United States Science Advisory EPA-SAB-EEC-98-007
Environmental Board (1400) April 1998
Protection Agency Washington DC
&EPA AN SAB REPORT: REVIEW OF
THE TOXICS RELEASE
INVENTORY (TRI) RELATIVE
RISK-BASED ENVIRONMENTAL
INDICATORS METHODOLOGY
A REVIEW BY THE ENVIRONMENTAL
ENGINEERING COMMITTEE (EEC)
-------
April 30, 1998
EPA-SAB-EEC-98-007
Honorable Carol M. Browner
Administrator,
U.S. Environmental Protection Agency
401 M Street, SW
Washington, DC 20460
Subject: An SAB Review: Review of the Toxics Release Inventory
Relative Risk-Based Environmental Indicators Methodology
Dear Ms. Browner:
The Office of Pollution Prevention and Toxics (OPPT) asked the Science
Advisory Board (SAB) to evaluate its Toxics Release Inventory (TRI) Relative Risk-
Based Environmental Indicator Methodology. The SAB's Environmental Engineering
Committee (EEC) met on July 2, 1997 at the National Risk Management Research
Laboratory in Cincinnati, Ohio, to review the technical merits of the methodology,
including toxicity weighting, exposure modeling, and exposed populations. The EEC's
Subcommittee on TRI Relative Risk-Based Environmental Indicators subsequently
prepared this report, which the EEC approved in November 1997 and the SAB's
Executive Committee approved on March 31, 1998.
We commend Agency representatives for their detailed understanding of the
indicator methodology, their clear presentation to the Subcommittee, and the
preparation of thorough reports that allowed the Subcommittee to grasp the toxicity,
exposure and population elements, assumptions and limitations that were employed in
the calculation of the indicators. The Agency's diligent preparation facilitated the SAB
review.
The attached report presents in detail the Subcommittee's findings and
recommendations. The Subcommittee would like to highlight the following points.
a) The methodology's consideration of exposure and populations in its
estimation of risk is an improvement over estimates of risk based solely
on the mass of annual releases or the toxicity-weighted releases as
proposed for the Sector Facility Indexing Project (SFIP) which was
recently reviewed by the Environmental Engineering Committee (SAB,
1997).
-------
b) The Subcommittee agrees with OPPT that the TRI Relative Risk-Based
Environmental Indicator, as presently structured, is developed to calculate
chronic human health risk but not ecological risk. The Subcommittee
therefore, encourages the Agency to implement its plans to include
ecological impacts in a separate indicator.
c) The Subcommittee concurs with the Agency's decision to reduce the
number of carcinogen categories from three to two.
d) The Subcommittee finds that the Agency's use of weights for binning is
inappropriate, because binning can result in inaccuracies of nearly an
order of magnitude and create artificial distinctions between chemicals
having similar toxicity values that fall on different sides of the cut point.
We, therefore, recommend that the Agency use the actual toxicity values
in the TRI Relative Risk-Based Environmental indicators methodology
and bin, if at all, at the end.
e) The Subcommittee recommends that separate indicators be considered
for cancer and non-cancer chronic health end-points, and that these
indicators be constructed in a manner that will allow them to be combined
into a single chronic human health indicator.
f) The Subcommittee recommends that actual population numbers be used
rather than the default minimum of 1000 because the default rural
population value will artificially increase relative risks for remote facilities.
g) The Subcommittee recommends that OPPT replace or modify the
deficient exposure models presently incorporated into the TRI Relative
Risk-Based Environmental Indicators with others that are presently used
within the Agency. The Subcommittee recommends that the Agency
review all exposure models but in particular focus on those for surface
water, land and POTW releases. The use of more appropriate exposure
models with region-specific data (and, when available, site-specific data)
will improve the precision and accuracy of the relative risk-based
indicators. The Subcommittee believes these improvements will be most
important when applying the methodology on a regional or site-specific
basis.
h) The Subcommittee disagrees with OPPT's position that the selection of
toxicity weights offers an opportunity to consider policy issues. The
Subcommittee recommends that the introduction of any Agency policy
should be transparent and reserved until after calculation of the TRI
indicators.
-------
i) The Subcommittee recommends that the Agency subject the TRI Relative
Risk-based Environmental Indicators methodology to additional sensitivity
and uncertainty analyses and also portray uncertainty in the final results.
This will enable users of this methodology to assign proper levels of
confidence when using the outputs.
The Subcommittee appreciates the opportunity to review the TRI Relative
Risked-Based Environmental Indicators methodology and looks forward to a written
response from the Office of Pollution Prevention and Toxics.
Sincerely,
Joan Daisey, Chair
Science Advisory Board
Dr. Ishwar P. Murarka, Past Chair
Environmental Engineering Committee
Dr. John P. Maney, Chair
Subcommittee on TRI Relative Risk-Based
Environmental Indicators
-------
NOTICE
This report has been written as part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice to
the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters related
to problems facing the Agency. This report has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the views
and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or
commercial products constitute a recommendation for use.
-------
ABSTRACT
The Science Advisory Board (SAB) assessed the technical merits of the Toxics
Release Inventory (TRI) Relative Risk-Based Environmental Indicator methodology
developed by the Office of Pollution Prevention and Toxics (OPPT). The methodology
employs the same toxicity weighting for chemical releases as the Sector Facility
Indexing Project previously reviewed by the SAB. The TRI Relative Risk-Based
Environmental Indicator methodology also considers fate, transport, and the exposed
population.
The methodology's consideration of exposure and populations in its estimation
of risk is an improvement over estimates based solely on the mass of annual releases
or solely on toxicity-weighted releases.
To improve the methodology, the Subcommittee recommends that the
methodology: a) use actual, rather than binned, toxicity values; b) use more appropriate
exposure models with region-specific data (and, when available, site-specific data); and
b) use actual population numbers rather than rural population default value of 1000.
The Subcommittee recommends that the EPA subject the TRI methodology to
sensitivity and uncertainty analyses and portray uncertainty in the final results. This
will allow potential users the ability to use the output with the proper confidence.
Keywords: Toxic Release Inventory, modeling, relative-risk, toxicity weights
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee
Subcommittee on TRI Relative Risk-Based
Environmental Indicators-July, 1997
CHAIR
Dr. John P. Maney, Environmental Measurements Assessment, Hamilton, MA
MEMBERS
Dr. Edgar Berkey, Concurrent Technologies Corp., Pittsburgh, PA
Dr. Ishwar P. Murarka, Environment Group, Electric Power Research Institute, Palo Alto, CA
Dr. Stephen L. Brown , R2C2, Oakland, CA (SAB/RAC)
Dr. Richard Kimerle, Monsanto Company (Retired), St. Louis, MO (SAB/RSAC)
CONSULTANTS
Mr. Terry Foecke, Waste Reduction Institute, St. Paul, MN
Dr. Wayne Kachel, MELE Associates, Brooks AFB, TX
Dr. Michael J. McFarland , Utah State University, Department of Civil and Environmental
Engineering, Logan, UT
Dr. William Pease, Environmental Defense Fund, Oakland, CA
Dr. Frederick G. Pohland , Department of Civil and Environmental Engineering, University of
Pittsburgh, Pittsburgh, PA
Dr. Rita Schenck, Eco Sense, West Rutland, VT
Dr. Nga Tran, Johns Hopkins University, School of Public Health, Baltimore, MD
SCIENCE ADVISORY BOARD STAFF
Kathleen W. Conway, Designated Federal Official, U.S. Environmental Protection Agency,
Science Advisory Board, Washington, DC 20460
Dorothy M. Clark, Staff Secretary, U.S. Environmental Protection Agency, Science Advisory
Board, Washington, DC 20460
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee (EEC)
CHAIR
Dr. Hilary I. Inyang, Center for Environmental Engineering and Science Technologies
(CEEST), University of Massachusetts, Lowell, MA
Dr. Ishwar P. Murarka, Environment Group, Electric Power Research Institute, Palo Alto, CA1
MEMBERS
Dr. Edgar Berkey, Concurrent Technologies Corporation, Pittsburgh, PA
Dr. Calvin C. Chien, E. I. DuPont Company, Wilmington, DE
Mr. Terry Foecke, Waste Reduction Institute, St. Paul, MM2
Dr. Nina Bergan French , SKY+, Oakland, CA
Dr. Hilary I. Inyang, Center for Environmental Engineering and Science Technologies
(CEEST), University of Massachusetts, Lowell, MA3
Dr. James H. Johnson, Jr., School of Engineering, Howard University, Washington, D. C.
Dr. JoAnn Slama Lighty, University of Utah, Salt Lake City, UT
Dr. John P. Maney, Environmental Measurements Assessment, Hamilton, MA
Ms. Lynne M. Preslo, Technical Programs, Earth Tech, Long Beach, CA
Dr. Wm. Randall Seeker, Energy & Environmental Research Corp., Irvine, CA3
Science Advisory Board Staff
Kathleen W. Conway, Designated Federal Official, U.S. Environmental Protection Agency,
Science Advisory Board, Washington, DC 20460
Dorothy M. Clark, Staff Secretary, U.S. Environmental Protection Agency, Science Advisory
'Chair during the TRI review.; serving as Past Chair for FY 1998.
Appointed to the EEC after the TRI review.
IV
-------
Board, Washington, DC 20460
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
1.1 Response to General Charge 1
1.1.1 General Charge Question 1 1
1.1.2 General Charge Question 2 2
1.1.2.1 Hazard 2
1.1.2.2 Exposure 3
1.1.2.3 Population 3
1.1.3 General Charge Question 3 3
1.1.4 General Charge Question 4 3
1.1.5 General Charge Question 5 5
1.1.5.1 Sensitivity and Uncertainty Analysis 5
1.1.5.2 Validation of Output 6
1.2 Specific Charges Regarding Toxicity Weighting 6
1.2.1 Compressing the Carcinogen Categories 6
1.2.2 Comparing Severity and Number of Effects 7
1.2.2.1 Severity of Effects 7
1.2.2.2 Multiple Effects 7
1.2.3 Binning of Toxicity Values 7
1.2.4 Toxicity Data Gaps 8
1.2.5 Separate Indicators for Cancer and Noncancer Impacts 8
1.2.6 Introduction of Policy into Toxicity Weighting 8
1.3 Specific Charges Regarding Exposure Modeling 9
1.3.1 Improving the Media- and Pathway-Specific Approaches
Used for Exposure Modeling 9
1.3.2 Adjustment Factors and Uncertainty in Evaluating Exposure .... 9
1.3.3 Alternate Values for Exposure Modeling Assumptions 10
1.4 Specific Charge Issues Regarding Exposed Populations 10
1.4.1 Filling Data Gaps for Interim Years Between Primary
Census Dates 10
1.4.2 Approaches for Weighting Data on Rural Populations 10
2. INTRODUCTION 11
2.1 Background 11
2.2 Review and Charge 11
3. RESPONSE TO THE CHARGE 14
3.1 Are Approaches for Hazard, Exposure and Population Appropriate
for Stated Objectives 14
3.1.1 Appropriateness of Approach for Assessing Hazard 14
3.1.1.1 Binning 15
VI
-------
3.1.1.2 Toxicity Data Gaps 16
3.1.1.3 Separate Indicators for Cancer and Noncancer
Impacts 19
3.1.1.4 Compression of Carcinogenic Categories 20
3.1.1.5 Accommodation of Severity 20
3.1.1.6 Accommodation of Multiple Effects 20
3.1.1.7 Introduction of Policy into Toxicity Weighting 21
3.1.2 Appropriateness of Approach for Assessing Exposure 22
3.1.2.1 General Comments 22
3.1.2.2 Comment Regarding Pathways 23
3.1.2.3 Comments Regarding Assumptions 24
3.1.3 Appropriateness of Approach for Assessing Exposed
Populations 27
3.2 Are Hazard, Exposure and Population Elements properly integrated? ... 28
3.2.1 Hazard 28
3.2.2 Exposure 29
3.2.3 Population 31
3.3 Will Methodology provide Reasonable Relative Risk-based
Analyses and Impacts of TRI Chemical Emissions? 32
3.4 Identify Future Research Needs 34
3.4.1 Sensitivity and Uncertainty Analysis 34
3.4.2 Validation of Output 35
3.4.3 Unintended Consequences 36
3.4.4 Other Research 37
REFERENCES CITED R-1
APPENDIX A A-1
APPENDIX B B-1
VII
-------
1. EXECUTIVE SUMMARY
This section of the report summarizes the Subcommittee responses to the
Charge. The structure of this section closely follows that of the Charge, using the
Charge questions as section heading. At the request of the Agency, the Charge
questions were responded to in depth and with detailed suggestions. Although this
section of the report summarizes the Subcommittee's responses, a more detailed
discussion of the our responses and suggestions is presented in Section 3.
1.1 Response to General Charge :
1.1.1 General Charge Question 1
Charge Question 1 asked the Subcommittee to assess the technical merits of
the methodology in order to evaluate whether appropriate approaches have been
selected to assess hazard, exposure and population parameters.
The Subcommittee found the methodology's consideration of exposure and
populations in its estimation of risk to be an improvement over estimates of risk based
solely on the mass of annual releases or the toxicity-weighted releases as proposed for
the Sector Facility Indexing Project (SFIP).
The Subcommittee agrees with OPPT's position, that the TRI Relative Risked-
based Environmental Indicator, as presently structured is solely an indicator of chronic
human health and does not address ecological impacts. The Subcommittee
encourages the Agency to achieve a balance between human health and ecological
issues by implementing its plans to include ecological impacts in a separate indicator.
The Subcommittee disagrees with OPPT's position that the selection of toxicity
weights offers an opportunity to consider policy issues. The Subcommittee
recommends that the introduction of any Agency policy should be transparent and
reserved until after calculation of the TRI indicators.
The Subcommittee found the methodology, after incorporating some
recommended changes (i.e. regarding binning, rural population default values and
separate indicators for cancer and noncancer impacts) could be employed to develop
scientifically defensible relative risk indicators for chronic human health at the national
level. However, limitations in the exposure component of the methodology would still
result in significant uncertainty for relative risk indicators at the regional or facility level.
-------
1.1.2 General Charge Question 2 :
The second Charge item asked the Subcommittee to assess the technical merits
of the methodology in order to determine if these elements have been properly
integrated within the methodology.
1.1.2.1 Hazard
To improve integration of the hazard components, the Subcommittee
recommends that toxicity values be used directly; carcinogenic potency values be
converted to Risk Specific Doses; and indicators be reported to only a few significant
figures. The TRI model has incorporated a number of approaches from the Hazard
Ranking System (MRS). In the past, commenters have stated that because the MRS
toxicity weighting scheme was developed for a different purpose, it should not be used
in the TRI Indicators Project. However, the MRS and TRI indicator objectives are not
sufficiently different to invalidate the use of the MRS system, especially if the
Subcommittee's recommendations for improvement are utilized. Both MRS and the TRI
indicators were developed with the intent of using surrogate measures of toxicity to
arrive at conclusions that relate to risk. SAB comments on the MRS emphasized the
need for risk-based rankings, and changes to the old MRS were made to improve that
relationship, making the MRS suitable for integration into the TRI Relative Risked-
based Methodology.
Toxicologists generally advise against summing Hazard Quotients (estimated
dose/RfD) over chemicals that do not have the same endpoint and mechanism of
action. However, if one adopts the probability viewpoint about the significance of the
indexes, then a facility with several chemicals will usually be more deserving of
attention than one with only one chemical with a similar component score. Regarding
summing over different media, the underlying (unstated) assumption is that the
exposure weights are roughly in the same proportion to risk for all of the media
calculations, after consideration of route-specific toxicity differences and the application
of exposure modeling uncertainty factors. That assumption is critical to the whole TRI
indicator toxicity-weighting scheme, and therefore, the Subcommittee believes that
summing adds no additional difficulties.
The Subcommittee believes that basing the hazard component on actual toxicity
values and incorporation of the above suggestions for risk-specific doses and the
number of reported significant figures offer a mechanism for properly integrating hazard
with exposure and population elements
-------
1.1.2.2 Exposure
The Subcommittee believes that there are superior exposure models available
within the Agency and the scientific community at large. The Subcommittee believes
that if the TRI methodologies are employed to generate disaggregate indicators (e.g.,
regional, facility specific indicators) the use of these models, regional-specific data and
when available, site-specific data should improve the relative risk-based indicators.
1.1.2.3 Population
The present model takes into account the total population for a model-specified
cell, area, reach or other zones of influence. Although the population estimates are
likely to contribute little to the uncertainty of national indicators, as the TRI model is
applied to smaller geographical areas, the contribution of population estimates to
indicator uncertainty is likely to be more significant. It is also recognized that
population changes in the zone of influence of a facility would increase or decrease its
component scores even though no change in its releases occurs. TRI documentation
should clearly explain this impact of population on TRI indicators and how local
population changes could result in misleading conclusions regarding a facility's
environmental management practices (Refer to Section 1.4.1).
The Subcommittee believes that actual population data, if available, should be
used in estimating risk(s) (Refer to Section 1.4.2).
1.1.3 General Charge Question 3
Because the Subcommittee felt most comfortable by not trying to discriminate
between general Charge issues 3 and 4, both of these issues are addressed together
in Section 1.1.4 below. Issue 3 is, "assess whether this screening-level tool will provide
reasonable results for relative risk- based analyses" and issue 4 is, "consider whether
the overall methodology accomplishes OPPT's objective to provide a measure of risk-
related impacts pertaining to TRI chemical emissions."
1.1.4 General Charge Question 4
As detailed above, Charge 4 addressed the overall methodology in the light of
the OPPT's objective.
According to the Executive Summary of the TRI Indicators document, "the
objective of [the indicators methodology] is to calculate a unitless value that reflects the
overall risk-related impacts of releases and transfers of all included TRI chemicals from
all reporting facilities to each environmental medium for a given year or years." With
one major exception and a number of minor ones discussed below, the methodology
-------
should serve this narrow objective. It will not serve as well the subsidiary uses that are
mentioned as possibilities: geographic analyses, chemical-specific analyses, industry
sector analyses, environmental justice analyses, and so on, principally because any
inaccuracies or biases will be magnified at lower levels of aggregation. Because the
temptation to apply the indicators methodology to such other uses will be strong, the
document should supply further cautions about its applicability.
A great strength of the TRI documentation is its clarity of presentation. The
Subcommittee found little trouble in understanding how the methodology was
developed or how it would operate. In particular, the document makes it clear that the
definition of risk, which it has adopted is the public health definition: simulating the
likely magnitude of impacts on the population, rather than the potential risks to highly
exposed persons. A minor exception occurs because the methodology establishes a
minimum size for the population at risk for a given facility's releases.
The Agency also has correctly chosen an approach that uses "proportional"
weights, such that the algorithm for calculating the indicator combines and scores the
weights in the same way they would be in a more complete risk assessment, again with
one glaring exception - the binning of toxicity values. In many respects, the current
indicator methodology for chronic human health amounts to a crude risk assessment
that should produce component scores roughly proportional to risk, albeit with
substantial uncertainties and conservative biases, which may not be consistent for
different components of the methodology. Because the Agency seemed to recognize
the virtues of the proportional approach, it is curious that it chose to use a severely
constrained version of that approach for the toxicity weight by binning toxicity values
and truncating their range. The Subcommittee recommends using toxicity values
directly rather than binning them and believes that incorporation of this
recommendation will result in more reasonable risk-based analyses.
The reliability of the indicators may depend strongly on the coverage of the TRI
list of reportable releases. If the Agency has any way of characterizing the effect of
omitted substances, it should present that information. Although this problem may not
have a strong effect on the reliability of a national chronic health risk indicator, it is
more likely to make comparisons on a smaller scale, such as interfacility or
environmental justice, unreliable. Similarly, the Agency should attempt to consider the
impact of not considering releases from facilities excluded from the TRI reporting
requirements and the impact of chemicals not presently listed by the TRI. The
Subcommittee recommends that the Agency identify the TRI slice of the universe of
environmental risks and clearly present these limitations in its hard copy and electronic
documentation and reports. The TRI relative risks indicators should be presented in
the context of non-TRI chemical releases, exposures from other pathways such as
indoor air, and the selective consideration of human populations at the exclusion of
others such as avian and terrestrial animals.
-------
The description of how the indicator methodology will be modified to
accommodate changes in TRI reporting requirements is not very specific. Although the
document states that it will continue to calculate a set of indicators based only on the
original list of facilities and chemicals, it is not clear whether that list is the 1987 or
1994 version, or what additional indicators might be constructed. The flexibility of the
TRI model makes it imperative that the Agency develops a means of ensuring that all
assumptions, defaults, bases of comparisons and changes from previous models or
reports are obvious to the potential user of the data.
The Agency has wisely recognized that the TRI indicators are ripe for intentional
and unintentional misuse. This recognition has resulted in the repeated declarations
found in the TRI documentation that the indicators are relative and should not be used
to measure risk. In conflict with this recognition is a tool that has so much ability for
displaying facility-specific information and comparisons among chemicals using
exposure and risk estimates as a sort. The Agency has to be realistic with regard to
the continuum of applications to which the TRI indicators are likely to be applied and
continue to incorporate even more prominent warnings into all new documentation and
reports, and where appropriate, into its computer program to decrease unintentional
misuse and to discourage intentional misuse.
1.1.5 General Charge Question 5
The fifth Charge issue requested an assessment of the technical merits of the
methodology in order to identify research needs that could influence future
enhancements and improvements of the methodology.
Although the Subcommittee recognizes that considerable work has already been
done to develop the indicator methodology, there is disagreement among the
Subcommittee members as to whether more development is required before the
indicator methodology can be credibly and reliably used for its intended purposes. To
decrease controversy regarding the relative risk indicators, the Subcommittee
recommends that the Agency subjects the methodology to sensitivity and uncertainty
analyses and where possible validate the outputs.
1.1.5.1 Sensitivity and Uncertainty Analysis
As currently presented, the output from the methodology (numbers as well as
graphs) implies an accuracy that is far greater than the quality of the input data and the
models used to generate it. The developers must address the uncertainty question as
a high priority and take steps to portray uncertainty in the final results. The
Subcommittee believes this will be a valuable and necessary exercise that will allow
potential users the ability to use the output with the proper confidence.
-------
The TRI indicators documentation as well as presentations during the review
imply that the developers of the methodology have not performed a sensitivity analysis,
with the exception of a stack height analysis, that relates variations in the final results
to variations in the value of input parameters and assumptions. The Subcommittee
believes it is critical to understand the internal sensitivities of the model as a means,
not only of understanding its limitations, but also focusing effort on improving the
accuracy of those input parameters that make the most difference. Sensitivity analysis
can result in numerous benefits, one of which is the ability to focus future research to
minimize uncertainty in the resulting indicators.
Sensitivity analysis will explain the relative influence of model components (i.e.,
hazard, exposure-dose, and population) on the final indicators. This kind of analysis
would provide some comfort level as to whether population size alone or some other
component or assumption had undue influence on the final indicator. It would be useful
for the developers to develop a summary table of the direction of bias in the default
parameters and assumptions to provide a basis for the up/down adjustment of
indicators. This summary table could facilitate future uncertainty analyses as default
parameters are replaced with more realistic values.
In summary, uncertainty and sensitivity analyses can overcome the present lack
of error bounds that make it difficult to distinguish "noise" levels from real changes in
risk. This problem is likely to be exaggerated at the disaggregated level, because at
this level the "noise" may be relatively large, requiring a sound understanding of
uncertainty before lower tier indicators could be used to support environmental
decision-making.
1.1.5.2 Validation of Output
The Subcommittee believes the model's output should be validated or "ground-
truthed" by applying the methodology to a series of cases for which exposure
concentrations were estimated using other accepted exposure models. The
Subcommittee recognizes that any validation is complicated by the relative nature of
the indicators and the uncertainty of the TRI release data, but in most situations at least
exposure model components should be capable of validation. Lacking some sort of
validation, questions will remain about the applicability of the model.
1.2 Specific Charges Regarding Toxicity Weighting
1.2.1 Compressing the Carcinogen Categories
The Subcommittee supports the approach for similar weighting of EPA Category
A, B1 and B2 carcinogens. The Subcommittee agrees that it is appropriate to treat
Category C carcinogens differently by reducing their potency value. The extent of the
-------
reduction (one order of magnitude in the current Indicator methodology) is a policy
judgment to be made by EPA; there is no scientific rationale for using a factor of ten.
For the long term, this approach to incorporating current weight of evidence
evaluations of carcinogens into the Indicator will need to be revised to accommodate
the new EPA carcinogen classification system that will emerge from the Agency's
revision of its cancer policy guidelines. This classification abandons the alphanumeric
categories used by the current Indicator methodology and establishes narrative
categories that are less likely to support quantitative adjustment factors.
1.2.2 Comparing Severity and Number of Effects
1.2.2.1 Severity of Effects
The Subcommittee noted that toxicity values are currently not derived using
factors to account for severity of endpoint, and that there is no generally accepted
weighting scheme for increasing or decreasing the value of a RfD or RfC based on
severity evaluations. [How to account for severity of endpoints is an important issue,
but the Agency cannot be expected to deal with it in the TRI indicator methodology. It
should be handled by the Agency in some other arena such as the Agency's Risk
Assessment Forum with input from outside parties.] Given the Indicator project's need
to rely on authoritative toxicity data and the value of having a quantitative hazard
component in the Indicator, the Subcommittee believes it is reasonable to assess
hazard based on most sensitive effect, without severity adjustments.
1.2.2.2 Multiple Effects
The Subcommittee noted that toxicity values are currently not derived using
factors that account for a chemical's ability to cause multiple organ effects, and that
there is no generally accepted approach to modifying the value of a RfD or RfC based
on multiple effects. The Subcommittee agrees with the Agency's position of not
incorporating a weighting mechanism for multiple effects into the present TRI
methodology.
1.2.3 Binning of Toxicity Values
Although the project's general approach to hazard assessment is appropriate,
the Subcommittee unanimously objected to the use of binning and truncation of toxicity
values rather than actual toxicity values in indicator calculations. The Subcommittee
believes that the binning of toxicity values is inappropriate, because it results in the
loss of information and can make artificial distinctions between chemicals that have
similar toxicity values that fall on different sides of the cut point. The Subcommittee
consensus is that binning, if done at all, should be done at the end of the process,
-------
when the final indicator is calculated, perhaps by presenting only one or two significant
figures and if binning is discarded, the Subcommittee indicated a preference against
truncating the lower and higher ends of toxicity values.
1.2.4 Toxicity Data Gaps
The Subcommittee recommends that chemicals without toxicity values that have
been assigned binned toxicity scores via EPA's disposition process remain in the
Indicator model. Assigned binned scores should be converted to interim toxicity values
(RfDs, RfCs, or potencies) based on the log midpoint of the dose scales that define the
bin responsible for a compound's toxicity scores. The Subcommittee also recommends
that the Agency should more actively explore the possibility of an expedited process in
which use of approved information from other reliable sources could result in Agency
adoption of toxicity numbers.
1.2.5 Separate Indicators for Cancer and Noncancer Impacts
Based on its review of the Indicator's approach to human health hazard
assessment, the Subcommittee recommends that separate indicators be considered for
cancer and noncancer chronic health impacts. The use of a toxicity scoring system that
assigns the same weights to both cancer and noncancer toxicity values enables the
project to generate a summary indicator of chronic human health impacts that
integrates across these different endpoints. While there are advantages to the current
TRI methodology's single summary chronic health indicator, it hides important value
judgments about the relative importance of these endpoints that cannot be defined
based on scientific considerations. A single summary indicator also precludes
independent evaluation of cancer and noncancer endpoints. The Subcommittee
recommends that separate indicators be considered for cancer and noncancer chronic
health impacts, and that these indicators be constructed in a manner that will allow
them to be combined into a single summary chronic human health indicator.
1.2.6 Introduction of Policy into Toxicity Weighting
The Agency has stated that the "selection (of) toxicity weights provide EPA with
an opportunity to consider important policy issues in determining final weights" (Page
30 of Toxic Release Inventory Relative Risk-Based Environmental Indicators: Interim
Toxicity Weighting Summary Document (EPA, 1997). The Subcommittee believes that
the introduction of policy, especially in a non-transparent manner, into a model that
needs to be scientifically defensible is inappropriate and will negatively impact the
credibility of the TRI indicators. The Subcommittee recommends that consideration of
the severity of endpoints and the introduction of Agency policy should be transparent
and reserved until calculation of the TRI indicators are completed.
8
-------
1.3 Specific Charges Regarding Exposure Modeling
The Subcommittee found the methodology, after incorporating some immediate
changes (i.e., regarding binning, rural population default values and separate indicators
for cancer and noncancer impacts) could be employed to develop scientifically
defensible national relative risk indicators for chronic human health. However, even
after incorporation of these immediate changes, the Subcommittee determined that
limitations and deficiencies in the exposure component of the methodology would result
in significant uncertainty for disaggregated relative risk indicators at the regional or
facility level.
1.3.1 Improving the Media- and Pathway-Specific Approaches Used for
Exposure Modeling
The Subcommittee believes that there may be more appropriate fate and
transport models than those employed by the TRI methodology. The EEC has
previously provided comments on such models, including where and how they should
be applied. The Agency should review alternative models to determine if they would be
more appropriate alternatives to those presently incorporated into the TRI
methodology.
The Subcommittee recommends that the Agency evaluate all potential exposure
pathways before excluding any from consideration. For example, the surface water
pathway with potential exposures through recreational use or bathing are pathways that
should be considered for inclusion in the TRI model. Another main route of exposure to
environmental contaminants, the dietary pathway, is not adequately accounted for in
the TRI model. For example some chemical releases through the air pathway can
result in more human exposure/risk via the diet than via inhalation. The Agency should
document the logic for including or excluding a pathway as well as the ramifications of
these decisions on the accuracy and uncertainty of the TRI model.
1.3.2 Adjustment Factors and Uncertainty in Evaluating Exposure
Due to the conservative assumptions made in some exposure models,
adjustment factors of 5 or 10 are used to adjust down "surrogate" doses. Nevertheless,
in certain situations, default assumptions may not be as conservative as suggested.
For example, the fish consumption value (6.5 g/day) in the surface water pathway is
based on the per capita mean for the U.S. population, yet the exposed population who
fish and eat fish are probably consumers in the upper percentile range. [If the 6.5
g/day figure is retained, it should probably apply to all the people in an area; otherwise
a higher consumption figure is probably appropriate and/or the adjustment factor
should be less.] The Subcommittee recommends that the Agency review adjustment
factors in light of the following discussion of pathway assumptions in Section 3.
-------
1.3.3 Alternate Values for Exposure Modeling Assumptions
The Subcommittee believes that the Agency should review its assumptions for
groundwater and air releases from land disposal units, population estimates, surface
water discharges and air dispersion models. A more detailed discussion of these
issues is presented in Section 3.
1.4 Specific Charge Issues Regarding Exposed Populations
1.4.1 Filling Data Gaps for Interim Years Between Primary Census Dates
The majority of the Subcommittee believes that an accepted method to address
population changes over time is to use the latest intercensual rate of change (i.e.,
average annual rate of change between 1980 census and 1990 census) to project
population size for postcensual years (Shrycock et. al, 1976). A dissenting
Subcommittee member voiced concern over this approach because it was perceived as
encouraging the extrapolation of data, a policy that the EEC has often opposed.
1.4.2 Approaches for Weighting Data on Rural Populations
Population considerations are a significant component of the TRI model, such
that population change could influence the indicator for regional and site-specific
applications. An increase or decrease of population will elevate or lower the indicator
even when no change in releases occurred.
Members of the Subcommittee had concerns regarding the 1000 exposed
persons minimum for air exposure for rural areas and recommend the use of actual
population numbers as opposed to this default minimum of 1000. Presently, the default
rural population value of 1000 may artificially increase relative risks for remote facilities
and remove the incentive to locate facilities in areas removed from adjacent
populations.
10
-------
2. INTRODUCTION
2.1 Background
The Emergency Planning and Community Right to Know Act of 1986 established
the requirement for the Toxics Release Inventory (TRI). Since 1987, implementation of
the TRI regulations has resulted in one of the Agency's largest and longer-term
databases. EPA's need to track changes in the environment and to set priorities
naturally led to the consideration of the TRI database as a means to measure risk and
changes in risk over time.
The TRI database is not a meaningful indicator of risk, because the annual mass
data ignores key components of risk. Risk is specific to those exposed and varies in
magnitude according to the degree of exposure and the toxicity of the chemical. OPPT
designed the TRI Relative Risk-Based Environmental Indicators methodology to
ameliorate these limitations and to calculate an indicator of relative risk-based impacts
on the nonworker, general population.
2.2 Review and Charge
The Office of Pollution Prevention and Toxics (OPPT) asked the Science
Advisory Board (SAB) to evaluate the TRI Relative Risk-Based Environmental Indicator
methodology. The Environmental Engineering Committee formed a Subcommittee on
TRI Relative Risk-Based Environmental Indicators, consisting of EEC members,
members of other relevant SAB committees, and consultants. Drs. Brown, Kimerle,
Maney, and Murarka had also participated in the related April 29, 1997 Special Topics
Subcommittee review (SAB, 1997) of the use of toxicity weighting factors in the Office
of Enforcement and Compliance Assurance's Sector Facility Indexing Project (SFIP).
The SFIP employed the toxicity weights as established by the TRI Relative Risked-
based Environmental Indicators Methodology. The EEC and the Subcommittee met on
July 2-3, 1997 at the National Risk Management Research Laboratory in Cincinnati to
address the following Charge:
a) General Charge: To assess the technical merits of the methodology to:
1) evaluate whether appropriate approaches have been selected to
assess hazard, exposure and population parameters;
2) determine if these elements have been properly integrated within
the methodology;
11
-------
3) assess whether this screening-level tool will provide reasonable
results for relative risk-based analyses;
4) consider whether the overall methodology accomplishes OPPT's
objective to provide a measure of risk-related impacts pertaining to
TRI chemical emissions; and
5) identify research needs that could influence future enhancements
and improvements of the methodology.
b) Specific Charge Issues Regarding Toxicity Weighting:
1) Does the SAB have any comments regarding the recommendation
to compress the carcinogen categories to two (one for A and B,
and one for C) with only a single factor of ten separating the
categories?
2) Can the SAB offer any recommendations as to how severity or
number of effects be compared?
c) Specific Charge Issues Regarding Exposure Modeling:
1) Does the SAB have any suggestions on how to improve the media-
and pathway-specific approaches used for exposure modeling?
2) Are there more appropriate adjustment factors, which may be
incorporated in the uncertainty categories used for evaluating
exposure?
3) Can the SAB suggest alternate values for these assumptions which
would be more appropriate?
d) Specific Charge Issues Regarding Exposed Population
1) At various levels (state, county, city, block group) could SAB
suggest potential approaches to filling data gaps for interim years
between primary census dates?
2) Can SAB recommend any other approach which would more
appropriately weight rural populations?
12
-------
The EEC and the Subcommittee reviewed materials sent in advance of the
meeting,4 listened to overview presentations, discussed the documentation
accompanying the presentations, and provided a verbal synopsis of findings and
recommendations to Agency staff before adjourning. The Executive Summary presents
the Subcommittee's findings and recommendations for each of the Charge questions.
In response to OPPT's request for a more detailed response to the Charge questions, a
detailed presentation of the Subcommittee's findings, recommendations and references
are presented in Section 3.
4Materials reviewed by the Subcommittee are listed in Appendix B.
13
-------
3. RESPONSE TO THE CHARGE
Detailed responses to the Charge issues are presented in this section. The
Charge identified general and specific issues and did not emphasize their close
relationship. While this format is constructive for a Charge, it can lead to unnecessary
redundancy within a report. This chapter has been structured to re-establish the
relationship between the general and specific issues. For example, questions
regarding the binning of toxicity values or the compression of carcinogenic categories
are considered in subsections under the section on hazard.
In addition, the Subcommittee determined that it was best not to discriminate
between two of the General Charge issues:
General Charge Question a)3) - assess whether this screening-level tool will
provide reasonable results for relative risk-based analyses; and
General Charge Question a)4) - consider whether the overall methodology
accomplishes OPPT's objective to provide a measure of risk-related impacts
pertaining to TRI chemical emissions;
Both of these Charge issues are addressed together in Section 3.3. To facilitate
consistency throughout the document and efficient perusal, the major findings are
summarized at the beginning of each subsection, with more detailed discussion,
comments and findings following.
3.1 Are Approaches for Hazard, Exposure and Population Appropriate
for Stated Objectives?
3.1.1 Appropriateness of Approach for Assessing Hazard
The assumptions underlying the hazard assessment component of the Chronic
Human Health Environmental Indicator are consistent with current scientific practice.
That is, the cancer and noncancer toxicity values developed by EPA to provide a
measure of inherent toxicity for indicator calculations. The TRI Indicators document
emphasizes that the current indicator addresses relative chronic health effects only.
This focus is appropriate given that the limitations of the TRI release data prevent
construction of an analogous quantitative indicator for acute health effects. The
Subcommittee agrees with the overall approach and provides, in the following
subsections, suggestions that can improve the mechanism for assessing hazard for
incorporation into the TRI indicator.
14
-------
3.1.1.1 Binning
Finding: The Subcommittee found that the binning of toxicity values is
inappropriate because binning may introduce more significant artifacts to the Indicator
than use of actual toxicity values. If a toxicity value should change and move across a
cut point, binning can improperly exaggerate the change by an order of magnitude.
Recommendation: The Subcommittee consensus is that binning, if done at all,
should be done at the end of the process, when the final indicator is calculated,
perhaps by presenting only one or two significant figures.
Recommendation: If binning is discarded and actual toxicity values are used,
the Subcommittee recommends that the Agency not truncate the lower and higher ends
of toxicity values.
Although the methodology's general approach to hazard assessment is
appropriate, the Subcommittee unanimously determined that the use of binning and
truncation of toxicity values rather than actual toxicity values in indicator calculations
was scientifically flawed. The Agency asserted before, during and after the review that
the indicator should be calculated using toxicity bins (assigned on the basis of
categorical ranges of toxicity values defined by the Hazard Ranking System) rather
than derived directly from the toxicity values. Four arguments have been offered by
Agency Staff to support the use of toxicity bins:
a) use of actual toxicity values would imply more accuracy than justified;
b) small changes in toxicity values should not be allowed to change overall
indicator scores (toxicity scores based on bin assignments would be more
unlikely to change);
c) use of weights will prevent the indicator tool from being misused to
produce risk estimates that might be misinterpreted as predictions of the
incidence of adverse health effects in exposed populations; and
d) that toxicologists are more confident in assigning order-of-magnitude
toxicity weights (even though the disposition process generates a point
estimate toxicity value that is later assigned to a bin) to TRI chemicals
missing toxicity values than to develop the actual values themselves
through EPA's disposition process.
The Subcommittee believes that the first three arguments (a, b, and c above) are
unsound, and that there are solutions to the problems posed in the fourth (d). Binning
the toxicity values creates inaccuracies of up to an order of magnitude resulting in
15
-------
artificial distinctions between chemicals of similar toxicity. Binning to account for
uncertainties, if done at all, should be done at the end of the process, when the final
indicator is calculated, perhaps by presenting only one or two significant figures. While
there are legitimate concerns about the uncertainty associated with toxicity values,
these concerns are not unique; there are also uncertainties associated with the other
major components, including reported release data, exposure and population
assumptions.
The Subcommittee is not persuaded by the rationale for the third argument (c
above): "toxicity weights cannot be easily manipulated mathematically to convert
indicator values from unitless numbers to precise (although probably inaccurate)
estimates of the number of cases of health effects associated with various chemical
emissions". This rationale is not persuasive because it is always possible for a person
of modest technical skill to replace weights with the actual toxicity values (included in
the data section) and then generate individual and population risk estimates for
carcinogens (or hazard indices and population exceedance for noncarcinogens). To
prevent people from using the methodology to generate risk estimates, the Agency
would have to make the methodology non-transparent and prevent public access to the
data elements required by its indexing algorithms.
Because the Agency is concerned about potential misuse of the screening-level
assessments, the Subcommittee recommends the Agency describe the limitations of
screening-level assessments and advise against misinterpreting risk estimates as
actual body counts. If the Agency wants to be sure that the public cannot misinterpret
the outputs as actual risk estimates, the Agency could also bin indicator results at the
end of the calculation process, by focusing on the exponents of the calculated indicator
values or by converting the exponent to letter scores that convey ordinal ranking
without any risk interpretation.
The fourth argument (d above) - the concern regarding the assignment of
toxicity scores to TRI chemicals missing toxicity value s- is addressed in the following
subsection. Options for calculating indicators with actual toxicity values are presented
in Section 3.1.1.3.
3.1.1.2 Toxicity Data Gaps
The Agency does not have data with which to develop toxicity values for all TRI
chemicals. Several strategies exist to deal with these data gaps. Currently the Agency
employs a disposition process by which experts assign these chemicals to "bins".
Recommendation : The Agency should more actively pursue development of an
expedited process which uses approved information from other reliable sources to
generate actual toxicity values.
16
-------
Recommendation: If, as recommended, the Agency uses actual toxicity values
instead of bins, there will be a subset of chemicals which lack actual toxicity values and
were assigned "bins" using the disposition process. The Subcommittee recommends
that the Agency convert the assigned binned toxicity scores for these chemicals to
interim toxicity values. Such values (RfDs, RfCs, or potencies) can be based on the log
midpoint of the dose scales that define the bin responsible for a compound's toxicity
scores.
The Subcommittee agrees that lack of IRIS and HEAST values, especially for
such important substances as lead and copper, creates a great impediment to the
successful implementation of the TRI Relative Risk-based Environmental Indicators
Methodology. Therefore, the Subcommittee agrees, in concept, with developing toxicity
values for those TRI chemicals not evaluated in IRIS or HEAST. However, the
Subcommittee recommends that the Agency establish a continuing process for
reviewing and updating the toxicity values to be consistent with best toxicological
practice in values used elsewhere in the Agency.
Presently 372 of the 600 chemicals covered by TRI have toxicity values that can
be used to create the chronic human health indicator. The Subcommittee encourages
the Agency to develop values for the remaining TRI chemicals. The Subcommittee
consensus is that it is better to use alternative approaches for filling toxicity data gaps,
than to ignore the chemical(s) because the alternative approaches have fewer
undesirable implications for the integrity of the overall indicator system.
The Subcommittee supports the hierarchy of data sources for toxicity values
utilized by the project (first IRIS, then HEAST, then the disposition process), plus
currently unutilized sources of credible toxicity values that could enrich the data
supporting the Indicator.
If the Agency accepts the SAB's advice to use actual toxicity values, there will
remain a subset of chemicals lacking actually toxicity values which were binned using
the disposition process. An short-term approach is to convert assigned binned scores
into interim toxicity values (RfDs, RfCs, or potencies) based on the log midpoint of the
dose scales that define the bin responsible for a compound's toxicity score.
In the medium term, the Agency should evaluate whether compounds without
IRIS or HEAST values do have toxicity values that have been generated other
organizations using procedures similar to EPA's risk assessment practice (e.g., the
California EPA). Where such toxicity values exist, EPA should promptly substitute
them for the current toxicity values based on the disposition process. Consistency in
methods used to derive toxicity values, is important, however, and some organizations
that derive toxicity values using different practices (e.g., ACGIH, ATSDR) may not be
appropriate sources.
17
-------
Outside organizations may also be valuable sources of toxicity values on
exposure scenarios that EPA has not developed values for, but would like to address in
future enhancements of the Indicator (e.g., CalEPA acute inhalation exposure guideline
limits).
The Agency should first adopt an explicit approach to setting priorities for those
chemicals that lack toxicity values and then define a structured approach for
establishing toxicity values for the higher priority chemicals. In the short term, the
summary Indicator is appropriately derived using just those compounds that can be
modeled based on available data. An additional subset of "missing data/unmodeled"
compounds could be indexed using default toxicity values for sensitivity analyses.
These default toxicity values could be based on various cut points observed in the
distribution of existing toxicity values (median, and some upper bound percentage).
Sensitivity analyses could then be run to examine whether addition of default-based
Indicators to the summary modeled Indicator has a significant impact. More resource
intensive approaches to developing "expedited" toxicity values could be explored in the
future as an effort to reduce the number of TRI chemicals excluded from the Indicator
because of toxicity data gaps. Alternatives to be explored include estimating cancer
potency based on the relationship between acute toxicity and carcinogenic potency,
and by estimating toxicity values based on structure activity relationships.
The Subcommittee also had the following recommendations regarding IRIS and
HEAST toxicity data;
a) Do not remove existing toxicity values from these data bases, because it
is disruptive to a variety of Agency programs among which the TRI project
is an example.
b) Make a commitment to improve the IRIS and HEAST databases by
completing work on important chemicals (e.g., lead and copper).
c) Facilitate the creation and use of interim toxicity values for specific
purposes such as the TRI Risked-based Environmental indicators. The
Subcommittee is mindful that proliferation of potentially inconsistent and
less reliable toxicity numbers is undesirable. However, properly qualified
toxicity numbers should be available for specific and limited purposes.
d) Explore more actively the possibility of an expedited process in which use
of approved information from other reliable sources could result in Agency
adoption of toxicity numbers based on the latest research without feeling
the need to defend the older numbers.
18
-------
3.1.1.3 Separate Indicators for Cancer and Noncancer Impacts
Recommendation: The Subcommittee recommends that separate indicators be
considered for cancer and noncancer chronic health impacts, and that these indicators
be constructed in a manner that will allow them to be combined into a single summary
chronic human health indicator.
Based on its review of the Indicator's approach to human health hazard
assessment, the Subcommittee recommends that separate indicators be considered for
cancer and noncancer chronic health impacts. The use of a toxicity scoring system that
assigns the same weights to both cancer and noncancer toxicity values enables the
project to generate a summary indicator of chronic human health impacts that
integrates across these different endpoints. While there are advantages to the current
summary chronic health indicator, it also obscures important value judgments about the
relative importance of these endpoints. A single summary indicator also precludes
independent evaluation of cancer and noncancer endpoints.
The Agency's toxicity weighting system establishes an equivalency of cancer
and noncancer impacts: a cancer risk of 10"4 is scored as equivalent to a noncancer
hazard index of 1. While this equivalency has always been implicit in the MRS scoring
system, it has never been publicly advertised as Agency policy and involves value
judgments that may vary substantially among stakeholders in risk debates. Lowering
the equivalence level to 10"5 or 10"6 would have the effect of focusing more attention on
carcinogens, while raising it would focus more attention on noncarcinogens. The
Subcommittee suggests that separate indicators be considered for cancer and
noncancer impacts and that these indicators be constructed in a manner that will allow
them to be combined into a single summary chronic human health indicator.
The Agency may require a scoring system that generates a single summary
chronic human health indicator. If so, there are approaches that use actual toxicity
values (not binned weights); these approaches can be designed such that they do not
predict the incidence of adverse health effects. These approaches are not very
susceptible to misuse or misinterpretation. To use this approach, the Agency would
first develop "benchmark" levels of acceptable exposure based on actual toxicity
values. These benchmarks would be established by Reference Concentrations/Doses
for noncarcinogens. Benchmarks for carcinogens would be established by converting
carcinogenic potency values to "Risk Specific Doses" (RSDs) by dividing the potency
into the risk value selected to be equivalent to exposure to the Reference Dose,
currently 10"4. Two examples of summary scoring systems are discussed in Appendix A.
19
-------
3.1.1.4 Compression of Carcinogenic Categories
Finding: The Subcommittee supports the approach employed for the compression of
the carcinogenic categories.
The Subcommittee supports the approach for similar weighting of EPA Category
A, B1 and B2 carcinogens. The Subcommittee agrees that it is appropriate to treat
Category C carcinogens differently by reducing their potency values. The extent of the
reduction (one order of magnitude in the current Indicator methodology) is a policy
judgment to be made by EPA; there is no scientific rationale for using a factor of ten.
For the long term, this approach to incorporating current weight of evidence
evaluations of carcinogens into the Indicator will need to be revised to accommodate
the new EPA carcinogen classification system that will emerge from the Agency's
revision of its cancer policy guidelines. This classification abandons the alphanumeric
categories used by the current Indicator methodology and establishes narrative
categories that are less likely to support quantitative adjustment factors.
3.1.1.5 Accommodation of Severity
Finding: The Subcommittee supports the approach employed for accommodating the
severity of endpoints.
The Subcommittee noted that toxicity values are currently not derived using
factors to account for severity of endpoint, and that there is no generally accepted
weighting scheme for increasing or decreasing the value of a RfD or RfC based on
severity evaluations [How to account for severity of endpoints is an important issue, but
the Agency cannot be expected to deal with it in the TRI indicator methodology. It
should be handled by the Agency in some other arena such as the Agency's Risk
Assessment Forum with input from outside parties.]. Given the Indicator project's need
to rely on authoritative toxicity data and the value of having a quantitative hazard
component in the Indicator, the Subcommittee believes it is reasonable to assess
hazard based on most sensitive effect, without severity adjustments.
3.1.1.6 Accommodation of Multiple Effects
Finding: The Subcommittee supports the approach employed for accommodating
multiple effects for the present version of the indicator methodology.
The Subcommittee noted that toxicity values are currently not derived using
factors that account for a chemical's ability to cause multiple organ effects, and that
there is no generally accepted approach to modifying the value of a RfD or RfC based
on multiple effects. The Subcommittee agrees with the Agency's position of not
20
-------
incorporating a weighting mechanism for multiple effects into the present TRI
methodology.
In the short term, the Agency could consider supplementing a core noncancer
impact indicator (which is a quantitative function of RfD/RfC, surrogate dose and
population) with qualitative indicators summarizing the mass release of compounds
affecting distinct endpoints/organ systems (e.g., pounds released of
developmental/reproductive toxicants). This would represent a first step towards
addressing concerns regarding the need to track trends of chemicals with multiple
effects. The Federal Register documentation supporting the listing of TRI compounds
identifies specific endpoints of concern for TRI chemicals (e.g., neurotoxicity,
immunotoxicity). These endpoints could be utilized to organize health endpoint
subsets of TRI chemicals. California's Air Toxics program has also classified a large
number or toxic air contaminants by the endpoint affected. Since the toxicity values
underlying the quantitative noncancer index are based on a compound's most sensitive
effect and are unlikely to be appropriate for scaling a compound's capacity to affect
multiple, different endpoints, it will not be possible to generate toxicity-weighted
releases or relative-risk adjusted releases for these subsets.
3.1.1.7 Introduction of Policy into Toxicity Weighting
Recommendation: The Subcommittee recommends that Agency introduce policy in a
transparent manner and do so only after completing calculation of the TRI indicators.
The review document, Toxic Release Inventory Relative Risk-Based
Environmental Indicators: Interim Toxicity Weighting Summary Document, states that
the "selection (of) toxicity weights provide EPA with an opportunity to consider
important policy issues in determining final weights". The document explains further
that lead is a priority chemical that the Agency may want to "highlight" and "target".
The Subcommittee believes that the introduction of policy, especially in a non-
transparent manner, into a model that needs to be scientifically defensible is
inappropriate and will detract from the credibility of the TRI indicators. Because the
present model is not designed to address severity of endpoints, an attempt to address
this limitation for selected chemicals is not only arbitrary but also compromises the
relative nature of the indicators and comparisons across chemicals. The Subcommittee
therefore recommends that both consideration of the severity of endpoints and the
introduction of Agency policy be transparent and reserved until calculation of the TRI
indicators are completed.
21
-------
3.1.2 Appropriateness of Approach for Assessing Exposure
Recommendation: The Subcommittee recommends that OPPT replace or modify the
deficient exposure models presently incorporated into the TRI Relative Risk-Based
Environmental Indicators with others that are presently used within the Agency. The
Subcommittee recommends that the Agency review all exposure models but in
particular focus on those for surface water, land and POTW releases. The use of more
appropriate exposure models with region-specific data (and, when available,
site-specific data) will improve the precision and accuracy of the relative risk-based
indicators. The Subcommittee believes these improvements will be most important
when applying the methodology on a regional or site-specific basis.
The TRI indicator methodology is a screening level tool whose proposed use is
for identifying trends and for prioritizing. Due in large part to its attempt to model
exposure, the Subcommittee finds the TRI indicators to be an improvement over
ranking and trending based on pounds alone. The following are comments regarding
the exposure approaches employed to calculate the TRI indicators. The comments are
categorized into three somewhat overlapping classes according to the nature of the
comments: general comments, comments pertaining to pathways, and comments
regarding assumptions.
3.1.2.1 General Comments
The Subcommittee believes there are more appropriate fate and transport
models than those employed by the TRI methodology. The EEC has previously
provided comments on such models, including where and how they should be applied.
The Agency should review alternative models (including Cal TOX) to determine
whether they would be more appropriate than those presently incorporated into the TRI
methodology.
The TRI indicators methodology is designed to accommodate various types of
analysis, such as aggregation and de-aggregation; various model options from pounds
alone in the model to the full model of hazard; surrogate dose and receptor population;
and modification of default parameters. However, uncertainty in the exposure-dose
approach increases as the analysis moves from a larger-scale, more aggregated level
(national) to smaller-scale more disaggregated levels (local/site-specific). With this
loss of resolution, the ability to say something meaningful about the indicators at the
local/site-specific level is compromised. Therefore, the Agency should provide some
guidance to users about the level of confidence in the indicators at different levels of
analysis/application, and at what level of analysis, site or regional-specific data are
preferable and should be used in place of default assumptions.
22
-------
Although the TRI indicator is a first step toward assessing trends and prioritizing
environmental impacts, the methodology offers little insight concerning actual impact(s)
of the TRI chemicals on community health. Community health is commonly assessed
through epidemiology, which is the study of factors of health and disease in human
populations. Epidemiological studies often rely heavily on more sophisticated analyses
of exposure and effects information. Individualized exposure and effects information is
especially useful in such analyses. Because the TRI release data are often estimates
based on crude mass balance calculations rather than actual monitoring, it is difficult to
determine what they mean in terms of individual exposures. Because assessment of
community health requires monitoring, surveillance, and sophisticated analyses beyond
that which the TRI Environmental Indicators methodology provides. Therefore the
Agency should repeatedly inform the users of the methodology about these limitations
and of the proper uses of the TRI indicators. Two special cautions follow.
a) First, although TRI is one the most extensive databases on environmental
releases, the TRI data are limited because they capture only a small
percent of actual releases and transfers of all chemicals from all sources.
For example, only manufacturers with ten or more employees who either
use 10,000 pounds or manufacture 25,000 pounds of one of the listed
chemicals must report. In the South and Southwest regions of
Philadelphia, only 11 of approximately 250 facilities listed in federal and
state environmental regulatory databases are required to submit TRI data
(Burke et. a/, 1997). Interpretation of TRI indicators should reflect this
limitation.
b) Secondly, the lack of error bounds make it difficult to distinguish "noise"
levels from changes in environmental quality or true difference between
chemicals and facilities. Particularly at the smaller, disaggregated scale,
the "noise" levels may be quite large.
3.1.2.2 Comment Regarding Pathways
The Subcommittee recommends that the Agency evaluate all potential exposure
pathways before excluding any from consideration. Then the Agency should document
the logic for including or excluding a pathway and also state the ramifications of these
decisions on the accuracy and uncertainty of the TRI model. For example, the surface
water pathway should be considered for inclusion in the TRI model because of the
potential for exposure through recreational use. The TRI model does not adequately
account for the dietary pathway because some chemical releases through the air
pathway can result in more human exposure/risk via the diet than via inhalation.
Vegetation, fruits and vegetables can become contaminated directly by atmospheric
deposition or indirectly through uptake from contaminated soil and this may be passed
23
-------
on to farm animals and poultry. Similarly, dietary exposure can occur when
contaminated surface water is used for irrigation.
3.1.2.3 Comments Regarding Assumptions
Due to the conservative assumptions made in some exposure models,
adjustment factors of 5 or 10 are used to adjust down "surrogate" doses. Nevertheless,
in certain situations, default assumptions may not be as conservative as suggested.
For example, the fish consumption value (6.5 g/day) in the surface water pathway is
based on the per capita mean for the U.S. population, yet the exposed population who
fish and eat fish are probably consumers in the upper percentile range [If the 6.5 g/day
figure is retained, it should probably apply to all the people in an area; otherwise a
higher consumption figure is probably appropriate.].
For the air dispersion model, a steady-state model is used to estimate
concentrations of pollution in each of the 441 cells surrounding a particular facility.
However, it is not clear in the TRI documentation whether this model is predicting the
maximum concentration in the cell (e.g., centerline concentration) or some other
parameter (e.g., maximum ground level concentration). During the review, Agency
personnel indicated that, for sparsely populated rural areas, the number of people
needed to reach 1000 were distributed evenly across the 441 100-hectare cells. The
Agency should document its chosen method of distribution for heavily or sparsely
exposed populations and provide an explanation for its choice.
EPA's choice of air modeling parameters minimizes plume rise. This choice will
generally predict higher ground-level concentrations close to the facility and lower
concentrations further away than would be the case if the plume rose higher. For the
air pathway, stack height is currently set at 10 meters. The actual range in stack
heights can be wide. Information from one set of air emission data showed stack height
can range from 6 to 76 meters. Perhaps, distribution of stack heights could be
empirically derived from existing data and the high, mid, and low values can be
identified, or possibly stack heights could be correlated with SIC or facility type. The
choice of stack heights is further complicated by the fact that the same model is
employed to model fugitive emissions. The Agency should comment on the rationale
and impact of their choice.
The Subcommittee believes that the Agency should review its assumptions for
groundwater releases from land disposal units. For the groundwater pathway, the
simplified TRI methodology for quantifying contaminant releases is prone to significant
errors because it does not accommodate the diversity of waste management facilities
and subsurface disposal schemes used for TRI wastes. A general assessment of the
configurations, designs, and construction materials used by industry for the land
disposal of TRI wastes would improve understanding of the source term(s).
24
-------
The methodology could be improved by establishing a subcategory for exposure
for groundwater contaminated by landfill leachate that accounts for the wide range of
landfill conditions actually seen. The current landfill model does not account for the
depth to groundwater when evaluating releases to land, instead the model sets
groundwater exposure concentration equal to a hundredth of the leachate
concentration. This does not fit with actual experience. In the west and southwestern
US, depths to groundwater in subtitle D landfills may be on the order of 100 to 400 feet.
Also, because of the high evapotranspiration rates in these areas, leachate may never
reach groundwater.
One approach would be for the TRI indicators methodology to employ one of the
several landfill models currently used by US EPA to evaluate landfill leachate, for
example the Hydrologic Evaluation of Landfill Performance (HELP) Model. The HELP
model uses publicly accessible data from sources such as the National Oceanagraphic
and Atmospheric Administration, the Natural Resources Conservation Service; and the
U.S. Department of Agriculture.
HELP is one of the most robust and popular landfill leachate generation and
migration models. Developed by the US Army Waterways Experiment Station, HELP
has undergone several revisions since it was first released in 1984. The latest
documentation of HELP is contained in an USEPA publication for HELP version 3.O.5
Other sources of subsurface data for the United States can be found in Guide to
Technical Resources for the Design of Land Disposal Facilities (EPA, 1988). Although
site-specific data would be the most desirable to use as input to any model, reasonable
ground water model outputs can be obtained using data supplied by the Natural
Resources Conservation Service (NRCS).6
The conservative nature of estimated leachate concentrations resulting from
wastes deposited in "non-hazardous" landfills, can be evaluated by comparisons to the
toxicity characteristic criteria. If the estimated leachate concentrations exceed the
criteria for hazardous waste, which are not allowed to be disposed in non-hazardous
landfills, the model may be too conservative.
5Since the release of the 1994 USEPA publication, HELP has undergone additional modifications. The latest version of HELP
is version 3.07. To obtain this latest version, either send two (2) 3.5", high density DOS formatted diskettes to Dr. Paul Schroeder, EE-P,
USAEWES, 3909 Halls Ferry Road, Vicksburg, MS 39180-6199, or get into the FTP site:
Host: 134.164.99.52
Username: helpS
Password: lloyd
(note: all files from the D:\HELP3 subdirectory should be transferred)
6The NRCS is an Agency of the US Department of Agriculture, 817-334-5292orpcole@ftw.nrcs.usda.gov). These data, which
are publicly accessible, include estimations of: (1) soil types, (2) hydraulic conductivities, (3) depth to groundwater, (4) soil chemistry, etc.
of the various geographical areas of the United States.
25
-------
It is unclear as to why the volatilization of substances from wastes deposited in
landfills must be mediated by leaching, as opposed to direct volatilization from the
waste. The TRI documentation should clarify whether this choice was a calculational
convenience or a statement of expected physical behavior.
The methodology documents do not clearly explain how the dilution volumes are
specified for the water discharges from individual facilities. If they are related to
reported river flow rates, then the averaging method should be explained (e.g., annual
average vs. harmonic mean).
The assumption that people drink contaminated surface water without any
benefit of water treatment may be excessively conservative, because most water
systems meet maximum contamination limits (MCLs). The Agency should evaluate the
scores to be sure they do not imply consumption at levels well above the MCLs.
The indicators methodology includes a component for individual public owned treatment we
(POTWs). The POTW release model assumes that all municipal sewage sludge is land filled. In ;
Agency discourages the practice of land filling of sewage sludge (or biosolids) and promotes bene
this material as a more environmentally attractive alternative as described in Land Application of S
Sludge and Domestic Septage (EPA, 1995).
The practice of sewage sludge land application is regulated by the Clean Water Act under 1
requirements set forth in 40 CFR Part 503 (known as the sludge rules). These regulations specify
conditions, sludge quality, sludge application rate, monitoring and sampling frequencies, record ke
that a POTW must comply with to legally land apply sewage sludge. These regulations, which we
by the USEPA, are risk- based and consider the effect of sewage sludge pollutants on highly expo
individuals (human, plant and animal). Results of the human health and environmental risk assess
land application of sewage sludge may be found in the following publications: Technical Support
Document for Land Application of Sewage Sludge (EPA, 1993a) and Standards for the Use
or Disposal of Sewage Sludge (EPA, 1993b).
In developing the sludge rules, the USEPA considered fourteen (14) exposure pathways, irr
nitrogen control for ground water protection, established surface water protection features, implem
pathogen reduction and vector attraction control requirements and imposed metal concentration lii
conservative approach employed by the USEPA in developing the technical basis for the sludge n
reasonable to expect that a POTW that is legally land applying its sludge would not significantly in
human health and/or environmental risk through this disposal practice.
Landfilling of sewage sludge is regulated by Subtitle D of the Resource Conservation and F
(RCRA) if it is co-disposed with other solid wastes (i.e., sludge monofills are regulated by 40 CFR
response to the Hazardous and Solid Waste Amendments of 1984, the USEPA promulgated a fin?
regarding new performance criteria for municipal solid waste landfills (EPA, 1993c). The new perf
criteria include requirements related to facility location, design, operations, monitoring, corrective i
and post closure care.
26
-------
With regard to the impact of landfill leachate on ground water quality, it is important to recoi
the federal mandated performance standards require that a landfill design prevent the concentratk
water pollutants from exceeding maximum contaminant levels (MCLs) at the point of compliance (\
normally the uppermost aquifer at the landfill boundary). The US EPA requires that landfills attem
the performance standard using alternative designs rely on site specific data collection and model
conservative nature of the federal landfill performance standards, it is not unrealistic to expect thai
sludge that is landfilled in a permitted Subtitle D facility would present an insignificant human heal
environmental risk. However, it can be argued that there are many landfills which presently are ei
without a Subtitle D permit or are in various stages of acquiring a permit
The TRI Subcommittee does not believe that the Agency should attempt to identify the type
sludge treatment processes employed at individual public owned treatment works (POTWs). Rath
Subcommittee has suggested that the TRI relative risk model account for the technical difference I
"landfilling" and "land application" (or beneficial use) of POTW sewage sludge.
3.1.3 Appropriateness of Approach for Assessing Exposed Populations
Finding: The present default value for minimum rural populations may artificially
increase relative risks or show a risk when one actually does not exist for these remote
facilities.
Finding: The majority of Subcommittee members believe that an accepted method to
address population changes over time is to use the latest intercensual rate of change
(i.e., average annual rate of change between 1980 census and 1990 census) to project
population size for postcensual years.
Population considerations are a significant component of the TRI model, such
that population change could influence the indicator. An increase or decrease of
population will elevate or lower the indicator even when no change in releases
occurred.
Members of the Subcommittee had concerns regarding the 1000 exposed
persons minimum for air exposure for rural areas and favored the use of actual
population numbers. The Members indicated that many government and industrial
facilities were located in remote areas for the sole purpose of being removed from
residential populations. Presently, the default rural population value of 1000 may
artificially increase relative risks for remote facilities and remove the incentive to locate
facilities in areas removed from adjacent populations. If facility-specific indicators are
developed, a further step of properly assigning the rural populations to individual cells
would also increase the accuracy of lower tier indicators. The Subcommittee is
concerned that the present default value for minimum rural populations may artificially
increase relative risks or show a risk when one actually does not exist for these remote
facilities.
27
-------
Although, population estimates are not usually significant contributors to overall
uncertainty in the model, projection of population changes for non-census years may be
important when the TRI indicators are applied to smaller geographical areas. For
example, population changes can be significant in urban environments. The population
in the south/southwest Philadelphia has decreased by 25% within the last 20 years.
(Burke et. al, 1997. The majority of the Subcommittee believes that an accepted
method to address population changes over time is to use the latest intercensual rate
of change (i.e., average annual rate of change between 1980 census and 1990 census)
to project population size for postcensual years (Shrycock et. al, 1976). A dissenting
Subcommittee member voiced concern over this approach because it was perceived as
encouraging the extrapolation of data, which the EEC has often opposed.
3.2 Are Hazard, Exposure and Population Elements properly integrated?
3.2.1 Hazard
Recommendation: To improve integration of the hazard components, the
Subcommittee recommends that toxicity values be used directly; carcinogenic potency
values be converted to Risk Specific Doses; and indicators be reported to only a few
significant figures.
The TRI model has incorporated a number of approaches from the Hazard
Ranking System (MRS). Some observers have stated that because the MRS toxicity
weighting scheme was developed for a different purpose, it should not be used in the
TRI Indicators Project. The Subcommittee agrees with the premise that the objective
must be considered in determining the scientific validity of a procedure. However, the
MRS and TRI indicator objectives are not sufficiently different to invalidate the use of
the MRS system, especially if the Subcommittee's recommendations for improvement
are utilized. Both MRS and the TRI indicators were developed with the intent of using
surrogate measures of toxicity to arrive at conclusions that relate to risk. SAB
comments on the MRS emphasized the need for risk-based rankings, and changes to
the old MRS were made to improve that relationship.
Although the MRS methodology is not seriously flawed, the Subcommittee
recommends that the Agency consider the three improvements discussed in Section
3.1.1. First, the toxicity values from IRIS, HEAST, and the disposition process should
be used directly rather than rounded by assigning them to bins of an order of
magnitude wide. Second, the carcinogenic potency values should be converted to
"Risk Specific Doses" (RSDs) to allow use of a single index scale for both cancer and
noncancer toxicities. Third, the various indicators should be reported to only a few
significant figures. Although changes in the third significant figure could be meaningful
for the national level indicators, most of the lower tier component scores would be
subject to significant levels of uncertainty. The computer implementation of the
28
-------
indicator methodology should actively discourage use of inappropriate numbers of
significant figures.
Toxicologists generally advise against summing Hazard Quotients (estimated
dose/RfD) over chemicals that do not have the same endpoint and mechanism of
action. However, if one adopts the probability viewpoint about the significance of the
indexes, then a facility with several chemicals will usually be more deserving of
attention than one with only one chemical with a similar component score. Regarding
summing over different media, the underlying (unstated) assumption is that the
exposure weights are roughly in the same proportion to risk for all of the media
calculations, after consideration of route-specific toxicity differences and application of
exposure modeling uncertainty factors. That assumption is critical to the whole TRI
indicator toxicity-weighting scheme, and therefore, the Subcommittee believes that
summing adds no additional difficulties.
The Subcommittee believes that basing the hazard component on actual toxicity
values and incorporation of the above suggestions for risk-specific doses and the
number or reported significant figures offer a mechanism for properly integrating hazard
with exposure and population elements.
3.2.2 Exposure
Recommendation: The Subcommittee recommends that the Agency identify and
evaluate more appropriate exposure models to be used with region-specific data (and,
when available, site-specific data), to improve the relative risk-based indicators.
Further, these models and regional data (and site-specific data when available) should
be used to improve the relative Risk-Based indicator.
In general, the Subcommittee found that the exposure modeling presently used
in the development of the TRI indicators constitutes a first step in identifying the critical
human health exposure pathways. However, there were several major assumptions
used in the model development that could potentially lead to significant errors in
predicting the risk impact of TRI releases. The Subcommittee believes that many of the
technical deficiencies in the exposure pathway models can be corrected using publicly
available models and information such as climatic, soil and groundwater conditions
from the various geographical regions of the United States. To maintain credibility of
the models, the choice of model parameters should be transparent, well documented
and scientifically based. Agency offices, such as the Office of Water, Office of Solid
Waste, and Office of Air and Radiation, routinely use many of the modeling tools that
could be easily imported into the TRI indicator model.
29
-------
The Agency requested detailed information on modeling tools that might be
imported into the TRI Indicator Model. The Subcommittee, therefore, is providing more
detail here.
Mathematical models are useful tools that provide insight into the effects of solid
waste facility design and operation on ground water quality. There are three basic
types of mathematical models that the USEPA currently uses to evaluate Subtitle D
facilities (e.g., landfills, surface impoundments and land treatment facilities). These
are: leachate generation models, leachate migration models, and geochemical models.
Leachate generation models predict the quantity and/or quality of leachate that
will be released from the bottom of a waste disposal facility. Leachate migration
models simulate the transport of contaminants from the source, through the
unsaturated and/or saturated zones, by ground water. Geochemical models evaluate
subsurface chemical processes that affect contaminant transformation and availability
such as adsorption/desorption, precipitation/dissolution, oxidation/reduction, aqueous
speciation and reaction kinetics. A thorough evaluation of the various mathematical
models have been summarized by the USEPA in the following report: Leachate
Generation and Migration at Subtitle D Facilities - A Summary and Review of Processes
and Mathematical Models (EPA, 1993d). It should be noted that numerous models are
described in this report which can be applied to the simulation of leachate generation
and migration. The report describes: a) 27 mathematical models used to predict
leachate generation; b) 39 mathematical models used to evaluate leachate migration
and exposure assessment; and c) six mathematical models used to evaluate
geochemical reactions. The models vary in complexity, from simple analytical solutions
(which can be solved with a hand held calculator) to complex numerical models which
require significant computer capabilities.7
Finally, some of the earlier (and simpler) mathematical models used to describe
contaminant transport may be found in another USEPA publication Ground Water
Transport: Handbook of Mathematical Models (EPA, 1984).
In addition to the Subtitle D facilities fate and transport models previously
described in Leachate Generation and Migration at Subtitle D Facilities - A Summary
Examples of some of the USEPA models described in the report include the following:
Leachate Generation Models
Approximating Pollutant Transport to Groundwater 1982 - USEPA Ada, Oklahoma
Simulations of Leachate Generation from Municipal Solid Wastes EPA/600/2-87/059
Modeling Chemical Emissions from Lagoons and Landfills 1984 - USEPA Athens, Georgia
Leachate Migration Models
Multimedia Exposure Assessment Model (MULTIMED) for Evaluating the Land Disposal of Wastes - Model Theory 1990 - USEPA
Athens, Georgia
A Subtitle D Landfill Application Manual for the Multimedia Exposure Assessment Model - (MULTIMED) 1990 - USEPA Athens,
Georgia
30
-------
and Review of Processes and Mathematical Models (EPA, 1993d), the USEPA has
recently developed its own composite model for landfills (EPACML). The EPACML
model is basically a contaminant fate and transport model which simulates both
saturated and unsaturated subsurface flow under steady and nonsteady conditions.
The model also simulates microbial and physico-chemical contaminant transformations
by incorporating first order reaction kinetics, adsorption and dispersion algorithms.
The EPACML model, which was developed primarily for regulatory rule making
purposes (e.g., toxicity characteristic rule), allows the estimation of the probability
distribution of contaminant concentrations at various exposure points. The model,
which employs a Monte Carlo technique, is not designed for site specific application,
rather, it uses general information including hydrogeologic parameters and relative
positions of exposure points to the contamination source as model inputs.8
With regard to improving the media and pathway approaches for exposure
modeling, the Subcommittee concluded that a sensitivity analysis be conducted as
either part of the overall indicator evaluation or as part of a subindicator analysis on
exposure pathway in order to determine the impact of the various components of the
model on the outcome of the final TRI indicator. For example, comparison of the value
of the TRI relative risk indicator using results from a simple groundwater model versus
the value of the TRI relative risk indicator using the results from the present generic
approach could be used to gauge the relative error involved in the present groundwater
exposure modeling approach. The uncertainty adjustment factor should then be
modified to reflect the magnitude of this error.
3.2.3 Population
The present model takes into account the total population for a model-specified
cell, area, reach or other zones of influence. While the population estimates are likely
to contribute little to the uncertainty of national indicators, as the TRI model is applied
to smaller geographical areas, the contribution of population estimates to indicator
uncertainty is likely to be more significant. It is also recognized that population
changes in the zone of influence of a facility would increase or decrease its component
scores even though no change in its releases occurs. TRI documentation should
clearly explain this impact of population on TRI indicators and how local population
changes could result in misleading conclusions regarding a facility's environmental
management practices.
The Subcommittee determined that actual population data, where available,
should be used in estimating risk(s).
Details ofthe EPACML model, including background documentation and model validation, may be obtained from the USEPA Office
of Solid Waste (Dr. Zubair Saleem - Tel. no. 703-308-0467).
31
-------
3.3 Will Methodology provide Reasonable Relative Risk-based
Analyses and Impacts of TRI Chemical Emissions?
Finding: Relative uncertainties increase as one moves from a larger, more aggregated
level (national) to smaller, more disaggregated levels, i.e., local/site-specific. With this
loss of resolution, the ability to say something meaningful about the indicators at the
local/site-specific level is compromised.
Recommendation: The Subcommittee recommends that the Agency identify and
clearly present in its documentation the specific universe of environmental risks
addressed by the TRI model.
Finding: The Subcommittee believes the Agency has to be realistic with regard to the
continuum of applications to which the TRI indicators are likely to be applied and
incorporate more warnings into its documentation and reports, and where appropriate,
into its computer program to decrease unintentional misuse and to discourage
intentional misuse.
Due to the similarity of its responses, the Subcommittee took the liberty of
combining in this section, responses for both General Charge questions a)3) and a)4).
According to the Executive Summary of the TRI Indicators document, "the
objective of [the indicators methodology] is to calculate a unitless value that reflects the
overall risk-related impacts of releases and transfers of all included TRI chemicals from
all reporting facilities to each environmental medium for a given year or years." With
one major exception and a number of minor ones discussed below, the methodology
should serve this narrow objective.
The methodology will not perform as well for the subsidiary uses that are
mentioned as possibilities: geographic analyses, chemical-specific analyses, industry
sector analyses, environmental justice analyses, and so on, principally because any
inaccuracies or biases will be magnified at lower levels of aggregation. Because the
temptation to apply the indicators methodology to such other uses will be strong, the
document should supply further cautions about its applicability.
One great strength of the TRI documentation is its clarity of presentation. The
Subcommittee found little trouble in understanding how the methodology was
developed or how it would operate. In particular, the document makes it clear that the
definition of risk that it has adopted is the public health definition: simulating the likely
magnitude of impacts on the population, rather than the potential risks to highly
exposed persons. A minor exception occurs because the methodology establishes a
minimum size for the population at risk for a given facility's releases.
32
-------
The Agency has correctly chosen a "proportional" weights approach. The
algorithm for calculating the indicator combines and scores the weights in the same
way they would be in a more complete risk assessment, again with one glaring
exception - the binning of toxicity values. In many respects, the current indicator
methodology for chronic human health amounts to a crude risk assessment that should
produce component scores roughly proportional to risk, albeit with substantial
uncertainties and biases, which may not be consistent for different components of the
methodology. Because the Agency seemed to recognize the virtues of the proportional
approach, it is curious that it chose to use a severely constrained version of that
approach for the toxicity weight by binning toxicity values and truncating their range.
As explained in Section 3.1.1.1, the Subcommittee recommends using toxicity values
directly rather than binning them and believes that incorporation of this
recommendation will result in more reasonable risk-based analyses.
The reliability of the indicators may depend on the coverage of the TRI list of
reportable releases. Although the proposed methodology considers the majority of
chemicals that have significant releases. If the Agency has any way of characterizing
the effect of omitted substances, it should present that information. Although this
problem may not have a strong effect on the reliability of a national chronic health risk
indicator, it is more likely to make comparisons on a smaller scale, such as interfacility
or environmental justice, unreliable. Similarly, the Agency should attempt to consider
the impact of not considering releases from facilities excluded from the TRI reporting
requirements and the impact of chemicals not presently listed by the TRI. The
Subcommittee recommends that the Agency identify the TRI slice of the universe of
environmental risks and clearly present these limitations in its hard copy and electronic
documentation and reports. The TRI relative risks indicators have to be presented in
the context of non-TRI chemicals releases, exposures from other pathways such as
indoor air, and the selective consideration of human populations at the exclusion of
others such as avian and terrestrial animals.
The description of how the indicator methodology will be modified to
accommodate changes in TRI reporting requirements is not very specific. Although the
document states that it will continue to calculate a set of indicators based only on the
original list of facilities and chemicals, it is not clear whether that list is the 1987 or
1994 version, or what additional indicators might be constructed. The flexibility of the
TRI model makes it imperative that the Agency develops a means of ensuring that all
assumptions, defaults, bases of comparisons and changes from previous models or
reports are obvious to the potential user of the data.
The Agency has wisely recognized that the TRI indicators are ripe for intentional
and unintentional misuse. This recognition has resulted in the repeated declarations
found in the TRI documentation that the indicators are relative and should not be used
to measure risk. In conflict with this recognition is a tool that has so much ability for
33
-------
displaying facility-specific information and comparisons among chemicals using
exposure and risk estimates as a sort. The Agency has to be realistic with regard to
the continuum of applications to which the TRI indicators are likely to be applied and
continue to incorporate even more prominent warnings into all new documentation and
reports, and where appropriate, into its computer program to decrease unintentional
misuse and to discourage intentional misuse.
3.4 Identify Future Research Needs
Although the Subcommittee recognizes that considerable work has already been
done to develop the indicator methodology, there is disagreement among the
Subcommittee members as to whether more development is required before the
indicator methodology can be credibly and reliably used for its intended purposes. The
Subcommittee recommends that the Agency address these fundamental research
needs and issues to verify the models scientific basis and decrease the uncertainties in
the resulting indicators.
3.4.1 Sensitivity and Uncertainty Analysis
Recommendation: The Subcommittee recommends that the Agency subject the
methodology to sensitivity and uncertainty analyses and portray uncertainty in the final
results. The Subcommittee believes this will allow potential users the ability to use the
output with the proper confidence.
As currently presented, the output from the methodology (numbers as well as
graphs) implies an accuracy that is far greater than the quality of the input data and the
models used to generate it. This is a typical problem with computational models that do
not explicitly consider uncertainty. The developers should take steps to limit the
number of significant figures in the final results to an appropriate number and to
incorporate a level of uncertainty into the final results. It is not sufficient to caveat the
limitations of the model in associated documentation. There should be an explicit
indication of uncertainty in the reported results. The Subcommittee believes that lack of
such qualifiers must be corrected if the results are to be used with confidence to draw
conclusions and make meaningful comparisons. The developers must address the
uncertainty question as a high priority item and take steps to portray uncertainty in the
final results. The Subcommittee believes this will be a valuable and necessary exercise
that will allow potential users the ability to use the output with the proper confidence.
The TRI indicators documentation as well as presentations during the review
imply that the developers of the methodology, except for stack heights, have not
performed a sensitivity analysis that relates variations in the final results to variations in
the value of input parameters and assumptions. In the case where the model's
sensitivity to certain parameters (i.e., variation in stack height) has been investigated,
34
-------
the results led to a far better appreciation of the limits of the model. The Subcommittee
believes it is critical to understand the internal sensitivities of the model as a limitations,
but also focusing effort on improving the accuracy of those input parameters that make
the most difference. Sensitivity analysis can result in numerous benefits, one of which
is the ability to focus future research to minimize uncertainty in the resulting indicators.
Sensitivity analysis will explain the relative influence of model components (i.e.,
hazard, exposure-dose, and population) on the final indicators. This kind of analysis
would provide some comfort level as to whether population size alone or some other
component or assumption had undue influence on the final indicator. It would be useful
for the developers to develop a summary table of the direction of bias in the default
parameters and assumptions to provide a basis for the up/down adjustment of
indicators. This summary table could facilitate future uncertainty analyses as default
parameters are replaced with more realistic values.
In summary, uncertainty and sensitivity analyses can overcome the present lack
of error bounds that make it difficult to distinguish "noise" levels from real changes in
risk. This problem is likely to be exaggerated at the disaggregated level, because at
this level the "noise" may be relatively large, requiring a sound understanding of
uncertainty before lower tier indicators could be used to support environmental
decision-making.
3.4.2 Validation of Output
Recommendation: The Subcommittee recommends that the model's output should be
validated or "ground-truthed" by applying the methodology to a series of cases, for
which exposure concentrations were estimated using other exposure models.
The Subcommittee believes the model's output should be validated or "ground-
truthed" by applying the methodology to a series of cases for which exposure
concentrations were estimated using other accepted exposure models. The
Subcommittee recognizes that any validation is complicated by the relative nature of
the indicators and the uncertainty of the TRI release data, but in most situations at least
exposure model components should be capable of validation. Lacking some sort of
validation, questions will remain about the applicability of the model. Facilities/locations
for which there are exposure monitoring data should be identified for this validation
efforts. The EPA-NHEXAS exposure data are another potential source for validation.
NHANES IV is in the planning phase at the current time; a consideration may be given
to design into NHANES IV a component to validate the TRI indicators.
The issue of ground-truthing can be applied to model inputs and interim model
results such as typical surface water contamination levels. Outreach efforts to other
sources in the Agency, and in other parts of the government (such as the USGS and
35
-------
NMFS and F&W service) as well as to NGO's such as the Nature Conservancy and the
Audubon Society, to secure additional sources of data. Under certain circumstances,
strategies to measure actual levels of key pollutants in the environment such as
ambient monitoring, "fence line" monitoring, household measurements, and individual
sampling may be appropriate.
The validation of model inputs can also be applied to the TRI reported releases.
During the review it was suggested that the quality of TRI data may vary by industry
segment or facility type. This variation reflects the different levels of sophistication in
data collection and management that exists across industries. A review of the
uncertainty in the reported releases and its impact on the relative ranking of indicators
should be considered.
3.4.3 Unintended Consequences
Recommendation: The Subcommittee recommends that the Agency review the model
to detect aspects of the model's algorithm that may result in unintended consequences.
Past tax laws encouraged personal debt and present tax laws can encourage
businesses to make decisions for tax purposes, not for the sake of their long-term
welfare. Environmental regulations can also encourage unwanted results. The Agency
should consider the possibility that some companies may manage their environmental
responsibilities for the sake of a favorable TRI rating as opposed to managing for the
sake of the environment and the public health.
The TRI indicator model, because it addresses many of the factors that reflect
the relative risks of releases, offers an opportunity to decrease unintended
consequences by encouraging decision makers to make decisions that have the
greatest chance of lowering risk. However, as presently constructed, the TRI
methodology may not accurately reflect the relative risk and could result in unintended
and unwanted consequences. An accurate allocation of relative risks to chemicals and
facilities would reward a facility manager with greater credit when focusing on the
greatest risk. The following model assumptions and structure do not accurately reflect
actual risks and could discourage a facility manager from addressing the most
significant risks.
The "severity-of-effect" is not considered when assigning toxicity weights, and
the model could encourage a facility manager to focus on a chemical that displays less
severe effects at lower concentration when the largest risk may be from a chemical with
a less sensitive yet more severe endpoint [Due to the subjective nature of assigning
values such as severity, the TRI model understandably does not address this
contentious issue. However, the methodology documentation and repeated use of
caveats should encourage decision-makers to consider severity when making
36
-------
comparisons across different chemicals.].
Onsite and offsite management of hazardous waste are ranked differently
(onsite air releases are incorporated into reported release estimates, while offsite air
releases are considered negligible and are not accounted for) by the TRI model,
possibly encouraging a facility manager to invest resources in a less ideal waste
management alternative that improves his TRI standing.
As discussed in Section 3.1.3, the default rural population value of 1000 may
artificially increase relative risks for remote facilities and remove the incentive to locate
facilities in areas removed from adjacent populations.
The TRI indicators capture only a percentage of total chemical releases and,
because releases from smaller businesses, utility power plants and transportation
sources are not considered, reported releases of certain chemicals from a given facility
could be a minor contributor to regional releases and not be recognized as such.
Recognizing that the complexities of the TRI regulations and the TRI indicators
can result in unintended consequences when used to support lower tiered decisions,
and that industry will consider its TRI ranking when making environmental-management
decisions, the Subcommittee recommends that the Agency review the model to 1)
ensure that its indicators reasonably reflect the relative magnitude of risks, and 2) to
detect aspects of the model's algorithm that may result in unintended consequences.
3.4.4 Other Research
As additional evaluations, field-testing and validations are completed, the TRI
indicators have the potential to lead toward a better understanding of environmental
and human health risks from toxic releases. To better exploit the TRI database and the
TRI indicators, the Subcommittee suggests that the Agency consider the following
topics for future research:
a) EPA should review the toxicity rankings from the disposition process with
an independent group of toxicologists (within and/or outside the Agency)
to identify any that seem unusual and verify whether the interpretation of
the base information was appropriate.
b) Evaluate alternative exposure models for the pathways presently
addressed by the TRI indicators and complementary exposure models for
pathways not presently addressed by the model (e.g., such as
consumption of foods contaminated from deposition of air pollutants).
c) Research into better sources of data for the model should be a continuing
37
-------
need that should be given continuing attention. In this search for better
data, the Agency should consider existing sources of data that could be
relevant to their work, such as facility monitoring data associated with air
permits and wastewater discharges and CERCLA releases. The need for
better data increases with the desire to disaggregate national level
information and move down to the local or plant-specific level.
d) Estimate what percentage of total chemical releases are captured by the
TRI database, because releases from smaller businesses, utility power
plants and transportation sources are not considered.
e) Identify risks such as those resulting from exposure to indoor air pollution
that are not captured by the TRI database and indicators so that the risk
of TRI chemical releases can be placed in the proper perspective.
f) Evaluate whether the TRI methodology lends itself to visualization
technologies such as Geographic Information Systems (CIS).
g) Elicit input and dialogue with local and state public health officials
regarding the TRI indicator project as part of its ground-truthing and to
devise means for local officials to better use the indicators and to better
address public questions/concerns regarding the TRI indicators.
38
-------
REFERENCES CITED
Burke, T.A., and N. Shalauta. 1997. Pilot Multimedia Environmental Health
Characterization Study of South and Southwest Philadelphia. John Hopkins
University, School of Hygiene and Public Health, Baltimore MD.
EPA. 1997. Toxic Release Inventory Relative Risk-Based Environmental Indicators:
Interim Toxicity Weighting Summary Document. U.S. Environmental Protection
Agency, Washington DC.
EPA. 1995. Land Application of Sewage Sludge and Domestic Septage. (625/R-95/001) U.S.
Environmental Protection Agency, Washington DC.
EPA. 1993a. Technical Support Document for Land Application of Sewage Sludge (vol. I
NTIS PB93-110575, U.S. Environmental Protection Agency, Washington DC.
EPA. 1993b. Standards for the Use or Disposal of Sewage Sludge (58 Federal Register 9248).
U.S. Environmental Protection Agency, Washington DC.
EPA. 1993c. So//cf Waste Disposal Facility Criteria - Federal Register 56 (196) 50978-51119. U.6
Environmental Protection Agency, Washington DC.
EPA. 1993d.. Leachate Generation and Migration at Subtitle D Facilities - A Summary
and Review of Processes and Mathematical Models (EPA/600/R-93/125) (USEPA
40 CFR Parts 257 and 258. U.S. Environmental Protection Agency, Washington DC.
EPA. 1988. Guide to Technical Resources for the Design of Land Disposal Facilities
(EPA/625/6-88/018). U.S. Environmental Protection Agency, Washington DC.
EPA. 1984. Ground Water Transport: Handbook of Mathematical Models - EPA-600
LT-84-051, NTIS PB 84-222694. U.S. Environmental Protection Agency,
Washington DC.
SAB. 1997. Review of the Guide to Technical ResourcesSector Facility Indexing
Project (EPA-SAB-EEC-97-012). Science Advisory Board. U.S. Environmental
Protection Agency, Washington DC.
Shrycock, S.H., Siegel, J.S. and Associates. 1976. Condensed Edition by E.G.
Stockwell. Study in Population - The Methods and Materials of Demography.
Academic Press, Inc. CA.
R-1
-------
APPENDIX A
Examples of Summary Scoring Systems Employing Actual Toxicology Values
Section 3.1.13 discusses the advantages of scoring system that support the
creation of a summary chronic human health indicator that combines cancer and
noncancer impacts. The remainder of this appendix discusses two options for
constructing chronic human health indexes that utilize actual toxicity values and not
binned weights. These options do not predict the incidence of adverse health effects,
and are therefore not likely to be susceptible to misuse or misinterpretation. Both of
these options require the Agency to develop "benchmark" levels of acceptable
exposure based on actual toxicity values. These benchmarks would be established by
Reference Concentrations/Doses for noncarcinogens. Benchmarks for carcinogens
would be established by converting carcinogenic potency values to "Risk Specific
Doses" (RSDs) by dividing the potency into the risk value selected to be equivalent to
exposure to the Reference Dose, currently 10"4.
Option 1: This option employs the measurement of potential health impact as
population-weighted "exceedances" of chemical-specific benchmarks.
Benchmark levels of "acceptable" exposure would be defined by RfCs or RfDs
for noncarcinogens, and RSDs for carcinogens. The index would be a
population-weighted count of when predicted surrogate doses for a chemical
from a facility exceed applicable benchmark levels. In essence, this option
extends the current practice of calculating hazard indices for noncarcinogens to
carcinogens. It is also analogous to the conventional way of doing noncancer
risk characterization for criteria air pollutants, where the population size exposed
above a NAAQS standard is estimated. The ratio of predicted surrogate dose to
RfC/RfD/RSD would be calculated and the population exposed to hazard indices
greater than 1 would be estimated. This option would allow users to calculate
separate cancer and noncancer indices, as well as an integrated chronic human
health index that would be constructed by summing the population-weighted
cancer and noncancer exceedance counts. The major assumption underlying
this option is that exposure to less than the benchmark level is not considered to
pose health risk and does not contribute to the index (i.e., hazard indices less
than 1 are considered "safe"). While this is a standard assumption in noncancer
risk assessment, not counting potentially carcinogenic exposures that pose risks
lower than the benchmark RSD level is likely to be more controversial. There is
considerable policy debate about what benchmark cancer risk level should be
considered cte minimis'. some stakeholders might argue that the RSD/RfC
equivalency should be set at 10"6 rather than 10"4. Because the relative
emphasis to be given to cancer vs. noncancer endpoints is a policy judgment
and not a scientifically resolvable issue, the Subcommittee recommends that the
TRI tool support user selection of a cte minimis "equivalence" level for cancer
and noncancer endpoints, even if EPA wants to establish as a matter of policy
A-1
-------
the MRS 10"4 equivalence level for its default national indicator construction.
This methodology is already being utilized to evaluate chronic human health
impacts in the Office of Policy Planning and Evaluation's Cumulative Exposure
Project, which has been reviewed positively by a different SAB Subcommittee, it
is to EPA's advantage to be consistent in these risk indexing methods.
Option 2: This option employs the measurement of potential impact as toxicity-
weighted population dose. This index could be derived by calculating chemical-
specific total doses to a population (population x surrogate dose) and then
dividing by a toxicity benchmark (RfC/D for noncarcinogens, or RSD for
carcinogens). For any given population dose, a larger contribution to the index
would be made by higher toxicity chemicals (because they will have lower
benchmark levels). Like the Agency's current indexing method, this option is not
entirely consistent with conventional noncancer risk assessment practice.
Rather than presume there is a threshold dose below which there is no risk of
deleterious adverse effects, these methods presume some potential for
noncancer impacts at any dose. Release and exposure situations that result in
an estimated surrogate dose that is less than the RfC/RfD still contribute to the
index developed using these methods (i.e., the index is calculated by multiplying
population exposed times chemical-specific hazard ratios of dose to benchmark,
even if hazard ratios are less than 1). This inconsistency with conventional
noncancer risk assessment practice might be warranted, however, in a
screening level tool. Option 1 only includes people whose surrogate dose is
above the benchmark dose, but none of those below. To the extent that
surrogate doses are biased toward overestimation (as a result of generic model
assumptions), Option 1 can make an artificial distinction between facilities
producing doses just above the benchmark and those just below. For screening
level assessments, Option 2 may better accommodate exposure-modeling
uncertainties.
Although Option 2's index is harder to interpret than the "body" counts produced
by risk assessment methods, or the "exceedance" counts produced by Option 1, this
might actually limit the potential for misinterpretation of the TRI indicators. The Option
2 index is neither a population risk for cancer nor an estimate of the number of people
potentially affected: it is an indicator of the relative probability of some adverse health
effect occurring, in the sense of the chance that some level of concern will be
exceeded. If the biases are reasonably uniform from facility to facility and chemical to
chemical, resulting rankings will be reliable, even though the relationship to real risk
may be exaggerated. Of course, the biases will not be uniform, and so there is a high
likelihood of mischaracterizing the relative risks among facilities, chemicals, and so on.
That is why the indicator will be most useful for discerning national trends, and much
more uncertain when it is applied at the micro-level for comparing facilities or
communities.
A-2
-------
APPENDIX B
Documents Reviewed
Bouwes, N.Wand S.M. Hassur. Toxic Release Inventory Relative Risk-Based
Environmental Indicators: Interim Toxicity Weighting Summary Document. June 1997.
Bouwes, N.W. and S.M. Hassur. Toxic Release Inventory Relative Risk-Based
Environmental Indicators Methodology. June 1997.
Bouwes, N.W. and S.M. Hassur. Toxic Release Inventory Relative Risk-Based
Environmental Indicators: Summary of Comments Received on the Draft 1992
Methodology and Responses to Comment. May 1997.
Hassur, S.M. Memorandum entitled "Issues Related to the Direct Use of Toxicity
Values", July 18, 1997.
Oakes, J.M., Letter from New England Research Institutes. 6/24/97
B-1
-------
DISTRIBUTION LIST
The Administrator
Deputy Administrator
Regional Administrators, Regions 1-10
Assistant Administrators
Directors, Office of Research and Development Laboratories
Director, Office of Air and Radiation
Director, Office of Atmospheric and Indoor Air Programs
Director, Office of Policy Analysis and Review
Director, Office of Environmental Processes and Effects Research
Director, Office of Policy Analysis
Director, Climate Change Division
EPA Headquarters Library
EPA Regional Libraries
National Technical Information Service
Congressional Research Service
Library of Congress
------- |