EPA Screening Evaluation Report
Presentation and Discussion of Uncertainty and Variability in IRIS
Assessments
July 2000
National Center for Environmental Assessment
Office of Research and Development
US Environmental Protection Agency
Washington, DC
-------
AUTHORS, CONTRIBUTORS, AND REVIEWERS
Author
Karen Hogan
IRIS Staff
National Center for Environmental Assessment
Office of Research and Development
US Environmental Protection Agency
Washington, DC
Internal EPA Reviewer
Hugh Tilson
National Health and Environmental Effects Research Laboratory
Office of Research and Development
US Environmental Protection Agency
Research Triangle Park, NC
-------
List of Tables
Table 1: Upper 95% bounds on 0% responses for selected toxicity study group sizes
Table 2: Criteria for classifying extent of presentation and discussion of variability and
uncertainty
Table 3: Extent of documentation of variability and uncertainty in a 10% random sample of
pre-Pilot IRIS Assessments.
Table 4: Extent of documentation and variability in post-Pilot IRIS Assessments
Table 5: Correspondence between two independent assignments of IRIS assessments to
categories of extent of documentation of variability and uncertainty.
Appendix A: Summary of classifications for screening sample of IRIS assessments
Appendix B: Independent verification of assignments to documentation categories
111
-------
Summary
In support of the study that Congress has directed EPA to conduct (via HR-106-379),
evaluating the extent of documentation of uncertainty and variability in IRIS assessments, EPA
has carried out a screening of 67 of the 536 IRIS assessments on-line as of 1/31/2000. The
purpose of this screening is to survey broadly the extent of this documentation in IRIS
assessments, in order to facilitate an in-depth evaluation of a smaller, but representative set of
IRIS assessments, to be carried out by a contractor. A simple random sample comprising 10% of
the pre-Pilot IRIS assessments (52/522), plus all of the 15 Pilot and post-Pilot IRIS assessments
were stratified into three categories, those with none/ minimal, some/ moderate, or extensive
presentation and discussion of uncertainty and variability. This report summarizes this screening
effort.
Introduction/Background
As pointed out in Science and Judgment in Risk Assessment (NRC, 1994) EPA has
historically incorporated uncertainty in health risk assessments in a qualitative manner. There are
no quantitative uncertainty analyses documented or referenced in IRIS. EPA formally stated its
position in the Policy for Use of Probabilistic Analysis in Risk Assessment (1997):
For human health risk assessments, the application of Monte Carlo and other probabilistic
techniques has been limited to exposure assessments in the majority of cases. The current
policy, Conditions for Acceptance and associated guiding principles are not intended to
apply to dose response evaluations for human health risk assessment until this application
of probabilistic analysis has been studied further.
The information provided in IRIS concerning variability and uncertainty has gradually
become more extensive as experience was gained in the IRIS program. At first, IRIS assessments
mainly provided the results of the deliberations culminating in consensus health hazard
conclusions. Gradually the assessments included more of the details of the data and of the
considerations which led to the consensus conclusions.
Note that some general aspects of the extent of variability and uncertainty can be
determined from the IRIS assessments. Concerning variability, upper bounds on response rates,
when provided in the IRIS Summary. A few examples for RfDs and RfCs generated from
NOAELS are provided in Table 1. For instance, for an RfC or RfD determined from a study with
6 animals per group, from which a NOAEL was identified, the 95% upper bound on the observed
0% response rate is approximately 50%. While this does not provide a confidence limit for the
corresponding exposure level, it demonstrates that there is considerable variability (and
uncertainty) in such a NOAEL. Also, the definition of RfDs and RfCs, accessible through the
IRIS Web Site, points out that there is perhaps an order of magnitude of uncertainty associated
with these determinations.
-1-
-------
The goal of this study is to examine more closely the strengths and weaknesses of the
documentation of the health hazard assessment conclusions available through IRIS, by examining
specific, representative IRIS assessments. On the advice of the Executive Committee of the
Science Advisory Board (11/29/99), the extent of documentation of variability and uncertainty in
IRIS assessments was established in two steps. The first step was to classify a random sample of
IRIS assessments into categories of documentation: none or minimal, some or moderate, or
extensive. Then assessments randomly drawn from these categories or strata are to be examined
in depth for their treatment of variability and uncertainty, given the state of the science and data
available at the time of the assessment.
Stratified random sampling is an efficient method for characterizing a population.
Through selection of strata which are reasonably internally homogeneous, we can expect that a
small number of assessments randomly chosen from a stratum can represent that stratum. Due to
the large number of pre-Pilot assessment, however, it is not possible to stratify the entire IRIS
database. A smaller subset must be used to characterize the overall extent of documentation of
uncertainty and variability in IRIS. The SAB recommended a 10% sample (52/522), which
NCEA also believes can adequately characterize the entire set of pre-Pilot IRIS assessments for
this evaluation. Thorough attention has been given during the entire IRIS program to generating
consistent summaries of adverse health effects associated with the chemicals considered.
Methods
The goal of the screening was to classify a random sample of assessments according to
the extent of the presentation and discussion of variability and uncertainty.
Selection of Screening Sample The total number of available IRIS assessments was fixed
by focusing on the chemicals listed on the IRIS Website. All toxicity values that were addressed
for each chemical - RfD, RfC, or cancer slope factors - were considered together. There were
522 pre-Pilot IRIS assessments available on-line as of 1/31/2000. These were numbered 1
through 522, in the order that the chemicals appear on-line, alphabetically. A table of random
numbers (Daniel, 1978) was used to select 52 numbers between 1 and 522, inclusive. Computer-
generated random numbers would have been appropriate, but this traditional method is more
straightforward to document. All post-Pilot assessments were stratified, since there were
relatively few post-Pilot assessments, and pre-Pilot and post-Pilot assessments are to be
compared.
Criteria for Assigning Assessments to Strata Since the available assessments were
generated over a period of approximately 14 years, it became clear that there was a continuum of
relevant factors to consider. NCEA developed criteria (see Table 2) to describe the degree of the
documentation and distinguish between qualitative and quantitative aspects of variability and
uncertainty. The first category, None/Minimal, describes assessments which presented results and
overall uncertainty and confidence conclusions, but no incidence rates or other quantitative effect
-2-
-------
levels for the available studies, nor rationale for the conclusions. Assessments with Some or
Moderate documentation contained quantitative effect levels and some discussion of variability of
effects, including variability across dose groups and temporal variability. In addition, there was
some discussion of the reasons for overall confidence in the assessment. Assessments with
Extensive documentation contained quantitative variability information, some comparison of
results across related studies, discussion of sources of uncertainty, comparison of uncertainties
across available studies, and rationales for confidence in the available studies and conclusions
drawn in the assessment.
In some cases, assessments contained somewhat more documentation of uncertainties
relative to variability, or vice versa. This was apparent within some sections addressing a health
hazard measure (RfC, RfD, or cancer unit risk), and between measures, especially when they were
completed a few years apart. The overall rating for an assessment was determined by the
characterization of the majority of the subsections. Appendix A provides a brief description of the
rationale for classifying each assessment.
For the purposes of this study, the determination of the extent of the presentation and
discussion of variability and uncertainty was restricted to what was explicitly provided in the on-
line IRIS database for each assessment. Specifically, for the pre-Pilot assessments, only the IRIS
Summary was examined. For the later IRIS assessments, the IRIS Summary and the
Toxicological Review were examined. EPA source documents and literature cited in the
assessments could not be consulted, due to the large volume of materials. Consequently, this
stratification addressed only the overall quality or approach to providing this information in the
on-line assessments, not the completeness of the summarized information nor the cited scientific
literature available at the time of each assessment.
Independent Review of Assignments to Documentation Categories The Executive
Committee of the SAB recommended that the assignment of the assessments in the screening
sample to the broad documentation categories go through an independent verification, to evaluate
the repeatability of the decision process. An EPA health scientist not routinely involved in IRIS
assessments applied the criteria developed above to the pre-screening sample of 67 assessments.
The report of this independent review is in Attachment A.
Results and Discussion
The results of applying the criteria in Table 2 to the pre-Pilot and the later IRIS
assessments are given in Tables 3 and 4, respectively. Of the 52 pre-Pilot IRIS assessments
screened (Table 3), 3/52 had extensive, 16/52 some or moderate, and 33/52 none or minimal
presentation or discussion of variability and uncertainty. In sharp contrast, assessments carried
out during or after the Pilot nearly unanimously (14/15) showed extensive treatment of variability
and uncertainty (see Table 4).
-3-
-------
As noted earlier, this evaluation was carried out to facilitate choosing assessments for in-
depth reviews. These in-depth evaluations were to cover a range of IRIS entry dates, types of
chemicals, scientific complexities, and toxic endpoints of concern. To that end, the sampled
assessments are listed in Tables 3 and 4 in chronological order within each of the three categories
of documentation, with broad categories of chemicals and critical health endpoints noted.
Note that a few pre-Pilot assessments referred to data which were not summarized in the
assessment, or otherwise included conclusions which were not supported by the available
summary. In these cases, assessments which otherwise fit the Extensive or Some/Moderate
criteria, but clearly omitted available data, were downgraded one category. These choices seemed
justified, since it seemed likely that if these assessments happened to be selected for the in-depth
phase of the evaluation, these deficiencies would be immediately apparent and remarked on.
These instances are noted in Appendix A.
Recall that 8 Pilot and 8 Pilot/post-Pilot assessments were to be randomly selected from
those screened, with 4 each from the Some/Moderate and Extensive categories within each of the
pre-Pilot and post-Pilot sets. As noted above, however, there were fewer than 4 assessments in 2
of the targeted subgroups. Specifically, only three assessments fell in the Extensive subset of the
pre-Pilot assessments sampled, so an additional Some/Moderate assessment must be evaluated
among the pre-Pilot assessments, resulting in a total of 5 Some/Moderate and 3 Extensive pre-
Pilot assessments. Similarly, among the post-Pilot assessments, only one assessment fell in the
Some/Moderate category, so 7 Extensive assessments must be evaluated to complete the in-depth
sample of post-Pilot assessments.
Due to time constraints, the independent verification of the classifications was carried out
after the in-depth assessments needed to be chosen. Overall, agreement was good (see Table 5),
with a Spearman rank correlation coefficient of 0.82 (Lehmann, 1975). Differences in
assignments for individual assessments were primarily due to the second reviewer classifying 11
'borderline' assessments in the next lower category than the first reviewer had. Also, there were
4 assessments which the first reviewer downgraded for omitting information referenced elsewhere
in the assessment; by the standards of the time the assessments were incomplete. The second
reviewer did not downgrade these assessments, emphasizing the quality of the approach to
presenting variability and uncertainty.
The second reviewer reached equally valid conclusions. Recognizing the subjectiveness
involved in drawing clear distinctions among characterizations which must consider a number of
heterogeneous issues, it is therefore constructive to consider the results of the two rankings
simultaneously. Among pre-Pilot assessments, approximately three-fourths (63-79%, from Table
5) contained none to minimal documentation of variability and uncertainty information. Note that
the vast majority of these assessments were completed before 1990 (see Table 3). Assessments
containing some to moderate documentation represented about 15-31% (Table 5) of the sample.
These were completed uniformly throughout the pre-Pilot period, at least among the assessments
in the screening sample.
-4-
-------
The distribution of assessments with extensive documentation of variability and
uncertainty clearly increased with time. Virtually all of the Pilot/post-Pilot (starting in 1995)
assessments demonstrated extensive treatment of variability and uncertainty information (93-
100%, Table 5). The earliest "Extensive" assessment in the screening sample was either 1,2-
dibromo-3-chloropropane (1991), according to the first reviewer, or manganese or 2,4-/2,6-
toluene diisocyanate mixture (1995), which both reviewers agreed upon.
References
Daniel, Wayne W. (1978). Biostatistics: A Foundation for Analysis in the Health Sciences.
John Wiley & Sons: New York.
Lehmann, E. L. (1975). Nonparametrics: Statistical Methods Based on Ranks. San Francisco:
Holden-Day, Inc.
National Research Council (1994). Science and Judgment in Risk Assessment. Washington,
D.C.: National Academy Press
U.S. EPA. (1986) Guidelines for carcinogen risk assessment. Federal Register 51(185):33992-
34003.
U.S. Environmental Protection Agency. (1996) Proposed guidelines for carcinogen risk
assessment, Notice, 1996. Federal Register 61 (79): 17960-18011.
US Environmental Protection Agency (1997). Policy for Use of Probabilistic Analysis in Risk
Assessment. National Center for Environmental Assessment: Washington, DC.
-5-
-------
Table 1 : Upper 95% bounds on 0%
responses for selected toxicity study group
sizes
Animals/sex/group
4
6
20
50
Upper 95% bound on
observed 0% response
60%
46%
17%
7%
Table 2: Criteria for classifying extent of presentation and discussion of variability and
uncertainty
Category
None/
Minimal
Some/
Moderate
Extensive
Variability
Any studies relevant to the conclusions
are listed, only qualitative dose-
responses indicated; no discussion.
Conditions for Minimal met, plus
adverse effect levels provided for
principal study; some discussion.
Conditions for Some/ Moderate met,
plus measures of variability or
discussion of variability of the results.
Uncertainty
Uncertainty factors listed, and overall
confidence stated; no discussion.
Uncertainty factors listed; some
discussion of uncertainty and
confidence in the assessment.
Discussion of the strengths and
weaknesses of the available studies,
some assessment of the level of
confidence in the body of evidence.
-6-
-------
Table 3: Extent of documentation of variability and uncertainty in a 10% random sample of pre-Pilot IRIS Assessments.
Chemical
Dibromo-3-chloropropane, 1 ,2-
Toluene diisocyanate mixture, 2,4-/2,6-
Manganese
Fluorine (soluble fluoride)
Ethylene glycol
Hexachlorodibenzo-p-dioxin, mixture
DichlorodiphenyltrichJoroethane, p,p'-
Fomesafen
Furmecyclox
Prochloraz
Propargite
Methylphenol, 4-
Dimethylformamide, N,N-
Vinyl acetate
Hexachlorobenzene
Pentachlorophenol
Aldicarb sulfone
Danitol
Arsenic, inorganic
Chlorsulfuron
Metalaxyl
Phosmet
Pronamide
Dibromobenzene, 1 ,4-
Overall
Documentation
Extensive
Extensive
Extensive
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
Some/Moderate
None/Minimal
None/Minimal
N one/Minim al
None/Minimal
None/Minimal
Last
Significant
Revision
10/01/1991
09/01/1995
11/01/1995
01/31/1987
09/30/1 987
03/01/1988
08/02/1 988
08/22/1988
09/07/1988
01/01/1989
05/01/1990
09/01/1990
10/01/1990
10/01/1990
03/01/1991
03/01/1991
11/01/1993
10/01/1994
06/01/1995
01/31/1987
01/31/1987
01/31/1987
01/31/1987
03/31/1987
Chemical
Type
Pesticide
Other
Metal
Element
Other
Other
Pesticide
Pesticide
Other
Pesticide
Pesticide
Other
Other
Other
Pesticide
Other
Pesticide
Pesticide
Metal
Pesticide
Pesticide
Other
Pesticide
Other
Key Health Endpoints"
RfC: testicular effects
RfC: chronic lung-function decline
RiD: CNS effects
RfC : impaired neurobehavioral function
Cancer: D, no human data, animal data inadeq.
RfD: dental fluorosis
RfD: kidney toxicity
Cancer: B2, liver tumors
RfD: liver lesions
Cancer: B2, liver tumors, benign and malignant
Cancer: C, hepatoadenomas, -carcinomas
Cancer: B2, liver carcinomas, neoplastic nodules
RfD: Increased SAP and liver wt, liver histopath
Cancer: C, liver tumors
RfD: None (at any dose tested)
Cancer: C, skin papillomas
RfC: digestive disturbances
RfC: Nasal epithelial lesions
RfD: liver effects
Cancer: B2, liver, thyroid, kidney tumors
RfD: liver, kidney pathology
Cancer: B2, hepatic, adrenal tumors; hemang.
RfD: brain ChE inhibition
RfD: tremors
RfD: hyperpigmentation, keratosis
Cancer: A, lung cancer
RfD: decreased BW
RfD: increased SAP; increased liver-to-brain wt ratio
RfC : inadequate data
RfD: reduced. BW; liver cell vacuolation; ChE inhibition
RfD: None (at any dose tested)
RfD: relative liver wt; hepatic microsomal enzyme induction
-7-
-------
Table 3: Extent of documentation of variability and uncertainty in a 10% random sample of pre-Pilot IRIS Assessments.
Chemical
Pydrin
Sodium azide
Tetrachlorovinphos
Cyromazine
Diphenamid
Hexazinone
Methamidophos
Butylphthalyl butylglycolate
N-Nitroso-N-methylethylamine
Carboxin
Vanadium pentoxide
Hexachlorophene
Dieldrin
Dimethyl phthalate
Maneb
Trichloroethane, 1,1,1-
Bis(chloromethyl)ether
Benomyl
Cypermethrin
Pursuit
Chlorocyclopentadiene
Octabromodiphenyl ether
beta-Chloronaphthalene
Brominated dibenzofurans
Dibromodichloromethane
Apollo
Dinitrobenzene, o-
Triethylene glycol monobutyl ether
Overall
Documentation
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/Minimal
None/MinimaJ
None/Minimal
None/Minimal
None/Minimal
Last
Significant
Revision
03/31/1987
03/31/1987
03/31/1987
09/30/1987
09/30/1987
09/30/1987
09/30/1987
03/01/1988
03/01/1988
06/30/1988
06/30/1988
08/22/1988
09/07/1988
09/07/1988
09/07/1988
09/07/1988
09/26/1988
03/01/1989
03/01/1989
01/01/1990
03/01/1990
08/01/1990
11/01/1990
12/01/1990
03/01/1991
06/01/1991
09/01/1992
09/01/1994
Chemical
Type
Pesticide
Other
Pesticide
Pesticide
Other
Other
Pesticide
Other
Other
Other
Other
Other
Pesticide
Other
Pesticide
Other
Other
Pesticide
Pesticide
Pesticide
Other
Other
Other
Other
Other
Pesticide
Other
Other
Key Health Endpoints"
RfD: neurological dysfunction
RiD: decreased BW; hunched posture
RfD: reduced BW gain; increased liver and kidney wt; RBC ChE inhibition
RfD: hematologic effects
RfD: liver toxicity
RfD: decreased BW
RfD: ChE inhibition
RfD: None (at any dose tested)
Cancer: B2, hepatic tumors
RfD: reduced BW gain, organ wt changes, death
RfD: decreased hair cystine
RfD: swollen salivary glands; status spongiosis in brain and optic nerve
RfD: liver lesions
Cancer: B2, hepatocarcinoma
Cancer: D, no data found
RfD: increased thyroid wt
Cancer: D, no human data, animal data inadeq.
RfC: inadequate data
Cancer: A, respiratory tract tumors
RfD: decreased pup BW
RfD: GI tract disturbances
RfD: decreased cell vol, HGB, erythrocytes
Cancer: D, no data found
RfD: induction of hepatic enzymes; liver histopath
Cancer: D, no data found
RfD: dyspnea; abnormal appear.; enlarged liver
Cancer: D, no data found
Cancer: D, no data found
RfD: liver effects; organ wt changes
Cancer: C, thyroid gland follicular cell tumors
Cancer: D, no human or animal data found
RfC: inadequate data
-8-
-------
a Note that RfDs or RfCs followed by 'None' indicate that no adverse health effects were seen; RfDs or RfCs were determined from
the particular exposure levels used in the cited experiments or studies.
Descriptions of cancer endpoints include cancer classifications from 1986 Cancer Guidelines (US EPA, 1986).
-9-
-------
Table 4: Extent of documentation and variability in post-Pilot IRIS Assessments
Last
Overall Significant
Chemical Documentation Revision Chemical Type
Key Health Endpoints"
Cumene
Tributyltin oxide
Trinitrobenzene, 1,3,5-
Chlordane
Extensive
Extensive
Extensive
Extensive
08/01/1997 Petroleum constituent RfD: increased kidney wt
Methylene diphenyl diisocyanate Extensive
Bentazon Extensive
Methyl methacrylate Extensive
Beryllium and compounds Extensive
Chromium VI Extensive
Naphthalene Extensive
Barium and compounds Extensive
Acetonitrile Extensive
Ethylene glycol monobutyl ether Extensive
09/01/1997 Pesticide
10/01/1997
02/07/1998
02/07/1998
03/02/1998
03/02/1998
Other
Pesticide
Other
Pesticide
Other
04/03/1998 Metal
09/03/1998 Metal
09/17/1998 Pesticide
01/21/1999 Metal
03/03/1999
12/30/1999
Other
Other
RfC: increased kidney, adrenal weights
Cancer: D/CBD, (see description)
RfD: immunosuppression
RfC: inadequate data
Cancer: D/CBD, benign pituitary, parathyroid tumors
RfD: methemoglobinemia; spleen-erythroid cell hyperplasia
RfD: hepatic necrosis
RfC: hepatic effects
Cancer: B2/L, hepatocarcinomas
RfC: olfactory epithelium hyperplasia
Cancer: D/CBD, data inadequate
RfD: blood loss into GI tract
Cancer: E/NL, (see description)
RfD: None (at any dose tested)
RfC: olfactory epithelium degeneration
Cancer: E/NL,
RfD: small intestinal lesions
RfC: Beryllium sensitivity, progression to CBD
Cancer: Bl/L, lung cancer
RfD: none (at any dose tested)
RfC: nasal septum atrophy, lower respiratory effects
Cancer: A/K, lung cancer
RfD: decreased BW
RfC: respiratory lesions
Cancer: C/CBD, respiratory tract tumors
RfD: none (at any dose tested)
RfC: unverifiable
Cancer: D/NL, (see description)
RfC: mortality
Cancer: D/CBD, (see description)
RfD: MCV changes
RfC: red blood cell count changes
Cancer: C/CBD, pheochromocytoma
-10-
-------
Table 4: Extent of documentation and variability in post-Pilot IRIS Assessments
Last
Overall Significant
Chemical Documentation Revision Chemical Type
Key Health Endpoints"
Benzene
Chromium III, insoluble salts
Extensive 01/19/2000 Solvent
Some/Moderate 09/03/1998 Metal
Cancer: A/K, leukemia
RfD: none (at any dose tested)
RfC: inadequate data
Cancer: D/CBD. (see description)
a Note that RfDs or RfCs followed by 'None' indicate that no adverse health effects were seen; RfDs or RfCs were determined from the particular exposure
levels used in the cited experiments or studies.
Descriptions of cancer endpoints include cancer classifications from 1986 Cancer Guidelines (USEPA, 1986), and abbreviations of the descriptors in the
proposed 1996 Cancer guidelines (USEPA, 1996):
CBD = Cannot be determined
NL = Not likely
L = Likely
K= Known
-11-
-------
Table 5: Correspondence between two independent assignments of IRIS assessments to categories of extent of documentation of
variability and uncertainty.
Pre-Pilot
Pilot/post-Pilot
Initial Classifications
Minimal
Moderate
Extensive
Totals
Initial Classifications
Minimal
Moderate
Extensive
Totals
Reviewer Classifications,
count (and %)
Minimal
31
10
-
41 (79%)
-
-
-
-
Moderate
2
5
1
8 (15%)
-
-
-
-
Extensive
-
1
2
3 (6%)
-
1
14
15 (100%)
Totals
33 (63%)
16 (31%)
3 (6%)
52
-
1 (7%)
14 (93%)
15
-12-
-------
Appendix A - Summary of classifications for screening sample of IRIS assessments
A=pre-Pilot
B=later
Chemical
Overall
Documentation
Comments
B
A
A
B
A
B
B
B
A
A
A
A
A
B
A
A
B
B
Acetonitrile
Aldicarb sulfone
Apollo
Arsenic, inorganic
Barium and compounds
Benomyl
Bentazon
Benzene
Beryllium and compounds
beta-Chloronaphthalene
Bis(chloromethyl)ether
Brominated dibenzofurans
Butylphthalyl butylglycolate
Carboxin
Chlordane
Chlorocyclopentadiene
Chlorsulfuron
Chromium III, insoluble salts
Chromium VI
Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Some/Moderate Relative effect levels reported, some discussion of effects across dose levels.
Some discussion of rationale for level of confidence reported.
None/Minimal Conclusions primarily ;some discussion of variability for cancer assessment
Some/Moderate Portions have extensive discussion of variability and uncertainty, but important
sections (human inhalation cancer studies) are missing; downgraded one
category.
Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
None/Minimal Conclusions only
Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
None/Minimal Limited discussion of consistency of results within principal study; some
discussion of consistency of results between studies.
None/Minimal Limited presentation of variability in results and discussion of uncertainty;
unclear uncertainty conclusions, downgraded one category
None/Minimal No chemical-specific data available; some discussion of relevance of structure-
activity information
None/Minimal Only one study available, with no adverse effects observed; no discussion of
medium confidence
None/Minimal Conclusions only
Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
None/Minimal Conclusions only
None/Minimal Conclusions only
Some/Moderate Incomplete reporting of subchronic studies cited, otherwise extensive
discussion of variability and uncertainty; downgraded one category
Extensive Detailed findings presented, relative significance of results discussed;
A-l
-------
Appendix A - Summary of classifications for screening sample of IRIS assessments
A=pre-Pilot
B=later
Chemical
Overall
Documentation
Comments
B
A
A
A
A
A
A
A
A
A
A
A
A
B
A
A
A
A
Cumene Extensive
Cypermethrin None/Minimal
Cyromazine None/Minimal
Danitol Some/Moderate
Dibromo-3-chloropropane, 1,2- Extensive
Dibromobenzene, 1,4- None/Minimal
Dibromodichloromethane None/Minimal
Dichlorodiphenyltrichloroethane, p,p'- Some/Moderate
Dieldrin None/Minimal
N,N-Dimethylformamide Some/Moderate
Dimethyl phthalate None/Minimal
o-Dinitrobenzene None/Minimal
Diphenamid None/Minimal
Ethylene glycol Some/Moderate
Ethylene glycol monobutyl ether Extensive
Fluorine (soluble fluoride) Some/Moderate
Fomesafen Some/Moderate
Furmecyclox Some/Moderate
Hexachlorobenzene Some/Moderate
rationales provided for uncertainty factors, confidence levels
Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Conclusions only
Conclusions; minimal discussion for not using several uncertainty factors
Extended discussion of variability for principal and supporting studies, but
incomplete; no discussion of uncertainty factors, some discussion of overall
confidence
Extensive discussion of adverse effects, including temporal variability; some
discussion of uncertainty factors and overall confidence
Conclusions mainly, with minimal discussion of confidence
No chemical-specific data available
Some discussion of variability among cancer studies; some discussion of
uncertainty and confidence
Mostly qualitative effect levels, except for SMRs; 13 slope factors with no
incidence data; minimal discussion of confidence or uncertainty
Thorough presentation of average effect magnitudes; some discussion of
confidence in data base
No human or animal studies available; some discussion of supporting data.
No human or animal studies available; some discussion of supporting data.
Conclusions only
Some discussion of variability between studies, and adverse effect magnitudes;
some discussion of confidence.
Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Some discussion of variability; assessment alludes to a 'large number of
studies' which were not summarized
Some discussion of variability among studies; some discussion of uncertainty
and confidence
Some discussion of variability among studies; some discussion of uncertainty
and confidence
Some discussion of variability across studies, confidence in assessment
A-2
-------
Appendix A - Summary of classifications for screening sample of IRIS assessments
A=pre-Pilot
B=later
Chemical
Overall
Documentation
Comments
A
A
A
A
A
A
A
B
B
A
B
A
A
A
A
A
A
A
A
A
A
A
B
Hexachlorodibenzo-p-dioxin, mixture Some/Moderate
Hexachlorophene None/Minimal
Hexazinone None/Minimal
Maneb None/Minimal
Manganese Extensive
Metalaxyl None/Minimal
Methamidophos None/Minimal
Methyl methacrylate Extensive
Methylene diphenyl diisocyanate Extensive
Methylphenol, 4- Some/Moderate
Naphthalene Extensive
N-Nitroso-N-methylethylamine None/Minimal
Octabromodiphenyl ether None/Minimal
Pentachlorophenol Some/Moderate
Phosmet None/Minimal
Prochloraz Some/Moderate
Pronamide None/Minimal
Propargite Some/Moderate
Pursuit None/Minimal
Pydrin None/Minimal
Sodium azide None/Minimal
Tetrachlorovinphos None/Minimal
Toluene diisocyanate mixture, 2,4-/2,6- Extensive
Tributyltin oxide Extensive
Trichloroethane, 1,1,1- None/Minimal
Some discussion of confidence in assessment
Conclusions only
Conclusions only
Conclusions only
Discussion of variability of results and of uncertainties and confidence in the
assessment
Conclusions only
Conclusions only
Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Incidence rates reported; some discussion of variability
Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Incidence rates reported; incomplete rationale for model defaults; minimal
discussion of uncertainty; downgraded one category
Conclusions only
Some discussion of variability
Conclusions only
Some discussion of variability
Conclusions only
Some discussion of variability
Conclusions only
Conclusions mainly; limited discussion of uncertainty
Conclusions only
Conclusions only
Discussion of variability and confidence in assessment
Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
Limited discussion of supporting data
A-3
-------
Appendix A - Summary of classifications for screening sample of IRIS assessments
A=pre-Pilot Overall
B=later Chemical Documentation Comments
A Triethylene glycol monobutyl ether None/Minimal No RFC, RfD or unit risk; some discussion of variability and uncertainty
B Trinitrobenzene, 1,3,5- Extensive Detailed findings presented, relative significance of results discussed;
rationales provided for uncertainty factors, confidence levels
A Vanadium pentoxide None/Minimal No quantitative response levels; no discussion of uncertainty
A Vinyl acetate Some/Moderate Some discussion of confidence in assessment
A-4
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14, 2000
Reviewer: Hugh Tilson
Variability
Uncertainty
Chemical
Acetonitrile
Overall Rating-Extensive based on variability and uncertainty narrative in the Toxicological Review
Aldicarb sulfone
Moderate
Not scored
Mimimal
Not scored
Not scored
RFD
RfC-Not listed
Cancer-Not listed Not scored
Overall Rating-Minimal
Apollo
RfD Minimal Minimal
RfC-Not listed Not Scored Not scored
Cancer Moderate Minimal
Overall Rating-Minimal
Arsenic, inorganic
RfD Extensive Moderate
RfC-Not listed Not Scored Not scored
Cancer Extensive Extensive
Overall Rating-Extensive
Barium and compounds
Overall Rating-Extensive for both variability and uncertainty based on Toxicological Review
Benomyl
RfD Minimal Minimal
RfC-Not listed Not Scored Not Scored
Cancer-Not listed Not Scored Not Scored
Overall Rating-Minimal
Bentazon
Overall Rating-Extensive for both variability and uncertainty based on Toxicological Review
Benzene
Overall Rating-Extensive for both variability and uncertainty based on Support Documents on IRIS
Beryllium and compounds
Overall Rating-Extensive for both variability and uncertainty based on Toxicological Review
beta-Chloronaphthalene
RfD Minimal Minimal
RfC-Not listed Not Scored Not Scored
Cancer-Not listed Not Scored Not Scored
Overall Rating-Minimal
Bis(chloromethyl)ether
RfD-Not listed Not scored Not scored
RfC-Not listed Not scored Not scored
Cancer Moderate Minimal
Overall Rating-Moderate
Butylphthalyl butylglycolate
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
B-l
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14, 2000
Reviewer: Hugh Tilson
Variability
Minimal
Not scored
Not scored
Not scored
Not scored
Minimal
Minimal
Not scored
Not scored
Chemical
Carboxin
RfD
RfC-Not listed
Cancer-Not listed
Overall Rating-Minimal
Chlordane
Overall Rating-Extensive for both variability and
Chlorocyclopentadiene
RfD-Not listed
RfC-Not listed
Cancer
Overall Rating-Minimal
Chlorsulfuron
RfD
RfC-Not listed
Cancer-Not listed
Overall Rating-Minimal
Chromium III, insoluble salts
Overall Rating-Extensive for both variability and
Chromium VI
Overall Rating-Extensive for both variability and
Cumene
Overall Rating-Extensive for both variability and
Cypermethrin
RfD
RfC-Not listed
Cancer-Not listed
Overall Rating-Minimal
Cyromazine
RfD
RfC-Not listed
Cancer-Not listed
Overall Rating-Minimal
Danitol
RfD
RfC-Not listed
Cancer-Not listed
Overall Rating-Moderate
Dibromo-3-chloropropane, 1,2-
RfD-Not listed
RfC
Cancer-Not listed
Overall Rating-Moderate
Dibromobenzene, 1,4-
RfD
Uncertainty
Minimal
Not scored
Not scored
Minimal
Not scored
Not scored
Moderate
Not scored
Not scored
Not scored
Moderate
Not scored
Minimal
Minimal
Not scored
Not scored
uncertainty based on Toxicological Review
Not scored
Not scored
Minimal
Minimal
Not scored
Not scored
uncertainty based on Toxicological Review
uncertainty based on Toxicological Review
uncertainty based on Toxicological Review
Minimal
Not scored
Not scored
Minimal
Not scored
Not scored
Minimal
Not scored
Not scored
Not scored
Minimal
Not scored
Minimal
B-2
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14, 2000
Reviewer: Hugh Tilson
Chemical
RfC-Not listed
Cancer-Mot listed
Overall Rating-Minimal
Dibromodichloromethane
RfD-Not listed
RfC-Not listed
Cancer
Variability
Not scored
Not scored
Uncertainty
Not scored
Not scored
Not scored Not scored
Not scored Not scored
Minimal Minimal
Overall Rating-Minimal (on the basis that a classification was made without presenting data)
Dichlorodiphenyltrichloroethane, p,p'-
Minimal
Not scored
RfD
RfC-Not listed
Cancer Minimal
Overall Rating-Minimal
Dieldrin
RfD Minimal
RfC-Not listed Not scored
Cancer Minimal
Overall Rating-Minimal
Dimethyl phthalate
RfD-Not listed Not scored
RfC-Not listed Not scored
Cancer Minimal
Overall Rating-Minimal
Dimethylformamide, N,N-
RfD-Not listed Not scored
RFC Moderate
Cancer-Not listed Not scored
Overall Rating-Moderate
Dinitrobenzene, o-
RfD-Not listed
RfC-Not listed
Cancer
Minimal
Not scored
Minimal
Minimal
Not scored
Minimal
Not scored
Not scored
Minimal
Not scored
Minimal
Not scored
Not scored Not scored
Not scored Not scored
Minimal Minimal
Overall Rating-Minimal (classified as non carcinogenic, supporting data are largely mechanistic and
not described in detail)
Diphenamid
Minimal Minimal
Not scored Not scored
Not scored Not scored
RfD
RfC-Not listed
Cancer-Not listed
Overall Rating-Minimal
Ethylene glycol
RfD Minimal
RfC-Not listed Not scored
Cancer-Not listed Not scored
Overall Rating-Minimal
Ethylene glycol monobutyl ether
Minimal
Not scored
Not scored
B-3
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14, 2000
Reviewer. Hugh Tilson
Chemical
Variability
Uncertainty
Overall Rating-Extensive for both variability and uncertainty based on Toxicological Review
Fluorine (soluble fluoride)
RfD Minimal
RfC-Not listed Not scored
Cancer-Not listed Not scored
Overall Rating-Minimal
Fomesafen
RfD-Not listed Not scored
RfC-Not listed Not scored
Cancer Minimal
Overall Rating-Minimal
Furmecyclox
RfD-Not listed Not scored
RfC-Not listed Not scored
Cancer Minimal
Overall Rating-Moderate
Hexachlorobenzene
RfD Minimal
RfC-Not listed Not scored
Cancer Minimal
Overall Rating-Minimal
Hexachlorodibenzo-p-dioxin, mixture
RfD-Not listed Not scored
RfC-Not listed Not scored
Cancer Minimal
Overall Rating-Minimal
Hexachlorophene
RfD Minima]
RfC-Not listed Not scored
Cancer-Not listed Not scored
Overall Rating-Minimal
Hexazinone
RfD Minimal
RfC-Not listed Not scored
Cancer-Not listed ' Not scored
Overall Rating-Minimal
Maneb
RfD Minimal
RfC-Not listed Not scored
Cancer-Not liested Not scored
Overall Rating-Minimal
Manganese
RfD Moderate
RfC Extensive
Cancer Not scored (moderate narrative)
Minimal
Not scored
Not scored
Not scored
Not scored
Minimal
Not scored
Not scored
Minimal
Minimal
Not scored
Minimal
Not scored
Not scored
Minimal
Minimal
Not scored
Not scored
Minimal
Not scored
Not scored
Minimal
Not scored
Not scored
Extensive
Extensive
Not scored (not carcinogenic)
B-4
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14,2000
Reviewer: Hugh Tilson
Chemical Variability Uncertainty
Overall Rating-Extensive
Metalaxyl
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Methamidophos
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Methyl methacrylate
Overall Rating- Extensive for both variability and uncertainty based on Toxicological Review
Methylene diphenyl diisocyanate
Overall Rating- Extensive for both variability and uncertainty based on Toxicological Review
Methylphenol, 4-
RfD-Withdrawn Not scored Not scored
RfC-Not listed Not scored Not scored
Cancer Moderate Minimal
Overall Rating- Moderate
Naphthalene
Overall Rating-Extensive based on both variability and uncertainty in Toxicological Review
N-Nitroso-N-methylethylamine
RfD-Not listed Not scored Not scored
RfC-Not listed Not scored Not scored
Cancer Moderate Minimal
Overall Rating- Moderate
Octabromodiphenyl ether
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not care. Not scored Not scored
Overall Rating-Minimal
Pentachlorophenol
RfD Minimal Minima]
RfC-Not listed Not scored Not scored
Cancer Moderate Minimal
Overall Rating-Minimal
Phosmet
RfD Minima] Minimal
RfC Not scored Not scored
Cancer Not scored Not scored
Overall Rating-Minimal
Prochloraz
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
B-5
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14, 2000
Reviewer: Hugh Tilson
Chemical
Variability
Moderate
Uncertainty
Minimal
Cancer
Overall Rating-Minimal
Pronamide
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Propargite
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer Not scored Not scored
Overall Rating-Minimal
Pursuit
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Pydrin
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Sodium azide
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Tetrachlorovinphos
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Toluene diisocyanate mixture, 2,4-/2,6-
RfD-Not listed Not scored Not scored
RfC- Extensive Moderate
Cancer-Not listed Not scored Not scored
Overall Rating-Extensive
Tributyltin oxide
Overall Rating-Extensive based on variability and uncertainty in Toxicological Review
Trichloroethane, 1,1,1-
RfD-Not listed Not scored Not scored
RfC-Not listed Not scored Not scored
Cancer- Minimal Minimal
Overall Rating-Minimal
Triethylene glycol monobutyl ether
B-6
-------
Appendix B - Independent Review of Assignments to Documentation Categories
July 14, 2000
Reviewer: Hugh Tilson
Chemical
Variability
Not scored
Minimal
Not scored
Uncertainty
Not scored
Minimal
Not scored
RfD-Not listed
RfC-Not verifiable
Cancer-Not listed
Overall Rating-Minimal
Trinitrobenzene, 1,3,5-
Overall Rating-Extensive on the basis of narrative for variability and uncertainty in the Support Document
Vanadium pentoxide
RfD Minimal Minimal
RfC-Not listed Not scored Not scored
Cancer-Not listed Not scored Not scored
Overall Rating-Minimal
Vinyl acetate
RfD-Not listed Not scored Not scored
RfC Moderate Minimal
Cancer-Not listed Not scored Not scored
Overall Rating-Moderate
Criteria for Classification:
Variability
Minimal No incidence generally
provided for RfD/RfC
Moderate Incidence, magnitude, onset
or duration mentioned with
some discussion, little discusion
about varability within or across
studies
Extensive Incidence, magnitude, onset
and duration mentioned
repeatedly; considerable discus-
sion about sources of variability
across and within studies
Uncertainty
Relatively terse description of what the
uncertainty factors were with little or
no discussion about their rationale;
no discussion about strengths or weaknesses
Rationale developed for the selection
of the uncertainty factors, strengths
or weaknesses mentioned
Considerable rationale for confidence
in studies and support provided for
conclusions, strengths or weaknesses
discussed
In some cases, assessments contained somewhat more documentation of uncertainties relative to
variability, and vice versa. If only one area qualified as Extensive, then the entire assessment was categorized as
Extensive. At the other end of the scale, if one part of an assessment contained Moderate documentation, the entire
assessment was categorized as Moderate. Several assessments were based on Supporting Documents or
Toxicological Reviews located on IRIS. All of these were categorized as Extensive.
B-7
------- |