Risk Evaluation for Methylene Chloride Systematic Review Supplemental File: Data Quality Evaluation of Human Health Hazard Studies Epidemiological Studies CASRN: 75-09-2


PEER REVIEW DRAFT, DO NOT CITE OR QUOTE
v/EPA
United States
Environmental Protection Agency
Office of Chemical Safety and
Pollution Prevention
Risk Evaluation for
Methylene Chloride
Systematic Review Supplemental File:
Data Quality Evaluation of Human Health Hazard Studies -
Epidemiological Studies
CASMN: 75-09-2
H
October, 2019, DRAFT

-------
Table Listing
1	Lash et al. 1991: Evaluation of Neurological/Behavior Outcomes		3
2	Of f et al. 1983: Evaluation of Mortality Outcomes		6
3	Cherry et al. 1983: Evaluation of Neurological/Behavior Outcomes		9
4	Windham et al. 2006: Evaluation of Neurological/Behavior Outcomes		14
5	Siemiatycki 1991: Evaluation of Cancer Outcomes		18
6	Cantor et al. 1995: Evaluation of Cancer Outcomes		21
7	Heineman et al. 1994: Evaluation of Cancer Outcomes		24
8	Seidler et al. 2007: Evaluation of Cancer Outcomes		28
9	Dosemeci et al. 1999: Evaluation of Cancer Outcomes		31
10	Wang et al. 2009: Evaluation of Cancer Outcomes		34
11	Infante-Rivard 2005: Evaluation of Cancer Outcomes		38
12	Miligi et al. 2006: Evaluation of Cancer Outcomes		42
13	Costantini et al. 2008: Evaluation of Cancer Outcomes		45
14	Radican et al. 2008: Evaluation of Cancer Outcomes 		49
15	Radican et al. 2008: Evaluation of Respiratory Outcomes 		52
16	Gold et al. 2010: Evaluation of Cancer Outcomes		55
17	Cocco et al. 1999: Evaluation of Cancer Outcomes		58
18	Barry et al. 2011: Evaluation of Cancer Outcomes		62
19	Bell et al. 1991: Evaluation of Growth (early life) and Development Outcomes . .	65
20	Hearne and Pifer 1999: Evaluation of Cancer for Employees in Roll Coating
Division Outcomes 		68
21	Hearne and Pifer 1999: Evaluation of Cancer for All Employees Outcomes ....	72
22	Hearne and Pifer 1999: Evaluation of Respiratory Outcomes 		76
	
23	Hearne and Pifer 1999: Evaluation of Hematological and Immune Outcomes ...	80
24	Gibbs et al. 1996: Evaluat ion of Cancer Outcomes		84
25	Lanes et al. 1990: Evaluation of Mortality Outcomes 		87
26	Lanes et al. 1990: Evaluation of Res piratory Outcomes		90
27	Lanes et al. 1990: Evaluation of Cancer Outcomes		93
28	Lanes et al. 1990: Evaluation of Cardiovascular Outcomes		96
29	Lanes et al. 1993: Evaluation of Respiratory Outcomes		99
30	Lanes et al. 1993: Evaluation of Cardiovascular Outcomes	102
31	Lanes et al. 1993: Evaluation of Cancer Outcomes	105
32	Lanes et al. 1993: Evaluation of Mortality Outcomes 	108
33	Taskinen et al. 1986: Evaluation of Reproductive Outcomes	Ill
34	Soden 1993: Evaluation of Cardiovascular Outcomes	116
35	Soden 1993: Evaluation of Neurological/Behavior Outcomes	120
36	Soden 1993: Evaluation of Hepatic Outcomes	124
37	Soden 1993: Evaluation of Hematological and Immune Outcomes 	128
38	Kalkbrenner et al. 2010: Evaluation of Neurological/Behavior Outcomes 	132
39	Tomeson 2011: Evaluation of Cancer Outcomes	136
40	Tomeson 2011: Evaluation of Cardiovascular Outcomes	141
41	Roberts et al. 2013: Evaluation of Neurological/Behavior Outcomes 	146
42	Christensen et al. 2013: Evaluation of Cancer Outcomes	149
43	Neta et al. 2012: Evaluation of Cancer Outcomes	152
44	Ruder et al. 2013: Evaluation of Cancer Outcomes	155
45	Vizcaya et al. 2013: Evaluation of Cancer Outcomes	158
46	Morales-Suarez-Varela et al. 2013: Evaluation of Cancer Outcomes	161
47	von Ehrenstein et al. 2014: Evaluation of Neurological/Behavior Outcomes ....	164
48	Talibov et al. 2014: Evaluation of Cancer Outcomes	167
49	Mattei et al. 2014: Evaluation of Cancer Outcomes 	172
50	Brender et al. 2014: Evaluation of Cardiovascular Outcomes 	175
51 Brender et al. 2014: Evaluation of Growth (early life) and Development Outcomes 178
1

-------
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
181
184
187
190
193
196
200
203
207
210
212
215
218
221
226
231
234
237
241
Brender et al. 2014: Evaluation of Neurological/Behavior Outcomes . . .
Silver et al. 2014: Evaluation of Cancer Outcomes	
Silver et al. 2014: Evaluation of Neurological/Behavior Outcomes	
Silver et al. 2014: Evaluation of Hepatic Outcomes	
Cliaigne et al 2015: Evaluation of Hematological and Immune Outcomes .
Talbott et al 2015: Evaluation of Neurological/Behavior Outcomes ....
Garcia et al. 2015: Evaluation of Cancer Outcomes 	
Kumagi et al. 2016: Evaluation of Cancer Outcomes	
Carton et al. 2017: Evaluation of Cancer Outcomes	
Purdue et al. 2016: Evaluation of Cancer Outcomes	
Celanese Fibers, Inc 1987: Evaluation of Hepatic Outcomes	
General Electric, Co 1990: Evaluation of Hepatic Outcomes	
General Electric, Co 1990: Evaluation of Neurological/Behavior Outcomes
Gibbs 1992: Evaluation of Cancer Outcomes	
Gibbs 1992: Evaluation of Respiratory Outcomes	
Dow Cliem, Co 1976: Evaluation of Skin and Connective Tissue Outcomes
Dow Cliem, Co 1972: Evaluation of Skin and Connective Tissue Outcomes
Ott et al. 1983: Evaluation of Hematological and Immune Outcomes . . .
Ott et al. 1983: Evaluation of Hepatic Outcomes	
2

-------
Table 1: Lash et al. 1991: Evaluation of Neurological/Behavior Outcomes
Study Citation: Lash, AA; Becker, CE; So, Y; Shore, M (1991). Neurotoxic effects of methylene chloride: Are they long lasting in humans? Occupational
and Environmental Medicine, 48(6), 418-426
Data Type: methylene chloride retired workers delayed verbal memory exposed-Neurological/Behavior
HERO ID: 13509
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
um x 0.4

Metric 2: Attrition

Low
x 0.4
Metric 3: Comparison Group
Medium x 0.2
0.8 Participants were retired airline mechanics who had
worked for the same, single airline and who were
members of the same labor union. Both the air-
line and the union provided information about the
study population and historical occupational methy-
lene chloride exposures. Retirees had to have worked
a minimum of 6 years in one or more of 14 target jobs
in order to be eligible. Medical and demographic
criteria for participants were we 11-documented in
the study report. Follow-ups with survey non-
res p o ndents/non-parti.ci.pants revealed that a higher
percentage of them had been diagnosed with heart
disease and/or gout compared to survey respon-
dents/participants, suggesting a bias toward lower
frequency of heart disease in the study population.
Additionally, the authors say that retirees that had
suffered strokes were excluded, but Table 3 shows
that 4 participants had had strokes.
1.2 Of the 91 potential study participants who met all
the medical and demographic criteria and were in-
vited to participate in the field study, only 46 ('25
solvent-exposed, 21 unexposed) participated. The
low participation rate is not explicitly explained, al-
though a logical assumption may be that these eli-
gible subjects elected not to participate.
0.4 The unexposed comparison group consisted of re-
tired airline mechanics who had worked in low-
or no-so 1 vent-exposure jobs (jet engine assembly
or routine aircraft maintenance). The unexposed
comparison group differed from the so 1 vent-exposed
group in some demographic criteria (e.g., ethnic mi-
nority, English-speaking), but models were not ad-
justed accordingly.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation: Lash, AA; Becker, CE; So, Y; Shore, M (1991). Neurotoxic effects of methylene chloride: Are they long lasting in humans? Occupational
and Environmental Medicine, 48(6), 418-426
Data Type: methylene chloride retired workers delayed verbal memory exposed-Neurological/Behavior
HERO ID: 13509
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
High x 0.4 0.4 Job-exposure matrices was determined using occu-
pational/historical exposure information from both
the airline and the labor union. Exposure was
confirmed by industrial hygiene assessments (per-
sonal and area air monitoring from 1975 through
1986), observation of current workplace practices,
and interviews with long-term employees. Addition-
ally, the study population consisted of retirees who
had worked for the same, single airline throughout
their careers, and thus their full work histories were
known.
X 0.2 0.6 The study examines two levels of exposure (solvent-
exposed and unexposed), based on occupational and
historical exposure information provided by the air-
line and the labor union.
High X 0.4 0.4 Study participants the solvent-exposed group
v/orked in these jobs for an average of 11.6 years
during the target years of 1970 to 1984, and for
an average of 23.8 years in all. For most, employ-
ment in these jobs was continuous. Participants
were assessed for neurological outcomes including
grip strength, motor speed, and memory.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
Medium
High
X 0.667 1.33 Participants were tested for a number of psychophys-
ical and psychological end points (grip strength, sen-
sory responses, motor speed, short-term visual mem-
ory, etc.) through seven test stations at the field site.
Tests were administered by specially trained examin-
ers (e.g., physicians, psychologists, nurses) who v/ere
blind to the participants' exposure group.
X 0.333 0.33 Means and standard deviations were reported for
each physiological and psychological test (along with
p values).
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
M<
High
X 0.5 1 The statistical analyses v/ere adjusted only for age.
X 0.25 0.25 Questionnaires, standardized tests, and interviews
by the research team and/or physicians were used
to determine participation eligibility and assess po-
tential confounders.
Continued on next page

-------
. continued from previous page
Study Citation: Lash, A A; Becker, CE; So, Y; Shore, M (1991). Neurotoxic effects of methylene chloride: Are they long lasting in humans? Occupational
and Envi r onmental Medicine, 48(6), 418-426
Data Type:	methylene chloride_retired workers_delayed verbal memory_exposed-Neurological/Behavior
HERO ID:	13509
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 11
Co-exposure Confounding
m
x 0.25
0.5
The issue of potential co-exposures was not ad-
dressed in the study, but there's also no evidence
that there were co-exposures that were improperly
adjusted for.
Domain 5: Analysis





Metric 12
Study Design and Methods
Medium
x 0.4
0.8
A small occupational cohort of airline mechanic re-
tirees with long-term methylene chloride exposure
was assessed for neurological outcomes. Data pre-
sented as means/standard deviations evaluated with
t-tests.
Metric 13
Statistical power
Medium
x 0.2
0.4
The study had limited sample size (25 exposed, 21
unexposed), but showed statistically significant re-
sults. Statistical power appears sufficient to detect
large effects.
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Results of neurological assessments were reported as
means/standard deviations. Analysis of effect esti-
mates is clearly described, and reproducible.
Metric 15
Statistical models
Medium
x 0.2
0.4
Continuous dependent variables analyzed using t-
tests. Composite scores for memory and attention
tests were standardized for the pooled group of sub-
jects.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.8

Extracted

Yes



* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score,; x MWF;) / J] . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 2: Of t et al. 1983: Evaluation of Mortality Outcomes
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride Scandinavian Journal of Work, Environment and Health, 9(Suppl 1 ,Suppl 1), 1-38
Data Type: DCM occupational retrospective cohort mortality-Mortality
HERO ID: 29149
Domain
Metric
Rating^
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2:
Metric 3:
Attrition
Comparison Group

Medium X 0.4 0.8 Participants were employees of a cellulose triacetate
and cellulose diacetate fiber manufacturing plant in
South Carolina who had worked in preparation or
extrusion areas for at least 3 months between 1954
and 1977. A total of 1271 employees from this plant
v/ere included in the mortality study. Control group
participants (948) were drawn from a non-DCM-
exposure reference acetate fiber manufactur i ng plant
in Virginia. Because work assignments at this plant
varied and day-to-day assignment records v/ere not
kept, employees who v/orked in comparable areas of
the plant (preparation or extrusion areas) could not
be identified.
Low X 0.4 1.2 Attrition was not reported/addressed in this report.
Medium X 0.2 0.4 Because of an absence of work records for employees
of the reference plant, it could not be ascertained
whether participants from this plant v/orked in sim-
ilar areas/operations as those of the participants
from the DCM-exposure plant. Additionally, details
on participants (e.g., race, sex, age, etc.) v/ere re-
portedly collected, but not reported in the study re-
port.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
High
Metric 5: Exposure levels
Medium
X 0.4 0.4 Eight-hr TWA concentrations and peak concentra-
tions were determined for both plants. Personal air
monitoring (>350 samples), area sampling (170 sam-
ples), and short-term excursion sampling (20 sam-
ples) were performed over the course of a 3.5-month
survey period in late 1977-early 1978. Details of the
personal air sampling methods are described in an
appendix to the study report.
X 0.2 0.4 Occupational DCM exposure v/as categorized into
three levels across a sufficient range.
Continued on next page

Metric 8: Reporting Bias

Low
X 0.667 1.33 Cause of death was determined from copies death
certificates of death certificates obtained through
company insurance records or state vital statistics
agencies. They were
coded by a nosologist according to the Revision of
the International Classification
of Diseases in force at the time of death. Mortality
within the exposed cohort was
compared with that of both the corresponding
United States population and the
reference population. Outcomes of a priori interest
were deaths due to ischemic heart
disease and malignant neoplasms.
X 0.333 1.0 Mortality information for participants is not re-
ported in this study report. Only median exposures
are reported.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
Unacceptable
X
0.5
0.25
There is no discussion of covariate adjustments.
Metric 10:
Covariate Characterization
Unacceptable
X
0.25
0.06
There is no discussion of covariate c h a r a c t e r i. z ati.on.
Metric 11:
Co-exposure Confounding
Low
X
0.25
0.75
The study report indicates that exposure to other
chemicals (e.g., methanol, acetone) was possible at
the South Carolina plant.
Domain 5: Analysis

Metric 12:
Study Design and Methods

X
0.667
0.44
Statistical analyses were not presented in this study
report, and therefore it is difficult to determine ac-
ceptability on the basis of study design.
Metric 13:
Statistical power
Medium
X
0.333
0.67
The study included 1,271 exposed employees and 948
unexposed employees, thus with a likely adequate
sample size.
Metric 14:
Reproducibility of analyses
Not Rated
NA
NA
Details of analyses are missing from this study re-

port.
Continued on next page ..

-------
. continued from previous page
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride Scandinavian Journal of Work, Environment and Health, 9(Suppl 1 ,Suppl 1), 1-38
Data Type:	DCM_occupational_retrospective cohort_mortality-Mortality
HERO ID:	29149
Domain
Metric

Rating^
MWF*
Score Comments^
Metric 15
Statistical models

Not Rated
NA
NA Details on statistical analyses were not presented in
this study report.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure


NA
NA
Metric 17
Effect biomarker


NA
NA
Metric 18
Method Sensitivity


NA
NA
Metric 19
Biomarker stability


NA
NA
Metric 20
Sample contamination


NA
NA
Metric 21
Method requirements


NA
NA
Metric 22
Matrix adjustment


NA
NA
Overall Quality Determination*

Unacceptable*'

2.8
Extracted


No


* Consistent with our Application of Systematic Review in TSCARisk Evaluations document, if a metric for a data source receives a score of Unacceptable (score = 4), EPA
will determine the study to be unacceptable. In this case, one or more of the metrics were rated as unacceptable. As such, the study is considered unacceptable and the score
is presented solely to increase transparency.
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T\ (Metric Score; x MWF,) / ]T\ MWF ;
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is crossed
out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type

-------
Table 3: Cherry et al. 1983: Evaluation of Neurological/Behavior Outcomes
Study Citation: Cherry, N; Venables, H; Waldron, HA (1983). The acute behavioural effects of solvent exposure Occupational Medicine, 33(1), 13-18
Data Type: Cohort Occupational DCM Behavior-Neurological/Behavior
HERO ID: 74582
Domain
Metric
Rating'
MWF*
Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
Medium
Unacceptable x 0.4 0.16
Low
x 0.4 0.8 Some key elements of the study design were not pre-
sented, but available information indicates a low risk
of selection bias. Factory C was the factory with
methylene chloride exposure. Some details provided
(type of shift work, age). There were three differ-
ent shifts and controls were selected from all three
shifts. However, participation rates and recruitment
methods were not reported.
Table II indicates loss of over half of the subjects,
with no explanations. Methods indicated that there
v/ere 56 exposed subjects and 36 control subjects
from factory C, but results in Table II indicate a
sample size of 44. It was also not indicated if the
44 were exposed subjects only or if they included
the control subjects. In addition, although they se-
lected subjects from all three shifts, there is no infor-
mation to indicate that those included in the results
were still from all three shifts.
X 0.2 0.6 No information about the similarity between groups
nor was there information to indicate that controls
were matched. Although it was noted that controls
were selected from each shift so that they v/orked the
same shift pattern as the exposed subjects. No other
information was provided including if the controls
were all men like the exposed workers. The mean age
of the exposed workers was stated to be 43.8 years
old, but no age was provided for the 36 controls.
In addition, only 12 of the controls were from the
areas of Factory C where there was no contact with
solvents. The other 24 were from another factory
belonging to the same parent group on a film making
process identical to the exposed me without solvent
exposure.
Domain 2: Exposure Characterization
Continued on next page

in carbon disulphide and solvent concentration was

analyzed using gas chromatography with a 2 m 8%
carbowax column. Blood samples were taken and
measured as well. There is no information provided

on QC methods or recovery rates for these methods.
Metric 5:
Exposure levels
Medium
x 0.2
0.4
The range of exposure reported was 28-173 ppm.
Blood solvent levels were not reported. Some results
were presented as only exposed vs. unexposed, but
combined the results for the different factories and
controls and were not specific for methylene chloride
exposure (i.e., Factory C)
Metric 6:
Temporality
Medium
x 0.4
0.8
Temporality is established, but it is unclear whether
exposures fall within relevant exposure windows for
the outcome of interest. Blood samples (used in the
analysis) were obtained at the beginning and end
of the shift. These appear to be the same times
that the outcome was tested. So although the sub-
jects likely worked around methylene chloride prior
to the outcomes, there is not enough information
provided on how long or when and measurements
were made at the same time as the outcome. How-
ever, the study authors appear to be looking at the
acute effects indicating that the timing may be ap-
propriate.
Domain 3: Outcome Assessii
cient

Metric 7:
Outcome measurement or characterization
Medium
x 0.667
1.33
Three tests were completed at the beginning and
the end of shift (i.e., visual analogue scales to re-
flect mood, the digit symbol substitution test from
the Wescsler Adult Intelligence Scale, and a test of
simple reaction time. Visual analogue scales are self-
reported rating scales that were noted to have been
shown to provide reliable and valid measure of mood.
Some details were provided on the other measures,
but it is not clear what the criteria being measured
were.
Continued on next page

-------
. continued from previous page
Study Citation: Cherry, N; Venables, H; Waldron, HA (1983). The acute behavioural effects of solvent exposure Occupational Medicine, 33(1), 13-18
Data Type: Cohort Occupational DCM Behavior-Neurological/Behavior
HERO ID: 74582
Domain
Metric
Rating^
MWF* Score
Commentstt
Metric 8: Reporting Bias
Low
X 0.333 1.0 Correlations were provided for methylene chloride
and 4 mood changes noted as part of the visual ana-
logue scales. No results v/ere provided for simple
reaction time in methylene chloride workers. Al-
though results were stated to be in Table III and may
have evaluated methylene chloride separate from the
styrene workers, there was no Table III in the re-
port nor is there a discussion of fi ndings for this test
in methylene chloride v/orkers. Digit symbol scores
v/ere just noted to show no difference.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
x 0.667

Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
Not Rated
Low
NA
x 0.333
NA
1

All the subjects were presumably male (not clear
that all the controls were male) and subjects in
both exposed and control group were selected from
all three shifts, but no other confounding variables
were discussed. Although subjects v/ere noted in the
methods to be selected from all three shifts, not all
subjects appear to have been included in the analy-
sis and it is not clear that this was still accounted for
in the results. Age v/as mentioned for the exposed
v/orkers, but v/as not mentioned for the control sub-
jects.
N/A because no covariates v/ere discussed.
Co-exposed to methanol (DCM:methanoI 9:1), but
the co-exposures v/ere not adjusted for. This co-
exposure would also likely bias results away from
the null, as it might contribute to effects seen. In
addition, controls v/ere exposed to other unspecified
compounds as part of the fi. 1 m making process that
could also have contributed to results in the control
and may bias the results tov/ards the null.
Domain 5: Analysis
Metric 12:
Study Design and Methods
Medium X 0.4 0.8 Study design was appropriate. The study was evalu-
ating acute neurobehaviorai effects and v/as designed
to test subjects before and after exposure. It also
contained controls that v/ere from the same plant
and unexposed, which would also help address if
the exposure had a chronic effect on the subjects
(thus lowering their initial score) and if the differ-
ences were just based on working 8 hours and not an
effect of exposure.
Continued on next page

-------
. continued from previous page
Study Citation: Cherry, N; Venables, H; Waldron, HA (1983). The acute behavioural effects of solvent exposure Occupational Medicine, 33(1), 13-18
Data Type: Cohort Occupational DCM Behavior-Neurological/Behavior
HERO ID: 74582
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 13:
Statistical power
Medium
x 0.2
0.4
The number of participants are adequate to detect
an effect in the exposed population and/or sub-
groups of the total population. The initial study
group was 56 exposed and 36 controls. The results
showed 44 subjects.
Metric 14:
Reproducibility of analyses
Low
x 0.2
0.6
It is apparent that the authors made several compar-

isons including correlations, but the specific meth-

ods used to determine significance or correlations
was not provided. Therefore, they could not be repli-
cated.
Metric 15:
Statistical models
Low
x 0.2
0.6
It is apparent that the authors made several compar-
isons including correlations, but the specific meth-
ods used to determine significance or correlations
was not provided. Therefore, they could not be repli-
cated.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:
Use of Biomarker of Exposure
Low
x 0.167
0.5
Methylene chloride was reported to be tested in
blood using a head space analyser and a 2 m 8%
carbowax column. Blood carboxvhaemoglobin con-
centration was also measured as noted by the study
authors carbon monoxide is a metabolite of methy-
lene chloride. There is no information provided for
QC or recovery rates. Nor is there any information
on how the metabolite information was included if at
all in the assessment of methylene chloride exposure.
This is probably not a very accurate method.
Metric 17:
Effect biomarker
Not Rated
NA
NA
No biomarker of effect was measured..
Metric 18:
Method Sensitivity
Low
x 0.167
0.5
LOD/LOQ values are not stated
Metric 19:
Biomarker stability
Low
x 0.167
0.5
There is no information on the storage or stability
of the samples nor was there information provided
on when the samples were tested in comparison to
when they were collected.
Metric 20:
Sample contamination
Low
x 0.167
0.5
There is no information about the collection and

storage of the sample in regards to contamination.
Metric 21:
Method requirements
Low
x 0.167
0.5
Sigma H6 head space analyser was used.
Metric 22:
Matrix adjustment
Low
x 0.167
0.5
There was no information about adjustments (or
not) for the matrix. There are no units provided
so it cannot be determined if there was any adjust-
ments or how the exposure was presented.
Overall Quality Determination
Unacceptable**

2.7

Extracted

Continued on next page

-------
. continued from previous page
Study Citation:	Cherry, N; Venables, H; Waldron, HA (1983). The acute behavioural effects of solvent exposure Occupational Medicine, 33(1), 13-18
Data Type:	Cohort_Occupational_DCM_Behavior-Neurological/Behavior
HERO ID:	74582
Domain	Metric	Rating^	MWF* Score	Comments^
** Consistent with our Application of Systematic Review in TSCARisk Evaluations document, if a metric for a data source receives a score of Unacceptable (score = 4), EPA
will determine the study to be unacceptable. In this case, one or more of the metrics were rated as unacceptable. As such, the study is considered unacceptable and the score
is presented solely to increase transparency
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
7], (Metric Scores x MWF'i) / . MWF_?-
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is crossed
out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 4: Windham et al. 2006: Evaluation of Neurological/Behavior Outcomes
Study Citation: Windham, GC; Zhang, L; Gunier, R; Croen, LA; Grether, JK (2006). Autism spectrum disorders in relation to distribution of hazardous
air pollutants in the San Francisco Bay area Environmental Health Perspectives, 114(9,9), 1438-1444
Data Type: California case control autism DCM OR_Q4-Neurological/Behavior
HERO ID: 103522
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
dium x 0.4

Metric 2: Attrition
Metric 3: Comparison Group
High
x 0.4
x 0.2
0.8 Cases were identified from the California Centers
for Autism and Developmental Disabilities Research
and Epidemiology (CADDRE) which draws informa-
tion on ASD by active surveillance of California De-
partment of Developmenta 1 Services (DDS) and the
Kaiser Permanente Medical Care Program. Authors
estimated that these methods captured 75-80% of
cases living in the area (Croen et al. 2002); authors
note that extreme ends of the socioeconomic status
were likely not well covered. Cases were included if
they were born in 1994 and resided in one of six San
Francisco Bay area counties. Controls were identi-
fied from a California 1994 linked birth-infant death
certificate database using the same inclusion crite-
ria. Controls were randomly selected and matched
on birth month and sex (2 to 1).
0.4 Of the cases identified in the databases, expert re-
view by the PI confirmed 83.3% ASD diagnoses, us-
ing the same criteria for all exc 1 usi.on/i.nc 1 usi.on by
expert review. Exclusion from the control popula-
tion was minimal (n= 18) and was sufficiently ex-
plained.
0.2 There is some evidence of differences between the
controls and cases; however, parental and child char-
acteristics such as race/ethnicity, maternal educa-
tion, and parity were considered as potential con-
founders in the statistical analysis. Demographic
details provided in Table 2.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4 0.8 Annual average concentration estimates were drawn
from EPA's National Air Toxics Assessment
(U.S.EPA; 4152303). Concentration estimates were
available by census tract for 1996 that matched the
geocoded addresses from birth certificates. Esti-
mates were calculated by summing concentrations
across various sources (mobile, point, and area
sources). This represents a well-established method
of determining exposure to HAPs and was assessed
consistently across groups.
Continued on next page

-------
. continued from previous page
Study Citation: Windham, GC; Zhang, L; Gunier, R; Croen, LA; Grether, JK (2006). Autism spectrum disorders in relation to distribution of hazardous
air pollutants in the San Francisco Bay area Environmental Health Perspectives, 114(9,9), 1438-1444
Data Type: California case control autism DCM OR_Q4-Neurological/Behavior
HERO ID: 103522
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 5: Exposure levels
Metric 6: Temporality
m x 0.2
x 0.4
0.4 For chemical specific analyses, quartil.es of exposure
were used. These were determined by exposure dis-
tribution quartiles in controls. This represents more
than two levels of exposure. Mean exposures were
0.64-0.68 ug/m3 (DCM), 0.60-0.61 ug/m3 (Perc),
and 0.17-0.19 ug/m3 (TCE).
1.2 Cases were diagnosed with Autism Spectrum Dis-
order by age 9 (sufficient window for diagnosis).
Cases and controls were drawn from a population
of children born in 1994; however, exposure was de-
termined from census tract-level exposure data for
birth address from 1996 exposure estimates (other
option was 1994). It is unclear how stable these es-
timates may be from year to year. Using exposure
data from 1996 may not accurately capture the ex-
posure that occurred during gestation, but instead
reflect an early childhood deve 1 opmental window.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization

x 0.667 0.67
Metric 8: Reporting Bias

um x 0.333 0.67
Cases were identified by CADDRE active surveil-
lance of California Department of Developmentai
Services and Kaiser Permanente records. Identified
cases were confirmed by the principal investigator by
diagnosis from a qualified medical professional, qual-
ification for special education under an autism ex-
ceptionality, or autistic behaviors appearing to meet
DSM-IV criteria for ASD. This represents a well-
established method of determining an autism diag-
nosis.
All outcomes outlined in the abstract, introduction,
and methods were provided in the results. The num-
ber of cases and controls was detailed for some anal-
yses, but not for chemi.ca 1 -speci.fi.c analyses which
would not al lowed for detailed extraction of the num-
ber of cases/controls. This is not expected to have
an appreciable impact on the results.
Domain 4: Potential Counfounding/Variable Control
Continued on next page

-------
.. . continued from previous page
Study Citation: Windham, GC; Zhang, L; Gunier, R; Croen, LA; Grether, JK (2006). Autism spectrum disorders in relation to distribution of hazardous
air pollutants in the San Francisco Bay area Environmental Health Perspectives, 114(9,9), 1438-1444
Data Type: California case control autism DCM OR_Q4-Neurological/Behavior
HERO ID: 103522
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 9:
Covariate Adjustment
High
X
0.5
0.5
Potential confounders included maternal age, race,
and education, parity, paternal race and age, low
birth weight, preterm delivery, and child race. The
final models include child race, maternal age, and
maternal education. Cases and controls were birth
month- and sex-matched. The authors stated they
did not include these two variables in the final model
as it made little difference.
Metric 10:
Covariate Characterization
High
X
0.25
0.25
For controls, demographic data were stated to be ab-
stracted from the birth certificate. Demographic in-
formation for cases was drawn from medical or DDS
records. These are both reliable methods of obtain-

Medium

ing covariate information.
Metric 11:
Co-exposure Confounding
X
0.25
0.5
Approximately 30 hazardous air pollutants (HAPs)
were considered in this study. The chlorinated sol-
vents (Perc, TCE, DCM, and vinyl chloride) tended
to be correlated with each other. TCE was noted

to be highly correlated to metals. Chemical-speci.fi.c

analyses did not control for exposure to other HAPs.
Although, there was no evidence of unbalanced co-
exposures by case status.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
A case-control study design was used to assess re-

lationships between exposure to HAPs during preg-
nancy/early childhood and the presence of ASD di-
agnosis at age 9.
Metric 13:
Statistical power
Medium
X
0.2
0.4
There were a sufficient number of cases and con-

trols to detect an effect.: 284 cases, 657 controls.
The study authors explicitly stated they kept birth
month- and sex-matched controls whose matched
cases did not meet the study's diagnostic criteria in
order to maintain a larger sample size.
Metric 14:
Reproducibility of analyses
m
X
0.2
0.4
The description of the analysis was sufficient. Cut-
points for quartiles of exposure and the procedure
for i n c 1 u sion/exc 1 usi.on of potential confounders was
described.
Metric 15:
Statistical models
Medium
X
0.2
0.4
Odds ratios were calculated for the two highest quar-
tiles of exposure using logistic regression. The mod-
els and decisions on categories of exposure were de-
scribed in detail in the methods.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16: Use of Biomarker of Exposure NA NA
Continued on next page

-------
. continued from previous page
Study Citation: Windham, GC; Zhang, L; Gunier, R; Croen, LA; Grether, JK (2006). Autism spectrum disorders in relation to distribution of hazardous
air pollutants in the San Francisco Bay area Environmental Health Perspectives, 114(9,9), 1438-1444
Data Type:	California	case	control	autism	DCM	OR_Q4-Neurological/Behavior
HERO ID:	103522
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination	Medium	1.7
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 5: Siemiatycki 1991: Evaluation of Cancer Outcomes
Study Citation: Siemiatycki, J (1991). Risk factors for cancer in the workplace ^journal#, ^volume# (tissue#), ^Pages#
Data Type:	DCM	worker any exposure	rectal cancer-Cancer
HERO ID:	157954
Domain
Metric

MWF* Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
High x
0.4
High x 0.2

0.4 Of 4576 eligible male cases from the Montreal
metropolitan area were ascertained between 1979-
1985, 3730 completed an interview during this study
(initiated in 1979 as a case-control design). Each
cancer was coded by the International Classification
of Disease for Oncology. Of 541 eligible popula-
tion male controls, 375 were interviewed and selected
from random digit calling, the provincial election of
1981, were noncancer patients hospitalized in the
same institutions as those with cancer - a subgroup
of control cancer cases unrelated to occupational ex-
posure or with cancer at another site deemed not
occupationally relevant was also interviewed
0.4 81.5% of eligible cases completed interviews. 72%
of controls. Nonresponses due to refusal, death, no
next of kin found, patient discharged, no valid ad-
dress, psychiatric cases, no translator, or physician
refusal
0.2 Baseline characteristics were collected from partic-
ipants and adjusted for; cases and controls v/ere
similar in that they were selected from Montreal,
Canada, between 35-70 years old, male and recruited
from 1979-1985.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Low
x 0.4
-_im x 0.2
Metric 6: Temporality
Low	x 0.4
1.2 Exposure determined by questionnaire, no occupa-
tional records. Chemi.st-hygi.eni.sts interview consul-
tants to better grasp the workings of particular in-
dustries, occupations were selected and coded as low
medium or high concentrations of exposure to a host
of chemicals based on job title
0.4 Any or substantial exposure was assigned to each
job title and patients were assigned to one of the
two categories for analysis. Assignments made by a
chemist-hygi.eni.st
1.2 Cases aged 35-70, time since first exposure not es-
timated; study was initiated in 1979 with exposures
occurring before or between 1945-1975.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Siemiatycki, J (1991). Risk factors for cancer in the workplace ^journal#, # volume# (tissue#), #Pages#
Data Type:	DCM	worker any exposure	rectal cancer-Cancer
HERO ID:	157954
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 7:
Outcome measurement or characterization
High
x 0.667
0.67
Histological or autopsy confirmation of primary tu-





mor site.
Metric 8:
Reporting Bias

x 0.333
0.33
ORs with 90% CIs.
Domain 4: Potential Counfounding/Variable Control




Metric 9:
Covariate Adjustment
High
x 0.5
0.5
For each association between occupational exposure





and cancer type adjustments were made included





age, height, place of birth, and race
Metric 10
Covariate Characterization
Medium
x 0.25
0.5
Confounders based on literature and questionnaire
Metric 11
Co-exposure Confounding
Medium
x 0.25
0.5
Adjustments for other occupational exposure types,





smoking, and alcohol intake were made.
Domain 5: Analysis





Metric 12
Study Design and Methods
Medium
x 0.4
0.8
This is a case-control study that collected cancer




type and lifetime occupational history from cancer





patients to determine if occupational history effected





cancer risk
Metric 13
Statistical power
Medium
x 0.2
0.4
DCM was not included in Table 1 results, which in-





cluded all associations where power was adequate to





detect a 2-fold risk (based on # participants and





at least 2% exposure). DCM was included in Ta-





ble 2 which shows elevated ORs only (irrespective of





pov/er to detect excess risk).
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Analysis was fully described a Man te 1- H ae ns ze 1 anal-





vsis was performed to analyze odds ratios for the
Metric 15
Statistical models
Medium
x 0.2
0.4
Method was transparent. A Mantel-Haenszel analy-





sis was performed to analyze odds ratios for the data.





p-values were computed by the Mante 1-Haensze 1 chi-





square test
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination
Medium

1.7

Extracted	Yes
Continued on next page

-------
. continued from previous page
Study Citation:	Siemiatycki, J (1991). Risk factors for cancer in the workplace #journal#, #volume#(#issue#)5 #Pages#
Data Type:	DCM_worker any exposure_rectal cancer-Cancer
HERO ID:	157954
Domain	Metric	Rating^ MWF* Score	Comments^
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =


]T\ (Metric Score; X MWF;)/]T. MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 6: Cantor et al. 1995: Evaluation of Cancer Outcomes
Study Citation: Cantor, KP; Stewart, PA; Brinton, LA; Dosemeci, M (1995). Occupational exposures and female breast cancer mortality in the United
States Journal of Occupational and Environmental Medicine, 37(3,3), 336-348
Data Type: DCM_breast cancer_occupational_case-control_OR_black2-Cancer
HERO ID: 194130
Domain
Metric
Rating*
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
0.4 Cases were women whose death certificates listed
breast cancer as the cause of death (across 24 U.S.
states). Controls were randomly selected from non-
cancer deaths, and frequency-matched for age, gen-
der, and race (four controls per case). Records were
from years 1984 to 1989, from a database supported
by the National Cancer Institute, NIOSH, and the
National Center for Health Statistics. Cases for
which 'homemaker' was the designated occupation
were excluded, leaving 29,397 white women cases,
102,955 white women controls, 4,112 black women
cases, and 14,839 black women controls.
0.4 Only cases for which 'homemaker' was the desig-
nated occupation were excluded (45.1% of white
women cases, 31.1% of black women cases; 51.7% of
white women controls, 37.9% of black women con-
trols).
0.2 Controls were recruited from records from the same
database and for the same time period as cases, and
were frequency-matched for age, gender, and race.
High x 0.4
High x 0.2
Me
Lium x 0.4
Metric 5: Exposure levels
Low x 0.2
0.8 Exposure was estimated using a job exposure ma-
trix, based on the assigned occupational codes, and
developed according to professional judgement of an
industrial hygi.eni.st, information in the general lit-
erature on occupational exposure, and NIOSH and
OSHA occupational exposure databases. Exposure
probability and level was estimated for 31 occupa-
tional exposure categories, of which DCM exposure
was one. Scores were assigned for probability and
level of exposure. There were no detailed employ-
ment records used.
0.6 Four levels of exposure are presented, including no
exposure. Detailed ranges for exposure are not in-
cluded in the present reference, but more details may
be available in HERO ID's 707912 and 1188.
Continued on next page

-------
. continued from previous page
Study Citation: Cantor, KP; Stewart, PA; Brinton, LA; Dosemeci, M (1995). Occupational exposures and female breast cancer mortality in the United
States Journal of Occupational and Environmental Medicine, 37(3,3), 336-348
Data Type: DCM breast cancer occupational case-control OR black2-Cancer
HERO ID: 194130
Domain Metric

Rating"'"
MWF*
Score
Comments^
Metric 6: Temporality

m
x 0.4
0.8
Exposure is likely to have occurred prior to the out-
come, but the exact timeline of occupational expo-
sures in relation to outcome isn't clear.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or
characterization
High
x 0.667
0.67
Outcome was assessed from causes of death listed
on official death certificates. Mortality from breast
cancer was determined using the underlying cause of
death (ICD-9, code 174) listed on the death certifi-
cate.
One outcome (breast cancer) was assessed, and is
appropriately identified in the study report. The
numbers of cases and controls included in the as-
sessment are also reported.
Metric 8: Reporting Bias

High
x 0.333
0.33
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High x 0.5 0.5 Analyses were adjusted for age at time of death,
and/or socioeconomic class. Results were stratified
by race.
Medium x 0.25 0.5 The assignment of SES was described in the current
reference as the SES status implied by the usual
occupation listed for an individual. This is not a
well-established method, but there is no evidence to
suggest that it is not a valid method.
Low x 0.25 0.75 The study authors discuss potential for "overlapping
exposures" and state this as a limitation, of the
study
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses

Medium x 0.4 0.8 This case-control study calculates odds ratios and
95% confidence intervals for probability and level of
exposure to DCM among breast cancer deaths across
24 states, from 1984 to 1989. The design is appro-
priate for investigating the effects of DCM on breast
cancer mortality.
m X 0.2 0.4 There were 29,397 white women cases, 102,955 white
women controls, 4,112 black women cases, and
14,839 black women controls included in the anal-
ysis. This was sufficient to detect an effect.
Low X 0.2 0.6 Some methods for covariate adjustments v/ere not
described. Assignment of SES was not fully de-
scribed.
Continued on next page

-------
. continued from previous page
Study Citation: Cantor, KP; Stewart, PA; Brinton, LA; Dosemeci, M (1995). Occupational exposures and female breast cancer mortality in the United
States Journal of Occupational and Environmental Medicine, 37(3,3), 336-348
Data Type:	DCM	breast cancer	occupational	case-control	OR	black2-Cancer
HERO ID:	194130
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 15: Statistical models	m X 0.2	0.4 Odds ratios were calculated for the odds of breast
cancer mortality, by the method published in Gart
(1970). Two models were presented (one age ad-
justed and the other age and SES adjusted). The
reasoning for inclusion of SES was discussed.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination*
High

1.6
Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
( 4	if any metric is Unacceptable
Overall rating = <
I	(Metric Score; X MWF;) / MWF,
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 7: Heineman et al. 1994: Evaluation of Cancer Outcomes
Study Citation: Heineman, EF; Cocco, P; Gomez, MR; Dosemeci, M; Stewart, PA; Hayes, RB; Zahm, SH; Thomas, TL; Blair, A (1994). Occupational
exposure to chlorinated aliphatic hydrocarbons and risk of astrocytic brain cancer American Journal of Industrial Medicine, 26(2),
155-169
Data Type: Case-control Occupational DCM AstrocyticBrainCancer Q2-C;
HERO ID: 194131
Domain
Metric
Rating^
MWF*
Score
Comments^
Domain 1: Study Participation

Medium

Metric 1:
Participant selection
x 0.4
0.8
Cases were gathered from death certificates of men
who died of brain or other central nervous system tu-
mors during 1978 to 1980 in southern Louisiana and
1979 to 1981 in northern New Jersey and Philadel-
phia, Pennsylvania. Interviews were conducted with
next-of-kin regarding occupational information. A
total of 300 cases, which reported a hospital diagno-
sis of astrocytic brain tumor, was used.
Metric 2:
Attrition
Medium
x 0.4
0.8
Among 483 cases with completed interviews (74% of
traced next-to-kin) a hospital diagnosis was reported
for 300 individuals. 229 cases had been pathologi-
cally confirmed. Of the matched controls 66 were
excluded due to a possible association between their
cause of death and occupational exposure to CAHs.
In logistic regression analysis, omitted 30 subjects
with electronics-related jobs.
Metric 3:
Comparison Group
Medium
x 0.2
0.4
Controls were frequency matched to cases by age,
year of death, cause of death other than brain

tumor/ cerebrovascular disease/ homicide/ suicide,

and study area. 320 total controls.
Domain 2: Exposure Characterization
1
Continued on next page . ..

-------
.. . continued from previous page
Study Citation: Heineman, EF; Cocco, P; Gomez, MR; Dosemeci, M; Stewart, PA; Hayes, RB; Zahm, SH; Thomas, TL; Blair, A (1994). Occupational
exposure to chlorinated aliphatic hydrocarbons and risk of astrocytic brain cancer American Journal of Industrial Medicine, 26(2),
155-169
Data Type: Case-control Occupational DCM AstrocyticBrainCancer Q2-Cancer
HERO ID: 194131
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
Low x 0.4
Metric 5: Exposure levels
Medi
um x 0.2
Metric 6: Temporality
Low x 0.4
1.2 Matrices were developed by first identifying the in-
dustry and occupation considered to entail potential
exposure to each of the CAHs based on data from lit-
erature, unpublished industrial hygiene reports and
inspection and by personal judgement of the project
industrial hygienist. Each industry and occupation
was assigned a semi-quanti.tati.ve estimate of proba-
bility and of intensity of exposure to each substance.
The matrices were then linked to the work histories
of the study subjects. Cumulative exposure indices
were calculated for each subject.
Judgments regarding exposure made by industrial
hygienists were based on work histories provided by
next-of-kin, who are likely to provide less accurate
information then subjects themselves or workplace
records. Poor specificity of some work histories for
specific solvents and the i.nter changeabi litv of sol-
vents for many applications probably reduced the
accuracy of exposure assignments.
0.4 Cumulative exposure score for each subject was cal-
culated as a weight sum of years in all exposed jobs,
with weight based on the square of the intensity of
exposure (low=l, medi.um=2, hi.gh=3) assigned to
each job. Average intensity was calculated over all
exposed jobs for each subjects based on same scores
without squaring, weighted by duration of employ-
ment in each job. Overall probability of exposure
was defined as highest probability score for that sub-
stance among their jobs.
1.2 Each industry and occupation was assigned positive
or zero decade indicators for each CAH according to
the likely use of the substance during each decade
between 1920 and 1980 because the use of CAHs
has changed over time. Matrices indicated if the ex-
posure was likely to occur by calendar period and
probability and intensity of exposure for each indus-
try and each occupation separately. Latency was
considered by lagging exposure by 10 or 20 years.
Domain 3: Outcome Assessment
Continued on next page

Metric 11: Co-exposure Confounding

Low
X 0.5 0.5 Adjusted for age, study area, employment, and prob-
ability of exposure to other chemicals of interest for
the logistic regression analysis,
um X 0.25 0.5 Characterized within methods, study population
section. Confounders not assessed by method
or instrument- used previous analyses to assess.
Cases and controls matched by confounding factors
(age, study area). Controlled for employment in
electronics-related occupations or industries (which
was associated with an excess risk of astrocytic brain
tumors in a previous analysis).
X 0.25 0.75 Co-exposure to electromagnetic fields was not as-
sessed or considered in the analysis.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Medium
Medium
T
X 0.4 0.8 Used appropriate statistical analyses and study de-
sign. Retrospective case-control included matrices
on likelihood of a certain chemical to have been
used in each industry and occupation by decade and
provided probability and intensity of exposure level.
Cumulative exposure indices were calculated for sub-
jects.
X 0.2 0.4 300 cases and 320 controls were used in the analysis.
x 0.2 0.6 It would be difficult to reproduce this analysis be-
cause of the lack of direct information on exposure
to various solvents. Information acquired from next-
of-kin was likely less accurate then information from
the subjects themselves or from industries that could
have provided it.
Continued on next page

-------
. continued from previous page
Study Citation: Heineman, EF; Cocco, P; Gomez, MR; Dosemeci, M; Stewart, PA; Hayes, RB; Zahm, SH; Thomas, TL; Blair, A (1994). Occupational
exposure to chlorinated aliphatic hydrocarbons and risk of astrocytic brain cancer American Journal of Industrial Medicine, 26(2),
155-169
Data Type:	Case-control	Occupational	DCM	AstrocyticBrainCancer	Q2-Cancer
HERO ID:	194131
Domain
Metric
Metric 15: Statistical models
Rating'
Medium
MWF* Score
Commentstt
x 0.2	0.4 Used maximum likelihood estimates of the OR and
95% CI adjusting for age and study area. Used
the statistical significance of linear trends by Man-
tel (1963). Logistic regression was used to evaluate
simultaneously the effects of the CAHs.
Domain 6: Other Considerations for Biomarker Selection and Mea.su
Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination"*"

Medium
2.1
Extracted
*	MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
*	The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =
(Metric Score; X MWF;) / J] . MWF,-
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the review r determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 8: Seidler et al. 2007: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Seidler, A; Mohner, M; Berger, J; Mester, B; Deeg, E; Eisner, G; Nieters, A; Becker, N (2007). Solvent exposure and malignant
lymphoma: A population-based case-control study in Germany Journal of Occupational Medicine and Toxicology, 2(tissue#), 2
>175 ppm*yrs DCM_B-NHL-Cancer-Cancer
194429
Domain
Metric
Rating*
MWF*
Score
Comments^
Domain 1: Study Participation





Metric 1:
Participant selection
High
x 0.4
0.4
Key elements of study design were reported includ-
ing description of study area, recruitment methods,
and participation rates. Rationale and study design
were previously published and cited (Becker et al.,
2004, HERO ID 729470). Complete details were re-
ported in that publication. Reported information
indicates selection in or out of the study and partic-
ipation is not likely to be biased.
Metric 2:
Attrition
Medium
x 0.4
0.8
Medium rating: participation rate among cases and
controls was 87.4% and 44.3%, respectively (controls
were recruited until 710 were selected), minimal ex-
clusion from the analysis sample and outcome data
and exposure were largely complete.
Metric 3:
Comparison Group
High
x 0.2
0.2
High rating: cases and controls were similar, for





each case, a gender, region and age-matched (± 1




year of birth) population control was drawn from the
population registration office; differences in baseline
characteristics of groups were also considered as po-
tential confounding variables and v/ere thereby con-
trolled by statistical analysis
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
High
x 0.4
Medi
Metric 6: Temporality
um x 0.2
Medium x 0.4
0.4 High rating: occupational population, question-
naires administered by trained interviewers that al-
lowed for construction of a job-matrix for entire work
history of exposure (i.e., cumulative exposures).
0.4 Medium rating: exposure was based on intensity
ranging from 0.5 to > 100 ppm and frequency rang-
ing from 1 to >30 percent, which were calculated
into cumulative ppm x years exposure. These were
separated into 3 or more levels of exposure including
a no exposure category.
0.8 Temporality is established but it is unclear whether
exposure fall within relevant windows for the out-
come of interest. A complete occupational history
was obtained, but there is no information provided
to indicate when exposures occurred in relation to
the cancer diagnosis.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Seidler, A; Mohner, M; Berger, J; Mester, B; Deeg, E; Eisner, G; Nieters, A; Becker, N (2007). Solvent exposure and malignant
lymphoma: A population-based case-control study in Germany Journal of Occupational Medicine and Toxicology, 2(tissue#), 2
Data Type: >175 ppm*yrs DCM B-NHL-Cancer-Cancer
HERO ID: 194429
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 7:
Outcome measurement or characterization High
X
0.667
0.67
Hospital and ambulatory physicians involved in the

diagnosis and therapy of malignant lymphoma v/ere

asked to identify cases; no assessment of validity (or

confirmation) of diagnosis was reported in the pa-

per but could be available in companion publications

that were cited, no evidence of differential misclas-

si.fi.cati.on
Metric 8:
Reporting Bias
High
X
0.333
0.33
High rating: all of the study's measured outcomes
are reported, effect estimates reported with confi-
dence interval; number of exposed reported for each
analysis.
Domain 4: Potential Counfounding/Variable Control
High

Metric 9:
Covariate Adjustment
X
0.5
0.5
High rating: appropriate adjustments or explicit
considerations were made for potential
confounders in the final analyses through the use of
statistical models for covariate
adjustment and matching by gender, region and age.
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
Medium rating: primary confounders (excluding co-
exposures) v/ere assessed. The paper
notes that trained interviewers administered ques-
tionnaires (medical history, lifestyle, occupation) to
subjects, did not describe if the questionnaire used
to collect information on education, smoking, etc.
has been previously validated.
Metric 11:
Co-exposure Confounding
Medium
X
0.25
0.5
Medium rating: co-exposures were measured and
modeled separately; the authors noted that a high
correlation was observed between PCE and TCE
(p=0.42). For this reason, it is difficult to disen-
tangle the specific effects of PCE and TCE on risk
of lymphoma.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
Medium rating: appropriate design (i.e., case con-
trol study of solvent exposure in relation to a rare
disease), and appropriate statistical methods (i.e.,
logistic regression analyses)
were employed to analyze data.
Continued on next page . ..

-------
. continued from previous page
Study Citation: Seidler, A; Mohner, M; Berger, J; Mester, B; Deeg, E; Eisner, G; Nieters, A; Becker, N (2007). Solvent exposure and malignant
lymphoma: A population-based case-control study in Germany Journal of Occupational Medicine and Toxicology, 2(tissue#), 2
Data Type:	>175 ppm*yrs DCM	B-NHL-Cancer-Cancer
HERO ID:	194429
Domain
Metric
Rating^
MWF*
Score
Comments^
Metric 13
Statistical power
m
x 0.2
0.4
Medium rating: authors noted that study pov/er
might have been insufficient to detect a slightly el-
evated lymphoma risk among DCM exposed sub-
jects or to detect an increased lymphoma risk among
PCE-exposed subjects. Note: For some subgroups,
effect estimate is based on a small number of cases
and controls.
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Medium rating: description of the analyses is suffi-
cient to understand what has been
done and to be reproducible with access to the data
Metric 15
Statistical models
Medium
x 0.2
0.4
Medium rating: logistic regression models were used
to generate Odds Ratios. Rationale
for variable selection is stated. Model assumptions
are met.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
High

1.5

Extracted

Yes



MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T\ (Metric Score,; x MWF;) / J] . MWF;
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 9: Dosemeci et al. 1999: Evaluation of Cancer Outcomes
Study Citation: Dosemeci, M; Cocco, P; Chow, WH (1999). Gender differences in risk of renal cell carcinoma and occupational exposures to chlorinated
aliphatic hydrocarbons American Journal of Industrial Medicine, 36(1), 54-59
Data Type: renal cancer and occupational DCM-Cancer
HERO ID: 194813
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
x 0.4
Medium x 0.4
Metric 3: Comparison Group

Medium x 0.2
0.4 Selection was provided in detail and indicates that
selection into or out of the study is not likely biased.
0.8 There was an overall 86% response rate that did not
differ between cases and controls. For the occupa-
tional analysis, 438 of the 690 cases and 687 of the
690 controls with complete personal interviews were
included. There does not appear to be any miss-
ing data for the included 438 cases and 687 controls.
However, all cases who died (35%) were excluded
from the analysis to avoid using next-of-kin inter-
views.
0.4 For subjects age 20-64 years, an age- and gender-
stratified random sample of white controls was ob-
tained with random digit dialing. For subjects age
65-85 years, an age-and gender-stratified systematic
sample of white controls was obtained from the list-
ing of the Health Care Financing Admi.ni.strati.on.
This is a po p u 1 at ion- based case control study in Min-
nesota. No information on characteristics were pro-
vided for comparing the cases and controls, but they
were similar in terms of age, sex, and ethnicity (all
were noted to be white).
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low
Low
x 0.2
x 0.4
0.8 Occupational history was obtained via interview.
Duration of employment in 13 specific occupa-
tions/industries and seven jobs with specific expo-
sures were obtained. Occupations and industries
were codes based on standard classifications and
JEMs were developed by the NCI for nine individ-
ual chemicals including Perc, CC14, TCE, and DCM.
Details of the J EM were provided (Dosemeci et al.,
1994; Gomez et ah, 1994 HERO ID 702154). The
J EM is based on probability and intensity scales.
0.6 Unclear, but appears to be exposed versus unex-
posed.
1.2 The temporality of exposure and outcome is uncer-
tain.
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: Dosemeci, M; Cocco, P; Chow, WH (1999). Gender differences in risk of renal cell carcinoma and occupational exposures to chlorinated
aliphatic hydrocarbons American Journal of Industrial Medicine, 36(1), 54-59
Data Type: renal cancer and occupational DCM-Cancer
HERO ID: 194813
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 7:
Outcome measurement or characterization
High
X
0.667
0.67
RCC were histologically confirmed and identified

through the Minnesota Cancer Surveillance System.
Metric 8:
Reporting Bias
Medium
X
0.333
0.67
All outcomes are reported, but not in a way that
would allow for detailed extraction.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
Medium
X
0.5
1
Results adjusted for age, gender, smoking, hyper-
tension, use of specific drugs, and BMI. There is not
enough information provided to know if SES would
be a potential confounder, but considering that con-
trols were randomly selected it is unlikely that this
would be a major potential confounder.
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
Information was collected via a questionnaire, but
validity and reliability were not reported.
Metric 11:
Co-exposure Confounding
Medium
X
0.25
0.5
There is no evidence to indicate that there v/ere co-
exposures that would appreciably bias the results.
Although this was occupational exposure, subjects
came from different occupations and areas; there-
fore, it is unlikely that there would have been differ-
ential co-exposures.
Domain 5: Analysis
Metric 12:
Metric 13:
Metric 14:
Study Design and Methods
Statistical power
Reproducibility of analyses
Metric 15: Statistical models
Medium
Medium
Medium
Medium
X
0.4
0.8
Study design was appropriate for the research ques-
tion.
X
0.2
0.4
Statistical pov/er should be sufficient.
X
0.2
0.4
The description of the analysis was sufficient to re-
produce with access to the analytical data.
X
0.2
0.4
Methods are transparent.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Metric 22
Matrix adjustment
NA
NA
Overall Quality Determination
Medium
1.9
Extracted
Yes
Continued on next page

-------
. continued from previous page
Study Citation: Dosemeci, M; Cocco, P; Chow, WH (1999). Gender differences in risk of renal cell carcinoma and occupational exposures to chlorinated
aliphatic hydrocarbons American Journal of Industrial Medicine, 36(1), 54-59
Data Type:	renal cancer and occupational DCM-Cancer
HERO ID:	194813
Domain	Metric	Rating^ MWF* Score	Comments^
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
]T. (Metric Score; X MWF;) / ^ . MWF;
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 10: Wang et al. 2009: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Wang, R; Zhang, Y; Lan, Q; Holford, TR; Leaderer, B; Zahm, SH; Boyle, P; Dosemeci, M; Rothman, N; Zhu, Y; Qin, Q; Zheng, T
(2009). Occupational exposure to solvents and risk of non-Hodgkin lymphoma in Connecticut women American Journal of Epidemiology,
169(2), 176-186
Non Hodgkin Lymphoma Connecticut women DCM-Cancer
626703
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
CO
Metric 2: Attrition
Metric 3: Comparison Group

um x 0.4
um x 0.4
Medium x 0.2
0.8 Authors reported that participants in this study
were women ages 21-84 years from Connecticut from
1996 to 2000. The cases were histologically con-
firmed with non-Hodgkins Lymphoma in Connecti-
cut and had no history of any type of cancer (except
nonmelanoma skin cancer). Controls with Connecti-
cut addresses (ages 65 or less) v/ere recruited by ran-
dom digit dialing or by random selection from Cen-
ters for Medicare and Medicaid Services files (ages 65
or older). Cases and controls were matched within
5-year age groups. Both cases and controls held 3-4
jobs during their lifetime but no table was provided
comparing covariates in cases vs. controls.
0.8 Of the NHL cases, 601 out of 832 (72%) completed
in person-intervi.ev/s. Of the controls, the partici-
pation rate for those identified via random digit di-
aling was 69% and it was 47% for those from the
Health Care Financing Ad ministration. In-person
interviews were completed for 717 controls. Out-
come data included information on all 601 cases and
717 controls.
0.4 The participants were from the same population
(Connecticut women) and they were matched within
5-vears of age. They v/ere adjusted for age, family
history of hematopoietic cancers, alcohol consump-
tion, and race.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Wang, R; Zhang, Y; Lan, Q; Holford, TR; Leaderer, B; Zahm, SH; Boyle, P; Dosemeci, M; Rothman, N; Zhu, Y; Qin, Q; Zheng, T
(2009). Occupational exposure to solvents and risk of non-Hodgkin lymphoma in Connecticut women American Journal of Epidemiology,
169(2), 176-186
Non Hodgkin Lymphoma	Connecticut women	DCM-Cancer
626703
Domain
Metric
Ratingt
Medium
Metric 4: Measurement of Exposure
MWF* Score
Commentstt
Metric 5: Exposure levels
Metric 6: Temporality
Medium
Medi
um

X 0.4	0.8 Exposure was based on the job classification by link-
ing the coded occupational data with a job-exposure
matrix updated by industrial hygienists at the NCI.
Every occupation and industry was assigned a semi-
quantitative estimate of intensity and probability ac-
cording to a scale of 0-3. Intensity was estimated on
the basis of expected exposure level and frequency
and exposure probability was the likelihood that a
specific substance was used by a worker in a given
industry or occupation. The final scores for average
exposure intensity and probability were categorized
as never exposed (0), low (< 3), medium (3-5), and
high intensity/probablity (> = 6). This method of
exposure classification could result in some misclas-
sifi.cati.on of exposure, since the occupational histo-
ries were self-reported.
X 0.2	0.4 The study used three distributions of exposure:
never, low, and medi.um-hi.gh which are sufficient to
determine an exposure-response relationship.
X 0.4	0.8 Participants provided information on their lifetime
occupational history. Exposure within 1 year be-
fore diagnosis/interview was excluded from the in-
terview process, however since non-Hodgkins Lym-
phoma takes many years to develop after exposure,
it is unclear if all exposures fell within the relevant
window to see the effect.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
Metric 8: Reporting Bias
x 0.667 0.67 The study said that cases of Non-Hodgkin Lym-
phoma were histologically confirmed, but presents
no further information on the procedure used to con-
firm the diagnosis
x 0.333 0.33 The results section presents tables that present the
number of cases and controls and the odds ratio and
95% confidence limits for exposure to each solvent
at the never, low, and medi.um-hi.gh exposure levels
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
High	X 0.5	0.5 All participants were Connecticut women. ORs for
cases and controls were adjusted for age, family his-
tory of hematopoieti.c cancers, alcohol consumption,
and race
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Wang, R; Zhang, Y; Lan, Q; Holford, TR; Leaderer, B; Zahm, SH; Boyle, P; Dosemeci, M; Rothman, N; Zhu, Y; Qin, Q; Zheng, T
(2009). Occupational exposure to solvents and risk of non-Hodgkin lymphoma in Connecticut women American Journal of Epidemiology,
169(2), 176-186
Non Hodgkin Lymphoma	Connecticut women	DCM-Cancer
626703
Domain
Metric
Ratine* MWF*
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
Hating
Medium
Score
Comments**
x 0.25
Medium x 0.25
0.5 In-person interviews using a standardized, struc-
tured questionnaire were used to collect information
on confounders. However, the authors don't report
that the questionnaire was validated.
0.5 The job histories were divided by potential exposure
to 8 specific organic, solvents, any organic solvent, or
chlorinated solvents in general. However, since the
occupational histories were self-reported, there is a
possibility of exposure misclassification which could
have resulted in non-reporting of co-exposures.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models

um x 0.4
Medium x 0.2
Medium x 0.2
Medium x 0.2
0.8 A case-control study was the appropriate type of
study to measure the possible association between
occupational exposure and development of Non-
Hodgkins Lymphoma and the statistical method
used - determination of Odds Ratio was appropri-
ate.
0.4 This study consisted of 601 cases and 717 controls
which are a sufficient number to detect the effect of
non-Hodgkins Lymphoma.
0.4 Description of the statistical methods was sufficient
to reproduce the logistic regression models and ad-
justment factors were included in the footnotes to
the tables.
0.4 Adjustment factors used in the final model were de-
termined based on logistic regression models and ad-
justment for other variables, such as level of educa-
tion, annual family income, tobacco smoking, and
medical history of immune-related disease did not
result in material changes for the observed associa-
tions and were not included in the final model.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Continued on next page . ..

-------
. continued from previous page
Study Citation: Wang, R; Zhang, Y; Lan, Q; Holford, TR; Leaderer, B; Zahm, SH; Boyle, P; Dosemeci, M; Rothman, N; Zhu, Y; Qin, Q; Zheng, T
(2009). Occupational exposure to solvents and risk of non-Hodgkin lymphoma in Connecticut women American Journal of Epidemiology,
169(2), 176-185
Data Type:	Non Hodgkin Lymphoma_Connecticut women_D CM - C ancer
HERO ID:	626703
Domain Metric
Rating1" MWF*
Score
Commentstt
Metric 22: Matrix adjustment
NA
NA

Overall Quality Determination*
Medium
1.7

Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always
Overall rating =
]T\ (Metric Score; x MWF;) / ^ . MWFj
)r a metric that has been categorized as High,
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to ^ 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 11: Infantc-Rivard 2005: Evaluation of Cancer Outcomes
Study Citation: Infante-Rivard, C; Siemiatycki, J; Lakhani, R; Nadon, L (2005). Maternal exposure to occupational solvents and childhood leukemia
Environmental Health Perspectives, 113(6,6), 787-792
Data Type: DCM Case-Control Children 2 Years Before Pregnancy ALL-Cancer
HERO ID: 630639

Domain
Metric
Rating*
MWF'
Score
Comments^

Domain 1: Study Participation

Metric 1:
Participant selection
High
X 0.4
0.4
Included 848 eligible cases. Cases of acute lym-
phoblastic, leukemia diagnosed between 1980 and
2000 in the province of Quebec, Canada were re-
cruited from tertiary care centers. Between 1980 and
1993 cases 0-9 yrs. at diagnosis were recruited, be-
tween 1994 and 2000 cases included up to 14 yrs. at
diagnosis. 790 parents were interviewed.

Metric 2:
Attrition
High
X 0.4
0.4
Children who v/ere adopted, lived in foster fami-
lies, families spoke neither English or French, who
did not reside in Canada, whose parents were both
unavailable for interviews were excluded. Reasons
for nonparticipation were confidential phone num-
CO

ber, refusal to participate, or inability to trace the
oo

High

family.

Metric 3:
Comparison Group
X 0.2
0.2
Population based controls were matched on sex and
age at the same time of diagnosis. They were concur-
rently selected. From 1980 to 1993 population-based
controls were chosen from family allowance files,
Regie des Rentes du Quebec, Quebec, Canada. This
data was the most complete census of children. Be-
tween 1994 and 2000, they used provincial universal
health insurance files, Regie de PAssurance Maldie
du Quebac,, Quebec, Canada, for controls. They
switched to this source because family allowances
were more often directly deposited in the mother's
bank account. 916 eligible controls were found.
Domain 2: Exposure Characterization
Continued on next page . ..

-------
.. . continued from previous page
Study Citation: Infante-Rivard, C; Siemiatycki, J; Lakhani, R; Nadon, L (2005). Maternal exposure to occupational solvents and childhood leukemia
Environmental Health Perspectives, 113(6,6), 787-792
Data Type: DCM Case-Control Children 2 Years Before Pregnancy ALL-Cancer
HERO ID: 630639
Domain
Metric
Rating^
MWF*
Score
Comments^
Metric 4:
Measurement of Exposure
High
X
0.4
0.4
Exposure coding was used. Carried out by assigning
each occupation a standard Canadian industrial title
and job titles. Job information was acquired through
questionnaires that asked for each job held by the
mother from 2 yrs. Before pregnancy and up to birth
of the index child. They determined whether there
was or was not exposure to specific solvents or chem-
ical mixtures with solvents. Questionnaire included
items to assess exposure to solvents at home. For
each question, they asked who carried out the activ-
ity and during what time period, specified as 1 vr.
before pregnancy, during pregnancy, and from birth
to the reference date.
Metric 5:
Exposure levels
[m
X
0.2
0.4
For exposure period ranging from 2 years before
pregnancy up to birth, they repeated analysis con-
trasting 'any exposure' and 'no exposure'. Exposure
was coded as level 0 (baseline), no exposure (de-
fined as none coded or 'possible' confidence); level
1, some exposure (exposure resulting in concentra-
tion x frequency < 4), and level 2, greater exposure
(concentration x frequency >= 4).
Metric 6:
Temporality
Medium
X
0.4
0.8
Study provides appropriate temporality between ex-
posure to methylene chloride and childhood acute
lymphoblastic leukemia of either 2 years before preg-
nancy or exposure while pregnant.
Domain 3: Outcome Assessment
High

Metric 7:
Outcome measurement or characterization
X
0.667
0.67
Acute lymphoblastic leukemia was assessed in cases

using we 11-estab 1 ished methods. Cases were deter-
mined to have acute lymphoblastic leukemia (In-
ternational Classification of Diseases, 9th Revision,
code 204.0) on the basis of clinical diagnosis by an
oncologist or hematologist.
Metric 8:
Reporting Bias

X
0.333
0.33
Chemists who carried out the exposure coding were
blind to the case/control status. Description of mea-
sured acute lymphoblastic leukemia is reported in
the methods section. Number of cases and controls
are reported for each analysis. Effect estimates are
reported with sufficient details (odds ratios and 95%
confidence intervals.) to allow for data extraction.
Domain 4: Potential Counfounding/Variable Control
Continued on next page . ..

naire administered by telephone. There is no infor-

mation on the reliability of the data obtained from

Medium

questionnaires.
Metric 11:
Co-exposure Confounding
X
0.25
0.5
No indication of unbalanced co exposures. Co-
exposures were appropriately measured or either di-
rectly or indirectly adjusted for.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
The case-control design was appropriate for this
study. Description of analysis is sufficient for un-

Medium

derstanding and the reproducibility of the data.
Metric 13:
Statistical power
X
0.2
0.4
Number of cases and controls is adequate. Identi-
fied 848 cases and interviewed 790 case parents. 916
eligible controls were identified and interviewed 790
control parents.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
Study design and methods can be reproducible with
information provided. Provided reasoning on how
categories were created for exposure levels, why co-

Medium

variates were used.
Metric 15:
Statistical models
X
0.2
0.4
Conditional logistic regression was used to estimate
odds ratio and 95% confidence intervals. Each agent,
mixture, and family were analyzed in a separate
model and analyses.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric
16
Use of Biomarker of Exposure
NA
NA
Metric
17
Effect biomarker
NA
NA
Metric
18
Method Sensitivity
NA
NA
Metric
19
Biomarker stability
NA
NA
Metric
20
Sample contamination
NA
NA
Metric
21
Method requirements
NA
NA
Metric
22
Matrix adjustment
NA
NA
Continued on next page . ..

-------
. continued from previous page
Study Citation: Infante-Rivard, C; Siemiatycki, J; Lakhani, R; Nadon, L (2005). Maternal exposure to occupational solvents and childhood leukemia
Environmental Health Perspectives, 113(6,6), 787-792
Data Type:	DCM_Case-Control_Children_2 Years Before Pregnaney_ALL-Cancer
HERO ID:	630639
Domain	Metric	Rating^ MWF* Score	Comments^
Overall Quality Determination'	High	1.5
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
7;. (Metric Score-; x MWFj) / ^ . MWF?	(round to the nearest tenth) otherwise
*	3	o.i
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 12: Miligi et al. 2006: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Miligi, L; Costantini, AS; Benvenuti, A; Kriebel, D; Bolejack, V; Tumino, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Croeignani,
P; Amadori, D; Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, GA; Mendico, I; Vineis, P (2006).
Occupational exposure to solvents and the risk of lymphomas Epidemiology, 17(5), 552-561
Very low/low DCM exposure intensity level-Car
630788
—
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
High
High

Metric 3: Comparison Group
High
X 0.4	0.4 High rating: key elements of study design were re-
ported, and the reported information indicates se-
lection in or out of the study and participation is
not likely to be biased.
X 0.4	0.4 High rating: minimal subject withdrawal from the
study, and outcome data and exposure were largely
complete: 1428 NHL cases (of 1719 eligible in the
8 areas [83%]), 304 HD cases (of 347 [88%]), and
1530 controls (of 2086 [73%]). The reasons for non-
participation v/ere refusal of interviews (11% of NHL
cases, 8% of HD cases, and 21 % of the controls), sub-
ject not traced (2.4%, 2.9%, and 3.0%, respectively),
and not interviewed because of illness or impairment
(3.2%, 1.4%, and 3.2%, respectively)
X 0.2	0.2 High rating: cases and controls were similar; con-
trols randomly selected from the general population
in each of the areas under study, differences in base-
line characteristics of groups were considered as po-
tential confounding or stratification variables (i.e,.
sex and 5-year age groups) and were thereby con-
trolled by statistical analysis.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Low	x 0.4
Metric 5: Exposure levels
Medium x 0.2
1.2 Low rating: Occupational study population with
exposure assessed using job-specific or industry-
specific questionnaires with subsequent expert rat-
ings to assign exposure to a definitive list of agents
(i.e., no employment records). Industrial hygiene ex-
perts from each geographic area examined data col-
lected in the questionnaires, and assessed a level of
probability and intensity of exposure to groups or
classes of solvents as well as certain individual sub-
stances. Reviewers blinded to disease status.
0.4 Medium rating: range and distribution of exposure
was sufficient to develop an exposure-response esti-
mate; 3 or more levels of exposure v/ere reported
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Miligi, L; Costantini, AS; Benvenuti, A; Kriebel, D; Bolejack, V; Tumino, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Crosignani,
P; Amadori, D; Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, GA; Mendico, I; Vineis, P (2006).
Occupational exposure to solvents and the risk of lymphomas Epidemiology, 17(5), 552-561
Very low/low DCM exposure intensity level-Cancer
630788
Domain
Metric
Metric 6: Temporality
Ratingt
Medium
MWF* Score
Commentstt
X 0.4 0.8 The study identified newly diagnosed cases of NHL
and assessed exposure via job^specific and industry
specific questionnaires. It is assumed that exposure
preceded the outcome but this is not clear.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
Metric 8: Reporting Bias
High
x 0.667 0.67 NHL cases were classified following the working for-
mulation proposed by the U.S. National Cancer In-
stitute. A panel of 3 pathologists reviewed all doubt-
ful NHL diagnoses (that is, cases for whom the local
pathologist had expressed uncertainties about the
allocation in a specific NHL category), as well as
a randomly selected 20% sample of all cases. The
NHL diagnosis was confirmed for all 334 cases that
were reviewed.
X 0.333 0.33 High rating: all of the study's measured outcomes
are reported, effect estimates reported with confi-
dence interval; number of exposed reported for each
analysis.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Medium x 0.25

Metric 11: Co-exposure Confounding
High X 0.5 0.5 High rating: appropriate adjustments or explicit
considerations were made for potential confounders
in the final analyses through the use of statistical
models for covariate adjustment
0.5 Medium rating: Primary confounders (excluding co-
exposures) were assessed. The paper did not de-
scribe if the questionnaire used to collect informa-
tion on education, smoking, etc. has been previously
validated.
X 0.25 0.5 Medium rating: co-exposures were measured and
modeled separately, and the authors noted that
'...high degree of correlation among exposures to
benzene, xylene, and toluene. For this reason, cau-
tion must be exercised when interpreting the evi-
dence for any one of these 3 solvents.' However,
there does not appear to be direct evidence of an co-
pollutant confounding of the relation between DCM,
TCE, PCE, and NHL.
Medium
Domain 5: Analysis
Continued on next page

-------
. continued from previous page
Study Citation: Miligi, L; Costantini, AS; Benvenuti, A; Kriebel, D; Bolejack, V; Tumino, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Crosignani,
P; Amadori, D; Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, GA; Mendico, I; Vineis, P (2006).
Occupational exposure to solvents and the risk of lymphomas Epidemiology, 17(5), 552-561
Data Type:	Very low/low DCM exposure intensity level-Cancer
HERO ID:	630788
Commentstt
Medium rating: appropriate design (i.e., case con-
trol study of DCM/TCE/PCE exposure in relation
to a rare disease, NHL), and appropriate statistical
methods (i.e., logistic regression analyses) were em-
ployed to analyze data.
The number of cases and controls are adequate to
detect an effect in the exposed population and/or
subgroups of the total population.
Medium rating: description of the analyses is suffi-
cient to understand what has been done and to be
reproducible with access to the data.
Medium rating: logistic, regression models were used
to generate Odds Ratios. Rationale for variable se-
lection is stated. Model assumptions are met.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure


NA
NA
Metric 17
Effect biomarker


NA
NA
Metric 18
Method Sensitivity


NA
NA
Metric 19
Biomarker stability


NA
NA
Metric 20
Sample contamination


NA
NA
Metric 21
Method requirements


NA
NA
Metric 22
Matrix adjustment


NA
NA
Overall Quality Determination*

High

1.6
Extracted	Yes
Domain	Metric	Ratine' MWF* Score
Metric 12: Study Design and Methods	Medium x 0.4 0.8
Hating
Medium
Metric 13: Statistical power	Medium x 0.2 0.4
Metric 14: Reproducibility of analyses	Medium x 0.2 0.4
Metric 15: Statistical models	Medium x 0.2 0.4
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWF,) / J] . MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 13: Costaiitini et al. 2008: Evaluation of Cancer Outcomes
Study Citation: Costantini, AS; Benvenuti, A; Vineis, P; Kriebel, D; Tumino, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Crosignani, P; Amadori, D;
Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, G; Mendico, I; Maltoni, S; Miligi, L (2008). Risk of
leukemia and multiple myeloma associated with exposure to benzene and other organic solvents: Evidence from the Italian Multicenter
Case-control study American Journal of Industrial Medicine, 51(11,11), 803-811
Data Type: DCM_population-based case-control_leukemia low-Cancer
HERO ID: 699230
Domain
Metric
ating
MWF*
Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
High

High
High

x 0.4 0.4 In this case-control study in 11 areas of Italy, all
cases of hemato 1 vmphopoieti.c malignancies in males
and females ages 20-74 years in the years 1991-1993
were identified. A total of 2,737 cases of malignan-
cies were interviewed and the control group consisted
of 1,779 subjects randomly selected through the de-
mographic files of municipalities in each of the areas
under study, stratified by sex and 5-year age group.
Table 1 presents information on the characteristics
of the cases and controls, showing that the demo-
graphic characteristics were similar.
x 0.4 0.4 Table 1 indicates that outcome data was generally
complete. Any missing information was minimal and
is not likely to appreciably bias the results.
X 0.2 0.2 The cases and controls were recruited from the same
populations (11 areas in Italy) and were of the same
age range and sex. The authors state that the con-
trol group was selected through demographic files of
the municipalities in each of the areas under study.
The authors do not describe how the cases were iden-
tified, but refer to Costantini et al. 2001. Potential
confounders were considered and analyzed and pre-
sented in Table 1, several covariates were adjusted
for in the final model.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation: Costantini, AS; Benvenuti, A; Vineis, P; Kriebel, D; Tumino, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Crosignani, P; Amadori, D;
Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, G; Mendico, I; Maltoni, S; Miligi, L (2008). Risk of
leukemia and multiple myeloma associated with exposure to benzene and other organic solvents: Evidence from the Italian Multicenter
Case-control study American Journal of Industrial Medicine, 51(11,11), 803-811
Data Type: DCM population-based case-control leukemia low-Cancer
HERO ID: 699230
Domain
Metric
MWF* Score
Commentstt
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
^ X 0.4 1.2 Exposure assessments were based on the utilization
of job or industry-specific questionnaires and subse-
quent expert ratings in order to assign a level of ex-
posure to the chemicals. Industrial hygiene experts
from each geographic area were selected to examine
questionnaires and assess a level of probability and
intensity of exposure to chemicals. The assessment
was blind with respect to case/control status. Ex-
posure was rated on two scales: probability, which
was classified into 3 levels (low, medium, and high),
and intensity, which was measured on a 4-point scale
(very low, low, medium, and high). To ensure a
standardized approach, the assessors were centrally
trained prior to and periodically during their inde-
pendent evaluation of questionnaires.
Low X 0.2 0.6 Only two levels of exposure were assessed in the
analysis: very low/low, and medium/high. These
limited exposure levels are not sufficient to provide
a high degree of accuracy in the exposure-response
assessment analysis. Analyses for duration of expo-
sure considered two levels: less than 15, and 15 or
more years.
Medium X 0.4 0.8 The outcomes assessed v/ere leukemia and multiple
myeloma identified in the years 1991-1993. Expo-
sure to the chemicals was assessed based on job
or industry-specific, questionnaires. It is unclear
whether the exposures fall within the relevant ex-
posure time-frame for development of leukemia and
multiple myeloma.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
Medium
High
x 0.667 1.33 Table 2 of the study report presents the I CD-9 codes
(leukemia, 204-208; chrnoic lymphatic leukemia,
204.1) that were used to identify cases of leukemia
or multiple myeloma in the study, details on case
ascertainment were not discussed in the current ref-
erence but are included in Costantini et al. 2001
(Not found in HERO).
x 0.333 0.33 The results for the association between leukemia or
multiple myeloma with DCM and other chemicals
were reported in Table 2.
Continued on next page

-------
. continued from previous page
Study Citation: Costantini, AS; Benvenuti, A; Vineis, P; Kriebel, D; Tumino, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Crosignani, P; Amadori, D;
Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, G; Mendico, I; Maltoni, S; Miligi, L (2008). Risk of
leukemia and multiple myeloma associated with exposure to benzene and other organic solvents: Evidence from the Italian Multicenter
Case-control study American Journal of Industrial Medicine, 51(11,11), 803-811
Data Type:	DCM	population-based case-control	leukemia low-Cancer
HERO ID:	699230
Domain
Metric
MWF* Score
Commentstt
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
x 0.5
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High x 0.25
Medium x 0.25
0.5 Information on education, tobacco smoking, bev-
erage consumption, occupational history, extra-
occupational exposure to solvents and pesticides,
hair dye use, lifelong residential history, previous
diseases, use of diagnostic, or therapeutic X-rays,
specific medications, family medical history, and re-
productive history was obtained by person-to-person
interviews that used a specific questionnaire admin-
istered by trained personnel. The study adjusted
for gender, age, education, and study area in the
final analysis. The study also examined the educa-
tion and smoking status of the cases and controls to
ensure the two groups were comparable.
0.25 The information on covariates was obtained by
person-to-person interviews that used a specific,
questionnaire done by trained personnel.
0.5 The information on co-exposures was obtained by
person-to-person interviews that used a specific,
questionnaire done by trained personnel.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Medium x 0.4 0.8
Medium x 0.2 0.4
Metric 14: Reproducibility of analyses
Medium x 0.2
0.4
The study used an appropriate design to assess the
relationship between chemical exposure and hema-
tolymphopoietic malignancies. The study calculated
odds ratios and the corresponding 95% confidence
limits using multiple logistic regression models, tak-
ing into account relevant potential confounders.
The study examined (in total) 355 cases and 811
controls (leukemia), 133 cases and 911 controls
(acute myeloid leukemia), 103 cases and 925 con-
trols (chronic lymphatic leukemia), and 163 cases
and 674 controls (multiple myeloma). This is a suffi-
cient number of cases and controls to detect an effect
in the exposed population. However, the number of
cases and controls exposed to DCM was quite small
(2-28) and may not have been sufficient to detect an
effect.
The description of the analysis was sufficient to un-
derstand what was done and to be conceptually re-
producible with access to the analytic data.
Continued on next page

-------
. continued from previous page
Study Citation: Costantini, AS; Benvenuti, A; Vineis, P; Kriebel, D; Tumi no, R; Ramazzotti, V; Rodella, S; Stagnaro, E; Crosignani, P; Amadori, D;
Mirabelli, D; Sommani, L; Belletti, I; Troschel, L; Romeo, L; Miceli, G; Tozzi, G; Mendico, I; Maltoni, S; Miligi, L (2008). Risk of
leukemia and multiple myeloma associated with exposure to benzene and other organic solvents: Evidence from the Italian Multicenter
Case-control study American Journal of Industrial Medicine, 51(11,11), 803-811
Data Type:	DCM_population-based case-control_leukemia low-Cancer
HERO ID:	699230
Domain
Metric
g*
MWF*
Score Commentstt
Metric 15
Statistical models
Medium
x 0.2
0.4 The use of the odds ratio for calculating the risk
estimates was transparent and was presented in the
paper in sufficient detail.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination
Medium

1.7
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWF,) / J] . MWF?
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this tvp

-------
Table 14: Radican et al. 2008: Evaluation of Cancer Outcomes
Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319
Data Type: Hill Air Force Base DCM BreastCancer Females-Cancer
HERO ID: 699234
Domain
Metric
Rating*
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
High x 0.4
x 0.2
0.4 This study consisted of an extended follow-up of
the Hill Air Force Base occupational cohort through
2000. The cohort is composed of former civilian em-
ployees, who worked at this aircraft maintenance fa-
cility for at least 1 year between January 1, 1952 and
December 31, 1956 (n= 14,455). The key elements of
the study design were reported. Selection into the
study was not likely to be biased. The cohort was
described in detail in previous publications (Spirtas
et al. 1991; Stewart et al. 1991; Blair et al. 1998).
0.4 There was no loss of subjects to follow-up reported
in the study (as of December 31 2000, 8580 subjects
had died and 5875 were still alive); exposure and
outcome data were largely complete.
0.2 Key elements of the study design are reported. Ef-
fects levels were adjusted for age, race, and/or sex.
The use of an internal comparison group likely re-
duces the risk of bias relative to the use of an exter-
nal reference group (e.g., the healthy worker effect).
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
Medium x 0.4 0.8
x 0.2 0.6
High x 0.4 0.4
The exposure assessment was conducted by the Na-
tional Cancer Institute (NCI), using job-exposure
matrices, based on information provided by the Air
Force. Although exposure misclassification was pos-
sible (because individual exposure records were not
available), misclassification was likely random and
not to appreciably bias the results.
For 21 chemicals (including TCE, Perc, CC14 and
DCM), exposure was classified as yes/no. No quan-
titative assessment of exposure was conducted.
The study presents the appropriate relationship be-
tween exposure and outcome. Outcome was ascer-
tained after information on exposure was obtained.
There was a long follow-up period.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319
Data Type: Hill Air Force Base DCM BreastCancer Females-Cancer
HERO ID: 699234
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization Medium x 0.667 1.33
Metric 8: Reporting Bias
.
x 0.333 0.33
The outcome was determined from death records
from the National Death Index (NDI). It was noted
in the study that mortality data can be mislead-
ing owing to inaccuracies captured in patient death
records.
A description of measured outcomes is provided in
the study report. Effects estimates are provided
with confidence limits; number of exposed cases is
included.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding

Low
X 0.5 1.5 Adjustments v/ere made for age, race, and gen-
der. However, there was indirect evidence that so-
cioeconomic status (SES) was considerably differ-
ent among exposed and non-exposed populations.
The proportion of non-exposed persons that were
salaried was 61% compared to < 1% in the ex-
posed cohort, suggesting a dissimilar SES. This dif-
ference may affect the results for some specific cancer
types / d iseases.
um X 0.25 0.5 Confounders were assessed using reliable methods
(database of employees and NDI). However, other
than age, gender, and race, data on other factors
(disease history, SES) were not available.
X 0.25 0.75 The study evaluated exposure to DCM and various
other chemicals. Exposures were not mutually ex-
clusive; therefore, it was not possible to evaluate the
risk of death from exposure to a singular chemical
while controlling for exposure to other chemicals.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Medium x 0.4 0.8
j.jm x 0.2 0.4
Medium x 0.2 0.4
The cohort design and calculation of hazard ratios
were appropriate for determining the association be-
tween exposure to TCE, Perc,, CC14 and DCM, and
all-cause, cancer, and non-cancer mortality.
The cohort was large (adequate for statistical anal-
yses). Despite the relatively large size of the cohort,
the number of cases for many causes of death was
small to evaluate associations.
The analysis (exposure estimation and statistical
modeling) is described in sufficient detail to un-
derstand what was done and is conceptually repro-
ducible.
Continued on next page

-------
. continued from previous page
Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319
Data Type:	Hill	Air	Force	Base	DCM	BreastCancer	Females-Cancer
HERO ID:	699234
Domain
Metric
Rating^
MWF*
Score
Comments^t
Metric 15
Statistical models
m
x 0.2
0.4
The method and model assumptions used to cal-
culate risk estimates for occupational exposure to
TCE, Perc, CC14 and DCM and all-cause and cause-
specific mortality (hazard ratios) are clearly de-
scribed in the study report.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.8

Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
( 1	if any metric is Unacceptable
Overall rating = <
I	(Metric Score; X MWF;) / MWF,
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 15: Radican et al. 2008: Evaluation of Respiratory Outcomes
Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319
Data Type: Hill Air Force Base DCM Bronchitis Males-Respiratory
HERO ID: 699234
Domain
Metric
Rating*
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
High x 0.4
x 0.2
0.4 This study consisted of an extended follow-up of
the Hill Air Force Base occupational cohort through
2000. The cohort is composed of former civilian em-
ployees, who v/orked at this aircraft maintenance fa-
cility for at least 1 year between January 1, 1952 and
December 31, 1956 (n= 14,455). The key elements of
the study design were reported. Selection into the
study was not likely to be biased. The cohort was
described in detail in previous publications (Spirtas
et al. 1991; Stewart et al. 1991; Blair et al. 1998).
0.4 There was no loss of subjects to follow-up reported
in the study (as of December 31 2000, 8580 subjects
had died and 5875 were still alive); exposure and
outcome data were largely complete.
0.2 Key elements of the study design are reported. Ef-
fects levels were adjusted for age, race, and/or sex.
The use of an internal comparison group likely re-
duces the risk of bias relative to the use of an exter-
nal reference group (e.g., the healthy worker effect).
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
Medium x 0.4 0.8
x 0.2 0.6
High x 0.4 0.4
The exposure assessment was conducted by the Na-
tional Cancer Institute (NCI), using job-exposure
matrices, based on information provided by the Air
Force. Although exposure misclassification was pos-
sible (because individual exposure records were not
available), misclassification was likely random and
not to appreciably bias the results.
For 21 chemicals (including TCE, Perc, CC14 and
DCM), exposure was classified as yes/no. No quan-
titative assessment of exposure was conducted.
The study presents the appropriate relationship be-
tween exposure and outcome. Outcome was ascer-
tained after information on exposure was obtained.
There was a long follow-up period.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319
Data Type: Hill Air Force Base DCM Bronchitis Males-Respiratory
HERO ID: 699234
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization Medium x 0.667 1.33
Metric 8: Reporting Bias
.
x 0.333 0.33
The outcome was determined from death records
from the National Death Index (NDI). It was noted
in the study that mortality data can be mislead-
ing owing to inaccuracies captured in patient death
records.
A description of measured outcomes is provided in
the study report. Effects estimates are provided
with confidence limits; number of exposed cases is
included.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding

-------
. continued from previous page
Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319
Data Type:	Hill	Air	Force	Base	DCM	Bronchitis	Males-Respiratory
HERO ID:	699234
Domain
Metric
Rating^
MWF*
Score
Comments^t
Metric 15
Statistical models
m
x 0.2
0.4
The method and model assumptions used to cal-
culate risk estimates for occupational exposure to
TCE, Perc, CC14 and DCM and all-cause and cause-
specific mortality (hazard ratios) are clearly de-
scribed in the study report.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.8

Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
( 1	if any metric is Unacceptable
Overall rating = <
I	(Metric Score; X MWF;) / MWF,
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 16: Gold et al. 2010: Evaluation of Cancer Outcomes
Study Citation: Gold, LS; Stewart, PA; Milliken, K; Purdue, M; Severson, R; Seixas,
relationship between multiple myeloma and occupational exposure to six
68(6), 391-399
Data Type:	Gold	DCM	exposed workers	cancer	lOyrlag	1-7 CE score-Cancer
HERO ID:	699241
N; Blair, A; Hartge, P; Davis, S; De Roos, AJ (2010). The
chlorinated solvents Occupational and Environmental Medicine,
Domain
Metric
Rating^
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Medium
Metric 3: Comparison Group
High
High


X 0.4	0.8 Study authors note a low participation rate of eli-
gible controls, with individuals in the youngest (35-
50) and oldest (65-75) age groups were less likely to
participate than those in the middle age group.
X 0.4	0.4 Low attrition for subjects that decided to participate
in study. Only one case was excluded because of
missing covariate information.
X 0.2	0.2 General population controls were selected from a
case-control study of non-Hodgkin's lymphoma un-
dertaken at the same time. Controls were identified
by random digit dialing with clear inclusion criteria.
A table of characteristics was not provided to evalu-
ate similarities, but adjustments were made for age,
race, site, gender, and years of education.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
bow
Medium
High
x 0.4	1.2 Use of a job-exposure matrix in a population based
study. Exposure based on participant interview
rather than detailed employment history records.
X 0.2	0.4 Reports referent group and 3 levels of exposure for
cumulative exposure and 10-year lagged cumulative
exposure.
X 0.4	0.4 Cases were diagnosed between 2000 and 2002 while
exposure was assessed from 1941 to time of study
enrollment.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
Metric 8: Reporting Bias
High
X 0.667 0.67 Cases were identified through the review of hospi-
tal medical records and records of selected pathol-
ogy laboratories, oncologists, radiologists and state
death certificates.
X 0.333 0.33 Effect estimates are reported with a confidence inter-
val. The number of cases and controls are included
in a tabular format for date extraction and analysis.
Domain 4: Potential Counfounding/Variable Control
Continued on next page

-------
. continued from previous page
Study Citation: Gold, LS; Stewart, PA; Milliken, K; Purdue, M; Severson, R; Seixas, N; Blair, A; Hartge, P; Davis, S; De Roos, AJ (2010). The
relationship between multiple myeloma and occupational exposure to six chlorinated solvents Occupational and Environmental Medicine,
68(6), 391-399
Data Type: Gold DCM exposed workers cancer lOyrlag 1-7 CE score-Cancer
HERO ID: 699241
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 9:
Covariate Adjustment
High
X
0.5
0.5
Covariates gender, age (35-50 years (referent), 51-
64 years and 65-74 years), race (only white (refer-
ent), any black, any Asian and other), education
(less than 12 years (referent), 1'2-15 years and 16
or more years) and SEER site (Seattle and Detroit).
Metric 10:
Covariate Characterization
um
X
0.25
0.5
Potential confounders were considered but method
validation not provided. However there is no evi-
dence that the method had poor validity.
Metric 11:
Co-expoeure Confounding

X
0.25
0.75
Exposure to other chlorinated solvents was also as-
sessed with JEM. Study authors note that they re-
port the percentages of control subjects exposed to
these chemicals alone and to two of these chemicals
and provide an estimate of the association with mul-
tiple myeloma for subjects who were exposed to all
four (TCE, CC14, DCM, PERC). But analyses were
not adjusted for these exposures.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
The case-control study design chosen was appropri-
ate for the exposure and outcome of interest.
Metric 13:
Statistical power
Medium
X
0.2
0.4
The overall number of cases and controls are ad-
equate to detect an effect, but the number in the
subsets are small.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
The description of the analysis is sufficient to under-

stand what has been done.
Metric 15:
Statistical models
Medium
X
0.2
0.4
There is sufficient information on how the ORs were
calculated.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination*
ffigfe —
Medium"
1.6 Metric mean score: 1.63.
Extracted

Yes

Continued on next page . ..

-------
. continued from previous page
Study Citation: Gold, LS; Stewart, PA; Milliken, K; Purdue, M; Severson, R; Seixas, N; Blair, A; Hartge, P; Davis, S; De Roos, A J (2010). The
relationship between multiple myeloma and occupational exposure to six chlorinated solvents Occupational and Environmental Medicine,
68(6), 391-399
Data Type:	Gold_DCM_exposed workers_cancer_ 10yr 1 ag_ 1-7 CE score-Cancer
HERO ID:	699241
Domain	Metric	Rating^ MWF* Score	Comments^
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
(round to the nearest tenth) otherwise
0.1
]T\ (Metric Score,; x MWF,;) / J] . MWF;
*
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating,
ft This metric met the criteria for high confidence as expected for this type of study
§ Evaluator's explanation for rating change: "The number of exposed cases and controls in the different subgroups is small and results should be interpreted with caution."

-------
Table 17: Cocco et al. 1999: Evaluation of Cancer Outcomes
Study Citation: Cocco, P; Heineman, EF; Dosemeci, M (1999). Occupational risk factors for cancer of the central nervous system (CNS) among US
women American Journal of Industrial Medicine, 36(1), 70-74
Data Type:	Case-Control	Occupational	DCM	MeningiomaMortality	Dichotomous-Cancer
HERO ID:	730500
Domain
Metric
Rating*
MWF*
Score
Comments^
Domain 1: Study Participation





Metric 1:
Participant selection
Low
X 0.4
1.2
Identified cases of cancer of the brain and other parts
of the CNS among women who died in 24 states be-
tween 1984 — 1992 via occupation and industry listed


Medium


on death certificate.
Metric 2:
Attrition
X 0.4
0.8
No mention of subject withdrawal. Specific inclusion
criteria implemented into study design.
Metric 3:
Comparison Group
Low
X 0.2
0.6
For each case, four controls were selected among
women who died from nonmalignant diseases, ex-
cluding neurological disorders, frequency-matched
by state, race, and 5-year age groups.
Domain 2: Exposure Characterization
oo	Continued on next page . ..

-------
. continued from previous page
Study Citation: Cocco, P; Heineman, EF; Dosemeci, M (1999). Occupational risk factors for cancer of the central nervous system (CNS) among US
women American Journal of Industrial Medicine, 36(1), 70-74
Data Type: Case-Control Occupational DCM MeningiomaMortality Dichotomous-Cancer
HERO ID: 730500
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
Medium
Medium
m x 0.4 0.8 Job-exposure matrices for 11 occupational hazards
were designed. An estimate of intensity level of
exposure and probability of exposure to each haz-
ard was developed by two authors (M.D. and P.C.)
for each 3-di.gi.t occupation and each 3-di.gi.t indus-
try U.S. Census code. The final intensity score and
probability score was developed for each occupa-
tion/industry combination appearing in study sub-
jects' death certificates. The final probability and
intensity score was created by combining the occu-
pation and industry scores in the following ways: 1)
If both occupation and industry involved exposure
to hazard, then the final intensity score was equal to
the product of the individual intensity scores. The
final probability score was that attributed to the in-
dustry code alone. 2) If exposure was related only
to occupation, regardless of industry, only the in-
tensity and probability scores related to occupation
were used to derive the final scores. Intensity score
was squared in these instances to maintain consis-
tency in units. The final intensity and probabil-
ity scores were then grouped into four levels (un-
exposed, low, medium, and high). Low, medium, or
high probability and intensity of exposure are meant
as comparisons within a given exposure and are not
comparable across exposures.
Occupation and industry listed on the death certifi-
cate represent only a fraction of the work history
for each subject, either the "usual" or the last oc-
cupation. The 3-di.gi.t US Census code may have
not been specific, enough to accurately identify ex-
posures. Thus, there is potential for exposure mis-
classification that may have impaired the specificity
of the job-exposure matrix and weakened positive
associations.
X 0.2 0.4 The range of exposure is sufficient. Some analyses
used three levels of exposure, but some only included
exposed and unexposed
X 0.4 0.8 It is assumed that exposure occurred before outcome
but it is unclear whether exposures fall within rele-
vant exposure windows.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
x 0.667 0.67
Obtained through death certificates and records.
I CD-9 codes 192.1 and 192.3.
Continued on next page

-------
.. . continued from previous page
Study Citation: Cocco, P; Heineman, EF; Dosemeci, M (1999). Occupational risk factors for cancer of the central nervous system (CNS) among US
women American Journal of Industrial Medicine, 36(1), 70-74
Data Type: Case-Control Occupational DCM MeningiomaMortality Dichotomous-Cancer
HERO ID: 730500
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 8:
Reporting Bias
High
X
0.333
0.33
Diagnostic bias was likely to occur in death certifi-
cates in case-controls studies since mortality from all
causes combined is generally greater and reliability
of death certificate is poorer among low SES groups.
Low SES occupations might be underrepresented in
cases and overrepresented in controls. They con-
trolled for SES.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
High
X
0.5
0.5
Adjusted for marital status (never vs. ever mar-

ried), SES (based on Green's Standardized Score for
Specific, Occupations, age (continuous), design (fre-
quency matching) state, race, age and sex.
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
To account for the uncertainty to control for con-
founding or effect modification by lifestyle factors or
other occupational exposures with death certificates,
they adjusted for marital status and residence in the
analysis to reduce the effect of lifestyle factors. They
adjusted for SES on three levels, based on Green's
Standardized Score for Specific Occupations and age
at death.
Metric 11:
Co-exposure Confounding
Medium
X
0.25
0.5
Introduces new analysis that was better designed
for job-exposure matrices which was validated in
another study. No indication of unbalanced co-
exposures.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
Case control is an appropriate study design for the

research question; this study design is used to assess
the association between exposure and rare diseases.
Metric 13:
Statistical power
Medium
X
0.2
0.4
OR and 95% CI were calculated with logistic re-

gression for each workplace exposure adjusting for
confounders mentioned above. ORs and 95% CI
were calculated with Wald method using GMBO
program in the Epicure software package. 13 cases
and 3229 controls. Provided reasoning on how cate-
gories were created for exposure levels, why covari-
ates were used, and what statistical analyses v/ere
put into place to gather comparative results for the
analysis.
Continued on next page

-------
. continued from previous page
Study Citation: Cocco, P; Heineman, EF; Dosemeci, M (1999). Occupational risk factors for cancer of the central nervous system (CNS) among US
women American Journal of Industrial Medicine, 36(1), 70-74
Data Type:	Case-Control	Occupational	DCM	MeningiomaMortality	Dichotomous-Cancer
HERO ID:	730500
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 14: Reproducibility of analyses	m X 0.2	0.4 Study design and methods can be reproducible with
information provided. Provided reasoning on how
categories were created for exposure levels, why co-
variates v/ere used. Covariates included in the re-
gression models are reported explicitly.
Metric 15: Statistical models	im X 0.2	0.4 Job-exposure matrices for the 11 occupational haz-
ards (one being DCM). The categorization of expo-
sure probability and intensity levels in the newly
designed matrices resulted in greater sensitivity in
identifying exposures particularly in the low proba-
bility/ low intensity groups. The number of people
exposed in this study is greater than if they used
the older matrices. OR and 95% CI were calculated
with logistic regression for each workplace exposure
adjusting for confounders mentioned above. ORs
and 95% CI were calculated with Wald method using
GMBO program in the Epicure software package
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination
Medium

1.8
Extracted

Yes


MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score,; x MWF,;) / J] . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 18: Barry et al. 2011: Evaluation of Cancer Outcomes
Study Citation: Barry, KH; Zhang, Y; Lan, Q; Zahm, SH; Holford, TR; Leaderer, B; Boyle, P; Hosgood, HD; Chanock, S; Yeager, M; Rothman, N;
Zheng, T (2011). Genetic variation in metabolic genes, occupational solvent exposure, and risk of non-hodgkin lymphoma American
Journal of Epidemiology, 173(4), 404-413
Data Type: Barry DCM exposed workers NHL-Cancer
HERO ID: 730513
Domain
Metric
Rating1
MWF*
Score
Comments^
Domain 1: Study Participation

Metric 1:
Participant selection
High
X
0.4
0.4
Participation rates provided as well as eligibility cri-
Metric 2:
Attrition
High
X
0.4
0.4
Study is a reanalysis of a case control study that

included only participations with blood and or buc-

cal cell samples (additional analyses evaluated geno-

types). The subset of cases and controls with sam-

ples was similar (86 and 83%, respectively). No fur-

ther attrition occurred.
Metric 3:
Comparison Group
High
X
0.2
0.2
Controls were frequency-matched to cases, identified

through random digit dialing and random selection

from Centers for Medicare and Medicaid Services

records. It is unclear if the controls were recruited

from the same eligible population. No comparison

between the groups are provided other than the ap-

plication of frequency matching for age.
Domain 2: Exposure Characterization

Metric 4:
Measurement of Exposure
Low
X
0.4
1.2
A standardized structured questionnaire was used

to collect information for the construction of a job-

exposure matrix. Exposure was not directly mea-

sured and detailed employment records were not uti-

lized.
Metric 5:
Exposure levels
Low
X
0.2
0.6
Exposure was characterized as 'ever or 'never' ex-

posed' (2 levels of exposure)
Metric 6:
Temporality
Medium
X
0.4
0.8
Little information is provided on the establishment

of exposure prior to the ascertainment of the out-

come.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High x 0.667 0.67 Outcome assessed using well-established methods.
Histologically confirmed incident NHL.
Metric 8: Reporting Bias High X 0.333 0.33 Effect estimate is reported with a confidence interval
with the number of cases and controls that would
allow with data extraction.
Domain 4: Potential Counfounding/Variable Control
Continued on next page . ..

-------
.. . continued from previous page
Study Citation: Barry, KH; Zhang, Y; Lan, Q; Zahm, SH; Holford, TR; Leaderer, B; Boyle, P; Hosgood, HD; Chanock, S; Yeager, M; Rothman, N;
Zheng, T (2011). Genetic variation in metabolic genes, occupational solvent exposure, and risk of non-hodgkin lymphoma American
Journal of Epidemiology, 173(4), 404-413
Data Type:	Barry	DCM	exposed workers	NHL-Cancer
HERO ID:	730513
Domain
Metric

Rating^
MWF*
Score
Commentstt
Metric 9:
Covariate Adjustment

High
x 0.5
0.5
Adjusted for age (continuous) and race
(white/nonwhite). The addition of family history
of hematopoietic disorders, alcohol consumption,
tobacco smoking, education, annual family income,
and medical history of i.mmune-re 1 ated disease did
not appreciably alter effect estimates for solvent
associations with NHL outcomes, and thus these
covariates were not included in the final models
Metric 10
Covariate Characterization


x 0.25
0.25
No method validation mentioned but no evidence
that the method had poor validity.
Metric 11
Co-exposure Confounding


x 0.25
0.75
Analyses not adjusted for co-exposure to other or-
ganic solvents evaluated by JEM
Domain 5: Analysis






Metric 12
Study Design and Methods

Medium
x 0.4
0.8
The study design chosen was appropriate for the re-
search question and an appropriate statistical meth-
ods was used to address the research question.
Metric 13
Statistical power

Medium
x 0.2
0.4
The number of cases and controls were adequate to





detect an effect.
Metric 14
Reproducibility of analyses

Medium
x 0.2
0.4
The description of the analysis was sufficient to un-
derstand what was done.
Metric 15
Statistical models

Medium
x 0.2
0.4
The model for calculating the OR was transparent.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure


NA
NA

Metric 17
Effect biomarker


NA
NA

Metric 18
Method Sensitivity


NA
NA

Metric 19
Biomarker stability


NA
NA

Metric 20
Sample contamination


NA
NA

Metric 21
Method requirements


NA
NA

Metric 22
Matrix adjustment


NA
NA

Overall Quality Determination*

High

1.6

Extracted


Yes



Continued on next page

-------
. continued from previous page
Study Citation: Barry, KH; Zhang, Y; Lan, Q; Zahm, SH; Holford, TR; Leaderer, B; Boyle, P; Hosgood, HD; Chanock, S; Yeager, M; Rothman, N;
Zheng, T (2011). Genetic variation in metabolic genes, occupational solvent exposure, and risk of non-hodgkin lymphoma American
Journal of Epidemiology, 173(4), 404-413
Data Type:	Barry_DCM_exposed workers_NHL-Cancer
HERO ID:	730513
Domain	Metric	Rating^ MWF* Score	Comments^
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
]T\ (Metric Score,; x MWF,;) / J] . MWF;
*
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
(round to the nearest tenth) otherwise
0.1
crossed out and an arrow points to the new rating.
.
This metric met the criteria for high confidence as expected for this type of study

-------
Table 19: Bell et al. 1991: Evaluation of Growth (early life) and Development Outcomes
Study Citation: Bell, BP; Franks, P; Hildreth, N; Melius, J (1991). Methylene chloride exposure and birthweight in Monroe County, New York
Environmental Research, 55(1,1), 31-39
Data Type: DCM birth weight of children of exposed residents birth weight Low vs no exposure-Growth (early life) and Development
HERO ID: 730515
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
High x 0.4
High x 0.2
0.4 The study examined data available on birth certifi-
cates of individuals near the Eastman Kodak Com-
pany at Kodak Park in Rochester, Monroe County,
New York. They excluded multiple births and in-
fants weighing less than 750 grams. Because of
the few births among nonwhites in the areas of
higher exposure, the study was restricted to white
births. The study population included white single-
ton births weighing 750 g or more, born to mothers
residing in Monroe County in 1976-1987.
0.4 The study obtained and analyzed data included on
birth certificates from all years 1976-1987. The
study indicated that outcome data was complete, no
attrition.
0.2 Because of the known major differences in the dis-
tribution of birthweight and in the relationship of
risk factors to birthweight between whites and non-
whites, the two groups were not considered together.
The study was restricted to white births because
of the few births among nonwhites in the areas of
higher exposure. Women included in the analysis
were recruited from the same geographical area.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Low x 0.4
Metric 5: Exposure levels
dium x 0.2
1.2 Exposure was determined using the Kodak Air Man-
agement Program (KAMP) on air dispersion mod-
eling system, which predicts average annual ground
level concentrations of substances in the surround-
ing community. Details on the model v/ere minimal
in the present reference and did not indicate that it
had been validated.
0.4 The KAMP model was used to generate a map of the
air dispersion pattern of point and nonpoint sources
of DCM within Kodak Park, designating exposure of
50, 25, 10, and 2 ug/m"3 DCM in the community.
Using the map, the study reported four exposure
levels: high (50 ug/m), moderate (25 ug/m), low
(10 ug/m), and none.
Continued on next page

-------
.. . continued from previous page
Study Citation: Bell, BP; Franks, P; Hildreth, N; Melius, J (1991). Methylene chloride exposure and birthweight in Monroe County, New York
Environmental Research, 55(1,1), 31-39
Data Type: DCM birth weight of children of exposed residents birth weight Low vs no exposure-Growth (early life) and Development
HERO ID: 730515
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 6:
Temporality
m
X
0.4
0.8
Census tract of residence at the time of birth of the

infant, obtained from the birth certificate, was the
surrogate measure of exposure to DCM during preg-
nancy. Temporality between exposure and outcome
is established, but there is some remaining uncer-
tainty using a cross-sectional measure of exposure.
Study authors state they included an interaction
term for 4-year intervals and exposure as well as sea-
sons and exposure.
Domain 3: Outcome Assessment

Metric 7:
Outcome measurement or characterization High
X
0.667
0.67
Birth weight data were obtained from birth certifi-

cates for all births in Monroe County in 1976-1987.
This is a we 11-estab 1 ished method of obtaining birth-
weight data.
Metric 8:
Reporting Bias
High
X
0.333
0.33
The study reported regression coefficients and odds
ratio for low birthweight with confidence intervals.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
High
X
0.5
0.5
Multiple linear regression was used to examine the

association between birthweight and multiple risk
factors, such as maternal education, parity, previous
losses, maternal age, late care, male sex, and com-
plicated pregnancy. No information was available on
smoking.
Metric 10:
Covariate Characterization
High
X
0.25
0.25
Potential confounders such as maternal age, parity,
and maternal education were obtained and assessed
from data available on birth certificates. This is a

Medium

valid method of obtaining covariate information.
Metric 11:
Co-exposure Confounding
X
0.25
0.5
Anv co-exposure to pollutants with potential to bias
the results was not likely present because the study
only included residents in Monroe County near the
Eastman Kodak Company where DCM emissions oc-
curred.
Domain 5: Analysis

Metric 12:
Study Design and Methods
ivieaium
X
0.4
0.8
Cross-sectional study design was used to examine

the relationship between birthweight and exposure
to emissions of DCM. The study used t-tests, corre-
lation coefficients, AN OVA, and multiple linear re-
gression to analyze the association between birth-
weight and risk factors.
Metric 13:
Statistical power
Medium
X
0.2
0.4
The number of participants were adequate for the
analysis. The study included over 90,000 birth
records from 1976 to 1987 for analysis.
Continued on next page

-------
. continued from previous page
Study Citation: Bell, BP; Franks, P; Hildreth, N; Melius, J (1991). Methylene chloride exposure and birthweight in Monroe County, New York
Environmental Research, 55(1,1), 31-39
Data Type:	DCM_birth weight of children of exposed residents_birth weight_Low vs no exposure-Growth (early life) and Development
HERO ID:	730515
Domain
Metric
Rating^ MWF* Score
Comments^
Metric 14:
Reproducibility of analyses
m x 0.2
0.4
The description of analysis is sufficient to reproduce
the analysis. The study used the DCM isopleth map
generated by the KAMP model, county census tracts
were classified into four exposure categories, high
(50 ug/m), moderate (25 ug/m), low (lOug/m), and
none. Birthweight and risk factors were gathered
from birth certificates of residents living in Monroe
County.
Metric 15:
Statistical models
Medium x 0.2
0.4
The study used t-tests, correlation coefficients, and
A NOVA to examine the relationship of birthweight
to risk factors. Multiple linear regression was used
to investigate the association between birthweight
and multiple risk factors.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure


NA
NA
Metric 17
Effect biomarker


NA
NA
Metric 18
Method Sensitivity


NA
NA
Metric 19
Biomarker stability


NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements


NA
NA
Metric 22
Matrix adjustment


NA
NA
Overall Quality Determination*

High

1.5
Extracted


Yes


MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
y\ (Metric Scores x MWF; )/£ MWF:
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 20: Hearne and Pifer 1999: Evaluation of Cancer for Employees in Roll Coating Division Outcomes
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1964-1970 roll coating cohort dose-response analysis total cancer-Cancer
HERO ID: 730525
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
High
x 0.4
x 0.4
Metric 3: Comparison Group
Medium x 0.2
0.4 All key elements of the study design were reported
and there is a low risk for selection bias. The total
study population of the 1964-1970 roll coating co-
hort was 1013 men. Women were excluded because
of the small number employed in film support oper-
ations during those years.
0.4 There was minimal subject loss to follow up during
the study. Only one death certificate was unavail-
able for the decedents.
0.4 Two referent populations were used in the analyses:
the general population of New York State men (ex-
cluding NYC) and an occupational population of all
RoChester-based Kodak hourly wage men (exclud-
ing the roll coating division). The authors reported
that, "previous studies of Roll Coating men demon-
strated no unusual smoking patterns compared with
other employees or with the population at large."
No other information was provided to indicate if the
workers were similar to the referent population char-
acteristics. There was no adjustment for race in the
analyses. For calculation of SMRs, a computer pro-
gram based on person-years by age, sex, and calen-
dar period was used to calculate the number of ex-
pected deaths by cause. For dose-response analysis,
the authors conducted Poisson regression modeling
to estimate the effect of career exposure on cause-
specific mortality rates while adjusting for age, cal-
endar year, and time from first exposure.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4 0.4 Air sampling data v/ere integrated with detailed oc-
cupational history to develop an index of career ex-
posure for each individual for their entire work his-
tory. Air sampling methods are described in the
companion paper Hearne et al. 1987 (HERO ID
730524). The rate estimates v/ere adjusted for res-
piratory protection and were based on more than
1200 area samples and 1000 personal breathing zone
samples collected over 5 decades.
Continued on next page

-------
.. . continued from previous page
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1964-1970 roll coating cohort dose-response analysis total cancer-Cancer
HERO ID: 730525
Domain Metric
Rating^
MWF*
Score
Commentstt
Metric 5: Exposure levels
m
X
0.2
0.4
The range and distribution of exposure is sufficient
or adequate to develop an exposure-response esti-

High

mate. There were 4 exposure categories.
Metric 6: Temporality
X
0.4
0.4
The study population was followed from 24-30 years,
depending on the date of entry. The median time
from first exposure was -35 years, which was suffi-
cient for the development of cancer and other chronic
illnesses. The employees were exposed to methylene
chloride for about 24 years on average.
Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization Medium
X
0.667
1.33
The vital status of workers was ascertained from

the corporate human resources database, including

death certificates collected for processing of life in-
surance claims. The Social Security Administra-
tion's Death Master File was searched through 1994
to determine the vital status of terminated employ-
ees. The underlying causes of death were coded by a
nosologist according to ICD-8 (deaths through 1978)
or I CD-9 (deaths after 1978). Causes of death v/ere
not confirmed with medical records, but there was
no evidence of outcome mi.sclassi.fi. cat ion.
Metric 8: Reporting Bias
Medium
X
0.333
0.67
Effect estimates are reported with confidence inter-

vals in Table 5 which reports SMRs for the entire
cohort. Table 6 reports SMRs for different exposure
categories but does not include confidence intervals.
All results tables include number of observed and
expected deaths for each outcome.
Domain 4: Potential Counfounding/Variable Control
Medium

Metric 9: Covariate Adjustment
X
0.5
1
Adjustments are briefly described. The results were
age- and sex-adjusted, but not adjusted or stratified
by race.
Metric 10: Covariate Characterization
High
X
0.25
0.25
Sex and age were ascertained from work records.
Metric 11: Co-exposure Confounding
Low
X
0.25
0.75
There is direct evidence of co-exposures in some co-
hort members and the co-exposures were not ad-
dressed in the analysis. Approxi.mately one third
of the subjects in the roll coating cohort were em-
ployed in that division before the mid-1940s when
methylene chloride was introduced, as thus received
occupational exposure to other solvents, primarily
acetone and methanol.
Domain 5: Analysis
Continued on next page

Medium x 0.2
Medium x 0.2
Medium x 0.2
0.8 The study design chosen was appropriate for the re-
search question and appropriate statistical methods
were used to address the research question (a com-
puter program was used to calculate the number of
deaths expected by cause; Poisson probability dis-
tribution was used to test the statistical significance
and to calculate confidence intervals for the SMRs).
Exposure-response relationship was also evaluated
(tests for trend were conducted using X2 statis-
tics for both internally and externally standardized
rates; Poisson regression modelling was performed to
assess the relationship between cause-specific mor-
tality and career exposure, adjusting for age, calen-
dar year, and time from first exposure). The Soft-
ware used for calculations was EGRET which was
developed by the Statistics and Epidemiology Re-
search Corporation.
0.4 The number of participants is adequate to detect an
effect in the exposed population. There were a total
of 1013 subjects, and the total observational period
generated 26,251 person-years of follow-up.
0.4 The description of the analysis is sufficient to under-
stand precisely what has been done and to be con-
ceptually reproducible. SMRs were calculated using
the person-years method. Numbers of observed and
expected deaths were provided.
0.4 The method used for calculating SMRs is transpar-
ent. Poisson probability distribution was used to
test the statistical significance and to calculate con-
fidence intervals for the SMRs. Tests for trend were
conducted using X.2 statistics with P value as prob-
ability of observed results, given no trend. Poisson
regression modeling was used to assess the relation-
ship between cause-specific mortality and career ex-
posure.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Continued on next page . ..

-------
. continued from previous page
Study Citation:
Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed

to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type:
Occupational	DCM	1964-1970 roll coating cohort	dose-response analysis	total cancer-Cancer
HERO ID:
730525
Domain
Metric Rating^ MWF* Score Commentstt
Metric 22: Matrix adjustment NA NA
Overall Quality Determination* ivieuium 1.7
Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
(Metric Score; x MWF;) / J] . MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 21: Hearne and Pifer 1999: Evaluation of Cancer for All Employees Outcomes
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1946-1970 cohort liver and biliary-Cancer
HERO ID: 730525
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
High x 0.4
Medium x 0.2
0.4 All key elements of the study design were reported
and there is a low risk for selection bias. The total
study population of the 1946-1970 methylene chlo-
ride cohort was 1311 men. Women were excluded be-
cause of the small number employed in film support
operations during those years. The authors stated
that, "to address the issue of potential selectivity
bias, we included all individuals who v/ere hired by,
or transferred to, the Roll Coating Division between
1946 and 1970, including those who died, terminated
employment, or transferred to other departments be-
fore 1964 "
0.4 There was minimal subject loss to follow up during
the study. The follow-up rate was >99%. Death
certificates were unavailable for four decedents.
0.4 The referent population was New York state men
residing outside of NYC from 1945-1990. The au-
thors reported that, "previous studies of Roll Coat-
ing men demonstrated no unusual smoking patterns
compared with other employees or with the popu-
lation at large." No other information was provided
to indicate if the v/orkers were similar to the refer-
ent population characteristics. The study popula-
tion (all male) was described as "almost all white,"
but no other baseline characteristics were provided.
There was no adjustment for race in the analyses.
For calculation of SMRs, a computer program based
on person-years by age, sex, and calendar period
was used to calculate the number of expected deaths
by cause. For dose-response analysis, the authors
also used an internal comparison (expected num-
bers of death based on intra-cohort distribution of
person-years) in addition to the New York state ex-
ternal comparison and conducted Poisson regression
modeling to estimate the effect of career exposure
on cause-specific mortality rates while adjusting for
age, calendar year, and time from first exposure.
Domain 2: Exposure Characterization
Continued on next page

Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
The vital status of workers was ascertained from

the corporate human resources database, including
death certificates collected for processing of life in-
surance claims. The Social Security Administra-
tion's Death Master File was searched through 1994
to determine the vital status of terminated employ-
ees. The underlying causes of death were coded by a
nosologist according to ICD-8 (deaths through 1978)
or I CD-9 (deaths after 1978). Causes of death v/ere
not confirmed with medical records, but there was
no evidence of outcome misclassification.
Metric 8:
Reporting Bias
Medium
X
0.333
0.67
Effect estimates are reported with confidence inter-
vals in Table 2 which reports SMRs for the entire
cohort. Table 4 reports SMRs for different exposure
categories but does not include confidence intervals.
All results tables include number of observed and
expected deaths for each outcome.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
Medium
X
0.5
1
Adjustments are briefly described. The results were
age- and sex-adjusted, but not adjusted or stratified
by race.
Metric 10:
Covariate Characterization
High
X
0.25
0.25
Sex and age were ascertained from work records.
Continued on next page . ..

-------
. continued from previous page
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1946-1970 cohort liver and biliary-Cancer
HERO ID: 730525
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 11: Co-exposure Confounding
m X 0.25 0.5 Any co-exposure to pollutants that are not the tar-
get exposure that would likely bias the results were
not likely to be present. The authors stated that, "to
ensure that methylene chloride was the cohort's pri-
mary solvent exposure, we selected employees who
were hired after the Roll Coating Division began us-
ing this material in the mid-1940s." Prior to the mid-
19405, acetone and methanol were the major solvents
used in film support manufac,turi.ng.
Domain 5: Analysis
Metric 12: Study Design and Methods
Medium x 0.4
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Medium x 0.2 0.4
m x 0.2
0.8 The study design chosen was appropriate for the re-
search question and appropriate statistical methods
were used to address the research question (a com-
puter program was used to calculate the number of
deaths expected by cause; Poisson probability dis-
tribution was used to test the statistical significance
and to calculate confidence intervals for the SMRs).
Exposure-response relationship was also evaluated
(tests for trend were conducted using X2 statis-
tics for both internally and externally standardized
rates; Poisson regression modelling was performed to
assess the relationship between cause-specific mor-
tality and career exposure, adjusting for age, calen-
dar year, and time from first exposure). The Soft-
ware used for calculations was EGRET which devel-
oped by the Statistics and Epidemiology Research
Corporation.
The number of participants is adequate to detect
an effect in the exposed population. There were a
total of 1311 subjects with over 200 people in each
quartile of exposure. The total observational period
generated 46,112 person-years of follow-up.
0.4 The description of the analysis is sufficient to under-
stand precisely what has been done and to be con-
ceptually reproducible. SMRs were calculated using
the person-years method. Numbers of observed and
expected deaths were provided.
Continued on next page . ..

-------
. continued from previous page
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type:	Occupational	DCM	1946-1970 cohort	liver and biliary-Cancer
HERO ID:	730525
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 15: Statistical models
m X 0.2	0.4 The method used for calculating SMRs is transpar-
ent. Poisson probability distribution was used to
test the statistical significance and to calculate con-
fidence intervals for the SMRs. Tests for trend were
conducted using X.2 statistics with P value as prob-
ability of observed results, given no trend. Poisson
regression modeling was used to assess the relation-
ship between cause-specific mortality and career ex-
posure.
Domain 6: Other Considerations for Biomarker Selection and Measu
Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination
High

1.6
Extracted

Yes


*	MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
*	The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =

]T\ (Metric Score; x MWF;) / J] . MWF
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 22: Hearne and Pifer 1999: Evaluation of Respiratory Outcomes
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1964-1970 roll coating cohort allrespiratorydiseases-Respiratory
HERO ID: 730525
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
High
x 0.4
x 0.4
Metric 3: Comparison Group
Medium x 0.2
0.4 All key elements of the study design were reported
and there is a low risk for selection bias. The total
study population of the 1964-1970 roll coating co-
hort was 1013 men. Women were excluded because
of the small number employed in film support oper-
ations during those years.
0.4 There was minimal subject loss to follow up during
the study. Only one death certificate was unavail-
able for the decedents.
0.4 Two referent populations were used in the analyses:
the general population of New York State men (ex-
cluding NYC) and an occupational population of all
RoChester-based Kodak hourly wage men (exclud-
ing the roll coating division). The authors reported
that, "previous studies of Roll Coating men demon-
strated no unusual smoking patterns compared with
other employees or with the population at large."
No other information was provided to indicate if the
workers were similar to the referent population char-
acteristics. There was no adjustment for race in the
analyses. For calculation of SMRs, a computer pro-
gram based on person-years by age, sex, and calen-
dar period was used to calculate the number of ex-
pected deaths by cause. For dose-response analysis,
the authors conducted Poisson regression modeling
to estimate the effect of career exposure on cause-
specific mortality rates while adjusting for age, cal-
endar year, and time from first exposure.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4 0.4 Air sampling data v/ere integrated with detailed oc-
cupational history to develop an index of career ex-
posure for each individual for their entire work his-
tory Air sampling methods are described in the
companion paper Hearne et al. 1987 (HERO ID
730524). The rate estimates v/ere adjusted for res-
piratory protection and were based on more than
1200 area samples and 1000 personal breathing zone
samples collected over 5 decades.
Continued on next page

-------
.. . continued from previous page
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1964-1970 roll coating cohort allrespiratorydiseases-Respiratory
HERO ID: 730525
Domain Metric
Rating^
MWF*
Score
Commentstt
Metric 5: Exposure levels
m
X
0.2
0.4
The range and distribution of exposure is sufficient
or adequate to develop an exposure-response esti-

High

Metric 7: Outcome measurement or characterization Medium
X
0.667
1.33
The vital status of workers was ascertained from

the corporate human resources database, including

Medium x 0.2
Medium x 0.2
Medium x 0.2
0.8 The study design chosen was appropriate for the re-
search question and appropriate statistical methods
were used to address the research question (a com-
puter program was used to calculate the number of
deaths expected by cause; Poisson probability dis-
tribution was used to test the statistical significance
and to calculate confidence intervals for the SMRs).
Ex pos u r e- r es po nse relationship was also evaluated
(tests for trend were conducted using X2 statis-
tics for both internally and externally standardized
rates; Poisson regression modelling was performed to
assess the relationship between cause-specific mor-
tality and career exposure, adjusting for age, calen-
dar year, and time from first exposure). The Soft-
ware used for calculations was EGRET which was
developed by the Statistics and Epidemiology Re-
search Corporation.
0.4 The number of participants is adequate to detect an
effect in the exposed population. There were a total
of 1013 subjects, and the total observational period
generated 26,251 person-years of follow-up.
0.4 The description of the analysis is sufficient to under-
stand precisely what has been done and to be con-
ceptually reproducible. SMRs were calculated using
the person-years method. Numbers of observed and
expected deaths were provided.
0.4 The method used for calculating SMRs is transpar-
ent. Poisson probability distribution was used to
test the statistical significance and to calculate con-
fidence intervals for the SMRs. Tests for trend were
conducted using X.2 statistics with P value as prob-
ability of observed results, given no trend. Poisson
regression modeling was used to assess the relation-
ship between cause-specific mortality and career ex-
posure.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Continued on next page . ..

-------
. continued from previous page
Study Citation:
Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed

to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type:
Occupational	DCM	1964-1970 roll coating cohort	allrespiratorydiseases-Respiratory
HERO ID:
730525
Domain
Metric Rating^ MWF* Score Commentstt
Metric 22: Matrix adjustment NA NA
Overall Quality Determination* ivieuium 1.7
Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
(Metric Score; x MWF;) / J] . MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 23: Hearne and Pifer 1999: Evaluation of Hematological and Immune Outcomes
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1964-1970 roll coating cohort infection mortality-Hematological and Immune
HERO ID: 730525
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
High
x 0.4
x 0.4
Metric 3: Comparison Group
Medium x 0.2
0.4 All key elements of the study design were reported
and there is a low risk for selection bias. The total
study population of the 1964-1970 roll coating co-
hort was 1013 men. Women were excluded because
of the small number employed in film support oper-
ations during those years.
0.4 There was minimal subject loss to follow up during
the study. Only one death certificate was unavail-
able for the decedents.
0.4 Two referent populations were used in the analyses:
the general population of New York State men (ex-
cluding NYC) and an occupational population of all
RoChester-based Kodak hourly wage men (exclud-
ing the roll coating division). The authors reported
that, "previous studies of Roll Coating men demon-
strated no unusual smoking patterns compared with
other employees or with the population at large."
No other information was provided to indicate if the
workers were similar to the referent population char-
acteristics. There was no adjustment for race in the
analyses. For calculation of SMRs, a computer pro-
gram based on person-years by age, sex, and calen-
dar period was used to calculate the number of ex-
pected deaths by cause. For dose-response analysis,
the authors conducted Poisson regression modeling
to estimate the effect of career exposure on cause-
specific mortality rates while adjusting for age, cal-
endar year, and time from first exposure.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4 0.4 Air sampling data v/ere integrated with detailed oc-
cupational history to develop an index of career ex-
posure for each individual for their entire work his-
tory Air sampling methods are described in the
companion paper Hearne et al. 1987 (HERO ID
730524). The rate estimates v/ere adjusted for res-
piratory protection and were based on more than
1200 area samples and 1000 personal breathing zone
samples collected over 5 decades.
Continued on next page

-------
.. . continued from previous page
Study Citation: Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed
to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type: Occupational DCM 1964-1970 roll coating cohort infection mortality-Hematological and Immune
HERO ID: 730525
Domain Metric
Rating^
MWF*
Score
Commentstt
Metric 5: Exposure levels
m
X
0.2
0.4
The range and distribution of exposure is sufficient
or adequate to develop an exposure-response esti-

High

Metric 7: Outcome measurement or characterization Medium
X
0.667
1.33
The vital status of workers was ascertained from

the corporate human resources database, including

-------
. continued from previous page
Study Citation:
Hearne, FT; Pifer, JW (1999). Mortality study of two overlapping cohorts of photographic film base manufacturing employees exposed

to methylene chloride Journal of Occupational and Environmental Medicine, 41(12), 1154-1169
Data Type:
Occupational	DCM	1964-1970 roll coating cohort	infection	mortality-Hematological and Immune
HERO ID:
730525
Domain
Metric Rating^ MWF* Score Commentstt
Metric 22: Matrix adjustment NA NA
Overall Quality Determination* ivieuium 1.7
Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
(Metric Score; x MWF;) / J] . MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 24: Gibbs et al. 1996: Evaluation of Cancer Outcomes
Study Citation: Gibbs, GW; Amsel, J; Soden, K (1996). A cohort mortality study of cellulose triacetate-fiber workers exposed to methylene chloride
Journal of Occupational and Environmental Medicine, 38(7,7), 693-697
Data Type: DCM exposed workers Mortality Prostate Cancer No Exposed-Cancer
HERO ID: 730533
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
X 0.4 0.4 All workers were included in the study, all key el-
ements were included. No loss was reported "The
workers included in the A nice lie cohort comprised
all individuals who were on the payroll on or after
January 1, 1970 and who had worked at the plant
for 3 or more months. The cohort consisted of 3211
white employees (2187 men and 1024 women)." The
plant and production process were described in de-
tail, including history of use and production.
0.4 No indication of loss from the 3211 initial partici-
pants. Outcome and covariate data were complete.
X 0.2 0.2 There were male and female workers in all exposure
groups, all where white. The study authors state
they present SMRs using Allegany County, MD as a
reference group as these are "preferred because they,
in effect, adjust for social, economic, ethnic, and cul-
tural factors related to disease."
mgn x 0.4
High
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Medium x 0.4
um x 0.2
Metric 6: Temporality
Medium x 0.4
0.8 Exposures were measured as reported in Ott et al.
1983 (HERO ID 29149). They describe personal
sampling and area sampling, but the exact method
and number of samples at the Amcelle plant is un-
clear. This is a direct exposure measurement, but
the method is not entirely clear.
0.4 Exposure monitoring data, first reported by Ott et
al 4 for the Celriver plant, were used to establish
high and low exposure ranges, which were 350 to
700 ppm and 50 to 100 ppm, respectively. Because
there were operations that did not involve methy-
lene chloride exposure, a "0" exposure category was
created as an internal control. The
distribution of workers according to exposure is
shown in Table 1.
0.8 Cause of death was determined and compared to ear-
lier exposures. Workers were followed for 9 years af-
ter the closing of the plant. This sufficiently estab-
lishes temporality between exposure and outcome.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW; Amsel, J; Soden, K (1996). A cohort mortality study of cellulose triacetate-fiber workers exposed to methylene chloride
Journal of Occupational and Environmental Medicine, 38(7,7), 693-697
Data Type: DCM exposed workers Mortality Prostate Cancer No Exposed-Cancer
HERO ID: 730533
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
High X 0.667 0.67 "Vital status was determined as of December 31,
1989. The follow-up included searching company
personnel records and pension files for persons still
living. Often, plant records could be used for iden-
tifying former employees who had died. In addi-
tion, the National Death Index was searched and
Social Security Death Master Files were examined
... causes of death were determined from death cer-
tificates and coded to the ninth revision of the In-
ternational Classification of Diseases by a qualified
nosologist." This is a well-established method of de-
termining mortality and cause of death.
X 0.333 0.33 All measured outcomes were reported. The study
authors state they published SMRs using the local
population as a reference only. This was thought to
be the most representative comparison.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
High
Medium
Metric 11: Co-exposure Confounding

Low
X 0.5 0.5 Covariates were assessed in the analysis. "The ra-
tio of observed to expected deaths in each 5-vear
interval from 1970 through 1989 was determined for
62 causes of death, and standardized mortality ra-
tios (SMRs) were calculated and controlled for age,
race, gender, and calendar period."
X 0.25 0.5 Covariate characterizati.on was not explicitly dis-
cussed. It is assumed age, race, and gender were
obtained from A nice lie employment records.
X 0.25 0.75 Exposures to acetone and finishing oils were present
and may have varied by task. There is no indication
that co-exposures were accounted for. More details
can be found in HERO ID 29149.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses

Medium X 0.4 0.8 This study looks at an causes of death in an occupa-
tional cohort with approximately 9 years of follow-
up. This is appropriate for the research question.
Medium X 0.2 0.4 There were over 3000 employees in this cohort. This
is sufficient to see an effect in the exposed popula-
tion.
Medium X 0.2 0.4 Categorization of exposure levels was adequately de-
scribed. Other details on the analysis v/ere included
so that the work could be reproduced.
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW; Amsel, J; Soden, K (1996). A cohort mortality study of cellulose triacetate-fiber workers exposed to methylene chloride
Journal of Occupational and Environmental Medicine, 38(7,7), 693-697
Data Type:	DCM	exposed workers	Mortality	Prostate Cancer	No Exposed-Cancer
HERO ID:	730533
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 15: Statistical models	m X 0.2	0.4 "The ratio of observed to expected deaths in each
5-year interval from 1970 through 1989 was deter-
mined for 62 causes of death, and standardized mor-
tality ratios (SMRs) were
calculated and controlled for age, race, gender, and
calendar period. Statistical analyses were done us-
ing OCMAP." The choice of a reference population
was adequately described.
Domain 6: Other Considerations for Biomarker Selection and Mea.su
Metric 16
Use of Biomarker of Exposure


NA
NA
Metric 17
Effect biomarker


NA
NA
Metric 18
Method Sensitivity


NA
NA
Metric 19
Biomarker stability


NA
NA
Metric 20
Sample contamination


NA
NA
Metric 21
Method requirements


NA
NA
Metric 22
Matrix adjustment


NA
NA
Overall Quality Determination*

High

1.6
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score,; x MWF,;) / . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 25: Lanes et al. 1990: Evaluation of Mortality Outcomes
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990).
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR all cause-Mortality
HERO ID:	730554
Mortality of cellulose fiber production workers Scandinavian
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
.
Metric 2: Attrition
Metric 3: Comparison Group

Medium
X 0.4	0.4 Study setting and participants are described as an
occupational cohort of 1271 employees, assembled in
1977. Employees worked for at least three months
between 1954 and 1977 in areas identified as having
methylene chloride exposure from an IH survey. De-
mographic details on the cohort are provided in-text.
Information regarding participation rate is provided
in companion publication (Ott et al. 1983).
X 0.4	0.4 Death certificates were obtained for 118/122 deaths.
The authors note that use of "the national death
index and the records of the Social Security Admin-
istration may fail to ascertain mortality by approx-
imately 10-20%"
X 0.2	0.4 Race and sex of the cohort are stratified in a table.
The chosen reference population was York County,
SC, the county in which 95% of the cohort resided
but only constituted <4% of the county population.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
High
x 0.2
x 0.4
0.8 The cohort was comprised of employees that v/orked
specifically in the two areas with concern for DCM
exposure—the preparation and extrusion areas. De-
tailed work histories v/ere only available for a small
subset of this cohort. An IH survey in 1977 reported
a ti.me-wei.ghted average for these two areas and it is
assumed that exposure was constantly present prior
to this survey.
0.6 As an SMR study, there are presumed to be two
levels of exposure. Those in the cohort are exposed
to DCM in the preparation and extrusion areas while
the reference population is unexposed.
0.4 Mortality was assessed for a follow-up period af-
ter employment at this facility of approximately 10
years (cohort formed in Jan 1977, follow-up until Sep
1986). Length of employment was assessed in select
cancer-related mortality outcomes.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990).
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR all cause-Mortality
HERO ID:	730554
Mortality of cellulose fiber production workers Scandinavian
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
Medium x 0.667 1.33
x 0.333 0.33
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
Mortality was assessed by searching the national
death index and records from the Social Security
Administration. This is not a gold standard method
and the authors note there may be a 10-20% margin
of error when assessing mortality by Social Security
Ad m i n is t r at io n records.
Mortality outcomes with a difference in observed vs
expected of more than one were included in the re-
sults. SMRs were presented with confidence inter-
vals in an easily read table.
x 0.5
Medium x 0.25
Low	x 0.25
0.5 Race, age, sex, and calendar period were all consid-
ered in the SMR calculation. There is also indirect
evidence to suggest the demographic distribution in
the sample population is similar to that of the ref-
erence population. Employees in the cohort were re-
ported to have v/orked in the preparation and extru-
sion areas; it is unclear whether there would be any
differential distribution of SES status in this sample
(i.e., managers vs non-managers).
0.5 Covariates such as age, sex, and race were obtained
through employment records. There is no evidence
to suggest this method has poor validity.
0.75 The industrial hygiene survey conducted in 1977
revealed 8-hour TWAs for three chemicals present
in these two areas of the textile manufac,turi.ng
plant. There were detectable concentrations of DCM
(1700 ppm), acetone (1600 ppm), and methanol (140
ppm). This indicates the presence of co-exposure,
but the distribution of this exposure among the co-
hort is unknown.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Medium
x 0.4
Medium x 0.2
0.8 The design of this study was appropriate for the
question of association between DCM and excess
mortality.
0.4 The occupational cohort contained over 1000 em-
ployees and was sufficiently large to detect an effect
of DCM.
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR all cause-Mortality
HERO ID:	730554
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 14
Reproducibility of analyses
m
x 0.2
0.4
Mortality among the cohort was compared to that
of the York County, SC population to generate
standardized mortality ratios. Reference population
death rates from 1962 were used for non-cancer out-
comes, as these rates were unavailable for the refer-
ence population for 1954-1961.
Metric 15
Statistical models
Medium
x 0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.7

Extracted

Yes



MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
(Metric Score; x MWF;) / J] . MWF?
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 26: Lanes et al. 1990: Evaluation of Respiratory Outcomes
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type: DCM Rock Hill occupational cohort SMR respiratory disease-Respiratory
HERO ID: 730554
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
.
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group

x 0.4
Medium x 0.2
0.4 Study setting and participants are described as an
occupational cohort of 1271 employees, assembled in
1977. Employees worked for at least three months
between 1954 and 1977 in areas identified as having
methylene chloride exposure from an IH survey. De-
mographic details on the cohort are provided in-text.
Information regarding participation rate is provided
in companion publication (Ott et al. 1983).
0.4 Death certificates were obtained for 118/122 deaths.
The authors note that use of "the national death
index and the records of the Social Security Admin-
istration may fail to ascertain mortality by approx-
imately 10-20%"
0.4 Race and sex of the cohort are stratified in a table.
The chosen reference population was York County,
SC, the county in which 95% of the cohort resided
but only constituted <4% of the county population.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
x 0.2
High x 0.4
0.8 The cohort was comprised of employees that v/orked
specifically in the two areas with concern for DCM
exposure—the preparation and extrusion areas. De-
tailed work histories v/ere only available for a small
subset of this cohort. An IH survey in 1977 reported
a ti.me-wei.ghted average for these two areas and it is
assumed that exposure was constantly present prior
to this survey.
0.6 As an SMR study, there are presumed to be two
levels of exposure. Those in the cohort are exposed
to DCM in the preparation and extrusion areas while
the reference population is unexposed.
0.4 Mortality was assessed for a follow-up period af-
ter employment at this facility of approximately 10
years (cohort formed in Jan 1977, follow-up until Sep
1986). Length of employment was assessed in select
cancer-related mortality outcomes.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type: DCM Rock Hill occupational cohort SMR respiratory disease-Respiratory
HERO ID: 730554
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
Medium x 0.667 1.33
x 0.333 0.33
Mortality was assessed by searching the national
death index and records from the Social Security
Administration. This is not a gold standard method
and the authors note there may be a 10-20% margin
of error when assessing mortality by Social Security
Administration records.
Mortality outcomes with a difference in observed vs
expected of more than one were included in the re-
sults. SMRs were presented with confidence inter-
vals in an easily read table.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
x 0.5
Medium x 0.25
Low x 0.25
0.5 Race, age, sex, and calendar period were all consid-
ered in the SMR calculation. There is also indirect
evidence to suggest the demographic distribution in
the sample population is similar to that of the ref-
erence population. Employees in the cohort were re-
ported to have v/orked in the preparation and extru-
sion areas; it is unclear whether there would be any
differential distribution of SES status in this sample
(i.e., managers vs non-managers).
0.5 Covariates such as age, sex, and race were obtained
through employment records. There is no evidence
to suggest this method has poor validity.
0.75 The industrial hygiene survey conducted in 1977
revealed 8-hour TWAs for three chemicals present
in these two areas of the textile manufac,turi.ng
plant. There were detectable concentrations of DCM
(1700 ppm), acetone (1600 ppm), and methanol (140
ppm). This indicates the presence of co-exposure,
but the distribution of this exposure among the co-
hort is unknown.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Medium
x 0.4
Medium x 0.2
0.8 The design of this study was appropriate for the
question of association between DCM and excess
mortality.
0.4 The occupational cohort contained over 1000 em-
ployees and was sufficiently large to detect an ef-
fect of DCM. The effect estimate is based on a small
number of cases.
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR respiratory disease-Respiratory
HERO ID:	730554
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 14
Reproducibility of analyses
m
x 0.2
0.4
Mortality among the cohort was compared to that
of the York County, SC population to generate
standardized mortality ratios. Reference population
death rates from 1962 were used for non-cancer out-
comes, as these rates were unavailable for the refer-
ence population for 1954-1961.
Metric 15
Statistical models
Medium
x 0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.7

Extracted

Yes



MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
(Metric Score; x MWF;) / J] . MWF?
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 27: Lanes et al. 1990: Evaluation of Cancer Outcomes
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990).
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR lung cancer-Cancer
HERO ID:	730554
Mortality of cellulose fiber production workers Scandinavian
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
.
Metric 2: Attrition
Metric 3: Comparison Group

Medium
X 0.4	0.4 Study setting and participants are described as an
occupational cohort of 1271 employees, assembled in
1977. Employees worked for at least three months
between 1954 and 1977 in areas identified as having
methylene chloride exposure from an IH survey. De-
mographic details on the cohort are provided in-text.
Information regarding participation rate is provided
in companion publication (Ott et al. 1983).
X 0.4	0.4 Death certificates were obtained for 118/122 deaths.
The authors note that use of "the national death
index and the records of the Social Security Admin-
istration may fail to ascertain mortality by approx-
imately 10-20%"
X 0.2	0.4 Race and sex of the cohort are stratified in a table.
The chosen reference population was York County,
SC, the county in which 95% of the cohort resided
but only constituted <4% of the county population.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
High
x 0.2
x 0.4
0.8 The cohort was comprised of employees that v/orked
specifically in the two areas with concern for DCM
exposure—the preparation and extrusion areas. De-
tailed work histories v/ere only available for a small
subset of this cohort. An IH survey in 1977 reported
a ti.me-wei.ghted average for these two areas and it is
assumed that exposure was constantly present prior
to this survey.
0.6 As an SMR study, there are presumed to be two
levels of exposure. Those in the cohort are exposed
to DCM in the preparation and extrusion areas while
the reference population is unexposed.
0.4 Mortality was assessed for a follow-up period af-
ter employment at this facility of approximately 10
years (cohort formed in Jan 1977, follow-up until Sep
1986). Length of employment was assessed in select
cancer-related mortality outcomes.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990).
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR lung cancer-Cancer
HERO ID:	730554
Mortality of cellulose fiber production workers Scandinavian
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
Medium x 0.667 1.33
x 0.333 0.33
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
Mortality was assessed by searching the national
death index and records from the Social Security
Administration. This is not a gold standard method
and the authors note there may be a 10-20% margin
of error when assessing mortality by Social Security
Ad m i n is t r at io n records.
Mortality outcomes with a difference in observed vs
expected of more than one were included in the re-
sults. SMRs were presented with confidence inter-
vals in an easily read table.
x 0.5
Medium x 0.25
Low	x 0.25
0.5 Race, age, sex, and calendar period were all consid-
ered in the SMR calculation. There is also indirect
evidence to suggest the demographic distribution in
the sample population is similar to that of the ref-
erence population. Employees in the cohort were re-
ported to have v/orked in the preparation and extru-
sion areas; it is unclear whether there would be any
differential distribution of SES status in this sample
(i.e., managers vs non-managers).
0.5 Covariates such as age, sex, and race were obtained
through employment records. There is no evidence
to suggest this method has poor validity.
0.75 The industrial hygiene survey conducted in 1977
revealed 8-hour TWAs for three chemicals present
in these two areas of the textile manufac,turi.ng
plant. There were detectable concentrations of DCM
(1700 ppm), acetone (1600 ppm), and methanol (140
ppm). This indicates the presence of co-exposure,
but the distribution of this exposure among the co-
hort is unknown.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Medium
x 0.4
Medium x 0.2
0.8 The design of this study was appropriate for the
question of association between DCM and excess
mortality.
0.4 The occupational cohort contained over 1000 em-
ployees and was sufficiently large to detect an effect
of DCM.
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR lung cancer-Cancer
HERO ID:	730554
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 14
Reproducibility of analyses
m
x 0.2
0.4
Mortality among the cohort was compared to that
of the York County, SC population to generate
standardized mortality ratios. Reference population
death rates from 1962 were used for non-cancer out-
comes, as these rates were unavailable for the refer-
ence population for 1954-1961.
Metric 15
Statistical models
Medium
x 0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.7

Extracted

Yes



MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
(Metric Score; x MWF;) / J] . MWF?
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 28: Lanes et al. 1990: Evaluation of Cardiovascular Outcomes
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type: DCM Rock Hill occupational cohort SMR cerebrovascular disease-Cardiovascular
HERO ID: 730554
Domain
Metric
Rating*
MWF*
Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
.
X 0.4 0.4 Study setting and participants are described as an
occupational cohort of 1271 employees, assembled in
1977. Employees worked for at least three months
between 1954 and 1977 in areas identified as having
methylene chloride exposure from an IH survey. De-
mographic details on the cohort are provided in-text.
Information regarding participation rate is provided
in companion publication (Ott et al. 1983).
X 0.4 0.4 Death certificates were obtained for 118/122 deaths.
The authors note that use of "the national death
index and the records of the Social Security Admin-
istration may fail to ascertain mortality by approx-
imately 10-20%"
um X 0.2 0.4 Race and sex of the cohort are stratified in a table.
The chosen reference population was York County,
SC, the county in which 95% of the cohort resided
but only constituted <4% of the county population.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4

Metric 5: Exposure levels
Metric 6: Temporality
x 0.2
High x 0.4
0.8 The cohort was comprised of employees that v/orked
specifically in the two areas with concern for DCM
exposure—the preparation and extrusion areas. De-
tailed work histories v/ere only available for a small
subset of this cohort. An IH survey in 1977 reported
a ti.me-wei.ghted average for these two areas and it is
assumed that exposure was constantly present prior
to this survey.
0.6 As an SMR study, there are presumed to be two
levels of exposure. Those in the cohort are exposed
to DCM in the preparation and extrusion areas while
the reference population is unexposed.
0.4 Mortality was assessed for a follow-up period af-
ter employment at this facility of approximately 10
years (cohort formed in Jan 1977, follow-up until Sep
1986). Length of employment was assessed in select
cancer-related mortality outcomes.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type: DCM Rock Hill occupational cohort SMR cerebrovascular disease-Cardiovascular
HERO ID: 730554
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
Medium x 0.667 1.33
x 0.333 0.33
Mortality was assessed by searching the national
death index and records from the Social Security
Administration. This is not a gold standard method
and the authors note there may be a 10-20% margin
of error when assessing mortality by Social Security
Administration records.
Mortality outcomes with a difference in observed vs
expected of more than one were included in the re-
sults. SMRs were presented with confidence inter-
vals in an easily read table.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
x 0.5
Medium x 0.25
Low x 0.25
0.5 Race, age, sex, and calendar period were all consid-
ered in the SMR calculation. There is also indirect
evidence to suggest the demographic distribution in
the sample population is similar to that of the ref-
erence population. Employees in the cohort were re-
ported to have v/orked in the preparation and extru-
sion areas; it is unclear whether there would be any
differential distribution of SES status in this sample
(i.e., managers vs non-managers).
0.5 Covariates such as age, sex, and race were obtained
through employment records. There is no evidence
to suggest this method has poor validity.
0.75 The industrial hygiene survey conducted in 1977
revealed 8-hour TWAs for three chemicals present
in these two areas of the textile manufac,turi.ng
plant. There were detectable concentrations of DCM
(1700 ppm), acetone (1600 ppm), and methanol (140
ppm). This indicates the presence of co-exposure,
but the distribution of this exposure among the co-
hort is unknown.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Medium
x 0.4
Medium x 0.2
0.8 The design of this study was appropriate for the
question of association between DCM and excess
mortality.
0.4 The occupational cohort contained over 1000 em-
ployees and was sufficiently large to detect an effect
of DCM. The effect estimate is based on a small
number of observed cases thus caution should be
taken in interpreting the SMR.
Continued on next page

-------
. continued from previous page
Study Citation: Lanes, SF; Cohen, A; Rothman, KJ; Dreyer, NA; Soden, KJ (1990). Mortality of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 16(4), 247-251
Data Type:	DCM Rock Hill occupational cohort SMR cerebrovascular disease-Cardiovascular
HERO ID:	730554
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 14
Reproducibility of analyses
m
x 0.2
0.4
Mortality among the cohort was compared to that
of the York County, SC population to generate
standardized mortality ratios. Reference population
death rates from 1962 were used for non-cancer out-
comes, as these rates were unavailable for the refer-
ence population for 1954-1961.
Metric 15
Statistical models
Medium
x 0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Medium

1.7

Extracted

Yes



MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
(Metric Score; x MWF;) / J] . MWF?
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 29: Lanes et al. 1993: Evaluation of Respiratory Outcomes
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM nonmalignant respiratory disease mortality-Respiratory
HERO ID: 730555
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
mgn x 0.4
Medium x 0.2

0.4 Cohort was assembled in 1977 for the purpose of
investigating potential health effects of exposure to
DCM. It included all 1271 workers employed in the
preparation and extrusion areas of the plant for at
least 3 months between January 1, 1954 and Jan-
uary 1, 1977. Demographic details on the cohort are
provided in-text. Information regarding participa-
tion rate is provided in companion publication (Ott
et al. 1983).
0.4 There does not appear to be any attrition.
0.4 Adjustment or stratification are not specifically de-
scribed. SMRs v/ere calculated using the local popu-
lation of York County, South Carolina controlled for
age, race, gender, and calendar period.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
0.8 Workers included worked in specific areas where
DCM exposure v/ould have occurred. Industrial
monitoring in 1977 revealed 8-h time weighted aver-
age concentrations of below detection to 1700 ppm
with median levels in the three areas of 140, 280, and
475 ppm. Respirators were not used until 1984 so ex-
posure is likely, although levels of exposure v/ere not
determined. Detailed work history was only avail-
able for 356 active employees and 119 employees who
terminated employment after 1979.
0.6 Exposure was only assessed as exposed in the occu-
pational cohort compared to unexposed in the local
population.
0.8 Temporality is established, but it is unclear whether
exposures fall within relevant exposure windows for
the outcome of interest.
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM nonmalignant respiratory disease mortality-Respiratory
HERO ID: 730555
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
Subjects vital status was identified through Na-
tional Death Index and the Social Security Admin-
istration's Death Master Files. Previous assessment
(HERO ID 730554) indicates that a nosologist re-
viewed the death certificates and coded the under-
lying cause of death in accordance with the ninth
revision of the I CD codes. Employees not identified
as deceased were assumed to be living at the end
of the study period. Previous assessment identified
1 '2*2 deaths through September 1986 with this study
following up through December 1990.
Metric 8:
Reporting Bias

X
0.333
0.33
Mortality outcomes with a difference in observed vs
expected of more than one were included in the re-
sults. SMRs were presented with confidence inter-
vals in an easily read table.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
High
X
0.5
0.5
SMRs accounted for age, race, gender, and calendar
period. Smoking was not discussed, but may not be
an issue as there was no increase in lung cancer.
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
Not reported, but likely obtained from death records
and the local rates.
Metric 11:
Co-exposure Confounding
Low
X
0.25
0.75
Although DCM was the principal solvent used (and
noted to be at the highest concentrations), methanol
and acetone were also present. Although methanol
was considerably lower with the upper concentration
of 140 ppm compared to the 1700 pp, for DCM, ace-
tone reached as high as 1600 ppm.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
The design of this study was appropriate for the
question of association between DCM and excess
mortality.
Metric 13:
Statistical power
Medium
X
0.2
0.4
There are sufficient subjects for statistical pov/er
overall, however many of the listed causes of death
have a small number of observed cases..
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
Sufficient details were reported to be reproducible
including the observed and expected numbers.
Metric 15:
Statistical models
Medium
X
0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type:	Cellulose fiber production workers DCM	nonmalignant respiratory disease mortality-Respiratory
HERO ID:	730555
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination*	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 30: Lanes et al. 1993: Evaluation of Cardiovascular Outcomes
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM ischemic heart disease mortality-Cardiovascular
HERO ID: 730555
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
tiign x 0.4
Medium x 0.2

0.4 Cohort was assembled in 1977 for the purpose of
investigating potential health effects of exposure to
DCM. It included all 1271 workers employed in the
preparation and extrusion areas of the plant for at
least 3 months between January 1, 1954 and Jan-
uary 1, 1977. Demographic details on the cohort are
provided in-text. Information regarding participa-
tion rate is provided in companion publication (Ott
et al. 1983),
0.4 There does not appear to be any attrition.
0.4 Adjustment or stratification are not specifically de-
scribed. SMRs v/ere calculated using the local popu-
lation of York County, South Carolina controlled for
age, race, gender, and calendar period.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
0.8 Workers included worked in specific areas where
DCM exposure v/ould have occurred. Industrial
monitoring in 1977 revealed 8-h time weighted aver-
age concentrations of below detection to 1700 ppm
with median levels in the three areas of 140, 280, and
475 ppm. Respirators were not used until 1984 so ex-
posure is likely, although levels of exposure v/ere not
determined. Detailed work history was only avail-
able for 356 active employees and 119 employees who
terminated employment after 1979.
0.6 Exposure was only assessed as exposed in the occu-
pational cohort compared to unexposed in the local
population.
0.8 Temporality is established, but it is unclear whether
exposures fall within relevant exposure windows for
the outcome of interest.
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM ischemic heart disease mortality-Cardiovascular
HERO ID: 730555
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
Subjects vital status was identified through Na-
tional Death Index and the Social Security Admin-
istration's Death Master Files. Previous assessment
(HERO ID 730554) indicates that a nosologist re-
viewed the death certificates and coded the under-
lying cause of death in accordance with the ninth
revision of the I CD codes. Employees not identified
as deceased were assumed to be living at the end
of the study period. Previous assessment identified
1 '2*2 deaths through September 1986 with this study
following up through December 1990.
Metric 8:
Reporting Bias

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type:	Cellulose fiber production workers DCM	ischemic heart disease mortality-Cardiovascular
HERO ID:	730555
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination*	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 31: Lanes et al. 1993: Evaluation of Cancer Outcomes
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM breast cancer mortality-Cancer
HERO ID: 730555
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
tiign x 0.4
Medium x 0.2

0.4 Cohort was assembled in 1977 for the purpose of
investigating potential health effects of exposure to
DCM. It included all 1271 workers employed in the
preparation and extrusion areas of the plant for at
least 3 months between January 1, 1954 and Jan-
uary 1, 1977. Demographic details on the cohort are
provided in-text. Information regarding participa-
tion rate is provided in companion publication (Ott
et al. 1983),
0.4 There does not appear to be any attrition.
0.4 Adjustment or stratification are not specifically de-
scribed. SMRs v/ere calculated using the local popu-
lation of York County, South Carolina controlled for
age, race, gender, and calendar period.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
0.8 Workers included worked in specific areas where
DCM exposure v/ould have occurred. Industrial
monitoring in 1977 revealed 8-h time weighted aver-
age concentrations of below detection to 1700 ppm
with median levels in the three areas of 140, 280, and
475 ppm. Respirators were not used until 1984 so ex-
posure is likely, although levels of exposure v/ere not
determined. Detailed work history was only avail-
able for 356 active employees and 119 employees who
terminated employment after 1979.
0.6 Exposure was only assessed as exposed in the occu-
pational cohort compared to unexposed in the local
population.
0.8 Temporality is established, but it is unclear whether
exposures fall within relevant exposure windows for
the outcome of interest.
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM breast cancer mortality-Cancer
HERO ID: 730555
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
Subjects vital status was identified through Na-
tional Death Index and the Social Security Admin-
istration's Death Master Files. Previous assessment
(HERO ID 730554) indicates that a nosologist re-
viewed the death certificates and coded the under-
lying cause of death in accordance with the ninth
revision of the I CD codes. Employees not identified
as deceased were assumed to be living at the end
of the study period. Previous assessment identified
1 '2*2 deaths through September 1986 with this study
following up through December 1990.
Metric 8:
Reporting Bias

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
The design of this study was appropriate for the
question of association between DCM and excess
mortality.
Metric 13:
Statistical power
Medium
X
0.2
0.4
There are sufficient subjects for statistical pov/er
overall, however many of the listed causes of death
have a small number of observed cases.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
Sufficient details were reported to be reproducible
including the observed and expected numbers.
Metric 15:
Statistical models
Medium
X
0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type:	Cellulose fiber production workers DCM	breast cancer mortality-Cancer
HERO ID:	730555
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination*	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 32: Lanes et al. 1993: Evaluation of Mortality Outcomes
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM all causes mortality-Mortality
HERO ID: 730555
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
tiign x 0.4
Medium x 0.2

-------
.. . continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type: Cellulose fiber production workers DCM all causes mortality-Mortality
HERO ID: 730555
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
Subjects vital status was identified through Na-
tional Death Index and the Social Security Admin-
istration's Death Master Files. Previous assessment
(HERO ID 730554) indicates that a nosologist re-
viewed the death certificates and coded the under-
lying cause of death in accordance with the ninth
revision of the I CD codes. Employees not identified
as deceased were assumed to be living at the end
of the study period. Previous assessment identified
1 '2*2 deaths through September 1986 with this study
following up through December 1990.
Metric 8:
Reporting Bias

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
The design of this study was appropriate for the
question of association between DCM and excess
mortality.
Metric 13:
Statistical power
Medium
X
0.2
0.4
There are sufficient subjects for statistical pov/er
overall, however many of the listed causes of death
have a small number of observed cases.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
Sufficient details were reported to be reproducible
including the observed and expected numbers.
Metric 15:
Statistical models
Medium
X
0.2
0.4
The choice of an SMR was appropriate and transpar-
ent to investigate the question of exposure to DCM
and excess mortality.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Lanes, S. F., Rothman, K. J., Dreyer, N. A., Soden, K. J. (1993). Mortality update of cellulose fiber production workers Scandinavian
Journal of Work, Environment and Health, 19(6), 426-428
Data Type:	Cellulose fiber production workers DCM	all causes mortality-Mortality
HERO ID:	730555
Domain	Metric	Rating^ MWF* Score	Comments^
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / V . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 33: TasMiien et al. 1986: Evaluation of Reproductive Outcomes
Study Citation: Taskinen, H., Lindbohm, M.L., Hemminki, K. (1986). Spontaneous abortions among women working in the pharmaceutical industry
British Journal of Industrial Medicine, 43(3,3), 199-205
Data Type: DCM exposed workers cases vs. controls spontaneous abortion-Reproductive
HERO ID: 730584
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
dium x 0.4

Metric 2: Attrition
Metric 3: Comparison Group
Medium x 0.4
Medium x 0.2
0.8 Women were employed in 8 Finnish pharmaceuti-
cal factories from 1973-1980 were matched to hospi-
tal records for pregnancy outcomes during and af-
ter employment (1973-1981). The total number of
pregnancies was 1795, which included 1179 deliv-
eries, 142 spontaneous abortions, and 474 induced
abortions. General population and matched controls
were used. A subset of 44 cases (spontaneous abor-
tion) and 130 controls (delivery) who worked in these
factories for at least 1 week in the first trimester of
pregnancy and completed questionnaires v/ere used
in a case-control analysis. The authors stated that
there were 8 factories, but only 4 v/ere included in
1975; it is unclear why these 4 factories were selected
and if the same factories employed the cases and the
matched controls.
0.8 Only subjects with completed exposure question-
naires were included in the case-control study. Thus,
3 cases (6.8%) and 9 controls (6.9%) were excluded.
This loss of subjects does not appear to be signifi-
cant and was adequately addressed in the study.
0.4 Cases and controls were employed in the same phar-
maceutical factories during the first trimester of
pregnancy, but job titles may have differed. Con-
trols were matched on age at time of conception
(within two and a half years). However, they were
not matched on any other characteristics and no ad-
justments were made in the statistical analyses.
Domain 2: Exposure Characterization
Continued on next page

Metric 6: Temporality
Low

X 0.4 1.2 Factory physician and nurses completed question-
naires on subjects of the case-control study based on
health cards, labor protection chiefs, and foremen
of departments. The questionnaire form requested
information on the individual worker's occupation
and main tasks, and exposure to solvents, including
DCM, antineoplastic agents and carcinogens, hor-
mones, and antibiotics. Coders were blinded to out-
come status. Since exposure was estimated based
on professional judgement, there is uncertainty in
the reliability of the exposure classification.
X 0.2 0.6 Exposure to DCM was classified based on frequency
of exposure (less than once a week or greater than
once a week). The intensity of solvent exposure was
evaluated based on frequency of collective solvent
use. Duration of exposure was not considered or
discussed in this assessment. These limited exposure
levels are not sufficient to provide a high degree of
accuracy in the exposure-response assessment anal-
ysis.
X 0.4 1.2 Pregnancy outcomes were assessed based on hospital
records for women working in pharmaceu t i ca 1 facto-
ries in Finland for at least one week during the first
trimester of pregnancy. While exposure during the
first trimester is anticipated to be an appropriate
window of exposure, it is unclear if the length of ex-
posure (1 week) is sufficient to detect an effect. No
details are provided regarding the average length of
employment or how that related to pregnancy out-
Domain 3: Outcome Assessment
Continued on next page

Pregnancy outcomes (delivery, spontaneous abor-
tion, induced abortion) were linked to workers v/ere
linked by personal identity number to a nation-wide
hospital discharge register and hospital polyclinic
data for 1973 to 1981. The reliability of the register
was described in a references (Lindbohm 1984, Hem-
minki. 1985, and Niemi. 1985). Women treated for
spontaneous abortions (ICD-8 codes 643 and 645)
were defined as cases. If the woman had one or more
spontaneous abortions, only one was randomly se-
lected. Three controls were selected for every case
from women who had given birth (ICD-8 codes 650-
662) but only one pregnancy per woman was in-
cluded. Unclear if women with both spontaneous
abortions and healthy pregnancies were in the orig-
inal subject pool.
Outcomes for case-control study fully presented, in-
cluding distribution by occupation (Table 1) and fre-
quency of DCM exposure (Table 2). Odds ratios
(OR) for spontaneous abortions presented in Table
3 and Table 5 by DCM exposure (never/ever) and
by frequency of exposure to DCM, respectively. No
results were presented in the tables for the odds ra-
tio for spontaneous abortions by intensity of expo-
sure to DCM or for the rate of spontaneous abor-
tions based on the year of employment (the authors'
stated in the text that the spontaneous abortion rate
decreased from about 15% to 9.5% for all employees
during the study).
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
X 0.5 1.5 The authors did not adjust for any covariates in
the analysis of the association between DCM and
spontaneous abortions. However, separate analy-
sis of the odds ratio of spontaneous abortions and
diseases and medications, type of work (sedentary,
varying and standing), and amount of heavy lifting
were presented; heavy lifting was significantly asso-
ciated with spontaneous abortions. Information on
smoking and previous pregnancies was available for
only 25% and 41% of the women, respectively, and
not presented. No consideration of alcohol intake or
socioeconomic status was presented.
Continued on next page

-------
. continued from previous page
Study Citation: Taskinen, H., Lindbohm, M.L., Hemminki, K. (1986). Spontaneous abortions among women working in the pharmaceutical industry
British Journal of Industrial Medicine, 43(3,3), 199-205
Data Type: DCM exposed workers cases vs. controls spontaneous abortion-Reproductive
HERO ID: 730584
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
X 0.25 0.75 Confounders were determined from the same occu-
pational questionnaire used to determine exposure
status. Few details are provided, but the low cap-
ture rate for smoking and previous pregnancy status
indicates this was not a reliable method.
X 0.25 0.75 Occupational co-exposure information was collected
from questionnaires completed by the occupational
physician or nurses at the factory. Exposure to
a number of solvents (aliphatic hydrocarbons, ali-
cyclic hydrocarbons, benzene, toluene, xylene, chlo-
roform), antieoplatic agents, oestrogens, progesto-
gens, androgens, antibiotics, and known carcinogens
were determined for these pharmaceutical factory
v/orkers. Correlations between these additional con-
taminates was not evaluated.
Domain 5: Analysis
Metric 12: Study Design and Methods
Medium x 0.4

Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models
Medium x 0.2
Medium x 0.2
Medium x 0.2
0.8 The case-control study design was used to assess the
relationship between exposure to solvents and spon-
taneous abortions. The study calculated odds ratios
for exposure with a logistic regression model for indi-
vidual matched data based on the conditional maxi-
mum likelihood. The p values for separate variables
were evaluated by comparing the respective stan-
dardized regression coefficients with normal distri-
bution. This is an appropriate statistical model.
0.4 The case-control study examined 44 women who had
a spontaneous abortion who were matched with 130
women who had a normal birth. This number of
cases and controls is not large, but is adequate to
detect an effect in the exposed population.
0.4 Odds ratios for DCM exposure and pregnancy out-
comes were determined with logistic regression. The
description of the analysis was sufficient to under-
stand what was done and to be conceptually repro-
ducible with access to the analytic data.
0.4 Odds ratio were calculated using logistic regression,
which was transparent and presented in the paper in
sufficient detail.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16: Use of Biomarker of Exposure
Metric 17: Effect biomarker
NA
NA
NA
NA
Continued on next page

-------
. continued from previous page
Study Citation: Taskinen, H., Lindbohm, M.L., Hemminki, K. (1986). Spontaneous abortions among women working in the pharmaceutical industry
British Journal of Industrial Medicine, 43(3,3), 199-205
Data Type:	DCM	exposed workers	cases vs. controls	spontaneous abortion-Reproductive 
HERO ID:	730584
Domain

Metric
Rating^
MWF*
Score
Comments^

Metric 18:
Metric 19:
Metric 20:
Metric 21:
Metric 22:
Method Sensitivity
Biomarker stability
Sample contamination
Method requirements
Matrix adjustment

NA
NA
NA
NA
NA
NA
NA
NA
NA
NA

Overall Quality Determination	Low	2.3
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T. (Metric Score; X MWF;) / ]T\ MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 34: Soden 1993: Evaluation of Cardiovascular Outcomes
Study Citation: Soden, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM unexposed workers irregular heartbeat-Cardiovascular
HERO ID: 730597
Domain
Metric

MWF* Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
Medi um x 0.4
Metric 2: Attrition
Medium x 0.4
Metric 3: Comparison Group
m x 0.2
0.8 There is a low risk for selection bias. Methods of
participant selection and i. nc 1 us io n/ exc, 1 us ion crite-
ria are reported. The exposed group consisted of all
of the 150 employees at the Rock Hill plant as of
December 31, 1986 who had worked for at least 10
years in the high methylene chloride exposure area
and had also participated in the company's health
monitoring program between 1984 and 1986. It is
unclear how many hi.gh 1 v-exposed employees were
excluded because they had not participated in the
health monitoring program.
0.8 Outcome data v/ere incomplete, especially for the
blood test parameters (hemato 1 ogi.c,ai and hepatic
outcomes). The missing outcome data were ex-
plained by the author and were balanced across
study groups with similar reasons for the missing
data. Outcome data were missing because not all
of the employees responded to every health history
question and not all of the employees underwent ev-
ery blood test during the study period because of
varying frequencies of examinations offered to em-
ployees based on age. There were a total of 150
exposed employees, and blood test data were only
reported for 90-103 of them depending on the test.
There were a total of 260 control subjects, and blood
test data were only reported for 120-126 of them.
The health history questionnaire data (neurological
and cardiovascular outcomes) were nearly complete
with 137-150/150 exposed employees responding to
the various questions and 247-258/260 of the con-
trols responding.
0.4 There is only indirect evidence from the author that
the exposed and control groups were similar. Work-
ers at another plant within the same company (a
polyester staple plant in Salisbury, NC) were cho-
sen as the non-exposed controls. The two plants
were reportedly "socioeconomi.ca 1 Iv and demograph-
ically similar as well as geographically proximate."
The controls were randomly selected and matched
for age, sex, and race.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM unexposed workers irregular heartbeat-Cardiovascular
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
1.2 The methods used to quantify exposure were not
fully described in this publication. The author
states, "Exposure assessment for both cohorts was
performed routinely as part of the HCC industrial
hygiene monitoring program. All monitoring was
done using standard, validated in hospital measuring
techniques and analysis was done by national certi-
fied laboratories." The exposed workers were chosen
from a larger cohort of workers (n=1271) that had
been followed for mortality with results reported in
Ott et ah, 1983 (not in HERO) and Lanes et ah 1990
(HERO ID 730554). The current study reports that
the average methylene chloride exposure of the em-
ployees was 475 ppm (8-hour TWA) for at least ten
years. No further details are provided.
0.6 There were only 2 levels of exposure (exposed vs.
non-exposed).
0.8 Temporality is established and the interval between
exposure and outcomes has an appropriate consid-
eration of relevant exposure windows for the out-
comes of interest. The study population was fol-
lowed from 1984-1986, and exposure occurred for at
least 10 years as of December 31, 1986.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Low
x 0.667
The outcome assessment method used for cardio-
vascular and neurological outcomes is an insensi-
tive measure. These results were taken from self-
reported information in a health history question-
naire. The hematological and liver outcomes v/ere
assessed using v/e 11-estab 1 ished methods. The meth-
ods used for drawing blood were not described, how-
ever it was part of the company's health monitoring
program that involved physicians and nurse practi-
tioners. All blood work was performed by a biomed-
ical laboratory.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM unexposed workers irregular heartbeat-Cardiovascular
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Metric 8: Reporting Bias
Medium x 0.333 0.67
All of the measured outcomes outlined in the meth-
ods are reported in a way that allows for data extrac-
tion. Incidence, prevalence, and samples sizes are re-
ported for the cardiovascular and neurological out-
comes. Sample sizes, means, standard deviations,
and the results of statistical analyses are reported
only for the hematological and hepatic outcomes
(continuous outcomes). Statistical significance for
comparison of prevalence measures are not reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
Low
X 0.5 0.5 Appropriate considerations were made for potential
confounders. Two controls were selected at random
for each exposed cohort member matching for age,
sex, and race. The author points out that no at-
tempt was made to control for the potential con-
founding effects of alcohol on the liver parameters
studied. However, the researchers found from pre-
vious studies utilizing the same health monitoring
data base that the socioeconomi.cs and demograph-
ics of both plants are similar. Since no differences
were found in the health parameters between the two
plants, more specific analysis including any potential
confounders was not considered necessary.
X 0.25 0.25 All of the chosen cohort members and controls par-
ticipated in the corporate health monitoring pro-
gram that would have collected information on age,
sex, and race.
x 0.25 0.75 There is direct evidence that there were unbal-
anced co-exposures across the study groups which
were not adjusted for. The exposed employees v/ere
also exposed to acetone and methanol. The re-
searchers considered the potential impacts of these
co-exposures on the results and determined it was
not an issue because significant differences were not
found between the two groups. "Theoretically, we
could postulate that the acetone and methanol ex-
posure in the exposed group might potentiate any ef-
fects from the methylene chloride exposure because
of their potential impact on the same target organs,
ie, liver, blood, and central nervous system but this
was clearly not the case as there were no clinically
significant differences found between the two groups.
This synergism or potentiation is not an issue based
upon the results of this study."
Domain 5: Analysis
Continued on next page

Rating?
MWF*
Score
Commentstt
Metric 12
Study Design and Methods

Medium
x 0.4
0.8
The study design chosen was appropriate for the
research question and the study used an appropri-
ate statistical method to address the research ques-
tion. For the hematological and hepatic outcomes,
the Student's t-test was used to compare the means.
For the cardiovascular and neurological outcomes.

the prevalence of responses was compared between

the two groups, but the statistical method was not
described.
Metric 13
Statistical power

Medium
x 0.2
0.4
The number of participants is adequate to detect an
effect in the exposed population. There were 150

Low

exposed subjects and 260 controls.
Metric 14
Reproducibility of analyses

x 0.2
0.6
The statistical test used to compare the responses
on the health history questionnaire (cardiovascular
and neurological outcomes) was not described.
Metric 15
Statistical models

Low
x 0.2
0.6
The method (one sided t-test) used for comparing
the means for the hematological and hepatic out-
comes is transparent. The statistical analyses are

not described for the cardiovascular and neurologi-

cal outcome measures.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*

Medium

2.2

Extracted
Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWF;) / J] . MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 35: Soden 1993: Evaluation of Neurological/Behavior Outcomes
Study Citation: Soden, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers memory loss-Neuroloeical/Behavior
HERO ID: 730597
Domain
Metric

MWF* Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
Medi um x 0.4
Metric 2: Attrition
Medium x 0.4
to
o
Metric 3: Comparison Group
m x 0.2
0.8 There is a low risk for selection bias. Methods of
participant selection and i nc. 1 us io n/ exc 1 us ion crite-
ria are reported. The exposed group consisted of all
of the 150 employees at the Rock Hill plant as of
December 31, 1986 who had worked for at least 10
years in the high methylene chloride exposure area
and had also participated in the company's health
monitoring program between 1984 and 1986. It is
unclear how many hi.gh 1 v-exposed employees were
excluded because they had not participated in the
health monitoring program.
0.8 Outcome data v/ere incomplete, especially for the
blood test parameters (hemato 1 ogi.c,ai and hepatic
outcomes). The missing outcome data were ex-
plained by the author and were balanced across
study groups with similar reasons for the missing
data. Outcome data were missing because not all
of the employees responded to every health history
question and not all of the employees underwent ev-
ery blood test during the study period because of
varying frequencies of examinations offered to em-
ployees based on age. There were a total of 150
exposed employees, and blood test data were only
reported for 90-103 of them depending on the test.
There were a total of 260 control subjects, and blood
test data were only reported for 120-126 of them.
The health history questionnaire data (neurological
and cardiovascular outcomes) were nearly complete
with 137-150/150 exposed employees responding to
the various questions and 247-258/260 of the con-
trols responding.
0.4 There is only indirect evidence from the author that
the exposed and control groups were similar. Work-
ers at another plant within the same company (a
polyester staple plant in Salisbury, NC) were cho-
sen as the non-exposed controls. The two plants
were reportedly "socioeconomi.ca 1 Iv and demograph-
ically similar as well as geographically proximate."
The controls were randomly selected and matched
for age, sex, and race.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers memory loss-Neurological/Behavior
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
1.2 The methods used to quantify exposure were not
fully described in this publication. The author
states, "Exposure assessment for both cohorts was
performed routinely as part of the HCC industrial
hygiene monitoring program. All monitoring was
done using standard, validated in hospital measuring
techniques and analysis was done by national certi-
fied laboratories." The exposed workers were chosen
from a larger cohort of workers (n=1271) that had
been followed for mortality with results reported in
Ott et ah, 1983 (not in HERO) and Lanes et ah 1990
(HERO ID 730554). The current study reports that
the average methylene chloride exposure of the em-
ployees was 475 ppm (8-hour TWA) for at least ten
years. No further details are provided.
0.6 There were only 2 levels of exposure (exposed vs.
non-exposed).
0.8 Temporality is established and the interval between
exposure and outcomes has an appropriate consid-
eration of relevant exposure windows for the out-
comes of interest. The study population was fol-
lowed from 1984-1986, and exposure occurred for at
least 10 years as of December 31, 1986.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Low
x 0.667
The outcome assessment method used for cardio-
vascular and neurological outcomes is an insensi-
tive measure. These results were taken from self-
reported information in a health history question-
naire. The hematological and liver outcomes v/ere
assessed using v/e 11-estab 1 ished methods. The meth-
ods used for drawing blood were not described, how-
ever it was part of the company's health monitoring
program that involved physicians and nurse practi-
tioners. All blood work was performed by a biomed-
ical laboratory.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers memory loss-Neurological/Behavior
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Metric 8: Reporting Bias
Medium x 0.333 0.67
All of the measured outcomes outlined in the meth-
ods are reported in a way that allows for data extrac-
tion. Incidence, prevalence, and samples sizes are re-
ported for the cardiovascular and neurological out-
comes. Sample sizes, means, standard deviations,
and the results of statistical analyses are reported
only for the hematological and hepatic outcomes
(continuous outcomes). Statistical significance for
comparison of prevalence measures are not reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
to
to
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
Low
X 0.5 0.5 Appropriate considerations were made for potential
confounders. Two controls were selected at random
for each exposed cohort member matching for age,
sex, and race. The author points out that no at-
tempt was made to control for the potential con-
founding effects of alcohol on the liver parameters
studied. However, the researchers found from pre-
vious studies utilizing the same health monitoring
data base that the socioeconomi.cs and demograph-
ics of both plants are similar. Since no differences
were found in the health parameters between the two
plants, more specific analysis including any potential
confounders was not considered necessary.
X 0.25 0.25 All of the chosen cohort members and controls par-
ticipated in the corporate health monitoring pro-
gram that would have collected information on age,
sex, and race.
x 0.25 0.75 There is direct evidence that there were unbal-
anced co-exposures across the study groups which
were not adjusted for. The exposed employees v/ere
also exposed to acetone and methanol. The re-
searchers considered the potential impacts of these
co-exposures on the results and determined it was
not an issue because significant differences were not
found between the two groups. "Theoretically, we
could postulate that the acetone and methanol ex-
posure in the exposed group might potentiate any ef-
fects from the methylene chloride exposure because
of their potential impact on the same target organs,
ie, liver, blood, and central nervous system but this
was clearly not the case as there were no clinically
significant differences found between the two groups.
This synergism or potentiation is not an issue based
upon the results of this study."
Domain 5: Analysis
Continued on next page

Rating?
MWF*
Score
Commentstt
Metric 12
Study Design and Methods

the prevalence of responses was compared between

the two groups, but the statistical method was not
described.
Metric 13
Statistical power

Medium
x 0.2
0.4
The number of participants is adequate to detect an
effect in the exposed population. There were 150

Low

exposed subjects and 260 controls.
Metric 14
Reproducibility of analyses

x 0.2
0.6
The statistical test used to compare the responses
on the health history questionnaire (cardiovascular
and neurological outcomes) was not described.
Metric 15
Statistical models

Low
x 0.2
0.6
The method (one sided t-test) used for comparing
the means for the hematological and hepatic out-
comes is transparent. The statistical analyses are

not described for the cardiovascular and neurologi-

cal outcome measures.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*

Medium

2.2

-------
Table 36: Soden 1993: Evaluation of Hepatic Outcomes
Study Citation: Soden, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers SGOT-Hepatic
HERO ID: 730597
Domain
Metric

MWF* Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
Medi um x 0.4
Metric 2: Attrition
Medium x 0.4
to
Metric 3: Comparison Group
m x 0.2
0.8 There is a low risk for selection bias. Methods of
participant selection and i. nc 1 us io n/ exc, 1 us ion crite-
ria are reported. The exposed group consisted of all
of the 150 employees at the Rock Hill plant as of
December 31, 1986 who had worked for at least 10
years in the high methylene chloride exposure area
and had also participated in the company's health
monitoring program between 1984 and 1986. It is
unclear how many hi.gh 1 v-exposed employees were
excluded because they had not participated in the
health monitoring program.
0.8 Outcome data v/ere incomplete, especially for the
blood test parameters (hemato 1 ogi.c,ai and hepatic
outcomes). The missing outcome data were ex-
plained by the author and were balanced across
study groups with similar reasons for the missing
data. Outcome data were missing because not all
of the employees responded to every health history
question and not all of the employees underwent ev-
ery blood test during the study period because of
varying frequencies of examinations offered to em-
ployees based on age. There were a total of 150
exposed employees, and blood test data were only
reported for 90-103 of them depending on the test.
There were a total of 260 control subjects, and blood
test data were only reported for 120-126 of them.
The health history questionnaire data (neurological
and cardiovascular outcomes) were nearly complete
with 137-150/150 exposed employees responding to
the various questions and 247-258/260 of the con-
trols responding.
0.4 There is only indirect evidence from the author that
the exposed and control groups were similar. Work-
ers at another plant within the same company (a
polyester staple plant in Salisbury, NC) were cho-
sen as the non-exposed controls. The two plants
were reportedly "socioeconomi.ca 1 Iv and demograph-
ically similar as well as geographically proximate."
The controls were randomly selected and matched
for age, sex, and race.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers SGOT-Hepatic
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4
to
Ox
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
1.2 The methods used to quantify exposure were not
fully described in this publication. The author
states, "Exposure assessment for both cohorts was
performed routinely as part of the HCC industrial
hygiene monitoring program. All monitoring was
done using standard, validated in hospital measuring
techniques and analysis was done by national certi-
fied laboratories." The exposed workers were chosen
from a larger cohort of workers (n=1271) that had
been followed for mortality with results reported in
Ott et ah, 1983 (not in HERO) and Lanes et ah 1990
(HERO ID 730554). The current study reports that
the average methylene chloride exposure of the em-
ployees was 475 ppm (8-hour TWA) for at least ten
years. No further details are provided.
0.6 There were only 2 levels of exposure (exposed vs.
non-exposed).
0.8 Temporality is established and the interval between
exposure and outcomes has an appropriate consid-
eration of relevant exposure windows for the out-
comes of interest. The study population was fol-
lowed from 1984-1986, and exposure occurred for at
least 10 years as of December 31, 1986.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Low
x 0.667
The outcome assessment method used for cardio-
vascular and neurological outcomes is an insensi-
tive measure. These results were taken from self-
reported information in a health history question-
naire. The hematological and liver outcomes v/ere
assessed using v/e 11-estab 1 ished methods. The meth-
ods used for drawing blood were not described, how-
ever it was part of the company's health monitoring
program that involved physicians and nurse practi-
tioners. All blood work was performed by a biomed-
ical laboratory.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers SGOT-Hepatic
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Metric 8: Reporting Bias
High
x 0.333 0.33
All of the measured outcomes outlined in the meth-
ods are reported in a way that allows for data extrac-
tion. Incidence, prevalence, and samples sizes are re-
ported for the cardiovascular and neurological out-
comes. Sample sizes, means, standard deviations,
and the results of statistical analyses are reported
only for the hematological and hepatic outcomes
(continuous outcomes). Statistical significance for
comparison of prevalence measures are not reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
to
02
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
Low
X 0.5 0.5 Appropriate considerations were made for potential
confounders. Two controls were selected at random
for each exposed cohort member matching for age,
sex, and race. The author points out that no at-
tempt was made to control for the potential con-
founding effects of alcohol on the liver parameters
studied. However, the researchers found from pre-
vious studies utilizing the same health monitoring
data base that the socioeconomics and demograph-
ics of both plants are similar. Since no differences
were found in the health parameters between the two
plants, more specific analysis including any potential
confounders was not considered necessary.
X 0.25 0.25 All of the chosen cohort members and controls par-
ticipated in the corporate health monitoring pro-
gram that would have collected information on age,
sex, and race.
x 0.25 0.75 There is direct evidence that there were unbal-
anced co-exposures across the study groups which
were not adjusted for. The exposed employees v/ere
also exposed to acetone and methanol. The re-
searchers considered the potential impacts of these
co-exposures on the results and determined it was
not an issue because significant differences were not
found between the two groups. "Theoretically, we
could postulate that the acetone and methanol ex-
posure in the exposed group might potentiate any ef-
fects from the methylene chloride exposure because
of their potential impact on the same target organs,
ie, liver, blood, and central nervous system but this
was clearly not the case as there were no clinically
significant differences found between the two groups.
This synergism or potentiation is not an issue based
upon the results of this study."
Domain 5: Analysis
Continued on next page

Rating?
MWF*
Score
Commentstt
Metric 12
Study Design and Methods

the prevalence of responses was compared between

the two groups, but the statistical method was not
described.
Metric 13
Statistical power

Medium
x 0.2
0.4
The number of participants is adequate to detect an
effect in the exposed population. There were 150

Low

exposed subjects and 260 controls.
Metric 14
Reproducibility of analyses

x 0.2
0.6
The statistical test used to compare the responses
on the health history questionnaire (cardiovascular
and neurological outcomes) was not described.
Metric 15
Statistical models

Low
x 0.2
0.6
The method (one sided t-test) used for comparing
the means for the hematological and hepatic out-
comes is transparent. The statistical analyses are

not described for the cardiovascular and neurologi-

cal outcome measures.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*

Medium

2.2

-------
Table 37: Soden 1993: Evaluation of Hematological and Immune Outcomes
Study Citation: Soden, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers hematocrit-Hematological and Immune
HERO ID: 730597
Domain
Metric

MWF* Score
Commentstt
Domain 1: Study Participation
Metric 1: Participant selection
Medi um x 0.4
Metric 2: Attrition
Medium x 0.4
to
oo
Metric 3: Comparison Group
m x 0.2
0.8 There is a low risk for selection bias. Methods of
participant selection and i. nc 1 us io n/ exc, 1 us ion crite-
ria are reported. The exposed group consisted of all
of the 150 employees at the Rock Hill plant as of
December 31, 1986 who had worked for at least 10
years in the high methylene chloride exposure area
and had also participated in the company's health
monitoring program between 1984 and 1986. It is
unclear how many hi.gh 1 v-exposed employees were
excluded because they had not participated in the
health monitoring program.
0.8 Outcome data v/ere incomplete, especially for the
blood test parameters (hemato 1 ogi.c,ai and hepatic
outcomes). The missing outcome data were ex-
plained by the author and were balanced across
study groups with similar reasons for the missing
data. Outcome data were missing because not all
of the employees responded to every health history
question and not all of the employees underwent ev-
ery blood test during the study period because of
varying frequencies of examinations offered to em-
ployees based on age. There were a total of 150
exposed employees, and blood test data were only
reported for 90-103 of them depending on the test.
There were a total of 260 control subjects, and blood
test data were only reported for 120-126 of them.
The health history questionnaire data (neurological
and cardiovascular outcomes) were nearly complete
with 137-150/150 exposed employees responding to
the various questions and 247-258/260 of the con-
trols responding.
0.4 There is only indirect evidence from the author that
the exposed and control groups were similar. Work-
ers at another plant within the same company (a
polyester staple plant in Salisbury, NC) were cho-
sen as the non-exposed controls. The two plants
were reportedly "socioeconomi.ca 1 Iv and demograph-
ically similar as well as geographically proximate."
The controls were randomly selected and matched
for age, sex, and race.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers hematocrit-Hematological and Immune
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4
to
CO
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.2
Medium x 0.4
1.2 The methods used to quantify exposure were not
fully described in this publication. The author
states, "Exposure assessment for both cohorts was
performed routinely as part of the HCC industrial
hygiene monitoring program. All monitoring was
done using standard, validated in hospital measuring
techniques and analysis was done by national certi-
fied laboratories." The exposed workers were chosen
from a larger cohort of workers (n=1271) that had
been followed for mortality with results reported in
Ott et ah, 1983 (not in HERO) and Lanes et ah 1990
(HERO ID 730554). The current study reports that
the average methylene chloride exposure of the em-
ployees was 475 ppm (8-hour TWA) for at least ten
years. No further details are provided.
0.6 There were only 2 levels of exposure (exposed vs.
non-exposed).
0.8 Temporality is established and the interval between
exposure and outcomes has an appropriate consid-
eration of relevant exposure windows for the out-
comes of interest. The study population was fol-
lowed from 1984-1986, and exposure occurred for at
least 10 years as of December 31, 1986.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Low
x 0.667
The outcome assessment method used for cardio-
vascular and neurological outcomes is an insensi-
tive measure. These results were taken from self-
reported information in a health history question-
naire. The hematological and liver outcomes v/ere
assessed using v/e 11-estab 1 ished methods. The meth-
ods used for drawing blood were not described, how-
ever it was part of the company's health monitoring
program that involved physicians and nurse practi-
tioners. All blood work was performed by a biomed-
ical laboratory.
Continued on next page

-------
. continued from previous page
Study Citation: So den, K.J. (1993). An evaluation of chronic methylene chloride exposure Journal of Occupational Medicine, 35(3,3), 282-286
Data Type: DCM exposed workers hematocrit-Hematological and Immune
HERO ID: 730597
Domain
Metric
Ratingt MWF* Score
Commentstt
Metric 8: Reporting Bias
Medium x 0.333 0.67
All of the measured outcomes outlined in the meth-
ods are reported in a way that allows for data extrac-
tion. Incidence, prevalence, and samples sizes are re-
ported for the cardiovascular and neurological out-
comes. Sample sizes, means, standard deviations,
and the results of statistical analyses are reported
only for the hematological and hepatic outcomes
(continuous outcomes). Statistical significance for
comparison of prevalence measures are not reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
CO
o
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
Low
X 0.5 0.5 Appropriate considerations were made for potential
confounders. Two controls were selected at random
for each exposed cohort member matching for age,
sex, and race. The author points out that no at-
tempt was made to control for the potential con-
founding effects of alcohol on the liver parameters
studied. However, the researchers found from pre-
vious studies utilizing the same health monitoring
data base that the socioeconomi.cs and demograph-
ics of both plants are similar. Since no differences
were found in the health parameters between the two
plants, more specific analysis including any potential
confounders was not considered necessary.
X 0.25 0.25 All of the chosen cohort members and controls par-
ticipated in the corporate health monitoring pro-
gram that would have collected information on age,
sex, and race.
x 0.25 0.75 There is direct evidence that there were unbal-
anced co-exposures across the study groups which
were not adjusted for. The exposed employees v/ere
also exposed to acetone and methanol. The re-
searchers considered the potential impacts of these
co-exposures on the results and determined it was
not an issue because significant differences were not
found between the two groups. "Theoretically, we
could postulate that the acetone and methanol ex-
posure in the exposed group might potentiate any ef-
fects from the methylene chloride exposure because
of their potential impact on the same target organs,
ie, liver, blood, and central nervous system but this
was clearly not the case as there were no clinically
significant differences found between the two groups.
This synergism or potentiation is not an issue based
upon the results of this study."
Domain 5: Analysis
Continued on next page

Rating?
MWF*
Score
Commentstt
Metric 12
Study Design and Methods

the prevalence of responses was compared between

the two groups, but the statistical method was not
described.
Metric 13
Statistical power

Medium
x 0.2
0.4
The number of participants is adequate to detect an
effect in the exposed population. There were 150

Low

exposed subjects and 260 controls.
Metric 14
Reproducibility of analyses

x 0.2
0.6
The statistical test used to compare the responses
on the health history questionnaire (cardiovascular
and neurological outcomes) was not described.
Metric 15
Statistical models

Low
x 0.2
0.6
The method (one sided t-test) used for comparing
the means for the hematological and hepatic out-
comes is transparent. The statistical analyses are

not described for the cardiovascular and neurologi-

cal outcome measures.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*

Medium

2.2

-------
Table 38: Kalkhrenner et al. 2010: Evaluation of Neurological/Behavior Outcomes
Study Citation: Kalkbrenner, A.E., Daniels, J.L., Chen, J.C., Poole, C., Emch, M., Morrissey, J (2010). Perinatal exposure to hazardous air pollutants
and autism spectrum disorders at age 8 Epidemiology, 21(5), 631-641
Data Type: DCM autism spectrum disorder (ASD) children-Neurological/Behavior
HERO ID: 737424
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2:
Attrition
Medium x 0.4
Metric 3: Comparison Group
High x 0.2
0.4 Cases identified through ADDM network in 8 NC
counties (2002-2004) or all of WV (2000-2002) and
based on DSM-IV-TR. Participants limited to chil-
dren who resided in study location at time of birth,
confirmed by matching birth certificates. In NC, 220
of 311 children identified with ASD had a matching
birth certificate, and 206 of those were born in the
surveillance counties and eligible for inclusion. In
WV, 189 of 257 children identified with ASD had
a matching birth certificate, and a census tract was
determined for 177 of those and they were eligible
for inclusion.
0.8 There was a moderate amount of exclusions, but rea-
sons were documented (i.e., those without in-state
birth certificates, a 1/3 random sampling of WV con-
trols, and those lacking Census tract data) and han-
dled adequately. Approximately 33% of NC cases,
30% of WV cases, 33% of NC controls, and 75% of
WV controls (or 23% of those randomly sampled)
were excluded from the analysis.
0.2 Controls identified during the same time period as
cases through school system based on speech and
language impairment w/o doc,umentati.on of other
developmental problems. Table 1 indicates cases
can controls were similar, except for covariates that
were included in statistical models (i.e., maternal
age, smoking in pregnancy, maternal marital status
and education, race, census tract median household
income, urbanicitv).
Domain 2: Exposure Characterization
Continued on next page . ..

-------
. continued from previous page
Study Citation: Kalkbrenner, A.E., Daniels, J.L., Chen, J.C., Poole, C., Emch, M., Morrissey, J (2010). Perinatal exposure to hazardous air pollutants
and autism spectrum disorders at age 8 Epidemiology, 21(5), 631-641
Data Type: DCM autism spectrum disorder (ASD) children-Neurological/Behavior
HERO ID: 737424
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
m x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
x 0.2
Medium x 0.4

0.8 Exposure based on modeled data because ambient
measurements not made during period of interest,
and residence at birth was used to assign Census-
tract-specific concentrations. Data for each census
tract based on National Air Toxics Assessment-1996
estimates, with primary inputs from the National
Emissions Inventory and additional inputs from me-
teorologic and sec,ondary-pollutant formation data.
Estimated PAH exposures are intended to reflect in-
dividual perinatal exposures. Authors note potential
for exposure m is c 1 ass i. fi. c at io n.
0.6 Provides clean air background levels of pollutants
and levels in NC and WV (urban, not urban, and
whole state). But analysis based only on compari-
son of 20th and 80th percentiles of 1 og-1ransformed
concentrations among controls.
0.8 Authors note exposure assigned during the perinatal
period, but subjects born between 1994-1996 (NC)
and 1992-1994 (WV) and exposure based on 1996
data, so unclear if exposure is within relevant win-
dow. Outcome measurements made between 2002-
2004 (NC) and 2000-2002 (WV).
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
Metric 8: Reporting Bias
X 0.667 0.67 Outcome based on DSM-IV-TR definition of ASD
regardless of previous diagnosis. Controls were chil-
dren in the surveillance system with speech and lan-
guage impairments, but no indication of other seri-
ous developmental problems (e.g., ASD, ID), iden-
tified from group with equivalent access to develop-
mental evaluations. All participants were 8 years
old, the age at which most ASD-affected children
have been identified.
X 0.333 0.33 OR and 95% CI reported, and number of cases and
total number of participants reported for each analy-
sis. All outlined statistical analyses, including sensi-
tivity analyses, were reported with sufficient detail.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
High X 0.5 0.5 Models adjusted for sampling variables, demo-
graphic information from birth certificate and cen-
sus (maternal age, smoking in pregnancy, maternal
marital status and education, race, census tract me-
dian household income, urbani.ci.ty), and co-varying
air pollutants.
Continued on next page

-------
.. . continued from previous page
Study Citation: Kalkbrenner, A.E., Daniels, J.L., Chen, J.C., Poole, C., Emch, M., Morrissey, J (2010). Perinatal exposure to hazardous air pollutants
and autism spectrum disorders at age 8 Epidemiology, 21(5), 631-641
Data Type: DCM autism spectrum disorder (ASD) children-Neurological/Behavior
HERO ID: 737424
Domain
Metric
Rating^
MWF*
Score
Comments^
Metric 10
Covariate Characterization
m
x 0.25
0.5
Demographic covariates determined from birth cer-
tificate and census data. Additional data source
for covariates is not explicitly reported, but demo-
graphic information is also assumed to have been
collected from the ADDM records. There is no evi-
dence of poor validity.
Metric 11
Co-exposure Confounding
Medium
x 0.25
0.5
All pollutants included in a semi-Bayes hierarchical
model that adjusted the beta coefficient for each pol-
lutant toward the mean of its exchangeability group.
Domain 5: Analysis

Medium

Metric 12
Study Design and Methods
x 0.4
0.8
Appropriate statistical methods v/ere used (Semi-
Bayes logistic regression accounting for multiple
comparisons in this case-control study).
Metric 13
Statistical power
Medium
x 0.2
0.4
Case and control sample sizes are sufficient to detect
an effect. In combined WV+NC analyses, 374 cases
and 2803 controls were included.
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
The statistical methods for the semi-Bayes hierar-
chical model were well described.
Metric 15
Statistical models
Medium
x 0.2
0.4
The assumptions for the statistical model were de-

scribed and met. Authors discussed reasoning for
including a priori covariates.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker
NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
High

1.6

Extracted

Yes

Continued on next page

-------
. continued from previous page
Study Citation: Kalkbrenner, A.E., Daniels, J.L., Chen, J.C., Poole, C., Emch, M., Morrissey, J (2010). Perinatal exposure to hazardous air pollutants
and autism spectrum disorders at age 8 Epidemiology, 21(5), 631-641
Data Type:	DCM autism spectrum disorder (ASD) children-Neurological/Behavior
HERO ID:	737424
Domain	Metric	Rating^ MWF* Score	Comments^
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
]T. (Metric Score; X MWF;) / ^ . MWF;
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 39: To meson 2011: Evaluation of Cancer Outcomes
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type: Cohort exposed workers DCM AUCancerMortality Dichotomous-Cancer
HERO ID: 787813
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition

Medium x 0.4

Metric 3: Comparison Group
Medium x 0.2
0.4 A cohort study was conducted of male workers em-
ployed at a photographic film base plant in Bran-
tham, UK from 1946 until its closure in 1988. 1,785
workers and of those, 1,473 has worked in jobs that
had exposure to methylene chloride. Information
was obtained through UK Medicai Research Infor-
mation Services. Exposed workers were predomi-
nantly manual v/orkers. Females were excluded be-
cause few had worked in production areas and many
had job titles where exposure was difficult to as-
sess. Median follow up time for exposed individu-
als was 36.8 years (IQR: 28.2-43.1) and 29.9 (IQR:
21.4-39.1) for unexposed individuals.
0.8 Attrition among the final study population with ex-
posure information was not significant. 59 v/orkers
emigrated, 10 were temporary foreign, and 10 v/ere
lost to follow-up. However, for 439 workers (30% of
study population) employment histories were insuf-
ficiently precise to calculate reliable estimates of cu-
mulative exposure-unspecified job histories and were
excluded from the dose-response analysis (most were
laborers and maintenance v/orkers).
0.4 Non -exposed group consisted of a variety of occu-
pations, but all were unlikely to have been exposed
either directly or indirectly, though very low concen-
trations may have existed. Most unexposed v/orkers
were office v/orkers in the technical and commercial
functions, and the unexposed group tended to be
older when hired and followed up for fewer years.
The exposed group consisted of mostly manual v/ork-
ers. Mortality statistics for England and Wales v/ere
used for comparison and a comparison with local
mortalities was also made for selected causes by com-
bining mortality information from four surrounding
districts to calculate an SMR. Authors attempted to
reduce potential for healthy v/orker effect by using a
suitable internal reference group.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type: Cohort exposed workers DCM AllCancerMortality Dichotomous-Cancer
HERO ID: 787813
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
Co
-a
Metric 5: Exposure levels
Medium x 0.2
Metric 6: Temporality
m X 0.4 0.8 Potential exposure to DCM was contained to the
time period for which the Brantham site was open
(1946-1988). Estimates of DCM exposure were gen-
erated for 20 work groups during four production
periods (before 1960, 1960-1969, 1970-1979, 1980-
1988) at an exposure duration of 8-h TWA, ppm.
Lifetime cumulative DCM exposure was calculated
by summing the products of mean level of exposure
and duration of employment for each job held by a
cohort member. A potential limitation is that per-
sonal monitoring data before 1980 and reliable area
monitoring data before 1975 was not available; how-
ever, good historical information on the casting ma-
chines and working conditions was available, and in-
formation on the number of incidents where workers
were affected by DCM vapors was consistent with
the pattern of exposure estimated for jobs and time
periods.
0.4 Levels of cumulative exposure to methylene chloride
were categorized as 0, 0-399, 400-799, and greater
than 800 ppm-years. These cut-off points were
chosen to enable comparisons with studies of the
Rochester film workers. Trend analyses were also
performed using cut-off points (36.4 and 299.1 ppm-
years) which gave equal numbers of deaths from all
causes in the 3 cumulative exposure categories. Cu-
mulative exposure was also modeled as a continuous
variable in the Cox regression models.
X 0.4 0.4 Study participants were unlikely to have been ex-
posed to DCM after the Brantham site closed in
1988, which strengthens the validity of the expo-
sure estimates occurring prior to the outcome as-
sessed. Cumulative exposure was treated as a time-
dependent variable, both as a continuous variable
and grouped by increasing exposure.
Time since hire was included in some models (not
the regression analyses), and analyses were also per-
formed with lagged cumulative exposure (15 years).
Median follow up time for exposed individuals was
36.8 years (IQR: 28.2-43.1) and 29.9 (IQR: 21.4-
39.1) for unexposed individuals. Long follow up
times for the cohort should be sufficient for the long
latency period of some chronic diseases assessed, in-
cluding cancer.
Continued on next page

Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
Low
X 0.5 1.5 SMRs were calculated to take into account the po-
tential confounding effect of age. Regression mod-
els included age as the time variable. Time since
hire was included in some models, and analyses were
also performed with lagged cumulative exposure (15
years). A limitation of this study includes the lack
of smoking histories for participants which suggests
potential residual confounding by smoking could in-
fluence the effect estimates.
X 0.25 0.25 Covariates considered included age and time since
hire. All were measured using appropriate, valid
methods using information provided in the UK Med-
ical Research Information Service database.
x 0.25 0.75 No co-exposures were considered in this study, nor
adjusted for in the analyses. It is unclear whether
other potential unmeasured co-exposures could have
influenced effect estimates.
Domain 5: Analysis
Continued on next page

-------
.. . continued from previous page
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type: Cohort exposed workers DCM AllCancerMortality Dichotomous-Cancer
HERO ID: 787813
Domain Metric Rating^ MWF* Score Commentstt
Metric 12:
Study Design and Methods
m
X
0.4
0.8
Cohort study design was conducted to follow work-
ers potentially exposed to methylene chloride at a
photographic, film base plant in Brantham, United
Kingdom. The outcome assessed included mortal-
ity due to different causes. The cohort study design
was appropriate for the research question involving
a relatively rare exposure, and the long follow up
time was suitable for the chronic disease outcomes
investigated.
Metric 13:
Statistical power
[m
X
0.2
0.4
The study population of 1,785 male employees, in-
cluding 1,473 exposed workers (1,034 exposed had an
exposure estimate) and 312 unexposed workers, was
sufficient to detect an effect for methylene chloride.
Statistical power not reported, but p values show
some statistically significant correlations. Pancre-
atic cancer was ultimately excluded from the regres-
sion analysis because too few cases were reported.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
Methods of statistical analysis was clearly described
and should be reproducible with information pro-
vided. Descriptions of the methods for calculating
SMRs were clearly described, and methods for re-
gression models were also described.
Metric 15:
Statistical models
Medium
X
0.2
0.4
SMRs were calculated for each mortality outcome of
interest, using the internal referent group and mor-
tality statistics for England and Wales, as well as
local mortalities from four surrounding districts and
two surrounding counties. For selected causes of
death, a multivariate regression analysis based on
Cox's proportional hazards model was conducted.
Model assumptions were met and the variables used
were clearly stated and appropriate.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Metric 22
Matrix adjustment
NA
NA
Overall Quality Determination
Medium
1.7
Continued on next page . ..

-------
. continued from previous page
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type:	Cohort_exposed workers_DCM_AHCancerMortality_Dichotomous-Cancer
HERO ID:	787813
Domain
Metric
Rating^ MWF* Score
Comments^
Extracted
Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score, x MWF;) / . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 40: To meson 2011: Evaluation of Cardiovascular Outcomes
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type: Cohort exposed workers DCM IschemicHeartDiseaseMortality Dichotomous-Cardiovascular
HERO ID: 787813
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition

Medium x 0.4

Metric 3: Comparison Group
Medium x 0.2
0.4 A cohort study was conducted of male workers em-
ployed at a photographic film base plant in Bran-
tham, UK from 1946 until its closure in 1988. 1,785
workers and of those, 1,473 has worked in jobs that
had exposure to methylene chloride. Information
was obtained through UK Medical Research Infor-
mation Services. Exposed workers were predomi-
nantly manual workers. Females were excluded be-
cause few had worked in production areas and many
had job titles where exposure was difficult to as-
sess. Median follow up time for exposed individu-
als was 36,8 years (IQR: 28,2-43,1) and 29,9 (IQR:
21.4-39.1) for unexposed individuals.
0.8 Attrition among the final study population with ex-
posure information was not significant. 59 v/orkers
emigrated, 10 were temporary foreign, and 10 v/ere
lost to follow-up. However, for 439 workers (30% of
study population) employment histories were insuf-
ficiently precise to calculate reliable estimates of cu-
mulative exposure-unspecified job histories and were
excluded from the dose-response analysis (most were
laborers and maintenance v/orkers).
0.4 Non -exposed group consisted of a variety of occu-
pations, but all were unlikely to have been exposed
either directly or indirectly, though very low concen-
trations may have existed. Most unexposed workers
were office v/orkers in the technical and commercial
functions, and the unexposed group tended to be
older when hired and followed up for fewer years.
The exposed group consisted of mostly manual v/ork-
ers. Mortality statistics for England and Wales v/ere
used for comparison and a comparison with local
mortalities was also made for selected causes by com-
bining mortality information from four surrounding
districts to calculate an SMR. Authors attempted to
reduce potential for healthy v/orker effect by using a
suitable internal reference group.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type: Cohort exposed workers DCM IschemicHeartDiseaseMortality Dichotomous-Cardiovascular
HERO ID: 787813
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
to
Metric 5: Exposure levels
Medium x 0.2
Metric 6: Temporality
m X 0.4 0.8 Potential exposure to DCM was contained to the
time period for which the Brantham site was open
(1946-1988). Estimates of DCM exposure were gen-
erated for 20 work groups during four production
periods (before 1960, 1960-1969, 1970-1979, 1980-
1988) at an exposure duration of 8-h TWA, ppm.
Lifetime cumulative DCM exposure was calculated
by summing the products of mean level of exposure
and duration of employment for each job held by a
cohort member. A potential limitation is that per-
sonal monitoring data before 1980 and reliable area
monitoring data before 1975 was not available; how-
ever, good historical information on the casting ma-
chines and working conditions was available, and in-
formation on the number of incidents where workers
were affected by DCM vapors was consistent with
the pattern of exposure estimated for jobs and time
periods.
0.4 Levels of cumulative exposure to methylene chloride
were categorized as 0, 0-399, 400-799, and greater
than 800 ppm-years. These cut-off points were
chosen to enable comparisons with studies of the
Rochester film workers. Trend analyses were also
performed using cut-off points (36.4 and 299.1 ppm-
years) which gave equal numbers of deaths from all
causes in the 3 cumulative exposure categories. Cu-
mulative exposure was also modeled as a continuous
variable in the Cox regression models.
X 0.4 0.4 Study participants were unlikely to have been ex-
posed to DCM after the Brantham site closed in
1988, which strengthens the validity of the expo-
sure estimates occurring prior to the outcome as-
sessed. Cumulative exposure was treated as a time-
dependent variable, both as a continuous variable
and grouped by increasing exposure.
Time since hire was included in some models (not
the regression analyses), and analyses were also per-
formed with lagged cumulative exposure (15 years).
Median follow up time for exposed individuals was
36.8 years (IQR: 28.2-43.1) and 29.9 (IQR: 21.4-
39.1) for unexposed individuals. Long follow up
times for the cohort should be sufficient for the long
latency period of some chronic diseases assessed, in-
cluding cancer.
Continued on next page

-------
.. . continued from previous page
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type: Cohort exposed workers DCM IschemicHeartDiseaseMortality Dichotomous-Cardiovascular
HERO ID: 787813
Domain Metric Rating^ MWF* Score Commentstt
Metric 12:
Study Design and Methods
m
X
0.4
0.8
Cohort study design was conducted to follow work-
ers potentially exposed to methylene chloride at a
photographic film base plant in Brantham, United
Kingdom. The outcome assessed included mortal-
ity due to different causes. The cohort study design
was appropriate for the research question involving
a relatively rare exposure, and the long follow up
time was suitable for the chronic disease outcomes
investigated.
Metric 13:
Statistical power
[m
X
0.2
0.4
The study population of 1,785 male employees, in-
cluding 1,473 exposed workers (1,034 exposed had an
exposure estimate) and 312 unexposed workers, was
sufficient to detect an effect for methylene chloride.
Statistical power not reported, but p values show
some statistically significant correlations. Pancre-
atic cancer was ultimately excluded from the regres-
sion analysis because too few cases were reported.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
Methods of statistical analysis was clearly described
and should be reproducible with information pro-
vided. Descriptions of the methods for calculating
SMRs were clearly described, and methods for re-
gression models were also described.
Metric 15:
Statistical models
Medium
X
0.2
0.4
SMRs were calculated for each mortality outcome of
interest, using the internal referent group and mor-
tality statistics for England and Wales, as well as
local mortalities from four surrounding districts and
two surrounding counties. For selected causes of
death, a multivariate regression analysis based on
Cox's proportional hazards model was conducted.
Model assumptions were met and the variables used
were clearly stated and appropriate.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Metric 22
Matrix adjustment
NA
NA
Overall Quality Determination
Medium
1.7
Continued on next page . ..

-------
. continued from previous page
Study Citation: Tomenson, J.A. (2011). Update of a cohort mortality study of workers exposed to methylene chloride employed at a plant producing
cellulose triacetate film base International Archives of Occupational and Environmental Health, 84(8,8), 889-897
Data Type:	Cohort_exposed workers_DCM_IschemicHeartDiseaseMortality_Dichotomous-Cardiovascular
HERO ID:	787813
Domain
Metric
Rating^ MWF* Score
Comments^
Extracted
Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score, x MWF;) / . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 41: Roberts et al. 2013: Evaluation of Neurological/Behavior Outcomes
Study Citation:
Data Type:
HERO ID:
Roberts, A.L., Lyall, K., Hart, J.E., Laden, F., Just, A.C., Bobb, J.F., Koenen, K.C., Ascherio, A., Weisskopf, M.G. (2013). Perinatal
air pollutant exposures and autism spectrum disorder in the children of Nurses' Health Study II participants Environmental Health
Perspectives, 121(8), 978-984
Nurses' Health Study II DCM case-control Autism endpoint males and females-Neurological/Behavior
1790951
Domain
Metric

Rating^
MWF*
Score
Comments^
Domain 1: Study Participation

Metric 1:
Participant selection

High
x 0.4
0.4
Data from the Nurses' Health Study II was used.
Study reported time frame in which all children
(cases and controls) were selected (2005-2008). Chil-
dren were born in all 50 US states. Exclu-
sion/inclusion criteria is described in the study.
Metric 2:
Attrition

High
x 0.4
0.4
The number of cases/controls included in the study
was 329 cases, 22098 controls. Reasons for excluding
subjects were clearly detailed. There was minimal
loss of subjects reported in results (325 cases/22101
controls)
Metric 3:
Comparison Group

High
x 0.2
0.2
Table 1 shows the demographic characteristics of
the cases and controls, which appear to be similar.
These include maternal age, year of birth, sex, state
of residence, smoking, income, and education infor-
mation. These were also considered in the analysis.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Low X 0.4 1.2 Exposure was determined based on the location of
the mothers beginning in 1989. Children born from
1987-1990 v/ere assigned the geographic location
of their mothers in 1989. The nurses address was
updated every other year after that and children
were assigned based on the closest date. "Hazardous
air pollutant (HAP) concentrations were assessed
by the U.S. EPA National Air Toxics Assessments in
1990, 1996, 1999, and 2002, which uses an inventory
of outdoor sources of air pollution, including
both stationary sources (e.g., waste incinerators,
small businesses) and mobile sources (e.g., traffic)
to estimate average ambient concentrations of
pollutants for each census tract based on dispersion
models (U.S. EPA 2011).R
The erratum states that the authors did not
use background exposures when determining the
qui.ni.tles in 1996, so the qui.nti.les are somewhat
different than as reported.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Roberts, A.L., Lyall, K., Hart, J.E., Laden, F., Just, A.C., Bobb, J.F., Koenen, K.C., Ascherio, A., Weisskopf, M.G. (2013). Perinatal
air pollutant exposures and autism spectrum disorder in the children of Nurses' Health Study II participants Environmental Health
Perspectives, 121(8), 978-984
Nurses' Health Study II	DCM	case-control	Autism endpoint	males and females-Neurological/Behavior
1790951
Domain
Metric
Metric 5: Exposure levels
Metric 6: Temporality
Ratingt
Medium
MWF* Score
Commentstt
High
X 0.2	0.4 Exposure levels ranged from 0.0006-41.9 ug/m3, and
divided into 5 qui.utiles. The range is sufficient to
determine a dose-response relationship
X 0.4	0.4 Exposures were measured during time and place of
birth from 1987-2002, autism spectrum disorder was
first assessed in 2005; therefore, a minimum of 3
years after exposure.
Domain 3: Outcome Assessment
Metric 8: Reporting Bias
High
x 0.667 0.67 ASD was reported by the mothers via this question
""Have any of your children been
diagnosed with the following diseases?" with autism,
Asperger's syndrome, or other ASD
listed as separate responses." The ASD diagnoses
were validated by telephone administration of the
Autism Diagnostic Interview-Revised (ADI-R), to a
randomly selected group of 50 monthers from the
study.
X 0.333 0.33 All measured outcomes were outlined in the meth-
ods, and information could be fulling extracted for
analysis. Some information was provided in supple-
mental information.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
Mediu
Medium
X 0.5	0.5 Covariates were included in the models, including:
socioeconomic indicators, smoking,
year of birth, maternal age at birth, and air pollution
prediction model year.
X 0.25 0.5 Confounders were assessed via questionnaires, but
there is no indication that the questionnaires v/ere
validated
X 0.25 0.5 Co-exposure analysis was included in the model: "To
investigate further whether one or
two pollutants were driving the association between
correlated pollutants and ASD, we
conducted analyses with diesel, lead, manganese,
cadmium, methylene chloride, and
nickel—the pollutants most strongly associated with
ASD based on tests of highest versus lowest quintile
as v/ell as linear trend—in a single model."
Domain 5: Analysis
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Roberts, A.L., Lyall, K., Hart, J.E., Laden, F., Just, A.C., Bobb, J.F., Koenen, K.C., Ascherio, A., Weisskopf, M.G. (2013). Perinatal
air pollutant exposures and autism spectrum disorder in the children of Nurses' Health Study II participants Environmental Health
Perspectives, 121(8), 978-984
Nurses' Health Study II DCM case-control Autism endpoint males and females-Neurological/Behavior
1790951
Domain
Metric
Ratine* MWF* Score
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models
Hating
Medium
Comments**
x 0.4 0.8
Medium x 0.2 0.4
Medi-
um x 0.2 0.4
Medium x 0.2 0.4
The case-control study design was appropriate for
assessing the possible association between autism
spectrum disorder and exposure to several different
compounds. The study design can get at prior ex-
posure to several exposures at once for a specific,
outcome from a large cohort.
The power was sufficient to detect effects (325 cases
and 22101 controls).
The methodology is clearly laid out, and could be re-
produced. Methods to calculate the odds ratios and
the covariates included were provided, and details
were provided on when they were not included.
Statistical methods were appropriate (calculation
of ORs, logistic regression models). Linear dose-
response was determined by dividing exposures into
qui.nti.les and using logistic regression with concen-
trations entered as a continuous independent vari-
able. Other analysis such as sex, correlation of heavy
metals, and covariate analysis v/ere employed.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Use of Biomarker of Exposure
NA
NA
Metric 17
Effect biomarker
NA
NA
Metric 18
Method Sensitivity
NA
NA
Metric 19
Biomarker stability
NA
NA
Metric 20
Sample contamination
NA
NA
Metric 21
Method requirements
NA
NA
Metric 22
Matrix adjustment
NA
NA
Overall Quality Determination* High

1.5
Extracted

MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]P. (Metric Score; X MWF;) / J] . MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 42: Christensen et al. 2013: Evaluation of Cancer Outcomes
Study Citation: Christensen, K.Y., Vizcaya, D., Richardson, H., Lavoue, J., Aronson, K., Siemiatycki, J. (2013). Risk of selected cancers due to
occupational exposure to chlorinated solvents in a case-control study in Montreal Journal of Occupational and Environmental Medicine,
55(2), 198-208
Data Type: Case-control study, occupational exposure to chlorinated solvents and various cancer types; DCM kidney cancer-Cancer
HERO ID: 2127914
Domain
Metric
Rating^
MWF*
Score
Comments^
Domain 1: Study Participation

Metric 1:
Participant selection
Medium
X
0.4
0.8
The presented work is a subset of the Montreal Can-
cer Case-Control Study, evaluating male Canadian
citizens aged 35-70 years diagnosed from 1979-1985
at the 18 largest Montreal area hospitals. Some key
elements of the study design were not present but
assumed to be present in related publications. Of

the cited studies, one was publicly available (Siemi-

atycki. et al 1987). Available information indicates a
low risk of selection bias.
Metric 2:
Attrition
Medium
X
0.4
0.8
No information was provided on subjects who de-
clined to be interviewed, but participation was rea-
sonable (82% for cases and 72% for controls). Out-
come data and exposure information were complete
for participants.
Metric 3:
Comparison Group
Medium
X
0.2
0.4
Study used both population control and cancer con-
trol groups.; both v/ere drawn from the region where
the cases were identified. Timing of the population
control selection was not reported. Characteristics
of cases and controls v/ere described.
Domain 2: Exposure Characterization

Metric 4:
Measurement of Exposure
Low
X
0.4
1.2
Exposure to a variety of chlorinated solvents, includ-

ing Perc, TCE, DCM and CCL4, was assessed based
on self-reported job history translated into exposure
by chemists and industrial hygi.eni.sts. Authors re-
ported that there was no indication that complete-
ness or validity of job histories differed between cases
and controls.
Metric 5:
Exposure levels
Medium
X
0.2
0.4
DCM exposure characterized as "any" or "substantial
exposure" (the latter assessed based on confidence,
frequency, and relative concentration of predicted
exposure). Referent group + 2 levels of exposure.
Continued on next page . ..

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Christensen, K.Y., Vizcaya, D., Richardson, H., Lavoue, J., Aronson, K., Siemiatycki, J. (2013). Risk of selected cancers due to
occupational exposure to chlorinated solvents in a case-control study in Montreal Journal of Occupational and Environmental Medicine,
55(2), 198-208
Case-control study, occupational exposure to chlorinated solvents and various cancer types; DCM kidney cancer-Cancer
2127914
Domain

Metric

Ratingt
MWF* Score
Comments^

Metric 6:
Temporality

Medium
x 0.4 0.8
Based on a related publication, (Siemiatycki et al
1987), during recruitment lung cancer cases were ex-
cluded in the second , third, and sixth years, rectal
cancer cases v/ere excluded in the first and second
year and prostate cancer case was excluded for some
of the fourth year and all of the fifth year.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization Medium
Metric 8: Reporting Bias
High
X 0.667 1.33 Cases were limited to incident, histologically con-
firmed cancers. Controls were interviewed to estab-
lish medical history for selected conditions but med-
ical records were not reviewed for confirmation.
X 0.333 0.33 Data for all outcomes (cancer incidence) and expo-
sure levels were reported in tables with measures of
precision.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
High
Medium
Metric 11: Co-exposure Confounding
Low
X 0.5 0.5 Distribution of primary covariates was reported and
did not differ substantially between groups for most
cancer types. Statistical methods for covariate ad-
justment were used.
X 0.25 0.5 Covariates and con founders assessed by subject in-
terview; there is no indication that this method had
poor validity. No method validation reported.
X 0.25 0.75 Co-exposures to other chlorinated solvents were
likely, given the overlapping job-exposure combina-
tions; the study did not control for co-exposures or
even report the distributions of co-exposures.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models
Medium x 0.4
Medium
Low
x 0.2
x 0.2
x 0.2
0.8 The large case-control study design was appropriate
for assessing risk of cancer with chlorinated solvent
exposure.
0.4 The 3730 cancer cases and 533 population controls
were sufficient to detect an effect.
0.4 Unconditional logistic regression was used to deter-
mine odds ratios (ORs). Description of analysis suf-
ficient to be conceptually reproducible.
0.6 The method for calculating risk estimates is trans-
parent, but the method for selecting covariates to
consider was not reported.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Christensen, K.Y., Vizcaya, D., Richardson, H., Lavoue, J., Aronson, K., Siemiatycki, J. (2013). Risk of selected cancers due to
occupational exposure to chlorinated solvents in a case-control study in Montreal Journal of Occupational and Environmental Medicine,
55(2), 198-208
Case-control study, occupational exposure to chlorinated solvents and various cancer types; DCM kidney cancer-Cancer
2127914
Domain

Metric
Rating1" MWF*
Score
Commentstt

Metric 16
Use of Biomarker of Exposure
NA
NA


Metric 17
Effect biomarker
NA
NA


Metric 18
Method Sensitivity
NA
NA


Metric 19
Biomarker stability
NA
NA


Metric 20
Sample contamination
NA
NA


Metric 21
Method requirements
NA
NA


Metric 22
Matrix adjustment
NA
NA

Overall Quality Determination*
Medium
2.0
Extracted
Yes


MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
y (Metric Scores x MWF,;) / ^ . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 43: Neta et al. 2012: Evaluation of Cancer Outcomes
Study Citation: Neta, G., Stewart, P.A., Rajaraman, P., Hein, M.J., Waters, M.A., Purdue, M.P., Samanic, C., Coble, J.B., binet, M.S. (2012).
Occupational exposure to chlorinated solvents and risks of glioma and meningioma in adults Occupational and Environmental Medicine,
69(11), 793-801
Data Type:	DCM	all	subjects	possibleexp	Glioma-Cancer
HERO ID:	2128240
Domain
Metric
Rating1
MWF*
Score
Comments^
Domain 1: Study Participation






Metric 1:
Participant selection
High
X
0.4
0.4
High rating: key elements of study design were re-
ported, and the reported information
indicates selection in or out of the study and partic-


High



ipation is not likely to be biased.
Metric 2:
Attrition
X
0.4
0.4
High participation rates: 92% and 94% for glioma
and meningioma cases, respectively. Participation
rate among controls was 86%
Metric 3:
Comparison Group
High
X
0.2
0.2
High rating: cases and controls were similar - con-
trols were patients admitted to the same hospitals as
cases for non-malignant conditions with frequency
matching by sex, age, race/ethnicity, hospital, and
proximity to hospital; differences in baseline
characteristics of groups were considered as poten-
tial confounding or stratification
variables (i.e,. sex and 5-year age groups) and were
thereby controlled by statistical
analysis
Domain 2: Exposure Characterization
Low




Metric 4:
Measurement of Exposure
X
0.4
1.2
Low rating: Occupational study population with ex-
posure assessed using in person interviews (i.e., no
employment records were utilized). Industrial hy-
giene experts from examined data collected in the
questionnaires, and assessed a level of probability
and levels of exposure to groups or classes of sol-
vents as well as certain individual substances.
Metric 5:
Exposure levels
Medium
X
0.2
0.4
Medium rating: range and distribution of exposure
was sufficient to develop an exposure response esti-
mate; 3 or more levels of exposure v/ere reported
Metric 6:
Temporality
High
X
0.4
0.4
High rating: temporality is established and the in-
terval between reconstructed exposure
and brain tumor risk has an appropriate considera-
tion of relevant exposure windows.
Domain 3: Outcome Assessment





Metric 7:
Outcome measurement or
characterization High
X
0.667
0.67
High rating: ICD-Onco 1 ogy codes listed; all partici-
pating case diagnoses were confirmed by microscopy
Continued on next page . ..

-------
. continued from previous page
Study Citation: Neta, G., Stewart, P.A., Rajaraman, P., Hein, M.J., Waters, M.A., Purdue, M.P., Samanic, C., Coble, J.B., Linet, M.S. (2012).
Occupational exposure to chlorinated solvents and risks of glioma and meningioma in adults Occupational and Environmental Medicine,
69(11), 793-801
Data Type: DCM all subjects possibleexp Glioma-Cancer
HERO ID: 2128240
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 8:
Reporting Bias
High
X
0.333
0.33
High rating: all of the study's measured outcomes
are reported, effect estimates reported
with confidence interval; number of exposed re-
ported for each analysis.
Domain 4: Potential Counfounding/Variable Control
High

Metric 9:
Covariate Adjustment
X
0.5
0.5
High rating: appropriate adjustments or explicit
considerations were made for potential
confounders in the final analyses through the use of
statistical models for covariate
adjustment (i.e., age group (<30, 30-49, 50-69,
70+), race (white vs non-white), sex, hospital site
and proximity of residence to the hospital)
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
Medium rating: primary confounders (excluding co-
exposures) were assessed. The paper
did not describe if the computer-based questionnaire
used to collect demographic information has been
previously validated.
Metric 11:
Co-exposure Confounding
Medium
X
0.25
0.5
Medium rating: potential co-pollutant confounding
was considered through the adjustment in statistical
models, of estimated cumulative occupational expo-
sures to lead, magnetic fields, herbicides and insecti-
cides. In addition, for ever/never analyses for partic-
ular solvents, the authors included all other solvents

in the model to account

for possible confounding by other solvent exposures.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
Medium rating: appropriate design (i.e., case control
study of chemical exposures in relation to a rare dis-
ease), and appropriate statistical methods (i.e., lo-
gistic regression analyses) were employed to analyze
data.
Metric 13:
Statistical power
Medium
X
0.2
0.4
Medium rating: the number of cases and controls are
adequate to detect an effect in the exposed popula-
tion for the primary analyses of probable/possi.ble
solvent exposure vs. unexposed in relation to risk
of glioma. The number of exposure cases of menin-
gioma was too small to have the power to conduct
stratified analyses or analyses of more detailed ex-
posure metrics.
Continued on next page . ..

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Neta, G., Stewart, P.A., Rajaraman, P., Hein, M.J., Waters, M.A., Purdue, M.P., Samanic, C., Coble, J.B., Linet, M.S. (2012).
Occupational exposure to chlorinated solvents and risks of glioma and meningioma in adults Occupational and Environmental Medicine,
69(11), 793-801
D CM	all	subj ects	possibleexp	Glioma- Cancer
2128240
Domain
Metric
Ratingt
MWF*
Score
Commentstt
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Medium rating: description of the analyses is suffi-





cient to understand what has been





done and to be reproducible with access to the data.
Metric 15
Statistical models
Medium
x 0.2
0.4
Medium rating: logistic regression models were used





to generate Odds Ratios. Rationale





for variable selection is stated. Model assumptions





are met
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination
High
1.5
Extracted
Yes


* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
(Metric Scores X MWF; )/V.mwf,-
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 44: Ruder et al. 2013: Evaluation of Cancer Outcomes
Study Citation: Ruder, A.M., Yiin, J.H., Waters, M.A., Carreon, T., Hein, M.J., Butler, M.A., Calvert, G.M., Davis-King, K.E., Schulte, P.A., Mandel,
J.S., Morton, R.F., Reding, D.J., Rosenman, K.D., Stewart, P.A., Brain Cancer Collaborative Study Group (2013). The Upper Midwest
Health Study: Gliomas and occupational exposure to chlorinated solvents Occupational and Environmental Medicine, 70(2), 73-80
Data Type: Upper Midwest Health Study DCM cumulative include proxy glioma-Cancer
HERO ID: 2128307
Domain
Metric
Rating^
MWF*
Score
Comments^
Domain 1: Study Participation

Metric 1:
Participant selection
High
X 0.4
0.4
Subjects were selected from the same area during
the same time frame. Cases were identified through
participating medical facilities and neurosurgeon of-
fices. Controls were identified from state driver's
license records.91.5% o f cases or their next of kin
participated and 70.4% of controls participated. Key
elements of the study design are reported.
Metric 2:
Attrition
High
X 0.4
0.4
Study population consisted of 1175 controls and 798

cases. 97& of the controls (1141/1175) were inter-
viewed and all cases had interviews with 360 being
proxy interviews. Some analysis was restricted to
cases that were directly interviewed.
Metric 3:
Comparison Group
High
X 0.2
0.2
Controls were randomly selected and age and sex
stratified. There were some differences in the level
of education, but this was adjusted for in the analy-
sis. Details comparing cases and controls as well as
ineligible and non-participants are detailed in com-
panion publication (Ruder et al. 2006).
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure Medium x 0.4 0.8 Complete occupational history was obtained using
a questionnaire modified from the one developed by
the National Cancer Institute. Jobs of at least one
years duration between the age of 16 and the end of
1992 were included. The questionnaire also asked
about specific exposures including solvent and on
which jobs and for how many hours a week these
exposures occurred. There is potential for cases to
have better recall. The probability, intensity, and
frequency of exposure in non-farm related jobs was
estimated based on occupation, industry, and decade
using an annotated appendix of sources of exposure
data as well as bibliographic databases of published
exposure levels. Complete descriptions of the meth-
ods were provided. JEM with complete job history,
but based on recalled jobs and some judgement on
exposure (although used several cited references).
Continued on next page . ..

-------
.. . continued from previous page
Study Citation: Ruder, A.M., Yiin, J.H., Waters, M.A., Carreon, T., Hein, M.J., Butler, M.A., Calvert, G.M., Davis-King, K.E., Schulte, P.A., Mandel,
J.S., Morton, R.F., Reding, D.J., Rosenman, K.D., Stewart, P.A., Brain Cancer Collaborative Study Group (2013). The Upper Midwest
Health Study: Gliomas and occupational exposure to chlorinated solvents Occupational and Environmental Medicine, 70(2), 73-80
Data Type: Upper Midwest Health Study DCM cumulative include proxy glioma-Cancer
HERO ID: 2128307
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 5:
Exposure levels
Medium
x 0.2
0.4
Exposure was estimated in cumulative exposure of
ppm-h and ppm-years.
Metric 6:
Temporality
Medium
x 0.4
0.8
Temporality is established, but it is unclear whether
exposures fall within relevant exposure windows for
the outcome of interest. Case diagnosis occurred be-
tween 1995 and 1997 with job history ending in 1992.
Domain 3: Outcome Assessment

Metric 7:
Outcome measurement or characterization
High
x 0.667
0.67
The study focused on histologically confirmed pri-
mary intracranial gliomas (ICD-O code 938-948).
Metric 8:
Reporting Bias
High
x 0.333
0.33
Sufficient information was reported. Effect esti-
mates are reported with a confidence interval.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
Medium
x 0.5
1
Adjusted for age group, sex, age, and education.
Metric 10:
Covariate Characterization
Medium
x 0.25
0.5
Information was obtained via a questionnaire some-
times via proxy.
Metric 11:
Co-exposure Confounding
Medium
x 0.25
0.5
Although this was occupational exposure, they in-
cluded people from different jobs at different times
and it is unlikely that there would be differential
co-exposures.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
x 0.4
0.8
Methods are appropriate and appropriate statistical
methods were used to address research question.
Metric 13:
Statistical power
Medium
x 0.2
0.4
The study included 798 cases and 1175 controls,
which is likely to provide sufficient statistical power.
For any given exposure there were more than 100
subjects except when evaluating women only or a
subset excluding proxy only. In these cases there
were as few as 34 subjects.
Metric 14:
Reproducibility of analyses
m
x 0.2
0.4
Enough information is provided to be reproducible
if data were available.
Metric 15:
Statistical models
ivieui um
x 0.2
0.4
Unconditional logistic regression models were used,
which were appropriate for the data and assump-
tions appear to have been met.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:
Use of Biomarker of Exposure

NA
NA

Metric 17:
Effect biomarker

NA
NA

Metric 18:
Method Sensitivity

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Ruder, A.M., Yiin, J.H., Waters, M.A., Carreon, T., Hein, M.J., Butler, M.A., Calvert, G.M., Davis-King, K.E., Schulte, P.A., Mandel,
J.S., Morton, R.F., Reding, D.J., Rosenman, K.D., Stewart, P.A., Brain Cancer Collaborative Study Group (2013). The Upper Midwest
Health Study: Gliomas and occupational exposure to chlorinated solvents Occupational and Environmental Medicine, 70(2), 73-80
Data Type:	Upper Midwest Health Study	DCM	cumulative	include proxy	glioma-Cancer
HERO ID:	2128307
Domain

Metric
Rating1" MWF*
Score
Comments^

Metric 19:
Biomarker stability
NA
NA


Metric 20:
Sample contamination
NA
NA


Metric 21:
Method requirements
NA
NA


Metric 22:
Matrix adjustment
NA
NA

Overall Quality Determination	High	1.6
Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T. (Metric Score; X MWF;) / ]T\ MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 45: Vizcaya et al. 2013: Evaluation of Cancer Outcomes
Study Citation: Vizcaya, D; Chris tensen, KY; Lavoue, J; Siemiatycki, J (2013). Risk of lung cancer associated with six types of chlorinated solvents:
Results from two case-control studies in Montreal, Canada Occupational and Environmental Medicine, 70(2), 81-85
Data Type: occupational case-control study Montreal (DCM substantial exposure pooled analysis extraction) - Cancer
HERO ID: 2128435
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
dium x 0.4 0.8

Low x 0.4 1.2
High x 0.2 0.2
This was a population based case-control study in
which subjects were restricted to Canadian citizens
who v/ere residents in the Montreal metropolitan
area. This report did not describe case ascertain-
ment, but cited references (HERO ID 2856585 and
091275) which indicate that histologically confirmed
cancer patients from 18 of the largest hospitals were
used as cases. Controls v/ere randomly selected fre-
quency matched by age and sex. Participation rates
were provided and v/ere slightly higher in the cases.
There appears to be a large amount of attrition that
v/as not adequately explained. It is likely that the
missing subjects from Table 1 did not have occupa-
tions with exposure codes.
Cases v/ere more likely to be French Canadians than
controls. Controls v/ere on average wealthier and
had a higher education. Cases were heavier smokers
than controls. These v/ere all controlled for in the
analysis.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
Low x 0.4 1.2
m x 0.2 0.4
Low x 0.4 1.2
A semi-structured questionnaire was used to obtain
details of each job that lasted at least 6 months. A
team of industrial chemists and hygi.eni.sts examined
each subject's questionnaire and translated each job
into potential exposures from a list of 294 substances
without knowledge of the subject's status. Exposure
based on collective judgement.
Only two groups v/ere compared and could not be
evaluated for trend. Exposed groups v/ere never ex-
posed, ever exposed, or substantial exposure.
The temporality of exposure and outcome is uncer-
tain. Although job history v/as obtained, there is no
information provided to determine that the jobs oc-
curred before diagnosis or even if the jobs v/ere prior
to diagnosis there is no information provided on how
long or how close to the diagnosis the jobs occurred.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: Vizcaya, D; Christensen, KY; Lavoue, J; Siemiatycki, J (2013). Risk of lung cancer associated with six types of chlorinated solvents:
Results from two case-control studies in Montreal, Canada Occupational and Environmental Medicine, 70(2), 81-85
Data Type: occupational case-control study Montreal (DCM substantial exposure pooled analysis extraction) - Cancer
HERO ID: 2128435
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 7:
Outcome measurement or characterization
High
X
0.667
0.67
Cases were histologically confirmed.
Metric 8:
Reporting Bias
High
X
0.333
0.33
Results were reported in sufficient details. A de-
scription of measured outcomes is reported in the
methods, abstract, and/or introduction. Effect es-
timates are reported with a confidence interval and
the number of cases/controls are reported for each
analysis.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
High
X
0.5
0.5
Results were adjusted by age, smoking habit, edu-
cational attainment, SES, and ethnicity.
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
Information was obtained from a questionnaire of
unknown reliability and validity. The authors note
that "Although it is very difficult to establish the va-
lidity of retrospective exposure assessments, we have
demonstrated satisfactory levels of reliability and va-
lidity in the job histories and in the expert exposure
assessments.'
Metric 11:
Co-exposure Confounding
Medium
X
0.25
0.5
It was noted that results were adjusted for exposure

to eight known carcinogens. Although there are po-
tential co-exposures for any given job, it is unlikely
that they were differential across jobs and within the
specific chemicals of interest. Supplemental Table
S2 indicated 5 different jobs with exposure to DCM
making it unlikely that co-exposure was consistent
across all 5 jobs in each category.
Domain 5: Analysis

Metric 12:
Study Design and Methods
Medium
X
0.4
0.8
Study design and statistical method were appropri-
ate for the research question. A case-control study
is the best design to study lung cancers when evalu-
ating many different possible exposures across mul-
tiple different jobs. The use of unconditional logistic
regression is appropriate for this data.
Metric 13:
Statistical power
m
X
0.2
0.4
Statistical pov/er should be sufficient. However,
some substantial exposure categories had a small
number of subjects.
Metric 14:
Reproducibility of analyses
Medium
X
0.2
0.4
The description of the unconditional logistic regres-
sion analysis used for estimates of odds ratios and
the confounders included is sufficient to understand
precisely what has been done and to be conceptually
reproducible with access to the analytic data.
Continued on next page . ..

-------
. continued from previous page
Study Citation: Vizcaya, D; Christensen, KY; Lavoue, J; Siemiatycki, J (2013). Risk of lung cancer associated with six types of chlorinated solvents:
Results from two case-control studies in Montreal, Canada Occupational and Environmental Medicine, 70(2), 81-85
Data Type:	occupational case-control study Montreal (DCM substantial exposure pooled analysis extraction) - Cancer
HERO ID:	2128435
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 15
Statistical models
m
x 0.2
0.4
The method for calculating the risk estimates (i.e.
odds ratios) is transparent and the model assump-
tions were met.
Domain 6: Other Considerat
Metric 16
Metric 17
Metric 18
Metric 19
Metric 20
Metric 21
Metric 22
ions for Biomarker Selection and Measurement
Use of Biomarker of Exposure
Effect biomarker
Method Sensitivity
Biomarker stability
Sample contamination
Method requirements
Matrix adjustment

NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA

Overall Quality Determination*
Medium

1.9

Extracted

Yes



MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
£ (Metric Score; x MWF;) / J] . MWF.;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type

-------
Table 46: Morales-Suarez-Varela et al. 2013: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Morales-Suarez-Varela, MM; Olsen, J; Villeneuve, S; Johansen, P; Kaerlev, L; Llopis-Gonzalez, A; Wingren, G; Hardell, L; Ahrens, W;
Stang, A; Merletti, F; Gorini, G; Aurrekoetxea, J J; Fevotte, J; Cyr, D; Guenel, P (2013). Occupational exposure to chlorinated and
petroleum solvents and mycosis fungoides Journal of Occupational and Environmental Medicine, 55(8), 924-931
Case-Control Occupational DCM MycosisPungoides OR_aboveMedian All-Cancer
2129849
Domain
Metric
t MWF*
Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
High x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
Medium x 0.4
x 0.2
High

Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure

0.4 140 cases ascertained from requests to hospitals and
pathology department, as well as regional/national
cancer and pathology registers. Patients from 6 Eu-
ropean countries: Denmark, Sweden, France, Ger-
many, Italy, and Spain. Controls from these coun-
tries selected from population registries or colon can-
cer registries. As such, the reported information in-
dicates selection in or out of the study and partici-
pation is not likely to be biased.
0.8 Moderate attrition due to patents removed from
study due to unconfirmed diagnosis (22) or lack of
availability for interview (18); participation rate of
84.75%. Of the eligible controls, 68.2% (3156) were
interviewed; only controls within the strata (5 year
age + gender) of MF patients used (2846).
0.2 Key elements of the study design are reported indi-
cate that that cases and controls were similar (e.g.,
recruited from the same eligible population with the
number of controls described, and eligibility crite-
ria and are recruited within the same time frame.
Specifically, 4 controls/case, frequency matched by
sex and age (5 years). Population registries and elec-
toral rolls used to select controls in Denmark, Swe-
den, France, Germany and Italy. Spanish controls
from colon cancer patients (no population register).
X 0.4 1.2 Interviews with standardized questionnaires to de-
termine occupational history. Next of kin completed
interviews for 4 cases and 95 controls. Exposure de-
termined with J EM developed by the French Insti-
tute of Health Surveillance using jobs/industries as-
signed based on interviews by trained coders using
international standards.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Morales-Suarez-Varela, MM; Olsen, J; Villeneuve, S; Johansen, P; Kaerlev, L; Llopis-Gonzalez, A; Wingren, G; Hardell, L; Ahrens, W;
Stang, A; Merle tti, F; Gorini, G; Aurrekoetxea, J J; Fevotte, J; Cyr, D; Guenel, P (2013). Occupational exposure to chlorinated and
petroleum solvents and mycosis fungoides Journal of Occupational and Environmental Medicine, 55(8), 924-931
Case-Control Occupational DCM MycoeisFungoides OR_aboveMedian All-Cancer
2129849
Domain
Metric
Metric 5: Exposure levels
Metric 6: Temporality
Rating^
Medium
MWF* Score
Comments^
X 0.2 0.4 Multiple levels of exposure. Classified by probabil-
ity of exposure, exposure frequency, and exposure
intensity. Results reported according to unexposed,
above median and below median. Details of expo-
sure intensity by chemical not reported. Sufficient
exposure to detect an effect.
X 0.4 0.4 Temporality is established and the interval between
the exposure (or reconstructed exposure) and the
outcome has an appropriate consideration of rele-
vant exposure windows. Specifically, the authors
considered lag times of 5, 10, or 15 years, which did
not make an impact (results not presented).
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High

Metric 8: Reporting Bias

High

X 0.667 0.67 Clinical and pathological mycosis fungoides (MF)
diagnosis from cancer/pathology registers and re-
quests of hospitals, using ICD codes. All diagnosis
were reviewed by the same pathologist for adherence
to morphological and topographical MF criteria; 22
cases were excluded on this basis.
X 0.333 0.33 The results discussed in the i. nt r o d u c t io n / met ho d s
were fully provided and extractable. All of the
study's measured outcomes are reported, effect es-
timates reported with confidence interval; number
of cases and controls reported for each analysis.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
High
x 0.5
Metric 11: Co-exposure Confounding
Medium x 0.25 0.5
Medium x 0.25
0.5 Confounders considered in adjusted analy-
sis; age, sex, country, current smoking habit
(cigarettes/day), alcohol intake, BMI, and educa-
tion level.
Primary confounders were assessed using a less-
established method with no reporting of validation
against we 11-estab 1 ished methods. Specifically, co-
variates were determined from interviews. Next of
kin completed interviews for 4 cases and 95 controls.
0.5 Co-exposures were not accounted for in this analysis,
but no direct evidence that co-exposures differ across
cases and controls.
Domain 5: Analysis
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Morales-Suarez-Varela, MM; Olsen, J; Villeneuve, S; Johansen, P; Kaerlev, L; Llopis-Gonzalez, A; Wingren, G; Hardell, L; Ahrens, W;
Stang, A; Merle tti, F; Gorini, G; Aurrekoetxea, J J; Fevotte, J; Cyr, D; Guenel, P (2013). Occupational exposure to chlorinated and
petroleum solvents and mycosis fungoides Journal of Occupational and Environmental Medicine, 55(8), 924-931
Case-Control	Occupational	DCM	MycoeisFungoides	OR_aboveMedian	All-Cancer
2129849
Domain
Metric
Rating*
MWF*
Score
Commentstt
Metric 12
Study Design and Methods
Medium
x 0.4
0.8
Case-control design was appropriate for investigat-





ing chlorinated solvents and a rare disease such as





MF, and appropriate statistical methods (logistic re-





gression) were employed to analyze data.
Metric 13
Statistical power
Medium
x 0.2
0.4
100 cases and 2846 controls. Exposed cases rela-





tively low (27 trichloroethy 1 ene, 6 perchloroethylene,





9 methylene chloride), but sufficient to detect an ef-
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Descri.pti.on of the analyses is sufficient to under-





stand what has been done and to be reproducible





with access to the data.
Metric 15
Statistical models
Medium
x 0.2
0.4
The model used for calculating risk estimate (i.e.,





odds ratios using logistic regression) is fully appro-





priate. Rationale for covariate selection is not pro-





vided, but model assumptions do not appear to be





violated.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
High

1.6

Extracted

Yes



* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
(Metric Score; X MWF;) / J] . MWF,
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 47: von Ehrensfein et al. 2014: Evaluation of Neurological/Behavior Outcomes
Study Citation: von Ehrenstein, OS; Aralis, H; Cockburn, M; Ritz, B (2014). In utero exposure to toxic air pollutants and risk of childhood autism
Epidemiology, 25(6), 851-858
Data Type: Case-Control DCM Childhood Autism OR_5km-Neurological/Behavior
HERO ID: 2453135
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
Medium x 0.4
High
X 0.4 0.4 Key elements of the study design are reported: chil-
dren born 1995-2006 to mothers res id i. ng within 5
km of ai.r-toxi.cs monitoring stations in Los Ange-
les County. Birth records linked to records of diag-
nosis of primary autistic disorder at the California
Department of Deve 1 opmenta 1 Services (1998-2009).
The reported information indicates selection in or
out of the study and participation is not likely to be
biased.
0.8 Moderate loss or exclusion of subjects: Linked 80%
of case records. Total cohort of 148,722 births were
included in the analysis. Birth records with im-
plausible gestational lengths or birth weights ex-
cluded (n=1436), and children who died before age
6 (n=492).
X 0.2 0.2 Differences in baseline characteristics of groups were
considered as potential confounding or stratification
variables and were thereby controlled by statisti-
cal analysis. Comparison group selected from some
regions and birth registries. Cases were predomi-
nantly male (81%), while controls were evenly dis-
tributed between genders. Cases had older moth-
ers with more education and a higher percentage of
private insurance. Potential that these factors may
have increased diagnosis, which were adjusted for in
the analysis.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
High
x 0.4
Metric 5: Exposure levels
Medium x 0.2
0.4 Exposure assessment is based on direct measurement
data of PCE, TCE, and DCM in air during the ac-
tual months of pregnancy in close proximity of the
mother's
residence: exposure for each trimester and entire
pregnancy estimated from air-toxics monitoring sta-
tions within 3-5 km of maternal address. Considered
24 pollutants with available data.
0.4 Average exposure per trimester and pregnancy pro-
vide continuous metrics sufficient to detect an
exposure-response estimate.
Continued on next page

-------
. continued from previous page
Study Citation: von Ehrenstein, OS; Aralis, H; Cockburn, M; Ritz, B (2014). In utero exposure to toxic air pollutants and risk of childhood autism
Epidemiology, 25(6), 851-858
Data Type: Case-Control DCM Childhood Autism OR_5km-Neurological/Behavior
HERO ID: 2453135
Domain
Metric

Rating^
MWF*
Score
Commentstt
Metric 6:
Temporality

High
X
0.4
0.4
Study tracks maternal exposure during pregnancy
and captures children until 6 years old, which es-
tablishes temporality and covers the critical expo-
sure window and expected diagnostic time.
Domain 3: Outcome Assessment

Metric 7:
Outcome measurement or characterization
High
X
0.667
0.67
Autism cases from the California Department of De-

velopmental Services diagnosed with severe autism
at 36-71 months (1998-2009) using the Diagnostic

and Statistical Manual of Mental Disorders. Valida-

tion studies are cited. Expressive-language pheno-
type was used a measure of severity. Possibility that
some controls are cases, if did not utilize the state
services (moved out of state, alternative treatments,
not aware of services offered),. However, this is un-
likely to result in differential reporting of autism by
exposure status.
Metric 8:
Reporting Bias

High
X
0.333
0.33
The results discussed in the i. nt r o d u c, t io n / met ho d s
were fully provided and extractable. Effect esti-

mates reported with confidence interval; number of

cases reported for each analysis.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment

High
X
0.5
0.5
Appropriate adjustments or explicit considerations
were made for potential con founders in the final
analyses through the use of statistical models for co-
variate adjustment. Specifically, risk estimates were
adjusted for maternal age, race/ethnicity, nativity,
education, insurance type (SES surrogate), mater-

Medium

nal birth place, parity, child sex, and birth year.
Metric 10:
Covariate Characterization

X
0.25
0.5
Source of covariate data not stated (presumed to be
the birth and diagnosis records), and it is unknown
whether method validation was conducted. How-
ever, there is little to no evidence that the source
was expected to introduce systematic bias.
Metric 11:
Co-exposure Confounding

Medium
X
0.25
0.5
The study considered the correlated nature of
the pollutant mixture. Specifically, perchloroethy-
lene was highly correlated (>90%) with benzene,
1,3-butadi.ene, toluene and ortho-xylene. How-
ever, methylene chloride and trichloroethylene not
strongly correlated with other pollutants. Moreover,
there does not appear to be direct evidence of an un-
balanced provision of additional co-exposures across
the primary study groups.
Continued on next page

-------
. continued from previous page
Study Citation: von Ehrenstein, OS; Aralis, H; Cockburn, M; Ritz, B (2014). In utero exposure to toxic air pollutants and risk of childhood autism
Epidemiology, 25(6), 851-858
Data Type:	Case-Control	DCM	Childhood	Autism	OR_5km-Neurological/Behavior
HERO ID:	2453135
Domain
Metric
Rating^
MWF*
Score
Comments^
Domain 5: Analysis





Metric 12
Study Design and Methods
im
x 0.4
0.8
Appropriate design (i.e., retrospective cohort for





assessment of a rare disease in relation to





PCE/TCE/DCM exposure, and appropriate statis-





tical methods (i.e., unconditional logistic regression





models) were employed to analyze data.
Metric 13
Statistical power
Medium
x 0.2
0.4
Sufficient study size to detect an effect. In the analy-





sis of risk of autism associated with exposures within





a 5 km buffer, there were 619 cases exposed to PCE,





641 cases exposed to DCM, and 624 cases exposed





to TCE (Table 2).
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Sufficient detail to understand analysis and repro-





duce if provided with all data.
Metric 15
Statistical models
Medium
x 0.2
0.4
Logistic regression modeling was used to generate





ORs. Rationale for variable selection is stated.





Model assumptions do not appear to be violated.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
High

1.4

Extracted	Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
I	if any metric is Unacceptable
:d
ride s
Overall rating =
]T\ (Metric Score,; x MWF;) / J] . MWF;
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 48: Talibov et al. 2014: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Talibov, M; Lehtinen-Jacks, S; Martinsen, JI; Kjserheim, K; Lynge, E; Sparen, P; Tryggvadottir, L; Weiderpass, E; Kauppinen, T;
Kyyronen, P; Pukkala, E (2014). Occupational exposure to solvents and acute myeloid leukemia: A population-based, case-control
study in four Nordic countries Scandinavian Journal of Work, Environment and Health, 40(5), 511-517
DCM nested case-control exposed workers AML cancer moderate-Cancer
2799600

Domain
Metric
Ratine^
MWF*
Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
High

Metric 2: Attrition
High
Metric 3: Comparison Group
X 0.4 0.4 Nested case-control study included cases and con-
trols identified from the Nordic Occupational Can-
cer Study (NOCCA) cohort. 15,332 incident cases
of AML diagnosed in Finland, Norway, Sweden and
Iceland from 1961-2005 and 76,660 controls matched
by year of birth, sex, and country included. Five
controls per case were randomly selected among per-
sons who v/ere alive and free from AML on the date
of diagnosis of the case (hereafter the "index date"
of the case-control set). Cases and controls could
have a history of any cancer other than AML and
were matched for the year of birth, sex, and coun-
try. Persons with minimum age of 20 years at index
date, and having occupational information from at
least one census record, were included in the present
study.
X 0.4 0.4 Cases and controls selected from very large cohort.
No subjects from Denmark were included because
individual records were not available. Initial sub-
jects were 1,5332 cases of AML in Finland, Norway,
Sweden, and Iceland diagnosed from 1961-2005 and
76,600 controls matched by year of birth, sex, and
country (5 matched controls per case). Of these, 350
cases (2.3%) and 2155 controls (2.8%) were excluded
because they were either <20 years or had no occu-
pational record.
X 0.2 0.2 Cases diagnosed from 1961-2005 and controls were
matched by year of birth, sex, and country (5
matched controls per case). For exposure analy-
sis (cases and controls combined), the comparison
group was unexposed based on JEM. No evidence
groups v/ere not similar.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Talibov, M; Lehtinen-Jacks, S; Martinsen, JI; Kj»rheim, K; Lynge, E; Sparen, P; Tryggvadottir, L; Weiderpass, E; Kauppinen, T;
Kyyronen, P; Pukkala, E (2014). Occupational exposure to solvents and acute myeloid leukemia: A population-based, case-control
study in four Nordic countries Scandinavian Journal of Work, Environment and Health, 40(5), 511-517
DCM	nested case-control	exposed workers	AML	cancer	moderate-Cancer
2799600
Domain
Metric
Ratingt
Medium
Metric 4: Measurement of Exposure
MWF* Score
Commentstt
x 0.4

Metric 5: Exposure levels
Medium x 0.2
0.8 Exposure to solvents and other occupational factors
was estimated based on conversion of occupational
codes to quantitative amounts of exposure with the
NOCCA job exposure matrix. Census records v/ere
used to determine occupational information for all
subjects which was then interpreted using the job
exposure matrix which covers 300 occupations and
29 exposure agents for periods: 1945-59, 1960-74,
1975-84, 1985-94. Estimates take into account pro-
portion of exposed, mean level of exposure in ex-
posed in specific time period and occupation. Cu-
mulative exposure estimated based on entire working
career. Main analysis only included exposures that
occurred prior to 10 years before index date (impor-
tance of earlier exposures for AML). Some potential
for exposure misclassifi cation due to: 1) heterogene-
ity in exposure levels within jobs, and 2)i.ndi.vidua!
work histories were based on census records that are
a snapshot of a job held by individual at the time
of the census. The data did not provide information
on the changes of the job or tasks during the entire
working career of an individual. In this study, we
assumed that an individual held his/her occupation
until the mid-year between two censuses.
0.4 Study selected values corresponding to the 50th
and 90th percentiles of cumulative exposure dis-
tribution among all exposed case/control subjects
as cut-off points for categorization. Defined expo-
sure values of 0—50th percentile inclusive as "low"
(TCE: <= 16.2 ppm/year; DCM: < = 9.9 ppm/year;
Perc: <-12.1 ppm/year), 50—90th percentile inclu-
sive as "moderate" (TCE: 16.2-121 ppm/year; DCM:
9.9-64.6 ppm/year; Perc: 12.1-106 ppm/year), and
>90th percentile of exposure distribution as "high"
(TCE: >121 ppm/year; DCM: >64.6 ppm/year;
Perc: >106 ppm/year). Individuals with 0 exposure
were used as the reference group.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Talibov, M; Lehtinen-Jacks, S; Martinsen, JI; Kj»rheim, K; Lynge, E; Sparen, P; Tryggvadottir, L; Weiderpass, E; Kauppinen, T;
Kyyronen, P; Pukkala, E (2014). Occupational exposure to solvents and acute myeloid leukemia: A population-based, case-control
study in four Nordic countries Scandinavian Journal of Work, Environment and Health, 40(5), 511-517
DCM nested case-control exposed workers AML cancer moderate-Cancer
2799600
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 6: Temporality
High X 0.4 0.4 Cumulative exposure estimated based on entire
working career, capturing all relevant exposure in-
formation. Main analysis only included exposures
that occurred prior to 10 years before index date
(importance of earlier exposures for AML). Study
sufficiently accounted for the long latency period of
AML.
Domain 3:
Outcome Assessment
Metric 7: Outcome measurement or characterization High
x 0.667 0.67
Metric 8: Reporting Bias
Medium x 0.333 0.67
Census records were linked to data from cancer reg-
istries and national population registries for infor-
mation on cancer, death and emigration. Acute
Myeloid Leukemia (AML) cases identified from
Nordic cancer registries, which are valid sources for
outcome measurement. Study does not provide sub-
stantial detail on the use of these registries.
The number of cases and controls in the "no expo-
sure" group used as a referent group was not explic-
itly stated, but can be calculated based on reported
total number of cases and control and reported sub-
ject numbers in low-, moderate, and high-exposure
groups. Data not shown for all of the analyses (e.g.
different lag-times). Sufficient description of mea-
sured outcomes is reported. Hazard Ratios with 95%
confidence intervals reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Medium x 0.5
Metric 10: Covariate Characterization
High
x 0.25
0.25
Controls were matched for sex, age, and country.
Analyses were stratified by sex and age. All analyses
were also done with different lag time assumptions.
Study did not control for smoking and genetic fac-
tors that have been previously linked to AML. Au-
thors note that smoking and genetic factors would
likely only have a minor confounding effect on the
estimates.
Sex, age, and country were all determined based
on valid Nordic national censuses (Finland, Iceland,
Norway, Sweden) in 1960, 1970, 1980/1981, and/or
1990.
Continued on next page

Medium x 0.4
Medium x 0.2
Medium x 0.2
Medium x 0.2
0.8 Nested case-control study within the larger Nordic
Occupational Cancer Study (NOCCA) cohort was
an appropriate study design to investigate the im-
pact of exposures on acute myeloid leukemia. Expo-
sure determined from job exposure matrices. Haz-
ard ratios with 95% confidence intervals estimated
by conditional logistic regression, which is appropri-
ated for the nested case-control design.
0.4 Study has large number of participants adequate to
detect an effect in the exposure population and sub-
groups (15,332 cases and 76,660 controls). Study
authors state: "These numbers are so high that our
study is unlikely to lack pov/er and miss an effect
should one exist in our data."
0.4 Detailed description of analysis is provided, includ-
ing process for selection variables and rationale for
stratification (see metric 15).
0.4 Model for calculating hazard ratio transparent and
all model assumptions were met. Conditional logis-
tic regression was used to estimate hazard ratios and
95% confidence intervals. Test for trend was per-
formed for a dose-response relationship between ex-
posure factors and AML. Variable selection for the
final main-effects model was based on the "purpose-
ful covariate selection" procedure. Two alternative
main-effects models included (see above). Analyses
stratified by age and sex was conducted to explore
potential age- and sex-specific interactions with ex-
posure. All analyses v/ere done with different lag-
time assumptions (0, 3, 5, 7, 10, and 20 years).
Domain 6: Other Considerations for Biomarker Selection and Measurement
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Talibov, M; Lehtinen-Jacks, S; Martinsen, JI; Kjasrheim, K; Lynge, E; Sparen, P; Tryggvadottir, L; Weiderpass, E; Kauppinen, T;
Kyyronen, P; Pukkala, E (2014). Occupational exposure to solvents and acute myeloid leukemia: A population-based, case-control
study in four Nordic countries Scandinavian Journal of Work, Environment and Health, 40(5), 511-517
DCM_nested case-control_exposed workers_AML_cancer_moderate-Cancer
2799600
Domain

Metric
Rating1" MWF*
Score
Commentstt

Metric 16
Use of Biomarker of Exposure
NA
NA


Metric 17
Effect biomarker
NA
NA


Metric 18
Method Sensitivity
NA
NA


Metric 19
Biomarker stability
NA
NA


Metric 20
Sample contamination
NA
NA


Metric 21
Method requirements
NA
NA


Metric 22
Matrix adjustment
NA
NA

Overall Quality Determination*
High
1.5
Extracted
Yes


MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
y (Metric Scores x MWF,;) / ^ . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 49: Mattei et al. 2014: Evaluation of Cancer Outcomes
Study Citation: Mattei, F; Guida, F; Matrat, M; Cenee, S; Cyr, D; Sanchez, M; Radoi, L; Menvielle, G; Jellouli, F; Carton, M; Bara, S; Marrer, E; Luce,
D; Strieker, I (2014). Exposure to chlorinated solvents and lung cancer: Results of the ICARE study Occupational and Environmental
Medicine, 71(10), 681-689
Data Type: ICARE cohort (DCM men CEI 1)-Cancer
HERO ID: 2799644

Domain Metric
Ratingt
MWF*
Score
Comments^

Domain 1: Study Participation

Metric 1: Participant selection
High
x 0.4
0.4
This is a is French multi-center population-based
case-control study conducted from 2001-2007. It in-
cluded a cancer registry. Case recruitment was per-
formed in collaboration with the French network of
cancer registries. Popu 1 ati.on-based controls v/ere se-
lected by incidence density sampling. All steps of
the participation v/ere provided.

Metric 2: Attrition
Medium
x 0.4
0.8
All attrition was clearly recorded. 10% of eligible
cases could not be located. 16% died, and 5% could
not be interviewed because of health status. 87%

of those remaining agreed to participate. 94% of

eligible controls were contacted and 81% agreed to
to

participate. There were a few subjects that were not
included in the analysis based on the numbers in the
table with out explanation, but this was < 10%.

Metric 3: Comparison Group
High
x 0.2
0.2
Controls v/ere selected based on incidence density
sampling and v/ere frequency matched to cases by
gender and age with further stratification to make
SES distribution comparable to the general popu-
lation living in the departments. Cases v/ere more
likely to be current smokers, but this was addressed
in the analysis.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure
Low
x 0.4
1.2
Data was collected via a questionnaire. For each job
held for at least 1 month, information v/as collected
on the tasks and specific exposures of interest. TCE
was the only chlorinated solvent specifically listed
and Perc, v/as stated to be the one agent that v/as
self-reported. Chlorinated solvents were assessed us-
ing a JEM. For each combination of ISCO and NAF
codes, J EM assigned three indices of exposure 1)
probability of exposure, 2) intensity of exposure, and
3) frequency of exposure. JEM provided an aver-
age level of exposure during a usual work day. Cu-
mulative Exposure Index (CEI) was calculated and
transformed into categorical variables. However, it
appears that exposure is solely based on self-report
and professional judgement.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Mattei, F; Guida, F; Matrat, M; Cenee, S; Cyr, D; Sanchez, M; Radoi, L; Menvielle, G; Jellouli, F; Carton, M; Bara, S; Marrer, E; Luce,
D; Stiicker, I (2014). Exposure to chlorinated solvents and lung cancer: Results of the ICARE study Occupational and Environmental
Medicine, 71(10), 681-689
ICARE cohort (DCM men CEI 1)-Cancer
2799644
Domain
Metric
Metric 5: Exposure levels
Metric 6: Temporality
Hating
Medium
Low
MWF* Score
Commentstt
X 0.2 0.4 Each chemical had at least 3 levels (control + 2 or
more CEI levels)
X 0.4 1.2 The temporality of exposure and outcome is uncer-
tain.

Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High x 0.667 0.67
Metric 8: Reporting Bias High x 0.333 0.33
All cases were histologically confirmed.
Sufficient details were provided.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
x 0.5
Medium x 0.25
Low
x 0.25
0.5 Confounders adjusted for included age at interview,
department, smoking history, number of jobs, and
SES. Genders were evaluated separately.
0.5 Information was obtained from a questionnaire with-
out reporting reliability or validity of the question-
naire.
0.75 Exposure to asbestos was adjusted for in the anal-
ysis. It was noted that exposure to one solvent
did not preclude exposure to the others, subjects
were categorized in into mutually exclusive exposure
groups according to various combinations of specific,
solvents. Combinations were evaluated separately.
However, it appears that there may be too much
correlation between exposure to some chemicals.
Domain 5: Analysis
Metric 12
Metric 13
Metric 14
Metric 15
Study Design and Methods Medium
Statistical power Medium
Reproducibility of analyses Medium
Statistical models Medium
x 0.4
x 0.2
x 0.2
x 0.2
0.8
0.4
0.4
0.4
Method is acceptable.
Likely sufficient.
Information was sufficient.
Methods are transparent and assumptions were met.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure
NA
NA

Metric 17
Effect biomarker
NA
NA

Metric 18
Method Sensitivity
NA
NA

Metric 19
Biomarker stability
NA
NA

Metric 20
Sample contamination
NA
NA

Metric 21
Method requirements
NA
NA

Metric 22
Matrix adjustment
NA
NA

Continued on next page . ..

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Mattei, F; Guida, F; Mat rat, M; Cenee, S; Cyr, D; Sanchez, M; Radoi, L; Menvielle, G; Jellouli, F; Carton, M; Bara, S; Marrer, E; Luce,
D; Stucker, I (2014). Exposure to chlorinated solvents and lung cancer: Results of the ICARE study Occupational and Environmenta 1
Medicine, 71(10), 681-689
ICARE cohort (DCM men CEI 1)-Cancer
2799644
Domain
Metric
Rating^ MWF* Score
Comments^
Overall Quality Determination'

Medium 1.8

Extracted

Yes


MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
y (Metric Scores x MWF.j) / . MWF?
z	^
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 50: Brender et al. 2014: Evaluation of Cardiovascular Outcomes
Study Citation: Brender, JD; Shinde, MTJ; Zhan, FB; Gong, X; Langlois, PH (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type: Developmental toxicity- septal heart defects methylene chloride-Cardiovascular
HERO ID: 2799700
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
High
Metric 3: Comparison Group
x 0.4 0.4 The key elements of the study design are reported
(including methods of case ascertainment); the in-
formation seems to indicate that selection for the
study was not biased.
X 0.4 0.4 Exclusion from the analysis sample was largely lim-
ited to elective terminations; however it was docu-
mented why they were excluded (lack of linkage to
a vital record).
X 0.2 0.2 Cases and controls v/ere recruited from the same
population (in Texas), during the same time period
(1996—2008) and within the same public health ser-
vice region (1 Iregions). The eligibility criteria for
cases (diagnosis of one of the selected birth defects)
was defined. Differences in baseline characteristics
(e.g., race/ethnicity, education) were controlled for
in statistical analyses.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality

Low X 0.4 1.2 Exposure was not directly assessed using a well-
established method. Exposure risk was estimated
based on proximity of maternal residence to DCM
emissions and the amounts of that chemical released
(Emission Weighted Proximity Model; EWPM).
EWPM values v/ere positively associated with air
measurements. There is no evidence that exposure
mi.sc 1 assi.fi.cation was different among cases and con-
trols.
Medium x 0.2 0.4 The range and distribution of exposure is sufficient
to develop an exposure-response measurement.
Medium X 0.4 0.8 Maternal residential address at the time of delivery
was used to evaluate the proximity to exposure. This
corresponds to the location of exposure during the
first trimester (relevant to morphogenesis) most of
the time, but not always. In evaluating the outcomes
of interest there is some uncertainty that exposure
as indicated occurred during the first trimester.
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: Brender, JD; Shinde, MIJ; Zhan, FB; Gong, X; Langlois, I'll (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type: Developmental toxicity- septal heart defects methylene chloride-Cardiovascular
HERO ID: 2799700
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
High
X
0.667
0.67
The outcomes of interest (birth defects) were eval-

uated in cases based by examination of medical
records by trained staff for the Texas Birth Defects

High

Registry (TBDR).
Metric 8:
Reporting Bias
X
0.333
0.33
The outcomes of interest are specified in the study
report. Effects estimates (ORs) are reported with
95% confidence intervals; the numbers of cases and
controls evaluated in each analysis are clearly de-
noted.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
Low
X
0.5
1.5
There is evidence that potential confounders were

not accounted for (e.g., the recurrence of birth de-

fects in subsequent pregnancies for case-women; a
known risk factor). All risk estimates v/ere ad-
justed for year of delivery, maternal age, education,
race/ethnicity, and public health region of residence.
Metric 10:
Covariate Characterization
Medium
X
0.25
0.5
Data on potential confounders were obtained from
birth and/or fetal death records. Certain character-
istics (e.g., smoking) appeared to be underreported
based on these records.
Metric 11:
Co-exposure Confounding
Medium
X
0.25
0.5
Co-exposures to pollutants (other chlorinated sol-
vents) were estimated using EWPM and were ad-
justed for.
Domain 5: Analysis
Study Design and Methods

Metric 12:
Medium
X
0.4
0.8
The study design chosen is appropriate to evalute
effects between exposure and outcome (i.e., case-
control study); appropriate statistical analyses were
performed.
Metric 13:
Statistical power
Medium
X
0.2
0.4
The number of cases and controls was sufficient to

detect effects. The offspring of 60,613 case-mothers
and 244,927 control-mothers were evaluated (large
sample size).
Metric 14:
Reproducibility of analyses
m
X
0.2
0.4
The description of estimation procedures and cate-
gorization of exposure risk for DCM were described
sufficiently to understand and conceptually repro-
duce the results.
Metric 15:
Statistical models
Medium
X
0.2
0.4
Methods for calculating risk estimates (ORs) are
transparent.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Brender, JD; Shinde, MIJ; Zhan, FB; Gong, X; Langlois, I'll (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type:	Developmental toxicity- septal heart defects	methylene chloride-Cardiovascular
HERO ID:	2799700
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 51: Brender et al. 2014: Evaluation of Growth (early life) and Development Outcomes
Study Citation: Brender, JD; Shinde, MTJ; Zhan, FB; Gong, X; Langlois, PH (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type: Developmental toxicity- oral cleft methylene chloride-Growth (early life) and Development
HERO ID: 2799700
Domain
Metric
Rating^
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
High
Metric 3: Comparison Group
x 0.4 0.4 The key elements of the study design are reported
(including methods of case ascertainment); the in-
formation seems to indicate that selection for the
study was not biased.
X 0.4 0.4 Exclusion from the analysis sample was largely lim-
ited to elective terminations; however it was docu-
mented why they were excluded (lack of linkage to
a vital record).
X 0.2 0.2 Cases and controls v/ere recruited from the same
population (in Texas), during the same time period
(1996—2008) and within the same public health ser-
vice region (1 Iregions). The eligibility criteria for
cases (diagnosis of one of the selected birth defects)
was defined. Differences in baseline characteristics
(e.g., race/ethnicity, education) were controlled for
in statistical analyses.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality

-------
.. . continued from previous page
Study Citation: Brender, JD; Shinde, MIJ; Zhan, FB; Gong, X; Langlois, I'll (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type: Developmental toxicity- oral cleft methylene chloride-Growth (early life) and Development
HERO ID: 2799700
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
High
X
0.667
0.67
The outcomes of interest (birth defects) were eval-

uated in cases based by examination of medical
records by trained staff for the Texas Birth Defects

High

Metric 9:
Covariate Adjustment
Low
X
0.5
1.5
There is evidence that potential confounders were

not accounted for (e.g., the recurrence of birth de-

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Brender, JD; Shinde, MIJ; Zhan, FB; Gong, X; Langlois, I'll (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type:	Developmental toxicity- oral cleft	methylene chloride-Growth (early life) and Development
HERO ID:	2799700
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 52: Brender et al. 2014: Evaluation of Neurological/Behavior Outcomes
Study Citation: Brender, JD; Shinde, MTJ; Zhan, FB; Gong, X; Langlois, PH (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type: Developmental toxicity- neural tube methylene chloride-Neurological/Behavior
HERO ID: 2799700
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
.
High
Metric 3: Comparison Group
x 0.4 0.4 The key elements of the study design are reported
(including methods of case ascertainment); the in-
formation seems to indicate that selection for the
study was not biased.
X 0.4 0.4 Exclusion from the analysis sample was largely lim-
ited to elective terminations; however it was docu-
mented why they were excluded (lack of linkage to
a vital record).
X 0.2 0.2 Cases and controls v/ere recruited from the same
population (in Texas), during the same time period
(1996—2008) and within the same public health ser-
vice region (1 Iregions). The eligibility criteria for
cases (diagnosis of one of the selected birth defects)
was defined. Differences in baseline characteristics
(e.g., race/ethnicity, education) were controlled for
in statistical analyses.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality

-------
.. . continued from previous page
Study Citation: Brender, JD; Shinde, MIJ; Zhan, FB; Gong, X; Langlois, I'll (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type: Developmental toxicity- neural tube methylene chloride-Neurological/Behavior
HERO ID: 2799700
Domain Metric Rating^ MWF* Score Commentstt
Metric 7:
Outcome measurement or characterization
High
X
0.667
0.67
The outcomes of interest (birth defects) were eval-

uated in cases based by examination of medical
records by trained staff for the Texas Birth Defects

High

Metric 9:
Covariate Adjustment
Low
X
0.5
1.5
There is evidence that potential confounders were

not accounted for (e.g., the recurrence of birth de-

Metric 16:
Use of Biomarker of Exposure

NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: Brender, JD; Shinde, MIJ; Zhan, FB; Gong, X; Langlois, I'll (2014). Maternal residential proximity to chlorinated solvent emissions
and birth defects in offspring: a case-control study Environmental Health: A Global Access Science Source, 13(#issue#), 96
Data Type:	Developmental toxicity- neural tube	methylene chloride-Neurological/Behavior
HERO ID:	2799700
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination	Medium	1.8
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 53: Silver et al. 2014: Evaluation of Cancer Outcomes
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type: NIO SH Occupational Cohort DCM BrainN ervousSystemCancer HazardRatio-Cancer
HERO ID: 2799800
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection

Metric 2: Attrition
Metric 3: Comparison Group
High

dium X 0.4 0.8 Retrospective NIOSH cohort of 34,494 workers em-
ployed in microelectronics and business machine fa-
cility for at least 91 days 1969-2001. Foreign nation-
als and those without a valid social security number
(1486) were excluded, as mortality was tracked using
this identifier. All key elements of the study design
are reported.
X 0.4 0.4 Small exclusion based on social security number
(—4%)., which was used to identify outcomes.
X 0.2 0.2 Controls were drawn from the full risk set, with
the conditions that controls started work at age less
than the case's death and survived longer than the
case. Mean data for the full cohort is available, but
not broken dov/n by case/control for each outcome.
While there may have been differences between cases
and controls, statistical models controlled for sex
and pay code. Cases could serve as controls for other
outcomes.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Low
x 0.2
0.8 Department/year-exposure matrix presented in pre-
vious publication (Fleming 3013 - HERO 2128566).
Chemical use and exposure from interviews and
company records: industrial hygiene monitoring
(1980-2002), industrial hygiene department docu-
ments (1974-2002), and environmental impact as-
sessments (1974-1980; 1985-2002). Estimates of
quantities of volatile organi.es from ATSDR study
of community air quality (1969-1980). Work histo-
ries from 2 company electronic personnel databases.
Cumulative exposure scores v/ere derived based on
department/ ye a r exposure matrix modified to incor-
porate intensity information and linked to individual
work history.
0.6 The range and distribution of the cumulative ex-
posure scores were presented (see Fleming 2013 -
HERO 2128566), and the prevalence of Perc was low
(e.g., 15.1% with likely Perc exposure among hourly
workers). This could bias effect estimates toward
the null.
Continued on next page

-------
. continued from previous page
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type: NIO SH Occupational Cohort DCM BrainN ervousSystemCancer HazardRatio-Cancer
HERO ID: 2799800
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 6: Temporality
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias

m X 0.4 0.8 Average of 24-29 years of follow-up with a 10 year
lag used, which is reasonable for cancer outcomes.
However, the population is noted to be relatively
young, so mortality rates may be bias towards the
null.
High X 0.667 0.67 Vital status determined in 2009 by searches of social
security administration death master file, national
death index, and internal revenue service. Death
certificates from state vital statistics offices when
COD not provided by NDI. ICD codes for cause of
death by a certified nosologist.
High X 0.333 0.33 Quantitative description of relevant outcomes from
the abstract/methods are fully provided and ex-
tractable. Data presented included number of ob-
servations, standardized mortality ratios with 95%
confidence intervals, and hazard ratio with 95% con-
fidence intervals.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Medium x 0.5
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding

Medium
Low
x 0.25
x 0.25
0.5
0.75
Covariates accounted for in the regression models,
including paycode (salaried or hourly) as a surrogate
for SES, birth year (20 year cohorts), duration of
employment prior to 1969, and manufacturing eras
(based on process and chemical use). Authors did
not adjust for race, due to missing data (16%) and
low variation (87% white). Variables with >20%
change was considered a confounder and included
in the regression models. Birth cohort adjustment
was an approach to consider smoking. Models for
hazard ratios were ultimately adjusted for paycode
and sex.
Covariates were determined from employment
records at the factory (2 databases with some con-
flicts).
Potential co-exposures were not fully quantified or
considered in the models, despite 3 chemicals and
3 chemical classes being considered explicitly within
the cohort.
Domain 5: Analysis
Metric 12: Study Design and Methods
Medium x 0.4
0.8 Study design was appropriate for the research ques-
tions. Use of regression models for hazard ratio are
appropriate.
Continued on next page

-------
. continued from previous page
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type:	NIO SH Occupational Cohort	DCM	BrainN ervousSystemCancer	HazardRatio-Cancer
HERO ID:	2799800
Domain
Metric

Rating^
MWF*
Score
Comments^
Metric 13
Statistical power

m
x 0.2
0.4
The cohort contains sufficient participants to detect





an effect.
Metric 14
Reproducibility of analyses

Medium
x 0.2
0.4
The process of creating the regression models was
described in detail.
Metric 15
Statistical models

Medium
x 0.2
0.4
Calculations for standardized mortality ratios and
regression models for hazard ratios were transparent
and assumptions were met.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure


NA
NA

Metric 17
Effect biomarker


NA
NA

Metric 18
Method Sensitivity


NA
NA

Metric 19
Biomarker stability


NA
NA

Metric 20
Sample contamination


NA
NA

Metric 21
Method requirements


NA
NA

Metric 22
Matrix adjustment


NA
NA

Overall Quality Determination*

Medium

1.8

Extracted


Yes




* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =

]T\ (Metric Score; X MWF;) / ]T\ MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 54: Silver et al. 2014: Evaluation of Neurological/Behavior Outcomes
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type: NIO SH Occupational Cohort DCM N ervous Syst emD ise ase HazardRatio-Neurological/Behavior
HERO ID: 2799800
Domain
Metric
Rating^
MWF*
Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection

Metric 2: Attrition
Metric 3: Comparison Group
High

dium X 0.4 0.8 Retrospective NIOSH cohort of 34,494 workers em-
ployed in microelectronics and business machine fa-
cility for at least 91 days 1969-2001. Foreign nation-
als and those without a valid social security number
(1486) were excluded, as mortality was tracked using
this identifier. All key elements of the study design
are reported.
X 0.4 0.4 Small exclusion based on social security number
(—4%)., which was used to identify outcomes.
X 0.2 0.2 Controls were drawn from the full risk set, with
the conditions that controls started work at age less
than the case's death and survived longer than the
case. Mean data for the full cohort is available, but
not broken dov/n by case/control for each outcome.
While there may have been differences between cases
and controls, statistical models controlled for sex
and pay code. Cases could serve as controls for other
outcomes.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Low
x 0.2
0.8 Department/year-exposure matrix presented in pre-
vious publication (Fleming 3013 - HERO 2128566).
Chemical use and exposure from interviews and
company records: industrial hygiene monitoring
(1980-2002), industrial hygiene department docu-
ments (1974-2002), and environmental impact as-
sessments (1974-1980; 1985-2002). Estimates of
quantities of volatile organi.es from ATSDR study
of community air quality (1969-1980). Work histo-
ries from 2 company electronic personnel databases.
Cumulative exposure scores v/ere derived based on
department/ ye a r exposure matrix modified to incor-
porate intensity information and linked to individual
work history.
0.6 The range and distribution of the cumulative ex-
posure scores were presented (see Fleming 2003 -
HERO 212856), and the prevalence of TCE was low
(e.g., 13.9% with likely TCE exposure among hourly
workers). This could bias effect estimates toward the
null.
Continued on next page

-------
. continued from previous page
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type: NIO SH Occupational Cohort DCM N ervous Syst emD ise ase HazardRatio-Neurological/Behavior
HERO ID: 2799800
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 6: Temporality
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias

-------
. continued from previous page
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type:	NIO SH Occupational Cohort	DCM	N ervous Syst emD ise ase	HazardRatio-Neurological/Behavior
HERO ID:	2799800
Domain
Metric

Rating^
MWF*
Score
Comments^t
Metric 13
Statistical power

m
x 0.2
0.4
The cohort contains sufficient participants to detect





an effect.
Metric 14
Reproducibility of analyses

Medium
x 0.2
0.4
The process of creating the regression models was
described in detail.
Metric 15
Statistical models

Medium
x 0.2
0.4
Calculations for standardized mortality ratios and
regression models for hazard ratios were transparent
and assumptions were met.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure


NA
NA

Metric 17
Effect biomarker


NA
NA

Metric 18
Method Sensitivity


NA
NA

Metric 19
Biomarker stability


NA
NA

Metric 20
Sample contamination


NA
NA

Metric 21
Method requirements


NA
NA

Metric 22
Matrix adjustment


NA
NA

Overall Quality Determination*

Medium

1.8

Extracted


Yes




* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =

]T\ (Metric Score; X MWF;) / ]T\ MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 55: Silver et al. 2014: Evaluation of Hepatic Outcomes
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type: NIO SH Occupational Cohort DCM LiverDisease SMR_malehourly-Hepatic
HERO ID: 2799800
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection

Metric 2: Attrition
Metric 3: Comparison Group
High

dillin X 0.4 0.8 Retrospective NIOSH cohort of 34,494 workers em-
ployed in microelectronics and business machine fa-
cility for at least 91 days 1969-2001. Foreign nation-
als and those without a valid social security number
(1486) were excluded, as mortality was tracked using
this identifier. All key elements of the study design
are reported.
X 0.4 0.4 Small exclusion based on social security number
(—4%)., which was used to identify outcomes.
X 0.2 0.2 Controls were drawn from the full risk set, with
the conditions that controls started work at age less
than the case's death and survived longer than the
case. Mean data for the full cohort is available, but
not broken dov/n by case/control for each outcome.
While there may have been differences between cases
and controls, statistical models controlled for sex
and pay code. Cases could serve as controls for other
outcomes.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Low
x 0.2
0.8 Department/year-exposure matrix presented in pre-
vious publication (Fleming 3013 - HERO 2128566).
Chemical use and exposure from interviews and
company records: industrial hygiene monitoring
(1980-2002), industrial hygiene department docu-
ments (1974-2002), and environmental impact as-
sessments (1974-1980; 1985-2002). Estimates of
quantities of volatile organi.es from ATSDR study
of community air quality (1969-1980). Work histo-
ries from 2 company electronic personnel databases.
Cumulative exposure scores v/ere derived based on
department/ ye a r exposure matrix modified to incor-
porate intensity information and linked to individual
work history.
0.6 The range and distribution of the cumulative ex-
posure scores were presented (see Fleming 2003 -
HERO 212856), and the prevalence of TCE was low
(e.g., 13.9% with likely TCE exposure among hourly
workers). This could bias effect estimates toward the
null.
Continued on next page

-------
. continued from previous page
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type: NIO SH Occupational Cohort DCM LiverDisease SMR_malehourly-Hepatic
HERO ID: 2799800
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 6: Temporality
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias

m X 0.4 0.8 Average of 24-29 years of follow-up with a 10 year
lag used, which is reasonable for cancer outcomes.
However, the population is noted to be relatively
young, so mortality rates may be bias towards the
null.
High X 0.667 0.67 Vital status determined in 2009 by searches of social
security administration death master file, national
death index, and internal revenue service. Death
certificates from state vital statistics offices when
COD not provided by NDI. ICD codes for cause of
death by a certified nosologist.
High. X 0.333 0.33 Quantitative description of relevant outcomes from
the abstract/methods are fully provided and ex-
tractable. Data presented included number of ob-
servations, standardized mortality ratios with 95%
confidence intervals, and hazard ratio with 95% con-
fidence intervals.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Medium x 0.5
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding

-------
. continued from previous page
Study Citation: Silver, SR; Pinkerton, LE; Fleming, DA; Jones, JH; Allee, S; Luo, L; Bertke, SJ (2014). Retrospective cohort study of a microelectronics
and business machine facility American Journal of Industrial Medicine, 57(4), 412-424
Data Type:	NIO SH Occupational Cohort	DCM	LiverDisease	SMR_malehourly-Hepatic
HERO ID:	2799800
Domain
Metric

Rating^
MWF*
Score
Comments^t
Metric 13
Statistical power

m
x 0.2
0.4
The cohort contains sufficient participants to detect





an effect.
Metric 14
Reproducibility of analyses

Medium
x 0.2
0.4
The process of creating the regression models was
described in detail.
Metric 15
Statistical models

Medium
x 0.2
0.4
Calculations for standardized mortality ratios and
regression models for hazard ratios were transparent
and assumptions were met.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure


NA
NA

Metric 17
Effect biomarker


NA
NA

Metric 18
Method Sensitivity


NA
NA

Metric 19
Biomarker stability


NA
NA

Metric 20
Sample contamination


NA
NA

Metric 21
Method requirements


NA
NA

Metric 22
Matrix adjustment


NA
NA

Overall Quality Determination*

Medium

1.8

Extracted


Yes




* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =

]T\ (Metric Score; X MWF;) / ]T\ MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 56: Cliaigne et al 2015: Evaluation of Hematological and Immune Outcomes
Study Citation: Chaigne, B; Lasfargues, G; Marie, I; Hiittenberger, B; Lavigne, C; Marchand-Adam, S; Maillot, F; Diot, E (2015). Primary Sjogren's
syndrome and occupational risk factors: A case-control study Journal of Autoimmunity, 60(tissue#), 80-85
Data Type: occupational (France) ever DCM exposure primary Sjogren's syndrome-Hematological and Immune
HERO ID: 2902069
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
um x 0.4 0.8 Some key elements of the study design were not
present but available information indicates a low risk
of selection bias. Eligibility and participation rates
were not reported, however exclusion criteria was
noted. It appears that all patients with primary Sjo-
gren's syndrome from different hospitals in France
from 2010-'2013 were included. Recruitment for con-
trols was not provided, but there is no indication of
selection bias.
High x 0.4 0.4 There is no apparent attrition.
High X 0.2 0.2 Controls were age and gender matched, and selected
from the same departments during the same time
period. Provided information does not indicate any
differences in terms of smoking habits, SES, or socio-
professional categories.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
bow
Low
x 0.2
x 0.4
0.8 Occupational exposure was assessed by industrial
hygi.eni.sts and occupational practitioners. Exposure
was semi.quanti.fied based on the experts' knowledge
of the industrial process and its evolution over time.
Exposure was also evaluated using the French job-
exposure matrix (link provided, but not working).
All employment periods in which subjects worked
more than 6 months was included. An exposure
score was calculated (methods reported).
0.6 Only evaluated as ever/never or low and high final
cumulative exposure score.
1.2 Although occupational exposure was retrospective 1 y
assessed, the study authors acknowledge that they
cannot distinguish between exposures that pre-dated
or post-dated the onset of the disease.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
X 0.667 0.67 Primary Sjogren;s syndrome was diagnosed in the
hospital and was defined according to the American-
European Consensus Group criteria.
Continued on next page

-------
. continued from previous page
Study Citation: Chaigne, B; Lasfargues, G; Marie, I; Hiittenberger, B; Lavigne, C; Marchand-Adam, S; Maillot, F; Diot, E (2015). Primary Sjogren's
syndrome and occupational risk factors: A case-control study Journal of Autoimmunity, 60(tissue#), 80-85
Data Type:	occupational (France) ever DCM exposure	primary Sjogren's syndrome-Hematological and Immune
HERO ID:	2902069
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 8: Reporting Bias
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High x 0.333 0.33
Medi
Low
For chemicals of interest all outcomes outlined in the
abstract, introduction, and methods were reported.
Effect estimates (odds ratios) are reported with a
95% confidence interval along with the number of
cases and controls.
Medium x 0.5
um x 0.25
x 0.25
1	The study does not appear to adjust for any covari-
ates. However, controls were sex and age matched
and there does not appear to be any differences be-
tween the groups in terms of smoking or SES.
0.5 Information was obtained during a 30-mi.nute in-
terview; a less established method to assess con-
founders with no method validation.
0.75 Subjects had several periods of exposure to different
categories of exposure that were not mutually exclu-
sive and these v/ere not adjusted for in the analysis.
Nor was there enough information provided on the
different types of work to know if there would be a
differential co-exposure that could affect the results.
Domain 5: Analysis
Metric 12: Study Design and Methods
Medium x 0.4
0.8 Study design is appropriate. The study is a case-
control study, which is appropriate for studying a
rare disease like primary Sjogren's syndrome espe-
cially when evaluating many different possible expo-
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models
Medium x 0.2
um x 0.2
Medium x 0.2
0.4 Sample size is sufficient overall (175 cases and 350
controls) but the number of exposed cases and con-
trols is small (e.g. 13 cases and 3 controls for
ever/never exposure)
0.4 It was only noted that a conditional maximum like-
lihood estimate was calculated, but this appears to
be sufficient information.
0.4 Method is transparent (a conditioned maximum like-
lihood estimate of the odds ratio and 95% confidence
intervals using GraphPad Prism version 6.00 soft-
ware) and assumptions were met.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Metric 17
Metric 18
Use of Biomarker of Exposure
Effect biomarker
Method Sensitivity
NA
NA
NA
NA
NA
NA
Continued on next page

-------
. continued from previous page
Study Citation: Chaigne, B; Lasfargues, G; Marie, I; Hiittenberger, B; Lavigne, C; Marchand-Adam, S; Maillot, F; Diot, E (2015). Primary Sjogren's
syndrome and occupational risk factors: A case-control study Journal of Autoimmunity, 60(tissue#), 80-85
Data Type:	occupational (France) ever DCM exposure	primary Sjogren's syndrome-Hematological and Immune
HERO ID:	2902069
Domain

Metric

Rating^
MWF*
Score
Comments^t

Metric 19:
Metric 20:
Metric 21:
Metric 22:
Biomarker stability
Sample contamination
Method requirements
Matrix adjustment


NA
NA
NA
NA
NA
NA
NA
NA

Overall Quality Determination
Medium
1.8
Extracted
Yes
MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
(Metric Score; X MWF;) / ]T\ MWFj
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 57: Talbott et al 2015: Evaluation of Neurological/Behavior Outcomes
Study Citation: Talbott, EO; Marshall, LP; Rager, JR; Arena, VC; Sharma, RK; Stacy, SL (2015). Air toxics and the risk of autism spectrum disorder:
The results of a population based case-control study in southwestern Pennsylvania Environmental Health: A Global Access Science
Source, 14(#issue#), 80
Data Type: CaseControl Childhood DCM AutismSpectrumDisorder OR_Q4-Neurological/Behavior
HERO ID: 3007486
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
High
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
Medium x 0.4
Medium x 0.2
0.4 217 autism spectrum disorder (ASD) cases born
2005-2009 were obtained from 6 counties in SW
Pennsylvania using an outreach campaign targeted
at ASD specialty diagnostic/treatment centers, pri-
vate pediatric/psychiatry practices, school-based
special needs programs, and autism support groups.
Approximately 43% of cases living in the area were
estimated to be obtained.
0.8 Of the 299 cases that wanted to participate, 56 were
excluded (see below), 26 were not interested or able
to complete the full interview. Of the 3254 mailed
requests for interview controls, 250 returned con-
tact sheets. Of these 24 were ineligible or unable
to be contacted. All eligible birth certificate con-
trols were included. Participants were excluded if
adopted, parents were non-English speaking, parent
wasn't available for interview, child lived outside the
US, or 2000 census tract could not be matched birth
certificate address.
0.4 Interview controls (224) were recruited from a
random selection of birth registries at same
time/counties as the cases; frequency matched to
year of birth, sex and race. Birth certificate con-
trols (4971) v/ere drav/n from birth registries in the
same time/counties weighted with sex ratio and year
of birth. An ASD diagnosis was not evaluated in the
birth certificate controls, although 16 cases captured
in this set were excluded. Cases had more preterm
birth and multiple births than controls. Interview
controls included more white and higher educated
mothers than cases. Birth certificate controls had
fewer white and higher educated mothers. All of
these differences were considered as potential con-
founders and/or analyzed via sensitivity analysis.
Domain 2: Exposure Characterization
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Talbott, EO; Marshall, LP; Rager, JR; Arena, VC; Sharma, RK; Stacy, SL (2015). Air toxics and the risk of autism spectrum disorder:
The results of a population based case-control study in southwestern Pennsylvania Environmental Health: A Global Access Science
Source, 14(#issue#), 80
CaseControl Childhood DCM AutismSpectrumDisorder OR_Q4-Neurological/Behavior
3007486
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 4: Measurement of Exposure
Low x 0.4
Metric 5: Exposure levels
im x 0.2
Metric 6: Temporality

Medium x 0.4
1.2 Ambient hazardous air pollution concentrations for
30 air toxics were estimated using modeled data from
the US EPA 2005 NATA assessment (average by cen-
sus tract), including DCM, PERC, and TCE, For
cases and interview controls, residential history from
3 months prior to pregnancy through 2 years old
were geocoded, verified, and assigned a census tract
(based on 2000 codes). Exposures were determined
for pregnancy, 1st and 2nd years of life. For analysis
using birth certificate controls, only the residence at
time of birth was used to estimate exposure.
0.4 Quartiles of exposure v/ere determined for cases, in-
terview controls and birth certificate controls for
methylene chloride (239-273 ng/m3), perchloroethy-
lene (94-267 ng/m3), and trichloroethylene (71-85
ng/m3). For cases evaluated against birth certifi-
cate controls, quartiles were split as follows: DCM
244.06 ng/m3, 266.47 ng/m3, 272.48 ng/m3; Perc
100.08 ng/m3, 214.81 ng/m3, 267.36 ng/m3; TCE
70.55 ng/m3, 74.33 ng/m3, and 82.46 ng/m3.
0.8 For cases and interview controls, exposure was mod-
eled using data from 3 months prior to pregnancy
through 2 years of age, which is anticipated to cover
the critical window of exposure. Age of children at
outcome assessment not stated. Participating chil-
dren were born 2005-2009, and the study was pub-
lished in 2015 with exposure data accessed in 2014.
Domain 3: Outcome Assessment
Metric 7:
Outcome measurement or characterization
Medium x 0.667 1.33
Metric 8: Reporting Bias
Medium x 0.333 0.67
The ASD outcome required a score of 15+ on the
Social Communi.cation Questionnaire (autistic fea-
tures screen), as well as written documentati.on of
a diagnosis by a child psychologist or psychiatrist.
Outcome was assessed in cases and interview con-
trols. The ASD outcome was not assessed in the
birth certificate controls.
Odds ratios reported with 95% confidence intervals
for adjusted models. Singleton sensitivity analy-
sis data included in supplemental material and Ta-
ble 5 for methylene chloride (statistically signifi-
cant). Number of cases/controls for each analysis
provided. Co-exposure correlations and factor anal-
ysis not fully presented.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Talbott, EO; Marshall, LP; Rager, JR; Arena, VC; Sharma, RK; Stacy, SL (2015). Air toxics and the risk of autism spectrum disorder:
The results of a population based case-control study in southwestern Pennsylvania Environmental Health: A Global Access Science
Source, 14(#issue#), 80
CaseControl Childhood DCM AutismSpectrumDisorder OR_Q4-Neurological/Behavior
3007486
Domain
Metric
Rating^ MWF* Score
Commentstt
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
High x 0.5
Metric 10: Covariate Characterization
um x 0.25
Metric 11: Co-exposure Confounding
Medium x 0.25

0.5 Adjusted for mother's age, education, race, smoking
status, as well as child's year of birth and sex. Sen-
sitivity analysis was conducted to evaluate the high
rate of multiple births in cases, relative to controls
(8.4% cases; ~4% controls).
0.5 Trained interviewers interviewed mothers with
structured questionnaire for demographics, SES, res-
idential history, occupational history (maternal and
paternal), family history of ASD, smoking history,
maternal reproductive history, and child's medical
history. Birth weight and preterm births were de-
termined from birth certificates.
0.5 Several of the air toxics studied v/ere reported to be
highly corelated, and PCA found 75% of the pollu-
tant variance could be attributed to 7 factors. De-
tails not provided. Abstract states "unclear if these
chemicals are risk factors themselves or if they re-
flect the effect of a mixture of pollutants." However,
no indication that these co-exposures differed across
cases and controls.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Medium x 0.4 0.8
Medium x 0.2 0.4
Medium x 0.2 0.4
A case-control study was utilized to construct OR
for ASD. Exposure quartiles determined with NATA
model using location data from pregancy-2 years.
Logistic regression utilized to determine OR across
quantises.
The 217 cases, 224 interview controls, and 4971 birth
certificate cases were sufficient to detect an effect for
methylene chloride and air pollutants not relevant to
this evaluation. Statistical pov/er not reported, but
p values show some statistically significant correla-
tions
Detailed description of analysis is provided. The
confounders used to adjust the OR models are clear
and provided. Only the factor analysis of co-
exposures correlation is insufficiently detailed to al-
low for replication, but this does not impact the
outcome-exposure correlations.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Talbott, EO; Marshall, LP; Rager, JR; Arena, VC; Sharma, RK; Stacy, SL (2015). Air toxics and the risk of autism spectrum disorder:
The results of a population based case-control study in southwestern Pennsylvania Environmental Health: A Global Access Science
Source, 14(#issue#), 80
CaseControl	Childhood	DCM	AutismSpectrumDisorder	OR_Q4-Neurological/Behavior
3007486
Domain
Metric
Metric 15: Statistical models
Rating'
Medium
MWF* Score
Commentstt
X 0.2	0.4 Logistic regression analysis used to compare in-
terquartile ORs. Spearman correlation and princi-
pal component analysis were used to assess air toxics
correlations. Model assumptions were met and the
variables used were clearly stated and appropriate.
Domain 6: Other Considerations for Biomarker Selection and Mea.su
Metric 16
Use of Biomarker of Exposure

NA
NA
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination"*"

Medium
1.9
Extracted
*	MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
*	The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =
(Metric Score; X MWF;) / J] . MWF,-
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the review r determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 58: Garcia et al. 2015: Evaluation of Cancer Outcomes
Study Citation: Garcia, E; Hurley, S; Nelson, DO; Hertz, A; Reynolds, P (2015). Hazardous air pollutants and breast cancer risk in California teachers:
A cohort study Environmental Health: A Global Access Science Source, 14(1), 14
Data Type: Cohort DCM CTS BreastCancer Q3-Cancer
HERO ID: 3014082
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
.

x 0.4
Metric 2: Attrition
to
o
o
Metric 3: Comparison Group
High x 0.4
High x 0.2
0.4 California Teachers Study including active and re-
tired female teachers and administrators were en-
rolled in the California State Teachers Retirement
System and completed a questionnaire. Study pop-
ulation was comprised on 5676 women. All partic-
ipants were included using the same inclusion and
exclusion criteria.
0.4 Large sample of study population excluded due to
women who v/ere not residing in California at base-
line, had unknown history of prior cancer, had prior
history of invasive or in situ breast cancer, asked to
be removed from study after joining, or had an ad-
dress that couldn't be geocoded. This represents ad-
equate explanation of attrition and is not expected
to bias the results.
0.2 Cases and controls were stated to be similar. Covari-
ates that were different between groups were consid-
ered and included as covariates in the final model.,
including a term for grouped personal risk factors.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Medium x 0.4
Metric 5: Exposure levels
ivieuium X 0.2
0.8 NATA identified and prioritized the air toxicants
with respect to their potential population health
risks. The first NATA was conducted based on 1996
emissions. EPA models annual ambient HAP con-
centrations using the Assessment System for Pop-
ulation Exposure Nationwide (ASPEN). This is a
we 11-estab 1 i.shed method of determining exposure.,
but may lead to some non-di.fferenti.al exposure mis-
classification.
0.4 By examining each compound individually, they cat-
egorized them into four quantises of concentration
without including exposure from any other com-
pound in the model. Level of exposure adequate.
Included four quantises of exposure, Q1 being no ex-
posure.
Continued on next page

-------
. continued from previous page
Study Citation: Garcia, E; Hurley, S; Nelson, DO; Hertz, A; Reynolds, P (2015). Hazardous air pollutants and breast cancer risk in California teachers:
A cohort study Environmental Health: A Global Access Science Source, 14(1), 14
Data Type: Cohort_DCM_CTS_BreastCancer_Q3-Cancer
HERO ID: 3014082
Domain
Metric
Rating^ MWF* Score
Comments^
Metric 6: Temporality

m X 0.4 0.8 Chose to use the 2002 ambient air concentration es-
timates for this study because that year was approx-
imately the mi.d-poi.nt for the follow up period. De-
cided against combining multiple years of estimate
due to inconsistent methodical approaches and tem-
poral variations in the level of agreement between
years of the assessments which could introduce ex-
posure misclassification.
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
Metric 8: Reporting Bias
High
x 0.667 0.67 CTS cohort is followed annually for cancer diagno-
sis, death, and change of address. Annual linkage
between CCR and cohort membership was used to
identify incident cancer rates. Defined a case as any
woman diagnosed with invasive breast cancer (ICD-
03 site codes C500-C509, excluding those with his-
tology codes for 9050-9055, 9140, and 9590-9992) af-
ter the date they completed their baseline question-
naire through Dec 31, 2011.
X 0.333 0.33 CCR maintains high standards for data quality and
completeness and is estimated to be 99% complete.
Ascertained date and cause of death from mortality
files as well as reports from relatives.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization

x 0.5
Metric 11: Co-exposure Confounding
Medium x 0.25
Medium x 0.25
0.5 All models were stratified by age and adjusted either
for race alone or for race and personal risk factors of
interest. For each compound, p-values no each non-
degenerative quantile HR were adjusted for multiple
testing across the ten subsets using False Discovery
Rates.
0.5 Covariates were obtained from the CTS baseline
questionnaire. This was self-reported information,
but there is no evidence to suggest that it is not a
valid method of obtaining covariate information.
0.5 No indication of unbalanced co exposures.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Medium x 0.4 0.8
Medium x 0.2 0.4
Cohort was appropriate study design. Examined the
relationship between risk of breast cancer and nu-
merous compounds of interest. Used two different
methods of parameterizing exposure in the models.
Number of subjects for estimated exposure was 5676
women. There were enough subjects to detect effects
for some chemicals and for some trends.
Continued on next page

-------
. continued from previous page
Study Citation: Garcia, E; Hurley, S; Nelson, DO; Hertz, A; Reynolds, P (2015). Hazardous air pollutants and breast cancer risk in California teachers:
A cohort study Environmental Health: A Global Access Science Source, 14(1), 14
Data Type:	Cohort_DCM_CTS_BreastCancer_Q3-Cancer
HERO ID:	3014082
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 14
Reproducibility of analyses
m
x 0.2
0.4
Study design and methods can be reproducible with





information provided. Provided reasoning on how





categories were created for exposure quantiles, why





covariates were used. Covariates included in the





models are reported explicitly.
Metric 15
Statistical models
Medium
x 0.2
0.4
Used COX proportional hazard models to estimate





hazard rate ratios. Parameterized exposures into





quantiles, modeled exposure as a continuous vari-





able, and tested for non-zero slope using a likelihood





ratio test.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination
High

1.5

Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWF,) / J] . MWF?
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 59: Kumagi et al. 2016: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Kumagai, S; Sobue, T; Makiuchi, T; Kubo, S; IJehara, S; Hayashi, T; Sato, KK; Endo, G (2016). Relationship between cumulative ex-
posure to 1,2-dichloropropane and incidence risk of cholangiocarcinoma among offset printing workers Occupational and Environmental
Medicine, 73(8), 545-552
DCM Osaka printing cohort cholangiocarcinoma IRR 5-year lag-Cancer
3419929
Domain
Metric
Rating
MWF*
Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Medium x 0.4

to
o
CO
Metric 2: Attrition
Metric 3: Comparison Group
High x
0.4
High x 0.2
0.8 Study setting and participant selection are reported
in detail. Employees were chosen from a company
list of workers in the proof-printing section of the
factory where they would be exposed to both 1,2-
DCP and DCM. There were some small differences
between the sub-population exposed to DCM com-
pared to the whole factory sample. Workers ex-
posed to DCM were slightly older, a larger propor-
tion male, and more likely to have a longer exposure
(larger cumulative exposure)
0.4 Of 116 v/orkers identified from the company list,
eight were excluded due to incomplete demo-
graph ic/employment information. Eleven other
v/orkers were excluded due to starting work after
termination of 1,'2-DCP (the main exposure in this
study) use in the plant.
0.2 For SIRs, the expected number of cases was "calcu-
lated using sex, calendar year and age-specific in-
cidence rates of cho 1 angi.ocarci.noma in the general
population in Japan." This demonstrates adjustment
for relevant characteristics as well as a clear selection
of an appropriate reference population.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
um X 0.4	0.8 Exposure was extrapolated back from a recreation of
the factory environment in a 2012 JNIOSH experi-
ment at one of the Osaka factories (Plant O-2). This
recreation used the appropriate mixture of 1,2-DCP
and DCM to obtain TWAs for a known quantity used
per hour. This was extrapolated back with worker's
histories and accounting records of purchased 1,2-
DCP and DCM for each specific plant to calculate
a cumulative exposure for each employee. In the
JNIOSH recreation, measurements of exposure were
also taken in the front office and delivery areas so
as to be able to assign exposure to workers that fell
into these categories.
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Kumagai, S; Sobue, T; Makiuchi, T; Kubo, S; TJehara, S; Hayashi, T; Sato, KK; Endo, G (2016). Relationship between cumulative ex-
posure to 1,2-dichloropropane and incidence risk of cholangiocarcinoma among offset printing workers Occupational and Environmental
Medicine, 73(8), 545-552
DCM Osaka printing cohort cholangiocarcinoma IRR 5-year lag-Cancer
3419929
Domain Metric
Rating^
MWF*
Score
Comments^
Metric 5: Exposure levels
Low
x 0.2
0.6
For the SIR, there are only two levels of exposure

which is defined by employment at the plants and no
exposure in the general population. For the incident
rate ratio, there was also only two levels of exposure
as it was included as a dichotomous variable.
Metric 6: Temporality
High
x 0.4
0.4
This study evaluates a rare cancer in employees dur-
ing a follow-up period (minimum 5 years) which es-
tablishes temporality between exposure and disease.
Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization
Medium
x 0.667
1.33
Health records were obtained for all employees from
the Japanese Ministry of Health, Labour, and Wel-
fare. These records were evaluated by one of the
study authors (Kubo). For comparison with the gen-
eral Japenese population, the specific I CD-9 codes
used were 155.1 and 156.1 (C22.1 and C24.0 in ICD-
10). This is not a gold standard, but there is no
evidence to suggest this method would have poor
validity.
Metric 8: Reporting Bias
High
x 0.333
0.33
All outcomes listed in the abstract, introduction,

and methods were provided both in-text and in eas-
ily read and extractable tables. Either standardized
incidence ratios or incidence rate ratios with 95%
confidence intervals and person-years and number of
subjects were reported.
Domain 4: Potential Counfounding/Variable Control

Metric 9: Covariate Adjustment
High
x 0.5
0.5
Relevant demographic and employment characteris-
tics v/ere drawn from employment records. For SIRs,
the expected number of cases was "calculated using
sex, calendar year and age-specific incidence rates
of cholangiocarcinoma in the general population in
Japan."
Metric 10: Covariate Characterization
m
x 0.25
0.5
Covariates were taken from employment records.
This is not a gold standard method, but there is no
evidence to indicate this method has poor validity.
Continued on next page . ..

-------
.. . continued from previous page
Study Citation: Kumagai, S; Sobue, T; Makiuchi, T; Kubo, S; TJehara, S; Hayashi, T; Sato, KK; Endo, G (2016). Relationship between cumulative ex-
posure to 1,2-dichloropropane and incidence risk of cholangiocarcinoma among offset printing workers Occupational and Environmental
Medicine, 73(8), 545-552
Data Type: DCM Osaka printing cohort cholangiocarcinoma IRR 5-year lag-Cancer
HERO ID: 3419929
Domain
Metric

Rating^
MWF*
Score
Commentstt
Metric 11
Co-exposure Confounding

Low
x 0.25
0.75
For the SIR, there was no adj ustment for exposure
to 1,2-DCP. All workers in the DCM exposed group
were also exposed to 1,2-DCP. Other co-exposures in
this setting include kerosene and potentially carcino-
genic inks although the study authors indicate that
these were present in low levels and were unlikely to
influence the DCM-cholangiocarcinoma relationship.
Domain 5: Analysis

Metric 12
Study Design and Methods

Medium
x 0.4
0.8
The study design was appropriate for investigat-
ing the relationship between exposure to DCM and
cholangiocarcinoma.
Metric 13
Statistical power

Medium
x 0.2
0.4
The number of subjects for DCM exposure was
rather small (n=33), but there were a sufficient num-
ber of cases to detect an effect.
Metric 14
Reproducibility of analyses

Medium
x 0.2
0.4
The analysis was described in detail. For exposure
measurement, the calculation of cumulative expo-

sure was moderately complex, but explained thor-
oughly.
Metric 15
Statistical models

Medium
x 0.2
0.4
The methods for calculating risk in both the case of
SIRs and RRs was appropriate and transparent. No
apparent issues.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*

Medium

1.7

Extracted

Yes

Continued on next page

-------
. continued from previous page
Study Citation: Kumagai, S; Sobue, T; Makiuchi, T; Kubo, S; Uehara, S; Hayashi, T; Sato, KK; Endo, G (2016). Relationship between cumulative ex-
posure to 1,2-dichloropropane and incidence risk of cholangiocarcinoma among offset printing workers Occupational and Environmenta 1
Medicine, 73(8), 545-552
Data Type:	DCM Osaka printing cohort cholangiocarcinoma IRR 5-year lag-Cancer
HERO ID:	3419929
Domain	Metric	Rating^ MWF* Score	Comments^
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
]T\ (Metric Score,; x MWF,;) / J] . MWF;
*
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
(round to the nearest tenth) otherwise
0.1
crossed out and an arrow points to the new rating.
.
This metric met the criteria for high confidence as expected for this type of study

-------
Table 60: Carton et al. 2017: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Carton, M; Barul, C; Menvielle, G; Cyr, D; Sanchez, M; Pilorget, C; Tretarre, B; Strieker, I; Luce, D (2017). Occupational exposure to
solvents and risk of head and neck cancer in women: A population-based case-control study in France British Medical Journal Open,
7(1), e012833
I CARE DCM He adNeck C anc er OR EverExposure-Cancer
3480125
Domain
Metric
Rating1"
MWF*
Score
Comments11
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
High x 0.4
Medium x 0.4

Metric 3: Comparison Group
High x 0.2
0.4 296 cases of head and neck squamous cell carcino-
mas and 775 controls were drawn from ICARE, a
French population-based case-control study (Luce
2011, HERO ID 1022113). Only women.
0.8 Parti.ci.pati.on rates in initial ICARE study v/ere
82.5% for cases and 80.6% for controls. Restrict-
ing to only females with squamous cell carcinomas
in areas of interest led to 296 cases and 755 controls.
0.2 Controls selected from general population based on
age, geographic region and SES. However, there are
statistically significant differences in terms of age,
geographic region, SES, smoking and alcohol con-
sumption. These covariates are all considered in
the analysis. Cases -2 years younger than controls,
lower SES, and more likely to smoke or drink alco-
hol.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Metric 6: Temporality
Low X 0.4 1.2 Employment history from in person interviews and
questionnaires. Employment of 1+ month coded
by trained coders blinded to status using Interna-
tional Standard Classification of Occupations and
the Nomenclature des Acti.vi.tes Franchises. Job-
exposure matrix; from French Institute of Health
Surveillance to predict exposure probability, inten-
sity, and frequency.
m X 0.2 0.4 Analysis includes dichotomous ever/never exposed,
as well as continuous exposure intensity, exposure
duration and cumulative exposure indices.
Low X 0.4 1.2 Time between potential occupational exposure and
diagnosis not stated.
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Carton, M; Barul, C; Menvielle, G; Cyr, D; Sanchez, M; Pilorget, C; Tretarre, B; Strieker, I; Luce, D (2017). Occupational exposure to
solvents and risk of head and neck cancer in women: A population-based case-control study in France British Medical Journal Open,
7(1), e012833
I CARE DCM He adNeck C anc er OR EverExpoeure-Cancer
3480125
Domain

Metric
Rating^
MWF*
Score
Comments^

Metric 7:
Outcome measurement or characterization
High
X 0.667
0.67
Cases identified from cancer registries in 10 ge-
ographical regions of France. Histologically con-
firmed diagnosis from 2001-2007 in women aged 18-
85. ICD-O-3 codes v/ere used to identify squa-

mous cell carcinomas in oral cavity, oropharynx, hy-

popharynx, oral cavity, and larynx (detailed list of
codes in text).

Metric 8:
Reporting Bias
High
X 0.333
0.33
Quantitative description of relevant outcomes (head

and neck cancers in women) from the ab-
stract/ methods are provided and extractable.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding

High
Medium
Medium
x 0.5 0.5 Analyses adjusted for geographical area, age, smok-
ing status, tobacco consumption (pack-years) and
alcohol consumption. Interaction terms for smok-
ing and alcohol were also included. SES considered
with last occupation and longest occupation, but did
not impact ORs and were not presented.
x 0.25 0.5 In person interviews with standardized question-
naire.
X 0.25 0.5 Exposures to TCE, Perc, and DCM were strongly
correlated. Rather than adjusting for co-exposures,
exclusive exposure to individual and combinations
of chlorinated solvents were analyzed.
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models
Medium
Medium
Low
x 0.4
x 0.2
x 0.2
Medium x 0.2
0.8 Study design was appropriate for the research ques-
tions. Logistic regression was used appropriately to
estimate ORs and CIs.
0.4 The cohort contains sufficient participants to detect
an effect for TCE, perc, and DCM. Insufficient data
for carbon tetrachloride, so it was excluded from
analysis beyond an ever/never OR.
0.6 Although the process of creating the regression mod-
els was described in detail, adjustments used for co-
variates were not explicitly stated.
0.4 Odds ratios and 95% confidence intervals were de-
termined using unconditional logistic regression ad-
justed for key covariates. Models were transparent
and assumptions were met.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Continued on next page

-------
. continued from previous page
Study Citation:
Data Type:
HERO ID:
Carton, M; Barul, C; Menvielle, G; Cyr, D; Sanchez, M; Pilorget, C; Tretarre, B; St ticker, I; Luce, D (2017). Occupational exposure to
solvents and risk of head and neck cancer in women: A population-based case-control study in France British Medical Journal Open,
7(1), e012833
ICARE_DCM_HeadNeckCancer_OR_EverExposure-Cancer
3480125
Domain

Metric
Rating1" MWF*
Score
Commentstt

Metric 16
Use of Biomarker of Exposure
NA
NA


Metric 17
Effect biomarker
NA
NA


Metric 18
Method Sensitivity
NA
NA


Metric 19
Biomarker stability
NA
NA


Metric 20
Sample contamination
NA
NA


Metric 21
Method requirements
NA
NA


Metric 22
Matrix adjustment
NA
NA

Overall Quality Determination*
Medium
1.8
Extracted
Yes


MWF = Metric Weighting Factor
High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating =
y (Metric Scores x MWF,;) / ^ . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 61: Purdue et al. 2016: Evaluation of Cancer Outcomes
Study Citation:
Data Type:
HERO ID:
Purdue, MP; Stewart, PA; Priesen, MC; Colt, JS; Locke, SJ; Hein, MJ; Waters, MA; Graubard, BI; Davis, F; Ruterbusch, J; Schwartz,
K; Chow, WH; Rothman, N; Hofmann, JN (2016). Occupational exposure to chlorinated solvents and kidney cancer: A case-control
study Occupational and Environmental Medicine, 74(4), 268-274
Case-control study of kidney cancer in workers exposed to chlorinated solvents - DCM	50-89% OR-Cancer
3482059
Domain
Metric

Rating^
MWF*
Score
Comments^
Domain 1: Study Participation







Metric 1:
Participant selection

High
X
0.4
0.4
Selection factors unlikely to be related to DCM ex-
posures
Metric 2:
Attrition

Medium
X
0.4
0.8
77% participation in cases; 54% participation in con-
trols; rationale was provided.
Metric 3:
Comparison Group

High
X
0.2
0.2
Age-, gender- and race-matched controls.
Domain 2: Exposure Characterization






Metric 4:
Measurement of Exposure
Medium
X
0.4
0.8
Job exposure matrix
Metric 5:
Exposure levels

Medium
X
0.2
0.4
Indicators of probability, frequency and intensity;
tertiles for cumulative hours exposed.
Metric 6:
Temporality

High
X
0.4
0.4
Exposure lagged to account for cancer latency.
Domain 3: Outcome Assessment






Metric 7:
Outcome measurement or
characterization
High
X
0.667
0.67
Cases identifies by cancer surveillance system and
many histologically confirmed.
Metric 8:
Reporting Bias

High
X
0.333
0.33
Odds ratios reported with 95% confidence inter-
vals for kidney cancer and exposure to TCE, CCL4,
DCM and Perc
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding
High
High
Medium
x 0.5
x 0.25
x 0.25
0.5
0.25
0.5
Adjusted for age, sex, race, study centre, education
level, smoking status, BMI and history of hyperten-
sion.
Some covariate information was self-reported (smok-
ing, hypertension, race)
Domain 5: Analysis
Metric 12:	Study Design and Methods
Metric 13:	Statistical power
Metric 14:	Reproducibility of analyses
Medium X 0.4	0.8 Case-control study used to evaluate occupational
TCE, Perc, DCM, and CC14 exposure and kidney
cancer.
Medium X 0.2	0.4 Between Medium and Unacceptable, Medium is the
better characterization. An elevated risk of TCE
was detected - it just wasn't stat sig.
Medium X 0.2	0.4 Odds ratios calculated with unconditional logistic
regression.
Continued on next page

-------
. continued from previous page
Study Citation: Purdue, MP; Stewart, PA; Friesen, MC; Colt, JS; Locke, SJ; Hein, MJ; Waters, MA; Graubard, BI; Davis, F; Ruterbusch, J; Schwartz,
K; Chow, WH; Rothman, N; Hofmann, JN (2016). Occupational exposure to chlorinated solvents and kidney cancer: A case-control
study Occupational and Environmental Medicine, 74(4), 268-274
Data Type:	Case-control study of kidney cancer in workers exposed to chlorinated solvents - DCM_50-89% OR-Cancer
HERO ID:	3482059
Domain
Metric

Ratingt MWF*
Score
Commentstt
Metric 15
Statistical models

Medium x 0.2
0.4
Adjustments used in determining ORs clearly stated.
Domain 6: Other Considerations for Biomarker Selection and Measurement



Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination'	High	1.4
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWFj) / ]T\ MWF?
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 62: Celanese Fibers, Inc 1987: Evaluation of Hepatic Outcomes
Study Citation: Celanese Fibers Inc (1987). Methylene chloride analysis of liver function tests with attachments and cover letter dated 091887 #jour-
nal#, #volume# (tissue#), #Pages#
Data Type: Celriver Plant DCM exposed workers Hepatic endpoint-Hepatic
HERO ID: 4213851
Domain
Metric
Rating^
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4
Metric 2: Attrition
Metric 3: Comparison Group
x 0.4
Medium x 0.2
1.2 The study reported that all individuals were work-
ers in the same company. Group A was employed
for 10 or more years, and Group B 5 or more years.
However it was not reported how long the unexposed
controls were employed with the company. No other
inclusion/exclusion criteria were reported (age, sex,
health status etc.) nor was recruitment or partici-
pation rate reported.
1.2 Group A = 37, Group B =59, Controls = 32. Results
on page 6 show minimal loss in each group (although
the reason for loss was not reported) and the control
group s ho wed no loss.
0.4 There is no direct evidence that the comparison
groups were similar (characteristics not reported).
However, all individuals were from the same com-
pany, and assumed to be tested within the same time
frame, so there is indirect evidence of similar com-
parison groups.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Unacceptable x 0.4
Metric 5: Exposure levels
Metric 6: Temporality
Low

x 0.2
Medium x 0.4
0.16 There is no information about how exposure was as-
sessed (only that employees were exposed to DCM
or not). It is reported that DCM levels in the ex-
posed groups was greater than 250 ppm. There is
no indication when this measure was taken (at the
time of the study, over the 10 years of previous em-
ployment, what job functions this measure applies
to. . . ). Normal ranges of parameters seem to be
obtained from Roche Biomedical Lab; no statistics
were run on analysis, no measures of exposure taken,
no details on population analyzed.
0.6 There are 3 levels of exposure (controls, exposed 5
years, exposed 10 years). Groups are divided by
exposure duration is not by exposure level; both
groups exposed to levels greater than 250ppm.
0.8 Employees were exposed at least 5 or 10 years to
DCM before outcome measurements were taken. It
is unclear if exposures fall within relevant window.
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: Celanese Fibers Inc (1987). Methylene chloride analysis of liver function tests with attachments and cover letter dated 091887 #jour-
nal#, ^volume#(tissue#), ^Pages#
Data Type: Celriver Plant DCM exposed workers Hepatic endpoint-Hepatic
HERO ID: 4213851
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 7:
Outcome measurement or characterization
Low
x 0.667
2
The study reported that the outcome assessments
v/ere "commonly used liver function tests (LD,
SGOT, SGPT, TOT, BIL)." However, the methods
were not reported. Time of sample collection was
not reported nor was time from collection to analy-
sis. Normal parameter values for these tests appear
to be obtained from Roche Biomedical lab.
Metric 8:
Reporting Bias
High
x 0.333
0.33
Means, SDs, and Ns are reported.
Domain 4: Potential Counfounding/Variable Control

Metric 9:
Covariate Adjustment
Not Rated
NA
NA
There are no reporting of confounders or con founder
adjustments.
Metric 10
Covariate Characterization
Not Rated
NA
NA
No indication of covariate assessment
Metric 11
Co-exposure Confounding
Low
x 1
3
Although it is unclear what other chemicals these
workers were exposed to (none are reported), it is
likely that there were co-exposures working in this
plant.
Domain 5: Analysis

Metric 12
Study Design and Methods
Medium
x 0.667
1.33
There is no detailed information about study design,

but it is acceptable. Statistics were not employed in
this study.
Metric 13
Statistical power
Not Rated
NA
NA
Statistics were not employed in this study. Group
A (n=37), Group B (n=59) and control (n=32) are
small to apply to the general population
Metric 14
Reproducibility of analyses
Low
x 0.333
1.0
Details of the study design are not reported, and
thus would be difficult to replicate.
Metric 15
Statistical models
Not Rated
NA
NA
No stats were employed.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Unacceptable*'

2.7

Extracted No
Continued on next page

-------
. continued from previous page
Study Citation: Celanese Fibers Inc (1987). Methylene chloride analysis of liver function tests with attachments and cover letter dated 091887 #jour-
nal# , #volume#(#issue#), #Pages#
Data Type:	Celriver Plant_DCM_exposed workers_Hepatic endpoint-Hepatic
HERO ID:	4213851
Domain	Metric	Rating^	MWF* Score	Comments^
** Consistent with our Application of Systematic Review in TSCARisk Evaluations document, if a metric for a data source receives a score of Unacceptable (score = 4), EPA
will determine the study to be unacceptable. In this case, one or more of the metrics were rated as unacceptable. As such, the study is considered unacceptable and the score
is presented solely to increase transparency.
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating = ^
]T. (Metric Score; X MWF;) / V . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is crossed
out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 63: General Electric, Co 1990: Evaluation of Hepatic Outcomes
Study Citation: General Electric Company (1990). Morbidity study of occupational exposure to methylene chloride using a computerized surveillance
system (final report) with cover sheets and letter dated 041190 ^journal#, #volume#(tissue#), #Pages#
Data Type:	Occupational	DCM	Hepatic	GGT	HighExpoeure-Hepatic
HERO ID:	4213921
Domain
Metric
Rating*
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
Medium	X 0.4	0.8 Employees were grouped into one of four exposure
categories, based on MeCL measurements from per-
sonal air monitoring. Only males who completed the
medical exam and worked in key job functions were
included in analysis; the only exception was in the
analysis of breast cancer. However, there was no
information about participation selection or rates.
0.8 There was very little information about attrition,
however it was reported that "5 v/orkers refused the
medical examination entirely in 1984." It is unclear
if this is the only attrition that occurred during the
study. Final numbers were 896 males (19 workers in
the high, 49 in the intermediate, 56 in the low, and
722 in the minimal/none).
Medium x 0.2	0.4 There is no information about the similarity of
groups, but they are from the same factories, so in-
direct evidence that they are similar..
Medium x 0.4
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
g
Medium
Metric 6: Temporality
Medium
X 0.4	0.4 Exposures were determined by personal and area
monitoring levels, duration of monitoring, work
zone, job classification, and method of sampling.
Exposure methods v/ere v/ell detailed.
X 0.2	0.4 There were 4 exposure groups: 1) low exposure
(mean = 3.3 ppm), 2) medium exposure (mean =
10.9 ppm), 3) high exposures (mean = 49.0 ppm),
4) "other groups" with minimal or no exposure to
MeCl (<1.0 ppm); all based on personal air mon-
itoring conducted 1979-1985. Exposures known to
reach up to 150 ppm during specific manufacturi.ng
steps.
X 0.4	0.8 Exposures occurred before medical examinations,
however it is not clear how long workers were em-
ployed before examination
Domain 3: Outcome Assessment
Continued on next page

-------
.. . continued from previous page
Study Citation: General Electric Company (1990). Morbidity study of occupational exposure to methylene chloride using a computerized surveillance
system (final report) with cover sheets and letter dated 041190 ^journal#, #volume#(tissue#), #Pages#
Data Type:	Occupational	DCM	Hepatic	GGT	HighExposure-Hepatic
HERO ID:	4213921
Domain
Metric

Rating^
MWF*
Score
Commentstt
Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
Medical data was collected by the plant physi-







cian (medical history and physical examination, and







medical equipment results: sphvgmo-manometer,







spirometer, electrocardiographs, audiogram, self-







reported family history which physician follow up



Medium



with worker about.)
Metric 8:
Reporting Bias

X
0.333
0.67
SD/SE are not reported; percentages were reported.
Domain 4: Potential Counfounding/Variable Control






Metric 9:
Covariate Adjustment

High
X
0.5
0.5
Sex was adjusted for in the final analysis (females






analysis was removed. The study reported that the







mean age was 35.3 years, predomi.nant 1 y white and



High



male. All analysis were adjusted for age and race
Metric 10:
Covariate Characterization

X
0.25
0.25
Age, race, sex were collected by a medical physician
during an annual checkup
Metric 11:
Co-exposure Confounding

Low
X
0.25
0.75
Co-exposure to phenol around reaction vessels, high
noise levels, and potentially other hazardous mate-







rials in small amounts at the BPA plant and phos-






gene, high noise level, and other catalysts at the







resin plant are mentioned but not adjusted for. High
noise level suggested to add to headaches
Domain 5: Analysis







Metric 12:
Study Design and Methods

Medium
X
0.4
0.8
Study design was acceptable for this type of cross-
sectional study; Workers at a BPA plant were cat-
egorized based on personal exposure and job titles
into exposure categories (little/none, low, medium
and high) and assessed for relationships with vertigo
experience)
Metric 13:
Statistical power

Medium
X
0.2
0.4
Final numbers v/ere 19 workers in the high, 49 in
the intermediate, 56 in the low, and 722 in the min-
imal/ none.
Metric 14:
Reproducibility of analyses

Medium
X
0.2
0.4
The methods of collection of exposure and outcome
data were clearly described.
Metric 15:
Statistical models

Low
X
0.2
0.6
Bivariate and multivariate analysis was achieved uti-
lizing an A NOVA to observe for differences between
groups, a cross-tabulation was performed using chi-
square to identify associations with categorical vari-
ables from the medical exam; very minimal explana-
tion of analysis provided
Domain 6: Other Considerations for Biomarker Selection and Measurement





Metric 16:
Use of Biomarker of Exposure

Not Rated
NA
NA

Continued on next page

-------
. continued from previous page
Study Citation: General Electric Company (1990). Morbidity study of occupational exposure to methylene chloride using a computerized surveillance
system (final report) with cover sheets and letter dated 041190 #journal#, #volume#(#issue#), #Pages#
Data Type:	Occupational_DCM_Hepatic_GGT_HighExposure-Hepatic
HERO ID:	4213921
Domain
Metric
Rating^ MWF* Score
Comments^
Metric 17:
Effect biomarker
High
x 0.2
0.2
Well established biomarkers for hepatic health were




used: serum gamma glutamyl transferase (GGT),
serum total bilirubin, serum aspartate amino-
transferase (AST), and serum alanine aminotrans-
ferase (ALT).
Metric 18:
Method Sensitivity
Medium
x 0.2
0.4
Limits of detection not stated, but values reported




for most of the subjects (missing some endpoints for
6 subjects out of the >800 presented).
Metric 19:
Biomarker stability
Medium
x 0.2
0.4
Storage history not described, but do not have a high
likelihood of biomarker instability.
Metric 20:
Sample contamination
Low
x 0.2
0.6
No documentation of steps used to ensure contami-
nation free from collection to measurement.
Metric 21:
Method requirements
Medium
x 0.2
0.4
Method of quantification not stated, but standard
clinical tests.
Metric 22:
Matrix adjustment
Not Rated
NA
NA

Overall Quality Determination1

Medium

1.9

Extracted

Yes



* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T\ (Metric Score; X MWF;) / . MWF;
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of si udy

-------
Table 64: General Electric, Co 1990: Evaluation of Neurological/Behavior Outcomes
Study Citation: General Electric Company (1990). Morbidity study of occupational exposure to methylene chloride using a computerized surveillance
system (final report) with cover sheets and letter dated 041190 ^journal#, #volume#(tissue#), #Pages#
Data Type:	Occupational	Neuro	DCM	High Exposed-Neurological/Behavior
HERO ID:	4213921
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
dium x 0.4

Medium x 0.4

Metric 3: Comparison Group
um x 0.2
0.8 Employees were grouped into one of four exposure
categories, based on MeCL me as u r e me nt s from per-
sonal air monitoring. Only males who completed the
medical exam and worked in key job functions were
included in analysis; the only exception was in the
analysis of breast cancer. However, there was no
information about participation selection or rates.
0.8 There was very little information about attrition,
however it was reported that "5 workers refused the
medical examination entirely in 1984." It is unclear
if this is the only attrition that occurred during the
study. Final numbers were 896 males (19 workers in
the high, 49 in the intermediate, 56 in the low, and
722 in the minimal/none).
0.4 There is no information about the similarity of
groups, but they are from the same factories, so in-
direct evidence that they are similar.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
x 0.4
Medium x 0.2
Metric 6: Temporality
um x 0.4
0.4 Exposures were determined by personal and area
monitoring levels, duration of monitoring, work
zone, job classification, and method of sampling.
Exposure methods v/ere well detailed.
0.4 There were 4 exposure groups: 1) low exposure
(mean = 3.3 ppm), 2) medium exposure (mean =
10.9 ppm), 3) high exposures (mean = 49.0 ppm),
4) "other groups" with minimal or no exposure to
MeCl (<1.0 ppm); all based on personal air mon-
itoring conducted 1979-1985. Exposures known to
reach up to 150 ppm during specific, manufacturing
steps.
0.8 Exposures occurred before medical examinations,
however it is not clear how long workers were em-
ployed before examination
Domain 3: Outcome Assessment
Continued on next page

-------
. continued from previous page
Study Citation: General Electric Company (1990). Morbidity study of occupational exposure to methylene chloride using a computerized surveillance
system (final report) with cover sheets and letter dated 041190 ^journal#, ^volume# (tissue#), ^Pages#
Data Type:	Occupational	Neuro	DCM	High Exposed-Neurological/Behavior
HERO ID:	4213921
Domain
Metric

Rating^
MWF*
Score
Comments^
Metric 7:
Outcome measurement or characterization
Medium
X
0.667
1.33
Medical data was collected by the plant physi-







cian (medical history and physical examination, and
medical equipment results: sphygmo-manometer.







spirometer, electrocardiographs, audiogram, self-







reported family history which physician follow up







with worker about.)
Metric 8:
Reporting Bias

Medium
X
0.333
0.67
SD/SE are not reported; percentages were reported.
Domain 4: Potential Counfounding/Variable Control






Metric 9:
Covariate Adjustment

High
X
0.5
0.5
Sex was adjusted for in the final analysis (females
analysis was removed. The study reported that the
mean age was 35.3 years, predomi.nantly white and



Hlgh



male. All analysis were adjusted for age and race
Metric 10:
Covariate Characterization

X
0.25
0.25
Age, race, sex were collected by a medical physician
during an annual checkup
Metric 11:
Co-exposure Confounding

Low
X
0.25
0.75
Co-exposure to phenol around reaction vessels, high
noise levels, and potentially other hazardous mate-







rials in small amounts at the BPA plant and phos-






gene, high noise level, and other catalysts at the






resin plant are mentioned but not adjusted for. High
noise level suggested to add to headaches
Domain 5: Analysis







Metric 12:
Study Design and Methods

Medium
X
0.4
0.8
Study design was acceptable for this type of cross-
sectional study; Workers at a BPA plant were cat-
egorized based on personal exposure and job titles
into exposure categories (little/none, low, medium
and high) and assessed for relationships with vertigo



Medium



experience)
Metric 13:
Statistical power

X
0.2
0.4
Final numbers were 19 workers in the high, 49 in
the intermediate, 56 in the low, and 722 in the min-
imal/none.
Metric 14:
Reproducibility of analyses

Medium
X
0.2
0.4
The methods of collection of exposure and outcome






data were clearly described.
Metric 15:
Statistical models

Low
X
0.2
0.6
Bi.vari.ate and multivariate analysis was achieved uti-
lizing an AN OVA to observe for differences between
groups, a cross-1abu 1 ation was performed using chi-
square to identify associations with categorical vari-
ables from the medical exam; very minimal explana-
tion of analysis provided
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16: Use of Biomarker of Exposure	NA	NA
Continued on next page . ..

-------
. continued from previous page
Study Citation: General Electric Company (1990). Morbidity study of occupational exposure to methylene chloride using a computerized surveillance
system (final report) with cover sheets and letter dated 041190 ^journal#, ^volume# (tissue#), ^Pages#
Data Type:	Occupational	Neuro	DCM	High Exposed-Neurological/Behavior
HERO ID:	4213921
Domain	Metric	Rating^ MWF* Score	Commentstt
Metric 17
Effect biomarker

NA
NA
Metric 18
Method Sensitivity

NA
NA
Metric 19
Biomarker stability

NA
NA
Metric 20
Sample contamination

NA
NA
Metric 21
Method requirements

NA
NA
Metric 22
Matrix adjustment

NA
NA
Overall Quality Determination*	Medium	1.9
Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4	if any metric is Unacceptable
Overall rating =
]T. (Metric Score; x MWF;) / J] . MWFj
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 65: Gibbs 1992: Evaluation of Cancer Outcomes
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), #Pages#
Data Type: Methylene chloride occupational prostate subcohort 1 high exposure >20 years latency-Cancer
HERO ID: 4214006
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection

Metric 2: Attrition
Metric 3: Comparison Group
X 0.4 0.4 All key elements of the study design are reported
including the setting, methods of participant
selection, participation rate at all steps of the study,
and inclusion/exclusion criteria. The total study
population including exposed and not exposed em-
ployees was n=3211 (2187 men, 1024 women).The
authors report that 3220 persons were eligible
for the study, but nine of those had inaccurate
information concerning employment dates. The
total number of exposed employees was n=2909
(1931 men, 978 women).
The authors explain that the cohort could
have included 4468 eligible employees if the initial
protocol had been followed. The original protocol
called for all eligible employees on the payroll in
1954 and subsequent years. However, there were
some issues with missing employee records for the
period 1954-1969. The issues are fully described by
the authors including all of the efforts taken to find
the missing records. In the end, the investigators
chose to only include employees on the payroll on
or after January 1, 1970.
0.4 There was minimal subject loss to follow up dur-
ing the study. Death certificates were obtained for
95,8% (252/263) of the decedents in the high expo-
sure category ("subcohort 1"), 97% (350/361) of the
decedents in the low exposure category ("subcohort
2"), and 98% (108/110) of the decedents in the not
exposed category ("subcohort 3").
ijm X 0.2 0.4 The mortality of exposed employees was compared
to three reference populations: the general popula-
tions of Allegany County and the State of Maryland,
and the total white population of the U.S. (used for
subcohort 1 only). The results were stratified by
sex, and the calculation of SMRs incorporated the
5-year age and sex specific mortality rates for the
reference populations. There were no adjustments
for or stratification by race.
High x 0.4
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, # volume# (tissue#), ^Pages#
Data Type:	Methylene chloride	occupational	prostate	subcohort 1 high exposure	>20 years latency-Cancer
HERO ID:	4214006
Domain
Metric
Rating^ MWF* Score
Commentstt
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4
to
to
to

Metric 5: Exposure levels
Medium x 0.2
1.2 Exposure was estimated solely using professional
judgement. The authors report that "measurements
of the concentrations of methylene chloride in the air
of the plant were not available." Because of this, the
authors used concentrations measured at another
plant owned by the same company to estimate ex-
posures for this study: "The median time-weighted
average concentration for jobs in the "Extrusion and
Preparation" areas at the Celriver plant (Ott et al
1983) was 475 ppm." Discussions with persons fa-
miliar with the Amcelle plant (current study) sug-
gested that the concentrations in the extrusion area
would have been about 7 times that in the bobbin
shops and other low exposure areas. Based on those
discussions and the industrial hygiene survey at the
Celriver plant, the authors categorized departments
with a range of 50-100 ppm as a " 1" for exposure and
the departments with concentrations in the range of
350-700 ppm as "7" for exposure. It was assumed
that the same concentrations were present through-
out the entire operation of the plant.
0.4 There were three exposure levels (high, low, and not
exposed). Each department was assigned a category
of methylene chloride exposure (0, 1, or 7), and this
was used to calculate an index representing the cu-
mulative exposure of each worker. The main co-
hort was divided into three subcohorts on the ba-
sis of exposure: Subcohort 1 included all persons
who ever worked in an area of the plant involving
high (category 7) concentrations of methylene chlo-
ride (could have been in any department but had
at least some time in a department considered high
exposure); Subcohort 2 included persons who ever
worked in an area of the plant with low (category 1)
methylene chloride concentrations (never worked in
high exposure department, but could have v/orked in
non-exposure areas); Subcohort 3 included persons
who according to their work histories never worked
in any methylene chloride exposed departments or
jobs.
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), ^Pages#
Data Type: Methylene chloride occupational prostate subcohort 1 high exposure >20 years latency-Cancer
HERO ID: 4214006
Domain Metric
Rating^
MWF*
Score
Comments^
Metric 6: Temporality
High
X
0.4
0.4
Temporality is established and consideration was
given to the interval between exposure and outcomes
of interest. Employees were eligible if they were on
the payroll or joined the company on or after Jan-
uary 1, 1970. In addition, they must have worked
for more than 3 months at the plant. Follow-up was
for the period 1970-1989. A latency of 20 years from
first exposure to death was included in the analyses
of malignant neoplasms.
Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization Medium
X
0.667
1.33
The causes of death were determined from death

certificates. The vital status of each employee was

ascertained using a variety of different approaches

including company records, the National Death In-

dex, and social security file searches performed by

two separate organizations. A nosologist reviewed

the death certificates and assigned the underlying

causes of death according to ICD-9. Medical records

were not obtained.
Metric 8: Reporting Bias
Medium
X
0.333
0.67
Confidence intervals are not reported for the SMRs.
The observed and expected numbers of deaths are
reported for each cause of death in all data tables.
The text and data tables indicate which effects v/ere
considered statistically significant with a p value <
0.05 or 0.01.
Domain 4: Potential Counfounding/Variable Control

Metric 9: Covariate Adjustment
Medium
X
0.5
1
The SMRs were calculated with 5-year age and sex
specific mortality rates. Results were stratified by
sex, but v/ere not adjusted for or stratified by race.
Metric 10: Covariate Characterization
High
X
0.25
0.25
The age and gender of each employee were ascer-
tained from company records.
Continued on next page . ..

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), ^Pages#
Data Type:	Methylene chloride	occupational	prostate	subcohort 1 high exposure	>20 years latency-Cancer
HERO ID:	4214006
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 11: Co-exposure Confounding
X 0.25 0.75 There is direct evidence of co-exposures in cohort
members which may have been unbalanced across
the study groups, and the co-exposures were not
addressed in the analyses. The authors note that,
"virtually all methylene chloride exposed workers
were exposed to acetone, methanol and "finishing
oils" and some workers were likely exposed to many
other chemicals." In addition, the authors make the
following comment regarding the significant excess
in prostate cancer mortality observed in the highly-
exposed employees: "Thus, while these men spent
many years exposed to methylene chloride, they may
have had even longer exposure to the cellulose ac-
etate extrusion process and other associated chemi-
cals."
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models

Medium x 0.4 0.8

Medium x 0.2 0.4
Low	x 0.2 0.6
Medium x 0.2 0.4
The study design chosen was appropriate for the re-
search question and the study uses an appropriate
statistical method to address the research question
(the Occupational Cohort Mortality Analysis Pro-
gram was used to perform mortality analyses).
The number of participants is adequate to detect an
effect in the exposed population. There were a total
of 2909 exposed subjects with 602 deaths analyzed.
The authors provide no description of the statistical
methods used to determine statistical significance.
The method used for calculating SMRs is transpar-
ent. The number of observed and expected deaths
in each 5 year interval from 1970 through 1989 in-
clusive were determined and SMRs were calculated
using the OCMAP (Occupational Cohort Mortality
Analysis Program) for personal computer. The 5-
vear age and sex specific, mortality rates which were
used in various analyses included those for the white
population of the United States, State of Maryland,
and Allegany county.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Metric 17
Metric 18
Metric 19
Use of Biomarker of Exposure
Effect biomarker
Method Sensitivity
Biomarker stability
NA
NA
NA
NA
NA
NA
NA
NA
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), ^Pages#
Data Type:	Methylene chloride	occupational	prostate	subcohort 1 high exposure	>20 years latency-Cancer
HERO ID:	4214006
Domain Metric
Rating^
MWF*
Score
Comments^t
Metric 20: Sample contamination
Metric 21: Method requirements
Metric 22: Matrix adjustment

NA
NA
NA
NA
NA
NA

Overall Quality Determination*
Medium

1.9

Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4
Overall rating =
]T\ (Metric Score; X MWF;) / J] . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 66: Gibbs 1992: Evaluation of Respiratory Outcomes
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 #journal#, #volume#(#issue#), #Pages#
Data Type: Methylene chloride_occupational_respiratory_subcohort 1 high exposure_>20 years latency-Respiratory
HERO ID: 4214006
Domain
Metric
Rating^" MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
x 0.4

Metric 2: Attrition
Metric 3: Comparison Group
High x 0.4
.LVA^VAJL !_im x 0.2
0.4 All key elements of the study design are reported
including the setting, methods of participant
selection, participation rate at all steps of the study,
and inclusion/exclusion criteria. The total study
population including exposed and not exposed em-
ployees was n=3211 (2187 men, 1024 women).The
authors report that 3220 persons were eligible
for the study, but nine of those had inaccurate
information concerning employment dates. The
total number of exposed employees was n=2909
(1931 men, 978 women).
The authors explain that the cohort could
have included 4468 eligible employees if the initial
protocol had been followed. The original protocol
called for all eligible employees on the payroll in
1954 and subsequent years. However, there were
some issues with missing employee records for the
period 1954-1969. The issues are fully described by
the authors including all of the efforts taken to find
the missing records. In the end, the investigators
chose to only include employees on the payroll on
or after January 1, 1970.
0.4 There was minimal subject loss to follow up dur-
ing the study. Death certificates were obtained for
95,8% (252/263) of the decedents in the high expo-
sure category ("subcohort 1"), 97% (350/361) of the
decedents in the low exposure category ("subcohort
2"), and 98% (108/110) of the decedents in the not
exposed category ("subcohort 3").
0.4 The mortality of exposed employees was compared
to three reference populations: the general popula-
tions of Allegany County and the State of Maryland,
and the total white population of the U.S. (used for
subcohort 1 only). The results were stratified by
sex, and the calculation of SMRs incorporated the
5-year age and sex specific mortality rates for the
reference populations. There were no adjustments
for or stratification by race.
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, # volume# (tissue#), ^Pages#
Data Type:	Methylene chloride	occupational	respiratory	subcohort 1 high exposure	>20 years latency-Respiratory
HERO ID:	4214006
Domain
Metric
Rating^ MWF* Score
Commentstt
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
x 0.4
to
to
-a

Metric 5: Exposure levels
Medium x 0.2
1.2 Exposure was estimated solely using professional
judgement. The authors report that "measurements
of the concentrations of methylene chloride in the air
of the plant were not available." Because of this, the
authors used concentrations measured at another
plant owned by the same company to estimate ex-
posures for this study: "The median time-weighted
average concentration for jobs in the "Extrusion and
Preparation" areas at the Celriver plant (Ott et al
1983) was 475 ppm." Discussions with persons fa-
miliar with the Amcelle plant (current study) sug-
gested that the concentrations in the extrusion area
would have been about 7 times that in the bobbin
shops and other low exposure areas. Based on those
discussions and the industrial hygiene survey at the
Celriver plant, the authors categorized departments
with a range of 50-100 ppm as a " 1" for exposure and
the departments with concentrations in the range of
350-700 ppm as "7" for exposure. It was assumed
that the same concentrations were present through-
out the entire operation of the plant.
0.4 There were three exposure levels (high, low, and not
exposed). Each department was assigned a category
of methylene chloride exposure (0, 1, or 7), and this
was used to calculate an index representing the cu-
mulative exposure of each worker. The main co-
hort was divided into three subcohorts on the ba-
sis of exposure: Subcohort 1 included all persons
who ever worked in an area of the plant involving
high (category 7) concentrations of methylene chlo-
ride (could have been in any department but had
at least some time in a department considered high
exposure); Subcohort 2 included persons who ever
worked in an area of the plant with low (category 1)
methylene chloride concentrations (never worked in
high exposure department, but could have v/orked in
non-exposure areas); Subcohort 3 included persons
who according to their work histories never worked
in any methylene chloride exposed departments or
jobs.
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), ^Pages#
Data Type: Methylene chloride occupational respiratory subcohort 1 high exposure >20 years latency-Respiratory
HERO ID: 4214006
Domain Metric
Rating^
MWF*
Score
Comments^
Metric 6: Temporality
High
X
0.4
0.4
Temporality is established and consideration was
given to the interval between exposure and outcomes
of interest. Employees were eligible if they were on
the payroll or joined the company on or after Jan-
uary 1, 1970. In addition, they must have worked
for more than 3 months at the plant. Follow-up was
for the period 1970-1989. A latency of 20 years from
first exposure to death was included in the analyses
of malignant neoplasms.
Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization Medium
X
0.667
1.33
The causes of death were determined from death

certificates. The vital status of each employee was

ascertained using a variety of different approaches

including company records, the National Death In-

dex, and social security file searches performed by

two separate organizations. A nosologist reviewed

the death certificates and assigned the underlying

causes of death according to ICD-9. Medical records

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), ^Pages#
Data Type:	Methylene chloride	occupational	respiratory	subcohort 1 high exposure	>20 years latency-Respiratory
HERO ID:	4214006
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 11: Co-exposure Confounding
X 0.25 0.75 There is direct evidence of co-exposures in cohort
members which may have been unbalanced across
the study groups, and the co-exposures were not
addressed in the analyses. The authors note that,
"virtually all methylene chloride exposed workers
were exposed to acetone, methanol and "finishing
oils" and some workers were likely exposed to many
other chemicals." In addition, the authors make the
following comment regarding the significant excess
in prostate cancer mortality observed in the highly-
exposed employees: "Thus, while these men spent
many years exposed to methylene chloride, they may
have had even longer exposure to the cellulose ac-
etate extrusion process and other associated chemi-
cals."
Domain 5: Analysis
Metric 12: Study Design and Methods
Metric 13: Statistical power
Metric 14: Reproducibility of analyses
Metric 15: Statistical models

Medium x 0.4 0.8

Medium x 0.2 0.4
Low	x 0.2 0.6
Medium x 0.2 0.4
The study design chosen was appropriate for the re-
search question and the study uses an appropriate
statistical method to address the research question
(the Occupational Cohort Mortality Analysis Pro-
gram was used to perform mortality analyses).
The number of participants is adequate to detect an
effect in the exposed population. There were a total
of 2909 exposed subjects with 602 deaths analyzed.
The authors provide no description of the statistical
methods used to determine statistical significance.
The method used for calculating SMRs is transpar-
ent. The number of observed and expected deaths
in each 5 year interval from 1970 through 1989 in-
clusive were determined and SMRs were calculated
using the OCMAP (Occupational Cohort Mortality
Analysis Program) for personal computer. The 5-
vear age and sex specific, mortality rates which were
used in various analyses included those for the white
population of the United States, State of Maryland,
and Allegany county.
Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16
Metric 17
Metric 18
Metric 19
Use of Biomarker of Exposure
Effect biomarker
Method Sensitivity
Biomarker stability
NA
NA
NA
NA
NA
NA
NA
NA
Continued on next page

-------
. continued from previous page
Study Citation: Gibbs, GW (1992). Mortality or workers employed at a cellulose acetate & triacetate fibers plant in Cumberland, MD (final report)
with cover letter dated 061792 ^journal#, #volume#(tissue#), ^Pages#
Data Type:	Methylene chloride	occupational	respiratory	subcohort 1 high exposure	>20 years latency-Respiratory
HERO ID:	4214006
Domain Metric
Rating^
MWF*
Score
Comments^t
Metric 20: Sample contamination
Metric 21: Method requirements
Metric 22: Matrix adjustment

NA
NA
NA
NA
NA
NA

Overall Quality Determination*
Medium

1.9

Extracted	Yes
* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
4
Overall rating =
]T\ (Metric Score; X MWF;) / J] . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 67: Dow Chem, Co 1976: Evaluation of Skin and Connective Tissue Outcomes
Study Citation: Dow Chemical Company (1976). In-use safety study with an aerosol spray deodorant #443181- 10 (633-65a) and an aerosol spray
antiperspirant #d4247-41a with cover letter dated 042181 ^journal#, # volume# (tissue#), #Pages#
Data Type: controlled randomized trial DCM Skin irritation-Skin and Connective Tissue
HERO ID: 4214072
Domain
Metric
Rating^
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
Low
Medium
X 0.4 1.2 There is no information on inclusion/exclusion cri-
teria, or from what population the participants were
selected.
X 0.4 0.8 There is no information about attrition in this study,
however, the study indicates no loss.
X 0.2 0.4 The age range was from 18-60 years (although there
was no indication of the age range of each group).
Group 1 had 47 males/28 females, group 2 had 25
males/25 females. No other information was pro-
vided on the two groups. In addition, there was no
control group that had a placebo spray.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Low
Metric 5: Exposure levels
Metric 6: Temporality
Unacceptable
High
X 0.4 1.2 This was a controlled trial and there was no specific
measurement of exposure. Samples tested contained
similar amounts of DCM (21.5% in one compound
and 20% in the other). However, exposures may
have varied by subjects as they were instructed to
spray the entire axillary vault of both arms for 2
seconds at a distance of 6 inches.
X 0.2 0.04 There is no control group with no DCM exposure,
each formulation consisted of —20% AEROTHENE
MM which is 99.5% methylene chloride,
x 0.4 0.4 For the response of skin irritation, the time frame
(12 v/eeks) is sufficient to see responses
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization High
Metric 8: Reporting Bias
High
X 0.667 0.67 A standardized checklist of skin symptoms was used
by a single dermatologist after 1, 2, 4, 8, and 12
weeks
X 0.333 0.33 All raw data are reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
Medium X 0.5 1 Age and sex were similar between the two groups.
Medium x 0.25 0.5 No information was provided in how age and sex
was obtained, but it was likely based on self-report
from the subjects and there is little concern for self-
reporting of age or sex.
Continued on next page

-------
.. . continued from previous page
Study Citation: Dow Chemical Company (1976). In-use safety study with an aerosol spray deodorant #443181- 10 (633-65a) and an aerosol spray
antiperspirant #d4247-41a with cover letter dated 042181 ^journal#, ^volume# (tissue#), ^Pages#
Data Type: controlled randomized trial DCM Skin irritation-Skin and Connective Tissue
HERO ID: 4214072
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 11
Co-exposure Confounding

x 0.25
0.75
Co-exposures from the other ingredients in the de-
odorants were not accounted for. There were no con-
trols that received placebo without the DCM. In ad-
dition, there appears to have been differences in the
concentrate used with one of the formulas using alu-
minum chlorohvdrate.
Domain 5: Analysis

Metric 12
Study Design and Methods
Unacceptable
x 0.667
0.44
Although the study design may have been accept-
able for the study purpose, it is not acceptable for
the purpose of determining if DCM is a skin irri-
tant. There were no control groups that did not
receive DCM exposures and there were additional
compounds that may have caused any irritation re-
ported.
Metric 13
Statistical power
Medium
x 0.333
0.67
There were 125 subjects included, which would have
had enough statistical power.
Metric 14
Reproducibility of analyses
Not Rated
NA
NA
The study did not conduct any analyses on the re-
sults. They just noted there was slight transient
erythema , which was considered safe for marketing.
Metric 15
Statistical models
Not Rated
NA
NA
The study did not conduct any analyses on the re-
sults. They j ust noted there was slight transient
erythema , which was considered safe for marketing.
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Unacceptable**

2.3

Extracted No
Continued on next page

-------
. continued from previous page
Study Citation: Dow Chemical Company (1976). In-use safety study with an aerosol spray deodorant #443181- 10 (633-65a) and an aerosol spray
antiperspirant #d4247-41a with cover letter dated 042181 #journal#, #volume#(#issue#), #Pages#
Data Type:	controlled randomized trial_DCM_Skin irritation-Skin and Connective Tissue
HERO ID:	4214072
Domain	Metric	Rating^	MWF* Score	Comments^
** Consistent with our Application of Systematic Review in TSCARisk Evaluations document, if a metric for a data source receives a score of Unacceptable (score = 4), EPA
will determine the study to be unacceptable. In this case, one or more of the metrics were rated as unacceptable. As such, the study is considered unacceptable and the score
is presented solely to increase transparency.
*	MWF = Metric Weighting Factor
*	High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
if any metric is Unacceptable
Overall rating = '
]T. (Metric Score; X MWF,) / V . MWF J (round to the nearest tenth) otherwise
*	3	I 0.1
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is crossed
out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 68: Dow Chem, Co 1972: Evaluation of Skin and Connective Tissue Outcomes
Study Citation:
Data Type:
HERO ID:
Dow Chemical Company (1972). Human repeated insult patch test with four aerosol antiperspirant products ^journal#, #vol-
ume|(tissue#), ^Pages#
Randomized controlled trial DCM_
4214073
Skin Irritation-Skin and Connective Tissue
Domain
Metric
Rating^
MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
Metric 2: Attrition
Metric 3: Comparison Group
Low
Medium
X 0.4 1.2 There is no information on participant selection (in-
clusion/exclusion etc), but likely all were from a the
same population (same time frame, etc)
x 0.4 0.8 No information on attrition, but no reports of loss
during the study. Presumably all subjects noted to
be tested were all those included in the study ini-
tially.
X 0.2 0.4 Table 1 shows similar sex (1:1) and race ratio, all
groups had similar age ranges (16-59). However,
comparison was made for four samples of aerosol
antiperspirant and there does not appear to be a
control group.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
Metric 5: Exposure levels
Low
Unacceptable
Metric 6: Temporality
Domain 3: Outcome Assessment
Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias
High
Medium
X 0.4 1.2 Exposure via skin patches, there was a uniform way
of making these patches, but no reporting of how
much DCM was in each patch. It was noted that
formula 14-2 and 14-4 contained 15% DCM.
X 0.2 0.04 Two of the samples contained 15% DCM. However,
how this was applied to the skin was not reported.
It was only noted that a patch was applied on Mon-
day, Wednesday, and Thursday and allowed contact
with the skin for 24 hours. Although there were 4
different formulas tests and two of the formulas con-
tained DCM. There is in essence one exposure group
with DCM at 15%.
X 0.4 0.4 The time frame (4 days/24 hours) was an appropri-
ate time frame for the outcome of skin irritation
x 0.667
x 0.333
1.33 A checklist of skin irritation is provided (table 2),
but it is unclear if a dermatologist carried out the
assessments.
0.33 Raw data reported.
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Medium
x 0.5
Sex, age, race were similar. No other covariates were
noted or considered.
Continued on next page

-------
.. . continued from previous page
Study Citation: Dow Chemical Company (1972). Human repeated insult patch test with four aerosol antiperspirant products ^journal#, #vol-
ume#(#issue#), #Pages#
Data Type: Randomized controlled trial DCM Skin Irritation-Skin and Connective Tissue
HERO ID: 4214073
Domain Metric Rating^ MWF* Score Commentstt
Metric 10
Covariate Characterization
Medium
x 0.25
0.5
It was not reported how the sex, race, and age were

obtained, but it is likely was self-report and there is

little concern for the self-report for these variables.
Metric 11
Co-exposure Confounding
Low
x 0.25
0.75
Co-exposures were present in the formulations, how-
ever, there is no information provided on what is in
the different formulas so we do not know how any
of them compare or if they may contain other com-
pounds that are potential skin irritants. It is just
noted that two of the 4 formulas contain 15% DCM.
Domain 5: Analysis

Metric 12
Study Design and Methods
Unacceptable
x 0.667
0.44
Although the study design may have been accept-

able for the study purpose, it is not acceptable for

the purpose of determining if DCM causes skin sensi-
tization. There were no control groups that did not
receive DCM exposures and there were additional
compounds that may have caused any sensitization
reported.
Metric 13
Statistical power
Medium
x 0.333
0.67
There were 50 subjects, which was likely of sufficient
power. It is unclear if the 50 subjects were separated
into 4 different groups, but this should still provide
sufficient power to detect skin sensitization.
Metric 14
Reproducibility of analyses
Not Rated
NA
NA
The study did not conduct any analyses on the re-
sults. They just noted there was no evidence of skin
sensitization
Metric 15
Statistical models
Not Rated
NA
NA
The study did not conduct any analyses on the re-
sults. They just noted there was no evidence of skin
sensitization
Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination*
Unacceptable**

2.4

Extracted No
Continued on next page

-------
. continued from previous page
Study Citation: Dow Chemical Company (1972). Human repeated insult patch test with four aerosol antiperspirant products ^journal#, #vol-
ume#(#issue#), #Pages#
Data Type:	Randomized controlled trial_DCM_Skin Irritation-Skin and Connective Tissue
HERO ID:	4214073
Domain	Metric	Rating^	MWF* Score	Comments^
** Consistent with our Application of Systematic Review in TSCARisk Evaluations document, if a metric for a data source receives a score of Unacceptable (score = 4), EPA
will determine the study to be unacceptable. In this case, one or more of the metrics were rated as unacceptable. As such, the study is considered unacceptable and the score
is presented solely to increase transparency.
*	MWF = Metric Weighting Factor
*	High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating = ^
]T. (Metric Score; X MWF;) / V . MWF;
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is crossed
out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 69: Of f et al. 1983: Evaluation of Hematological and Immune Outcomes
Study Citation:
Data Type:
HERO ID:
Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride: Clinical laboratory evaluation Scandinavian Journal of Work, Environment and Health, 9(1)(tissue#), 17-25
DCM	occupational	retrospective cohort	exposed	white women	aspartate aminotransferase-Hematological and Immune
5240267

Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
dium x 0.4

CO
-4



Metric 2: Attrition
Medium x 0.4
0.8 Study population was described in a methodologi-
cal paper covering all aspects of the health surveil-
lance project (Ref ID: 24149). Briefly, exposed par-
ticipants were employees of a cellulose triacetate
and cellulose diacetate fiber manufacturing plant in
Rock Hill, South Carolina, exposed to were exposed
to methylene chloride, acetone, and methanol, the
methanol being present in a ratio of approximately 1
to 10 to methylene chloride. Unexposed participants
were from a non-DCM-exposure acetate fiber man-
ufacturing plant in Narrows, Virginia, who were ex-
posed to similar concentrations of acetone but were
not exposed to methylene chloride or ethanol. Par-
ticipation in the health examination was on a volun-
teer basis and was estimated to cover about 61 % of
the employees in the plant with methylene chloride
exposure and 55 % of the employees in the reference
plant.
0.8 Participation in the health examination was on a
volunteer basis, with 266 exposed and 251 unexposed
employees. There was no other specific mention of
attrition reported/addressed in this report.
ii next p
Continued o
age

-------
. continued from previous page
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride: Clinical laboratory evaluation Scandinavian Journal of Work, Environment and Health, 9(1)(tissue#), 17-25
Data Type: DCM occupational retrospective cohort exposed white women aspartate aminotransferase-Hematological and Immune
HERO ID: 5240267
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 3: Comparison Group
High X 0.2 0.2 Details on participants (e.g., race, sex, age, and
cigarette smoking.) were reported in the study re-
port. Cigarette smoking varied with sex and race;
however, there were no differences between the ex-
posed and reference groups within the sex-by race
subgroups. Among the exposed volunteers only 9 of
266 (3.4 %) had been employed less than one year
and 169 (63.5 %) had been employed for more than
five years at the time of the examination. In the
reference plant, the percentages were 13.9 and 55.1
%, respectively. In addition, the regression analy-
ses controlled for sex, race, age, cigarette smoking
history, time of venipuncture. The authors acknowl-
edge potential differences in the collection and han-
dling of the blood specimens between exposed and
unexposed workers that might bias the results, and
hence did not perform direct comparisons of labora-
tory findings between exposed and unexposed.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
High x 0.4 0.4
Metric 5: Exposure levels
m x 0.2
0.4
The results of the industrial hygiene monitoring of
the work environment are detailed in another re-
port (Ref ID: 29149.) (Eight-hr TWA concentrations
and peak concentrations were determined for both
plants. Personal air monitoring (>350 samples),
area sampling (170 samples), and short-term excur-
sion sampling (20 samples) were performed over the
course of a 3.5-month survey period in late 1977-
early 1978. Details of the personal air sampling
methods are described in an appendix to the study
report.). Median time weighted average concentra-
tions of methylene chloride for an 8-h day were pre-
sented in this report for exposed employees in vari-
ous work areas.
Occupational DCM exposure was categorized into
four levels across a sufficient range: unexposed, 60
and 140 ppm DCM, 280 ppm, and 475 ppm DCM.
Continued on next page

um x 0.333 0.67
Analyses of blood samples for both exposed ad un-
exposed employees were performed by the same lab-
oratory. Analyses are described in detail and are
adequate. However, there were differences in the
collection (posture, time of day, altitude) and han-
dling of the blood specimens between exposed and
unexposed workers that might bias the results, and
hence did not perform direct comparisons of labora-
tory findings between exposed and unexposed.
The blood constituents examined were red cell
count, hemoglobin, hematocrit, mean corpuscu-
lar volume, mean corpuscular hemoglobin, mean
corpuscular hemoglobin concentration, car boxy-
hemoglobin, aspartate aminotransferase, alanine
aminotransferase, lactate dehydrogenase, alkaline
phosphatase, total bilirubin, and
albumin. Regression results are mainly presented
with effect estimates and p-values (lacking standard
errors or confidence intervals).
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
High X 0.5 0.5 Covariates used in the analyses were sex, race,age,
cigarette smoking history, time of venipuncture,.
Additional variables evaluated as potential covari-
ates were date of examination, and intensity of ace-
tone exposure within the reference plant.
JJU w x 0.25 0.75 There is no direct information in this report on cov-
rariate characterization, however it is likely that the
main source of information is the health evaluation
and/or company records.
Continued on next page

-------
. continued from previous page
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride: Clinical laboratory evaluation Scandinavian Journal of Work, Environment and Health, 9(l)(#issue#), 17-25
Data Type:	DCM_occupational_retrospective cohort_exposed_white women_aspartate aminotransferase-Hematological and Immune
HERO ID:	5240267
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 11
Co-exposure Confounding

x 0.25
0.75
Intensity of acetone exposure within the reference
plant. was evaluated as potential important co-
variate for blood constituents. The study report
also indicates that exposure to other chemicals (e.g.,
methanol, acetone) was possible at the South Car-
olina plant.
Domain 5: Analysis





Metric 12
Study Design and Methods
Medium
x 0.4
0.8
Study design (retrospective cohort) and analyses
were adequate for the research question.
Metric 13
Statistical power
Medium
x 0.2
0.4
The study included 266 exposed and 251 unexposed
workers.
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Statistical analyses are briefly described and likely
to be conceptually reproducible given access to the
analytic data.
Metric 15
Statistical models
ivOW
x 0.2
0.6
Regression analyses and covariates considered are
briefly described. There is no detail on model as-
sumptions, model selection, or sensitivity analyses.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination
Medium

1.8

Extracted





* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWF;) / J] . MWF?
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------
Table 70: Of f et al. 1983: Evaluation of Hepatic Outcomes
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride: Clinical laboratory evaluation Scandinavian Journal of Work, Environment and Health, 9(1)(tissue#), 17-25
Data Type:	DCM	occupational	retrospective cohort	total bilirubin	exposed	white women-Hepatic
HERO ID:	5240267
Domain
Metric
Rating^ MWF* Score
Comments^
Domain 1: Study Participation
Metric 1: Participant selection
dium x 0.4





Metric 2: Attrition
Medium x 0.4
0.8 Study population was described in a methodologi-
cal paper covering all aspects of the health surveil-
lance project (Ref ID: 24149). Briefly, exposed par-
ticipants were employees of a cellulose triacetate
and cellulose diacetate fiber manufacturing plant in
Rock Hill, South Carolina, exposed to were exposed
to methylene chloride, acetone, and methanol, the
methanol being present in a ratio of approximately 1
to 10 to methylene chloride. Unexposed participants
were from a non-DCM-exposure acetate fiber man-
ufacturing plant in Narrows, Virginia, who were ex-
posed to similar concentrations of acetone but were
not exposed to methylene chloride or ethanol. Par-
ticipation in the health examination was on a volun-
teer basis and was estimated to cover about 61 % of
the employees in the plant with methylene chloride
exposure and 55 % of the employees in the reference
plant.
0.8 Participation in the health examination was on a
volunteer basis, with 266 exposed and 251 unexposed
employees. There was no other specific mention of
attrition reported/addressed in this report.
ii next p
Continued o
age

-------
. continued from previous page
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride: Clinical laboratory evaluation Scandinavian Journal of Work, Environment and Health, 9(1)(tissue#), 17-25
Data Type: DCM occupational retrospective cohort total bilirubin exposed white women-Hepatic
HERO ID: 5240267
Domain
Metric
Rating^ MWF* Score
Commentstt
Metric 3: Comparison Group
High X 0.2 0.2 Details on participants (e.g., race, sex, age, and
cigarette smoking.) were reported in the study re-
port. Cigarette smoking varied with sex and race;
however, there were no differences between the ex-
posed and reference groups within the sex-by race
subgroups. Among the exposed volunteers only 9 of
266 (3.4 %) had been employed less than one year
and 169 (63.5 %) had been employed for more than
five years at the time of the examination. In the
reference plant, the percentages were 13.9 and 55.1
%, respectively. In addition, the regression analy-
ses controlled for sex, race, age, cigarette smoking
history, time of venipuncture. The authors acknowl-
edge potential differences in the collection and han-
dling of the blood specimens between exposed and
unexposed workers that might bias the results, and
hence did not perform direct comparisons of labora-
tory findings between exposed and unexposed.
Domain 2: Exposure Characterization
Metric 4: Measurement of Exposure
High x 0.4 0.4
Metric 5: Exposure levels
m x 0.2
0.4
The results of the industrial hygiene monitoring of
the work environment are detailed in another re-
port (Ref ID: 29149.) (Eight-hr TWA concentrations
and peak concentrations were determined for both
plants. Personal air monitoring (>350 samples),
area sampling (170 samples), and short-term excur-
sion sampling (20 samples) were performed over the
course of a 3.5-month survey period in late 1977-
early 1978. Details of the personal air sampling
methods are described in an appendix to the study
report.). Median time weighted average concentra-
tions of methylene chloride for an 8-h day were pre-
sented in this report for exposed employees in vari-
ous work areas.
Occupational DCM exposure was categorized into
four levels across a sufficient range: unexposed, 60
and 140 ppm DCM, 280 ppm, and 475 ppm DCM.
Continued on next page

um x 0.333 0.67
Analyses of blood samples for both exposed ad un-
exposed employees were performed by the same lab-
oratory. Analyses are described in detail and are
adequate. However, there were differences in the
collection (posture, time of day, altitude) and han-
dling of the blood specimens between exposed and
unexposed workers that might bias the results, and
hence did not perform direct comparisons of labora-
tory findings between exposed and unexposed.
The blood constituents examined were red cell
count, hemoglobin, hematocrit, mean corpuscu-
lar volume, mean corpuscular hemoglobin, mean
corpuscular hemoglobin concentration, carboxyhe-
moglobin, aspartate aminotransferase, alanine
aminotransferase, lactate dehydrogenase, alkaline
phosphatase, total bilirubin, and
albumin. Regression results are mainly presented
with effect estimates and p-values (lacking standard
errors or confidence intervals).
Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment
Metric 10: Covariate Characterization
High X 0.5 0.5 Covariates used in the analyses were sex, race,age,
cigarette smoking history, time of venipuncture,.
Additional variables evaluated as potential covari-
ates were date of examination, and intensity of ace-
tone exposure within the reference plant.
JJU w x 0.25 0.75 There is no direct information in this report on cov-
rariate characterization, however it is likely that the
main source of information is the health evaluation
and/or company records.
Continued on next page

-------
. continued from previous page
Study Citation: Ott, MG; Skory, LK; Holder, BB; Bronson, JM; Williams, PR (1983). Health evaluation of employees occupationally exposed to
methylene chloride: Clinical laboratory evaluation Scandinavian Journal of Work, Environment and Health, 9(l)(#issue#), 17-25
Data Type:	DCM_occupational_retrospective cohort_total bilirubin_exposed_white women-Hepatic
HERO ID:	5240267
Domain
Metric
Rating^
MWF*
Score
Commentstt
Metric 11
Co-exposure Confounding

x 0.25
0.75
Intensity of acetone exposure within the reference
plant. was evaluated as potential important co-
variate for blood constituents. The study report
also indicates that exposure to other chemicals (e.g.,
methanol, acetone) was possible at the South Car-
olina plant.
Domain 5: Analysis





Metric 12
Study Design and Methods
Medium
x 0.4
0.8
Study design (retrospective cohort) and analyses
were adequate for the research question.
Metric 13
Statistical power
Medium
x 0.2
0.4
The study included 266 exposed and 251 unexposed
workers.
Metric 14
Reproducibility of analyses
Medium
x 0.2
0.4
Statistical analyses are briefly described and likely
to be conceptually reproducible given access to the
analytic data.
Metric 15
Statistical models
ivOW
x 0.2
0.6
Regression analyses and covariates considered are
briefly described. There is no detail on model as-
sumptions, model selection, or sensitivity analyses.
Domain 6: Other Considerations for Biomarker Selection and Measurement




Metric 16
Use of Biomarker of Exposure

NA
NA

Metric 17
Effect biomarker

NA
NA

Metric 18
Method Sensitivity

NA
NA

Metric 19
Biomarker stability

NA
NA

Metric 20
Sample contamination

NA
NA

Metric 21
Method requirements

NA
NA

Metric 22
Matrix adjustment

NA
NA

Overall Quality Determination
Medium

1.8

Extracted





* MWF = Metric Weighting Factor
t High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.
+ The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.
Overall rating =
]T\ (Metric Score; x MWF;) / J] . MWF?
if any metric is Unacceptable
(round to the nearest tenth) otherwise
where High = > 1 to < 1.7; Medium = > 1.7 to < 2.3; Low = > 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.
t This metric met the criteria for high confidence as expected for this type of study

-------