Draft Risk Evaluation for Carbon Tetrachloride Systematic Review Supplemental File: Data Quality Evaluation of Epidemiological Studies CASRN 56-23-5 January 2020


PEER REVIEW DRAFT, DO NOT CITE OR QUOTE

SEPA

United States	Office of Chemical Safety and

Environmental Protection Agency	Pollution Prevention

Draft Risk Evaluation for
Carbon Tetrachloride

Systematic Review Supplemental File:

Data Quality Evaluation of Epidemiological Studies

CASRN 56-23-5

ci

C'""CI

cr
ci

January 2020

1

-------
PEER REVIEW DRAFT, DO NOT CITE OR QUOTE

Table Listing

1	Davis 1934: Evaluation of Acute Toxicity/Poisoning Outcomes	 3

2	Radican et al. 2008: Evaluation of Cancer Outcomes 	 6

3	Radican et al. 2008: Evaluation of Respiratory Outcomes 	 9

4	Gold et al. 2010: Evaluation of Cancer Outcomes	 12

5	Roberts et al. 2013: Evaluation of Neurological/Behavior Outcomes 	 15

6	Goldman et al. 2012: Evaluation of Neurological/Behavior Outcomes	 18

7	Neta et al. 2012: Evaluation of Cancer Outcomes	 22

8	Ruder et al. 2013: Evaluation of Cancer Outcomes	 25

9	Vizcaya et al. 2013: Evaluation of Cancer Outcomes	 28

10	Morales-Suarez-Varela et al. 2013: Evaluation of Cancer Outcomes	 31

11	Heck et al. 2013: Evaluation of Cancer Outcomes 	 34

12	Davis 1934: Evaluation of Hematological and Immune Outcomes	 38

13	Mattei et al. 2014: Evaluation of Cancer Outcomes 	 41

14	Garcia et al. 2015: Evaluation of Cancer Outcomes 	 44

15	Carton et al. 2017: Evaluation of Cancer Outcomes	 47

16	Nelson et al. 2012: Evaluation of Cancer Outcomes	 50

17	Purdue et al. 2016: Evaluation of Cancer Outcomes	 53

18	Tomenson et al. 1995: Evaluation of Hepatic Outcomes	 55

19	Tomenson et al. 1995: Evaluation of Hematological and Immune Outcomes .... 60

20	Dow Chemical, Co 1992: Evaluation of Cancer Outcomes 	 65

21	Davis 1934: Evaluation of Renal Outcomes	 68

22	Davis 1934: Evaluation of Cardiovascular Outcomes	 71

23	Siemiatycki 1991: Evaluation of Cancer Outcomes	 74

24	Heineman et al. 1994: Evaluation of Cancer Outcomes	 77

25	Seidler et al. 2007: Evaluation of Cancer Outcomes	 81

26	Dosemeci et al. 1999: Evaluation of Cancer Outcomes	 84

27	Wang et al. 2009: Evaluation of Cancer Outcomes	 87

2

-------
1

Table 1: Davis 1934: Evaluation of Acute Toxicity/Poisoning Outcomes

Study Citation: P. A. Davis (1934). Carbon tetrachloride as an industrial hazard Journal of the American Medical Association, 103(13,13), 962-966

Data Type: Davis CC14 controlled inhalation exposure clinicalobs-Acute Toxicity/Poisoning

HERO ID: 3611

Domain

Metric

Ratingt MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Low

Metric 2: Attrition

Metric 3: Comparison Group

Medium
Low

X 0.4 1.2 Eight controlled experiments were conducted in to-

tal. Each experiment consisted of three to four indi-
viduals and one group of individuals was used for two
experiments. Age and basic clinical measurements
were provided for each subject. Some subjects may
have been used for multiple experiments, but this
is unclear. The method of recruitment was not de-
scribed and demographic details, including sex, were
not provided.

X 0.4 0.8 Subjects differed for all experiments but one. The

reason for this change from experiment to experi-
ment is not fully described.

X 0.2 0.6 No control group was used in this study. The mea-

sured outcomes were presumably compared to refer-
ence values, but the details are not clear.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low

Metric 5: Exposure levels

Metric 6: Temporality

Low

X 0.4 1.2 The inhalation chamber was adequately described.

The method of creating the inhalation exposure and
the method to monitor the exposure level were not
described. Source and purity of the test article are
not reported. Exposure duration varied by exposure
level. The seventh experiment described determin-
ing the carbon tetrachloride concentration by the al-
cohol potassium hydroxide and combustion method,
but it is unclear if this was used for other experi-
ments.

X 0.2 0.6 Multiple exposure levels were examined in this study

including 76 ppm, 158 ppm, 317 ppm, 1191 ppm,
2300 ppm and additional unreported levels, but ex-
posure duration varied by exposure concentration.

X 0.4 1.2 This study was a controlled inhalation exposure.

The timing of outcome measurement was not fully
described in the text and remains unclear, although
it is presumed that measurements were taken after
controlled exposure to carbon tetrachloride.

Domain 3: Outcome Assessment

Continued on next page . ..

-------
.. . continued from previous page

Study Citation: P. A. Davis (1934). Carbon tetrachloride as an industrial hazard Journal of the American Medical Association, 103(13,13), 962-966

Data Type: Davis CC14 controlled inhalation exposure clinicalobs-Acute Toxicity/Poisoning

HERO ID: 3611

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 7:

Outcome measurement or characterization

Low

x 0.667

Clinical observations were described, if present.
Hematology, urinalysis, and vital measurements
were taken, but the methods or other details on out-
come measurement were not reported. It was not re-
ported whether outcome investigators were blinded
to exposure during treatment.

Metric 8:

Reporting Bias

Medium

x 0.333

0.67

Outcomes were outlined throughout the paper and
clinical observations were described.

Domain 4: Potential Counfounding/Variable Control

Metric 9:

Covariate Adjustment

Low

x 0.667

A statistical analysis was not conducted. Age of

the test subjects was provided, but no other demo-
graphic information was presented or adjusted for.

Metric 10

Covariate Characterization

Not Rated

Covariates, besides age, were not collected.

Metric 11

Co-exposure Confounding

Medium

x 0.333

0.67

There was no indication of co-exposures being
present or measured for during the controlled inhala-
tion exposure.

Domain 5: Analysis

Metric 12

Study Design and Methods

Medium

x 0.5

This study utilized an inhalation chamber to ex-

amine the effects of acute inhalation exposures to
carbon tetrachloride. No concurrent control group
was used and clinical measurements were presum-
ably compared to reference standards. No statistical
analysis was applied to the results.

Metric 13

Statistical power

Medium

x 0.25

0.5

Three to four subjects were used in each controlled
inhalation experiment. This is a low number of in-
dividuals per experiment and results should be in-
terpreted with caution.

Metric 14

Reproducibility of analyses

Low

x 0.25

0.75

The inhalation chamber is described, but the
method of used to achieve the inhalation exposure
and ensure maintenance of an accurate dose are not
described. Also, timings of exposure and measured
outcomes were not reported.

Metric 15

Statistical models

Not Rated

Results were compared to reference values and de-
scribed qualitatively only.

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16

Use of Biomarker of Exposure

Metric 17

Effect biomarker

Metric 18

Method Sensitivity

Metric 19

Biomarker stability

Metric 20

Sample contamination

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:
Data Type:
HERO ID:

P. A. Davis (1934). Carbon tetrachloride as an industrial hazard Journal of the American Medical Association, 103(13,13), 962-966

Davis	CC14	controlled	inhalation	exposure	clinicalobs-Acute Toxicity/Poisoning

3611

Domain

Metric Rating^

MWF*

Score Comments^



Metric 21: Method requirements
Metric 22: Matrix adjustment

NA
NA

NA
NA

Overall Quality Determination1" Low



2.6

Extracted	Yes

MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; x MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 2: Radican et al. 2008: Evaluation of Cancer Outcomes

Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319

Data Type: Hill Air Force Base CC14 BreastCancer Females-Cancer

HERO ID: 699234

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

High X 0.4 0.4 This study consisted of an extended follow-up of

the Hill Air Force Base occupational cohort through
2000. The cohort is composed of former civilian em-
ployees, who worked at this aircraft maintenance fa-
cility for at least 1 year between January 1, 1952 and
December 31, 1956 (n= 14,455). The key elements of
the study design were reported. Selection into the
study was not likely to be biased. The cohort was
described in detail in previous publications (Spirtas
et al. 1991; Stewart et al. 1991; Blair et al. 1998).

High X 0.4 0.4 There was no loss of subjects to follow-up reported

in the study (as of December 31 2000, 8580 subjects
had died and 5875 were still alive); exposure and
outcome data were largely complete.

High X 0.2 0.2 Key elements of the study design are reported. Ef-

fects levels were adjusted for age, race, and/or sex.
The use of an internal comparison group likely re-
duces the risk of bias relative to the use of an exter-
nal reference group (e.g., the healthy worker effect).

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Medium x 0.4

Metric 5: Exposure levels

Metric 6: Temporality

Low
High

x 0.2
x 0.4

0.8 The exposure assessment was conducted by the Na-
tional Cancer Institute (NCI), using job-exposure
matrices, based on information provided by the Air
Force. Although exposure misclassification was pos-
sible (because individual exposure records were not
available), misclassification was likely random and
not to appreciably bias the results.

0.6 For 21 chemicals (including TCE, Perc, CC14 and
DCM), exposure was classified as yes/no. No quan-
titative assessment of exposure was conducted.

0.4 The study presents the appropriate relationship be-
tween exposure and outcome. Outcome was ascer-
tained after information on exposure was obtained.
There was a long follow-up period.

z
o

O
»
o

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

Data Type: Hill Air Force Base CC14 BreastCancer Females-Cancer

HERO ID: 699234

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 7:

Outcome measurement or characterization

Medium

x 0.667

1.33

The outcome was determined from death records
from the National Death Index (NDI). It was noted
in the study that mortality data can be mislead-
ing owing to inaccuracies captured in patient death
records.

Metric 8:

Reporting Bias

High

x 0.333

0.33

A description of measured outcomes is provided in
the study report. Effects estimates are provided
with confidence limits; number of exposed cases is
included.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

Metric 11: Co-exposure Confounding

Low X 0.5 1.5 Adjustments were made for age, race, and gen-

der. However, there was indirect evidence that so-
cioeconomic status (SES) was considerably differ-
ent among exposed and non-exposed populations.
The proportion of non-exposed persons that were
salaried was 61% compared to < 1% in the ex-
posed cohort, suggesting a dissimilar SES. This dif-
ference may affect the results for some specific cancer
types / d iseases.

Medium X 0.25 0.5 Confounders were assessed using reliable methods

(database of employees and NDI). However, other
than age, gender, and race, data on other factors
(disease history, SES) were not available.

Low X 0.25 0.75 The study evaluated exposure to CC14 and various

other chemicals. Exposures were not mutually ex-
clusive; therefore, it was not possible to evaluate the
risk of death from exposure to a singular chemical
while controlling for exposure to other chemicals.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Medium x 0.4 0.8

Medium x 0.2 0.4

The cohort design and calculation of hazard ratios
were appropriate for determining the association be-
tween exposure to TCE, Perc, CC14 and DCM, and
all-cause, cancer, and non-cancer mortality.

The cohort was large (adequate for statistical anal-
yses). Despite the relatively large size of the cohort,
the number of cases for many causes of death was
small to evaluate associations.

The analysis (exposure estimation and statistical
modeling) is described in sufficient detail to un-
derstand what was done and is conceptually repro-
ducible.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319

Data Type:	Hill	Air	Force	Base	CC14	BreastCancer	Females-Cancer

HERO ID:	699234

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 15

Statistical models

Medium

x 0.2

0.4

The method and model assumptions used to cal-
culate risk estimates for occupational exposure to
TCE, Perc, CC14 and DCM and all-cause and cause-
specific mortality (hazard ratios) are clearly de-
scribed in the study report.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

Medium



1.8



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

^ This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 3: Radican et al. 2008: Evaluation of Respiratory Outcomes

Data Type: Hill Air Force Base CC14 NonMalignantRespiratoryDisease-Respiratory

HERO ID: 699234

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

High X 0.4 0.4 This study consisted of an extended follow-up of

High X 0.4 0.4 There was no loss of subjects to follow-up reported

in the study (as of December 31 2000, 8580 subjects
had died and 5875 were still alive); exposure and
outcome data were largely complete.

High X 0.2 0.2 Key elements of the study design are reported. Ef-

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Medium x 0.4

Metric 5: Exposure levels

Metric 6: Temporality

Low
High

x 0.2
x 0.4

0.6 For 21 chemicals (including TCE, Perc, CC14 and
DCM), exposure was classified as yes/no. No quan-
titative assessment of exposure was conducted.

0.4 The study presents the appropriate relationship be-
tween exposure and outcome. Outcome was ascer-
tained after information on exposure was obtained.
There was a long follow-up period.

z
o

O
»
o

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

Data Type: Hill Air Force Base CC14 NonMalignantRespiratoryDisease-Respiratory

HERO ID: 699234

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 7:

Outcome measurement or characterization

Medium

x 0.667

1.33

Metric 8:

Reporting Bias

High

x 0.333

0.33

A description of measured outcomes is provided in
the study report. Effects estimates are provided
with confidence limits; number of exposed cases is
included.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

Metric 11: Co-exposure Confounding

Low X 0.5 1.5 Adjustments were made for age, race, and gen-

Medium X 0.25 0.5 Confounders were assessed using reliable methods

(database of employees and NDI). However, other
than age, gender, and race, data on other factors
(disease history, SES) were not available.

Low X 0.25 0.75 The study evaluated exposure to CC14 and various

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Medium x 0.4 0.8

Medium x 0.2 0.4

The cohort design and calculation of hazard ratios
were appropriate for determining the association be-
tween exposure to TCE, Perc, CC14 and DCM, and
all-cause, cancer, and non-cancer mortality.

The cohort was large (adequate for statistical anal-
yses). Despite the relatively large size of the cohort,
the number of cases for many causes of death was
small to evaluate associations.

The analysis (exposure estimation and statistical
modeling) is described in sufficient detail to un-
derstand what was done and is conceptually repro-
ducible.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation: Radican, L; Blair, A; Stewart, P; Wartenberg, D (2008). Mortality of aircraft maintenance workers exposed to trichloroethylene and
other hydrocarbons and chemicals: Extended follow-up Journal of Occupational and Environmental Medicine, 50(11), 1306-1319

Data Type:	Hill	Air	Force	Base	CC14	NonMalignantRespiratoryDisease-Respiratory

HERO ID:	699234

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 15

Statistical models

Medium

x 0.2

0.4

The method and model assumptions used to cal-
culate risk estimates for occupational exposure to
TCE, Perc, CC14 and DCM and all-cause and cause-
specific mortality (hazard ratios) are clearly de-
scribed in the study report.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

Medium



1.8



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

^ This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 4: Gold et al. 2010: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Gold, LS; Stewart, PA; Milliken, K; Purdue, M; Severson, R; Seixas, N; Blair, A; Hartge, P; Davis, S; De Roos, AJ (2010). The
relationship between multiple myeloma and occupational exposure to six chlorinated solvents Occupational and Environmental Medicine,
68(6), 391-399

Gold CC14 exposed workers cancer 1-4 yrs-Cancer

699241

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Medium x 0.4

Metric 3: Comparison Group

High
High

x 0.4

x 0.2

0.8 Study authors note a low participation rate of eli-
gible controls, with individuals in the youngest (35-
50) and oldest (65-75) age groups were less likely to
participate than those in the middle age group.

0.4 Low attrition for subjects that decided to participate
in study. Only one case was excluded because of
missing covariate information.

0.2 General population controls were selected from a
case-control study of non-Hodgkin's lymphoma un-
dertaken at the same time. Controls were identified
by random digit dialing with clear inclusion criteria.
A table of characteristics was not provided to evalu-
ate similarities, but adjustments were made for age,
race, site, gender, and years of education.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Metric 5: Exposure levels

Metric 6: Temporality

Low x 0.4 1.2 Use of a job-exposure matrix in a population based

study. Exposure based on participant interview
rather than detailed employment history records

Medium X 0.2 0.4 Reports referent group and 3 levels of exposure for

cumulative exposure and 10-year lagged cumulative
exposure.

High X 0.4 0.4 Cases were diagnosed between 2000 and 2002 while

exposure was assessed from 1941 to time of study
enrollment.

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization

Metric 8: Reporting Bias

High X 0.667 0.67 Cases were identified through the review of hospi-

tal medical records and records of selected pathol-
ogy laboratories, oncologists, radiologists and state
death certificates.

High X 0.333 0.33 Effect estimates are reported with a confidence inter-

val. The number of cases and controls are included
in a tabular format for date extraction and analysis.

z
o

O
»
o

Domain 4: Potential Counfounding/Variable Control

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Gold, LS; Stewart, PA; Milliken, K; Purdue, M; Severson, R; Seixas, N; Blair, A; Hartge, P; Davis, S; De Roos, AJ (2010). The
relationship between multiple myeloma and occupational exposure to six chlorinated solvents Occupational and Environmental Medicine,
68(6), 391-399

Gold	CC14	exposed workers	cancer	1-4 yrs-Cancer

699241

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 9:

Covariate Adjustment

High

x 0.5

0.5

Covariates gender, age (35-50 years (referent), 51-
64 years and 65-74 years), race (only white (refer-
ent), any black, any Asian and other), education
(less than 12 years (referent), 12-15 years and 16
or more years) and SEER site (Seattle and Detroit).

Metric 10

Covariate Characterization

Medium

x 0.25

0.5

Potential confounders were considered but method
validation not provided. However there is no evi-
dence that the method had poor validity.

Metric 11

Co-exposure Confounding

Low

x 0.25

0.75

Exposure to other chlorinated solvents was also as-
sessed with JEM. Study authors note that they re-
port the percentages of control subjects exposed to
these chemicals alone and to two of these chemicals
and provide an estimate of the association with mul-
tiple myeloma for subjects who were exposed to all
four (TCE, CC14, DCM, PERC). But analyses were
not adjusted for these exposures.

Domain 5: Analysis











Metric 12

Study Design and Methods

Medium

x 0.4

0.8

The case-control study design chosen was appropri-
ate for the exposure and outcome of interest.

Metric 13

Statistical power

Medium

x 0.2

0.4

The overall number of cases and controls are ade-
quate to detect an effect.

Metric 14

Reproducibility of analyses

Medium

x 0.2

0.4

The description of the analysis is sufficient to under-
stand what has been done.

Metric 15

Statistical models

Medium

x 0.2

0.4

There is sufficient information on how the ORs were
calculated.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

High —

Medium5





Extracted



Yes









O

H

o

O

z
o

H

n

HH

H

H

O
»
o

d

o

H

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Gold, LS; Stewart, PA; Milliken, K; Purdue, M; Severson, R; Seixas, N; Blair, A; Hartge, P; Davis, S; De Roos, AJ (2010). The
relationship between multiple myeloma and occupational exposure to six chlorinated solvents Occupational and Environmental Medicine,
68(6), 391-399

Gold	CC14	exposed workers	cancer	1-4 yrs-Cancer

699241

Domain

Metric

Rating^ MWF* Score

Comments^

MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study
§ Evaluator's explanation for rating change: "The number of cases in this subgroup is small (n=4) and caution should be taken when interpreting the findings."



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 5: Roberts et al. 2013: Evaluation of Neurological/Behavior Outcomes

Study Citation:

Data Type:
HERO ID:

Roberts, A.L., Lyall, K., Hart, J.E., Laden, F., Just, A.C., Bobb, J.F., Koenen, K.C., Ascherio, A., Weisskopf, M.G. (2013). Perinatal
air pollutant exposures and autism spectrum disorder in the children of Nurses' Health Study II participants Environmental Health
Perspectives, 121(8), 978-984

Nurses' Health Study II CC14 case-control Autism endpoint-Neurological/Behavior

1790951

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

High X 0.4 0.4 Data from the Nurses' Health Study II was used.

Study reported time frame in which all children
(cases and controls) were selected (2005-2008). Chil-
dren were born in all 50 US states. Exclu-
sion/inclusion criteria is described in the study.

High X 0.4 0.4 The number of cases/controls included in the study

was 329 cases, 22098 controls. Reasons for excluding
subjects were clearly detailed. There was minimal
loss of subjects reported in results (325 cases/22101
controls)

High X 0.2 0.2 Table 1 shows the demographic characteristics of

the cases and controls, which appear to be similar.
These include maternal age, year of birth, sex, state
of residence, smoking, income, and education infor-
mation. These were also considered in the analysis.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low X 0.4 1.2 Exposure was determined based on the location of

the mothers beginning in 1989. Children born from
1987-1990 were assigned the geographic location
of their mothers in 1989. The nurses address was
updated every other year after that and children
were assigned based on the closest date. "Hazardous
air pollutant (HAP) concentrations were assessed
by the U.S. EPA National Air Toxics Assessments in
1990, 1996, 1999, and 2002, which uses an inventory
of outdoor sources of air pollution, including
both stationary sources (e.g., waste incinerators,
small businesses) and mobile sources (e.g., traffic)
to estimate average ambient concentrations of
pollutants for each census tract based on dispersion
models (U.S. EPA 2011)."

The erratum states that the authors did not
use background exposures when determining the
quinitles in 1996, so the quintiles are somewhat
different than as reported.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Roberts, A.L., Lyall, K., Hart, J.E., Laden, F., Just, A.C., Bobb, J.F., Koenen, K.C., Ascherio, A., Weisskopf, M.G. (2013). Perinatal
air pollutant exposures and autism spectrum disorder in the children of Nurses' Health Study II participants Environmental Health
Perspectives, 121(8), 978-984

Nurses' Health Study II	CC14	case-control	Autism endpoint-Neurological/Behavior

1790951

Domain Metric

Rating^

MWF*

Score

Comments^

Metric 5: Exposure levels

Medium

x 0.2

0.4

Exposure levels ranged from 0.0006-41.9 ug/m3, and
divided into 5 quintiles. The range is sufficient to
determine a dose-response relationship

Metric 6: Temporality

High

x 0.4

0.4

Exposures were measured during time and place of
birth from 1987-2002, autism spectrum disorder was
first assessed in 2005; therefore, a minimum of 3
years after exposure.

Domain 3: Outcome Assessment









Metric 7: Outcome measurement or characterization

High

x 0.667

0.67

ASD was reported by the mothers via this question







""Have any of your children been

diagnosed with the following diseases?" with autism,

Asperger's syndrome, or other ASD

listed as separate responses." The ASD diagnoses

were validated by telephone administration of the

Autism Diagnostic Interview-Revised (ADI-R), to a

randomly selected group of 50 monthers from the

study.

Metric 8: Reporting Bias

High

x 0.333

0.33

All measured outcomes were outlined in the meth-





ods, and information could be fulling extracted for
analysis. Some information was provided in supple-
mental information.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

Metric 11: Co-exposure Confounding

High	X 0.5	0.5 Covariates were included in the models, including:

socioeconomic indicators, smoking,
year of birth, maternal age at birth, and air pollution
prediction model year.

Medium X 0.25 0.5 Confounders were assessed via questionnaires, but

there is no indication that the questionnaires were
validated

Medium X 0.25 0.5 Co-exposure analysis was included in the model: "To

investigate further whether one or
two pollutants were driving the association between
correlated pollutants and ASD, we
conducted analyses with diesel, lead, manganese,
cadmium, methylene chloride, and
nickel—the pollutants most strongly associated with
ASD based on tests of highest versus lowest quintile
as well as linear trend—in a single model."



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

Domain 5: Analysis

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Nurses' Health Study II CC14 case-control Autism endpoint-Neurological/Behavior

1790951

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Metric 15: Statistical models

Medium x 0.4 0.8

Medium x 0.2 0.4

The case-control study design was appropriate for
assessing the possible association between autism
spectrum disorder and exposure to several different
compounds. The study design can get at prior ex-
posure to several exposures at once for a specific
outcome from a large cohort.

The power was sufficient to detect effects (325 cases
and 22101 controls).

The methodology is clearly laid out, and could be re-
produced. Methods to calculate the odds ratios and
the covariates included were provided, and details
were provided on when they were not included.
Statistical methods were appropriate (calculation
of ORs, logistic regression models). Linear dose-
response was determined by dividing exposures into
quintiles and using logistic regression with concen-
trations entered as a continuous independent vari-
able. Other analysis such as sex, correlation of heavy
metals, and covariate analysis were employed.

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16

Use of Biomarker of Exposure

Metric 17

Effect biomarker

Metric 18

Method Sensitivity

Metric 19

Biomarker stability

Metric 20

Sample contamination

Metric 21

Method requirements

Metric 22

Matrix adjustment

Overall Quality Determination1"

High

1.5

Extracted

Yes

MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; x MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

^ This metric met the criteria for high confidence as expected for this type of study

z
o

O
»
o

-------
Table 6: Goldman et al. 2012: Evaluation of Neurological/Behavior Outcomes

Study Citation:

Data Type:
HERO ID:

Goldman, S.M., Quinlan, P.I., Ross, G.W., Marras, C., Meng, C., Bhudhikanok, G.S., Comyns, K., Korell, M., Chade, A.R., Kasten,
M., Priestley, B., Chou, K.L., Fernandez, H.H., Cambi, F., Langston, J.W., Tanner, C.M. (2012). Solvent exposures and Parkinson
disease risk in twins Annals of Neurology, 71(6), 776-784

WW2 Twins CC14 Parkinson's dichotomous pairwise OR-Neurological/Behavior
2127988

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

High

Metric 2: Attrition

Metric 3: Comparison Group

Medium x 0.4

High

X 0.4 0.4 Key elements of study are reported: participants

were selected from the National Academy of Sci-
ences/National Research Council WWII Veteran
Twins Registry, an all-male twin cohort. Cases were
selected through telephone screening of the entire
reachable cohort; concurrently, searches of VA med-
ical databases, the Health Care Financing Adminis-
tration, and the National Death Index were under-
taken to identify other cases. It was stated that age
at PD diagnosis or interview was similar between
those pairs that completed the interview and those
pairs that did not complete the interview. As such,
the reported information indicates selection in or out
of the study and participation is not likely to be bi-
ased.

0.8 Occupational histories were completed by 63.6% of
twins with PD and 60.1% of twins without PD lead-
ing to a final total of 99 twin pairs. This is moderate
exclusion from the analysis sample. Rates of com-
pletion were similar between twins with and without
PD.

x 0.2 0.2 In both paired and unpaired analysis, smoking was

an included covariate. In unpaired analysis, an age
index was also adjusted for. Other important de-
mographic factors in the paired analysis would be
highly controlled as the analysis was of twin pairs.
The type of twin (monozygotic or dizygotic) was also
included as a covariate in the paired analysis.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low X 0.4 1.2 This method relies on self-reported occupational his-

tories. There may be some misclassification due re-
call bias in addition to any bias introduced by accu-
racy of response for participant proxies.

z
o

O
»
o

Continued on next page

-------
.. . continued from previous page

Study Citation: Goldman, S.M., Quinlan, P.I., Ross, G.W., Marras, C., Meng, C., Bhudhikanok, G.S., Comyns, K., Korell, M., Chade, A.R., Kasten,
M., Priestley, B., Chou, K.L., Fernandez, H.H., Cambi, F., Langston, J.W., Tanner, C.M. (2012). Solvent exposures and Parkinson
disease risk in twins Annals of Neurology, 71(6), 776-784
Data Type: WW2 Twins CC14 Parkinson's dichotomous pairwise OR-Neurological/Behavior

HERO ID: 2127988

Domain Metric

Rating^

MWF*

Score

Comments^

Metric 5: Exposure levels

Medium

0.2

0.4

For logistic regression using duration of exposure or
cumulative exposure indices, ORs addressed risk as-
sociated with a one tertile change in the respective
marker of exposure. This represents three or more
levels of exposure.

For the Ever/Never analysis, only two levels of ex-
posure are used. Ever exposure was defined as ex-
posure to a solvent for at least 2% of work time or
1 hour per week.

Metric 6: Temporality

High

0.4

This study investigated occupational exposures be-
ginning at a young age and their association with
Parkinson's Disorder later in life. The interval be-
tween exposure and outcome measurement is appro-
priate to measure this association.

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization

High

0.667

0.67

Cases were identified through searches of records
in the Department of Veteran's Affairs, the Health
Care Financing Administration, and the National
Death Index. Participants suspected of having
Parkinson's underwent in-person examination with a
trained movement disorder specialist. This outcome
assessment represents a well-established method.
Both neurologists followed standard criteria for PD
diagnosis and made their diagnosis by video. There
is no mention of blinding during this evaluation., al-
though participants were unaware of study hypothe-

Metric 8: Reporting Bias

High

0.333

0.33

All outcomes mentioned in the abstract, introduc-
tion, and methods were presented clearly in the re-
sults. ORs are contained in easily extractable tables,
including number of participants used in each anal-
ysis accompanied by summary measures of exposure
in the analyses of cumulative exposure.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

High

0.5

In the paired analysis (paired twins), the conditional
logistic regression model included terms for respon-
dent type (monozygotic/dizygotic) and smoking. In
the unpaired analysis, respondent type, smoking,
and age were all included in the analysis. Models
including head injury were stated to be similar to
the results shown.

Continued on next page . ..

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

WW2 Twins CC14 Parkinson's dichotomous pairwise OR-Neurological/Behavior
2127988

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 10: Covariate Characterization

Medium x 0.25

Metric 11: Co-exposure Confounding

Medium x 0.25

0.5 In some cases, questionnaires/surveys were com-
pleted by proxies such as a spouse or sibling. For
several covariates including head injury or smoking,
this is not a well-established method, but there was
little evidence that the method had poor validity It
should also be noted that results were presented for
an analysis excluding twin pairs using proxy respon-
dents. The results of this analysis were in agreement
with the main analyses.

0.5 Co -exposures to other solvents was measured in this
study. Overall, six different solvents were included
in the exposure analysis: TCE, PERC, CC14, n-
hexane, toluene, and xylene. Several analysis strate-
gies were presented to elucidate any effects of co-
exposures. Analyses were done for the relationship
between PD and exposure to TCE or PERC as well
as an analysis of the relationship between exposure
to any of the 4 solvents, excluding TCE and PERC.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Metric 15: Statistical models

Medium x 0.4 0.8

Medium x 0.2 0.4

The retrospective study design is appropriate to in-
vestigate long-term or chronic exposure to industrial
solvents and development of the neurodegenerative
Parkinson's Disease. Appropriate statistical meth-
ods (i.e., conditional logistical modeling) were em-
ployed to analyze the matched data.

There is an adequate number of discordant twin
pairs (n=99) for the pairwise analysis and an ad-
equate number of participants in the unpaired anal-
ysis (n=126 cases exposed, n=110 controls exposed)
to detect an effect in the exposed population.

The description of the analysis is sufficient to repro-
duce the results if given original data. No apparent
issues.

The method (logistic regression modeling) of calcu-
lating risk is transparent and appropriate. Rationale
for variable selection is stated. Model assumptions
do not appear to be violated.

Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16: Use of Biomarker of Exposure
Metric 17: Effect biomarker

NA
NA

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Goldman, S.M., Quinlan, P.I., Ross, G.W., Marras, C., Meng, C., Bhudhikanok, G.S., Comyns, K., Korell, M., Chade, A.R., Kasten,
M., Priestley, B., Chou, K.L., Fernandez, H.H., Cambi, F., Langston, J.W., Tanner, C.M. (2012). Solvent exposures and Parkinson
disease risk in twins Annals of Neurology, 71(6), 776-784

WW2 Twins CC14 Parkinson's dichotomous pairwise OR-Neurological/Behavior
2127988

Domain



Metric

Ratingt MWF*

Score

Comments^



Metric 18:

Method Sensitivity

NA

NA





Metric 19:

Biomarker stability

NA

NA





Metric 20:

Sample contamination

NA

NA





Metric 21:

Method requirements

NA

NA





Metric 22:

Matrix adjustment

NA

NA



Overall Quality Determination1"

High

1.6

Extracted

Yes





MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 7: Neta et al. 2012: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Neta, G., Stewart, P.A., Rajaraman, P., Hein, M.J., Waters, M.A., Purdue, M.P., Samanic, C., Coble, J.B., Linet, M.S. (2012).
Occupational exposure to chlorinated solvents and risks of glioma and meningioma in adults Occupational and Environmental Medicine,
69(11), 793-801

CC14	all	subjects	possibleexp	Glioma-Cancer

2128240

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

Higll	X 0.4	0.4 High rating: key elements of study design were re-

ported, and the reported information
indicates selection in or out of the study and partic-
ipation is not likely to be biased.

High	X 0.4	0.4 High participation rates: 92% and 94% for glioma

and meningioma cases, respectively. Participation
rate among controls was 86%

High	X 0.2	0.2 High rating: cases and controls were similar - con-

trols were patients admitted to the same hospitals as
cases for non-malignant conditions with frequency
matching by sex, age, race/ethnicity, hospital, and
proximity to hospital; differences in baseline
characteristics of groups were considered as poten-
tial confounding or stratification

variables (i.e,. sex and 5-year age groups) and were

thereby controlled by statistical

analysis

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Metric 5: Exposure levels

Metric 6: Temporality

Low	X 0.4	1.2 Low rating: Occupational study population with ex-

posure assessed using in person interviews (i.e., no
employment records were utilized). Industrial hy-
giene experts from examined data collected in the
questionnaires, and assessed a level of probability
and levels of exposure to groups or classes of sol-
vents as well as certain individual substances.
Medium X 0.2	0.4 Medium rating: range and distribution of exposure

was sufficient to develop an exposure response esti-
mate; 3 or more levels of exposure were reported
High	X 0.4	0.4 High rating: temporality is established and the in-

terval between reconstructed exposure
and brain tumor risk has an appropriate considera-
tion of relevant exposure windows.

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization High

x 0.667 0.67



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

High rating: ICD-Oncology codes listed; all partici-
pating case diagnoses were confirmed by microscopy

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Neta, G., Stewart, P.A., Rajaraman, P., Hein, M.J., Waters, M.A., Purdue, M.P., Samanic, C., Coble, J.B., Linet, M.S. (2012).
Occupational exposure to chlorinated solvents and risks of glioma and meningioma in adults Occupational and Environmental Medicine,
69(11), 793-801

CC14	all	subjects	possibleexp	Glioma-Cancer

2128240

Domain





Metric

Rating^

MWF* Score

Comments^



Metric 8:

Reporting Bias



High

x 0.333 0.33

High rating: all of the study's measured outcomes
are reported, effect estimates reported
with confidence interval; number of exposed re-
ported for each analysis.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

High

x 0.5

Metric 10: Covariate Characterization

Metric 11: Co-exposure Confounding

Medium x 0.25

Medium x 0.25

0.5 High rating: appropriate adjustments or explicit
considerations were made for potential
confounders in the final analyses through the use of
statistical models for covariate

adjustment (i.e., age group (<30, 30—49, 50—69,
70+), race (white vs non-white), sex, hospital site
and proximity of residence to the hospital)

0.5 Medium rating: primary confounders (excluding co-
exposures) were assessed. The paper
did not describe if the computer-based questionnaire
used to collect demographic information has been
previously validated.

0.5 Medium rating: potential co-pollutant confounding
was considered through the adjustment in statistical
models, of estimated cumulative occupational expo-
sures to lead, magnetic fields, herbicides and insecti-
cides. In addition, for ever/never analyses for partic-
ular solvents, the authors included all other solvents
in the model to account

for possible confounding by other solvent exposures.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Medium x 0.4 0.8

Medium x 0.2 0.4

Medium rating: appropriate design (i.e., case control
study of chemical exposures in relation to a rare dis-
ease), and appropriate statistical methods (i.e., lo-
gistic regression analyses) were employed to analyze
data.

Medium rating: the number of cases and controls are
adequate to detect an effect in the exposed popula-
tion for the primary analyses of probable/possible
solvent exposure vs. unexposed in relation to risk
of glioma. The number of exposure cases of menin-
gioma was too small to have the power to conduct
stratified analyses or analyses of more detailed ex-
posure metrics.



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

Continued on next page

-------
. continued from previous page

N>

Study Citation:

Data Type:
HERO ID:

Neta, G., Stewart, P.A., Rajaraman, P., Hein, M.J., Waters, M.A., Purdue, M.P., Samanic, C., Coble, J.B., Linet, M.S. (2012).
Occupational exposure to chlorinated solvents and risks of glioma and meningioma in adults Occupational and Environmental Medicine,
69(11), 793-801

CC14	all	subjects	possibleexp	Glioma-Cancer

2128240

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 14

Reproducibility of analyses

Medium

x 0.2

0.4

Medium rating: description of the analyses is suffi-











cient to understand what has been











done and to be reproducible with access to the data.

Metric 15

Statistical models

Medium

x 0.2

0.4

Medium rating: logistic regression models were used











to generate Odds Ratios. Rationale











for variable selection is stated. Model assumptions











are met

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

High

1.5

Extracted

Yes





MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 8: Ruder et al. 2013: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Ruder, A.M., Yiin, J.H., Waters, M.A., Carreon, T., Hein, M.J., Butler, M.A., Calvert, G.M., Davis-King, K.E., Schulte, P.A., Mandel,
J.S., Morton, R.F., Reding, D.J., Rosenman, K.D., Stewart, P.A., Brain Cancer Collaborative Study Group (2013). The Upper Midwest
Health Study: Gliomas and occupational exposure to chlorinated solvents Occupational and Environmental Medicine, 70(2), 73-80

Upper Midwest Health Study CC14 cumulative include proxy glioma-Cancer

2128307

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

High X 0.4 0.4 Subjects were selected from the same area during

the same time frame. Cases were identified through
participating medical facilities and neurosurgeon of-
fices. Controls were identified from state driver's
license records.91.5% o f cases or their next of kin
participated and 70.4% of controls participated. Key
elements of the study design are reported..

High X 0.4 0.4 Study population consisted of 1175 controls and 798

cases. 97& of the controls (1141/1175) were inter-
viewed and all cases had interviews with 360 being
proxy interviews. Some analysis was restricted to
cases that were directly interviewed.

High X 0.2 0.2 Controls were randomly selected and age and sex

stratified. There were some differences in the level
of education, but this was adjusted for in the analy-
sis. Details comparing cases and controls as well as
ineligible and non-participants are detailed in com-
panion publication (Ruder et al. 2006).

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Medium x 0.4

0.8 Complete occupational history was obtained using
a questionnaire modified from the one developed by
the National Cancer Institute. Jobs of at least one
years duration between the age of 16 and the end of
1992 were included. The questionnaire also asked
about specific exposures including solvent and on
which jobs and for how many hours a week these
exposures occurred. There is potential for cases to
have better recall. The probability, intensity, and
frequency of exposure in non-farm related jobs was
estimated based on occupation, industry, and decade
using an annotated appendix of sources of exposure
data as well as bibliographic databases of published
exposure levels. Complete descriptions of the meth-
ods were provided. JEM with complete job history,
but based on recalled jobs and some judgement on
exposure (although used several cited references).

z
o

O
»
o

Continued on next page

-------
.. . continued from previous page

Study Citation:

Data Type:
HERO ID:

Upper Midwest Health Study CC14 cumulative include proxy glioma-Cancer

2128307

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 5:

Exposure levels

Medium

x 0.2

0.4

Exposure was estimated in cumulative exposure of
ppm-h and ppm-years.

Metric 6:

Temporality

Medium

x 0.4

0.8

Temporality is established, but it is unclear whether
exposures fall within relevant exposure windows for
the outcome of interest. Case diagnosis occurred be-
tween 1995 and 1997 with job history ending in 1992.

Domain 3: Outcome Assessment

Metric 7:

Outcome measurement or characterization

High

x 0.667

0.67

The study focused on histologically confirmed pri-

mary intracranial gliomas (ICD-O code 938-948).

Metric 8:

Reporting Bias

High

x 0.333

0.33

Sufficient information was reported. Effect esti-
mates are reported with a confidence interval.

Domain 4: Potential Counfounding/Variable Control

Metric 9:

Covariate Adjustment

Medium

x 0.5

Adjusted for age group, sex, age, and education.

Metric 10:

Covariate Characterization

Medium

x 0.25

0.5

Information was obtained via a questionnaire some-
times via proxy.

Metric 11:

Co-exposure Confounding

Medium

x 0.25

0.5

Although this was occupational exposure, they in-
cluded people from different jobs at different times
and it is unlikely that there would be differential
co-exposures.

Domain 5: Analysis

Metric 12:

Study Design and Methods

Medium

x 0.4

0.8

Methods are appropriate and appropriate statistical
methods were used to address research question.

Metric 13:

Statistical power

Medium

x 0.2

0.4

The study included 798 cases and 1175 controls,
which is likely to provide sufficient statistical power.
For any given exposure there were more than 100
subjects except when evaluating women only or a
subset excluding proxy only. In these cases there
were as few as 34 subjects.

Metric 14:

Reproducibility of analyses

Medium

x 0.2

0.4

Enough information is provided to be reproducible
if data were available.

Metric 15:

Statistical models

Medium

x 0.2

0.4

Unconditional logistic regression models were used,
which were appropriate for the data and assump-
tions appear to have been met.

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16:

Use of Biomarker of Exposure

Metric 17:

Effect biomarker

Metric 18:

Method Sensitivity

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Ruder, A.M., Yiin, J.H., Waters, M.A., Carreon, T., Hein, M.J., Butler, M.A., Calvert, G.M., Davis-King, K.E., Schulte, P.A., Mandel,
J.S., Morton, R.F., Reding, D.J., Rosenman, K.D., Stewart, P.A., Brain Cancer Collaborative Study Group (2013). The Upper Midwest
Health Study: Gliomas and occupational exposure to chlorinated solvents Occupational and Environmental Medicine, 70(2), 73-80

Upper Midwest Health Study	CC14	cumulative	include proxy	glioma-Cancer

2128307

Domain



Metric

Ratingt MWF*

Score

Comments^



Metric 19:

Biomarker stability

NA

NA





Metric 20:

Sample contamination

NA

NA





Metric 21:

Method requirements

NA

NA





Metric 22:

Matrix adjustment

NA

NA



Overall Quality Determination1"

High

1.6

Extracted

Yes

MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

U This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 9: Vizcaya et al. 2013: Evaluation of Cancer Outcomes

Study Citation: Vizcaya, D; Christensen, KY; Lavoue, J; Siemiatycki, J (2013). Risk of lung cancer associated with six types of chlorinated solvents:

Results from two case-control studies in Montreal, Canada Occupational and Environmental Medicine, 70(2), 81-85
Data Type: occupational case-control study Montreal (CC14 any exposure Study II analysis extraction)-Cancer

HERO ID: 2128435

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Medium x 0.4

Metric 2: Attrition

Metric 3: Comparison Group

Low

High

x 0.4

x 0.2

0.8 This was a population based case-control study in
which subjects were restricted to Canadian citizens
who were residents in the Montreal metropolitan
area. This report did not describe case ascertain-
ment, but cited references (HERO ID 2856585 and
091275) which indicate that histologically confirmed
cancer patients from 18 of the largest hospitals were
used as cases. Controls were randomly selected fre-
quency matched by age and sex. Participation rates
were provided and were slightly higher in the cases.

1.2 There appears to be a large amount of attrition that
was not adequately explained. It is likely that the
missing subjects from Table 1 did not have occupa-
tions with exposure codes.

0.2 Cases were more likely to be French Canadians than
controls. Controls were on average wealthier and
had a higher education. Cases were heavier smokers
than controls. These were all controlled for in the
analysis.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Metric 5: Exposure levels

Metric 6: Temporality

Low x 0.4 1.2 A semi-structured questionnaire was used to obtain

details of each job that lasted at least 6 months. A
team of industrial chemists and hygienists examined
each subject's questionnaire and translated each job
into potential exposures from a list of 294 substances
without knowledge of the subject's status. Exposure
based on collective judgement.

Medium x 0.2 0.4 Only two groups were compared and could not be

evaluated for trend. Exposed groups were never ex-
posed, ever exposed, or substantial exposure.

Low X 0.4 1.2 The temporality of exposure and outcome is uncer-

tain. Although job history was obtained, there is no
information provided to determine that the jobs oc-
curred before diagnosis or even if the jobs were prior
to diagnosis there is no information provided on how
long or how close to the diagnosis the jobs occurred.

z
o

O
»
o

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

Study Citation: Vizcaya, D; Christensen, KY; Lavoue, J; Siemiatycki, J (2013). Risk of lung cancer associated with six types of chlorinated solvents:

HERO ID: 2128435

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 7: Outcome measurement or characterization High x 0.667 0.67
Metric 8: Reporting Bias High x 0.333 0.33

Cases were histologically confirmed.

Results were reported in sufficient details. A de-
scription of measured outcomes is reported in the
methods, abstract, and/or introduction. Effect es-
timates are reported with a confidence interval and
the number of cases/controls are reported for each
analysis.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

High
Medium

x 0.5
x 0.25

Metric 11: Co-exposure Confounding

Medium x 0.25

0.5 Results were adjusted by age, smoking habit, edu-
cational attainment, SES, and ethnicity.

0.5 Information was obtained from a questionnaire of
unknown reliability and validity. The authors note
that "Although it is very difficult to establish the va-
lidity of retrospective exposure assessments, we have
demonstrated satisfactory levels of reliability and va-
lidity in the job histories and in the expert exposure
assessments.'

0.5 It was noted that results were adjusted for exposure
to eight known carcinogens. Although there are po-
tential co-exposures for any given job, it is unlikely
that they were differential across jobs and within the
specific chemicals of interest. Supplemental Table
S2 indicated 5 different jobs with exposure to CC14
making it unlikely that co-exposure was consistent
across all 5 jobs in each category.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Medium x 0.4

Medium
Medium

x 0.2

0.8 Study design and statistical method were appropri-
ate for the research question. A case-control study
is the best design to study lung cancers when evalu-
ating many different possible exposures across mul-
tiple different jobs. The use of unconditional logistic
regression is appropriate for this data.

0.4 Statistical power should be sufficient. However,
some substantial exposure categories had a small
number of subjects.

0.4 The description of the unconditional logistic regres-
sion analysis used for estimates of odds ratios and
the confounders included is sufficient to understand
precisely what has been done and to be conceptually
reproducible with access to the analytic data.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation: Vizcaya, D; Christensen, KY; Lavoue, J; Siemiatycki, J (2013). Risk of lung cancer associated with six types of chlorinated solvents:

Results from two case-control studies in Montreal, Canada Occupational and Environmental Medicine, 70(2), 81-85
Data Type:	occupational case-control study Montreal (CC14 any exposure Study II analysis extraction)-Cancer

HERO ID:	2128435

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 15

Statistical models

Medium

x 0.2

0.4

The method for calculating the risk estimates (i.e.
odds ratios) is transparent and the model assump-
tions were met.

Domain 6: Other Considera
Metric 16
Metric 17
Metric 18
Metric 19
Metric 20
Metric 21
Metric 22

ions for Biomarker Selection and Measurement
Use of Biomarker of Exposure
Effect biomarker
Method Sensitivity
Biomarker stability
Sample contamination
Method requirements
Matrix adjustment



NA
NA
NA
NA
NA
NA
NA

NA
NA
NA
NA
NA
NA
NA



Overall Quality Determination1"

Medium



1.9



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 10: Morales-Suarez-Varela et al. 2013: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Morales-Suarez-Varela, MM; Olsen, J; Villeneuve, S; Johansen, P; Kaerlev, L; Llopis-Gonzalez, A; Wingren, G; Hardell, L; Ahrens, W;
Stang, A; Merletti, F; Gorini, G; Aurrekoetxea, J J; Fevotte, J; Cyr, D; Guenel, P (2013). Occupational exposure to chlorinated and
petroleum solvents and mycosis fungoides Journal of Occupational and Environmental Medicine, 55(8), 924-931

Case-Control Occupational CC14 MycosisFungoides OR_aboveMedian All-Cancer

2129849

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

High x 0.4 0.4 140 cases ascertained from requests to hospitals and

pathology department, as well as regional/national
cancer and pathology registers. Patients from 6 Eu-
ropean countries: Denmark, Sweden, France, Ger-
many, Italy, and Spain. Controls from these coun-
tries selected from population registries or colon can-
cer registries. As such, the reported information in-
dicates selection in or out of the study and partici-
pation is not likely to be biased.

Medium X 0.4 0.8 Moderate attrition due to patents removed from

study due to unconfirmed diagnosis (22) or lack of
availability for interview (18); participation rate of
84.75%. Of the eligible controls, 68.2% (3156) were
interviewed; only controls within the strata (5 year
age + gender) of MF patients used (2846).

High X 0.2 0.2 Key elements of the study design are reported indi-

cate that that cases and controls were similar (e.g.,
recruited from the same eligible population with the
number of controls described, and eligibility crite-
ria and are recruited within the same time frame.
Specifically, 4 controls/case, frequency matched by
sex and age (5 years). Population registries and elec-
toral rolls used to select controls in Denmark, Swe-
den, France, Germany and Italy. Spanish controls
from colon cancer patients (no population register).

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Metric 5: Exposure levels

Low X 0.4 1.2 Interviews with standardized questionnaires to de-

termine occupational history. Next of kin completed
interviews for 4 cases and 95 controls. Exposure de-
termined with JEM developed by the French Insti-
tute of Health Surveillance using jobs/industries as-
signed based on interviews by trained coders using
international standards.

Low X 0.2 0.6 Reports only 2 levels of exposure for CC14 (ex-

posed/ unexposed)

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Case-Control Occupational CC14 MycosisFungoides OR_aboveMedian All-Cancer

2129849

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 6: Temporality

Higll X 0.4 0.4 Temporality is established and the interval between

the exposure (or reconstructed exposure) and the
outcome has an appropriate consideration of rele-
vant exposure windows. Specifically, the authors
considered lag times of 5, 10, or 15 years, which did
not make an impact (results not presented).

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization High

Metric 8: Reporting Bias High

X 0.667 0.67 Clinical and pathological mycosis fungoides (MF)
diagnosis from cancer/pathology registers and re-
quests of hospitals, using ICD codes. All diagnosis
were reviewed by the same pathologist for adherence
to morphological and topographical MF criteria; 22
cases were excluded on this basis,
x 0.333 0.33 The results discussed in the introduction/methods
were fully provided and extractable. All of the
study's measured outcomes are reported, effect es-
timates reported with confidence interval; number
of cases and controls reported for each analysis.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

High

x 0.5

Metric 11: Co-exposure Confounding

Medium x 0.25

0.5 Confounders considered in adjusted analy-
sis: age, sex, country, current smoking habit
(cigarettes/day), alcohol intake, BMI, and educa-
tion level.

0.5 Primary confounders were assessed using a less-
established method with no reporting of validation
against well-established methods. Specifically, co-
variates were determined from interviews. Next of
kin completed interviews for 4 cases and 95 controls.

0.5 Co-exposures were not accounted for in this analysis,
but no direct evidence that co-exposures differ across
cases and controls.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Medium x 0.4 0.8

Medium x 0.2 0.4

Case-control design was appropriate for investigat-
ing chlorinated solvents and a rare disease such as
MF, and appropriate statistical methods (logistic re-
gression) were employed to analyze data.

100 cases and 2846 controls. Exposed cases rela-
tively low (27 trichloroethylene, 6 perchloroethylene,
9 methylene chloride), but sufficient to detect an ef-
fect.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Morales-Suarez-Varela, MM; Olsen, J; Villeneuve, S; Johansen, P; Kaerlev, L; Llopis-Gonzalez, A; Wingren, G; Hardell, L; Ahrens, W;
Stang, A; Merletti, F; Gorini, G; Aurrekoetxea, J J; Fevotte, J; Cyr, D; Guenel, P (2013). Occupational exposure to chlorinated and
petroleum solvents and mycosis fungoides Journal of Occupational and Environmental Medicine, 55(8), 924-931

Case-Control	Occupational	CC14	MycosisFungoides	OR_aboveMedian	All-Cancer

2129849

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 14

Reproducibility of analyses

Medium

x 0.2

0.4

Description of the analyses is sufficient to under-
stand what has been done and to be reproducible
with access to the data.

Metric 15

Statistical models

Medium

x 0.2

0.4

The model used for calculating risk estimate (i.e.,
odds ratios using logistic regression) is fully appro-
priate. Rationale for covariate selection is not pro-
vided, but model assumptions do not appear to be
violated.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

High



1.6



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 11: Heck et al. 2013: Evaluation of Cancer Outcomes

Study Citation: Heck, JE; Park, AS; Qiu, J; Cockburn, M; Ritz, B (2013). An exploratory study of ambient air toxics exposure in pregnancy and the

risk of neuroblastoma in offspring Environmental Research, 127 1-6
Data Type: Case-Control_Children_CC14_Neuroblastoma_OR_IQR_2_5km-Cancer

HERO ID: 2225094

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

High
Low

x 0.4
x 0.4

Metric 3: Comparison Group

Medium x 0.2

0.4 Authors included all cases of neuroblastoma listed
in the California Cancer Registry (1990-2007).

1.2 The study attained a 89% matching rate to Califor-
nia birth certificate (probabilistic linkage program
(LinkPlus, Atlanta, GA) and included up to 75 cases
and 14,602 controls (depending on the air toxic eval-
uated as exposure), who lived within 5 km of an air
toxics monitor. According to the authors, excluded
children (781 cases and 146,763 controls) were more
likely to live in a rural county (20% vs. 4%), to
have a mother who was White non-Hispanic (35%
vs. 26%) and to be born in the US(56% vs. 50%).

0.4 Controls randomly selected from California birth
records (no cancer diagnosis before age 6), frequency
matched by year of birth; excluded children who had
died of other causes prior to age 6. Large number
excluded due to missing information on length of
gestation. In general, demographic characteristics
of cases and controls were similar but there were
some differences, for example, in ethnicity (e.g. 40%
cases were White non-Hispanic vs 26.1% controls)
and neighborhood socio-economic index (e.g. 18.7%
of cases vs 29.2% of controls in lowest level).

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low

x 0.4

Metric 5: Exposure levels

Medium x 0.2

1.2 Exposure based on data from community-based air
pollution monitors for participants living within 5
km of an air pollution monitor. For participants
born in the period 1998-2007, geocoding based on
exact home address, but for those born in 1990-1997,
geocoding based on zipcode (potential for exposure
misclassification). Additional potential source of
bias due to assumption that birth certificate address
was consistent throughout the pregnancy.

0.4 Exposure-response estimate obtained for several air
toxics, including CC14, Perc and TCE, for interquar-
tile range and in some cases for across quartiles, con-
sidering different buffer sizes (5km, 4km, 3km, 2.5
km) around air toxics' monitors.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation: Heck, JE; Park, AS; Qiu, J; Cockburn, M; Ritz, B (2013). An exploratory study of ambient air toxics exposure in pregnancy and the
risk of neuroblastoma in offspring Environmental Research, 127 1-6

Data Type: Case-Control Children CC14 Neuroblastoma OR IQR 2 5km-Cancer

HERO ID: 2225094

Domain

Metric

Rating^

MWF* Score

Comments^

Metric 6:

Temporality

High

x 0.4 0.4

Exposure assessed for full extent of pregnancy and
for each trimester. Neuroblastoma has a high inci-
dence in infants, so assessing though 6 years old is
appropriate.

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization

Metric 8: Reporting Bias

Medium x 0.667 1.33

Medium x 0.333 0.67

Outcome assessed using International Classification
of Childhood Cancer, version3 (ICCC-3) code 041 as
reported in the California Cancer Registry, but di-
agnosis was not confirmed. It is not clear if absence
of cancer diagnosis in controls was confirmed.

For CC14, both OR for IQR at different buffer sizes
(2.5km, 3km, 4km, and 5km) and for each quar-
tile (vs. 1st quartile) are reported; however, when
reporting results for each quartile it is not clearly
stated whether or not these are for the 5km buffer
size. For Perc and TCE, OR per interquartile in-
crease reported only for two buffer sizes (2.5km and
5 km) and results for each quartile are not reported.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Medium x 0.5

Metric 10: Covariate Characterization

Medium x 0.25

0.5

Selection of potential confounders was based on
literature review and relationship in sample be-
tween demographic and perinatal factors and out-
come. Several relevant covariates were considered
and retained in final analysis [mother's age, mother's
race/ethnicity, birth year, socioeconomic indicator
(method of payment for prenatal care)]. However,
other potential confounders noted as relevant by
the authors in the Introduction section (e.g. birth-
weight, maternal and paternal alcohol intake and
smoking status, paternal occupational exposures)
were not evaluated.

Demographic and socio-economic data obtained
from birth certificates (mother's age, mother's
race/ethnicity, birth year) and US Census data
(socio-economic data). SES was assessed through
both insurance type and census tract data.

Continued on next page

-------
. continued from previous page

Data Type: Case-Control Children CC14 Neuroblastoma OR IQR 2 5km-Cancer

HERO ID: 2225094

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 11: Co-exposure Confounding

Medium x 0.25

0.5 Co -exposures to pollutants were measured but not
adjusted for in the regression models. Authors state
that, according to cited study (Heck et al., in press),
they found that Perc was highly correlated with
traffic-related toxics, while other air toxics "were not
as strongly correlated with each other." No differ-
ences expected between exposure groups.

Domain 5: Analysis

Metric 12: Study Design and Methods

Medium x 0.4

Metric 13: Statistical power

Medium x 0.2

Metric 14: Reproducibility of analyses

Metric 15: Statistical models

Medium x 0.2

0.8 A case-control study design was used to evaluate the
relationship between prenatal exposure to air toxics
(CC14, PERC, TCE) and neuroblastoma (childhood
cancer). Logistic regression was used to determine
OR for IQR of increase in exposure to each air toxic
and, for CC14, the OR for each quartile relative to
the lowest quartile of exposure was also evaluated.

0.4 Statistically significant effects were determined for
some air toxics using each respective sample size,
but no statistical power was reported. For CC14,
the analysis included 40 cases and 7443 controls, for
Perc 67 cases and 12041 controls were included and
for TCE 67 cases and 12086 controls were included,
for a 5km radius around air pollution monitors.

0.4 Detailed description of statistical analysis provided.

The covariates adjusted for in the logistic regres-
sion explicitly stated for each model. Number of
cases/controls used in each analysis presented for
5km and 2.5 km radii.

0.4 Logistic regression appropriately used to determine
ORs. Study presents models adjusted just for birth
year, or for all confounders that were collected (birth
year, maternal age, maternal race/ethnicity, and
method of payment - SES). Potential confounders
identified from literature and in a previous study
(Heck 2009).

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16

Use of Biomarker of Exposure

Metric 17

Effect biomarker

Metric 18

Method Sensitivity

Metric 19

Biomarker stability

Metric 20

Sample contamination

Metric 21

Method requirements

Continued on next page . ..

z
o

O
»
o

-------
. continued from previous page

Study Citation: Heck, JE; Park, AS; Qiu, J; Cockburn, M; Ritz, B (2013). An exploratory study of ambient air toxics exposure in pregnancy and the
risk of neuroblastoma in offspring Environmental Research, 127 1-6

Data Type:	Case-Control	Children	CC14	Neuroblastoma	OR IQR	2	5km-Cancer

HERO ID:	2225094

Domain Metric

Ratingt MWF*

Score

Comments^

Metric 22: Matrix adjustment

NA

NA



Overall Quality Determination1"

Medium

2.0



Extracted

Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 12: Davis 1934: Evaluation of Hematological and Immune Outcomes

Study Citation: P. A. Davis (1934). Carbon tetrachloride as an industrial hazard Journal of the American Medical Association, 103(13,13), 962-966

Data Type: Davis CC14 controlled inhalation exposure hematology-Hematological and Immune

HERO ID: 3611

Domain

Metric

Ratingt MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Low

Metric 2: Attrition

Metric 3: Comparison Group

Medium
Low

X 0.4 1.2 Eight controlled experiments were conducted in to-

X 0.4 0.8 Subjects differed for all experiments but one. The

reason for this change from experiment to experi-
ment is not fully described.

X 0.2 0.6 No control group was used in this study. The mea-

sured outcomes were presumably compared to refer-
ence values, but the details are not clear.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low

Metric 5: Exposure levels

Metric 6: Temporality

Low

X 0.4 1.2 The inhalation chamber was adequately described.

X 0.2 0.6 Multiple exposure levels were examined in this study

including 76 ppm, 158 ppm, 317 ppm, 1191 ppm,
2300 ppm and additional unreported levels, but ex-
posure duration varied by exposure concentration.

X 0.4 1.2 This study was a controlled inhalation exposure.

The timing of outcome measurement was not fully
described in the text and remains unclear, although
it is presumed that measurements were taken after
controlled exposure to carbon tetrachloride.

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

Study Citation: P. A. Davis (1934). Carbon tetrachloride as an industrial hazard Journal of the American Medical Association, 103(13,13), 962-966

Data Type: Davis CC14 controlled inhalation exposure hematology-Hematological and Immune

HERO ID: 3611

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 7:

Outcome measurement or characterization

Low

x 0.667

Metric 8:

Reporting Bias

Medium

x 0.333

0.67

Outcomes were outlined throughout the paper and
clinical observations were described.

Domain 4: Potential Counfounding/Variable Control

Metric 9:

Covariate Adjustment

Low

x 0.667

A statistical analysis was not conducted. Age of

the test subjects was provided, but no other demo-
graphic information was presented or adjusted for.

Metric 10

Covariate Characterization

Not Rated

Covariates, besides age, were not collected.

Metric 11

Co-exposure Confounding

Medium

x 0.333

0.67

There was no indication of co-exposures being
present or measured for during the controlled inhala-
tion exposure.

Domain 5: Analysis

Metric 12

Study Design and Methods

Medium

x 0.5

This study utilized an inhalation chamber to ex-

Metric 13

Statistical power

Medium

x 0.25

0.5

Three to four subjects were used in each controlled
inhalation experiment. This is a low number of in-
dividuals per experiment and results should be in-
terpreted with caution.

Metric 14

Reproducibility of analyses

Low

x 0.25

0.75

Metric 15

Statistical models

Not Rated

Results were compared to reference values and de-
scribed qualitatively only.

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16

Use of Biomarker of Exposure

Metric 17

Effect biomarker

Metric 18

Method Sensitivity

Metric 19

Biomarker stability

Metric 20

Sample contamination

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:
Data Type:
HERO ID:

P. A. Davis (1934). Carbon tetrachloride as an industrial hazard Journal of the American Medical Association, 103(13,13), 962-966

Davis	CC14	controlled	inhalation	exposure	hematology-Hematological and Immune

3611

Domain

Metric Rating^

MWF*

Score Comments^



Metric 21: Method requirements
Metric 22: Matrix adjustment

NA
NA

NA
NA

Overall Quality Determination1" Low



2.6

Extracted	Yes

MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; x MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

^ This metric met the criteria for high confidence as expected for this type of study

-------
Table 13: Mattei et al. 2014: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Mattei, F; Guida, F; Matrat, M; Cenee, S; Cyr, D; Sanchez, M; Radoi, L; Menvielle, G; Jellouli, F; Carton, M; Bara, S; Marrer, E; Luce,

D; Stiicker, I (2014). Exposure to chlorinated solvents and lung cancer: Results of the ICARE study Occupational and Environmental

Medicine, 71(10), 681-689

ICARE cohort (CC14 women CEI 1)-Cancer

2799644

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

Metric 3: Comparison Group

High x 0.4 0.4 This is a is French multi-center population-based

case-control study conducted from 2001-2007. It in-
cluded a cancer registry. Case recruitment was per-
formed in collaboration with the French network of
cancer registries. Population-based controls were se-
lected by incidence density sampling. All steps of
the participation were provided.

Medium X 0.4 0.8 All attrition was clearly recorded. 10% of eligible

cases could not be located. 16% died, and 5% could
not be interviewed because of health status. 87%
of those remaining agreed to participate. 94% of
eligible controls were contacted and 81% agreed to
participate. There were a few subjects that were not
included in the analysis based on the numbers in the
table with out explanation, but this was <10%.

High X 0.2 0.2 Controls were selected based on incidence density

sampling and were frequency matched to cases by
gender and age with further stratification to make
SES distribution comparable to the general popu-
lation living in the departments. Cases were more
likely to be current smokers, but this was addressed
in the analysis.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low X 0.4 1.2 Data was collected via a questionnaire. For each job

held for at least 1 month, information was collected
on the tasks and specific exposures of interest. TCE
was the only chlorinated solvent specifically listed
and Perc was stated to be the one agent that was
self-reported. Chlorinated solvents were assessed us-
ing a JEM. For each combination of ISCO and NAF
codes, JEM assigned three indices of exposure 1)
probability of exposure, 2) intensity of exposure, and
3) frequency of exposure. JEM provided an aver-
age level of exposure during a usual work day. Cu-
mulative Exposure Index (CEI) was calculated and
transformed into categorical variables. However, it
appears that exposure is solely based on self-report
and professional judgement.

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Mattei, F; Guida, F; Matrat, M; Cenee, S; Cyr, D; Sanchez, M; Radoi, L; Menvielle, G; Jellouli, F; Carton, M; Bara, S; Marrer, E; Luce,

D; Stiicker, I (2014). Exposure to chlorinated solvents and lung cancer: Results of the ICARE study Occupational and Environmental

Medicine, 71(10), 681-689

ICARE cohort (CC14 women CEI 1)-Cancer

2799644

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 5:
Metric 6:

Exposure levels
Temporality

Medium
Low

x 0.2
x 0.4

0.4
1.2

Each chemical had at least 3 levels (control + 2 or
more CEI levels)

The temporality of exposure and outcome is uncer-
tain.

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization
Metric 8: Reporting Bias

High
High

x 0.667
x 0.333

0.67
0.33

All cases were histologically confirmed.
Sufficient details were provided.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

Metric 11: Co-exposure Confounding

High

Medium x 0.25

Low

X 0.5 0.5 Confounders adjusted for included age at interview,

department, smoking history, number of jobs, and
SES. Genders were evaluated separately.
0.5 Information was obtained from a questionnaire with-
out reporting reliability or validity of the question-
naire.

X 0.25 0.75 Exposure to asbestos was adjusted for in the anal-
ysis. It was noted that exposure to one solvent
did not preclude exposure to the others, subjects
were categorized in into mutually exclusive exposure
groups according to various combinations of specific
solvents. Combinations were evaluated separately.
However, it appears that there may be too much
correlation between exposure to some chemicals.

Domain 5: Analysis

Metric 12

Study Design and Methods

Medium

x 0.4

0.8

Method is acceptable.

Metric 13

Statistical power

Medium

x 0.2

0.4

Likely sufficient.

Metric 14

Reproducibility of analyses

Medium

x 0.2

0.4

Information was sufficient.

Metric 15

Statistical models

Medium

x 0.2

0.4

Methods are transparent and assumptions were met.

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16

Use of Biomarker of Exposure

Metric 17

Effect biomarker

Metric 18

Method Sensitivity

Metric 19

Biomarker stability

Metric 20

Sample contamination

Metric 21

Method requirements

Metric 22

Matrix adjustment

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Mattei, F; Guida, F; Matrat, M; Cenee, S; Cyr, D; Sanchez, M; Radoi, L; Menvielle, G; Jellouli, F; Carton, M; Bara, S; Marrer, E; Luce,

D; Stiicker, I (2014). Exposure to chlorinated solvents and lung cancer: Results of the ICARE study Occupational and Environmental

Medicine, 71(10), 681-689

ICARE cohort (CC14 women CEI 1)-Cancer

2799644

Domain

Metric

Rating^

MWF* Score

Comments^

Overall Quality Determination1"



Medium

1.8



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 14: Garcia et al. 2015: Evaluation of Cancer Outcomes

Study Citation: Garcia, E; Hurley, S; Nelson, DO; Hertz, A; Reynolds, P (2015). Hazardous air pollutants and breast cancer risk in California teachers:
A cohort study Environmental Health: A Global Access Science Source, 14(1), 14

Data Type: Cohort CC14 CTS BreastCancer Q4-Cancer

HERO ID: 3014082

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

-i^
-i^

Metric 3: Comparison Group

High X 0.4 0.4 California Teachers Study including active and re-

tired female teachers and administrators were en-
rolled in the California State Teachers Retirement
System and completed a questionnaire. Study pop-
ulation was comprised on 5676 women. All partic-
ipants were included using the same inclusion and
exclusion criteria.

High X 0.4 0.4 Large sample of study population excluded due to

women who were not residing in California at base-
line, had unknown history of prior cancer, had prior
history of invasive or in situ breast cancer, asked to
be removed from study after joining, or had an ad-
dress that couldn't be geocoded. This represents ad-
equate explanation of attrition and is not expected
to bias the results.

High X 0.2 0.2 Cases and controls were stated to be similar. Covari-

ates that were different between groups were consid-
ered and included as covariates in the final model.,
including a term for grouped personal risk factors.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Medium x 0.4

Metric 5: Exposure levels

Medium x 0.2

0.8 NATA identified and prioritized the air toxicants
with respect to their potential population health
risks. The first NATA was conducted based on 1996
emissions. EPA models annual ambient HAP con-
centrations using the Assessment System for Pop-
ulation Exposure Nationwide (ASPEN). This is a
well-established method of determining exposure.,
but may lead to some non-differential exposure mis-
classification.

0.4 By examining each compound individually, they cat-
egorized them into four quantiles of concentration
without including exposure from any other com-
pound in the model. Level of exposure adequate.
Included four quantiles of exposure, Q1 being no ex-
posure.

Continued on next page

-------
. continued from previous page

Data Type: Cohort CC14 CTS BreastCancer Q4-Cancer

HERO ID: 3014082

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 6: Temporality

Medium x 0.4

0.8 Chose to use the 2002 ambient air concentration es-
timates for this study because that year was approx-
imately the mid-point for the follow up period. De-
cided against combining multiple years of estimate
due to inconsistent methodical approaches and tem-
poral variations in the level of agreement between
years of the assessments which could introduce ex-
posure misclassification.

Domain 3: Outcome Assessment

Metric 7: Outcome measurement or characterization

Metric 8: Reporting Bias

High X 0.667 0.67 CTS cohort is followed annually for cancer diagno-

sis, death, and change of address. Annual linkage
between CCR and cohort membership was used to
identify incident cancer rates. Defined a case as any
woman diagnosed with invasive breast cancer (ICD-
03 site codes C500-C509, excluding those with his-
tology codes for 9050-9055, 9140, and 9590-9992) af-
ter the date they completed their baseline question-
naire through Dec 31, 2011.

High X 0.333 0.33 CCR maintains high standards for data quality and

completeness and is estimated to be 99% complete.
Ascertained date and cause of death from mortality
files as well as reports from relatives.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

High

x 0.5

Metric 11: Co-exposure Confounding

Medium x 0.25

0.5 All models were stratified by age and adjusted either
for race alone or for race and personal risk factors of
interest. For each compound, p-values no each non-
degenerative quantile HR were adjusted for multiple
testing across the ten subsets using False Discovery
Rates.

0.5 Covariates were obtained from the CTS baseline
questionnaire. This was self-reported information,
but there is no evidence to suggest that it is not a
valid method of obtaining covariate information.

0.5 No indication of unbalanced co exposures.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Medium x 0.4 0.8

Medium x 0.2 0.4

Cohort was appropriate study design. Examined the
relationship between risk of breast cancer and nu-
merous compounds of interest. Used two different
methods of parameterizing exposure in the models.
Number of subjects for estimated exposure was 5676
women. There were enough subjects to detect effects
for some chemicals and for some trends.

Continued on next page

-------
. continued from previous page

Study Citation: Garcia, E; Hurley, S; Nelson, DO; Hertz, A; Reynolds, P (2015). Hazardous air pollutants and breast cancer risk in California teachers:
A cohort study Environmental Health: A Global Access Science Source, 14(1), 14

Data Type:	Cohort	CC14	CTS	BreastCancer	Q4-Cancer

HERO ID:	3014082

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 14

Reproducibility of analyses

Medium

x 0.2

0.4

Study design and methods can be reproducible with
information provided. Provided reasoning on how
categories were created for exposure quantiles, why
covariates were used. Covariates included in the
models are reported explicitly.

Metric 15

Statistical models

Medium

x 0.2

0.4

Used COX proportional hazard models to estimate
hazard rate ratios. Parameterized exposures into
quantiles, modeled exposure as a continuous vari-
able, and tested for non-zero slope using a likelihood
ratio test.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

High



1.5



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 15: Carton et al. 2017: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Carton, M; Barul, C; Menvielle, G; Cyr, D; Sanchez, M; Pilorget, C; Tretarre, B; Stiicker, I; Luce, D (2017). Occupational exposure to
solvents and risk of head and neck cancer in women: A population-based case-control study in France British Medical Journal Open,
7(1), e012833

ICARE_CC14_HeadNeckCancer_OR_EverExposure-Cancer
3480125

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

Metric 2: Attrition

High

Medium x 0.4

Metric 3: Comparison Group

High

X 0.4 0.4 296 cases of head and neck squamous cell carcino-

mas and 775 controls were drawn from ICARE, a
French population-based case-control study (Luce
2011, HERO ID 1022113). Only women.
0.8 Participation rates in initial ICARE study were
82.5% for cases and 80.6% for controls. Restrict-
ing to only females with squamous cell carcinomas
in areas of interest led to 296 cases and 755 controls.

X 0.2 0.2 Controls selected from general population based on

age, geographic region and SES. However, there are
statistically significant differences in terms of age,
geographic region, SES, smoking and alcohol con-
sumption. These covariates are all considered in
the analysis. Cases ~2 years younger than controls,
lower SES, and more likely to smoke or drink alco-
hol.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Metric 5: Exposure levels

Metric 6: Temporality

Low X 0.4 1.2 Employment history from in person interviews and

questionnaires. Employment of 1+ month coded
by trained coders blinded to status using Interna-
tional Standard Classification of Occupations and
the Nomenclature des Activites Frangaises. Job-
exposure matrix from French Institute of Health
Surveillance to predict exposure probability, inten-
sity, and frequency.

Medium X 0.2 0.4 Analysis includes dichotomous ever/never exposed,

as well as continuous exposure intensity, exposure
duration and cumulative exposure indices.

Low x 0.4 1.2 Time between potential occupational exposure and

diagnosis not stated.

z
o

O
»
o

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

ICARE_CC14_HeadNeckCancer_OR_EverExposure-Cancer
3480125

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 7:

Outcome measurement or characterization

High

x 0.667

0.67

Cases identified from cancer registries in 10 ge-

ographical regions of France. Histologically con-
firmed diagnosis from 2001-2007 in women aged 18-
85. ICD-O-3 codes were used to identify squa-
mous cell carcinomas in oral cavity, oropharynx, hy-
popharynx, oral cavity, and larynx (detailed list of
codes in text).

Metric 8:

Reporting Bias

High

x 0.333

0.33

Quantitative description of relevant outcomes (head
and neck cancers in women) from the ab-
stract/methods are provided and extractable.

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization
Metric 11: Co-exposure Confounding

High X 0.5 0.5 Analyses adjusted for geographical area, age, smok-

ing status, tobacco consumption (pack-years) and
alcohol consumption. Interaction terms for smok-
ing and alcohol were also included. SES considered
with last occupation and longest occupation, but did
not impact ORs and were not presented.

Medium X 0.25 0.5 In person interviews with standardized question-
naire.

Medium X 0.25 0.5 Exposures to TCE, Perc, and DCM were strongly

correlated. Rather than adjusting for co-exposures,
exclusive exposure to individual and combinations
of chlorinated solvents were analyzed.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Metric 15: Statistical models

Medium X 0.4 0.8 Study design was appropriate for the research ques-

tions. Logistic regression was used appropriately to
estimate ORs and CIs.

Medium X 0.2 0.4 The cohort contains sufficient participants to detect

an effect for TCE, perc, and DCM. Insufficient data
for carbon tetrachloride, so it was excluded from
analysis beyond an ever/never OR. For analysis in-
volving ever exposure to CCL4, the number of cases
and controls is relatively small.

Low X 0.2 0.6 Although the process of creating the regression mod-

els was described in detail, adjustments used for co-
variates were not explicitly stated.

Medium X 0.2 0.4 Odds ratios and 95% confidence intervals were de-

termined using unconditional logistic regression ad-
justed for key covariates. Models were transparent
and assumptions were met.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Carton, M; Barul, C; Menvielle, G; Cyr, D; Sanchez, M; Pilorget, C; Tretarre, B; Stiicker, I; Luce, D (2017). Occupational exposure to
solvents and risk of head and neck cancer in women: A population-based case-control study in France British Medical Journal Open,
7(1), e012833

ICARE_CC14_HeadNeckCancer_OR_EverExposure-Cancer
3480125

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 6: Other Considerations for Biomarker Selection and Measurement

Metric 16

Use of Biomarker of Exposure

NA

NA

Metric 17

Effect biomarker

NA

NA

Metric 18

Method Sensitivity

NA

NA

Metric 19

Biomarker stability

NA

NA

Metric 20

Sample contamination

NA

NA

Metric 21

Method requirements

NA

NA

Metric 22

Matrix adjustment

NA

NA

Overall Quality Determination1"

Medium

1.8

Extracted

Yes





MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 16: Nelson et al. 2012: Evaluation of Cancer Outcomes

The

Study Citation: Nelson, JS; Burchfiel, CM; Fekedulegn, D; Andrew, ME (2012). Potential risk factors for incident glioblastoma multiforme:
Honolulu Heart Program and Honolulu-Asia Aging Study Journal of Neurooncology, 109(2), 315-321

Data Type: HHP-HAAS CCL4 glioblastoma high occupational-Cancer

HERO ID: 3481852

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

High x 0.4

Metric 2: Attrition

Metric 3: Comparison Group

High x 0.4

High x 0.2

0.4 Cohort of aging men of Japanese ancestry born be-
tween and 1900 and 1919 and between age 45-68
at time of initial examination (1965-1968). Par-
ticipants identified through WWII selective service
records. Of 14,426 men estimated to be Oahu res-
idents, 11,148 were located and 8,006 completed a
baseline examination (>70% of target population).

0.4 Participants followed through series of six follow-up
examinations from 1968-2000, and less than 1% lost
to follow-up (5/8,006). Occupational exposure data
available for entire cohort based on information col-
lected in first and third examinations.

0.2 Participants identified through WWII selective ser-
vice records. All were born between 1900-1919 and
were aged 45-68 at time of initial examination (1965-
1968). There is no evidence that participants were
not similar in health status.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Low x 0.4

Metric 5: Exposure levels

Metric 6: Temporality

Medium x 0.2
Medium x 0.4

1.2 Participants reported present and usual jobs and
years worked at these jobs during the first and
third examinations. Jobs were coded according
to U.S. Bureau of the Census and unique occupa-
tion/industry combinations were identified and in-
dependently assessed by three industrial hygienists.
Likelihood of exposure was assigned by consensus as
none, low, medium, and high. An intensity score
was calculated using the likelihood of exposure mul-
tiplied by number of years worked in usual occupa-
tion.

0.4 Exposure levels categorized as none, low or medium,
and high, but corresponding numerical levels not
presented.

0.8 Exposure based on responses during first and third
examinations (1965-1968 and 1971-1974). GBM de-
veloped during the follow-up periods between 1974-
1995. However, unclear whether exposures fall
within relevant exposure window for outcome.

z
o

O
»
o

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

The

Data Type: HHP-HAAS CCL4 glioblastoma high occupational-Cancer

HERO ID: 3481852

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 7:

Outcome measurement or characterization

High

x 0.667

0.67

All GBM cases were confirmed by histological ex-
amination. The source of initial diagnosis was not
reported, but is assumed to have come from follow-
up examinations, hospital discharge records, and/or
death certificates or searches of the National Death
Index.

HR and 95% CI reported for outcome outlined.
Number of cases and non-cases also reported for each
analysis. All outlined statistical analyses were re-
ported with sufficient detail.

Metric 8:

Reporting Bias

High

x 0.333

0.33

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Metric 10: Covariate Characterization

Medium x 0.5

Medium x 0.25

Metric 11: Co-exposure Confounding

Medium x 0.25

1 Adjustment methods not explicitly described, but

were made for other risk factors included in the
model (age, education, triceps skinfold, sugar con-
sumption, coffee consumption, tea consumption,
chest surgery, blood transfusion).

0.5 Basic demographic, occupational and socioeconomic
data, medical history (chest surgery, blood transfu-
sion, herpes), and lifestyle factors including usual
physical activity, smoking habits, alcohol intake,
and dietary habits identified from questionnaires
completed from the first three examinations (self-
reported), but no report of validation. Exposure
based on self-report of jobs and classification by in-
dependent industrial hygienists. Additional risk fac-
tors (e.g., tricep skinfold thickness) were assessed
during the first three examinations, but no detailed
description of methods provided.

0.5 Cases and non-cases were similar in exposure to sol-
vents, pesticides, and metals.

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Medium x 0.4 0.8

Medium x 0.2 0.4

The study design (prospective cohort) and statisti-
cal methods (including a multivariate analysis to es-
timate the hazard ratio associated with exposure to
CC14, using the Cox proportional hazards regression
model) were appropriate for the research question.
Cohort size (8,006) is sufficient to detect an effect,
but only 9 cases resulting in low statistical power.

z
o

O
»
o

Continued on next page

-------
. continued from previous page

The

Study Citation: Nelson, JS; Burchfiel, CM; Fekedulegn, D; Andrew, ME (2012). Potential risk factors for incident glioblastoma multiforme:
Honolulu Heart Program and Honolulu-Asia Aging Study Journal of Neurooncology, 109(2), 315-321

Data Type:	HHP-HAAS	CCL4	glioblastoma	high occupational-Cancer

HERO ID:	3481852

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 14

Reproducibility of analyses

Medium

x 0.2

0.4

Exact logistic regression relating to each potential











risk factor was performed to obtain exact p-values











which were then used to assess linear trend. Mul-











tivariate analysis performed using Cox proportional











hazards regression model to estimate hazard ratio.

Metric 15

Statistical models

Medium

x 0.2

0.4

Model assumptions were described and met.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

Medium



1.7



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 17: Purdue et al. 2016: Evaluation of Cancer Outcomes

Study Citation:

Data Type:
HERO ID:

Purdue, MP; Stewart, PA; Friesen, MC; Colt, JS; Locke, SJ; Hein, MJ; Waters, MA; Graubard, BI; Davis, F; Ruterbusch, J; Schwartz,
K; Chow, WH; Rothman, N; Hofmann, JN (2016). Occupational exposure to chlorinated solvents and kidney cancer: A case-control
study Occupational and Environmental Medicine, 74(4), 268-274

Case-control study of kidney cancer in workers exposed to chlorinated solvents - CC14	90% OR-Cancer

3482059

Domain

Metric

Rating^

MWF*

Score

Comments^

Domain 1: Study Participation













Metric 1:

Participant selection

High

X

0.4

0.4

Selection factors unlikely to be related to CC14 ex-













posures

Metric 2:

Attrition

Medium

X

0.4

0.8

77% participation in cases; 54% participation in con-













trols; rationale was provided.

Metric 3:

Comparison Group

High

X

0.2

0.2

Age-, gender- and race-matched controls.

Domain 2: Exposure Characterization











Metric 4:

Measurement of Exposure

Medium

X

0.4

0.8

Job exposure matrix

Metric 5:

Exposure levels

Medium

X

0.2

0.4

Indicators of probability, frequency and intensity;













tertiles for cumulative hours exposed.

Metric 6:

Temporality

High

X

0.4

0.4

Exposure lagged to account for cancer latency.

Domain 3: Outcome Assessment











Metric 7:

Outcome measurement or characterization

High

X

0.667

0.67

Cases identifies by cancer surveillance system and











many histologically confirmed.

Metric 8:

Reporting Bias

High

X

0.333

0.33

Odds ratios reported with 95% confidence inter-













vals for kidney cancer and exposure to TCE, CCL4,













DCM and Perc

Domain 4: Potential Counfounding/Variable Control











Metric 9:

Covariate Adjustment

High

X

0.5

0.5

Adjusted for age, sex, race, study centre, education













level, smoking status, BMI and history of hyperten-

Metric 10:

Covariate Characterization

High

X

0.25

0.25

Some covariate information was self-reported (smok-











ing, hypertension, race)

Metric 11:

Co-exposure Confounding

Medium

X

0.25

0.5

TCE exposure did not confound Perc results.

Domain 5: Analysis

Metric 12:	Study Design and Methods

Metric 13:	Statistical power

Metric 14:	Reproducibility of analyses

Medium x 0.4 0.8
Medium x 0.2 0.4
Medium x 0.2 0.4

Case-control study used to evaluate occupational
TCE, Perc, DCM, and CC14 exposure and kidney
cancer.

Between Medium and Unacceptable, Medium is the
better characterization. An elevated risk of TCE
was detected - it just wasn't stat sig.

Odds ratios calculated with unconditional logistic
regression.



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

Continued on next page

-------
. continued from previous page

ui
-i^

Study Citation:

Data Type:
HERO ID:

Purdue, MP; Stewart, PA; Friesen, MC; Colt, JS; Locke, SJ; Hein, MJ; Waters, MA; Graubard, BI; Davis, F; Ruterbusch, J; Schwartz,
K; Chow, WH; Rothman, N; Hofmann, JN (2016). Occupational exposure to chlorinated solvents and kidney cancer: A case-control
study Occupational and Environmental Medicine, 74(4), 268-274

Case-control study of kidney cancer in workers exposed to chlorinated solvents - CC14	90% OR-Cancer

3482059

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 15

Statistical models

Medium

x 0.2

0.4

Adjustments used in determining ORs clearly stated.

Domain 6: Other Considerations for Biomarker Selection and Measurement









Metric 16

Use of Biomarker of Exposure



NA

NA



Metric 17

Effect biomarker



NA

NA



Metric 18

Method Sensitivity



NA

NA



Metric 19

Biomarker stability



NA

NA



Metric 20

Sample contamination



NA

NA



Metric 21

Method requirements



NA

NA



Metric 22

Matrix adjustment



NA

NA



Overall Quality Determination1"

High



1.4



Extracted



Yes







MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; X MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

M

O
»
o

d

o

H

-------
Table 18: Tomenson et al. 1995: Evaluation of Hepatic Outcomes

Study Citation: Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514

D at a Type: No-direct- C CL4-exposure workers ALT-hepatotoxicity-Hepatic

HERO ID: 3688717

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

High

Metric 2:

Attrition

High

X 0.4 0.4 Authors reported that the study group consisted

of 135 workers, 83% of those eligible for inclusion.
These workers were from 3 sites in the northwest of
England who had worked on one of the processes
with full exposure to carbon tetrachloride. The con-
trols consisted of 276 workers from the same plants
but had no risk of exposure to carbon tetrachloride
or other hepatotoxic chemicals. It was reported that
the study and control groups were well matched for
age, height, weight, type of job, and alcohol con-
sumption. The authors did not report the sex of the
workers and a table was provided providing evidence
that the alcohol consumption was similar between
the exposed and control groups, and the ages of both
groups were roughly normally distributed, but no ev-
idence on the other covariates was provided.

X 0.4 0.4 The authors reported that there were 135 workers in

the study group (83% of those eligible for inclusion)
and 276 in the control group (77% of the total). The
authors stated that a short questionnaire was given
to all study and control workers in advance of the
study and was used to select eligible participants.
The authors stated that the exposed workers had
to have potential exposure to carbon tetrachloride
either as full-time or on a regular basis. Workers
were excluded from the control group if they had
worked in or on any of a predefined list of workplaces
where there was potential for exposure to carbon
tetrachloride over the past 5 years.

Continued on next page

-------
. continued from previous page

Study Citation: Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514
D at a Type: No-direct- C CL4-exposure_workers_ALT-hepatotoxicity-Hepatic

HERO ID: 3688717

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 3: Comparison Group

Medium X 0.2 0.4 The authors stated that the study and control

groups were well matched for age, height, weight,
type of job, and alcohol consumption, however no
evidence was provided for this other than a table
for alcohol consumption, which was not divided by
exposure level (only showed study and controls). In
addition, the study group and the controls were from
one of the same sites, however the controls were also
from an additional site located nearby where car-
bon tetrachloride was not handled, which may have
resulted in some differences between the 2 groups.

Domain 2: Exposure Characterization

Metric 4: Measurement of Exposure

Metric 5: Exposure levels

Metric 6: Temporality

Higll X 0.4 0.4 Exposure was estimated based on historical monitor-

ing data for each job category. According to the esti-
mate, study group members were categorized as hav-
ing high, medium, or low exposure to carbon tetra-
chloride. Most work groups had historical personal
monitoring data and the mean of these results was
calculated. For groups of workers where no monitor-
ing data had taken place, categorization was done by
judgment of likely exposure from comparison with
other groups. This judgment was done by a profes-
sional industrial hygienist in association with each
plant manager.

Medium X 0.2 0.4 There were 4 exposure categories: mean results of

none, low (1 ppm or less), medium (1-3 ppm), or
high (4 ppm or more) . This distribution of expo-
sure is adequate to determine an exposure-response
relationship.

High X 0.4 0.4 The outcome assessed, hepatic effects, was based on

blood analysis for all workers and controls. The ex-
posure to carbon tetrachloride was assessed based on
work history which was obtained from a question-
naire that also contained a question on the length
of service in a job exposed to carbon tetrachloride.
This information is sufficient to establish a time or-
der for exposure and outcome.

Domain 3: Outcome Assessment

Continued on next page

-------
. continued from previous page

Study Citation: Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514

D at a Type: No-direct- C CL4-exposure workers ALT-hepatotoxicity-Hepatic

HERO ID: 3688717

Domain

Metric

Rating^

MWF*

Score

Comments^

Metric 7:

Outcome measurement or characterization

High

x 0.667

0.67

The outcomes assessed were hepatic enzymes (ala-

nine transaminase, aspartate transaminase, alka-
line phosphatase, glutamyl transferase, total bile
acids, and 5-nucleotidase), that were measured in
the blood. These tests are well established and have
been used in clinical practice for many years.

Metric 8:

Reporting Bias

Low

x 0.333

1.0

All of the outlined analyses are presented in the
results table. There was no adjustment made for
covariates., but the authors reported that the con-
trols and workers were well matched for age, height,
weight, type of job, and alcohol consumption. How-
ever, alcohol consumption could have been a sig-
nificant factor that affected the results because the
study did not evaluate the difference between expo-
sure groups in terms of alcohol consumption.

The authors stated that the study group and the
controls were well matched for age, height, weight,
type of job, and alcohol consumption. However, the
study was scheduled to start in November 1986 but
after a 2 week period of sample collection (about 60
subjects) there was a problem with the availability
of controls due to a plant breakdown. It was decided
to restart the study in February 1987 and the rest of
the samples were taken during a period of about 8
weeks. Therefore, it is possible that the results could
be different between the samples taken in November
and those in February. The authors analyzed for
a synergistic reaction between exposure to carbon
tetrachloride and alcohol consumption was examined
by including an interaction term between the two
factors in the linear model.

The confounders were assessed based on a question-
naire that was given by one occupational health
nursing officer trained for this purpose, but the au-
thors don't report that the questionnaire was vali-
dated.

Continued on next page . ..

Domain 4: Potential Counfounding/Variable Control
Metric 9: Covariate Adjustment

Low

x 0.5

1.5

Metric 10: Covariate Characterization

Medium

x 0.25

0.5

-------
. continued from previous page

Study Citation: Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514

D at a Type: No-direct- C CL4-exposure workers ALT-hepatotoxicity-Hepatic

HERO ID: 3688717

Domain

Metric

Rating^ MWF* Score

Comments^

Metric 11: Co-exposure Confounding

Medium x 0.25

0.5 The exposure to potential co-exposures was assessed
based on the work history of the workers and con-
trols. Workers were excluded from the control group
if they had worked in or on any of a predefined list of
workplaces were there was exposure to carbon tetra-
chloride or other known hepatotoxins during the pre-
vious 5 years; therefore the potential for exposure to
other chemicals appear minimal.

z
o

O
»
o

Domain 5: Analysis

Metric 12: Study Design and Methods

Metric 13: Statistical power

Metric 14: Reproducibility of analyses

Metric 15: Statistical models

Medium X 0.4 0.8 The cross-sectional design appears to be appropri-

ate for the question of whether carbon tetrachloride
exposure is associated with hepatotoxic effects.

Medium X 0.2 0.4 The number of participants, 135 workers and 276

controls, appears adequate to detect an effect in the
exposed population.

Medium X 0.2 0.4 The description of the analysis is sufficient to be

understandable and reproducible. The results were
presented as the geometric means after logarithmic
transformation for each exposure group.

Medium X 0.2 0.4 Linear models were fitted to the logarithmically

transformed data. The terms in the model included
exposure category, age, sampling time, and measure
of alcohol consumption.

Domain 6: Other Considerations for Biomarker Selection and Measurement
Metric 16: Use of Biomarker of Exposure
Metric 17: Effect biomarker

Metric 18: Method Sensitivity
Metric 19: Biomarker stability

Not Rated NA
Medium x 0.2

Low
Low

x 0.2
x 0.2

NA
0.4

0.6

Of the biomarkers examined, only ALT is specific to
the liver. AST can also be associated with the liver,
but it could indicate damage to another organ. Both
of these biomarkers measure tissue damage but do
not measure functional changes to the liver.
Analytical methods measured biomarkers are ade-
quately reported. No LOQ/LOD reported.

On the morning of collection, blood samples were
transported to the ICI central toxicology lab for
analysis; samples were taken from roughly the same
ratio of study and control participants; stability was
not stated. To minimize any effect of laboratory
variation, blood samples were taken from a roughly
constant ratio of study and control group subjects
on each day.

Continued on next page

-------
. continued from previous page

Study Citation:

Data Type:
HERO ID:

Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514

No-direct-CCL4-exposure	workers	ALT-hepatotoxicity-Hepatic

3688717

Domain

Metric

Rating^

MWF*

Score

Comments^



Metric 20: Sample contamination

Low

x 0.2

0.6

There are no known or measured contamination is-



Metric 21: Method requirements
Metric 22: Matrix adjustment

High

Not Rated

x 0.2
NA

0.2
NA

Instrumentation allows for the biomarker with a high
degree of confidence. Biochemical variables mea-
sured with Vitatron PA800 analyser or a Kone CD
analyser.

Overall Quality Determination"1'

Medium



1.8



Extracted



Yes









MWF = Metric Weighting Factor

High = 1; Medium = 2; Low = 3; Unacceptable = 4; N/A has no value.

The overall rating is calculated as necessary. EPA may not always provide a comment for a metric that has been categorized as High.

Overall rating =

]T\ (Metric Score; x MWF;) / J] . MWFj

if any metric is Unacceptable
(round to the nearest tenth) otherwise

where High => 1 to < 1.7; Medium => 1.7 to < 2.3; Low => 2.3 to < 3.0. If the reviewer determines that the overall rating needs adjustment, the original rating is
crossed out and an arrow points to the new rating.

^ This metric met the criteria for high confidence as expected for this type of study



O

H

o

O

z
o

H

n

HH

H

H

O
»
o

d

o

H

-------
Table 19: Tomenson et al. 1995: Evaluation of Hematological and Immune Outcomes

Study Citation: Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514

Data Type: High-CCL4-exposure workers hemaglobin-Hematological and Immune

HERO ID: 3688717

Domain

Metric

Rating^ MWF* Score

Comments^

Domain 1: Study Participation

Metric 1: Participant selection

High

Metric 2:

Attrition

High

X 0.4 0.4 Authors reported that the study group consisted

X 0.4 0.4 The authors reported that there were 135 workers in

Continued on next page

-------
. continued from previous page

Study Citation: Tomenson JA; Baron CE; O'Sullivan JJ (1995). Hepatic function in workers occupationally exposed to carbon tetrachloride. Occupa-
tional and Environmental Medicine, (52), 508-514
Data Type: High-CCL4-exposure_workers_hemaglobin-Hematological and Immune