Nex
ADVANCING THE NEXT GENERATION (NEXGEN) OF RISK ASSESSMENT:
THE PROTOTYPES WORKSHOP
-------
EPA/600/R-11/100
September 2011
http://epa.gov/risk/nexgen
r/EPA
ADVANCING THE NEXT GENERATION (NEXGEN) OF RISK ASSESSMENT:
THE PROTOTYPES WORKSHOP
NOVEMBER 1-3,2010
RESEARCH TRIANGLE PARK, NORTH CAROLINA
National Center for Environmental Assessment
Office of Research and Development
U.S. Environmental Protection Agency
Washington, DC 20460
-------
DISCLAIMER
This document summarizes the discussions presented at an experts' workshop held Nov 1-3, 2010, in
Research Triangle Park, NC. The purpose of the workshop was to review conceptual approaches to
prototype development. This document is not all inclusive or binding. Conclusions and
recommendations to the U.S. EPA may not represent full consensus. The views expressed in this
document are those of the Workshop Participants and do not necessarily reflect the views and
policies of the U.S. Environmental Protection Agency. Mention of trade names or commercial
products does not constitute endorsement or recommendation for use.
This document was prepared initially by ICF Inc., an EPA contractor (Contract No EP-C-09-009
Work Plan, Budget, Work Assignment 1-37). This report captures the main points and highlights of
the meeting. It is not a complete record of all detailed discussion, nor does it embellish, interpret, or
enlarge upon matters that were incomplete or unclear. Statements represent the individual views of
each participant.
-------
CONTENTS
1. Background and Objectives of the Workshop 1
2. Introduction 1
2.1. Towards a Framework for NexGen Risk Assessment 2
2.2. NexGen Risk Assessment Issues 3
3. Data-rich Prototype Breakout Groups 4
3.1. Lung Injury-Ozone 4
3.2. Developmental Impairment-Thyroid Hormone Disrupters 6
3.3. Cancer- Benzene 7
3.4. Cancer - PAHs 9
3.5. Panel Discussions and Cross-cutting Themes 10
4. Day 3 - Approaches for Chemicals with Less Data (Tier 2 Assessments) 13
4.1. Day 3 Presentations 14
Dr. Derek Knight, Approaches from the European Union: REACH 14
Dr. Karen Leach, Approaches for Safety Assessment to Pharmaceuticals 14
Dr. Michael DeVito, Tox21 Targeted Testing 15
Dr. Christopher Portier, Genetic and Genomic Risk Assessment for Identifying Hazards 15
Dr. Alexander Tropsha, Combined Application of Chemical and Molecular Biology Information 16
Dr. David Reif, AToxicological Priority Index (ToxPi) for Prioritizing Chemicals based on the ToxCast
Data 16
Dr. Richard Judson, A Framework for High-Throughput Risk Assessment 16
Dr. Russell Thomas, Can Genomics Be Used to Derive a Meaningful Points-of-Departure for Cancer
and Noncancer Risk Assessment? 17
4.2. Panel and Open Discussion 18
5. References 20
Appendix A. Final Agenda: Advancing the Next Generation (NexGen) of Risk Assessment: The
Prototypes Workshop A-l
Appendix B. Participants in the Advancing the Next Generation (NexGen) of Risk Assessment: The
Prototypes Workshop B-l
-------
1. Background and Objectives of the Workshop
The U.S. Environmental Protection Agency (EPA), in collaboration with other federal and state agencies,
is advancing the next generation of risk assessment through a project named "NexGen." The project
aims to better incorporate recent advances in molecular and systems biology into risk assessment,
thereby potentially making risk assessments faster, less expensive and/or more scientifically robust.
This transition is expected to evolve over the next 10-20 years as new knowledge and approaches
become available. NexGen partner organizations include the National Institute of Environmental Health
Sciences (NIEHS), National Toxicology Program (NTP), Centers for Disease Control/Agency for Toxic
Substances and Disease Registry, National Human Genome Research Institute, and the State of
California's Environmental Protection Agency (CalEPA).
EPA convened a 3-day expert workshop on November 1-3, 2010, in Research Triangle Park, North
Carolina to discuss a draft framework, early draft prototypes, research, and other project elements. The
workshop sought individual input, rather than consensus, in meeting its discussion goals.
Days 1 and 2 of the workshop focused on deliberative drafts of data rich prototype health assessments
(i.e., Tier 3 assessments). The goals for the first two days were to:
• Refine early-stage (i.e., early draft) case study health effects assessments of data-rich chemicals,
referred to as "Prototypes."
• "Reverse engineer" from molecular system biology data to "known" public health risk estimates
based on in vivo human and animal bioassay data and, ultimately, demonstrate proof of
concept, elucidate value of information, and characterize decision rules with the final
prototypes.
• Summarize options for expanded future work and research needs.
Day 3 of the workshop focused on approaches applicable to assessing the potential risks posed by
chemicals with limited or no traditional data (i.e., Tier 2 assessments). The goals of the third day of the
workshop were to:
• Identify and discuss a wider variety of new data, methods, and knowledge to help characterize
data limited chemicals.
• Consider how this information may augment, extend, or replace traditional data in health
assessment.
• Summarize options for expanded future work and research needs.
The workshop was attended by approximately 40 federal and non-federal experts and 80 EPA and
NexGen partner organization staff.
2.
Dr. lla Cote, EPA, provided a brief introduction to the NexGen project and the meeting. She noted that
the NexGen Prototype assessments are not intended to change the current risk assessments for the
specific chemicals evaluated, but rather will attempt to demonstrate proof of concept, characterize the
value of information, and explore decision rules for appropriate use of molecular and systems biology
data in general. For this initial effort, a narrowly defined set of diseases/disorders, causative chemicals,
and mechanisms of action were used. Prototypes also rely on illustrative rather than comprehensive
-------
data sets. A broad set of molecular systems biology disciplines and assays will be considered as data are
available.
Day 1 continued with two plenary presentations of a proposed framework for NexGen risk assessment
and an overview of NexGen risk assessment issues, each of which was followed by an associated
question and answer session. Four breakout groups—one for each of the four draft Prototypes-
deliberated for the remainder of Day 1 and the morning of Day 2 and reported back to the plenary on
Day 2. A panel discussion on cross-cutting themes of the breakout groups concluded Day 2 of the
workshop.
2.1. a
Dr. Daniel Krewski, of the University of Ottawa and Risk Sciences International, presented a draft
framework for conducting NexGen risk assessments. This NexGen Risk Assessment Framework is
comprised of the following three building blocks: Toxicity Testing in the 21st Century (NRC, 2007),
Mclaughlin Centre Framework for Population Health Risk Assessment (Krewski et al., 2007), and Science
and Decisions: Advancing Risk Assessment (NRC, 2009)
Independently, each of these building blocks serve to advance the field of risk assessment. For example,
the first building block, National Research Council's (NRC's) Toxicity Testing in the 21st Century report,
provides a vision for the future of toxicity testing based on the identification and prevention of
perturbations of toxicity pathways. The vision presented in this report focuses on predicting chemical
properties and characteristics, where possible and appropriate, by using computational tools. The vision
also emphasizes incorporating high throughput approaches using cells or cell lines, preferably of human
origin, into toxicity testing. The risk assessment goal is to employ high throughput assays and
computational methods in toxicology to efficiently identify potential toxic agents, and subsequently
establish human exposure guidelines that will avoid pathway perturbations. Figure 1 illustrates the risk
assessment process, where the four stages of risk assessment (as presented in the NRC's "Red Book,"
Risk Assessment in the Federal Government: Managing the Process (NRC, 1983)), overlap with the new
scientific tools and technologies. This figure demonstrates that while this new NRC vision incorporates
the traditional four stages of risk assessment, the technical activities conducted within each stage will
change dramatically.
The second building block, the McLaughlin Centre Framework for Population Health Risk Assessment,
addresses risk assessment on a population level. This Framework is based on the concept of integrating
traditional human health risk assessment with population risk assessment, a comprehensive assessment
of health risks in the general population based on biological, genetic, environmental, occupational,
social, and behavioral determinants of health. By bringing together these two parallel fields and
recognizing that there are a number of determinants of health outcomes, this Framework offers a more
multidisciplinary and robust approach to the assessment and management of health risk issues, which is
important for better assessing potential health risks to human populations.
The NRC's Science and Decisions: Advancing Risk Assessment report, informally known as "The Silver
Book," is the third building block of the NexGen Risk Assessment Framework. This report provides
guidance on new directions in risk assessment methodology, such as evaluating uncertainty and
variability in risk in order to derive a distribution of risks that can be used as a more complete basis for
risk management decision making. While the Silver Book includes the four core phases of risk
assessment as presented in "The Red Book" (NRC, 1983), it places greater emphasis on the first phase of
problem formulation and scoping. Specifically, the Silver Book promotes early and thorough planning
that tailors the assessment's level and complexity to the demands of the problem and provides
approaches for obtaining clearer estimates of population risk and advancing cumulative risk assessment.
-------
Chemical
;haracterization
Mode of Action
Assess ^^
Biological (5j
Perturbation(sV'^
Hazard Identification
Dose-response Assessment
Affected
Pathway
Measures
of Dose
In Vitro
Dose-response
Analysis for
Toxicrty Pathway
Perturb at on(s)
It *t
Population-based Studies
XT
Calibrating In Vitro
and Human Dosimetry
XT
Human Exposure Data
Exposure Assessment
Risk Characterization
Figure 1. Scientific Tools and Technologies that can be used in Risk Assessment. (1) High throughput
screens, (2) Stem cell biology, (3) Functional Genomics, (4) Bioinformatics, (5) Systems biology, (6)
Computational systems biology,(7) Physiologically-based pharmacokinetic models, (8) Structure-activity
relationships, (9) Biomarkers, (10) Molecular and genetic epidemiology.
Together, these three building blocks can help produce a NexGen framework that will shape the future
of health risk science. However, it is important to keep in mind that although each of these
buildingblocks will be useful in identifying future risk assessment principles, procedures, and practices,
their integration into an overarching NexGen Risk Assessment Framework will continue to evolve as the
technologies and the decision analysis approaches develop. As a result, the challenge in the near term is
to begin the process of integrating these components into an overarching NexGen framework and
developing acceptance of this broader view as risk assessment is transitioned from the current to the
new paradigm in order to meet the demanding goals of human health risk assessment into the 21st
century. An additional goal would be to build capacity for analyzing the data from the most recent
scientific advances and incorporating them into risk assessment.
2.2. NexGen Risk Assessment Issues
The challenges and opportunities for NexGen risk assessment were addressed by Dr. Weihsueh Chiu of
the Environmental Protection Agency's National Center for Environmental Assessment, Office of
Research and Development. Dr. Chiu first identified the key challenges facing NexGen quantitative
dose-response assessment as integration of new data and models, including high-throughput systems, in
NexGen risk assessment; production of higher throughput assessments; definition of key terms like
"adversity" from a quantitative perspective; prediction of metabolism and effects at environmental
levels using in vitro assays; integration of assessments across different biological and temporal scales
and determination of uncertainty and variability across these scales; assessment of cumulative
interactions of multiple stressors; and achievement of diverse risk management goals.
-------
Dr. Chiu outlined the previously proposed approaches to addressing these challenges, which included:
(1) a procedurally simple approach similar to a point-of-departure/uncertainty factor approach
(consistent with Toxicity Testing in the 21st Century), (2) a probabilistic dose-response assessment
addressing incremental risk through evaluation of apical endpoints at the individual and population
level, and (3) a biologically based dose-response modeling to predict apical endpoints. This would
provide a quantitative link between precursor effects and adversity or risk, and a mechanistic basis for
assessing cumulative endpoints. He noted, however, that there are some disadvantages to each
approach individually, and existing proposals did not respond to all of the challenges for NexGen risk
assessment. As a result, Dr. Chiu proposed a three-pronged approach to fill the remaining gaps. The
first prong would involve a point-of-departure-based approach for screening and/or prioritization of
chemicals, but augmented, as needed, by considerations for susceptibility and background conditions,
as well as possibly probabilistic methods. The second prong would be an "off-the-shelf" adaptation of
existing human biomarkers and prediction models, taking advantage of existing biomedical knowledge,
in order to quantify different degrees effects across a population and integrate the effects of different
stressors. The third prong would further extend this approach through the use of new (likely molecular)
biomarkers and prediction models developed by integrating the next generation of biological data and
understanding.
Dr. Chiu's three-pronged approach was well-received by the other participants. The need for further
collaboration with the medical community and other entities was proposed as a source of additional
research and an important component of developing needed expertise.
3. Data-rich Prototype Breakout Groups
Each Prototype breakout group considered the following general discussion questions:
1. Are the right questions being asked?
2. Have the most useful methods been identified?
3. What kinds of data are anticipated and how can results be used to (a) identify potential adverse
health effects, (b) inform us about dose-response, (c) help link dose to exposures, and (d)
improve our understanding of important ^^^^^^^^^^^^^_^^^^^^^^^^^^^^
issues such as sensitive subpopulations
and mixtures exposures?
4. What are the weight of evidence criteria,
key uncertainties, and areas of scientific
disagreement that require particular
consideration?
Thyroid Hormone Disrupters
Breakout groups also discussed prototype-
specific questions. Summaries of the breakout > Cancer-Polycyclic Aromatic
group discussions are provided in the sections Hydrocarbons (PAH)
that follow in the order shown in the
accompanying text box.
Initial Draft Prototypes Discussed
at the Workshop
> Lung Injury-Ozone
> Developmental Impairment-
> Cancer - Benzene Prototype
3.1. Lung Injury - Ozone
The goal of this prototype is to evaluate the utility of molecular biology data in understanding health
outcomes and the feasibility of developing a biologically based dose-response (BBDR) model using an
integrated systems approach combining laboratory experiments and computational modeling for the
-------
data-rich chemical ozone. This prototype is designed to help develop an in vitro model to predict in vivo
effects for selected endpoints in toxicity pathways associated with ozone-induced lung injury and
inflammation. Furthermore, this prototype may illustrate how BBDR modeling can be used to integrate
diverse kinds of data at different scales of biological organization and how a toxicity pathway approach
can be used to better understand the cellular and molecular events that underlie ozone-induced
inflammation.
Dr. Robert Devlin, EPA's National Health and Environmental Effects Research Laboratory (NHEERL), led
the Lung Injury - Ozone prototype breakout group with a presentation outlining the aims of the project,
the proposed study design, and the basis for choosing ozone as a prototype chemical. Dr. Devlin noted
that two projects are associated with this prototype, the first of which asks, "Can we expose cells in vitro
and in vivo, run microarray analyses, and determine how well in vitro data can predict in vivo
responses?" And the second project asks, "Can we model in vitro intracellular events that might explain
why cells are making lnterleukin-8 (IL-8)?"
The Lung Injury - Ozone breakout group discussion was loosely based around these questions and the
nine more specific questions. The discussion points during the breakout group sessions were highly
representative of the set of comments received prior to the workshop in response to these nine
questions. The main topics of discussion were focused on clarification of the prototype terminology and
approach, whether animal data are needed to supplement human data, whether toxicity pathway data
are quantitative enough to use as model inputs, key considerations for incorporating population
variability into models, whether other toxicants should be assessed concurrently with ozone to help
validate models, whether multiple cell lines should be employed, whether the toxicity pathway
approach is appropriate for this project, and whether there is value added from upstream real-time
measurement of biomarker events.
The Lung Injury - Ozone breakout group concluded that in vitro modeling could potentially be used as a
tool to rank the toxicity of various pollutants for risk assessment, and correlations could be sufficient to
allow for prioritization of in-depth, chemical-specific analysis. Most participants were in support of
supplementing the human data with animal data to allow for testing of more doses, time points, and
measured variables. Some participants argued, however, that animal data was not needed for this
phase of the project, but could be useful to fill data gaps for chemical-specific analyses beyond ozone.
Ultimately, the group agreed that animal data was necessary for building a quantitative model because
the human data alone was insufficient. One participant suggested that a comprehensive review of what
is already known about ozone be conducted first to establish what kind of modeling is feasible given the
animal, human, and in vitro data that already exist.
A few participants emphasized the importance of collecting data for other toxicants associated with an
inflammatory response, and the group agreed that this was necessary for the acceptance of a predictive
model within the scientific community. One participant specifically noted that the data used to
generate a model cannot then be used to validate that model, thus generally necessitating incorporation
of additional data from the same or other toxicants. The participants also generally agreed that if the
budget could support only analysis of multiple toxicants or analysis of effects in multiple cell lines,
priority should be given to the multi-toxicant approach. However, participants agreed that using
multiple cell lines offered another valuable perspective, but that those cell lines would have to be well
characterized. The group agreed that at this stage of the process, it was appropriate to use human
primary cells.
The participants reached few conclusions on strategies to incorporate population variability into the in
vitro models. While some participants argued that it was important to understand the "normal" state
first by observing variability among healthy human cells, others argued that cells from humans with
-------
susceptibility factors such as asthma should be used as the basic model. This discussion raised a number
of important questions about whether toxicity pathways would be different for high responders (i.e.,
susceptible populations), whether the data are quantitative enough to distinguish between high and low
responders, and whether sample sizes would be large enough to capture population variability. In the
end, the question of whether genetic variability can be built into in vitro studies remained unanswered.
The Lung Injury- Ozone breakout group expressed some confusion over the meaning of toxicity
pathways and the relevance of this approach to the ozone project. One participant remarked that
multiple "toxicity pathways" are likely to lead to the same effects, and that trying to define a "toxicity
pathway" in light of the many aspects of homeostasis is highly problematic. Another participant
recommended that the term "toxicity pathway" be changed to "toxic signature," recognizing that there
are examples of consistent signatures across different compounds, while another participant
recommended using "stress pathways" instead because many of these pathways have already been
identified and are distinct.
Finally, the group discussed the relevance of modeling upstream events in this prototype and strongly
agreed that experimental data on the kinetics and dynamics for immediate, far upstream events (e.g.,
calcium changes, free radicals) was needed in addition to kinetic and dynamic data for midstream events
(e.g., signal transduction pathways), and final downstream events (e.g., transcription factor activation).
Participants further suggested that some of the highest-priority endpoints to analyze might be catalase
activity, nuclear respiratory factor (Nrf), nuclear factor kappa-light-chain-enhancer of activated B cells
(NF-KB), glucose metabolisms/solute carrier (SLC) transporters , and lipid mediator metabolism
(cytochrome 4fl5). One participant also suggested identifying five to ten variables in each known stress
pathway to measure and map to microarray analyses.
3.2. Developmental Impairment - Thyroid Hormone Disrupters
Drs. Mary Gilbert and Kathleen Raffaele, EPA, opened the Developmental Impairment - Thyroid
Hormone Disrupters prototype breakout group by discussing the relationship between endocrine
disruption and the thyroid hormonal pathway. Disruption of thyroid hormone homeostasis has been
linked to adverse neurological and developmental effects, including low intelligence and learning
disabilities, making it a significant public health concern. The hypothalamic-pituitary-thyroid (HPT) axis,
an important regulator of neurodevelopment, is well-studied and can provide insight into the toxicity
pathways for thyroid hormone disruption. Distinct elements in these pathways are known to be
disrupted by exposure to many environmental toxicants, including many chemicals that have been
evaluated by EPA's IRIS Program. Available screening assays provide important data regarding specific
elements of the toxicity pathways and many more are currently under development.
The ultimate goal of the breakout group was to develop approaches for predicting adverse impacts on
brain development from exposure to environmental chemicals via interference with thyroid hormone
homeostasis. The breakout group recognized, however, that developing such approaches would be
difficult due to current data gaps. For example, an adverse outcome in the brain is not induced by direct
interaction between the chemical and the brain, but is secondary due to changes in thyroid status.
Additionally, the relationship between the magnitude of disruption and adverse outcome is not well
characterized; timing and magnitude of disruption is critical for predicting an adverse impact on brain
development. Furthermore, the mechanism by which thyroid hormone disruption interferes with brain
development is not fully understood. In order to more accurately and efficiently screen chemicals for
potential to disrupt thyroid hormone homeostasis, the group focused on the following three issues:
(1) assay identification and refinement, (2) algorithm development for toxicity and hazard prediction,
and (3) assay conduct, data analysis, and data reporting for risk assessment needs.
-------
The participants addressed assay identification and refinement by stating that the ToxCast™ database,
developed by EPA's National Center for Computational Toxicology, in collaboration with the National
Institutes of Health National Chemical Genomics Center (NCGC) and NIEHS, needs to be expanded to
include more assays that probe thyroid disruption toxicity pathways. They acknowledged that hepatic
catabolism is well covered by available assays; however, there is minimal coverage of the other nodes in
the HPT axis. While new assays are currently being developed, the breakout group identified
opportunities for refining existing assays. The participants stressed the importance of comparing results
from available assays to in vivo data from primary sources (e.g., ToxRef, EDSP, NTP), secondary sources
(IRIS, ATSDR), and peer-reviewed literature. They also highlighted an existing opportunity to run known
thyroid disrupting chemicals through available assays to better assess their predictive power and
potentially elucidate other modes of action. For future assay development, the breakout group
recommended incorporating quantitative capabilities to available assays to yield data for future use
whenever feasible. They also suggested that assays be designed to cover more aspects of each pathway
and assess across different nodes to assist in grouping of toxicants and reading across data.
Regarding algorithm development, the participants identified several keys to developing predictive
algorithms including providing a probabilistic landscape of inputs, deriving a weight of evidence
approach for integrating results from multiple assays, and utilizing new approaches for capturing higher
level information (e.g., curve class descriptor and dose-response information for predictive modeling).
Algorithm development is difficult because thyroid hormone disrupters potentially act on multiple
molecular target sites and tissues, which is further complicated by the fact that extrathyroidal target
tissues are less well understood than thyroidal target tissues and homeostatic pathways. Thus, the
participants underscored the importance of addressing both direct and downstream impacts of
perturbations to thyroid hormone homeostasis. For target sites such as hepatic catabolism and
elimination of thyroid hormones, the participants demonstrated confidence in the ability to develop
algorithms. For other nodes, however, more assay development is needed. They suggested that assay-
specific and system-specific biological context be incorporated into the algorithms, and the results could
then be ranked according to levels of confidence with the ultimate goal of optimizing assays for
inclusion. Additionally, with sufficient concentrations and time points, it will be possible to model data
and determine concentration-time-response curves.
In addition to reliable dose-response information, the participants agreed that interpretation, data
analysis, and assay reliability are critical factors for use in risk assessment. Assays that take into account
timing, sensitive life stages and population variability, and tissue-specific differences in response are
desirable when assessing the numerous target sites potentially associated with thyroid hormone
disruption. There also exists a need for concordance in responses between different types of tissues in
order to achieve a systems integration approach; however, it is not clear what level of systems
integration is necessary for using screening data for risk assessment. The breakout group concluded
that the thyroid disrupting compounds will serve as a good initial case study. Despite clear data needs,
the participants stressed that risk assessment is an iterative process and enough data exist currently to
begin evaluating how approaches identified from the thyroid hormone disruption case study could
inform regulatory decision making.
3.3. -
Dr. Martyn Smith, University of California-Berkeley, Dr. Bob Sonawane, EPA, and Dr. Kate Guyton, EPA,
led the Cancer - Benzene prototype breakout group discussion. As discussed in the draft prototype,
benzene exposure at high doses causes acute myeloid leukemia (AMI) and myelodysplastic syndromes
and has been associated with lymphoproliferative disorders including childhood lymphoblastic leukemia.
Biological plausibility for a causal role of benzene (or its metabolites) in these diseases comes from its
-------
genotoxic effects and toxicity to hematopoietic stem cells or progenitor cells, from which leukemias
arise. The impact of this toxicity is manifested as lowered blood counts (hematotoxicity). However, the
mechanism of action for benzene-induced leukemia is still unknown, making assessment of risk in the
low-dose region uncertain. The draft prototype proposes a systems biology approach, encompassing
toxicogenomic, epigenomic, and phenomic endpoints relevant to leukemia. The prototype proposes
using a biomarker of early effect that is predictive of leukemia to examine dose-response relationship in
low-dose region (e.g., hematotoxicity, chromosome changes and altered gene expression).
The Cancer - Benzene breakout group addressed four main questions: (1) what new data are available
and can these new data and methods improve our understanding of risk in a meaningful way? (2) how
can this new type of information best be incorporated into health assessments? (3) what new policies
and procedures are needed? (4) what are the next steps to take to move forward with the goals set out
in this prototype?
The breakout panel identified many sources of new data that can be used to further risk assessment
procedures. Since the last EPA Integrated Risk Information System (IRIS) dose-response assessment was
conducted for benzene in 2005, more than 60 new epidemiologic studies have been published and could
be relevant for a NexGen assessment. A plethora of 'omics data is also now available, including work to
identify the disease pathway initiated by benzene (or metabolites) exposure. Disease pathway data for
benzene will establish a pattern and help identify effects from different chemicals that show a similar
pattern. Hematoxicity and chromosome damage data, genetic factors (i.e., SNPs), toxicokinetic
variability, life stage susceptibility (i.e., in utero), and birth defects data might be used to help
understand the dose-response for benzene exposures. Recent studies have shown that pre-existing
conditions, such as obesity and blood disorders, can increase an individual's susceptibility to benzene
related diseases. Lastly, reproductive outcomes, such as reduced sperm count, are also showing
potential as predictive endpoints for benzene-induced leukemia.
The breakout group discussed the best methods by which this new data can be used in the risk
assessment paradigm. There was wide support to use the epidemiological data to verify the results
from the new 'omics data, rather than incorporate the two data types together. Specifically, Dr. Smith
suggested looking at the dose-response relationship of the new 'omics data to develop a point of
departure. Some participants disagreed, stating that the epidemiological data and 'omics data will not
align because the dosing regimens are varied and certain genes are expressed at low doses that are not
at high doses and vice versa. If the 'omics' data can be used to support the interpretation of the
epidemiologic studies, this will decrease the uncertainty of the assessment. Also, new data on
preexisting conditions can be used to identify susceptible populations. Lastly, new mechanism data can
possibly predict adverse co-exposures.
To incorporate this new data, new policies are needed, and procedures standardized, to ensure that the
data can be compared across studies. New guidance and protocols are needed on the use of 'omics'
data in a risk assessment. In addition to the guidance, training courses and communications are needed
to support effective implementation and understanding of procedures among researchers and risk
assessors.
The Cancer - Benzene breakout group recommended a number of research initiatives that could be
pursued including: (1) developing a testing regimen that uses in vitro stem cells in a 3D niche;
(2) exploring quantitative approaches (e.g., blood counts) for continuous health outcomes;
(3) evaluating hematological data in a biomarker-based approach for parameters or states that predict
leukemia risk; (4) conducting dose-response modeling of 'omics data and biomarker-based approaches
to evaluate the predictability of comparing with the epidemiological data; (5) integrating single and
multiple datasets (e.g., phenomics data, low exposure human studies, and disease-specific pathway
-------
data) into a systems biology model that predicts risk from exposure to benzene; and (6) identifying data
gaps and associated opportunities for model refinement.
Dr. Peter McClure and Ms. Heather Carlson-Lynch, Syracuse Research Corporation, led the breakout
group discussion of their draft Cancer - PAHs prototype, which set out to evaluate whether 'omics data
in combination with existing epidemiology, rodent bioassay data, and mechanistic data (1) can improve
existing methods for evaluating human cancer risk of PAH mixtures, and (2) can lead to development of
predictive tools for hazard identification and dose response for PAH mixtures. The initial objective of
the project was to identify available 'omics data and examine how it could be used to inform human
cancer risk assessment for PAHs.
Dr. McClure described the drivers for this case study. PAHs occur almost exclusively as complex
mixtures, and in the environment, weathering alters mixture composition. Several complex PAH
mixtures and/or occupations with PAH exposure have been shown to be carcinogenic in humans (e.g.,
coke oven emissions, diesel exhaust, tobacco smoke). Many individual PAHs and complex mixtures have
been tested in animal bioassays and have been shown to be carcinogenic, but hundreds of known PAHs
and most complex mixtures have not been tested. Given the universe of PAHs and potential mixtures of
PAHs, testing all of them in carcinogenicity bioassays is not feasible.
Dr. McClure described the results of their literature review that examined studies of 'omics endpoints
following exposure to benzo[a]pyrene and at least one other PAH. Their focus was on discriminating
between carcinogenic and noncarcinogenic PAHs, high potency and low potency carcinogenic PAHs, and
strongly genotoxic and less genotoxic PAHs and evaluating the carcinogenicity, genotoxicity, or potency
of PAH mixtures relative to benzo[a]pyrene. The initial PAH Prototype analysis found that
carcinogenicity in animal bioassays appears to correlate with modulation of genes in the p53 pathway.
The results suggested that genes and gene products in the Mdm2-p53 network can serve as markers for
DNA-damaging effect of PAH or PAH mixtures.
The Cancer - PAHs breakout group discussed the draft prototype analysis and concluded that while
existing 'omics data are limited, they show promise for meeting the overall goal. They suggested that a
directed research program will be required. In the short term, the group recommended identifying
networks or pathways that serve as signatures for PAH-induced cancer. To support this, Dr. McClure
presented the breakout group's short-term recommendations to the plenary:
• Take advantage of the literature for benzo[a]pyrene for further network or pathway inferences.
• Compare the benzo[a]pyrene 'omics signature with signatures for relevant cancers (skin, lung,
bladder).
• Mine the literature and evaluate the weight of evidence for various pathways.
• Obtain and further analyze raw data from comparative studies of PAHs and mixtures.
• Evaluate consistency of 'omics data from in vitro and in vivo studies and across species.
• Assemble multiple data sets and consider possible meta-analysis.
The Cancer - PAHs breakout group also recommended assembling all information for 'omics and
traditional bioassays to identify patterns associated with potency or carcinogenicity.
-------
The group's longer-term recommendations were to design a specific research program strategy to relate
'omics data to carcinogenicity and/or cancer potency and obtain 'omics data in conjunction with apical
endpoint data on a series of standardized complex mixtures and individual PAHs.
3.5. Panel Discussions and Cross-cutting Themes
Several questions developed prior to the workshop
served as starting points for the panel discussions.
The questions posed to each invited panelist are
included in the text box to the right. In addition
to providing answers to these questions, each
panelist provided their insight on next steps for
the NexGen initiative.
Dr. Krewski commented on the progress that has
been made since 2007 in developing the context
for toxicity testing in the 21st century, including
advances in technologies, methodological
approaches, and capacity. He indicated,
however, that there is still a long path forward in
terms of developing data to be used in a NexGen
risk assessment context. He suggested that the
NexGen concept as it currently stands, should be
used as a concrete example of the direction in
which risk assessment needs to proceed.
Developing data to be used in a NexGen risk
assessment will be iterative; therefore, it will be
important to continually revisit the NexGen
approach as new data emerges.
Day 2 Panel Discussion Questions
> Are we making progress in
developing NexGen data and
approaches that can be used in
chemical assessments?
> What did we learn about using
NexGen data and approaches to
identify effects caused by
chemicals?
> What did we learn about using
NexGen data and approaches in
quantitative assessments...
o To evaluate relative potencies
of similarly acting chemicals?
o To account for susceptible
populations in assessments?
o To perform screening level risk
assessments?
The second panelist, Dr. Bernie D. Goldstein,
University of Pittsburgh, emphasized that a 20-year approach is realistic for this type of transformation
in the field of risk assessment to be fully realized. He noted that the idea of starting with disease and
working backwards is an enormous shift from the traditional approaches for toxicity testing and
regulating chemicals; creating this shift will take time. He suggested that for future meetings, it would
be beneficial to look at case studies that start with diseases (e.g., asthma, infertility) and work
backwards. This would help in developing a better molecular understanding of disease. He also
stressed that it is important to effectively communicate the reasons for improving approaches to risk
assessment. Regardless of the number of tests and assays available, there will always be uncertainties
associated with chemical risk assessment, which may result in chemicals with toxic effects entering into
commerce. For this reason, more effort should be directed towards approaches and tools that can
effectively manage the uncertainty aspects of risk assessment and fortify the decision making process.
Dr. Ken Ramos from the University of Louisville noted that this workshop is an effective step in the right
direction and is taking place at the right time. The debate is no longer whether risk assessment can be
advanced, but how risk assessment should be advanced. Going forward, he recommended asking more
targeted research and application questions to enable the development of tangible products in a shorter
timeframe. As new knowledge builds, risk assessors need new approaches for embracing and
capitalizing on that knowledge, while still remaining scientifically grounded in a process. He cautioned
that there is no need to reinvent the wheel; rather, it is important to look at existing resources and
10
-------
applications (e.g., the NRC reports Toxicity Testing in the 21st Century and Science and Decisions:
Advancing Risk Assessment) and ensure that they are being fully utilized.
Dr. Gary Ginsberg of the Connecticut Department of Public Health described some current limitations
associated with transitioning from the traditional animal based toxicity testing to a new generation
where toxicity testing data is derived from higher throughput systems. There are limitations in any
system in terms of uncovering how they function and respond across nodes within a cell, across cells
and tissues, and in regards to timing (e.g., periodicity, development). He noted that there is always the
question of whether we are testing the right things at the right time within a system to effectively
predict human biology; this question is even more relevant for in vitro systems as there is the potential
for issues with simulating dosimetry in culture versus in vivo systems, as well as metabolic differences,
and determining whether the cell types are biologically relevant (i.e., do they simulate human systems).
There is also a limitation in determining population risk using in vitro systems. He concluded by sharing
his thoughts on some immediate uses for NexGen data, which include conducting hazard identification
(e.g., chemical screening against well anchored prototypes), understanding mechanistic pathways and
responses (e.g., screening not only for toxicant fingerprints but also for various points along pathway),
screening for chemical-chemical interactions, developing biomarkers, and conducting contextual dose-
response modeling. A potential longer term use for NexGen data would be to conduct quantitative risk
assessments.
Dr. Martyn Smith of the University of California-Berkley, highlighted the importance of starting to
conceptualize how the emerging research will fit into the risk assessment process. He noted that a
critical aspect of advancing the NexGen concept and advancing risk assessment is building funding
capacity and providing opportunities for emerging scientists to further develop risk assessment
approaches and concepts. For example, resources are needed to develop robust assays; before moving
to high throughput testing, it is important to ensure that the assays that have already been developed
are relevant for low throughput testing.
Dr. Frederic Bois, Institut National de I'Environnement Industrie! et des Risques (INERIS), spoke about
the deluge of data resulting from the European Union's REACH (Registration, Evaluation, Authorisation
and Restriction of Chemical substances) regulation and how people are struggling with approaches to
expedite the registration process. He noted that one of the lessons learned from this registration
process is that since this is a multidisciplinary effort and, since people have different ways of
interpreting the same terms, there is a need to establish a common vocabulary that allows everyone to
communicate with each other. In terms of NexGen, he recommended developing guidance and a range
of agreed upon approaches that will enable interaction between prototypes. However, he cautioned
that care should be taken when extrapolating from prototypes to other chemicals; this will require
thinking about the process and prioritization of the process so that the outcomes from these prototypes
are more generically able to be applied. It is important to develop a continuum of chemical information
that allows for risk based decision making within a variety of contexts.
Overall, the panel felt that it is important to determine which methods could be currently utilized in risk
assessment, and to set priorities and target future research needs. Based on the four prototype
discussions and presentations described in Section 2.4, the panel and workshop participants identified
several cross-cutting research needs, which are presented in the following text box.
11
-------
I Days 1 & 2 Cross-Cutting Themes
> Develop specific plans for what molecular and systems biology approaches could
be utilized in the near term
> Start using approaches as they currently exist and recognize that these
approaches are iterative and will continue to develop in stages
o Feedback into the development process as we learn new information
o Refine initial data and approaches as needed
> Benchmark case studies against:
o Suite of tools that are currently available
o Risk assessment methodology criteria
o Public health population outcomes and other human data, as available
o NexGen framework
> Elucidate networks or related pathways that serve as signatures, i.e., we have
identified many key events/nodes in toxicity pathways for four prototypes, but
generally need to more broadly elucidate critical pathway(s)
o Mine the literature and molecular biology databases, and evaluate the
weight of evidence for various pathways
o Use data-rich chemicals to extrapolate mechanisms and responses for
data-poor chemicals when feasible
> Develop approaches that incorporate population variability and susceptibility
that are biologically relevant (e.g., occurring at environmentally relevant
concentrations)
> Obtain data on a series of standardized complex mixtures and develop
approaches for analysis
> Further explore dose-response information to develop more informative dose-
response curves
> Integrate and compare data sets (e.g., epidemiology, exposure, biomarker) in a
systems biology approach to develop integrated models of human risk
> Evaluate the consistency of data obtained from in vitro and in vivo approaches
across species
> Consider variabilities to the extent feasible, including assays, interspecies,
intraspecies (i.e., male/female/lifestage/physiological condition) exposure
scenarios, and progression of disease processes
> Identify: data gaps, opportunities for methods, model/assay refinement, and the
needs for additional reseach, articulate options from future applications of
molecular and systems biology to risk assessment
12
-------
4. Day 3 - Approaches for Chemicals with Less Data (Tier 2 Assessments)
Dr. Stan Barone, EPA, opened Day 3 of the workshop. The focus of which was whether high-throughput
screening assays can help solve the problem of the paucity of data for chemicals in the environment.
The goal is to design new approaches that will allow us to:
• Screen and rank thousands of chemicals for further evaluation rapidly and relatively cheaply.
• Identify potential adverse effects and relative potencies for specific effects for hundreds of
chemicals.
• Derive points of departure for many chemicals with limited data with the additional application
of reverse dosimetry.
• Provide information on mixtures interactions.
• Provide EPA program offices with a way to address the many chemicals for which there are no
or inadequate data, e.g., Office of Air and Radiation's National Air Toxic Assessment, urban air
sheds, and residual risk, Office of Water unregulated contaminants, Superfund chemicals, and
emergency urgent response (e.g., World Trade Center, Katrina, Gulf Oil spill).
Dr. Barone described how this collaborative effort involve the EPA Office of Research and Development
labs and centers, NIEHS, NTP, NCGC, and the Centers for Disease Control/Agency for Toxic Substance
and Disease Registry. The specific programs include, but are not limited to, ToxCast Phase I and II,
Tox21, and NexGen. Dr. Barone presented risk assessment questions that were provided to the Day 3
speakers and considered by the panelists later in the day.
Day 3 Risk Assessment Questions
Hazard Identification
> How can key information be rapidly and effectively summarized in an automated
fashion?
> What toxicity pathways are affected by the chemical(s) in question?
> What are the implications of pathway alteration for specific adverse effects?
> Can specific weight of evidence criteria for high throughput (HT)/high content (HC) assay
data be articulated that would indicate a known, likely or suggestive relationship
between chemical exposure and adverse effect?
Dose-Response
> How can relative potencies, and/or dose-response be estimated?
> Can upstream events that predict well characterized public health risks, based on
traditional data, be identified?
> How can recent scientific advances help describe adaptation, additivity to disease
background, & implications for low-response rates?
> How can recent scientific advances help describe probability of harm and uncertainty?
Both Hazard Identification and Dose-Response
> How can recent scientific advances help describe human variability and susceptible
subpopulations?
> How can recent scientific advances help describe the impacts of exposures to mixtures?
13
-------
4.1. Day 3 Presentations
Dr. Derek Knight, Approaches from the European Union: REACH
Dr. Derek Knight from the European Chemicals Agency (ECHA) presented an overview of ECHA's REACH,
Registration, Evaluation and Authorisation of CJHemicals Regulation.
Registration of substances is central to REACH and the evaluation, authorization, and restrictions
processes for chemicals rely on the registration data collected. There is a system of targeted
registration deadlines so that the highest priority chemicals will be registered first. Thus, the first phase
of registration includes high production and high volume chemicals (HPV), substances that are
carcinogenic, mutagenic and toxic to reproduction, and medium tonnage substances that are classified
as toxic to aquatic organisms and that may cause long-term adverse effects. ECHA expected that
approximately 4,500 substances would be registered by the November 30, 2010 deadline for Phase I
chemicals, but since multiple companies might submit dossiers for registration of a particular chemical,
the total number of dossiers could reach 30,000. EHCA estimates that companies will register an
additional couple of thousand substances in Phase II and several thousand more in Phase III. For
registration, ECHA requires standard data including chemical information, information on the use of the
chemical throughout its life cycle and potential exposures, and an assessment of the hazardous
properties of the substance at each stage of the life cycle. In addition, for chemicals produced in excess
of 10 tonnes per year, the registrant must submit a chemical safety report including a risk assessement
for hazardous substances. Registration for high and medium tonnage substances must include
proposals to fill data gaps for higher-tier toxicological and environmental studies, such as long term
mammalian toxicology studies as well as soil or sediment studies; however, ECHA emphasizes that
conducting new studies, particularly animal studies, should be a last resort. Instead, registrants should
consider using "non-standard data" including existing data, weight of evidence, QSAR, in vitro methods,
and chemical groupings and read-across approaches, which may provide adequate information and
hence be acceptable. While registrants are encouraged to consider all of these options, they must
provide robust scientific arguments to justify the use of non-standard data. In September 2010, ECHA
conducted an experts' workshop to discuss with experts the challenges and uncertainties related to
using non-test data in a regulatory context; this includes both the scientific uncertainty of using the data
and the uncertainty associated with applying that data in a risk management context.
ECHA expects that Phase I registration data will be available on a public Web site in early to mid 2011,
and then it will be possible to use this information for scientific purposes such as developing new QSAR.
Dr. Karen Leach, Approaches for Safety Assessment to Pharmaceuticals
Dr. Karen Leach from the Compound Safety Prediction group, which is part of the Medicinal Chemistry
Division of Pfizer Incorporated discussed the methods and applications of compound safety predictions
in the drug development process at Pfizer. In drug development, toxicity accounts for approximately
60% of drug attrition; that is, 60% of potential drugs are abandoned because of toxic effects discovered
during either preclinical or clinical Phase 1, 2 or 3 testing. When these potential toxic effects are
discovered earlier in the development process, the overall cost of developing a drug can decrease since
money and time are no longer invested in drugs that are later found to be toxic. Previously, drug
development focused first on pharmacology then on pharmacokinetics and lastly on safety. This
sequential approach is being abandoned in favor of assessing all three simultaneously primarily through
predictive assays that are based on pathway knowledge, computational analysis, and in silico models.
Dr. Leach's department at Pfizer develops predictive screening tests that can help inform the drug
design process. Adverse safety events resulting from compound treatment can be the result of the
primary pharmacology, the chemical structure of the compound, its reactive metabolites, the
14
-------
physicochemical properties of the compound, and the off-target or secondary pharmacology effects.
Using a combination of in vitro assays (e.g., genetic toxicity assays such as Ames test, cell viability assays,
mitochondrial toxicity assays), researchers will be able to make more informed predictions of in vivo
outcomes. Recently, scientists at Pfizer physical-chemical property associations and created a map of
physical properties such as molecular weight and polar surface area for central nervous system drugs
that are currently on the market. By comparing the physical-chemical properties of a potential drug to
those of existing drugs, one can produce a probability-type estimate about chemical safety, rather than
a simple binary, yes or no prediction of chemical safety. Dr. Leach emphasized that the challenges in
developing predictive assays provides an opportunity for scientists from industry, academia, and
regulatory groups to collaborate when investigating toxicity mechanisms, identifying acceptable
biomarkers and high throughput screening applications, and determining how this data can be
combined to support decisions.
Dr. Michael DeVito, ToxZl Targeted Testing
Dr. Michael DeVito, NIEHS's NTP, presented an overview of Tox21, a collaborative partnership between
the NTP, NGCG, EPA, and U.S. Food and Drug Administration (FDA). The goal of Tox21 is to pool
resources and expertise towards developing predictive toxicity models and high throughput screening
assays based on mechanisms of chemically-induced biological activity to prioritize chemicals for more
extensive toxicological evaluation, and to develop toxicity data that can be used to support risk
management decisions. Dr. DeVito presented a brief overview of a targeted testing study that evaluated
a predictive model for non-genotoxic liver carcinogens. The study was designed to answer questions
such as "Do human assays predict rodent in vivo results?", "What is the impact of metabolism?", and "Is
a hit in vitro really a hit in vivo?" Tier 1 testing will begin with 30 chemicals known to either cause liver
tumors in Sprague-Dawley rats or not cause liver tumors; the animals will be dosed once daily for two
years, and in vitro assay signatures will be compared to biomarkers measured in the animals.
Dr. DeVito also presented an example of the challenges of quantitatively extrapolating from in vitro data
to predict in vivo responses. Using a toxicokinetic model, Dr. DeVito and his team determined how well
in vitro data on the metabolism and distribution of deltamethrin derived in cells predict
pharmacokinetic data for deltamethrin derived from in vivo studies. The team also had in vivo data on
changes in motor activity in rats. The results included accurate predictions of in vivo blood
concentrations, but in vivo brain tissue concentrations were not accurately predicted. Dr. DeVito
presented several caveats regarding interpretation of these data. For example, further study is needed
on the relationship between in vitro cell exposures and chemical concentrations and interactions in the
media, and how best to model this relationship. Similar approaches will be evaluated in the Liver
Targeted Testing study; the details of that study were presented later in Day 3 by Dr. Richard Judson.
In conclusion, the ongoing Tox21 efforts are providing insight into the capabilities and uncertainties of
extrapolating HTS data to predict in vivo biological responses.
Dr. Christopher Portier, Genetic and Genomic Risk Assessment for Identifying Hazards
Dr. Christopher Portier, Centers for Disease Control, National Center for Environmental Health/Agency
for Toxic Substances and Disease Registry, discussed the importance of linking genomics to pathway
perturbations to guide prioritization of chemicals for high-throughput screening. He stressed the need
for using a systems biology approach, in which data from human clinical laboratory, epidemiology,
animal model, tissue culture, cell culture, and molecular biology studies are used in conjunction to
predict human health risk. It is also important to take into account how humans interact with their
environment as well as other human characteristics (e.g., nutrition, socioeconomic status). Once human
disease pathways are identified, genomic signatures of chemicals can be linked to these pathways to
15
-------
predict risk from chemical exposure. Determining the gene target for pathways often linked to human
diseases can guide chemical prioritization for high-throughput screening. The NIH Genetic Association
Database and the Comparative Toxciogenomics Database contain gene-centered data, but Dr. Portier
emphasized the growing need for genome-wide association studies.
Audience participants pointed out that a method must be developed to screen out and prioritize which
genes are really linked to disease. Dr. Portier responded that the development of such a method is
beyond the scope of current research but must be addressed in the future. An audience participant
highlighted the opportunity to use genome sequencing previously conducted for certain cancers as a
tool to derive pathways. Another audience participant suggested using multiple strains rather than a
single strain to identify pathways. Dr. Portier agreed, arguing that once the pathways in the mouse are
identified, this information can be used to extrapolate to humans.
Dr. Alexander Tropsha, Combined Application of Chemical and Molecular Biology
Information
The presentation by Dr. Alexander Tropsha, University of North Carolina-Chapel Hill, focused on the
combined application of cheminformatics and high-throughput screening data to improve chemical
safety assessment. Dr. Tropsha expressed the need to begin building multiple quantitative structure-
activity relationship (QSAR) models as a virtual screen database of chemicals that allows for
prioritization of chemicals for high-throughput screening based on toxicity. QSAR modeling uses
statistical techniques to relate a characterized chemical structure to biological data. Dr. Tropsha pointed
out that this process is only effective when chemical structure knowledge and biological response data
are error free, prompting the need for thorough curation of existing chemical and biological data. High-
throughput screening in vitro data or chemical descriptors alone do not predict in vivo results as well as
the combination of those data. Dr. Tropsha emphasized that in vitro data, especially concentration-
response high-throughput screening profiles, can improve the results of the QSAR modeling of in vivo
endpoints. Concentration-response biological high-throughput screening descriptors further enhance
the accuracy of the models. Dr. Tropsha concluded that any developments in combining
cheminformatics and high-throughput screening data should be made publically available.
Dr. David Reif, A Toxicological Priority Index (ToxPi) for Prioritizing Chemicals based on the
ToxCast Data
Dr. David Reif, EPA, presented an overview of Toxicological Priority Index (ToxPi), a flexible prioritization
support software tool that incorporates ToxCast bioactivity profiles, inferred toxicity pathways, dose
estimates, and chemical structural descriptors. Instead of developing an absolute threshold, ToxPi
consists of a numerical index that is more flexible for different prioritization tasks and can better
accommodate new data, chemicals, and data adjustments. The data can be sorted according to
pathway, and combined with additional information such as genetic susceptibility factors and chemical-
specific factors. Dr. Reif described efforts to examine confidence in ranking, alternative chemical sets,
and treatment of missing data to better communicate uncertainties and increase transparency in
decision making.
Dr. Richard Judson, A Framework for High-Throughput Risk Assessment
Dr. Richard Judson, EPA, discussed the benefits of using high throughput risk assessment (HTRA) to
inform decisions regarding health protective exposure levels for chemicals. HTRA aims to use in vitro
data to estimate the dose at which a given pathway is perturbed in vivo, and it may potentially be used
to evaluate hundreds to thousands of chemicals with little to no in vivo data. Presently, he proposed
focusing on Tier 1 chemicals with molecular pathways and targets where existing data suggest a link
16
-------
between perturbation and signs of adversity. These results would then be used to prioritize chemicals
for inclusion in Tier 2 and Tier 3 assessments.
He presented five key ideas that comprise the HTRA approach, which include defining biological
pathways whose alteration can lead to adverse outcomes, developing in vitro assays that measure
chemical activity in biological pathways, determining the in vitro concentration required to alter a
pathway (i.e., the biological pathways altering concentration [BPAC]), estimating the oral dose required
to reach the BPAC (i.e., the biological pathway altering dose [BPAD]), and incorporating variability and
uncertainty. The BPAD required to reach the BPAC is determined using the Reverse Toxicokinetics (RTK)
approach. RTK yields a concentration at steady state based on human plasma protein binding data that
accounts for population variability. Uncertainty is incorporated by taking the 95% bound on the lower
99% tail of the BPAD, resulting in a more protective lower bound. As assays are developed to measure
chemical activity in biological pathways, he emphasized the power of this approach to quantitatively
predict in vivo human responses from different target sites and pathways.
Audience participants suggested additional potential uses for the HTRA approach. For example, one
participant pointed out that a tremendous amount of pharmacodynamic data exist, presenting an
opportunity to interface this approach with in vivo data. Another observer suggested taking a relative
potency approach for cases where you have a lot of uncertainty (e.g., Tier 2). The presenter explained
that there is not enough data to build models in these instances, but such an approach would be
qualitatively informative. A third person stressed the need to take into account gender differences, but
the presenter asserted that such details are beyond the scope of the approach at the present time, and
that the outlined approach is only the first step in the process of developing HTRA.
Dr. Russell Thomas, Can Genomics Be Used to Derive a Meaningful Points-of-Departure for
Cancer and Noncancer Risk Assessment?
The final presentation by Dr. Russell Thomas from the Hamner Institutes for Health Sciences focused on
incorporating genomics into Tier 2 risk assessments. He reviewed how the field of genomics has
matured over the past decade and indicated that numerous studies have demonstrated the sensitivity
and reproducibility of current gene expression microarray technology. Genomics has the potential to be
useful in risk assessment because it can provide quantitative information on the dose at which cellular
processes are affected and the underlying biology of dose-dependent transitions. It also has the
potential to increase efficiency, reduce animal numbers, and cut costs associated with chemical risk
assessment.
For example, to determine how relatively short-term genomic changes correlate with apical toxicity
endpoints as a function of dose, he exposed whole animals to chemicals and measured transcriptional
changes in selected target tissues using microarrays. Each gene was then fit with a statistical model, a
benchmark dose was calculated, genes were grouped by cellular function (e.g., proliferation, apoptosis),
and a dose at which cellular function was perturbed was estimated. He then identified points of
departure and used these values to estimate provisional references doses (RfDs) or cancer slope factors.
To further demonstrate the promise of this approach, he presented the results from 90-day exposure
studies using five chemicals that had evidence of tumor development when tested by NTP. For the most
sensitive gene ontology (GO) category, he reported slightly more sensitive results, suggesting that there
is good correlation between transcriptomic dose response alterations and both noncancer- and cancer-
related apical endpoints. While this approach is promising, more work needs to be conducted in the
interpretation and use of genomic data for mode-of-action risk assessment. As a result, future work
includes evaluating the use of genomics in Tier 2 risk assessment over a three-year period in conjunction
with EPA's Office of Research and Development.
17
-------
4.2.
A panel convened on Day 3 which included the speakers and was moderated by Drs. Stan Barone and
David Dix, EPA. The panel considered each others' presentations and the risk assessment questions
followed by an open discussion among all of the workshop participants. Discussion points are
summarized in the following paragraphs.
It was recognized that the focus of Tier 2 assessments to date has been on toxicity assessment and
exposure is not addressed. A lot is unknown about how the chemical gets to the target tissues and gets
into the cell and modulates the cell activity. Pharmacokinetics could be added as a factor for the
screening approach. Tier 1 ADME will be in silico and in vitro.
There was agreement that single predictive values are inadequate and that we need to look deeper to
understand the shape of the dose-response curve. It is important to understand where the exposure is
on the dose-response curve. It was asked if there were plans to compare dose-response curves between
in vivo and in vitro. Dr. Judson remarked that indeed they are starting this. They are comparing oral
equivalents of bioactivity with estimated human exposure, which provides context to get at bioactivity.
Three different approaches were suggested for semi-quantitiative estimates. One was to take the
median transcriptional BMDL and develop an ordinal scale of severity.
It was asked whether we have progressed for being able to screen for neurotoxicity, immunotoxitiy,
reproductive toxicity, and endocrine disruption. Dr. Leach replied that there is work going on in these
areas, especially immunotoxicity and reproductive toxicity. The least work is being done on
neurotoxicity. Cardiotoxicity is also challenging. Liver and cardiotoxicity are major challenges for the
pharmaceutical industry. There is a group in EPA ORD looking at cardiotoxins.
There was discussion about where the different studies fit on the continuum between Tier 1, Tier 2, and
Tier 3 assessments and how much information is needed at each Tier. Dr. Cote clarified that Tier 1 is
exclusively high throughput testing based. It was noted that decision-makers and stakeholders need
information quickly and in a format that they can use. How can the information be integrated into the
decision-making process earlier? The need for stakeholder engagements (e.g., with States and EPA
Regions) was acknowledged.
Following the open discussion, Dr. Dix summarized key points from the discussions. He projected Figure
2 which was refined by the workshop participants. It is a preliminary proposal for what delineates Tier 1
from Tier 2.
Key points from the Day 3 discussion are provided in the text boxes on page 19 and 20.
18
-------
NTS
QSAR
ADME
Disease
pathways
ToxPi
HTRA/BPAD
In vitro organotypicmodels
Alternative species
Short-term/targeted rodent assays
In silico human tissues
Exposure information
Tierl
Tier 2
Figure 2. The NexGen Tier 1 and 2 continuum with a vertical dashed line distinguishing between data
elements and approaches relevant to these tiers. Tier 1 data elements of High-throughput Screening
(HTS), Quantitative Structure Activity Relationships (QSAR), Absorption-Distribution-Metabolism-
Elimination (ADME), and disease pathways inform a Toxicological Prioritization index (ToxPi) for ranking
chemicals for Tier 2 evaluation. Tier 2 High-Throughput Risk Assessments (HTRA) yielding Biological
Pathway Altering Doses (BPAD)would benefit from addition data from in vitro organotypic models,
alternative non-mammalian species, short-term and targeted rodent assays, and in silico human tissue
models. Exposure information would be important for both Tier 1 and 2 assessments.
Day 3 Key Points
> With respect to in vitro to in vivo correlation, what are we benchmarking high-
throughput screening data against (i.e., human disease interactomes or in vivo data
from experimental animal data)?
> Grouping criteria/metrics for chemical mixtures are needed.
> There is uncertainty in the biological data that are in-hand. There may be even more
uncertainty in how the information is subsequently employed in decision making/risk
management.
> There is a need to refine a tiered framework for Tier 2 assessments.
> Optimization analysis (i.e., inclusion of assays that address, in part, both kinetics and
dynamics) should be considered.
> Decision makers are faced with a variety of situations to be addressed, e.g. ranking
chemicals for futher research to regulatory decision making. NexGen-type
approaches/data will vary depending on the type of situation to be addressed.
(Continued on next page)
19
-------
Day 3 Key Points (Continued)
> Variabilities should be considered to the extent feasible, including assays, interspecies,
intraspecies (i.e., male/female/lifestage/physiological condition) exposure scenarios, and
progression of disease processes.
> We have been focused primarily on hazard identification and dose-response assessment.
Exposure assessment is a critical piece and needs consideration.
> There is a need to investigate approaches to correlating biological perturbations with the
incidence/severity of apical events both as a function of dose-response and duration of
exposure (e.g., an extension of Dr. Thomas' approach). For example, transcriptomic
changes at a BMR of X correlates with apical phenotype/condition Y.
> How might screening/prioritization feed into a more risk assessment centric outcome
(e.g., quantitative dose-response data, identification of the point-of-departure)?
5. References
Krewski, D., Hogan, V., Turner, M.C., Zeman, P.L., McDowell, I., Edwards, N., and Losos, J. (2007). An
integrated framework for risk management and population health. Human and Ecologicial Risk
Assessment 13: 1288-1312.
National Research Council (NRC). 2009. Science and Decisions: Advancing Risk Assessment. The National
Academies Press, Washington, DC.
National Research Council (NRC). 2007. Toxicity Testing in the 21st Century: A Vision and a Strategy. The
National Academies Press, Washington, DC.
National Research Council (NRC). 1983. Risk Assessment in the Federal Government: Managing the
Process. The National Academies Press, Washington, DC.
20
-------
Appendix A. Final Agenda: Advancing the Next Generation (NexGen) of
Risk Assessment: The Prototypes Workshop
Day 1 - November 1, 2010 - EPA Campus at Research Triangle Park
Data Rich Prototypes (Tier 3 Assessments)
Morning Plenary Session - EPA Conference Room Clll A/B
EPA Security Check-in and Registration - Directions provided in General Travel and
8:00-8:50
Workshop Information document.
9:00-9:20 Welcome and Introduction - Origin of effort, goals and objectives, structure for this
workshop- Dr. lla Cote, U.S. EPA
9:20-10:30 Framework for Prototype Development - Presentation (30 minutes); Q&A and
facilitated discussion (30 minutes) - Dr. Daniel Krewski, University of Ottawa/Risk
Sciences International
10:30-10:45 Break (EPA's Lakeside Cafe will be open for beverage purchases)
10:45-11:15 NexGen Risk Assessment Issues- Presentation (20 minutes) and Q&A (10 minutes)
- Dr. Weihsueh Chiu, U.S. EPA
11:15-11:45 General Charge to Breakout Groups - Presentation (20 min) and Q&A (10 min) -
Dr. lla Cote, U.S. EPA
(1) Lung Injury-Ozone
(2) Developmental Impairment-Thyroid Hormone Disrupters
(3a) Cancer-Benzene
(3b) Cancer - Polycyclic Aromatic Hydrocarbons (PAHs)
11:45-1:00 Lunch (EPA's Lakeside Cafe, on your own)
Afternoon Breakout Sessions
1:00-3:15 Breakout Sessions
• Prototype overview
presentations by
Prototype Team Leads
• Discussion
Breakout Conference Rooms
(1) Lung Injury-Ozone: Room C111A
(2) Developmental Impairment-Thyroid
Hormone Disrupters: Room C111B
(3a) Cancer- Benzene: Room C112
(3b) Cancer- PAHs: Room C113
3:15-3:30 Break (Lakeside Cafe will be open for beverage purchases)
3:30-5:00 Breakout Sessions - Continued discussion
A-l
-------
Day 2 - November 2, 2010 - EPA Campus at Research Triangle Park
Data Rich Prototypes (continued)
Morning Breakout Sessions
7:30-8:30 EPA Security Check-in
8:30-10:00 Breakout Sessions - (Same conference rooms as Day 1)
• Conclude discussion
• Summarize
10:00-10:30 Break (Lakeside Cafe will be open for beverage purchases)
'ay 2 Plenary Session - EPA Conference Room Clll A/B
10:30-12:00 Breakout Group Report by Prototype Chairs (each 30 minute presentation &
15 minute Q&A)
(1) Lung Injury-Ozone
(2) Developmental Impairment-Thyroid Hormone Disrupters
12:00-1:00 Lunch (Lakeside Cafe, on your own)
1:00-2:30 Breakout Group Reports by Prototype Chairs (continued)
(3a) Cancer - Benzene
(3b) Cancer - PAHs
2:30-3:15 Panel Discussion - Moderated by Dr. Lauren Zeise, California EPA
Dr. Gary Ginsberg—Connecticut Department of Public Health
Dr. Bernard Goldstein—University of Pittsburg
Dr. Daniel Krewski—University of Ottawa
Dr. Ken Ramos—University of Louisville
Dr. Martyn Smith—University California—Berkeley
3:15-3:45 Break (Lakeside Cafe will be open for beverage purchases)
3:45-4:30 Panel Discussion (continued) - Discussion and Q&A
4:30-5:15 Refinement of Cross-Cutting Key Points - Moderated by Dr. Rob DeWoskin,
U.S. EPA
5:15-5:30 Next Steps, Discussion of 3rd Day, and Close - Dr. lla Cote, U.S. EPA
A-2
-------
Day 3 - November 3, 2010 - Hilton Raleigh-Durham Airport at
Research Triangle Park, Grand Ballroom
Approaches for Chemical with Less Data (Tier 2 Assessments)
Dr. Stan Barone and Dr. David Dix, U.S. EPA - Co-chairs
7:30-8:30 Workshop registration at the Hilton (outside of Grand Ballroom)
8:30-8:45 Introduction - What is Tier 2 and what questions are we trying to address?
- Dr. Stan Barone, U.S. EPA
• In a risk assessment context, how is Tier 2 different from Tier 3? What is being
learned from the Tier 3 prototypes about proof of concept, value of
information, and decision rule that can inform Tier 2?
• Can the Tier 2 approach be applied to 100 to 1000s of chemicals per year?
• Is the Tier 2 approach acceptable for selected regulatory and policy decisions
even though it provides less WOE than Tier 3?
8:45-10:15 Speakers - 20 minute presentations, 10 minute Q&A
Example Questions Posed to Each Speaker—Moderated by Dr. Stan Barone, U.S.
EPA
• What kind of information does this approach provide about potential adverse
effects when combined with in vivo data and in the absence of in vivo data?
• How could these data inform potency estimates or dose-response
relationships?
• How qualitatively or quantitatively predictive is the data of in vivo human
responses the information generated from this approach?
• How could this information be use in a weight of evidence scheme?
• Can this approach inform us about:
• Variability and susceptibility in the human population?
• Mixtures interactions?
• What are the strengths and weaknesses of this approach for assessing risks in
the human population?
• Are there other approaches that you are aware of that might provide similar
or improved information on these topics?
A-3
-------
Day 3 - November 3, 2010 (continued)
8:45-10:15 Speakers-
• Approaches from the European Union: REACH - Dr. Derek Knight, European
Chemicals Agency
• Approaches for Safety Assessment to Pharmaceuticals - Dr. Karen Leach,
Pfizer
• Tox21 Approaches -Dr. Michael DeVito, National Institute of Environmental
Health Sciences
10:15-10:30 Break
10:30-12:00 Speakers (continued)
• Genetic and Genomic Risk Assessment for Identifying Hazards—Dr.
Christopher Portier, CDC-NCEH/ATSDR
• Combined Application of Chemical and Molecular Biology Information -
Dr. Alexander Tropsha, University of North Carolina-Chapel Hill
• AToxicological Priority Index (ToxPi) for Prioritizing Chemicals based on the
ToxCast Data - Dr. David Reif, U.S. EPA
• First Steps Towards High Throughput Risk Assessment (HTRA) -
Dr. Richard Judson, U.S. EPA
12:00-1:15 Lunch (Hotel restaurant on your own; buffet with beverage available for $ll+tax)
1:15-1:45 Speakers (continued)
• Can Genomic Data be Used to Derive Meaningful Points-of-Departure for
Cancer and Noncancer Risk Assessment? - Dr. Rusty Thomas, The Hamner
Institutes for Health Sciences
1:45-3:00 Panel and Open Discussion
3:00-3:10 Development of Key Meeting Observations - Facilitated by Dr. Jason Lambert and
Dr. lla Cote, U.S. EPA
3:10-3:30 Break
3:30-5:00 Development of Key Meeting Observations (continued)
• Integrated toolbox approaches targeted for further development in health
assessment applications
• Weight of evidence issues
• Needed next steps
5:00-5:15 Next Steps and Close - Dr. Stan Barone and Dr. David Dix, U.S. EPA
A-4
-------
Appendix B. Participants in the Advancing the Next Generation (NexGen)
of Risk Assessment: The Prototypes Workshop
First Name
Scott
Tina
William
Stan
Souad
Don
Cathy
Fredric
Philip
James
Barbara
Lyle
David
Heather
Chao
Weihsueh
Becki
Elaine
Rory
Las, Name
Auerbach
Bahadori
Baird
Barone
Benromdhane
Bergfelt
Blake
Bois
Bromberg
Brown
Buckley
Burgoon
Bussard
Carlson-Lynch
Chen
Chiu
Clark
Cohen-Hubal
Conolly
Affiliation
National Institutes of
Environmental Health Sciences
American Chemistry Council
Oregon State University
EPA National Center for
Environmental Assessment
EPA Office of Air Quality
Planning and Standards
EPA National Center for
Environmental Assessment
University of Illinois, Urbana-
Champaign
Institut National de
I'Environnement Industrie! et
des Risques
University of North Carolina-
Chapel Hill
EPA National Center for
Environmental Assessment
EPA National Center for
Environmental Assessment
EPA National Health and
Environmental Effects Research
Laboratory
EPA National Center for
Environmental Assessment
Syracuse Research Corporation
EPA National Center for
Environmental Assessment
EPA National Center for
Environmental Assessment
EPA National Center for
Environmental Assessment
EPA National Center for
Computational Toxicology
EPA National Center for
Computational Toxicology
7™;;"
auerbachs@niehs.nih.gov
tina_bahadori@americanchemi
stry.com
william.baird@orst.edu
barone.stan@epa.gov
benromdhane.souad@epa.gov
bergfelt.don@epa.gov
clblake@illinois.edu
frederic.bois@ineris.fr
pwspar@med.unc.edu
brown.james@epa.gov
buckley.barbara@epa.gov
burgoon.lyle@epa.gov
bussard.david@epa.gov
hclynch@srcinc.com
chen.chao@epa.gov
chiu.weihsueh@epa.gov
clark.becki@epa.gov
hubal.elaine@epa.gov
conolly.rory@epa.gov
n/a
Thyroid
PAHs
Thyroid
Ozone
Thyroid
PAHs
Benzene
Ozone
Ozone
Ozone
Benzene
n/a
PAHs
n/a
Ozone
n/a
PAHs
Ozone
B-l
-------
FirstNan-e LastNan-e
Dan
Ha
Kevin
Maxine
Sally
David
Mike
Robert
Robert
David
Sanjivani
David
David
Stephen
Nicole
Hisham
Lynn
Brenda
Stiven
Bruce
Costa
Cote
Crofton
Croteau
Darney
DeMarini
DeVito
Devlin
DeWoskin
Diaz-Sanchez
Diwan
Dix
Eastmond
Edwards
Edwards
EI-Masri
Flowers
Foos
Foster
Fowler
Afli,ia,,on
EPA Office of Research
Development
EPA National Center for
Environmental Assessment
EPA Office of Research
Development
Risk Sciences International
EPA Office of Research
Development
EPA National Health and
Environmental Effects Research
Laboratory
National Institute of
Environmental Health Sciences
EPA National Health and
Environmental Effects Research
Laboratory
EPA National Center for
Environmental Assessment
EPA National Health and
Environmental Effects Research
Laboratory
EPA National Center for
Environmental Assessment
EPA National Center for
Computational Toxicology
University of California -
Riverside
EPA National Health and
Environmental Effects Research
Laboratory
EPA National Center for
Environmental Assessment
EPA Office of Research
Development
EPA National Center for
Environmental Assessment
EPA Office of Children's Health
Protection
EPA Office of Solid Waste and
Emergency Response
CDC Agency for Toxic Substances
and Disease Registry
^^M^^^^I^^M
costa.dan@epa.gov
Cote.lla@epamail.epa.gov
crofton.kevin@epa.gov
mcroteau@uottawa.ca
darney.sally@epa.gov
demarini.david@epa.gov
devitom@niehs.nih.gov
devlin.robert@epa.gov
dewoskin.rob@epa.gov
diaz-sanchez.david@epa.gov
diwan.sanjivani@epa.gov
dix.david@epa.gov
david.eastmond@ucr.edu
edwards.stephen@epa.gov
edwards.nicole@epa.gov
el-masri.hisham@epa.gov
flowers.lynn@epa.gov
foos.brenda@epa.gov
foster.stiven@epa.gov
bxf9@cdc.gov
Ozone
n/a
Thyroid
PAHs
Thyroid
n/a
Thyroid
Ozone
Thyroid
Ozone
Thyroid
Thyroid
Benzene
Ozone
n/a
Ozone
PAHs
Thyroid
Benzene
Benzene
B-2
-------
FirstNan-e LastNan-e
Mark
John (Jef)
Rebecca
Annette
Mary
Gary
Helen
Michael
(Rocky)
Bernard D.
Ami
Kate
Maureen
Gary
Dale
Andrew
Annie
Melanie
Jennifer
Ryan
Richard
Ramzi
Ray
Channa
Frampton
French
Fry
Gatchett
Gilbert
Ginsberg
Goeden
Goldsmith
Goldstein
Gordon
Guyton
Gwinn
Hatch
Hattis
Hotchkiss
Jarabek
Jardim
Jinot
Jones
Judson
Kafoury
Kent
Keshava
Afli,ia,,on
University of Rochester
National Institute of
Environmental Health Sciences
University of North Carolina-
Chapel Hill
EPA National Center for
Environmental Assessment
EPA Office of Research and
Development
Connecticut Department of
Public Health
Minnesota Department of
Health
EPA National Exposure Research
Laboratory
University of Pittsburgh
ICF International
EPA National Center for
Environmental Assessment
EPA National Center for
Environmental Assessment
EPA Off ice of Research
Development
Clark University
EPA National Center for
Environmental Assessment
EPA National Center for
Environmental Assessment
EPA Office of Research
Development
EPA National Center for
Environmental Assessment
EPA National Center for
Environmental Assessment
EPA National Center for
Computational Toxicology
Jackson State University
EPA Office of Pesticide Programs
EPA National Center for
Environmental Assessment
^^M^^^^I^^M
mark_frampton@urmc.rochest
er.edu
french@niehs.nih.gov
rfry@email.unc.edu
gatchett.annette@epa.gov
gilbert.mary@epa.gov
gary.ginsberg@po. state. ct. us
helen.goeden@state.mn.us
goldsmith.rocky@epa.gov
bdgold@pitt.edu
agordon@icfi.com
guyton.kate@epa.gov
gwinn.maureen@epa.gov
hatch.gary@epa.gov
dhattis@aol.com
hotchkiss.andrew@epa.gov
jarabek.annie@epa.gov
jardim.melanie@epa.gov
jinot.jennifer@epa.gov
jones.ryan@epa.gov
judson.richard@epa.gov
ramzikafoury@gmail.com
kent.ray@epa.gov
keshava.channa@epa.gov
Ozone
Benzene
Ozone
n/a
Thyroid
Benzene
Benzene
PAHs
Benzene
Thyroid
Benzene
Benzene
Ozone
Thyroid
n/a
Ozone
Ozone
Benzene
Ozone
Thyroid
Ozone
n/a
PAHs
B-3
-------
FirstNan-e LastNan-e
Nagu
Whitney
Andrew
Derek
Thomas
Daniel
Jason
Karen
Janice
William
George
Susan
Amalia
Kristen
Matt
Peter
William
David
Fred
Mark
Holly
Stephen
Jennifer
Kim
Keshava
Kihlstrom
Kligerman
Knight
Knudsen
Krewski
Lambert
Leach
Lee
LeFew
Leikauf
Makris
Marenberg
Marin
Martin
McClure
McDonnell
Miller
Miller
Miller
Mortensen
Nesnow
Orme-Zavaleta
Osborn
Afli,ia,,on
EPA Office of Research
Development
ICF International
EPA National Health and
Environmental Effects Research
Laboratory
European Commission, Joint
Research Centre
EPA Office of Research
Development
University of Ottawa
EPA National Center for
Environmental Assessment
Pfizer
EPA National Health and
Environmental Effects Research
Laboratory
EPA Office of Research
Development
University of Pittsburgh
EPA National Center for
Environmental Assessment
ICF International
ICF International
EPA National Center for
Computational Toxicology
Syracuse Research Corporation
Independent Consultant
EPA National Center for
Environmental Assessment
Independent Consultant
California EPA
EPA Office of Research
Development
EPA National Health and
Environmental Effects Research
Laboratory
EPA Office of Research and
Development
ICF International
^^M^^^^I^^M
keshava.nagu@epa.gov
wkihlstrom@icfi.com
kligerman.andrew@epa.gov
Derek.knight@echa.europa.eu
knudsen.thomas@epa.gov
dkrewski@uottawa.ca
lambert.jason@epa.gov
karen.l.leach@pfizer.com
lee.janices@epa.gov
lefew.william@epa.gov
gleikauf@pitt.edu
makris.susan@epa.gov
amarenberg@icfi.com
kmarin@icfi.com
martin.matt@epa.gov
mcclure@srcinc.com
Mcdonnell.william@earthlink.n
et
miller.david@epamail.epa.gov
fjmiller@nc.rr.com
mmiller@oehha.ca.gov
mortensen.holly@epa.gov
nesnow.stephen@epa.gov
orme-
zavaleta.jennifer@epa.gov
kosborn@icfi.com
n/a
Thyroid
n/a
n/a
Thyroid
PAHs
Thyroid
n/a
n/a
Ozone
Ozone
Thyroid
Ozone
Benzene
Thyroid
PAHs
Ozone
Ozone
Ozone
Thyroid
Thyroid
PAHs
n/a
Benzene
B-4
-------
FirstNan-e LastNan-e
Greg
Fred
David
Chris
Kathleen
Ken
David
Karen
Craig
Ivan
James
Martha
Deborah
Mary Jane
Woodrow
Imran
Steve
Martyn
Bob
Julie
Cecilia
Russell
(Rusty)
Ray
Paoli
Parham
Phillips
Portier
Raffaele
Ramos
Reif
Rowland Yeo
Rowlands
Rusyn
Samet
Sandy
Segal
Selgrade
Setzer
Shah
Simmons
Smith
Sonawane
Stickney
Tan
Thomas
Tice
Afli,ia,,on
Risk Sciences International
National Institute of
Environmental Health Sciences
Institute of Cancer Research
National Center for
Environmental Health/ Agency
for Toxic Substances and Disease
Registry
EPA National Center for
Environmental Assessment
University of Louisville
EPA National Center for
Computational Toxicology
Simcyp
The Dow Chemical Company
University of North Carolina-
Chapel Hill
EPA National Health and
Environmental Effects Research
Laboratory
California EPA
EPA National Center for
Environmental Assessment
ICF International
EPA National Center for
Computational Toxicology
EPA National Center for
Computational Toxicology
EPA National Health and
Environmental Effects Research
Laboratory
University of California- Berkeley
EPA National Center for
Environmental Assessment
Syracuse Research Corporation
EPA National Exposure Research
Laboratory
The Hamner Institutes for Health
Sciences
National Institutes of
Environmental Health Sciences
^^M^^^^I^^M
gpaoli@RiskScienceslnt.com
parham@niehs.nih.gov
david.phillips@icr.ac.uk
cportier@cdc.gov
raffaele.kathleen@epamail.epa
.gov
ksramo01@louisville.edu
reif.david@epamail.epa.gov
k.r.yeo@simcyp.com
jcrowlands@dow.com
iir@unc.edu
samet.james@epa.gov
msandy@oehha.ca.gov
segal.deborah@epa.gov
mselgrade@icfi.com
setzer.woodrow@epa.gov
shah.imran@epamail.epa.gov
simmons.steve@epa.gov
martynts@berkeley.edu
sonawane.bob@epa.gov
stickney@srcinc.com
tan.cecilia@epa.gov
RTHOMAS@thehamner.org
tice@niehs.nih.gov
PAHs
Thyroid
PAHs
Ozone
Thyroid
PAHs
Thyroid
Thyroid
Thyroid
Thyroid
Ozone
Benzene
Thyroid
n/a
Thyroid
PAHs
n/a
Benzene
Benzene
PAHs
Benzene
PAHs
Benzene
B-5
-------
FirstNan-e LastNan-e
Alex
Audrey
John
John
Nina
Amy
Katrina
Scott
Margit
Ronald
Darrell
Doug
Lauren
LuoPing
Jay
Tropsha
Turley
Vandenberg
Wambaugh
Wang
Wang
Waters
Wesselkamper
Westphal
White
Winner
Young
Zeise
Zhang
Zhao
Afli,ia,,on
University of North Carolina-
Chapel Hill
ICF International
EPA National Center for
Environmental Assessment
EPA National Center for
Computational Toxicology
EPA National Center for
Environmental Assessment
EPA National Center for
Computational Toxicology
Pacific Northwest National
Laboratory
EPA National Center for
Environmental Assessment
Risk Sciences International
Johns Hopkins Bloomberg School
of Public Health
EPA National Center for
Environmental Assessment
EPA Office of Research
Development
California EPA/ Office of
Environmental Health Hazard
Assessment
University of California -
Berkeley
EPA National Center for
Environmental Assessment
^^M^^^^I^^M
alex_tropsha@unc.edu
aturley@icfi.com
vandenberg.john@epa.gov
wambaugh.john@epa.gov
wang.nina@epa.gov
wang.amy@epa.gov
katrina.waters@pnl.gov
wesselkamper.scott@epa.gov
m west020@ uotta wa . ca
rwhite@jhsph.edu
winner.darrell@epa.gov
young.douglas@epa.gov
lzeise@oehha.ca.gov
luoping@berkeley.edu
zhao.jay@epa.gov
PAHs
Ozone
Ozone
Benzene
Benzene
n/a
Benzene
PAHs
PAHs
Ozone
n/a
n/a
Thyroid
Benzene
PAHs
B-6
------- |