UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
May 22, 2014
EPA-SAB-14-006
The Honorable Gina McCarthy
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: SAB Advice on Advancing the Application of CompTox Research for EPA Chemical
Assessments
Dear Administrator McCarthy:
Risk assessment is central to the mission of the U.S. Environmental Protection Agency (EPA), and the
Science Advisory Board (SAB), as well as the National Academy of Sciences, has encouraged the EPA
to incorporate 21st century approaches into its risk assessment practices and to modify its single-
chemical approach. Tens of thousands of chemicals are currently in commerce and hundreds more are
introduced every year, yet only a small fraction have been adequately assessed for potential hazard. To
meet this challenge, the EPA established the Computational Toxicology (CompTox) Research Program
to explore and develop ways to exploit modern advances in molecular biology, chemistry, exposure
science and computer science to more effectively and efficiently assess chemical hazards and along with
exposure information, their risks. An effort central to the CompTox program is ToxCast, in which data
from a battery of rapid in vitro assays have been assembled for a variety of compounds.
The CompTox research program has the potential to provide the agency with a means to evaluate
individual chemicals more rapidly. Moreover, it can facilitate the shift from the EPA's traditional focus
on single stressors, endpoints, sources, pathways and environmental media to evaluate, more broadly,
multiple factors simultaneously. To assist the EPA in this process, the SAB asked its Exposure and
Human Health Committee (EHHC) to evaluate how the products from the CompTox research program
are being used by the EPA, whether the program outputs align with the needs of the EPA's programs
and whether limitations or challenges to using CompTox hazard and exposure data for risk assessment
and decision-making can be identified and addressed. The EHHC, along with two members of the EPA
Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific Advisory Panel, received
briefings from EPA representatives regarding the use of CompTox research program outputs as one
component of EPA risk assessments. In the enclosed report, the SAB provides its analysis and advice
regarding the issues that the agency should consider as it moves forward with implementation. This
letter highlights the SAB's major recommendations.
The challenges that the EPA faces regarding the various applications of CompTox data are substantial
and are well known to the agency. These include, but are not limited to, detailed characterization of each
-------
individual assay; determining the accuracy of the assays - or a battery of assays -to predict adverse
effects in animals; and the ways in which these patterns of data predict the risk of human disease. The
SAB offers recommendations on several applications for ToxCast data:
Using ToxCast data to develop Adverse Outcome Pathways (AOPs): The agency is currently
evaluating whether information derived from ToxCast is predictive of in vivo toxicity, using the
Endocrine Disrupter Screening Program (EDSP) as the focus of this validation work. This is a
reasonable focus and the agency is reporting some success. However, the SAB recommends that the
agency look not only at whether the ToxCast dataset for the EDSP validation is predictive of in vivo
toxicity, but also whether the endpoints captured in vivo have associated molecular endpoints in
ToxCast. This balance in information derived from the in vitro and in vivo setting is critical to the
validation process, and evaluating information flow in both directions will be crucial to validating the
ToxCast enterprise and streamlining the required battery of assays. The agency also is in the process of
developing "Adverse Outcome Pathways (AOP)" that will help link results from in vitro high-
throughput screening assays to human disease. As the agency develops these AOPs for the human
population, the SAB recommends that the agency explore partnerships with professional societies and
institutions devoted to the development of new information and therapeutic approaches for specific
diseases.
Using ToxCast data in a consistent manner: Following the Deepwater Horizon accident in 2010, a
subset of ToxCast assays was used to evaluate the endocrine toxicity of eight different commercial
dispersants being considered for use in the Gulf of Mexico. Considering this application as a case study,
the SAB finds that the logic underlying the specific choice of assays - even the choice of endocrine
toxicity as opposed to other mechanisms of toxicity - as well as the interpretation of the data derived
from these assays requires a more structured and transparent approach. EPA should examine their
decision to use these ToxCast assays to determine if the information they provided was the most
appropriate for the intended purpose (i.e., one that minimized risks to human health and the
environment). The SAB recommends that the agency develop a structured approach, such as a Data Use
Guide, to obtain and use ToxCast data, especially in emergency situations. A retrospective study of the
Deepwater Horizon accident can help inform the development of this structured approach. A Data Use
Guide or other structured approach would be useful for other ToxCast applications as well.
Using ToxCast data for evaluating the toxicity of mixtures: The CompTox research program offers the
agency a unique opportunity to develop data on environmentally relevant mixtures. This is an important
element of the rationale for ToxCast, but requires development of the practical aspects of studying
mixtures using in vitro assays and validation of the predictive nature of the assays since a large dataset
of in vivo data on mixtures is not available. The SAB recommends that the agency begin a formal
approach to address these challenges in the field of mixture research using ToxCast and other CompTox
technologies.
The CompTox research program is at a stage of maturity where researchers within the EPA - as well as
partners working along with the agency - would benefit from input regarding the applicability of
program outputs. Therefore, the SAB recommends that the agency use an advisory committee to
promote engagement with a multi-disciplinary group of external scientists, including scientists with
expertise in social and decision science, epidemiology and basic science, among others disciplines. The
goal of this advisory group would be to help ensure that the agency continues to generate tools that are
useful not only within the EPA but also to a broad array of extramural stakeholders. This will help the
ToxCast program remain relevant to the overall mission of the agency and will improve the ability of the
-------
outside community to provide thoughtful input to the program. In addition, the SAB notes that the
CompTox data frequently will be used in conjunction with other hazard and exposure information and
recommends that the program seek input on the integration of CompTox products with other data
essential to assessing risk.
The SAB commends the CompTox program for efforts to reach out to EPA program offices and
stakeholders to communicate the value and utility of the research program. These efforts are laudable
and should continue. The program has developed Computational Toxicology Communities of Practice,
composed of more than 300 people from over 50 public and private sector organizations interested in the
application of computational toxicology and exposure science to EPA's risk assessments. This
development is an especially worthy effort and should be continued.
The SAB appreciates the opportunity to provide these recommendations and we look forward to
continued progress in ToxCast and other CompTox research programs.
Sincerely,
/signed/
/signed/
David Allen, PhD
Chair, Science Advisory Board
Thomas Zoeller, PhD
Chair, SAB Exposure and Human Health Committee
Enclosure
-------
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board, a public
advisory group providing extramural scientific information and advice to the Administrator and other
officials of the Environmental Protection Agency. The Board is structured to provide balanced, expert
assessment of scientific matters related to the problems facing the agency. This report has not been
reviewed for approval by the agency and, hence, the contents of this report do not necessarily represent
the views and policies of the Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does mention of trade names or commercial products constitute a
recommendation for use. Reports of the EPA Science Advisory Board are posted on the EPA website at
http ://www. epa.gov/sab.
-------
U.S. Environmental Protection Agency
Science Advisory Board
Exposure and Human Health Committee (2013) and Liaisons
CHAIR
Dr. R. Thomas Zoeller, Professor, Department of Biology, University of Massachusetts, Amherst, MA
MEMBERS
Dr. Claude Emond, Adjunct Clinical Professor, Department of Environmental and Occupational
Health, Faculty of Medicine, University of Montreal, Montreal, QC, Canada
Dr. Anna Fan-Cheuk, Chief, Pesticide and Environmental Toxicology Section, California
Environmental Protection Agency, Oakland, CA
Dr. Alfred Franzblau, Professor, Department of Environmental Health Sciences, School of Public
Health, University of Michigan, Ann Arbor, MI
Dr. Maida Galvez, Assistant Professor, Department of Community and Preventive Medicine,
Department of Pediatrics, Mount Sinai School of Medicine, New York, NY
Dr. Chris Gennings, Professor, Department of Biostatistics, Virginia Commonwealth University,
Richmond, VA
Dr. Gary Ginsberg, Toxicologist, Environmental & Occupational Health, Connecticut Department of
Public Health, Hartford, CT
Dr. Robert L. Goble, Research Professor, George Perkins Marsh Institute, Clark University, Worcester,
MA
Dr. Russ Hauser, Frederick Lee Hisaw Professor, Department of Environmental Health, Harvard
School of Public Health, Boston, MA
Dr. Laurie Haws, Principal Health Scientist, ToxStrategies, Inc., Austin, TX
Dr. Darryl Hood, Professor, Department of Neuroscience and Pharmacology, Center for Environmental
Health Disparities and Medicine, Meharry Medical College, Nashville, TN
Dr. Kirsten Moysich, Professor of Oncology, Department of Cancer Prevention and Control, Roswell
Park Cancer Institute, Buffalo, NY
Dr. Gloria Post, Research Scientist, Office of Science, New Jersey Department of Environmental
Protection, Trenton, NJ
11
-------
Dr. Virginia Rauh, Professor and Deputy Director, Department of Population and Family Health,
School of Public Health, Columbia University, New York, NY
Dr. P. Barry Ryan, Professor of Exposure Science and Environmental Chemistry, Environmental and
Occupational Health, Rollins School of Public Health, Emory University, Atlanta, GA
Dr. John Vena, Professor and Department Head, Department of Epidemiology and Biostatistics,
College of Public Health, University of Georgia, Athens, GA
Dr. Clifford Weisel, Professor, Department of Environmental and Occupational Medicine, University
of Medicine and Dentistry of New Jersey, Piscataway, NJ
Dr. Robert Wright, Associate Professor, Pediatrics, Division of Environmental Health, Harvard School
of Public Health, Boston, MA
LIAISONS FROM THE FEDERAL INSECTICIDE, FUNGICIDE AND RODENTICIDE ACT
SCIENTIFIC ADVISORY PANEL
Dr. Janice Chambers, William L. Giles Distinguished Professor and Director, Center for
Environmental Health Sciences, College of Veterinary Medicine, Mississippi State University,
Mississippi State, MS
Dr. Daniel Schlenk, Professor of Aquatic Ecotoxicology & Environmental Toxicology, Department of
Environmental Sciences, University of California, Riverside, CA
SCIENCE ADVISORY BOARD STAFF
Dr. Suhair Shallal, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board (1400R), 1200 Pennsylvania Avenue, NW, Washington, DC
in
-------
U.S. Environmental Protection Agency
Science Advisory Board (2013)
CHAIR
Dr. David T. Allen, Gertz Regents Professor of Chemical Engineering and the Director of the Center
for Energy and Environmental Resources, The University of Texas, Austin, TX
MEMBERS
Dr. George Alexeeff, Director, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA
Dr. Pedro Alvarez, Department Chair and George R. Brown Professor of Engineering, Department of
Civil & Environmental Engineering, Rice University, Houston, TX
Dr. Joseph Arvai, Svare Chair in Applied Decision Research, Institute for Sustainable Energy,
Environment, & Economy, Haskayne School of Business, University of Calgary, Calgary, Alberta,
Canada
Dr. Thomas Burbacher, Professor, Department of Environmental and Occupational Health Sciences,
School of Public Health, University of Washington, Seattle, WA
Dr. Ingrid Burke, Director, Haub School and Ruckelshaus Institute of Environment and Natural
Resources, University of Wyoming, Laramie, WY
Dr. Thomas Burke, Professor and Jacob I. and Irene B. Fabrikant Chair in Health, Risk and Society
Associate Dean for Public Health Practice, Johns Hopkins Bloomberg School of Public Health, Johns
Hopkins University, Baltimore, MD
Dr. Edward T. Carney, Departmental Senior Science Leader and Director of Predictive Toxicology
Center, Toxicology & Environmental Research and Consulting, The Dow Chemical Company, Midland,
MI
Dr. Terry Daniel, Professor of Psychology and Natural Resources, Department of Psychology, School
of Natural Resources, University of Arizona, Tucson, AZ
Dr. George Daston, Victor Mills Society Research Fellow, Global Product Stewardship, The Procter &
Gamble Company, Mason, OH
Dr. Costel Denson, Managing Member, Costech Technologies, LLC, Newark, DE
Dr. Otto C. Doering III, Professor, Department of Agricultural Economics, Purdue University, W.
Lafayette, IN
Dr. Michael Dourson, President, Toxicology Excellence for Risk Assessment, Cincinnati, OH
Dr. Joel Ducoste, Professor, Department of Civil, Construction, and Environmental Engineering,
College of Engineering, North Carolina State University, Raleigh, NC
iv
-------
Dr. David A. Dzombak, Walter J. Blenko, Sr. University Professor of Environmental Engineering,
Department of Civil and Environmental Engineering, College of Engineering, Carnegie Mellon
University, Pittsburgh, PA
Dr. T. Taylor Eighmy, Vice Chancellor for Research and Engagement, University of Tennessee,
Knoxville, TN
Dr. Elaine Faustman, Professor and Director, Environmental and Occupational Health Sciences,
University of Washington, Seattle, WA
Dr. R. William Field, Professor, Department of Occupational and Environmental Health, and
Department of Epidemiology, College of Public Health, University of Iowa, Iowa City, IA
Dr. H. Christopher Frey, Distinguished University Professor, Department of Civil, Construction and
Environmental Engineering, College of Engineering, North Carolina State University, Raleigh, NC
Dr. John P. Giesy, Professor and Canada Research Chair, Veterinary Biomedical Sciences and
Toxicology Centre, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
Dr. Cynthia M. Harris, Director and Professor, Institute of Public Health, Florida A&M University,
Tallahassee, FL
Dr. Robert Johnston, Director of the George Perkins Marsh Institute and Professor, Economics, Clark
University, Worcester, MA
Dr. Kimberly L. Jones, Professor and Chair, Department of Civil Engineering, Howard University,
Washington, DC
Dr. Bernd Kahn, Professor Emeritus and Associate Director, Environmental Radiation Center, Georgia
Institute of Technology, Atlanta, GA
Catherine Karr, Associate Professor - Pediatrics and Environmental and Occupational Health Sciences
and Director - NW Pediatric Environmental Health Specialty Unit, University of Washington, Seattle,
WA
Dr. Madhu Khanna, Professor, Department of Agricultural and Consumer Economics, University of
Illinois at Urbana-Champaign, Urbana, IL
Dr. Nancy K. Kim, Senior Executive, Health Research, Inc., Albany, NY
Dr. Francine Laden, Mark and Catherine Winkler Associate Professor of Environmental
Epidemiology, Harvard School of Public Health, and Channing Division of Network Medicine, Brigham
and Women's Hospital and Harvard Medical School, Boston, MA
Dr. Cecil Lue-Hing, President, Cecil Lue-Hing & Assoc. Inc., Burr Ridge, IL
-------
Dr. Elizabeth Matsui, Associate Professor, Pediatrics, School of Medicine, Johns Hopkins University,
Baltimore, MD
Dr. Surabi Menon, Director of Research, ClimateWorks Foundation, San Francisco, CA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. Christine Moe, Eugene J. Gangarosa Professor, Hubert Department of Global Health, Rollins
School of Public Health, Emory University, Atlanta, GA
Dr. Horace Moo-Young, Dean and Professor, College of Engineering, Computer Science, and
Technology, California State University, Los Angeles, CA
Dr. Eileen Murphy, Director of Research and Grants, Ernest Mario School of Pharmacy, Rutgers
University, Piscataway, NJ
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Duncan Patten, Director, Montana Water Center, and Research Professor, Hydroecology Research
Program, Department of Land Resources and Environmental Sciences, Montana State University,
Bozeman, MT
Dr. Martin Philbert, Dean and Professor, Environmental Health Sciences, School of Public Health,
University of Michigan, Ann Arbor, MI
Dr. Stephen Polasky, Fesler-Lampert Professor of Ecological/Environmental Economics, Department
of Applied Economics, University of Minnesota, St. Paul, MN
Dr. C. Arden Pope, III, Professor, Department of Economics, Brigham Young University, Provo, UT
Dr. Stephen M. Roberts, Professor, Center for Environmental and Human Toxicology, University of
Florida, Gainesville, FL
Dr. Amanda Rodewald, Professor of Wildlife Ecology, School of Environment and Natural Resources,
The Ohio State University, Columbus, OH
Dr. James Sanders, Director and Professor, Skidaway Institute of Oceanography, Savannah, GA
Dr. William Schlesinger, President, Gary Institute of Ecosystem Studies, Millbrook, NY
Dr. Gina Solomon, Deputy Secretary for Science and Health, Office of the Secretary, California
Environmental Protection Agency, Sacramento, CA
Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics,
University of Southern California, Los Angeles, CA
VI
-------
Dr. Peter S. Thorne, Director, Environmental Health Sciences Research Center and Professor and
Head, Department of Occupational and Environmental Health, College of Public Health, University of
Iowa, Iowa City, IA
Dr. Paige Tolbert, Professor and Chair, Department of Environmental Health, Rollins School of Public
Health, Emory University, Atlanta, GA
Dr. Jeanne Van Briesen, Professor, Department of Civil and Environmental Engineering, Carnegie
Mellon University, Pittsburgh, PA
Dr. John Vena, University of Georgia Foundation Professor in Public Health and
Head, Department of Epidemiology and Biostatistics, Georgia Cancer Coalition Distinguished Scholar,
College of Public Health , University of Georgia, Athens, GA
Dr. R. Thomas Zoeller, Professor, Department of Biology, University of Massachusetts, Amherst, MA
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board (1400R), 1200 Pennsylvania Avenue, NW, Washington, DC
vn
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 3
3. FINDINGS AND RECOMMENDATIONS 6
3.1. APPLICATIONS OF COMPTOX DATA TO EPA PROGRAMS 6
3.1.1. Challenges of Relating ToxCast Outputs to in vivo Assay Results 6
3.1.2. Development of Adverse Outcome Pathways 7
3.1.3. Current Use of ToxCast Outputs 7
3.2. EVALUATING COMPTOX OUTPUTS FOR DECISION-MAKING 8
3.2.1. Specificity, Sensitivity, Reliability, and Relevance of ToxCast Data 8
3.2.2. Incorporating Exposure Information 10
3.2.3. Structured Approaches for Data Use 11
3.3. BUILDING SCIENTIFIC ACCEPTANCE OF COMPTOX 11
3.4. COMMUNICATING COMPTOX APPROACHES AND OUTPUTS 14
3.5. OTHER ISSUES 15
GLOSSARY 17
REFERENCES 19
APPENDIX A. HISTORY OF THE COMPTOX PROGRAM A-l
APPENDIX B. ASSAY PERFORMANCE CRITERIA B-l
-------
1. EXECUTIVE SUMMARY
The EPA's Computational Toxicology Research Program (CompTox) is part of the EPA's broader
research program on Chemical Safety for Sustainability. Traditional chemical toxicity testing is
expensive, time-consuming and uses a significant number of animals. As a result, a large number of
chemicals in commerce have not been evaluated sufficiently - or at all - for safety yet the agency must
make hundreds of decisions each year based on limited information. CompTox is researching new, more
efficient, ways to evaluate the safety of chemicals, particularly in assessing chemicals for potential risk
to human health and the environment. EPA's National Center for Computational Toxicology (NCCT)
was established in 2005 to coordinate computational toxicology research on chemical screening and
prioritization, informatics and systems modeling. A core program managed by the NCCT is the ToxCast
program. ToxCast is building a database of chemical activities by evaluating a large number of
chemicals in a very large (700+) battery of high-throughput in vitro assays. The agency is publishing an
increasing number of research papers describing the results of this program and the ways in which the
information can be validated for use in agency decisions.
To assist the EPA in this process, the SAB asked its Exposure and Human Health Committee (EHHC) to
evaluate how the products from the CompTox research program are being used by the agency, whether
the program outputs align with the needs of the EPA's programs and whether limitations or challenges
to using CompTox hazard and exposure data in decision-making for risk assessment can be identified
and addressed. The EHHC, along with two members of the EPA Federal Insecticide, Fungicide, and
Rodenticide Act (FIFRA) Scientific Advisory Panel, received briefings from EPA representatives
regarding the use of CompTox research program outputs as one input to EPA risk assessments. This
report provides analysis and advice regarding the issues that the agency should consider as it moves
forward with implementation. Because the ToxCast program is more advanced than other programs
within CompTox, much of the focus of this report is on ToxCast.
Using ToxCast data to develop Adverse Outcome Pathways (AOPs). ToxCast outputs currently are
being used principally within a research context. The agency is evaluating whether information derived
from ToxCast is predictive of in vivo toxicity, using the Endocrine Disrupter Screening Program (EDSP)
as the focus of this validation work. This is a reasonable focus and the agency is reporting some success.
However, the SAB recommends that the agency look not only at whether the ToxCast dataset is
predictive of in vivo toxicity, but also whether the endpoints from in vivo studies have associated
molecular endpoints in ToxCast. This balance in information derived from the in vitro and in vivo
setting is critical to the validation process, and evaluating information flow in both directions will be
crucial to validating the ToxCast enterprise and streamlining the required battery of assays. The agency
also is in the process of developing "Adverse Outcome Pathways (AOP)" that will help link results from
high-throughput in vitro assays to human disease. As the agency develops these AOPs for the human
population, the SAB recommends that the agency explore partnerships with professional societies and
institutions devoted to the development of new information and therapeutic approaches for specific
diseases.
Using ToxCast data in a consistent manner. The SAB learned about the Deepwater Horizon accident,
where a subset of ToxCast assays were used to evaluate the endocrine toxicity of eight different
commercial dispersants. This application illustrates important benefits of the ToxCast program, as well
as opportunities to strengthen it for use in future emergencies and other uses. Examination of the
Deepwater Horizon case showed that it would have been helpful to have a more structured and
1
-------
transparent rationale for the choice of specific assays, the choice of endocrine toxicity as opposed to
other mechanisms of toxicity, as well as the interpretation of the data derived from these assays, and
whether the ToxCast assays provided the right information. Therefore, the SAB recommends that the
agency develop a Data Use Guide or other structured approach to the use of ToxCast data in various
applications, including the interpretation of the resulting information. The Deepwater Horizon accident
should be studied in retrospect to help inform this structured approach.
Using ToxCast data for evaluating the toxicity of mixtures. The CompTox research program offers the
agency a unique opportunity to develop data on environmentally relevant mixtures. This is an important
element of the rationale for ToxCast, but requires development including the practical aspects of
studying mixtures in in vitro assays, and in validating the predictive nature of the assays in the absence
of a large dataset of in vivo data on mixtures. The SAB recommends that the agency begin a formal
approach to addressing these challenges in the field of mixture research using ToxCast and other
CompTox technologies.
The SAB recommends that performance characteristics of assays that relate to specificity, sensitivity and
reliability be made available on the EPA website. The SAB understands that the suitability of in vitro
data for regulatory decisions depends on the type and level of decisions to be made. This underscores
the importance of a principal research goal of the program, that is, to characterize the data generated
from ToxCast assays in terms of the specificity, sensitivity and reliability of the individual assays, as
well as their ability to predict toxicity either alone or in combination with other findings.
The CompTox research program, specifically ToxCast, is at a stage of maturity where researchers within
the EPA - as well as partners working along with the agency - would benefit from input regarding the
applicability of program outputs. Therefore, the SAB recommends that the agency use an advisory
committee to promote engagement with external scientists. This will help the ToxCast program remain
relevant to the overall mission of the agency and will improve the ability of the outside community to
provide thoughtful input to the program. In addition, the SAB notes that the ToxCast data frequently will
be used in conjunction with metabolic and exposure information and recommends that the program seek
input on the integration of ToxCast products with other data essential to assessing risk.
-------
2. INTRODUCTION
EPA's Computational Toxicology (CompTox) Research Program has been underway since 2003. EPA
established the National Center for Computational Toxicology in 2005. The goal of this Center is to
oversee the development of innovative in vitro tools and methodologies as alternatives to animal studies
for use in chemical risk assessment. The EPA defines computational toxicology as the integration of
modern computing and information technology with molecular biology and chemistry to improve risk
assessment and prioritization of the data EPA requires to assess chemicals. The three long-term goals for
the program are: (a) to provide EPA risk assessors with improved methods and tools to better understand
and describe linkages across the source-to-outcome paradigm; (b) to provide EPA Program Offices with
advanced hazard characterization tools to prioritize and screen chemicals for toxicological evaluation;
and (c) to provide EPA risk assessors and regulators with new models based on the latest science to
reduce uncertainties in dose-response assessment, cross-species extrapolation, and quantitative risk
assessment. A more extensive account of the history and goals of the program are included in Appendix
A of this report.
In 2007, the National Research Council (NRC) Committee on Toxicity Testing and Environmental
Assessment published a study, Toxicity Testing in the 21st Century: A Vision and a Strategy (NRC
2007). In this report, the NRC Committee recommended developing a program that would incorporate
modern tools to provide information about chemical toxicity for use in risk assessments. The overall
goal of such a program would be to enhance the efficiency and effectiveness of chemical safety
determinations. Consistent with the recommendations of the NRC report, the EPA in that same year
launched ToxCast, an initiative central to the CompTox research program, to develop rapid automated
chemical toxicity tests. Including ToxCast, the CompTox research program manages 10 research areas
(Table 1). These research areas are intended to provide innovative research that integrates advances in
molecular biology, chemistry and innovative computer science, and to communicate this information so
as to more effectively and efficiently rank chemicals based on risks.
Table 1. CompTox Research Program: Managed Research
Areas
ACToR - Aggregated Computational Toxicology Resource
Determining Uncertainty
Distributed Structure-Searchable Toxicity (DSSTox) Public Database
Network
ExpoCast
ToxCast
Toxicological Priority Index (ToxPi)
Toxicity Reference Database (ToxRefDB)
Tox21
Virtual Liver Project (v-Liver)
Virtual Embryo Project (v-Embryo)
In a report on the FY2012 EPA research budget, the EPA Science Advisory Board (SAB) noted its
concern that, ".. .there is no proactive budget initiative to develop ways of employing the results of the
[Chemical Safety for Sustainability] CSS program, including high throughput data, into hazard or risk
assessment" (U.S. EPA SAB 2011). The CSS program, one of six transdisciplinary research programs
within the EPA's Office of Research and Development, is responsible for coordinating the activities of
the CompTox research program. . In addition to allowing more rapid evaluation of a large number of
3
-------
individual chemicals, CompTox data and technologies overall has the potential to provide the agency
with a means of modifying its traditional focus on single stressors, endpoints, sources, pathways, and
environmental media to a broader focus on evaluation of these factors in combination to evaluate the
potential effects of the co-occurrence of multiple chemicals. Recognizing these potential benefits and
impacts of CompTox data and technologies for the EPA, the SAB has requested that its Exposure and
Human Health Committee (EHHC) develop advice to assist in advancing the application of CompTox
research products for human health risk assessment to meet EPA's programmatic needs.
The members of the SAB Exposure and Human Health Committee (EHHC) were joined for this review
by two members of the EPA Federal Insecticide, Fungicide, and Rodenticide Act Scientific Advisory
Panel (FIFRA SAP) who had reviewed elements of the ToxCast program in 2011.1 Members of the
panel engaged in discussions with the Office of Research and Development (ORD) and EPA program
offices that currently use or plan to use the CompTox research outputs. Discussions addressed the
following questions:
1. How are the outputs of the CompTox research program being used currently by EPA? What
are the challenges that EPA has had to overcome when using these outputs?
2. How well do the outputs of the CompTox research program match with EPA's environmental
assessment and management goals and/or the needs of key partners?
a. Are there plans to align the outputs and needs more closely?
b. What is the timetable and are those plans appropriate?
c. How can EPA align the outputs of the CompTox research program and EPA
assessment/management needs more closely?
3. Are there obstacles (conceptual, legal, policy, institutional, other) preventing EPA from using
the outputs from the computational toxicology program in EPA assessments and to support
management decisions? How significant are these barriers? How might they be overcome?
What would be a realistic timetable for overcoming them? What are the requirements of data
from high-throughput screening that would allow them to be informative to hazard
characterization and/or risk assessment?
4. How would the outputs of the computational toxicology program be effectively
communicated to the public as part of a new EPA assessment approach? What research or
other steps should be taken to build an effective approach for communicating this new
science?
The EPA representatives described the overall scope, structure and organization of the CompTox
research program, and the use of CompTox information within agency programs. The CompTox
research program has generated a large number of products including web sites and web-based tools,
and publications describing various research activities and outcomes of the research. These were
effectively reviewed by the EPA's Board of Scientific Counselors (BOSC) in 2009 (U.S.EPA BOSC,
2010). The goal of the current SAB process was not to duplicate the work of the BOSC but rather to
specifically evaluate how CompTox tools are being implemented within the agency to date. As a result
of the information presented by EPA representatives to the panel, this report focuses largely on ToxCast.
1 Report available at the following URL: http://www.epa.gov/scipoly/sap/meetings/201 I/may705241 lminutes.pdf
4
-------
ToxCast products are a central element of CompTox research and have begun to be used in specific
situations.
The ToxCast program2 currently consists of nearly 700 pre-existing individual in vitro assays provided
by nine companies. A foundational element of ToxCast is a chemical library in which a large number of
chemicals are tested simultaneously to create toxicity profiles in these assays. In Phase I, chemicals for
which there is a substantial amount of toxicity data (including 309 pesticide active ingredients and
commercial chemicals) have been evaluated in the ToxCast battery of in vitro assays. Data generated
from this exercise have been analyzed to provide a "proof of concept"; i.e., the results will be used to
develop toxicity profiles and evaluate the ability of the assays to predict toxicity. In Phase II, about
2,000 chemicals are being evaluated similarly from a broad range of sources including industrial and
consumer products, food additives, "green" products that reduce or eliminate the use or generation of
hazardous substances, nanomaterials and drugs that never made it to the marketplace. This information
will be used to identify pathways of toxicity - patterns of responses observed in the CompTox assays
that are plausibly and causally related to observations of apical effects or empirically verifiable
outcomes of exposure (e.g., developmental anomalies) in the in vivo assays.
A draft SAB committee report was developed during the summer of 2012 and discussed at a
teleconference on September 24, 2012. The draft report was then considered by the chartered SAB on
March 7, 2013 and revised based on member's comments. The following report outlines the SAB's
recommendations on how to enhance the utility of the products generated by the CompTox research
program, and more specifically the ToxCast program outputs.
More information regarding the ToxCast program is available at the following URL: http://www.epa.gov/ncct/toxcast/ and
in a fact sheet at: http://www.epa.gov/ncct/download files/factsheets/Tox Cast Fact Sheet.pdf
5
-------
3. FINDINGS AND RECOMMENDATIONS
3.1. Applications of CompTox Data to EPA Programs
Study Question 1. How are the outputs of the CompTox program being used currently by EPA? What
are the challenges that EPA has had to overcome when using these outputs?
Currently, the outputs of CompTox are being used largely within a research context. It is essential to
recognize that this is a highly focused applied research program designed to determine whether, and
how, large sets of in vitro data may be used to enhance the mission of the EPA in the short and long-
term. Therefore, at present, CompTox outputs, and specifically ToxCast, have been used primarily
within ORD to compare in vivo data with the resulting in vitro data gathered from a variety of assays to
determine their predictive capacity and reliability for use in various types of decision-making (e.g.,
screening, prioritization) by agency programs. The EPA faces significant challenges to understanding
how information derived from ToxCast can be employed to inform the various decisions required of the
agency. These efforts will require communication between the different programs within the agency to
ensure that the outputs meet the needs of the specific programs. At the SAB meeting in May 2012, EPA
representatives from ORD and program office (e.g., pesticides, water or toxics) devoted considerable
time to describing how this coordination is occurring.
The ToxCast program within CompTox is designed to evaluate hundreds to thousands of chemicals in
hundreds of in vitro tests; many of these chemicals have known toxicological phenotypes for cancer,
reproductive impairment and developmental disorders. The initial goal is to acquire sufficient
information on a range of chemicals so that "bioactivity profiles" or "/'« vitro signatures" can be
discerned that predict patterns of toxic effects or phenotypes corresponding to those observed in
traditional animal toxicity testing. An underlying assumption in the design and execution of this
research program is that the pattern of responses observed in the in vitro assays will match toxicological
profiles in traditional in vivo studies, despite the fact that neither (in vitro assays or in vivo studies) was
designed with this intention in mind.
3.1.1. Challenges of Relating ToxCast Outputs to in vivo Assay Results
The ToxCast research program is at the stage where enough data are available to compare the in vitro
profiles against in vivo data; therefore, soon researchers will be able to know the extent to which the
program will be predictive and how they may be able to use it. A straightforward approach to address
the characterization of the ToxCast database is to determine whether the data from in vitro assays in
ToxCast are predictive of in vivo toxicity using a database of toxicity studies conducted with guideline,
in vivo test systems in addition to data derived from peer-reviewed non-guideline studies. In general,
chemicals that acted on estrogen- or androgen-sensitive assays in ToxCast were reported to have a high
degree of probability of acting on estrogenic or androgenic endpoints in vivo. However, chemicals
acting on thyroid-sensitive assays in ToxCast were not identified as acting on thyroid-sensitive
endpoints in vivo. These findings indicate that the ToxCast data will be reasonably predictive of
endpoints of estrogen and androgen toxicity in the EDSP, but will not be predictive of thyroid toxicity.
This application of ToxCast data represents a good beginning to test reliability because there are
considerable data in vivo to compare with the ToxCast data. In addition, the SAB recommends that the
agency consider information flow in both directions; that is, from in vitro to in vivo and from in vivo to
in vitro. In this manner, the questions become whether data from specific assays in ToxCast are
predictive of effects observed in vivo, and whether molecular and cellular events that lead to effects on
endpoints identified in tier 1 of the EDSP are represented by specific assays in ToxCast.
6
-------
3.1.2. Development of Adverse Outcome Pathways
The structured framework within which ToxCast data may be linked to adverse outcomes (i.e., be
predictive of toxicity), both in the EDSP and potentially in other applications including in the human
population, is called the Adverse Outcome Pathway (AOP). Building these AOPs represents a very
important link from in vitro high-throughput assays to human disease, and this effort is just beginning.
The SAB recommends that the EPA explore AOPs not only based upon how a chemical can perturb
biological systems but also from the perspective of how aging and disease processes have underlying
AOPs which may be sensitive to chemical effect. By considering upstream events, the EPA can use
CompTox data to evaluate how chemical exposure and disease AOPs may intersect leading to a more
complete understanding of chemical action (NRC 2009). Building and characterizing these AOPs would
benefit from the expertise of those in the scientific community who conduct research on specific health
outcomes, and the SAB recommends that the agency partner with professional societies and/or public
institutions to engage with these experts. Examples might include the American Lung Association or the
National Heart, Lung and Blood Institute to aid in building AOPs related to lung disease or disorders.
As ToxCast data are evaluated for deployment in different applications, the approach to test reliability
becomes less obvious and has not been articulated. The agency is exploring the possibility that ToxCast
data can be combined with large databases of experimental data at the level of the genome, epigenome,
proteome and metabolome to provide integrated data within the context of AOPs. In principle, weight-
of-evidence approaches would be developed to guide the integration of this information into different
kinds of practices for hazard identification and perhaps risk assessment. The SAB agrees that, if
successful, this effort would be enormously valuable, but the realization of this is distant.
3.1.3. Current Use of ToxCast Outputs
One example where the EPA has employed the ToxCast program to help inform an important decision
was after the Deepwater Horizon accident. In this situation, the agency used a subset of ToxCast assays
to assess the endocrine toxicity of eight oil dispersants that had the potential to be employed in the Gulf
after the oil spill. Having this research program in place provided an important example of how the
system can be used now and in the future. Moreover, this situation illustrates a number of important
ancillary benefits of this program, including having developed an infrastructure that would allow rapid
data generation so that the agency can make better-informed decisions in an emergency situation.
Another benefit is that development of the overall CompTox research program required a great deal of
interaction between various EPA offices. This interaction will foster greater communication about data
needs and data interpretation. This interaction also helps to ensure that the intramural research program
aligns with the routine as well as the sometimes unanticipated needs of the agency, and helps risk
assessors identify early the data gaps that may be filled by ToxCast.
However, the Deepwater Horizon accident also illustrates that in emergency situations ToxCast data
may be generated and used very rapidly without the opportunity to fully evaluate the rationale behind
the selection of a subset of assay data or the interpretation of those data for agency decisions, and
without an obvious results-based validation that the decisions made based on these data were the best.
Therefore, the SAB strongly encourages the agency to engage in strategic planning so that the endpoints
and assays employed in a specific set of circumstances are maximally useful for that situation. Maximal
utility would include a thoughtful analysis of the strengths and weaknesses of the subset of ToxCast data
collected for a particular situation. The SAB recommends that the agency develop Data Use Guides to
assist the agency in the design of assays and interpretation of ToxCast data generated in an emergency
situation. The SAB also recommends that the agency use the Deepwater Horizon incident as a case
study for the use of ToxCast data.
-------
The ToxCast program also provides a unique way of evaluating the risk posed by exposures to mixtures
of chemicals. This may be a means of enhancing the agency's traditional, more limited, focus on single
stressors, endpoints, sources, pathways, and environmental media to a broader focus on the combination
of these factors or the potential co-occurrence among these factors. The SAB recommends that the
agency consider the use of ToxCast to explore the impact of mixtures.
3.2. Evaluating CompTox Outputs for Decision-making
Study Question 2. How well do the outputs of the CompTox program match with EPA 's environmental
assessment and management goals and/or the needs of key partners?
a) Are there plans to align the outputs and needs more closely?
b) What is the timetable and are those plans appropriate?
c) How can EPA align the outputs of the CompTox program and EPA assessment/management
needs more closely?
This question is related to the first but is focused specifically on the degree to which the outputs from
the CompTox research program are tailored to match the needs of the agency and key partners. In
general, the administrative structure within ORD is designed to ensure that the work being performed -
and the outputs generated - is matched to EPA's assessment and management goals. This includes both
planning and implementation of ORD's research programs and periodic reviews by the BOSC, as well
as feedback from regional offices and national program directors. Thus, there is every reason to believe
that the organizational structure of the CompTox research program within ORD is being constantly
evaluated and tailored to meet the agency's needs.
While the structure is in place to ensure that CompTox outputs are relevant to the agency and its
partners, there are two fundamental issues that influence the use of these data in decision-making at
different levels. First is what might be called "data integrity." That is, the data from each individual
assay should be specific, sensitive and reliable. The second is "relevance." That is, the data should be
informative at the level of prioritization, screening or green chemistry. These issues are briefly
discussed below because these issues impact the degree to which CompTox outputs will become useful
within the agency.
3.2.1. Specificity, Sensitivity, Reliability, and Relevance of ToxCast Data
The technical characteristics of the many assays that make up ToxCast - data integrity - were not
discussed in detail at the May 30, 2012 meeting because reviewing these characteristics was beyond the
scope of the committee's charge. The SAB expects that these technical issues of performance
characteristics have been evaluated by the agency and suggests that this information be made available.
Performance characteristics for individual assays were not discernible from the data posted to the
website, but may influence the weight that a single assay should have within a battery.
The relevance of the data to decision-making is another matter. This will depend on the level of
decision (prioritization, screening or green chemistry). There are, however, additional situations where
the data may be used in different ways, such as in the case of Deepwater Horizon accident. It will be
important to proactively envision the various ways that these data may be employed and develop a
structured approach to demonstrate relevance for that application.
A central question at this time is whether the in vitro high-throughput assays will produce data that will
be suitable for decision-making such that, eventually, these data could replace in vivo testing in various
kinds of regulatory decisions. The answer to this question will undoubtedly depend on the level of
-------
decisions to be made. For decisions with significant regulatory consequences, an important - if not
essential - goal will be to sufficiently demonstrate the relevance of and obtain widespread acceptance of
the data generated from ToxCast by risk assessors, risk managers and the regulated community. The
data also will need to be consistent with statutory requirements for the evidence the EPA uses to take
regulatory action. Thus, a principal goal of the research domain of the program is to characterize the
data generated from ToxCast assays in terms of the specificity, sensitivity and reliability of the
individual assays, as well as their ability to predict toxicity either alone or in combination with other
findings.
The EPA has adopted two general strategies for testing the value of ToxCast data for agency use. The
first strategy is to identify the patterns of responses for each chemical in the battery of ToxCast assays
and correlate these with the biological activities observed in in vivo studies associated with the same
chemical. This strategy is made possible by the considerable amount of in vivo data associated with the
Phase I chemicals. Of course, the assays included in ToxCast were pre-existing high-throughput assays
developed for the pharmaceutical industry; they were not designed for ToxCast to correlate with
endpoints employed in in vivo studies. Therefore, the SAB recommends that the agency consider
developing the theoretical framework that would support the effectiveness of this strategy. That is, rather
than evaluating how well ToxCast data predict the in vivo data, develop a structure that predicts how
well ToxCast data should predict the in vivo data. Essentially, this amounts to developing "AOPs" for
the in vivo studies and matching the in vitro assays to those AOPs.
The EPA's second strategy is to develop AOPs for human disease that may be reflected in the ToxCast
data. This strategy highlights an important weakness in these two strategies that can be addressed in
building these AOPs. Specifically, the ToxCast assays were not designed by the agency to inform in vivo
endpoints, and the guideline in vivo endpoints were not designed overtly to inform human disease. Thus,
to build a credible system, the EPA needs to focus on making the case that there is a relationship
between what is observed in the ToxCast assays, what is observed in the guideline studies, and what is
observed (or expected) in the human population. For example, according to Judson et al. (2010),
CompTox models developed to screen for chronic, developmental and reproductive toxicity endpoints
display high specificity (few false positives) but only moderate sensitivity (multiple false negatives).
The concept is that by evaluating the behavior of known toxicants in the ToxCast battery, patterns of
toxicity linking this high-throughput behavior to adverse outcomes and thereby enhancing predictability
will become apparent and will serve as validation of the predictive capability of the assay. However,
this is a very retrospective approach that implies that strong patterns of activity within a subset of the
700 assays of ToxCast will be readily visible. The fact that investigators outside the agency evaluating
ToxCast data are not finding the same predictive capacity of the data (Thomas et al. 2012) may indicate
that the agency should take a more prospective approach to evaluating the relevance of these data.
Ultimately, the usability of a given result will be dependent upon the availability of additional data for
the chemical in question and information regarding the tests and pathways affected by that chemical.
The advantage of ToxCast is that thousands of tests can be conducted. These need to be inclusive of as
many potential health effects as possible. EPA should make clear the value of the information obtained
from the breadth of assays. For example, the testing may be relevant to some cancers, developmental
and reproductive endpoints, and endocrine and metabolic endpoints, but not, for example, eye health or
neurological health. Positive results on subsets of assays or patterns of effects in certain biochemical
pathways could implicate further testing and/or in vivo studies. Of particular importance for public
health is the relevance of a negative result in an assay system, which in a screening step could result in a
9
-------
decision not to proceed with further testing. Regarding more advanced uses of ToxCast outputs beyond
hazard identification (e.g., use in dose-response assessment and risk assessment), the following
additional concerns were raised by the SAB:
Have the most predictive assays been identified in the ToxCast assays (which were pre-existing
and not designed for this application per se)?
How well do these ToxCast assays relate to apical endpoints such as carcinogenesis, endocrine
disruption, organ toxicity, neurotoxicity and immunotoxicity?
How would the uncertainty factors used when starting with in vivo data (e.g., interspecies,
intraspecies, acute to subchronic to chronic study duration, database quality and completeness) be
applied and/or modified for in vitro screening data?
How would the in vitro dose-response relate to in vivo behaviour when considering route of
entry, metabolic activation and detoxification systems that may not be present in vitro!
How would the in vitro dose-response relate to in vivo behaviour when considering other
toxicokinetic factors governing the external dose associated with a particular concentration at the target
cell or receptor that may not be taken into account in vitro! These factors include metabolic activation
and detoxification, as well as absorption through relevant route(s) of entry, distribution, and excretion;
and
Related to uncertainty factors, how well do the in vitro test methods capture intra-human
variability in terms of susceptible sub-populations and life stages including genetic polymorphisms and
disease states?
The SAB also notes that the ToxCast research program should develop strategies for studying
environmental chemical mixtures - not just the effects of one chemical at a time. The importance of
using ToxCast to characterize the hazard, and ultimately the risk, of environmental chemical mixtures
cannot be overstated. Moving in this direction requires establishing a scientifically defensible
foundation; for example, by defining appropriate AOPs, developing testing methods that address a wide
array of AOPs, and evaluating the accuracy, sensitivity and specificity of the tests. While assessment of
mixtures may ultimately be a long range goal, the path to studying and estimating risk from mixtures
should be outlined. Examples of critical questions include the following: (1) How will relevant mixtures
be identified? (2) Can methods be developed to predict the hazard and/or risk of mixtures of chemicals
from ToxCast data on individual chemicals which affect the same AOP(s)? (3) How will risk be
quantified for different types of endpoints based on effects on relevant AOPs? and (4) How much risk is
acceptable for a given AOP?
3.2.2. Incorporating Exposure Information
The SAB emphasizes the importance of considering the potential for exposure to the chemical when
determining the degree of testing required. Even if initial screens of a chemical find little reason for
concern, in vivo confirmation may still be desirable if the chemical's exposure potential is high.
Conversely, low exposures may diminish the need for extensive toxicity testing that might be needed for
agents to which exposure is greater. These considerations underscore the need for good
exposure/biomonitoring information, which at this point appears to be a limitation of CompTox
modeling. However, work is currently underway to address these issues (Gangwal et al. 2012; Wetmore
etal. 2012).
10
-------
As already noted, exposure is a key component of risk assessment and one that needs greater attention
within the CompTox program. A general approach, based solely on chemical properties, that evaluates
transport from large sources and partitioning based on fugacity concepts to predict the distribution of
chemicals from their sources to a population will not provide a full exposure evaluation and may lead to
misclassification of exposure. For example, an agent produced in relatively small quantities but used in
personal products can result in a higher exposure than a high production volume chemical emitted from
point sources located away from populations. The higher exposure potential of a low production
chemical would not be predicted based on an exposure model that does not include information on its
use and potential contact with people. If these two agents were equally hazardous, the low production
compound would present greater risk, but it is unclear if the current assays used in the CompTox
program would account for this situation.
3.2.3. Structured Approaches for Data Use
A key issue affecting use of ToxCast data is the need for a transparent structured approach to obtain and
interpret data for various applications, especially in emergency situations. While the data are meant to be
used within a weight-of-evidence context that requires integration across all of the available data (e.g., in
vivo toxicology data, structure-activity relationship (SAR), read-across approaches, other supporting in
vitro data), it would be beneficial to establish general principles for the use and interpretation of the
output for specific situations in Data Use Guidelines. Guidance for data needs (and sufficiency or
appropriateness of data) must come from a clear characterization of EPA programmatic needs -
identification of both the intended goals of a risk assessment or a prioritization effort - and examples in
which ToxCast information appears to add real value. Key aspects regarding the performance of assays
that are used should also be addressed in such a guidance document. Suggested criteria for assessing
these assays can be found in Appendix B.
The concept of a Data Use Guideline is not new. For example, the Centers for Disease Control/ National
Health and Nutrition Examination Survey (CDC/NHANES3) biomonitoring data release provides
important information for each exposure type including the normative range in the population, any
relevant workplace or environmental standards (e.g., Occupational Health and Safety Administration
(OSHA) Biological Exposure Indices), and limitations (e.g., specificity, sensitivity) of the biomarker
itself. This information is meant to aid in the interpretation of the data by various stakeholders and avoid
the over-interpretation of the data.
The SAB also recommends that the agency work toward developing a clear rationale describing the
different uses of the data depending upon specific effects. For example, the CompTox program's
"ToxPi" (or Toxicological Priority Index) pie chart of endocrine-related effects for a chemical appears to
be a useful way to illustrate the types of biological activities of a chemical, but not the meaning and
importance of individual slices relative to other slices. In addition, in cases where only a subset of assays
is employed for a particular application, there is no transparency in the logic about the construction of
that subset as opposed to other subsets.
3.3. Building Scientific Acceptance of CompTox
Study Question 3. Are there obstacles (conceptual, legal, policy, institutional, other) preventing EPA
from using the outputs from the computational toxicology program in EPA assessments and to support
management decisions? How significant are these barriers? How might they be overcome? What
would be a realistic timetable for overcoming them? What are the requirements of data from high-
3 Information can be accessed at the following URL: http://www.cdc.gov/nchs/nhanes.htm
11
-------
throughput screening that would allow them to be informative to hazard characterization and/or risk
assessment?
There are a number of challenges facing the ToxCast program with respect to preparing outputs for use
in agency decisions and by the broader scientific community. These challenges are specific to the
potential types of applications for these data, including informing decisions when other information is
not available, prioritizing chemicals for further toxicity analysis and as the basis for risk assessment. In
all cases, a common concern is whether the data generated from high-throughput in vitro assays can be
applied reliably, i.e., that the data will have been shown to be sufficiently predictive of toxicity in vivo
and ultimately in humans, relative to traditional approaches, such that agency decisions can be
scientifically defensible within an acceptable level of uncertainty. Barriers to the use of ToxCast data
include:
If an in vivo endpoint is not well anchored in an AOP or read-across approach, then perturbation
of that endpoint may be difficult to detect in ToxCast assays and thus it will be difficult to determine the
reliability of the in vitro assay;
Lack of understanding of the impact of the route of entry, metabolic activation and detoxification
systems and other toxicokinetic factors that may not be present in vitro;
Dose-response assessment that does not take into account in vitro to in vivo extrapolation
including metabolism and other toxicokinetic factors, application of uncertainty factors and special
consideration of vulnerable sub-groups;
The potential for false negative results when considering a chemical for increased usage based
upon ToxCast results; and
Limited exposure information, which is a key part of any screening and prioritization program,
as well as necessary for risk assessment.
The ToxCast program is in the development stage, as noted in response to Study Question 1, and so its
use is still restricted to a largely research domain. The program has not had sufficient time to
demonstrate that it can deliver on its promise. Questions remain about the reliability of individual
assays, the availability of assays predictive of the full range of relevant endpoints, the power of "pattern
recognition" as a predictor of toxicity, and the value of the current design of the system to generate the
kind of information needed to be predictive; these are legitimate questions that require time and
experience to answer. Considering the importance of these goals and the complexity of the issues
involved, there will be unavoidable "blind alleys." However, the number of these "blind alleys" may be
minimized by being more proactive about building AOPs and pathways of toxicity. In this regard, there
are currently no internationally accepted methods in the scientific literature for performing a weight-of-
evidence analysis for such pathways. While this task is not the purview of the ToxCast program per se,
the ability of the EPA to employ peer-reviewed science in the Tox21 initiative, which combines the
efforts of several federal agencies (i.e., EPA, NIEHS and FDA) to screen thousands of chemicals for
potential toxicity, would be enhanced by developing an accepted method of analysis for determining the
ability of ToxCast assays to predict human disease. In the absence of such an accepted method, the
agency will be limited in associating ToxCast data to data generated from guideline assays and this
would be a severe limitation.
One of the ways to improve acceptance of CompTox outputs and overcome some of the barriers to their
use is to demonstrate that they provide equivalent (or more accurate) answers relative to the currently
accepted methods for characterizing hazard and estimating risk. Moreover, if CompTox does so with
12
-------
fewer resources (e.g., cost and time), thereby allowing for the characterization of the large number of
agents that the EPA must make decisions about, then it will quickly become the methodology of choice.
There also is a need to commit similar resources to develop other components of the CompTox program,
such as ExpoCast, in parallel to ToxCast to more fully support the needs of EPA programs. This will
require not only acceptance by scientists at EPA's National Exposure Research Laboratory (NERL) but
also a recognition by others within the agency that exposure is a key component of risk assessment, risk
characterization and risk management and that the volume of an emission is not equivalent to exposure
and the dose humans receive.
According to the EPA's 2009 strategy document on evaluating chemical toxicity (U.S. EPA 2009), the
agency appears to be following the recommendation of the NRC 2007 committee that, "...in vitro tests
would be developed not to predict the results of current [animal] apical toxicity tests but rather as
[human] cell-based assays that are informative about mechanistic responses of human tissues to toxic
chemicals. The NRC committee is aware of the implementation challenges that the new toxicity-testing
paradigm would face." With this in mind, the EPA is currently conducting research to identify AOPs
that can serve as predictors of toxicity; the need to relate these AOPs to currently understood toxicity
endpoints is critical. Once appropriate AOPs are established, the EPA will be positioned to transition to
the methodologies recommended by the NRC. However, as the EPA pursues this path, the agency will
need to address several questions:
How well do the in vitro and in silico (computer simulation) tests translate to human systems?
How predictive of human pathways are the identified AOPs? Data on this is important to share
and make public.
How do the testing methods account for differences between in vitro/in vivo animal testing and
human toxicokinetics, particularly metabolism but also absorption, distribution, and excretion. For
instance, how are chemicals that are cleared through multiple pathways (renal, gastro-intestinal, etc.)
treated in the analysis? How do these testing methods account for chemicals that are actively reabsorbed
by renal organic anion transporters or those that are strongly bound to plasma proteins, lipids, etc.? And
how would the testing methods determine the toxicity of chemicals that are initially metabolized in one
organ and further metabolized to the ultimate toxic metabolite in another organ?
Given that there are multiple methods to estimate pharmacokinetic behavior (as described in
Rotroff et al. 2010) and since the results may differ based on which methods are employed, how will
decisions be made regarding which ones to use, their accuracy and certainty?
Are the proposed tests useful for chemicals that are stored in humans (e.g., adipose tissue depot
or other sites)?
How are human exposure characterization and biomonitoring data used in the prioritization and
testing of chemicals? Although the tests are designed to identify chemical hazards, if exposure is low or
non-existent, then how should the chemical be prioritized?
Where will human exposure data come from, and how will they be used (upper bounds, central
tendency, etc.)? and
How are existing data from the scientific literature incorporated into these AOPs and how will
the AOPs remain current?
The scientific acceptance of ToxCast approaches in a weight of evidence for decision-making will
depend on the accuracy, sensitivity and specificity of the computational toxicity testing for predicting
actual and potential human health effects. To assist in gaining acceptance, a transparent strategy should
13
-------
be developed for quantifying the endpoints upon which risk assessments will be based. The agency also
should indicate what issues should be considered for EPA applications such as chemical screening,
prioritization, risk assessment and green chemistry. Finally, the interactions between ORD and the
various EPA programs (e.g., pesticides, water or toxics) that will use ToxCast data are commendable
and should continue in order to understand what would make such data most useful. In addition to
providing opportunities for program office scientists to spend time in the ORD laboratories to become
familiar with the ToxCast program, extensive remote learning and training modules could be developed
to reduce the cost and logistic challenges. This may also serve to engage more EPA scientists outside of
the ORD laboratories located in Research Triangle Park, North Carolina.
3.4. Communicating CompTox Approaches and Outputs
Study Question 4. How should the use of the CompTox program be effectively communicated to
stakeholders? How can the communication be enhanced?
The EPA appears to be doing a very thorough job of communicating to stakeholders about the CompTox
program. The agency has created multiple web-based learning tools and models such as webinars and
dashboards for the interested public to learn more about the program and to access the data it is
generating. The EPA has actively sought input from stakeholders as it developed the CompTox program
and also has disseminated information to the scientific community through publications and
presentations at scientific conferences. The EPA established the Computational Toxicology
Communities of Practice, composed of more than 300 people from over 50 public and private sector
organizations that are interested in the application of computational toxicology and exposure science in
risk assessments. The SAB commends this effort; it is a powerful tool for keeping up with technical
issues that the EPA is confronting and addressing as a part of the ToxCast program.
EPA's communication effort has focused on two areas. First is conveying the importance of the
approach and the value of the strategy to stakeholders including the public. Many in the regulated
community have worked at developing computational toxicology models of various kinds, often quite
specific to their products; they are, obviously, convinced of the strength of the approach or they would
not be pursuing it. If stakeholders are included in the process of development, validation and application
of these methods in a collaborative fashion, they may be more likely to accept the results and provide
constructive feedback. Second is providing the data to all stakeholders, including the general public. The
CompTox website (http://www.epa.gov/ncct/) is relatively easy to navigate, but it would be useful to
provide some information about strategies for extracting relevant data.
As EPA moves forward with the development of the CompTox program, communication can be
enhanced by being transparent about the limitations and uncertainties in the use of ToxCast assays to
predict any particular endpoint in isolation and in combination with data from other ToxCast assays, and
providing a broader understanding of what is known about a chemical's biological activity based upon
ToxCast data in conjunction with Structure Activity Relationships (SAR), in vivo testing, etc. It may
also be useful to provide stakeholders with some summary statistics about the results, perhaps along the
lines of AOPs, with a transparent, easily accessible (e.g., on a website) location for the details of the
testing and the raw data. However, it should be kept in mind that uninitiated evaluators of large datasets
are often daunted by the sheer volume of data and may not consider the quality and limitations of those
data. As ExpoCast develops, the website (http://www.epa.gov/ncct/expocast/) should incorporate
estimates of exposure to chemicals and mixtures (especially upper bounds if possible) potentially
stratified by age, gender, regions of the country, population density (rural, suburban, and urban),
ethnicity and so forth.
14
-------
Outreach to epidemiologists and clinical investigators needs to be part of the communication process. It
may be difficult for some health scientists to discern the potential relevance of computational toxicology
to human exposure and health effects. Data generated and provided by EPA (and collaborators) can be
used to demonstrate that the tests utilized are relevant to human health effects. In addition, the agency
should clarify in what situations the data may fall short and be inadequate. For instance, there is a higher
level of uncertainty for specific AOPs, outcomes and/or for specific classes of chemicals. Thus, it is
essential to combine ToxCast outputs with data on toxicokinetics, particularly metabolism, of the
chemical in humans. Finally, biomonitoring data, exposure pathways, chemical source information and
human activity patterns related to human exposure need to be included in the assessment of chemical
risk.
The EPA should continue to partner with existing academic health science centers to disseminate
information on ToxCast. The agency can utilize existing relationships via community outreach and
translation cores which support gene and cell therapy clinical trials and include laboratories for basic
research, testing, scale-up, and clinical work. This would allow for the analysis of high-throughput data
and development of predictive modeling using ToxCast data sets. As AOPs are developed, it would also
be useful for the agency to develop partnerships with relevant professional societies or institutions. For
example, a group within the EPA developing an AOP on asthma would benefit from a partnership with
the American Lung Association or the National Heart, Lung, and Blood Institute to access physicians
and researchers in this field. The EPA also may benefit from more collaboration with international
agencies regarding data sources, data access and technology transfer. Another important group of
stakeholders to target for outreach are risk assessors and public health professionals in state and tribal
environmental and health agencies. Additional suggestions for research regarding communication and
achieving a broader understanding of the potential contributions and limitations of these approaches
include: (1) an evaluation of the pesticide stakeholder dialog process
(http://www.epa.gov/oppfeadl/cb/ppdc/tfabout) by an independent expert (group) in communication and
stakeholder participation to see what can be learned from that experience; and (2) pursuing a mental
model study to compare expert and public understandings of how CompTox findings could be
informative. Such a study might identify structural reasons why there might be communication
difficulties and how they could be addressed.
3.5. Other Issues
In addition to addressing the study questions, the SAB identified additional issues that should be
considered by the EPA as it continues to develop the CompTox research program and apply the program
outputs.
1) The SAB recommends that the agency clarify the goals and objectives for ToxCast with respect
to chemical screening, prioritization and risk assessment. How will application of ToxCast information
improve current EPA practice? Because risk assessments are conducted for a variety of purposes,
demands on the information base will necessarily differ among situations, but are there context-specific
criteria for assessing the utility of particular types of information? Resolution of some of these
structural issues could be a useful contribution of the ToxCast program even before it is producing
actionable information. The Deepwater Horizon provides an example of a programmatic need -
provision of information in emergency or other fast-moving settings - for which guidance is lacking.
2) There is a need to better delineate what ToxCast can and cannot contribute, both now and in the
future. Which contributions might be feasible over the next few years versus which ones will take longer
to develop? What chemicals will be evaluated? What health effects can be assessed at the current time
15
-------
and what health effects may be assessed in the future? The identification of critical pathways is an
important step toward clarifying key risk challenges - mixtures, interactions with background exposures,
existing conditions and susceptibilities - and it provides an attractive possible approach for using
ToxCast data to assess risk, but are there risks that may be obscured or ignored when an approach based
on critical pathways is used?
3) How well developed are EPA's capabilities for synthesizing and using fragmentary and
incomplete information? For the near term, ToxCast results will be quite limited and their best use
likely will be in combination with limited information from other sources. Current EPA practice tends to
be chemical-specific and to focus on particular types of information. How far along is the EPA in
developing multi-chemical and multi-factor risk assessments? A future vision for CompTox is that the
data will deliver a complete identification of critical pathways and a measure of the response along
them, but realizing such a vision is still remote. For some period of time, perhaps indefinitely, the
information provided by ToxCast will be fragmentary and new methods will be needed for its
interpretation. The primary challenges thus are transitional - how to build analytic structures that can
incorporate new kinds of information in incremental steps.
4) Analytic capabilities are a major consideration for the ToxCast program, but it is also important
to think about institutional capabilities for developing, organizing, and using the information. Are data
resources constantly updated and expanded and are there ongoing improvements in accessibility and
analytic flexibility? How can staff and scientist training and development in use of new ToxCast data be
accomplished? Is there an institutional culture that identifies opportunities for the use of new
information and is vigilant to detect warning signs concerning new issues and new difficulties? Is there
good communication between groups that might use the same or similar information and methods? Can
the institutions develop and support incremental changes? Can they engage stakeholders and other
governmental and non-governmental organizations as supporters of such change?
5) Critical data for steps in the transition from current risk assessment practices will only partly
come from ToxCast; those data must be synthesized with other, more familiar, types of information.
Data needs and requirements for data quality must be addressed as well.
6) How will the EPA handle contradictions between in vivo or human studies and the ToxCast data?
As the science moves forward, there may/will be results generated from in vivo and/or epidemiologic
studies that contradict or are not consistent with the ToxCast results. This is an inherent characteristic of
science and is not unique to the ToxCast program. What will be EPA's approach to handling comments
and perceptions regarding whether the ToxCast data either overestimated the risk of a chemical or did
not identify the hazard(s) of a chemical? What would the implications be for the ToxCast program and
the use of its outputs?
7) Clear and effective communication with all stakeholders, both within and outside the agency,
will be essential to the long term use of ToxCast and ExpoCast findings.
8) A community of scientists should be developed to provide feedback on ExpoCast in a parallel
fashion to ToxCast.
9) Finally, the SAB recommends that the EPA consider establishing an ongoing external advisory
process to help institutionalize a long term program built around the idea of incremental transformation.
An independent perspective on the current program and prospects for the future can be provided along
with constructive suggestions.
16
-------
GLOSSARY
AOP - or Adverse Outcome Pathways provide a scientific approach for linking mechanistic information
to responses considered relevant to risk assessment and management. AOPs refers to pathways that
would: (1) proceed from an initiating molecular event in which a chemical interacts with a biological
target (e.g., DNA binding, protein oxidation, or receptor/ligand interaction); (2) continue on through a
sequential series of biological activities (e.g. gene activation, or altered cellular chemistry or tissue
development); and (3) ultimately culminate in an adverse outcome of relevance to human or ecological
risk assessors (e.g., mortality, disrupted reproduction, cancer, or extinction).
Apical Effects or Endpoints - empirically verifiable outcomes of exposure, such as developmental
anomalies, breeding behaviors, impaired reproduction, physical changes and alterations in the size and
histopathology of organs, or death.
CompTox - or the EPA's Computational Toxicology Research Program is part of EPA's broader
Chemical Safety research efforts. The CompTox research program conducts innovative research that
integrates advances in molecular biology, chemistry and innovative computer science to more
effectively and efficiently rank chemicals based on risks. CompTox is researching new, more efficient,
ways to assess chemical safety for potential risk to human health and the environment that can replace
traditional chemical toxicity testing that is expensive, time consuming and uses a significant number of
animals.
CSS - or Chemical Safety for Sustainability is one of six transdisciplinary research programs in EPA's
Office of Research and Development which focuses on the use of systems-approaches to advance the
understanding of the links between exposures to chemicals and toxicity pathways that lead to the
development of disease.
ExpoCast - a collaborative effort across the exposure and risk assessment community to provide the
exposure science required for interpretation of high-throughput in vitro toxicity data. The overall goal of
this program is to develop novel approaches and tools for evaluating and classifying chemicals, based on
potential for biologically-relevant human exposure, to inform prioritization and toxicity testing.
Omics - a suffix that refers to a broad field of study in biology that includes molecular biology
specialties such as genomics, proteomics, and transcriptomics.
NCCT - or the National Center for Computational Toxicology is the largest component of EPA's
Computational Toxicology Research Program. It was established in 2005 to coordinate computational
toxicology research on chemical screening and prioritization, informatics and systems modeling.
Tox21 - a collaborative effort of the EPA, National Institutes of Environmental Health
Sciences/National Toxicology Program, National Institutes of Health and the Food and Drug
Administration that is pooling federal resources and expertise in the use of robotics technology to screen
thousands of chemicals for potential toxicity, use screening data to predict the potential toxicity of
chemicals and develop a cost-effective approach for prioritizing the thousands of chemicals that need
toxicity testing.
17
-------
ToxCast - A multi-year effort launched by EPA in 2007 to develop a cost-effective approach for
prioritizing the thousands of chemicals that need toxicity testing. ToxCast uses advanced science tools to
help understand how human body processes are impacted by exposure to chemicals and to determine
which exposures are most likely to lead to adverse health effects.
ToxPi - or Toxicological Priority Index is a tool which profiles the interactions of chemicals with
biological processes in ways the public and decision makers can easily understand. ToxPi is a flexible
prioritization support software tool that incorporates ToxCast bioactivity profiles, inferred toxicity
pathways, dose estimates and chemical structural descriptors to provide a visual representation of the
relative contribution of each data domain to an overall priority score.
ToxRef - a comprehensive relational database of mammalian toxicity studies developed by EPA's
Office of Research and Development in collaboration with EPA's Office of Pesticide Programs (OPP).
18
-------
REFERENCES
Gangwal, S., D.M. Reif, S. Mosher, P.P. Egeghy, J.F. Wambaugh, R.S. Judson, E.L. Hubal. 2012.
Incorporating exposure information into the toxicological prioritization index decision support
framework. Sci. Total Environ. 435-436: 316-25.
Judson, R., D. J. Dix, K. A. Houck, M. T. Martin, T. B. Knudsen, and R. J. Kavlock. 2010. Predictive
Signatures from ToxCast Data for Chronic, Developmental and Reproductive Toxicity
Endpoints. Presented at Society of Toxicology Annual Meeting, Salt Lake City, UT, March 07 -
11,2010.
National Research Council. 2007. Toxicity Testing in the 21st Century: A Vision and a Strategy.
Washington, DC: The National Academies Press. ISBN-10: 0-309-15173-2, ISBN-13: 978-0-
309-15173-3.
National Research Council. 2009. Committee on Improving Risk Analysis Approaches Used by the U.S.
EPA, "Front Matter." Science and Decisions: Advancing Risk Assessment. Washington, DC: The
National Academies Press, ISBN-10: 0-309-12046-2, ISBN-13: 978-0-309-12047-0.
Rotroff, D. M., B. A. Wetmore, D. J. Dix, S. S.Ferguson, H. J. Clewell, K. A. Houck, E. L. Lecluyse, M.
E. Andersen, R. S. Judson, C. M. Smith, M. A. Sochaski, R. J. Kavlock, F. Boellmann, M. T.
Martin, D. M. Reif, J. F. Wambaugh, R. S. Thomas. 2010. Incorporating human dosimetry and
exposure into high-throughput in vitro toxicity screening. Toxicol Sci. 117(2):348-58. Epub 2010
Jul 16.
Thomas, R.S., M. B. Black, L. Li, E. Healy, T-M Chu, W. Bao, M. E. Andersen, and R. D. Wolfmger.
2012. Response to "Incorporating Biological, Chemical, and Toxicological Knowledge Into
Predictive Models of Toxicity" Toxicol. Sci. 130 (2): 442-443. Epub 2012 September 20.
U.S. EPA (U.S. Environmental Protection Agency). 2009. The U.S. Environmental Protection Agency's
Strategic Plan for Evaluating the Toxicity of Chemicals, March 2009. Office of the Science
Advisor. EPA/100/K-09/001 I, 2009. Available at
http://www.epa.gov/osa/spc/toxicitytesting/docs/toxtest strategy 032309.pdf
U.S. EPA (U.S. Environmental Protection Agency) Board of Scientific Counselors. 2010. Review of
ORD's National Center for Computational Toxicology. Available at
http ://epa. gov/osp/bosc/pdf/ctox 1004rpt.pdf
U.S. EPA (U.S. Environmental Protection Agency) Science Advisory Board. 2011. Science Advisory
Board Comments on the President's RequestedFY 2012 Research Budget. EPA-SAB-11 -007.
Wetmore, B.A., J.F. Wambaugh, S.S. Ferguson, M.A. Sochaski, D.M. Rotroff, K. Freeman, H.J. Clewell
3rd, D.J. Dix, M.E. Andersen, K.A. Houck, B. Allen, R.S. Judson, R. Singh, R.J. Kavlock, A.M.
Richard, R.S. Thomas. 2012. Integration of Dosimetry, Exposure, and High-Throughput
Screening Data in Chemical Toxicity Assessment. Toxicol Sci. 2012 Jan;125(l):157-74. Epub
2011 Sep26.
19
-------
APPENDIX A. HISTORY OF THE COMPTOX PROGRAM
[The information contained in this appendix was extracted from the U.S. EPA Office of Research and
Development Computational Toxicology Research Program Implementation Plan for Fiscal Years 2009 to 2012,
http://epa.gov/ncct/download_files/basic_information/CTRP2_Implementation_Plan_FY09_12.pdf]
History of the Computational Toxicology Research Program (CTRP) and the National Center for
Computational Toxicology (NCCT)
A. Defining the Mission of Computational Toxicology at EPA
Computational toxicology applies mathematical and computer models and molecular biological and
chemical approaches to explore both qualitative and quantitative relationships between chemical
exposure and adverse health outcomes. Recent technological advances make it possible to develop
molecular profiles using high-throughput and high-content methods that identify the impacts of
environmental exposures on living organisms.
CTRP Mission Statement: To integrate modern computing and information technologies with
molecular biology to provide the agency with decision support tools for high-throughput risk
assessment.
With these tools, scientists can produce a more detailed understanding of the hazards and risks of a much
larger number of chemicals. The integration of modern computing with molecular biology and chemistry
is enabling scientists to better understand a chemical's progression through the environment to the target
tissue within an organism and, ultimately, to the key steps that trigger an adverse health effect. Currently,
risk estimates most often are based on gross outcomes of disease, such as occurrence of cancer, a
neurological disorder, or a visible birth defect. The National Research Council (NRC) in its 2007 report,
Toxicity Testing in the 21st Century: A Vision and a Strategy, called for a concerted effort to move
toxicology from a primarily descriptive science to a more predictive one by utilizing largely human based
in vitro studies to understand the biological pathways by which chemically induced diseases occur. The
Environmental Protection agency's Computational Toxicology Research Plan is working to aid this
transformation by evaluating the key molecular changes occurring in the function of critical human
toxicity pathways within cells, tissues, individuals, and populations. The key will be connecting these
changes quantitatively and systemically to the types of adverse health effects that have been the
traditional basis of EPA risk assessments and to use this understanding to reduce the current uncertainties
in the extrapolation of effects across dose, species, and chemicals. The Office of Research and
Development CTRP is composed of three main elements. The largest component is the National Center
for Computational Toxicology , which was established in 2005 to coordinate research on chemical
screening and prioritization, informatics, and systems modeling. The second element consists of related
activities in the National Health and Environmental Effects Research Laboratory (NHEERL) and
National Exposure Research Laboratory (NERL) of ORD. The third and final component consists of
academic centers working on various aspects of computational toxicology and funded by the EPA
Science to Achieve Results (STAR) Program.
The rapid and ongoing success of the CTRP is impacting hazard and exposure identification and helping
to close data gaps, identify toxicity pathways, suggest modes of action, and make for more efficient
utilization of precious resources on the highest priority chemicals. Besides these initial outcomes from
the higher throughput approach of the CTRP, informatics and modeling efforts will provide more in-
A-l
-------
depth and quantitative molecular understanding of how biological systems respond to environmental
chemicals. These knowledge databases and in silico tools will reduce or quantify uncertainties relating to
biological susceptibility, species differences, and dose response as part of a faster and more intelligent
targeted testing paradigm in support of quantitative risk assessments.
B. Timeline of CTRP Development
In fiscal year 2002, Congress ordered a redirection of $4 million from available EPA funds, "... for the
research, development and validation of non-animal alternative chemical screening and prioritization
methods, such as rapid, non-animal screens and Quantitative Structure Activity Relationships (QSAR),
for potential inclusion in EPA's current and future relevant chemical evaluation programs."
To fulfill this directive, EPA embarked on development of a research program that (1) was consistent
with the Congressional mandate, (2) complemented and leveraged related ongoing agency sponsored
efforts to consider alternative test methods, (3) further advanced the research to support the agency's
mission, and (4) would not duplicate the mission and programs in this area conducted by other agencies
(see Figure 1 below for a timeline of CTRP development).
Thus the CTRP was initiated to target these goals and, in the process, significantly advance toxicology
and risk assessment as currently practiced by the agency and the broader environmental sciences
community. In FY2002 to 2003 pilot projects were funded to demonstrate computational toxicology
could be adapted to the study of endocrine disrupters. Early successes of these efforts included
refinement of estrogen receptor ligand binding data for development of quantitative structure-activity
models, evaluation of EPA-developed cell lines for detecting estrogen and androgen activities from
various species and the development of an alternative test method for evaluating effects on
steroidogenesis in H295R cells (see EDSP Assay Status).
With increasing attention to and expectations for the CTRP in FY2003, ORD developed^ Framework
for a Computational Toxicology Research Program, which was published in FY2004 and provided
strategic direction for the program. This document was the product of a cross- ORD design team of
scientists and was reviewed by the Science Advisory Board (SAB). ORD hosted a workshop in Research
Triangle Park, NC, in late FY2003 to introduce the CTRP framework from which the three objectives for
EPA computational toxicology were translated into the three initial long-term goals (LTGs) for the
program:
1) risk assessors use improved methods and tools to better understand and describe the linkages of the
source-to-outcome paradigm,
2) EPA program offices use advanced hazard characterization tools to prioritize and screen chemicals
for toxicological evaluation, and
3) EPA assessors and regulators use new and improved methods and models based on the latest science
for enhanced dose-response assessment and quantitative risk assessment.
With issuance of the CTRP framework, ORD began the process of implementing a more formalized
program. A cross-agency working group, the Computational Toxicology Implementation Steering
Committee (CTISC) was formed in FY2004 to oversee the selection and funding of projects across ORD.
Seven cross-ORD projects were initiated as result of CTISC action, and these "new start" projects
became a critical component of the first generation CTRP implementation plan.
A-2
-------
In October 2004, then EPA Science Advisor and Assistant Administrator for ORD, Dr. Paul Oilman,
announced the formation of the NCCT, which began official functions in February 2005. The
announcement states:
"The Center will advance the science needed to more quickly and efficiently evaluate the potential risk of
chemicals to human health and the environment. The Center will coordinate and implement EPA's
research on computational toxicology to provide tools to conduct more rapid risk assessments and
improve the identification of chemicals for testing that may be of greatest risk."
NCCT quickly became the hub for ORD CTRP research. NCCT formed key partnerships with the other
laboratories and centers within ORD, which formed the second critical element. Partnerships with
NHEERL, NERL, the National Risk Management Research Laboratory, and the National Center for
Environmental Assessment helped in the execution of not only the seven cross-ORD "new start" projects
awarded by the CTISC in 2004, but also in several original NCCT-led projects, including the Distributed
Structure-Searchable Toxicity Database (DSSTox), Toxicity Forecasting Project (ToxCast), Toxicity
Reference Database (ToxRefDB), and the virtual tissues projects looking at liver and embryo.
FIGURE 1
ORD Computational Toxicology
Research Program Development
CTRP Framework
CTISC
1st CTRP
Implementation Plan
ToxCast Design
Congressional Redirect Seven CTRP Proof of Concepts 1st and 2nd STAR Centers
EDC Proof of Concepts STAR Systems Biology RFA BOSC II
NCCT Staffing Complete
NAS 21st Century Vision
EPA 21st Century Strategy
Tox21
MOU
DSSTox
v2
3rd STAR Center
BOSC III
CTRP Design Team
SAB and BOSC reviews RTP CTRP
Workshop STAR HTS RFA
NCCT Launch
DSSTox vl
ToxCast Concept
BOSC I
1st Title 42 Hire
Chemical Prioritization
CoP
ToxCast Launch
v-Tissues Launch
Intl Science Forum
2nd CTRP
Implementation
Plan
ACToR Launch
ExpoCast Launch
ToxRefDB Launch
ToxCast I Complete
1st ToxCast Summit
ToxCast II
Launch v-
Tissues 09
4th STAR Center
BOSC IV
Thomas RS, Black MB, Li L, Healy E, Chu TM, Bao W, Andersen ME, Wolfinger RD. 2012. A comprehensive
statistical analysis of predicting in vivo hazard using high-throughput in vitro screening. Toxicol Sci 128:398-
417.
A-3
-------
APPENDIX B. ASSAY PERFORMANCE CRITERIA
1. Name of the assay;
2. Description of assay design;
3. Name of company that developed the assay;
4. Information on any proprietary constraints of the assay;
5. Positive control and other agents used to characterize the assay;
6. Dynamic range of the assay;
7. Where the endpoint fits within one or more AOPs;
8. Related CompTox endpoints (i.e., endpoints likely to be within the same AOP or that are
indicative of similar biological activity but in an independent test system);
9. Interpretative value of the endpoint if altered in isolation;
10. Interpretative value if altered in conjunction with other "aggregated" endpoints;
11. Rate of false positive and negative results if it is to be used for predictive purposes (e.g.,
to forecast in vivo endocrine activity);
12. Shape of the dose-response curve (e.g., monotonic, non-monotonic, threshold, linear,
hermetic and essential);
13. Potential for the endpoint to be used as a biomarker in toxicity testing or in epidemiology
studies;
14. Whether the endpoint is also affected by disease processes that might potentially lead to a
chemical/disease interaction;
15. Limitations and uncertainties of the endpoint; and
16. Cross reference with other assays that assess the same endpoint(s) and comparison of
reliability of the assay in comparison.
B-l
------- |