science in ACTION
BUILDING A SCIENTIFIC FOUNDATION FOR SOUND ENVIRONMENTAL DECISIONS
www.epa.gov
Ensuring the Reliability and Relevance of High-Throughput Assays
A Case Study Testing for Potential Endocrine Activity of Chemicals
March 2,2012
Background
Tens of thousands of chemicals are
currently in commerce, and
hundreds more are introduced every
year. To date, only a small fraction
of chemicals have been fully
assessed for potential risk. For
example, EPA's Endocrine
Disruption Screening Program
iKDSl'i has thousands of chemicals
that need to be screened for
endocrine disruption and could use
a faster method to prioritize these
chemicals for additional testing.
There are two major issues with
current toxicity testing methods that
limit the ability of scientists to
assess these chemicals efficiently
and effectively:
1.	Current in vivo (animal) testing
is expensive, time consuming
and often uses large numbers
of animals which has led to a
backlog of untested chemicals.
Results from animal testing are
then extrapolated to evaluate
dose, species and life stage
differences which lead to
uncertainties when assessing
hazards and risks.
2.	Current whole animal testing
cannot provide clear
mechanistic insight into human
toxicity pathways. However,
high-throughput in vitro testing
methods can provide data on
activities at the human cellular
level.
The U.S. EPA and its partners have
developed methods of toxicity
testing that will help usher chemical
toxicity testing into the 21?t century.
These testing methods (referred to
as "To\2l" and Toxicity Forecaster
Definitive Tests
Chemical
Library
ft
r Validate	cnemica
Set'	HTS Ass;
HTS Assays	jj|
11/
High evidence
chemicals from
HTS Assay Set i
Develop Set i	Run Set i
HTS Assays	HTS Assays
tt	I V
Learn and	H	VM
	 Improve ^
>	1 for	AI1 ch
ase use of	ro[
Chemical
Library
Propose use of
new research-level
HTS Assays
"ToxCast") include high-throughput
Learn and
Improve
for
Set i+1
of HTS assays
chemicals
recycled for
HTS Assay Set i+1
Overview
Fig. 1. Conceptual model of a continuously improving battery of HTS assays to be
used for prioritization
screening (HTS) assays and robotic
technologies that significantly
reduce the time, expense and
resource demands of current
chemical toxicity testing. Tox21 is
currently screening approximately
10,000 chemicals using HTS assays.
A key obstacle to adoption of these
HTS methods by regulators is the
need for validation - that is,
demonstration that the new
approaches are relevant, reliable and
fit for the intended purpose. Formal
validation is needed to make sure
that the data can stand up to legal
challenges within regulatory
processes. Although they are slow
and inefficient, current toxicity
testing methods are accepted by the
regulated community. In order for
all stakeholders to accept the usage
of Tox21 and ToxCast assays to
help make chemical regulatory
decisions, these new methods must
be validated to ensure they are of
high quality.
Perspectives on Validation of High-
Throughput Pathway-Based .faws
Supporting the 21st Century
Toxicity Testing Vision by Judson et
al. describes proposed alterations to
standard validation practices that
could be used for HTS assays.
Alterations are suggested to the
validation process developed by
Interagency Coordinating
Committee on the Validation of
Alternative Methods (ICCVAM),
the European Center for the
Validation of Alternative Methods
iPCV.WI i and the Organization for
Economic Co-operation and
Development (QECD). The
validation process includes defining
the testing, conducting intra- and
inter-laboratory testing and defining
the relevance of the work.
Although it is important to ensure
the validity of HTS assays, this
formal validation process is lengthly
and has can be a barrier to
innovation.

-------
It is desirable to develop a
streamlined validation process to
evaluate both the relevance and
reliability of HTS assays.
"Relevant" means that an assay
must test some aspect of biology
that will help assess the safety of a
chemical. "Reliable" means that the
assay must produce similar results
over time, across reagent batches,
etc. HTS assay data generally
provides quantitative read-outs with
a focused and mechanistically
simple interpretation. These
attributes should make evaluation of
the relevance and reliability of the
HTS assays, peer review and
decisions on regulatory acceptance
relatively straightforward.
Judson et al. presents two ideas that
would lower the cost and time
requirements of HTS validation and
suggests criteria for assay
evaluation and performance without
compromising reliability:
1.	Eliminating the requirement
for round-robin cross-
laboratory testing.
2.	Developing a streamlined,
online peer review process.
The argument for eliminating the
need for cross-laboratory validation
is based on three premises. Most
importantly, with HTS assays one
can run many positive and negative
reference chemicals both during the
validation phase and during routine
production which provides ongoing
quality assurance. Secondly, many
HTS assays are run in laboratories
with expensive customized, one-of-
a kind equipment that could not be
replicated elsewhere, making true
replication of assay protocols
difficult. Thirdly, for many HTS
assays, the originating lab has the
capacity to run all of the world's
chemicals to be tested for a
particular assay, thereby eliminating
the need to have other labs be
validated.
Developing an online peer review
process would simplify review of
any number of HTS assays because
all the data would be in a common
format in a centralized web-
accessible database. Any group
wishing to propose a new assay for
use in a regulatory application
would then have immediate access
to all existing validation information
on similar assays, and could submit
their validation package into the
central system to be queued up for
subsequent peer review. This rapid
and continuous preparation of
validation documentation would
facilitate the continuous
improvement of assays to be used in
regulatory prioritization [see Fig. 1],
One real world application of using
HTS data discussed in Judson's
paper is using ToxCast and Tox21
to prioritize chemicals for EPA's
Endocrine Disruption Screening
Program (EDSP). The prioritization
would determine which of the
thousands of potential endocrine
disrupting chemicals are in most
need of further testing.
Using In Vitro High-Throughput
Screening Assays to Identify
Potential Endocrine Disrupting
Chemicals (Rotroff et al)
investigated using EPA's ToxCast
assays for assessing chemicals for
estrogen, androgen, steroidogenic
and thyroid disruption. The study
used assay data from the 309
chemicals in ToxCast Phase I and
data on an additional 23 reference
chemicals and compared it to results
from animal toxicity tests. The
study demonstrates that current
ToxCast HTS assays can accurately
identify chemicals with the potential
to interact with the estrogenic and
androgenic pathways, with a
balanced accuracy of over 90%.
These results suggest that current
ToxCast assays can accurately
identify chemicals with potential to
interact with the estrogenic and
androgenic pathways, and could
help prioritize chemicals for EDSP
T1S assays.
Conclusions
These two potential alterations to
standard assay validation practice
proposed by Judson et al. could
significantly streamline the
acceptance criteria for new HTS
technologies used for prioritization
applications. The elimination of the
requirement to do cross-laboratory
testing already has precedent, while
the development of an online peer
review process offers not only
greater efficiency, but also
additional transparency relative to
current practice.
Although both of these
recommendations might be
considered controversial due to their
departure from current validation
practice, both suggestions merit
serious consideration given the
significant advantages offered by
HTS assays.
General Information
U.S. EPA and its partners developed
Tox21and ToxCast to create new
methods for efficiently screening
chemicals and prioritizing limited
testing resources to chemicals that
represent the greatest potential
hazard to human health and the
environment.
Tox21 research is performed by the
U.S. EPA National Center for
Computational Toxicology in
collaboration with the National
Institutes of Environmental Health
Sciences/National Toxicology
Program, the National Institutes of
Health (NIH)/National Human
Genome Research Institute, the NIH
Center for Translational
Therapeutics (NCTT) and the Food
and Drug Administration (FDA).
Originating Organization for Fact
Sheet
EPA
Reference
Judson, et al. "Perspectives on
Validation of High-Throughput
Pathway-Based Assays Supporting
the 21st Century Toxicity Testing
Vision" (2012).
Rotroff et al. "Using In Vitro High-
throughput Screening Assays to
Identify Potential Endocrine
Disrupting Chemicals" (2012).
Contact:
Monica Linnenbrink
Office of Research & Development
Chemical Safety for Sustainability
(919)-541-1522
linnenbrink.monica@epa.gov

-------