Toxicity Reference Database Version 3.0 User Guide


Office of Research arid Development

Center for Computational Toxicology and Exposure

v-/EPA

United States
Environmental
Protection Agency

Toxicity Reference
Database Version 3.0

User Guide

-------
EPA Report Number 601B25001
August 2025

Toxicity Reference Database
Version 3.0
User Guide

U.S. Environmental Protection Agency
Office of Research and Development
Center for Computational Toxicology and Exposure
Research Triangle Park, North Carolina

-------
Table of Contents

Table of Contents	2

Purpose	3

Abstract	3

Disclaimer	3

Overview	4

History	4

Summary of ToxRefDB v3.0	5

Table 1: Summary Coverage	5

Figure 1: Study-Level Data Landscape	7

Figure 2: Chemical-Level Data Landscape	7

Changes between ToxRefDB v2.1 and v3.0	8

Table 2: Changes between ToxRefDB v2.1 and v3.0	8

Accessing information in ToxRefDB	9

Installing PostgreSQL and Loading ToxRefDB	9

Installing PostgreSQL	9

Create Database and Import Schema	9

Example SQL Queries	10

Programmatic Access	11

Python	11

R	13

CompTox Chemicals Dashboard	14

CTX Hazard API	15

Database Structure	16

Figure 3: ToxRefDB v3.0 ERD	16

Figure 4: Simplified Schema Overview	17

Data Curation Process	18

Legacy Curation	18

Figure 5: Data Extraction and Review Workflow	18

Application-Based Curation	19

Quality Assurance in Data Extraction	19

Efforts to Reduce Error Rate	19

Unit Standardization	20

Study Reliability with ToxRTool	20

Table 3: ToxRTool Guideline Adherence Score	20

Guideline Profiles	20

Table 4: Series 870 - Health Effects Test Guidelines Guideline Profile Coverage	21

DNT Vocabulary and Guideline Profile Curation	23

Endpoint Terminology	26

Figure 6: Hierarchical endpoint terminology example	26

Ontology mappings	26

Figure 7: Cross-referenced Terminology Sources	27

Negative Endpoints and Effects	28

Table 5: Endpoint Observation Status	28

Figure 8: Decision tree for identification of negative endpoints and effects	29

Figure 9: Example Observation Status Interpretation	29

Data Dictionary	30

Acknowledgements	38

References	39

-------
Purpose

This documentation describes how to access and use the Toxicity Reference Database (ToxRefDB)
version 3.0. The latest data can be accessed through EPA's Clowder site: https://clowder.edap-
cluster.com/datasets/61147fefe4b0856fdc65639b#folderld=6850327be4b096bca8848301. More

information about ToxRefDB and its development can be found in the publications below.

• Feshuk M, Kolaczkowski L, Watford S, Paul Friedman K. ToxRefDB v2.1: update to curated in
vivo study data in the Toxicity Reference Database. Front Toxicol. 2023 Sep 11;5:1260305. doi:
10.3389/ftox.2023.1260305. PMID: 37753522; PMCID: PMC10518696.

• Watford, S., Pham, L.L., Wignall, J., Shin, R., Martin, M.T., and Friedman, K.P. (2019). ToxRefDB
version 2.0: Improved utility for predictive and retrospective toxicology analyses. Reproductive
Toxicology, 89, 145-158. DOI: 10.1016/j.reprotox.2019.07.012

• Pham, L.L., Watford, S., Friedman, K.P., Wignall, J.A., and Shapiro, A.J. (2019). Python BMDS:
A Python interface library and web application for the canonical EPA dose-response modeling
software. Reproductive toxicology. DOI: 10.1016/j.reprotox.2019.07.013

Abstract

ToxRefDB contains in vivo study data from over 6400 guideline or guideline-like studies for over 1200
chemicals. It largely comprises curated animal study data from repeat dose studies conducted according
to Health Effects Series 870 guidelines, with many studies (over 3,800) orginating from registrant-
submitted toxicity studies known as data evaluation records (DERs) from the U.S. EPA's Office of
Pesticide Programs (OPP). By employing a controlled vocabulary for enhanced data quality, ToxRefDB
serves as a resource for study design, quantitative dose response, and endpoint testing status
information given guideline specifications. The database can serve as a resource for retrospective and
predictive toxicology applications, such as read-across and quantitative structure activity relationship
modeling as well as facilitate validation of in vitro high throughput screening of chemicals.

Disclaimer

The research project described in this document has been funded by the United States Environmental
Protection Agency (U.S. EPA). Mention of products, companies or trade names does not indicate
endorsement or recommendation for use by the Agency.

-------
Overview

The Toxicity Reference Database (ToxRefDB) serves as a resource for structured animal toxicity data
for many retrospective and predictive toxicology applications. ToxRefDB contains in vivo study data from
over 6000 guideline or guideline-like human health relevant studies for over 1000 chemicals. The goal of
ToxRefDB is to provide a public database that better supports the needs of predictive toxicology by
increasing the qualitative and quantitative information available and by facilitating the interoperability of
legacy in vivo hazard information with other tools and databases.

The study types covered in ToxRefDB include repeat dose study designs utilizing various administration
routes (predominantly oral) namely:

•	chronic (CHR; typically 1-2 year exposures depending on species and study design) conducted
predominantly in rats, mice, and dogs;

•	subchronic (SUB; 90 day exposures) conducted predominantly in rats, mice, and dogs;

•	subacute (SAC; 14-28 day exposures depending on the source and guideline) conducted
predominantly in rats, mice, and dogs;

•	prenatal developmental (DEV) conducted predominantly in rats and rabbits;

•	multigeneration reproductive toxicity studies (MGR) conducted predominantly in rats;

•	reproductive (REP) toxicity studies conducted largely in rats;

•	developmental neurotoxicity (DNT) studies conducted predominantly in rats;

and a small number of studies with designs characterized as acute (ACU), neurological (NEU), or "other"
(OTH).

Many of the studies (3607) come from registrant-submitted toxicity studies known as data evaluation
records (DERs) from the U.S. EPA's Office of Pesticide Programs (OPP). Accordingly, the majority of
chemicals in the database are categorized as pesticides. Ongoing curation efforts since 2009 have
expanded ToxRefDB to incorporate toxicity studies from ten additional sources, including the National
Toxicology Program (NTP), peer-reviewed primary research articles (OpenLit), and pharmaceutical pre-
clinical toxicity studies (Pfizer, Sanofi, GSK, Merck), among others (RIVM, PMRA, unpublished and
unassigned sources). These efforts have broadened the chemical coverage of the database beyond
pesticides.

ToxRefDB serves as a resource for study design, quantitative dose response, and endpoint testing status
information given guideline specifications from the US Environmental Protection Agency (US EPA) and
the National Toxicology Program (NTP) headquartered at the National Institute of Environmental Health
Sciences. The legacy and current data curation workflow is described in more detail in later sections. An
important component of ToxRefDB is its controlled vocabulary for studies and effects observed for
enhanced data quality.

History

The first version of ToxRefDB (ToxRefDB v1.0) was initially released as a series of spreadsheets, which
are still available on EPA's FTP site and referenced in FigShare
(https://doi.org/10.23645/epacomptox.6062545.v1). ToxRefDB then underwent significant updates,
including extraction of quantitative (i.e. dose-response) data, that are described in Watford et al. (2019)
and was released as ToxRefDB v2.0. ToxRefDB v2.0 and its associated summary files can be found
here: https://doi.org/10.23645/epacomptox.6062545.v3

ToxRefDB v2.1 represented a minor update of ToxRefDB v2.0 to address issues discovered with the
compilation script that caused some extracted values, notably toxicity effects, not to import properly from
the original MS AccessDB curation files, such as failure to import some effects. Although the overall total
number of studies and chemicals remained unchanged, the v2.1 update included additional data as
previously curated studies with extracted dose treatment groups and effects were made fully accessible.
The additional data improved the utility of ToxRefDB as a resource for curated legacy in vivo information
by providing more complete information of the past animal studies conducted. The minor update was

-------
described in the Feshuk et al. (2023) publication. ToxRefDB v2.1 and its associated summary files can
be found here: https://doi.org/10.23645/epacomptox.6062545.v4

Summary of ToxRefDB v3.0

ToxRefDB v3.0 contains summary information from 6409 guideline or guideline-like human health
relevant studies for 1228 chemicals. The quantitative (i.e. dose-response) data has now been completed
for 4320 studies (indicated with processed=TRUE) with plans to extract and release the remaining data
in subsequent database versions.

ToxRefDB v3.0 represents the next iteration of the ToxRefDB project featuring several key updates:

• Improved Curation Workflow: New data in ToxRefDB v3.0 was curated using an application-
driven workflow. The Data Collection Tool (DCT), an Oracle APEX software, was developed and
utilized to support continued curation efforts. In the future, Health Assessment Workspace
Collaborative (HAWC) is anticipated to be leveraged to support ToxRefDB curation, data access,
and visualization.

• Vocabulary Expansion for New Study Type: Expanded controlled vocabulary and a new
guideline profile was curated to capture the OPPTS 870.6300 developmental neurotoxicity (DNT)
guideline and "non-guideline" studies.

• MySQL -> PostgreSQL Migration: The database was migrated from MySQL into PostgreSQL
format. PostgreSQL is a more broadly used, open-source solution, requested by ToxRefDB users,
for enhanced data interoperability.

• New Study Extractions: Data extraction from source documents (e.g. OPP DERs and NTP
PFAS reports (TOX-96 and TOX-97)) expanded both the chemical and study coverage of
ToxRefDB.

• Improved Data Provenance: Source chemical information was added as new JSON column with
confirmed DSSTox Substance Identifiers (DTXSIDs) mappings stored in the chemical table.

• Improved Citation Management: NTP source report identifiers were added as ToxRefDB study
source ids. Efforts are underway to expand Health and Environmental Research Online database
(HERO) interoperability for improved citation management.

• Quality Control: Quality assurance, such as vocabulary standardization to continue to provide
more detailed effect and study-level information allowing for a more streamlined, interoperable
database efforts. Errors were identified for systematic correction, such as non-standard values in
free-text fields and effect size comparisons to flag poorly extracted data.

The .sql export of ToxRefDB 3.0 and summary files are available for public download here:
https://clowder.edap-

cluster.com/datasets/61147fefe4b0856fdc65639b#folderld=6850327be4b096bca8848301

Herein we provide the reader with a summary of the scope and coverage of the database. ToxRefDB
was filtered to present only data where a full curation with guideline profile observations was complete.
This is achieved using a 'processed' flag set to TRUE within the study table.

Table 1 presents the number of chemicals and number of studies for each study source, study type, and
species. Study type abbreviations are as follows: CHR = Chronic, DEV = Prenatal-Developmental, MGR
= Multigeneration Reproductive, SAC = Subacute, SUB = Subchronic, DNT = Developmental
Neurotoxicity, NON = Non-guideline.

Table 1: Summary Coverage

Study type

Study source

Species

Number of studies

Number of chemicals

CHR

NTP

mouse

178

173

rat

169

164

OpenLit

mouse

rat

-------


OPP DER

dog

337

304





hamster

4

3





mouse

341

302





primate

1

1





rat

399

329

Total CHR

1438

564

DEV

NTP

mouse

1

1





rabbit

3

3





rat

6

6



OpenLit

rat

1

1



OPP DER

mouse

27

16





rabbit

481

372





rat

560

479



Other

mouse

1

1





rabbit

1

1





rat

4

4

Total DEV





1085

541

DNT

OPP DER

rat

79

74

Total DNT





79

74

MGR

OpenLit

rat

1

1



OPP DER

mouse

2

2





rat

376

339



Other

rat

19

19

Total MGR

361

331

NON

OPP DER

mouse

1

1





rabbit

1

1





rat

79

49

Total NON

81

50

SAC

NTP

mouse

29

26





rat

38

37



OPP DER

dog

2

2





mouse

14

14





rabbit

6

6





rat

48

38

Total SAC





137

86

SUB

NTP

hamster

1

1





mouse

119

107





rat

127

114



OpenLit

mouse

2

2





rat

4

4



OPP DER

dog

227

205





hamster

4

4





mouse

144

131





primate

3

3





rabbit

5

4





rat

466

369

Total SUB





1102

533

Database totals





4320

849

-------
Figure 1 depicts a breakdown of studies by study source, study type, and species.
Figure 1: Study-Level Data Landscape

Study Type

1500

Species

Study Source

4000

2000

3000

(/)

'-5 1000
(/)

CD
_Q

500

CO
.0
'~o

CD
_Q

E 1000

w
0
"O
13

O 2000

L—

0
_Q

1000

671

Study Type

* 6C V7/

-------
Changes between ToxRefDB v2.1 and v3.0

The following table details a summary of differences between ToxRefDB v2.1 and v3.0. ToxRefDB v3.0
contains summary information from 6341 guideline or guideline-like human health relevant studies for
1228 chemicals. As part of the ToxRefDB 2.0 update, quantitative (i.e. dose-response) data were
extracted. For v3.0, this curation has been completed for 4320 studies and 849 chemicals (indicated with
processed=TRUE) with plans to extract and release the remaining data in subsequent database versions.
Overall, ToxRefDB grows by 235680 database rows from v2.1 to v3.0, or a 40% increase in database
size, likely attributed to the high number of effects and lifestages included in DNT study type extractions.

Table 2: Changes between ToxRefDB v2.1 and v3.0

Output

v2.1

v3.0

Change

Total number of studies with complete curation

3871

4320

+449

Number of studies with extracted effects

3662

4070

+408

Total number of chemicals

748

849

+ 101

Total database rows, including studies with no extracted

344868

580548

+235680

effects

Total effects extracted

335281

368437

+33156

Dose treatment groups with effects

40905

45553

+4648

Unique effects: Cholinesterase endpoint category

6008

7204

+ 1196

Unique effects: Developmental endpoint category

9640

10835

+ 1195

Unique effects: Reproductive endpoint category

5775

6416

+641

Unique effects: Systemic endpoint category

302674

324532

+21858

Unique effects: Neurological endpoint category

3194

+3194

Unique critical effects: Cholinesterase endpoint category

796

1095

+299

Unique critical effects: Developmental endpoint category

1276

1496

+220

Unique critical effects: Reproductive endpoint category

645

700

+55

Unique critical effects: Systemic endpoint category

20989

24044

+3055

Unique critical effects: Neurological endpoint category

390

+390

Treatment-related effects define the lowest effect levels (LELs) and no effect levels (NELs); these
effects are simply treatment-related and significantly different from control. In contrast, toxicological
opinion informs selection of a critical effect designation to define the lowest observable adverse effect
and no observable adverse effect levels (LOAELs, NOAELs).

-------
Accessing information in ToxRefDB

A SQL database export and summary files of ToxRefDB v3.0 are available for public download, see
https://doi.org/10.23645/epacomptox.6062545.v5. The summary spreadsheet contains study and
chemical-level information for reference. ToxRefDB information is also summarized with calculated point-
of-departure values at the chemical and study level for inclusion in the summary-level database, the
Toxicity Value Database (ToxValDB), which is accessible via the CompTox Chemicals Dashboard.
ToxValDB v.9.6.1 presently relies on ToxRefDB v2.1. ToxRefDB v3.0 values are provided with the point
of departure (pod) table, which will be incorporated in a later ToxValDB release.

Below is documentation on how to install PostgreSQL, load ToxRefDB, and access the data using both
SQL and programmatic access using either Python or R. Another useful tool to access the data is
pgadmin, which provides a user interface to interact with any PostgreSQL database. ToxRefDB v3.0 is a
generic SQL export, therefore can effectively bridge the gap between different database GUIs. Install
instructions and example queries for the PostgreSQL format are in scope of this user guide.

installing PostgreSQL and Loading ToxRefDB

Steps to install PostgreSQL and load ToxRefDB are detailed below for Windows. More comprehensive
documentation for installing PostgreSQL on other operating systems can be found online.

Installing PostgreSQL

1. Download the Installer: Download the PostgreSQL installer from the official website or a
reputable source like EnterpriseDB.

2. Run the Installer: Double-click the downloaded file to start the installation wizard.

3. Choose Installation Options: The installer will guide you through the process, allowing you to
customize the installation path, select components (like pgAdmin for a GUI), and choose the
database data location.

4. Configure PostgreSQL: You'll need to set a password for the PostgreSQL superuser and specify
the port the server will listen on. By default, this will be '5432'

5. Complete the Installation: Follow the remaining prompts to complete the installation.

Create Database and Import Schema

1. Open psql: After installation, open the psql command-line tool or launch psql from the command
from e the directory where PostgreSQL is installed e.g. in Windows this will likely be: C:\Program
Files\PostqreSQL\17\bin

2. Connect to the Database: Use the psql command to connect to the PostgreSQL server,
providing the necessary credentials.

4. Create the Database: Use the CREATE DATABASE command to create a new database

CREATE DATABASE resjoxref;

5. Create the Schema:

CREATE SCHEMA prod_toxrefdb_3_0;

6. Import the Schema: Use the \i command (or the \COPY command for larger files) to import the
data from the file into the database.

pg_restore -h -U -d res_toxref toxrefdb_3_0.dump

7. Verify the Import: Check the database to ensure the data was imported correctly.

8. Connect to the Database: From the command line, this would take the form:

psql "postgresql://username:password@hostname:5432/res_toxref?options=-
c%20search_path%3Dprod_toxrefdb_3_0"

-------
Example SQL Queries

Once a connection to the database has been established, the user should be able to query the database.
The following sample queries can be tailored for exploratory data analysis, research questions based the
specific use case, or risk assessment workflows. Review the Database Structure or Data Dictionary
section for more details on individual table structures and relevant columns.

# Return number of studies per study type where a full curation with guideline profile observations was
complete (the counts match the totals represented in the summary coverage presented in Table 1)
SELECT study_type, COUNT(study_id) FROM study where processed = TRUE

GROUP BY study_type;

# Return number of studies per study type and species
SELECT study_type,species, COUNT(study_id) FROM study
GROUP BY study_type,species;

# Return number of studies per source

SELECT study_source, COUNT(studyJd) FROM study
GROUP BY study_source;

# Return all study information for chronic studies
SELECT * FROM study WHERE study_type='CHR';

# Return all treatment group and dosing information for a single chemical
SELECT * FROM chemical

INNER JOIN study ON chemical.chemical_id=study.chemical_id
INNER JOIN tg ON tg.study_id=study.study_id
INNER JOIN dose ON dose.study_id=study.study_id
INNER JOIN dtg ON dtg.tg_id=tg.tg_id AND dose.dose_id=dtg.dose_id
WHERE dsstox_substance_id ='DTXSID7034676'

# Return number of studies per endpoint

SELECT endpoint_category, endpoint_type, endpoint_target,

COUNT(DISTINCT study.studyjd) AS "number of studies" FROM study
INNER JOIN tg ON study.study_id=tg.studyjd
INNER JOIN tg_effect ON tg.tg_id=tg_effect.tg_id
INNER JOIN effect ON effect.effect_id=tg_effect.effect_id
INNER JOIN endpoint ON endpoint.endpoint_id=effect.endpoint_id
GROUP BY endpoint_category,endpoint_type,endpoint_target;

# Return all dose-response data for a study
SELECT * FROM chemical

INNER JOIN study ON study.chemical_id=chemical.chemical_id

INNER JOIN tg ON tg.study_id=study.studyjd

INNER JOIN dose ON dose.studyjd=study.studyjd

INNER JOIN dtg ON dtg.tg_id=tg.tg_id AND dose.doseJd=dtg.doseJd

INNER JOIN tg_effect ON tg.tgjd=tg_effect.tg_id

INNER JOIN effect ON effect.effectjd=tg_effect.effectJd

INNER JOIN endpoint ON endpoint.endpointJd=effect.endpoint_id

INNER JOIN dtg_effect ON tg_effect.tg_effect_id=dtg_effect.tg_effect_id AND
dtg. dtg J d=dtg_eff ect. dtg J d

WHERE study.studyJd=687;

-------
Programmatic Access

Beyond the use of the command-line tool psql or the graphical tool like pgAdmin, ToxRefDB can be
accessed programmatically using R and python. Below are examples of how to connect to the database
in Python and R. The user will need to install language specific connectors in order to connect to the
database.

Python

In the example below, the python packages sqlalchemy, pandas, and psycopg2 are required. These can
be installed using pip instair. You can, however, use any type of connector. Any SQL query can replace
the one provided in this example.

# Load libraries
import sqlalchemy as sa
import pandas as pd

# Establish connection
username = ""
password = ""
host = ""
database = ""
schemas = ""

engine=sa.create_engine(f"postgresql+psycopg2://{username}:{password}@{host}:5432/{database}?
options=-c%20search_path%3Dprod_toxrefdb_3_0")

# Return guideline profiles

results = pd.read_sql( SELECT gd.guidelinejd,

gd.guideline_number,

gd.guideline_name,

gd.profile_name,

gd.description,

gdp.guideline_profile_id,

gdp.obs_status,

gdp.description as gdp_description,
ed.endpointjd,
ed.endpoint_category,
ed.endpoint_type,

ed.endpoint_target FROM guideline as gd

INNER JOIN guideline_profile as gdp ON gd.guideline_id=gdp.guideline_id
INNER JOIN endpoint as ed ON ed.endpoint_id=gdp.endpoint_id .engine)

# Export to excel

writer = pd.ExcelWriter("guideline_profiles.xlsx")
results.to_excel(writer,index=False,merge_cells=False)

writer.closeQ

In the example below, the R package RPostgreSQL required. Any SQL query can replace the one
provided in this example.

# Load library
library(RPostgreSQL)

# Establish connection

-------
con <- dbConnect(drv = "PostgreSQL", user="", password = "", db =
, options = ="-c search_path=")

# Return all ToxRefDB information for subchronic studies
output <- dbGetQuery(con, "SELECT chemical.casrn,
chemical. preferred_name,
study.studyjd,
study. study_type,
study. study_year,
study. study_source,
study.species,
study.strain_group,
study.admin_route,
study.admin_method,
endpoint.endpoint_category,
endpoint.endpoint_type,
endpoint.endpoint_target,
endpoint.endpointjd,
tg_effect. Iife_stage,
tg_effect.tg_effect_id,
effect. effect_id,
effect. effect_desc,
tg.sex,

tg.generation,

dose.dosejevel,

dtg.dose_adjusted,

dtg.dose_adjusted_unit,

dtg_effect.treatment_related,

dtg_effect. critical_effect,

tested_status,

reported_status FROM chemical

INNER JOIN study ON chemical.chemical_id=study.chemical_id

LEFT JOIN dose ON dose.study_id=study.study_id

LEFT JOIN tg ON tg.study_id=study.study_id

LEFT JOIN dtg ON tg.tg_id=dtg.tg_id AND dose.dose_id=dtg.dose_id

LEFT JOIN tg_effectON tg.tg_id=tg_effect.tg_id

LEFT JOIN dtg_effect ON tg_effect.tg_effect_id=dtg_effect.tg_effect_id AND
dtg. dtgj d=dtg_eff ect. dtgj d

LEFT JOIN effect ON effect.effect_id=tg_effect.effect_id
LEFT JOIN endpoint ON endpoint.endpoint_id=effect.endpoint_id

LEFT JOIN obs ON obs.study_id=study.study_id AND obs.endpoint_id=endpoint.endpoint_id
WHERE study_type='SUB'")

-------
CompTox Chemicals Dashboard

Data from ToxRefDB is summarized in the Toxicity Values Database (ToxValDB), including inferred
NEL and NOAEL effect levels based on the reported LEL and LOAEL, respectively. These dose
response summary values are accessible via the CompTox Chemicals Dashboard (CCD).ToxRefDB
v3.0 values will be incorporated in the next ToxValDB release, as far as is practicably feasible. An
image of the CCD Hazard tab for Bisphenol A is depicted below:

CompTox Chemicals Dashboard V2.5.3 Home Search ~ Lists ~ About - Tools "

Submit Comments

Search all data

Bisphenol A

80-05-7I DTXSID7020182

Searched by Synonym.

Hazard: Dose Response Summary Value ©

Data Description

Dose Response Summary Value v

Human-Relevant Dose Response Summary Value Data

— Type Subtype Risk

| ToxRefDB

i 11 1 C

~ c

& mmms*
E &

LOAEL

L£k

LOAEl

160 rngkg-daj reproduction de/£ - cral

16C mgkg-^y reproducer cteve.. cral

128et3 mgjlcg-day de,>ebe=ntai cral

1.286+3 mg,*g-day

1.28e+3 rrg-lcg-day

ToxRefDB effects data are also available via Batch Search as an Enhanced Data Sheet option, wherein
users can export all effects data for a given chemical if available:

Enhanced Data Sheets

| | Atejra^_Siftw_lnfwSJFib

it Band Identifiers

J ubst-snce reistiorisruoa

| | AC

| | Jc«V^iPB_Details
H ToxF«jp_B_Detail3
~ PhysicochefTiical Prd

~ ASPECT Gf
0 ATSDRLST Gf

~ ATSDRMRL5 Gf

Jfr

menai Col

£v5EPk,S2i
AJSDft Mir
ss NAV1GA1

Selecting this checkbox provides a separate Excel worksheet
containing all extracted dose-treatment group-effect
information from ToxRefDB for the chemicals listed in the input
Data from ToxRefDB is also summarized in ToxValDB, including
inferred NEL and NOAEL effect levels based on the reported LEL
and LOAEL, respectively.

-------
CTX Hazard API

To expand user accessibility, ToxRefDB will be available via the CTX Hazard API. The CTX API
Documentation provides more information on use and versioning. The following endpoints are available:

• Effects: Export of all extracted dose-treatment group-effect information from ToxRefDB for the
chemicals, studies, or study types listed in the input. This view is also available as an enhanced
data sheet using the batch search capability within the CompTox Chemicals Dashboard.

• Data: Export of all extracted dose-treatment group-effect information as well as doses not eliciting
effects from ToxRefDB for chemicals, studies, or study types listed in the input.

• Obs: Export of all observations and endpoint observation status according to a relevant guideline
profile for chemicals, studies, or study types listed in the input. Given ToxRefDB curations are
guideline studies with specific testing requirements, guideline profiles are used to populate a list
of observations that should be reported and tested within the study, i.e. endpoint observation
status. Endpoint Observation status enables automated distinction of true negatives (i.e. tested
with no effect observed) and better understanding of false negative (i.e. missing) effects.

• Summary: Export of summary chemical and study level metadata available within ToxRefDB.

-------
Database Structure

This entity-relationship diagram (ERD) of the ToxRefDB schema can be used to understand the
relationships between tables. The 'pod' table reflects summary-level point of departure levels provided in
a format to be consumed by the Toxicity Values Database (ToxValDB).

Figure 3: ToxRefDB v3.0 ERD

' chemical

tbefftkaljd

dsstra., sub-stance.
' casm

' preftrrtd_nam#

^ ftu^.ekswdfcf
study ckiwder id

rtu-dyd

' sourcefilename
' »urci.fl)C^pc

jource_ckjv»Tder_UKl

orceLfilervsme

e*cel_clo*idhsr_iiid

' atces*db_ci&wl€

1o«re*db_f>eid

3= ontology

MitofcHjyid |

' onEolagy_name
4 lid
L&d_type
' prtftrrtdj-ibf I
description
identifier„uri

= Etudy

- study Jd

dhernkal.id
' study.saurcejd
' studycitatmn
5tudy_y_ id

time

lr° te.c

V! iimtjtfwt

"r gtwatien

dtg_efiect_comment

d««j5
-------
Figure 4: Simplified Schema Overview

dosing, and significant treatment-related and critical effects.
Part 2:

Observation status for ToxRefDB endpoints

Reported status

Was the endpoint described in the study literature?

Tested status

is "assumed" based on the
default from guideline profile

Tested status

— Were data collected for the
endpoint?

TCo~

(not tested)

No effect data recorded for
the endpoint in database

Treatment group effect data

• Life stage

• Direction of net change
across all doses
(increase/decrease)

Qualitative

• Treatment
related?

• Critical effect?

Treatment-related endpoint effects

Was the data collected described as at least
one of the following?

1. Toxicologicallysignificant

2. Biologically significant

3. Statistically significant

4. Used to derive LOEL/NOEL

5. Treatment-related or Dose-related

6. Quantitative data suggests trend across doses

Yes

Effect data information

Method information describing the
data collected for each applicable
endpoint's effect

Part 2 provides more context about the data entry method. The portion of ToxRefDB v1.0 that was carried
over to v2.0 remains unchanged from v1.0 if indicated as processed = FALSE and yet to undergo re-
curation. The previously extracted information from ToxRefDBvl was checked for accuracy and
modified/added for QA purposes.

Per the workflow in Part 2, a curator first assigns an endpoint testing status according to a guideline
profile. A decision tree which classifies 400 standardized endpoints as described in study reports is used
for this purpose. Guideline profiles developed that matched language found in the studies are used to
infer negative endpoints/effects. Observed Endpoints classified as "tested" are evaluated for treatment-
related effects. Treatment-related effects are indexed by endpoint and method information pertaining to
the data collected.

Where available, complete dose-response effect qualitative and/or quantitative data for each dose was
extracted.

-------
Data Curation Process

The following sections detail ToxRefDB's data curation process.

Legacy Curation

Initially, ToxRefDB v1.0 provided only summary effect levels and lacked quantitative dose response
information. This task initially proceeded using an Excel file-based extraction; however, the process
required manual corrections after uploading study extractions to the ToxRefDB MySQL database,
including inconsistent comments, different number of animals for the same treatment group, and added
effects outside of the controlled terminology. The quantitative information and its application in ToxRefDB
v2.0 served as a strong impetus to re-extract the studies.

An MS Access database (MS AccessDB) file was generated from the MySQL database for each study
in v1.0, and this approach offered several improvements including standardized options for more
consistent reporting in some fields, such as the units on time and dose, dose-treatment group, and effect
information; checkbox reporting for observation status on each endpoint and effect; and a log for tracking
changes and facilitating QA. Nearly 32% of the studies were extracted using the Excel-based approach,
with the remaining studies extracted using the Access database approach. Switching to Access database
files from Excel files significantly reduced errors and increased standardization of reporting items such
as units, endpoints, and effects.

Figure 5: Data Extraction and Review Workflow

Figure 5 details the workflow of the overall data extraction process for ToxRefDB v2.G. Access databases
files were generated for each study in ToxRefDB v1.0 and bundled with the corresponding source files
for data extraction. The data in the MS AccessDB files were curated with additional data extracted from
the source files with up to three levels of review. The MS AccessDB files were returned by the reviewers
and the data is imported back into the MySQL database with the study table designation of processed=1.

ToxRefDB v2.0 curation also included the implementation of guideline profiles to guide curation.
Endpoints were annotated (e.g. "required", "not required") according to guidelines for subacute,
subchronic, chronic, developmental, and multigenerational reproductive designs, distinguishing negative

Generate Access database fries for

ToxRelDB

-------
responses from untested. Implementation of controlled vocabulary improved data quality; standardization
to guideline requirements and cross-referencing with United Medical Language System (UMLS)
connected ToxRefDB v2.0 observations to vocabularies linked to UMLS, including PubMed medical
subject headings (MeSH). The endpoint terminology and its hierarchical nature is described in later
sections.

Application-Based Curation

For ToxRefDB v3.0, an application-driven workflow was piloted with the Data Collection Tool (DCT) to
create a more sustainable process for loading curated information to a database. The DCT improved
upon the legacy ToxRefDB curation workflow to provide document allocation, curation and workflow
management among users, and management review with data conflict resolution, resulting in records
that directly link quality-controlled curations to source documents. The DCT offered flexibility via its
modular workflow for curating the heterogeneous and complex in vivo study designs. Given the DCT is
an Oracle APEX software tool, the ToxRefDB data will be migrated to the Health Assessment Workspace
Collaborative (HAWC) application in the future. HAWC is a modular, web-based content management
system designed to store, display, and synthesize multiple data sources for the purpose of supporting
the development of human health and environmental risk assessments of hazardous pollutants. Updates
to HAWC are in development to support curation as well as public accessibility and visualization of
ToxRefDB data. HAWC will provide an open-source, scalable solution for ToxRefDB curation and data
access while promoting a more integrated hazard data landscape within EPA. Within these tools, a multi-
layer review process will continue to be implemented to ensure data integrity and minimize data entry
error.

Quality Assurance in Data Extraction

Guidance for data extraction was stratified first by study type (e.g., CHR, SUB, DEV, MGR) and study
source (e.g., OPP DER and NTP) because of the differences in both study design and adverse effects
required for reporting as stated in guidelines. The process used to extract study information was also an
important aspect of QA efforts for ToxRefDB v2.0. First, a primary reviewer extracted study, dose,
treatment group, effect, and endpoint observation information. The instructions detailed how to review
the toxicological data and extract it from the original data sources consistently across reviewers using
the MS Access database files. This was reviewed by a second, senior reviewer, who was asked to review
all extracted information as though they were extracting it again and, also, to review the comment log
from the primary reviewer. Finally, if either the primary or secondary reviewer noted that it was necessary,
an additional senior toxicologist reviewed the comment logs, extracted information, and resolved any
conflicts or questions prior to finalization of the extraction. The final, tertiary review occurred for
approximately 10% of the studies. Review by a manager to resolve any differences between the primary
and secondary reviewer served to inform any training needs or gaps for the reviewers. During this
process, subject matter experts could also be consulted to resolve questions. For release of ToxRefDB
v2.0, the full quantitative data extraction for all CHR and SUB studies were completed, with quantitative
data extraction completed for many other study types and sources as well.

Efforts to Reduce Error Rate

Error rate is an inherent problem for legacy databases as much of the source information was entered
manually and human errors resulting from transcription are impossible to completely avoid. However, as
part of the ToxRefDB v2.0 curation effort, more robust QA processes were implemented to promote
greater fidelity of the information extracted and numerous quality control (QC) checks to verify data
integrity.

First, studies were extracted utilizing a defined QA process, with multiple levels of review and MS Access
form-based entry (described previously) to prevent extraction errors. Upon being uploaded into
ToxRefDB v2.0, these extractions were required to pass specific QC checks because, although the MS
Access database files enforce the MySQL database constraints as well minimize data entry error by
standardizing vocabulary used, logical errors can persist. After the extracted data was uploaded through

-------
the import script, a series of potential logical errors were identified through unit tests where their curated
value could be assumed. Flagged logical errors that have been corrected included:

• Dose level numbering did not correspond to the total number of doses;

• Duplication of concentration/dose values, including two control doses;

• No concentration and no dose adjusted value for a reported effect (possible extraction error or
possibly that the effect was qualitatively reported);

• The critical effect level is at a dose below where treatment-related effects were observed; and/or,

• The control was incorrectly identified as a critical effect level.

Any of these issues that could not be resolved systematically were flagged to undergo a second round
of extraction and review to correct. Though QC is an ongoing and evolving process, these QC checks
are serving as an improvement to the overall database and database development process.

Unit Standardization

An additional ongoing problem for reporting quantitative data from clinical or related laboratory findings
is unit standardization. No guidance is provided on how to report findings in the OCSPP guidelines nor
from any other sources, so units were extracted exactly as they were presented in the reports. The units
were standardized by eliminating duplicate entries for the same units that were originally entered
differently or with typographical errors. Units were only standardized, and no conversions of effect value
units are introduced in the current database. For non-inhalation exposures, a mg/kg/day is computed
using species-specific conversion factors. The value is stored within the dtg table's mg_kg_day_value
column. Species-specific conversion scaling factors used to convert ppm to mg/kg/day were:

• Primate: 0.05 • Guinea Pig: 0.04

• Mouse: 0.15 • Rabbit: 0.03

• Rat: 0.05 • Dog: 0.025

Ongoing efforts include further standardization of units and defining conversions that cannot be
systematically automated.

Study Reliability with ToxRTool

Most studies referenced within ToxRefDB were extracted via summaries from OPP DERs, and these
studies typically follow OCSPP 870 series Health Effects Testing Guidelines. As ToxRefDB was
expanded, additional studies needed to be assessed for reliability and guideline adherence. The scope
of ToxRefDB has shifted curation efforts to focus on guideline adherent in vivo studies submitted for
statutory requirements over studies published in open literature. Using the ToxRTool to assess study
reliability before curation was not implemented for new document extractions in v3.0.

The Toxicological Data Reliability Assessment Tool (ToxRTool) was adapted for reliability assessment.
ToxRTool is an Excel application that includes questions across 5 criteria with numerical responses that
are summed to lead to a Klimisch score: a score ranging from 1-4 that captures an overall assessment
of reliability.

A total of 522 OpenLit studies were assessed with the ToxRTool with scores ranging from 8 to 23. As
explained in the table below, most studies reviewed for ToxRefDB v2.0 corresponded to Klimisch quality
scores of 1 (ToxRTool score of > 18) or 2 (ToxRTool score of 13-18). The ToxRTool scores could be
used as a quality flag both to qualify and prioritize studies for the extraction process, or by users who are
performing reviews of information on a single chemical basis.

Table 3: ToxRTool Guideline Adherence Score

Score

Description

Adheres to modern* OECD/EPA guideline for repeat-dose toxicity studies (explicitly
stated by authors; broad endpoint coverage and ability to assess dose-response)

Adheres to an existing or previous guideline (explicitly stated by authors; previous

-------

version of OECD/EPA guidelines or FDA guidelines)

Not stated to adhere to guideline but guideline-like in terms of endpoint coverage and
ability to assess dose-response (e.g., NTP). Please see Quick Guide to EPA Guidelines
for chronic and subchronic studies. In this table, you can easily assess whether the
study was guideline-like in terms of the animals used (species, sex, age, number),
dosing requirements, and reporting recommendations.

Unacceptable adherence to guideline (intended to adhere to guideline but had major
deficiencies)

Unacceptable (no intention to be run as a guideline study, purely open literature or
specialized study)

A study is considered as adhering to "modern" OECD/EPA guidelines if it was published after
1998, which is the date that many Health Effect 870 series guidelines were re-published. Note that many
of the studies extracted, particularly from sources like the NTP and OpenLit, were never intended to
adhere to a guideline and as such "unacceptable" in this case only refers to their guideline adherence
and not the study design itself.

Guideline Profiles

Within a curation, study records are linked to a guideline profile. OPP DERs follow the Series 870 - Health
Effects Test Guidelines, described here. NTP reports follow NTP specifications. Other subsources cannot
be uniformly mapped, but some curations may be assigned a guideline profile based on how closely the
study design adheres to a guideline.

Guideline profiles for study endpoints were created from the Office of Chemical Safety and Pollution
Prevention (OCSPP) series 870 Health Effects Testing Guidelines and NTP specifications (Table 4). This
allows for analysis of guideline adherence for both guideline and non-guideline studies. Additional efforts
are underway to develop new profiles. The following table includes guideline profiles available for the
Series 870 - Health Effects Test Guidelines, but additional profiles exist within ToxRefDB.

Table 4: Series 870 - Health Effects Test Guidelines Guideline Profile Coverage

Study Type

Guideline
Profile ID

Guideline Profile Description

CHR:

Carcinogenicity

The objective of a long-term carcinogenicity study is to observe
test animals for a major portion of life span for development of
neoplastic lesions during or after exposure to test substance by an
appropriate route of administration. The dose period generally
lasts a year or longer, typically 12, 18, or 24 months, and
observations will exclude developmental and neurological effects.
See OPPTS 870.4200 Carcinogenicity.

CHR: Chronic
Toxicity

The objective of a chronic toxicity study is to determine the effects
of a substance in a mammalian species following prolonged and
repeated exposure. A chronic toxicity study should generate data
to identify chronic effects and define long-term dose-response
relationships. The dose period generally lasts a year or longer,
typically 12, 18, or 24 months, and observations will exclude
developmental and neurological effects. See OPPTS 870.4100
Chronic Toxicity.

CHR:

Combined
Chronic

Toxicity /
Carcinogenicity

The objective of a combined chronic toxicity/carcinogenicity study
is to determine the effects of a substance in a mammalian species
following prolonged and repeated exposure. Following updates to
the 870 Series Health Effects Guidelines in 1998, this combined
study was preferred to separate submissions of 870.4100 and
870.4200. The design and conduct should allow for the detection

-------

of neoplastic effects and a determination of the carcinogenic
potential as well as general toxicity. The dose period generally
lasts a year or longer, typically 12, 18, or 24 months, and
observations will exclude developmental and neurological effects.
See OPPTS 870.4300 Combined Chronic

T oxicitv/Carcinoqenicitv.

DEV: Prenatal
Developmental
Toxicity Study

This guideline for developmental toxicity testing is designed to
provide general information concerning the effects of exposure of
the pregnant test animal on the developing organism; this may
include death, structural abnormalities, or altered growth and an
assessment of maternal effects. The dose period is usually
gestational (in utero) and the animal is sacrificed prior to delivery.
See OPPTS 870.3700 Prenatal Developmental Toxicity Studv

DNT:

Developmental
Neurotoxicity

This study is designed to develop data on the potential functional
and morphological hazards to the nervous system which may
arise in the offspring from exposure of the mother during
pregnancy and lactation. The test substance is administered to
several groups of pregnant animals during gestation and early
lactation, one dose level being used per group. Offspring are
randomly selected from within litters for neurotoxicity evaluation.
The evaluation includes observations to detect gross neurologic
and behavioral abnormalities, determination of motor activity,
response to auditory startle, assessment of learning,
neuropathological evaluation, and brain weights. This protocol
may be used as a separate study, as a followup to a standard
developmental toxicity and/or adult neurotoxicity study, or as part
of a two-generation reproduction study, with assessment of the
offspring conducted on the second (F2) Generation. See OPPTS
870.6300 Developmental Neurotoxicitv Studv

MGR: Multi-
generational
reproductive
toxicity study

7 or 13 (pre-
1998)

This guideline for two-generation reproduction testing is designed
to provide general information concerning the effects of a test
substance on the integrity and performance of the male and
female reproductive systems, including gonadal function, the
estrous cycle, mating behavior, conception, gestation, parturition,
lactation, and weaning, and on the growth and development of the
offspring. The study may also provide information about the
effects of the test substance on neonatal morbidity, mortality,
target organs in the offspring, and preliminary data on prenatal
and postnatal developmental toxicity and serve as a guide for
subsequent tests. Additionally, since the study design includes in
utero as well as postnatal exposure, this study provides the
opportunity to examine the susceptibility of the immature/neonatal
animal. The dose period begins in adolescent F0 males and
females and continues until the terminal generation. Some of the
litters deliver their pups, while others may be sacrificed prior to
deliverv. See OPPTS 870.3800 Reproduction and Fertilitv Effects.

Note: There are two guideline profiles due to a 1998 guideline
change. The post-1998 guideline was likely used for MGR studies
that started in 1996.

-------
REP: Fertility
(Segment 1)

This guideline is designed to generate limited information
concerning the effects of a test substance on male and female
reproductive performance such as gonadal function, mating
behavior, conception, development of the conceptus, and
parturition. This screening test guideline can be used to provide
initial information on possible effects on reproduction and/or
development, either at an early stage of assessing the
toxicological properties of chemicals, or on chemicals of high
concerns focused on early postnatal evaluation, with sacrifice of
dams and offspring at postnatal dav 4. See OPPTS 870.3550
Reproduction and Fertility Effects.

REP: Peri- and
post-natal
toxicity study
(Segment III)

The study may provide information about the effects of the test
substance on neonatal morbidity, mortality, target organs in the
offspring, and preliminary data on prenatal and postnatal
developmental toxicity and serve as a guide for subsequent tests.
Additionally, since the study design includes in utero as well as
postnatal exposure, this study provides the opportunity to examine
the susceptibility of the immature/neonatal animal (F1 generation).
See OPPTS 870.3550 Reproduction and Fertility Effects.

REP:

Reproductive /
developmental
toxicity
screening test

This guideline is designed to generate limited information
concerning the effects of a test substance on male and female
reproductive performance such as gonadal function, mating
behavior, conception, development of the conceptus, and
parturition. This screening test guideline can be used to provide
initial information on possible effects on reproduction and/or
development, either at an early stage of assessing the
toxicological properties of chemicals, or on chemicals of high
concern. See OPPTS 870.3550 Reproduction and Fertility Effects.

SAC: Sub-acute
dermal toxicity

A 21/28 day repeated dose dermal study will provide information
on possible health hazards likely to arise from repeated dermal
exposure to a test substance for a period of 21/28 days. Dose
period is typically 21-28 days with dermal exposure route, and
observations will exclude developmental and neurological effects.
See OPPTS 870.3200 21/28-Dav Dermal Toxicity.

SAC: Sub-acute
repeat dose
toxicity

The objective of a sub-acute repeat dose toxicity study is to
determine the adverse effects of a substance in a mammalian
species occurring after short-term dosing duration. Determination
of acute toxicity is usually an initial step in the assessment and
evaluation of the toxic characteristics of a substance. Dose period
is typically 21-28 days with varied exposure routes, and
observations will exclude developmental and neurological effects.
See 870.3050 - Repeated Dose 28-Day Oral Toxicity Study in
Rodents

SUB:

Subchronic
dermal toxicity

The subchronic dermal study has been designed to permit the
determination of the no-observed-effect level (NOEL) and toxic
effects associated with continuous or repeated exposure to a test
substance for a period of 90 days. It can provide useful information
on the degree of percutaneous absorption, target organs, the
possibilities of accumulation, and can be of use in selecting dose
levels for chronic studies and for establishing safety criteria for
human exposure. The dose period is typically 90 days or 13
weeks, but may be as long as 6 months, via dermal routes of

-------

exposure. Observations will exclude developmental and
neuroloaical effects. See OPPTS 870.3250 90-Dav Dermal

Toxicity.

SUB:

Subchronic

inhalation

toxicity

The subchronic inhalation study has been designed to permit the
determination of the no-observed effect-level (NOEL) and toxic
effects associated with continuous or repeated exposure to a test
substance for a period of 90 days. It will provide information on
target organs and the possibilities of accumulation, and can be
used to select concentration levels for chronic studies and
establishing safety criteria for human exposure. The dose period
is typically 90 days or13 weeks, but it may be as long as 6 months,
via inhalation routes of exposure. Observations will exclude
developmental and neuroloaical effects. See OPPTS 870.3465
90-Dav Inhalation Toxicitv.

SUB:

Subchronic oral
toxicity in
nonrodent

The subchronic oral study has been designed to permit the
determination of the no-observed-effect level (NOEL) and toxic
effects associated with continuous or repeated exposure to a test
substance for a period of 90 days. It provides information on target
organs, the possibilities of accumulation, and can be of use in
selecting dose levels for chronic studies and for establishing
safety criteria for human exposure. The dose period is typically 90
days or 13 weeks, but it may be as long as 6 months, via oral
routes of exposure in any nonrodent species. Observations will
exclude developmental and neuroloaical effects. See OPPTS
870.3150 90-Dav Oral Toxicitv in Nonrodents.

SUB:

Subchronic oral
toxicity in
rodents

The subchronic oral study has been designed to permit the
determination of the no-observed-effect level (NOEL) and toxic
effects associated with continuous or repeated exposure to a test
substance for a period of 90 days. It provides information on target
organs, the possibilities of accumulation, and can be of use in
selecting dose levels for chronic studies and for establishing
safety criteria for human exposure. The dose period is typically 90
days or 13 weeks, but may be as long as 6 months, via oral routes
of exposure in rodent species, typically rats and mice.
Observations will exclude developmental and neurological effects.
See OPPTS 870.3100 90-Dav Oral Toxicitv in Rodents.

DNT Vocabulary and Guideline Profile Curation

A Developmental Neurotoxicity study (DNT) assesses behavioral and neurobiological parameters in
order to ascertain the effects of chemicals on the developing animal. The rodent DNT study paradigm
has evolved over time with the most recent test guidelines updated in 2007 with the introduction of OECD
guideline 426. In vivo DNT study data could inform benchmarking for new approach methods for DNT
and also help characterize what has been learned from available DNT studies. To establish a resource
of in vivo DNT data, a new vocabulary and list of required observations (i.e. guideline profile) was first
developed according to the 870.6300 DNT guideline. A standardized term list ensures data can be
consistently extracted across studies.

The DNT studies are often complex with the guideline requirement for observational and functional
assessments be collected at specified time points. Given the possible variations in assessment times
and flexibility in types of tests considered, observations often include required time periods within the

-------
neurological endpoint category. A review of endpoint types within the neurological endpoint category is
provided below:

• Detailed clinical observations ("detailed clin obs") are observational and functional tests to evaluate
overall neurological function in both the dam and offspring. Functional observational batteries (FOB)
are not required by the DNT guideline perse, but evaluations will encompass the following specific
observations: Autonomic function: lacrimation, salivation, piloerection, exophthalmos, urination,
defecation, pupillary function; Convulsions, tremors, abnormal movements; Posture, gait
abnormalities; Unusual or abnormal behaviors.

o Dams are evaluated at least twice during the gestational period and at least twice during
lactation, as reflected by dam gestation #1, dam gestation #2, dam lactation #1 or dam
lactation #2.

o Offspring are evaluated on PNDs 4,11,21, 35, 45, 60, where PND groupings define lifestages
accordingly as:

¦ early neonate: PNDO-7

¦ mid-lactation neonate: PND8-17

¦ weaning neonate: PND18-22

¦ early juvenile: PND23-35

¦ late juvenile: PND36-45

¦ young adult: PND46-90

• Motor activity tests assess locomotor activity although modern automated test equipment provide a
wide array of activity measures including ambulatory activity (movement between locations),
exploratory movements (rearing or vertical movements) as well as a variety of fine motor movements
(e.g., sniffing, scratching, grooming).

o Offspring are evaluated for motor activity on PND13, 17, 21, and 60 (±2 days), where PND
groupings define lifestages accordingly as mid-lactation neonate #1 or mid-lactation neonate
#2 (PND8-17), weaning neonate (PND18-22), or young adult: (PND46-90)

• Auditory startle response testing assesses reflexive muscle movements elicited by a sudden intense
sensory stimulus.

o Offspring are evaluated for startle response around the time of weaning and around day 60,
where PND groupings define lifestages accordingly as weaning neonate: (PND18-22) or
young adult (PND46-90)
o The most common startle response metrics are:

¦ Amplitude: Maximum startle response recorded in grams for load cells and as
equipment-specific units for accelerometers;

¦ Habituation: Calculated metric for characterizing the change in startle response
magnitude over repeated exposure to the startle stimulus;

¦ Latency: Time in msec from the startle stimulus onset to the beginning or to the peak
of the startle response.

• Associative learning is assessed either a change across several repeated learning trials or sessions,
or in tests involving a single trial. This test includes controls for non-associative effects of the training
experience, as well as some measure of memory (retention) in addition to the original learning
(acquisition) task, where appropriate.

o Offspring are evaluated for learning and memory with at least one type of test completed at

weaning and at PND 60.
o For the respective test types, acquisition and retention results are reported separately and by
sex:

¦ Active avoidance: Animal must move to avoid stationary footshock region while on a
rotating platform

• Common Effect Descriptions: Errors, Latency (time to enter shock region,
mean ± sd), Number failed, Trials - total number of trials

¦ Passive avoidance: Animal avoids footshock by choosing not to move from stationary
bright region into dark stationary footshock region

• Common Effect Descriptions: Errors, Latency (time to enter shock region,

-------
mean ± sd), Number crossings (number of trials in which the animal crossed
into shock region), Number failed, Trials (total number of trials)

¦ Dry maze: Animal makes a binary choice to turn left or right to escape the water, with
correct turns reinforced by food reward. These land-based mazes are usually in the
shape of letters (Y, M, E, T) and test position discrimination learning.

¦ Water maze: Animal makes a binary choice to turn left or right to escape the water,
with correct turns reinforced by reaching platform. These land-based mazes are
usually in the shape of letters (Y, M, E, T) and test position discrimination learning.

• Common Effect Descriptions: Criterion (including Number of trials to criterion
achievement (median or mode), Number of errors to criterion achievement
(mean ± sd), Percentage or number of animals per group to achieve criterion
on each trial, Number of animals to achieve criterion); Errors (number of errors
by trial, mean ± sd); Errors by day, mean ±sd; Number of errors per group on
each trial, median or mode); Failed; Latency by trial (time to escape from water,
mean ± sd); Latency by day, mean ± sd; Path (direction taken or length, mean
± sd); Time in each quadrant, mean ± sd; Number platform crossings, mean ±
sd; Swim speed, mean ± sd; Successful attempts; Trials

The reader is directed to this Developmental Neurotoxicity (DNT) Guidance Document for a
comprehensive review of guideline requirements for DNT studies and extended one-generation
reproduction studies that evaluate DNT.

-------
Endpoint Terminology

ToxRefDB employs controlled terminology standardized to better reflect both the OCSPP Health Effects
870 series guidelines and DER summary reporting. This hierarchical relationship of effects and endpoints
was adapted from the vocabulary developed for earlier versions of ToxRefDB based on the data types
curated. Novel values can be added when found during a curation.

Figure 6: Hierarchical endpoint terminology example

Observation

endpoint category, endpoint type, and
endpoint target.

reproductive | reproductive performance |
postimplantation loss

^ Effect

specific condition associated endpoint
target

Effect Description

postimplantation loss

TreatmentGroup

Life Stage

Target

Effect Description Free

Effect

Site

Lifestage, location,

adult

uterus

postimplantation site

verbatim text

pregnancy

loss: mean

An example of the terminology hierarchy is demonstrated for an effect described as "postimplantation
loss". The finding is recorded as is in the "effect description free" field, which is the verbatim wording used
in the study report. The remaining fields are part of the ToxRefDB controlled terminology. The endpoint
category is reproductive, the endpoint type is reproductive performance, the endpoint target is
postimplantation loss, the effect description is postimplantation loss, and the specific observation of
"postimplantation loss" was made in the adult pregnancy life-stage at the specific target site, the uterus.

Ontology mappings

It is increasingly apparent that many toxicology research questions will require the integration of public
data resources, both with those containing the same types of information, as well as with other databases
to connect different kinds of information. ToxRefDB vocabulary, therefore, promotes increased
connections to other resources, which has greatly enhanced its quantitative and qualitative utility for
predictive toxicology.

For example, efforts linking in vitro effects in ToxCast to in vivo outcomes using predictive models may
help to identify rapid, more efficient chemical screening alternatives. To connect the ToxRefDB endpoint
and effect terminology with other resources, the ToxRefDB terminology was standardized and cross-
referenced to the United Medical Language System (UMLS). UMLS cross-references enable mapping of
in vivo pathological effects from ToxRefDB to PubMed (via Medical Subject Headings or MeSH terms),
which may be relevant for toxicological research and systematic review. This enables linkage to any
resource that is also connected to PubMed or indexed with MeSH.

-------
Figure 7: Cross-referenced Terminology Sources

Over 1,800 UMLS concept codes were mapped to endpoints and effects in ToxRefDB via a manual
process. Only 500 of those concept codes are a part of the CDISC-SEND terminology. All of the concept
codes are a part of vocabularies within both National Cancer Institute Thesaurus (NCIt) as well as UMLS.

The result of updating the ToxRefDB
terminology and linking to the UMLS
concepts is that ToxRefDB may be used
to better anchor or compare to new
approach method (NAM) information,
including data from ToxCast or structure-
activity relationship models, as well as
other in vivo databases of toxicological
information, such as eChemPortal.
Integration of these data resources is a
major hurdle toward to evaluating the
reproducibility and biological meaning of
both traditional, legacy toxicity
information and the data from NAMs.
Additional work may be performed to link
to other ontologies and to assist
stakeholders in mapping their ontologies
to the ToxRefDB and UMLS ontologies.

-------
Negative Endpoints and Effects

As part of the v2.0 update to ToxRefDB, negative endpoints and effects are inferred from guideline
profiles and the testing and reporting statuses of endpoints. Given the list of all observations required for
the relevant guideline profile, the curator indicates which endpoints were missing (meaning not tested)
or negative (meaning tested with no effect observed) by setting tested and reported status accordingly.
Endpoint observation status enables automated distinction of true negatives and a better understanding
of false negative effects. Users can access the current inferred negatives and calculate inferences for a
specific subset. Curation for ToxRefDB v3.0 expanded on this process with the incorporation of the new
'DNT' guideline profile.

The database has inferred study-level negative effects and negative endpoints available in two tables:
"negative_effect" and "negative_endpoint". These tables were created from stored procedures
(repopulate_negative_effect and repopulate_negative_endpoint) that are also available with the full
database. The logic for the stored procedures follows the inference workflow seen in Figure 6. Endpoint
Observation Status distinguishes negative and missing (not tested) effects based on the study's specific
guideline requirements. An effect is negative if the study has gone through the data extraction process,
the effect was tested (regardless of being reported), and no effect was seen in the study. An endpoint is
negative for a study if all effects for that endpoint are also negative in the study.

Table 5: Endpoint Observation Status

Tested Reported Assumption
Status Status

Yes

The text of the study document explicitly stated the endpoint was measured,
or data was presented in tables for the endpoint. This is the combination if
required by the guideline for that study type and data is provided within the
document, even the effects measured were not significant.

Yes

This is the combination if the study document explicitly states the endpoint
was not measured or data was not collected, even though the endpoint was
required by the study guidelines.

Yes

The text of the study document does not state the endpoint was measured
and data for the endpoint is not present. However, other evidence suggests
that the endpoint was measured. This is the default for endpoints required
by the study guideline and should only be changed in the face of direct
evidence from the document.

Within the long table of observations from all study guidelines, this is the
default setting for the endpoints not required by the alternative study
guidelines and they should not be changed. Interpret these observations as
irrelevant since they are not serving the selected guideline, therefore not
required to be tested nor reported.

-------
Figure 8: Decision tree for identification of negative endpoints and effects

Negative Effects can be
inferred from studies that have
gone through the data
extraction and additional OA
processes

Study: Data
Extraction and QA
Complete

Reported and Tested Status for each
end point

R1T1

R1T0

R0T1

ROTO

Was a treatment group-related effect
seen for the given endpoint?

YES

Negative effect conclusion

Endpoint was tested
and effects were seen

Endpoint was tested
and it was negative

Endpoint was not
tested

Endpoint was tested
and it was negative

Endpoint was not
tested

Negative endpoints and effects can only be identified in studies that have gone through data extraction
and any subsequent QA processes because this ensures confidence in decisions made about the
adherence and/or deviations from the corresponding guideline profiles. We can infer negatives based on
whether or not an endpoint was tested and no treatment group-related effects were seen. The example
below shows how reported results are interpreted given the study's guideline profile.

Figure 9: Example Observation Status Interpretation

X VolUM I glucoee I

X pH

X protein

X bet one*
X bilirubin
X specific
~~ gravity

X occult blood
X urobilinogen
X appearance

osmolality R1.T 0

X alcroscoplc *x*al

nation
nents

of sedl- R1.T 1

3. Secrogsy Crocs lesions vere noted. |Fof organs with histopath, R=i, T=1
e. '«>l
-------
Data Dictionary

A data dictionary is found in the database in the toxrefdb_dd table.

ToxRefDB Table

ToxRefDB Field

Description

chemical

chemicaljd

PK: Autoincremented unique identifier for the
chemical tested.

dsstox_substance_id

Identifier assigned through DSSTox chemical
curation.

preferred_name

Curated chemical name from DSSTOX chemical
curation.

casrn

Curated casrn from DSSTOX chemical curation.

dose

studyjd

FK: A unique numeric identifier for each study in the
database.

cone

Concentration of a test chemical, typically reported in
ppm within the exposure matrix (e.g., feed or water).

dose level

Numeric rank indicating the level of dose
administered to test animals, with lower dose levels
indicating lower concentrations of a chemical (e.g., 0
= vehicle, 1 = lowest dose, etc.). The dose level for
some studies may be staggered since concentrations
may vary by sex (e.g, male treatment group: 0 =
vehicle, 1 = lowest dose, 3 = second lowest dose,
etc.).

dosejd

PK: Autoincremented unique identifier for a dose

dose_comment

NULL if no additional comment needed. Field can be
used to explain any differences in dosing over the
dosing interval and/or clarifying comments on how the
dose was administered.

vehicle

The media used in administration of the chemical.

conc_unit

Unit associated with a concentration of a test
chemical, typically reported as ppm.

dtg

dose_id

FK: A unique numeric identifier for each dose in the
database.

tg_id

FK: A unique numeric identifier for each treatment
group in the database.

dose_adjusted

The amount of the chemical administered in mg/kg of
body weight/day (mg/kg/day). This value is typically
different between male and female groups receiving
the same dose concentration (cone) due to
differences in bodyweight. . If dose_adjusted values
were not provided in a study, then they were
calculated using species scaling factors (FAO/WHO,
2000).

dose_adjusted_unit

Unit associated with the adjusted dose of a chemical,
typically reported in mg/kg/day.

dtg_comment

NULL if no additional comment needed. Field
provides additional explanation of the dose-treatment
group-effect row in the table, including statistical
significance.

dtg_id

PK: Autoincremented unique identifier for a dosed-
treatment group.

mg_kg_day_value

The mg/kg/day species-specific converted value,

-------

usually converted from ppm concentration.

dtg_effect

dtg_effect_id

PK: Autoincremented unique identifier for a dosed-
treatment group effect

dtg_id

FK: A unique numeric identifier for each dosed
treatment group in the database.

tg_effect_id

FK: A unique numeric identifier for each treatment
group effect in the database.

dtg_effect_comment

NULL if no additional comment needed; provides
additional explanation of the dose-treatment-group-
effect row in the table, including statistical
significance.

effect_val

Numeric value of a measured effect, can be
continuous or dichotomous (incidence) data.

effect_val_unit

Unit associated with the effect value.

effect_var

Measurement of the variance for a set of data
associated with a measured effect, generally reported
as the standard deviation (SD) or standard error (SE).

effect_var_type

Name of the variance metric used to determine the
effect variance, typically the standard deviation (SD)
or standard error (SE). Other effect_var types include:
interquartile range, 95% confidence limit, and none.

sample_size

Number of animals used for an examination for a
particular effect.

time

Numeric value associated with the duration of the
exposure at which a particular effect was measured
or observed, typically reported in hours, days, weeks,
or months.

critical_effect

Boolean description for an effect by dose treatment
group. TRUE corresponds to a toxic or adverse effect
denoted in the study summary or via expert
judgement using a weight-of-evidence approach.
FALSE indicates that although an effect is produced
at this level, it is not considered adverse, nor
immediate precursors to specific adverse effects. If
there are several critical effects, the no observed
adverse effect level (NOAEL) is determined from the
highest dose level without critical effects. The lowest
dose level at which the critical effect was observed in
a study is the lowest observed adverse effect level
(LOAEL.)

treatment_related

Boolean description for an effect by dose treatment
group. TRUE indicates there was a statistically
significant difference from the control group for the
effect; FALSE indicates there was no difference from
control group. The highest dose level at which no
significant observable adverse effects were observed
corresponds to the no effect level (NEL). The lowest
effect level (LEL) can be inferred by treatment related
effects.

time_unit

Unit associated with the duration of the exposure at
which a particular effect was measured or observed,
typically reported in hours, days, weeks, or months.

-------
effect

endpointjd

FK: A unique numeric identifier for each endpoint in
the database.

effect_desc

More specific description for an effect than
endpoint_category, usually detailing a specific
condition associated with an endpointjarget (e.g.
dysplasia, atrophy, necrosis, etc.).

effect_id

PK: Autoincremented unique identifier for an effect

cancer_related

Indicates if effect is considered cancer related (1) or
not.

endpoint

endpoint_category

The broadest descriptive term for an endpoint.
Possible endpoint categories include: systemic,
developmental, reproductive, and cholinesterase.

endpoint_type

The subcategory for endpoint_category, which is
more descriptive for a particular endpoint (e.g.
pathology gross, clinical chemistry, reproductive
performance, etc.)

endpointjd

PK: Autoincremented unique identifier for an endpoint

endpoint_target

More specific description than endpointjype, often
indicating where or how the sample was collected to
supply data for a particular endpoint. Target may
describe an organ, tissue, metabolite or protein
measured.

guideline

description

Information pertinent to a studys guideline. For
example, MGR studies conducted post-1998 required
the testing of developmental landmarks, which is
notable for observation status.

profile_name

Abbreviated name of the particular Office of Chemical
Safety and Pollution Prevention (OCSPP) guideline
that a study adheres to or most closely adheres to.
See abbreviations section for profile name list.

guidelinejd

PK: Autoincremented unique identifier for a guideline

guideline_number

Number associated with the guideline. OPP DERs
follow the Series 870 - Health Effects Test Guidelines.

guideline_name

Name of guideline, such as the particular Office of
Chemical Safety and Pollution Prevention (OCSPP)
guideline or NTP specifications.

guideline_profile

endpointjd

FK: A unique numeric identifier for each endpoint in
the database.

guidelinejd

FK: A unique numeric identifier for each guideline in
the database.

description

Provides a description of the rationale for an endpoint
observation status.

guideline_profile_id

PK: Autoincremented unique identifier for a guideline
profile.

obs_status

Indicator of whether or not an endpoint is required to
be tested according to the particular guideline a study
adheres to. The observation status for an endpoint
can be required, not required, or triggered.

obs

status

The status regarding whether or not an endpoint was
required, recommended, triggered, not required, or
mentioned by the studys most closely related
guideline. Assumes that an endpoint was tested if the

-------

guideline the study adheres to requires that endpoint
to be tested.

default

The status regarding whether or not an endpoint
was tested and reported in a study. Assumes that
an endpoint was tested if the guideline the study
adheres to requires that endpoint to be tested.

tested_status

Indicates if an endpoint was tested (1) or not tested
(0). If an endpoint was tested, it was examined or
measured.

reported_status

Indicates if an endpoint was reported (1) or not
reported (0). If an endpoint was reported, it appears
somewhere in the text of the report.

ontology

description

The associated description for the identifier.

label

The associated label for the idenfier.

uid

Unique identifier from respective terminology
resource

uid_type

Type of identifier

uri

Uniform resource identifier

ontologyjd

PK: Autoincremented unique identifier for an ontology
class

ontology_toxrefdb

ontology_id

FK: A unique numeric identifier for each ontology
class in the database.

toxrefdb_table

The associated table in ToxRefDB.

ontology_toxrefdb_id

PK: Autoincremented unique identifier for an ontology
class associated with a concept in ToxRefDB.

toxrefdbjd

Primary key from associated toxrefdb_table

toxrefdb_field

The associated field from toxrefdb_table linked to a
term

pod

study_id

FK: A unique numeric identifier for each study in the
database.

study_type

Classification to describe animal toxicity testing that
was conducted.

dsstox_substance_id

Identifier assigned through DSSTox chemical
curation

preferred_name

Curated name from DSSTOX chemical curation

toxval study source i
d

Unique string for each POD grouping. Formatted as
study_id_study_type if no effects are extracted, else
study_id_life_stage_generation_sex_endpoint_categ
ory. Some study types will have required treatment
groupings by sex, lifestage, and generations that will
be filled by the highest concentration tested if no
effects were observed.

toxval_effect_list

Concatenated listing of all effects in the endpoint
category used to establish the POD.

dose_level

Dose level at which the POD was seen

max_dose_level

Maximum dose level tested with relation to where the
POD was captured

calc_pod_type

LEL, NEL, LOAEL, or NOAEL

qualifier

A
A

ii
ii

mg_kg_day_value

Converted mg/kg/day value

-------

admin_method

Specific path by which the test substance was
administered via the administration route. Examples
include capsule, diet, gavage, or topical.

admin_route

Path by which test substance was administered to
animal. Options include oral, dermal, inhalation,
injection, or other.

dose_end

Time during an animals life that the administration of
a test substance stopped.

dose_end_unit

Unit of time associated with the end of the dose
(dose_end).

dose_start

Time during an animals life that the administration of
a test substance began.

dose_start_unit

Unit of time associated with the start of the dose
(dose_start).

study_year

Year in which the study was reported as finished.

species

Species of the animal test subject used in a study.

strain_group

Descriptive category for a group of test animals that is
more general than the strain.

strain

Intraspecific description of group of animals used in a
study; generally, a stock of animals that share a
uniform morphological or physiological character, or
group that is genetically uniform.

study

dose_end

Time during an animals life that the administration of
a test substance stopped.

dose_end_unit

Unit of time associated with the end of the dose
(dose end).

dose_start

Time during an animals life that the administration of
a test substance began.

dose_start_unit

Unit of time associated with the start of the dose
(dose start).

species

Species of the animal test subject used in a study.

strain_group

Descriptive category for a group of test animals that is
more general than the strain.

study_type_guideline

Description that combines the study_type and
guideline name for a study.

strain

Intraspecific description of group of animals used in a
study; generally, a stock of animals that share a
uniform morphological or physiological character, or
group that is genetically uniform.

study_id

PK: Autoincremented unique numeric identifier for
each study in the database.

guidelinejd

FK: Unique numeric identifier for each guideline in the
database.

chemicaljd

FK: Unique numeric identifier for each chemical in the
database.

study_source_id

Unique document identifier provided by the study
source.

study_type

Classification to describe animal toxicity testing that
was conducted. Based on either dose period, subjects
involved, or measured effects of interest, study types
include: ACU (acute): Dose period typically a day or
less. Excludes developmental and neurological

-------

studies.; SAC (subacute): Dose period is typically 21-
28 days. Excludes developmental and neurological
studies.; SUB (subchronic): Dose period is typically
13 weeks, but may be as long as 6 months. Excludes
developmental and neurological studies.; CHR
(chronic): Dose period is typically 12, 18, or 24
months (generally any dosing lasting a year or
longer). Excludes developmental and neurological
studies.; DEV (developmental): Gestational (in utero)
dose period. Sacrificed prior to delivery.; MGR
(multigenerational reproductive): Dose period begins
in adolescent FO males and females and continues
until terminal generation. At least some of the litters
deliver their pups, some may be sacrificed prior to
delivery.; NEU (neurological): Study contains
functional observation battery or other battery of
behavioral testing that occurs during or after dosing.
Pathology has specific interest in the brain (i.e.
regions, morphology, biochemistry, et cetera),
excludes developmental studies; DNT
(developmental neurotoxicity): dose period occurs
anytime during development (i.e. in utero, lactational,
adolescent [after weaning, before adulthood]). Study
contains functional observation battery or other
battery of behavioral testing that occurs during or after
dosing, typically during adulthood. Pathology has
specific interest in the brain (i.e. regions, morphology,
biochemistry, etc.)

study_source

Organization that provided or authored the document
for curation.

study_year

Year in which the study was reported as finished.

source.

_chemical

JSON describing tested substance as reported within
the source document. Fields include: name, type,
casrn, pc_code, purity_min, purity_max, lot_batch,
substance_comment, composition, and image of
structure.

processed

Boolean description for curation status. TRUE
corresponds to complete, whereas FALSE indicates
only summary-level information is available.

admin_

method

Specific path by which the test substance was
administered via the administration route. Examples
include capsule, diet, gavage, or topical.

admin_

route

Path by which test substance was administered to
animal. Options include oral, dermal, inhalation,
injection, or other.

study_citation

Citation of source document.

source.

_chemical

JSON of chemical substance information as reported
within the source document.

study_comment

Field for the executive summary or any pertinent
information about the study in general, such as poor
document quality (e.g., poor scan), missing pages,
etc.

study_toxrtool

toxrtoo

_id

FK: A unique numeric identifier for each ToxRTool

-------

question in the database.

studyjd

FK: A unique numeric identifier for each study in the
database.

score

The associated score for the ToxRTool question

toxrtool_comment

The corresponding comment further describing the
score

study_toxrtool_id

PK: Autoincremented unique identifier for a ToxRTool
question associated with a study

study_id

FK: A unique numeric identifier for each study in the
database.

dose_duration

Amount of time a group is dosed. This varies within
studies depending on the dose period of a particular
treatment group.

dose_duration_unit

Unit of time associated with the dose duration.
Typically in days or months.

dose_period

Period within a groups lifetime that the animals were
dosed and the sample for an endpoints data was
taken (when the animals were sacrificed).

generation

Generation a test animal belongs to. The FO
generation is the first generation mating group for
MGR studies and is the default for non-reproductive
studies (CHR, SUB, SAC). F1 is the second
generation mating group, selected from either F1a or
F1b litters. F2 is the third generation mating group,
selected from either F2a or F2b litters. F1a and F1b
are the first and second litter groups produced by FO
matings, F2a and F2b are the first and second litter
groups produced by F1 matings, and F3a and F3b are
the first and second litter groups produced by F2
matings. The fetal generation is the group produced
by FO matings in DEV studies, and are typically
removed from a female via cesarean section in DEV
studies.

sex

The biological sex of a test animal. The sex of fetal
groups is denoted as MF for both males and females.

tg_comment

NULL if no additional comment needed; contains
information that the extractor/curator found helpful in
describing issues related to a treatment-group (e.g.
animals dosed via capsule so concentration not
reported, added recovery groups, etc.).

tg_id

PK: Autoincremented unique identifier for a treatment
group

Number of animals per treatment group

tg_effect

no_quant_data_report
ed

Indicates if qualitative (1) or quantitative (0) effect
data was reported.

effect_id

FK: A unique numeric identifier for each effect in the
database.

tg_id

FK: A unique numeric identifier for each treatment
group in the database.

direction

Description of the net change across all doses that
indicates whether the numerical data increased,
decreased, or stayed the same. Also can be used to

-------

describe effects that did not have numerical data, but
were still described in the study source.

effect_comment

NULL if no additional comment needed; contains
information that the extractor/curator found helpful in
describing issues related to a treatment-group-effect
(e.g. units not reported, effect only reported for certain
treatment groups, etc.).

tg_effect_id

PK: Autoincremented unique identifier for a treatment
group effect

effect_desc_free

Brief verbatim text from study file that was entered if
the effect description differed from predetermined
endpoint terminology.

life_stage

Stage of life that a measurement was taken. CHR,
SUB, and SAC studies typically only have adult for
life_stage, whereas DEV and MGR studies will always
be characterized by multiple life stages. The different
life stages in the database include: fetal, juvenile,
adult, adult-pregnancy and pregnancy. DNT studies
include distinct lifestages by post-natal day (PND) to
capture the required observations and functional tests
conducted at different timepoints.

tg_effect

target_site

A more specific description than effect_target. Can
describe a specific tissue within an organ, type of cell,
etc.

toxrtool

criteria

The ToxRTool comprises a list of evaluation criteria to
assess study reliability that are subdivided into five
groups: test substance identification, test system
characterization, study design description, study
results documentation, and plausibility of study design
and data.

question

Question used as part of the ToxRTool evaluation
criteria to assess study reliability.

question_number

Number indicating the question as part of the
ToxRTool evaluation criteria to assess study
reliability.

toxrtooljd

PK: Autoincremented unique identifier for a ToxRTool
question

-------
Acknowledgements

ToxRefDB is a longstanding project with many collaborators over the years. Authors would like to
acknowledge the following individuals for their contributions:

• ToxRefDB Team: Madison Feshuk, Sean Watford, Kelly Carstens, Elizabeth Gilson (ORAU-
NSSC), Lori Kolaczkowski (ORAU-NSSC), Katie Paul Friedman

• DCT Curation and Reviewers (ICF): Scott Studenberg, Lisa Prince*, Ellen Lee, Maricruz Zarco,
Lauren Browning, Denyse Marquez Sanchez, Meredith Clemons, Hannah Eglinton, Monique
Slowly, Emily Pak, Grace Cooney, Jennifer Seed, Anna Kolanowski, Andrea Santa Rios, Sarah
Saucier, Sheerin Shirajan, Whitney Fies*

• DCT Project Management (ICF): Whitney Fies, Grace Chappell, and Lisa Prince

• DCT Software Development (GDIT): Michael Cordts, Paula Scott, Paul Dutrow, and Kristine
Sissons

• Computational Toxicology and Exposure (CTX) API and CompTox Chemicals Dashboard
Support: Asif Rashid, Briana Dirks, Nisha Sipes

• DSSTOX Chemical Curation: Antony Wlliams

• Technical Reviewers and ToxRefDB Users: Grace Patlewicz (GenRA), Chelsea Weitekamp
(ToxValDB), Alison Harrill

• Data Infrastructure and ETL support: Sarangapani Addanki (GDIT), Jamil Shah, James Renner

• Management: Colleen Elonen, Amar Singh, Norman Adkins, Stephen Edwards

-------
References

1. Feshuk M, Kolaczkowski L, Watford S, Paul Friedman K. ToxRefDB v2.1: update to curated in
vivo study data in the Toxicity Reference Database. Front Toxicol. 2023 Sep 11;5:1260305. doi:
10.3389/ftox.2023.1260305. PMID: 37753522; PMCID: PMC10518696.

2. Watford, S., Pham, L.L., Wgnall, J., Shin, R., Martin, M.T., and Friedman, K.P. (2019).
ToxRefDB version 2.0: Improved utility for predictive and retrospective toxicology
analyses. Reproductive Toxicology, 89, 145-158.

3. Pham, L.L., Watford, S., Friedman, K.P., Wignall, J.A., and Shapiro, A.J. (2019). Python BMDS:
A Python interface library and web application for the canonical EPA dose-response modeling
software. Reproductive toxicology.

4. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.3100 90-Day Oral Toxicity in Rodents.

5. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.3150 90-Day Oral Toxicity in Nonrodents.

6. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.3200 21/28-Day Dermal Toxicity.

7. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.3250 90-Day Dermal Toxicity

8. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.3465 90-Day Inhalation Toxicity.

9. U.S. Environmental Protection Agency. (2000). Health Effects Test Guidelines: OPPTS
870.3550 Reproduction/ Developmental Toxicity Screening Test.

10. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.3700 Prenatal Developmental Toxicity Study.

11. U.S. Environmental Protection Agency (1998). Health Effects Test Guidelines: OPPTS
870.3800 Reproduction and Fertility Effects 1998.

12. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.4100 Chronic Toxicity.

13. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.4200 Carcinogenicity.

14. U.S. Environmental Protection Agency. (1998). Health Effects Test Guidelines: OPPTS
870.4300 Combined Chronic Toxicity/Carcinogenicity.

-------