ENY/JM/MONO(2016)12 | 3
OECD Environment, Health and Safety Publications
Series on Testing and Assessment
No. 233
USERS' HANDBOOK SUPPLEMENT TO THE GUIDANCE DOCUMENT FOR DEVELOPING
AND ASSESSING AOPs
INTER-ORGANIZATION PROGRAMME FOR THE SOUND MANAGEMENT OF CHEMICALS
A cooperative agreement among FAQ, ILO, UNDP, UNEP, UNIDO, UNITAR, WHO, World Bank and OECD
Environment Directorate
ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT
Paris 2016
Unclassified
-------
ฎ ปOECD
Organisation for Economic Co-operation and Development
ENV/JM/MONO(2016)12
Unclassified English - Or. English
14 February 2018
ENVIRONMENT DIRECTORATE
JOINT MEETING OF THE CHEMICALS COMMITTEE AND THE WORKING PARTY
ON CHEMICALS, PESTICIDES AND BIOTECHNOLOGY
Cancels & replaces the same document of 27 January 2017
USERS' HANDBOOK SUPPLEMENT TO THE GUIDANCE DOCUMENT FOR
DEVELOPING AND ASSESSING AOPs
Series on Testing & Assessment
No. 233
Series on Adverse Outcome Pathways
No. 1
Second Edition of the Users' Handbook, replacing the version dated 2017.
JT03426727
This document, as weU as any data ami map included herein, are without prejudice to the status of or sovereignty over any territory, to the
delimitation of international frontiers ami boundaries ami to the name of any territory, city or area.
-------
2 | ENV/JM/MONO(2016)12
Unclassified
-------
4 | ENV/JM/MONO(2016)12
About the OECD
The Organisation for Economic Co-operation and Development (OECD) is an intergovernmental
organisation in which representatives of 35 industrialised countries in North and South America, Europe
and the Asia and Pacific region, as well as the European Commission, meet to co-ordinate and harmonise
policies, discuss issues of mutual concern, and work together to respond to international problems. Most
of the OECD's work is carried out by more than 200 specialised committees and working groups
composed of member country delegates. Observers from several countries with special status at the
OECD, and from interested international organisations, attend many of the OECD's workshops and other
meetings. Committees and working groups are served by the OECD Secretariat, located in Paris, France,
which is organised into directorates and divisions.
The Environment, Health and Safety Division publishes free-of-charge documents in twelve different
series: Testing and Assessment; Good Laboratory Practice and Compliance Monitoring; Pesticides;
Biocides; Risk Management; Harmonisation of Regulatory Oversight in Biotechnology; Safety of
Novel Foods and Feeds; Chemical Accidents; Pollutant Release and Transfer Registers; Emission
Scenario Documents; Safety of Manufactured Nanomaterials; and Adverse Outcome Pathways.
More information about the Environment, Health and Safety Programme and EHS publications is
available on the OECD's World Wide Web site (www.oecd.org/chemicalsafety/).
This publication was developed in the IOMC context. The contents do not necessarily reflect the
views or stated policies of individual IOMC Participating Organizations.
The Inter-Organisation Programme for the Sound Management of Chemicals (IOMC) was established
in 1995 following recommendations made by the 1992 UN Conference on Environment and
Development to strengthen co-operation and increase international co-ordination in the field of
chemical safety. The Participating Organisations are FAO, ILO, UNDP, UNEP, UNIDO, UNITAR,
WHO, World Bank and OECD. The purpose of the IOMC is to promote co-ordination of the policies
and activities pursued by the Participating Organisations, jointly or separately, to achieve the sound
management of chemicals in relation to human health and the environment.
Unclassified
-------
ENY/JM/MONO(2016)12 | 5
This publication is available electronically, at no charge.
For this and many other Environment,
Health and Safety publications, consult the OECD's
World Wide Web site (www.oecd.org/ehs)
or contact:
OECD Environment Directorate,
Environment, Health and Safety Division
2, rue Andre-Pascal
75775 Paris cedex 16
France
Fax : (33-1) 44 30 61 80
E-mail: ehscont@oecd.org
ฉ OECD 2016
Applications for permission to reproduce or translate all or part of this material should be made
to: Head of Publications Service, RIGHTS@oecd.org, OECD, 2 rue Andre-Pascal, 75775
Paris Cedex 16, France
Unclassified
-------
6 | ENV/JM/MONO(2016)12
ฃnvord
This document is the Users' Handbook supplement to the Guidance Document for
developing and assessing Adverse Outcome Pathways (AOPs) [ENV/JM/MONO(2013)6,
Second Edition], The latter provides a historical background for the AOP development
programme, and outlines the elements required to construct an AOP as well as the
principles of the AOP framework.
The Users' Handbook supplement was prepared initially in June 2014 by a subgroup of
the OECD's Extended Advisory Group on Molecular Screening and Toxicogenomics
(EAGMST). At that time it was acknowledged that the Handbook should be revised once
expert groups and member countries acquire experience in developing and assessing
AOPs. Since then, some experience has been gained in developing AOPs, and the
material related to the AOP knowledgebase (AOP-KB) and the weight of evidence (WoE)
considerations has significantly evolved. For these reasons, the Handbook was revised by
the same subgroup of EAGMST to amend and incorporate new material based on lessons
learned.
The Users' Handbook was reviewed and discussed by EAGMST at the 10th meeting of
the EAGMST, in June 2017, and endorsed by EAGMST through written procedure in
July 2017. It was subsequently sent to the Working Group of the National Coordinators of
the Test Guidelines Programme and to the Working Party on Hazard Assessment who
approved it by written procedure.
The Joint Meeting of the Chemicals Committee and the Working Party on Chemicals,
Pesticides and Biotechnology agreed to declassification of this document in January 2018
This document is being published under the responsibility of the Joint Meeting of the
Chemicals Committee and the Working Party on Chemicals, Pesticides and
Biotechnology.
Unclassified
-------
ENY/JM/MONO(2016)12 | 7
Table of can ten ts
Foreword 6
Executive Summary 9
Introduction to Adverse Outcome Pathways (AOPs) 12
Obtaining Author Access to the AOP-KB 16
A Note on AOP Descriptions in the AOP-KB 17
Section 1 - AOP Description 18
AOP Identifier and Title 18
Graphical Representation of the AOP 19
Authors of AOP 20
Status and Date Modified 21
Abstract 24
Background (Optional) 24
KEand KER Tables 25
Network View 25
Stressors 26
Section 2 - KE Descriptions 27
KE ID 27
KE Title 27
Short Name 28
Level of Biological Organisation 28
KE Components and Biological Context 28
Other AOPs that use this KE 29
KE Description 29
How it is Measured or Detected 30
Biological Domain of Applicability 30
MIE-Specific Content 32
AO-Specific Content 34
References 34
Section 3 - KER Descriptions 35
KER ID 37
KER Title 37
AOPs Referencing Relationship 37
Biological Domain of Applicability 37
KER Description 39
Evidence Supporting this KER 39
Quantitative Understanding 42
Unclassified
-------
8 | ENV/JM/MONO(2016)12
References 46
Section 4 - Overall Assessment of the AOP 49
Define the Biological Domain of Applicability of the AOP 49
Assess the Essentiality of All KEs 50
Assess the Evidence Supporting All KERs 52
Quantitative WoE considerations (optional) 55
Review the Quantitative Understanding for Each KER 55
References 57
Annex 1: Guidance for Assessing Relative Level of Confidence in the Overall AOP 59
Annex 2: General guidance for characterising the level of quantitative understanding of a KER as
low, moderate, or high 62
Unclassified
-------
ENY/JM/MONO(2016)12 | 9
Executive Summary
This document is a supplement to the Guidance Document for developing and assessing
Adverse Outcome Pathways (AOPs) [ENV/JM/MONO(2013)6] (AOP Guidance
hereafter).
The AOP Guidance, originally published in 2013 and revised in 2017, provides an
introduction to the terminology and concepts of AOP development, including the
identification and use of relevant scientific data and resulting knowledge. The Guidance
also briefly outlines some potential applications of AOPs.
While the AOP Guidance provides a set of definitions and the conceptual background
behind AOP development, the present document, the OECD AOP Users' Handbook, is
designed to provide focused, in-depth, and practical instructions concerning development
and review of AOP-descriptions disseminated through the internationally harmonised
AOP knowledgebase (AOP-KB, http://aopkb.org or https://aopkb.oecd.org/). The Users'
Handbook can be thought of as being analogous to the "instructions for authors" used in
preparing a journal article. However, in this case, rather than describing the preparation of
a technical manuscript, this Handbook details how to structure an AOP description in the
AOP-KB. This handbook contains an updated template for AOP development that is
organised into sections. Each section of the handbook described below aligns with
corresponding sections within the pages to be constructed within the AOP-KB (Table 1).
In this manner, the Handbook is intended to assist in identifying, organising and
evaluating critical information on key events (KEs) as well as linkages between KEs,
termed key event relationships (KERs), within the AOP (i.e., AOP development). It also
provides more explicit guidance on how to assemble and assess the weight of evidence
(WoE) (degree of confidence) supporting the AOP and its relevance for different life
stages, sex, taxa, etc. The content of the Handbook is specifically designed to support
entry of AOP information into the AOP-Wiki ("http://aopwiki.org. one module of the
AOP-KB), and will be updated and harmonised as the AOP-KB evolves.
As with the AOP Guidance itself, this Handbook is not intended to provide a review or
summary of the literature informing the AOP concept. Instead, it focuses on practical
aspects of AOP development and assessment. Likewise, the Handbook is not intended to
provide guidance on determining the appropriate or inappropriate regulatory application
of AOPs. However, by following the template and practices outlined in the Users'
Handbook, AOP developers should be in a position to systematically and efficiently
assemble information pertinent to their AOP (the focus of Sections 1-3), and evaluate the
underlying WoE (the focus of Section 4). This should provide transparent assessment of
the level of confidence in the overall AOP, as well as critical gaps and uncertainties that
Unclassified
-------
10 I ENV/JM/MONO(2016)12
are relevant to decisions regarding appropriate regulatory applications. Although potential
regulatory applications (e.g., developing Test Guidelines, forming categories, informing
integrated approaches to testing and assessment (IATA), or risk assessments within
different regulatory contexts) may be described in Section 4, this information is
considered optional.
AOP descriptions developed as part of the OECD AOP Development Programme are
peer-reviewed as per procedures outlined by the OECD. Because AOP descriptions
within the AOP-KB are viewed as living documents, they are expected to continue to
evolve over time as new evidence supporting or rejecting AOPs are generated and/or new
knowledge is gained. Consequently, AOPs that are reviewed and endorsed by the OECD
will have multiple versions, namely, the version that existed at the time of the review and
endorsement and the current version that exists in the AOP-KB. Reviews are performed
on "snapshots" of content from the AOP-KB, as it existed when review was initiated.
These snapshots are permanently stored in the AOP-KB along with the living document
to clearly distinguish between the version of the AOP that has been endorsed and the
current state of knowledge. The snapshot corresponding to the endorsed version of the
AOP are also published in the OECD series on Adverse Outcome Pathways
rhttp://www.oecd-ilibrarv.org/environment/oecd-series-on-adverse-outcome-
pathways 2415170x1. The AOP-KB allows the download of both current AOP
information and all snapshots in PDF form. It also provides tools for examining the
differences between any snapshot and the current version of the AOP.
The sections in the handbook are presented in the order in which information should be
assembled during AOP development. Based on feedback from previous AOP reviewers, it
was determined that this is not optimal for AOP review. As a result, while the
"snapshots" extracted from the AOP-KB capture all contents as described in this Users"
Handbook sections, the order in which they are presented is different. Specifically, the
"snapshots" contain a brief summary of the AOP followed by the overall assessment of
the AOP, whereas the detailed description of the KEs and evidence supporting the KERs
are presented as appendices (Table 1).
Table 1. Overview of the organisation of content pages in the AOP-KB and reviewer
snapshots relative to sections of the Handbook
SECTIONS OF AOP-KB AND/OR SNAPSHOT REPORT
HANDBOOK
SECTION
AOP Description
Key Event (KE) Descriptions
AOP-KB: Each KE description is on a separate page that
is reached via a link from the KE Table in the AOP
Summary Section
Snapshot: Each KE description is found in Appendix 1
Key Event Relationship (KER) Descriptions
AOPiKB: Each KER description is on a separate page
Section 1
Section 2
Section 3
Unclassified
-------
ENY/JM/MONO(2016)12 | 11
that is reached via a link from the KER Table in the AOP
Summary Section
Snapshot: Each KER description is found in Appendix 2
Overall Assessment of the AOP Section 4
See (https://aomiiki.ore/info paees/2/info linked pases/5) for the details regarding snapshots created from
the AOP-KB.
Unclassified
-------
12 | ENV/JM/MONO(2016)12
Introduction to Adverse Outcome Pathways (AOPs)
An AOP describes a sequence of events commencing with initial interaction(s) of a
stressor with a biomolecule within an organism that causes a perturbation in its biology
(i.e., molecular initiating event, MIE), which can progress through a dependent series of
intermediate key events (KEs) and culminate in an adverse outcome (AO) considered
relevant to risk assessment or regulatory decision-making (Table 2). AOPs are typically
represented sequentially, moving from one KE to another. In this respect, AOPs define a
series of measurable biological changes that can be expected to occur if the perturbation
is sufficiently severe (i.e., in terms of potency, duration, frequency) to drive the pathway
all the way to the AO. Importantly, AOPs do not describe every detail of the biology but
instead focus on describing critical steps or check-points along the path to adversity,
which are both measurable and have potential predictive value. While the focus of AOP
development is to capture and organise what is known, the process of AOP development
may also identify current knowledge gaps which, if filled, could further improve
predictive utility.
Table 2. Definitions of key terms and abbreviations used in this Handbook (see AOP
guidance for additional terminology relevant to the AOP framework and its application).
A specialised type of key event that represents the initial
point of chemical/stressor interaction at the molecular level
within the organism that results in a perturbation that starts
the AOP.
A change in biological or physiological state that is both
measurable and essential to the progression of a defined
biological perturbation leading to a specific adverse
outcome.
A scientifically-based relationship that connects one key
event to another, defines a causal and predictive relationship
between the upstream and downstream event, and thereby
facilitates inference or extrapolation of the state of the
downstream key event from the known, measured, or
predicted state of the upstream key event.
A specialised type of key event that is generally accepted as
being of regulatory significance on the basis of
correspondence to an established protection goal or
equivalence to an apical endpoint in an accepted regulatory
guideline toxicity test.
Molecular
initiating MIE
event
Key event KE
Key event KER
relationship
Adverse . ^
^ A0
Outcome
Unclassified
-------
ENY/JM/MONO(2016)12 | 13
KEs are measurable biological changes that are essential to the progression along an
AOP. Essentiality implies that the KEs play a causal role in the pathway such that if the
KE is prevented or fails to occur, progression to subsequent KEs in the pathway will not
occur. While KEs are essential to progression along the AOP, they are not necessarily
sufficient. Just because a particular KE is observed, does not mean the perturbation will
necessarily progress all the way to the AO. Rather, the conditions under which
progression can be expected are described as part of the KERs that link one KE to another
in sequence to form an AOP.
The AOP framework provides a transparent and scientifically-based means to organise
and present current knowledge of predictable relationships between MIEs, subsequent
KEs and AOs. The objective underlying AOP development is to ultimately support
inference or extrapolation from one KE to another. Most notably, consistent with the
proposed vision for regulatory toxicology in the 21st century, there is considerable
interest in extrapolating from KE measurements that may be made efficiently and cost-
effectively, typically at low levels of biological organisation, to adverse effects at higher
levels of organisation that are relevant to regulatory protection goals and decision-making
(Krewski et al. 2010). The overall WoE and level of certainty underlying the inference
and extrapolation will in turn dictate the most suitable application of the AOP.
Assessment of AOPs and evaluation of their suitability for application in different
regulatory contexts and the assimilation of the relevant characterisation of supporting
biological information relies in part on (1) the confidence and precision with which the
KEs can be measured, (2) the level of confidence in the relationships between the KEs
linked in an AOP (KERs) based on biological plausibility, and empirical support for the
KERs; and (3) WoE for the overall hypothesised pathway, taking into account a number
of additional considerations, including any uncertainties and inconsistencies. Therefore,
overall assessment of AOPs is best supported by providing thorough descriptions of the
KEs [Section 2], relationships between those KEs [i.e., KERs, Section 3], and by robust
consideration of supporting evidence for the biological plausibility and empirical support
for KERs [Section 3D] and essentiality of KEs [Section 4], Consequently, the Handbook
and AOP-KB are structured in a manner that prompts AOP developers to provide relevant
types of supporting information.
Principles of AOP Development and their Implications for AOP Description
As a pragmatic convention, AOPs are conceptualised as a single sequence of events
proceeding from the MIE to the AO via a series of intermediate KEs. That is, they
describe how one particular molecular perturbation may cause one AO, not every possible
AO that perturbation may cause, nor every way a particular AO may arise. It is
recognised that MIEs, KEs, and AOs may be shared by more than one AOP.
Consequently, it is desirable to describe KEs as discrete (modular) units without reference
to a specific MIE, AO, or other KEs. Likewise, it is useful to describe relationships
between discrete pairs of KEs (KERs), without reference to other elements of the AOP.
This facilitates generation of generic KE or KER descriptions that can be linked to
Unclassified
-------
14 | ENV/JM/MONO(2016)12
multiple other AOPs. Such an approach both fosters consistency and increases
efficiencies in the AOP development process by eliminating the need for AOP developers
to completely re-describe biological measurements (KEs) or evidence supporting
inference from one KE to another (KERs) that another developer may have already
detailed. Maintaining KE and KER descriptions as discrete units that avoid reference to
other elements of the AOP also facilitates the updating of KE and KER descriptions as
new methods for measuring KEs or new evidence supporting KERs are developed.
Finally, it facilitates the construction and conceptualisation of AOP networks.
An AOP network is defined as an assembly of two or more AOPs that share one or more
KEs in common. If the components of an AOP (KEs and KERs) are described in a
modular fashion, AOP networks emerge from the description of individual AOPs that
share KEs. In a network, KEs represent nodes while KERs represent directed edges that
link those nodes together. Through their defacto construction as more AOPs are added to
the AOP-KB, AOP networks can be viewed as capturing broader knowledge concerning
the range of possible AOs a perturbation may cause, or the range of ways in which a
particular adverse outcome may occur. AOP networks are also critical for addressing
exposures to multiple stressors that lead to the same AO or to individual stressors that
perturb multiple MIEs (Knapen et al., 2015; Villeneuve et al., 2014a, b) and for
understanding potential interactions between co-occurring AOPs.
In describing the KEs and KERs that make up an AOP, each information field of the KE
or KER description should be completed as thoroughly as is practical and supported
through citation of primary literature and other references as appropriate. It is recognised
that AOP descriptions reflect current knowledge and will evolve as additional information
becomes available. In this respect, AOP descriptions should be regarded as "living
documents". Not all sections described need to be completed immediately. It is expected
that AOPs may have gaps that may be addressed over time as the science progresses or as
other researchers contribute. Likewise, collaboration and contributions from other
developers is encouraged.
Indeed, AOPs provide a relevant construct to promote collaboration between experts in
various areas of research and the regulatory risk assessment community as a basis to
better coordinate and tailor research to practical application, such as the development of
KE-based testing strategies. Collaboration between a range of experts with expertise in
these different areas in the development and assessment of AOPs is therefore strongly
encouraged. The AOP-Wiki facilitates this collaboration by providing a tool to organise
and share the relevant data and information. Consequently, it is recommended that
descriptions are structured in a way that facilitates addition and revision of information as
it is developed; for example, through the use of bullets or tables and organisation into
topical subsections rather than development of extensive narrative text. Rather than
representing a daunting compilation of information that must be assembled to adequately
describe an AOP, the sections defined in the Handbook and AOP-KB should be viewed
as an organisational structure for assembling a transparent record of scientific support for
an AOP and a basis for clear delineation of current gaps in our knowledge. Organisation
Unclassified
-------
ENY/JM/MONO(2016)12 | 15
of information in this manner is intended to facilitate collaborative development,
transparency, and appropriate use of the information assembled.
In this handbook, particular emphasis is placed on sections of the template related to the
description of the MIE, KEs and AO in an AOP (Section 2), the assembly of available
scientific evidence supporting the KERs (Section 3) and the summation of the support for
the AOP as a whole (Section 4) as a basis to consider its potential application.
If the author aims to publish the AOP in the OECD Series on AOPs, it is strongly
encouraged that the text of the AOP description, including the list of references, should
conform, to the extent possible, with the OECD Style Guide
(https: //www. oecd.org/about/publishing/OECD -Style -Guide -Third-Edition .pdf) (OECD.
2015).
Unclassified
-------
16 | ENV/JM/MONO(2016)12
Obtaining Authr .ฆ :ess to the
Read-access to all contents of the AOP-KB ("http://aopkb.org/) and the AOP-Wiki
(https://aopwiki.org) is publicly available world-wide without need to create a user
profile, login ID, or password.
Commentor access: A self-created user account, with a verified email address, grants the
user the ability to comment on all pages in the AOP-Wiki including AOPs, KEs, and
KERs. Users can create an account on the AOP-Wiki by following the instructions here:
https://aopwiki.org/info pages/2#Contributing to the AOP Wiki
Author Access: In order to create or edit AOPs, KEs, or KERs, the user must request
author access to the AOP-Wiki by following the instructions here:
https://aopwiki.org/info pages/2 Red nesting Author Access.
Unclassified
-------
ENY/JM/MONO(2016)12 | 17
A Note on . scriptions in the'
AOP descriptions in the AOP-KB can be viewed as consisting of two types of
information, structured information and free text.
Structured information is derived from standardised ontologies available through look-
up tables or by making selections from a drop-down list. Structured information fields
within the AOP-KB populate a back-end database. The terms and information in that
database is machine-readable and can be used to aid various computational analyses,
querying, and searching of the AOP-KB. For example, construction of AOP networks
from the modular units of individual AOP descriptions relies on these structured
annotation fields.
Free text sections in the AOP-KB provide AOP developers with much greater descriptive
flexibility than structured information fields. While free text is searchable, it is not
standardised and machine-readable, and has much more limited use from a computational
standpoint.
As a means to balance computational accessibility along with the desire for descriptive
accuracy and richness, the AOP-KB incorporates both elements. Consequently, AOP
developers are encouraged to complete both the structured information and free text
sections of the AOP descriptions to the extent they are able.
Unclassified
-------
18 | ENV/JM/MONO(2016)12
Section 1 - AOP Description
This section is for information that describes the overall AOP. The information described
in section 1 is entered on the upper portion of an AOP page within the AOP-Wiki. This is
where some background information may be provided, the structure of the AOP is
described, and the KEs and KERs are listed.
AOP Identifier and Title
This subsection provides guidance for naming the AOP.
A OP Identifier
Each AOP is automatically given a numerical AOP identifier when it is created.
AOP Title
Each AOP should be given a descriptive title that takes the form "MIE leading to AO".
For example, "Aromatase inhibition [MIE] leading to reproductive dysfunction [AO]" or
"Thyroperoxidase inhibition [MIE] leading to decreased cognitive function [AO]". In
cases where the MIE is unknown or undefined, the earliest known KE in the chain (i.e.,
furthest upstream) should be used in lieu of the MIE and it should be made clear that the
stated event is a KE and not the MIE.
In some cases, AOPs linking the same MIE to the same AO may proceed through
different intermediate KEs. Naming based on MIE and AO alone can result in a series of
distinct AOPs with the same title. While these are distinguished in the AOP-KB by their
AOP page ID numbers, each of which has a distinct URL, it can be hard for users to
discriminate them at a glance. In such cases, an additional descriptor should be added to
the title using the form "MIE leading to AO via distinctive KE". For example,
"cyclooxygenase inhibition [MIE] leading to reproductive dysfunction [AO] via
inhibition of pheromone release" versus "cyclooxygenase inhibition [MIE] leading to
reproductive dysfunction [AO] via interference with spindle assembly checkpoint".
Unclassified
-------
ENY/JM/MONO(2016)12 | 19
Short Name
A short name should also be provided that succinctly summarises the information from
the title. This name should not exceed 90 characters.
Implementation in the AOP-Wiki
https://aopwiki.org/info pages/2/info linked pages/l#A Create a new AOP
Graphical Representation of the AOP
A graphical summary of the AOP listing all the KEs in sequence, including the MIE (if
known) and AO, and the pair-wise relationships (links or KERs) between those KEs
should be provided. This is easily achieved using the standard box and arrow AOP
diagram (Figure 1).
KER
KER
KER,
MIE
AO
n-l
Figure 1. Example of a generic AOP diagram in its simplest form (see also
htti)s://aoi)wiki.org/aoi)s/23#grai)hical representation)
Development tip 1 - Graphical Representation: The graphical representation (AOP
diagram) serves as a useful road-map to guide AOP development in the AOP-KB. For
this reason, it is recommended that an AOP diagram be developed prior to creating an
AOP description in the AOP-KB. Starting with the graphical summary provides a useful
overview of the KE and KER pages that will need to be included. Ideally, development of
a graphical overview of the AOP to be developed should be followed by a search of
existing content to determine whether analogous AOPs and/or synonymous KEs or KERs
may already exist in the knowledgebase. This can avoid duplicative effort and help to
ensure that KEs and KERs are shared among AOPs, allowing for de facto creation of
AOP networks. Once existing KE and KER pages relevant to the AOP have been
identified, the developer then knows which pages in the AOP-KB will need to be created
de novo.
The graphical summary is prepared and uploaded by the user (templates are available)
and is often included as part of the proposal when AOP development projects are
submitted to the OECD AOP Development Workplan.
Unclassified
-------
20 | ENV/JM/MONO(2016)12
The graphical representation or AOP diagram provides a useful and concise overview of
the KEs that are included in the AOP, and the sequence in which they are linked together.
This can aid both the process of development, as well as review and use of the AOP.
Development tip 2 - Number of KEs to include: Determining the number of KEs to
include in an AOP and the specificity with which they are defined is one of the
more challenging aspects of AOP development. In describing KEs within an
AOP, it is important to recognise their distinction from "mechanism of action".
AOPs provide a description of a limited number of essential, measurable events
(check-points) leading to induction of the relevant toxicity endpoint. They do not
necessarily provide a comprehensive molecular description of every aspect of the
biology involved. With that in mind, the following "rules of thumb" can help
guide the process of KE definition (Villeneuve et al. 2014a, b):
Where possible and appropriate for application, try to include at least one
KE at each major level of biological organisation (molecular, cellular,
tissue, organ, individual).
Where feasible/appropriate, focus on KEs that can be measured in a
relatively routine manner over those that require highly specialised
expertise, equipment, or supplies to measure. These will tend to be the
KEs for which empirical evidence to support KERs is more likely to be
available to support the WoE evaluation.
Select a limited number of KEs that are measurable and for which
evidence supports plausibility and potential predictive utility. Where
relevant, more detailed description of the underlying biology involved can
be incorporated into the descriptions of the biological plausibility linking
two KEs (see section 3 - KER descriptions).
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#B Graphical Representation of the
AOP
Authors of AOP
This section provides guidance on author identification.
Authors and Affiliations
List the name and affiliation information of the individual(s)/organisation(s) that
created/developed the AOP. In the context of the OECD AOP Development Workplan,
this would typically be the individuals and organisation that submitted an AOP
development proposal to the EAGMST. Significant contributors to the AOP should also
Unclassified
-------
ENY/JM/MONO(2016)12 | 21
be listed. A corresponding author with contact information may be provided here. This
author does not need an account on the AOP-KB and can be distinct from the point of
contact below. The list of authors will be included in any snapshot made from an AOP.
Point of Contact
Indicate the point of contact for the AOP-KB entry itself. This person is responsible for
managing the AOP entry in the AOP-KB and controls write access to the page by
defining the contributors as described below. Clicking on the name will allow any wiki
user to correspond with the point of contact via the email address associated with their
user profile in the AOP-KB. This person can be the same as the corresponding author
listed in the authors section but isn't required to be. In cases where the individuals are
different, the corresponding author would be the appropriate person to contact for
scientific issues whereas the point of contact would be the appropriate person to contact
about technical issues with the AOP-KB entry itself. Corresponding authors and the point
of contact are encouraged to monitor comments on their AOPs and develop or coordinate
responses as appropriate.
AOP-KB Contributors
List user names of all authors contributing to or revising pages in the AOP-KB that are
linked to the AOP description. This information is mainly used to control write access to
the AOP page and is controlled by the point of contact.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#C Authors of AOP
Status and Date Modified
This section provides guidance on the various status trackers for AOPs.
Author Status
The status section is used to provide AOP-KB users with information concerning how
actively the AOP page is being developed, what type of use or input the authors feel
comfortable with given the current level of development, and whether it is part of the
OECD AOP Development Workplan and has been reviewed and/or endorsed. "Author
Status" is an author defined field that is designated by selecting one of several options
from a drop-down menu (Table 3). The "Author Status" field should be changed by the
point of contact, as appropriate, as AOP development proceeds.
Unclassified
-------
22 | ENV/JM/MONO(2016)12
Table 3. Drop-down options for "Author status" field
Selection
Explanation
This is the default status assigned when a new AOP page is
created in the AOP-Wiki. It is used to indicate that the
project team is actively developing the pages and that the
author(s) have new content they expect to add such that
commenting on or citing the existing content is premature.
Under
development: not
open for comment;
Do not cite
Open for
comment; do not
cite
This status is used to indicate that the authors have added
the primary content they wish to include and they invite the
community to comment on that content via the Discussion
pages. However, this designation indicates that the authors
do not feel the AOP should be cited in its current form. For
example, perhaps they have identified major uncertainties or
gaps that still need to be addressed. This is a common
designation to use for AOPs that represent a hypothesised
AOP for which supporting evidence has not yet been
assembled.
This status is used to indicate that the author(s) have added
the content they wish to include on their AOP page (and the
associated KE and KER pages) and they invite the
Open for citation community to comment on that content via the Discussion
and comment pages and cite the AOP in its current form, if desired. This
designation usually indicates that the authors stand behind
their contribution and take responsibility for the scientific
content.
Open for adoption
This refers to "adoption" in the sense of new authors taking
over responsibility for further development of the AOP. It
should not be confused with an AOP that should be
considered for endorsement or use. This status is used to
indicate that the primary author(s) of the AOP are no longer
actively working on the page, but would like to invite others
from the community to take-over development of the AOP.
An open for adoption status also signals the curators of the
AOP-Wiki that the authors feel the content provided
warrants further development. AOPs that are open for
adoption will not be deleted from the AOP-KB without first
consulting the current Point of Contact.
Not under active
development
This status indicates the primary author(s) of the AOP are
no longer actively working on the page. Others may still
contact the authors about taking-over development of the
pages if desired. However, the content provided may or may
not warrant further development. AOPs with this status
Unclassified
-------
ENY/JM/MONO(2016)12 | 23
designation are subject to deletion at the discretion of the
curators of the AOP-KB.
OECD Status
For AOPs that are included in a project that has been accepted into the OECD AOP
Development Workplan (see http: //www. oecd.org/chemicalsafetv/testing/proi ects -
adverse -outcome -pathways .htm). status with regard to progress through OECD review
and endorsement processes is tracked by the OECD EAGMST. 'OECD status" tracks the
level of review/endorsement the AOP has been subjected to. This designation is managed
and updated by the OECD. It is not selected by the AOP author(s).
OECD Project Number
The OECD project number is also indicated along with the current status of the AOP with
regard to the OECD Development Workplan. This designation is managed and updated
by the OECD. It is not selected by the AOP author(s).
SAA OP Status
All AOPs under development in the AOP-KB are monitored by curators who are
members of the Society for the Advancement of AOPs (SAAOP). These curators
maintain a separate status designation for AOPs based on their evaluation of the current
state of the AOP. These designations (Table 4) are managed and updated by the SAAOP
curators. They are not selected by the AOP author(s). Currently the SAAOP status list
includes the following:
Table 4. Explanation for SAAOP status
SAAOP Status Explanation
Included in the An AOP development project proposal has been reviewed by
OECD work OECD EAGMST, accepted into the workplan, and a project
plan number assigned.
Proposed for A SAAOP curator has encouraged the author to submit a
OECD work proposal to OECD. Indicates well developed content that is
plan likely suitable for review.
Under Indicates the SAAOP views the content as still under
development development and not ready for formal review.
Indicates that the entry is likely to be deleted. AOPs with an
archived status are not listed when a user is browsing the AOPs
Archive but they will show up when a search is made. This is typically
for AOPs that are not under active development and not suitable
for adoption.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#D Status of an AOP
Unclassified
-------
24 | ENV/JM/MONO(2016)12
Date Modified
The date the AOP was last modified is automatically tracked by the AOP-KB. The date
modified field can be used to evaluate how actively the page is under development and
how recently the version within the AOP-Wiki has been updated compared to any
snapshots that were generated.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#Historv of Modifications
Abstract
In the abstract section, authors should provide a concise and informative summation of
the AOP under development that can stand-alone from the AOP page. Abstracts should
typically be 200-400 words in length (similar to an abstract for a journal article).
Suggested content for the abstract includes the following: (1) the background/purpose for
initiation of the AOP's development (if there was a specific intent); (2) a brief description
of the MIE, AO, and/or major KEs that define the pathway; (3) a short summation of the
overall WoE supporting the AOP and identification of major knowledge gaps (if any); (4)
if a brief statement about how the AOP may be applied (optional). The aim is to capture
the highlights of the AOP and its potential scientific and regulatory relevance.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#E AOP Abstract
https://aopwiki.org/info pages/2/info linked pages/1 -
05%20To%20edit%20AQP%20abstract
Background (Optional)
This optional subsection should be used to provide background information for AOP
reviewers and users that is considered helpful in understanding the biology underlying the
AOP and the motivation for its development. The background should NOT provide an
overview of the AOP, its KEs or KERs, which are captured in more detail below. A few
examples of potential uses of the optional background section include:
If the AOP was a result of research funded through a particular grant or research program,
the authors may want to provide information regarding the source of funding for the
research that led to development of the AOP and the scope and key research questions the
over-arching research effort was designed to address.
Unclassified
-------
ENY/JM/MONO(2016)12 | 25
If the AOP is one of a series of related AOPs that the author(s) developed as part of a
network-guided approach to AOP development, the authors may want to make explicit
reference to other AOPs that were also developed as part of the effort in this section.
In the case of AOPs that were developed as a regulatory application case study or to
support a particular regulatory decision, the authors may want to provide a bit of
background on the problem formulation that motivated development of the AOP.
If there is some particularly interesting biology that is encompassed by the AOP that is
not necessarily evident from the KE and KER descriptions, but would likely be of interest
to other investigators with an interest in the AOP, those details could be provided here.
In general, this section is suitable for any additional information that does not necessarily
fit in other parts of the AOP description, but may be of interest to readers/users.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#F Background Information
KE and KE les
Tables listing each KE and KER are automatically created in the AOP-KB as KE pages to
link to the AOP are selected or created and as KERs are defined.
a. KE Table: This table summarises all of the KEs of the AOP, including the MIE
and AO. This table is populated in the AOP-Wiki as KEs are added to the AOP.
Each table entry acts as a link to the individual KE description page. For
guidance on completing the KE descriptions see Section 2.
b. KER Table: This table summarises all of the KERs of the AOP and is populated
in the AOP-Wiki as KERs are added to the AOP. Each table entry acts as a link
to the individual KER description page. For guidance on completing the KER
descriptions see Section 3.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#G KE and KER Tables
Network View
The AOP-Wiki automatically generates a network view of the AOP. This network
graphic is based on the information provided in the MIE, KEs, AO, KERs and WoE
summary tables. The width of the edges representing the KERs is determined by its WoE
Unclassified
-------
26 | ENV/JM/MONO(2016)12
confidence level, with thicker lines representing higher degrees of confidence. This
network view also shows which KEs are shared with other AOPs.
Examples in the AOP Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#H AOP Networks
https://aopwiki.org/aops/15#network view
https://aopwiki.Org/aops/23#network view
https://aopwiki.Org/aops/3 8#network view
Stressors
The stressor field is a structured data field that can be used to annotate an AOP with
standardised terms identifying stressors known to trigger the MIE/AOP. Most often these
are chemical names selected from established chemical ontologies. However, depending
on the information available, this could also refer to chemical categories (i.e., groups of
chemicals with defined structural features known to trigger the MIE). It can also include
non-chemical stressors such as genetic or environmental factors. Although AOPs
themselves are not chemical or stressor-specific, linking to stressor terms known to be
relevant to different AOPs can aid users in searching for AOPs that may be relevant to a
given stressor.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#Stressors
Unclassified
-------
ENY/JM/MONO(2016)12 | 27
Section 2 - KE Descriptions
Development tip 3 - Sharing of KEs:
Use existing KEs when possible - when adding KEs to an AOP it is strongly
recommended to use KEs that already exist in the AOP-KB as much as possible.
Existing KE requires modification - If an existing KE requires modification to
make it suitable, changes to the content on that page should be coordinated with
the point(s) of contact for other AOPs sharing the KE to ensure that the original
meaning is not altered.
Creating new KEs - If no suitable KEs are available in the knowledgebase, or if
the revisions needed to make an existing KE description suitable for the AOP
under-development would make it unsuitable for use in AOPs it is already linked
to, then a new KE should be created.
AOP-KB Etiquette - When using an existing KE, it is the responsibility of the
person making changes to ensure that KEs used in multiple AOPs are not altered
in such a way as to diminish the applicability of that KE for the existing AOPs.
Please be courteous to your fellow AOP developers.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#A-D Create a New Key Event
KE ID
When a KE is created, an ID number is automatically assigned to it. This number is used
for tracking the KE in the AOP-KB.
KE Title
The KE title should describe a discrete biological change that can be measured. It should
generally define the biological object or process being measured and whether it is
increased, decreased, or otherwise definably altered relative to a control state. For
example "enzyme activity, decreased", "hormone concentration, increased", or "growth
rate, decreased", where the specific enzyme or hormone being measured is defined.
Unclassified
-------
28 | ENV/JM/MONO(2016)12
Short Name
The KE short name should be a reasonable abbreviation of the KE title and is used in
labelling this object throughout the AOP-Wiki. The short name should be less than 80
characters in length.
Level of Biological Organisation
Structured terms, selected from a drop-down menu, are used to identify the level of
biological organisation for each KE. Note, KEs should be defined within a particular level
of biological organisation. Only KERs should be used to transition from one level of
organisation to another. Selection of the level of biological organisation defines which
structured terms will be available to select when defining the Event Components (below).
KE Components and Biological Context
Because one of the aims of the AOP-KB is to facilitate de facto construction of AOP
networks through the use of shared KE and KER elements, authors are also asked to
define their KEs using a set of structured ontology terms (Event Components). In the
absence of structured terms, the same KE can readily be defined using a number of
synonymous titles (read by a computer as character strings). In order to make these
synonymous KEs more machine-readable, KEs should also be defined by one or more
"event components" consisting of a biological process, object, and action with each
term originating from one of 22 biological ontologies (Ives, et al., 2017;
https://aopwiki.org/info pages/2/info linked pages/7#List). Biological process describes
dynamics of the underlying biological system (e.g., receptor signalling). The biological
object is the subject of the perturbation (e.g., a specific biological receptor that is
activated or inhibited). Action represents the direction of perturbation of this system
(generally increased or decreased; e.g., 'decreased" in the case of a receptor that is
inhibited to indicate a decrease in the signalling by that receptor).
Development tip 4 - How specifically should my KEs be defined:
The following are some general recommendations and "rules of thumb" concerning how
specifically to define a KE (see also Villeneuve et al. 2014a, b):
Define the KE with enough specificity that one would know what to measure to
determine the state of the KE. For example "histological changes" is too broad;
"oocyte atresia" or "hyperplasia" would be better.
KEs should refer to/focus on a single measurable event within a specific
biological level of organisation, rather than compounding events together. For
example, it would be better to define a KE as "enzyme activity, increased" (if that
can be measured), rather than "transcription and translation leading to enzyme
activity, increased".
The biological context of the KE (e.g., the tissue type/taxa/life stage/sex etc.)
Unclassified
-------
ENY/JM/MONO(2016)12 | 29
should only be restricted (e.g., "enzyme activity in liver, decreased" or "hormone
concentration in females, increased") to the extent that function changes with
context. If the function is equivalent in both sexes, do not restrict the context by
sex. If the function is equivalent in all cell types, do not restrict to a specific cell
type.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#E Event Components and
Biological Context
Other AOPs that use this KE
All of the AOPs that are linked to this KE will automatically be listed in this subsection.
This table can be particularly useful for derivation of AOP networks including the KE.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#F AOP
Informationhttps://aopwiki.org/info pages/2/info linked pages/3 - E Event Components
and Biological Context
KE Description
A description of the biological state being observed or measured, the biological
compartment in which it is measured, and its general role in the biology should be
provided. For example, the biological state being measured could be the activity of an
enzyme, the expression of a gene or abundance of an mRNA transcript, the concentration
of a hormone or protein, neuronal activity, heart rate, etc. The biological compartment
may be a particular cell type, tissue, organ, fluid (e.g., plasma, cerebrospinal fluid), etc.
The role in the biology could describe the reaction that an enzyme catalyses and the role
of that reaction within a given metabolic pathway; the protein that a gene or mRNA
transcript codes for and the function of that protein; the function of a hormone in a given
target tissue, physiological function of an organ, etc. Careful attention should be taken to
avoid reference to other KEs, KERs or AOPs. Only describe this KE as a single isolated
measurable event/state. This will ensure that the KE is modular and can be used by other
AOPs, thereby facilitating construction of AOP networks.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#Kev Event Description
Unclassified
-------
30 | ENV/JM/MONO(2016)12
How it is Measured or Detected
One of the primary considerations in evaluating AOPs is the relevance and reliability of
the methods with which the KEs can be measured. The aim of this section of the KE
description is not to provide detailed protocols, but rather to capture, in a sentence or two,
per method, the type(s) of measurements that can be employed to evaluate the KE and the
relative level of scientific confidence in those measurements. Methods that can be used to
detect or measure the biological state represented in the KE should be briefly described
and/or cited. These can range from citation of specific validated test guidelines, citation
of specific methods published in the peer reviewed literature, or outlines of a general
protocol or approach (e.g., a protein may be measured by ELISA).
Key considerations regarding scientific confidence in the measurement approach include
whether the assay is fit for purpose, whether it provides a direct or indirect measure of the
biological state in question, whether it is repeatable and reproducible, and the extent to
which it is accepted in the scientific and/or regulatory community. Information can be
obtained from the OECD Test Guidelines website
("http://www.oecd.org/chemicalsafetv/testing/oecdguidelinesforthetestingofchemicals.htm
) and the EURL ECVAM Database Service on Alternative Methods to Animal
Experimentation (DB-ALM) (https://ecvam-dbalm.irc.ec.europa.eu/).
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#How it is Measured or Detected
Biological Domain of Applicability
The relevant biological domain(s) of applicability of the KE in terms of sex, life-stage,
taxa, and other aspects of biological context are defined in this section. In essence, the
taxa/life-stage/sex applicability is defined based on the groups of organisms for which the
measurements represented by the KEs can feasibly be made with the same functional
interpretation.
Defining the taxonomic, life stage and sex relevance of each KE helps to bound the
domain of applicability of the AOP as a whole and provides an understanding of how
broadly data represented by a KE measurement may be extrapolated, including potential
human relevance. As a general guide, there are two primary considerations associated
with defining the applicability domain of a KE:
1. Structure: Is the biological object being measured/observed present/conserved in
the taxa/sex/life-stage of interest? Here biological object may refer to a protein, a cell
type, an organ, etc.
Unclassified
-------
ENY/JM/MONO(2016)12 | 31
2. Function: Is the function of that biological object and the process being measured
via the KE conserved and relevant in the taxa/sex/life-stage of interest. Does it play the
same role?
For example, if the KE involves binding to the estrogen receptor, but invertebrates lack a
functional homolog of the estrogen receptor, one could reasonably conclude that the AOP
is not relevant to invertebrates on the basis of a lack of conserved structure. Likewise, if
the KE involves a measurement in ovary tissue, its applicability domain in terms of sex
would be restricted to females. If a KE involves altered organogenesis (e.g., heart
formation), the KE would only be relevant to the life-stage during which the heart is
actually formed, and not to the adult life-stage in which organ development has already
completed.
Biological Domain of applicability is defined in the AOP-KB using a combination of
structured fields and free text. Structured terms can be selected to identify the taxa, life
stage, sex, and the level of biological organisation (e.g., cell, tissue or organ) for which
the KE is known to be applicable. Selection of structured terms to describe the
applicability domain can aid AOP network construction as well as facilitating other types
of computational processing and searching of information captured in the AOP-KB.
At the time that developers select structured ontology terms to help define the domain of
applicability of the KE, there is also an option to make evidence calls related to
applicability of the specific KE in question to that category term. These calls should be
based on expert knowledge of the biology and the extent of supporting experimental
evidence. Recommendations for these calls are:
Low: With the understanding that by definition a KE must be measurable in the
species/taxonomic group/lifestage/sex defined, no such measurements have been
reported or shown experimentally to date;
Moderate: The measurement associated with the KE can plausibly be made for the
species/taxonomic group/lifestage/sex, and there is at least some supporting
experimental evidence, although that may be something other than direct
measurement of the KE;
High: The measurement associated with the KE has been made repeatedly or
frequently and/or with multiple orthogonal methods for the species/taxonomic
group/lifestage/sex.
Taxonomic Applicability
Latin or common names of a species or broader taxonomic grouping (e.g., class, order,
family) can be selected from an ontology. In many cases, individual species identified in
these structured fields will be those for which the strongest evidence used in constructing
the AOP was available in relation to this KE.
Unclassified
-------
32 | ENV/JM/MONO(2016)12
Life Stage Applicability
The structured ontology terms for life-stage are more comprehensive than those for taxa,
but may still require further description/development and explanation in the free text
section.
Sex Applicability
The authors must select from one of the following: Male, female, mixed, asexual, third
gender, hermaphrodite, or unspecific.
Evidence for Biological Domain of Applicability
This free text section should be used to elaborate on the scientific basis for the indicated
domains of applicability and the WoE calls (if provided). While structured terms may be
selected to define the taxonomic, life stage and sex applicability (see structured
applicability terms, above) of the KE, the structured terms may not adequately reflect or
capture the overall biological applicability domain (particularly with regard to taxa).
Likewise, the structured terms do not provide an explanation or rationale for the selection.
The free-text section on evidence for taxonomic, life stage, and sex applicability can be
used to elaborate on why the specific structured terms were selected, and provide
supporting references and background information.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#I Biological Domain of
Applicability for KE
1M IE-Specific Con ten t
An MIE is a specialised KE that represents the beginning (point of interaction between a
stressor and the biological system) of an AOP. Description of an MIE should include all
the information listed above for KEs and also requires two additional fields of
information: evidence that the event can be triggered by a chemical (or other stressor),
and a list of known stressors. If the KE is being described is not an MIE, simply indicate
"not an MIE" in this section
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#J MIE-Specific Content
Unclassified
-------
ENY/JM/MONO(2016)12 | 33
Evidence for Perturbation of MIE by Stressor
The MIE involves a chemical interaction (e.g., a reaction, covalent binding, hydrogen
bonding, electrostatic interaction, etc.) between a chemical stressor and chemically
defined biomolecules within an organism. In some cases, this may be a highly specific
interaction, for example between an exogenous ligand and a specific receptor. In other
cases, it may be non-specific, as in the case of a reactive chemical that can covalently
modify a wide array of proteins. In still other cases, non-chemical stressors of various
types may initiate a biological perturbation through interaction with a defined biological
target (for example actions of a virus in a host cell, physical damage to gill tissue in a
fish, effects of UV radiation on DNA, etc.). Any of these cases can be described as an
MIE, provided that the general nature of the stressor-biomolecule interaction is
understood. Therefore, when a specific MIE can be defined (i.e., the molecular target and
nature of interaction is known), in addition to describing the biological state associated
with the MIE, how it can be measured, and its taxonomic, life stage, and sex applicability,
it is useful to list stressors known to trigger the MIE and provide evidence supporting that
initiation. This will often be a list of prototypical compounds demonstrated to interact
with the target molecule in the manner detailed in the MIE description to initiate a given
pathway (e.g., 2,3,7,8-TCDD as a prototypical AhR agonist; 17a-ethynyl estradiol as a
prototypical ER agonist). Depending on the information available, this could also refer to
chemical categories (i.e., groups of chemicals with defined structural features known to
trigger the MIE). Known stressors should be included in the MIE description, but it is not
expected to include a comprehensive list. Rather initially, stressors identified will be
exemplary and the stressor list will be expanded over time.
Stressors
This is a structured field used to identify specific agents (generally chemicals) that can
trigger the KE. Stressors identified in this field will be linked to the KE in a machine-
readable manner, such that, for example, a stressor search would identify this as an event
the stressor can trigger. NOTE: intermediate or downstream KEs in one AOP may
function as MIEs in other AOPs, meaning that stressor information may be added to the
KE description, even if it is a downstream KE in the pathway currently under
development.
Information concerning the stressors that may trigger an MIE can be defined using a
combination of structured and unstructured (free-text) fields. For example, structured
fields may be used to indicate specific chemicals for which there is evidence of an
interaction relevant to this MIE. By linking the KE description to a structured chemical
name, it will be increasingly possible to link the MIE to other sources of chemical data
and information, enhancing searchability and inter-operability among different data-
sources and knowledgebases. The free-text section "Evidence for perturbation of this
MIE by stressor" can be used both to identify the supporting evidence for specific
stressors triggering the MIE as well as to define broad chemical categories or other
properties that classify the stressors able to trigger the MIE for which specific structured
terms may not exist.
Unclassified
-------
34 | ENV/JM/MONO(2016)12
Specific Content
An AO is a specialised KE that represents the end (an adverse outcome of regulatory
significance) of an AOP. For KEs that are designated as an AO, one additional field of
information (regulatory significance of the AO) should be completed, to the extent
feasible. If the KE is being described is not an AO, simply indicate "not an AO" in this
section.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/3#K AO-Specific Content
Regulatory Significance of the AO
A key criterion for defining an AO is its relevance for regulatory decision-making (i.e., it
corresponds to an accepted protection goal or common apical endpoint in an established
regulatory guideline study). For example, in humans this may constitute increased risk of
disease-related pathology in a particular organ or organ system in an individual or in
either the entire or a specified subset of the population. In wildlife, this will most often be
an outcome of demographic significance that has meaning in terms of estimates of
population sustainability. Given this consideration, in addition to describing the
biological state associated with the AO, how it can be measured, and its taxonomic, life
stage, and sex applicability, it is useful to describe regulatory examples using this AO.
References
List of the literature that was cited for this KE description. Ideally, the list of references,
should conform, to the extent possible, with the OECD Style Guide
(https: //www. oecd.org/about/publishing/OECD -Style -Guide -Third-Edition .pdf) (OECD.
2015).
Unclassified
-------
ENY/JM/MONO(2016)12 | 35
Section 3 riptions
The utility of AOPs for regulatory application is defined, to a large extent, by the
confidence and precision with which they facilitate extrapolation of data measured at low
levels of biological organisation to predicted outcomes at higher levels of organisation
and the extent to which they can link biological effect measurements to their specific
causes. Within the AOP framework, the predictive relationships that facilitate
extrapolation are represented by the KERs. Consequently, the overall WoE for an AOP is
a reflection in part, of the level of confidence in the underlying series of KERs it
encompasses. Therefore, describing the KERs in an AOP involves assembling and
organising the types of information and evidence that defines the scientific basis for
inferring the probable change in, or state of, a downstream KE from the known or
measured state of an upstream KE. Before describing a KER, carefully consider the
following guidance:
KERs are always described in the form of a directed relationship (one-way arrow) linking
an upstream "causing" event to a downstream "responding" event. The pair of KEs linked
via a KER may either be adjacent to one another in the sequence of KEs that define a
given AOP, or non-adjacent. Regardless of adjacency, one event is always positioned
upstream of the other. By convention (and for clarity), KERs linking adjacent KEs in an
AOP are represented using solid arrows, while KERs that link KEs that are not adjacent
to one another in sequence are linked via dashed arrows (e.g., Figure 2). This is a
graphical convention only which has no bearing on the type of content to include in the
KER description.
A KER description has to be created for each adjacent upstream-downstream pair of KEs
in the pathway. Graphically speaking, there should always be at least one solid arrow path
connecting each KE in the pathway into a sequence. There should be no KEs that are
unconnected or are only connected via a non-adjacent path (represented as a dashed
arrow) only.
Inclusion and description of non-adjacent KERs within an AOP can be particularly useful
for assembling evidence supporting the AOP. For example, some KE measurements may
be fairly difficult to make, such that they are rarely made in routine studies. While there
may be sufficient data or plausibility to establish an intermediate KE as part of the AOP,
much of the available WoE may ignore or "leap over" that particular KE. Including KER
descriptions for non-adjacent KE pairs allows the WoE for these relationships to be
readily described and linked to other AOPs without compromising the principle of
modularity with regard to the KER descriptions. With this in mind, the upstream-
downstream pair of KEs linked via a KER may be adjacent in one AOP and non-adjacent
Unclassified
-------
36 | ENV/JM/MONO(2016)12
in another (Figure 3A). In cases where the upstream-downstream sequence was reversed,
a separate KER would be described for each upstream-downstream orientation (Figure
3B).
Non-adjacent KER
MIE
kerl
ke2
>
ker2
Kt^ ]
KERn_i
KER,
AO
Non-adjacent KER
Figure 2. Generic AOP diagram illustrating the graphical convention for depicting KERs
linking adjacent (solid arrow) versus non-adjacent (dashed arrow) upstream-downstream
KE pairs within an AOP. Regardless of adjacency, each KER represents a predictive relationship
between a pair of KEs and can be supported by WoE. Each AOP diagram should portray at least
one direct path sequence through all KEs in the pathway (a solid arrow path from MIE to AO,
connecting all KEs in the pathway). It is not necessary to create a KER for every non-adjacent
pairing, although that can be done if the available supporting evidence warrants.
KER
KE
KERi-3
KE,
KER!,
KE:
KE:
KER
3-1
KE,
Figure 3. Graphical depiction of the modular functionality of KERs connecting KE1 to KE3.
In case (A), the content of KER1-3 is identical despite the fact that the KE1 and KE3 are adjacent in one AOP
and non-adjacent in the other. In case (B), KER1-3 is not equivalent to KER3-1. They would be represented
as separate pages in the AOP-KB supported by different KER descriptions and evidence.
Overall, the subsections of the KER descriptions are intended to aid the user in collecting
relevant information that will support evaluation of the level of confidence in each KER,
which in turn contributes to the assessment of the WoE of the AOP overall (section 4).
Unclassified
-------
ENY/JM/MONO(2016)12 | 37
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/2#A-C Create a New Key Event
Relationship
K
When a KER is created, an ID number is automatically assigned to it. This number is
used for tracking the KER in the AOP-KB.
KER Title
The title of the KER should clearly define the two KEs being considered and the
sequential relationship between them (i.e., which is upstream and which is downstream).
Consequently all KER titles take the form "upstream KE leads to downstream KE".
eferencing Relation ship
All of the AOPs that are linked to this KER will automatically be listed in this subsection.
Biological Domain of Applicability
Developers have the option to select one or more structured terms that help to define the
biological applicability domain of the KER. In general, this will be dictated by the more
restrictive of the two KEs being linked together by the KER. For example, if the upstream
KE is relevant to all vertebrates but the downstream KE is relevant only to sexually
mature, egg-laying female vertebrates, the KER would be relevant to sexually mature
egg-laying female vertebrates. Generally speaking, the biological domain of applicability
of a KER can never be broader than the more restrictive of the two KEs it links together.
Thus, the biological applicability domains of the two KEs being linked is a strong
determinant of the biological domain of applicability of a KER. However, in some cases,
the biological applicability domain of the KER may be even more restrictive. This is
because in addition to structural and functional conservation, the KER also considers the
conservation of a regulatory relationship between two KEs. That is, KEupstream has to
regulate KEdownstream. Therefore, with regard to KERs, the three considerations that
generally guide definition of the biological domain of applicability are:
1. Structure: Are the biological object(s) being measured/observed in the context of the
two KEs being linked present/conserved in the taxa/sex/life-stage of interest?
2. Function: Are the functions of those biological objects and the processes being
measured in the two KEs conserved and relevant in the taxa/sex/life-stage of interest?
Does the object/process play the same role?
Unclassified
-------
38 | ENV/JM/MONO(2016)12
3. Regulation: Is the regulation of the KEdownstream by KEupstream conserved and
relevant in the taxa/sex/life-stage of interest?
Selection of structured terms to describe the biological applicability domain can aid AOP
network construction as well as facilitating other types of computational processing and
searching of information captured in the AOP-Wiki.
Upon selection of structured biological applicability domain terms, developers have the
option to classify the extent of the supporting evidence for the terms they have selected:
Low the relationship is biologically plausible, but hasn't been shown
experimentally in this species/taxonomic group/lifestage/sex;
Moderate the relationship is biologically plausible, and there is some limited
supporting experimental evidence in the species/taxonomic group/lifestage/sex of
interest;
High the relationship is biologically plausible, and there is considerable
supporting evidence in the species/taxonomic group/lifestage/sex, including
evidence of temporal, dose-response, and/or incidence concordance between the
two KEs for the group in question.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/2#D Biological Domain of
Applicability for KER
Taxonomic Applicability
Authors can indicate the relevant taxa for this KER in this subsection. The process is
similar to what is described for KEs (Section 2).
Life Stage Applicability
Authors can indicate the relevant life stage for this KER in this subsection. The process is
similar to what is described for KEs (Section 2).
Sex Applicability
Authors can indicate the relevant sex for this KER in this subsection. The process is
similar to what is described for KEs (Section 2).
Evidence Supporting the Biological Domain of Applicability
As for the KEs, there is also a free-text section of the KER description that the developer
can use to explain his/her rationale for the structured terms selected with regard to
Unclassified
-------
ENY/JM/MONO(2016)12 | 39
taxonomic, life stage, or sex applicability, or provide a more generalizable or nuanced
description of the applicability domain than may be feasible using standardised terms.
R scription
Provide a brief, descriptive summation of the KER. While the title itself is fairly
descriptive, this section can provide details that aren't inherent in the description of the
KEs themselves (see Section 5, recommendations regarding number of KEs to include).
For example, if the upstream KE was antagonism of a specific receptor, the description
could stipulate that "persistent antagonism of the receptor for a period of days" will
trigger the downstream KE. Shorter term antagonism of the same receptor (i.e., same
upstream KE) may trigger a different downstream KE, and thus would be described as a
different KER. This description section can be viewed as providing the increased
specificity in the nature of upstream perturbation (KEupstream) that leads to a particular
downstream perturbation (KEdownstream), while allowing the KE descriptions to remain
generalised so they can be linked to different AOPs. The description is also intended to
provide a concise overview for readers who may want a brief summation, without
needing to read through the detailed support for the relationship (covered below). Careful
attention should be taken to avoid reference to other KEs that are not part of this KER,
other KERs or other AOPs. This will ensure that the KER is modular and can be used by
other AOPs
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/2#E Describe the KER
Evidence Supporting this KER
Assembly and description of the scientific evidence supporting KERs in an AOP is an
important step in the AOP development process that sets the stage for overall assessment
of the AOP (Section 4). To do this, biological plausibility, empirical support, and the
current quantitative understanding of the KER are evaluated with regard to the predictive
relationships/associations between defined pairs of KEs as a basis for considering WoE
(Section 4). In addition, uncertainties and inconsistencies are considered.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/2#F Evidence Supporting this KER
Biological Plausibility
Define, in free text, the biological rationale for a connection between KEupstream and
KEdownstream. What are the structural or functional relationships between the KEs? For
example, there is a functional relationship between an enzyme's activity and the product
of a reaction it catalyses.
Unclassified
-------
40 | ENV/JM/MONO(2016)12
Supporting references should be included. However, it is recognised that there may be
cases where the biological relationship between two KEs is very well established, to the
extent that it is widely accepted and consistently supported by so much literature that it is
unnecessary and impractical to cite the relevant primary literature. Citation of review
articles or other secondary sources, like text books, may be reasonable in such cases. The
primary intent is to provide scientifically credible support for the structural and/or
functional relationship between the pair of KEs if one is known.
In general, the structural and/or functional relationship supporting plausibility is based on
understanding of normal biological function, rather than response to a specific stressor.
The description of biological plausibility can also incorporate additional mechanistic
detail that helps inform the relationship between KEs, but is not practical/pragmatic to
represent as separate KEs due to the difficulty or relative infrequency with which it is
likely to be measured. For example, in the case of G protein coupled receptor activation
(KEupstream) leading to increased activity of a specific enzyme (KEdownstream), there
may be numerous mechanistic steps in between those KEs (e.g., alterations in signal
transduction pathways, transcriptional regulation, post-translational modifications, etc.).
These underlying details, if known, can be captured in the description of biological
plausibility (if desired) rather than represented as independent KEs. The KER
descriptions are an appropriate place for "compounding" or "embedding" that type of
biological detail without compromising the reusability of KE descriptions within the
AOP-Wiki. However, it should be kept in mind that added detail should only be included
to the extent that it enhances the predictive utility of the AOP. In part, the AOP is
intended to filter through much of the "biological noise" to focus on what is causally
related to the adversity. Thus, efforts should be made to keep the descriptions focused.
Empirical Evidence
In this section authors are encouraged to cite specific evidence that supports the idea that
a change in the upstream KE (KEupstream) will lead to, or is associated with, a
subsequent change in the downstream KE (KEdownstream), assuming the perturbation of
KEupstream is sufficient.
In particular, it is useful to cite evidence showing that stressors that perturb KEupstream
also perturb KEdownstream. Because this section of the KER description cites evidence
from specific studies, it is also helpful to provide as much detail about the toxicological
and biological context in which the measurements were made, as is feasible, including the
stressor(s) tested, the effective doses at each KE, etc. While the KER itself is not intended
to be stressor-specific, those details can aid the overall assessment of the individual AOPs
that include that KER. These details also help inform the question of consistency of
supporting data, consistency across different biological contexts for which the KER is
relevant, and the applicability domain of the KER. However, authors are cautioned that
this evidence should focus on data that only relate KEupstream to KEdownstream, and
should avoid reference to other KEs, KERs and AOPs as much as possible in order to
maintain modularity of the KER.
Unclassified
-------
ENY/JM/MONO(2016)12 | 41
Given the likelihood that new empirical support will be developed over time, particularly
as various AOPs are tested and applied, it is most practical to provide empirical support
in the form of bulleted lists or tables that include a short description of the nature of the
empirical support along with the corresponding reference(s).
Dose Concordance
In the case of dose-response concordance, the aim is not to show dose-dependence of a
single KE in the pair, but rather to establish that KEupstream is generally impacted at
doses (or stressor severities) equal to or lower than those at which KEdownstream is
impacted.
Temporal Concordance
In the case of temporal concordance, it is desirable to assemble evidence showing that
effects on KE upstream are observed earlier in a time-course than effects on the
downstream KE.
Incidence Concordance
In the case of incidence concordance, evidence should be assembled that addresses
whether, at an equivalent dose or stressor severity, KEupstream occurs more frequently
than KEdownstream .
Other Evidence (optional)
Although evidence that demonstrates dose, temporal or incidence concordance are
preferred, other evidence that empirically supports the relations that a sufficient change in
KEupstream will lead to a change in KEdownstream, but do not fall into the above three
categories, can be cited in this subsection.
Uncertainties and Inconsistencies
In addition to outlining the evidence supporting a particular linkage, it is also important to
identify inconsistencies or uncertainties in the relationship. This could include, for
example, empirical evidence showing changes in KEupstream that did not elicit
alterations in KEdownstream. It could also include descriptions of gaps in biological
understanding that lend to uncertainties in understanding of the exact nature of the
structural or functional relationship between the two KEs. Additionally, while there are
expected patterns of concordance that support a causal linkage between the KEs in the
pair, it is also helpful to identify experimental details that may explain apparent
deviations from the expected patterns of concordance. An example of this would be a
case where methods for measuring the upstream KE are relatively insensitive compared
Unclassified
-------
42 | ENV/JM/MONO(2016)12
to those for measuring the downstream KE, leading to the appearance of dose-response or
incidence discordance that is simply an artefact of the measurement techniques employed.
In this regard, when assembling information from multiple disparate studies, it is
important to capture variables that directly influence how well concordance can be
assessed (i.e., information regarding the doses tested in various experiments and the time-
points at which various KE measurements were made). Identification of uncertainties and
inconsistencies contributes to evaluation of the overall WoE supporting the AOPs that
contain a given KER (see Section 4) and to the identification of research gaps that
warrant investigation.
Given that AOPs are intended to support regulatory applications, AOP developers should
focus on those inconsistencies or gaps that would have a direct bearing or impact on the
confidence in the KER and its use as a basis for inference or extrapolation in a regulatory
setting. Uncertainties that may be of academic interest but would have little impact on
regulatory application don't need to be described. In general, this section details evidence
that may raise questions regarding the overall validity and predictive utility of the KER
(including consideration of both biological plausibility and empirical support). It also
contributes along with several other elements to the overall evaluation of the WoE for the
KER (see, Section 4).
Quantitative Understanding
The quantitative understanding section of the KER description is intended to capture
information that helps to define how much change in the upstream KE, and/or for how
long, is needed to elicit a detectable and defined change in the downstream KE. While
empirical support (F) addresses whether data between the two KEs are consistent with the
patterns that are expected if the upstream event is causing the downstream event to occur,
the quantitative understanding section helps to define the precision with which the state of
the downstream KE can be predicted from knowledge of the state of the upstream KE.
These quantitative relationships may be defined in terms of correlations, response-
response relationships, dose-dependent transitions or points of departure (i.e., a threshold
of change in KEupstream needed to elicit a change in KEdownstream), etc. They may
take the form of simple mathematical equations or sophisticated biologically-based
computational models that consider other modulating factors such as compensatory
responses, or interactions with other biological or environmental variables. Regardless of
form, the idea is to briefly describe what is known regarding the quantitative relationship
between the KEs and cite appropriate literature that defines those relationships and/or
provides support for them.
Data that confers quantitative understanding of a KER are not necessarily mutually
exclusive from those addressing other weight of evidence considersations. In that respect,
the quantitative understanding section of the KER description is not intended to be
redundant with the other WoE sections. Rather, it is intended to aid application of the
AOP by allowing a reader to rapidly identify the relationships that would support
quantitative prediction of the probability or magnitude of change in KEdownstream based
on a known state of KEupstream. For transparency, the toxicological and biological
Unclassified
-------
ENY/JM/MONO(2016)12 | 43
context in which the quantitative relationships were defined should be indicated within
the description. However, the ultimate goal is to identify quantitative relationships that
generalise across the entire applicability domain of the two KEs being linked via the
KER.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/2#G Quantitative Understanding of
the KER
Based on recommendations from workshops held in September 2015 (Wittwehr et al.
2016) and April 2017 (LaLone et al. 2017), description of the quantitative understanding
of the KER has been organised into subsections in order to more consistently capture
information that would be informative for both quantitative AOP and AOP network
applications. As with other areas of the AOP descriptions, authors are encouraged to
complete the sections to the extent that is feasible, but it is recognised that supporting
information may not be adequate to address all sections.
Response-response relationship
This subsection should be used to define sources of data that define the response-response
relationships between the KEs. In particular, information regarding the general form of
the relationship (e.g., linear, exponential, sigmoidal, threshold, etc.) should be captured if
possible. If there are specific mathematical functions or computational models relevant to
the KER in question that have been defined, those should also be cited and/or described
where possible, along with information concerning the approximate range of certainty
with which the state of the KEdownstream can be predicted based on the measured state
of the KEupstream (i.e., can it be predicted within a factor of two, or within three orders
of magnitude?). For example, a regression equation may reasonably describe the
response-response relationship between the two KERs, but that relationship may have
only been validated/tested in a single species under steady state exposure conditions.
Those types of details would be useful to capture.
Time-scale
This sub-section should be used to provide information regarding the approximate time-
scale of the changes in KEdownstream relative to changes in KEupstream (i.e., do effects
on KEdownstream lag those on KEupstream by seconds, minutes, hours, or days?). This
can be useful information both in terms of modelling the KER, as well as for analysing
the critical or dominant paths through an AOP network (e.g., identification of an AO that
could kill an organism in a matter of hours will generally be of higher priority than other
potential AOs that take weeks or months to develop). Identification of time-scale can also
aid the assessment of temporal concordance. For example, for a KER that operates on a
time-scale of days, measurement of both KEs after just hours of exposure in a short-term
experiment could lead to incorrect conclusions regarding dose-response or temporal
Unclassified
-------
44 | ENV/JM/MONO(2016)12
concordance if the time-scale of the upstream to downstream transition was not
considered.
Known modulating factors
O Jr
This sub-section presents information regarding modulating factors/variables known to
alter the shape of the response-response function that describes the quantitative
relationship between the two KEs (for example, an iodine deficient diet causes a
significant increase in the slope of the relationship; a particular genotype doubles the
sensitivity of KEdownstream to changes in KEupstream). Information on these known
modulating factors should be listed in this subsection, along with relevant information
regarding the manner in which the modulating factor can be expected to alter the
relationship (if known). Note, this section should focus on those modulating factors for
which solid evidence supported by relevant data and literature is available. It should NOT
list all possible/plausible modulating factors. In this regard, it is useful to bear in mind
that many risk assessments conducted through conventional apical guideline testing-based
approaches generally consider few if any modulating factors.
Known Feedback loops influencing this KER
X o
This subsection should define whether there are known positive or negative feedback
mechanisms involved and what is understood about their time-course and homeostatic
limits? In some cases where feedback processes are measurable and causally linked to the
outcome, they should be represented as KEs (see development tip 4). However, in most
cases these features are expected to predominantly influence the shape of the response-
response, time-course, behaviours between selected KEs. For example, if a feedback loop
acts as compensatory mechanism that aims to restore homeostasis following initial
perturbation of a KE, the feedback loop will directly shape the response-response
relationship between the KERs. Given interest in formally identifying these positive or
negative feedback, it is recommended that a graphical annotation indicating a positive or
negative feedback loop (Figure 4) is involved in a particular upstream to downstream KE
transition (KER) be added to the graphical representation, and that details be provided in
this subsection of the KER description.
1 -2
i "*
Figure 4. Recommended graphical annotation to indicate that a known (A) positive
feedback (i.e., feedforward) or (B) negative feedback loop is involved in the transition
from one KE to the next in the AO P. Note: This is an optional annotation. See Section 6D for
Unclassified
-------
ENY/JM/MONO(2016)12 | 45
more information on describing positive and negative feedback processes using the AOP
framework.
Development tip 5 - Capturing information on positive or negative feedback loops.
Ways to capture/represent known positive or negative feedback loops have
emerged as a frequently asked question in relation to use of the AOP framework.
Thus, a few general guidelines are provided here.
In cases where feedback loops play a direct causal role in the progression
of a biological perturbation leading to an AO, they can be included as KEs
as long as they are measurable. For example, for an AOP in which a
negative feedback process results in decreased hormone signalling that
leads to the AO, a measurable event indicative of or involved in the
activation of the negative feedback could be included as a KE.
In cases where a feedback loop may act as a key compensatory or adaptive
mechanism that dictates how severely the KEupstream needs to be
impacted in order of affect the KEdownstream, but does not play a direct
causal role in the AOP (other than defining the relevant point of
departure), the feedback should not be included as a separate KE. Rather it
should be detailed as part of the quantitative understanding section of the
KER description. In the user supplied graphical representation, a forward
or backward looping symbol could be added above the arrow linking the
two KEs to indicate that a known positive or negative feedback loop is
involved in the transition (Figure 4B).
In cases where two measurable KEs in an AOP are part of a positive
feedback loop, it can be challenging to define which should be upstream
and which downstream, as they are amplifying or altering one another in a
cycle. A two headed arrow is undesirable as it can incorrectly suggest that
the AOP is reversible. However, in practice an AOP with a positive
feedback loop could be accurately represented as two different AOPs in
the AOP-Wiki, in which the KEs involved in the positive feedback are
presented in either order. This effectively creates a bi-directional arrow
when the AOP network is assembled. Rather than creating two nearly
identical AOP pages with the KE order reversed for each, the current
recommendation is to select either order for the KEs and connect them
with a unidirectional arrow, but add a forward looping symbol above the
arrow in the user-supplied graphical representation to indicate that a
known feedforward loop is involved (Figure 4A).
Classification of quantitative understanding
To aid in overall assessment of the AOP and whether it is fit-for-purpose for various
applications, developers are also asked to classify the extent of quantitative understanding
of the KER as low, moderate, or high. General guidance for classification of the level of
Unclassified
-------
46 | ENV/JM/MONO(2016)12
quantitative understanding of a KER as low, moderate, or high (Annex 2) is based on
several key considerations:
The extent to which a change in KEdownstream can be precisely predicted based
on KEupstream.
The precision with which uncertainty in the prediction of KEdownstream can be
quantified.
The extent to which known modulating factors or feedback mechanisms are
accounted for.
The extent to which the relationships described can be reliably generalised across
the biological applicability domain of the KER.
References
List of the literature that was cited for this KER description using the appropriate format.
Unclassified
-------
ENY/JM/MONO(2016)12 | 47
A.
B.
C.
Figure 5. Illustration of general guidance regarding inclusion of simple AOP networks or
branched AOP structures (A) on a single AOP page. Brandling representing independent
actions leading to more than AO should not be included in an AOP description (B). Brandling
indicating multiple KEs (including MIEs) that MUST occur for the pathway to progress
downstream should be included in an AOP description.
Development tip 6 - Branching of AOPs captured on a single AOP page
In general, individual AOPs are defined as a single, non-branching sequence of
KEs, linked by KERs that connect a single MIE to an AO (Villeneuve et al.
2014a). In most cases, this is viewed as the most pragmatic unit for development
and evaluation of AOP descriptions. Consequently, most AOPs pages should
define a single, non-branching, sequence of KEs linked by KERs. However, it is
recognised that in some cases there may be exceptions for which representation of
a simple AOP network on an AOP page is a more pragmatic unit of development
and evaluation (see Leist et al. 2017 for examples and further explanation). Under
certain circumstances, representation of a branched structure on an AOP page is
Example of acceptable branching on an AOP
page. Represents additive/joint ("and"
relationships) actions of a single MIE that
contribute to a single AO.
MIE
KE
KE
Note, this would still be considered an AOP
network, not a single AOP, but describing on
a single page may be pragmatic.
KE
H
ACCEPTABLE
1 -
ซ
AO
,
GEH
1 ^
MIE2
KE
KE
KE
KE
Examples of branching that should not be
included on a single AOP page. In this case,
the branching represents independent
action ("or" relationships).
NOT
ENCOURAGED
KE
KE
AOl
KE
KE
A02
MIE1
Example of acceptable branching on an AOP page.
Represents case where more than one MIE must occur
I simultaneously for the downstream KEs to occur.
MIE2
-i simi
J I
&
H
& i
KE
KE
KE
KE
Because both MIEs are essential to progression along
the pathway, this would be considered a single AOP,
despite the branched structure.
ACCEPTABLE
KE
AOl
Unclassified
-------
48 | ENV/JM/MONO(2016)12
acceptable, so long as the principles of modularity of the KEs and KERs and
overall coherence to the framework is maintained.
For example, representation of branching on an AOP may become pragmatic
when there are multiple KEs, causally linked to the MIE and AO that are
occurring concurrently and likely acting in concert to drive the downstream
effects. In such cases, the various KEs cannot necessarily be placed neatly into a
single temporal sequence because they are effectively occurring simultaneously.
Likewise it cannot necessarily be determined which of the concurrent KEs is most
essential or critical, because there are multiple KEs (measurable biological
changes) contributing jointly in an additive manner such that it cannot be
effectively determined whether one could cause the pathway to progress without
the other. This is contrasted with cases where KEs act independently such that one
event or the other, alone, would allow progression toward the outcome.
In cases where an additive (and) relationship must be assumed, representation of a
simple AOP network on a single AOP page within the AOP-KB may be more
practical from both a development and use stand-point than breaking those
multiple highly related pathways into separate AOP descriptions. As long as KEs
and associated KERs are each represented as separate modular pages in the AOP-
KB (as described below), capturing such networks on single AOP pages does not
create problems for modular AOP network building. Indeed, it can actually
strengthen the overall AOP by capturing the evidence for pleiotropic effects of the
same MIE that ultimately contribute to the same outcome.
Note, such branched AOP structures should only be included on a single AOP
page when all the branches diverge from a common MIE (or MIEs in the case that
two or more MIEs MUST occur to drive the pathway) and converge to a common
AO (Figure 5A) and two or more of the KEs contributing causally to the AO
occur concurrently such that it is experimentally intractable to isolate and identify
which is playing the dominant causal role (i.e., in all likelihood both KEs are
contributing) and both (all KEs) measurements are deemed to have predictive
value.
Branched structures should not be included on a single AOP page when they
diverge to independent outcomes (e.g., Figure 5B) and/or are operating largely
independent of one another and can be resolved from one another in space or
time, experimentally. Following this logic, two or more MIEs may occur on an
AOP page, when more than one event MUST happen simultaneously in order for
the pathway to be triggered (Figure 5C).
Unclassified
-------
ENY/JM/MONO(2016)12 | 49
Section 4 - Overall Assessineir
This section addresses the relevant biological domain of applicability (i.e., in terms of
taxa, sex, life stage, etc.) and WoE for the overall AOP as a basis to consider appropriate
regulatory application (e.g., priority setting, testing strategies or risk assessment). The
goal of the overall assessment is not to reproduce or reiterate all the content assembled as
part of sections 1-3, but rather to provide a high level synthesis and overview of the
relative confidence in the AOP and where the significant gaps or weaknesses are (if they
exist). Users or readers can drill down into the finer details captured in the KE and KER
descriptions, and/or associated summary tables, as appropriate to their needs.
Determination of confidence in the overall AOP is based on the biological plausibility,
empirical support, and extent of quantitative understanding for the KERs (Section 3) and
the evidence supporting essentiality of the KEs.
Assessment of the AOP is organised into a number of steps. Guiding questions that
inform evaluation at each step are included in Annexes 1 and 2. The questions are
designed to facilitate assignment of categories of high, moderate, or low confidence for
each consideration. While it is not necessary to repeat lengthy text that appears elsewhere
in the AOP description (or related KE and KER descriptions), a brief explanation or
rationale for the selection of high, moderate, or low confidence should be made, in light
of the guiding questions detailed below.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#J Overall Assessment of the AOP
Define the Biological Domain of Applicability of the
The relevant biological domain(s) of applicability in terms of sex, life-stage, taxa, and
other aspects of biological context are defined in this section. Biological domain of
applicability is informed by the "Description" and "Biological Domain of Applicability"
sections of each KE and KER description (see sections 2G and 3E for details). In essence
the taxa/life-stage/sex applicability is defined based on the groups of organisms for which
the measurements represented by the KEs can feasibly be measured and the functional
and regulatory relationships represented by the KERs are operative.
The relevant biological domain of applicability of the AOP as a whole will nearly always
be defined based on the most narrowly restricted of its KEs and KERs. For example, if
Unclassified
-------
50 | ENV/JM/MONO(2016)12
most of the KEs apply to either sex, but one is relevant to females only, the biological
domain of applicability of the AOP as a whole would be limited to females. While much
of the detail defining the domain of applicability may be found in the individual KE and
KER descriptions, the rationale for defining the relevant biological domain of
applicability of the overall AOP should be briefly summarised on the AOP page.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#6iological Domain of
Applicability
Assess the Essentiality of All KEs
An important aspect of assessing an AOP is evaluating the essentiality of its KEs. The
essentiality of KEs can only be assessed relative to the impact of manipulation of a given
KE (e.g., experimentally blocking or exacerbating the event) on the downstream sequence
of KEs defined for the AOP. Consequently evidence supporting essentiality is assembled
on the AOP page, rather than on the independent KE pages that are meant to stand-alone
as modular units without reference to other KEs in the sequence.
The nature of experimental evidence that is relevant to assessing essentiality relates to the
impact on downstream KEs and the AO if upstream KEs are prevented or modified. This
includes:
Direct evidence: directly measured experimental support that blocking or
preventing a KE prevents or impacts downstream KEs in the pathway in the
expected fashion.
Indirect evidence: evidence that modulation or attenuation in the magnitude of
impact on a specific KE (increased effect or decreased effect) is associated with
corresponding changes (increases or decreases) in the magnitude or frequency of
one or more downstream KEs.
When assembling the support for essentiality of the KEs, authors should organise relevant
data in a tabular format (e.g., Table 5). The objective is to summarise briefly the nature
and numbers of investigations in which the essentiality of KEs has been experimentally
explored either directly or indirectly. In some cases, the impact of blocking or modifying
an early KE on all downstream KEs in the pathway has been determined; in other cases,
the impact only on a single adjacent or non-adjacent downstream KE has been measured.
When assembling support for essentiality of the KEs, it is not necessary to repeat lengthy
text on the design or results of relevant investigations that may appear in other parts of
the AOP description (e.g., as WoE for a KER). Rather, the entries should briefly address
the extent of the supporting and contradictory data through a short description of the
nature of the direct or indirect evidence addressing essentiality, along with relevant
references. The objective is to provide an overview of the extent and nature of supporting
Unclassified
-------
ENY/JM/MONO(2016)12 | 51
and inconsistent data on essentiality of the KEs in a format that will facilitate a "call" on
the overall degree of support for essentiality across the AOP (Section 8). Some examples
of brief narratives addressing support for essentiality are included here. See
https://aopwiki.org/info pages/2/info linked pages/6 for additional examples:
For direct evidence:
Knock-out of KE1 or early KEs leads to blockage of all downstream KEs
One or more downstream KEs is blocked or reversed by inhibiting (or allowing
recovery of) upstream KEs
Overexpression in repair enzyme for early KEs leads to decreased incidence of
downstream KEs
Antagonism or agonism of upstream KE leads to expected pattern of effects on
downstream KEs
For indirect evidence:
Impact on a modulating factor for early KEs leads to expected pattern of effects
on later KEs
Table 5. Example of a Table Format for Assembling the Data on Essentiality of KEs:
MIE
KE1
KE2
KE3
KEn
Where there is no experimental model to prevent or augment a specific KE in the context
of the overall pathway, this should be indicated as "No data".
Uncertainties or Inconsistencies:
In addition to outlining the evidence supporting essentiality, it is also important to
identify inconsistencies or uncertainties, as presented in one of the columns in Table 5.
This could include, for example, evidence in specific studies that did not support that
blockage or attenuation of an early KE impacted later KEs in the AOP. Discordance with
the results of other studies should be considered based on evaluation of the adequacy of
study design, taking into account, for example, the sensitivity of the detection of impact.
It could also include, for example, gaps in knowledge concerning the essentiality of the
MIE or particular KEs where there are data on essentiality only for one or a few. To the
Event
Direct Indirect No experimental
Evidence Evidence evidence
Contradictory
experimental
evidence
Unclassified
-------
52 | ENV/JM/MONO(2016)12
extent possible, inconsistencies and uncertainties should focus on data gaps important for
potential envisaged regulatory applications as a basis for indicating priorities for further
research.
Based on the assembled evidence on essentiality for the KEs, confidence in the
supporting data on essentiality is considered for the entire AOP, including KERs and
KEs. This is commonly based on the extent of direct and/or indirect evidence for one,
several or all of the KEs.
Confidence in the supporting data for essentiality of KEs within the AOP is considered:
High if there is direct evidence from specifically designed experimental studies
illustrating prevention or corresponding impact on downstream KEs and/or the
AO if upstream KEs are blocked or modified (e.g., via stop exposure/reversibility
studies, antagonism, knock out models, etc.);
Moderate if there is indirect evidence that modification of one or more upstream
KEs is associated with a corresponding (increase or decrease) in the magnitude or
frequency of downstream KEs [e.g., augmentation of proliferative response
(KEupstream) leading to increase in tumour formation (KEdownstream or AO)];
Low if there is no or contradictory experimental evidence that blocking or
modulating/attenuating any of the KEs influences the KEs downstream or AO
(Annex 1).
These considerations, as well as those related to biological plausibility and empirical
support draw upon experience in application of WoE analysis in mode of action analysis
in chemical specific regulatory application, an important envisaged application of AOPs.
As such, they reflect broad collective and evolving experience in regulatory application of
mechanistic data, tailored to maintain balance between relevant aspects of application
with envisaged modular development of AOPs. For essentiality, considerations also
reflect the nature of experimental data that optimally informs this critical component for
regulatory application. Supporting experimental investigations which address the impact
of early key events on all subsequent KEs in hypothesized AOPs obviate the need for
studies on the essentiality of individual KEs.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/l#Essentialitv of the Key Events
Assess the Evidence Supporting All KERs
The biological plausibility, empirical support, and quantitative understanding from each
KER in an AOP are assessed together:
Unclassified
-------
ENY/JM/MONO(2016)12 | 53
Review the Biological Plausibility of Each KER
Biological plausibility of each of the KERs in the AOP is the most influential
consideration in assessing WoE or degree of confidence in an overall hypothesised AOP
for potential regulatory application (Meek et al., 2014; 2014a). The defining question for
biological plausibility (Annex 1) is: Is there a mechanistic (i.e., structural or functional)
relationship between KEupstream and KEdownstream consistent with established
biological knowledge? WoE for the biological plausibility of the KERs would be
considered:
High if it is well understood based on extensive previous documentation and has
an established mechanistic basis and broad acceptance (e.g., increased follicle
stimulating hormone signalling leading to increased estrogen synthesis, increased
incidence of alkylated DNA leading to increased incidence of mutations);
Moderate if the KER is plausible based on analogy to accepted biological
relationships but scientific understanding is not completely established;
Low if there is empirical support for a statistical association between KEs but
structural or functional relationship between them is not understood.
Review the Empirical Support for Each KER
Empirical support entails consideration of experimental data in terms of the associations
between KEs - namely dose-response concordance and temporal relationships between
and across multiple KEs. It is examined most often in studies of dose-response/incidence
and temporal relationships for stressors that impact the pathway. While less influential
than biological plausibility of the KERs and essentiality of the KEs (Meek et al., 2014;
2014a), empirical support can increase confidence in the relationships included in an
AOP.
It is important to recognise that empirical support relates to the "concordance" of dose
response, temporal and incidence relationships for KERs; the defining question is not
whether or not there is a dose response relationship for a specific KE but rather, whether
there is expected concordance with the dose-response relationships for KERs - i.e.,
between KEs.
The defining questions for empirical support (Annex 1) are: Does KEupstream occur at
lower doses and earlier time points than KEdownstream; is the incidence or frequency of
KEupstream greater than that for KEdownstream for the same dose of tested stressor?
Inconsistencies in empirical support across taxa, species and stressors that don't align
with the expected pattern for the hypothesised AOP as described in Section 3 should be
identified and their basis considered.
Empirical support for each of the KERs would be considered:
High if there is dependent change in both events following exposure to a wide
range of specific stressors (extensive evidence for temporal, dose-response and
incidence concordance) and no or few data gaps or conflicting data;
Unclassified
-------
54 | ENV/JM/MONO(2016)12
Moderate if there is demonstrated dependent change in both events following
exposure to a small number of specific stressors and some evidence inconsistent
with the expected pattern that can be explained by factors such as experimental
design, technical considerations, differences among laboratories, etc.;
Low if there are limited or no studies reporting dependent change in both events
following exposure to a specific stressor (i.e., endpoints never measured in the
same study or not at all), and/or lacking evidence of temporal or dose-response
concordance, or identification of significant inconsistencies in empirical support
across taxa and species that don't align with the expected pattern for the
hypothesised AOP.
Tables summarising the relevant experimental data for tested stressors may be helpful in
considering the extent of empirical support. For example, benchmark doses (BMDs) for
specified similar increases in of each of the KEs are entered in the cells of the table. If
the hypothesised linkages in the AOP are supported by empirical data, there is a pattern
of increasing BMDs from the top left hand corner to the bottom right hand corner for
each of the tested stressors. Presentation in this manner readily identifies any exceptions
to the expected patterns that are considered as inconsistencies and diminish from the
overall weight of empirical support (see Tables 6).
Table 6. Generic example of a concordance table for evaluating empirical support of the
KERs.
Type the subtitle here. If you do not need a subtitle, please delete this line.
Species tested Stressor Time pt KEla KE2 KE3 KE4 KE5
FHM A 6 h 1 No effect No effect No data No effect
FHM A 24 h 1 10 No effect No data No effect
FHM A 4 d No effect 1 10 20 50
FHM A 8 d No effect 1 10 10 20
FHM A 21 d No effect 1 10 10 10
FHM B 24 h 25 25 50 No data No data
FHM B 10 d 10 10 25 25 25
RBT A 12 h 0.2 0.2 10 10 No effect
RBT A 24 h 0.2 0.2 1 10 10
RBT A 8 d 0.1 0.2 0.5 0.5 0.5
RBT A 21 d 0.1 0.1 0.2 0.5 0.5
a: Benchmark dose at which a specified level of change in the KE relative to controls M'as inferred, based on
the empirical results. (Note, where concentrations tested are inadequate to determine a BKID, LOEC or
NOEC could also be considered, but concentrations tested in different studies must be taken into account).
Additional examples of concordance tables:
https://aopwiki.Org/wiki/images/4/45/Aromatase inhibition dose-
response concordance table revl.pdf
Unclassified
-------
ENY/JM/MONO(2016)12 | 55
https://aopwiki.org/svstem/dragonflv/production/2017/05/19/1 qoq 9ky7zb AOP15
supporting evidence.pdf
https://a0pwiki.0rg/wiki/index.php/File:Mechanistic data niatrix.ipg
https://aopwiki.org/svstem/dragonflv/production/2017/03/20/lk2chhiib AR agoni
sm concordance table updated 2017 03 14.pdf
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/1 #Evidence Assessment
Quantitative WoE considerations (optional)
Some proof of concept examples to address the WoE considerations for AOPs
quantitatively have recently been developed, based on the rank ordering of the relevant
Bradford Hill considerations (i.e., biological plausibility, essentiality and empirical
support) (Becker et al., 2017; Becker et al, 2015; Collier et al., 2016). Suggested
quantitation of the various elements is expert derived, without collective consideration
currently of appropriate reporting templates or formal expert engagement. Though not
essential, developers may wish to assign comparative quantitative values to the extent of
the supporting data based on the three critical Bradford Hill considerations for AOPs, as a
basis to contribute to collective experience.
Review the Quantitati lerstanding for Each KER
The extent of quantitative understanding of the KERs in an AOP is also critical in
consideration with regard to potential regulatory application. For some applications (e.g.,
dose- response analysis in in-depth risk assessment), quantitative characterisation of
downstream KERs may be essential, while for others quantitative understanding of
upstream KERs may be most important (e.g., QSAR modelling for category formation for
testing). Because evidence that contributes to quantitative understanding of the KER is
generally not mutually exclusive with the empirical support for the KER (i.e., expected
patterns of quantitative relationships), evidence that contributes to quantitative
understanding will generally be considered to some extent as part of the evaluation of the
WoE supporting the KER (see Section 3.E. and Annex 1, footnote b). However, specific
attention is also given to how precisely and accurately one can potentially predict an
impact on KEdownstream based on some measurement of KEupstream. This is captured
in the form of quantitative understanding calls for each KER. As noted in section 3,
general guidance for characterising the level of quantitative understanding of a KER as
low, moderate, or high (Annex 2) is based on several key considerations:
The extent to which a change in KEdownstream can be precisely predicted based
on KEupstream.
The precision with which uncertainty in the prediction of KEdownstream can be
quantified.
The extent to which known modulating factors or feedback mechanisms are
accounted for.
Unclassified
-------
56 | ENV/JM/MONO(2016)12
The extent to which the relationships described can be reliably generalized across
the applicability domain of the KER.
As with the other parts of the overall assessment of the AOP, it is not necessary to repeat
all the details provided in the KER descriptions. The overall evaluation of the quantitative
understanding should briefly explain the rationale for the assigned level of quantitative
understanding of each KER.
Implementation in the AOP-Wiki:
https://aopwiki.org/info pages/2/info linked pages/1 Quantitative Considerations
Unclassified
-------
ENY/JM/MONO(2016)12 | 57
References
Becker RA, Ankley GT, Edwards SW, Kennedy SW, Linkov I, Meek B, Sachana M,
Segner H, Van Der Burg B, Villeneuve DL, Watanabe H, Barton-Maclaren TS. (2015).
Increasing scientific confidence in adverse outcome pathways: application of tailored
Bradford-Hill considerations for evaluating weight of evidence. Regul Toxicol Pharmacol
72: 514-537.
Becker RA, Dellarco V, Seed J, Kronenberg JM, Meek B, Foreman J, Palermo C, Kirman
C, Linkov I, Schoeny R, Dourson M. (2017). Quantitative weight of evidence to assess
confidence in potential modes of action. Regul Toxicol Pharmacol 86: 205-220.
Collier ZA, Gust KA, Gonzalez-Morales B, Gong P, Wilbanks MS, Linkov I, Perkins EJ.
(2016). A weight of evidence assessment approach for adverse outcome pathways. Regul
Toxicol Pharmacol 75: 46-57.
Knapen, D., Vergauwen, L., Villeneuve, D.L. and Ankley GT. (2015) The potential of
AOP networks for reproductive and developmental toxicity assay development. Reprod
Toxicol. 56: 52-55.
Krewski D, Acosta D Jr., Andersen M, Anderson H, Bailar J.C. 3rd, Boekelheide K,
Brent R, Charnley G, Cheung VG, Green S Jr, Kelsey KT, Kerkvliet NI, Li AA, McCray
L, Meyer O, Patterson RD, Pennie W, Scala RA, Solomon GM, Stephens M, Yager J,
Zeise L. (2010). Toxicity testing in the 21st century: a vision and strategy. J Toxicol
Environ Health B Crit Rev. 13: 51-138.
LaLone CA, Ankley GT, Belanger SE, Embry MR, Hodges G, Knapen D, Munn S,
Perkins EJ, Rudd MA, Villeneuve DL, Whelan M, Willett C, Zhang X, Hecker M.
(2017.) Advancing the adverse outcome pathway framework - an international horizon
scanning approach. Environ Toxicol Chem. (in press).
Leist M, Ghallab A, Graepel R, Marchan R, Hassan R, Bennekou SH, Limonciel A,
Vinken M, Schildknecht S, Waldmann T, Danen E, van Ravenzwaay B, Kamp H,
Gardner I, Godoy P, Bois FY, Braeuning A, Reif R, Oesch F, Drasdo D, Hohme S,
Schwarz M, Hartung T, Braunbeck T, Beltman J, Vrieling H, Sanz F, Forsby A, Gadaleta
D, Fisher C, Kelm J, Fluri D, Ecker G, Zdrazil B, Terron A, Jennings P, van der Burg B,
Dooley S, Meijer AH, Willighagen E, Martens M, Evelo C, Mombelli E, Taboureau O,
Unclassified
-------
58 | ENV/JM/MONO(2016)12
Mantovani A, Hardy B, Koch B, Escher S, van Thriel C, Cadenas C, Kroese D, van de
Water B, Hengstler JG. (2017) Adverse outcome pathways: opportunities, limitations,
and open questions. Regulat. Toxicol. DOI: 10.1007/s00204-017-2045-3
Meek ME, Klaunig JE. (2010). Proposed mode of action of benzene induced leukemia:
interpreting available data and identifying critical data gaps for risk assessment. Chem.
Biol. Interact. 184: 279-285.
Meek ME, Boobis AR, Cote I, Dellarco V, Fotakis G, Munn S, Seed J, Vickers C.
(2014a). New developments in the evolution and application of the WHO/IPCS
framework on mode of action/species concordance analysis. J Appl Toxicol. 34: 1-18.
Meek ME, Palermo CM, Bachman AN, North, CM, Lewis RJ. (2014b). Mode of Action
Human Relevance (MOA/HR) Framework - Evolution of the Bradford Hill
Considerations and Comparative Analysis of Weight of Evidence. J Appl Toxicol. 34:
595-606.
OECD (2015), OECD Style Guide third edition, OECD Publishing, Paris.
https ://www. oecd. org/about/publishing/OECD -Style -Guide -Third-Edition .pdf
Villeneuve DL, Crump D, Garcia-Reyero N, Hecker M, Hutchinson TH, LaLone CA,
Landesmann B, Lettieri T, Munn S, Nepelska M, Ottinger MA, Vergauwen L, Whelan M.
(2014a) Adverse outcome pathway (AOP) development I: strategies and principles.
Toxicol Sci. 142: 312-320.
Villeneuve DL, Crump D, Garcia-Reyero N, Hecker M, Hutchinson TH, LaLone CA,
Landesmann B, Lettieri T, Munn S, Nepelska M, Ottinger MA, Vergauwen L, Whelan M.
(2014b) Adverse outcome pathway development II: best practices. Toxicol Sci. 142: 321 -
330.
Wittwehr C, Aladjov H, Ankley G, Byrne HJ, de Knecht J, Heinzle E, Klambauer G,
Landesmann B, Luijten M, MacKay C, Maxwell G, Meek ME, Paini A, Perkins E,
Sobanski T, Villeneuve D, Waters KM, Whelan M. (2017) How Adverse Outcome
Pathways can aid the development and use of computational prediction models for
regulatory toxicology. Toxicol Sci. 155: 326-336.
Unclassified
-------
ENY/JM/MONO(2016)12 | 59
Annex 1: Guidance for Assessing Relative Level of Confidence in the Overall
AOP
Examples of complete tables for selected AOPs are available:
AOP Assessment Summary File
https://aopwiki.o https://aopwiki.org/svstem/dragonflv/production/2017/05/19/7s 1 ibrunwt RevisedAss
rg/aops/15 essmentSummarvAop 15.pdf
https://aopwiki.o https://aopwiki.org/svstem/dragonflv/production/2017/03/20/3usvv7naq8 Annexl fo
rg/aops/23 r AOP 23 AR reproductive dvs 2017 03 20.pdf
https://aopwiki.o https://aopwiki.Org/aops/38#evidence
rg/aops/38
https://aopwiki.o https://aopwiki.org/svstem/dragonflv/production/2017/03/24/6u60ihkip8 TPO AOP
rg/aops/42 Summary Tables.pdf
Defining Question High2
Moderate
Low
1. Support for
Biological
Plausibility of
KERs1
4MIE => KE1:
(copy and
paste the KER
description
into this cell)
Is there a mechanistic
(i.e., structural or
functional)
relationship between
KEup and KEdown
consistent with
established biological
knowledge?
Extensive
understanding
based on extensive
previous
documentation and
broad acceptance
-Established
mechanistic basis
The KER is plausible
based on analogy to
accepted biological
relationships but
scientific
understanding is not
completely
established.
Biological Plausibility of the MIE => KE1 is xxx.
Rationale:
There is empirical
support for a statistical
association between
KEs (See 3.), but the
structural or functional
relationship between
them is not understood.
1 Rank ordered Bradford Hill considerations adapted from Meek et al. (2014b)
2 The guidance for "high", "moderate" and "low" draws on limited current experience. Additional
delineation of the nature of relevant evidence in these broadly defined categories requires more
experience with larger numbers of documented AOPs.
3 "Direct evidence" implies specifically designed experiments to consider the relevant element.
"Indirect evidence" may overlap with other elements.
4 To the extent possible, each of the relevant Bradford Hill considerations is addressed for each of
the KERs (biological plausibility and empirical support) and KEs (essentiality) and separate
rationales provided.
Unclassified
-------
60 | ENV/JM/MONO(2016)12
KE1 => KE2:
(copy and
, Biological Plausibility of KE1 => KE2 is xxx
Udutv LXlw t-\ . -|
Rationale:
description
into this cell)
KE2 => KE3
(copy and
, .. pR Biological Plausibility of KE 1 => KE2 is xxx.
Udutv LXlw t-\ j * 1
Rationale:
description
into this cell)
2. Support for
Essentiality of
KEs5
Defining
Question
What is the
impact on
downstream
KEs and/or the
AO if an
upstream KE
is modified or
prevented?
High
Direct evidence from
specifically designed
experimental studies
illustrating prevention
or impact on
downstream KEs
and/or the AO if
upstream KEs are
blocked or modified
Moderate
Indirect evidence that
modification of one or
more upstream KEs is
associated with a
corresponding
(increase or decrease)
in the magnitude or
frequency of
downstream KEs
Low
No or contradictory
experimental evidence of
the essentiality of any of
the KEs.
AOP
Rationale for Essentiality of KEs in the AOP is xxx:
3. Empirical
Support3 for
KERs
Defining
Questions
Does KEup
occur at lower
doses and
earlier time
points than KE
down and at
the same dose
of stressor, is
the incidence
of KEup >
High
Multiple studies
showing dependent
change in both events
following exposure to a
wide range of specific
stressors. (Extensive
evidence for temporal,
dose- response and
incidence concordance)
and no or few critical
Moderate
Demonstrated
dependent change in
both events following
exposure to a small
number of specific
stressors and some
evidence inconsistent
with expected pattern
that can be explained
by factors such as
Low
Limited or no studies
reporting dependent
change in both events
following exposure to a
specific stressor (i.e.,
endpoints never measured
in the same study or not at
all); and/or significant
inconsistencies in
empirical support across
5 While the extent of the supporting data on the essentiality of each of the KEs is addressed
separately (Table 5), delineation of the degree of confidence is based on consideration of evidence
for all of the KEs within the AOP and therefore, only one rationale is required. This call is
normally based on the extent of the available evidence for a range of KEs in the AOP.
Unclassified
-------
ENY/JM/MONO(2016)12 | 61
than that for
data gaps or conflicting
experimental design.
taxa and species that don't
KEdown?6,7
data
technical
align with expected pattern
considerations.
for hypothesised AOP
Are there
differences among
inconsistencies
laboratories, etc.
in empirical
support
across taxa.
species and
stressors that
don't align
with
expected
pattern for
hypothesised
AOP?
MIE => KE1:
(copy and
paste the KER Empirical Support of the MIE => KE1 is. xxx. Rationale:
description
into this cell)
KE1 => KE2 :
(copy and
paste the KER Empirical Support of the KE1 => KE2 is xxx. Rationale:
description
into this cell)
KE2 => KE3
(copy and
paste the KER Empirical Support of the KE1 => KE2 is xxx. Rationale:
description
into this cell)
a In many cases, evidence that contributes to quantitative understanding (Section 3-KER descriptions) will
also provide empirical support for the relationship. Consequently, relevant information from the
"Quantitative Understanding" section of the KER description should be considered as part of the overall
weight of evidence evaluation of the concordance of empirical observations and consistency for the KER.
6 This is normally considered on the basis of tabular presentation of available data on temporal and
dose-response aspects, in a template that documents the extent of support. See, for example. Table
6.
7 Note that this relates to concordance of dose response, temporal and incidence relationships for
KERs rather than the KEs; the defining question is not whether or not there is a dose response
relationship for the KE but rather there is concordance with that for earlier and later KEs. This is
normally demonstrated in studies with different types of stressors.
Unclassified
-------
62 | ENV/JM/MONO(2016)12
Annex 2: General guidance for characterising the level of quantitative
understanding of a KER as low, moderate, or high.
Extent of
Quantitative
Understanding
High
Moderate
Characteristics
Change in KEdownstream can be precisely predicted based on a
relevant measure of KEupstream.
Uncertainty in the quantitative prediction can be precisely
estimated from the variability in the relevant measure of
KEupstream.
Known modulating factors and feedback/feedforward
mechanisms are accounted for in the quantitative description.
There is evidence that the quantitative relationship between the
KEs generalises across the relevant applicability domain of the
KER.
Change in KEdownstream can be precisely predicted based on a
relevant measure of KEupstream.
Uncertainty in the quantitative prediction is influenced by factors
other than the variability in the relevant measure of KEupstream.
Quantitative description does not account for all known
modulating factors and/or known feedback/feedforward
mechanisms.
The quantitative relationship has only been demonstrated for a
subset of the overall applicability domain of the KER (e.g., based
on a single species).
Only a qualitative or semi-quantitative prediction of the change in
KEdownstream can be determined from a measure of
KEupstream.
Known modulating factors and/or known feedback/feedforward
mechanisms are not accounted for.
The quantitative relationship has only been demonstrated for a
narrow subset of the overall applicability domain of the KER
(e.g., based on a single species).
Unclassified
------- |