Guidance for Considering and Using Open Literature
Toxicity Studies to Support Human Health Risk Assessment
PROCEDURES FOR REVIEWING RELEVANT EFFECTS DATA PUBLISHED IN THE
OPEN LITERATURE
FOR USE IN OPP's HUMAN HEALTH RISK ASSESSMENTS
8/28/2012
Office of Pesticide Programs
U.S. Environmental Protection Agency
1
-------
Table of Contents
1. Introduction 3
1.1. Purpose 3
1.2. Organization of the Document 3
2. Screening the Open Literature Studies 4
2.1. Accepted Journal Articles/Publications by OPP 4
2.2. Documenting Relevant Journal Articles/Publications 4
3. Reviewing the Open Literature Studies 5
3.1. Study Categorization 5
3.2. Guidance for Open Literature Study Review 6
3.2.1. Guidance for Evaluating the Acceptability of Open Literature Studies 6
3.2.2. Guidance for Differentiating Between Qualitative and Quantitative Studies 8
3.2.3. Special Notes on Epidemiologic Data 10
3.3. Completion of Data Evaluation Records (DERs) for Journal Articles 11
3.3.1. Completion and Submittal of Journal DERs for Data Used Quantitatively 12
3.3.2. Completion and Submittal of Journal DERs for Data Used Qualitatively 12
3.3.3. Completion and Submittal of Journal DERs for Unacceptable Studies 12
4. Use of Open Literature in OPP's Risk Assessments 13
4.1. Use of "Quantitative" Open Literature Data in Risk Assessment 13
4.2. Use of "Qualitative" Open Literature Data in Risk Assessment 13
5. References 15
Attachment 1: Criteria for Invalidation of Open Literature Studies 16
2
-------
1. Introduction
1.1. Purpose
Toxicological and metabolism data for pesticide chemicals (active and inert ingredients) are
provided by the registrants as required in 40 CFR Part 158. Guidelines for conducting studies to
meet these requirements are available on the OCSPP Harmonized Test Guideline webpage:
http://www.epa.gov/ocspp/pubs/frs/home/guidelin.htm. In addition to the information submitted
by the registrants, effects data from studies published in the open literature may also be
considered in risk assessments conducted in the Office of Pesticide Programs (OPP).
The purpose of this document is to provide further information and clarification to assist in the
selection and review of relevant publications available in the open literature (i.e., non-test
guideline studies) for use in hazard and dose response assessment. This guidance was developed
to assist OPP scientists and is intended for use in OPP's risk assessments. This guidance draws
from OPP's long standing experience and guidance for review of registrant-submitted studies
submitted in response to the 40 CFR Part 158 data requirements. It is intended to ensure
consistent consideration, use, and documentation of information in the open literature by OPP
scientists and risk assessors when evaluating the potential adverse effects on human health. This
document is also intended to make transparent how OPP judges the scientific quality of open
literature publications of relevance and importance to human health risk assessment.
Although this guidance focuses on mammalian in vivo toxicity studies, its general principles and
criteria also apply to pharmacokinetic/metabolism, mechanism of toxicity and in vitro studies.
1.2. Organization of the Document
This guidance is divided into the following three sections:
• Screening the Open Literature Studies: Discusses how to determine which journal
articles / publications to consider relevant to the specific purpose of human health risk
assessment.
• Reviewing the Open Literature Studies: Provides study categorizations, criteria for
study reviews, and preparation of documentation of reviewed open literature [i.e.,
preparation of a Data Evaluation Record (DER) or Abbreviated Data Evaluation Record
(AbDER)].
• Use of Open Literature Studies in Risk Assessment: Provides guidance for use of
quantitative and qualitative data in OPP's risk assessments conducted for Registration
Review and Registration actions.
3
-------
2. Screening the Open Literature Studies
When evaluators conduct a literature search, the evaluator should keep the search parameters and
the results of the literature search as a record. It is possible a large amount of published papers
will be identified that are of possible interest. An initial screening process is needed to identify
those papers that are appropriate for the purposes of addressing the critical questions of human
health risk assessment (e.g., what are the potential toxicities of the chemical, at what doses are
effects found, what lifestages are impacted, how does the chemical causes its toxicity). The
purpose of this section of the guidance is to discuss the screening process used to identify
potentially suitable and useful open literature journal articles/publications. The screening criteria
for accepted journal articles/publications that are described below in Section 2.1 are taken from
the Pesticide Reregi strati on Rejection Rate Toxicology document (USEPA.1993).
2.1. Accepted Journal Articles/Publications by OPP
In order to be eligible for consideration, journal articles/publications need to meet the following
minimum criteria:
1. The toxic effects are related to defined chemical exposure;
2. The toxic effects are on an appropriate test animal species;
3. The presence or absence of toxicological effects is observed;
4. A chemical concentration/dose or application rate is reported;
5. An explicit duration of exposure is included;
6. Toxicology information is reported for the chemical of interest or its structural
analog;
7. The article is available in the English language;
8. The study results are presented as a full article (i.e., not an abstract);
9. The paper is a publically available document;
10. The paper is the primary source of the data;
11. Treatment(s) are compared to acceptable controls;
12. The location of the study (e.g., laboratory vs. field) is reported;
13. Adequate data are provided on the chemical tested (i.e., test article characterization);
14. Adequate data are provided on the species tested;
15. The study results (findings) are adequately reported; and
16. The study findings are relevant to assessing human health risks
2.2. Documenting Relevant Journal Articles/Publications
Once the determination is made that the open literature article/publication is eligible for
consideration (based on the screening criteria discussed in Section 2.1) and may be used
quantitatively or qualitative in risk assessments, a Master Record Identification number (MRID),
for the article should be requested. The purpose of assigning an MRID to the open literature
study is to ensure that the study is documented as part of the study bibliography for the chemical
4
-------
in the Office of Pesticide Program's Information Network (OPPIN) and electronically available
via Documentum. In order to obtain an MRID number for an open literature study, an electronic
copy of the study should be provided to the Alternate Contracting Officer Representative
(ACOR) of the Data Management Contract in the Information Technology Resources and
Management Division (ITRMD). Currently, the ACOR point of contact in ITRMD is Teresa
Downs (703-305-5363, downs.teresa@epa.gov). An electronic copy of the study should be
provided to the ACOR in an email and the MRID number is typically assigned within 2 to 10
days. Once the MRID is assigned to the open literature study, the citation for the study will
appear in the OPPIN bibliography. In addition, a ".tif' file of the study will be available in
Documentum approximately one month after the MRID is assigned to the study.
Journal articles are sometimes submitted to the Agency by outside stakeholders such as
environmental groups or the registrant. When this occurs, the article is processed in the same
manner as standard test guideline studies (i.e., scanned into Documentum, entered into OPPIN
and assigned an MRID by ITRMD).
3. Reviewing the Open Literature Studies
All open literature journal articles/publications that are identified as potentially useful based on
the selection criteria discussed in Section 2 should be reviewed, categorized, and documented.
A description of the open literature study categorizations, guidelines for study reviews, and
completion/documentation of open literature data summaries is provided in Sections 3.1
through 3.3, respectively.
3.1. Study Categorization
Open literature studies that may provide additional information on measurement doses/endpoints
should be reviewed and categorized as to their usefulness in a risk assessment. The three general
categories for open literature studies are:
• Quantitative: Appropriate for quantitative use [i.e., establishing a point of departure such
as No Observed Adverse Effect Level (NOAEL), Lowest Observed Adverse Effect Level
(LOAEL), Benchmark Dose (BMD), cancer slope factor, etc.] in risk assessment;
• Qualitative: Not appropriate for quantitative use, but is of sufficient quality, relevant to
issues and questions within the risk assessment of a chemical, and can be used
descriptively in the weight of the evidence and risk characterization; and
• Unacceptable: Inappropriate for quantitative or qualitative use in risk assessment
because it is of insufficient quality and lacks scientific reliability and defensibility.
Further description of the guidelines for open literature study categorization as "quantitative",
"qualitative", or "unacceptable" is provided below in Section 3.2.
5
-------
3.2. Guidance for Open Literature Study Review
This guidance will enable scientists to consistently differentiate open literature studies into the
three categories outlined in Section 3.1. The scientist must also use best professional judgment,
in addition to the considerations discussed below to determine the appropriate study
categorization for open literature studies. While a single factor may result in categorization of
the study as unacceptable (e.g., excessive control mortality), more typically, several issues
combine to render the study of questionable reliability and utility.
3.2.1. Guidance for Evaluating the Acceptability of Open Literature
Studies
Consistent with guidance to determine whether a study meets the criteria outlined in pesticide
testing guidelines, general information that should be considered as important in determining the
reliability and utility of an open literature study in risk assessment includes the following:
• Nature of the test substance (percent active ingredient). The study needs to indicate the
exact nature and source of the pesticide; the percent active ingredient and/or the purity of the
test compound should also be reported. If a solvent vehicle is used, the vehicle should not
interfere with the absorption, distribution, metabolism or the elimination (ADME) of the test
substance nor alter the behavior/response of the test organisms. Studies which use a solvent
vehicle should also include solvent vehicle controls.
• Test organism. Species, age, sex, size, health and life stage and source of the test species
should be reported. Any observed diseases and treatment need to be reported.
• The number of organisms tested per concentration and the number of concentrations or
dosage levels evaluated. This type of information should be reported and be sufficient to
yield statistically sounding data. An inadequate number of test organisms per test level can
also produce unreliable results. The appropriate comparable guideline study Standard
Evaluation Procedure (SEP) should be consulted for further information on the adequate
number of test organisms per test level.
• Husbandry conditions. Guideline studies have been developed using particular species to
establish conditions under which the test organisms are most likely to thrive and where
husbandry conditions will not confound the interpretation of the study. Reviewers need to be
cognizant of husbandry conditions and verify whether the environmental conditions of the
study are adequately described and/or addressed to ensure that the test organisms are not
adversely affected. This description should include the number of animals per cage or test
container (i.e., biological loading rate); nature and composition of bedding used for
mammalian studies (if available); ambient temperature and humidity; photoperiod;
description of the diet; source of the animal feed; dimensions of the test container.
6
-------
• Exposure method, route, and frequency of administration and length of the treatment
period. The dose administered (test substance plus carrier) to each organism (in feed or
water) at each time administration is made need to be reported. In addition, the frequency of
administration and duration of the exposure need to be reported. For all studies, the exposure
conditions need to be clearly described and documented. Additionally, the reviewer needs to
consider whether test conditions may not sufficiently preclude exposure to other chemicals
that could potentially confound the study. In such cases, the reviewer should consider the
variability associated with the measured endpoints from the controls.
• Controls. A suitable number of controls need to be run to test whether study conditions are
adequate. Control performance should be used as an indicator of whether study conditions
and animal performance are adequate. To this end, controls need to be run concurrent with
the study; failure to do so would render the study unacceptable. As mentioned previously,
studies which rely on solvent vehicles should report concurrent solvent controls. As an
indicator of study conditions, control performance in terms of mortality and disease should
be carefully evaluated to determine the adequacy of the study. Mortality of greater than 10%
in controls for most test species is sufficient to conclude that the study is unacceptable.
Ideally, studies should also report the measured concentrations of test chemical in the
controls.
• Performance of test species. Normal development times (where available) should be
compared to those reported for the test species. Where the development time for the control
animals differs substantially from normal reported values, the reviewer needs to determine
whether study conditions have impaired the animals' ability to thrive. In cases where
development time is substantially different than what is typically observed for the test
organisms, the study should be considered as unacceptable as the study's ability to
distinguish treatment effects is uncertain.
• Macroscopic observations of the test animals. During the course of the study, a detailed
description of the nature, incidence, time of occurrence, severity, and duration of all observed
toxic effects, including death and any other abnormal or unusual signs and symptoms {i.e.,
sub lethal effects) should be reported.
• Microscopic observations of the test animals. Tissues and organs for microscopic
examination should be fixed in 10% buffered formalin or recognized fixative. Reporting of
microscopic evaluations should consist of accurate diagnosis of all non-neoplastic {e.g.,
atrophy, hypertrophy, hyperplasia, and dysplasia) and neoplastic {i.e., tumors) lesions
observed in the control and treated groups. Neoplastic findings {i.e., tumors) where
applicable, may be reported as benign and malignant. This evaluation is important for
integrated interpretation of the findings to identify and characterize the histopathological
findings of a study. Microscopic evaluation of the slides should follow the guidelines
established in the Society of Toxicological Pathology's Best Practices Guideline paper
(Toxicological Pathology 32:126-131; 2004).
7
-------
• In vitro studies should include the following data: description of the test system/test method;
purity/composition/origin of the test substance; data on dose/concentration tested; data on
solubility, impurities, and pH; presence of absence of metabolic activation; appropriate
negative/positive controls; and the appropriateness of the method of analysis performed.
• Statistical method used to derive the test endpoints. Verification of the statistical analysis
is an integral part of the data evaluation process. As such, studies should provide descriptive
statistics that report measures of central tendency (e.g., means, medians) and measures of
dispersion (e.g., standard deviations, standard errors) along with associated sample sizes (N
values). The report should state which methods of statistical comparison (e.g., t-test,
ANOVA, chi square) were used and the presumed nature of the data (parametric versus
nonparametric) and whether the data supported use of parametric analyses.
• Information necessary to provide a complete and accurate description of test
procedures and evaluation of the test results. Each report should include a summary of
the data, a description of the statistical analysis of the data, and a statement of conclusions
drawn from the analysis that allows the reader to independently understand the conclusions
of the author. Sometimes it is important to obtain raw data from the study authors.
• Important information missing from the study. Inconsistencies or deviations with
recommended methodologies, as discussed in the appropriate comparable guideline study
SEP and/or 870 guideline for each of the respective studies, should be addressed. SEPs
and/or 870 test guidelines can provide additional measures of gauging the reliability of study
conditions.
• The toxic effects must be able to be attributed to exposure from the chemical.
An acceptable open literature study may have some limitations but will still contribute
information to the assessment. Unacceptable open literature studies, however, are those that are
not considered scientifically sound and as such do not provide useful/reliable information. These
can include studies that were performed under conditions that deviated significantly from
scientifically accepted methods or recommended protocols such that the scientific integrity of the
study is uncertain and the results should not be used to support risk assessment. In addition to
the guidance discussed in this section, a list of additional factors that could result in an open
literature study being categorized as 'unacceptable' is provided in Attachment 1.
3.2.2. Guidance for Differentiating Between Qualitative and Quantitative
Studies
If a study is considered to be acceptable based on the guidance described in Section 3.2.1. and/or
Attachment 1, a determination is made regarding whether the information provided in the study
is adequate for "qualitative" or "quantitative" use in risk assessment. For OPP's purposes,
"qualitative" refers to data that can be used in a weight of evidence evaluation to support
8
-------
conclusions regarding potential hazard. "Quantitative" means the dose response data from the
study can be used for establishing a point of departure for risk assessment.
To be used quantitatively, the data reported in the open literature need to meet all of the
following criteria:
• The dose from the open literature study is lower (i.e., more sensitive) than the lowest
dose from a comparable registrant-submitted study;
• The open literature data are reported in (or have the ability to be converted to) units that
can be compared to other study results; and
• Sufficient information is provided in the open literature to substantiate whether the study
conclusions/endpoints/doses are accurate, reliable, and reasonable and a judgement can
be made that the study findings could potentially be replicated (as per Section 3.2).
If a scientifically valid study does not meet any of these three criteria, the data from the study
should be categorized as "qualitative." OPP recognizes that the third criterion (i.e., sufficient
information is provided to substantiate whether the conclusions/endpoints/doses are accurate)
requires best professional judgment. The most reliable means of determining whether study
conclusions can be verified is through access to the raw data; however, it is recognized that very
few open literature journal articles/publications provide this type of information. Therefore, the
quantitative use of open literature requires that the study provide a relatively comprehensive
understanding of the conditions under which the study was conducted and of the data generated
by the study. If the open literature study is important to the risk assessment and this
comprehensive understanding is not provided, the reviewer should attempt to obtain missing
information from the study, including the raw data from the study authors.
To assess the third criterion the reviewer should consider whether the study reports relatively
detailed measures of the variability associated with the data and the methods used to analyze the
data. Reviewers should note whether the statistical tests used in the study are appropriate to the
design of the study, the nature of the measurement endpoint, and of the data generated in the
study. Tests using parametric statistics should indicate whether the conditions for such tests (i.e.,
normal distribution and homogeneity of variance) have been met.
Where raw data cannot be obtained or are not available to verify the study results, the reviewer
needs to discuss the uncertainties associated with quantitative use of the data relative to studies
where raw data are provided. Consideration needs to be given as to the extent to which results
are aligned with other lines of evidence. Open literature values that are inconsistent with similar
measures of toxicity should be carefully scrutinized to determine their reliability.
Ultimately, distinguishing between data that can be used qualitatively versus quantitatively will
largely depend on professional judgment.
9
-------
3.2.3. Special Notes on Epidemiologic Data
In addition to experimental toxicological evidence, OPP is interested in querying the peer review
literature for observational epidemiology studies of potential adverse acute and chronic health
effects linked to pesticide use. Epidemiologic research utilizing cohort, case-control or cross-
sectional study designs may provide information to strengthen OPP's understanding of the
potential hazards, exposure-response characterization, exposure scenarios or assessment
methods, and ultimately risk characterization (Van den Brandt, 2002). In addition, at times
compelling case reports or case series analysis may illumine a health effect or mechanism of
action previously unidentified.
Recently, OPP has developed draft guidance for incorporating epidemiologic research into the
risk assessment process. OPP anticipates increased use of these types of data in our risk
assessment process as epidemiologic cohorts such as the National Cancer Institute's Agricultural
Health Study, among others, continue to mature (increased time on study), and associations
between pesticide use and adverse cancer and non-cancer outcomes are refined and clarified.
To perform a query of the published epidemiologic data, OPP scientists utilize biomedical search
tools such as MedLine/PubMed, Web of Science, and Google.scholar. These three biomedical
search tools are among the most well-developed and characterized for use by epidemiologists
(Falagas 2006). Working in conjunction with EPA reference librarians, internal data query
experts, and using best professional judgment, OPP scientists develop a search string appropriate
to the research question of interest. Use of Medical Subject Heading (MeSH) indexing is
particularly helpful in developing a comprehensive search string. Manually searching reference
lists of key/pivotal articles (secondary level searching) is also recommended. Using the I SI/Web
of Science search tool, OPP scientists can also perform citation mapping in which articles that
cite key/pivotal research are automatically identified. In this way, additional studies potentially
pertinent to the research question may be identified. Ultimately, the particular needs of the risk
assessment will dictate the level of sophistication of the biomedical literature review. In all
cases, search string variables, date of search, and original reference lists can be retained to
delineate literature search methodology and allow replication, if needed.
Regarding the selection of epidemiologic studies identified in the peer reviewed literature for use
in risk assessment, OPP is currently in the process of developing a guidance document detailing
the characteristics of epidemiology studies deemed desirable for this purpose. The guidelines
discussed in this document with respect to querying the experimental toxicological literature may
or may not be used for the purposes of selecting observational epidemiology studies.
Generally speaking, the quality epidemiologic research, sufficiency of documentation of the
study (study design and results), and relevance to risk assessment will be considered when
selecting epidemiology studies from the open literature for use in OPP's risk assessments. These
include:
10
-------
1. Clear articulation of the hypothesis, even if the study is hypothesis-generating in nature;
2. Adequate assessment of exposure for the relevant critical windows of the health effects,
the range of exposure of interest for the risk assessment target population, and the
availability of a dose/exposure-response trend from the study, among other qualities of
exposure assessment,
3. Reasonably valid and reliable outcome ascertainment (the correct identification of those
with and without the health effect in the study population),
4. Appropriate inclusion and exclusion criteria that result in a sample population
representative of the target population, and absent systematic bias,
5. Adequate measurement and analysis of potentially confounding variables, including
measurement or discussion of the role of multiple pesticide exposure, or mixtures
exposure in the risk estimates observed,
6. Overall characterization of potential systematic biases in the study including errors in the
selection of participation and in the collection of information, including performance of
sensitivity analysis to determine the potential influence of systematic error on the risk
estimates presented (e.g., Greenland's formula)
7. Evaluation of the statistical power of the study, if under-powered to observed an effects,
appropriate discussion and/or presentation of power estimates,
8. Use of appropriate statistical modeling techniques, given the study design and the nature
of the outcomes under study.
OPP will finalize the draft Framework for incorporating epidemiology into risk assessment,
including factors to consider when selecting studies for inclusion in qualitative or quantitative
aspects of the risk assessment. FIFRA Scientific Advisory Panel comments on OPP's draft
framework can be found at:
http://www.epa.gOv/scipolv/sap/meetings/2010/020210meeting.html#transcripts
3.3. Completion of Data Evaluation Records (DERs) for Journal Articles
Once a journal article has been determined useful for risk assessment (qualitatively or
quantitatively), OPP staff should complete a DER (or AbDER) for the review using the standard
toxicology template form that is the best fit for the type of study described in the article
following the established Standard Operating Procedures (e.g., HED SOP 2001.02 and 2001.03).
The purpose of completing the DER is to ensure an efficient and consistent process for
documenting reviews of open literature and avoiding duplicative and possibly conflicting efforts
associated with study. The procedures for completion and submittal of DERs for endpoints that
are categorized as "qualitative," "quantitative" or "unacceptable" are described below in
Sections 3.3.1 through 3.3.3, respectively.
An MRID number needs to be obtained for any open literature used or qualitatively or
quantitatively in risk assessments (based on the screening criteria discussed in Section 2.1). See
Section 2.2 for instructions on obtaining an MRID number. Once the risk assessor has obtained
11
-------
an MRID number and completed the journal DER (including secondary and/or peer review), the
journal DER should be out-processed as for any other OPP work.
3.3.1. Completion and Submittal of Journal DERs for Data Used
Quantitatively
Review summaries of open literature data that are used quantitatively {i.e., to establish endpoints
and points of departure for risk assessment) should include all available information that would
normally be included as part of the current guideline/non-guideline DER templates. Although
the journal DER should include the same type of information, it is expected that they will be
reduced in length and detail as compared to standard DERs because raw data are generally not
available for review and only the data in the published study are being evaluated. The basic
study requirements should be verified and reported using the standard DER template that best fits
the article content. The review should document all statistically or biologically significant
effects. In addition, the duration of exposure, the magnitude of the effect, and the test
concentration (nominal, measured, and time-weighted average, if it can be determined) at which
the effect was observed should be documented. In addition, the reviewer is encouraged to
include relevant figures and tables from the study that include key findings; table and figure
captions should properly cite the relevant publication if the figure and/or table is copied from the
publication.
All open literature studies that are categorized as "quantitative" need to undergo secondary
review and/or peer review within OPP.
3.3.2. Completion and Submittal of Journal DERs for Data Used
Qualitatively
At the discretion of the evaluator, DERs may be completed for open literature studies that
include data to be used qualitatively in the risk assessment. The evaluator should consider
preparing a DER for studies that provide novel information and are critical to the conclusions of
the assessment. In addition, DERs for qualitative assessments should include the same type of
information and level of detail as reviews that are completed for quantitative assessments. DERs
for qualitative data need to include descriptions of the study limitations which preclude their
quantitative use. These DERs should undergo secondary review.
3.3.3. Completion and Submittal of Journal DERs for Unacceptable Open
Literature Studies
Literature studies that are determined to be unacceptable do not require a DER since they will
not be considered for use in the OPP risk assessment. However, DERs should be completed for
unacceptable studies that are submitted to the Agency by outside stakeholders. The level of
12
-------
detail for an "unacceptable" review relative to "quantitative" and "qualitative" reviews should be
significantly reduced. The DERs for unacceptable studies should be condensed into 1-2 pages
and focus on the limitations of the study which preclude its use in hazard/risk assessment.
Detailed description of the experimental design is not required for studies that are categorized as
"unacceptable."
4. Use of Open Literature in OPP's Risk Assessments
The extent to which open literature data categorized as either "qualitative" or "quantitative"
should be used in the risk assessment is discussed below in Sections 4.1.1 and 4.1.2,
respectively. Open literature studies that pass the initial screen and are determined to be
"unacceptable" based on the risk assessor's review should not be included in the risk assessment.
4.1. Use of "Quantitative" Open Literature Data in Risk Assessment
As previously discussed in Section 3.2.2, endpoints and points of departure from the open
literature that are more sensitive (or lower) than the lowest registrant-submitted study and are
categorized as "quantitative" may be used for establishing a point of departure for risk
assessment.
Where data from open literature are deemed to be of sufficient quality to permit their use
quantitatively in OPP's risk assessment, the assessment needs to provide a relatively
comprehensive review of the open literature study associated with the dose/endpoint. Any open
literature data that are categorized as 'quantitative' and used for endpoint and dose selection in
the risk assessment needs to be fully described in the toxicological effects section of the
assessment, with particular emphasis on those open literature endpoints that result in lower
values than those used in previous risk assessments. In addition, the risk assessor should cite the
DERs for all 'quantitative' endpoints and doses in the risk assessment. If applicable, the risk
assessor needs to provide clear and transparent rationale for quantitatively using the open
literature data over guideline and GLP-compliant data. As described in more detail in Section
3.2., the criteria used to evaluate test guideline studies and best professional judgment should be
used to determine the appropriate use of an open literature study in risk assessment.
4.2. Use of "Qualitative" Open Literature Data in Risk Assessment
Although data from the open literature that are categorized as "qualitative" are not appropriate
for quantitative use {i.e., dose selection), they should be discussed in the toxicological effects and
risk characterization sections of the risk assessment as additional lines of evidence to support risk
conclusions regarding metabolism, adverse effects of concern, life stage susceptibility, and mode
of toxic action. A clear rationale should be provided in the effects section that describes why the
data were not used quantitatively. These reasons might include limitations in the study design,
lack of sufficient information to substantiate whether the conclusions/endpoints/doses are
13
-------
accurate, and other uncertainties that confound the ability to discriminate a dose-related effect.
As previously stated, best professional judgment should be used to determine the appropriate use
of an open literature study in risk assessment.
Conclusion
In summary, this document provides guidance for the use of open literature publications and
criteria forjudging its quality and relevance in support of OPP human health risk assessments.
To permit independent review of the study findings, the study method and findings need to be
sufficiently documented and transparent. In principle, the more details of the methodology and
findings, the greater the confidence in the publication's reliability. Studies that use scientifically
sound and appropriate methodology and relevant routes of exposure are important to consider
because they may provide valuable information for the risk assessment.
14
-------
5. References
Crissman et al (2004) Best Practices Guideline: Toxicologic Histopathology. Toxicologic Pathology
32: 126-131.
Falagas (2006) Peer Review in Open Access Scientific Journals. Open Medicine Vol. 1, No. 1.
Klimisch, H.J., Andreae, M. and Tillman, U. (1997). A Systematic Approach for Evaluating the
Quality of Experimental Toxicological and Ecotoxicological Data; Regulatory Toxicology and
Pharmacology 25: 1-5.
Society of Toxicological Pathology's Best Practices Guideline paper (Toxicological Pathology
32:126-131; 2004).
U.S. Environmental Protection Agency (USEPA). Series 870 Health Effects Test Guidelines
(http://www.epa.gov/ocspp/pubs/frs/publications/Test Guidelines/series870.htm)
USEPA. 1993. Pesticide Reregistration Rejection Rate Analysis Toxicology. Office of
Prevention, Pesticides, and Toxic Substances. EPA 738-R-93-004.
U.S. EPA. 2010. Evaluation Guidelines for Ecological Toxicity Data in the Open Literature,
Procedures for Screening, Reviewing, and Using Published Open Literature Toxicity Data in
Ecological Risk Assessments; November 4, 2010
Van den Brandt et al. (2002) The Contribution of Epidemiology. Food and Chemical Toxicology
Vol. 40 Issues 2-3: Feb.-March 2002: 387-424.
15
-------
Attachment 1:
Criteria for Invalidation of Open Literature Studies
• Lack of characterization of the test substance
• Lack of characterization of vehicle/solvent controls used
• Inadequate or missing analytical data
• Insufficient number of animals tested
• Poorly controlled test environment
• Insufficient number of dose levels tested
• Insufficient number of parameters evaluated
• Lack of clinical pathology data
• Lack of macroscopic and/or histopathology data
• Lack of appropriate statistical methodology
• Deficiencies in reporting of study data
16
------- |