nnual Rep
I
ntegrity
&EFK
-------
-------
Scientific Integrity at EPA
Annual Report Fiscal Year 2014
-------
The U.S. Environmental Protection Agency is charged by Congress with
protecting the Nation's land, air, and water resources. Under a mandate of
national environmental laws, the Agency strives to formulate and implement
actions leading to a compatible balance between human activities and the
ability of natural systems to support and nurture life.To meet this mandate,
EPA programs provide data and technical support for solving environmental
problems today and building a science knowledge base necessary to manage
our ecological resources wisely, understand how pollutants affect our health,
and prevent or reduce environmental risks in the future.
EPA's Scientific Integrity Official (ScIO) champions scientific integrity
throughout the Agency.The ScIO chairs the Scientific Integrity Committee
comprised of Deputy Scientific Integrity Officials who represent each EPA
program office and region. Science is the backbone of EPA's decision-making.
The Agency's ability to pursue its mission to protect human health and the
environment depends upon the integrity of the science on which it relies.
The full text of this report is available on EPA's website at:
http://www.epa.gov/scientificintegrity/2014annualreport/
Cover Design by Armando Valentino Chagolla, October 2014.
About the CovenThe cover is adapted from a new Scientific Integrity poster.
The poster will be printed and distributed throughoutthe Agency to increase
awareness of Scientific Integrity.The poster also will be made available to
internal and external groups at conferences and other outreach events.
EPA Publication number 601R14008
Printed on 100% Postconsumer, Process Chlorine Free Recycled Paper
-------
Contributors &. ,
Acknowledgments
Acknowledgments
The contributors wish to acknowledge:
The Scientific Integrity Committee, fortheir role in championing Scientific
Integrity throughout the Agency, their assistance in gathering the information
needed for this report, and their helpful review of an earlier draft.
Barbara Martinez, Ph.D., for her contributions to the design of the Scientific
Integrity Policy Evaluation process and the related figures.
Armando Valentino Chagolla, for his assistance formatting the figures, layout
and overall design of this report.
Contributors
FrancescaT. Grifo
Francesca is the Scientific Integrity Official at the U.S. Environmental
Protection Agency. Previously she was the Director of the Scientific Integrity
Program at the Union of Concerned Scientists. She holds a BA in biology from
Smith College and a Ph.D. in plant systematics from Cornell University.
Martha Otto
Martha is the Scientific Integrity Program Lead in the Office of the Science
Advisor. Martha has over 25 years of experience in hazardous waste site
remediation, innovative treatment technologies, and policy, regulation, and
guidance development. She holds a BS in biology and an MS in environmental
science and engineering from VirginiaTech.
Valerie Askinazi
Valerie is a student services contractor in the Office of the Science Advisor.
She holds a BS in science of natural and environmental systems from Cornell
University and an MPA in environmental science and policy from Columbia
University.
-------
Tab
of,
ontents
Contributors and Acknowledgments iii
Figures and Boxes vi
Acronyms vii
Executive Summary 1
1. Introduction 3
EPA Scientific Integrity Milestones 3
What is Scientific Integrity? 4
EPA's Scientific Integrity Policy 6
Promoting a Culture of Scientific Integrity at EPA 6
Releasing Information to the Public 7
Peer Review and the Use of Federal Advisory Committees 7
Professional Development of Government Scientists and Engineers 7
2. Scientific Integrity in FY 2014 s
Ongoing Scientific Integrity Activities 8
The Scientific Integrity Committee 8
The Annual Report on Scientific Integrity 8
Annual Meeting/Conversation with the Scientific Integrity Official 8
Scientific Integrity Training 9
Strengthening Contractor-Managed Peer Review 9
Quarterly Coordination Meetings with the Office of Inspector General 10
New Scientific Integrity Initiatives 10
Procedures for Reporting and Resolving Allegations of a Loss of Scientific
Integrity 10
FMFIA:The Federal Managers Financial Integrity Act 12
Policy Evaluation 12
Peer Review Handbook 15
Scientific Integrity Outreach 16
Quarterly Meetings of the Union Working Group 16
EthicsTraining Module 16
-------
Scientific Integrity Accomplishments 17
Promoting a Culture of Scientific Integrity 17
Release of Information to the Public 18
Peer Review and Federal Advisory Committees 19
Professional Development 20
3. Opportunities for Improvement 22
Reporting and Resolving Allegations of a Loss of Scientific Integrity ..22
Reducing Confusion about Designating Publication Authorship 24
Increasing Transparency 25
Media Access to EPA Scientists 25
Decision Making 25
Public Access 25
Addressing Constraints to Full Implementation of the Policy 25
Maintaining Scientific Expertise in aTime of Reduced Workforce 25
Outreach and Training 26
Defining the Timely Release of Agency Science and Scientific Products.26
Enhancing Peer Review 26
4. Areas for Future Investment 28
Annual Activities and Requirements 28
Ongoing Projects 28
New Initiatives 28
Transparency and the PublicTrust 28
Nurturing a Culture of Robust Scientific Discourse 29
Extending EPA's Culture of Scientific Integrity 29
5. Conclusions so
Sources and End Notes 32
Appendix 1 34
Appendix 2 38
Members of the Scientific Integrity Committee 39
-------
Figures and Box
Figures and
oxes
Figures
Figure 1.Timeline of Scientific Integrity at EPA 2
Figure 2. Scientific Integrity at EPA 6
Figure 3. The Annual Cycle of Ongoing Scientific Integrity Activities 9
Figure 4. Procedures for Reporting and Resolving Allegations (Draft) 11
Figure 5.The Evaluation of the Scientific Integrity Policy 14
Figure 6. An Example of One Logic Model Activity 14
Figure 7. Logic Model Conditions EPA is Striving to Achieve 15
Figure 8.Total Allegations Received by FY Quarter/Year (as of 9/30/14) 22
Figure 9. Informal and Formal Allegations Received in Fiscal Year 2014 23
Figure 10. Allegations by Quarter, FY14 23
Boxes
Box 1. EPA's Principles of Scientific Integrity 5
Box 2. Case Summary 13
-------
Acronyms
Acronyms
A A Assistant Administrator
DScIO Deputy Scientific Integrity Official
EPA U.S. Environmental Protection Agency
FMFIA Federal Managers Financial Integrity Act
HIS A Highly Influential Scientific Assessment
IRIS Integrated Risk Information System
ISI Influential Scientific Information
OAR Office of Air and Radiation
OCSPP Office of Chemical Safety and Pollution Prevention
OECA Office of Enforcement and Compliance Assurance
OGC Office of General Counsel
OIG Office of Inspector General
ORD Office of Research and Development
OS A Office of the Science Advisor
QMP Quality Management Plan
RA Regional Administrator
RFAC Regional Field Advisory Council
RSSC Regional Science Steering Council
ScIC Scientific Integrity Committee
ScIO Scientific Integrity Official
SclP Scientific Integrity Policy
SGE Special Government Employee
SOP Standard Operating Procedures
TSA Technical Systems Audit
-------
tVii
T 'ft?
-------
Executive
oummary
The Annual Report chronicles the implementation of EPA's Scientific Integrity
Policy in fiscal year (FY) 2014. Since February 2012, EPA's Scientific Integrity
Policy has provided both a vision and a roadmap for ensuring scientific
integrity at the Agency.The Policy lists the components of a culture of scientific
integrity and offers a framework for ensuring Agency-wide participation in that
culture. Although scientific integrity is treated as a single issue in the Policy,
maintaining scientific integrity requires investment from and the collaboration
of many parts of EPA.This report documents the investments made across
EPA in FY2014 and identifies areas of focus for future initiatives.
The 2014 Annual Report on Scientific Integrity represents a model for future
years by using the Federal Managers Financial Integrity Act (FMFIA) process
to assess the state of scientific integrity across the Agency. Each program
and regional office submitted a checklist to the Office of the Chief Financial
Officer, highlighting successes that exemplify a culture of scientific integrity,
transparency of EPA research, proper peer review of scientific documents and
the professional development of EPA's scientific staff.
Several of EPA's scientific integrity activities were first implemented prior
to FY2014.These activities provide ongoing support for scientific integrity
activities at EPA.These include quarterly meetings of the Scientific Integrity
Committee (ScIC), production of an annual report, the Annual Meeting/
Conversation with the Scientific Integrity Official (ScIO), training in scientific
integrity, oversight of contractor-led peer review and coordination with the
Office of Inspector General.
The Scientific Integrity Committee made significant strides creating new
scientific integrity initiatives across the Agency in FY2014. New draft
procedures were written for reporting and resolving allegations of a loss of
scientific integrity at the Agency.The Scientific Integrity Committee has
also begun evaluating the Scientific Integrity Policy, a process that will
identify the effectiveness of the Policy in meeting its goals.The Scientific
Integrity Committee has used a variety of outlets to reach EPA employees
who supervise, manage, perform, communicate or are engaged in scientific
research.These activities will continue to make scientific integrity a visible
component of EPA culture. New outreach activities included internal
presentations at Headquarters and regional offices for EPA employees, and
convening quarterly meetings of a scientific integrity Union Working Group.
In 2014, EPA program and regional offices have taken a variety of approaches
to enhance a culture of scientific integrity at EPA. Research-focused offices
have made advances in the public release of large data sets, dashboards
-------
2-3
and other mechanisms that lead to greater transparency and accessibility of
Agency science. Other offices have revised procedures to assure the quality of
Agency science, such as those for reviewing and approving scientific products
and for conducting peer review.The FMFIA program review also provided
an opportunity for offices to highlight activities to support the professional
development of EPA scientists and engineers so that they may engage with
their scientific community and be recognized as leaders in their fields.
Scientific integrity is an ongoing priority for EPA. While we have made
progress in 2014, we must continue to build on our successes to fully ensure a
robust culture of scientific integrity at EPA.The Annual Report shares several
highlights from the last year, but also acknowledges areas for improvement in
FY2015.
Figure 1
Timeline of
Scientific
Integrity
at EPA
Looking forward to fiscal year 2015 and beyond, six priority issues
present opportunities for ongoing investment:
1. Reporting and resolving allegations of a loss of scientific integrity.
2. Reducing confusion about designating publication authorship.
3. Increasing transparency.
4. Addressing constraints to full implementation of the Policy.
5. Defining the timely release of Agency science and scientific products.
6. Enhancing peer review.
Agency investments in these activities ensure the credibility of, and maintain
the public trust in, Agency science.
Science at EPA is robust and ready to meet the task of guiding our work,
protecting human health and the environment. Our investment in enhancing
scientific integrity is part of embracing EPA as a high-performing organization.
From the earliest formation of a scientific question to the application of
research results, scientific integrity creates protections for science from
inappropriate interference, manipulation and suppression.These protections
ensure that EPA decisions are informed by the best science the Agency, its
contractors, grantees and collaborators have to offer. The Scientific Integrity
Official and the Scientific Integrity Committee will continue to work with the
Senior Counsel for Ethics, the Office of Inspector General and the rest of the
Agency to safeguard our science and keep the public's trust in the quality and
integrity of our work every day. Scientific integrity gets us there.
1983
William Ruckelshaus
promises that EPA
employees will
"Conduct themselves
with the opneness and
integrity which alone
can insure public trust
in the Agency" in his
"fishbowl memo"
1999
National
Partnership
Council
releases EPA's
Principles
of Scientific
Integrity
January 2009
President
Obama
promises
to "restore
science to
its rightful
place" in his
inaugural
address
March 2009
President
Obama issues a
Scientific Integrity
Memorandum
giving OSTP 120
days to "develop
a plan aimed
at ensuring the
integrity of federal
science"
May 2009
EPA
Administrator
Lisa Jackson
issues her
"compass
memo" pledging
a culture
of scientific
integrity at EPA
-------
1. In
production
nirocTuciion
Continuing the tradition first articulated in 1983 by former EPA administrator
William Ruckelshaus "that EPA would operate in a fishbowl,"1 this report
is part of the Agency's ongoing commitment to transparency. EPA released
its Scientific Integrity Policy in February 2012 and its first annual report in
November 2013. EPA's Scientific Integrity Policy provides both a vision and
a roadmap for ensuring high standards of scientific integrity at the Agency.
The Policy lists the components of a culture of scientific integrity and offers
a framework for ensuring Agency-wide compliance. EPA assesses the overall
implementation of the Policy at the end of each fiscal year during a review
of all the scientific integrity activities at the Agency.The annual review for
FY2014 culminated in the publication of this Annual Report.The Annual Report
provides an opportunity to highlight scientific integrity accomplishments and
identify areas for future improvement and action.
EPA Scientific Integrity Milestones
In 1999, EPA unions and managers came together to write the Agency's
scientific integrity principles,2 which were developed by the National
Partnership Council.They are a call for employees to ensure that their
scientific work is of the highest integrity, to represent it fairly, acknowledge
the intellectual contributions of others and avoid financial conflicts (Box 1.
EPA's Principles of Scientific Integrity). In his inaugural address in 2009,
President Obama pledged to "...restore science to its rightful place..." A few
months later, he issued a Scientific Integrity Memorandum in which he asked
the White House Office of Science andTechnology Policy (OSTP) to create
a plan to establish strong scientific integrity standards.3 The resulting OSTP
Guidance Memorandum required agencies and departments to create or
improve policies related to foundations of scientific integrity in government;
public communications; use of federal advisory committees; and professional
December 2010
OSTP issues
guidance requiring
agencies and
departments to
create or improve
policies related to
scientific integrity
August 2011
EPA releases a
draft scientific
integrity policy
and opens a public
comment period
through which EPA
receives thousands
of comments
February 2012
EPA releases
its final
Scientific
Integrity
Policy
November 2013
EPA hires its first
full-time ScIO
April 2014
EPA
Administrator
Gina McCarthy
addresses
The National
Academy of
Sciences
-------
4-5
development of scientists and engineers. It acknowledged the differences in
structure and degree of regulatory responsibility in agencies and departments
and gave them some latitude on developing their policies.4 EPA was among
a few agencies to put its draft policy out for public comment and was one of
the first agencies to release a final Scientific Integrity Policy in February 2012
(Figure 1).
What is Scientific Integrity?
Scientific integrity is the adherence to professional values and practices
when conducting, communicating and applying the results of science and
scholarship. Scientific integrity ensures objectivity, clarity, reproducibility and
utility. It provides insulation from bias, fabrication, falsification, plagiarism,
outside interference and censorship. For example, if Agency end-users
directed research and results, or manipulated, suppressed or delayed science
or scientific reports, those actions could represent a loss of
scientific integrity.
"The work we do together to preserve the
integrity of our science is as critical as
ever...With science as our North Star-- EPA
has steered America away from health risk
and toward a higher quality of life."
Gina McCarthy April 29, 2014
It would also be seen if there were a lack of transparency around an Agency
decision, a lack of impartiality, or a conflict of interest in peer review or
advisory committees. A lack of scientific integrity may also manifest itself
as inappropriate discouragement of professional development or a culture
unwelcoming to differing scientific opinions.The Agency may make final
decisions that weigh other factors besides science and be consistent with
EPA's governing statutes. Such decisions, even if they are not consistent with
the science, do not in and of themselves constitute scientific integrity issues.
In addition, a denial of travel or train ing due to budget constraints or concerns
about an employee's ability to handle workload is also not a scientific integrity
issue. It also is not a denial of permission to publish when the research
is flawed nor is it a lack of freedom over one's research topics. Although
scientific integrity is treated as a single issue in the Scientific Integrity Policy,
implementing scientific integrity requires inputs from a wide variety of sources
toward that common goal (Figure 2).
-------
Box1: EPA's,Prin
niegr
National Partnership Council
Agency Labor Unions and Management in Partnership
Fostering Scientific Integrity
EPA's Principles of Scientific Integrity
It is essential that EPA's scientific and technical activities be
of the highest quality and credibility if EPA is to carry out its
responsibilities to protect human health and the environment.
Honesty and integrity in its activities and decision-making
processes are vital if the American public is to have trust and
confidence in EPA's decisions. EPA adheres to these Principles
of Scientific Integrity listed below.
EPA employees, whatever their grade level, job or duties must:
Ensure that their work is of the highest integrity.This means
that their work is to be performed objectively, without
predetermined outcomes using the most appropriate
techniques. Employees are responsible and accountable
for the integrity and validity of their own work. Fabrication
or falsification of work results are direct assaults on the
integrity of EPA and will not be tolerated.
Represent their work fairly and accurately. When representing
the work of others, employees must seek to understand the
results and the implication of the work and also represent it
fairly and accurately.
Represent and acknowledge the intellectual contributions of
others in representing their work to others or in published
writings such as journal articles or technical reports.To do
otherwise is plagiarism. Employees should also refrain from
taking credit for work with which they were not materially
involved.
Avoid financial conflicts of interest and ensure impartiality
in the performance of their duties by respecting and adhering
to the principles of ethical conduct and implementing
standards contained in Standards of Ethical Conduct for
Employees of the Executive Branch and in supplemental
Agency regulations.
Be cognizant of and understand the specific programmatic
statutes that guide the employee's work.
Accept the affirmative responsibility to report any breach of
these principles.
Welcome differing views and opinions on scientific and
technical matters as a legitimate and necessary part of
the process to provide the best possible information to
regulatory and policy decision-makers.
-------
Scientific
Integrity
Public
Affairs
Environmental
Statutes
Ethics
Peer
Review and
Advisory
Committees
Data
Access
Figure 2
Scientific
Integrity
at EPA
EPA's Scientific Integrity Policy
EPA's Scientific Integrity Policy outlines five specific goals to ensure scientific
integrity throughout the Agency. Engaging in these activities strengthens the
credentials of Agency scientists and the credibility of future Agency research.
1. EPA environmental policies, decisions, guidance and regulations
grounded in robust, high quality science.
2.Transparency within Agency scientific processes.
3. Scientific research and results communicated openly and with integrity,
accuracy and timeliness.
4. Appropriate use of peer review and federal advisory committees.
5. Professional development of the Agency's scientists, engineers and other
technical staff.
To address these goals, the Policy is divided into four sections:
1. Promoting a culture of scientific integrity at EPA.
2. Releasing information to the public.
3. Peer review and the use of federal advisory committees.
4. Professional development of government scientists and engineers.
Promoting a Culture of Scientific Integrity at EPA
This section of the Policy establishes an expectation that scientific research is
generated in a timely manner, characterized appropriately for Agency policy-
-------
making and communicated clearly to the public. Honesty, transparency and
a commitment to producing high-quality scientific data are key components
of a culture of scientific integrity. All EPA employees, contractors, grantees
and volunteers are explicitly forbidden to suppress, manipulate or otherwise
alter scientific data.This assures that EPA decisions are informed by the
best science the Agency, its contractors, grantees and collaborators have to
offer. A culture of scientific integrity is also one that protects employees who
report suspected allegations of misconduct. Similarly, employees who express
differing scientific opinions should neither fear nor experience retaliation.
Releasing Information to the Public
The Policy aims to foster a culture of transparency regarding the results of
research, scientific activities and technical findings. EPA encourages open
communication, free from political or other interference. Communication
activities may include news releases, Web postings, responses to
Congressional inquiries and publication in peer-reviewed journals.The
clear and timely release of science facilitates a free flow of information and
increases public confidence in the Agency.
"Science must be the compass guiding our
environmental protection decisions....While
the laws that EPA implements leave room
for policy judgments, the scientific findings
on which these judgments are based should
be arrived at independently using well-
established scientific methods, including
peer review, to assure rigor, accuracy, and
impartiality. "
Lisa Jackson May 9th, 2009
Peer Review and the Use of Federal Advisory Committees
Independent peer review is a necessary component of quality control in science
and a crucial aspect of scientific integrity. EPA's review process is outlined in
the Agency Peer Review Handbook5 which is currently under revision. External
Federal Advisory Committees offer additional opportunities for review of
scientific activities and provide additional scientific expertise.
Professional Development of Government Scientists and Engineers
EPA employees are encouraged to participate in professional development
activities to fully engage with their scientific communities and become leaders
in their fields. Professional development activities may include presenting at
scientific meetings or conferences, participating in professional societies, or
serving on editorial boards of peer-reviewed journals.
-------
Ongoing Scientific Integrity Activities
Several of EPA's scientific integrity activities were first implemented prior to
FY2014.These activities provide ongoing support for the evolving scientific
integrity activities at EPA.These annual activities are illustrated in Figure 3
and described in this section.
The Scientific Integrity Committee
The Scientific Integrity Policy established a Scientific Integrity Committee,
chaired by the Scientific Integrity Official.The Committee consists of senior
program office and regional officials who are designated Deputy Scientific
Integrity Officials (DSclOs).They provide leadership forthe Agency on
scientific integrity, jointly assist in the implementation of the Policy and
promote Agency compliance with the Policy.The Committee meets quarterly.
The ScIO communicates regularly with Committee members to discuss
approaches to emerging issues and work together to resolve allegations.The
Committee's participation ensures that a variety of experiences and viewpoints
are considered.The members are listed in a table located inside the back cover
with their email addresses.
The Annual Report on Scientific Integrity
The ScIO, with input from the DSclOs, is responsible for generating an annual
report to the Science Advisor on the status of scientific integrity at the Agency.
This report is also publicly available online6 and in print form.The report
highlights successes, identifies areas for improvement and includes plans for
addressing critical weaknesses if any are reported. For the first time in 2014,
the annual report used information gathered under the auspices of the Federal
Managers Financial Integrity Act.
Annual Meeting/Conversation with the ScIO
The Annual Meeting/Conversation with the ScIO provides an opportunity for
EPA employees to learn about scientific integrity at EPA and ask questions.
The ScIO, Dr. Francesca Grifo, presented to a live audience at Headquarters
and to the rest of the Agency through a well-attended webinar in July.The
conversation improved the visibility of the Scientific Integrity Policy and
increased awareness among EPA employees and other stakeholder groups.
The sessions emphasized the broad applications of the Policy across EPA and
encouraged employees to recognize and bring forward any concerns they might
have. A summary of this meeting is included in Appendix 2.
-------
Scientific Integrity Annual Calendar
January
Committee
Meeting
Union Working
Group Meeting
November
Scientific
Integrity
Cross-Agency
Implementation
Process
September
FMFIA
Committee
Meeting
Union Working
Group Meeting
Committee
Meeting
Union Working
Group Meeting
Annual
Stakeholder
Meetings
Committee
Meeting
Union Working
Group Meeting
Figure 3 The Annual Cycle of Ongoing Scientific Integrity Activities
Scientific IntegrityTraining
An initial training on the Scientific Integrity Policy was deployed in November
2013. The training was mandatory for supervisors and other employees
designated by each Deputy Scientific Integrity Official. More than 5,700
employees, one-third of all employees, have taken the online course. All 24
members of the Scientific Integrity Committee have certified that the designated
employees in their program, office or region have taken the training.The training
module is also an ongoing resource for current and new EPA employees to learn
about scientific integrity and its applicability to EPA activities.7
Strengthening Contractor-Managed Peer Review
In FY2013, EPA strengthened the Agency's oversight of contractor-led peer
review panels by developing a new Conflict-of-interest Review Process for
Contractor-Managed Peer Reviews.8 The new process includes two new
opportunities for public involvement for identifying and selecting panel
members. The new process is designed to enhance the transparency of
contractor-led peer reviews, increase internal oversight of these peer reviews
and reduce the potential for organizational or personal conflict-of-interest
concerns through greater public participation and more rigorous internal
-------
10-11
review.The process was used to improve three contractor-managed peer
reviews in 2014.
Quarterly Coordination Meetings with the Office of Inspector General
The Scientific Integrity Official (ScIO) and the Office of Inspector General
(OIG) have maintained regular communication through quarterly meetings
in which they discuss the status of current allegations of a loss of scientific
integrity under review and anticipated courses of action.9 Coordination
between the two offices exemplifies the Agency-wide nature of the Scientific
Integrity Policy implementation.
The handling of scientific misconduct, which includes fabrication, falsification,
plagiarism or misrepresentation in proposing, performing or reviewing scientific
or research activities, is governed by EPA's Scientific Misconduct Policy10 and
overseen by the OIG. In 2014, five allegations were received through the OIG
hotline and referred to the ScIO and three allegations of scientific misconduct
came to the ScIO and were referred to the OIG.
New Scientific Integrity Initiatives
New work in 2014 focused on developing procedures to implement the Scientific
Integrity Policy; initiating an evaluation of the content, implementation and
impact of the Policy; consulting with the EPA unions and other outreach; and
updating the EPA Peer Review Handbook.
Procedures for Reporting and Resolving Allegations of a Loss of
Scientific Integrity
There are no formal processes for receiving or resolving allegations included
in the Policy. In 2014, the Scientific Integrity Committee developed draft
procedures for reporting and resolving allegations of lapses of scientific
integrity at the Agency.The goal was to create a procedure that honored
transparency, confidentiality, consistency, timeliness and fairness. Figure 4
illustrates the resulting draft procedures.The following is a summary of the
draft procedures that are currently being used and finalized with the input of
the Scientific Integrity Committee.
Allegations of misconduct may be received by the Scientific Integrity Official,
any Deputy Scientific Integrity Official or the Office of Inspector General.
Once an allegation has been received, the process begins with the question:
Is there a reasonable basis to believe the allegation of a lapse in scientific
integrity has merit? In other words, if everything the complainant alleges were
found to be true, would the allegation be a lapse in scientific integrity or is it
a different kind of problem? If the allegation concerns waste, fraud, or abuse
or other criminal violations, the allegation would be referred to the OIG. If the
allegation involves reprisal, it would be referred to the OIG or the Office of
Special Counsel.11 If the allegation concerns a financial conflict of interest or
other ethics issue involving federal employees it would go to the appropriate
Deputy Ethics Official or Office of General Counsel/Ethics, or Human Subjects
Research Review Official, as appropriate. If the allegation has been submitted
previously and provides no new information, or if it has no merit, it would be
dismissed and all parties informed of this action.
-------
Evaluation
by the ScIO and another
DScIO to determine if
there's reasonable basis to
conclude the allegation has
merit as a Scientific
Integrity issue. Proceed
.. to inquiry phase. . '
Allegations of fraud, waste, and abuse or
criminal law violations go to EPA Office of
Inspector General.
Allegations of reprisal or retribution go to
Office of Inspector General or Office of Special
Council.
Allegations of financial conflicts of interest
or ethics involving federal employees go to
the appropriate ethics official.
If the allegation has no merit or was
previously investigated and no significant
new information is provided - the case is
dismissed.
Inquiry
by the ScIO and DScIO
working with supervisors
or other managers to
collect information from
all involved parties.
Review
of the science or other
information related to
the allegation by DSclOs
and ScIO to decide if
the allegation has been
substantiated.
If the allegation is not substantiated - the
case is dismissed.
If the allegation is substantiated, a review
team recommendation report is written and
administrative and or corrective action is
determined by the direct line supervisor or
other manager.
Figure 4 Procedures for Reporting and Resolving Allegations (Draft)
-------
12-13
If the claim is identified as a potential loss of scientific integrity, then the
inquiry phase begins and information is collected from the subject of the
allegation, the complainant and other relevant sources.The information is
reviewed by the ScIO, the relevant DScIO and a second DScIO who may have
expertise in the field of inquiry.The second question in the flow chart asks if the
allegations have been substantiated. If the evidence is insufficient, the case is
dismissed. One appeal is permitted for a dismissed case, if additional evidence
can be provided. If a case is substantiated, the ScIO is responsible for drafting
a case summary and recommendations for corrective actions to safeguard the
involved science. Other actions may be deemed necessary by the supervisor
or other manager.The Scientific Integrity Committee will check in periodically
to be sure the scientific recommendations are carried out. In addition, the
resolved allegations are summarized in the annual report and on the Internet
in a way that protects the identity of the parties involved. A summary of a case
closed in August 2014 is in Box 2. Additional information on allegations and
their disposition can be found in section 3.
FMFIA: The Federal Managers Financial Integrity Act
The Federal Managers Financial Integrity Act requires that federal agencies
assess the effectiveness of programmatic and financial internal controls. EPA
Assistant Administrators (AAs) and Regional Administrators (RAs) must
certify that their programs comply each year through an assurance letter to the
EPA Administrator, who delivers an overall statement of assurance to
the President and Congress. FY2014 marks the first year that AAs and RAs
were required to submit an attachment certifying internal controls for
scientific integrity.
The Office of the Science Advisor (OSA)12 worked closely with the Office
of the Chief Financial Officer (OCFO) to develop a process for management
certification of compliance with the Scientific Integrity Policy. Based on
the requirements outlined in the Scientific Integrity Policy, the template
probed for accomplishments, potential weaknesses and overall progress in
implementing the Agency's Scientific Integrity Policy. Each office assessed the
status of scientific integrity in its processes for generating, managing, using,
communicating or otherwise working with scientific information.
The FY2014 FMFIA process provided a structured assessment of EPA's
scientific integrity activities across the Agency. Although several opportunities
for improvement emerged, senior managers did not identify any material
weaknesses in their offices' implementation activities and they identified a
number of successes.
Policy Evaluation
The Scientific Integrity Policy requires that the Scientific Integrity Committee
review the Policy every two years to ensure its effectiveness and adherence
with applicable rules and regulations. In 2014 the Committee began a formal
evaluation of the Policy including a systematic investigation of the merit, worth
and significance of the Agency's scientific integrity efforts.
-------
The evaluation, which will extend into FiscalYear
2015, will examine the content, implementation
and impacts of the Policy. It is designed to be
a practical, ongoing assessment that involves
the Scientific Integrity Committee and other
stakeholders.The evaluation process will identify
criteria to assess performance, standards
that must be reached to consider the program
successful, and evidence needed to indicate
performance relative to the standards. A logic
model will synthesize the main program elements
into a picture of how the program is supposed to
work and make explicit the sequence of events
that are presumed to bring about change (Figure
6). Logic models articulate available resources,
activities, outputs and outcomes, including
changes in awareness, behavior and condition.
The scientific integrity logic model is illustrated
in Figures 5, 6 and 7.
ase Summary
Case Number: EPA-00005
Report Issued: Aug 15, 2014
Allegation: The complainant, a
laboratory equipment provider, alleged
that an EPA employee with a conflict of
interest sabotaged approval of a new
analytical method.
Summary: The charges were found to
lack merit and were not substantiated.
The EPA employee did not have a conflict
of interest with competitors of the
complainant, nor was the employee in a
position to make decisions for EPA to
approve or disapprove of the analytical
method in question.The review of the
method was done in accordance with
applicable laws, regulations, policies,
procedures and guidance.The Scientific
Integrity Official concluded that the
allegation had no basis and it was,
therefore, dismissed.
-------
14-15
Content
Implementation
Impacts
Figure 5 The Evaluation of the Scientific Integrity Policy
Resources I Activities I Outputs I Awareness I Behavior
Condition
Staff
Budget
Scientific
Integrity
Committee
Write
additional
policies
needed to
fully
implement
the Scientific
Integrity
Policy
\
t Employee
procedures
r Reporting I
d Resolving I
{/legations
I
1 Increased
knowledge of:
Who to
consult and
how to
report an
allegation
EPA community
recognizes
potential
scientific integrity
problems when
they see them
Allegations are
reported and
resolved
Awareness of
scientific integrity
prevents
non-compliance 1
with the Scientific
Integrity Policy
Issues are
prevented or
recognized and
resolved early
Compliance with
Scientific
Integrity Policy
requirements
Work of EPA is based
on robust science
There is transparency
within scientific
processes and open,
accurate, timely
communication of
science
There is public trust in
Agency science
Scientists are able to
do their best work
Figure 6 An Example of One Logic Model Activity
-------
Work of EPA is based
on robust science
Work of EPA is based on robust science
Scientific Integrity ensures:
The validity of scientific information
Objectivity
A commitment to evidence
Credibility and quality of Agency science
Scientific studies undergo independent peer review
There is transparency
within scientific
processes and open,
accurate, timely
communication of
science
Transparency
Free flow of Agency scientific information
Scientific findings are generated, reviewed, and disseminated in a timely and
transparent manner
There is public trust in
Agency science
Public trust in Agency science
Increased public understanding and appreciation of EPA's scientific work
Scientists are able to
do their best work
Scientists are able to do their best work
Robust culture of dissent
Scientists have confidence in the processes of reporting and resolving allegations
Professional development
Figure 7 Logic Model Conditions EPA is Striving to Achieve
The Evaluation will also identify barriers to full implementation, document
success in accomplishing objectives, demonstrate accountability, gather
success stories, and support organizational change and improvement. It will
also be a valuable tool in annual planning and setting priorities.
Logic model columns, such as those in Figure 6, are linked by a series of if-then
questions.The columns are connected by asking, "If an activity is completed,
then will the outcome in the next column happen?" For example if the
procedures are written, then will EPA employees have increased knowledge? If
employees have that increased knowledge, then would the behaviors take place?
And if those behaviors take place, then will there be a change in condition?
Metrics will allow measurement of successes at each of the column junctures.
This kind of analysis will reveal where additional investments would need to
be made to achieve the changes in condition the Policy is designed to achieve.
Figure 7 lists the conditions the Agency is trying to achieve.These, like the
behaviors listed in the logic model, are taken from the Scientific Integrity Policy.
Peer Review Handbook
The Peer Review Handbook provides guidance for EPA employees and
managers planning and conducting peer review.The third edition of the
Peer Review Handbook has been the Agency-wide resource for peer review
since 2006. EPA's Peer Review Advisory Group (PRAG) is currently working
-------
16-17
on a fourth edition of the Peer Review Handbook.The draft document is in
the Agency-wide review process and is being prepared for final review and
approval by the Science Advisor.
Scientific Integrity Outreach
Outreach on scientific integrity is key to the full implementation of the
Policy. In 2014, outreach consisted of 23 presentations to four regions,
several offices, union representatives and external stakeholders. Last year's
annual report identified the construction of an EPA Web portal on scientific
integrity as a critical priority. OSA has launched the redesign of the scientific
integrity Internet site for deployment in FY2015. In the interim, background
documents are currently hosted on ORD's website.13 The scientific integrity
Web portal will provide a comprehensive resource for policies and documents
related to scientific integrity at EPA.The Web portal plans also include a
comment box, which will allow visitors to submit questions or suggestions
anonymously. Concerned employees or other stakeholders who prefer to
remain unidentified also will be able to use this feature to report allegations
of a loss of scientific integrity.
Quarterly Meetings of the Union Working Group
The Scientific Integrity Official established a work group composed of
representatives of EPA employee unions.Their meetings roughly coincide
in timing and content with the Scientific Integrity Committee's quarterly
meetings. Agency labor union representatives are able to comment on
scientific integrity issues and discuss recommendations directly with the SclO.
Ethics Training Module
The professional development section of the Policy does not make reference
to the many ethical considerations that must be a part of seeking permission,
for example, to present a paper, hold office in a scientific society or choose to
undertake an outside activity.The SclO worked with the Senior Ethics Counsel
to develop a training that will help scientists and their supervisors navigate
these tricky situations.This module is designed to be a resource with easy
access to various topics without having to take the entire training. It includes
case studies and examples.The training module is expected to be offered to
staff online in FY2015.
t
cn
-------
Scientific Integrity Accomplishments
In 2014, EPA program and regional offices took many approaches to enhance a
culture of scientific integrity at EPA. Research-focused offices made advances
in the public release of large data sets, dashboards and other mechanisms that
lead to greater transparency and accessibility of Agency science. Other offices
have revised procedures to assure the quality of Agency science, such as those
for reviewing and approving scientific products and for conducting peer review.
The following are examples of scientific integrity accomplishments across the
Agency in FY2014.
Promoting a Culture of Scientific Integrity
A culture of scientific integrity promotes the quality, collection, processing
and communication of scientific information. Many quality assurance systems
are already in place to ensure the integrity of the scientific research process.
In FY2014, several new initiatives were introduced to demonstrate EPA's
commitment to evidence, objectivity and the quality of scientific information.
Data Management
The Office of Research and Development (ORD) uses a Research Management
System (RMS) to provide EPA and its stakeholders with access to up-to-date
information on ORD research.
Clearance Procedures
Clearance procedures increase transparency in the release of research results,
ensuring timely review and discouraging unreasonable delays.They also
ensure that scientific products are reviewed by the appropriate supervisors and
technical managers before being released to the public. Several regional and
program offices have developed their own clearance procedures for scientific
research, as the Scientific Integrity Committee develops a framework for
Agency-wide clearance procedure guidelines.
ORD uses the Scientific andTechnical Information Clearance System
(STICS) to verify that scientific and technical products are properly
reviewed before their release.
Region 1's Scientific Product Clearance Review Procedures require that
external subject matter experts review science products in addition to the
formal peer review process.
Region 6's Standard Operating Procedure (SOP) A-10: Procedures
for Review and Approval ofTechnical Presentations or Publications
establishes procedures for internal approval of technical products to
ensure the consistency and accuracy of publicly released documents.
Region 9 has developed guidelines for scientists seeking to publish their
work or engage in publication review.
Quality Assurance
A variety of mechanisms work to ensure the quality and integrity of EPA
scientific products, in addition to those mentioned above. Quality Management
Programs (QMPs) play a large role in the quality assurance of scientific
information in offices. Collectively, these programs contribute to a culture that
emphasizes the validity of scientific information.
-------
18-19
The Office of Environmental Information (OEI) develops Agency-wide
policies to manage environmental data and quality-related activities for
EPA science.
Region 3 has chartered an internal workgroup, the Regional Field Advisory
Council (RFAC), to develop and implement consistent guidelines for field
operations when collecting, analyzing and reporting data.
Region 9 is performingTechnical Systems Audits (TSAs) of state and
tribal monitoring programs to ensure the integrity of scientific data used
for regulatory decisions.
Region 10's Science Steering Council (RSSC) undertookan initiative
to evaluate regional priority science needs and provide staff and
management with training and resources to address the needs.
Release of Information to the Public
EPA encourages the transparency of Agency activities through
communications tools such as online blogs, newsletters, news releases and
official publications. EPA also maintains several online databases to provide
open access to Agency information. Special user interfaces allow the public
to navigate EPA databases easily. Online tools such as dashboards and
calculators allow users to access a variety of datasets, input their own data
and model personalized scenarios.
The Office of Research and Development (ORD) launched a beta version
of the interactive Chemical Safety for Sustainability (iCSS) dashboard,
which provides public access to the toxicity screening information of over
10,000 chemicals.
ORD released the ExposureToolbox, "EPA-Expo-Box,"14 which includes
guidance documents to help the public assess chemical exposure risks.
ORD updated the Chemical and Product Categories (CPCat)15 database,
which provides public access to chemical exposure information for over
43,000 chemicals used in consumer products.
ORD created a National Stormwater Calculator (SWC)16 desktop tool to
help users model and manage stormwater runoff on their property.The
Calculator accesses geographic data to project stormwater retention
values under various scenarios.
ORD is updating the Integrated Climate and Land Use Scenarios tool
(ICLUS)17to include datasets that will facilitate climate adaptation
assessments and planning at local, regional and national scales.
ORD completed a pilot simulation to incorporate real-time data in
EPANET18 software. EPANET allows water utilities to manage the
consequences of contamination events by producing an automated
simulation of the water distribution system.
ORD created two tools, the Community-Focused Exposure and Risk
ScreeningTool (C-FERST) and theTribal-Focused Environmental Risk
and SustainabilityTool (T-FERST) to help communities understand public
health risks when making decisions.
ORD publicly released EnviroAtlas,19 a new interactive Web-based tool
that combines maps, graphs and other analysis tools to help the public
make well-informed natural resources planning and policy decisions.
The Office of Air and Radiation (OAR) launched a revised Power Plant
-------
EmissionTracking website with improved public access to current data on
power sector emissions and released the third edition of EPA's Climate
Change Indicators in the United States.
Office of Chemical Safety and Pollution Prevention (OCSPP) released
the Aquatic Life Benchmarks tool20 to provide consistent data for
stakeholders interested in aquatic ecosystem protection.The tool contains
information from the ecological risk assessments of over 384 chemicals.
EPA's Particulate Matter (PM) Designation Mapping Tool21 enabled the
public to access the data used for air quality decision-making.
Region 6 piloted a SharePoint site to facilitate the sharing of documents
between EPA and state officials managing public water systems.The
PublicWater System Supervision (PWSS) SharePoint site encourages
communication and collaboration between Region 6 and state drinking
water staff.
The Office of Enforcement and Compliance Assurance (OECA) continued
to publish a monthly Criminal Enforcement Case Bulletin to publicize its
investigations of noncompliance with national environmental laws.
Peer Review and Federal Advisory Committees
Scientific integrity ensures the quality of scientific and technical products
by promoting adherence to proper scientific procedures. In FY2014, EPA
continued its efforts to promote peer review as an essential component of
quality scientific research products.
EPA submitted an annual report to the White House Office of Management
and Budget (OMB), showing that EPA completed proper peer review
reports for Influential Scientific Information (ISI) and Highly Influential
Scientific Assessments (HISAs) in FY2013.
ORD reviewed the Science Inventory, a comprehensive catalogue of EPA
research, to ensure that all information was up to date, accurate, well-
documented and defensible.
-------
20-21
Professional Development
EPA encourages professional development activities so that EPA's scientists
and engineers can maintain their expertise, be active members of their
scientific communities and become leaders in their fields.Training activities
may include online courses, webinars, in-person workshops or conferences.
EPA provides several professional development opportunities for employees
and encourages their participation in professional societies.
ORD created Risk AssessmentTraining and Experience (RATE) Program
modules to improve Agency training in hazard identification, dose-
response assessment, exposure assessment and risk characterization.
Region 1's Laboratory Branch Chief led a national webinar on Ethics
and Data Integrity for Environmental Labs for labs accredited under the
National Environmental Laboratory Accreditation Program (NELAP).The
webinar was attended by over 900 people.
Region 8's training officer collaborated with their Scientific Integrity
workgroup to conduct a training needs assessment.The Human Resources
Program will use the assessment to allocate funds for training activities.
Region 10's Scientific/Technical Scholarship Program provided over
$15,000 in FY2014 for Region 10 employees to attend university classes,
conferences or other professional development opportunities.
EPA employees were recognized by multiple organizations in FY2014,
including Research! America, the Partnership for Public Service and the
White House.
Region 3's Air Protection Division hosted a two-day Air Monitoring Quality
Assurance workshop in Philadelphia.
EPA co-sponsored the National Environmental Monitoring Conference
(NEMC) under a cooperative agreement with the NELAC Institute (TNI).
-------
-------
22-23
3. Opportunities ,
Tor Improvement
EPA has made great strides in 2014 in scientific integrity. Looking forward
to fiscal year 2015 and beyond, six priority issues present opportunities for
improvement:
Reporting and resolving allegations of a loss of scientific integrity.
Reducing confusion about designation of publication authorship.
Increasing transparency.
Addressing constraints to full implementation of the Policy.
Defining the timely release of agency science and scientific products.
Enhancing peer review.
Agency investments in these activities ensure the credibility of, and maintain
the public trust in, Agency science.
Reporting and Resolving Allegations ot a Loss ot
Scientific Integrity
EPA has received 43 allegations of a loss of scientific integrity since the
release of the Scientific Integrity Policy and 40 in fiscal year 2014 (Figure 8). It
is important to remember that the allegations represent less than 0.3% of EPA
employees and an even smaller percentage if one includes the contractors,
grantees and volunteers who are also covered by the Policy.
0)
o;
C
.O
ns
en
CD
-Q
£
3
04
2073
Q1
2014
Date
Q2
2014
Q3
2014
Q4
2014
Figure 8 Total Allegations Received by FY Quarter/Year (as of 9/30/14)
-------
Formal allegations can be reported officially, revealing the identity of the person
making the allegation.These currently go through the draft processes outlined
on pages 10-11 for reporting and resolving allegations.There are also instances
where complainants do not want to reveal their identity to anyone except the
Scientific Integrity Official.The ScIO is still interested in obtaining information
about these allegations and can take some steps to resolve them.These are
informal allegations. In FY2014, of the 40 allegations received, 17 were made
formally and 23 were made informally (Figure 9).
Formal
Informal
External
Internal
Figure 9 Informal and Formal Allegations Received in Fiscal Year 2014
Of the 23 informal reports, only one came from outside the Agency (Figure 9).
Fourteen came from EPA offices and programs and eight came from regional
offices. Of the 17 formal allegations in FY2014, 11 came from outside the Agency,
three from EPA offices and programs and three from regional offices.
Allegations by Quarter, FY14
Internal Formal
Internal Informal
External Informal
External Formal
Figure 10 Allegations by Quarter, FY14
-------
24-25
The allegations relate to many different areas of the Scientific Integrity Policy.
For example, 13 allegations concern suppression or delay, nine relate to peer
review and five concern authorship disputes, including allegations of plagiarism.
Of the 17 formal allegations received in FY2014, three are closed, three are
being investigated by the Inspector General, one is being resolved in the
affected region and the other 10 are in the inquiry phase.
The increase in the reporting of allegations in fiscal year 2014 coincides with
both the arrival of the Agency's first full-time Scientific Integrity Official and
significant increases in outreach about the Policy and hence is probably not as
notable as it might seem. Allegations from outside the Agency have dropped
while those from inside the Agency have increased (Figure 10).This is a
positive development that may indicate that the Agency is resolving concerns
before they are shared with outside parties.The number of informal allegations
is larger than the number of formal allegations and this disparity has grown
recently. Increasing outreach could reduce the hesitancy of these employees to
report issues formally. It is important to continue to watch these trends closely.
Reducing Confusion about Designating
Publication Authorship
Many Agency publications have specific individuals designated as authors.
While the majority of formal scientific publications come from ORD, offices
and regions produce many other kinds of authored scientific products. Five
allegations concerning authorship designation were reported in FY2014. For
example, they involve confusion about when a supervisor is an author and when
the contributions of contractors are recognized with authorship. Four labs in
ORDNational Center for Environmental Assessment, National Exposure
Research Laboratory, National Health and Environmental Effects Research
Laboratory and National Risk Management Research Laboratoryhave
authorship guidelines that address these and other issues.These guidelines,
along with authorship rules from major journals, will form the basis of a series
of best practices for designating authorship that the Committee will release in
FY2015.
"The Administration is committed to
ensuring that, to the greatest extent and
with the fewest constraints possible and
consistent with law and the objectives set
out below, the direct results of federally
funded scientific research are made
available to and useful for the public,
industry, and the scientific community.
Such results include peer-reviewed
publications and digital data."
John Holdren February 2013
-------
Increasing Transparency
Media Access to EPA Scientists
In April 2014, the Scientific Integrity Official met with a group of external
stakeholders to discuss scientific integrity at EPA.The stakeholders expressed
concern about barriers to media access to EPA scientists and engineers.
Media access to Agency scientists has been the subject of several letters
to the Agency by coalitions of groups espousing increased transparency in
government.22 In addition, four members of the media, nine scientists and two
staff members have brought the issue of access both by the media to scientists
and by scientists to the media to the attention of the SclO. While designating
spokespeople for Agency policy issues is necessary for clear and consistent
communications, it is vital that EPA scientists have both training and
permission to communicate their research results.The role of EPA scientists
as translators of research is ever more important to reporters being able to
write well about EPA's research accomplishments and to enhancing public
trust in the Agency.
The Office of Public Affairs has increased their existing efforts to ensure
timely and accurate responses to press inquiries.They have added five new
staff to increase their responsiveness. For example, the Office of Research and
Development's records indicate an increase in the numbers of interviews and
the ability of EPA to schedule them within reporter's deadlines.
Decision Making
Agency scientists have raised concerns that they are sometimes unsure about
how the work they do is reflected in Agency decisions. It will be important for
the SclO to learn more about these concerns and, if merited, look at ways to
ascertain if this is a matter of scientists and others not knowing where to look
or if the Agency can do better at being more transparent.
Public Access
In response to a memorandum from the Office of Science andTechnology
Policy to the heads of executive departments and agencies, the Agency
submitted a plan to increase public access to its federally funded scientific
research.23 EPA's plan for public access to scientific research data will begin
with a pilot program focused on our intramural research.This pilot will inform
the design of the requirements that could be implemented more broadly across
the Agency's intramural research program and extramural research programs.
Addressing Constraints to Full Implementation
of the Policy
Maintaining Scientific Expertise in a Time of Reduced Workforce
Scientists at EPA use technologies and practices to make key contributions
to the Agency's programs.Their analytical work is central to environmental
protection and partnerships with states, tribal nations and local governments.
The Agency has recently gone through a reduction in the size of its scientific
staff. While challenging, this in and of itself, is not a scientific integrity issue;
however, if such reductions lead to changes in process or practice that make
scientific conclusions more vulnerable to a loss of scientific integrity, then this
could become an issue of concern to the SclO.
-------
26-27
Outreach and Training
Promoting a culture of scientific integrity at EPA requires tremendous
outreach to all parts of the Agency.The outreach efforts in 2014 represent a
good start, with over one-third of the Agency employees taking the training
on the Scientific Integrity Policy, many employees participating in the annual
Conversation with the ScIO webinar, and the ScIO giving 23 presentations
on scientific integrity. Additional work is required to ensure that all EPA
employees, contractors, grantees and volunteers are familiar with the Policy,
understand that it applies to them and know what to do if they experience
something that they feel may compromise the integrity of EPA science.
Defining theTimely Release of Agency Science and
Scientific Products
The Scientific Integrity Policy mentions timeliness 11 times as it relates to
the release of scientific work, findings, conclusions, information, the data
underlying Agency decisions, and scientific activities, research and results.
The Inspector General has asked the Scientific Integrity Committee to
define "timeliness" as it is used in the Policy.The Committee is working to
develop a definition that would "ensure scientific findings are generated and
disseminated in a timely and transparent manner..."24 as the Policy requires.
To support a culture of scientific integrity
within the Agency, this policy:
- Prohibits all EPA employees, including
scientists, managers, and other Agency
leadership, from suppressing, altering,
or otherwise impeding the timely release
of scientific findings or conclusions.
Scientific Integrity Policy Section IV, Part A
Enhancing Peer Review
The purpose of the EPA Peer Review Handbook is to ensure that peer review is
used consistently and appropriately across the Agency. Peer review is critical
to maintaining the quality of our scientific products.The release of the new
edition of the Peer Review Handbook and the accompanying training will help
to ensure the consistency of peer review of the science used by EPA.
-------
-------
28-29
4. Areas for. , ,
Fuiure Investment
The FY2015 work plan for scientific integrity includes:
1. Annual activities and requirements of the Policy,
2. Ongoing projects initiated in FY2014 (pages 8-9), and
3. New initiatives (pages 10-16)
Annual Activities and Requirements
The annual requirements of the Scientific Integrity Policy include the annual
report, annual meeting and other webinars, annual stakeholder meetings, the
use of the Federal Managers Financial Integrity Act process to request annual
certification of compliance with the Policy, Scientific Integrity Committee and
Union Working Group quarterly meetings and coordination of our work with the
Inspector General.
Ongoing Projects
The Evaluation of the Scientific Integrity Policy will continue with the
collection of data on our progress through a survey of Agency personnel. We
will continue to assist with the oversight of contractor-managed peer review.
We will finalize the procedures for receiving and resolving allegations as well
as continue to receive them and resolve those we have. We will release best
practices for designating publication authorship as described on page 24.
New Initiatives
Transparency and the Public Trust
/. Agency Framework for Clearance Procedures
The Scientific Integrity Policy lists the following among the roles and
responsibilities of the Scientific Integrity Committee:
Develop an Agency framework for clearance procedures for scientific
products as guidance for program offices and regions.
Evaluation of program offices' and regions' clearance procedures
for scientific products and make recommendations as appropriate to
promote standardization across the Agency.
In FY 2014, the Scientific Integrity Committee will work on creating an Agency
framework for clearance procedures.This entails identifying the procedures
currently in place and sharing best practices.
2. Transparency in Decisions Informed by Science
The Scientific Integrity Official will learn more about these concerns
previously outlined, and if merited, look at ways to work with the Scientific
Integrity Committee to address them.
-------
3. Increasing Access to Agency Science and Scientists
We plan to continue to work with the Office of Public Affairs to increase
media access to scientists and engineers reporting research results.The
ScIO will explore options for communications and media training as well.
Nurturing a Culture of Robust Scientific Discourse
/. Differing Professional Opinion Policy
The Scientific Integrity Committee and the Union Working Group will
consult with other federal agencies that have policies that govern
dissenting or differing professional opinions. Following this, the Scientific
Integrity Committee will draft a policy for EPA. Care will be taken to
distinguish differing opinions on the interpretation of scientific results
from differing opinions on Agency policy decisions. The former is a
scientific integrity issue; the latter is not.
2. Professional Development Ethics Training
Professional development opportunities also can present ethics issues.
To assist scientists and their supervisors in navigating this, the ScIO and
the Counsel for Ethics will finish and release a training module and once
released, publicize its availability.
Extending EPA's Culture of Scientific Integrity
/. Outreach
The ScIO plans increased efforts to reach out with information on
scientific integrity to all employees in the EPA program and regional
offices and also outside the Agency. In addition to meetings, webinars
and discussions, the ScIO and Scientific Integrity Committee plan to
produce outreach products in FY2015. A particular focus of the outreach
will be those involved with the FMFIA annual review process, designated
federal officials managing EPA's advisory committees and special
government employees (SGEs) serving on those committees. EPA has
21 Federal Advisory Committees. Some consist of representatives of
various stakeholders, some are populated with scientists appointed as
special government employees and some have both.The Policy applies to
members appointed as SGEs.
2. Language for Contracts and Grants
The Scientific Integrity Policy applies to contractors and grantees.The ScIO
will work with those who write and manage these agreements to make sure
this is reflected in the agreements that govern the Agency's relationships
with grantees and contractors.
-------
30-31
5. Conclusions
Conclusions
Preparing this Annual Report has been a very important process of reflection
and strategic thinking as we look back on last year and look ahead to next year.
We want to emphasize that while this report raises many important topics,
the science at EPA is robust and ready to meet the task of guiding our work
towards protecting public health and the environment.
From the earliest formation of a scientific question to the application of those
results, scientific integrity creates protections for science from inappropriate
interference, manipulation and suppression.This assures that EPA decisions
are informed by the best science the Agency, its contractors, grantees and
collaborators have to offer.
The Scientific Integrity Committee will continue to work with the Senior
Counsel for Ethics, the Office of Inspector General and all of the other parts of
the Agency that together safeguard our science and earn the public trust in the
quality and integrity of our work every day. Scientific integrity gets us there.
-------
'
-------
32-33
Source
En
otes
Sources
This report's content was gathered from across EPA. Each of the program
offices and regional offices provided an assessment of scientific integrity
activities in their respective offices through the Federal Managers Financial
Integrity Act (FMFIA) annual review process.The ScIO supplied additional
information from efforts to resolve allegations of lapses in scientific integrity
in FY2014. Stakeholder meetings (summary in Appendix 1), the Annual
Meeting/Conversation with the ScIO (summary in Appendix 2) and the work of
the Scientific Integrity Committee provided additional valuable information.
End Notes
1 Ruckelshaus, William. Fishbowl Memo (1983): http://www2.epa.gov/aboutepa/
ruckelshaus-takes-steps-improve-flow-agency-information-fishbowl-policy
2 EPA's Principles of Scientific Integrity: http://www2.epa.gov/sites/
product ion/files/2014-11/documents/epa-pri nciples-of-scientific-integrity.pdf
3 Memorandum from the President for Heads of Executive Department
Agencies (2009): www.whitehouse.gov/the-press-office/memorandum-heads-
executive-departments-and-agencies-3-9-09
4 Memorandum from the Science Advisor for the Heads of Executive
Departments and Agencies (2010): www.whitehouse.gov/sites/default/files/
microsites/ostp/scient ific-integrity-memo-12172010.pdf
5 US EPA Peer Review Handbook, 3rd Edition (2006): http://www.epa.gov/
peerreview/pdfs/peer_review_handbook_2012.pdf
6 US EPA Annual Report on Scientific Integrity (2013): http://www2.epa.gov/
sites/product ion/files/2014-12/documents/2013_scientific_integrity_annual_
report.pdf
7 http://www2.epa.gov/programs-office-science-advisor-osa/training-epas-
scientific-integrity-policy
8 US EPA Annual Report on Scientific Integrity, Appendix E (2013): http://
www2.epa.gov/sites/product ion/files/2014-12/documents/2013_scientific_
integrity_annual_report.pdf
9 US EPA Annual Report on Scientific Integrity, Appendix D (2013): http://
www2.epa.gov/sites/product ion/files/2014-12/documents/2013_scientific_
integrity_annual_report.pdf
-------
10 EPA Order 3120.5, Policy and Procedures for Addressing Research
Misconduct, http://ori.hhs.gov/sites/default/files/epapolicy.pdf
11 http://www.osc.gov/
12 The ScIO and the Scientific Integrity Program Lead
are a part of the Office of the Science Advisor. Work on FMFIA was initiated
by the Office of the Science Advisor together with the ScIO.
13 http://www.epa.gov/research/htm/scientific-integrity.htm
14 http://epa.gov/risk/expobox/
15 http://actor.epa.gov/cpcat/faces/home.xhtml
16 http://www2.epa.gov/water-research/national-stormwater-calculator
17 http://www.epa.gov/ncea/global/iclus/
18 http://www.epa.gov/NRMRL/wswrd/dw/epanet.html
19 http://enviroatlas.epa.gov/enviroatlas/atlas.html
20 http://www.epa.gov/oppefed1/ecorisk_ders/aquatic_life_benchmark.htm
21 http://geoplatform2.epa.gov/pm_map/index.html
22 Union of Concerned Scientists, EPA SAB Letter (2014): http://www.ucsusa.
org/sites/default/files/legacy/assets/documents/center-for-science-and-
democracy/epa-sab-letter-8-12-14.pdf
23White House Office of Science andTechnology Policy, Increasing Access
to the Results of Federally Funded Scientific Research (2013): http://www.
whitehouse.gov/sites/def ault/files/microsites/ostp/ostp_public_access_
memo_2013.pdf
24 U.S. Environmental Protection Agency Scientific Integrity Policy (2012):
http://www2.epa.gov/sites/product ion/files/2014-02/documents/scientific_
integrity_policy_2012.pdf
-------
34-35
A
ppendix.1: Stakeholder
Meeting TXlotes
April 29, 2014
The annual stakeholder meetings are an opportunity for stakeholders to air
any concerns and ask questions.The following meeting notes reflect this and
do not include EPA responses.The Scientific Integrity Official responded to
these concerns and questions at the meeting, over the course of FY2014 by
phone and email, by addressing some of the larger issues in our work plan
for FY2014 and for FY2015 and by working with other parts of the Agency. For
example, the Agency has added additional public affairs staff to increase our
responsiveness to media inquiries. We have drafted procedures for reporting
and resolving allegations of a loss of scientific integrity, which address
questions about our handling of allegations. We are currently working on a
differing scientific opinions policy which will address those questions.These
notes capture a moment in time, while our work to enhance our culture of
scientific integrity is ongoing.
Attendees: Natural Resources Defense Council, Environmental Defense
Fund, Public Employees for Environmental Responsibility, Center for
Effective Government, Society for Environmental Journalism, Government
Accountability Project, Coalition for Sensible Safeguards, Union of Concerned
Scientists, Society for Conservation Biology, and Climate Science Watch.
Following a presentation by the ScIO, the following
topics were raised:
Transparency
The Policy states that the "public affairs staff...should attend interviews
with members of the media..." (IV B3) Is following this provision the
responsibility of the employee being interviewed or of the public affairs
staff? Can an EPA scientist be subjected to discipline for speaking with a
reporter outside the presence of public affairs staff? If not, where is that
stated?
Why can't a scientist speak to a reporter on background without going
through many layers of phone calls which result in consistently missed
deadlines?
Does EPA provide media training for its scientists?
How can or does EPA ensure the ability of its scientific and technical
experts to communicate freely about fracking environmental issues?
There are scientific integrity issues in EPA's communications about EPA's
Corexit dispersant policy.
Is EPA considering revising its policy regarding minders on calls with
scientists?
Transparency builds trust. What is EPA afraid of when it muzzles its
-------
scientists? Reporters need to speak to scientists to report well on the
good work EPA is doing.
Conflict of Interest/Peer Review
EPA should address conflict of interest disclosure and research
sponsorship issues in Integrated Risk Information System (IRIS)
systematic review procedures and in Science Advisory Board meetings of
the full Chemical Assessment Advisory Committee panel vs meetings of
the subpanels
EPA should mount a more vigorous response to accusations that receipt
of EPA funding constitutes a Clean Air Science Advisory Committee
membership conflict of interest.
Who determines if the contractors doing peer review have conflict of
interest?
EPA does not consider its IRIS panels to be Federal Advisory Committees,
despite the fact that these panels provide science-based human health
assessments to support the Agency's regulatory activities. Conflict
reviews are conducted by contractors and the Agency has taken the
position that these conflict reviews are not even subject to the Freedom of
Information Act.
Shouldn't the fully transparent treatment of the recruitment and review of
Federal Advisory Committee panels also apply to IRIS panels conducted
by contractors?
Implementation of the Scientific Integrity Policy
Who polices or enforces the prohibition that "policy makers shall not
knowingly misrepresent, exaggerate, or downplay areas of scientific
uncertainty associated with policy decisions"? (IV A3)
Does this violation require that the victim of the "managers and other
Agency leadership" file a complaint? How would these violations ever be
investigated?
The Policy references "EPA clearance procedures" (III and IV B 1) but does
not define them. Where are these procedures spelled out?
In a footnote (4), the Policy provides that "[t]he EPA Scientific Integrity
Committee will develop an Agency-wide framework for approval of
scientific communications." Has this been done? What is the function/
purpose of an Agency-wide framework?
Are allegations and their outcomes confidential? May the complainant
publicly disclose the contents of his/her complaint?
May complaints be filed anonymously? May they be filed by third parties,
such as a union?
Who investigates violations of the Policy (such as suppressing dissent or
preventing participation in a scientific society)?
By what mechanism are all the provisions of the Pol icy-other than the
prohibition against scientific misconduct-actually enforced?
Does the subject of the complaint have any due process rights, such as
being able to examine the evidence behind the allegation? If so, where are
those rights spelled out?
In allegations involving the chain-of-command, is there a process to wall
off members of the Scientific Integrity Committee from cases/allegations
-------
36-37
involving their direct superiors? What conflict rules apply to Scientific
Integrity Committee members?
The Policy "mandates the ScIO, with input from the Deputy SclOs, to
develop a transparent mechanism for Agency employees to express
differing scientific opinions." What is the status of this mechanism and
what does it look like?
When has EPA ever featured the dissenting or differing scientific views of
its staff scientists?
Working with the Inspector General
The OIG principally is staffed by investigators and auditors. In an
allegation involving highly technical matters, how is the OIG supposed to
handle matters beyond its expertise?
Has the OIG voluntarily assented to subject itself to the EPA Scientific
Integrity Policy?
Does EPA have a memorandum of agreement with the OIG governing how
scientific misconduct investigations will be conducted or is it entirely up to
the discretion of the OIG?
Appendix D of the Annual Report on Scientific Integrity for FiscalYear
2013 states that the OIG will report the status of a case after 30 days. Is
there no other timeline? What prevents an OIG from investigating a case
indefinitely?
The Policy cautions that its implementation is "subject to the availability
of appropriated funds." (Ill) Since OIG did not receive a supplemental
appropriation for these actions, does OIG have an incentive to assign
these misconduct cases a low priority?
May the ScIO withdraw a case from the OIG? If so, under what
circumstances?
Who investigates allegations of scientific misconduct involving the OIG?
In cases where "There is no further OIG interest in the allegation," does
the ScIO have any protocol for investigating such complaints or is it
completely ad hoc, perhaps varying from case to case?
Whistleblower Protections
By what mechanism are whistleblower protections extended to scientific
dissidents? Who exactly enforces these protections?
TheWhistleblower Protection Act (WPA) creates an affirmative defense
against a slate of named adverse personnel actions, such as demotions,
suspensions, etc. By contrast, the Policy references "retaliation or other
punitive actions." Is the zone of whistleblower protection under the Policy
broader than the WPA? For example, would the Policy protect an EPA
scientist whom a manager prevents from submitting an article to a peer-
reviewed journal as retaliation for the scientific content of the work? If so,
then specifically how?
By way of footnote 13, the Policy implies that its whistleblower protections
are limited to WPA. Why does EPA not include the whistleblower
protections of statutes which are implemented by EPA, such as the Clean
Air Act, Clean Water Act, Resource Conservation and Recovery Act, Safe
Drinking Water Act or Superfund?
-------
-------
38-39
Appendix 2: Annua Meeting
with ;he Sc O
DISCUSSION: July 17,2014
Attendees: Dr. FrancescaT. Grifo, U.S. EPA's ScIO (ScIO), held a webinar
conversation with EPA employees on July 17, 2014. Dr. Grifo gave a
presentation in which she explained the Agency's Scientific Integrity Policy
(SclP), shared current scientific integrity initiatives, and discussed future
scientific integrity plans. More than 140 participants attended the webinar. Dr.
Grifo addressed questions after her presentation.
Allegations of a Loss of Scientific Integrity
In response to questions, Dr. Grifo explained that:
Employees can report allegations of a loss of scientific integrity by
approaching their management chain or they can contact Dr. Grifo directly.
The SclO's office discusses concerns confidentially and reviews options
with scientists who are unsure whether there may be a scientific integrity
issue. Dr. Grifo noted that her office has a spreadsheet containing
allegations that have not been formally reported. She and senior EPA
officials want to know about allegations, even if the people making
allegations are reluctant to provide their names.
Regarding confidentiality for formal allegations, Dr. Grifo recognizes the
need for EPA staff to feel safe about contacting her with concerns. OGC is
working to assist Dr. Grifo in advancing her objectives of maintaining full
confidentiality.
The OIG would handle concerns about excessive work or unachievable
goals, if they involve waste, fraud or abuse. However, an employee with
such concerns may contact Dr. Grifo to discuss the matter so that she can
help determine the appropriate avenue for addressing the concern based
on specific facts.
Regarding concerns about the integrity of peer reviewed published studies
used by EPA in assessments or regulatory actions, Dr. Grifo would need to
be presented with specific examples to determine whether the issue was
one for her office or another entity to address. However, she encouraged
employees to bring this type of issue to her attention.
If an allegation against an employee were substantiated, the Scientific
Integrity Committee reviewers would not be the officials to recommend
disciplinary actions; this would be the purview of the employee's
supervisor or other manager.The procedures to address allegations
change none of the existing rights of employees.The Scientific Integrity
Committee review is strictly concerned with the science.
The decision on whether an allegation has been substantiated may be
appealed.The appeal would be reviewed by a different set of members of
the Scientific Integrity Committee than those who had originally reviewed
-------
the allegation.The members of the Scientific Integrity Committee, who are
senior officials designated by each office and region, are listed on the EPA
scientific integrity website.
EPA's Scientific Integrity Policy
In response to questions, Dr. Grifo explained that:
The logic model to evaluate the SclP is a series of if/then statements. If a
certain activity were followed, then a certain result would occur, producing
specified outcomes that in turn would have certain outcomes producing
specific behavioral changes and long-term changes in the Agency.The
draft logic model starts with the SclP, which defines the outcome EPA is
seeking vis-a-vis scientific integrity, and then works backward to activities
that will produce the end result; the model also analyzes current activities
in this way.
Implementation of the SclP will involve the OIG, the Office of Special
Counsel, the Senior Counsel for Ethics and others who Dr. Grifo had
mentioned. Dr. Grifo works closely with those mentioned and others. For
example, her office recently met with EPA quality assurance specialists
to understand how they can work together. Every few months, Dr. Grifo's
office meets with the OIG to coordinate with them. EPA has a conflict
resolution group that Dr. Grifo hopes to contact soon. She is reaching out
one-by-one to all of the relevant groups to ensure mutual understanding of
each other's functions and to promote coordination.
The Deputy Scientific Integrity Officials (DSclOs) communicate policies
and procedures regarding scientific integrity to regional management.
There is one DScIO in each program and EPA regional office.The Scientific
Integrity Committee members take that responsibility very seriously and
in all cases communicate regularly with their regional administrators. Dr.
Grifo requested that she be contacted if there are any instances of the
policies not being communicated to regional management. Dr. Grifo has
visited several regions and hopes to visit others this coming year.
Dr. Grifo's office is not responsible for the integrity of all of the science
at EPA.The SclO's office is responsible for the processes that ensure
the integrity of the science. For example, when scientists wish to publish
journal articles, they should go through their office's usual clearance
procedures.The SclO's office can examine specific issues, but the SclO's
office does not scrutinize a priori every piece of science in EPA.
Closing Remarks
Dr. Grifo thanked the participants for attending the webinar and said
that she looked forward to another webinar in the fall when she will
have a sense of the scientific integrity agenda for 2015. In the meantime,
participants can call or send an email message to her with any questions.
-------
38-39
M»,
-
-------
Scientific Integrity Committee
Contact Information
Office/Region
Scientific Integrity Official
: Scientific Integrity Program Lead
JOAR
JOARM
JOCFO
JOCSPP
JOECA
JOEI
JOGC
JOITA
JAO(OP)
JORD
JOSWER
low
: AO (SABSO, OFACMO, OEAEE,
JOCHP)
JOSA
j Region 1
j Region 2
| Region 3
j Region 4
j Region 5
| Region 6
| Region 7
| Region 8
j Region 9
j Region 10
! Official
Francesca Grifo
I Martha Otto
j Betsy Shaw
j DonnaVizian
j David Bloom
j David Dix
j Carol Rushin
j Ron Borsellino
j Carol Ann Siciliano
j Martin Dieu
JAI McGartland
: Robert Kavlock
j Barry Breen
j Mike Shapiro
: John Reeder
| Mary Greene
: Robert Maxfield
j Marie O'Shea
: John Forren
j Thomas Baugh
j Carole Braverman
:RickMcMillin
jCeciliaTapia
: DebThomas
j Eugenia McNaughton
j Joyce Kelly
Email
: Grifo.francesca@epa.gov
: Otto.martha@epa.gov
j Shaw.betsy@epa.gov
j Vizian.donna@epa.gov
j Bloom.david@epa.gov
j Dix.david@epa.gov
j Rushin.carol@epa.gov
j Borsellino.ron@epa.gov
j Siciliano.carolann@epa.gov
j Dieu.martin@epa.gov
j McGartland.al@epa.gov
j Kavlock.robert@epa.gov
j Breen.barry@epa.gov
j Shapiro.mike@epa.gov
: Reeder.john@epa.gov
| Greene.mary@epa.gov
j Maxfield.robert@epa.gov
j Oshea.marie@epa.gov
: Forren.john@epa.gov
j Baugh.thomasL@epa.gov
j Braverman.carole@epa.gov
| McMillin.rick@epa.gov
: Tapia.cecilia@epa.gov
: Thomas.debrah@epa.gov
j Mcnaughton.eugenia@epa.gov
j Kelly.joyce@epa.gov
-------
http.'Hwww.epa.govlscientificintegrity
------- |