Fiscal Year 2018
oicegrDiEy
AnnuaD [R@port
SEPA
www.epa.gov/scientificintegrity
-------
-------
Scientific Integrity at EPA
Annual Report
Fiscal Year 2018
-------
The U.S. Environmental Protection Agency is charged by Congress with
protecting the Nation's land, air, and water resources. Under a mandate of national
environmental laws, the Agency strives to formulate and implement actions leading to
a compatible balance between human activities and the ability of natural systems to
support and nurture life.To meet this mandate, EPA programs provide data and technical
support for solving environmental problems today and building a science knowledge base
necessary to manage our ecological resources wisely, understand how pollutants affect
our health, and prevent or reduce environmental risks in the future.
The EPA Scientific Integrity Official (ScIO) champions scientific integrity
throughout the Agency.The ScIO chairs the Scientific Integrity Committee (the
Committee) that is comprised of Deputy Scientific Integrity Officials (DSclOs) who
represent every EPA program office and region. Science serves as the backbone for
decision-making at EPA.The ability of the Agency to pursue its mission to protect human
health and the environment depends upon the integrity of the science on which it relies.
The full text of this report is available on EPA's website at:
Cover Design by Armando Valentino Chagolla
EPA Publication number 601 R2000I
Printed on 100% Postconsumer, Process Chlorine Free Recycled Paper
-------
r Contributors ,& ,
^ Acknowledgments
Acknowledgments
The contributors wish to acknowledge:
The Scientific Integrity Committee, for its role in championing Scientific Integrity
throughout the Agency assistance in gathering the information needed for this report,
and helpful review of an earlier draft.
Contributors
FrancescaT. Grifo
Francesca is the Scientific Integrity Official (ScIO) at the U.S. Environmental Protection
Agency. Previously, she was the Director of the Scientific Integrity Program at the Union
of Concerned Scientists. She holds a BA in biology from Smith College and a Ph.D. in
plant systematics from Cornell University.
Daniel D'Arcy
Daniel was an Oak Ridge Associated Universities contractor in the Office of the Science
Advisor. He holds a BS in meteorology and political science from Florida State University,
and an MS in public policy from the Georgia Institute ofTechnology.
Blake Neumann
Blake is an Oak Ridge Associated Universities contractor in the Office of the Science
Advisor. He holds a BS in geography from Western Kentucky University, an MS in
Environmental Science from SUNY-ESF and an MPA in environmental policy from
Syracuse University.
Vincent Cogliano
Vincent is the Deputy to the Scientific Integrity Official at the U.S. Environmental
Protection Agency. He was previously the director of EPA's Integrated Risk Information
System (IRIS).Vincent holds a BS in mathematics from the Catholic University of America,
an MS in mathematics from the University of Michigan, and a Ph.D. in operations research
from Cornell University.
Martha Otto
Martha is the Scientific Integrity Program Lead in the Office of the Science Advisor.
Martha has over 30 years of experience in hazardous waste site remediation, innovative
treatment technologies, and policy, regulation, and guidance development. She holds a BS
in biology and an MS in environmental science and engineering from Virginia Tech.
-------
^^^¦SSlllalBH
Contributors & Acknowledgements iii
List of Figures vii
Executive Summary I
Introduction 3
Scientific Integrity in 2018 3
What is Scientific Integrity? 4
EPA's Scientific Integrity Policy 5
Promoting a Culture of Scientific Integrity 5
Releasing Information to the Public 5
Peer Review and the Use of Federal Advisory Committees 5
Professional Development of Government Scientists & Engineers 5
Scientific Integrity in FY2018 6
FYI8 Scientific Integrity Program Initiatives 6
Best Practices for Clearance of Scientific Products at EPA 6
Management Dialogues on Scientific Integrity 6
Scientific Integrity Language for Grant Agreements 7
Scientific Integrity Language for Contracts 7
Detecting Plagiarized Text 7
Ongoing Scientific Integrity Activities 8
The Scientific Integrity Committee 8
Annual Employee Conversation wI the Scientific Integrity Official.... 8
New Employee OnboardingTraining 9
Quarterly Coordination Meetings with OIG and OGC 9
Additional Training 9
Contractor-Managed Peer Review 10
Web Analytics 10
-------
FMFIA: Certifying Compliance w/ the Scientific Integrity Policy 10
Allegations and Requests for Advice Thru the End of FY 18 I I
New Procedures for Allegations and Requests for Advice I I
Annual Update on Allegations and Advice 12
Alternative Dispute Resolution 16
Summary of Adjudicated Allegations in FY 18 16
Accomplishments Across EPA 19
Areas for Future Investment 20
IncreasingVisibility of Scientific Integrity 20
Outreach Materials and Publications 20
Best Practices of Scientific Integrity Handbook 20
Embracing and Modeling Scientific Integrity 20
Subject Matter Whiteboard Videos 21
Increasing Access to Scientific Integrity Expertise 21
Protecting and Maintaining EPA's Culture of Scientific Integrity 21
Differing Scientific Integrity Opinions (DSO) Policy 21
Electronic Clearance System 22
Assessing the Implementation of EPA's Scientific Integrity Policy 23
OIG Scientific Integrity Audit 23
GAO Scientific Integrity Engagement 23
Conclusions 24
Acronyms 26
Appendices 30
Appendix I: Annual Employee Conversation on Scientific Integrity 30
Appendix II: Accomplishments Across EPA 34
Appendix III: Members of the Scientific Integrity Committee 59
Appendix IV: Endnotes 60
-------
vi-vii
-------
Figure I. Foundations of Scientific Integrity 2
Figure 2.What is Scientific Integrity? 4
Figure 3. Flyer for the 2018 Annual Conversation with the ScIO 8
Figure 4. Monthly completion in FY 18 of Scientific Integrity Onboard Training 9
Figure 5. How to seek scientific integrity advice or report an allegation 12
Figure 6. Procedure for submitting and evaluating an allegation 12
Figure 7. Number of scientific integrity queries received by quarter 13
Figure 8. FY 12-17 vs FY 18 scientific integrity queries 14
Figure 9. FY 12-17 requests for advice versus FY 18 requests for advice 14
Figure 10. Comparison of all FY 12-17 allegations versus FY 18 allegations 15
Figure I I. Current status of all allegations and requests for advice received 15
Figure 12. Sources and subjects of FY 18 queries 16
Figure 13. Scientific Integrity at EPA Whiteboard Overview 22
-------
-------
Executive
Summary
This Annual Report chronicles the implementation of the EPA's Scientific Integrity
Policy (the Policy) in fiscal year 2018 (FY 18). Since February 2012, the Policy has
provided both a vision and a roadmap for ensuring scientific integrity at the Agency.The
Policy lists the components of a culture of scientific integrity and offers a framework
for ensuring Agency-wide participation in that culture. Although scientific integrity is
treated as a single issue in the Policy, maintaining scientific integrity requires investment
and collaboration from many parts of EPA.This report documents the investments
made across EPA in FY 18 and identifies areas of focus for future initiatives.
In FY20I8, the Scientific Integrity Program (the Program) introduced new and exciting
initiatives across the Agency that strengthened EPA's culture of scientific integrity.The
newly released Best Practices for Clearance of Scientific Products at EPA offers guidance
to program offices and regions for developing durable clearance procedures that will
ensure rigorous review and the timely release of information.The Program initiated
management dialogues on scientific integrity, through which EPA leaders are having
open conversations with the ScIO about their experiences in scientific integrity and
the role that they play in contributing to the Agency's culture of scientific integrity.
The Program developed language for future Agency grants and contracts to ensure
compliance with the Policy, further enhancing EPA's culture of scientific integrity.
Several initiatives that provide ongoing support for scientific integrity at EPA include
convening the Scientific Integrity Committee (the Committee) for quarterly meetings,
producing the annual report, holding the Annual Employee Conversation with the
Scientific Integrity Official, providing scientific integrity training, overseeing contractor-
led peer reviews, and coordinating with both the Office of Inspector General (OIG)
and the Office of General Counsel (OGC).
Scientific integrity remains an ongoing priority for EPA. While many scientific integrity
successes occurred in FY20I8, further progress must be made to fully ensure a robust
culture of scientific integrity at EPA.This annual report details several highlights from
the last year and looks forward to future areas for improvement. In FY20I8 and
beyond, three priority issues present opportunities for ongoing investment:
1. Increasing the visibility of scientific integrity at EPA
2. Embracing and modeling scientific integrity across the Agency
3. Protecting and maintaining EPA's culture of scientific integrity
Agency investments in these activities ensure the credibility of, and maintain the public
trust in, EPA science.The ScIO and the Committee will continue to work with the
Agency Science Advisor, the Senior Counsel for Ethics, the Office of Inspector General
(OIG), and the rest of the Agency to safeguard science and maintain public trust in the
quality and integrity of EPA's work.
-------
r ^
Firm Commitment
to Evidence
r 1
Science-Based
Management
Review
Science Not
Influenced by Policy
Implications
J
r ^
r ^
Transparency
^ -J
Free Flow of
Science
r "
Timeliness
r ^
Acknowledged
Collaborators
r ^
No Conflicts of
Interest
r 1
Peer Review
* 1
Qualified Scientists
Robust Scientific
Discourse
F 1
Honest
Investigation
J
r ^
Quality Assurance
L
Accredited Labs
and Facilities
^ -J
r 1
Independently
Validated Methods
-J
' 1
J
Figure I: Foundations of Scientific Integrity
Scientific integrity is the compass that guides EPA in its mission to protect human health
and the environment (Figure I). Scientific integrity ensures that the science that is
conducted and utilized at EPA is objective and of the highest quality. Scientific integrity
prevents conflicts of interest or policy implications from interfering with or influencing
scientific results. Scientific integrity encourages robust scientific discourse, welcomes
differing scientific opinions, and supports the professional development of staff. Scientific
integrity requires that others be acknowledged for their intellectual contributions.
Scientific integrity guarantees that science is communicated openly, transparently, and in
a timely manner.Together, each of these elements create a culture of scientific integrity
at EPA that inspires public trust in the Agency and ensures that EPA achieves its mission
of protecting human health and the environment.
-------
Introduction
In 1983, former EPA Administrator William Ruckelshaus stated that, "EPA would operate
in a fishbowl.'"This is a proud tradition that EPA has continued for thirty-five years.This
annual report serves as a portion of the Scientific Integrity Program's (the Program's)
contribution to the Agency's ongoing commitment to transparency.
In March 2009, an executive memorandum directed the White House Office of Science
and Technology Policy to develop a plan to ensure the integrity of federal science. EPA
released its Scientific Integrity Policy (the Policy) in February 2012 and its first annual
report on scientific integrity in November 2013.The Policy provides both a vision and
a roadmap for ensuring high standards of scientific integrity at the Agency.The Policy
lists the components of a culture of scientific integrity and offers a framework for
ensuring Agency-wide compliance. At the end of each fiscal year, EPA assesses the overall
implementation of the Policy during a review of all the scientific integrity activities at
the Agency.The FY 18 annual review culminated in the publication of this annual report,
which serves to highlight the status of scientific integrity within EPA, promote new
scientific integrity initiatives, celebrate FY 18 scientific integrity accomplishments at EPA,
and identify areas for future investment.
Scientific Integrity in 2018
Calendar year 2018 marked the start of an initiative to separate concerns reported to
the scientific integrity team into two categories - advice and allegations (see page 24
of this report).The response to this change has been overwhelmingly positive. It has
enabled the program to avert many allegations by providing early support and assistance
to our submitters.
Any change in administration may bring with it many changes to EPA's procedures and
practices. Some of the actions taken can be easily misconstrued to be lapses in scientific
integrity. It is the task of the scientific integrity team and the Committee to carefully
examine each concern and take appropriate actions.This represents a large investment
of time but is necessary for the maintenance of our culture of scientific integrity and for
successful transitions from one set of leaders to another.
This fiscal year marked the initiation of a training specifically geared towards managers
and supervisors.The impacts of this will be seen as more leaders complete the training
in 20l9.The workshop challenges our leaders to uphold a culture of scientific integrity,
encourage good policies and practices, lead the way, and mitigate negative influences on
scientific integrity.These tenets of scientific integrity play a critical role in our ability to
successfully use science to inform Agency decisions.
As in previous years, the Agency worked to promote a culture of scientific integrity
in the conduct, communication, supervision and utilization of science to carry out its
mission-driven work. As evinced by the accomplishments of the Scientific Integrity
-------
Program and across the Agency in FY 18, it is critical to promote the Scientific Integrity
Policy through continued training and outreach, having other programs promote
scientific integrity in the Agency, and continuing the work of the Committee.
What is Scientific Integrity?
Scientific integrity is the adherence to professional values and practices when conducting,
communicating, supervising, and utilizing the results of science and scholarship. Scientific
integrity ensures objectivity, clarity, reproducibility, and utility (Figure 2). It provides
insulation from bias, fabrication, falsification, plagiarism, outside interference, and
censorship.
The Agency may make final policy decisions that weigh other factors besides science
but are still consistent with EPA's governing statutes. Such decisions, even if they are
not consistent with the science, do not necessarily constitute scientific integrity issues.
Implementing the Policy requires input from a wide variety of sources across the Agency,
which interact to promote and maintain a culture of scientific integrity.
RESEARCH INTEGRITY,
SCIENTIFIC INTEGRITY
SCIENTIFIC METHOD
RESEARCH RESULTS
QC& PEER REVIEW
SUPERVISE, COMMUNICATE, UTILIZE
PUBLICATION
Figure 2.What is scientific integrity?
-------
EPA's Scientific Integrity Policy
The Policy generally applies to all EPA employees including scientists, managers and
political appointees. Additionally contractors, grantees, collaborators, student volunteers
and other EPA-affiliates may fall under its purview. However, some exceptions apply
in the cases of agreements predating the Policy and contracts that do not specifically
address science.The Policy builds upon existing Agency and government-wide policies
and guidance documents to enhance EPA's overall commitment to scientific integrity.
Promoting a Culture of Scientific Integrity
The Policy establishes an expectation that the Agency will foster honest investigation,
open discussion, refined understanding and a firm commitment to evidence, and
scientific research that is generated in a timely manner, characterized appropriately
for Agency policy-making, and communicated clearly to the public. All EPA employees
are explicitly forbidden from suppressing, manipulating, or otherwise altering scientific
data.This assures that EPA decisions are informed by the best science that the Agency,
its contractors, grantees, and collaborators can offer. A culture of scientific integrity is
also one that protects employees who report allegations of suspected violations of the
policy. Similarly, employees who express differing scientific opinions should neither fear
nor experience retaliation.
Releasing Scientific Information to the Public
The Policy fosters a culture of transparency regarding the results of research, scientific
activities, and technical findings. EPA encourages open communication that is free from
political or other interference.The clear and timely release of science facilitates a free
flow of information and increases public confidence in the Agency.
Peer Review and the Use of Federal Advisory Committees
Independent peer review is a necessary component of quality control in science and
thus a crucial aspect of scientific integrity. EPA's review process is outlined in the Agency
Peer Review Handbook". External federal advisory committees offer further
opportunities for review of scientific activities and provide additional scientific expertise.
Professional Development of Government Scientists and Engineers
EPA employees are encouraged to participate in professional development activities
to fully engage with their scientific communities and become leaders in their fields.
Professional development activities may include presenting at scientific meetings or
conferences, participating in professional societies, or serving on editorial boards of
peer-reviewed journals.
-------
FY18 Scientific Integrity Program Initiatives
New initiatives in FY 18 centered on enhancing EPA's culture of scientific integrity.This
was achieved by providing additional scientific integrity resources, having conversations
with Agency leaders about scientific integrity, and promoting awareness of how to
properly acknowledge the intellectual contributions of others.
Best Practices for Clearance of Scientific Products at EPA
EPA is committed to ensuring the timely release of scientific information to the public.
The release of EPA's scientific products is preceded by clearance, a process of attaining
management approval for release.The Policy mandates the Committee to "develop a
framework for Agency clearance procedures for scientific products as a guidance for
Program Offices and Regional Offices." On May 15,2018, the Program released Best
Practices for Clearance of Scientific Products at EPA'".This document should be referred to
when clearance processes are being developed, evaluated, or revised. Some best
practices include:
• Establishing essential elements of clearance procedures
• Determining and defining which scientific products are required to be cleared
• Verifying that appropriate scientific reviews occur before clearance is initiated
• Training employees on clearance procedures
The best practices promote transparency, clarity, timeliness, predicatability, and
consistency in clearance processes.The Agency is developing an Agency-wide electronic
clearance system that would complement the best practices.
Management Dialogues on Scientific Integrity
In August 2018, the Program launched an outreach initiative for managers. Supervisors
from almost every office, program, region, and laboratory will participate in Management
Dialogues on Scientific Integrity led by Francesca Grifo (ScIO).These conversations
provide an opportunity for managers to learn about their scientific integrity
responsibilities as leaders at EPA, understand what scientific integrity is, know what
resources are available, identify lapses in scientific integrity, and discuss their experiences
with scientific integrity. Attendees are also provided the Scientific Integrity Brochure,
the Scientific Integrity fact sheet, and scientific integrity posters.
Throughout August and September of FYI8, four Management Dialogues on Scientific
Integrity sessions were held in the Office of Chemical Safety and Pollution Prevention
(OCSPP) and the Office of Land and Emergency Management (OLEM). Ninety-nine
-------
1. Best Practices for
Clearance of Scientific
Products at EPA
2. Scientific Integrity
Language for Grant
Agreements & Contracts
3. Scientific Integrity
Management Dialogue
Training
4. Plagiarism Detection
Software
managers and supervisors participated in conversations
with the SclO.To further improve the quality of this nascent
program, the ninety-nine attendees were asked to complete
an evaluation of the session that they attended.The
participants that responded provided positive feedback that
the session was useful, will help them to do their job more
effectively, and that they would recommend a colleague
attend.
Scientific Integrity Language for Grant
Agreements
The Policy defines "science" and "scientific" as expansive
terms that refer to the full spectrum of scientific endeavors,
e.g., basic science, applied science, engineering, technology,
economics, social sciences, and statistics. Beginning in
FY 18, if the recipient of a grant is engaged in conducting
science, supervising science, communicating science, or
using or applying the results of science, the recipient
and the project team must review the Policy and comply
with its requirements as part of the agreement with
EPA. Additionally, the grantee need not necessarily be an
individual, but may be an organization.These requirements
are detailed in Section 33, "Scientific Integrity Terms and
Conditions" of "EPA General Terms and Conditions,
Effective October 1,2018.iv"
Scientific Integrity Language for Contracts
In FY20I8, EPA issued a proposed rulev to address
applicability of scientific integrity requirements to EPA contracts by creating a clause
in solicitations and contracts under which a contractor may be required to perform
scientific activities or use scientific information to perform advisory and assistance
services. When the rule is final, this clause will complement the EPA Scientific Integrity
Policy to ensure that all scientific work developed and used by EPA is accomplished with
scientific integrity.
Detecting Plagiarized Text
As stated in The Best Practices for Designating Authorship", all Agency employees must
"appropriately characterize, convey, and acknowledge the intellectual contributions of
others." In FY 18, the Program conducted a pilot project of making a software program
that evaluates the originality of document content available to EPA scientists and
supervisors. Authors were encouraged to use the program to check the work and
ensure appropriate attribution in EPA work products.Twelve drafts were submitted
during the FY 18 pilot project.
-------
KM
Ongoing Scientific Integrity Activities
The annual activities described in this section were successfully completed in FY 18 and
contributed to ongoing support for the evolving scientific integrity activities at EPA over
multiple fiscal years.
The Scientific Integrity Committee
The Policy established a Scientific Integrity Committee (the Committee), chaired by the
Scientific Integrity Official (SclO).The Committee meets quarterly and consists of senior
program, office, and regional officials who are designated as Deputy Scientific Integrity
Officials (DSclOs).They provide leadership for the Agency on scientific integrity, jointly
assist in implementing the Policy, and promote Agency compliance with the Policy.The
ScIO regularly communicates with Committee members to discuss potential approaches
to emerging issues and work together to resolve allegations of a loss of scientific
integrity and respond to requests for advice.The participation of the Committee
ensures that a variety of experiences and viewpoints are considered in decision-making.
The Program wishes to recognize David Bloom (OCFO), Carole Braverman (R5), Al
McGartland (OP), Betsy Shaw (OAR), Carol Ann Siciliano (OGC), and Deb Thomas
(Region 8) as the longest-serving members of the Committee (five or more years) and
thank them for their expertise and guidance.
The Annual Employee Conversation with the Scientific Integrity Official
The Annual Employee Conversation with the ScIO (Figure 3) provides an opportunity
for EPA employees to learn about scientific integrity at EPA and ask questions on
this topic.The ScIO, Dr. Francesca Grifo, presented to a live audience at headquarters
and to the rest of the Agency through a webinar in June 2018.The meeting had an in-
person attendance of 45 and 301 webinar participants for a total of 346 attendees.
These conversations improve the visibility of the Policy and increase awareness about
scientific integrity among EPA employees.The session emphasized the broad applications
of the Policy across EPA and encouraged employees to recognize and bring forward any
&Obji
Annual Employee Conversation Sa
with the Scientific Integrity Official
ective
ience.
Tuesday, June 12th, 3:00 - 4:00 pm EDI ^
WJC-N 5020 (in DC)
intranet.ord.epa.gov/scientific-integritv/aonual-conversation
S-EFft
intranet.ord.epa.gov/scientific-integrity
Figure 3. Flyer for the 2018 Annual Conversation with the ScIO
-------
concerns.The EPA Science Advisor, Dr. Jennifer Orme-Zavaleta, provided introductory
remarks and the Deputy to the Scientific Integrity Official, Dr.Vincent Cogliano, discussed
the new allegations procedures (Appendix I).
New Employee OnboardingTraining
Since January 2017, all new EPA employees are required to take online scientific integrity
training, which consists of a video of the ScIO conducting a training session.This training
session features the introductory whiteboard video and discussion, followed by a short
quiz. Showing this training to new employees helps them to establish a personal commit-
ment to scientific integrity from the moment that they arrive at the Agency.This greatly
contributes to the overall culture of scientific integrity at EPA. In FY 18, 83 new EPA
employees completed the training.The monthly number of trainees is depicted below in
Figure 4. By the end of FY 18,22 months after the mandatory training was implemented,
498 EPA employees had completed scientific integrity onboarding training.
25
Figure 4. Monthly completion in FY 18 of Scientific Integrity Onboarding Training
Additional Training
In July 2018, the Deputy to the Scientific Integrity Official also provided general training
to the Office of Underground Storage Tanks (USTs) in the Office of Land and Emergen-
cy Management (OLEM).This training provided an overview of the Policy and scientific
integrity at EPA.
Quarterly Coordination Meetings with OIG and OGC
The ScIO maintains regular communication with both the Office of Inspector General
(OIG) and the Office of General Counsel (OGC) through quarterly meetings. During
these meetings, the status of current allegations of a loss of scientific integrity under
review and the anticipated courses of action are discussed. Identifying information is
-------
omitted in any cases where the submitter of the allegation has requested confidentiality.
Coordination between these offices exemplifies the Agency-wide participation and col-
laboration in implementing the Policy.
The handling of scientific misconduct, which includes fabrication, falsification, plagiarism,
or misrepresentation in proposing, performing, or reviewing scientific or research activ-
ities, is governed by EPA's Scientific Misconduct Policy and is overseen by the OIG, with
the exception of plagiarism which may be handled by the ScIO as described in the ScIO/
OIG Coordination Procedures™. In FY 18, one allegation was received through the OIG
hotline and referred to the ScIO. No allegations were referred to the OIG by the ScIO
during FY 18.
Web Analytics
In FY 18, the scientific integrity home page on the intranet was visited 1,3 15 times.This
was a 15% increase from FY 17 (1,138). After the homepage,The Federal Managers
Financial Integrity Act (FMFIA) section was the most visited scientific integrity page, with
453 views, in FY 18.The 2018 Annual Employee Conversation with the Scientific Integrity
Official was the third most popular page with 45 I visits.Two initiatives that were new
in FY 18, iThenticate and Best Practices for Clearance of Scientific Products, were the
fourth and fifth most commonly accessed scientific integrity webpages and were visited
274 and 265 times, respectively.
On the internet, the scientific integrity homepage attracted 5,573 hits.This was a 32%
drop from FY 17 (8,184). However, the 5,573 visits in FY 18 fell roughly halfway between
the visits recorded in FY 16 (2,371) and FY 17 (8,184). Interest in the Policy generated
1,458 visits to the Policy webpage. While this was a 20% decrease from FY 17 (1,835), it
was also about twice as high as it was in FY 16 (663). Best Practices for Authorship was
accessed on 285 occasions (the third most), in FY20l8.The Dr. Chris KirkpatrickWhis-
tleblower Protection Act of 2017 was the fourth most viewed page with 195 visits.The
Best Practices for Clearance was the fifth most visited page with 103 views.
Contractor-Managed Peer Review
EPA strengthened the Agency's oversight of contractor-managed peer review panels in
FY2013 by developing a conflict-of-interest review process for contractor-managed peer
reviews. Conflicts of Interest Review Process for Contractor-Managed Peer Reviews of
EPA HISA and ISI documents™ specifically applies to all future Agency technical docu-
ments (following publication of the process) that are designated as Influential Scientific
Information (ISI) or Highly Influential Scientific Information (HISA).The process is de-
signed to enhance the transparency and internal oversight of these peer reviews and
reduce the potential for organizational or personal conflict-of-interest concerns by in-
creasing public participation and more rigorous internal review.This process was applied
to one contractor-managed peer review in FY20I8.
FMFIA: Certifying Compliance with the Scientific Integrity Policy
The Federal Managers Financial Integrity Act (FMFIA) requires that federal agencies as-
sess the effectiveness of programmatic and financial internal controls. EPA Assistant Ad-
ministrators (AAs) and Regional Administrators (RAs) must certify that their programs
-------
comply each year through an assurance letter to the EPA Administrator, who delivers an
overall statement of assurance to the President and Congress. FY20I8 marked the fifth
year that AAs and RAs were required to submit a certification of internal controls for
scientific integrity. Based on the requirements that are outlined in the Policy programs,
offices, and regions were asked to report their accomplishments, potential weaknesses,
and overall progress in implementing the Agency's Policy.
The FY20I8 FMFIA process provided a structured assessment of EPA's scientific integrity
activities across the Agency. On behalf of their offices, programs, or regions, respondents
highlighted their accomplishments, showcased their innovations in scientific integrity,
detailed problems or challenges related to scientific integrity, provided issues that they
would like for the Committee to address, and discussed any vulnerabilities or weakness-
es related to scientific integrity within their organizations or within the Agency.
Allegations and Requests for Advice Through End of FY18
The Presidential Memorandum on scientific integrity (March 9,2009) directs that "Each
agency should have in place procedures to identify and address instances in which the
scientific process or the integrity of scientific and technological information may [have
been] compromised." Accordingly, the Scientific Integrity Program provides a procedure
for seeking advice to prevent lapses in scientific integrity and for reporting allegations of
possible violations of EPA's Scientific Integrity Policy (Figure 5).
In FY 18, the Program drafted a new procedure creating a two-pronged approach sep-
arating those seeking advice about scientific integrity concerns from those reporting
allegations of a loss of scientific integrity. Several allegations were evaluated and closed
during the year. At the same time, the Program received a record number of requests
for advice from people concerned about scientific integrity at EPA.
New Procedures for Allegations and Requests for Advice
Following six years of experience, the Scientific Integrity Program developed a two-track
procedure in FY 18 that separated reporting allegations from seeking advice and assis-
tance when there is a concern that EPA's Scientific Integrity Policy is not being upheld.
In general, the new advice track was designed to resolve a concern before it becomes a
formal allegation.
The aim of the advice-and-assistance track is early preventive action to uphold EPA's cul-
ture of scientific integrity. Anyone with a question or a concern is encouraged to have a
conversation with the ScIO (Francesca Grifo), the Deputy to the ScIO (Vince Cogliano),
or a Deputy ScIO in a program or regional office.These officials provide timely advice or
assistance. If the issue is not one of scientific integrity, they can assist in redirecting it as
appropriate. Allegations of research misconduct that involve waste, fraud, or abuse are
referred to the Office of Inspector General (Figure 5).
-------
12-13
INSPECTOR
GENERAL
hotline:
-888-5^6-87*
SCIENTIFIC
INTEGRITY
OFFICIAL
oEPUTy
AfWV
official*
Figure 5. How to seek scientific integrity advice
or report an allegation
If advice and assistance
does not resolve the issue,
an allegation may be filed.
The aim of the new allega-
tion procedure is to evalu-
ate and take corrective
scientific action, when
circumstances warrant. Any
person may report an
allegation to the ScIO, any
Deputy ScIO, or the Office
of Inspector General.
Allegations should include,
when possible, the provision of EPA's Scientific Integrity Policy that was violated, sup-
porting evidence with a timeline, and the names of witnesses who can provide pertinent
information. Once received, the Scientific Integrity Program screens the allegation,
gathers pertinent information, and makes a determination based on the available evi-
dence, drawing on the experience of the Scientific Integrity Committee, as needed
(Figure 6).The determination includes recommendations for corrective scientific action
and other preventive measures as appropriate.Throughout the process, confidentiality is
maintained to the extent the law allows and knowledge about the identity of persons
submitting or otherwise involved in the allegation is limited to those who need to know.
r ^
Submission
L. ^
^¦1
r *
inn
veilii iy
Office of
Inspector
General
r l
Inquiry
r i
Determination
L J
(Internal
Subjects)
Inquiry
(External
Subjects)
1
External
organization
Figure 6. Procedure for submitting and evaluating an allegation
Annual Update on Allegations and Advice
Following the development of the two-track procedure described above, the Scientific
Integrity Program reviewed all prior allegations and reclassified many of them as re-
quests for advice. Since EPA's Scientific Integrity Policy was issued in 2012 and through
September 30,2018, there have been I 10 requests for advice and 74 allegations.These
figures are audited and updated annually, every August.
-------
Figure 7 shows the number of allegations and requests for advice by quarter.
The number of requests for advice accelerated during FY 18, when there were
forty-nine requests for advice and four allegations.These 53 queries mean that
nearly a third (29%) of the total 184 queries received over the six-and-a-half-year
history of the Program were received during the past year. Although the number
of requests for advice greatly increased during FY 18, the number of allegations
was the lowest since FY 13.
Figure 8 compares the topics of all queries between FY 17 and FY 18 with the
topics of queries received during FY 18. Forty-three percent of queries received
during FY 18 concerned interference with science, compared with 10% historical-
ly before FY 18. In other areas, although the numbers are small, the numbers of
queries received during FY 18 related to peer review or to data quality/methods
were 4 and 3, respectively, compared with 2 and I received from 2012 to the end
of FY 17.
Conversely, queries related to the expression of a differing scientific opinion,
historically the third most common type of query, declined during FY 18, when it
was the topic of only one query. Another apparent trend is a decline in queries
related to ethics (which should be reported to a Deputy Ethics Official or to the
Office of General Counsel) or in queries that otherwise were not a matter of
scientific integrity. This may be due to more people becoming familiar with EPA's
Scientific Integrity Policy, though more information is needed to confirm this.
Queries by Fiscal Quarter
40
35
30
25
20
15
10
5
0
2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018
¦ Advice ¦ Allegation
Figure 7. Number of scientific integrity queries received by quarter
. 1
II
. . .1.1
1234123412341234123412341234
-------
14-15
Tr*f»ipa/er»cy: 3
fabrication,'
Falsification
Peer ; 2
Revie# 1 Unknown: 2
. , I _^_Data {Jualltyi'Melliods/QA: 1
Not Si-Ethics: 1
Suppression'
K
Clearance: 30
Nd1 SI- Other 7
~SO: 21
Ethics: 131 liiUfifwence:
Ptatfutan: 1 \ si-Pmonnel: 1
FabricatKHii'FalsifiCdlion : 2
Dala Qualrty.'Fieth^QA: 3
Peer Review: 4
Suppression^
Del*yf
Clearance: 10
FY 12-FY 17 Queries (131)
•Hal MOtrtrr: dill quafef/Mrirtcxf^Ey, [Xj. 0SJ>Mt fnaml virm |J^ frtuhn (t|. PsAi [l|
FY18 Queries (53)
Figure 8. FY 12-17 scientific integrity queries versus FY 18 scientific integrity queries
Figure 9 compares the topics of all requests for advice during FY 12 through FY 17 with
the topics of queries received during FY 18,The patterns seen in Figure 9 for all queries
(requests for advice plus allegations) are also apparent in this large subset of all
queries.
FafcritiliOW
F«l*lfic«tto
-------
Interference 1
Peer Review: i
Suppression/
Deity i'
Clearance: 1
Not SI: Da",a QiMlHy/Mettiod&'GA (1), Petition ('
Policy <1). DSO (1). Personal Vwws (1)
No-1 Si-Personnel: 2
Suppression,'
D#l«yi
Clearance: 12
Peer Review: 2
Fabrication1
F»ltdic»tieft: 3
KotSI-£thks:
Interference: 3
Pligiafisn: 4
FV12-17 Allegations (70) FV18 Allegations (4)
Figure 10, Comparison of all FY 12-17 allegations versus FY 18 allegations
Figure I I shows the status of all allegations and requests for advice at the end of
FY 18.There was substantial progress in clearing the backlog of allegations, and there
were only four active allegations at the end of FY 18. Since 2012, five allegations have
been transferred to the Office of Inspector General, which has the responsibility for
investigating allegations of research misconduct or waste, fraud, or abuse. Among
allegations adjudicated by the Scientific Integrity Program, similar numbers of allegations
were substantiated as were not substantiated.
Since 2012, among requests for advice, many (40) have been concluded without the filing
of an allegation. Conversely, a similar number (41) are still active, with the individuals
involved still seeking solutions to their concerns about potential violations of the
Scientific Integrity Policy.
InsuPTcie-nl KenwuumiiMw
IrtfOrmaSipri: 1
On Hold:
^ i, 3 / Transferred to Allegation: f
Active: i
OIG-Trander 5
Transferred: 5
Not Substantiated 21
Aetna: 41
A! I Allegations Al I Advice
Figure I I. Current status of all allegations and requests for advice received
-------
FY18 Queries by
Submitter's Office
FY18 Queries by
Subject's Office
Unknown: 1
f
Programs/Offices: 49
Prog ram sfOffices: 44
Purple: Programs/Offices: 44
Red: Regions: 4
Orange: External: 3
Green: Unknown: 2
Purple: Programs/Offices: 49
Red: Regions: 3
Green: Unknown: 1
Figure 12. Sources arid subjects of FY 18 queries
Figure 12 breaks down the queries received during FY 18 by organization. Most queries
involved EPA's program offices.Three queries originated outside EPA, demonstrating the
interest of external stakeholders in the Agency's culture of scientific integrity.
Alternative Dispute Resolution
EPA addresses some allegations of a loss of scientific integrity with the help of EPA's
Conflict Prevention and Resolution Center using alternate dispute resolution (ADR).The
Scientific Integrity Program has used its mediation services for two types of scientific
integrity issues: differing scientific opinions and authorship disputes.To date, two cases
have been resolved by working with the Conflict Prevention and Resolution Center's
staff and accessing mediators through their contract. As of the end of FY 18, another
ADR project was ongoing.
The benefits of using ADR to resolve scientific integrity issues include;
• Stakeholders agree to the process, appreciate the neutrality of the mediators,
and accept the results;
• Scientific integrity is upheld; and,
• The case is resolved in a timely manner.
Summary of Adjudicated Allegations in FY 18
Eleven allegations were adjudicated during FY 18. Following are summaries of the
allegations adjudicated during FY 18.
I. Political Interference with Scientific Presentations
Allegation: Scientists employed by or affiliated with ORD were told on a Friday that they
could not honor their previous commitment to present results of their research at a
-------
scientific conference the following Monday.The cancellation was reported to have been
initiated by the Office of Public Affairs.
Outcome: Administrator Pruitt acknowledged the incident and wrote that it would not
recur, that ORD has authority to make decisions about event participation, and that he
was committed to upholding EPA's Scientific Integrity Policy.
2. Data Quality
Allegation:The Scientific Integrity Program received a Request for Reconsideration
submitted by an external organization under the Data Quality Act.The external
organization alleged that the findings of some papers published in a scientific journal
by authors from another federal agency were unreliable and asked EPA to remove
information from its website that had been based on these published papers.
Outcome: EPA has an established process under the Data Quality Act to evaluate
such complaints, and the external organization used this process in submitting a
Request for Correction and subsequent Request for Reconsideration. EPA followed its
established process and convened a panel of senior officials to consider the Request
for Reconsideration.The panel reaffirmed the utility, objectivity, and integrity of the
information on EPA's website.
3. Plagiarism
Allegation: Plagiarism-detection software flagged some grant proposals submitted to an
EPA Office as containing text from other sources.
Outcome: A careful review found that most instances of flagged text came from
references listed in the grant proposals but that were not attributed to a specific
reference or placed in quotation marks. Only a few instances of flagged text came from
other sources. Although proper attribution is important, the apparent plagiarism seemed
to be due to sloppiness and to the space restrictions imposed by the grant application
form.The Scientific Integrity Program worked with the program that reviews these grant
proposals to prevent future attribution issues.
4. Clearance Delay
Allegation: An employee alleged that clearance was delayed when a manager requested a
third internal review of a manuscript for submission to a journal, when only two reviews
are standard.
Outcome:The office's clearance policy and procedures allow managers to request
additional review of manuscripts if they disagree, as they did in this case.The request for
a third review was appropriate.
5. Scientific Credit
Allegation: An employee who made a significant contribution to a project was
improperly excluded from a group award.
Outcome: Management acknowledged that this was a mistake and nominated the
employee for another award.The employee considered this to be an acceptable
resolution to the allegation.
-------
18-19
6.Authorship Credit
Allegation: Employees were listed as co-authors in a draft EPA report, but not listed as
co-authors in the final report. Some authors pointed out that their work was included
in the final report and they should have been authors.
Outcome: Upon review, it was found that the final report provided extensive lists of
authors, reviewers, and acknowledgements. A list of the draft report co-authors is
prominently provided in the final report. Co-authorship of the final document was
not extended to those who did not work on the revisions that led to the final report.
Some of these employees were also recognized in an acknowledgement section.
The best practice would have been early and repeated conversations among the
authors to discuss authorship in both drafts and final versions, and to explicitly agree
on what to do in the event of authors leaving the project and/or the Agency.
7.Authorship Credit
Allegation:A university employee was unfairly excluded from authoring journal
publications that were developed under an EPA grant.
Outcome:The principal investigator, who determined authorship, is a university
employee.Therefore, this was determined to be a matter internal to the university.
8. Differing Scientific Opinion
Allegation:An employee in a program office alleged that EPA's methods for assessing
health risks from radionuclides are outdated.
Outcome:The employee was able to express this differing opinion to management,
including through three internal reports written on official time.The Scientific Integrity
Policy provides that employees can express differing scientific opinions and their
rationale, preferably in writing, without fear of retaliation.The Policy does not require
that the differing opinion be accepted.The Policy was not violated.
9. Differing Scientific Opinion
Allegation:A manager in a regional office alleged that preliminary remediation goals for
radionuclides are not scientifically defensible and that their use should be suspended
and replaced with a different approach.
Outcome:The manager had ample opportunity, which was utilized, to express this
differing opinion to higher management.Therefore, it was determined that the Policy
was not violated.
10. Differing Scientific Opinion
Allegation:An employee alleged that EPA does not accurately communicate its
greenhouse gas footprint since renewable energy certificates, according to the
allegation, do not significantly reduce greenhouse gas emissions.
Outcome: This was an instance of a differing scientific opinion, where the employee
had expressed the scientific opinion to the responsible officials.Therefore, it was
-------
determined that the Policy was not violated.
I I. Peer Review Requirements
Allegation:An employee objected to EPA's use of a greenhouse gas calculator that
was not peer-reviewed.
Outcome:The inquiry found that there is no requirement that calculators undergo
peer review.
In FY 18, three other allegations were determined to not involve scientific integrity
and another allegation was withdrawn by the submitter. Additionally, sixty-one
queries that were previously considered allegations prior to FY 18 have been re-
classified as requests for advice under the new procedure that was developed this
past year.These requests for advice are not reported in this summary of adjudicated
allegations, but are summarized in Figures 7, 8, 9, I I, and 12.
Accomplishments Across EPA
In FY 18, EPA program and regional offices took many approaches to enhance a
culture of scientific integrity at EPA. Offices throughout the Agency accomplished
this by featuring scientific integrity language in grants and employee handbooks and
by establishing regional science councils. Employees were provided training to learn
new skills and technologies. Innovative systems transparently released data and
scientific information to the public. Efficient clearance procedures were created with
an emphasis on timely release. New quality assurance procedures streamlined the
data collection and management process.Additional data was released to the public
and existing public datasets were made more user-friendly.The science that underlies
EPA's decisions was reinforced through the Agency's use of independent peer review
and federal advisory committees. EPA employees also expanded their knowledge,
learned new skills, and remained leaders in their fields by participating in profes-
sion-al societies, attending conferences, and speaking on expert panels. See Appendix
II for examples of scientific integrity accomplishments across the Agency in FY 18.
-------
20-21
4
Areas for Furure Investment
Beyond 2018
Increasing Visibility of Scientific Integrity
Implementing the Policy and fostering a culture of scientific integrity is most effec-
tive when all employees, contractors, grantees, and student volunteers understand
the Policy and how they contribute to EPA's culture of scientific integrity. While the
Annual Report on Scientific Integrity and the Annual Employee Conversation with
the Scientific Integrity Official increase awareness of the Policy and understanding of
scientific integrity, the Scientific Integrity team will pursue additional opportunities to
increase the visibility of scientific integrity at EPA.
Outreach Materials and Publications
New outreach efforts include "tombstone" signs that are prominently placed in EPA
headquarters lobbies.These posters call upon everyone to be responsible for scien-
tific integrity and alert employees to upcoming events related to scientific integrity.
They will also be used to inform employees of new policies and procedures related
to scientific integrity. Also, the Scientific Integrity internet and intranet websites will
be continually updated to announce new scientific integrity initiatives and provide
innovative scientific integrity resources.The scientific integrity brochures will also be
revised to assist those with a scientific integrity concern, so that they understand the
new two-track allegation procedure.
Best Practices of Scientific Integrity Handbook
The Program has received multiple requests from across the Agency to develop a
Handbook with scientific integrity best practices.The Program is excited to have this
opportunity to provide a one-stop-shop on everything scientific integrity at EPA. In
the current development phase, the handbook is envisioned to address key aspects
of scientific integrity, including releasing scientific results in a timely manner, commu-
nicating science to internal and external audiences, ensuring proper attribution in
citations, and offering equitable professional development opportunities to staff.
Embracing and Modeling Scientific Integrity
Since its inception, EPA has embraced a culture of scientific integrity. For six years,
implementation of the Policy has re-enforced the Agency's commitment to scientific
integrity. In the upcoming years, the Program and Committee look forward to fur-
ther assisting the Agency in ensuring that scientific integrity is embraced and mod-
eled by all employees, contractors, grantees, and volunteers.
-------
Subject Matter Whiteboard Videos
In 2016, the Program produced an award-winning whiteboard video, Scientific
Integrity at EPA, that provides an overview of both scientific integrity and the
Policy. Scientific Integrity at EPA50" is available to view on the scientific integrity
intranet website (Figure 13).Three additional case study videos about manipulating
science, suppressing scientific findings, and delaying the release of science were also
produced in 20l6.The Program plans to draft storyboards and scripts for additional
training videos.
Increasing Access to Scientific Integrity Expertise
The Committee, with its 24 members, has representation from every office and
region. However, the geographic distribution of EPA campuses, satellite locations,
and laboratories causes some sites to lack a DScIO with a physical presence.The
Program and Committee are working to increase access to scientific integrity exper-
tise at these locations and throughout the Agency. One way that this issue is being
addressed is that the ScIO will begin holding regular, open office hours on scientific
integrity every Wednesday, from I 1:30 am to 1:30 pm Eastern Time. She is available
for walk in (or call in) consultations in her office in Washington, D.C., in the Ronald
Reagan Building, room 51 142 (Call 202-564-1687).
Protecting and Maintaining EPA's Culture of
Scientificlntegrity
EPA's ability to protect human health and the environment is dependent on the
integrity of the science that is conducted and utilized at the Agency. As a result, it is
everyone's responsibility to protect and maintain EPA's culture of scientific integrity.
The Program continues to develop guidance, standard operating procedures, policies,
and systems for the Agency to adopt.
Differing Scientific Integrity Opinions (DSO) Policy
The Policy re-affirms that a culture of scientific integrity "welcomes differing views
and opinions on scientific technical matters as a legitimate and necessary part of the
scientific process." Therefore, the Policy mandates the ScIO, with input from DSclOs,
"to develop a transparent mechanism for Agency employees to express differing
scientific opinions (DSOs)."A DSO arises when an employee, who is substantively
engaged in the science, disagrees with scientific data, interpretation, or conclusions
that will be relied upon to inform an Agency's policy decision. If this happens, the em-
ployee is encouraged to express, in writing, her/his dissenting opinion and reasons for
holding this opinion. It is then expected that DSOs will be resolved during internal
deliberations or the peer review process.The peer review panel report, along with
other deliberative documents, are provided to policy makers.The Program is cur-
rently drafting a detailed DSO Policy for the Agency that is anticipated to be released
in the near future.
-------
Electronic Clearance System
The Program is working towards unveiling an electronic clearance system in 2019
that will further promote transparency, clarity, timeliness, predictability, and
consistency across the Agency when clearing scientific products.This versatile
system is expected to be an important component in implementing the Plan to
Increase Public Access to EPA-Funded Research. Other anticipated benefits
include automatic notifications to approvers and submitters, version control, and
record-keeping.While the Office of Research and Development (ORD) has already
adopted an electronic clearance system, this would provide other offices the
opportunity to utilize electronic clearance.
/samwvL
-/ ^SEVIEVl
Q
-1 WW
£* --» fiNGwM
77
KHIK
-"=11
Tt&AlTy
LSnsW
SCIENTIFIC
iWXE&ftrTY
OFFICII*.
D**u Fy
Ifr,
AAn A f/\
PSOTfCT HUlMN H^LTH
r* ENYttOHMSW
Figure 13. Scientific Integrity at EPA Whiteboard Overview
-------
v Assessing the Implementation
^ of EPA's Scientific Integrity Policy
Three initiatives began in FY 18 that provide opportunities to assess the effectiveness
of the Program and Committee's efforts in implementing the Policy and to fortify
EPA's culture of scientific integrity.
OIG Scientific Integrity Audit
On August 30,2018, the Office of Inspector General (OIG) for the EPA began a
self-initiated project to study the implementation of the Policy.The project's objec-
tive is to determine "whether the EPA's Scientific Integrity Policy is being implement-
ed as intended to assure scientific integrity throughout the EPA." The project focuses
on four key areas:
1. Extent and type of employee concerns, if any, with scientific integrity at the EPA
2. Employees'awareness of the EPA's Scientific Integrity Policy, including the
process for reporting potential violations
3. Reasons potential violations may not be reported
4. Adjudication process in general and any related concerns (e.g., satisfaction with
complaint resolution, timeliness of resolution, and other process-related issues)
The OIG will also conduct an anonymous survey of EPA employees and contractors.
This survey is expected to be open for about a month between November and
December 2018. The results of this survey and a final report of these findings are
anticipated by September 2019.
GAO Scientific Integrity Engagement
On June 14,2018, the U.S. Government Accountability Office (GAO) informed EPA
that it began an engagement on implementation of government scientific integri-
ty policies after receiving a request from Senator Bill Nelson. This review will focus
on three key questions:
1. What are the main components of selected agencies'scientific integrity
policies?
2. To what extent do selected agencies have processes in place to reasonably
ensure that the objectives of their scientific integrity policies are achieved?
3. To what extent have agencies established processes for reporting and
investigating allegations of violations of their scientific integrity policies?
This report is expected to be released in the spring of 2019.
-------
24-25
<
Conclusions
Preparation of this Annual Report provided the Program with a very important op-
portunity to reflect on implementing the Policy and scientific integrity initiatives, and
to also create a strategic roadmap for enhancing EPA's culture of scientific integrity
in the coming years.
In FY20I8, the ScIO, the Program, and the Committee continued to promote a
culture of scientific integrity in all of EPA's programs, offices, and regions.This was
achieved by hosting an Agency-wide conversation with the ScIO, training nearly 100
new employees on scientific integrity, engaging in dialogues with almost 100 manag-
ers about scientific integrity at EPA, and working in close coordination with partners
throughout the Agency including OGC and OIG.The establishment of new clearance
and quality assurance procedures in Agency offices promoted transparency and the
timely release of scientific data. EPA's use of peer review and federal advisory com-
mittees contributed to the high-quality science produced at the Agency in FY20I8.
EPA scientists and researchers learned new skills, met fellow leaders in their fields,
and presented their findings in FY 18 because their supervisors encouraged their
professional development.
New initiatives in FY 18 created exciting opportunities to strengthen the culture of
scientific integrity at EPA. With the release of the Best Practices for Clearance of
Scientific Products at EPA, programs, offices, and regions can reference these guide-
lines when developing, evaluating, or revising their clearance procedures to promote
transparency, clarity, timeliness, predictability, and consistency.The advent of manage-
ment dialogues on scientific integrity is fostering engaging discussions on scientific
integrity that will continue into FY 19. Expanding scientific integrity language to the
Agency's grant agreements and contracts re-affirms EPA's commitment to scientific
integrity.The pilot program to detect plagiarized text was informative for participants
to better understand how to represent the intellectual contributions of others.
In FY20I8, there were fifty-three submitted queries about scientific integrity. Eigh-
teen queries were evaluated in FY20l8.Two of eleven allegations of a loss in scientific
integrity at EPA were resolved. While nine of the allegations were found to have not
been losses in scientific integrity, it is vital that everyone, internally and externally, has
an opportunity to be heard and their concerns addressed.
In FY20I9, the ScIO, the Program, and the Committee will continue their commit-
ment to promote a culture of scientific integrity, the timely release of information
to the public, the appropriate use of peer review and federal advisory committees,
and the professional development of EPA employees.Annual and ongoing initiatives,
such as the Employee Conversation with the Scientific Integrity Official and evalua-
tion of scientific integrity queries, will continue. FY 19 will bring with it the debut of
new outreach materials and publications including additional whiteboard videos, a
-------
new electonic clearance system, increased access to scientific integrity expertise, and
the release of a Differing Scientific Opinions policy.These initiatives will contribute
to increasing the visibility of scientific integrity at EPA, the Agency embracing and
modeling scientific integrity, and protecting and maintaining EPA's culture of scientific
integrity.The completion of the OIG audit, and GAO engagement, will also provide
analyses of Policy implementation and scientific integrity at EPA.These results and
recommendations will guide futur e scientific integrity initiatives and fortify EPA's
culture of scientific integrity.
EPA's successful ability to protect human health and the environment requires a cul-
ture of scientific integrity.This means science that is conducted with the highest rigor
and using independently validated scientific methods, science that is independent of
policy implications, is supervised and utilized with integrity, and is communicated in a
timely, transparent manner.
-------
Acronyms
i
AD Anaerobic Digestion
ADR Alternative Dispute Resolution
APD Air Protection Division
AV Assistance Visit
CASAC Clean Air Scientific Advisory Committee
CASTNET Clean Air Status and Trends Network
CIO Chief Information Officer
DC Data Competency
DRA Deputy Regional Administrator
DScIO Deputy Scientific Integrity Official
DSO Differing Scientific Opinion
EAID Environmental Assessment and Innovation Division
EPA US Environmental Protection Agency
EQMD Environmental Quality Management Division
FACA Federal Advisory Committee Act
FBT Freshwater Biology Team
FEMA Federal Emergency Management Agency
FIT Field Implementation Team
FMFIA Federal Managers Financial Integrity Act
FY Fiscal Year
GAO US Government Accountability Office
GIS Geographic Information Systems
GRTS Grants Reporting and Tracking System
ICR Information Collection Request
IDP Individual Development Plan
IND Improvised Nuclear Device
ISO International Organization for Standardization
-------
LD 18 Liberty Down 2018
MOVES Motor Vehicle Emission Simulator
NARS National Aquatic Resource Surveys
NCER National Center for Environmental Research
NEIC National Enforcement Investigations Center
NERL National Exposure Research Laboratory
NMDA N-methyl-D-aspartate
OAR Office of Air and Radiation
OCFO Office of the Chief Financial Officer
OCGR Office of Communications and Government Relations
OCSPP Office of Chemical Safety and Pollution Prevention
OECA Office of Enforcement and Compliance Assistance
OEI Office of Environmental Information
OEIP Office of Enterprise Information Programs
OGC Office of General Counsel
OGWDW Office of Groundwater and Drinking Water
OIG Office of Inspector General
OLEM Office of Land and Emergency Management
OMB Office of Management and Budget
OP Office of Policy
ORD Office of Research and Development
OSA Office of the Science Advisor
OSIM Office of Science Information Management
OST Office of Science and Technology
OW Office of Water
OWOW Office of Wetlands, Oceans and Watersheds
PFAS Per- and polyfluoroalkyl substances
PIE Planning, Implementation & Evaluation System
PM Particulate Matter
-------
PMC PubMed Central
PPG Performance Partnership Grants
PWS Public Water System
QA Quality Assurance
QAFAP Quality Assurance Field Activities Procedure
QAPP Quality Assurance Project Plan
QC Quality Control
R2P2 Regional Research Partnership Program
RCRA Resource Conservation and Recovery Act
RDX Royal Demolition Explosive
RESES Regional Sustainable Environmental Science program
RFR Request for Reconsideration
ROCS-Net Regional ORD Community of Science Networking program
RSC Regional Science Council
RSL Regional Science Liaison
RVP Reid Vapor Pressure
SAB Science Advisory Board
SABSO Science Advisory Board Staff Office
Scl Scientific Integrity
ScIO Scientific Integrity Official
SDM Scientific Data Management
SESD Science and Ecosystem Support Division
SFM Sustainable Food Management
SOP Standard Operating Procedures
TMDL Total Maximum Daily Load
TSCA Toxic Substances Control Act
UCMR Unregulated Contaminant Monitoring Rule
VADEQ Virginia Department of Environmental Quality
WPD Water Protection Division
-------
-------
Appendices
<
Appendix I
Annual Employee Conversation on Scientific Integrity
Introductory Remarks
Dr. Francesca Grifo (ScIO), welcomed the participants and introduced Dr. Jennifer
Orme-Zavaleta (Science Advisor and Principal Deputy Assistant Administrator for
Science, Office of Research and Development). Dr. Orme-Zavaleta remarked that
science helps to inform decisions and Scl is the core value of the Agency. Dr. Orme-
Zavaleta encouraged participants to contact Dr. Grifo and the Scl Committee for any
questions or concerns. Scl remains a topic of interest for the current U.S. government
administration. Mr. Scott Pruitt (EPA Administrator) has expressed his commitment to
upholding EPA's Scl Policy.
Dr. Grifo acknowledged EPA staff assisting the Scl Committee—Ms. Martha Otto (EPA/
Office of the Science Advisor [OSA]), Dr. Cheryl Hawkins (OSA) and Mr. Daniel D'Arcy
(Contractor to OSA). A special thanks was given to Dr. Kevin Teichman (Senior Science
Advisor, Office of Research and Development) who served as the interim Committee
Chair in Dr. Grifo's absence. Dr. Grifo expressed her gratitude to staff who contributed
to her medical leave bank and encouraged attendees to participate in EPA's Skills
Marketplace, where the Scientific Integrity Program has an open project announcement.
Scientific Integrity Policy Highlights
The purpose of the Scl Policy is to ensure adherence to professional values and
practices when conducting, managing, influencing, utilizing, and communicating science.
These guidelines apply to all EPA scientists, managers, and political appointees. Dr. Grifo
commented that research integrity must be managed and supervised. It is important
that objectivity, clarity, reproducibility, and utility are maintained and that individuals
are insulated from bias, fabrication, falsification, plagiarism, outside interference, and
censorship.The Policy states that ensuring Scl involves validating scientific methods,
retaining qualified scientists, accrediting laboratories and facilities, providing information
on quality and quality assurance, a peer review process with no conflicts of interest, and
the free flow of science with a timely release of research results. Dr. Grifo remarked that
the Agency is committed to establishing public trust of EPA science, transparent scientific
processes, and open scientific communication. Scientists must be allowed to perform
research in an environment that upholds Scl.
Highlighting the various strengths of the Policy, Dr. Grifo stated that it prohibits all
EPA employees from suppressing, altering, or otherwise impeding the timely release of
Chair: Dr. Francesca Grifo, Scientific Integrity Official
Tuesday, June 12,2018
3:00-4:00 p.m. EDT
-------
scientific findings.The Policy requires EPA to expand and promote access to scientific
information and prohibits Agency leadership from intimidating or coercing scientists
to alter scientific data, findings, or professional opinions. Employees are permitted to
express their personal opinions, albeit they must adhere to exceptions (e.g., no lobbying
or electioneering). Another strength is that the Agency allows researchers to review
documents that substantively rely on their science for accuracy before release.
Dr. Grifo clarified what constitutes Scl. She described the difference between policy
versus science and commented that a change in an employee's assignments and a limit
to opportunities for travel may not be scientific integrity issues.The Policy indicates that
allegations could be reported to the ScIO, any of the 24 Deputy SclOs, or the Office of
Inspector General (OIG) hotline. Participants are encouraged to review the Scl policies
and guidelines.
Dr. Grifo mentioned EPA's Scientific Misconduct Policy, which is adjudicated by the
Agency's OIG and outlines procedures for addressing research misconduct (i.e.,
fabrication, falsifying research data, or plagiarism).
Allegations Update
The Scl Program received 137 reported allegations from February 2012 to March
3 1,2018. In total, 23 are active, 57 adjudicated (3 I were substantiated and 26 were
dismissed), 12 are "unable to proceed", 26 were withdrawn, 12 were not Scl related
and 7 were reassigned. Most allegations were related to suppressing/delaying report or
information, interference with science by a manager, authorship, and conflict of interest.
Dr. Grifo commented that fewer authorship allegations were received, perhaps because
the Committee published its best practices for designating authorship.The external
allegations that were formally submitted exceeded the number of those submitted
informally; internal allegations were mainly informal submissions.
New Procedures for Allegations
Dr.Vincent Cogliano (Deputy to the ScIO) outlined the new procedures for reporting
allegations.This process emphasizes transparency, confidentiality, consistency, timeliness,
effectiveness, and fairness.The allegation process contains two paths: advice and
assistance and a procedure for reporting allegations.The purpose of advice and
assistance is to avert allegations early and with minimal senior-level organizational
involvement. Someone with a scientific integrity concern can receive coaching to
address an issue before it rises to the level of an allegation. If an allegation is reported,
it is screened, an inquiry is conducted, and a review panel may be convened to
determine whether a violation has occurred.The determination may also recommend
corrective action and preventive measures as needed.The allegation procedure
includes a description of topics covered and not covered, a statement of safeguards and
confidentiality for those who report or are the subject of allegations, and targets for
timeliness:
Other Updates
Dr. Grifo informed the participants of upcoming discussions (fall 2018) with EPA
management.These conversations are aimed at informing senior-level employees of their
role in upholding a culture of Scl, encouraging good policies and practices as well as
-------
leadership. A manager who upholds a culture of Scl must demonstrate the importance of
Scl to achieve EPA's mission of protecting human health and the environment. Managers
also should ensure that those who look to them for leadership accomplish their work
in a manner that upholds Scl. Managers should hold employees accountable and support
individuals who report lapses in Scl. She encouraged participants to review the clearance
best practices on EPA's website.The goal of good policies and best practices is to
provide transparency and timeliness of procedures that prohibit delays or suppression of
science.
Dr. Grifo announced the development of an electronic clearance system, which will
incorporate the best practices for clearance of scientific products at EPA. Clearance best
practices are to be predictable, clear, transparent, consistent and timely, to prevent the
delay or suppression of science.The clearance system also will enhance the capability to
implement the plan for increasing public access to EPA-funded research.
EPA scientists and supervisors can verify the source of written work via iThenticate, a
service that is being provided by the Scientific Integrity Program.This software helps
employees to accurately attribute written documents, which in turn helps to avoid Scl
allegations.
Questions and Answers
In discussion, the following points were made:
Dr. Grifo reminded the participants that before reporting, potential allegators must
determine whether their actions will impact the science.
A participant asked about the enforcement of Scott Pruitt's proposed Transparency Rule.
Dr. Grifo replied that this rule has not been finalized and a public meeting will be held
in July 2018, to discuss this proposal. A participant questioned whether the Transparency
Rule could hinder science. Dr.Tom Sinks (Director, OSA) replied that a public hearing
will be conducted on July 17,2018, and the comment period has been extended to
August 16,2018. Dr. Grifo added that it is important for individuals to provide specific
instances when submitting their comments.
Dr. Grifo clarified a participant's question regarding allegations that concern the EPA
administration. Dr. Grifo alluded to the allegation from the Sierra Club regarding a public
comment made by Scott Pruitt. She reiterated the point that all Agency employees are
encouraged to express their personal opinions. EPA encourages a free flow of ideas and
information.
In response to a question regarding EPA's Regional Science and Technology laboratory,
Dr. Grifo replied that the Committee is open to address their Scl-related issues.Any
group that believes that their issue is related to scientific integrity should report an
allegation.
32-33
-------
A participant asked about the scientific integrity awareness of EPA political appointees.
Dr. Grifo responded that all appointees are trained in Scl as part of the Agency's
onboarding process. She re-emphasized that the goal of the Agency is to safeguard the
process that allows individuals to express their opinions.
Summary of Action Items
• Employees may review clearance procedures on the EPA Scientific Integrity Intranet
website.
• Employees may submit any potential comments on the Transparency Rule at the
public hearing session.
• Managers are encouraged to attend an Agency-wide managers training in 2019.
• All should participate in next year's Annual Employee Conversation on Scientific
Integrity; date to be announced.
-------
Appendix II
Accomplishments Across EPA
In FY 18, EPA program and regional offices took many approaches to enhance a culture
of scientific integrity at EPA. Offices throughout the Agency accomplished this by fea-
turing scientific integrity language in grants and employee handbooks, and by establish-
ing regional science councils. Employees were provided training to learn new skills and
technologies. Innovative systems transparently released data and scientific information
to the public. Efficient clearance procedures were created with an emphasis on timely
release. New quality assurance procedures streamlined the data collection and man-
agement process.Additional data was released to the public and existing public datasets
were made more user-friendly.The science that underlies EPA's decisions was reinforced
through the Agency's use of independent peer review and federal advisory committees.
EPA employees also expanded their knowledge, learned new skills, and remained leaders
in their fields by participating in professional societies, attending conferences, and speak-
ing on expert panels.The following are examples of scientific integrity accomplishments
across the Agency in FY 18.
Promoting a Culture of Scientific Integrity
A culture of scientific integrity strives for transparency, limits management review to
scientific quality considerations, and encourages robust scientific discourse. A culture of
scientific integrity uses independently validated methods, accredited labs, and certified
data. In a culture of scientific integrity, science is conducted, supervised, communicated,
and utilized with honesty, transparency, and integrity.
Upholding a Culture of Scientific Integrity
The Office of Research and Development (ORD) upheld a culture of scientific integrity
by expanding scientific integrity language to grants controlled by the National Center
for Environmental Research (NCER), while the Office of Chemical Safety and Pollution
Prevention (OCSPP) included reviewing the Policy in its onboarding handbook. Region 8
enhanced its Regional Science Council in its second year and Region 4 re-established its
own Regional Science Council.
Office of Research and Development (ORD)
National Center for Environmental Research (NCER)
Under the guidance of the Agency's Scientific Integrity Program, NCER staff (Sheryl
Law) worked with the Scientific Integrity Program staff to ensure that the Agency's
scientific integrity culture and principles were incorporated in the "Terms and Con-
ditions" of the assistance agreements that NCER manages. For more information,
see the EPA Research Grants webpage.ix
-------
Office of Chemical Safety and Pollution Prevention (OCSPP)
The Office of Pesticide Programs (OPP) developed a new employee handbook that was
deployed in June 2018.This handbook includes a manager's and an employee's checklist
of what to do during the first months on the job. Both checklists include,"Review scien-
tific integrity information at [OSA's scientific integrity landing page.]x" The handbook can
be found on the OPP@work website."
Region 4
In FY 18, the Region 4 Regional Science Liaison (RSL) re-established the Regional Science
Council (RSC).The RSC is a group of Regional scientists (each Division/Office has repre-
sentation) who meet monthly to discuss issues, work on specific projects of Region-wide
interest, and make recommendations to management concerning matters dealing with
science. Activities include:
~ Serving as a forum for the identification and communication of the scientific,
technical support, and research needs of the Region and its partners (States,
Tribes, Local, etc.);
~ Developing and keeping current a list of the Region's science issues and needs;
~ Maximizing opportunities for funding and training (e.g., RARE, RESES, R2P2,
ROCS-Net) to address the Region's science issues and needs;
~ Providing scientific support to national and regional workgroups, task forces, and
other such entities;
~ Serving as a scientific sounding board to the Regional Senior Leadership Team on
emerging scientific issues;
~ Working to improve communication between the Region and ORD;
~ Serving as a forum for the identification and implementation of scientific training
/educational programs that benefit the Region and their partners.
Region 8
In FY 18, the Region 8 Science Council entered its second year of operation.The Coun-
cil's mission is to ensure the continued enhancement of science capacity in Region 8 by:
~ Serving as a resource to staff and managers to assist with integrating sound
science in the decision-making process;
~ Enhancing the Region's ability to provide new and current employees with the
scientific and technical skills they will need as their careers develop;
~ Enhancing communication and coordination on science activities among
program offices and with Regional leadership; and
~ Serving as a clearinghouse for science activities in the Region, with a goal of re-
ducing redundancy.
-------
36-37
Examples of Science Council accomplishments in FY 18 include:
~ The Council elected its first class of new members, a total of six staff.
~ To provide room for the new members, six members stepped down to
emeritus status;total membership is 17 active and 6 emeritus (I retired)
members.
~ The Council also elected a new Chair and Vice-Chair in FY 18.
~ The Council held an all-day retreat in January 2018 to develop a work plan
for the upcoming calendar year.
The following Science Council Committees were active in FY 18:
~ Science Integrity Committee:
• Completed development of draft Clearance/Peer Review process for
Region 8.
• Next steps include briefing DRA and Regional leadership.
~ Science Seminar Committee:
Committee organized and held several seminar presentations in FY 18.
Examples include:
• Explorers for Bats: film screening and expert panel, citizen science and bat
conservation.
• EPA Region 8 Air Program and ORD Research Collaborations.
• ROCS-Net and NMDA: Attempting to address challenges of an unregulat-
ed contaminant in drinking water.
• Addressing Regional Research Needs through ORD Collaboration: PFAS
point of entry treatment and metals inhalation exposure at mining and
mineral processing sites.
• Application of bioanalytical tools to identify sources and effects of con-
taminants in surface water.
• Committee also hosted webinars to facilitate staff participation and
discussion in-house, as opposed to individuals participating at their desks.
Example:
• Community powered citizen science: How people use sensors to test air
and water across the globe.
~ Science Needs Committee:
• Committee continued to increase awareness and promote use of elec-
tronic science needs form.The intent of the form is to provide a mecha-
nism for staff and management to seek assistance on science issues.
~ Communication Committee:
• Committee continued to advertise science council seminars and events
on the 8-net, atrium monitor, via email, and by other means (e.g., lobby
meet-and-greet).
-------
Committee continued to track science council accomplishments over the
course of the year, including attendance at seminars and events.
A priority for Region 8 is implementing EPA's QA Field Activities Procedure (QAFAP).
This initiative affects all programs involved in field activities and includes every office,
the Region 8 Laboratory, and both field offices in Montana and South Dakota. In FY 18,
the following were key focus areas to assure the integrity of Region 8 field activities: (I)
institutionalize/optimize QAFAP policies/procedures; (2) conduct training/continuing ed-
ucation; (3) further strengthen field work through workgroup collaboration; (4) perform
annual internal audits; and (5) address findings through corrective actions.
~ Revised Overarching QAFAP Standard Operating Procedures (SOPs) - The Re-
gion 8 Field Operations program revised the Regional Overarching QAFAP SOPs
to reflect streamlined processes as well as to address findings during internal and
external assessments.All procedures and other resources, including the updated
SOPs, are available on Region 8's intranet/SharePoint site.xii
~ Workgroup Collaboration - Field Implementation Team (FIT) - The FIT, which
is comprised of members from all Regional program offices where field activ-
ities occur, meets monthly during the field off-season (October through April)
to strengthen integration of the QAFAP standards into field activities that are
conducted by EPA Region 8 personnel.The FIT accomplished this by: (I) updating
SCPs as needed; (2) advising on technical field issues/challenges; (3) leading/train-
ing their programs in QAFAP requirements; and (4) assessing QAFAP implemen-
tation by serving as auditors in internal audits.
~ Updated Regional Digital Image SOP - The FIT members collaborated to update
the Regional SOP for taking, recording, and managing digital images to reflect new
guidance from the Office of Enforcement and Compliance Assistance (OECA).
This exceptional practice ensures consistency and continuity in digital image man-
agement across programs.
Training
Ensuring data integrity is critical to maintaining a culture of scientific integrity.The Na-
tional Exposure Research Laboratory (NERL), Region 4, and Region 8 provided training
for their scientists to improve data-collection and quality assurance strategies in the field.
Office of Research and Development (ORD)
National Exposure Research Laboratory (NERL)
The NERL QATeam training, along with a training facilitator, conducted team train-
ing to discuss training for the newly published NERL Quality Management Plan
(QMP) and to initiate plans to write NERL specific Quality Assurance (QA) Program
SOPs, and to identify a process to ensure each project has an approved Quality
Assurance Project Plan (QAPP).
-------
Region 4
Environmental Sampling and Analysis Course:The Region 4 Superfund Division collabo-
rated with the Science and Ecosystem Support Division to develop a 3-day course that
familiarizes project managers with the field and laboratory aspects of the site work.
Through classroom instruction, hands-on field demonstrations, and detailed tours of the
Regional laboratory, the course provides participants with the tools needed to improve
data collection strategies and to optimize the use of available investigative techniques for
site decision making. In the initial offering of the course, 19 Region 4 staff attended.
Region 8
Deliver FY 18 Annual QA Field Activities Procedure (QAFAP) Training - The Region 8
Field Operations program distributed QAFAP training to all field personnel and their
supervisors.This I-hour course was delivered online through a learning management
system.To receive credit for the course, individuals were required to: (I) complete a
knowledge test with a score of 90% or better; and (2) document/acknowledge review of
the updated Regional Overarching QAFAP SOPs.
Data Management
EPA's ability to protect human health and the environment is heavily dependent on the
quality of its data. As part of EPA's Public Access Plan,The Office of Research and De-
velopment (ORD) continued its effort to improve its ScienceHub System to transpar-
ently release scientific data to the public.The Office ofWater (OW) released the new
Assessment Total Maximum Daily Load (TMDL) Tracking and Implementation System
(ATTAINS), improved its Section 3 19 Grants Reporting and Tracking System (GRTS), and
provided additional data for public consumption on EPA's website.The Office of Enforce-
ment and Compliance Assurance (OECA) and the Office of Environmental Information
(OEI) fortified their testing procedures. Modern and efficient data collection methods
and storage systems were created in Regions 3,4,6, and 8.
Office of Research and Development (ORD)
Office of Science Information Management (OSIM)
OSIM continued to work with other ORD Assessable Units (AUs) to implement
its newly promulgated ORD Scientific Data Management policy and continued
to improve the related ScienceHub portal.™'The OSIM-managed ScienceHub is a
system that is used to help manage ORD's research data throughout the life of a
research project. Data and metadata are made publicly available in accordance with
EPA's Public Access Plan, and better guarantees the transparency of and easy access
to ORD's scientific data used in published articles and documents. In this way, OSIM
helped ORD to collaborate and meet data transparency requirements and the ex-
pectations of external customers.
-------
Office of Enforcement and Compliance Assurance (OECA)
The National Enforcement Investigations Center (NEIC) is working with the Office of
Land and Emergency Management (OLEM) to improve and strengthen the testing re-
quired to determine the Resource Conservation and Recovery Act (RCRA) character-
istic of ignitability.This work led to the recent publication of a new American Society for
Testing and Materials (ASTM) method for making a RCRA determination.
Office of Environmental Information (OEI)
New Chief Information Officer (CIO) Quality Directive: Issued Final CIO Notification
Procedure for Environmental Data Quality Issues; CIO 2105-P-03.0; April 06,2018.This
procedure describes the due diligence process that EPA organizations are to follow in
evaluating potential risks and impacts from data quality issues that may adversely affect
EPA's environmental data operations.The EPA ScIO will be informed of any potential
scientific integrity concerns.
Office of Water (OW)
o In January 2018, occurrence data collection began under the Unregulated Contami-
nant Monitoring Rule (UCMR 4).The Office of Groundwater Drinking Water (OG-
WDW) oversees that program, which involves gathering data on 30 contaminants of
emerging concern from all large public water systems (PWSs) and a representative
set of small PWSs. Beginning with this monitoring cycle, EPA has integrated Quali-
ty Control (QC) data collection into the web-based UCMR reporting system.This
ensures that laboratories meet QC criteria before they can successfully submit their
data.
o In FY20I8, the Office of Wetlands, Oceans and Watersheds (OWOW) released the
new Assessment Total Maximum Daily Load (TMDL) Tracking and Implementation
System,xiv which streamlines the Integrated Reporting process and allows for more
accurate, transparent data reporting. Improvements include improved tracking of as-
sessment units from one Integrated Reporting cycle to the next, as well as improved
identification of assessment units with TMDLs, alternative restoration, or protection
approaches. States now have the flexibility to use different approaches and tools to
measures progress in water quality in ways that work best for them.The EPA can
process the state data to calculate a common unit of measure to provide consistent
reporting across all states. States are beginning to use this new system for reporting
their 2018 303(d)/305(b) Integrated Reports, and the Regions are using this new
system to review and approve the state 303(d) lists.This system will also be critical
to automating the report of agency metrics on water quality and communicating this
information to the public.
o The Office ofWetlands, Oceans, and Watersheds (OWOW) continued to enhance
its Section 3 19 Grants Reporting and Tracking System (GRTS) and incorporated a
module for the submittal and review of Section 3 19 success stories, their primary
program metric of water quality success. Previously, Success Story submittal was
-------
40-41
done via email, with much back and forth to fill any data gaps and finalize the sub-
mittal.The new Success Stories database tracks workflow between the states, EPA
regions, and headquarters and uses a set of mandatory fields that help to reduce
data entry errors.To count as a success story, states and EPA must follow guidance
that requires scientifically-sound documentation of the water quality impairment
and the improvement of water quality to achieve water quality standards.To further
ensure a rigorous process, the Nonpoint Source Program will accept a Success Sto-
ry only if the water is expected to meet the requirements for de-listing under the
303(d) Program.
o OW conducted an ELMS (EPA Lean Management System) exercise to streamline
the data processing, review, and interpretation activities to improve delivery of data,
metadata, and results to their state and tribal collaborators as well as the public.
OWOW expanded use of electronic field data apps for tablet devices enabling
state, tribal, EPA, and contract field crews to collect data electronically and submit it
directly to EPA for upload to the data management system.The use of the field apps
and tablets enhance the quality of data and speed input of data into the National
Aquatic Resource Surveys (NARS) database.
o OW actively supports increased web-based reporting of results and findings from
national water quality assessments, an important complement to local assessment
unit results. OWOW continued efforts to provide NARS data through interactive
dashboards as a means of providing key measures of national water quality condition
and changes over time for external exploration and transparency. Building from the
successful launch of the National Lakes Assessment Interactive Dashboard™ and the
National Coastal Condition Assessment results in a similar format at https://coast-
alcondition.epa.gov/, OW developed a draft dashboard for the National Rivers and
Streams Assessment, which will be released with the report.
Region 3
Land and Chemicals Division (LCD)
The Land and Chemicals Division (LCD) developed an Information Collection Re-
quest (ICR) that will allow the Sustainable Food Management (SFM) Program to ob-
tain capacity data for anaerobic digestion (AD) facilities processing wasted food.The
SFM program promotes diversion of organic wastes (including wasted food) from
landfills.This ICR gives EPA the ability to establish a baseline capacity for process-
ing organic materials in anaerobic digesters.The SFM Program will then trackAD
processing capacity over time to gain a better understanding of the factors affecting
the growth of the AD industry.The information collected will also help the program
develop future activities designed to further increase capacity. Region 3 generated
the Report of Findings for the first year of data collection (2017) under the ICR.
This product was cleared through Region 3 and released in May 2018.The report
is available at this URL: https://www.epa.gov/anaerobic-digestion/anaerobic-diges-
tion-tools-and-resources#ADdata.
-------
Water Protection Division (WPD)
The Water Protection Division (WPD) Drinking Water Branch developed a Qual-
ity Assurance Project Plan (QAPP) as part of a Regional Applied Research Effort
(RARE) project to determine how to collect and analyze water samples. A mod-
eling QAPP was also developed to strategize how to develop models to predict
total trihalomethane exceedances. In addition, as part of the file review process, the
Drinking Water Branch works to detect discrepancies between Public Water Sys-
tem (PWS) data in the primacy agency flies/database and the data reported to Safe
Drinking Water Information System (SDWIS) Federal
Reporting System, and to ensure that the primacy agency is determining
compliance in accordance with federal regulations.
Region 4
Superfund Division (SD)
~ The Region 4 Superfund Division leveraged two additional EQulS technology
tools from Earthsoft in FY 18 to further develop their database of environmental
data.The Sample Planning Module (SPM) was implemented to track field events
and identify potential sources of environmental data for loading to the database.
The EQulS Data Gathering Engine (EDGE)xvi is being pilot-tested during FY 18 to
expand Region 4's capabilities to collect electronic data in the field in a database
ready format.
~ Asbestos Soil Methods Study: Region 4 Superfund Division staff continued work
on this effort, which is a collaborative effort with Region 10, the Office of Re-
search and Development, and State partners that began under a Regional Applied
Research Effort (RARE). In FY 18, soil samples were collected from sites in Region
4 and Region 10 by using incremental soil sampling methods.The samples will be
used to demonstrate best practices for sample collection and to critically eval-
uate sample processing and analytical methods.The generated data will inform
the Agency in improving methods for collecting and analyzing soil samples from
asbestos-contaminated Superfund sites. A Region 4 Risk Assessor and On-Scene
Coordinator (OSC) will present on the applicability of these efforts at a meeting
of southeastern state asbestos programs in September FY 18.
Water Protection Division (WPD)
To meet the requirements under the "EPA Policy to Assure the Competency of
Organizations Generating Environmental Measurement Data Under Agency-Funded
Assistance Agreements" or what is commonly referred to as the Data Competency
Policy, the Water Protection Division (WPD) worked across the divisions in Re-
gion 4 to create a process that satisfies the data competency (DC) requirements
for all the programs within each state.This new process eliminates the need for
grantees to submit multiple annual DC packages for every grant issued >$200K in
federal funds for collecting, generating, and/or using environmental data.This stream-
-------
lined process requires a cumulative and comprehensive DC package to address all
programs under Performance Partnership Grants (PPGs) and individual grants.To
ensure states have a consistent point of contact for the PPG, the PPG Coordina-
tor collects and disperses the documentation to the divisions for the review and
approval.The DC packages for grantees that are managed within the WPD, approx-
imately 35 grantees, will be collected by WPD Project Officers, and then reviewed
and approved by WPD Quality Assurance staff. The WPD QA Coordinator will
prepare a written procedure to reflect current practices in FY 19.
Region 6
o Initiated "real-time" (within 2-3 days) inspection report generations in the lead-
based paint program
o Converted Underground Injection Control (UIC) well data base from paper to elec-
tronic format.This allows for more efficient analyses.
Region 8
The Region 8 Superfund program developed Electronic Data Capturing Devices (EDCD)
to allow automated, real-time data collection and transfer from the field.This reduces
the potential for errors associated with manual data collection
methods.
Clearance Procedures
Clearance procedures increase transparency in the release of research results, ensure
timely review, and discourage unreasonable delays.They also ensure that scientific prod-
ucts are reviewed by the appropriate supervisors and technical managers before they are
released to the public. Using the guidance provided in the Best Practices for Clearance
of Scientific Integrity Products at EPA,The Office of Air and Radiation (OAR) and Region
3 developed clear and consistent practices for clearing their products. A new Standard
Operating Procedure (SOP) for clearance was also utilized in the Office of Chemical
Safety and Pollution Prevention (OCSPP).
Office of Chemical Safety and Pollution Prevention (OCSPP)
The Office of Science Coordination and Policy (OSCP) began using the Document
Clearance SOP for OCSPP in FY 18.The SOP covers the clearance of scientific products
that are developed by an EPA author, or a group of authors including at least one EPA
author, as part of his/her official duties.These include journal articles, meeting presenta-
tions, and other scientific products that have an EPA employee listed as an author.The
SOP was finalized on May 10,2018.
Office of Air and Radiation (OAR)
OAR evaluated its processes for the review and clearance of scientific products devel-
oped by OAR staff, against the recently issued Best Practices for Clearance of Scientific
-------
Products at EPA.That document provided useful suggestions that have helped OAR to
develop a more consistent process for clearance of scientific products across its oper-
ations, identifying a common set of information elements to be addressed by all OAR
programs in the clearance of such documents.
Region 3
The Region 3 DScIO, Jennifer Fulton, housed in the Environmental Assessment and Inno-
vation Division (EAID), drafted Region 3 Clearance Procedures for Release of Scientific
Products to provide clarification and expectations for employees who release scientific
products externally.This document was developed using the Best Practices for Clear-
ance of Scientific Products at EPA, as established by the Committee and distributed by
the SclO.The clearance procedures were drafted to meet the goals of the EPA Scientific
Integrity Policy, which states that,"Each Program Office and Regional Office will develop
and document procedures for review and approval, consistent with the Scientific Integ-
rity Committee's framework.The procedures will include guidance for review elements,
time frames for review and approval, and a process for redress if review procedures are
not met." The ScIO asked DSclOs to consider how the Best Practices related to their
office's current clearance
procedures, with the goal of using the document to assist in developing, evaluating, or
revising clearance procedures to promote transparency, clarity, timeliness, predictability,
and consistency.
Quality Assurance
A variety of mechanisms work to ensure the quality and integrity of EPA scientific prod-
ucts. Quality Assurance Project Plans (QAPP) and Standard Operating Procedures (SOP)
contribute to a culture that emphasizes the validity of scientific information. In FY 18,
innovative QAPPs and SOPs were implemented across the Agency including a Citizen
Science Quality Assurance Project Plan Handbook developed by Region I.
Office of Research and Development (ORD)
National Exposure Research Laboratory (NERL)
~ The NERL QATeam is actively involved in the cross ORD collaboration of sev-
eral highly visible projects including PFAS and Lead, and they have worked with
other labs to standardize the application of QA on these research efforts.
~ In FY 18, 12 NERL Quality System SOPs were developed. Of these SOPs, eight
have completed NERL staff/union review.Among these eight, three were ap-
proved for implementation by NERL management.A NERL intranet site was also
developed to allow all NERL researchers access to technical SOPs.
-------
Office of Enforcement and Compliance Assurance (OECA)
The National Enforcement Investigations Center (NEIC) is continuing to work on a
special project to continuously improve and better document their quality control sys-
tems. NEIC's Quality group is facilitating the development of a Quality Control Platform
(QCP) that will be beneficial in tracking consistently-gathered quality control data across
projects for select analytical methods.The QCP allows for the monitoring of data quality
on a regular basis by applying user group and method defined acceptance criteria, along
with collecting quality control (QC) data to allow for performance evaluation and trend
analysis.The scope of the development of a QCP is focusing on developing systems that
will create efficiencies and consistency in evaluating QC measurement data.
Office of Environmental Information (OEI)
o Quality System Assessments: EPA's quality management system specifies periodic
assessment of implemented environmental data programs to assure effectiveness
and conformance to national requirements and EPA's policy.These assessments
identify best practices, potential vulnerabilities, and opportunities for improving the
production and use of scientifically sound data and information.The Office of Enter-
prise Information Programs (OEIP) assessed two regions and two program offices in
FY20I8.
o EPA Laboratory Competency: OEIP will compile reports for the Office of the
Science Advisor on laboratory compliance with EPA's policy directive, "Assuring
the Competency of Environmental Protection Agency Laboratories™" for activities
conducted in FY 2018.To assure competency, all EPA-operated laboratories, includ-
ing government-owned contractor operated laboratories, are required to maintain
a documented Quality System that at a minimum complies with the requirements
of the EPA Quality System. As a component of the OEI annual Quality Assurance re-
porting process, OEIP receives the individual reports on documentation of indepen-
dent assessments and participation in inter- laboratory comparisons or programs,
consolidates the information, and sends a report to the Forum on Environmental
Measurement, which is then shared with the SclO.
o Quality Community Information Exchange: OEIP hosts monthly conferences with
the EPA Quality Assurance Community (the National Program Offices, Office of
Research and Development, and Regions). During these meetings, OEIP addresses
topics about quality processes and scientific expectations for the data and informa-
tion used to support agency decisions.
Office of Water (OW)
o Issued/edited/revised multiple QAPPs and SOPs.
o Participated in internal assessments of various programs to judge conformance with
the Agency's Field Operations Guidelines
o Prepared for FY 2018 internal Technical System Assessments of OGWDW's labora-
tory work
-------
o Audited the contract laboratories engaged to analyze UCMR 4 samples from small
PWSs
o OW developed and implemented Quality Assurance Project Plans for both contract
work and Agency work, including the EPA/State National Aquatic Resource Surveys.
Region I
Region I took the lead in developing a Citizen Science Quality Assurance Project Plan
Handbook, which provides citizen scientists with tools and procedures that can assist
them with documenting information about the data being collected. It also conveys
information about the appropriate quality assurance documentation for environmental
studies.
Region 2
o Facilitated foilow-up by an EPA Region and OEI-EQMD of a reported lab fraud issue
that could have impacts on multiple EPA organizations, including Region 2.
o Developed courtesy QAPP (or other QA documentation) review response/dis-
claimer language for environmental projects when EPA does not provide funding or
use the data (e.g. some citizen science, voluntary),
o Began a series of periodic QA Training/Outreach to all Regional personnel titled:
Let's Dish Up Some QA PIE.
o Facilitated 2018 on-site and webinar training through the Region 5 2018 QA Work-
shop, attended by Region 2 Quality Assurance Officers (QAOs), Region 2 Tribes, and
New York State Department of Environmental Conservation personnel,
o Participated with OEI-EQMD in the Region I Quality System Assessment,
o Participated in: a national Citizen Science QAPP Work Group, developing a QAPP
handbook, templates and examples; the OEI-led EtQ QA Enterprise Management
System Pilot Work Group; and an OEI-EQMD-led e-Enterprise/Lean State/Tribal
QAPP Initiative.
Region 3
o Air Protection Division (APD): Quality Assurance Project Plan Tracker Updates -
APD further updated their QAPP database.These updates are based on suggestions
of monitoring staff, to better capture information they need to review and track
QAPPs.
o National Rivers and Streams Assessment (NRSA) - NRSA, part of the National
Aquatic Resource Survey (NARS), is a probabilistic survey of the nation's streams
and rivers collaboratively sampled by states, tribes, or subcontractors.To ensure
quality data collection, each sampling team completes a 4-day intensive training and
is then assessed during an Assistance Visit (AV). During AVs, assistance to states,
regions, and tribes in collecting quality data is emphasized, rather than an audit or a
grade of sampling ability. For the FY 18 AV, the Office of Monitoring and Assessment
Freshwater Biology Team (FBT) was accompanied by a representative from Great
Lakes Environmental Consulting, who is contracted by EPA OWOW to provide
-------
46-47
support for NARS.The FBT successfully completed the AV and has provided AVs for
state and river basin commission sampling teams throughout the Region.
o Taxonomic QA/QC - The FBT provides annual genus-level taxonomic QA/QC for
Virginia Department of Environmental Quality's (VADEQ) monitoring and assess-
ment program.VADEQ biologists initially completed a random round-robin set,
re-identifying benthic samples collected by different regional offices. QA statistics
were calculated, and then a set of samples were sent to FBT for re-identification.
Once QA statistics were again calculated between FBT and VADEQ, reconciliation
discussions proceeded for remaining taxonomic errors. Over the past 5 years, FBT
has seen strong improvements inVADEQ's taxonomic skills and overall data quality.
FBT provides similar taxonomic QA/QC for Maryland and West Virginia.
o Significant work was completed by the staff of EAID's Office of Analytical Services
and Quality Assurance (OASQA) to streamline and better communicate the pro-
cesses for validating environmental organic and inorganic data. Currently, many of
the protocols and procedures on data validation are not well circulated or rely
heavily on professional judgement of the data validator.This effort is to clearly de-
scribe all data validation protocols and ensure training is conducted to all staff.This
project is anticipated to be completed in FY 19.
o Engaged the National Enforcement Investigations Center (NEIC) - (Denver, CO) to
analyze 2 sites (S.H. Bell located at East Liverpool, Ohio and Glasgow, Pennsylvania);
and Palmerton/AZR, (Palmerton, PA) for hazardous pollutants collected from ambi-
ent air monitors and onsite premises. NEICs reports helped bolster enforcement
cases.
Region 4
Science and Ecosystem Support Division (SESD)
In 2017, SESD's Field and Analytical Services Branches completed a merger of the
two previously independent quality systems into one uniform system supporting all
SESD operations.The culmination of this merger was the successful completion of
the remote assessment performed byANSI-ASQC National Accreditation Board
(ANAB) that evaluated SESD's newly merged quality system.The assessment con-
cluded SESD's laboratory and field quality management system continues to operate
as intended, complying with all ISO 17025 and Forensic accreditation requirements.
In FY 18, SESD staff conducted a combined field and laboratory internal audit of the
laboratory and field accreditation procedures and processes.This was the first SESD
audit to utilize audit teams comprised of field and laboratory staff paired together
to evaluate the administrative and technical requirements contained in SESD's ISO
17025 Accreditation to determine conformance with those requirements. One
corrective action was initiated based on audit findings.The combined internal audit
proved beneficial on several levels.The audit was more efficient and generated feed-
back from different perspectives because of the diversity of the audit teams.
-------
Water Protection Division (WPD)
TheWPD updated a Quality Assurance Field Activities Procedure, National Pollutant
Discharge Elimination System (NPDES) Permitting and Enforcement Branch Records
Management procedure, to reflect new practices within the branch.
Region 6
Ecosystems Protection Branch:Water Division (WD)
Conducted a workshop for tribes on stream monitoring, assessment, and quality
assurance - published by Office ofWater.*™
Region 8
Conducted Year I of a Multi-Year Internal Audit Cycle - Following an approved audit
plan forYear I of the internal audit cycle, the FIT members conducted and completed
an assessment of the effectiveness of the QAFAP implementation for 10 of 30 programs
that are involved in field activities. In FY 18, eight internal audits, representing ten pro-
grams, were conducted in the Denver office.The evaluation of internal and management
controls involved assessing each program's field quality management system, identifying
program vulnerabilities, and providing recommendations for identified problems, where
appropriate. Audited programs were required to develop corrective action plans by June
30,2018, to address the findings.
Release of Information to the Public
EPA encourages the transparency of Agency activities through communications tools
such as online blogs, newsletters, news releases, and official publications. EPA also main-
tains several online databases that provide open access to Agency information. Special
user interfaces allow the public to navigate EPA databases easily. Online tools including
dashboards and calculators allow users to access a variety of datasets, input their own
data, and model personalized scenarios.The Office of Air and Radiation (OAR) and The
Office of Water (OW) provided timely access to data on air quality, greenhouse gas
emissions, and watershed conditions to the public. Regional offices also clearly com-
municated scientific information, potential dangers to human health, and environmental
concerns to the residents that they serve.
Office of Chemical Safety and Pollution Prevention (OCSPP)
In June 2018,The Office of Pollution Prevention and Toxics (OPPT) released the Appli-
cation of Systematic Review in TSCA Risk Evaluation documentxix for public comment.
This document describes the implementation of these standards throughout the risk
evaluation process.This document will guide the Agency's selection and review of studies
and provide the public with continued transparency regarding how EPA plans to evaluate
scientific information.This document accompanies EPA's initial work on systematic re-
view that was described in the supplemental files for each TSCA scope document.These
-------
included the Strategy for Conducting Literature Searches** and the Bibliography for each
chemical.**'
Office of Environmental Information (OEI)
EPA Information Quality Guidelines (IQGs):The Office of Enterprise Information Pro-
grams (OEIP) oversees the administrative process for responding to the public's request
for correction of EPA's disseminated information that doesn't meet the standards for
"objectivity integrity and reproducibility" established in the Information Quality Act.
In FY 2018, OEI responded to one Request for Reconsideration (RFR).The RFR also
challenged the "scientific integrity" of the information disseminated by EPA. EPA's IQG
describes Integrity as "keeping information 'unaltered,' i.e., free from unauthorized or ac-
cidental modification or destruction." "Scientific integrity" is defined In the EPA Scientific
Integrity Policy.The Office of the Science Advisor issued a response to the requestor on
this issue, saying that no violation of the Scientific Integrity Policy had occurred.
Office ofWater (OW)
o The Office Wetlands, Oceans, and Watersheds (OWOW) continued implementing
the catchment-based indexing approach to process and use state-submitted geospa-
tial data for EPA's analytical purposes.The use of catchments allows for state-sub-
mitted geospatial data to be available in a timelier manner, while improving the
accuracy of the information being presented. Also, in FY20I8, OWOW continued
to compile metrics of watershed condition that were measured from established
national geospatial datasets and assisted states in their determination of relative
watershed health, vulnerability to impairments, and priority for management actions.
These activities followed Agency protocols, were guided by Quality Assurance Proj-
ects Plans, and were supplemented by web-based access to the data and metadata.
OWOW also began using these metrics and information to develop new methods
for communicating to the public through a new How's My Waterway Application
(currently under development) with a plan to release these new approaches in
FY20I9.
o OW and ORD jointly developed and are about to publish a document entitled
Guidelines for Measuring Changes in Seawater pH and Associated Carbonate Chem-
istry in Coastal Environments of the Eastern United States.The basis for this project
is that coastal monitoring of acidification poses a unique challenge due to greater
variability of pH in the coastal environment.The intended audience is far-reaching
-from shellfish growers interested in monitoring pH with inexpensive equipment, to
citizen monitoring groups, to advanced chemistry laboratories interested in expand-
ing their capabilities to monitor carbonate chemistry.
Office of Air and Radiation (OAR)
o OAR participated on the cross-Agency Forum on Increasing Public Access to EPA
Scientific Research and its associated workgroups. OAR took a leadership role,
co-chairing the data access workgroup created under the Forum. After the initial
48-49
-------
efforts by the Forum to ensure that peer reviewed journal articles that are authored
or co-authored by EPA staff are uploaded to PubMed Central (PMC), the OAR-ied
workgroup developed a plan for the next implementation step, to make the data un-
derlying the conclusions of those journal articles publicly accessible.This plan elabo-
rates on proposed EPA procedures to achieve this objective, describing data hosting
options as well as guidance for posting information to EPA's Environmental Data
Gateway (EDG) beginning in October 2018. OAR is also working with other Offices
to develop training and informational materials to support authors in making data
publicly accessible, as well as with ORD and OEI to expand existing data applica-
tions/infrastructure to meet Agency needs in this area. Finally, OAR has also focused
on procedures within their Office to ensure appropriate guidance and protocols are
followed.
OAR programs maintain up-to-date websites as one way of providing public access
to scientific information developed by their programs. Examples of information add-
ed in FY 18 include announcements for OAR-developed products including Green
Vehicle Guide, Fuel Economy Guide, Emissions Standards Reference Guide, Smart-
Way and Clean Diesel, and Fuel Economy Trends Report. OAR's CASTNET moni-
toring system provides public data access through EPA's website. Air Quality System
(AQS), and AIRNow, to support scientific research, inform policy and regulatory
decisions, and generate local air quality aierts.The ready availability of this data has
resulted in over 100 peer-reviewed articles citing CASTNET data in 2017.
FY 18 has also seen a major upgrade of OAR's AirNow system, which can now pro-
vide air quality data at the local level, while still providing air quality information at
state, national, and world views. A new interactive map, with zoom function, allows
the user to get the big picture or drill down to see data for a single air quality
monitor.The system proved highly useful to communities downwind during recent
wildfire events.
OAR continues to improve data on Greenhouse Gas (GHG) emissions, develops
the Inventory of U.S. Greenhouse Gas Emissions and Sinks: 1990-2016, and makes it
available on their website. OAR updates this data from key sectors through public,
transparent processes including for example, in 2018, an enhanced public review
process. Several divisions within OAR also make scientific publications available
through the Agency website.*' '
ion 3
Office of Communications and Government Relations (OCGR)
The Office of Communications and Government Relations (OCGR) plays an inte-
gral role in communicating scientific information to the public. OCGR works with
the Region's divisions and program offices to inform the public of the availability of
scientific reports and information, as well as significant public health-based decisions
rooted in scientific research and findings. In 2018, OCGR worked closely with the
Hazardous Site Cleanup Division (HSCD), Land and Chemicals Division (LCD), and
-------
50-51
the Agency for Toxic Substances and Disease Registry (ATSDR) to effectively com-
municate to communities in Philadelphia, PA, and the media about public health risks
associated with lead in soils and reducing children's exposure to lead based paint.
Best practices included:
~ Working with EPA Program offices to coordinate meetings with city and state
officials to determine collaborative ways of addressing lead hazards in
residential neighborhoods.
~ Ensuring consistent messaging about lead in soil and associated health risks when
addressing community concerns and responding to media inquiries.
~ Educating media reporters on the complexities associated with determining the
potential sources of lead in soils and EPA's limited authorities in
addressing lead in soils.
Region 4
Superfund Division (SD)
Urban Background Study - Region 4 Superfund Division continued work on this
study, which is a collaborative effort with the Brownfields Program, the Office of Re-
search and Development, and Region 4 states.This project began under a Regional
Applied Research Effort (RARE). In FY 18, background levels of polyaromatic hydro-
carbons and metals were characterized for Columbia, SC.The data that is generated
informs the Agency in distinguishing background/anthropogenic levels of contami-
nants from site-related releases of contaminants. Region 4 Risk Assessors presented
a webinar**"' in FY 18 highlighting the usefulness of the data collected to date, and the
study has been highlighted by ORD in several external communications.
Peer Review and Federal Advisory Committees (FACA)
A culture of scientific integrity ensures high quality scientific and technical products are
produced by adhering to proper scientific procedures. In FY20I8, EPA continued its ef-
forts to promote peer review and federal advisory committees as essential components
for producing high quality scientific research products.
Office of Chemical Safety and Pollution Prevention (OCSPP)
The final TSCA Risk Evaluation Process Rule requires that all draft risk evaluations
undergo peer review, and the Office of Pollution Prevention and Toxics (OPPT) uses
the Agency's Peer Review Handbook and OMB guidance for this provision. Peer review
activities in FY 18, by the Science Advisory Committee on Chemicals,*™ a FACA commit-
tee, include providing independent scientific advice and recommendations to EPA re-
garding the Exposure and Use Assessment of Five Persistent, Bioaccumulative and Toxic
Chemicals, the accompanying Supplemental Information document, and the Environmen-
tal and Human Health Hazards of five Persistent, Bioaccumulative, and Toxic Chemicals
under the authority of TSCA.
-------
Administrator's Office (AO)
Science Advisory Board Staff Office (SABSO)
The Science Advisory Board Staff Office (SABSO) has reviewed and updated annual
SOPs and guidelines related to FACA, Environmental Research Development and
Demonstration Authorization Act (ERDDAA), and General Services Administration
(GSA) requirements.The office heavily utilizes a database located in Lotus Notes,
built for the office which drives population of the website, thus, increasing transpar-
ency and the Agency's visibility to the public. By posting "real-time" information, the
public has the information it needs and provides the SABSO staff and members with
increased efficiency.The management of the database assists the staff office in man-
aging the personnel process and paperwork for the Special Government Employees
(SGEs), their ethics training and other mandatory training requirements, tracks the
HR actions and onboarding process, and flawlessly tracks completed reports and
ensures recordkeeping compliance with FACA and National Archives and Records
Administration (NARA). Due to SABSO's work:
~ Clean Air Scientific Advisory Committee (CASAC)-Completed CAS AC Review
of the EPA's Policy Assessment for the Review of the Primary National Ambient
Air Quality Standards for Sulfur Oxides, Review of the Primary National Ambient
Air Quality Standard for Sulfur Oxides: Risk and Exposure Assessment Planning
Document, and Review of the EPA's Integrated Science Assessment for Oxides of
Nitrogen, Oxides of Sulfur, and Particulate Matter—Ecological Criteria.
~ SAB-Completed SAB Review of Consideration of EPA Proposed Rule: Strength-
ening Transparency in Regulatory Science; Consideration of EPA Planned Actions
in the Fall 2017 Unified Agenda of Regulatory and Deregulatory Actions and their
Supporting Science; Consideration of EPA Planned Actions in the Spring 2017
Unified Agenda of Regulatory and Deregulatory Actions and their Supporting
Science,'Advice on the Use of Economy-Wide Models in Evaluating the Social
Costs, Benefits, and Economic Impacts of Air Regulations; Review of EPA's Draft
Assessment entitled Toxicological Review of RDX.
Office of Water (OW)
o In March 2018, the Office of Groundwater Drinking Water (OGWDW) completed
expert peer review of quantitative tools to evaluate neurodevelopmental effects that
could arise from drinking water exposure to perchlorate.These tools are consid-
ered a Highly Influential Scientific Assessment and, as such, expert peer review was
conducted in accordance with Agency guidance. Expert peer review is an important
component of the scientific process.The critical feedback, suggestions, and new ideas
provided by the peer reviewers stimulate creative thought, strengthen the interpre-
tation of the reviewed material, and confer credibility on the product.
-------
o
OW put both the Gold King Mine Biological Report and the National Rivers and
Streams Report out for independent peer review as well as partner review by their
state and tribal collaborators.
o In FY2018, the Office of Wetlands, Oceans, and Watersheds (OWOW) and the
Office of Research and Development (ORD) completed an economic model for
calculating economic impacts on the shellfish industry due to ocean acidification
and climate change.The package of materials for the model includes the model itself
that can be downloaded onto a computer, a users' guide for how to use it with their
own data, and an executive summary. OW completed a formal peer review of this
model, incorporated the edits, and plans to publish this work in a peer-reviewed
journal.
Office of Air and Radiation (OAR)
In FY20I8, the Clean Air Scientific Advisory Committee (CASAC) provided an indepen-
dent scientific review for Sulfur Dioxides (S02) Primary National Ambient Air Quality
Standards (NAAQS) - CASAC conducted a review of the Risk and Exposure Assess-
ment, and Policy Assessment. Other products undergoing peer review in FY 18 include:
~ Speciation ofTotal Organic Gas and Particulate Matter Emissions from On-road
Vehicles in the Next Version of MOVES
~ Exhaust Emission Rates for Heavy-Duty On-road Vehicles in the Next Version of
MOVES
~ Exhaust Emission Rates for Light-Duty On-road Vehicles in the Next Version of
MOVE
~ Population and Activity of On-road Vehicles in the Next Version of MOVES
~ Nonroad Engine Growth Estimates in the Next version of MOVES
~ Technical Report on Aircraft Emissions Inventory and Stringency Analysis
~ Aircraft C02 Cost/Technology and Industry Characterization Report Update
~ EPA's Method of Calculating Concentrations of Airborne Lead near Airports Na-
tionwide.
Region 6
Water Division (WD)
National Pollution Discharge Elimination Systems (NPDES) Permits &
Total Maximum Daily Loads (TMDLS) Branch
Since late 2009, Region 6, Oklahoma, Arkansas, and the Cherokee Nation have
worked to address nutrient impairments in the Illinois River Watershed of north-
west Arkansas and northeast Oklahoma, and efforts have yielded
preliminary models covering both rivers/streams and LakeTenkiller. EPA has com-
pleted the calibration and validation of the watershed and lake models. EPA has also
completed sensitivity and uncertainty analyses for both models. EPA's independent
external peer review panel was selected through a competitive selection process of
-------
academic or research institutions which then selected and convened a third-party
panel of external experts to review the EPA's modeling efforts. Oklahoma and Ar-
kansas agencies recently reached a shared
understanding that the developed models were highly calibrated to the actual
available data and that government officials at all levels, as well as industries in the
watershed, could use the models to realistically predict the water quality effects of
different load-reduction scenarios.The model is designed to reproduce conditions
of the Illinois River. In doing so, the model can be used to project potential clean up
options and evaluate potential cleanup alternatives for the watershed.
Professional Development
EPA encourages professional development activities so that EPA's staff can maintain their
expertise, be active members of their professional communities, and become leaders in
their fields.Training activities may include online courses, webinars,
in-person workshops, or conferences. EPA provides several professional development
opportunities for employees. Examples in FY 18 include retiring staff members
passing their knowledge onto the rising generation, enforcement officers participating in
mock trials, and staff members receiving training on communicating science. In FY 18, EPA
staff experienced career growth by authoring publications, receiving recognition through
awards, participating in presentations and panels at renowned conferences, and being
active members in professional science societies.Two
committees on the Region 8 Regional Science Council were instrumental in
promoting career development.The Technical Training Committee utilized a survey to
determine the technical training needs of their staff and implement a strategic plan.The
Professional Society Participation Committee developed a manager's resource guide to
provide standard, transparent, and equitable processes for allocating resources to sup-
port professional society participation.
Office of Enforcement and Compliance Assurance (OECA)
o A retiring senior chemist developed and presented a series of knowledge
management seminars and transferred some of her over 30-years of practical
laboratory expertise.
o The National Enforcement Investigations Center (NEIC) developed a series of video
presentations designed to help Special Agents better understand some of the com-
plexities of sampling and analyses associated with criminal investigations.
o The National Enforcement Investigations Center (NEIC), Office of Criminal
Enforcement, Forensics, and Training (OCEFT), Regional, and Department of
Justice (DOJ) attorneys, developed a "mock trial" for NEIC forensic scientists.The
trial used an actual NEIC criminal case and the field and laboratory
scientists involved in the project.This mock trial took place in October 2017.
-------
Office of Water (OW)
o In FY 18, the Office of Science and Technology (OST) scientists represented EPA and
promoted their Office's mission at conferences held by the following professional
organizations: Society of Environmental Toxicology and Chemistry Society of Toxi-
cology Society of Freshwater Sciences, Society of Epidemiologic Research,American
College of Toxicology, Coastal and Estuarine Research Foundation, World Health
Organization, National Environmental Health Association, North American Lake
Management Society, American Public Health Association, American Society of Civil
Engineers, Allied Social Sciences Associations, Applied and Agricultural Economics
Association, Society for Benefit Cost Analysis, Association of Environmental and
Resource Economics, as well as University of North Carolina Water Institute, Great
Lakes Beach Association and the Engineering Society ofWestern Pennsylvania.
o Additionally, staff participated in meetings held by multiple stakeholder associations:
Association of Clean Water Administrators, Interstate Shellfish Sanitation Confer-
ence, Environmental Council of States, National Association of Clean Water Agen-
cies, Water Environment Federation, Water Research Foundation, National Rural
Water Association, Intelligent Water Networks,World Health Association, Interna-
tional Water Association, and American WaterWorks Association.
o Staff also attended meetings with states and other stakeholders on program imple-
mentation and on technical topics such as nutrients, harmful algal blooms, recre-
ational criteria/swimming advisories, coliphage (viral indicator), perfluorinated com-
pounds, emerging contaminants and effluent guidelines. OST staff are encouraged
to publish their research. In FY 2018, OST scientists also contributed ten peer-re-
viewed publications to scientific journals, and most of their scientists gave oral and/
or poster presentations at professional conferences.
o Furthermore, OST staff are encouraged to have an Individual Development Plan
(IDP) and to discuss their professional development goals with their manager at
least twice per year. As a major accomplishment compared to past years, 98% of
OST staff have their negotiated and approved IDPs in place for FY 2018 (77% in
2016 and 92% in 2017).
Office of Air and Radiation (OAR)
Professional development and recognition of OAR scientific/technical staff is strongly
encouraged and accomplished through webinars and other mechanisms including spon-
sored training, funded off-site training, and by staff participation in scientific conferences
and workshops. OAR's scientific work is on the cutting-edge of many fields and its
scientists continue to publish papers in broadly recognized, high quality scientific jour-
nals. OAR programs report numerous examples of publications in the major scientific
journals related to automotive, fuels, electric vehicle, and transportation research includ-
ing Environmental Health Perspectives, Environmental Science and Technology, and the
American Journal of Epidemiology.
-------
Examples of publications, awards, and participation in professional activities include:
• Effect of Fuel Composition on Fuel Economy/C02 Emissions in Light-Duty Gasoline
Vehicles
• Understanding Real World Activity Data for Light-Duty Conventional, Hybrid, and
Plug-in Hybrid Vehicles
• Identifying Areas of High NOx Operation on Heavy-Duty Vehicles
• Modeling the Impact of Aftertreatment System on Nonroad Tier 4 Compression
Ignition Engines
• Fuel Trends Report: Gasoline 2006 - 2016
• Updated Evaluation of MOVES Light-Duty Gasoline NOx Emissions Rates with Real
World Measurements
• The Impact of SoakTime on Vehicle Start Emissions
• A Pilot Study ofTSO and RVP Effects on PM Emissions from Light-Duty Gasoline
Direct Injection Vehicles
• Consumer Satisfaction with New Vehicles Subject to Greenhouse Gas and Fuel
Economy Standards
OAR staff participated and presented at conferences and symposia including:
• Society for Benefit-Cost Analysis Annual Meeting
• Portable Emissions Measurement Systems Conference
• Science of the Total Environment
• Community Modeling and Analysis System Conference
• American Geophysical Union Fall Meeting
• Transportation Research Board Annual Meeting
• SAE Hybrid and Electric Vehicle Conference
• American Fuel and Petrochemical Manufacturers Conference
• Coordinating Research Council - Real World Emissions Workshop and Mobile
Source Air Toxics Workshop
• Health Effects Institute Annual Conference
• World Conference on Quality and Improvement
Training activities for individual or groups of staff included:
• The science of science communication
• Excellence in supervision
• Situational frontline leadership
• Data mining and screening experiments
• Confidence in analytical results
• Basics of petroleum refining for nontechnical personnel
• Conflict resolution skills
• Building a collaborative culture
• Building a positive team culture
• Diesel engine technologies
• Supply chain management principles Certified green supply chain professional
-------
Office of Land and Emergency Management (OLEM)
OLEM provided training opportunities related to scientific and technical information
to staff and project managers on capabilities of technologies and best practices (online
and classroom). For example, the National Association of Remedial Project Managers
(NARPM) National training this year included over 50 courses and opportunities for
project managers and technical staff to interact and share information on new technolo-
gies.
Region 3
Air Protection Division (APD)
o Technical Audit National Workgroup - APD Quality Assurance staff
participated in a national workgroup and completed chapters of the guidance
document, "Technical Systems Audit Quality Assurance Guidance Document
(TSA QAGD)." EPA-454/B-17-004, November 2017.
o Conference attendance - Staff attendance and participation in the Regional/State/
Local modeling conference (Boston, MA) and the National Air
Monitoring Conference (Portland, OR).
o S02 and Ozone national workgroups - APD staff actively participated in S02
and Ozone national workgroups which focused on the use of data in determining
designations for the National Ambient Air Quality Standards (NAAQS).To align
with national efforts, Region 3 senior staff formed and led regional workgroups
to evaluate and discuss data and to apply consistent approaches in support of
designations.
o The Science of Improvised Nuclear Devices (INDs) -The Philadelphia
Federal Executive Board Preparedness and Security Council assembled a panel
of subject matter experts to conduct presentations to help those planning to
attend the Liberty Down 2018 (LD 18) interagency continuity exercise, as the
LD 18 exercise scenario focused on a fictitious IND event.The regional radiation
program manager presented.The focus was expected impacts from the scenario
that affected the agencies' ability to perform functions per Federal Continui-
ty Directive I,'the unique safety concerns that revolve around an IND, such as
damage zones, plume, radiation and the need to shelter-in-place and/or evacuate
the area, etc.; and the role of the Commonwealth and the City in the IND event.
APD shared the stage with the Federal Emergency Management Agency (FEMA),
the Pennsylvania Department of Environmental Protection, and the Philadelphia
Office of Emergency Management.
Region 5
Region 5 held a workshop on Science Communication in November 2017 and they have
another one scheduled for 20l8.The workshop was coordinated through the R5 training
office and was open to anyone in R5 who wanted to register.
-------
Region 6
Water Systems - Ecosystems Protection Branch
Trained new staff in wetlands and wetland health for decision-making in applying
regulations, assessing for loss and compensation of wetlands, and stream
resources.
Superfund Program
The Superfund staff completed technical/scientific training within their core disci-
plines to maintain their respective certification.
Region 8
Technical Training Committee
The Technical Training Committee is one of the Region 8 Science Council
Committees.
~ Pilot FY 18/FY19 Technical Training Plan - The Committee developed a strategic
plan for technical training that includes training for priority needs identified from
the 2017 Science Survey issued by the Council: water, statistics, GIS, and data
management were identified as highest priority needs.The plan also serves as a
clearing house to connect Region 8 staff and managers and the Region 8 Human
Resources Office with information about available technical training. Region 8
hosted six training/webinars.
~ Training Needs Survey for Microsoft Access - The Committee distributed a sur-
vey to further define and specify the Access training that will best fit
Regional needs. Based upon the survey results, Region 8 will provide training in
FY 19 that covers mid-level to advanced topics.
~ Technical Training Offered in FY 18 - The Committee delivered several trainings to
Region 8, including:
• Applied Environmental Statistics - 5-day training on advanced topics in envi-
ronmental statistics and data evaluation (confidence, prediction, and tolerance
intervals; regression models and trend tests in many varieties, parametric,
non-parametric, and permutation tests) using R statistical software package.
30 attended from Region 8.
• R Mini-Training Series - Bi-monthly training series designed to maintain and
further build capacity in the use of R, as well as transitions Excel users to R
for data analyses.Two mini-trainings were held in May and July with 15 attend-
ees.
-------
Professional Society Participation Committee
The Professional Society Participation Committee is one of the Region 8 Science
Council Committees
~ Committee worked on development of a Managers Resource Guide which out-
lines standard, transparent, and equitable processes for allocating resources to
support professional society participation.
~ Committee identified a management champion to assist with the rollout of the
Managers Resource Guide, which is expected in FY 19.
~ Finally, the committee worked with the Region 8 comptroller to determine the
feasibility of utilizing a central pot of money to support professional society par-
ticipation
Region 8/ORD Collaborative Research
~ Regional Applied Research Effort (RARE):
Three projects were selected for funding in FY 18
o Toolbox for computational evaluations of subsurface impacts.
o Application of 21 st century bioanalytical tools to identify sources and
effects of bioactive contaminants associated with select municipal
wastewater discharges to the South Platte River and Colorado River
Watersheds.
o Remediation of fentanyl contaminated indoor environments.
~ Regional ORD Community of Science Networking (ROCS-Net):
• EPA Region 8 scientist, Seth Tourney, was selected to participate in ROCS-Net
along with Colorado Department of Public Health and Environment scientist,
Kristy Richardson, to address knowledge gaps related to N-Nitrosodimethyl-
amine in drinking water.
~ Regional Research Partnership Program (R2P2):
• EPA scientist, Dianna Hammer, was selected to participate in the R2P2 re-
search program to research nature-based strategies for managing excess
nutrients in the environment, specifically nitrogen.
-------
Appendix III
Members of the Scientific Integrity Committee
Office/Region Official
Scientific Integrity Official Francesca Grifo
Deputy to the Scientific Integrity Official Vincent Cogliano
Scientific Integrity Program Lead Martha Otto
OAR Betsy Shaw
OARM Lynnann Hitchens
OCFO David Bloom
OCSPP Stan Barone
OECA Erica Canzler
OEI Harvey Simon
OGC Carol Ann Siciliano
OITA Martin Dieu
OP Al McGartland
ORD Bruce Rodan
OLEM Nigel Simon
OW Benita Best-Wong
AO Helena Wooden-Aguilar
OSA Tom Sinks
Region I Art Johnson
Region 2 Anahita Williamson/Linda Mauel
Region 3 Jennifer Fulton
Region 4 Dawn Taylor
Region 5 Carole Braverman
Region 6 David (Wes) McQuiddy
Region 7 Cecilia Tapia
Region 8 Deb Thomas
Region 9 Duane James
Region 10 David Allnutt
-------
Appendix IV
Sources & End Notes
Some of this report's content was gathered from across EPA. Each of the program
offices and regional offices provided an assessment of scientific integrity activities and
accomplishments in their respective offices through the Federal Managers Financial
Integrity Act (FMFIA) annual review process.The ScIO supplied additional information
from efforts to resolve allegations of lapses in scientific integrity in FY20l8.The Annual
Meeting/Conversation with the ScIO (summary in Appendix I) and the work of the
Scientific Integrity Committee provided additional valuable information.
Disclaimer: Links to intranet versions of EPA documents or intranet sources will not
be accessible to non-intranet users.
(i) USEPA. (2016). Memorandum. Retrieved from https://archive.epa.gov/epa/
aboutepa/ruckelshaus-takes-steps-improve-flow-agency-information-fishbowl-
policy.html#memo
(ii) USEPA. (2015). Peer Review Handbook, 4th Edition. Retrieved from https://www.
epa.gov/osa/peer-review-handbook-4th-edition-2015
(iii) USEPA. (2018). Best Practices for the Clearance of Scientific Products. Retrieved
from https://www.epa.gov/osa/best-practices-clearance-scientific-products-epa
(iv) USEPA. (2018). EPA General Terms and Conditions. Retrieved from
https://www.epa.gov/grants/epa-general-terms-and-conditions-effective-
october-1 -2018
(v) Environmental Protection Agency Acquisition Regulation; Scientific Integrity, 83
Fed. Reg. 48581 - 48584, (affecting 48 CFR Parts 1503 and 1552). Retrieved from
https://www.regulations.gov/document?D=EPA-HQ-OARM-2015-0657-0001
(vi) USEPA. (2016). Best Practices for Designating Authorship. Retrieved from
https://www.epa.gov/osa/authorship-best-practices
(vii) USEPA. (2015). Office of the Science Advisor Coordination Procedures between
the Scientific Integrity Official and the Office of Inspector General regarding
Research Misconduct Allegations. Retrieved from https://www.epa.gov/sites/
production/files/2016-10/documents/oig coordination procedures.pdf
(viii) USEPA. (2014). Conflicts of Interest Review Process for Contractor-Managed
Peer Reviews of EPA HISA and ISI Documents. Retrieved from
https://www.epa.gov/sites/production/files/2015-01 /documents/epa-process-for-
contractor_0.pdf
-------
(ix) USEPA. (2019), Research Grants. Retrieved from https://www.epa.gov/research-
grants
(x) USEPA. (2019). Basic Information about Scientific Integrity. Retrieved from https://
www.epa.gov/osa/basic-information-about-scientiflc-integrity
(xi) USEPA. (2018). Office of Pesticide Programs New Employee Onboarding
Handbook. Retrieved from http://intranet.epa.gov/opp00002/hr/onboarding/new-
emp!oyee-onboarding-handbook.pdf
(xii) USEPA. (n.d.). Field Operations. Retrieved from http://r8net.epa.gov/pages/field-
operations.
(xiii) USEPA. (n.d.). ScienceHub. Retrieved from https://intranet.ord.epa.gov/science/
sciencehub
(xiv) USEPA. (2019).ATTAINS. Retrieved from https://www.epa.gov/waterdata/attains
(xv) USEPA. (2016). U.S. EPA National Lakes Assessment 2012: Percentage of Lakes in
Most Disturbed Condition. Retrieved from https://nationaIIakesassessment.epa.
gov/
(xvi) USEPA. (2019). Region 4 Superfund: Electronic Data Submission. Retrieved from
https://www.epa.gov/superfund/region-4-superfund-eIectronic-data-submission
(xvii) USEPA. (2015). Assuring the Competency of Environmental Protection Agency
Laboratories: Agency Policy Directive. Retrieved from https://www.epa.gov/sites/
production/fiies/2016-02/documents/internal_measurement_competency_-_
reaffirmed l I20l5.pdf)
(xviii) USEPA. (2017). National Water Program: Performance,Trends, and Best
Practices Report, [p. 38]. Retrieved from https://www.epa.gov/sites/production/
files/2017-03/documents/nwp_report_fy_2016_web_final.pdf
(xix) USEPA. (2018). Application of Systematic Review inTSCA Risk Evaluations.
Retrieved from https://www.epa.gov/assessing-and-managing-chemicals-under-
tsca/application-systematic-review-tsca-risk-evaluations
(xx) USEPA. (2017). Strategy for Conducting Literature Searches for Asbestos:
Supplemental Document to theTSCA Scope Document. Retrieved from
https://www.epa.gov/assessing-and-managing-chemicais-under-tsca/application-
systematic-review-tsca-risk-evaluations
(xxi) USEPA. (n.d.). Asbestos (CASRN: 1332-21 -4) Bibliography: Supplemental File
for theTSCA Scope Document. Retrieved from https://www.epa.gov/sites/
production/fiies/2017-06/documents/abestos_comp_bib.pdf
-------
(xxii) USEPA. (2017).Technical Air Pollution Resources. Retrieved from https://www.
epa.gov/technical-air-pollution-resources
(xxiii) USEPA. (2019). Urban Background Study Webinar Slides. Retrieved from https://
www.epa.gov/research/urban-background-study-webinar
(xxiv) USEPA. (20I9).TSCA Scientific Peer Review Committees. Retrieved from https://
www.epa.gov/tsca-peer-review
(xxv) USEPA. (n.d.). Scientific Integrity at EPA. Retrieved from https://intranet.ord.epa.
gov/scientific-integrity/what-scientific-integrity
-------
-------
http://www.epa.gov/scientificintegrity
~
To report allegations or concerns:
http://www2.epa.gov/osa/forms/anonymous-scientific-integrity-concerns-and-suggestions
For additional information or to report an allegation:
Scientific Integrity Official Scientific Integrity Program Lead
FrancescaT. Grifo, PhD Martha Otto
Grifo.francesca@epa.gov Otto.martha@epa.gov
(202) 564-1687 (202) 564-2782
To report fraud, waste or abuse, contact the hotline (Office of Inspector General):
E-mail: OIG_Hotline@epa.gov Write:
Phorie: 1-888-546-8740 EPA Inspector General Hotline
Fax: 202-566-2599 1200 Pennsylvania Avenue NW
Online: http://www.epa.gov/oig/ Mailcode 2431T
hotline.htm Washington, DC 20460
------- |