oEPA
November 2013
United States
Environmental Protection
Agency
Annual Report
on Scientific Integrity
-------
('over photo by Richard Rertfro
Hoosic River near Ml Greyiock m Western Massachusetts
-------
Message from the Scientific Integrity Official
EPA's ability to protect human health and the environment depends upon the integrity of the
science on which it relies. The EPA has a long history of promoting scientific integrity. For
example, early in EPA's history, then Administrator William Ruckelshaus established a culture
of integrity and transparency by promising that the Agency would operate "in a fishbowl," as
openly as possible. In 1999, the Agency published its Principles of Scientific Integrity,
developed in conjunction with the EPA's National Partnership Council, which is comprised of
representatives of Agency labor unions and management. The Principles laid out the basic rules
for ethical behavior by all EPA employees in conducting scientific research, in interpreting and
presenting results, and in using scientific information and data. Training was also made
available at that time on the Scientific Integrity Principles.
Since then, EPA has continued to make significant progress institutionalizing scientific integrity
through a wide variety of policies, guidance and actions. These efforts were captured in the
EPA's Scientific Integrity Policy, issued in February 2012. Because of the EPA's strong
commitment to scientific integrity, the Agency was recently rated highly in two separate reports
comparing scientific integrity and transparency among federal agencies.1
While our early work continues to expand in implementing the Agency's new Policy, I am
confident that we are moving forward on actions that will further weave scientific integrity into
the fabric of how EPA operates.
Glenn Paulson, Ph.D.
Interim Scientific Integrity Official
U.S. Environmental Protection Agency
1 Two recent Union of Concerned Scientists reports: the first in March 2013 entitled, "Grading Government
Transparency: Scientists' Freedom to Speak (and Tweet) at Federal Agencies"
http://www.ucsusa.org/assets/documents/scientific integrity/grading-government-transparencv-report.pdf
and another March 2013 report entitled, "Federal Agency Scientific Integrity Policies: A Comparative Analysis."
http://www.ucsusa.org/assets/documents/scientific integritv/SI-policies-comparative-analvsis.pdf
-------
Listening sessions that included internal and external stakeholders, EPA employee
unions' representatives, and EPA senior managers;
A webinar open to all EPA employees to discuss scientific integrity; and,
Publication of this Annual Report.
Scientific Integrity Committee
The Scientific Integrity Committee and its membership were established by the Policy; it has
very specific duties, both within its members' organizations as well as across the Agency. The
Committee is comprised of Deputy Scientific Integrity Officials, senior managers from across
the EPA. A list of the current Deputy Scientific Integrity Officials is in Appendix A.
These Officials provide oversight for the implementation of the Policy, act as liaisons for their
respective offices and regions, and are available to address questions and concerns regarding the
Policy. In addition to their role on the Committee, the Deputy Officials work with the Scientific
Integrity Official and others to address issues and allegations of scientific misconduct and
violations of the Policy, provide input into the development of this Report as part of the annual
review, and are to certify compliance with the Policy on behalf of their organizations.
The Committee has met regularly to discuss elements of the Policy, consistent implementation
across the Agency, and issues related to process and procedures for dealing with allegations of
violation of the Policy. By meeting regularly, the Committee provides a critical cross-agency
resource for conveying information and providing leadership in implementing the Policy.
Listening Session
Listening Sessions (the "annual meeting" for Fiscal Year 2013) were held on June 25, 2013, and
were comprised of four sessions:
External non-federal stakeholders
Federal stakeholders
EPA union representatives
EPA senior managers
For each session, participants were invited to provide input on EPA's Policy implementation and
future challenges. Several of the external stakeholders praised the EPA Scientific Integrity
Policy. The Union of Concerned Scientists (UCS) said that EPA's Policy is among the three
strongest of the 23 agencies, departments and offices that the UCS surveyed. UCS also praised
EPA as one of only two agencies that have appointed or have plans to appoint a full-time
Scientific Integrity Official.
Recommendations received from listening session participants included that the EPA should:
1) set up a process for evaluating training on scientific integrity; 2) set up a scientific integrity
website with links to related policies, processes and documents; and 3) review mechanisms for
Agency employees to express differing scientific opinions. The key points made at the sessions
are summarized in Appendix B.
2
-------
2013 Annual Report on Scientific Integrity
Section 1: Introduction
In March 2009, President Obama issued an executive memorandum that expressed the need for
sound science to inform and guide decisions by Executive Branch departments and agencies. In
response, the White House Office of Science and Technology Policy (OSTP) provided guidance
for the development of scientific integrity policies by federal agencies.
The EPA's Scientific Integrity Policy issued, in February 2012, in response to OSTP builds upon
EPA's significant earlier scientific integrity efforts, focusing on: the promotion of a culture of
scientific integrity throughout the EPA, the release of scientific information to the public, the
consistent use of peer review and federal advisory committees, and the professional development
of government scientists. The Policy established a Scientific Integrity Committee (the
Committee) to provide oversight for its implementation. The Committee, led by the Scientific
Integrity Official, will encourage consistent Policy implementation and further bolster the EPA's
broader efforts to ensure the integrity of the Agency's scientific, engineering, and other technical
work.
The EPA's 2012 Policy also requires that the Scientific Integrity Official prepare an annual
report, with input from the Committee, on the status of scientific integrity within the Agency.
This Report satisfies that requirement. It describes the implementation efforts; the
accomplishments, challenges and lessons learned during the previous year; and includes
recommendations for future actions by the Scientific Integrity Committee to ensure continuous
improvement in implementing the Policy. This Report also summarizes the input from both
annual listening sessions on scientific integrity and an interactive webinar titled, "A
Conversation with the Interim Scientific Integrity Official."
Section 2: Scientific Integrity Implementation Activities
This section provides a description of the implementation and outreach activities conducted
during the first year after issuance of the Scientific Integrity Policy. In addition, although the
review and certification processes are still under development, their major components are
described in Section 2B.
A. The First Year
Implementation activities have been structured to ensure regular, substantive input on issues and
priorities and, when needed, the development of new procedures, guidance, or other actions to
promote greater consistency in the implementation of the Policy.
Implementation activities for the first year consisted of the following, which are discussed in
more detail below:
Regular meetings (currently quarterly) of the Committee to discuss implementation of
the Policy;
1
-------
A Conversation with the Scientific Integrity Official
A webinar providing a direct interaction between EPA employees and the Interim Scientific
Integrity Official was held on August 20, 2013. Compared to the listening sessions, this webinar
allowed for a broader group of participants and provided an opportunity for the Official to
educate and train employees on the Policy, while also providing a venue for EPA staff to ask
questions and provide input. At least 140 employees participated online, with many more
participating by telephone in conference rooms across the country. Many suggestions were made
during the webinar on a wide range of issues; these suggestions are summarized in Appendix C.
B. Future Implementation Activities
This section describes proposed future implementation activities. To facilitate consistent
implementation, the Committee meetings that support the cross-agency implementation of the
Scientific Integrity Policy will continue. This includes outreach efforts with stakeholders,
partners, and employees. Each office and region also has both the authority and the responsibility
to ensure implementation of the Policy and to annually review and certify that its activities are in
compliance with the Policy. A biennial review of the Policy by the Scientific Integrity
Committee will serve to evaluate the effectiveness of the Policy and EPA's adherence with the
applicable rules and regulations, as well as identify any needed changes.
Scientific Integrity Committee Meetings and Outreach Efforts
Fundamental to the implementation of the Policy are the quarterly meetings of the Scientific
Integrity Committee. (See Figure 1 below for a description of the proposed overall cross-agency
process.) The October meeting would be used to prioritize the work of the Committee for the
new Fiscal Year. If needed, workgroups could be formed to address specific priority topics, e.g.,
training. The January and April meetings provide an opportunity for Committee members to
discuss progress addressing current priorities and issues, and to further ensure consistent
implementation across the Agency. During the July meeting, the Committee would discuss
recommendations and suggestions that had been provided during earlier Committee meetings, as
well as from partners, stakeholders, and employees during outreach efforts.
2 The future annual implementation activities for scientific integrity are still being discussed with the Committee
Members.
3
-------
October
September
Annual Report Cycle Ends
November
Key: o
Committee Meeting
Agencywide Webinars ~
Figure 1. Scientific Integrity Cross-Agency Implementation Process
In March, the Committee will conduct formal "listening sessions" with representatives of the
Agency's external stakeholders (e.g., the Science Advisory Board, the Office of Science and
Technology Policy, the Office of Inspector General, the Union of Concerned Scientists, and the
National Academies' National Research Council), EPA employee unions, and EPA's senior
leadership; this is a key element of its outreach efforts. The purposes of listening sessions are to
solicit input that informs the Annual Report on scientific integrity and to help the Committee
identify new concerns and obtain recommendations. Listening sessions are held early in the
calendar year so that input from them can be considered when offices and regions evaluate
whether there are any weaknesses or challenges in their organization's implementation of the
Policy that need further action in the near term.
Additional outreach efforts also will include one or two Agency-wide webinars each year to
provide any interested EPA employees an opportunity to share their thoughts and suggestions on
scientific integrity. The webinars are scheduled to provide timely input into priority setting for
Committee activities and information that regions and offices can consider in planning their
internal control reviews. Additional outreach activities also will be undertaken to ensure a
significant and robust discussion on scientific integrity issues and activities. For example, union
representatives have requested regular opportunities to engage in a discussion of scientific
integrity issues, and provide recommendations on priorities and potential changes in the Policy.
4
-------
Review. Certification and Reporting on Compliance with the Policy
All Agency employees, including scientists, managers, and political appointees, are required to
follow the Scientific Integrity Policy. While the Deputy Scientific Integrity Officials provide
implementation oversight, act as the liaison for their respective organizations, and are available
to address questions and concerns from individuals within their offices, as the Policy itself states,
implementation of the Policy is the responsibility of all EPA personnel. The Deputy Scientific
Integrity Official for each office and region should engage management and staff to identify and
promote activities that support scientific integrity. Annually, each Deputy Official will need to
review these activities, develop information needed for the Annual Report, and certify
compliance (for the office or region) with the Policy.
Review of Activities for the Annual Report
The Deputy Scientific Integrity Official for each organization will oversee a review of that
organization's activities and determine whether their activities are in compliance with the
Scientific Integrity Policy. Specifically, the four implementation areas in the Policy to be
addressed are:
Promoting a culture of Scientific Integrity
Releasing Scientific Information to the Public
Conducting Peer Review and Using Federal Advisory Committees
Ensuring Professional Development of EPA's Scientists, Engineers, and Other Technical
Personnel
For each of the areas listed above, each organization that generates, manages, communicates or
otherwise deals with scientific information or issues should develop organizationally specific
review questions that will help in assessing its progress in implementing the Policy. (In his May
8, 2013 memorandum to the Committee members, Bob Perciasepe, then Acting Administrator,
specifically directed that questions be developed to facilitate the review of compliance with the
Policy.) Because each organization is unique, these questions may need to be tailored to
adequately address scientific integrity in each organization.
The review questions are intended to aid each organization in determining: 1) whether internal
controls are needed or are in place to prevent violations of the Policy; 2) how well the Policy is
being implemented; and 3) whether there are any weakness or challenges that need to be
addressed. Programs and regions should also compile information and examples of
accomplishments and lessons learned.
Certification of Compliance
At the completion of an annual review, each Deputy Scientific Integrity Official should be in a
position to "certify" compliance of his or her organization with the Policy, as well as report on
the implementation of the Scientific Integrity Policy, including dealing with any misconduct
issues within his or her program or region. To effectively achieve the certification requirement
of the Policy, either a separate certification process is needed or each Deputy Scientific Integrity
5
-------
Official would address scientific integrity policy compliance in the relevant Federal Managers'
Financial Integrity Act (FMFIA) assurance letter.3 A specific "assurance" statement in the
organization's FMFIA assurance letter will satisfy the certification requirement in Section V.D.
of the Scientific Integrity Policy, while also informing the FMFIA process. The May 8, 2013
memorandum previously mentioned also directed that the Committee develop a certification
statement and address scientific integrity in FMFIA assurance letters starting in FY 2014.
Preparation of the Annual Report on Scientific Integrity
Besides the above certification, each Deputy Scientific Integrity Official must report to the
Scientific Integrity Official on the promotion of a culture of scientific integrity, the release of
scientific information to the public, the consistent use of peer review, and the professional
development of government scientists, engineers, and other technical personnel within his or her
organization.
The Scientific Integrity Official, with input from the Deputy Scientific Integrity Officials, is
responsible for generating and making publicly available an Annual Report to the Science
Advisor on the status of scientific integrity in the Agency. The Annual Report is to highlight
scientific integrity successes, as well as identify areas for improvement and include a plan for
addressing critical weaknesses, if needed. The Annual Report will also include lessons learned
during the previous year, a summary of input from the outreach efforts, and recommendations for
future actions by the Committee to ensure continuous improvement in implementing the Policy.
Information that programs and regions gather as part of their annual review and certification
process is valuable input for the Annual Report.
Conduct of the Biennial Policy Review
The Policy requires that the Committee conduct a review every two years to ensure the
effectiveness of the Policy. The Policy shall be reviewed and, if necessary, revised as
recommended by the Committee and the Scientific Integrity Official, and approved by the
Science Advisor. The initial biennial review will occur in FY 2014.
Section 3: Scientific Integrity Policy Accomplishments
The accomplishments discussed below are organized around the four major components of the
Policy. The information provided in this report establishes the baseline from which progress will
be determined in future years. Despite resource limitations, including not having a full-time
Scientific Integrity Official during the first year of Policy implementation, the Agency has
accomplished a variety of scientific integrity related actions.
3 The Office of Research and Development (ORD) has already fully integrated its Scientific Integrity review and
certification into its FMFIA process and will continue to do so in the future. Other EPA offices could adopt this
approach or develop an alternative one.
6
-------
A. Promoting a Culture of Scientific Integrity
As stressed in the Policy, successful application of science in Agency policy decisions relies on
the integrity of the scientific process both to ensure the validity of scientific information and to
engender public trust in the Agency. Scientific integrity is promoted across the Agency by
supporting a culture of scientific integrity, enhancing transparency within scientific processes,
and protecting Agency scientists (e.g., by prohibiting management from intimidating or coercing
scientists to alter scientific data, findings, or professional opinions). Below are examples from
across the Agency.
Supporting a Culture of Scientific Integrity
Scientific Integrity Official
Somewhat unique among federal agencies, EPA has chosen to hire a senior level employee that
will provide focus and leadership on scientific integrity. Most other federal agencies have this
role as a collateral duty for a senior person with other responsibilities. The Agency is in the
process of hiring its first full-time Scientific Integrity Official, who will act as a champion for
scientific integrity throughout the Agency. The position is expected to be filled early in FY 2014.
Application of the Policy
In addition to all EPA employees, and as specifically noted in the Applicability section of the
Policy, all contractors, grantees, collaborators and volunteers who engage in scientific activities
at or for the EPA are expected to uphold the standards established in the Policy. For example,
the Office of Enforcement and Compliance Assurance (OECA) conducts compliance monitoring
of laboratories that perform environmental and safety studies in support of pesticide registrations
under the Federal Insecticide, Fungicide and Rodenticide Act (FIFRA) or pursuant to a consent
agreement under the Toxic Substances Control Act (TSCA). Compliance monitoring is carried
out to assure that these studies are conducted in compliance with the FIFRA or TSCA Good
Laboratory Practice (GLP) regulations at 40 CFR parts 160 and 792, respectfully. Instances of
non-compliance are referred to the Office of Chemical Substances and Pollution Prevention
(OCSPP) for regulatory action. The people working in these outside labs are expected to act in
accordance with the Policy.
The EPA's Forum on Environmental Measurements (FEM) is implementing another Agency-
wide policy requiring organizations generating or using environmental data under certain
Agency-funded assistance agreements to submit documentation of their competency prior to
award of the agreement, or if that is not practicable, prior to beginning any work involving the
generation or use of environmental data under the agreement. That policy was originally
approved on December 12, 2012, by the Science and Technology Policy Council and compliance
with it is to have been accomplished by October 1, 2013. The policy applies to competitive and
non-competitive assistance agreements expected to exceed a total maximum value of $200,000
(in federal funding) that will involve the generation or use of environmental data. Requiring that
these organizations provide documentation of their competency will help to ensure integrity of
the data and compliance with the Scientific Integrity Policy.
7
-------
ORD is revising terms and conditions for grant awards to require grant recipients to provide
annual assurance that they understand and comply with the terms and conditions of EPA's
Scientific Integrity Policy that are designed to prevent research misconduct.
Consistent Field Operations
The Policy also promotes a culture of scientific integrity by fostering honest investigation,
discussion, refined understanding, and a firm commitment to evidence. To this end, the Deputy
Administrator identified improving the consistency of field operations as an Agency-wide area of
focus. The Regional Science and Technology (RS&T) Directors' Field Operations Group (FOG)
was created and charged with promoting and ensuring technical consistency among the Agency's
many field activities. EPA has an obligation to ensure that reliable, accurate environmental data
of known quality is generated, and that the process of generating data, from planning to
implementation to reporting, is traceable and transparent. In FY 2013, the FOG developed ten
operational guidelines for field activities to ensure consistency in managing field practices and to
reduce potential vulnerabilities. The FOG Guidelines are based on best practices for data
collection as determined by EPA field groups, EPA quality requirements, and concepts of
management systems established by the International Organization for Standardization (ISO).
Management and staff at OECA's National Enforcement Investigations Center (NEIC) have led
implementation efforts including developing and delivering national training. In March 2013,
the EPA Deputy Administrator directed full implementation of the FOG Guidelines within three
years. Program and regional offices have developed and are carrying out implementation
strategies.
Electronic Tracking and Chain-of-Custody Procedures for Environmental Field Samples
Chain-of-custody procedures are an essential quality control element that ensures that sample
integrity can be tracked and documented, reducing the risk of lost samples and compromised
data quality. The EPA ORD implemented electronic tracking and chain-of-custody procedures
for the collection and analysis of environmental field samples for the Congressionally-requested
Hydraulic Fracturing Drinking Water Study. This innovative system enabled the field and
laboratory staff to create a thorough electronic chain-of-custody file. A handheld barcode
scanning computer system and software program are being used to track samples and sample
bottles from bottle preparation through analysis to data storage for this high profile project. With
this system, the status, current location of every sample, and who had performed each step in the
sample's lifetime, can be traced and verified.
Scientific Integrity Training
As part of its mandate, the EPA Scientific Integrity Committee has overseen the development of
training on the Scientific Integrity Policy for Agency employees. Contractors, cooperators,
grantees, and volunteers also will be encouraged to take this training, and may in the future be
required to do so if such training is part of their respective agreements with the EPA.
The first training module, developed by a workgroup under the Committee with representatives
from program and regional offices and EPA unions, is now available through EPA's training
8
-------
portal. This training module is required for all Agency managers and direct-line supervisors, as
well as for personnel within their offices that they have designated. A summary of the Policy and
an invitation to take the training will be sent to all EPA employees to help continue to ensure that
they are familiar with the Policy and are aware of the importance of scientific integrity.
In addition to this newly developed training specifically on the Scientific Integrity Policy, there
are many other training courses that are relevant to scientific integrity. Among them are the
following examples.
EPA laboratories provide annual Laboratory Ethics and Data Integrity Training for scientists
engaged in generating scientific data to support cleanups, enforcement, and environmental
assessments. This annual scientific ethics training fulfills accreditation standards (as applicable)
and directly reinforces an understanding of the laboratory ethics policy, as well as indirectly of
the Scientific Integrity Policy.
During the past year, EPA scientific and technical staff participated in activities such as EPA's
Forum for Environmental Measurement and the National Environmental Monitoring Conference,
both of which promote a high level of data integrity. EPA's Office of Water conducted annual
Chemistry and Microbiology training for Regional and State Certification Officers responsible
for the oversight of drinking water laboratories.
Additionally, several EPA regions conduct annual quality assurance training for scientific and
technical staff. The training covers the region's quality assurance procedures, quality assurance
project plan requirements and procedures, and the Field Operations Group and information
quality guidelines. In addition, some EPA regional offices offer an annual Quality Assurance
Conference. The conferences include speakers from within the Agency, universities, and the
private sector.
All of these other training activities are focused on one or more specific components of the
Scientific Integrity Policy.
Scientific Integrity Official/Office of Inspector General Coordination Procedures
The efficient evaluation and disposition of decisions regarding scientific misconduct allegations
require close coordination between the Scientific Integrity Official and the Office of Inspector
General (OIG). This past year, the Scientific Integrity Official personally led the work to
develop procedures for this coordination on scientific misconduct allegations. The
"Coordination Procedures between the Scientific Integrity Official and the Office of Inspector
General regarding Scientific Misconduct Allegations" are found in Appendix D.
Scientific Misconduct Allegations
The Scientific Integrity Official, working closely with the EPA OIG and the EPA Science
Advisor, investigated three allegations of violations of the Scientific Integrity Policy. These
allegations came from entities outside of the Agency, and included issues in the areas of release
of scientific information, peer review, and data interpretation. Two of the allegations were found
9
-------
to have no merit. The third allegation, referred from the OIG for Agency evaluation, is still
under review to determine whether any action is needed.
Enhancing Transparency
To enhance transparency, the Scientific Integrity Policy recognizes the value of independent
review of the Agency's scientific facilities and testing activities, as occurs with accreditation by
national and internationally recognized sanctioning bodies. While the testing conducted at many
of EPA's laboratories is unique, EPA has pursued appropriate accreditation and certification to
demonstrate laboratory integrity prior to the issuance of the overarching Scientific Integrity
Policy in February 2012.
Accreditation of EPA Laboratories
All EPA laboratories are addressing the 2004 Agency Policy Directive, "Assuring the
Competency of Environmental Protection Agency Laboratories"
(http://www.epa.gov/fem/pdfs/labdirective.pdf). Most EPA laboratories addressed the
competency policy directive by becoming accredited by the National Environmental Laboratory
Accreditation Program (NELAP) or by obtaining International ISO 17025 standard certification.
For laboratories with more specialized research, for which there is not yet a national or
international accreditation process, a third-party external assessment was performed to
demonstrate and document that the laboratory was adhering to the policies and procedures
described in their documented data quality system.
All EPA regional laboratories maintain NELAP accreditation or ISO 17025 certification. Some
EPA labs go far beyond what is required by NELAP and ISO. For example, because of its
portfolio of specific projects, the Region 8 Laboratory has had three reviews during the last two
years. These included not only an onsite NELAP review, but also an ORD Quality
Assurance/Quality Control review for the National Hydro-Fracking Study, and an Office of
Ground Water and Drinking Water review for Drinking Water Certification of the Region 8
Laboratory.
The Office of Air and Radiation (OAR) National Analytical Radiation Environmental
Laboratory (NAREL) maintains NELAP accreditation. The OAR National Vehicle Fuel &
Emissions Laboratory and the OCSPP Biological and Economic Analysis Division Laboratories
maintain ISO 17025 certification.
OECA's environmental forensic science center, the NEIC, is accredited by an internationally
recognized accrediting body (American National Standards Institute/American Society for
Quality/National Accreditation Board/Forensic Quality Services, ANSI/ASQ/NAB/FQS). The
scope of this accreditation includes both field and laboratory activities. NEIC's quality
management system conforms to ISO 17025:2005 requirements and supplemental requirements
for forensic science service providers. Annual external assessments are conducted by this
accrediting body.
10
-------
The OCSPP/Office of Pesticide Programs - Microarray Research Laboratory and several ORD
Laboratories use non-EPA third party external auditors to demonstrate and document that the
laboratory is adhering to the policies and procedures described in their documented quality
system, since a formal accreditation program is not available for the specialized research
performed at these labs.
The Office of Solid Waste and Emergency Response (OSWER) manages the Environmental
Response Laboratory Network (ERLN), which is comprised of approximately 140 Federal
(including EPA labs), State, and private laboratories. There are extensive requirements for
laboratories to become members of the ERLN, with the most basic requirement being that each
laboratory must be accredited under a nationally recognized accreditation program (such as
NELAP) or provide written proof of compliance with all ISO 17025 laboratory certification
requirements. Laboratories choosing this second means of ERLN membership are subjected to
an EPA-sponsored laboratory audit before they are allowed to analyze samples under the ERLN.
OSWER has established capability in five EPA regional laboratories for the sole purpose of
analyzing environmental samples for chemical warfare agents (CWA). These laboratories are all
NELAP accredited for general laboratory activity with CWA analysis and subjected to annual
Proficiency Tests to ensure their competency for this type of analysis, since there is no available
accreditation for this area of analysis. OSWER also manages two specialized mobile
laboratories, both with NELAP accreditation.
Review of Data from Other Organizations
The Agency's quality assurance is further supported by, "A Summary of General Assessment
Factors for Evaluating the Quality of Scientific and Technical Information" and its addendum,
"Guidance for Evaluating and Documenting the Quality of Existing Scientific and Technical
Information." This document describes the assessment factors and considerations used by the
Agency to evaluate the quality and relevance of scientific and technical information.
The addendum was updated this year, to establish minimum review and documentation
requirements for assessing and accepting data from other organizations.
Protecting Employees
When the Whistleblower Protection Enhancement Act of 2012 was signed into law on November
27, 2012, the law bolstered the protections and rights found in the Whistleblower Protection Act
of 1989. Among other things, the enhanced Act provides whistleblower protection for
government scientists who challenge censorship of scientific information or make whistleblower
disclosures related to the integrity of scientific processes. In response to additional requirements
of the Act, the EPA OIG has designated Howard Nicholson, an attorney with the EPA OIG, as
the Agency's whistleblower protection ombudsman to be responsible for educating employees
about whistleblower protections, rights and remedies. Mr. Nicholson is available to respond to
individual inquiries and to address staff gatherings and to provide training on this important
issue.
11
-------
B. Release of Scientific Information to the Public
Scientific research and analysis are the foundation of all EPA policy decisions. The Agency
works to ensure that scientific research and results are presented openly, with integrity, accuracy,
timeliness, and full public scrutiny. EPA encourages and supports transparency and active, open
communications through various forms, including but not limited to publication in peer-reviewed
journals, conference papers, presentations, media interviews, responses to Congressional
inquiries, web postings, and news releases. Full and open communication is a shared
responsibility throughout EPA.
EPA program and regional offices work continuously to promote access to relevant scientific
information. Below are examples of recent accomplishments regarding the release of scientific
information to the public. What follows is only a partial picture ("snapshot") since every day all
EPA Offices are providing some types of scientific information to segments of the American
public. Here we highlight only a few that either started or reached a major stage of
accomplishment in the past year.
Cross-Agency Efforts
Increasing Public Access to the Results of EPA-funded Scientific Research
In a February 2013 memorandum, the Director of the White House Office of Science and
Technology Policy (OSTP) asked all federal Departments and Agencies to prepare and submit
draft plans for increasing public access to the results of their federally funded scientific research.
The EPA has prepared a draft Plan for how to make more fully available to the public both peer-
reviewed scholarly publications and also the associated digitally formatted scientific data
resulting from unclassified research funded wholly or in part by the Agency.
Hydraulic Fracturing Study
EPA has conducted extensive outreach to states, Tribes, the public, the scientific community,
industry and other interested stakeholders in the development and execution of its Study of the
Potential Impacts of Hydraulic Fracturing on Drinking Water Resources (Hydraulic Fracturing
Study). In the Fall of 2012, the Agency announced an enhanced stakeholder outreach process to
inform the synthesis and analysis in the draft report of results expected in late 2014. As part of
this effort, over the last year the Agency has held public and stakeholder webinars; announced a
request for information; organized and led a series of technical roundtables; and held five
technical workshops. Workshop topics are Analytical Methods, Well Construction/Operation and
Subsurface Modeling, Wastewater Treatment and Related Modeling, Water Acquisition
Modeling and Case Studies. All workshops are followed by a public webinar, and information
from the roundtables and workshops is posted on the study web site (www.epa.gov/hfstudy).
These forums provided an opportunity for EPA scientists to interact with technical experts, as
well as inform EPA of current hydraulic fracturing technology and uses and identify related
research. A key feature of all of them was the identification of both data sources EPA was using
and also the approaches EPA is using to analyze external data and develop new data. EPA
12
-------
believes a transparent, research-driven approach with significant stakeholder involvement can
address questions about hydraulic fracturing and strengthen our clean energy future.
Next Generation Compliance
With advances in information technology and emissions monitoring capabilities, the EPA, states,
citizens, and industry will have more complete and timely information regarding environmental
conditions, emissions and compliance. EPA is developing and implementing a new initiative,
Next Generation Compliance, which uses advances in both emissions monitoring and
information technology, along with better designed rules, to improve environmental protection.
The EPA regions are participating with OECA in the development of the Next Generation
Compliance approaches. One of the areas in this effort involves the greater use of field and
remote sensing instrumentation, as well as working with other Agency partners in developing
methods to advance this area. OECA, ORD, and OAR also are working on addressing any data
quality issues associated with the varying uses of advanced monitoring technology. Information
on these new technologies and their capabilities is being provided publicly as specific projects
are completed.
Mixing Zone Modeling Webinar Workshop
With ORD's help, Agency-wide events titled The Mixing Zone Modeling Webinar Workshop
Series were held in mid-January 2013. In conjunction with the Agency's Superfund and
Technology Liaisons (STLs) and Office of Science Policy, the webinar series on water quality
modeling and decision support systems was designed to focus on the complex topic of mixing
zones. The webinars were open to all EPA employees, as well as to members of all Tribes,
States, and other Federal Agencies. This 3-day series, which provided a basic understanding of
mixing zone assessments and modeling, was offered in response to a critical national need. The
several hundred national and international participants included representatives from almost
every EPA region, at least 36 States' Departments of Environmental Quality or Departments of
Health, and numerous Tribes, plus many others. The webinars have been archived so that
National Pollutant Discharge Elimination System permit writers can access them when they
actually need to review a mixing zone study or use CORMIX, an EPA-supported mixing zone
model and decision support system for environmental impact assessment of regulatory mixing
zones that are a result of continuous point source discharges.
Mine Waste Characterization Webinar
Another three-part webinar series, Mine Waste Characterization, featured an agenda with
nationally-recognized experts from across the country. The first session had over 400
participants, including colleagues from other agencies and outside firms. The webinars focused
on evaluating the potential for contaminant release from hardrock mine sites, a serious issue in
Region 10 and other parts of the country. The workshops helped participants understand the key
issues regarding water chemistry predictions, identify the potential sources of contamination
from mine sites, and learn methods to mitigate or reclaim facilities to protect natural resources.
Collaborating agencies were EPA Headquarters, EPA regional offices, ORD, US Geological
Survey and US Forest Service.
13
-------
These and other webinars provide an opportunity for scientific and technical staff across the
Agency to engage with people from other organizations on current complex scientific issues in
order to learn, share ideas and expertise, and work together to solve the challenging issues faced
in protecting human health and the environment.
Office of Water Activities
The EPA Office of Water (OW) is finding new ways to use the Internet to make data it has more
readily available to the public. For example, the National List of Fish Advisories Database,
which includes all available information describing state-, tribal-, and federally-issued fish
consumption advisories provided to EPA by the states, territories, and tribes, used to only be
available to users who requested copies of compact discs of the data. Now it is available directly
from the OW website, where any users can directly and easily search the database to address
their needs.
How's My Waterway
In mid-2011, OW initiated the Water Data Project to actively promote better outreach and
communication of EPA water data. In October 2012, "How's My Waterway" was released. It
is both a desktop application as well as an "App," which uses plain English and the Global
Positioning System (GPS) capabilities of a smart phone to locate the position and information on
the quality of the water body close to you or of a water body in a location that you specify. In
addition, as part of a new partnership with the Smithsonian Institution, "How's My Waterway" is
now up on MySI.edu. On the Smithsonian website, many more people will have access to
EPA's Water Quality data.
Office of Air and Radiation
The EPA Office of Air and Radiation (OAR) has a number of recent initiatives to increase
accessibility of scientific information to the public, including:
o Near-real time air quality observations and forecasts (airnow.gov)
o Quarterly data on power sector emissions of sulfur dioxide, nitrogen oxides, and carbon
dioxide (ampd.epa.gov)
o Peer-reviewed climate change indicators presented through an easily accessible website
and slide show (www.epa. gov/climatechange/science/indicators)
o Information on airborne radioactivity in the environment (www.epa.gov/radnet/)
o Information on indoor air quality that is directed at many types of audiences, including
the general public, schools, scientists, and industry (www.epa.gov/iaq/)
14
-------
Environmental Benefits Mapping and Analysis Program Community Edition
Another example of OAR's efforts to expand access to the scientific information that underlies
Agency decisions is the development and release of the "Environmental Benefits Mapping and
Analysis Program - Community Edition" (BenMAP-CE). BenMAP, the tool EPA uses to
estimate the health impacts and economic benefits occurring when populations experience
changes in air quality, was developed in the early 2000s as proprietary software. Recognizing
the increasing need for transparent, publicly accessible tools for benefits assessment, the OAR
BenMAP software development team redesigned BenMAP from the ground-up as an open-
source and community-owned tool, using a modern software language and geographic
information system. The result of this effort is the first open-source version of EPA's benefits
software.
Office of Research and Development
In FY 2013, ORD implemented proactive measures to strategically integrate multiple outreach
and communication mechanisms to leverage opportunities and effectively amplify messages
conveying the impact of EPA science. These proactive measures included the use of news
releases, interviews with scientists and technical experts, public events, creating several
communities of practice (such as for computational toxicology), coordinated social media efforts
(such as Tweets and "Tweet ups," blog posts, and podcasts), topical (single-issue) newsletters,
and the formation of new partnerships to amplify outreach campaigns.
Green Heart
The Green Heart initiative is an example of such a partnership. Working with the EPA scientists
and partners in the Office of Air and Radiation and the Office of the Administrator, ORD
launched the "Green Heart" initiative to raise awareness of air pollution's role in cardiovascular
disease. The Green Heart outreach and communication campaign targeted doctors, nurses and
other healthcare providers to increase the awareness of actions that at-risk individuals can take to
protect their cardiovascular health from exposure to unhealthy air. The campaign included the
following: a Green Heart website, extensive social media outreach and amplification using the
@EPAresearch Twitter account (29,000 followers accounting for "retweets" with a total
outreach of 993,750), newsletter articles in EPA's Science Matters newsletter (about 40,000
subscribers), the posting of numerous blog posts on EPA's It All Starts with Science blog, and a
featured EPA podcast.
Village Green Project
In 2012, ORD initiated the Village Green Project (VGP), which is a research study to develop a
next-generation air monitoring station for community environments. A partnership was
established via a memorandum of understanding with the Durham County Library in North
Carolina to test the prototype VGP air monitor at a public library location and collaborate on
educational outreach. The prototype station, entirely solar-powered, measures fine particles,
ozone, and meteorology minute by minute. The data are wirelessly streamed to a database,
where the instrument diagnostics are quality-checked; acceptable data are then posted to a public
website. The website - villagegreen.epa.gov - was designed to be user-friendly and allow the
15
-------
public to explore the data collected. To support educational outreach, the project team has been
writing blog entries in the EPA It All Starts with Science blog, describing the technology, time
series observations, and educational connections.
Office of Solid Waste and Emergency Response
OSWER uses the internet, and in particular webinars, to provide broad public access to
information on science and technology. One of OSWER's most successful efforts is the
Contaminated Site Clean-up Information website (http://www.clu-in.org/). which is supported by
the Office of Superfund Remediation and Technology Innovation. This website provides
information to the public about innovative treatment and site characterization technologies by
offering dozens of internet webinars each year, regular posting of technical support documents,
and publication of a number of newsletters. The webinars are archived for continued public
access to the technical information in them.
Region 10 Initiative
The EPA Region 10 office has created a Regional Science Steering Council (RSSC) to improve
communications of science products. As part of this effort, the RSSC has sponsored and co-
hosted an initiative to expand the use of webinars. RSSC members spearheaded an effort to
identify and fill important knowledge gaps among staff from around the country, with attention
to gaps in the public's understanding of particular issues.
C. Peer Review and the Use of Advisory Committees
Peer review, fundamental to the scientific process, helps ensure the quality, rigor and objectivity
of our work. EPA has a long history of conducting independent peer reviews of Agency science
documents. In addition to a long-established Peer Review Policy, EPA's Peer Review Handbook
is now in its 3rd Edition.
EPA also utilizes Federal Advisory Committees to gain expert input on a wide variety of issues,
including scientific documents, and in accordance with the Federal Advisory Committee Act,
does this in a transparent public way.
Peer Review
To ensure that scientific products undergo appropriate peer review by qualified experts, the EPA
relies on its Peer Review Policy and US EPA Peer Review Handbook, 3rd Edition (2006). The
Handbook defines the approach to peer review for all of EPA, including Headquarters program
offices and regional offices. EPA's Peer Review Advisory Group (PRAG), a cross-Agency
workgroup, under the authority of the Science and Technology Policy Council and supported by
the Office of the Science Advisor, is currently revising and updating the Peer Review Handbook.
The revision will incorporate changes made since the 2006 edition and will include the guidance
in a 2009 addendum, which provided clarity for the regulatory definition of "appearance of a
lack of impartiality" for individuals who serve on peer review panels, and other process
16
-------
improvements put in place since 2006. The 4th Edition of the Handbook should be released in FY
2014.
This year, at the specific request of then-Administrator Lisa Jackson, EPA strengthened the
Agency's oversight of contractor-led peer review panels by developing a new, "Conflict-of-
interest Review Process for Contractor-Managed Peer Reviews." The new process includes two
opportunities for public involvement not previously available for identifying and selecting panel
members. The new process is designed to enhance the transparency of contractor-led peer
reviews and is intended to reduce the potential for organizational or personal conflict-of-interest
concerns through greater public participation and more rigorous internal review. The PRAG also
developed an implementation plan for the new process to ensure its consistent application across
the Agency. The new process and the Acting Administrator's cover memorandum on it are in
Appendix E.
As needed, the Agency also requests the National Academies of Science (NAS) to perform peer
review of important or controversial scientific documents. For example, EPA contracted with
National Research Council (NRC) of NAS to conduct a peer review of the draft EPA report titled
State of the Science Evaluation: Nonmonotonic Dose Responses as They Apply to Estrogen,
Androgen, and Thyroid Pathways and EPA Testing and Assessment Procedures. The NRC
assembled a panel of scientific experts, conducted a public meeting on the document, and
collected and transmitted public comments to the panel. The peer review report is expected in
early 2014. The Agency also has requested that NAS develop a peer review report on EPA's
external draft of the Integrated Risk Information System (IRIS) Toxicological Review of
Inorganic Arsenic.
Federal Advisory Committee Act Committees
The EPA may use formal advisory committees to gain input and bring the public into a
productive advisory process. Federal Advisory Committee Act (FACA) committees include
experts in, for example, environmental, physical, social and technological sciences, and are
formed to provide advice on a broad range of environmental issues. FACA committees are an
important tool that the EPA may use to ensure the credibility and quality of Agency science,
enhance the transparency of the peer review process, and gain input from the EPA's diverse
stakeholders.
FACA committees may provide advice on broader features of EPA's scientific and technical
activities (e.g., the Board of Scientific Counselors (BOSC)) or could focus on certain specific
areas (e.g., the Clean Air Scientific Advisory Committee (CASAC)). The BOSC, a FACA
committee, provides advice, information, and recommendations to EPA's ORD on technical and
management issues of its research programs. The CASAC provides an independent review to the
EPA Administrator on the technical bases for EPA's national ambient air quality standards.
EPA ensures that the management of the BOSC and CASAC strictly adheres to all FACA
requirements and the Scientific Integrity Policy. Nominations are sought in an open, transparent
manner, including through the Federal Register and professional organizations. Members are
selected based on their expertise, knowledge and contribution to the relevant area, while also
17
-------
providing a balanced and diverse committee. Reports produced by the BOSC and CASAC are
recognized as products of the Committees and are not revised by the EPA programs.
D. Professional Development
Scientific leadership is essential to the EPA's ability to advance its mission. One key activity for
this leadership is the ability to facilitate the professional development of our scientific,
engineering, and technical staff. With budget and travel reductions, as well as new requirements
to track costs of attendance at conferences, the EPA now faces considerable obstacles to
overcome to promote professional development of Agency personnel. Nevertheless, below are
some examples of the activities over the past year:
The EPA has continued to support professional activities by its technical staff, including
presenting papers and posters at scientific meetings, organizing technical sessions at conferences
(e.g., the National Environmental Monitoring Conference, Pittsburgh Analytical Chemistry
Conference (PittCon)), and actively participating in professional societies (e.g., Society of
Toxicology, Society for Risk Analysis, Society of Environmental Toxicology and Chemistry, Air
& Waste Management Association, and American Geophysical Union). In addition to helping
spread the word about EPA's scientific contributions to many fields, these and other meetings
allow the Agency's professional staff to learn about cutting edge research and developments and
interact with other professionals and stakeholders in their field.
Agency employees also are encouraged to present and engage with their peers in academia,
industry, government, and non-governmental organizations, consistent with their work
responsibilities. Such activities allow for individual professional growth and relationship-
building opportunities; information exchange and data sharing between scientists and policy
makers; and an identification of data gaps, discussions on how to fill gaps, and how science can
better support policy decision making. All of these benefits directly contribute to smarter and
more defensible decision-making actions.
The EPA supports and encourages its staff to obtain and keep current appropriate professional
certifications and publish their work in peer reviewed journals and scientific books. Many EPA
employees serve as members on editorial boards, officers on specialty sections and boards of
directors of professional societies and international scientific organizations. For example, a
National Exposure Research Laboratory scientist is a former President of the International
Society of Exposures Sciences. Also, a National Risk Management Research Laboratory
scientist has recently been nominated to serve on the board of the International Water
Association's Health Related Water Microbiology group.
As previously noted, multiple EPA offices and regions are supporting online webinars and
training courses. Here are some additional examples that are more directly related to professional
development.
OSWER is increasingly using webinars and other means to virtually attend meetings on science
and technology subjects related to its work. OSWER also provides specialized training to its staff
on a wide range of topics including health and safety, toxicology and risk assessment, contract
18
-------
management, and the operation of EPA's Emergency Operations Center. In addition, OSWER
sponsors training for members of the National Incident Coordination Team (NICT) and the
Response Support Corps that support emergency response activities.
The Office of Air and Radiation, in collaboration with other offices, provides webinar access to
both internal and outside expert presentations on cutting edge scientific issues, such as a recent
webinar on Designing Research to Assess Air Quality and Health Outcomes from Air Pollution
Regulations. OAR also provides training to other government agencies, such as through the
multi-agency training and guidance documents associated with cleaning up radioactively
contaminated sites.
A network of Regional Science Councils across EPA regions provides a forum for regional
scientists and engineers to focus on the use of science to address environmental issues. The
Councils work to increase science communication and improve science knowledge and expertise
at the regional level by hosting regional seminars, webinars and training. Some highlights in FY
2013 include:
At the EPA Region 6 office, seminars had previously been limited to regional staff.
However, beginning in 2013, the seminars became available as webinars with universities
with which the Region has memoranda of understanding. The August 2013 seminar
titled Waterborne Pathogen Research to Support Water Quality Regulations was by Dr.
George DiGiovanni of the School of Public Health at the University of Texas.
EPA Region 8 hosts "Science Shorts" and/or Brown Bag Seminars that detail select
scientific projects within the Region. The latest examples by Region 8 scientists include
"A Photovoltaic Array on a Green Roof in Denver, Colorado, Improves Resilience of
Green Roof Plants by Expanding Habitat Heterogeneity" by Thomas Slabe, and "A
Pesticide/Waste Indicator/ and Pharmaceuticals Summary of Region 8 Sites" by Kenneth
Dahlin.
Regional scientists are often asked to accompany project managers on site visits with local, state,
regional, and national customers to serve as technical experts on the science aspects of joint
projects. In 2013, Region 8 Laboratory scientists participated in site visits associated with the
ORD Hydro-fracking project, the Front Range Colorado Department of Public Health and
Environment (CDPHE) Emerging Contaminants Project, and the Regional Tribal Surface-Water
Pesticides Project, among others.
EPA employees also have access to university courses. For instance, the EPA Region 10 office
has a competitive Scientific/Technical Scholarship Program that provides funds to successful
applicants (Region 10 employees) to support them in maintaining and enhancing their scientific
and technical expertise.
Section 4: Challenges and Lessons Learned over the Past Year
As it implements the Policy, the Agency has been evaluating its guidance, processes, and
policies relevant to scientific integrity. This section of the report highlights the lessons learned
during the first year of implementation of the Policy.
19
-------
A. Progress Has Been Slowed by the Absence of a Full-time Scientific Integrity Official
The Policy established a Scientific Integrity Official to promote scientific integrity throughout
the Agency. The Official is independent of the EPA program offices and regions, so that s/he
may maintain an impartial perspective. The Official chairs the Scientific Integrity Committee
and serves as a champion for scientific integrity across the Agency. The Agency is in the process
of hiring a full-time Scientific Integrity Official during a time of resource constraints,
demonstrating the importance that the EPA places on scientific integrity.
B. Contractor-Managed Peer Review Process Needed Strengthening
As reported earlier, this year, the EPA developed a new process for Agency oversight of
contractor-led peer review panels. This was done in response to concerns raised by external
stakeholders regarding EPA's level of oversight on the selection of members of contractor-
managed peer review panels in regard to conflict-of-interest issues. The new process will reduce
the potential for organizational or personal conflict-of-interest concerns through greater public
participation and more rigorous internal review.
C. Implementation Process for the Policy
The Scientific Integrity Committee recognized the need to develop a process that would allow
for more input and help to ensure that the highest priority scientific integrity issues were
addressed. The outreach approach described in Section 2B is a way to align the review of
scientific integrity activities with existing processes to make it more efficient and to provide
additional opportunities for input from internal and external stakeholders.
In FY 2013, ORD required each of its Laboratories, Centers, and Offices (L/C/Os) to certify
compliance with the Agency Scientific Integrity Policy through their FMFIA annual assurance
process. In the FY 2014 annual assurance process, ORD's annual guidance to the L/C/Os will
require each L/C/O to submit specific categories of information needed to complete ORD's
contribution to the annual scientific integrity report.
Please refer to Section 2B Review of Activities for the Annual Report for future cross-Agency
implementation activities.
D. Coordination Procedures on Scientific Misconduct Allegations
While evaluating allegations concerning scientific integrity issues, the Scientific Integrity
Official and the Office of Inspector General recognized the need for an understanding of each
other's procedures and requirements, including the OIG investigation procedures for allegations
of scientific misconduct. The two groups also recognized the need for close coordination
between the Scientific Integrity Official and the OIG. Recently, the "Coordination Procedures
between the Scientific Integrity Official and the Office of Inspector General regarding Scientific
Misconduct Allegations" was finalized; it provides clear coordination procedures between the
Scientific Integrity Official and the OIG, as well as procedures to ensure communication with the
20
-------
relevant manager (i.e., the manager of the Office that is the focus of the allegation). The
coordination procedures also provide guidance in the disposition of allegations reviewed by the
Scientific Integrity Official and the communication back to the complainant. The Coordination
Procedures are in Appendix D.
E. Increasing Proficiency of Technical Staff
With reduced ability to hire new employees with skills to meet the EPA's current and future
scientific needs, it has become more important that the Agency evaluate ways of developing the
current workforce to maintain the technical proficiency and potential to be able to take on new
scientific and technical work.
The Regional Science Council (RSC) conducted a Science Needs Survey in 2012 that identified
a need to increase regional efforts to maintain the proficiency of technical and scientific staff.
The RSC proposed measures to address this weakness. Several have been adopted by Region 1
and acted upon; others are under consideration by regional leadership.
Technical training to support regional scientists and engineers (FY12 - Applied Statistics,
Toxicology; FY13 - Technical Writing)
Professional development seminar series. An annual series of approximately monthly
seminars dealing with a broad range of topics relevant to regional issues. Seminars host
both internal and external speakers with the intention of providing opportunities to enrich
staff understanding of technical and scientific issues.
As a result of the FY13 Science Forum, the Office Directors are considering the
establishment of a regional Advanced Monitoring Group. One function of that group will
be supporting professional development working with offices to coordinate Region 1
attendance at key environmental technology monitoring conferences (e.g. PittCon,
National Water Well Association, American Society of Civil Engineers, etc.) and other
events and then to transfer information on new monitoring technology to interested
regional staff.
Section 5: Recommendations for Future Actions by the Scientific Integrity Committee
This section describes topics that have been recommended for future action by the Scientific
Integrity Committee. It includes topics raised by Committee members and the Interim Scientific
Integrity Official during FY 2013 and by others within and outside of EPA.
Training
As previously noted, the Agency will deploy the first training module on the EPA Scientific
Integrity Policy in the Fall of 2013. The Committee will then consider whether additional
training is needed and if so, what its content should be. In addition, training on the new Peer
Review Handbook is planned for FY 2014.
21
-------
EPA Scientific Integrity Policy Review
The Policy provides for a biennial review cycle for the Committee to evaluate the Policy to
ensure its effectiveness; the Policy review is scheduled for FY 2014. The Committee will need to
develop the process for review and potential revision of the Policy.
Communications and Education Mechanisms
The Committee will consider developing improved communications and education mechanisms
to promote scientific integrity throughout the Agency. For example, the Committee could look
for existing EPA networks to more broadly communicate scientific integrity information. In
particular, the EPA quality community has an extensive network spreading across all program
and regional offices. The Office of Environmental Information (OEI) Quality Staff sponsors
monthly meetings (the "Quality Community Information Exchange", QCIX), which could be a
useful venue. In addition, a consolidated web page as a source of scientific integrity information
would be highly beneficial. A web page may also provide a means for submission of anonymous
information on scientific integrity issues. Currently there is no formal mechanism for submitting
such information within EPA.
Expressing Differing Scientific Opinions
The EPA Scientific Integrity Policy provides that the Scientific Integrity Committee develop a
mechanism for Agency employees to express differing scientific opinions. The Committee may
start by identifying the current mechanisms for Agency employees to express differing opinions,
and then work to improve them, fill gaps, etc. Other options on this particular topic may need to
be developed.
Section 6: Closing Statement and Request for Comments
This first Annual Report on Scientific Integrity was prepared by staff in the Office of the Science
Advisor. It will be obvious to the reader that, with the help of the members of the Scientific
Integrity Committee and their colleagues, many people contributed to its content. An early draft
was reviewed by Committee members and appropriate changes made. The final draft was
reviewed, revised and approved for release by the Interim Scientific Integrity Official, who is
therefore responsible for any errors of commission or omission. He and the OSA staff welcome
any comments or suggestions, but ask that they be focused on making the second Annual Report
better than the first.
22
-------
APPENDICES
A. List of Deputy Scientific Integrity Officials
B. Summary of Comments and Recommendations from the Scientific Integrity Annual
Listening Session
C. Summary of Recommendations from the Webinar, "A Conversation with the Interim
Scientific Integrity Official"
D. Coordination Procedures between the Scientific Integrity Official and the Office of Inspector
General regarding Scientific Misconduct Allegations
E. Conflict-of-interest Review Process for Contractor-Managed Peer Reviews
23
-------
APPENDIX A
EPA Scientific
Integrity Committee - Deputy Scientific Integrity Officials i
Office/Region
Official
CHAIR & EPA Interim
Scientific Integrity Official
Glenn Paulson
OAR
Betsy Shaw
OARJVT
Nanci Gelb
OCFO
David Bloom
OCSPP
David Dix
OECA
Carol Rushin
OEI
Vaughn Noga
OGC
Carol Ann Siciliano
OITA
Michael Stahl
OP
A1 McGartland
ORD
Robert Kavlock
OSWER
Barry Breen
OW
Mike Shapiro
AO (SABSO, OFACMO,
OEAEE, OCHP)
John Reeder
OSA
Mary Greene
Region 1
Robert Maxfield
Region 2
Marie O'Shea
Region 3
John Forren
Region 4
Thomas Baugh
Region 5
Carole Braverman
Region 6
David Neleigh
Region 7
Ronald Hammerschmidt
Region 8
Debra Thomas
Region 9
Brenda Bettencourt
Region 10
Joyce Kelly
October 2013)
24
-------
APPENDIX B
Summary of Comments and Recommendations from
the Scientific Integrity Annual Listening Session
Session 1: Union of Concerned Scientists
EPA is one of the three strongest agencies, departments and offices of the 23 that the
UCS surveyed (in terms of the strength of their scientific integrity policies).
EPA and DOI should be praised for appointing a full-time Scientific Integrity
Official, as all other agencies have Officials with many other duties in their portfolio.
Concentrate scientific integrity information on one website.
Review the mechanisms for Agency employees to express differing scientific
opinions.
Allow scientists to speak on "background" to reporters without a public affairs person
in attendance.
Set up a process for evaluating the training, because training is critical.
Reach out to other federal agencies to identify common issues, share best practices,
etc.
Session 2: Science Advisory Board (SAB) and Office of the Inspector General (OIG)
SAB
"EPA is in good shape" on science integrity issues.
Explore ways to track any political influence on EPA science products.
Elicit information from Regional and Program Office personnel, perhaps via an
anonymous survey.
OIG
The Office of Investigations is the point of contact for any information forwarded on
science integrity issues, and the OIG would have exclusive oversight of potentially
criminal matters. Otherwise, once an allegation is made, the Scientific Integrity
Official and OIG would coordinate in responding to the allegation based on its scope
and other attributes as well as the respective responsibilities of the OIG and the
Scientific Integrity Committee and Official.
Agency officials should ensure that training covers how to report violations of the
policy through the OIG hotline.
25
-------
Session 3: EPA Unions
EPA unions support the formation of committees of scientists and engineers to
resolve disputes over science integrity issues at the lowest possible management level
(e.g., branch level) within the EPA organization.
An implementation plan should include employee input and then be revised every
other year, involving managers, staff and union work groups to assist in resolving
employee and management issues.
The OSA website should include links to scientific integrity related processes and
documents.
Scientific integrity work groups comprised of staff should interact using SharePoint
software once the Agency launches that application.
Standing committees should address quality management assurance and other
subtopics related to the unique challenges faced by field scientists, laboratory staff,
engineers and managers.
Session 4: Regional Administrators (RAs) and Assistant Administrators (AAs)
There are limited time and resources available for training in the regions; do not
duplicate the content of ethics training.
Compile and make available early memoranda and scientific integrity related policies
that pre-date the Policy.
Create a solid foundation going forward, by applying the policy consistently and
promoting buy-in among staff.
26
-------
APPENDIX C
SUMMARY OF RECOMMENDATIONS FROM THE WIHIVAR.
"A CONVERSATION WTTH THE INTERIM SCIENTIFIC INTEGRITY OFFICIAL"
Modify the Scientific Integrity Policy to take into account differences in office
cultures regarding clearance procedures, expressing personal opinions, etc.
Focus on the process for expressing different scientific opinions. There were several
suggestions on this, including:
o Require written responses from management to the differing professional
opinions
o Make the expression of differing professional opinions part of the decision
process and include it in the decision record
o Explore options to encourage people to speak out on scientific issues
Review NAS's Science for Environmental Protection: the Road Ahead (2012) report
for applications to scientific integrity issues.
Consider involving the Scientific Integrity Committee in the Lab Enterprise Study.
Increase the budget for training scientific personnel:
o Some employees are not sufficiently trained in current technologies
o We need to hire more scientists and use those scientists in non-contract
oversight capacities
Clarify how upholding scientific integrity principles might overlap with reporting
waste, fraud, and abuse.
Emphasize that the Scientific Integrity Policy applies to all employees, not just
scientists and engineers.
Scientific Integrity Committee members may not necessarily be scientists or
engineers, may have conflicts of interest with respect to the Agency regulatory
agenda, and may not necessarily understand the issues that they are charged with
acting on (including a personal understanding of a defensible and credible scientific
product).
Create an ad hoc committee to review scientific misconduct allegations. The
suggestion included developing details on how the process would work.
The Scientific Integrity Policy and the Data Quality Act/Information Quality
Guidelines both address transparency and integrity. How does the Scientific Integrity
Policy correlate to the IQG requirements in these areas?
Right now the ethics rules have been interpreted to say that if a scientist is not funded
to travel to a scientific meeting, the scientist cannot attend the meeting except as a
private citizen and also cannot discuss the EPA scientific work that s(he) is doing.
There should be a mechanism to have any presentation or poster screened through the
Agency and released for presentation at a scientific meeting as a way to forward our
science to the public in a timely way and also allow for professional development.
If meeting organizers are paying for travel of EPA scientists, may the scientists
attend?
27
-------
Is there any recourse if managers deny conference travel, not because of a lack of
money in the travel budget, but because it might "look bad" to send scientists to
present research in conferences in the current "belt-tightening" political climate?
EPA experts were invited to participate in a workshop that was bringing together
diverse stakeholders on a topic on which EPA is actively working. It took 3 months of
negotiation, with the outcome that no one from EPA was allowed to participate.
Another organization was paying for travel of the participants. Why is there
reluctance at EPA to allow its scientists to participate in meetings on controversial
topics?
Could you possibly tell us more about how the 2012 Policy applies to EPA
contractors? What if the contract was awarded prior to February 2012?
Information on scientific integrity should be much more visible and easy to find at
both epa.gov and intranet.epa.gov. For example, I found it difficult or impossible to
find info to answer the following
o Who do I call if I have questions or an "issue" to discuss / report? Where can I
find a list of Deputy Scientific Integrity Officials?
o Where can I find a suggestion box?
One of the highest value issues is that the EPA is viewed as a top echelon scientific
organization. With that in mind, we need to identify the highest scientific concerns
and ensure that the science leads the discussion e.g. dioxin, TCE, lead, arsenic, etc.
Will there be any efforts made in the coming year to ensure that journalists covering
EPA issues can have timely access to EPA's technical experts?
What have been the results of issues identified this last year? Will you have yearly
reporting of results?
28
-------
APPENDIX D
Office of the Science Advisor
Coordination Procedures between the Scientific Integrity Official and the Office of
Inspector General regarding Scientific Misconduct Allegations
September 19, 2013
For the purposes of this document, the term Scientific Misconduct Allegation is defined as an
assertion of scientific misconduct, which includes fabrication, falsification, plagiarism or
misrepresentation in proposing, performing or reviewing scientific or research activities.
Scientific misconduct does not include honest error or differences of opinion.
A. Scientific Misconduct Allegations First Received by the Agency Scientific Integrity
Official (ScIO)
1. Upon receipt of an allegation, the ScIO will within seven days: a) refer the allegation
to the Office of Inspector General (OIG) Hotline, and b) with appropriate members of the
Scientific Integrity Committee (ScIC), review the allegation in consultation with the OIG
and develop and implement a plan for its disposition. Note that some allegations may
require the ScIO to take immediate action before a formal meeting of the ScIC. These
actions will be communicated to the OIG.
2. Within 30 days of the receipt of the referral, the OIG will inform the ScIO of the
status of any action on the referral.
B. Scientific Misconduct Allegations First Received by the Office of Inspector General
1. The OIG will contact the ScIO to discuss the allegation, as appropriate.
2. The OIG will then inform the ScIO about its decision regarding disposition of the
allegation. The OIG disposition will consist of one of the three following options:
a. There is no further OIG interest in the allegation.
b. OIG needs to gather additional information prior to making a determination
regarding the disposition of the allegation, or
c. OIG will start an audit, evaluation, or other action and will contact the ScIO
for assistance as needed. The audit or evaluation will follow OIG Hotline and/or
investigative policy and procedures.
3. If option B2(c) is selected - i.e., the OIG is going to start an action, then the OIG will
send a memorandum to the ScIO to that effect.
29
-------
C. Communication with the Relevant Manager
The ScIO will provide information to the relevant manager or office for further action, as
appropriate. The SciO may also request further information from the relevant manager or office,
as needed.
D. Disposition of Allegations Reviewed by the ScIO (Section A)
For allegations reviewed by the ScIO and ScIC, the ScIO will document the resolution of the
issue, including the response to the complainant. This documentation will be reviewed by the
ScIC and sent to the OIG Hotline Coordinator.
E. Communication Back to the Complainant
The ScIO will respond to the complainant for allegations in which the OIG has no further
interest. The ScIO will work with the ScIC and other senior Agency personnel, as appropriate,
to develop the response to the complainant.
30
-------
APPENDIX E
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D C. 20460
MAY - 2 2013
OFFICE OF THE
ADMINISTRATOR
MEMORANDUM
SUBJECT: Conflict-of-interest Review Process for Contractor-Managed Peer Reviews
FROM: Bob Perciasepe, Acting Administrator
TO
Assistant Administrators
Regional Administrators
As you may already know, at the March 13 meeting of the Science and Technology Policy Council,
agency managers discussed and approved the attached conflict-of-interest review process. I want to
thank you for your council representative's thoughtful views and attention to this important process
change. The new process will apply to all scientific or technical work products designated as influential
scientific information or a highly influential scientific assessment in which expert peer reviews will be
conducted by panels selected and managed by an independent contractor to the U.S. Environmental
Protection Agency.
Under the new process there will be a Federal Register notice soliciting nominees for each peer-review
panel, followed later by notices or a posting online to obtain feedback on potential candidates and
announce both the panel meeting and the names of the peer-review panel members. Fundamental to the
new process are these elements of public involvement not previously undertaken in identifying panel
members. The new process will enhance the transparency of contractor-led peer reviews and is intended
to reduce any potential for organizational or personal conflict-of-interest concerns through greater public
participation and more rigorous internal review.
Please continue ongoing contractor-led peer reviews; the new process will apply to future activities.
However, it is critically important that contract vehicles that support peer review be modified to
implement the process changes in the attached description as soon as possible. To accomplish that, I am
asking that you identify the relevant contracts and that the Office of Administration and Resources
Management assist you in modifying the contracts to enable implementation.
Adopting this new process will also require developing an implementation plan. To facilitate
incorporating the new process into the Peer Review Handbook and the timely implementation of the
process, I am requesting that the Peer Review Advisory Group complete within the next 60 days
implementation-plan development with assistance from the Science and Technology Policy Council
Steering Committee.
Internet Address (URL) http://www.epa.gov
Recycled/Recyclable Printed with Vegetable Oil Based Inks on 100% Postconsumer, Process Chlorine Free Recycled Paper
-------
Thank you for your support and attention to this important change in our peer review process.
Attachment
Cc: Science Advisor
STPC
STPC Steering Committee
Peer Review Advisory Group
Kenneth Redden
John Bashista
-------
Conflicts of Interest Review Process for Contractor-Managed Peer Reviews
Of EPA HISA and ISI Documents 1
This process will apply to all future technical documents designated as Influential Scientific Information
(ISI) or Highly Influential Scientific Assessment (HISA) where independent peer reviews will be
conducted by panels selected and managed by an independent Contractor under contract with EPA. No
Contractor is required to comply with this process until it is incorporated into the Contractor's peer
review contract and tasking documents, as appropriate.2
1. The relevant EPA office will publish a Call for Experts in the Federal Register which will identify
the types of disciplinary expertise needed and announce either the availability of the document to
be reviewed or publish a brief synopsis of the document and indicate that an announcement of the
availability of the document will follow at the time of its release. The Call will be open for a
minimum of three weeks. The notice will direct members of the public, stakeholder groups, and
other organizations to submit nominations to the Contractor. In addition, experts can nominate
themselves. As an alternative to publishing subsequent notices in the Federal Register, this first
notice may provide a link to an appropriate website for the public to request automatic
announcements of the peer review panel formation process and activities.
2. At the same time, the Contractor will use traditional techniques (e.g., a literature search) to
identify additional qualified candidates in the disciplines identified by EPA. The Contractor will
screen the nominees submitted as a result of the public Call for Experts to ensure that they have
the types of disciplinary expertise listed in the Federal Register notice. The Contractor will use
this screening combined with the list of qualified candidates identified by the Contractor through
traditional techniques to produce a list of potentially qualified candidates.
3. The Contractor will ask the potentially qualified candidates whether they are interested and
available for the date of the peer review panel's meeting. From each person who is interested and
available, the Contractor will obtain: (a) a current resume; (b) a completed conflict of interest
form containing the information specified by EPA and (c) if not in the resume, a list of his or her
pertinent publications during the past two years, with identification of the funding source(s) for
1 This process is focused on the rigor of the process for identifying and evaluating actual or potential conflicts of
interest and not on how conflicts of interest are to be substantively addressed. Implementation guidance will be
developed for this process.
2 This process does not apply to the National Academy of Sciences.
-------
the work presented in each publication. The Contractor will supplement the information obtained
from the candidates by Internet searches and other appropriate methods.
4. Based on the information it obtains or collects, the Contractor will develop an interim list of
potential peer review panel members based on: (a) expertise, knowledge and experience of each
individual, and (b) a detailed identification, review, and analysis of any potential organizational
or personal conflict of interest concerns.
5. EPA will then publish, in the Federal Register (or post on the website identified in the Call for
Experts notice); the names, principal affiliations, and Contractor-prepared abbreviated resumes of
those on the interim list. The Contractor will solicit public comment for a minimum of three
weeks on their candidacy. After the close of the comment period, the Contractor will collect and
review the information received from the public.
6. The Contractor will select, after considering information about the candidates received from the
public, a proposed final peer review panel based on: (a) expertise, knowledge, and experience of
each individual, (b) adherence to the conflict of interest requirements in the contract,3 and (c)
panel balance with respect to the expertise required to conduct the review and the diversity of
relevant scientific and technical perspectives.
7. After selecting a proposed final peer review panel, the Contractor will schedule a consultation
with the EPA Science Advisor or his/her designee (hereinafter "EPA Science Advisor") and the
EPA Contracting Officer managing the contract to review the process used by the Contractor to
review organizational or personal conflicts of interests related to the panel, to explain the manner
in which the Contractor (and the proposed subcontractor peer reviewers) followed applicable
conflicts of interest guidance and requirements, and to identify and provide input on any issues
concerning potential conflicts. If a specific potential conflict of interest is identified, the EPA
Contracting Officer, in consultation with the EPA Science Advisor, OGC and, as appropriate,
3 Among them are requirements established in the contract to: (1) Evaluate and update specific conflict of interest-
related financial and professional information related to peer review candidates and their spouses (e.g., assets,
liabilities, funding sources for research, employment information, compensated consulting and expert witness
activities, and previous public statements related to the chemical or matter under review); (2) Follow guidance
articulated in EPA's current Peer Review Handbook and related documents (currently the 2006 Science Policy
Council Peer Review Handbook, 3rd Edition, and the 2009 Addendum to the Peer Review Handbook) that address
identification and resolution of appearances of a loss of impartiality; (3) Certify that the contractor has evaluated
each candidate, and that there is no significant evidence to support a finding of an actual or potential conflict of
interest, or an appearance of a lack of impartiality; and (4) Report and disclose to EPA the nature of such conflicts
or appearances when the certification cannot be made.
-------
with other Agency offices (e.g., program office), will determine whether the Contractor's
proposed plan to avoid, mitigate or neutralize the conflict is acceptable and, if not, direct the
Contractor to take additional measures as may be required. The Contractor will incorporate these
measures into a plan for avoiding, neutralizing or mitigating the conflict of interest and will
submit the plan to the EPA Contracting Officer for approval. Once the plan is approved the
Contractor will select the final peer review panel and finalize its subcontractor arrangements with
the peer reviewers. If at anytime, the Contracting Officer, following consultation with the EPA
Science Advisor and OGC, determines that an individual is not qualified to serve due to a conflict
of interest that cannot be avoided, neutralized or mitigated, the Contracting Officer will direct the
Contractor that the individual is ineligible to serve as a peer reviewer on this specific matter.
8. EPA will publish, in the Federal Register (or post on the website identified in the Call for Experts
notice), the subject, date, time and place of each peer review panel meeting at least three weeks
before the panel meeting, along with the names of the peer review panel members. This notice
will also include the following language: "EPA requests that no individual or organization contact
in any way its contractor or the subcontractor panel members regarding the subject of the peer
review meeting, send them written materials regarding the subject of the meeting, or make any
offers or requests to any of them that appear to be linked to their participation in the peer review.
The Contractor shall direct the panel members to report any such contacts to the Contractor, who
will take appropriate action in consultation with EPA to ensure the independence and impartiality
of the peer review."
9. The Contractor will obtain updated conflicts of interest information from the panel members two
weeks prior to each peer review meeting. The Contractor will examine any updated forms and
assess whether the individuals are still eligible to serve. The Contractor will notify the EPA
Contracting Officer who will consult with the EPA Science Advisor and OGC on any conflict of
interest issues. If a specific potential conflict of interest is identified, the EPA Contracting
Officer, following consultation with the EPA Science Advisor, OGC, and as appropriate, with
other Agency offices (e.g. program office) will determine whether appropriate measures can be
taken to avoid, mitigate or neutralize the conflict and, if so, the Contracting Officer will provide
those to the Contractor. Should the EPA Contracting Officer, following consultation with the
EPA Science Advisor and OGC, determine that an individual is not qualified to serve due to a
conflict of interest that cannot be avoided, neutralized or mitigated, the Contracting Officer will
direct the Contractor that the individual is ineligible to serve as a peer reviewer on this specific
matter. The Contractor will incorporate these measures and directions into a plan for avoiding,
-------
neutralizing or mitigating the conflict of interest and will submit the plan to the EPA Contracting
Officer for approval.
10. Members of the panel will review, update if necessary, and re-sign their conflict of interest forms
at the start of each peer review meeting. For conflicts issues identified at the start of the meeting,
the EPA Contracting Officer will direct the Contractor to implement the appropriate measures or
directions immediately. Subsequently, the Contractor will, as soon as practicable, incorporate
those measures or directions into a plan for avoiding, neutralizing or mitigating the conflict of
interest and submit the plan to the EPA Contracting Officer for his/her approval.
11. As soon as practicable after this process is formally adopted, EPA will provide training and
guidance to Contractors working under an existing contract involving peer review panel
selections and in future years at the start of each new contract involving peer review panel
selections. The training and guidance will cover applicable conflicts of interest guidance and
requirements and the appropriate implementation of the process outlined above.
In addition to this specific process for each peer review covered by this policy, EPA will undertake,
through the Office of the Science Advisor, and in coordination with the EPA Office of Acquisition
Management and, as appropriate, the Office of General Counsel, a periodic review of the conflict of
interest forms and other relevant information for individuals selected to serve on Contractor-led peer
review panels. The qualifications and diversity of panel members, and the balance of biases on panels,
may also be considered during these periodic reviews.
Approved by the Science and Technology Policy Council at its meeting on March 13, 2013.
Glenn Paulson, Ph.D.
Date
Science Advisor
-------
^ ^ _ y 'O
|ฎj
What Did You Think?
We strive to constantly provide the highest level of value for you. Please take a few minutes
to tell us about your experience using this product.
To be taken to a short consumer satisfaction survey, please click here or copy and paste the
following URL into your browser:
https://www.surveymonkev.eom/r/OSAconsumerfdbck?
product=Annual Report on Scientific Integrity
Thank you for your feedback.
Sincerely,
Office of the Science Advisor
United States Environmental Protection Agency
www.epa.gov/OSA@epa.gov
------- |