y§z)
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Audit Report
Office of Research and
Development Should
Increase Awareness of
Scientific Integrity Policies
Report No. 11 -P-0386
July 22, 2011
-------
Report Contributors: Patrick Gilbride
Erin Barnes-Weaver
Alicia Mariscal
Ashley Sellers-Hansen
Karen L. Hamilton
Luke Stolz
Abbreviations
EPA U.S. Environmental Protection Agency
GAO U.S. Government Accountability Office
NPC National Partnership Council
OIG Office of Inspector General
ORD Office of Research and Development
OSTP Office of Science and Technology Policy
STPC Science and Technology Policy Council
Cover photo: Analysis performed at the Cincinnati Laboratory. (EPA photo)
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
e-mail: OIG Hotline@epa.gov write: EPA Inspector General Hotline
phone: 1-888-546-8740 1200 Pennsylvania Avenue NW
fax: 703-347-8330 Mailcode 8431P (Room N-4330)
online: http://www.epa.gov/oig/hotline.htm Washington, DC 20460
-------
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
11-P-0386
July 22, 2011
Why We Did This Review
The U.S. Environmental
Protection Agency (EPA),
Office of Inspector General,
initiated this review to
determine the process EPA
uses to address
scientific/research misconduct
I Specifically, we sought to
determine whether EPA's
Office of Research and
Development (ORD) has
controls to address scientific
integrity and research
misconduct, and whether those
controls are effective.
Background
Since 2000, a number of
federal and EPA policies on
ensuring the integrity of
government science have been
issued. ORD is EPA's lead
office for integrating science
into environmental protection
policies. EPA Order 3120.5
implements the federal policy
on research misconduct, and
ORD and others formulated the
Principles of Scientific Integrity
and the Principles of Scientific
Integrity E-Training to further
highlight professional ethics for
EPA scientists.
For further information,
contact our Office of
Congressional, Public Affairs
and Management at
(202)566-2391.
The full report is at:
www.epa.qov/oiq/reports/2011/
20110722-11-P-0386.pdf
Catalyst for Improving the Environment
Office of Research and Development Should Increase
Awareness of Scientific Integrity Policies
What We Found
ORD has internal controls that include policies, procedures, training, and peer
review. However, ORD should improve how it evaluates the effectiveness of its
policies and procedures for scientific integrity and research misconduct.
Currently, ORD does not test its policies and procedures because ORD asserts
that few reported instances of misconduct means that it generally does not occur.
However, few identified instances of research misconduct could signal that staff
lacks awareness of key criteria and reporting requirements necessary to identify
and report misconduct.
We issued an electronic survey to 1,371 ORD science staff and received 488
responses. We found that 65 percent of respondents were unaware of EPA
Order 3120.5, and 32 percent were unaware of EPA's Principles of Scientific
Integrity. We also found that ORD has not updated the Principles of Scientific
Integrity E-Training since June 2005. The existing e-training is not mandatory
for ORD staff and does not include actual examples to aid understanding by
training participants. Those who have not completed the training may be
unaware of key criteria regarding scientific integrity. To facilitate reporting of
research misconduct, ORD should increase awareness of the process. Without
these additional internal control efforts, ORD risks having its science called into
question, potentially lessening the credibility of its work.
What We Recommend
We recommend that the Assistant Administrator for Research and Development
periodically test the effectiveness of controls in place to address scientific
integrity and research misconduct. We also recommend that ORD work with
EPA offices to initiate outreach on EPA Order 3120.5 to raise awareness on
roles/responsibilities and reporting steps, and to identify EPA staff and managers
who should complete the Principles of Scientific Integrity E-Training. Lastly, we
recommend that ORD continue to work with unions to update and implement
e-training. Such updates should include making the e-training mandatory for all
ORD staff, ensuring that the updated course contains actual examples, and
creating a system for maintaining current contact information for reporting
instances of scientific integrity and research misconduct. ORD agreed with our
recommendations and subsequently followed up with its corrective action plan.
We have reviewed the corrective action plan and milestone dates, and found
them acceptable. As such, we are closing this report upon issuance, and no
further response is required.
-------
f
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT:
FROM:
TO:
July 22, 2011
Office of Research and Development Should Increase
Awareness of Scientific Integrity Policies
Report No. ll-P-0386
Arthur A. Elkins, Jr.
Inspector General
Paul Anastas
Assistant Administrator for Research and Development
This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). This report contains findings that describe the
problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established audit resolution procedures.
The estimated direct labor and travel costs for this report are $266,685.
Action Required
On July 7, 2011, your office provided its response and subsequently provided its corrective
action plan. We have reviewed the corrective action plan and milestone dates, and found them
acceptable. As such, we are closing this report upon issuance, and no further response is
required. Your response will be posted on the OIG's public website, along with our
memorandum commenting on your response. We have no objections to the further release of this
report to the public. We will post this report to our website at http://www.epa.gov/oig.
If you or your staff have any questions regarding this report, please contact Melissa Heist,
Assistant Inspector General for Audit, at (202) 566-0899 or heist.melissa@epa.gov: or Patrick
Gilbride, Director, Risk Assessment and Program Performance Audits, at (303) 312-6969 or
gilbride.patrick@epa.gov.
-------
Office of Research and Development Should Increase
Awareness of Scientific Integrity Policies
11-P-0386
Table of Contents
1 Introduction 1
Purpose 1
Background 1
Noteworthy Achievements 5
Scope and Methodology 5
2 Opportunities Exist for ORD to Better Address
Scientific Integrity and Research Misconduct 7
ORD Has Controls in Place to Address Scientific Integrity but
Does Not Test Them 7
ORD Staff Unaware of Key Criteria and Reporting Requirements 8
Principles of Scientific Integrity E-Training Should Be
Updated and Mandatory 10
Conclusion 11
Recommendations 11
Agency Comments and OIG Evaluation 12
Status of Recommendations and Potential Monetary Benefits 13
Appendices
A Survey Instrument 14
B Survey Analysis Results by Question 18
C Agency Response to Draft Report 22
D Distribution 35
-------
Chapter 1
Introduction
Purpose
We sought to determine the process the U.S. Environmental Protection Agency
(EPA) uses to address issues related to scientific integrity and research
misconduct. Because the Office of Research and Development (ORD) leads
integrating science into environmental protection policies within EPA, we focused
on ORD using the following objectives:
• Does ORD have controls to address scientific integrity and research
misconduct?
• How effective are the controls ORD has in place to address scientific
integrity and research misconduct?
Background
ORD is EPA's lead office for the production, review, and integration of scientific
and technical knowledge into environmental protection policies and regulations.
ORD conducts work within its labs, centers, and offices. ORD is headquartered in
Washington, DC, and has laboratories and centers across the country, with its
main research facilities in Ohio and North Carolina (figure 1). ORD's enacted
fiscal year 2011 budget was $582.1 million, authorizing 1,907.2 full-time staff.
Figure 1: ORD's locations
Corvallis OR
Newport OR |s_r-><_ Duiuth.MN Cincinnati, OH
GrosselleJ
rA^
Narragansett, Rl
Edison, NJ
Washington, DC
•Research Triangle
Park, NC
Las Vegas, NV \ / J ^k^^rC^Y ^Athens GA
y
Ada, OK
Source: ORD intranet site (accessed May 2011).
11-P-0386
-------
History of Policies on Scientific Integrity and Research Misconduct
In 2000, EPA's National Partnership Council (NPC)1 developed the Principles of
Scientific Integrity as a policy statement establishing a set of professional ethics
for EPA scientists. The EPA Administrator promulgated the Principles of
Scientific Integrity on March 28, 2000. They state:
It is essential that EPA's scientific and technical activities be of
the highest quality and credibility if EPA is to carry out its
responsibilities to protect human health and the environment.
Honesty and integrity in its activities and decision-making
processes are vital if the American public is to have trust and
confidence in EPA's decisions.
The Principles of Scientific Integrity require all employees to:
• Ensure that their work is of the highest integrity
• Represent their own work fairly and accurately
• Represent and acknowledge the intellectual contributions of others
• Avoid financial conflicts of interest and ensure impartiality
• Be cognizant of and understand the specific programmatic statutes
• Accept the affirmative responsibility to report any breach
• Welcome differing views and opinions on scientific and technical matters
On December 6, 2000, the Executive Office of the President issued the
government-wide Federal Policy on Research Misconduct. The Federal Policy on
Research Misconduct applies to all federally funded and conducted research and
research proposals. The Federal Policy on Research Misconduct required all
federal agencies that conduct or support research to implement the policy by
December 6, 2001.
On March 18, 2003, EPA implemented its Policies and Procedures for Addressing
Research Misconduct through EPA Order 3120.5. The order defines research
misconduct as fabrication, falsification, or plagiarism in proposing, performing, or
reviewing research, or in reporting research results; or ordering, advising or
suggesting that subordinates engage in research misconduct. EPA Order 3120.5
applies to all research conducted, sponsored, or funded, in whole or in part, by
EPA (including research done by EPA contractors), as well as to research
proposals submitted to EPA. The order applies to all Agency employees,
contractors, and recipients of assistance agreements. The order directs each to
1 According to its charter, the NPC promotes implementation of partnership concepts throughout the Agency to
investigate, study, discuss, and propose solutions to the Administrator on a wide range of issues affecting EPA
employees, or to make specific decisions when delegated authority to do so by the Administrator. The NPC
comprises one officer from each local union that represents EPA bargaining unit employees, the president of any
national councils having exclusive recognition in EPA, and a group of senior management officials designated by
the Administrator.
11-P-0386 ;
-------
report allegations to the appropriate EPA staff and/or management, and to the
OIG under certain conditions.2 While EPA retains the ultimate oversight authority
for EPA-supported research, research entities bear primary responsibility for
preventing and detecting research misconduct.
In June 2005, EPA's NPC developed the Principles of Scientific Integrity
E-Training to acquaint EPA's professional scientists with the details and
implications of the Principles of Scientific Integrity. ORD played a major role in
assisting the NPC's development of the Principles of Scientific Integrity and the
e-training.
On March 9, 2009, the President signed the "Memorandum for the Heads of
Executive Departments and Agencies—Scientific Integrity." The memorandum
directed the President's Office of Science and Technology Policy (OSTP) to
"develop recommendations for Presidential action designed to guarantee scientific
integrity throughout the executive branch."
EPA responded to the President's memorandum on May 9, 2009. In the
memorandum, "Scientific Integrity: Our Compass for Environmental Protection,"
the EPA Administrator asked EPA's Science Policy Council3 to strengthen policies
and procedures that ensure scientific integrity within the Agency, including:
• Inventorying all of EPA's guidelines and policies that relate to
scientific integrity to look for gaps and possible areas for improvement
• Updating and reaffirming EPA's Peer Review Handbook and
recommending how to improve implementation of peer review policies
across programs and regions
• Working with the NPC to reaffirm the Agency' s Principles of
Scientific Integrity and update the e-training.
On December 17, 2010, the director of OSTP issued a memorandum to guide
executive departments and agencies in implementing the Administration's
policies on scientific integrity. OSTP's memorandum outlined five areas:
1. Foundations of Scientific Integrity in Government
2. Public Communications
3. Use of Federal Advisory Committees
EPA OIG must be notified immediately of any allegation of research misconduct that involves the following:
public health or safety is at risk, agency resources or interests are threatened, circumstances where research activities
should be suspended, there is a reasonable indication of possible violations of civil or criminal law, federal action is
required to protect the interests of those involved in the investigation, the research entity believes that the inquiry or
investigation may be made public prematurely so that appropriate steps can be taken to safeguard evidence and
protect the rights of those involved, and circumstances where the research community or public should be informed.
3 EPA established the Science Policy Council in 1993 and reconstituted it in 2010 as the Science and Technology
Policy Council (STPC). The STPC serves as a mechanism for addressing EPA's many significant science policy
issues that go beyond regional and program boundaries. The STPC is chaired by the Science Advisor to the Agency
(ORD's Assistant Administrator) and comprises senior managers from EPA programs and regions.
11-P-0386 3
-------
4. Professional Development of Government Scientists and Engineers
5. Impl ementati on
The memorandum directed all agencies to report within 120 days the actions
taken to develop and implement policies on scientific integrity. EPA responded to
this memorandum on April 18, 2011 (see "Noteworthy Achievements" section).
Internal Control Standards
A November 1999 report by the U.S. Government Accountability Office (GAO),
Standards for Internal Control in the Federal Government, defines "internal
control" as an integral component of an organization's management that provides
reasonable assurance of effective and efficient operations and compliance with
applicable laws and regulations. Office of Management and Budget Circular
A-123 (revised 2004) provides corresponding goals, stating, among other matters,
that agency managers take timely and effective action to correct internal control
deficiencies. As GAO recognized, an internal control comprises the plans,
methods, and procedures used to meet missions, goals, and objectives and, in
doing so, supports performance-based management. Internal control is not one
event, but a series of actions and activities that occur throughout an entity's
operations and on an ongoing basis. In this sense, internal control is management
control built into the entity to help managers achieve their aims on an ongoing
basis. GAO identified five internal control standards (table 1).
Table 1: GAO's Standards for Internal Control in the Federal Government
Control
Environment
Risk Assessment
Control Activities
Information and
Communication
Monitoring
This establishes and maintains an environment throughout the
organization that sets a positive and supporting attitude toward
internal control and conscientious management. This includes
establishing goals, objectives, and performance measures at the
entity and activity level.
A precondition to risk assessment is the establishment of clear,
consistent agency objectives. The internal control risk
assessment process includes assessing risks the agency faces
from both internal and external sources. Managers need to
comprehensively identify risks and should consider all significant
interactions between the entity and other parties as well as
internal factors at both the entity-wide and activity levels.
These are policies, procedures, techniques, and mechanisms
that implement management's direction to achieving goals.
These help ensure that management's directives are carried out.
This includes data and information (performance and financial)
to determine whether the organization meets its goals and
objectives and maintains accountability over resources.
This assesses the quality of performance over time and ensures
that audits and other review findings are promptly resolved.
Source: OIG's summary of GAO's Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1, November 1999.
All EPA organizations are required to develop and implement a systematic
strategy for reviewing internal controls. These reviews provide managers and
11-P-0386
-------
employees reasonable assurance that their programs operate effectively and
efficiently. It is important for a science organization such as ORD to have internal
controls to ensure adequate review and action on its research products and those
produced by others receiving EPA funds.
Noteworthy Achievements
ORD developed an ad hoc committee to work with the Science and Technology
Policy Council (STPC) to respond to OSTP's December 17, 2010, memorandum.
EPA's April 18, 2011, response stated that the Agency would strengthen its
commitment to scientific integrity by developing an Agency-wide scientific
integrity policy by October 15, 2011. The response also said that EPA would
establish a senior scientific integrity official to champion scientific integrity
across EPA and form a standing committee of officials representing each EPA
office. EPA's response also emphasized Agency activities in four areas:
foundations of scientific integrity in government, public communications, use of
federal advisory committees, and professional development of government
scientists and engineers. Additionally, ORD continues to work with the NPC and
employee unions to update the e-training and believes that the e-training could
serve as a best practice for other agencies.
ORD has an extensive process for peer review that addresses both internal and
external peer reviews, as well as conflicts of interest. EPA believes that its peer
review process can function as a model for other federal agencies. According to
the Peer Review Handbook, the peer review process helps the Agency ensure that
it bases decisions or positions on credible science products.
Lastly, to address internal-control-related program risks, ORD stated that it
recently implemented a risk assessment protocol for its labs, centers, and offices.
The protocol included scientific integrity as one potential risk category and asked
respondents to address the likelihood of risk events that compromise scientific
integrity and actions to mitigate risks. ORD plans to complete an office-wide risk
assessment by August 31,2011.
Scope and Methodology
We conducted our work from July to October 2010 and from February to June
20114 in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform our review to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our objectives. We believe that the evidence obtained
provides a reasonable basis for our findings and conclusions.
4 We suspended our work from October 2010 until February 2011, for a total of 120 days, to address a higher-
priority assignment.
11-P-0386
-------
To address our first objective, we reviewed policies, procedures, and guidance on
scientific integrity and research misconduct. We also interviewed EPA staff to
determine the number of allegations of misconduct and internal controls in place
to mitigate allegations. Specifically, we interviewed OIG hotline and
investigations staff, ORD staff and managers, and EPA union representatives.
To address our second objective, we compared controls EPA had in place against
those of other science agencies, such as the National Science Foundation, the U.S.
Department of Health and Human Services, the U.S. Department of the Interior,
and the National Oceanic and Atmospheric Administration. We also discussed
various control activities with ORD managers.
We also issued an electronic survey to 1,371 ORD staff to better understand how
many instances of research misconduct staff reported, as well as to gauge staff
awareness of key scientific integrity criteria and reporting requirements. We
received 488 responses, for a response rate of over 35 percent.5 We analyzed
survey results to address our objectives. Appendix A contains our blank survey
instrument, and appendix B provides summary survey results by question.
Prior Audit Coverage
We reviewed two EPA OIG reports issued in 2009, as well as a U.S. Department
of the Interior OIG report issued in 2010:
• In No Violations Found Regarding Removal of Comments from an
External Peer Review, Report No. 09-P-0084, issued January 16, 2009, the
EPA OIG Office of Counsel found that EPA did not violate existing
federal law or guidance when it removed comments from a peer-reviewed
report and published an explanatory message in the report and online.
• In EPA Can Improve Its Process for Establishing Peer Review Panels,
Report No. 09-P-0147, issued April 29, 2009, the EPA OIG determined
that, although laws, regulations, guidance, and other relevant requirements
governing EPA's peer review process are adequate to produce objective
scientific review, EPA could improve in some areas.
• In Interior Lacks a Scientific Integrity Policy, Report No. WR-EV-MOA-
0014-2009, issued April 28, 2010, the U.S. Department of the Interior OIG
found that Interior had no comprehensive scientific integrity policy to
consistently address scientific misconduct allegations. The report found
that all Interior bureaus but one lacked wide-ranging scientific integrity
policies or procedures to track allegations of scientific misconduct.
5 Given the anonymous nature of our survey and the length of time the survey was available, respondents could have
potentially submitted more than one survey. We did not address this possibility in our analysis.
11-P-0386 6
-------
Chapter 2
Opportunities Exist for ORD to Better Address
Scientific Integrity and Research Misconduct
Although ORD has internal controls in place, it should improve the way in which
it evaluates the effectiveness of its policies and procedures for scientific integrity
and research misconduct. Currently, ORD does not test its policies and procedures
because ORD asserts that few reported instances of misconduct means that it
generally does not occur. However, few identified instances of research
misconduct could signal that staff lacks awareness of key criteria and reporting
requirements necessary to identify and report misconduct. Our electronic survey
of 1,371 ORD science staff found—that out of 488 responses—65 percent of
respondents were unaware of EPA Order 3120.5, and 32 percent were unaware of
EPA's Principles of Scientific Integrity. In addition, ORD has not updated the
Principles of Scientific Integrity E-Training since June 2005. The existing
e-training is not mandatory for ORD staff and does not include actual examples to
aid understanding by training participants. Those who have not completed the
training may be unaware of key criteria regarding scientific integrity. To facilitate
reporting of research misconduct, ORD should increase awareness of the process
as well as ensure the confidentiality of those ORD staff who report alleged
instances to managers or supervisors. Without these additional internal controls,
ORD risks having its science called into question, potentially lessening the
credibility of its work.
ORD Has Controls in Place to Address Scientific Integrity but Does
Not Test Them
ORD has controls in place to address internal control standards, such as:
Control Environment
• Culture and commitment to integrity
• EPA Order 3120.5
• Principles of Scientific Integrity
• Peer Review Handbook
Monitoring
• Rigorous internal peer review
• External peer review
Information and Communication
• Principles of Scientific Integrity E-Training
11-P-0386
-------
ORD believes the controls put in place have resulted in few instances of
misconduct. Indeed, in fiscal year 2010, the OIG hotline received 5 allegations of
lab, scientific, or research fraud out of 514 formal hotline referrals. Moreover, our
electronic survey of 1,371 ORD science staff found—that of 488 responses—
14 percent of respondents had knowledge of an instance of research misconduct in
the past 5 years, with 11 percent reporting one to two instances (figure 2).
Figure 2: Number of instances of research misconduct reported by respondents
Number of Instances
None
11-2
3-5
I Over 6
Source: EPA OIG survey results.
However, ORD cannot assert with certainty the effectiveness of controls because
ORD does not test its controls. ORD should periodically test controls to ensure
staff awareness of how to identify and report instances of research misconduct.
Testing controls will help ensure ORD's research is of the highest quality.
ORD Staff Unaware of Key Criteria and Reporting Requirements
Overall, our electronic survey results demonstrated that many respondents were
unaware of key criteria intended to help them identify and report instances of
research misconduct, and were also unaware of reporting requirements. Figure 3
illustrates that 32 percent of survey participants were unaware of EPA's
Principles of Scientific Integrity, 65 percent were unaware of EPA Order 3120.5,
and 66 percent did not complete the e-training. Appendix B provides a summary
of survey results by question.
11-P-0386
-------
Figure 3: Levels of awareness of key criteria and training completion
Aware of Principles
Aware of EPA Order 3120.5
I No
I Yes
Completed E-Training
I No
I Yes
Source: EPA OIG survey results.
In the final section of our survey, we asked for additional feedback and allowed
participants to share comments or concerns. The following recurring narrative
responses may explain the low number of reported instances:
• Uncertainty regarding steps to take to report an instance
• Fear of retaliation or that managers and supervisors will not maintain
confidentiality
• Need for better or additional formal training or outreach on scientific
integrity and research misconduct
This feedback demonstrates the importance of testing the effectiveness of controls
ORD has in place to help staff identify and report allegations related to scientific
integrity and research misconduct. ORD's Deputy Assistant Administrator for
Science has committed to addressing our survey results by raising staff awareness
of criteria and reporting requirements.
11-P-0386
-------
Principles of Scientific Integrity E-Training Should Be Updated and
Mandatory
In her May 2009 memorandum on scientific integrity, the EPA Administrator
tasked the STPC to work with the NPC to update the Principles of Scientific
Integrity E-Training. Although the NPC in turn directed EPA's unions and ORD
to update the e-training, it has not been updated since its creation in June 2005.
Narrative responses to our survey question inviting ORD staff to provide
comments related to the e-training included staff:
• Not remembering whether they took the course
• Being unaware of the existence of the e-training or having no recollection
of being asked to take the training
• Not remembering the e-training course content
In 2010, ORD formed the Principles of Scientific Integrity Workgroup, consisting
of EPA union and management representatives, to update the e-training. Since its
inception, the workgroup has met twice via teleconference to discuss updates to
the e-training. As part of this process, ORD will work the with EPA Office of
General Counsel Ethics Program staff and the Office of Administration and
Resources Management Labor and Employee Relations staff to update contact
lists that staff would use to report instances of scientific integrity or research
misconduct.
An ORD senior advisor indicated that the primary purpose of the e-training course
is to provide scientists and engineers with contact information should they
experience or identify instances of scientific integrity or research misconduct.
However, the e-training summary /certificate screen links to outdated EPA union
and OIG contact information. However, that screen also contains a link that takes
the trainee directly to the Office of General Counsel Ethics Program updated
webpage that lists current contact information for EPA Ethics Officials. As the
workgroup updates the e-training, it should consider linking directly to updated
web pages with current union representative and OIG contact information, as it
does for contact information for EPA's ethics officials.
The currently available 2005 version of the e-training includes a discussion of the
Principles of Scientific Integrity, and contains a section on "Misconduct:
Violations of the Principles of Scientific Integrity." However, since the current
e-training is not mandatory for ORD staff, those who do not complete the
e-training may be unaware of available internal control guidance for scientific
integrity issues, such as the Principles of Scientific Integrity or EPA Order
3120.5. Making the e-training mandatory for appropriate ORD staff would clearly
connect staff to the e-training as well as increase staff awareness of scientific
integrity guidance.
11-P-0386 10
-------
The workgroup should also add actual examples of misconduct into the e-training
to aid participants' understanding and to make the training more meaningful. The
National Science Foundation OIG takes this approach by combining civil/criminal
criteria with actual stories, and it posts completed case summaries on-line as
educational materials. The workgroup should also consider more formal training
in addition to the e-training format;6 however, one workgroup member
emphasized that online training can potentially reach more attendees.
ORD management estimates that the workgroup will complete its updates to the
e-training by July 2012. At that point, ORD will submit the updated e-training for
concurrent review and approval by the NPC's executive committee and the STPC.
ORD hopes to offer the updated e-training to ORD staff by September 2012. The
workgroup should consider the narrative responses to our electronic survey when
updating the e-training.
Conclusion
Periodically testing its controls would help assure that ORD utilizes the right
control activities while striving to achieve scientific integrity. Further, raising
awareness of key criteria and updating the e-training will help strengthen ORD's
internal control environment to address instances of research misconduct. These
efforts could improve the credibility of ORD's scientific research.
Recommendations
We recommend that the Assistant Administrator for Research and Development:
1. Periodically test the effectiveness of controls to address scientific
integrity and research misconduct.
2. Work with Agency offices to:
a. Initiate proactive outreach on EPA Order 3120.5 to raise
awareness of roles/responsibilities and reporting steps.
b. Identify staff and managers outside of ORD who should
complete mandatory Principles of Scientific Integrity
E-Training.
3. Continue working with the unions to update and implement the
Principles of Scientific Integrity E-Training. Changes to the course
should include:
a. Making the e-training mandatory for all ORD staff.
6The National Science Foundation OIG considers online training to be an inappropriate delivery format for such an
important subject as research misconduct and believes that formal, in-class training represents the best method for
communicating how to address instances of scientific integrity and research misconduct.
11-P-0386 11
-------
b. Ensuring that the updated course contains real-life
examples.
c. Creating a system for linking to current contact information
for reporting instances of scientific integrity and research
misconduct.
Agency Comments and OIG Evaluation
ORD agreed with our recommendations and noted many steps it has already taken
to enhance scientific integrity at the Agency. Examples of how EPA addresses
scientific integrity include mandatory ethics training, quality systems
requirements, a peer review policy and Peer Review Handbook., and establishing
cross-Agency councils such as EPA's STPC.
In response to our review, the Agency will make the Principles of Scientific
Integrity E-Training mandatory for scientific and technical staff. The updated
course will contain real-life examples and will have links to current contact
information for reporting instances of scientific and research misconduct. ORD
expects to have the course content completed by July 2012 and training completed
by September 2012.
EPA is also in the process of developing a new Agency-wide policy on scientific
integrity. EPA has named an acting scientific integrity official and will establish
deputy scientific integrity officials in each of the regions and offices. The Agency
noted that these individuals will together become EPA's Scientific Integrity
Committee, responsible for overseeing scientific integrity, and will serve as one
option (the OIG being the other) to address allegations of scientific integrity
violations.
ORD agreed with our conclusion regarding periodically testing its controls and
noted that the Management Integrity Program has recently implemented a new
risk assessment protocol. The risk assessment protocol is centered on program
operations and facilitates the periodic testing of controls. These protocols may
need to be augmented as the Agency puts the Scientific Integrity Committee and
the Agency-wide policy into place.
We believe ORD's planned corrective actions address the intent of our
recommendations. Appendix C includes ORD's full response.
11-P-0386 12
-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
Rec. Page
No. No.
Subject
Status1
Action Official
Planned
Completion
Date
POTENTIAL MONETARY
BENEFITS (in SOOOs)
Claimed Agreed-To
Amount Amount
Periodically test the effectiveness of controls to
address scientific integrity and research
misconduct.
Work with Agency offices to:
a. Initiate proactive outreach on EPA
Order 3120.5 to raise awareness of roles/
responsibilities and reporting steps.
b. Identify staff and managers outside of
ORD who should complete mandatory
Principles of Scientific Integrity E-Training.
Continue working with the unions to update and
implement the Principles of Scientific Integrity
E-Training. Changes to the course should include:
a. Making the e-training mandatory for all
ORD staff.
b. Ensuring that the updated course contains
real-life examples.
c. Creating a system for linking to current
contact information for reporting instances
of scientific integrity and research
misconduct.
Assistant Administrator for 12/31/2012
Research and Development
Assistant Administrator for 12/31/2011
Research and Development
Assistant Administrator for 09/30/2012
Research and Development
0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
11-P-0386
13
-------
Appendix A
Survey Instrument
SCIENTIFIC INTEGRITY AND RESEARCH MISCONDUCT SURVEY
Our intent is to determine the effectiveness of Agency policies and procedures in place to help EPA staff identify and report instances/allegations related to scientific
integrity and research misconduct.
If necessary, did on the toistie (>) next to each section heading to access the questions, Please answer each question in the space provided. Your responses may trigger secondary
questions. All survey questions in Sections I and II are mandatory, but associated text input responses are optional. This survey must be completed in its entirety once the survey is
started and should take approximately 15 minutes to complete, When you have completed the survey and pressed the "Submit Survey" button (as part of Section IV), the completed
survey will be sent to the Office of Inspector General and the survey database will dose,
Section I; Respondent Information
The following information is for OIG purposes only and will remain confidential.
1. What is your current grade level?
2. How long have you worked for EPA's Office of Research and Development (ORD)?
f* Less than 1 year
r 1-2 years
f 3-5 years
<" 6-9 years
r 10-20 years
r 21+years
3. Which of the fol lowing best describes your current job function?
f Manager
(~ Manager with research responsibilities
f~ Supervisor
f" Supervisor with research responsibilities
c Research Scientist
C Research Assistant
ff Other
(Please specify [max 1,000 characters])
4. In which job series is your current position listed?
C 0028 - Environmental Protection Specialist
f 0400 - Natural Resources Management and Biological Sciences
f 0800 - Engineering and Architecture
r 1300 - Physical Sciences
f 1500 - Mathematics and Statistics
«• Other
i. (Please specify [max 1,000 characters])
11-P-0386
14
-------
5. Current Lab, Center, or Office?
(~ Immediate Office of the Assistant Administrator (IOAA)
C National Center for Computational Toxicology (NCCT)
<~ National Center for Environmental Assessment (NCEA)
<" National Center for Environmental Research (NCER)
f National Exposure Research Laboratory (NERL)
<" National Health and Environmental Effects Research Laboratory (NHEERL)
r National Homeland Security Research Center (NHSRC)
(~ National Risk Management Research Laboratory (NRMRL)
<" Office of Administrative and Research Support (OARS)
C Office of Resources Management a Administration (ORMA)
r Office of Science Information Management (OSIM)
r Office of Science Policy (OSP)
r Office of the Science Advisor (OSA)
-------
3, Did you complete the E-Training course on the Principles of Scientific Integrity?
(f Yes
("No
On a scale from one to five, where one is not at all useful and five is very useful, please indicate how useful you found the E-Training,
1 = Not Useful 5 = Very Useful
ri o if 3 n ("5
Please provide any suggestions related to the training [max 1,000 characters].
4, Do you know how to report instances/allegations relating to scientific integrity and research misconduct?
(f Yes
("No
To whom would you feel comfortable reporting your information;
I, Supervisor
f Yes
if No
Why not [max 1,000 characters]?
Union
T Yes
if No
Why not [max 1,000 characters]?
iii, DIG
("Yes
if No
Why not [max 1,000 characters]?
iv, Other (please specify [max 1,000 characters])
11-P-0386 16
-------
5, If you disagree with a decision ORD management makes which you believe does not align with the results of your scientific research, do you have a process in which to
register your disagreement?
ff Yes
c No
Please describe the process [max 1,000 characters].
5. If you disagree with a decision EPA management makes which you bel ieve does not align with the results of your scientific research, do you have a process in which to
register your disagreement?
Yes
NO
Please explain (e.g. the need for a process, or how the process could be improved) [max 1,000 characters].
7, In the past 5 years, how many instances/allegations related to scientific integrity and/or research misconduct do you have personal/direct knowledge or involvement?
r None
If 1-2
r 3-5
r over 6
Please indicate whether these instances were reported and to whom [max 1,000 characters]:
8. Do you think it would be helpful for the Agency to provide additional outreach/education on scientific integrity and research misconduct (i.e. on identifying and reporting
instances)?
<• Yes
r NO
Section III: Additional Comments
Please provide any additional comments, questions, or input regarding EPA's process for addressing scientific integrity and research misconduct, You may also use this
space to communicate any concerns to the DIG team and, if you feel comfortable, include your name and contact information so that the OIG team can contact you to
follow-up, if necessary. Your name and contact information will remain confidential. [4000 character limit]
Section IV: Survey Completion
Once you have completed the survey, please press the "Submit Survey" button to send it to the Office of Inspector General and close the survey database, Use this button only
when you are ready to submit the completed survey.
Thank you for your participation!
Submit Survey
11-P-0386 17
-------
Appendix B
Survey Analysis Results by Question
17
120
155
179
11
6
488
3.48%
24.59%
31 .76%
36.68%
2.25%
1 .23%
100.00%
Section I: Respondent Information:
1. What is your current grade level?
GS-7/9
GS-11/12
GS-13
GS-14/15
SES
Title 42 Appointee
2. How long have you worked for EPA's Office of Research and Development
(ORD)?
Less than 1 year 25 5.12%
1-2 years 41 8.40%
3-5 years 52 10.66%
6-9 years 54 11.07%
10-20 years 185 37.91%
21 + years 131 26.84%
488 100.00%
3. Which of the following best describes your current job function?
Manager 15 3.07%
Manager with research
responsibilities 39 7.99%
Other 52 10.66%
Research Assistant 41 8.40%
Research Scientist 308 63.11%
Supervisor 7 1.43%
Supervisor with research
responsibilities 26 5.33%
488 100.00%
4. In which job series is your current position listed?
0028 (Environmental Protection Specialist) 12 2.46%
0400 (Natural Resources Management &
Biological Sciences) 169 34.63%
0800 (Engineering and Architecture) 59 12.09%
1300 (Physical Sciences) 181 37.09%
1500 (Mathematics and Statistics) 20 4.10%
Other 47 9.63%
488 100.00%
11-P-0386 18
-------
5. Current Lab, Center, or Office?
IOAA
NCCT
NCEA
NCER
NERL
NHEERL
NHSRC
NRMRL
ORMA
OSA
OSP
Other
6
8
41
8
108
184
14
97
2
6
11
3
1 .23%
1 .64%
8.40%
1 .64%
22.13%
37.70%
2.87%
19.88%
0.41%
1 .23%
2.25%
0.61%
488 100.00%
Section II: Survey Questions
1. Are you aware of EPA's Principles of Scientific Integrity?
No 158 32.38%
Yes 330 67.62%
488 100.00%
How useful are the Principles?
1 (Not Useful) 15 4.55%
2 20 6.06%
3 142 43.03%
4 109 33.03%
5 (Very Useful) 44 13.33%
330* 100.00%
*Please note that the usefulness percentages reported above are for a total number of 330
responses, corresponding to the number of respondents who are aware of EPA's Principles.
2. Are you aware of EPA Order 3120.5 - EPA's Policy and Procedures for Addressing
Research Misconduct?
No 319 65.37%
Yes 169 34.63%
488 100.00%
How useful is EPA Order 3120.5?
1 (Not Useful)
2
3
4
5 (Very Useful)
*Please note that the usefulness percentages reported above are for a total number of 169
responses, corresponding to the number of respondents who are aware of EPA Order 3120.5.
11-P-0386 19
6
14
58
66
25
169*
3.55%
8.28%
34.32%
39.05%
14.79%
100.00%
-------
3. Did you complete the E-Training course on the Principles of Scientific
Integrity?
No 321 65.78%
Yes 167 34.22%
488 100.00%
On a scale from one to five, where one is not at all useful and five is very useful,
please indicate how useful you found the E-Training.
1 (Not Useful) 9 5.39%
2 9 5.39%
3 75 44.91%
4 55 32.93%
5 (Very Useful) 19 11.38%
167* 100.00%
*Please note that the usefulness percentages reported above are for a total number of 167
responses, corresponding to the number of respondents who completed the E-Training course.
4. Do you know how to report instances/allegations relating to scientific
integrity and research misconduct?
No 252 51.64%
Yes 236 48.36%
488 100.00%
To whom would you feel comfortable reporting your information:
i. Supervisor
No 60 12.30%
Yes 428 87.70%
488 100.00%
ii. Union
No 241 49.39%
Yes 247 50.61%
488 100.00%
iii. OIG
No 171 35.04%
Yes 3T7 64.96%
488 100.00%
5. If you disagree with a decision ORD management makes which you believe
does not align with the results of your scientific research, do you have a
process in which to register your disagreement?
No 291 59.63%
Yes 197 40.37%
488 100.00%
11-P-0386 20
-------
6. If you disagree with a decision EPA management makes which you believe
does not align with the results of your scientific research, do you have a
process in which to register your disagreement?
No 307 62.91%
Yes 181 37.09%
488 100.00%
7. In the past 5 years, how many instances/allegations related to scientific
integrity and/or research misconduct do you have personal/direct knowledge
or involvement?
None 419 85.86%
1-2 54 11.07%
3-5 9 1.84%
Over 6 6 1.23%
488 100.00%
8. Do you think it would be helpful for the Agency to provide additional
outreach/education on scientific integrity and research misconduct (i.e. on
identifying and reporting instances)?
No 156 31.97%
Yes 332 68.03%
488 100.00%
Section III: Additional Comments
Please provide any additional comments, questions, or input regarding EPA's
process for addressing scientific integrity and research misconduct.
153 of the 488 respondents (31%) provided additional comments.
11-P-0386 21
-------
Appendix C
Agency Response to Draft Report
D.C.
July 7, 20 11
MEMORANDUM
SUBJECT: Office of Research and Development (ORD) Response Office of Research and
Development Should Increase Awareness of Scientific Integrity Policies, Project
No.OA-FYIO-0193
FROM: Kevin Y. Teichman
Deputy Assistance Administrator of Science
Office of Research Development
TO: Melissa M. Heist
Assistant Inspector General
Office of Audit
Thank you for the opportunity to comment on the Office of Inspector General (OIG) draft
audit report, Office of Research and Development Should Increase Awareness of Scientific
Integrity Policies (Project No. OA-FYIO-0193), dated June 8, 2011. We appreciate the OIGs
recognition that EPA has a commitment to scientific integrity and that "ORD has [scientific
integrity] controls in place that include policies, procedures, training, and peer review. " While
we also appreciate that the recommendations in your draft report are subject to change, we agree
with the OIGs recommendations as currently proposed. In fact, EPA is already taking these and
additional steps to enhance scientific integrity at the Agency.
The Agency has taken many steps to address scientific integrity including mandatory
ethics training, quality systems requirements, our peer review policy, our Peer Review
Handbook, and establishing cross- Agency Councils such as EPA's Science and Technology
Policy Council. Attached is a copy of the memorandum we sent to Dr. John P. Holdren, Director
of the Office of Science Technology Policy, where we provide greater detail regarding the many
actions the Agency has undertaken to address scientific integrity.
Recently, the Agency has established a workgroup to revise and update the Principles of
Scientific Integrity e-training course. Under the aegis of the EPA national Partnership Council,
the workgroup consists of representatives from every major EPA union and several management
officials. In response to advice from the OIG, the Agency will make the training mandatory for
scientific and technical staff. Further, the updated course will contain real-life examples and will
11-P-0386 22
-------
have links to current contact information for reporting instances of scientific and research
misconduct. We expect to have the course completed by September 30, 2011.7
Relevant to the subject OIG draft report, EPA is in the process of developing a new
Agency-wide policy on scientific integrity. EPA has also named an Acting Scientific Integrity
Official, and Deputy Scientific Integrity Officials will be established in each of the Regions and
Offices. Together, these individuals will become EPA's Scientific Integrity Committee and will
be responsible for overseeing scientific integrity and serving as one option (the OIG being the
second) to address allegations of scientific integrity violations. EPA's Scientific Integrity
Committee will also be charged with standardizing the Agency's scientific integrity training and
ensuring that appropriate EPA staff members complete the appropriate training courses.
We agree with your conclusion that "periodically testing controls that ORD has in place
would help assure that ORD utilizes the right control activities while striving to achieve
scientific integrity." We would like to point out that ORD's Management Integrity
Program has recently implemented a new risk assessment protocol. The risk assessment protocol
is centered on program operations and facilitates the periodic testing of controls. These protocols
may need to be augmented as the Scientific Integrity Committee and the Agency-wide policy are
put into place.
Finally, in your transmittal memorandum, you state:
"This draft report is subject to revision by the OIG and, therefore, does not represent the
final position of the OIG on the subjects reported."
We understand that the findings and conclusions are the premise to developing your
recommendations. As required by EPA Order 2750, our written response to the final report will
address any recommendations that may be included at that time. We will consider any
recommendations on their own merit and, if applicable, provide a corrective action plan and/or
offer alternative solutions to the report's recommendations. If you have any questions, please
contact Norman Adkins at (919) 541-0872.
Attachment
cc: Arthur Elkins
Paul Anastas
Erin Barnes-Weaver
Kevin Teichman
Bill Sanders
Amy Battaglia
Mary Greene
Norman Adkins
7 On July 21, 2011, ORD provided their corrective action plan and adjusted the training completion date to
September 2012.
11-P-0386 23
-------
ft a
I *&> 5 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
| , .. j WASHINGTON, D.C. 20460
April 18,2011
OFFICE OF
ATTACHMENT RESEARCH AND DEVELOPMENT
Dr. John P. Holdren
Assistant to the President for Science and Technology
and Director of the Office of Science and Technology Policy
Executive Office of the President
Washington, DC 20502
Dear Dr. Holdren:
In your December 17,2010 memorandum1 on the subject of "scientific integrity," you
charged the Federal agencies with the development of policies in four areas: foundations of
scientific integrity in government, public communications, use of Federal advisory committees,
and professional development of government scientists and engineers. You also asked agencies
to report on actions taken to "develop and implement policies" in these four areas within 120
days. This letter and its attachment are EPA's response to your request.
Science is the backbone of EPA's decision-making. Scientific integrity is central to our
identity and the credibility of our work. Our ability to pursue the Agency's mission to protect
human health and the environment depends upon it. The policies, decisions, guidance and
regulations that impact the lives of all Americans every day are grounded, at a most fundamental
level, in sound science. It is therefore the responsibility of this Agency to always conduct,
utilize, and communicate science with the highest degree of honesty, integrity, and transparency
to the American public.
For forty years, EPA has implemented policies that establish and uphold these principles
of scientific integrity. EPA is strengthening this commitment by developing an Agency -wide
scientific integrity policy to be finalized by October 15,201 1. This policy will make our
historic, exemplary programs for quality, peer review, and independence even stronger and more
cohesive to ensure that our scientific work is conducted in an environment that is free of political
interference or suppression. The development of this policy on scientific integrity will be
conducted under the same principles of transparency and openness that it aims to implement.
Additionally, we will establish an Agency senior scientific integrity official to champion
scientific integrity across EPA. This official will chair a standing committee of scientific
integrity officials representing each EPA office. These senior-level employees will provide
oversight for the implementation of scientific integrity at EPA.2 We plan to designate the senior
1 http://www .whitehouse.gov/sites/default/files/microsites/ostp/scientifi c-integrity-memo-121720 10.pdf
2 EPA's Office of the Inspector General will maintain its independent authority to conduct research misconduct
investigations as authorized by EPA Order 3120.5 (Policy and Procedures for Addressing Research Misconduct).
Page 1
11-P-0386 24
-------
EPA scientific integrity official and assemble the standing committee by May 15,2011, and
develop operating procedures for the committee by June 15,2011.
As part of our comprehensive scientific integrity policy, EPA will take the following
specific actions to address the four policy areas that you outlined on December 17, 2010:
Foundations of Scientific Integrity in Government: EPA has an existing
framework of principles that was formed in collaboration with union partners of the
National Partnership Council. We are updating EPA's Principles a/Scientific
Integrity and will incorporate them in our Agency-wide policy.
Public Communications: EPA is currently developing formal Agency-wide policy
for the communication of science and technology information by our employees.
This policy will be part of our Agency-wide scientific integrity policy and will ensure
accurate, transparent, and accessible communications of our science to the public.
Further, while it is already current Agency practice to encourage scientists to engage
with the public and media, this new policy will ensure that there are no barriers for
scientists when communicating with the press and the public and reinforce our
expectation that they adhere to their fields of expertise when doing so.
Use of Federal Advisory Committees: EPA has a strong tradition of accessing
external scientific expertise through the appropriate use of federal advisory
committees (FACs). Following your lead, we have directed our FACs to follow
procedures that are consistent with the expectations outlined in your December 17,
2010 memo. In addition, we will finalize and disseminate a new Agency-wide
handbook for use by these committees and, as always, we will continue to comply
with all relevant statutory and regulatory requirements.
Professional Development of Government Scientists and Engineers: We will
continue to encourage our scientists and engineers to engage and interact with the
wider scientific community and will incorporate critical information about
professional development incentives and opportunities in our updated scientific
integrity training module.
The attached document details the Agency's current collection of policies, actions, and
programs that uphold the four areas of scientific integrity. While we emphasize those that are
EPA-specific, we also include some government-wide requirements that are key components of
scientific integrity at EPA.
Thank you for this important opportunity to document actions that EPA has taken, is
taking, and will take to implement and promote scientific integrity. If you have additional
questions, please do not hesitate to contact me at 202-564-6620.
Sincerely yours,
Paul T. Anastas, Ph.D.
EPA Science Advisor
Page 2
11-P-0386 25
-------
Attachment:
Scientific Integrity at the U.S. Environmental Protection Agency
Foundations of Scientific Integrity in EPA
Ensure a Culture of Scientific Integrity
At EPA, ensuring a culture of scientific1 integrity is central to our identity and the
credibility of our work. To operate an effective research and regulatory Agency like EPA, it is
essential that political officials not suppress or alter scientific or technological findings. Soon
after entering office as EPA Administrator, Lisa Jackson issued the "Transparency in EPA's
Operations" (23 April 2009)2 memorandum to reaffirm EPA's commitment to operate "in a
fishbowl" and provided guidelines for ensuring transparency in EPA's interactions with all
members of the public. This was a reassertion of values that Administrator William Ruckelshaus
had first articulated in his "Fishbowl Memo" (19 May 1983).3 In that memo, Administrator
Ruckelshaus sought to establish a culture of integrity and openness for all employees by
promising that under his leadership, EPA would operate "in a fishbowl" and "will attempt to
communicate with everyone from the environmentalists to those we regulate, and we will do so
as openly as possible."
To reinforce and advance the principles in the President's scientific integrity memo,
Administrator Lisa Jackson issued her statement on scientific integrity, the "Compass Memo" (8
May 2009),4 to all EPA staff. In this communication, the Administrator stated that "Science must
be the compass guiding our environmental protection decisions." Shortly afterward, in testimony
before the Senate Environment and Public Works (SEPW) Committee (9 June 2009),5 she
"pledged to uphold values of scientific integrity every day."
Furthermore, in her "Compass Memo" and SEPW testimony, Administrator Jackson
described how EPA has fostered this culture of scientific integrity through its Principles of
Scientific Integrity and voluntary online training for scientists and engineers. These Principles
were developed with EPA's National Partnership Council (NPC), a partnership of Agency labor
unions and management. Administrator Carol Browner issued the Principles on November 29,
1999, and Administrator Christine Todd Whitman reaffirmed them on July 10, 2002. The
Principles of Scientific Integrity sets forth the Agency's commitment to conducting science
objectively, presenting results fairly and accurately, and avoiding conflicts of interest. Currently,
the Agency is working with the NPC to review, and revise if needed, the Principles of Scientific
Integrity and to update our online training. The improved training will include additional
information for scientists and engineers about opportunities for professional development and
how EPA encourages their professional growth. When the revised online training is complete,
1 In this document, "science" and "scientific" are expansive terms that refer to the full spectrum of the scientific
process, including basic science, applied science, engineering, and technology.
http://www.epa.gov/Administrator/operationsmemo.html
3 http://www.epa.gov/hlstory/topics/policy/fishbowl.htmSmemo
4 http://www.epa.gov/Administrator/scientificmemo.htm I
5 http://www.epa.gov/ocir/hearings/testimony/H1_2009_2010/2009_060B-'pj.pdf
11-P-0386 26
-------
EPA staff will be encouraged to take it, whether as a refresher or for the first time, and it will be
provided to new employees. In addition, as part of EPA's commitment to transparency, the
training will be shared on the internet to help the public understand how EPA operates.
EPA employees are provided with mandatory annual ethics training pursuant to 5 CFR
2638 Subpart G (Executive Agency Training Programs). In addition, EPA has provided its online
ethics training to more than a dozen different Federal Agencies for their use as well. This
training refreshes staff on general ethics topics and focuses on a specific theme or topic of
interest. Over the past several years, these focus areas have included:
> 2010: A Window to the World: Ethics and Social Media
> 2009: Quest for the Ultimate Ethics Treasure
> 2008: Adventures in the House of Ethics: Misuse of Position
> 2007: Outside Activities: Your Life Outside of EPA
> 2006: Essential Ethics for EPA Employees
> 2005: Interacting With Contractors
> 2004: The Hatch Act
> 2003: Post-Government Employment
> 2002: Conflicts of Interest
> 2001: Misuse of Office & Resources
In addition to general ethics training, EPA laboratory scientific staff received annual
scientific ethical conduct training. Accredited EPA laboratories provide annual Laboratory
Ethics and Data Integrity Training for scientists and engineers engaged in generating scientific
data to support cleanups, enforcement, and environmental assessments. This training serves as a
refresher that fulfills accreditation standards and reinforces an understanding of the laboratory
ethics policy.
Strengthen the actual and perceived credibility of Government research
EPA has always been committed to using external independent consultations, including
external peer review, in order to ensure the scientific integrity of its scientific work products.
EPA renews this commitment in its formal scientific integrity policy and emphasizes that
political interference with the scientific process or Agency scientists is not acceptable whether
that interference comes from within the Agency or other parts of the Federal government.
Quality environmental information is integral to EPA's mission to protect human health
and the environment. EPA has robust systems for quality and peer review to ensure that data and
research used to support policy decisions are credible and high quality. EPA's Quality Policy was
updated in 2008 (CIO Policy 2106.0).6 The Agency's Chief Information Officer (CIO) is the
Senior Management Official responsible for EPA's Quality Program. EPA's Quality and
Information Council (QIC), a group of cross-Agency senior managers, reviews and evaluates
how well EPA is meeting its quality goals and objectives.
6 http://www.epa.gov /irmpoli8/policies/21060.pdf
11-P-0386 27
-------
EPA's Quality Program includes Agency -wide policies procedures, guidance and
standards that enhance scientific integrity of EPA's environmental data and research results. The
Policy and Program Requirements for Mandatory Agency-wide Quality Systems (CIO Policy
2105.0)7 requires EPA environmental programs to implement Quality Management Systems that
comply with ANSI/E4 - 1994.8 The primary goal of this policy is to ensure that environmental
data are of sufficient quantity and quality to support their intended use. Under this Quality
System, EPA regional and program offices develop and implement supporting quality systems.
EPA's quality requirements may also apply to contractors, grantees, and other recipients of EPA's
financial assistance. EPA's Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by the Environmental Protection
Agency (IQG, 2002)9 focus on the dissemination of products and services that are credible,
reliable and reproducible with the goal of promoting openness and transparency in
communicating with the public.
Following applicable EPA quality assurance and peer review policies and procedures
helps to ensure that the Agency produces scientific products that are consistent with "scientific
integrity criteria."10 For example, EPA's regional offices frequently make multi-million dollar
permit and cleanup decisions that may face controversy and litigation. Adherence to EPA's
quality assurance policies builds confidence that the information supporting these high-impact
decisions are technically and legally defensible.
To assure that scientific products undergo appropriate peer review by qualified experts,
EPA issued a Peer Review Policy (2006)11 and Peer Review Handbook (2006).12 The Peer
Review Handbook is a how-to manual that is used by staff across EPA and is often referred to by
external stakeholders as a model of good peer review practices. It should be noted that peer
review is not new in the Agency, as the Agency -wide peer review policies have been in place in
the Agency since 1993.13 The Peer Review Policy establishes EPA's policy for peer review of
scientifically and technically based work products, including economic and social science
products, that are intended to inform Agency decisions. It includes specific expectations for
varying levels of scientific products including influential scientific information (1SI) and highly
influential scientific assessments (HISA). In compliance with OMB's 2004 Final Information
Quality Bulletin for Peer Review, EPA posts a Peer Review Agenda14 for its HISAs and ISIs.
In an approach that is similar to peer review, EPA's National Remedy Review Board
reviews proposed Superfund response decisions at both National Priority List (NPL) and non-
NPL sites for proposed actions that exceed $25 million. In addition, some EPA regional offices
7 http://www .epa.gov/irmpoli8/policies/21 050.pdf
8
Later updated as ANSI/ASQ E4-2004, Quality Systems for Environmental Data and Technology Programs Requirements
with Guidance for Use, Milwaukee, Wisconsin 2004.
9 http://www.epa. gov/quality/informationguidelines/documents/EPAJnfoQualityGuidelines.pdf
10 "Quality Science in the Courtroom: U.S. EPA Data Quality and Peer Review Policies and Procedures Compared to
the Daubert Factors," G. M. Brills, J. C. Worthington, A. D. Wait, Environmental Forensics, Vol. 1, No. 4, December
2000, pp. 197-203.
11 http://www.epa. gov/peerreview/pdfs/peer_re View_policyand_memo.pdf
12 http://www.epa.gov/peerreview/pdfs/peer Jeview_handbook_2006.pdf
13 http://www.epa.gov/peerrevlew
14 http://cfpub.epa. gov/si/si_public_pr_agenda.cfm
11-P-0386 28
-------
have established a parallel process for less expensive cleanups. Before completing the Record of
Decision for the cleanup of a Superfund site, the Remedial Project Manager must present and
defend the rationale for the recommended remedy to a regional board to ensure its soundness.
EPA's quality and peer review programs are further supported by its Summary of General
Assessment Factors for Evaluating the Quality of Scientific and Technical Information (2003).
This document describes the assessment factors and considerations generally used by the Agency
to evaluate the quality and relevance of scientific and technical information. These general
assessment factors are founded in the Agency guidelines, practices, and procedures that
constitute EPA's information and quality systems, including existing program-specific quality
assurance policies.
The Agency has in place clearly articulated policies against scientific misconduct,
including inappropriate alterations of work products, in two important documents:
> Scientific Misconduct in the Conduct and Discipline Manual (EPA Order 3120.1)
includes discipline guidelines for fabrication, plagiarism, misrepresentation, and
causing a subordinate to engage in scientific misconduct.
> Policy and Procedures for Addressing Research Misconduct (EPA Order 3120.5)
provides policy on reporting, procedures, investigations, and adjudication of research
misconduct by EPA employees, contractors, and recipients of assistance agreements.
Through several cross-Agency councils, EPA has developed guidance documents for the
development and application of science in specific functional areas. These guidance documents
promote transparency and quality for our work products. The Council for Regulatory
Environmental Modeling (CREM), a cross-Agency council of senior managers, was established
to promote consistency and consensus among environmental model developers and users. The
CREM developed the Guidance Document on the Development, Evaluation and Application of
Environmental Models (2009).15 This guidance provides a simplified, comprehensive resource
for modelers across the Agency on best modeling practices. These guidelines are intended to
ensure the quality, utility, and regulatory relevance of the models that EPA develops and applies
and the transparency of modeling analyses and model-based decisions. The Forum on
Environmental Measurements (FEM), a standing committee of senior EPA managers, issued its
Policy for Assuring the Competency of Environmental Protection Agency Laboratories in
2004.16 This FEM policy requires all Agency laboratories to maintain a Quality System and
undergo independent assessments. EPA's Program in Human Research Ethics (PHRE)17 includes
the Agency's Human Subjects Research Review Official, who provides review for all Agency
human subjects' activities. In addition, the PHRE includes the Human Studies Review Board, a
federal advisory committee (F AC) that provides advice to EPA on third party human studies and
other aspects of human studies research. The basic policy for EPA's protection of human subjects
in research is found in 40 CFR Part 26.
15 http://www.epa.gov/crem/library/cred~uidance_0309.pdf
16 http://www.epa.gov /fem/pdfs/labdirective.pdf
17 http://www.epa.gov/phre/
11-P-0386 29
-------
The Administrator's June 2009 SEPW testimony also included details of new EPA
processes that advance transparency and scientific integrity:
> EPA's new process for developing Integrated Risk Information System (IRIS)
assessments is more transparent and timely, while ensuring the highest level of
scientific integrity. For example, all written scientific comments received through
interagency science consultation and discussion become part of the public record. To
guarantee the scientific quality of IRIS assessments, the process includes the
opportunity for public comment and relies on a rigorous, open, and independent
external peer review. Changes in EPA's scientific judgments from public and peer
reviewers' comments are documented and explained clearly to maximize the
transparency of the final product.
> EPA's new process for National Ambient Air Quality Standards (NAAQS) considers
the latest peer-reviewed science and the expert advice of the Clean Air Scientific
Advisory Committee (CASAC)18 on the science and the standards. This 5-year
process ensures timeliness, scientific integrity, and transparency. It preserves steps
that contribute to these attributes, such as the kickoff workshop, the integrated review
plan, and more concise, policy-relevant assessments of science, as well as risk and
exposure. At the same time, because of concerns expressed by the CASAC and
others, the use of an advance notice of proposed rulemaking has been discontinued. In
its place, EPA staff prepared a policy assessment document. By reinstating the policy
assessment document in the revised NAAQS process, EPA ensures that both the
public and CASAC will be able to see, and comment on, a transparent staff analysis
of the scientific basis for alternative policy options for consideration by the
Administrator.
EPA has implemented appropriate whistleblower protections. Agency employees are
entitled to the protections afforded to them under the Whistleblower Protection Act of 1989 and,
in certain circumstances, to the employee protection provisions of certain environmental statutes.
In addition, the Agency's Office of the Inspector General (OIG) maintains a confidential and
anonymous hotline to which employees can report known or suspected violations of law. In
addition, in compliance with the Notification and Federal Employee Antidiscrimination and
Retaliation Act of 2002, all EPA employees are required to take annual training designed to
familiarize them with their rights under the applicable anti-discrimination and whistleblower
protection laws.
Facilitate the free flow of scientific and technological information
As part of the Administration's Open Government Initiative, EPA's Office of
Environmental Information (OEI) has a robust Open Government Initiative.19 OEI efforts are
intended to expand and promote access to scientific and technological information by making it
available online in open formats, including through the Data.gov portal. In addition, EPA
continues to use conventional means of open government, including reading rooms to share final
18 The CASAC is a FACA committee consisting entirely of SGEs.
19 http://www.epa.gov/open
11-P-0386 30
-------
opinions and orders made in adjudicating cases, final statements of policy and interpretations that
were not published in the Federal Register, and other materials of public interest. DEI also
administers EPA's responses to requests for information under the Freedom of Information Act
(EPA regulations at 40 CFR Part 2).
EP A maintains a number of publicly available databases that store environmental data
including standard terminology to assist in their interpretation and models underlying our
regulatory proposals and policy decisions. The Envirofacts20 data warehouse contains
information from many of the Agency's major programs, including air, drinking water,
wastewater, solid waste, Toxics Release Inventory, Brownfields, Superfund, and compliance
information. In addition, the System of Registries21 and System Inventory Services (SIS)22
provide users with extensive information on terminology within EPA's data systems, inventories
of data systems, and other essential descriptive information on the resources.
Another example of EPA's efforts to improve transparency can be found in the CREM's
Models Knowledge Base23, an inventory of the computational models that are developed, used,
and/or supported by EPA's offices. For each model, the Models Knowledge Base provides
metadata on its development, conceptual basis, scientific detail and evaluation; technical
requirements and how to use it; information on its inputs and outputs; and directions for
downloading it and links to further information. CREM is integrating its Models Knowledge
Base into SIS to streamline management of information on models.
Establish principles for conveying scientific and technological information to the public
EPA has several mechanisms for communicating scientific and technological findings to
the public and the research community. The Science Inventory is a searchable database of EPA
science activities conducted by EPA and through EPA-funded assistance agreements. Records in
the Science Inventory provide descriptions of the activity, associated products, contact
information, peer review actions, and links to available printed material and Web sites. The
Science Inventory also contains EPA's Peer Review Agenda, a compilation of planned and
ongoing EPA science activities and products that meet the Office of Management and Budget's
peer review requirements for "influential scientific information" or "highly influential scientific
assessments."
EPA's Action Development Process (ADP, 2004) provides detailed processes and
procedures for the development of regulations, including the development of Analytic Blueprints
that formalize scientific input at the beginning of the process. In addition, we wish to explore
approaches that may better distinguish science and policy judgments in proposed rules.24 This
20 http://www.epa.gov/enviro
21 http://iaspub.epa.gov/soUnternet/registry/sysofregfhome/overview/home.do
22 http://iaspub.epa.gov/sor_internet/registry/systmreg/home/overview/home.do
23 http://www.epa.gov/crem/knowbase/index.htm
24 To that end, a recommendation of the Final Report from the Science for Policy Project (Bipartisan Policy Center,
August 2009, http://www.bipartisanpolicy.org/library/report/science-policy-proJect-final-report) is being
considered. It states that agencies should develop guidance "to ensure that when ... developing regulatory
policies, they explicitly differentiate, to the extent possible, between questions that involve scientific judgments
and questions that involve judgments about economics, ethics and other matters of policy."
11-P-0386 31
-------
development of risk information for the regulatory process is supported by EPA's Guidance for
Risk Characterization25 and Risk Characterization Handbook26 The Guidance contains
principles for developing and appropriately characterizing risk in EPA's assessments. The
Handbook presents technical approaches that promote scientific integrity by ensuring that critical
information from each stage of risk assessment is used to form conclusions. When used together
with the ADP, these documents encourage an accurate presentation of risk science information to
inform decision making.
Public Communications
EPA is currently developing formal Agency-wide guidance for the communication of science
and technology information by our employees. This guidance will be part of our Agency-wide
scientific integrity policy and will ensure accurate, transparent, and accessible communications
of our science to the public. Further, while it is current Agency practice to allow scientists access
to the media, this new guidance will both ensure that there are no barriers for scientists when
communicating with the press and the public and reinforce our expectation that they adhere to
their fields of expertise when doing so.
Use of Federal Advisory Committees
At EPA, FACs are overseen by the Office of Federal Advisory Committee Management
and Outreach (OFACMO) with legal support from the Office of General Counsel. All EPA
FACs comply with the requirements of the Federal Advisory Committee Act (5 USC App. 2) and
the regulations issued by the General Services Administration (GSA) (41 CFR Part 102-3). In
addition, EPA's scientific FACs have been directed to follow procedures that are consistent with
the policy expectations of the December 17, 2010, memo. OFACMO is finalizing a new Agency-
wide guidance manual and a ten-chapter FACA Handbook for use by FACA committees.
EPA adheres to current standards governing conflict of interest as defined in statute and
implementing regulations. The Office of General Counsel Ethics Office oversees the procedures
for Special Government Employees (SGEs) who serve on scientific F ACs. These procedures
include the Confidential Financial Disclosure Form for SGEs serving on Advisory Committees
(EPA Form 3110-48), EPA Ethics Advisory 08-02: "Ethics Obligations for Special Government
Employees," and an online Office of Government Ethics course available on the internet. In
addition, the 2009 Addendum to EPA's Peer Review Handbook entitled: "Appearance of a Lack
of Impartiality in External Peer Reviews,"27 provides an elucidation of the regulatory definition
of "appearance of a lack of impartiality" for individuals who serve on peer review panels, criteria
for applying this definition, and illustrative examples.
25 http://www.epa.gov/spc/pdfs/rcguide.pdf
26 http://www.epa.gov/spc/pdfs/rchandbk.pdf
27 http://www .epa.gov/peerreview/pdfs/spc_peer_rvw_handbook_addendum.pdf
11-P-0386 32
-------
Professional Development of Government Scientists and Engineers
Professional development of the Agency's scientists and engineers is critical to
maintaining and enhancing the high quality of our EPA staff. To this end, the Agency has a host
of policies that promote and facilitate, as permitted by law, the professional development of EPA
scientists and engineers.28 In particular, scientists and engineers in ORD and other EPA offices
are encouraged to publish their research findings in peer-reviewed journals and to present their
research findings at professional meetings. We take pride in our exceptional staff and their
accomplishments in the external scientific community. Currently many EPA staff participate in
scientific societies as presenters, peer reviewers, session and conference chairs, and officers. For
example, in March 2011 more than 100 EPA scientists participated in the Society of
Toxicology's annual meeting in Washington DC.
EPA scientists and engineers are allowed to receive honors and awards for their research
and discoveries. EPA encourages its scientists and engineers to be part of the external science
and engineering enterprise and wants them to accrue the professional benefits of such honors and
awards. However, it is important to note certain ethical restrictions and limitations that may
apply to these external honors and awards.29
28 In accordance with EPA Ethics Advisory 92-04, the Agency encourages employees to participate and hold office
in professional societies and associations, subject to applicable conflict of interest considerations. With regard to
Government scientists and engineers serving as editors or editorial board members of journals or in positions of
responsibility within professional or scholarly societies, some clarification of the December 17, 2010 Holdren
memo may be required. If the employee has fiduciary interest in the organization, the goal in the memo may
conflict with ethics rules for Federal scientists. EPA cannot appoint anyone to serve in their official capacity in an
organization where they have a fiduciary duty except where specific statutory authority exists (e.g., 20 USC
5590(b)(1)(B) that directs the Administrator to appoint the Director of the Office of Environmental Education as an
"ex-officio member" of the Board of Directors of the Environmental Education Foundation). Otherwise, unless a
waiver is obtained, it is a conflict of interest under 18 USC 208 for Federal employees to serve in their official
capacities in a position where there is a fiduciary duty. Generally speaking, serving on an editorial board or as
journal editor would not be considered to be positions with fiduciary duties. However, serving as an officer or on
the board of directors of an organization would often be considered positions with fiduciary duties. It should also
be noted that Government employees may serve in their personal capacities In organizations even when they have
a fiduciary role. However, in these cases the interests of the organization would be imputed to the employee. As
such, the employee would have a conflict of Interest concerning the organization that could be waived subject to
provisions under 18 USC 208.
At EPA, as for the Federal government in general, honors and awards are considered "gifts from outside sources"
that are regulated by the Standards of Ethical Conduct for Employees of the Executive Branch (5 CFR Part 2635
204(d)). These gifts are limited to an aggregate market value of $200 if such gifts are a bona fide award or incident
to a bona fide award that is given for meritorious public service or achievement by a person who does not have
interests that may be substantially affected by the performance or nonperformance of the employee's official
duties or by an association or other organization the majority of whose members do not have such interests.
There is one exception: "Awards and Honorary Degrees" can permit gifts from outside sources with an aggregate
market value in excess of $200 (e.g., Nobel Prize) may be accepted upon a written determination by an agency
ethics official that the award is made as part of an established program of recognition. Similarly, an employee may
accept an honorary degree from an Institution of higher education based on a written determination by an agency
ethics official that the timing of the award of the degree would not cause a reasonable person to question the
employee's impartiality in a matter affecting the institution.
11-P-0386 33
-------
In addition, EPA provides internal and government-wide awards for its scientists and
engineers, including the Science Achievement Awards (SAA), the Scientific and Technological
Achievement Awards (STAA), and the Presidential Early Career Award for Scientists and
Engineers (PECASE). As part of EPA's honor award program, the SAA recognize scientists and
engineers who have excelled in their disciplines. Nominations are sought for EPA employees
who have excelled in a scientific area related to environmental protection, including: air quality,
biology/ecology, chemistry, earth sciences, engineering, health sciences, waste management,
water quality, and environmental economics. The awards are given in conjunction with the
appropriate professional society that participates in the selection process. The ST AA awards are
selected by a FACA panel established by EPA's Science Advisory Board so that this internal
award is determined by independent experts. PECASE awardees are selected for their pursuit of
innovative research at the frontiers of science and technology and their commitment to
community service as demonstrated through scientific leadership, public education, or
community outreach.
Through the Federal Technology Transfer Act (FITA, 1986), EPA scientists and
engineers can collaborate with their external peers. The FIT A (15 USC 3701 to 3716) provides
incentives for the transfer of technologies developed in Federal government laboratories to
private industry. At EPA, the FIT A program promotes collaboration between private and federal
researchers. Staff from EPA's FTTA program and OGC provide on-site training and one-on-one
visits with Agency scientists to explain and encourage patenting. The FIT A external partners
benefit from Cooperative Research and Development Agreements (CRADA) by tapping into
EPA's resources and knowledge base to conduct joint research and technology
commercialization. At any given time, EPA's FTTA program has approximately 100 active
CRADAs.
Each year about ten new patents are issued to EPA scientists. Under the FTTA, U.S.
government agencies are required to pay the employee(s)-inventor(s) at least 15% of the
royalties the agency receives under any licensing agreement (the remaining 85% goes to the
government lab). EPA exceeds this requirement by giving 35% to the employee(s)-inventor(s)
(and 65% to the government lab).
11-P-0386 34
-------
Appendix D
Distribution
Office of the Administrator
Agency Followup Official (the CFO)
Assistant Administrator, Office of Research and Development
Agency Followup Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Audit Followup Coordinator, Office of Research and Development
11-P-0386 35
------- |