Status of Scientific Integrity Concerns The Presidential Memorandum on Scientific lnteqrit\ (March 9, 2009) directs that "Each agency should have in place procedures to identify and address instances in which the scientific process or the integrity of scientific and technological information may [have been] compromised." EPA's SI Policy requires "mechanisms to ensure accountability." Allegations may be reported to the Scientific Integrity Official, any Deputy Scientific Integrity Official, or the Inspector General Hotline. INSPECTOR GENERAL hotline SCIENTIFIC INTEGRITY OFFICIAL 0pfICIAV.6 igure 1. How to seek scientific integrity advice or report an allegation In fiscal year (FY) 2018, the Program drafted a new procedure creating a two-pronged approach separating those seeking advice about scientific integrity concerns from those reporting allegations. In general, the new advice track was designed to resolve concerns before they became a formal allegation by giving informal and early counsel. Seventeen allegations and 56 requests for advice were received during FY 2020. Annual Update on Allegations and Advice Advice Lane The aim of the advice track is early preventive action to uphold EPA's culture of scientific integrity. Anyone with a question or a concern is encouraged to have a conversation with the Scientific Integrity Official (Francesca Grifo), the Deputy to the ------- Scientific Integrity Official, or any of the Agency's Deputy Scientific Integrity Officials who are in each program or regional office. These officials can provide timely advice or assistance. If the issue is not one of scientific integrity, they can assist in redirecting it as appropriate such as directing retaliation, waste, fraud, or abuse to EPA's Office of the Inspector General. If advice and assistance do not resolve the issue, an allegation may be filed with the Scientific Integrity Official or Deputy Scientific Integrity Officials. Following the development of the two-track procedure described in the box below, the Scientific Integrity Program reviewed all prior allegations and reclassified many of them as requests for advice. Box 1. Advice or Allegation? Advice or Allegation? Advice ¦ First conversation. ¦ Is it scientific integrity? • Next steps are clear. • Informational conversation. • Not high profile or directly linked to a threat to public health. • Can be anonymous. Allegation • Based on current information, it would be a violation of the Policy. • The submitter is aware of our limitations on confidentiality and wishes to proceed. • Advice is not appropriate. ¦ Previous advice was not effective or effective enough. • Urgent or high profile. • Expertise or support of the Scientific Integrity Committee is warranted. Advice and Allegations Through FY 2020 Between February 2012 and September 30, 2020, there have been 235 requests for advice and 101 allegations. Figure 2 illustrates allegations, indicated in green as well as advice requests, indicated in blue, by year since the Policy was adopted. For a breakdown of submissions by quarter, see Figure 3. ------- Queries by Fiscal Year 80 2012 2013 2014 2015 2016 2017 2018 2019 2020 ¦ Advice ¦ Allegation Figure 2. Allegations and advice by year 40 35 30 25 20 15 10 5 0 Queries by FY Quarter i ¦ , ¦ ¦ _ ¦ lillil i . . .h 1 ii Mill 1 2 3 4 12 3 4 12 3 4 12 3 4 12 3 4 12 3 4 12 3 4 12 3 4 1 2 3 2012 2013 2014 2015 2016 2017 2018 2019 2020 l Advice i Allegation Figure 3. Number of scientific integrity queries received by quarter ------- Requests for Advice in Fiscal Year 2020 In fiscal year (FY) 2020, we received 56 requests for advice (Figure 4). These ranged from questions about peer review and attribution (13%) to delay and suppression of scientific products (12%) to inappropriate interference (59%). FY2012-19 FY2020 Plagiariim 1% Other \% Clearance 2% Retaliation 2% General Advice 2% Peer Review Plagiarism 2% 2% Delay/Suppression 22% Data Quality 9% Delay/Suppression 12% Not SI 7% Interference 39% Interference 59% N = 179 N = 56 Figure 4. Advice Requests by Topic Increases in Advice Queries There are increases in two critical categories of queries - interference and suppression/delay. The number of advice queries that involved interference rose from 22 in fiscal year (FY) 2018 to 30 in FY 2019 and to 33 in FY 2020. One possible explanation for these increases is that advice queries can be submitted anonymously. Many of these advice queries were accompanied by a stated fear of retaliation, retribution, or other forms of reprisal and a clear statement that without that fear, they would have submitted formal allegations. Reprisal and retaliation are prohibited by federal law and all those reporting this to the Scientific Integrity Official are directed to report any instances to the EPA Office of the Inspector General. (For more information on advice, see Box 1. Advice or Allegation? and Box 2. What is Interference?) ------- Box 2. What is Interference? What is Interference? The altering of scientific products without scientific justification. For example: • Manipulation of science used in decision making. • Removing studies, cherry picking studies for inclusion, or narrowing the scope of the science without scientific justification. ¦ Rejection of models, new methods, information, or procedures. • Downplaying or exaggerating uncertainty. • Using inadequate, outdated, or substandard science • Risk management considerations driving risk assessment decisions. • Changes to minimize risk conclusions or removal of hazards in assessments. Summary of Allegations in FY 2020 Allegations in FY 2020 When advice does not resolve an issue, is not appropriate, or an issue is novel or complex, employees may file an allegation. If an issue concerns an unaddressed significant risk to public health or the environment, submitters are directed to EPA's elevation procedures or the Office of Inspector General. Any person from within EPA may report an allegation to the Scientific Integrity Official, any Deputy Scientific Integrity Official, or the Office of Inspector General. To allow the SIO or DSIO to more efficiently address allegations, allegation reports should include, when possible, detailed references to the specific provision(s) of EPA's Scientific Integrity Policy that were violated; supporting evidence with a timeline; and the names of witnesses who can provide pertinent information. Once received, the Scientific Integrity Program screens the allegation, gathers additional pertinent information, and makes a determination based on the available information, drawing on the experience and expertise of the Scientific Integrity Committee as needed. The determination includes recommendations for corrective scientific action and other preventive measures as appropriate. It is important to note that recommendations are not directed at individual employees but rather at safeguarding the science. Relevant managers and supervisors are informed of the outcomes of allegations as disciplinary and other corrective actions are their responsibility and not within the purview of the Scientific Integrity Program. Throughout the process, confidentiality is maintained to the extent the law allows and knowledge about the identity of persons submitting or otherwise involved in the allegation is limited to those who need to know. ------- In fiscal year (FY) 2020, we received 17 allegations (Figure 5). This an increase from the 11 allegations received in FY 2019. These ranged from questions about peer review and attribution to interference. Figure 6 breaks down the status of allegations between FY 2012-2020. Figure 5. Allegations by Topic Closed Figure 6. Status of Allegations (as of the end of FY 2020) ------- Summary of Closed Allegations in FY 2020 Five allegations were closed during fiscal year (FY) 2020. Summaries of the allegations adjudicated during FY 2020 are detailed below. An allegation of failure to follow authorship best practices was substantiated. An external coauthor alleged that a draft manuscript had been posted on EPA's website without his/her knowledge or consent. The Scientific Integrity Program found this allegation to be substantiated. An allegation of failure to acknowledge authorship was not substantiated. A former Oak Ridge Institute for Science and Education (ORISE) participant alleged that his/her name was inappropriately excluded from the authorship list of a published journal article. The editors of the journal conducted an independent investigation into the allegation. The Scientific Integrity Program was consulted by the editors of the journal as part of their investigation. The editors of the journal found the allegation to be unsubstantiated. An allegation of failure to acknowledge authorship was not substantiated. An EPA scientist alleged that his/her name was inappropriately excluded from the authorship list of a journal article. The Scientific Integrity Program found the allegation to be unsubstantiated. An allegation of inadequate peer review was closed. The subject matter at issue in this allegation was identified as being under pending litigation. The Scientific Integrity Program does not adjudicate legal claims or conduct parallel investigations of issues pertaining to legal claims. This allegation was closed. An allegation of interference was closed due to insufficient information being provided by the submitter. Office of Inspector General Report on Scientific Integrity On May 20, 2020, EPA's Office of Inspector General (OIG) issued the report #20-P- 01734, "Further Efforts Needed to Uphold Scientific Integrity Policy at EPA," which examined whether the EPA's Scientific Integrity Policy was being implemented as intended to assure scientific integrity throughout the EPA. The OIG audit examined the "extent and type of employee concerns with SI at the EPA; employee awareness of EPA's SI Policy, including the process for reporting potential violations; reasons potential violations may not be reported; and the adjudication process for allegations of SI Policy violations." The OIG compared their November 2018 survey (included in this audit) results with EPA's 2016 Scientific Integrity Survey and found an increased awareness of the Scientific Integrity Policy and how to report an allegation or violation of the Scientific ------- Integrity Policy. However, the survey comparison also found reduced perceived leadership support of scientific integrity and reduced perceived knowledge of review and clearance procedures among respondents. The report included recommendations of actions designed to help the SIO, SIC, Office of the Administrator, and other offices that consistently implement the Scientific Integrity Policy across the Agency such as finalizing procedures to address allegations of SI violations, tracking mandatory scientific integrity training, and supporting release of scientific products through a centralized clearance system. The Program adjusted its work plan to implement corrective actions in response to the report's recommendations. In fiscal year (FY) 2020, EPA completed three recommendations: institution of a tracking system to monitor new employee onboard scientific integrity training completion status; a system that reports progress on a quarterly basis to the SIC for any necessary follow up; and completed the SIC Charter. As detailed in Figure 7, EPA is continuing work to address the remaining recommendations. No. OIG Recommendation EPA Status 1 Determine the extent and cause of the culture and "tone at the top" concerns, based on the indicators from the OIG's scientific integrity (SI) survey. Issue the results to all EPA staff and make available to the public. On track 2 With the assistance of the Scientific Integrity Committee (SIC), develop and identify which performance measures will be used to define SI Program success and effective Scientific Integrity Policy (SI Policy) implementation. On track 3 With the assistance of the SIC, develop and execute a plan, including resource needs and milestones, to address the remaining action items identified by the agency to improve the implementation of its SI Policy." (Appendix A) On track 4 In coordination with OMS and the SIC, develop and implement a process for tracking completion of SI training for all new employees, including senior leadership and political appointees Completed 5 Provide updated information on SI training completion rates to SIC members and supervisors. Completed 6 In coordination with OMS, complete the development and implementation of the electronic clearance system for scientific products across the agency. On track 7 With the assistance of the SIC, finalize and release the draft procedures for addressing allegations of a violation of the SI Policy and incorporate the procedures into SI outreach and training materials. On track ------- 8 With the assistance of the SIC, develop and implement a process to adjudicate allegations of SI Policy violations involving high-profile issues or senior officials in the agency for which the SIO or SIC does not feel it can adequately adjudicate via existing procedures; include an indicator for when the process should be used. On track 9 With the assistance of the SIC, finalize and implement a charter or procedures to clarify the roles and responsibilities of SIC members. Completed 10 Include in the SI Program's annual reporting on allegations of SI violations (as applicable and to the extent that privacy allows): (a) adjudication outcome; (b) description of the process used to reach the adjudication outcome; (c) description of corrective actions and/or any longer-term changes or consequences to address the cause of substantiated violations; (d) whether and how the allegation was resolved through the advice/assistance process." On track 11 With the assistance of the SIC, finalize and post to the EPA's public website prior year Annual Reports on SI. On track 12 Develop a timeline or procedure that ensures the prior fiscal year annual report on SI is completed and distributed before the annual agency wide meeting on SI. On track Figure 7 Status of OIG Recommendations (End of FY2020) ------- |