Comprehensive list of EPA Scientific Integrity Activities FY 2021 Agencywide Meeting EPA AGENCYWIDE MEETING ON SCIENTIFIC INTEGRITY March 31, 2021 Virtual Meeting MEETING SUMMARY Participants Over 3,200 participants attended the virtual meeting and represented every EPA program office and region. Introductions Dr. Jennifer Orme-Zavaleta (ORD), acting administrator for the Office of Research and Development (ORD) and EPA Science Advisor, welcomed attendees and provided a brief description of Scientific Integrity and the meeting ahead. She introduced new EPA Administrator Michael Regan. Administrator Michael Regan spoke about the critical role scientific integrity plays in protecting the health and wellbeing of both Americans and the environment. He stated that as Administrator, he is committed to reaffirming scientific integrity as a core EPA value and to using it as a compass to guide the EPA. He reiterated a previous announcement that the EPA administration will investigate political interference in science by the previous administration to prevent future abuses from happening. He informed the group that many actions to implement the President's memorandum are already underway, including reviewing and evaluating the agency's scientific federal advisory committees to ensure that they include top tier experts to provide scientific and technical advice free of any conflicts of interest. The federal advisory committees will also review and update agency policies, processes and practices that may prevent the best available science and data from informing the equitable delivery of policies and programs. Administrator Regan called on the community to protect Scientific Integrity, and to offer and welcome differing scientific opinions as a legitimate and necessary part of the scientific process. Administrator Regan pledged to hear what he needs to hear and not only what he wants to hear. He stated that retaliation and other forms of reprisal will not be tolerated under his administration. Dr. Chris Frey (ORD) was introduced as the ORD Deputy Assistant Administrator for Science Policy. Dr. Frey provided a statement on the importance of scientific integrity and its critical role in developing sound policies to protect public health and the ------- environment. He then introduced Dr. Francesca Grifo, EPA Scientific Integrity Official, who expressed her thanks to all who spoke before her and opened the meeting to its first formal presentation. Role of the Office of the Inspector General Kristen Kafka from the Office of the Inspector General (OIG) provided an overview of the mission of OIG. This mission includes conducting audits and investigations in EPA with a specific focus on preventing and detecting fraud, abuse, or mismanagement; promoting efficiency and effectiveness; and adjudicating allegations of research or scientific misconduct. The goal of the OIG is to keep the Administrator, Congress, and the public informed about the problems and deficiencies of the EPA. She described the three outward facing parts that make up OIG: Office of Audit, Office of Evaluation, and Office of Investigation. Ms. Kafka informed the community of their responsibilities as they relate to reporting wrongdoing to the OIG and helping to achieve its mission. She provided the contact information for the Hotline coordinator as well as the EPA Whistleblower Protection Coordinator. Scientific Integrity Presentation Dr. Grifo presented the history and details of EPA's Scientific Integrity Policy since its adoption nearly 12 years ago, following President Obama's Memorandum on Scientific Integrity. Dr. Grifo echoed and reinforced the statement made by Administrator Regan that it is the responsibility of the "U.S. Federal employees, contractors, grantees, partners, volunteers to successfully navigate applying those statutes appropriately, hearing from our stakeholders in creating, using, and communicating, rigorous and independent science to inform our critical work to absolutely ensure the equitable delivery of our programs." She described the progress that the committee has made, even over the course of the past year, to make things right and "banish inappropriate influences on our science." Dr. Grifo posed the question to the community: "How has our culture of scientific integrity been challenged?" She then described two types of challenges: when the Scientific Integrity Policy was violated, and when EPA experienced interference not covered by EPA's Scientific Integrity Policy. Dr. Grifo described the ways that issues can be brought forward to the Scientific Integrity Committee: advice and allegations. Contacting the Scientific Integrity Committee for advice begins the informal, anonymous conversation to determine if the issue in question is a matter of scientific integrity. Typically, if advice is not effective or if the matter is either high profile or urgent, then it is elevated to an allegation. The classification of advice or allegation is typically decided by the person raising the issue; rarely is something classified as an allegation unless it is urgent, a public health risk, or another extenuating circumstance. Dr. Grifo noted that these conversations always include a discussion of confidentiality; while the Scientific Integrity Committee aims to keep records confidential, there exists a possibility for information to become revealed through such mechanisms as Congressional depositions. Dr. Grifo presented slides on the statistics that covered the percentages of which offices and regions had advice and ------- allegation lanes opened. In Fiscal Year 2020, there were 50 advice requests and 17 allegations of violations of the Scientific Integrity Policy. Dr. Grifo reiterated Ms. Kafka's point on reporting retaliation, reprisal, and any form of bullying to the hotline. Dr. Grifo then shifted focus to how the EPA is making changes and holding a high standard for strong science and scientific integrity with the new administration. The Executive Memorandum makes a commitment to protecting public health and to restoring science to tackle the climate crisis, and it also creates a White House-based task force of agency representatives to cover and take part in a range of topics, such as creating a framework for regular assessments of iterative improvement of the agency's scientific integrity policies. Agencies will be able to compare their scientific integrity policy against the framework and implement any changes or improvements consistent with the framework. Dr. Grifo added that the Scientific Integrity program is working through the backlog of allegations and placing them into two different groups: issues where the policy was ignored and issues where the policy was not strong enough. Dr. Grifo urged the community to reach out to her and the Scientific Integrity Committee for anything to be added or reviewed related to the work the committee members are doing for the Executive Memorandum. The ongoing work of the Scientific Integrity Program is looking at strengthening our Scientific Integrity Policy and looking to enhance the "norms" that ensure scientific integrity, such as honesty, rigor, transparency, and a firm commitment to evidence. Dr. Grifo hopes to increase awareness of the Scientific Integrity Policy and maintain it as an ongoing priority, especially as new hires are brought in. Resources to Expect from Scientific Integrity There will be an Agencywide anonymous survey distributed in April, put together by fellows, Dana Williamson, and Angie Boyce. The goals of this survey are to assess the culture of scientific integrity across the agency, to query employee awareness, their experience of scientific integrity, and to learn new ways to improve the policy implementation. There will also be a new onboarding Whiteboard training video on best practices, as well as the launch of a biennial scientific integrity training initiative, which will take place sometime in the late fall. Additionally, the program is looking at innovative ways to enhance the culture of scientific integrity, such as lessons learned from other agencies and researching barriers or ways to incentivize scientific integrity, and it is rewriting the allegations procedure. Dr. Grifo encouraged the community to reach out with any ideas for strengthening scientific integrity. Scientific Integrity Committee Presentations Betsy Shaw (OAR) provided an overview of the role that the Scientific Integrity Committee has played and currently plays in the agency. She spoke about how the committee is determined to promote continuous improvement. She highlighted the feedback loops to allow for continuous adjustments when learning from experience and to allow for policy adjustments when needed. Some of the feedback loops Ms. Shaw highlighted include the Annual Scientific Integrity Meeting, the survey, and the biennial trainings that Dr. Grifo touched on previously. There are also quarterly meetings with ------- the OIG, office hours, and all hands meetings. Ms. Shaw then highlighted some of the recent focus areas of the committee, which include the Whiteboard video trainings. She cited that the committee has trained over 800 managers and new political appointees. Carol Ann Siciliano (OCSPP) spoke about how the Scientific Integrity Committee is aiming to build capacity in all levels within the offices at EPA. She then provided background on the process of an allegation. When the allegation is made, an investigation is launched to understand all sides of the issue. The committee examines documents, conducts interviews, and produces an investigation report based on their findings. This report is sent to a group of panelists for review to determine if the issue was a scientific integrity violation and to provide a recommendation on what the appropriate response should be. It then moves back to the relevant scientific integrity deputy to take action in their office. Ms. Siciliano reminded the audience that Scientific Integrity program is not about punishment but about restoring scientific integrity and securing the science in all places. She cited an old EPA slogan as: "Think globally, act locally" and encouraged everyone to bring scientific integrity into their everyday work. Stakeholder Meeting BIENNUAL EPA STAKEHOLDER MEETING ON SCIENTIFIC INTEGRITY June 15, 2021 Virtual Meeting MEETING SUMMARY On June 15, 2021, the U.S. Environmental Protection Agency held its biannual stakeholder and partner meeting on scientific integrity. The external stakeholder meeting has been held since 2013 as a forum and opportunity for external stakeholders to hear about scientific integrity from the EPA Scientific Integrity Official and to comment on, or ask questions about, scientific integrity at the Agency. Transparency is a key component of scientific integrity, and this meeting represents an opportunity for EPA to demonstrate transparency. At this year's meeting, the EPA Scientific Integrity Official hosted special guests and shared information about current scientific integrity initiatives, discussed future plans for scientific integrity at EPA, and held a question-and-answer session. Dr. Jennifer Orme-Zavaleta, Acting Assistant Administrator for the Office of Research and Development (ORD) and EPA Science Advisor, welcomed participants and provided a brief description of Scientific Integrity and the meeting ahead. EPA Deputy Administrator Janet McCabe delivered opening remarks and introduced the Presidential Memorandum and the work that the agency has been doing and continues to do to comply with the memorandum. ------- Guest speaker, Dr. Alondra Nelson, Deputy Director for Science and Society at the White House Office of Science and Technology Policy informed the audience of the Presidential Memorandum on scientific integrity, released by President Biden on January 27, 2021. The memorandum signals the importance of scientific integrity and is helping to build the American people's confidence in this new era. Dr. Francesca Grifo, EPA Scientific Integrity Official, provided an overview of EPA's Scientific Integrity Policy, the Scientific Integrity team's responsibilities, challenges to Scientific Integrity at EPA and how the program reacts to those challenges, presented the audience with a graph to describe the allegations and advice from Fiscal year 2012 through March 31, 2021, and hosted a question-and-answer session. External Tribal Partner Meeting Scientific Integrity External Tribal Partner Meeting July 27, 2021 Virtual Meeting MEETING SUMMARY Introductory Remarks and Presidential Memorandum on Scientific Integrity JoAnn Chase, Director of the U.S. Environmental Protection Agency (EPA) American Indian Environmental Office, outlined the unique legal and political government- government relationship that the U.S. government has with tribal nations, forming the foundation of engagement with tribal partners. The mission of EPA's American Indian Environmental Office is to protect human health and the environment in Indian country. Recent presidential memoranda address restoring trust in the government based on scientific integrity and evidence-based policymaking and restoring trust with tribes. Ms. Chase noted that, in past administrations, the Commission on Environmental Cooperation (CEC) was able to engage tribal experts from Canada, the United States and Mexico in discussions to arrange for the implementation of traditional ecological knowledge (TEK). Scientific Integrity Dr. Francesca Grifo, Scientific Integrity Official at EPA, provided information on scientific integrity and the Scientific Integrity Policy at EPA. Scientific integrity is the adherence to professional values when conducting, communicating, influencing, and using scientific information. Dr. Grifo expressed hope for an open scientific culture and a culture of honest investigation, a firm commitment to evidence, and robust scientific inquiry and ------- discussion, which creates public trust in science and allows EPA to do its best work. Dr. Grifo emphasized that policy implications should not influence scientific conclusions. EPA scientists also need to be able to amicably disagree. Dr. Grifo emphasized that the government must be guided by the best science to ensure the integrity of federal decision-making. Dr. Grifo reiterated that her team works to ensure that EPA science is high quality and independent, that EPA performs its work and conveys the results of the science in a manner aligned with the Scientific Integrity Policy regardless of internal or external pressures, and that scientific conclusions are distinct from their policy implications. Q&A Dr. Grifo and Ms. Chase responded to attendee questions. Although the Scientific Integrity Policy is not a law, Dr. Grifo emphasized that the greatest potential barrier to scientific agency is an inhospitable culture at EPA, so her office frames its work in terms of creating a strong culture of scientific integrity at EPA. Scientific integrity is not a partisan issue. Although everyone has bias, the only way to address bias is to balance it by including all relevant voices and significant points of view in the discussion. Ms. Chase noted her intent to prioritize TEK in EPA's decision-making. Agency staff require additional, more creative training on engaging with tribal partners and incorporating traditional cultural knowledge into scientific decision-making. Dr. Grifo and Ms. Chase agreed with a comment on the importance of good data—as well as resources—to supporting good science and good decisions. Dr. Grifo emphasized that tribal partners know what they want and need, and she supports their decisions. Ms. Chase added that one of EPA's responsibilities is the direct implementation of programs in tribal lands, which requires legitimate engagement and inclusion of tribes 'science. Complete Listing of FY 2021 Scientific Integrity Committee Members Vii'iililii InKuril) Olliiial J Office of the Administrator Wes Carpenter j Office of Air and Radiation Betsy Shaw | Office of Chemical Safety and I Pollution Prevention Carol Ann Siciliano J Office of the Chief Financial Officer David Bloom | J OA/Office of Childrens Health Protection Jeanne Briskin j Office of Enforcement Compliance Assurance Erica Canzler | | Office of General Counsel Helen Serassio j Office of International I and Tribal Affairs Martin Dieu | ------- | Office of Land and Emergency Management Barry Breen Office of Mission Support Lynnann Hitchens J OA/Office of Policy A1 McGartland J OA/Science Advisory Board Tom Brennan Office of Research I and Development Bruce Rodan | Office of Water Benita Best-Wong Region 1 Johanna Hunter Region 2 Anahita Williamson / Linda Mauel Region 3 Bill Jenkins Region 4 John Blevins/ Dawn Taylor Region 5 Carole Braverman Region 6 David (Wes) McQuiddy Region 7 Cecilia Tapia Region 8 Sandra Spence J Region 9 Duane James J Region 10 Michael Szerlog Listing of FY 2021 Scientific Integrity Activities by EPA Offices and Regions About the lli > 'i I¦. < ;hnical an n' II \ eir Review National Center of Environmental Economics (NCEE) demonstrates its commitment to Scientific Integrity through several ongoing activities. NCEE assessed their office's effectiveness at implementing their Quality Management Plan. When no major findings were discovered, approval was sought and obtained by the Office of Mission Support, and NCEE updated their Quality Management Plan. For social science works, NCEE provides support to the management of peer review contracts and helps to supply external peer reviews of those products. They also initiate peer reviews of working papers prior to their release online to the public. ------- Office of Air and Radiation office prioritizes the peer review process of their technical products, and they ensure principal investigators and authors understand their obligations towards internal and external peer reviews. Office of Children's Health Protection (OCHP) led two updates in FY 2021 that the director is intending to utilize in FY 2022. They initiated an update to its Quality Management Plan and made a clearance form promoting the management review of scientific presentations, papers, posters, and similar products. Office of Pesticides Program (OPP) has an internal peer review system to meet scientific integrity and quality assurance goals. Managers are responsible for the level of peer review and overall scope of the review for individual risk assessment cases. In FY 2022, OPP plans to assess the quality assurance of a pesticide registration and check whether the Scientific Integrity Policy is being adhered to by examining the following: ¦ The Peer Review Record- The record will be examined to see if the appropriate subject matter experts were involved in the review of the project. In the peer review, OPP will look for any dissenting opinions and if those dissenting opinions were heard and kept as part of the record. Then finally OPP will look for the resolution of all comments. ¦ Significant Drafts of Documents- The drafts will be examined for any changes to the original. If changes occurred, then OPP will determine if those changes were documented properly and adhered to the Scientific Integrity Policy. ¦ Selected Scientists and Management- Interviews will be scheduled for only those who are involved in the application process and decision- making. Some questions will be framed to investigate if the Scientific Integrity Policy is being followed. ¦ Potential violations- If any findings reveal violations of the Scientific Integrity Policy, they will be reported to the OPP Office Director with recommended corrective actions. The Office of Chemical Safety and Pollution (OCSPP) - Office of Program Support (OPS) is a new office within OCSPP and started several initiatives. For quality assurance, they updated OCSPP's Standard Operating Procedures and are developing a Quality Management Plan draft. The Office of Program Support (OPS) in the Office of Chemical Safety and Pollution Prevention (OCSPP) is designing an electronic workflow for the clearance process using the Agency's best practices. In FY 2022, OPS plans to complete its first Quality Action Plan. The Peer Review and Ethics Branch will offer Peer Review Training so that staff are aware of internal and external peer review options as well as the times in which the OPS's two Federal Advisory Committees (FACs) should be used. OPS's two FACs ------- are FIFRA Scientific Advisory Panel (SAP) and TSCA Scientific Advisory Committee on Chemicals (SACC). Also, OPS is planning to train all OCSPP employees on the clearance process for technical documents and peer review requirements for the office and any additional peer review guidance from the Agency Peer Review Handbook. Lastly, the Deputy Scientific Integrity Official resolved an authorship dispute in FY 2021 and has begun to investigate other allegations. Office of Pollution Prevention and Toxics (OPPT) drafts are peer reviewed as required by the Risk Evaluation Process Rule and Prioritization Process Rule. They also sought additional feedback from the public on five final rules for Persistent, Bioaccumulative and Toxic (PBT) chemicals. By maintaining their accreditation, Office of Enforcement and Compliance Assurance/National Enforcement Investigations Center (OECA/NEIC) has demonstrated that they operate efficiently and with a robust quality management system. A few non-conformities discovered, but they were addressed through corrective actions, preventative actions, or corrections. Currently, a tracking database and a methodology to documenting unified measurement uncertainty (MU) is under development at NEIC. NEIC is continuing to utilize its Proficiency Testing (PT) program which guarantees that samples and analytical technologies meet the established criteria. The Center for Environmental Measurement and Monitoring's (CEMM) director of Quality Assurance is developing a new internal review form for outside collaborations and technical assistance. It will serve as a temporary measure until a long-term solution is found. Much of the Center for Environmental Solutions and Emergency Response's (CESER) work has focused on improving quality assurance through tracking or training employees on sites like ScienceHub. Their divisions regularly coordinate and track peer reviews ensuring that experts only review one product within a short timeframe. CESER is currently developing ORD Assist for the purpose of tracking ORD Technical Support requests and responses. ORD Assist will also provide reminders for quality assurance. Additionally, CESER used the web to quickly make COVID-19 research publicly available through 5 interim releases. Office of Science Information Management (OSIM) focused on data management and market analyses of their Electronic Laboratory Notebooks (ELNs) and Laboratory Information Management Systems (ELNs). Under data management initiatives, OSIM is conducting Information Technology Reviews that are intended to increase public accessibility to scientific data. The market analyses are for the improvement of documentation and management of research activities. The Office of Water (OW) staff produced several peer reviewed publications in FY 2021. Additionally, OW updated the project level Quality Assurance Project Plan and field protocols for Stream Duration Assessment Method Development (SDAM). They also ------- published a beta SDAM for the Arid West. It is planned that every region will have a beta testing period where a formal peer review process will take place before the final version is revealed to the public. OWs Water Quality Standards (WQS) are supported through the external and public peer review of OWs white papers and technical documents. For the public review process and to encourage different scientific perspectives, OW develops focused charged questions to the public and external peer reviewers. OWs Standards and Health Protection Division maintains both an internal and an external database. The WQS Action Tracking Application (WATA) is a database with state and tribal CWA WQS. The National Aquatic Resource Survey (NARS) program's products and reports are subject to Information Quality Controls and peer reviews. A regular ongoing activity for the NARS program is the development and implementation of Quality Assurance Project Programs for staff partners and other affiliates. Deepwater Horizon Natural Resource Damage Assessment Louisiana Trustee Implementation Group (TIG) and Open Ocean TIG Monitoring and Adaptive Management Development (DHNRDA TIG): Deepwater Horizon Trustees see the necessity of monitoring and adaptive management (MAM) for restoration planning and the several factors that contribute to the ongoing Deepwater Horizon oil spill restoration effort. Monitoring directly supports adaptive management as a feedback mechanism. The Trustee TIG MAM has also formed a workgroup that develops and releases a 5- Year Programmatic Review. The publicly available information includes the data analyses, the restoration plan, the current restoration status, and an overall summary of activities through 2020. Effluent Limitation Guidelines (ELGs): OW produces ELGs using quality assurances, and ELGs are subject to public comment before they are completely finalized. Lastly, OW intends to make wastewater treatment technology easier for the public to access. Quality assurance is promoted in Region 2 by conducting reviews for NJ's Department of the Environment, responding to inquiries by the NJDEP and New York State's Department of Environmental Conservation. Additionally, the Quality Assurance Project Plan for the Assistance Agreements and Brownfields Cooperative Agreements were reviewed and have been streamlined in FY 2021. Training sessions have been offered for the new streamline process. Region 2's Enterprise Quality Management Division (EQMD) leads the State and Tribal quality assurance project plan's metrics for their Environmental Lean Management System (ELMS) where the Deputy Scientific Integrity Official serves as EQMD's point of contact. The Deputy Scientific Integrity Official assists by defining backlog metrics, identifying and providing corrections, recommending changes for 60-day spreadsheets, and submitting a monthly metrics chart to EQMD. Lastly, the Peer Review Coordinator, ------- Scientific Integrity Manager, and the Regional Science Policy Forum Lead reviewed several other documents including the Peer Review Advisory Group Charter, product review for Science and Technical work products, and reports by EQMD. Region 2 also constructed a quality assurance annual report and work plan. In FY 2022, EQMD proposed quality assurance activities to track critical metrics. Laboratory Services and Applied Sciences Division (LSASD) Quarterly Quality Assurance meetings were held for LSASD staff and management. Air and Radiation Division (ARD) sends automated reminder emails to staff about Section 105 deadlines which aids ARD's tracking of State quality assurance plans and project plans. Lastly, ARD understands that the data for vulnerable communities is variable and unique to each community. To be more inclusive, ARD uses an Integrated Assessment when analyzing data because the approach is more flexible. ARD is developing a process for their federal equivalent method (FEM) monitors to be consistently deployed due to possible data quality issues. Chesapeake Bay Program Office (CBPO) conducts peer reviews on major projects and an annual assessment of Best Management Practice (BMP) data for purposes such as tracking, assurance of TMDL targets, and accountability. An interagency agreement with USGS was made to ensure the quality of environmental data. As part of the agreement, CBPO performs the following environmental data activities regarding quality assurance: ¦ Documentation and review ¦ Assistance with control policies, procedures, and requirements ¦ Reviews requirements for contracts, grants, and interagency agreements ¦ Validates data and usability of data ¦ Audits and other types of assessments of laboratory and field activities Land, Chemicals, Redevelopment Division (LCRD): Region 3 has developed a new Regional Quality Management Plan, and LCRD has aligned its training and SOPs to reflect any changes to the plan. Staff can access Quality Assurance information through LCRD's SharePoint site, and all staff received quality assurance training in FY 2021. Laboratory Services and Applied Science Division (LSASD) focuses heavily on quality assurances by conducting audits, using SOPs, and adhering to the Quality Management Plan and correcting any vulnerabilities identified. The Water Division (WD) team improved the process for receiving, assigning, and reviewing program Quality Assurance Project Plans. Region 4 also conducted a peer review and presented National Aquatic Resource Survey information to States. WD staff who work with interagency division implementation formed workgroups that addressed clarification concerns for trainings. ------- In fiscal year 2021, Region 4's Enforcement and Compliance Assistance Division (ECAD) updated several Standard Operating Procedures including: Water Enforcement Branch Field Documentation, Water Enforcement Branch Planning Inspections/Investigations and Preparation of Reports, Conducting Compliance Monitoring Inspections Federal Insecticide, Fungicide and Rodenticide Act, Conducting Compliance Monitoring Inspections Toxic Substances Control Act Polychlorinated Biphenyl. Region 4 and the Gulf of Mexico Division (GMD) finalized the Quality Management Plan (QMP) with updates intended to improve the consistency of quality assurance in assistance agreements. The updates concern a graded approach language which sets a quality standard that must be met for assistance agreement projects. The updates also change the programmatic terms and conditions awards that help to produce environmental information with the latest changes to the QMP. Region 4's Air and Radiation Division (ARD) implemented the Atlanta Rail and Port Sensor Project (RAPS) which is a pilot study that is intended to improve air sensor research and citizen science. The team has finished the monitoring component of the study and is finalizing the final report. ARD also initiated another pilot program called the Regional Applied Research Effort (RARE), and RARE has 3 potential purposes: improve Volatile Organic Compound emissions measurements near bulk gasoline terminals; further the development of low-cost monitoring; and find different methods of identifying and quantifying air toxics. Region 4's Water Division (WD) continues to examine the review process of their Water Division Quality Assurance Project Plans (QAPPs). The WD team drafted new Standard Operating Procedures that includes a new process for receiving, assigning, and reviewing program QAPPs. In FY 2019, Region 4 identified inefficiencies due to changes in its organization's structure. In response to some inefficiencies discovered, Region 4 has been trying to streamline the quality system and improve consistency throughout the divisions like the Laboratory Services and Applied Sciences Division (LSASD). The result is the integration of the Quality Assurance Field Activities Procedure (QAFAP) into the overall quality management system and a decrease of Quality Management Plans (QMPs). In FY 2021, Region 4's QMP has been updated to include the streamline efforts and organization changes. Region 6 displays its adherence to scientific integrity through their quality assurance efforts. They provide recommendations to States, local organizations, and Tribes as well as Technical System Audits (TSAs). Region 6 now offers online Quality Assurance trainings to staff when a 4 day in-person training was typically offered. The training includes Data Quality objectives and introduces basic management issues within EPA's Quality Assurance Program. The Houston Environmental Laboratory Standard Operating Procedures (SOPs) and Quality Assurance Manual outline and address important elements of Quality Assurance such as quality sample preparation, peer reviews, and publishing papers. ------- Region 7 focused on scientific integrity and Quality Assurance trainings in each division. The Region offered a 3-day in-person training Quality Assurance training to the EPA, state, and tribes as well as an online Quality Assurance refresher course. The overall quality of LSASD's science and peer reviews has improved due to the revival of the cross-EPA peer review workgroup. ORD's EPA-Funded Research Data Implementation Plan has also helped LSASD by clarifying responsibilities and providing additional training and support. Lastly, LSASD regularly works on improving their clearance procedures. I m kase of Scientiiin II111ormation National Center of Environmental Economics (NCEE) is open to feedback and has instituted the following in support of releasing information to the public: ¦ NCEE has developed a computable general equilibrium model (SAGE) which was reviewed in late FY 2020. The public will be able to access the model and data for regulatory analysis purposes. ¦ NCEE continued to support the Environmental Lean Management System (ELMS) project. ELMS helps the office with improvements for starting projects and tracking ongoing research. The ELMS project aids the office's ability to attain timely, constructive feedback which enables staff to take steps to making the research accessible to the public. ¦ Greenhouse Gas Reporting Program (GGRP) redeployed the Greenhouse Gas Reporting Tool (e-GGRT) that improved validation and verification checks on reported data. Center for Computational Toxicology and Exposure (CCTE) promotes scientific integrity with an emphasis on quality assurance and the release of information to the public. To establish a high-quality data collection and adhere to Quality Assurance Project Plans, the Great Lakes Toxicology and Ecology Division (GLTED) uses STICS and Science HUB for controlling and reviewing products that will be used internally or released to the public. In FY 2021, CCTE continued their pilot program that tracks CCTE's research projects with the aid of their dedicated peer review coordinator. The goal of the pilot program is to determine the tools and publications most used by clients as well as the predominant research areas that should be focused on in the future. CCTE distributes information to the public through the EPA website, the Git Hub, and other online sites. For FY 2022, CCTE intends to inventory all research projects from FY 2021; determine the best peer review for each research project; and track the projects to ensure that peer reviews are conducted appropriately and timely. They are also transitioning digital object identifier (DOI) research products to EPA's cloud-based system, and CCTE hopes that the new system will correct some intermittent access errors that occurred on the previous site. Lastly, the Scientific Integrity Coordinator in GLTED can develop ------- educational tools and reminders on the importance of Scientific Integrity throughout the year in the weekly newsletter, PowerPoint trainings, and email blasts. Unregulated Contaminant Monitoring Rule (UCMR): UCMR minimizes data-entry errors through its protocols. For example, laboratories post data directly to EPA's web-based reporting system which has quality control checks for reliability. The Office of Research and Development's (ORD) Office of Science Advisor, Policy, and Engagement (OSAPE) co-chairs the Agency's Public Access Forum that discussed the development of more training resources for Agency scientists concerning public access requirements. Also, OSAPE holds a leadership role for ORD's Clearance Policy and Procedures and has been developing a detailed Standard Operating Procedure (SOP) for clearance policy implementation. ORD's Office of Science Information Management (OSIM) met with ORD's new Information Management Advisory Board (IMAB) which determined a goal of improved accessibility for customers to ORD's research data. OSIM began working on a Research Data Catalog with the following subgoals that would: enable a complete inventory of ORD's research datasets, enable a collection of descriptive metadata to facilitate research datasets, ensure findability, accessibility, interoperability, and reusability. The Office of Water (OW) published websites for the public and stakeholders called Nutrient Scientific Technical Exchange Partnership & Support (N-Steps Online), the National Listing of Fish Advisory, and the Beach Advisory and Closing Online Notification (BEACON) database. N-Steps online provides the public with information on nutrient criteria development and technical assistance, and BEACON provides information on pollution occurrences within coastal recreation waters. A public app was also developed called the Industrial Wastewater Treatment Technologies (IWTT). Region 1's Water Division (WD) inventoried scientific research and activities, and then they added products to the National Database. Region 1's Mission Support Division (MSD) collaborated with other groups and individuals to finalize the region's Clearance Procedures for Scientific Products which included an automated checklist that guides employees through the process. Office of Air and Radiation's (OAR) Office of Transportation and Air Quality The (OTAQ) made a resource webpage with informational resources for staff such as public access requirements for journal publications. OAR's Office of Atmospheric Programs (OAP) created a SharePoint with information on clearance for publications intended for the public. In May 2021, OAP re-launched the Climate Indicators in the United States public website. They also updated the Long- Term Monitoring Program for the lakes and streams website which includes an interactive map with monitoring site information such as photographs and aquatic ecosystem health trends. ------- The Air and Radiation Division (ARD) is working on an ongoing project through the Regional Applied Research Effort (RARE) Program called the Odor Explore App. The App is to serve as a reporting tool to help ARD address odor issues in a community as well as improving the overall transparency while determining strategies. It is still currently in the testing phase. Superfund and Emergency Management Division (SEMD) team members have been awarded ORD grants to develop and display some soil sampling methods. In collaboration with local officials, county partners, and state departments, SEMD has been conveying timely and transparent information to the public for the Davidson Asbestos Site. The Site collected samples from residential yards to determine asbestos contamination, and contaminated soils are being removed. The data is being reported transparently and in real-time through the development of a Story Map. SEMD has been working on increasing external data visibility through an external part review of the Airborne Spectral Photometric Environmental Collection Technology (ASPECT) reports. After internal and external reviews are conducted, the reports are posted online for the public. Region 4's Water Division (WD) assisted with the redesign of a public database called the Assessment, Total Maximum Daily Load (TMDL) Tracking and Implementation System (ATTAINS). WD has been reviewing the script and troubleshooting the system with a goal to improve user-friendliness and the database Region 6's SEMD started reviewing their Airborne Spectral Photometric Environmental Collection Technology (ASPECT) reports internally and externally. The report details Superfund Emergency Response Action missions such as their purpose and any findings or detections. After the reviews, the ASPECT reports will become available to the public online. Region 8 is partnering with ORD IRIS to speed up the IRIS assessment process to provide health information to the public faster. Professional Development and Outreach Office of Administration/Science Advisory Board (OA-SAB): One of the two FACs, the Clean Air Scientific Advisory Council, is reviewing the National Ambient Air Quality Standards (NAAQS) reconsideration of Particulate Matter. During the COVID-19 pandemic, OA-SAB held virtual meetings that enabled participation from Special Government Employees and the public. Office of Transportation and Air Quality (OTAQ) has two outreach programs for staff that entail mentoring. The Leadership for Non-Supervisors (LNS) program has a strong ------- mentoring component, and the MentorMatch program, pairs new staff with mentors in the office. The Office of Science Advisor, Policy, and Engagement (OSAPE) promotes professional development with training resources for staff. In FY 2022, they plan to lead the Agency in the Public Access Forum for the development of training materials concerning the requirements for public access. For professional development, the Office of Water (OW) encourages staff to publish their products and to create an Individual Development Plan (IDP) for their professional development goals. Then staff can discuss their IDPs with their manager at least twice a year. OWs SharePoint contains resources for developing an IDP, other professional opportunities, and training suggestions. In FY 2021, the Office of Water (OW) staff participated in several conferences and presented their individual publications. They also participated in several stakeholder association meetings. Lastly, OW developed and led training sessions on Water Quality Standards (WQS) through WQS Academy and the WQS webinar training for new EPA WQS Managers. Outreach and External Engagement Office (OEEO) regularly presents scientific information regarding Superfund sites to communities and through fact sheets. Superfund and Emergency Management Division (SEMD) collaborated with a Technical Assistance Grantee and offered Risk Assessment Training to a community on scientific decision-making processes for a Superfund Site. Region 1 began the R1 Science Exchange on a Microsoft Teams site to increase the Region's communication. The site contains regional science documents, data, and reports that may not have been published by EPA and would not typically be accessible to staff. Regional staff also have access to the calendar within R1 Science Exchange site where trainings, meetings, and webinars are shared. Lastly, train-the-trainer documents were created to educate the regional Scientific Integrity Coordinators on the changes to the Clearance Procedures. Region 4's Superfund and Emergency Management Division (SEMD) team has been working with the local community, State, and other organizations of the Westside Lead Site and adopted the SoilSHOP program. SoilSHOP provides lead education to the community about exposure and possible reductions, and SEMD has expanded the program to address lead concerns in urban soils. Region 4's Enforcement and Compliance Assistance Division (ECAD) inspectors are required to complete an Annual Refresher Training to keep their credentials as well as obtain recredentials every 4 years. In FY 2021, 93 inspectors were recredentialed ------- through 13 group training sessions and 14 one-on-one training sessions. ECAD also provided training for state and federal inspectors on the following topics: FDEP Flow Measurement Training, FDEP NPDES Sampling, OECA Intro to the EPA Quality Assurance Field Activities Procedure, FDEP Post FedTalent Basic Inspector Training Q&A Session, FDEP RCRA Hazardous Waste Advanced Inspector Training. ECAD also produced a public Lead and Copper Rule Sampling instructional video that illustrated collection methods for tap water to determine lead and copper levels in public drinking water systems. Region 8's Science Council and Training Officer collaborated to produce two rounds of training on Statistical Methods concerning the utilization of environmental data in the region. The three courses offered include: Applied Environmental Statistics 1, Applied Environmental Statistics 2, Non-Detects and Data Analysis In FY 2022, the Superfund and Emergency Management Division (SEMD) is intending to initiate a community outreach project at the Westside Lead site with funding from the RARE grant if selected. The outreach projects would be within the environmental justice community and will improve tree plantings for the neighborhood. SEMD intends to partner with community groups and a nonprofit tree planting organization if they receive the RARE grant. If1!ih • IIiIIih • II11 i ; i i 'M;¦ ii i 'min1 ^ i h nti'li'i linegrity Acroi I I Office of Radiation and Indoor Air (ORIA) has two Centers of Excellence that provide required training programs. As part of its continuing education campaign, ORIA offered an Advanced Health Physics course, and they developed training modules on taking quality radioactivity measurements. Outreach activities included brown-bag lunches and more development of their mentoring programs. Office of Atmospheric Programs (OAP) provides science training internally in an on- going series of lectures. Scientists also served as mentors in the Office of the Federal Chief Information Officer's (OFCIO) Data Science Training Program. OFCIO's Data Science Training Program is a pilot program that trains EPA workers on data science techniques and their application to data gathering, analysis, and presentation to decision makers. In FY 2021, OAP also provided briefings to various groups on ongoing analytical projects related to climate impacts in the United States. Office of Air Quality Planning and Standards (OAQPS) implemented an electronic flow board to track the review of proposals which gives staff more visibility to the tracking progress and allowing them to determine if it is progressing in a timely manner. OAR is developing a procedure to safeguard a Quality Assurance Project Plan for tracking ------- purposes. The new developments intend to increase transparency for individual projects or manuscripts in the clearance process. Office of General Counsel (OGC) included a session on scientific integrity in the July 2021 National Counseling Attorneys Conference. During the conference, the Scientific Integrity Official presented information about the agency's Scientific Integrity Program. Office of the Inspector General (OIG) oversees the Scientific Integrity Policy and has strict quality assurance standards for its documentation and reports. Any scientific information is audited internally by several teams to ensure that the information complies with the Scientific Integrity Policy. The Center for Environmental Solutions and Emergency Response's (CESER) staff are informed of Scientific Integrity and are expected to promote and follow through on scientific and ethical standards. Internal Trainings are offered on the Scientific and Technical Information Clearance System (STICS) and ScienceHub. Lastly, Research Planning and Initiative Staff (RPIS) assisted ORD staff by participating in the ORD Clearance and Peer Review workgroups. They served as a resource on the requirements of scientific products by providing guidance and clarity. Office of Science Advisor, Policy, and Engagement (OSAPE) houses the Scientific Integrity Program (the Program) and the Agency's Scientific Integrity Official. As such, it is a leader in many Scientific Integrity Initiatives. The Program holds regular meetings concerning scientific integrity such as the quarterly meetings with the Scientific Integrity Official, an annual employee conversation with the Scientific Integrity Official, quarterly meetings with the Office of the Inspector General, and quarterly meetings with the Office of General Counsel. The Program continues to provide ongoing training opportunities through the mandatory training for new hires at EPA and the management dialogue series. For its products, the Office of Water (OW) strictly follows the Agency's Peer Review handbook and has a strong quality assurance and quality control program including quality assurance checks for contracts. OWhas a Quality Assurance and Quality Control Coordinator who is responsible for quality assurance and quality control protocols for new work assignments. The Drinking Water Laboratory Certification Program in the Office of Water (OW) leads certification training for regions and states. OW also oversees all regional programs and provides technical support as needed. Stream Duration Assessment Method Development (SDAMD): Like with the other programs, the Stream Duration Assessment Method Development (SDAMD) follows by Agency protocols and Quality Assurance Project Plans. ------- Enforcement and Compliance Assistance Division (ECAD) supports compliance with the Scientific Integrity Policy through fieldwork, inspections, and their Divisional Standard Operating Procedures (SOPs). Their SOPs were developed to ensure best management practices are followed when ECAD is sampling, performing field data collection, and producing compliance reports. A Standard Operating Procedure was also created for the Integrated Compliance Information Database (ICIS) which made requirements for data submissions that support a continuous flow of data. Laboratory Services and Applied Sciences Division (LSASD) is crafting new trainings on the new Standard Operating Procedures for data validation. One data validation training targets Superfund Remedial Project Managers (RPMs), Onsite Coordinators (OSCs), and Site Assessment Managers for the purpose of informing RPMs, OSCs, and SAMs about the appropriate validation level that should be designated for projects. LSASD offers different trainings such as a Quality Assurance Training every 2 years for the region and an annual training on laboratory ethics for all laboratory employees. National Wetlands Condition Assessment (NWCA) offered quality data collection training to staff and was assessed during an Assistance Visit (AV). During an AV, Region 3 aids in quality data collection to States, Regions, and Tribes instead of an audit. Superfund and Emergency Management Division (SEMD): SEMD offers training on the development of Quality Assurance Plans and Field Sampling Plans to Remedial Project Managers (RPMs), On Scene Coordinators (OSCs), and Site Assessment Managers (SAMs). SEMD is collaborating with the LSASD for several activities including: ¦ Tracking systems ¦ Expedited Quality Assurance Guidance for emergency actions The development of a training on data validation for RPMs, OSCs, and SAMs because of the new SOPs for Data Validation being drafted by LSASD. SEMD provides regular training and outreach to staff on current science. For example, they offered training sessions on human and ecological risk assessment. They also offered trainings multiple times that mostly discussed scientific methods of identifying risk due to contaminations of both the Region and an EPA national. Region 3's Water Division's (WD) staff offers a regional Area-Wide Optimization Program (AWOP) for States to obtain tools and methodologies for their respective sanitary survey programs. WD also uses the Compliance Monitoring Data Portal (CMDP) to submit Safe Drinking Water Act compliance data to assist with the reduction of compliance reporting information errors. Region 4 initiated several pilot projects towards improving scientific integrity and quality scientific data. Notably, Region 4 has been trying to streamline the quality system and increase consistency throughout all the divisions. Because of the streamlining efforts, ------- the organizational realignment in 2019, and the new Enforcement and Compliance Assurance Division, the Quality Assurance Field Activities Procedure has been able to integrate into the overall quality management system. Gulf of Mexico Division (GMD) has assistance agreements with individuals who collect water quality data by assisting them with inputting information into the Water Quality Exchange tool. The information inputted also includes the level of quality assurance. Office of the Regional Administrator (ORA)- Strategic Programs Office (SPO) provided Environmental Justice Screen Training for EPA staff, State partners from Alabama and Mississippi, Federal partners from the FEMA and NOAA, and community members and organizations. The purpose of the training was to help attendees use the information in the database appropriately while stressing the importance of the assumptions and limitations in the underlying environmental data. Lastly, SPO provided wetlands delineation training for new National Environmental Policy Act staff. Region 3 WD's Drinking Water Section staff led the Region's Area-Wide Optimization Program (AWOP) and offered tools, approaches, and networking opportunities for those involved in sanitary survey programs. This initiative is due to the differences of waterways across the region; and consequently, technical assistance and training resources vary between states. WD is working on a Compliance Monitoring Data Portal for staff to input compliance data related to the Safe Drinking Water Act with the intention of reducing reporting errors. Region 5 held its annual awareness and training day in February 2021, and the Scientific Integrity Official was the speaker. Region 6's Air and Radiation Division (ARD) has been implementing virtual TSAs for air quality organizations who operate air monitoring network equipment. ARD is currently developing a SharePoint site to centralize training materials. In FY 2021, Region 8's Air and Radiation Division (ARD) hosted midyear discussions with grantees and a Quality Assurance representative. Region 8 has offered training courses on statistics to improve the science. The Deputy Regional Counsel updates and provides scientific integrity training through Region 9's Quality Assurance Team. Lab Services and Applied Sciences Division gave an overview of scientific integrity as part of their LSASD 101 presentation. Region 10's Superfund and Emergency Management Division (SEMD) Remedial Cleanup Branch is collaborating with LSASD in a workgroup to update the Data Management Plan. After the plan is developed, Remedial Project Managers and LSASD staff will receive training on the changes including updates, necessary guidance, and software ------- |