United States Environmental Protection Agency Fiscal Year 2025 Justification of Appropriation Estimates for the Committee on Appropriations Tab 14: Evaluation and Evidence-Building EPA-190R24002 March 2024 www.epa.gov/cj ------- Environmental Protection Agency FY 2025 Annual Performance Plan and Congressional Justification Table of Contents - Evaluation and Evidence-Building for FY 2025 EPA Evaluation and Evidence-Building for FY 2025 2 FY 2025 Annual Evaluation Plan 3 FY 2025 Annual Plan for Evidence-Building Activities 16 FY 2025 Evaluation and Evidence-Building Activities - Supplemental Funds 62 1 ------- EPA Evaluation and Evidence-Building for FY 2025 EPA's ability to protect human health and the environment depends on its use of high-quality evidence to support the development of its policies, decisions, guidance, and regulations. EPA programs collect data about their implementation and outcomes to monitor their effectiveness (e.g., the extent to which targets are achieved), efficiency (e.g., the extent to which activities are delivered on schedule and within budget), and equity (e.g., the extent to which all people regardless of background have fair access to program benefits). Consistent with the Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act). EPA works to promote a culture of using evidence to inform decision making and support continuous improvement in delivering programs effectively, efficiently, and equitably. Programs analyze their data to assess progress in achieving performance results, identify challenges in carrying out strategies, and consider opportunities to mitigate risks and barriers. Programs identify successful strategies and approaches, as well as areas for development, which then inform recommendations for best practices and improvement strategies that can be implemented across the Agency. EPA publishes an Annual Evaluation Plan (AEP), as required by the Evidence Act starting with l:\ 2022. which highlights EPA's planned investment in, and intended use of program evaluations and other evidence-building activities to improve programs effectiveness, efficiency, and equity. EPA's FY 2025 AEP describes key program evaluations and other evidence-building activities the Agency plans to undertake. Final program evaluation reports will be available on EPA's evaluation website unless otherwise indicated. This document is organized into three sections: 1. FY 2025 Annual Evaluation Plan: program evaluations proposed by each EPA office 2. FY 2025 Evidence-Building Activities: evidence-building activities proposed (a) to support EPA's Learning Agenda, and (b) by each EPA office 3. FY 2024 Evaluation and Evidence-Building Activities - Supplemental Funds: evaluations and evidence-building activities proposed to support (a) programs funded by the Bipartisan Infrastructure Law (BIL), also known as the Infrastructure Investment and Jobs Act (IIJA); and (b) the Inflation Reduction Act (IRA). 2 ------- FY 2025 Annual Evaluation Plan Office of Kntorcement and ( ompliancc Assurance Title Assessing the effectiveness of offsite compliance monitoring FY 2025 Lend Office Office of Enforcement and Compliance Assurance l ink lo EPA Strategic Plan Goal 3: Improve compliance with the nation's environmental laws and hold violators accountable. Objective 3.2: Ensure high levels of compliance with federal environmental laws and regulations through effective compliance tools - including inspections, other monitoring activities, and technical assistance supported by evidence and advanced technologies. Start Dale October 2022 Completion Dale September 2025 Note This evaluation is being conducted over multiple fiscal years. Purpose and brief description: This project is part of OECA's Compliance Learning Agenda which promotes collaboration between EPA, states, tribes, and academics to identify the most pressing programmatic questions, and develop evidence-based enforcement tools and techniques that will ensure the biggest impact on environmental compliance. The pandemic restricted our ability to do onsite inspections and provided insight that a broader portfolio of Off-site Compliance Monitoring (OfCM) activities may provide us with additional tools for our enforcement and compliance programs. To assess what EPA has learned from the extended use from 2020-2022 and gain insight into the efficacy of OfCM tools relative to onsite inspections, we conducted a preliminary, short-term assessment using readily available data and information to inform interim guidance and best practices. The findings of the Short-Term Assessment provided general information and trends that can be used by EPA to better understand how OfCM can enhance the National Compliance Monitoring Program. The findings indicated that: Effectiveness of OfCM activities varied by program and activity employed. OfCM activities do not supplant the need for onsite inspections. Formal enforcement rates resulting from the use of OfCM varied significantly between programs with rates as low as 0 percent and as high as 43 percent. Some programs were able to produce sizable formal enforcement cases without onsite inspections. There are trends for situations where OfCM is more useful and effective and where they are less useful and effective. 3 ------- EPA is using these results to guide longer-term evaluations of OfCM and the best uses of these tools going forward. EPA anticipates that the answers to these questions will involve multiple evaluation efforts given the range of programs and OfCM tools that will need to be assessed. Question(s) to be addressed: 1. How does the effectiveness of offsite compliance monitoring activities compare to onsite inspections? 2. What outcomes does EPA get from offsite compliance monitoring? 3. What is the most effective use for OfCM? (does it depend on the tool, the Program, and on the compliance history of the facility?) 4. Do OfCM tools support enforcement activities? Methodological and Analytical approach Data collection method: This project will use existing data for further preliminary research, followed by a randomized control trial (RCT). Data sets: The state/EPA inspection data, enforcement data, and state OfCM data from the Integrated Compliance Information System (ICIS) is created and available to EPA. The data from state associations, academic databases, and RCT will be created by an external party who will make it available to the Agency. Analytic approach: We will continue to work with academic partners to uncover which analytical method will be used on this project. Tools and/or equipment: We will continue to work with academic partners to uncover which, if any, tools and/or equipment might be used on this project. Anticipated challenges and proposed solutions: There are potential data limitations associated with this activity. For example, because of the broad categories in ICIS, the definition of the OfCM activity performed in ICIS may not be indicative of the actual activity performed. Additionally, there are limited links between OfCM activity and enforcement actions. To combat this, EPA will use an array of different data sources to obtain as much specific, credible information as possible to minimize irregularities. For the randomized control trial, EPA anticipates there could be hesitation from regions and states to participate. In an effort to mitigate these challenges, EPA has partnered with the E-Enterprise Leadership Council and have invited ECOS, states, and tribes to participate in the workgroup to complete learning agenda projects. EPA also will be proactive in marketing the benefits of the results of an RCT and how the results could be beneficial to regions and states as they make compliance monitoring decisions. Dissemination of findings: We anticipate making project findings public on EPA.gov. 4 ------- Office of Research and Development Title Research Planning Review for Strategic Research Action Plan 2023-2026 l.ead Office Office of Research and Development Link lo I.IW Strategic Plan Cross-Agency Strategy 1: Ensure Scientific Integrity and Science-Based Decision Making Start Date FY 2023 Completion Dale FY 2028 Note This evaluation is being conducted over multiple fiscal years. Purpose and brief description: ORD's research planning process is highly complex by design, such that many different types of participants and research are included. As a scientific organization, ORD leadership is interested in learning from staff about their experiences in the most recent research planning cycle, which informed ORD's Strategic Research Action Plans for Fiscal Years 2023-2026. One objective of the Research Planning Review (RPR) is to understand the effects of process improvement strategies that were implemented in the research planning cycle that resulted in development of the FY 2023-2026 Strategic Research Actions Plans (StRAPs). A second objective is understanding our partners' experiences in the most recent research planning process. Finally, the RPR will explore the planning process that other federal research organizations use with a goal of understanding the objectives, structures, participants, challenges, and ensuring science-based decision making. The RPR is intended to strengthen communication among ORD employees in the context of research planning, guide our ability to use evidence-based decision making, and continuously improve ORD's research planning process. Question(s) to be addressed: The overall goal of the RPR project is to identify what worked well in the most recent research planning cycle and where improvements could be addressed in future research planning cycles. The insights will be synthesized and used to identify specific strategies that may be implemented in future research planning cycles. Methodological and analytical approach Data collection method: The project design includes a mixed-methods explanatory approach and includes a range of primary data collection from individuals that will take place over the evaluation time period including but not limited to: o Surveys to internal EPA employees (Qualtrics) o Ad Hoc 1:1 informal conversations o Focused small group interviews o Workgroup strategy development discussions Data sets: Qualitative data will be produced from the survey sent to internal EPA Employees. 5 ------- Analytic approach: A mixed-method analytical approach will include thematic coding of open-ended responses and quantitative analysis of Likert-scaled questions. Tools and/or equipment: The survey instrument used will be Qualtrics, and responses will be analyzed using Qualtrics data visualization tools and Microsoft Excel. Anticipated challenges and proposed solutions: This evaluation will produce qualitative responses that will require analysis, interpretation, review coordination, task prioritization, and regular interactions internally within ORD and with the program partners (e.g., OAR, OCSPP, OECA, OEJECR, OLEM, OW, Regional Offices) to ensure future Strategic Research Action Plan Research Planning processes meet ORD's mission and our partner's needs. Dissemination of findings: Evaluation findings will be posted on the ORD intranet webpage. The findings from this evaluation will be included in EPA's FY 2028 Evaluation Report. 6 ------- Office of Water Title NEP Program Evaluation FY 2025 Lead Office Office of Water Link to LPA Strategic Plan (ioal 5: Ensure clean and safe water for all communities. Objective 5.2: Protect and restore waterbodies and watersheds. Start Date October 2024 Completion Dale September 2025 Note This evaluation is conducted each fiscal year. Purpose and brief description: The primary purpose of the Program Evaluation (PE) process is to help EPA assess how the National Estuary Programs (NEPs) are making progress in achieving programmatic and environmental results through implementation of their Comprehensive Conservation and Management Plans (CCMPs). The PE process has proven to be an effective, interactive management process that ensures national program accountability and transparency, while incorporating local priority considerations. It also demonstrates the value of federal investment in estuarine and coastal watershed restoration and protection at the local and regional levels. The PE process was revised, and new guidance distributed to the 28 NEP locations at the end of 2021. The 28 NEPs are evaluated on a rotating basis over a five-year cycle, so each NEP is evaluated every five years, but all NEPs are not evaluated in the same year. The PE process also is useful for: Transferring lessons learned among NEPs, EPA, and stakeholders through the sharing of case studies and transferable examples; documenting the value added to environmental management of estuarine systems using the partnership model of the national program and its individual NEPs, including their role in convening stakeholders for decision-making and interpreting science for management actions; demonstrating continued stakeholder commitment; and highlighting achievements and successes of each NEP, as well as suggestions for continued program improvements. Question(s) to be addressed: The evaluation process for NEP locations informs the Agency on the progress of the NEP program. It also ensures the locations are delivering environmental results and are well-managed programs so that they can continue to receive annual grants from EPA which are matched 1:1 with non-federal dollars. The program evaluation is focused on the National Estuary Program as described in Section 320 of the Clean Water Act. The PE goals are to: ensure submissions enable objective and consistent evaluations among the different NEPs; ensure a consistent and transparent process to determine NEP CCMP implementation progress; further align the PEs with individual NEP CCMP priorities and related NEP annual work plan goals and accomplishments; determine progress in achieving programmatic and environmental results by documenting NEP contributions to improving or reducing pressures on their coastal watersheds and enabling all NEPs to successfully serve as local 7 ------- implementation partners for EPA programs; and identify areas of improvement to assist NEPs in becoming stronger programs and achieving environmental results. Data collection methods and datasets: The PE consists of several phases: A) development and submission of a package of required information, B) PE team site visit to each NEP under evaluation, and C) documentation of PE findings via formal letter from EPA Headquarters. The PE Narrative Submission should report on the NEP's five-year evaluation period and include a concise, five-year cumulative self-reflection on the three key topics: 1) NEP Environmental/Programmatic Workplan Accomplishments, 2) NEP Program Implementation, and 3) NEP Ecosystem and Community Status. More information about the required information and format is available in the NEP Program Evaluation Guidance from September 2021, which the program can share upon request. Anticipated challenges and proposed solutions: The regular PE process examines each NEP location on a variety of topics as listed below. NEP Administration and Governance Structure Grant Obligations and Finance including budget summary Healthy Ecosystems (e.g., fish, shellfish, plant, eelgrass, and wildlife populations; habitat protection/restoration, natural resources, land use, hydrological and ecological restoration, invasive species) Community and Stakeholder Engagement Education and Outreach Monitoring and Assessment Clean Water Act Programs Relationship EPA Priorities (nutrient pollution, water reuse and conservation, marine litter reduction, green infrastructure, environmental justice, climate change) The challenge is to identify and relay recommendations for improvement based upon the categories above. The solution is to use discussions between the PE team and NEP location to review the recommendations, and then to follow-up these discussions by submitting a final PE letter to each NEP location. Dissemination of findings: Summary information on the NEP is available on the EPA's NEP website. EPA acknowledges the importance of NEP partnerships and proactive actions of most NEP location activities which are mostly non-regulatory and highly leveraged offering EPA an average value of $17 for every $1 of EPA investment. Individual PE results are typically not made available to the public. 8 ------- Title Charting a course beyond 2025 Lead Office Office of Water l ink lo I.PA Strategic Plan Goal 5: Ensure clean and safe water for all communities. Objective 5.2: Protect and restore waterbodies and watersheds. Start Dale October 2023 Completion Dale December 2026 Note This evaluation will be conducted over multiple fiscal years. Purpose and brief description: The Chesapeake Bay Program's (CBP) Chesapeake Executive Council (EC) has charged the CBP Principals' Staff Committee (PSC) with recommending a critical path forward that prioritizes and outlines the next steps for meeting the goals and outcomes of the 2014 Chesapeake Bay Watershed Agreement (2014 Agreement) leading up to and beyond 2025. At the 2024 annual EC meeting, the PSC is to prepare recommendations that continue to address new advances in science and restoration, along with a focus on the CBP partnership for going beyond 2025. A value assessment of the 2014 Agreement and a program evaluation of the CBP will be conducted as part of fulfilling the EC charge. Question(s) to be addressed: Questions to be addressed include: Can Chesapeake Bay water quality goals and the way attainment is measured be revised to increase attention to, and potential for, other water quality investments to improve living resources? What policy and implementation options offer the potential to deliver substantial and sustained reductions in nonpoint source loads? What processes and analytical approaches are available to improve learning, especially as it pertains to the first two questions above? What are the existing and emerging challenges that the CBP partnership will need to address beyond 2025 to be able to meet its goals and objectives? How will these challenges affect the CBP partnership's ability to meet its goals and objectives? How can these challenges be prioritized by the CBP partnership to ensure resources are being used most cost-effectively? What actions can be used to address multiple water quality goals/outcomes which are not being used to address multiple goals/outcomes currently? Who are the relevant communities the CBP partnership should engage to improve the health of the ecosystem? What are their distributions (geographic and otherwise)? What are the restoration and protection needs of these communities? To what extent is the CBP partnership meeting the conservation needs of these communities? If so, how? If not, why? Who are the CBP partnership's stakeholders? What routes do the stakeholders have in reaching (i.e., contacting) the partnership? To what extent do stakeholders use/access the defined routes to reach the CBP partnership and its data or other resources? 9 ------- To what extent do stakeholders feel the CBP partnership has listened to their needs? To what extent does the CBP logic model reflect actual operations? Data collection methods: Literature reviews, focus groups, (scientific) document review and synthesis, surveys, and expert elicitation with CBP partnership staff and management. Anticipated challenges and proposed solutions: Program evaluation, including the development of recommendations, to fulfill the EC charge is currently on a tight timeframe. In addition, the CBP partnership group tasked with developing recommendations is a large group with representation from across the Chesapeake Bay watershed. To address these challenges, EPA has leveraged contractor resources to assist with conducting the program evaluation and facilitating the CBP partnership group responsible for this work. Dissemination of findings: The findings will be made publicly available on EPA.gov. Title FY 2025 EPA Annual Assessment of the Jurisdictions' Progress toward Meeting the Chesapeake Bay Total Maximum Daily Load (Bay TMDL) Lead Office Office of Water / Region 3 l ink to i:r \ Strategic Plan (ioal 5: Ensure clean and safe water for all communities. Objective 5.2: Protect and restore waterbodies and watersheds. Start Dale 1 )ccember 2023 Completion Dale August 2026 Note This project is being conducted over multiple fiscal years. Purpose and brief description: Through the 2014 Chesapeake Bay Watershed Agreement, the Chesapeake Bay Program has committed to having 100 percent of pollution-reducing practices in place that would achieve all the nitrogen, phosphorus, and sediment reductions necessary to meet the goals outlined in the Bay TMDL by 2025. These estimates are generated by the Chesapeake Bay Watershed Model and are derived from land use data, implementation and effectiveness of best management practices and the most up-to-date water quality monitoring data. The Chesapeake Bay Program assesses water quality by the amount of dissolved oxygen in the Bay, chlorophyll a (a measure of algae growth) and water clarity (using underwater grass acreage). Question(s) to be addressed: The seven watershed jurisdictions, in coordination with local governments, businesses, non-governmental organizations and individuals, have installed pollution-reducing best management practices to lower the amount of nitrogen, phosphorus and sediment entering tributaries of the Chesapeake Bay. The conservation practices reported by the seven watershed jurisdictions, along with land use, manure, and fertilizer information, are entered into a sophisticated suite of modeling tools to estimate the progress that each jurisdiction is making in meeting their individual nitrogen, phosphorus, and sediment goals as outlined in the Bay TMDL. This project will incorporate the best available data into the computer simulations and pollution 10 ------- load estimates and will seek to give EPA and the larger partnership a more holistic view of how conservation practice installation and improved management actions are helping to improve Bay water quality. Questions to be addressed include: To what extent have different jurisdictions made progress toward their pollution reduction goals? What is progress that each jurisdiction is making toward reducing nitrogen, phosphorus and sediment pollution entering not only the Chesapeake Bay, but also their local waterways? Methodological and Analytical approach: Data collection methods: Data solicitation and mining from the seven Bay watershed jurisdictions (Delaware, District of Columbia, Maryland, New York, Pennsylvania, Virginia, West Virginia). Data sets: Best Management Practice data (created by the state agencies and made available to EPA), agricultural data (created by state and federal agencies and used by EPA), monitoring data (created by state and federal agencies and made available to EPA), and land use information created by state and federal agencies and made available to EPA). Analytic approaches: Statistical regression and trend analysis will be used. Tools and/or equipment: Chesapeake Assessment Scenario Tool, National Environmental Information Exchange Network, and Land Use Change Model. Anticipated challenges and proposed solutions: Anticipated challenges include late submission of data by the state agencies to EPA, incorrect data received, and processing errors. To address these challenges, EPA has developed Quality Assurance/Quality Control procedures and protocols for the submission and processing of data inputs and outputs and has established deadlines for the submission and release of progress and verification data to the public. Dissemination of findings: Findings will be made publicly available on EPA.gov. 11 ------- Title FY 2025 Program Evaluation of Habitat Restoration Practices and Tools in the Gulf of Mexico Watershed Lead Office Office of Water / Region 4 l ink lo EPA Strategic Plan Goal 5: Ensure clean and safe water for all communities. Objective 5.2: Protect and restore waterbodies and watersheds. Start Dale October 2024 Completion Dale No projected end date Note This project will be conducted over multiple fiscal years Purpose and brief description: The purpose of the evaluation is to assess progress made toward restoring, enhancing, or protecting habitats in the Gulf of Mexico watershed. The evaluation of progress is examined through outputs reported from recipients of assistance agreements and staff led efforts centered around projects or activities on agricultural lands and on watershed-based non- agricultural approaches supporting the use of nutrient management and reduction practices and tools. To quantifiably assess progress, habitat acres restored, enhanced, or protected will be reported quarterly and tracked on the Gulf of Mexico SharePoint page. Staff led or supported efforts yielding restored, enhanced, or protected habitat acres contribute to the overall reporting of data. Question(s) to be addressed: By tracking habitats restored, enhanced, or protected, the Gulf of Mexico will be able to respond to the following: Are quantifiable goals attained? If not, why not? What adjustments to action plans are needed to achieve the goals? Methodological and analytical approach Data collection method: To quantifiably collect data, outputs derived from assistance agreements and staff led efforts will be reported on quarterly basis and tracked on the Gulf of Mexico SharePoint site. The methodology for computation of habitats that are restored, enhanced, and/or protected is: once a reporting year no matter how many activities are done to enhance the same acreage. each reporting year when management activities are completed annually on the same acreage. as acres; however, it may be reported in another measurement and converted to acres. Data sets: The Gulf of Mexico will rely primarily on data submitted in required assistance agreement progress reports and numbers attained through staff led efforts. The assistance agreements progress and staff led efforts are tracked and reported every three months. All assistance agreement reports are stored on EPA's Next Generation Grants System. In addition to official electronic grant file, data are stored on the Gulf of Mexico SharePoint, where the staff led efforts; results are tracked as well. This data allows EPA to assess what federal 12 ------- investments are accomplishing. EPA also will evaluate progress in attaining habitat targets, quantifiable outcomes from actions. Analytic approaches: The data will be consolidated on the Gulf of Mexico SharePoint. Project data will be aggregated, allowing a cumulative analysis of what specific actions investments are contributing to and what is being accomplished within each action. Tools and/or equipment: The Gulf of Mexico has a StoryMap providing a depiction of financial investments, regional expanse of projects, and the varied project types underway to restore, enhance or protect habitats. This information will be located at ilf of Mexico Division StoryMap (arceis.com). Anticipated challenges and proposed solutions: The Gulf of Mexico has a history of awarding projects that yield acres associated with habitat restoration, enhancement, or protection. Inclement weather conditions may impact projects and delay outputs. This is remedied by extending project end dates. Recipients of assistance agreements and staff report on acres restored, enhanced, or protected quarterly. This data resides on the Gulf of Mexico SharePoint and is found in the EPA Next Generation Grants System. The public may submit a FOIA request to see data and some project information is located on the Gulf of Mexico StoryMap website. Dissemination of findings: This information will be available on the Gulf of Mexico StoryMap on an annual basis. Title FY 2025 Program Implementation Evaluations in response to GAO 18-410: Long Island Sound Restoration: Improved Reporting and Cost Estimates Could Help Guide Future Efforts l.esul Office Office of Water/ Regions 1 and 2 l ink lo KPA Strategic Plan Goal 5: Ensure clean and safe water for all communities. Objective 5.2: Protect and restore waterbodies and watersheds. Start Date October 2023 Completion Dale December 2025 Note This project is being conducted over multiple fiscal years. Purpose and brief description: The purpose of the evaluation is to assess progress made toward meeting the goals, actions, and schedules of the Long Island Sound Comprehensive Conservation and Management Plan (CCMP), including quantifiable targets of ecosystem condition. The evaluation is conducted through an EPA SharePoint Power Business Intelligence (BI) application reporting and tracking system. Data will be added to the online tracking and reporting system to show progress in implementing the CCMP. EPA also will evaluate progress in attaining ecosystem targets, quantifiable outcomes from individual actions. Data on ecosystem targets will be collected from several sources, including environmental monitoring and project assessments. In 13 ------- combination, these assessment efforts will evaluate progress to actions funded with FY 2024 appropriations and before. Work funded with FY 2025 appropriations will be assessed in FY 2026. Question(s) to be addressed: The Long Island Sound online reporting and tracking system is completed and in full use by the Study. Data are added to the system approximately every six months to allow the Study to evaluate progress toward goal implementation. The online reporting and tracking system addresses the leading practice of reporting recommended by the GAO, which is to evaluate actions to support outcome goals. By tracking ecosystem targets and implementation actions, the Study will be able to respond to the following questions: Are goals being met? If not, why not? What adjustments to action plans are needed to achieve the goals? Methodological and Analytical approach: Data collection methods: The purpose of this evaluation is for EPA to meet the statutory requirement under the Clean Water Act Section 119 for the Long Island Sound Office to issue biennial reports to Congress summarizing the progress made in implementing the CCMP, any modifications to the CCMP, and recommendations concerning the CCMP. To accomplish this, the Program will use grant progress report data that is entered into an internal EPA SharePoint site. That data is used to compare intended to actual performance in accomplishing the targets and actions in the CCMP. EPA also will evaluate progress in attaining ecosystem targets, quantifiable outcomes from individual actions. Data on ecosystem targets will be collected from a number of sources, including monitoring and project assessments. Data sets: EPA will rely primarily on data submitted in required grant and interagency assistance agreement progress reports. The reports are required every six months. All reports are stored in EPA's official electronic grant file system. Data from the reports includes what CCMP actions are being implemented, costs, output metrics, and project status. This data allows EPA to assess what federal investments are accomplishing. EPA also will evaluate progress in attaining ecosystem targets, quantifiable outcomes from individual actions. Data on ecosystem targets will be collected from a number of sources, including monitoring and project assessments. All ecosystem target reporting is documented and available to the public at LISS Ecosystem Targets and Supporting Indicators - Lone Island Sound Study. Analytic approach: EPA will consolidate data on an EPA SharePoint Power BI application. Project data will be linked and aggregated to specific implementation actions in the CCMP, allowing a cumulative analysis of what specific actions investments are contributing to and what is being accomplished within each action. Tools and/or equipment: EPA will consolidate data on an EPA SharePoint Power BI application. This application is accessible to EPA staff. The public version of the LISS online reporting and tracking system is available at: https://longislandsoundstudv.net/program- implementation-and-progress/. 14 ------- Anticipated challenges and proposed solutions: The Long Island Sound online reporting and tracking system was developed in response to GAO 18-410: Long Island Sound Restoration: Improved Reporting and Cost Estimates Could Help Guide Future Efforts. One of the report recommendations was that EPA should develop a reporting format that fully incorporates leading practices of performance reporting. The overall evaluation system fulfills that requirement by showing past conditions and progress over time toward ecosystem targets in the recovery plan. The online reporting and tracking system addresses the leading practice of reporting recommended by the GAO, which is to evaluate actions for unmet goals. By tracking implementation actions, the Study will be able to provide suggestive evidence about why goals are not being met and create plans and schedules to achieve the goals. The key challenge is to work with program partners to develop appropriate metrics for progress reports and work with partners to have them develop complete and timely reports. Dissemination of findings: The public version of the LISS online reporting and tracking system is available at: https://longislandsoundstudv.net/program-implementation-and-progress/. All ecosystem target reporting is documented and available to the public at rstem Targets and Supporting Indicators - Long Island Sound Study. In addition, "Reports to Congress" summarizing evaluations are made available to the public. 15 ------- FY 2025 Annual Plan for Evidence-Building Activities EPA's FY 2025 Annual Plan for Evidence-Building Activities describes agency plans for significant evidence-building across a range of program areas. In this section EPA describes evidence-building activities other than program evaluations, such as data analysis, foundational fact finding, research, statistical analysis, continuous process improvement, and performance measurement. This document shares examples of evidence-building that supports EPA's decision- making in response to Administration priorities, Congressional mandates, and management priorities. The first section presents a summary of EPA's evidence-building activities in support of the Agency's Learning Agenda. The subsequent sections summarize the evidence-building activities planned by each national program office. HPA Learnin« Agenda The Evidence Act provides a framework to promote a culture of evaluation, continuous learning, and decision-making using the best available evidence. EPA's FY 2022 - FY 2026 Strategic Plan incorporates learning priority areas for the first time as required by the Evidence Act, which is a significant part of developing this culture. EPA has identified four learning priority areas: 1. Drinking Water Systems Out of Compliance - What EPA/state drinking water program policies (tools, guidance, training, funding mechanisms) are most effective in increasing system compliance? 2. Expanding EPA's Toolkit of Air Benefits Assessment Methodologies and Practices - How can EPA more comprehensively characterize the health benefits associated with improved air quality and improve approaches for quantifying and valuing air pollution effects among populations most susceptible and vulnerable to poor air quality? 3. Workforce - How can EPA ensure it has employees with the competencies needed to achieve its mission now and in the future, including identifying or developing leading practices in recruitment, retention, succession planning and knowledge management? 4. Grant Commitments Met - How can EPA assess the extent to which commitments achieve the intended environmental and/or human health results and identify possible next steps in establishing a comprehensive grant reporting system? This section summarizes the planned evidence-building activities in FY 2025 that will support the Agency's learning agenda. 16 ------- Priority Area Drinking Water Systems out of Compliance Lead Office Office of Enforcement and Compliance Assurance Link lo I.PA Strategic Plan Goal 3: Enforce environmental laws and ensure compliance Objective 3.2: Detect violations and promote compliance Starl Dale FY 2022 Completion Date September 2026 Note This project is being conducted over multiple fiscal years. Purpose and brief description: The Office of Enforcement and Compliance Assurance (OECA), Office of Water (OW), and the Drinking Water Systems Out of Compliance learning priority workgroup are assessing drinking water data reported to EPA to determine whether it accurately measures national compliance and substantiates EPA policy decisions; considering noncompliance root causes and corresponding technical/managerial/financial (TMF) factors; and testing efficacy of technical assistance, enforcement, and state oversight. The assessments, once complete, will identify key water system characteristics for which EPA and states should focus its policies and the most effective way to apply compliance assurance tools for increasing compliance in the drinking water program. Through FY 2025, OECA will continue evaluations and other empirical analyses for Question 2 (root causes of noncompliance), Question 3 (efficacy of enforcement on compliance), and Question 5 (Oversight). In FY 2023, EPA continued its work on Question 1 (data availability and reliability), continued its work on Question 2 (root cause of noncompliance in Public Water Systems (PWSs)), and initiated work to evaluate Question 3 (how and under what conditions do inspections and enforcement help water systems achieve compliance). Programmatic or policy decisions this activity will inform: Applying compliance assurance tools to effectively increase drinking water compliance rates. Questions to be addressed: Does increased use of compliance assurance tools (inspections and enforcement) improve system compliance, and if so under what circumstances? How can EPA determine if a system has the TMF capacity to provide safe water on a continuous basis to its customers? Methodological and analytical approach Data collection methods: EPA anticipates using several different tools for the evaluation of Questions 3 (Efficacy of enforcement), 4 (TMF), and 5 (EPA oversight) including survey instruments, literature reviews, data mining, and advanced statistical analysis such as machine learning and other regression approaches. Data sets: For evaluation of potential technical, managerial, and financial metrics (Question 4) the Agency anticipates needing to pull from various places such as federal databases at EPA (SDWIS), Department of Commerce Census Bureau, and USD A Rural Utilities Service (RUS) 17 ------- loan program data and information gleaned from the State Revolving Fund work, state Capacity Development annual reports, and sanitary survey checklists. Analytic approaches: Question 2 Root Cause Analysis: Preliminary analyses were conducted in FY 2023. Predictive tools developed by regions and/or states that were able to identify systems of risk of noncompliance in tests included in common the following systems characteristics: sources of water, financial questions, system size, presence of violations and significant deficiencies, presence of certified operators, source water quality, and presence of management plan. These analyses will continue and may be expanded upon in FY 2024. The expanded analysis could include looking further into drinking water systems "defying the odds," systems which despite their predicted noncompliance status have continued to remain in compliance, to determine if EPA can garner further insight into best practices of system compliance, as well as what managerial structures and other factors may be influencing the degree to which each factor effects system compliance/ noncompliance. Question 3 on Enforcement and Inspection Efficacy: The Agency plans to empirically test the impact of increased use of compliance monitoring inspections. This priority question complements the Drinking Water National Enforcement Compliance Initiative (NECI). While the Agency determined it could not plan inspections such that those activities could form the basis of a prospective study to inform the overall evaluation process, the Agency is considering a retroactive analysis of inspections already completed. Additionally, the use of OECA's Enforcement and Compliance History Online database will be used to do a retrospective analysis of enforcement activity. Tools and/or equipment: Statistical software Anticipated challenges and proposed solutions: Data availability may slow and/or limit progress on analytical activities that need to be carried out to conduct planned evaluations and other empirical studies. For example, the volume of compliance assurance work may be too low to support methodologies that use a randomization approach to Question 3 (efficacy of enforcement). States and water systems may not agree to participate in a survey study to identify attitudes on enforcement actions. Since the Agency does not collect TMF information in a consistent format, there is no national data set on these system characteristics, insufficient TMF data could limit our ability to identify effective metrics for TMF capacity. Ongoing work to modernize SDWIS should address some of these issues. Dissemination of findings: Final evaluation reports and other empirical analyses for this learning priority area will be made available on EPA"s evaluation website. Quarterly data reports are shared publicly via the SDWIS FED Data Warehouse. 18 ------- Priority Area Expanding EPA's Toolkit of Air Benefits Assessment Methodologies and Practices l.ead Office Office of Air and Radiation l ink lo KPA Strategic Plan Goal 4: Ensure clean and healthy air for all communities Objective 4.1: Improve air quality and reduce localized pollution and health impacts Start Date Subject to funding Completion Dale Subject to funding Note This project is subject to funding availability and will be conducted over multiple fiscal years. Purpose and brief description: EPA uses well-established methods for estimating the health benefits associated with reductions in criteria and air toxic pollutants. However, as noted by scientific bodies including the U.S. EPA Science Advisory Board (SAB)1, opportunities exist for EPA to improve its approach for quantifying the number and economic value of air pollution- related health effects; this includes estimating benefits that EPA does not currently quantify and monetize. Question(s) to be addressed: What are the health benefits of reducing human exposures to air pollutants not currently quantified, particularly those related to hazardous air pollutants (HAPs)? What are the health benefits of reducing the risk of air pollution-related effects that are challenging to quantify but nonetheless important to the exposed populations? What are the benefits of health outcomes that cannot yet be valued using Willingness-to-Pay or other measures of economic value? How can EPA account for sequelae and the progression of disease when quantifying benefits? Methodological and analytical approach Data collection methods: EPA will use well-established methods for estimating the health benefits associated with reductions in criteria and air toxic pollutants, including the use of the newly revised cloud-based version of the environmental Benefits Mapping and Analysis Program - Community Edition (Benft Data sets: EPA will use economic and health datasets providing information on the incidence of adverse health effects, novel health outcomes not previously quantified, and health care expenditures. Such datasets may include those published by the U.S. Small-Area Life Expectancy Estimates Project (USA-LEEP), the Healthcare Cost and Utilization Project 1 National Research Council. 2002. Estimating the Public Health Benefits of Proposed Air Pollution Regulations. Washington, DC: The National Academies Press, https://doi.org/10.1.7226/1.051.1. National Research Council. 2008. Estimating Mortality Risk Reduction and Economic Benefits from Controlling Ozone Air Pollution. Washington, DC: The National Academies Press. https://doi.org/10.1.7226/1.21.98. 19 ------- (HCUP), the U.S. Census and data reported in epidemiologic studies. Information reported in publicly available datasets (USA-LEEP, HCUP, Census) must be adapted for use in EPA health benefits analyses. For example, USA-LEEP reports life tables, but death rates must first be calculated from these tables before they may be used in a health benefits analysis. EPA will separately draw upon results reported in peer-reviewed epidemiologic studies (e.g., Odds Ratios and Hazard Ratios). Analytic approaches: When adapting data for use in EPA health benefits analyses, relatively simple calculations will be performed to quantify death rates and the Cost of Illness (COI) for adverse effects. When extracting Odds Ratios and Hazard Ratios from published epidemiologic studies, EPA commonly converts these measures of association to a beta coefficient. Tools and/or equipment: EPA will use existing agency tools, including the cloud-based version of the environmental Benefits Mapping and Analysis Program-Community Edition (BenMAP-CE). Anticipated challenges and proposed solutions: Addressing questions of the scope and complexity of this project will require significant contract resources and additional FTE (in particular, economists, biostatisticians, and air pollution epidemiologists). Dissemination of findings: EPA anticipates working collaboratively with NAS in developing this project. Any NAS reports will be disseminated by the NAS, although EPA will provide links to those reports through EPA's website, as appropriate. Information and any findings also will be shared with EPA staff and management through other venues (e.g., meetings, presentations, etc.). Priority Area Workforce Planning Lead Office Office of Mission Support Link lo I.PA Strategic Plan Cross-Agency Strategy 3: Advance EPA's organizational excellence and workforce equity Start Dale IV 2023 Completion Dale September 2025 Note This project is being conducted over multiple fiscal years. Purpose and description: EPA identified Human Capital Management as an Enterprise Risk due to the high number of staff eligible for retirement and EPA's aging workforce. The Workforce learning priority area in EPA's Learning Agenda will develop an evidence-based roadmap for how EPA can ensure it has employees with the competencies needed to achieve its mission now and in the future. It also will help determine the overall processes required to cultivate and manage the workforce, while anticipating internal and external changes, and continuously maximizing the efficiency and effectiveness of the Agency's workforce. 20 ------- Programmatic or policy decisions this activity will Inform: Near and long-term strategies to attract, recruit, train and retain a diverse and effective workforce. Question(s) this activity will address: What key factors contribute to EPA's organizational health and how do those metrics impact the best strategies to attract, recruit, train and retain a diverse workforce? What makes people stay in the Agency long-term? Methodological and analytical approach Data collection methods and data sets: EPA has various data sets and dashboards to capture employee demographic, hiring and attrition data. EPA will continue compiling information from these and other sources to create an even more robust body of evidence aligned with OMB's new Organizational Health and Organizational Performance initiative (M-23-15). This data will greatly enhance our understanding of what are the best strategies to attract, recruit, train and retain a diverse workforce and what makes people stay in the Agency long-term. EPA's current measures of organizational health include, among others, metrics covering recruitment and staffing, well-being, learning and development and succession management. Examining recruitment metrics, such as percent of positions available for entry-level talent and number of entry level positions hired, will provide background on how EPA is able to attract and recruit a diverse workforce. Other metrics, such as the Employee Viewpoint Survey, internal pulse surveys on work-life flexibility and exit survey data, will support understanding of EPA's ability to attract and retain a diverse workforce. Analytic approach: Implementing recommendations in a forthcoming white paper focused on preventing burnout prepared by the Office of Human Resources also will support EPA's efforts to bolster the retention, as well as wellbeing, of the workforce. EPA will examine a variety of training metrics including hours of discretionary training completed, investment in training dollars and participation in leadership training programs such as the Leaders and Learners Program. Examining these metrics will supply a foundation for understanding employee development and its possible effects on retaining the Agency's diverse workforce. Additionally, EPA will analyze and compile regional and program office succession management plans completed over FY 2023 into a comprehensive agency succession management plan. This will allow EPA to align agency strategy with implementation, to maintain EPA's most critical positions, enhancing our understanding of critical skills and EPA's overarching succession management needs. Anticipated challenges and proposed solutions: There might be low participation among stakeholders in the assessment and analysis of the workforce priority questions. This possible challenge is being mitigated by continuing to enlist the buy-in and support of senior leaders and other key stakeholders to help promote the importance of our processes prior to their start and keeping in constant contact with those stakeholders during the evaluation and analysis process. Dissemination of findings: The identified workforce activities are considered key components of management's strategic decision-making process; findings will be shared consistent with 21 ------- requirements related to privileged information. It is anticipated relevant results will be shared with internal stakeholders, including senior leaders and EPA's Human Resource Officer/Program Management Officer community. Aggregate information on findings might be shared with other federal agencies and/or publicly. Priority Area Grant Commitments Met Lead Office Office of the Administrator and Office of the Chief Financial Officer Link lo KPA Strategic Plan Cross-Agency Strategy 4: Strengthen tribal, state, and local partnerships and enhance engagement Start Dale IV 2021 Completion Dale September 2025 Note This project is being conducted over multiple fiscal years. Purpose and brief description: Grant Commitments Met is one of the Learning Priorities in the amine Agenda. EPA awards over $5 billion in annual funding to grants and other assistance agreements. New agency funding provided by the American Rescue Plan.2 the Bipartisan Infrastructure Law.3 and Inflation Reduction Act4 to fund grants and other assistance agreements underscores the importance of this Learning Priority.5 EPA helps to protect human health and the environment through these grants and the work of its grantees. The management and tracking of the individual grant awards are dispersed amongst staff at EPA headquarters and EPA's ten regional offices, which makes tracking results at the national level challenging. The Grant Commitments Met work is guided by the overarching learning question: How can EPA assess the extent to which commitments achieve the intended environmental and/or human health results and identify possible next steps in establishing a comprehensive grant reporting system? In the initial phase (Year 1 / FY 2021) of work, EPA addressed the question: How do EPA's existing grant award and reporting systems identify and track grant commitments? EPA organized an extensive survey that gathered 462 responses from grant programs across the Agency. The survey responses were analyzed to identify what data (e.g., outputs and outcomes) are being collected and how programs are reporting on grant activities across EPA. Year 1 also included a request for National Program Managers (NPMs) to provide background information on EPA's grant programs. EPA analyzed the survey responses and other documents to identify what data 2 H.R. 1319: American Rescue Plan Act of 2021. 3 H.R.3684: Infrastructure Investment and Jobs Act. 4 H.R. 5 375: Inflation Reduction Act of 2022. 5 The American Rescue Plan, Bipartisan Infrastructure Law, and Inflation Reduction Act provide around $100 million, $60.89 billion, and $350 million in additional EPA funding, respectively, for a total of around $61.34 billion in additional funding. See https://www.epa.gov/arp/about-epas-american-rescue-plan-arp-funding. https://www.epa.gov/infrastructure/explore-epas- bipartisan-infrastructure-law-funding-allocations. https://www.epa.gov/inflation-reduction-act/inflation-reduction-act-programs- fight-climate-change-reducing-embodied. accessed January 23, 2023. 22 ------- grant programs collect and how programs report on activities across EPA. The effort culminated in the Year 1 Report, published in September 2022. In the second year of the project (Year 2 / FY 2022), EPA addressed the question: What EPA practices and tools (1) effectively track grantee progress towards meeting workplan grant commitments including outputs and outcomes, and/or (2) support communication of national program level outputs and outcomes? Year 2 data efforts included approximately 30 in-depth interviews and some additional analysis of data previously collected in the Year 1 survey. Grant programs were selected with pre-defined considerations for individual or small group interviews with project officers or NPMs. This process built upon previous efforts as an in-depth study of a smaller number of programs to understand what the data can tell us about the effectiveness of EPA grant programs. The effort resulted in a r 2 Report, published in March 2023. In the third year of the project (Year 3 / FY 2023), EPA addressed the question: What could EPA do to prepare grant programs to report on consistently defined outputs and outcomes? To address this question, EPA developed draft standard agencywide definitions for outputs, a potential list of standard behavioral change outcomes and environmental human health outcomes (including climate and equity related metrics), and standard approaches for collecting output and outcome data. To accomplish this, EPA relied on the Strategic Plan, additional analysis of survey data and national program documents collected in Year 1, Year 2 interview data, and feedback from an advisory group comprised of key EPA staff including representatives from all ten region and all program offices. This culminated in a list for use in a pilot study, in the fourth year of the project (Year 4/ FY 2024) with four EPA grant programs to test the application of the definitions and approaches. In the fifth year of the project (Year 5/ FY 2025), EPA will address the question: To what extent do the EPA's grant programs have measures that support the reporting of intended results? EPA expects to use a methodology that relies on document review, and analysis of quantitative and qualitative data focused on a subset of agency grant programs to review for environmental, human health, or other priority strategic programmatic outcomes. EPA expects to use the cumulative information from the studies and pilots conducted from FY 2021 through FY 2024 in addition to FY 2025 analysis to address the overarching learning agenda question and to inform executive- level decisions about grants management, including improvement to processes, guidance and implementation of data and reporting. Programmatic or policy decisions this activity will inform: Practices and tools to effectively assess the extent to which commitments achieve the intended environmental and/or human health results and identify possible next steps in establishing a comprehensive grant reporting system. Outcomes from the first three years of this work will inform the Agency's overarching efforts to improve enterprise-wide grant management and reporting. Questions to be addressed: The Grant Commitments Met work is guided by the overarching learning question (in the EPA. Learning Agenda): How can EPA assess the extent to which commitments achieve the intended environmental and/or human health results and identify possible next steps in establishing a comprehensive grant reporting system? 23 ------- Progress, results, and interim findings: The Year 1 and Year 2 reports are currently available on EPA's website. Methodological and analytical approach: EPA expects to use a methodology that relies on document review, and analysis of quantitative and qualitative data focused on a subset of agency grant programs to review for environmental, human health, or other priority strategic programmatic outcomes. EPA plans to use the cumulative information from the studies and pilots conducted from FY 2021 through FY 2024 in addition to FY 2025 analysis to address the overarching learning agenda question "How can EPA assess the extent to which commitments achieve the intended environmental and/or human health results and identify possible next steps in establishing a comprehensive grant reporting system?" and to inform executive-level decisions about grants management, including improvements to processes, guidance, and implementation of data and reporting. Anticipated challenges and proposed solutions: Success depends on high stakeholder engagement and participation, including that of regional and NPM staff and management. EPA will address these challenges by relying on a group of regional and NPM points of contact and leveraging access to senior leadership calls. It will be challenging to keep up with the rapidly changing landscape regarding grant funding at EPA. Grant programs at EPA continue to expand in size and number. The high visibility of this additional funding further highlights the importance of accountability in grant reporting. Partnerships supporting this evidence-building effort: EPA will continue to engage with and inform states and tribes of EPA efforts through ECOS, the e-Enterprise Leadership Council (EELC), and other appropriate fora. Dissemination of findings: All final reports for the Grant Commitments Met learning priority work can be found on ience Act website. The Year 1 and Year 2 reports are currently available. 24 ------- Office of the Administrator Title Estimating the Social Cost of Greenhouse Gases (SC-GHG) Lead Office Office of the Administrator l ink lo l l\\ Strategic Plan Goal 1: Tackle the climate crisis. Objective 1.1: Reduce emissions that cause climate change. Slaii Dale January 2024 Completion Dale December 2028 Note This assessment is being conducted over multiple fiscal years. Purpose and brief description: A robust and scientifically founded assessment of the positive and negative impacts that an action can be expected to have on society facilitates evidence-based policy making. Estimates of the social cost of carbon (SC-C02), social cost of methane (SC-CH4), and social cost of nitrous oxide (SC-N20) allow analysts to incorporate the net social benefits of reducing emissions of each of these greenhouse gases, in benefit-cost analysis, and when appropriate, in decision making and other contexts. Collectively, these values are referred to as the "social cost of greenhouse gases" (SC-GHG). The SC-GHG is the monetary value of the future stream of net damages associated with adding one ton of that GHG to the atmosphere in a given year. The SC-GHG, therefore, also reflects the societal net benefit of reducing emissions of the gas by one ton. The academic literature has published estimates of the social cost of carbon and other GHGs since at least the early 1990s. As early as 2002 researchers began conducting reviews that combined lines of evidence across early SC-C02 estimates (Clarkson and Deyes 2002). The EPA began regularly incorporating SC-C02 estimates in regulatory impact analyses following a 2008 court ruling in which an agency was ordered to consider the SC-C02 in the rulemaking process. The SC-C02 estimates initially presented in EPA analyses in 2008 and early 2009 were derived from the academic literature. Beginning in September 2009, EPA's regulatory impact analyses applied SC-C02 estimates that were developed through a U.S. Government interagency working group (IWG) process, supported by EPA analysis. In January 2017, the National Academies released a report, Valuing Climate Damages: Updating Estimation of the Social Cost of Carbon Dioxide, that recommended specific criteria for future updates to the SC-C02 estimates, a modeling framework to satisfy the specified criteria, and research needs pertaining to various components of the estimation process (National Academies 2017). In the regulatory impact analysis of EPA's November 2022 Supplemental Notice of Proposed Rulemaking, "Standards of Performance for New, Reconstructed, and Modified Sources and Emissions Guidelines for Existing Sources: Oil and Natural Gas Sector Climate Review," in addition to using the current recommended interim SC-GHG estimates from the IWG process, EPA included a sensitivity analysis of the climate benefits of the proposed rule using a new set of SC-GHG estimates. These new estimates incorporate recent research addressing near term 25 ------- recommendations in the National Academies report (2017). EPA solicited public comment on the sensitivity analysis and the external review draft of the accompanying technical report, "Report on the Social Cost of Greenhouse Gases: Estimates Incorporating Recent Scientific Advances," that explains the methodology underlying the new set of SC-GHG estimates, in the docket for the proposed Oil and Gas rule. EPA also conducted an external peer review of the report. As noted in EPA's technical report containing the updated SC-GHG estimates numerous categories of climate change damages are not currently quantified. The EPA is committed to expanding lines of evidence, including more robust methodologies for estimating the magnitude of the various direct and indirect damages from GHG emissions and addressing long term recommendations from the National Academies (2017), that could further improve SC-GHG estimation going forward. In FY 2025, EPA plan to continue its efforts to develop and implement new damage categories and modeling improvements that will allow for more fulsome estimates of the SC-GHG in future updates. Question to be addressed: What are the benefits of incremental reductions in GHG emissions? Methodological and Analytical approach Data collection methods: Literature reviews, acquisition and processing of administrative data, and modeling. Data sets: The specific data set used will depend on the damage categories and modeling improvements selected. Analytic approach: The specific analytic approaches will depend on the damage categories and modeling improvements selected but are likely to include econometric analysis and structural modeling. Tools and/or equipment: The specific tools will depend on the damage categories and modeling improvements selected but are likely to include the R, python, and julia programming languages consistent with current tools used to develop the SC-GHG estimates. Anticipated challenges and proposed solutions: As is usual in scientific research, there are always some risks. However, EPA has a rich experience leading the Federal government's efforts on the SC-GHG, and EPA knows the data and modeling challenges that need to be overcome to develop more complete estimates of the SC-GHG. We have sought to mitigate those risks through engagement with a robust group of experts with opportunities for cross-fertilization and scientific dialogue through the 2017 National Academies report and recent SC-GHG peer review that provide thoughtful and reasoned direction for future research and advancements. Dissemination of findings: The technical report describing the methodology and SC-GHG estimates, along file replication files and source code, are posted on EPA's website. 26 ------- Office of Air and Radiation Tille Inventory of U.S. Greenhouse Gas Emissions and Sinks Lead Office Office of Air and Radiation l ink lo KIW S(rale«ic Plan Goal 1: Tackle the climate crisis Objective 1.1: Reduce emissions that cause climate change Si art Dale May 2024 Completion Dale April 2025 Nole This project is conducted each fiscal year. The start and completion dates indicate the timeframe for the annual report. Purpose and brief description: EPA has prepared the official Inventory of U.S. Greenhouse Gas Emissions and Sinks since the early 1990s. This annual report provides a comprehensive accounting of total greenhouse gas (GHG) emissions from all man-made sources in the United States over time. The gases covered by the Inventory include carbon dioxide, methane, nitrous oxide, hydrofluorocarbons, perfluorocarbons, sulfur hexafluoride, and nitrogen trifluoride. The Inventory also calculates carbon dioxide removal from the atmosphere by "sinks" (e.g., through the uptake of carbon and storage in forests, vegetation, and soils) from management of lands in their current use and as lands are converted to other uses. The national greenhouse gas inventory is submitted to the United Nations in accordance with the Framework Convention on Climate Change. Starting in 2022, EPA also has released the Inventory of U.S. Greenhouse Gas Emissions and Sinks by State (hereafter referenced as the Inventory), which provides state-by-state data consistent with the national greenhouse gas inventory and with international standards. As with the national inventory, the state-level greenhouse gas inventory provides the latest annual data and will be updated each year. Question(s) to be addressed: How does EPA provide comprehensive accounting of total greenhouse gas emissions from all man-made sources in the United States? How does data regarding national GHG emissions and sinks contribute to discussions regarding climate change? How does the Inventory support U.S. obligations to the United Nations Framework Convention on Climate Change? Methodological and analytical approach Data collection methods: This inventory adheres to both: (1) a comprehensive and detailed set of methodologies for estimating sources and sinks of anthropogenic greenhouse gases; and (2) a common and consistent format that enables Parties to the United Nations Framework Convention on Climate Change (UNFCCC) to compare the relative contribution of different emission sources and greenhouse gases to climate change. 27 ------- Data sets: In following the UNFCCC requirement under Article 4.1 and related decisions to develop and submit annual national greenhouse gas emission inventories, the emissions and sink categories are calculated using internationally accepted methods provided by the Intergovernmental Panel on Climate Change (IPCC) in the 2006 IPCC Guidelines for National Greenhouse Gas Inventories and where appropriate, its supplements and refinements. In applying methods, the Inventory makes use of data submitted to EPA's Greenhouse Gas Reporting Program (GHGRP) as well as data from other federal agencies, such as fuel consumption data published in the Energy Information Administration's (EIA) Monthly Energy Review and the U.S. Department of Defense data on military fuel consumption and use of bunker fuels. EPA collects greenhouse gas emissions data from individual facilities and suppliers of certain fossil fuels and industrial gases through its Greenhouse Gas Reporting Program (GHGRP), which is complementary to the U.S. Inventory. The GHGRP requires reporting by over 8,000 sources or suppliers in 41 industrial categories and applies to direct greenhouse gas emitters, fossil fuel suppliers, industrial gas suppliers, and facilities that inject carbon dioxide (CO2) underground for sequestration or other reasons. Annual reporting is at the facility level, except for certain suppliers of fossil fuels and industrial greenhouse gases, with a threshold of 25,000 metric tons or more of CO2 equivalent per year. Methodologies used in EPA's GHGRP are consistent with the 2006 IPCC Guidelines however, it does not provide full coverage of total annual U.S. greenhouse gas emissions and sinks (e.g., the GHGRP excludes emissions from the agricultural, land use, and forestry sectors), yet it does provide an important input to the calculations of national-level emissions in the Inventory. The GHGRP dataset provides annual emissions information, annual information such as activity data and emission factors that can improve and refine national emission estimates, as well as trends over time. GHGRP data also allows EPA to disaggregate national inventory estimates in new ways that can highlight differences across regions and sub-categories of emissions, along with enhancing application of Quality Assurance/Quality Control (QA/QC) procedures and assessments of uncertainties. Further, the Inventory also makes use of data from research studies and trade publications as described in detail within the report. Analytic approaches: Emissions and sink categories are calculated using internationally accepted methods provided by the IPCC using the 2006 IPCC Guidelines. Additionally, under this international agreement, the calculated emissions and removals in a given year for the United States are presented in a common manner in line with the UNFCCC reporting guidelines for the reporting of inventories. The use of consistent methods to calculate emissions and removals by all nations providing their inventories to the UNFCCC ensures that these reports are comparable. The presentation of emissions and removals provided in this Inventory does not preclude alternative examinations, but rather this Inventory presents emissions and removals in a common format consistent with how countries are to report inventories under the UNFCCC. Tools and/or equipment: EPA will use existing data collection methodologies and the GHGRP. 28 ------- Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major challenges in completing this project. Dissemination of findings: The findings are published annually on EPA's website in report format. Data from each report is made available through the GHG Data Explorer and supplemental CSV files. Future data and reporting will be published on the EPA website. Title Climate Change Indicators in the United States Lead Office Office of Air and Radiation l ink lo KPA Strategic Plan Goal 1: Tackle the climate crisis Objective 1.1: Reduce emissions that cause climate change Start Dale October 2024 Completion Dale September 2025 Note This project is conducted each fiscal year. Purpose and brief description: The EPA's Climate Change Indicators in the United States, was created with the primary goal of informing readers' understanding of climate change. It also is designed to be useful for the public, scientists, analysts, decision-makers, educators, and others who can use climate change indicators as a tool for communicating climate change science. EPA partners with more than 50 data contributors from various government agencies, academic institutions, and other organizations to compile a key set of indicators related to the causes and effects of climate change. These indicators also provide important input to the quadrennial National Climate Assessment and other efforts to understand and track the science and impacts of climate change. Question(s) to be addressed: How do the indicators help to increase understanding of the impacts of climate change? How do the indicators help to track trends? How can the indicators be used to help inform science-based decision making in the Office of Air and Radiation? Methodological and analytical approach Data collection methods: EPA partners with more than 50 data contributors from various U.S. and international government agencies, academic institutions, and other organizations to compile these key indicators of climate change. Data sets: EPA chooses indicators that meet a set of 10 criteria that consider data quality, transparency of analytical methods, and relevance to climate change. Based on the availability of these data, some indicators present a single measure or variable while others have multiple 29 ------- measures, reflecting different data sources or different ways to group, characterize, or zoom in on the data. The criteria EPA uses to select indicators are: 1. Trends over time: Data are available to show trends over time. Ideally, these data will be long-term, covering enough years to support climatically relevant conclusions. Data collection must be comparable across time and space. Indicator trends have appropriate resolution for the data type. 2. Actual observations: The data consist of actual measurements (observations) that are representative of the target population. 3. Broad geographic coverage: Indicator data are national in scale or have national significance that are representative of the region/area. 4. Peer-reviewed data: The quality of underlying source data sound, credible, reliable, and have been peer-reviewed and published. 5. Uncertainty: Information on sources of uncertainty is available and evaluations of the indicators have been made that clearly address both variability and limitations. 6. Usefulness: The indicator informs issues of national importance, addresses issues important to human or natural systems, and complements existing indicators. 7. Connection to climate change: The relationship between the indicator and climate change is supported by published, peer-reviewed science and data. A climate signal is evident among stressors, even if the indicator itself does not yet show a climate signal and the relationship to climate change is easily explained. 8. Transparency, reproducibility, and objectivity: The data and analysis are scientifically objective, methods are transparent, and biases, if known, are documented, minimal, or judged to be reasonable. 9. Understandability by the public: The data provide a straightforward depiction of observations and are understandable to the average reader. 10. Feasibility to construct: The indicator can be constructed or reproduced within a reasonable timeframe, and data sources allow for routine updates of the indicator. Analytic approaches: EPA ensures the scientific integrity of the climate change indicators through a rigorous development process. For every indicator, EPA also develops technical documentation that describes the data sources, analytical methods used, and ensures the information is accessible each indicator. Tools and/or equipment: Existing data is pulled from 50 data contributors to compile the key indicators previously identified. Anticipated challenges and proposed solutions: At this time, the Office of Air and Radiation does not anticipate any major challenges in completing this project. Dissemination of findings: These indicators characterize observed changes from long-term records related to the causes and effects of climate change; the significance of these changes; and their possible consequences for people, the environment, and society. Examples of indicators include: Heat waves: trends in the number of heat waves per year (frequency); the average length of heat waves in days (duration); the number of days between the first and last heat wave of the 30 ------- year (season length); and how hot the heat waves were, compared with the local temperature threshold for defining a heat wave (intensity). Coastal flooding: tracks periodic inundation based on measurements from tide gauges at locations along U.S. coasts. Glaciers: examines the balance between snow accumulation and melting in glaciers, and it describes how glaciers in the United States and around the world have changed over time. Growing season: looks at the impact of temperature on the length of the growing season in the contiguous 48 states, as well as trends in the timing of spring and fall frosts. Wildfire ', tracks four aspects of wildfires over time: the total number of fires (frequency), the total land area burned (extent), the degree of damage that fires cause to the landscape (severity), and the acreage burned by fires starting in each month of the year (seasonal patterns). Future updates will be posted to the EPA website: https:/Avww.epa.fzov/climate-iridicators. Til le Power Sector Programs - Progress Report Lead Office Office of Air and Radiation Link to EPA Strategic Plan Goal 4: Ensure clean and healthy air for all communities Objective 4.1: Improve air quality and reduce localized pollution and health impacts Start Dale October 2023 ( 0111 pletion Dale September 2024 Note This project is being conducted over multiple fiscal years, with an update each year. Purpose and brief description: Under the Clean Air Act, EPA implements regulations to reduce emissions from power plants, including the Acid Rain Program (ARP), the Cross-State Air Pollution Rule (CSAPR), the CSAPR Update, the Revised CSAPR Update, and the Mercury and Air Toxics Standards (MATS). These programs require fossil fuel-fired electric generating units to reduce emissions of sulfur dioxide (SO2), nitrogen oxides (NOx), and hazardous air pollutants including mercury (Hg) to protect human health and the environment. This reporting year marks the seventh year of CSAPR implementation, the fifth year of the CSAPR Update implementation, the first year of Revised CSPAR Update implementation, the 27th year of the ARP, and the fifth year of MATS implementation. This report summarizes annual progress through 2021, highlighting data that EPA systematically collects on emissions for all five programs and on compliance for the ARP and CSAPR. Commitment to transparency and data availability is a hallmark of these programs and a cornerstone of their success. Question(s) to be addressed: This annual activity assesses implementation of multiple regulations to reduce air pollution from power plants. Specific questions of interest include: Have the regulations met their emission reduction goals? 31 ------- What is the compliance record of air pollution sources controlled under these regulations? What is the air quality and environmental response of implementing these regulations? Methodological and analytical approach Data collection methods: EPA's Clean Air Markets Division (CAMP) systematically collects emissions data for the Acid Rain Program. Cross-State Air Pollution Rule (CSAPR), CSAPR Update, and the Mercury and tcs Standards (MATS). Transparency and data availability are a hallmark of these programs, and a cornerstone of their success. CAMD provides an array of reports, resources, and tools, to access and understand these data and environmental results of emission reductions at varying levels of detail. Data sets: Accurate and consistent emissions monitoring data are critical to ensure program results and accountability. Most emissions from affected sources are measured by continuous emission monitoring systems (CEMS). Analytic approaches: Compliance for the Acid Rain Program (ARP) and each of the Cross- State Air Pollution Rule (CSAPR) trading programs is assessed on an annual basis. Each regulated facility must hold an amount of allowances equal to or greater than its emissions for the relevant compliance period6 . Historically, these programs have had exceptionally high rates of compliance. This performance continued in 2021 as 100 percent of the facilities in each of these programs held sufficient allowances to cover their emission obligations. In contrast to the ARP and CSAPR, the Mercury and Air Toxics Standards (MATS) rule is issued under section 112 of the Clean Air Act and is not an emissions trading program. Tools and/or equipment: EPA will use existing tools for each program identified above. Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major challenges in completing this project. Dissemination of findings: Future information will be published on the EPA website. 6 These emissions trading programs also are known as "allowance trading programs" or "cap-and-trade" 32 programs. ------- Title Title V Permitting Program Reviews Lead Office Office of Air and Radiation l ink to i:r \ Strategic Plan (ioal 4: Ensure clean and healthy air for all communities Objective 4.1: Improve air quality and reduce localized pollution and health impacts Start Dale October 2024 Completion Dale September 2025 Note This project is conducted each fiscal year. Purpose and brief description: EPA periodically assesses state and local permitting programs, including the sufficiency of fees collected, under Title V of the Clean Air Act as part of its responsibility to oversee delegated and approved air permitting programs. Question(s) to be addressed: What are some good practices and areas of improvement in state and local permitting programs under Title V of the Clean Air Act? How can EPA help the permitting agencies improve their performance? Are fees collected sufficient to ensure effective operation of the Program? Methodological and analytical approach Data collection methods: In general, EPA uses a questionnaire to gather preliminary information, reviews files maintained on permits, conducts site visits, and follows up with the permitting program to clarify information in conducting a Title V program assessment. Data sets: EPA uses preliminary information gathered from questionnaires to conduct a Title V program assessment. This data is created and available to EPA. Analytic approaches: N/A Tools and/or equipment: N/A Anticipated challenges and proposed solutions: The Agency conducts these analyses annually and does not anticipate challenges. Dissemination of findings: The Title V Permit analyses are posted on EPA's website. Information and any findings also will be shared with appropriate EPA staff and management. 33 ------- Title Our Nation's Air: Status and Trends Through 2024 Lead Office Office of Air and Radiation Link lo LPA Slralegic Plan Goal 4: Ensure clean and healthy air for all communities Objective 4.1: Improve air quality and reduce localized pollution and health impacts Si art Dale October 2024 Completion Dale September 2025 Note This project is conducted each fiscal year. Purpose and brief description: EPA is committed to protecting public health and the environment by improving air quality and reducing air pollution. This annual report presents the trends in the nation's air quality and summarizes the detailed information found at EPA's Air Trends website and other air quality and emissions data. Question(s) to be addressed: Where are areas experiencing air quality above the national ambient air quality standards? Are these areas trending toward improving air quality? Methodological and analytical approach Data collection methods: EPA will use the National Emission Inventory (NEI) and Air Quality System (AQS) to gather data. Data sets: EPA pulls existing data from several sources to generate the report, such as the National Emission Inventory (NEI) and Air Quality System (AQS), both of which EPA created and can access. Analytic approaches: In addition to relying on existing publicly available analyses, this report will use trends analyses for air quality and emissions information. Tools and/or equipment: This report uses SAS and a variety of data visualization software. Anticipated challenges and proposed solutions: The Agency produces this report annually and does not anticipate challenges. This activity is contingent upon air quality data availability from state, local, and tribal air pollution control agencies. Dissemination of findings: EPA will share the results of these efforts on EPA's website, https://www.epa.gov/air-trends. 34 ------- Office of C hemical Safety and Pollution Prevention Title Reducing Use of Animals in Chemical Testing in FY 2025 Lead Office Office of Chemical Safety and Pollution Prevention l ink lo CPA S(rale«ic Plan Goal 7: Ensure safety of chemicals for people and the environment. Objective 7.1: Ensure chemical and pesticide safety. Si art Dale October 2024 Completion Dale September 2025 Nole This project builds on workshops and reports developed in FY 2022 and FY 2023. Purpose and brief description: The Office of Chemical Safety and Pollution Prevention (OCSPP) and the Office of Research and Development (ORD) have been world leaders in advancing the science for moving away from the use of animals for toxicity testing. In December 2021, EPA released the updated "New Approach Methods Work Plan: Reducing Use of Animals in Chemical Testing" which provides a workplan to develop metrics for reducing the use of mammalian laboratory animals in both research and for safety evaluations for pesticides and industrial chemicals. Additionally, the U.S. Government Accountability Office (GAO) released a report to Congress in 2019 recommending that Federal agencies develop metrics to assess the progress made toward reducing, refining, and replacing animal use in testing. EPA implemented activities and policies over the past several years that demonstrate significant reductions in the number of animals used in testing and saving resources for the Agency and stakeholders. In March 2021, in response to the GAO report, the Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) published its report entitled "Measuring U.S. Federal Agency Progress T ovvard Implementation of Alternative Methods in Toxicity Testing." Question(s) to be addressed: EPA has funded a report by the U.S. National Academies of Sciences, Engineering, and Medicine study that will assess the variability and relevance of existing mammalian toxicity tests and reviews frameworks for validation and establishing scientific confidence in testing methods. In FY 2022, two public workshops were held by the NAS in support of this work. In 2023, the NAS released its report, "Building Confidence in New Evidence Streams for Human Health Risk Assessment: Lessons Learned from Laboratory Mammalian Toxicity Tests." There are two additional milestones for FY 2025. EPA is nearing completion of a report of existing statutes, programmatic regulations, policies, and guidance that relate to vertebrate animal testing and the implementation and use of appropriate NAMs for regulatory purposes. As started in 2022, EPA will continue to provide progress and summary metrics on reducing vertebrate animal testing requests and use across ORD and OCSPP. 35 ------- Methodological and analytical approach Data collection methods: OCSPP tracks the reduction and replacement metrics through internal committees, primarily the Hazard and Science Policy Council (HASPOC) and the Chemistry and Acute Toxicology Science Advisory Council (CATSAC) and division-level processes. Data sets: For OPP, critical data sets are created by EPA using the number of waivers considered and recommended for through internal committees, such as HASPOC and CATSAC. The number of submissions for particular study types also are compiled through the division. Analytic approach: OCSPP is nearing completion of the development of a new process (including baseline ranges) that will provide the foundation for animal reduction metrics for TSCA-specific activities in this area. Anticipated challenges and proposed solutions: Under TSCA, there is no defined set of toxicology data requirements which makes establishing baselines difficult. Accelerating progress towards adopting new approach methods requires the availability of approaches that are "equal to or better than" the typically used animal studies. Other activities described in the updated 2021 workplan will address this challenge. Dissemination of findings: EPA efforts to reduce use of animals in chemical testing is reported in the Annual Reports on PRIA Implementation (https://www.epa.gov/pria-fees/annual-reports- pria-implementation). OPP publishes metrics on its website (https://www.epa.gov/pesticide- science-and-assessing-pesticide-risks/strategic-vision-adopting-new-approach). OPPT expects to begin publishing this information in 2024. 36 ------- Title Pesticide Registration Review in FY 2025 Lead Office Office of Chemical Safety and Pollution Prevention Link to I.IW Strategic Plan Goal 7: Ensure safety of chemicals for people and the environment. Objective 7.1: Ensure chemical and pesticide safety. Si art Dale October 2024 Completion Date September 2025 Note This project is conducted each fiscal year. Purpose and brief description: Review will assess the degree of progress and timely completion of docket openings, draft risk assessments, and case completions for the second cycle of pesticide registration review. Question(s) to be addressed: Whether OCSPP's suite of pesticide registration review performance measures and processes for meeting pesticide registration review statutory timeframes warrant further revision. Methodological and analytical approach Data collection method: Data will be collected from quarterly reports of registration review actions completed, and registration review action tracking databases maintained by the program. Data sets: Critical data sets include performance metric targets and results and any other data sets that could point to a need for operational improvements. Tools and analytical methods would not be needed for this exercise. Anticipated challenges and proposed solutions: OCSPP does not anticipate any major challenges in gathering performance data currently. Expert input will be brought to bear on any challenges and possibility that solutions will be needed. Dissemination of findings: Indicate whether the findings will be made publicly available on EPA.gov. The expectation is that EPA's program evaluation findings will be available to the public, in line with EPA's Policy on Evaluations and Other Evidence-Building Activities. If you anticipate not sharing the findings publicly, please explain your rationale. OCSPP intends to make performance results publicly available. Under GPRA, any measures considered external will be transmitted to OMB and the Congress and made public. OCSPP will publish quarterly updates to the pesticide registration review schedule (https://www.epa.eov/pesticide-reevaliiation/upcomine-reeistration-review-actions). 37 ------- Title IT Modernization of EPA pesticide tracking system in FY 2025 Lead Office Office of Chemical Safety and Pollution Prevention Link to I.IW Strategic Plan Goal 7: Ensure safety of chemicals for people and the environment. Objective 7.1: Ensure chemical and pesticide safety. Si art Dale October 2024 Completion Date September 2025 Note This project builds on activities conducted between April 2019 and September 2024. Purpose and brief description: In April 2019, EPA kicked off Phase 1 of a multi-year digital transformation to create a fully electronic workflow for EPA registration and reevaluation activities. This effort builds on the 2016 launch of the Pesticide Submission Portal, a secure, web- based portal in EPA's Central Data Exchange (CDX) environment through which the public can electronically submit applications for EPA evaluation. In early 2020, in advance of the launch of the new system, EPA developed performance metrics and established baselines of performance using the current agency systems for review of applications. These metrics will allow EPA to measure the impact of the digital transformation on meeting the targets and objectives described in the EPA Strategic Plan. Performance measures were developed addressing 1) timeliness of review, 2) efficiencies realized as a result of the transformation effort, and 3) employee engagement. In FY 2020, a pilot of the new system went live for one of the three regulatory divisions within OPP, as well as the Information Technology, and Resource Management Division (ITRMD) which in-processes all applications. In FY 2021, a second regulatory division in OPP entered the pilot. The pilot is specific to registration application workflows under the Pesticide Registration Improvement Act (PRIA) and its reauthorizations. Full expansion to all registering divisions and workflows will occur by the end of FY 2023, as well as some development to additional divisions in the Office of Pesticide Programs that support reevaluation regulatory activities. In FY 2024 and FY 2025 there will be expansion to the outward-facing aspects of the digital transformation effort, improving the ability of the regulated community, other stakeholders, partners, and the American public to directly engage with the regulatory and science efforts. Improvements to the front-end portal by which companies submit applications also will occur in FY 2024/25. Question(s) to be addressed: Potential for mission transformation through digitalization is enormous. Having a single system through which all data are captured, both for workflow and information needed for work, is a game changer. Managers will be able to see who is working on what task throughout their organizational unit while leaders will be able to see how all registrations and registration review cases are progressing and whether the overall trajectory is predictive of completion on time or not. Predictive algorithms will help determine where skills gaps lie so targeted hiring decision can be applied to remove bottlenecks. Employees will have access to all data they need to work on an assessment at their fingertips and won't have to go searching for data needed for work. 38 ------- Digital transformation is expected to improve employee job satisfaction significantly. By having access to quality information instantaneously available to assess the risk will enhance productivity and allow for a better work-life balance. Augmented intelligence tools being built into the new system will eventually automate administrative tasks allowing staff to focus on tasks more that bring a higher efficiency and rigor to the science. Surveys conducted one year after the launch of the pilot that included three divisions is already showing a significant savings of time and thereby a better work-life balance. Methodological and analytical approach Data collection method and datasets: Information from EPA's PRISM and OPPIN systems will allow EPA to establish baselines for how much time is spent at each stage of risk assessment and assess improvement in the overall review processes for registration and registration review cases. The Salesforce interface currently being piloted for antimicrobial and biopesticide applications will allow EPA to establish baselines for how much time is spent at each stage and assess improvement in review processes supporting new active ingredients registration determinations. The Employee Engagement metric will be tracked by evaluating results to specific questions and focus areas on the EPA Employee Viewpoint Survey and comparing responses from OPP staff before and after implementation of the IT-modernization effort. Analytic approach and tools: In addition, the augmented intelligence and advance data analytics within Salesforce will allow EPA to identify stages in the review process that present bottlenecks, allowing further system development and/or resource allocation to address identified concerns. Robotic Process Automation (RPA) will enable automation of many routine tasks allowing the scientists and regulatory specialists to focus on higher value work. Anticipated challenges and proposed solutions: OCSPP is currently awaiting award of the Mission Support IT Contract to continue work on the Digital Transformation. Current contracts supporting development and operations & Maintenance of systems expire in November thereby making the award of the new contract urgent. Office of Acquisition Services (OAS) is currently projecting an award date of September 15. Dissemination of findings: Process improvements relating to pesticide registration and registration review activities, as well as information technology improvements, are described annually in the PRIA annual report (https://www.epa.gov/pria-fees/annual-reports-pria- implementation). 39 ------- Title ESA Effects Determinations for Listed Species in FY 2025 Lead Office Office of Chemical Safety and Pollution Prevention Link to I.IW Strategic Plan Goal 7: Ensure safety of chemicals for people and the environment. Objective 7.1: Ensure chemical and pesticide safety. Si art Dale October 2024 Completion Date September 2025 Note This project is conducted every fiscal year. Purpose and brief description: The Endangered Species Act (ESA) requires that the actions of federal agencies do not jeopardize the continued existence of federally threatened or endangered species or destroy or adversely modify their critical habitat. EPA is developing a process to incorporate ESA determinations into its new active ingredient registration process and to work towards more routine considerations of ESA determinations for registration review decisions. EPA anticipates increasing ESA considerations into its registration and registration review decisions at an increasing frequency over the next 5 years. In FY 2022, EPA posted the ESA workplan- https://www.epa.gov/system/files/dociiments/2022-04/balancing-wildlife-protection-and- responsible-pesticide-use final.pdf - to provide to the public the framework for ESA implementation into pesticide regulatory activities Question(s) to be addressed: Whether OCSPP's suite of performance measures and processes for developing ESA effects determinations warrant further revision. Methodological and analytical approach Data collection method: EPA solicits input, data, and general comments from stakeholders and the general public on its ESA activities as they are developed and each time they are incorporated into a pesticide registration or registration review decision. Data set: Critical data sets include EPA workflow tracking systems and stand-alone reports on ESA-related risk assessment activity and label mitigation as well as public comments EPA receives on its ESA activities. Tools and analytical methods would not be needed for this exercise. Anticipated challenges and proposed solutions: Describe any anticipated challenges and how they will be addressed. Include discussion of challenges to making new tools or data developed publicly available where appropriate. Identify any other activities this activity is contingent on. At this time, OCSPP does not anticipate any major challenges in gathering performance data. Expert input will be brought to bear on any challenges and possibility that solutions will be needed. Dissemination of findings: OCSPP intends to make performance results publicly available. Under GPRA, any measures considered external will be transmitted to OMB and the Congress and made public. 40 ------- Title Safer Choice Consumer Survey in FY 2025 Lead Office Office of Chemical Safety and Pollution Prevention Link lo LPA Strategic Plan Goal 7: Ensure safety of chemicals for people and the environment. Objective 7.1: Promote Pollution Prevention - Encourage the adoption of pollution prevention and other stewardship practices that conserve natural resources, mitigate climate change, and promote environmental sustainability. Start Dale April 2023 Completion Dale March 2025 Note This project is conducted over multiple fiscal years, with repeated surveys. Purpose and brief description: As part of EPA's Strategic Plan, EPA will implement the Safer Choice Program and will conduct outreach to communicate the benefits of Safer Choice. To assess the effects of these outreach activities and the Program more broadly, the Agency will conduct an annual survey of consumers to determine the awareness and perception of Safer Choice-certified products. Data from this survey also will be used to support additional goals in EPA's Strategic Plan such as increasing the number of Safer Choice-certified products. OPPT/Safer Choice will conduct an annual survey of 2,000 consumers to assess their awareness and perceptions of Safer Choice-certified products. Data from this survey will help assess the implementation of Safer Choice outreach efforts and increase interest in product certification. Question(s) to be addressed: What are the awareness levels and consumer perception of the primary Safer Choice label, alternate labels (i.e., fragrance-free Safer Choice label) and Design for the Environment logo (used on antimicrobial products that meet the Safer Choice Standard)? How have OPPT/Safer Choice program implementation and outreach activities contributed to changes in awareness and perceptions of Safer Choice over time? Methodological and analytical approach Data collection method: Online survey of 2,000 consumers. Data set: Existing OPPT/Safer Choice Consumer Survey data, which has been created by and is available to EPA. Analytic approaches: Descriptive statistics (e.g., percentages) and potentially trend analysis with previous FY survey data. Anticipated challenges and proposed solutions: Though unlikely, there may be concerns with publishing the survey results, but OPPT/Safer Choice will work to address them as appropriate, for example by only including high-level data points. Dissemination of findings: Safer Choice plans to make findings publicly available on EPA.gov. 41 ------- Office of Hnforcement and Compliance Assurance Title Identifying interventions that are effective at overcoming the impediments to municipal compliance FY 2025 Lead Office Office of Enforcement and Compliance Assurance l ink lo I.IW S(rale«ic Plan (ioal 3: Improve compliance with the nation's environmental laws and hold \ iolators accountable Objective 3.2: Ensure high levels of compliance with federal environmental law s and regulations through effective compliance tools - including inspections, other monitoring activities, and technical assistance supported In evidence and advanced technologies Start Dale IV 2023 Completion Dale Through FY 2025 Note This project is being conducted over multiple fiscal years. Purpose and brief description: This project is a part of OECA's Compliance Learning Agenda (CLA) which collaborates with states, tribes, and academics to identify the most pressing programmatic questions, and create a venue for EPA, states, tribes, and territories to collaborate in the development of evidence-based enforcement tools and techniques that will ensure the biggest impact on environmental compliance. EPA has heard about causes of noncompliance for small municipal systems from many sources over some time. Through this research, EPA hopes to identify the root causes that lead to noncompliance and that also render agency interventions (enforcement, technical assistance, etc) unsuccessful at returning systems to compliance. We anticipate this effort to involve multiple research projects under both the NPDES and SDWA programs. Questions to be addressed: In addition to furthering the efforts of OECA's Compliance Learning Agenda, the results of this activity will be used to improve agency efforts and interventions to ensure that they are effective at returning systems to compliance. The following questions will be addressed: 1. What are the Root Causes of Municipal (Wastewater Treatment Plants and Drinking Water systems) Noncompliance that Can Render EPA and State Enforcement and Technical/Financial Assistance Efforts Unsuccessful? 2. Considering the root causes of municipal noncompliance, what are the impediments to compliance that prevent technical assistance/financial assistance/enforcement tools from being effective in producing compliance? 3. What Alternate or Supportive Interventions are effective in producing compliance? 4. What is the effectiveness of the application of various compliance tools to municipal noncompliance, e.g., enforcement actions, technical assistance, etc. in producing compliance - or improved compliance? 42 ------- Methodological and analytical approach Data collection methods: Methods used include data analysis, survey, and follow-up interviews. Data sets: The state/EPA inspection data, enforcement data, and state violation data from ECHO, ICIS-NPDES, SDWIS, and other government databases is created and available to EPA. Population and inequality data will be obtained from an external party and be made available to the Agency. The data from state associations, academic databases, and survey responses has or will be created by an external party who will make it available to the Agency. Analytic approach: Statistical analysis will be used, and EPA will continue to work with academic partners to uncover which, if any, other analytical method might be used on this project. Tools and/or equipment: We will continue to work with academic partners to uncover which, if any, tools and/or equipment will be use. Anticipated challenges and proposed solutions: 1. Effectiveness of enforcement (and other compliance tools) in producing compliance may vary state to state for various reasons. We will make every effort to account for this in the study. 2. There are multiple likely drivers of noncompliance and variations of the drivers of noncompliance between states. We will likely need a large study dataset to analyze the associations between these drivers of noncompliance and the effectiveness of enforcement actions to become evident. 3. There is uncertainty about ease of obtaining reliable information about the drivers of noncompliance for individual enforcement action. To help overcome this challenge, EPA has partnered with the E-Enterprise Leadership Council and have invited ECOS, states, and tribes to participate in the workgroup to complete learning agenda projects. Dissemination of findings: We anticipate making project findings public on EPA.gov. 43 ------- Office of Land and Kmergencv Management Title FY 2025 Redevelopment economics at federal facilities Lead Office Office of Land and Emergency Management l ink lo EPA Strategic Plan (ioal 6: Safeguard and revitalize communities Objective 6.1: Clean up and restore land for productive uses and healthy communities Start Date October 2024 Completion Dale January 2025 Note The Economic Analysis commenced in 2016 and is updated/expanded annually. Start and end dates are for expected FY 2025 activities. Purpose and brief description: Cleaning up contaminated sites at federal facilities can serve as a catalyst for economic growth and community revitalization. The Superfund Federal Facilities Program facilitates the redevelopment of federal facility sites across the country by assisting other federal agencies (OFAs) expedite activities related to CERCLA response actions, while protecting human health and the environment. Collaborative efforts among OF As; developers; and state, local, and tribal partners encourage restoration of sites. Since federal facility Superfund sites often encompass thousands of acres with buildings, roads, and other infrastructure, their effective and efficient cleanup and reuse can play a pivotal role in a community's economic growth. EPA has initiated efforts to collect economic data at a subset of federal facility Superfund sites. Question(s) to be addressed: The analysis will provide current, reliable business-related information for a subset of federal facility Superfund sites in reuse and continued use. What information can EPA provide about federal facility Superfund sites in reuse and continued use, including the variety of purposes that some innovative business owners and organizations reuse Superfund sites9 How these uses help economically revitalize communities near Superfund sites9 Methodological and analytical approach Data collection method: The FY 2025 Federal Facilities Superfund Economic Analysis is an update and expansion of research efforts in 2016, 2018, 2019, 2020, 2021 and 2022. These efforts provide current, reliable business-related information for a subset of federal facility Superfund sites in reuse and continued use. The research process uses the following methodology: Verification and/or update of economic information for previously identified site businesses. Discovery of new active businesses that may not have been operating previously at sites, or that may not have been identified previously, and collection of economic information for those newly identified site businesses. 44 ------- Discovery of previously identified site businesses that may have closed or moved off site. Quality control/quality assurance (QA/QC) review of economic data collected during the update. Data sets: Hoovers/Dun & Bradstreet (external party data set) is used to obtain data on businesses, jobs, and annual sales. ReferenceUSA (external party data set) is used to obtain data on businesses, jobs, and annual sales. Manta database (external party data set) is used to obtain data on businesses, jobs, and annual sales. The Bureau of Labor Statistics (BLS) Quarterly Census of Employment and Wages (external party data set) is used to estimate annual income based on the number of jobs identified at the business and the average weekly wage reported by BLS for each business's primary NAICS code and location. Analytic approach: The study estimates economic activity at federal facilities Superfund sites based on methodology developed by EPA's Superfund Redevelopment Program. Data on businesses, jobs and annual sales were obtained from Hoovers/Dun & Bradstreet, Reference Solutions, Manta.com and other published reports identified online. These databases and reports include data reported by businesses. Accordingly, some reported values might be underestimates or overestimates. In general, economic information gathered for sites in reuse is conservative, as it is not always possible to identify all businesses on site. Wage data are from the Bureau of Labor Statistics (BLS) Quarterly Census of Employment and Wages and are used to estimate annual income based on the number of jobs identified at the business and the average weekly wage reported by BLS for each business's primary NAICS code and location. Tools and/or equipment: Excel Anticipated challenges and proposed solutions: The Economic Analysis commenced in 2016 and is updated/expanded annually. The Economic Analysis is an established activity that provides valuable metrics for the Program and is expected to continue without challenges. Dissemination of findings: The summary of the results will be shared on Redevelopment Economics eral Facilities website. In addition, economic data are included in budget justifications to Congress and are used in general communication with other Federal agencies and the public. 45 ------- Title FY 2025 Redevelopment economics at remedial sites (non-federal facility) Lead Office Office of Land and Emergency Management l ink lo KPA Strategic Plan Goal 6: Safeguard and revitalize communities Objective 6.1: Clean up and restore land for productive uses and healthy communities Si art Dale October 2024 Completion Dale January 2025 Nole This project is conducted each fiscal year. Purpose and brief description: Cleaning up contaminated sites can serve as a catalyst for economic growth and community revitalization. The Superfund Redevelopment Program (SRP) facilitates the redevelopment of sites across the country while protecting human health and the environment. Collaborative efforts among state, local, and tribal partners, redevelopers, and other federal agency programs encourage restoration of sites. Since Superfund sites often encompass buildings, roads, and other infrastructure, their effective and efficient cleanup and reuse can play a pivotal role in a community's economic growth. EPA has initiated efforts to collect economic data at a subset of Superfund sites. Each year, the data collected is made available on EPA's webpages as part of the Redevelopment Economics at Superfund Sites StoryMap webpage and corresponding pages on National Beneficial Effects and related topics. EPA has created a Superfund Redevelopment Economics Notebook that provides a general overview of EPA's efforts to quantify some of the economic benefits associated with the cleanup and reuse of Superfund sites. In addition, Regional Economic Profiles summarize economic data collected for Superfund sites within an EPA region. They also highlight successes and put them in the context of aggregated data within the state and EPA region. Economic data are updated annually; regions receive a full regional economic profile or a data supplement to update the prior year's full regional economic profile on alternate years. Economic data are included in budget justifications to Congress and are used in general communication with key stakeholders and the public. Questions to be addressed: The analysis will provide current, reliable business-related information for a subset of Superfund sites in reuse and continued use: What information can EPA provide about Superfund sites in reuse and continued use, including the variety of purposes that some innovative business owners and organizations reuse Superfund sites? How does this use help economically revitalize communities near Superfund sites? 46 ------- Methodological and analytical approach Data collection methods: Each year, SRP collects the following types of economic information for site businesses: the names of businesses operating at sites, the number of people employed at site businesses, wage and income information, and annual business sales. During each update, referred to as the Annual National Economic Information Update, SRP gathers economic information from high-quality, online economic databases. Economic information also comes from site stakeholders and businesses, local media, and online resources. Wage values come from the U.S. Bureau of Labor Statistics' Quarterly Census of Employment and Wages. SRP uses those wage values to calculate estimated annual income for each site business based on North American Industrial Classification System (NAICS) codes. To identify new sites that may potentially support revenue-generating businesses, the annual update includes a review of sites in commercial, industrial, recreational, agricultural, and residential reuse, as well as sites in planned reuse. SRP uses the information from the Annual National Economic Information Update to track progress in returning sites to beneficial use and to respond to federal and Congressional information requests. At the end of each Annual National Economic Information Update, SRP compiles all site-level economic information and calculates the estimated beneficial effects of site reuse at the national level. In 2022, SRP gathered economic information for 671 sites in reuse. Data sets: EPA information on site reuse collected through Annual National Economic Information Update (it will be created by EPA). Dun & Bradstreet Hoovers platform (external party date). U.S. Bureau of Labor Statistics' Quarterly Census of Employment and Wages (external party data). Anticipated challenges and proposed solutions: The Economic Analysis commenced in 2011 and is updated/expanded annually. The Economic Analysis is an established activity that provides valuable metrics for the Program and is expected to continue without challenges. Dissemination of findings: Each year, the data collected is made available on EPA's webpages as part of the Redevelopment Economics at Superfund Sites page and corresponding pages, as well as the Putting Sites to Work - How Superfund Redevelopment is Making a Difference in Communities Across the United States: Compendium of 2021 Economic Data. Economic data are included in budget justifications to Congress and are used in general communication with key stakeholders and the public. 47 ------- Title FY 2025 Planned analyses of economic benefits at Resource Conservation and Recovery Act (RCRA) corrective action facilities Lead Office Office of Land and Emergency Management, Office of Resource Conservation and Recovery l ink lo EPA Strategic Plan Goal 6: Safeguard and revitalize communities Objective 6.1: Clean up and restore land for productive uses and healthy communities Start Dale November 2020 Completion Dale September 2025 Note This project is being conducted over multiple fiscal years and may recur annually after FY 2025 depending on resource availability. Purpose and brief description: Cleaning up contaminated facilities serves as a catalyst for economic growth and community revitalization and can help to preserve existing business operations. The Resource Conservation and Recovery Act (RCRA) economic benefits study provides information on currently active businesses now operating at former RCRA Corrective Action (CA) facilities that are now in reuse after cleanup and remediation. Economic impacts associated with facilities in reuse highlight how cleanup performed under RCRA CA can set the stage for a wide range of new development. These developments can often attract new businesses and bolster local economies. In some cases, reuse priorities are incorporated into the remedial design process, resulting in cleanups that directly facilitate future reuse. Such facilities can serve as models of what is possible when EPA and RCRA-authorized states, other state and local entities, and facility stakeholders work together to address cleanup and consider reuse priorities early in the cleanup process. Since RCRA facilities often encompass buildings, roads, and other infrastructure, their effective and efficient cleanup for continued use and/or reuse/redevelopment can play a pivotal role in a community's economic growth. Additionally, this study reveals how cleanup performed under RCRA CA also can facilitate safe, continued operations of long-time facility businesses, while also protecting human health and the environment through remediation. EPA has initiated efforts to collect economic data at a subset of RCRA facilities to gain evidence of such economic benefits. Questions to be addressed: The ongoing analysis of economic benefits provides current, reliable business-related information for a subset of RCRA Corrective Action Facilities now in reuse after they have been cleaned up. The study helps to highlight the significant economic benefits that can occur when such facilities are remediated. The analyses furthermore help the RCRA cleanup program characterize the many types of redevelopment that can occur at RCRA Corrective Action facilities. To leverage these economic findings, the Program also is producing facility case studies that showcase the cleanup and current uses so that they may be used as examples of what may be replicable at other RCRA cleanups. 48 ------- Methodological and analytical approach Data collection methods: The 2021 RCRA economic benefits study involves the collection and research of current, reliable, publicly available business-related information for businesses that are currently operating in the footprint of a subset of RCRA Corrective Action (CA) facilities that are now either in reuse or continued use after cleanup and remediation. Data sets: Information on the number of employees and sales volume for on-site businesses typically comes from the "Hoovers/Dun & Bradstreet" (D&B) database. When D&B database research is not able to identify employment and sales information for on-site businesses, EPA uses the "Reference Solutions" and "Manta" databases. These databases include data reported by businesses. In some instances where necessary, business and employment information come from publications such as company annual reports, business websites, and news media reports. Finally, employee income is estimated using average wages from the Bureau of Labor Statistics for the specific industries at each site. Analytic approach: This project primarily focuses on the collection of business-related economic data for purposes of aggregation, and basic comparative analyses so that it can be made publicly available for the Program. Tools and/or equipment: Only very common data collection and statistical software applications such as MS EXCEL are necessary for this project. Anticipated challenges and proposed solutions: As this research is 100 percent reliant on the availability of facility boundary maps, the research cannot be conducted when such maps don't exist. A second limitation can sometimes also be the lack of publicly available data for some facilities and businesses. In these circumstances, such facilities are simply excluded from the study at this time. A complete report of all findings and the underlying research methodology also is made available on our webpaee. Our office is dedicated to conducting this data collection and disseminating it to the public on an annual basis, with the only contingency being the availability of funding for the study. Dissemination of findings: Economic data findings are included in budget justifications to Congress and are used in general communication with key stakeholders and the public. A webpage was launched to make these findings and associated facility case studies broadly available to the public All past and future economic benefit findings will be made publicly available on the RCRA Hazardous Waste and Corrective Action webpage. 49 ------- Title FY 2025 OLEM near site population analysis Lead Office Office of Land and Emergency Management l ink lo I.PA Strategic Plan (ioal 6: Safeguard and revitalize communities Objective 6.1: Clean up and restore land for productive uses and healthy communities Si art Dale April 2025 Completion Dale July 2025 Nole This project is conducted each fiscal year. Purpose and brief description: This is a descriptive study. The purpose is to conduct a bi-annual analysis to support evidence-based descriptions of who benefits from EPA's cleanup and prevention work, by collecting data on the population living within three miles and within one mile of a Superfund site, Brownfields site, Resource Conservation and Recovery Act (RCRA) Corrective Action (CA) site, Leaking Underground Storage Tank (LUST) site, and Underground Storage Tank (UST) facility that exist in thousands of communities across the United States ranging from remote to large urban settings. This analysis also supports EPA's America's Children and the Environment Report.1 by estimating the number of children and their socioeconomic/demographic characteristics who live within one mile of a RCRA CA or Superfund site that may not have had all human health protective measures in place at the time of the analysis. Aspects of these results are included in EPA's annual budget reviews and are included in the annual President's Budget submitted to Congress. Results also are used in general communications with press, other government agencies, and the public. Question(s) to be addressed: This analysis estimates the population living within three miles and within one mile of a Superfund site, Brownfield site, RCRA CA site, removal site, LUST site and UST facility by: Race: people who self-identify as white, black, Asian, Native American, Hawaiian/pacific islander, or other. Ethnicity, people of all races who self-identify as Hispanic or non-Hispanic. Minority: all race and ethnicity combinations except "non-Hispanic whites." Income: below poverty level, and incomes twice or more above poverty level. Education: less than high school education. Age: Under 5, Under 18, over 64. Linguistically isolated, households where all members do not speak English as a first language or "very well." Populations that are more minority, low income, linguistically isolated, or less likely to have a high school education than the U.S. population as a whole, may have fewer resources with which 7 The Report may be accessed here: www.epa.gov/aiiiericascliildreiieiwiromiieiit. 50 ------- to address concerns about their health and environment. EPA includes these factors in population analyses to understand the potential for these vulnerabilities in relation to cleanup sites at the national level. Methodological and analytical approach Data collection methods: The population data will be downloaded from the US Census's American Community Survey 5-Year Estimates and the site location data will be downloaded from the EPA datasets listed below. Data sets: Site location and status data from the Assessment, Cleanup and Redevelopment Exchange System (ACRES), Superfund Enterprise Management System (SEMS) and RCRA Info for Brownfields, Superfund and RCRA CA, respectively. (EPA dataset). Site location and status data for LUST sites and UST facilities from ORD's state LUST/UST database (EPA dataset). Population data from the most recent American Community Survey 5-Year Estimates (external party dataset). Analytic approaches: Latitude and longitude coordinates are used to map site locations. Then 1- and 3- mile buffers are drawn from the site location. Depending on data availability, the site location is either a point, a modeled circular site boundary based on site acreage around a point, or the actual site boundaries. Using census block group centroids and the 1- and 3- mile buffers, the population and characteristics are estimated. If the census block centroid falls within the buffer, then the population of that census block is included in the estimation of the near site population. EPA compares the near site populations to the overall U.S. population to identify differences in the characteristics listed above. EPA follows the methods used in the America's Children and the Environment Report Indicators E10 and Ell.8 Tools and/or equipment: ArcGIS, R, and Excel will be used for this analysis. Anticipated challenges and proposed solutions: Geospatial data available to map site boundaries is limited. EPA continues to work to improve geospatial data on Superfund and RCRA Corrective Action site boundaries. The LUST/UST data used was obtained from the USTFinder. The USTFinder is a new web map application containing a comprehensive, state-sourced national map of underground storage tank (UST) and leaking UST (LUST) data. USTFinder was made possible by a large ORD data collection effort. Ability to update estimates for LUST/UST in the future depends on whether ORD updates data in the USTFinder. Dissemination of findings: EPA will share the results of these analyses on EPA's OLEM program benefits website and include the information in agency documents that are available to the public. 8 For more details on the methods, see https://www.epa.gov/americaschildrenenviromiient/ace-enviromnents-and-contamiiiants- contaminated-lands#Methods. 51 ------- Office of Research and Development Title Environmental Justice, Cumulative Impacts, and Vulnerable Populations l.ead Office Office of Research and Development l ink lo IPA Strategic Plan Goal 2: Take decisive action to advance environmental justice and civil rights Objective 2.1: Promote environmental justice efforts at the federal, Tribal, state, and local levels Start Dale October 2022 Completion Dale September 2026 Note This project is being conducted over multiple fiscal years. Purpose and brief description: Environmental justice (EJ) is an integral part of EPA's mission to protect human health and the environment. EJ is achieved when all people are fully protected from environmental and health hazards and have equitable access to decision-making processes to maintain a healthy environment in which to live, learn, play, and work. Low-income communities, disadvantaged groups, and indigenous peoples are often more vulnerable to environmental health challenges due to heightened exposure to pollutants, historical inequities, and social stressors, and limited adaptive capacity to address emerging stressors such as climate change. Similarly, cumulative health impacts from these chemical and nonchemical stressors vary with lifestages, as well as inherent sensitivities. Children, older persons, and people with disabilities or pre-existing health conditions may be most susceptible and vulnerable to climate changes and associated environmental stressors. Coordinating research across the six National Research Programs (NRP) will lead to a better understanding of how cumulative impacts (exposure and health effects) and health disparities can arise from unequal environmental conditions, including impacts from climate change and exposures to pollution, and inequitable social and economic conditions. ORD research efforts will be designed to strengthen the scientific foundation and generate evidence for actions at the Agency, state, tribal, local, and community levels to address cumulative impacts and environmental and health inequalities in vulnerable populations, lifestages, and communities with environmental justice and equity concerns. ORD's FY 2023 - FY 2026 Strategic Research Action Plans (StRAP 4) include focus on six cross-cutting research priorities, two of which are environmental justice and cumulative impacts. For cumulative impacts, EPA published the peer-reviewed Cumulative Impacts: Recommendations for ORD Research9 to guide development of StRAP 4 research. Researchers responded and are currently working on over 90 research products that address the recommendations included in that report. There are a total of nearly 200 unique products being developed for cumulative impacts and environmental justice combined. 9 For more details on the methods, see https://www.epa.gov/systeiii/files/docunieiits/2023- 05/CUMULATIVE%20IMPACTS%20RESEARCH-FINAL%20REPORT-EPA%20600-R-22-014A%20%2B12%29.PDF. 52 ------- Question(s) to be addressed: ORD's research will address multiple questions related to understanding and addressing cumulative and disproportionate impacts and environmental justice concern. These include expanding scientific understanding and generating evidence on environmental health disparities resulting from exposure to chemical and non-chemical stressors. Research also investigates intertwined social and environmental variables affecting community resilience and vulnerability across population groups and lifestages to inform development of policy solutions. The research also includes characterizing and assessing disproportionate exposures, risks, and impacts across media and considering climate change. ORD will use methods such as cumulative impact assessments to identify, compare, and evaluate evidence-based solutions. These solutions aim to reduce impacts and improve health and environmental equity with communities that historically have been underserved and overburdened. Methodological and analytical approach Data collection methods: A variety of methods and approaches will be used across the cumulative impacts and environmental justice research portfolios to assess who, where, and how environmental health and wellbeing are disproportionately affected and identify tools, approaches, and potential solutions for reducing these cumulative and disproportionate impacts. These methods include systematic literature reviews and meta-analyses, surveys, primary data collection, health/ecological/environmental impact assessments related to specific decisions, clustering analyses, multiple regressions and other statistical approaches, biomonitoring, biological aging, allostatic load, and analysis of large datasets. Where appropriate, models, such as EJScreen, may be used. Data sets: EPA will use existing and new data sets to carry out the environmental justice and cumulative impacts research. Any datasets EPA creates or have created for us that underly publications on these topics will be made publicly available through Science Hub. For example, states maintain health databases which may prove useful for cumulative impacts and environmental justice research. Additionally, as EPA becomes aware of large datasets, EPA will explore what is in those datasets and whether the Agency can obtain those datasets for scientific study. Analytic approaches: Multiple scientific and statistical approaches will be used for the cumulative impacts and environmental justice research. These include development of indicators and indices, epidemiological and toxicology-based studies examining, for example, allostatic load and biological aging, and exploration of the effects of non-chemical stressors on health and wellbeing. Other statistical approaches may include clustering analyses and multiple regressions. Tools and/or equipment: A variety of tools and equipment will be used to carry out this research. These include statistical software, geospatial tools, scientific equipment, and low- cost community sensors (e.g., Purple Air). Anticipated challenges and proposed solutions: This research area will produce many scientific deliverables which required complex research planning and connection with internal partners (e.g., OLEM, OEJECR, OCHP, Regional Offices) and external partners and stakeholders. Throughout 53 ------- the ongoing implementation of the research, varying levels of coordination, cooperation, and collaboration have been and will be needed, which requires commitment on the part of our partners and stakeholders. This is necessary to ensure deliverables/products address partner needs. In FY 2025, ORD will continue to develop more efficient and effective methods of project implementation and tracking. Dissemination of findings: Environmental Justice and cumulative impacts research findings will take a variety of publicly available forms including journal publications, open-access web-based tools and models, data sets, webinars, and technical fact sheets. EPA makes these available to the public via Science Inventory, the GeoPlatform, and on topical epa.gov webpages. Title Climate Change Research Lead Office Office of Research and Development Link lo CPA Slrale«ic Plan Goal 1: Tackle the climate crisis Objective 1.1: Reduce emissions that cause climate change Objective 1.2: Accelerate resilience and adaptation to climate change impacts Si art Dale October 2022 Completion Dale September 2026 Note This project is being conducted over multiple fiscal years. Purpose and brief description: Climate change is impacting public and environmental health and these impacts are likely to increase and compound over time. Changing climate patterns exacerbate the frequency, duration and intensity of wildland fires, extreme heat, flooding, drought, and harmful algal blooms; and change transportation and energy usage, for example increases in air conditioner use. These climate related events adversely impact air and water quality, availability of clean water, and infrastructure among other consequences. ORD's FY 2023 - FY 2026 Strategic Research Action Plans (StRAP 4) include focus on six cross-cutting research priorities including climate change. Coordinating research across the six National Research Programs (NRP), this research will improve understanding of these climate-driven changes, developing knowledge to support science-based decision making, and supporting climate induced disaster preparation, response and recovery, resiliency of ecosystems and the services they provide, community resilience and sustainability, and protection of human health and the environment. ORD research will generate evidence on the impacts of climate change on human health and ecosystems and societal responses, evaluate the effectiveness of greenhouse gas (GHG) mitigation approaches and strategies, and identify and evaluate adaptation and resilience approaches. This evidence can inform mitigation, adaptation, and resilience decisions at multiple levels of governance including local, tribal, state, regional, and national. Question(s) to be addressed: ORD climate change research will address questions related to GHG mitigation technologies and strategies (e.g., emissions reductions and carbon removal and 54 ------- sequestration) alternative sources of water for safe reuse, coastal acidification and hypoxia, forecasting and early detection of harmful algal blooms, natural infrastructure for coastal adaptation, and building community and infrastructure resilience to climate related extreme events and longer term stressors. In addition, ORD research will provide tools and data to assist EPA, state, tribal, local government, and communities in predicting how air quality, water quality, ecosystems, and human health will change as a result of the changing climate and the potential mitigation strategies that are adopted. ORD research will address questions related to the disproportionate impacts of climate change to inform decisions, sustainable transitions, and efforts to decrease disparities. ORD research also will address questions related to EPA responses to climate-related disasters, including public drinking water supply, drinking and wastewater infrastructure recovery, debris management, and environmental contamination cleanup (oil spill, pesticide, hazardous waste, mold, etc.). Many of these response activities benefit from capabilities developed from research supporting disaster response and recovery. Methodological and analytical approach Data collection methods: This research area will use multiple quantitative and qualitative methods to produce data, methods, and tools to advance the understanding of adverse health impacts among people, changes to air quality, changes to water quality and quantity, changes to contaminant loading in sediments and soils, and changes to ecosystem functions and services that are associated with changing climate. This research area also will produce methods and tools to improve community preparation for, response to, and recovery from climate induced disasters, as well as to improve the long-term resilience of communities to climatic change with respect to human health and welfare. Methods used in this research area may include but are not limited to literature reviews, computer modeling, environmental monitoring, health data collection, clinical studies, toxicological studies, statistical analyses, text analysis, surveys, interviews, and focus groups. Data sets: EPA will use existing and new data sets to carry out the climate change research. Any datasets EPA creates or has created for us that underly publications on these topics will be made publicly available through the EPA Science Inventory. Examples of data sets to be used include observational data used for epidemiological studies, results of toxicological studies, air emissions data, environmental measurements, and downscaled climate modeling outputs. Analytic approach: Multiple scientific and statistical approaches will be used for climate change research. These include development of indicators and indices, epidemiological and toxicology-based studies examining, for example, interactions of climate change and air pollution and impacts of wildland fire smoke on health, and ecological studies, including for example place-based studies of strategies to adapt to increased risks of flooding using nature- based solutions. Tools and/or equipment: Multiple tools and equipment will be used for climate change research. These include but are not limited to climate change models and outputs, air quality models, water distribution system models, systems models, ecological models, water quality and air sensors, satellite and other remote sensing data, and geographic information systems. 55 ------- Anticipated challenges and proposed solutions: This research area will produce scientific deliverables which will require complex research planning, facilitation, review coordination, task prioritization, and regular interactions with the program and regional partners (e.g., AO, OAR, OW, OLEM, OHS, Regional Offices) to ensure deliverables/products address partner's needs. In FY 2025, ORD will continue to develop more efficient methods of project implementation and tracking. Dissemination of findings: Research area findings will take a variety of publicly available forms such as technical reports, journal publications, open-access web-based tools and models, data sets, webinars, and technical fact sheets aimed at promoting translation of results to inform solutions. Findings will be made publicly available through the Science Inventory. Tools also will be available through the Global Change Explorer. 56 ------- Office of Water Title Public Water System Supervision (PWSS) Program Reviews and Drinking Water State Revolving Fund State Reviews Lead Office Office of Water Link lo I.PA Strategic Plan (ioal 5: Ensure clean and safe water for all communities. Objective 5.1: Ensure safe drinking water and reliable water infrastructure. Si art Dale October 2024 Completion Dale September 2025 Nole This project is conducted each fiscal year. Purpose and brief description: EPA annually conducts reviews of agencies with Public Water System Supervision (PWSS) primacy (55 reviews) and reviews of each state Drinking Water State Revolving Fund program (51 reviews). Questions to be addressed: These reviews assess if primacy entities are effectively implementing the PWSS program to oversee community water system compliance with the Safe Drinking Water Act and evaluate if states are effectively implementing the Drinking Water State Revolving Fund program to facilitate public water system compliance with the Safe Drinking Water Act (SDWA). Questions addressed include: Are primacy entities effectively implementing the range of activities in the PWSS program to oversee community water system compliance with the Safe Drinking Water Act? Are states effectively implementing the Drinking Water State Revolving Fund program to facilitate public water system compliance with the Safe Drinking Water Act, addressing public health protection and affordability, assisting disadvantaged communities with access to funding, applying fiscal integrity and controls, effectively using Bipartisan Infrastructure Law funds, and complying with the EPA's State and Tribal Assistance Grant program requirements? Data collection methods: EPA PWSS review results are reported annually in each of the individual 55 primacy agency Performance Evaluation Reports. In addition, the EPA DWSRF review results are reported out in each individual state specific Performance Evaluation Reports annually. The reports function similarly to base line monitoring reports for grant programs. Because reports are state specific, there is not a national report of overall program performance. Examples of items included in the review include: The results of reviews of state program files for system compliance with PWSS and DWSRF rules and cross cutting requirements. The results of regional transaction testing for federal cash draws. State performance in key PWSS and DWSRF program metrics, such as funding to disadvantaged communities and using Bipartisan Infrastructure Law funds. Success (or lack of success) in addressing past issues raised. 57 ------- Anticipated challenges and proposed solutions: Not applicable. Dissemination of findings: EPA PWSS review results are reported annually in each of the individual 55 primacy agency Performance Evaluation Reports. In addition, EPA DWSRF review results are reported out in each individual state specific Performance Evaluation Reports annually. EPA shares PWSS information on water system compliance rates across and within states. EPA makes publicly available an annual report on the status of the national DWSRF program. EPA also shares project and financial data at the national and state level. Title Public Water System Supervision (PWSS) National Community Water System Non-Compliance Review Lead Office Office of Water Link lo KPA Strategic Plan Goal 5: Ensure clean and safe water for all communities. Objective 5.1: Ensure safe drinking water and reliable water infrastructure. Si art Dale October 2024 Completion Dale September 2025 Nolc This project is conducted each fiscal year. Purpose and brief description: EPA conducts a review quarterly of the PWSS National Community Water System (CWS) health-based non-compliance data. Question(s) to be addressed: This review assesses the trends and causes of non-compliance. This assessment is used to inform technical, managerial, and financial state and public water system capacity building training or future drinking water regulation needs, in support regulatory drinking water compliance. The question addressed was: What are the barriers and challenges of CWS systems maintaining compliance with health- based drinking water standards? Data collection method and data set: Data are provided from the EPA's Safe Drinking Water Information System (SDWIS) database. There is a non-compliance review of CWS systems with health-based violations by regulation type, geographical distribution, and system source type. Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major challenges in completing this project. Dissemination of findings: The findings from the program reviews will be publicly shared. Quarterly data reports are shared publicly via the SDWIS FED Data Warehouse. 58 ------- Title Clean Water State Revolving Fund State Reviews Lead Office Office of Water l ink to KPA Strategic Plan Goal 5: Ensure clean and safe water for all communities. Objective 5.1: Ensure safe drinking water and reliable water infrastructure. Start Dale October 2024 Completion Dale September 2025 Note This project is conducted each fiscal year. Purpose and brief description: EPA conducts annual reviews of each state Clean Water State Revolving Fund program (51 reviews). Question(s) to be addressed: The reviews assess if states are effectively implementing the Clean Water State Revolving Fund program in compliance with the Clean Water Act (CWA). Questions these reviews address include: Are states effectively implementing the Clean Water State Revolving Fund program in compliance with the Clean Water Act, addressing water quality priorities and affordability, assisting disadvantaged communities with access to funding, applying fiscal integrity and controls, effectively using Bipartisan Infrastructure Law funds, and complying with the EPA's State and Tribal Assistance Grant program requirements? Data collection methods and data sets: EPA CWSRF review results are reported out in 51 state specific Performance Evaluation Reports annually. The reports function similarly to base line monitoring reports for grant programs. Because reports are state specific, there is not a national report of overall program performance. Examples of items included in the review include: The results of reviews of state program files for compliance with CWSRF rules and cross cutting requirements. The results of regional transaction testing for federal cash draws. State performance in key CWSRF program metrics, such as funding to disadvantaged communities and using Bipartisan Infrastructure Law funds. Success (or lack of success) in addressing past issues raised. Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major challenges in completing this project. Dissemination of findings: EPA CWSRF review results are reported out in 51 state specific Performance Evaluation Reports annually. EPA makes publicly available an annual report on the status of the national CWSRF program. EPA also shares project and financial data at the national and state level. 59 ------- Title FY 2025 Lake Pontchartrain Basin Restoration Program (PRP) program assessment report l.esul Office Office of Water / Region 6 Link lo EPA Strategic Plan Goal 5: Ensure clean and safe water for all communities. Objective 5.2: Protect and restore waterbodies and watersheds. Start Dale October 2024 Completion Dale March 2026 Note This project is being conducted over multiple fiscal years. Purpose and brief description: EPA will assess (1) the suitability of the Management Conference and the program's organizational structure in achieving the program's objectives; (2) the grantee's performance related to PRP grants; and (3) the program's progress toward achieving the PRP equity strategy goals. Question(s) to be addressed: Is the Management Conference performing the required program actions? Are work plan commitments under the PRP grants being met? What percentage of BIL funds are being applied towards disadvantaged communities? Methodological and analytical approach Data collection method: EPA will upload the program documents (or grant deliverables) received from the grantee into internal shared folders. EPA will review the grant deliverables and communicate any deficiencies to the grantee. EPA will manage record keeping spreadsheets. Data sets: Question No 1: Is the Management Conference performing the required program actions? To answer question number 1, EPA will evaluate meeting notes and will develop an EPA spreadsheet to track action items from the Management Conference meetings. Question No 2: Are work plan commitments under the PRP grants being met? To answer question number 2, EPA will review the PRP Annual Evaluation Report. Question No 3: What percentage of BIL funds are being applied towards disadvantaged communities? To answer question 3, EPA will develop a spreadsheet and track PRP tracking Justice40 investments. Analytic method and tool: EPA will use Microsoft Excel to compare the data and will report in Microsoft Word. 60 ------- Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major challenges in completing this project. Dissemination of findings: EPA will upload significant program fundings and reports to the Lake Pontchartrain Basin Restoration Program's public website. 61 ------- FY 2025 Evaluation and Evidence-Building Activities - Supplemental Funds Bipartisan Infrastructure Law The Bipartisan Infrastructure Law (BIL) expanded EPA's historic role as a regulatory and scientific agency to be a large-scale funder of critical infrastructure. In FY 2022 and FY 2023 BIL programs at EPA designed and planned a series of evidence-building projects that address four priority areas: Identification of program investments. This priority area assesses the extent to which funds are being distributed to disadvantaged and underserved communities, including people of color, low-income groups, Tribes, and rural communities. This data will document EPA's progress in reversing decades of underinvestment in communities most impacted by environmental hazards, pollution, and climate change. Examination of how well programs are being implemented. This priority area identifies program implementation with a focus on target schedules and milestones, including the development of deliverables such as products (e.g., reports), services (e.g., technical assistance), and events (e.g., community meetings). This data can help identify inconsistencies, bottlenecks, and gaps in the process of carrying out programs, which can then be targeted for improvement. More broadly, understanding a program's implementation enables EPA to identify the reasons why a program is successful. Documentation of program outcomes. As a priority, BIL programs seek to deliver outcomes across various important dimensions: environmental (e.g., number of acres of coastline restored); community-level health indicators (e.g., child asthma rates); behavior change (e.g., individual or organizational recycling behaviors); social and economic benefits (e.g., access to green spaces, job creation); climate change mitigation (e.g., diesel emissions reduction); and climate change adaptation (e.g., flood risk reduction). Collecting data about these outcomes over the next few years will enable programs to assess their longer-term effects. Identification of key stakeholders and their environmental priorities. Stakeholder engagement is prioritized to ensure that programs are involving communities and groups who are most affected by activities and decisions of the program. Working with stakeholders helps programs harness valuable insights and experiences about local priorities and creates a shared vision for identifying and solving problems; such collaborations increase the likelihood of program success. Each BIL program has developed an evidence-building strategy to systematically collect data that will address one or more of these priority areas, using different methods such as document reviews, interviews, surveys, and focus groups. In FY 2024 and FY 2025, programs will collect data to build an evidence base that can be used to improve their effectiveness, efficiency, and equity. EPA will share summary results and information in subsequent Annual Performance Reports and will post related evaluation reports on the Agency's evaluation website. Furthermore, significant evaluations will be included in future Annual Evaluation Plans. 62 ------- Inflation Reduction Act The Inflation Reduction Act (IRA) enables EPA to take aggressive action in tackling the climate crisis. EPA's IRA-funded programs are being designed and launched during FY 2023 and FY 2024, during which time the Agency is identifying and implementing appropriate evaluation and evidence-building activities to support program implementation and assess results. In FY 2025, EPA will continue its efforts to use evaluation and evidence-building for IRA programs with the following goals in mind: executing programs efficiently and effectively; promoting transparency and building trust; maintaining accountability to taxpayers; and advancing equity priorities. EPA will share information and results for these activities in subsequent Annual Performance Reports and will post evaluation reports on the Agency's evaluation website. Furthermore, significant evaluations will be included in future Annual Evaluation Plans. 63 ------- |