United States
Environmental Protection Agency
FISCAL YEAR 2024
Justification of Appropriation
Estimates for the
Committee on Appropriations
Tab 14: Evaluation and Evidence-Building for FY 2024
March 2023
EPA-190-R-23-001 www.epa.gov/cj
-------
EPA Evaluation and Evidence-Building for FY 2024
The Foundations for Evidence-Based Policymaking Act (Evidence Act) provides a framework to
promote a culture of evaluation and continuous learning to ensure Agency decisions are made
using the best available evidence. EPA's FY 2024 Annual Evaluation Plan (AEP) describes
significant program evaluations and other evidence-building activities the Agency plans to
undertake in FY 2024. The Agency's FY 2024 AEP includes program evaluations that assess
program outcomes, support program improvement, and aid decision making. Final program
evaluation reports will be available at EPA's website: https://www.epa.eov/evaluate unless
otherwise indicated.
FY 2024 Annual Evaluation Plan
Office of Chemical Safety and Pollution Prevention (OCSPP)
Title
EPA-Supported WPS Training of Farmworkers
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
October 2023
Planned completion date
September 2024
Purpose and brief Description: This study uses pre- and post-training assessments to gauge the
effectiveness of this recurring training. EPA provides funding through a 5-year cooperative grant
to train farmworkers in accordance with the Agricultural Worker Protection Standard (WPS) rule.
WPS pesticide safety training is an annual requirement. This evaluation will track the number of
individuals trained and the effectiveness of the training by assessing participant knowledge and
understanding before and after the training.
Programmatic or policy decisions this activity will inform: Effectiveness and scope of the
EPA-supported WPS training.
Question(s) this activity will address:
• How many farmworkers are receiving EPA-supported annual training required under the
WPS rule, and what is their knowledge of the material at completion of the course?
Data, tools, method/analytical approach: Critical data sets include information provided by the
grantee on number of individuals trained. The evaluation will include pre- and post-training,
assessments of those trained. EPA can follow up with the grantee for assessment of factors leading
to performance results against quarterly and annual targets.
Anticipated challenges and proposed solutions: The COVID-19 public health emergency may
continue to influence the grantee's training reach. Social distancing and other in-person protection
strategies means smaller training groups and fewer individuals trained. Remote training can
present technical challenges for some workers. Additional COVID-related impacts experienced in
FY 2022 include farmers and growers' refusal to let trainers onto farms to conduct training for fear
of exposing workers to COVID, and decreased capacity within the grantee's national network of
-------
training organizations (hiring and retention challenges, loss of established relationships with the
agricultural community). In-person training is a preferred method to engage with trainees; the
grantee and agricultural establishments will continue to implement strategies to address the
pandemic-related obstacles while meeting the training needs. The grantee is currently
aligning/redirecting resources to institutions with training capacity and will continue reaching out
to their network and partnering organizations, including local agencies, nonprofit organizations,
community leaders and agricultural employers, to better understand their capacity and increase
efforts to return to in-person training.
Dissemination of findings: Evaluation results will be made publicly available in the Annual
Reports on PRIA Implementation ("https://www.epa.gov/pria-fees/annual-reports-pria-
implementation).
Title
Effectiveness of OCSPP Pollution Prevention Activities
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.2: Promote Pollution Prevention
Planned start date
Continuing from
FY 2022
Planned completion date
September 2024
Purpose and brief Description: The Pollution Prevention (P2) Program seeks to alleviate
environmental problems by leveraging business-relevant approaches to achieve significant
reductions in the generation of hazardous releases to air, water, and land; reductions in the use of
hazardous materials, which also advances EPA's chemical risk reduction and management goals
under the Toxic Substances Control Act (TSCA); reductions in the generation of greenhouse gases;
and reductions in the use of water. As a result of these preventative approaches, the P2 Program
helps businesses and others reduce costs and access market opportunities.
Programmatic or policy decisions this activity will inform: Review will assess the degree of
progress and outcomes of the P2 programs, as well as the utility of performance measures for the
program.
Question(s) this activity will address: The overarching evaluation question the program intends
to address is "What are the outcomes of EPA's P2 programs?" More specific questions will be
aligned to individual programs and initiatives, such as the environmental justice aspects of
implementation of the BIL and the climate aspects of implementation of the Inflation Reduction
Act (IRA). Additionally, EPA intends to address questions about outcomes of the Safer Choice
Program considering increased leadership interest and support for the program.
Data, tools, method/analytical approach: EPA will review a range of data assets to determine
how the information and data collected as part of the P2 grant programs, the existing list of Safer
Choice products, and other performance metric targets can be used for FY 2024 conduct of
evaluation of effectiveness and outcomes, as well as for continuous improvements of day-to-day
operations.
-------
Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major
challenges in gathering relevant data. This assessment may change pending the review of data
assets and evaluation plans developed in FY 2023.
Dissemination of findings: EPA anticipates making evaluation findings public through EPA's
website www.epa.gov/evaluate, as well as other program venues as appropriate.
Office of Enforcement and Compliance Assurance (OECA)
Title
Assessing the effectiveness of offsite compliance monitoring
Lead National Program
Office of Enforcement and Compliance Assurance
Strategic Goal and
Objective supported
Goal 3: Enforce Environmental Laws and Ensure Compliance
Objective 3.2: Detect Violations and Promote Compliance
start date
Continuing from
FY 2022
Planned completion date
September 2024
Purpose and brief Description: This project is a part of OECA's Compliance Learning Agenda
which collaborates with states, tribes, and academics to identify the most pressing programmatic
questions, and create a venue for EPA, states, tribes, and territories to collaborate in the
development of evidence-based enforcement tools and techniques that will ensure the biggest
impact on environmental compliance. As EPA moves out of the pandemic that restricted our ability
to do onsite inspections, it recognizes that a broader portfolio of Off-site Compliance Monitoring
(OfCM) activities may provide the Agency with additional tools for our enforcement and
compliance programs. To assess what EPA has learned from the extended use of the past two years
and gain insight into the efficacy of OfCM tools relative to onsite inspections, the Agency is
conducting an exploratory evaluation using readily available data and information to inform
interim guidance and best practices. EPA will then use those results to guide a longer-term
evaluation of OfCM and the best uses of these tools going forward. The Agency anticipates that
the answers to these questions will involve multiple evaluation efforts given the range of programs
and OfCM tools that will need to be assessed.
Programmatic or policy decisions this activity will inform: In addition to furthering the efforts
of OECA's Compliance Learning Agenda, the results of this activity will inform future Agency
policy and guidance related to Off-site Compliance Monitoring.
Question(s) this activity will address:
• How does the effectiveness of off-site compliance monitoring activities compare to onsite
inspections?
• What outcomes does the Agency achieve from off-site compliance monitoring?
• What is the best use for OfCM? (Does it depend on the tool, the program, and on the
compliance history of the facility?)
Do OfCM tools support enforcement activities?
-------
Data, tools, method/analytical approach: Data mining and analysis will use State/EPA
inspection data, enforcement data, and State OfCM data from ICIS, other EPA data systems, and
State associations. Some EPA regional OfCM data will be analyzed from a short questionnaire for
the short-term study. An algorithm may be used to establish links between OfCM inspection
activities and enforcement actions. The Agency also plans to partner with academics to develop
methodology for longer term evaluations.
Anticipated challenges and proposed solutions: There are potential significant data limitations
associated with this activity. For example, because of the broad categories in ICIS, the definition
of the OfCM activity performed in ICIS may not be indicative of the actual activity performed. To
address this, EPA will use an array of different data sources to obtain as much specific, credible
information as possible to minimize data irregularities. The Agency will develop and use
algorithms to establish direct links and/or correlations between OfCM activities and enforcement
actions. Longer term evaluations will require participation by our state and tribal partners to be
successful. EPA has partnered with the E-Enterprise Leadership Council (EELC) and have invited
the Environmental Council of States (ECOS), states, and tribes to participate in the workgroup to
complete learning agenda projects.
Dissemination of findings: EPA anticipates making project findings public through EPA's
website, www.epa.gov/evaluate, as well as other public venues as appropriate.
References and Relevant Sources
OECA Compliance Learning Agenda
site Compliance Monitoring Project
EPA. OfCM Project Partnership Portal
Title
Identifying interventions that are effective at overcoming the impediments
to municipal compliance
Lead National Program
Office of Enforcement and Compliance Assurance
Strategic Goal and
Objective supported
Goal 3: Enforce Environmental Laws and Ensure Compliance
Objective 3.2: Detect Violations and Promote Compliance
Planned start date
October 2023
Planned completion date
Through FY 2024
Purpose and brief Description: This project is a part of OECA's Compliance Learning Agenda
(CLA) which collaborates with state, tribal, and territorial government environmental
professionals, and academics, to identify the most pressing programmatic questions, and create a
venue for EPA, states, tribes, and territories to collaborate in the development of evidence-based
enforcement tools and techniques that will ensure the biggest impact on environmental
compliance. EPA has heard about causes of noncompliance for small municipal water systems
from many sources over some time. The Agency's goal is to identify the root causes that lead to
noncompliance and that also render agency interventions (e.g., enforcement, technical assistance,
etc.) unsuccessful at returning systems to compliance. EPA anticipates this project to involve
multiple evaluation and research projects under both the National Pollutant Discharge Elimination
System (NPDES) and Safe Drinking Water Act (SDWA) programs.
-------
Programmatic or policy decisions this activity will inform: Project results will be used to
improve Agency efforts and interventions to ensure that they are effective at returning systems to
compliance.
Question(s) this activity will address:
• What are the root causes of municipal (wastewater treatment plants and drinking water
systems) noncompliance that can render EPA and state enforcement and technical/financial
assistance efforts unsuccessful?
• Considering the root causes of municipal noncompliance, what are the impediments to
compliance that prevent technical assistance/financial assistance/enforcement tools from
being effective in producing compliance?
• What alternate or supportive interventions are effective in producing compliance?
• What is the effectiveness of the application of various compliance tools to municipal
noncompliance, e.g., enforcement actions, technical assistance, etc. in producing
compliance - or improved compliance?
Data, tools, method/analytical approach: EPA expects to use a wide range of evaluation tools
and methods to address priority questions., For example, EPA plans use data collected by state
associations' surveys, as well as engaging with evaluation and other academic experts to learn
about evaluation and research into the causes of municipal noncompliance. Data mining and
analysis will use both NPDES data from the ICIS-NPDES data system (and possibly state-specific
NPDES data systems) and drinking water data from SDWIS-Fed.
Anticipated challenges and proposed solutions:
• Effectiveness of enforcement (and other compliance tools) in producing compliance may
vary state to state for various reasons. EPA will account for this variance in the study.
• There are multiple likely drivers of noncompliance and variations of the drivers of
noncompliance between states. The Agency will likely need a large study dataset to analyze
the associations between these drivers of noncompliance and the effectiveness of
enforcement actions to become evident.
• There is uncertainty about ease of obtaining reliable information about the drivers of
noncompliance for individual enforcement action. To help overcome this challenge, we
have partnered with the EELC and have invited ECOS, states, and tribes to participate in
the workgroup to complete learning agenda projects.
Dissemination of findings: EPA anticipates making project findings public through EPA's
website, www.epa.gov/evaluate, as well as other public venues as appropriate.
-------
Office of Water (OW)
Title
Progress Evaluation of the National Estuary Program
Lead National
Program
Office of Water/Office of Wetlands, Oceans and Watersheds
Strategic Goal and
Objective supported
Goal 5: Ensure Clean and Safe Water for All Communities
Objective 5.2: Protect and Restore Waterbodies and Watersheds.
Planned start date
October 2023
Planned completion date
September 2024
Purpose and brief Description: This program evaluation activity is focused on the National
Estuary Program as described in Section 320 of the Clean Water Act. The primary purpose of the
Progress Evaluation of the National Estuary Programs (NEPs) is to help EPA assess progress in
achieving programmatic and environmental results through implementation of Comprehensive
Conservation and Management Plans (CCMPs). The program evaluation process has proven to be
an effective, interactive management process that ensures national program accountability and
transparency, while incorporating local priorities and demonstrating the value of federal
investment in estuarine and coastal watershed restoration and protection at the local and regional
levels. The program evaluation process was revised and published in the 2021 NEP Program
Evaluation Guidance This new guidance was distributed to the 28 NEP locations at the end of FY
2021.
The NEP program evaluation process also: transfers lessons learned among NEPs, EPA, and
stakeholders through the sharing of case studies and transferable examples; documents the value
added to environmental management by the national program and individual NEPs, including their
role in convening stakeholders and interpreting science for management; demonstrates continued
stakeholder commitment; and highlights achievements and successes of each NEP, as well as
suggestions for continued program improvements.
Programmatic or policy decisions this activity will inform: Seven Program Evaluations will be
conducted in FY 2024. The regular evaluation process for NEP locations informs the
administration of the NEP program. It ensures the locations are delivering environmental results
and are well-managed programs so that they can continue to receive annual grants from EPA which
are matched 1:1 with non-federal dollars.
Question(s) this activity will address:
• Can NEP locations determine progress in achieving programmatic and environmental
results?
• Can NEP locations document contributions to improving or reducing pressures on their
coastal watersheds and enable NEPs to successfully serve as local implementation partners
for EPA programs?
• Can NEP locations identify areas of improvement to assist NEPs in becoming stronger
programs and achieving environmental results?
Progress, results, and interim findings (if applicable): The NEP program evaluation process
runs on an annual cycle such that each location within the NEP is evaluated every five years. The
-------
program evaluation process uses a two-category determination of Proficient and Progressing, as
defined in the guidance. Proficient means an NEP is adequately meeting programmatic and
environmental results. Progressing means there are missing criteria that need to be addressed
before the next cycle and will catalyze a timeline to address those missing elements or
opportunities for improvement before the next cycle. This determination is informed by the entire
Program Evaluation (PE) package (narrative submission, NEPORT (NEP Online Reporting Tool)
data, annual work plans, and EPA required annual end of year reports), on-site visit, and through
discussions with the NEP under review.
Data, tools, method/analytical approach: Information for the evaluation process includes annual
reports and work plans, data submitted to NEPORT, existing CCMPs, and other documentation of
key decisions., In addition, any supplementary documents the NEP locations may have developed
including communications plans, financial plans, monitoring plans, etc. are also reviewed. The
NEP locations assemble these materials and make them available to the program evaluation team
which include EPA staff from headquarters and applicable region along with a director from a
different NEP.
Anticipated challenges and proposed solutions: The regular program evaluation process
examines each NEP location on a variety of topics listed below. Each presents a potential challenge
and can be addressed through the discussions between the program evaluation team and NEP
locations. The results include recommendations for improvement based upon the following
categories and are submitted to each program as a final program evaluation, letter:
• NEP Administration and Governance Structure
• Grant Obligations and Finance including budget summary
• Healthy Ecosystems (e.g., fish, shellfish, plant, eelgrass, and wildlife populations; habitat
protection/restoration, natural resources, land use, hydrological and ecological restoration,
invasive species)
• Community and Stakeholders Engagement
• Education and Outreach
• Monitoring and Assessment
• Clean Water Act Programs Relationship
• EPA Priorities (Nutrient pollution, water reuse and conservation, marine litter reduction,
green infrastructure, environmental justice, climate change)
Dissemination of findings: Information about the Progress Evaluation of the National Estuary
Program is posted on EPA's website at https://www.epa.gov/nep/progress-evaluation-national-
estuary-program.
-------
FY 2024 Evaluation and Evidence-Building Activities in Support of the
Bipartisan Infrastructure Law and Inflation Reduction Act
In FY 2022 the Bipartisan Infrastructure Law (BIL) expanded EPA's historic role as a regulatory
and scientific agency to be a large-scale funder of critical infrastructure and the Inflation Reduction
Act (IRA) enables EPA to take aggressive action in tackling the climate crisis. In FY 2022 and FY
2023, in support of the new and expanded programs supported by BIL and IRA, EPA is identifying
and implementing appropriate evaluation and evidence-building activities to assess results and
support program implementation. In FY 2024, EPA will continue its efforts to use evaluation and
evidence-building for BIL and IRA programs with the following goals in mind: executing programs
efficiently and effectively; promoting transparency and building trust; maintaining accountability to
taxpayers; and advancing equity priorities. EPA will share results and information in subsequent
Annual Performance Reports found on www.epa.gov/planandbudget. in addition to posting related
evaluation reports at www.epa. gov/evaluate
FY 2024 Evidence-Building Activities
The Foundations for Evidence-Based Policymaking Act (Evidence Act) provides a framework to
promote a culture of evaluation and continuous learning to ensure Agency decisions are made
using the best available evidence. EPA's FY 2024 Annual Plan for Evidence-Building Activities
describes Agency plans for significant evidence-building across a range of program areas. In this
section EPA describes evidence-building activities other than program evaluations, such as data
analysis, foundational fact finding, research, statistical analysis, continuous process improvement,
and performance measurement. This document shares examples of evidence-building that supports
EPA's decision-making in response to Administration priorities, Congressional mandates, and
management priorities.
The first part of this document gives an overview of EPA's evidence-building activities in support
of the Agency's Learning Agenda, which is part of the FY 2022 - 2026 EPA Strategic Plan. This
part is organized by Learning Priority Area. The second part of this document, the Other Evidence-
Building Activities, is organized by national program.
Evidence-Building Activities Supporting EPA's Learning Agenda
Expanding EPA's Toolkit of Air Benefits Assessment Methodologies and Practices
Title
Expanding EPA's Toolkit of Air Benefits Assessment Methodologies and
Practices
Lead National Program
Office of Air and Radiation
Strategic Goal and
Objective supported
Goal 4: Ensure Clean and Healthy Air for All Communities
Objective 4.1: Improve Air Quality and Reduce Localized Pollution and
Health Impacts
Planned start date
Subject to Planned completion date Subject to Funding
Funding
Purpose and brief Description: EPA uses well-established methods for estimating the health
benefits associated with reductions in some pollutants. However, as noted by scientific bodies
-------
including the National Academy of Sciences (NAS) and Science Advisory Board (SAB)1, there
are areas where the science of air pollution effects continues to advance and there are benefits that
EPA does not currently quantify and monetize.
Programmatic or policy decisions this activity will inform: This activity will improve EPA's
ability to: (1) characterize the health benefits of improved air quality within Environmental Justice
communities; (2) account for the role of air pollution in promoting the progression of chronic
disease and subsequent death; (3) quantify the health benefits of reducing toxic air pollutants.
Question(s) this activity will address:
• What are the health benefits of reducing human exposures to air pollutants not currently
quantified, particularly those related to hazardous air pollutants (HAPs)?
• What are the health benefits of reducing the risk of air pollution-related effects that are
challenging to quantify but nonetheless important to the exposed populations?
• What are the benefits of health outcomes that cannot yet be valued using Willingness-to-
Pay or other measures of economic value?
• How can EPA account for sequelae and the progression of disease when quantifying
benefits?
Progress, results, and interim findings: In FY 2023 EPA anticipates hiring dedicated staff and
initiating work on reports detailing methodological improvements and case studies applying those
improvements. In addition, EPA plans to convene an NAS panel to advise the Agency and review
the output of key reports/case studies.
Data, tools, method/analytical approach: Addressing the above questions will require access to
economic and health datasets providing information on health effect incidence, health outcomes,
and health care expenditures. EPA would apply these newly developed techniques using existing
Agency tools, including the newly revised cloud-based version of the environmental Benefits
Mapping and Analysis Program (BenMAP).
Anticipated challenges and proposed solutions: Addressing questions of the scope and
complexity of those above will require significant contract resources and additional FTE (in
particular, economists, biostatisticians, and air pollution epidemiologists).
Partnerships supporting this evidence-building effort: EPA anticipates working collaboratively
with NAS in developing this project. EPA also anticipates consulting with industry, academia,
1 National Research Council. 2002. Estimating the Public Health Benefits of Proposed Air Pollution Regulations. Washington,
DC: The National Academies Press, https://doi.org/10.1.7226/1.051.1.. National Research Council. 2008. Estimating Mortality Risk
Reduction and Economic Benefits from Controlling Ozone Air Pollution. Washington, DC: The National Academies Press.
https://doi.org/10.1.7226/1.21.98.
-------
environmental organizations. The Agency will seek input and review of these parties in case
studies and methodological proposals.
Dissemination of findings: Any NAS reports will be disseminated by the NAS, although EPA
will provide links to those reports through EPA's website, www.epa.gov/evaluate, as appropriate.
Information and any findings also will be shared with EPA staff and management through other
venues (e.g., meetings, presentations, etc.).
Drinking Water Systems Out of Compliance
Title
Drinking Water Systems out of Compliance
Lead National Program
Office of Enforcement and Compliance Assurance
Strategic Goal and
Objective supported
Goal 3: Enforce Environmental Laws and Ensure Compliance Objective 3.2:
Detect Violations and Promote Compliance
Planned start date
Continuing from
FY 2022
Planned completion date
Through 2026
Purpose and brief Description: Office of Enforcement and Compliance Assurance (OECA),
Office of Water (OW), and the Drinking Water Systems Out of Compliance learning priority
workgroup are assessing drinking water data reported to EPA to determine whether it accurately
measures national compliance and substantiates EPA policy decisions; considering noncompliance
root causes and corresponding technical/managerial/financial (TMF) factors; and testing efficacy
of technical assistance, enforcement, and state oversight. The assessments, once complete, will
identify key water system characteristics for which EPA and states should focus its policies and
the most effective way to apply compliance assurance tools for increasing compliance in the
drinking water program.
OECA anticipates FY 2024 funds will support continuation of evaluations and other empirical
analyses for Question 3 (in the Agency Learning Agenda Learning Priority Area: Drinking Water
Systems Out of Compliance) - efficacy of enforcement on compliance, and for Question 4 -
identifying metrics for TMF, and initiation of work on question 5 - EPA oversight best practices.
EPA plans to begin work on Question 4 in FY2023, using results from Question 2 - root cause of
noncompliance, and EPA anticipates that work will continue into FY 2024.
Programmatic or policy decisions this activity will inform: Applying compliance assurance
tools to effectively increase drinking water compliance rates.
Question(s) this activity will address:
• Does increased use of compliance assurance tools (inspections and enforcement) improve
system compliance, and if so under what circumstances?
• How can EPA determine if a system has the TMF capacity to provide safe water on a
continuous basis to its customers?
• What EPA oversight activities are effective at assessing and improving state programs'
ability to drive compliance?
-------
Progress, results, and interim findings: In FY2022, EPA made significant progress on Questions
2 due to existing work from which the Agency was able to learn water system characteristics most
correlative to noncompliance. Specifically, predictive tools developed and tested in the State of
California and EPA Regions 3 and 7 resulted in statistical data indicating that important system
characteristics of a water system that relate to system success are: number of sources of water,
financial factors, system size, history of violations, operator training level, complexity of treatment
required, and management factors. The results align with professional observation of
characteristics associated with noncompliance. The next phase will involve examining the
potential for additional, original analysis to gain further insights into root causes of noncompliance
and conducting such work. In FY2022, for Question 3, EPA worked with the General Services
Administration's Office of Evaluation Scienci S) academic partners from Georgetown
University to explore possible study designs for evaluating the effect of EPA inspections in the
drinking water program (as compared to the current approach of only having state sanitary surveys
of systems). Additionally, EPA has entered into an agreement with the University of Kansas to
study if and under what conditions enforcement in the drinking water program improves
compliance. EPA anticipate study work on Question 3 will begin in FY2023 and will continue in
FY2024.
Data, tools, method/analytical approach:
• Question 2 Root Cause Analysis: Going forward, the analysis will review findings to date
and determine if additional data sets can be accessed or created to do deeper analysis on
certain characteristics (e.g., find specific aspects of management structure that correlate
with ongoing compliance). If so, advanced statistical analytical methods may be applied to
better understand underlying causes versus correlations to noncompliance. For Question 3
on Enforcement and Inspection Efficacy, the Agency hopes to empirically test the impact
of increased use of compliance monitoring inspections and conduct surveys of public water
system operators to evaluate effects of enforcement actions as compared to the status quo
practice of heavily relying on other types of assistance. This priority question complements
the Drinking Water National Compliance Initiative (NCI). Increased use of inspections
could be planned such that those activities form the basis of a prospective study to inform
the overall evaluation process. The results of the study could inform future compliance
assurance strategies either as part of or beyond the NCI. Additionally, the use of OECA's
Enforcement and Compliance History Online database will be used to do a retrospective
analysis of enforcement activity.
• EPA will search for available data sets that provide technical, managerial, and financial
information to base the Question 4 (TMF metrics) evaluation. The Agency anticipates
needing to pull from various places such as federal databases at EPA (SDWIS), Department
of Commerce Census Bureau, and USD A Rural Utilities Service (RUS) loan program data
and information gleaned from the State Revolving Fund work, state Capacity Development
annual reports, and sanitary survey checklists.
• EPA anticipates using several different tools for the evaluation of Questions 3 (efficacy of
enforcement), 4 (TMF metrics), and 5 (EPA oversight) including statistical software,
-------
survey instruments, literature reviews, data mining, and advanced statistical analysis such
as machine learning and other regression approaches.
Anticipated challenges and proposed solutions: Data availability may slow-down and/or limit
progress on the root-cause analysis, as well as other analytical activities that need to be carried out
to conduct planned evaluations and other empirical studies. For example, the volume of
compliance assurance work may be too low to support methodologies that use a randomization
approach to Question 3 (efficacy of enforcement). States and water systems may not agree to
participate in a survey study to identify attitudes on enforcement actions. Insufficient TMF data
could limit our ability to identify effective metrics for TMF capacity.
Dissemination of findings: Final evaluation reports and other empirical analyses for this learning
priority area will be made available through EPA's website www.epa.gov/evaluate.
Workforce
Title
Workforce
Lead National
Program
Office of Mission Support
Strategic Goal and
Objective supported
Cross-Agency Strategy 3: Advance EPA's Organizational Excellence and
Workforce Equity
Planned start date
Continuing from FY Planned completion September 2024
2023 date
Purpose and brief description: EPA identified Human Capital Management as an Enterprise
Risk due to the high number of staff eligible for retirement and EPA's aging workforce, and the
related concern of replacing departing employees' institutional knowledge and skills. The
Workforce learning priority area in EPA's Learning Agenda will develop an evidence-based
roadmap for how EPA can ensure it has employees with the competencies needed to achieve its
mission now and in the future. It also will help determine the overall processes required to cultivate
and manage the workforce, while anticipating internal and external changes, and continuously
maximizing the efficiency and effectiveness of the Agency's Human Resources services.
Programmatic or policy decisions this activity will inform: Near- and long-term strategies to
attract, recruit, train, and retain a diverse and effective workforce.
Question(s) this activity will address:
• Does EPA have access to the tools and strategies needed to analyze and understand the
Agency's near- and long-term workforce and succession needs?
• What are the critical skills needed to support the Agency's mission, now and in the future?
• What are the best strategies to attract, recruit, train, and retain a diverse workforce? What
makes people stay in the Agency long-term?
• What is the best way to ensure knowledge is transferred from outgoing to current and incoming
staff to support succession planning?
-------
Data, tools, method/analytical approach: EPA has various data sets and tools to capture
employee demographic, hiring, and attrition data, but no current Agencywide data sets exist on
current and future employee skills and competencies for leadership and other critical positions. To
develop such data sets, EPA will compile information gathered from various internal and external
stakeholders during its workforce planning and succession management activities. Stakeholders
include but are not limited to: EPA employees and supervisors, the Human Resource
Officer/Program Management Officer (HRO/PMO) community, First Line Supervisor Advisory
Group (FLAG), senior leaders, and members of the external human resources academic and
practitioner community. Surveys, literature reviews, focus groups, interviews, and other
quantitative and qualitative methods will be used to obtain needed information. Cost-benefit
analysis, benchmarking, and appropriate quantitative and qualitative analyses will be used along
with other analytical approaches. Data will be managed consistent with security and privacy
requirements.
Anticipated challenges and proposed solutions: Due to a general sense of "survey fatigue"
within EPA, there is a risk of low participation among key internal stakeholders in the assessment
and analysis of the four workforce priority questions. This possible challenge will be mitigated by
enlisting the buy-in and support of senior leaders, the Human Resources Council, and other key
stakeholders to help promote the process prior to its start and keeping in constant contact with
those stakeholders during the evaluation and analysis process. Additionally, analytical approaches
may be constrained by limitations due data security and privacy requirements; EPA will work
carefully to identify appropriate alternatives.
Dissemination of findings: The identified workforce activities are considered key components of
management's strategic decision-making process; findings will be shared consistent with
requirements related to information that may be privileged or prohibited from disclosure. The
Agency anticipates that relevant results will be shared with internal stakeholders, including senior
leaders and EPA's Human Resource Officer/Program Management Officer community. Aggregate
information on findings might be shared with other federal agencies and/or publicly.
Grant Commitments Met
Lead National Program
Office of the Administrator/Office of Congressional and
Intergovernmental Relations
Cross-Agency Strategy
supported
Cross-Agency Strategy 4: Strengthen Tribal, State, and Local Partnerships
and Enhance Engagement
Planned start date
Continuing from
FY 2021
Planned completion date
Through 2024
Purpose and brief Description: Historically, EPA has annually awarded over $4 billion in grants
and other assistance agreements. With these grants, EPA helps to protect human health and the
environment through the work of its grantees. The management and tracking of the individual
awards are dispersed amongst thousands of staff throughout EPA's headquarters and ten regional
offices, which makes tracking results at the national level challenging. This Learning Agenda
priority area focuses on how to better understand how the EPA's grant programs track, report, and
analyze its outputs and outcomes achieved to assess and communicate the environmental and
health results accomplished.
-------
The importance and visibility of this effort has increased with the additional funding provided by
American Rescue Plan (ARP), IIJA, and, most recently, the IRA. The potential funding level for
EPA's programs could reach over $100 billion, making it critical to have mechanisms in place to
track, report and assess the Agency's progress in protecting human health and the environment
through its grant programs.
This learning priority area outlines work to establish the baseline, assess the current state, and
define the future state of grant result reporting. The effort spans multiple fiscal years, beginning in
FY 2021. In FY 2021, the Grant Commitments Met Workgroup used a survey instrument to
establish a baseline knowledge of grant results reporting practices at EPA. FY 2022 was focused
on interviewing regional and headquarters National Program Manager (NPM) contacts to collect
best practices. Efforts in FY 2023 and 2024 are focused on using the gathered data to implement
grant program reviews and inform grant result tracking systems to better communicate and assess
the environmental and health results achieved through EPA's grant programs.
Programmatic or policy decisions this activity will inform: Practices and tools to effectively
track whether grantees are fulfilling their workplan grant commitments, including outputs and
environmental outcomes.
Question(s) this activity will address:
• What data and information exists to provide a baseline assessment of the Agency's grant
and tracking systems?
• Which criteria are used to assess the ability of programs to successfully monitor grantee
performance?
• How are the Agency' s grant programs meeting their intended purpose?
Progress, results, and interim findings: In Year 1, EPA surveyed all active EPA grant programs
to determine the universe of existing grant reporting and tracking systems. The surveys provided
the data and information needed to understand existing Agency approaches and processes for
collecting, monitoring, reporting, and evaluating grant commitments. EPA learned that 99% of
programs collect output data, but only 31% collect long-term outcomes. Word documents are the
most common method of collecting grantee data. Common challenges to grantee data collection
include labor intensity (i.e., the time and knowledge required for EPA staff and grantee staff to
conduct data collection activities), poor communication with grantees, and capacity issues internal
to grantees.
In Year 2, EPA used the Year 1 survey responses as a launching point to establish criteria for
conducting the next set of interviews within EPA. Interview criteria included prioritizing programs
receiving additional BIL or ARP funding, programs that reported best practices, and programs that
addressed administration priorities. EPA has identified 31 Regional and NPM interviewees and
has completed most of the interviews.
EPA is comparing the current state of grants management to an ideal future state, considering the
programmatic and statutory requirements unique to each grant program, and available tools for
-------
programmatic monitoring. A workgroup will develop criteria to assess the ability of programs to
successfully monitor grantee performance, with a specific focus on tracking environmental outputs
and outcomes. This activity will inform the next of phase (Learning Agenda Question 3), which
will analyze the Agency's ability to review progress made in protecting human health and the
environment through its grant programs and demonstrate how EPA's grants programs are
achieving the intended environmental results.
Data, tools, method/analytical approach: The third phase of this project will address the
question: Are the commitments established in grant agreements achieving the intended results?
Using Year 1 Survey results and Year 2 interview responses, EPA will design an appropriate
empirical study to answer this key question.
The effort will culminate in overall findings report including several recommendations with the
goal of reaching actionable results that aid in the implementation of a new Agency-wide approach
to collect and communicate environmental results.
Anticipated challenges and proposed solutions: Success depends on high stakeholder
engagement and participation, including that of regional and NPM staff and management. EPA
will address these challenges by relying on a group of regional and NPM points of contact and
leveraging access to senior leadership calls. It will be challenging to keep up with the rapidly
changing landscape regarding grant funding at EPA. Grant programs at EPA continue to expand
in size and number. The high visibility of this additional funding further highlights the importance
of accountability in grant reporting.
Partnerships supporting this evidence-building effort: EPA will continue to engage with and
inform states and tribes of EPA efforts through ECOS, the e-Enterprise Leadership Council
(EELC), and other appropriate fora.
Dissemination of findings: Final reports will be posted publicly on EPA's website
www, epa. gov/evaluate; the Year 1 report has been posted at
https://www.epa.gov/svsterri/files/docurrients/2Q22~Q9/leaming~agenda~grants~corrirriitrrients~
met.pdf.
Other EPA Evidence-Building Activities
Office of Air and Radiation (OAR)
Title
Title V Permitting Program Reviews
Lead National Program
Office of Air and Radiation
Strategic Goal and
Objective supported
Goal 4: Ensure Clean and Healthy Air for All Communities
Objective 4.1: Improve Air Quality and Reduce Localized Pollution
and Health Impacts
Planned start date
October 2023
Planned completion date
September 2024
-------
Purpose and description: EPA periodically assesses state and local permitting programs,
including the sufficiency of fees collected, under Title V of the Clean Air Act as part of its
responsibility to oversee delegated and approved air permitting programs.
Programmatic or policy decisions this activity will inform: In general, these analyses document
areas needing improvement and inform how EPA can help the permitting agencies improve their
performance.
Question(s) this activity will address:
• What are some good practices and areas of improvement in state and local permitting
programs under Title V of the Clean Air Act?
• How can EPA help the permitting agencies improve their performance?
Data, tools, method/analytical approach: In general, EPA uses a questionnaire to gather
preliminary information, reviews files maintained on permits, conducts site visits, and follows up
with the permitting program to clarify information in conducting a Title V program assessment.
Anticipated challenges and proposed solutions: The Agency conducts these analyses annually
and does not anticipate challenges.
Dissemination of findings: The Title V Permit analyses are posted on ^ website. Information
and any findings will also be shared with appropriate EPA staff and management.
Title
Our Nation's Air: Status and Trends Through 2023
Lead National Program
Office of Air and Radiation
Strategic Goal and
Objective supported
Goal 4: Ensure Clean and Healthy Air for All Communities
Objective 4.1: Improve Air Quality and Reduce Localized Pollution
and Health Impacts
Planned start date
October 2023
Planned completion date
September 2024
Purpose and description: EPA is committed to protecting public health and the environment by
improving air quality and reducing air pollution. This annual report presents the trends in the
nation's air quality and summarizes the detailed information found at EPA's Air Trends website
and other air quality and emissions data.
Programmatic or policy decisions this activity will inform: This activity provides an annual
assessment of air quality in an accessible format, allowing EPA, states, and other stakeholders to
understand how air quality is changing both in their local area and across the nation. Stakeholders
can use this information to help inform their decisions in their air quality programs.
-------
Question(s) this activity will address:
• Where are areas experiencing air quality above the national ambient air quality standards?
• Are these areas trending toward improving air quality?
Data, tools, method/analytical approach: Existing data is pulled from several sources to generate
the report such as the National Emission Inventory (NEI) and Air Quality System (AQS).
Anticipated challenges and proposed solutions: The Agency produces this report annually and
does not anticipate challenges. This activity is contingent upon air quality data availability from
state, local, and tribal air pollution control agencies.
Dissemination of findings: This report is annually included on :mds website.
Information and any findings will also be shared with appropriate EPA staff and management.
Office of the Chief Financial Officer (OCFO)
Title
Growing Capacity for Evidence Building
Lead National
Program
Office of the Chief Financial Officer
Cross-Agency Strategy
Supported
Strategy 3: Advance EPA's Organizational Excellence and Workforce Equity
Planned start date
Continuing from FY
2023
Planned Completion
Date
Throughout 2024
Purpose and brief Description: EPA will engage in a variety of Evidence Act-related activities
in FY 2024 that will support policy development and decision making. These activities reflect
EPA's commitment to implement a framework that promotes a culture of evaluation and
continuous learning and ensures Agency decisions are made using the best available evidence. For
example, in FY 2023 EPA is engaging an Agency-wide effort to integrate evidence-building in the
implementation of BIL and IRA by identifying evidence-building priorities for BIL and IRA-
funded programs. In FY 2024, EPA will carry out plans across the board for evaluation and other
empirical analyses.
EPA will lead a coordinated cross-agency process to support the design and execution of
evaluations of BIL investments. EPA began this coordinated effort in FY 2023 and will continue
to lead this effort in FY 2024.This effort will also include developing capacity for equity in
evaluation by advancing approaches to conducting evaluations more equitably and be culturally
responsive and/or develop new approaches reflecting EPA's unique understanding and role in
advancing environmental justice.
Dissemination of findings: EPA will share the results of these efforts on EPA's website,
www, epa. gov/evaluate.
-------
Office of Chemical Safety and Pollution Prevention (OCSPP)
Title
Assessing IT modernization of EPA pesticide tracking systems
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
April 2019
Planned completion date
September 2024
Purpose and brief Description: In April 2019, EPA kicked off Phase 1 of a multi-year digital
transformation to create a fully electronic workflow for EPA registration and reevaluation
activities. In early 2020, in advance of the launch of the new system, EPA developed performance
metrics and established baselines of performance using the current Agency systems for review of
applications. These metrics will allow EPA to measure the effects of the digital transformation on
meeting the targets and objectives described in the FY 2022 - 2026 EPA Strategic Plan. In FY
2020, a pilot of the new system went live and continues to be implemented in phases.
EPA has designed an ongoing assessment of the pilot focused on how the system meets
performance metrics. Also, the pilot is designed to assess how effectively the use of predictive
algorithms will help in determining skills gaps and hiring needs. The pilot is also designed to assess
time savings and effects related to work-life balance. For example, OPP is tracking targeted
Employee Viewpoint Survey (EVS) questions to track workload satisfaction. We believe digital
transformation success will impact the results in a positive direction.
Programmatic or policy decisions this activity will inform: Results of this assessment will
inform additional IT system development and facilitate enterprise resilience through strategic
planning, proactive risk management, effective organizational change management and capacity
planning, as well as emergent technologies.
Question(s) this activity will address:
• Does this pilot approach show the expected potential for mission transformation through
digitalization?
• What is the effectiveness of predictive algorithms used in this pilot in determining where
skills gaps lie, and how does this help with resources decisions intended to remove
bottlenecks?
• Does this pilot show how EPA can assess time savings and as well as outcomes related to
work-life balance?
Data, tools, method/analytical approach: Information from EPA's PRISM and OPPIN systems
will allow EPA to establish baselines for how much time is spent at each stage of risk assessment
and assess improvement in the overall review processes for registration and registration review
cases. The Employee Engagement metric will be tracked by evaluating results to specific questions
and focus areas on the EPA Employee Viewpoint Survey and comparing responses from OPP staff
before and after implementation of the IT-modernization effort.
-------
Anticipated challenges and proposed solutions: EPA is currently waiting for the award of the
Mission Support IT Contract to continue work on the Digital Transformation. Current contracts
supporting development and operations & maintenance of systems expire in November thereby
making the award of the new contract urgent. Office of Acquisition Services (OAS) is currently
projecting an award date of September 15, 2022.
Dissemination of findings: Process improvements relating to pesticide registration and
registration review activities, as well as information technology improvements, are described
annually in the PRIA annual report ("https://www.epa.gov/pria-fees/annual-reports-pria-
implementation).
Title
TSCA Risk Evaluation Review and Assessments
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
October 2023
Planned completion date
September 2024
Purpose and brief Description: These recurring reviews assess the degree of progress, timely
completion of risk evaluations mandated under TSCA, and the utility and quality of program
performance measures.
Programmatic or policy decisions this activity will inform: EPA will use information from
these reviews to make decisions on whether any changes are needed to performance measures.
Information also will be used to determine needed changes in the process for completion of
chemical risk evaluations within TSCA statutory time frames or other time frames designated by
the Agency.
Question(s) this activity will address:
• Do EPA's suite of performance measures and processes for developing TSCA risk
evaluation warrant further revision?
Data, tools, method/analytical approach: Critical data sets include performance metric targets
and results and any other data sets that could point to a need for operational improvements.
Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major
challenges in gathering performance data; however, challenges in chemical risk evaluation data
gathering can exist. Expert input will be brought to bear on any challenges and possibility that
solutions will be needed.
Dissemination of findings: EPA intends to make performance results publicly available, through
a variety of venues, including but not limited to, Agency performance reporting at
https://www.epa.eov/planandbudeet and other targeted stakeholder outreach and communications
-------
Title
TSCA Risk Management Activities
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
October 2023
Planned completion date
September 2024
Purpose and brief Description: These recurring reviews will assess the degree of progress and
timely completion of risk management actions taken by EPA under TSCA, and performance
measures (APG, LTPG, internal operational metrics).
Current measures for this program include 1) Draft risk management rules anticipated to not be
completed by statutory deadline; 2) Final risk management rules anticipated to not be completed
by statutory deadline; and percent of existing chemical TSCA risk management actions initiated
within 45 days of a final chemical risk evaluation. EPA anticipates proposing and finalizing some
number of risk management actions in FY 2024 and will assess the evidence for these measures at
that time. EPA will not have the ability to fully determine the effectiveness of these actions in FY
2024 given the anticipated finalization during that year.
Programmatic or policy decisions this activity will inform: Decisions on whether changes are
needed in the measurement of, process for developing, or implementation of chemical risk
management actions.
Question(s) this activity will address: Whether EPA's suite of performance measures and
processes for developing TSCA risk management actions warrant further revision. EPA anticipates
assessing the effectiveness of its risk management actions in future years following finalization of
the actions.
Data, tools, method/analytical approach: Critical data sets include performance metric targets
and results and any other data sets that could point to a need for operational improvements.
Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major
challenges in gathering relevant performance data; however, chemical data and information to
inform risk management quality can be limited in quantity or quality, or hard to obtain. Expert
input will be brought to bear on any challenges and will address any possibility that solutions will
be needed.
Dissemination of findings: Any risk management actions proposed or finalized will be a matter
of public record. EPA intends to make performance results publicly available, through a variety of
venues, including but not limited to, Agency performance reporting at
https://www.epa.gov/planandbudget and other targeted stakeholder outreach and communications.
Title
TSCA New Chemicals Activities
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
October 2023
Planned completion date
September 2024
-------
Purpose and brief Description: Under TSCA section 5, EPA may impose certain actions by
issuing an order and/or significant new use rule (SNUR) that may contain risk mitigation
requirements put in place to protect human health and the environment. EPA's Office of Pollution
Prevention and Toxics (OPPT) will develop a process to review compliance with the requirements
of past Orders and SNURs issued under TSCA section 5, looking back from October 2021. EPA
will review compliance with restrictions in TSCA section 5 orders or SNURs by cross-walking
action requirements with information reported to the 2020 Chemical Data Reporting (CDR) rule.
This process would include any chemical with a TSCA section 5 order and/or SNUR that was also
reported to CDR.
Programmatic or policy decisions this activity will inform: OPPT will use the information
reported to CDR to check adherence with the terms of past TSCA section 5 orders or SNURs.
Instances of potential non-compliance will be referred to EPA's Office of Enforcement and
Compliance Assurance (OECA). This could trigger OECA follow-up actions, including an
information request for records, subpoena for specific documents and answers to questions, virtual
records auditing, on-site audits, issuance of compliance advisories or guidance, and
modifications/updates to TSCA section 5 Consent Order, SNURs, or other applicable regulations,
as appropriate.
Question(s) this activity will address: This activity will be the first time the New Chemicals
Program has conducted a systematic review of information reported to EPA. Questions include
• Does the information reported to help ensure chemical manufactures and importers
comply with regulations designed to protect workers, consumers, communities, and the
environment?
• Does the process to review compliance with these TSCA requirements meet the needs of
the Agency in identifying non-compliance?
Progress, results, and interim findings: The new chemicals program has started creating the
datasets required to crosscheck new chemicals data with CDR data. Additionally, the program has
developed methods to crosscheck new chemical actions with data reported to CDR.
Data, tools, method/analytical approach: Critical data sets include the 2020 dataset from
Chemical Data Reporting rule and a dataset of the regulatory terms of all TSCA section 5 consent
orders and SNURs.
Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major
challenges in gathering relevant performance data; however, there can be challenges in cross-
walking historic chemical data and information to more recent economic and enforcement
information (e.g., current company status, ownership changes, location changes, etc.) Further,
limitations in chemical data and information exist. Expert input will be brought to bear on any
challenges and will address any possibility that solutions will be needed.
Dissemination of findings: EPA intends to make performance results publicly available through
a variety of venues, including but not limited to, Agency performance reporting at
https://www.epa.eov/planandbudeet and other targeted stakeholder outreach and communications.
-------
Title
Pesticide Registration Review
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
October 2023
Planned completion date
September 2024
Purpose and brief Description: These recurring reviews assess the degree of progress and
timely completion of docket openings, draft risk assessments, and case completions for the second
cycle of pesticide registration review.
Programmatic or policy decisions this activity will inform: Decisions on whether any changes
are needed to pesticide registration review performance measures or the process for completion of
pesticide registration review activities.
Question(s) this activity will address:
• Does OCSPP's suite of pesticide registration review performance measures and processes
for meeting pesticide registration review statutory timeframes warrant further revision?
• Does the suite of pesticide registration review performance measures affect the quality of
the draft risk assessments and risk management decisions?
Data, tools, method/analytical approach: Critical data sets include performance metric targets
and results and any other data sets that could point to a need for operational improvements.
Anticipated challenges and proposed solutions: At this time, OCSPP does not anticipate any
major challenges in gathering performance data. Expert input will be brought to bear on any
challenges and possibility that solutions will be needed.
Dissemination of findings: EPA intends to make performance results publicly available, through
a variety of venues, including but not limited to, Agency performance reporting at
https://www.epa.eov/planandbudeet and quarterly updates to the pesticide registration review
schedule (https://www.epa.eov/pesticide-reevaluation/iipcomine-reeistration-review-actions).
Title
ESA Effects Determinations for Listed Species
Lead National Program
Office of Chemical Safety and Pollution Prevention
Strategic Goal and
Objective supported
Goal 7: Ensure Safety of Chemicals for People and the Environment
Objective 7.1: Ensure Chemical and Pesticide Safety
Planned start date
October 2023
Planned completion date
September 2024
Purpose and brief description: The Endangered Species Act (ESA) require that the actions of
federal agencies do not jeopardize the continued existence of federally threatened or endangered
species or destroy or adversely modify their critical habitat. EPA is developing a process to
incorporate ESA determinations into its new active ingredient registration process and to work
towards more routine considerations of ESA determinations for registration review decisions. EPA
anticipates increasing ESA considerations into its registration and registration review decisions at
-------
an increasing frequency over the next 5 years. In FY 2022, EPA posted the ESA workplan2 to
provide to the public the framework for ESA implementation into pesticide regulatory activities.
Data collection for this activity occurs annually.
Programmatic or policy decisions this activity will inform: Decisions on whether any changes
are needed to performance measures or the process for incorporating ESA effects determinations
into OSCPP's risk assessments supporting registration and registration review activities. EPA is
evaluating whether targets established at the time of the FY2022-2026 SP development are in
alignment with the implementation schedule communicated in the EPA ESA workplan posted to
the EPA webpage and has proposed adjusting annual targets appropriately. Stages of
implementation are dependent on additional resources.
Question(s) this activity will address:
• Do processes for developing ESA effects determinations warrant further revision?
• Should EPA develop a new suite of performance measures to measure current or new
processes, and if so, what are the options?
Data, tools, method/analytical approach: Critical data sets include EPA workflow tracking
systems and stand-alone reports on ESA-related risk assessment activity and label mitigation.
Tools and analytical methods listed above would not be needed for this exercise.
Anticipated challenges and proposed solutions: At this time, EPA does not anticipate any major
challenges in gathering performance data. Expert input will be brought to bear on any challenges
and possibility that solutions will be needed.
Dissemination of findings: EPA intends to make performance results publicly available through
a variety of venues, including but not limited to, Agency performance reporting at
https://www.epa.eov/planandbudeet and other targeted stakeholder outreach and communications.
Office of Land and Emergency Management (OLEM)
Title
OLEM Population Analysis
Lead National Program
Office of Land and Emergency Management
Strategic Goal and
Objective supported
Goal 6: Safeguard and Revitalize Communities
Objective 6.1: Cleanup Up and Restore Land for Productive Uses and
Healthy Communities
Planned start date
March 2024
Planned completion date
July 2024
Purpose and brief Description: This is a bi-annual descriptive study. The purpose is to conduct
a bi-annual analysis to support evidence-based descriptions of who benefits from EPA's cleanup
and prevention work, by collecting data on the population living within three and one mile(s) of a
Superfund site, Brownfields site, Resource Conservation and Recovery Act (RCRA) Corrective
Action (CA) site, Leaking Underground Storage Tank (LUST) site, and Underground Storage
2 The ESA Workplan may be accessed at: https://www.epa.gov/system/files/documents/2022-04/balanciiig-wildlife-protection-
and-responsible-pesticide-use_final.pdf.
-------
Tank (UST) facility that exist in thousands of communities across the United States ranging from
remote to large urban settings.
This analysis also supports EPA's America's Children and the Environment Report? by estimating
the number of children and their socioeconomic/demographic characteristics who live within one
mile of a RCRA CA or Superfund site that may not have had all human health protective measures
in place at the time of the analysis.
Programmatic or policy decisions this activity will inform: Aspects of these results are included
in EPA's annual budget reviews and are included in the annual President's Budget submitted to
Congress. Results also are used in general communications with press, other government agencies,
and the public.
Question(s) this activity will address: This analysis estimates the population living within three
and one mile(s) of a Superfund site, Brownfield site, RCRA CA site, removal site, LUST site and
UST facility by:
• Race: people who self-identify as white, black, Asian, Native American, Hawaiian/pacific
islander, or other.
• Ethnicity, people of all races who self-identify as Hispanic or non-Hispanic.
• Minority: all race and ethnicity combinations except "non-Hispanic whites."
• Income: below poverty level, and incomes twice or more above poverty level.
• Education: less than high school education.
• Age: Under 5, Under 18, over 64.
• Linguistically isolated, households where all members do not speak English as a first
language or "very well."
Populations that are more minority, low income, linguistically isolated, or less likely to have a
high school education than the U.S. population as a whole, may have fewer resources with which
to address concerns about their health and environment. EPA includes these factors in population
analyses to understand the potential for these vulnerabilities in relation to cleanup sites at the
national level.
Data, tools, method/analytical approach:
• Data
o Site location and status data from the Assessment, Cleanup and Redevelopment
Exchange System (ACRES), Superfund Enterprise Management System (SEMS) and
RCRA Info for Brownfields, Superfund and RCRA CA, respectively,
o Site location and status data for LUST sites and UST facilities from ORD's state
LUST/UST database
o Population data from the most recent American Community Survey 5-Year Estimates
• Methods/Analytical approach:
3 The Report may be accessed here: www.epa.gov/aiiiericascliildreiieiwiromiieiit.www.epa.gov/americaschildrenenvironment.
-------
o Latitude and longitude coordinates are used to map site locations. Then 1- and 3- mile
buffers are drawn from the site location. Depending on data availability, the site
location is either a point, a modeled circular site boundary based on site acreage around
a point or the actual site boundaries,
o Using census block group centroids and the 1- and 3- mile buffers, the population and
characteristics are estimated. If the census block centroid falls within the buffer, then
the population of that census block is included in the estimation of the near site
population.
o EPA compares the near site populations to the overall U.S. population to identify
differences in the characteristics listed above,
o EPA follows the methods used in the America's Children and the Environment Report
Indicators E10 and El l.4
• Tools
o This spatial analysis is done using ArcGIS and R software suites
Anticipated challenges and proposed solutions: Geospatial data available to map site boundaries
is limited. EPA continues to work to improve geospatial data on Superfund and RCRA Corrective
Action site boundaries. The LUST/UST data used was obtained from the USTFinder. The
USTFinder is a new web map application containing a comprehensive, state-sourced national map
of underground storage tank (UST) and leaking UST (LUST) data. USTFinder was made possible
by a large ORD data collection effort. Ability to update estimates for LUST/UST in the future
depends on whether ORD updates data in the USTFinder.
Dissemination of findings: EPA will share the results of these analyses on EPA's OLEM program
benefits website at https://www.epa.gov/aboutepa/ofFice~land~and~emergencv~management~olem~
program-benefits and include the information in Agency documents that are available to the public.
Title
Redevelopment Economics at Remedial Sites (non-federal facility)
Lead National Program
Office of Land and Emergency Management
Strategic Goal and
Objective supported
Strategic Goal 6: Safeguard and Revitalize Communities
Objective 6.1: Cleanup Up and Restore Land for Productive Uses and
Healthy Communities
Planned start date
October 2023
Planned completion date
January 2024
Purpose and brief Description: Cleaning up contaminated sites can serve as a catalyst for
economic growth and community revitalization. The Superfund Remedial Program facilitates the
redevelopment of sites across the country while protecting human health and the environment.
Collaborative efforts among state, local, and tribal partners, redevelopers, and other federal agency
programs encourage restoration of sites. Since Superfund sites often encompass buildings, roads,
and other infrastructure, their effective and efficient cleanup and reuse can play a pivotal role in a
community's economic growth. EPA has initiated efforts to collect economic data at a subset of
Superfund sites. Each year, the data collected is made available on EPA's webpages as part of the
Redevelopment Economics at Superfund Sites page and corresponding pages, as well as the
4 For more details on the methods, see https://www.epa.gov/americaschildrenenviromiient/ace-enviromiients-and-contaminants-
contaminated-lands#Methods.
-------
Putting Sites to Work - How Superfund Redevelopment is Matins a Difference in Communities
Across the United States: Compendium c 2 , onomic Data. In addition, Regional Economic
Profiles summarize economic data collected for Superfund sites within an EPA region. They also
highlight successes and put them in the context of aggregated data within the state and EPA region.
Economic data are updated annually; regions receive a full regional economic profile or a data
supplement to update the prior year's full regional economic profile on alternate years.
Programmatic or policy decisions this activity will inform: Economic data are included in
budget justifications to Congress and are used in general communication with key stakeholders
and the public.
Question(s) this activity will address: The analysis will provide current, reliable business-related
information for a subset of Superfund sites in reuse and continued use:
• What information can EPA provide about Superfund sites in reuse and continued use,
including the variety of purposes that some innovative business owners and organizations
reuse Superfund sites9
• To what extent and how do these uses help economically revitalize communities near
Superfund sites9
Data, tools, method/analytical approach: The study estimates economic activity at Superfund
sites in reuse from reputable sources based on methodology developed by EPA's Superfund
Redevelopment Initiative and outlined on the public webpage: Redevelopment Economics at
Superfund Sites. Information on the number of employees and sales volume for on-site businesses
typically comes from Hoovers/Dun & Bradstreet, the Reference! LSA and Manta databases.
Anticipated challenges and proposed solutions: Given that most sites with known economic
activities have been included in the report in prior years and that relatively few Superfund sites are
added to the NPL or become newly economically productive each year, the analysis does not
typically reveal large changes year-to-year. In order to focus resources on the most salient analyses,
the program alternates the development of Regional economic profiles every two years and may
consider a similar effort with a national profile if needed in the future. In the meantime, the
program has also increased a focus on Beneficial Economic Effect Case Studies which offer
expanded insights into the economic data and highlight replicable strategies for productive reuse
and redevelopment.
Dissemination of findings: The summary of the results will be shared on EPA's Superfund
Redevelopment website.5
Title
Redevelopment Economics at Federal Facilities
Lead National Program
Office of Land and Emergency Management
Strategic Goal and
Objective supported
Strategic Goal 6: Safeguard and Revitalize Communities
Objective 6.1: Cleanup Up and Restore Land for Productive Uses and
Healthv Communities
Program Project Title/
Federal Facilities Restoration and Reuse Office
5 Found at: https://www.epa.gov/superfmd-redevelopment/redeveloTOmt-economics-superfund-sites.
-------
Appropriations
Planned start date
October 2023
Planned completion date
January 2024
Purpose and brief Description: Cleaning up contaminated sites at federal facilities can serve as
a catalyst for economic growth and community revitalization. The Superfund Federal Facilities
Program facilitates the redevelopment of federal facility sites across the country by assisting other
federal agencies (OFAs) expedite activities related to CERCLA response actions, while protecting
human health and the environment. Collaborative efforts among OF As; developers; and state,
local, and tribal partners encourage restoration of sites. Since federal facility Superfund sites often
encompass thousands of acres with buildings, roads, and other infrastructure, their effective and
efficient cleanup and reuse can play a pivotal role in a community's economic growth. EPA has
initiated efforts to collect economic data at a subset of federal facility Superfund sites which is
outlined on the public webpage Redevelopment Economics at Federal Facilities.6 This economic
analysis is updated and expanded annually.
Programmatic or policy decisions this activity will inform: Economic data are included in
budget justifications to Congress and are used in general communication with other Federal
agencies and the public.
Question(s) this activity will address: The analysis will provide current, reliable business-related
information for a subset of federal facility Superfund sites in reuse and continued use.
• What information can EPA provide about federal facility Superfund sites in reuse and
continued use, including the variety of purposes that some innovative business owners and
organizations reuse Superfund sites.
• To what extent and how do these uses help economically revitalize communities near
Superfund sites9
Data, tools, method/analytical approach: The study estimates economic activity at federal
facilities Superfund sites in reuse from reputable sources based on methodology developed by
EPA's Superfund Redevelopment Initiative, which is outlined in more detail at Redevelopment
Economics at Fede ilities1. Information on the number of employees and sales volume for
on-site businesses typically comes from Hoovers/Dun & Bradstreet, the Reference! LSA and
Mania databases.
Anticipated challenges and proposed solutions: The Economic Analysis commenced in 2016
and is updated/expanded annually. The Economic Analysis is an established activity that provides
valuable metrics for the program and is expected to continue without challenges.
Dissemination of findings: The summary of the results will be shared on eral Facilities
website.8
6 Found at: https://www.epa.gov/fedfac/redevelopment-federal-facilities.
7 Found at: https://www.epa.gov/fedfac/redevelopment-federal-facilities
8 See https://www.epa.gov/fedfac/redevelopment-federal-facilities.
-------
Office of Research and E
~evelopment (ORD)
Title
Climate Change Research
Lead National Program
Office of Research and Development
Cross-Agency Strategy
supported
Cross-Agency Strategy 1: Ensure Scientific Integrity and Science-Based
Decision Making
Strategic Goal and
Objective supported
Goal 1: Tackle the Climate Crisis
Objective 1.2: Accelerate Resilience and Adaptation to Climate Change
Impacts
Planned start date
October 2023 Planned completion November 2026
date
Purpose and brief description: Climate change is impacting public health, air, and water quality
today and will exacerbate environmental challenges in the future. Potential impacts from climate
change include increases in scale and frequency of hurricanes, wildland fires, flooding and
drought, and changes in transportation and energy usage. These changes impact air and water
quality as well as the health of both humans and ecosystems. Coordinating research across the six
National Research Programs (NRP), this research will improve understanding of these climate-
driven changes, developing knowledge to support science-based decision making, and supporting
climate induced disaster preparation, response and recovery, resiliency of ecosystems services,
community resilience and sustainability, and management of human health and the environment.
Programmatic or policy decisions this activity will inform: ORD research efforts will be
designed to strengthen the scientific foundation for actions at the Agency, state, tribal, local,
territory, and community levels to address environmental and health inequalities in vulnerable
populations, lifestages, and communities with environmental justice and equity concerns.
Question(s) this activity will address: This research area will assist EPA in addressing scientific
questions related to environmental and health inequalities and is supported by multiple national
research programs. EPA, state, tribal, local government, and communities need tools and data to
predict how air quality, water quality, ecosystems, and human health will change because of the
changing climate and the potential mitigation strategies that are adopted. ORD research will inform
decisions, sustainable transitions, and efforts to decrease disproportionate impacts of climate
change. The climate induced disasters need EPA response activities including public drinking
water supply, drinking and wastewater infrastructure recovery, debris management, and
environmental contamination cleanup (oil spill, pesticide, hazardous waste, mold, etc.). Many of
these response activities benefit from capabilities developed from research supporting chemical,
biological, and radiological incident response.
Data, tools, method/analytical approach: This research area will produce data, methods, and
tools to advance the understanding of adverse health impacts among people, changes to air quality,
changes to water quality and quantity, changes to contaminant loading in sediments and soils, and
changes to ecosystem functions and services that are associated with changing climate. This
research area will also produce methods and tools to improve community preparation for, response
to, and recovery from climate induced disasters, as well as to improve the long-term resilience of
communities to climatic change with respect to human health and welfare.
-------
Anticipated challenges and proposed solutions: This research area will produce scientific
deliverables which will require complex research planning, facilitation, review coordination, task
prioritization, and regular interactions with the program partners (e.g., AO, OAR, OW, OLEM,
Regional Offices) to ensure deliverables/products address partner's needs. In FY 2024, ORD will
continue to develop more efficient methods of project implementation and tracking.
Dissemination of findings: Research area findings will take a variety of publicly available forms
such as technical reports, journal publications, open-access web-based tools and models, data sets,
webinars, and technical fact sheets aimed at promoting translation of results to inform solutions.
Title
Environmental Justice, Cumulative Impacts, and Vulnerable Populations
Lead National Program
Office of Research and Development
Cross-Agency Strategy
supported
Cross-Agency Strategy 1: Ensure Scientific Integrity and Science-Based
Decision Making
Strategic Goal and
Objective supported
Goal 2: Take Decisive Action to Advance Environmental Justice and Civil
Rights
Objective 2.1: Promote Environmental Justice Efforts at the Federal, Tribal,
State, and Local Levels
Planned start date
October 2023 Planned completion November 2026
date
Purpose and brief description: Environmental justice (EJ) is an integral part of EPA's mission
to protect human health and the environment. EJ is achieved when all people are fully protected
from environmental and health hazards and have equitable access to decision-making processes to
maintain a healthy environment in which to live, learn, play, and work. Low-income,
disadvantaged communities and indigenous peoples are often disproportionately vulnerable to
environmental health challenges because they are faced with greater exposure to many sources of
pollutants or contaminants (chemical stressors), disadvantaged due to long term environmental
policies resulting in wealth and health inequities or disparities, and underserved in adaptive
capacity to respond to and cope with emerging environmental stressors including those caused by
climate change and catastrophic incidents. Similarly, cumulative health impacts from these
chemical and nonchemical stressors vary with lifestages, as well as inherent sensitivities. Children,
older persons, and people with disabilities or pre-existing health conditions may be most
susceptible and vulnerable to climate changes and associated environmental stressors.
Coordinating research across the six National Research Programs (NRP) will lead to a better
understanding of how health disparities can arise from unequal environmental conditions,
including impacts from climate change and exposures to pollution, and inequitable social and
economic conditions.
Programmatic or policy decisions this activity will inform: ORD research efforts will be
designed to strengthen the scientific foundation for actions at the Agency, state, tribal, local,
territory, and community levels to address cumulative impacts and environmental and health
inequalities in vulnerable populations, lifestages, and communities with environmental justice and
equity concerns.
Question(s) this activity will address: ORD's research will address multiple questions such as:
-------
• How can EPA address the challenge of expanding the state of scientific understanding for
addressing environmental health disparities and shortening of lifespan related to exposure
to chemical and nonchemical stressors in vulnerable populations and life stages?
• How can EPA address the challenge of investigating the intertwined social and
environmental variables that affect community resilience and vulnerability to
environmental contamination incidents including natural disasters?
• How can EPA address the challenge of characterizing and assessing disproportionate
exposures, risks, and impacts, such as through cumulative impact assessment, while
identifying, comparing, and evaluating evidence-based solutions to reduce these impacts
in, and improve the health and wellbeing of, communities with EJ and equity concerns?
Data, tools, method/analytical approach: In this research area a large amount of data, as well as
methods, and tools, will be developed that help support decision-making and empower EPA, states,
tribes, local governments, and overburdened and disadvantaged communities to take action for
revitalization, resilience, and sustainability. This research will enhance human health by
supporting the development of new technologies, data, models, and tools as well as resources and
trainings for risk communication and risk management, outreach, and community engagement. In
January 2022, ORD published the external review draft white paper, Cumulative Impacts:
Recommendations _, lesearch to guide development of ORD's FY 2023-2026 research
portfolio and address critical science gaps.9 ORD has developed scores of research products to
address cumulative impacts. Over time, ORD will be able to assess progress towards addressing
the research recommendations contained in the white paper.
Anticipated challenges and proposed solutions: This research area will produce scientific
deliverables which will require complex research planning, facilitation, review coordination, task
prioritization, and regular interactions with the program partners (e.g., OLEM, OEJ, OCHP,
Regional Offices) to ensure deliverables/products address the partner needs. In FY 2024, ORD
will continue to develop more efficient methods of project implementation and tracking.
Dissemination of findings: Research area findings will take a variety of publicly available forms
including journal publications, open-access web-based tools and models, data sets, webinars, and
technical fact sheets.
Office of Water (OW)
Title
Clean Water Infrastructure Revolving Fund State Reviews
Lead National Program
Office of Water
Strategic Goal and
Objective supported
Goal 5: Ensure Clean and Safe Water for All Communities.
Objective 5.1: Ensure Safe Drinking Water and Reliable Water
Infrastructure.
Planned start date
October FY 2023 Planned completion date Throughout FY 2024
9 Additional information may be found at: https://www.epa.gov/healthresearch/cumulative-impacts-research.
-------
Purpose and brief Description: EPA completes annual reviews of each state's Clean Water State
Revolving Fund (CWSRF) program.
Programmatic or policy decisions this activity will inform: These reviews will help assess if
states are effectively implementing the CWSRF program by increasing the amount of non-federal
dollars leveraged. The reviews will also be used to encourage states to direct funding to projects
that address climate resiliency and equity.
Question(s) this activity will address:
• Are states effectively implementing the CWSRF program by leveraging non-federal funds?
• Are the states complying with EPA's State and Tribal Assistance Grant program
requirements?
• What steps are the states taking to promote climate resiliency and equity through CWSRF
funding?
Progress, results, and interim findings (if applicable): Results from the annual reviews are
documented on EPA's Clean Water State Revolving Fund Report website
rhttps://www.epa.gov/cwsrf/clean-water-state-revolving-fund-cwsrf-reportsl
Data, tools, method/analytical approach: Data are provided from each state CWSRF program
review that is conducted by EPA Headquarters and the Regions.
Dissemination of findings: The findings from the annual state reviews are documented in
Program Evaluation Reports, which are provided to EPA Headquarters by the Regional Offices.
EPA Headquarters periodically updates the guidance based on these findings. Revised guidance is
made available to states and stakeholders through EPA's CWSRF website.10
Title
Public Water System Supervision (PWSS) Program Reviews &
Drinking Water Infrastructure Revolving Fund State Reviews
Lead National Program
Office of Water
Strategic Goal and
Objective supported
Goal 5: Ensure Clean and Safe Water for All Communities.
Objective 5.1: Ensure Safe Drinking Water and Reliable Water
Infrastructure.
Planned start date
October FY Planned completion date Throughout FY 2024
2023
Purpose and brief Description: EPA annually conducts reviews of agencies with Public Water
System Supervision (PWSS) primacy (55 reviews) and reviews of each state's Drinking Water
State Revolving Fund (DWSRF) program.
Programmatic or policy decisions this activity will inform: These reviews assess if primacy
entities are effectively implementing the PWSS program to oversee community water system
10 Additional information may be found at: https://www.epa.gov/cwsrf.
-------
compliance with the Safe Drinking Water Act (SDWA) and assess if states are effectively
implementing the DWSRF program to facilitate community water system compliance with the
SDWA.
Question(s) this activity will address:
• Are primacy entities effectively implementing the range of activities in the PWSS program
to oversee community water system compliance with the SDWA?
• Are states effectively implementing the DWSRF program to facilitate community water
system compliance with the SDWA and complying with EPA's State and Tribal Assistance
Grant program requirements?
Data, tools, method/analytical approach: Data is provided via program review reports by
agencies with primacy for the PWSS program. The reports are reviewed by EPA, and include
elements such as state use of the funds and the associated outcomes, compliance, and
implementation of SDWA regulations, alignment of the program with national enforcement and
compliance priorities, and public communication efforts.
DWSRF data is provided from each state DWSRF program review conducted by EPA
Headquarters and the Regions.
Dissemination of findings: EPA's regional offices engage and share results with primacy agencies
under their purview. EPA shares PWSS information on water system compliance rates across and
within states. EPA makes publicly available an annual report on the status of the national DWSRF
program. EPA also shares project and financial data at the national and state level. The most recent
annual report, 2019 DWSRF annual report, is available here:
https://www.epa.gov/sites/default/files/2020-
10/documents/2 inual report final 508compliant.pdf.
Title
Safe Drinking Water Information System (SDWIS) National
Community Water System Non-Compliance Review
Lead National Program
Office of Water
Strategic Goal and
Objective supported
Goal 5: Ensure Clean and Safe Water for All Communities.
Objective 5.1: Ensure Safe Drinking Water and Reliable Water
Infrastructure.
Planned start date
October FY 2023 Planned completion date Throughout 2024
Purpose and brief Description: EPA conducts a review quarterly of the Safe Drinking Water
Information System (SDWIS) National Community Water System (CWS) health-based non-
compliance data.
Programmatic or policy decisions this activity will inform: This review assesses the trends and
causes of non-compliance to inform technical, managerial, and financial state and public water
system capacity building training or future drinking water regulation needs, in support of
regulatory drinking water compliance.
Question(s) this activity will address:
-------
• What are the barriers and challenges of CWS systems maintaining compliance with health-
based drinking water standards?
Data, tools, method/analytical approach: Data are provided from EPA's SDWIS database.
There is a non-compliance review of CWS systems with health-based violations by regulation
type, geographical distribution, and system source type.
Anticipated challenges and proposed solutions: There are no anticipated challenges.
Dissemination of findings: The findings from the program reviews will be publicly shared.
Quarterly data reports are shared publicly via the SDWIS FED Data Warehouse11.
11 For additional information, please visit: https://ofmpub.epa.gov/apex/sfdw/f?p=108%3A200%3A%3A%3A%3A%3A%3A.
------- |