ESTUARY PROGRAM National Estuary Program Program Evaluation Guidance Updated December 5, 2023 i ------- National Estuary Program Program Evaluation Guidance Table of Contents Section I: Purpose, Updates, and Goals 3 A. Purpose 3 B. 2021 Updates to the Program Evaluation Guidance 3 C. Goals 5 Section II: Scope and Format of the Program Evaluation 6 A. Format for PE Narrative Submission 6 B. On-Site Visit 12 C. Program Evaluation Findings 13 1. Criteria for Determinations 13 2. Program Evaluation Improvement Timeline 16 Section III: Program Evaluation Process 17 A. Program Evaluation Team Structure 17 B. Responsibilities for the Parties Involved in the Program Evaluation Process 17 C. Program Evaluation Team Responsibilities 19 Section IV: NEP Groups and Program Evaluation Schedule 20 A. NEP Groups 20 B. Program Evaluation Schedules 21 1. 2022: Group A Program Evaluation Schedule 21 2. 2023: Group B Program Evaluation Schedule 23 3. 2024: Group C Program Evaluation Schedule 25 4. 2025: Group D Program Evaluation Schedule 27 Appendix A: History of Program Evaluation Guidance Revisions 29 Appendix B: NEP Reference Documents 30 2 ------- Section I: Purpose, Updates, and Goals A. Purpose The primary purpose of the Program Evaluation (PE) process is to help the U.S. Environmental Protection Agency (EPA) assess how the National Estuary Programs (NEPs) are making progress in achieving programmatic and environmental results through implementation of their Comprehensive Conservation and Management Plans (CCMPs). The PE results are used cumulatively to help the national program demonstrate its impact and help design the next generation of CCMPs and programs. The PE process has proven to be an effective, interactive management process that ensures national program accountability and transparency, while incorporating local priorities considerations. It also demonstrates the value of federal investment in estuarine and coastal watershed restoration and protection at the local and regional levels. The PE process is also useful for: • transferring lessons learned among NEPs, EPA, and stakeholders through the sharing of case studies and transferable examples; • documenting the value added to environmental management by the national program and individual NEPs, including their role in convening stakeholders and interpreting science for management; • demonstrating continued stakeholder commitment; and • highlighting achievements and successes of each NEP, as well as suggestions for continued program improvements. B. 2021 Updates to the Program Evaluation Guidance In 2020-21, a PE Revision Team composed of EPA HQ staff and EPA NEP Regional Coordinators participated in a reevaluation process that led to the 2021 NEP Program Evaluation Guidance. One of the main goals of the PE Revision Team was to examine how the EPA assessed performance and whether the metrics and process adequately reflected an evolving program. Based on insights from that examination, certain aspects of the process were updated, modified, or eliminated. The updated guidance reflects these changes, including evaluation timing and reporting requirements. Some new areas of assessment were also identified and included in the guidance. The significant changes are described below: • The PE package submission was revised to include a narrative format with questions that require direct responses from the NEPs. These questions are mostly derived from the 2016 PE standardized performance measures. 3 ------- • The tiered self-assessment based on the standardized performance measures from the 2016 PE Guidance has been eliminated. The change to a PE Narrative Submission has been implemented to provide for more cumulative results of CCMP implementation. • The workplan elements in the PE Narrative Submission were broadened to three categories: 1) Healthy Ecosystems, 2) Clean Waters, and 3) Strong Communities, allowing the NEPs to report their workplan elements under larger, but still consistent categories, and at the same time enable the EPA to report on the NEP progress at a national scale. Appendix B lists the documents the EPA referenced when determining what information to request from the NEPs. • PE requirements are based on the NEP Funding Guidance 2021-2024 rather than the PE Logic Model developed in the 2007 PE Guidance. The NEP's workplan goals and activities responses no longer need to follow the PE Logic Model. • To shorten the PE review schedule and provide a longer amount of time to develop the PE package, the deadline for PE package submission is changed from March 15th to April 1st. If April 1st falls during a weekend, the due date for the PE package submission is the following Monday. • To align with the federal fiscal year (Oct. 1 - September 30) schedule, the PE review period is in transition from the state fiscal year that ends in June to the federal fiscal year that ends in September. Therefore, the review period for this five-year cycle will start in July of the first reporting year and end in September of the last reporting year. By ending with a September timeframe, the next PE cycle will be consistent with the federal fiscal year with an October start date. This schedule will enable more accurate NEPORT reporting for habitat and leveraging. By 2027, all PE reviews will occur based on the federal fiscal year schedule, and updated PE schedules will be issued. • On-site visits are preferred, if possible, due to the value added by visiting the NEP in person. Site visits should include a Management Conference or board meeting whenever possible. Virtual site visits may be considered under certain circumstances and on a case-by-case basis if agreed upon by all PE team members and the NEP being evaluated. • A two-category (Proficient and Progressing) determination has been introduced, and the Fail determination from the previous final review criteria has been eliminated. For definitions of the terms "Proficient" and "Progressing," refer to Section II. C. The two- category determination focuses on continuous improvement and makes sure each NEP becomes the strongest program possible. • Rather than identifying "challenges," we will now use the term "opportunities for improvement." All PEs will still identify such opportunities for continuous improvement for the NEP. • Progressing NEPs will have a PE Improvement Timeline to address their opportunities for improvement, including unmet or underperforming requirements, prior to their next evaluation period. Any unmet or underperforming requirements that have not been 4 ------- resolved before the NEP's next evaluation period may be identified during the EPA's award of the CWA 320 NEP grants (base and Bipartisan Infrastructure Law funds). The time between site visits and the PE letter is shortened to expedite the PE schedule by 1.5 months. The timeline includes deadlines the PE teams should meet to ensure the PE process is timely and efficient. For PE letter deadlines, refer to Section IV.B. C. 2023 Updates to the Program Evaluation Guidance In the fall of 2023, updates were made to the 2021 PE Guidance to include reporting on Bipartisan Infrastructure Law accomplishments and budget, consistent with language used in the NEP Bipartisan Infrastructure Law Implementation Memo. The NEPs will receive Bipartisan Infrastructure Law funding from 2022-2026, but projects may continue past those years as the funds are no-year. D. Goals As noted above, the PE process aims to demonstrate the value of a sustained National Estuary Program and its premier status as an efficient and effective delivery system for EPA programs at the local level. The goals of this PE Guidance are twofold: At the national level, to: • consistently document the cumulative value added to environmental management by the national program and individual NEPs, including their role in convening stakeholders and interpreting science for management; and • capture the return on the federal investment in estuarine and coastal watershed restoration and protection at the local and regional levels. At the program level, to • ensure that submissions enable objective and consistent evaluations among the different NEPs; • ensure a consistent and transparent process to determine NEP CCMP implementation progress; • further align the PEs with individual NEP CCMP priorities and related NEP annual work plan goals and accomplishments; • determine progress in achieving programmatic and environmental results by documenting NEP contributions to improving or reducing pressures on their coastal watersheds. This will enable all NEPs to successfully serve as local implementation partners for EPA programs; and • identify areas of improvement to assist NEPs in becoming stronger programs and achieving environmental results. 5 ------- Section II: Scope and Format of the Program Evaluation The PE consists of several phases: A) development and submission of a package of required information, B) PE team site visit to each NEP under evaluation, and C) documentation of PE findings via formal letter from EPA HQ. A. Format for PE Narrative Submission The PE Narrative Submission should report on the NEP's five-year evaluation period and include a concise, five-year cumulative self-reflection on the three key topics: 1) NEP Environmental/Programmatic Workplan Accomplishments, 2) NEP Program Implementation, and 3) NEP Ecosystem and Community Status. Please follow the format outlined for this PE Narrative Submission rather than organizing in accordance with individual NEP CCMP structures. The PE Narrative Submission should be concise but fully address all the topics and associated questions clearly, providing details about how progress and outcomes are being achieved rather than yes/no responses. Use quantitative metrics and results where possible. Every topic listed should be addressed in enough detail to fully describe what was accomplished, along with whatever substantiating data are necessary. Because of the changes made to streamline the process, we anticipate this will dictate a shorter PE Narrative Submission with a suggested length of approximately 30-50 pages. Each NEP's PE Narrative Submission should report information from their evaluation period: • Group A/2022: July 1, 2016 - September 30, 2021 • Group B/2023: July 1, 2017 - September 30, 2022 • Group C/2024: July 1, 2018 - September 30, 2023 • Group D/2025: July 1, 2019 - September 30, 2024 Topic 1: NEP Environmental / Programmatic Workplan Accomplishments: The following information will illustrate the environmental and programmatic progress the NEP has made towards achieving the goals and objectives identified in its CCMP through selected workplan accomplishments. • Workplan Activities: CWA §320 (Base) Funds: The NEP's long-term goal, as specified in the CWA §320, is to restore and maintain the ecological integrity of estuaries of national significance by addressing the quantity and quality of their waters, habitats, and living resources with the goal of making waters fishable and swimmable. The NEP should evaluate where its 6 ------- individual workplan goals and activities fit into each of the following three categories and reflect on the key accomplishments during the review period, specifying the NEP's role. These categories are broad to allow each NEP to fit its individual workplan goals and activities more easily into each of them. Please consider how climate resiliency plays a role in these areas as appropriate. o Healthy Ecosystems: e.g., fish, shellfish, plant, eelgrass, and wildlife populations; habitat protection/restoration, natural resources, land use, hydrological and ecological restoration, invasive species o Clean Waters: e.g., point and non-point sources of pollution, nutrients, harmful algal blooms (HABs), pathogens, ocean and coastal acidification, toxics, stormwater runoff, green infrastructure, sediments, plastics and other litter, TMDL implementation, freshwater flow, quality and quantity of drinking water sources o Strong Communities: e.g., environmental justice, water equity, investment of resources into the local economy, technical assistance to communities, long- term stakeholder relationships, integration of projects into the fabric of the community, conservation of water, land, and energy When describing the environmental and programmatic workplan accomplishments, consider the instructions in the NEP Funding Guidance 2021-2024: "include significant outputs and outcomes, examples of transferable activities and tools, and whether CCMP goals were achieved... [This] should include important key environmental and programmatic accomplishments, completed workplan activities, and highlight noteworthy lessons learned. NEPs should highlight success stories from the [past five] years, including but not limited to quantifiable outcomes related to workplan goals (see above) and the priority areas of special interest (listed in Topic 3). This information is needed to illustrate each NEP's CCMP implementation progress and should be presented in enough detail to fully describe what was accomplished, along with whatever substantiating data are available." To further demonstrate accomplishments to workplan goals and activities for the PE cycle, the EPA HQ PE team leader will provide the team members the following information from NEPORT or the NEP Qlik site. • A cumulative total of the funds leveraged for the years covered in the PE cycle, and breakdown by primary and significant leveraging roles. • A cumulative total of acres protected and restored, and breakdown by habitat type for the years covered in the PE cycle. • A cumulative total of the funds leveraged for the years covered in the PE cycle, and breakdown by primary and significant leveraging roles. In addition; 7 ------- • Workplan Activities: Bipartisan Infrastructure Law (Bipartisan Infrastructure Law) Funds: The Bipartisan Infrastructure Law provides $132 million to 28 longstanding NEPs for fiscal years 2022 through 2026. A core emphasis of the NEP Bipartisan Infrastructure Law funding is the acceleration of environmental and community goals within the CCMPs. NEP projects funded through Bipartisan Infrastructure Law should seek to: o Accelerate and more extensively implement CCMPs: Multi-year Bipartisan Infrastructure Law funds provides an opportunity for NEPs to execute long-term projects within communities they serve, leverage additional resources, and work with stakeholders to advance a wide range of projects identified in CCMPs. o Ensure that benefits reach disadvantaged communities: Ensure that the benefits of federal investments are shared equitably by communities benefiting from estuary program projects. The Bipartisan Infrastructure Law NEP funds are covered under the Justice40 initiative, and the national program as a whole has a target of ensuring that at least 40 percent of the benefits from Bipartisan Infrastructure Law flow to disadvantaged communities. Each estuary program was required to develop an Equity Strategy outlining approaches to contribute to the nationwide NEP Justice40 target. o Build the adaptive capacity of ecosystems and communities: NEPs should use Bipartisan Infrastructure Law resources to continue to expand on their climate change adaptation, hazard mitigation, and resilience activities, where appropriate, including protection and restoration of key habitats that increase resiliency. o Leverage additional resources: As NEPs select Bipartisan Infrastructure Law-funded projects, the EPA expects estuary programs to collaborate with other federal agencies and new partners and identify opportunities to leverage these funds. The NEP should evaluate where its individual Bipartisan Infrastructure Law workplan goals and activities fit into each of the following three categories and reflect on the key accomplishments during the review period, specifying the NEP's role. These categories are broad to allow each NEP to fit its individual workplan goals and activities more easily into each of them. Please consider how climate resiliency plays a role in these areas and how benefits flow to disadvantaged communities, as appropriate. o Healthy Ecosystems: e.g., fish, shellfish, plant, eelgrass, and wildlife populations; habitat protection/restoration, natural resources, land use, hydrological and ecological restoration, invasive species o Clean Waters: e.g., point and nonpoint sources of pollution, nutrients, harmful algal blooms (HABs), pathogens, ocean and coastal acidification, toxics, stormwater runoff, green infrastructure, sediments, plastics and other litter, TMDL implementation, freshwater flow, quality and quantity of drinking water sources 8 ------- o Strong Communities: e.g., environmental justice, water equity, investment of resources into the local economy, technical assistance to communities, long-term stakeholder relationships, integration of projects into the fabric of the community, conservation of water, land, and energy Topic 2: NEP Program Implementation: The following information will describe the NEP's organizational and operational health and functionality to demonstrate a successful path forward for overcoming challenges and achieving current and future CCMP goals. Please fully address all the topics and associated questions clearly, providing details about how progress and outcomes are being achieved rather than yes/no responses. • NEP Administration and Governance Structure o How does the NEP organizational structure provide a clear and transparent decision-making process for actions based on both stakeholders' priorities and good science, facilitate decision-making autonomy for the Management Conference from the host entity, and allow the NEP to be seen as a leader in watershed management? How is the NEP ensuring that its Management Conference includes input from diverse populations and interests? o How do the NEP's staffing structure and planning promote stability and continuity of succession within the organization? o How does the NEP plan to continue operations during emergencies? o Highlight particularly beneficial characteristics as well as areas for improvement. • Grant Obligations and Finance o Has the NEP consistently met all its EPA §320 (Base) and Bipartisan Infrastructure Law grant obligations? o Have there been any challenges or problems encountered with cost sharing or implementing its federal NEP award? o What were the sources of the required non-federal cost share of the NEP award? o Have grant dollars been drawn down promptly in accordance with the terms and conditions of the grant for implementation of the EPA-approved workplan? o Are there strategies in place for obtaining additional funding beyond the EPA §320 (Base) and Bipartisan Infrastructure Law funds to implement CCMP actions (i.e., financial strategy)? o Highlight particularly successful efforts and approaches as well as unique institutional challenges or difficulties in obtaining funding. • Budget Summary o CWA §320 (Base) Funds: Provide a five-year tabular or graphic budget summary with an accompanying brief narrative showing how the EPA funding and match 9 ------- has been used since the last review (specific projects, NEP staffing, and other activities). Budget summaries that vary from the PE review period due to a state fiscal schedule constraint may be considered on a case-by-case basis if agreed upon by all PE team members and the NEP being evaluated. o Bipartisan Infrastructure Law Funds: Provide a tabular or graphic budget summary with an accompanying brief narrative showing how the Bipartisan Infrastructure Law funds have been used and leveraged since the initial grant award of Bipartisan Infrastructure Law funds until the end of the review period cycle. • Opportunities for Improvement and NEP Priorities o How has the NEP addressed challenges (referred to in this guidance as opportunities for improvement) identified in the previous PE? o What kind of obstacles, if any, has the NEP faced with CCMP implementation (political, institutional, etc.) and what has the NEP done to overcome those obstacles? How can the EPA (Regions/HQ) support the NEP's efforts to address these obstacles? o How can the EPA (Regions/HQ), states, and/or other Federal agencies support the NEP's efforts to address climate and equity priorities more effectively? o What difficulties or priorities does the NEP anticipate during the next five years? Topic 3: NEP Ecosystem and Community Status: The following information shows how the NEP applies and connects the everyday work of the NEP with the foundational goals of the CWA and the EPA priorities for achieving them. Please fully address all the topics and associated questions clearly, providing details about how progress and outcomes are being achieved rather than yes/no responses. • Community and Stakeholders Engagement o How does the NEP ensure that the public has access to the decision-making process and engagement opportunities? o How has the NEP engaged the variety of community members and stakeholders in the NEP study area, including in underrepresented segments? o What is the level of engagement from the stakeholders and the public? o Where and how could the level of engagement be improved? • Education and Outreach o How is the NEP effectively promoting and creating widespread recognition of the Program? o What are some of the impacts of outreach and educational activities? 10 ------- o What are some ways these activities could be improved? • Monitoring and Assessment o How do the NEP's monitoring plan and indicators produce data to support a comprehensive and integrated analysis of environmental conditions (e.g., environmental progress report that communicates ecosystem status and trends, aka State of the Bay/Estuary Reports)? o How does the NEP use monitoring results to re-direct management actions and programs implemented under the CCMP? o How are research efforts used to identify missing data that warrant additional monitoring or sampling (if applicable)? • Clean Water Act Programs Relationship o How does the NEP support the goals of the CWA? Highlight the best examples not already identified in previous sections. An example does not need to be provided for each CWA Program listed below. CWA Programs include but are not limited to: o Strengthening Water Quality Standards o Improving Water Quality Monitoring o Developing Total Maximum Daily Loads o Controlling Nonpoint Source Pollution on a Watershed Basis o Strengthening National Pollutant Discharge Elimination Systems (NPDES) Permits o Supporting Sustainable Wastewater Infrastructure • EPA Priorities o How does the NEP incorporate relevant aspects of the EPA priorities into its workplans consistent with locally generated concerns? Highlight the best examples of where the NEP has made collective impacts not already identified in previous sections related to CWA §320 (Base) funds or Bipartisan Infrastructure Law funds. An example does not need to be provided for each priority area listed. The EPA's Office of Water also has several priority areas of interest relevant to the NEPs, included in the NEP Funding Guidance 2021-2024: o Reduction in nutrient pollution and HABs o Water reuse and conservation o Marine litter reduction o Green infrastructure 11 ------- Additional EPA priority areas of national interest, especially for Bipartisan Infrastructure Law funds, include: o Environmental justice o Climate change The PE package should be submitted electronically by the NEP Director to the entire PE team and should contain the following: • Narrative Submission (provided by NEP Director), • NEPORT data (provided by PE team lead), • Annual workplans (provided by PE team lead), and • EPA required annual end of year reports for the PE review period (provided in coordination with PE team lead and Regional PE team member). Other documents that are being summarized in the Narrative Submission may be submitted as attachments. The NEP may use its professional judgment to decide which documents best showcase the NEP's progress toward accomplishing workplan and CCMP goals. For more details on submission schedules, see Section IV.B. If the PE review team cannot determine that an NEP is making adequate progress towards implementing its CCMP based on the PE package (See Figure 1 below summarizing the criteria for determining progress), the NEP can provide supplemental documentation addressing specific questions or information gaps identified by the PE review team. B. On-Site Visit The on-site visit is an opportunity for the EPA to gain a greater understanding of the NEP, including sharing programmatic and administrative projects and discussing initial evaluation findings with the NEP along with Management Conference members. The visits are typically one to three days in length. On-site visits also provide opportunities for the PE team (PE team leader, EPA Regional PE team member, and Ex-officio NEP Director PE team member) to view on-the-ground projects and view NEP collaboration with partners. Refer to Section III.B for additional role definition. The PE team will work with the NEP Director to determine the best time to schedule the visit, preferably synchronized with NEP Management Conference meeting(s). The site visit can be made virtual if agreed upon by all PE team members and the NEP being evaluated. On-site visits are preferred if possible due to the values and benefits associated with seeing the NEP in person. The NEP should use the PE on-site visit to: 12 ------- • demonstrate successes and accomplishments, especially those that are innovative and have technical transfer possibilities; • showcase their partners and examples of connected leadership; • showcase on-the-ground projects; • expand upon workplan progress summarized in the workplan section of the Narrative Submission; • describe opportunities for potential improvements to CCMP implementation (political, institutional, etc.); • describe potential opportunities for meeting Justice40 targets; and • demonstrate how external factors may be influencing progress toward environmental milestones and targets. The PE team members should use the on-site visit to: • meet and build relationships among the EPA and NEP partners to help inform national program policies and direction and identify potential new resources; • listen to NEP Policy and/or Management Conference members, stakeholders, and partners share their insights as well as any support and/or and concerns about CCMP implementation; • view on-the-ground projects; • discuss any outstanding questions or issues with submission of PE materials with the NEP; • discuss preliminary findings (strengths and opportunities for improvement) with the Management Conference; and • identify and discuss program recommendations with NEP, if feasible. C. Program Evaluation Findings 1. Criteria for Determinations The PE team will make a final determination based on the Figure 1 criteria informed by the entire PE package (narrative submission, NEPORT data, annual workplans, and the EPA required annual end of year reports), on-site visit, and discussions with the NEP under review. The 2021 PE Guidance revision introduces a two-category final PE letter determination, "Proficient" and "Progressing" toward proficiency, and it eliminates the Fail category from the final review determination. Each NEP will receive a final PE letter with a determination of Proficient or Progressing at the end of the PE process. A Proficient rating is defined as a NEP meeting all requirements identified in Figure 1, column 1. A Progressing rating is defined as a NEP with 13 ------- missing or underperforming elements (based on the PE team's best professional judgment) from Figure 1, column 1, as defined in column 2. Proficient NEPs will be re-evaluated on these criteria at the next five-year PE cycle. If a NEP is missing elements from the Proficient criteria, a NEP may receive a Progressing rating, catalyzing a timeline for improvement to address those missing elements or opportunities for improvement before the next PE cycle. The goal of the new Proficient and Progressing determinations is to focus on continuous improvement and make sure each NEP has the strongest program possible. Figure 1. PE Findings Proficient Criteria Progressing Criteria • Funding Guidance 2021-2024 requirements are met: o All required elements of the workplan are addressed. o CCMPs and associated documents (finance, monitoring, habitat, and education and outreach plans) are up to date. o Timely Obligations and Expenditure of CWA §320 (Base) Grant and Bipartisan Infrastructure Law funds. o Reporting shows environmental results and return on investment (Environmental Results and Leveraged Resources through NEPORT). In addition: * Narrative Submission elements are answered in a way that makes it clear to the PE team members that: o Challenges from the previous PE cycle are addressed. • If any one of the Funding Guidance 2021-2024 requirements criteria listed in the Proficient column are missing or consistently underperforming, it will prompt the PE team to grant a Progressing rating to the NEP. OR • If the NEP hasn't addressed the challenges identified in the previous PE cycle, it will prompt the PE team to grant a Progressing rating to the NEP. OR • If the answers to two or more of the Narrative Submission elements in the Proficient column are missing or underperforming, it will prompt the PE team to grant a Progressing rating to the NEP. A Progressing rating will prompt a PE Improvement Timeline for addressing missing elements before the next PE cycle. 14 ------- o Workplan goals are met and there is progress toward CCMP environmental milestones or targets. o NEP Administration directs day-to-day operations, fosters a positive work environment, and coordinates and supports Management Conference responsibilities. o NEP Governance Structure considers input from diverse perspectives, sets priorities to produce tangible improvements in the health of the ecosystem and can make decisions independently and operate autonomously from host entity to meet the Program's goals and objectives. o Community and Stakeholder Engagement fosters collaborative problem solving and sets a course for the NEP. o Education and Outreach efforts are promoting recognition of the NEP and advancing CCMP priority issues. o Monitoring and Assessment efforts inform decision- making, evaluate the effectiveness of actions, and produce data to support analysis of environmental conditions. o NEPs are successfully delivering and/or supporting relevant CWA Programs. 15 ------- o NEPs are supporting local priorities and, in addition, doing their best to support the EPA's current priorities, including those listed in the Funding Guidance. 2. Program Evaluation Improvement Timeline If an NEP receives a Progressing rating, this triggers a PE Improvement Timeline for the NEP to address its opportunities for improvement, including meeting unmet or underperforming requirements, before the next PE cycle. Any unmet or underperforming requirements that have not been resolved before the NEP's next evaluation period may be identified during the EPA's award of the CWA 320 NEP grants (base and Bipartisan Infrastructure Law funds). This provides accountability for the NEP's continuous improvement based on its unique set of opportunities for improvement or unmet requirements and appropriate individual timeline. The PE team has the flexibility to decide whether: 1) the PE letter includes the entire timeline for meeting requirements or 2) the PE letter proposes a timeline based on follow-up communication. The EPA should consider the input of the ex-officio team member when developing the PE Improvement Timeline. The Regional PE team member will lead the development of the timeline in close coordination with the PE team lead. The EPA Regions and the NEP will coordinate on the NEP's progress and keep HQ apprised. It is the goal of the EPA and the NEP to continue to work together to strengthen our programs. 16 ------- Section III: Program Evaluation Process A. Program Evaluation Team Structure The PE teams for each NEP will include o the PE team leader (EPA HQ NEP Coordinator), o the EPA Regional PE team member (EPA Regional NEP Coordinator), and, if possible, o an ex-officio NEP Director PE team member. The purpose of the ex-officio NEP Director in the PE process is to allow an exchange of information from one NEP to another (e.g., provide technical transfer assistance to the NEP undergoing the PE and take lessons learned back to their NEP). The EPA values the input and unique perspective that the ex-officio NEP Director brings to the PE process and will consider the input of the ex-officio as an additional information source in making final determinations. Responsibilities for the PE team members are outlined below in Section III.B. The EPA HQ and Regional Coordinators are expected to include PE on-site visits for their NEPs in their annual travel budget plans; however, travel for on-site visits is dependent on the availability of funds. Alternative arrangements in the absence of travel funds include webinars or video-conferences. The schedule and groupings for upcoming PEs and their associated on- site visits are presented in Section IV below. B. Responsibilities for the Parties Involved in the Program Evaluation Process EPA HQ: • Internal to the EPA, there is one HQ PE Coordinator who should: o oversee PE process for all PE cycles o set up PE teams including the call for volunteer ex-officio NEP Directors and distribute NEP PE Guidance • The PE team leader (EPA HQ NEP Coordinator) should: o help interpret the PE Guidance and/or provide feedback on the draft PE package o schedule conference calls with members of the team and the NEP Director o send electronic copies of the NEP workplans for the years covered within the PE cycle to the members of the team o review and comment on the NEP PE package in coordination with the rest of the PE team 17 ------- o collect electronic comments from members of the team o share written comments with the NEP Director, with a cc to the EPA Regional PE team member and ex-officio NEP Director PE team member, prior to the conference call with the NEP Director o coordinate and actively participate in the on-site visit o lead the presentation of preliminary findings during the on-site visit o draft the PE letter in collaboration with the EPA Regional PE team member and the ex-officio NEP Director PE team member o help develop the PE Improvement Timeline if one is required o provide input to PE Improvement Timeline development of "missing" Narrative elements if required o send the PE letter to all PE team members for review and signature by the EPA management o send final PE letter to the NEP EPA Regions: • The EPA Regional PE team member (EPA Regional NEP Coordinator) should: o help interpret the PE Guidance and/or feedback on the draft PE package o actively participate in conference calls o review and assist the PE team in interpreting the NEP PE package o submit electronic comments to the PE team leader o coordinate and actively participate in the on-site visit o participate in the presentation of preliminary findings during the on-site visit o help draft the PE letter o review, comment and concur with the final draft PE letter o help develop the PE Improvement Timeline if one is required o provide input to PE Improvement Timeline development of "missing" Narrative elements if required NEPs: • The ex-officio NEP Director PE team member should: o provide technical transfer assistance to the NEP, such as help addressing common challenges through discussion at the on-site visit and through review of the PE package and PE letter 18 ------- o volunteer in November to serve as ex-officio and identify which NEP they think is a good fit for them to review o actively participate in conference calls o review the NEP PE package o submit written comments to the PE team leader o coordinate and actively participate in the on-site visit o have a speaking role at the presentation of preliminary findings during the on- site visit o offer unique perspective and present shared experiences or challenges o provide technical transfer assistance to the NEP undergoing the PE o review and comment on the draft PE letter o help develop the PE Improvement Timeline if one is required o provide input to PE Improvement Timeline development of "missing" Narrative elements if required • NEPs undergoing the PE should: o prepare and submit the PE package to the entire PE Team by April 1st o participate in conference calls o address the PE team comments and provide any additional information requested by the PE team o coordinate and host the NEP on-site visit which may include a Management Conference or board meeting whenever possible. C. Program Evaluation Team Responsibilities The PE team is responsible for reviewing materials consistent with this guidance, planning the site visits, and developing preliminary findings, ideally to present to the NEP during the site visit. Evaluations will be based on the following: • Program Evaluation Narrative Submission and supporting documentation (see Section II.A). A qualitative assessment will be done on the PE Narrative Submission that includes discussion of key NEP workplan goals and activities to help implement the CCMP goals and objectives and contribute to environmental improvements. This assessment will include attention to programmatic, ecosystem, and community status. • Observations made during the on-site visit, and team discussions with NEP staff, Management Conference, stakeholders, and partners. The on-site visit will be used to discuss any questions or issues with the PE submission, discuss preliminary findings, and explore possible recommendations. It is a chance to visit project sites and meet with 19 ------- NEP Policy and/or Management Conference members, stakeholders, and partners to listen to their insights and concerns about CCMP implementation. Collaboratively, the PE team will document its findings in writing based on comments provided by each PE team member (electronic comments will be submitted to the PE team leader by each PE team member). The findings will reflect the PE package, the on-site visit, and discussions with the NEP, including MC members, if applicable (optional). The PE team will develop a final PE letter for EPA management's signature based on the PE team's documented findings (see Section II.C). Section IV: NEP Groups and Program Evaluation Schedule A. NEP Groups NEPs are distributed in four groups (A, B, C, and D), as shown in Figure 2. If there are any changes to these groups, the EPA will notify the NEPs by October of the year before the scheduled PE in Figure 2. The PE schedule for each group is shown below in Section IV.B. Figure 2: PE Groups A-B-C-D PE Group A PE Group B PE Group C PE Group D PE:2022 PE: 2023 PE:2024 PE: 2025 (6 Programs) (8 Programs) (7 Programs) (7 Programs) Barataria- Tampa Bay Buzzards Bay Coastal & Heartland Terrebonne Casco Bay Coastal Bend Bays Partnership for the Delaware Estuary Morro Bay Indian River Lagoon Albemarle-Pamlico Long Island Sound Lower Columbia Estuary Massachusetts Bays Delaware Inland Bays Santa Monica Bay Barnegat Bay San Juan Bay Narragansett Bay New York /New Jersey Harbor Piscataqua Region Estuaries Partnership Tillamook Bay Sarasota Bay Puget Sound San Francisco Bay Estuary *Galveston Bay Maryland Coastal Bays Mobile Bay **Peconic Bay *Galveston Bay will return to its original Group B in 2023. **Peconic Bay will return to its original Group A in 2027. 20 ------- B. Program Evaluation Schedules Below are the schedules for Groups A-D (2022-2025). It is highly recommended to have an individual PE team schedule set by the end of January so that all PE team members and the NEP have an advanced idea of what the year will look like and so that the on-site visit is aligned with the Management Conference or board meetings early on. These dates can be changed down the line if necessary. 1. 2022: Group A Program Evaluation Schedule Period covered: July 1, 2016 - September 30, 2021 NEPs: Barataria-Terrebonne, Casco Bay, Indian River Lagoon, Massachusetts Bay, San Juan Bay, Tillamook Bay November 5, 2021 November 12, 2021 NEP Directors should determine whether they can volunteer to serve on a PE team and notify the PE Coordinator at EPA HQ. EPA HQ will set up PE teams for Group A NEPs. December 17, 2021 Deadline for PE team leader to hold a conference call with members of the team and the NEP Director to discuss logistics on preparing and submitting the PE package, among other issues. Initiate conversation on individual PE schedules (conference calls with PE team members, conference calls with the NEP Director, and the on-site visit). January 14, 2022 Deadline for PE team leader to finalize an individual PE schedule with firm dates for calls and the on-site visit, preferably synchronized with Management Conference or board meetings. April 1, 2022 Due date for PE package submittal. An electronic copy should be supplied to each member of the PE team (PE team lead, Region, ex- officio NEP Director). The EPA HQ Coordinator is responsible for making a file copy. The PE team leader sends electronic copies of NEP workplans covered during the review period (FY16 - FY21) to the ex-officio NEP Director. April 15, 2022 Deadline for PE team leaders to hold a conference call for the PE team members to compare notes after reviewing the PE package and submitting written comments to the PE team leader. May 6, 2022 Deadline for PE team leaders to hold a conference call with the NEP Director and the PE team to discuss additional documentation needs 21 ------- and identify issues that should be addressed during the on-site visit. May 9 - 20, 2022 Period for PE team members and NEP Director to have further conversations, if needed, prior to the on-site visit. May 23-July 15, 2022 Period for on-site visits. The on-site visit should be scheduled to present preliminary findings to the Management Conference toward the end of the visit, when possible. LETTER DEVELOPMENT** Six weeks after the site visit is the deadline for final PE letter submission. Here is an example schedule: One week after visit Period to resolve any concerns raised during the on-site visit. Deadline for the team leader to prepare draft letter documenting the PE team's findings, recommendations, and rating. Two weeks after visit Deadline for PE team (Regional Coordinator and ex-officio NEP Director) to review and provide comments on draft letter. Three weeks after visit PE team reviews updated draft letter to submit to NEP Director. Four weeks after visit Deadline for NEP Director to review and provide comments on draft letter. Five weeks after visit PE team reviews updated draft letter to submit to PPB Branch Chief. Six weeks after visit Deadline for addressing outstanding issues and concurrence and signature by PPB Branch Chief. August 31, 2022 All seven PE letters should be signed and sent out to the respective NEPs. ** The schedule for the PE letter development assumes no major issues arise from the PE that requires extensive consultation. 22 ------- 2. 2023: Group B Program Evaluation Schedule Period covered: July 1, 2017 - September 30, 2022 NEPs: Tampa Bay, Coastal Bend Bays, Albemarle-Pamlico, Delaware Inland Bays, Galveston Bay*, Sarasota Bay, Narragansett Bay, **Peconic Bay November 4, 2022 November 11, 2022 December 16, 2022 NEP Directors should determine whether they can volunteer to serve on a PE team and notify the PE Coordinator at EPA HQ. EPA HQ will set up PE teams for Group B NEPs. Deadline for PE team leader to hold a conference call with members of the team and the NEP Director to discuss logistics on preparing and submitting the PE package, among other issues. Initiate conversation on individual PE schedules (conference calls with PE team members, conference calls with the NEP Director, and the on-site visit). April 3, 2023 Due date for PE package submittal. An electronic copy should be supplied to each member of the PE team (PE team lead, Region, ex- officio NEP Director). The EPA HQ Coordinator is responsible for making a file copy. The PE team leader sends electronic copies of NEP workplans covered during the review period (FY17 - FY22) to the ex-officio NEP Director. EXEMPTIONS: *Galveston Bay goes back to its original group. Review period covered (July 1, 2018 - September 30, 2022). Workplans covered during the review period (FY18 - FY22). **Peconic Bay was moved from Group A to B for this PE cycle. Review period covered (July 1, 2016 - September 30, 2022). Workplans covered during the review period (FY16 - FY22). April 21, 2023 Deadline for PE team leaders to hold a conference call for the PE team members to compare notes after reviewing the PE package and submitting written comments to the PE team leader. May 5, 2023 Deadline for PE team leaders to hold a conference call with the NEP Director and the PE team to discuss additional documentation needs, schedule the on-site visit, and identify issues that should be addressed during the on-site visit. May 8 - 19, 2023 Period for PE team members and NEP Director to have further 23 ------- conversations, if needed, prior to the on-site visit. May 22 - July 14, 2023 Period for on-site visits. The on-site visit should be scheduled to present preliminary findings to the Management Conference toward the end of the visit, when possible. LETTER DEVELOPMENT** Six weeks after the site visit is the deadline for final PE letter submission. Here is an example schedule: One week after visit Period to resolve any concerns raised during the on-site visit. Deadline for the team leader to prepare draft letter documenting the PE team's findings, recommendations, and rating. Two weeks after visit Deadline for PE team (Regional Coordinator and ex-officio NEP Director) to review and provide comments on draft letter. Three weeks after visit PE team reviews updated draft letter to submit to NEP Director. Four weeks after visit Deadline for NEP Director to review and provide comments on draft letter. Five weeks after visit PE team reviews updated draft letter to submit to PPB Branch Chief. Six weeks after visit Deadline for addressing outstanding issues and concurrence and signature by PPB Branch Chief. August 31, 2023 All seven PE letters should be signed and sent out to the respective NEPs. ** The schedule for the PE letter development assumes no major issues arise from the PE that requires extensive consultation. 24 ------- 3. 2024: Group C Program Evaluation Schedule Period covered: July 1, 2018 - September 30, 2023 NEPs: Buzzards Bay, Partnership for the Delaware Estuary, Long Island Sound, Santa Monica Bay, New York/ New Jersey Harbor, Puget Sound*, and Maryland Coastal Bays November 10, 2023 NEP Directors should determine whether they can volunteer to serve on a PE team and notify the PE Coordinator at EPA HQ. November 17, 2023 EPA HQ will set up PE teams for Group C NEPs. December 15, 2023 Deadline for PE team leader to hold a conference call with members of the team and the NEP Director to discuss logistics on preparing and submitting the PE package, among other issues. Initiate conversation on individual PE schedules (conference calls with PE team members, conference calls with the NEP Director, and the on-site visit). April 1, 2024 Due date for PE package submittal. An electronic copy should be supplied to each member of the PE team (PE team lead, Region, ex- officio NEP Director). The EPA HQ Coordinator is responsible for making a file copy. The PE team leader sends electronic copies of NEP workplans covered during the review period (FY18 - FY23) to the ex-officio NEP Director. EXEMPTION: *Puget Sound goes back to its original group. Its review period covered (July 1, 2019 - September 30, 2023). Workplans covered during the review period (FY19 - FY23). April 19, 2024 Deadline for PE team leaders to hold a conference call for the PE team members to compare notes after reviewing the PE package and submitting written comments to the PE team leader. May 10, 2024 Deadline for PE team leaders to hold a conference call with the NEP Director and the PE team to discuss additional documentation needs, schedule the on-site visit, and identify issues that should be addressed during the on-site visit. May 13 -24, 2024 Period for PE team members and NEP Director to have further conversations, if needed, prior to the on-site visit. May 27-July 12, 2024 Period for on-site visits. The on-site visit should be scheduled to present preliminary findings to the Management Conference toward 25 ------- the end of the visit, when possible. LETTER DEVELOPMENT** Six weeks after the site visit is the deadline for final PE letter submission. Here is an example schedule: One week after visit Period to resolve any concerns raised during the on-site visit. Deadline for the team leader to prepare draft letter documenting the PE team's findings, recommendations, and rating. Two weeks after visit Deadline for PE team (Regional Coordinator and ex-officio NEP Director) to review and provide comments on draft letter. Three weeks after visit PE team reviews updated draft letter to submit to NEP Director. Four weeks after visit Deadline for NEP Director to review and provide comments on draft letter. Five weeks after visit PE team reviews updated draft letter to submit to PPB Branch Chief. Six weeks after visit Deadline for addressing outstanding issues and concurrence and signature by PPB Branch Chief. August 30, 2024 All seven PE letters should be signed and sent out to the respective NEPs. ** The schedule for the PE letter development assumes no major issues arise from the PE that requires extensive consultation. 26 ------- 4. 2025: Group D Program Evaluation Schedule Period covered: July 1, 2019 - September 30, 2024 NEPs: Coastal & Heartland, Morro Bay, Lower Columbia Estuary, Barnegat Bay, Piscataqua Region Estuaries, San Francisco Estuary, Mobile Bay November 6, 2024 NEP Directors should determine whether they can volunteer to serve on a PE team and notify the PE Coordinator at EPA HQ. November 13, 2024 EPA HQ will set up PE teams for Group D NEPs. December 13, 2024 Deadline for PE team leader to hold a conference call with members of the team and the NEP Director to discuss logistics on preparing and submitting the PE package, among other issues. Initiate conversation on individual PE schedules (conference calls with PE team members, conference calls with the NEP Director, and the on-site visit). April 1, 2025 Due date for PE package submittal. An electronic copy should be supplied to each member of the PE team (PE team lead, Region, ex- officio NEP Director). The EPA HQ Coordinator is responsible for making a file copy. The PE team leader sends electronic copies of NEP workplans covered during the review period (FY19 - FY24) to the ex-officio NEP Director. April 18, 2025 Deadline for PE team leaders to hold a conference call for the PE team members to compare notes after reviewing the PE package and submitting written comments to the PE team leader. May 12 - 23, 2025 Period for PE team members and NEP Director to have further conversations, if needed, prior to the on-site visit. May 26-July 18, 2025 Period for on-site visits. The on-site visit should be scheduled to present preliminary findings to the Management Conference toward the end of the visit, when possible. LETTER DEVELOPMENT** Six weeks after the site visit is the deadline for final PE letter submission. Here is an example schedule: One week after visit Period to resolve any concerns raised during the on-site visit. 27 ------- Deadline for the team leader to prepare draft letter documenting the PE team's findings, recommendations, and rating. Two weeks after visit Deadline for PE team (Regional Coordinator and ex-officio NEP Director) to review and provide comments on draft letter. Three weeks after visit PE team reviews updated draft letter to submit to NEP Director. Four weeks after visit Deadline for NEP Director to review and provide comments on draft letter. Five weeks after visit PE team reviews updated draft letter to submit to PPB Branch Chief. Six weeks after visit Deadline for addressing outstanding issues and concurrence and signature by PPB Branch Chief. August 30, 2025 All seven PE letters should be signed and sent out to the respective NEPs. ** The schedule for the PE letter development assumes no major issues arise from the PE that requires extensive consultation. 28 ------- Appendix A: History of Program Evaluation Guidance Revisions 1997-2006 The EPA began a NEP Implementation Review (IR) process in 1997 to determine which NEPs with approved CCMPs qualified for continued funding. The IR process was initially conducted every two years. In 2000, the process was streamlined, and the review cycle was extended from every two to every three years for those programs that had already undergone the biennial review. In 2003, the IR process was revised with the intent to make IRs less burdensome to the NEPs while still collecting sufficient information to evaluate NEP progress and technical transfer. The IR cycle remained a three-year cycle. 2006-2016 In 2006, the IR process was reevaluated due to increased federal program accountability, e.g., the Program Assessment Rating Tool (PART). An IR Reassessment Team composed of EPA Headquarters (HQ) staff, EPA NEP Regional Coordinators, and NEP Directors participated in the reevaluation process that led to the 2007 NEP Program Evaluation Guidance. In 2011, the PE process was reassessed to identify further streamlining opportunities. The PE framework laid out in the 2007 PE Guidance was maintained. Still, the Tracking/Reporting sub-element under the Program Management element was eliminated except for two of the criteria that were transferred to the Outreach and Public Involvement sub-element. The PE cycle was also changed from a three-year cycle to a five-year cycle (four consecutive years with the fifth year to be spent producing a findings report). 2016-2020 In 2016, the PE process was reassessed again to identify opportunities for improvement. Changes were made to better define the EPA's expectations for the format of documentation for the PE package. NEPs were no longer expected to re-submit work plans with the PE package, report separately on CWA implementation, or report on habitat restored and protected and leveraged funds data (primary and significant role) using NEPORT. The EPA HQ will provide these cumulative totals for the years covered in the PE cycle. The element Tools was eliminated from the work plan core elements. Language is updated to better define the expectations for the on-site visit and the role of the ex-officio NEP Director. The timeline between the on-site visit and final PE letter was updated to expedite the communication of PE findings to the NEP Director. The PE cycle remained a five-year cycle (four consecutive years followed by one year off to prepare a PE finding report of all 28 NEPs). 29 ------- Appendix B: NEP Reference Documents The information requested in the PE package is derived from the following documents found on the NEP SharePoint and NEP Website: • 2016 Program Evaluation Guidance: provides process and criteria guidance for the requisite PEs that all NEPs must undergo periodically; • NEP Funding Guidance 2021-2024: guidance on annual workplans, reporting requirements, and major assistance agreement policies to the 28 NEPs; • NEP Funding Guidance 2017-2019: guidance on annual workplans, reporting requirements, and major assistance agreement policies to the 28 NEPs; • 2015 Frequently Asked Questions on National Estuary Program Governance: FAQs related to operations and requirements of the NEP; • 2005 Community-Based Watershed Management Handbook: handbook describing innovative approaches to watershed management by 28 NEPs; and • 1992 CCMP Content and Approval Guidance: guidance on the CCMP development and language for Monitoring, Finance, Outreach Strategies, and the Management Conference role. 30 ------- |