&EPA
United States
Environmental Protection
Agency
Office of Policy
(1807T)
July 2013
EPA-100-K-13-003
Evaluation of the
National Water Program
Climate Change
Adaptation Strategy
Final Report
July 22, 2013
prepared for:
EPA Evaluation Support Division
EPA Office of Water
prepared by:
Industrial Economics, Incorporated
Ross Strategic
Promoting Environmental Results
Through Evaluation
-------
EC
ACKNOWLEDGEMENTS
This report, Evaluation of the Office of Water Climate Change Strategy, was developed for the United
States Environmental Protection Agency's (EPA) Office of Policy under Contract EP-BPA-12-H-0035
between EPA and Industrial Economics, Inc. (lEc) of Cambridge, Massachusetts. The evaluation team
(also referred to as the "evaluators" throughout this report) consisted of Angela Helman, Tracy Dyke-
Redmond, Andrew Schwarz, John McKenzie, and Josh Wolff of lEc; and Tim Larson, Andy Chinn, and
Megan Parker of Ross Strategic.
Joel Corona of the Office of Water provided critical assistance and background information throughout
the course of this study. Britta Johnson and Carl Koch of EPA's Office of Policy Evaluation Support
Division provided technical support and advice for the evaluation.
This report was developed under the Program Evaluation Competition, sponsored annually by EPA's
Office of Policy. Program Evaluation is one of the performance management tools EPA uses to assure
itself, the public, and other interested stakeholders that EPA programs are protecting human health and
the environment effectively and efficiently. The information obtained through program evaluations can
shed light on whether EPA programs are meeting their goals and objectives, provide the evidence and
road map needed to replicate successes, and identify those aspects of EPA programs needing
improvement. To access copies of this or other EPA program evaluations, please visit EPA's Evaluation
Support Division's website at http://www.epa.gov/evaluate.
-------
lEc
TABLE OF CONTENTS
EXECUTIVE SUMMARY
CHAPTER 1 | INTRODUCTION AND BACKGROUND
NWPaiK ' -. '•; !,' ,' S 17
'••-'_• _• ,. 18
1 " • . menlalum 18
2 ' ^79
- ' .. • ' > ncnialioii To-! )alc 20
CHAPTER 2 | LOGIC MODELS
CHAPTER 3 | METHODOLOGY
Methodological Approach lor the Retrospectse Kuiluation 29
1 )aia Sources: Retrospective Pxaluation 29
Dala Analysis: Relrospeeliye pyalualion 31
Methodological Approach to Prospectne Performance Measurement 32
1 )aia Sources: Prospective Measui'enieni Approach 32
Dala Anah'sis: i'i'ospecli\'e Measui'enieni Approach 34
CHAPTER 4 | LESSONS LEARNED FROM EVALUATION OF 2008 STRATEGY
IMPLEMENTATION
Findings and Lessons 36
1 km \\cll is climate change mamstreamed into the N\\ P'.' 36
\\ hal goals, iniplenienlalion experience, or lessons Inun Ihe .,!(•«: 18 Slraleux' coukl be uselul So liuule iniplenienlalion
oftlie 2012 Ssralcu\ 42
\\liat uoals and strategic actions in she 2i'l2 Strateus slioulii l-'l'A lieadi|iiarters and regional programs prioritise'.' 56
CHAPTERS | PERFORMANCE MEASUREMENT FOR THE 2012 STRATEGY
OWs Approach So dale in Measuring Progress on Ihe 2
-------
lEc
CHAPTER 6 | CONCLUSIONS AND RECOMMENDATIONS
Roc 0 in in c n cl al ion s 90
Keinvitiorate N\\ !' management and sin! I commitment to the Climate Chantie Strateiiv 90
Create management practices that keep climate ehansie integration Iron! ami center 91
APPENDICES:
Appendix A: Connections in the Logic Models A-1
Appendix B: SummaiA of Strategic Action Baseline Data 6-7
Appendix C: SummaiA of Re ports and Online Resources that Describe Progress on CM male
Chan p,c A d a p I a I i o n c- 1
Appendix D: Categories of Strategic Actions D-1
Appendix t: Dala Consistence Qualih E-1
111
-------
EC
EXECUTIVE SUMMARY
The National Water Program Strategy: Response to Climate Change was first published in 2008. OW
created one of the first climate change strategies in the Agency, and is considered a model for other
emerging efforts within the federal government. The 2008 Strategy describes the likely effects that
climate change will have on water resources and implications for EPA's Office of Water (OW) and the
National Water Program (NWP). In 2012, OW developed an updated strategy that describes NWP's long-
term goals for sustainably managing water resources in light of climate change. It is intended as a
roadmap to guide future program work and inform the Agency's annual planning process. In late 2012,
OW's Immediate Office (IO), which is responsible for coordinating OW climate change work, requested
support from EPA's Evaluation Support Division (ESD) to conduct a retrospective evaluation of lessons
from the 2008 Strategy, and to develop a prospective measurement framework to track the progress on
the 2012 Strategy. Industrial Economics, Incorporated (lEc) and its subcontractor Ross Strategic,
henceforth referred to as the evaluation team, carried out the study. This report presents the results of
these coordinated efforts.
EVALUATION QUESTIONS
The project was guided by the following research questions:
Evaluating Implementation of the 2008 Strategy:
A How well is climate mainstreamed into OW programs? What are the barriers to mainstreaming and
how might this be better accomplished?
A What goals, implementation experience, or lessons from the 2008 Strategy could be useful to guide
implementation of the 2012 Strategy?
-^ What goals and strategic actions in the 2012 Strategy should EPA headquarters (HQ) and regional
programs prioritize?
Developing Prospective Measurement Approach for the 2012 Strategy:
A What is the measurement approach that can be used to measure adaptation progress in five areas:
infrastructure, watersheds and wetlands, ocean and coastal waters, water quality, and working with
tribes?
A What specific elements need to be applied to the phased approach to tracking progress outlined in
the 2012 Strategy, to make it a robust measurement framework?
A What, if, any, revisions should EPA make to its baseline data collection process to ensure that data
collected are meaningful and objective?
-^ How can OW's measurement approach inform measuring progress in the EPA-wide Adaptation Plan,
and to inform development of the next Agency Strategic Plan?
DATA SOURCES AND APPROACH
This study draws on multiple data sources to inform the lessons learned related to our research
questions. Key sources of information include:
-------
EC
A Interviews with EPA representatives from IO and from all four OW program offices and several
regional offices, as well as external organizations. An EPA interview summary is presented in Exhibit
ES-1 below; in total the evaluators interviewed 26 EPA staff and 4 non-EPA staff. The evaluators
spoke with staff in all four OW program offices as well as staff from four different regions.
A An EPA focus group to discuss lessons from the 2008 Strategy and recommendations for
implementation of the 2012 Strategy. Most focus group participants are also among the
interviewees discussed above.
A Review of documents related to the NWP climate change strategies and their implementation,
including EPA memoranda and other policy documents.
A Review of baseline data collected by IO to assess progress towards the strategic actions included in
the 2012 Strategy.
A Review of existing literature and online publications that address climate change adaptation
strategies and activities.
EXHIBIT ES-1. EPA INTERVIEW SUMMARY
EPA OFFICE/REGION
OW/IO
OAR
OGWDW
OP
ORD
OST
OWM
OWOW
Region 1
Region 3
Region 6
Region 10
Total
NUMBER OF INTERVIEWEES
5
2
5
1
1
1
4
3
1
1
1
2
27
LESSONS FROM IMPLEMENTATION OF THE 2008 STRATEGY
The findings and lessons from the retrospective evaluation of the 2008 Strategy are presented in
Chapter 4 and are summarized below by evaluation questions.
•*• How well is climate mainstreamed into OW programs?
Overall, evaluation participants felt that the degree of integration of climate change into the NWP is low
but improving. There are a few NWP programs in which substantial integration progress has been made,
but most divisions, offices, and programs are in the early stages. The evaluators framed this discussion
by defining a fully mainstreamed situation as one in which climate change factors into daily routines,
duties, and decision making. Evaluation participants were asked to rate their opinion of mainstreaming
on a 1 to 5 scale, with 1 being "not at all" and 5 being "completely integrated." Participants were then
asked to explain the rationale behind their rating. The average rating of all responses to this question
was 2.5; Exhibit ES-2 below illustrates the range of responses by respondent affiliation.
-------
EC
EXHIBIT ES-2. EXTENT OF MAINSTREAMING ACROSS OW PROGRAMMING, PERSPECTIVES BY
AFFILIATION
Fully Integrated
Not Integrated
OGWDW OWIO OWOST OW OWM OWOW Regional Office
As shown in the exhibit above, interviewee perception is that mainstreaming is further along in OGWDW
in other OW offices. Also, external interviewees perceive that OW is further along in mainstreaming, and
credited OW for its willingness to begin a dialogue with state and tribal partners around climate change,
but noted that progress has been slow and halting.
•*• What are the barriers to mainstreaming and how might this be better accomplished?
The evaluators found a range of barriers preventing the integration of climate change considerations
into OW's daily operations. Key barriers included competing priorities, lack of resources, characteristics
of climate change as an issue, organizational and structural aspects of OW, and external influences.
Many interviewees observed that in the context of severe budget and staffing constraints, it can be
difficult for issues such as climate change to compete for limited time and attention. The NWP faces
numerous priorities, and those which are driven by legislative, regulatory, or court-mandated
requirements are often prioritized.
Several interviewees noted that the inherent complexity, uncertainty, variability, and long-term time
frame of climate change were barriers to mainstreaming into the NWP. This was coupled with the
overarching nature of the problem, in that climate change affects multiple sectors and exacerbates
many other issues, ranging from water availability to water quality to sea level rise. Participants noted
that while EPA permits are generally written for five year increments, the programs most impacted by
climate change should think about setting standards over a 20 - 30 year time frame. A related barrier
cited by several interviewees was the knowledge base within OW. Several interviewees noted that
offices must go through the thought exercise of considering climate impacts on their core
responsibilities to internalize the concept that failing to account for climate change could have severe
consequences for their programs.
The lack of continuity associated with a constantly shifting roster of branch chiefs, division directors, and
deputy office directors was noted as a barrier to integration. A participant in the evaluation remarked "it
takes 10,000 repetitions to create muscle memory. The same is true of integrating this type of priority
across an organization; it has to be part of the regular conversation."
-------
EC
Finally, interviewees noted that political and public support and emphasis for climate change policy is
capricious and often based on reaction to major events (e.g., hurricanes, flooding). Public awareness and
acceptance of climate change is still not at a favorable tipping point in some regions. Regional
interviewees remarked that EPA has just begun the dialogue with states around climate change, and
most environmental programs are implemented by states. They observed that in some states, state
government officials face political constraints regarding their ability to recognize climate change as an
issue and EPA regional offices must be sensitive to these politics. In these situations, EPA regional offices
have had some success in engaging state and local partners on specific topics associated with climate
change, such as storm water management needs associated with severe weather events.
Interviewees noted that climate change is better
mainstreamed within voluntary programs than
regulatory programs. Throughout the discussions,
evaluation participants consistently brought up
two programs as examples of successful
integration of climate change considerations into
day-to-day operations: Climate Ready Water
Utilities (CRWU) and Climate Ready Estuaries
(CRE) - see sidebar for program descriptions. Both
programs work with a well-established network of
organizations outside of EPA; in the case of
CRWU, OW staff works with national utility
associations and CRE taps into the expertise of
organized coastal watershed groups. The CRWU
program emphasize creating practicable, usable
tools - filling a niche which had until then been
the provenance of high level atmospheric
scientists. The CRE program is well-marketed and
benefits from immediate buy-in at the state level,
which is critical since states can advocate for
programs at EPA.
Climate Ready Water Utilities
The CRWU initiative provides several useful
tools and resources for utility owners and
operators. These include the Climate
Resilience Evaluation and Awareness Tool
(GREAT), Adaptation Strategies Guide, a
searchable resource library, and an Extreme
Events Workshop Planner.
http://water.epa.gov/infrastructure/watersecu
rity/climate/index.cfm
Climate Ready Estuaries
The Climate Ready Estuaries program works
with the National Estuary Programs and the
coastal management community to: (1) assess
climate change vulnerabilities, (2) develop and
implement adaptation strategies, and (3)
engage and educate stakeholders. CRE shares
NEP examples to help other coastal managers,
and provides technical guidance and assistance
about climate change adaptation.
http://water.epa.gov/type/oceb/cre/index.cfm
A widely held perception among interviewees was
that more work is needed to integrate climate
change into the regulatory side of EPA activities.
Interviewees acknowledged that incorporating
climate change considerations into regulatory programs and existing work requires a significant
commitment of time and resources. OW's CWA and Safe Drinking Water Act (SDWA) authorities include
adaptive management processes that allow updates to permits and standards based on changing
conditions, of which climate change is one. Another interviewee noted, however, that until climate
change is explicitly acknowledged within OW's regulatory processes, it will not be fully integrated into
the NWP. For example, regulatory drivers such as NPDES regulations and SDWA regulations for carbon
capture and storage have been discussed within OW but there has been no action to date.
•*• What goals, implementation experience, or lessons from the 2008 Strategy could be useful to
guide implementation of the 2012 Strategy?
The evaluation team distilled several findings and associated lessons on the overall effectiveness of the
National Water Program Strategy: Response to Climate Change (the 2008 Strategy)—as both a
document and a process—for affecting change in and through the NWP. Key findings are highlighted
below.
-------
lEc
Strategy use, influence, and value: Despite resource constraints and competing priorities, the NWP
has made significant progress in responding to climate change since 2008 in both voluntary and
regulatory program areas. The evaluators found evidence that significant progress has been made
since 2008 in understanding the potential impacts of climate change to the NWP and in taking
responsive actions to support mitigation and adaptation efforts by diverse partners. However, most
evaluation participants stated that while progress to date has been substantial, major work lies
ahead to integrate climate change into NWP programs. Although the 2008 Strategy is widely
regarded as a pioneering effort to take on the task of addressing climate change within a major EPA
program office, overall the individuals interviewed for this evaluation reported that the 2008
Strategy was not a significant driver of this activity by OW divisions and programs. Many
interviewees indicated that the strategy addressed actions already planned or underway, and that
other drivers (such as stakeholder needs and the presence of a management-level champion) had
much greater influence than the strategy effort on OW programs' progress to respond to climate
change. The story is somewhat different outside of EPA headquarters. The 2008 Strategy was
actively used by some EPA Regional Offices and state and tribal partners to raise awareness and
motivate action, and the 2008 Strategy has been used by other EPA offices and other federal
agencies to inform climate change adaptation planning. New implementation strategies—such as
enhanced communications and management engagement efforts—will be needed to make the 2012
Strategy more useful in driving progress.
Communication and outreach: Overall, the evaluators found that OW is viewed as an effective
partner on climate change activities, and in some cases OW's efforts have spurred other
organizations' ability to work on climate change. However, more attention to communications and
outreach for the 2012 Strategy would be beneficial for implementation success. Several existing
communications and outreach mechanisms, such as the National Water Program Climate Change
Workgroup, the "Highlights of Progress" documents, and the NWP climate change strategy website,
have been helpful for engaging EPA regional offices and partners outside of EPA. The State and
Tribal Climate Change Council (STC3) was noted as an important effort to engage key external
partners, although several evaluation participants indicated that there are substantial opportunities
and need to expand partner engagement activities and the use of the STC3.
Management support: Conversations with evaluation participants revealed strong high-level
management support for the overall NWP climate change strategies but uneven support at the
division, office, and branch manager level during implementation. Regular management support is
vital to progress; finding ways to sustain management engagement with the 2012 Strategy is key.
Staffing and resources: Overall, the evaluators found limited resources and staff for both
development of the 2008 and 2012 Strategies and implementation of climate change activities.
However, implementation of some climate change programs and initiatives can occur with existing
resources by doing current work in a slightly different manner, while other efforts will require
dedicated staff time and funding.
Training and capacity building: The evaluators overall assessment is that additional training for EPA
staff on impacts of climate change, particularly impacts on areas specific to staff members'
programmatic duties, would be extremely helpful.
Measurement: The evaluators found that OW divisions and offices outside of IO found little value in
2008 Strategy measurement activities, and did not find evidence that measurement informed
decisions or motivated action. This finding informed the evaluator's suggested measurement
approaches presented in this report.
-------
EC
J What goals and strategic actions in the 2012 Strategy should EPA headquarters (HQ) and regional
programs prioritize?
IO is currently developing prioritization criteria in order to effectively allocate resources toward climate
related activities. Key criteria under consideration include: urgency, risk, geographic scale, programmatic
scale, and probability of occurrence. OW could use these criteria to prioritize activities identified in the
2013 Implementation Plan. The evaluators encountered a broad range of perspectives on prioritization
of the strategic actions described in the 2012 Strategy. Interviewees did not advocate for prioritization
of particular strategic actions; instead, they provided a suite of prioritization schemes for OW
consideration. For example, offices that work directly with partner organizations or associations, such as
utilities, tend to prioritize those activities developed jointly with partners. Some participants focused on
regulatory changes to implement the Strategy; some respondents suggested prioritizing strategies
connected to existing EPA authorities such as TMDLs; and still others suggested prioritizing climate
change guidance in the use of EPA categorical grants, performance partnership grants (PPGs) and CW
and DW SRF programs.
PROSPECTIVE MEASUREMENT APPROACH FOR 2012 STRATEGY
In Chapter 5, the evaluation team provides options for measurement approaches for IO to consider for
measuring progress on the 2012 Strategy. These options are informed by several factors, including the
goals of the 2012 Strategy and its phased approach to adaptive management; feedback from
interviewees on the 2008 Strategy regarding measurement limitations; feedback from interviewees on
options for measuring progress on the 2012 Strategy; and adherence to core principles of robust
performance measurement. The evaluation team summarizes findings and lessons learned for this
evaluation question below, organized by evaluation question:
A What is the measurement approach that can be used to measure adaptation progress in five
areas: infrastructure, watersheds and wetlands, ocean and coastal waters, water quality, and
working with tribes?
The 2012 Strategy contains 53 strategic actions in the five vision areas (infrastructure, watersheds and
wetlands, coastal and ocean waters, water quality, and working with tribes) and cross-cutting areas of
program support. In the 2012 Strategy, OW adopted a seven-phased approach to adaptive
management, described below, that show progress towards achieving stated goals and strategic actions
described in the 2012 Strategy:
1. Initiation: Conduct a screening assessment of potential implications of climate change to mission,
programs, and operations.
2. Assessment: Conduct a broader review to understand how climate change affects the resources in
question. Work with stakeholders to develop an understanding of the implications of climate change
to the mission, programs, and operations.
3. Response Development: Identify changes necessary to continue to reach program mission and
goals. Develop initial action plan. Identify and seek the research, information, and tools needed to
support actions. Begin to build the body of tools, information, and partnerships needed to build
capacity internally and externally.
4. Initial Implementation: Initiate actions in selected priority programs or projects.
5. Robust Implementation: Programs are underway and lessons learned are being applied to
additional programs and projects.
-------
EC
6. Mainstreaming: Climate is an embedded component of the program.
7. Monitoring and Adaptive Management: Continue to monitor and integrate performance, new
information, and lessons learned into programs and plans.
The seven phases of adaptive management could be used in two different ways to assess progress:
1. To assess progress on goals, which are mostly outcome-based. Outcomes are changes in awareness,
behavior, or condition that result from EPA activities.
2. To assess OW's progress on implementing strategic actions, which are mostly EPA outputs. Outputs
are products or services that EPA provides, such as technical assistance, trainings, and decision-
support tools.
At this stage of climate change adaptation, the evaluators suggest that OW programs should be
accountable for outputs, and learn from outcomes. Thus, we suggest that OW apply the seven phases to
track progress towards goals articulated the 2012 Strategy by undertaking an "outputs plus priority
outcomes approach" described below and in Chapter 5.
^ What specific elements need to be applied to the phased approach to tracking progress outlined
in the 2012 Strategy, to make it a robust measurement framework?
IO needs to develop objective criteria for each phase of adaptive management to facilitate consistent
measurement. For example, if the goal selected is reduce water infrastructure vulnerability to climate
change risks, criteria may be defined as shown in Exhibit ES-3. Note that Phases 1-3 are likely be
assessed in terms of producing outputs and may be closely linked to existing strategic actions contained
in the 2012 Strategy. Phases 4 and later are intended to track outcomes; tracking outcomes will require
data collection from EPA partners such as state and local governments. Note that we suggest that IO set
specific, quantitative thresholds for meeting each stage, but determining appropriate thresholds will
require pilot testing.
EXHIBIT ES-3. ILLUSTRATIVE CRITERIA FOR AN OUTPUTS PLUS PRIORITY OUTCOMES APPROACH
PHASE
CRITERIA FOR ASSESSING PROGRESS ON REDUCING WATER INFRASTRUCTURE VULNERABILITY
1-Initiation
OW conducted a screening assessment to identify the potential implications of climate change for water
infrastructure.
2-Assessment
3-Response
Development
4-lnitial
Implementation
5-Robust
Implementation
6-Mainstreaming
7-Monitoring and
Adaptive
Management
OW has conducted a broad review to better understand how climate change affects water infrastructure,
including consulting water utilities.
In collaboration with partners, OW has developed and distributed information, guides, and tools to assist
water utilities in undertaking adaptation, efficiency, and demand/supply management measures.
At least 30% of water utilities have conducted initial planning steps and updated planning documents to
address climate change risks and a few water utilities have undertaken substantive, on-the-ground
adaptation, efficiency, and demand/supply management measures.
At least 30% of water utilities have undertaken substantive, on-the-ground efficiency, and demand/supply
management measure, and of water utilities that have identified adaptation measures to be implemented
in the short-term, at least 30% have undertaken substantive, on-the-ground adaptation measures.
At least 70% of water utilities have undertaken substantive, on-the-ground efficiency, and demand/supply
management measures and have integrated climate change considerations into their normal processes and
operations, and of water utilities that have identified adaptation measures to be implemented in the
short-term, at least 70% have undertaken substantive, on-the-ground adaptation measures.
The water utility sector, independently or in conjunction with OW or other federal agencies, has
implemented mechanisms to monitor and evaluate water utility progress, identify lessons learned,
incorporate new climate data into planning, and continually improve performance on climate planning and
programming.
-------
EC
For each goal, IO will need to identify if the phase of adaptive management has been met, based on the
specific criteria defined. For example, the evaluation team's review of baseline data indicates progress
on the above goal, using the illustrative criteria, may be as described in Exhibit ES-4 below.
The evaluation team recommends piloting the outputs plus priority outcomes approach with a few
priority goals. Given this, the evaluators do not see a reason to aggregate across results, at least not in
the near term. However, if IO aggregates results, it needs to be very careful in applying any weighting
(implicitly or explicitly). IO needs to be clear in all assumptions, and any weighting should be deliberate
and transparent. Note that if results are aggregated without explicit weighting, then all results are
implicitly given equal weight.
EXHIBIT ES-4. PROGRESS TOWARDS GOALS
PHASE
CRITERIA FOR ASSESSING PROGRESS ON GOALS
(Green = Phase Has Been Met, Yellow = Phase May Be Met, Orange = Phase Not Yet Met)
1-Initiation
2-Assessment
3-Response
Development
OW conducted a screening assessment to identify the potential implications of climate change for water
infrastructure. (Met)
OW has conducted a broad review to better understand how climate change affects water infrastructure,
including consulting water utilities. (Met)
In collaboration with partners, OW has developed and distributed information, guides, and tools to assist
water utilities in undertaking adaptation, efficiency, and demand/supply management measures. (In
Progress/ May be met)
4-lnitial
Implementation
5-Robust
Implementation
6-Mainstreaming
At least 30% of water utilities have conducted initial planning steps and updated planning documents to
address climate change risks and a few water utilities have undertaken substantive, on-the-ground
adaptation, efficiency, and demand/supply management measures. (In Progress/ May be met)
At least 30% of water utilities have undertaken substantive, on-the-ground efficiency, and demand/supply
management measures and of water utilities that have identified adaptation measures to be implemented
in the short-term, at least 30% have undertaken substantive, on-the-ground adaptation measures. (Not yet
met)
At least 70% of water utilities have undertaken substantive, on-the-ground efficiency, and demand/supply
management measures and have integrated climate change considerations into their normal processes and
operations and of water utilities that have identified adaptation measures to be implemented in the short-
term, at least 70% have undertaken substantive, on-the-ground adaptation measures. (Not yet met)
7-Monitoring and
Adaptive
Management
The water utility sector, independently or in conjunction with OW or other federal agencies, has
implemented mechanisms to monitor and evaluate water utility progress, identify lessons learned,
incorporate new climate data into planning, and continually improve performance on climate planning and
programming. (Not yet met)
Exhibit ES-5 below presents the advantages and disadvantages of the outputs plus priority outcomes
approach. These considerations are also discussed in Chapter 5.
EXHIBIT ES-5. PROS AND CONS OF THE OUTPUTS PLUS PRIORITY OUTCOMES APPROACH
ADVANTAGES
DISADVANTAGES
Approach tracks progress on key goals: goes beyond
assessing outputs to assessing the state of climate change
adaptation in the field, which is ultimately what matters.
Identifying key outcomes (particularly using the logic model
framework) would be a helpful step in clarifying OW's
priorities on climate change adaptation.
Difficulty of collecting data from external partners.
Uncertainty about attributing progress on goals to EPA actions
because the approach involves assessment of progress on
activities outside of EPA's direct sphere of influence.
Lack of buy-in from OW staff for an approach that includes
outcomes.
Criteria for each vision area would need to be carefully defined
by expert program staff, and then pilot tested.
-------
EC
Above, we have illustrated the key elements for robust measurement vis-a-vis the outputs plus priority
outcomes approach. Chapter 5 provides more detail on this approach, and notes several alternatives for
measuring outcomes. Chapter 5 also details an alternative approach to measurement that would cover
outputs only.
A What, if, any, revisions should EPA make to its baseline data collection process to ensure that data
collected are meaningful and objective?
The evaluation team reviewed the approach that IO took in measuring progress on the 2012 Strategy to
date, as presented in the 2012 Highlights of Progress report. We documented several data quality and
consistency challenges with reported data. Moving forward, we recommend that IO take steps to
adhere to key tenets of data quality and consistency described in our recommendations (see below).
Core among these, IO will need to develop data reporting templates, clear instructions for reporting,
and institute a quality control plan. The evaluation team also noted several issues regarding
assumptions, implicit weighting factors, and transparency with lO's previous approach. Thus, we
recommend that IO take a different approach to measurement moving forward, as described above.
A How can OW's measurement approach inform measuring progress in the EPA-wide Adaptation
Plan, and to inform development of the next Agency Strategic Plan?
One of this project's original evaluation questions asked:
How can OW's measurement approach inform measuring progress in the EPA-wide
Adaptation Plan, and to inform development of the next Agency 4-year Strategic Plan?
After conducting the evaluation, the evaluation team, in conjunction with EPA, decided to reframe this
question to ask:
How can lessons learned from this evaluation inform measuring progress in the EPA-wide
Adaptation Plan, and inform development of the next Agency 4-year Strategic Plan?
Lessons Learned on Measurment Relevant to the Agency-wide Adaptation Plan
A central goal of the EPA-wide Adaptation Plan is to strengthen the capacity of EPA staff and partners
across the country to anticipate and respond to the effects of climate change. Strengthening capacity
will help EPA staff and partners integrate climate adaptation into everyday work by providing them with
needed data, information, and tools. The Adaptation Plan includes a list of ten priority actions that the
Agency will take to integrate climate change adaptation into its programs, rules, and operations:
1. Fulfill Strategic Measures in FY2011-2015 EPA Strategic Plan
2. Protect Agency facilities and operations
3. Factor legal considerations into adaptation efforts
4. Strengthen adaptive capacity of EPA staff and partners through training
5. Develop decision-support tools that enable EPA staff and partners to integrate climate adaptation
planning into their work
6. Identify cross-EPA science needs related to climate adaptation
7. Partner with tribes to increase adaptive capacity
8. Focus on most vulnerable people and places
9. Measure and evaluate performance
-------
EC
10. Develop Program and Regional Office Implementation Plans (the OW 2012 Strategy serves as the
implementation plan for OW)
Following the release of the agency-wide Adaptation Plan, each EPA office and region is required to
develop an implementation plan and will need to track progress on it. OW's 2012 Strategy is its
implementation plan in response to the agency-wide Adaption plan. Overall, we recommend that OW
strive to measure progress on a few priority outcomes, at least on a pilot basis. Other EPA offices could
go through a similar process of using logic models to select priority outcomes, and developing measures,
criteria, and data collection strategies relevant to those priority outcomes. As with OW, we recommend
that programs pilot this approach with mature programs, where outcomes may be apparent and
mechanisms for measurement (e.g., grant reporting requirements, or regulations that incorporate
reporting mechanisms) may be available.
Alternatively, other EPA offices could to continue to use existing measures, and track progress on them
in light of climate change. Using this approach, it would be important to understand the potential for
climate change to affect existing measures. It may be necessary to collect contextual information to
make it possible to understand the extent to which climate change is making it more difficult to meet
targets for existing goals. The evaluation team does not make a recommendation on which of these
approaches is preferable; however, we do think that any such approach should be pilot tested for a few
outcomes or measures.
Whatever approach other offices choose to take in measuring progress on climate change adaptation,
we recommend they keep the following general lessons on measurement in mind from this evaluation:
A Focus on a few priorities;
A Weigh the merits of measuring outputs vs. outcomes;
A Be transparent about assumptions and weighting; and
A Ensure data quality.
Lessons Learned on Measurement for the Next Strategic Plan
The current Strategic Plan's emphasis is to mainstream climate change into operations by 2015; the
evaluation team believes the three measures of mainstreaming that the Agency adopted are clear,
concise, and measurable. In the next Strategic Plan, once EPA has demonstrated the ability to
mainstream climate change into operations, the current three measures will no longer be as relevant.
To develop new measures, we recommend that EPA engage in fundamental strategic planning, and
grapple with the particular value that the Agency can add with regard to climate change adaptation.
Using a logic model approach with a particular focus on EPA's role and key external influences relative to
key audiences and desired outcomes, may help the Agency focus in on where investment in climate
change adaptation is most warranted.
Given the priority actions identified in the Adaptation Plan, and EPA's key strengths and institutional
capabilities, three key areas of EPA expertise may include providing data, decision-support tools, and
training to partners related to climate change adaptation. Further work is needed to define a few
specific measures that are relevant across the agency, plan for data collection associated with these
measures, and determine the degree of EPA's contribution to outcomes achieved.
CONCLUSIONS AND RECOMMENDATIONS
The 2008 and 2012 OW climate change strategies are important milestones in the continued evolution
of the NWP. The evaluators—and many of the people we interviewed as part of this evaluation-
anticipate that recognition of the value of these early efforts to understand and address challenges
10
-------
EC
posed by climate change will grow as real impacts to on the ground water resource management
become more apparent. The NWP's climate change strategy work is maturing, entering the seventh year
since the Climate Change Workgroup was launched in 2007. At the same time, continued resource and
staffing pressures and the lack of statutory drivers weaken the ability of climate change considerations
to compete for limited NWP time and attention.
The evaluation findings suggest that fresh implementation approaches are needed to reignite
enthusiasm for implementation of the 2012 Strategy. More attention is needed in several critical areas:
reinvigorating NWP management and staff commitment to the Climate Change Strategy; create
management practices that keep climate change integration front and center; empower EPA staff and
state, tribal and local partners; and clarify the purpose of measurement and pilot a measurement
approach that includes outcomes. Recommendations for IO for moving forward in implementing the
2012 Strategy include the following:
EXHIBIT ES-6. RECOMMENDATIONS SUMMARY
Reinvigorate NWP management and staff commitment to the Climate Change
Strategy.
1. Clarify the purpose of the 2012 Strategy.
2. Clarify commitments and roles associated with the 2012 Strategy.
3. Seek buy-in for the 2012 Strategy among OW management and staff.
Create management practices that keep climate change integration front and
center.
4. Schedule regular management-level strategic discussions adaptation and Strategy implementation.
5. Ask key climate change questions relentlessly up and down the management chain.
6. Shift the balance of implementation focus toward "customer service" and learning.
7. Recognize and reward climate change integration progress.
Empower EPA staff and state, tribal, and local partners.
8. Focus education and training support on connecting climate change to practical work.
9. Expand engagement on the strategy with State, Tribal, and local partners.
10. Attract and plan for resources.
Clarify the purpose of measurement and pilot a meaurement approach that
includes outcomes.
11. Seek buy-in for measuring progress on Strategy implementation.
12. Consider adopting an outputs plus priority outcomes measurement approach.
13. Within the measurement approach selected, acknowledge the iterative and evolving nature of this work.
14. Ensure data quality and consistency in collecting measurement data.
11
-------
lEc
Reinvigorate NWP management and staff commitment to the Climate Change
Strategy.
1. Clarify the purpose of the 2012 Strategy. Clarifying the purpose of the 2012 Strategy, and
communicating that purpose within OW, is critical for informing strategy implementation,
identifying reporting and measurement needs, and understanding how measurement data will be
used and communicated. The evaluation team heard conflicting rationales for the 2008 and 2012
strategies; some OW staff view the strategies as encompassing only goals and activities that are
within EPA's purview, whereas others view the documents as more broadly encompassing goals and
activities that involve EPA's partners. Also, it is not clear to OW staff if the strategy is primarily an
internal planning document, or primarily a document for communicating OW's vision and goals to
external audiences. Notably, during interviews on implementation of the 2008 Strategy, staff
indicated that the previous strategy was more successful as an external communications document
than as an internal planning document.
2. Clarify commitments and roles associated with the 2012 Strategy. Much of the current language in
the 2012 Strategy discussing coordination and collaboration with EPA's partners is vague. It is not
clear what EPA's specific roles or investment will be as distinct from partners. IO should clarify EPA
roles; this is essential for successful strategy implementation, as well as for measurement. Also, IO
needs to harmonize goals and strategic actions at the national and regional levels. While not all
national goals and strategic actions will apply to all regions, having two sets of goals and actions is
unwieldy and confusing.
3. Seek buy-in for the 2012 Strategy among OW management and staff. Lack of buy-in for both the
2008 and 2012 strategies is apparent at both the managerial and staff levels. The evaluation team
heard that managers rarely reference the strategy as part of day-to-day business, and that they do
not participate in the workgroup designed to coordinate implementation of the strategy. Staff
indicated that the strategy is viewed as an IO strategy, that priorities for staff time and funding are
not informed by the strategy, and that the strategy lacks a connection to daily work. Notably, some
staff interviewed indicated that they were not aware of strategic actions that had been officially
assigned to their purview; other staff indicated that they disagreed with the inclusion or wording of
certain strategic actions in the 2012 Strategy.
While there is no single magic bullet for attaining buy-in, a few key decisions could go a long way:
A OW should prioritize specific goals. IO should involve program office and regional staff in this
prioritization in a meaningful way. Resource decisions should be tied to the priorities.
A Managers should keep climate change as a front and center topic with their staff; they should
regularly discuss how the office's daily work relates to and is informed by climate change, and to the
climate change priorities selected as part of the strategic planning process. Managers should also re-
engage with the workgroup and attend meetings regularly instead of relying on designees.
A OW should engage in open discussions with EPA's Office of Policy and Office of Administration and
Resource Management to develop a pilot initiative for Senior Executive Service (SES) candidates and
managers that encourages and supports the development of climate change "champions." This pilot
could provide focused assessment criteria and professional development guidance for advancing
climate change as a cross-cutting issue under the "leading change" SES assessment area.
12
-------
lEc
Create management practices that keep climate change integration front and
center.
4. Schedule regular management-level strategic discussions adaptation and Strategy
implementation. OW should create more opportunities for meaningful strategic discussions among
management on climate change, its implications for water programs, and efforts to mainstream
climate change considerations into voluntary and regulatory programs. While much can be handled
at the staff level, there is no substitute for periodic management engagement on strategic topics. It
will be important to think carefully about the framing of management level discussions so they are
viewed as appropriate and to recognize that some issues may be relevant to some divisions, offices,
and programs but not others. An option for creating space is to designate one monthly climate
change workgroup meeting every 3 to 6 months for management-level discussions. Another
opportunity for strategic discussions is to include time for such discussions on other management
meeting agendas. Potential strategic discussions could include:
A Given current resource constraints, how can we best incentivize progress on mainstreaming
climate change within voluntary and regulatory programs?
A What type of measurement system can best support our efforts over the long-term to respond
to climate change? How can we build toward such a system in the near and mid-term?
A What scenarios do we see for how climate change may impact local integrated water resource
management and governance? What do these scenarios mean for EPA water programs?
A How might evolving scientific understanding and emerging climate change impacts (e.g., ocean
acidification) affect the NWP?
A What strategic priorities or emergent opportunities are important to address?
A What are we learning from climate change integration efforts to date?
A What types of support, information, tools, and resources are needed to drive more rapid
progress in responding to climate change?
A suggested support tool for these discussions is a concise (less than 10 pages) adaptation of the
2012 Strategy Executive Summary to serve as a stand-alone resource to guide management
discussions. Soliciting input from managers on concise information that could be added to this
document will make it more useful to them.
5. Ask key climate change questions relentlessly up and down the management chain. One tried and
true approach to breathe life into strategy implementation efforts is to ensure that managers at the
top consistently ask a set of simple and clear questions. These questions keep the strategy present
and send clear signals that the topic is important and must be considered in routine program
planning and decisions. Questions could include:
A Have you considered climate change and its impacts (in your program, plans, analyses, or
decision process)?
A How is climate change likely to affect your program's ability to deliver results and meet goals?
How confident are you about this?
A What work is needed to improve understanding of how climate change will affect your program
or to integrate climate change considerations into your program? How can we help support
these efforts?
13
-------
lEc
A What is your current understanding of the outstanding adaptation issues and needs, and the
opportunities to address them?
A Which of those opportunities do you think are the most important to address- do you have
resources to do so? How can we support these efforts?
6. Shift the balance of implementation focus toward "customer service" and learning. Much of the
strategy implementation process is currently viewed as an obligation by divisions, offices, and
branches in the NWP. Take steps (including those described in recommendations below) to ask
managers and staff in the NWP what they need and how IO can best support mainstreaming of
climate change across programs. Foster a culture of learning around the strategy; ask managers and
staff what they are learning from efforts underway. Consider periodic deployment of a web-based
survey to solicit information from NWP staff to understand the state of climate change awareness,
informational needs and questions, and staff-level perceptions of opportunities, accomplishments,
and lessons. Compile and communicate input received and consider ways to be responsive to them.
Set a clear tone that IO recognizes the challenges of integrating climate change but that the need
will only intensify and IO wants to support program offices on this journey.
7. Recognize and reward climate change integration progress. Many evaluation participants noted
that recognition and modest incentives can go a long way to inspire and encourage managers and
staff to take extra steps to advance climate change efforts. IO should build on the "highlights of
progress" efforts to showcase accomplishments and lessons, and consider modest opportunities to
capture and share successes within and external to EPA in newsletters, intranet postings, staff
meetings, and other venues. IO could consider incentives such as "climate champion" badges and
other forms of recognition. Although some of these ideas may sound frivolous, their cumulative
effect can be powerful and can motivate people to keep the strategy - and the goals and actions that
it encompasses - present in the workplace.
Empower EPA staff and state, tribal, and local partners.
8. Focus education and training support on connecting climate change to practical work. While
general presentations and discussion of climate change and climate impacts can raise awareness,
they can also fall flat if they fail to help people answer the question: "What does this mean for me
and the work that I do?" More work is needed to help answer this question in divisions, offices,
branches and programs across the NWP. All hands meetings and training sessions have their
advantages, but they will miss key opportunities if they do not connect broader issues to practical
work. One way to explore these connections is to create space during webinars and presentations
for direct discussion on this question. Creating space for interaction can also provide insights into
managers' and staff members' questions about climate change. A periodic survey could support
these efforts. Shifting webinar formats to use real-time polling and facilitated peer exchange can
also make education, training, and communication efforts more relevant to participants. EPA's State
and Local Climate and Clean Energy Program in the Office of Air is a leader within the Agency in
using these approaches. OW should also work closely with the Office of Policy team supporting the
Agency-wide Climate Adaptation strategy in exploring innovative ways to conduct education,
training, and communications. Interaction may lead to identification of new adaptation
opportunities.
9. Expand engagement on the strategy with State, Tribal, and local partners. Interest in climate
change adaptation and mitigation appears to be growing at the state and local level. Even in
jurisdictions where there are political constraints to talking about climate change, local officials are
increasingly interested in responses to related issues such as extreme weather, ocean acidification,
14
-------
lEc
and integrated water resource management needs. Preliminary evidence suggests that the 2008 and
2012 Strategies have been useful tools for engaging state, tribal, and local partners in discussions
and joint planning and projects to respond to climate change. There are significant opportunities to
expand these discussions and increase communications between NWP partners about activities,
accomplishments, lessons, needs, and challenges. Broader discussions on the response to climate
change with external partners can build on existing program and regional office relationships and
discussions. OW can encourage coordination with national partners such as the Association of State
Drinking Water Agencies and the Association of State Wetland Managers to leverage resources and
communication opportunities.
10. Attract and plan for resources. IO should explore opportunities to assemble even a modest reserve
of resources to support or seed priority climate change related projects. Programs could compete
for these resources on a periodic basis and applications could be selected by peers in the NWP.
Engage senior managers in discussions to identify creative opportunities to attract limited resources
to support high priority activities. IO could explore opportunities to expand the use of pilot projects
to test integration approaches with limited funds, and to capture and share lessons from existing
pilots and examine resource-efficient ways to expand successful approaches.
Clarify the purpose of measurement and pilot a meaurement approach that
includes outcomes.
11. Seek buy-in for measuring progress on Strategy implementation. Attaining buy-in for measuring
the outcomes of the 2012 Strategy is first predicated on attaining buy-in for the strategy itself, and
also on clarifying the purpose of the strategy. However, currently OW staff are skeptical about
measurement, and IO cannot implement a meaningful measurement effort without the support and
engagement of staff in the program offices and regions. IO should take several steps specific to
measurement to increase program office and regional support. IO should communicate with offices
and regions about what it plans to do with measurement data, and how data will be aggregated and
communicated. IO should provide assurances that measurement data will not be used as
justification for cutting budgets. Finally, if IO pursues an outcome approach, it should pursue
measurement primarily for a small set of outcomes, discussed below.
12. Consider adopting an outputs plus priority outcomes measurement approach. IO will need to
weigh the costs and benefits of potential measurement approaches. Measuring progress on
strategic actions is largely an output-based approach, and in and of itself, cannot measure progress
on the ultimate goal of fostering climate change adaptation on the ground. As such, the evaluation
team recommends pursuing the outputs plus priority outcomes approach. Specifically, we suggest
that IO pilot this approach with no more than a few priority outcomes given resource constraints.
To select outcomes for piloting an outputs plus priority outcomes approach, IO should consider
selecting outcomes with an existing mechanism for data collection, such as programs with a grant
component. IO may want to focus on more mature efforts, such as CRWU or CRE, where outcomes
are more likely be present. IO should develop a data collection and measurement plan in
collaboration with partners, such as the State/Tribal Council, and test the plan for a few years. At
the conclusion of the pilot, IO and its partners should assess progress, determine lessons learned,
and contemplate scaling up the approach to include more outcomes.
Focusing measurement on a few priority outcomes will serve to investigate proof of concept before
scaling up a measurement approach. Trying to implement a comprehensive outcome-based
approach from the outset may inadvertently add more confusion than insight; as noted by
15
-------
lEc
interviewees, there is a real danger that overinvestment in measurement could result in loss of staff
support for tracking progress on climate change adaptation.
13. Within the measurement approach selected, acknowledge the iterative and evolving nature of
this work. The seven stages of adaptive management make sense conceptually, but as a practical
matter, EPA will need to revisit each stage on a regular basis. A more cyclical framework may
provide a better model, with the reach of EPA's activities expanding over time to a wider circle of
partners.
14. Ensure data quality and consistency in collecting measurement data. The baseline data OW
collected illustrate challenges of data consistency and quality that are important to address in any
measurement system. Moving forward, IO needs to develop data collection templates, instructions,
and a quality control plan that ensure high quality data. Key steps include:
• Clearly communicate the data collection plan to all stakeholders involved.
• Identify specific data to be provided, including units, within data collection templates and
instructions. Also indicate the specific data to be provided by specific stakeholders. Do not use
open-ended questions to collect measurement data.
••"- Require data collection at regular interviews (e.g., annually), and on specific outputs and/or
outcomes previously agreed upon. IO should not accept reports of outputs or outcomes that
were not included in the measurement plan, at least not as a substitute for previously agreed
upon metrics.
• Clarify the time period for activities to be reported on. IO should not accept reports of activities
conducted and/or outcomes realized prior to the reporting period. Similarly, IO should not
accept descriptions of future activities to satisfy reporting requirements.
• Consider developing an online reporting system to facility easier reporting, review, and storage
of measurement data.
• Assign an individual or team within IO to review data submitted; check for completeness,
consistency, clarity, and adherence to reporting instructions. Follow up with reporting entities
for revisions where necessary.
16
-------
lEc
CHAPTER 1 | INTRODUCTION AND BACKGROUND
The National Water Program Strategy: Response to Climate Change was first published in 2008. It
describes the likely effects that climate change will have on water resources and implications for EPA's
Office of Water (OW) and the National Water Program (NWP). Four years later, OW developed an
updated strategy that describes NWP's long-term goals for sustainably managing water resources in
light of climate change. The National Water Program 2012 Strategy: Response to Climate Change (the
2012 Strategy) reflects current climate change adaptation efforts underway nationwide and is intended
as a roadmap to guide future program work and inform the Agency's annual planning process.
In 2012, OW's Immediate Office (IO), which is responsible for coordinating OW climate change work,
requested support from EPA's Evaluation Support Division (ESD) through EPA's Program Evaluation
Competition to conduct a retrospective evaluation of the implementation of the 2008 Strategy and to
develop a prospective measurement framework to track the progress on the 2012 Strategy. ESD offered
support to OW through EPA's Program Evaluation Competition; ESD's support included contractor
assistance provided by Industrial Economics, Incorporated (lEc) and its subcontractor Ross Strategic
(hereafter referred to as the evaluation team or the evaluators).
The remainder of this chapter provides background information the 2008 and 2012 strategies and their
implementation. Subsequent chapters of the report are organized as follows:
A Chapter 2 presents logic models for the 2012 Strategy.
A Chapter 3 discusses the evaluation methodology.
-^ Chapter 4 presents findings and lessons from the retrospective evaluation of implementing the 2008
Strategy.
-^ Chapter 5 presents findings and lessons for implementing the 2012 Strategy.
A Chapter 6 presents conclusions and recommendations.
Several appendices follow the main text and provide additional project detail.
NWP AND CLIMATE CHANGE: EARLY EFFORTS
In March 2007, then EPA Assistant Administrator (AA) for Water Benjamin Grumbles issued a
memorandum to EPA Office of Water (OW) Office Directors, Regional Water Division Directors, and
Great Waterbody Program Office Directors on Climate Change and the NWP.1 The memorandum
launched a NWP Climate Change Workgroup, charged with developing a climate change strategy for the
NWP, and requested each OW Office Director to name a senior manager to serve on the workgroup. The
workgroup met throughout 2007 and into 2008 to develop program response actions in the areas of
mitigation, adaptation, and research. In September 2008, OW published the results in the document:
National Water Program Strategy: Response to Climate Change.2 NWP was one of the earlier Federal
programs to develop a strategy to address mitigation and adaptation to climate change (see Exhibit 1-2
for timeline).
1 http://www.ecy.wa.gov/programs/wq/partnership/03 15 07/epamemo.pdf
2 http://water.epa.gc
Climate-Change.pdf
2 http://water.epa.gov/scitech/climatechange/upload/2008-National-Water-Program-Strategy-Response-to-
17
-------
EC
2008 Strategy Development
The focus of the 2008 Strategy was on raising
awareness about NWP vulnerabilities to
climate change and immediate actions OW
and its partners could take to reduce the
risks of floods, droughts, and other climate
change impacts. Initial drafts of the 2008
Strategy contained 60 to 70 strategic actions,
but given resource constraints, the Climate
Workgroup narrowed the list to 44 key near-
term actions that could be accomplished
with existing resources. AA Grumbles
requested commitments from management
and staff to implement the 2008 Strategy,
and an OW staff member was designated as
the lead for each strategic action. The
Climate Workgroup continued to meet
monthly; however, senior executive
participation transitioned to staff level
participation overtime.
2008 Strategy Implementation
During the first two years of implementation,
IO staff contacted each strategic action lead
semi-annually to check on progress toward
the key actions. OW documented results
from 2008 in a progress report dated January
2009.3 OW released a second progress report
in January 2010 covering activities during
2009.4 The second report was accompanied
by a report on progress and highlights from
2007-2009 related to implementation of the
2008 Strategy and EPA's Regional and Large Aquatic Ecosystem Program highlights.5
In October 2009, Principal Deputy AA for Water Mike Shapiro signed the charter for the State and Tribal
Climate Change Council (STC3) of the NWP.6 Among the key goals of the STC3 was to "disseminate
information to State and Tribal governments concerning implementation of the National Water Program
Strategy: Response to Climate Change and engage States and Tribes in this implementation process."7
EXHIBIT 1-1. EFFORTS ACROSS THE FEDERAL
GOVERNMENT ON CLIMATE CHANGE ADAPTATION
In 2009, the Obama Administration convened the
Interagency Climate Change Adaptation Task Force, a
body comprising senior leadership from more than
23 federal departments, agencies, and offices. On
October 5, 2009, President Obama signed Executive
Order 13514 directing the Task Force to report to
him within one year on what Federal agencies are
currently doing to support a national climate change
adaptation strategy, and what more the Federal
Government should be doing. On March 4, 2011, the
White House Council on Environmental Quality (CEQ)
issued a set of Implementing Instructions for Federal
Agency Adaptation Planning. The instructions
informed agencies on how to integrate climate
change adaptation into their planning, operations,
policies, and programs. Agencies were required to
include initial adaptation plans as part of their annual
Strategic Sustainability Performance Plans, which are
submitted to OMB. On February 7th, 2013 agencies
released their most recent Sustainability Plans.
All Federal agency plans are available at
http://sustainability.performance.gov/
EPA's draft Climate Change Adaptation Plan is
available at: http://epa.gov/climatechange/impacts-
adaptation/fed-programs.html.
3 http://water.epa.gov/scitech/climatechange/upload/2008-lmplementing-the-National-Water-Program-Strategy-
Response-to-Climate-Change-Progress-Report.pdf
http://water.epa.gov/scitech/climatechange/upload/2009-lmplementing-the-National-Water-Program-Strategy-
Response-to-Climate-Change-Progress-Report.pdf
5 http://water.epa.gov/scitech/climatechange/upload/2007-2009-Regional-and-Large-Aquatic-Ecosystems-
Program-Highlights.pdf
6 http://water.epa.gov/scitech/climatechange/upload/State-Tribal-Climate-Change-Council-Charter-10-2-09.pdf
7 Ibid.
18
-------
EC
In August 2010, OW issued a Key Action Update to indicate the status of the 44 key actions from the
2008 Strategy, reflect on challenges, and examine new opportunities.8 The Key Action Update was an
interim step toward development of a new NWP strategic plan for climate change, scheduled for release
in 2012.
In late 2010 and 2011, OW shifted its reporting strategy to annually solicit information on broader
programmatic accomplishments related to climate change, rather than progress related to the 44 near-
term actions from the 2008 Strategy. OW's third progress report, published in December 2011, was
markedly different from its predecessors in both length and format, and more closely resembled the
2007-2009 Highlights of Progress report.9 Whereas previous progress reports analyzed the 44 key
actions and measured progress towards each, the third progress report broadly described OW's climate-
related activities under each of the five goal areas from the 2008 Strategy (Mitigation of Greenhouse
Gases, Adaptation to Climate Change, Climate Change Research, Water Program Education on Climate
Change, and Management of Climate Change). Some activities described in the third progress report
were not clearly connected to the 44 actions contained in the 2008 Strategy.
2012 STRATEGY
In 2010 the Climate Change Workgroup began developing a new climate change strategy for the NWP.
OW intended the new strategy take a longer term view than the previous strategy (approximately 20 -
30 years) and provide a more robust framework to guide management system development, action
planning, and assessment of progress. OW also intended the 2012 Strategy as a framework under which
OW programs would perform annual (or biannual) action planning and make commitments.
On April 2, 2012 EPA released a draft of the 2012 Strategy for a 45-day public comment period; in May
the Agency held a public webinar to provide an overview of the 2012 Strategy's contents and outline the
process for submitting comments on the draft strategy. EPA received forty-four sets of comments during
the public comment period. The STC3 also provided feedback on the 2012 Strategy and is expected to
continue to provide feedback throughout its implementation. The final 2012 Strategy was published in
December 2012,10 as was a summary of public comments received.11
The 2012 Strategy was organized around a series of visions, goals, and strategic action areas. The vision
areas are as follows:
1. Infrastructure
2. Wetlands and Watersheds
3. Coastal and Ocean Waters
4. Water Quality
5. Working with Tribes
http://water.epa.gov/scitech/climatechange/upload/2010-2011-National-Water-Program-Climate-Change-
Strategv-Kev-Action-Update-August-2010.pdf
9 http://water.epa.gov/scitech/climatechange/upload/2010-2011-lmplementing-the-National-Water-Program-
Strategv-Response-to-Climate-Change-National-and-Regional-Highlights-of-Progress.pdf
10 http://water.epa.gov/scitech/climatechange/upload/epa 2012 climate water strategy full report final.pdf
11 http://water.epa.gov/scitech/climatechange/upload/EPA-NWP-2012-Climate-Strategy-Public-Comment-
Summary.pdf
19
-------
lEc
Goals included under the 2012 Strategy include items such as sustainably managing water resources,
maintaining a network of healthy watersheds and habitat corridors, and incorporating climate resilience
into watershed restoration (for a full listing of goals and strategic actions by vision area, see Exhibit 3-3
in Chapter 3). In addition to the five vision areas and associated goals and strategic actions, the 2012
Strategy identifies three cross-cutting areas of program support (communication/collaboration,
measuring outcomes, and research) and related goals and strategic actions. Together the five vision
areas and three cross-cutting areas of program support encompass 19 goals and 53 strategic actions. In t
2012 Strategy identifies goals and strategic actions for each of eight climate regions defined by the U.S.
Global Change Research Program. In the 2012 Strategy, OW also adopted a seven-phased approach to
adaptive management, described below, that show progress towards achieving stated goals and
strategic actions described in the 2012 Strategy:
1. Initiation: Conduct a screening assessment of potential implications of climate change to mission,
programs, and operations.
2. Assessment: Conduct a broader review to understand how climate change affects the resources in
question. Work with stakeholders to develop an understanding of the implications of climate change
to the mission, programs, and operations.
3. Response Development: Identify changes necessary to continue to reach program mission and
goals. Develop initial action plan. Identify and seek the research, information, and tools needed to
support actions. Begin to build the body of tools, information, and partnerships needed to build
capacity internally and externally.
4. Initial Implementation: Initiate actions in selected priority programs or projects.
5. Robust Implementation: Programs are underway and lessons learned are being applied to
additional programs and projects.
6. Mainstreaming: Climate is an embedded component of the program.
7. Monitoring and Adaptive Management: Continue to monitor and integrate performance, new
information, and lessons learned into programs and plans.
2012 Strategy Implementation To-Date
In October 2012 Deputy Assistant Administrator Mike Shapiro sent a memorandum to the NWP Office
Directors and Regional Water Division Directors requesting information on implementation to date. In
November and December 2012, OW collected baseline information from HQ and regional contacts on
progress to-date implementing strategic actions identified in the 2012 Strategy, and actions planned for
Fiscal Year 2013. Progress on achieving the goals and strategic actions and progress toward
implementing regional strategies will be compiled and reported annually. The final FY2013 Climate
Change Implementation Action Plan was distributed internally in early 2013. In March 2013, EPA
published the 2012 Highlights of Progress report summarizing the accomplishments of the NWP in
2012.12 This is the first progress report organized around the vision areas described in the 2012 Strategy.
A timeline for OW's climate change strategy work, and related milestones, is included below.
12 http://water.epa.gov/scitech/climatechange/upload/Final-2012-NWP-Climate-Highlights-Report-3-18-13.pdf
20
-------
lEc
EXHIBIT 1-2. EPA OFFICE OF WATER CLIMATE CHANGE STRATEGY TIMELINE
•March: EPA Assistant Administrator Ben Grumbles launches an OW Climate Workgroup to
develop the first OW climate change strategy. The Workgroup meets monthly from this time
to present.
r
2008
•September: 2008 National Water Program Strategy: Response to Climate Change released.
2009
•January: National Water Program Strategy: Response to Climate Change Progress Report
for 2008 released (first progress report of the 2008 Strategy).
•October: EPA launches the State and Tribal Climate Change Council of the NWP (STC3).
•October: President Obama signed an Executive Order directing the Interagency Climate
Change Adaptation Task Force to develop a report with recommendations for how the
Federal Government can strengthen policies and programs to better prepare the Nation to
adapt to the impacts of climate change.
2010
•January: National Water Program Strategy: Response to Climate Change Progress Report
for 2009 released (second progress report of the 2008 Strategy).
•February: Highlights of Progress 2007-2009: Implementing the NWP Strategy: Response to
Climate Change released.
•April: OW Climate Change Workgroup began developing a new climate strategy for the
NWP.
•August: National Water Program Climate Change Strategy: Key Action Update for 2010-
2011 released.
201
•June: EPA Administrator issued a policy statement on climate change adaptation directing
EPA to complete its Draft Climate Change Adaptation Plan.
•December: National Water Program Strategy: Response to Climate Change, 2010-2011
Nationaland Regional Highlights of Progress released (third progress report of the 2008
Strategy).
2012
•October: Michael Shapiro sent a memo to all NWP Office Directors and Regional Water
Division Directors requesting implementation activities defined for2013and reporting on
progress in 2012.
•December: EPA publishes National Water Program 2012 Strategy: Response to Climate
Change.
2013
•March: EPA publishes 2012 Highlights of Progress: Responses to Climate Change by the
National Water Program.
21
-------
lEc
CHAPTER 2 | LOGIC MODELS
As a first step in developing performance measures for government programs, evaluators typically
develops a logic model that describes how the program is intended to work. In particular, logic models
demonstrate how program resources and activities are designed to influence key audiences to affect
desired changes in understanding or awareness, behavior, and ultimate conditions.
The 2012 Strategy includes a vision for each of the five areas (and multiple goals associated with each
vision area). Since the reach of the 2012 Strategy is so comprehensive, the evaluation team modified its
typical approach to developing a logic model and has identified series of roles that EPA seeks to fulfill to
support each of its vision areas, rather than identifying specific program resources and activities. The
team then identified the primary audiences that Office of Water (OW) seeks to influence (e.g., states,
tribes, local authorities, or utilities) to achieve its goals. And finally, the logic models illustrate the
desired outcomes: changes in audience awareness and behavior that will lead to the desired conditions
if OW's strategy is successful. (See Exhibit 2-1.)
EXHIBIT 2-1. ORGANIZATION OF THE 2012 STRATEGY LOGIC MODELS
EPA Role I Primary I Audience I Audience I Conditions
I Audiences I Awareness I Behavior I
Developing logic models are helpful to inform the development of performance measures. Changes in
awareness, behavior, and conditions can translate into outcome measures. While implementing
outcome measures can be a challenge, which is discussed in Chapter 5, they are necessary to
understand program impact.
For the OW Strategy, it is important to note that external factors likely contribute to (or prevent) desired
changes in behavior and conditions, given that OW's direct sphere of influence for implementing the
2012 Strategy is limited, as illustrated with dotted lines on the logic models. The logic models identify
examples of external influences on the audiences OW seeks to affect, in a light green box on each
model. Measures based on changes in audience behavior cannot, in isolation, indicate the degree to
which OW's work led to the desired changes in behavior. By understanding the other factors outside of
OW control that shape the behaviors of key audiences, OW can consider where it has the most influence
and what roles it is best able to serve.
Exhibits 2-2 through 2-6 show the logic models that the evaluators developed for each of the five areas.
Note that they represent a simplified model of how the 2012 Strategy is intended to work. For a
complete articulation of OW's visions, goals, and strategic actions related to the NWP's Response to
Climate Change, see the full 2012 Strategy document.
Logic models typically include a depiction of the connections between the various components, e.g.,
how specific OW actions will affect particular audiences to lead to specified changes in awareness,
behavior, and conditions. We displayed such connections in the Area 1 logic model for infrastructure as
an example. We developed tables that describe the connections for the other logic models (see
Appendix A. While implementation of the 2012 Strategy will likely include feedback loops (e.g., changes
in behavior of one key audience may affect changes in awareness for another key audience), for the
sake of simplicity we have only shown linear relationships from left to right on the logic model (i.e., from
OW's vision and goals, to roles influencing audiences, to changes in awareness, behavior, and
conditions).
22
-------
lEc
EXHIBIT 2-2. AREA 1 LOGIC MODEL: INFRASTRUCTURE
Vision
EPA Role
Area 1
Infrastructure:
Resilient a
adaptable
drinking
water,
wastewater, &
stormwater
utilities
Incorporate
climate change
into planning &
decision-
making
Support
Integrated
Water
Resources
Management
Develop
partnerships
Foster
research
Develop water-
efficient
product
specifications
EPA Sphere of
Direct Influence
Examples of additional influences on primary audiences
• Utility commissions
• Federal, state & local government requirements
• Utility rates & funding for infrastructure
• Operating costs 6 electricity rates
• Customer behavior
• Population & economic growth
• Weather & climate conditions
1 Technological advances
Primary
Audiences
Provide tools &
information,
including
location-
specific
information
about impacts
Audience
Awareness
Audience
Behavior
Conditions
Understand
potential
climate change
impacts &
prioritize risks
Understand
adaptation
options
Understand
energy
efficiency and
sustainability
options
Understand
how to
promote
supply 6
demand
management
Implement
adaptation
measures
Reduced
vulnerability to
climate change
risks
Implement
sustainability
and efficiency
measures
Reduced utility
energy use
Take action to
manage water
supply &
demand
Consumers &
industry use less
water
23
-------
EC
EXHIBIT 2-3. AREA 2 LOGIC MODEL: WATERSHEDS AND WETLANDS
Vision
EPA Role
Primary
Audiences
Audience
Awareness
Audience
Behavior
Conditions
Area 2
Watersheds 6
Wetlands:
Watersheds &
wetlands are
protected,
maintained,
enhance
climate
preserve the
functions 6
benefits they
provide
EPA Sphere of
Direct Influence
Identify, protect,
6 maintain
healthy
watersheds 6
habitat corridor
networks
Incorporate
resilience into
watershed
floodplain
management
protection
practices
incorporate
Source Water
Protection to
protect drinking
water supplies
Incorporate
climate change
considerations
into CWA 404
program
Improve baseline
. f
wetland extent,
condition, 6
performance
Develop
partnerships
Develop policy
6 guidance
Develop tools
& technical
resources, 6
provide
technical
decision
support
Support
development
of data and
maps
Develop
indicators 6
track progress
Provide
Communicate
key issues
Coordinate
stakeholders
EPA Core
Programs
Other Federal
Agencies
States
Inter-state
Agencies
Tribes
Local
Watershed
Managers
Permitting
authorities
Private
Landowners
Water Sector
Utilities
NGOs
The Public
Awareness of
relationship
between 1 )
existing
programs (e.g.,
permitting 6
funding), 2)
watershed
climate change
Understand
importance of
wetlands
Sbwatersheds to
climate change
resilience
Understand
extent 6
condition of
wetlands 6
watersheds, 6
how they may be
climate change
Awareness of
tools 6 resources
to protect
watersheds &
wetlands
Integrate
climate change
and healthy
watersheds/
wetlands
considerations
into core
CWA 404 &
tory mitigation
Implement
projects to
protect and/or
restore
watersheds,
wetlands, and
techniques
that are
climate
resilient
Incorporate
factors into
plans for
watershed
restoration,
floodplain
management,
6 source water
Watersheds and
wetlands
continue to
provide
ecological
services, as
well as social
and economic
benefits, in
spite of climate
change
Examples of additional
influences on primary
audiences:
Statutory authority
and program funding
for government
agencies,
Land prices
Cost of capital
infrastructure for
private landowners,
water sector utilities.
24
-------
EC
EXHIBIT 2-4. AREA 3 LOGIC MODEL: COASTAL AND OCEAN WATERS
EPA Role
Primary
Audiences
Audience
Awareness
Audience
Behavior
Conditions
Area 3
Coastal and
Ocean: Ocean
and coastal
environment
protected
against climate
change and
against
unintended
adverse
consequences
of responses to
climate
change.
Support
collaborations
creating and
sharing of
information and
best practices
Develop
partnerships that
assist effective
adaptation action
for coastal and
ocean
environments
Ensure that
mitigation and
adaptation
measures are
environmentally
sound
Adjust EPA
programs to
incorporate
shifting
environmental
conditions and
other emerging
threats
EPA Sphere of
Direct Influence
l_
Foster
partnerships,
collaborations,
and
information
sharing
Provide
technical
assistance
Promote best
practices for
climate-
planning
Develop
climate-
readiness
federal
programs,
agencies, and
authorities
Develop
1 environmental
safety criteria
for offshore
renewables
and CCS
r
!
EPA's NWP and
NEP programs
U.S. National
Ocean Council
Regional ocean
organizations
State and local
watershed
organizations
Coastal
communities
and planners
Coastal
infrastructure
owners and
operators
r Examples of add
primary audie
• Shoreline deve
trends
• Weather, tidal
L* Technological c
CCS, renewabk
• Offshore navigc
• Renewable par
• Competing plar
• Emerging clima
Understand
strategies for
incorporating
adaptation into
federal policies &
programs
Aware of
adaptation
options
Aware of
relevant partners
6 opportunities
to collaborate/
share
information
Mindful of the
potential hazards
that offshore
renewables and
CCS may pose to
coastal and
ocean resources
tional Influences on
ices:
opment and real estate
and climate conditions
dvances (particularly
>s, and IT)
itional dredging
tfolio standards
ning considerations
te threats
Integrate
adaptation
considerations
into policies 6
programs at
the federal
level
Incorporate
climate change
& adaptation
considerations
into regional,
state a local
programs 6
plans
Engage in
collaborative
partnerships
that ensure
information-
sharing and
prevent
duplication of
efforts
Adjust offshore
renewables
and CCS
permitting
criteria to
consider
adverse effects
to ocean &
coastal
resources
Climate-change-
induced risks to
coastal and
ocean
ecosystems and
infrastructure
are minimized
Coastal and
ocean
environments
continue to
provide current
levels of
ecosystem
services and
socioeconomic
benefits
Coastal and
ocean
infrastructure
and ecosystems
are protected
against adverse
effects of climate
change
adaptation and
mitigation efforts
25
-------
EC
EXHIBIT 2-5. AREA 4 LOGIC MODEL: WATER QUALITY
Area 4
Water Quality:
Quality of
surface water,
drinking
water, 6
ground water
is protected 6
climate change
induced risks
to human
health and the
environment
are diminished
via adaptation
a mitigation
strategies
EPA Sphere of
Direct Influence
EPA Role
Protect waters
of the US a
promote
sustainable
management
of surface
water
resources
Protect water
resources from
unintended
consequences
of national
efforts to
reduce green
house gas
emissions a
develop
alternative
energy
Collaborate to
make
hydrological
and climate
data a
projections
available
Encourage
integration of
climate change
consideration
into existing
water quality
protection
measures,
including
permitting
Promote best
practices
Communicate
key issues
Develop
guidance a
provide tech.
decision
support
Coordinate
stakeholders
Provide policy
clarification
Support data
a new analytic
method
development
Primary
Audiences
Examples of additional influences on primary audiences
Federal, state, & local
government
requirements
Weather & climate
conditions
Economic considerations
Other EPA
offices
States
Inter-state
Agencies
Tribes
Local
Watershed
Managers
Permitting
Authorities*
Other Federal
Agencies
Audience
Awareness
Aware of
potential for
existing
programs ft
resources to
address
climate change
concerns
Understand
relationship
between best
practices (e.g.,
Gl a LID) a
climate change
mitigation/
adaptation
Aware of tools
a resources to
protect water
quality
Understand
potential
effects of
climate change
on current
water quality
standards
Audience
Behavior
Integrate
climate change
considerations
into federal,
state, a local
water quality
decisions
Update
hydrological
data a enhance
statistical
methods to
account for
changing
climate
conditions
Conditions
Climate change
induced risks to
human health a
the environment
are minimized
Surface water,
ground water, a
drinking water
quality remains
protected
+ln cases where EPA has not granted a state
primacy, permitting authorities may fall under
the Agency's direct sphere of influence.
26
-------
EC
EXHIBIT 2-6. AREA 5 LOGIC MODEL: WORKING WITH TRIBES
EPA Role
Area 5
Working with
Tribes:
Tribes are able
to preserve,
adapt, 6
maintain the
viability of
their culture,
traditions,
natural
resources, 6
economies in
the face of a
changing
climate
EPA Sphere of
Core programs
incorporate
climate change
considerations
a NWP
collaborates
with other EPA
offices, federal
agencies to
work with
tribes on
climate change
issues
Tribes have
access to
information on
climate change
for decision
making
Develop
guidance 6
provide
technical
decision
support
Support data
development
Provide &
facilitate use
of funding for
change
adaptation
and building
sustainability
Coordinate
stakeholders
Communicate
key issues
Primary
Audiences
Tribes
Audience
Awareness
EPA Core
Programs
Other EPA
offices
Other Federal
Agencies
Aware of
potential for
existing
programs 6
resources to
address
climate change
concerns
Examples of additional influences on primary audiences
- Federal, state, 6 local government requirements
• Weather 6 climate conditions
• Economic considerations
• Technological developments
Perspectives of community members
Understand
potential
effects of
climate change
on tribal
resources
Audience
Behavior
Conditions
Integrate
climate change
considerations
into
implementation
of core
programs for
tribal Nations
Utilize
established
programs and
funding
mechanisms to
address climate
change issues
Tribes maintain
and protect their
traditional
cultures, values,
and resources in
spite of climate
change
27
-------
lEc
CHAPTERS | METHODOLOGY
This evaluation sought to address the following research questions:
Evaluating Implementation of the 2008 Strategy:
-^ How well is climate mainstreamed into OW programs? What are the barriers to mainstreaming and
how might this be better accomplished?
A What goals, implementation experience, or lessons from the 2008 Strategy could be useful to guide
implementation of the 2012 Strategy?
A What goals and strategic actions in the 2012 Strategy should EPA headquarters (HQ) and regional
programs prioritize?
Developing Prospective Measurement Approach for the 2012 Strategy:
-^ What is the measurement approach that can be used to measure adaptation progress in five areas:
infrastructure, watersheds and wetlands, ocean and coastal waters, water quality, and working with
tribes?
-^ What specific elements need to be applied to the phased approach to tracking progress outlined in
the 2012 Strategy, to make it a robust measurement framework?
A What, if, any, revisions should EPA make to its baseline data collection process to ensure that data
collected are meaningful and objective?
A How can OW's measurement approach inform measuring progress in the EPA-wide Adaptation Plan,
and to inform development of the next Agency 4-year Strategic Plan?
The evaluation team used several methods to address these questions, including review of existing
literature and data, phone interviews, and a focus group. An EPA interview summary is presented in
Exhibit 3-1 below; in total the evaluators interviewed 27 EPA staff and 4 non-EPA staff. The evaluators
spoke with staff in all four OW program offices as well as staff from four different regions. More details
on interviews are provided later in this chapter.
The evaluation team undertook a literature review based on information sources provided by EPA as
well as other sources identified through online research. The following key EPA reports were referred to
extensively for both the retrospective and prospective components of the evaluation:
A EPA. 2012. National Water Program 2012 Strategy: Response to Climate Change. U.S. Environmental
Protection Agency, Office of Water, Washington, DC
^ EPA. 2008. National Water Program Strategy Response to Climate Change. EPA 800-R-08-001. U.S.
Environmental Protection Agency, Office of Water, Washington, DC
^ EPA. 2011. National Water Program Strategy: Response to Climate Change, 2010-2011 National and
Regional Highlights of Progress. U.S. Environmental Protection Agency, Office of Water, Washington,
DC
A EPA. 2010. Implementing the National Water Program Strategy: Response to Climate Change.
Progress Report for 2009. U.S. Environmental Protection Agency, Office of Water, Washington, DC
28
-------
EC
A EPA. 2010. EPA Regional and Large Aquatic Ecosystem Programs, Highlights of Progress 2007-2009,
Implementing the NWP Strategy: Response to Climate Change. U.S. Environmental Protection
Agency, Office of Water, Washington, DC
A EPA. 2009. Implementing the National Water Program Strategy: Response to Climate Change.
Progress Report for 2008. U.S. Environmental Protection Agency, Office of Water, Washington, DC
EXHIBIT 3-1. EPA INTERVIEW SUMMARY
EPA OFFICE/REGION
OW/IO
OAR
OGWDW
OP
ORD
OST
OWM
OWOW
Region 1
Region 3
Region 6
Region 10
Total
NUMBER OF INTERVIEWEES
5
2
5
1
1
1
4
3
1
1
1
2
27
The evaluation methodology is discussed in more detail below. First we discuss the methodological
approach used for the retrospective evaluation, followed by a discussion of the approach used for the
prospective measurement component.
METHODOLOGICAL APPROACH FOR THE RETROSPECTIVE EVALUATION
Data Sources: Retrospective Evaluation
The retrospective evaluation used multiple data sources: 1) interviews with EPA representatives and
representatives of organizations external to EPA, 2) a focus group to further discuss lessons from the
2008 Strategy and recommendations for implementation of the 2012 Strategy, and 3) a review of
existing literature related to the NWP climate change strategies and there implementation, including
EPA memoranda and other policy documents.
Interviews with EPA Representatives and Representatives of Organizations External to EPA
The evaluation team conducted a series of discussions with EPA OW staff as well as representatives from
organizations outside of EPA with a connection to OW's climate change strategy. OW staff from EPA HQ
and regional offices provided insights on "vertical" implementation (i.e., dissemination and
implementation of strategy from OW HQto regional offices) and "horizontal" implementation (i.e.,
dissemination and implementation of strategy across OW offices) of the 2008 Strategy. Representatives
from external organizations provided additional perspective on OW's implementation activities.
Interviewee affiliations and number of interviewees are summarized in Exhibit 3-2 below.
29
-------
EC
EXHIBIT 3-2. RETROSPECTIVE EVALUATION: INTERVIEW SUMMARY
EPA INTERVIEWEE
AFFILIATION
ow- 10
OW - OST
OW - OGWDW
OW - OWM
OW- OWOW
OAR
OP
ORD
Region 1
Region 3
Region 6
Region 10
NUMBER OF
INTERVIEWEES
5
1
5
4
3
2
1
1
1
1
1
2
EXTERNAL INTERVIEWEE
AFFILIATION
Association of State Drinking
Water Agencies (ASDWA)
Association of State Wetland
Managers (ASWM)
Port Gamble S'Klallam Tribe
NUMBER OF
INTERVIEWEES
1
2
1
A set of evaluation questions were included in a set of discussion guides tailored to each group of
participants (EPA staff, federal partner agency staff, State-Tribal Climate Change Council, and other EPA
partners and stakeholders). Each evaluation participant received a discussion guide prior to his or her
interview.
Supplemental Focus Group Information Collection
In addition to phone interviews with evaluation participants, the evaluators conducted a half-day focus
group in Washington, D.C. on Monday, March 25. The group consisted of representatives from IO,
OWM, OGWDW, OWOW, OWM, Region 1, and OST. Eight out of the nine focus group participants were
also interviewed individually by the evaluation team. The purpose of the focus group meeting was to
provide a structured opportunity for selected OW policy staff and managers to reflect together about
lessons from implementation of the 2008 Strategy, and to jointly explore potential recommendations for
improving implementation of the 2012 Strategy. The focus group was timed to occur after most of the
data collection was completed by the evaluators. This enabled the evaluators to explore with the focus
group potential findings from interviews that may diverge from the perspectives raised in the focus
groups. It also provided a valuable opportunity for the evaluators to better understand potential issues
and trade-offs inherent to the framing of various types of recommendations.
Additional Literature Review
In addition to the key references noted above, the evaluation team consulted the following literature for
the retrospective evaluation:
A EPA. 2009. Charter for EPA State-Tribal Climate Change Council of the National Water Program. U.S.
Environmental Protection Agency, Office of Water, Washington, DC
A EPA. Climate Change and the National Water Program. Memorandum from Assistant Administrator
for Water Benjamin Grumbles
Also, during the course of the evaluation, discussion participants also suggested additional documents,
some of which were provided to the evaluation team:
A Summary of comments from ASDWA on draft 2012 NWP Climate Change Strategy
30
-------
EC
A Agendas and minutes from OW Climate Change Workgroup
A Memorandum from Deputy Assistant Administrator Mike Shapiro "Reporting of Climate Change
Progress and Development of FY13 Climate Change Implementation Action Plan"
Data Analysis: Retrospective Evaluation
The evaluation questions were considered in the context of basic strategy development and
implementation cycle, as illustrated in Exhibit 3-3 below. The questions in the evaluation focused on the
strategy implementation process/systems and effectiveness. The evaluation did not explicitly focus on
the strategy development process; although the evaluators captured and shared some findings related
to the strategy development process that interviewees indicated may affect the success of strategy
implementation. Similarly, the evaluation did not focus on the extent to which implementation of the
2008 Strategy resulted in specific outcomes related to climate change and adaptation, except to the
extent that interviewees indicated that specific outcomes (or lack thereof) informed understanding of
the effectiveness of the strategy or influenced actions or decisions under the National Water Program.
EXHIBIT 3-3. STRATEGY DEVELOPMENT AND IMPLEMENTATION CYCLE
Strategy '^ Strategy '^^ Strategy ^^^ Strategy
D it ^r lmP'ementat'on ^^r Implementation ^^* Implementation
,^^r Process/Systems ^^p Effectiveness ^B Outcomes
In addition to strategy implementation effectiveness and strategy implementation process/systems, the
evaluators explored perspectives on the degree to which climate change is integrated into the National
Water Program (NWP). To the extent possible, the evaluators also solicited perspectives on the
connections between strategy implementation effectiveness, strategy implementation process/systems,
and climate change integration. Finally, the evaluators collected perspectives on whether and how
actions should be further prioritized for implementation under the 2012 Strategy.
The evaluators analyzed collected information question-by-question to develop findings and
recommendations. The evaluators analyzed discussion notes and identified common perspectives and
notable outliers with respect to the topics discussed. They examined patterns across discussions to
identify whether, for example, lessons learned are specific to certain circumstances or perspectives.
Where possible and appropriate, the evaluators summarized findings in quantifiable terms (such as all,
many, most, only one) to provide a sense of scope or scale of thematic findings.
To the extent possible, information provided by evaluation participants was supplemented by
information in existing written materials. Instances where written information and information from
discussions contradict each other is noted and analyzed. Inconsistencies between EPA and external
personnel's perspectives is also noted and analyzed, as are differences in responses across divisions and
offices within EPA. The evaluators' final conclusions are based on the entire set of data sources,
including an analysis both within and across the overarching topic areas. In the sections that follow, the
terms "evaluation participant(s)", "participant(s)", and "interviewee(s)" are used interchangeably to
refer to the individuals who provided their perspectives to the evaluators.
31
-------
lEc
METHODOLOGICAL APPROACH TO PROSPECTIVE PERFORMANCE MEASUREMENT
Data Sources: Prospective Measurement Approach
The prospective analysis drew upon multiple data sources to inform the lessons learned related to our
research questions on measurement. Key sources of information include: (1) baseline data collected by
OW to assess progress towards the strategic actions included in the 2012 Strategy, (2) phone interviews
with EPA representatives from all four program offices and in IO, and (3) a review of existing literature
and online publications that address climate change adaptation strategies and activities. OW gathered
the strategic action baseline data, while the evaluation team gathered the remaining data conducted all
the analyses. The following text describes these data sources in greater detail. It should be noted that
proposed measurement approaches and lessons learned discussed in this chapter are also informed by
feedback provided by participants in the retrospective evaluation of the 2008 Strategy, and specifically
feedback from focus group participants and interviewees.
Baseline Data Collected by 10
In November and December 2012, IO collected baseline information from HQ and regional contacts on
progress to-date implementing the 53 strategic actions identified in the 2012 Strategy in the five vision
areas (infrastructure, watersheds and wetlands, coastal and ocean waters, water quality, and working
with tribes) and cross-cutting areas of program support, and actions planned for Fiscal Year 2013. For
each strategic action, HQ offices and regions were directed to 1) report on recent progress,
outputs/outcomes, milestones achieved, and key partners; 2) report on current activities underway; and
3) assess which of the seven phases best described progress toward implementing the strategic action.
The baseline data submitted is anecdotal in nature. Nine regions and seven offices/divisions submitted
baseline data for 46 of the 53 strategic actions; for most strategic actions regions and offices/divisions
reported three or fewer related activities. Appendix B shows the count of activities submitted and the
reporting HQ offices and regions for each strategic action. Note: as part of the baseline data collection
OW also asked regions to report on a separate set of strategic actions for each climate region in the
Geographic Region section of the 2012 Strategy. The evaluation team did not review this information as
part of our assessment because it was not directly related to the evaluation questions, which primarily
focused on measuring progress in the five vision areas.
Interviews With EPA Representatives
The evaluation team conducted a separate set of interviews with OW staff and managers to discuss
approaches to measuring progress on climate change adaptation. We conducted interviews with one or
two representatives working on each vision area in the 2012 Strategy, and we also interviewed a
representative IO for a broader perspective related to all five vision areas. Many of these participants
were also interviewed for the retrospective evaluation. Interviews specific to each vision area focused
on how to measure progress towards the strategic actions and towards the adaptive management phase
for each vision area. During the interviews, we also: addressed ways to collect data from stakeholders
outside of EPA; gathered feedback on our preliminary assessment of progress in each vision area; and
made specific inquiries about baseline data submissions. The broad interview related to all five vision
areas focused on OW's needs and challenges related to measuring progress on climate change
adaptation, and the interests and concerns of groups within OW related to measurement. Exhibit 3-4
provides a summary of the EPA OW offices that participated in each interview.
32
-------
EC
EXHIBIT 3-4. PROSPECTIVE EVALUATION: INTERVIEW SUMMARY
VISION AREA
Area 1:
Area 2:
Area 3:
Area 4:
Area 5:
Infrastructure
Watersheds and Wetlands
Coastal and Ocean Waters
Water Quality
Working with Tribes
All Areas
OW OFFICES REPRESENTED
OGWDW, OWM
OGWDW, OWOW
OWOW, OST
OST, 10
10
10
Additional Literature Review
In addition to the key references noted above, the evaluation team reviewed the following documents
from the UK government related to measuring climate change adaptation:
A Adapting to Climate Change: Guidance notes for NI188, 2010
A Measuring Adaptation to Climate Change - A Proposed Approach, 2010
A Adapting to Climate Change: A Guide for Local Councils, 2010
A Adapting to Climate Change in the UK: Measuring Progress - Adaptation Sub-Committee Progress
Report, 2011
The evaluation team reviewed these documents because the UK government preceded EPA in defining
key steps in the process of climate change adaptation, and developing a general approach for assessing
the extent to which government agencies and their partners are undertaking those steps. The UK
government developed an approach to gauge progress of local areas in:
A Comprehensively assessing the risks and opportunities of climate change;
A Developing an adaptation strategy and action plan that identifies risks and priority areas;
A Taking action in priority areas; and
A Implementing, assessing, and monitoring the actions on an ongoing basis. 13
EPA's proposed framework with seven phases of adaptive management echoes the UK concept of
measuring progress in the process of climate change adaptation.
In addition, the evaluation team reviewed several reports and online resources that describe progress
on climate change adaptation. The names of the resources reviewed are shown in Exhibit 3-5, and a
summary of each resource is provided as Appendix C. In both Exhibit 3-5 and Appendix C, we categorize
the resources by their primary author (i.e. EPA Resources, Other Federal Government Resources, State
and Local Resources, Other Resources). Appendix C indicates the vision areas that each resource
addresses.
UK Department for Environment, Food, and Rural Affairs, 2010. Adapting to Climate Change: Guidance notes for
NI188, March 2010. Available at: http://www.defra.gov.uk/environment/climate/sectors/local-authorities/; and
http://archive.defra.gov.uk/corporate/about/with/localgov/indicators/documents/nil88-guidance.pdf.
33
-------
EC
EXHIBIT 3-5. SUMMARY OF EXISITING REPORTS/RESOURCES REVIEWED
RESOURCE
CATEGORY
RESOURCE TITLE (AUTHOR)
EPA Resources
Climate Ready Estuary Projects-Where you live-2008 - 2012 (EPA)*
GREAT analytics data (EPA)*
EPA Coastal Area Impacts and Adaptation (EPA)*
EPA's Water Infrastructure Website (EPA)*
Workshop Proceedings on Water Infrastructure Sustainability and Adaptation to Climate Change (EPA)
Other Federal
Government
Resources
Addressing Climate Change in Long-Term Water Resources Planning and Management (DOI and U.S. Army
Corps of Engineers)
National Climate Assessment, Adaptation Section - draft 2013 (National Climate Assessment and
Development Advisory Committee)
NOAA Coastal Climate Adaptation and Action Plans (NOAA)*
State and Local
Resources
Climate Change Adaptation for Maryland Water Utilities (Maryland DOE)
Climate Change Handbook for Regional Water Planning (California DWR)
Confronting Climate Change - An early analysis of water and wastewater adaptation costs (Association of
Metropolitan Water Agencies)
Managing an Uncertain Future - Climate Change Adaptation Strategies for California's Water (California DWR)
Preliminary Report of the Interagency Climate Adaptation Team (Minnesota PCA)
State and Local Adaptation Plans (Georgetown Climate Center)*
The Oregon Climate Change Adaptation Framework (Oregon DOE)
Other
Resources
Adaptation Planning - What U.S. States and Localities are Doing - 2009 (Pew Center for Global Climate
Change/Center for Climate and Energy Solutions)
Climate Change Adaptation Plan for Coastal and Inland Wetlands in the State of Michigan - 2012 (The
Association of State Wetland Managers, Inc.)
Climate Change Adaptation: What Federal Agencies Are Doing - 2012 (Center for Climate and Energy
Solutions)
State Wetland Climate Change Adaptation Summaries - 2010 (The Association of State Wetland Managers,
Inc.)
StormSmart Coasts (StormSmart Coasts Network)*
Swinomish Climate Change Initiative - Climate Adaptation Action Plan (Swinomish Indian Tribal Community)
The Role of Coastal Zone Management Programs in Adaptation to Climate Change (Coastal States
Organization)
Tribal Climate Change Project (University of Oregon)*
Tribes & Climate Change (Institute for Environmental Professionals)*
Yurok Tribe and Climate Change: An Initial Prioritization Plan (Kathleen Sloan and Joe Hostler - Yurok Tribe
Environmental Program)
'indicates website
Data Analysis: Prospective Measurement Approach
To explore developing a performance measurement framework, the evaluation team conducted a
qualitative analysis of responses to the interview questions, the strategic action baseline data submitted
by the HQ offices and regions, and the existing reports and resources regarding progress on climate
change adaptation. The specifics of the approach vary for the evaluation questions and are described
below:
a) What is the measurement approach that can be used to measure adaptation progress in five
areas: infrastructure, watersheds and wetlands, ocean and coastal waters, water quality, and
working with tribes?
b) What specific elements need to be applied to the phased approach to tracking progress
outlined in the 2012 Strategy, to make it a robust measurement framework?
To develop the lessons learned related to these questions, we analyzed OW's baseline data for the
strategic actions, assessed input from interviews with EPA staff and managers, and reviewed existing
34
-------
EC
publications relevant to climate change adaptation. In addition, we assessed OW's progress on the
seven phases of adaptive management, based on the criteria OW established in the 2012 Strategy and
the baseline data submitted by HQ offices and regions.
What, if, any, revisions should EPA make to its baseline data collection process to ensure that data
collected are meaningful and objective?
The evaluation team reviewed and assessed each strategic action baseline data submission to develop
lessons learned for this evaluation question. We considered the format of the baseline data collection
template as well as the content of the submissions. Based on this review, we developed
recommendations to improve data quality in future collection efforts.
How can OW's measurement approach inform measuring progress in the EPA-wide Adaptation Plan,
and inform development of the next Agency 4-year Strategic Plan?
To address this evaluation question, we identified lessons learned from this evaluation effort that inform
other offices' effort to develop implementation plans for the EPA-wide Adaptation plan, and that inform
development of the next EPA Strategic Plan.
35
-------
EC
CHAPTER 4 | LESSONS LEARNED FROM EVALUATION OF 2008 STRATEGY
IMPLEMENTATION
In this chapter, we describe findings and lessons from a retrospective evaluation of the 2008 Strategy.
The purpose of the retrospective evaluation is to gather perspectives about how EPA OW could improve
implementation of the NWP's climate change strategy. Key questions addressed in this evaluation are:
A How well is climate mainstreamed into OW programs? What is the status of mainstreaming, what
are the primary mainstreaming barriers, and are there examples of successful mainstreaming?
A What goals, implementation experience, or lessons from the 2008 Strategy could be useful to guide
implementation of the 2012 Strategy?
A What goals and strategic actions in the 2012 Strategy should headquarters and regional programs
prioritize?
FINDINGS AND LESSONS
How well is climate change mainstreamed into the NWP?
The evaluators' overall assessment is that mainstreaming of climate change in the NWP has begun but is
generally at an early stage with substantial opportunity for integration remaining. EPA OW appears to be
generally viewed as any early leader and pioneer within the U.S. Government in responding to climate
change, although there is a perception within and external to OW that other agencies and programs are
catching up. A few OW programs and regional offices are regarded as being further along with
mainstreaming due in part to the needs of key stakeholders and presence of management-level
champions. The following section explores the status of efforts to mainstream consideration of climate
change throughout programs and activities in the NWP and provides examples of successful
mainstreaming programs. These findings provide lessons for implementing the 2012 Strategy, which are
further explored in the recommendations section of this report.
Mainstreaming Status
Overall, the evaluators found that the degree of integration of climate change into the National Water
Program is low but improving. External interviewees credited OW for its willingness to begin a
dialogue around climate change but noted that progress has been slow and halting.
In addition to determining the value and influence of the 2008 Strategy, the retrospective evaluation
gathered feedback on interviewees' impression of the degree to which climate change is integrated (or
"mainstreamed") into the NWP. The evaluators framed this discussion by defining a fully mainstreamed
situation as one in which climate change factors into daily routines, duties, and decision making.
Evaluation participants were asked to rate their opinion of mainstreaming on a 1 to 5 scale, with 1 being
"not at all" and 5 being "completely integrated". Participants were then asked to explain the rationale
behind their rating.
The average rating of all responses to this question was 2.5; Exhibit 4-1 below illustrates the range of
responses by respondent affiliation.
36
-------
EC
EXHIBIT 4-1. EXTENT OF MAINSTREAMING ACROSS OW PROGRAMMING, PERSPECTIVES BY
AFFILIATION
Fully Integrated
Not Integrated
External OGWDW OW 10 OW OST OW OWM Regional Office
Evaluation participants generally indicated that climate change is not at the forefront of OW staff
concerns. One OW staff member noted that although climate change appears to be a significant issue
for the IO, prioritizing projects with climate change components is not being emphasized at the staff
level. There was a sense among many evaluation participants that integration of climate change into the
NWP was following a positive trend but as yet it is at a nascent stage of development. Participants
acknowledged the difficulty in integrating climate change into daily OW operations, given the magnitude
and cross-cutting nature of the problem. At the same time, several interviewees mentioned that the
very attributes that make climate change a complicated challenge - significant magnitude coupled with
cross-cutting application - also created a unique opportunity to advance cooperation across OW
divisions and encourage collaboration on groundbreaking, cost-effective solutions.
Interviewees outside of EPA rated OW's climate change integration efforts between 1 and 4, with a
mean rating of 3.4. External interviewees that gave a higher rating noted that OW is more consistently
referring to climate change as a background issue when discussing other topics, and grant applications
often include either a mandatory climate change component or the flexibility to address climate
considerations. External interviewees that rated OW's integration efforts as 1 to 2 observed that OW has
consistently signaled, through the 2008 Strategy and 2012 Strategy, that it is keenly interested in
tackling climate change; however, OW's actual ability to incorporate climate change into specific areas
covered under the Clean Water Act is less clear, not only to outside entities but also, it seems, within
OW itself. External interviewees commented that OW appears willing to consider these types of
discussions but EPA has not to date deciphered the legal boundaries around incorporating climate
change considerations.
Respondents were also asked to provide feedback on the status of climate mainstreaming within their
division or regional office. The results of these responses are illustrated in Exhibit 4-2 below.
37
-------
EC
EXHIBIT 4-2. EXTENT OF MAINSTREAMING WITHIN EACH OFFICE
Fully Integrated
Not Integrated
OGWDW OWIO OWOET OW OWM OWOW Regional Office
Some participants observed greater progress within specific OW offices (for example, OGWDW),
whereas most OW offices were rated, on average, as having a low degree of mainstreaming. Regional
offices reported a wide variation in regional investment in climate change. Other OW offices reported a
wide range of integration among divisions and offices, although mainstreaming climate change is
generally recognized as an important goal.
Mainstreaming Barriers
The evaluators found a range of barriers preventing the integration of climate change considerations
into OW's daily operations. Key barriers included competing priorities, lack of resources,
characteristics of climate change as an issue, organizational and structural aspects of OW, and
external influences.
Many interviewees observed that in the context of severe budget and staffing constraints, it can be
difficult for issues such as climate change to compete for limited time and attention. The NWP faces
numerous priorities, and those which are driven by legislative, regulatory, or court-mandated
requirements are often prioritized.
Several interviewees noted that the inherent complexity, uncertainty, variability, and long-term time
frame of climate change were barriers to mainstreaming into the NWP. This was coupled with the
overarching nature of the problem, in that climate change affects multiple sectors and exacerbates
many other issues, ranging from water availability to water quality to sea level rise. Participants noted
that while EPA permits are generally written for five year increments, the programs most impacted by
climate change should think about setting standards over a 20 - 30 year time frame.
A related barrier cited by several interviewees was the knowledge base within OW. One interviewee
noted that offices must go through the thought exercise of considering climate impacts on their core
responsibilities to internalize the concept that failing to account for climate change could have severe
consequences for their programs.
One participant noted that OW still lacks a strategic perspective on mainstreaming climate change into
existing programs, noting that the 2008 Strategy and 2012 Strategy more closely resemble workplans or
reporting mechanisms rather than true strategies. The interviewee remarked, "If you are building an
overarching strategy for climate change in OW, the document should be no more than ten pages long."
38
-------
EC
An external interviewee observed that the NWP has many responsibilities rooted in engineering and
OW's institutional knowledge and expertise is based in engineering (e.g., building wastewater treatment
plants). An issue such as climate change impacts watershed ecology and biology, which requires a
different set of knowledge and expertise.
Several interviewees commented on the difficulty in breaking down institutional silos and changing the
mindset of program staff to think of themselves and their work as fitting into part of a larger
organization.
The lack of continuity associated with a constantly shifting roster of branch chiefs, division directors, and
deputy office directors was noted as a barrier to integration. A participant in the evaluation remarked "it
takes 10,000 repetitions to create muscle memory. The same is true of integrating this type of priority
across an organization; it has to be part of the regular conversation." In addition, participants noted
multiple initiatives emerging from IO, including urban waters, integrated water resource management
(IWRM), and several programs aimed at promoting energy efficiency in the water sector. Each initiative
requires a significant set of resources and participants noted no clear signal on which was most
important; and if they are all important there is only so much that can be done for each, and not much
will be accomplished.
One participant noted that OW's CWA and Safe Drinking Water Act (SDWA) authorities include adaptive
management processes that allow updates to permits and standards based on changing conditions, of
which climate change is one. Another interviewee noted, however, that until climate change is explicitly
acknowledged within OW's regulatory processes it will not be fully integrated into the NWP. For
example, regulatory drivers such as NPDES regulations and SDWA regulations for carbon capture and
storage have been discussed within OW but there has been no action to date. One participant posed the
question "Can OW consider the carbon sequestration potential of a wetland when making a permit
decision?", and noted that the ability to provide a clear response to these and other questions will send
a strong signal about OW's commitment to integrating climate change.14 In addition, drinking water
regulations are also often tied to state or local regulatory structures around water availability, water
rights, and water withdrawal rates, which further complicates OW's ability to add climate change to the
regulatory lexicon.
Interviewees noted that political and public support and
emphasis for climate change policy is capricious and
often based on reaction to major events (e.g.,
hurricanes, flooding). Public awareness and acceptance
of climate change is still not at a favorable tipping point,
and the perception persists in some quarters that EPA is
a "job killing" regulatory agency. Interviewees added that
this obstacle is exacerbated by current economic
conditions, with public and political attention
concentrated on lowering the unemployment rate and
"We are still suffering from some crazy
torturing of language due to climate
change skeptics; this is why we now call it
climate change rather than global
warming. The external conversation is
still unclear, and in some cases we still
have to approach it more obliquely than
we prefer."
- OW Manager
The 2012 Strategy includes 7 strategic actions related to regulatory programs, including significant degradation
as part of Section 404 wetland permitting; however, the 2012 Strategy is silent on integrating climate change into
any new or planned rulemakings. In March 2013 OW released a "Highlights of Progress" document on NWP
response to climate change. The highlights included integration of climate change into 2012 HQ Wetland Program
Development Grants RFP announcement.
http://water.epa.gov/scitech/climatechange/2012-National-Water-Program-Strategy.cfm
39
-------
EC
encouraging growth. Furthermore, the public at large perceive the impacts of climate change will occur
far in the future, lowering the sense of urgency to take action.
Despite more frequent and damaging storm events, the utility sector still perceives climate impacts as
something that can be dealt with within regular operations. Utilities have to explain to their board of
local elected officials why they need to take action on climate change; they need the knowledge base to
explain climate projections and uncertainty in order to make a compelling case. Climate models are very
good at the macro level but often lack the ability to translate impacts down to the local scale. This type
of information is critical for local planners to be effective.
Regional interviewees remarked that EPA has just begun the dialogue with states around climate
change, and most environmental programs are implemented by states. They observed that in some
"red" states, state government officials face political constraints regarding their ability to recognize
climate change as an issue and EPA regional offices must be sensitive to these politics. In these
situations, EPA regional offices have had some success in engaging state and local partners on specific
topics associated with climate change, such as storm water management needs associated with severe
weather events.
Mainstreaming Successes
The evaluators found greater flexibility in voluntary OW programs as opposed to regulatory programs;
Climate Ready Water Utilities (CRWU) and Climate Ready Estuaries (CRE) were frequently cited as
positive examples of integrating climate change considerations into the NWP. At the EPA regional
level, participants cited Region 1 as a strong example of successfully integrating climate change into
operations.
Interviewees' examples of mainstreaming
successes fell broadly into three categories:
voluntary programs, regulatory programs, and
programs targeting non-climate related outcomes.
Examples of each type of success are described in
greater detail below.
Throughout the discussions, evaluation participants
consistently brought up two programs as examples
of successful integration of climate change
considerations into day-to-day operations: Climate
Ready Water Utilities (CRWU) and Climate Ready
Estuaries (CRE) - see sidebar for program
descriptions. Both programs work with a well-
established network of organizations outside of
EPA; in the case of CRWU, OW staff works with
national utility associations and CRE taps into the
expertise of organized coastal watershed groups.
The CRWU program emphasize creating
practicable, usable tools - filling a niche which had
until then been the provenance of high level
atmospheric scientists. At least one participant
commented that because CRWU is mostly focused
on its own constituencies, there is little interaction
with the rest of OW. This effectively limits
Climate Ready Water Utilities
The CRWU initiative provides several useful
tools and resources for utility owners and
operators. These include the Climate
Resilience Evaluation and Awareness Tool
(GREAT), Adaptation Strategies Guide, a
searchable resource library, and an Extreme
Events Workshop Planner.
http://water.epa.gov/infrastructure/watersecu
rity/climate/index.cfm
Climate Ready Estuaries
The Climate Ready Estuaries program works
with the National Estuary Programs and the
coastal management community to: (1) assess
climate change vulnerabilities, (2) develop and
implement adaptation strategies, and (3)
engage and educate stakeholders. CRE shares
NEP examples to help other coastal managers,
and provides technical guidance and assistance
about climate change adaptation.
http://water.epa.gov/type/oceb/cre/index.cfm
40
-------
EC
communications around creative tools and approaches that could potentially provide value to other OW
offices. The CRE program is well-marketed and benefits from immediate buy-in at the state level, which
is critical since states can advocate for programs at EPA.
Participants characterized states in Region 1 as having a strong environmental ethic, with member states
in close proximity reinforcing the importance of coordination. Region 1 Administrators dating back to at
least 2001 have been interested in climate change issues; Regional Administrator Robert Varney created
a team to work on climate-related issues, calling it an "energy" team in deference to political conditions.
His successor, current Region 1 Administrator Curt Spalding is concerned with climate change and
previously worked for environmental organizations on coastal and estuarine issues. Region 1 academic
institutions with significant climate expertise were also cited as strong contributors to the overall
regional emphasis on climate change, and recent significant climate events reinforce the need for
climate change action and keep the issue at the forefront of citizens' concerns. As one participant noted:
"Every year there has been a 'case study' of climate change impacts: Hurricane Irene, flooding in Rhode
Island, New Hampshire, Maine, Hurricane Sandy, variability in snowfall, blizzards. Residents are seeing
these issues and seeing the vulnerability to climate change."
A widely held perception among interviewees was that more work is needed to integrate climate change
into the regulatory side of EPA activities. Interviewees acknowledged that incorporating climate change
considerations into regulatory programs and existing work requires a significant commitment of time
and resources. Examples of initial efforts to incorporate climate change into OW regulatory programs
include:
-^ Region 10 is in the midst of a pilot nonpoint source TMDL for the Nooksack River as an example of
incorporating climate change into daily program operations. Pilot project partners include the
University of Washington Climate Impacts Group, Nooksack Tribe staff, Lummi Nation staff, EPA
ORD - Corvallis, and Washington State Department of Ecology. As the pilot concludes, Region 10
hopes to identify elements that can be replicated in other TMDL processes.15
-^ In 2010 OWM hired a contractor to develop an analysis of the NPDES program with
recommendations on how the program could respond to climate change. The contractor concluded
that there was significant flexibility under NPDES to incorporate climate change. A few interviewees
expressed disappointment that the results of the analysis have not been widely shared and that
there does not appear to be significant follow-up work in recent months.16
Although not directly addressing climate change, WaterSense, Green Infrastructure, and programs
focused on water utility energy efficiency were also mentioned as success stories for mainstreaming of
climate change considerations, since both programs provide climate benefits. One participant noted
15 This example is also described in the 2012 "Highlights of Progress" document.
16 The 2012 "Highlights of Progress" document notes that the 2012 version of the NPDES Permit Writers' Manual
includes new text for consideration of climate change:
• Section 5.2.2.7: Apply Additional Regulatory Considerations in Calculating Thermal Discharge Limits - Clean
Water Act Section 316(a) Variance: Permitting authorities should be aware that the effects of global climate
change could alter the thermal profile of some receiving waters making the historical record of thermal conditions
less representative of future conditions.
• Section 6.2.4.2: Receiving Water Critical Conditions Receiving Water Upstream Flow: Modelers should be aware
that the effects of climate change could alter historical flow patterns in rivers and streams, making these historical
flow records less accurate in predicting current and future critical flows.
41
-------
EC
that OWM's stormwater program within the Green Infrastructure Program provides a great dual benefit
of improved stormwater management within the urban atmosphere while allowing certain climate
considerations to be used as those types of green infrastructure adaptation measures/improvements
are put in place. Participants also mentioned greater receptivity to green infrastructure concepts-
particularly those concepts that emphasize system resiliency - in coastal communities where the sense
of climate impacts is much more imminent. Region 10 also reported that it chose to incorporate climate
change and sustainability into the CW SRF program as part of its on-site review process, particularly for
facility planning. Oregon DEQ has been receptive to this concept and has invested in circuit riders to
provide information and expertise to small communities in the state. Region 10 is working with the
national EPA SRF coordinator to revise the SRF annual review guidance to include climate adaptation.
What goals, implementation experience, or lessons from the 2008 Strategy could
be useful to guide implementation of the 2012 Strategy?
This section includes findings (and associated lessons) on the overall effectiveness of the National Water
Program Strategy: Response to Climate Change (the 2008 Strategy)—as both a document and a
process—for affecting change in and through the NWP. Where appropriate, the section includes
reflections and findings related to the 2012 Strategy, recognizing that implementation of the 2012
Strategy is just beginning.
Strategy Purpose and Focus
The evaluators' assessment is that the 2008 Strategy purpose and focus was generally appropriate and
useful, although the strategy was quickly outgrown by the NWP. A key finding mentioned by several
evaluation participants was that the 2008 and 2012 Strategies do not go far enough to accommodate
the fundamental needs that climate change may necessitate for water resource management.
The purpose of the 2008 Strategy was to bring attention in the NWP to the issue of climate change and
to drive near-term efforts to expand understanding and action within EPA, while also demonstrating
EPA's climate change commitment to external audiences. The structure and focus of the 2008 Strategy
clearly reflects this purpose. First, the strategy documents diverse "climate impacts on water resources"
to raise awareness of climate change's relevance to water programs. Second, the strategy organizes 44
near-to-mid-term "key actions" under five goals. While two of the goals (and key actions under them)
focus on direct efforts to support climate change mitigation and adaptation activities, the other three
address research, education, and management systems to enhance NWP capacity to respond to climate
change.
Language in the 2008 Strategy suggests that it was intended to serve
a diverse set of audiences—both internal to and external to the OW.
The strategy made the case for expanding understanding and
capacity within EPA to respond to climate change across its water
programs, and included commitments from EPA programs to
advance progress towards these goals. The strategy also stated its
intent with regard to external audiences: "This document expresses
the National Water Program's commitment to work in cooperation with national partners, State and
local government, and public and private stakeholders to understand the science, develop tools, and
implement actions to address the impacts of climate change on water resources."17
Most evaluation participants regarded the 2008 Strategy as "a good first effort." The document
"The 2008 Strategy was EPA's
first attempt at doing this. It
was a good first try, but
clearly a first try."
- External Interviewee
17
2008 Strategy, p. iii.
42
-------
EC
described the relevance and importance of climate change to the NWP and provided a consolidated list
of near-term actions to advance progress in responding to climate change. While some evaluation
participants regarded the 2008 Strategy more as "a workplan/' it did signal EPA's commitment to
address the topic, backed by specific near-term actions, to internal and external audiences. Evaluation
participants widely observed that the limitations of the 2008 Strategy were recognized by OW
leadership. This was reflected by activities, beginning in 2010, to engage representatives from across the
NWP in developing a new strategy which could provide a longer-term framework for responding to
climate change.
While the 2012 Strategy retains the mam overall purpose and ,,_,., -----^ ^ ,
, . . , The 2012 Strategy shows
audiences as the 2008 Strategy, it shifts its focus to providing a , ....
, , , , how our approach is evolving.
longer-term road map for responding to climate change. In doing so, „. , ,...,.,.
, , , ,., , , , , , ltismoreforwardthinking.lt
the 2012 Strategy reads less like a workplan and is more clearly , . . , .
, , ,. , , , has more buy-in from people.
designed to serve as a framework under which annual planning can
occur.18 Many evaluation participants regarded this shift in focus -, , ^ / - /
, . with what we are doing and
away from including specific commitments as an important
The content is more in tune
with what we are doing c
where we are heading."
- OW Staff Member
evolution in order for the 2012 Strategy to be useful over a longer
time horizon. However, some evaluation participants observed that
in practice, some of the strategic actions in the 2012 Strategy are
overly specific or worded in a way that makes it difficult to plan actions and conduct reporting under. A
few participants noted that the vision statement and the ten principles in the 2012 Strategy provide an
important starting point for on-going strategic discussions that will be important for the NWP in the
years to come.
Some evaluation participants stated that they believe that the 2008 and 2012 Strategies "miss the mark"
in not going far enough to accommodate and address the fundamental needs that climate change may
necessitate at the national and local levels. For example, a few participants noted that more attention is
needed to frame a strategic vision for what it may mean for climate change to be "mainstreamed" into
the NWP, and to set in motion management-level discussions to explore this topic in a meaningful way.
Some participants also observed that the strategies do not adequately focus on the fundamental
changes in water resource management and governance that will be needed at the local level due to
climate change and it does not adequately address EPA's role in supporting such transitions. For
example, a few participants noted that despite references in the strategies to integrated water resource
management (IWRM), the strategies do not explore the fundamental idea that climate readiness will
require new and unprecedented levels of coordinated governance and action at the local level,
addressing integrated management of water quantity and quality across watersheds and areas
supported by shared aquifers. These participants argued that even if the strategy does not clearly
articulate a vision for how these issues could be addressed; it needs to do more to frame and drive such
discussions.
Strategy Use, Influence,and Value
Despite resource constraints and competing priorities, the evaluators found that the NWP has made
significant progress in responding to climate change since 2008 in both voluntary and regulatory
program areas. Evidence suggests that the 2008 Strategy has not been a significant driver of activity
18 This shift is clearly articulated in the Memorandum from Michael Shapiro to National Water Program Office
Directors, "Reporting of Climate Change Progress and Development of FY13 Implementation Action Plan," October
17, 2012.
43
-------
EC
by EPA water programs, although it was valuable in providing a clear signal within and external to EPA
that climate change mitigation and adaptation have important implications for the NWP.
The evaluators found evidence that significant progress has been made since 2008 in understanding the
potential impacts of climate change to the NWP and in taking responsive actions to support mitigation
and adaptation efforts by diverse partners. Review of strategy progress reports and "highlights of
progress" reports, complemented by evaluation interviews, shows that substantial progress has been
made across each of the goals in the 2008 Strategy and in areas not explicitly addressed by the strategy.
Although evaluation participants commonly noted more rapid progress being made in voluntary
programs, the evaluators also found substantial evidence of progress in exploring how climate change
considerations fit into regulatory programs. This progress has been noteworthy to many evaluation
participants given the context of severe budget constraints and numerous competing priorities since
2008.
Most evaluation participants stated that while progress to date has been substantial, major work lies
ahead. Many participants noted the NWP is in the early stage of a long-term journey to respond to
climate change. Additional findings related to the status of efforts to "mainstream" climate change into
EPA water programs are presented later in this chapter.
While progress has been made in responding to climate change,
the evaluators found very little evidence that the 2008 Strategy
played a role in motivating or driving this progress. Across nearly
every OW division and program, evaluation participants
reported that the strategy (and its implementation) has not had
a substantive and discernible influence on program activities and
direction.
"The strategy does not in any
way shape or form drive my
decisions on what I do and don't
do."
- OW Manager
Evaluation participants offered several explanations to support their perception of the strategy having
little or no influence on OW activities and decisions. First, most participants stated that the lack of new
resources to support strategy implementation (and the erosion of staff and budget resources overall)
translated into offices and programs pursuing actions that, while related to the strategy on paper, they
already had planned to undertake in response to stakeholder needs and program plans. Second, many
participants noted that despite a strong general commitment to
address climate change by OW leadership, climate change has
not been integrated as a recurring and salient issue to be
addressed in routine discussions up and down the management
chain within OW and within EPA management. Third, some
participants mentioned that since the strategy resides outside of
mainstream EPA planning and accountability systems, it has
weaker influence on actions and decisions than if it was
incorporated into the Agency's routine planning. Fourth, many
participants indicated that the length of 2008 Strategy was a
barrier and that most OW staff had not read it. Several
participants noted that the document length could also pose
challenges for the success of the 2012 Strategy. Some evaluation
participants argued that a 10-page version of the 2012 Strategy is needed to make the strategy
accessible. A few participants suggested that the Executive Summary of the 2012 Strategy could be
adapted into a useful document for NWP managers, but that some additional information may need to
be incorporated. For example, one participant suggested including some "framing questions" to guide
"Climate change does not always
compete well against other
priorities vying for management
attention. Until we get good at
asking about how climate change
matters in every discussion, the
strategy won't have real influence
in EPA."
- OW Manager
44
-------
EC
on-going management-level strategic discussions about the mainstreaming of climate change into water
programs (voluntary and regulatory), challenges, and opportunities.
With the above caveat about making a shorter version of the strategy available, most evaluation
participants indicated that the 2012 Strategy is "good enough" and subsequent attention should focus
on how to use the strategy implementation process to yield the greatest value for the NWP. Most
participants stated that attention should focus on getting management-level input on and buy-in to a
shared vision for 2012 Strategy implementation—and to a set of implementation activities that support
this vision.
Many evaluation participants cautioned that based on the experience of implementing the 2008
Strategy, leadership attention is needed to ensure that the 2012 Strategy does not become a
bureaucratic exercise with little influence on real actions or decisions. While some division, office, and
branch-level managers are clearly bought in to addressing climate change, interviewees generally
indicated that the 2008 Strategy and its implementation process did not succeed in driving or inspiring
meaningful on-going discussions on program strategy within or between OW water programs. While this
is not surprising given resource constraints and competing priorities, lessons from the 2008 Strategy
implementation experience (discussed later in this chapter) suggest steps that could be taken to make
the 2012 Strategy more influential within the NWP. It is important to note, however, that even if the
strategy does not have a major influence on strategic direction and actions within OW, it can and has
been useful in other ways, such as a tool for engaging external partners.
The evaluators found that the 2008 Strategy has been used to
"The strategy enables us to say,
'look, this is an issue we are
wrestling with too. We are
committed to working with you
on this important issue.'"
- EPA Regional Office
Representative
engage state and tribal partners in discussions about the
implications of climate change for water resource management.
For example, some EPA Regional Office representatives
indicated that the strategy has supported their efforts to raise
climate change as an issue with state and tribal representatives
within their jurisdictions. Similarly, some state and tribal
partners reported that the strategy has prompted discussions
and reflection among state and tribal water program managers,
even among representatives from jurisdictions in which
discussion of climate change is constrained by politics. Several
evaluation participants stated that more attention should be given to engaging with States, tribes, local
governments, and other partners as part of 2012 Strategy implementation.
The evaluators found evidence that the 2008 Strategy has informed adaptation thinking and planning
among other EPA offices (outside of OW) and among other federal agencies. Several federal officials
outside of the NWP commented that the 2008 and 2012 Strategies have informed climate change
strategy development and action planning within their own organizations. Several officials also reported
that they have appreciated opportunities to have conversations with OW managers and staff involved in
the strategy work and (in some cases) to observe Climate Change Workgroup discussions.
As mentioned above, framing the 2012 Strategy as a longer-term "road map" to respond to climate
change addresses some key limitations of the 2008 Strategy. However, fresh thinking on implementation
and renewed leadership engagement are needed to make the 2012 Strategy relevant and useful within
the NWP. Several evaluation participants observed that the strategy could be used to create openings (a
reason and time for discussions) to discuss questions among EPA managers that may not otherwise get
explored as quickly or thoroughly, such as:
45
-------
EC
A What are the common challenges we are experiencing as we work to incorporate climate change
into our voluntary and regulatory programs?
A What lessons are we learning?
-^ Where do we need to prioritize activity, or to push harder and faster?
A Do we believe that the sum of our efforts are sufficient (given EPA's roles) to address the potential
scale of the challenges posed by climate change? If not, what should we do about that?
A What does climate change mean for how local actors will need to manage and make decisions about
integrated water resource management in the future? What will this mean for EPA's programs?
Management Support
Conversations with evaluation participants revealed strong high-level management support for the
overall NWP climate change strategies but uneven support at the division, office, and branch manager
level during implementation.
Many participants noted strong initial and consistent support for climate change integration from IO,
starting with former Assistant Administrator for Water Benjamin Grumbles, who commissioned the
National Water Program Climate Change Workgroup. Some interviewees noted that OWtook a lead role
at EPA in dealing with climate change during a less favorable political environment, and several
evaluation participants outside of OW noted that OW has been considered a model for how to integrate
climate change considerations into other EPA offices. One interviewee remarked that the early
leadership demonstrated by OW on climate change, at the least, set the precedent for OW staff to
openly discuss the issue. A significant number of participants also noted continued support for climate
change activities under current Deputy Assistant Administrator for Water Michael Shapiro and Assistant
Administrator for Water Nancy Stoner.
Interviewees noted the initial high-level participation in the OW Climate Change Workgroup and
subsequent delegation of workgroup participation and roles to staff. While some interviewees believed
this progression made sense as the workgroup transitioned from focusing on strategic direction and
messaging to specific actions and updates, many also identified this as a challenge to keeping
management informed and engaged with the importance of the work and to engaging them on strategic
issues related to climate change.
Despite the continued support of senior OW management for addressing climate change, participants
indicated uneven support at the division, office, and branch manager level. Evaluation participants
observed that while division, office, and branch managers in certain parts of OW demonstrated their
commitment during the 2008 Strategy development processes, that commitment has waned during
strategy implementation. Interviewees noted varying reasons for lack of support from managers,
although the primary reason cited is resource and staffing constraints. Interviewees stated that the flow
of resources to programs and projects is the most significant indicator of management prioritization,
and without additional resources to support OW's climate change strategy, management support has
been inconsistent. One factor connected to resources is the competition climate change considerations
face from a host of other priorities vying for limited organizational time and attention. Lack of
knowledge of climate change impacts to specific programs was cited as another reason for lack of
support at the division, office, and branch manager level; one interviewee noted that there is "not a lot
of management buy-in to implementation. Some management doesn't even know how climate is going
to affect their program, so they might disregard the work their staff is doing."
46
-------
EC
Interviewees cited OGWDW management as strong proponents of climate change within their office,
noting that OGWDW has diverted significant resources to climate-related activities. As one OGWDW
staff member noted "We are one of the few divisions, if not the only, that is actually doing that." A
significant number of interviewees also identified climate activities within OGWDW as an example of
successful climate change integration (further information is provided in a subsequent report section on
mainstreaming successes).
Despite current funding challenges and competition with core responsibilities, several interviewees
remarked that management support remains critical if OW is to continue to make progress addressing
climate change. Management support for integrating climate change considerations may vary from
office to office but could include activities such as advocating for and assigning staffing and resources for
climate change-related activities, encouraging staff to look at climate change considerations in their
current scope of work, and integrating climate change as a standing discussion topic during regular OW
management and budget meetings.
Staffing and Resources
Overall, the evaluators found limited resources and staff for both development of the 2008 and 2012
Strategies and implementation of climate change activities. However, implementation of climate
change programs and initiatives can occur with existing resources by doing current work in a slightly
different manner.
For development of the 2008 and 2012 Strategies, it was primarily staff that worked on the day-to-day
development and kept their managers informed on progress. Interviewees noted that slightly different
staffing methods were used in the creation of the 2008 and 2012 Strategies. While both were led by
assigned IO staff, staff in other OW offices participated more during the 2012 Strategy development
process than the 2008 Strategy development process. The 2008 Strategy was written in large part by IO
staff with review and comment by other OW office staff, while the 2012 Strategy was a collaborative
effort from the outset. Staff in different parts of OW collaborated on developing and reviewing the
strategies through the workgroup. This work was considered somewhat of an "add-on" to the regular
duties of these staff members. Beyond limited staff time, there were no additional resources allocated
to the development of the strategies.
The shift to involving more staff across OW in the development of the 2012 is generally regarded as a
positive development that has potential to expand "ownership" of the 2012 Strategy and to build a
broader group of climate change champions across the NWP. In practice, this appears to be somewhat
true among staff-level participants involved in developing the strategy, although the evaluation team did
not find evidence indicating that this involvement has enhanced management support for the 2012
Strategy at the division and office levels. Some interviewees observed that lack of clarity around how
their input was incorporated into the 2012 Strategy may be a factor in undermining the ability of
participation in strategy development to translate into perceived ownership of the strategy.
The few staff members working exclusively on climate change are dedicated and put forth significant
effort. Interviewees working on climate-related activities noted their personal commitment to the
subject as a driver for beginning and continuing this work, even in a resource-limited era. Those
individuals who worked on the development of the 2008 and 2012 Strategies or implementation of
climate change activities noted that they often do so outside of their primary responsibilities.
Most interviewees noted that OW staff and management are primarily focused on core responsibilities,
including court mandated deadlines and other immediate concerns which restrict the ability to
47
-------
EC
undertake the type of holistic, long-term strategic planning required to integrate climate change
considerations into most programs. Lack of resources was cited by interviewees as a factor in limiting
the number of key actions listed in the 2008 Strategy. Initially the strategy contained 60-70 key actions
but once it was clear that there would be no additional resources to support the activities, the number
was reduced to 44.
Some interviewees compared the challenge in integrating climate change considerations to other cross-
cutting issues such as children's health and environmental justice. These issues connect with multiple
aspects of different programs across EPA but are not built into programs' critical missions. Given the
nature of integrating climate change considerations, limited budget and staffing resources, and the
federal budget situation overall, interviewees conceded that climate change programs will likely be
among the first to suffer from budget reductions. Interviewees noted that the two climate-focused
initiatives considered as compelling examples successful integration within OW are projected to see
budget reductions in the near future.
Some interviewees, both inside and outside of OW, held a strong opinion that there are yet unrealized
opportunities to integrate climate change considerations into existing programs without additional
resources. For example, interviewees noted that OW could consider the effects of changing water levels
or temperatures on permit conditions using scenarios and models that incorporate updated information
from current climate models. Interviewees noted that further opportunities exist to more strategically
integrate climate change considerations into OW programs and offices without additional resources, but
acknowledged that this would requires a significant shift in thinking around OW's traditional operations.
Other evaluation participants countered that integration of
climate change considerations into existing programs and
activities can sometimes require significant investments in staff
time, and in some cases, specialized technical assistance or
extramural resources for consultant analyses.
Communication and Outreach
The evaluators found that more attention to communications
and outreach for the 2012 Strategy would be beneficial for
implementation success. Several communications and outreach
mechanisms, such as the National Water Program Climate
Change Workgroup, the "Highlights of Progress" documents,
and the NWP climate change strategy website, are important
pieces of an overall communications and outreach strategy.
- EPA Official
In addition to the original memo from former Assistant
"For me, one of the lessons
learned is that we need to do a
better job of helping people
understand that although there
are some things that do require
more resources, there are an
awful lot of things that we could
be doing with existing resources
but doing them in a slightly
different way that would allow for
them to account for climate
change."
Administrator for Water Benjamin Grumbles commissioning the National Water Program Climate
Change Workgroup and the 2008 Strategy, IO released several memos associated with requests for
review of components of the strategy during the drafting process. Most interviewees' recollections of
communication from the front office identified communication around drafts of the Strategies as
opposed to implementation. In addition, IO communicated with points of contact for specific key actions
in the 2008 Strategy for reporting and measurement purposes.
Interviewees suggested that further communication around the 2012 Strategy would be useful to
ensure successful implementation. They also suggested that integrating climate change into the NWP
would require communications efforts around knowledge-building as well as communicating about the
strategy.
48
-------
EC
Examples of climate change communication activities from individual offices include an all-hands
meeting in OWOW and webinars offered through OWM and OST. The all-hands meeting was led by
OWOW staff with participation by front office staff connected to the 2008 Strategy. The main purpose
was to describe OWOW's specific commitments in the strategy. Most of the communication around the
2008 and 2012 Strategies is conducted with the climate change leads of individual OW offices through
the workgroup.
The workgroup acts as the main communication mechanism for climate change activities. The
workgroup is a venue for offices and regions to share information on activities underway at HQ and in
the regions. Regional interviewees observed that the workgroup provides an important communications
mechanism for work that is going on in other regions and headquarters that they would not know of
otherwise.
Some interviewees noted that the online presence from the
.. . . . .. . . . . . .. ... ... .. The NWP Climate Change
climate change website is a great tool and that it will continue
,.£...,. . , . , , >_, . Workgroup meets monthly and
to serve as such if it is kept current. Interviewees observed that
.. . .. . , , . . .. receives updates from regions
the website is focused more on external communications as
, . , .. . ,,. . .. . .,-..,„ . . and guest presentations. Since
opposed to educating staff internally about OWs strategy or
,. . , rr . there is no formal climate division
climate change efforts.
within EPA, this is how
Both EPA and external interviewees noted that the 2008 and communication occurs."
2012 Strategy and the "Highlights of Progress" documents
showed a commitment by EPA, and OW in particular, to " OW Staff Member
address climate change. The strategies allowed the regions to
engage on tackling climate change issues more directly and were a way for them to have conversations
about these issues with states and partners. Regional staff also noted that the strategies are also an
important signal of support for the various topics contained therein.
Training and Capacity Building
The evaluators overall assessment is that additional training for EPA staff on impacts of climate
change, particularly impacts on areas specific to staff members' programmatic duties, would be
extremely helpful.
After releasing the 2008 Strategy, OW created training slides with an overview of the water cycle, a
definition of climate change, how climate change fits into the water cycle, and the impact of climate
change on water resources. The slides were posted on a previous version of the climate change website
and used as part of the Water Quality Standards Academy. There were also various webinars on climate
change produced by individual OW offices (OST, OWM), the OAR's State and Local Climate and Energy
program, and the Office of Policy. Because it was recently released there has not yet been significant
training associated with the 2012 Strategy; several interviewees noted the new strategy provides an
opportunity for OW to engage its staff with training and education.
Interviewees perceived a greater degree of climate change knowledge among staff members that
worked on the 2012 Strategy and a lesser degree of knowledge among staff who have yet to engage
with climate change as part of their work. OW hosts periodic brownbag sessions related to climate
change, but these are only communicated to the people on the climate change distribution list and not
office-wide. Some interviewees noted that the organizational nature of OW - specifically, the "silo"
mentality of vertical barriers between offices and programs - impacts how individuals think about
climate change and that providing a larger watershed perspective could be useful in training and
implementation of climate-related initiatives. These interviewees noted that regions could help OW
49
-------
EC
learn how to solve problems from a watershed perspective. Some interviewees noted that additional
training could focus on: 1) educating a larger number of OW staff, including managers, on climate
change science at a general level, and 2) educating OW staff and managers on how climate change could
impact their programs and day-to-day responsibilities.
Interviewees noted that for the most part, OW staff have a basic understanding of climate change and
agree on its importance, but there is a lack of understanding on how to integrate climate change
considerations into core programs and day-to-day activities. EPA has begun to address this with some
training, including a January 2013 webinar in conjunction with the US Global Change Research Program
(UWGCRP) to introduce EPA staff to the national climate assessment scenarios and sea level rise
scenarios, which addressed how to use the information in decision-making. There are other voluntary
opportunities to receive additional training, with webinars presented by OW OST and IO as well as the
OAR's state and local climate webinar series. It remains unclear, however, how many OW staff
participate in these opportunities. Some regional interviewees noted effective strategies such as
leveraging webinars to provide basic instruction and knowledge, followed by discussion sessions about
how the information presented connects with relevant work in the region. Interviewees noted that
managers also need to be aware of climate change effects, and asking staff for this information in the
context of regular responsibilities is an important step in normalizing climate into daily activities.
EXHIBIT 4-3. IMPLEMENTATION STATUS CHART
Status of Implementing 44 Key Actions
2009
Measurement
The evaluators found that OW divisions and
offices outside of IO found little value in
2008 Strategy measurement activities and
the evaluators did not find evidence that
measurement informed decisions or
motivated action.
Measurement of progress under the 2008
Strategy focused primarily on qualitative
descriptions of activities and outputs
undertaken by EPA relevant to the five
specific goals and forty-four key actions
articulated in the strategy. In some cases,
quantitative measures were used to
complement descriptions of outputs. For
example, to assess the influence of the 2008
Strategy, IO measured the number of
subscribers to the EPA Climate Change and
Water News e-newsletter.19 Similarly, the
Climate Ready Estuaries program measured
the number of times relevant reports were
downloaded from the EPA website.20 In a few instances, EPA programs included outcome-focused
measures previously developed to address program-related needs. For example, EPA's WaterSense
ICompleted BUnderway • To Be Initiated D New Activities
Source: 2009 Progress Report, page 6.
U.S. EPA, Implementing the National Water Program Strategy: Response to Climate Change - Progress Report for
2009, p. 38, http://water.epa.gov/scitech/climatechange/upload/2009-lmplementing-the-National-Water-
Program-Strategy-Response-to-Climate-Change-Progress-Report.pdf.
20 Ibid, p. 26.
50
-------
EC
program included measures of total water and cost savings from the use of WaterSense labeled
products.21
Progress under the 2008 Strategy was also measured in aggregate at the key action level, as summarized
in the 2008 and 2009 Progress Reports. IO coded the progress under each key action as completed,
underway, to be initiated, or new activities, and counts were generated summing the number of key
actions in each status category (see Exhibit 4-3). Although these aggregated metrics were not prepared
after 2009, the 2010-2011 Key Action Update includes a table categorizing the status of specific Key
Actions as completed, ongoing, initiated, proposed, or deleted.22
The focus on measuring outputs under the 2008 Strategy was expected by most evaluation participants,
given the strategy's focus on near-term actions and the scope of data available to support measurement
related to the strategy.
Most evaluation participants indicated that the measurement
approach under the 2008 Strategy was not structured to support
or inform learning or decisions at the office, division, or program
levels. Numerous evaluation participants observed that this
accountability focus to measurement is challenging—and
potentially counter-productive—in the combined context of:
A Uncertain and declining budget and staffing resources;
A Lack of statutory drivers or court-orders requiring the National Water Program to address and
measure progress on its response to climate change;23 and
A The Strategy and measurement activities residing outside of the Agency's mainstream strategic
planning processes and measurement systems.
"Intentions are good, but the
measurement approach creates
busy work - it doesn't enhance
the value of our work."
- OW Staff Member
Several evaluation participants observed that the 2008 and 2012
Strategies face similar challenges for measurement and
accountability to those experienced by other cross-cutting EPA
initiatives such as environmental justice and children's health.
Many interviewees talked about the importance of transparency
and demonstrating tangible commitment and progress to key
partners and the public on climate change. However, most
perceived that the 2008 Strategy (and the early 2012 Strategy)
measurement approach felt more like "bean counting" as
implemented. They generally argued that other approaches are
better suited to support transparency and demonstrate
commitment and progress externally given the current context
bulleted above. For example, as discussed below in the reporting
section, several division and regional Office participants
indicated that the qualitative "measurement" of "bright spots"
"There has been a very vibrant
debate between OWIO and
program offices over the
difference between - and value of-
aspirational goals versus
measurable metrics [for cross-
cutting issues like climate
change], especially in extremely
resource-constrained times. That
debate gets played out in this
con text frequen tly."
- OW Manager
Ibid, p. 9.
22 This discussion of measurement approach is specific to the 2008 Strategy; discussion of the measurement
approach that IO employed to date for the 2012 Strategy is contained in Chapter 5.
23 It is important to note that requirements supporting federal agency adaptation planning are strengthening. See
sidebar on page 2 of this report.
51
-------
EC
"/ think there are a lot of people
throughout the offices that are
not comfortable with that
approach. But others want to
know: Where are we?"
- OW Staff Member
captured in the Highlights of Progress reports was a more
useful example of measurement to demonstrate commitment
and progress to partners. Essentially, the participants argued
for developing illustrative success stories in lieu of
comprehensive measurement.
Several evaluation participants outlined how even a subtle
accountability-focused measurement approach in the current
context can frame the measurement process as "a least common denominator" obligation rather than
one that inspires action within and outside of the Agency. Several OW division and office
representatives discussed perceived fears that failure to meet commitments in the strategy could have a
negative impact on future program budget allocations or personnel performance reviews. Even though
the practical risk of this occurring was downplayed, participants stated that "no one wants to look bad"
in a measurement system, particularly given external budget and staffing circumstances and competing
priorities that are outside their sphere of control.
In the context of severe budget pressures and weak strategy
drivers, the output and accountability-focused measurement
approach tacitly encourages divisions and offices to be very
conservative in the commitments they propose under the
strategy. The evaluators also heard reports of some office
representatives spending significant time shepherding the
wording of their office's commitments (and the implied
measures of progress they would set) through the strategy
development and action planning processes. These factors
appear to have fueled internal perceptions of the strategy as
an obligation instead of as a tool for achieving progress. The
framing of the 2012 Strategy somewhat mitigates against this by moving commitments from the
strategy to an annual action planning process; however, the initial measurement approach under the
2012 Strategy does not appear to have "reset" broadly held perceptions of "bean counting" in the
context of budget reductions.
"/ know that that is hard to do but
pushing [measurement] in the
direction of outcomes would
really begin to make it real in
terms of intellectual commitment
as well as bureaucratic
commitment."
- OW Manager
Several evaluation participants recognized the challenge of
designing a robust and meaningful measurement system—one
that informs and drives progress in responding to climate
change—given the current context. While there are no easy
answers, the evaluation team summarizes some ideas in the
recommendations section, drawing on ideas from evaluation
participants.
"Short term, qualitative
[measurement and] reporting is
the best that we are able to
do...unless there's a huge influx of
funds and regulatory capacity...."
- OW Staff Member
As implied in the previous discussion, measurement activities under the 2008 Strategy (and to date
under the 2012 Strategy) were largely perceived to be for the benefit of IO. The evaluators did not find
evidence that divisions, offices, or programs in the National Water Program outside of IO have used
measurement information collected under the strategy to inform decisions or motivate action. Several
division and office representatives also raised questions about how useful the measurement
information supporting the 2008 Strategy actually was in practice to inform IO decisions and actions.
The qualitative summaries included in the brochure-like 2008-2009 and 2010-2011 Highlights of
Progress fact sheets and report are a notable exception. Several regional office participants talked about
how these materials and the information in them have been useful for informing state and other
partners about EPA's commitment and activities to respond to climate change. Evaluation participants
52
-------
EC
"/ think everyone is in a really
tough position now because we
don't want to be negative. We
want to step up and show many
of the good things we're doing—
but the way that some of the
strategic actions are written, we
can't report to them. This is
problematic."
- OW Staff Member
outside of EPA did not report much familiarity with the measurement approach and information for the
2008 Strategy, indicating that the strategy itself has been the main information they have interacted
with and used to inspire and motivate action.
A few evaluation participants acknowledged the potential value
of measures that address behavior change among partners and
target audiences or that address outcomes relevant to the
strategy. Such measures would likely be important if
measurement under the strategy were to be useful for
informing program-level decision making. However, these
participants were quick to raise cautions about the time and
resources needed to collect information to support such
measures, if the data were accessible at all. Paperwork
Reduction Act requirements and the potential need for
Information Collection Requests (ICRs) were also identified as
challenges to expanding measurement approaches to address
behavior change and outcomes. One participant also raised the
concern that it would likely be difficult to determine the contribution or attribution of progress to EPA
programs and activities, and that outcome measures might show that climate mitigation and adaptation
actions, while valuable and important, are not of a scale commensurate to address evolving impacts and
challenges. The evaluators acknowledge these as legitimate concerns, and they are addressed in
Chapter 5 and in discussion of options for measurement approaches moving forward.
Overall, based on the feedback received on the 2008 Strategy, the evaluators found that IO will need to
lead important internal discussions with offices and regions to cultivate a clear and aligned vision for
how future measurement activities could best be structured to achieve progress in realizing the vision
and goals of the 2012 Strategy. Clarifying the use and value of measurement information will be vital to
this vision.
Reporting
The evaluators found a broad range of staff time devoted to reporting under the 2008 Strategy. In
addition, most reports prepared under the 2008 Strategy were not designed to meet the needs of
audiences outside of IO and were not likely to be read by many people.
"We have the capacity to take a
fairly simple exercise and make it
so complicated."
- OW Manager
Some division and office representatives reported minor
amounts of time (e.g., a few hours) required to support annual
reporting under the 2008 Strategy whereas others reported a
substantial time commitment (e.g., multiple days).
Interviewees suggested several reasons for this variation.
First, the number of key actions assigned to program offices
varied. Second, EPA regional offices were given a fair degree
of discretion in what they chose to report on under the strategy. Third, the number of program staff that
reporting leads needed to coordinate with to collect information varied substantially across offices and
divisions. Fourth, the level of detail of information collected and provided by reporting leads varied. One
reporting lead suggested that it would have been helpful and potentially more efficient to have an
example of the appropriate level of detail expected in the periodic requests for progress reporting under
the strategy.
In general, the reports produced under the 2008 Strategy support the accountability and output-
focused approach taken with the measurement system discussed above. As a result, most evaluation
53
-------
EC
participants indicated that they see IO as the primary audience for the reports prepared under the
strategy. Given the length of most reports prepared under the strategy, many participants questioned
whether IO is deriving sufficient value from the reports to warrant the effort required to develop them.
A few evaluation participants noted that it is not necessarily clear who the intended audience is for
reports produced under the strategy—IO, managers and staff throughout EPA, state and tribal partners,
industry and NGO representatives, or the interested public writ large. They suggested that more work is
needed to clarify the priority audiences, their specific needs, and then determine what types of reports
or other materials are needed to support these information flows.
"Reporting information seems like
it goes into a black hole. We don't
get more or less resources
because we are doing great or
lousy. There are no rewards or
punishment for how good or bad
you do. There aren't follow-up
discussions."
- OW Staff Member
As mentioned in the Measurement Section, several regional
office representatives noted that the Highlights of Progress
documents—particularly the shorter versions with photos and
graphics—were useful for sharing with broader audiences.
Other evaluation participants reflected that they appreciated
the opportunity to report on climate change-related activities
and accomplishments in which their programs were engaged
that did not necessarily fit under specific key actions. They
indicated that standard progress reports were not necessarily
conducive "for reporting what their office or division was
excited about in a way that that others would find interesting
and engaging."
Evaluation participants outside of OW pointed to the Climate Ready Estuaries Program 2012 Progress
Report as a useful example of a progress report accessible to external partner audiences.24 They noted
the use of visuals (e.g., graphics and photos) and engaging narrative and also noted that the report
discusses key lessons from program activities, and not just descriptions of activities. They suggested that
this type of approach can be powerful for recognizing and inspiring internal action and for engaging
external partners, demonstrating progress, and talking openly about issues and challenges.
Evaluation and Learning
The evolution from the 2008 Strategy to the 2012 Strategy reflects considerable organizational
learning by OW.
As discussed under the Strategy Effectiveness section, the evaluators found evidence that the
development and implementation experience with the 2008 Strategy provided a context for learning
that strengthened the vision and goals framework developed for the 2012 Strategy. Many evaluation
participants observed that the 2008 Strategy was a good first step and signaled that among EPA's
National Program Offices, the National Water Program was ahead of the curve on thinking about climate
change and adaptation. The 2012 Strategy is generally seen as an important evolution in the National
Water Program's thinking around climate change.
Evaluation participants indicated that more work is needed to accelerate learning and improvement in
how the strategy—as an artifact and as a process—can be used to drive more rapid mainstreaming of
climate change within diverse voluntary and regulatory programs across the National Water Program.
Many evaluation participants appreciate that IO has sought to evaluate implementation of the 2008
Strategy and that the focus is on identifying lessons to inform future implementation activities. A few
participants noted that the timing for the evaluation might have been more appropriate if conducted
24 Report available at: http://water.epa.gov/type/oceb/cre/upload/CRE 2012Report 122612a.pdf
54
-------
lEc
before the 2012 Strategy was released. They encouraged IO to be open to exploring lessons raised in the
evaluation, even if these lessons signal the need for changes to the initial implementation approach
used for the 2012 Strategy.
Alignment with Other Federal Agency Climate Plans
OW created one of the first climate change strategies in the Agency and the evaluators found that OW
is considered a model for other emerging efforts within the federal government.25
OW is participating in a broader inter-agency task force to create climate plans for federal agencies
affected by climate change. One interviewee familiar with the process noted that the OW plan and the
inter-agency task force plan show remarkable alignment given differences in scale and program
priorities.
Ongoing parallel climate change planning processes can provide support for the 2012 Strategy. These
other plans reinforce the importance of integrating climate change considerations into the NWP and can
be used to help staff and managers understand its priority status within the Agency. OW can also learn
from other federal agencies' implementation processes, once they have reached this stage.
Some interviewees also noted that to truly integrate climate change considerations into the NWP, these
efforts needed to not just be laid out in a separate strategy, but be integrated into the NWP's strategic
plan.
External Partnerships and Engagement
Overall, the evaluators found that OW is viewed as an effective partner on climate change activities,
and in some cases OW's efforts have spurred other organizations' ability to work on climate change.
The 2008 and 2012 Strategies contain actions internal to OW but also ways to support external partners
including states, tribes, and local governments. These partners have been engaged both by the regions
who work directly with them and by certain programs within OW that work with them on a more regular
basis, such as those that work with estuaries and utilities.
Regions were often identified as having conducted significant outreach with states and other partners
on the 2008 Strategy and/or climate change in general. Media organization partners that participated in
the State-Tribal Climate Change Council (STC3) also conducted outreach on the 2008 Strategy with their
members. In addition to states, interviewees identified outreach with groups like the NE Regional Ocean
Council, Gulf of Maine Council, and regional meetings of basin commissions and state water directors.
Region 1 states have independently pressed for climate action, especially with high profile events such
as Hurricane Sandy. Conversely, states in other regions cannot discuss climate change due to the local
political environment, but they are able to engage in planning around events and initiatives related to
drought, wildfires, and emergency preparedness.
Interviewees identified OW as an important partner that has coordinated well on climate change
activities. OW is seen as a model because of its early efforts to strategize incorporation of climate
change considerations.
Regional interviewees noted that OW's climate change strategy provides a tool to educate states and
other partners and has given credence to pursue implementation of climate change mitigation and
adaptation activities.
25 Alignment across the OW strategies, EPA Strategic Plan, and EPA Adaptation plan is discussed in more detail in
the previous chapter.
55
-------
lEc
While interviewees made it clear that OW and the regional offices did conduct some outreach related to
the 2008 Strategy, some interviewees noted that OW could have a better dialogue with states on the
strategy. Even regions that have conducted some outreach noted that this could be improved; one
mentioned that he was not sure of the lead climate change person to reach out to in each state.
The STC3 initially met on regular basis but meeting frequency has declined. Many interviewees
identified this as a place where OW could foster further engagement. Interviewees both within EPA and
externally, noted that the STC3 is useful but does not have a clearly defined role at this point. They
suggested that EPA could use this group in a more effective manner, which it might mean reconstituting
it with broader membership for greater engagement.
EPA also has an opportunity to engage with inter-agency work and workgroups. Some regions have done
this on a regional basis and collaborate with other federal agencies in their interactions with states. This
could be expanded to other regions. Some interviewees noted that this would be an effective way to
leverage resources available for adaptation activities.
Interviewees noted that the strategy references EPA's role as creating tools and providing information
to fill gaps and enable states and other stakeholders to apply the tools to take action. Interviewees
noted that when these tools are produced or information becomes available, EPA can also utilize these
opportunities to engage with external partners and stakeholders on concrete climate change adaptation
and mitigation solutions. One interviewee said that this "reminds everyone that [the strategy is] a real
living document that EPA is taking seriously and provokes people to go back and think about the
strategy in context."
Many interviewees noted that having the strategy as an agenda item for meetings occasionally was
another way to keep it visible and on the minds of both EPA staff and external partners. This could be
practiced with groups such as the STC3, regional internal and external meetings, and even within OW.
What goals and strategic actions in the 2012 Strategy should EPA headquarters
and regional programs prioritize?
A final piece of the retrospective evaluation was to gather interviewee perspectives on prioritization of
strategic actions in the 2012 Strategy. This section presents findings on prioritization for OW's
consideration in the context of OW's current climate related prioritization activities.
The evaluators encountered a broad range of perspectives on prioritization of the strategic actions
described in the 2012 Strategy. Interviewees did not advocate for prioritization of particular strategic
actions; instead, they provided a suite of prioritization schemes for OW consideration.
Offices that work directly with partner organizations or associations, such as utilities, tend to prioritize
those activities developed jointly with partners. These priorities reflect what the partners want and will
use, and in the absence of regulations these offices will work with target audiences to see what types of
voluntary efforts will gain the most traction. One interviewee noted that for actions involving external
partners, priority should be placed on educating partners on what impacts will affect them, how to
respond to those impacts, and how to perform vulnerability assessments.
Some respondents noted that regulations are already implemented under a wide variety of climactic
conditions, and for now that may sufficient. But as the variant increases and parameters change, there
may be a need for regulatory change. Other respondents suggested prioritizing climate change guidance
in the use of EPA categorical grants, performance partnership grants (PPGs) and CW and DW SRF
56
-------
EC
programs. Respondents also suggested prioritizing strategies connected to existing EPA authorities such
as TMDLs because these will be most effective and will have definite environmental protection
outcomes.
Evaluation participants suggested several concepts to consider when designing a prioritization
framework:
A Use scientific research to inform the prioritization strategy. For example, a recent OW - NOAA
meeting may lead to some joint action on ocean acidification based on recent scientific findings.
-^ It is more effective to work harder to maximize implementation effectiveness from ongoing
successful programs such as CRWU and CRE rather than ask staff to drop everything and work on a
different strategy.
A Prioritization should be placed on decisions and actions that have longer term impacts, such as the
SRF program and other water infrastructure investments.
A Prioritize strategies/actions that benefit the entire agency; it is easier to get buy-in if offices realize
they do not have to do everything on their own.
A Prioritize based on the scale of impact (high, medium, or low) and add a filter for likelihood of
occurrence; this will avoid the need to refer to availability of resources.
-^ Include an explicit statement that prioritization will not result in diversion of resources from other
areas; instead, as resources become available they will be targeted toward priorities.
A Before prioritization can occur, EPA must make hydrological and climate information available at the
watershed/aquifer scale to the IWRM councils.
A EPA should use a shotgun approach: Try to launch as many things as possible and see which ones
gain traction.
A Prioritize actions in coastal programs, as they will feel the most immediate significant impacts.
IO is currently developing prioritization criteria in order to effectively allocate resources toward climate
related activities. Key criteria under consideration include: Urgency, risk, geographic scale,
programmatic scale, and probability of occurrence. OW could use these criteria to prioritize activities
identified in the 2013 Implementation Plan.
57
-------
lEc
CHAPTER 5 | PERFORMANCE MEASUREMENT FOR THE 2012 STRATEGY
In this chapter, we describe findings and lessons learned related to developing performance measures
for the National Water Program 2012 Strategy: Response to Climate Change. Specifically, we address
lessons learned related to the following topics and questions:
Measurement Approach:
-^ What is the measurement approach that can be used to measure adaptation progress in five areas:
infrastructure, watersheds and wetlands, ocean and coastal waters, water quality, and working with
tribes?
A What specific elements need to be applied to the phased approach to tracking progress outlined in
the 2012 Strategy, to make it a robust measurement framework?
A What, if, any, revisions should EPA make to its baseline data collection process to ensure that data
collected are meaningful and objective?
Informing Agency-Wide Plans:
A How lessons learned from this projects evaluation of OW's approach inform measuring progress in
the EPA-wide Adaptation Plan, and inform development of the next Agency 4-year Strategic Plan?
FINDINGS AND LESSONS
This section begins by describing how OW has measured progress to date on the 2012 Strategy, and the
advantages and disadvantages of this approach. The evaluators then answer each of the evaluation
questions in turn, with particular emphasis on the first question, where we lay out two alternative
measurement approaches that could be used to track adaptation progress and variations on those
approaches. We conclude by discussing how lessons learned from this evaluation can inform Agency-
wide efforts to track progress on climate change adaptation.
OW's Approach to date in Measuring Progress on the 2012 Strategy
As part of the 2012 Strategy, the OW said it would "adopt a phased approach to track programmatic
progress towards Strategic Actions" and develop outcome measures.26 The 2012 Strategy addresses 19
goals across 5 vision areas and includes 53 strategic actions that will help achieve the goals (see Exhibit
5-1).27
OW collected baseline information on progress to date on strategic actions related to climate change
adaptation. The IO asked HQ offices and regions submitting baseline data to assess which of the seven
phases best described progress toward implementing the strategic action (although not all HQ offices
and regions provided this information).
26 2012 Strategy, Goal 18 (Tracking Progress and Measuring Outcomes), Strategic Action 52
27 The 2012 Strategy also includes goals and strategic actions for each climate region; these are not included in this
report.
58
-------
lEc
The IO used the baseline information on strategic actions to develop its 2012 Highlights of Progress
report/8 which includes an assessment of progress at the goal (not strategic action level). To prepare
the report, IO examined the baseline data and assessments of progress at the strategic action level, and
then followed up with HQ offices and regions to resolve questions. The IO then assigned one of the
seven phases of adaptive management to each goal and an associated score of "1" through "7." For
example, goals deemed to be at the phase of "initiation," received a score of "1," goals at the phase of
"assessment" received a score of "2," and so on through the seven phases of adaptive management, up
to a possible score of "7" for each goal. The IO then summed the scores assigned across the 19 goals.
This process resulted in a 2012 baseline assessment of a score of 42 out of a total possible score of 133
(19 goals with the potential to achieve a score of "7" for each strategic action equates to a total
potential score of 133).
This approach of developing a single numeric score for progress on all goals is appealing in its apparent
simplicity. However, in the evaluation team's view, this approach requires a number of implicit
assumptions that may not be merited and in some cases are not what the IO intended:
A Assigning scores at the goal level, rather than the strategic action level, weights the importance of
the strategic actions relative to achieving the goal, although the criteria for weighting are not
explicit.
-^ Assigning a numeric score of "1" to "7" for the seven steps of adaptive management effectively
assumes that each step in adaptive management is of equal importance.
A Assigning a numeric score of "1" to "7" for the seven steps of adaptive management implies that
goal can be at only one "step" in the process, whereas the evaluation team's research indicates that
implementation of strategic actions is not completely linear.
A Summing the scores for all goals together assumes that each goal is equally important and thus
contributed equally to the score.
A Presenting the assessed total score in the context of the total possible score assumes that all seven
phases of adaptive management are relevant to every goal (i.e., it would be feasible for each goal to
reach the phase of monitoring and adaptive management, and thus receive a score of "7.")
Thus, in the evaluation team's view, the simple result of this process (a single score compared to a
potential score) obscures the many subjective decisions and implicit weighting factors used to develop
the score. In the next section, we propose two alternative approaches to track progress on adaptive
management which the evaluation team believes are more transparent and rigorous.
28 See http://water.epa.gov/scitech/climatechange/2012-National-Water-Program-Strategy.cfm for a copy of the
report: 2012 Highlights of Progress: Responses to Climate Change by the EPA National Water Program.
59
-------
EC
EXHIBIT 5-1. OW VISION AREAS, GOALS, STRATEGIC ACTIONS, AND EPA OFFICES RELATED TO CLIMATE CHANGE ADAPTATION
GOALS AND STRATEGIC ACTIONS
LEAD OFFICE (ft PARTNERS)
VISION AREA 1: INFRASTRUCTURE: In the face of a changing climate, resilient and adaptable drinking water, wastewater, and stormwater utilities (i.e.,
the water utility sector) ensure clean and safe water to protect the nation's public health and environment by making smart investment decisions to
improve the sustainability of their infrastructure and operations and the communities they serve, while reducing greenhouse gas emissions through greater
energy efficiency.
Goal 1 :
Build the body of information
and tools needed to incorpo-
rate climate change into
planning and decision making.
Goal 2:
Support Integrated Water
Resources Management to
sustainably manage water
resources.
SA1: Improve access to vetted climate and hydrological science, modeling, and assessment
tools through the Climate Ready Water Utilities program.
SA2: Assist wastewater and water utilities to reduce greenhouse gas emissions and increase
long-term sustainability with a combination of energy efficiency, co-generation, and
increased use of renewable energy resources
SAB: Work with the states and public water systems, particularly small water systems, to
identify and plan for climate change challenges to drinking water safety and to assist in
meeting health based drinking water standards.
SA4: Promote sustainable design approaches to provide for the long-term sustainability of
infrastructure and operations.
SA5: Understand and promote through technical assistance the use of water supply
management strategies.
SA6: Evaluate and provide technical assistance on the use of water demand management
strategies.
SA7: Increase cross-sector knowledge of water supply climate challenges and develop
watershed specific information to inform decision making.
OGWDW (OWM)
OWM (OGWDW)
OGWDW
OWM (OGWDW)
OWM (OGWDW)
OWM (OGWDW)
OW IO (All OW Offices)
VISION AREA 2: WATERSHEDS Et WETLANDS: Watersheds are protected, maintained, and restored to provide climate resilience and to preserve the
ecological, social, and economic benefits they provide; and the nation's wetlands are maintained and improved using integrated approaches that recognize
their inherent value as well as their role in reducing the impacts of climate change.
Goal 3:
Identify, protect, and
maintain a network of healthy
watersheds and supportive
habitat corridor networks.
SA8: Develop a national framework and support efforts to protect remaining healthy
watersheds and aquatic ecosystems.
SA9: Collaborate with partners on terrestrial ecosystems and hydrology so that effects on
water quality and aquatic ecosystems are considered.
SA10: Integrate protection of healthy watersheds throughout the NWP core programs.
SA1 1 : Increase public awareness of the role and importance of healthy watersheds in
reducing the impacts of climate change.
OWOW
OWOW
OWOW
OWOW
60
-------
EC
GOALS AND STRATEGIC ACTIONS
Goal 4:
Incorporate climate resilience
into watershed restoration
and floodplain management.
Goal 5:
Watershed protection
practices incorporate Source
Water Protection to protect
drinking water supplies.
Goal 6:
EPA incorporates climate
change considerations into its
wetlands programs, including
the CWA 404 program, as
appropriate.
Goal 7:
Improve baseline information
on wetland extent, condition,
and performance to inform
long term planning and
priority setting that takes into
account the potential added
benefits for climate change
adaptation and carbon
sequestration.
SA1 2: Consider a means of accounting for climate change in EPA funded and other
watershed restoration projects.
SA13: Work with federal, state, interstate, tribal, and local partners to protect and
restore the natural resources and functions of riverine and coastal floodplains as a means
of building resiliency and protecting water quality.
SA14: Encourage states to update their source water delineations, assessments or
protection plans to address anticipated climate change impacts.
SA15: Continue to support collaborative efforts to increase state and local awareness of
source water protection needs and opportunities, and encourage inclusion of source water
protection areas in local climate change adaptation initiatives.
SA16: Consider the effects of climate change, as appropriate, when making significant
degradation determinations in the CWA Section 404 wetlands permitting and enforcement
program.
SA17: Evaluate, in conjunction with the U.S. Army Corps of Engineers, how wetland and
stream compensation projects could be selected, designed, and sited to aid in reducing the
effects of climate change.
SA18: Expand wetland mapping by supporting wetland mapping coalitions and training on
use of the new federal Wetland Mapping Standard.
SA19: Produce a statistically valid ecological condition assessment of the nation's
wetlands.
SA20: Work with partners and stakeholders to develop information and tools to support
long term planning and priority setting for wetland restoration projects.
LEAD OFFICE (ft PARTNERS)
owow
owow
OGWDW
OGWDW
OWOW
OWOW
OWOW
OWOW
owow
VISION AREA 3: COASTAL AND OCEAN WATERS: Adverse effects of climate change along with collective stressors and unintended adverse consequences of
responses to climate change have been successfully prevented or reduced in the ocean and coastal environment. Federal, tribal, state and local agencies,
organizations, and institutions are working cooperatively; and information necessary to integrate climate change considerations into ocean and coastal
management is produced, readily available, and used.
Goal 8:
Collaborate so that
information and
methodologies for ocean and
coastal areas are collected,
produced, analyzed, and
easily available.
SA21 : Collaborate so that synergy occurs, lessons learned are transferred, federal efforts
effectively help local communities, and efforts are not duplicative or at cross -purposes.
SA22: Work within EPA and with the U.S. Global Change Research Program and other
federal, tribal, and state agencies to collect, produce, analyze, and format knowledge and
information needed to protect ocean and coastal areas and make it easily available.
owow
owow
61
-------
EC
GOALS AND STRATEGIC ACTIONS
Goal 9:
Support and build networks of
local, tribal, state, regional
and federal collaborators to
take effective adaptation
measures for coastal and
ocean environments through
EPA's geographically targeted
programs.
Goal 10:
Address climate driven
environmental changes in
coastal areas and provide that
mitigation and adaptation are
conducted in an
environmentally responsible
manner.
Goal 1 1 :
Protect ocean environments
by incorporating shifting
environmental conditions and
other emerging threats into
EPA programs.
SA23: Work with the NWP's larger geographic programs to incorporate climate change
considerations, focusing on both the natural and built environments.
SA24: Address climate change adaptation and build stakeholder capacity when
implementing NEP Comprehensive Conservation and Management Plans and through the
Climate Ready Estuaries Program.
SA25: Conduct outreach and education, and provide technical assistance to state and local
watershed organizations and communities to build adaptive capacity in coastal areas
outside the NEP and Large Aquatic Ecosystem programs.
SA26: Support coastal wastewater, stormwater, and drinking water infrastructure owners
and operators in reducing climate risks and encourage adaptation in coastal areas.
SA27: Support climate readiness of coastal communities, including hazard mitigation, pre-
disaster planning, preparedness, and recovery efforts.
SA28: Support preparation and response planning for impacts to coastal aquatic
environments.
SA29: Consider climate change impacts on marine water quality in NWP ocean
management authorities, policies, and programs.
SA30: Use available authorities and work with the Regional Ocean Organizations and other
federal and state agencies through regional ocean groups and other networks so that
offshore renewable energy production does not adversely affect the marine environment.
SA31 : Support the evaluation of sub-seabed sequestration of CO2 and any proposals for
ocean fertilization.
SA32: Participate in interagency development and implementation of federal strategies
through the NOC and the NOC Strategic Action Plans.
LEAD OFFICE (ft PARTNERS)
OWOW (regions)
OWOW
OWOW
OWOW
OWOW
OWOW
OWOW
OWOW (regions)
OWOW
OWOW
VISION AREA 4: WATER QUALITY: Our Nation's surface water, drinking water, and ground water quality are protected, and the risks of climate change to
human health and the environment are diminished, through a variety of adaptation and mitigation strategies.
Goal 1 2:
Protect waters of the United
States and promote
management of sustainable
surface water resources.
SA33: Encourage states and communities to incorporate climate change considerations into
their water quality planning.
SA34: Encourage green infrastructure and low-impact development to protect water
quality and make watersheds more resilient.
SA35: Promote consideration of climate change impacts by National Pollutant Discharge
Elimination System permitting authorities.
SA36: Encourage water quality authorities to consider climate change impacts when
developing wasteload and load allocations in TMDLs where appropriate.
OWOW
OWM (OWOW)
OWM
OWOW
62
-------
EC
GOALS AND STRATEGIC ACTIONS
Goal 13:
As the nation makes decisions
to reduce greenhouse gases
and develop alternative
sources of energy and fuel,
work to protect water
resources from unintended
adverse consequences.
Goal 14:
Collaborate to make
hydrological and climate data
and projections available.
SA37: Identify and protect designated uses that are at risk from climate change impacts.
SA38: Clarify how to re-evaluate aquatic life water quality criteria on more regular
intervals; and develop information to assist states and tribes who are developing criteria
that incorporate climate change considerations for hydrologic condition.
SA39: Continue to provide perspective on the water resource implications of new energy
technologies.
SA40: Provide assistance to states and permittees to assure that geologic sequestration of
CO2 is responsibly managed.
SA41 : Continue to work with States to help them identify polluted waters, including those
affected by biofuels production, and help them develop and implement Total Maximum
Daily Loads (TMDLs) for those waters.
SA42: Provide informational materials for stakeholders to encourage the consideration of
alternative sources of energy and fuels that are water efficient and maintain water quality.
SA43: As climate change affects the operation or placement of reservoirs, work with other
federal agencies and EPA programs to understand the combined effects of climate change
and hydropower on flows, water temperature, and water quality.
SA44: Monitor climate change impacts to surface waters and ground water.
SA45: Collaborate with other federal agencies to develop new methods for use of updated
precipitation, storm frequency, and observational streamflow data, as well as methods for
evaluating projected changes in low flow conditions.
SA46: Enhance flow estimation using National Hydrography Dataset Plus (NHDPlus).
LEAD OFFICE (ft PARTNERS)
OST (OWM)
OST
OWM (OGWDW)
OGWDW (OWOW)
OGWDW (OWOW,
OWM)
OWM (OW IO)
OWM
OWOW (OGWDW)
OWIO
OWOW
VISION AREA 5: WORKING WITH TRIBES: Tribes are able to preserve, adapt, and maintain the viability of their culture, traditions, natural resources, and
economies in the face of a changing climate.
Goal 15:
Incorporate climate change
considerations in the
implementation of core
programs, and collaborate
with other EPA Offices and
federal agencies to work with
tribes on climate change
issues on a multi-media basis.
Goal 16:
Tribes have access to
information on climate change
for decision making.
SA47: Through formal consultation and other mechanisms, incorporate climate change as a
key consideration in the revised NWP Tribal strategy and subsequent implementation of
CWA, SDWA, and other core programs.
SA48: Incorporate adaptation into tribal funding mechanisms, and collaborate with other
EPA and federal funding programs to support sustainability and adaptation in tribal
communities.
SA49: Collaborate to explore and develop climate change science, information, and tools
for tribes, and incorporate local knowledge.
SA50: Collaborate to develop communication materials relevant for tribal uses and tribal
audiences.
OWIO
OWIO
OWIO
OWIO
63
-------
EC
GOALS AND STRATEGIC ACTIONS
LEAD OFFICE (ft PARTNERS)
CROSS-CUTTING PROGRAM SUPPORT
Goal 1 7:
Communicate, Collaborate,
and Train.
Goal 18:
Track Progress and Measure
Outcomes
Goal 19:
Identify Climate Change and
Water Research Needs
SA51: Continue building the communication, collaboration, and training mechanisms needed
to effectively increase adaptive capacity at the federal, tribal, state, and local levels.
SA52: Adopt a phased approach to track programmatic progress towards Strategic Actions;
achieve commitments reflected in the Agency Strategic Plan; work with the EPA Work Group
to develop outcome measures.
SA53: Work with ORD, other water science agencies, and the water research community to
further define needs and develop research opportunities to deliver the information needed to
support implementation of this 2012 Strategy, including providing the decision support tools
needed by water resource managers.
OWIO
OWIO
OST (OW IO)
64
-------
EC
What approach can be used to measure adaptation progress?
Overall, the evaluators suggest that the IO move away from developing a single score across all goals
and strategic actions. Instead, we suggest that the seven phases of adaptation could be used in one of
two ways to assess progress:
1. Outputs-based approach: Assess OW's progress on implementing strategic actions, which are
mostly EPA outputs. Outputs are products or services that EPA provides, such as technical
assistance, trainings, and decision-support tools.
2. Outputs plus priority outcomes-based approach: Assess progress on goals, which are mostly
outcome-based. Outcomes are changes in awareness, behavior, or condition that result from EPA
activities. This approach will include assessing progress on EPA outputs in the initiation and
assessment phases of adaptive management, and outcomes in the later stages of adaptive
management.
The evaluation team believes outputs plus priority outcomes approach described above will best convey
progress towards goals. This approach it will require identifying outcomes associated with each goal
using a logic model approach, and gathering data about progress partners are making toward the goals.
Note that at this stage of climate change adaptation, given the many uncertainties associated with
climate change adaptation and the many factors outside of EPA's control, the evaluators suggest that
OW programs should be accountable for outputs, but the focus of tracking progress on outcomes should
be for learning, not accountability.
If IO does undertake an outputs plus priority outcomes approach, the summary result for each key
outcome could be conveyed as shown in Exhibit 5-2, which provides an example of conveying progress
toward the outcome of infrastructure resilience, within Vision Area 1 (Infrastructure). A similar table
could be prepared for each priority outcome. The evaluators do not see a reason to aggregate across
results, at least not in the near term. However, if IO aggregates results, it needs to be very careful in
applying any weighting (implicitly or explicitly). IO needs to be clear in all assumptions, and any
weighting should be deliberate and transparent.
EXHIBIT 5-2. REDUCED VULNERABILITY OF INFRASTRUCTURE TO CLIMATE CHANGE RISKS
PHASE
CRITERIA FOR ASSESSING PROGRESS ON GOALS
(Green = Phase Has Been Met, Yellow = Phase May Be Met, Orange = Phase Not Yet Met)
1-Initiation
2-Assessment
3-Response
Development
4-lnitial
Implementation
5-Robust
Implementation
OW conducted a screening assessment to identify the potential implications of climate change for water
infrastructure. (Met)
OW has conducted a broad review to better understand how climate change affects water infrastructure,
including consulting water utilities. (Met)
In collaboration with partners, OW has developed and distributed information, guides, and tools to assist
water utilities in undertaking adaptation, efficiency, and demand/supply management measures. (In
Progress/ May be met)
At least 30% of water utilities have conducted initial planning steps and updated planning documents to
address climate change risks and a few water utilities have undertaken substantive, on-the-ground
adaptation, efficiency, and demand/supply management measures. (In Progress/ May be met)
At least 30% of water utilities have undertaken substantive, on-the-ground efficiency, and demand/supply
management measures and of water utilities that have identified adaptation measures to be implemented
in the short-term, at least 30% have undertaken substantive, on-the-ground adaptation measures. (Not yet
met)
65
-------
EC
PHASE
CRITERIA FOR ASSESSING PROGRESS ON GOALS
(Green = Phase Has Been Met, Yellow = Phase May Be Met, Orange = Phase Not Yet Met)
At least 70% of water utilities have undertaken substantive, on-the-ground efficiency, and demand/supply
.... . management measures and have integrated climate change considerations into their normal processes and
° operations and of water utilities that have identified adaptation measures to be implemented in the short-
term, at least 70% have undertaken substantive, on-the-ground adaptation measures. (Not yet met)
7 M -t • . The water utility sector, independently or in conjunction with OW or other federal agencies, has
, , .. ° implemented mechanisms to monitor and evaluate water utility progress, identify lessons learned,
ap ive incorporate new climate data into planning, and continually improve performance on climate planning and
Management programming. (Not yet met)
As a first step in assessing progress on strategic actions, the evaluation team designated each type of
strategic action (e.g., technical resources/data development; fostering partnerships/collaboration, etc.)
in italics within the description of each strategic action. The evaluation team assigned each strategic
action to one of five categories; proposed categories and their definitions are as follows:
^ Policy and/or Guidance Change: The strategic action directly integrates climate change
considerations into new or existing policies, including guidance documents or other policy
interpretations.
A Financial Incentive: The strategic action encourages consideration of climate change
mitigation/adaptation efforts within new or existing financial incentives, including grant and loan
programs.
A Technical Resources/Data Development: The strategic action develops or refines data, databases, or
analytical tools designed to account for climate change scenarios.
^ Technical Assistance/Training: The strategic action provides partners with technical assistance or
training needed to respond to climate change.
A Fostering Partnerships/Collaboration: The strategic action establishes partnerships or collaboration
among EPA and its partners to address climate change.
Exhibit 5-3 presents the results of this categorization, and Appendix D identifies the assigned category
for each strategic action. Overall, the evaluation team found that the majority of strategic actions were
focused on providing technical assistance and training, or on providing technical resources and data
development.29
29 The evaluation team assigned strategic actions to the above categories according to the desired outcome of the
strategic action. For example, Strategic Action 44 (Monitor climate change impacts to surface waters and ground
water) is assigned to Technical Resources/Data Development because the intended outcome is for EPA to develop
the resources and data necessary to effectively monitor climate change impacts to surface and ground water. The
evaluation team assigned strategic actions to a single category to facilitate clarity and ease in communicating
results, however in some cases more than one category may be applicable to a single strategic action. In these
cases, the evaluation team used the strategic action's most fundamental desired outcome to categorize it. In
addition, we only assigned strategic action to the Fostering Partnerships/Collaborations category when no other
desired outcome is apparent, as fostering partnerships/collaborations is typically not an outcome in and of itself,
and measuring progress on it can be challenging.
66
-------
EC
EXHIBIT 5-3. FREQUENCY OF STRATEGIC ACTIONS BY TYPE
TYPE OF STRATEGIC ACTIONS
Technical Assistance/Training
Technical Resources/Data Development
Policy and/or Guidance Change
Fostering Partnerships/ Collaboration
Financial Incentive
NUMBER OF STRATEGIC ACTIONS
19
17
7
5
2
Next, to develop a consistent and transparent outputs-based approach to assessing progress along the
seven phases of adaptive management, the evaluation team used the baseline information on strategic
actions and the description of the seven phases of adaptive management in the 2012 Strategy to assess
progress on a sample of strategic actions. The evaluation team's process differed from OW's baseline
data interpretation described above in that we conducted a centralized review for all baseline data using
explicit considerations for assessing progress, namely the explanations and examples of each phase of
adaptive management provided by EPA in the 2012 Strategy (see Exhibit 5-4). Note that even with
explicit considerations, the process of assigning strategic actions to a phase of adaptive was subjective
because of limited baseline data available. For example, where some regions or HQ offices did not
submit baseline data, or where the data provided gave an incomplete picture of the work underway, the
evaluation team may have concluded that work has not progressed as far as it has in reality. Thus,
moving forward, it is important for IO to collect consistent and comprehensive data to be able to assess
progress as objectively as possible.
67
-------
EC
EXHIBIT 5-4. CONSIDERATIONS FOR ASSESSING PROGRESS AT EACH PHASE OF ADAPTIVE
MANAGEMENT (FROM THE 2012 STRATEGY)
NWP PHASES
EXPLANATION
EXAMPLES OF EVIDENCE OF ACHIEVEMENT
1. Initiation
2. Assessment
3. Response
Development
4. Initial
Implementation
5. Robust
Implementation
Conduct a screening assessment of
potential implications of climate
change to mission, programs, and
operations.
Conduct a broader review to
understand how climate change
affects the resources in question.
Work with stakeholders to develop
an understanding of the implications
of climate change to the mission,
programs, and operations.
Preliminary information is developed to evaluate relevance of
climate change to the mission or program; a decision is made
as to whether to prepare a response to climate change; further
exploration of climate change implications has been
authorized.
Accountabilities and responsibilities are assigned at appropriate
levels within the organization and resources are available to
develop a more in-depth assessment.
Review science literature and assessments to understand how
climate change affects the resources being protected (threat to
mission); Engage internal staff and external stakeholders in
evaluation.
Identify climate change issues and concerns and communicate
with internal and external stakeholders and partners.
Identify which specific programs are threatened and what
specific information or tools need to be developed.
Communicate findings to partners and stakeholders and engage
them in dialogue on building adaptive capacity.
Identify changes necessary to
continue to reach program mission
and goals.
Develop initial action plan.
Identify and seek the research,
information, and tools needed to
support actions.
Begin to build the body of tools,
information, and partnerships
needed to build capacity internally
and externally.
Develop initial program vision and goals for responding to
climate change.
Identify needed response actions or changes that will allow the
organization to begin to address climate impacts on its mission.
Initiate strategies and actions in a few key areas to begin to
build organizational ability to use climate information in
decision processes.
Identify program partners' needs for building adaptive
capacity.
Begin working with an external "community of practice" to
engage in tool and program development.
Rudimentary methods are put in place to track progress.
Develop a research strategy and partnerships to obtain
additional needed research.
Initiate actions in selected priority
programs or projects.
Programs are underway and lessons
learned are being applied to
additional programs and projects.
Make it clear within the organization that incorporating climate
change into programs is critical.
Initiate actions and plans identified in Step 3.
Initiate cooperative projects with partners.
Develop a range of needed information and tools.
Begin to institute changes to incorporate climate change into
core programs.
Some program partners have begun to implement response
actions.
Lessons learned are evaluated and strategies are refined.
Efforts are initiated to consider climate change in additional,
or more complex, program elements.
Continue to institute institutional changes to incorporate
climate change into core programs.
External communities of practice are in place to support
ongoing capacity development.
6. Mainstreaming
Climate is an embedded, component
of the program.
The organization's culture and policies are aligned with
responding to climate change.
All staff have a basic understanding of climate change causes
and impacts.
All relevant programs, activities, and decision processes
intrinsically incorporate climate change.
Methods for evaluating outcomes are in place.
7. Monitoring and
Adaptive
Management
Continue to monitor and integrate
performance, new information, and
lessons learned into programs and
plans.
Progress is evaluated and needed changes are implemented.
As impacts of climate change unfold, climate change impacts
and organizational responses are reassessed.
68
-------
EC
As an illustration of how the framework could work, the evaluation team has categorized progress for
strategic actions related to one goal within each vision area (see Exhibit 5-5). For each strategic action,
we assessed if work within each phase of adaptive management is fully underway, partially underway,
may have just begun, or may not yet have begun. The evaluation team's assessment depends not only
on the nature of the activities underway, but also the extent to which multiple regions were undertaking
activities. If only one to two regions reported activities underway related to a certain phase of adaptive
management, The evaluation team concluded the "phase has begun," while three to five regions
reporting activities within a phase indicated the "phase is partially underway," and six or more regions
reporting work within a phase indicated the "phase is fully underway." Note that where a phase is fully
underway, OW will loop back to reassess as new information becomes available (except in the case of
the initiation phase). In other cases we did not have enough information to assess the status of the
strategic action, or the phase is not applicable to the strategic action, or the strategic action is not
expressed in a measurable way. The key below designates these different statuses within each phase of
adaptive management. Exhibit 5-6 documents examples of activities OW has undertaken as part of each
of the four strategic actions and the evaluation team's rationale for categorizing these strategic actions
according to the identified phases of adaptive management.
The results of this assessment suggest, based on available data, indicate that OW is further along in
supporting adaptive management for goals 1 and 3 compared to goals 11, 13, and 15, but that OW has
not progressed beyond the "initial implementation" stage for any of the five goals assessed.
Note that this assessment is intended only as an illustration, as the evaluation team has made the
assessment based on limited baseline information as discussed above. If OW moves forward with using
this measurement approach, it should review the evaluation team's work with the description of each
phase of adaptive management in mind, and modify the assessments as needed. In addition, we suggest
that the IO request that HQ offices and regions review, and if necessary, update or supplement the
baseline data they have provided. The IO could then implement this assessment process for the full set
of strategic actions, and as part of the process, document the rationale for categorizing the SAs
according to the phases of adaptive management. The IO could summarize assessment results in graphic
and narrative form. If OW decides to identify priority strategic actions, then it would be necessary to
apply this approach only to those strategic actions.
69
-------
lEc
EXHIBIT 5-5. ASSESSMENT OF STRATEGIC ACTIONS ACCORDING TO PHASES OF ADAPTIVE MANAGEMENT, FOR FIVE SAMPLE GOALS
KEY:
Phase is fully underway; for all phases except initiation, OW will loop back to reassess as new information becomes available
Phase is partially underway
Phase has begun
Phase has not yet begun
The evaluation team does not have enough information to assess status
Phase is not applicable to strategic action
Strategic action is not measurable
Abbreviations used in the table: Area (A); Goal (G); Strategic Action (SA)
AREA
A1:
Infrastructure
GOAL
G01: Build the body of
information and tools
needed to incorporate
climate change into
planning and decision
making.
STRATEGIC ACTION
SA01: Improve access to vetted climate and hydrological science, modeling,
and assessment tools through the Climate Ready Water Utilities program.
(technical resources/ data development)
SA02: Assist wastewater and water utilities to reduce greenhouse gas
emissions and increase long-term sustainability with a combination of
energy efficiency, co- generation, and increased use of renewable energy
resources, (technical assistance /training)
SA03: Work with the states and public water systems, particularly small
water systems, to identify and plan for climate change challenges to
drinking water safety and to assist in meeting health based drinking water
standards, (technical assistance/training)
SA04: Promote sustainable design approaches to provide for the long-term
sustainability of infrastructure and operations, (technical
assistance /training)
PHASE OF ADAPTIVE MANAGEMENT
1
z
O
[_
<
1-
z
2
z
Ul
5
H |_
Z Q,
O "^
§5
70
-------
lEc
AREA
A2: Watersheds
a Wetland
A3: Coastal a
Ocean Waters
GOAL
G3: Identify, protect,
and maintain a
network of healthy
supportive habitat
corridor networks
G 1 1 : Protect ocean
environments by
incorporating shifting
environmental
conditions and other
emerging threats into
EPA programs
STRATEGIC ACTION
SA8: Develop a national framework and support efforts to protect remaining
healthy watersheds and aquatic ecosystems
(technical resources/ data development)
SA9: Collaborate with partners on terrestrial ecosystems and hydrology so
that effects on water quality and aquatic ecosystems are considered
(fostering partnership/collaboration)
SA10: Integrate protection of healthy watersheds throughout the NWP core
programs (policy and/ or guidance change)
SA11: Increase public awareness of the role and importance of healthy
watersheds in reducing the impacts of climate change
(technical assistance /training)
SA29: Consider climate change impacts on marine water quality in NWP
ocean management authorities, policies, and programs, (policy and/or
guidance change)
SA30: Use available authorities and work with the Regional Ocean
Organizations and other federal and state agencies through regional ocean
groups and other networks so that offshore renewable energy production
does not adversely affect the marine environment, (technical
assistance /training)
SA31: Support the evaluation of sub-seabed sequestration of C02 and any
proposals for ocean fertilization, (technical assistance/training)
SA32: Participate in interagency development and implementation of
federal strategies through the NOC and the NOC Strategic Action Plans.
(fostering partnership /collaboration)
PHASE OF ADAPTIVE MANAGEMENT
1
z
o
!<
H
Z
2
z
LU
LO
LU
UJ
O
?
4
O
1-
bP
Z Q;
i|
71
-------
lEc
AREA
Water Quality
Working with
Tribes
GOAL
G13* As the nation
makes decisions to
reduce its greenhouse
gas emissions and
develop alternative
sources of energy and
fuel, the NWP will
work to protect water
resources from
unintended adverse
consequences.
G15: Incorporate
climate change
considerations in the
implementation of
core programs and to
collaborate with
other EPA offices and
federal agencies to
work with tribes on
climate change issues
on a multi-media
basis.
STRATEGIC ACTION
SA39: Continue to provide perspective on the water resource implications of
new energy technologies, (technical assistance/ training)
SA40: Provide assistance to states and permittees to assure that geologic
sequestration of C02 is responsibly managed, (technical assistance/training)
SA41: Continue to work with States to help them identify polluted waters,
including those affected by biofuels production, and help them develop and
implement Total Maximum Daily Loads (TMDLs) for those waters, (technical
resources and data development)
SA42: Provide informational materials for stakeholders to encourage the
consideration of alternative sources of energy and fuels that are water
efficient and maintain water quality, (technical resources and data
development)
SA43: As climate change affects the operation or placement of reservoirs,
EPA will work with other federal agencies and EPA programs to understand
the combined effects of climate change and hydropower on flows, water
temperature, and water quality, (technical resources and data a
development)
SA47: Through formal consultation and other mechanisms, incorporate
climate change as a key consideration in the revised NWP Tribal strategy and
subsequent implementation of CWA, SDWA, and other core programs, (policy
change)
SA48: Incorporate adaptation into tribal funding mechanisms, and collaborate
with other EPA and federal funding programs to support sustainability and
adaptation in tribal communities, (financial incentive change)
PHASE OF ADAPTIVE MANAGEMENT
1
z
o
!<
H
Z
2
z
li 1
t/1
l/l
LU
LO
3
3
1-
uj 5
LU
O
4
o
1-
_1 <
Pz
1 — ll 1
Z 5
— uj
_l
Q.
5
o
1-
1- <
lz
CO ii i
OS
OL uj
_l
Q.
6
O
—
^
g
1-
t/1
7
*S
O O
Z 5
OL UJ
O >
bP
Z Q;
i|
72
-------
lEc
EXHIBIT 5-6. EXAMPLES OF WORK UNDER STRAETGIC ACTIONS ft RATIONALE FOR LINKING STRATEGIC ACTIONS TO PHASES OF ADAPTIVE
MANAGEMENT(AM)
STRATEGIC ACTION
EXAMPLES OF WORK UNDER STRATEGIC ACTION
RATIONALE FOR LINKING TO PHASE OF ADAPTIVE MANAGEMENT
SA01: Improve access to vetted
climate and hydrological
science, modeling, and
assessment tools through the
Climate Ready Water Utilities
program.
SA02: Assist wastewater and
water utilities to reduce
greenhouse gas emissions and
increase long-term
sustainability with a
combination of energy
efficiency, co- generation, and
increased use of renewable
energy resources.
OGWDW has developed a new version of the Climate Resilience
Evaluation and Awareness Tool (CREAT) that includes embedded basic
and advanced video training modules and allows utilities to conduct
analysis comparison scenarios for multiple time periods. Pilots were
conducted in Oakland, California and Wilmington, Delaware for the
updated software.
OGWDW developed trainings on using the CREAT tool as part of its
Climate Ready Water Utilities webinar series.
For three recent pilot projects intended to increase coordination
between EPA's Climate Ready Water Utilities and Climate Ready Estuaries
programs, CREAT was used to conduct a joint risk assessment with
watershed partners to identify current and future climate threats and
adaptation options.
In February 2012, OWM finalized and released the Planning for
Sustainability Handbook for Water and Wastewater Utilities (EPA-832-R-
12-001).
In February 2012, OGWDW updated the CRWU Toolbox, a website that
provides hundreds of climate-related resources for water utilities.
During 2012, OWM conducted five webinars to educate water and
wastewater utilities about the handbook and about facility energy
management.
In Region 1, over 100 municipal water/wastewater facilities have
participated in roundtable discussions and have been trained on energy
management plans. Two wastewater facilities are near zero net energy
and at least four others are working on plans to reach Zero Net Energy.
Region 1 has begun drafting an academic paper analyzing existing energy
use by the municipal wastewater sector in AAA, Rl, and VT. Region 4 is
developing an energy efficiency toolkit for energy efficiency for water
and wastewater utilities.
Region 4 developed and implemented an Energy Management Initiative
(EMI) with the collaboration of the Tennessee Department of Environment
and Conservation (TDEC) to work with water & wastewater public utilities
in TN to save energy and reduce their Carbon footprint
Region 7 supported the Kansas Department of Health and the
Environment with the implementation of an asset/energy management
training focused on Small Systems.
Region 9 conducted a webinar program designed to help water and
wastewater utilities identify and implement energy efficiency or clean
energy generation projects.
Region 9 continued its partnership with the U.S. Department of Energy
Industrial Assessment Centers to provide 10 energy audits to wastewater
Rationale for linking to AM phase 2/3:
EPA has developed a new version of its CREAT tool that contains
embedded training modules and enables more powerful analyses.
Moreover, EPA has initiated trainings on the tool and is using the tool to
facilitate dialogue among internal programs and external partners about
climate risks and adaptation options. These actions correspond to some of
the criteria for AM phase 3, including "begin to build the body of tools,
information, and partnerships needed to build capacity internally and
externally" and "begin working with an external community of practice
to engage in tool and program development."
Rationale for linking to AM phase 3/4:
EPA HQ and EPA regions have begun creating and disseminating a range of
technical resources to assist partners in reducing greenhouse gas
emissions from utility operations. Likewise, utilities have begun to take
action to reduce their emissions. Nevertheless, there remains limited on
the ground activity in terms of renewables, energy efficiency, and
cogeneration. Therefore, phase 4 is partially underway. The actions of
EPA and its partners align with some of the criteria for phase 4, including
"make it clear within the organization that incorporating climate change
into programs is critical," "initiate cooperative projects with partners,"
and "develop a range of needed information and tools."
73
-------
lEc
STRATEGIC ACTION
EXAMPLES OF WORK UNDER STRATEGIC ACTION
RATIONALE FOR LINKING TO PHASE OF ADAPTIVE MANAGEMENT
treatment facilities in Region 9.
During 2012, Region 9 expanded its Biogas Mapping Tool, researched
biogas characteristics, and worked with California agencies and other
interested entities to remove technical and regulatory barriers that
currently discourage the use of biogas.
SA03: Work with the states and
public water systems,
particularly small water
systems, to identify and plan
for climate change challenges
to drinking water safety and to
assist in meeting health based
drinking water standards.
Under EPA and USDA's Rural Development Rural Utilities Service MOA,
"Promoting Sustainable Rural Water and Wastewater Systems", the two
agencies co-sponsored two workshops in California and Michigan for small
system managers and operators that provided training in utility
management principles, based on the proven management approaches
including EPA's own primer on "Effective Utility Management". Workshop
attendees learned techniques to evaluate each utility's strengths and
weaknesses and to improve and measure utility performance.
Region 7 supported the Kansas Department of Health and the
Environment with the implementation of an asset/energy management
training focused on Small Systems.
Rationale for linking to AM phase 2:
EPA and USDA have created an MOA to promote sustainable small water
systems, and have begun doing so through a series of workshops. Region 7
has supported a partner with energy management training. These actions
correspond to the criteria for AM phase 2, including "communicate
findings to partners and stakeholders and engage them in dialogue on
building adaptive capacity" and "work with stakeholders to develop an
understanding of climate change implications."
SA04: Promote sustainable
design approaches to provide
for the long-term sustainability
of infrastructure and
operations.
SA8: Develop a national
framework and support efforts
to protect remaining healthy
watersheds and aquatic
ecosystems.
In February 2012, OWM finalized and released the Planning for
Sustainability Handbook for Water and Wastewater Utilities (EPA-832-R-
12-001). OWM conducted five webinars to educate water and wastewater
utilities about the handbook and about facility energy management.
NWP is working with the Office of Sustainable Community and 3 states
(NY, MD, and CA) to identify action that can be taken to integrate EPA-
HUD-DOT partnership sustainability principles into the Clean Water SRF
program.
Region 4 developed and implemented an Energy Management Initiative
(EMI) with the collaboration of the Tennessee Department of Environment
and Conservation (TDEC) to work with water & wastewater public utilities
in TN to save energy and reduce their Carbon footprint.
In Region 6, Dallas-Fort Worth Airport is investing $1.75 billion in changes
that will utilize water efficiency, green infrastructure and green
management concepts for terminals A, B, C and E. Terminal restroom
changes are currently 90% complete with 230 WaterSense labeled urinals
and 600 water efficient toilets and sinks installed.
Rationale for linking to AM phase 3:
EPA has released a technical guidance handbook and is working with
partners to identify sustainable infrastructure practices and take steps to
conserve water at utilities and an airport. These actions correspond to AM
phase 3 criteria, including but not limited to "begin working with an
external community of practice to engage in tool and program
development" and "identify program partners' needs for building
adaptive capacity."
In February of 2012 the EPA released the technical guide "Identifying and
Protecting Healthy Watersheds: Concepts, Assessments, and Management
Approaches" This document provides a framework for supporting state
efforts to protect a national framework of remaining healthy watersheds.
This document was created with many partners across multiple state and
federal agencies.
In partnership with VA DEQ, EPA helped complete a hydrologic flow
assessment with the state of Virginia in July 2012, technical document,
"Virginia Ecological Limits of Hydrologic Alteration (ELOHA): Development
of Metrics of Hydrologic Alteration." This document ties landscape
alteration and flow alteration together and examines the effects on
aquatic life, (originally classified under SA9)
In 2012 OGWDW completed PWS density maps by state or HUC 12 which
are available in MyWaters at "http://watersgeo.epa.gov/mwm/" - these
Rationale for linking to AM phase 3/4:
EPA has released a technical guide for identifying and protecting
watersheds and has completing some maps, therefore it seems that EPA
meeting the criteria of "beginning to build the body of tools, information,
and partnerships needed to build capacity" described in AM Phase 3; in
addition EPA has developed partnerships with VA and other states, thus
the agency has "Initiated cooperative projects with partners" as
described in AM Phase 4
74
-------
lEc
STRATEGIC ACTION
EXAMPLES OF WORK UNDER STRATEGIC ACTION
RATIONALE FOR LINKING TO PHASE OF ADAPTIVE MANAGEMENT
maps identify the names and population by PWSs. (originally classified
under SA9)
SA9: Collaborate with partners
on terrestrial ecosystems and
hydrology so that effects on
water quality and aquatic
ecosystems are considered.
OGWDW provided logistical support for several dozen state meetings on
source water protection matters over the last few years.
OGWDW regularly participated in interagency and public climate change
forums to better understand the technical issues in meeting climate
change challenges
In 2009 - 2012 OGWDW met periodically with federal agencies (e.g.,
USAGE, NOAA, USGS, USER) to coordinate our planning for technical
assistance to states and water utilities in climate change adaptation.
Rationale for linking to AM phase 2/3:
EPA is meeting with partners, including states and other federal agencies,
therefore it seems that the agency is "engaging partners and stakeholders
in dialog" (AM Phase 2) and starting to develop an external "community
of practice" (AM Phase 3). However, the baseline data does not mention
"initiating cooperative projects with partners" therefore it does not
appear that the Agency has yet entered AM Phase 4.
SA10: Integrate protection of
healthy watersheds throughout
the NWP core programs.
319 grant guidelines have been revised to include more flexibility in use
of funds for protecting healthy watersheds
Rationale for linking to AM phase 4: EPA has "begun to institute changes
to incorporate climate change into core programs." However, the
baseline data does not describe work that would fit under AM phase 2 or
3.
SA11: Increase public
awareness of the role and
importance of healthy
watersheds in reducing the
impacts of climate change.
Release of technical document
http://water.epa.gov/polwaste/nps/watershed/hw_techdocument.cfm
and Action Plan
http://water.epa.gov/polwaste/nps/watershed/hwi_action.cfm
SE region: The program participates in the Nature's Notebook project of
the National Phenology Network. Nature's Notebook trains citizens to
monitor how keystone plant and animal species in their own backyards or
nearby wild areas are responding to climate change. Workshops to train
citizen-scientists were held in 2012.
SE region: The Mobile Bay National Estuary Program initiated a watershed
management planning effort for Three Mile Creek just north of Mobile
Alabama. The project includes a climate change vulnerability assessment
and a community outreach effort to educate an EJ community regarding
potential vulnerabilities.
Rationale for linking to AM phase 1: EPA HQ and one region has begun to
increase public awareness, but efforts to date are limited, and there is
not yet evidence of a coordinated outreach campaign.
SA29: Consider climate change
impacts on marine water
quality in NWP ocean
management authorities,
policies, and programs.
HQ: OWOW issued a memo recommending consideration in listings of
waters for pH impairments if appropriate. OCPD has assigned staff to
monitor the issue. EPA's ocean acidification workgroup has been meeting
monthly to keep interested people informed. Likewise, the workgroup is
doing a screening to see what could be done with regard to acidification
problems in coastal/near shore waters.
Rationale for linking to AM phase 1 /2:
EPA has issued a memo recommending assessment of acidification
effects, and has mobilized staff to monitor the problem. Furthermore, a
workgroup has begun screening potential solutions to acidification. These
actions correspond to some of the criteria for AM phase 2, including
"identify climate change issues and concerns and communicate with
internal and external stakeholders and partners" and "communicate
findings to partners and stakeholders and engage them in dialogue on
building adaptive capacity." This SA does not appear to have entered AM
Phase 3.
SA30: Use available authorities
and work with the Regional
Ocean Organizations and other
federal and state agencies
through regional ocean groups
and other networks so that
offshore renewable energy
production does not adversely
affect the marine environment.
No action taken.
No known action to date.
75
-------
lEc
STRATEGIC ACTION
EXAMPLES OF WORK UNDER STRATEGIC ACTION
RATIONALE FOR LINKING TO PHASE OF ADAPTIVE MANAGEMENT
SA31: Support the evaluation of
sub-seabed sequestration of
C02 and any proposals for
ocean fertilization.
HQ: EPA serves on the US delegation to the London Convention and
London Protocol and has supported convention activities, including a
statement regarding ocean fertilization and guidance for sub seabed
carbon capture and sequestration.
HQ: OWOW, in coordination with OGC, OGWDW, and OAR, has formed an
interagency working group with the Department of Interior/Bureau of
Ocean Energy Management to address technical, regulatory, and other
issues associated with offshore carbon capture and storage (sub-seabed
sequestration of C02).
SA32: Participate in
interagency development and
implementation of federal
strategies through the NOC and
the NOC Strategic Action Plans.
EPA has been actively involved in writing and implementing the National
Ocean Council Strategic Action Plan for the last three to four years.
Rational for linking to AM phase 1 /2:
EPA has provided guidance on sub seabed carbon capture to the London
Convention and Protocol and has formed an interagency working group to
begin addressing the technical and regulatory issues associated with
carbon capture. These actions demonstrate achievement of some phase 2
criteria, including "identify climate change issues and concerns and
communicate with internal and external stakeholders and partners" and
"communicate findings to partners and stakeholders and engage them in
dialogue on building adaptive capacity."
Rationale for linking to AM phase 4:
EPA's active involvement in writing and implementing the National Ocean
Council's Strategic Plan fulfills several criteria for AM phase 4, including
"initiate cooperative projects with partners" and "begin to institute
changes to incorporate climate change into core programs."
SA39: Continue to provide
perspective on the water
resource implications of new
energy technologies.
OW published "Principles for an Energy-Water Future" at
http://water.epa.gov/action/energywater.cfmincluding the following
concepts: end-user water and energy efficiency, a water-wise energy
sector, an energy-wise water sector, viewing wastewater as a source of
renewable resources, integrated resource planning, and maximizing social
benefits.
OW is engaging in dialogue with DOE
Rationale for linking to AM phase 3/4 :
OW has published a guidance document that provides perspective on the
water resource implications of new energy technologies, indicating that
they have begun to "build the body of tools, information, and
partnerships needed to build capacity" described in AM phase 3.
Additionally, the publication of this document and the engagement in
dialogue with DOE serves as evidence of "developing a range of needed
information and tools" and "initiating cooperative projects with
partners" as described in AM phase 4. Finally, this SA calls for EPA to
"continue to provide perspective" meaning that EPA will have to reassess
progress as new information becomes available.
SA40: Provide assistance to
states and permittees to assure
that geologic sequestration of
C02 is responsibly managed.
OWOW works directly with stakeholders to support potential permits for
C02 sequestration under the Underground Injection Control program.
OWOW plans to Participate in and present at national meetings and
workshops to communicate rule requirements to ensure that states and
permit applicants are able to submit Class VI permit applications that
meet the Class VI requirements and ensure protection of USDWs.
OWOW plans to coordinate with states interested in primacy for Class VI:
developing and evaluating a regulatory crosswalk and other pieces of a
Class VI primacy application.
OWOW plans to conduct the formal rulemaking process for any states that
finalize their primacy applications during the FY13 timeframe.
Region 4 submitted comments on a Class V C02 experimental permit
application and draft UIC permit which was eventually issued by the
Alabama Department of Environmental Management to Denbury Onshore.
Region 4 received and reviewed post injection monitoring reports from
permittees for two Class V C02 experimental wells in Kentucky (Dl state).
Region 4 submitted comments to the Florida Department of
Environmental Protection on a Class V C02 experimental permit
application for a proposed C02 sequestration project near Tampa,
Florida.
Rationale for linking to AM phase 4 :
EPA has worked with the States and other stakeholders to support
permitting of C02 sequestration projects as described in AM phase 4. The
Alabama DEM issued an experimental Class V C02 permit application, thus
meeting the AM phase 4 requirements that "some program partners have
begun to implement response actions." Additionally, by a partner issuing
an experimental permit, the AM phase 4 description "Initiate actions in
selected priority programs or projects" has been met.
76
-------
lEc
STRATEGIC ACTION
EXAMPLES OF WORK UNDER STRATEGIC ACTION
RATIONALE FOR LINKING TO PHASE OF ADAPTIVE MANAGEMENT
SA41: Continue to work with
States to help them identify
polluted waters, including
those affected by biofuels
production, and help them
develop and implement Total
Maximum Daily Loads (TMDLs)
for those waters.
No Baseline Data Received
Rationale for not assessing AM status:
The evaluation team did not receive any baseline data for this strategic
action.
SA42: Provide informational
materials for stakeholders to
encourage the consideration of
alternative sources of energy
and fuels that are water
efficient and maintain water
quality.
OWM completed an Energy Management Progress Report documenting
OWM and Regional efforts to assist utilities in developing energy
management plans based on OWM's Energy Management Guidebook.
OWM conducted three webinars on energy management for utilities and
two on the Planning for Sustainability Handbook.
SA43: As climate change
affects the operation or
placement of reservoirs, EPA
will work with other federal
agencies and EPA programs to
understand the combined
effects of climate change and
hydropower on flows, water
temperature, and water
quality.
Rationale for linking to AM phase 3:
EPA has conducted a series of webinars on energy management for
utilities and planning for sustainability, indicating they have "identified
program partners' needs for building adaptive capacity" as described in
AM phase 3. Also, by completing an Energy Management Progress Report
that documents OWM and Regional efforts to assist utilities in developing
energy management plans based on OWM's Energy Management
Guidebook, EPA has met the AM phase 3 criteria of "rudimentary methods
are put in place to track progress."
OWM is involved with a project to develop a tool to estimate critical low
flow conditions for all waters within the National Hydrography Dataset
(NHD).
Rationale for not assessing AM status:
The evaluation team does not have enough information about EPA's
involvement in the tool to estimate critical low flow conditions for all
waters within the National Hydrography Dataset (NHD) to assess progress.
SA47: Through formal
consultation and other
mechanisms, incorporate
climate change as a key
consideration in the revised
NWP Tribal strategy and
subsequent implementation of
CWA, SDWA, and other core
programs. (Policy Change)
SA48: Incorporate adaptation
into tribal funding mechanisms,
and collaborate with other EPA
and federal funding programs
to support sustainability and
adaptation in tribal
communities. (Financial
Incentive Change)
Region 1 has conducted tribal consultation with 10 federally recognized
tribes in New England regarding the draft National Water Program
Climate Change Strategy, including briefing on monthly RTOC conference
calls and site visits.
Region 1 Held a New England Federal Partners session at the October
2011 Regional Tribal Environmental Training Workshop to discuss federal
agencies' climate change activities and services/products available to
tribes.
Rationale for linking to AM Phase 2:
Region 1 has worked with stakeholders and partners to develop an
understanding of the implications of climate change by consulting with
federally recognized tribes on the draft National Water Program Climate
Change Strategy. Additionally, they have communicated with internal and
external stakeholders and partners the New England Federal Partners
Session at the 2011 Regional Tribal Environmental Training Workshop.
Region 5 reports that several tribes in the region have included climate
change adaptation work as a priority in their Tribal Environmental
Agreements.
Rationale for not assessing AM status:
The baseline data for this strategic action is too vague to determine an
appropriate AM phase.
77
-------
lEc
Considerations for Using an Outputs-Based Approach
The outputs-based measurement approach, which designates what phase of adaptive management OW
has reached in implementing the strategic actions, is, in the evaluation team's view, a reasonable and
feasible approach to measuring progress on the 2012 Strategy. One particular advantage of this
approach is that it uses data that OW HQ offices and regions can provide, and does not require
collecting data from partners outside of EPA (e.g., other federal agencies, states, local agencies, or
NGOs). In addition, this measurement approach focuses on the work that OW can directly influence (i.e.,
the strategic actions), rather than work undertaken by partners which may be largely outside of OW's
control. Moreover, this approach does not require aggregation across diverse strategic actions, and the
corresponding complications of determining an appropriate weighting scheme. However, this
measurement approach does raise issues and challenges, as described below.
•*• Uncertain connection to progress on goals: Measuring progress on outputs does not necessarily
gauge progress on OW's goals. Since achieving the outcomes relies on the work of many external
partners and factors outside of OW's control, it is possible that OW may fully implement strategic
actions and yet progress on the ultimate goals may fall short. This approach says more about HQ
office and regional activities than achievement of ultimate climate change adaptation goals.
A Imperfect alignment of strategic actions and adaptive management phases: Some strategic actions
cannot be mapped well to the seven phase adaptive management framework. For example, SA39:
"Continue to provide perspective on the water resource implications of new energy technologies," is
framed as an ongoing activity that does not fit well with the progress from initiation to response
development to implementation assumed in the seven phases of adaptive management. Other
strategic actions fit within the first few phases of adaptive management, but the later phases (e.g.,
robust implementation, mainstreaming, and monitoring and adaptive management) do not apply.
For example, SA11: "Increase public awareness of the role and importance of healthy watersheds in
reducing the impacts of climate change," does not fit within the concepts of implementation,
mainstreaming, or adaptive management, where the focus is on implementing programs or projects
rather than raising awareness.
A Progress through phases of adaptive management is not linear: The seven phases of adaptive
management imply that HQ offices and regions will make stepwise progress, from initiation and
assessment to response development, implementation, and so on. In reality, HQ offices and regions
may not conduct a thorough assessment before developing responses, and early steps in
implementation (e.g., pilot projects) may in fact be part of the assessment phase, as EPA and
partners work together to determine what responses are needed to climate change. In addition, HQ
offices and regions will likely need to double back and reconsider earlier steps as new information
becomes available. Thus while the seven phases of adaptive management presents a clear
conceptual framework that makes sense overall, for any particular strategic action OW's work may
not progress in a linear fashion. The scoring framework shown in Exhibit 5-4 seeks to convey the
potential non-linear nature of OW's work.
•*• Specific nature of the OW commitment: Many of the strategic actions do not clearly define the
specific nature of OW's commitment. They are often broadly written and do not identify specific
activities that OW is planning to take. For example, in strategic actions involving partners, it is often
unclear what actions OW will take relative to the other partners (e.g., providing funding, technical
assistance, developing tools, etc.). Additionally, many of the strategic actions employ vague and/or
ambiguous terms such as intend, consider, encourage, promote, and collaborate, which muddle
78
-------
EC
OW's expected outputs. To facilitate measurement, the strategic actions should clearly define OW's
role and provide an anticipated output that can be used to gauge progress.
A Audiences: Many actions relate to partnerships or conveying information to key audiences, but
those audiences are often not well defined. Often there are multiple levels of audiences NWP is
trying to reach. It will be important for HQ offices and regions to identify specifically which
audiences they seek to influence, to the extent possible. For example, when working on a technical
document, OW Offices should articulate who they expect will read and use the document, and
which partners will ultimately be influenced by the work.
A Criteria for measuring progress: The seven phases currently don't have clear and universally
applicable criteria for meeting each phase. However, given the diversity of strategic actions, it is
inherently difficult to devise a single set of criteria that applies to strategic actions in a diverse range
of areas (e.g. Infrastructure & Working with Tribes). Nonetheless, this leaves measurement open to
interpretation and lack of consistency. As such, using the above approach requires careful judgment
to apply general descriptions associated with each phase of adaptive management included in the
2012 Strategy, combined with synthesis of information submitted on each strategic action using
clear decision rules.
-^ Anecdotal data: It is difficult to gauge how much progress has been made based on examples of
work, without full information about work undertaken that is relevant to each strategic action. It is
not clear whether the baseline data collected to date represents a comprehensive description of HQ
office and regional efforts on climate change adaptation. To use the data to measure progress, it will
be important to have a comprehensive and detailed description of work completed and underway.
A Scale: It is difficult to gauge progress without a good sense of the full scope of work to be completed
(e.g., how many partners need to be engaged, how many regions need to take action, how many
guidance documents need to be written). As part of the description of work completed and
underway, HQ offices and regions would ideally describe their efforts in the context of the full scope
of activities they intend to complete (e.g., "we have engaged eight out of 10 identified partners.")
However, it may be inherently difficult to know upfront the scope of the work needed, particularly
at the assessment and response development phases.
•*• Data Consistency and Quality: The baseline data OW collected illustrate challenges of data
consistency and quality that are important to address in any measurement system. For example, HQ
offices and regions sometimes cited future plans or past activities as examples of current progress
on strategic actions. Some HQ offices and regions also included activities that were only tangentially
related to the strategic actions. Finally, the template OW used to collect baseline data was open
ended, which allowed inconsistent responses. For example, HQ offices and regions could describe
progress on any strategic action they chose. The template also asked about future plans instead of
focusing on current progress. These challenges resulted in a baseline dataset that lacked coherence
and prevented comparison among HQ offices and regions. See Appendix E for more details on these
issues, and general guidance for ensuring data consistency and quality moving forward.
The evaluation team has summarized the advantages and disadvantages of an outputs-based approach,
compared to an outputs plus priority outcomes based approach, in Exhibit 5-7 below.
79
-------
EC
EXHIBIT 5-7. PROS AND CONS OF THE OUTPUTS-BASED APPROACH
ADVANTAGES
DISADVANTAGES
EPA staff are more comfortable measuring progress where
they have direct influence
Data can be collected from internal sources only
Ability to make progress on outputs is more certain
If outputs are accomplished, it can generally be attributed
to EPA efforts
Achieving strategic actions does not necessarily lead to
achieving goals; output measures do not directly track
progress on goals
Some OW strategic actions do not map well to seven phases
of adaptive management
It is difficult to develop broadly applicable criteria for
meeting each phase of adaptive management
Progress through phases of adaptive management is not
always linear
Outputs plus Priority Outcomes-Based Approach
Given the potential limitations of focusing on outputs for measuring progress on adaptive management,
the evaluation team recommends developing a measurement approach that encompasses outputs and
priority outcomes, and uses the framework of the seven phases of adaptive management to assess
progress. The evaluation team finds that the seven phases represent a reasonable approach to assessing
adaptation progress on the 2012 Strategy, given the broad and diverse nature of the goals identified
therein. The seven phases provide a general framework to summarize progress for each of the five
vision areas and for the strategy as a whole. It can be applied across many different types of goals and
activities, and at different geographic scales (e.g., states, regions, and nation-wide). This section
describes how OW could develop an outputs plus priority outcomes-based approach by applying the
seven phases to the goals included in the 2012 Strategy.
Using an outputs plus priority outcomes approach, OW would interpret the seven phases differently
than discussed in the previous section on using strategic actions to track progress on adaptive
management. In the previous section, we interpreted OW's progress on the seven phases as related to
progress on implementing OW's strategic actions; here we interpret "implementation" in Phases 4-7 to
refer to on-the-ground activities, which are typically conducted by OW's partners.
The evaluation team suggests starting with the logic models to define the outcomes that OW is seeking
to achieve. The logic models identify changes in condition sought for each of the five vision areas. For
example, Area 1: Infrastructure changes in condition sought are reduced vulnerability to climate change
risks, reduced utility energy use, and reduced water use by consumers and industry (see Exhibit 5-8).
OW could measure progress towards adaptive management on these outcomes. Note that because this
measurement approach will likely require an investment of time to gather and assess the needed data,
the evaluation team suggests that OW use this measurement approach only for priority outcomes.
80
-------
EC
EXHIBIT 5-8. AREA 1: INFRASTRUCTURE OUTCOMES
Audience
Awareness
Audience
Behavior
Conditions
Understand
potential
climate change
impacts &
prioritize risks
Understand
adaptation
options
Understand
energy
efficiency and
sustainability
options
Understand
how to
' promote
supply &
demand
management
\
\
i/
Implement
adaptation
measures
Implement
sustainability
and efficiency
measures
Take action to
manage water
supply &
demand
Reduced
vulnerability to
climate change
risks
Reduced utility
energy use
Consumers &
industry use less
water
Once OW identifies high priority outcomes it seeks to achieve, then it will be necessary to identify
criteria that needed to be satisfied to meet each adaptation phase, for each outcome. In Exhibit 5-9, the
evaluation team provides examples of such criteria for the outcome of reduced vulnerability of
infrastructure to climate change risks, and our assessment of progress toward achieving these outcomes
based on the criteria and available data.
A Note that Phases 1-3 incorporate OW outputs, e.g., conducting a screening assessment to identify
the potential implications of climate change for water infrastructure and developing and
distributing information and tools to water utilities.
A Phases 4-7 cover outcomes, specifically water utility progress on planning for climate change, and
undertaking on-the-ground measures where called for by plans.
Phases 4-7 are set up as graduated phases; for example, Phase 4/lnitial Implementation can be pegged
to 30% of the target audience taking action, while Phase 5 may be pegged to 50% or 70%. Phases 4-7 are
shaded in the Exhibit 5-9 to denote that these phases are generally outside of OW's direct sphere of
influence, although OW may play a role in Phase 7. It is important to note that these are just examples
of criteria; if OW were to take the above approach, it would need to develop clear and appropriate
criteria in collaboration with staff responsible for each priority outcome.
After OW has defined appropriate criteria, it will be necessary to collect data on what progress has been
achieved toward the outcomes, and then assess progress based on whether there is evidence of
meeting the criteria at each stage for each priority outcome. OW could document this information as in
Exhibit 5-9 below. OW may want to pilot this approach with just a few priority outcomes. After a pilot
81
-------
EC
phase, OW would have more information to determine if the approach is workable and could be scaled
up to encompass the entire 2012 Strategy.
EXHIBIT 5-9. EXAMPLE CRITERIA AND ASSESSMENT OF PROGRESS ON REDUCING VULNERABILITY
OF WATER INFRASTRUCTURE TO CLIMATE CHANGE RISKS
PHASE
CRITERIA FOR ASSESSING PROGRESS FOR AREA 1:
REDUCED VULNERABILITY OF INFRASTRUCTURE
TO CLIMATE CHANGE RISKS
HAS THE PHASE OF ADAPTIVE
MANAGEMENT BEEN MET TO SUPPORT THE
GOAL?
1-Initiation
OW conducted a screening assessment to identify the
potential implications of climate change for water
infrastructure.
Yes: OW has conducted a broad review to
better understand how climate change affects
water infrastructure
2-Assessment
OW has conducted a broad review to better
understand how climate change affects water
infrastructure, including consulting water utilities.
Yes: OW has conducted a broad review to
better understand how climate change affects
water infrastructure
3-Response
Development
4-lnitial
Implementation
5-Robust
Implementation
6-Mainstreaming
7-Monitoring and
Adaptive
Management
In collaboration with partners, OW has developed and
distributed information, guides, and tools to assist
water utilities in undertaking adaptation, efficiency,
and demand/supply management measures.
Maybe: OW has developed, refined, and
distributed the GREAT tool to partners
At least 30% of water utilities have conducted initial
planning steps and updated planning documents to
address climate change risks.
and
A few water utilities have undertaken substantive,
on-the-ground adaptation, efficiency, and
demand/supply management measures.
At least 30% of water utilities have undertaken
substantive, on-the-ground efficiency, and
demand/supply management measures.
and
Of water utilities that have identified adaptation
measures to be implemented in the short-term, at
least 30% have undertaken substantive, on-the-ground
adaptation measures.
At least 70% of water utilities have undertaken
substantive, on-the-ground efficiency, and
demand/supply management measures and have
integrated climate change considerations into their
normal processes and operations.
and
Of water utilities that have identified adaptation
measures to be implemented in the short-term, at
least 70% have undertaken substantive, on-the-ground
adaptation measures.
The water utility sector, independently or in
conjunction with OW or other federal agencies, has
implemented mechanisms to monitor and evaluate
water utility progress, identify lessons learned,
incorporate new climate data into planning, and
continually improve performance on climate planning
and programming.
Not clear/data not available
Unlikely
Unlikely
Unlikely
82
-------
EC
Considerations for Using an Outputs plus Priority Outcomes-Based Approach
The evaluation team suggests that OW consider the following factors specific to implementing an
outputs plus priority outcomes-based approach to measuring progress on climate change adaptation.30
•*• Barriers to collecting data from partners: To assess progress on outcomes, OW will need to collect
data from its partners, as state and local governments and water utilities are the primary
implementers of climate change adaptation efforts. For example, using the overarching framework,
data collected on OW's activities will generally inform the assessment of progress on Phases 1-3, but
in most cases will not address whether criteria have been met for Phases 4-7, because those phases
are generally implemented by OW partners. Collecting data from OW partners poses several
challenges.
• Most if not all interviewees for interviewed for the retrospective evaluation emphasized the
extreme budget and resource shortages currently faced by the Agency, and indicated that
current budgets would not support the comprehensive data collection efforts required to
understand outcomes of OW's efforts on climate change adaptation.
• Many interviewees pointed out that EPA does not have the regulatory authority to compel data
collection from partners, and many partners would not respond unless they were mandated to
do so. For example, interviewees noted that water utilities are already required to report data
to EPA under several different regulatory programs, and are thus unlikely to participate in a
voluntary data collection.
• Finally, for OW to collect data from partners, it would need to go through an ICR request and
approval process with OMB. Seeking ICR approval can be a lengthy process; it make take up to
nine months for OMB to approval requests, and significant staff and/or contractor time is
required prepare analyses and documentation for ICR requests.
• One potential solution to the above data collection barriers would be to focus on outcomes with
an existing mechanism for data collection, such as grant programs, and investigate if IO could
include additional data collection for measurement purposes into existing grant reporting
processes.
• OW could also confer with outside organizations noted in Appendix C that have conducted
relevant data collections with EPA partners. If an outside organization(s) plans to conduct the
data collection in the future, an opportunity may exist for OW to insert questions into the data
collection, overcoming the resources and authority barriers. However, these questions must
have utility for the outside organization to avoid the ICR barrier. Moreover, OW will be limited in
the number and type of questions it can ask of partners using this approach.
A Attribution issues: Seeking to assess progress on outcomes raises the question: can progress be
attributed to OW efforts? Would the outcome have occurred in absence of OW activities? With
topics as complex as climate change adaptation, it is likely that many factors lead to outcomes
achieved, including policies and influences exerted by other federal agencies, and by state and local
governments. Moreover, as discussed above, limited existing data sources address EPA or OW
contribution to outcomes. As such, OW will need to focus on characterizing its contribution to
outcomes seen by focusing on drawing clear connections between OW program activities and
outcomes realized. As part of collecting data from partners, OW will need to inquire not only about
30 Note that some of the considerations for using an outputs-based approach also apply with an outcomes-based
approach, e.g., the need to define key audiences and criteria for assessing progress.
83
-------
EC
outcomes, but about the role of OW resources in leading to those outcomes. OW may want to
retain the services a trained evaluator to design data collections that probe on OW's contribution in
a standardized fashion. A trained evaluator will employ question formats yielding data that can be
readily aggregated and analyzed.
•*• Lack of buy-in from OW staff for outcome-based approach: Over the course of this evaluation OW
staff raised serious concerns about measuring progress on the 2012 Strategic Plan given the severe
resource constraints faced by the Agency and the lack of resources dedicated to climate change
programming. Interviewees expressed concern that measurement for accountability purposes
provides limited value and diverts resources from mainstreaming climate change into programs.
Interviewees contacted for the retrospective evaluation raised particular concerns with an outcome-
based measurement approach, compared to a measurement approach based on OW's strategic
actions, given resource constraints and the above concerns about data collection from partners. OW
will need to address this key barrier to successfully implement the 2012 Strategy.
The evaluation team has summarized the advantages and disadvantages of an outputs plus priority
outcomes-based approach in Exhibit 5-10 below.
EXHIBIT 5-10. PROS AND CONS OF THE OUTPUTS PLUS PRIORITY OUTCOMES APPROACH
ADVANTAGES
DISADVANTAGES
Approach tracks progress on key goals: goes beyond
assessing outputs to assessing the state of climate change
adaptation in the field, which is ultimately what matters.
Identifying key outcomes (particularly using the logic model
framework) would be a helpful step in clarifying OW's
priorities on climate change adaptation.
Difficulty of collecting data from external partners.
Uncertainty about attributing progress on goals to EPA
actions because the approach involves assessment of progress
on activities outside of EPA's direct sphere of influence.
Lack of buy-in from OW staff for an approach that includes
outcomes.
Criteria for each vision area would need to be carefully
defined by expert program staff, and then pilot tested.
Using Existing Data to Assess Outcomes
The evaluation team considered whether existing data could help assess progress towards OW's goals
for climate change adaptation. As presented in Chapter 2 and in Appendix C, the evaluation team
reviewed several existing resources and reports that describe progress on climate change adaptation.
These resources provide context for OW's efforts to encourage adaptation planning and can help to
define the universe associated with OW's work on climate change adaptation. For example, the
Georgetown Climate Center tracks state and local climate adaptation plans; it provides a brief
description and links to planning documents. The site is sortable and searchable. The National Climate
Assessment, which is updated regularly, summarizes climate change adaptation efforts including those
specific to coastal areas and wetlands and watersheds.
In some cases, existing resources provide a good source of information for understanding progress on
adaptive management that has a direct connection to OW activities. For example:
A GREAT Analytics Data: GREAT is a software tool designed to assist drinking water and wastewater
utility owners and operators in understanding potential climate change impacts and in assessing the
related risks at their utilities. Users download and register for the software via the OW website.
84
-------
EC
Analytic data that tracks the number of downloads could, along with registration data, be used to
assess the tool's market penetration in the water utility industry.
•*• Climate Ready Estuary Project Information: The OW's program's webpage contains project
summaries and links to Climate Ready Estuary projects in each OW region. Many of the projects
pertain to analysing coastal watersheds for climate vulnerability. However, a large portion of the
projects are also intended to improve coastal resilience to climate change. In all, the site contains 36
project summaries, and each one links to an external website for the project or the project partners.
OW could use the project summaries and linked information to understand progress on adaptive
management related to the Climate Ready Estuaries program.
However, in most cases, existing data sources do not present information on adaptive management
efforts in a way that connects these efforts to OW activities. Therefore, most of these resources cannot
be used independently as a substitute for collecting outcome data on OW efforts. For some outcomes,
OW may be able to combine information gleaned from existing resources with internal information on
OW policies, technical assistance, training, and other activities that connect OW with the outcomes
discussed in existing resources. Another limitation of using existing resources is that many resources
identified appear to be one-time reports, as opposed to resources that are continually updated; one-
time reports will be of limited utility to OW measurement efforts.
Given all of the limitations of existing resources, if OW seeks to measure progress on outcomes, it will
need to develop a strategy for collecting new data for each outcome measure developed, which as
noted earlier will involve collecting data from EPA partners.
Using Existing Measures to Track Progress on Climate Change Adaptation
Given that EPA already has a detailed performance management structure, with many performance
measures already defined for each program, some reviewers have asked if OW should develop
performance measures specifically related to climate change adaptation at all. One alternative would be
to train staff to account for the potential impacts of climate change when assessing progress on existing
measures.
For example, one OW measure in the Drinking Water program tracks the "Percent of community water
systems that meet all applicable health-based standards through approaches that include effective
treatment and source water protection." Climate change will likely impact the ability of community
water systems to meet applicable health-based standards, because higher temperatures and drought
may affect water supplies and levels of pathogens in the water. Rather than develop a measure
specifically related to efforts to prepare community water systems for the impacts of climate change,
the Drinking Water program could simply continue to track progress on its existing measure,
understanding that targets will need to be met in spite o/climate change.
The evaluation team believes this measurement approach merits consideration. However, given the
many uncertainties about how climate change will affect progress on existing measures, this approach
may not provide sufficient information to program managers about whether they are making
appropriate progress in addressing the challenges of climate change. Nevertheless, the evaluators fully
support the concept that climate change adaptation should be integrated into existing work (rather than
functioning as a separate, add-on set of activities), and over time we encourage OW to consider whether
existing measures may be sufficient to capture progress on agency goals in light of climate change.
85
-------
EC
What specific elements need to be applied to the phased approach to tracking
progress outlined in the 2012 strategy, to make it a robust measurement
framework?
IO needs to develop objective criteria for each phase of adaptive management to facilitate consistent
measurement. Regardless of whether IO selects an outputs based approach or an outputs plus priority
outcomes approach, the criteria for meeting each phase of adaptive management should be clear and
objective, so that different reviewers assessing the same data would reach the same conclusions about
progress made. If IO selects an outputs plus priority outcomes approach, it will be necessary to define
criteria for each phase of adaptive management for each outcome as discussed above. We suggest that
IO set specific, quantitative thresholds for meeting each stage; however, determining appropriate
thresholds will require pilot testing.
In addition, we understand that IO may seek to aggregate results across outcomes or strategic actions.
We caution that any aggregation should be undertaken with care so that any assumptions are explicit
and all weighting is deliberate and transparent. Any effort to "add up" or summarize progress across
strategic actions, vision areas, or outcomes implicitly involves weighting, and this should be carefully
considered. Note that if results are aggregated without explicit weighting, as OW has done previously,
then all results and steps in the adaptive management framework are implicitly given equal weight.
What, if, any, revisions should EPA make to its baseline data collection process to
ensure that data collected are meaningful and objective?
The evaluation team reviewed the approach that IO took in measuring progress on the 2012 Strategy to
date, as presented in the 2012 Highlights of Progress report. We documented several data quality and
consistency challenges with reported data. Moving forward, we recommend that IO take steps to
adhere to key tenets of data quality and consistency described in our recommendations. Core among
these, IO will need to develop data reporting templates, clear instructions for reporting, and institute a
quality control plan. The evaluation team also noted several issues regarding assumptions, implicit
weighting factors, and transparency with lO's previous approach. Thus, we recommend that IO take a
different approach to measurement moving forward, as described earlier in this chapter. Additional
details on recommendations for data reporting are provided in Chapter 6.
How can lessons learned inform measuring progress in EPA-wide efforts?
One of this project's original evaluation questions asked:
How can OW's measurement approach inform measuring progress in the EPA-wide
Adaptation Plan, and inform development of the next Agency 4-year Strategic Plan?
After conducting the evaluation, the evaluation team, in conjunction with EPA, decided to reframe this
question to ask:
How can lessons learned from this evaluation of OW's approach inform measuring progress in
the EPA-wide Adaptation Plan, and inform development of the next Agency 4-year Strategic
Plan?
To answer this question we first explore lessons learned from this evaluation project that are relevant to
other offices as they develop implementation plans in response to the Agency-wide Climate Change
Adaptation plan. Then, we explore lessons learned from this evaluation project that relate to the
Agency-level as it contemplates climate change adaptation measures for the next Strategic Plan.
86
-------
EC
Lessons Learned on Measurment Relevant to the Agency-wide Adaptation Plan
A central goal of EPA's Climate Change Adaptation Plan is to strengthen the capacity of EPA staff and
partners across the country to anticipate and respond to the effects of climate change. Strengthening
capacity will help EPA staff and partners integrate climate adaptation into everyday work by providing
them with needed data, information, and tools. The Adaptation Plan includes a list often priority actions
that the Agency will take to integrate climate change adaptation into its programs, rules, and
operations:
1. Fulfill Strategic Measures in FY2011-2015 EPA Strategic Plan
2. Protect Agency facilities and operations
3. Factor legal considerations into adaptation efforts
4. Strengthen adaptive capacity of EPA staff and partners through training
5. Develop decision-support tools that enable EPA staff and partners to integrate climate
adaptation planning into their work
6. Identify cross-EPA science needs related to climate adaptation
7. Partner with tribes to increase adaptive capacity
8. Focus on most vulnerable people and places
9. Measure and evaluate performance
10. Develop Program and Regional Office Implementation Plans (the OW 2012 Strategy serves as
the implementation plan for OW)
Following the release of the agency-wide Adaptation Plan, each EPA office and region is required to
develop an implementation plan and will need to track progress on it. OW's 2012 Strategy is its
implementation plan in response to the agency-wide Adaption plan. Earlier in this chapter we discussed
in detail alternative options for OW to track progress on its implementation plan (the 2012 Strategy).
Overall, we recommend that OW strive to measure progress on a few priority outcomes, at least on a
pilot basis. Like OW, other EPA offices could go through a similar process of using logic models to select
priority outcomes, and developing measures, criteria, and data collection strategies relevant to those
priority outcomes. We recommend that programs pilot this approach with mature programs, where
outcomes may be apparent and mechanisms for measurement (e.g., grant reporting requirements, or
regulations that incorporate reporting mechanisms) may be available.
Alternatively, other EPA offices could to continue to use existing measures, and track progress on them
in light of climate change. Using this approach, it would be important to understand the potential for
climate change to affect existing measures. It may be necessary to collect contextual information to
make it possible to understand the extent to which climate change is making it more difficult to meet
targets for existing goals. The evaluation team does not make a recommendation on which of these
approaches is preferable; however, we do think that any such approach should be pilot tested on a few
outcomes or measures.
Whatever approach other offices choose to take in measuring progress on climate change adaptation,
we recommend they keep the following general lessons on measurement in mind:
87
-------
EC
A Focus on relatively few priorities. One of the key lessons learned in developing measurement
approaches for the OW 2012 Strategy is that measuring something as multifaceted and complex as
adaptation to climate change is very difficult, and even more so when the measures are very
specific, detailed, and numerous. At this early stage, and in the current environment of extreme
resource constraints, developing just a few higher level measures, or measuring progress on just a
few priority goals, is likely the best approach for the other offices to take in developing their
implementation plans for the Adaptation Plan. We recommend selecting a small set of outcome
measures. We do not recommend using a comprehensive list of strategic actions/outputs as an
organizing principle for performance measurement (though it may be helpful for program managers
to internally track progress on strategic activities or actions).
A Weigh the merits of measuring outputs vs. outcomes. Other EPA offices will need to wrestle with
some of the same fundamental decisions that OW is considering: measuring outputs of EPA's work,
which are easier to measure and more under EPA's direct control but less related to the desired end
results; versus measuring outcomes of EPA's work, which are harder to measure and often depend
on partners' efforts. Where possible, the evaluation team recommends focusing measurement
efforts on outcomes, even though we recognize that doing so will require an investment of time and
resources, and will therefore also require prioritization.
A Be transparent about assumptions and weighting. It is difficult to describe progress succinctly on
something as complex, with as many diverse aspects, as climate change adaptation. At this point, we
urge caution in aggregating performance measure data, and we encourage offices to be clear about
any weighting, implicit or explicit, in their scoring or aggregation schemes
A Ensure data quality. All EPA offices will need to attend to fundamental principles of measurement
discussed in this report as they measure progress on climate change adaptation. These principles
include developing clear, consistent criteria for measurement, establishing a comprehensive data
collection process for the measures selected, and ensuring data accuracy through a central quality
assurance/quality control process.
Lessons Learned on Measurment for the Next Strategic Plan
The FY 2011-2015 EPA Strategic Plan identifies three high-level strategic measures that the Agency will
use to evaluate its progress in mainstreaming climate change into operations by 2015:
1. Integrate climate change adaptation into five rulemaking process;
2. Integrate considerations of climate change impacts and adaptive measures into five major grant,
loan, contract, or technical assistance programs; and
3. Integrate climate change trend and scenario information to five major scientific models or decision
support tools used to implement environmental management programs.
The OW plan reflects these goals in several ways. For example, in support of efforts to develop decision
support tools that incorporate climate change information, the NWP is deploying an upgraded version of
GREAT, as well as a comprehensive toolbox of water-related climate resources, to better assist water
and wastewater utilities in becoming more resilient to climate change. The NWP is also working to
incorporate climate change considerations in the development and implementation of a rulemaking by
2015. In addition, the NWP will help NEP grantees consider as a potential priority climate adaptation and
resilience in their Comprehensive Conservation and Management Plans s and develop climate
adaptation plans and implementation strategies where considered a priority.
88
-------
EC
The current Strategic Plan's emphasis is to mainstream climate change into operations by 2015; the
evaluation team believes the three measures of mainstreaming that the Agency adopted are clear,
concise, and measurable. In the next Strategic Plan, once EPA has demonstrated the ability to
mainstream climate change into operations, the current three measures will no longer be as relevant.
To develop new measures, we recommend that EPA engage in fundamental strategic planning, and
grapple with the particular value that the Agency can add with regard to climate change adaptation.
What role can EPA play that no other agency or private market actor can or will play? What role is EPA
especially able to play? EPA will need to consistently consider climate change adaptation in all of its
existing programs going forward, but the Agency will need to decide where to invest its effort in
concentrated new activities related to climate change adaptation. Using a logic model approach with a
particular focus on EPA's role and key external influences relative to key audiences and desired
outcomes may help the Agency focus in on where investment in climate change adaptation is most
warranted.
Given the priority actions identified in the Adaptation Plan, and EPA's key strengths and institutional
capabilities, three key areas of EPA expertise may include providing data, decision-support tools, and
training to partners related to climate change adaptation. Further work is needed to define a few
specific measures that are relevant across the agency, plan for data collection associated with these
measures, and determine the degree of EPA's contribution to outcomes achieved. For example, EPA
could consider measuring the percent (or number) of partners that have used EPA information to
integrate climate change adaptation into their existing training. This would require defining who the
partners are, and learning from the partners the extent to which they have integrated EPA climate
change adaptation partnership into their training. Such information may be relatively easy to collect if
EPA is working directly with the partners to develop the training. However, a concerted data collection
would be necessary if partners include organizations that EPA is not working with directly, but who
benefit from EPA information to inform their training efforts. EPA could also consider measuring the
percent (or number) of partners that incorporate EPA decision support tools into their planning
processes. This measure would involve similar considerations for defining partners and planning for
data collection. In some cases, e.g., the GREAT tool, EPA has worked collaboratively with partners to
develop climate change adaptation decision support tools and test using them in their planning. In
other cases, EPA has developed existing decision support tools that partners may benefit by using; in
these cases EPA would need to collect information from partners about how they are using these
decision support tools to plan for climate change resilience.
89
-------
lEc
CHAPTER 6 | CONCLUSIONS AND RECOMMENDATIONS
The 2008 and 2012 OW climate change strategies are important milestones in the continued evolution
of the NWP. The evaluators—and many of the people we interviewed as part of this evaluation-
anticipate that recognition of the value of these early efforts to understand and address challenges
posed by climate change will grow as real impacts to on the ground water resource management
become more apparent. The NWP's climate change strategy work is maturing, entering the seventh year
since the Climate Change Workgroup was launched in 2007. At the same time, continued resource and
staffing pressures and the lack of statutory drivers weaken the ability of climate change considerations
to compete for limited NWP time and attention.
In this context, our findings suggest that fresh approaches are needed to ensure robust implementation
of the 2012 Strategy. Our findings also suggest substantial limitations to a strategy implementation
approach based primarily on extracting annual commitments from programs and requiring output-based
reporting. More attention is needed in several critical areas: engaging management in meaningful ways,
creating time and space for strategic discussions, incentivizing and recognizing successes, harvesting and
analyzing lessons from implementation, forging buy-in to a longer-term measurement approach,
increasing the depth of knowledge of climate change implications for OW programs among relevant
staff, and relentlessly asking a set of key questions around climate change until they become a core
element of the NWP DNA. The shift that needs to be made now in OW can be compared to the shift that
OW had to make over the course of the last decade to adopting a watershed approach, although the
climate challenge is arguably more complex.
We are confident that OW will continue to make progress on climate change integration regardless of
how the 2012 Strategy is implemented. However, we believe that implementation of the 2012 Strategy
has potential to inspire, inform, and drive more progress faster if implementation is undertaken
carefully and deliberately. Our recommendations below are largely drawn from ideas provided by
managers and staff in the National Water Program. While some may seem simplistic, we believe that the
combined impact of these recommendations can set 2012 Strategy implementation on a different
trajectory with the potential to motivate more progress.
RECOMMENDATIONS
Reinvigorate NWP management and staff commitment to the Climate Change
Strategy.
1. Clarify the purpose of the 2012 Strategy. Clarifying the purpose of the 2012 Strategy, and
communicating that purpose within OW, is critical for informing strategy implementation,
identifying reporting and measurement needs, and understanding how measurement data will be
used and communicated. The evaluation team heard conflicting rationales for the 2008 and 2012
strategies; some OW staff view the strategies as encompassing only goals and activities that are
within EPA's purview, whereas others view the documents as more broadly encompassing goals and
activities that involve EPA's partners. Also, it is not clear to OW staff if the strategy is primarily an
internal planning document, or primarily a document for communicating OW's vision and goals to
external audiences. Notably, during interviews on implementation of the 2008 Strategy, staff
90
-------
lEc
indicated that the previous strategy was more successful as an external communications document
than as an internal planning document.
2. Clarify commitments and roles associated with the 2012 Strategy. Much of the current language in
the 2012 Strategy discussing coordination and collaboration with EPA's partners is vague. It is not
clear what EPA's specific roles or investment will be as distinct from partners. IO should clarify EPA
roles; this is essential for successful strategy implementation, as well as for measurement. Also, IO
needs to harmonize goals and strategic actions at the national and regional levels. While not all
national goals and strategic actions will apply to all regions, having two sets of goals and actions is
unwieldy and confusing.
3. Seek buy-in for the 2012 Strategy among OW management and staff. Lack of buy-in for both the
2008 and 2012 strategies is apparent at both the managerial and staff levels. The evaluation team
heard that managers rarely reference the strategy as part of day-to-day business, and that they do
not participate in the workgroup designed to coordinate implementation of the strategy. Staff
indicated that the strategy is viewed as an IO strategy, that priorities for staff time and funding are
not informed by the strategy, and that the strategy lacks a connection to daily work. Notably, some
staff interviewed indicated that they were not aware of strategic actions that had been officially
assigned to their purview; other staff indicated that they disagreed with the inclusion or wording of
certain strategic actions in the 2012 Strategy.
While there is no single magic bullet for attaining buy-in, a few key decisions could go a long way:
A OW should prioritize specific goals. IO should involve program office and regional staff in this
prioritization in a meaningful way. Resource decisions should be tied to the priorities.
A Managers should keep climate change as a front and center topic with their staff; they should
regularly discuss how the office's daily work relates to and is informed by climate change, and to the
climate change priorities selected as part of the strategic planning process. Managers should also re-
engage with the workgroup and attend meetings regularly instead of relying on designees
A OW should engage in open discussions with EPA's Office of Policy and Office of Administration and
Resources Management to develop a pilot initiative for Senior Executive Service (SES) candidates
and managers that encourages and supports the development of climate change "champions." This
pilot could provide focused assessment criteria and professional development guidance for
advancing climate change as a cross-cutting issue under the "leading change" SES assessment area.
Create management practices that keep climate change integration front and
center.
4. Schedule regular management-level strategic discussions adaptation and Strategy
implementation. OW should create more opportunities for meaningful strategic discussions among
management on climate change, its implications for water programs, and efforts to mainstream
climate change considerations into voluntary and regulatory programs. While much can be handled
at the staff level, there is no substitute for periodic management engagement on strategic topics. It
will be important to think carefully about the framing of management level discussions so they are
viewed as appropriate and to recognize that some issues may be relevant to some divisions, offices,
and programs but not others. An option for creating space is to designate one monthly climate
change workgroup meeting every 3 to 6 months for management-level discussions. Another
opportunity for strategic discussions is to include time for such discussions on other management
meeting agendas. Potential strategic discussions could include:
91
-------
EC
EXHIBIT 6-1. RECOMMENDATIONS SUMMARY
Reinvigorate NWP management and staff commitment to the Climate Change
Strategy.
1. Clarify the purpose of the 2012 Strategy.
2. Clarify commitments and roles associated with the 2012 Strategy.
3. Seek buy-in for the 2012 Strategy among OW management and staff.
Create management practices that keep climate change integration front and
center.
4. Schedule regular management-level strategic discussions adaptation and Strategy implementation.
5. Ask key climate change questions relentlessly up and down the management chain.
6. Shift the balance of implementation focus toward "customer service" and learning.
7. Recognize and reward climate change integration progress.
Empower EPA staff and state, tribal, and local partners.
8. Focus education and training support on connecting climate change to practical work.
9. Expand engagement on the strategy with State, Tribal, and local partners.
10. Attract and plan for resources.
Clarify the purpose of measurement and pilot a meaurement approach that
includes outcomes.
11. Seek buy-in for measuring progress on Strategy implementation.
12. Consider adopting an outputs plus priority outcomes measurement approach.
13. Ensure data quality and consistency in collecting measurement data.
14. Ensure data quality and consistency in collecting measurement data.
Given current resource constraints, how can we best incentivize progress on mainstreaming
climate change within voluntary and regulatory programs?
What type of measurement system can best support our efforts over the long-term to respond
to climate change? How can we build toward such a system in the near and mid-term?
What scenarios do we see for how climate change may impact local integrated water resource
management and governance? What do these scenarios mean for EPA water programs?
How might evolving scientific understanding and emerging climate change impacts (e.g., ocean
acidification) affect the NWP?
What strategic priorities or emergent opportunities are important to address?
What are we learning from climate change integration efforts to date?
What types of support, information, tools, and resources are needed to drive more rapid
progress in responding to climate change?
92
-------
EC
A suggested support tool for these discussions is a concise (less than 10 pages) adaptation of the 2012
Strategy Executive Summary to serve as a stand-alone resource to guide management discussions.
Soliciting input from managers on concise information that could be added to this document will make it
more useful to them.
5. Ask key climate change questions relentlessly up and down the management chain. One tried and
true approach to breathe life into strategy implementation efforts is to ensure that managers at the
top consistently ask a set of simple and clear questions. These questions keep the strategy present
and send clear signals that the topic is important and must be considered in routine program
planning and decisions. Questions could include:
• Have you considered climate change and its impacts (in your program, plans, analyses, or
decision process)?
• How is climate change likely to affect your program's ability to deliver results and meet goals?
How confident are you about this?
• What work is needed to improve understanding of how climate change will affect your program
or to integrate climate change considerations into your program? How can we help support
these efforts?
••"- What is your current understanding of the outstanding adaptation issues and needs, and the
opportunities to address them?
• Which of those opportunities do you think are the most important to address- do you have
resources to do so? How can we support these efforts?
6. Shift the balance of implementation focus toward "customer service" and learning. Much of the
strategy implementation process is currently viewed as an obligation by divisions, offices, and
branches in the NWP. Take steps (including those described in recommendations below) to ask
managers and staff in the NWP what they need and how IO can best support mainstreaming of
climate change across programs. Foster a culture of learning around the strategy; ask managers and
staff what they are learning from efforts underway. Consider periodic deployment of a web-based
survey to solicit information from NWP staff to understand the state of climate change awareness,
informational needs and questions, and staff-level perceptions of opportunities, accomplishments,
and lessons. Compile and communicate input received and consider ways to be responsive to them.
Set a clear tone that IO recognizes the challenges of integrating climate change but that the need
will only intensify and IO wants to support program offices on this journey.
7. Recognize and reward climate change integration progress. Many evaluation participants noted
that recognition and modest incentives can go a long way to inspire and encourage managers and
staff to take extra steps to advance climate change efforts. IO should build on the "highlights of
progress" efforts to showcase accomplishments and lessons, and consider modest opportunities to
capture and share successes within and external to EPA in newsletters, intranet postings, staff
meetings, and other venues. IO could consider incentives such as "climate champion" badges and
other forms of recognition. Although some of these ideas may sound frivolous, their cumulative
effect can be powerful and can motivate people to keep the strategy - and the goals and actions that
it encompasses - present in the workplace.
93
-------
lEc
Empower EPA staff and state, tribal, and local partners.
8. Focus education and training support on connecting climate change to practical work. While
general presentations and discussion of climate change and climate impacts can raise awareness,
they can also fall flat if they fail to help people answer the question: "What does this mean for me
and the work that I do?" More work is needed to help answer this question in divisions, offices,
branches and programs across the NWP. All hands meetings and training sessions have their
advantages, but they will miss key opportunities if they do not connect broader issues to practical
work. One way to explore these connections is to create space during webinars and presentations
for direct discussion on this question. Creating space for interaction can also provide insights into
managers' and staff members' questions about climate change. A periodic survey could support
these efforts. Shifting webinar formats to use real-time polling and facilitated peer exchange can
also make education, training, and communication efforts more relevant to participants. EPA's State
and Local Climate and Clean Energy Program in the Office of Air is a leader within the Agency in
using these approaches. OW should also work closely with the Office of Policy team supporting the
Agency-wide Climate Adaptation strategy in exploring innovative ways to conduct education,
training, and communications. Interaction may lead to identification of new adaptation
opportunities.
9. Expand engagement on the strategy with State, Tribal, and local partners. Interest in climate
change adaptation and mitigation appears to be growing at the state and local level. Even in
jurisdictions where there are political constraints to talking about climate change, local officials are
increasingly interested in responses to related issues such as extreme weather, ocean acidification,
and integrated water resource management needs. Preliminary evidence suggests that the 2008 and
2012 Strategies have been useful tools for engaging state, tribal, and local partners in discussions
and joint planning and projects to respond to climate change. There are significant opportunities to
expand these discussions and increase communications between NWP partners about activities,
accomplishments, lessons, needs, and challenges. Broader discussions on the response to climate
change with external partners can build on existing program and regional office relationships and
discussions. OW can encourage coordination with national partners such as the Association of State
Drinking Water Agencies and the Association of State Wetland Managers to leverage resources and
communication opportunities.
10. Attract and plan for resources. IO should explore opportunities to assemble even a modest reserve
of resources to support or seed priority climate change related projects. Programs could compete
for these resources on a periodic basis and applications could be selected by peers in the NWP.
Engage senior managers in discussions to identify creative opportunities to attract limited resources
to support high priority activities. IO could explore opportunities to expand the use of pilot projects
to test integration approaches with limited funds, and to capture and share lessons from existing
pilots and examine resource-efficient ways to expand successful approaches.
Clarify the purpose of measurement and pilot a meaurement approach that
includes outcomes.
11. Seek buy-in for measuring progress on Strategy implementation. Attaining buy-in for measuring
the outcomes of the 2012 Strategy is first predicated on attaining buy-in for the strategy itself, and
also on clarifying the purpose of the strategy. However, currently OW staff are skeptical about
94
-------
EC
measurement, and IO cannot implement a meaningful measurement effort without the support and
engagement of staff in the program offices and regions. IO should take several steps specific to
measurement to increase program office and regional support. IO should communicate with offices
and regions about what it plans to do with measurement data, and how data will be aggregated and
communicated. IO should provide assurances that measurement data will not be used as
justification for cutting budgets. Finally, if IO pursues an outcome approach, it should pursue
measurement primarily for a small set of outcomes, discussed below.
12. Consider adopting an outputs plus priority outcomes measurement approach. IO will need to
weigh the costs and benefits of potential measurement approaches. Measuring progress on
strategic actions is largely an output-based approach, and in and of itself, cannot measure progress
on the ultimate goal of fostering climate change adaptation on the ground. As such, the evaluation
team recommends pursuing the outputs plus priority outcomes approach. Specifically, we suggest
that IO pilot this approach with no more than a few priority outcomes given resource constraints.
To select outcomes for piloting an outputs plus priority outcomes approach, IO should consider
selecting outcomes with an existing mechanism for data collection, such as programs with a grant
component. IO may want to focus on more mature efforts, such as CRWU or CRE, where outcomes
are more likely be present. IO should develop a data collection and measurement plan in
collaboration with partners, such as the State/Tribal Council, and test the plan for a few years. At
the conclusion of the pilot, IO and its partners should assess progress, determine lessons learned,
and contemplate scaling up the approach to include more outcomes.
Focusing measurement on a few priority outcomes will serve to investigate proof of concept before
scaling up a measurement approach. Trying to implement a comprehensive outcome-based
approach from the outset may inadvertently add more confusion than insight; as noted by
interviewees, there is a real danger that overinvestment in measurement could result in loss of staff
support for tracking progress on climate change adaptation.
13. Within the measurement approach selected, acknowledge the iterative and evolving nature of
this work. The seven stages of adaptive management make sense conceptually, but as a practical
matter, EPA will need to revisit each stage on a regular basis. A more cyclical framework may
provide a better model, with the reach of EPA's activities expanding over time to a wider circle of
partners.
14. Ensure data quality and consistency in collecting measurement data. The baseline data OW
collected illustrate challenges of data consistency and quality that are important to address in any
measurement system. Moving forward, IO needs to develop data collection templates, instructions,
and a quality control plan that ensure high quality data. Key steps include:
J Clearly communicate the data collection plan to all stakeholders involved.
4 Identify specific data to be provided, including units, within data collection templates and
instructions. Also indicate the specific data to be provided by specific stakeholders. Do not use
open-ended questions to collect measurement data.
J Require data collection at regular interviews (e.g., annually), and on specific outputs and/or
outcomes previously agreed upon. IO should not accept reports of outputs or outcomes that
were not included in the measurement plan, at least not as a substitute for previously agreed
upon metrics.
95
-------
EC
J Clarify the time period for activities to be reported on. IO should not accept reports of activities
conducted and/or outcomes realized prior to the reporting period. Similarly, IO should not
accept descriptions of future activities to satisfy reporting requirements.
4 Consider developing an online reporting system to facility easier reporting, review, and storage
of measurement data.
4 Assign an individual or team within IO to review data submitted; check for completeness,
consistency, clarity, and adherence to reporting instructions. Follow up with reporting entities
for revisions where necessary.
96
-------
lEc
APPENDIX A | CONNECTIONS IN THE LOGIC MODELS
EXHIBIT A-1. LOGIC MODEL CONNECTIONS MATRIX: WETLANDS AND WATERSHEDS
Logic Model for the National Water Program 2012 Strategy: Response to Climate Change
© Vision
© Goal
®?rimary • ©Xudience • ©Audience |© Condi
Audiences I Awareness • Behavior
J Area 2
Watersheds 6
Wetlands:
Watersheds &
wetlands are
protected,
maintained,
restored, a
improved to
enhance
climate
resiliency ft
preserve the
functions 8
benefits they
provide
^Identify, protect,
& maintain
healthy
watersheds a
habitat corridor
networks
Incorporate
climate
resilience into
watershed
restoration 6
floodplain
management
Watershed
protection
practices
incorporate
Source Water
Protection to
protect drinking
water supplies
Incorporate
climate change
considerations
into CWA 404
program
EPA Sphere of
Direct Influence
Q) Improve baseline
information on
wetland extent,
condition, 8
performance
11 Develop
partnerships
Coordinate
stakeholders
i
I
3 EPA Core
Programs
2X)ther Federal
Agencies
ID States
d) Inter-state
P Tribes
D Local
Watershed
Managers
"^\
Lf Permitting
authorities
D Private
Landowners
j-' Water Sector
Utilities
p KGOs
c
(
(
(
y Awareness of
relationship
existing
programs (e.g.,
permitting &
funding), 2)
wetlands/
watershed
climate change
j) Understand
importance of
ftwatersheds to
climate change
resilience
L) Understand
extent ft
wetlands &
watersheds, &
how they may be
impacted by
climate change
Si
I/ Awareness of
tools a resources
to nrot*»rt
) Integrate
climate change
and healthy
watersheds/
wetlands
considerations
into core
programs, e.g.,
CWA 404 a
compensen-
tory mitigation
/ Implement
projects to
protect and/or
restore
watersheds,
wetlands, and
forests, or use
techniques
that are
climate
resilient
The Public
watersheds 6
wetlands
'Watersheds and
wetlands
continue to
provide
ecological
services, as
well as social
and economic
benefits, in
spite of climate
change
A-1
-------
EC
A-2
-------
lEc
EXHIBIT A-2. LOGIC MODEL CONNECTIONS MATRIX: COASTAL AND OCEAN WATERS
Logic Model for the National Water Program 2012 Strategy: Response to Climate Change
© Vision
© Goal
ID Primary
Audiences
3) Area 3
Coastal and
Ocean: Ocean
environment
protected
aeainst climate
(
) Support
collaborations
creating and
information and
best practices
(
y Foster
partnerships,
collaborations,
information
sharing
©EPA's NWP and
INEP programs
U.S. National
Ocean Council
(
) Understand
Strategies for
incorporating
adaptation into
t , i i * • f
federal policies a
programs
J Integrate
adaptation
considerations
into policies &
programs at
the federal
level
change and
against
unintended
adverse
consequences
of responses to
climate
change.
/ Develop
partnerships that
assist effective
adaptation action
for coastal and
ocean
environments
Provide
technical
assistance
F Regional ocean
organizations
Ensure that
mitigation and
adaptation
measures are
environmentally
sound
Promote best
practices for
climate-
readiness
planning
State and local
watershed
organizations
Coastal
communities
and planners
Adjust EPA
programs to
incorporate
shifting
environmental
conditions and
other emerging
threats
' Develop
climate-
readiness
guidance for
federal
programs,
agencies, and
authorities
t
Aware of
adaptation
options
Aware of
relevant partners
& opportunities
to collaborate/
share
information
Coastal
infrastructure
owners and
operators
EPA Sphere of
Direct Influence
Develop
environmental
safety criteria
for offshore
renewables
and CCS
j
f Mindful of the
potential hazards
that offshore
renewables and
CCS may pose to
coastal and
ocean resources
luftaas
t and rtal estate
trends
Weather, tidal, and climate conditions
Tech n ological advances (particu larly
CCS, renewobles, and IT)
Offshore navigational dredging
Renewable portfolio standards
Competing planning considerations
Emerging climate threats
•J Incorporate
climate change
& adaptation
considerations
into regional,
state & local
programs &
plans
v' Engage in
collaborative
partnerships
that ensure
information-
sharing and
prevent
duplication of
efforts
^/Adjust offshore
renewables
and CCS
permitting
criteria to
consider
adverse effects
to ocean &
coastal
resources
© Conditions
L/ Climate-change-
induced risks to
coastal and
ocean
ecosystems and
infrastructure
are minimized
Coastal and
ocean
environments
continue to
provide current
levels of
ecosystem
services and
socioeconomic
benefits
Coastal and
ocean
infrastructure
and ecosystems
are protected
against adverse
effects of climate
change
adaptation and
mitigation efforts
A-3
-------
EC
Primal
Audience:
Dl: Cl, C2, C3, C4,
C5
D2: Cl, C4, C5
D3: Cl, C2, C5
D4: Cl, C2, C3
E: Audience
Awarene
El: Dl, D2
E2: Dl, D2, D3, D4,
D5, D6
E3: Dl, D2, D3, D4,
D5
E4: Dl, D3, D4, D5
F: Audiei
Behavii
Fl: El, E2
F2: El, E2, E3
F3:E3
F4:E4
D5: Cl, C2, C3
D6: C2, C3
Conditions
Gl: Fl, F2, F3
G2: Fl, F2, F3,
F4
G3:F4
A-4
-------
lEc
EXHIBIT A-3. LOGIC MODEL CONNECTIONS MATRIX: WATER QUALITY
Logic Model for the National Water Program 2012 Strategy: Response to Climate Change*
[BM icjmmm^ [DJJ.I,,HI« rEffffffiBB
Vision
EPA Role
Audiences
l) Area 4
Water Quality:
Quality of
surf ace water,
drinking
water, a
ground water
is protected ft
climate change
induced risks
to human
health and the
environment
are diminished
via adaptation
a mitigation
strategies
EPA Sphere of
Direct Influence
^Protect waters
of the US a
promote
sustainable
management
of surface
water
resources
^Protect water
resources from
unintended
consequences
of national
reduce green
house gas
emissions a
develop
alternative
energy
[/Collaborate to
make
hydrological
and climate
HatA A
projections
available
Q Encourage
integration of
climate change
consideration
into existing
water quality
protection
measures
^^ Promote best
practices
(^ Communicate
key issues
I I
Q Develop
guidance a
provide tech.
decision
support
CSj Coordinate
stakeholders
1
^ Provide policy
clarification
^^
^Support data ft
new analytic
method
development
Other EPA
offices
Additional Influences on Primary Audiences
- Federal, state, a local . Weather a climate
government conditions
requirements • Economic considerations
States
Inter-state
Agencies
Tribes
Local
Watershed
Managers
Permitting
Authorities
3ther Federal
Agencies
Awareness
/ Aware of
potential for
existing
programs &
resources to
address
climate change
concerns
2J Understand
relationship
between best
practices (e.g.,
Gl 8 LID) 8
climate change
mitigation/
adaptation
3'Aware of tools
a resources to
protect water
quality
/ Understand
potential
effects of
climate change
on current
water quality
standards
Behavior
nditions
lj Integrate
climate change
considerations
into federal,
state, & local
water quality
decisions
) Update
hydrological
data a enhance
statistical
methods to
account for
changing
climate
conditions
1J Climate change
induced risks to
human health ft
the environment
are minimized
f Surf ace water,
ground water, 8t
drinking water
quality remains
protected
•For a complete articulation of EPA's
visions, goals, a strategic actions related
to the National Water Program's
Response to Climate Change, see the
2012 Strategic Plan at
http://water.epa.gov/scitech/dimatech
ange/2012-National-Water-Program-
Strategy, cfm
A-5
-------
EC
C1:B1,B3
Primal
Dl: C2, C4, C6,
C7
E: Audience
Awareness
E1:D1,D2, D3, D4,
D5, D6
: Audience
ehavior
Fl: El, E2, E3, E4
Conditions
G1:F1,F2
B3: Al
C2:B1,B2
D2: Cl, C2, C3,
C4, C6, C7
E2: D2, D4, D5, D6,
D7
F2: E2, E3, E4
G2: Fl, F2
C3:B1,B2
D3:C1
E3: D2, D4, D5, D6
C4:B1,B2
D4: Cl, C2, C3,
C4, C6, C7
E4: Dl, D2, D3, D4,
D5, D6, D7
C5:B1,B3
D5: Cl, C2, C4,
C7
C6:B1,B2
D6: Cl, C2, C4,
C7
C7:B1,B2, B3
D7: Cl, C2, C4,
C7
A-6
-------
lEc
EXHIBIT A-4. LOGIC MODEL CONNECTIONS MATRIX: WORKING WITH TRIBES
Logic Model for the National Water Program 2012 Strategy: Response to Climate Change"
Vision
A Role
-------
EC
D: Primary
Audiences
Dl: Cl, C2, C3,
C4, C5
D2:C2
D3: Cl, C3
D4: C3, C5
E: Audience
Awareness
E1:D1,D3, D4
F: Audience
Behavior
E2: Dl, D2, D3,
D4
F1:E1,E2
F2:E1
Conditions
G1:F1,F2
A-8
-------
EC
APPENDIX B: SUMMARY OF STRATEGIC ACTION BASELINE DATA
STRATEGIC ACTION
SA 1: Improve access to vetted climate and hydrological science,
modeling, and assessment tools through the Climate Ready Water
Utilities program.
SA 2: Assist wastewater and water utilities to reduce greenhouse
gas emissions and increase long-term sustainability with a
combination of energy efficiency, co-generation, and increased
use of renewable energy resources.
SA 3: Work with the states and public water systems, particularly
small water systems, to identify and plan for climate change
challenges to drinking water safety and to assist in meeting health
based drinking water standards.
SA 4: Promote sustainable design approaches to provide for the
long-term sustainability of infrastructure and operations.
SA 5: Understand and promote through technical assistance the
use of water supply management strategies.
SA 6: Evaluate and provide technical assistance on the use of
water demand management strategies.
SA 7: Increase cross-sector knowledge of water supply climate
challenges and develop watershed specific information to inform
decision making.
SA 8: Develop a national framework and support efforts to protect
remaining healthy watersheds and aquatic ecosystems.
SA 9: Collaborate with partners on terrestrial ecosystems and
hydrology so that effects on water quality and aquatic ecosystems
are considered.
SA 10: Integrate protection of healthy watersheds throughout the
NWP core programs.
SA 1 1 : Increase public awareness of the role and importance of
healthy watersheds in reducing the impacts of climate change.
SA 12: Consider a means of accounting for climate change in EPA
funded and other watershed restoration projects.
SA 13: Work with federal, state, interstate, tribal, and local
partners to protect and restore the natural resources and
functions of riverine and coastal floodplains as a means of building
resiliency and protecting water quality.
SA 14: Encourage states to update their source water
delineations, assessments or protection plans to address
anticipated climate change impacts.
SA 15: Continue to support collaborative efforts to increase state
and local awareness of source water protection needs and
opportunities, and encourage inclusion of source water protection
areas in local climate change adaptation initiatives.
SA 16: Consider the effects of climate change, as appropriate,
when making significant degradation determinations in the CWA
Section 404 wetlands permitting and enforcement program.
SA 17: Evaluate, in conjunction with the U.S. Army Corps of
Engineers, how wetland and stream compensation projects could
be selected, designed, and sited to aid in reducing the effects of
climate change.
SA 18: Expand wetland mapping by supporting wetland.
SA 19: Produce a statistically valid, ecological condition
assessment of the nation's wetlands.
SA 20: Work with partners and stakeholders to develop
information and tools to support long term planning and priority
setting for wetland restoration projects.
COUNT OF
ACTIVITIES
3
8
4
4
4
5
3
1
2
1
2
3
1
2
2
1
2
1
1
3
REGION/OFFICE
Region 3, Region 6, OGWDW
Region 1 , Region 4, Region 5,
Region 6, Region 7, Region 8,
Region 9, OWM
Region 1, Region 3, Region 7,
OGWDW
Region 4, Region 6, Region 7, OWM
Region 3, Region 5, Region 6, OWM
Region 5, Region 6, Region 8, OWM
Region 3, Region 6, Region 8,
OWOW/AWPD
Region 4, OWOW/AWPD
OWOW/AWPD
Region 4, OWOW/AWPD
Region 5, Region 7, OWOW/AWPD
OWOW/AWPD
Region 7, OGWDW
Region 1, OGWDW
OWOW
Region 7, OWOW
OWOW
OWOW
Region 1, Region 4, OWOW
B-1
-------
EC
STRATEGIC ACTION
SA 21 : Collaborate to ensure that synergy occurs, lessons learned
are transferred, federal efforts effectively help local communities,
and efforts are not duplicative or at cross-purposes.
SA 22: Work within EPA and with the U.S. Global Change Research
Program and other federal, tribal, and state agencies to collect,
produce, analyze, and format knowledge and information needed
to protect ocean and coastal areas and make it easily available.
SA 23: Work with the NWP's larger geographic programs to
incorporate climate change considerations, focusing on both the
natural and built environments.
SA 24: Address climate change adaptation and build stakeholder
capacity when implementing NEP Comprehensive Conservation and
Management Plans and through the Climate Ready Estuaries
Program.
SA 25: Conduct outreach and education, and provide technical
assistance to state and local watershed organizations and
communities to build adaptive capacity in coastal areas outside
the NEP and Large Aquatic Ecosystem programs.
SA 26: Support coastal wastewater, stormwater, and drinking
water infrastructure owners and operators in reducing climate
risks and encourage adaptation in coastal areas.
SA 27: Support climate readiness of coastal communities,
including hazard mitigation, pre-disaster planning, preparedness,
and recovery efforts.
SA 28: Support preparation and response planning for diverse
impacts to coastal aquatic environments.
SA 29: Consider climate change impacts on marine water quality
in NWP ocean management authorities, policies, and programs.
SA 30: Use available authorities and work with the Regional Ocean
Organizations and other federal and state agencies through
regional ocean groups and other networks so that offshore
renewable energy production does not adversely affect the marine
environment.
SA 31 : Support the evaluation of sub-seabed sequestration of C02
and any proposals for ocean fertilization.
SA 32: Participate in interagency development and
implementation of federal strategies through the NOC and the NOC
Strategic Action Plans.
SA 33: Encourage states and communities to incorporate climate
change considerations into their water quality planning.
SA 34: Encourage green infrastructure and low-impact
development to protect water quality and make watersheds more
resilient.
SA 35: Promote consideration of climate change impacts by
National Pollutant Discharge Elimination System permitting
authorities.
SA 36: Encourage water quality authorities to consider climate
change impacts when developing wasteload and load allocations in
TMDLs where appropriate.
SA 37: Identify and protect designated uses that are at risk from
climate change impacts.
SA 38: Clarify how to re-evaluate aquatic life water quality
criteria on more regular intervals; and develop information to
assist states and tribes who are developing criteria that
incorporate climate change considerations for hydrologic
condition.
SA 39: Continue to provide perspective on the water resource
implications of new energy technologies.
SA 40: Provide assistance to states and permittees to assure that
geologic sequestration of C02 is responsibly managed.
COUNT OF
ACTIVITIES
1
2
3
3
3
2
3
1
1
1
1
1
2
3
2
2
2
1
1
3
REGION/OFFICE
owow
Region 1, OWOW
Region 1, Region 10, OWOW
Region 1, Region 4, OWOW
Region 4, OWOW, GLNPO
Region 10, OWOW
Region 1, Region 10, OWOW
OWOW
OWOW
OWOW
OWOW
OWOW
Region 5, OWOW/AWPD
Region 1, Region 8, OWM
Region 5, OWM
Region 10, OWOW/AWPD
OW/OST
OW/OST
Baseline data submitted but
reporting office not identified
Region 4, OGWDW
B-2
-------
EC
STRATEGIC ACTION
SA 41 : Continue to work with States to help them identify polluted
waters, including those affected by biofuels production, and help
them develop and implement Total Maximum Daily Loads (TMDLs)
for those waters.
SA 42: Provide informational materials for stakeholders to
encourage the consideration of alternative sources of energy and
fuels that are water efficient and maintain water quality.
SA 43: As climate change affects the operation or placement of
reservoirs, EPA will work with other federal agencies and EPA
programs to understand the combined effects of climate change
and hydropower on flows, water temperature, and water quality.
SA 44: Monitor climate change impacts to surface waters and
ground water.
SA 45: Collaborate with other federal agencies to develop new
methods for use of updated precipitation, storm frequency, and
observational streamflow data, as well as methods for evaluating
projected changes in low flow conditions.
SA 46: Enhance flow estimation using National Hydrography
Dataset Plus (NHDPlus).
SA 47: Through formal consultation and other mechanisms,
incorporate climate change as a key consideration in the revised
NWP Tribal strategy and subsequent implementation of CWA,
SDWA, and other core programs.
SA 48: Incorporate adaptation into tribal funding mechanisms, and
collaborate with other EPA and federal funding programs to
support sustainability and adaptation in tribal communities.
SA 49: Collaborate to explore and develop climate change
science, information, and tools for tribes, and incorporate local
knowledge.
SA 50: Collaborate to develop communication materials relevant
for tribal uses and tribal audiences.
SA 51: Continue building the communication, collaboration, and
training mechanisms needed to effectively increase adaptive
capacity at the federal, tribal, state, and local levels.
SA 52: Adopt a phased approach to track programmatic progress
towards strategic actions; achieve commitments reflected in the
Agency's Strategic Plan; work with the EPA workgroup to develop
outcome measures.
SA 53: Work with EPA's Office of Research and Development,
other water science agencies, and the water research community
to further define needs and develop research opportunities to
deliver the information needed to support implementation of this
2012 Strategy, including providing the decision support tools
needed by water resource managers.
COUNT OF
ACTIVITIES
0
2
1
0
0
1
1
1
1
0
0
0
1
REGION/OFFICE
No baseline data submitted
Region 6, OWM
OWM
No baseline data submitted
No baseline data submitted
OWOW/AWPD
Region 1
Region 5
Region 10
No baseline data submitted
No baseline data submitted
No baseline data submitted
IO/OST
B-3
-------
EC
APPENDIX C: SUMMARY OF REPORTS AND ONLINE RESOURCES THAT DESCRIBE
PROGRESS ON CLIMATE CHANGE ADAPTATION
EPA Resources
• Resource/Report title: Climate Ready Estuary Projects—Where you live—2008 - 2012
^ Author: EPA
A Link: http://water.epa.gov/type/oceb/cre/live.cfm
http://water.epa.gov/type/oceb/cre/upload/CRE 2011Report RiskManagementPu
llout 508.pdf
A Partners included: EPA Climate Ready Estuaries program, various states and
communities
A Description: This webpage summarizes and links to Climate Ready Estuary projects in
each EPA region. Many of the projects pertain to analysing coastal watersheds for
climate vulnerability. However, a large portion of the projects are also intended to
improve coastal resilience to climate change. For example, a study by the New York and
New Jersey Harbor Estuary Program is examining adaptation options for the sewage
authority. Likewise, the Lower Columbia River Estuary Partnership is working to
incorporate climate considerations into the estuary program's Comprehensive
Conservation and Management Plan to guide adaptation efforts. In all, the site contains
36 project summaries, and each one links to an external website for the project or the
project partners. A separate PDF report (see second link above) categorizes these
projects according to the ISO 31000 Risk Management Framework.
A Updates: Frequency of updates depends on the particular project
x Applicable areas: Infrastructure, Wetlands and Watersheds
• Resource/Report title: GREAT analytics data
^ Author: EPA
A Link: http://water.epa.gov/infrastructure/watersecurity/climate/creat.cfm
•^ Partners included: Water and wastewater utilities
A Description: GREAT stands for Climate Resilience Evaluation and Awareness Tool. It is a
software tool designed to assist drinking water and wastewater utility owners and
operators in understanding potential climate change impacts and in assessing the
related risks at their utilities. Users download and register for the software via the EPA
website. Analytic data that tracks the number of downloads could, along with
registration data, be used to assess the tool's market penetration in the water utility
industry.
A Updates: It is unclear how often EPA updates the GREAT software or the analytics
programs that track downloads of GREAT
A Applicable areas: Infrastructure, Water Quality
• Resource/Report title: EPA Coastal Area Impacts and Adaptation
^ Author: EPA
A Link: http://www.epa.gov/climatechange/impacts-adaptation/coasts.htmltfadapt
^ Partners included: Links to other federal agencies
C-1
-------
lEc
A Description: This EPA website is a clearinghouse for information on coastal climate
impacts and the adaptation approaches being taken by in states and communities
around the country. In addition to discussions of how climate change will affect coastal
communities and ecosystems, this website contains examples of EPA, interagency, and
state adaptation efforts. Examples include state and federal efforts to map sea level rise,
state regulations that require construction contractors to consider climate effects, and
an online forum for communities to share information about adaptation. The site
includes subpages that report on region-specific problems and adaptation efforts. For
example, the Northwest Impacts and Adaptation subpage features the summary
Olympia, Washington, Plans for Sea Level Rise.
^ Updates: Website last updated June 14, 2012; it is not clear whether the website is
regularly updated
A Applicable areas: Infrastructure, Wetlands and Watersheds.
• Resource/Report title: EPA's Water Infrastructure Website
^ Author: EPA
A Link: http://water.epa.gov/infrastructure/
A Partners included: States, municipalities, water utilities
A Description: EPA's water infrastructure website is a clearinghouse of resources on
climate adaptation in the water supply and wastewater management sector. It contains
links to surveys and case studies about what specific utilities and other infrastructure
systems are doing to adapt to climate change, as well as more generic tools and
resources designed to provide technical assistance to infrastructure stakeholders.
A Updates: Website was last updated December 5, 2012; it is not clear whether the
website is regularly updated
A Applicable areas: Wetlands and watersheds
• Resource/Report title: Workshop Proceedings on Water Infrastructure Sustainability and
Adaptation to Climate Change
^ Author: EPA
A Link: http://www.epa.gov/nrmrl/wswrd/wq/wrap/workshop.html
A Partners included: Various states, cities, and infrastructure owners and operators
A Description: These proceedings include links to workshop materials created by and/or
for water infrastructure owners and operators. Many of the materials contain case
studies on successful capital investments for adapting water utilities to climate change.
Examples include presentations about water supply adaptation efforts in Boston, water
availability forecasting in South Florida, and preparations for climate change among
water utility operators in East Bay, California. More general materials include abstract
tips and strategies for adapting municipal water systems to climate change.
A Updates: It is unlikely that the proceedings would be updated
A Applicable areas: Wetlands and Watersheds
Other Federal Government Resources
Report title: Addressing Climate Change in Long-Term Water Resources Planning and Management
A Report author: Department of the Interior (Bureau of Reclamation) and U.S. Army Corps
of Engineers
C-2
-------
EC
A Report link: http://www.usbr.gov/climate/userneeds/
A Partners included: Many (Fed., State, Local gov't agencies, private industries, and non-
profits)
A Description: The report describes the need for information and tools to support long-
term climate change planning efforts by various water management agencies (Fed.,
State, and Local level). The findings are intended to identify tool and information gaps
and in turn, focus future research efforts. The gaps are categorized into eight technical
steps by which findings are summarized. The eight steps are as follows:
o Summarize relevant literature
o Obtain climate change information
o Make decisions about how to use climate change information
o Assess natural systems response
o Assess socioeconomic and institutional response
o Assess system risks and evaluate alternatives
o Assess and characterize uncertainties
o Communicating results and uncertainties to decision makers
A Updates: Report was last updated on 4/24/2011; it is unclear if a future update is
planned.
A Applicable areas: Infrastructure
• Report title: National Climate Assessment, Adaptation Section (draft 2013)
A Report author: National Climate Assessment and Development Advisory Committee
A Report Link: http://ncadac.globalchange.gov/download/NCAJanll-2013-
publicreviewdraft-chap28-adaptation.pdf
A Partners included: Many (Fed., State, Local gov't agencies, private industries, and non-
profits)
A Description: Report summarizes the latest information on climate change adaptation
efforts across the US. The report concludes that "Substantial adaptation planning is
occurring in the public and private sectors and at 20 all levels of government, however,
few measures have been implemented and those 21 that have appear to be incremental
changes." The report finds that, "Most adaptation efforts to date have occurred at local
and regional levels." A few of the examples of climate change adaptation provided
relate to wetlands and watersheds. For example, the report states that the city of
Groton, Connecticut, "Partnered with federal, state, regional, local, non-governmental,
and academic partners through the EPA's Climate Ready Estuaries program to assess
vulnerability to and devise solutions for sea level rise." The report finds that the general
stages in the adaptation process include 1) understanding risks and vulnerabilities, 2)
planning, assessing, and selecting options, 3) implementation, 4) monitoring and
evaluation, and 5) revising strategy and research and sharing lessons learned. These
steps can occur simultaneously, in a different order, or may be omitted completely.
Evaluation and monitoring efforts to date and focused on process-based rather than
outcome indicators, however efforts are underway to develop outcome-based
indicators (including efforts at EPA).
A Updates: Every four years
A Applicable areas: All areas
C-3
-------
EC
• Resource/Report title: NOAA Coastal Climate Adaptation and Action Plans
^ Author: NOAA
^ Link:
http://collaborate.csc.noaa.gov/climateadaptation/Lists/Resources/AdaptationAction%
20Plans.aspx
•*• Partners included: States and municipalities
A Description: This clearinghouse of adaptation plans contains reports and action plans
created by states and cities to plan and document their adaptation efforts. The website
characterizes each plan according to the sectors and impacts discussed. Several dozen
documents are characterized and linked to. An example is the San Diego Bay Sea Level
Rise Adaptation strategy, which lays out a comprehensive vulnerability assessment and
a series of broad recommendations for building the resilience of community assets.
A Updates: Latest plan added on March 27, 2013; new plans are added regularly
•*• Applicable areas: Infrastructure, Wetlands and Watersheds, Water Quality, Tribes.
State and Local Resources
Report title: Climate Change Adaptation for Maryland Water Utilities
A Report author: Maryland Depart of Environment's Water Supply Program
•*• Report link:
http://www.mde.state.md.us/programs/Water/Water Supply/Documents/120516 CCb
rochure Web.pdf
A Partners included: NA
A Description: This report is intended to help local water utilities in Maryland plan and
prepare for the impacts of climate change. The report lays out potential impacts to
drinking water along with guidelines to choosing appropriate adaptation measures.
Recommended adaptation measures to address changes to water quality are as follows:
o Acquire and manage forested/vegetated lands
o Improve monitoring
o Increase treatment capability
o Green infrastructure
A Updates: Updated on 9/21/2012, it is not clear whether the report is regularly updated.
A Applicable areas: infrastructure
• Report title: Climate Change Handbook for Regional Water Planning
A Report author: California DWR, US Army Corps of Engineers, and US EPA Region 9
A Report link:
http://www.water.ca.gov/climatechange/docs/Climate Change Handbook Regional
Water Planning.pdf
A Partners included: Many (Fed., State, Local gov't agencies, private industries, and non-
profits)
A Description: This report is a handbook that lays out an analytical framework for
incorporating climate change impacts into regional and watershed level planning
processes. The handbook focuses on the California Integrated Regional Water
Management Planning but is intended to inform any water planning process. The
handbook addresses adaptation and mitigation techniques and describes various
decision support tools and techniques that water planning agencies can use. The use of
C-4
-------
EC
these tools and techniques are illustrated through case studies. Additionally, Section 6
of the handbook discusses evaluation of climate change adaptation and mitigation
programs. It stresses the need for having quantifiable metrics in order to measure
performance.
A Updates: Prepared in Nov. of 2011 it does not appear the report is regularly updated.
A Applicable areas: Infrastructure, Coastal and Ocean Waters, Wetlands and Watersheds
Report title: Managing an Uncertain Future - Climate Change Adaptation Strategies for California's
Water
•*• Report author: California DWR
•*• Report link:
http://www.water.ca.gov/climatechange/docs/ClimateChangeWhitePaper.pdf
A Partners included: NA
A Description: The report recommends ten capacity building climate change adaptation
strategies for state and local water managers. The recommended strategies are very
specific, laying out precise actions that the State of California should take. Some of the
strategies involve immediate actions and others require further planning and
consultation.
A Updates: Prepared in 2008, it does not appear to be regularly updated
A Applicable areas: Infrastructure, Wetlands and Watersheds, Coastal and Ocean Waters
Report title: Adapting to Climate Change in Minnesota: Preliminary Report of the Interagency
Climate Adaptation Team
A Report author: Minnesota Pollution Control Agency
A Report link: http://www.pca.state.mn.us/index.php/view-document.html?gid=15414
^ Partners included: MN DOA, DOC, DOH, DNR, Public Safety and Transportation
A Description: This report outlines anticipated climate change effects on the State of
Minnesota. It also describes various state agencies' current climate change adaptation
efforts. With regard to water quality, the Minnesota Pollution Control Agency (MPCA)
has conducted the following activities:
o Initiated a watershed-focused monitoring design to help provide better spatial
resolution of stressors.
o Initiated a statewide random monitoring program to develop long-term trends
on a large scale.
o Integrate biological, physical and chemical monitoring to develop relationships
between environmental factors (including climate) and aquatic life.
o Developed biological indices for fish and invertebrates to measure response.
o Developed approaches to reduce storm-water runoff through low impact
development.
A Updates: Prepared in August of 2010, it does not appear to be regularly updated.
A Applicable areas: Infrastructure
C-5
-------
EC
Report title: The Oregon Climate Change Adaptation Framework
A Report author: Oregon DOE
A Report link:
http://www.oregon.gov/ENERGY/GBLWRM/docs/Framework Final DLCD.pdf
•^ Partners included: Numerous state agencies including: Oregon DOA, DOE, DEQ, DFW,
and many others...
A Description: The report describes a framework for the development of strategies and
plans to address climate change impacts in the State of Oregon. The framework
discusses Oregon's climate change risks, adaptive capacity, and short priority actions
and a long term process to build Oregon's adaptive capacity. The report does not focus
specifically on water quality but it is addressed in various portions of the report.
A Updates: Prepared in December of 2010, it does not appear to be regularly updated.
•*• Applicable areas: Infrastructure, Wetlands and Watersheds, Coastal and Ocean Waters
• Website title: State and Local Adaptation Plans
•*• Website author: Georgetown Climate Center
A Website link: http://www.georgetownclimate.org/adaptation/state-and-local-plans
•*• Partners included: State and local governments
A Description: This website provides links to state and local adaptation plans.
A Updates: Website appears to be continually updated; recent plans from December 2012
are included.
A Applicable areas: All areas
Other Resources
• Report title: Adaptation Planning - What U.S. States and Localities are Doing (2009)
A Report author: Pew Center for Global Climate Change/Center for Climate and Energy
Solutions
A Report link: http://www.c2es.org/publications/state-local-adaptation-planning
•*• Partners included: States and localities
A Description: This report describes state and local climate change adaptation planning
efforts, with links to more information.
A Updates: Prepared in 2009, it does not appear that the report is regularly updated
A Applicable areas: All areas
• Report title: Climate Change Adaptation Plan for Coastal and Inland Wetlands in the State of
Michigan (2012) (Included as an example of a state adaptation plan specific to wetlands)
*• Report author: The Association of State Wetland Managers, Inc.
A Report link: http://aswm.org/wetland-science/climate-change/climate-change-
adaptation/3214-climate-change-adaptation-plan-for-coastal-a-inland-wetlands-in-the-
state-of-michigan
A Partners included: Michigan Department of Environmental Quality Wetlands Program
and Coastal Management Program
A Description: The report describes how wetlands are a tool to help adapt to climate
change, identifies wetland adaptation planning in other states, and provides
recommended actions for Michigan related to wetlands adaptation.
C-6
-------
EC
A Updates: The report was prepared in September 2012 in draft format; a partnership of
state, local, and regional agencies and groups will need to review the recommendations
reach agreement on specific actions that can be taken to ensure that wetland resources
are included in broader strategies to address and adapt to climate change. It does not
appear there are currently plans to update the report.
A Applicable areas: Wetlands and Watersheds, Coastal and Ocean Waters
• Report title: Climate Change Adaptation: What Federal Agencies Are Doing (2012)
•*• Report author: Center for Climate and Energy Solutions
A Report link: http://www.c2es.org/publications/climate-change-adaptation-what-
federal-agencies-are-doing
•*• Partners included: US Federal Agencies
A Description: For each Department, the report highlights specific adaptation initiatives
(such as a program office or strategic plan) that are in place at the Department level.
These are followed by an overview of each agency or bureau within that Department
and relevant adaptation activities, which are typically divided into: (1) initiatives and
strategies, (2) programs and institutional mechanisms, and (3) tools and resources. This
January 2012 update also includes several examples of federal projects that incorporate
the impacts of climate change and adaptive actions into the planning, design, and
implementation process. These projects further highlight the leadership role federal
departments and agencies are taking to promote a more climate resilient economy,
society, and environment.
A Updates: the report was originally developed in 2010, and updated in 2012.
A Applicable areas: All areas
• Report title: State Wetland Climate Change Adaptation Summaries (2010)
A Report author: Association of State Wetland Managers
A Report link: http://aswm.org/wetland-science/climate-change/climate-change-
adaptation/1200-climate-change-adaptation-summaries
^ Partners included: States
A Description: This web-based report describes states that have respond to climate
change through adaptation planning, studies, and monitoring. The report describes the
status of the activity (e.g., in progress or completed) and whether adaptation planning
includes wetlands. Each example includes a brief description and links to websites
and/or contacts for more information.
A Updates: Prepared in 2010, it does not appear that the report is regularly updated.
A Applicable areas: All areas
• Website title: StormSmart Coasts
A Author: StormSmart Coasts Network
A Link: http://stormsmartcoasts.org/
•^ Partners included: EPA, NOAA, Gulf of Mexico Alliance, Northeast Regional Ocean
Council
A Description: StormSmart Coasts is "a resource for coastal decision makers looking for
the latest and best information on how to protect their communities from weather and
climate hazards." While the website contains some information that may not be strictly
considered adaptation (e.g., best practices for assessing storm damage), the majority of
C-7
-------
EC
the site is dedicated to sharing information on activities more traditionally considered
under the banner of adaptation, including analysing sea level, flood, and storm
vulnerability; creating community, hazard, and adaptation plans; and drafting new
building and siting codes that account for climate change. StormSmart has individual
websites for the following states: Alabama, Connecticut, Delaware, Florida, Louisiana,
Maine, Massachusetts, Mississippi, New Hampshire, Rhode Island, and Texas. Some of
the site's offerings differ among states. For example, the Texas site includes a "tools"
section, which links to tools such as a vulnerability atlas of the Texas coast, while the
Florida site does not. This tool and others could provide some data on adaptation
efforts.
A Updates: The website is updated as new states join the network and add adaptation
information, but a regular update schedule is unavailable
A Applicable areas: Infrastructure
• Report title: Swinomish Climate Change initiative - Climate Adaptation Action Plan
A Report author: Swinomish Indian Tribal Community: Office of Planning and Community
Development
A Report link: http://www.swinomish-
nsn.gov/climate change/Docs/SITC CC AdaptationActionPlan complete.pdf
A Partners included: University of Washington, Center for Science in the Earth System,
Climate Impacts Group
A Description: This report provides a comprehensive picture of the threats that climate
change poses to the Swinomish Tribe, identifies specific adaptation/mitigation actions
that the Tribe can take, and assesses their ability to fulfil these actions in a specified
time frame. Additionally, the report uses parameters to prioritize each of the
recommended actions. The report was written in 2010 and is part of the Swinomish
Tribe's on-going climate change adaptation effort.
A Updates: Prepared in October of 2010, it does not appear to be regularly updated.
A Applicable areas: Working with Tribes
• Resource/Report title: The Role of Coastal Zone Management Programs in Adaptation to
Climate Change
A Author: Coastal States Organization
^ Link: http://www.coastalstates.org/wp-content/uploads/2010/07/CSO-2008-Climate-
Change-Report2.pdf
^ Partners included: Coastal states
A Description: This report explores state coastal program's climate change initiatives, as
well as states' assessments about national policy needs. It is the end result of a survey of
30 U.S. coastal states and territories about climate change adaptation efforts and needs.
The data in the report constitute an attempt to rank coastal climate change issues,
assess work done to date, and estimate a cost associated with state adaptation
activities. A results section toward the end of the report documents about a dozen
adaptation strategies and actions being undertaken in coastal states around the nation.
A Updates: While this report will not be updated, additional reports will likely be released
in the future
•*• Applicable areas: Infrastructure, Wetlands and Watersheds, Water Quality
C-8
-------
EC
• Website title: Tribal Climate Change Project
•*• Website author: University of Oregon and the USDA Forest Service Pacific Northwest
Research Station
•*• Website link: http://tribalclimate.uoregon.edu/
•*• Partners included: University of Oregon, Environmental Studies Program and the USDA
Forest Service Pacific Northwest Research Station
A Description: This website provides information on the Tribal Climate Change Project,
part of the USDA Forest Service's 2010 Coordinated Approach to Tribal Climate Change
Research. The project addresses the following key research areas:
o Tribal adaptation and mitigation planning
o Management of off-reservation resources
o Tribal consultation in the context of climate change
A Updates: Website appears to be continually updated as projects are developed
A Applicable areas: Working with Tribes
• Website title: Tribes & Climate Change
A Website author: Institute for Tribal Environmental Professionals
A Website link: http://www4.nau.edu/tribalclimatechange/tribes/index.asp
^ Partners included: Northern Arizona University
A Description: This website contains profiles of various Tribes and describes the potential
impacts climate change may have on them. Additionally, the site provides links to
numerous resources about climate change that Tribes can use to help with adaptation
and mitigation efforts. The resources include tools, reports, funding opportunities, and
information about traditional tribal knowledge related to climate change.
A Updates: Site was last updated 8/27/2012, there does not appear to be a set schedule
for updates.
A Applicable areas: Working with Tribes
• Report title: Yurok Tribe and Climate Change: An Initial Prioritization Plan
A Report author: Kathleen Sloan and Joe Hostler-Yurok Tribe Environmental Program
•*• Report link:
http://www.yuroktribe.org/departments/vtep/documents/YurokTribeandClimateChang
ePrioritizationPlan.pdf
•*• Partners included: US EPA Environmental Justice Small Grants Program
A Description: The report provides a preliminary assessment of available climate change
information and the potential impacts to the "Yurok People, Resources, and Lifeways."
The report is intended to provide information to tribal decision-makers and help guide
climate change research and planning efforts. The report focuses on the first two steps
of a more robust plan:
o Communicate and consult
o Monitor and review
A Updates: Prepared in September of 2011, it does not appear to be regularly updated.
A Applicable areas: Working with Tribes
C-9
-------
EC
APPENDIX D: CATEGORIES OF STRATEGIC ACTIONS
STRATEGIC ACTION
POLICY
AND/OR
GUIDANCE
CHANGE
FINANCIAL
INCENTIVE
TECHNICAL
RESOURCES /
DATA
DEVELOPMENT
TECHNICAL
ASSISTANCE /
TRAINING
FOSTERING
PARTNERSHIPS/
COLLABORATION
Infrastructure
1: Improve access to vetted
climate and hydrological
science, modeling, and
assessment tools through the
Climate Ready Water Utilities
program.
2: Assist wastewater and water
utilities to reduce greenhouse
gas emissions and increase long-
term sustainability with a
combination of energy
efficiency, co-generation, and
increased use of renewable
energy resources.
3: Work with the states and
public water systems,
particularly small water systems,
to identify and plan for climate
change challenges to drinking
water safety and to assist in
meeting health based drinking
water standards.
4: Promote sustainable design
approaches to provide for the
long-term sustainability of
infrastructure and operations.
5: Understand and promote
through technical assistance the
use of water supply management
strategies.
6: Evaluate and provide
technical assistance on the use
of water demand management
strategies.
7: Increase cross-sector
knowledge of water supply
climate challenges and develop
watershed specific information
to inform decision making.
X
X
X
X
X
X
X
Watersheds and Wetlands
8: Develop a national framework
and support efforts to protect
remaining healthy watersheds
and aquatic ecosystems.
9: Collaborate with partners on
terrestrial ecosystems and
hydrology so that effects on
water quality and aquatic
ecosystems are considered.
X
X
D-1
-------
EC
STRATEGIC ACTION
10: Integrate protection of
healthy watersheds throughout
the NWP core programs.
11: Increase public awareness of
the role and importance of
healthy watersheds in reducing
the impacts of climate change.
12: Consider a means of
accounting for climate change in
EPA funded and other watershed
restoration projects.
13: Work with federal, state,
interstate, tribal, and local
partners to protect and restore
the natural resources and
functions of riverine and coastal
floodplains as a means of
building resiliency and
protecting water quality.
14: Encourage states to update
their source water delineations,
assessments or protection plans
to address anticipated climate
change impacts.
15: Continue to support
collaborative efforts to increase
state and local awareness of
source water protection needs
and opportunities, and
encourage inclusion of source
water protection areas in local
climate change adaptation
initiatives.
16: Consider the effects of
climate change, as appropriate,
when making significant
degradation determinations in
the CWA Section 404 wetlands
permitting and enforcement
program.
17: Evaluate, in conjunction with
the U.S. Army Corps of
Engineers, how wetland and
stream compensation projects
could be selected, designed, and
sited to aid in reducing the
effects of climate change.
18: Expand wetland mapping by
supporting wetland.
19: Produce a statistically valid,
ecological condition assessment
of the nation's wetlands.
20: Work with partners and
stakeholders to develop
information and tools to support
long term planning and priority
setting for wetland restoration
projects.
POLICY
AND/OR
GUIDANCE
CHANGE
X
X
X
X
FINANCIAL
INCENTIVE
TECHNICAL
RESOURCES /
DATA
DEVELOPMENT
X
X
X
X
TECHNICAL
ASSISTANCE /
TRAINING
X
X
FOSTERING
PARTNERSHIPS/
COLLABORATION
X
D-2
-------
EC
STRATEGIC ACTION
POLICY
AND/OR
GUIDANCE
CHANGE
FINANCIAL
INCENTIVE
TECHNICAL
RESOURCES /
DATA
DEVELOPMENT
TECHNICAL
ASSISTANCE /
TRAINING
FOSTERING
PARTNERSHIPS/
COLLABORATION
Coastal and Ocean Waters
21: Collaborate to ensure that
synergy occurs, lessons learned
are transferred, federal efforts
effectively help local
communities, and efforts are not
duplicative or at cross-purposes.
22: Work within EPA and with
the U.S. Global Change Research
Program and other federal,
tribal, and state agencies to
collect, produce, analyze, and
format knowledge and
information needed to protect
ocean and coastal areas and
make it easily available.
23: Work with the NWP's larger
geographic programs to
incorporate climate change
considerations, focusing on both
the natural and built
environments.
24: Address climate change
adaptation and build stakeholder
capacity when implementing NEP
Comprehensive Conservation and
Management Plans and through
the Climate Ready Estuaries
Program.
25: Conduct outreach and
education, and provide technical
assistance to state and local
watershed organizations and
communities to build adaptive
capacity in coastal areas outside
the NEP and Large Aquatic
Ecosystem programs.
26: Support coastal wastewater,
stormwater, and drinking water
infrastructure owners and
operators in reducing climate
risks and encourage adaptation
in coastal areas.
27: Support climate readiness of
coastal communities, including
hazard mitigation, pre-disaster
planning, preparedness, and
recovery efforts.
28: Support preparation and
response planning for diverse
impacts to coastal aquatic
environments.
29: Consider climate change
impacts on marine water quality
in NWP ocean management
authorities, policies, and
programs.
X
X
X
X
X
X
X
X
X
D-3
-------
EC
STRATEGIC ACTION
30: Use available authorities and
work with the
Regional Ocean Organizations
and other federal and state
agencies through regional ocean
groups and other networks so
that offshore renewable energy
production does not adversely
affect the marine environment.
31: Support the evaluation of
sub-seabed sequestration of C02
and any proposals for ocean
fertilization.
32: Participate in interagency
development and
implementation of federal
strategies through the NOC and
the NOC Strategic Action Plans.
POLICY
AND/OR
GUIDANCE
CHANGE
FINANCIAL
INCENTIVE
TECHNICAL
RESOURCES /
DATA
DEVELOPMENT
TECHNICAL
ASSISTANCE /
TRAINING
X
X
FOSTERING
PARTNERSHIPS/
COLLABORATION
X
Water Quality
33: Encourage states and
communities to incorporate
climate change considerations
into their water quality
planning.
34: Encourage green
infrastructure and low-impact
development to protect water
quality and make watersheds
more resilient.
35: Promote consideration of
climate change impacts by
National Pollutant Discharge
Elimination System permitting
authorities.
36: Encourage water quality
authorities to consider climate
change impacts when developing
wasteload and load allocations in
TMDLs where appropriate.
37: Identify and protect
designated uses that are at risk
from climate change impacts.
38: Clarify how to re-evaluate
aquatic life water quality
criteria on more regular
intervals; and develop
information to assist states and
tribes who are developing
criteria that incorporate climate
change considerations for
hydrologic condition.
39: Continue to provide
perspective on the water
resource implications of new
energy technologies.
X
X
X
X
X
X
X
D-4
-------
EC
STRATEGIC ACTION
40: Provide assistance to states
and permittees to assure that
geologic sequestration of C02 is
responsibly managed.
41: Continue to work with States
to help them identify polluted
waters, including those affected
by biofuels production, and help
them develop and implement
Total Maximum Daily Loads
(TMDLs) for those waters.
42: Provide informational
materials for stakeholders to
encourage the consideration of
alternative sources of energy
and fuels that are water
efficient and maintain water
quality.
43: As climate change affects
the operation or placement of
reservoirs, EPA will work with
other federal agencies and EPA
programs to understand the
combined effects of climate
change and hydropower on
flows, water temperature, and
water quality.
44: Monitor climate change
impacts to surface waters and
ground water.
45: Collaborate with other
federal agencies to develop new
methods for use of updated
precipitation, storm frequency,
and observational streamflow
data, as well as methods for
evaluating projected changes in
low flow conditions.
46: Enhance flow estimation
using National Hydrography
Dataset Plus (NHDPlus).
POLICY
AND/OR
GUIDANCE
CHANGE
FINANCIAL
INCENTIVE
TECHNICAL
RESOURCES /
DATA
DEVELOPMENT
X
X
X
X
X
X
TECHNICAL
ASSISTANCE /
TRAINING
X
FOSTERING
PARTNERSHIPS/
COLLABORATION
Working with Tribes
47: Through formal consultation
and other mechanisms,
incorporate climate change as a
key consideration in the revised
NWP Tribal strategy and
subsequent implementation of
CWA, SOW A, and other core
programs.
48: Incorporate adaptation into
tribal funding mechanisms, and
collaborate with other EPA and
federal funding programs to
support sustainability and
adaptation in tribal
communities.
X
X
D-5
-------
EC
STRATEGIC ACTION
49: Collaborate to explore and
develop climate change science,
information, and tools for tribes,
and incorporate local
knowledge.
50: Collaborate to develop
communication materials
relevant for tribal uses and tribal
audiences.
Totals
POLICY
AND/OR
GUIDANCE
CHANGE
7
FINANCIAL
INCENTIVE
2
TECHNICAL
RESOURCES /
DATA
DEVELOPMENT
X
X
17
TECHNICAL
ASSISTANCE /
TRAINING
19
FOSTERING
PARTNERSHIPS/
COLLABORATION
5
D-6
-------
EC
APPENDIX E: DATA CONSISTENCY AND QUALITY
The evaluation team's assessment of strategic actions according to the seven phases of adaptive
management was based in part on baseline data collected for strategic actions by OW in December.
These data illustrated challenges of data consistency and quality that are important to address for any
measurement system. Some of these challenges derive from the problems discussed in the report (see
the section entitled, "Considerations for Mapping Strategic Actions to the Phases of Adaptive
Management"). For example, broadly written strategic actions may lead to radically different
interpretations among partners and ultimately to significant variations in the amount and type of data
reported. Other issues with the data consistency and quality cannot be directly attributed to the way
EPA has written its strategic actions. The evaluation team noted the following issues with baseline data:
A Reporters often discussed future plans instead of current progress.
A Reporters cited activities that occurred many years ago as evidence of progress.
A When providing examples of progress, reporters cited activities that were only vaguely related
to each strategic action.
A The baseline data survey itself needs more specificity.
Data consistency and quality were adversely affected when reporters cited future plans or activities
from the distant past as current progress on strategic objectives. Data become incomparable when each
reporter makes their own assumptions about relevant timeframes. For example, if one region discusses
activities going back three years and another going back five years, it is difficult or impossible to
determine which region has made more progress. EPA should specify an appropriate timeframe, within
which reporters can cite relevant past and future activities as examples of progress.
Data consistency and quality also faltered because the activities cited as examples of progress had
varying degrees of relevance to the strategic actions. It becomes difficult to compare various datasets
when one reporter takes a conservative view and provides a few activities that definitively relate to a
specific strategic action, while another provides a multitude of examples that are only vaguely relevant.
To prevent this problem, EPA should consider drafting strategic actions that are more narrowly worded
and providing examples of activities that are and are not appropriate citations of progress.
Finally, the template EPA used to collect baseline data does not promote consistency. Most significantly,
the template asks open ended questions that lend themselves to a high degree of inconsistency.
Likewise, it permits reporters to describe—or not describe—progress on any strategic action they
choose, rather than requiring them to cite progress or lack of progress on all strategic actions in their
purview. The outcome is a patchwork of results from HQ offices and regions. Additionally, by asking
about future plans and challenges, the template distracts from its intended purpose: to gauge progress
on strategic actions. EPA could improve the template by utilizing check boxes or pick lists instead of
open ended questions, requiring progress reports for all strategic actions, focusing questions on
progress instead of challenges and future plans, and asking for progress reports in terms of strategic
action stages rather than adaptive management phases.
E-1
------- |