ง
<
3J
\
r"
*L P R
T>
2
LU
O
T
J
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Evaluation Report
Sustained Commitment
Needed to Further Advance
Watershed Approach
Report No. 2005-P-00025
September 21, 2005
-------
Report Contributors:
Dan Engelberg
George Golliday
Doug LaTessa
Renee McGhee-Lenart
Abbreviations
CWSRF Clean Water State Revolving Fund
EPA Environmental Protection Agency
HUC Hydrologic Unit Code
NPDES National Pollutant Discharge Elimination System
PAM Program Activity Measure
TMDL Total Maximum Daily Load
Cover photo: A view of the upper watershed of the North Fork Lucie River, Florida
(from South Florida Water Management District Web site).
-------
^tos%
&
0"V U.S. Environmental Protection Agency 2005-P-00025
Office of Inspector General September 21,2005
At a Glance
prqI^
Catalyst for Improving the Environment
Why We Did This Review
We undertook this evaluation
to determine how well the
U.S. Environmental Protection
Agency (EPA) is doing in four
critical elements to advance the
watershed approach. These
four elements are integration,
stakeholder participation,
strategic planning, and
performance measurement.
Background
A watershed refers to a
geographic area in which water
drains to a common outlet. A
watershed includes not only all
water resources, such as lakes
and rivers, but also the land that
drains into these resources.
The watershed approach is a
strategy for achieving clean
water that relies on
decentralized decision making
and stakeholder involvement to
effectively protect and restore
aquatic ecosystems.
For further information,
contact our Office of
Congressional and Public Liaison
at (202) 566-2391.
To view the full report, click on the
following link:
www.epa.qov/oiq/reports/2005/
20050921-2005-P-00025.pdf
Sustained Commitment Needed to
Further Advance Watershed Approach
What We Found
If EPA is committed to the watershed approach, it needs to make improvements
in four key elements:
Integrating watershed activities into its core water programs.
Addressing stakeholder concerns to increase their participation.
Refining and improving key aspects of its strategic planning process.
Improving the watershed performance measurement system.
EPA adopted the watershed approach to help focus existing, traditional water
pollution control programs in a more comprehensive manner and address
emerging problems. The premise is that many water quality problems are best
solved at the overall watershed level rather than the individual waterbody or
discharger level. It is a holistic approach that considers cumulative impacts from
a variety of sources, and represents an effort to enhance EPA's ability to improve
and protect the Nation's water quality.
Although progress has been made in each of the four critical elements that we
reviewed, further improvements are needed for each. EPA has made progress
integrating watershed approach principles into some of its core water programs,
but needs to address challenges to ensure further success. Stakeholders were
enthusiastic about the watershed approach, but identified a number of obstacles
when adopting the approach. EPA has made important strides incorporating the
watershed approach into its strategic plans, but it must improve some key steps.
Although EPA developed a performance measurement system for improving
water quality on a watershed basis, EPA did not develop measures to evaluate
key programs and activities, and its national outcome measures were not
understandable, comparable, and reliable.
What We Recommend
We recommend that EPA address challenges to integrating watershed approach
principles into its core programs, as well as obstacles identified by stakeholders
concerning the watershed approach. EPA also needs to improve its strategic
plans and performance measurement system that address the watershed approach.
If EPA is committed to the watershed approach, it will make these needed
improvements. EPA provided comments in response to our draft report. The
Agency generally agreed with our findings and recommendations.
-------
^STA%
X UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
| mM \ WASHINGTON, D.C. 20460
US?/
OFFICE OF
INSPECTOR GENERAL
September 21, 2005
MEMORANDUM
SUBJECT: Sustained Commitment Needed to Further Advance Watershed Approach
Report No. 2005-P-00025
FROM: Kwai-Cheung Chan /s/
Assistant Inspector General for Program Evaluation
TO: Benjamin Grumbles
Assistant Administrator
Office of Water
This is our final report on how well the U.S. Environmental Protection Agency (EPA) is
advancing the watershed approach. This report contains findings that describe the issues
identified by the EPA Office of Inspector General (OIG) and recommended corrective actions.
This report represents the opinion of the OIG, and the findings contained herein do not
necessarily represent the final EPA position. Final determinations on matters discussed in this
report will be made by EPA managers in accordance with established audit resolution
procedures.
The OIG issued a draft report on June 21, 2005, to EPA for review and comment. A response
was submitted on July 28, 2005. EPA's response highlighted its efforts to integrate the
watershed approach into its core programs and develop partnerships with stakeholders. EPA also
responded to our recommendations on improving strategic planning and performance measures.
The OIG has incorporated these comments, as well as the technical corrections and supplemental
information provided by EPA, into the final report.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days of the date of this report. You should include a corrective action
plan for agreed upon actions, including milestone dates. We have no objections to the further
release of this report to the public. For your convenience, this report will be available at
-------
http://www.epa.gov/oig. In addition to providing a written response, please e-mail an electronic
version to mcghee-lenart.renee@epa.gov.
If you or your staff have any questions regarding this report, please contact me at (202) 566-0827,
Dan Engelberg, Director of Program Evaluation - Water Issues, at (202) 566-0830, or Renee
McGhee-Lenart, Assignment Manager, at (913) 551-7534.
-------
Table of Contents
At a Glance
Chapters
1 Introduction 1
Purpose 1
Background 1
Scope and Methodology 4
Results in Brief 4
2 Progress Made in Integrating Watershed Principles into Core Programs,
but Challenges Remain 6
Guiding Principles of the Watershed Approach 6
National Pollutant Discharge Elimination System Program 7
Total Maximum Daily Load Program 9
Nonpoint Source Program 10
Water Quality Monitoring Program 12
Water Quality Standards Program 13
Clean Water State Revolving Fund Program 14
3 Stakeholders Identify Advantages of and Obstacles with Implementing the
Watershed Approach 16
Advantages and Opportunities Noted to Watershed Approach 16
Stakeholders Identified a Variety of Obstacles 17
4 EPA's Watershed Plan Hampered by Inadequate Preparation 21
Strategic Planning Critical for Allocating Resources 21
Significant Progress Made in Strategic Planning 22
Limitations in Data and Analysis Undermine Baseline Measure 23
Unachievable Goals Hamper EPA's Ability to Guide Activities 25
Regional Plans Inadequate to Implement National Strategies 27
5 Progress Made Developing Performance Measurement System,
but Improvements Needed 28
Aspects of Watershed Approach Performance and Measurement 28
Performance Measurement System Incomplete 29
National Outcome Measures Need Improved Design 30
6 EPA's Commitment Must Be Sustained by Addressing Critical Issues 33
Recommendations 34
Agency Comment and OIG Evaluation 35
-------
Appendices
A Details on Scope and Methodology 36
B Clean Water Act Watershed Framework 40
C EPA's Data Processing Techniques for Developing
Watershed Restoration Baseline 41
D Subobjective 2.2.1 Program Activity Measures 45
E Agency Comments on Draft Report 48
F OIG Evaluation of Agency Comments 53
G Distribution 56
ii
-------
Chapter 1
Introduction
Purpose
The U.S. Environmental Protection Agency (EPA) has supported a "Watershed
Protection Approach" for well over a decade to help meet the Agency's Clean
Water Act and Safe Drinking Water Act obligations to achieve clean water. The
Office of Inspector General (OIG) undertook this evaluation to determine how
well EPA is doing in four critical elements to advance the watershed approach.
We evaluated those elements using the following questions:
Integration: What progress has EPA made in integrating watershed approach
principles into its water programs and activities?
Stakeholder Participation: What advantages/opportunities and challenges
exist for EPA to persuade stakeholders to adopt the watershed approach?
Strategic Planning: How well has EPA strategically planned for
implementing the watershed approach?
Performance Measurement: How adequately does EPA measure the
success of the watershed approach?
Background
A watershed refers to a geographic area in which water drains to a common
outlet. The watershed includes not only the water resources, such as lakes and
rivers, but the land surrounding these resources. The watershed approach is a
strategy for effectively protecting and restoring aquatic ecosystems and protecting
human health that relies on decentralized decision making and significant
stakeholder involvement. EPA adopted this approach to help focus existing,
traditional water pollution control programs in a more comprehensive manner and
address emerging problems such as nonpoint source pollution. An example of the
watershed approach is EPA's National Estuary Program, which provides funding
to restore and protect 28 of America's nationally significant estuaries. An estuary
is the region of interaction between rivers and near-shore ocean waters, where
fresh and salt water mix. The National Estuary Program is an inclusive,
community-based approach on the watershed level.
Historically, EPA has worked to achieve clean and safe waters through
implementing the Clean Water Act by employing a variety of regulatory programs
and tools. The conventional water programs tend to focus on particular sources,
pollutants, industries, or facilities, and have resulted in a fragmented approach to
managing water quality. Despite the success of reducing impacts of point source
discharges, the nation's water quality has remained at risk due to the emergence
of nonpoint source pollution, which comes from diffuse sources and is generally
1
-------
carried by rainfall or snowmelt moving over the ground. As a result, EPA
adopted the watershed approach to address these challenges.
The watershed approach has as its premise that many water quality problems are
best solved at the watershed level rather than the individual waterbody or
discharger level. While there is some overlap between the activities of the two
approaches, States are still required to fulfill regulatory requirements. For
instance, EPA or States must issue National Pollutant Discharge Elimination
System permits to point sources dischargers. These permits can be issued to
individual facilities or on a watershed basis. Regardless, EPA and States must
implement these activities from a fixed pool of resources. EPA must fund its
conventional and watershed approaches within its current budget, which may
decrease in future years.
Although EPA has supported a "Watershed Protection Approach" since the early
1990s, the approach gained limited acceptance as the means to implement water
programs. For example, on December 3, 2003, the then Assistant Administrator
for Water noted there were "substantial gaps in actual implementation," and that
"now is the right time to focus and re-invigorate our efforts to more fully
institutionalize the approach - both on the ground and as a cornerstone of our core
water programs." The current Assistant Administrator for Water has stated,
".. .if we are going to leave our water purer than we found it, we must redouble
our efforts to implement a watershed management approach in every part of our
country."
EPA elevated the importance of the watershed approach by creating subobjective
2.2.1 in its 2003-2008 Strategic Plan. According to the Strategic Plan,
successfully protecting and improving water quality on a watershed basis depends
on: implementation of core water programs, including integration on a watershed
basis; engaging diverse stakeholders in solving problems; and applying innovative
ideas, such as water quality trading, to deliver cost-effective water pollution
control. EPA also developed two national outcome measures and an
Implementation Plan for Subobjective 2.2.1.
EPA is to assess how well it is improving water quality on a watershed basis
through two national outcome measures:
Number of the Nation's watersheds where water quality standards are met in
at least 80 percent of the assessed waters segments.
Number of the Nation's watersheds where all assessed water segments
maintain their quality and at least 20 percent of assessed water segments show
improvements above conditions as of 2002.
EPA also developed an Implementation Plan for Subobjective 2.2.1 to improve
water quality on a watershed basis. According to the Implementation Plan, the
watershed approach "should be the fulcrum of Federal and State restoration and
2
-------
protection efforts, and those of our many stakeholders, both private and public.
EPA has both a national interest in, and responsibility for, supporting watershed
goals and approaches and believes that such an approach is one of the most
important environmental guiding principles to maintain and restore the chemical,
physical and biological integrity of the Nation's waters." The Implementation
Plan sets forth the following three-part strategy to improve water quality on a
watershed basis:
Implementing core clean water programs, including on a watershed basis.
Accelerating watershed protection.
Applying an adaptive management framework.
To implement core clean water programs to improve water quality on a watershed
basis, EPA will:
Figure 1.1: Actions to Implement the Core Clean Water Act Programs
ฆ/ Strengthen the National Pollutant Discharge Elimination System Permit Program
ฆS Develop Total Maximum Daily Loads and Related Plans
ฆS Implement Effective Nonpoint Source Practices on a Watershed Basis
ฆS Improve Water Quality Monitoring and Assessment
ฆS Strengthen the Water Quality Standards Program
ฆS Support Sustainable Wastewater Infrastructure through the Clean Water State
Revolving Fund
In addition, EPA is integrating watershed principles into these six key program
areas. However, EPA recognizes that implementation of water programs on a
watershed basis, which is the first part of its three-part national strategy, is not
enough to accomplish EPA's watershed goals.
To accelerate watershed protection, EPA supports local watershed efforts by
working in collaboration with multiple partners, including other Federal agencies,
States, local governments, and environmental organizations. Local watershed
organizations develop watershed plans to help achieve clean and safe water. EPA
is developing tools and technical assistance to help these partners. To initiate and
strengthen watershed protection efforts for critical watersheds and waterbodies,
EPA created the Targeted Watershed Grants program. The program first provided
funding in 2003 to a variety of watershed projects designed to encourage
community-based approaches to protect and restore waters. Over the past 2 years,
EPA awarded 34 grants, totaling over $28 million.
The final part of EPA's strategy to accomplish its watershed objectives is to apply
an "adaptive management" framework. According to EPA's Implementation Plan
for Subobjective 2.2.1, an adaptive management framework applied to watershed
protection involves several key components, including setting challenging but
realistic goals, improving assessment and monitoring, and identifying barriers to
implementation. The Plan also states that the adaptive management framework
will allow EPA to analyze progress and obtain feedback regarding the
3
-------
effectiveness of different approaches, which can then be used to adjust and realign
the goals and specific program management and activities to make progress and
achieve clean water goals.
The Government Performance and Results Act does not specifically require that
performance measurement be applied to the watershed approach, since it is not
identified as a program under the Clean Water Act. Nonetheless, EPA has
developed a performance measurement system to assess how well it is improving
water quality on a watershed basis and implementing its three-part national
strategy.
Scope and Methodology
We conducted our evaluation from October 2003 through February 2005 in
accordance with Government Auditing Standards, issued by the Comptroller
General of the United States. We evaluated the watershed approach by analyzing
four critical elements key to the success of this approach: integration of watershed
activities into core water programs; partnerships; strategic planning; and
development of performance measures and goals.
To evaluate these four critical elements, we evaluated EPA headquarters and
regional guidance for the watershed approach and core programs. We also
reviewed EPA's Fiscal 2003-2008 Strategic Plan; the Implementation Plan for
Subobjective 2.2.1; the National Water Program Guidance for Fiscal 2005; and
the 2004 EPA Regional Plans for Regions 1, 4, 7, and 10. We interviewed all
four of the primary Office Directors within the Office of Water, as well as
managers and program staff in each of the six core programs. We also spoke with
EPA regional water program directors and staff from Regions 1, 4, 7, and 10.
Additionally, we interviewed representatives of nine large and small watershed
organizations, and water staff from six States, to identify advantages and
obstacles that EPA needs to address regarding the watershed approach.
Appendix A provides further details on our scope and methodology, including
prior coverage of this topic by OIG and others.
Results in Brief
While EPA has made progress in implementing the four key elements that we
reviewed regarding the watershed approach, improvements in each of the
elements are needed to ensure the success of the approach. EPA has taken steps
to integrate the watershed approach into its core water programs, but a number of
challenges exist that may prevent further integration. Stakeholders, who are vital
to successful implementation of the watershed approach, identified a number of
advantages to the watershed approach, as well as obstacles that threaten the
implementation of the approach. EPA has made strides in strategic planning for
the watershed approach, but phases of the planning process must be improved.
4
-------
While EPA has developed a performance measurement system, it needs to be
improved so that critical performance information is conveyed to EPA on the
implementation of the watershed approach.
We recommend that EPA address the challenges that impede integration of the
watershed approach into core water programs, as well as the obstacles that
prevent stakeholders from adopting and implementing the watershed approach.
We also recommend that EPA improve its strategic plans and performance
measurement system for implementing the watershed approach. If EPA is
committed to advancing the watershed approach, it will need to improve these
areas to ensure its success.
In July 2005, EPA provided comments in response to our draft report. EPA's
comments to the OIG draft report are in Appendix E. We provide our
evaluation of those comments in Appendix F.
5
-------
Chapter 2
Progress Made in Integrating Watershed Principles
into Core Programs, but Challenges Remain
EPA has made progress integrating watershed principles into core water
programs, but obstacles remain. One of the principal challenges to integrating the
nation's various clean water protection programs into a comprehensive
watershed-based effort is the fact that most of the programs do not have a
watershed origin. They came from individual requirements of the Clean Water
Act, with a programmatic focus. Most EPA water protection efforts occur
through six "core programs," and EPA has found building watershed principles
into these core programs to be a challenge. For example, expanding the
geographic focus and increasing stakeholder involvement in regulatory programs
may require more time and resources to carry out program requirements.
Obtaining the resources needed may be difficult in the current budget
environment.
Guiding Principles of the Watershed Approach
EPA's guiding principles1 for the watershed approach call for a geographic focus
and partnerships with those people most affected by watershed management
issues. The geographic focus principle encourages activities to be directed within
specific geographic areas such as watersheds. Partners, such as States and
watershed groups, are to be involved in making key decisions. We evaluated
EPA's six core Clean Water Act programs (see Figure 2.1) to determine whether
they supported these principles.
Figure 2.1: Six Core Clean Water Act Programs
ฆ/ National Pollutant Discharge Elimination System
ฆ/ Total Maximum Daily Loads
ฆ/ Nonpoint Source Pollution Control
ฆS Water Quality Monitoring
ฆS Water Quality Standards
ฆS Clean Water State Revolving Fund
According to EPA's planning documents, these Clean Water Act programs form
the foundation for the "watershed framework." This process is briefly described
in Appendix B.
1 The third guiding principle of the watershed approach is Sound Management Techniques Based On Strong Science
and Data. This principle was not assessed as it is not unique to the watershed approach.
6
-------
Details on what we found when comparing the conventional and watershed
approaches for each of the six programs follow.
National Pollutant Discharge Elimination System Program
EPA has integrated watershed approach principles into the National Pollutant
Discharge Elimination System (NPDES) permit program by promoting
watershed-based permitting and encouraging States to synchronize issuing
NPDES permits by water basins through the "rotating basin approach." The
advantages of watershed permitting are uncertain, however, because this is a new
tool, and limited empirical information exists on its benefits and the extent of its
use by stakeholders. Regardless of the approach taken for issuing permits, the
regulatory requirements of the Clean Water Act must be met.
Conventional Approach
The NPDES program controls water pollution by regulating point sources that
discharge pollutants into waters. Point sources are discrete conveyances, such as
pipes or man-made ditches. Facilities must obtain NPDES permits if they
discharge directly to surface waters, and the permit indicates a specified amount
of a pollutant that can be discharged. EPA has authorized 45 States to administer
components of their permitting programs with EPA oversight, while EPA
administers programs for the remaining five States and the District of Columbia.
Typically, permit requirements are arrived at between the States (or EPA) and the
affected facility, with limited participation by community groups. The two basic
types of NPDES permits are:
An individual permit, specifically tailored for an individual facility or point
source.
A general permit, to cover multiple facilities within a specific category.
General permits may be written to cover categories of point sources having
common elements, such as those in a similar industry, and may only be issued
to dischargers within a specific geographical area.
Watershed Approach
EPA has integrated watershed principles into the NPDES program by encouraging
watershed-based permitting. This involves developing NPDES permits that
coordinate requirements for control of dischargers within the delineated
watershed area. EPA's Watershed-Based NPDES Permitting Implementation
Guidance - published in December 2003 - emphasizes considering watershed
goals and the impact of multiple pollutant sources and stressors, and encouraging
increased stakeholder involvement for watershed permits. These efforts expanded
the geographic focus of the program to consider the overall conditions of the
watershed and other sources affecting water quality. Watershed-based permitting
7
-------
can combine aspects of the more traditional individual and general permits
described above. Examples include:
Watershed-Based Permit for Multiple Permitees - A single permit is issued to
multiple point sources.
Watershed General Permit - This is similar to existing general permits, but
the watershed boundary - and not the type of discharge - would be used to
determine the sources covered by the permit.
Individual Watershed Permits with Coordinated Requirements
An example of a watershed-based permit is the permit issued to the Neuse River
Compliance Association and its co-permitees in North Carolina. Association
members include facilities that discharge treated wastewater into the Neuse River
basin under existing individual NPDES permits. A single watershed-based permit
was issued to Association members governing the total amount of nitrogen
discharged into the watershed. The permit established a ceiling of nitrogen to be
discharged and allocated this limit among the existing dischargers. The permit
allows for trading among Association members to meet the total limit. Pollutants
other than nitrogen are controlled under each member's individual permit.
Watershed-based permitting relies significantly on EPA's partners, such as the
regulated entities and contributors of nonpoint source pollution. As the basis for
making permitting decisions expands from a single point source and water body
to a collection of sources in a watershed, the number of parties with an interest
expands. Enlisting voluntary involvement by stakeholders - particularly those
currently unregulated (such as farmers) - can be a challenge, but is important.
Including more stakeholders may require additional time and complexity to the
permit development process. As a result, watershed permits may require more
resources and thus be more expensive to develop. EPA recognized this in its
Watershed-Based National Pollutant Discharge Elimination System (NPDES)
Permitting Implementation Guidance issued in December 2003. The guidance
stated that "engaging a wider variety of stakeholders means that the permitting
authority and the permit writer will have to consider a broader range of interests
and watershed goals when developing the permit, potentially adding technical
complexity and time to the permit development process." This may be a
challenge for States with limited resources. Further details on stakeholder
involvement are in Chapter 3.
Ensuring all sources of stressors in a watershed are taken into account can also be
challenging. For example, the sources of air deposition that impair watersheds
(such as mercury emitted from the smokestacks of coal-fired utilities) may come
from other States, and from the air rather than water, and thus can be difficult to
control under the Clean Water Act.
The benefits of watershed permitting are uncertain. Since watershed permitting is
a new tool, limited empirical information exists on its benefits. In addition, EPA
8
-------
does not have data on the extent of its use by stakeholders. However, EPA has
recently begun to track the number of such permits, and this information is critical
for evaluating the effectiveness of watershed-based permitting in achieving clean
water goals and improving efforts.
In addition to promoting watershed permitting, EPA has advanced the watershed
approach by encouraging stakeholders to synchronize NPDES permit issuance
within water basins (a "rotating basin approach"). This approach involves a cycle
of management programs (e.g., monitoring, planning, permit issuance) around
State river basins over an established time frame (usually 5 years). EPA
encouraged such a process for issuing NPDES permits in its 1994 NPDES
Watershed Strategy. This process may prove difficult because dischargers are
often not evenly distributed across States' water basins, and other Federal
initiatives (such as EPA efforts to reduce the NPDES permit backlog) may divert
resources from basin permitting cycles according to an EPA report, A Review of
Statewide Watershed Management Approaches, issued in April 2002.
Total Maximum Daily Load Program
EPA has integrated principles of the watershed approach into the Total Maximum
Daily Load (TMDL) program by encouraging States to develop TMDLs on a
watershed basis rather than by individual water segments. Stakeholder
involvement with TMDLs is critical for both the conventional and watershed
approaches, but the broader watershed approach may expand the number of
stakeholders. Expanding both the geographic scale and the number of
stakeholders may result in additional time and resources required to develop these
TMDLs. Regardless of the approach taken for development of TMDLs, the
regulatory requirements of the Clean Water Act must be met.
Conventional Approach
A TMDL is a calculation of the maximum amount of a pollutant a waterbody can
receive from all sources and still meet water quality standards. The Clean Water
Act requires States to identify a list of impaired waters not meeting State water
quality standards, set priorities for TMDL development, and develop a TMDL for
those pollutants causing the impairment. EPA has approval authority over the
lists of impaired waters and corresponding TMDLs, and if EPA disapproves a
State submission it must develop the list of impaired waters and corresponding
TMDLs itself. TMDLs are geographically based, although the Clean Water Act
does not specify the scale at which States should develop TMDLs. In many cases,
TMDLs are developed for individual waterbodies rather than entire watersheds.
Because EPA is under court orders and consent decrees in some States to develop
TMDLs for impaired waterbodies, developing TMDLs for individual waterbodies
may be quicker.
9
-------
Watershed Approach
EPA encourages States to develop TMDLs on a watershed basis because many
water pollution concerns are found in the same geographic area and are caused by
multiple dischargers, multiple pollutants, or nonpoint sources. Developing more
comprehensive TMDLs may take longer due to the complexity of the problems in
the watershed and more stakeholders being involved, which may also result in the
need for more resources. Similar to NPDES permitting, EPA also encourages a
rotating basin approach for TMDL development as watershed strategies suggest
that related activities within a basin should be coordinated to achieve greater
benefits.
The watershed approach to developing TMDLs relies more heavily on partners
and stakeholders than the conventional approach. EPA encourages States to
establish local advisory groups to assist State regulators during the TMDL
development process. Also, EPA has recently taken steps to further the use of
water quality trading in impaired waters for certain pollutants, such as nutrients
and sediment. In January 2003, EPA issued a "Final Water Quality Trading
Policy," and in November 2004 published its "Water Quality Trading Assessment
Handbook." The latter discusses how to broaden stakeholder participation by
identifying and engaging potential participants.
There are several barriers to developing watershed TMDLs. As is the case for the
NPDES permitting program, developing TMDLs for multiple waters and pollutant
sources may be more complex than doing so for individual segments and
pollutants. This may increase the time needed to complete watershed TMDLs.
Increased stakeholder involvement may add to the complexity as well. For these
reasons, developing watershed TMDLs may require more resources than
conventional TMDLs. Further, some States have found it difficult to complete
TMDL development within rotating basin schedules. Some States may find
resultant delays of particular concern because they are under court orders and
consent decrees to finish TMDLs under specified timeframes.
Jurisdictional issues are another challenge to developing TMDLs on a watershed
basis. Many watersheds cross State boundaries, as do the effects of pollution
sources. Because pollution control is a State-level activity, downstream States
with waters affected by an upstream source in a different State have no way to
directly control that pollution source. The issue of air deposition noted previously
is an example of this jurisdictional issue. A number of State staff we interviewed
identified jurisdictional issues as an important area in which EPA should continue
being involved. Chapter 3 contains further details.
Nonpoint Source Program
EPA's Nonpoint Source program has incorporated watershed principles into the
program by encouraging the development of geographically based watershed
10
-------
plans and involving partners when carrying out programmatic requirements. EPA
encourages the development of these plans and is relying on States, working with
local watershed groups and other stakeholders, to develop and implement them.
Interviews with stakeholders suggest that local watershed groups often lack the
technical capacity to develop these plans, which could be an ongoing challenge.
Conventional Approach
Nonpoint source pollution is seen as the largest remaining source of uncontrolled
pollution. Rather than coming from a specific source, such as an industrial plant,
nonpoint source pollution is generally caused by rainfall or snowmelt moving
over and through the ground. As the runoff moves, it picks up and carries natural
and human-made pollutants, depositing them into surface waters.
The nonpoint source program has historically embraced watershed principles
because the 1987 amendments to the Clean Water Act, which created the
nonpoint source program, preceded EPA's watershed approach by only a few
years. The amendments require that States implement nonpoint source programs
on a watershed basis to the maximum extent practicable, and that States involve
partners when developing and implementing the programs. In the early phases of
the program, some States and EPA regional offices focused their programs on
implementation of management practices and demonstration projects of particular
technologies in limiting nonpoint source runoff. As a result, EPA has issued
numerous publications and reports highlighting specific pollutant reductions or
other measurable improvements attributable the nonpoint source program.
Despite these efforts, nonpoint source pollution continues to be a leading cause of
water pollution. According to the 1998 Clean Water Act Section 303(d) list,
43 percent of water quality impairments were attributed exclusively to nonpoint
source pollution. An additional 47 percent were attributed both to point and
nonpoint source pollution.
Watershed Approach
EPA has indicated that developing and implementing watershed plans are
important in solving nonpoint source problems. These plans are designed to
identify the causes of pollution and suggest controls, known as nonpoint source
management measures. Beginning in 1999, EPA and the States sharpened the
focus of the nonpoint source program on solving water problems in impaired
waters, and in 2002 issued updated guidance to emphasize watershed-based
efforts. Moreover, the Office of Water recently established a "program activity
measure" to track the number of watershed-based plans supported under State
nonpoint source management programs.
EPA has encouraged partnerships through efforts to increase public awareness
and understanding of nonpoint source pollution in their watersheds. EPA has
conducted focus groups to identify media outreach plans; developed guides on
11
-------
how to put together watershed outreach campaigns; and assembled radio,
television, and newspaper public service announcements. EPA holds conferences
on community involvement, and past conferences have included sessions on
watershed protection. Nonetheless, maintaining community interest in watershed
issues remains a challenge, particularly when there is not a crisis that affects the
community directly.
EPA and States often rely on local stakeholders, such as local watershed
organizations, to develop comprehensive watershed plans to address nonpoint
source impairments. Interviews with stakeholders indicated that watershed
organizations often lack the technical capacity to develop these complex plans.
To help stakeholders facilitate an effective watershed planning process, EPA
established a Watershed Planning Team. The team is assisting stakeholders by
describing the elements of a comprehensive watershed plan, and developing a
Web-based tool to facilitate the development of such plans.
Water Quality Monitoring Program
EPA has introduced watershed principles into the water quality monitoring
program primarily by encouraging States to monitor and assess geographic areas,
such as watersheds, using a rotating basin approach. EPA has promoted the use
of a rotating basin approach for monitoring since it may be more effective than
traditional approaches. Although a recent survey by the Association of State and
Interstate Water Pollution Control Administrators indicated that some States have
adopted a rotating basin approach to monitor State waters, the percentage of
assessed waters throughout the nation has remained low.
Conventional Approach
States are responsible for monitoring and reporting to EPA on the condition of
their waters. The Clean Water Act does not specify the scale to which States must
monitor the quality of their waters. Historically, EPA has relied on submission of
Clean Water Act section 305(b) reports to determine that States have monitored
water conditions. States have traditionally used a variety of monitoring
techniques, such as fixed monitoring networks, probability design, targeted
monitoring, and judgmental sampling. However, the percentage of assessed
waters in the United States has remained relatively low. For example, from 1992
through 2000, EPA indicated the number of assessed river miles remained about
19 percent. Although EPA does not have all State data for the 2002 reporting
cycle, the Agency noted that States assessed a small percentage of their waters for
this reporting cycle as well.
To leverage limited resources and in recognition that other stakeholders collect
monitoring data, EPA encourages partnerships with other Federal agencies, State
monitoring councils, and volunteer organizations that collect data on water
quality. For example, EPA is beginning to work more closely with the U.S.
12
-------
Geological Survey to share monitoring data. Through its work on the National
Water Quality Monitoring Council, EPA has worked to form monitoring councils
to coordinate monitoring organizations.
Watershed Approach
EPA encourages States to employ a rotating basin approach for monitoring
watersheds, since it may result in greater monitoring coverage of State waters
than historical practices. In addition, in its 2003 guidance document, Elements of
a State Water Monitoring and Assessment Program, EPA provided guidance to
States to upgrade their monitoring programs and advocated rotating basin
approaches. According to EPA's response to our draft report, it is working
closely with States to implement this guidance and has indicated that all States
and territories are on schedule to have comprehensive State monitoring strategies
finalized by September 2005. However, as noted, the percentage of assessed
waters in the United States has remained relatively low.
Water Quality Standards Program
The opportunity to incorporate watershed principles (geographic focus and
partnerships) is limited in the Water Quality Standards program. For example,
because uses vary from place to place, one component of standards - designated
uses - may need to vary from site to site. However, EPA has incorporated the
geographic focus principle of the watershed approach into the Water Quality
Standards program for some pollutants.
Conventional Approach
Water quality standards define the water quality goals for a waterbody. These
standards include three components: (1) the designated use or uses; (2) pollutant
criteria necessary to protect the uses; and (3) a policy to prevent or limit water
quality degradation. These serve as one of the regulatory bases for permitting
programs, and provide the basis for the TMDL allocations and nonpoint source
controls. States are responsible for adopting and revising water quality standards.
EPA approves or disapproves the State standards; if EPA disapproves a standard,
it can promulgate a new standard for the State. EPA also develops and revises
water quality criteria for certain pollutants, which can be used by States.
Watershed Approach
EPA encourages States to use a watershed as the geographic unit of analysis when
adopting criteria for certain parameters, such as nutrients. For example, nutrient
levels in the nation's waters vary from one "ecoregion" to another, so it is not
appropriate to develop a single national water quality criterion for nutrients. EPA
has published nutrient criteria for different ecoregions across the country. It is
13
-------
EPA's expectation that States tailor such ecoregion criteria to site-specific
conditions within a State - such as watersheds.
Water quality standards tend to be waterbody-specific rather than adopted for
watersheds. Assigning the same designated uses to an entire watershed would be
questionable (for example, only some waters in a watershed would be used as
public water supplies). However, some regulatory requirements of the Water
Quality Standards program support watershed approach principles. For example,
State water quality standards must consider downstream uses. EPA guidance also
affords States the opportunity to adopt site-specific criteria for the protection of
aquatic life. Site-specific criteria may be appropriate when taking into account
the conditions at a site such as a watershed.
EPA's measure of success is dependent upon whether individual waters within
watersheds are meeting their applicable water quality standards. EPA also
recognizes that to protect water resources, it is critical to address the condition of
land areas within watersheds, to help control nonpoint source pollution.
Landscape level indicators have been suggested as a way to assess the linkage
between pollution sources and aquatic health.
Clean Water State Revolving Fund Program
EPA has incorporated watershed approach principles into the Clean Water State
Revolving Fund (CWSRF) program, a funding mechanism available to pay for
watershed projects. EPA has encouraged States to consider the goals of
watersheds when making funding decisions. However, according to EPA
guidance, CWSRF funds have been under-utilized for non-point source watershed
projects.
Conventional Approach
The CWSRF is a funding mechanism available to pay for a variety of both point
and nonpoint source projects. The 1987 amendments to the Clean Water Act
created the CWSRF program, designed to replace the construction grants
program. To date, the CWSRF has funded over $43 billion dollars for eligible
projects. EPA makes grants to States for the purpose of establishing a revolving
fund that provides loans for the following projects:
Construction of publicly owned wastewater facilities.
Implementation of nonpoint source control projects.
Estuary management projects.
States provide matching funds (a minimum of 20 percent) and then fund low-
interest loans to eligible projects within the State. Although EPA provides
oversight and technical assistance to States to administer their programs, each
State has the authority to fund its own priorities. Most of the CWSRF funding has
14
-------
been targeted toward wastewater facilities rather than nonpoint source and estuary
projects. EPA's 2004 Nonpoint Source Grant Guidelines stated that. .most
States have under-utilized this resource (CWSRF) to date... States are
encouraged to increase their use of this copious financial resource to help
implement their nonpoint source watershed-based plans and other nonpoint source
projects...."
Watershed Approach
EPA has incorporated watershed principles into the CWSRF program by
providing States with guidance on how to consider the goals of different
geographic scales, such as a watershed, when making funding decisions. In 1996,
EPA recognized that States were interested in funding other types of projects
found within a watershed, such as nonpoint source and estuary projects. As a
result, EPA provided guidance to States on how to prioritize projects eligible for
CWSRF funding to encourage more flexibility in funding decisions. EPA
encourages States to use the CWSRF as a source of funding for nonpoint source
projects identified by watershed plans, although EPA admits this source of
funding has been under-utilized by States for such projects. As of November
2003, 14 States had not used any of their revolving funds for nonpoint source
pollution projects. Given the significance of nonpoint source pollution and the
funding available through the CWSRF program, opportunities exist to expand the
use of this program to mitigate pollution in watersheds.
15
-------
Chapter 3
Stakeholders Identify Advantages of and Obstacles
with Implementing the Watershed Approach
Stakeholders identified a variety of advantages and obstacles that EPA should
take into account when trying to persuade other stakeholders to adopt the
watershed approach. Although most stakeholders we interviewed see the benefits
of the watershed approach, they expressed concerns about how to operationalize
the approach. Concerns cited included the need for more public involvement, as
well as better coordination and communication. Because stakeholders at the State
and local levels contribute most of the labor and thus play a crucial role in
achieving improved water quality, EPA should address their concerns.
Advantages and Opportunities Noted to Watershed Approach
State staff and watershed organization officials we spoke with expressed
enthusiasm about the watershed approach. They identified a number of
advantages to the approach, as follows.
Watershed Approach Leads to Increased Participation by Stakeholders
A number of representatives from local watershed organizations, as well as State
staff, told us they believe the watershed approach affords local citizens, the
regulated community, local governments, and environmental organizations the
opportunity to identify priorities and strategies for accomplishing mutual goals.
In their view, the watershed approach provides the opportunity for more local
involvement because decisions about water quality are made from the 'bottom-up'
rather than driven by regulatory agencies. Robust local involvement is critical to
the success of the watershed approach.
Watershed Approach Focuses on Environmental Results
Staff from a number of States interviewed indicated that the watershed approach
is more results-oriented than EPA's regulatory programs, and thus provides an
opportunity for improved water quality. In their view, the watershed approach is
synonymous with a focus on environmental results, and that regulatory programs
are too narrow in focus.
Watershed Approach Affords Opportunity to Address Jurisdictional Issues
Staff from three of the six States we interviewed said the watershed approach
enables them to better resolve jurisdictional issues. Since watershed boundaries
16
-------
cross political lines, such as State and county borders, the watershed approach
better enables EPA to become involved in resolving issues that cross
jurisdictional boundaries. Staff noted instances of EPA assistance in resolving
disputes between political entities, such as municipalities.
Watershed Approach Affords Opportunity to Include Additional Programs
A number of State staff indicated the watershed approach affords EPA the
opportunity to incorporate additional pollution control programs in its strategy of
achieving clean water. Some interviewees added that they would like to see other
programs incorporated into the watershed approach, such as efforts related to
ground water, water quantity, and air programs. The watershed approach is
designed to be holistic in nature, and therefore focuses on comprehensive
solutions to water quality problems. This can be particularly useful when a source
of contamination is located outside of a watershed. For example, mercury emitted
into the air by coal-fired utilities can result in mercury being deposited in water
bodies hundreds of miles away. Mercury deposition in waters is one of the
leading causes of fish advisories throughout the country.
Stakeholders Identified a Variety of Obstacles
Stakeholders cited a number of obstacles that EPA must contend with to ensure
increased adoption and implementation of different watershed approach mixes.
By not addressing these obstacles, the environmental benefits of the watershed
approach may not materialize. Details on these obstacles follow.
Generating and Maintaining Public Involvement Challenging
While recognizing that the success and value of the watershed approach depends
on enlisting community involvement and support, officials from some watershed
organizations noted that one of the biggest challenges facing EPA is encouraging
and maintaining public involvement. For example, an official from the River
Network, a national environmental organization
that supports the watershed movement, stated the
public must have a fundamental understanding of
what their watershed is and understand how their
behaviors impact the watershed. An official from
another organization said it is much easier to get
people involved and become active in their
watershed when there is a perceived problem; it is
difficult to mobilize people in the absence of a perceived crisis. A representative
from yet another watershed organization said that problems generally motivate
people and thus drive involvement.
A representative from one watershed organization suggested that EPA work more
closely with local organizations in efforts to educate the public. For example, a
" The biggest challenge
facing EPA is watershed
education for the public...
Once the public has a better
understanding of these
issues, more is possible."
President, River Network
17
-------
representative from the Center for Watershed Protection said that while EPA has
some difficulty in conveying certain messages that are controversial, enlisting
local organizations to get out controversial messages can be more effective since
messages coming from local groups may come across as more credible.
Improved Coordination and Communication Needed
A number of State staff suggested that channels of coordination and
communication with EPA need to improve. Water staff from four of the six
States we visited were unsure as to what the watershed approach is, what EPA's
plan is for implementing it, or what is expected of the States. For example, staff
from one State said it is difficult to transition from a program-by-program
approach to a "place-based" one, and they were unclear how the existing
regulatory programs fit into the watershed approach. Staff from another State
said that EPA should better communicate what the watershed approach means to
specific stakeholders. These State staff implied that the benefits of the watershed
approach have not been demonstrated, and local decision makers need to be
educated as to the benefits before the approach can be successful.
Interviewees from States and watershed organizations indicated that EPA must do
a better job of sharing information with stakeholders. For example,
representatives from three of the four umbrella environmental organizations
interviewed (River Network, Southeast Watershed Forum, and Center for
Watershed Protection) indicated that EPA could do a better job of disseminating
the existence of successful watershed projects and case studies. Stakeholders rely
on the successful experiences of others when designing and implementing
projects - they do not want to "reinvent the wheel." One representative indicated
watershed organizations would implement successful projects more quickly if
they thought they would work in their geographical area. EPA has published a
number of successful practices in its "Nonpoint Source Success Stories" series.
Also, EPA uses a listserver, Watershed-News, to inform stakeholders of topics of
interest, although EPA estimates only about 2,600 of approximately 6,000 local
watershed groups belong to the listserver.
Staff from three of the six States also expressed confusion regarding the scale
chosen by EPA to measure results. EPA selected the 8-digit Hydrologic Unit
Code (HUC) to measure results, because this represents a natural geographic
delineation of hydrologic regions in the United States and the availability of data
at this scale. There are 2,262 8-digit HUC watersheds in the United States, with
an average of 1,563 square miles and 1,635 miles of rivers and streams in each
watershed. However, staff from Colorado, Georgia, and Ohio were unclear why
EPA chose the 8-digit code. They were concerned as to the scale at which the
code would measure water quality improvements, or how water improvements at
smaller scales would be accounted for within the larger scale. Staff added that it
is difficult for them to manage water quality on such a large scale because it does
18
-------
not match up well with the work done at the State level, since States generally
implement programs on stream segments or reaches.
Watershed Approach Increases State Workload
A number of State staff we interviewed noted the watershed approach increases
their workload. While EPA is encouraging States to adopt watershed approaches,
it is still also requiring them to account for programmatic outputs required by the
Clean Water Act. For example, staff from one State noted undertaking required
programmatic activities, such as NPDES permitting, could detract from other
activities directly focused on achieving environmental results - like working with
stakeholder groups to implement watershed projects. Staff added that the
programmatic work and the watershed approach activities are both full-time jobs,
and working on one takes away from the other.
Funding Considered Necessary for Additional Workload
Because of the additional workload previously discussed, staff from four of the
six States we interviewed said more funding is needed because the watershed
approach requires them to do things beyond what is required by the Clean Water
Act. Because these activities take time away from statutory requirements, they
believe they should receive additional funding.
Officials from watershed groups emphasized the need for long-term funding
commitments. As noted by one watershed organization representative, temporary
project funding is the "bane" of watershed organizations because it makes it
difficult to plan for fixed expenses, such as salaries and overhead. Recognizing
the importance of sustained funding to watershed organizations, Office of Water
established a Sustainable Finance Team to build the capacity of watershed
organizations to develop and implement finance strategies to obtain and leverage
funding. The team is also working with organizations that provide funding to
increase their awareness of the economic benefits of watershed management.
Watershed Organizations Need Assistance Developing Plans
Many stakeholders said technical assistance
is needed in developing watershed plans (see
Chapter 2). Recognizing the importance
that the Agency has placed on developing
comprehensive watershed plans, the Office
of Water has established a Watershed
Planning Team to assist stakeholders by
describing the elements of a comprehensive
watershed plan and developing a Web-based
tool to facilitate the development of such plans. Despite EPA's efforts,
stakeholders indicated they need more assistance. In many cases, locally driven,
" The two key steps needed to
solve nonpoint source problems
within a watershed context are the
development of a watershed-based
plan.. .and the actual
implementation of the plan."
EPA Nonpoint Source Guidelines,
October 23, 2003
19
-------
citizen-led watershed organizations consisting of volunteers do not have the
technical expertise and skills to develop comprehensive watershed plans that met
EPA requirements. States and EPA rely on these watershed organizations to
develop watershed plans, and therefore the expertise to write such plans is critical.
Watershed plans have become more important in the past few years. EPA now
requires States to direct a significant portion of the States' nonpoint source
funding to watershed plan development.
20
-------
Chapter 4
EPA's Watershed Plan Hampered by
Inadequate Preparation
EPA needs to improve key aspects of its strategic planning process for the
watershed approach to maximize its benefits. Although the Agency has made
considerable progress in strategic planning, improvements are needed in three
critical phases:
Identifying the baseline, or current level of achievement.
Setting meaningful goals.
Developing a plan that will move performance from the baseline to goals.
EPA's baseline is undermined by limitations in underlying data and analyses.
Also, EPA did not set realistic goals, which may limit its ability to manage water
programs and activities. Finally, regional plans are inadequate to implement
EPA's national strategies to improve water quality on a watershed basis.
Improvements in these key aspects of EPA's strategic planning process are
needed to help achieve its watershed goals.
Strategic Planning Critical for Allocating Resources
Strategic planning is a required and critical component of organizational success.
The Government Performance and Results Act requires executive agencies to
develop strategic plans. Moreover, strategic planning is critical because it:
Provides a roadmap and direction and sets priorities.
Allocates resources for maximum efficiency and effectiveness.
Establishes measures of success so that progress can be measured.
Gains commitment to the plan by involving the organization in its
development.
Coordinates actions of diverse parts of the organization into unified programs.
While the content and style of strategic plans vary based on industry and type of
organization, the planning process is fairly standardized. The process consists of
sequential phases that help an organization answer critical questions that, when
properly addressed, will result in an adequate strategic plan. Successful strategic
planning is accomplished by answering the four critical questions in Figure 4.1.
We evaluated EPA's strategic plans, listed below, to determine how well EPA has
addressed these questions.
21
-------
Figure 4.1
Strategic Planning Process
Critical Questions
Where are we
now?
Where do we want to
be?
How will we get
there and who
must do what?
How are we
doing?
Baseline Water
Quality Status
of Watersheds
Improving Water
Quality on Watershed
Basis (National
Outcome Measures)
Implementation
and Regional
Plans
Performance
Measurement
System
Critical Watershed Approach Planning Phase
Significant Progress Made in Strategic Planning
EPA made important strides when developing subobjective 2.2.1 in its 2003-2008
Strategic Plan, which focused the goal of improving water quality on a watershed
basis as part of the overall Agency strategic planning process. These strategic
plans are found in the following documents:
EPA's FY2003 - FY2008 Strategic Plan
Implementation Plan for Subobjective 2.2.1
National Water Program Guidance for FY2005
EPA Regional Plans
Through these documents, the Office of Water presents its vision of using the
watershed approach to improve water quality. This vision is to be accomplished
through a three-part national strategy and regional plans involving:
(1) implementing core water programs, (2) accelerating watershed protection, and
(3) applying an adaptive management framework. Progress made toward
achieving water quality on a watershed basis is assessed using two national
outcome measures:
Table 4.1: National Outcome Measures
Watershed
Restoration
Goal
(Baseline: 453 watersheds as of 2002) 500 watersheds by 2005 and
600 by 2008 where 20 percent of water segments in the watershed are
assessed and water quality standards are met in at least 80 percent of
the assessed water segments.
Watershed
Improvement
Goal
(Baseline: 0) 200 watersheds by 2008 where all assessed water
segments maintain their quality and at least 20 percent of assessed
water segments show improvement above conditions as of 2002.
22
-------
Limitations in Data and Analysis Undermine Baseline Measure
The first critical element of strategic planning is to determine the baseline water
quality status of watersheds nationwide. An accurate assessment of the baseline
is critical because it provides the foundation upon which EPA will develop its
watershed restoration and watershed improvement goals, and determine the
strategies, programs, and activities needed to improve water quality on a
watershed basis. A flawed baseline indicator could limit EPA's ability to set and
achieve useful goals.
Data Sources Contribute to Uncertainty of Baseline Estimate
In developing a baseline for its watershed restoration goal, EPA faced challenges
imposed by data sources in attempting to convey the nation's water quality in a
single number. The limitations of data sources contribute to the uncertainty of the
baseline estimate. We were unable to assess the impact of data limitations on the
baseline estimate, but they diminish the reliability that Congress, EPA, and the
public can place on EPA's ability to plan and run its clean water programs.
In developing the measure, EPA relied on State Figure 4.2
assessments of the quality of their waters as
contained in the biennial national 305(b) report to
Congress on water quality. However, the report has
a number of limitations. States assessed only a
small portion of their waters as of 2000 (see Figure
4.2). In addition, this national number disguises a
wide variability from State to State (States reported
different things). Moreover, because States assess
their waters in different ways, using a mixture of
objective and subjective information, the
assessments are not comparable across State
borders. Some States also extrapolate from
individual monitoring points to a larger coverage,
resulting in significant variances. These problems
have already been noted in a number of EPA OIG
and U.S. Government Accountability Office
Reports. In 2001, a Committee of the National
Academy of Sciences recommended that EPA
develop uniform, consistent approaches to data
collection. Over the years, EPA has attempted to
increase the uniformity of State 305(b) reports.
699,946 m i les = 1 996 assessed
Total miles: 3,692,830*
nil
"si 11-.. i o'^'.'issed
1ซ31*8 .i ฆฆฆ i; i; 11 ฆ - . .... ssed
k': Total miles: 3,662,255^
mil's:-:. - I ^'-i ..twrised
ITill'-:. -1 r-o-l. I
1994
Tota
1992 m 642,881 miles = 1 8% assessed
SB Total mites-. 3,551,247ฎ
aSour ! ate and tribal Section 30 5(b)
rvf. rt
^Sour ซ* l-aซ. ปiate and tribal Section 30 5(b*J
^Souixt-- 1 fate and tribal Section 305(b)
reports.
*%our e 1 tate and tribal Section 30 5(b)
fM| -,rt
-Source: 199.2 state and tribal Section 305(b)
r>=nh;.
23
-------
Some Uncertainty Also Results from EPA's Analysis
Although EPA developed a process to estimate the watershed restoration baseline,
the process contributes an unknown degree of error in the baseline. These data
processing steps, which use 305(b) data, may have introduced errors into the
baseline estimate by:
Excluding certain classes of waters.
Combining classes of water without adjusting for significance.
Screening classes of waters differently.
Employing inconsistent scales to estimate data sufficiency.
EPA's process is detailed in Appendix C.
Excluding Classes. When characterizing the quality of complex systems as "the
nation's waters" or "watersheds," EPA used three waterbody types to classify
water resources: rivers, lakes, and estuaries. It did not include other classes of
water, such as wetlands and groundwater. Although it may be acceptable to
eliminate certain classes of water if they will not materially affect the estimate,
ideally, a national assessment should be based on the entire "population" of
waters. EPA also based the "rivers" portion of the baseline assessment on
"perennial" rivers - that portion of rivers that flow year-round. The Agency
excluded "intermittent" and "ephemeral" waters, which can constitute a large
share of the total miles of rivers in some parts of the country.
Combining Classes. Because watersheds are composed of different types of
waters, an overall assessment of quality required EPA to combine distinctly
different waterbody types. EPA averages the assessment numbers of each of the
three waterbody types (rivers, lakes, and estuaries). Although simple and
straightforward, this approach disregards the relative size of the classes; in EPA's
approach, all three classes are treated equally even in watersheds dominated by
one of them. Averaging waterbody types without regard for relative size may
introduce error into the baseline estimate by inappropriately including or
excluding certain watersheds from being counted as part of the baseline.
Screening Differently. EPA screened out watersheds based on the amount of
available information to prevent watersheds with too little data being labeled as
meeting watershed goals. EPA applied a "data sufficiency" requirement that
excluded assessments of "rivers" and "small lakes" if the State had assessed less
than 20 percent of the river miles or lake acres in that watershed. However, it is
important to analyze data uniformly to achieve non-biased results. EPA applied
the "data sufficiency" requirement to some but not all classes of waters (such as
"estuaries" and "large lakes"). We were unable to establish EPA's reason for this
differential treatment, and did not assess whether it was reasonable. This
inconsistent treatment could introduce error into the baseline if the true state of
excluded rivers or small lakes differs significantly from included estuaries and
large lakes in a watershed.
24
-------
Scaling Inconsistently. EPA determined whether there was sufficient data to
derive an estimate by calculating an "assessment ratio" (e.g., the number of miles
of rivers assessed divided by the number of miles of rivers in the watershed).
For such a calculation to be valid, both of these numbers should be reported at the
same scale. However, EPA's analysis procedures did not ensure that assessment
ratios were determined using consistent scales. Because the numerator and
denominator of the assessment ratios were sometimes calculated using different
geographic scales, EPA's calculations may be mathematically incorrect. It is
possible that some portions of rivers in some watersheds were excluded from the
denominator during calculation of the assessment ratio as a result of this
inconsistency. We were unable to establish EPA's rationale for these exclusions.
Unachievable Goals Hamper EPA's Ability to Guide Activities
The second critical element of strategic planning is setting challenging yet
realistic goals. The goals EPA developed in its initial strategic plan for
subobjective 2.2.1 were unachievable because the Office of Water did not
effectively communicate that EPA Regions would be responsible for achieving
national watershed goals. Also, Regions did not realize they would subsequently
be asked to direct their programs and activities to achieve those goals. As a
result, there is a significant disconnect between the goals for subobjective 2.2.1
and what the Agency is able to achieve. This disconnect prevents EPA from
effectively managing its water programs and activities.
Office of Water based the national watershed goals for 2005 and 2008, in part, on
best professional judgment of national water program managers, as well as trend
analysis of watershed data previously collected. The goal-setting process was
collaborative and also included input from other EPA media offices, EPA
Regions, and a State/Tribal Steering Committee including representatives from
the Association of State and Interstate Water Pollution Control Administrators.
In its strategic plans, the Agency published the following watershed restoration
and watershed improvement goals:
Table 4.2: Goals for Number of Watersheds to Be Restored or Improved
Baseline
(2002) 2005 Goal 2008 Goal
Watershed Restoration Goal 453 500 600
Watershed Improvement Goal 0 - 200
25
-------
In December 2003, after EPA
published its national watershed
goals, the Office of Water again
consulted with regional offices,
this time asking them to estimate
their regional commitments for
restoring and improving a fixed
number of watersheds equivalent
to meeting national goals.
However, the initial estimated
commitments by Regions were
collectively unable to meet the
goals that were established
earlier that year. Those estimates
were published in the April 2004
National Water Program
Guidance. Office of Water asked
EPA Regions to continue
consulting with States about
regional commitments during the
summer of 2004. Despite
continued consultation with
States, the final regional
commitments, made available
during October 2004, fell drastically short of national watershed goals (see
Figures 4.3 and 4.4). For the watershed restoration goal, EPA was able to commit
to a gain of only nine watersheds by 2005 and 15 by 2008, instead of the 47 and
147 watersheds, respectively, it published in the strategic plan. For the watershed
improvement goal, EPA was able to commit to only 47 instead of the 200 it
published in the strategic plan.
The key reason explaining this disconnect between goals and commitments is that
the national watershed goals were established without a common understanding
among EPA headquarters and regional offices that each would be responsible for
achieving those goals. In particular, the initial request by the Office of Water
asked only for "estimates," instead of "commitments," which would have
indicated to the Regions that they would be responsible for achieving watershed
goals. According to an Agency official, because this was the first time EPA set
national watershed goals and expressed them as regional commitments, EPA
regional personnel involved in the planning process may have felt disconnected
from the national goals and believed they were Office of Water goals, not regional
goals.
Figure 4.3: Comparison of Regional Commitment
and Watershed Improvement Goal
200
150
100
50
0
l Regional
Commitment
I Watershed
Improvement
FY2008
(Baseline = Zero)
Figure 4.4: Comparison of Regional
Commitments and Watershed Restoration Goals
w 600
I Regional
Commitment
I Watershed
Restoration
Goal
26
-------
Regional Plans Inadequate to Implement National Strategies
The third critical element of strategic planning is to develop an implementation
plan that allows Office of Water and EPA Regions to move efficiently from the
baseline to its strategic goals. However, EPA regional plans are not adequate to
implement EPA's national strategy to improve water quality on a watershed basis
and achieve watershed restoration and watershed improvement goals. EPA
regional plans were recommended by the Office of the Chief Financial Officer to
provide "road maps" outlining their contributions toward achievement of the
Agency's goals. Further, Office of Water specifically requested each Region to
identify how national strategies will be implemented at regional levels.
Our analysis of the regional plans indicates that they are inadequate to support
achievement of watershed restoration and watershed improvement goals. We
analyzed plans from Regions 1, 4, 7,
and 10 to determine whether each
plan acknowledged the watershed
restoration and watershed
improvement goals and each part of
EPA's three-part national strategy
(see box) was incorporated at the
regional level.
Of the four regional plans analyzed, only Region 7 acknowledged the national
watershed restoration and watershed improvement goals and the regional
commitment. Regions 1, 4, and 10 acknowledged the national watershed
restoration and watershed improvement goals but not the regional commitments.
All four regional plans identified the core water programs that would be
implemented to achieve subobjective 2.2.1. However, the regional plans did not
emphasize or provide specific strategies, procedures, or policies for implementing
core water programs on a watershed basis. The regional plans also lacked specific
regional strategies, policies, or activities to support accelerating local watershed
protection efforts and implementing an adaptive management approach.
Despite the shortcoming noted, Regions are engaging in watershed planning.
Region 1 commented they have been using the watershed approach for so long
that some watershed-oriented activities are not fully reflected in the regional plan.
Regions 4 and 7 developed water division operational plans to help address the
lack of details regarding implementation of the watershed approach in the
regional plans. Although these individual efforts are helpful, regional planning
efforts remain inadequate to implement EPA's national strategy.
National Strategy to Improve Water
Quality on Watershed Basis
1) Implement Core Water Programs
2) Accelerate Watershed Protection
3) Apply Adaptive Management Framework
27
-------
Chapter 5
Progress Made Developing Performance
Measurement System, but Improvements Needed
EPA took important steps to advance the watershed approach by developing a
performance measurement system for subobjective 2.2.1. However, issues still
need to be addressed. EPA did not develop measures to evaluate key programs
and activities, including implementation of some core water programs on a
watershed basis. Further, while EPA's national outcome measures were relevant,
they were not understandable, comparable, and reliable. Without these
improvements, the ability of EPA's performance measurement system to convey
useful information on EPA's strategy to improve water quality on a watershed
basis will be hampered.
Aspects of Watershed Approach Performance and Measurement
EPA plans to improve water quality on a watershed basis by implementing its
three-part national strategy involving voluntary and regulatory programs. The
logic model in Figure 5.1 details the relationship between critical aspects of
performance, both in general terms and using the watershed approach, and the
system of measures to assess each critical aspect of performance.
Figure 5.1
Watershed Approach Performance Aspects and Measurement System
Program Activity Measures
1) Watershed Restoration Goal
2) Watershed Improvement
Goal
National Outcome Measures
INPUTS ~
ACTIVITIES
OUTPUTS
OUTCOMES
1) Dollars
2) Staff
3) Technology
4) Capital
Improving Water Quality on A
Watershed Basis
(Subobjective 2.2.1)
1) Implement Core Programs,
Including On Watershed Basis
a) Water Quality Standards
b) NPDES
c) Monitoring/Assessment
d) Total Maximum Daily
Loads and other plans
e) Nonpoint Source
f) State Revolving Funds
2) Accelerate Watershed
Protection
a) Support local watershed
protection efforts
b) Critical watershed
protection
3) Apply Adaptive Management
Framework
Implement 3-part National
Strategy
2) Accelerate Watershed Protection
a) Technical assistance, development
of local watershed plans, formation of
watershed groups
b) Implementation of Targeted
Watershed Grants programs and
activities
3) Adaptive management
1) Core Program Outputs (Examples)
a) Pollutant Criteria Recommendations
b) Issue Watershed Permits
c) Guidance to help States Improve
Monitoring
d) Develop Watershed TMDLs
e) Distribution of NPS funds to develop
watershed management plans
f) Distribution of SRF funds for water
pollution control projects
Outputs of 3-part National Strategy
-------
It is important for the Agency to measure critical aspects of performance (outputs
and outcomes) so that it can determine what the program is accomplishing and
whether the intended results are being achieved. Output performance measures,
which EPA refers to as program activity measures (PAMs) for subobjective 2.2.1,
are most beneficial when they fulfill a set of minimum characteristics and
measure goods and services produced by a program or organization. Complete
PAMs are able to indicate what each critical strategy and program is
accomplishing through an indicator, numeric target, and timeframes. Outcome
performance measures help the Agency demonstrate what it is achieving as a
result of its programs and activities.
Performance Measurement System Incomplete
Office of Water made significant progress developing a performance
measurement system with both output and outcome performance measures.
However, EPA currently lacks measures to provide performance information for
all critical strategies and programs, including implementation of some core water
programs on a watershed basis.
To evaluate the outputs from its national strategies, programs, and activities,
Office of Water developed 35 PAMs, which are listed in Appendix D. The
PAMs are the basis for monitoring progress in implementing programs to
accomplish improved water quality on a watershed basis. Complete PAMs
provide useful performance information about all critical strategies, programs,
and activities, and provide an indicator of what is being accomplished, a numeric
target, and a timeframe. However, EPA's PAMs were incomplete because they
could not provide tracking or performance information for all critical national
strategies. Moreover, while EPA could track programmatic information for the
water quality standards, NPDES, and State Revolving Fund programs, it could not
provide performance information about how these programs were being
implemented on a watershed basis.
Of the 35 PAMs, 10 provide information on directing core programs on a
watershed basis, and only 6 of those 10 are "performance measures" in the sense
that they include a specific target and timeframe. The remaining 25 measures
provide information about the core programs but not in terms of watershed
aspects. Fifteen of those 25 are true performance measures that include a target
and timeframe. The results of this analysis are shown in the following table for
two of the three critical national strategic objectives (there were no PAMs for the
third objective on applying adaptive management).
29
-------
Table 5.1: Watershed-Based Performance and Tracking Measures
by National Strategy and Key Program/Activity for Subobjective 2.2.1
Critical National
Strategic Objectives
Key Program/Activity
Watershed-Based
Measures
Non-Watershed (Core
Program)-Based Measures
Performance
Measures
Tracking
Measures
Performance
Measures
Tracking
Measures
1. Implement Core Water
Programs; including
innovations that apply to
programs on watershed
basis
Water Quality Standards
0
0
5
1
Monitoring/Assessment
2
0
2
0
Watershed Planning,
TMDL, Nonpoint Source
3
2
1
4
NPDES Program and
National Regulations
0
2
5
4
State Revolving Fund
0
0
2
1
2. Accelerate Watershed
Protection
Support local watershed
protection efforts
0
0
0
0
Initiate or strengthen
watershed protection for
critical watersheds/
waterbodies
1
0
0
0
Total
6
4
15
10
EPA developed a mixture of performance measures for its core programs. Some
do not have a watershed basis, while others are helpful in advancing that
approach. An example of a measure for a core program implemented on a
watershed basis is PAM 68: "Number of watersheds in which a watershed
permit(s) has been issued and the number of States issuing NPDES permits using
a rotating basin process." EPA is currently only tracking this measure. If it were
to develop a meaningful target and timeframe, this measure could provide more
helpful performance information to program managers.
EPA does not have performance measures for some key programs and activities.
For instance, the NPDES program has tracking measures but no performance
measures on a watershed basis; the water quality standards and State Revolving
Fund programs have neither on a watershed basis. Also, EPA does not have
performance and tracking measures to evaluate its support for local watershed
protection efforts.
National Outcome Measures Need Improved Design
EPA developed two national outcome measures to assess the environmental
impact of the watershed approach as implemented through EPA's national
strategy: the watershed restoration and watershed improvement goals (see Figure
5.2). EPA uses these outcome measures to reflect its progress toward improving
water quality on a watershed basis and determine the environmental impact of the
watershed approach as implemented through its three-part national strategy.
Outcome measures are critical to demonstrating what an organization is
accomplishing, whether results are being achieved, and providing a structured
approach for focusing an organization's strategic plans. These measures need to
be:
30
-------
relevant - relates to Agency objectives, attributable to programs and activities,
and useful to the Agency;
comparable - allows comparison over time to indicate future performance;
understandable - clearly and consistently defined and measurable; and
reliable - represents what it claims to indicate; extent to which data and
analysis are free from error; and verifiable.
The national outcome measures are "relevant" because they relate directly to
EPA's objective of improving water quality on a watershed basis and are
attributable to the Agency's
national strategies, programs, and
activities. Further, these measures
are useful for the Agency to
demonstrate improvements to
water quality on a watershed basis.
The watershed restoration and
watershed improvement goals do
not completely fulfill the
"understandability" criterion,
because they leave key terms
undefined. Incomplete or
ambiguous definitions of key terms
may prevent the performance
measure from yielding valuable
information to EPA. EPA
identified the key terms "watershed" and "water quality standards." However, it
did not identify such key terms as "assessed water segment," "maintain their
quality," and "show improvement." While States may provide some of these
definitions, it is important for these terms to be defined on a national basis.
Leaving key definitions undefined may limit the usefulness of the outcome
measure.
The watershed restoration and watershed improvement goals also appear to fulfill
the "comparability" condition because they are based on a simple calculation of
the increase or decrease in the number of watersheds that meet certain conditions,
which can be compared with a baseline to assess progress. However, the
comparability of the measure is undermined for two reasons. First, EPA does not
clarify exactly how it will compare watershed data across time periods or how
variations in monitoring coverage affect the comparability of the measures. For
instance, increases above the baseline for the watershed restoration goal could
simply represent increases in the coverage of monitoring as opposed to actual
improvements in water quality. Watersheds where greater than 80 percent of
assessed water segments already meet water quality standards could be counted as
meeting goals by simply increasing the percent of assessed waters above the data
Figure 5.2
National Outcome Measures
Watershed Restoration Goal
2005 Goal'. 500 watersheds
2008 Goal'. 600 Watersheds
Baseline (as of 2002)'. 453 Watersheds
Conditions'. At least 20 percent of waters in
watershed must be assessed and at least 80
percent of assessed waters must attain water
quality standards
Watershed Improvement Goal
2008 Goal'. 200 watersheds
Baseline (as of 2002)'. 0 watersheds
Conditions'. All assessed water segments maintain
their quality and at least 20 percent of assessed
water segments show improvement above
conditions as of 2002.
31
-------
sufficiency condition of 20 percent through monitoring. While technically
meeting the watershed restoration goal, actual water quality improvements have
not occurred. Second, there are shortcomings regarding baselines, such as
limitations in data and analysis, as discussed in Chapter 4.
The "reliability" of the watershed restoration and watershed improvement goals
needs to be enhanced to ensure the usefulness of the goals as national outcome
measures. A reliable outcome measure represents what it claims to indicate and is
based on data and analysis that are verifiable. As noted in Chapter 4, there are
limitations in the data used by EPA to establish its baseline for meeting
conditions. Further, EPA data analysis procedures introduce errors that impact
the reliability of the measures. The reliability of both the watershed restoration
and watershed improvement goals is significantly diminished due to the extent of
errors associated with the data and analysis.
32
-------
Chapter 6
EPA's Commitment Must Be Sustained by
Addressing Critical Issues
EPA has taken various actions to advance the watershed approach but, based on
continuing and anticipated challenges, needs to do more. While addressing the
four elements that were the focus of our report may not guarantee success,
ignoring them will hinder the ability of EPA to achieve clean and healthy water.
If EPA is committed to further advancing the watershed approach, it needs to
continue to integrate watershed principles into core water programs, address
obstacles facing stakeholders, and improve key planning steps and performance
measures. Due to the overlapping nature of the issues discussed in Chapters 2
through 5, we are providing overall conclusions as well as recommendations in
this chapter.
EPA needs to make additional progress integrating the watershed approach into
some of its core programs to assist EPA and its partners in achieving the nation's
clean water goals. For example, increasing geographic focus and stakeholder
involvement may increase the complexity, resources, and time required to
implement programs. This, in turn, may result in the need for additional
resources. Obtaining such resources in the current budget environment may be
difficult.
EPA needs to make additional progress in partnering with stakeholders.
Stakeholders, such as States and watershed groups, play a critical role as partners
with EPA in furthering clean water goals through the watershed approach. EPA
recognizes the importance of these groups and has taken several steps to involve
them, but faces an immense task in coordinating with so many stakeholders. EPA
needs to coordinate and communicate with these vital stakeholders to advance the
watershed approach.
EPA needs to make further progress in planning to implement the watershed
approach and measuring its effectiveness. Planning and measurement are critical
to ensuring that the Agency provides direction, sets priorities, allocates resources
efficiently and effectively, and determines whether results are being achieved. To
continue advancing the watershed approach, EPA needs a sustained commitment
to make improvements to critical areas of the strategic plan and measurement
system. Without these improvements, EPA's progress will continue to be limited.
33
-------
Recommendations
If EPA is committed to the watershed approach, we recommend that the Assistant
Administrator for Water:
6-1 Continue to integrate watershed activities into its core water programs.
Specifically, EPA could:
Commit the necessary resources to support, where appropriate, the
development of watershed permits, watershed TMDLs, and watershed
plans;
Evaluate the benefits and costs of watershed permitting and watershed
TMDLs; and
Work closely with States to ensure that the CWSRF is used as an
additional source of funding for nonpoint source and other watershed-
related projects.
6-2 Work in partnership with stakeholders to ensure obstacles with implementing
the watershed approach are addressed. Specifically, EPA could:
Work closely with watershed advocacy groups to educate the public
about the value of water resources and how public participation is
critical to safeguard these resources;
Improve coordination and communication with States and watershed
organizations to help ensure the success of the watershed approach in
achieving clean and safe water; and
Provide technical assistance to stakeholders, particularly in developing
watershed plans.
6-3 Continue to refine and improve key aspects of its strategic planning
process. Specifically, EPA could:
Improve the accuracy and reduce the uncertainty of the baseline;
Ensure that Office of Water and EPA Regions clearly understand their
roles in setting and achieving national watershed goals; and
Improve Regional plans to help achieve watershed goals.
6-4 Continue to improve key aspects of its performance measurement system.
Specifically, EPA could:
Ensure that the performance of all critical national strategies and
implementation of core water programs on a watershed basis can be
assessed since EPA does not have performance measures for some key
programs and activities; and
Improve the design of the watershed restoration and watershed
improvement goals to be more understandable, comparable, and reliable.
34
-------
Agency Comment and OIG Evaluation
EPA generally concurred with our recommendations and in some cases has taken
actions to address them. EPA's response to our draft report recommendations is
in Appendix E, and our OIG evaluation of the response is in Appendix F. The
OIG has incorporated technical corrections and clarifications provided by EPA
into the final report as appropriate.
35
-------
Appendix A
Details on Scope and Methodology
We conducted our evaluation from October 2003 through February 2005 in accordance with
Government Auditing Standards, issued by the Comptroller General of the United States. This
evaluation focused only on EPA's efforts regarding the watershed approach and not those of
other Federal agencies, States, or watershed groups. We nonetheless interviewed State and
watershed groups to evaluate EPA's role in watershed efforts.
We evaluated the watershed approach by analyzing four key elements that help ensure its
success:
Integration of watershed activities into core water programs;
Partnerships;
The process of strategic planning for the watershed approach; and
The development of performance measures.
These four elements came from the synthesis of numerous EPA documents reviewed by the
team. Details on how we performed our evaluation for each element follow.
Integration
We evaluated EPA's efforts to integrate the two of three guiding principles (Partnerships,
Geographic Focus, and Sound Management Techniques) of the watershed approach into EPA's
programs. We did not evaluate Sound Management Techniques because (1) sound management
techniques are not unique to the watershed approach; and (2) EPA is employing a new
management technique for the watershed approach, adaptive management. Since 2005 is the
first year that the adaptive management process will be applied to the watershed approach, we
did not evaluate the process.
We selected the six core water programs (NPDES, TMDL, Nonpoint Source, Water Quality
Monitoring, Water Quality Standards, and CWSRF) highlighted in subobjective 2.2.1 of EPA's
Strategic Plan. We determined the extent to which watershed approach principles have been
integrated into the programs. To do so, we interviewed all four of the primary Office of Water
office directors, as well as the managers and program staff in each the six core programs, and
spoke with EPA regional water program directors and staff from Regions 1, 4, 7, and 10. We
obtained and analyzed program guidance, regulatory and statutory requirements, and guidelines
for each program. We also analyzed the recommendations of EPA's Program Integration Team
and other Office of Water efforts to identify any other EPA actions involving integration of the
watershed approach into programmatic activities.
Partnerships
Using a structured interview instrument, we interviewed representatives of umbrella
environmental and watershed organizations. Using a different structured interview instrument,
36
-------
we interviewed water staff from six States, to identify both advantages/opportunities and
challenges that EPA needed to address regarding the watershed approach. The table notes the
States and watershed organizations interviewed:
States
Umbrella Environmental
Organizations
Watershed Organizations
Massachusetts
Southeast Watershed Forum
Charles River Watershed Association
Georgia
River Network
Cherry Creek Stewardship Partners
Oregon
Center for Watershed Protection
Rathburn Land and Water Alliance
Colorado
Georgia River Network
West Creek Preservation Committee/
White Oak Creek Watershed Partners
Ohio
Iowa
Strategic Planning
We researched academic and professional textbooks, the Government Performance and Results
Act, and information from the National Partnership for Reinventing Government to identify the
critical process elements that must be addressed to ensure successful strategic planning. We
evaluated EPA's Fiscal 2003-2008 Strategic Plan; Implementation Plan for subobjective 2.2.1;
the National Water Program Guidance for Fiscal 2005; and 2004 EPA Regional Plans for
Regions 1, 4, 7, and 10 against these criteria. We also reviewed EPA Regional Plans and
guidance from the Office of the Chief Financial Officer. Further, we interviewed Office of
Water program directors for the Office of Wetlands, Oceans, and Watersheds; Office of Science
and Technology; Office of Wastewater Management; and the Office of Ground Water and
Drinking Water; as well as four regional water directors and other senior policy advisors, to
understand the strategic planning process and determine how well strategic planning for the
watershed approach occurred. We reviewed EPA's Data Processing Techniques to confirm how
EPA calculated the baseline number of watersheds meeting assessment and attainment
conditions for subobjective 2.2.1. Our analysis also compared the watershed restoration and
watershed improvement goals for subobjective 2.2.1 for Fiscal 2005 and Fiscal 2008 against
EPA regional commitments.
In developing a baseline for its watershed restoration goal, EPA had data limitations. These
limitations contributed to the uncertainty of the baseline estimate. These limitations have been
noted in previous Government Accountability Office and OIG reports. We were unable to assess
the impact of data limitations on the baseline estimate. The data limitations did not impact OIG
findings and conclusions. More details on watershed data quality can be found in Chapter 4.
Performance Measurement
We researched reports from similar auditing organizations in Federal and foreign governments to
identify criteria relating to performance measurement by which to assess EPA's Management
Matrix System. We identified three distinct sets of criteria useful for assessing critical aspects of
EPA's system. First, we reviewed EPA's overall system to ensure that it is capable of
37
-------
communicating vital performance information about critical national strategies and programs
designed to improve water quality on a watershed basis. Second, we reviewed EPA's national
outcome measures, the watershed restoration and watershed improvement goals, to ensure the
measures are properly designed to convey performance information about the cumulative impact
of EPA's efforts to improve water quality on a watershed basis. Finally, we evaluated EPA's
PAMs to ensure that each measure contained an indicator, target, and timeframe to determine if
it could convey performance information about that program.
EPA's overall system of performance measures was evaluated to ensure that it is complete by
measuring those key factors, strategies, or activities that allow the Agency to improve water
quality on a watershed basis. The criteria for completeness can be found in a December 1997
report by the Office of the Auditor General of Canada regarding how Canadian federal
organizations can make performance measurement work.
We reviewed EPA's national outcome measures - the watershed restoration and watershed
improvement goals - to ensure the measures are properly designed to convey performance
information about the cumulative impact of EPA's efforts to improve water quality on a
watershed basis. Properly designed outcome performance measures, such as EPA watershed
restoration and improvement goals, should fulfill a set of criteria in order to be useful. We used
criteria developed by the Office of the Auditor General of Canada that require outcome
performance measures to be: (1) Understandable, (2) Relevant, (3) Comparable, (4) Reliable, and
(5) Practical. We chose to exclude the element of practicality; we were not able to determine
what each aspect of practicality meant in terms of EPA's national outcome measures.
We evaluated the individual PAMs to ensure that each measure contained an indicator, target,
and timeframe to determine whether it could convey performance information about that
program. We evaluated EPA's PAMs using criteria set forth by the Office of Management and
Budget in its 2003 Supporting Documentation to the Instructions for Completing the Program
Assessment Rating Tool (PART).
Management Controls
We identified the following management controls as applicable to our objectives: effectiveness
of program operations and validity and reliability of data. Review of compliance with applicable
laws and regulations was not applicable to this evaluation since the watershed approach is
voluntary and no laws and regulations apply to the actual approach.
Effectiveness of Program Operations
Although the watershed approach is not a program, we evaluated the effectiveness of the
watershed approach by evaluating the Office of Water's strategic planning process for the
watershed approach and evaluating watershed performance measures. Details on what we found
are in Chapters 4 and 5 of the report.
38
-------
Validity and Reliability of Data
Individual data measurements in databases were not directly analyzed or used to support findings
and conclusions. Our evaluation of the national outcome measure, which uses data from the
National Assessment Database in terms of establishing a baseline status of watersheds
nationwide, focused on data processing techniques and not the actual raw data. Furthermore, the
quality and reliability of data in the National Assessment Database were previously examined in
other EPA OIG and Government Accountability Office reports, some of which are discussed in
Chapter 4 and listed in this appendix under Prior Coverage.
Prior Coverage
Agency
Report
EPA OIG
"Stronger Leadership Needed to Develop Environmental
Measures for Clean Water State Revolving Fund,"
2004-P-00022, June 2004
EPA Office of Water
"A Review of Statewide Watershed Management Approaches,"
April 2002
U.S. Government Accountability
Office
"Watershed Management: Better Coordination of Data
Collection Efforts Needed to Support Key Decisions,"
GAO-04-382, June 2004
"Performance Budgeting: Observations on the Use of OMB's
Program Assessment Rating Tool for the Fiscal Year 2004
Budget," GAO-04-174, January 2004
"Program Evaluation: Strategies for Assessing Information
Dissemination Contributes to Agency Goals," GAO-02-923,
September 2002
"Water Quality: Key EPA and State Decisions Limited by
Inconsistent and Incomplete Data," GAO/RCED-OO-54,
March 2000
"Grant Programs: Design Features Shape Flexibility,
Accountability, and Performance Information,"
GAO/GGD-98-137, June 1998
"Evaluating a Performance Measurement System - A Guide for
the Congress and Federal Agencies," FGMSD-80-57,
May 12, 1980
Office of Management and Budget
"Performance Measurement Challenges and Strategies,"
June 18, 2003
Association of State and Interstate
Water Pollution Control Administrators,
Water Quality Monitoring Programs
"Status and Future of State Ambient Water Quality Monitoring
Programs," 2002
Committee to Assess the Scientific
Basis of the Total Maximum Daily Load
Approach to Water Pollution
Reduction, Water Science and
Technology Board, National Research
Council
"Assessing the TMDL Approach to Water Quality
Management," ISBN: 0-309-07579-3, 2001
39
-------
Appendix B
Clean Water Act Watershed Framework
Monitor/Assess l/l/QS Attainment
Assessment methodology
305(b) Report
303(d) List of Impaired Waters
EPA approves 303(d) list
TMDL
Continuing
Planning Process
(CPP)
WLAs to individual point sources
LAs to nonpoint sources
EPA approval
Watershed Plan
N PD E S permits for point sources
EPA discretion to object
Manage nonpoint sources
w/ Federal/State/other $
EPA's support for watershed approaches centers on the goals of the Clean Water Act. The
framework for watershed protection and restoration in the Clean Water Act is focused on the
attainment of water quality standards as set by the States and approved by EPA. Water quality
standards define the water quality goals for waterbodies, such as rivers and lakes. These
waterbodies are then periodically assessed against those standards and reported in Clean Water
Act section 305(b) reports submitted by the States. Waters not meeting water quality standards
are then placed on a list required by Clean Water Act section 303(d). For that list of waters,
TMDLs are developed that set load reduction targets for individual point sources and categories
of nonpoint sources. More and more watershed plans are being developed that take those load
reduction targets and expand them into a more detailed step-by-step approach that typically
includes broad stakeholder involvement. These TMDLs and watershed plans include the
eventual wasteload reductions required of individual NPDES permittees and nonpoint source
management practices used to obtain load reductions in a given watershed. Periodically, the
waters are assessed to determine the success of the remedial actions designed to reduce point and
nonpoint source loadings. Although not depicted, the CWSRF is a source of funding for several
types of projects found within watersheds. This cyclical return to the top of the Clean Water Act
Watershed Framework flowchart is the Continuous Planning Process.2
2 Office of Water Document Provided to OIG: The Watershed Approaches at EPA
40
-------
Appendix C
EPA's Data Processing Techniques for Developing
Watershed Restoration Baseline
Step by Step Data Processing Techniques for Developing a
Watershed Indicator of Water Quality Standards Attainment and the
Assumptions made as part of that process
RTI International - February 10, 2005
BRTI
1N T F ฆ Pf A.T IO N A 1.
1. Developing Watershed Indicators from the National Assessment Database
The current Office of Water outcome measure indicator of watershed water quality standard
attainment is presented in terms of scores for USGS subbasin (HUC8) watersheds. These
watershed scores are derived from underlying designated use assessment information organized
around reporting segments representing reaches along streams, lakes, and estuaries. This
segment-level information comes from databases States develop as part of the Clean Water Act
Section 305(b) process. EPA assembles available State assessment database into a National
Assessment Database (the NAD).
A number of data processing operations are performed on the primary NAD segment-level
information. Only certain waterbody types are used in the watershed indicator. These include
rivers, small (freshwater) lakes, large lakes, and estuaries. For freshwater lakes, a size threshold
of 2,000 surface acres is used to divide smaller lakes from large lakes. Through the 2000 305(b)
reporting cycle, the PC-based data systems EPA encouraged States to use included fields to
record the size of an assessment segment (for instance, 500 acres on an arm of a larger reservoir)
as well as a waterbody size (e.g., 10,000 acres for an entire large reservoir which could include
segment components on its arms and the main storage pool near the hydrostructure). This
waterbody size was the basis for assigned a lake segment as representing either a small lake or a
large lake.
The appropriate waterbody type use attainment data records for a given NAD baseline must have
georeferencing information adequate to assign the segments to USGS subbasins (HUC8s).
These HUC8 assignments can be derived from GIS coverages or from the contents of special
basin location fields in the State databases. Segments that appear in the NAD database but that
cannot be reasonably assigned to HUC8 units are not processed for the development of the
41
-------
watershed use attainment indicator. For the selected use attainment database records in the
NAD, there will usually be an assessment for a given segment involving multiple designated
uses. For the watershed indicator, derived overall water quality standards attainment status
information is developed for each segment. This involves generating a data record that records a
"worst case" condition looking at the assessment provided for all reported uses. For instance, if
an aquatic life use shows impairment, then the overall status for the segment would be rated as
impaired even if other uses, for instance, a primary contact recreation use, were deemed to be
attaining these other uses.
The overall water quality attainment status information is then grouped by waterbody strata
(rivers, small lakes, large lakes, and estuaries) and also grouped (or "sub-totaled") according to
HUCs or portions of complete HUCs contained within a State. These HUC-within-state
polygons are called STHUC units. The summary information for these STHUC units is the basis
for a series of data adequacy tests. For large lakes and for estuaries, these waterbody types are
considered "unique" - and the STHUC results for these waterbody type strata are always
maintained. For the rivers and for the small lakes, certain minimum levels of assessment effort
are required. Where information is available in the EPA (RF3-based) Total Waters Database, the
miles of stream assessed in a STHUC unit are compared with a value that is 20% of the
estimated miles of perennial streams in the STHUC polygon. For small lakes, the acres of small
lakes assessed in a STHUC unit are compared with a value that is 20% of the estimated acres of
lakes with surface acreage less than 2,000 acres in the STHUC polygon. Where these tests
cannot be performed (e.g., RF3 was never fully implemented for States in EPA Region 10), then
it is assumed that the data adequacy test is met. Where the river or small lake values show levels
of assessment effort below these thresholds, then the STHUC information is not used.
2. Step-by-Step development of the Watershed Indicator
For this step-by-step discussion, it is already assumed that the NAD data have been compiled,
and that the STHUC subtotals have been calculated.
Step 1. Capture prior HUC scores from previous cycles
Take the previous HUC scores develop from the previous GPRA reporting cycle
(the 2000 baseline scores) - and hold this until the final step. These scores will be used
to fill in the gaps where no information is available for a given HUC for 2002.
Step 2. Compile STHUC summaries for the major water types and perform data
adequacy test
Take STHUC summaries for 2002 reporting for use attainment information for the
4 major waterbody types of: rivers, small lakes, large lakes, estuaries. Apply data
adequacy test (see Assumptions: Data Adequacy Test) (where the TOTAL WATERS
information is available) for the STHUC results for rivers and small lakes.
For rivers, the test is relative to TOTAL WATERS (see Assumptions: Total Waters)
estimates of perennial stream/river miles. Keep all results for large lakes and estuaries.
42
-------
Keep river and small lakes results where TOTAL WATERS information is not available
for rivers or small lakes.
NOTE: For 9 Small Lake STHUCs and for 22 River STHUCs it was discovered that a
check that should have been ">19.999" was apparently implemented in the original
production work as ">19" a keypunch error that resulted in a decision to "keep" the
information.
These cases are flagged in the river and small lakes STHUC files with a " Y" in a field
called USED19 (see Appendix A: Field Descriptions).
Step 3. Refine the STHUC Dataset to only those STHUCs that passed the Data
Adequacy Test
For the water types of lakes and estuaries, it is assumed that they pass the data adequacy
test. Rivers and small lakes are treated independently of one another in regards to the
data adequacy test. For example, if <20% of a STHUC's perennial rivers were assessed,
but >20% of the STHUC's small lakes were assessed, then the result would be to discard
the river information for that STHUC while at the same time keeping the small lake
information.
Step 4. Create an Average Ratio for All Lakes
To create a score for each HUC, the first step is to average the ratios from the differing
water types. The first step in this is to average the two lake water types (small and large)
together.
Step 5. Create an Average Ratio across all water types and develop HUC scores
Use available STHUC use attaining ratios for rivers, Lakes (small/large combined), and
estuaries and average the ratios over HUC watersheds to develop new 2002 HUC ratios
and GPRA scores.
Step 6. Create an Average Ratio across all water types and develop HUC scores
Take the new 2002 HUC scores and carry over HUC scores from the previous
GRPA HUC scores (see step 1) where new 2002 HUC scores are not available to create
the complete 2002 HUC-level set of scores.
Definition of number scores assigned to watersheds
WATERSHED PERCENT RANGE
SCORE
LABEL
80% to 100%
0
80 - 100%) meeting all uses
>=50% to <80%
1
50 - 79%) meeting all uses
>=20% to <50%
2
20 - 49% meeting all uses
<20%
3
<20% meeting all uses
-1
Data Not Available to Develop Score
43
-------
3. Assumptions
Data Adequacy Test: Only rivers and small lakes are subjected to the Data Adequacy
Test. To determine whether or not to include values for rivers or small lakes for a given
STHUC, the State must have assessed at least 20% of that water type (see Assumptions:
Determining Percent Assessed). Rivers and small lakes are treated independently, meaning
that one can be kept while the other is left out. If a STHUC passes the Data Adequacy Test
because there was no Total Waters data available (see Assumptions: Total Waters Not
Available), it is NOT assumed that the State has assessed 100% of the water, but rather that the
State has assessed at least 20% of the water. Estuaries and Large Lakes are not subject to the
Data Adequacy Test, and are therefore always included.
Total Waters: Total waters numbers were derived from EPA's RF3-based Total Waters
Database. Reach File 3 (RF3) is a stream coverage at a scale of 1-100,000. Since the
development of RF3, the USGS has developed a new stream coverage called the National
Hydrography Dataset (NHD) which exists at three possible different scales: 1-100,000, 1-24,000,
and local scales (meaning scales that are specific to an individual State). Because of these
differing stream coverages and scales, what the State considers their total waters may differ from
what exists in the RF3-based Total Waters Database.
Determining Percent Assessed: To determine the percent assessed for Rivers and Small
Lakes in a given STHUC, a ratio is developed by taking the total size reported by the State in
that STHUC divided by the total size (see Assumptions: Perennial Rivers) in the STHUC
provided from the Total Water Database. Because the source for the numerator is different from
the source for the denominator, there may be cases where the total assessed size is greater than
the total size for that STHUC. This could result from a number of possibilities, including
differing scales of the source data for the two numbers (see Assumptions: Total Waters), to
differing ways in which the sizes were determined. If the ratio of assessed waters to total waters
is greater than 100%, then it is assumed that the STHUC passes the Data Adequacy Test (see
Assumptions: Data Adequacy Test).
Perennial Rivers: The total waters value for the purpose of the Data Adequacy Test for
rivers is based solely on Perennial Rivers.
Total Waters Not Available: If a total waters value was not available for a given
STHUC, it was assumed that the rivers and small lakes passed the Data Adequacy Test.
44
-------
Appendix D
Subobjective 2.2.1 Program Activity Measures
PAM
2005
2008
No.
Program Activity Measure Description
Target
Target
Type
Water Quality Standards
38
Number of States & authorized Tribes that have completed a review of water
quality standards within three years of the previous triennial review under
Section 303(c) of the Clean Water Act. (56 State/Territories, & 22 authorized
Tribes)
N/A
N/A
Core Program/
Tracking
39
Number of new or revised criteria documents for water pollutants published
providing the scientific information necessary for State adoption or revision of
a water quality standard protocols and methods for the pollutant, including
needed implementation protocols and methods.
5
15
Core Program/
Performance
40
Number of States that have adopted into their water quality standards, and
EPA has approved, nutrient criteria for fresh water (rivers/streams, lakes, and
reservoirs).
5
25
Core Program/
Performance
41
Number of States that have adopted into their water quality programs for
streams and small rivers, biological criteria designed to support determination
of attainment of water quality standard use designations standards.
[Note: biological criteria may include quantitative endpoints or narrative criteria
with quantitative implementation procedures or translators!
17
45
Core Program/
Performance
42
Number of Tribes that have water quality standards approved by EPA.
33
33
Core Program/
Performance
43
Each year, percentage of State/Tribal water quality standards submissions
that are approved/disapproved by EPA within 90 days.
73%
75%
Core Program/
Performance
Monitoring
44
Each year, the number of States & Territories that have adopted and begun
implementing a comprehensive monitoring strategy [including a State
approach to putting data into the Storage and Retrieval Data System
(STORET) consistent with national guidance, (i.e., March 2003 guidance
describing 10 key monitoring elements).
56
56
Watershed/
Performance
45
Number of States, Interstate Agencies, and Territories that provide
comprehensive integrated assessments of the condition of their waters
consistent with sections 305(b) and 303(d) of the Clean Water Act and EPA's
integrated assessment guidance. (56 State/Territories)
41
56
Watershed/
Performance
46
Number of Tribes that currently receive EPA funding that have developed
comprehensive monitoring strategies that serve all water quality management
needs, and address all tribal waters, including all water body types and that
provide their water quality data in a system accessible for storage in EPA's
STORET.
Under
develop-
ment
90
45
Core Program/
Performance
47
EPA reports results of a statistical survey of the condition of the Nation's
water, conducted in cooperation with the States.
N/A
56
Core Program/
Performance
Watershed Planning, TMDLs, and Nonpoint Source
48
The number of watershed based plans (and water miles/acres covered),
supported under State Nonpoint Source Program grants (section 319) since
the beginning of FY 2002 that are under development and the number of
watershed based plans, (and water miles/acres covered), where watershed
based plans are being implemented.
N/A
N/A
Watershed/
Tracking
49
Number of watershed based plans (and miles covered), supported under State
Nonpoint Source Programs (section 319) since the beginning of FY 2002 that
have been substantially implemented.
44
TBD
50
5,000
Watershed/
Performance
50 **
Number of national significant watersheds where a watershed approach to
protecting and restoring water quality is being fostered using Watershed
Initiative grant assistance (cumulative).
60
100
Watershed/
Performance
51
Percentage of TMDLs approved since the beginning of 2004 that were
developed as part of a larger, watershed planning process that addressed
restoration and protection of all waters within a watershed.
N/A
N/A
Watershed/
Tracking
52
Percentage of the TMDLs required for waters currently on the 303(d) list that
are established or approved by EPA within 13 years of listing consistent with
national policy. Annual targets will be based on State schedules or straight-
line rates that ensure that the national policy is met.
76%
100%
Core Program/
Performance
53
Number of Tribes that currently receive EPA funding in 2004 that have
participated with States &/or EPA in development of measures (e.g., TMDLs
or watershed-based plans) to restore and protect watersheds with impaired
waters.
24
20
Watershed/
Performance
45
-------
54
Percentage of TMDL approvals occurring since the beginning of FY 04 for
which EPA took approval action within 30 days of submission.
N/A
N/A
Core Program/
Tracking
55
Percentage of TMDLs approved by EPA, or watershed plans developed for
restoration of waters on a State impaired waters list that address nutrient
impairments that specifically identify a "trading margin."
25
200
Watershed/
Performance
56
Number of waterbodies identified by States in 2000 as being impaired by
nonpoint sources or by both point & nonpoint sources that are fully restored
(cumulative). [Estimated 6,264 waterbodies impaired solely or partially by
nonpoint source]
N/A
N/A
Core Program/
Tracking
57
Annual reduction in lbs/tons of nitrogen, phosphorus, and sediment from
nonpoint sources to waterbodies
N/A
N/A
Core Program/
Tracking
58
Number and dollar value of projects financed with Clean Water SRF loans to
prevent polluted runoff (cumulative).
N/A
N/A
Core Program/
Tracking
Permitting and National Regulations
59
Percentage of all NPDES permits that are considered current and, beginning
in 2005, the percentage of high priority permits are also current; permits for
facilities in Indian Country and to meet the same standard/schedule. [Targets
to be reevaluated once universe of priority permits is defined in cooperation
with States/Tribesl
87%
95%
88%
95%
90%
95%
90%
95%
Core Program/
Performance
60
Number of States that have updated regulations and/or statutes where
necessary to reflect new Concentrated Animal Feeding Operations (CAFO)
requirements; number of States that have issued Statewide general permits,
or otherwise substantially implemented the permit program, consistent with
these new requirements.
35
37
44
49
Core Program/
Performance
61
Percentage of States/Regions that have issued NPDES general permits
requiring storm water management programs for Phase II municipalities
(MS4S) (estimated annual load reduction of 4.1 billion pounds of pollutants).
(Note: assumes continued availability of general permits)
93%
100%
Core Program/
Performance
62
Percentage of States/Regions that have issued NPDES general permits
requiring storm water pollution prevention plans for Phase II construction
(estimated annual load reduction of 17 billion pounds of pollutants).
(Note: assumes continued availability of general permits)
98%
100%
Core Program/
Performance
63
Percentage of Significant Industrial Users (SlUs) in Publicly Owned Treatment
Works (POTWs) with Pretreatment Programs and percentage of known
Categorical Industrial Users (ClUs) in non-pretreatment POTWs that have
control mechanisms in place that implement applicable pretreatment
requirements.
N/A
N/A
Core Program/
Tracking
64
Number of pounds of pollution loadings to waterbodies from industrial
dischargers reduced (2004-2008) as a result of national industrial water
pollution control regulations.
1.0
2.4
Core Program/
Performance
65
Estimated annual reduction in pounds of pollutants discharged to waters as a
result of NPDES permits for storm water, POTWs, CAFOs, Combined Sewer
Overflows (CSOs), and industrial discharges, (annual reduction in 2003)
N/A
N/A
Core Program/
Tracking
66
Using the planning process called for in section 304(m) of the Clean Water
Act, identify any industrial categories where discharges to waterbodies or
releases to POTWs pose a significant risk to water quality and determine
whether to develop new national pollution control regulations, revise existing
regulations, or develop other control tools.
N/A
By 2006
Core Program/
Tracking
67
Number of dischargers with permits providing for trading between the
discharger and other water pollution sources and the number of dischargers
that carried out trades.
N/A
N/A
Watershed/
Tracking
68
Number of watersheds in which a watershed permit(s) has been issued and
the number of States issuing NPDES permits using a rotating basin process.
N/A
N/A
Watershed/
Tracking
69
Percentage of NPDES program authorities where a comprehensive
assessment of NPDES program integrity has been conducted (beginning in
FY 04) and the percentage of assessed programs that are complying with
implementation schedules for all those follow-up actions for which a schedule
has been established.
N/A
N/A
Core Program/
Tracking
State Revolving Fund
70
Fund utilization rate [cumulative loan agreement dollars to the cumulative
funds available for projects! for the CWSRF.
90%
94%
Core Program/
Performance
71
Return on Federal investment [cumulative dollar amount of assistance
disbursements to projects divided by cumulative Federal outlays for projects]
for the CWSRF.
N/A
N/A
Core Program/
Tracking
72
Number of States using integrated planning and priority systems to make
CWSRF funding decisions.
29
28
Core Program/
Performance
46
-------
Type Codes:
Watershed/Performance - indicates a PAM designed to assess implementation of core water programs on a watershed basis, which
is also a "true" performance measure that includes the indicator, target, and timeframe.
Watershed/Tracking - indicates a PAM designed to assess implementation of core water programs on a watershed basis, which
lacks either a target or timeframe and is classified as a tracking measure.
Core Program/Performance - indicates a PAM designed to assess implementation of core water programs, which is also a "true"
performance measure that includes the indicator, target, and timeframe.
Core Program/Tracking - indicates a PAM designed to assess implementation of core water programs, which lacks either a target or
timeframe and is classified as a tracking measure.
In Table 5.1 of the report, we classified this measure as addressing the critical national strategic objective of accelerating
watershed protection (specifically initiating or strengthening watershed protection for critical watershed/waterbodies). The measure
was not classified in the "Watershed Planning, TMDL, Nonpoint Source" category in Table 5.1.
47
-------
Appendix E
Agency Comments on Draft Report
MEMORANDUM
SUBJECT: Draft Evaluation Report: "Sustained Commitment Needed to Further Advance
Watershed Approach"
FROM: Benjamin H. Grumbles
Assistant Administrator
TO: Dan Engelberg
Director, Water Issues
Office of Program Evaluation
Office of Inspector General
Thank you for your memorandum dated July 19, 2004, transmitting the draft report on the
subject evaluation, No. 2003-001564. We appreciate your interest in the watershed approach
which continues to be one of the Administrator's highest priorities for the water program as
evidenced in the 500-day plan goal of Restoring Watersheds and Coastal Waters and the
Principles of "a better way" (http://www.epa.gov/adminweb/administrator/500davplan.htm).
Overall, the report has done a good job of capturing the essence of the issues and the
recommendations are relatively general. Although, it recognizes the benefits of the watershed
approach it also correctly points out the challenges of achieving full success as a result of our
limited ability to provide individual support to all local watershed organizations due to the
resource challenges as pointed out in various parts of the report. The Office of Water's
continuing commitment to the watershed approach has spanned more than ten years and recently
been broadened with the inclusion of a watershed sub-objective in the Agency's Strategic Plan
which has focused even more energy and enthusiasm into improving our support in watershed
protection.
I am pleased to respond to the draft report's specific recommendations in attachment 1 of
this memorandum. In addition, attachment 2 to this memorandum provides some additional
clarifications based on our review of the main body of your report.
Attachments
cc: Nikki Tinsley
48
-------
ATTACHMENT 1
Memorandum from Benjamin H. Grumbles to Dan Engelberg
Subject: "Draft Evaluation Report: "Sustained Commitment Needed to Further Advance
Watershed Approach."
Recommendation 1 (Report Recommendation 6.1):
Continue to integrate watershed activities into its core water programs. Specifically EPA could:
Commit the necessary resources to support, where appropriate, the development of
watershed permits, watershed TMDLs and watershed plans;
Evaluate the benefits and costs of watershed permitting and watershed TMDLs; and,
Work closely with state to ensure that the CWSRF is used as an additional source of
funding for nonpoint source and other watershed related projects.
We are and will continue to integrate our core programs into our watershed approach.
This December we plan to publish a draft technical guidance on watershed based permitting to
further assist states and local stakeholders. EPA is encouraging a watershed approach to TMDLs
by developing guidance for States on effective ways to pursue TMDL development in a
watershed context. We are working with interested Regions and select States to generate a draft
of this guidance this fiscal year, with particular attention to existing examples, successful
models, and lessons learned. The guidance will integrate approaches of the permits and non-
point source programs to facilitate successful implementation at the watershed level. In addition,
the Regions are evaluating opportunities to financially and technically support watershed
approaches to TMDL development in specific cases including the costs and benefits of
watershed TMDLs.
Beginning in 2002, the NPS Program and Grants Guidelines have required that States
dedicate $100 million of the 319 funds to remediating 303(d) - listed waters through the
development and implementation of watershed-based plans. The plans must be consistent with
any TMDLs that have been developed and serve as a mechanism to coordinate monitoring and
planning on a watershed basis, and provide the foundation for effective implementation using
federal and other funding sources. To implement watershed projects, States first must develop
watershed-based plans that identify pollutants and the significant sources of those pollutants and
determine the most appropriate mechanisms to address those sources. In most cases, the
planning and implementation is led by local communities with State assistance.
As noted in the report, EPA has provided states with guidance and flexibility to utilize the
CWSRF for state watershed priorities. EPA has consistently emphasized priority setting systems
based on water quality information and watershed needs. Since, the watershed approach
includes both point and nonpoint sources of pollution, the two sources need to be considered
together. Twenty-seven states have voluntarily adopted "Integrated Priority Setting Systems"
that enhance their ability to target funds to watershed priorities. While EPA does not have a
strategic planning target for CWSRF nonpoint source funding, the EPA strategic plan does have
a CWSRF Performance Activity Measure for Integrated Planning and Priority Systems.
49
-------
Recommendation 2 (Report Recommendation 6.2):
Work in partnership with stakeholders to ensure obstacles with implementing the watershed
approach are addressed. Specifically, EPA could:
Work closely with watershed advocacy groups to educate the public about the value of
water resources and how public participation is critical to safeguard these resources;
Improve coordination and communication with states and watershed organizations to
help insure the success of the watershed approach in achieving clean and safe water; and
Provide technical assistance to stakeholders, particularly in developing watershed plans.
We agree and have initiated a number of new activities designed to assist stakeholders in
implementing the watershed approach in addition to continuing to refine existing and develop
new watershed support tools already in the pipeline. Already this calendar year we have held
two "dialogues" with groups of national, regional and local organizations to discuss their both
their support needs and better understand the obstacles to their success. We have also begun
hosting a series of monthly Webcast Learning Seminars intended to deliver watershed training to
a broader national audience. The first Webcast held in June 2005 on the Eight Tools for
Watershed Protection in Developing Areas attracted more than 600 participants from 40 states,
the Virgin Islands, and Puerto Rico. Just this week, OWOW will launch a new web-based
Watershed Discussion Board. This tool will offer watershed practitioners a platform to exchange
ideas and hopefully stimulate innovative solutions that can be easily shared. Our goal is to
engage watershed leaders from around the country in these interactive, on-line discussions and
encourage the sharing of expertise and experience. We are also planning to build on the national
success through our partnership with the Weather Channel on the "After the Storm" video that
highlighted the importance of water resources to the general public. In addition, through the
Environmental Financing Network, a nationwide group of educational centers located at major
universities around the country, EPA has funded the development and distribution of a suite of
tools on how to obtain funding and resources for completing watershed work.
By the end of the calendar year we expect to release drafts of two major tools that will
support the development of watershed plans. One is guidance developed by our nonpoint source
program, a Watershed Planning Handbook that can be used as a technical resource by
stakeholders to improve the technical basis of watershed plans. The second is a web-based
watershed planning tool that complements the guidance by helping watershed planners follow a
stepwise, user-defined process to integrate the programmatic aspects of the Clean Water Act into
their watershed plans. EPA has requested input from EPA Regions, States, watershed
organizations, and other stakeholders in developing these draft tools. EPA is planning to provide
training for the use of these tools in the coming fiscal year.
The Watershed Managers Forum, the group of OW and regional managers most closely
aligned with the day to day operations of the watershed sub-objective, is developing a revised
national strategy to increase the capacity of local watershed groups. By focusing national
support to watershed organizations on some key support activities, we hope to ease the resource
pressure that the regions are feeling to work one on one with more and more local watershed
groups.
50
-------
One major omission from your recommendations on partnerships is the importance of
other government agencies and programs to ultimate success at the watershed level. For
example, integration of the conservation programs contained in the Farm Bill in agriculture
dominated watersheds and those contained in the Highway Bill for urban and suburban areas
dealing with stormwater runoff programs can be critical to local watershed planning and
implementation. Since a large percentage of funding for addressing nonpoint source problems,
most of the technical assistance resources, and much of the authority and responsibility
appropriated by Congress come primarily through the Farm and Highway Bill programs, these
programs need to be recognized as a significant part of the watershed solution.
The report also fails to make the connection between our partnerships with local
watershed NGOs and our need to both measure and demonstrate success. This approach is
especially important in light of the resource constraints on EPA and the States that the report
does emphasize in various places. Finally, both tribes and local planning agencies can play
critical roles in watershed approaches and are not mentioned in the report.
Recommendation 3 (Report Recommendation 6.3):
Continue to refine and improve keys aspects of its strategic planning process. Specifically, EPA
could:
Improve the accuracy and reduce the uncertainty of the baseline;
Ensure the Office of Water and EPA Regions clearly understand their roles in setting and
achieving national watershed goals; and,
Improve Regional plans to help achieve watershed goals.
Thank you for acknowledging in the body of the report that we have made important
strides in developing our watershed sub-objective. We agree that the accuracy is critical to
targeting and measuring our success. The importance of data quality was emphasized at our first
ever EPA Watershed Managers Forum meeting in December when the Watershed Managers
Forum agreed that improving our Assessment Database (ADB) was a critical first step in sub-
objective success. Further, the report documents the challenges in setting performance measures
based on watershed improvements and contains some reasonable recommendations to work
incrementally to improve these measures. We have already begun this process. An ad-hoc
national Measures/Data Workgroup is now examining a range of issues related to data needs,
data quality and measures of success and expected to make recommendations for improvements
in time for the next strategic planning cycle.
The Watershed Managers Forum which was officially Chartered on Earth Day 2005,
has been a key focal point for improving the two-way, regional/headquarters communication on
the watershed sub-objective. Working with this group, we have supplemented the broad regional
plans that are part of out strategic planning process. Each region has developed a much more
specific watershed sub-objective "game plan". Just this week, we have sent the regions a general
critique of their initial game plans with recommendations for improvements. We believe these
actions along with the broad participation of the regions in our various workgroups working on
the measurement and local watershed capacity building activities are key actions to help achieve
overall watershed success.
51
-------
Recommendation 4 (Report Recommendation 6.4):
Continue to improve key aspects of its performance measurement system. Specifically, EPA
could:
Ensure that the performance of all critical national strategies and implementation of core
water programs on a watershed basis can be assessed; and
Improve the design of the watershed restoration and watershed improvement goals.
We agree that improvements are needed in our ability to measure success and as
mentioned above we are working jointly with our regions to identify possible improvements in
both measures and the data needed to support them. Our strategic planning program activity
measures underwent a critical analysis with much regional input that led to changes in 2006. We
expect these activities along with the current PART reviews by OMB will have a significant
impact on our next strategic plan.
A major focus of Office of Water is to strengthen data management systems to track
water quality status and trends and measure progress in the nation's watersheds. We are linking
our IT efforts and the statistically-based monitoring efforts to the Strategic Plan. These efforts
include redesigning STORET into a new system, tentatively called Water Quality Exchange
(WQX). WQX will contain the data from probability surveys that characterize condition of
nation's water resources. WQX also will contain data that supports measures of incremental
progress towards restoration or protection of waterbody segments or watersheds. In addition,
WQX will provide data that is used to make state assessment decisions, which are reported in the
National Assessment Database (http://www.epa.gov/waters/305b/index.html). The assessment
decisions are linked to the National TMDL Tracking System
(http://www.epa.gov/waters/tmdl/index.html). which contains information on TMDLs underway
and those completed.
Statistically-based monitoring provides a benchmark for large classes of waters of the
U.S., allowing us to track changes across the whole resource and determine what proportion of
the waters are moving up or down the scale in certain categories like e.g., good, fair and poor.
The site specific data on waterbody segments allow us to track the improvements associated with
specific actions in individual watersheds by measuring changes in chemical, physical and fish
tissue data. The state monitoring strategies now being developed address both the types of
monitoring designs needed to generate the data needed to track changes at these different scales
and the data systems needed to store and manage the data.
52
-------
Appendix F
0/G Evaluation of Agency Comments
EPA generally concurred with our recommendations and in some cases has taken actions to
address them. EPA's response to our draft recommendations is in Appendix E. Following is our
evaluation of the EPA's specific comments on our recommendations.
Recommendation 6-1
According to EPA's response to the OIG draft report, EPA plans to issue draft technical
guidance by the end of the fiscal year on watershed permitting to further assist States and
stakeholders. In addition, the response stated that Regions are evaluating opportunities to
financially and technically support watershed approaches to TMDL development in specific
cases, including the costs and benefits of watershed TMDLs. The response also noted EPA's
support for watershed plans through the use of 319 funds and for watershed priorities through
CWSRF funds. EPA mentioned that 27 States have voluntarily adopted "Integrated Priority
Setting Systems" that enhance their ability to target funds to watershed priorities.
EPA's listed activities to integrate watershed activities into its core water programs generally
satisfy the recommendations set forth in the draft report. However, EPA should also study the
benefits and cost of watershed permitting and watershed TMDLs. The Agency has recognized in
previous guidance that watershed permitting may require more time to develop permits. For
example, in its Watershed-Based National Pollutant Discharge Elimination System (NPDES)
Permitting Implementation Guidance, issued in 2003, EPA states the following:
... An expansion in stakeholders presents a challenge to and a new role for the permitting
authority (coordinator). Engaging a wider variety of stakeholders means that the
permitting authority and the permit writer will have to consider a broader range of
interests and watershed goals when developing the permit, potentially adding technical
complexity and time to the permit development process. An expansion in stakeholder
involvement will also challenge the other stakeholders as they take the time to understand
one another's goals for and concerns about the watershed, and determine how to best
structure the watershed-based permitting process to meet these goals.
Similar issues exist in developing watershed TMDLs. Evaluating the benefits and costs of
watershed permitting and watershed TMDLs may provide critical information for stakeholders as
they seek to identify innovative solutions to reduce loadings into their watersheds. EPA is in a
unique position to provide such information to those permitting authorities.
Recommendation 6-2
According to EPA's response, EPA is planning to work with the Weather Channel to further
highlight the importance of water resources to the general public. To improve coordination and
communications with States and watershed groups, EPA (1) has held two "dialogues" with
groups of national, regional, and local organizations to discuss both their support needs and
53
-------
better understand the obstacles to their success; (2) began hosting a series of monthly Webcast
Learning Seminars intended to deliver watershed training to a broader national audience; and
(3) will launch a new Web-based Watershed Discussion Board. EPA's response also stated that
the Agency is working with the Environmental Financing Network to fund the development and
distribution of a suite of tools on how to obtain funding and resources for completing watershed
work. Additionally, EPA is developing a Watershed Planning Handbook that can be used as a
technical resource by stakeholders and a Web-based watershed planning tool.
EPA was concerned that we omitted the importance of other government agencies and programs
in the success of the watershed approach. We understand that other Federal, State, and local
agencies also play a fundamental role in the success of the watershed approach. In this report,
we evaluated only EPA's efforts.
EPA's listed activities to work in partnership with stakeholders satisfy the recommendations in
the draft report. Continuing to support these types of activities will be crucial for the success of
the watershed approach.
Recommendation 6-3
In response to the first part of our recommendation, the Agency acknowledges the importance of
accuracy in terms of targeting and measuring success and states that an ad-hoc national
workgroup is examining a wide range of issues related to data quality and strategic planning.
The response states that improving the Assessment Database is a critical first step in the success
of the watershed subobjective. However, the Agency does not offer specific actions that it will
take to improve the accuracy of the baseline. For instance, it does not offer any actions it could
take to change how the baseline is calculated or how to compensate for errors in the Assessment
Database. The Agency does state that an ad-hoc national Measures/Data Workgroup is now
examining a range of issues related to data needs, data quality, and measures of success.
However, the recommendations for improvements will not be ready until the next strategic
planning cycle.
The Agency states that the Watershed Managers Forum will help communication between the
Regions and headquarters and that each Region is now developing regional "game plans" for the
watershed subobjective that will help address improvement to the regional plans. While a
workgroup may help communication, the Agency does not offer specific actions that should be
taken to help improve communication during the planning process. EPA's response did not
address how the Agency would ensure that the Office of Water and EPA Regions clearly
understand their roles in setting and achieving national watershed goals. Moreover, without
knowing the specific content of the "game plans," we are unable to determine whether they will
be beneficial in improving regional plans.
Recommendation 6-4
The Agency agreed that improvements are needed to measure success and stated that they are
working jointly with Regions to identify possible improvements in both the measures and the
54
-------
data needed to support them. The Agency expects that the current Program Assessment Rating
Tool reviews by the Office of Management and Budget will have a significant impact on the next
strategic plan. The response states that the Agency's major focus of the Office of Water is to
strengthen data management systems to track water quality status and to measure trends in the
progress in the nation's watersheds by redesigning the Storage and Retrieval Data System
(STORET) into a new system tentatively called Water Quality Exchange.
The Agency does not offer specific actions that it will take to address this recommendation.
The Agency does not address whether the mix of performance measures will be revised to better
determine the success of key initiatives or strategies. For example, in our draft report, we stated
that the NPDES program has tracking measures but no performance measures on a watershed
basis; the water quality standards and State Revolving Fund programs have neither on a
watershed basis. Additionally, EPA does not address how it plans on improving the design of
the watershed restoration and watershed improvement goals to be more understandable,
comparable, and reliable. EPA's 90-day response should address actions the Agency plans on
taking to address this recommendation.
55
-------
Distribution
Appendix G
Office of the Administrator
Assistant Administrator, Office of Water
Director, Office of Wetlands, Oceans, and Watersheds
Director, Office of Wastewater Management
Director, Office of Science and Technology
Director, Office of Ground Water and Drinking Water
Agency Followup Official (the CFO)
Agency Followup Coordinator
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
General Counsel
Inspector General
56
------- |