j
\
i
-------
Report Contributors: Dan Engelberg
George Golliday
Kathryn Hess
Renee McGhee-Lenart
Gerry Snyder
Abbreviations
CSO Combined Sewer Overflow
EPA U.S. Environmental Protection Agency
FY Fiscal Year
LDAR Leak Detection and Repair
MACT Maximum Achievable Control Technology
OECA Office of Enforcement and Compliance Assurance
OIG Office of Inspector General
OMB Office of Management and Budget
Cover photos: Examples of the three enforcement priority areas reviewed, clockwise from
left: an industrial plant emitting air toxics; a combined sewer overflow
outlet; and a cement plant engaged in mineral processing (EPA photos).
-------
,V16D Sr/|,
I
5
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
08-P-0278
September 25, 2008
Catalyst for Improving the Environment
Why We Did This Review
We undertook this review to
determine how well the
U.S. Environmental Protection
Agency (EPA) planned to
ensure success in its
enforcement priority areas.
We reviewed current priority
area strategies for air toxics,
combined sewer overflow,
and mineral processing.
Background
EPA's Office of Enforcement
and Compliance Assurance
(OECA) focuses on core
program activities and a
limited number of national
priorities. Through the
national priorities, OECA
directs its resources to
significant environmental
problems where patterns of
noncompliance have been
established and where a direct
federal role is needed.
EPA Has Initiated Strategic Planning for Priority
Enforcement Areas, but Key Elements Still Needed
For further information,
contact our Office of
Congressional and Public
Liaison at (202) 566-2391.
To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2008/
20080925-08-P-0278.pdf
What We Found
OECA has instituted a process for strategic planning in its national enforcement
priority areas. It has developed strategic planning guidance and a strategy
template to facilitate continual review and improvement of the strategies. The
Fiscal Years 2008-2010 strategic plans we reviewed, for air toxics, combined
sewer overflows, and mineral processing, contain an overall goal, a problem
statement, a description of the current status of the priority area, anticipated
environmental benefits, the facilities to be addressed, the tools to be used, and
OECA Headquarters and regional responsibilities.
However, each of the plans is missing key elements to monitor progress and
accomplishments and efficiently utilize Agency resources. All three strategies
lack a full range of measures to monitor progress and achievements. Two
strategies lack detailed exit plans. Additionally, the combined sewer overflow
strategy does not address the States' key roles in attaining the strategy's overall
goal. The absence of these elements hinders OECA from monitoring progress and
achieving desired results in a timely and efficient manner.
What We Recommend
We recommend that the Assistant Administrator for Enforcement and Compliance
Assurance issue policy that requires strategy documents for the priority areas to
include:
• a full range of performance measures;
• exit plans; and
• the States' roles, where needed.
We also recommend that the Assistant Administrator develop a cost-effective
methodology for measuring resource inputs in the national priorities.
OECA concurred with the recommendation for making improvements in priority
strategy documents, and indicated how it plans to address our concerns. However,
OECA did not agree with our recommendation on developing a methodology for
measuring resource inputs. OECA stated that the use of input measures is an
Agency-level issue and should be addressed at that level. For reasons detailed in
the report, we believe the recommendation is valid.
-------
% UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
S WASHINGTON, D.C. 20460
«j-
*
4-t
OFFICE OF
INSPECTOR GENERAL
September 25, 2008
MEMORANDUM
SUBJECT: EPA Has Initiated Strategic Planning for Priority Enforcement Areas,
but Key Elements Still Needed
Report No. 08-P-0278
FROM: Wade Najjum
Assistant Inspector General, Offic'e' of Program Evaluation
TO: Granta Nakayama
Assistant Administrator, Office of Enforcement and Compliance Assurance
This is our report on the subject evaluation conducted by the Office of Inspector General (OIG)
of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe
the problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established resolution procedures.
The estimated cost of this report - calculated by multiplying the project's staff days by the
applicable daily full cost billing rates in effect at the time - is $544,912.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days. You should include a corrective actions plan for agreed upon
actions, including milestone dates. We also ask that you reconsider your position not to accept
the report's second recommendation. We have no objections to the further release of this report
to the public. This report will be available at http://www.epa.gov/oig.
If you or your staff have any questions regarding this report, please contact me at (202) 566-0832
or najjum.wade@epa.gov: or Dan Engelberg, Director of Program Evaluation, Enforcement and
Water Issues, at (202) 566-0830 or engelberg.dan@epa.gov.
-------
EPA Has Initiated Strategic Planning for 08-P-0278
Priority Enforcement Areas, but Key Elements Still Needed
Table of Contents
Purpose 1
Background 1
Air Toxics Priority Area 1
Combined Sewer Overflow Priority Area 2
Mineral Processing Priority Area 2
Noteworthy Achievements 3
Scope and Methodology 4
Missing Strategy Elements Hinder Measuring Program Progress 5
Strategies Do Not Contain Full Range of Measures 5
Two Strategies Do Not Include a Plan for Exiting the Priority Area 7
States' Roles in Accomplishing Overall CSO Strategy Goal Not Addressed 7
Conclusions 8
Recommendations 8
Agency Comments and OIG Evaluation 8
Status of Recommendations and Potential Monetary Benefits 9
Appendices
A OECA's Template for Developing a Performance-Based Strategy
for National Compliance and Enforcement Priorities 10
B Prior Reports 12
C Agency Comments 13
D OIG's Evaluation of Agency Comments 23
E Distribution 35
-------
08-P-0278
Purpose
The U.S. Environmental Protection Agency's (EPA's) Office of Enforcement and Compliance
Assurance (OECA) has prepared performance-based strategies for its national compliance and
enforcement priority areas. EPA's focus on national priorities is important because it allows
OECA to direct its resources to significant environmental problems where a pattern of
noncompliance has been established and a direct federal role is needed. The Office of Inspector
General (OIG) undertook this evaluation to determine how well EPA strategically planned to
ensure progress and accomplishments in its national enforcement priority areas. We reviewed
three priority area strategies: air toxics, combined sewer overflow (CSO), and mineral
processing
Background
OECA is responsible for maximizing compliance with 10 distinct federal environmental statutes
dealing with prevention and control of air pollution, water pollution, hazardous waste, and toxic
substances. OECA's national program is organized into two major components: (1) core
program activities that implement the requirements of all environmental laws, and (2) a limited
number of national priorities that focus on significant environmental risks and noncompliance
patterns. OECA evaluates candidate priorities and selects final national priorities using three
criteria: (1) significant environmental benefit, (2) pattern of noncompliance, and (3) appropriate
EPA responsibility. OECA selected nine priority areas for Fiscal Year (FY) 2008-2010. The
three we reviewed are emphasized in the following list:
• Air Toxics
• New Source Review/Prevention of Significant Deterioration
• Wet Weather - Concentrated Animal Feeding Operations
• Wet Weather - CSOs
• Wet Weather - Sanitary Sewer Overflows
• Wet Weather - Storm Water
• Financial Responsibility
• Mineral Processing
• Tribal Compliance Assurance
Air Toxics Priority Area
Air toxics are those air pollutants that are known or suspected to cause cancer or other
serious health problems. Through the 1990 Clean Air Act amendments, Congress
directed EPA to use a technology and performance-based approach to significantly
reduce emissions of air toxics from major sources of air pollution. "Major" sources are
defined as sources that emit 10 tons per year of any specific air toxics, or 25 tons per year
of a mixture of air toxics. Under the technology-based approach, EPA develops
standards for controlling the routine emissions from each major type of facility within an
industry group. These standards, known as maximum achievable control technology
(MACT) standards, are based on emissions levels that are already being achieved by the
-------
08-P-0278
better-controlled and lower-emitting sources in an industry. As of February 1, 2008, EPA
had developed and published MACT standards for 96 major source categories.
OECA established air toxics as a priority area in 2000. The objective was to distribute
the substantial MACT implementation workload between EPA Headquarters and the
regions through a regional "Adopt-a-MACT" program. The focus of the air toxics
priority shifted in April 2003 from primarily a compliance assistance and tool
development effort to compliance monitoring and enforcement. For FY 2005-2007,
EPA's air toxics goal was to achieve an annual reduction of at least 36,000 pounds of air
emissions regulated by the MACT standards. In April 2006, EPA set an even more
aggressive goal of reducing 750,000 pounds of air emissions over the 3-year period. The
FY 2008-2010 air toxics strategy focuses on three specific problem areas: leak detection
and repair (LDAR), flares, and surface coating.
Combined Sewer Overflow Priority Area
CSO discharges are identified as a significant source of impairment to receiving waters.
Contaminants discharged include bacteria, viruses, metals, nutrients, and oxygen-
consuming compounds. A combined sewer system carries both storm water and sanitary
sewage. Normally, its entire flow goes to a wastewater treatment facility. But, during a
heavy storm, the volume of water may be so great as to cause overflows of untreated
mixtures of storm water and sewage into receiving waters. According to a 2004 Report
to Congress, annual CSO discharges are about 850 billion gallons. The systems serve
approximately 46 million people, in over 700 communities, in 32 States, mostly in the
Northeast and upper Midwest. In 1994, EPA established policy that laid out steps needed
to eliminate or minimize the impacts of CSOs. This policy was codified by amendments
to the Clean Water Act in 2000.
OECA established CSO as a priority area in 1998. Under the priority area, EPA strives to
get communities with combined sewer systems, where required by the Clean Water Act,
to establish and implement long-term control plans that have enforceable schedules.
These schedules are important because of the high costs and the extensive time - as long
as 20 years - that will be needed for the communities to complete the major infrastructure
construction activities set out in their plans. For the FY 2008-2010 priority area strategy,
EPA's primary focus is to ensure that communities representing significant population
centers are making appropriate progress towards addressing CSO problems and
violations.
Mineral Processing Priority Area
Environmental impacts caused by mineral processing and mining are significant.
According to EPA, mineral processing and mining facilities generate more wastes that
are corrosive or contain toxic metals than any other industrial sector. Many of these
facilities have contaminated groundwater, surface water, and soil due to noncompliance
with State or federal environmental requirements or legally permissible waste
management practices. The United States has approximately 100 to 150 mineral
-------
08-P-0278
processing facilities and 300 to 500 active mining sites. Some facilities are located in
close proximity to large populations, and the health risk to people living near these
facilities is of significant concern to EPA.
In 2004, OECA developed a national strategy covering FY 2005-2007. The overall goal
of this strategy was to ensure that high-risk facilities in mineral processing and mining
were in compliance or on a path to compliance, or were otherwise working to reduce risk
to human health and the environment through such measures as implementing best
management practices. The strategy placed special emphasis on mineral processing
facilities that produced phosphoric acid, but also focused on non-phosphoric acid
facilities and mining facilities. The FY 2008-2010 strategy contains the same overall
goal for mineral processing as the previous strategy, but the focus of the goal for mining
facilities was changed from enforcement activities to compliance assistance.
Strategic planning is important because it:
• provides a roadmap and direction,
• sets priorities,
• allocates resources for maximum efficiency and effectiveness,
• establishes measures of success so that progress can be measured,
• gains commitment to the plan by involving the organization in its development, and
• coordinates actions of diverse parts of the organization into unified programs.
An important early step in strategic planning is developing a range of performance measures to
indicate what is being accomplished and whether results are being achieved. According to
Office of Management and Budget (OMB) guidance, performance measurement should include
goals, annual and long-term outcome performance measures, and output performance measures.
The guidance also states that output and outcome performance measures need to include a
description of what is being measured, a target, and a timeframe to effectively measure
performance. OMB also encourages input measures to determine cost efficiency and program
effectiveness. According to OMB guidance, outcomes describe the specific intended result or
consequence that will occur from carrying out the program. Outputs are the goods and services
produced by the program that contribute toward achieving outcomes and goals. Inputs are
resources, often measured in dollars or full-time equivalents, used to produce the outputs and
outcomes. Input measures complement performance measures by providing information on how
resources and efforts are being or should be allocated to ensure efficiency and effectiveness.
Noteworthy Achievements
OECA has placed an increasing emphasis on strategic planning in recent years. To assist in this
initiative, OECA formed teams for each priority area to plan and implement the work that needed
to be done. Since 2003, OECA's teams have developed priority area strategies for all of their
priority areas. These documents help communicate the vision of OECA Headquarters to the
regions concerning work that needs to be completed in the nine priority areas. To correct
deficiencies it identified in the FY 2003-2005 priority area strategies, OECA in 2004 issued
internal guidance for priority strategies. According to OECA, OECA 's Template for Developing
-------
08-P-0278
a Performance-Based Strategy for National Compliance and Enforcement Priorities (see
Appendix A) was used as the standard against which the FY 2005-2007 strategies were
reviewed. In March 2007, OECA issued a Guide for Addressing Environmental Problems:
Using an Integrated Strategic Approach.
In the three priority area strategies for FY 2008-2010, the current status of the priority area is
well defined. Each priority strategy has an overall goal, problem statement, the anticipated
environmental benefits, the facilities to be addressed, the tools to be used, and OECA
Headquarters and regional responsibilities. Additionally, the mineral processing strategy
contains a detailed exit plan with steps to discontinue the priority area and return it to the core
program, and a maintenance plan.
Scope and Methodology
We conducted this performance evaluation from August 2007 through June 2008 in accordance
with generally accepted government auditing standards. Those standards require that we plan
and perform the evaluation to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our evaluation objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and conclusions based on our
evaluation objectives.
We selected the air toxics, CSO, and mineral processing priorities for our review because they
covered a broad range of media (air, water, and land) and represented a mix of newer and older
priorities. We did not evaluate how EPA selected the priority areas.
We reviewed the FY 2008-2010 Performance-Based Strategies for the air toxics, CSO, and
mineral processing priorities using a combination of criteria derived from several sources,1
including OMB guidance. Based on those elements, we reviewed the extent to which the
strategies addressed:
• the current status of the priority areas;
• the results EPA wants to achieve in its priority areas;
• how EPA, regions, and States will get to where EPA wants to be;
• the responsibilities of major stakeholders; and
• the progress and accomplishments of EPA's goals and performance measures for the
selected priorities.
We interviewed staff in OECA, the Office of Air and Radiation, the Office of Water, and the
Office of Solid Waste and Emergency Response. We also interviewed staff in EPA Regions 1
through 9. We reviewed those internal controls that were relevant to our objectives. We also
considered findings in prior EPA OIG reports (see Appendix B).
^cam-Based Strategic Planning: A Complete Guide to Structuring, Facilitating, and Implementing the Process.
Fogg, C. Davis, 1994; Guide for Addressing Environmental Problems: Using an Integrated Approach. U.S. EPA,
OECA, March 2007, EPA 305-R-07-001; and Performance Measurement Challenges and Strategies. OMB, June
18, 2003.
-------
08-P-0278
Missing Strategy Elements Hinder Measuring Program Progress
The three priority area strategies reviewed are missing key elements to monitoring progress and
accomplishments. They lack a full range of measures to monitor progress and achievements.
Two of the strategies lack exit plans. Additionally, the CSO strategy does not address the States'
key roles in accomplishing the strategy's overall goal. Adding these elements would help OECA
monitor progress so that it can achieve results in a timely and efficient manner.
Strategies Do Not Contain Full Range of Measures
OECA's three priority area strategies do not contain a full range of performance and
input measures (see Table 1). According to OMB, performance measurement should
include goals, annual and long-term outcome performance measures, and output
performance measures. The guidance also states that output and outcome performance
measures need to include a description of what is being measured, a target, and a
timeframe to effectively measure performance. OMB also encourages that input
measures be used to determine cost efficiency and program effectiveness. None of the
three strategies reviewed has annual outcome measures or input measures, and one does
not contain a long-term outcome performance measure. OECA did set overall goals and
had one output performance measure for each strategy in its Annual Commitment
System. However, because the components of these measures are designed to work
together, this incomplete suite of measures significantly limits the utility of the
measurement structure.
Table 1: FY 2008-2010 Priority Area Strategy Measures
Goals and Measures
Overall Outcome Goal
Long-Term Outcome Performance Measures
Annual Outcome Performance Measures
Output Performance Measures
Input Measures
Air
Toxics
Yes
Yes2
No
Yes
No
CSOs
Yes
No
No
Yes
No
Mineral
Processing
Yes
Yes2
No
Yes
No
Source: OIG analysis of OECA's FY 2008-2010 strategies
Air Toxic Performance Measures
The long-term outcome performance measure in the air toxics strategy measures
the extent to which EPA has addressed facilities in the three problem areas. For
example, EPA defines "addressing facilities" as issuing an administrative order or
referring a case to the Department of Justice. While these activities will likely put
2 The air toxics and mineral processing long-term outcome measures capture the extent to which EPA addressed
facilities by certain activities such as enforcement actions. Although they are certainly correlated with outcomes, we
believe OECA could develop better outcome measures that include such things as bringing facilities into compliance
over a certain period of time.
-------
08-P-0278
facilities on the path to compliance and lead to outcomes, we also believe that
OECA could develop outcome measures that include such things as bringing
facilities into compliance over a certain period of time. It might be difficult to
develop targets for this type of outcome performance measure because facilities
often do not achieve full compliance for many years after an action is taken, and
the OECA priority areas are of fixed duration. However, developing this type of
measure is important because it would capture the strategy's results.
OECA needs to establish annual outcome performance measures for the air toxics
strategy. Such measures would allow OECA to monitor annual progress toward
achieving its long-term outcome performance measure and, if necessary,
implement any changes needed to keep the program on track. Moreover, any
annual outcome performance measures OECA uses should be associated with its
annual output performance measure. Having long-term and annual outcome
performance measures would help OECA monitor program effectiveness.
CSO Performance Measures
The CSO strategy does not have long-term or annual outcome performance
measures. It does include an overall goal focused on ensuring that communities
representing significant population centers are making appropriate progress
towards addressing CSO problems and violations. This overall goal could also be
considered a long-term outcome measure if it had a timeframe associated with it.
OECA managers told us the overall goal was not meant to be accomplished
during the 3-year strategy cycle. Without outcome measures that contain targets
and timeframes, OECA is not able to gauge whether the pace of progress is
satisfactory.
Mineral Processing Performance Measures
Like the air toxics strategy, the mineral processing strategy contains long-term
outcome performance measures that gauge the extent to which OECA addressed
facilities through activities such as enforcement actions. The points we raised
with respect to the long-term outcome performance measure for the air toxics
strategy also apply to the mineral processing strategy. Further, as with the other
strategies, the mineral processing strategy does not have annual outcome
performance measures. The strategy has outcome measures, but they lack targets
and timeframes, so they are not performance measures. For example, one of the
measures is the percentage of facilities in significant noncompliance that are
returned to compliance; the measure does not have a target or a timeframe against
which to assess performance. This measure could be written as an annual
outcome performance measure by adding a target and timeframe. For example:
Twenty-five percent of facilities in significant noncompliance
will be brought back into compliance in FY 2008.
-------
08-P-0278
This type of information would inform OECA of its annual progress toward
achieving its goals and enable it to make needed changes.
None of the three priority strategies includes input measures to assess the resources
needed and expended to accomplish OECA's goals. As a result, OECA cannot measure
the efficiency of its resource utilization or plan, nor adjust for changing resource needs.
There are two aspects to measuring inputs. The amount of resources needed to conduct
priority area activities needs to be projected at the beginning of the planning period.
Further, the amount of resources used for activities needs to be measured. Both measures
can be used to assist the Agency in planning and assessing its performance. OMB has
placed increasing emphasis on agencies measuring efficiency. Senior level OECA
officials said that developing input measures could be difficult and costly. We agree
additional resources would be needed, but without gauging resource utilization OECA is
unable to assess the efficiency of its priority area work.
Two Strategies Do Not Include a Plan for Exiting the Priority Area
Two strategies - CSO and air toxics - do not include plans for leaving the priority area
and transit!oning to the core program. According to EPA's template (Appendix A), each
strategy should include a plan for how EPA will exit its national compliance and
enforcement focus in the priority area once the overall goal is achieved. The plan needs
to address resource obligations, regulated facilities not addressed as part of the strategy,
and actions to safeguard improvements achieved under the strategy. These steps should
enable the priority area to transfer smoothly back to the core program when appropriate.
The CSO strategy states an exit plan will be developed after regions have determined a
more precise universe of CSO permittees needing Federal attention. The air toxics
strategy includes a plan to exit the three national problem areas addressed in the
FY 2008-2010 strategy, but does not address how OECA will discontinue priority
treatment of air toxics as a whole.
The mineral processing strategy includes an exit plan. This plan lays out steps to
discontinue the priority area, as well as the costs of on-going enforcement activities, the
entities not addressed under the strategy, and activities required to maintain the emissions
reductions achieved.
States' Roles in Accomplishing Overall CSO Strategy Goal Not Addressed
The CSO strategy does not address the States' roles in accomplishing the strategy's
overall goal, and we believe doing so would be beneficial. The other two strategies do
not address States' roles either, but their roles are not as critical as for the CSO strategy.
According to EPA's template, a component of each strategy is to clearly assign roles and
responsibilities. EPA's dependence on State actions and the lack of a discussion in the
strategy create uncertainty on whether the overall CSO goal can be achieved.
We expect State permitting and enforcement actions to continue to significantly
contribute to attaining the overall CSO goal. According to our analysis of OECA data,
-------
08-P-0278
28 percent of the systems serving large population centers have long-term control plans
with enforceable schedules in place. One-quarter of these plans resulted from State
enforcement actions.3 We believe specific information on the States' roles in
accomplishing the overall CSO goal needs to be included in the strategy, even though
other EPA guidance documents lay out authorized State obligations under the Clean
Water Act. While the strategy acknowledges State environmental agencies as having
"important roles to play in the CSO strategy," it does not define those roles.
Conclusions
OECA's strategic planning initiative for priority enforcement created a performance structure for
strategic planning in the three priority areas we examined. This initiative helps EPA ensure
compliance with environmental regulations and better protect human health and the environment.
The three strategies we reviewed would be improved by adding input measures, a full range of
performance measures, and detailed exit plans. The CSO strategy would also be improved by
addressing the States' roles in accomplishing the overall goal of the strategy. The absence of
these elements hinders OECA from monitoring progress, reporting efficiencies, and achieving
desired results in a timely and efficient manner.
Recommendations
We recommend that the Assistant Administrator for Enforcement and Compliance Assurance:
1. Issue policy that requires strategy documents for the priority areas to include:
• a full range of performance measures (output and outcome measures, and an overall
outcome goal) with targets and timeframes;
• exit plans, including steps to transition the priority area back to the core program; and
• the States' roles for those areas where State involvement will be critical in
accomplishing overall outcome goals.
2. Develop a cost-effective methodology for measuring resource inputs in the national
priorities.
Agency Comments and OIG Evaluation
OECA concurred with Recommendation 1 and indicated how it plans to address our concerns.
OECA did not concur with Recommendation 2. OECA's detailed comments and OIG's
evaluation of the comments are in Appendices C and D, respectively. The OIG has incorporated
technical corrections and clarifications from OECA's comments into the final report as
appropriate.
OECA continues to evaluate State permitting and enforcement actions in coordination with the Office of Water and
the regions to determine whether such actions will result in enforceable and complete long-term control plans.
-------
08-P-0278
Status of Recommendations and
Potential Monetary Benefits
POTENTIAL MONETARY
RECOMMENDATIONS BENEFITS (in $OOOs)
Planned
Rec. Page Completion Claimed Agreed To
No. No. Subject Status1 Action Official Date Amount Amount
1 8 Issue policy that requires strategy documents for 0 Assistant Administrator for 4/30/09
the priority areas to include: Enforcement and
. a full range of performance measures Compliance Assurance
(output and outcome measures, and an
overall outcome goal) with targets
and timeframes;
• exit plans, including steps to transition the
priority area back to the core program; and
• the States' roles for those areas where State
involvement will be critical in accomplishing
overall outcome goals.
2 8 Develop a cost-effective methodology for U Assistant Administrator for
measuring resource inputs in the national priorities. Enforcement and
Compliance Assurance
0 = recommendation is open with agreed-to corrective actions pending;
C = recommendation is closed with all agreed-to actions completed;
U = recommendation is undecided with resolution efforts in progress
-------
08-P-0278
Appendix A
OECA's Template for Developing a
Performance-Based Strategy for National
Compliance and Enforcement Priorities
I. Problem Statement
A. Define and characterize the environmental or non-compliance problem for the priority area
B. Provide baseline information on the size of the priority universe, geographic distribution of
the problem, level of compliance, and any environmental justice issues
C. Address how the problem relates to each OECA criteria for priority selection
D. Address how the problem affects Environmental Justice (all strategies) and Federal Facilities
issues (where appropriate).
E. Describe known or suspected causes of the problem
F. Identify existing policies and guidance that address the priority; and past strategies or
initiatives used to address the problem
II. Goals and Measures
A. Establish an outcome goal for the priority area
1. The goal should address the priority area, be measurable, and reasonably achievable in
the given time frame
2. Achievement of the goal will serve as the end point for the priority
B. Establish outcome and output measures for the priority area
1. Outcome Measures should be directly linked to, and serve as the primary indicator of
progress toward achieving the outcome goal
2. Output Measures serve as indicators of progress toward implementing the components of
the priority strategy, and will enable managers to track implementation milestones
3. Measures should be developed for EJ (all strategies) and Federal Facilities (where
appropriate)
C. Establish baselines for all measures, or develop a plan for developing a baseline for each of
the measures
III. Priority Strategy
A. Identify a mix of tools that will be used to address the priority area, and how they will result
in achieving the outcome goal.
B. Describe how the tools will be used in conjunction with each other to achieve the best results.
1. Include the sequence in which tools will be used, targeting priorities, how long they will
be used, implementation milestones, resource issues, and implementation roles (i.e.,
headquarters, regions, others)
2. Include a description of how Environmental Justice (all strategies) and Federal Facility
issues (where appropriate) will be addressed by the application of the tools
C. Develop a plan for periodically reviewing progress and making adjustments to the strategy as
needed.
10
-------
08-P-0278
D. Develop internal and external communication strategies. The internal strategy should focus
on providing managers information on strategy implementation and periodic performance
measure updates to enable them to track progress and make needed corrections. The external
strategy should inform the regulated community and stakeholders, help improve strategy
effectiveness, and communicate results.
E. Assign Roles and Responsibilities. Identify all of the parties that will play a role in
developing and implementing the strategy, and assign responsibilities for all of the
development and implementation tasks. Responsibilities should be broken out by
headquarters, regions, and others.
IV. Exit Strategy
A. Identify steps to discontinue formal priority treatment once the outcome goal has been met.
B. Develop a maintenance plan to include:
1. Significant resource obligations
2. Whether and how to address regulated entities not addressed as part of the strategy
3. Actions to safeguard to improvements achieved; including periodic monitoring, review
and adjustment
V. Workforce Deployment Strategy
A. The priority sector or problem area should be assessed to identify the skill sets needed to
effectively address the priority and implement the priority strategy. This assessment should
anticipate the different skills needed at different points in the strategy (e.g., compliance
assistance early on, more enforcement towards the end).
B. Taking into account the strategy's ultimate goal, time line, and exit strategy, responsible
parties in headquarters and regions should estimate the activity levels needed to achieve
strategy's ultimate goal in the given timeframe.
C. Gap Analysis - headquarters and regions should identify workforce gaps that will impact
achieving priority goals, including deficits in overall number of employees to address a
problem area, and in particular skill sets.
D. Workforce Deployment Recommendations - based on the gap analysis recommendations for
making workforce adjustments within individual regions and headquarters, and between
regions, and regions and headquarters should be identified to maximize strategy
effectiveness.
VI. Schedule
Develop an overall schedule for addressing the priority area.
11
-------
Prior Reports
08-P-0278
Appendix B
Title
EPA OIG Reports
Report No.
Date
Improvements in Air Toxics Emissions Data
Needed to Conduct Residual Risk Assessments
08-P-0020
October 31, 2007
Assessment of EPA's Projected Pollutant
Reductions Resulting from Enforcement Actions
and Settlements
2007-B-00002
July 24, 2007
Consolidated Report on OECA's Oversight of
Regional and State Air Enforcement Programs
E1GAE7-03-0045-8100244 September 25, 1998
Title
Other Notable Enforcement Reports/Policies
Report No.
Date
Report to Congress: Impacts and Control of
CSOs and SSOs, U.S. Environmental Protection
Agency
EPA 833-R-04-001,
August 2004
Combined Sewer Overflow Control Policy,
U.S. Environmental Protection Agency
59 Fed. Reg. 18688.
April 19, 1994
Report to Congress - Wastes from the Extraction
and Beneficiation of Metallic Ores, Phosphate
Rock, Asbestos, Overburden from Uranium
Mining, and Oil Shale, U.S. Environmental
Protection Agency
EPA/530-SW-85-033
December 1985
12
-------
08-P-0278
Appendix C
Agency Comments
August 14, 2008
MEMORANDUM
SUBJECT: Response to the Office of the Inspector General Draft Evaluation Report,
"EPA Needs to Improve Strategic Planning for Priority Enforcement Areas,"
Project Number 2007-00967, dated July 16, 2008
FROM: Catherine R. McCabe (signed)
Principal Deputy Assistant Administrator
TO: Dan Engelberg
Director of Water and Enforcement Issues
Office of Inspector General
Thank you for the opportunity to review and comment on the draft evaluation report,
"EPA Needs to Improve Strategic Planning for Priority Enforcement Areas," Project Number
2007- 00967, which focuses on improving strategic planning for priority enforcement areas. The
Office of Enforcement and Compliance Assurance (OECA) has completed its review and would
like to provide comments in response to this report. We also appreciate your consideration of
comments this office made in response to the Office of Inspector General's (OIG) discussion
draft released on April 17, 2008.
OECA agrees that the national priority strategies could be enhanced in a number of ways
suggested by the IG. OECA will revise its guidance regarding development of the strategies to
require that output and outcome measures with targets and associated timeframes, as appropriate,
goals, milestone schedules, exit strategies and the role of states, if any, be explicitly identified in
the documents.
We continue to have concerns about the utility and need for input measures tracked by
national priority. As we discussed at a meeting regarding this draft report, the use of input
measures is an Agency-level issue and should be addressed at that level.
As we have discussed, OECA believes that the title of this report is overly negative given
the findings of the report. We would suggest that the title focus more on potential enhancements
to strategic planning documents for the priority areas.
OECA would like to request your consideration of a number of remaining comments
which we believe could be incorporated into the final report that will more accurately describe
the use of the performance measures and exit strategies in the national priority strategies. We ask
13
-------
08-P-0278
that you review our attached comments and consider incorporating these comments into the final
report.
Within 90 days of the date of the final report, OECA will issue a policy memo directing
the Strategy Implementation Teams to clearly identify output and outcome measures with targets
and associated timeframes, as appropriate, goals, milestone schedules, exit strategies and the role
of states, if any, in the strategy documents. These elements will be reflected in the revised
strategies by April 30, 2009.
Again, we appreciate the opportunity to review and comment on this draft report. Should
you have any questions or concerns regarding this response, please contact OECA's Audit
Liaison, Gwendolyn Spriggs on 202-564-2439.
Attachment
14
-------
08-P-0278
OECA Response to OIG Draft Report
EPA Needs to Improve Strategic Planning for Priority Enforcement Areas
(Project No. 2007-000967)
This draft report evaluates three OECA compliance and enforcement priority strategies [Air
Toxics, Combined Sewer Overflows (CSOs) and Mineral Processing]. The draft report raises a
number of issues with each strategy. Our response to the recommendations is provided below:
I. OECA Response to Draft Report Recommendations
OECA agrees that the national priority strategies could be enhanced in a number of ways
suggested by the IG. Within 90 days from the date of the final OIG report, a memorandum from
the Assistant Administrator or the Principal Deputy Assistant Administrator will be issued to
OECA headquarters and regional managers and staff involved in the management and
implementation of the national compliance and enforcement strategies. The memo will require
that the national priority strategy documents clearly identify output and outcome measures with
targets and associated timeframes as appropriate, goals, milestone schedules and exit strategies.
The policy will state that the revisions suggested by the OIG and accepted by OECA will be
incorporated as part of the regularly-scheduled review of the strategies scheduled for the end of
the first year that the strategy has been in effect. The revisions will be reflected in the strategies
by April 30,2009.
The policy will state that the strategy documents will address output and outcome measures, and
an overall outcome goal since OECA believes that these measures are most important in ensuring
the effectiveness of the strategies. While OECA believes that most of the strategies include
milestones and exit strategies, the policy will direct the Strategy Implementation Teams to
develop clearer milestone schedules and more explicit exit strategies where needed.
OECA views its state oversight responsibilities as a core program function and is addressed by
the State Review Framework (SRF). Given that state oversight responsibilities apply to the
national compliance and enforcement program as a whole (rather than the priorities) and that the
priority strategies deal with initiatives that are almost exclusively the responsibility of
EPA/OECA, we do not think it is appropriate to address oversight responsibilities in the
strategies. However, the policy to be issued by OECA will direct the SITs to address the role of
states (if any) in the implementation of the strategies by describing the explicit responsibilities
states will be asked to undertake.
Please note that the Office of Water (OW), rather than OECA, is responsible for oversight of
state permitting programs. Therefore, if a large number of combined sewer systems are
addressed through state permits, state oversight of those systems is the responsibility of the
Office of Water (OW).
OECA is not convinced of the value of input measures in determining the effectiveness of the
national priority strategies. The problems selected for national priorities are such that federal
15
-------
08-P-0278
efforts are necessary to address them, thus input measures are less relevant to their success than
the outcome measures for each strategy. In previous discussions with the IG about this draft
report concerning input measures, the IG agreed that this is an Agency-level issue, and that it
needs to be raised and addressed at that level.
OECA does set targets, and uses outcome measures in monitoring progress of all of the
strategies, although some of these are through other mechanisms rather than stated explicitly in
the strategies. As such, OECA believes that the title of this report is overly negative in stating
that OECA needs to improve its strategic planning, and respectfully requests that the title be
revised to suggest that the strategy documents be made more explicit.
II. OECA's Specific Comments and Corrections on Text of the Report
At a Glance
Revise draft report, as follows: "All three strategies lack some of the full range of measures
recommended by OMB and ...."
Table of Contents
Revise draft report, as follows: Delete reference to "Appendix C, FY 2008 - 2010 Exit Strategy
for Mineral Processing."
Background
Revise draft report at page 1, as follows: "OECA evaluates candidate priorities and selects final
national priorities using three criteria: (1) significant environmental benefit, (2) pattern of
noncompliance, and (3) appropriate EPA responsibility federal involvement.
Air Toxics Priority Area
Revise draft report at page 2 as follows: "The FY 2008-2010 air toxics strategy
focuses on three specific problem areas: leak detection and repair or LDAR,
flares, and surface coating.
Combined Sewer Overflows Priority Area
Revise draft report at page 2, as follows: "... Under the priority area, EPA strives to get
combined sewer systems, where required by the Clean Water Act, to establish and fully
implement long-term control plans (LTCPs) that have enforceable schedules.
Mineral Processing Priority Area
Revise draft report at page 3, as follows: "... The FY 2008-2010 strategy contains the
same overall goal as the previous strategy and includes a compliance assistance focus for
mining but has dropped the focus on mining. "
16
-------
08-P-0278
Subheading Missing
Revise draft report at page 3, as follows: Insert subheading titled "Strategic Planning."
Noteworthy Achievements
Revise draft report at page 4, as follows: "In March 2007, OECA . . .Both of these documents
were used in the development of the 2008 - 2010 strategies."
Strategies Do Not Contain Full Range of Measures
OECA Response to page 5 including Table 1.1: The proposed edits below reflect that the final
Air Toxics strategy does have outcome goals and outcome and output measures, the CSO
strategy does have an outcome goal (additional discussion at "Performance Measures" below) as
well as annual outcome performance measures [i.e., Annual Commitment System (ACS)
process], and the Mineral Processing Strategy does include annual outcome performance
measures (i.e., ACS process).
Revise draft report at page 5, as follows: "None of the three strategies reviewed has annual
outcome measures or input measures, and two are missing long term outcome performance
measures."
Revise draft report at Table 1 page 5, as indicated below.
Table 1.1: FY 2008 - 2010 Priority Area Strategy Measures
Overall Outcome Goal
Long Term Outcome Performance Measures
Annual Outcome Performance Measures
Output Performance Measures
Input Measures
Air Toxics
Yes
NeYes
NeYes
Yes
No
CSOs
Yes
NeYes
Ne-Yes
Yes
No
Mineral
Processing
Yes
Yes
NeYes
Yes
No
Air Toxics Performance Measures
OECA Response to pages 5 and 6: The July 16, 2008 draft report indicates that the FY
2008-2010 Air Toxics Strategy lacks long term and annual outcome performance
measures, as well as input measures and an overall exit strategy. We believe we have
responded to these comments in a previous response to your office in May 2008.
17
-------
08-P-0278
As we stated previously, the Air Toxics Strategy includes a specific, measurable long-
term performance measure that will allow OECA to determine the performance of the Air
Toxics Strategy over time. The goal of the FY 08-10 Air Toxics Strategy is for each EPA
Region to address facilities in the regionally selected MACT universe(s) within the
National Problem Areas of LDAR, flares and surface coating. We believe this measure
serves as both a strategy goal and a long term measure.
Regarding annual performance measures, the Air Toxics Strategy requires each region to
submit annual milestones and a schedule for attainment of the goal. As part of this
process, each region has established annual commitments for compliance evaluations and
facilities addressed within their regionally selected universes. These milestones and
schedules have been reviewed and approved by the Strategy Implementation Team.
OECA is also developing a tracking system on the OTIS data system to assist both
headquarters and the regions in tracking progress towards attaining the goal. In addition,
outcomes from this strategy will be reported annually under the Government
Performance and Results Act (GPRA).
In addition, the final Air Toxics Strategy includes the following primary measures of
progress, as well as outcomes and outputs. The primary measures of progress for the
FY08-10 Air Toxics Strategy will be: 1) number of sources evaluated within each focus
area; and 2) percent of facilities addressed.
CSO Performance Measures
OECA Response to page 6: The CSO Strategy does have long-term outcome measures
which are reflected in Goal 1 of the Strategy. The Strategy does not explicitly state the
calendar dates by which the long-term outcome measures will be met because insufficient
data is available upon which to determine baselines and benchmarks.
Goal 2 is focused on ensuring that the Regions coordinate with their authorized States to
determine which of the smaller CSO communities are appropriate for Federal
enforcement in accordance with an April 10, 2005 memo signed by then ECOS President
and then Assistant Administrator for OECA. The Strategy states that the Regions have
until the end of FY08 to develop this list with their states, as follows: "By the end of
FY2008, EPA Regions will provide a list of appropriate Federal enforcement targets to
OECA and discuss setting appropriate goals for addressing them during the remainder of
the FY08 -FY10 cycle. Numeric goals and commitments for this universe will be
developed at that time."
Revise draft report at page 6, as follows. We recommend deleting the paragraph: "The
CSO strategy does not..." and replacing it with language that will reflect these points:
The CSO strategy does have long-term outcome measures which are reflected in Goal 1
of the Strategy. The Strategy's overall goal is focused on ensuring that communities
representing significant population centers are making appropriate progress toward
addressing CSO problems and violations. However, the Strategy does not explicitly state
18
-------
08-P-0278
the calendar dates by which the long-term outcome measures will be met because
insufficient data is available upon which to base valid baselines and benchmarks. States
and EPA have until very recently been unable to report the needed data into PCS, the
former database of record. A modernized system, ICIS-NPDES, now allows entry of this
data. EPA is taking steps to require the data through the rulemaking process. In the
meantime, OECA has begun collecting the data manually from the regions. OECA is, on
an ongoing basis, monitoring the Regions' progress in meeting this goal through the ACS
process, and through semi-annual data submission by the Regions to the Office of
Compliance.
While Goal 2 of the Strategy does not presently have an overall outcome goal, the
language in the Strategy suggests that one will be developed prior to the end of the FY08-
10 cycle.
Mineral Processing Performance Measures
OECA Response to page 6: While OECA has set annual targets for numbers of
inspections, it is not possible to determine the length of time necessary after an inspection
to develop an enforcement case and bring it to resolution. A number of factors make the
timing of case conclusions wholly unpredictable, most notably the cooperativeness of the
defendant in sharing information and negotiating an agreement. If a case goes into
litigation, case conclusion will depend on the schedule set by the court, which often
results in a multi-year process. This means that it is also not possible to develop annual
targets for numbers of enforcement actions, consent agreements or final orders.
We recommend deleting the two paragraphs on page 6 beginning with "Like the other
strategies, mineral processing..." and ending with "... enable it to make needed changes"
and replacing them with language that makes the following points:
While OECA has not set annual targets, except for inspections, it fully expects to address
the universe of facilities in the priority by FY 2010. For this priority, EPA defines
"facilities addressed" as facilities in compliance, on the path to compliance, and/or
implementing measures to address existing harm and reduce risks of potential harm. Due
to the variable length of time necessary to develop an enforcement case and bring it to
resolution, it is more realistic to set 3-year goals rather than set annual targets. A number
of factors make the timing of case conclusions wholly unpredictable, most notably the
cooperativeness of the defendant in sharing information and negotiating an agreement. If
a case goes into litigation, case conclusion will depend on the schedule set by the court,
which often results in a multi-year process. This means that it is also not possible to
develop annual targets for numbers of enforcement actions, consent agreements or final
orders.
OECA does monitor annual performance in terms of inspections, concluded cases,
percentage of universe brought into compliance, and pollutant reductions across the
regions and headquarters for their contributions to meeting the overall goal of the
Strategy by FY 2010. Further, the aggregated results provide the SIT the information
19
-------
08-P-0278
they need to determine whether and when the priority can be returned to the core
program.
Subheading Missing
Revise draft report at page 6, as follows: Insert subheading titled "Input Measures."
OECA Response to page 6: OECA is not convinced of the value of incorporating input measures
to determine the effectiveness of the national priority strategies. The problems selected for
national priorities are such that federal efforts are necessary to address them, thus input measures
are less relevant to their success than the outcome measures for each strategy. OECA agrees
with comments made by the OIG in meetings concerning this report that input measures are an
Agency-wide issue and should be dealt with at that level.
Two Strategies Do Not Include a Plan for Exiting the Priority Area
OECA Response to page 7: The July 16, 2008 draft stated that the Air Toxics Strategy includes a
plan to exit the three national problem areas but does not address how to discontinue priority
treatment of air toxics as a whole. As indicated in the draft report, the current exit strategy
addresses the three national problem areas, which constitute the entire scope of the FY 08-10
strategy. Since the current strategy does not cover the air toxics program as a whole, it would be
inappropriate to include an exit strategy for the entire air toxics program. The air toxics program
includes numerous regulations and it is possible that it will continue to be an enforcement
priority in the future with new national problem areas on which to focus.
As stated in the CSO strategy, OECA believes it is premature to develop an exit strategy due to:
1) the current lack of adequate information related to the CSO permittees with populations under
50,000 that are appropriate for Federal enforcement; and 2) the need for additional information
from the Regions and the Office of Water regarding the current status of enforceable long-term
remedial measures for numerous systems with populations over 50,000. OECA expects
significant progress through enforcement actions in addressing CSOs by the end of FY 2010.
However, without the necessary information on each system, the CSO SIT cannot develop an
appropriate final exit strategy. The CSO SIT continues to work with the Regions to collect
accurate information on the status of these systems. This information is not available in ICIS,
nor are States required to report this information to EPA. As a result, the CSO SIT developed a
database to collect and track this information. This requires manual reporting from the Regions
and detailed review and management of this information by the SIT. The CSO SIT intends to
continue to manage this information, monitor progress to determine when CSO compliance and
enforcement can be returned to the "core" NPDES program, and will develop an exit strategy as
soon as sufficient information is available.
States' Roles in Accomplishing Overall CSO Strategy Goal Not Addressed
OECA Response to page 7: The draft report states that the Strategy does not address the states'
roles in accomplishing the Strategy's overall goal. In fact, the Strategy references an April 10,
2005 memo signed by then ECOS President and then Assistant Administrator for OECA which
20
-------
08-P-0278
provides specific detail about the roles and responsibilities of states and EPA, and what roles the
states may play in accomplishing the outcome performance measure. Additionally, the "Roles
and Responsibilities" section of the Strategy references the regional role in working with states
to ensure that appropriate progress is made towards addressing the CSO permittees. The
Strategy is clear as to what is an appropriate action to address a CSO system. Regions are
expected to ensure that they are reviewing the enforcement actions by their authorized states to
meet these criteria.
As we noted in comments on an earlier draft of the report, OECA submitted an updated
spreadsheet reflecting OECA's ongoing evaluation of 14 New York state enforcement actions as
well as 25 state permits. This information should be reflected in the report.
Please revise the draft report at page 7, as follows: "We expect State permitting and enforcement
actions to continue to significantly contribute to attaining the overall CSO goal. According to
our analysis of OECA and regional data, about half of the systems serving large population
centers have LTCPs with enforceable schedules in place. Of these, about one-third resulted from
State enforcement actions. ..
Recommendations
1. Issue a policy that requires strategy documents for the priority areas to include
a. Output and outcome measures, and overall outcome goals with targets and timeframes.
OECA Response: Within 90 days from the date of the final report, OECA will issue a policy
memorandum requiring agreed-to revisions to be incorporated as part of the regularly-scheduled
review of the strategies scheduled for the end of the first year that the strategy has been in effect.
The revisions will be completed by April 30, 2009.
The policy will state that the strategies should include output and outcome measures with targets
and associated timeframes, as appropriate, and an overall outcome goal, since OECA believes
that these measures are most important in ensuring the effectiveness of the strategies. As noted
above, annual targets for enforcement cases are not appropriate, but other annual indicators can
be used to help monitor and assess progress toward a multi-year end goal.
b. Exit plans, including steps to transition the priority area back to the core program
OECA Response: While OECA believes that most of the strategies include milestones and exit
strategies, the policy will instruct the SITs to develop clearer milestone schedules and more
explicit exit strategies where needed.
c. States' roles for those areas where state involvement will be critical in
accomplishing overall outcome goals
21
-------
08-P-0278
OECA Response: The policy to be issued by OECA will instruct the SITs to address the role of
states (if any) in the implementation of the strategies by describing the explicit responsibilities
states will be asked to undertake.
2. Develop a cost-effective methodology for measuring resource inputs in the national
priorities.
OECA Response: OECA does not agree with this.
22
-------
08-P-0278
Appendix D
OIG's Evaluation of Agency Comments
No.
Agency Comments (see Appendix C)
OIG Evaluation
I. OECA Response to Draft Report
Recommendations
OECA agrees that the national priority strategies could
be enhanced in a number of ways suggested by the IG.
Within 90 days from the date of the final OIG report, a
memorandum from the Assistant Administrator or the
Principal Deputy Assistant Administrator will be issued
to OECA headquarters and regional managers and staff
involved in the management and implementation of the
national compliance and enforcement strategies. The
memo will require that the national priority strategy
documents clearly identify output and outcome
measures with targets and associated timeframes as
appropriate, goals, milestone schedules and exit
strategies.
The policy will state that the revisions suggested by the
OIG and accepted by OECA will be incorporated as part
of the regularly-scheduled review of the strategies
scheduled for the end of the first year that the strategy
has been in effect. The revisions will be reflected in the
strategies by April 30, 2009.
The policy will state that the strategy documents will
address output and outcome measures, and overall
outcome goal since OECA believes that these measures
are most important in ensuring the effectiveness of the
strategies. While OECA believes that most of the
strategies include milestones and exit strategies, the
policy will direct the Strategy Implementation Teams to
develop clearer milestone schedules and more explicit
exit strategies where needed.
We agree with the actions OECA
proposes to address Recommendation 1.
23
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
OECA views its state oversight responsibilities as a core
program function and is addressed by the State Review
Framework (SRF). Given that state oversight
responsibilities apply to the national compliance and
enforcement program as a whole (rather than the
priorities) and that the priority strategies deal with
initiatives that are almost exclusively the responsibility
of EPA/OECA, we do not think it is appropriate to
address oversight responsibilities in the strategies.
However, the policy to be issued by OECA will direct
the SITs (strategy implementation teams) to address the
role of states (if any) in the implementation of the
strategies by describing the explicit responsibilities
states will be asked to undertake.
Please note that the Office of Water (OW), rather than
OECA, is responsible for oversight of state permitting
programs. Therefore, if a large number of combined
sewer systems are addressed through state permits, state
oversight of those systems is the responsibility of the
Office of Water (OW).
We do not state in the draft report that
EPA should address its oversight
responsibilities in the strategies. We
believe that State responsibilities need
to be addressed in the strategies if their
work is critical to a strategy's overall
goal, as it is in the CSO strategy. For
example, State actions are key to EPA
meeting its overall CSO priority
strategy goal since only about 28
percent of facilities serving large
population centers have long-term
control plans in place and about
25 percent of those resulted from State
actions. Effective planning for this
priority would therefore necessarily
factor in the ability of States to support
the goals of the priority. As was stated
in the draft report," EPA's dependence
on State actions and the lack of a
discussion in the strategy create
uncertainty on whether the overall CSO
goal can be achieved."
We agree with OECA's proposed actions
of directing the strategy implementation
teams to address the role of States in the
implementation of the strategies.
OECA is not convinced of the value of input measures
in determining the effectiveness of the national priority
strategies. The problems selected for national priorities
are such that federal efforts are necessary to address
them, thus input measures are less relevant to their
success than the outcome measures for each strategy. In
previous discussions with the IG about this draft report
concerning input measures, the IG agreed that this is an
Agency-level issue, and that it needs to be raised and
addressed at that level.
We disagree. OECA cannot measure
the efficiency of its resource utilization
or adjust for changing resource needs
without input measures. The fact that
"federal efforts are necessary" for these
activities does not distinguish them
from others engaged in by EPA. Even
though the problems selected for
national priorities require federal effort,
management still needs to know what
resources they are trading-off between
the core programs and the selected
priorities to ensure program
effectiveness.
Further, the fact that this is an Agency-
wide problem does not excuse OECA
from a responsibility to address it.
OECA has been proactive in other
aspects of strategic planning; it can do
so on this topic as well.
24
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
OECA does set targets, and uses outcome measures in
monitoring progress of all of the strategies, although
some of these are through other mechanisms rather than
stated explicitly in the strategies. As such, OECA
believes that the title of this report is overly negative in
stating that OECA needs to improve its strategic
planning, and respectfully requests that the title be
revised to suggest that the strategy documents be made
more explicit.
We changed the title to: EPA Has
Initiated Strategic Planning for Priority
Enforcement Areas, but Key Elements
Still Needed.
II. OECA's Specific Comments and Corrections on
Text of the Report
At a Glance
Revise draft report, as follows: "All three strategies lack
some of the full range of measures recommended by
OMB and. ..."
We do not believe these changes are
warranted. See analysis below.
No change.
Table of Contents
Revise draft report, as follows: Delete reference to
"Appendix C, FY 2008 - 2010 Exit Strategy for Mineral
Processing."
We will delete the reference.
Background
Revise draft report at page 1, as follows: "OECA
evaluates candidate priorities and selects final national
priorities using three criteria: (1) significant
environmental benefit, (2) pattern of noncompliance,
and (3) appropriate EPA responsibility federal
involvement.
We do not have support for this change.
The OECA Web site:
http: //www. epa. gov/compliance/data/pl
anning/priorities/index.html uses the
same three elements mentioned in our
draft report.
No change.
Air Toxics Priority Area
Revise draft report at page 2 as follows: "The FY 2008-
2010 air toxics strategy focuses on three specific
problem areas: leak detection and repair or LDAR,
flares, and surface coating."
We will add in the abbreviation
"LDAR."
25
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
Combined Sewer Overflows Priority Area
Revise draft report at page 2, as follows: "... Under the
priority area, EPA strives to get combined sewer
systems, where required by the Clean Water Act, to
establish and fully implement long-term control plans
(LTCPs) that have enforceable schedules.
We agree to add "where required by the
Clean Water Act."
We also agree to add in the word
"implement," even though that is what
is also implied by "enforceable
schedules." Because the long-term
control plans will take years (even
decades) to implement, we do not
support adding "fully" as a modifier in
front of "implement." Most
municipalities will be continuing to
implement their long-term control plan
long after OECA discontinues CSOs as
a priority area.
10 Mineral Processing Priority Area
Revise draft report at page 3, as follows: "... The FY
2008-2010 strategy contains the same overall goal as the
previous strategy and includes a compliance assistance
focus for mining but has dropped the focus on mining.
We will rewrite the sentence as follows:
The FY 2008-2010 strategy generally
contains the same overall goal for
mineral processing facilities as the
previous strategy, but for mining
facilities the focus of the strategy
changed from enforcement activities to
compliance assistance.
11 Subheading Missing
Revise draft report at page 3, as follows: Insert
subheading titled "Strategic Planning."
The priority area titles are indented to
show that they are separate sections of
the background. The main background
starts again with "Strategic planning is
important..."
No change.
12 Noteworthy Achievements
Revise draft report at page 4, as follows: "In March
2007, OECA . . .Both of these documents were used in
the development of the 2008 - 2010 strategies."
We do not have support for OECA's
proposed revision. During our entrance
conference, we were told by OECA
staff that the March 2007 document was
not used in the development of the FY
2008-2010 strategies because it was
tool-based not problem-based.
No change.
26
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
13 Strategies Do Not Contain Full Range of Measures
OECA Response to page 5 including Table 1.1: The
proposed edits below reflect that the final Air Toxics
strategy does have outcome goals and outcome and
output measures, the CSO strategy does have an
outcome goal (additional discussion at "Performance
Measures" below) as well as annual outcome
performance measures [i.e., Annual Commitment
System (ACS) process], and the Mineral Processing
Strategy does include annual outcome performance
measures (i.e., ACS process).
Revise draft report at page 5, as follows: "None of the
three strategies reviewed has annual outcome measures
ef input measures, and two aro missing long torm
outcome performance measures.
Revise draft report at Table 1 page 5, as indicated below.
We disagree with almost all of the
proposed revisions to the report and
Table 1. We do agree to change our
response for the long-term outcome
performance measure for air toxics to
"Yes." See explanations below under
Air Toxics, CSO, and Mineral
Processing performance measures.
Table 1.1: FY 2008 - 2010 Priority Area Strategy
Measures
Overall Outcome
Goal
Long Term
Outcome
Performance
Measures
Annual Outcome
Performance
Measures
Output
Performance
Measures
Input Measures
Air
Toxics
Yes
NeYes
NeYes
Yes
No
CSOs
Yes
NeYes
Ne-Yes
Yes
No
Mineral
Processing
Yes
Yes
NeYes
Yes
No
27
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
14 Air Toxics Performance Measures
OECA Response to pages 5 and 6: The July 16, 2008
draft report indicates that the FY 2008-2010 Air Toxics
Strategy lacks long term and annual outcome
performance measures, as well as input measures and an
overall exit strategy. We believe we have responded to
these comments in a previous response to your office in
May 2008.
As we stated previously, the Air Toxics Strategy
includes a specific, measurable long-term performance
measure that will allow OECA to determine the
performance of the Air Toxics Strategy overtime. The
goal of the FY 08-10 Air Toxics Strategy is for each
EPA Region to address in the regionally selected MACT
universe(s) within the National Problem Areas of
LDAR, flares and surface coating. We believe this
measure serves as both a strategy goal and a long term
measure.
Regarding annual performance measures, the Air Toxics
Strategy requires each region to submit annual
milestones and a schedule for attainment of the goal. As
part of this process, each region has established annual
commitments for compliance evaluations and facilities
addressed within their regionally selected universes.
These milestones and schedules have been reviewed and
approved by the Strategy Implementation Team. OECA
is also developing a tracking system on the OTIS data
system to assist both headquarters and the regions in
tracking progress towards attaining the goal. In addition,
outcomes from this strategy will be reported annually
under the Government Performance and Results Act
(GPRA).
• In addition, the final Air Toxics Strategy
includes the following primary measures of
progress, as well as outcomes and outputs. The
primary measures of progress for the FY08-10
Air Toxics Strategy will be: 1) number of
sources evaluated within each focus area; and 2)
percent of facilities addressed.
The long-term outcome performance
measure in the air toxics strategy
measures the extent to which EPA has
addressed facilities in the three problem
areas. While these activities will put
facilities on the path to compliance and
lead to outcomes, we also believe that
OECA could develop outcome
measures that include such things as
bringing facilities into compliance over
a certain period of time. It might be
difficult to develop targets for this type
of outcome performance measure
because facilities often do not achieve
full compliance for many years after an
action is taken, and OECA priority
areas are of fixed duration. For the
purposes of Table 1, we will count it as
a long-term outcome measure.
We disagree that air toxics has annual
outcome performance measures. The
annual commitments made by the
Regions are the number of compliance
evaluations for LDAR, flares, or surface
coating. Each region's commitment
number is an output not an outcome.
OECA needs to establish annual
outcome performance measures for the
air toxics strategy. Such measures
would allow OECA to monitor annual
progress toward achieving its long-term
outcome performance measure and, if
necessary, implement any changes
needed to keep the program on track.
28
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
15 CSO Performance Measures
OECA Response to page 6: The CSO Strategy does
have long-term outcome measures which are reflected in
Goal 1 of the Strategy. The Strategy does not explicitly
state the calendar dates by which the long-term outcome
measures will be met because insufficient data is
available upon which to determine baselines and
benchmarks.
Goal 2 is focused on ensuring that the Regions
coordinate with their authorized States to determine
which of the smaller CSO communities are appropriate
for Federal enforcement in accordance with an April 10,
2005 memo signed by then ECOS President and then
Assistant Administrator for OECA. The Strategy states
that the Regions have until the end of FY08 to develop
this list with their states, as follows: "By the end of
FY2008, EPA Regions will provide a list of appropriate
Federal enforcement targets to OECA and discuss
setting appropriate goals for addressing them during the
remainder of the FY08 -FYIO cycle. Numeric goals and
commitments for this universe will be developed at that
time."
Revise draft report at page 6, as follows. We
recommend deleting the paragraph: "The CSO strategy
does not..." and replacing it with language that will
reflect these points:
The CSO strategy does have long-term outcome
measures which are reflected in Goal 1 of the Strategy.
The Strategy's overall goal is focused on ensuring that
communities representing significant population centers
are making appropriate progress toward addressing CSO
problems and violations. However, the Strategy does not
explicitly state the calendar dates by which the long-
term outcome measures will be met because insufficient
data is available upon which to base valid baselines and
benchmarks. States and EPA have until very recently
been unable to report the needed data into PCS, the
former database of record. A modernized system, ICIS-
NPDES, now allows entry of this data. EPA is taking
steps to require the data through the rulemaking process.
In the meantime, OECA has begun collecting the data
manually from the regions. OECA is, on an ongoing
basis, monitoring the Regions' progress in meeting this
goal through the ACS process, and through semi-annual
data submission by the Regions to the Office of
Compliance. While Goal 2 of the Strategy does not
presently have an overall outcome goal, the language in
the Strategy suggests that one will be developed prior to
the end of the FY08-10 cycle. 29
Goal 1 of the CSO strategy does contain
long-term outcome measures.
However, without baselines and
benchmarks, these are not performance
measures. The opening sentence of the
paragraph will be changed to include
"performance" as is implied by the
subsection header and as is included in
the opening sentence of the paragraphs
for the other two priority areas: "The
CSO strategy does not have any long-
term or annual outcome performance
measures."
We understand the focus of Goal 2 in
ensuring that the regions coordinate
with the States to determine which
smaller CSO communities are
appropriate for federal enforcement
action. However, as written, Goal 2 is
an annual output measure and not an
annual outcome performance measure.
We did not include an evaluation of
ICIS (Integrated Compliance
Information System) and PCS (Permit
Compliance System) in our field work.
Therefore we are not in a position to
include the database issues in our
report. We recognize OECA's ability to
track progress is hampered because of
these database issues. However, OECA
has taken steps to collect this data
manually throughout the last priority
area cycle (FY 2005-2007). Therefore,
the effort has been underway longer
than implied by the language OECA has
requested ("OECA has begun collecting
the data manually from the regions.").
This effort has been ongoing for long
enough that it is our opinion that
baseline information should be well
established.
No change.
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
16 Mineral Processing Performance Measures
OECA Response to page 6: While OECA has set annual
targets for numbers of inspections, it is not possible to
determine the length of time necessary after an
inspection to develop an enforcement case and bring it
to resolution. A number of factors make the timing of
case conclusions wholly unpredictable, most notably the
cooperativeness of the defendant in sharing information
and negotiating an agreement. If a case goes into
litigation, case conclusion will depend on the schedule
set by the court, which often results in a multi-year
process. This means that it is also not possible to
develop annual targets for numbers of enforcement
actions, consent agreements or final orders.
We recommend deleting the two paragraphs on page 6
beginning with "Like the other strategies, mineral
processing ..." and ending with "... enable it to make
needed changes" and replacing them with language that
makes the following points:
While OECA has not set annual targets, except for
inspections, it fully expects to address the universe of
facilities in the priority by FY 2010. For this priority,
EPA defines "facilities addressed" as facilities in
compliance, on the path to compliance, and/or
implementing measures to address existing harm and
reduce risks of potential harm. Due to the variable
length of time necessary to develop an enforcement case
and bring it to resolution, it is more realistic to set 3-
year goals rather than set annual targets. A number of
factors make the timing of case conclusions wholly
unpredictable, most notably the cooperativeness of the
defendant in sharing information and negotiating an
agreement. If a case goes into litigation, case conclusion
will depend on the schedule set by the court, which
often results in a multi-year process. This means that it
is also not possible to develop annual targets for
numbers of enforcement actions, consent agreements or
final orders. OECA does monitor annual performance
in terms of inspections, concluded cases, percentage of
universe brought into compliance, and pollutant
reductions across the regions and headquarters for their
contributions to meeting the overall goal of the Strategy
by FY 2010. Further, the aggregated results provide the
SIT the information they need to determine whether and
when the priority can be returned to the core program.
We agree that it can be difficult to set
annual targets. However, OECA's own
policy requires that appropriate
enforcement response be completed
within specific timeframes (see
Hazardous Waste Civil Enforcement
Response Policy, December 2003). For
example, significant noncompliance
must be addressed with a formal
enforcement response that mandates
compliance and initiates either an
administrative or civil action (see page
9). The policy provides response time
guidelines which can be used to set
targets for issuing unilateral or initial
compliance orders, Department of
Justice referrals, or final or consent
orders (see pages 10-12). It seems that
these types of enforcement actions
would occur prior to any efforts to
negotiate a settlement and the
timeframe could be estimated for when
these actions would occur.
We agree that targets cannot be readily
set for settlement negotiations with
Department of Justice or case
conclusions, as these may take years to
accomplish.
Currently, OECA only sets targets for
inspections. OECA could also set
targets using its existing Annual
Commitment System process for initial
enforcement actions for the ensuing
fiscal year based on the results of these
inspections. Further, OECA could
establish 3-year goals for case
conclusions.
No change.
30
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
17 Subheading Missing
Revise draft report at page 6, as follows: Insert
subheading titled "Input Measures."
The discussion on input measures,
where the Agency suggested we insert a
heading, applies to all of the priority
areas. Therefore, this text is indented
only once. Input measures fall under
the subsection heading on page 5,
"Strategies Do Not Contain Full Range
of Measures."
No change.
18 OECA Response to page 6: OECA is not convinced of
the value of incorporating input measures to determine
the effectiveness of the national priority strategies. The
problems selected for national priorities are such that
federal efforts are necessary to address them, thus input
measures are less relevant to their success than the
outcome measures for each strategy. OECA agrees with
comments made by the OIG in meetings concerning this
report that input measures are an Agency-wide issue and
should be dealt with at that level.
See response No. 3.
19 Two Strategies Do Not Include a Plan for Exiting the
Priority Area
OECA Response to page 7: The July 16, 2008 draft
stated that the Air Toxics Strategy includes a plan to exit
the three national problem areas but does not address
how to discontinue priority treatment of air toxics as a
whole. As indicated in the draft report, the current exit
strategy addresses the three national problem areas,
which constitute the entire scope of the FY 08-10
strategy. Since the current strategy does not cover the
air toxics program as a whole, it would be inappropriate
to include an exit strategy for the entire air toxics
program. The air toxics program includes numerous
regulations and it is possible that it will continue to be
an enforcement priority in the future with new national
problem areas on which to focus.
OECA chose air toxics as its FY 2008-
2010 priority area and is choosing to
address it by focusing on three problem
areas. This decision does not negate
OECA's responsibility to disclose its
plans for the air toxics area as a whole.
We understand that it may be premature
to indicate with specificity at this time
when the air toxics priority area will be
completed. However, the priority area
strategy does not even indicate in broad,
general terms what factors will be
considered when deciding to exit the
whole priority area. As time goes on,
more specificity can be added, but as
stated in the report, some thought
should be given to this so as to allow
the priority area to transfer smoothly
back to the core program when the time
comes.
31
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
20 As stated in the CSO strategy, OECA believes it is
premature to develop an exit strategy due to: 1) the
current lack of adequate information related to the CSO
permittees with populations under 50,000 that are
appropriate for Federal enforcement; and 2) the need for
additional information from the Regions and the Office
of Water regarding the current status of enforceable
long-term remedial measures for numerous systems with
populations over 50,000. OECA expects significant
progress through enforcement actions in addressing
CSOs by the end of FY 2010. However, without the
necessary information on each system, the CSO SIT
cannot develop an appropriate final exit strategy. The
CSO SIT continues to work with the Regions to collect
accurate information on the status of these systems. This
information is not available in ICIS, nor are States
required to report this information to EPA. As a result,
the CSO SIT developed a database to collect and track
this information. This requires manual reporting from
the Regions and detailed review and management of this
information by the SIT. The CSO SIT intends to
continue to manage this information, monitor progress
to determine when CSO compliance and enforcement
can be returned to the "core" NPDES program, and will
develop an exit strategy as soon as sufficient
information is available.
We understand that OECA lacks
information it needs to completely
formulate a detailed exit plan for the
CSO strategy. The strategy lays out a
time frame for obtaining the
information related to defining the
universe of CSO permittees where
federal involvement will be appropriate.
The strategy also lays out the
responsibility of the CSO strategy
implementation team to work with
Office of Water to reconcile and
coordinate on information management.
The Region's responsibilities to collect
and report results for the key measures
are also outlined in the strategy. If this
information gathering plan is
implemented, the CSO strategy
implementation team should be able to
add an exit plan to the priority strategy.
That said, OECA established CSO as a
priority area in 1998. We fail to
understand why, after 10 years, OECA
still struggles with baseline information
in this priority area. Milestones need to
be set and monitored in order for OECA
to solve these long-lived data
management issues.
32
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
21 States' Roles in Accomplishing Overall CSO Strategy
Goal Not Addressed
OECA Response to page 7: The draft report states that
the Strategy does not address the states' roles in
accomplishing the Strategy's overall goal. In fact, the
Strategy references an April 10, 2005 memo signed by
then ECOS President and then Assistant Administrator
for OECA which provides specific detail about the roles
and responsibilities of states and EPA, and what roles
the states may play in accomplishing the outcome
performance measure. Additionally, the "Roles and
Responsibilities" section of the Strategy references the
regional role in working with states to ensure that
appropriate progress is made towards addressing the
CSO permittees. The Strategy is clear as to what is an
appropriate action to address a CSO system. Regions are
expected to ensure that they are reviewing the
enforcement actions by their authorized states to
meet these criteria. As we noted in comments on an
earlier draft of the report, OECA submitted an updated
spreadsheet reflecting OECA's ongoing evaluation of 14
New York state enforcement actions as well as 25 state
permits. This information should be reflected in the
report.
Please revise the draft report at page 7, as follows: "We
expect State permitting and enforcement actions to
continue to significantly contribute to attaining the
overall CSO goal. According to our analysis of OECA
and regional data, about half of the systems serving
large population centers have LTCPs with enforceable
schedules in place. Of these, about one-third resulted
from State enforcement actions. ..
Guidelines for Federal Enforcement in
CSO/SSO (Sanitary Sewer Overflow)
Cases is the subject of the April 10,
2005, memo cited in OECA's response
and in the CSO strategy. The memo
includes a purpose statement: "to
present guidelines developed by the
CSO/SSO Workgroup on federal
enforcement in authorized states." The
focus of this memo is on federal roles,
not on State roles.
The strategy, under a section titled
"Roles and Responsibilities," states:
"EPA Headquarters and Regional
Offices and State environmental
agencies all have important roles to play
in the CSO strategy. The specific
responsibilities of each party are
described below." However, the
strategy does not include a description
of the specific responsibilities of the
State environmental agencies.
Moreover, we analyzed the updated
spreadsheet OECA provided us.
Because the spreadsheet is OECA's
interpretation of the data provided to it
by the regions, we chose to label this
spreadsheet as "OECA data," not
"OECA and regional data."
The spreadsheet tracks 202 permits.
Of these, 56 are indicated as having
been addressed per Goal 1. This is
28 percent. We will change the text to
be more specific (28 percent) rather
than "less than one-third."
Of the 56 permits listed as being
addressed per Goal 1, 14 are listed as
resulting from State enforcement
actions. This is 25 percent, as indicated
in the draft report.
33
-------
08-P-0278
No.
Agency Comments (see Appendix C)
OIG Evaluation
22 Recommendations
1. Issue a policy that requires strategy documents for the
priority areas to include
a. Output and outcome measures, and overall outcome
goals with targets and timeframes.
OECA Response: Within 90 days from the date of the
final report, OECA will issue a policy memorandum
requiring agreed-to revisions to be incorporated as part
of the regularly-scheduled
review of the strategies scheduled for the end of the first
year that the strategy has been in effect. The revisions
will be completed by April 30, 2009.
The policy will state that the strategies should include
output and outcome measures with targets and
associated timeframes, as appropriate, and an overall
outcome goal, since OECA believes that these measures
are most important in ensuring the effectiveness of the
strategies. As noted above, annual targets for
enforcement cases are not appropriate, but other annual
indicators can be used to help monitor and assess
progress toward a multi-year end goal.
We agree with OECA's response to this
part of the recommendation.
23 b. Exit plans, including steps to transition the priority
area back to the core program
OECA Response: While OECA believes that most of
the strategies include milestones and exit
strategies, the policy will instruct the SITs to develop
clearer milestone schedules and more
explicit exit strategies where needed.
We disagree that air toxics and CSO
strategies include milestones and exit
strategies for their priority areas. We
agree for the need for strategy
implementation teams to develop
clearer milestone schedules and explicit
exit strategies for the air toxics and
CSOs priority areas.
24 c. States' roles for those areas where state involvement
will be critical in
accomplishing overall outcome goals
OECA Response: The policy to be issued by OECA
will instruct the SITs to address the role of states (if any)
in the implementation of the strategies by describing the
explicit responsibilities states will be asked to
undertake.
We agree with the proposed actions in
OECA's response. We would add that
OECA needs to ensure that the CSO
priority area is revised to include a
description of States' roles.
25 2. Develop a cost-effective methodology for measuring
resource inputs in the national
priorities.
OECA Response: OECA does not agree with this
recommendation.
See Response #3.
34
-------
08-P-0278
Appendix E
Distribution
Office of the Administrator
Assistant Administrator for Enforcement and Compliance Assurance
Principal Deputy Assistant Administrator for Enforcement and Compliance Assurance
Director, Office of Compliance, Office of Enforcement and Compliance Assurance
Director, Office of Civil Enforcement, Office of Enforcement and Compliance Assurance
Agency Follow-up Official (the CFO)
Agency Follow-up Coordinator
Office of General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Audit Follow-up Coordinator, Office of Enforcement and Compliance Assurance
Deputy Inspector General
35
------- |