United States      Office of Policy
Environmental Protection  (1807T)
Agency
                   December 2012
                   EPA-100-K-12-012
Measuring the
Effectiveness of the
Ocean Dumping
Management Program
Final Report
December 15, 2012

-------
ACKNOWLEDGEMENTS

This evaluation of the Ocean Dumping Management Program was developed for the U.S.
Environmental Protection Agency's Office of Policy (OP) under Contract EP-W-10-002
between EPA and Industrial Economics, Inc. (IEc) of Cambridge, MA.  Eastern Research
Group (ERG) of Lexington, MA assisted in the evaluation as a subcontractor to IEc.
Andy Schwarz, Tracy Dyke Redmond, and Josh Wolff represented IEc. Lou Nadeau and
Melanie Sands represented ERG.

Ryan Gross and Davie Nguyen provided important background information on the Ocean
Dumping Management Program.  Matt Keene of OP's Evaluation Support Division
provided technical support and advice for the evaluation. Special thanks go to EPA staff
at Headquarters and the Regions who shared their perspectives and feedback to inform
the evaluation.

This report was developed under the Program Evaluation Competition, sponsored
annually by EPA's Office of Policy. Program Evaluation is one of the performance
management tools US EPA uses to assure itself, the public, and other interested
stakeholders that US EPA programs are protecting human health and the environment
effectively and efficiently. The information obtained through program evaluations can
shed light on whether US EPA programs are meeting their goals and objectives, provide
the evidence and road map needed to replicate successes, and identify those aspects of
US EPA programs needing improvement. To access copies of this or other EPA program
evaluations, please go to EPA's Evaluation Support Division website at
http://www.epa.gov/evaluate.

-------
TABLE OF CONTENTS


GLOSSARY

EXECUTIVE  SUMMARY

CHAPTER 1   |  INTRODUCTION AND BACKGROUND
Introduction 1-1
   The Evaluation 1-2
     Goal and Scope of the Evaluation  1-2
     Evaluation Roadmap 1-3
Evaluation Questions 1-4

CHAPTER 2   |  METHODOLOGY
Measures and Sources of Data 2-1
   Literature Review 2-4
   Pre-Interview Survey of EPA Staff 2-4
   Interviews 2-4
   Focus Groups 2-4
Analysis of Data  2-5
   Overview 2-5
   Data Storage 2-5
   Interview Data Analysis 2-5
   Survey Data Analysis 2-5
   Document Review 2-6
   Synthesis 2-6
Data Limitations  2-6

CHAPTERS   |  PROGRAM THEORY OF CHANGE
Ocean Dumping Management Program Logic Model  3-1
Findings 3-1
Program Processes and  Activities 3-3
   Headquarters  3-3
   Regions  3-4
Program Assumptions  3-4
Program Information Flows and Feedback Mechanisms 3-5
   Summary of Ocean Dumping Management Program in Practice 3-6
Allocation of Program Resources 3-10
Alignment of Partner Goals 3-11

-------
Conclusions  3-14
Recommendations  3-14

CHAPTER 4  |   EMERGING TRENDS, OPPORTUNITIES, AND CHALLENGES
Findings 4-1
Historical Trends and Emerging Issues 4-1
   Decreases in Ocean Dumping, Increased Focus on Beneficial Reuse 4-1
   Advances in Science and Understanding of Contaminants 4-3
   Changes in Program Resources  4-6
   Staff Turnover 4-9
   Trends and Emerging Issues Specific to EPA Regions 4-9
Obstacles and Delays 4-11
Opportunities for Improvement 4-12
   Opportunity to Exercise Greater Coordination and Leadership from EPA HQ 4-13
   Opportunity to Update Regulations and Guidance 4-14
   Opportunity Related to London Protocol Ratification 4-14
Conclusions  4-15
Recommendations  4-16

CHAPTERS  |   RESOURCE ALLOCATION
Findings 5-1
Resources Required vs. Those  That Can Be Reallocated 5-1
Opportunities to Improve Alignment of Resources with Intended Program Outcomes 5-1
Opportunities to Improve Alignment of Resources Across Regions 5-2
Opportunities, Risks, and Impacts of Resource Reallocation 5-3
Conclusions  5-3
Recommendations  5-4

CHAPTER 6  |   PERFORMANCE MEASUREMENT
Findings 6-2
Current Measures 6-2
Strengths and Weaknesses of Current Measures 6-3
Potential Alternative measures 6-4
   Outcome Measures  6-4
   Output/Efficiency Measures 6-5
Conclusions  6-5
Recommendations  6-6

-------
APPENDICES:

APPENDIX A: SURVEY QUESTIONS
APPENDIX B: INTERVIEW GUIDES
APPENDIX C: FOCUS GROUP DISCUSSION GUIDE
APPENDIX D: FTE RESOURCES AND CONTRACT DOLLARS APPLIED TO THE
           LOGIC MODEL
                                                            111

-------
GLOSSARY
Acronym         Definition
CWA            Clean Water Act
BSD             U.S. EPA Evaluation and Support Division
FTE             Full Time Equivalent
FY              Fiscal Year
HQ              Headquarters
MPRSA          Marine Protection, Research & Sanctuaries Act
NGO            Non-Governmental Organization
ORISE           Oak Ridge Institute for Science and Education
OW             U.S. EPA Office of Water
OWOW          U.S. EPA Office of Wetlands, Oceans, and Watersheds
PCB             Polychlorinated Biphenyl
PEC             U.S. EPA Program Evaluation Competition
SINKEX         U.S. Navy Sinking Exercise
USAGE          U.S. Army Corps of Engineers
                                                                          IV

-------
EXECUTIVE SUMMARY
The U.S. Environmental Protection Agency (EPA) Marine Pollution Control Branch,
within the Office of Water (OW), administers the Ocean Dumping Management
Program in coordination with each of EPA's seven coastal Regions (Regions 1, 2, 3, 4, 6,
9, and 10).  The Ocean Dumping Management Program managers applied for and were
awarded resources to evaluate the program in order to provide information that will help
managers better align resources and activities with intended outcomes. This information
should set the stage for future program improvements by increasing awareness among
Ocean Dumping Management Program staff about how the program is working and
leverage points where the program could change its approach to be more effective.
Based on interviews with program staff and stakeholders, a survey of program staff, and
reviews of the literature and program documentation, the evaluation team answered four
primary evaluation questions and multiple sub-questions.  These questions, and the
findings, are summarized below:

1. What is  the Ocean Dumping Management Program's theory of change?
•  What are the processes and activities that the Ocean Dumping Management
   Program uses to achieve its outcomes? The EPA HQ Ocean Dumping Management
   Program has five key processes/functions: 1) establishing environmental
   criteria/guidance for implementing MPRSA; 2) reviewing, concurring, and issuing
   ocean dumping permits; 3) designating ocean disposal sites; 4) establishing and
   managing Site Management and Monitoring Plans;  and 5) enforcing MPRSA. To
   support these roles, the program seeks to undertake  several activities, including: 1)
   developing and updating legislation; 2) working with international organizations and
   the State Department on international treaties and protocols;  3) developing policy on
   emerging issues; 4) developing regulations and policies, and ensuring that existing
   policies stay current; 5) issuing and updating guidance for implementing the
   program; 6) addressing petitions, letters, and other inquiries; 7) issuing special or
   general (non-dredge material) permits; 8) assisting Regions in reviewing and (where
   appropriate) concurring on dredged material permits issued by USAGE; 9) assisting
   Regions in conducting site monitoring and site designation; 10) assisting Regions and
   U.S. Coast Guard with regulatory enforcement; 11) reviewing regional programs;  12)
   coordinating across Regions; and 13) conducting strategic planning.  Evaluation
   findings suggest that the program is currently conducting most but not all of these
   actions.
•  What are assumptions that lead from activities/outputs to outcomes? The
   program theory rests on assumptions that:  1) the regulations are current, 2) there is
   adequate information to make program decisions, 3) there is  clear communication
                                                                           ES-1

-------
   within EPA, 4) there are adequate program resources, and 5) program partners
   cooperate with EPA. The evaluation finds that these assumptions do not hold true in
   all cases.
•  How are the program's resources allocated across the outcomes? The program's
   FTEs are allocated to activities intended to lead to the following long-term outcomes
   in descending priority order: environmentally acceptable conditions resulting from
   disposal (46 percent); coordinated and effective Ocean Dumping Management
   Program (29 percent); no legal vulnerability (15 percent); and maintaining an
   international leadership role (10 percent). The program's contract dollars are
   deployed in a similar manner across outcomes: Environmentally acceptable
   conditions resulting from disposal (38  percent); maintaining an international
   leadership role (31 percent); coordinated and effective Ocean Dumping Management
   Program (22 percent); and no legal vulnerability (9 percent).
•  What are the feedback mechanisms  from outputs/outcomes to activity planning?
   Information flows can serve as a feedback mechanism that helps program staff and
   managers adapt their activities. Current information flows include: sharing
   information from EPA Regions; interagency meetings; complaints from program
   partners; Regional Dredging Team conversations; and public feedback. There are
   opportunities to improve information flows, e.g., by conducting focused regular
   meetings between the EPA Regions and HQ.
•  At a high level, what are the goals of the non-EPA program partners? How do
   these goals align with the goals of the Ocean Dumping Management Program?
   For EPA Regions, the goals are to maintain environmentally acceptable conditions
   resulting from disposal. For USAGE, the goals related to ocean dumping are to
   maintain navigation and meet regulatory responsibilities for dredged material with a
   focus on cost effectiveness. Other program partners include the EPA Clean Water
   Act 404 Program, the Coast Guard, Department of Interior, Environmental Non-
   Governmental Organizations, U.S. Fish and Wildlife Service, National Oceanic and
   Atmospheric Administration, State Agencies, the U.S. Navy, and individual ports and
   harbors. These partners have a wide array of goals, from natural resource protection
   to dredging, regulating mineral rights,  and vessel disposal.  As a regulatory program,
   EPA's goals do not align with all of its customers' and partners' goals, which is to be
   expected.  The one area of misalignment that seems to cause the most challenges for
   the Ocean Dumping Management Program is the tension in goals between EPA and
   USAGE.

2.  How can the program  deal  with emerging issues,  challenges,  and
   opportunities for national and regional implementation of  the program?
•  What are the emerging issues, challenges, and opportunities for the Ocean
   Dumping Management Program? Are these the same for the national program
   and the Regions? Historical trends and emerging issues common to all the EPA
   Regions include: 1) decreases in ocean dumping nationwide, and an increased focus
   on beneficial reuse, 2) advances in science and understanding of contaminants, 3)
                                                                           ES-2

-------
changes in program resources, and 4) staff turnover. Emerging issues specific to
Regions include Gulf of Mexico restoration, major harbor and channel deepening
projects in the southeastern US, ensuring proper disposal of debris from the Japanese
tsunami on the west coast, ensuring proper disposal offish waste in Alaska and on
the west coast, and protecting sensitive near shore habitats (e.g., reefs) in the
Caribbean from effects of dredged material transport.
How have the environmental issues addressed by the program changed over the
past 20 years, and how have these changes been reflected in program design and
resource allocation?  What changes in ocean dumping-related environmental
issues and associated goals can reasonably be expected over the next 10 years?
The environmental issues addressed by the program have indeed changed over the
last 20 years, particularly since ocean disposal of sewage sludge was prohibited in all
EPA Regions in 1992.  Since that  time, the focus of the program has  shifted to
primarily regulating ocean disposal of dredging material, and preventing ocean
dumping of most other wastes. While we do not have budgetary information about
the program that stretches back as far as the  1990s, interviewees report that the
program has fewer resources now  than it has had historically, and program staff say
that the value of the program is less appreciated in recent years than it has been
historically.  Going forward, the Ocean Dumping Management Program is likely to
be asked to address an emerging set of issues, particularly related to emerging
contaminants and climate change.
What guidance and other assistance should be provided to Regions to assist
them in dealing with the emerging issues, challenges, and opportunities?
Program staff and stakeholders express concern that the regulations and guidance
documents have not kept pace with advances in science and therefore need to be
updated.  Regions have requested  a range of guidance to assist them with emerging
issues, challenges, and opportunities, as well as advances in science.  Regions request
guidance and coordination on emerging issues such as contaminants (e.g., pyrethroids
and pharmaceuticals) and issues related to climate change. Additional requests
include updating the testing guidance (Greenbook); guidance on fish waste, vessel
disposal, site management, beneficial reuse and the Coastal Zone Management Act;
advanced maintenance dredging/overdepth/sampling; and guidance on the Water
Resources Development Act of 1992.
What innovations or improvements are needed to deal with the emerging issues,
challenges, and opportunities? Regions  seek a centralized repository of checklists,
resource lists, and other resources  to improve consistency of program
implementation. Staff praise the recent Ocean Dumping Coordinators meeting as an
opportunity to foster coordination across the Regions and seek creative means to
gather Ocean Dumping coordinators from across the country in focused
conversations. Regions seek support from HQ establishing consistent expectations
for USAGE Districts
                                                                        ES-3

-------
•   Is the program encountering obstacles or delays? If so, what are those
    obstacles/delays and what can be done to overcome them? The program is
    encountering obstacles, such as unclear program boundaries and priorities,
    conflicting goals of some partners, and lack of coordination.  These obstacles lead to
    delays in activities such as updating guidance documents and reviewing regional
    programs.  The program may overcome some of its obstacles by more clearly
    communicating its value, more clearly articulating its mission and priorities, and
    fostering greater communication and consistency across Regions.

3.  Given stable or reduced program resources, what are the opportunities for
    improving the alignment between resource deployment and the  program's
    intended or desired outcomes?1
•   Does the current resource allocation reflect the desired allocation across
    program goals? If not, where should the emphasis be? Yes, the current resource
    allocation across its activities does reflect the program's goals and priorities.
    However, resource alignment is not currently optimized, and interviewees generally
    agree that the program could do a better job of focusing its efforts, working more
    efficiently, and being more proactive. Program managers could improve resource
    alignment by confirming the program's priorities, articulating the minimum resources
    needed to meet those priorities at both the HQ and regional level, and requesting
    support from senior leadership to focus the program's limited resources on those
    priorities.
•   What resources are required/fixed in the current allocation? What resources
    can be reallocated and under what conditions? It appears that the program's
    current resources are required  (and in fact may not be adequate to support the scope
    of the program's work).  Thus it does not appear that resources can be reallocated
    away from the program without reducing its scope. However, it may be desirable to
    reconsider the reallocation of resources across Regions to ensure equity given the
    variations in scale of work to be accomplished.
•   Of the resources that can be reallocated, what risks and/or opportunities does
    their reallocation pose? Resource reallocation (whether within the program or
    outside of the program) poses  risks of taking time and effort away from program
    work. Program staff and managers have not developed a shared view of the
    minimum requirements that HQ and the Regions must fulfill, and the resources
    needed to complete those tasks. Without such an agreement, it is difficult to pinpoint
    opportune areas for resource reallocation.
1 Additional sub-questions, which are answered as part of the findings for sub-questions described above, include: Would
 resources being reallocated be as effective in other places/roles within the Ocean Dumping Management Program as in
 their current place/role? What would be the impacts of resource reallocation on the program's customers? Would those
 impacts lead to feedback risks or opportunities for the program?
                                                                              ES-4

-------
•   Do the ultimate goals/targeted outcomes reflect the program's mission? Do the
    Regions share these goals and outcomes? Yes, the program goals are consistent
    with the program's mission, and the Regions share these goals.

4. How can  the Ocean Dumping Management Program measure its success?
•   Are the current measures understood throughout the program? Yes, the current
    measure is understood, although it is not interpreted consistently.
•   Are the current metrics valid measures for the program? A performance measure
    is valid if it accurately represents the condition or phenomenon that it is purporting to
    represent. The current measure would be valid if the key term "environmentally
    acceptable conditions" were consistently defined and interpreted.
•   Are there other (more valid or reliable) measures? If so, what are the pros/cons
    of the other measures relative to the current ones? There are supplementary
    outcome and output measures that could help measure program results. Key criteria
    for assessing measures are the extent to which the program has control over the
    measure and the extent to which the measure reflects program goals.
•   What can the current or potentially new measures tell us about program
    success? Supplementary measures could help assess the quality and efficiency of the
    program's work.
In light of the findings above, the evaluation team concludes that the Ocean Dumping
program faces numerous challenges, including significant budget restructuring, and at
times operates in an inconsistent and somewhat reactive mode as new issues arise. We
offer the following recommendations.
    1.  Clarify and communicate the importance of the program, focusing on why
       this program is essential to protecting ocean ecosystems.  We suggest that
       program managers articulate the importance of the program with regard to
       managing dredge materials and encouraging beneficial reuse, and state the
       benefits of the program in terms of environmental benefits (as well as
       implementing regulations).
    2.  Seek to foster improved communication and partnership with USAGE. We
       recommend that the Ocean Dumping Management Program work towards more
       productive communication with USAGE by acknowledging differences in
       mission and organizational constraints, and focusing on areas of shared
       responsibility.
    3.  Update program guidance and use the London Protocol ratification process
       as an opportunity to update the regulations. We believe the program will
       need to identify approaches to take on the additional responsibilities that will
       come with ratification. As  a part of updating program guidance, program
       managers at HQ should work to improve communications to Regions about how
       to implement the regulations, standard procedures, and expectations for program
                                                                           ES-5

-------
    partners (e.g., USAGE Districts). HQ should seek to facilitate communication
    and consistency across Regions.
4.   Strengthen information flows to inform program adaptation, building on a
    suite of performance measures that encompass both outcome and output
    measures.  We particularly recommend the program consider the following
    measures: volume of contaminated material not allowed to be ocean dumped
    (and of this amount, percentage that is used for beneficial uses); percentage of
    monitored sites found to be in compliance; average number of years since ocean
    disposal sites were monitored; and average number of years since Site
    Management and Monitoring Plans have been reviewed and updated.
5.   Assess resource needs:  We suggest that HQ define the minimum requirements
    that the Regions must fulfill. The program should then research what future
    program activity levels (e.g., permit review,  site designation activities) can be
    anticipated in each Region, given trends in ocean dumping and port development.
    The program can then analyze what resources will be required in each Region to
    meet minimum program requirements given anticipated activity levels.  Finally,
    the program should consider resource reallocation across Regions if necessary to
    ensure a balance between effort required and resources provided.  HQ will also
    need to assess its own minimum resource requirements to carry out its core
    activities at the national level, given work on the London Protocol and the need
    to update the regulations and provide additional guidance to Regions.
                                                                        ES-6

-------
CHAPTER  1   |   INTRODUCTION AND BACKGROUND
This chapter introduces the purpose and scope of the Ocean Dumping Management
Program evaluation. The chapter goes on to provide background on the program and its
origins and describe the evaluation questions and key audiences for the evaluation
findings.

INTRODUCTION
Enacted in 1972, the Marine Pollution, Research, and Sanctuaries Act (MPRSA)
regulates transportation for the purpose of dumping of all types of materials into ocean
waters and prevents or strictly limits the dumping into ocean waters of any material that
would adversely affect human health, welfare, or amenities; or the marine environment,
ecological systems, or economic potentialities.2  The MPRSA implements the
requirements of the London Convention, an international treaty governing ocean
dumping.3
Certain materials, such as high-level radioactive waste, medical waste, sewage sludge,
and industrial waste, are banned from ocean disposal. The primary source of material that
may be dumped in the ocean, with appropriate oversight and permitting, is navigational
dredging undertaken to remove sediment from navigation channels and vessel berthing
areas.  Other items that may be disposed in the ocean include vessels that are no longer in
operation, fish wastes, and human remains.
EPA is responsible for designating ocean disposal sites (both for dredged and non-
dredged material), establishing and managing Site Management and Monitoring Plans for
all ocean disposal sites, and establishing environmental criteria for the evaluation of
permit applications. Though the U.S. Army Corps of Engineers (USAGE) issues ocean
dumping permits for dredged material (subject to EPA review and concurrence), EPA is
responsible for issuing ocean dumping permits (special, general, research, or emergency)
for all other substances.
EPA's Marine Pollution Control Branch, within the Office of Water (OW) in
Washington, B.C., administers the Ocean Dumping Management Program in
2Title I of the Marine Protection, Research, and Sanctuaries Act (MPRSA), also referred to as the Ocean Dumping Act,
 generally prohibits (1) transportation of material from the United States for the purpose of ocean dumping; (2)
 transportation of material from anywhere for the purpose of ocean dumping by U.S. agencies or U.S.-flagged vessels; (3)
 dumping of material transported from outside the United States into the U.S. territorial sea. A permit is required to deviate
 from these prohibitions.

3 The U.S. has signed but not yet ratified the 1996 London Protocol, which is designed to update and clarify the London
 Convention. As a signatory, the United States is obliged not to take any action to defeat the 1996 Protocol's object and
 purpose.
                                                                                 1-1

-------
coordination with each of EPA's seven coastal Regions (Regions 1, 2, 3, 4, 6, 9, and 10).
The Headquarters (HQ) program addresses national policy and programmatic issues,
leads guidance development in coordination with the Regions, issues permits under the
MPRSA, addresses cross-regional issues, and represents the program on/in intra-agency,
interagency, and international workgroups and forums. The HQ program, in coordination
with the regional offices, regularly addresses public and other agency inquiries related to
dumping or placement of substances in the ocean and addresses a variety of issues during
and following large-scale emergencies (e.g., Hurricane Katrina, Deepwater Horizon Oil
Spill).  Regional ocean dumping programs generally focus on dredged material
disposal—including review and concurrence on USACE-issued permits, site
designations, and site management and monitoring activities—as well as vessel, fish
waste, burial at sea, and emergency issues.

The  Evaluation
An evaluation of the Ocean Dumping Management Program's past, present, and future is
timely. MPRSA was enacted forty years ago and this is the first systematic evaluation of
the program outside of OW.  Like many programs within the federal government, the
Ocean Dumping Management Program is operating within the context of diminishing
resources. To better understand the program's priorities and to improve management of
program resources, Ocean Dumping Management Program managers applied for and
were awarded evaluation resources through the EPA Program Evaluation Competition
(PEC). The PEC is an ongoing Agency effort overseen by the Evaluation and Support
Division (ESD), within the Office of Policy, that promotes the effective use and
integration of program evaluation throughout the Agency.
This evaluation is a collaborative effort between ESD and Ocean Dumping Management
Program  staff, with contractor support.  The overall evaluation design and objectives  are
the result of collective discussion and planning by the evaluation team, comprising a lead
evaluator from ESD, one primary Ocean Dumping Management Program representative,
an Oak Ridge Institute for Science and Education (ORISE) Fellow from OW, and the
contracting team of Industrial Economics, Inc. (lEc) and its subcontractor Eastern
Research Group, Inc. (ERG). In the remainder of this report, we use the term "the
evaluators" or "we" to refer to the contractors who prepared this report under the
oversight and guidance of the EPA staff.

Goal and Scope of the Evaluation
The goal of the evaluation is to provide information to allow the program to align
resources and activities with intended outcomes. This information should set the stage
for future program improvements by increasing awareness among Ocean Dumping
Management Program staff4  about how the program is working and leverage points
where the program could change its approach to be more effective.  The evaluation team
agreed on the following scoping decisions:
4 In general, we use the term "staff" to indicate both managers and staff. Where we specifically intend to indicate program
 managers, we use that term rather than "staff."
                                                                             1 -2

-------
                    • Focus: The evaluation will focus on the national Ocean Dumping Management
                      Program and how it operates as a whole, including the regional components.
                    • EPA Regions as Partners: The evaluation will consider EPA Regions as partners
                      of the national Ocean Dumping Management Program staff to the extent that
                      they:  a) contribute to the development of priorities for the program, and b)
                      conduct activities that contribute to program outputs. Otherwise, the Regions are
                      HQ's customers in the process.
                    • Other Federal Agencies: Other Federal agencies (e.g., USAGE) involved in
                      implementing requirements under the MPRSA may provide input to the
                      evaluation, as appropriate, but will not be the focus of the evaluation.

               Evaluation  Roadmap
               Exhibit 1-1 provides an overview of the evaluation process and how it will inform
               program design. Starting at the top of the figure and proceeding clockwise, the evaluation
               team first established an understanding of the Ocean Dumping Management Program's
               mission, goals, and objectives and then revised a logic model that had been provided by
               the program to reflect this information. Next, the team developed evaluation questions
               based on the logic model designed to support the goals of the evaluation. The evaluation
               questions then flowed into the team's discussion of the discrete measures that would be
               most appropriate to address each question and the data that would be needed to develop
               these measures. We also considered how and from what sources the data would be
               gathered, and how they would be analyzed. The evaluation methodology, described in
               Chapter 2, documents each of these elements of the planned evaluation process. The team
               gathered data and analyzed findings in accordance with the methodology and prepared
               this report to present findings, conclusions, and recommendations for potential
               adaptations to the program and/or adjustments in how the program defines its work.
EXHIBIT 1-1.   STEPS IN THE  EVALUATION PROCESS
                        Adapt the Pi-.
                                                 Describe the Program
                            Communicate Results
                              for Program Use
                                  \
 Program
Evaluation
  Cycle
                 Establish the Evaluation
                     Questions
I
                                  Conduct the Evaluation
             Define the Measures
              and Data Sources
                                                                                            1 -3

-------
EVALUATION  QUESTIONS
The evaluation team, in consultation with representatives from the Ocean Dumping
Management Program, identified four primary questions:
    1.  What is the Ocean Dumping Management Program's theory of change? The
       purpose of this question is to better understand the various components of the
       Ocean Dumping Management Program (inputs, outputs, and outcomes), their
       relationships, and how they function in relation to the program's intended goals
       and outcomes. Answering this question provides an opportunity to learn from
       key partners, particularly EPA Regions.  For example, with regard to ensuring
       environmentally acceptable conditions resulting from disposal, the evaluation
       will address how the program expects its regulations, policies, and enforcement
       activities will help achieve the goal. The report will address where the program
       is currently focusing efforts and resources and how that focus aligns with the
       priorities of partners and stakeholders in the program.  The findings will inform
       the remainder of the evaluation by pointing to places in the program where
       resource adjustment, improved communication, changes in information tracked,
       or a realignment of program focus may be needed to obtain targeted goals.
       Program staff may use the findings to communicate key aspects of their program
       and assumptions about how the program works.
    2.  How can the program deal with emerging issues, challenges, and
       opportunities for national and regional implementation of the program?
       This question serves three purposes: 1) to describe how changes in science,
       technology, and environmental issues have influenced the evolution of the
       program's design and management over time, 2) to better understand issues and
       relationships that influence the program's ability to meet its intended goals, and
       3) to identify  and explore aspects of the program (e.g., existing strengths or
       positive relationships) that can be leveraged to improve program functioning and
       resilience.
    3.  Given stable or reduced program resources, what are the opportunities for
       improving the alignment between resource deployment and the program's
       intended or desired outcomes? This question takes stock of the Ocean
       Dumping Management Program's resources and how they are allocated in order
       to determine whether or not this allocation reflects the program's goals and
       priorities. This question delves into which program resources are fixed, which
       resources may be reallocated, the structure of resource flows and delays, and the
       implications of changing the resource allocation, should that be necessary to
       achieve different program outcomes or due to decreases in program funding.
    4.  How can the program measure its success? The purpose of this question is to
       understand how the program can assess and track the extent to which it is
       meeting its goals.  The question helps identify current program measures of
       success, assess whether those measures are valid and/or easily understood by
       staff and external stakeholders, and suggest new or additional measures that
       might be beneficial in gauging success.
                                                                             1 -4

-------
             Exhibit 1-2 identifies a series of sub-questions that are designed to inform the primary
             questions, along with the key audiences and communication mechanisms for the findings
             for each primary question.

EXHIBIT 1-2.  EVALUATION QUESTIONS AND THE  KEY AUDIENCE AND COMMUNICATION
             MECHANISMS FOR FINDINGS


EVALUATION QUESTIONS AND SUB-QUESTIONS


KEY AUDIENCES
COMMUNICATION
MECHANISMS FOR
FINDINGS
1 . What is the Ocean Dumping Management Program's theory of change?
a) What are the processes and activities that
the Ocean Dumping Management Program
uses to achieve its outcomes?
b) What are assumptions that lead from
activities /outputs to outcomes?
c) How are the program's resources allocated
across the outcomes?
d) What are the feedback mechanisms from
outputs/outcomes to activity planning?
e) At a high level, what are the goals of the
non-EPA program partners? How do these
goals align with the goals of the Ocean
Dumping Management Program?

Program Manager for
Ocean Dumping
Management Program
Director of the Oceans
and Coastal
Protection Division
Director and Deputy
Director and Director
of the Office of
Wetlands, Oceans and
Watersheds (OWOW)



Program logic model
and accompanying
narrative theory of
change. The
evaluation team will
document the
evolution of the logic
model over the course
of the evaluation.


2. How can the program deal with emerging issues, challenges, and opportunities for national and
regional implementation of the program?
a) What are the emerging issues, challenges,
and opportunities for the Ocean Dumping
Management Program? Are these the same
for the national program and the Regions?
b) How have the environmental issues
addressed by the program changed over
the past 20 years, and how have these
changes been reflected in program design
and resource allocation? What changes in
ocean dumping-related environmental
issues and associated goals can reasonably
be expected over the next 10 years?
c) What guidance and other assistance should
be provided to Regions to assist them in
dealing with the emerging issues,
challenges, and opportunities?
d) What innovations or improvements are
needed to deal with the emerging issues,
challenges, and opportunities?
e) Is the program encountering obstacles or
delays? If so, what are those
obstacles/delays and what can be done to
overcome them?









Same audiences as for
Evaluation Question 1 ,
and potentially,
partner agencies
(e.g., Coast Guard
and USACE)


















Narrative in final
evaluation report










                                                                                        1 -5

-------


EVALUATION QUESTIONS AND SUB-QUESTIONS


KEY AUDIENCES
COMMUNICATION
MECHANISMS FOR
FINDINGS
3. Given stable or reduced program resources, what are the opportunities for improving the
alignment between resource deployment and the program's intended or desired outcomes?
a) Does the current resource allocation
reflect the desired allocation across
program goals? If not, where should the
emphasis be?
b) What resources are required /fixed in the
current allocation? What resources can be
reallocated and under what conditions?
c) Of the resources that can be reallocated,
what risks and/or opportunities does their
reallocation pose?
d) Would resources being reallocated be as
effective in other places/roles within the
Ocean Dumping Management Program as in
their current place/role?
e) What would be the impacts of resource
reallocation on the program's customers?
Would those impacts lead to feedback
risks or opportunities for the program?
f) Do the ultimate goals/targeted outcomes
reflect the program's mission? Do the
Regions share these goals and outcomes?









Same audiences as for
Evaluation Question 1


















Narrative in the final
evaluation report









4. How can the Ocean Dumping Management Program measure its success?
a) Are the current measures understood
throughout the program?
b) Are the current metrics valid measures for
the program?
c) Are there other (more valid or reliable)
measures? If so, what are the pros/cons of
the other measures relative to the current
ones?
d) What can the current or potentially new
measures tell us about program success?




Same audiences as for
Evaluation Question 1






Narrative and
examples in the final
evaluation report



1 -6

-------
CHAPTER 2  |   METHODOLOGY
This chapter describes how the evaluation team planned to answer the evaluation
questions, specifically: 1) the measures used and sources of data; 2) approaches to
analyzing data gathered; and 3) expected limitations of the analysis.
MEASURES AND SOURCES OF DATA
There are four key sources of information used in this evaluation:

    1.  Literature review
                                                      EXHIBIT 2-1.  SEQUENCE
                                                      OF DATA COLLECTION
    3.  Interviews with:
2.   Survey of program staff
    4.
   •  EPA Ocean Dumping Management Program
     HQ staff
   •  EPA staff from each of the seven Regions
     participating in the program
   •  Stakeholders, including representatives from
     EPA's Clean Water Act Section 404 (inland
     disposal) Program, USAGE, an environmental
     group that has engaged EPA's Ocean Dumping
     Management Program in the past, and a former
     program manager (now retired).
   Focus groups with EPA Ocean Dumping
   Management Program staff and managers.
Exhibit 2-1 shows the sequence of data collection used in
the evaluation. These sources of information will inform
measures that are used to answer the primary evaluation
questions and their sub-questions.  Exhibit 2-2 shows the
measures that are associated with each evaluation
question and the source of those data.
                                                       Documentation Review
                                                       Survey of Program Staff
                                                       Program Staff Interviews
   Regional Ocean
Dumping Coordinators
      Meeting
    Focus Groups
                                                   Additional Program Staff
                                                         Interviews
                                                    Stakeholder Interviews
                                                                             2-1

-------
EXHIBIT 2-2.   EVALUATION DESIGN MATRIX: MEASURES AND  DATA SOURCES
         EVALUATION QUESTIONS AND MEASURES [A]
                                                                                                 DATA SOURCES
LITERATURE

  REVIEW
SURVEY OF

 PROGRAM

  STAFF
                                                                                   INTERVIEWS WITH PROGRAM STAFF FROM...
EPA HQ
EPA REGIONS
STAKEHOLDERS
 FOCUS

GROUPS
SUMMATIVE

   FROM

FINDINGS1"1
1.    What is the Ocean Dumping Management Program's theory of change?
Summary of  the processes and activities that the program
uses to achieve its outcomes (1a)
Description of the assumptions that lead from
activities/outputs to outcomes (1b)
Quantification of the program's resources (Full Time
Equivalent (FTE) hours and contract dollars) allocated across
the outcomes (1c)
Description of the feedback mechanisms from
outputs/outcomes to activity planning (1d)
Description of the high-level goals of the non-EPA program
partners and how they align with the goals of the Ocean
Dumping Management Program (1e)
2.   How can the program deal with emerging issues, challenges and opportunities for national and regional implementation of the program?
Description of the emerging issues, challenges, and
opportunities for the Ocean Dumping Management Program at
the national and regional levels (2a)
Description of trends in environmental issues addressed by
the program over the past 20 years and related changes in
program design and resource allocation over the same period
(2b)
Description of guidance/assistance needed to deal with the
emerging issues, challenges, and opportunities  (2c)
Description of innovations or improvements are needed to
respond to emerging issues, challenges, and opportunities
(2d)
Description of obstacles or delays encountered by the
program and potential responses (2e)
                        mm.
3.  Given stable or reduced program resources, what are the opportunities for improving the alignment between resource deployment and the program's intended or
    desired outcomes?
Description of current resource allocation and emphasis (3a)
    ^     I

                                                                                                                                                         2-2

-------
         EVALUATION QUESTIONS AND MEASURES [A]
                                                                                              DATA SOURCES
LITERATURE

  REVIEW
SURVEY OF

 PROGRAM

  STAFF
                                                                                INTERVIEWS WITH PROGRAM STAFF FROM...
EPA HQ
EPA REGIONS
STAKEHOLDERS
 FOCUS

GROUPS
SUMMATIVE

  FROM

FINDINGS1"1
Description of desired resource allocation (3a)
W///A
Description of required/fixed elements of the resource
allocation (3b)
Description of elements that can be reallocated under what
conditions (3c)
Risks and opportunities of reallocation (3c)
                        mm,
                                                         m
Description of the effectiveness of reallocating resources as
compared to current place (3d)
Impacts of reallocation on the Ocean Dumping Management
Program's customers, e.g., EPA regional staff and potentially
other stakeholders
Description of the alignment of targeted goals/outcomes with
program mission (3f)
Description of the whether or not targeted goals/outcomes
are shared by Regions (3f)
4.   Given the answer to question #3, how can the Ocean Dumping Management Program measure its success?
Description of current program measures (4a)
                        mm.
Description of how program understands current program
measures (4a)
Description of the validity of current program measures (4b)
Potential alternative measures and/or suggested revisions to
current measures and their pros and cons (4c)
Description of how current or new measures gauge program
success (4d)
[A] Parenthetical references following the measures link the measure to the sub-questions listed in Exhibit 1 -2.
[B] Builds on findings from other evaluation questions
                                                                                                                                                    2-3

-------
Each of the four data collection methods is described in further detail below.

Literature Review
The evaluation team reviewed documents that are publicly available or made available by
program staff in order to: 1) better understand the Ocean Dumping Management Program
and its functioning, 2) identify key budgetary, technical, and managerial issues and trends
related to ocean dumping, and 3) provide context on relevant historical trends and events.

Pre-lnterview Survey of EPA Staff
Prior to interviews, the evaluation team distributed a brief online survey to 40 Ocean
Dumping Management Program staff at HQ and the Regions.  The purpose of the survey
was to use staff time as efficiently as possible by collecting information for selected
questions in a standardized way, and to identify topics to address in greater detail in
subsequent interviews. The evaluation team received 13 responses (a 33 percent response
rate), representing HQ and six of the seven Regions involved in the program.  The survey
gathered respondent opinions, perspectives, and feedback regarding emerging issues,
challenges, and opportunities for national and regional implementation of the program.
The evaluation team developed the survey questions with input from Ocean Dumping
Management Program HQ staff. Appendix A contains a copy of the survey questions.

Interviews
Following review of the survey results, the evaluation team conducted interviews with
EPA staff and individuals outside the agency. The interviews occurred in two sets: first,
the evaluation team interviewed EPA regional staff and a representative of EPA's Office
of Research and Development, which provides consultation to the Ocean Dumping
Management Program. We summarized the findings of these interviews at the Regional
Ocean Dumping Coordinators Meeting in August 2012.  Following the meeting, the
evaluation team conducted a second set of interviews with EPA HQ staff, a representative
of EPA's Clean Water Act (CWA) Section 404 program, and external stakeholders
(including USAGE, an environmental group, and a former Ocean Dumping Management
Program employee). The evaluation team developed tailored interview guides for each
set of interviewees in consultation with Ocean Dumping Management Program HQ staff.
Exhibit 2-3 summarizes the interviews conducted.  Appendix B contains the interview
guides used.

Focus  Groups
The evaluation team conducted three one-hour focus groups with program staff at the
Regional Ocean Dumping Coordinators meeting in Washington, D.C., in August 2012 to
add depth and detail to the information learned through the interview and survey.  The
discussions focused primarily on verifying and refining the theory of change, use of
program resources, effectiveness of HQ activities, and engagement with partners.
Appendix C includes the discussion guide for the focus groups.
                                                                             2-4

-------
EXHIBIT 2-3.   SUMMARY OF INTERVIEWS CONDUCTED
GROUP INTERVIEWED
EPA INTERVIEWEES
Regional staff
Office of Research and Development
HQ staff
CWA Section 404 staff
EXTERNAL INTERVIEWEES
USAGE
Environmental group (Clean Ocean Action)
Former Ocean Dumping Management Program
employee
NUMBER OF INTERVIEWS
CONDUCTED

6
1
2
1

1
1
1
               ANALYSIS OF DATA

               Overview
               The general analytical approach for the evaluation included a content analysis of the
               information gathered. In general, the analysis was qualitative, but where possible we
               included numerical analysis of quantitative data points such as resource and FTE trends.

               Data Storage
               The evaluation team developed a master set of all notes in Microsoft Word to serve as the
               foundation of the analysis. This file, along with the survey results, but excepting any
               information deemed confidential by the interviewees, will be available to EPA upon
               completion of the evaluation.

               Interview Data Analysis
               Given the manageable scope of data collected through the interviews, the evaluation team
               did not code interview responses, but rather identified common themes and gathered
               evidence from all data sources to address each evaluation question.  The evaluation team
               compiled quotes where appropriate to illustrate key points (quotes are not attributed). We
               also noted where there were areas of consensus, and where there were differences in
               perspective across different groups of interviewees (e.g., across different groups of
               Regions, or between HQ and Regions).

               Survey Data Analysis
               The evaluation team compiled and analyzed survey data to look for trends and  patterns
               related to the Ocean  Dumping Management Program.  The survey results include the
               percentage of respondents selecting specific responses or identifying particular themes.
                                                                                             2-5

-------
Document Review
The evaluators reviewed documents provided by the Ocean Dumping Management
Program and searched for publicly available literature relevant to this evaluation using
sources such as the EPA website, grey literature, and peer-reviewed journal articles. The
literature review helped us understand the Ocean Dumping Management Program, as well
as the historical context for the program, emerging issues, and resource trends.

Synthesis
To develop our conclusions and recommendations, the evaluation team synthesized
information across the data sources. We used multiple data sources to answer each
evaluation question and to verify apparent trends and patterns. Where different data
sources suggested different conclusions, we noted this and, when appropriate, searched
for additional data to confirm the analysis.

DATA LIMITATIONS
Throughout this study, the evaluation team sought to collect representative, objective, and
robust data. However, as with all program evaluations, data collection and analytical
limitations exist, and we make these transparent in our findings. We currently note the
following limitations to this evaluation methodology:
      •  Non-experimental research design:  This evaluation is not suited to use of
        comparison or control groups.  However, the evaluation does involve
        comparisons, particularly comparing information about the program over time
        (e.g., current program activities and resource allocation, recollections about how
        the program operated in the past, and expected future opportunities and
        challenges).
      •  Limited information drawn from outside of EPA: The evaluation primarily
        drew on information from within the Ocean Dumping Management Program
        itself (e.g., program documentation, survey responses, and interview notes).  The
        evaluation team did gather some perspectives from outside EPA (e.g., through a
        limited number of stakeholder interviews), but the evaluation does not
        necessarily present a representative summary of viewpoints from stakeholders
        outside EPA.
      •  Potential uncertainties in data on program resource allocation data: We
        analyzed program resource quantities (FTE hours and contract dollars) allocated
        across program outcomes based on data provided by program staff, including
        work plans with staff allocations to specific program activities. If these  work
        plans and program staff input did not accurately reflect the activities that FTE
        hours and contract dollars support, the accuracy of our results would be
        compromised.
                                                                              2-6

-------
CHAPTERS  |   PROGRAM THEORY OF  CHANGE
In this chapter we consider how the Ocean Dumping Management Program is intended to
work, and how it works in practice, based on input from program staff and stakeholders.
The section begins with a brief description of the program logic model that we developed
in conjunction with program staff at the outset of the evaluation, and then describes our
findings about particular facets of the program theory and experience, including:
     • Processes and activities that the program uses to achieve its outcomes;
     • Assumptions about how the program's activities and outputs transition to
       program outputs;
     • Program information flows and feedback mechanisms;
     • Allocation of program resources; and
     • Alignment of non-EPA partner goals.

OCEAN DUMPING MANAGEMENT PROGRAM LOGIC  MODEL
The program logic model presented in Exhibit 3-1, which was developed by the Ocean
Dumping Management Program and refined by the evaluation team, serves as a starting
point in the evaluation. It depicts key elements of the HQ program, including program
inputs, activities, outputs, customers, desired outcomes, and external factors that
influence results. The evaluation team used the information provided in the logic model
to clarify various aspects of the program's theory of change. The logic model also serves
as a template against which the program's actual experience can be compared, to learn if
there are aspects of the program that differ from the program design.
                                                                            3-1

-------
EXHIBIT 3-1.   OCEAN DUMPING MANAGEMENT PROGRAM  LOGIC MODEL
         Activities & Partners
                                       Outputs
                                                                  Customers
Short-term Outcomes
(Awareness)
Medium-term Outcomes
(Behavior)
Assist regions & USCG to
enforce regulations


Assistance w/ civil
penalties
USAGE and Permittees
/

US ACE & EPA aware of
monitoring requirements
and conditions at sites
/ \

Efficient and effective
evaluation of permit
proposals
Long-term Outcomes
(Conditions)
Develop and update
legislation (i.e. MPRSA)

Work w/ int'l orgs and
State Dept

Develop policy on
emerging issues (ie. CCS)


Develop regulations &
policies, keep current w/



Address petitions, letters.
and other inquiries
Issue special or general
(non-dredged material)
permits
Assist regions to review &
concur on dredged
material permits issues
	 >



\
\^
\
\
\
^






\»
/

Legislation package
(MPRSA Amendments)

International technical &
training documents

Annual report to IMO on
ocean dumping permits


Regulations & policies






Ocean dumping permits


	 »



\
V
^^»



\
\
\
\

\
— \—*
\ \
\\
US Congress





non-contracting parties

Gov't entities, regions.
regulated community,
env. groups, public



Regulated community.
env. groups, public

Ports and port- dependent
industries








	 »


	 >




\
\
	 ^
/
/
Congress informed
regarding MPRSA and LP



EPA & State Dept.
communicate w/ int'l
conmunfty (LC/LP)

Stakeholders understand
role and requirements of
ODP policy






Stakeholders understand
& aware of requirements
for disposal







	 >


	 *




/
/
N.
\\,
\>
Congress amends MPRSA
and ratifies LP




waste

Stakeholders conform to
ODP regulations and
policies



Limit ocean disposal of
wastes

permit

acceptable ocean disposal



*
/
/



\
\
\
\
•

//
' /
/




int'l leadership role









acceptable
conditions resulting
from disposal




Develop guidance
documents

Assist regions in site
monitoring & designation
issues

Review regional
programs

Conduct monthly call &
semi-annual meeting for
OD coordinators

Strategic Planning









Guidance documents

Assistance with site
monitoring data, SMMPs,
site designation

Regional program review
reports

Coordinators' call notes,
meeting reports

EPA reports & planning
documents
4
7
/


Regional program offices

Division, office, OW
management, public





Increased regional
awareness of policy and
intent of policy

Mgmt aware of program
activities and direction
<


Guidance hip roves
•nplementation ofOD
management program

Consistent program
administration across
regions

Mgmt support of program
activities and budget
\
\
/
No legal
vulnerability

Coordinated &
effective OD mgmt
program

                                                                                                                                                               3-2

-------
Findings
PROGRAM PROCESSES AND ACTIVITIES
The program's goal of ensuring that material placed in the ocean does not negatively
impact the environment and/or human health through the implementation of MPRS A has
remained constant since the program's inception in 1972.  However, since the program's
beginning, the types of materials being disposed of in the  ocean have changed, shifting
primarily toward the disposal of dredged material. Over time, this shift in the types of
materials placed in the ocean has shaped the activities and emphasis of the Ocean
Dumping Management Program. The program serves five primary functions, including:
    •   Establishing environmental criteria/guidance  for implementing MPRSA;
    •   Reviewing, concurring on, and issuing ocean dumping permits;5
    •   Designating ocean disposal sites;
    •   Establishing and managing Site Management and Monitoring Plans; and
    •   Enforcing MPRSA.
These functions are carried out by EPA HQ and Regions,  as described below.

Headquarters
HQ interviewees conveyed that their core program responsibilities mirror those listed in
the Activity column of Exhibit 3-1 (e.g.,  dealing with statutory issues, providing guidance
to the Regions), however,  funding and near-term priorities have shaped HQ activities. For
example, interviewees noted that the reduction in program funding has caused HQ to
operate in a "bare bones" fashion; staff have had to focus  on addressing issues requiring
immediate attention, e.g., decommissioning the program's research and monitoring
vessel, the Bold.6 Interviewees report that some longer-term program initiatives, such as
those listed below, are not receiving adequate attention and resources:
      • Updating the  ocean disposal testing manual,
      • Developing an amendments package for MPRSA,
      • Dealing with  Region-specific issues (e.g., fish waste in Region 10), and
      • Conducting certain types of activities related to strategic planning activities, such
        as regional reviews.
5 EPA is responsible for issuing ocean dumping permits (special, general, or emergency) for all material placed in the ocean
 other than dredged material, which is permitted by the USAGE and subject to EPA review and concurrence.

5 According to one interviewee, approximately half of the Bold's activities supported monitoring related to ocean dumping.
 In 2010, the Bold's survey missions included monitoring ocean dredged material disposal sites, coastal eutrophication and
 toxicity assessments, monitoring ocean outfalls, assessing critical coral reef habitats, and monitoring hypoxia in the Gulf of
 Mexico. Source: EPA's Ocean Survey Vessel Bold 2010 Annual Report,
 http://water.epa.gov/type/oceb/assessmonitor/bold/reports.cfm
                                                                                   3-3

-------
Regions
EPA's seven coastal Regions (EPA Regions 1, 2, 3, 4, 6, 9, and 10) implement the Ocean
Dumping Management Program in conjunction with HQ. At the regional level,
interviewees conveyed that they primarily focus on site designation, monitoring,
compliance assistance, and stakeholder relationships. Although all the Regions work on
many aspects of Ocean Dumping—including evaluation of project proposals and scoping
documents, and development of and review of sampling/testing data— Regions
emphasize different activities depending on the needs and priorities of the Region. For
example, regional staff noted that they currently devote significant attention to the
following activities:
      • Enforcement (Region 2);
      • Dredging teams (Regions 1, 6);
      • Beneficial use (Regions 6, 9, 10); and
      • Disposal of fish waste (Region 10).

PROGRAM ASSUMPTIONS
There are several assumptions that underlie the program's theory about how activities
will lead to intended outcomes.
    •  Current Regulations: the program is operating with up-to-date regulations that
       reflect current contaminants in material disposed of through ocean dumping.
    •  Adequate Information: the program has adequate information to establish
       program criteria, assess the suitability of material proposed for ocean dumping,
       and determine whether ocean dumping sites are meeting environmentally
       acceptable conditions.
    •  Clear Communication: HQ provides Regions with sufficient information that
       will enable them to implement the program consistently in accordance with
       MPRSA, and program managers at the Office of Water understand the purpose
       and function of the Ocean Dumping Management Program. For example, when
       delivering program outputs, such as reports, legislative packages, and guidances,
       the theory rests on the assumption that the HQ program clearly communicates
       information to their customers and partners.
    •  Adequate Program Resources: there is sufficient funding and staff time to
       support all of the activities necessary to achieve program goals.
    •  Program Partners Cooperate with EPA: program partners will cooperate, e.g.,
       by sharing needed information, and thus help achieve program goals.
The evaluation examined the validity of these assumptions, and found that not all of them
hold true.  For example, see the section "Summary of Ocean Dumping Management
Program in Practice" and Exhibit 3-3 for a discussion of examples where the assumptions
are not valid.
                                                                             3-4

-------
PROGRAM INFORMATION FLOWS AND  FEEDBACK MECHANISMS
According to systems theory, "missing feedback is one of the most common causes of
system malfunction. Adding or restoring information can be a powerful intervention" that
causes people to behave differently.7  Thus information flows can serve as a feedback
mechanism that helps program staff and managers adapt their activities. Regional
interviewees commented that they currently rely on the following sources of feedback to
help gather information about the Ocean Dumping Management program and its
functioning:
      •  Information and insights from Ocean Dumping Coordinators in the Regions;
      •  Interagency meetings;
      •  Complaints from program partners;
      •  Regional Dredging Team conversations; and
      •  Public feedback.
In addition, interviewees cited monthly conference calls  for Ocean Dumping
Coordinators as a potentially useful feedback mechanism that currently lacks sufficient
focus and specificity to inform action. For example, instead of providing a brief
overview of regional updates, focus group participants suggest that the calls  could focus
on specific topics or questions raised by the Regions.
Interviewees pointed out there is a need for more consistent transfer of information as to
what is working and not working in the program.  For example, one interviewee said, "I
don't think we've really had time to do that. We've been in a total  crisis mode for years.
The adaptation we've done is: when a crisis hits, we focus on it until the next crisis."
When survey respondents were asked to rate their level of agreement with the following
statement: "I am provided or can access information about program results that helps
inform my work," less than one-third  agreed or strongly  agreed with the statement (See
Exhibit 3-2).
7 Meadows, Donella. Leverage Points: Places to Intervene in a System. The Sustainability Institute, 1999.
                                                                               3-5

-------
EXHIBIT 3-2.   ACCESS TO INFORMATION THAT INFORMS WORK EFFORTS*
                          "I am provided or can access information about program
                                    results that helps inform my work."
                    Agree/Strongly Agree
                Disagree/Strongly Disagree
                Neither Agree nor Disagree
                     Don't Know/Not Sure 0%
                                       0%   5%  10%  15%  20%  25%  30%  35%  40%  45%
               *ln Exhibit 3-2 and subsequent exhibits showing survey respondent agreement or disagreement
               with a statement, agreement is shown in the lighter teal bar, and strong agreement (where it
               occurs) is shown in the darker teal bar. Disagreement is shown in the lighter purple bar, and
               strong disagreement (where it occurs) is shown with dark purple.

               Summary of Ocean Dumping Management Program in Practice
               We can assess how the program theory is working in practice by considering what HQ
               and regional staff and stakeholders said about their experience with program
               implementation. Exhibit 3-3 shows the program logic model, with three additional
               features identified based on interview notes: 1) leverage points that strongly influence
               program effectiveness, 2) process delays and need for improved processes, and 3)
               processes that are not occurring according to interviewees.  Each of these elements is
               described below and illustrated in Exhibit 3-3.

               Leverage Points
               Systems theorists use the term "leverage points" to indicate "places within a complex
               system.. .where a small shift in one thing can produce big changes in everything."8  We
               identify three leverage points within the system of agencies and individuals responsible
               for implementing MPRSA:
                   •   The presence of current regulations seems to be a key leverage point that
                       strongly influences program effectiveness. Lack of current regulations drives a
                       feedback loop, whereby the USAGE does not consistently cooperate with EPA
                       and Regions do not have information to consistently implement MPRSA, which
                       undermines the credibility and effectiveness of the program.  Updating the
                       regulations could lead to improved understanding and cooperation with USAGE
               8 Meadows, Donella.  Leverage Points: Places to Intervene in a System. The Sustainability Institute, 1999.
                                                                                              3-6

-------
        and more consistent and efficient implementation of the program at the regional
        level. Interviewees mentioned the following factors as impeding updates to the
        regulations:  concern about litigation particularly from environmental NGOs,
        difficulty gaining agreement from other agencies (USAGE and NOAA) about
        how to change the regulations, and concern that opening the regulations to
        revision could lead to weakening of the regulations. In order to change the
        current feedback loop, program managers will need to understand and address
        the factors leading to the current situation.
    •   The presence of current guidance is another leverage point that influences
        program effectiveness.  As one interviewee said, "If regulatory reform is not
        going to occur, then the program needs to get [its] guidance to the best place
        possible so that people in the Regions can do the best job possible given the
        circumstances."  Lack of current guidance and training on key topics e.g., testing
        guidance, fish waste, vessel  disposal, and artificial reefs impedes consistent
        program implementation and administration, which increases legal vulnerability
        and reduces program efficiency. Interviewees mentioned the following factors
        as impeding updates to the guidance:  lack of updates to the regulations and
        competing priorities. Updating the guidance would lead to more effective
        program implementation, and could ultimately reduce resource demands by
        requiring less HQ staff time to assist Regions on a case-by-case basis.  One
        interviewee said, "Guidance would save EPA and permittees time and money."
        Another interviewee said, "Having strong consistent coordination and
        implementation and good guidance would really help our program would help
        new folks come on board. It would help us ensure we're being environmentally
        protective across the board."
    •   Effective coordination across Regions is a third leverage point that influences
        program effectiveness. This leverage point is shown in the logic model by the
        activity of conducting a monthly call and semi-annual meeting for ocean
        dumping coordinators.  The program has not held regular in-person meetings in
        recent years, which has reduced the Regions' ability to ensure consistent
        program implementation. Interviewee experience at a recent Ocean Dumping
        Coordinators meeting in Washington B.C. suggests that more regular focused
        conversations between Regions and HQ can improve information flows.
Process Delays and Weed for Improved Processes
In Exhibit 3-3, yellow diamonds represent junctures where interviewees  indicated that a
program process or activity is delayed or impeded. For example, HQ  staff indicated that
their focus on certain near-term activities (e.g., resource reallocation) has shifted their
focus away from addressing some programmatic strategic planning issues, so a delay
symbol appears after the "Strategic planning" activity box. Interviewees  also report
delays  associated with developing policy on emerging issues, and assisting Regions in
site monitoring issues, which influence subsequent elements of the logic model. As
discussed in the text on leverage points, there are also delays associated with updating
guidance and a need for process improvement related to coordinating across Regions.
                                                                               3-7

-------
Regional interviewees also commented that process delays can also stem from friction in
relationships with program partners. For example, although USAGE and EPA may be
"aware of monitoring requirements and conditions at sites," regional staff require
cooperation and adequate information from USAGE in order to conduct "efficient and
effective evaluation of permit proposals."
Processes not Occuring
The red stop sign symbols shown in Exhibit 3-3 indicate processes or activities that are
currently not being undertaken by the program. For example, interviewees conveyed that
regional reviews, conducted by HQ, have not been occurring. While HQ intends to
conduct the regional reviews and does allocate resources to this effort in its work plans,
other near term priorities have taken precedence. Regional personnel see the reviews as
an opportunity to communicate their concerns and feedback to HQ and generate
information that could help improve the program at the regional level. Therefore, the
absence of these reviews represents a missed opportunity to gather feedback on program
operations. In another example discussed above, regulations have not been updated, and
the fact that this process has not occurred is strongly influencing work with USAGE.
Examples of program processes not occurring, being delayed, or needing improvement
point to places in the program theory where the underlying assumptions do not hold true.
For example, the need for updated regulations and guidance suggests that the assumptions
of current regulations, adequate information,  and clear communication do not fully hold
true. Instances of friction with USAGE reduce the extent to which the program has
adequate information and cooperation from program partners. Feedback from program
staff raises  questions about whether the program has adequate resources.
                                                                              3-8

-------
EXHIBIT  3-3.    OCEAN DUMPING  LOGIC MODEL  REFLECTING PROGRAM LEVERAGE POINTS CONVEYED DURING  INTERVIEWS
   Act ivities& Partners
                                    Outputs
                                                                Customers
         policy
Develop and update
legislation (i.e. MPRSA)

Work w/ int'l orgs and
State Dept

Develop policy on
emerging issues (ie. CCS)

Develop regulations &
-o~
\
vs
^
Legislation package
(MPRSA Amendments)

International technical &
t rah rig documents

Annual report to IMO on
ocean dump rig permits

Regulations & policies
                                                                US Congress
Assist regions & USCG to
enforce regulations


Assistance w/ civil
penalties

Develop guidance
documents

Assist regions in site
monitoring & designation
issues

Review regional
programs

Conduct monthly call &
semi-annual meeting for
OD coordinators

Strategic Planning
-&•
-•*
-&*
-0-
Guidance documents (

Assistance with site
monitoring data, SMMPs,
site designation

Regional program review
reports

Coordinators' call notes, /
meeting reports

EPA reports & planning
documents
                                                             USAGE and Permittees
Regional
program offices
         Leverage points that strongly influence program effectiveness


    x
Response letters

Ocean dump rig permits

\ ,
\
\.
\v
Regulated community,
env. groups, public

Ports and port-dependent
industries

\
/

Stakeholders understand
& aware of requirements
for disposal

   USACE & EPA aware of
  monitoring requirements
   and conditions at sites
                                                                                          Increased regional
                                                                                        awareness of policy and
                                                                                           intent of policy
                          Long-term Outcomes
                          (Conditions)
Division, office, OW
management, public


Mgmt aware of program
activities and drection


Mgmt support of program
activities and budget
                 Logic Model Assumptions include:
                 1) Current Regulations
                 2) Adequate Information
                 3) Clear Communication
                 4) Adequate Program Resources
                 5) Program Partners Cooperate with
                 EPA
                                                                                                                                                                          3-9

-------
EXHIBIT 3-4.
ALLOCATION OF PROGRAM RESOURCES
To assess the current allocation of Ocean Dumping Management Program resources, the
evaluation team analyzed how resources in the program's fiscal year (FY) 2012 work
plan support the program's long-term goals.9 Exhibit 3-4 shows the percentage of Ocean
Dumping Management Program FTE hours and ORISE contract dollars10 that are
allocated to each activity in the FY 2012 work plan, based on the total number of FTE
hours and non-FTE hours (i.e. contract dollars). Based on this analysis, Exhibit 3-4
shows that the largest percentage of the program's FTE hours are allocated to activities
associated with developing policy on emerging issues (19 percent) and strategic planning
(16 percent). In terms of ORISE contract dollars, the largest percentage of these
resources is allocated to developing and updating legislation (24 percent) and developing
policy on emerging issues,  e.g., carbon capture and storage (21 percent). However, it is
important to note that in practice staff do not conduct all of the activities that are  allocated
staff resource in the work plan. For example, as noted earlier, regional reviews have not
occurred, even though they received  an allocation of six percent of the FTE hours. Thus
intended allocation may differ somewhat from actual resource allocation.
ALLOCATION OF FTE HOURS AND CONTRACT DOLLARS APPLIED TO OCEAN
DUMPING ACTIVITIES FOR FISCAL YEAR 2012
Develop policy on emerging issues
Strategic Planning
Address petitions, letters, and other inquiries
Conduct monthly call & semi-annual meeting for OD...
Review regional programs
Assist regions in site monitoring & designation issues
Assist regions & USCG to enforce regulations
Assist regions to review & concur on dredged material...
Develop and update legislation
Develop guidance documents
Issue special or general (non-dredged material) permits
Develop regulations & policies, keep current w/ policy
Work w/ int'l orgs and State Dept



0% 6
0%
^™
^^
W
— •-

10%
* 10%
\
10%
7%




-5!°^


24%

0% 5% 10% 15% 20% 25%
• Allocation of FTE MRS Allocation of Contract Dollars
                Program staff gave the evaluation team a copy of the FY2012 work plan.

               10 The work allocation of ORISE fellows are considered non-FTE hours in the work plan and have been converted to contract
                dollars for representation in the logic model. Contract costs associated with the OSV Bold are not included in this analysis.
                                                                                              3-10

-------
               When the resources for specific activities are assigned to subsequent elements of the logic
               model based on the connections in the model, we find that activities leading to
               "environmentally acceptable conditions resulting from disposal" receive the largest
               percentage of FTE hours (46%) and contract dollars (38%), while the long-term goal of
               "no legal vulnerability" receives the fewest FTE hours (15%) and contract dollars (9%).n
               Exhibit 3-5 shows the resource allocation for each of the four long-term program goals.
               (For the details of this analysis, and a depiction of resources across the logic model, see
               Appendix D.)  As noted earlier, these percentages reflect intended allocations of effort,
               and in practice some staff time appears to be diverted to other near term priorities.
EXHIBIT 3-5.   RELATIVE  PRIORITIZATION  OF PROGRAM GOALS FOR FY 2012
                  Environmentally acceptable
                   conditions resulting from
                          disposal

                   Coordinated and effective
                       Ocean Dumping
                    management program
                       No legal vulnerability
                 Maintaining an international
                      leadership role
                                                                         I Allocation of FTE
                                                                         MRS
                                                                         Allocation of
                                                                         Contract Dollars
                                          0%   10%  20%  30%  40%  50%
               ALIGNMENT OF PARTNER GOALS
               The Ocean Dumping Management Program has a variety of program partners whose
               goals span from protection of natural resources to maintaining navigational waterways.
               Exhibit 3-6 shows the program's key partners, which include federal, state, and local
               entities, along with their goals related to ocean dumping.  As noted in the introduction,
               the evaluation team decided to consider EPA Regions as partners of the national Ocean
               Dumping Management Program staff to the extent that they: a) contribute to the
               development of priorities for the program, and b) conduct activities that contribute to
               program outputs. EPA Regions are instrumental in achieving the program's goals.
               1 Contract costs associated with the OSV Bold are not included in this analysis.
                                                                                              3-11

-------
EXHIBIT 3-6.   OCEAN DUMPING PROGRAM  PARTNERS AND GOALS RELATED TO OCEAN DUMPING
PROGRAM PARTNER
EPA Regions
USAGE
Clean Water Act 404 Program
Coast Guard
Department of Interior
Environmental NGOs
Fish and Wildlife Service
National Oceanic and Atmospheric
Administration
State Agencies
Navy
Ports and Harbors
PARTNER GOAL(S) RELATED TO OCEAN DUMPING
Maintaining environmentally acceptable
conditions resulting from disposal.
Maintaining navigation and meeting regulatory
responsibilities for dredged material with a
focus on cost effectiveness.
• Issuing permits for dredged material placed in
ocean waters (from baseline12 to three
nautical miles only) for a purpose other than
dumping, i.e., for the purpose of raising the
bottom elevation (e.g., beneficial reuse)
• Regulating discharge of dredged material to
US waters, including wetlands
Surveillance and enforcement of dumping sites
Overseeing issues pertaining to mineral rights,
carbon capture and storage, and platform/rig
reefing and abandonment.
Natural resource quality and protection
Ensure consistency of state water quality
standards/requirements for dumping that occurs
in state waters
Vessel Disposal (SINKEX)
Materials being dredged and/or placed in these
locations
               To assess how the goals of partners outside EPA align with the goals of EPA's Ocean
               Dumping Management Program, the evaluation team asked program staff to rate their
               agreement with the following statement during the pre-interview survey: "The goals of
               the non-EPA program partners align with the goals of the Ocean Dumping program." No
               respondents agreed with the statement, and over three-quarters of respondents disagreed
               or strongly disagreed with the statement (See Exhibit 3-7).
               12 The United States baseline is the mean lower low water line along the coast, as shown on official United States nautical
                charts. The baseline is drawn across river mouths, the opening of bays, and along the outer points of complex coastlines.
                                                                                                 3-12

-------
EXHIBIT 3-7.   ALIGNMENT OF GOALS WITH NON-EPA PROGRAM PARNTERS
                         "The goals of the non-EPA program partners align with the
                                   goals of the Ocean Dumping program."
                     Agree/Strongly Agree 0 Vo
                 Disagree/Strongly Disagree
                 Neither Agree nor Disagree
                      Don't Know/Not Sure
               69%
8%
                                          4-
                                       0%   10%  20%  30%  40%  50%  60%  70%  80%  90%
              Interviewees specifically noted a disconnect with the goals of USAGE. USACE's goal is
              to maintain navigational waterways while meeting regulatory responsibilities in a cost-
              effective manner and with minimal delays. At times, this focus conflicts with the Ocean
              Dumping Management Program's focus on ensuring adequate testing, monitoring, and
              review. This has led to friction between EPA and USAGE in some Regions and Districts.
              For example, in EPA Region 6, the program struggles to obtain program-related data and
              information from USAGE.
              Aside from USAGE, interviewees pointed out occasional overlaps in responsibilities and
              potential tension with other program partners. For example, while the Ocean Dumping
              Management Program, along with USAGE, oversee the disposal of dredged material, the
              CWA 404 program also becomes involved when dredged material is placed in the ocean
              within 0-3 miles from the baseline for a purpose other than disposal (i.e., raising the
              bottom elevation).  Where both MPRSA (ocean dumping) and CWA 404 programs
              intersect,  some interviewees questioned which program should be providing guidance and
              authority. Interviewees indicated that at HQ, there is a clear delineation between the
              CWA 404 and Ocean Dumping Management Program staff, but in some Regions the
              same individuals may work in both programs. In another example, one interviewee  noted
              that U.S. Fish and Wildlife Service has shifted their regional relationship from one of
              proactive collaboration to an increasing level of litigation.
                                                                                         3-13

-------
Conclusions
The program's activities and outputs appear well structured to lead to desired outcomes,
and the program's intended resource prioritization is appropriate and consistent with
EPA's mission, since environmental protection is the top priority. Nevertheless, several
assumptions that underlie the program theory do not hold true, and it will be important for
the program to address these challenges.
In order to operate effectively the program will need to address key leverage points (e.g.,
updating regulations and guidance, and ensuring coordination across Regions).
While some misalignment amongst agency and organizational missions is typical, tension
with USAGE is posing a serious challenge for the Ocean Dumping Management
Program, even though some Regions have a positive working relationship with USAGE.
Ensuring a more consistently productive relationship with USAGE appears important to
helping the program achieve its goals.
Recommendations
The evaluation team recommends that the Ocean Dumping Management Program:
    •   Update program regulations and/or guidance to improve consistency of program
       implementation and potentially improve relationships with USAGE;
    •   Work towards more productive communication with USAGE by acknowledging
       differences in mission and organizational constraints, and focusing on areas of
       shared responsibility;
    •   Clarify and communicate to senior EPA management the importance of the
       program, focusing on why this program is essential to protecting ocean
       ecosystems; and
    •   Consider rebranding the program with a new name and new focused outreach
       approaches designed to communicate to key audiences (e.g., co-regulators at
       federal and state levels, permittees, and environmental groups).
                                                                             3-14

-------
CHAPTER 4  |   EMERGING TRENDS, OPPORTUNITIES,  AND
                  CHALLENGES
The Ocean Dumping Management Program has encountered significant changes over the
past two decades, and faces emerging trends, opportunities, and challenges. This section
describes our findings related to: 1) historical trends and emerging issues, 2) obstacles
and delays, and 3) opportunities for improvement. At the end of the section we present
our conclusions and recommendations.
FINDINGS
HISTORICAL TRENDS AND  EMERGING ISSUES
We identified four historical trends important to understanding the context of the Ocean
Dumping Management Program: 1) decreases in ocean dumping, and an increased focus
on beneficial reuse, 2) advances in science and understanding of contaminants, 3)
changes in program resources, and 4) staff turnover. Though there is some variation,
these trends are common across all  of the Regions.  After considering common historical
trends and emerging issues, we point out trends and issues that are specific to particular
Regions.

Decreases in  Ocean Dumping,  Increased Focus on Beneficial Reuse
Since the 1972 passage of MPRSA, ocean dumping of many types of substances has been
phased out (with the notable exception of material from navigational dredging, which is
discussed below).  In particular, sewage sludge, which was dumped in the ocean by many
cities starting in the 1920s, was outlawed by the Ocean Dumping Ban Act, which
prohibited all ocean dumping of industrial waste and municipal sludge after December
31, 1991. The final city to cease dumping, New York City, ended the practice on June 30,
1992. On that date, the Deputy Administrator for EPA's Region 2 stated that "EPA is
committed to  ... ensuring] that this problem is not just moved from the ocean to land,
but that long-term, land-based disposal options are implemented. We will also continue to
encourage the use of sludge as a benefit to the environment."13
The theme of limiting ocean dumping and instead finding new uses for materials
previously dumped in the ocean has continued, with a new focus on dredged material.
Dredged material from navigation channels is now the predominant substance disposed of
through ocean dumping, and managing this disposal has occupied a substantial portion of
the Ocean Dumping  Management Program's time and resources. Between 1998 and
2009, there were 651 dredged material permits issued by USAGE following EPA review
13 U.S. EPA Region 2 Press Release, "EPA Declares End of Ocean Dumping as New York City Ceases; EPA Committed to Long-
 Term Beneficial Alternatives, "June 30, 1992, http://www.epa.gov/aboutepa/history/topics/mprsa/03.html
                                                                            4-1

-------
and concurrence. During this same period, an EPA general permit enabled the disposal of
50 vessels, seven emergency permits were issued by EPA, and two fish waste permits
were issued (and reissued) by EPA.14 EPA has also issued permits to the U.S. Navy and
the National Science Foundation for the transportation of target vessels and ice piers in
Antarctica, respectively.
Exhibit 4-1  shows that on a national level, there was a pronounced decline in ocean
dumping of dredged material starting in the early 2000s, with a record low in 2007,
although in recent years the national trend has started to reverse.  The amount of dredged
material dumped at sea depends not only on permitting decisions by the Ocean Dumping
Management Program and USAGE, but also  on the amount of dredging occurring, the
contamination of dredged material, and the funds available for land disposal or beneficial
reuse.
Dredging trends vary widely by Region and fluctuate frequently. Regions 4 and 6 have
concurred on permits for the greatest volume of ocean dumping in most years of the
program's history.  Ports in these EPA Regions are currently undergoing major harbor
and channel deepening projects as they seek to accommodate post-Panamax vessels, a
very large class of ship that will begin transporting goods internationally in coming
years.15  The USAGE has identified the Southeast and Gulf coast ports as top candidates
for economically justified port expansion/dredging projects.16 One interviewee noted,
"We have a number of ports that want to deepen for the post-Panamax vessels - [these
projects] put a strain on the program, [because they require]  a lot of testing to review a lot
of material."
M National Ocean Disposal Activities Excel file, dated 9-11-12, provided by EPA Ocean Dumping Staff. There are also other
 activities covered by EPA permits not listed in the spreadsheet. A burial at sea permit was issued in 1977; EPA does not
 report individual burial at sea activities. The National Ocean Disposal Activities Excel file does not list activities related to
 transportation for target practice permits in the time period described.

15 http://www.pancanal.com/common/maritime/advisories/2009/a-02-2009.pdf

16 Institute for Water Resources, U.S. Army Corps of Engineers, "U.S. Port and Inland Waterways Modernization: Preparing for
 Post-Panamax Vessels: Report Summary." June 20, 2012. http://www.iwr.usace.army.mil/index.php/us-port-and-inland-
 waterways-moderinzation-strategy
                                                                                    4-2

-------
EXHIBIT 4-1.  OCEAN DUMPING OF DREDGED MATERIAL
                  120,000,000
                  100,000,000
                   80,000,000
                   60,000,000
                   40,000,000
                   20,000,000
                           1976 1978 1980 1982 19841986 1988 19901992 1994 19961998 2000 2002 2004 2006 2008 2010
               Source: USAGE. 2011. Ocean Disposal Database, http://el.erdc.usace.armv.mil/0cean Dumping/amount by all districts.asp
               EPA has encouraged beneficial reuse or land disposal rather than ocean dumping where
               feasible or necessary to protect marine habitats. USAGE reports that currently "about 20
               to 30 percent of port and waterway dredged material is used for habitat creation and other
               beneficial use."17 Program staff note the connection between climate change, sea level
               rise, and beneficial reuse. For example, one survey respondent identified "climate
               change" as one of the top two emerging trends for the Ocean Dumping Management
               Program and said, "More and more, we are reviewing projects where sediments are being
               proposed for beneficial reuse, as protective measures for repair of dikes and providing
               sandy sediments for beaches and other coastal areas for protection of infrastructure."
               Another respondent identified "alternatives to ocean disposal, especially in light of
               climate change and sea level rise" as one of the top two emerging trends. These responses
               highlight the extent to which EPA and its partners are increasingly considering beneficial
               reuse as an alternative to ocean dumping.

               Advances in Science and Understanding of Contaminants
               Over the last two decades, EPA and USAGE have jointly developed and updated several
               guidance documents for implementing various aspects of MPRSA. These include
               document:
               17 Institute for Water Resources, U.S. Army Corps of Engineers, "U.S. Port and Inland Waterways Modernization: Preparing for
                Post-Panamax Vessels: Report Summary." June 20, 2012. http://www.iwr.usace.army.mil/index.php/us-port-and-inland-
                waterways-modernization-strategy
                                                                                                   4-3

-------
    •  For evaluating alternative disposal methods for dredged material: Evaluating
       Environmental Effects of Dredged Material Management Alternatives — A
       Technical Framework18 issued jointly by EPA and USAGE in 1992 and revised
       in 2004. This document provides a general technical framework for evaluating
       the environmental acceptability of dredged material management alternatives,
       including open-water disposal (including ocean dumping), confined disposal (this
       includes disposal on nearshore or upland locations), and beneficial uses. The
       framework augments other existing program guidance, and is intended to foster
       enhanced consistency and coordination in USACE/USEPA decisions regarding
       dredged material management.
    •  For assessing the suitability of dredged material for ocean disposal:
       Evaluation of Dredged Material Proposed for Ocean Disposal Testing Manual,19
       or the Green Book, which EPA issued jointly with USAGE in February 1991.
       The manual contains technical guidance for determining the suitability of
       dredged material for ocean disposal through chemical, physical, and biological
       evaluations. The Green Book updated the prior testing manual, the Ecological
       Evaluation of Proposed Discharge of Dredged Material into Ocean Waters,
       published in 1977. The 1991 update introduced atiered testing approach that
       sequentially increases the amount of information collected to fully consider
       biological impacts.20
    •  For preparing ocean dredged material disposal site management plans:
       Guidance Document for Development of Site Management Plans for Ocean
       Dredged Material Disposal Sites developed jointly by EPA and USAGE in 1996.
       This document lays out a recommended framework for site management plan
       development and content.
EPA and USAGE have provided several updates to these guidance documents. For
example, in the 1990s, EPA made  changes to its bioaccumulation testing procedures that
had a substantial impact on the amount of material that could be placed in a major ocean
dumping site off of the Port of New York and New Jersey. These changes resulted in re-
designating the location as a remediation site that would accept only uncontaminated
sediment.21  Between 2000 and 2003, EPA tightened its standards for substances
contaminated with polychlorinated biphenyls, or PCBs, permitted for ocean dumping in
the same EPA Region. These changes in guidance and standards were the subject of (and
some say a product of) a series of lawsuits initiated by Clean Ocean Action and other
environmental advocacy groups pushing for more rigorous bioaccumulation testing
procedures, and a lawsuit from the United States Gypsum Company arguing for EPA to
 http://water.epa.gov/type/oceb/oceandumping/dredgedmaterial/upload/2004_08_20_oceans_regulatory_ dumpdredged_
 f ramework_techf ramework. pdf

19 http://water.epa.gov/type/oceb/oceandumping/dredgedmaterial/upload/gbook.pdf

20 Burroughs, Richard and Christine Santora, "Disposal of Contaminated Sediment from The Port of New York," University of
 Rhode Island Transportation Center, June 2004.

21 Ibid.
                                                                               4-4

-------
maintain its current PCB standards for the purpose of its permit application. The complex
history of these lawsuits and related negotiations, public involvement process, and
involvement of Congressional delegations and Vice President Gore is beyond the scope
of the current document, but is described in existing literature.22
The most recent update made to the ocean dumping guidance was the 2004 framework
for evaluating alternative disposal methods for dredged material. The underlying
legislation (MPRSA) was last updated in 1992 by the Water Resources Development Act
of 1992 (Public Law 102-580).23 As noted earlier, in FY2012, the Ocean Dumping
Management Program prioritized activities including strategic planning and developing
policy on emerging issues; in contrast the activity of developing guidance documents was
the focus for only three percent of total FTE hours  (see Exhibit  3-4).
Overall, program staff and stakeholders expressed concern that  the regulations and
guidance documents have not kept pace with advances in science and therefore need to be
updated. Several program staff mentioned outdated regulations, new contaminants, and
the need to revise testing and assessment methods as one of the top two emerging trends
for the Ocean Dumping program. For example, one survey respondent discussed
"emerging contaminants," explaining that "Pyrethroids are coming to the forefront in
various watersheds ... we are beginning to look into this class of pesticides as a source of
acute and chronic toxicity, but we will soon need guidance from HQ and ORD [the EPA
Office of Research and Development]. Similarly, pharmaceuticals may be implicated in
contributing to chronic toxicity in sediments." Another respondent said that a top
challenge is "sediment testing improvements (including ocean dumping regulation
revisions): how/when to incorporate emerging contaminants, combining the Ocean and
Inland Testing Manuals, etc."  For example, one interviewee said, "We know so much
more about  contamination and carcinogens nowadays, but we don't sample for them
because the  1970's regulations don't [cover] them."
A new class of substances related to climate change that may be proposed for ocean
dumping, including sub-seabed carbon sequestration and geo-engineering (e.g., ocean
fertilization), constitutes an important emerging issue that will likely receive increased
public attention overtime, particularly in light of their potential for environmental
impacts. For example, Canada's environmental ministry is currently investigating an
incident involving the Haida Salmon Restoration Corporation and of a California
businessman who reportedly dumped iron dust approximately 200 nautical miles of the
coast of British Columbia.24 The iron sparked the growth of plankton, which was
intended to aid the recovery of the  local salmon fishery for the native Haida as well as to
sequester carbon, because plankton absorbs carbon dioxide and settles deep in the ocean
22 For a detailed history, see Burroughs, Richard and Christine Santora, "Disposal of Contaminated Sediment from The Port of
 New York," University of Rhode Island Transportation Center, June 2004. Additional analysis of the case is included in
 Brown, N. and R.W. Knecht.  1998. "The New York/New Jersey Harbor dredging conflict." In J.R. Waldman, W.C. Nieder
 (eds.), Final Reports of the Tibor T. Polgar Fellowship Program, 1997. Hudson River Foundation.

23 http://www.epa.gov/owow/oceans/ndt/publications/pdf71996_smmp_guidance.pdf

24 Fountain, Henry, "A Rouge Climate Experiment has Ocean Experts Outraged." New York Times, section A1, Friday October
 19, 2012.
                                                                                 4-5

-------
when it dies.25 This issue was raised at the 29 October -2 November 2012 London
Convention/London Protocol meeting where the Parties to the London
Convention/London Protocol issued a statement of concern regarding this ocean
fertilization activity. The EPA Ocean Dumping Management Program could be called
upon to prevent or enforce against similar activities off the U.S. coast in the future.

Changes in Program Resources
As funding for  OWOW and the Office of Water overall have increased, funding for the
Marine Pollution Control Branch, which includes the Ocean Dumping Management
Program, has declined 14 percent in the last six years (Exhibit 4-2). During the same
period, while the number of HQ OWOW staff FTEs allocated to the Ocean Dumping
Management Program has more than doubled, the overall FTE allocated to the HQ
program remains relatively small (less than 4 FTE) (Exhibit 4-3). There is broad
consensus among regional and HQ program staff interviewed that they are being asked to
do "more with less," and that they have fewer resources to do their jobs.  Ten of 13
survey respondents cited reduced or insufficient funding as  a top emerging issue or
challenge for the program. One survey respondent said a top challenge for the program is
"limited resources and a continually growing portfolio (a broad array of issues)."
Program staff have been actively discussing EPA's disinvestment in the Ocean Survey
Vessel Bold and its implications for the Ocean Dumping Management Program. Ten of
13 survey respondents identified the loss of the Bold or maintaining sufficient  funding for
monitoring as one of the top two emerging issues or challenges for the program.
Interviewees from several regional offices, particularly those on the east coast, were
concerned about the impact of the loss of the Bold on monitoring. One interviewee
explained that the vessel was the program's most important monitoring tool and its only
solid asset. Interviewees worried that the loss of the ship would open the door to future
budget cuts.  As one interviewee explained, "One of the things about having a  ship is that
you can't cut [the budget] below $5 million. [But] once you get rid of it you can cut it
whenever you want." Another interviewee remarked, "We need to be developing a
strategy for that [monitoring] money [to replace the Bold] so we don't lose it, so it's
predictable,  [and] so we can plan." Other interviewees expressed concern about the
monitoring work the Bold used to do for programs other than the Ocean Dumping
Management Program (e.g., related to corals and Total Maximum Daily Load
monitoring), which will no longer be supported. On the other hand, some interviewees
thought that  the disinvestment from the Bold could bring more equity in regional
monitoring resources.  One interviewee said "With the east coast having the majority use
of the vessel, their approach to monitoring [has been] really different than ours.... But
now there's  an opportunity to have all the Regions look at how ocean disposal sites are
managed and monitored in a consistent manner."
25 Fountain, Henry, "A Rouge Climate Experiment has Ocean Experts Outraged." New York Times, section A1, Friday October
 19, 2012.
                                                                              4-6

-------
Program managers note that budget cutting exercises take time in and of themselves,
which could be better directed toward substantive planning for the program. For
example, one interviewee said, "We have spent a lot of time on budgeting exercises for
the last year and a half... with all of those resources we could have done things that
would have been more important. Rather than working on the budget level allocation, we
should be working on planning [for the future]... for example how to meet required
monitoring levels [and] how to even out resources between the Regions."
Program staff report they are concerned that they are less involved in resource decisions
than they have been in the past. One interviewee said, "Before the last two or so years,
program staff across Regions had a bottom-up style of communication that fostered team
decision-making  and a stewardship approach to work. For example, if a budget cut had to
made, we would all talk it out until we could come to some understanding of how best to
proceed. Now, we have managers at the  next level above us making decisions that we
don't necessarily agree with, and they inform us of those decision through rather poor
messaging. ... There is a lack of transparency."  One manager from the Office of Water
reports that the Agency as a whole is under severe budget constraints, and that the Ocean
Dumping Management Program needs to clearly communicate what it does and the value
of the program to the EPA mission.
                                                                             4-7

-------
EXHIBIT 4-2.   RESOURCE TRENDS WITHIN THE OFFICE OF WATER (DOLLARS IN THOUSANDS)'


PROGRAM

- RELATED GOAL(S)
Office of Water
- Clean and Safe Water/
Protecting America's Waters
Office of Wetlands, Oceans,
and Watersheds
Protect Water Quality
Marine Pollution
PERCENT
CHANGE OVER

SIX YEAR
PERIOD

21%


55%

(14%)

FY2008

ACTUAL
BUDGET

$3,119,201.2


$1,658,310.4

$13,430.4

FY2009

ACTUAL
BUDGET

$8,887,323.4


$5,538,892.8

$13,064.7

FY2010

ACTUAL
BUDGET

$4,989,963.6


$3,375,542.5

$9,783.7

FY2011

ACTUAL
BUDGET

$5,085,863.7


$3,553,462.7

$15,570.5

FY2012

ENACTED
BUDGET

$4,094,452.5


$2,798,913.5

$12,898.0

FY2013

PRESIDENT'S
BUDGET

$3,782,228.0


$2,565,462.0

$11,587.0
T Source: EPA Congressional Justifications, available at http://www.epa.gov/planandbudget/archive.html. Includes contract costs associated with the OSV Bold.


EXHIBIT 4-3.   RESOURCE TRENDS WITHIN THE OFFICE OF WATER (ANNUALIZED FULL-TIME EQUIVALENTS)'

PROGRAM
RELATED GOAL(S)
Office of Water
- Clean and Safe Water/
Protecting America's Waters
Marine Pollution
EPA HQ Ocean
Dumping Management
Program*
OWOW FIE
PERCENT
CHANGE OVER
SIX YEAR
PERIOD
21 .4%
2.1%
128%

FY2008
ACTUAL
BUDGET
2,815.1
42.8
1.71

FY2009
ACTUAL
BUDGET
2,868.3
44
1.30

FY2010
ACTUAL
BUDGET
3,471.3
41.5
1.00

FY2011
ACTUAL
BUDGET
3,510.3
44.5
2.5

FY2012
ENACTED
BUDGET
3,423.6
43.7
3.6

FY2013
BUDGET
3,418.9
43.7
3.9
 Source: EPA Congressional Justifications, available at http://www.epa.gov/planandbudget/archive.html

* Source: Work plans for FY 08 through FY 13 provided by Ocean Dumping Management Program staff. Where possible, values taken from subsequent year's work plan (to
more closely account for FTEs actually used, not just planned).
                                                                                                                                               4-8

-------
Staff Turnover
The Ocean Dumping Management Program has experienced considerable staff turnover
in recent years.  One interviewee noted that three of the seven Ocean Dumping Regions
have lost staff with long tenures over the last seven years. Another interviewee noted that
"It's hard to know what to do and how to do it in this program unless you've been in it a
long time. We need checklists...  and other documentation to get people up to speed
quickly." Loss of institutional knowledge about the Ocean Dumping Management
Program's processes and procedures may lead to inefficiencies in permitting, monitoring,
and other activities.

Trends and Emerging Issues Specific  to EPA  Regions
In addition to common historical trends and emerging issues cited by respondents, several
issues were cited by only one or two Regions as a high priority in their jurisdiction.
These issues include:
     •  Restoring the Gulf of Mexico, including post British Petroleum Deepwater
        Horizon work in the Gulf of Mexico related to Ocean Dredged Material Disposal
        Sites and harbors;
     •  Managing major harbor and channel deepening projects in the southeastern
        US.  As noted earlier, ports in  this area are expected to undergo major expansion
        in coming years to accommodate post-Panamax vessels;
     •  Ensuring proper disposal of  debris from the Japanese tsunami on the west
        coast, and avoiding introduction of invasive species on the west coast;
     •  Ensuring proper disposal of  fish waste in Alaska and on the west coast; and
     •  Protecting sensitive near shore habitats (e.g., reefs) in the Caribbean from
        effects of dredged material transport.
Survey respondents are relatively evenly divided over whether HQ and Regions face the
same emerging issues and challenges:  just over half (54 percent) of survey respondents
report that EPA HQ and Regions face the same emerging  issues, and a slightly smaller
share of survey respondents (46 percent) agree that EPA HQ and Regions face the same
challenges (see Exhibits 4-4  and 4-5).  The survey responses do not  suggest any notable
patterns of agreement or disagreement  based on the Region of the country or the tenure of
staff responding.
                                                                            4-9

-------
EXHIBIT 4-4.   SHARED HEADQUARTERS AND  REGIONAL EMERGING ISSUES
                        "Headquarters and regional staff face the same emerging
                                               issues."
                    Agree/Strongly Agree
               Disagree/Strongly Disagree
               Neither Agree nor Disagree 0'/0
                    Don't Know/Not Sure
8%
                                     0%     10%    20%     30%    40%     50%    60%
EXHIBIT 4-5.   SHARED HEADQUARTERS AND  REGIONAL CHALLENGES
                        "Headquarters and regional staff face the same program
                                             challenges."
                    Agree/Strongly Agree
               Disagree/Strongly Disagree
               Neither Agree nor Disagree     8%
                    Don't Know/Not Sure
                                     0%      10%      20%      30%      40%      50%
                                                                                        4-10

-------
              OBSTACLES AND DELAYS
              Findings from this evaluation suggest that the Ocean Dumping Management Program has
              faced obstacles and delays in recent years.  Eighty-four percent of program staff surveyed
              agree or strongly with the statement that "Over the past three years the Ocean Dumping
              Program has encountered significant obstacles and/or delays to accomplishing its
              mission" (Exhibit 4-6).  Obstacles cited include:
                  •   Lack of adequate time and resources;
                  •   Lack of clear priorities and "mission creep;"
                  •   Limited senior management direction/feedback;
                  •   Administrative "fire drills;"
                  •   Loss of the Bold and associated increases in administration time to find other
                      vessels to serve this function;
                  •   Lack of funding for monitoring Ocean Dredged Material Disposal Sites so effects
                      to marine species are not adequately understood;
                  •   USAGE disagreement or inflexibility and lack of communication with some
                      USAGE Districts;
                  •   Lack of cooperation between federal and state agencies, despite best efforts of
                      Regional Dredging Team; and
                  •   Threat of lawsuits.
EXHIBIT 4-6.  ENCOUNTERING OBSTACLES AND DELAYS
                         "Over the past three years the Ocean Dumping program has
                             encountered significant obstacles and/or delays to
                                         accomplishing its mission."
                    Agree/Strongly Agree
                Disagree/Strongly Disagree 0'/0
                                Neutral
                    Don't Know/Not Sure 0 Vo
                                      0%   10%  20%   30%  40%   50%  60%  70%  80%  90%
                                                                                          4-11

-------
              OPPORTUNITIES FOR IMPROVEMENT
              Program managers seeking to improve the Ocean Dumping Management Program will
              need to anticipate emerging issues, challenges, and opportunities. Survey results suggest
              that to date, the Ocean Dumping Management Program has done a better job of
              anticipating emerging issues compared to anticipating challenges and opportunities, but
              in general the program has room to improve in both of these areas. Specifically, only 31
              percent of respondents agreed or strongly agreed with the statement "Over the past three
              years, the  Ocean Dumping program as a whole has done a good job of anticipating
              emerging issues" (roughly two-thirds of respondents disagreed with this statement or did
              not express an opinion) (see Exhibit 4-7). Only 15 percent of respondents agreed with
              the statement "Over the past three years, the Ocean Dumping program as a whole has
              done a good job of anticipating challenges and opportunities," and no respondents
              strongly agreed (see Exhibit 4-8). These findings suggest that the Ocean Dumping
              program needs a renewed focus on strategic planning, anticipating emerging challenges,
              and seizing potential opportunities. In the following sections, we highlight three areas of
              opportunity for the Ocean Dumping program to focus on improvements: 1) exercising
              greater coordination and leadership from EPA HQ, 2) updating regulations and guidance,
              and 3) supporting London Protocol ratification.
EXHIBIT 4-7.  ANTICIPATING EMERGING ISSUES
                       "Over the past three years the Ocean Dumping program as a
                       whole has done a good job of anticipating emerging issues."
                    Agree/Strongly Agree
                Disagree/Strongly Disagree
                Neither Agree nor Disagree
                     Don't Know/Not Sure      8%
                                       0%    5%    10%   15%   20%    25%    30%   35%
                                                                                           4-12

-------
EXHIBIT 4-8.  ANTICIPATING  EMERGING CHALLENGES AND OPPORTUNITIES
                       "Over the past three years the Ocean Dumping program as a
                        whole has done a good job of anticipating challenges and
                                              opportunities."
                     Agree/Strongly Agree
                Disagree/Strongly Disagree
                Neither Agree nor Disagree
                     Don't Know/Not Sure
                                       0%     10%    20%    30%    40%     50%     60%
              Opportunity to Exercise Greater Coordination and Leadership from EPA HQ
              Interviewees and focus group participants suggest that, particularly in light of the recent
              staff turnover, EPA HQ could provide more leadership, facilitation, and coordination for
              the Regions. One survey respondent reported that a major challenge for the program was
              the "lack of consistency and standard operating procedures between Regions." Regions
              seek a centralized repository of identified checklists, resource lists, and other resources.
              Participants in the Ocean Dumping Coordinators meeting acknowledged that the creation
              of training documentation may represent a large investment of time and effort in the short
              term but is a necessary step to address staff turnover and foster the long term resilience of
              the program.
              Staff praised EPA HQ's recent convening of an  Ocean Dumping Coordinators meeting in
              Washington, D.C. as an opportunity to foster coordination across the Regions.
              Participants at the meeting acknowledged that regular in-person meetings might not be
              possible due to resource constraints, but suggested that EPA HQ seek creative means to
              gather Ocean Dumping coordinators from across the country in focused conversations
              (e.g., through facilitated conference calls and/or meetings using online document sharing
              and participation software such as GotoMeeting, WebEx, or Microsoft 365).  Meeting
              participants also acknowledged that EPA HQ does hold regular conference calls, but
              pointed out that they tend to be an opportunity for updates, rather than focusing on a
              particular topic of concern.
                                                                                           4-13

-------
In addition to coordination across Regions, staff seek greater leadership from EPA HQ.
During the strengths, weaknesses, opportunities, and threats analysis at the Ocean
Dumping Coordinators meeting, participants identified an absence of leadership as a key
threat or weakness for the Ocean Dumping Management Program.  One interviewee said,
"There is a national consistency issue: [we need to] get HQ on the same page [and get]
information coming down from the top telling [USAGE] Districts what they need to do."
Another interviewee reported, "We have five or six voices telling you how to do things.
[There is]  unclear program direction [and a] lack of a unified voice from leadership."

Opportunity to Update Regulations and Guidance
Although there is broad-based agreement among Ocean Dumping Management Program
staff and program stakeholders that it is important to update the regulations and program
guidance,  several barriers make this task difficult.  First, the regulations themselves are
quite detailed and need to be at least partially updated, if the program is to fully update
the technical guidance. Moreover, stakeholders do not agree on how to update the
regulations. One interviewee explained that the program's technical guidance cannot be
changed because it was codified into the original regulations. Another challenge is that
current resource constraints and demands on program staff time have taken focus away
from efforts to update the regulations and guidance. As noted earlier, the program has
updated its testing guidance in the past, which was the subject of litigation. One of the
four long term outcomes that the program says it is trying to achieve is "no legal
vulnerability." One interviewee suggested that the program's emphasis on avoiding
litigation has "stymied program creativity." Another interviewee  remarked that "It would
save everyone a lot of work to update the regulations because they are not working," but
that the "risk of changing the regulations is to EPA, not USAGE," presumably because
once open for negotiation the regulations could become less protective of the marine
environment.
The fact that the Ocean Dumping Management Program has not updated the guidance in
recent years is straining relations with USAGE and making program implementation
more challenging for the Regions. The Ocean Dumping Management Program is
planning to update the MPRSA regulations in conjunction with the London Protocol
Ratification process, which could have the added benefit of potentially improving
relations with USAGE. The London Protocol Ratification process is described in the next
section.

Opportunity Related to London Protocol Ratification
The London Convention, which the U.S. ratified in 1975, covers the deliberate disposal
of wastes or other matter from vessels, aircraft, platforms, and other man-made structures
at sea. In 1992, the 80  Parties to the London Convention began a comprehensive review
of the Convention, resulting in a new treaty called the 1996 Protocol.26  The London
Protocol, which the U.S. has signed but not yet ratified, updates and improves upon the
London Convention.  Currently, some Parties to the London Protocol are interested in
 6 http://water.epa.gov/type/oceb/oceandumping/dredgedmaterial/londonconvention.cfm
                                                                             4-14

-------
developing a binding regulation to address marine geoengineering activities, including
ocean fertilization, through an amendment to the London Protocol.
The United States signed the 1996 London Protocol in 1998. Since then, the United
States has been working on the ratification process via implementing legislation
(amendments to the MPRSA) and seeking necessary action by the Senate. In 2007, the
White House submitted the London Protocol to the U.S. Senate for its advice and
consent. The London Protocol is on the Obama Administration's treaty priority list. The
proposed implementing legislation submitted to Congress in 2008 is undergoing review
prior to re-submittal to Congress. The Ocean Dumping Management Program is currently
reviewing the MPRSA legislation package submitted to Congress in 2008 to align with
the requirements of the London Protocol. One interviewee noted that, "Updating MPRSA
would allow us .. .to fix a few things to be more efficient for how we implement the
Ocean Dumping Management Program." For example, one interviewee explained that
"right now [the program] can only issue an emergency permit if it's  a threat to human
health. We don't have the authority to issue permits to deal with an  emergency if the
threat is to environmental health."
If the MPRSA is revised, the Ocean Dumping Management Program will also have an
opportunity to  adjust the scope of the regulation. For example, the program may be
called upon to  help mitigate emerging issues like ocean acidification and climate change
(e.g., ocean fertilization). Interviewees disagreed about whether the program should
expand its scope to address emerging ocean  problems,  such as acidification, sea level
rise, and carbon capture. Some program staff felt that the program should expand its
purview beyond dumping to address these issues. For example, one regional interviewee
suggested that  EPA is likely "missing out" on helping with issues such as "ocean
acidification, invasive species, and marine debris." A survey respondent remarked "[The
program] need[s] to expand [the] scope of [the] program beyond just ocean dumping; we
need to be looking at other ocean issues, and there are a lot of them." Other staff felt that
expanding the  scope of the program would be a distraction for a program with a narrowly
focused role and a lack of resources. Some interviewees expressed skepticism about the
viability of carbon sequestration in deep ocean waters.  Considering  which of these issues
to include in its scope will be a major strategic decision for the Ocean Dumping
Management Program going forward.
CONCLUSIONS
The Ocean Dumping Management Program has been challenged to clearly articulate the
scope and benefits of the program, given shifts in the types of materials proposed for
ocean disposal, multiple agencies involved in ocean dumping permitting, and reduced
public attention to environmental and human health risks of ocean disposal.
The changing nature of contaminants and advances in monitoring and testing
technologies require updates to EPA requirements and guidance. While the  Ocean
Dumping Management Program has jointly developed and updated several guidance
                                                                            4-15

-------
documents for implementing various aspects of MPRSA with USAGE, the differences in
perspective between the two agencies, multiple program priorities, and limits to EPA
staff resources have constrained EPA's ability to keep pace with needed updates.
The Ocean Dumping Management Program is working with a very small staff at HQ, and
is challenged to provide effective coordination, leadership, and guidance to the Regions.
Staff turnover and associated loss of institutional knowledge, insufficient guidance and
documentation, limited opportunities for coordination across Regions, and varying levels
of cooperation from USAGE have hampered consistent implementation of the program
across the country.
RECOMMENDATIONS
The evaluation team recommends that the Ocean Dumping Management Program:
    •   Update program guidance and use the London Protocol ratification process as an
       opportunity to update MPRSA;
    •   Identify approaches to take on the additional responsibilities that will come with
       ratification;
    •   As a part of updating program guidance on regulations, work to improve
       communications to Regions about how to implement the regulation, standard
       procedures, and expectations  for program partners (e.g., USAGE Districts);
    •   Foster communication and consistency across Regions; and
    •   Seek to foster improved communication and partnership with USAGE.
                                                                            4-16

-------
CHAPTER 5  |   RESOURCE ALLOCATION
Given current resource trends and a broadening portfolio of work, the Ocean Dumping
Management Program seeks to identify opportunities for improving the alignment
between resource deployment and the program's intended or desired outcomes.  We
group our findings into four main topics: 1) resources that are required vs. those that can
be reallocated, 2) opportunities to improve alignment of resources with intended program
outcomes, 3) opportunities to improve alignment of resources across Regions, and 4)
opportunities, risks, and impacts of resource reallocation. Conclusions and
recommendations follow the findings.
FINDINGS
RESOURCES REQUIRED VS. THOSE THAT CAN BE REALLOCATED
Most individuals we interviewed believe that the program is operating with minimal
resources and that there is no opportunity to be gained by moving resources out of the
program. For example, one interviewee said, "This is a program that in my view never
had any 'fat.' This program is running on empty.  It's really hard for me to think what
they could focus their resources on better; they're only doing the absolute core essential
elements of the program because that's all they can do." Another interviewee said, "Can
the program be cut more and still meet the regulatory requirements? There are certain
aspects of [the program] that really aren't discretionary; when EPA looks at
disinvestment, [it should] look at whether this is something some other agency is doing,
[and whether the Ocean Dumping  Management Program] has authorities that no one else
has.  [Resource reductions are] cutting into the bones of the agency we're supposed to
keep alive." Interviewees point out that the Ocean Dumping Management Program has
authority for enforcement, permits (specifically concurring on dredged material ocean
dumping permits and issuing ocean dumping permits for all other substances), site
management, and site monitoring, and that EPA cannot meet these responsibilities with
fewer resources. Only one interviewee (outside of EPA) suggested that the program had
resources to spare: this person said "EPA's 35-person staff is still the same size as it was
back when everything (not just dredged material) was dumped in the ocean." Program
FTE data shown in Exhibit 4-3 suggests that this size staff more likely relates to the
Marine Pollution Control Branch as a whole, not just the Ocean Dumping Management
Program.

OPPORTUNITIES TO IMPROVE ALIGNMENT  OF RESOURCES WITH  INTENDED
PROGRAM OUTCOMES
While not explicitly addressing opportunities to improve resource alignment,
interviewees did suggest situations where resource alignment is not currently optimized.
                                                                            5-1

-------
For example, interviewees generally agree that the program could do a better job of
focusing its efforts and working more efficiently.  Some interviewees suggested that
program staff were trying to work on too many disparate issues and that they needed to
remain focused on core regulatory requirements. For example, one regional staff person
said, "We have suffered from mission creep ... HQ requires Regions to help enforce
ocean dumping [requirements] but also help with the policy for the oceans27 ...  But we
only get FTEs for the dumping portion.  This [mismatch] in resource allocation and
duties presents confusion in the program."
Some interviewees suggested that the program is operating in a reactive mode because it
is so under-resourced that it does not have the systems and procedures in place to
anticipate and manage workloads. One person said, "We're just trying to hold the key
things together and deal with administrative [requirements], political directives,
emergencies, and prepare for international meetings [related to the London
Protocol]... .We respond to emergencies well, but what happens is a lot of other stuff
languishes... .It would be great just to finish the guidance on the testing manual, but
we've made slow progress. We don't have the resources to get over the hump."
Interviewees also noted that lack  of resources and attention to documenting procedures
and ensuring consistency between Regions (e.g., establishing consistent site monitoring
frequencies) leads to inefficiencies in work. The lack of effective manuals and guidance
leads to increased time and resources needed to bring new staff up to speed when they
replace experienced staff.  This has been a particular challenge in Regions that have
historically experienced management challenges and/or significant staff turnover. For
example, one Region is working to create electronic filing systems, standard operating
procedures, updated research, and a new understanding with USAGE about what is
required under the regulations.  This renewed effort has required an investment of
resources not only within the Region but also at EPA HQ.

OPPORTUNITIES TO IMPROVE ALIGNMENT OF RESOURCES ACROSS REGIONS
Several interviewees suggested changes to how resources are allocated across Regions,
though there was no clear consensus on how to improve this allocation.  Two
interviewees said that resources should not be allocated equally between the Regions, but
rather should be based on the number of ocean dumping sites within the Regions.
However, another interviewee cautioned that this sort of pro-rated allocation had been
tried in the past, and that it had led to some Regions that received relatively more funds
redirecting funds away from ocean dumping to other activities.
A few interviewees suggested that there is a minimum level of resources required for
monitoring, and that each Region should be allocated sufficient resources to conduct
required monitoring. One interviewee (in the east) said, "What we've had to do is bare
bones monitoring.  We have adapted how we are monitoring sites by reducing the
number of samples and replicates, and instead of doing everything the regulations say, we
27 The interviewee most likely was referring to the National Ocean Policy described at
 http://www.whitehouse.gov/administration/eop/oceans/policy
                                                                               5-2

-------
have pulled back to the absolute minimum where we could still get useful information.
At some point, data is so minimal it is not useful."  On the other hand, another
interviewee said, "the east coast has had the [Bold] vessel available and has had an
incentive to keep it busy, so they've done extra work, while we can't even do the basics
on the west coast. There needs to be a nationally recognized minimum monitoring
policy." Interviewees point out the importance of monitoring to protect human health
and the environment. For example, one regional representative noted that monitoring is
important to understand how activities during disposal affect marine environment.
Another regional representative said,  "Site monitoring is a feedback mechanism.
[Monitoring allows us to] look for trends to see whether testing and management is
adequate; without the monitoring results, we wouldn't be able to make changes in how
materials are placed [in the ocean to protect the marine environment.]" Another
interviewee pointed out that ensuring adequate monitoring frequency is necessary to
ensure that EPA is meeting its regulatory obligations and that the Agency is not
vulnerable from a legal perspective. Until the minimum requirements for monitoring are
agreed on and Regions determine how they will conduct monitoring without the Bold
vessel, it remains unclear what are the minimum dollar resources Regions will need to
contract out adequate monitoring.
With regard to allocation of staff across Regions, the current allocation is based on the
historical number of ocean dumping sites in each Region. One  interviewee argued this
no longer makes sense. However, this interviewee said "it doesn't matter because
Regions allocate staff the way they want to anyway... .and HQ does the same thing.... a
lot of staff who have nothing to do with ocean dumping are officially 'Ocean Dumping
FTEs.'" One interviewee said that each Region should have a minimum of two staff
persons, to allow for backup/redundancy in case  one person leaves. In a group discussion
among program staff, participants suggested that the Program assess the FTE allocation
across regional offices and revisit the workload model from the 1990s.

OPPORTUNITIES, RISKS, AND IMPACTS OF RESOURCE REALLOCATION
As noted earlier, the Ocean Dumping program has recently undergone a major resource
reallocation, namely disinvestment from the ocean monitoring vessel, the Bold. While
several Regions lamented the loss of the vessel, others felt it would lead to an appropriate
reallocation of resources. In addition, interviewees pointed out that the act of
disinvestment itself has taken resources and has been a considerable distraction for the
program in recent years.  Some interviewees expressed concern that disinvestment from
the Bold will actually end up costing EPA more resources in the long term.
CONCLUSIONS
The findings suggest that the Ocean Dumping Management Program does have potential
to improve alignment of its resources with intended outcomes.  We conclude that the
program managers could help align resources by confirming the program's priorities,
                                                                              5-3

-------
articulating the minimum resources needed to meet those priorities at both the HQ and
regional level, and requesting support from senior leadership to focus the program's
limited resources on those priorities. Findings from interviews suggest that the HQ
program is vitally important to coordinate, guide, and ensure consistency across Regions.
In addition, HQ is essential to representing the Agency in cross-agency efforts, e.g., the
interagency workgroup on the London Convention. In our opinion, these areas of activity
should be the primary focus of the Ocean Dumping Management Program.
Whenever resources are shifted, there are likely to be "winners" and "losers," and it is
important to be sure that the change provides enough benefits to be worth the friction and
distraction of implementing the change. Program staff and managers have not developed
a shared view of the minimum requirements that HQ and the Regions must fulfill, and the
resources needed to complete those tasks. Without such an agreement, it is difficult to
pinpoint opportune areas for resource reallocation.
It is not clear that the program has adequate resources to address its core functions, take
on new emerging issues, and cope with the transaction costs associated with budget
restructuring.  To operate with existing resources, the program may need to prioritize its
activities with a focus on core program functions (e.g., providing guidance to the
Regions), while devoting  less attention to EPA initiatives that are important but not
directly related to the program's mission (e.g., administrative work associated with
issuing monitoring grants in lieu of the Bold, and coordinating with other agencies on
carbon capture and storage). On the other hand, in order to address emerging issues (e.g.,
related to climate change) the program may need more resources.
RECOMMENDATIONS
The evaluation team recommends that the Ocean Dumping Management Program take
the following steps:
    •  Assess resource allocations for Regions going forward: Define the minimum
       requirements that the Regions must fulfill. Research what future program
       activity levels (e.g., permit review, site designation activities) can be anticipated
       in each Region, given trends in ocean dumping and port development.  Analyze
       what resources will be required in each Region to meet minimum program
       requirements given anticipated activity levels. Finally, consider resource
       reallocation across Regions if necessary to ensure a balance between effort
       required and resources provided.
    •  Assess minimum resource requirements for the HQ program: given work on
       the London Protocol and the need to update the regulations and provide
       additional guidance to Regions,  determine what resources will be needed to
       adequately carry out core program activities.
                                                                               5-4

-------
CHAPTER 6  |   PERFORMANCE MEASURES
Performance measures provide essential feedback to assess program performance and to
inform ongoing program learning and evaluation. There are a number of factors which
should inform the design of performance measures for any program. These factors
include, for example, the goals of the program, the aspects of the program over which
program managers exert some control or influence, and the availability of reliable and
meaningful data.  Programs may develop a
suite of performance measures that include
both output and outcome measures intended
to provide a comprehensive picture of
program results.  Not all aspects of a
program can be measured; the best measures
provide an indicator of program
effectiveness (See Exhibit 6-1 for additional
characteristics of effective program
measures.)
The Ocean Dumping Management Program
seeks to measure its success in a complex
environment. First, the program is trying to
achieve goals that have been articulated in
several different ways and the definition of
the goals is currently evolving.
Traditionally, the program has articulated its
goals in terms of implementing the MPRSA
and the London Convention. The FY13
Congressional Justification describes the
program's major areas of effort as
"Designating, monitoring, and managing
ocean dumping sites and implementing
provisions of the National Dredging Policy."  As part of the draft "EPA Operational
Priorities for FY13 Ocean and Coastal Protection" the Ocean Dumping Management
Program is helping define an operational priority for ocean dumping, which addresses
monitoring of ocean disposal sites to ensure protection of environmental health,
appropriate testing of material to be disposed, and encouraging beneficial reuse of
materials. The program is also considering a process of engaging the users of ocean
disposal sites in financing site designation and management costs.
 EXHIBIT 6-1.  CHARACTERISTICS OF
 EFFECTIVE  PERFORMANCE MEASURES
Data are:
   •  Available
   •  Measurable
   •  Verifiable
Measures are:
   •  Timely and actionable
     Responsive to the needs of
     stakeholders (e.g., staff, managers,
     public, flinders)
     Comprehensible and credible to
     stakeholders
     Motivating
     Comparable across jurisdictions
     and programs, and over time
                                                                            6-1

-------
Program staff say they are working to minimize the amount of substances placed into the
ocean and to ensure that any matter that is dumped is tested appropriately, meets
applicable standards, and is managed and monitored appropriately. However, the
program has no direct influence over the amount of material that is dumped; this is
largely dependent on dredging projects overseen by USAGE and influenced by economic
conditions and priorities, as well as the amount of sedimentation in navigation channels.
While the Ocean Dumping Management Program seeks to encourage beneficial reuse
where possible, this approach is often logistically or economically infeasible.28 In the
remainder of this chapter we present our findings regarding the current performance
measures in use, strengths and weaknesses of the current measures, and potential alternate
measures, along with our conclusions and recommendations.
FINDINGS
CURRENT MEASURES
Overall, the Ocean Dumping program focuses on measuring outcomes, namely the
percentage of Ocean Dumping sites which meet specified conditions. Specifically, the
Ocean Dumping program has a strategic measure which states that, "By 2015, 95 percent
of active dredged material Ocean Dumping sites,  as determined by 3-year average, will
have achieved environmentally acceptable conditions (as reflected in each site's
management plan and measured through onsite monitoring programs)."29 (See Exhibit 6-2
for a description of performance on this measure). In addition, the Ocean Dumping
program tracks the "number of active dredged material Ocean Dumping sites that are
monitored in the reporting year."30 The FY 13 Congressional Justification states that
"During FY 2013, the EPA will collect environmental data from several offshore areas
for use in the designation of... dredged material disposal sites and .. .monitor, as required,
the 67 active dredged material ocean disposal sites." In addition to documented measures,
program staff shared that they track additional measures and are working on developing
new measures.  For example, one interviewee described an output measure under
development that would track the number of projects and suitability determinations
processed each year. Another interviewee said the program tracks the timeframes of how
often site monitoring is conducted and when site management plans are updated, to track
whether the site management plans are reviewed and revised every 10 years as required.
28 EPA Operational Priorities for FY13 Ocean And Coastal Protection, Draft August 27, 2012

29 Source FY 2011 - 2015 EPA Strategic Plan, http://www.epa.gov/planandbudget/strategicplan.html. The universe of active
 dredged material ocean disposal sites was 60 in 2005, 65 in 2009, and is currently 67 sites.

30 EPA Office of Water, Fiscal Year 2013 National Water Program Guidance, Appendix A Measures Summary, April 2012.
                                                                                6-2

-------
EXHIBIT 6-2.  PERFORAMANCE ON OCEAN DUMPING STRATEGIC PLAN MEASURE


MEASURE

Target
Actual*
Percent of Active Dredged Material Ocean Dumping Sites
that Will Have Achieved Environmentally Acceptable
Conditions (as Reflected in Each Site's Management Plan).
FY08
95
99
FY09
98
99
FY10
98
90.1
FY11
98
93
FY12
95

FY13
95



UNITS

Percent
sites
                 *Due to variability in the universe of sites, results vary from year to year (e.g., between 85 percent
                 and 99 percent). While this much variability is not expected every year, the results are expected to
                 have some change each year.)

                 Explanation of Results: Gulfport Western Ocean Dredged Material Disposal Sites (ODMDS) has exceeded
                 the minimum depth limitation. The Miami ODMDS has elevated PCB levels. In addition, multiple Gulf of
                 Mexico ODMDS likely do not meet environmentally acceptable conditions due to the Deep Water
                 Horizon Oil Spill and need to be evaluated.

                 Source: FY 13 Congressional Justification, page 853-854
               STRENGTHS AND WEAKNESSES OF CURRENT MEASURES
               Interviews with Ocean Dumping Management Program staff suggest that while the
               current strategic plan measure is understood in theory, different Regions have different
               definitions of what constitutes "environmentally acceptable conditions," and thus the
               Regions do not use the measure in a consistent way.  One interviewee stated that program
               staff have discussed what the term "environmentally acceptable conditions" means, but
               have reached no consensus.  Another interviewee said that by definition, only sites that
               had achieved environmental acceptable conditions would be approved, and therefore by
               definition the measure should always be 100 percent. Another regional interviewee said
               that one state in the Region would not sign off on some site management plans (in an
               effort to make  sure all dredge material was used  for beneficial reuse) without the state's
               agreement the  Region could not say the sites had achieved environmentally acceptable
               conditions.  These incompatible perspectives mean that the values reported for the
               strategic plan measure are not consistent or meaningful.
               A review of performance on the strategic plan measure by Region shows that all but two
               Regions rate themselves as having 100 percent of their active dredged material ocean
               dumping sites that achieve environmentally acceptable conditions.31  For the two Regions
               that miss the target, one interviewee notes that one Region understands the issues at their
               sites (e.g.,  elevated PCB levels), and the other Region believes that environmental
               conditions at the ocean dumping sites are not adequately characterized.  While the current
               measure does flag potential problems in these two Regions, it does not distinguish
               31 EPA Office of Water, Fiscal Year 2013 National Water Program Guidance, Appendix E Detailed Measures Appendix, April
                2012, page 12.
                                                                                                  6-3

-------
between the two different types of problems these Regions face. More importantly, it
suggests that performance has been maximized in all the other Regions, which interviews
suggest is not the case. Overall, definitions underlying the performance measures reflect
different Regions' conceptions of the measure more than variations in performance.
In addition to concerns about inconsistency in interpreting the strategic plan measure,
interviewees express concern that the measure does not capture the quality of the work
the program does, or the broad range of program activities conducted. Overall, it does
not appear that the current measure provides feedback for staff about the effectiveness of
their efforts, because staff feel they have  little control over the measure.

POTENTIAL ALTERNATIVE MEASURES
Interviewees and focus group participants suggested a number of options for developing
alternative performance measures, and both HQ and the Regions informally track some
data in addition to the Strategic Plan Measure. Possible alternate measures for the
program include outcome and output/efficiency measures, as described below.

Outcome Measures
Effective performance measures should track progress toward a goal (in addition to
meeting the criteria described earlier).  For the Ocean Dumping Management Program,
goals include  site designation and monitoring to ensure that substances dumped in the
ocean do not have a negative impact on human health or the environment.  Another
objective is to increase the beneficial use of dredge material.  Potential measures related
to these goals include:
     • Volume of material dumped at ocean dumping sites. Since one overall goal
       of the program is to minimize ocean dumping, this measure would indicate the
       environmental benefit of the program, if the volume of matter disposed of
       through ocean dumping declines overtime.
     • Volume and percentage of contaminated material proposed for ocean
       disposal that is not allowed to be ocean dumped (e.g., sent for land disposal
       instead).  This measure speaks directly to the environmental benefit of the
       program, since it would show the extent to which the program prevents the
       dumping of contaminated matter.
     • Percentage of monitored sites found to be in compliance. This measure would
       document whether sites are in compliance.
     • Percentage of projects where material is diverted to beneficial reuse rather
       than  ocean dumping.  This measure would show whether the program is
       making progress in encouraging  beneficial reuse for uncontaminated dredging
       material, which is a goal of the program.
 All of the outcome measures described here may be strongly influenced by factors
 outside the program's control.  Thus while these measures are important to
 understanding progress toward the program's goals, they must be interpreted carefully,
 and changes  in an outcome measure should not be assumed to be cause by the Ocean
                                                                             6-4

-------
 Dumping Management Program.  Evaluation is a useful tool that can assess contribution
 of a program toward observed outcomes.32

Output/Efficiency Measures
In addition to outcome measures, the Ocean Dumping Management Program may wish to
also include output or efficiency measures, which it can more directly control. Several
options are listed below.
      • Average number (and range) of years since site monitoring has occurred.
       This measure would document whether the program is meeting the expectation of
       monitoring all sites within the past 10 years, which is a key program activity.
       Note that some sites need to be monitored more frequently than every 10 years.
       In order for the measure to be comparable and meaningful, it would be necessary
       to define what constitutes adequate monitoring.
      • Average number (and range) of years since Site Management and
       Monitoring Plans (SMMPs) that have been reviewed and updated. This
       measure would document whether the program is keeping site management plans
       up to date and meeting the requirements that SMMPs are reviewed and revised at
       least every 10 years.
      • Average time to complete specific EPA activities (e.g., leading to site
       designation, suitability determinations, or permit reviews). This type of
       measure would track program efficiency, and managers would seek to reduce the
       measure over time to demonstrate improved efficiency. This type of measure
       would need to be balanced with an adequate review of quality of EPA's work, in
       order to ensure there is no perverse incentive to rush the work.
      • Count of EPA activities (e.g., inquiries EPA responds to, permits reviewed,
       etc.). This type of measure would not indicate program performance, but rather
       would provide context for other measures by indicating the extent to which there
       are ocean dumping activities that are potentially subject to  EPA oversight and the
       volume of work EPA is completing.
CONCLUSIONS
Overall, the current measure is not consistently interpreted by the Regions and does not
provide particularly useful feedback for understanding program effectiveness. A valid
performance measure is one that accurately represents the condition or phenomenon that
it is purporting to represent. The current measure would be valid if the key term
"environmentally acceptable conditions" were consistently defined.
32 In some cases, programs can attribute outcomes to a program, but this requires a higher standard of evidence, and is
 difficult to do in the context of government program evaluations.
                                                                              6-5

-------
The Ocean Dumping Management Program could improve information flows by
developing a suite of performance measures that reflect both outcomes related to the
programs goals and outputs over which the program has direct control.
RECOMMENDATIONS
The evaluation team recommends that the Ocean Dumping Management Program:
   •   Strengthen information flows to inform program adaptation, building on a suite
       of performance measures that encompass both outcome and output measures; and
   •   Consider the following measures: volume of contaminated material not allowed
       to be ocean dumped (and of this amount, percentage that is  used for beneficial
       uses); percentage of monitored  sites found to be in compliance; average number
       of years since ocean disposal sites were monitored; and average number of years
       since Site Management and Monitoring Plans have been reviewed and updated.
                                                                           6-6

-------