U.S. ENVIRONMENTAL PROTECTION AGENCY
        OFFICE OF INSPECTOR GENERAL
                        Catalyst for Impro ving the En vironment
Evaluation Report
       Regional Public Liaison Program
       Needs Greater Focus on Results
       and Customer Awareness
       Report No. 09-P-0176

       June 24, 2009

-------
Report Contributors:            Bram Hass
                               Larry Dare
                               Eric Lewis
Abbreviations

EPA         U.S. Environmental Protection Agency
FTE         Full-Time Equivalent
OIG         Office of Inspector General
OSWER      Office of Solid Waste and Emergency Response
RPL         Regional Public Liaison (formerly Regional Superfund Ombudsman)
RSO         Regional Superfund Ombudsman

-------
                  U.S. Environmental Protection Agency
                  Office of Inspector General

                  At   a  Glance
                                                           09-P-0176
                                                        June 24, 2009
                                                               Catalyst for Improving the Environment
Why We Did This Review

The Office of Inspector
General (OIG) evaluated the
actions taken by the U.S.
Environmental Protection
Agency's (EPA's) Office of
Solid Waste and Emergency
Response (OSWER) to
implement its Regional Public
Liaison (RPL) program.
We also reviewed program
changes resulting from a 2003
OIG report.
Background
Responding to the 2003 OIG
report, EPA in 2004 revised its
Superfund Regional
I Ombudsman program,
renamed staff RPLs, and
established national program
(guidance.  The RPLs help the
public and regulated
community by (1) providing
information and facilitating
informal contact with EPA
staff, (2) helping resolve
problems, and (3) making
recommendations for
improvement to Agency
senior management.
For further information,
contact our Office of
Congressional, Public Affairs
and Management at
(202)566-2391.

To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2009/
20090624-09-P-0176.pdf
Regional Public Liaison Program Needs Greater
Focus on Results and Customer Awareness
 What We Found
OSWER's RPL program does not sufficiently focus on or measure specific
outputs and outcomes and is not consistently implemented across offices.  RPLs
report results in varied formats, and OSWER does not consolidate program results
into a comprehensive report. As a result, RPLs reported annual results that could
not be readily consolidated to show what RPLs had achieved. The program does
not sufficiently focus on results because it lacks clear program logic. Developing
a logic model would help define outputs and outcomes and encourage a results-
oriented approach to program implementation. A more consistent and
comprehensive approach would also raise stakeholder awareness of the RPL
program.

Despite limited resources, RPLs have assisted many stakeholders since the
program was revised in 2004. New guidance more clearly described program
expectations and sought to provide more consistent program implementation
across regional offices.
 What We Recommend
We recommend that OSWER use a logic model approach to revise the RPL
program to help focus on outputs and outcomes and ensure stakeholders are aware
of the RPL resource.  Also, OSWER should revise the 2004 RPL guidance to
reflect program revisions and build in minimum requirements for stakeholder
awareness activities, including a national RPL Website.

OSWER concurred and described planned corrective actions to address all of our
recommendations. OSWER stated it has requested assistance from OSWER's
Center for Program Analysis to conduct a logic model assessment starting during
the RPLs' National Meeting, June 16-18, 2009. OSWER's response explains that
this assessment will lead to revised guidance that will include minimum
requirements for stakeholder awareness activities.

-------
                    UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
                                   WASHINGTON, D.C. 20460
                                                                             OFFICE OF
                                                                         INSPECTOR GENERAL
                                      June 24, 2009
MEMORANDUM
SUBJECT:
FROM:
TO:
Regional Public Liaison Program Needs Greater Focus
on Results and Customer Awareness
Report No. 09-P-0 176
Wade T. Najjum
Assistant Inspector General
Office of Program Evaluation
Mathy Stanislaus
Assistant Administrator
Office of Solid Waste and Emergency Response
This is our final report on the subject review conducted by the Office of Inspector General (OIG) of
the U.S. Environmental Protection Agency (EPA). This report contains findings that describe
concerns the OIG has identified and corrective actions the OIG recommends. This report represents
the opinion of the OIG and does not necessarily represent the final EPA position. Final
determinations on matters in this report will be made by EPA managers in accordance with
established resolution procedures.

The estimated cost of this report - calculated by multiplying the project's staff days by the applicable
daily full cost billing rates in effect at the time - is $355,483.

Action Required

We believe the written response you provided to the draft report meets the requirements of EPA
Manual 2750 for a written response within 90 days of the date of this report. We request that you
inform us as you complete each corrective action so we may update our tracking system. We have
no objections to the further release of this report to the public. This report will be available at
http ://www. epa. gov/oig.

If you or your staff have any questions regarding this report, please contact Eric Lewis at
202-566-2664 or lewis.eric@epa.gov, or Larry Dare at 202-566-2138 or dare.larry@epa.gov.

-------
Regional Public Liaison Program Needs                                    09-P-0176
Greater Focus on Results and Customer Awareness
                     Table of Contents
Chapters
   1   Introduction 	    1

           Purpose 	    1
           Background  	    1
           Noteworthy Achievements	    2
           Scope and Methodology	    3

   2   Focusing on Results Would Provide Greater Program
       Consistency and Facilitate Measuring Results	    4

           RPL Program Needs Greater Focus on Outputs and Outcomes  	    4
           OSWER Should Analyze Program Logic to Focus on Results	    6
           Regional Resources Devoted to RPL Program Varied Greatly 	    6
           Performance Agreements Should Include RPL Role 	    7
           Consistent Implementation Would Improve Stakeholder Awareness	    7
           Conclusions	    9
           Recommendations 	    9
           Agency Response and OIG Comment	   10


   Status of Recommendations and Potential Monetary Benefits	   11
Appendices
   A   Agency Response to Draft Report	   12

   B   Distribution 	   15

-------
                                                                           09-P-0176
                                Chapter  1
                                Introduction
Purpose
             The Office of Inspector General (OIG) reviewed the U.S. Environmental
             Protection Agency's (EPA's) Regional Public Liaison (RPL) program within the
             Office of Solid Waste and Emergency Response (OSWER) to answer the
             following questions:

                •   Is the RPL program being implemented consistently across offices?

                •   How does EPA measure the results (outputs and outcomes) of this
                   program, including its impact on the involved communities and the
                   environmental problems they face?

             We also looked at whether the program implemented recommendations made in
             our March 13, 2003, report, EPA Regional Super fund Ombudsmen Program
             Needs Structure (2003-S-00004).
Background
             EPA created Regional Superfund Ombudsman (RSO) positions in each regional
             office as part of the Superfund Administrative Reforms effort in 1996.  The RSOs
             facilitate resolution of stakeholder concerns and provide a forum so States and
             communities can be more informed and involved in clean-up decisions. Prior to
             this reform, stakeholders raised concerns with regional personnel but had no
             formal mechanism to have their issues elevated. When EPA appointed the 10
             RSOs in June 1996, it issued a press release saying:

                   The establishment of an ombudsman position in each EPA region
                   will help resolve Superfund issues that fall through the cracks in
                   the current system. Through these ombudsmen, who will work
                   closely with the EPA headquarters ombudsman, the public will
                   have  access to an EPA employee who will be able to cut  through
                   red tape to investigate complaints and arrange meetings with
                   appropriate staff to try to resolve problems. The ombudsmen will
                   have  the authority to cut across bureaucratic lines to get answers
                   and resolve problems quickly.

             In our March 2003 report, we noted that EPA does not have a management
             system in place to ensure its RSOs are accountable for fulfilling their
             responsibilities. The RSO program was generally a collateral duty within the

-------
                                                                       09-P-0176
       Superfund program. As a result, there was a perceived lack of independence and
       impartiality. Further, a lack of guidance had caused uncertainty over the RSO
       program. The report pointed out that changing the title of the RSOs would allow
       them to continue to provide a valuable service by informally resolving issues at a
       local level, and would alleviate the perception that they should meet the American
       Bar Association's core characteristics of impartiality and independence.  Also, to
       ensure consistent performance and results, we recommended that OSWER
       provide guidance describing the roles and responsibilities of the position.

       Responding to our report recommendations, OSWER issued guidance in March
       2004, OSWER Guidance for the Regional Public Liaisons (OSWER 9200.0-48).
       The guidance changed the "Regional Superfund Ombudsman" title to "Regional
       Public Liaison,"  and allowed regions, at their discretion, to broaden the former
       RSOs' scope beyond Superfund to include programs that OSWER administers
       under nine other Acts. It described the purpose of the RPLs as "... to provide
       information and assistance to the public in resolving issues and concerns raised
       about the programs administered by OSWER." The guidance explains that the
       RPL may be called on to serve in a number of capacities:

          •  Provide information and facilitate  informal contact with EPA staff.
          •  Help resolve problems.
          •  Make recommendations to Agency senior management aimed at
             improving OSWER programs.

       The RPL program provides stakeholders the opportunity to be involved in
       OSWER programs and  serves as an important internal control to reduce the risk
       that stakeholders' concerns will go unanswered.  In issuing the guidance,
       OSWER's Deputy Assistant Administrator's cover memo stated that".. .the RPLs
       have been a useful resource for people outside the Agency who have needed
       assistance or had concerns about the program. They have helped numerous
       individuals and defused many potentially  difficult situations." He pointed out that
       the purpose of the guidance was to strengthen the effectiveness of the RPLs by
       establishing a clear and consistent set of operating principles and procedures. He
       viewed the guidance as a sound framework for RPL activities that still allowed
       each region to implement a program that was consistent with its own needs and
       organization.

Noteworthy Achievements

       RPLs have assisted many stakeholders since the program was revised in  2004.
       New guidance more clearly described program expectations and sought to provide
       more consistent program implementation  across regional offices.  The name of the
       program was changed from Ombudsman to Regional Public Liaison to more
       accurately reflect the nature of the program.

-------
                                                                       09-P-0176
Scope and Methodology

      We reviewed OSWER's RPL program from January 2008 through May 2009 in
      accordance with generally accepted government auditing standards. Those
      standards require that we plan and perform the evaluation to obtain sufficient,
      appropriate evidence to provide a reasonable basis for our findings and
      conclusions based on our evaluation objectives. We believe that the evidence
      obtained provides a reasonable basis for our findings and conclusions based on
      our evaluation objectives. We limited our review of management controls and
      compliance to those directly relating to the issues identified.  Office of
      Management and Budget Circular A-123 notes that internal controls are a means
      of managing the risk associated with federal programs and operations.

      We analyzed the current RPL guidance  and identified documentation and
      guidance from predecessor programs. We obtained annual reports, resource
      estimates, job descriptions, performance agreements, meeting minutes, and other
      information about how RPLs carried out their duties. We conducted searches of
      EPA's national and regional Websites to determine whether stakeholders could
      readily find the RPLs, whether RPL names were up to date, and whether RPLs
      were listed on Superfund site descriptions. We communicated with eight current
      and five former RPLs. We also visited Region 2 to  obtain an understanding of
      how an extensive RPL effort operated.

      As previously noted, we reviewed a March 2003 OIG report to determine whether
      our recommendations  in that prior report had been implemented. We also
      reviewed Government Accountability Office reports and testimony.

-------
                                                                         09-P-0176
                               Chapter 2

 Focusing  on Results Would Provide  Greater Program

      Consistency  and Facilitate Measuring  Results

            OSWER's RPL program does not sufficiently focus on or measure specific
            outputs and outcomes and is not consistently implemented across offices.
            Although RPLs are dedicated to helping their customers, the current design and
            implementation limit the program's potential to effectively serve the public and
            measure results. RPL guidance describes three primary roles: helping members
            of the public gain access to information, helping resolve problems, and initiating
            positive institutional change to prevent similar problems in the future.  However,
            because regional offices implement the program differently, offices report results
            that are not comparable and do not provide a clear picture of the program's
            overall accomplishments. A more consistent and comprehensive approach would
            also raise stakeholder awareness of the RPL program.

RPL Program Needs Greater Focus on Outputs and Outcomes

            The 2004 RPL guidance does not define outputs or outcomes for RPLs to track as
            they carry out their responsibilities. As a result, RPLs reported annual results that
            could not be readily consolidated to show what RPLs had achieved. Without a
            clear link between program objectives and expected outputs and outcomes, the
            program will have little chance  of achieving its intended results.

            EPA's Office of Policy, Economics, and Innovation defines outputs and outcomes
            as follows:

               •   Outputs: A measure of products and services provided as a direct result
                   of program activities.
               •   Outcomes:  Accomplishments of program goals and objectives.

            The RPL guidance states that each RPL should provide an annual report to his/her
            Regional Administrator with a copy to the Assistant Administrator for OSWER
            describing numbers and types of issues that the RPL addressed. It goes on to note
            that the annual report could be used to make process and organizational
            recommendations and address publicly issues of concern, but does  not require a
            consolidated report.

            Based on this requirement, RPLs should be reporting outputs ("numbers.. .of
            issues and concerns... addressed"). Categorizing them by "types of issues and
            concerns" is left up to the RPLs. Because each RPL categorizes reporting outputs
            differently and there was no shared format, RPL reports we reviewed were not
            consistent.  For example:

-------
                                                                               09-P-0176
                 •  One RPL provided several breakouts, including information requests
                    versus assistance requests, and categorization by EPA program area as
                    well as the source of the request.
                 •  One RPL showed number of contacts received, number that were site-
                    related, and number referred.
                 •  One RPL with only a few cases described them and noted whether they
                    were open or closed at the end of the fiscal year.
                 •  One RPL reported no major cases but referred 15-20 calls to other
                    divisions.

             None of the RPLs summarized activities by the three roles of the RPL (providing
             information, cutting red tape, and initiating institutional  change). Unless the
             RPLs use comparable categories when tracking and reporting on inquiries and
             cases, it will be difficult to summarize overall RPL program results.

             While the reporting requirement does not require RPLs to track outcomes, we
             believe it would be beneficial. We found two outcome categories in sections
             where the guidance discussed  other subjects:  (1) "resolving problems," and
             (2) "members of the public.. .gain access to information.. .that will help them
             participate more fully in established Agency processes."  Program outcomes also
             could be described in terms of the RPLs' three key roles. Outputs and outcomes
             should be integrated into an overall program design for tracking and reporting.

             Although EPA reported it had measures for the earlier RSO program, one RPL
             told us that no common output or outcome measures were ever provided to the
             regions.  None are currently being used. These measures are listed in the
             Government Accountability Office's May 2000 report on Superfund
             Administrative Reforms.1 One measure was the number of cases for which EPA
             conducted investigations and mediations. Another was whether the public's
             perceptions of EPA's decisions improved.

             To capture  and assess outputs  and outcomes, RPLs must collect and report
             comparable data regarding program accomplishments. They instead use different
             methods to track cases and collect different data about program accomplishments.
             For example, one RPL uses a spreadsheet to log cases while others use less formal
             methods. Annual reports also capture results differently; some RPLs simply
             summarize the results of each  case they handled during the year, while others
             produce relatively detailed reports.  Further, not all RPLs sent annual reports each
             year. Without comparable data and reporting, the RPL program cannot conduct the
             kind of analysis necessary to evaluate results,  identify areas for improvement in the
             RPL program, or encourage institutional reform in OSWER-related programs.
1 SUPERFUND: Extent to Which Most Reforms Have Improved the Program Is Unknown, May 2000, GAO/RCED-
00-118.

-------
                                                                            09-P-0176
             The RPL program coordinator does not produce a consolidated annual report that
             would show the RPL program's accomplishments. National guidance requires each
             RPL to submit a report annually but does not require a consolidated report.
             Nonetheless, we believe a national report would be beneficial.

OSWER Should Analyze Program Logic to Focus on Results

             Given its limited resources, we believe the RPL program would benefit by using a
             logic model. We analyzed OSWER's 2004 guidance for RPLs to determine if a
             logic model could be constructed from its description of the program.  The guidance
             did not provide all the elements needed for a usable logic model.  If the RPL
             program design was based on a logic model, then it would focus more specifically
             on defining outputs and outcomes and thus more clearly on program results.

             EPA training materials explain that a logic model is a diagram and text that
             describes/illustrates the logical (causal) relationships among program elements
             and the problem to be solved, thus defining measurements of success.  Logic
             models help managers answer key questions and understand and check
             assumptions on how the program is supposed to work.  These models can support
             program improvement and evaluation, helping to answer such questions as:

                •  What am I doing, with whom, and to whom/what?
                •  How well am I doing it?
                •  Is anybody (anything) better off in either the short or long term?
                •  What role, if any, did my program play in the results?
                •  What role, if any, did factors unrelated to the program play in the results?
                •  Were there any unintended outcomes, and if so, why?

             EPA and many other organizations use logic models to better define the
             relationship of resources; customers; outputs; and short-, medium-, and long-term
             results (or outcomes). This technique enables one to determine whether the
             program is set up to  succeed and facilitates program evaluation.  The EPA Office of
             Policy, Economics, and Innovation's logic model training stresses that meaningful
             measuring  of program performance requires a clear understanding of what a
             program does and the results it is intended to accomplish. EPA and the Office of
             Management and Budget endorse and make use of the logic model approach.

Regional Resources Devoted to  RPL Program Varied Greatly

             Resources used by the various regions for the RPL program varied greatly, and
             one benefit of using  a logic model approach is that it could help OSWER and the
             regional offices assess the adequacy of resources.

             According  to estimates provided by the RPLs and the program coordinator, the
             entire RPL program  used about 2.5 FTEs in 2008.  Implementing the RPL

-------
                                                                            09-P-0176
             program is a collateral duty for most RPLs.  One RPL (Region 2) estimated his
             region's effort at 1.9 FTEs. He and two assistants handle several hundred
             inquiries per year among other duties. Their inquiries come from the region's
             main Web inquiry system, a Superfund hotline, direct contacts from stakeholders,
             and referrals from EPA staff.  Another RPL reported spending about 35 percent of
             his time on RPL matters. RPL estimates for other regions ranged from 1  percent
             to 10 percent of one FTE. We believe that spending such a small amount of time
             may not allow RPLs to carry out the full extent of their duties. Further, two
             regional offices do not have anyone designated as RPL.  Several RPLs and other
             regional staff attributed the limited RPL resources devoted to cases in their
             regions to excellent community relations.

Performance Agreements Should Include RPL Role

             The 2004 RPL guidance states that RPL duties should be included in the
             individual's annual performance agreement:

                    ... The RPL is accountable for the successful performance of
                   his/her official duties, as defined in his/her position description
                   and more specifically delineated in his/her annual performance
                   agreement.

             We found that two of the eight current RPL performance agreements did  not
             mention their RPL roles.  Four RPLs had general statements related to their RPL
             duties.  Two had more specific wording related to their RPL role. None referred to
             the 2004 OSWER guidance. We found no common wording  or description of
             duties among those we reviewed.  A common understanding and written description
             of RPL duties in performance agreements would help ensure that the regions
             implemented the RPL program more consistently and achieved program outputs
             and outcomes.

Consistent Implementation Would Improve Stakeholder Awareness

             A more consistent and comprehensive approach would raise stakeholder
             awareness of the RPL program. EPA should ensure that stakeholders who may
             need RPLs are aware of their existence and can reach them easily.  Currently, the
             guidance does not address awareness efforts. Based on information provided by
             RPLs and our checks of regional Websites, we found that EPA's efforts to inform
             stakeholders about the RPL program are inconsistent. For example, 4 of  10
             regional Websites did not mention the program.  Stakeholders may learn  of the
             RPL program from the RPL directly (at public meetings, for example), from other
             EPA staff, or from EPA documents and Internet sites.

-------
                                                                  09-P-0176
RPL Guidance Does Not Address Stakeholder Awareness

The 2004 RPL guidance does not advise RPLs on ensuring that stakeholders
know the RPLs exist. The guidance states that "If community members believe
that the official Agency procedures have failed to adequately address their
concerns, they may ask the RPL to help them get information or assistance from
within the system." It does not describe how RPLs inform stakeholders outside
EPA of their availability.

When the RPLs' predecessor RSO program started in 1996, implementation plans
prepared by several regional offices described methods for publicizing the
existence of that program. For example, Region 2's 1996 Implementation Plan
stated that".. .The degree to which the public is made aware of the ... Program
and its availability and accessibility will determine the effectiveness  of the
Program.  Therefore, it is imperative that EPA's stakeholders become aware of
the program's availability, capability and limitations."  The plan listed various
activities including creating a toll free telephone number, distributing fliers at
certain Superfund sites describing the program, and providing handout materials
to elected officials and the public. The plan also suggested coordinating with a
wide range of other regional EPA staff and State agencies; developing an RSO
Web page; and meeting with citizens, community members, and special interest
groups. Implementation plans for Regions 7 and 10 listed  similar activities.

A handbook for an even earlier ombudsman program stated that the program's
effectiveness depended to a large extent on public awareness, noting that"... Any
system for handling complaints is of no use unless the public can actually get into
the system when the need arises." It specified certain actions, such as: developing
program fliers  and brochures, providing a Federal Register notice, contacting
congressional representatives,  listing the ombudsman separately in telephone
directories, and including a description of the program in employee orientation.

Various Techniques Used by Regions to Raise Awareness

Raising awareness of the RPL program varies among regions, and some
techniques could be easily implemented. Some RPLs have taken specific steps to
ensure  stakeholders are aware  of their program.  Others appear to take  a passive
approach,  only reacting to inquiries if and when they receive them. Although
RPL staff discussed a draft early in the program, they have never produced a
national brochure.

Some RPLs told us that they have attended public meetings to explain their role
and share their contact information.  One RPL noted that he developed a brochure
and handed it out at public meetings.  He also mailed a notification about his
availability to all Potentially Responsible Parties associated with the sites in his
region and to the local jurisdictions where the sites are located. Further, he
reported joining EPA colleagues on site visits.

-------
                                                                             09-P-0176
             Greater Visibility on Websites Could Help

             Some regional Websites lack any apparent mention of the RPL. EPA's Websites
             are one key avenue for reaching people and organizations that might benefit from
             RPL assistance.  EPA Policy 2191.0 emphasizes the importance of the Internet by
             stating, "The EPA Web site is a fundamental part of every Agency program; taken
             together, these pages are the foundation of the Agency's environmental outreach
             and communication program." We found no reference to the RPL on four
             regional Websites. Also, when we checked regional Superfund site fact sheets
             listed on regional Websites, we found that only two regions included RPL contact
             information with the site.  On one site we found that the RPL was included in the
             body of the site description document; in the other, the RPL was mentioned in a
             side banner next to the site description. Because EPA considers Websites such an
             important part of its awareness and communication efforts, the RPLs should be
             prominently displayed on each Website and with site descriptions.

             The list  of RPLs on EPA's national Website was last updated in 2003 and is
             currently outdated. That list follows a description of the program's origin in the
             1996 Superfund Administrative Reforms. However, we recently found that
             EPA's community involvement Websites now include a link to a Regional Public
             Liaison  site that is under development.  Program managers and RPLs have
             discussed plans to develop an RPL Website, prepare a brochure, and add an
             article about the RPLs to the community involvement program's Web page.
             These steps should be undertaken only  after the program has defined its program
             logic and intended outputs and outcomes.
Conclusions
             The RPL program provides an important link between concerned stakeholders and
             an EPA liaison who can provide assistance. It should provide OSWER an internal
             control to reduce the risk that significant stakeholders' concerns go unaddressed.
             However, the program currently has design and operational deficiencies that hinder
             its chances of success.  These deficiencies can be addressed by defining the
             program logic and performance measures (outputs and outcomes), better use of
             performance agreements, and improving stakeholder awareness of RPLs.
Recommendations
             We recommend that the Assistant Administrator for Solid Waste and Emergency
             Response:

             2-1   Develop an RPL program logic model to:
                       •  identify outputs and outcomes,
                       •  assess resource needs, and
                       •  revise the program's design to help RPLs achieve the desired
                          outputs and outcomes.

-------
                                                                             09-P-0176
             2-2    Adopt common data reporting elements across all regional offices and
                    collect, summarize, and report comparable data.

             2-3    Report results annually in a consolidated, national report.

             2-4    Develop and use a common performance agreement element for all RPLs.
                    The element should refer to the national RPL program guidance when
                    defining the RPL role and expected outputs and outcomes.

             2-5    Make stakeholders aware of the RPL program by sharing contact
                    information through regional  staff, site descriptions, and a national
                    Website linked to each regional RPL Website.

             2-6    Revise the 2004 RPL guidance as necessary to reflect revisions in the
                    program and build in minimum requirements for stakeholder awareness
                    activities.

Agency Response and OIG Comment

             OSWER concurred and described planned corrective actions to address all of our
             recommendations.  OSWER stated is has obtained contractor support to develop a
             national RPL Website and has requested assistance from OSWER's Center for
             Program Analysis to conduct a logic  model  assessment starting during the RPLs'
             National Meeting, June 16-18, 2009.  The week after the meeting, OSWER told
             us that it started work on the logic model during the national meeting and will get
             contractor support to help complete it. OSWER's response to our draft report said
             that the assessment will lead to common data reporting elements, revised
             guidance that will include minimum  requirements for stakeholder awareness
             activities, and a common RPL performance  agreement element. After adopting
             common data  reporting elements, the RPL program will begin submitting
             consolidated national reports.  We believe these actions will address our
             recommendations and we will close out the  recommendations as those actions are
             completed.
                                          10

-------
                                                                                                         09-P-0176
                      Status of Recommendations  and
                           Potential  Monetary Benefits
                                   RECOMMENDATIONS
Rec.
No.
Page
 No.
                            Subject
                                                   Status1
                                                              Action Official
 Planned
Completion
   Date
                                                                                        POTENTIAL MONETARY
                                                                                         BENEFITS (in $OOOs)
Claimed    Agreed To
Amount     Amount
2-1      9   Develop an RPL program logic model to (1) identify
            outputs and outcomes, (2) assess resource needs,
            and (3) revise the program's design to help RPLs
            achieve the desired outputs and outcomes.

2-2      10   Adopt common data reporting elements across all
            regional offices and collect, summarize, and report
            comparable data.

2-3      10   Report results annually in a consolidated, national
            report.
2-4     10   Develop and use a common performance
            agreement element for all RPLs.  The element
            should refer to the national RPL program guidance
            when defining the RPL role and expected outputs
            and outcomes.

2-5     10   Make stakeholders aware of the RPL program by
            sharing contact information through regional staff,
            site descriptions, and a national Website linked to
            each regional RPL Website.

2-6     10   Revise the 2004 RPL guidance as necessary to
            reflect revisions in the program and build in
            minimum requirements for stakeholder awareness
            activities.
                                                     Assistant Administrator
                                                      for Solid Waste and
                                                     Emergency Response


                                                     Assistant Administrator
                                                      for Solid Waste and
                                                     Emergency Response

                                                     Assistant Administrator
                                                      for Solid Waste and
                                                     Emergency Response

                                                     Assistant Administrator
                                                      for Solid Waste and
                                                     Emergency Response
                                                     Assistant Administrator
                                                      for Solid Waste and
                                                     Emergency Response


                                                     Assistant Administrator
                                                      for Solid Waste and
                                                     Emergency Response
                                                                          09/30/2009
03/31/2010
03/31/2011
 10/01/2010
                                                                          09/30/2009
 10/01/2010
 0 = recommendation is open with agreed-to corrective actions pending
 C = recommendation is closed with all agreed-to actions completed
 U = recommendation is undecided with resolution efforts in progress
                                                        11

-------
                                                                          09-P-0176
                                                                      Appendix A

                Agency Response to Draft Report
June 10, 2009


MEMORANDUM
SUBJECT:   Response to Draft Public Liaison Report: "Regional Public Liaison Program Needs
             Greater Focus on Results and Customer Awareness" (Project No. OCPL-FY08-003)

FROM:      Barry N. Breen/s/
             Deputy Assistant Administrator

TO:         Eric Lewis, Director, Special Reviews
             Office of Program Evaluation
PURPOSE

      The purpose of this memorandum is to provide a written response to the findings of the
Draft Evaluation Report: "Regional Public Liaison Program Needs Greater Focus on Results
and Customer Awareness" (Project No. OCPL-FY08-0003). My staff has reviewed the draft
report and agrees with the conclusions presented in the draft report. Below you will find the
corrective actions we have initiated or planned for each recommendation.
RECOMMENDATIONS AND OSWER RESPONSE

Recommendation 2-1
Develop a Regional Public Liaison (RPL) program logic model to (1) identify outputs and
outcomes; (2) assess resource needs, and (3) revise the program's design to help RPLs achieve
the desired outputs and outcomes.

OSWER Response:  The RPL program has initiated discussions with staff in the Office of Solid
Waste and Emergency Response, Center for Program Analysis, to conduct a logic model
assessment. The logic model assessment will be conducted during the Regional Public Liaison
National Meeting, June 16-18, 2009.  The RPL program expects to complete the logic model
assessment by September 30, 2009.
                                        12

-------
                                                                             09-P-0176
Recommendation 2-2
Adopt common data reporting elements across all regional offices and collect, summarize, and
report comparable data.

OSWER Response: The RPL program will adopt common data reporting elements across all
regional offices once the logic model assessment is complete.  Initial discussions on this
recommendation will be conducted during the Regional Public Liaison National Meeting, June
16-18, 2009. The RPL program expects to implement common data reporting elements across
all regional offices by March 31, 2010.

Recommendation 2-3
Report results annually in a consolidated national report.

OSWER Response: The RPL program will report results in a consolidated national report after
common data reporting elements have been adopted across all regional offices. The RPL
program expects to submit the first consolidated national report by March 31, 2011.  This report
will include information from calendar year 2010.

Recommendation 2-4
Develop and use a common performance agreement element for all RPLs.  The element should
refer to the national RPL  program guidance when defining the RPL  role and expected outputs
and outcomes.

OSWER Response: The RPL program will develop and use a common performance agreement
element for all RPLs.  Initial discussions on this recommendation will be conducted during the
Regional Public Liaison National Meeting, June 16-18, 2009.  The development of a final
common element for performance  agreements is dependent on the completion of the logic model
assessment.  The RPL program expects to implement a common performance element by
October 1, 2010.

Recommendation 2-5
Make stakeholders aware of the RPL program by sharing contact information through regional
staff, site descriptions, and a national website linked to each regional RPL website.

OSWER Response: The RPL program will develop a national website linked to each regional
RPL website. The RPL program has obtained contractor support to  develop a national website.
A draft version of the national website will be available for review and comment during the RPL
National Meeting, June 16-18, 2009. The RPL program expects to publish the final website by
September 30, 2009.

Recommendation 2-6
Revise the 2004 RPL guidance as necessary to reflect revisions in the program and build in
minimum requirements for stakeholder awareness activities.

OSWER Response: The RPL program will revise the 2004 RPL guidance to reflect revisions in
the program ascertained from the logic model assessment and build  in minimum requirements
                                          13

-------
                                                                              09-P-0176
for stakeholder awareness activities. The RPL program expects to implement the revised
guidance by October 1, 2010.
CONCLUSIONS

If you have any questions about these comments, please contact Karen L. Martin at
703-603-9925, or martin.karenl@epa.gov.
                                          14

-------
                                                                             09-P-0176


                                                                         Appendix B


                                 Distribution


Office of the Administrator
Assistant Administrator, Office of Solid Waste and Emergency Response
Acting Regional Administrators, Regions 1-10
Principal Deputy Assistant Administrator, Office of Solid Waste and Emergency Response
Director, Office of Superfund Remediation and Technology Innovation
Acting Director, Office of Regional Operations
Acting General Counsel
Acting Associate Administrator for Congressional and Intergovernmental Relations
Acting Associate Administrator for Public Affairs
Agency Follow-up Official (the CFO)
Agency Follow-up Coordinator
Audit Follow-up Coordinator, Office of Solid Waste and Emergency Response
Audit Follow-up Coordinators, Regions 1-10
Public Affairs Officers, Regions 1-10
Acting Inspector General
                                          15

-------