U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Catalyst for Impro ving the En vironment Evaluation Report Regional Public Liaison Program Needs Greater Focus on Results and Customer Awareness Report No. 09-P-0176 June 24, 2009 ------- Report Contributors: Bram Hass Larry Dare Eric Lewis Abbreviations EPA U.S. Environmental Protection Agency FTE Full-Time Equivalent OIG Office of Inspector General OSWER Office of Solid Waste and Emergency Response RPL Regional Public Liaison (formerly Regional Superfund Ombudsman) RSO Regional Superfund Ombudsman ------- U.S. Environmental Protection Agency Office of Inspector General At a Glance 09-P-0176 June 24, 2009 Catalyst for Improving the Environment Why We Did This Review The Office of Inspector General (OIG) evaluated the actions taken by the U.S. Environmental Protection Agency's (EPA's) Office of Solid Waste and Emergency Response (OSWER) to implement its Regional Public Liaison (RPL) program. We also reviewed program changes resulting from a 2003 OIG report. Background Responding to the 2003 OIG report, EPA in 2004 revised its Superfund Regional I Ombudsman program, renamed staff RPLs, and established national program (guidance. The RPLs help the public and regulated community by (1) providing information and facilitating informal contact with EPA staff, (2) helping resolve problems, and (3) making recommendations for improvement to Agency senior management. For further information, contact our Office of Congressional, Public Affairs and Management at (202)566-2391. To view the full report, click on the following link: www.epa.qov/oiq/reports/2009/ 20090624-09-P-0176.pdf Regional Public Liaison Program Needs Greater Focus on Results and Customer Awareness What We Found OSWER's RPL program does not sufficiently focus on or measure specific outputs and outcomes and is not consistently implemented across offices. RPLs report results in varied formats, and OSWER does not consolidate program results into a comprehensive report. As a result, RPLs reported annual results that could not be readily consolidated to show what RPLs had achieved. The program does not sufficiently focus on results because it lacks clear program logic. Developing a logic model would help define outputs and outcomes and encourage a results- oriented approach to program implementation. A more consistent and comprehensive approach would also raise stakeholder awareness of the RPL program. Despite limited resources, RPLs have assisted many stakeholders since the program was revised in 2004. New guidance more clearly described program expectations and sought to provide more consistent program implementation across regional offices. What We Recommend We recommend that OSWER use a logic model approach to revise the RPL program to help focus on outputs and outcomes and ensure stakeholders are aware of the RPL resource. Also, OSWER should revise the 2004 RPL guidance to reflect program revisions and build in minimum requirements for stakeholder awareness activities, including a national RPL Website. OSWER concurred and described planned corrective actions to address all of our recommendations. OSWER stated it has requested assistance from OSWER's Center for Program Analysis to conduct a logic model assessment starting during the RPLs' National Meeting, June 16-18, 2009. OSWER's response explains that this assessment will lead to revised guidance that will include minimum requirements for stakeholder awareness activities. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 OFFICE OF INSPECTOR GENERAL June 24, 2009 MEMORANDUM SUBJECT: FROM: TO: Regional Public Liaison Program Needs Greater Focus on Results and Customer Awareness Report No. 09-P-0 176 Wade T. Najjum Assistant Inspector General Office of Program Evaluation Mathy Stanislaus Assistant Administrator Office of Solid Waste and Emergency Response This is our final report on the subject review conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe concerns the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established resolution procedures. The estimated cost of this report - calculated by multiplying the project's staff days by the applicable daily full cost billing rates in effect at the time - is $355,483. Action Required We believe the written response you provided to the draft report meets the requirements of EPA Manual 2750 for a written response within 90 days of the date of this report. We request that you inform us as you complete each corrective action so we may update our tracking system. We have no objections to the further release of this report to the public. This report will be available at http ://www. epa. gov/oig. If you or your staff have any questions regarding this report, please contact Eric Lewis at 202-566-2664 or lewis.eric@epa.gov, or Larry Dare at 202-566-2138 or dare.larry@epa.gov. ------- Regional Public Liaison Program Needs 09-P-0176 Greater Focus on Results and Customer Awareness Table of Contents Chapters 1 Introduction 1 Purpose 1 Background 1 Noteworthy Achievements 2 Scope and Methodology 3 2 Focusing on Results Would Provide Greater Program Consistency and Facilitate Measuring Results 4 RPL Program Needs Greater Focus on Outputs and Outcomes 4 OSWER Should Analyze Program Logic to Focus on Results 6 Regional Resources Devoted to RPL Program Varied Greatly 6 Performance Agreements Should Include RPL Role 7 Consistent Implementation Would Improve Stakeholder Awareness 7 Conclusions 9 Recommendations 9 Agency Response and OIG Comment 10 Status of Recommendations and Potential Monetary Benefits 11 Appendices A Agency Response to Draft Report 12 B Distribution 15 ------- 09-P-0176 Chapter 1 Introduction Purpose The Office of Inspector General (OIG) reviewed the U.S. Environmental Protection Agency's (EPA's) Regional Public Liaison (RPL) program within the Office of Solid Waste and Emergency Response (OSWER) to answer the following questions: • Is the RPL program being implemented consistently across offices? • How does EPA measure the results (outputs and outcomes) of this program, including its impact on the involved communities and the environmental problems they face? We also looked at whether the program implemented recommendations made in our March 13, 2003, report, EPA Regional Super fund Ombudsmen Program Needs Structure (2003-S-00004). Background EPA created Regional Superfund Ombudsman (RSO) positions in each regional office as part of the Superfund Administrative Reforms effort in 1996. The RSOs facilitate resolution of stakeholder concerns and provide a forum so States and communities can be more informed and involved in clean-up decisions. Prior to this reform, stakeholders raised concerns with regional personnel but had no formal mechanism to have their issues elevated. When EPA appointed the 10 RSOs in June 1996, it issued a press release saying: The establishment of an ombudsman position in each EPA region will help resolve Superfund issues that fall through the cracks in the current system. Through these ombudsmen, who will work closely with the EPA headquarters ombudsman, the public will have access to an EPA employee who will be able to cut through red tape to investigate complaints and arrange meetings with appropriate staff to try to resolve problems. The ombudsmen will have the authority to cut across bureaucratic lines to get answers and resolve problems quickly. In our March 2003 report, we noted that EPA does not have a management system in place to ensure its RSOs are accountable for fulfilling their responsibilities. The RSO program was generally a collateral duty within the ------- 09-P-0176 Superfund program. As a result, there was a perceived lack of independence and impartiality. Further, a lack of guidance had caused uncertainty over the RSO program. The report pointed out that changing the title of the RSOs would allow them to continue to provide a valuable service by informally resolving issues at a local level, and would alleviate the perception that they should meet the American Bar Association's core characteristics of impartiality and independence. Also, to ensure consistent performance and results, we recommended that OSWER provide guidance describing the roles and responsibilities of the position. Responding to our report recommendations, OSWER issued guidance in March 2004, OSWER Guidance for the Regional Public Liaisons (OSWER 9200.0-48). The guidance changed the "Regional Superfund Ombudsman" title to "Regional Public Liaison," and allowed regions, at their discretion, to broaden the former RSOs' scope beyond Superfund to include programs that OSWER administers under nine other Acts. It described the purpose of the RPLs as "... to provide information and assistance to the public in resolving issues and concerns raised about the programs administered by OSWER." The guidance explains that the RPL may be called on to serve in a number of capacities: • Provide information and facilitate informal contact with EPA staff. • Help resolve problems. • Make recommendations to Agency senior management aimed at improving OSWER programs. The RPL program provides stakeholders the opportunity to be involved in OSWER programs and serves as an important internal control to reduce the risk that stakeholders' concerns will go unanswered. In issuing the guidance, OSWER's Deputy Assistant Administrator's cover memo stated that".. .the RPLs have been a useful resource for people outside the Agency who have needed assistance or had concerns about the program. They have helped numerous individuals and defused many potentially difficult situations." He pointed out that the purpose of the guidance was to strengthen the effectiveness of the RPLs by establishing a clear and consistent set of operating principles and procedures. He viewed the guidance as a sound framework for RPL activities that still allowed each region to implement a program that was consistent with its own needs and organization. Noteworthy Achievements RPLs have assisted many stakeholders since the program was revised in 2004. New guidance more clearly described program expectations and sought to provide more consistent program implementation across regional offices. The name of the program was changed from Ombudsman to Regional Public Liaison to more accurately reflect the nature of the program. ------- 09-P-0176 Scope and Methodology We reviewed OSWER's RPL program from January 2008 through May 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the evaluation to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our evaluation objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our evaluation objectives. We limited our review of management controls and compliance to those directly relating to the issues identified. Office of Management and Budget Circular A-123 notes that internal controls are a means of managing the risk associated with federal programs and operations. We analyzed the current RPL guidance and identified documentation and guidance from predecessor programs. We obtained annual reports, resource estimates, job descriptions, performance agreements, meeting minutes, and other information about how RPLs carried out their duties. We conducted searches of EPA's national and regional Websites to determine whether stakeholders could readily find the RPLs, whether RPL names were up to date, and whether RPLs were listed on Superfund site descriptions. We communicated with eight current and five former RPLs. We also visited Region 2 to obtain an understanding of how an extensive RPL effort operated. As previously noted, we reviewed a March 2003 OIG report to determine whether our recommendations in that prior report had been implemented. We also reviewed Government Accountability Office reports and testimony. ------- 09-P-0176 Chapter 2 Focusing on Results Would Provide Greater Program Consistency and Facilitate Measuring Results OSWER's RPL program does not sufficiently focus on or measure specific outputs and outcomes and is not consistently implemented across offices. Although RPLs are dedicated to helping their customers, the current design and implementation limit the program's potential to effectively serve the public and measure results. RPL guidance describes three primary roles: helping members of the public gain access to information, helping resolve problems, and initiating positive institutional change to prevent similar problems in the future. However, because regional offices implement the program differently, offices report results that are not comparable and do not provide a clear picture of the program's overall accomplishments. A more consistent and comprehensive approach would also raise stakeholder awareness of the RPL program. RPL Program Needs Greater Focus on Outputs and Outcomes The 2004 RPL guidance does not define outputs or outcomes for RPLs to track as they carry out their responsibilities. As a result, RPLs reported annual results that could not be readily consolidated to show what RPLs had achieved. Without a clear link between program objectives and expected outputs and outcomes, the program will have little chance of achieving its intended results. EPA's Office of Policy, Economics, and Innovation defines outputs and outcomes as follows: • Outputs: A measure of products and services provided as a direct result of program activities. • Outcomes: Accomplishments of program goals and objectives. The RPL guidance states that each RPL should provide an annual report to his/her Regional Administrator with a copy to the Assistant Administrator for OSWER describing numbers and types of issues that the RPL addressed. It goes on to note that the annual report could be used to make process and organizational recommendations and address publicly issues of concern, but does not require a consolidated report. Based on this requirement, RPLs should be reporting outputs ("numbers.. .of issues and concerns... addressed"). Categorizing them by "types of issues and concerns" is left up to the RPLs. Because each RPL categorizes reporting outputs differently and there was no shared format, RPL reports we reviewed were not consistent. For example: ------- 09-P-0176 • One RPL provided several breakouts, including information requests versus assistance requests, and categorization by EPA program area as well as the source of the request. • One RPL showed number of contacts received, number that were site- related, and number referred. • One RPL with only a few cases described them and noted whether they were open or closed at the end of the fiscal year. • One RPL reported no major cases but referred 15-20 calls to other divisions. None of the RPLs summarized activities by the three roles of the RPL (providing information, cutting red tape, and initiating institutional change). Unless the RPLs use comparable categories when tracking and reporting on inquiries and cases, it will be difficult to summarize overall RPL program results. While the reporting requirement does not require RPLs to track outcomes, we believe it would be beneficial. We found two outcome categories in sections where the guidance discussed other subjects: (1) "resolving problems," and (2) "members of the public.. .gain access to information.. .that will help them participate more fully in established Agency processes." Program outcomes also could be described in terms of the RPLs' three key roles. Outputs and outcomes should be integrated into an overall program design for tracking and reporting. Although EPA reported it had measures for the earlier RSO program, one RPL told us that no common output or outcome measures were ever provided to the regions. None are currently being used. These measures are listed in the Government Accountability Office's May 2000 report on Superfund Administrative Reforms.1 One measure was the number of cases for which EPA conducted investigations and mediations. Another was whether the public's perceptions of EPA's decisions improved. To capture and assess outputs and outcomes, RPLs must collect and report comparable data regarding program accomplishments. They instead use different methods to track cases and collect different data about program accomplishments. For example, one RPL uses a spreadsheet to log cases while others use less formal methods. Annual reports also capture results differently; some RPLs simply summarize the results of each case they handled during the year, while others produce relatively detailed reports. Further, not all RPLs sent annual reports each year. Without comparable data and reporting, the RPL program cannot conduct the kind of analysis necessary to evaluate results, identify areas for improvement in the RPL program, or encourage institutional reform in OSWER-related programs. 1 SUPERFUND: Extent to Which Most Reforms Have Improved the Program Is Unknown, May 2000, GAO/RCED- 00-118. ------- 09-P-0176 The RPL program coordinator does not produce a consolidated annual report that would show the RPL program's accomplishments. National guidance requires each RPL to submit a report annually but does not require a consolidated report. Nonetheless, we believe a national report would be beneficial. OSWER Should Analyze Program Logic to Focus on Results Given its limited resources, we believe the RPL program would benefit by using a logic model. We analyzed OSWER's 2004 guidance for RPLs to determine if a logic model could be constructed from its description of the program. The guidance did not provide all the elements needed for a usable logic model. If the RPL program design was based on a logic model, then it would focus more specifically on defining outputs and outcomes and thus more clearly on program results. EPA training materials explain that a logic model is a diagram and text that describes/illustrates the logical (causal) relationships among program elements and the problem to be solved, thus defining measurements of success. Logic models help managers answer key questions and understand and check assumptions on how the program is supposed to work. These models can support program improvement and evaluation, helping to answer such questions as: • What am I doing, with whom, and to whom/what? • How well am I doing it? • Is anybody (anything) better off in either the short or long term? • What role, if any, did my program play in the results? • What role, if any, did factors unrelated to the program play in the results? • Were there any unintended outcomes, and if so, why? EPA and many other organizations use logic models to better define the relationship of resources; customers; outputs; and short-, medium-, and long-term results (or outcomes). This technique enables one to determine whether the program is set up to succeed and facilitates program evaluation. The EPA Office of Policy, Economics, and Innovation's logic model training stresses that meaningful measuring of program performance requires a clear understanding of what a program does and the results it is intended to accomplish. EPA and the Office of Management and Budget endorse and make use of the logic model approach. Regional Resources Devoted to RPL Program Varied Greatly Resources used by the various regions for the RPL program varied greatly, and one benefit of using a logic model approach is that it could help OSWER and the regional offices assess the adequacy of resources. According to estimates provided by the RPLs and the program coordinator, the entire RPL program used about 2.5 FTEs in 2008. Implementing the RPL ------- 09-P-0176 program is a collateral duty for most RPLs. One RPL (Region 2) estimated his region's effort at 1.9 FTEs. He and two assistants handle several hundred inquiries per year among other duties. Their inquiries come from the region's main Web inquiry system, a Superfund hotline, direct contacts from stakeholders, and referrals from EPA staff. Another RPL reported spending about 35 percent of his time on RPL matters. RPL estimates for other regions ranged from 1 percent to 10 percent of one FTE. We believe that spending such a small amount of time may not allow RPLs to carry out the full extent of their duties. Further, two regional offices do not have anyone designated as RPL. Several RPLs and other regional staff attributed the limited RPL resources devoted to cases in their regions to excellent community relations. Performance Agreements Should Include RPL Role The 2004 RPL guidance states that RPL duties should be included in the individual's annual performance agreement: ... The RPL is accountable for the successful performance of his/her official duties, as defined in his/her position description and more specifically delineated in his/her annual performance agreement. We found that two of the eight current RPL performance agreements did not mention their RPL roles. Four RPLs had general statements related to their RPL duties. Two had more specific wording related to their RPL role. None referred to the 2004 OSWER guidance. We found no common wording or description of duties among those we reviewed. A common understanding and written description of RPL duties in performance agreements would help ensure that the regions implemented the RPL program more consistently and achieved program outputs and outcomes. Consistent Implementation Would Improve Stakeholder Awareness A more consistent and comprehensive approach would raise stakeholder awareness of the RPL program. EPA should ensure that stakeholders who may need RPLs are aware of their existence and can reach them easily. Currently, the guidance does not address awareness efforts. Based on information provided by RPLs and our checks of regional Websites, we found that EPA's efforts to inform stakeholders about the RPL program are inconsistent. For example, 4 of 10 regional Websites did not mention the program. Stakeholders may learn of the RPL program from the RPL directly (at public meetings, for example), from other EPA staff, or from EPA documents and Internet sites. ------- 09-P-0176 RPL Guidance Does Not Address Stakeholder Awareness The 2004 RPL guidance does not advise RPLs on ensuring that stakeholders know the RPLs exist. The guidance states that "If community members believe that the official Agency procedures have failed to adequately address their concerns, they may ask the RPL to help them get information or assistance from within the system." It does not describe how RPLs inform stakeholders outside EPA of their availability. When the RPLs' predecessor RSO program started in 1996, implementation plans prepared by several regional offices described methods for publicizing the existence of that program. For example, Region 2's 1996 Implementation Plan stated that".. .The degree to which the public is made aware of the ... Program and its availability and accessibility will determine the effectiveness of the Program. Therefore, it is imperative that EPA's stakeholders become aware of the program's availability, capability and limitations." The plan listed various activities including creating a toll free telephone number, distributing fliers at certain Superfund sites describing the program, and providing handout materials to elected officials and the public. The plan also suggested coordinating with a wide range of other regional EPA staff and State agencies; developing an RSO Web page; and meeting with citizens, community members, and special interest groups. Implementation plans for Regions 7 and 10 listed similar activities. A handbook for an even earlier ombudsman program stated that the program's effectiveness depended to a large extent on public awareness, noting that"... Any system for handling complaints is of no use unless the public can actually get into the system when the need arises." It specified certain actions, such as: developing program fliers and brochures, providing a Federal Register notice, contacting congressional representatives, listing the ombudsman separately in telephone directories, and including a description of the program in employee orientation. Various Techniques Used by Regions to Raise Awareness Raising awareness of the RPL program varies among regions, and some techniques could be easily implemented. Some RPLs have taken specific steps to ensure stakeholders are aware of their program. Others appear to take a passive approach, only reacting to inquiries if and when they receive them. Although RPL staff discussed a draft early in the program, they have never produced a national brochure. Some RPLs told us that they have attended public meetings to explain their role and share their contact information. One RPL noted that he developed a brochure and handed it out at public meetings. He also mailed a notification about his availability to all Potentially Responsible Parties associated with the sites in his region and to the local jurisdictions where the sites are located. Further, he reported joining EPA colleagues on site visits. ------- 09-P-0176 Greater Visibility on Websites Could Help Some regional Websites lack any apparent mention of the RPL. EPA's Websites are one key avenue for reaching people and organizations that might benefit from RPL assistance. EPA Policy 2191.0 emphasizes the importance of the Internet by stating, "The EPA Web site is a fundamental part of every Agency program; taken together, these pages are the foundation of the Agency's environmental outreach and communication program." We found no reference to the RPL on four regional Websites. Also, when we checked regional Superfund site fact sheets listed on regional Websites, we found that only two regions included RPL contact information with the site. On one site we found that the RPL was included in the body of the site description document; in the other, the RPL was mentioned in a side banner next to the site description. Because EPA considers Websites such an important part of its awareness and communication efforts, the RPLs should be prominently displayed on each Website and with site descriptions. The list of RPLs on EPA's national Website was last updated in 2003 and is currently outdated. That list follows a description of the program's origin in the 1996 Superfund Administrative Reforms. However, we recently found that EPA's community involvement Websites now include a link to a Regional Public Liaison site that is under development. Program managers and RPLs have discussed plans to develop an RPL Website, prepare a brochure, and add an article about the RPLs to the community involvement program's Web page. These steps should be undertaken only after the program has defined its program logic and intended outputs and outcomes. Conclusions The RPL program provides an important link between concerned stakeholders and an EPA liaison who can provide assistance. It should provide OSWER an internal control to reduce the risk that significant stakeholders' concerns go unaddressed. However, the program currently has design and operational deficiencies that hinder its chances of success. These deficiencies can be addressed by defining the program logic and performance measures (outputs and outcomes), better use of performance agreements, and improving stakeholder awareness of RPLs. Recommendations We recommend that the Assistant Administrator for Solid Waste and Emergency Response: 2-1 Develop an RPL program logic model to: • identify outputs and outcomes, • assess resource needs, and • revise the program's design to help RPLs achieve the desired outputs and outcomes. ------- 09-P-0176 2-2 Adopt common data reporting elements across all regional offices and collect, summarize, and report comparable data. 2-3 Report results annually in a consolidated, national report. 2-4 Develop and use a common performance agreement element for all RPLs. The element should refer to the national RPL program guidance when defining the RPL role and expected outputs and outcomes. 2-5 Make stakeholders aware of the RPL program by sharing contact information through regional staff, site descriptions, and a national Website linked to each regional RPL Website. 2-6 Revise the 2004 RPL guidance as necessary to reflect revisions in the program and build in minimum requirements for stakeholder awareness activities. Agency Response and OIG Comment OSWER concurred and described planned corrective actions to address all of our recommendations. OSWER stated is has obtained contractor support to develop a national RPL Website and has requested assistance from OSWER's Center for Program Analysis to conduct a logic model assessment starting during the RPLs' National Meeting, June 16-18, 2009. The week after the meeting, OSWER told us that it started work on the logic model during the national meeting and will get contractor support to help complete it. OSWER's response to our draft report said that the assessment will lead to common data reporting elements, revised guidance that will include minimum requirements for stakeholder awareness activities, and a common RPL performance agreement element. After adopting common data reporting elements, the RPL program will begin submitting consolidated national reports. We believe these actions will address our recommendations and we will close out the recommendations as those actions are completed. 10 ------- 09-P-0176 Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS Rec. No. Page No. Subject Status1 Action Official Planned Completion Date POTENTIAL MONETARY BENEFITS (in $OOOs) Claimed Agreed To Amount Amount 2-1 9 Develop an RPL program logic model to (1) identify outputs and outcomes, (2) assess resource needs, and (3) revise the program's design to help RPLs achieve the desired outputs and outcomes. 2-2 10 Adopt common data reporting elements across all regional offices and collect, summarize, and report comparable data. 2-3 10 Report results annually in a consolidated, national report. 2-4 10 Develop and use a common performance agreement element for all RPLs. The element should refer to the national RPL program guidance when defining the RPL role and expected outputs and outcomes. 2-5 10 Make stakeholders aware of the RPL program by sharing contact information through regional staff, site descriptions, and a national Website linked to each regional RPL Website. 2-6 10 Revise the 2004 RPL guidance as necessary to reflect revisions in the program and build in minimum requirements for stakeholder awareness activities. Assistant Administrator for Solid Waste and Emergency Response Assistant Administrator for Solid Waste and Emergency Response Assistant Administrator for Solid Waste and Emergency Response Assistant Administrator for Solid Waste and Emergency Response Assistant Administrator for Solid Waste and Emergency Response Assistant Administrator for Solid Waste and Emergency Response 09/30/2009 03/31/2010 03/31/2011 10/01/2010 09/30/2009 10/01/2010 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is undecided with resolution efforts in progress 11 ------- 09-P-0176 Appendix A Agency Response to Draft Report June 10, 2009 MEMORANDUM SUBJECT: Response to Draft Public Liaison Report: "Regional Public Liaison Program Needs Greater Focus on Results and Customer Awareness" (Project No. OCPL-FY08-003) FROM: Barry N. Breen/s/ Deputy Assistant Administrator TO: Eric Lewis, Director, Special Reviews Office of Program Evaluation PURPOSE The purpose of this memorandum is to provide a written response to the findings of the Draft Evaluation Report: "Regional Public Liaison Program Needs Greater Focus on Results and Customer Awareness" (Project No. OCPL-FY08-0003). My staff has reviewed the draft report and agrees with the conclusions presented in the draft report. Below you will find the corrective actions we have initiated or planned for each recommendation. RECOMMENDATIONS AND OSWER RESPONSE Recommendation 2-1 Develop a Regional Public Liaison (RPL) program logic model to (1) identify outputs and outcomes; (2) assess resource needs, and (3) revise the program's design to help RPLs achieve the desired outputs and outcomes. OSWER Response: The RPL program has initiated discussions with staff in the Office of Solid Waste and Emergency Response, Center for Program Analysis, to conduct a logic model assessment. The logic model assessment will be conducted during the Regional Public Liaison National Meeting, June 16-18, 2009. The RPL program expects to complete the logic model assessment by September 30, 2009. 12 ------- 09-P-0176 Recommendation 2-2 Adopt common data reporting elements across all regional offices and collect, summarize, and report comparable data. OSWER Response: The RPL program will adopt common data reporting elements across all regional offices once the logic model assessment is complete. Initial discussions on this recommendation will be conducted during the Regional Public Liaison National Meeting, June 16-18, 2009. The RPL program expects to implement common data reporting elements across all regional offices by March 31, 2010. Recommendation 2-3 Report results annually in a consolidated national report. OSWER Response: The RPL program will report results in a consolidated national report after common data reporting elements have been adopted across all regional offices. The RPL program expects to submit the first consolidated national report by March 31, 2011. This report will include information from calendar year 2010. Recommendation 2-4 Develop and use a common performance agreement element for all RPLs. The element should refer to the national RPL program guidance when defining the RPL role and expected outputs and outcomes. OSWER Response: The RPL program will develop and use a common performance agreement element for all RPLs. Initial discussions on this recommendation will be conducted during the Regional Public Liaison National Meeting, June 16-18, 2009. The development of a final common element for performance agreements is dependent on the completion of the logic model assessment. The RPL program expects to implement a common performance element by October 1, 2010. Recommendation 2-5 Make stakeholders aware of the RPL program by sharing contact information through regional staff, site descriptions, and a national website linked to each regional RPL website. OSWER Response: The RPL program will develop a national website linked to each regional RPL website. The RPL program has obtained contractor support to develop a national website. A draft version of the national website will be available for review and comment during the RPL National Meeting, June 16-18, 2009. The RPL program expects to publish the final website by September 30, 2009. Recommendation 2-6 Revise the 2004 RPL guidance as necessary to reflect revisions in the program and build in minimum requirements for stakeholder awareness activities. OSWER Response: The RPL program will revise the 2004 RPL guidance to reflect revisions in the program ascertained from the logic model assessment and build in minimum requirements 13 ------- 09-P-0176 for stakeholder awareness activities. The RPL program expects to implement the revised guidance by October 1, 2010. CONCLUSIONS If you have any questions about these comments, please contact Karen L. Martin at 703-603-9925, or martin.karenl@epa.gov. 14 ------- 09-P-0176 Appendix B Distribution Office of the Administrator Assistant Administrator, Office of Solid Waste and Emergency Response Acting Regional Administrators, Regions 1-10 Principal Deputy Assistant Administrator, Office of Solid Waste and Emergency Response Director, Office of Superfund Remediation and Technology Innovation Acting Director, Office of Regional Operations Acting General Counsel Acting Associate Administrator for Congressional and Intergovernmental Relations Acting Associate Administrator for Public Affairs Agency Follow-up Official (the CFO) Agency Follow-up Coordinator Audit Follow-up Coordinator, Office of Solid Waste and Emergency Response Audit Follow-up Coordinators, Regions 1-10 Public Affairs Officers, Regions 1-10 Acting Inspector General 15 ------- |