Office of Inspector General Memorandum Report EPA Needs Better Integration of the National Environmental Performance Partnership System 2000-M-000828-000011 March 31, 2000 ------- March 31, 2000 MEMORANDUM SUBJECT: EPA Needs Better Integration of the National Environmental Performance Partnership System Report No. 2000-M-000828-000011 TO: W. Michael McCabe Acting Deputy Administrator We have done a series of regional audits and reviews evaluating regional and state National Environmental Performance Partnership System (NEPPS) and performance partnership grant (PPG) program implementation. The Environmental Protection Agency (EPA) regional offices have committed to make changes in their NEPPS and PPG program implementation to address our regional audit recommendations. Our work shows that NEPPS as well as PPGs have not been well-integrated into EPA and not all EPA managers and staff buy-into or accept the new performance partnership system. We believe that EPA can help increase NEPPS/PPG success by establishing a more formal structure to set direction, establish goals, provide training, oversee accomplishments, and ensure accountability. Attached is a summary of our review results and recommendations that require your attention to better integrate the performance partnership system throughout EPA. Action Required We have designated you as the Action Official for this report. In accordance with EPA Order 2750, we request that you provide this office with a written response within 90 days of the final report date. For corrective actions planned, but not completed by the response date, reference to specific milestone dates will assist in deciding whether to close this report. We have no obj ections to the release of this report to the public. This audit report contains findings that the Office of Inspector General (OIG) has identified and corrective actions OIG recommends. This audit report represents the opinion of OIG, and the findings in this report do not necessarily represent the final EPA position. Final determinations on matters in this audit report will be made by EPA managers in accordance with established EPA audit resolution procedures. ------- If you have any questions, please call me on (202) 260-3137 or Jeff Hart, Audit Manager, at (303) 312-6169. Attachment /s/ Nikki L. Tinsley cc: Diane Thompson, Associate Administrator, Office of Congressional and Intergovernmental Relations ------- EPA NEEDS BETTER INTEGRATION OF THE NATIONAL ENVIRONMENTAL PERFORMANCE PARTNERSHIP SYSTEM The National Environmental Performance Partnership System (NEPPS) has not been well- integrated into the Environmental Protection Agency (EPA) and not all EPA managers and staff buy-into or accept NEPPS. NEPPS requires a change in EPA organizational culture that has not yet occurred. Regional NEPPS coordinators and headquarters staff that work directly with implementing NEPPS have adopted NEPPS components into their daily activities. However, many other EPA program managers and staff have not integrated NEPPS into their day-to-day operations or into helping them accomplish EPA's mission. NEPPS CREATED A NEW FRAMEWORK FOR ACHIEVING ENVIRONMENTAL RESULTS On May 17, 1995, EPA and state leaders signed the NEPPS agreement creating a new framework for working together as partners. NEPPS was designed to strengthen EPA and state protection of public health and the environment by directing scarce public resources toward improving environmental results, allowing states greater flexibility to achieve those results, and enhancing EPA and state accountability to the public. The NEPPS agreement outlined and described seven principal components of the new system: 1. increased use of environmental goals and indicators 2. new approach to program assessments by states 3. performance partnership agreements 4. differential oversight 5. performance leadership programs 6. public outreach and involvement 7. joint system evaluation EPA's Strategic Plan, dated September 1997, states that EPA would develop policies, guidance documents, and training materials as needed to enhance EPA and state capacity to implement elements of the performance partnership system. However, as discussed later, EPA has not developed guidance documents or provided training to implement the performance partnership system. In addition to NEPPS, EPA implemented performance partnership grants (PPGs) beginning in 1996 that allowed states and tribes to combine multiple EPA grants into one grant. PPGs are important tools for implementing NEPPS and share many of the same objectives as NEPPS: focusing on environmental results, creating incentives for improved performance, allowing increased flexibility to improve performance, and enhancing accountability to the public. Specifically, PPGs are to provide recipients with flexibility to address their highest environmental priorities, reduce administrative burdens and costs, strengthen partnerships toj ointly accomplish goals, and improve environmental results and more effectively 1 ------- link program activities with environmental goals and program outcomes. A change in organizational culture such as that represented by NEPPS requires strong leadership providing the direction, expectations, and goals of the new system. Staff should have a clear understanding of what the new system is trying to accomplish and how the new system will help them do their jobs. Clearly defined goals and objectives for implementing the new system should be developed along with specific performance measures that are tracked throughout the implementation and are used for monitoring. FACTORS CONTRIBUTING TO LACK OF NEPPS INTEGRATION AND CULTURAL CHANGE NEPPS was not well-integrated into EPA because of four factors: lack of (1) leadership that provided a clear direction and set expectations, (2) training and guidance, (3) trust in NEPPS due to fear of change and losing control, and (4) goals and related performance measures to monitor and measure progress on achieving better environmental results. Lack of Leadership EPA did not have a leader or champion for NEPPS. Many EPA and state staff did not perceive that senior EPA leadership was committed to NEPPS. EPA had not clearly established a central authority or responsibility for NEPPS and senior EPA management had not clearly communicated its expectations about NEPPS and PPGs. Although various memorandums and policy statements were issued from the Deputy Administrator, EPA staff often did not know where to turn for specific information on direction, expectations, and clarification. For many EPA staff, NEPPS was perceived as a policy that was implemented only if a state wanted it and even then the state could choose which NEPPS components it wanted to participate in. Some EPA staff also interpreted NEPPS to apply just to those states that had performance partnership agreements and PPGs. EPA and state staff also had the following concerns: Some state officials were reluctant to participate in NEPPS or to take advantage of some of the flexibility offered by PPGs because they were afraid that NEPPS was an EPA initiative that would disappear. Some state officials stated that the lack of senior leadership and support for NEPPS indicated that EPA was not committed to NEPPS and the idea of performance partnerships to better accomplish environmental results. Some EPA staff have commented that the NEPPS components should be updated and the expectations for each component clearly defined. For example, the performance leadership program component was to provide state programs that meet specific criteria (to be developed jointly by EPA and states) with national recognition and afforded minimum allowable EPA oversight. However, EPA and state staff found that defining the criteria for the leadership program was very difficult. As a result, the leadership program is not being pursued at this time. 2 ------- Lack of Training and Guidance EPA and state staff have not had any training that would educate staff on NEPPS as well as begin to support the kind of cultural change involved in adopting NEPPS. NEPPS is a different approach to working with states and achieving improved environmental results than EPA's traditional command-and- control approach that emphasized accomplishing activities such as the number of permits and inspections. EPA has also not developed any NEPPS guidance. The lack of training and clear guidance has resulted in many EPA managers and staff having little direction about how to incorporate NEPPS into their programs. EPA managers and staff did not understand how NEPPS would help them accomplish the work of the agency. Staff perceived that their responsibility was to ensure that states were implementing effective programs to address environmental problems. NEPPS has created a great deal of concern among some EPA managers and staff who believe NEPPS could eliminate program and financial accountability. For example, EPA and states have not yet agreed on how to provide states flexibility along with accountability. EPA and state managers struggled with how to provide states flexibility to address their highest environmental priorities while continuing to implement and report on core program requirements such as permitting, inspections, and enforcement. These core program requirements are based on statutes, regulations, and standing legal agreements between EPA and the states. EPA and states had generally not moved toward funding regional or state priorities beyond the normally funded core program activities. EPA and state managers had not yet determined where, or if, disinvestments in some of the core program activities could be made. As a result, performance partnership agreements often included a layering of basic core program requirements (activities) with some outcome- based or indicator measures. The layering of outcome-based measures on top of traditional activity-based measures often created state work plans that had several hundred measures and these measures often did not link to environmental outcomes. For example, the majority of one state's 365 measures in its performance partnership agreement for fiscal 1998-1999 were output measures. The measures included 43 outcome-based measures, or 11.9 percent, and environmental indicators accounted for only 35, or 9.5 percent, of the 365 measures. Another state had approximately 386 measures in its fiscal 1997 and 1998 performance partnership agreement and only 36, or 9.3 percent, of these measures were environmental indicators. In one region, we reviewed a sample of five state performance partnership agreements for fiscal 1998 and the majority of the 727 measures reviewed were activity measures. These measures included the national core performance measures. National core performance measures represent a limited number of program and multi-media performance measures, including enforcement measures, that are designed to gauge progress toward protection of the environment and human health as well as provide a national picture of the status of the environment. EPA headquarters launched NEPPS without any guidance for the Regions and the states, hoping to encourage innovation and maximize flexibility. While some regional and state staff were able to use some new innovative and flexible approaches to address environmental problems, the lack of guidance, training, and clear direction resulted in many EPA and state staff having widely different impressions and understanding of NEPPS and what it was designed to accomplish. The following are examples of the 3 ------- different impressions and understanding of NEPPS by EPA and state staff: Regional and state officials had differing expectations concerning reductions in oversight and reporting activities. State officials expected considerable reduction in oversight and reporting activities, but regional staff often could not meet these expectations because national EPA guidance still requested specific information and because regional staff were reluctant to relinquish their oversight role. In addition, regional EPA managers and staff had different impressions among each other regarding NEPPS components. Many EPA managers and staff also believed NEPPS had removed or substantially diminished the ability of EPA to hold states accountable for delegated programs and for federal funding. States, on the other hand, complained that EPA was still seeking specific, activity-based commitments from them, often outside the NEPPS process. Some regional program managers complained that their programs were no longer able to obtain specific work commitments from the states as they had under the old system of categorical grants and media-specific work plans and that the states were no longer accountable to EPA. Managers stated that they were no longer able to obtain specific work commitments because state officials would argue that the notion of specific commitments was inconsistent with acting as partners. Further, some senior EPA officials would agree with state officials that specific commitments in the work plans were not necessary. As a result, some EPA managers and staff continued to ask for the old-style activity commitments outside the context of NEPPS and sometimes held states responsible for additional, verbal commitments that were never incorporated into a state's work plan or grant agreement. Lack of Trust in the New System Many EPA and state staff were still embedded in their media-specific, activity-based culture and lacked trust in the new system. NEPPS required a new working relationship, a new measuring system, and ultimately a change in how managers and staff have managed their programs. Many EPA and state staff viewed their activity-based authorities under the media-specific statutes as having priority and had difficulty reconciling these media-specific activities with NEPPS' cross-media, priority-setting process that focuses on environmental results rather than on the number of permits and inspections. The following are examples of EPA and state staff fears and concerns regarding NEPPS: Many EPA and state program managers were fearful of losing control over their programs under NEPPS. NEPPS and PPGs emphasize a cross-media, priority-setting process that would allow media-specific funds to be moved to other higher priority areas. Many EPA and state program managers did not want to lose control over their program's funding. They believed it would be difficult to adequately prioritize cross-media priorities and shift funding without having to give up some of the basic core program requirements. Many EPA regional program managers and staff were uncomfortable with changing how 4 ------- they worked with state officials. Some EPA regional program managers were uncomfortable with states asserting more control and independence in how state officials managed their programs. EPA program managers believed that some state officials were asserting too much control and using NEPPS as the basis for trying to eliminate any federal oversight and involvement in environmental programs. EPA and state staff had used activity measures as a basis of evaluating a state's program performance for many years and were unsure how to develop outcome-based measures that provided the same type of information and assurance on an annual basis of how well a program was being implemented. While staff recognized that outcome-based measures focusing on environmental results were important to develop, they had not seen any good outcome-based measures that provided information they needed to evaluate a program. Regional officials were still predominantly using activity-based measures to hold states accountable and had not disinvested in these types of measures because (1) outcome-based measures were hard to develop, (2) data did not exist on which to base outcome-based measures, and (3) national program managers were still requesting activity-based information. In addition, many regional program managers and staff were very comfortable with using activity-based or output measures for measuring results and found it difficult and resource-intensive to develop outcome measures. Regional officials were also using performance measures that were not time-specific, measurable, quantifiable, and verifiable. In one region, we sampled five state performance partnership agreements and of the more than 700 measures we reviewed, only 76, or about 10.5 percent, met all four attributes. These measures included the national core performance measures. Lack of Goals and Performance Measures Although NEPPS and PPGs have their own overall goals, EPA has not defined its performance measures and related milestones to measure how EPA and its partners are progressing toward accomplishing those goals. NEPPS and PPGs focus on achieving improved environmental results. However, EPA has not defined specific measurable goals for evaluating whether it is making progress toward obtaining environmental results and whether NEPPS and PPGs are contributing to those results. EPA staff often provided summaries of how many states had performance partnership agreements and PPGs, but then stated that these numbers did not indicate whether EPA and its partners were successful or not in implementing NEPPS. However, the emphasis on these numbers and the lack of any other performance measures have made the number of participating states appear to be the measure of success for NEPPS. EFFECT OF INADEQUATE NEPPS INTEGRATION Environmental improvements may not be happening as rapidly or to as great an extent as they might be if there was better integration and acceptance of NEPPS and meaningful performance measurement. EPA and states have not been able to redirect scarce resources to improving environmental results. For many EPA regional managers and staff NEPPS has not focused on environmental results or deferred work that 5 ------- was not a priority, but rather has been added to a long list of traditional work responsibilities. Further, some regional program managers and staff viewed NEPPS as only a paperwork exercise to get a performance partnership agreement in place. PROGRESS TO DATE EPA has taken several steps to try to implement and integrate NEPPS. For example, EPA has Established a NEPPS coordinator for each region that coordinates and communicates NEPPS within the regions. Established a national NEPPS work group, consisting of regional NEPPS coordinators and headquarters staff, that regularly discusses issues and problems regarding NEPPS implementation. Coordinated national NEPPS workshops for EPA staff as well as j oint workshops including both EPA and state staff. EPA staff also established work groups to address issues from these workshops. Developed agreement with states on national core performance measures for each of EPA's media offices as well as enforcement. Included some NEPPS components, such as the use of core performance measures, in national program guidance. Encouraged states to participate in both performance partnership agreements and PPGs. EPA and state staff have improved communication through the use of these agreements and grants. Staff have had more open discussions on each other's priorities and goals and on using a priority-setting process to address those priorities and goals. Worked on revising Code of Federal Regulations 40 Part 35 to build into the regulations the performance partnership concepts so that grant requirements are streamlined and include PPG requirements. However, despite these efforts, NEPPS has not been well-integrated and accepted within EPA. EPA has used recent assessments of NEPPS by the U.S. General Accounting Office, Office of Inspector General (OIG), and EPA-state staff to improve its activities under NEPPS. EPA has initiated the following steps to better clarify its directions for NEPPS: On March 17, 2000, the Acting Deputy Administrator signed a memorandum reaffirming EPA's commitment to NEPPS as well as directing the Office of Congressional and Intergovernmental Relations to be the national program manager for NEPPS. The 6 ------- memorandum also asks that all staff work closely with the NEPPS national program manager to integrate NEPPS into EPA's planning and budgeting processes; management and personnel standards; and reporting and communication activities. The memorandum states that EPA's goal for NEPPS in fiscal 2000 is to make the performance partnership agreements and the process for developing them so valuable and integral to protecting the environment that all states will be interested in participating. To accomplish this goal, the memorandum outlines four objectives that EPA needs to focus on: Continued improvement aimed at more useful performance partnership agreements; Expansion of the use of PPGs to support NEPPS; Identification and implementation of burden reduction opportunities for states; and Continued efforts to enhance the use of outcome-oriented core performance measures. - On March 13, 2000, the Office of Congressional and Intergovernmental Relations, in coordination with the Office of the Chief Financial Officer issued a memorandum that provided a crosswalk between EPA's Annual Performance Measures and the national core performance measures. The memorandum stated that when states reported core performance measures data to EPA under NEPPS, EPA could use that same data to report to Congress for purposes of the Government Performance and Results Act. The memorandum stated that the Office of Congressional and Intergovernmental Relations would work with national program managers and regions to create tools and training for EPA use at all organizational levels. Joint evaluation of NEPPS by EPA and state officials planned for calendar year 2000. EPA's implementation of the above planned actions as well as our recommendations below should help integrate NEPPS within EPA, and increase NEPPS success in achieving improved environmental results. RECOMMENDATIONS EPA can better integrate NEPPS and PPGs into EPA and the states and ultimately achieve improved environmental results by implementing the following steps: 1. Continue to provide strong and consistent leadership for NEPPS and PPGs as demonstrated in your March 17, 2000 memorandum reaffirming EPA's commitment to NEPPS. 2. Work with EPA and state staff to establish clear direction, expectations, goals, performance measures, and milestones for NEPPS and PPGs. 7 ------- 3. Ensure the new NEPPS national program manager has full authority and responsibility for implementing NEPPS and PPGs so that leadership is strengthened and NEPPS goals are being developed and progress is evaluated. 4. Train EPA and state staff on how (1) NEPPS can be effective in finding solutions to their key environmental problems and in better managing their programs, (2) to move from an activity-based to outcome-based measurement system, and (3) to move from a historic command-and-control relationship to partnering. 5. Develop a NEPPS guidance document that takes into consideration the lessons learned to date. (U.S. General Accounting Office also reached the same conclusion suggesting that, as part of the joint EPA/state evaluation of NEPPS, attention should be paid to "developing a set of flexible guidelines, to be used a tool by state and EPA regional NEPPS negotiators, that could help clarify the appropriate performance expectations and other conditions that states must meet to achieve reduced oversight in carrying out their environmental programs...") OBJECTIVES. SCOPE AND METHODOLOGY In fiscal 1998 and 1999, OIG divisional audit offices and EPA regional staff conducted joint reviews in 8 of the 10 regions. These reviews were the first formal activities that OIG and regional staff jointly conducted and represented OIG's new approach of working collaboratively with EPA. These reviews were performed in accordance with the review guide. The reviews were not audits and were not performed in accordance with Government Auditing Standards. The goal of these joint reviews was to identify lessons learned in implementing NEPPS and PPGs and to allow regions and states to take corrective action before OIG conducted formal audits. Each review team evaluated a state's PPG and the state's performance partnership agreement. The team analyzed a sample of program performance measures and interviewed regional and state staff. The review teams issued memorandums to the regions and states that summarized the results and offered suggestions for improvement. OIG and EPA regional staff conducted the following reviews: Review of Connecticut's fiscal 1997 and 1998-1999 performance partnership agreements and PPGs, dated April 21, 1999 • Review of New Jersey's fiscal 1998 PPG and 1997-1998 performance partnership agreement, dated October 14, 1999 Review of Delaware's fiscal 1997PPGand 1997 performance partnership agreement, dated April 12, 1999 Review of Georgia's fiscal 1997 PPG and 1997 performance partnership agreement, dated April 23, 1999 Review of Indiana's fiscal 1997 PPG and 1997 performance partnership agreement, dated 8 ------- March 22, 1999 Review of Texas' fiscal 1997 PPG and 1997 performance partnership agreement, dated April 8, 1998 Review of Colorado's fiscal 1997 PPG and 1997 and 1998 performance partnership agreements, dated July 2, 1998 Review of Alaska's fiscal 1998 PPG and 1998-1999 performance partnership agreement, dated July 29, 1999 In fiscal 1999 and 2000, we completed PPG program implementation and oversight audits in Regions 4, 5, 6, and 8. These audits were performed in accordance with Government Auditing Standards (1994 Revision) issued by the Comptroller General of the United States. The purpose of the audits was to determine whether regions were effectively implementing their PPG programs to accomplish the four PPG goals: (1) administrative savings, (2) flexibility to address state priorities, (3) strengthened partnerships, and (4) improved environmental results. Our audits also addressed components ofNEPPS becauseNEPPS and PPGs are so interrelated. We specifically addressed performance partnership agreements, differential oversight, environmental indicators, and results-oriented performance measures. The audit teams reviewed several PPG work plans, analyzed a sample of program performance measures, evaluated end-of-year reports, interviewed regional and state staff, and evaluated regional approaches to providing flexibility along with accountability. Each audit included specific recommendations for improving regional NEPPS and PPG implementation. Audit Reports GRANTS: Region 4's Implementation and Oversight of Performance Partnership Grants, September 27, 1999, Report No. 1999-P00216 GRANTS: Improving Region 5's EnPPA/PPG Program, February 29, 2000, Report No. 2000-P- 00008 PERFORMANCE PARTNERSHIP GRANTS: Region 6 Oversight of Performance Partnership Grants, September 21, 1999, Report No. 1999-000208-R6-100282 PERFORMANCE PARTNERSHIP GRANTS: Region 8 Needs to Improve Its Performance Partnership Grant Program to Ensure Accountability and Improved Environmental Results, September 29, 1999, Report No. 1999-000209-R8-100302 We also attended and participated in several national NEPPS workshops for EPA staff as well as joint workshops including both EPA and state staff. We also participated in the national NEPPS work group conference calls. 9 ------- |