Ui ,r,ø St.,.. £nviro,,.r .., Pro ciien Aq CY Oflc. of So u a$Is Ern.rg.r Rsioos. &EPA IiVPP t)’ 1fli P ( QIfl! D DIREC TIVE DIREC TIVE /3 DIRECTIVENUMBER. 954 5.00-6A TITLE: RCRA Program Evaluation Guide APPROVAL DATE: 08/11/88 EFFECTIVE DATE: 08/11/88 ORIGINATING OFFICE: Office of Solid Waste FINAL DRAFT j A— Pending O? approval STATUS: [ J B— Pending AA—OSWER approval [ C— For review &for coent [ ] D— In development or circulating REFERENCE (other documents): headqua r g “RCRA Evaluation Guide” (OSWER Directive 119545.00—6) ------- United States nvironmen1ai Protection Agency — EP 1 A Washinglon DC OSWER Directive Initiation Request I Directive Number 9545 . 00 - k’ 2 Or1ç thator information Name of Contact Person IM Code I0 e ITelephone Code Kathy S. Margolis WH—563 OSW I 382—2232 3 Title RCR Program Evaluation Guide 4 Summary of Directive (include brief statement of purpose) This revised evaluation guide establishes procedures and guidelines to assist EPA Regions in conducting State program reviews. This evaluation guide establishes minimum requirements applicable nationwide in evaluating the quality of State hazard- ous wast programs. 5 Keyworøs Program Evaluation I Oversite / RCRA / Program Capability / State Program 6a Does This Directive Superseoe Previous Directive(s)” No XX Yes What directive (number title) # 9545.00—6 b Does It Supplement Previous Directive(s)” XX No Yes What directive (number title) 7 Draft Level A — Signed by AND&A B — Signed by Ofl ce Director C — For Review & Comment 0 — ri Development 8. Document to be distributed to States by Headquarters? Li Yes No This Request Meets OSW R Directives System Format Standards 9 Signature of Lead office Difectives oorthnator\J k . U, 10 Name and Title of Approving Official Date Date 8/26/88 J. Winston Porter, Assistant A iiinistrator, OSWER 8/11/88 EPA Form 1315-17 (Rev. 5—87) Previous editions are obsolete Os VE WE DI R Os RECTIVE WER DI Os RECTIVE WER DI 0 RECTIVE ------- OSWER Directive #9545.OO-6A D Sr 4 ,. UNITED STATES ENVIRONMENTAL PROTECTION AGENCY _____ WASHINGTON, D.C. 20460 4 ‘ t p o AUG I ‘ ic c OFFICE OF SOUD WASTE AND EMERGENCY RESPONSE MEMOR.AN DUM SUBJECT: The 1988 Revised RCRA Evaluation Guide (OSWER Directive #9545.00—GA) FROM: J. Winston Porter ACLJ ’ Assistant Administra lid aste and Emergency Response TO: Regional Administrators Regions I—X In December 1984, the FY 1985 Edition of the RCRA Evaluation Guide was released. The guide was developed by a special Task Force to provide a reference point for implementing national requirements and to provide protocols for conducting State program evaluations. Since the guide’s release, the Regions have continued to develop valuable expertise in State program oversight. This expertise has been tapped to provide revisions to the FY 1989 edition of the guide. At the Regions’ request, the RCRA Evaluation Guide has been modified and restructured to provide a more usable guide to ensure nationally consistent State program evaluations while allowing flexibility to address specific areas of concern and allow for appropriate levels of differential oversight. The attached Guide is provided in two documents, the revised guide and Appendices. I hope that this revised guide proves useful in conducting RCRA State program performance evaluations and in negotiating annual State evaluation protocols. Attachments cc: Waste Management Division Directors (no attachment) RCRA Branch Chiefs (no attachment) State Authorization Section Chiefs (attachment and extra copies) ------- ‘.. ,-. --‘ - : .4—. ‘ t __ u- EvaluationGuide. ------- 9545.OO-6A RCRA PROGRAM EVALUATION GUIDE T&BLE OF CONT1 NTS SECTION TITLE PAGE INTRODUCTION . . . 1 Purpose and Use of Guide . . 1 Relationship to Other Guidance and Policy . . 2 The RCRA Oversight and Evaluation Process . . S II REVIEW PROCEDURES . . . 9 Conducting Program Reviews . . . . . 9 A. Pre-meeting Preparation/Planning Session(s). . 10 B. On-site Review . 11 1. File Reviews . . . 12 2. Introductory Session 13 3. Discussion of Program Elements 13 4. Exit Briefing 14 C. Written Report 14 D. Follow-up Actions . . 16 Elements Common to a Successful Review . . . . 16 III PROGRAM REVIEWS. . . . . 18 A. Permit Program . 18 B. Compliance Monitoring and Enforcement Program. . . 21 C. Closure Progran 26 D . Corrective Action Program . 28 E. Management Program . 31 IV GUIDELINES FOR EPA RESPONSE TO STATE PERFORMANCE 36 ------- 9545 .OO-6A -1- SECTION I: INTRODUCTION In December of 1984, EPA released the FY 1985 edition of the Resource Conservation and Recovery Act Evaluation Guide. This document was developed as a tool to incorporate the existing Interim National Criteria for a quality }lazardous Waste Management Program under RCRA into EPA’s oversight activities. The original guide was a reference point for implementing national requirements and to provide protocols for conducting State program reviews. Since this document’s release, the Regions have continued to develop expertise in overseeing and implementing quality RCRA programs. Although review and oversight of the RCRA program must be tailored to specific Regional/State situations, it is clear that a baseline exists for conducting a quality evaluation. In order to ensure nationally consistent RCRA program evaluations and in order to allow enough flexibility to address specific circumstances in those evaluations, the RCRA Evaluation Guide has been modified and simplified. The revised RCRA Program Evaluation Guide emphasizes the process for conducting program evaluations. The Agency has res.tructured this revised guide to make it more usable for the Regions in evaluating State programs. The discussion of national requirements (previously Section 2) is now included as part of the discussion of program goals for each program element in Section III. In addition, the previous -Section 3 (Incorporating National Requirements into Grant Workplan) is incorporated into the Introduction. This new guide includes a separate section (II) that sets forth appropriate procedures for conducting program reviews. Section III now outlines the review for each RCRA program element and presents general questions representing the minimum requirements for both the pre-meeting preparation/planning session and the on-site review. The revised RCRA Program Evaluation Guide is presented in two volumes. The first volume includes an Introduction, Review Procedures, Program Reviews, and Guidelines for EPA response to State performance. This volume is designed to stand alone when used to develop a State Program Review. The second volume is a group of five appendices that include more detailed checklists for each program element. Although this second volume is not required to develop a program review, it i provided for those who desire a more detailed approach. PURPOSE AND USE OF GUIDE This revised evaluation guide establishes procedures and guidelines to assist EPA Regions in conducting State program reviews. The requirements summarized in the Guide apply equally to the EPA Regions and States The Guide may be used both to oversee States with RCRA program grants - - whether authorized or under cooperative arrangements - - and to evaluate the Regions (where States are not authorized) ------- 9545 .OO-6A -2 This evaluation guide establishes minimum requirements applicable nationwide in evaluating the quality of State hazardous waste programs. This document incorporates the program goals and key questions that are described in the National Criteria for a Quality Hazardous Waste Management Program Under RCRA (1986). In addition to presenting suggested procedures for conducting State program reviews, this document provides detailed Appendices which provide a number of useful checklists and questions for use in conducting evaluations. It is important to note, however, that EPA Headquarters and the Regions are not required to follow the questions exactly as contained in this evaluation guide. The questions are intended as guidance for developing a review tailored to the individual Region/State relationship and the status and characteristics of the State programs. It is clearly the responsibility of the Region and the State to develop specific oversight procedures. 40 CFR §35.150 states, in part, “. . . In consultation with the applicant, the Regional Administrator will develop a process for evaluating the recipient’s performance. This document, however, aims to ensure a nationally consistent framework by which to evaluate the quality of State hazardous waste programs by: • Presenting program review procedures each Region should follow (Section II); • Illustrating model evaluation questions that incorporate the quality criteria and related requirements (Section III); and • Outlining appropriate EPA response actions to State performance (Section IV). RgLATIONSHIP TO Qr I GUIDANCE AND POLICY The guide will be revised or supplemented as necessary to reflect emerging program priorities and new policies identified in the Agency Ooerating Guidance , the RCRA Inmlementation Plan and other operating guidance. It may also be revised periodically as experience is gained in its use. The RCRA Program Evaluation Guide is integrally related to a number of existing Agency guidance and policies. To conduct a fully successful RCRA program evaluation, the following documents should be used as a part of the evaluation process. • AgencY Operating Guidance . This document sets the policy and planning directions of the Agency in a given fiscal year. Based on this document, the annual RCRA Implementation Plan and the goals for the Strategic Planning Management System are established. ------- 9545 .OO-6A -3- • °The RCRA Implementation Plan (RIP) , issued annually. This document provides a summary of the program priorities, reporting requirements and forms, and targets for Region/State grants management. It cross-references virtually all the related documents affecting fiscal year planning and oversight. • The National Criteria for a Oualitv Hazardous Waste Management Program Under RCRA , issued in revised form in 1986. This document defines the key questions and performance expectations for managing and evaluating the RCRA program under existing regulations and policies. It is multi-year in scope. The quality criteria form the basis for the grant work agreement, program element format, program reviews and other oversight tools. • Strategic Flannina and Management System (SPMSI , conducted annually. This process defines the RCRA measures, commitments, and reports that senior Agency management will use to evaluate progress in overall program implementation. It establishes quarterly reporting requirements for EPA program offices and the Regions, and establishes achievement targets for some of these requirements. • National. Permits Strategy , revised August 1984. This document provided national direction for developing individual Region and State multi-year strategies for addressing the permit workload. • RCR.A Enforcement ResDonse o1icv (ERPI , revised in 1987. This . document defines a classification scheme for identifying and reporting violations, describes timely and appropriate responses for each violation type, and outlines various levels of program response to bring violators back into compliance. • RCRA Ground-Water Monitoring Enforcement Guidance , August 1985. This document includes both the RCRA Ground-Water Monitoring Compliance Order Guidance and the Draft RCRA Ground-Water Monitoring Technical Enforcement Guidance Document . These documents provide comprehensive guidance on how to identify and rectify ground-water monitoring violations. ------- 9545 .OO-6A -4- Late and Incomnlete Part B Policy. This policy memorandum outlines appropriate enforcement procedures (e.g., notices of deficiency, warning letters, compliance orders, etc.) to deal with facilities submitting late and incomplete Part B permit applications. • State/EPA Memorandum of Agreement (MOkl , for authorized states. These documents are negotiated agreements between EPA and authorized States, and set forth the respective roles and responsibilities of each agency in administering the RCRA program in that State. • Azencv Policy Framework for State/Federal Enforcement Agreement , issued June 26, 1984. This document provides Agency-wide requirements for developing enforcement agreements and overseeing authorized enforcement programs. • Guidance on Developing RCRA Coumliance/Enforcement Strategy , issued on June 12, 1984. This document provides national direction for developing individual Region and State multi-year strategies for compliance monitoring and enforcement. • Capability Assessment Guidance , memo andum, signed April 8, 1987. This memorandum describes the procedures the Regions should follow in conducting capability assessments of State programs, prior to approving applications for state authorization for HSWA provisions. • Protocol for Evaluating Interim Status Closure/Post- Closure Plans , issued August 1986. This document provides detailed checklists and forms for evaluating closure/post- closure plans for completeness, procedural correctness, and consistency with regulations. • RCRA Permit Quality Protocol , issued August 1986. This document provides detailed checklists for evaluating the completeness, consistency with regulations, technical soundness, and enforceability of RCRA permits. • EPA Policy on Performance-Based Assistance , May, 1985. This document outlines procedures for releasing funds to a State in consideration of performing deficiencies. ------- 9545 .OO-6A -5- • RCRA State Oversight Inspection Guidance , December, 1987. This document provides a uniform procedure under which State oversight inspections should be performed and promotes a nationally consistent approach to evaluating State inspection performance. The interrelationship of these documents and the process for oversight planning, monitoring and evaluation, and feedback and follow-up is detailed in Exhibit 1-1, RCRA Oversight Process Flow Chart. ‘filE RCRA OVERSIGHT AND EVALUATION PROCESS The RCRA oversight and evaluation process relies on the guidance, policies, and negotiated agreements cited above. The process can be approached through a three-tiered planning and evaluation structure. Oversight planning; monitoring and evaluation; and feedback/follow-up response comprise the three basic stages in the process. Each stage is discussed in more detail below, and is illustrated in Exhibit I-i. Oversight PlnTming The RCRA oversight process begins with the issuance of the “Agency Operating Guidance,” and its supplement, the “Strategic Planning and Management System” (SPMS). The operating guidance establishes broad goals and objectives for all EPA programs, while SPMS details specific accountability neasures. Based on these documents, the RCRA national program issues the annual “RCRA Implementation Plan” (RIP), which describes the national program’s strategy for implementing the Agency’s operating guidance. This is the basis for both Regional development of its own commitments under SPMS and the guidance it provides to the State for program grant planning. In conjunction with the development of Regional commitments, the Region and State develop the annual State grant work agreement. This document, which establishes annual State commitments and performance expectations, should reflect the grant guidance and the MOA, as well as the RCRA Quality Criteria and RIP. All RCRA State grant work agreements should be performance-based. This approach improves the accountability of the State program by expressing commitments on a quarterly basis, and by explicitly identifying the resources associated with those commitments The emphasis on funding and commitments also aids the States in the management of their program by encouraging close supervision of resources and the tracking of outputs ------- Exhibit 1.1 RCRA Oversight Process Flow Chart Oversight Planning Program Goals Negotiation of Issue Obligate Complete Established Commitments Performance— Grant Development Based Grant Funds of Evaluation • Agency Operating • Regional SPMS Protocol Guidance Commitments p.,. • RIP • State Grant Work Agreement • SPMS Monitoring Monthly Monitoring Mid— and End-of-Year Reviews and Evaluation (HWDMS) • Program Reviews — All Elements • Compliance and Enforcement —. •State Grant Work Logs Agreement Review • Permit Activities • File Review Monthly Status Report • MOA Review -4 _ I Feedback and Issue S —l j Showing • Transfer Good Ideas to Other States and Regions lZea sPublicize State Program and Accomplishments Follow-up Program Re- I view_Re I ri Exemplary •Tectinical and Financial Assistance for State Special or _______ I rmance Innovative Projects - Feedback Into ___________ Later Reviews Response EPA • Provide Technical Assistance Year Planning State Response • Suggest Minor Change in State or Regional Procedure and Future • Increase Reporting Frequency/Level of Oversight •Ralse Performance Issue to Higher Level Management For Arees I S Revise Future Work Program • Explanations • Corrective L .I Needing -‘ • Greatly Increase Oversight Actions i Improvement I • Negotiate “Corrective Action Plan” •Feedback Ofl i •Revoke Letter of Credit/Institute Reimbursable Grant Regional •Postpone Release of Funds, or Deobligate Performance • Initiate Program Withdrawal S82065— lb ------- 9545 .OO-6A -7- The State grant work agreement summarizes the activities the Region and State agree should be performed by the State during the fiscal year for which the grant is awarded. It also indicates the level of grant resources to be devoted to each of the various activities. As such, it is an excellent tool for establishing and documenting how a State will incorporate program priorities and guidance into their operations, and how the Region will support and oversee such activities. It is also a tool that can serve as the basis for any grant sanctions that may be imposed as a result of non-performance. Conditions should be established in the State grant work agreement to address areas of poor.performance during the preceding year, activities mandated by the State capability assessment, and procedures for release of grant funds and adjustments to funding in the event of non-performance. Monitoring and Evaluation The RCRA monitoring program and evaluation activities include monthi monitoring through use of the Hazardous Waste Data Management System (HWDMS) and other reports, as well as program performance reviews (mid-year, end-of- year, and quarterly/monthly, if necessary). Throughout the year, oversight inspections, record reviews, and permit reviews are also conducted to complement State monitoring and evaluation activities. In addition, less formal monitoring and evaluation activities, such as routine meetings, monthly permitting and enforcement conference calls, and telephone conversations with State personnel, can and should be used to enhance program monitoring and evaluation. Feedback and Follow-up Response The final part of the RCRA oversight and evaluation process involves feedback on program performance. It is critical that feedback be a two-way street, with the objective being to improve future performance by both the Region and the State. The States receive informal feedback from the Regions throughout the year and formal feedback by means of the program review reports prepared by the Regions to describe the findings of program evaluations. When program performance has been exemplary, the Regional response may include publicity for State accomplishments, and technical or financial assistance for special State projects as appropriate and available. A new data system called Resource Conservation and Recovery Information System (RCRIS) is currently being field tested Over the course of the next year, HWDMS will be phased-out and RCRIS will take its place This change will improve our reporting efficiency and enhance our ability to analyze data ------- 9545.OO-6A -8- When program performance shows need for improvement, Regional responses may range from increasing the level of monitoring and evaluation to providing technical assistance, changing grant pay-out procedures, negotiating a “corrective action plan”, or, in the most extreme cases, initiating withdrawal of program authorization. ------- 9545.0O-6A -9- SECTION II: REVIEW PROCEDURES EPA is required under 40 CFR 35.150 to review the States’ performance toward completing the outputs agreed to in the State grant work agreement. This section provides procedures for conducting these required program reviews followed by a list of elements common to a successful review process. A detailed description of the recommended format for the program review report is also provided. CONDUCTING PROGRAM REVIEWS The on-site program review provides a forum for a face-to-face detailed review and discussion of a State’s performance in meeting established goals and objectives in the State grant work agreement. The RCRA Implementation Plan (RIP) requires that at least one on-site program review be conducted each year. Most Regions conduct two reviews: mid-year and end-of-year. Additional evaluations, usually less formal, should be conducted as the budget allows, or if a State’s performance warrants increased oversight. For those States which have Regional offices, these additional reviews are recommended. Mid- and end-of-year reviews should be conducted as soon after the period being evaluated as possible. Whereas lags in data entry may necessitate a delay in an immediate evaluation, they should be conducted as soon as po sib1e and no later than 45 days from the end of a review period. This will help expedite the release of the final report and the initiation of steps to correct and program deficiencies. Mid- and end-of-year reviews will usually require a minimum of two working days on-site. The reviews should be comprehensive, detailed evaluations of all aspects of a State’s performance. Additional time will be required if file audits are also scheduled. Whenever possible, State performance commitments should be directly incorporated into the State grant work agreement. However, there are other existing mechanisms utilized by Regions and States. In conducting a program review, all State/Regional commitments should be considered, including the following: • State grant work agreement • Memorandum of Agreement (MOA) • Letter of Intent/ ”Corrective Action Plan” (where appropriate) • State-specific implementation strategies • Other EPA/State agreements ------- 9545 .O0-6A - 10 - The National Criteria for a Quality Hazardous Waste Management Program Under RCRA (revised 1986) serve as the basis for determining the adequacy of a State’s/Region’s performance. Exhibits in Section 3 of this Guide (revised as necessary for individual Region/State ituations) should be used for file reviews, analysis of individual State outputs, etc. Creating a program review staff with an appropriate skill mix is crucial in preparing for a successful review. Conducting a comprehensive program review requires a team approach; that is, input from all program elements is necessary. A review team should consist of individuals who are familiar with the State’s performance in each of the following areas: enforcement, permitting, closure, corrective action, regulatory development, reporting/data management, and resource utilization/program management. Consideration should be given to OSW and OWPE participation in on-site reviews, particularly in States having persistent performance problems and/or in those States for which a capability assessment is pending or underway. The Grant Project Officer responsible for the particula State or a suitable supervisor from the Regional Office should serve as team leader for the review. The actual evaluation process consists of four major components: preparation/ planning session(s) prior to the on-site review; the actual on-site review; a written report summarizing the findings and observations of the team with recommendations for resolving problems; and follow-up actions. A. Pre -meeting Preparation/P1RI1nI ng Session(s) In order to minimize the time spent on-site, it is essential to have careful planning and preparation prior to the actual State program review. If the Region and the State have established oversight mechanisms, and have routinely utilized them, the on-site review should hold few, if any, surprises. Regions already receive a large amount of information from the States in the form of State monthly reports, permit reviews, oversight inspections, copies of land disposal facility documents, etc.; therefore, most of the quantitative information and significant portions of the qualitative information can be compiled prior to the on-site review. However, since the review draws upon input from numerous sources and will be time-intensive for many State and Regional employees, it must be carefully organized. The entire review team should meet at least once approximately three to four weeks prior to the on-site review to develop an agenda, define individual roles or responsibilities, to discuss the State’s progress in performing casks established in the State grant work agreement and in other mechanisms (MOA, etc ), to determine the State files to be reviewed, and to discuss preliminary ------- 9545.00-6A - 11 - ratings for each program element. 2 Where 05W and/or OWPE personnel plan to participate in the on-site review, their roles should also be discussed in advance. The team should review the last written evaluation, in order to verify that recommendations for correcting previously-identified deficiencies have been implemented. Individual team members will have completed checklists and prepared summaries of performance (where possible) for discussion at the planning session(s). Where specific deficiencies (or potential deficiencies) are identified, the team should prepare recommendations for correcting these deficiencies. As far as possible, potential conflicts in performance ratings should be resolved prior to the on-site review. It is important that input from the State be sought at this early stage. What issues would they like to have included on the agenda? Have they identified specific guidance or technical assistance needs? Do they wish to request the attendance of specific personnel from the Region or Headquarters, for clarification or interpretation of a particular policy? Is there an impending change or ituation of which the Region needs to be aware, and for which careful planning is needed? At least two weeks prior to the scheduled review, the Region should send the State the following: a) an agenda for the review, including items requested - by t ie State; b) procedures/checklists for conducting the review; c) a list of specific issues/questions related to each program element or commitment being evaluated; and d) a roster of EPA participants. A list of files to be reviewed should be provided to the State at least 10 days prior to the scheduled review date. Whenever possible, the Region should also provide the State with completed checklists and/or summaries of performance prior to the review to allow the State time to prepare responses. State personnel familiar with the various aspects of the program and specific issues should be available to meet with the review team. State personnel should be prepared to provide the review team with copies of any documents which the team needs to document accomplishments or clarify issues raised during the review. B. On-Site Review The on-site review will require two to five working days, depending on whether file reviews are to be conducted, the amount of issue-resolution required, and other factors. It is important that sufficient time be allowed for detailed and substantive discussion of issues, deficiencies (if any), and 2 Although not all Regions currently give specific ratings for each program element, a basic decision on the adequacy of the State’s performance in each area should be made as a part of the evaluation process. ------- 9545.OO-6A 12 - recommendations. The Region should also use this opportunity to receive feedback on its own performance, to identify training and/or guidance development needs, and to plan for the future. It is essential that the on-site review team consists of individuals with expertise in compliance monitoring and enforcement, permitting, closure/post- closure, corrective action, and program management. File reviews of each of these program areas should be performed by the member of the review team who has expertise in that area. The actual on-site review will generally proceed as follows: a. File reviews (as necessary) b. Introductory session c. Discussion of program elements d. Exit briefing In addition, the Region may wish to augment the on-site review by encouraging the participation of State program personnel in the review process. For example, the Region could recommend that a State conduct its own evaluation prior to the on-site review. Oftentimes, an internal State program review enables State managers to verify EPA’s findings. Alternatively, the Region could request an experienced State program staff person to accompany the EPA review team. The State participant would be involved throughout the information gathering process and would be able, to verify the accuracy of information collected in addition to the validity of the corresponding program review’s conclusions. Both these processes may prove effective in strengthening Regional findings. 1. File reviews . The Region should conduct a review of State files on a regular basis, and should ensure that file reviews are completed and summarized prior to program reviews so that their results may be used during the program review. Many Regions find it necessary to conduct file reviews as part of the on-site program review, to minimize travel time. In such cases, the file reviews should be scheduled at the beginning of the visit, to allow sufficient time to incorporate the results into the review. File reviews should be conducted for all aspects of the authorized program: compliance monitoring and enforcement, permitting, closure/post-closure, corrective action and program management. The percentage of file reviews conducted and the selection of files should be based on criteria established during the grant negotiation process. The extent of file review coverage is based on Regional office needs for effective oversight arid may vary annually The RCRA Permit quality Protocol (August 1986) and the Protocol for Evaluating Interim Status Closure/Post-Closure Plans (August 1986) are useful tools for conducting file reviews Both protocols provide detailed checklists useful for evaluating permits arid closure/post-closure plans for completeness. procedural correctness, consistency with regulations, and enforceability ------- 9545.OO-6A - 13 - If file reviews are to be performed at the same time as the on-site review, the first two or three days should be set aside for the file reviews. The file reviews are primarily the responsibility of the appropriate program staff (e.g., enforcement, permitting); however, the entire team may participate. During the file reviews, the Regional review team should interact with State staff to clear up any minor questions which may arise. A representative number of each type of file should be reviewed in order for the Region to evaluate the quality and timeliness of the State’s activities. The State grant work agreement usually defines the number and type of files to be reviewed, the number and type of oversight inspections to be conducted, and the extent of record reviews to be conducted at the State. Where possible, the Region will already have completed its review of documents routinely sent from the State to the Region (e.g., all land disposal files) prior to the on-site visit. It is important that the State receive a list of files to be audited prior to the review, so that all necessary materials are r dily available. This is particularly critical when the State has field or district offices which maintain records. In such a case, the Region might consider doing an occasional file review or portion of a program review in a district office. If the file reviews are conducted as par; of the on-site review, the Regional review team should meet following the file reviews and prior to meeting with the State. The purpose of this meeting is to discuss results of the file reviews, to determine the impact of the preliminary performance rating(s), and to define roles/responsibilities for communicating the file review results to the State. The technical specialist or supervisor for that particular area usually assumes this responsibility. 2. Introductory session . The opening session of the on-site program review should include introductions, and a general overview of how the review is to proceed. The ground rules for the process (including roles/format/time frames) should be understood by all parties, and agreement reached on any last minute adjustments Co the agenda. 3. Discussion of program e1e ents . Through staff interviews and roundtable discussions, all program elements and objectives in the State grant work agreement and other commitments are discussed. The State’s commitments should be briefly summarized, then progress towards meeting them discussed in detail in terms of quality, quantity, and timeliness of the outputs. These discussions should serve to fill in information gaps and to verify information collected in advance of the review. The Region and State should discuss successes and deficiencies with emphasis on identifying the reasons for successes and developing alternatives for correcting problems that exist. The Region and State should also discuss the work years used to date and, if required, the need for reprogramming grant funds The Region should seek feedback on its own performance as each program element is discussed, and together the Region and the State should identify training and guidance development needs At the conclusion of the discussion of each major program ------- 9545.OO-6A - 14 - element, the team should meet with key State staff and managers to discuss the team’s findings and observations. 4. Exit briefing . There should be an exit briefing on the last day of the review. At that time, senior Regional managers or agreed-upon representatives will meet with senior State agency managers, as well as key State hazardous waste personnel, to discuss the major findings and observations of the review team. Any verbal agreements reached between the Region and the State should be reiterated. It is important that managerial personnel from both the Region and the State participate in this exit briefing, so that all are aware of the issues raised and the recommendations discussed. In this way, recommendations for corrective action can be implemented immediately, rather than waiting for the issuance of the written report. C. Written Report Following the on-site program review, the Region should prepare a written report which summarizes the findings obtained through all oversight mechanisms: program reviews, file reviews, State monthly reports, oversight inspections, document reviews, permit reviews, etc. The Project Officer is responsible for preparing the draft and final report with input from all members of the review team and other appropriate personnel. The report should include findings and observations on all program elements and objectives, an evaluation of the State’s progress in performing those tasks, and specific recommendations for improving program performance, where necessary. The report should summarize major points, and detailed charts, graphs, and matrices should be included as attachments. Information included in the attachments should document State accomplishments and deficiencies. Regions should use standardized attachments to assist EPA in evaluating consistency among State programs. Exhibit 11-1 provides a suggested format for the program review report. The Region should work to finalize the report as expeditiously as possible, through a schedule previously agreed upon by the Region and State. The Region should send a draft report to the State, usually within 15 calendar days of the review, or as otherwise agreed upon. Again, it is critical to finalize the report as soon as possible so that the overriding goal of the review, the correction of program deficiencies found, can be efficiently met. The Region should provide the State with an opportunity to comment on the draft report, correcting any factual errors and clarifying any issues and recommendations for follow-up actions. The State should respond within 15 calendar days from receipt of the draft report, or as otherwise agreed upon. A final report is then issued, incorporating and responding to the State’s comments, as appropriate. The final report should be issued within 15 calendar days from receipt of the State’s comments, or as otherwise agreed upon ------- 9545 .OO-6A - 15 - HIBIT 11-1 PROGRAM REVIEW REPORT FORMAT A. Executive Si ” ry: The first page or two of the report should present a brief Executive Summary. This summary is the portion of the report most frequently read by others, including senior managers at both the State and the Region. It should be concise, emphasizing the major findings an recommendations of the program review, and highlighting significant issues and areas of concern. Positive aspects of a State’s performance should also be highlighted. Information should be presented in “bullet” format, with reference to the specific page or section of the report where that particular issue is discussed in more detail. B. Introduction: The introduction to the review report should be very brief. It should state the objective of the review, identify the period being evaluated, and offer a list of the State and Federal participants in the review. C. Findings and Observations: A narrative discussion of the State’s performance under each program element is presented in the body of the report, and should include the following: • A brief statement describing the State’s commitment in terms of work years allocated, type and number of outputs, and timeframes for submission. • A summary of State performance to date, including quality of the work, timeliness, and utilization of work years. • A discussion of State methodologies, procedures, structures, etc., which contribute to successes or deficiencies. This discussion should serve to substantiate the “Findings and Observations”. D. Conclusions: This section should contain the following: • An assessment of State performance, specifically, a brief analysis of problems, why they occurred, and a discussion of alternative solutions. • A summary of any agreements reached between the RegLon and the State during the on-site review, commitments and/or recommendations made, and timeframes agreed upon for resolution of issues. • A discussion of work years earned or reprogrammed, if necessary • A closing statement which makes a judgment call as to the State’s performance ------- 9545 .OO-6A - 16 - D. Follow-up Actions In addition to issuing the written report, it is critical that Regions/States establish tracking systems to ensure that all recommended actions are completed on schedule. It should also be emphasized that steps to initiate actions to correct program deficiency can be taken prior to the issuance of the final report. To track progress on recommended actions, many Regions have monthly conference calls with the States. The Region should be prepared to take an escalated response action (according to predetermined guidelines) if recommendations are not followed. EL TS CC MON TO A SUCCESSFUL REVIEW Each Region should tailor the structure of the RCRA program review to its own operating procedures and unique Region/State relationship. However, certain elements are coon to a successful process: • The performance expectations underlying all program review questions should be clearly defined in the State grant work agreement, MOA, or other Region/State agreement. • Since the review is concerned with program performance, it must consider both State and Regional performance of responsibilities (i.e., technical support, training, etc.); follow-up actions should be developed not only for the State, but for the Region as well. • The purpose of the review should be to improve future program performance, with the emphasis on constructive feedback to identified problems, rather than on sanctions. However, Regions/States are expected to deal firmly with persistent problems, taking whatever measures are necessary and appropriate to remedy performance. • A major objective of the process is to identify performance issues early and to take steps to correct problems that would otherwise prevent implementation of a quality program. • The review should also recognize program accomplishments, with an eye toward the transfer of good ideas, procedures, etc. to other States/Regions. • Program review findings should be viewed relative to past performance and the importance of the review area Apparent successes or failures should be assessed according to whether actual improvements or slippage in performance have occurred. ------- 9545. OO-6A - 17 - • All overs ight tools used throughout the year should be centrally coordinated so that information for evaluation is collected once and used consistently. • Evaluation reports for all States should be centrally coordinated and follow a standardized format to establish consistency within the Region. ------- 9545.OO-6A - 18 - SECTION III: PROGRAM REVIEWS A. PERMIT PROGRAM Program Coals The permit program’ s primary goal is to establish increased regulatory control over the design and operation of hazardous waste treatment, storage, and disposal facilities. One of the key objectives is to issue quality, enforceable permits to land disposal and incineration facilities in accordance with the statutory deadlines and to environmentally significant treatment and storage facilities. To achieve this goal, the permit program requires the aggressive use of enforcement authorities to obtain compliance with all Part B informat ion requirements. The permit program also plays a critical role in ensuring future storage, treatment, and disposal capacity and alternatives to the land disposal of hazardous waste. The permit program is placing increased emphasis on processing new facility applications, research, development, and demonstration permits (RD&D) and permit modifications. Permit modifications are becoming in creasingly important, since many address schedules of compliance for corrective action. In addition, the permit program will include increased requests for changes to existing interim status facilities for the expansion of storage, treatment and incineration capacity. Pre -neeting Preparation/P1A T ng Prior to the on-site review, the Region should organize the appropriate permit staff to evaluate the status and adequacy of multi-year permit strategies and MOAs. This preliminary review will assist Regional staff in determining the need .for .revisions/adjustments to projected accomplishments and changes in the division of permitting responsibilities. In addition, the appropriate permit staff should evaluaçe the latest monthly HWDMS reports to determine how the State is progressing in meeting its quarterly commitments. A review of HWDMS data should also highlight potential discrepancies or areas for follow-up during the on-site review. Individual facility management plans, if complete and up-to-date, may also be used to evaluate State progress on specific permit actions. The following questions are for use in the pre-planning stage and address the issues discussed above A more detailed checklist is found in Appendix A. ------- 9545 .OO-6A - 19 - MULTI -YEAR PVJ MTT STRATEGY • Has the State developed a multi-year permitting strategy? • Does the strategy address all environmentally signific ant facilities? • Does the strategy require final determinations for existing facilities in accordance with national priorities? • Does the strategy include commitments for intermediate milestones in the permit process? • Is the strategy periodically reviewed and/or updated to address changes in facility status? HW • Are the schedules agreed upon in the State grant work agreement for Part B call ins, permit processing and final determinations being met? • For each facility type, what is the progress of permit issuance to date? • Is the State issuing Notices of Deficiencies (NODs) for Part B applications within a timely manner? • Is appropriate follow-up action being taken (i.e., permit denial, enforcement action) if permit deficiencies are not resolved within appropriate time frames? o What is the status of permit modifications? Are modifications being issued for corrective action activities? o Has the time period for processing permits improved for permits issued this year versus prior years? ------- 9545. OO-6A - 20 - On-site Review The focus of the on-site visit is on reviewing State permit procedures for quality and completeness. File reviews and staff interviews are the primary information sources for the on-site visit. The field review should evaluate permits for clarity and enforceability and determine whether all procedural and substantive requirements are being fulfilled. For example, incorporation of public comments into final decisions is an important objective of the permit program and should be covered in the review. The Region should also review the State’s record in winning permit appeals. The ability of the State to have its final determinations upheld on technical, legal and procedural grounds may be a measure of permit quality. The Office of Solid Waste (OSW) developed the RCRA Permit Quality Protocol (August 1986) to assist the Regions in evaluating a permit for completeness, procedural correctness, and consistency with regulations. In addition, this guidance provides for the review of a permit’s technical soundness, and enforceability. This guidance provides detailed checklists and forms to be filled out by the reviewer. These forms may be used during the on-site file review. - Finally, meetings with appropriate staff shpuld be used to discuss problem areas and coordination of the permit process with other State program elements. The following questions address the primary issues covered during the on-site review. A more detailed checklist is found in Appendix A. FILE REVIEW • Are permit files and administrative records complete with all necessary correspondence and documentation? • Are permit conditions clear, enforceable and properly documented? • Do the permits fully describe and define requirements and frequencies for facility monitoring, reporting, inspection, and analyses after permit issuance 9 ------- 9545 .OO-6A - 21 - FILE REVIEW (cont’ d) • Are all regulatory requirements for public participation being met? • Do final permit decisions consider all relevant public comments received on the draft permit? • What is the status of permit appeals? Are permit determinations being upheld on technical, legal, and procedural grounds? B. COMPLIANCE MONITORING AND FORC (EWT Program Coals The goal of the compliance monitoring and enforcement program is to ensure an improving level of compliance withi ’h the regulated community and to discourage non-compliant behavior through the use of formal enforcement actions and penalty assessments. In accordance with the time frames described in the revised Enforcement Response Policy (ERP), the Regions and States should initiate appropriate enforcement action and assess penalties against violators. The compliance monitoring and enforcement program is shifting its focus from enforcing pre-HSWA interim status requirements to enforcing requirements in permits and closure plans. In addition, the program will show increased emphasis on compelling corrective actions and enforcing against schedules of compliance to implement corrective measures in accordance with approved plans. Furthermore, the compliance monitoring and enforcement program will ensure compliance with all mandatory inspections and HSWA requirements, in particular, the land disposal ban. Pre -meeting Preparation/Planning To minimize the time spent on-site, it is important to compile and analyze all relevant information before the on-site review The State Compliance Monitoring and Enforcement Strategy should be reviewed for consistency with the National Strategies. The strategy should include procedures for inspections, record reviews, and enforcement actions and should be updated as necessary Similarly, the Region should review the State/EPA Enforcement Agreement for appropriate division of responsibilities and oversight ------- 9545 .OO-6A - 22 - requirements. Another important information source is the monthly HWDMS reports. HWDMS data quantitatively describe whether the State has met its commitments for performing inspections and record reviews, and whether the State has been successful in bringing the regulated community into compliance HWDMS data will also help in determining whether enforcement actions have been timely and appropriate. Any conclusions made from Regional oversight inspections and oversight record reviews should be summarized during the pre- planning stage, and used later in the on-site review to support the results of the file reviews. Below is a series of questions that should be asked to assess the compliance monitoring and enforcement program at the pre-planning stage. For a more detailed checklist which expands on these questions, see Appendix B. STATE COMPlIANCE MONITORING AND YORC 1T STRATEGY/ 1YORC T AGRRPX NT • Dies the State Compliance Monitoring and Enforcement Strategy include procedures for: (1) Conducting all mandatory inspections in accordance with the RIP? (2) Taking timely and appropriate enforcement response action? (3) Completing record reviews of all handlers? (4) Classifying violations? (5) Inspecting all TSDFs in conjunction with permit applications? (6) Identifying non-notifiers and handlers operating without permits or manifests? (7) Verifying that facilities have closed in accordance with approved closure and post- closure plans, and that requests for withdrawal are valid? ------- 9545 .OO-6A - 23 - STATE COMPLLANCE MONITORING AND ENFORC fENT STRATECY/ENYORC cENT A RE ENT (cont’ d) • When was the strategy last updated to address new program priorities? • Does the State/EPA Enforcement Agreement contain clear oversight criteria, clear criteria for direct EPA enforcement in authorized States, and adequate provisions for state reporting of enforcement activities? • Is the State meeting its inspection and record review commitments in accordance with the RIP State grant work agreement and statutory requirements? • Are enforcement actions timely and appropriate? Does the State respond to every instance of known non-compliance and issue penalties to all high priority violators? Are penalties commensurate with violations according to the State’s Penalty Policy or more than de minimus if there is no State penalty policy? • What is the compliance rate of various types of handlers? How have compliance rates changed over time? Has the State been successful in bringing handlers into compliance? ------- 9545 .OO-6A - 24 - On- site Re’ci.ew In evaluating the compliance monitoring and enforcement program, file reviews are the most important source of information during the on-site review. File reviews should focus on assessing the quality of enforcement files and various documents including CMEs, GEls, record reviews, and administrative orders. All enforcement actions should contain clear, specific language, and documentation should be sufficient to support the type of actions taken. In addition, particularly in States without administrative order authority, a visit to the State Attorney General’s (AG) office may be desireable. Whereas the AG’s performance on taking actions and negotiating settlements will not be considered a factor in the State’s program performance, it may provide valuable information on ways to improve upon the State’s overall timely and appropriate enforcement response. Below is a series of questions to assess the compliance monitoring and enforcement program performance during the on-site review. For a more detailed checklist which expands on these questions, see Appendix B. Appendix B also includes a form for use in reviewing each document during a file review. - QUALITY OF ENFORCEMENT FILES • Are the files well-organized, complete and up- to - date? • Are the respondent’s work products, progress reports, correspondence included in the file? • Is there a procedure for checking out and returning files? • Do the files indicate the current compliance status? Are facilities that return to compliance (RTC) well-documented through field verifications? correspondence? ------- 9545 .OO-6A - 25 - QUALITY OF QIEs • Did the CME contain a review of the design and operation of GWM system, a review of the sampling and analysis plan, and a site map? • Does the CME adequately evaluate the facility’ s GWM system? • Is there a firm basis for all violations cited? • Does the CME report stand alone as an enforcement document? • For contractor-conducted CKEs, was the inspection adequately critiqued? • Was the CME checklist used? If not, what was used to evaluate CMEs? QUALITY OF CEIs/RECORD REVIEWS • Did the State accurately identify all Class I violations at handlers which received either an inspection or a record review? • Are inspection checklists and record review checklists complete and accurate? • Is there a firm basis for all violations cited? • Does the inspection report stand alone as an enforcement document? ------- 9545.OO-6A - 26 - QUALITY OF ADMINISTRATIVE ORDERS • Is there an administrative record to support the order? Is the administrative record complete? • Is there a “Findings of Fact” section in the Order that clearly describes the violations and how they were discovered? Is there sufficient documentation? • Are the actions to be taken clearly described in the Order with the responsibilities for the respondent and State clearly defined? Does the Order specify work products, reporting requirements, and schedules? • Is the State’s mechanism for overseeing the requirements specified in the Order adequate? • How are modifications to the Order made? C. CLOSURE PROGRAM Program Goals The closure program’s primary goals are to minimize the post-closure release of hazardous constituents into the environment and to take corrective action to remedy existing hazards. In order to attain these goals, the Regions and States must ensure that closing land disposal facilities have adequate ground- water monitoring systems and that all releases are detected. The Regions and States should compel corrective action, as necessary, through enforcement or post-closure permitting authorities. Furthermore, the Regions and States should ensure, through inspections and review of closure certifications, that the approved plans are implemented properly. Pie -meeting Preparation/Planning The pre-planning stage focuses on review of the State’s multi-year closure strategy and its progress in meeting schedules and facility priorities State monthly HWDMS reports and the State grant work agreement are the primary information sources for this stage of the review ------- 9545.OO-6A - 27 - The Region should evaluate the multi-year strategy and HWDMS reports for the level of commitment devoted to processing closure plans, conducting closure inspections and integrating corrective action into the closure process. The following questions are for use in the pre-planning stage and address the issues discussed above. A more detailed checklist is found in Appendix C. MULTI -YEAR STRATEGY AND MONfl LY REPORTS • Is the schedule for completing closure actions consistent with the State’s multi-year strategy? • What is the State’s progress in processing closures? • Does the strategy include commitments to conduct inspections at closing or closed land disposal facilities? • Does the strategy integrate corrective action requirements into the closure process? On-site REview The on-site review focuses on the quality of the closure plans and adherence to procedural requirements. Staff interviews and file reviews are the primary information sources. Files are checked for completeness of information (e.g., ground-water and soil monitoring data), consistency with regulatory requirements and incorporation of public comment. OSW developed the Protocol for Evaluating Interim Status Closure/Post-Closure Plans (August 1986) to assist the Regions in evaluating closure/post-closure plans for completeness and procedural correctness. In addition the guidance provides for the review of the documentation associated with a closure/post-closure plan and other substantiation of techical soundness. This guidance provides detailed checklists and forms which may be filled out by the reviewer during the on- site file review. The following questions address the primary issues covered during the on-site review. A more detailed checklist is found in Appendix C. ------- 9545.OO-6A - 28 - FILE REVIEW • Are the States reviewing closure plans in accordance with statutory and regulatory requirements? • Are all statutory requirements for public participation being met? • Are closure decisions based on adequate soil and ground-water monitoring information? How does the State deal with closing facilities which have inadequate ground-water monitoring systems? • Does the closure plan clearly explain and describe in detail the activities the owner/operator must conduct to ensure enforceability? • Do closure plans specify cleanup levels in adequate detail and provide mechanisms for measuring achievement of performance standards? • How is the State overseeing closure actions? Is this approach adequate to ensure compliance with closure requirements? • How do States follow up on facilities that close by removal to ensure that no soil and ground- water contamination exists? D. COfiRECTIVI ACTION PROGRAM Progra i Goals The goal of the corrective action program is to ensure remediation of releases of hazardous waste or hazardous constituents from treatment, storage and disposal facilities as necessary to protect human health and the environment To the extent possible, corrective action activities should be integrated with permitting and closure activities Whenever possible, all releases associated with a facility should be investigated at one time One objective of the RCRA corrective action program is to use an approach which is consistent with CERCLA, while meeting the specific needs and objectives of RCRA Regions and States should consider the use of CERCLA, TSCA, or other appropriate State authorities in conjunction with RCRA authorities to secure appropriate action The corrective action program should focus on identifying and remediating releases at facilities which pose the greatest threat to human health and the ------- 9545.OO-6A - 29 - environment. The Regions and States will use the FMP process and grant negotiations to identify facilities requiring priority attention. PROGRAM REVIEW Pre -meeting PreparationfPbuming In order to prepare for the on-site review, the Region should review State priorities to ensure that proper procedures are in place for identifying releases and potential releases, and for setting priorities for further corrective action activities. HWDMS data, including monthly reports, should .be compiled and sujmnRrized. HWDMS data quantitatively describe the progress of the State’s corrective action program in meeting commitments in its grant work program. A recommended series of questions to assess the corrective action program during the pre-planning stage is listed below. A more detailed checklist is found in Appendix D. STATE PRIORITIES • Is there a process for reviewing all facility- specific information to identify all potential releases at RCRA facilities? • How are facilities prioritized for scheduling RFAs? What information sources, if any, are used to prioritize facilities for RFAs? • What procedures are in place to identify facilities for priority attention in initiating the corrective action process? • What is the State’s progress in meeting RFA, RFI, CMS, and CMI activities specified in the State grant work agreement? On-site Review In evaluating the corrective action program through the on-site review process, file reviews and discussions with State staff members are the most important sources of information Through file reviews, the Region can determine whether all relevant data were used to detect releases which require corrective action. The Region can also determine whether RFAs and RFIs are ------- 9545.00-6A - 30 - conducted on schedule and in a technically adequate manner. Discussion with staff members should be used to answer questions which remain after the file review. This discussion should also be used to determine how well corrective action activities are being coordinated with other State program elements and other State and Federal regulatory authorities. A recommended series of questions to assess corrective action program performance at the on-site review is listed below. A more detailed checklist is found in Appendix D. FILE REVIEW • What is the quaMty of the RFAs? Are RFAs prepared according to existing guidance? Are visual site inspections conducted prior to completion of RFAs? • Are all conclusions in the RFA clearly explained and documented to support specific permit or compliance order conditions? (e.g., need for RFI, interim measures, etc.)? • Do RFAs identify and evaluate all SWMUs and all known/likely releases at a facility? • Are schedules of compliance for RFIs clear and enforceable? Are monitoring mechanisms and milestones for R.FIs established in the permit or order? • Does the State oversee approved RFI workplans/ sampling plans? • Have specific activities and schedules for public involvement been included in plans for all sites where an order or permit includes corrective action? • Are the schedules of compliance for corrective measures studies specific in detail and enforceable 2 • Are procedures in place to ensure that the owner/operator is implementing corrective measures in accordance with the agreed upon work plans? ------- 9545.OO-6A - 31 - FILE REVIEW (cont’ d) • Are there mechanisms available to measure the effectiveness of the corrective measures? , STAFF DISCUSSIONS • Has the State identified facilities for priority attention in initiating and implementing the corrective action process? • Has the State been able to identify and respond quickly to situations requiring immediate corrective measures? • In making decisions on RCRA corrective action activities, are other relevant authorities taken into account (e.g., CERCLA, TSCA, State authorities, etc.)? • Are corrective action activities effectively coordinated between permitting and enforcement? E. MANAG (ENT PROGRAM Prograii Objectives The main objective of the RCRA Management Program is to provide support and oversight to the other program elements through management of resources, training, and data management. This support ensures that all of the RCRA program elements work in concert to provide a high quality RCRA program over time. Smooth operation of program management is ensured by effective resource allocation. The resource allocation for e ich fiscal year is planned and agreed upon by EPA and the States in the State grant work agreement. This agreement outlines how many workyears will be expended in the operation of ------- 9545 .OO-6A - - each program element. This agreement should also inc .ude a plan for how federal grant money will be allocated to agencies other than the lead State agency A well trained staff is also critical to effective management of the State RCRA program. Management should develop and implement an annual training plan that discusses the areas and programs that would benefit from training and how this training will occur. A third area of concern for the Management Program is the use of an appropriate data system. State RCRA programs receive and process vast amounts of data. The State should establish a data system that provides timely and accurate information to meet the needs of State program decision makers. Moreover, the State must report this information to EPA using established schedules and formats. By implementing and maintaining an adequate data system and Quality Control Program, managers are able to facilitate this process. Finally, in order to ensure a consistent RCRA program nationwide, management must stay abreast of new Federal requirements. States must also keep EPA informed of potential changes in State legislation and regulations and strive to achieve or maintain an authorized RCR.A program under HSWA. Pre -i.eeting PreparatioufP1 ming Most of the Management Program review should take place prior to the on-site review. Through careful review of documents during the pre -meeting preparation/planning phase, the review team can develop a clear understanding of the management process in the State. Perhaps the most valuable source of information for this review is the State Grant Work Agreement. This document should contain the State’s plans for resource allocation and staff training as well as schedules for achieving RCRA program goals. Other important sources that should be reviewed prior to the on-site review are the State’s HWDMS reports and Data Quality Assurance: Plan. The information in these sources should be reviewed to determine if the Data Management System used in the State is adequate. The following questions should be addressed during the pre-meeting preparation/planning review. More detailed questions are included in Appendix E. ------- 9545.OO-6A STAFF TRAINING • What training will be conducted this year and which programs will benefit? • How are training needs evaluated? • How will training be conducted and by whom? INYORMATION MANAG T SYST ( • Have EPA and the State established consistent definitions for: reporting: data? • Are quality control. procedures established, documented, and implemented? • Are the data in HWDMS up-to-date? - 33 - RESOURCE ALLOCATION • How are work years allocated to each program element? • How are adjustments made during the year? • Are grant funds allocated to other State agencies? For what purposes? ------- 9545.OO-6A - 34 - On-site Reviev The on-site review of the Management Program is an opportunity for the review team to follow-up on any discrepancies discovered in the pre-planning review as well as to discuss State and Regional performance with members of the State staff. The primary information sources for the on-site review should be file- reviews and staff discussions. In addition to any issues that may develop as a result of the pre-meecing preparation/planning review, the following areas should be examined: RESOURCE ALlOCATION • Is the State using work years according to the RCRA allocation in the State grant work agreement? Explain deficiencies/deviations from the allocation. • Is the initial allocation sufficient to achieve program commitments or are changes required? If changes are needed, why? • Are all staff vacancies being filled? Is the skill mix appropriate for each program or are particular skills/expertise missing? What plans are in place to fill these gaps? • Are grant funds being diverted to other programs (Superfund, Subtitle D, etc.)? Are procedures in place to ensure no diversion? AFF TRAINING • Are staff trained in accordance with annual training plan? • Is the current training plan adequate or have new needs developed? What changes are necessary? ------- 9545.OO-6A - 35 - r INFORM&TION MANAG 1T SYST ( • Is the State Universe identical to HWDMS? Explain these discrepancies and how they will be fixed. • Does the Data Management System provide up-to- date activity and exception reports for all required activities? • Does program management use the data system to track and follow-up on key program elements? AUTHORIZATION • Is the State meeting its schedule for adopting regulatory changes for HSWA and non-HSWA clusters? • Jhat are the causes, if any, for these delays? • What actions should be taken to expedite HSWA authorization? ------- 9545 .OO-6A - 36 - SECTION IV: GUIDELINES FOR EPA RESPONSE TO STATE PERFORMANCE This section recommends procedural actions that can be followed to respond effectively to both positive and negative State performance as identified during the data collection and program review process described earlier in this guide. Part 3 of the National Criteria for a Quality Hazardous Waste Management Program under RCRA outlines the general principles to use when determining how EPA and the States should respond to performance against the criteria and related requirements. RESPONSE PRINCIPLES The appropriate response to performance against the criteria will depend on a number of conditions, including: • the relative importance of the criteria to program objectives; - • the seriousness of the failure or extent of success in meeting the criteria; • the frequency of any problems/failures or successes; • the number of criteria failed; and • the past response to plans to correct deficiencies. Certain situations demand a nationally consistent response. However, most situations require a response based on what will work for the indiv-idual Region and State, reflecting the general principles outlined here and in the Agency’s Performance-Based Grant Policy. Therefore, the response action must be tailored to the needs of the situation, recognizing that the principal goal is to strengthen the credibility, capability, and performance of the implementing agency. The level and severity of the response action should be consistent for similar performance problems, whether the lead Agency is the EPA or the State However, the available range of EPA response actions differs depending on whether the Region or the State is the lead Agency. Oversight in an authorized State is based on the MOA, the annual State grant work agreement, or other State-EPA agreements, and response actions are to be in accord with Agency policies. Response actions where the Region is the lead Agency are based on performance tracking and evaluations and personnel performance standards. ------- 9545 .OO-6A - 37 - RESPONDING TO 1PLARY P RMANCE In conducting effective oversight of State programs, it is as important to recognize and provide incentives for good performance as it is to identify problems and ensure that corrective actions are taken. The first response to effective State performance is simply to give credit where credit is due. Focus management attention on those areas where the State is operating in an exemplary way, and publicize the State program and its accomplishments. Related to this effort is the need to analyze what accounts for a State’ s exceptional performance in some area. Is it due to a particular State procedure? If so, could such a procedure (idea, approach, or project) be effectively employed in other States? These ideas should be transferred to other States in the form of memos, guidance, or technical notes, and/or included in training programs. A third response to positive State performance is to reduce the degree of oversight levels as appropriate for that State. For example, if the State’s inspections follow the established procedures and are always thorough and well documented, the Region may reduce the number of oversight inspections. Additionally, where performance has been demonstrated to be of consistently high quality, the Region may reduce the frequency and scope of performance reviews. R.ESPONDING TO PER. RMANCE PROBL (S Where the State fails to meet the performance criteria, the Regional Office will initiate plans to correct deficiencies. The primary emphasis of these plans will be on meeting the enforcement, permits, closure, and corrective action criteria and maintaining an equivalent program. The plans should clearly identify the corrective actions that are agreed upon as well as which Agency is responsible for the action (the State or EPA). Actions to correct deficiencies should be clearly established with real, measureable outputs and realistic timeframes. Review of progress towards these actions should be included in the next review. Where a State has problems meeting the actions, the Region will take a more rigorous look at how the State is handling certain management criteria, such as effective use of resources and training of staff In some cases, failure to meet the criteria will trigger a direct EPA action. This is particularly true in the enforcement area, where EPA has the authority under §3008(a)(2) to bring Federal enforcement action if the State action is not timely or appropriate. The quality criteria define “timely and appropriate” and establish a system of escalating enforcement actions based on the seriousness of the violation (e.g., Class I violations) and the available authorities (e.g , administrative penalty authority) If the State fails to conform to these criteria, EPA involvement may be triggered ------- 9545 .OO-6A - 38 - EPA may also take direct action in response to State-prepared permits that are inconsistent with the authorized State permit regulations. EPA will prepare comments in writing to the State for all permits designated for EPA review through the specific Regional/State Memorandum of Agreement. Based on 40 CFR §271.19, comments reflecting requirements of the authorized State program are enforceable by EPA, even if not included in the final State-issued permit. For the remainder of the criteria, the response will depend on the pattern of performance evaluated through program monitoring and the mid-year and end-of- year reviews. Where failure to meet a criterion has a minor impact on the overall quality of the State’s program, the Regional Office may slightly increase oversight or urge the State to modify a procedure. Where a State consistently fails selected criteria and shows little intent to correct the situation, the Region will take stronger actions. This should include award of conditional grants and quarterly or semi-annual release of funds, directly linked to performance. Listed below in roughly ascending order are examples of responses to failure to meet the criteria. The responses are not mutually exclusive: • Suggest a minor change to State or Regional procedures (e.g., - a change to filing procedures to improve availability of enforcement information). • Provide technical assistance (e.g., provide State permit writers with training on drafting groundwater corrective action provisions). • Slightly increase oversight (e.g., where file reviews indicate inspection checklists are not being properly completed, an increased number of oversight inspections may be appropriate). • Raise performance issues to a higher level of management, both at the Region and State offices (e.g., include as agenda item at routine BA/State Director meetings). • Revise future work program (e.g., add additional grant conditions requiring program management changes designed to correct problems with meeting performance criteria). • Greatly increase oversight (e g., where State consistently drafts permits that are not in accord with S€ate regulations, oversee more permits) ------- 9545.OO-6A - 39 - • Negotiate plans to correct deficiencies to address non- performance (e.g , implementation of plan would be evaluated at subsequent reviews and/or worked into State Grant Work Agreement). • Award conditional grants or revoke a State’s letter of credit (e.g., where States consistently fail to take timely enforcement actions, future grant awards should be tied to improved performance). • Initiate procedures to deobligate grant funds. • Initiate program withdrawal. Consistent with the Policy on Performance-Based Assistance (May 31, 1985), EPA will award conditional grants and release them on a quarterly or semi-annual basis where a consistent pattern of problems has developed for an individual State. Subsequent increments should be released only if the State corrects performance deficiencies. Following’ the mid-year review, the Regions should monitor State progress in addressing any performance problems discovered during the program review. The Region should know, prior to the next fiscal year’s grant negotiation, if State performance is improving or if the State still needs to take action to correct deficiencies. In most instances, during the end-of-year review, the Region, assesses State follow-up to recommended actions. The end-of-year review, however, occurs after the Region and State have negotiated the next fiscal year’s grant workplan. By waiting until the end-of-year to reexamine State performance, the Region may miss the opportunity to use the grant as a leverage for improved State program performance. Where a State consistently fails to follow through on the plans to correct deficiencies, or the State’s legal. authorities are no longer equivalent, EPA may initiate withdrawal of the State’s authorization. The criteria for withdrawal of the program are outlined at 40 CFR Part 271.22. ------- |