$ < 3) O % ^ PRO^ CD T s* U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL The EPA Should Assess the Utility of the Watch List as a Management Tool Report No. 13-P-0435 September 30, 2013 Scan this mobile code to learn more about the EPA OIG. ------- Report Contributors: Dan Engelberg Khadija Walker Tiffine Johnson-Davis Genevieve Soule Andre von Hoyer Abbreviations ECHO Enforcement and Compliance History Online EPA U.S. Environmental Protection Agency OECA Office of Enforcement and Compliance Assurance OIG Office of the Inspector General Hotline To report fraud, waste, or abuse, contact us through one of the following methods: email: OIG Hotline@epa.gov write: EPA Inspector General Hotline phone: 1-888-546-8740 1200 Pennsylvania Avenue, NW fax: 202-566-2599 Mailcode 2431T online: http://www.epa.gov/oiq/hotline.htm Washington, DC 20460 ------- * ' U.S. Environmental Protection Agency 13-P-0435 § \ Office of Inspector General September 30, 2013 s —'—'J" \ vjv " At a Glance Why We Did This Review We sought to determine how the U.S. Environmental Protection Agency (EPA) uses the Watch List as a tool to target enforcement actions and whether there are opportunities for improvement. Used by the EPA since 2004, the Watch List is designed to provide EPA regions and states with a list of facilities that are in significant violation of environmental laws and that appear not to have been addressed by timely and appropriate enforcement. The Watch List initially was just used as an in-house management tool, but in 2011 the EPA made its data available to the public. This report addresses the following EPA theme: • Embracing EPA as a high performing organization. The EPA Should Assess the Utility of the Watch List as a Management Tool For further information, contact our public affairs office at (202) 566-2391. The full report is at: www.epa.aov/oia/reports/2013/ 20130930-13-P-0435.pdf What We Found According to the EPA, the Watch List is a management tool used to identify long-standing significant violations. However, through interviews, regional questionnaires, and data analysis, we found that: • Use of the Watch List differs among headquarters and regional enforcement programs. • Trends in the number of facilities on the Watch List differ among enforcement programs. • EPA staff suggest multiple opportunities for improvement in the Watch List. • The public version of the Watch List has limited search capabilities and information. Wthout a proper assessment of the Watch List, the agency runs the risk of maintaining a management tool that does not assist in tracking facilities with long- standing significant violations, has limited transparency and utility to the public, and does not meet the needs of EPA users. Recommendations and Planned Agency Corrective Actions We recommend that the EPA Office of Enforcement and Compliance Assurance assess the Watch List's utility as a management tool. If the agency determines that the tool is useful, it should: • Ensure that Watch List criteria are consistent with relevant enforcement response policies, and reassess criteria to determine relevance. • Develop an approach for identifying and/or removing facilities on the list that have been referred elsewhere or are under a consent decree. • Improve transparency of the publicly available Watch List to allow users to search and receive information similar to what is available internally. • Identify and implement other improvements to the Watch List identified in the EPA's assessment. The agency agreed with the recommendation to assess the Watch List by September 30, 2014. If the agency decides to retain the list, it will act on the remainder of the recommendations by December 30, 2014. The recommendations are resolved with corrective actions pending. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL September 30, 2013 MEMORANDUM SUBJECT: The EPA Should Assess the Utility of the Watch List as a Management Tool Report No. 13-P-0435 FROM: Arthur A. Elkins Jr. TO: Cynthia Giles, Assistant Administrator Office of Enforcement and Compliance Assurance This is a report on the subject evaluation conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report describes issues the OIG identified and makes recommendations to address those issues. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established audit resolution procedures. Action Required The report recommendations are resolved with corrective actions pending. Therefore, the agency is not required to provide a final response to this report. However, if you choose to provide a final response, we will post your response on the OIG's public website, along with our memorandum commenting on your response. You should provide your response as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as amended. We will post this report to our website at http://www.epa.gov/oig. If you or your staff have any questions regarding this report, please contact Assistant Inspector General for Program Evaluation Carolyn Copper at (202) 566-0829 or copper.carolyn@epa.gov. or Director for Water Issues Dan Engelberg at (202) 566-0830 or engelberg.dan@epa.gov. ------- Purpose The purpose of this review was to determine how the U.S. Environmental Protection Agency (EPA) uses the Watch List as a tool to target enforcement actions and whether there are opportunities for improvement. Background In 2003, the assistant administrator for the Office of Enforcement and Compliance Assurance (OECA) requested that a tool be developed that would allow senior enforcement officials to track facilities that were in noncompliance with regulations (specifically, the Clean Air Act, Clean Water Act, and Resource Conservation and Recovery Act) without timely and appropriate enforcement actions. This request resulted in the implementation of the Watch List in 2004. The Watch List, one of several management tools used by the EPA enforcement programs, was designed to provide OECA and the EPA's regions, as well as states, with a list of facilities that data indicate are in significant violation of these environmental laws and have not been addressed by timely and appropriate enforcement. The system identifies facilities as high-priority violators under the Clean Air Act or significant noncompliers under the Clean Water Act's National Pollutant Discharge Elimination System and the Resource Conservation and Recovery Act. The Watch List contains a subset of violations identified under EPA enforcement response policies. The policies define when a violation of environmental laws and regulations becomes significant. This automated management tool is generated using compliance information from the EPA's data systems, including the Air Facility System, the Integrated Compliance Information System-National Pollutant Discharge Elimination System (previously the Permit Compliance System), and the Resource Conservation and Recovery Act Information System (RCRAInfo). In 2011, in response to a Freedom of Information Act request, the EPA made a modified version of the Watch List available to the public on its Enforcement and Compliance History Online (ECHO) website; the Watch List had previously only been used as an in-house management tool. Scope and Methodology We conducted this evaluation in accordance with generally accepted government auditing standards. Those standards require that we plan and perform our work to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our evaluation objectives. We conducted our evaluation from February through August 2013. To answer our objective, we identified the laws, regulations, policies, procedures and guidance used by the EPA for enforcement. We also reviewed documents 13-P-0435 1 ------- related to the EPA's roles and responsibilities, oversight/management requirements, performance tracking and goals for the Watch List. We interviewed the EPA program and enforcement staff about data reliability. We interviewed agency officials and sent a questionnaire to staff in all 10 EPA regions to determine their use of the Watch List and the areas for which it could be improved. We also conducted an analysis of Watch List data to identify potential trends and anomalies for further discussion with OECA and the regions and comparison with agency information. Results of Review According to OECA, the Watch List, which the agency has used for almost 10 years, is a management tool to identify long-standing significant noncompliance. We found the use of the Watch List differs among headquarters and regional enforcement programs. Further, we observed that trends in the number of facilities on the Watch List differ among enforcement programs. Additionally, staff suggested multiple opportunities for improvement. We also found that the public version of the Watch List has restricted capabilities. Without a proper assessment of the Watch List, the agency runs the risk of maintaining a management tool that does not assist in tracking facilities without timely and appropriate enforcement actions. In its current state, the list also has limited transparency to the public and may not meet the needs of its users, potentially limiting its utility for public users. Enforcement Programs Utilize the Watch List Differently Information we gathered from interviews with headquarters and questionnaires sent to all 10 regions showed that some enforcement staff in regional program offices use the Watch List to assist in their daily work (e.g., tracking facilities to determine whether they have returned to compliance and/or proceeded to follow-up with the state), while others only use it as a part of quarterly meetings with states and/or EPA headquarters to update the data on the list. Interviews and questionnaires indicated that the following factors might have affected the level of utilization of the Watch List: • Seven of 10 regions reported the availability of duplicative/alternate data sources. • Four of 10 regions and two headquarters program offices reported facilities remaining on the Watch List that have been referred to other offices/programs (such as the Department of Justice or the Superfund program) or are under a consent decree. • Three of 10 regions and one headquarters program office reported inaccurate reporting and timeliness problems with the data. • OECA headquarters and one region reported the Watch List not keeping up with technological advancements or changes in criteria definitions since it was created nearly a decade ago. 13-P-0435 2 ------- Enforcement Programs Have Differing Trends in Watch List Data If the Watch List was an effective and key management tool for addressing facilities in longstanding noncompliance, we would expect to see some decreasing trends in the number of facilities on the list. However, our review of national Watch List data, pertaining to three acts (the Clean Air Act, Clean Water Act, and Resource Conservation and Recovery Act), from headquarters programs and all 10 regions, showed different trends in the number of facilities on the Watch List from 2004 to 2013. Some regions had decreasing numbers of facilities on the Watch List in certain programs, others had increasing numbers, and others showed no obvious pattern of increase or decrease. The information gathered through interviews with OECA and questionnaires sent to the regions cited several reasons for the differences. They included states not reporting in a timely manner or having data entry issues, interface issues resulting from the switch from the Permit Compliance System to the Integrated Compliance Information System-National Pollutant Discharge Elimination System, and states or the agency not updating status codes timely and appropriately. Since most of the reasons cited were data quality issues, it is possible that, absent these issues, the trends would look different. Therefore, we are not able to draw a conclusion from the data about whether the Watch List has assisted EPA in addressing long-term noncompliance. Headquarters and Region Staff Identified Opportunities for Improvement Through interviews and regional questionnaires, Watch List users identified opportunities for improvement, including the following: • Incorporate a Watch List status in the ECHO facility search outputs (e.g., a "Watch List" column). • Create an efficient process to identify and/or remove facilities that are on the list in error or that the agency has referred to other programs or have been placed under a consent decree. • Ensure that criteria on the Watch List match the criteria in the relevant enforcement response policy. • Finalize Clean Water Act criteria. • Add the Safe Drinking Water Act to the Watch List • Allow OECA and regions to create region-specific "Watch Lists" based on mutually determined criteria while continuing to identify violations that meet overall Watch List criteria. • Initiate Web-based quarterly reporting to accept status codes and explanations from the regions. • Make the data on Watch List real time. • Enhance the Watch List standard operating procedures. • Add additional status codes that better describe the status of the current situation of a facility (make codes more specific). 13-P-0435 3 ------- Publicly Available Watch List Lacks Many of the Features and Information Available Internally The Watch List capabilities on the public website (ECHO) are very different from those of the internal Watch List. The public site does not provide users with the ability to search the Watch List to return only information that they are interested in. In addition, the information provided is in a "PDF" format and therefore cannot be easily sorted or analyzed by users, decreasing the utility of the list. The public Watch List provides limited information - only facility name, ID and location. The internal Watch List tool (available through the Online Tracking Information System) provides a substantial amount of additional enforcement information that allows the user to put the Watch List designation into context with the facility's enforcement history. Given the agency's long-stated goal of transparency, the information available to the public should contain all information that is not deemed enforcement sensitive. Conclusion Given the initial reasons for the Watch List implementation, its current differing use among regions, and differing data trends, the EPA should determine whether the Watch List is still a useful management tool. Our review identified opportunities for improvement, including updating and reviewing its approach for listing facilities to ensure transparency and accuracy of information as well as the public utility of the information. OECA is currently modernizing its enforcement data system, which is scheduled to be completed by the end of fiscal year 2013. We believe this is an opportune time for OECA to make changes to the Watch List should that be its decision. Recommendation We recommend that the assistant administrator, Office of Enforcement and Compliance Assurance: 1. Assess the Watch List's utility as a management tool in assisting the agency in monitoring long-standing alleged significant violators. If the agency determines that the tool is useful, it should: a. Ensure that Watch List criteria are consistent with relevant enforcement response policies and reassess the criteria to determine relevance. b. Develop an approach for identifying and/or removing facilities on the list that have been referred to other offices/programs or are under a consent decree. 13-P-0435 4 ------- c. Improve transparency of the publicly available Watch List to allow users to query and receive information similar to what is available through internal data systems. d. Identify and implement other improvements to the Watch List identified in the EPA's assessment. Agency Comments and OIG Evaluation The agency agreed with the recommendation to assess the Watch List by September 30, 2014. OECA intends to assess the overall appropriateness and utility of the Watch List as a multi-media tool as it revises some of the media- specific compliance and enforcement policies. In follow-up communications with OECA staff, we were told that if OECA decides to retain the Watch List, it will implement the remainder of the recommendations by December 30, 2014. The recommendations are resolved with corrective actions pending. 13-P-0435 5 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Action Official Planned Completion Date Claimed Amount Ag reed-To Amount Assess the Watch List's utility as a management tool in assisting the agency in monitoring long- standing alleged significant violators. If the agency determines that the tool is useful, it should: a. Ensure that Watch List criteria are consistent with relevant enforcement response policies and reassess the criteria to determine relevance. b. Develop an approach for identifying and/or removing facilities on the list that have been referred to other offices/programs or are under a consent decree. c. Improve transparency of the publicly available Watch List to allow users to query and receive information similar to what is available through internal data systems. d. Identify and implement other improvements to the Watch List identified in the EPA's assessment. Assistant Administrator, 9/30/14 Office of Enforcement and Compliance Assurance 12/30/14 12/30/14 12/30/14 12/30/14 1 O = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is unresolved with resolution efforts in progress 13-P-0435 6 ------- Appendix A Agency Response UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 OFFICE OF ENFORCEMENT AND COMPLIANCE ASSURANCE July 25,2013 MEMORANDUM SUBJECT: Comments on "Quick Reaction Report: The EPA Should Assess the Utility of the Watch List as a Management Tool" Thank you for the opportunity to review the Office of Inspector General (OIG) "Quick Reaction Report: The EPA Should Assess the Utility of the Watch List as a Management Tool". We agree that it is an opportune time to discuss the utility of the Watch List as a management tool in assisting the Agency in monitoring long-term alleged significant violators. We are currently in the process of revising some of the media-specific compliance and enforcement policies. As part of this process, we can better assess the overall appropriateness and utility of the Watch List as a multi-media management tool, As we are conducting this assessment, we will keep in mind the findings and initial recommendations of the OIG and commit to the following corrective action: Recommendation Lead Office CA Target Date Corrective Action If you have any questions regarding this response, please contact Gwendolyn Spriggs, the OECA Audit Liaison, at 202-564-2439. FROM: Cynthia Giles, Assistant Administrator TO: Carolyn Copper, Assistant Inspector General Office of Program Evaluation Assess the Watch List's OC/OCE 1 9/30/2014 Evaluate the utility of the Utility as a management tool. the Watch List as a Management tool, and identify appropriate next steps. 13-P-0435 7 ------- cc: Lisa Lund Susan Shinkman Mamie Miller Lauren Kabler Gwendolyn Spriggs 13-P-0435 8 ------- Appendix B Distribution Office of the Administrator Assistant Administrator for Enforcement and Compliance Assurance Agency Follow-Up Official (the CFO) Agency Follow-Up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for External Affairs and Environmental Education Principal Deputy Assistant Administrator for Enforcement and Compliance Assurance Audit Follow-Up Coordinator, Office of Enforcement and Compliance Assurance Audit Follow-Up Coordinator, Office of Solid Waste and Emergency Response Audit Follow-Up Coordinator, Office of Water Audit Follow-Up Coordinator, Office of Air and Radiation 13-P-0435 9 ------- |