$ < 73 \ Ml r ppo^ O 2 Lll O T A? OFFICE OF INSPECTOR GENERAL Catalyst for Improving the Environment Evaluation Report EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed Report No. 2007-P-00006 December 5, 2006 ------- Report Contributors: Carolyn Copper Angela Bennett Jennifer Dwyer Andrew Lavenburg Madeline Mullen Michael Owen Abbreviations CERCLA CERCLIS EPA OIG OSWER SARA Comprehensive Environmental Response, Compensation, and Liability Act Comprehensive Environmental Response, Compensation, and Liability Information System U.S. Environmental Protection Agency Office of Inspector General Office of Solid Waste and Emergency Response Superfund Amendments and Reauthorization Act ------- < 33 \ ^t0SrX & V PRO^4-0 o 2 Lll o U.S. Environmental Protection Agency Office of Inspector General At a Glance 2007-P-00006 December 5, 2006 Why We Did This Review Prior studies have identified weaknesses in the Superfund five-year review process. We evaluated whether the U.S. Environmental Protection Agency (EPA) has improved the quality, completeness, and timeliness of five-year reviews, and what impact the review process has had on remedies at Superfund sites. Background EPA's Superfund five-year review process examines the remedies at hundreds of Superfund sites where hazardous substances remain at levels that potentially pose an unacceptable risk. The purpose of the reviews is to determine whether remedies are, or will be, protective of human health and the environment. For further information, contact our Office of Congressional and Public Liaison at (202) 566-2391. To view the full report, click on the following link: www.epa.aov/oia/reports/2007/ 20061205-2007-P-00006.pdf Catalyst for Improving the Environment EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed What We Found Since our last review in 1999, EPA has taken actions to improve the five-year review process. These actions included issuing the Comprehensive Five-Year Review Guidance, providing training, and reducing the review backlog. While these actions have resulted in improvements, EPA needs to take additional steps to better support and communicate conclusions, continue to improve review timeliness, and provide fuller assurance that cleanup actions are protective of human health and the environment. While we did not conclude that remedies were unsuccessful at achieving protection of human health and the environment, our evaluation of a random sample of 39 five-year review reports issued between Fiscal Years 2002 and 2004 showed that: • 21 percent did not fully support their protectiveness conclusions • 21 percent did not provide complete protectiveness conclusions • 21 percent did not have sufficient information to implement recommendations • 23 percent did not meet public notification requirements EPA has not assessed the overall impact of its five-year review process on implementing and performing remedies because a system to provide complete information on the results of reviews had not been implemented. What We Recommend We recommend that EPA expand the scope of quality assurance reviews of five- year review reports, and revise guidance to more clearly define short- and long- term protectiveness determinations. To improve timeliness, we recommend that EPA evaluate the regions" workloads and available resources for five-year reviews for meeting due dates. We also recommend that EPA use data in a new information system module to measure the effectiveness and impacts of five-year reviews. EPA generally concurred with our recommendations. The Agency will need to provide further details on its plans to address Office of Inspector General recommendations within 90 days. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 OFFICE OF INSPECTOR GENERAL December 5, 2006 MEMORANDUM SUBJECT: EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed Report No. 2007-P-00006 TO: Susan Parker Bodine Assistant Administrator Office of Solid Waste and Emergency Response This is our report on the subject evaluation conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established resolution procedures. The estimated cost of this report - calculated by multiplying the project's staff days by the applicable daily full cost billing rates in effect at the time - is $530,299. Action Required In accordance with EPA Manual 2750, you are required to provide a written response to this report within 90 calendar days. You should include a corrective actions plan for agreed upon actions, including milestone dates. We have no objections to the further release of this report to the public. This report will be available at http://www.epa.gov/oig. If you or your staff have any questions, please contact me at (202) 566-0847 or roderick.bill@epa.gov: or Carolyn Copper, Director for Program Evaluation, Hazardous Waste Issues, at (202) 566-0829 or copper.carolyn@epa.gov. Sincerely, —BilhA^odenck Acting Inspector General ------- EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed Table of C Chapters 1 Introduction 1 Purpose 1 Background 1 Prior Evaluations and Agency's Actions 2 Scope and Methodology 3 2 EPA Needs to Further Improve the Quality and Completeness of Reviews 5 Actions Taken to Improve Five-Year Reviews 5 Five-Year Review Reports Need Further Improvements 6 Supporting Documentation Not Maintained 8 Primary Causes for Quality and Completeness Problems Noted 9 Conclusion 10 Recommendations 10 Agency Comments and OIG Evaluation 11 3 Progress Made in Completing Reviews Timely But Further Improvement Needed 12 EPA's Annual Planning Processes Generally Successful 12 Majority of Reviews Conducted Timely, But Further Actions Needed 13 Limited Resources and No Tracking Systems Caused Delays 14 Conclusion 15 Recommendations 15 Agency Comments and OIG Evaluation 16 4 Reviews Provide an Effective Oversight Mechanism, But More Data Needed 17 Five-Year Reviews Identify Protectiveness Issues 17 EPA Has Not Evaluated Overall Impacts of Five-Year Reviews 18 Recommendations 19 Agency Comments and OIG Evaluation 19 Status of Recommendations and Potential Monetary Benefits 20 - continued - ------- EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed Appendices A Details on Scope and Methodology 22 B Details on Reports without Fully Supported or Complete Protectiveness Conclusions 25 C Agency Response to Draft Report 29 D OIG Evaluation of Agency Response 40 E Distribution 44 ------- Chapter 1 Introduction Purpose This report focuses on the U.S. Environmental Protection Agency's (EPA's) five- year review activities for Superfund sites. Overall, we sought to determine how successful the five-year review process has been at assuring that remedies at Superfund sites are protective of human health and the environment. We addressed three questions: • How successful has EPA been at improving the quality and completeness of five-year reviews? • How does EPA identify sites requiring five-year reviews and ensure that the reviews are conducted timely? • What impact has EPA's five-year review process had on the implementation and performance of remedial actions at Superfund sites? Background About 11 million people in the United States, including 3 to 4 million children, live within 1 mile of a Superfund site. Section 121(c) of the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), as amended by the Superfund Amendments and Reauthorization Act (SARA) of 1986, requires EPA to review the remedies at Superfund sites where hazardous substances, pollutants, or contaminants will remain onsite. These reviews are required to be conducted every five years. The purpose of five-year reviews is to evaluate the implementation and performance of remedies and determine whether remedies are, or will be, protective of human health and the environment. When a five-year review identifies problems or deficiencies with a remedy, EPA should ensure appropriate corrective actions are taken. EPA uses two types of five-year reviews: statutory and policy. Statutory reviews are required by CERCLA. These reviews are for sites where post-SARA remedial actions leave hazardous substances, pollutants, or contaminants on site. The initial trigger date for statutory reviews is the initiation of the first remedial action leaving contaminants on site. Policy reviews are not required under CERCLA, but are performed as a matter of EPA policy. These reviews are triggered by the date that remedial action construction is completed at a site. Policy reviews are required for sites where a: (1) pre-SARA remedial action 1 ------- leaves hazardous substances, pollutants, or contaminants onsite; or (2) pre- or post-SARA remedial action will allow for unlimited use and unrestricted exposure, but requires five years or more to complete. Policy reviews are also performed for removal-only National Priority List sites where hazardous substances, pollutants, or contaminants were left onsite at levels that do not permit unlimited use and unrestricted exposure. Both statutory and policy reviews should be completed within five years of their trigger dates, with subsequent five- year reviews completed five years after the prior review is issued. The five-year review process is the same for both types of reviews. The process is based on information obtained from decision documents, operational data, and input of those responsible for and affected by the actions at the site. The process includes several components, such as: • Community involvement and notification: Issue public notices announcing the initiation and completion of the five-year review, and provide results of the review in a local site repository. • Document review: Gather and review all relevant documents, data, and other information in support of the five-year review. • Site inspection: Conduct a site inspection to visually confirm and document the conditions of the remedy and the site. • Interviews: Gather additional information about the site through interviews with site personnel, regulatory authorities, local officials, and/or community action groups. • Data review and evaluation: Review and analyze the data collected as part of a technical assessment of the remedy and the site. • Protectiveness statements: Based on the technical assessment, make a determination as to whether the remedy is protective of human health and the environment. EPA regions generally conduct the five-year reviews, but they may also be conducted by States, Federal facilities, the U.S. Army Corps of Engineers, or a contractor. In all cases, the components identified above are used to evaluate the performance and protectiveness of the remedy. Prior Evaluations and Agency's Actions Prior reports by the EPA Office of Inspector General (OIG) and Resources for the Future, a non-profit independent research organization, have identified weaknesses in EPA's five-year review process. Specifically: • EPA OIG Report No. 5100229, Backlog Warrants Higher Priority for Five-Year Reviews, March 24,1995: This report disclosed that only 30 percent of the required five-year reviews were completed as of 1994, and predicted that the backlog would increase for Fiscal Years 1995 through 1997. 2 ------- • EPA OIG Report No. 1999-P-219, Backlog of Five-Year Review Reports Increased Nearly Threefold, September 30,1999: This followup report concluded that: (a) the backlog of overdue reviews significantly increased since 1995, (b) 28 percent of five-year review reports evaluated did not include a conclusion on protectiveness of remedies or did not adequately support conclusions, (c) 50 percent of the reports reviewed with recommendations did not include sufficient information for the recommended actions, and (d) 63 percent of the reports were issued late. • Resources for the Future, in the publication Superfund's Future, What Will It Cost, issued in 2001: This publication indicated that 32 percent of the 151 five-year review reports reviewed had protectiveness statements that were insufficiently substantiated or were questionable. In response to the findings and recommendations in these reports, EPA in 2001 developed three initiatives to improve the management of its Five-Year Review program and the quality of five-year review reports. Table 1.1 summarizes these initiatives. Table 1-1. EPA's Five-Year Review Initiatives Initiative Major Actions Improve the Quality and Consistency of Reports • Complete and implement the Comprehensive Five-Year Review Guidance. • Develop and provide five-year review training to EPA staff in the regions and others involved with the reviews. • Conduct quality reviews of selected five-year review reports prepared by each EPA region. • Clarify and reaffirm protectiveness statements for reports questioned by Resources for the Future. • Track corrective actions and recommendations in five-year review reports. Provide Public Access to Reports • Make completed five-year review reports available to the general public through the EPA regions' Internet Web pages. Eliminate Backlog and Ensure Timely Completion of Overdue Reports • Ensure the Agency does not continue to add to the backlog, and clear the backlog by the end of Fiscal Year 2002. Source: Office of Solid Waste and Emergency Response Memorandum on Five-Year Review Program Initiatives, August 27, 2002 Scope and Methodology We conducted our evaluation from November 2004 to May 2006. We performed our evaluation in accordance with Government Auditing Standards, issued by the 3 ------- Comptroller General of the United States. We considered the findings of prior EPA OIG reports and a Resources for the Future publication related to EPA's Five-Year Review program. To evaluate the Agency's Five-Year Review program, we interviewed managers and program staff from the EPA Office of Solid Waste and Emergency Response (OSWER) and from 5 of EPA's 10 regions. We also reviewed a random sample of 39 of 555 five-year review reports issued during Fiscal Years 2002 through 2004. Our sample of 39 reports included at least 1 report from each of EPA's 10 regions. Further, we reviewed key documents and data supporting information presented in 28 of the reports in our sample (representing the 5 regions where we also conducted interviews), including Records of Decision, site sampling results, and site inspection records.1 Appendix A provides further details on our scope and methodology. 1 Due to time and resource constraints, we could not visit all 10 regions and conduct detailed site file analyses for each of the 39 sites in our sample. Therefore, we selected 28 five-year review reports from our sample covering Regions 2, 3, 4, 5, and 9. Reports from these five regions were selected primarily because we sought to obtain geographical coverage for the eastern, central, and western sections of the nation. 4 ------- Chapter 2 EPA Needs to Further Improve the Quality and Completeness of Reviews While EPA has taken actions to address quality and completeness issues identified in prior evaluations of the Agency's five-year review activities, further improvement is needed. An analysis of a random sample of 39 completed five- year review reports showed the following: Table 2-1. Issues Regarding Reports Issue Percent Protectiveness conclusions not fully supported 21 % Protectiveness conclusions incomplete 21% Information to implement recommended corrective actions insufficient 21% Reviews did not meet public notification requirements 23% Source: EPA OIG analysis of five-year review reports Further, 79 percent of the 28 reports examined for documentation did not have complete documentation for at least one of three supporting activities: public notices, interviews, or site inspections. The issues noted primarily occurred because EPA did not establish effective quality assurance processes, sufficient definitions for short- and long-term protectiveness conclusions, effective public notification procedures, and specific documentation requirements. We did not conclude that Superfund remedies may be unsuccessful at achieving intended protectiveness results. However, without complete and fully supported report information, EPA has less assurance that remedies at Superfund sites protect human health and the environment. EPA also has inadequate assurance that Congress and the public are accurately informed on the status of remedial actions.2 Actions Taken to Improve Five-Year Reviews Since 2001, EPA has taken several actions under its initiative Improve the Quality and Consistency of Reports to address issues identified by the EPA OIG and Resources for the Future. Specifically: 2 Five-year reviews are required, under Section 121 of CERCLA, to assure that human health and the environment are being protected by the remedial action being implemented. The Act requires that Congress be notified of the results of all such reviews, and any actions taken as a result of such reviews. Additionally, as part of its 2001 initiatives to improve the five-year review program, EPA committed to making five-year review results available to the public via the Internet. 5 ------- • Guidance: OSWER developed and implemented the Comprehensive Five-Year Review Guidance during Fiscal Year 2001. This document supersedes prior EPA guidance for five-year reviews, and is designed to clarify EPA's policy, facilitate consistency across regions, and note roles and responsibilities. • Training: OSWER conducted training covering the major components of the Comprehensive Five-Year Review Guidance during Fiscal Years 2001 and 2002. According to OSWER program staff, more than 300 remedial project managers in EPA regions received the training. • Reevaluations: The regions reevaluated the 48 five-year review reports identified by Resources for the Future as not having clearly supported protectiveness statements during Fiscal Year 2002. Based on the regions' reevaluations, OSWER concluded no exposure issues existed at the sites. • Quality Assurance Reviews: OSWER has conducted quality assurance reviews of selected draft five-year reviews since Fiscal Year 2002 to help the regions meet guidelines outlined in the Comprehensive Five-Year Review Guidance. Five-Year Review Reports Need Further Improvements Although EPA has taken several actions since 2001 to improve the quality and consistency of five-year reviews, further improvement is needed. We did not conclude that the remedies at the sites we reviewed may be unsuccessful at achieving intended protectiveness results. However, our review of a random sample of 39 five-year review reports issued between Fiscal Years 2002 and 2004 showed various issues, as discussed below. Reviews Contained Unsupported Protectiveness Conclusions Eight of the reports reviewed, or 21 percent, had remedy protectiveness conclusions that were not fully supported. EPA's Comprehensive Five-Year Review Guidance states that "the [five-year review] report should contain the data and information necessary to support all findings and conclusions." See Appendix B for details on reports with unsupported protectiveness conclusions. Examples of insufficient support follow: • In one report, the information appeared to contradict the conclusion. The report stated that the remedy was implemented in accordance with the Record of Decision and remained "fully protective of human health and the environment." However, the report also stated ".. .the current off-site extent of the contaminants is unknown. As such, the protectiveness of the ground water remedies is unknown... There is an urgent need for the evaluation of current ground water concentrations off-site...." 6 ------- • The technical assessment in another report did not address all of the elements of the evaluation framework prescribed by the Comprehensive Five-Year Review Guidance. The technical assessment did not provide sampling data and other specific information to support the evaluation and protectiveness conclusions. Further, the Document Review section of the report did not cite any monitoring/sampling reports that covered the review period. • A third report disclosed that the erosion of an earthen dam at the site was a "serious issue that would potentially undermine the stability of the recovery well... and ultimately the dam itself." The report identified this issue as impacting both the current and future protectiveness of the remedy. However, the report concluded the remedy was protective of human health and the environment and did not mention the dam stability issue in the protectiveness statement. Reviews Contained Incomplete Protectiveness Conclusions An additional eight reports reviewed, or 21 percent, did not contain all required elements for protectiveness conclusions. We found these protectiveness conclusions to be incomplete because they either did not cover all operable units at the sites (five reports) or did not address short- and long-term protectiveness (three reports). This condition existed for reports from 4 of the 10 regions. According to the Comprehensive Five-Year Review Guidance, a protectiveness statement should be developed for each operable unit at which a remedial action has been initiated. OSWER managers also specified that protectiveness statements should address both short- and long-term protectiveness. Further Improvement Needed with Recommended Corrective Actions Eight of the reports reviewed, or 21 percent, did not contain sufficient information to implement the recommended corrective actions, or did not provide recommendations to correct remedy deficiencies. While this is an improvement from OIG's 1999 report, which showed that 50 percent of the reports with recommendations lacked sufficient information to implement the recommended actions, further improvement is needed. Six of the reports in our sample contained one or more recommendations that did not identify milestone dates, the responsible party for implementation, and/or the responsible oversight agency. This information is required by the Comprehensive Five-Year Review Guidance and is critical to ensuring recommendations are timely and effectively implemented. In one example, a report identified that the site was not achieving compliance standards in specific wells, and that the ground water gradient was not responding to specific measures taken to control the gradient. Followup actions were recommended for these issues, but the report did 7 ------- not identify a responsible party, oversight agency, milestone dates, and the impact on current and future protectiveness. The other two reports did not include recommendations to address all issues that had a potential impact on the protectiveness of the remedy. For example, one of the reports disclosed that ground water system evaluation reports indicated that wells outside the remediation system were showing some degree of contamination and "the remedy [was] not functioning exactly as intended." However, the report did not include a recommendation to address this issue. Public Notification Requirements Not Met Nine of the reports reviewed, or 23 percent, consisted of reviews that did not meet public notification requirements. In four reports, EPA or the appropriate five-year review lead agency did not provide public notifications of the commencement and completion of the evaluations. The other five reports were based on reviews where the applicable lead agency did not issue a public notice for the commencement of the review. The Comprehensive Five-Year Review Guidance requires that public notices be issued before and after each review. These notices are important for informing the public about the five-year review process, obtaining public participation, and disclosing the results of the review. Supporting Documentation Not Maintained In addition to reviewing reports themselves, we reviewed documentation for 28 of the reports in our random sample. For 22 of those reports, or 79 percent, we found that the supporting documentation was not prepared and/or maintained by the regions for one or more major five-year review components: • Public notification details (25 percent of reports) • Details on interview with officials or others (32 percent of reports) • Site inspection details (64 percent of reports) As a result, we could not verify public notice, interview, and site inspection details in the reports. While technical data (sampling reports, monitoring plans, and operation and maintenance documentation) are the primary basis for the technical assessment and protectiveness determinations, these other major review activities help ensure that the review is based on complete information. The Comprehensive Five-Year Review Guidance provides examples for conducting and documenting interviews and site inspections. The guidance also specifies that the regions "... should establish appropriate record keeping procedures to minimize efforts needed to gather all necessary documents for subsequent five-year reviews." Therefore, maintaining documentation for five- year review supporting information should streamline the process for future reviews, especially in instances where there has been a change in remedial project 8 ------- managers. Also, regional managers, independent reviewers, and the public cannot verify important details in the reports without supporting documents. Documentation supporting the details of the major components of the five-year review is critical to ensuring the quality and completeness of the reviews. Primary Causes for Quality and Completeness Problems Noted Although OSWER and the five regions included in our sample have quality assurance processes for five-year reviews, their processes have not been effective in ensuring the reviews met the Comprehensive Five-Year Review Guidance. OSWER has conducted quality assurance reviews on selected reports since Fiscal Year 2002; however, the reviews since Fiscal Year 2003 generally have been limited to regional requests. Our discussions with program managers and staff from the five regions identified that one region conducted quality assurance reviews on selected five-year review reports and the other four regions conducted reviews on all reports. We were unable to evaluate the scope and depth of the Agency's quality assurance reviews because the reviews were generally not documented. However, the quality and completeness issues we identified show that the Agency's quality assurance activities have not been effective. Therefore, OSWER needs to perform quality assurance reviews on a higher percentage of five-year review reports and the regions need to conduct more comprehensive quality assurance reviews. The primary cause for incomplete protectiveness statements in five-year review reports was that some region personnel believed having such determinations for each operational unit at a site was confusing and potentially redundant. While there may be similar protectiveness conclusions for different operational units at a site, the Comprehensive Five-Year Review Guidance clearly requires that separate protectiveness statements be made for each operational unit at a site. Also, the guidance does not sufficiently define short- and long-term protectiveness determinations, which created inconsistencies in conclusions across regions. Regarding lack of public notifications, the regions had not established procedures to ensure that the notifications were issued. For example, in one region, remedial project managers did not always coordinate their reviews with the community involvement coordinator responsible for issuing public notifications. The Comprehensive Five-Year Review Guidance does not include specific documentation requirements for five-year reviews. As a result, we found that the regions did not always ensure that public notices were included in site files. Also, some regions did not always consider interview and site inspection documentation to be necessary when they were highly involved at the sites. 9 ------- Conclusion The purpose of five-year reviews is to evaluate implementation and performance of Superfund remedies and to establish appropriate corrective actions when problems with remedies are identified. Corrective action recommendations in five-year review reports have substantially improved since 1999 when our last report was issued. However, EPA needs to take additional actions to improve the overall quality and completeness of information and conclusions in five-year reports. While we did not conclude that these problems prevented remedies from achieving intended protectiveness results, the problems undermine assurances that remedies protect human health and the environment, and that Congress and the public are being accurately informed on the protectiveness status of remedial actions. Recommendations We recommend that the Assistant Administrator for Solid Waste and Emergency Response: 2-1 Expand the Agency's scope of the quality assurance reviews conducted under the initiative Improve the Quality and Consistency of Five-Year Review Reports. At a minimum, conduct quality assurance reviews for a representative sample of reports from each region. 2-2 Revise the Comprehensive Five-Year Review Guidance to more clearly define short- and long-term protectiveness determinations. Also, revise the guidance to include specific requirements for conducting and documenting quality assurance reviews of five-year review reports, and maintain documentation to support the information in the reports. These requirements should emphasize the need for: a) Report information that provides complete support for conclusions on remedy protectiveness. b) Complete information supporting recommendations, including milestones, oversight agency, and responsible party. c) Documentation supporting report information and quality assurance results. d) Regional quality assurance programs for five-year reviews that ensure the reviews are complete and fully supported, including site inspections. 2-3 Communicate to the regions the need for: (a) public notifications for the commencement and completion of five-year reviews; and (b) protectiveness conclusions that address each operable unit at a site. 10 ------- Agency Comments and OIG Evaluation EPA agreed with Recommendations 2-1 and 2-3 and responded that increased review by Headquarters improves the quality and consistency of reports. The Agency did not agree with Recommendation 2-2, but described several efforts currently underway that appear to generally meet the intent of the recommendation. In its comments on Recommendation 2-2, the Agency said that interviews are not required by the Comprehensive Five-Year Review Guidance and are only one of many community involvement activities to solicit feedback on the remedies being reviewed. We agree with the Agency's comment on interviews and made appropriate modifications to the chapter and recommendation. Although the Agency described corrective actions taken and planned to address the recommendations, it will need to provide more specific details on these actions for Recommendations 2-1 and 2-2, as well as specific milestones for completing corrective actions for all three recommendations, within 90 days. The Agency also provided comments on our findings for some five-year reviews discussed in Chapter 2 and Appendix B of the report. We made changes based on these comments as appropriate. The Agency's complete written response is in Appendix C. Our evaluation of those comments is in Appendix D. 11 ------- Chapter 3 Progress Made in Completing Reviews Timely But Further Improvement Needed EPA identified sites subject to five-year reviews through an annual planning process and successfully addressed a backlog of overdue five-year reviews. EPA has also improved the timeliness of five-year reviews, with 67 percent of the 39 reports in our sample being issued on time. However, further actions are needed to ensure all reviews are completed within five years of their trigger date and recommended corrective actions implemented by due dates. The untimely reviews were primarily attributable to large workloads and turnover of remedial project managers. Late implementation of recommendations was primarily caused by the absence of formal tracking systems in the regions. The late reviews and implementing of recommendations resulted in delayed corrective actions that potentially impacted the protectiveness of some remedies. Also, EPA did not always timely inform affected residents and Congress on the effectiveness of remedies. EPA's Annual Planning Processes Generally Successful OSWER and the regions identified sites subject to five-year reviews and extramural funding requirements through an annual planning process. This process includes review of EPA's Comprehensive Environmental Response, Compensation, and Liability Information System (CERCLIS) by the regions and OSWER to identify sites subject to review and their due dates. OSWER discusses and confirms each region's review workload and necessary extramural funding requirements to support the workload for the year through planning meetings. Our comparison of five-year review tracking information maintained by the five regions we visited with five-year review data in CERCLIS for Fiscal Years 2002 through 2004 indicated that EPA's process to identify sites subject to five-year review is generally effective. With few exceptions, the review information maintained by the regions matched that in CERCLIS. The most significant exception was that one region's tracking information did not include six Federal facility sites subject to the five-year review requirement, and although these reviews were omitted from the region's tracking information they were still completed on time. Also, since 1999, EPA has been successful in reducing its backlog of overdue five-year reviews. Our 1999 evaluation identified a backlog of 143 reviews overdue by an average of 17 months. This backlog represented 40 percent of the reviews planned for completion through March 1999. As of the end of Fiscal Year 2004, CERCLIS showed a backlog of overdue five-year reviews for 14 sites, 12 ------- representing only 2.5 percent of the five-year reviews planned for completion between Fiscal Years 2002 and 2004 (see Table 3.1). Table 3-1. Uncompleted Five-Year Reviews for Fiscal Years 2002 through 2004 Region Number of Planned Reviews 2002-2004 Number of Reviews Not Completed by End of 2004 Percent of Reviews Not Completed by End of 2004 1 50 0 0% 2 70 1 1% 3 75 4 5% 4 65 6 9% 5 115 0 0% 6 34 0 0% 7 36 1 3% 8 20 1 5% 9 43 1 2% 10 47 0 0% Total 555 14 2.5% Source: EPA OIG analysis of data provided by OSWER Majority of Reviews Conducted Timely, But Further Actions Needed EPA also improved the timeliness of five-year reviews, with over two-thirds of the reports in our random sample being issued on time. However, further actions are still needed to ensure all reviews are completed within five-years of their trigger date and recommended corrective actions implemented by established dues dates. Our 1999 evaluation report disclosed that 63 percent of the 356 five-year review reports issued since the inception of the five-year review program were issued late by an average of 17 months. EPA has significantly improved its timeliness since then. Our random sample of 39 reports issued between Fiscal Years 2002 and 2004 showed that 67 percent of the reviews were completed on time and 5 of the 10 regions completed all reports in the sample timely. The sample also showed that the late reports were overdue by an average of 12 months (see Table 3.2). Table 3-2. Timeliness of Five-Year Review Reports Reviews in Number of Range of Overdue Region Our Sample Late Reviews Reports (months) 1 2 0 0 2 7 4 6-25 3 7 0 0 4 6 2 7-11 5 6 0 0 6 2 2 12-27 7 4 3 9-12 8 2 2 8-60 9 2 0 0 10 1 0 0 Total 39 13(33%) 6-60 Source: EPA OIG analysis of five-year review reports 13 ------- While reducing the number of untimely reports from 63 percent to 33 percent is a significant accomplishment, further improvement is needed. One review in our sample appeared to miss an entire five-year review cycle, and was completed approximately five years past its due date (we did not include this review in our calculation of average time overdue because we considered it an extreme case). The other 12 late reviews in the sample were completed an average of 12 months past their due date. The overdue reports contained 42 recommended followup actions that address remedy protectiveness and other site-related issues identified during the five-year reviews. We noted in our evaluation of the recommendations in 28 of the reports in the sample that for 43 percent of them, one or more of the recommended corrective actions were not implemented by established due dates. In 25 percent of the sample, implementation due dates were missed for recommendations addressing current or future remedy protectiveness issues. Limited Resources and No Tracking Systems Caused Delays Discussions with program staff from two of the five regions with untimely reports disclosed that the primary contributors to the delays were large Superfund site workloads for the remedial project managers and a turnover of those managers for some of the sites. OSWER managers told us that some regions, particularly those that conduct most five-year reviews using regional staff, view a shortage of staff resources as a significant issue. According to the OSWER managers, extramural funding (such as funding for work performed under contracts, grants, or interagency agreements) for five-year reviews is sufficient and available for all regions. In fact, some regions receive extramural funding to hire the U.S. Army Corps of Engineers to assist with their five-year workload. However, the majority of regions have not chosen to use the extramural funding for five-year review commitments. Also, planned completion dates for some five-year reviews in CERCLIS were not accurate and may have contributed to delays. For five untimely reviews in our sample, CERCLIS showed planned due dates different than those in five-year reviews reports. CERCLIS identified some reviews as completed timely when they were actually up to two years overdue. An OSWER staff member indicated this discrepancy occurred because the data for many reviews were entered into CERCLIS retroactively. According to this staff member, all planned and actual completion dates for upcoming five-year reviews will be accurate, and OSWER expects the new CERCLIS 3 Module to correctly reflect all trigger dates and due dates for five-year reviews. The staff member said OSWER plans to fully implement the module during Fiscal Year 2006, and it will provide for each site: • Planned and actual due dates for five-year reviews • Protectiveness determinations • Descriptions of issues and recommended corrective actions 14 ------- • Implementation status of recommendations We also found that the five regions in our sample did not have formal tracking systems for ensuring recommended corrective actions were implemented by specified milestones. Instead, the regions generally relied on the remedial project managers to track and resolve recommended corrective actions. As a result, there was not central regional visibility or awareness of the status of recommended corrective actions and the effectiveness of those actions in addressing issues identified by the five-year reviews. Conclusion EPA has made substantial progress in addressing the backlog of five-year reviews and improving the timeliness of reviews since our last report in 1999. However, additional improvement is needed to ensure all reviews are completed timely and recommended corrective actions are completed by established due dates. The late reviews and implementation of recommendations resulted in delayed corrective actions. OSWER should evaluate the regions' workloads and available resources for conducting five-year reviews through annual planning with the regions. Extramural funding should be used to obtain additional resources for conducting five-year reviews where insufficient EPA staffing will delay completion of reviews. The new CERCLIS 3 Module, once tested and fully implemented, should assist the Agency with tracking and followup of five-year review activities, including implementation of recommendations. Recommendations We recommend that the Assistant Administrator for Solid Waste and Emergency Response: 3-1 To assess discrepancies or gaps in resources, evaluate annual five-year review workloads and available resources as part of the annual planning process with the regions. Communicate to the regions that the U.S. Army Corps of Engineers or other contractor sources should be used for reviews where insufficient staffing or changes in remedial project managers will delay completion. 3-2 Work with region officials to correctly identify five-year review due dates and enter these dates into the new CERCLIS 3 Module. 3-3 After testing and validation of the CERCLIS 3 Module, monitor the status of five-year reviews and the recommended corrective actions established by completed reviews using the module and ensure they are completed by specified due dates. 15 ------- Agency Comments and OIG Evaluation EPA generally agreed with our recommendations. Although the Agency described corrective actions taken or planned to address the recommendations, it will need to provide more specific details on these actions as well as specific milestones for completing corrective actions for the recommendations within 90 days. With regard to Recommendation 3-1, the Agency commented that discussions on five-year review workloads and resources have been included in the annual planning process with the regions for the last few years and priority funding is provided to five-year reviews to ensure they are completed on time. We agree with the Agency's comment. However, we found that the Agency's current planning process has not been completely effective in assuring that sufficient resources are allocated to five-year reviews. Therefore, EPA will need to more effectively evaluate the regions' five-year review workloads and resource requirements during the planning process. The Agency said in response to Recommendation 3-2 that it has aggressively undertaken a data quality effort to ensure CERCLIS 3 reflects correct due dates for all planned five-year reviews. Regarding Recommendation 3-3, the Agency said that the need to promptly input five-year review report issues and recommendations into CERCLIS 3 was discussed with the regions and that regional branch chiefs agreed to review and verify progress for recommendations that affect protectiveness at least twice a year. We are encouraged by the Agency's comments on Recommendations 3-2 and 3-3. However, the Agency will need to provide more details on its corrective action for Recommendation 3-2. The Agency's corrective action for Recommendation 3-3 does not completely address the recommendation. Therefore, the Agency will need to provide a more comprehensive corrective action or actions for the recommendation in the response to the final report. The Agency disagreed with a recommendation in our draft report to require regions to establish contingency plans that help ensure that five-year reviews meet due dates when there are changes in remedial project managers. The Agency said that it does not believe the activity is significant enough to require the development of a contingency plan. The Agency also said the regions can plan for any foreseen transition of project managers. Based on these comments, we removed the recommendation from the report and modified Recommendation 3-1 to include consideration of remedial project manager changes as part of the annual five-year review planning process with the regions. The Agency's complete written response is in Appendix C. Our evaluation of those comments is in Appendix D. 16 ------- Chapter 4 Reviews Provide an Effective Oversight Mechanism, But More Data Needed Five-year reviews provide EPA with an effective mechanism to identify and address significant issues regarding the protectiveness of remedial actions for Superfund sites. However, EPA has not assessed the overall impact of its five- year review process on remedies at these sites. The Agency could not complete this assessment at the time of our review because a system to provide complete information on review results and recommended corrective actions had not been implemented. As a result, EPA has been unable to measure the effectiveness of its Five-Year Review program at assuring protection of human health and the environment. EPA's new CERCLIS 3 Module, once tested and fully implemented, should assist it with evaluating the effectiveness of the program in the future. Five-Year Reviews Identify Protectiveness Issues Five-year reviews provide EPA with a mechanism to identify and address significant issues that impact the protectiveness of remedial actions at Superfund sites. The reviews enable EPA to make recommendations for corrective actions, and formally communicate site conditions to Congress and the public. Our evaluation of 28 randomly selected five-year review reports showed that nearly two-thirds (64 percent) identified issues and provided recommended corrective actions at sites. These reports provided a total of 79 recommendations. Thirty of these recommendations addressed issues that potentially impacted current or future protectiveness of the remedial actions. For example, one report included a recommendation to conduct a remedial investigation/feasibility study to identify alternatives for a failing remedy. The other 49 recommendations addressed less significant issues that did not appear to have an impact on protectiveness. In total, 29 recommendations (37 percent) from the 28 reviews we examined had been implemented as of September 2005. Many of the corrective actions taken addressed important remedy issues at sites. Examples of actions taken included: • Optimization of ground water extraction systems. • Approval of alternative remedial actions that may be more efficient or cost effective in reducing site contaminants. • Identification of new contaminants of concern and modification of sampling programs. • Completion of studies to examine new potential exposure pathways. 17 ------- EPA Has Not Evaluated Overall Impacts of Five-Year Reviews Although five-year reviews are an important and required aspect of the Superfund program, EPA has not assessed the overall impact of the process on the implementation and performance of the remedies at sites. EPA had not done so because a system to provide complete information on the results of reviews and recommended corrective actions had not been implemented, but is expected to be by the end of Fiscal Year 2006. An OSWER manager and staff member agreed that five-year reviews were important. According to OSWER staff, the five-year review process provides an opportunity for reconsidering remedies proposed at similar sites based on failed, ineffective, or inefficient remedies identified as a result of five-year reviews. They also indicated the process assisted in discovery of new contaminants at some sites that otherwise could not or would not have been identified. Region staff indicated similar impacts, including the identification of new potential exposure pathways, and cited the public visibility of five-year reviews as an incentive for responsible parties to take corrective actions and avoid the negative public relations. The Comprehensive Five-Year Review Guidance establishes an annual reporting requirement intended to inform OSWER on national progress of five-year reviews. The guidance requires that each EPA region submit a report annually to OSWER that identifies key five-year review information and results, including: (1) the sites subject to five-year reviews for that fiscal year; (2) a summary of issues, recommended corrective actions, implementation schedule, and protectiveness determination(s) for each site; and (3) the status of recommended corrective actions for sites from previous fiscal years. However, OSWER waived the reporting requirement for annual five-year review progress reports from the regions until the Agency's new CERCLIS 3 Module is fully implemented. According to OSWER program staff, the new CERCLIS 3 Module, when fully implemented, will include the report information specified by the Comprehensive Five-Year Review Guidance and make the regions' annual reports unnecessary. Although the CERCLIS 3 Module has been under development since approximately 2002, it had not been fully implemented at the time of our review. OSWER staff expects the module to be fully implemented in Fiscal Year 2006. The new module, once fully tested and implemented, should provide the data to measure the effectiveness and impacts of EPA's five-year program. For example, the module should enable the Agency to measure the timeliness of reviews, number of reviews with protectiveness issues, timeliness of implementing recommended corrective actions addressing protectiveness issues, and actual/potential results from implementing recommended corrective actions. 18 ------- Recommendations We recommend that the Assistant Administrator for Solid Waste and Emergency Response: 4-1 Fully implement the CERCLIS 3 Module, after testing and validation, for the five-year review program. Require the regions to comply with the annual reporting requirement specified in the Comprehensive Five-Year Review Guidance by populating the module with reporting data. 4-2 Use CERCLIS 3 Module data to measure the effectiveness and impacts of EPA's five-year review program, such as measuring the timeliness of reviews, number of reviews with and without protectiveness issues, timeliness of implementing recommended corrective actions addressing protectiveness issues, and actual/potential results from implementing recommended corrective actions. Agency Comments and OIG Evaluation EPA agreed with our recommendations and responded that the CERCLIS 3 Module has been updated and the second generation module has been in production since June 2006. The Agency also said significant progress has been made entering data into the module for five-year reviews completed since Fiscal Year 2003, which allows OSWER to track issues, recommendations, and progress. The Agency described corrective actions intended to address the recommendations, but these actions did not completely address the recommendations. The Agency will need to provide more specific details on actions taken and/or planned, as well as specific milestones for completing the corrective actions, within 90 days. The Agency's complete written response is in Appendix C. Our evaluation of those comments is in Appendix D. 19 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s)2 Rec. No. Page No. Subject Status1 Planned Completion Action Official Date Claimed Amount Agreed To Amount 2-1 10 Expand the Agency's scope of the quality assurance reviews conducted under the initiative Improve the Quality and Consistency of Five-Year Review Reports. At a minimum, conduct quality assurance reviews for a representative sample of reports from each region. 2-2 10 Revise the Comprehensive Five-Year Review Guidance to more clearly define short- and long- term protectiveness determinations. Also, revise the guidance to include specific requirements for conducting and documenting quality assurance reviews of five-year review reports, and maintain documentation to support the information in the reports. 2-3 10 Communicate to the regions the need for: (a) public notifications for the commencement and completion of five-year reviews; and (b) protectiveness conclusions that address each operable unit at a site. 3-1 15 To assess discrepancies or gaps in resources, evaluate annual five-year review workloads and available resources as part of the annual planning process with the regions. Communicate to the regions that the U.S. Army Corps of Engineers or other contractor sources should be used for reviews where insufficient staffing or changes in remedial project managers will delay completion. 3-2 15 Work with region officials to correctly identify five- year review due dates and enter these dates into the new CERCLIS 3 Module. 3-3 15 After testing and validation of the CERCLIS 3 Module, monitor the status of five-year reviews and the recommended corrective actions established by completed reviews using the module and ensure they are completed by specified due dates. 4-1 19 Fully implement the CERCLIS 3 Module, after testing and validation, for the five-year review program. Require the regions to comply with the annual reporting requirement specified in the Comprehensive Five-Year Review Guidance by populating the module with reporting data. Assistant Administrator OSWER Assistant Administrator OSWER Assistant Administrator OSWER Assistant Administrator OSWER Assistant Administrator OSWER Assistant Administrator OSWER Assistant Administrator OSWER 20 ------- RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s)? Claimed Agreed To Amount Amount 4-2 19 Use CERCLIS 3 Module data to measure the 0 Assistant Administrator effectiveness and impacts of EPA's five-year OSWER review program, such as measuring the timeliness of reviews, number of reviews with and without protectiveness issues, timeliness of implementing recommended corrective actions addressing protectiveness issues, and actual/potential results from implementing recommended corrective actions. Planned Rec. Page Completion No. No. Subject Status1 Action Official Date 1 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is undecided with resolution efforts in progress 2 Identification of potential monetary benefits was not an objective of this evaluation. 21 ------- Appendix A Details on Scope and Methodology We conducted our evaluation from November 2004 to May 2006 in accordance with Government Auditing Standards, issued by the Comptroller General of the United States. Our scope generally covered EPA's five-year review activities during the period from Fiscal Years 2002 through 2004. To gain an understanding of EPA's Five-Year Review program, we interviewed managers and program staff from OSWER and Regions 2, 3, 4, 5, and 9. We also reviewed CERCLA, EPA's Comprehensive Five-Year Review Guidance, and other program documents and records applicable to EPA's five-year review activities. To assess management controls, we evaluated the Agency's quality assurance procedures and records for five-year reviews covering this three-year period. We found that the Agency needs to enhance quality assurance activities to improve the quality, completeness, and timeliness of five- year reviews. We evaluated five-year review procedures for Regions 2, 3, 4, 5, and 9. We selected these regions primarily because we sought to obtain geographical coverage for the eastern, central, and western sections of the nation. We made site visits to these regions and reviewed documents and records relevant to their five-year review activities. In addition, we interviewed managers and staff from OSWER and Regions 2, 3, 4, 5, and 9 to answer all three of our objectives presented below. Our evaluation work took into account the findings of the prior reports noted in Chapter 1. To evaluate how successful EPA has been at improving the quality and completeness of five- year reviews, we selected a random sample of 39 five-year review reports out of the universe of 555 completed during Fiscal Years 2002 and 2004 (see Table A-l on next page). The sample included at least 1 report for each of the 10 regions. We evaluated the content of these reports to determine whether the reports met EPA's Comprehensive Five-Year Review Guidance and had fully supported conclusions on remedy protectiveness. For the 28 reports from Regions 2, 3, 4, 5, and 9 in the sample, we also reviewed key site documents and data to verify the reports' technical assessment information and protectiveness conclusions. The supporting documents and data reviewed included Records of Decision, site sampling results, and site inspection records. 22 ------- Table A-1. Summary of Selected Details for Five-Year Review Reports in Random Sample Review Date Listed NPL Review Review Completion Review Region Site on NPL Status Type Sequence Date Lead 1 Beacon Heights Landfill 09/08/83 Final Statutory Third 09/19/03 EPA Plymouth Harbor/Cannon's 09/08/83 Deleted Policy Third 09/26/03 EPA Engineering Corporation 2 Higgins Farm 06/24/88 Final Policy First 09/29/03 EPA Caldwell Trucking 09/08/83 Final Statutory First 09/29/02 EPA Kin-Buc Landfill 09/08/83 Final Statutory Second 09/30/04 EPA Vestal Water Supply Well 1 -1 09/08/83 Final Statutory Second 09/30/03 EPA Chemical Insecticide Corporation 08/30/90 Final Statutory Second 12/29/03 EPA Old Bethpage Landfill 09/08/83 Final Statutory Second 09/30/02 EPA Naval Weapons Station Earle 08/30/90 Final Statutory First 02/09/04 Federal Facility 3 River Road Landfill 10/04/89 Deleted Statutory First 09/30/04 EPA Chem-Solv 08/30/90 Final Policy First 09/26/03 EPA Rhinehart Tire Fire 06/10/86 Final Statutory Second 11/06/02 EPA Tyson's Dump 09/21/84 Final Statutory Second 09/27/04 EPA Harvey and Knott Drum Site 09/08/83 Final Statutory Third 09/30/03 EPA Eastern Diversified 10/04/89 Final Statutory Second 02/12/03 EPA Woodbridge Research Facility Not on NPL Non NPL Federal Facility Statutory First 09/25/03 Federal Facility/ USACE 4 Lewisburg Dump 09/08/83 Deleted Statutory Second 09/11/02 EPA Beulah Landfill 02/21/90 Deleted Statutory Second 09/25/03 EPA/ USACE Geigy Chemical Corporation 10/04/89 Final Policy First 09/23/03 EPA/ USACE Sangamo Weston/Twelve Mile Creek/ 02/21/90 Final Statutory First 09/21/04 EPA Lake Hartwell Polychlorinated Biphenyls Contamination Site Harris Corporation (Palm Bay Plant) 07/22/87 Final Statutory First 02/03/04 EPA Fort. Hartford Coal Co. Stone Quarry 08/30/90 Final Statutory First 08/08/02 EPA/ USACE 5 Allied Chemical and Ironton Coke 09/08/83 Final Statutory Second 09/13/04 EPA Kummer Sanitary Landfill 06/10/86 Deleted Statutory First 03/13/03 State E.H. Schilling Landfill 09/08/83 Final Statutory Second 09/02/02 EPA Eau Clair Municipal Well Field 09/21/84 Final Statutory Second 09/27/02 EPA Springfield Township Dump 09/08/83 Final Statutory Second 09/24/04 EPA Velsicol Chemical Corporation 09/08/83 Final Policy Second 09/25/02 EPA 6 Fourth Street Abandoned Refinery 03/31/89 Final Statutory Second 07/29/02 EPA French Limited 09/08/83 Deleted Statutory Second 03/12/02 EPA 7 Oronogo-Duenweg Mining Belt Site 08/30/90 Final Statutory First 09/27/02 EPA Vogel Paint and Wax 06/10/86 Final Statutory Second 09/24/04 State Aidex Corporation 09/08/83 Deleted Policy Third 01/30/04 EPA Red Oak City Landfill 03/31/89 Final Statutory First 09/10/02 EPA 8 Mystery Bridge Road 08/30/90 Final Statutory Second 09/27/04 EPA Whitewood Creek 09/08/83 Deleted Statutory First 07/17/02 EPA 9 Fairchild Semiconductor- Mountain View Site 02/11/91 Final Policy First 09/30/04 EPA TRW Microwave 02/21/90 Final Policy Second 09/30/04 State 10 Standard Steel 08/30/90 Deleted Statutory First 04/23/03 EPA NPL National Priorities List USACE U.S. Army Corps of Engineers Source: Five-year review data provided by OSWER 23 ------- We also reviewed and evaluated tracking systems used by the five regions to monitor implementation of recommendations included in five-year review reports. The review included a review of available documents supporting the implementation status of recommendations included in the 28 reports in our random sample. Additionally, we reviewed quality assurance documents and records related to five-year reviews maintained by the five regions visited. To evaluate how EPA identifies sites requiring five-year reviews and ensures that the reviews are conducted timely, we tested EPA's process for identifying sites subject to those reviews for Fiscal Years 2002 through 2004. We obtained the universe of sites subject to five-year reviews during these fiscal years from CERCLIS data provided by OSWER. We did not test EPA's management controls for CERCLIS. However, we did test the accuracy of the universe through a comparison of the CERCLIS information with five-year review inventory information obtained from the five regions. We assessed the accuracy of five-year review dates in CERCLIS by comparing date information provided by the 39 reports in our sample with the review dates in CERCLIS. We also obtained input on the accuracy of the CERCLIS data for selected reviews from Agency managers and program staff. Additionally, we reviewed the CERCLIS data to identify reviews exceeding the five-year period specified by CERCLA and EPA policy. We classified reviews overdue by more than six months as being untimely. Our testing of the CERCLIS data did not reveal any significant discrepancies in the universe of five-year reviews. However, we found that some five-year review dates maintained in CERCLIS were inaccurate and that inaccurate dates may have contributed to untimely five-year reviews. This issue is discussed in Chapter 3. To evaluate impacts of EPA's five-year review process on the implementation and performance at Superfund sites, we identified impacts of five-year review activities through an assessment of the issues and recommendations in the 28 reports from Regions 2, 3, 4, 5, and 9 in our random sample. We also reviewed available documents supporting the implementation status of recommendations included in the reports. 24 ------- Appendix B Details on Reports without Fully Supported or Complete Protectiveness Conclusions Reports with Protectiveness Conclusions Not Fully Supported Eight out of 39 reports in our random sample presented protectiveness conclusions that were not fully supported. These reports either did not include enough information to support protectiveness conclusions, or presented information that did not support the conclusions. Plymouth Harbor/Cannon's Engineering Corporation - Region 1 The report identified that the site's perimeter fence must be reconstructed and maintained to control access to the property. Although the report disclosed that inadequate access controls were an issue impacting current protectiveness, the protectiveness conclusion stated that the remedy at the site was currently protective of human health and the environment. Higgins Farm - Region 2 The report's technical assessment stated that wells outside the remedial system were showing some degree of contamination and the remedy was not functioning exactly as intended. Also, a U.S. Army Corps of Engineers' optimization study for the site stated that "the offsite extent of the contaminant plumes is unknown and there is a potential for impacts on ground water users ... the protectiveness of the ground water remedies is unknown." Despite this information, the five- year review concluded that the remedial actions were protective. Chemical Insecticide Corporation - Region 2 The technical assessment for this report did not address all of the elements of the evaluation framework prescribed by the Comprehensive Five-Year Review Guidance. Although the site included three operable units, the five-year review was based primarily on a recent Record of Decision for one operable unit under construction. Therefore, the protectiveness conclusion was not based on an evaluation of data and information for all operable units at the site. Tyson's Dump - Region 3 The report's technical assessment did not address all of the elements of the evaluation framework prescribed by the Comprehensive Five-Year Review Guidance. The technical assessment and document review sections of the report did not provide sampling data and other specific information to support the evaluation and protectiveness conclusions. 25 ------- Geigy Chemical Corporation (Aberdeen Plant) - Region 4 The report stated the site remedy was protective of human health and the environment. However, the issues section disclosed that there was insufficient data to determine the impact of an off-site trichloroethylene contamination plume on the remedy. Further, interview information included in the report disclosed that an environmental engineer for the State of North Carolina's Department of Environment and Natural Resources had some concerns about the undefined source of the trichloroethylene contamination plume and its potential effects on the remedy. Therefore, information in the report noting a concern about the future protectiveness of the remedy appeared to contradict the protectiveness conclusion. E.H. Schilling Landfill - Region 5 The report concluded that the remedy was functioning as intended and protective of human health and the environment. However, the report identified erosion around the face of the dam onsite as a significant issue with the remedy. The report stated: "it was noted during the site visit that the erosion on the face of the dam was a serious issue that would potentially undermine the stability of the Recovery Well... and ultimately the dam itself." While this issue did not directly impact human exposure to contaminants at the site, the report disclosed that it affected both the current and future protectiveness of the remedial action. Therefore, this issue appeared to contradict the report's protectiveness conclusion. Fourth Street Abandoned Refinery - Region 6 The report's protectiveness conclusion included a statement that monitoring data suggested that the remedy was functioning as required and achieving ground water cleanup goals. However, the report also disclosed that site contaminants were present above action levels, and that data from current well locations may not provide sufficient information to assess the influence of off- site contaminant sources, horizontal migration of site-related contaminants, or natural attenuation of contaminants. This information appeared to contradict the protectiveness conclusion. Mystery Bridge Road - Region 8 The protectiveness statement concluded that the remedy was protective of human health and the environment because the contaminated ground water at the site was not used by residents. However, the report's technical assessment found that benzene concentrations in the ground water were a concern because they spiked up to 90 times the maximum contaminant levels during the first two years of the five-year review period. Further, the report disclosed that institutional controls prohibiting the use of ground water from the site were not in place even though they were required by the Record of Decision. Therefore, data and information presented in the report appeared to contradict the protectiveness conclusion. 26 ------- Reports with Incomplete Protectiveness Conclusions Eight out of 39 reports in our random sample had incomplete protectiveness conclusions. We considered the conclusions incomplete because they did not cover all operable units at the site and/or address both the short- and long-term protectiveness of the remedial actions. Old Bethpage Landfill - Region 2 The report's protectiveness conclusion was incomplete because it did not address both short- and long-term protectiveness of the remedy, despite identifying that a lack of institutional controls potentially affected the long-term protectiveness of the remedy. Because the report identified an issue affecting future protectiveness, the protectiveness conclusion should have addressed the lack of institutional controls and presented a statement for an operating remedy that was protective in the short term. Vestal Water Supply Well 1-1 - Region 2 The protectiveness conclusion for this report addressed the protectiveness of the remedies for the next five years, but did not address long-term protectiveness. The report's technical assessment identified that further evaluation of the soil vapor intrusion pathway was necessary. Given the recommendation for further evaluation of this pathway, future protectiveness of the remedies should have been addressed by the report's conclusion. Also, the report provided a site-wide protectiveness conclusion without addressing the protectiveness of each of the site's two operable units. Kin-Buc Landfill - Region 2 Although the site has multiple operable units in the construction complete phase, the report provided a site-wide protectiveness conclusion without addressing the protectiveness of each operable unit. Further, the report presented a protectiveness conclusion covering the next five- year period but did not specifically address long-term protectiveness of the remedy. Sangamo Weston/Twelve Mile Creek/Lake Hartwell Polychlorinated Biphenyls Contamination Site - Region 4 The protectiveness conclusion for this report only addressed one of the two operable units at the site. The five-year review did not address the remedy for Operable Unit 1 as part of the technical assessment even though this unit was the source for the contamination in Operable Unit 2. Velsicol Chemical Corporation - Region 5 The protectiveness conclusion for this report only addressed one of the two operable units at the site. Since both operable units have remedies either in place or under construction, the report should have addressed the protectiveness of the remedies for both operable units. 27 ------- Allied Chemical and Ironton Coke - Region 5 A protectiveness conclusion for each of the three operable units at this site was not presented in the report. The report only provided a site-wide protectiveness conclusion. Kummer Sanitary Landfill - Region 5 The report only provided a site-wide protectiveness conclusion. A protectiveness conclusion for each of the three operable units at this site was not presented in the report. French Limited - Region 6 The conclusion for this report did not address future protectiveness of the remedy. The conclusion covered both the source control and ground water remedies and generally addressed short-term protectiveness. However, results discussed in the report, such as ground water units not meeting compliance criteria and unsuccessful measures to direct the onsite ground water gradient, were not identified in the protectiveness conclusion as issues that could potentially impact the long-term protectiveness of the remedy. 28 ------- Appendix C Agency Response to Draft Report September 21, 2006 MEMORANDUM SUBJECT: OSWER Response to the Office of Inspector General draft report "EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed" FROM: Susan Parker Bodine/s/ Assistant Administrator TO: Bill A. Roderick Acting Inspector General Thank you for the opportunity to review and respond to the Office of Inspector General's (OIG) Draft Evaluation Report entitled "EPA Has Improved Five-Year Review Process for Superfund Remedies, But Further Steps Needed." We also appreciate the meetings with your staff to discuss your findings prior to issuance of the draft report. We agree that since the last review in 1999, EPA has taken actions to improve the five- year review (FYR) process. We issued the Comprehensive Five-Year Review Guidance, conducted training in all ten Regional offices and at NARPM meetings, significantly reduced the backlog of FYRs, and pursued the development and implementation of a tracking system for FYRs within the CERCLIS 3 application. We developed a comprehensive Five-Year Review web page (http://www.epa.gov/superfund/action/postconstruction/5yr.htm) which includes the "Comprehensive Five-Year Review Guidance;" an internet-based application that allows the public to search for Five-Year Reviews by state, site name, EPA ID, Region, keyword, or fiscal year; community involvement fact sheets; and "Questions and Answers" related to FYRs. We currently have a program review effort focused on documenting Regional processes for managing FYRs and implementing recommendations. The program review will summarize Regional management practices for tracking and conducting FYRs, implementation of FYR recommendations, and the usefulness of FYR guidance and training. In addition, the program review seeks to better understand Regional support needs and facilitate the sharing of FYR best practices nationwide. This program review is scheduled to be completed by the end of FY 2007. We also are taking steps to better support and communicate review findings, improve review timeliness, and provide greater assurance that cleanup actions are protective of human health and the environment. 29 ------- Our attached response includes comments on the findings, agreement/disagreement with the recommendations, a summary of our recent accomplishments and planned corrective actions, and an action plan for implementing the recommendations. For the sites identified in Appendix B, we provide clarifying language submitted by the Regions for selected sites that may alter the percentages in the OIG's final report. If you have any questions regarding this response, please contact Rafael Gonzalez at (703) 603-8892. Attachment 30 ------- EPA Response to OIG Recommendations Recommendation 2-1 Expand the Agency's scope of the quality assurance reviews conducted under the initiative Improve the Quality and Consistency of Five-Year Review Reports. At a minimum, conduct quality assurance reviews for a representative sample of reports from each region. See Appendix D Note 1 EPA Response We agree with the recommendation. As part of the initiative to "Improve the Quality and Consistency of Five-Year Reviews," we initially reviewed a representative sample of draft five- year reviews from the Regions. We have continued to expand the effort and recently have reviewed approximately half of all five-year reviews planned or due during the fiscal year. We will continue to increase the number of draft five-year reviews evaluated under the initiative to ensure compliance with the guidance and improve supporting information in the reports. We agree that increased review by Headquarters improves the quality and consistency of reports. Recommendation 2-2 Revise the Comprehensive Five-Year Review Guidance to more clearly define short- and long-term protectiveness determinations. Also revise the guidance to include specific requirements for conducting and documenting quality assurance reviews offive-year review reports, and maintain documentation to support the information in the reports. These requirements should emphasize the needfor: a) Information that provides complete support for conclusions on remedy protectiveness. b) Information supporting recommendations, including milestones, oversight agency, and responsible party. c) Documentation of supporting report information and quality assurance results. d) Explanations for excluding interviews of site personnel, regulatory officials, and local residents from the five-year reviews. e) Regional quality assurance programs for five-year reviews that ensure the reviews are complete and fully supported, including site inspections. See Appendix D Note 2 EPA Response We do not agree with the recommendation because we have several efforts currently underway that accomplish the same result. 31 ------- a/b) We are aggressively pursuing a refresher FYR training effort with the Regions, we have increased the number of draft FYR reports that we review, and have engaged in discussions with Regional managers to increase awareness of the need to have protectiveness determinations fully supported in the reviews. c) We agree that Regions should maintain documentation of support activities (i.e., site inspections and interviews) to verify these activities occurred. However, we do not agree that these need to be included in the FYR document. d) We disagree that the Regions should explain in the FYR document why interviews are omitted from the process. Interviews are not required by the guidance but are suggested as one of many community involvement activities to solicit feedback on the remedies being reviewed. e) During the last 18 months or so we have worked with the Regions to ensure all FYR documents were being reviewed for quality and completeness. We found that each Region has a document review process in place utilizing a variety of expertise from senior project managers, policy advisors, attorneys, risk assessors, community involvement coordinators, section chiefs, branch chiefs, and five-year review coordinators. Regions also routinely solicit feedback from State counterparts and other stakeholders. Recommendation 2-3 Communicate to the regions the needfor: (a) public notifications for the commencement and completion of five-year reviews; and (b) protectiveness conclusions that address each operable unit at a site. See Appendix D Note 3 EPA Response We agree with the recommendation. We recently met with Regional community involvement staff and managers and shared with them the list of planned and due FYRs for FY 2006 and FY 2007 to increase awareness and ensure that appropriate community involvement activities are conducted. In addition, in our training and Headquarters review, we are emphasizing that each operable unit should have a protectiveness conclusion. Recommendation 3-1 To assess discrepancies or gaps in resources, evaluate annual five-year review workloads and available resources as part of the annual planning process with the regions. Communicate to the regions that the U.S. Army Corps of Engineers or other contractor sources should be usedfor reviews where insufficient staffing will delay completion. See Appendix D Note 4 32 ------- EPA Response We agree with the recommendation. For the last few years we have included this discussion in our annual work planning process with the Regions. Each Region is provided a list of FYRs for the upcoming fiscal year and expected due dates. We review their requests for contractor funding, U.S. Army Corps of Engineers support, cooperative agreements with states, or whether they will conduct FYRs "in-house." We provide priority funding for FYRs to ensure they are completed on time. Recommendation 3-2 Require regions to establish contingency plans that help ensure that five-year reviews meet due dates when there are changes in remedial project managers. The plans should ensure all site data/information is timely transferred to newly assigned remedial project managers. See Appendix D Note 5 EPA Response We do not agree with the recommendation. We do not believe this activity is significant enough to require the development of a contingency plan. By providing an annual list of FYRs due to the Regions, managers can plan for any foreseen transition of remedial project managers. Recommendation 3-3 Work with region officials to correctly identify five-year review due dates and enter these dates into the new CERCLIS 3 Module. See Appendix D Note 6 EPA Response We agree with the recommendation. We have aggressively undertaken a data quality effort to ensure CERCLIS 3 reflects correct due dates for all planned FYRs. During the work planning sessions for FY 2007, Regions were also given a list of FYRs due over the next three years to verify due dates. For NPL Federal Facility reviews, we have sent letters to all Federal Agencies indicating when FYRs are due and have asked our Federal Facility Managers to review FYR information with their Federal Agency field personnel. 33 ------- Recommendation 3-4 After testing and validation of the CERCLIS 3 Module, monitor the status of five-year reviews and the recommended corrective actions established by completed reviews using the module and ensure they are completed by specified due dates. See Appendix D Note 7 Recommendation 4-1 Fully implement the CERCLIS 3 Module, after testing and validation, for the five-year review program. Require the regions to comply with the annual reporting requirement specified in the Comprehensive Five-Year Review Guidance by populating the module with reporting data. See Appendix D Note 8 Recommendation 4-2 Use CERCLIS 3 Module data to measure the effectiveness and impacts of EPA 's five- year review program, such as measuring the timeliness of reviews, number of reviews with and without protectiveness issues, timeliness of implementing recommended corrective actions addressing protectiveness issues, and actual/potential results from implementing recommended corrective actions. See Appendix D Note 9 EPA Response We agree with the recommendations 3-4, 4-1 and 4-2. The FYR CERCLIS 3 Module has been updated, and the second generation module has been in production since June 2006. Headquarters met with Regional Branch Chiefs to discuss issues related to FYRs, specifically the need to promptly input issues and recommendations identified in the reports into CERCLIS 3. Regional Branch Chiefs also agreed to review and verify progress for recommendations that affect protectiveness at least twice a year. The Regions are required to enter completion dates, issues and recommendations, and protectiveness statements into CERCLIS 3 in lieu of the annual reporting requirements specified in the guidance. Significant progress has been made entering data into the FYR CERCLIS 3 Module for FYRs completed since FY 2003. This allows us to generate reports that track issues and recommendations and the progress toward implementing them. Some of these reports have been made available to the Regions so they can closely monitor the status of recommendations, particularly those that impact protectiveness. 34 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Planned Completion Action Official Date Claimed Amount Agreed To Amount 2-1 10 Expand the Agency's scope of the quality assurance reviews conducted under the initiative Improve the Quality and Consistency of Five-Year Review Reports. At a minimum, conduct quality assurance reviews for a representative sample of reports from each region 2-2 10 Revise the Comprehensive Five-Year Review Guidance to more clearly define short- and long- term protectiveness determinations. Also revise the guidance to include specific requirements for conducting and documenting quality assurance reviews of five-year review reports, and maintain documentation to support the information in the reports 2-3 11 Communicate to the regions the need for: (a) public notifications for the commencement and completion of five-year reviews; and (b) protectiveness conclusions that address each operable unit at a site 3-1 15 To assess discrepancies or gaps in resources, evaluate annual five-year review workloads and available resources as part of the annual planning process with the regions. Communicate to the Regions that the U.S. Army Corps of Engineers or other contractor sources should be used for reviews where insufficient staffing will delay completion 3-2 15 Require regions to establish contingency plans that help ensure that five-year reviews meet due dates when there are changes in remedial project managers. The plans should ensure all site data/information is timely transferred to newly assigned remedial project managers 3-3 15 Work with region officials to correctly identify five- year review due dates and enter these dates into the new CERCLIS 3 Module 3-4 15 After testing and validation of the CERCLIS 3 Module, monitor the status of the five-year reviews and the recommended corrective actions established by completed reviews using the module and ensure they are completed by specified due dates. 4-1 18 Fully implement the CERCLIS 3 Module, after testing and validation, for the five-year review program. Require the regions to comply with the annual reporting requirement specified in the Comprehensive Five-Year Review Guidance by populating the module with reporting data Assistant Administrator OSWER Assistant Administrator OSWER Ongoing N/A N/A N/A Assistant Administrator OSWER Assistant Administrator OSWER Ongoing Ongoing N/A N/A Assistant Administrator OSWER N/A N/A Assistant Administrator Ongoing OSWER Assistant Administrator Ongoing OSWER Assistant Administrator Ongoing OSWER N/A N/A N/A 35 ------- RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Action Official Planned Completion Date 4-2 18 Use CERCLIS 3 Module data to measure the effectiveness and impacts of EPA's five-year review program, such as measuring the timeliness of reviews, number of reviews with and without protectiveness issues, timeliness of implementing recommended corrective actions addressing protectiveness issues, and actual/potential results from implementing recommended corrective actions Assistant Administrator OSWER Ongoing Claimed Amount Agreed To Amount N/A 1 0 = recommendation is open with agreed-to corrective actions pending; C = recommendation is closed with all agreed-to actions completed; U = recommendation is undecided with resolution efforts in progress 36 ------- Appendix B - Reports Without Fully Supported or Complete Protectiveness Conclusions Plymouth Harbor - Region 1 OIG Finding: The report identified inadequate access controls as an issue impacting current protectiveness. The report identified that the site's perimeter fence must be reconstructed and maintained to control access to the property. The report further stated that there was inadequate data to determine the risk at the site. Because the access controls to the site are not functioning, and there is inadequate data to calculate risk for the site, there is uncertainty in determining the current protectiveness of the remedy. However, the protectiveness statement in the report stated that the remedy at the site is currently protective of human health and the environment. See Appendix D Note 10 EPA Response: Although inadequate access controls is an issue that may impact protectiveness, the extent of the problem was not significant enough to affect the protectiveness determination at the time of the review. Most of the site was enclosed with proper access controls except for the northern portion which was only accessible by boat. Since then, the northern portion has been closed off. Also, the 5YR pointed out that the risk assessment was conservative using the highest cPAH data available which overstated the estimated risk. This conservative approach should be re-evaluated because it may impede future development. Tyson's Dump - Region 3 OIG Finding: The report's technical assessment did not address all of the elements of the evaluation framework prescribed by the Comprehensive Five-Year Review Guidance. The technical assessment and document review sections of the report did not provide sampling data and other specific information to support the evaluation and protectiveness conclusions. See Appendix D Note 11 EPA Response: The sampling data supporting the conclusions are referenced on page 16 which states, "The results of the 2003 Site-Wide Monitoring Program (SWMP) demonstrate that groundwater pump and treat as a remedy for the deep aquifer is working..." Page 17 discusses this conclusion which includes reductions in contamination over a seven year period, decreasing trends of site-related contamination, and quarterly monitoring results from the adjacent river which show it is "free of site-related compounds". Geigy Chemical Corp. (Aberdeen Plant) - Region 4 OIG Finding: The report stated the site remedy was protective of human health and the environment. However, the technical assessment section disclosed that there was insufficient data to determine the impact of an off-site trichloroethylene contamination plume on the remedy. Further, interview information included in the report disclosed that a representative of the 37 ------- primary responsible party for the site said increased trichloroethylene levels could require changes to the remedial design for the site. Therefore, information in the report noting a concern about the future protectiveness of the remedy appeared to contradict the protectiveness conclusion. See Appendix D Note 12 EPA Response: The remedy design changes for addressing TCE referenced in the document relate to using larger activated carbon filters or potentially adding air strippers. Remedies often undergo design modifications, and we do not believe that the potential modification requires a non-protective conclusion. Sangamo Weston/Twelve Mile Creek/Lake Hartwell Poly chlorinated Biphenyls Contamination Site - Region 4 OIG Finding: The report concluded that the remedy for the site's Operable Unit 2 was adequately protective of human health and the environment, and prescribed long-term monitoring at the site to assure future protectiveness. However, data and information presented in the report and supporting documents indicated the remedy was not completely functioning as intended. Information and data in the report and supporting documents identified that fish contamination levels for some species had not changed, indicating that the remedy for the site's Operable Unit 1 was not completely addressing the source of contamination for Operable Unit 2. Also, the five-year review and report did not address the remedy for Operable Unit 1 as part of the technical assessment, even though this unit was the source for the contamination in Operable Unit 2. See Appendix D Note 13 EPA Response: The remedy at Operable Unit 2 focuses on human health consumption of contaminated fish and monitored natural recovery of sediments. As part of the five-year review, EPA conducted surveys and collected blood samples for PCBs from nearby residents. Results of the surveys showed that the fish advisories and public outreach programs were working. Also, results from the blood tests showed that participants had blood levels at or below the mean of the U.S. population. The report also clearly indicated that sediment PCB levels are decreasing through monitored natural recovery. A five-year review was conducted for Operable Unit 1 in FY 2005 and was not intended to be included in the FY 2004 review. 4th Street Abandoned Refinery - Region 6 OIG Finding: The report concluded that the remedy was expected to be protective upon completion, and in the interim all exposure pathways were being controlled. According to the report, monitoring data suggested that the remedy was functioning as required and achieving ground water cleanup goals. However, the report also disclosed that data from current well locations did not provide sufficient information to assess the influence of off-site contaminant 38 ------- sources, horizontal migration of site-related contaminants, or natural attenuation of contaminants. The report indicated that a lack of data to assess those issues was grounds for considering the remedy to be failing. This information appeared to contradict the report's statement that the remedy was functioning as required. See Appendix D Note 14 EPA Response: The report identifies potential factors that may assist in the evaluation of the remedy as it operates. The Region and the State considered these observations as areas of concern and integrated them into an on-going monitoring program to ensure the remedy is working as intended. The statements do not imply that the remedy is not working. Appendix B - Reports With Incomplete Protectiveness Conclusions Beulah Landfill - Region 4 OIG Finding: The report's protectiveness conclusion did not address the actions needed to ensure the future protectiveness of the remedy. The report stated that the ground water at the site was not in compliance with Florida's drinking and surface water standards, which was necessary to satisfy the closure requirements for the site specified in the Record of Decision. According to the report, the State was working on a remedial action plan to address this issue. However, the protectiveness conclusion did not discuss what actions are necessary to remediate any ground water that poses a threat to surface waters, and ensure future protectiveness for human health and the environment. See Appendix D Note 15 EPA Response: The recommendation to develop a remedial action plan is sufficient to meet the five-year review requirement. The protectiveness conclusion did not include the specific actions necessary to address the issue because the appropriate response action plan was being developed by the State and the PRP at the time of the review. 39 ------- Appendix D OIG Evaluation of Agency Response Note 1 - The Agency's corrective actions appear to adequately address the recommendation. However, the Agency will need to be more specific on the scope of quality assurance reviews for Fiscal Years 2007 and later in its response to the final report. Note 2 - The Agency's comments describe actions taken and planned that appear to generally address parts 2-2(a) and 2-2(b). However, the Agency will need to provide additional details in its response to the final report that explain how the refresher training, the increase in the number of draft five-year review reports reviewed by OSWER, and discussions with the regions will improve the report information supporting protectiveness conclusions and recommendations. Regarding the Agency's comments for part 2-2(c), we are encouraged that the Agency agrees that the regions should maintain documentation supporting interviews. We agree that the documentation does not need to be included in five-year review reports. The Agency will need to include specific corrective actions taken and/or planned to address this part of the recommendation, including documentation supporting public notifications and quality assurance reviews, in its response to the final report. We agree with the Agency's comments on part 2-2 (d). We removed the "Reviews Did Not Include Important Verification Activities" section in Chapter 2 as well as the applicable recommendation from the report. We are encouraged by the Agency's comments for part 2-2 (e). This corrective action appears to meet the intent of our recommendation. However, the Agency will need to provide more specific details about its work with the regions to improve the regions' quality assurance reviews in its response to the final report. As discussed in our report, the regions' quality assurance reviews had not been effective in ensuring the quality and completeness of five-year reviews as of the end of Fiscal Year 2004. The Agency will also need to include milestones for completing the corrective actions for each part of Recommendation 2-2 in the response to the final report. 40 ------- Note 3 - We agree with the actions taken and planned by the Agency. EPA will need to include specific milestones for completing these actions in its response to the final report. Note 4 - The Agency's agreement with the recommendation is encouraging. We acknowledge that OSWER has discussed five-year workloads with the regions during annual planning meetings. However, large workloads for remedial project managers and a turnover of those managers for some sites were primary contributors to untimely five-year review reports. Therefore, OSWER's current planning process has not been completely effective in assuring that sufficient resources are allocated to five-year reviews. In its response to the final report, the Agency will need to describe the actions taken or planned during the annual planning process to better ensure the regions have sufficient resources to timely complete five-year reviews. OSWER will also need to include specific milestones for completing these actions. Note 5 - We agree that Agency managers can plan for any foreseen transition of remedial project managers, and have removed the recommendation from the report. Recommendation 3-1 addresses the Agency's annual five-year review planning process. Therefore, we have modified Recommendation 3-1 by recommending the use of contractor resources for reviews when changes in remedial project managers will delay completion. Note 6 - We are encouraged by actions taken and underway by the Agency. To completely address the recommendation, EPA will need to describe the data quality assurance efforts taken to correct due dates in the CERCLIS 3 Module prior to Fiscal Year 2007 in its response to the final report. The Agency will also need to include in its response specific milestones for completing actions taken and underway. Note 7- We are encouraged that the Agency agrees with the recommendation and the regions have agreed to review and verify progress for recommendations addressing protectiveness issues twice a year. However, the Agency's comments do not describe actions taken and/or planned by OSWER to monitor timely completion of five-year reviews and implementation of recommendations established by five-year reviews. The Agency will need to provide additional details on action taken and/or planned to address the recommendation as well as a milestone for completing each action in its response to the final report. Note 8- Although the Agency agrees with the recommendation, its comments do not disclose whether the regions are required to populate the CERCLIS 3 Module with all the reporting data specified in the Comprehensive Five-Year Review Guidance. For example, the Agency does not identify whether report data specified by the guidance, such as implementation schedules for recommended corrective actions, are required to be populated in CERCLIS 3. The Agency 41 ------- will need to provide additional details on actions taken and/or planned to address the recommendation as well as a milestone for completing each action in its response to the final report. Note 9- The Agency's agreement with the recommendation is promising. We are also encouraged that the Agency has made significant progress in entering data into the CERCLIS 3 Module, which allows EPA to generate reports that track issues, recommendations, and progress. However, the Agency's comments do not indicate whether specific data in CERCLIS 3 will be used to measure the effectiveness and impacts of the five-year review program. Therefore, EPA will need to provide more specific details on corrective actions taken and/or planned as well as milestones for completing the actions in its response to the final report. Note 10- The report specifically identifies the inadequate access controls as an issue that impacts the current and future protectiveness of the site, yet the report provides a protectiveness statement that cites the site is currently protective. We agree with the Agency's comments on the risk assessment after further review of the five-year review report, and we made appropriate revisions to the paragraph. Note 11 - The data review section on pages 16 and 17 of the five-year review report provide only general conclusions taken directly from page 12 of the 2003 Annual Monitoring Report for the Tyson Superfund Site. The report does not provide any sampling results or any further details to support the evaluation and protectiveness conclusions. Further, the document review section of the five- year review report does not cite any monitoring or sampling reports that cover the five-year review period. Note 12 - The five-year review report states that increasing trichloroethylene contamination levels have shortened the life of carbon absorption canisters in the treatment facility; however, the long-term effects of this change on the remedy is unknown. Groundwater treatment at the site is achieved by removal of the contaminants of concern through these carbon absorption canisters. Therefore, the carbon absorption canisters are critical components of the groundwater remedy. Given the lack of certainty regarding the impact of trichloroethylene contamination on the remedy's effectiveness, it is unclear from the details presented in the five-year review report how the protectiveness conclusion was reached. We made revisions to the paragraph for clarity. Note 13 - We agree that the protectiveness conclusion for Operable Unit 2 was adequately supported based on the Agency's comments and further review of the five-year review report. However, the five-year review report provides a site-wide protectiveness conclusion without addressing Operable Unit 1 in the technical assessment section. Therefore, we reclassified the report's protectiveness conclusion from "not fully supported" to "incomplete." We made appropriate revisions to Chapter 2 and Appendix B. 42 ------- Note 14 - We agree that the five-year review report identifies early indicators of potential remedy failure in the technical assessment that assist in the evaluation of the remedy. The five-year review report identifies that site contaminants were present above action levels, and that data from current well locations may not provide sufficient information to assess the influence of off-site contaminant sources, horizontal migration of site-related contaminants, or natural attenuation of contaminants. This information appears to contradict a statement in the protectiveness conclusion disclosing that the remedy was functioning as intended and was achieving ground water cleanup goals. We modified the paragraph for clarity. Note 15 - We agree with the Agency after further review of the five-year review report. We made appropriate revisions to Chapter 2 and removed the paragraph in question from Appendix B. 43 ------- Appendix E Distribution Office of the Administrator Assistant Administrator, Office of Solid Waste and Emergency Response Agency Followup Official (the CFO) Agency Followup Coordinator Audit Followup Coordinator, Office of Solid Waste and Emergency Response Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for Public Affairs General Counsel Acting Inspector General 44 ------- |