United States Environmental Protection 401 M St., S.W. Agency Washington, D.C. 20460 May 1985 Emergency and Remedial Response SEPA ASSESSING THE SUPERFUND REMEDIAL INVESTIGATION AND FEASIBILITY STUDY PROCESS 1 ------- ASSESSING THE SUPERFUND REMEDIAL INVESTIGATION AND FEASIBILITY STUDY PROCESS FINAL REPORT Prepared for: Thomas Sheckells Hazardous Site Control Division Office of Emergency and Remedial Response United States Environmental Protection Agency Washington, D.C. Prepared by: ICF Incorporated 1850 K Street, N.W. Suite 950 Washington, D.C. 20006 Task Number 17 EPA Contract Number 68-01-6872 This contract was awarded competitively. May 1985 ------- ACKNOWLEDGEMENTS This report was prepared by ICF Incorporated under Delivery Order Number 17 of Contract Number 68-01-6872. The EPA Task Managers were Thomas Sheckells and Anthony Diecidue of the Hazardous Site Control Division and Patricia Cohn of the Policy Analysis Staff, Office of Emergency and Remedial Response. The ICF staff for this project included Marc Tipermas, Project Manager, and Abigail Amsterdam, Erik Beecroft, Jason Cheever, Charles Debelius, James Duffy, Michael Goldman, Jim Janis, Marc Lieber, Peter Linquiti, Robin Sandenburgh, Barbara Thegze, Carol Trippe, Zella Williams, and Jim Woldahl. We would like to acknowledge the cooperation of EPA headquarters and regional staff who provided the valuable information used to develop the report. ------- ASSESSING THE SUPERFUND REMEDIAL INVESTIGATION AND FEASIBILITY STUDY PROCESS EXECUTIVE SUMMARY EPA's Hazardous Site Control Division (HSCD) conducted this study to document the time required, and causes of delays encountered, i:r performing key activities for remedial investigations and feasibility studies (RI/FS). The study included two phases: Phase I: Collection and analysis of quantitative data on RI/FS milestone activities for 139 federal-lead sites and 67 state-lead sites with work assignments occurring between 1982 and 1984; and Phase II: Site reviews of 26 sites to gather qualitative information on sources of RI/FS delays. At the completion of Phase II, HSCD developed specific recommendations regarding possible methods to reduce RI/FS delays at future sites. PHASE I: DATA ANALYSIS EPA headquarters remedial staff and the zone contractors provided site data, which regional staff latBr verified to the extent possible.. Major findings for federal-lead sites include the following: Average total duration from work assignment to submission of final FS report is 587 days (over 19 months). An average of over six months is required for developing the work plan and obtaining final authorization to begin field work. The interval from work assignment to final authorization averages 31 percent of total RI/FS duration. The interval from final authorization to actual or current estimated FS completion averages 69 percent of total RI/FS duration. Average overall slippage is 141 days and most of this slippage occurs during the RI. Slippage estimates for federal-lead sites may underestimate actual slippage, particularly for RI/FS with planned completion dates in calendar year 1985. Average slippage for sites with a planned completion date before December 10, 1984 is over six months, but the corresponding average slippage for sites with completion dates after December 10, 1984 is less than three months. These findings raise concerns about RI/FS projected for completion in FY85 and FY86, and for budgeting of remedial design and construction funds. ------- ES-2 Major findings for state-lead sites include the following: Average total duration is 680 days (23 months). Slippage averages nearly five months, but constraints on the size of the sample (N=13) require that one use caution in drawing conclusions from these results. PHASE II: SITE REVIEWS Phase II addressed many questions that were raised in Phase I regarding RI/FS delays and slippages. Major common sources of delay include: Allocation and turnaround in the Contract Laboratory Program (CLP); Review of CLP data results by regional Environmental Services Divisions (ESDs); Work plan development; Technical performance of contractors and subcontractors; Planning and communication among headquarters, the region, and the contractor.; State review of reports and work plans; Late subcontracting decisions; Headquarters policy and guidance to regional and state staff; Headquarters processing of documents; Inadequate regional management tracking and oversight; and Unique or unavoidable factors such as weather and unexpected findings. MANAGEMENT IMPROVEMENTS TO DATE HSCD has already instituted several policy changes based on Superfund program experience of the past four years. The new policies and procedures include: Improved work assignment and award fee procedures; Efforts, including a task force, to increase laboratory capacity and reduce turnaround time; ------- ES-3 Delegation of contract-related paperwork to the regional offices; Training workshops for state personnel; Class deviations from procurement regulations for state-lead s ites; Multi-site cooperative agreements; and Guidance to states on zone, pre-authorized, and prequalified contractors. Regional staffs are instituting RI/FS tracking systems so that they may stay apprised of site progress and notify headquarters and regional management as soon as problems arise. OPTIONS FOR FURTHER IMPROVEMENT This study indicates that EPA headquarters and regional staff, contractors, and state personnel may all improve performance in specific areas to reduce RI/FS delays. The site reviews indicate that most delays are not caused by uncontrollable or unexpected factors. Recommendations for headquarters include the following: Shorten CLP turnaround time, by increasing l'aboratory capacity, upgrading sample flow management, and defining data-quality objectives; Shorten data review time by increasing data review staff, expanding reliance on CLP staff expertise, increasing the role of the SMO, adjusting CLP delivery criteria, increasing automation of data review, and providing contractor access to unreviewed data; Increase training for RSPOs; and Improve oversight of the RI/FS process with the assistance of a management tracking system. Recommendations to regional management include: Identify critical path tasks, and perform other RI/FS activities during delays in these tasks whenever feasible; Improve communication with and management of contractors and subcontractors; Expand general headquarters guidance, including efforts to train RSPOs; Improve tracking of RI/FS activities; and ------- ES-4 Increase communication with state site project managers, regional ESDs, and health and safety review staff. Recommendations to state staff include: Decrease time for procurement and review of work plans and reports; and Improve communication with regional staff. Recommendations to contractors include: Improve subcontractor management; Increase communication with regional staff; and Improve scheduling and tracking of critical path tasks. ------- ASSESSING THE SUPERFUND REMEDIAL INVESTIGATION AND FEASIBILITY STUDY PROCESS 1. INTRODUCTION This report presents the results of a study of Superfund remedial investigation and feasibility studies (RI/FS). The purpose of the study is to document the time required, and the possible reasons for delays, in accomplishing RI/FS milestones such as completion of work plans, remedial investigation activities, and feasibility study reports. By identifying sources of delay, EPA can learn from Superfund program experience to date and develop management techniques to reduce delays. The study included two phases. During Phase I, the Hazardous Site Control Division (HSCD) collected cost figures and dates of major RI/FS milestones such as work assignment, final authorization, remedial investigation (RI) and feasibility study (FS) planned completion, and actual or current estimated completion date for the RI and FS. The information was analyzed to determine the average duration for each milestone and for the entire RI/FS. During Phase II, HSCD staff and ICF Incorporated visited regional offices to review RI/FS at selected sites, identify causes of delay, and recommend management steps to reduce delays. The data for Phase I were provided by EPA headquarters remedial staff and the two zone contractors, NUS Corporation and CH2M-Hill, and were verified by regional staff. ICF Incorporated calculated average durations between milestones and costs. Some of the data accuracy could be improved through additional verification by regional and headquarters staff. The results presented in this report therefore may be revised pending such verification. Following are the major findings from Phase I: The average federal-lead RI/FS lasts over 19 months, from work assignment to submission of the final FS report; Over six months on average are required for developing a work plan and granting final authorization to proceed with field work; Slippage in completing the RI accounts for over five months on average compared to the schedule in the final work plan; Slippage estimates for federal-lead sites may underestimate actual slippage, particularly for RI/FS planned for completion in the coming year. Average slippage for sites with a planned RI/FS completion date before December 10, 1984 (date of data collection) is more than 6 months, but the corresponding average for RI/FS to be completed after December 10, 1984 is less ------- - 2 - than three months. These findings raise concern about projects currently projected to be completed in FY85 and FY86, and the amount of funds to be budgeted for remedial design and construction. The average state-lead RI/FS lasts nearly 23 months. Slippage for state-lead RI/FS averages nearly five months, but the sma.> ! number of sites in the data base (N=13) indicates the ieed for caution in interpreting these results. Appendix A presents the findings on duration and cost from Phase I in more detail. The Phase I analysis raised many questions that were addressed in Phase II. The Phase II site reviews attempted to collect information, for example, on why it takes six months to plan an RI/FS, and what events cause average delays of five months in the RI. This report presents the findings of the site reviews. The major finding of the site reviews is that no single cause or participant bears primary responsibility for the delays that have occurred in RI/FS. EPA headquarters and regional staff, contractors, and states all need to improve performance in specific areas to reduce RI/FS delays. Moreover, the site reviews indicated that most delays are not caused by uncontro-1 lable or unexpected events. Although some sites experienced significant delays due to weather and unexpected findings, improved planning and communication can likely reduce most delays. The site reviews also identified several sites that were completed relatively quickly as a result of superior effort, planning, communication, and technical knowledge, including Mountain View Mobile Home Estates (federal-lead) and Highlands Acid Pit (state-lead). The remainder of this report includes the following sections: Section 2 presents the findings or causes of delay from the Phase II site reviews; and ° Section 3 presents implications for management improvements to reduce RI/FS delays. ------- - 3 - 2. SITE REVIEW RESULTS: CAUSES OF RI/FS DELAYS EPA's Hazardous Site Control Division (HSCD), assisted by ICF Incorporated, conducted detailed reviews of federal- and state-lead hazardous waste sites in several regions in order to document and analyze the RI/FS process and identify areas of potential time savings. In selecting sites to be reviewed, HSCD sought sites that could provide general lessons on the RI/FS process, and avoided RI/FS that were among the first performed in the Superfund program, or that experienced highly unusual delays during the RI/FS. HSCD attempted to choose sites that were representative of all RI/FS projects, including a variety of contaminant problems and facility types, at which most of the RI/FS activities had been completed. To date, HSCD has conducted a total of 26 site reviews. Exhibit 1 presents the sites chosen for review. The statistical analysis in Phase I identified average durations for the RI/FS planning and work phases, including slippage caused by delays in site progress. The purpose of the site reviews was to explain the data results by identifying causes of the delays. The site reviews identified several common sources of delay causing significant slippage at all or most of the sites. Based on the site reviews, this section identifies and discusses sources of delay in the RI/FS process. Section 2.1 describes the major common sources of delay at sites, including the following: Allocation and turnaround in the Contract Laboratory Program (CLP); Review of CLP data results by regional Environmental Services Divisions (ESDs); Work plan development; Technical performance of contractors and subcontractors; Planning and communication among headquarters, the region, and the contractor; State review of reports and work plans; Late subcontracting decisions; Headquarters policy and guidance to regional and state staff; and Headquarters processing of documents. Section 2.2 briefly discusses unique, uncontrollable causes of delay such as inclement weather or unexpected findings, that resulted in significant slippages in the RI/FS schedules. ------- - 4 - Region I II III IV V VI EXHIBIT 1 RI/FS SITES REVIEWED Site Lead Type Hocomonco Pond, MA Federal Nyanza, MA Federal McKin, HE State GEMS Landfill, NJ Federal Swope Oil, NJ Federal Burnt Fly Bog, NJ State Olean Wells, NY State Marathon Battery, NY State Spence Farm, NJ State Drake Chemical, PA Heleva Tyson's Dump, PA Distler Brickyard, KY Pepper's Steel and Alloys, FL Whitehouse Waste Oil Pits, FL Federal Federal Federal Federal Federal State Fields Brook, OH Federal Cross Bros., IL State Rose Township, MI State Cecil Lindsey, AK Federal Cleve Reber, LA Federal Highlands Acid Pit, TX State Old Inger, AK State Del Norte, CA Federal Iron Mountain Mine, CA Federal Purity Oil, CA State Mountain View Mobile Home Estates, AZ Federal Total sites Federal-lead State-lead 26 15 11 ------- - 5 - 2.1 COMMON SOURCES OF DELAY AT RI/FS SITES 2.1.1 CLP Allocation and Analysis Virtually every federal-lead site encountered delays in CLP laboratory processing of site samples. This is a serious concern, because laboratory results are a "critical path" item, i.e., other site work and preparation of the RI report cannot proceed or be completed until the contractor receives sample analysis results. One of the problems cited frequently by regional site project officers (RSPOs) is that of obtaining CLP allocations, particularly during the summer months. In several instances, the regional sample control center in the region's ESD notified the RSPO shortly prior to the scheduled sampling start date that CLP laboratories would not be able to accept samples and that the ESD would continue its attempts to secure an allocation at some future date. For example, at Cleve Reber, on-site investigation had been tentatively scheduled to begin on May 25, 1984. At that time, the RSPO was notified that sample shipments from Cleve Reber could not be accepted during May or June, due to competing demands from other sites across the country. The RSPO said that because the delay was indefinite, the subcontractor reassigned Cleve Reber site personnel to other projects. The RSPO was informed late in June that CLP allocation space was available during July, but that all sampling would have to be completed by July 31. Site contractor staff had to be mobilized immediately to meet the deadline and the subcontractor had to spend a few additional days orienting a new site team, including the primary project engineer. The RSPO estimated that the CLP allocation system was thus responsible for a delay of at least five weeks and a slight cost escalation due to the demobilization. At Cecil Lindsey, the RSPO said that the site was mobilized and demobilized on three occasions due, in part, to CLP allocation delays; moreover, the RSPO said that sampling activities that could have been completed in a two-week period were spread over six weeks in order to keep within the region's CLP allocation. Similarly, sampling was delayed three weeks at Distler Brickyard and four weeks at Rose Township until CLP allocation space was available. At Drake Chemical, in part because a very large number of samples were collected, CLP could not absorb a sudden increase in demand. Several samples were stored for up to three months awaiting analysis. Once the contractor delivers samples to the CLP laboratories, further delays frequently occur in laboratory turnaround, specifically in completion of laboratory analysis and shipment of results to the ESD. According to the RSPOs, the CLP laboratories often exceed the one-month deadline specified in CLP contracts for performing sample analyses and forwarding results t~o the appropriate EPA data review staff. In an extreme case, CLP turnaround required over five months for Nyanza, and the RSPO said that he was finally forced to drop one sampling round and allow the contractor to begin the RI report with unvalidated data. CLP turnaround was three months at both Rose ------- - 6 - Township and Fields Brook, according to the RSPOs. The CLP laboratory took two months, or twice as long as was scheduled, to complete the analysis of samples from Hocomonco Pond. Similarly, according to the RSPO, analysis of samples from Iron Mountain Mine was completed two weeks later than originally scheduled. In comparison with the CLP difficulties, state-lead sites in general have had a much more satisfactory experience with privately contracted laboratories. The firm with which the state of Louisiana contracted to perform laboratory analysis for Highlands Acid provided sample turnaround within six to eight weeks. Moreover, when two additional samples were taken, the RSPO said that the laboratory was again able to accept, analyze, and return verified results in under eight weeks. In the case of McKin, the Maine state environmental laboratory was able to provide results two weeks after samples were taken. Such reliable and relatively rapid turnaround assisted the state contractor in making key decisions in the RI, permitting the elimination of some areas of investigation and confirming the need for others. With the exception of Spence Farm, which encountered laboratory capacity delays similar to those experienced at federal-lead sites, the state-lead sites have not regularly been hindered by the two most common CLP problems that plague the federal-lead sites. First, and most serious, the lack of available CLP slots often delays site work so that essential elements of the RI are postponed indefinitely. Such delays have led to added costs for site demobilization and mobilization, 'and have delayed sampling work. Second, turnaround time for the analysis was typically shorter for state- than for federal-lead RI/FS. 2.1.2 Quality Assurance of Laboratory Data Results Regional ESD staff, often assisted by the field investigation team (FIT) contractors, are responsible for reviewing, validating, and interpreting the raw data results from the laboratory analyses. In the past, most regions forbade the contractor to proceed with the RI using unvalidated data. Significant delays in RI/FS data review have compounded the problem of late laboratory analysis at almost every federal-lead site reviewed. Validation delays of one to two months have been common, although work plans generally allot two weeks for this procedure. According to the respective RSPOs, quality assurance of data for Iron Mountain Mine, Nyanza, and Distler Brickyard took approximately one month each, and validation of Hocomonco Pond samples required two months, despite an RI schedule estimating that it would take three weeks. In the most extreme case, data validation took one year at Fields Brook in Region V. This delay was caused primarily by a lack of staff and contractor data review resources, combined with repeated regional management decisions to postpone data review so that the resources could be devoted to higher-priority sites. 2.1.3 Vork Plan Development The statistical analysis of the RI/FS program data for federal-lead sites showed that the national mean duration from work assignment to final authorization is more than six months. Three factors in particular appear to ------- - 7 - have contributed to delays during the planning phase: contractor preparation of planning documents, state and regional review, and subcontractor procedures. Delays due to procurement of subcontractors and cooperative agreement and contract document review appear to be relatively minor problems for most of the sites, and are discussed in a later section. In almost every site studied, the work plan and the health and safety, quality assurance/quality control (QA/QC), and sampling plans prepared by the contractor or subcontractor required multiple drafts, sometimes adding as much as three months to the RI/FS planning phase. Draft documents often suffered from technical inaccuracies, omissions, or imprecise statements. For example, the RSPO at Cleve Reber said that the subcontractor submitted a total of seven draft work plans over a period of three months; the RSPO noted that intermediate drafts contained revisions he had not requested and failed to resolve financial and technical concerns that he had mentioned in his review comments. The RSPO at Cecil Lindsey said that the initial sampling plan failed to specify on-site responsibilities; the subcontractor did not submit a final sampling plan until six weeks later. The Nyanza RSPO said that the contractor included a drinking water analysis in the work plan, although the water affected by hazardous wastes was not used for drinking; the contractor corrected this and other technical discrepancies in the final work plan, submitted two months later. Work plan development has also been a source of slippage at state-lead sites. The EPA RSPO indicated that the state's contractor for McKin spent nine weeks revising the health and safety and QA/QC plans to incorporate state and regional concerns about coordination of laboratory analysis and the precise location of the decontamination area. The Region II RSPO for Burnt Fly Bog and the Region V RSPO for Rose Township said that draft QA/QC, sampling, health and safety, and drilling plans all required extensive revisions by the contractor, delaying on-site activities by several months. One possible explanation for this type of problem is that current procedures may not, in general, give the contractor (and particularly, new contractors) sufficient direction and strong enough incentives to prepare correct documents at the outset, necessitating lengthy revision periods until the contractor submits acceptable site plans. Regions commonly authorize contractors to begin preliminary tasks while completing preparation of a final work plan. The purpose of this practice is to permit immediate initiation of RI/FS field work by beginning certain tasks necessary to perform field work. For example, before taking samples, the contractor often must prepare a health and safety, quality assurance, and sampling plan. By completing these tasks early, field work can begin immediately after the final work plan is approved. To the extent that the contractor begins these preliminary tasks, delays in field work are avoided. In none of the sites reviewed, however, were interim tasks completed before final authorization and approval of the final work plan. In the opinion of RSPOs and headquarters staff, there is nothing wrong with completing preliminary tasks after final authorization. Rather, any tasks initiated before final authorization are seen as saving time. However, any benefit from beginning and completing initial tasks early and saving time in field work ------- - 8 - must be weighed against the risk, of completing tasks before final authorization, which would result in contractor downtime. HSCD's revised work assignment procedures, discussed in Section 3.1, appear to promote early completion of initial tasks. 2.1.4 Technical Performance of the Contractor or Subcontractor In numerous cases, subcontractors have delayed the RI/FS because of technical errors and inexperience. The contractor must provide sufficient guidance and supervision throughout the RI/FS to ensure that subcontractors complete tasks in a timely manner and according to work plan specifications. For example, the RSPO for Cecil Lindsey said that the subcontractor was unfamiliar with sample quality assurance and chain of custody procedures. As a result, the RSPO indicated that subcontractor workers (1) packed water samples incorrectly, with the result that one broke in transit; (2) sent several samples in insufficient volumes for the requested analyses; (3) failed to log correctly the precise location and identification of samples taken; and (4) completed sample traffic reports incorrectly, causing a delay of approximately one month until the contractor relocated the "lost" samples. Regional and state staff cited subcontractor drillers specifically as a common source of on-site delay. The contractor for Nyanza required almost five months to subcontract drilling tasks. This delay was due, according to the RSPO, to inaccurate drilling specifications provided by the contractor and failure to locate a qualified subcontractor. Slow drilling progress, due primarily to the subcontractor's inexperience with hazardous wastes, caused a ten-week delay at Swope. The RSPO at Cecil Lindsey said that the drilling team did not know how to operate a hollow-stem auger for core samples, but had experience primarily with solid augers used for drilling wells. Site work was delayed several days until the contractor was able to provide a more appropriately experienced team. At Pepper's Steel, the RSPO said that the contractor had considerable difficulty in locating a competent driller. The original low-bid drilling firm was not certified at the time the contractor hired it, and subsequently failed to receive certification. A second driller quit after one month, and a third driller was finally hired after a delay of over three months. At some of these sites, problems may have been due in part to zone contractor efforts at hiring small and disadvantaged subcontractors. Contractor errors and inexperience occasionally have delayed site work at state-lead sites. According to the EPA RSPO, the first batch of samples taken at Burnt Fly Bog were contaminated with methylene chloride, a cleaning solution -- a problem that might have occurred because the contractor or the laboratory did not follow all quality control procedures precisely. At McKin, the EPA site officer observed instances in which the contractor mobilized inappropriate equipment or provided insufficient personnel, delaying site work by almost two weeks. The EPA site officer for Spence Farm said that the subcontractor neglected to bring needed protective equipment to the site and made other errors that caused two to three weeks of slippage in the RI. The RSPO at Rose Township said that the drilling subcontractor used a drilling rig that was too small for the depth and hardness of soil and rock at the site. The driller experienced repeated equipment problems, including a broken compressor followed by malfunctioning of the back-up compressor. Finally, although not a technical error, the contractor at Cross Brothers virtually ------- - 9 - stopped work for three months during corporate reorganization following its acquisition by another firm. 2.1.5 Communication Among the Key RI/FS Participants Several sites experienced delays because the contractor did not follow guidance on objectives and constraints from EPA and the state, either because the contractor did not receive clear guidance or because the contractor did not adequately understand such guidance. Improved communication would enable contractors to make more efficient use of RI/FS time. An extreme example of insufficient communication occurred at GEMS where, according to regional staff, the contractor began preparing the FS report without notifying the region that the sampling data had been found to be inconclusive. As a result, two months were lost during which a supplementary RI could have been planned and approved. At Del Norte, headquarters and regional coordination and communication resulted in confusion and delays. The RSPO noted that the region communicated primarily with the subcontractor, at the same time that headquarters gave conflicting advice and instructions to the contractor. Where good communication is maintained between the region and contractor, it is more likely that problems that constitute potential sources of delay can be detected early, and measures can be taken to minimize the impacts. At Distler Brickyard, the RSPO spoke by telephone to the contractor's project manager at least once a week during the on-site phase of the RI, enabling him to remain apprised of contractor activities during this important phase. At times, communications have been hindered because the region's suggestions were conveyed first to the contractor and then to the subcontractor. For example, the RSPO for Cleve Reber commented that because the subcontractor did not receive review comments on the draft work plans directly from the region, work plan development was delayed over six weeks while the work plan went through many drafts. Inadequate communication has been a common problem at state-lead sites as well. In two instances, the state did not express its preference for a low-cost remedial alternative until after reviewing the contractor's draft FS report. At Spence Farm and Highlands Acid, the respective RSPOs said that the state told the contractors to restructure the FS and to look at a range of lower-cost alternatives. The National Contingency Plan requires that a wide range of remedial alternatives be considered in choosing the cost-effective remedy. In both cases, however, had the state communicated its concerns from the start without limiting the range of alternatives considered, over a month's time might have been saved. An example of a site where effective communication minimized time delays is Old Inger. The EPA site officer said that the contractor provided the state and region with monthly technical status and individual task completion reports throughout the RI/FS work phase, and the state and region provided frequent comments, mostly by telephone, on various aspects of site progress. As a result, the state and region approved the RI and FS reports after one round of revisions on each. The turnaround time between the draft and final RI and FS reports was a relatively short two and four weeks, respectively. ------- - 10 - Increased communication in the early planning phase at two state-lead sites could potentially have improved the thoroughness of project planning, resulting in fewer unexpected events and delays in the RI/FS process overall, because contingencies could be anticipated and planned for. Examples of sites where RSPOs indicated that better planning and technical judgment could have made resampling unnecessary are Whitehouse, where two additional sampling rounds led to a total RI/FS delay of seven months, and Burnt Fly Bog, where initial sampling was too shallow to determine the full extent of contamination. At Burnt Fly Bog, the RSPO said that the remedial design contractor essentially had to repeat the RI because sampling did not detect fully the contaminants at the site. Regional staff also voiced the opinion that fast-tracking may lead to delays at some sites because it increases the likelihood of technical errors and increased RI/FS costs. Fast-track schedules allow little or no time for contingencies such as unexpected findings, equipment malfunction, or any of the other sources of delay that often occur during RI data collection and field work, and may put pressure on RSPOs to be less thorough in order to save time. At GEMS, a fast-tracked site, the RSPO said that EPA and the contractor agreed to reduce the number of sampling wells from the number that had originally been planned in the final work plan. The RSPO said that the abridged drilling program failed to penetrate the ground water plume of contamination; the need to resample eventually led to a ten-month delay in RI completion. It appears that advance planning of" all RI/FS documents can help to expedite the process. The RSPO directed the contractor for Cleve Reber to prepare sections of a Record of Decision (ROD) document simultaneously with the RI report. When the RI is completed, the RSPO has expressed his intent to brief headquarters and receive guidance comments before preparing the FS. This method increases the likelihood that the alternatives in the FS report will be acceptable, and that headquarters will approve the ROD without many revisions or lengthy delays. 2.1.6 State Review of Reports and Work Plans Several RSPOs pointed out that lengthy state reviews often have delayed the RI/FS. Although regions generally assign personnel and specify deadlines for review, states -- particularly in the case of sites undertaken early in the program -- often have not had a formal review procedure or schedule. For instance, the RSPO for Cecil Lindsey said that the state of Arkansas took one month to provide comments to the contractor on the first draft work plan for the site. The RSPO for Swope Oil recalled that New Jersey took two months to review the draft work plan; eight offices in the New Jersey Department of Environmental Protection reviewed work plans and no one person had the authority to set deadlines. In response to this problem, Region II now allows thirty days for state review, after which EPA proceeds, with or without state comments. ------- - 11 - 2.1.7 Late Subcontracting Decision At a few of the sites, the contractor did not arrange to subcontract work until three or more weeks after work assignment, delaying the RI/FS by at least as long as the time needed to complete subcontract negotiations. For example, according to the RSPO at Hocomonco, the contractor decided to subcontract work five weeks after work authorization, necessitating an additional draft work plan and delaying RI progress until contracts were finalized. At the Tysons Dump site, the zone contractor prepared a draft work plan before deciding to subcontract the RI/FS; the subcontractor did not complete the final work plan until four months after the draft work plan had been submitted. Similarly, at Cecil Lindsey, the RSPO said that late subcontracting delayed preparation of the draft work plan by approximately one month. 2.1.8 Contracting Procedures at State-Lead Sites Most of the problems in this category that caused significant delay were due to unique situations that will not necessarily arise in the future. Because contracting did account for a portion of the RI/FS delay at several state-lead sites, however, it is included here as a minor common source of delay. At early state-lead sites, the lack of defined procedures and staff uncertainty caused substantial delay. At Spence Farm, the EPA RSPO indicated that the state did not award contracts until fourteen months after signing the cooperative agreement. The EPA site officer for Old Inger said that the state took six months to complete the contract awards process. Since then, states have become more skilled at preparing requests for proposals, reviewing and evaluating proposals, and selecting contractors. Even with improved skills and techniques such as pre-qualified lists of contractors (see Section 3.2.7), however, existing state-lead contracting procedures require more time for selection of a contractor team and starting fieldwork than use of zone contractors. The RSPO for Whitehouse indicated that states (in this case, the state of Florida) often have difficulty complying with EPA contracting requirements and, at the same time, adhering to their own internal contracting procedures. The RSPO stated that Florida required approximately four months to complete negotiations with contractors for the RI at Whitehouse, primarily because of conflicts with and confusion about EPA contracting requirements. The Purity Oil RI/FS also experienced delays because of complications arising from the state's need to conform with EPA procedures. Site work was held up six weeks while the contractor prepared necessary additional documents. Attention to contract coordination with states would help alleviate such delays in the future. 2.1.9 Headquarters Policy and Guidance Some regions noted that, because revised record of decision (ROD) criteria now require that Superfund remedies comply with RCRA, headquarters must be prepared for some amount of unavoidable RI/FS slippage. For example, the RSPO for Nyanza, a site which completed its planning phase prior to the policy ------- - 12 - change, anticipates that the feasibility study may be delayed by several more months in order to complete additional work to comply with RCRA. At Cleve Reber, mentioned above, the regional staff are uncertain as to how headquarters will respond to their potential request for a waiver to exempt the site from RCRA compliance. To avoid preparing an unacceptable ROD, the regional staff plan to request detailed guidance from headquarters before beginning the FS. At a few sites, RSPOs have avoided delays thji". could have been caused by revised headquarters policies. By dividing response work into phases, the aspects of a RI/FS that are not affected by policy changes can be responded to first while considering the effects policy decisions might have on other parts of the site. For example, response work at Tysons Dump will be phased. The first phase will deal with on site contamination, but response to a contaminated floodplain off site will be addressed later in order to consider carefully EPA's policies affecting floodplains and wetlands. At the Drake Chemical site, sediments will be removed from a leachate bed and temporarily deposited in a single-lined lagoon on site; this action is temporary because it may not meet RCRA requirements. A second phase of remedial action will ensure that a permanent remedy is taken that will be consistent with RCRA. Headquarters efforts to expedite the RI/FS can have a significant and positive impact. One example of headquarters' success in this respect is Del Norte, one of the first sites to use the recently revised work assignment procedures, described in detail in the next section. Headquarters' new streamlined work assignment procedures enable the region to allow the contractor to begin field work shortly after the work assignment is given. The RSPO at Del Norte indicated that the required procedures could shorten RI/FS for all regions, once regional and contractor staff become experienced in following the procedures. On occasion, legitimate policy concerns arise, and the RI/FS cannot proceed until headquarters resolves the issue. A situation of this type occurred at Mountain View Mobile Home, which was initially classified as an enforcement-lead RI/FS. Some legal uncertainty existed as to whether the former asbestos mine could qualify for cost-recovery under CERCLA, and the site's RI/FS was delayed for four months while headquarters determined appropriate enforcement policy. Once these policy issues were resolved, however, the RI/FS was considered as a "fast-track" response, and was completed in only one month. Finally, in addition to site-specific delays, several RSPOs commented that RI/FS do not proceed as quickly as they might because the nature of the RI/FS process presents disincentives for the RSPO to accelerate completion of RI/FS tasks. The present Superfund procedures encourage the RSPO to be extremely thorough in order to ensure that, at any time, activities at the site can withstand the scrutiny of cost-recovery proceedings, congressional or public inquiries, and periodic headquarters review. At the same time, there are extremely few positive incentives (e.g., SCAP activity targets) to encourage the region to minimize RI/FS slippages. ------- - 13 - 2.1.10 Headquarters Processing of Initial and Final Authorization Documents At Distler Brickyard and Swope Oil, regional staff indicated that headquarters required over one month to process the final authorization, whereas at Hocomonco the contractor received final authorization nine days after submitting the final work plan. Similarly, at Fields Brook, the RSPO said that headquarters took nearly four months to review a request to begin Phase II sampling while awaiting Phase I data review. Given that EPA headquarters need only provide approvals and complete contract administration tasks at the time a satisfactory work plan has been completed, this period could be reduced. 2.2 OTHER SOURCES OF DELAY Some delays are unique or unavoidable. The site reviews indicated, however, that such delays are the exception rather than the rule, and are not responsible for most RI/FS slippages. Two such causes of delay, weather and unexpected findings, are discussed below. 2.2.1 Weather Extreme weather conditions caused RI/FS slippage at several sites. The Iron Mountain Mine RSPO explained that a drought in California prevented necessary wet weather sampling at the site for an entire year. This site was the only one reviewed in which uncontrollable causes, and not planning and management, were responsible for a significant RI/FS delay. Annual spring flooding of the Mississippi River led to RI and FS slippages of eight and five weeks, respectively, at Old Inger, The RSPO said that the Old Inger RI/FS was delayed an additional three weeks when the laboratory contracted to analyze samples was severely damaged in .a lightning storm. Summer heat and humidity wer£ recurrent problems in some regions, lowering worker productivity and limiting site activities to a few early morning hours. At Cleve Reber, the RSPO said that one driller was hospitalized for heat exhaustion and other drillers walked off the site, causing a delay of several days. 2.2.2 Unexpected Findings Unanticipated results have the potential to delay any RI/FS. At Cleve Reber, the RSPO said that the site team found that the subsurface clay layer was 250 feet deep rather than 150, as had been expected -- and an additional two days of drilling were required to penetrate the clay. At Rose Township, unexpected discovery of high concentrations of vinyl chloride and metals caused an increased scope of work and more stringent health and safety procedures. At Olean, sampling was delayed one month when the contractor discovered that the aquifer was 100 feet deep rather than 75 feet. In the case of GEMS Landfill, regional staff indicated that neither EPA nor the contractor realized that the placement of site wells did not adequately trace the ground water plume. Because GEMS was one of the sites selected for fast-tracking, in the opinion of regional staff, the region and contractor were unable to respond flexibly to new insights into the nature of the site. ------- - 14 - At Heleva, an on-site quarry filled with water several months after ground water samples were taken. An additional, unanticipated, round of sampling was needed to determine how the filled quarry might have affected ground water quality. ------- 3. WHAT CAN BE DONE: IMPLICATIONS FOR MANAGEMENT The site reviews identified several major and minor causes of delay for Superfund RI/FS. This section discusses steps that will be taken in an effort to reduce delays. The key finding is that no single action will reduce RI/FS delays significantly. Rather, the site reviews indicate that EPA needs to devote more attention to several areas of concern and to all participants -- EPA headquarters and regions, states, and contractors -- in the RI/FS process. The section begins with a discussion of improved procedures that HSCD has already instituted, followed by implications of the site reviews for further improvements in the RI/FS process. 3.1 IMPROVED PROCEDURES ALREADY INSTITUTED BY HSCD To date, HSCD has instituted several changes in its policies and requirements based, in large part, on experience gained during the first four years of the program. This section describes the major changes that have been made to date for both federal- and state-lead sites. 3.1.1 Improved HSCD Procedures to Date for Federal-Lead Sites In an effort to reduce the amount of delay and improve the overall efficiency at federal-lead sites, HSCD has adopted several new policies and procedures, the most significant of which are described below. Revised work assignment procedures. In the past, EPA could only authorize initial activities after a draft work plan had been submitted, a process that can take several weeks. Program experience indicated, however, that certain RI/FS tasks could be undertaken during the work plan phase of a project. Under the REM II contract, the RSPQ signals the contractor to begin interim tasks after the headquarters contract officer initiates the work assignment. The contractor acknowledges these tasks in a Work Plan Memorandum -- an abbreviated version of a work plan that establishes budgets and schedules to track progress of interim tasks. EPA acknowledges and accepts the budgets and schedules, allowing the contractor to begin a wide range of interim activities -- including developing QA/QC and health and safety plans, conducting site reconnaissance activities, and even initial sampling -- before submitting a complete draft work plan. Award Fee Policy. In the past, for each project conducted under the REM/FIT contracts, EPA paid a quarterly award fee, depending on the quality of the work performed at the site during that quarter. In June 1984, EPA revised the award fee procedures to provide a greater incentive for high quality performance and timeliness. Under the new procedures, the contractor receives only part of the award fee three times a year; the remaining award fee is held in reserve in an award fee pool until the project has been completed. At that time, EPA evaluates the project and awards some or all of the award fee pool to the contractor. The initial fee awarded to the contractor every four months is called Phase I; the fee held in reserve and awarded at the project's completion is called Phase II. ------- - 16 - CLP Task Force. In August 1984, the Director of the Office of Emergency and Remedial Response (OERR) requested that a task force be convened to study several contract laboratory program (CLP) issues, including delays and quality control problems related to the CLP. The task force's findings were released on October 30, 1984. One of its recommendations was that all data packages be reviewed by the Sample Management Office (SMO) after analysis by the CLP but before the packages are sent to the regional ESD. Data packages of insufficient quality will be returned to the CLP laboratory that performed the analysis, which will have to supplement the data. The CLP would not receive payment for the analysis until the SMO had approved the data packages and forwarded them to the regional ESD. Delegation to Regional Offices. HSCD has delegated much of the contract-related paperwork, which had previously been completed by headquarters staff, to the regional offices. The goal was to cut one to two weeks from the project schedule by removing headquarters staff from the process entirely, except for giving the contract officer final approval on each work assignment. Contractor Responsibilities. Both the REM/FIT zones 1 and 2 and REM II contractors have designated a full-time regional manager for each region under their jurisdiction. The regio'nal manager oversees all of the sites in the region, coordinates project schedules, and works closely with regional staff to ensure that activities at various sites are conducted efficiently. This function has proven extremely valuable and HSCD expects to require contractors on all future REM projects to provide regional managers with expanded responsibilities and support staff as needed. Subcontractor Procurement. For the REM II and future contracts, headquarters contracts staff will require that all technical work performed by prime or subcontractors be paid from level-of-effort funds rather than the subcontractor fund pool. The effect of this change is to reduce the number of instances in which subcontracting occurs compared to the existing REM/FIT contracts. 3-1.2 Improved HSCD Procedures at State-Lead Sites HSCD has also implemented several changes in the policies and procedures that apply to state-lead sites in order to simplify the RI/FS process and reduce the amount of delays that occur at such sites. Training Workshops. In the fall of 1983, EPA and the Association of State and Territorial Solid Waste Management Officials (ASTSWMO) sponsored training sessions for state personnel which covered a range of topics related to CERCLA, including procurement regulations and other program requirements. The objective of these sessions was to familiarize state staff with the program so that they would be able to fulfill program requirements more quickly and easily. ------- - 17 - Class Deviations from Procurement Regulations. The original procurement regulations governing CERCLA (codified in 40 CFR Parts 30 and 33) required states to issue a request for proposals (RFP) to conduct the RI/FS and a second RFP to conduct the remedial design. This requirement resulted in significant delay, because state staff had to go through the entire contracting process twice. In November 1983, the requirement was changed, and states may now select a single contractor to conduct both the RI/FS and the remedial design for a site. Multi-site Cooperative Agreements. In February 1984, EPA approved a multi-site cooperative agreement (MSCA) with the state of Pennsylvania. The MSCA authorized RI/FS activities at twelve sites in the state, and saved considerable time for both the state and EPA compared to the amount of time that would have been spent negotiating twelve individual cooperative agreements. Several other states, including Illinois, Minnesota, Utah, Washington, and Texas, have entered into MSCAs and/or are now considering using MSCAs for multiple sites in their jurisdictions. State "Zone" Contract. The state of Michigan recently procured a contractor that will serve, at the state level, much the same function as the REM/FIT contractors do at the regional level. The state will write a separate work assignment for each site project, and the contractor will handle all remedial projects in the state. HSCD is encouraging other states to consider similar contracting arrangements, in the belief that they will simplify and accelerate state-lead projects. Pre-Authorization Procurement. EPA has recently encouraged states to begin the process of procuring a contractor'before the cooperative agreement has been finalized. This procedure was used at the Purity Oil site in Fresno, California and reduced the lag time that usually occurs while the state awaits final EPA approval of the cooperative agreement. Prequalified List of Contractors. The states of New Jersey and New York, among others, established a list of contractors qualified to conduct RI/FS activities. For each new project, the state selects a few contractors and asks them to submit a proposal for conducting the needed work. The state then reviews the proposals and selects one contractor to perform the RI/FS. This process has significantly reduced the time and resources the state devotes to contractor procurement. Site Project Officer Manual. EPA is currently developing a manual that defines and describes the responsibilities of regional and state site project officers and the role of the RSPO in state-lead sites. EPA plans to make the manual available to all regional and state staff involved in CERCLA projects. 3.2 FURTHER IMPROVEMENTS TO BE MADE 3.2.1 Shorten CLP Turnaround Time Longer-than-expected CLP turnaround time was the most frequently mentioned source of federal-lead RI/FS delays. Although CLP delay is the largest single ------- - 18 - source of RI/FS delays, it is not responsible for the majority of total RI/FS delay. The long CLP turnaround time was expressed in the site reviews as a result of three problems. These are: Allocation delay; Turnaround delay; and Incomplete sampling result delay. OERR reco'uiizes these three major CLP problems and steps are being taken to increase the number and quality of CLP laboratories. The regions did not offer specific suggestions for resolving the CLP capacity problems that cause much of the turnaround delay, although regional staff did express frustration over the lack of solutions to date. For example, lack of CLP capacity and slow turnaround time prevent RSPOs from being able to sample, evaluate and resample in order to adopt a phased RI approach that promotes an accurate RI report. There are several ways to shorten CLP turnaround time. OERR should continue its work to study and implement these remedies. They are: Increase laboratory capacity; Increase staff to assist CLP laboratories in increasing speed and quality of analysis; Upgrade sample flow management to decrease laboratory demand; and Define data quality objectives. Increase Laboratory Capacity. The allocation and turnaround problems causing CLP delays are due primarily to insufficient laboratory capacity. Demand for laboratory services is growing faster than-capacity, resulting in backlogs and unavailability of laboratory analysis when needed. Additional CLP capacity may be developed from extramural sources or from expanding the capacity of ESD laboratories and currently participating CLP firms. If additional laboratories are to be attracted or existing CLP participants are to expand their capacity, however, EPA may need to consider several actions, including: Increase CLP prices by screening out low bidders who, although qualified, may not deliver contracted capacity in the future; and Retain additional analytical laboratories (including REM contractor laboratories) to supply added capacity during periods of peak demand. Increase Staff to Assist CLP Laboratories. A second method to shorten CLP turnaround time is to provide experienced EPA staff to help new laboratories learn the required procedures and help existing laboratories ------- - 19 - maintain the required quality standards. Continual oversight and visits by EPA personnel may help CLP laboratories avoid unnecessary delays and reduce the number of incomplete data packages. The proposed addition of technical contractors at the SMO will provide some of this assistance, but more is needed. The increased SMO role is discussed in Section 3.2.2. Upgrade Sample Flow Management. In addition to increasing the supply of laboratory capacity and staff, improving the management of demand for laboratory services may also reduce CLP turnaround time. Stronger management of demand includes: Rigorous control and planning of sampling needs; Limiting sample analyses to pertinent fractions; and Increasing use of on-site remedial contractor laboratories for preliminary screening. These sample demand management steps are now being taken to a limited extent. The award fee process encourages careful planning of sampling needs. HSCD has directed REM contractors to limit sample analyses to pertinent fractions. Regions have begun encouraging contractors to use mobile screening techniques. Even with these steps, stronger management of demand may still be required. Laboratory capacity remains relatively constant throughout the year but demand for laboratory services fluctuates between heavy demand in the summer and lighter demand in the winter. The allocation system has addressed this fluctuation in demand by increasing allocations of CLP space to southern regions (IV, VI, IX) during the winter. The site reviews revealed, however, that laboratory bottlenecks still occur primarily in the summer. More rigorous management of sample flows by program managers and the regional sample control center (RSCC) may help alleviate summer bottlenecks by allowing CLP laboratories to utilize their capacity fully year round. For example, regional program managers should ensure that RSPOs begin sampling in the winter. In addition, regions and contractors should be encouraged to plan and schedule all their sampling needs to make the most effective use of monthly allocations. For example, if site access is denied at one site, sampling activities should begin at another site so as to utilize all of the regional allocation. Such coordination is possible only if managers ensure that contingency plans are prepared monthly. Demand for laboratory services may also be reduced if sample analyses are limited to relevant fractions only, particularly for organics where capacity shortages are most severe. Sampling plans could be reviewed carefully to determine the pertinent fractions, thus saving time and resources that could be applied to other analyses. Remedial contractor laboratories have been used at some sites for "first round" sampling to help alleviate CLP capacity problems. The Del Norte site contractor, for example, has responded to CLP capacity problems by setting up ------- - 20 - an on-site laboratory and using a portable gas chromatograph to conduct preliminary sample analyses. The CLP laboratory was used only to confirm the preliminary results. The Iron Mountain Mine contractor also has the capacity to conduct on-site sample analyses. The contractor claims that if his mobile laboratory is used, four to eight weeks of CLP delay can be avoided. Increased use of on-site laboratories such as these may help eliminate inadequate CLP capacity. Several potential risks, such as lower quality data and lack of standardized results, must be considered, however, in deciding whether or not to use on-site laboratory '.s for preliminary screening. Use of these laboratories should therefore be managed carefully and determined on a case-by-case basis. Define Data Quality Objectives. Headquarters policy specifies that CLP laboratories be capable of providing analytic support for cost recovery and enforcement activities under CERCLA -- activities that may involve case development against responsible parties. Because laboratory samples may be used as evidence in litigation, laboratories are required to follow strict procedures for chain of custody and document control. In addition, laboratories must follow standard analytical protocol to ensure the validity of the sampling results. These quality control procedures for samples and analyses that might be used as evidence consume significant laboratory time and resources. Some of the required procedures are necessary to ensure fundamentally consistent results of a known quality. However, an analysis of the objectives of the data quality procedures may reveal revisions or simplifications to the procedures that can be made without compromising overall data quality. A first step in this regard is to define clearly the quality requirements for data used for legal evidence and engineering decision-making, and to identify possible revisions to these requirements. To the extent that the procedures can be simplified, laboratory time per sample would be reduced, which would free capacity to analyze more samples reducing turnaround time. 3.2.2 Shorten Data Review Time Another major cause of RI/FS delay brought up in the site reviews is longer-than-expected data review time. At the Cleve Reber site, for example, the RSPO said that the ESD laboratory had a three month backlog of data packages to be reviewed. At the Hocomonco site, the RSPO indicated that data review took five weeks longer than scheduled. Three major causes of data review delay are: Insufficient ESD data review capacity; Incomplete CLP data packages; and Current data quality objectives make data review a labor intensive, complex process. There are several measures that can be taken to reduce data review delays and relieve time pressures on RI/FS completions. Some of these measures are: Increase data review staff; Increase reliance on CLP expertise; ------- - 21 - Increase SMO role in data review; Change criteria for date of delivery under CLP contracts; Increase automation of data review; and Provide contractor with access to raw data. Increase Data Review Staff. One way to reduce data review time is to add EPA staff to the ESDs, or to provide contractor support to ESDs to assist in reviewing data. When the Cleve Reber site faced a three month backlog in Region VI's Houston ESD laboratory, for example, contract workers on the region's FIT team were called in to assist with data review. Nearly every region uses contractors to review data and provide audit reports to ESD data review staff, but the site reviews indicated that existing resources may be insuf f icient. Increase Reliance on CLP Expertise. CLP laboratories currently provide ESDs with unreviewed data packages for review and interpretation. ESD staff must analyze the data results without any technical assistance or appropriate cautionary information from the CLP laboratory staff on the reliability of the results. This process is both time consuming and labor-intensive. Greater reliance on CLP professional judgment to provide preliminary interpretation or highlight potential problems and discrepancies may reduce data review time. CLP laboratories provide information on quality concerns and reliability of results currently for private clients. Extension of the activity for ESDs could require significant training and administrative changes under current data quality objectives. If pursued in conjunction with respecification of such objectives, however, increased reliance on CLP expertise could reduce the time needed to interpret the results and allow ESDs to concentrate on their role as reviewers. Several potential drawbacks to using CLPs to conduct more sophisticated interpretation should be considered, however, including: Potentially higher prices for CLP analysis; Reduced standardization of analytical protocol; and Potentially reduced validity of CLP results as legal evidence. Increase SMO Role in Data Review. Another method to reduce pressure on ESDs is to use the SMO to conduct an initial screening of CLP sample results before they are sent to the ESDs -- one of the recommendations made by the CLP task force in its report of October 30, 1984. Using the SMO to check data packages for completeness and quality would conserve regional resources and time by catching potential problems before the data package reaches the ESD. Change Criteria for Date of Delivery. Currently, CLP laboratory contract conditions are considered fulfilled if the laboratory delivers a data package to the region by the agreed-upon delivery date, usually 30 days after samples have been sent to CLP laboratories. If the delivery date is met, according to the task force, CLP laboratories receive a partial payment whether or not the data packages are complete. In an attempt to avoid the lengthy data review time that results when incomplete data packages are submitted, the CLP task force has recommended that the definition of a satisfactory data package be ------- - 22 - changed. Under the new definition, the contract terms would be met only upon ESD receipt of a complete data package, as determined by the SMO. This procedural change would increase the CLP firms' incentives to improve the quality and ensure completeness of its documentation the first time through, thus reducing ESD review time. Increase Automation of Data Review. Increased automation of the data review process could also reduce ESD turnaround time. Automation of this process would involve electronic transfer of the data packages tc .the ESDs where objective, unambiguous requirements could be verified using automated equipment. For example, tasks that could be automated are: Verification that sample holding times do not exceed the appropriate limits; " Verification that a GC/MS Tuning and Mass Calibration Form V has been submitted at the required frequency and properly completed; Verification that a Pesticides Evaluation Standards Summary Form has been submitted at the required frequency and properly completed; and Verification of completeness of the Reagent Blank Summary and that results have been rep.orted at the required frequency. Automation of the data verification and evaluation procedures has advantages and disadvantages. Possible advantages include: Reduction of Q/A review turnaround time; Allowance of higher productivity of -data package review; and Reduction of costs in the long run. Conversely, possible disadvantages that could result from automation are: Potentially decrease quality of review (e.g., some errors that require human judgment may be missed); Increase costs in the short run while purchasing automation equipment; and Increase delays and staff dislocation while automation is implemented. Contractor Access to Raw Data. Site reviews revealed several cases where the RSPO provided copies of unreviewed CLP laboratory data to the contractor as soon as they were available. This approach not only can reduce delay but also may improve the quality of RI results by allowing phasing of the RI. At Iron Mountain Mine, for example, the RSPO said that access to unvalidated data allowed early recognition of a potential problem area and a change in the sampling program (with no increase in cost) to achieve more useful results. ------- - 23 - Similarly, at the Nyanza site, the RSPO provided the contractor with preliminary laboratory data to enable portions of the RI report to be written while validation procedures were underway. Release of data in advance of data review results should not be recommended routinely because decisions based on data that are eventually found to be faulty would have to be reversed. Rather, release of unreviewed data should be considered on a case-by-case basis but encouraged, as determined by the RSPO in consultation with data review staff. Such an approach may save time and improve final RI results, but possible risks must also be considered carefully. For example, resources and time spent on site work based on unvalidated CLP laboratory results may be lost if data review reveals that the results are faulty. It is essential, therefore, that RSPOs work closely with ESDs when requesting site work to proceed with preliminary laboratory results. 3.2.3 EPA Regional Management Procedures In discussions held with RSPOs during the site reviews, several management issues arose that are not limited to any one site but are found across all sites in all regions. These issues are fundamental to the timely completion of any RI/FS, including: Identification of critical path tasks; Frequent communication with contractor; Expansion of RSPO training; and Creation of regional RI/FS management tracking systems. Identification of Critical Path Tasks. RI/FS tasks that make up the critical path are those whose length determine the total duration of the RI/FS. Delays in these tasks, therefore, are most damaging in terms of delaying the RI/FS completion. The following tasks comprise some-of the critical path tasks in a "typical" RI/FS: Drilling program; Sampling; and Data analysis. If delays occur beyond the scheduled time, the RSPO and REM/FIT project officer should identify other non-critical path tasks that can be accelerated to minimize the effect on the RI/FS completion. Steps of this type were taken at Cleve Reber in Region VI, where the RSPO is preparing the ROD concurrently with the RI report; and at Nyanza in Region I, where the RSPO said that he authorized the preparation of the RI report to proceed with unvalidated data. By understanding and analyzing the scheduling of critical path tasks, the region and the REM/FIT contractor can modify the RI/FS schedule to minimize the impact of delays in these tasks. Frequent Communication with Contractors. Almost without exception, frequent communication was the norm at those RI/FS with only minimal delays, ------- - 24 - such as McKin in Region I, Swope Oil in Region II, and Old Inger in Region VI. Frequent communication was noted as the key factor in the success of the two state-lead sites, McKin and Old Inger. Lack of communication, however, caused major delays at federal-lead sites including GEMS Landfill in Region II and Cleve Reber in Region VI. Similarly, at state-lead sites including Burnt Fly Bog in Region II, Whitehouse in Region IV, and Highlands Acid Pit in Region VI, poor communication necessitated multiple drafts of both planning documents and RI and FS reports. By better explaining their objectives in each RI/FS and by following closely the daily implementation of the objectives, the state site project officers may be able to eliminate delays caused by misunderstandings. At McKin in Region I, frequent communications between the state, regional staff and the contractor contributed to the less than average slippage in the RI/FS. In part because the contractor's offices were near the Region I office, the EPA site officer was included in many daily exchanges of information between the state and the contractor. This involvement clarified any questions that arose during the RI/FS and also provided a sense of management continuity to the project. Several methods of communication with contractors may be used to avoid misunderstandings which cause schedule and budget over-runs, including: 1. Holding a kick-off meeting at the time of work authorization to discuss: Information on the site such as the RAMP or results of state- or owner-sponsored investigations; Goals of the RI/FS; Foreseeable problems such as site access or enforcement delays; and Schedule for reporting progress during the RI/FS. 2. Obtaining weekly progress reports during the RI/FS planning phase. These could be phone calls, memos or meetings, depending on the RSPO's preference. 3. Holding a final authorization meeting to discuss the RI schedule, subcontractor procurement and any progress which has been made on interim tasks. 4. Maintaining daily contact during RI field work and visit the site during major pperations. Additional meetings should be held whenever significant problems or delays are encountered. ------- - 25 - 5. Holding a FS scoping meeting toward the conclusion of the RI to discuss data results, remedial action alternatives, and the appropriate direction for the FS report. Although frequent communication will not eliminate all delays, it can help RI/FS participants to anticipate problems and resolve them before delays occur. Expansion of RSPO Training and Headquarters Guidance. Several EPA regional staff and contractor personnel indicated a belief that many RI/FS are "overdesigned" in terms of both number of samples taken and types of data analysis (e.g., full priority pollutant scans) performed at the laboratory. The knowledge tht some RI/FS in the past lacked sufficient data to document a ROD provides an incentive for some RSPOs to overdesign. Overdesigning an RI/FS not only adds time to the schedule directly, but also leaves the entire process more vulnerable to delays. There are several reasons why sampling and analysis redundancies are being designed into RI/FS. Headquarters policy has placed a heavy emphasis on quality and thoroughness, particularly in requiring that results be legally defensible for cost recovery purposes and ensuring compliance with RCRA standards. In addition, there is some degree of uncertainty in the regions as to what headquarters expects to be included in an RI/FS and what it requires in order to approve a ROD. To deal with this uncertainty, several RSPOs indicated a tendency to design RI/FS to cover all possible headquarters requirements. Finally, many RSPOs expressed a desire for additional training on the supervision and design of RI/FS. Several RSPOs noted that a lack of training adds to the uncertainty over what is required in an RI/FS and the subsequent ROD. Approaches to alleviate the uncertainty at the regional level include expansion or acceleration of current headquarters efforts both to expand RSPO training and to publish guidance on RI/FS. In order to capitalize on the experience of the RSPOs who have been involved in the Superfund program over the past two years, HSCD should compile a short manual or guidance document for new RSPOs. The guidance should cover problems that have been encountered at sites and the most effective responses to each of these problems. HSCD is currently developing a seminar to aid state and federal site project officers in implementing headquarters guidance on risk assessment and ROD document preparation. The objective of the guidance suggested from the site reviews should be to create a "working" reference document to which RSPOs can refer when problems occur during an RI/FS. Development and dissemination of such practical guidance enables experience to be shared across regions so that one region will not be inventing a new response to a problem that another region has already addressed. This short document could be used most effectively as part of training courses for RSPOs. In addition, clarification of policy is needed in the areas of RCRA compliance and the ROD approval process. By issuing policy directives and increasing the frequency of contact with regions, headquarters can inform RSPOs as to what is needed to obtain approval of a ROD. ------- - 26 - Regional RI/FS Management Tracking System. Some of the regions visited did not use a standard, comprehensive procedure to update program managers on progress of RI/FS. In some regions, RSPOs followed sites individually and reported progress as needed. Other regions used regular weekly or monthly reports to managers that required the RSPO to learn and report progress at specified intervals. Use of regularly scheduled reports could enhance management of RI/FS by encouraging RSPOs to communicate with states and contractors, and by providing regional managers with the information necessary for de". ding when, whether, and how to improve or expedite RI/FS progress. 3.2.4 Effective Management of Subcontractors To minimize delays in the RI/FS due to subcontractor problems, both EPA and the REM/FIT contractors must place additional emphasis on managing the entire subcontracting process. Specifically, additional policies or procedures are needed in the following areas: Procurement; Subcontractor charges; and Daily management. Subcontractor Charges. The site reviews identified several instances in which subcontractor.technical performance could have been improved substantially. In addition, nearly every site using a drilling subcontractor experienced delays and, in the.opinion of the respective RSPOs, inadequate technical performance: EPA should encourage the REM/FIT contractors to use an award fee mechanism for its engineering services subcontractors similar to that used by EPA for its contractors. To the extent that the award fee serves as an incentive for better performance, it should be used to spur better performance by subcontractors. In addition, drillers and other technical support subcontractors are now being procured under fixed-unit-cost contracts, such as cost per foot drilled. Although engineering services are often procured through level of effort contracts, technical services are generally procured under fixed fee contracts. Fixed price contracts convey the incentives for timely completion of drilling and other technical services on time and within budget. Daily Management. At the McKin site in Region I and Iron Mountain Mine in Region IX, frequent communication with the subcontractor effectively mitigated many problems and, therefore, minimized delays. Frequent communication among the subcontractor, the REM/FIT contractor, and EPA is essential to the timely completion of an RI/FS. Subcontractor management should therefore be emphasized in the REM/FIT review and award fee process. 3.2.5 EPA Headquarters Management Procedures HSCD has indicated a desire to expedite the RI/FS process whenever "bottlenecks" occur. Specifically, headquarters should intervene, when requested by the regions, to resolve disputes with the REM/FIT contractor, other headquarters offices, and the CLP. ------- - 27 - In order to expedite projects, headquarters is developing a more accurate RI/FS management tracking system to stay apprised of the progress at active RI/FS. The site reviews revealed numerous instances of uncertainty over the accuracy of RI/FS progress data. The study team noted discrepancies and gaps in regional site files, differences in dates recorded by regional, contractor, and headquarters staff, and significant differences between existing headquarters data collected from regions and the data collected by the study team from the same regions. The FY85 Headquarters Remedial Management Plan includes procedures for regional managers at headquarters to track RI/FS progress and update schedules monthly. This tracking and updating is accomplished primarily by telephone. HSCD may wish to consider an automated RI/FS tracking system that updates, on a monthly or bi-monthly basis, progress on RI/FS and changes in expected completion dates. This system could either use information from regional management tracking systems or be independent of such systems. 3.2.6 Improvement in Site Files In the course of conducting the site reviews, the study team observed that the quality of site files and records varied considerably. At EPA headquarters, for example, a summary document containing key RI/FS contract data showed different dates from the actual dates in documents in the files. More importantly, regional site files sometimes were neither complete nor well organized. Regional records must be in order at all times during the RI/FS, to ensure that key dates, documents, and other information are readily accessible. Coherent files are crucial when cost-recovery issues arise, when the ROD document is prepared, or when public or congressional inquiries are made. In addition, when a new RSPO is assigned to a site mid-way through the RI/FS, good site files make it much easier for the RSPO to become familiar with the complexities of the site and begin immediately to direct and supervise contractor and subcontractor RI/FS activities. In a few extreme cases, the region could not easily locate final versions of plans and reports, correspondence records were incomplete or contained undated material, and the organization of the file made it difficult to discern precisely the chronology of events and the measures taken to resolve documented concerns. In light of these problems, regional program managers and, if necessary, EPA headquarters staff, should conduct periodic management review of regional files. This task could be performed by headquarters or regional staff, or by a contractor such as the FIT. For example, headquarters could issue guidance on file contents, and regions would be responsible for implementing the guidance. Headquarters could oversee regional implementation or provide resources or other assistance to regions to keep site files comprehensive and up-to-date. Headquarters should make an effort to ensure a standard level of file quality across all ten regions to avoid legal or policy complications in the future. ------- - "»8 - 3.2.7 Shorten Contracting Time Lengthy contracting delays occurred at several state-lead sites, and was a particular problem at early sites when state personnel had little or no experience in evaluating proposals or negotiating with contractors. Since then, several states have developed formal contracting procedures and a few have prepared prequalified lists of contractors. This method has met with success in New Jersey and New York. According to Region II staff, when the state of New Jersey first began rontracting RI/FS, five to seven months were required to issue a RFP, review pioposals, and select a contractor. Under the revised contracting procedures, a prequalified list of 14 contractors has been established. For each site contract, three of the 14 contractors are selected and asked to submit a proposal from which the state selects one to conduct the RI/FS. State staff estimate that the process now takes from two to three months and requires less staff time to review and evaluate the proposals. 3.2.8 Expand Staffing to Expedite State Review Increasing state staff resources is one way to decrease the amount of time spent by the state in procuring contractors and reviewing plans and reports throughout the RI/FS. The state project officer for Old Inger had primary responsibility for negotiations with the contractor chosen for the site. Because of conflicting commitments, however, he was unable to devote sufficient time to the contract award process. The state project officer commented that if he had had assistance in preparing contracts at this stage, contract awards could have been made one month earlier. He further suggested that the creation of a separate state-level contracts management division would allow project officers to focus more on technical aspects of the RI/FS and less on the administrative requirements of EPA's contracting process. States that take lead responsibility for an increasing number of RI/FS site projects might find it advantageous to allocate duties in this way. Alternatively, the permanent staff of state project officers could be expanded so that each officer handled fewer sites. At Highlands Acid Pit in Region VI, EPA headquarters assigned a member of its staff, at the state's request, to assist in the review and preparation of the ROD after the RI/FS completion. Supplementing state staffs with members of the regional and headquarters staff has two advantages from the state's perspective, the state receives assistance in completing RI/FS tasks on time and is able to benefit from the greater variety of experience of the regional or headquarters staff. Alternatively, EPA could encourage states with insufficient staffing to opt for federal-lead RI/FS. 3.3 SUMMARY AND CONCLUSIONS The preceding analysis indicates that management improvements in the RI/FS process are needed in several areas. A clear conclusion of the site reviews is that every party involved in RI/FS work must participate in the effort to upgrade RI/FS quality and reduce delays. Moreover, the site reviews indicate that the experience of the Superfund program to date can be used to guide future RI/FS, so that EPA, state, and contractor staff can learn the lessons from past RI/FS and implement an effective national program. ------- APPENDIX A RI/FS DATA ANALYSIS Prior to initiating the reviews of selected sites that are the basis for the body of this report, HSCD collected and analyzed data on 67 state-lead and 139 federal-lead RI/FS. The purpose of this analysis was to determine the average duration and cost for both state- and federal-lead RI/FS, and to identify the extent to which RI and FS completion dates had "slipped" from their originally-scheduled dates. The findings from Phase I became the starting point for Phase II -- the 26 site reviews analyzed in this report. This appendix summarizes the results of the analysis for the Phase I data collection. Section A. 1 presents the method used to collect, verify, and analyze the data; Section A.2 discusses the findings of the analysis of federal-lead sites; and Section A.3 describes the findings of the analysis of state-lead sites. The final section, Section A.4, presents the conclusions of Phase I of the RI/FS study. A.1 METHOD The method employed in the Phase I analysis included a three-stage data collection and verification process and a series of calculations performed to estimate average duration, cost, and slippage in schedules. This section describes the method used, the types of data collected, and the gaps that still remain in these data. A.1. 1 Data Collection and Verification HSCD collected data for both federal-lead and state-lead RI/FS sites. Federal-lead data on 139 sites obligated between June 29, 1982 and June 30, 1984 were supplied by the zone contractors, NUS Corporation and CH2M-Hill. State-lead data on 67 sites obligated between October 1, 1982 and June 29, 1984 were collected from EPA headquarters files and interviews with EPA regional Superfund staff. The dates for each RI/FS milestone were compiled in an automated data base by ICF Incorporated. All data collected on each region were then sent to the appropriate regional office, where staff made corrections and filled in missing data. For purposes of this analysis, the cutoff date for receiving verified data was December 10, 1984. No new or verified data were incorporated after that date. As a result, the data analyzed in this report are not complete, and most estimates were calculated from a subset of the total data base. A.1.2 Description of Data Collected All of the data elements collected for this analysis are either RI/FS milestone completion dates or project cost figures. Both types of data elements are given in planned dates or amounts (i.e., as appeared in the ------- A-2 final work plan or cooperative agreement) and actual completion dates or cost figures (or current estimates, for sites that have not yet been completed). Because a federal-lead RI/FS has a greater number of interim milestones, the federal-lead site data base contains a greater number of data elements than does the state-lead data base, as shown in Exhibit A-l. A.1.3 Calculations Performed With respect to the project milestone data, two types of calculations were made. First, estimates of average duration were derived, including duration of the entire RI/FS and duration of several specific phases of activity. Second, estimates of average slippage in the RI/FS, the RI alone, and the duration between RI and FS completion, were calculated. Both the calculations and the findings of the analysis are discussed in detail in Sections A.2 and A.3 below. For both state- and federal-lead sites, cost data were analyzed to determine the average planned and actual cost (or current estimates, for sites that have not vet been completed) of an.RI/FS, and the average difference between the two. Cost calculations and findings are discussed in sections A.2.4 and A.3.4 of this Appendix. Because specific data elements were missing for several sites, as enumerated in Exhibit A-2, each of the calculations is based on a different number of sites. The number of observations used to calculate each estimate is provided in the discussion of the calculation. A.2 FEDERAL-LEAD SITES: FINDINGS This section discusses the findings of the data analysis conducted for federal-lead RI/FS sites. The first part describes briefly the sources and type of data collected for federal-lead sites. The next section discusses calculations to determine the.average duration of the RI/FS process at federal-lead sites. For purposes of this section the RI/FS is defined to comprise two components, a planning phase and a work phase. The third part of this section focuses exclusively on slippage, or average delays, in the RI/FS process. The final part of this section discusses data collected on federal-lead RI/FS costs. A.2.1 Federal-Lead RI/FS Data Collected Duration and cost data were collected for 139 federal-lead RI/FS sites. NUS was the contractor at 61 of these sites, CH2M-Hill the contractor at the other 78. Most of the data collected were dates for certain milestones in the RI/FS process, although some cost data were collected also. The milestone dates are: work assignment, submission of draft work plan, submission of final work plan, interim authorization, final authorization, RI completion date estimated in the final work plan, RI current estimated or actual completion date, FS completion date estimated in the final work plan, and FS current estimated or actual completion date. The cost data collected were: original obligation amount, approved work plan cost, and cost at completion. ------- A-3 EXHIBIT A-1 DATA ELEMENTS COLLECTED FOR PHASE I ANALYSIS Federal-Lead Sites RI/FS Milestone Dates Work assignment Submission of draft work plan Submission of final work plan Interim authorization Final authorization RI planned completion-" RI actual or current estimated completion FS planned completion'- FS actual or current estimated completion Cost Data Original obligation amount Approved work plan cost Cost at completion State-Lead Sites Cooperative agreement Contract award RI/FS start RI planned completion- RI actual or current estimated completion FS planned completion- FS actual or current estimated completion Original obligation amount Cost at completion For federal-lead sites, these are the dates given in the final work plan. For state-lead sites they are the dates given in the cooperative agreement. ------- A-4 EXHIBIT A-2 DATA ELEMENTS COLLECTED FOR PHASE I ANALYSIS Federal-Lead Sites (N = 139) Number of Sites with Missing Data RI/FS Milestone Dates Work assignment 0 Submission of draft work plan 11 Submission of final work plan 18 Interim authorization 45 Final authorization 58 RI planned completion- 34 RI actual or current estimated completion 51 FS planned completion- 25 FS actual or current estimated completion 41 Cost Data Original obligation amount 0 Approved work plan cost 9 Cost at completion 12 State-Lead Sites fN = 67) RI/FS Milestone Dates Cooperative agreement 1 Contract award 14 RI/FS start 1 RI planned completion- 25 RI actual or current estimated completion 45 FS planned completion- 8 FS actual or current estimated completion 45 Cost Data Original obligation amount 2 Cost at completion 41 For federal-lead sites, these are the dates given in the final work plan. For state-lead sites they are the dates given in the cooperative agreement. ------- A-5 Appendix B lists, for each data element, the definition used in collecting and verifying data for this analysis. The federal-lead data were originally supplied by the zone contractors at the request of HSCD. The data collected were preliminary; numerous gaps existed and the possibility existed that the definitions of the data elements distributed to regional staffs and zone contractors were not interpreted in exactly the same way by different regions or contractors. A verification effort was therefore made through correspondence and telephone calls between HSCD and EPA regional staff and by a review of headquarters site files. Further verification of selected sites was undertaken during trips to seven regional offices, by checking regional site files, and through brief interviews with regional site project officers (RSPOs). The primary purpose of these regional visits was to interview RSPOs for the Phase II RI/FS site reviews, but the visits were also used to verify RI/FS data. For regional offices not visited, data were verified through correspondence and telephone cal Is. Although HSCD made substantial efforts to verify the RI/FS data, some gaps and errors still exist. Roughly half of the sites have dates missing for one or more milestones, as shown in Exhibit A-2. For some sites no dates were given because the milestone had not yet been reached (e.g., sites that have not yet received final authorization) or had never occurred. For other sites, dates that should be available were missing, e.g., some sites had dates for submission of final work plan but did not have RI or FS completion dates from the work plan. In addition to these gaps, errors and inconsistencies were noticed in the data collected, even after verification attempts were made. For example, one site had an FS planned completion date only two weeks later than the date the final work plan was submitted, implying that, at the time the final work plan was submitted, the contractor believed the FS could be finished in two weeks. Even for extremely small or uncomplicated sites, this estimate is far too short a period of time to complete an RI/FS report. HSCD and regional staff tried to identify and correct all obvious data inconsistencies. Less obvious errors and inconsistencies were also corrected when they were identified, but the possibility exists that errors remain in the data base. Overall, however, HSCD believes the data collected to be of sufficient accuracy for this analysis, and the likelihood of significant errors and inconsistencies in the verified data is small. The majority of the data has been verified, in most cases by the RSPO in charge of the site. The results of the data analysis performed to date are discussed in the following subsections. A.2.2 Average Duration of RI/FS The primary focus of the analysis is on duration of the RI/FS: actual, intended, and the difference between the two, which is defined as slippage and discussed in Section A.2.3. This part focuses on statistics calculated for actual average duration of the RI/FS process. Exhibit A-3 presents selected results of the data analysis for federal-lead RI/FS. ------- A-6 EXHIBIT A-3 SELECTED DURATION STATISTICS, FEDERAL-LEAD RI/FS National National Mean Median N * Interval from Work Assignment Date to Actual 587 619 98 or Current Estimated Date for FS Completion (Overall duration) Interval From Work Assignment Date to Final Authorization Date (Planning phase) (a) In days 194 183 91 (b) As a percentage of total RI/FS period 33% 31% 75 Interval from Final Authorization to Actual or Current Estimated FS Completion Date (Work Phase) (a) In days 412 420 71 (b) As a percentage of total RI/FS period 69% 69% 71 Interval from Planned to Actual or Current Estimated RI Completion Date (RI slippage) (a) In days 156 134 86 (b) As a percentage of the period from final authorization to planned RI completion 72% 65% 61 Interval from Planned to Actual or Current Estimated FS Completion Date (Total slippage) (a) In days 141 121 92 (b) As a percentage of the period from final authorization to planned FS completion 44% 43% 64 "N" refers to the number of sites included in the calculation. ------- A-7 As presented in Exhibit A-4, the average duration of a federal-lead RI/FS, from work assignment to the actual or current estimated completion date for the FS, is 587 days, or a little more than one and one-half years. This calculation is based on data from 98 federal-lead sites (54 CH2M-Hill sites and 44 NUS sites). The remaining 41 sites in the data base were missing either a work assignment date or an actual or current estimated FS completion date and were therefore excluded from this calculation. There was little variation in the duration of RI/FS performed by NUS and CH2M-Hill. NUS averaged 591 days from work assignment to FS completion and CH2M-Hill averaged 583 days. A shortcoming of the results generated by this analysis is that, because the same sites were not used in every calculation, the statistics are not directly comparable, as indicated by the last column of Exhibit A-3, which lists the number of sites in each calculation. None of the calculations in Exhibit A-3 use all 139 federal-lead sites and nearly every statistic is based on a different number of observations or "N," an unavoidable consequence of the incompleteness of the data. For example, to calculate overall RI/FS duration, it was necessary to have both the work assignment date and the actual or current estimated FS completion date. The N for this calculation is 98, which means that the data base contained both milestone dates for 98 sites. Calculations of the two phases of the RI/FS were not based on the same number of sites, however, because of missing data. To be included in the calculation of planning phase duration, for example, a site had to-have both a work assignment and final authorization date. Of the total data base, 91 sites had both dates. To calculate work phase duration, it was necessary to have both a final authorization date and an actual or current estimated FS completion date; the number of sites included in this calculation is 71. Thus, the duration estimates for the planning phase (194 days) and the work phase (412 days) do not total the overall duration estimate of 587 days because the three estimates are each based on a different number of sites. Definition of Phases. The RI/FS process was divided into two phases for this analysis, and duration statistics were calculated for each. The planning phase, defined as the period from work assignment to final authorization, is the phase in which work plans are drafted and reviewed; sampling, site safety, and quality assurance/quality control plans are prepared; and subcontractors are hired. The planning phase is roughly the period before field work begins at a site, although at many sites some field work does not begin until a few weeks after the contractor receives final authorization. The work phase, defined as the period from final authorization to the actual or current estimated FS completion date, is primarily the point at which soil and ground water sampling is conducted, sampling results are analyzed and validated, and the RI and FS reports are written. Duration of Planning Phase. The average duration of the planning phase at federal-lead sites is 194 days, one-third of an entire RI/FS. This calculation is based on data from 91 sites. The planning phase was subdivided for this analysis into four intervals, depicted in Exhibit A-5, with the duration of each interval as a percent of the total planning phase. For some ------- Exhibit A-4 FEDERAL-LEAD RI/FS: MEAN DAYS TIME LINE* Work Assignment Initial Authorization Final Authorization Actual or Current Estlme FS Complet a DAYS 143 188 587 PERCENT 0% 24% 32% 100% # Shaded portion represents planning phase of RI/FS. ------- A-9 EXHIBIT A-5 STEPS IN THE PLANNING PHASE OF FEDERAL-LEAD RI/FS Percent of Total Planning Phase Step 21% (1) Work assignment to submission of draft work plan (N = 128)*; 38% (2) Submission of draft work plan to initial authorization (N = 90)*; 16?i (3) Initial authorization to submission of final work plan (N = 87)*; 25% (4) Submission of final work plan to final authorization (N = 88)*. * "N" refers to the numbers of sites used in the calculation. ------- A-10 RI/FS, the third step is reversed, i.e., the contractor submits the final work plan before initial authorization is granted. Duration of Work Phase. The average duration of the work phase of an RI/FS is 412 days, based on data from 71 sites. As noted, this is about two-thirds of the average total RI/FS duration. Conducting the RI accounts for the majority of the work phase, which is not surprising because the field work and laboratory analysis for an RI/FS occurs during the RI. The FS portion is used primarily for writing the FS re^. rt. The planned duration of the RI portion of the work phase -- defined as tue interval between final authorization and the RI completion date in the final work plan -- averaged 210 days. This estimate is more than twice as much as the duration of the planned FS portion of the work phase -- defined as the period from RI planned completion to the FS completion date in the final work plan -- which averaged 95 days. FS work, however, is probably not confined only to the period between RI completion and FS completion; the amount of time attributable to the FS thus may be somewhat greater than 95 days. As discussed in the next subsection, the data verify that the RI/FS process is taking more time to complete than EPA had originally projected. A longer RI/FS duration means that the entire remedial action process, including remedial design and construction, will also take longer to complete because the RI/FS is prerequisite to other steps in the remedial process. A.2.3 RI/FS Slippage Slippage is the delay in the RI/FS process, or the difference between actual and planned completion dates. Average slippage was calculated in three ways: for the entire RI/FS, for the RI alone, and for the FS alone. Key Definitions. RI/FS slippage is defined to be the difference in days between the FS completion date in the final work plan and the actual or current estimated FS completion date, the last step in the RI/FS. In other words, the amount of time by which FS completion is delayed is the overall RI/FS delay. Delays in other stages -- for example, delays in the RI -- may be partly made up during the FS, and thus would not necessarily be equal to the overall delay in an RI/FS. Results of Analysis. RI/FS slippage averaged 141 days for federal-lead sites (N=92) or 46 percent of the planned duration of the work phase (305 days), as shown in Exhibit A-6. In other words, for federal-lead sites the RI/FS work phase is nearly one and one-half times as long as planned. This finding is based on the definition used here that all slippage occurs during the work phase. Slippage is calculated using the completion date in the final work plan. For this data analysis, slippage that occurs in the planning phase, e.g., during the preparation, review, or approval of work plans, is not taken into account. There is no readily available way to measure slippage during the early stages of an RI/FS because no formal schedule exists. Total slippage of 141 days was broken down into two components, RI slippage and FS slippage. RI slippage is defined to be the difference between ------- Exhibit A-6 RI/FS SLIPPAGE IS DUE TO SLIPPAGE IN THE Rl Final Authorization Rl Planned Completion FS Planned Completion ORIGINAL PLANNED WORK PERIOD DAYS ? 10 305 450 Rl Aclual or FS Actual or Current Estimated Current Estimated Completion Completion ACTUAL OR CURRENT ESTIMATED WORK PERIOD 156 Da y s : Rl Slippage -11 Da ya: FS Slippage (Negative) ------- A-12 the RI completion date in the final work plan and the current estimated or actual RI completion date. FS slippage is defined to be the difference between total slippage and RI slippage. Thus, given total slippage of 141 days (based on data for 92 sites) and RI slippage of 156 days (based on data for 86 sites), FS slippage is -15 days. As measured here, FS slippage refers only to that portion of the FS conducted after RI completion, although FS work may begin in some cases before the RI is completed. Negative FS slippage suggests, first, that all slippage occurs during the RI and, second, that some attempt is made during the FS to make up for delay in the RI, or at least .< it to lose any additional time. For example, some RSPOs begin preliminary FS work during delays in the RI, such as while awaiting laboratory analysis results. Most sites in the analysis, however, have not yet reached the FS stage and negative slippage may therefore be due to optimistic projections by RSPOs and zone contractors rather than to actual acceleration of the FS. As more FSs are completed, the negative FS slippage may be erased and slippage may become positive. The possibility that the negative slippage estimates may not be borne out over time is supported by the analysis, which suggests that slippage estimates increase for sites with imminent planned completion dates. Average slippage estimates were compared for sites in the following two categories: Sites with a planned FS completion date on or before December 10, 1984 (49 sites); and Sites with a planned FS completion date after "December 10, 1984 (43 sites). The average total slippage for sites in the first category is 187 days, while total slippage for sites in the second category averages 88 days, or less than half the estimate for the first category. This finding is presented in Exhibit A-7, a scatter diagram showing the relationship between slippage and proximity of the FS planned completion date. The horizontal axis in Exhibit A-7 represents the proximity of the FS planned completion date to the data verification cutoff date of December 10, 1984. Negative numbers on this axis indicate that the planned completion date had already passed on December 10, 1984; positive numbers indicate that completion was planned after December 10, 1984. The vertical axis represents slippage in the RI/FS, or the difference, in days, between the FS planned completion date and the actual or current estimated FS completion date. A negative value on this axis indicates that the actual or current planned completion date is earlier than the planned date; a positive number indicates that the actual or current estimated FS completion date is later than the planned date -- i.e., that the schedule has slipped. The horizontal lines on either side of the "0" days line represent the mean slippage for the two categories of sites. The findings presented in Exhibit A-7 suggest that average slippage estimates may increase over time as more RI/FS near completion. To the extent that this finding is correct and actual slippage is underestimated, regional offices may complete RI/FS even later than they currently estimate, and therefore may not meet their FY85 SCAP commitments. The implication for EPA ------- A-13 Exhibit A-7 Rl SLIPPAGE ESTIMATES DECREASE FOR FUTURE ACTIVITIES RI/FS Slippage 700 - aoo 600 400 - 300 - 200 100 0 - 1 00 - -200 - -300 - -400 - -600 - -600 - -700 - Mean: 187 Days Slippage t b . ' *1 * # Mean = 88 Days Slippage ' 1 1 1 1 1 1 1 1 hi 1 1 1 r -700 -600-600-400 -300 -200 -100 0 100 200 300 400 600 Proximity of FS Planned Completion Date to 12/10/84 ------- A-14 headquarters to consider is that funds budgeted or planned for remedial design and construction in FY85 and beyond may not be spent in a particular fiscal year because RI/FS schedules have slipped. A.2.4. Cost Analysis Although the data analysis focused on dates used to calculate duration and slippage statistics, a few cost statistics were calculated using three cost milestones: original obligation amount, approved work plan cost, and cost at completion, as shown in Exhibit A-8. The most significant result of the cost analysis is that, on average, cost escalations do not appear to be very significant for federal-lead RI/FS sites. The average difference between cost at completion and the approved work plan cost is $31,000, approximately seven percent of the average approved work plan cost. In order to determine the degree to which this cost estimate is affected by the proximity of the actual or current estimated completion date, the data were divided into two categories, as described in section A.2.3, and the estimate was recalculated. Using this approach, the average cost differential for sites with an actual or current estimated FS completion date on or before December 10, 1984 is an increase of $38,000 (based on data from 79 sices). The average differential for the 36 sites with an actual or current estimated completion date after December 10, 1984 was an increase of $28,000. These estimates suggest that, as appears to be the case with slippage, cost escalation may be underestimated until completion of the RI/FS is imminent. EXHIBIT A-8 SELECTED COST STATISTICS, FEDERAL-LEAD RI/FS National National Mean Median N * Approved Work Plan Cost (in $1000) 475 410 131 Actual or Current Planned Cost at Completion (in $1000) 501 434 127 Difference Between Approved Work Plan Cost and Actual or Current Planned Completion Cost (Negative indicates cost escalation) (a) In $1000 (b) As a percentage of completion cost -31 -7% 0 126 0% 126 * N refers to the number of sites used in the calculation. ------- A-15 To assess the extent to which outlying observations affect the estimates, a third calculation was performed in which nine sites -- four with exceptionally large cost savings and five with exceptionally high cost overruns -- were excluded from the calculation. Excluding these outlying observations lowers the average cost escalation from $31,000 to $29,000, not a significant change and one that suggests that these outlier observations essentially cancel each other out. The average of each of the three cost milestones is: Estimated or actual cost at completion -- $501,000; Approved work plan cost -- $475,000; and Original obligation amount -- $331,000. The original obligation amount does not necessarily represent an initial estimate of cost, because at many sites the original obligation amount is intended to cover only preliminary RI/FS activities. A.3. STATE LEAD SITES: FINDINGS This section discusses the findings of the data analysis conducted for state-lead sites for which cooperative agreements were signed between June 29, 1982 and June 29, 1984. -The first part discusses the data base used and calculations made to estimate RI/FS duration. Like the federal-lead analysis, this section divides the RI/FS process into two components, a planning and a work phase. \ The second part of this section focuses on slippage, or average delays in the RI/FS process, and the final part discusses RI/FS costs. A.3.1 State-Lead RI/FS Data Collected The data collected on state-lead sites, like the data collected for federal-lead sites, consists of milestones and costs for state-lead RI/FS. Data were provided by regional site project officers, who reported to HSCD. As presented in Exhibit A-l, the data base for the state-lead sites includes nine elements: cooperative agreement date, original obligation amount, contract award date, RI/FS start date, RI planned completion date, RI actual or current estimated completion date, FS planned completion date, FS actual or current estimated completion date, and cost at completion. In general, the data collected on each state-lead site are not complete, as shown in Exhibit A-2; only three sites contain a complete data set including data on all nine elements. Conclusions drawn from this data must therefore consider the low number of sites used in many of the calculations when interpreting the data. Appendix B lists, for each data element, the definition used in collecting and verifying data for this analysis. A.3.2 Average Duration of State-Lead RI/FS The duration of a state-lead RI/FS is defined as the period from the cooperative agreement date to the actual or current estimated FS completion ------- A-16 date. Of the total 67 state-lead sites in the data base, 45 did not provide an actual or current estimated FS completion date. In calculating duration estimates, if no actual or current estimated FS completion date was provided, then the planned completion date was substituted for the actual or current estimated FS completion date in the calculation. The use of this substitute created a composite date termed the "merged" RI or FS completion date. Use of merged completion dates throughout the analysis of duration greatly increased the number of sites that could be included in the calculations. For example, using the merged FS completion date, the average duration was approximately 680 days, as illustrated in Exhibit A-9, based on the 61 sites for which data were available rather than the 22 sites for which actual or current planned completion dates were provided. Using the FS merged completion date therefore increased the number of sites that could be included in this calculation by more than a factor of three. Similar increases occurred for other calculations, as discussed below. Exhibit A-10 presents selected results of the data analysis for state-lead RI/FS. Definition of Phases. As in the discussion of federal-lead sites, the total time spent on the RI/FS is split, for the purpose of analysis, into two phases. The first of these, the planning phase, begins on the date the cooperative agreement is signed and ends on the RI start date, which is defined as the date the state's contractor has completed all planning and begins field work. The second phase, the work phase, begins on the RI start date and ends on the actual or current estimated FS completion date. The definitions of planning and work phases for state- and federal-lead sites are similar but not identical, so that comparisons of the duration of, for exdmple, state versus federal planning phases could be misleading. The federal-lead planning phase ends with final authorization, which is the final approval of the contractor's work plan. Although many federal-lead sites grant interim authorization to the contractor to begin preparation of the site safety, sampling, and quality assurance plans, these plans were not complete at the time of final authorization for many of the federal-lead sites reviewed in Phase II. One would expect, therefore, that the work phase for state-lead sites would be, on average, two to four weeks shorter than the work phase for federal-lead sites. Similarly, for the planning phase, the state-lead data includes the time required for procuring a contractor. Because federal-lead RI/FS need not procure a new contractor for each site, one would expect the duration for the federal-lead work phase to be shorter, on average, than for state-lead. Comparisons of the durations of state- and federal-lead sites are therefore potentially misleading, and should be viewed with caution. Duration of Planning Phase. Analysis of the state-lead data indicated that an average of 233 days (N=65) elapsed between the cooperative agreement obligation date and the RI start date. This estimate represents approximately 36% of the average total RI/FS duration, from the cooperative agreement obligation date to the "merged" FS completion date. Duration of Work Phase. The work phase, defined as the period from the RI start date to the "merged" FS completion date, took an average of 438 days (N=61). This estimate represents approximately 64% of the total duration of the RI/FS process. ------- Exhibit A-9 STATE-LEAD RI/FS: MEAN DAYS TIME LINE* Cooperative Agreement Obligation Oate RI/FS Start Oate Actual or Current Eatlmated FS Completion DAYS 0 680 PERCENT 0% 35% 100% # Shaded portion represents planning phase. ------- A-18 EXHIBIT A-10 SELECTED DURATION STATISTICS, STATE-LEAD RI/FS Duration Statistics National National Mean Median Interval from Cooperative Agreement > \te to Merged FS Completion (Overall duration) Interval From Cooperative Agreement Date to RI/FS Start Date (Planning phase) (a) In days (b) As a percentage of total RI/FS period Interval from RI/FS Start Date to Merged FS Completion Date (Work Phase) (a) In days (b) As a percentage of total RI/FS period Interval from Planned to Actual or Current Estimated RI Completion Date.(RI s 1 ippage) (a) In days (b) As a percentage of the period from final authorization to planned RI completion Interval from Planned to Actual or Current Estimated FS Completion Date (Total slippage)'- (a) In days (b) As a percentage of the period from final authorization to planned FS completion 680 233 36% 438 64% 89 20° 177 76% 666 230 36% 380 64% 54 22% 162 56% 61 65 60 61 60 14 13 13 13 * N refers to the number of sites used in the calculation. The 60-day exclusion rule was used to calculate these estimates ------- A-19 A.3.3 RI/FS Slippage In addition to calculating the the duration of specific phases of the RI/FS, the analysis addressed the extent to which RI and FS completion dates had slipped from the dates originally scheduled in the cooperative agreement. In order to calculate slippage, data must be available on both planned and actual or current estimated completion dates. Key Definitions. Three slippage definitions are important to this analysis. The first is overall RI/FS slippage, defined as the duration from the FS planned completion date that appears in the cooperative agreement, and the actual or current estimated FS completion date. The second is RI slippage, defined as the duration from the planned RI completion date to the actual or current estimated RI completion date. Finally, FS slippage is defined as overall RI/FS slippage minus RI slippage. In addition, a decision rule termed the "sixty-day exclusion rule" was used in some of the slippage calculations. The purpose of this rule is to eliminate sites with completion dates that may be unreliable because they are more than 60 days hence. When this rule is used, data on sites with an FS completion date beyond February 8, 1985 -- sixty days after the data verification cutoff date of December 10, 1984 -- are excluded from the analysis. Without the rule all sites with the relevant statistics are included in the analysis. Results of Analysis. Slippage at state-lead sites was calculated both with and without the sixty-day exclusion rule. First, without the use of the sixty-day rule, overall slippage -- ot the difference between the planned FS completion date and the actual or current estimated FS completion date -- is 145 days. This calculation is based on data from 17 sites. Based on data from 16 sites, RI slippage accounts for 77 days or 53% of the total slippage, and slippage in the FS process accounts for the rest (approximately 66 days although, because of the different number of sites used in each slippage calculation, the RI and FS slippage durations do not add to 145 days). As expected, the "sixty-day exclusion rule" works to increase the estimates of slippage. Using the sixty-day exclusion rule, total slippage is 177 days, compared to 145 days using all sites. RI slippage is 89 days (versus approximately 68 days). Again, it must be emphasized that these results should be treated with caution, as they are based on data for only 13 sites. Although the sixty day rule decreases the number of sites used in the calculation -- from sixteen sites to fourteen sites in the RI calculations, and from seventeen to thirteen in the FS calculations -- the smaller base may be the more accurate one. Projected dates more than two months in the future are more likely to change than dates closer to the present. As with the federal lead data, then, the average slippage for the sites with relatively close completion dates is probably the more reliable estimate. A.3.4. Cost Analysis The study team calculated the variance between the original obligation amount and the actual or current estimated cost at completion, as presented in ------- A-20 Exhibit A-ll. Forty-one sites did not provide a current estimate of the cost at completion. Because state-lead RI/FS generally use fixed price contracts, it was assumed that cost escalation should not be imputed simply because an actual or current estimated cost was not provided. The original obligation amount was therefore used for these forty sites, making the variance zero at these sites. With a data base of sixty-five sites, the cost at completion exceeded the original obligation amount, on average, by approximately $46,000, at 6.6% average cost escalation. Applying the sixty-day rule to this analysis increases the average cost escalation to $56,000 or 10.5% over the original obligation cost. EXHIBIT A-ll SELECTED COST STATISTICS, STATE-LEAD RI/FS National National Duration Statistics Mean Median N- Original obligation Amount (in $1000) 484 434 65 Actual or Current Planned Cost at Completion (in $1000) 524 457 66 Difference Between Original Obligation Amount and Actual or Current Planned Completion Cost (Negative indicates cost escalation) (a) In $1000 -46 0 65 (b) As a percentage of completion cost -7% 0% 65 N refers to the number of sites used in the calculation. ------- A-21 A. 4 SUMMARY AND CONCLUSIONS The Phase I analysis was undertaken to determine the extent to which the RI/FS process has been subject to delays or cost escalations. The key findings of the Phase I effort are: The average federal-lead RI/FS lasts over 19 months, from work assignment to submission of the final FS report; Over six months on average are required for developing a work plan and granting final authorization to proceed with field work; Slippage in completing the RI accounts for over five months on average compared to the schedule in the final work plan; Slippage estimates for federal-lead sites may underestimate actual slippage, particularly for RI/FS to be completed in the coming months. Average slippage for sites with a planned RI/FS completion date before December 10, 1984 is more than 6 months, but the corresponding average for RI/FS to be. completed after December 10, 1984, is less than three months. These findings raise concern about projects currently projected to be completed in FY85 and beyond, and the amount of funds to be budgeted for remedial design and construction. The average state-lead RI/FS lasts nearly 23 months. Slippage for state-lead RI/FS averages nearly five months, but the small number of sites in the data base (N = 13) indicates the need for caution in interpreting these results. These findings suggest that both schedule slippage and, to a lesser extent, cost escalation are significant problems meriting further investigation and analysis. Phase II of this study was undertaken to address the questions raised by the Phase I findings, and to assess the specific causes of the delay and cost escalation at selected remedial sites. ------- APPENDIX B RI/FS PROGRAM DATA CHECKLIST FOR DATA VERIFICATION As part of the statistical analysis on the RI/FS program data, HSCD is requesting that the regional staffs review and verify the data used. This paper is intended to serve as a checklist for this process to ensure that verification efforts across the regions are as uniform and as complete as possible. It is organized as follows: 1. Approach to Data Verification: Lists, step-by-step, the procedure to be used in verifying the existing data base. 2. Definition of Terms: Defines the different authorization dates in termsof the work being performed. Any additional comments or suggestions on the RI/FS data base which may arise during this verification process should be noted. Future analyses of this data will attempt to incorporate suggestions from both regional staffs and contractors. 1 . APPROACH TO DATA VERIFICATION For each site, the procedure is as follows: 1. Check the definition of the data label for each column. 2. Note the specific data entry in each column. 3. If the entry is incorrect or missing, note the correct entry. If the data are unavailable, note "N/A" on the form and explain. 4. If the data are unavailable but a proxy exists, record the proxy, but explain how it differs from the data requested. 2. DEFINITIONS Regional staff should comment on whether they find these definitions appropriate and helpful toward understanding the RI/FS process. In some cases, informal communications between EPA and the contractor may allow work to proceed in the absence of formal document agreements. Staff and contractors should supply these informal agreement dates whenever possible. ------- B-2 2.1 Federal-Lead Data Work Authorization (Obligation) Date: The date EPA decides that there will be an RI/FS at this site and gives the contractor the go-ahead to begin planning. Original Obligation Amount: Dollar estimate by EPA of RI/FS costs. D,ift Work Plan Date: The date the contractor turns in a draft work plan to EPA~. Initial (Interim) Authorization Date: The date EPA authorizes preliminary field work to begin, presumably based upon review of draft work plan. Final Work Plan Date: The date the contractor submits the final work plan to EPA after all redrafts and revisions are completed. Final Authorization Date: The date EPA approves the final work plan. Approved Work Plan Cost: Cost determined by the contractor in the final work plan. RT Planned Completion Date: Completion date, determined by the contractoro in the final work plan, on which the contractor expects to submit the final version of the RI report.- FS Planned Completion Date: Completion date, determined by the contractor in the final work plan, on which the contractor expects to submit the final version of the FS report which would be available for public comment. RI Actual- or Current Planned Completion Date: Date on which the final RI report was in fact received or the latest revised planned date on which the final RI report will be received. FS Actual- or Current Planned Completion Date: Date on which the final FS report (the version to be available for public comment) was in fact received or the latest revised planned date on which the final FS report will be received. Cost at Completion: Actual or current planned completion cost. Delay Comments: Comments A through F are provided by the contractors to explain delays in RI/FS completion. Comments G through L are provided in the EPA award fee evaluations. "Please indicate actual RI or FS completion dates with an asterisk (*). ------- B-3 Comment Codes: Contractor Comments A EPA/State review delays B Site access delays C CLP delays D Data validation delays E Policy problems (e.g., sites on Indian lands which have jurisdictional ambiguities, need for more study to complete Recorii of Decision). F Enforcement involvement delays EPA Comments G Contractor unresponsiveness H Contractor technical inadequacy I Contractor scheduling and management problems J Contractor communication or reporting delays K Contractor problems in managing subcontractor L Weather delays State-Lead Data Cooperative Agreement- Qbligation Date: The date the cooperative agreement is signed by the state. Original! Obligation Amount: The total amount awarded to the state by EPA in the original cooperative agreement. Contract Award Date: The date the state awards the contract for site work to a private contractor. RI/FS Start Date: The date the contractor begins on-site work. If this date is not available, use the date state authorizes the contractor to begin on-site work. RI Planned Completion: Completion date determined by the contractor in the final work plan. FS Planned Completion: Completion date determined by the contractor in the final work plan. RI Actual or Current Planned Completion: Date on which RI was in fact completed or the latest revised planned completion date. FS Actual or Current Planned Completion Date: Date on which the final RI/FS report was submitted by the contractor or the most current planned estimate of the completion date. Cost at Completion: Actual cost at completion, including state costs and any cost amendments or current lanned completion cost. ------- |