^60 sr^ V. \ PR01^° OFFICE OF INSPECTOR GENERAL Catalyst for Improving the Environment Audit Report Response Action Contracts: Structure and Administration Need Improvement Report No. 2005-P-00001 December 6, 2004 ------- Report Contributors: Stephen Burbank Heather Layne Denton Stafford Charles Watts Myra Mackall Michael Petscavage Abbreviations CPAF Cost Plus Award Fee EPA Environmental Protection Agency FAR Federal Acquisition Regulation ID/IQ Indefinite Delivery/Indefinite Quantity LoE Level-of-Effort NIH CPS National Institutes of Health Contractor Performance System OAM Office of Acquisition Management OARM Office of Administration and Resources Management OIG Office of Inspector General OSRTI Office of Superfund Remediation and Technology Innovation OSWER Office of Solid Waste and Emergency Response PBS A Performance-Based Service Acquisition RAC Response Action Contract RACMIS Remedial Action Contract Management Information System USACE U.S. Army Corps of Engineers ------- ** U.S. Environmental Protection Agency 2005-P-00001 » r. nff;ranflncna^r:Qnarol December 6,2004 I o Office of Inspector General *> / ^ PRO*^ At a Glance Catalyst for Improving the Environment Why We Did This Audit We conducted this audit to determine how effectively and efficiently EPA is administering Response Action Contracts (RACs). We looked at the following areas: Acquisition Planning: How are RACs structured and funded? Source Selection: How does EPA decide with whom to contract? Is past performance considered? Contract Administration: Are there good measures for assessing contractor performance? Contract Information Systems: Do contract managers have the information needed to evaluate results and make decisions? Background RACs are used to obtain professional Architect-Engineer, technical, and management services in support of EPA" s Superfund cleanup responsibilities. Current RACs, which expire between 2005 through 2009, have a maximum potential value of more than $4 billion. For further information, contact our Office of Congressional and Public Liaison at (202) 566-2391. To view the full report, click on the following link: www.epa.gov/oig/reports/2005/20041 206-2005-P-00001.pdf Response Action Contracts: Structure and Administration Need Improvement What We Found EPA can improve the structure of RACs to better protect the Government's interests. Current RACs, which are Cost Plus Award Fee Level of Effort contracts, assign to EPA a disproportionate share of the risk of cost overruns; expose EPA to the risk of loss of funds through litigation; limit competition; and forego potential cost savings associated with other approaches to contracting, such as Performance-Based Service Acquisition. EPA regions do not consistently document the rationale used to decide what procurement option to utilize for Superfund cleanup activities as required by established policy. Further, EPA does not have a process to measure and disseminate information on the U.S. Army Corps of Engineers" past performance in support of EPA. The Agency has measures in place to assess contractor performance at the work assignment level. However, evaluations at the contract level were not being documented timely and consistently, as required, because they were not given the necessary priority. Not consistently documenting evaluations in a timely manner does not permit EPA and other Federal agencies to consider contractors" past performance and could be detrimental to contractors who have performed well. Contract managers have, or can obtain, the information needed to evaluate results and make decisions, but the information in the national automated database is not always readily available. The Remedial Action Contract Management Information System is underutilized by regional staff, and the system does not collect national data as originally intended. As a result, EPA is expending approximately $1.5 million a year on a system that is not being fully utilized. What We Recommend We recommend that the Office of Solid Waste and Emergency Response, in coordination with the Office of Administration and Resources Management, develop and implement a plan with milestones that will increase the use of different contract types, require regional staff to document the rationale for all source selection decisions, develop a method for holding Contracting Officers accountable for conducting past performance evaluations timely and accurately, and conduct a cost benefit analysis to determine whether the Remedial Action Contract Management Information System should be retained. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY fff mmr Twfc 7* | mLim % WASHINGTON, D.C. 20460 \ pantf.^ OFFICE OF n° INSPECTOR GENERAL December 6, 2004 MEMORANDUM SUBJECT: Response Action Contracts: Structure and Administration Need Improvement Report No. 2005-P-00001 FROM: Robert Mitchell, Director for Contract Audits \Signed\ Office of Inspector General (242IT) TO: Thomas P. Dunne, Acting Assistant Administrator Office of Solid Waste and Emergency Response (5101T) David J. O'Connor, Acting Assistant Administrator Office of Administration and Resources Management (3101 A) This is our final report on the subject audit conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe problems the OIG has identified and corrective actions the OIG recommends. This audit report represents the opinion of the OIG and the findings contained in this report do not necessarily represent the final EPA position. Final determinations on matters in this audit report will be made by EPA managers in accordance with established audit resolution procedures. Action Required In accordance with EPA Manual 2750, the Acting Assistant Administrator for the Office of Solid Waste and Emergency Response, as the designated Action Official, is required to provide a written response to the report within 90 calendar days of the report date. The Action Official should include a corrective action plan for agreed upon actions, including milestones dates. If you or your staff have any questions, please contact me at Mitchell.Robert@epa.gov. or Stephen Burbank, Assignment Manager, at Burbank. Stephen@epa.gov. ------- Tables of At a Glance Chapters 1 Introduction 1 2 Response Action Contracts Need to be Restructured 6 3 Procurement Decisions Not Adequately Documented 14 4 Past Performance Evaluations Not Documented Timely and Consistently 18 5 Utilization of Remedial Action Contract Management Information System Could be Improved 21 Appendices A Schedule of Response Action Contracts Reviewed 23 B Agency Response 24 C Distribution 29 ------- Chapter 1 Introduction Purpose We conducted this audit to determine how efficiently and effectively the Environmental Protection Agency (EPA) is administering Response Action Contracts (RACs). RACs are used to obtain professional Architect-Engineer, technical, and management services in support of EPA's Superfund cleanup responsibilities. The current RACs, which expire between 2005 through 2009, have a maximum potential value of more than $4 billion (see Appendix A.). The Agency is planning to replace these contracts with a new set of contracts (RAC II) as they expire, which will have an estimated maximum potential value of $4 billion as well. Our objectives were to answer the following questions: Acquisition Planning: How are RACs structured and funded? Source Selection: How does EPA decide with whom to contract (other government agencies, private sector)? Does past performance inform the source selection decision? Contract Administration: Are there good measures for assessing contractor performance? Contract Information Systems: Do contract managers have the information needed to evaluate results and make decisions? Background Congress enacted the Comprehensive Environmental Response, Compensation, and Liability Act of 1980, commonly referred to as "Superfund," to address threats to human health or the environment resulting from releases or potential releases of hazardous substances. EPA has primary responsibility for managing cleanup and enforcement activities under Superfund. In the cleanup of Superfund sites, EPA uses a variety of instruments (such as contracts, assistance agreements, and interagency agreements) to obtain Superfund removal and remedial services. EPA assigns work to RAC contractors, States, the U.S. Army Corps of Engineers (USACE), or the U.S. Bureau of Reclamation based upon factors such as: Site Characteristics Remedy Characteristics Local/Public Interest Experience/Regional Infrastructure Capability/Capacity of Contractors ------- Conflict of Interest Unique Site Needs RACs provide professional Architect-Engineer, technical, and management services in support of EPA's remedial response, enforcement oversight, and non- time critical removal activities at Superfund sites. Contract services performed under RACs include site management, remedial investigation and feasibility studies that define the nature and extent of contamination, engineering services to design remedial actions, subcontracting for remedial actions, construction management, and enforcement support. Services may also include technical and management services supporting EPA's coordination and oversight of remedial activities performed by a State, the USACE, the U.S. Bureau of Reclamation, or responsible parties identified in enforcement actions. Through this "one- program" approach, where most services needed can be ordered under one contract, EPA expected to reduce disputes between design and construction contractors, promote program integration, and give the regions flexibility to pursue various enforcement options. EPA Headquarters offices and the regions are responsible for administering RACs. In the Office of Solid Waste and Emergency Response (OSWER), the Office of Superfund Remediation and Technology Innovation (OSRTI) is responsible for overall management of the Superfund program. Within OSRTI, the Contracts Management Branch provides leadership for Headquarters Superfund contracts. OSRTI also provides planning, oversight, and support for all regionally-administered and regional-support contracts, including RACs. The Office of Acquisition Management (OAM), within the Office of Administration and Resources Management (OARM), is responsible for the policies, procedures, operations, and support of the Agency's procurement and contracts management programs, from contract planning through closeout. Within each region, Remedial Program Managers, Contracting Officers, and Project Officers administer RACs. Scope and Methodology We performed this audit from April 2003 to July 2004 in accordance with Government Auditing Standards, issued by the Comptroller General of the United States. Our audit included site visits to EPA Headquarters and four regions, as well as attending the 2003 National Superfund Project Officer/Contracting Officer Training Conference. We reviewed all 16 active RACs and data from EPA Headquarters and Regions 1, 2, 5, 6, 7, 9, and 10. 2 ------- The regions were selected based upon the following criteria: Contract capacity (maximum potential value). Number of sites on the National Priority List within each region. Amount of unallocated obligations on contracts (bulk funding). Use of Interagency Agreements (number and dollar amounts of agreements). To determine how RACs are structured and funded, we interviewed staff in Headquarters and the regions to gain an understanding of the planning, budgeting, and funding process for RACs. We reviewed the Federal Acquisition Regulation (FAR) Part 7, Acquisition Planning', the Contracts Management Manual; the RACs Users Guide; the RACs Statement of Work; the EPA Acquisition Regulation; and other pertinent documentation. We also interviewed staff at the USACE. We did not perform work at the U.S. Bureau of Reclamation because they receive a small percentage of Superfund money compared to the USACE. To determine how EPA decides with whom to contract and whether past performance informs source selection decisions, we interviewed headquarters staff, Remedial Project Managers, Project Officers, and Contracting Officers. We reviewed guidance issued to the regions on determining to whom to assign cleanup work. We also reviewed Interagency Agreements documents; Interagency Agreement Decision Memorandums; and an EPA report, "Evaluation of the Performance of the Corps of Engineers in Support of EPA's Superfund Program," on the USACE's support of Superfund. We also reviewed EPA's implementation plan to address the recommendations from the above report and reviewed, on a test basis, the documentation for selected source decisions. We interviewed OAM staff to identify the processes and systems the regions should be using to assess contractor performance. We interviewed regional contracting officers to determine the processes and systems actually being used. We reviewed pertinent documentation such as the RACs Users Guide and Award Fee Plans. We reviewed the FAR and EPA requirements pertaining to conducting performance evaluations. We reviewed award fee and annual evaluations for compliance with those requirements. We also tested the accuracy and timeliness of performance evaluations that were recorded in the National Institutes of Health Contractor Performance System (NIH CPS) by comparing them to the source documents. To determine whether contract managers have the information needed to evaluate results and make decisions, we conducted interviews in the regions and Headquarters to determine the information needed and the information systems used to administer the RACs, and assessed whether these systems readily provide needed information. 3 ------- Prior OIG Reports We reviewed the following Office of Inspector General (OIG) reports and determined that the recommendations in these reports, as they pertained to this audit, have been resolved: Acquisition Management: OMB Requested Review of EPA Contracting, Report No. E1SKF7-04-0037-7100301, issued September 30, 1997. Superfund: Superfund Interagency Agreements, Report No. 2001 -P-00011, issued June 22, 2001. Internal Control Structure In planning and performing the audit, we reviewed management controls relating to our objectives, and our concerns with the lack of controls or the effectiveness of existing controls are discussed in the following chapters. We also examined the Fiscal Year 2003 Federal Managers' Financial Integrity Act Annual Assurance Letters issued to the then Acting EPA Administrator by the Principal Deputy Assistant Administrator for OSWER and the Assistant Administrator for OARM. Neither assurance letter recommended any material, Agency-level weaknesses nor management challenges in the administration of RACs. Compliance With Laws and Regulations EPA complied with the laws and regulations (the FAR) pertaining to its efforts to administer and manage RACs with the exception of the requirement for timely input of past performance evaluations into the NIH CPS as discussed in Chapter 4. Needed process improvements are discussed in Chapters 2, 3, 4, and 5. Results of Review EPA can improve the structure of the RACs to better protect the Government's interests. The current RACs, which are Cost Plus Award Fee Level of Effort contracts, assign to EPA a disproportionate share of the risk of cost overruns; expose EPA to the risk of loss of funds through litigation; limit competition; and forego potential cost savings associated with other contracting approaches, such as Performance-Based Service Acquisition. Also, EPA regions do not consistently document the rationale used to decide what procurement option to utilize for Superfund cleanup activities, as required by established policy. The Agency has measures in place to assess contractor performance at the work assignment level, but evaluations at the contract level were not being documented timely and consistently as required. Contract managers have, or can obtain, the information needed to evaluate results and make decisions, but it is not always readily 4 ------- available; the Remedial Action Contract Management Information System is underutilized by regional staff, and the system does not collect national data as originally intended. We recommended that OSWER, in coordination with OARM, develop and implement a plan for RAC II that increases use of Indefinite Delivery/Indefinite Quantity, task order and site specific contracts. We also made recommendations to review and revise the recertification policy, issue clarifying guidance on the "Superfund Policy for Assigning Remedial Work to the US Army Corps of Engineers (USACE)," provide training and hold Contracting Officers more accountable for evaluating contractor performance, and complete the proposed cost benefit analysis to determine whether the Remedial Action Contract Management Information System should be retained. We held an exit conference with the Agency on November 17, 2004. EPA provided written comments to our draft report on October 29, 2004. Where appropriate, we made revisions to our report or incorporated the Agency's comments and our evaluation of EPA's comment. We have included EPA's complete response as Appendix B. 5 ------- Chapter 2 Response Action Contracts Need to be Restructured EPA can improve the structure of RACs to better protect Government interests. The current RACs, which are Cost Plus Award Fee (CPAF) Level of Effort (LoE) contracts, assign to EPA a disproportionate share of the risk of cost overruns; expose EPA to the risk of loss of funds through litigation; limit competition; and forego potential cost savings associated with other approaches to contracting, such as Performance-Based Service Acquisition (PBSA). Some Agency officials maintain that the CPAF-LoE contracts offer the following advantages: their flexibility allows the Agency to devote resources to other sites when warranted; they are funded in a way that allows the Agency to continue work when appropriations are delayed; and they prevent managers from having to return funds to Headquarters for recertification. While there are benefits associated with the current contract type, there are better contracting approaches available that should, among other things, reduce EPA's cost risk and risk of loss through litigation. Options Available to Reduce EPA's Risks EPA should utilize available historical data to move away from CPAF LoE contracts. The services obtained under the RACs have been performed for EPA for more than 20 years and EPA has extensive historical data to draw upon. In addition, the nature of cost reimbursement contracts requires EPA to monitor contractor performance closely and places more risk for cost and performance on the Government. According to FAR 16.103(a), selecting the contract type is generally a matter for negotiation. It requires Contracting Officers to exercise sound judgment and avoid protracted use of a cost-reimbursement or time-and-materials contract after experience provides a basis for firmer pricing. The objective is to negotiate a contract type and price (or estimated cost and fee) that will result in reasonable contractor risk and provide the contractor with the greatest incentive for efficient and economical performance. FAR 16.104(d) and (k) state that complex requirements, particularly those unique to the Government, usually result in greater risk assumption by the Government. As a requirement recurs or as quantity production begins, product descriptions or descriptions of services to be performed can be defined more clearly, the cost risk should shift to the contractor, and a fixed-price contract should be considered. 6 ------- There are various types of contracts available for use by EPA that offer advantages over the present contract type. Regions 6, 7, and 10 have decided to move away from CPAF work assignment contracts and to Indefinite Delivery/ Indefinite Quantity (ID/IQ) task order contracts for the next round of RACs. Task order contracts offer the following benefits: Task orders can be ceiling price (best effort) or fixed price (completion). Funds are obligated by task order and cannot be used to fund another task order without deobligation first. Cost recovery is easily identified to a site because of site specific accounting and reporting. An ID/IQ contract is a contract for services that does not procure or specify a firm quantity of service and that provides for the issuance of orders for the performance of tasks during the period of the contract. Benefits of ID/IQ contracts include: Providing flexibility in both quantities and delivery scheduling. Limiting the Government's obligation to the minimum quantity specified in the contract. Offering the ability to place term or completion type orders, cost reimbursement, or fixed priced orders. Providing funding flexibility (funding in increments). Providing opportunity and potential for costs savings through better contractor cost performance and accountability. Enabling the obligation of funds under individual task orders. Providing better opportunity for PBSA. Region 10 awarded new ID/IQ contracts (one small and one large business) in July and December 2003, but Regions 6 and 7 have not awarded their contracts. The contract specialist and the Project Officer for Region 10 both believe that the new contract requires more initial planning but less overall contract administration. For example, task orders must be planned and funded site specifically, which requires better initial planning and cost estimating than traditional work assignments. However, these new contracts do not have award fees, so performance evaluation boards are not needed. Also, these contracts are loaded fixed-rate (all overhead and program management expenses are built into the hourly labor rate) and do not have separate program management accounts to administer nor require routine indirect cost rate audits. Region 10's new contract generally has the same statement of work as the current RAC, as well as some of the same flexibilities in the type of task orders issued. Task orders issued can be cost-reimbursable (term form or completion form) or fixed price. However, the task order contract provides better funds control because the task order obligates the funds. 7 ------- The USAGE procures remedial services similar to EPA, and faces many of the same uncertainties and funding hurdles. However, term form contracts are discouraged by USACE and are not used when procuring remedial services. Under their Pre-placed Remedial Action Contract, the USACE awards a task order, detailing the services required much like EPA. USACE purchases complete tasks rather than labor hours. If site conditions change, USACE can issue a "Work Variance Notice," which will change the task order requirements within the overall scope of work of the task order. This demonstrates that remedial cleanup work can be accomplished with contracts structured as other than CPAF LoE. We believe EPA is moving in the right direction with three regions changing contract type. The Agency should expedite a lessons learned analysis of the new ID/IQ contracts as soon as sufficient data is available. RACs are 10-year contracts, and EPA should confirm whether in fact the ID/IQ approach reduces risk and lessens contract administration effort. If the results of the review shows this to be the case, then the RAC II contracts in the seven remaining regions should not be CPAF-LoE. If this information is not available in time for award of the remaining contracts, and any of the contracts awarded are CPAF-LoE, then the results of the review should be factored into the decision whether to exercise the option periods. Contract Structure and Size Limit Competition RACs are full-service Architect-Engineer contracts that can provide most Superfund site cleanup services. The contracts' maximum value can exceed several hundred million dollars. EPA has historically been reluctant to reduce the overall scope of the RAC, which currently can provide most services required over the lifetime of a site. The wide array of services currently provided, along with the RACs' long performance periods, may limit competition to only the largest firms capable of providing these services. This limits the participation of smaller firms, which may further limit competition. Per FAR 16.306(d)(2) and (3) Two Forms of Cost-reimbursable Contracts Completion form contracts describe the scope of work as a clearly defined task or job, with a specific end product required. If the contractor does not complete the work within the estimated cost, the Government may require more effort without an increase in fee, provided the Government increases the estimated cost. Term form describes the work in general terms and obligates the contractor to devote a specified level of effort for a stated period of time. At the completion of the performance period, the contractor is paid cost and fee if performance is considered satisfactory. Additional work is considered a new acquisition. 8 ------- EPA's Contracts 2000 workgroup recommended in 1999 that the design and construction portions of the RACs be broken out into separate contracts. Advantages of separating design from construction are: Increased opportunities for fixed price, completion form, and performance based contracting. Increased competition in the contracting process. Increased small, minority, and women-owned business participation in the Superfund contracting program. Disadvantages of separating design from construction are: Limited EPA experience awarding and managing construction contracts. Resources needed to develop construction management expertise. Instead of mandating the separation of design and construction contracts, the later Design Construction workgroup advocated, and EPA adopted, a menu approach whereby regions could choose to break out the design and construction portions, or continue to utilize full service RACs. Agency officials stated that regional resistance to change prompted the switch to the menu approach. While the menu approach seems to balance national consistency with regional flexibility, there is limited assurance that it will lead in all cases to increased competition, increased small, minority, and women-owned business participation, or use of performance based, multiple award, and fixed price contracts. There is no guarantee that the regions will select other types of contracts, and they may stay with the existing contract type, CPAF. OSRTI staff advised that in the remaining contract awards, the regions will be required to identify at least one contract for Small Business Set-Aside. They also advised that while one of the remaining seven regions is not changing contract type, the others are considering doing so. EPA at Risk of Losing Funds Through Litigation Obligating funds on an LoE contract in excess of identified needs recently resulted in losing contract funds as a result of litigation, and this could happen again. In January 2002, the U.S. Court of Appeals for the Federal Circuit held EPA liable to pay a contractor all funds allotted to a contract. In this case, EPA ordered LoE far below levels called for by the contract, and the contractor was able to prove resulting damages. The contract, although not a RAC, contained clauses identical to those in the current LoE RACs. 9 ------- As a result of the court ruling, EPA issued a document stating, in part, "OAM and Program Offices needed to guard against the practice of funds being allotted to a LoE contract for the purpose of preserving funds for other uses, rather than for use in reimbursing incurred contract costs. . . Regions order work by issuing work assignments. Work assignments are tasking documents but are not obligating documents, so RAC funds are obligated at the contract level, then work assignments are issued against those obligations. This practice, known as bulk funding, places funds on the contract before work is ordered, in some cases years in advance. Regional offices use bulk funding to preserve funds, and to provide the flexibility to move funds between sites. For example, if unanticipated delays occur, regional managers are able to redirect contractor services and shift those funds to higher priorities within the region. Regional officials stated that the amount of bulk-funded obligations on each contract generally corresponds to work ordered on work assignments. However, regional officials informed us there will be a small percentage of the total bulk funding that is not yet supported by approved work assignments, which represents planned new work, some program management, and award fee. The Agency needs to actively monitor RAC funding activity. As of February 2004, there was approximately $23.4 million of bulk funding on RACs. The Regional Project Officers had plans to order approximately $14.8 million of work against these obligations. The remaining $8.6 million appears to be funding in excess of the regions' identified needs and "at risk" in light of the court decision. EPA's April 2004 report, SUPERFUND: Building on the Past, Looking to the Future, stated that some of the regions, at their present expenditure rate, had placed enough funding on their RACs to be able to utilize them for 1.2 years to 4.6 years in the future without placing additional funds on the contracts. The report recommended that OARM and OS WER work closely with the regions to monitor contracts to ensure that the regions have not funded their contracts into the future to an extent where they cannot appropriately use the funds during the contract period. Regional Desire and Funding Issues Drive the Need for Flexibility The RACs' structure and funding method support the regions' desire for flexibility to deal with unpredictable site conditions. However, EPA does not actively track the frequency and scope of these unknown conditions. Agency officials said these unpredictable conditions are an inherent part of their business, and the RAC's flexibility allows contract managers to easily adjust. They stated they need this flexibility because of the way EPA funds its work and because of its deobligation and re-certification procedures, which direct that 75 percent of deobligations be returned to Headquarters. 10 ------- Regional officials stated that for the past several years, they have received their appropriation late in the fiscal year, as late as the second or third quarter. In these situations, regional officials believe it would be difficult to manage and start new task orders. With task orders, work has to be identified and costs negotiated before task orders are awarded and funds obligated. They stated regional offices, therefore, would have difficulty issuing new task orders and obligating funds before the end of the fiscal year. By contrast, under current procedures the RACs are bulk funded. Work assignments are subsequently issued and amounts negotiated. If site conditions change and cause an unexpected work stoppage, funds can easily be shifted to other work. Moreover, under the current deobligation and recertification procedures, managers using task orders would have to return 75 percent of the deobligated funds to Headquarters, rather than just shifting the funds to a different work assignment. We do not believe that the desire to avoid EPA's funds recertification process should be allowed to drive the contract structure. Contract type should be determined based on Agency needs and the risks involved. As discussed above, EPA has been managing site cleanups for more than 20 years, and this experience should allow for the use of contract types that shift risk from the Government to the contractors. EPA Acknowledges that Benefits Can be Achieved Through Different Contract Type and Structure The EPA April 2004 Superfund report previously mentioned concluded that "while pursuing alternative types of contracts (i.e., performance-based, site- specific, and task order contracts) will require a greater investment in Agency and Superfund program time and personnel, done properly, these different contract types can result in significant cost savings to the program." The report recommended that OARM, OSWER, and the regions work together to encourage the use of alternative contract types, and that OARM and regional contracting officers should offer regular training for contract personnel, regional project managers, on-scene-coordinators, and project officers in alternative contract mechanisms. Procuring remedial actions on a site-specific basis offers certain benefits. It provides more opportunity for fixed price work, eliminates some potential conflicts of interest, provides more competition for remedial action work, provides more opportunity for small businesses, and provides more opportunities for PBSA. EPA could reap many of these benefits by increasing its use of site-specific contracts. With the expiration of the current RACs, the Agency has the opportunity to reduce costs and increase the availability of funds for the remedial program by moving away from CPAF contracts to ID/IQ task order contracts, separating design from construction and utilizing more site specific contracts in the new round of RACs. 11 ------- Recommendations We recommend that the Acting Assistant Administrator for OSWER, in coordination with the Acting Assistant Administrator for OARM: 2-1 Develop and implement a plan with milestones for RAC II which: a) increases use of ID/IQ, task order, and site specific contracts, and b) increases the use of separate contracting for design, construction, and other remedial services. 2-2 Conduct a lessons learned analysis of the new ID/IQ contracts as soon as sufficient data is available and develop a plan to share and utilize the results of the analysis. 2-3 Stop funding RACs in excess of identified needs. 2-4 Review and revise the recertification policy so as to remove it as an impediment to utilization of better contract types. The policy should reflect a balance between the need to return funds to Headquarters for re- distribution to where the most pressing needs are identified, and the degree of flexibility Headquarters is willing to grant to the regions to move funds between sites as they now do with work assignments. Agency Comments and OIG Evaluation EPA generally agreed with most of the recommendations. In response to recommendation 2-1, the Agency stated it developed plans to increase the use of ID/IQ contracts and that its current strategy (Contracts 2000 - Design/Construction Contracts Decision memorandum dated September 15, 2000) already allows for separate contracts for design, construction, and other remedial services; and that Regions 7 and 10 are piloting this concept. In addition to Region 10 mentioned in the report, Regions 6 and 7 will award ID/IQ contracts and other regions are considering using this contract type. We do not believe that EPA's current strategy of providing a menu approach is the same as developing a plan with milestones that would increase the use of different contract types and structures, such as ID/IQ contracts. When providing a response to the final report, EPA should provide a plan to increase the use of ID/IQ contracts. EPA agreed with recommendation 2-2 to conduct a lessons learned analysis of the new ID/IQ contracts and stated they would share the results. We believe EPA should go beyond just sharing the results of the analysis. If results of the lessons 12 ------- learned demonstrate that ID/IQ contracts are more beneficial, EPA should develop guidance that lays out the circumstances when certain types of contracts, such as ID/IQ, should be used. Any deviation from the preferred contract type should be justified. EPA did not agree that RACs are funded in excess of identified needs, but agreed to examine the extent to which regions are "forward" funding project needs. We conducted a detailed contract-by-contract analysis of the RAC funding and determined that regions had funds allocated to contracts in excess of identified needs and planned work. EPA agreed that there needs to be a revision or clarification of the recertification policy, especially as it applies to RACs that use task orders or other contract types. At the exit conference, EPA staff stated that regions will be exempt from having to return 75 percent of the deobligated funds to Headquarters if funds deobligated from a task order are obligated to a task order under the same RAC concurrently. 13 ------- Chapter 3 Procurement Decisions Not Adequately Documented EPA regions do not consistently document the rationale used to decide what procurement option to utilize for Superfund cleanup activities, as required by established policy. Further, EPA does not have a process to measure and disseminate information on USACE past performance in support of EPA. As a result, the Agency has limited assurance that sound procurement decisions are being made. OSWER Requirements Were Not Followed OSWER's May 19, 2003, policy, "Superfund Policy for Assigning Remedial Work to the US Army Corps of Engineers (USACE)," set forth factors to consider when selecting the appropriate procurement options (see box). Specifically it states, "As with all procurements, analysis should be conducted to select the appropriate contracting vehicles . . . All analyses should be documented in the site or contract file. The documentation should outline the reason behind the selection, especially any factors that were used to make the decision." In six regions, the 2003 policy was not being followed and inconsistent evaluation methods were being used. Four regions (2, 5, 6, and 9) had no documentation of factors considered to support decisions made to assign (or not assign) work to the USACE. The other two regions (1 and 7) documented some of the factors used to make their decision, but did not fully comply with the May 2003 policy. Region 1 utilizes a "Decision Matrix." The factors listed in the Region 1 Decision Matrix are similar to those required by the May 2003 policy. However, the decision matrix does not consider Remedy Characteristics and Experience/Regional Infrastructure. Region 7 utilizes an "option paper," which evaluates the different alternatives available for cleanup activity. The option paper does not address some of the factors listed in the May 2003 policy, such as conflict of interest, remedy characteristics, and unique site needs. Regions 2, 5, 6, and 9 advised they used informal discussions among the Project Officers, Remedial Project Managers, and May 2003 Policy Source Selection Factors Site Characteristics Remedy Characteristics Local/Public Interest Experience/Regional Infrastructure Capability/Capacity of Contractors Conflict of Interest Unique Site Needs 14 ------- Branch Chiefs when deciding with whom to contract. These informal discussions are not documented. Headquarters staff stated that the regions should be documenting their decisions in all cases. However, some regional staff stated they were unsure whether they had to document their decision in all cases or only when they use the US ACE. By not documenting the reasons behind the selection of a procurement option, the Agency has limited assurances that sound decisions are being made in accordance with applicable policy. USACE Past Performance is Not a Factor in EPA's Source Selection Considerations EPA does not have a formal process to measure and disseminate information on USACE past performance in support of EPA cleanup activities. As of February 2003, the EPA Superfund program had approximately $453 million in Interagency Agreements with the USACE to support Superfund cleanup activities. The May 2003 policy lists numerous source selection considerations; however, past performance of the USACE is not a factor. As a result, there is an increased risk that a USACE district office with performance problems in one EPA region could be awarded additional work in other regions. Regional staff oversee USACE activities through progress reports, meetings, and e-mails. However, no process currently exists for regional staff to systematically assess USACE performance and share it with EPA staff nationwide. EPA is starting to address these issues. EPA's August 2003 report, "Evaluation of the Performance of the Corps of Engineers in Support of EPA's Superfund Program," reported on how well the USACE was supporting Superfund programs in EPA regional offices. Some of the recommendations were that: New Interagency Agreements include criteria against which USACE performance will be measured and evaluated. USACE and EPA Headquarters implement annual oversight of their respective field programs. EPA and regional offices institute formal annual oversight programs of USACE District offices managing or overseeing Superfund projects and suggested criteria on which the USACE should be evaluated. OSWER developed an action plan to implement these recommendations and to conduct regional evaluations. However, milestone dates have not been implemented for all action items. The Agency has developed a plan to conduct regional evaluations of the USACE. After regional evaluations are completed, the Regional Liaison is required to summarize the results of the evaluations and forward them to Headquarters. However, the plan is unclear as to how regions would have access to other regions' evaluations. The Agency should develop a 15 ------- system or database in order to share the evaluations nationwide. The full implementation of the action plan should allow EPA to better monitor the costs, the quality of the work, and the timeliness of performance provided by the USACE. Recommendations We recommend that the Acting Assistant Administrator for OSWER: 3-1 Issue clarifying guidance on "Superfund Policy for Assigning Remedial Work to the US Army Corps of Engineers (USACE)" requiring that: a) past performance of USACE be considered in source selection, and b) the rationale for all source selection decisions be documented. 3-2 Develop and implement a plan to evaluate USACE performance and share this information nationwide. Agency Comments and OIG Evaluation EPA stated that as part of the source selection considerations the regions determine the best source from which to procure and manage construction services. However, Regional Reviews will look into how the performance of USACE was considered and documented, as well as ways of sharing performance issues at national meetings, such as the Senior Superfund Regional Managers Acquisition Council. We concur with EPA actions of developing feedback on USACE. However, EPA has not demonstrated how it intends to utilize this information beyond sharing the results at meetings. We are therefore recommending that EPA develop a system to disseminate this information nationwide to aid in the source selection decisions. EPA agreed with the recommendation that more documentation be required, and stated that the factors to be considered, as outlined in current policy, are guidelines rather than requirements. EPA further stated it will be reviewing how each region documents its decision as part of the planned Superfund Contracts Regional Review program. EPA plans to share performance issues on a national basis at national meetings, such as the Senior Superfund Regional Managers Acquisition Council. We agree with EPA actions to conduct regional reviews. EPA should not only share results of these reviews, but issue/reissue any guidance deemed necessary as a result. EPA agreed that sharing information about contracts procured by USACE and the process used will be a valuable management tool, but did not agree it needed a formal process to evaluate USACE's past performance. EPA stated it is testing a feedback report intended to be completed by both EPA and USACE project 16 ------- managers that will provide progress, quality, and status of ongoing work. This feedback, in addition to routine project communications, will highlight for managers potential problems that need to be resolved. EPA also stated they are developing a completion report to provide information to EPA and US ACE personnel on the overall satisfaction of US ACE's management of the project and the USACE contractor's performance, and this information will be available to all regions. We are not advocating the development of any additional plans beyond what EPA has started to develop. What we are recommending is that EPA make evaluation information more readily available to EPA staff nationwide. EPA stated that both the feedback and completion reports will be available to all regions. However, EPA did not provide us with its means of doing this. The Senior Superfund Regional Managers Acquisition Council meeting, which is held twice a year, is a good avenue to discuss the results of evaluations, but providing regions with access to these evaluations as needed would be more beneficial. 17 ------- Chapter 4 Past Performance Evaluations Not Documented Timely and Consistently The Agency has measures in place to assess contractor performance at the work assignment level, but evaluations at the contract level were not being documented timely and consistently as required because they were not given the necessary priority. There is no EPA individual held accountable for assuring past performance evaluations are documented in the National Institutes of Health Contractor Performance System (NIH CPS). Annual evaluations allow Contracting Officers to recognize contractors performing well, and to correct problems before they jeopardize contract completion. Contractors can gain or lose contracts based on past performance information, which in turn can mean millions of dollars in revenue. Therefore, not consistently documenting evaluations in a timely manner does not permit EPA and other Federal agencies to consider contractors' past performance and it could be detrimental to contractors who have preformed well. Past Performance Evaluations at Contract Level Are Not Documented Timely and Consistently in NIH CPS Environmental Protection Agency Acquisition Regulations 1509.170 and 1552.209-76 provide time-frames for completing contractor evaluations and require the Contracting Officer to complete the process within 90 business days after the end of each 12 months of contract performance period in the MH CPS. The NIH CPS collects and maintains contractor performance evaluations. It allows instant access to multiple agency evaluations in Federal departments and independent agencies. As of May 7, 2004, 34 interim evaluations for the RACs should have been in the NIH CPS. However, only 26 evaluations were in the system. Nineteen of the 26 evaluations were finalized, and available to the contracting community. However, many of the evaluations took a long time to be finalized. Eleven of the 19 (58 percent) completed evaluations took from 152 to 351 days. Seven of the 26 evaluations were started and "in progress," and had been in process from 128 to 858 days from the end of the evaluation period. 18 ------- There is inconsistent evaluation information in NIH CPS. For the 26 evaluations in the NIH CPS: Four had evaluation periods exceeding a year, ranging from 4 years to more than 8 years. Beginning periods for these evaluations were the contract award dates, 1995, 1996 (2), and 1998. One evaluation was for a 6-month period of performance instead of the required 12 months. Three evaluations for one contract were for the same performance period. Two evaluations were listed as final, even though the contracts do not end until 2006. There is no EPA individual responsible for assuring past performance evaluations are documented timely and consistently in the NIH CPS. Each region is responsible for regional contractor evaluation information. Additionally, there are various measurement points during the evaluation process, but no one is monitoring these measuring points to ensure timely completion. The May 2000 Office of Federal Procurement Policy, "Best Practices for Collecting and Using Current and Past Performance Information," states that contractors and industry associations emphasized the power of past performance as a tool for motivating contractors to make their best efforts, and raised concerns that many assessments are not being done, or are being done inconsistently. Interim evaluations allow Contracting Officers to recognize contractors performing well, and to correct problems before they jeopardize contract completion. Interim evaluations also provide current performance information on comparable contracts to source selection teams. Completing these assessments improves the quantity and quality of performance information available to aid in source selection, not just by EPA but other Federal agencies as well. Contractors can gain or lose contracts based on past performance information, which in turn can mean millions of dollars in revenue. Therefore, not consistently documenting evaluations in a timely manner is a disservice to EPA and other Federal agencies, as well as EPA's best contractors. By not adequately documenting evaluations, the Agency is not fully meeting its requirement of providing past performance evaluations to the contracting community. FAR 9.104-1 (c) requires a prospective contractor to have a satisfactory performance record to do business with the Government. Past performance can be used to help the Government decide whether a contractor is capable of performing and provides a good indicator of future performance. 19 ------- Lack of formal training may have resulted in inconsistent and inaccurate evaluation information. EPA has designated one individual to provide training on the NIH CPS. Most staff nationwide have not been trained due to lack of travel funds. OAM has developed a "tickler" system to remind Contracting Officers to complete evaluations. However, OAM needs to develop controls to ensure that Contracting Officers document evaluations timely and consistently in the NIH CPS. Recommendations We recommend that the Acting Assistant Administrator for OSWER, in coordination with the Acting Assistant Administrator for OARM and the regions: 4-1 Provide NIH CPS training to applicable regional and headquarters staff. 4-2 Develop a method for holding Contracting Officers accountable for documenting evaluations of contractor performance in the NIH CPS timely and consistently. Agency Comments and OIG Evaluation EPA agreed with the recommendations and noted that more training and accountability are required in the area of contractor performance evaluations. However, EPA did not give any detailed information on how staff will be trained , or how it plans to hold Contracting Officers accountable if evaluations are not documented. EPA will need to decide on a cost effective method of providing training so that data can be input timely and consistently. 20 ------- Chapter 5 Utilization of Remedial Action Contract Management Information System Could be Improved Contract managers have, or can obtain, the information needed to evaluate results and make decisions, but the information in the national automated database is not always readily available. The Remedial Action Contract Management Information System (RACMIS) is underutilized by regional staff and the system does not collect national data as originally intended. Most regional staff do not use RACMIS due to its limited reporting capability, and some regions have decided not to require contractor information through RACMIS due to its costs. As a result, EPA is expending approximately SI.5 million a year on a system that is not being fully utilized and may not be used for its intended purposes. The Administrator's Task Force Report of October 1991 required establishing a management information system to monitor resources and establish regional capacity for providing Independent Government Cost Estimates. RACMIS was created to meet this requirement, and the system was deployed in 1996 and 1997. RACMIS is currently utilized primarily by Headquarters to report on contract obligations, expenditure limits, expenditures, type of work performed, regional crossover work, and creation of Independent Government Cost Estimates. However, most regional staff use locally based spreadsheets and do not use RACMIS due to the limited retrieval and reporting capability available in the regions. Currently there is a lag time between the regions' requests for data and the generation of reports to meet those requests. Some regions believe the information can in fact be captured at a lower cost. Because of this, Regions 7 and 10 chose, and were allowed, to opt-out of reporting in RACMIS for the new RACs. According to Region 10 staff, "this was done because the program in Region 10 did not want to bear the cost of RACMIS when essentially the same information could be obtained at a lower cost." To address the problem of underutilization, the Agency plans to implement Business Objects, a report-writing tool that will allow the regions to readily access RACMIS to create reports. Headquarters provided Business Objects training to some regional staff in April and May 2004. Indications from staff attending the training is that RACMIS will now be utilized more in the regions. The OSRTI staff advised that Region 7 now wants to utilize RACMIS on its contracts and OAM staff stated that all the other contractors for the next round of RACs will be required to utilize RACMIS. Region 10, however, has not changed its plans. Without data from all the contractors, complete national data on RACs 21 ------- will not be available in RACMIS, compromising the ability to report timely national data. OSRTI staff also advised that they plan on performing an analysis before the end of calender year 2004 to assess whether RACMIS is truly needed, whether the cost of RACMIS is reasonable for the information being provided, or whether contractors' financial reporting requirements can be met at a lower cost. They advise that, currently, the cost of reporting through RACMIS is SI.5 million a year, less than 1 percent of total RAC costs. If the Agency decides that RACMIS is the system it wants to use, then reporting through RACMIS should be mandatory for all regions. Recommendations We recommend that the Acting Assistant Administrator for OSWER, in coordination with the Acting Assistant Administrator for OARM and the regions: 5-1 Complete the proposed cost benefit analysis to determine whether RACMIS should be retained or other more cost effective methods should be used to collect RAC financial data. 5-2 If EPA decides to continue using RACMIS, develop and implement a strategy to improve regional utilization of RACMIS (or its replacement system if so decided), and require that financial information be collected for all RACs. Agency Comments and OIG Evaluation EPA stated that it is conducting a cost benefit analysis of RACMIS and plans to discuss the results of its analysis at the Senior Superfund Regional Managers Acquisition Council meeting in December 2004. EPA's response addresses our first recommendation; however, EPA still needs to address how it will assure that the system (RACMIS or its replacement) will be utilized for all RACs. 22 ------- Appendix A Schedule of Response Action Contracts Reviewed No. Contractor Contract Number Maximum Contract Value Date of Award Period of Performance Base Plus Option Years Re- gion 1 Tetra Tech NU 68-W6-0045 $423,248,994 08/29/1996 08/29/1996-08/28/2006 1 2 Metcalf & Eddy 68-W6-0042 $459,536,934 09/30/1996 09/30/1996-09/29/2006 1 3 Foster Wheeler Environmental Corp. 68-W-98-214 $243,770,460 07/06/1998 070/6/1998-07/05/2008 2 4 CDM Federal Programs Corp. 68-W-98-210 $217,731,330 07/06/1998 07/06/1998-07/05/2008 2 5 Tetra Tech NUS 68-S6-3003 $168,720,501 09/30/1996 09/30/1996-09/29/2006 3 6 Tetra Tech/Black & Veatch 68-S7-3002 $168,763,187 06/12/1997 06/12/1997-06/11/2007 3 7 CDM Federal Programs Corp. 68-S7-3003 $169,329,900 09/30/1997 09/30/1997-09/29/2007 3 8 Black & Veatch Special Projects Corp. 68-W-99-043 $346,336,991 06/17/1999 06/17/1999-06/16/2009 4 9 CH2M Hill 68-W6-0025 $288,327,444 09/30/1996 09/30/1996-09/26/2006 5 10 Weston Solutions, Inc. 68-W7-0026 $290,046,890 05/01/1997 05/01/1997-04/30/2007 5 11 CH2M Hill 68-W6-0036 $290,728,258 07/08/1996 07/08/1996-07/07/2006 6 12 Tetra Tech Environmental Mgmt. 68-W6-0037 $253,261,985 09/13/1996 09/13/1996-09/12/2006 6 13 Black & Veatch 68-W5-0004 $302,858,596 06/15/1995 06/15/1995-06/14/2005 7 14 CDM Federal Programs 68-W5-0022 $237,629,770 020/5/1997 09/29/1995-09/28/2005 8 15 CH2M Hill 68-W-98-225 $303,984,009 09/29/1998 10/01/1998-09/30/2008 9 16 URS Greiner 68-W-98-228 $130,466,269 09/30/1998 10/01/1998-09/30/2008 10 Maximum RACs Value: $4,294,741,518 23 ------- Appendix B ;SS) NW" UNIT! 0 STAIf.S I NVIHONMt NIAL PROTECTION AOENCV WASMNQTON. O-C. 20460 MEMORANDUM OCT 29 m qfucsc#1 SUBJECT Draft Audit Report. Res pun*; Action C?onu»cts Structure and Administration Need itinre'wefHCnt Project No. 2003-00814 FROM: Thomas P Daag^i^^W' . Acting Assistant Administrator TO Robert Mitchell, Director for Contract Audita Qffci of Inspector Gswal Thank you for the opportunity to review the draft audit report: Response Action Contracts Structure and Administration Needs Improvement. The comments below reflect consolidation of the Office of Acquisition Management's (OAM) and the Office of Solid Waste and Emergency Response's (OSWER) review of the report. First, we address each of the recommendations and provide information about planned or initiated actions related to those recommendations. Second, we provide a few other comments to ensure that the final audit report contains accurate and current information to substantiate the proposed recommendations. CHAPTER 2 - RESPONSE ACTION CONTRACTS NEED TO BE RESTRUCTURED Recommendation 2-1 "Develop a plan for RACII which: a) increases the use oflD/IQ, task order, site specific contracts, and b) allows for separate contracting for design, construction, and other remedial services. " Recommendation 2-2 "Conduct a lessons learned analysis of the new ID/IQ contracts as soon as sufficient data is available." EPA RESPONSE: EPA's current strategy (Contracts 2000 - Design/Construction Contracts Decision memorandum dated/ September 15, 2000) already allows for separate contracts for design, construction, and other remedial services, and Regions 7 and 10 are piloting this concept. We also 24 ------- have developed plans to increase the use of ID/IQ contracts. In addition to the Region 10 contract discussed in the audit report, Regions 6 and 7 will award ID/IQ contracts and other regions are considering using this contract type. The audit report recommendation should indicate this progress. OARM and OSRTI will be conducting a review of the Region 10 Architect and Engineering Services (AES) contracts this fall and will share lessons learned and best practices with the other regions. Recommendation 2-3 "Stop funding the Response Action Contracts (RACs) in excess of identified needs." EPA RESPONSE: As noted in our comment related to Page 9 below, we do not agree that EPA is funding RACs in excess of identified needs. We will examine, however, the extent to which Regions are forward funding project needs. Recommendation 2-4 "Review and Revise the recertification policy so as to remove it as an impediment to utilization of better contract types. EPA RESPONSE: We agree that there needs to be a revision or clarification of this policy, especially as it applies to RACs that use task orders or other contract types, and we will develop such guidelines for the regions. OTHER EPA COMMENTS ON CHAPTER 2: Page 8, fifth full paragraph: "While the menu approach seems to balance national consistency with regional flexibility, there is limited assurance that it will lead in all cases to increased competition, increased small, minority and women owned business participation, or use of performance based, multiple award, and fixed price contracts. There is no guarantee that the regions will select other types of contracts and they may stay with the existing contract type, CPAF." While we agree that the menu approach does not assure these goals, I want to highlight the actions we are taking to achieve them. Regional implementation plans approved September 28, 2004, show that all regions will be awarding at least two contracts and have committed to increase participation of disadvantaged and small business. At least five regions (Regions 5, 6, 7, 8, and 10) have committed to awarding 50 percent of their requirement to small business. In addition, the full-service RACs will require fixed priced subcontracts for construction to be performance based. While the regions have not been directed to award specific contract types, three regions will award Indefinite Delivery/Indefinite Quantity (ID/IQ) contracts. Page 9, third full paragraph, fourth sentence: "The remaining $8.6 million appears to be funding in excess of the regions' identified needs and "at risk" in light of the court decision. " 25 ------- While we agree that the Agency needs to actively monitor RAC funding activity, the audit report does not clearly establish that $8.6 million is "at risk". It is not an accurate assumption to use composite amounts of all RACs for this purpose. In order to assess if funds are at risk in light of the court decision, the Level of Effort (LOE) ordered and obligated has to be analyzed on a contract by contract basis. Nonetheless, we agree that LOE contracts could pose a vulnerability if not closely monitored and appropriately modified with regard to funds obligated and available level of effort. CHAPTER 3 - PROCUREMENT DECISIONS NOT ADEQUATELY DOCUMENTED Recommendation 3-1 "Issue clarifying guidance on the Superfund "Policy for assigning remedial work to the US ACE: requiring that (a) past performance of US ACE be considered in source selection, (b) the rationale for all source selection decisions be documented." EPA RESPONSE: EPA recognizes the need to evaluate how each Region is documenting its rationale of which procurement option to utilize for Superfund construction services. It is important to clarify that the factors to be considered, as outlined in current policy, are guidelines rather than requirements. Thus, Regions do not have to address each factor in their documentation, only those that apply. EPA will be reviewing how each Region documents their decision as part of the Superfund Contracts Regional Review program. We agree with the recommendation that more documentation is required. As part of source selection considerations, EPA does consider who is the best source to procure and manage construction services. Under the source selection factor, experienced staff and managers evaluate which organization is most appropriate to place and manage a particular requirement. Experience, capabilities, performance, staffing knowledge and availability, and capabilities, all play into evaluations made under this factor. As part of our review of source selection documentation during Regional Reviews, we will look into how the performance of the US ACE is explained under this factor. We will also look into ways of sharing performance issues on a national basis at national meetings, such as the Senior Superfund Regional Managers Acquisition Council (SRMAC). Recommendation 3-2 Develop and implement a plan to share EPA evaluations of the USACE nationwide. EPA RESPONSE: While we do not agree that we need a formal process to evaluate the USACE's past performance, we do agree that sharing information about contracts procured by USACE and the process used will be a valuable management tool. The USACE partners with EPA in providing construction contracting, placement and oversight. Performance of contractors procured by USACE are evaluated using standard Federal contract management procedures. Poor 26 ------- performance by a contractor is not necessarily indicative of poor performance of USACE personnel managing the contract. We believe that what is beneficial to the contracting process is close coordination between EPA and the USACE project manager overseeing the project. To this end, to enhance coordination we are testing a feedback report intended to be completed by both the EPA and USACE project managers that will provide progress, quality and status of ongoing work. This feedback, in addition to routine project communications, will highlight for managers potential problems that need to be resolved. We are also developing a completion report to provide information to EPA and USACE personnel on the overall satisfaction of the USACE's management of the project and the USACE contractor's performance. This information will be available to all Regions. OTHER EPA COMMENT ON CHAPTER 3: Page 14, third full paragraph: "EPA information pertaining to USACE past performance is anecdotal" is contradicted by the next sentence which states that "Regional staff oversees USACE activities through progress reports, meetings, and e-mails. " Progress reports and meetings are not anecdotal, thus this term should be removed from the first sentence. CHAPTER 4 - PAST PERFORMANCE EVALUATIONS NOT DOCUMENTED TIMELY AND CONSISTENTLY Recommendation 4-1 "Develop a method for holding Contracting Officers accountable for conducting timely and consistent evaluations of contractor performance in the NIH CPS. " Recommendation 4-2 "Provide NIH CPS training to applicable regional and headquarters staff." EPA RESPONSE We agree with recommendations 4.1 and 4.2 that more accountability and training are required in the area of contractor performance evaluations. In addition to the corrective actions mentioned in the report, the following additional actions are planned or initiated: a. OAM will conduct reviews every six months to ensure that the performance evaluations of the RAC contracts are completed on time. b. OAM designates an annual National Institute of Health (NIH) Stand-down Day. Contracting Officers are directed to prioritize the completion of the contractor performance evaluations on this day. c. Training on the NIH system has been made available to the regions. 27 ------- CHAPTER 5 - UTILIZATION OF REMEDIAL ACTION CONTRACT MANAGEMENT INFORMATION SYSTEM COULD BE IMPROVED Recommendation 5-1 "Complete the proposed cost benefit analysis to determine whether RACMIS should be retained or other more cost effective methods should be used to collect RACfinancial data." EPA RESPONSE: We are conducting the cost benefit analysis called for in this recommendation. We plan to present the results to the SRMAC meeting in December 2004. We will share the presentation with you. Recommendation 5.2 If EPA decides to continue using RACMIS, develop and implement a strategy to improve regional utilization of RACMIS (or its replacement system if so decided), and require the financial information be collected for all RACs. EPA RESPONSE: We agree that if we continue to use RACMIS we will require that the information be collected for all RACs. OTHER EPA COMMENT ON CHAPTER 5: Page 21, 6th paragraph, first sentence: "OSRTI staff advised that Region 10 now wants to utilize RACMIS on its contracts " is incorrect. Whereas there may have been discussion among staff about RACMIS, OSWER has not received official notification by Region 10 that they want to utilize RACMIS. Second sentence: "Region 7, however, has not changed its plans. " Region 7 has indicated to OAM that they will utilize RACMIS in the future. Thank you for the opportunity to review the draft report. If you have any questions regarding our comments please contact Johnsie Webster at 202-566-1912 or Barbara McDonough at (703) 603-9042. cc: Judy Davis Mike Cook Johnsie Webster Susan Janowiak Paige Peck Yvette Garner 28 ------- Appendix C Distribution EPA Headquarters Acting Assistant Administrator, Office of Solid Waste and Emergency Response (510IT) Director, Office of Superfund Remediation and Technology Innovation (5201G) Acting Assistant Administrator, Office of Administration and Resources Management (310 IT) Director, Office of Acquisition Management (3801R) Agency Followup Official (the CFO) (271 OA) Agency Followup Coordinator (2724A) Audit Followup Coordinator, Office of Administrator (1104) Audit Followup Coordinator, Office of Solid Waste and Emergency Response (5103) Audit Followup Coordinator, Office Administration and Resources Management (3102A) Associate Administrator for Congressional and Intergovernmental Relations (1301 A) Associate Administrator for Public Affairs (1101 A) Inspector General (2410) EPA Regions Regional Administrator (1-10) Regional Audit Followup Coordinators (1-10) 29 ------- |