$ < 73 \ Ml (T b 2 ui (J T J U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Catalyst for Improving the Environment Audit Report EPA Can Improve Its Preparation and Use of Independent Government Cost Estimates for Superfund Contracts Report No. 10-P-0065 February 16, 2010 ------- Report Contributors: Nancy Dao Doug LaTessa Kevin Lawrence Michael Petscavage Abbreviations CMM Contracts Management Manual EPA U.S. Environmental Protection Agency ERRS Emergency and Rapid Response Services ESAT Environmental Services Assistance Team FAR Federal Acquisition Regulation GAO Government Accountability Office IGCE Independent Government Cost Estimate OAM Office of Acquisitions Management OARM Office of Administration and Resources Management OIG Office of Inspector General OSRTI Office of Superfund Remediation and Technology Innovation OSWER Office of Solid Waste and Emergency Response QAP Quality Assessment Plan RAC Remedial Action Contract SOW Statement of Work START Superfund Technical Assessment and Response Team TDF Technical Direction Form ------- $ < 73 \ V PRO^4-0 o LU a U.S. Environmental Protection Agency Office of Inspector General At a Glance 10-P-0065 February 16, 2010 Catalyst for Improving the Environment Why We Did This Review We conducted this audit to determine whether the U.S. Environmental Protection Agency (EPA) is preparing quality Independent Government Cost Estimates (IGCEs) that can be used to evaluate whether EPA receives the best value for dollars expended on Superfund contracts. Background An IGCE is a detailed estimate of what a reasonable person should pay to obtain the best value for a product or service. EPA's training guide for IGCEs states that, "they are an integral part of any effective acquisition program." IGCEs are used to plan costs for new contracts and new work under existing contracts. Once program personnel prepare the IGCE, the IGCE should then be compared to the contractor's proposal to help determine price reasonableness. For further information, contact our Office of Congressional, Public Affairs and Management at (202) 566-2391. To view the full report, click on the following link: www.epa.qov/oiq/reports/2010/ 20100216-10-P-0065.pdf EPA Can Improve Its Preparation and Use of Independent Government Cost Estimates for Superfund Contracts What We Found EPA can improve its Superfund IGCEs and the corresponding cost estimating process. In 30 of the 42 cases we reviewed, EPA did not sufficiently document information in its Superfund IGCEs. Additionally, in 9 of the 42 cases, EPA did not update the IGCEs when significant changes occurred. In 8 of the 42 cases, EPA program staff accepted the contractor's estimate without evaluating why it differed from the IGCE. Finally, in some cases EPA did not prepare an IGCE for actions with a potential value in excess of $100,000, the Federal Acquisition Regulation threshold for simplified acquisitions. These actions are contrary to the Government Accountability Office Cost Estimating and Assessment Guide and the EPA Contracts Management Manual. They occurred because there is an overall lack of emphasis by EPA management on the preparation and use of IGCEs. EPA limits its ability to negotiate a fair and reasonable price when it does not have a well-supported IGCE. What We Recommend We recommend that EPA place greater emphasis on IGCEs through training and tools for creating IGCEs. EPA should prepare IGCEs for all contract actions expected to exceed the simplified acquisition threshold and discontinue the practice of relying only on the contractors' estimates. EPA agreed with all of our recommendations and provided a corrective action plan for most of the recommendations. EPA indicated it will provide its entire corrective action plan as part if its response to the final report. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 OFFICE OF INSPECTOR GENERAL February 16, 2010 MEMORANDUM SUBJECT: EPA Can Improve Its Preparation and Use of Independent Government Cost Estimates for Superfund Contracts Report No. 10-P-0065 FROM: Melissa M. Heist Assistant Inspector General for Audit TO: Craig E. Hooks Assistant Administrator Office of Administration and Resources Management Mathy Stanislaus Assistant Administrator Office of Solid Waste and Emergency Response Karl Brooks Regional Administrator, Region 7 This is our report on the U.S. Environmental Protection Agency's (EPA's) preparation and use of Independent Government Cost Estimates (IGCEs). This audit report contains findings that describe the problems the EPA Office of Inspector General (OIG) has identified and corrective actions the OIG recommends. This report represents the position of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established audit resolution procedures. The estimated cost of this report - calculated by multiplying the project's staff days by the applicable daily full cost billing rates in effect at the time - is $449,120. Action Required In accordance with EPA Manual 2750, you are required to provide a written response to this report within 90 calendar days. You should include a corrective actions plan for agreed-upon actions, including milestone dates. We have no objections to the further release of this report to the public. This report will be available at http://www.epa.gov/oig. ------- If you or your staff have any questions, please contact me at 202-566-0899 or heist.melissa@epa.gov; or Janet Kasper, Product Line Director, at 312-886-3059 or kasper.ianet@epa.gov. ------- EPA Can Improve Its Preparation and Use of Independent Government Cost Estimates for Superfund Contracts 10-P-0065 Table of C Chapters 1 Introduction 1 Purpose 1 Background 1 Noteworthy Achievements 2 Scope and Methodology 2 2 EPA Superfund Can Improve Its Independent Government Cost Estimates .. 5 Superfund Staff Need to Better Document Rationale, Assumptions, and Changes 5 Superfund Staff Should Not Rely on Contractor Proposed Amounts 6 Superfund Staff Need to Prepare IGCEs for All Contract Actions Over the Simplified Acquisition Threshold 7 EPA Should Place Greater Emphasis on IGCEs 8 Conclusion 9 Recommendations 10 Agency Comments and OIG Evaluation 10 3 EPA Region 7 Improved Cost Management for Its ESAT Contract 12 Status of Recommendations and Potential Monetary Benefits 13 Appendices A List of Superfund Contracts Reviewed 14 B Agency Response 16 C Distribution 19 ------- 10-P-0065 Chapter 1 Introduction Purpose The U.S. Environmental Protection Agency's (EPA's) Office of Inspector General (OIG) conducted this audit to determine whether the Agency is preparing quality Independent Government Cost Estimates (IGCEs) that can be used to evaluate whether it receives the best value for dollars expended on Superfund contracts. Background An IGCE is a detailed estimate of what a reasonable person should pay to obtain the best value for a product or service. It should include priorities and assumptions that exist at the time the estimate is made. In a training guide on IGCEs, EPA's Office of Acquisition Management (OAM) states, "They are an integral part of any effective acquisition program." OAM further states, "EPA's goal for cost estimates is to achieve predictions that are as accurate as possible using the information available and the best estimating technique for a given situation. Both involve the use of any historical data available." Quality cost estimates help EPA ensure it obtains fair and reasonable prices for desired goods and services. EPA's Strategic Plan states, "EPA is committed to being a good steward of our environment and a good steward of America's tax dollars. To provide the public with the environmental results it expects and deserves, we must operate as efficiently and effectively as possible...." EPA uses IGCEs to plan costs for new contracts as well as modifications, new work assignments, and task or delivery orders under existing contracts. EPA's program offices are responsible for preparing IGCEs. In preparing the IGCE, the program office uses various methods to estimate the resources necessary to perform the activities detailed in the Statement of Work (SOW). The program office compares the IGCE to the contractor's proposal to help determine cost reasonableness. According to the Federal Acquisition Regulation (FAR), the contracting officer's responsibility is to provide necessary support for contractual issues, including assuring the reasonableness of costs. EPA's Office of Solid Waste and Emergency Response (OSWER) oversees the Superfund program. Superfund is the name given to the environmental program established to address abandoned hazardous waste sites. It is also the name of the fund established by the Comprehensive Environmental Response, Compensation, and Liability Act of 1980, as amended. This law was enacted in the wake of the discovery of toxic waste dumps. It allows the EPA to clean up such sites and to 1 ------- 10-P-0065 compel responsible parties to perform clean-ups or reimburse the government for EPA-led clean-ups. Noteworthy Achievements We identified noteworthy achievements in the form of a student guide and tools available to assist EPA personnel in preparing IGCEs. However, these resources need to be updated and re-emphasized. EPA made IGCE related resources available following previous OIG and Government Accountability Office (GAO) reports that cited issues regarding EPA's preparation and use of IGCEs. In 1998, OAM developed a guide for preparing and using IGCEs. The guide details the importance of IGCEs and the important components that should be contained in IGCEs. The guide also specifies the potential uses of an IGCE. The Office of Superfund Remediation and Technology Innovation (OSRTI) developed a cost estimating toolbox. The toolbox is a database resource that provides comprehensive information for staff to use when preparing IGCEs for Remedial Action Contracts (RACs). It defines IGCEs and the associated cost elements and provides the purpose for developing IGCEs. Spreadsheet templates are provided for several phases of RAC contracts and can serve as a basis for staff to prepare their own IGCEs. The toolbox emphasizes the importance of documenting assumptions when preparing IGCEs. It also provides model SOWs, as well as IGCE checklists. Scope and Methodology GAO and the OIG have historically raised concerns regarding EPA's preparation and use of IGCEs. In a report issued in 2007, the OIG discussed problems relating to EPA's use and management of IGCEs. To address these concerns, we performed this audit from June 2008 to October 2009 in accordance with generally accepted government auditing standards issued by the Comptroller General of the United States. These standards require that we plan and perform the audit to obtain sufficient and appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The scope of this report deals only with IGCEs for EPA Superfund contracts. We reviewed the active contracts listing to determine which to include in our sample. We judgmentally selected contracts based on factors such as contract type, region or area where the contract is located, dollar value, period of performance, etc. In total, we selected 17 Superfund contracts from Headquarters and Regions 2, 4, 5, and 7. For each contract we reviewed, we judgmentally selected multiple work assignments, task orders, and/or technical direction documents for further review. We reviewed the corresponding files for information relating to cost estimates. 2 ------- 10-P-0065 Appendix A contains a complete list of the Superfund contracts we reviewed as well as the work assignments, task orders, and/or technical direction documents reviewed from each. We conducted site visits to each location within our sample. During the site visits, we reviewed the overall contract, SOW, and documents from the preaward file relating to cost estimating for each contract in our sample. We reviewed documents ordering work (i.e., work assignments, task orders, and technical direction documents) from each contract for information relating to cost estimating. We interviewed contracting officers and project officers associated with the contracts in our sample. In several instances, we also interviewed personnel at the level where work is ordered to gain a better understanding of the cost estimating and negotiation processes. We reviewed IGCEs to determine their overall quality. To measure the quality of IGCEs, we developed a list of the following five criteria that were taken from EPA's Guide for Preparing Independent Government Cost Estimates and an exposure draft of GAO's Cost Assessment Guide, which became final in March 2009. • The task is clearly identified: Estimator must be provided with a clear description of the task, including ground rules, assumptions, and technical and performance characteristics relating to the task. The estimate's constraints and conditions must be clearly identified to ensure the preparation of a well-documented estimate. A well-written SOW is integral to the development of an IGCE. • Relevant data is identified and used: To the extent possible, multiple sources of data should be used. The estimator(s) should determine the availability of historical data and use that which relates to the applicable task. • A defined/structured estimating process is used: The estimator(s) should determine and document the process for estimating prior to starting the process and document why the method was selected. The estimating process should be accomplished at the task level through the use of the work breakdown structure, created using either a bottom-up or top-down approach. • The cost estimate is revised as program changes occur because large changes that affect costs can significantly influence program decisions: Cost estimates need to be fluent and refined as work and planning progress. A fluent cost estimating process provides an allowance for uncertainties to be addressed as they become known. 3 ------- 10-P-0065 • The assumptions for the IGCE are documented, and the IGCE is complete: The rationale for the IGCE should be documented and the IGCE should contain all required tasks. Related Audit Coverage While conducting this audit, we determined that EPA's Acquisition Regulation contains language that requires EPA to provide total labor hour estimates to the contractor within work assignments. If this information is provided to the contractor prior to receiving the proposal, EPA jeopardizes the assurance of cost reasonableness and overall effectiveness of the estimating process. To address this issue, we issued a separate report, "EPA Should Stop Providing Estimates of Total Labor Hours to Contractors, " on September 9, 2009 (OIG Report No. 09-P-0229). Internal Control Structure In planning and performing our audit, we reviewed management controls related to our objectives. Specifically, we examined EPA's Contracts Management Manual (CMM), EPA's Acquisition Regulation, the OAM guide for developing IGCEs, an Office of Federal Procurement Policy letter, and several OSWER Directives. We reviewed Quality Assessment Plans (QAP) from each program operating division. We also looked at OAM's review of the QAPs. We reviewed documents EPA completed in compliance with the Federal Managers' Financial Integrity Act. This included a review of numerous documents within EPA's Office of Administration and Resources Management's (OARM) Fiscal Year 2008 Federal Managers' Financial Integrity Act Assurance Letter. EPA did not report any material or Agency weaknesses related to its preparation and use of IGCEs. 4 ------- 10-P-0065 Chapter 2 EPA Superfund Can Improve Its Independent Government Cost Estimates EPA can improve Superfund IGCEs and the cost estimating process. Superfund program staff are not sufficiently documenting rationale, assumptions, and changes relating to its IGCEs. Program staff sometimes rely on the contractor's cost estimates without evaluating why it differs from the IGCE. In some cases, EPA program staff do not prepare an IGCE for contract actions in excess of the simplified acquisition threshold. GAO and OAM guides address these issues by indicating that all aspects of cost estimating should be documented and IGCEs should be updated when significant changes occur. Further, the CMM requires project officers to submit IGCEs for all contract actions with a potential value that will exceed the simplified acquisition threshold of $100,000. A primary reason Superfund IGCEs need improvement is because there is an overall lack of emphasis by management on the preparation and use of IGCEs. The guide and other resources related to IGCEs are outdated and employees responsible for preparing and reviewing IGCEs are undertrained. As a result, the value of the IGCE as a tool for ensuring fair and reasonable contract prices is diminished. Superfund Staff Need to Better Document Rationale, Assumptions, and Changes Superfund IGCEs lacked documentation for the basis of the estimated costs for 30 of the 42 contract actions we reviewed. OAM's IGCE guide specifies that the reasons costs are included within IGCEs should be documented as well as any assumptions made that may impact costs. Documenting the rationale and assumptions associated with cost estimates will provide sound support to use during negotiations with the contractor. GAO's Cost Estimating and Assessment Guide specifies that all aspects of cost estimates should be documented. Documentation supporting IGCEs should include information that will allow a cost analyst unfamiliar with the program to determine how the estimate was developed. The IGCEs were missing supporting documentation such as the reason for proposed labor hours, the labor mix, travel costs, etc. For example, one work assignment under a RAC contract was broken down into nine different tasks. The IGCE contained level-of-effort estimates by task and was developed using a template that factors in historical rates such as average labor category rates, and historical indirect rates. However, Superfund program staff did not document assumptions for specific level-of-effort estimates for the number of hours or other direct cost amounts included in the IGCE. As another example, a task order under an Emergency and Rapid Response Services (ERRS) contract contained a SOW 5 ------- 10-P-0065 that clearly described 11 different tasks to be carried out. Even though the SOW clearly described the tasks, the IGCE did not explicitly document the assumptions, rationale, or historical basis (if any) for the number of hours estimated. GAO recommends that to the extent possible, multiple sources of data should be used when preparing an IGCE. GAO also recommends that cost estimators determine the availability of historical data and use that which relates to the applicable task. Without adequate documentation of the rationale, assumptions, or historical data to support cost estimates, the IGCE does not provide the information needed to evaluate the contractor's proposed prices. For 9 of 42 contract actions, Superfund program staff did not update IGCEs to reflect significant changes to the scope of work. GAO's cost estimating guide states that estimates should be updated to reflect changes, especially large changes, since they can significantly impact program decisions. However, nine IGCEs were not updated when significant changes occurred to the corresponding task order/work assignment. For example, one work assignment under a RAC 2 contract underwent changes resulting in a substantial decrease in the cost estimate, but the IGCE was not updated. The initial IGCE, completed in June 2006, estimated costs would range from $21 million to $31.5 million. After a third contract revision in March of 2008, the contractor's estimate was approved for $15.8 million. For each revision, the contractor submitted a revised workplan. However, program staff did not revise the IGCE during a 2Vi year span of the work assignment. The only revised IGCE was completed in October 2008, one week prior to the OIG site visit. If IGCEs are not updated when significant changes occur, Superfund program staff may not have the necessary information to make important program decisions, such as distribution of resources and negotiation of contract price. Superfund Staff Should Not Rely on Contractor Proposed Amounts When there was a difference between the IGCE and contractor's proposed price, Superfund program staff accepted the contractor's price without additional analysis in 8 of 42 (19 percent) cases reviewed. The IGCE is a tool program staff use when reviewing the contractor's proposed price. Program staff should compare the hours and rates contained in the proposal to those estimated within the IGCE. Such an analysis would allow program staff to determine whether proposals overstate or understate work, or whether prices are too high or low, and determine the fair and reasonable price for labor hours. Superfund program staff are not always using the IGCE to conduct this analysis. In eight cases, the final agreed-to price was the same as the contractor's proposal and there was no evidence that program staff examined the reasons for the difference between the IGCE and proposed price (see Table 2-1 below). 6 ------- 10-P-0065 Table 2-1: Examples where Superfund relied on contractor estimates Between Contractor H Approved H Contractor Example J IGCE J Estimate J Amount J Estimate 1 $12,935 $64,988 $64,988 $52,053 2 $71,383 $72,435 $72,435 $1,052 3 $27,862,037 $14,845,475 $14,845,475 $-13,016,562 4 $2,193,769 $2,332,238 $2,332,238 $138,469 5 $109,789 $119,316 $119,316 $9,527 6 $65,046 $48,151 $48,151 $-16,895 7 $110,452 $117,335 $117,335 $6,883 8 $111,948 $106,531 $106,531 $-5,417 Source: OIG analysis of EPA data. Program staff sometimes accept the contractor's proposal if it is within a certain percentage of the IGCE. However, the percentage used was not consistent and varied among those interviewed. For example, the program staff responsible for items 7 and 8 in Table 2-1 stated that they generally accept any proposed costs within 10 percent of the IGCE without further analysis. In the case of example 6, program staff stated that they generally accept any proposal within 20 percent of the IGCE without additional analysis. In this case, the program staff accepted the contractor's proposal without additional analysis even though the proposal was 26 percent less than the IGCE. Even though the amount proposed was less, Superfund program staff cannot be reasonably assured that some of the individual amounts within the proposal were not overstated. The costs may be significantly understated and relying on these amounts could impact the program's ability to reasonably plan future work. By not analyzing why there was a difference in the price, or documenting the analysis of reasons for the difference, Superfund program staff diminished the value of the IGCE in determining the reasonableness of the final price. Superfund Staff Need to Prepare IGCEs for All Contract Actions Over the Simplified Acquisition Threshold Superfund program staff are not always producing IGCEs for work ordered with estimated costs expected to exceed the simplified acquisition threshold of $100,000. The CMM states that, "POs [project officers] shall submit IGCEs for all contract actions with a potential value in excess of the FAR threshold for simplified acquisitions." The CMM further states that, "COs [contracting officers] may also require IGCEs for actions below the FAR threshold for simplified acquisitions" if deemed necessary and appropriate. However, for three contracts in our sample, POs did not prepare IGCEs as required. • The PO for the Region 5 Superfund Technical Assessment and Response Team (START) contract did not prepare IGCEs at the task order or technical direction level. This contract establishes large task orders that generally describe the work to be done in broad terms. These task orders act as a sort of overall budget for the task. During contract performance, 7 ------- 10-P-0065 work is ordered under these task orders using technical direction documents. These technical direction documents specifically describe the work to be done (location, scope, etc.). The project officer indicated that she does not prepare IGCEs for the START contract because she does not believe they are useful. • Region 7 does not prepare IGCEs at the technical direction level for its Environmental Services Assistance Team (ESAT) contract. The ESAT contract uses 11 task orders that describe the work to be performed in broad terms. Under these task orders, work is later specifically defined and ordered using technical direct documents. However, Region 7 only prepares IGCEs at the broader task order level. For example, one of the task orders broadly directs work to be performed for Superfund program support. Later, specific descriptions of work to be performed are presented in a technical direction document, such as "Analyze 1 PT water sample for Selenium, by Region 7 approval methods. Process the data and prepare final data reports." • Region 4 did not prepare an IGCE for one task order totaling almost $1.3 million under its ERRS contract. The task order was funded through the Coast Guard's National Pollution Fund (but managed by EPA) and no IGCE was prepared. EPA staff informed us that the on-site coordinator is warranted up to $50,000 to respond to an oil spill emergency. Beyond that, approval must be given by the Coast Guard case officer. The initial award for this task order was for $150,000 and no IGCE was done because it was an emergency situation. However, the task order was amended three times, bringing the total approved cost ceiling to almost $1.3 million, and no IGCE was ever prepared. While we recognize that the initial emergency should be responded to without delay, we believe Superfund program staff should prepare an IGCE once the situation has stabilized and before it awards further funding. EPA Should Place Greater Emphasis on IGCEs A primary reason Superfund IGCEs need improvement is because management does not adequately emphasize the preparation and use of IGCEs. The OAM IGCE guide has not been updated since 1998. Additionally, training on IGCEs has been minimal and program office and regional reviews of the acquisition process do not directly address IGCEs. The lack of emphasis has resulted in the conditions identified in this audit and diminished the value of the IGCE as a tool for ensuring that contractor prices are reasonable. OAM and the Superfund program office developed tools and guidance relating to IGCEs more than 10 years ago. The CMM refers program staff to the student guide for guidance on preparing IGCEs. The guide has not been updated in the last 10 years, and does not include some current practices, such as using a 8 ------- 10-P-0065 technical direction document to order work. Also, in March 2009, GAO completed a cost estimating guide. EPA's guide needs to reflect the information contained in GAO's cost estimating guide. Superfund program staff involved in the cost estimating process informed us that training relating to IGCEs has been minimal within the last 10 years. EPA does not have a specific training course for IGCEs. Project Officers receive some IGCE training as part of their Contracting Officer Representative training. However, IGCEs make up only a few pages of the training text. Additionally, some employees were not aware of the guidance and tools available to assist in preparing IGCEs. For example, OSRTI developed a cost estimating toolbox. It provides comprehensive information for staff to use when preparing IGCEs for RAC contracts. Six of the nine RAC project officers/work assignment managers interviewed were not aware of the toolbox. This toolbox is of little use for those employees who are not aware it exists. EPA also needs to better emphasize IGCEs in its QAPs. QAPs are management plans developed by EPA program offices and regions to help ensure an overall effective acquisition process. The QAPs for EPA program offices and regions did not address the preparation and/or use of IGCEs. Conclusion The IGCE is a tool that the Superfund program can use to evaluate the reasonableness of the contractor's proposed price. A quality IGCE includes documentation of assumptions and reflects updates based on changed conditions. When Superfund prepares IGCEs, the assumptions and rationale are not always documented and updated. Additionally, Superfund does not always document the reasons for accepting the contractor's price when it differs from the IGCE. Finally, IGCEs are not used for all contract actions. Superfund program staff should be preparing IGCEs and comparing the estimated hours and rates to those contained within the contractor's proposal. This comparison will enhance the program's ability to determine whether proposals overstate or understate work, or whether prices are too high or low. Without a fully supported IGCE, Superfund program staff cannot conduct such an analysis. As a result, Superfund is diminishing the IGCE's value as a tool to ensure fair and reasonable contract prices. 9 ------- 10-P-0065 Recommendations We recommend that the Assistant Administrator for Administration and Resources Management: 2-1 Require OAM to update its IGCE guide to address: • Technical direction documents that exceed the simplified acquisition threshold. • Emergency acquisitions. • The GAO Cost Estimating and Assessment Guide. 2-2 After the OAM IGCE guide is updated, require EPA regions and program operating divisions to conduct a review to verify compliance with the updated guide. We recommend that the Assistant Administrator for Administration and Resources Management and the Assistant Administrator for Solid Waste and Emergency Response: 2-3 Eliminate the practice of accepting contractor estimates that differ from IGCEs without examining the reason for the difference. We recommend that the Assistant Administrator for Solid Waste and Emergency Response: 2-4 Require the Superfund program to update, distribute, and maintain its supplemental tools and guidance used for IGCE preparation (e.g., OSRTI's cost estimating toolbox). We recommend that the Assistant Administrator for Administration and Resources Management and the Assistant Administrator for Solid Waste and Emergency Response: 2-5 Instruct OAM and the Superfund program office to provide training to Superfund program staff on IGCE tools and databases, as well as OAM and Superfund IGCE guidance. Agency Comments and OIG Evaluation EPA agreed with recommendations 2-1, 2-4, and 2-5, and provided milestone dates for completion of the following items: • Revise OAM's IGCE guide (June 1, 2010) • Update, distribute, and maintain Superfund's supplemental tools for IGCE preparation (September 30, 2010) 10 ------- 10-P-0065 • Provide training to Superfund staff on IGCE tools and guidance (September 30, 2010). The Agency's response will address these recommendations. Regarding recommendation 2-2, EPA did not agree that it should incorporate compliance with the IGCE guide into its QAP because the QAP is an OAM tool and not a program tool. EPA stated that preparation of IGCEs is the responsibility of the program office. EPA believes that by updating the IGCE guide and providing training to the program office, IGCE quality will improve. At the exit conference, OAM officials agreed that a review to verify compliance was needed, but expressed a preference to not include that as part of the QAP. Based on the discussion, we revised the recommendation. EPA will need to provide an action plan, with milestone dates, for addressing the recommendation when responding to the final report. In response to recommendation 2-3, the Agency stated that the recommendation should be addressed to OSWER and that the offices will work together to update the IGCE guide and training. We revised the recommendation to include OSWER as suggested. At the exit conference, OAM officials clarified that the updated IGCE guide will address the practice of accepting contractor estimates that differ from IGCEs without examining the reason for the difference. With the clarification, the response addresses the recommendation. See EPA's full response in Appendix B. 11 ------- 10-P-0065 Chapter 3 EPA Region 7 Improved Cost Management for Its ESAT Contract Based on our audit work, EPA Region 7 took actions to improve the cost management of the ESAT contract. EPA Region 7 had not been reviewing and managing ESAT contract costs at the Technical Direction Form (TDF) level, which is the level where work is ordered. EPA's CMM provides that it is the policy of the Government to review contract invoices thoroughly for cost reasonableness to facilitate payment of costs which are allowable, allocable, and reasonable. Adequate invoice reviews were not occurring because the Region 7 ESAT contract gives the project officer the flexibility to determine the level of detail that needs to be contained in monthly financial reports. The PO did not require the contractor to provide cost data at the TDF level. Without detailed cost data broken out by TDF, the project officer could not determine the reasonableness of invoiced costs charged for specific work performed. During the audit, we discussed with Region 7 management and staff our concerns about the reporting under the ESAT contract. Region 7 believed that the invoices did contain all the necessary information to ensure charges were fair and reasonable. However, the Region did acknowledge that the information was not contained in an effective format for review and that efficiencies in the review process could be realized by requiring data at the TDF level in monthly reports. Therefore, Region 7 management and staff fully implemented our planned recommendation to require the contractor to include detailed cost breakouts by task order and TDF in its monthly invoices. Based on Region 7's timely response, we determined no further recommendations are needed regarding this finding in this report. 12 ------- 10-P-0065 Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Action Official Planned Completion Date Claimed Amount Agreed To Amount 2-1 10 Require OAM to update its IGCE guide to address: • Technical direction documents that exceed the simplified acquisition threshold. • Emergency acquisitions. • The GAO Cost Estimating and Assessment Guide. 2-2 10 After the OAM IGCE guide is updated, require EPA regions and program operating divisions to conduct a review to verify compliance with the updated guide. 2-3 10 Eliminate the practice of accepting contractor estimates that differ from IGCEs without examining the reason for the difference. 2-4 10 Require the Superfund program to update, distribute, and maintain its supplemental tools and guidance used for IGCE preparation (e.g., OSRTI's cost estimating toolbox). 2-5 10 Instruct OAM and the Superfund program office to provide training to Superfund program staff on IGCE tools and databases, as well as OAM and Superfund IGCE guidance. Assistant Administrator for 06/01/2010 Administration and Resources Management Assistant Administrator for Administration and Resources Management Assistant Administrator for 06/01/2010 Administration and Resources Management and Assistant Administrator for Solid Waste and Emergency Response Assistant Administrator for 09/30/2010 Solid Waste and Emergency Response Assistant Administrator for Administration and Resources Management and Assistant Administrator for Solid Waste and Emergency Response 09/30/2010 1 O = recommendation is open with agreed-to corrective actions pending; C = recommendation is closed with all agreed-to actions completed; U = recommendation is undecided with resolution efforts in progress 13 ------- 10-P-0065 Appendix A List of Superfund Contracts Reviewed Region Contract WA/TO/TDD* Type of Contract Region 4 68-W-99-043 WA-015 RAC 68-W-99-043 WA-648 RAC EP-S4-07-02 TO 0035 ERRS III EP-S4-07-02 TO 0035 - Mod 1 ERRS III EP-S4-07-02 TO-0035 - Mod 2 ERRS III EP-S4-07-02 TO-0035 - Mod 3 ERRS III EP-S4-07-02 TO 0039 ERRS III EP-S4-07-02 TO 0052 ERRS III EP-W-05-053 TO 001/TDD 026 START III EP-W-05-053 TO 001/TDD 049 START III EP-W-05-053 TO 003/TDD 041 START III Region 5 EP-S5-06-04 TDD-S05-0005-0609-067 START III EP-S5-06-04 TDD-S05-0001-0711-022 START III EP-S5-06-01 WA 021-RARA-1523 RAC 2 EP-S5-06-01 WA 044-RDRD-B51Z RAC 2 EP-S5-06-06 TO 3 ROC 3 EP-S5-06-06 TO 8 ROC 3 Headquarters EP-W-06-019 TO 3 Region 7ESAT EP-W-06-019 TO 10 Region 7 ESAT 68-W-02-073 WA 7 OSRE2 68-W-02-073 WA 19 OSRE2 EP-W-05-060 WA 1-08 HW Methods Support EP-W-05-060 WA 1-11 HW Methods Support Region 7 EP-S7-05-06 TO 005 RAC II EP-S7-05-06 TO 0027 RAC II EP-S7-05-06 TO 0062 RAC II EP-S7-05-06 TO 0010 RAC II EP-S7-05-05 TO 0028 A&E Services EP-S7-05-05 TO 003 A&E Services EP-S7-05-05 TO 0010 A&E Services EP-S7-05-05 TO 015 A&E Services 14 ------- 10-P-0065 Region Contract WA/TO/TDD* Type of Contract EP-R7-07-02 TO 0011 ERRS, Region 10 EP-R7-07-02 TO 0013 ERRS, Region 10 EP-R7-07-02 TO 007 ERRS, Region 10 EP-S7-06-01 TO 0023 START III EP-S7-06-01 TO 0048 START III EP-S7-06-01 TO 0051 START III Region 2 EP-W-04-054 TO 0065 ERRS EP-W-04-054 TO 0057 ERRS 68-W-98-214 153-RDRD-0291 RAC 68-W-98-214 158-RARA-01G1 RAC EP-W-05-041 TO 0010 ROC III EP-W-05-041 TO 014 ROC III EP-W-05-048 WA 081 Site Assessment Team 2 EP-W-05-048 WA 057 Site Assessment Team 2 Source: OIG analysis of contract data. A&E Architect and Engineering HW Hazardous Waste OSRE Office of Site Remediation Enforcement ROC Regional Oversight Contract TDD technical direction document TO task order WA work assignment 15 ------- 10-P-0065 Appendix B Agency Response December 17, 2009 MEMORANDUM SUBJECT: Response to Draft Audit Report: EPA Superfund Can Improve its Preparation and Use of Independent Government Cost Estimates, Project OA-FY08-0255 FROM: Craig E. Hooks Assistant Administrator TO: Janet Kasper Director, Contracts and Assistance Agreements Office of the Inspector General We appreciate the opportunity to comment on the draft report entitled "EPA Superfund Can Improve its Preparation and Use of Independent Government Cost Estimates," dated November 13, 2009. Our comments on the report and recommendations are below: Specific Recommendations and Responses: Recommendation 2-1 - We recommend the Assistant Administrator for OARM require OAM to update its IGCE guide to address: • Technical direction documents that exceed the Simplified Acquisition Threshold • Emergency acquisitions • The Government Accountability Office's cost estimating guide Response - We concur with this recommendation. This will be a development process to fold in the three new areas above into the current IGCE guide. OAM will begin the development and stakeholder review process immediately with completion of guide revisions by June 1, 2010. Recommendation 2-2 - We recommend that the Assistant Administrator for OARM, after the OAM IGCE guide is updated, require EPA regions and program operating divisions to include compliance with the IGCE guide as part of their Quality Assurance Programs. 16 ------- 10-P-0065 Response - While we agree that OAM will update the IGCE guide (see Recommendation 2-1 response above) and work with OSWER to provide IGCE training to Superfund program staff (see Recommendation 2-5 response below), we do not believe that contracting offices should be required to modify their QAPs. Section 4.2 of the Acquisition Handbook states that QAPs shall contain oversight activities for "all primary contracting workload functions performed by the contracting organization." Page 1 of this report states that "EPA's program offices are responsible for preparing IGCEs." We do not believe that the QAPs need to be modified to specifically focus on primary activities which fall within EPA's program offices. The Contract Management Manual Section 7.3, "Procurement Initiation and Related Documentation," lists the Guide for Preparing IGCEs as a guidance source for program officials, in addition to providing an on-line link to this document. The updated guidance will be posted on the intranet by June 30, 2010. By updating the guide, assisting OSWER in providing IGCE training to program officials, and performing current QAP oversight activities related to reviewing procurement request packages, we believe OAM will help improve the quality of IGCEs prepared by its customers. Recommendation 2-3 - We recommend the Assistant Administrator for OARM eliminate the practice of accepting contractor estimates that differ from IGCEs without examining the reason for the difference. Response - We believe this recommendation should be directed to the Assistant Administrator of OSWER, as well as OARM. These Offices share IGCE responsibilities, and they will work together to update the guidance and provide training, which will effectively address this recommendation by June 1, 2010. Recommendation 2-4 - We recommend the Assistant Administrator for OSWER: require the Superfund program to update, distribute and maintain its supplemental tools and guidance used for IGCE preparation (e.g., OSRTI Cost Estimating Toolbox). Response - We concur with this recommendation. OSWER will work closely with OAM to complete this action by September 30, 2010. It should be noted that OSWER provided formal IGCE training at the Superfund National Association of Remedial Project Managers' (NARPM) annual training conference and the Superfund Project Officers/Contracting Officers' (PO/CO) annual conference in July 2009. Recommendation 2-5 - We recommend the Assistant Administrator for OARM and Assistant Administrator for OSWER instruct OAM and the Superfund program office to provide training to Superfund program staff on IGCE tools and databases, as well as OAM and Superfund IGCE guidance. 17 ------- 10-P-0065 Response - We concur with this recommendation. OSWER will work closely with OAM (after guidance updates are completed) to complete the action by September 30, 2010. General Comments: Abbreviations - RAC stands for Remedial Action Contract, it is written as Response Action Contract in the report. This also needs to be corrected in Chapter 1, under Noteworthy Achievements. Chapter 1 - Scope and Methodology, page two, second paragraph, first sentence - We recommend listing the various types of Superfund contracts included in review (e.g., Removal, Lab, Remedial, Federal Facilities or Enforcement). Chapter 2 - Superfund Needs to Better Document Rationale, Assumptions, and Changes, page five, first paragraph, last sentence - We recommend changing the sentence to read, "Documentation supporting IGCEs should be included that will allow a cost analyst unfamiliar with the program to determine how the estimate was developed." 18 ------- 10-P-0065 Appendix C Distribution Office of the Administrator Assistant Administrator for Administration and Resources Management Assistant Administrator for Solid Waste and Emergency Response Regional Administrator, Region 7 Agency Follow-up Official (the CFO) Agency Follow-up Coordinator General Counsel Director, Office of Superfund Remediation and Technology Innovation, Office of Solid Waste and Emergency Response Acting Director, Office of Acquisition Management, Office of Administration and Resources Management Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for Public Affairs Audit Follow-up Coordinator, Office of Administration and Resources Management Audit Follow-up Coordinator, Office of Solid Waste and Emergency Response Audit Follow-up Coordinator, Region 7 Acting Inspector General 19 ------- |