ฃ < 33 O \ O * 2E I U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Improved Contract Administration Needed for the Customer Technology Solutions Contract Report No. 13-P-0398 September 16, 2013 Scan this mobile code to learn more about the EPA OIG. ------- Report Contributors: Michael D. Davis Teren Crawford Nancy Dao Kevin Haas Debra C off el Abbreviations CO Contracting Officer COTR Contracting Officer Technical Representative CPARS Contractor Performance Assessment Reporting System CSIRC Computer Security Incident Response Capability CTS Customer Technology Solutions EPA U.S. Environmental Protection Agency EPAAR Environmental Protection Agency Acquisition Regulations FAR Federal Acquisition Regulations ISO Information Security Officer NIH-CPS National Institutes of Health's Contractor Performance System OAM Office of Acquisition Management OARM Office of Administration and Resources Management OIG Office of Inspector General OMB Office of Management and Budget OTOP Office of Technology Operations and Planning PO Project Officer PPIRS Past Performance Information Retrieval System QAM Quality Assurance Manager QASP Quality Assurance Surveillance Plan SP Service Provider SRI Service Request Interface WCF Working Capital Fund Hotline To report fraud, waste, or abuse, contact us through one of the following methods: email: OIG Hotline@epa.gov write: EPA Inspector General Hotline phone: 1-888-546-8740 1200 Pennsylvania Avenue, NW fax: 202-566-2599 Mail code 2431T online: http://www.epa.gov/oiq/hotline.htm Washington, DC 20460 ------- ^ฃDsrx A v lSi U.S. Environmental Protection Agency Office of Inspector General At a Glance 13-P-0398 September 16, 2013 Why We Did This Review The U.S. Environmental Protection Agency Office of Inspector General conducted this audit to determine whether the EPA implemented effective contract administration for its Working Capital Fund contract EPW08034. The Office of Administration and Resources Management's Office of Acquisition Management and the Office of Environmental Information's Office of Technology Operations and Planning perform contract administration activities with contracting officers, project officers and quality assurance managers for the Customer Technology Solutions contract EPW08034. Contract administration involves those activities performed by government officials after a contract has been awarded to determine how well the government and the contractor performed to meet the contract requirements. Contract administration assists the government in assuring it receives the goods and services for which it paid. This report addresses the following EPA theme: Embracing EPA as a high performing organization. For further information, contact our Office of Congressional and Public Affairs at (202) 566-2391. The full report is at: www.epa.qov/oiq/reports/2013/ 20130916-13-P-0398.pdf Improved Contract Administration Needed for the Customer Technology Solutions Contract What We Found Based on our review of the WCF contract EPW08034, which ended September 2012, the EPA needs to improve its contract administration to assist in managing other similar type contracts. The EPA did not, as stated by Office of Management and Budget, Federal Acquisition Regulations and agency guidelines: Use performance standards to measure cost outcomes. Complete any of the required contractor performance evaluation reports. Maintain required contract administration documents. The EPA did not have policies in place that would require performance metrics and standards to be linked to cost outcomes and procedures to ensure contract administrators maintain sufficient documents in the official contract files. The EPA did not complete contractor performance reports because of insufficient guidance and inadequate communications during contractor performance system changes and personnel reassignments. The EPA's contractor performance systems include the National Institutes of Health's Contractor Performance System and subsequently the Contractor Performance Assessment Reporting System, which transmits information into the Past Performance Information Retrieval System. The EPA's ineffective contract administration may have hindered the ability of EPA staff to ensure that the contractor successfully met agency needs, as well as its ability to determine whether the EPA achieved the best value for the $85 million expended on the WCF contract. Recommendations and Planned Agency Corrective Actions We recommend that the OARM assistant administrator update its policies and procedures to ensure that contract performance metrics and standards link to cost outcomes. We also made recommendations involving updating internal programs to provide oversight and accountability for linking metrics to cost outcomes and the review and submission of contractor performance evaluation reports for the contract reviewed. Further, we recommend that OARM develop contract administration procedures related to the transfer of documents when reassigning contract administrative staff. OARM concurred with four of the six recommendations and did not concur with recommendations 1 and 2. The OARM's corrective actions for recommendations 3 through 6 do not contain milestones nor completely address the recommendations and are unresolved. Noteworthy Achievements The Office of Acquisition Management implemented the Balanced Scorecard program in fiscal year 2011 for the EPA Acquisition Systems to provide the necessary checks and balances to ensure procurement documents are of the highest quality and comply with applicable laws, regulations and policies. ------- ^ED sx ฆ ฃ% ' < rJK-7 5 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY | W\/C/ ฎ WASHINGTON, D.C. 20460 "U PRO^ THE INSPECTOR GENERAL September 16, 2013 MEMORANDUM SUBJECT: Improved Contract Administration Needed for the Customer Technology Solutions Contract Report No. 13-P-0398 FROM: Arthur A. Elkins Jr. /fiM*f L* ฆ (/^r^ TO: Craig E. Hooks, Assistant Administrator Office of Administration and Resources Management Renee P. Wynn, Acting Assistant Administrator and Chief Information Officer Office of Environmental Information This is our report on the subject audit conducted by the Office of Inspector General of the U.S. Environmental Protection Agency. This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Action Required The agency agreed with four of the six recommendations in this report. However, all six recommendations are considered unresolved pending our receipt of EPA's corrective action plan and estimated completion dates that correspond with the specific recommended actions. Therefore, for all recommendations, you are required to provide a corrective action plan with planned completion dates within 60 days of report issuance. We will post this report to our website at http://www.epa.gov/oig. If you or your staff have any questions regarding this report, please contact Richard Eyermann, acting assistant inspector general for the Office of Audit, at (202) 566-0565 or evermann.richard@epa.gov; or Mike Davis, director for Efficiency Audits, at (513) 487-2363 or davis.michaeld@epa.gov. ------- Improved Contract Administration Needed for the Customer Technology Solutions Contract 13-P-0398 Table of C Chapters 1 Introduction 1 Purpose 1 Background 1 Noteworthy Achievements 3 Scope and Methodology 4 2 Ineffective Administration of Contractor Performance 6 OMB, FAR and EPAAR Provide Guidance for Managing Contractors 6 Performance Metrics and Standards Not Linked to Cost Outcomes 7 Contractor Performance Evaluation Reports Not Completed 9 Conclusion 11 Recommendations 11 Agency Comments and OIG Evaluation 12 3 Inadequate Contract Records Management 13 Federal and Agency Guidance on Records Management 13 Agency Official Contract Files Missing Required Documents 14 Conclusion 16 Recommendations 16 Agency Comments and OIG Evaluation 16 Status of Recommendations and Potential Monetary Benefits 17 Appendices A Contract Performance Requirements Summary 18 B PO, CO and QAM Reassignments on CTS Contract 22 C Relevant Recordkeeping Criteria 23 D Agency Comments on Draft Report 25 E Distribution 32 ------- Chapter 1 Introduction Purpose The U.S. Environmental Protection Agency Office of Inspector General conducted this audit to determine whether the EPA implemented effective contract administration for its Working Capital Fund contract EPW08034. Even though the contract ended on September 30, 2012, the EPA awarded subsequent contracts for the same services. In addition, the EPA can apply lessons learned from this audit to improve its contract administration that will assist in managing other similar types of contracts (see chapters 2 and 3). Background Contract administration involves government activities performed after an agency awards a contract to determine whether the contractor met contract requirements. Contract administration activities include controlling costs, monitoring, measuring and reporting contractor performance, and maintaining records in the official contract files. Contract administration constitutes that primary part of the procurement process that assures the government gets what it pays for. The WCF is a revolving fund authorized by law to finance a cycle of operations in which federal agencies can charge the costs for goods or services provided to its customers. WCF customers include the EPA's program offices and regions. The role of the WCF is to provide a centralized source of administrative and support services for the EPA. The WCF strives to reduce the costs of services in the agency through improved efficiencies gained by achieving economies of scale, greater consumer bargaining power, and reduction in overhead. The Office of Administration and Resources Management's Office of Acquisition Management and the Office of Environmental Information's Office of Technology Operations and Planning provide contract administration on the WCF contract EPW08034. OTOP personnel manage the work performed under the contract, which is with Customer Technology Solutions. CTS, a WCF data processing service, provides and coordinates all information technology end-user support and services for the EPA's headquarters and field offices. Critical participants in the contract administration of the WCF contract include: OAM's contracting officers, who have overall responsibility for the contract. OTOP's project officers, quality assurance managers and quality assurance evaluators, who monitor contractor performance and customer satisfaction. 13-P-0398 1 ------- Even though the Office of the Chief Financial Officer manages the WCF, including reviews and processing of invoices, OAM and OTOP provide contract administration functions. In April 2008, the EPA awarded an indefinite delivery/indefinite quantity hybrid contract. This hybrid contract also contained provisions for performance-based award fees and fixed hourly rates and material costs for a time and materials contract. This contract required the contractor to provide technology services to approximately 12,000 program office users scattered throughout 16 states and the District of Columbia. The total potential value of the contract was over $200 million and covered the contract base period of June 1, 2008, to September 30, 2012, and an option period available through December 31, 2016. The EPA decided not to exercise the option period because the contractor had not met the required performance metrics. Figure 1 summarizes the WCF's top five data processing service contracts obligated as of May 2012. Figure 1: Top Five WCF obligations as of May 20121 WCF Contracts (millions of obligations)* ฆ Network Security and New Custom Application ฆ Cominercial-Off-The- Shelf ฆ Customer Technology Solutions ฆ Infrastructure ฆ Hosting Source: OIG analysis of EPA data. The agency identified approximately $72 million of funds obligated for the CTS contract EPW08034 as of May 2012. By October 2012, the EPA's Compass Financial2 system reported that the agency's expenditures for this CTS contract increased to approximately $85 million. 1 OTOP's WCF coordinator provided this information on May 11, 2012, in the form of an email. 2 Compass is the EPA's official financial database. 13-P-0398 2 ------- The Federal Acquisition Regulation, Subsection 37.601(b), requires that: Performance-based contracts for services shall include(1) A performance work statement (PWS); (2) Measurable performance standards (i.e., in terms of quality, timeliness and quantity, etc.) and the method of assessing contractor performance against performance standards; and (3) Performance incentives where appropriate. The FAR Subsection 37.603(a) requires that performance standards establish the performance level required by the government to meet the contract requirements. In addition, this same FAR Subsection requires the performance standards to be measurable and structured to permit an assessment of the contractor's performance. Th eEPA Contracts Management Manual [CMM], Section 11.1.5.7, "Developing a Performance Based Service Contract Quality Assurance Surveillance Plan," states that a: The quality assurance surveillance plan (QASP) defines what the Government must do to ensure that the contractor has performed in accordance with the PWS performance standards. The QASP objective is to assess the ability of the contractor to achieve the defined performance standards and fulfill contractual obligations. According to the CTS contract, the EPA defined the performance standards in the contract's Technical Exhibit under the title "Performance Requirement Summary." This CTS performance-based contract has a performance work statement, measurable performance standards, performance incentives and a QASP. The Performance Requirement Summary identified 44 performance metrics and standards. According to the QAM, the EPA used no other performance metric or standard to measure contractor performance. In addition, the EPA performed weekly and monthly evaluations based on the 44 performance metrics to monitor contractor performance per the FAR. According to the contract's award fee plan, the EPA also used the 44 performance metrics and standards to determine whether the EPA would provide an award fee to the contractor. The 44 performance metrics identified categories that relate to quality and technical performance (a degree of excellence), timeliness and scheduling (meeting milestones), and business relations (customer satisfaction). Details are in appendix A. Noteworthy Achievements OAM implemented the Balanced Scorecard Program in fiscal year 2011 for the EPA Acquisition Systems. The internal control program works in conjunction with the Balanced Scorecard Program to provide the necessary checks and balances to ensure that procurement documents are of the highest quality and in compliance with applicable laws, regulations and policies. The Balanced Scorecard Program also seeks to include measures of quality, cost, timeliness, customer service and employee alignment with skills, to provide an in-depth, 13-P-0398 3 ------- predictive performance management system. Even though the EPA implemented specific measures in the Balance Scorecard Program to ensure that past performance information is timely and included in the Contractor Performance Assessment Reporting System, the specific measures did not start until the end of fiscal year 2012. Scope and Methodology We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We conducted the audit from May 2012 to June 2013 at OTOP, OAM and the Office of the Chief Financial Officer's Research Triangle Park Finance Center. During the audit, we obtained the top five WCF data processing service contracts and reviewed the contract with the highest reported obligated amount to determine whether the EPA was effectively managing WCF contracts. We limited our review to auditing the EPA's contract administration for the CTS EPW08034 contract because it accounted for approximately $72 million (60 percent) of the approximately $120 million in the EPA WCF contracts (see figure 1). We reviewed memoranda, regulations, guides and other documents to gain an understanding of the requirements and processes used to provide effective contract administration. The specific documents reviewed included: Office of Management and Budget memoranda. Office of Federal Procurement Policy Memoranda, and Guides. FAR Parts 1, 4, 16, 32, 37, 42 and 46. Environmental Protection Agency Acquisition Regulations Sections 1509.170 and 1542.15 The EPA CMM. Appendix C includes relevant recordkeeping criteria. We looked at OMB, FAR, agency polices and guidance for contract and contract administration requirements. We discussed with OAM, OTOP and Research Triangle Park Finance Center staff their procedures used to manage contractor performance, invoices, monthly progress reports and other needed contract administration functions used for effective internal controls. We also discussed other background information that pertains to WCF procedures and associated key players. The WCF background information obtained involved the systems used, documents managed and other WCF funding procedures used. Other WCF key players included OTOP's Director of Customer Business Support Staff, who oversees WCF workload, collections, billings and reports. The director also interacts with WCF customers who 13-P-0398 4 ------- provide feedback on service quality. We reviewed contract invoices, checklists, monthly progress reports, QASPs, prior audit reports, award fee determination reports, as well as other supporting documentation, to determine whether the EPA conducted effective contract administration. Prior Agency OIG Reports The EPA OIG issued the following reports with findings related to CTS contract administration: Report No. 10-P-0194, EPA Needs to Improve Management Practices to Ensure a Successful Customer Technology Solutions Project, August 23, 2010. The agency's lack of acquisition planning led to questions involving: The quality of the helpdesk supporting the project. A quality management program that was not finalized. Key business processes to support ongoing operations not being defined. Vacant leadership positions needed to facilitate communication and coordination with customers about CTS equipment deployments. We did not follow up on the contractor providing helpdesk support on the project because the EPA cancelled the contract. We did follow up on areas that affected contract administration. We found the following: The EPA finalized the QASP. The EPA ensured the contractor defined key business process documents. The EPA filled vacant leadership positions. Report No. ll-P-0705, EPA Contract Oversight and Controls Over Personal Computers Need Improvement, September 26, 2011. The EPA paid the CTS contractor a total of $489,734 over an 11-month period for 3,343 seats. A standard seat includes a leased computer with accessories and technical support, which was not ordered by the agency during the aforementioned period. We recommended that the EPA should improve controls for updating data in the fixed assets database, and should retain property acquisition documentation in accordance with retention requirements. In addition, we recommended that the agency should have a separation of duties in its property staff positions and consider assigning permanent property positions. We did not follow up on this condition because the EPA decided not to exercise the contract option award terms because the contractor had not met the required performance metrics. However, during this review, we found similar internal control issues in terms of updating databases, including: Retaining appropriate documentation. Maintaining permanent contract positions (see the following chapters). 13-P-0398 5 ------- Chapter 2 Ineffective Administration of Contractor Performance OAM did not effectively evaluate cost outcomes and report any of the four required annual contractor performance evaluation reports in the Past Performance Information Retrieval System between September 2009 and March 2013. OMB and FAR require agencies to assess cost outcomes, while the FAR and EPAAR require agencies to report on contractor performance information in the PPIRS. Deficiencies occurred because of insufficient guidance and inadequate communications during contractor performance system changes and personnel reassignments. Consequently, the EPA did not effectively monitor costs to ensure that the contract, with expenses at approximately $85 million, achieved the best possible cost outcomes and value. In addition, the EPA did not provide other federal acquisition community users with contractor past performance data so that other federal government agencies can make effective management decisions on whether to conduct business with the same contractor. OMB, FAR and EPAAR Provide Guidance for Managing Contractors OMB and the FAR established requirements for evaluating contractor performance for cost outcomes. OMB policies state that evaluation factors be directly linked to cost performance results.3 Another OMB policy4 addresses the evaluating of cost controls based on the contractor's: Ability to perform within or below budget. Use of cost efficiencies. Relationship of negotiated costs to actual costs. Submission of reasonably priced change proposals. Providing current, accurate and complete billing in a timely manner. This policy addressed the assessing of the contractor's effectiveness in forecasting, managing and controlling contract cost. This policy also stated cost controls are needed for fixed price contracts. The FAR5 requires that, for incentive fee type and time and materials contracts, evaluating factors be linked to cost performance results and cost controls, respectively. The FAR and EPAAR regulations establish requirements for reporting contractor performance information. Specifically, the FAR6 requires that agencies prepare an annual evaluation of contractor performance that closely parallels the award fee 3 OMB memorandum. [ppropriale Use of Incentive Contracts, December 4, 2007; and OMB memorandum. Improving Government Acquisition, dated July 29, 2009, 4 OMB policy Best Practices for Collecting and Using Current and Past Performance Information, May 2000. 5 FAR Subsections 16.401(e)(2), 16.402-l(a) and 16.601(c)(1). 6 FAR Subpart 42.1500, Subsections 42.1503(b), and 42.1503(c). 13-P-0398 6 ------- determinations and submit past performance information electronically to the PPIRS in accordance with agency procedures. The agency's acquisition regulations7 require the agency to use the National Institutes of Health's Contractor Performance System, subsequently replaced by CPARS, to record contractor performance information. The systems collect and transmit information into the PPIRS. Performance Metrics and Standards Not Linked to Cost Outcomes The EPA's 44 performance metrics and standards identified in the Performance Requirements Summary for the EPW08034 CTS contract did not include cost outcomes as required by the FAR and addressed in OMB guidance. The EPA used the 44 performance metrics and standards to evaluate contractor performance on the CTS contract and determine whether the contractor would receive an award fee. The QASP identifies the procedures used by the QAM and quality assurance evaluator to review contractor performance based on the Performance Requirement Summary and its 44 performance metrics and standards. The QAM assembled biannual reports that identified and rated each of the 44 performance metrics and standards used to determine whether the contractor would be granted an award fee. Appendix A identifies the 44 metrics and standards that relate to quality, schedule and customer satisfaction outcomes. The EPA did not include any metrics and standards relating to cost outcomes as required by the FAR. The 44 performance metrics and standards did not include the OMB-identified definitions for cost controls that cover: The contractor's ability to perform within or under budget. Use of cost efficiencies. Relationship of negotiated costs to actual costs. Submission of reasonably priced change proposals. The contractor providing current, accurate and complete billing in a timely manner. Also, the CTS contract performance work statement identified a requirement for the contractor to analyze potential cost savings from new technologies and business process changes. FAR Subsection 37.602(b) (3) requires agencies to "rely on the use of measurable performance standards and financial incentives in a competitive environment to encourage competitors to develop and institute innovative and cost-effective methods of performing the work." The EPA did not use the 44 performance metrics and standards or incentives to encourage competitors to develop and institute innovative and cost-effective methods of performing the work per FAR requirements. As part of the EPA Balanced Scorecard Program approach, OAM identified measures for cost control in contractor performance evaluations as principles 7 EPAAR Subpart 1542.1503(a) (CPARS), which replaced 1509.170-3(c) and 5(c) (NIH-CPS). 13-P-0398 7 ------- needed to improve its acquisition process. However, the EPAAR and CMM did not require that for incentive fee and time and material type contracts the performance evaluation factors include cost outcomes. In accordance with the December 2007 OMB memorandum, chief acquisition officers and senior procurement executives review agency's acquisition policies to ensure that incentive fees are linked to acquisition outcomes such as cost, schedule, and performance results. Current EPA policy does not address this requirement. Even though this contract has ended, having this requirement will ensure that existing and future contracts will contain the required cost outcome metrics and standards. By not linking the award fee to cost outcomes as required, the EPA did not effectively hold the contractor accountable for cost inefficiencies and inaccuracies. We found examples where the contractor billed for services that were not rendered. For example, Table 1 identifies examples of 13 invoices totaling nearly $1 million that the agency suspended for inaccurate billings. Suspended contractor invoice payments due to inaccurate billings indicate potential problems exist with contract cost. Table 1: Invoices suspended Invoice Suspended amounts Bill date Reason for suspension 1 $2,734.41 07/20/10 Units not deployed. Invoice is inaccurate (data does not support the billable amount for the time period). 2 3,125.04 10/26/10 Units not deployed. 3 117,775.00 08/13/09 Dispute over number of printers installed. 4 4,330.57 08/21/09 Fifteen student/intern seats deployed on a Saturday. User cannot accept deployment on Saturday. Need to count these seats next month. 5 281,050.00 10/13/09 Dispute over printers installed. 6 391,650.00 11/20/09 Dispute over printers installed. 7 3,767.62 08/12/10 Units not deployed. 8 53,082.25 02/27/12 Work not completed 9 15,961.82 11/13/12 Typo error on prior payment. 10 29,784.47 05/21/12 This invoice indicates a double billing for the same service. 11 9,674.35 08/12/10 Units not deployed. 12 130.21 10/22/10 Units not deployed. 13 27,433.72 07/06/12 Questioned billing method of rounding. Total $940,499.46 Source: OIG data analysis. In addition, since FAR requires cost outcomes from past contractor performance be inputted into PPIRS, the EPA should have evaluated the cost outcome based on contractor performance standards. As a result, the EPA cannot sufficiently inform other federal agencies about the contractor's cost performance. Even though the agency did not award the contractor an award fee, there could have been additional corrective actions implemented by having cost outcomes tied 13-P-0398 8 ------- to contractor performance standards. This would have provided additional enforcement measures to prevent recurring or escalating problems. According to the QASP, the QAM has the responsibility to monitor the contractor's performance to control cost and take corrective actions. The performance work statement requires the contractor to look for cost savings through enhanced technology. The EPA did not tie the requirement to performance standards so the EPA did not use this requirement for determining an award fee. Including cost outcomes could have driven the overall ratings up if the contractor identified savings and could have driven the overall ratings further down if the contractor did not identify savings. Such actions would facilitate a determination as to whether the contractor is performing to meet the overall contract needs and providing the most value for the government. Contractor Performance Evaluation Reports Not Completed The EPA did not complete the four required annual contractor past performance evaluation reports. The EPA completes contractor past performance evaluations when an agency official enters the finalized reports in CPARS. Prior OIG reports identified similar issues where the EPA did not complete the evaluations or report past performance information into required systems in a timely fashion.8 Both the FAR and EPAAR require the agency to conduct and report the evaluations in the past performance system after the end of each 12-month contractor performance period. The initial EPAAR Subpart 1509.170, dated 2002, required the EPA to use the NIH-CPS to report past performance information within 90 business days from the date the contracting officer initiates the evaluation. As of September 2010, the new EPAAR Subsection 1542.1502 replaced Subpart 1509.170, and requires the use of the Department of Defense's CPARS [see prior Footnote 7], CPARS requires reporting of past performance information within 120 days. Table 2 identifies the specific contractor past performance data not timely entered into the system. Table 2: Timeliness of Contractor Performance Evaluations for EPW08034 Evaluation period Reporting system * Evaluation Date due Interim/final evaluations # Calendar Days Past Evaluation Due Date (04/01/2013=OIG cut-off date) 10/01/08-09/30/09 NIH-CPS 02/11/10 (1) Not prepared 1145 10/01/09-09/30/10 CPARS 02/11/11 (1) Not prepared 780 10/01/10-09/30/11 CPARS 02/10/12 (1) Not prepared 416 10/01/11-09/30/12 CPARS 01/28/13 (2) Not prepared 63 *Past performance information in NIH-CPS/CPARS is automatically transmitted to the PPIRS. (1) 95 business days allowed for years 1-3's evaluation process. (2) 120 calendar days allowed for evaluation process for year 4 per new EPAAR Subsection 1542.1502. 8 Report No. 10-R-0113, EPA Should Improve Its Contractor Performance Evaluation Process for Contractors Receiving Recovery Act Funds, issued April 26, 2010; and Report No. 12-P-0417, Weaknesses in EPA's Management of the Radiation Network System Demand Attention, issued April 19, 2012. 13-P-0398 9 ------- Source: OIG analysis of contractor performance evaluations as of 04/01/2013. The 44 performance metrics and standards (see appendix A) identify the performance categories quality, timeliness and business relations. Quality assurance evaluators review contractor performance based on the metrics and standards and submit semiannual reports that contain needed information to input into the CPS or CPARS, but the EPA did not use such information to complete contractor past performance evaluation reports in the systems. The EPA did not have the contractor performance information in PPIRS because of inadequate communication during the NIH-CPS and CPARS transitions and personnel reassignments. NIH-CPS and CPARS are the official systems used by the agency to download past performance information into PPIRS. Even though the FAR required contractor performance information to be entered into PPIRS, the EPA did not provide clear guidance to enforce the reporting of contractor information for existing contracts. The OAM intranet website stated that the CPARS would be used for new solicitations issued on October 3, 2011, or later. In addition, OAM's Intranet also emphasized EPAAR changes for new solicitations but did not require the EPA to input contractor performance information into CPARS for existing contracts. The Interim Policy Notice 10-03 issued by OAM in March 2010 also emphasizes that solicitations issued, contracts awarded and options exercised after May 14, 2010, are to follow the EPAAR deviations to report contractor performance information in CPARS. The EPAAR deviations did not require existing contracts to report contractor performance information. Since the CTS contract, awarded in 2008, was an active contract at the time and EPA did not provide clear guidance on existing contracts, the EPA did not report on four contractor performance evaluation periods (see table 2 above). In addition, when the EPA's OAM and OTOP reassigned staff, these offices did not communicate adequate procedures to ensure reporting of contractor performance information. The initial QAM stated she asked the CO about putting information into NIH-CPS, but the CO said that, since the EPA changed systems, they should do nothing. One PO, assigned to the contract for a few months, did not fully understand the CPARS assessment process and did not have access to the CPARS. One CO stated that she transitioned out before the end of the 12- month evaluation period. Another CO said she requested the PO to assess the contractor performance but received no response from the PO. With many personnel reassignments (see appendix B), the EPA has no effective procedure to communicate what CPARS evaluations have or have not been done. The Department of Defense CPARS policy guide9 states an interim evaluation in CPARS is required upon a significant change within the agency provided 6 months of performance has occurred. The changes include change in program or 9 The PPIRS website, www.ppirs.gov, and OMB memorandum, Improving Contractor Past Performance Assessments: Summary of the Office of Federal Procurement Policy's Review, and Strategies for Improvement, dated January 2011, acknowledges the DOD policy as guidance for federal agencies on contractor performance information. The DOD policy was subsequently revised in June. 13-P-0398 10 ------- project management responsibility and transfer of a contract to a different contracting activity. It also states that an interim evaluation in CPARS shall be started prior to transfer of assessing official duties from one individual to another, to ensure continuity. If COs did not report the contractor performance information in the past performance systems for all contract performance periods, performance reports are not available to inform the EPA and other federal agencies about the contractor's performance. According to the OMB memorandum identified in the footnote 9, this information is used in the source selection process to assist other agencies in assessing past performance when making future acquisitions. Without timely availability of contractor past performance information that includes information on contract cost, the EPA and other federal agencies may inadvertently award contracts to contractors that have a history of poor performance on current or past government contracts. Since the EPA did not provide an award fee throughout the entire life of this 4-year performance-based contract, there may be concerns with the contractor performance that should be reflected and available for consideration by other agency offices and government agencies. An OMB memorandum, Clarifying Chief Acquisition Officer Roles and Responsibilities, dated October 2012, states that chief acquisition officers ". . . should lead efforts to, among other things, improve the value of contractor past performance assessments and increase the transparency of contractor business integrity data so that the Federal Government only does business with reputable firms." Conclusion Cost controls and contractor performance information have been the focus of OMB memoranda and the Balanced Scorecard Program addressed in chapters 1 and 2 above. The EPA did not evaluate cost outcomes and report on any required contractor performance information as required. Having effective cost controls and past contractor performance information for this contract valued at approximately $85 million will help the EPA assure that the government gets what it paid for. It will also provide transparency so that other federal government agencies can make effective management decisions on whether to conduct business with the same contractor. Recommendations We recommend that the assistant administrator, Office of Administration and Resources Management: 1. Update the policies to include procedures that would ensure that, for all incentive and time and material contracts, contract performance metrics and standards include cost outcomes. 13-P-0398 11 ------- 2. Update internal programs that provide oversight and accountability to ensure that, for all incentive type and time and material contracts, the EPA performance metrics and standards link to cost outcomes. 3. Require OTOP, in conjunction with OAM, to review contractor performance reports generated per the QASP. Require OTOP to coordinate all actions taken with the COs and POs and require the COs to input contractor performance information into CPARS. 4. Require COs to input interim reports into CPARS prior to their reassignments. Agency Comments and OIG Evaluations OARM did not concur with recommendations 1 and 2, but concurred with recommendations 3 and 4. However, recommendations 3 and 4 are unresolved pending receipt of corrective actions that correspond with the causes and recommendations. OARM disagreed with recommendations 1 and 2. OARM stated EPA does not have to control cost or manage the contract-based cost incentives for this WCF contract because the contract has fixed rates so it is a fixed unit price contract. FAR Subsections 16.202-1 and 203-l(b) requirements state that for fixed price contracts, the award fee or incentive is based solely on factors other than costs; however, this contract's requirements and data indicated a contract type other than a fixed price contract. FAR Subsection 16.201(b) states "time and material contracts and labor hour contracts are not fixed price contracts." The EPA's Acquisition System and the Federal Procurement Data System identified this contract to be a time and materials contract. The contract also identified changes to fixed hourly rate and negotiated hours throughout the contract 4-year time period and identified fixed rate and other elements that is consistent with FAR Subpart 16.6 for time and materials contracts. Recommendations 1 and 2 also reflect on the performance standards having cost outcomes and the decision to award a fee based on cost outcomes. FAR Subsection 1.102-2 states that performance standards satisfy the customer in terms of cost, quality and timeliness of delivered product or service, so performance standards as well as the decision to award a fee should link to cost outcomes. OARM stated during the exit conference that issues found and addressed under recommendations 3 and 4 relate to inadequate management oversight. Based on the causes addressed above, we found no EPA procedures in place to direct the actions of inputting interim performance information into CPARS when there are staff reassignments. This detailed direction would assist management in providing the oversight to ensure the contractor performance information is input into CPARS. In addition, this is a recurring issue because prior audits recommended improvements to management oversight that the agency agreed to yet the conditions still exist (see footnote 8). As a result, we concluded that this further direction is needed. 13-P-0398 12 ------- Chapter 3 Inadequate Contract Records Management The official contract files maintained by COs, POs and the QAM were missing required official documents. These missing documents were: Thirty-four of 48 monthly progress reports. A post award conference meeting document. Three of four CO annual contract invoice reviews. A financial monitoring review report and the CO's applicable responses. Initial or finalized QASPs. Six of eight letters sent to the contractor providing its performance ratings. FAR Subsection 4.802 and agency policies require that contract files be maintained under central control to ensure effective documentation, safeguarding and efficient retrieval. These required documents were missing due to ineffective oversight of the policies and procedures in place to ensure the easy retrieval of official documents during the many staff reassignments. Because the EPA did not maintain adequate files on an approximately $85 million contract, the agency could not provide assurance that the EPA made informed business decisions to ensure contractors met contract requirements and the EPA complied with criteria. As a result, the EPA cannot also provide assurance that the government effectively got what it paid for. Federal and Agency Guidance on Records Management FAR Subsection 4.801, General, states documentation in the files shall be sufficient for purpose of providing a complete background for informed decisions and furnishing essential facts in the event of litigations or congressional inquiries. FAR Subsection 4.802, Contract Files, paragraphs (c) and (d), states: Files must be maintained at organizational levels that ensure: (1) Effective documentation of contract actions; (2) Ready accessibility to principal users; (3) Minimal establishment of duplicate and working files; (4) The safeguarding of classified documents; and (5) Conformance with agency regulations for file location and maintenance. If the contract files or file segments are decentralized (e.g., by type or function) to various organizational elements or to other outside offices, responsibility for their maintenance must be assigned. A central control and, if needed, a locator system should be established to ensure the ability to locate promptly any contract files. 13-P-0398 13 ------- The EPA policies and procedures emphasize the need for creating and managing the records necessary to document the agency's official activities and actions. The following agency policies are applicable. EPA Classification No. CIO 2155, EPA Records Management Policy, requires that all agency employees are responsible for creating, managing and filing records for safe storage and efficient retrieval. The EPA Records Management Manual maintains that agency staff must capture records by filing, storing or otherwise systematically maintaining them in a recordkeeping system to ensure information is accessible to all authorized staff, including related records stored on special media or in different locations. EPA Record Series, Schedule 202, NARA Disposition No. Nl-412-06- 6/5, calls for COs and contracting officer representatives to work together to determine who is responsible for maintaining specific documents to minimize duplication while still providing an adequate audit trail (see appendix C for more criteria). Agency Official Contract Files Missing Required Documents OAM COs and OTOP POs had inadequate records management based on FAR, EPAAR and the EPA policy. The official contract file and files maintained by each CO and PO were missing the following: Thirty-four of 48 monthly progress reports. A post award conference meeting document. Three of four CO annual contract invoice reviews. A financial monitoring review report and the CO's applicable responses. Initial or finalized QASPs. Six of eight letters sent to the contractor providing its performance ratings. In addition, the EPA did not centrally control the documents that did exist, as required by the FAR Subsection 4.802(d), as stated above. EPA's OAM procedures did not establish a requirement for a central repository. The EPA's various COs, POs and QAMs individually maintained selected documents. For example, the COs did not maintain the finalized QASP in the official contract file. The PO maintained the finalized QASP separately in their office computer. Of the 14 monthly progress reports received, the current CO provided only 11, one PO provided two, and another PO provided one. These monthly progress reports assist in determining whether the contractor met contract requirements. We asked the current CO for contractor performance evaluations, but he could not provide any. Two different COs provided six award fee evaluation reports. The evaluation reports assist in determining whether EPA complied with FAR and EPAAR requirements for measuring contractor performance. 13-P-0398 14 ------- The OIG has noted this problem with the agency's administration of its contracts during multiple audits. The EPA's OAM Balanced Scorecard Program also identified similar issues. Table 3 lists the OIG reports and the OAM document. Table 3: Related audit reports Report Finding EPA OIG 11-P-0705, EPA Contract Oversight and Controls Over Personal Computers Need Improvement We found similar internal control issues in terms of updating databases, retaining appropriate documentation, and maintaining permanent contract positions. EPA OIG draft report, Improvements Needed in EPA's Smartcard Program to Ensure Consistent Physical Access Procedures and Cost Reasonableness Staff did not assure adequate data maintenance and that incomplete contract files resulted from issues within both the project management and contract management offices. OAM Balanced Scorecard An internal audit identified contract documentation issues. Source: OIG analysis. The agency's inadequate record management persists because OAM and OTOP did not have adequate oversight of policies and procedures in place to ensure the easy retrieval of official documents during the many staff reassignments (see appendix B). OAM and OTOP procedures did not hold responsible personnel accountable for adequate records management during CO and PO personnel reassignments. The CO and PO did not perform an inventory of required documents to ensure the prior CO and PO maintained documentation. In addition, OAM and OTOP did not implement procedures to require the transference and maintenance of data in a centralized location or key locator system to identify where documents are located during CO and PO personnel changes. The FAR Subsection 4.802(d) states: "A central control and, if needed, a locator system should be established to ensure the ability to locate promptly any contract files." As a result of ineffective records management, files cannot be readily accessible to principal users who are required to facilitate and ensure the effective accomplishment of contract requirements and contract closeout. An overall analysis of all documented monthly progress reports and annual invoice reviews would have provided a better indication of the contractor achieving contract requirements. In addition, the EPA may be unable to make effective management decisions and provide reasonable assurance that the agency complied with applicable regulations. If the CO centrally maintained all contractor performance evaluations, the CO would have had past performance information readily available to ensure compliance with FAR and EPAAR (see chapter 2). Consequently, the EPA cannot reasonably make informed decisions based on missing documents and ensure the government got what it paid for. 13-P-0398 15 ------- Conclusion Without complete and organized contract files, the EPA is unable to make effective management decisions and provide reasonable assurance that the contractor met contract requirements and the EPA complied with applicable regulations. Although the contract has ended, the EPA needs to maintain adequate records to make effective and informed decisions on contractor past performance for contract closeout purposes and in the event of litigation or congressional inquiries. Since the agency continues to acquire those same services, records would provide needed lessons learned to improve management of the new contracts. Recommendations We recommend that the assistant administrator, Office of Administration and Resources Management: 5. Develop OAM procedures to require that, before there are personnel reassignments, COs, POs and QAMs perform an inventory of required documents in their possession or in the official contract file to ensure maintenance of accurate and complete records. 6. Update internal procedures and programs to include a process to transfer all required documents to the newly appointed COs and POs or establish a key locator system identifying the location of required contracting documents. Agency Comments and OIG Evaluations OARM concurred with recommendations 5 and 6, but the recommendations are unresolved pending receipt of corrective actions that correspond with the causes and recommendations. OARM stated during the exit conference that issues found and addressed under recommendations 5 and 6 should relate to a management oversight issue, yet we found no EPA procedures in place to direct the actions of record management when there are staff reassignments or new appointments. The criteria addressed in appendix C does not address the local procedures when there are CO, PO and QAM reassignments and new appointments. In addition, this is a recurring issue because prior audits recommended improvements in records management that the agency agreed to yet the conditions still exist (see table 3 above). As a result, we concluded that this further direction is needed. 13-P-0398 16 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Action Official Update the policies to include procedures that would ensure that, for all incentive and time and material contracts, contract performance metrics and standards include cost outcomes. Planned Completion Date Claimed Amount Ag reed-To Amount Assistant Administrator, Office of Administration and Resources Management 12 Update internal programs that provide oversight and accountability to ensure that, for all incentive type and time and material contracts, the EPA performance metrics and standards link to cost outcomes. Assistant Administrator, Office of Administration and Resources Management 12 Require OTOP, in conjunction with OAM, to review contractor performance reports generated per the QASP. Require OTOP to coordinate all actions taken with the COs and POs and require the COs to input contractor performance information into CPARS. Assistant Administrator, Office of Administration and Resources Management 12 Require COs to input interim reports into CPARS prior to their reassignments. 16 Develop OAM procedures to require that, before there are personnel reassignments, COs, POs and QAMs perform an inventory of required documents in their possession or in the official contract file to ensure maintenance of accurate and complete records. Assistant Administrator, Office of Administration and Resources Management Assistant Administrator, Office of Administration and Resources Management 16 Update internal procedures and programs to include a process to transfer all required documents to the newly appointed COs and POs or establish a key locator system identifying the location of required contracting documents. Assistant Administrator, Office of Administration and Resources Management 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is unresolved with resolution efforts in progress 13-P-0398 17 ------- Appendix A Contract Performance Requirements Summary (Performance Metrics and Standards) Applicable performance work statement Performance Frequency of sections/section # ID Metric standard Surveillance method surveillance 1 Percentage of all requests 90% Monitoring monthly data As needed responded to and routed within 15 minutes of submission to the Service Request Interface during core hours 2 Percentage of all requests 99% Monitoring monthly data As needed responded to and routed within 60 minutes of submission to the SRI during core hours Service request 3 Percentage of tickets correctly 75% Monitoring monthly data Monthly interface routed or escalated on initial and Customer Support (Section 2.1.1) attempt Representative input 4 Percentage of tickets correctly 95% (months Monitoring monthly data Monthly routed or escalated on second 05-28) and Customer Support attempt 98% (months Representative input 29-100) 5 Percentage of end-user requests 98% Monitoring monthly As needed (requiring more than 1 business sample day to resolve) receiving daily updates 6 Percentage of assets accurately 98% Planned sample: cross Periodic identified in tracking system reference random sample of assets with tracking system; cross reference random sample of tracking system with assets 7 Percentage of unaccounted assets 99% Planned sample: audit Periodic accurately accounted for and sampling of missing recorded in tracking system or assets records with Asset management identified as missing within 30 information in the days of previous surveillance Remedy system and (Section 2.1.2) check Ebusiness 8 Percentage of accurate updates 90% Evaluate moves, adds, Monthly made in tracking system within and changes cross- 1 business day of change being referenced against date made of update made to the asset tracking system 9 Percentage of accurate updates 99% Evaluate moves, adds Monthly made in tracking system within and changes cross- 2 business days of change being referenced against date made of update made to the asset tracking system 10 Percentage of accounts in 90% Monitoring of monthly Monthly compliance with the EPA policy data provided by Service and standards Provider Account access and management Monitor monthly Bind (Section 2.1.2.3) view reports 11 Percentage of accounts 99% Monitoring of monthly Monthly established or routed to proper data provided by Service authority within 4 hours Provider 13-P-0398 18 ------- Applicable performance work statement sections/section # ID Metric Performance standard Surveillance method Frequency of surveillance 12 Percentage of systems that are patched and have latest virus definition files by CSIRC required date 90% Monthly inspection of data Monthly 13 Percentage of systems that are patched have latest virus definition files within 10 business days of CSIRC required date 99% Monthly inspection of data Monthly 14 Percentage of incidents that are reported to ISO and contracting officer technical representative within 4 hours of incident 90% Random sampling Monthly Security (Section 2.1.3) 15 Percentage of incidents that are reported to ISO, CSIRC and COTR within 2 days of incident 99% Random sampling Periodic 16 Percentage of incidents of personally identifiable information loss that are reported to ISO, CSIRC and COTR within 1 hour of incident 90% Random sampling Periodic 17 Percentage of incidents of personally identifiable information loss that are reported to ISO, CSIRC and COTR within 4 hours of incident 99% Random sampling Periodic 18 Percentage of contractor staff who complete required training on time 95% Random Monitoring Once a year 19 Percentage of hardware installs scheduled within 4 hours of submission of approved request to the SRI for standard desktops/laptops or upon receipt of other hardware 80% Review monthly data from SP and validation through analysis of data from various sources Monthly Hardware provisioning (Section 2.2.1.1) 20 Percentage of hardware installs scheduled within 2 business days of submission of approved request to the SRI for standard desktops/ laptops or upon receipt of other hardware 99% Review monthly data from SP and validation through analysis of data from various sources Monthly 21 Percentage of hardware installed by the scheduled install date and time; includes loaner hardware 90% Review monthly data from SP and validation through analysis of data from various sources Monthly 22 Percentage of hardware installed within 1 business day of schedule date and time; includes loaner 99% Review monthly data from SP and validation through analysis of data Monthly hardware from various sources 13-P-0398 19 ------- Applicable performance work statement sections/section # ID Metric Performance standard Surveillance method Frequency of surveillance 23 Percentage of software installs scheduled within 4 hours of submission of approved request to the SRI for standard software or upon receipt of other software 80% Review monthly data from SP and validation through analysis of data from various sources Monthly Software provisioning (Section 2.2.1.2) 24 25 Percentage of software installs scheduled within 2 business days of submission of approved request to the SRI for standard software or upon receipt of other software Percentage of software installed by the scheduled install date and time 99% 90% Review monthly data from SP and validation through analysis of data from various sources Review monthly data from SP and validation through analysis of data from various sources Monthly Monthly 26 Percentage of software installed within 1 business day of schedule date and time 99% Review monthly data from SP and validation through analysis of data from various sources Monthly 27 Percentage hardware collected and removed from desk within 1 business day of submission of approved request to the SRI 90% Review monthly data from SP and validation through analysis of data from various sources Monthly Deprovisioning (Section 2.3) 28 Percentage hardware collected and removed from desk within 2 business days of submission of approved request to the SRI 99% Review monthly data from SP and validation through analysis of data from various sources Monthly 29 Percentage of accounts deactivated/disabled on scheduled date 99% Review monthly data from SP and validation through analysis of data from various sources Monthly Operations (Section 2.4) 30 31 Percentage of systems in compliance with the EPA policy and standards Percentage of network printers fully operational 95%(months 05-16) 98%(months 17-100) 98% Compare sample of selected CTS desktop configurations to the EPA standard configuration to ensure compliance Sampling of printer data Monthly Monthly 32 Percentage of preventative maintenance schedules met 90% Inspection of reports Periodic 33 Percentage of password resets completed within 15 minutes after notification of request 90% Sampling monthly data from remedy Periodic 34 Percentage of password resets completed within 60 minutes after notification of request 99% Sampling monthly data from remedy Periodic User support (Section 2.4.3) 35 Percentage of hardware and software issues resolved or reprovisioned within 4 hours of submission of request for service to the SRI 90% Review monthly data from SP and validation through analysis of data from various sources Monthly 36 Percentage of hardware and software issues resolved or reprovisioned within 2 business days of submission of request for service to the SRI 99% Review monthly data from SP and validation through analysis of data from various sources Monthly 13-P-0398 20 ------- Applicable performance work statement Performance Frequency of sections/section # ID Metric standard Surveillance method surveillance 37 Percentage of moves, changes, and upgrades completed within 90% Review monthly data from SP and validation Monthly Moves, changes, 4 hours of scheduled time through analysis of data from various sources and upgrades (Section 2.4.3.3) 38 Percentage of moves, changes, 99% Review monthly data Monthly and upgrades completed within 2 business days of scheduled time from SP and validation through analysis of data from various sources 39 Percentage of reports delivered in accordance with reporting 90% Monitoring submission dates and quality of Monthly Reporting schedule reports (Section 3.5) 40 Percentage of reports delivered within 5 business days of due date 99% Monitoring submission dates and quality of reports Monthly 41 Percentage of customer satisfaction surveys rated at an average overall score of 4.0 or higher (scale of 0 - 5 with 5 being highest) 95% Review sampling of surveys Monthly 42 Percentage of IT training class evaluations rated at an average 95% Review sampling of surveys Monthly Customer satisfaction overall score of 4.0 or higher (scale of 0 - 5 with 5 being highest) 43 Percentage of requests properly completed at first attempt 95% Monitoring of monthly data sample and Customer Support Representative input Monthly 44 Percentage of requests properly completed at second attempt 99% Monitoring of monthly data sample and Customer Support Representative input Monthly * Note: All 44 metrics relate to quality; Metrics 1,2,5, 7-9, 11,13-29, 32-40 relate to timeliness; and Metrics 41-44 relate to business relations. Source: CTS Contract Modification 6. 13-P-0398 21 ------- Appendix B PO, CO and QAM Reassignments on CTS Contract A IV J J A S O N D J F IV A IV j j A S 0 N D J F IV A M J j A S O N D J F IV A IV J J A s 0 N D J F IV A IV J J A S p 0 1 5/1/08 -10/8/08 2 10/8/08 - 3/11/09 3 3/11/09 - 12/23/09 4 12/23/09 - 6/21/11 1* 6/21/11 - 9/30/12 A IV J J A S O N D J F IV A IV j j A s 0 N D J F IV A M j j A S O N D J F IV A IV J J A s 0 N D J F IV A IV J J A S c 0 1 4/1/08 - 3/11/09 2 3/11/09 - 6/2/10 3 6/2/10 -11/14/11 4 11/14/11 - 9/30/12 A IV J J A S O N D J F IV A IV j j A s 0 N D J F IV A M j j A S O N D J F IV A IV J J A s 0 N D J F IV A IV J J A S Q A M vacant 1 9/1/08 - 3/31/11 2 7/3/11 -10/31/11 vacant 3 4/8/12 - 8/5/12 2008 2009 2010 2011 2012 *This CO is the same person but serviced during a different period of time. Source: OIG analysis. 13-P-0398 22 ------- Appendix C Relevant Recordkeeping Criteria The following table illustrates the criteria for government contract records. Regulation/policy/ instruction/guidance Requirement/suggestion/recommendation FAR Subsection 46.104 (c) Maintain, as part of the performance records of the contract, suitable records reflecting (1) the nature of government contract quality assurance actions, including, when appropriate, the number of observations made and the number and type of defects; and (2) decisions regarding the acceptability of the products, the processes, and the requirements, as well as action to correct defects. EPAAR Subsection 1542.1503 (f) Copies of the evaluation, contractor response, and review comments (if any) shall be retained as part of the evaluation, and hard copies shall be contained in contract files. EPA Classification No. CIO 2155, EPA Records Management Policy, paragraph 8(h)(2)(i)1&3 "All agency employees are responsible for (1) creating and managing the records necessary to document the agency's official activities and actions, including those records generated by the EPA contractors and grantees, in accordance with the EPA recordkeeping requirements;.... (3) filing records for safe storage and efficient retrieval and maintaining personal papers and non-record materials separately from official agency records." EPA Records Management Manual, Chapter 3 After creating or receiving a record, the EPA staff must capture it by filing, storing or otherwise systematically maintaining it in a recordkeeping system. The EPA offices must capture records in a recordkeeping system that facilitates maintenance and use of the records in an efficient and cost-effective manner. EPA CMM, paragraph 7.3.5.5(E) Contract files must document the contractor's performance to assist the agency in future procurements. EPA CMM, Section 11.2, NOTE (11) A properly documented file should be kept by both the PO/Delivery Order PO and work assignment manager, along with copies of invoices reviewed and approval documents. Relevant work assignment manager issues should also be communicated and filed in the PO file. The CO should file formal correspondence in the contract file. EPA CMM, Section 11.2, NOTE (15) The Research Triangle Park Finance Center maintains the official file documenting invoice payments. The PO/Delivery Order PO and CO maintain secondary files which generally include added data supporting payment decisions, suspensions and disallowances. EPA CMM, Section 11.2, paragraph 11.2.5.2 (E) - Filing/Recordkeeping (1) POs should update their financial records to reflect information concerning the invoice, the amount paid, and the account used; and (2) both the PO and work assignment manager should maintain files of approved invoices and all associated documentation. These files will eventually be sent to the CO at the completion of the contract. The PO should consolidate PO/work assignment manager invoice files before they are sent to the CO. 13-P-0398 23 ------- Regulation/policy/ instruction/guidance Requirement/suggestion/recommendation EPA CMM, Appendix 42.1, (4) COR Work plan (a) Set-up a file system containing all relevant documentation including the basic contract, list of contracting officer representatives under the contract, all correspondence and meetings related to the contract, technical direction, contract deliverables received and reviewed, payment file and other items that will provide an audit trail of the contract-level contracting officer representative's actions under the contract. Maintain files in accordance with agency National Records Management Program policy. Guidance on maintaining the EPA Series 202, Contract Management Records, is available on the Intranet at httD://www.eDa.aov/records/Dolicv/schedule/sched/202.htm. EPA Record Series, Schedule 202, NARA Disposition No. N1 -412-06- 6/5, Contract Management Records COs and contracting officer representatives should work together to determine who is responsible for maintaining specific documents to minimize duplication while still providing an adequate audit trail. Source: OIG analysis of EPA data. 13-P-0398 24 ------- Appendix D Agency Comments on Draft Report #% mm STATU EflfliQIIlWlL PROTECTION AGENCY WASHINGTON DC 20*69 *.ฆ .;'i ฆ , ...".ni . * H SB the Customer Tmbvioig/ Selulimm Cmlmi i--s ซ*' v.-P 'VvflfMltt| OifaoflitlKirititfli! The Office of Administration and Resources Management and the Office of Environmental Information have reviewed the draft OIG audit report OA-FY12-0494 and provide the following comments and corrective actions in response to subject audit findings and recommendations. 13-P-0398 25 ------- General Comments on Report Page 2 - The OIG writes, "In April 2008, the EPA awarded an indefinite delivery/indefinite quantity hybrid contract. This hybrid contract also contained provisions for performance-based incentive fees and fixed hourly rates and material costs for a time and materials contract. This contract required the contractor to provide technology services to approximately 12,000 program office users scattered throughout 16 states and the District of Columbia." Contract EP-W-08-034 was a "Seat Service" contract which refers to all the desktop hardware and software required for EPA employees to perform work responsibilities. In the contract, the EPA estimated the maximum number of seats that may be required by the agency was 12,000, and the contractor proposed a monthly fixed price of $169.75 per seat to ensure all EPA employees - up to 12,000 - were provided with desktop hardware and software. Ninety percent of contract spend used this pricing methodology - the Government both ordered and agreed to pay a fixed amount per seat per month, thus accepting the pricing risk in the event the estimated 12,000 seat calculation was understated, and the contractor accepted the pricing risk in the event the 12,000 seat calculation was overstated. The remaining 10% of the contract price was for labor categories proposed by the contractor to support various potential contract tasks, such as Disaster Recovery and Emergency Response Support and Specialized Projects relating to the PWS, and other ancillary services that the EPA required in support of seat service. These additional services were procured via task orders, under which the EPA and the contractor negotiated the hours needed to perform the required work using the labor categories and fixed rates set forth in contract. For this effort, the hours were negotiated but the Government agreed to pay the fixed hourly rates. Given the above-described contract terms and conditions, the OIG's characterization of the contract as "provide technology services to approximately 12,000 program office users" is a misleading description of a contracting and pricing arrangement. OIG Evaluation. This report included statements obtained from and supported by the audited contract. EPA awarded an Indefinite Delivery/ Indefinite Quantity contract that issued task orders for supplies and services with time and materials provisions that the fixed rates include wages, overhead, general and administrative expenses and profit. According to the FAR Subsection 16.600, time and materials contracts are not fixed price contracts and Subsection 16.601 states fixed hourly rates are supposed to be specified. Based on our review of this contract, modifications (mods) indicated changes to the rates, hours and user requirements, so the possibility that fixed rates were used and the overall cost would not change appear to be misleading. Below shows changes to rates per seat: Year 1 Year 2 Year 3 Year 4 Contract $169.75 $139.27 $139.43 $140.13 Task Order 2 $175.00 (mod 4) $147.28 (mod 8) $146.70 (mod 15) $147.28 (mod 23). If the overall cost is determined based on the changing fixed rate multiplied by the seats, the costs will reflect upward and downward revisions to the stated overall contract price. With these changes, this contract is not a firm fixed price contract and cost controls are needed. FAR Subsectionl6.601 states that for a time and materials contract, appropriate government surveillance of contractor performance is required to give reasonable assurance that efficient methods and effective cost controls are being used. 13-P-0398 26 ------- Page 8: The OIG writes "By not linking award fee to cost outcomes as required, the EPA did not effectively hold the contractor accountable for cost inefficiencies and inaccuracies.. .Even though the agency did not award the contractor an incentive fee, there could have been additional corrective actions implemented by having cost outcomes tied to contracting activity." Pursuant to the contract terms and conditions, contractor performance was evaluated against contract performance metrics, and the contractor was held accountable when performance against those metrics did not meet the level required for payment of incentive fees. Under the afore-mentioned contract terms and conditions, performance was incentivized and the contractor was not awarded an incentive fee when performance did not meet contract metrics. Accordingly, contract incentives worked exactly as designed and intended in accordance with contract terms and conditions. Under a fixed price arrangement, controlling costs is inherent in this contract type and is therefore, not incentivized. The contractor has to deliver the services at the fixed rate in the contract or absorb any additional costs associated with the services. OIG Evaluation. According to the FAR Subsection 16.202, a firm fixed price contract provides for a price that is not subject to any adjustment on the basis of the contractor's cost experience in performing the contract. A firm fixed price contract type places upon the contractor maximum incentive for the contractor to control costs and can use award fee or incentives based on factors other than costs. There is nothing in the audited contract that states this is a fixed price contract. EPA awarded an Indefinite Delivery/ Indefinite Quantity contract that issued task orders for supplies and services with time and materials provisions that the fixed rates include wages, overhead, general, and administrative expenses, and profit. FAR Subsection 16.601 states that a time and materials contract provides no positive incentive to the contractor for cost control or labor efficiency. Therefore, appropriate government surveillance of contractor performance is required to give reasonable assurance that efficient methods and effective cost controls are being used. Since this contract has an award fee plan, FAR Subsection 16.401 9(e)(2) states award shall not be earned if the overall cost, schedule and technical performance is not at a satisfactory level. Responses to the Recommendations Recommendation 1: Update the policies and procedures to include controls that would ensure contract performance metrics and standards include cost outcomes for all incentive and time and materials contracts. Recommendation 2: Update internal programs that provide oversight and accountability to ensure that, for all incentive type and time and materials contract, the EPA performance metrics link to cost outcomes. QAM Response: OAM disagrees with these recommendations and disagrees with the OIG's recommendation that cost is an appropriate incentive for all incentive and time and materials contracts. EPA's CTS Contract EP-W-08-034 is an Indefinite Delivery/Indefinite Quantity type contract with time and materials pricing, under which the contractor may earn an incentive fee for performance that exceeds metrics set forth in the contract. Attachment 11 of the contract contains contract pricing. Under the Attachment 11 Base Period, 90% of the contract price is for standard seat service, "Seat Service" refers to all the desktop 13-P-0398 27 ------- hardware and software required for EPA employees to perform work responsibilities. In the contract, EPA estimated the maximum number of seats that may be required by the agency was 12,000, and the contractor proposed a monthly fixed price of $169.75 to ensure all EPA employees - up to 12,000 - are provided with desktop hardware and software. Again 90% of contract pricing used this methodology. The other pricing under Attachment 11 Base Period, which totals 10% of the contract price, is for labor categories proposed by the contractor to support various potential contract tasks, such as Disaster Recovery and Emergency Response Support and Specialized Projects relating to the PWS, that the EPA may require in support of seat service. These additional services were procured via task orders, under which EPA and the contractor negotiated the hours needed to perform the required work using the labor categories and fixed rates set forth in Attachment 11. For this effort, the hours were negotiated but the rates were fixed. Attachment 6 of the contract sets forth performance standards/metrics for seat service and other tasks against which contractor performance is measured for the purpose of determining the award fee. The performance standards/metrics identified are specifically tied to Performance Work Statement tasks the contractor is performing to ensure all EPA employees have the necessary desktop hardware and software to perform their work responsibilities. As such, the criteria set forth in Attachment 6 of the contract is designed to measure critical contract performance - timeliness, responsiveness, system security, user support, and customer satisfaction. Attachment 12 of the contract prescribes how the contractor will be paid an incentive fee for elevated levels of performance, for example, if the performance metric is met at 100%, the vendor receives 100% of the incentive fee, if met at 75%, the vendor receives 75%, etc. The final fee is calculated against the Attachment 6 metrics, and paid against the Attachment 11 pricing. OAM believes that writing a policy that "includes cost outcomes for all incentive and time and materials contracts" is not necessary. Additionally, FAR Subpart 16.1 states selecting the contract type is a matter for negotiation and requires the exercise of sound judgement. Factors that should be considered in this negotiation are risk, complexity of the requirement, expected price competition, urgency of the requirement, period of performance of the requirement (the longer the more risk), and the contractor's technical capability and financial responsibility. The FAR also states that the objective is to negotiate a contract type and price that will result in reasonable contractor risk and provide the contractor with the greatest incentive for efficient and economical performance. In the above pricing arrangement, the number of seats was controlled by the Government, and the price per seat was fixed, so the only risk to the Government was with contractor performance. FAR Subsection 16.402 states that performance incentives may be considered in connection with specific product characteristics (e.g., a missile range, an aircraft speed, an engine thrust, or vehicle maneuverability) or other specific elements of the contractor's performance. These incentives should be designed to relate profit or fee to results achieved by the contractor, compared with specified targets. OAM incentivized contractor performance characteristics in the areas of timeliness, responsiveness, system security, user support, and customer satisfaction, and fee was awarded based upon the contractor's performance against metrics set forth in these areas. Regardless of the amount of fee awarded against these metrics, the vendor was entitled to collect the fixed price per seat set forth in the contract. Under the 13-P-0398 28 ------- above pricing structure, OAM properly assessed contract risk and established incentive fee metrics to mitigate and shift the burden of that risk to the contractor. OAM does not agree with writing a policy requiring cost be an incentive for all incentive and time and materials contracts. OIG Evaluation. The OAM response does not sufficiently address recommendations 1 and 2. OAM states OIG is recommending cost as an appropriate incentive, yet we are emphasizing FAR Subsections 16.401, 16.402, and 16.601, and OMB requirements that, for incentive (contracts with an award fee) and time and material contracts, agencies link cost outcomes to performance standards and metrics. Also, OAM's response is stating they have the right to determine contract type when the issue is with the FAR additional requirements for the contract types already selected. To ensure the EPA is complying with FAR and OMB requirements addressed above, the EPA policies/procedures should require that for incentive (contracts with an award fee) and time and material contracts performance standards and metrics are linked to cost outcomes. In addition for recommendation 2, per OMB Circular A 123, agencies are responsible for establishing internal controls (policies and procedures) to achieve compliance with applicable laws and regulations and should design management structures that help ensure accountability for results. Recommendation 3: Require OTOP, in conjunction with OAM, to review contractor performance reports generated per the QASP. Require OTOP to coordinate all actions taken with the COs and PO's and require the CO's to input contractor performance information into CPARS. Recommendation 4: Require CO's to input interim reports into CPARS prior to their reassignments. OAM Response: OAM agrees with these recommendations. With regard to the CTS contract, OAM has been working with the program office to complete the required past performance evaluation(s). The contract was registered and interim reports initiated in CPARS on Feb 1, 2013. Evaluation narratives were drafted by OAM based on information obtained from the award-fee assessments; and finalization of these reports is pending upon additional input from the program office (OEI). With regard to overall past performance reporting, OAM believes adequate policy already exists in the FAR and EPAAR with regard to reporting requirements. As part of the OAM Balanced Scorecard Performance Measurement and Management Peer Review and Oversight Process, OAM reviews individual office's procedures for monitoring Contractor Performance Assessment Reporting System and evaluates whether the office has an effective oversight program. Furthermore, in March 2013, OFPP began monitoring CPARS compliance and required all federal agencies to establish compliance goals to be monitored both internally and externally. In order to monitor and oversee both internal and external past reporting initiatives, OAM designated a CPARS lead to monitor agencywide past performance reporting compliance on a monthly basis. The lead ensures compliance by comparing PPIRS reports with the CPARS database and disseminates the results to all agency CPARS focal points and their managers. Beginning in February 2013, the CPARS representative is requiring Agency focal points report status on any incomplete CPARS. Under the above described internal OAM compliance program, OAM has already improved compliance by 15% and anticipates continual 13-P-0398 29 ------- improvement over the next few months. Additionally, as of June 2013, OAM appears to be on target to meet our Agency CPARS compliance goal of 75 percent by the end of FY 2013. Based upon EPA/OAM's established, aggressive and continuous CPARS compliance oversight and monitoring toward achieving and reporting against external goals, and given CPARS monitoring is a recognized ongoing effort satisfies recommendation 4 and OARM considers this recommendation closed. OIG Evaluation. Even though the agency agreed to recommendations 3 and 4, the comments are not completely responsive to the recommendations. The comments should address requiring that COs put interim reports into CPARS prior to their reassignments. This additional procedure will assist the agency to ensure that the contractor performance reports are entered into CPARS. The response did not address how or when OARM would implement the agreed-to recommendation to require COs input interim evaluation prior to reassignments in CPARS. This should be addressed as part of the 60-day response to the final report. According to EPA Manual 2750, if the OIG determines that the agency's intended corrective actions meet the intent of the recommendations, the OIG reports recommendations as resolved in the OIG's audit tracking system. Recommendation 5: Develop OAM procedures to require that before there are personnel reassignments, CO's, PO's, and QAM's perform an inventory of required documents in their possession or in the official contract file to ensure maintenance of accurate and complete records. Recommendation 6: Update internal procedures and programs to include a process to transfer all required documents to the newly appointed CO's and PO's or establish a key locator system indentifying the location of required contracting documents. OAM Response: OAM agrees with these recommendations although regarding the need for policy updates with regard to adequate file documentation, OAM believes sufficient internal and external policy and procedures with regard to contract documentation requirements already exists in the FAR, the Contract Management Manual, and the Acquisition Handbook. OAM also believes that meaningful oversight and review processes are necessary to ensure compliance with the applicable statutes, regulations, policies and procedures. Accordingly, OAM's Balanced Scorecard Performance Measurement and Management Program consists of self-assessment and peer review oversight components that utilize a significant file review component to assess compliance with statues, regulations, policies, and procedures as documented in the contract. Under this approach, contract file documentation adequacy is internally reviewed by each division, and also independently by an independent team of senior level contracting professionals. Since program implementation in FY 2012, OAM has been assessing the quality and compliance of EPA's acquisition system agency-wide, and using review results to identify acquisition system weaknesses, and correct problems areas through training and/or implementation of policy and process guidance. In peer reviews to date, the need for improved contract file documentation has been identified as a systems issue finding, and as a result, individual organizations have developed corrective action plans and OAM has been including documentation requirements under various comprehensive new acquisition policies on Acquisition Planning and the CPARS. 13-P-0398 30 ------- In addition, Acquisition Handbook, Section 4.1, is being updated to prescribe requirements for streamlined internal controls (e.g., higher-level transactional reviews), to ensure the sufficiency and quality of file documentation in support of support contracting officer decisions and actions and an updated standardized checklist for contract file documentation. Under the BSC PMMP program, all OAM staff are accountable for the quality and health of the EPA acquisition system. Information on OAM's BSC PMMP Program may be found on the OAM intranet under the "Balanced Scorecard" tab on the left. Based upon EPA/OAM's established and aggressive BSC PMMP compliance oversight and monitoring program, OAM requests the above described ongoing actions be considered and these recommendations be closed. OIG Evaluation. The agency agreed with the recommendations 5 and 6. However, the agency's comments are not completely responsive to the recommendations. The recommendations discussed procedures as opposed to policy and OAM procedures are necessary to provide details on what should be done to ensure maintenance of accurate and complete contract records. The policies OAM mentioned do not address procedures needed when there are CO, PO and QAM reassignments and new appointments. Procedures to perform an inventory of the required contract documents would assist in ensuring maintenance of accurate and complete records. In addition, procedures are needed to include a process to transfer all required documents to the newly appointed COs and POs or establish a key locator system indentifying the location of required contracting documents. According to EPA Manual 2750, if the OIG determines that the agency's intended corrective actions meet the intent of the recommendations, the OIG reports recommendations as resolved in the OIG's audit tracking system. Should you have any questions regarding the OEI issues, please contact Scott Dockum at (202) 566-1914; any questions regarding OAM's responses can be directed to Rebecca Snipe at (202) 564-8346. 13-P-0398 31 ------- Appendix E Distribution Office of the Administrator Assistant Administrator for Administration and Resources Management Assistant Administrator for Environmental Information and Chief Information Officer Agency Follow-Up Official (the CFO) Agency Follow-Up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for External Affairs and Environmental Education Director, Office of Acquisition Management, Office of Administration and Resources Management Director, Office of Technology Operations and Planning, Office of Environmental Information Audit Follow-Up Coordinator, Office of Administration and Resources Management Audit Follow-Up Coordinator, Office of Environmental Information 13-P-0398 32 ------- |