'"
OFR Or: OF 1 NSPECTOR GEN FLRAI,
^
Audit Report
Response Action Contracts:
Structure and Administration
Need Improvement
Report No. 2005-P-00001
December 6, 2004
-------
Report Contributors:
Stephen Burbank
Heather Layne
Denton Stafford
Charles Watts
Myra Mackall
Michael Petscavage
Abbreviations
CPAF Cost Plus Award Fee
EPA Environmental Protection Agency
FAR Federal Acquisition Regulation
ID/IQ Indefinite Delivery/Indefinite Quantity
LoE Level-of-Effort
NIH CPS National Institutes of Health Contractor Performance System
0AM Office of Acquisition Management
OARM Office of Administration and Resources Management
OIG Office of Inspector General
OSRTI Office of Superfund Remediation and Technology Innovation
OSWER Office of S olid Waste and Emergency Response
PBSA Performance-Based Service Acquisition
RAC Response Action Contract
RACMIS Remedial Action Contract Management Information System
USACE U.S. Army Corps of Engineers
-------
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
2005-P-00001
December 6, 2004
C,:,,:,:,:_-
g^^^^piis^ip^^ll^
;| ^«^M^^i|?paBi»^:^ :vi> !:-!f'f
^ft^;^a^ii^||^;jj^^:?s
»* .* >s'v'y -XxliM-i-:- -it:': .;':"-" ^:'*>":>*::i:i::t':":':":::':'.V: ~-.*iifi:K.~X''-:
ext
Response Action Contracts: Structure and Administration Need Improvement
What We Found
EPA can improve the structure of RACs to better protect the Government's
interests. Current RACs, which are Cost Plus Award Fee Level of Effort
contracts, assign to EPA a disproportionate share of the risk of cost overruns;
expose EPA to the risk of loss of funds through litigation; limit competition; and
forego potential cost savings associated with other approaches to contracting, such
as Performance-Based Service Acquisition.
EPA regions do not consistently document the rationale used to decide what
procurement option to utilize for Superfimd cleanup activities as required by
established policy. Further, EPA does not have a process to measure and
disseminate information on the U.S. Army Corps of Engineers' past performance
in support of EPA.
The Agency has measures in place to assess contractor performance at the work
assignment level. However, evaluations at the contract level were not being
documented timely and consistently, as required, because they were not given the
necessary priority. Not consistently documenting evaluations in a timely manner
does not permit EPA and other Federal agencies to consider contractors' past
performance and could be detrimental to contractors who have performed well.
Contract managers have, or can obtain, the information needed to evaluate results
and make decisions, but the information in the national automated database is not
always readily available. The Remedial Action Contract Management
Information System is underutilized by regional staff, and the system does not
collect national data as originally intended. As a result, EPA is expending
approximately $1.5 million a year on a system that is not being fully utilized.
What We Recommend
We recommend that the Office of Solid Waste and Emergency Response, in
coordination with the Office of Administration and Resources Management,
develop and implement a plan with milestones that will increase the use of
different contract types, require regional staff to document the rationale for all
source selection decisions, develop a method for holding Contracting Officers
accountable for conducting past performance evaluations timely and accurately,
and conduct a cost benefit analysis to determine whether the Remedial Action
Contract Management Information System should be retained.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
December 6,2004
MEMORANDUM
SUBJECT: Response Action Contracts: Structure and Administration
Need Improvement
Report No. 2005-P-00001
FROM: Robert Mitchell, Director for Contract Audits \Signed\
Office of Inspector General (242IT)
TO: Thomas P. Dunne, Acting Assistant Administrator
Office of Solid Waste and Emergency Response (510 IT)
David J. O'Connor, Acting Assistant Administrator
Office of Administration and Resources Management (3101 A)
This is our final report on the subject audit conducted by the Office of Inspector General (OIG)
of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe
problems the OIG has identified and corrective actions the OIG recommends. This audit report
represents the opinion of the OIG and the findings contained in this report do not necessarily
represent the final EPA position. Final determinations on matters in this audit report will be
made by EPA managers in accordance with established audit resolution procedures.
Action Required
In accordance with EPA Manual 2750, the Acting Assistant Administrator for the Office of Solid
Waste and Emergency Response, as the designated Action Official, is required to provide a
written response to the report within 90 calendar days of the report date. The Action Official
should include a corrective action plan for agreed upon actions, including milestones dates.
If you or your staff have any questions, please contact me at Mi tchell. Robert;giepa. go v. or
Stephen Burbank, Assignment Manager, at Burbank.Stephen;&epa.gov.
-------
Tables of Contents
At a Glance
Chapters
1 Introduction 1
2 Response Action Contracts Need to be Restructured 6
3 Procurement Decisions Not Adequately Documented 14
4 Past Performance Evaluations Not Documented Timely
and Consistently 18
5 Utilization of Remedial Action Contract Management
Information System Could be Improved 21
Appendices
A Schedule of Response Action Contracts Reviewed 23
B Agency Response 24
C Distribution 29
-------
Chapter 1
Introduction
Purpose
We conducted this audit to determine how efficiently and effectively the
Environmental Protection Agency (EPA) is administering Response Action
Contracts (RACs). RACs are used to obtain professional Architect-Engineer,
technical, and management services in support of EPA's Superfund cleanup
responsibilities. The current RACs, which expire between 2005 through 2009,
have a maximum potential value of more than $4 billion (see Appendix A.). The
Agency is planning to replace these contracts with a new set of contracts (RACII)
as they expire, which will have an estimated maximum potential value of $4
billion as well.
Our objectives were to answer the following questions:
Acquisition Planning: How are RACs structured and funded?
Source Selection: How does EPA decide with whom to contract (other
government agencies, private sector)? Does past performance inform the
source selection decision?
Contract Administration: Are there good measures for assessing contractor
performance?
Contract Information Systems: Do contract managers have the information
needed to evaluate results and make decisions?
Background
Congress enacted the Comprehensive Environmental Response, Compensation,
and Liability Act of 1980, commonly referred to as "Superfund," to address
threats to human health or the environment resulting from releases or potential
releases of hazardous substances.
EPA has primary responsibility for managing cleanup and enforcement activities
under Superfund. In the cleanup of Superfund sites, EPA uses a variety of
instruments (such as contracts, assistance agreements, and interagency
agreements) to obtain Superfund removal and remedial services. EPA assigns
work to RAC contractors, States, the U.S. Army Corps of Engineers (USAGE), or
the U.S. Bureau of Reclamation based upon factors such as:
Site Characteristics
Remedy Characteristics
Local/Public Interest
Experience/Regional Infrastructure
Capability/Capacity of Contractors
-------
Conflict of Interest
Unique Site Needs
RACs provide professional Architect-Engineer, technical, and management
services in support of EPA's remedial response, enforcement oversight, and non-
time critical removal activities at Superfund sites. Contract services performed
under RACs include site management, remedial investigation and feasibility
studies that define the nature and extent of contamination, engineering services to
design remedial actions, subcontracting for remedial actions, construction
management, and enforcement support. Services may also include technical and
management services supporting EPA's coordination and oversight of remedial
activities performed by a State, the USAGE, the U.S. Bureau of Reclamation, or
responsible parties identified in enforcement actions. Through this "one-
program" approach, where most services needed can be ordered under one
contract, EPA expected to reduce disputes between design and construction
contractors, promote program integration, and give the regions flexibility to
pursue various enforcement options.
EPA Headquarters offices and the regions are responsible for administering
RACs. In the Office of Solid Waste and Emergency Response (OSWER), the
Office of Superfund Remediation and Technology Innovation (OSRTI) is
responsible for overall management of the Superfund program. Within OSRTI,
the Contracts Management Branch provides leadership for Headquarters
Superfund contracts. OSRTI also provides planning, oversight, and support for all
regionally-administered and regional-support contracts, including RACs.
The Office of Acquisition Management (OAM), within the Office of
Administration and Resources Management (OARM), is responsible for the
policies, procedures, operations, and support of the Agency's procurement and
contracts management programs, from contract planning through closeout
Within each region, Remedial Program Managers, Contracting Officers, and
Project Officers administer RACs.
Scope and Methodology
We performed this audit from April 2003 to July 2004 in accordance with
Government Auditing Standards, issued by the Comptroller General of the United
States. Our audit included site visits to EPA Headquarters and four regions, as
well as attending the 2003 National Superfund Project Officer/Contracting Officer
Training Conference. We reviewed all 16 active RACs and data from EPA
Headquarters and Regions 1, 2, 5, 6, 7, 9, and 10.
-------
The regions were selected based upon the following criteria:
Contract capacity (maximum potential value).
* Number of sites on the National Priority List within each region.
Amount of unallocated obligations on contracts (bulk funding).
Use of Interagency Agreements (number and dollar amounts of agreements).
To determine how RACs are structured and funded, we interviewed staff in
Headquarters and the regions to gain an understanding of the planning, budgeting,
and funding process for RACs. We reviewed the Federal Acquisition Regulation
(FAR) Part 7, Acquisition Planning; the Contracts Management Manual; the
RACs Users Guide; the RACs Statement of Work; the EPA Acquisition
Regulation; and other pertinent documentation. We also interviewed staff at the
USACE. We did not perform work at the U.S. Bureau of Reclamation because
they receive a small percentage of Superfund money compared to the USACE.
To determine how EPA decides with whom to contract and whether past
performance informs source selection decisions, we interviewed headquarters
staff, Remedial Project Managers, Project Officers, and Contracting Officers. We
reviewed guidance issued to the regions on determining to whom to assign
cleanup work. We also reviewed Interagency Agreements documents; Interagency
Agreement Decision Memorandums; and an EPA report, "Evaluation of the
Performance of the Corps of Engineers in Support of EPA's Superfund Program,"
on the USACE's support of Superfund. We also reviewed EPA's implementation
plan to address the recommendations from the above report and reviewed, on a
test basis, the documentation for selected source decisions.
We interviewed OAM staff to identity the processes and systems the regions
should be using to assess contractor performance. We interviewed regional
contracting officers to determine the processes and systems actually being used.
We reviewed pertinent documentation such as the RACs Users Guide and Award
Fee Plans. We reviewed the FAR and EPA requirements pertaining to conducting
performance evaluations. We reviewed award fee and annual evaluations for
compliance with those requirements. We also tested the accuracy and timeliness
of performance evaluations mat were recorded in the National Institutes of Health
Contractor Performance System (NIH CPS) by comparing them to the source
documents.
To determine whether contract managers have the information needed to evaluate
results and make decisions, we conducted interviews in the regions and
Headquarters to determine the information needed and the information systems
used to administer the RACs, and assessed whether these systems readily provide
needed information.
-------
Prior OIG Reports
We reviewed the following Office of Inspector General (OIG) reports and
determined that the recommendations in these reports, as they pertained to this
audit, have been resolved:
Acquisition Management: OMB Requested Review of EPA Contracting,
Report No. E1SKF7-04-0037-7100301, issued September 30,1997.
Superfund: Superfund Interagency Agreements, Report No. 2001 -P-00011,
issued June 22, 2001.
Internal Control Structure
In planning and performing the audit, we reviewed management controls relating
to our objectives, and our concerns with the lack of controls or the effectiveness
of existing controls are discussed in the following chapters. We also examined
the Fiscal Year 2003 Federal Managers' Financial Integrity Act Annual Assurance
Letters issued to the then Acting EPA Administrator by the Principal Deputy
Assistant Administrator for OSWER and the Assistant Administrator for OARM.
Neither assurance letter recommended any material, Agency-level weaknesses nor
management challenges in the administration of RACs.
Compliance With Laws and Regulations
EPA complied with the laws and regulations (the FAR) pertaining to its efforts to
administer and manage RACs with the exception of the requirement for timely
input of past performance evaluations into the Nffl CPS as discussed in Chapter 4.
Needed process improvements are discussed in Chapters 2, 3,4, and 5.
Results of Review
EPA can improve the structure of the RACs to better protect the Government's
interests. The current RACs, which are Cost Plus Award Fee Level of Effort
contracts, assign to EPA a disproportionate share of the risk of cost overruns;
expose EPA to the risk of loss of funds through litigation; limit competition; and
forego potential cost savings associated with other contracting approaches, such as
Performance-Based Service Acquisition. Also, EPA regions do not consistently
document the rationale used to decide what procurement option to utilize for
Superfund cleanup activities, as required by established policy. The Agency has
measures in place to assess contractor performance at the work assignment level,
but evaluations at the contract level were not being documented timely and
consistently as required. Contract managers have, or can obtain, the information
needed to evaluate results and make decisions, but it is not always readily
-------
available; the Remedial Action Contract Management Information System is
underutilized by regional staff, and the system does not collect national data as
originally intended.
We recommended that OSWER, in coordination with OARM, develop and
implement a plan for RAC II that increases use of Indefinite Delivery/Indefinite
Quantity, task order and site specific contracts. We also made recommendations
to review and revise the recertification policy, issue clarifying guidance on the
"Superfund Policy for Assigning Remedial Work to the US Army Corps of
Engineers (USAGE)," provide training and hold Contracting Officers more
accountable for evaluating contractor performance, and complete the proposed
cost benefit analysis to determine whether the Remedial Action Contract
Management Information System should be retained.
We held an exit conference with the Agency on November 17, 2004. EPA
provided written comments to our draft report on October 29, 2004. Where
appropriate, we made revisions to our report or incorporated the Agency's
comments and our evaluation of EPA's comment. We have included EPA's
complete response as Appendix B.
-------
Chapter 2
Response Action Contracts Need to be Restructured
EPA can improve the structure of RACs to better protect Government interests,
The current RACs, which are Cost Plus Award Fee (CPAF) Level of Effort (LoE)
contracts, assign to EPA a disproportionate share of the risk of cost overruns;
expose EPA to the risk of loss of funds through litigation; limit competition; and
forego potential cost savings associated with other approaches to contracting, such
as Performance-Based Service Acquisition (PBSA). Some Agency officials
maintain that the CPAF-LoE contracts offer the following advantages: their
flexibility allows the Agency to devote resources to other sites when warranted;
they are funded in a way that allows the Agency to continue work when
appropriations are delayed; and they prevent managers from having to return
funds to Headquarters for recertification. While there are benefits associated with
the current contract type, there are better contracting approaches available that
should, among other things, reduce EPA's cost risk and risk of loss through
litigation.
Options Available to Reduce EPA's Risks
EPA should utilize available historical data to move away from CPAF LoE
contracts. The services obtained under the RACs have been performed for EPA
for more than 20 years and EPA has extensive historical data to draw upon. In
addition, the nature of cost reimbursement contracts requires EPA to monitor
contractor performance closely and places more risk for cost and performance on
the Government.
According to FAR 16.103(a), selecting the contract type is generally a matter for
negotiation. It requires Contracting Officers to exercise sound judgment and
avoid protracted use of a cost-reimbursement or time-and-materials contract after
experience provides a basis for firmer pricing. The objective is to negotiate a
contract type and price (or estimated cost and fee) that will result in reasonable
contractor risk and provide the contractor with the greatest incentive for efficient
and economical performance.
FAR 16.104(d) and (k) state that complex requirements, particularly those unique
to the Government, usually result in greater risk assumption by the Government.
As a requirement recurs or as quantity production begins, product descriptions or
descriptions of services to be performed can be defined more clearly, the cost risk
should shift to the contractor, and a fixed-price contract should be considered.
-------
There are various types of contracts available for use by EPA that offer
advantages over the present contract type. Regions 6, 7, and 10 have decided to
move away from CPAF work assignment contracts and to Indefinite Delivery/
Indefinite Quantity (1D/IQ) task order contracts for the next round of RACs. Task
order contracts offer the following benefits:
Task orders can be ceiling price (best effort) or fixed price (completion).
Funds are obligated by task order and cannot be used to fund another task
order without deobligation first.
Cost recovery is easily identified to a site because of site specific accounting
and reporting.
An ID/IQ contract is a contract for services that does not procure or specify a firm
quantity of service and that provides for the issuance of orders for the
performance of tasks during the period of the contract. Benefits of ID/IQ
contracts include:
Providing flexibility in both quantities and delivery scheduling.
Limiting the Government's obligation to the minimum quantity specified in
the contract.
Offering the ability to place term or completion type orders, cost
reimbursement, or fixed priced orders.
Providing funding flexibility (funding in increments).
Providing opportunity and potential for costs savings through better contractor
cost performance and accountability.
Enabling the obligation of funds under individual task orders.
Providing better opportunity for PBSA.
Region 10 awarded new ID/IQ contracts (one small and one large business) in
July and December 2003, but Regions 6 and 7 have not awarded their contracts.
The contract specialist and the Project Officer for Region 10 both believe that the
new contract requires more initial planning but less overall contract
administration. For example, task orders must be planned and funded site
specifically, which requires better initial planning and cost estimating than
traditional work assignments. However, these new contracts do not have award
fees, so performance evaluation boards are not needed. Also, these contracts are
loaded fixed-rate (all overhead and program management expenses are built into
the hourly labor rate) and do not have separate program management accounts to
administer nor require routine indirect cost rate audits.
Region 10's new contract generally has the same statement of work as the current
RAC, as well as some of the same flexibilities in the type of task orders issued.
Task orders issued can be cost-reimbursable (term form or completion form) or
fixed price. However, the task order contract provides better funds control
because the task order obligates the funds.
-------
The USAGE procures remedial services similar to EPA, and faces many of the
same uncertainties and funding hurdles. However, term form contracts are
discouraged by USAGE and are not used when procuring remedial services.
Under their Pre-placed
Remedial Action Contract, the
USAGE awards a task order,
detailing the services required
much like EPA. USAGE
purchases complete tasks rather
than labor hours. If site
conditions change, USAGE can
issue a "Work Variance
Notice," which will change the
task order requirements within
the overall scope of work of the
task order. This demonstrates
that remedial cleanup work can
be accomplished with contracts
structured as other than CPAF
LoE.
Completion form contracts describe the scope
of work as a clearly defined task or job, with a
specific end product required. If the contractor
does not complete the work within the estimated
cost, the Government may require more effort
without an increase in fee, provided the
Government increases the estimated cost.
Term form describes the work in general terms
and obligates the contractor to devote a
specified level of effort for a stated period of
time. At the completion of the performance
period, the contractor is paid cost and fee if
performance is considered satisfactory.
Additional work is considered a new acquisition.
We believe EPA is moving in the right direction with three regions changing
contract type. The Agency should expedite a lessons learned analysis of the new
ID/IQ contracts as soon as sufficient data is available. RACs are 10-year
contracts, and EPA should confirm whether in fact the ID/IQ approach reduces
risk and lessens contract administration effort. If the results of the review shows
this to be the case, then the RAG II contracts in the seven remaining regions
should not be CPAF-LoE. If this information is not available in time for award of
the remaining contracts, and any of the contracts awarded are CPAF-LoE, then the
results of the review should be factored into the decision whether to exercise the
option periods.
Contract Structure and Size Limit Competition
RACs are full-service Architect-Engineer contracts that can provide most
Superfund site cleanup services. The contracts' maximum value can exceed
several hundred million dollars. EPA has historically been reluctant to reduce the
overall scope of the RAC, which currently can provide most services required
over the lifetime of a site. The wide array of services currently provided, along
with the RACs' long performance periods, may limit competition to only the
largest firms capable of providing these services. This limits the participation of
smaller firms, which may further limit competition.
-------
EPA's Contracts 2000 workgroup recommended in 1999 that the design and
construction portions of the RACs be broken out into separate contracts.
Advantages of separating design from construction are:
Increased opportunities for fixed price, completion form, and performance
based contracting.
Increased competition in the contracting process.
Increased small, minority, and women-owned business participation in the
Superfund contracting program.
Disadvantages of separating design from construction are:
* Limited EPA experience awarding and managing construction
contracts.
Resources needed to develop construction management expertise.
Instead of mandating the separation of design and construction contracts, the later
Design Construction workgroup advocated, and EPA adopted, a menu approach
whereby regions could choose to break out the design and construction portions,
or continue to utilize full service RACs. Agency officials stated that regional
resistance to change prompted the switch to the menu approach.
While the menu approach seems to balance national consistency with regional
flexibility, there is limited assurance that it will lead in ail cases to increased
competition, increased small, minority, and women-owned business participation,
or use of performance based, multiple award, and fixed price contracts. There is
no guarantee that the regions will select other types of contracts, and they may
stay with the existing contract type, CPAF.
OSRTI staff advised that in the remaining contract awards, the regions will be
required to identify at least one contract for Small Business Set-Aside. They also
advised that while one of the remaining seven regions is not changing contract
type, the others are considering doing so.
EPA at Risk of Losing Funds Through Litigation
Obligating funds on an LoE contract in excess of identified needs recently resulted
in losing contract funds as a result of litigation, and this could happen again. In
January 2002, the U.S. Court of Appeals for the Federal Circuit held EPA liable to
pay a contractor all funds allotted to a contract. In this case, EPA ordered LoE far
below levels called for by the contract, and the contractor was able to prove
resulting damages. The contract, although not a RAC, contained clauses identical
to those in the current LoE RACs.
-------
As a result of the court ruling, EPA issued a document stating, in part, "OAM and
Program Offices needed to guard against the practice of funds being allotted to a
LoE contract for the purpose of preserving funds for other uses, rather than for use
in reimbursing incurred contract costs...."
Regions order work by issuing work assignments. Work assignments are tasking
documents but are not obligating documents, so RAC funds are obligated at the
contract level, then work assignments are issued against those obligations. This
practice, known as bulk funding, places funds on the contract before work is
ordered, in some cases years in advance. Regional offices use bulk funding to
preserve funds, and to provide the flexibility to move funds between sites. For
example, if unanticipated delays occur, regional managers are able to redirect
contractor services and shift those funds to higher priorities within the region.
Regional officials stated that the amount of bulk-funded obligations on each
contract generally corresponds to work ordered on work assignments. However,
regional officials informed us there will be a small percentage of the total bulk
funding that is not yet supported by approved work assignments, which represents
planned new work, some program management, and award fee.
The Agency needs to actively monitor RAC funding activity. As of February
2004, there was approximately $23.4 million of bulk funding on RACs. The
Regional Project Officers had plans to order approximately $14.8 million of work
against these obligations. The remaining $8.6 million appears to be funding in
excess of the regions' identified needs and "at risk" in light of the court decision.
EPA's April 2004 report, SUPERFUND: Building on the Past, Looking to the
Future, stated that some of the regions, at their present expenditure rate, had
placed enough funding on their RACs to be able to utilize them for 1.2 years to
4.6 years in the future without placing additional funds on the contracts. The
report recommended that OARM and OSWER work closely with the regions to
monitor contracts to ensure that the regions have not funded their contracts into
the future to an extent where they cannot appropriately use the funds during the
contract period.
Regional Desire and Funding Issues Drive the Need for Flexibility
The RACs' structure and funding method support the regions' desire for
flexibility to deal with unpredictable site conditions. However, EPA does not
actively track the frequency and scope of these unknown conditions. Agency
officials said these unpredictable conditions are an inherent part of their business,
and the RAC's flexibility allows contract managers to easily adjust. They stated
they need this flexibility because of the way EPA funds its work and because of
its deobligation and re-certification procedures, which direct that 75 percent of
deobligations be returned to Headquarters.
10
-------
Regional officials stated that for the past several years, they have received their
appropriation late in the fiscal year, as late as the second or third quarter. In these
situations, regional officials believe it would be difficult to manage and start new
task orders. With task orders, work has to be identified and costs negotiated
before task orders are awarded and funds obligated. They stated regional offices,
therefore, would have difficulty issuing new task orders and obligating funds
before the end of the fiscal year. By contrast, under current procedures the RACs
are bulk funded. Work assignments are subsequently issued and amounts
negotiated. If site conditions change and cause an unexpected work stoppage,
funds can easily be shifted to other work. Moreover, under the current
deobligation and recertification procedures, managers using task orders would
have to return 75 percent of the deobligated funds to Headquarters, rather than just
shifting the funds to a different work assignment.
We do not believe that the desire to avoid EPA's funds recertification process
should be allowed to drive the contract structure. Contract type should be
determined based on Agency needs and the risks involved. As discussed above,
EPA has been managing site cleanups for more than 20 years, and this experience
should allow for the use of contract types that shift risk from the Government to
the contractors.
EPA Acknowledges that Benefits Can be Achieved Through Different
Contract Type and Structure
The EPA April 2004 Superfund report previously mentioned concluded that
"while pursuing alternative types of contracts (i.e., performance-based, site-
specific, and task order contracts) will require a greater investment in Agency and
Superfund program time and personnel, done properly, these different contract
types can result in significant cost savings to the program." The report
recommended that O ARM, OSWER, and the regions work together to encourage
the use of alternative contract types, and that OARM and regional contracting
officers should offer regular training for contract personnel, regional project
managers, on-scene-coordinators, and project officers in alternative contract
mechanisms. Procuring remedial actions on a site-specific basis offers certain
benefits. It provides more opportunity for fixed price work, eliminates some
potential conflicts of interest, provides more competition for remedial action
work, provides more opportunity for small businesses, and provides more
opportunities for PBSA. EPA could reap many of these benefits by increasing its
use of site-specific contracts. With the expiration of the current RACs, the
Agency has the opportunity to reduce costs and increase the availability of funds
for the remedial program by moving away from CPAF contracts to ID/IQ task
order contracts, separating design from construction and utilizing more site
specific contracts in the new round of RACs.
11
-------
Recommendations
We recommend that the Acting Assistant Administrator for OSWER, in
coordination with the Acting Assistant Administrator for OARM:
2-1 Develop and implement a plan with milestones for RAC II which:
a) increases use of ID/IQ, task order, and site specific contracts, and
b) increases the use of separate contracting for design, construction, and
other remedial services.
2-2 Conduct a lessons learned analysis of the new ID/IQ contracts as soon as
sufficient data is available and develop a plan to share and utilize the
results of the analysis.
2-3 Stop funding RACs in excess of identified needs.
2-4 Review and revise the recertification policy so as to remove it as an
impediment to utilization of better contract types. The policy should
reflect a balance between the need to return funds to Headquarters for re-
distribution to where the most pressing needs are identified, and the degree
of flexibility Headquarters is willing to grant to the regions to move funds
between sites as they now do with work assignments.
Agency Comments and OIG Evaluation
EPA generally agreed with most of the recommendations.
In response to recommendation 2-1, the Agency stated it developed plans to
increase the use of ID/IQ contracts and that its current strategy (Contracts 2000 -
Design/Construction Contracts Decision memorandum dated September 15, 2000)
already allows for separate contracts for design, construction, and other remedial
services; and that Regions 7 and 10 are piloting this concept. In addition to
Region 10 mentioned in the report, Regions 6 and 7 will award ID/IQ contracts
and other regions are considering using this contract type.
We do not believe that EPA's current strategy of providing a menu approach is
the same as developing a plan with milestones that would increase the use of
different contract types and structures, such as ID/IQ contracts. When providing a
response to the final report, EPA should provide a plan to increase the use of
ID/IQ contracts.
EPA agreed with recommendation 2-2 to conduct a lessons learned analysis of the
new ID/IQ contracts and stated they would share the results. We believe EPA
should go beyond just sharing the results of the analysis. If results of the lessons
12
-------
learned demonstrate that ID/IQ contracts are more beneficial, EPA should develop
guidance that lays out the circumstances when certain types of contracts, such as
ID/IQ, should be used. Any deviation from the preferred contract type should be
justified.
EPA did not agree that RACs are funded in excess of identified needs, but agreed
to examine the extent to which regions are "forward" funding project needs. We
conducted a detailed contract-by-contract analysis of the RAC funding and
determined that regions had funds allocated to contracts in excess of identified
needs and planned work.
EPA agreed that there needs to be a revision or clarification of the recertification
policy, especially as it applies to RACs that use task orders or other contract types.
At the exit conference, EPA staff stated that regions will be exempt from having
to return 75 percent of the deobligated funds to Headquarters if funds deobligated
from a task order are obligated to a task order under the same RAC concurrently.
13
-------
Chapter 3
Procurement Decisions Not Adequately Documented
EPA regions do not consistently document the rationale used to decide what
procurement option to utilize for Superfund cleanup activities, as required by
established policy. Further, EPA does not have a process to measure and
disseminate information on USAGE past performance in support of EPA. As a
result, the Agency has limited assurance that sound procurement decisions are
being made.
OSWER Requirements Were Not Followed
OSWER's May 19, 2003, policy, "Superfund Policy for Assigning Remedial
Work to the US Army Corps of Engineers (USAGE)," set forth factors to consider
May 20Q3 Policy ',
Sourea SeJeeioft , '
FJtefos*
Site Characteristics
Remedy Characteristics
Local/Public Interest
Experience/Regional
Infrastructure
Capability/Capacity
of Contractors
Conflict of Interest
Unique Site Needs
when selecting the appropriate procurement
options (see box). Specifically it states, "As with
all procurements, analysis should be conducted to
select the appropriate contracting vehicles... All
analyses should be documented in the site or
contract file. The documentation should outline
the reason behind the selection, especially any
factors that were used to make the decision."
In six regions, the 2003 policy was not being
followed and inconsistent evaluation methods
were being used.
Four regions (2, 5, 6, and 9) had no documentation
of factors considered to support decisions made to
assign (or not assign) work to the USAGE. The other two regions (1 and 7)
documented some of the factors used to make their decision, but did not fully
comply with the May 2003 policy.
Region 1 utilizes a "Decision Matrix." The factors listed in the Region 1 Decision
Matrix are similar to those required by the May 2003 policy. However, the
decision matrix does not consider Remedy Characteristics and
Experience/Regional Infrastructure.
Region 7 utilizes an "option paper," which evaluates the different alternatives
available for cleanup activity. The option paper does not address some of the
factors listed in the May 2003 policy, such as conflict of interest, remedy
characteristics, and unique site needs. Regions 2, 5, 6, and 9 advised they used
informal discussions among the Project Officers, Remedial Project Managers, and
14
-------
Branch Chiefs when deciding with whom to contract. These informal discussions
are not documented.
Headquarters staff stated that the regions should be documenting their decisions in
all cases. However, some regional staff stated they were unsure whether they had
to document their decision in all cases or only when they use the USAGE. By not
documenting the reasons behind the selection of a procurement option, the
Agency has limited assurances that sound decisions are being made in accordance
with applicable policy.
USAGE Past Performance is Not a Factor in EPA's Source Selection
Considerations
EPA does not have a formal process to measure and disseminate information on
USAGE past performance in support of EPA cleanup activities. As of February
2003, the EPA Superrund program had approximately $453 million in Interagency
Agreements with the USAGE to support Superrund cleanup activities. The May
2003 policy lists numerous source selection considerations; however, past
performance of the USAGE is not a factor. As a result, there is an increased risk
that a USAGE district office with performance problems in one EPA region could
be awarded additional work in other regions.
Regional staff oversee USAGE activities through progress reports, meetings, and
e-mails. However, no process currently exists for regional staff to systematically
assess USAGE performance and share it with EPA staff nationwide. EPA is
starting to address these issues. EPA's August 2003 report, "Evaluation of the
Performance of the Corps of Engineers in Support of EPA's Superfund Program,"
reported on how well the USAGE was supporting Superrund programs in EPA
regional offices. Some of the recommendations were that:
New Interagency Agreements include criteria against which USAGE
performance will be measured and evaluated.
USAGE and EPA Headquarters implement annual oversight of their
respective field programs.
EPA and regional offices institute formal annual oversight programs of
USAGE District offices managing or overseeing Superrund projects and
suggested criteria on which the USAGE should be evaluated.
OSWER developed an action plan to implement these recommendations and to
conduct regional evaluations. However, milestone dates have not been
implemented for all action items. The Agency has developed a plan to conduct
regional evaluations of the USAGE. After regional evaluations are completed, the
Regional Liaison is required to summarize the results of the evaluations and
forward them to Headquarters. However, the plan is unclear as to how regions
would have access to other regions' evaluations. The Agency should develop a
15
-------
system or database in order to share the evaluations nationwide. The full
implementation of the action plan should allow EPA to better monitor the costs,
the quality of the work, and the timeliness of performance provided by the
USAGE.
Recommendations
We recommend that the Acting Assistant Administrator for OSWER:
3-1 Issue clarifying guidance on "Superfund Policy for Assigning Remedial
Work to the US Army Corps of Engineers (USACE)" requiring that:
a) past performance of USACE be considered in source selection, and
b) the rationale for all source selection decisions be documented.
3-2 Develop and implement a plan to evaluate USACE performance and share
this information nationwide.
Agency Comments and OIG Evaluation
EPA stated that as part of the source selection considerations the regions
determine the best source from which to procure and manage construction
services. However, Regional Reviews will look into how the performance of
USACE was considered and documented, as well as ways of sharing performance
issues at national meetings, such as the Senior Superfund Regional Managers
Acquisition Council. We concur with EPA actions of developing feedback on
USACE. However, EPA has not demonstrated how it intends to utilize this
information beyond sharing the results at meetings. We are therefore
recommending that EPA develop a system to disseminate this information
nationwide to aid in the source selection decisions.
EPA agreed with the recommendation that more documentation be required, and
stated that the factors to be considered, as outlined in current policy, are
guidelines rather than requirements. EPA further stated it will be reviewing how
each region documents its decision as part of the planned Superfund Contracts
Regional Review program. EPA plans to share performance issues on a national
basis at national meetings, such as the Senior Superfund Regional Managers
Acquisition Council. We agree with EPA actions to conduct regional reviews.
EPA should not only share results of these reviews, but issue/reissue any guidance
deemed necessary as a result.
EPA agreed that sharing information about contracts procured by USACE and the
process used will be a valuable management tool, but did not agree it needed a
formal process to evaluate USACE's past performance. EPA stated it is testing a
feedback report intended to be completed by both EPA and USACE project
16
-------
managers that will provide progress, quality, and status of ongoing work. This
feedback, in addition to routine project communications, will highlight for
managers potential problems that need to be resolved. EPA also stated they are
developing a completion report to provide information to EPA and USAGE
personnel on the overall satisfaction of USAGE'S management of the project and
the USAGE contractor's performance, and this information will be available to all
regions. We are not advocating the development of any additional plans beyond
what EPA has started to develop. What we are recommending is that EPA make
evaluation information more readily available to EPA staff nationwide. EPA
stated that both the feedback and completion reports will be available to all
regions. However, EPA did not provide us with its means of doing this. The
Senior Superfund Regional Managers Acquisition Council meeting, which is held
twice a year, is a good avenue to discuss the results of evaluations, but providing
regions with access to these evaluations as needed would be more beneficial.
17
-------
Chapter 4
Past Performance Evaluations Not Documented
Timely and Consistently
The Agency has measures in place to assess contractor performance at the work
assignment level, but evaluations at the contract level were not being documented
timely and consistently as required because they were not given the necessary
priority. There is no EPA individual held accountable for assuring past
performance evaluations are documented in the National Institutes of Health
Contractor Performance System (NIH CPS). Annual evaluations allow
Contracting Officers to recognize contractors performing well, and to correct
problems before they jeopardize contract completion. Contractors can gain or
lose contracts based on past performance information, which in turn can mean
millions of dollars in revenue. Therefore, not consistently documenting
evaluations in a timely manner does not permit EPA and other Federal agencies to
consider contractors' past performance and it could be detrimental to contractors
who have preformed well.
Past Performance Evaluations at Contract Level Are Not Documented
Timely and Consistently in NIH CPS
Environmental Protection Agency Acquisition Regulations 1509.170 and
1552.209-76 provide time-frames for completing contractor evaluations and
require the Contracting Officer to complete the process within 90 business days
after the end of each 12 months of contract performance period in the NIH CPS.
The NIH CPS collects and maintains contractor performance evaluations. It
allows instant access to multiple agency evaluations in Federal departments and
independent agencies.
As of May 7, 2004, 34 interim evaluations for the RACs should have been in the
NIH CPS. However, only 26 evaluations were in the system.
Nineteen of the 26 evaluations were finalized, and available to the contracting
community. However, many of the evaluations took a long time to be
finalized. Eleven of the 19 (58 percent) completed evaluations took from
1.52 to 351 days.
Seven of the 26 evaluations were started and "in progress," and had been in
process from 128 to 858 days from the end of the evaluation period.
18
-------
There is inconsistent evaluation information in NIH CPS. For the 26 evaluations
in the NIH CPS:
Four had evaluation periods exceeding a year, ranging from 4 years to more
than 8 years. Beginning periods for these evaluations were the contract award
dates, 1995, 1996 (2), and 1998.
One evaluation was for a 6-month period of performance instead of the
required 12 months.
Three evaluations for one contract were for the same performance period.
Two evaluations were listed as final, even though the contracts do not end
until 2006.
There is no EPA individual responsible for assuring past performance evaluations
are documented timely and consistently in the NIH CPS. Each region is
responsible for regional contractor evaluation information. Additionally, there are
various measurement points during the evaluation process, but no one is
monitoring these measuring points to ensure timely completion.
The May 2000 Office of Federal Procurement Policy, "Best Practices for
Collecting and Using Current and Past Performance Information," states that
contractors and industry associations emphasized the power of past performance
as a tool for motivating contractors to make their best efforts, and raised concerns
that many assessments are not being done, or are being done inconsistently.
Interim evaluations allow Contracting Officers to recognize contractors
performing well, and to correct problems before they jeopardize contract
completion. Interim evaluations also provide current performance information on
comparable contracts to source selection teams. Completing these assessments
improves the quantity and quality of performance information available to aid in
source selection, not just by EPA but other Federal agencies as well. Contractors
can gain or lose contracts based on past performance information, which in turn
can mean millions of dollars in revenue. Therefore, not consistently documenting
evaluations in a timely manner is a disservice to EPA and other Federal agencies,
as well as EPA's best contractors.
By not adequately documenting evaluations, the Agency is not fully meeting its
requirement of providing past performance evaluations to the contracting
community. FAR 9.104-l(c) requires a prospective contractor to have a
satisfactory performance record to do business with the Government. Past
performance can be used to help the Government decide whether a contractor is
capable of performing and provides a good indicator of future performance.
19
-------
Lack of formal training may have resulted in inconsistent and inaccurate
evaluation information. EPA has designated one individual to provide training on
the N1H CPS. Most staff nationwide have not been trained due to lack of travel
funds.
OAM has developed a "tickler" system to remind Contracting Officers to
complete evaluations. However, OAM needs to develop controls to ensure that
Contracting Officers document evaluations timely and consistently in the NIH
CPS
Recommendations
We recommend that the Acting Assistant Administrator for OSWER, in
coordination with the Acting Assistant Administrator for OARM and the regions:
4-1 Provide NIH CPS training to applicable regional and headquarters staff.
4-2 Develop a method for holding Contracting Officers accountable for
documenting evaluations of contractor performance in the NIH CPS timely
and consistently.
Agency Comments and OIG Evaluation
EPA agreed with the recommendations and noted that more training and
accountability are required in the area of contractor performance evaluations.
However, EPA did not give any detailed information on how staff will be trained ,
or how it plans to hold Contracting Officers accountable if evaluations are not
documented. EPA will need to decide on a cost effective method of providing
training so that data can be input timely and consistently.
20
-------
Chapter 5
Utilization of Remedial Action Contract Management
Information System Could be Improved
Contract managers have, or can obtain, the information needed to evaluate results
and make decisions, but the information in the national automated database is not
always readily available. The Remedial Action Contract Management
Information System (RACMIS) is underutilized by regional staff and the system
does not collect national data as originally intended. Most regional staff do not
use RACMIS due to its limited reporting capability, and some regions have
decided not to require contractor information through RACMIS due to its costs.
As a result, EPA is expending approximately $ 1.5 million a year on a system that
is not being fully utilized and may not be used for its intended purposes.
The Administrator's Task Force Report of October 1991 required establishing a
management information system to monitor resources and establish regional
capacity for providing Independent Government Cost Estimates. RACMIS was
created to meet this requirement, and the system was deployed in 1996 and 1997.
RACMIS is currently utilized primarily by Headquarters to report on contract
obligations, expenditure limits, expenditures, type of work performed, regional
crossover work, and creation of Independent Government Cost Estimates.
However, most regional staff use locally based spreadsheets and do not use
RACMIS due to the limited retrieval and reporting capability available in the
regions. Currently there is a lag time between the regions' requests for data and
the generation of reports to meet those requests. Some regions believe the
information can in fact be captured at a lower cost.
Because of this, Regions 7 and 10 chose, and were allowed, to opt-out of reporting
in RACMIS for the new RACs. According to Region 10 staff, "this was done
because the program in Region 10 did not want to bear the cost of RACMIS when
essentially the same information could be obtained at a lower cost."
To address the problem of underutilization, the Agency plans to implement
Business Objects, a report-writing tool that will allow the regions to readily access
RACMIS to create reports. Headquarters provided Business Objects training to
some regional staff in April and May 2004. Indications from staff attending the
training is that RACMIS will now be utilized more in the regions.
The OSRTI staff advised that Region 7 now wants to utilize RACMIS on its
contracts and 0AM staff stated that all the other contractors for the next round of
RACs will be required to utilize RACMIS. Region 10, however, has not changed
its plans. Without data from all the contractors, complete national data on RACs
21
-------
will not be available in RACMIS, compromising the ability to report timely
national data.
OSRTI staff also advised that they plan on performing an analysis before the end
of calender year 2004 to assess whether RACMIS is truly needed, whether the
cost of RACMIS is reasonable for the infomiation being provided, or whether
contractors' financial reporting requirements can be met at a lower cost. They
advise that, currently, the cost of reporting through RACMIS is $1.5 million a
year, less than 1 percent of total RAC costs.
If the Agency decides that RACMIS is the system it wants to use, then reporting
through RACMIS should be mandatory for all regions.
Recommendations
We recommend that the Acting Assistant Administrator for OSWER, in
coordination with the Acting Assistant Administrator for OARM and the regions:
5-1 Complete the proposed cost benefit analysis to determine whether
RACMIS should be retained or other more cost effective methods should
be used to collect RAC financial data.
5-2 If EPA decides to continue using RACMIS, develop and implement a
strategy to improve regional utilization of RACMIS (or its replacement
system if so decided), and require that financial information be collected
for all RACs.
Agency Comments and OIG Evaluation
EPA stated that it is conducting a cost benefit analysis of RACMIS and plans to
discuss the results of its analysis at the Senior Supertund Regional Managers
Acquisition Council meeting in December 2004. EPA's response addresses our
first recommendation; however, EPA still needs to address how it will assure that
the system (RACMIS or its replacement) will be utilized for all RACs.
22
-------
Appendix A
Schedule of Response Action Contracts Reviewed
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Contractor
Tetra Tech NU
Metcalf & Eddy
Foster Wheeler
Environmental Corp.
COM Federal
Programs Corp.
Tetra Tech NUS
Tetra Tech/Black &
Veatch
COM Federal
Programs Corp.
Black & Veatch
Special Projects Corp.
CH2M Hill
Weston Solutions, Inc.
CH2M Hill
Tetra Tech
Environmental Mgmt.
Black & Veatch
COM Federal
Programs
CH2M Hill
URS Greiner
Contract
Numb4F
68-W6-0045
68-W6-0042
68-W-98-214
68-W-98-210
68-S6-3003
68-S7-3002
68-S7-3003
68-W-99-043
68-W6-0025
68-W7-0026
68-W6-0036
68-W6-0037
68-W5-0004
68-W5-0022
68-W-98-225
68-W-98-228
ilfc&mSfft
Contract .
₯«ta*
$423,248,994
$459,536,934
$243,770,460
$217,731,330
$168,720,501
$168,763,187
$169.329.900
$346,336,991
$288,327,444
$290,046,890
$290,728,258
$253,261,985
$302,858,596
$237,629,770
$303,984,009
$130,466,269
Date of
Award
08/29/1996
09/30/1996
07/06/1998
07/06/1998
09/30/1996
06/12/1997
09/30/1997
06/17/1999
09/30/1996
05/01/1997
07/08/1996
09/13/1996
06/15/1995
020/5/1997
09/29/1998
09/30/1998
Pariftdof
Performance Base
Phis Option Yaarfe
08/29/1 996-08/28/2006
09/30/1996-09/29/2006
070/6/1998-07/05/2008
07/06/1998-07/05/2008
09/30/1996-09/29/2006
06/12/1997-06/11/2007
09/30/ 1 997-09/29/2007
06/17/1999-06/16/2009
09/30/ 1 996-09/26/2006
05/01/1997-04/30/2007
07/08/1996-07/07/2006
09/13/1996-09/12/2006
06/1 5/1995-06/14/2005
09/29/1995-09/28/2005
10/01/1998-09/30/2008
10/01/1998-09/30/2008
t*«-
fliw*
i
1
2
2
3
3
3
4
5
5
6
6
7
8
9
10
Maximum RACs Value: $4,294,741,518
23
-------
CHAPTER 5 - UTILIZATION OF REMEDIAL ACTION CONTRACT MANAGEMENT
INFORMATION SYSTEM COULD BE IMPROVED
Recommendation 5-1 "Complete the proposed cost benefit analysis to determine whether
RACM1S should be retained or other more cost effective methods should be used to collect
RAC financial data."
EPA RESPONSE:
We are conducting the cost benefit analysis called for in this recommendation. We plan
to present the results to the SRMAC meeting in December 2004. We will share the presentation
with you.
Recommendation 5.2 If EPA decides to continue using RACMIS, develop and implement a
strategy to improve regional utilization of RACMIS (or its replacement system if so
decided), and require the financial information be collected for all RACs.
EPA RESPONSE:
We agree that if we continue to use RACMIS we will require that the information be
collected for all RACs.
OTHER EPA COMMENT ON CHAPTER 5:
Page 21, ^paragraph, first sentence: "OSRTI staff advised that Region 10 now wants to utilize
RACMIS on its contracts " is incorrect. Whereas there may have been discussion among staff
about RACMIS, OSWER has not received official notification by Region 10 that they want to
utilize RACMIS. Second sentence: "Region 7, however, has not changed its plans. "
Region 7 has indicated to OAM that they will utilize RACMIS in the future.
Thank you for the opportunity to review the draft report. If you have any questions
regarding our comments please contact Johnsie Webster at 202-566-1912 or Barbara McDonough
at (703) 603-9042.
cc: Judy Davis
Mike Cook
Johnsie Webster
Susan Janowiak
Paige Peck
Yvette Gamer
28
-------
Appendix C
Distribution
EPA Headquarters
Acting Assistant Administrator, Office of Solid Waste and Emergency Response (5101T)
Director, Office of Superfund Remediation and Technology Innovation (5201G)
Acting Assistant Administrator, Office of Administration and Resources Management (3101T)
Director, Office of Acquisition Management (3 801R)
Agency Followup Official (the CFO) (2710A)
Agency Followup Coordinator (2724A)
Audit Followup Coordinator, Office of Administrator (1104)
Audit Followup Coordinator, Office of Solid Waste and Emergency Response (5103)
Audit Followup Coordinator, Office Administration and Resources Management (3102A)
Associate Administrator for Congressional and Intergovernmental Relations (1301 A)
Associate Administrator for Public Affairs (1101 A)
Inspector General (2410)
EPA Regions
Regional Administrator (1-10)
Regional Audit Followup Coordinators (1-10)
29
------- |