UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, O.C. 20460
MAR 28 1980
OFFICE Of
THE INSPECTOR GENERAL
V MEMORANDUM
SUBJECT:
'FROM!
TO:
Audit Report No. E1SKF9-11-0031-0100228
The SITE Demonstration Program: Much
Promise But Delayed Results
eneral
Kenneth A. Konz
Acting Assistant Inspector
for Audit (A-109)
Erich W. Bretthauer
Assistant Administrator for Research
and Development (RD-672)
Don R. Clay
Assistant Administrator for Solid Waste
and Emergency Response (OS-100)
Charles L.. Grizzle
Assistant Administrator for Administration
and Resources Management (PM-208)
SUMMARY OF FINDINGS
The Office of Research and Development (ORD) and the Office of
Solid Waste and Emergency Response (OSWER) need to actively
increase efforts to demonstrate innovative treatment technolo-
gies and to disseminate the results to regional cleanup decision-
makers. Our review of the Superfund Innovative Technology Eval-
uation (SITE) Demonstration Program showed that: (1) matching
technologies to hazardous waste sites has been a slow process;
(2) lengthy reporting period? and delays in reporting demonstra-
tion results have provided little detailed information to assist
regional decision-makers; (3) insufficient attention has been
placed on contractor performance and SITE costs; and (4) the
method selected to provide Federal assistance discouraged tech-
nology developers. In addition, about 30 percent of regional On-
Scene Coordinators (OSCs) and Remedial Project Managers (RPMs) we
contacted had either not heard of the SITE Program, or were not
familiar enough with it to answer our questions. Further, Super
fund cleanup contractors and technology developers expressed
CD
OT
OT
to
CM
HEADQUARTERS LIBRARY
ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
-------
numerous concerns about the program. In fact, four of the five
developers we talked with indicated that, if given the chance,
they would hot participate in the SITE Program again.
For some time, Congress has been extremely concerned over the
effectiveness and cost of conventional methods for handling
hazardous wastes at Superfund sites. Because of these concerns,
Congress provided direction to the Environmental Protection
Agency (EPA) on the use of alternative technologies to replace
these conventional cleanup methods. Both the Superfund Amend-
ments and Reauthorization Act (SARA) and resulting Agency
guidance point out this intent and the importance of the
Demonstration Program to testing and reporting on innovative
technologies. While there may be other scattered efforts to
test alternative technologies, the SITE Demonstration Program
was developed to be the only comprehensive Agency-wide program .
to demonstrate and report on innovative technologies.
The Demonstration Program's primary objective is to develop
information on innovative technologies so they can be adequately
considered in remediation decision-making for hazardous waste
sites. These demonstrations are important because they provide
information to assist cleanup personnel in making sound decisions
as to the applicability of the technology for the specific site
and to compare a technology's effectiveness and cost to other
alternatives.
1. IMPROVEMENTS NEEDED IN MATCHING TECHNOLOGIES WITH
HAZARDOUS WASTE SITES FOR DEMONSTRATION PURPOSES
The Demonstration Program has been delayed because of problems
in locating sites for the technology demonstrations. Matching
technologies to specific sites has been a difficult and time-
consuming task, frequently taking about a year or longer. SITE
officials repeatedly asked the regions to volunteer prospective
sites since a universe of hazardous waste sites with known
characteristics was not yet available. This caused time delays
in site matching. The reliance placed on the regions to suggest
sites was a major cause of the problem with site matching.
Recently program officials have developed a data base to help the
site matching process, but they will still have to rely on the
regions to provide timely and accurate detailed information on
specific sites identified in the data base.
Some regional Superfund officials are reluctant to recommend
sites and participate in the program because of the limited time
they have available to clean up sites. In addition, some RPMs
and OSCs we contacted were unwilling to recommend sites. There
was little or no incentive for them to do so, since there is
little documented information on successful technology
demonstrations to convince them to participate in the SITE
Program. We further found that these individuals are not tracked
YSAflaiJ 2R3THAUOOA3K
H01T33TOJW JATi13MHCfllWU
OSWK .CUJ .
-------
or held accountable for participation in the SITE Program but
instead are primarily accountable for cleanup of sites.
Developers turning down sites, developers exiting from the
program, and proposed technologies having a limited scope of
treatment have also impacted the Agency's site matching process.
It is especially important that the SITE Program address all
these factors to increase the number of demonstrations and thus
provide reliable, documented information to RPMs and OSCs to aid
in cleanup decision-making at Superfund sites.
2. IMPROVEMENTS NEEDED IN REPORTING RESULTS FOR COMPLETED
DEMONSTRATIONS
The SITE Program needs to address lengthy reporting periods and
delays in issuing the final reports of the technology demonstra-
tions. For example, these reports have taken as long as 22
months to be published. The reports are critical because they
are the principal means of notifying RPMs, OSCs and Superfund
cleanup contractors about the in-depth demonstration results.
Extensive rewrites and a long review process by EPA, the demon-
stration contractor, the technology developer, and other parties
that have reviewed the reports have delayed the reporting pro-
cess, while ORD has taken some actions to address reporting
concerns, additional emphasis is needed to shorten the reporting
period and eliminate delays in issuing the results of the tech-
nology demonstrations. In addition, ORD must shift from their
normal long range focus of 3-5 years to the shorter period of
under two years involved in the SITE process. Currently, SITE
management has set a new timeframe of 18 months for the entire
demonstration process. SITE officials should explore possible
alternatives to identify and eliminate the bottlenecks in this
reporting process.
The lack of reported information and documented results has
handicapped the success of the program. In fact, 30 percent
of the individuals we contacted in the regions told us they
had not heard of the SITE Program or were not familiar enough
with it to answer our questions. These OSCs and RPMs are the
ultimate EPA users of the demonstrated technologies. While those
who were familiar with the program thought it was a very good
idea, to date they had not received sufficient information about
the results of specific demonstrations to encourage them to use
innovative technologies. Aa a result, the use of these innova-
tive technologies to more permanently clean up Superfund sites
has been hindered.
-------
3. IMPROVEMENTS NEEDED IN ENSURING ADEQUATE CONTRACTORS'
PERFORMANCE AND COST CONTROL
RPMs, OSCs, developers and SITE officials repeatedly expressed
concerns about contractors' performance in such areas as
experience, writing reports, and time delays. In addition,
program managers and SITE management did not appear to be
adequately controlling contractor costs and overall project
costs. The nine projects we reviewed frequently had significant
cost increases and time extensions. For example, we found
increases to original cost estimates that were as high as 137
percent with an overall average of over 98 percent. This cost
growth is especially important because the contractors received
significant portions of the Demonstration Program's budgeted
funds during fiscal 1987 through 1989.
We found that award fees were not always consistent with the
project time delays and increased costs. SITE management needs
to ensure that the award fee determinations are adequately used
to communicate performance results and the need for high quality
work to the contractors. Further, the SITE Program needs to make
improvements to insure adequate contractor performance and cost
control. Given the degree and scope of the concerns on perfor-
mance and continuing cost increases, the Agency needs to better
address project performance monitoring, cost control and time
delays..
4. THE SITE DEMONSTRATION PROGRAM DISCOURAGED DISTRIBUTION
OF FEDERAL FUNDS TO DEVELOPERS
The Demonstration Program discouraged distribution of Federal
assistance to technology developers, and as of October 1, 1989,
no developer has received funding assistance. SITE officials
took a conservative approach in providing funding because they
told us they were unclear as to what Congress intended. In
addition, they told us that they were concerned that funds pro-
vided to developers would limit the amount of available funds
remaining to conduct the program. Due to this conservative
approach, four of five developers we talked with believed funding
was an impediment to the program, and two indicated they would
not participate again in the program because of a lack of fund-
ing. Further, we found that while no technology developers
received funding, SITE contractors conducting the demonstrations
and writing the reports were paid approximately 80 percent of the
program's $34.6 million budget during fiscal 1987 through 1989.
Finally, 11 developers have exited the program without demonstra-
ting their technology. Little or no benefits accrued to the
Agency, at a cost of approximately $2.6 million. As a result,
because developers were not provided actual funding assistance,
the Demonstration Program may have discouraged developers of
other promising technologies from applying for admission to the
program.
-------
To correct the problems described above, we are making recommen-
dations concerning: improving the matching of technologies with
sites and increasing regional incentives and participation;
considering alternatives to eliminate lengthy reporting periods
and delays; establishing methods and alternative strategies to
deal with contractor performance and cost control; clarifying
Congressional intent regarding developer funding; and requesting
additional funds, if necessary, for developer funding.
We held an exit conference with the Director, Office of
Environmental Engineering and Technology Demonstration, the
Director, Risk Reduction Engineering Laboratory, and other ORD
and OSWER officials on February 7, 1990, to discuss our findings
and recommendations and to ensure a clear understanding of our
report by management. At this conference, these SITE Program
officials discussed their position relative to our findings and
recommendations.
The Assistant Administrator for Research and Development provided
us formal written comments on our draft report in a memorandum
dated March 12, 1990. His comments addressed the views of both
ORD and OSWER. The Assistant Administrator agreed with most of
our recommendations. However, he disagreed with two of them,
regarding retaining some.program functions in-house if contractor
performance does not improve, and decreasing the amount of
financial information required from developers interested in
financial assistance. To provide a balanced understanding of the
issues, we have summarized management's position at appropriate
locations in the report and provided.their verbatim comments in
Appendix 1.
ACTION REQUIRED
In accordance with EPA Directive 2750, the action official is
required to provide this office with a written response to the
audit report within 90 days,of the audit report date. Since this
report deals primarily with the management of the SITE Program,
the Assistant Administrator for Research and Development was
designated as the action official. The other two Assistant
Administrators should coordinate their comments with the
Assistant Administrator for Research and Development. For
corrective actions planned but not completed, reference to
specific milestone dates will assist this office in closing this
report.
-------
SCOPE AND OBJECTIVES
The purpose of the audit was to determine how successful the SITE
Demonstration Program has been in encouraging the development
and use of innovative technologies to achieve more permanent
protection of human health, welfare, and the environment. The
objectives of the audit were to determine if: (1) technology
demonstrations were being planned in accordance with SARA; (2)
technology demonstrations were being conducted and evaluated in a
timely manner; (3) completed demonstrations were leading to a
more widespread use of the technologies and more effective site
cleanups; and (4) adequate internal controls were in place to
guard against fraud, waste and abuse.
We performed our review from January 9, 1989, to October 13,
1989. To accomplish our audit objectives, we interviewed
SITE Program managers and other officials, such as the project
officers, the quality assurance officer, and the economist, at
the Risk Reduction Engineering Laboratory (RREL) in Cincinnati,
Ohio (where the Demonstration Program is administered) and in
Edison, New Jersey. We also interviewed appropriate SITE Program
officials at EPA Headquarters Office of Research and Development
and Office of Solid Waste and Emergency Response.
We selected nine of the 36 technologies which had been admitted
to the Demonstration Program as of April 18, 1989, to review
the progress of these projects since their acceptance into the
program. The selections were generally random. However, we
replaced two technologies in our original sample with those
recommended by SITE Program officials, to obtain a broader
perspective on the types of technologies in the program (see
Exhibit A for a summary of each of these nine technologies}.
We reviewed the extensive technology demonstration files and
interviewed project managers of the selected technologies to
determine whether the demonstrations were planned and conducted
in accordance with SARA requirements. We also analyzed the
procedures for the exit program and reviewed files for three of
the ten technologies involved.
We performed a nationwide telephone survey of 28 out of 361 RPMs,
25 out of 131 OSCs, 15 out of 32 Superfund contractors, and five
technology developers that had completed their demonstrations.
The RPMs, OSCs, and Superfund contractors were randomly selected
based on a statistical sample. We are 90 percent confident that
the results of our statistical sample represent the total uni-
verse of RPMs, OSCs, and Superfund contractors. However, the
sample of developers was not statistically valid because of the
small universe size. The purpose of our telephone survey was to
determine if: (1) the individuals with the responsibility for
determining which technology to use for cleaning up a hazardous
waste site were familiar with the Demonstration Program; (2) the
-------
documents generated by the program were reaching the perspective
users; (3) the documents received were useful in remediation
decision -making; and (4) the technologies in the Demonstration
Program were being considered for Superfund site cleanups.
We visited two Superfund hazardous waste sites, the McColl site
in Fullerton, California, and the Stringfellow site in Pyrite,
California, to observe the preparations for conducting the demon-
strations . We also interviewed the technology developer at a
state lead, site, GATX Terminals Corporation site, in San Pedro,
California. Lastly, in June 1989, we attended both the EPA Forum
on Innovative Hazardous Waste Treatment Technologies: Domestic
and International, -and the National RPM/OSC Meeting on Innovative
Technologies and the Superfund Program, held in Atlanta, Georgia.
We initiated the audit as part of our annual audit workplan.
During our preliminary review, Headquarters SITE Program manage-
ment informed us that problems in matching technologies with
Superfund sites have slowed the progress of the program and the
reporting of test results. However, they stated they had imple-
mented corrective actions to solve the problems. Therefore we
reviewed and tested the actions taken to correct the problems to
determine whether they were effective.
Our audit was conducted in accordance with the Government Audit-
ing Standards (1988 Revision) issued by the Comptroller General
of the United States. Our assessment of the program is presented
in the Findings and Recommendations section of this report. No
other issues came to our attention which we believed warranted
further detailed audit work.
BACKGROUND
SARA Section 209(b) amends the Comprehensive Environmental
Response, Compensation and Liability Act of 1980 (CERCLA) by
adding Section 311. This section directs the EPA to "carry out a
program of research, evaluation, testing, development, and
demonstration of alternative or innovative treatment technologies
. . . which may be utilized in response actions to achieve more
permanent protection of human health and welfare and the
environment."
Histor of the SITE
Prior to the enactment of SARA, concerns had been growing among
the scientific community, citizens, and government officials over
the effectiveness and cost of conventional methods for handling
hazardous wastes at Superfund sites. Historically, many Super-
fund cleanups had primarily involved removing contaminated
material, such as soil, and transporting it to a landfill, or
capping part or all of the site. These methods were not always
completely effective, and were increasing in cost. SARA
-------
emphasized more effective and permanent cleanups by stating that
an assessment of permanent solutions and alternative treatment
technologies should be conducted that would lead to a permanent
and significant decrease in the toxicity, mobility, or volume of
the hazardous wastes. The scientific and engineering communities
recognized that the demand for such treatment often outstripped
the availability and capability of existing technologies.
In response to these growing concerns, the Agency moved ahead in
early 1986 to develop a technology demonstration program within
the existing statutory and budgetary authority. A strategy was
developed to increase the use of innovative treatment technolo-
gies at Superfund sites. EPA convened a review group composed of
experts from large and small companies, acaderaia, and engineering
firms to help develop this strategy. The review group provided
much of the rationale for developing a technology demonstration
program, citing the need for an objective evaluation of new
technologies being marketed to the EPA regional offices and
states for cleanup efforts.
As a result of this initial work, EPA was able to respond with a
program when SARA was enacted that was already in the planning
stages, although funding was not available. EPA's Office of
Research and Development and Office of Solid Waste and Emergency
Response created the SITE Program, with two major purposest
(1) to accelerate the development, demonstration and use of new
or innovative treatment technologies; and (2) to demonstrate and
evaluate new innovative measurement and monitoring technologies.
The SITE Program has five components.
1 . The Demonstration Program develops performance
engineering and cost information on innovative
alternative technologies so that they can be
adequately considered in remediation decision-
making for hazardous waste sites.
2 . The Emerging Technologies pyogfyam performs
laboratory pilot- and bench-scale evaluations of
technologies that are not yet ready for field
demonstration .
3 . The Technology Transfer Program includes numerous
components that incprporate a variety of outreach
activities. This program disseminates demonstration
and waste remediation data from all components of the
SITE Program to regional and state managers of Super-
fund cleanup activities, Federal agencies, the
engineering community, related industries, and the
public .
-------
4 . The Measurement and Monitoring Technologies
Development Program seeks to improve Superfund site
characterization efforts by continually developing
new and innovative measurement and monitoring
technologies .
5 . The Innovative Technologies Program was an outgrowth
of early research and development efforts. It promotes
transfer of EPA-developed technologies to the private
sector for commercialization and use at Superfund sites
Although SARA was enacted in October 1986, funds were not appro-
priated until March 1987. Congress initially appropriated $9.1
million to support the SITE Program. The Demonstration Program
received approximately 88 percent of the program's overall bud-
get, and has received approximately 70 percent of the program's
budgeted funds in the two succeeding years.
The Demonstration
The Demonstration Program is the primary focus of the SITE Pro-
gram. As previously stated, its major objective is to develop
extensive performance engineering and cost information on inno-
vative alternative technologies so that they can be adequately
considered in remediation decision-making for hazardous waste
sites. Demonstrations are designed to provide information to
assist cleanup personnel in making sound judgments as to the
applicability of the technology for a specific site and to
compare the technology's effectiveness and cost to other
alternatives. The demonstrations usually occur at Superfund
sites or under conditions that duplicate or closely simulate
actual wastes and conditions found at Superfund sites. This
helps assure the reliability of the information collected and
the acceptability of data to users.
Demonstration projects are a1 partnership between EPA and the
developers. Each party has a specific role, and both must carry
out their responsibilities for the program to work. Developers
are responsible for demonstrating their innovative systems at
selected sites. They are expected to pay the costs to transport
equipment to the site, operate the equipment on-site during the
demonstration, and remove the equipment from the site after test
operations are complete. However, SARA Section 311(b)(5)(J)
states that Federal assistance may be provided to any developer
who can demonstrate that it cannot obtain appropriate private
financing on reasonable terms and conditions sufficient to carry
out the demonstration.
-------
EPA is responsible for project planning, sampling and analysis,
data quality assurance and quality control, report preparation,
and information dissemination. To carry out the Agency's respon-
sibility, the SITE Program has contracted with three engineering
firms to perform most of these activities. The contractors'
duties and responsibilities include, but are not limited to,
planning and preparing a detailed demonstration plan, preparing
the site, selecting and collecting samples, analyzing the
samples, collecting other data during the actual demonstration,
and drafting and finalizing the reports at the end of each
demonstration.
Technologies are accepted into the program through an annual
solicitation published in the "Commerce Business Daily." This
annual solicitation, known as the Request for Proposals (RFP),
allows interested developers to submit proposals to EPA. The
proposals are then reviewed by an EPA panel, which selects
approximately 10 technologies for the program each year.
The RFP for SITE solicitation 001 was advertised in the "Commerce
Business Daily" on February 13, 1986. The RFP was a modification
to a previous one of November 29, 1985, entitled "Demonstration
of Available Technologies for Hazardous Waste Site Cleanup." Any
proposals received in response to this earlier RFP were automati-
cally considered in SITE 001. While the SITE 001 RFP was issued
prior to the enactment of SARA, the solicitation was based on the
anticipated Superfund legislation. Subsequent to the enactment
of SARA, EPA has issued three more solicitations. EPA planned to
issue another RFP on January 5, 1990, and planned to take
responses through March 2, 1990.
There are two major outputs for each completed demonstration,
the Technology Evaluation Report (TER) and the Applications
Analysis Report (AAR). The TER documents the performance data
resulting from the demonstration, and the AAR evaluates the
potential uses of the technology for other sites and wastes, and
provides cost estimates for these applications. As of October 1,
1989, EPA had published six TERs and four AARs on test data
generated from six of the 12 completed demonstration projects.
Headquarters SITE management stated they measure the success of
the Demonstration Program in various ways. These include the
number of proposals received in response to the RFP, the number
of innovative technologies selected for use in the Records of
Decision (RODs), and the ultimate commercialization of the
technologies that participate in the program.
10
-------
Other EPA Efforts Involving Innovative Technologies
The SITE Program is currently the only comprehensive national
effort to encourage the development and use of innovative tech-
nologies to treat hazardous wastes. However, Headquarters SITE
management stated that they never intended for their program to
be the only means for accomplishing the Agency's SARA goals.
Other offices within the Agency have performed undertakings
similar to the SITE Program. For example, in fiscal 1985 through
1987, EPA Region 3 assisted a contractor in the development of a
soil washing process in an attempt to clean*up a site contami-
nated with polychlorinated biphenyls (PCB). EPA spent $800,000
toward the development of the technology. However, due to
repeated breakdowns with the machinery, Region 3 went back to
using a conventional method for cleaning the site. Also, during
fiscal 1987, EPA Region 4 initiated and supervised a removal
action at the Peak Oil Superfund site in Brandon, Florida. The
region contracted with an emergency removal contractor to destroy
sludge contaminated with PCB and lead, using an innovative
incineration process. The Office of Inspector General reviewed
the cleanup of this site (see the report entitled, "Review of
Region IVs Management of Significant Superfund Removal Actions,"
No. E5EH7-04-0181-81927, dated September 26, 1988), and found
that the incinerator did not destroy the hazardous materials to
required standards; the region allowed the burning to continue
after tests showed the incinerator did not work; PCBs and
hazardous particulates were released into the atmosphere; and
wastewater containing unacceptable levels of lead was discharged
to the local wastewater treatment plant. Nevertheless, the
Demonstration Program was able to collect appropriate data to
evaluate the technology's effectiveness.
In addition to exploring other developments of innovative treat-
ment methods, EPA has financed other contractors whose tasks are
similar to those of the SITE Program. A January 1989, Office of
Technology Assessment report highlights examples of policy
support contracts in which the Agency has awarded contractors
approximately $40 million to perform a variety of tasks, includ-
ing: (1) developing methods for technology transfer and determin-
ing if technology transfer activities are effective; (2) develop-
ing reports and briefing technology transfer committees on a
variety of new emerging technologies; (3) assisting in evaluating
the economic and technical feasibility of various alternative
technologies; and (4) analyzing SITE Program issues and results
and making policy recommendations.
11
-------
FINDINGS AND RECOMMENDATIONS
1. IMPROVEMENTS NEEDED IN MATCHING TECHNOLOGIES WITH HAZARDOUS
WASTE SITES FOR DEMONSTRATION PURPOSES
The Demonstration Program has been delayed because of problems
in locating sites for the demonstrations. In fact as of
September 30, 1989, almost 2 1/2 years after Congress funded the
SITE Program, only 12 of 36 technology demonstrations were
completed. SITE matching is one of the most crucial early steps
in planning the demonstrations. EPA must be able to match each
technology to an appropriate site so that it can be adequately
tested. But in some cases, site matching can take over a year
and longer (see Exhibit B), and nearly 20 percent of the tech-
nologies in the program still remain to be matched to sites.
This condition occurred because: (1) EPA lacked an inventory of
preplanned sites at which to conduct demonstrations; (2) many
regional officials lacked familiarity with and had concerns about
technology demonstrations; (3) technology developers could reject
sites they believed were unfavorable to their technology;
(4) some technologies could only treat a limited selection of
wastes; and (5) 10 technology developers "exited" the program for
various reasons. Due to these conditions, little documented
information is available for use by RPMs and OSCs in cleanup
decision-making at Superfund sites.
Both SARA and Agency guidance discuss the intent and importance
of the Demonstration Program. SARA Section 311(b)(6) directs the
Administrator to "initiate or cause to be initiated at least 10
field demonstration projects of alternative treatment technolo-
gies at sites in fiscal year 1987 and each of the succeeding
three fiscal years." ORD officials stated that they interpreted
SARA as requiring EPA to start 10 field demonstrations each year.
They believed that this requirement covers only the period from
the acceptance of the technology into the SITE Program through
the actual demonstration, but does not include the report writing
phase of the demonstrations themselves. EPA's March 1988, Long-
Ranoe Research Agenda for the Period 1989-1993 states "The Agency
will conduct 10 demonstrations per year to accelerate the commer-
cialization of innovative/alternative treatment technologies that
will clean up priority sites." In addition, the December 1986,
SITS Strategy and Program Plan states that, "The demonstration
program is a cornerstone of the SITE program. SARA calls for at
least 10 demonstrations annually." This same document identifies
matching technologies with sites as one of the principal steps
for conducting demonstrations, and indicates that once EPA has
chosen promising technologies ready for demonstration, the Agency
12
-------
will determine which hazardous waste site is most appropriate for
each demonstration. Finally, the September 1989, Administrator's
Task Force report, Management Review of the Superfund Program,
expressed concerns about the Agency's Superfund program. The
report discussed the importance of alternative technologies to
the Superfund program, and especially the importance of the SITE
Program. The report mentioned that improvement is needed in the
area of demonstrations, and specifically recommended that the
SITE Program "conduct additional demonstrations".
Matching Technologies To Specific Sites Was Difficult And
Time Consuming
While 21 technologies are currently matched to Superfund sites,
we found that this matching has been a very slow process. For
example, in our sample of nine technologies, site selection
frequently took about a year or longer. For these nine technolo-
gies, the time to match sites varied from 0 to 17 months. Table
1 shows the time required for site selection for these nine
technologies.
TABLE 1. TIME TAKEN IN SITE SELECTION
Months Required Demonstration
Technology for Site Selection Started as
Developer Solicitation as of 9/30/89 of 9/30/89
1. IWT 001 12 I/ yes
2. Terra Vac 001 6 yes
3. Ogden 001 0 2/ no
4. Retech 002 11 no
5. Chemfix 002 12 yes
6. Ultrox 003 4 yes
7. Toxic
Treatments 003 4 ,3_/ yes
8. Biotrol 003 ' 11 yes
9. Silicate 003 17 4/ no
I/ While a site was submitted with the application, the site
owner did not give official permission to conduct the demon-
stration until a year after the technology had been accepted
into the program.
2/ While a site was submitted with the application, as of 9/30/89
problems with civic concerns have not allowed the
demonstration to take place.
3/ While a site was submitted with the application, official site
selection took four months.
4/ The SITE Program was negotiating for a replacement site for
this technology as of 9/30/89.
13
-------
We obtained the above information from our file review of the
technologies. We also talked to the technology project managers,
who concurred with these dates for their individual technology's
site selection. One SITE official in Headquarters told us some
of these dates were different from the dates they had in Head-
quarters files. However, we believe that our file review and
discussions with the project managers provided us with accurate
dates.
A December 20, 1988, Congressional Research Service report also
found site matching to be a major problem. The report indicated
that EPA officials stated site matching was an impediment to the
SITE Program. This report also showed that site matching was a
major weakness of the program, causing time delays in the overall
program.
During our audit, we found several reasons why these difficul-
ties occurred. These reasons were: (1) in the past an inventory
of sites which could be used for demonstrations was not avail-
able; (2) many regional officials were not familiar with the SITE
Program and were unwilling to recommend sites without incentives
for them, or were reluctant to participate in the program due to
time constraints on performing cleanup work; (3) technology
developers were involved in the approval process and could reject
sites they perceived as not ideal for them; (4) 10 (28 percent)
of the 36 technology developers "exited" the Demonstration
Program; and (5) some technologies were so narrow in scope of
treatment that EPA has had trouble selecting a suitable site.
SITE Officials Repeatedly Asked Regions To Volunteer
Prospective Sites Due To A Lack Of A Site Inventory
For the first three solicitations, EPA did not have an inven-
tory of available Superfund sites to use for the technology
demonstrations. SITE officials did not know the characteristics
of the universe of available sites so that they themselves could
identify suitable locations for demonstrations. Instead these
officials relied on the regions to volunteer sites. However,
SITE officials told us they believed they did not have authority
to require, but could only urge, regions to suggest sites. SITE
officials also relied on the developers, who could propose a site
along with their applications to the program. But in most cases
the technology was first accepted into the SITE Program, and then
a search for a suitable site, was made. We believe that the
reliance placed on the regions and developers to suggest sites
was the major cause of the problem with site matching.
14
-------
At the end of fiscal 1988 there were 797 sites on the National
Priorities List (NPL) and another 378 sites that have been
proposed for listing. In addition, there are several thousand
more >non-NPL sites. This is an enormous number of sites from
which the SITE Program has to select. However, this large volume
also complicates the task SITE officials have in sorting through
them to find likely candidates at which to conduct demonstra-
tions. We believe that this points to the importance of having
a viable identified inventory of sites already prepared for
demonstrations.
One of SARA'S criteria was that EPA should have an inventory of
sites established for demonstrations. However/ we found that
SITE officials had not done this until very recently and had
instead been relying on regional officials to suggest sites.
In fact, ORO and OSWER officials have repeatedly urged regional
officials to nominate Superfund sites for demonstrations of the
SITE technologies. From the beginning of the program, SITE offi-
cials believed it was more practical to first determine which
technologies would be in the program, and then select suitable
sites for the demonstrations. However, in a July 15, 1988,
memorandum, the OSWER Director of the Office of Program Manage-
ment and Technology indicated that new procedures were being
set up for the selection of demonstration sites. These new
procedures would help expedite the technology/site matching by
collecting information on potential sites prior to the admittance
of technologies into the program. Matching of sites would then
be made from this inventory of information. The memorandum also
indicated that the proposed sites would also be evaluated for
future technology/site matches. This is essentially what SARA
seemed to be requiring.
However, in a later memorandum dated March 30, 1989, the Assist-
ant Administrators for OSWER and ORD again asked the regions to
consider Superfund sites as possible locations for the next round
of demonstrations. The memorandum stated that to expedite the
site matching process, the offices would work more closely with
the regions on an informal basis to identify potential sites.
However, it was only a request for potential demonstration sites.
There was no formal direction to participate. Finally, in an
August 31, 1989, memorandum on nominating Superfund sites for
demonstration purposes, the regions were yet again asked to
nominate sites. The memorandum encouraged the regions to con-
sider sites and identify those for a technology demonstration.
During our review, project managers told us that there was not
enough awareness among the regions during the site selection
process and that the regions had not been involved enough in the
15
-------
process. One program manager stated that the SITE Program it-
self, through its contractors, had acquired the majority of sites
used for demonstrations, rather than rely on what the regions had
submitted.
We discussed this site matching process with SITE officials at
RREL, and they agreed that it was a problem. The officials told
us that while the SITE Program is the flagship program of ORD,
the regions do not need the SITE Program. However, the officials
stated the SITE Program needs the regions to be involved and to
volunteer sites for demonstrations. We believe that if the SITE
Program is indeed the flagship of ORD, that the regions should be
more actively involved in providing Superfund sites for demon-
stration of technologies. Further, we believe that the regions
should be provided incentives to offer sites for technology
demonstrations.
According to Headquarters SITE officials, a Superfund site selec-
tion support system has been developed to conduct a preliminary
screening of NFL sites for potential demonstrations. A data base
will be used to help facilitate site selection for technologies
admitted under solicitation 004. To date, the data base contains
information on sites with RODs scheduled for 3rd and 4th quarters
in fiscal 1990 and all of fiscal 1991. The goal is to conduct
demonstrations on those sites early enough in the Remedial
Investigation/Feasibility Study (RI/FS) process to determine
whether the technology may be useful for cleanup of the site.
According to SITE officials, it is important to note that infor-
mation on sites which have an unsigned ROD is limited to the
description provided in NPL background documents. These descrip-
tions usually contain general information on the type of site,
waste source, waste media, and contaminants. Therefore7 the data
base can only serve as a rough screen of sites that may meet
technology requirements. Additional information will need to be
gathered from regional offices on more detailed site characteris-
tics, status, logistics, and regional interest before specific
sites can be identified. In our review, we found that the first
computer runs of the system were being prepared during October
1989. A SITE official told us that they are currently refining
the system.
We commend SITE officials for developing this data base inventory
and believe this is a positive action in helping to identify
potential sites. However, the SITE Program will still have to
rely on the regions to provide timely and accurate detailed
information on specific sites identified in the data base.
16
-------
Regional Superfund Officials Have Concerns On The SITE Program
Some regional Superfund officials are reluctant to participate
in the program due to the time constraints on performing clean-
ups. In fact, we learned that RPMs and OSCs are under signifi-
cant pressure to complete their cleanup work within established
timeframes. For this reason many are unwilling to get involved
in a SITE demonstration. In a discussion we had with a Superfund
cleanup contractor, the contractor also indicated that RPMs and
OSCs are under tremendous pressure and have critical dates and
milestones they must meet. With this pressure, a SITE
demonstration may be another factor that just adds to their
problems. For example, a demonstration requires additional time
consuming work and planning from the OSC and RPM, such as obtain-
ing permits, gaining acceptance by responsible parties, etc.
A further issue is that some RPMs and OSCs may be unwilling to
recommend sites because there is little or no incentive for them
to do so. One cleanup contractor we interviewed reinforced the
idea that RPMs and OSCs do not have an incentive to participate
in the SITE Program. The contractor told us this is due to the
fact that little documented information is available on the
detailed results of demonstrations to convince the RFMs and OSCs
that it is worthwhile to participate. In addition, the regions
are not tracked or held accountable for participation in the SITE
Program. However, they are tracked and held accountable for
cleanup of Superfund sites. Our telephone survey of RPMs and
OSCs also indicated*that they place more emphasis on.site
cleanup. They stated that:
— demonstrations interfere with on-going cleanup
activities;
-- time constraints dictated against being involved in the
program;
— demonstrations delay cleanup at sites;
— high potential risks are involved for both the developer
and OSC;
— too much red tape is involved which places a limitation
on both the program and participant in the program;
— the SITE Program lacks an adequate information channel on
how to become a participant in the program; and
— there is a lack of overall procedures when setting up
some SITE demonstrations.
17
-------
We found that the Agency's Strategic Planning and Management
System (SPMS), which tracks critical milestones and projects,
does not contain or track any element of the SITE Program. We
also found that the Superfund Comprehensive Accomplishments Plan
(SCAP), which is used to track Headquarters and regional Super-
fund data and accomplishments, does not contain any SITE Program
elements. We believe that a program of this scope and importance
to cleaning up Superfund sites should be included and tracked
within one or both of these systems. This would help provide the
incentive needed for the regions to participate in the Demonstra-
tion Program, which, in turn, would provide the SITE Program with
the necessary regional support to recommend Superfund sites for
technology demonstrations.
Additional Concerns Have Impacted SITE Matching For
Technology Demonstrations
During our review, we identified other concerns that have
affected site selection and thus the number of technology demon-
strations conducted. These concerns include developers that
turned down sites, exited from the program, and proposed tech-
nologies with a limited scope of treatment. EPA must also
address these concerns to enable the Demonstration Program to
have a significant impact on cleaning up the nation's hazardous
waste sites.
We found that developers have the authority to turn down sites
that do not meet their treatment requirements. This has caused
delays because developers naturally want the best possible site
conditions to test their respective technologies. In our review 2
of nine technologies, we found four of the nine developers turned
down at least one site. One of the developers turned down two
sites and another, three sites before agreeing on a site to test
the technology. In the former case, it took 12 months to finally
match the technology to a site, and in the latter case 11 months.
SITE officials told us that in the past they have worked with the
developers on selection of sites. In addition, they told us that
in the future they will definitely exit a developer from the pro-
gram after one or two chances during the site selection process.
We believe EPA must be firm in this area to reduce the amount of
time involved.
A concern related to this issue of developers turning down a
site is the SITE "exit" program. As of October 1, 1989, 10 j
(28 percent) of the 36 technology developers selected from the •
beginning of the program through the third solicitation have •
exited for various reasons. In 4 of these 10 cases, EPA and the
developers could not agree on a location for the demonstration.
For example, we found the following:
18
-------
— One of the technologies was selected in September 1986
and was not exited until September 1989. In three years,
a technology demonstration had .never been scheduled
because a suitable site had not been found.
— The second technology was accepted in September 1987 and
was exited in September 1989. Three sites were consider-
ed; however, because it was difficult to find a site that
met the technology requirements, the developer rejected
all sites and exited the program.
— The third technology was accepted in May 1988 and was
exited in September 1989. EPA inspected four sites that
may have been suitable for the technology; however, the
developer decided to exit the program.
— The fourth technology was selected in September 1987 and
was exited in September 1988. One of the major reasons
it was exited was due to site selection problems in
choosing a site to adequately demonstrate the technology.
This is an issue that impacts on the total number of technologies
available to be demonstrated in the SITE Program. In addition,
EPA spent $537,000 on these technologies without demonstrations
ever being performed. We believe Demonstration Program officials
should thoroughly review and analyze the reasons developers have
exited the program. The lessons learned from this analysis could
then be used as part of the selection criteria for future solici-
tations .
Another area of concern is that a technology may.be so narrow in
its scope of treatment that EPA and the developer may have
trouble selecting a suitable site. Because a technology may have
inherent limitations, it can prove difficult to find an adequate
site for testing. For one technology in our review, we found six
months were spent trying to find a site that had a limited amount
of organic and metal waste to conduct the demonstration because
of the treatment capability of the technology. SARA Section
311(b)(7)(c) requires that "the availability of sites for demon-
strating such technologies" be one factor considered in the
selection of technologies for the program. In addition, EPA's
1986 SITE Strategy and Program Plan states that:
"because of numerous waste types and complex
settings at Superfund sites, a single innovative
technology will seldom be a sufficient remedy for
an entire facility. Such technologies, however,
may be very useful as part of an overall process-
ing scheme and will be demonstrated, where
appropriate."
19
-------
EPA's Management Review of the Superfund Program report supports
this. The report states:
"Other expansions have been suggested by
those familiar with the SITE program.
Evaluating combinations of technologies is
one example. Currently, the SITE program
focuses on individual technologies, but often
two or more technologies must be combined to
treat a waste. For example, soil contami-
nated with metals and volatile organics might
require use of a thermal process to extract
the organics, followed by use of a washing
process to chemically remove the metals.
Each of these steps involve subsequent steps
to treat the extracted organics and metals..
It may be difficult to find vendors willing
to team-up on site demonstrations, but to
date a focused effort has not been made to do
so."
We believe the SITE Program should investigate the possibilities
of using more than one technology for some demonstrations to more
completely address the realities of Superfund site cleanups.
ORD COMMENTS Aj
-------
RECOMMENDATIONS
We recommend that the Assistant Administrator for Research and
Development:
1. Review and analyze the reasons why developers exit the
Demonstration Program so that this information could be
used in selecting future technologies.
2. Initiate SITE technology demonstrations that would
include use of more than one technology for the
demonstration.
3. Formalize the number of site locations at which a
developer can refuse to conduct a demonstration.
We *recommend that the Assistant Administrator for Solid Waste and
Emergency Response:
1. Add SITE demonstrations to the performance indicators or
elements of the SPMS or SCAP management tracking
systems.
2. Explore and develop other incentives for regional RPMs
and OSCs to allow their sites to be used in SITE
technology demonstrations.
21
-------
2. IMPROVEMENTS NEEDED IN REPORTING RESULTS FOR COMPLETED
DEMONSTRATIONS
The SITE Program has been adversely impacted by lengthy reporting
periods and delays in issuing the final Technology Evaluation and
Applications Analysis Reports. For example, TBRs have taken as
long as 17 months and AARs as long as 22 months to be published
(see Exhibit C). As a result, little reliable detailed
information is available to guide cleanup decision-makers.
While EPA is using many different techniques for distributing
demonstration information, users consider the final reports
issued at the end of the demonstrations to be the most reliable.
OSCs and RPMs advised us that they do not have sufficient,
timely information on these new technologies with which to make
decisions on their use. We believe this has hindered the direct
impact of the SITE Program on the development and use of these
innovative technologies. While SITE officials have made some
improvements, we believe additional emphasis is needed on the
reporting process. The SITE Program must increase the efforts to
produce more timely final reports so that users will have the
means to consider innovative technologies for hazardous waste
cleanups.
Reporting Demonstration Results IsSlow
We found the results of the completed demonstrations can take 22
months to be published. Even with recent increased management
emphasis in this area, reports have still been delayed. In
addition, we found actual distribution of the printed reports can
often take many months beyond the published date. The reports
are critical because they are the principal means of notifying
RPMs, OSCs and Superfund cleanup contractors about in-depth
demonstration results.
The in-depth results of the technology demonstrations are
presented in two reports called the TER and AAR. These two
reports are prepared following the completion of each-
demonstration .
A. Technology Evaluation Report. This report focuses on the
actual demonstration. It presents a discussion on how
the demonstration was conducted; the results that were
obtained; problems that were incurred in the demonstra-
tion and process; and advantages and limitations of the
technology. This report is intended for the person who
requires a detailed understanding of the technology, the
logistics of implementing the demonstration, the methodo-
logy used in conducting the demonstration, and the
performance of the technology during the demonstration.
22
-------
8. Applications Analysis Report. The purpose of this report
is to provide the decision-maker with enough information
that he/she can make a preliminary screening of the
applicability of the technology to a specific problem.
This report combines information from the demonstration
project, other demonstrations/ treatability studies,
pilot studies, and other literature. It provides the
reader with an understanding of the technology, the
types of wastes and waste matrix for which it may be
applicable, where it may fit into the treatment process,
the limitations of the technology, and its economics.
The Applications Analysis Report is designed for use
during preliminary screening of treatment alternatives
for a specific site.
From our review and discussions with SITE officials, we learned
there are other products designed to provide information quickly.
These include technology profiles, fact sheets, demonstration
bulletins, an electronic bulletin board, journal articles, pro-
ject summaries, and the annual SITE report to Congress. How-
ever, as previously stated, the TER and the AAR are the principal
means of getting an in-depth understanding of the evaluation and
analysis to individuals who make technology selection decisions
for cleaning up hazardous waste sites. In fact, the Director of
RREL has written that the AAR is the most significant and widely
distributed document in the program.
SARA Section 311(b)(5)(H) and (I) state that each demonstration
project should be completed within such time as is established
in the Demonstration Plan, and that EPA may extend any deadline
by mutual agreement with the developer. He found that SITE
officials continually revised reporting milestones, we also
found in our telephone survey that developers were concerned
about lengthy report preparation and review periods.
In a July 1, 1988, memorandum that summarized a June 1988 SITE
review meeting, the SITE officials acknowledged time delays in
reviewing and publishing reports. The memorandum indicated that
a new publication policy was being developed to address this
problem. Consequently, in an August 1, 1988, SITE Publication
Guidance memorandum/ SITE officials laid out new reporting
procedures and timing. From the completion of the demonstration,
the final TER was to be published in 331 days or about 11 months,
and the AAR in 391 days, or about 13 months. In other words, the
AAR should be published 60 days after the TER.
Because only two of our sample technologies had proceeded far
enough into the reporting phase for us to evaluate the process,
we reviewed timeframes on all completed reports.. These reports
had either been published or were in-process and were close to
being published at the time of our review.
23
-------
We found that EPA did not meet the timeframes established. In
fact, none of the TERs or AARs have been published on time.
Table 2 shows the number of months taken to publish the reports.
As Table 2 shows, the TERs varied from 1-6 months late and the
AARs were from 1-9 months late as of October 1, 1989. In
addition, IWT and CF Systems still had reports in-process at that
time.
TABLE 2
LENGTH OF TIME TO PUBLISH REPORTS
Date Months to Months to
Technology Demonstration Publish Publish
Developer Completed TER AAR
Haztech/Shirco August 1987 13 22
Hazcon October 1987 16 19
Shirco/
Rosetownship November 1987 17 19
American
Combustion January 1988 15 17 I/
April 1988 14 18 &/
Terra Vac April 1988 12 15
CF Systems September 1988 13 I/ 13 I/
I/ While the report was published, it had not been
officially distributed as of 9/30/89.
2J The report was not published as of 9/30/89.
SITE officials again recognized the reporting problem, and in a
March 8, 1989, memorandum the Director of RREL established new
milestones to speed up the process. The memorandum stated that
"While this schedule may appear to be somewhat severe, we can all
agree that the need to resolve this report backlog requires
severe actions." In addition, the memorandum contained a revised
milestone chart for the reports in-process at that time. The
memorandum further indicated that for these reports, the earlier
established SITE Publication Guidance would be met. We reviewed
the new dates and found that these intensive report milestones
were also missed in most cases. Table 3 on the following page
shows the expedited dates and actual dates as of October 1, 1989.
24
-------
TABLE 3
REVISED MILESTONES AND ACTUAL PUBLICATION DATES
Technology
Developer
Hazcon
Shirco/
Eosetownship
American
Combustion
IWT
Terra Vac
CF Systems
Month/Year of
TER Pub1i cat ion
Proposed
2/89
4/89
4/89
4/89
3/89
5/89
Actual
2/89
4/89
4/89
6/89
4/89
y
Month/Year of
AAR Publication
Proposed Actual
4/89
5/89
5/89
6/89
5/89
6/89
5/89
6/89
6/89 I/
7/89
I/
I/
This report was published in 6/89 but was not
officially distributed as of 9/30/89.
Not published as of 9/30/89.
Even with the expedited dates and increased management emphasis
and attention, in nine of the twelve cases the reports were late.
In addition to reviewing the dates for completing the reports, we
found that the distribution of the actual documents can be months
later than the published date shown on the report. Table 4 shows
the published date on the reports and the actual date of
distribution of the reports.
TABLE 4
PUBLICATION DATES VS. DISTRIBUTION DATES
TER
Technology Publication Distribution
Developer Date Date
AAR
Publication Distribution
Date Date
Haztec/
Shirco 9/88
Hazcon 2/89
Shirco/Rose-
township 4/89
American
Combustion 4/89
IWT 6/89
Terra Vac 4/89
3/89
4/89
6/89
6/89
9/89
6/89
6/89
5/89
6/89
6/89
I/
7/89
9/89
7/89
9/89
I/
I/
10/89
I/ Not published/distributed as of 9/30/89
25
-------
Agency officials believe the guidance issued in March 1989, on
SITE reports solved this problem. However, we found that most
reports were still delayed. The SITE Program must take increased
actions to shorten the reporting period and eliminate delays in
issuing the results of the technology demonstrations.
The Report Review Process Needs Improvement
One of the major causes of the delays was a long review process
that appeared to go back and forth between EPA, the contractor
writing the reports, the developer, and the various other parties
that reviewed the reports for the technical details. Because of
the technical nature of the reports and the large volume of
samples taken to authenticate demonstration results, a contractor
writes the reports. We found that extensive rewrites and changes
delayed the reporting process. Some of the concerns raised were
due to quality assurance/quality control (QA/QC) aspects that had
to be explained and covered satisfactorily before the report
could be finalized.
We also found some additional reasons for the delays. For
example, there were delays in determining the format for the
first reports. EPA officials told us that they were breaking
ground with these reports and were trying to determine what the
TER and AAR should contain. This took time early in the program
to finalize the first reports. In addition, ORD was adjusting
from their normal long range focus (3 to 5 years) on research to
a shorter period (2 years and lass) involved in the SITE process.
For two of the nine technologies in our sample that had reached
the report stage, long review processes delayed issuance of the
final reports. One technology, IWT, was originally supposed
to have a TER published in January 1989. However, it was not
published until June 1989. The AAR for the technology was orig-
inally due to be published in February 1989, but as of October 1,
1989, it had not been published. The other technology, Terra
vac, had a review period that ran for over eight months. Both
technologies averaged about three or four rewrites for their
reports.
In discussions with project managers, we learned about many
concerns with the report process. First, the project managers
indicated that the report process had been a learning process for
both EPA and the SITE contractors. There had been a lack of
communication between EPA and the contractors on how the final
reports would appear and what they would actually contain. In
addition, initially there was an internal debate between ORD and
OSWER on how the reports would eventually be structured. The
project managers also stated the review process takes too long
for reports. Three project managers indicated that there were
26
-------
problems with OSWER and Headquarters reviews and that the reviews
in many cases have taken too long. In fact, we received one
comment that the review process can take longer than the process
to have the technology actually demonstrated and tested. Project
managers also criticized the large number of reviewers involved
and the time necessary to incorporate the reviewers' comments
into the reports.
The March 1989 guidance on SITE reports acknowledges that the
revised schedule for completed demonstrations rests on the
assumption that the reports received from the contractors will
not require extensive changes or rewrites. The assumption is
based on the fact that most of the reports had been through two
or three drafts already up to that point. This memorandum
further discusses ongoing projects by stating:
"For ongoing projects, indications are that the
schedule outlined in the SITE Publications
Guidance can be met. A key factor will be the
ability to produce only one draft which requires
ORD/OSWER review and then a final report."
The memorandum further stated that if EPA receives drafts of
sufficient quality that major rewrites would not be necessary,
then the guidance schedule could be met.
In discussions with SITE Headquarters officials, they told us
that the report process had been corrected by the March memoran-
dum. However/ in a discussion on reports, SITE officials at RREL
indicated that in the last few months the SITE project managers
were writing some of the reports because the reports coming from
the contractors were still not of the sufficient quality they
wanted. While we recognize and commend the Agency's efforts in
this area, we believe that the issuance of these reports is still
a significant problem. We believe that additional attention
should be directed to this issue due to the Importance of this
published information to decision-makers who clean up the
nation's hazardous waste sites.
One additional aspect is the fact that typically ORD's focus is
long range (3-5 years) in duration. However, the Demonstration
Program focus has been a period of about 2 years. This has put a
lot of pressure on SITE management to adjust to a much shorter
term focus. Currently, SITE'management has set a new timeframe
of 18 months for the total SITE technology demonstration process
from beginning to end. The 18-month timeframe will add further
pressure on SITE management to accomplish the entire process
under this shorter timefr«m«
27
-------
Timeliness and productivity issues have been receiving increas-
ing attention within the Federal Government in recent years.
The President's Quality and Productivity Improvement Program
has spearheaded this effort under the auspices of the Office of
Management and Budget (OMB). One of this program's recent
projects involved the Lewis Research Center of the National
Aeronautics and Space Administration (NASA). After studying
their reporting process, the center made substantial improvement
in issuing technical publications, reducing processing time by
nearly 50 percent. NASA did the following to help accomplish
this.
— Established a cross-functional team to identify
improvements to the report generation process. They
identified the specific trouble spots or choke points,
addressed these specific concerns, and eliminated any
steps which did not add value to the process.
— Considered new technologies to enhance the publishing
process. They began to do more things in-house by using
newer equipment.
— Reordered the priorities of project managers to raise the
emphasis on writing the technical reports.
Total Quality Management (TQM) ia another OMB productivity effort
designed to improve timeliness and productivity at Federal
agencies. He discussed our reporting concerns with two Agency
components responsible for implementing TQM at EPA: the Resource
Management Division in the Office of Comptroller; and the Office
of Modeling, Monitoring Systems, and Quality Assurance in ORD.
This ORD office has previously helped the SITE Program to address
some QA/QC issues on a demonstration. However, they are also
able to perform detailed Management Systems Reviews to identify
and help resolve specific choke points and inefficiencies in ORD
processes, such as these reporting problems. These types of
reviews are very valuable because they provide an independent
appraisal of the process in question.
Because of the importance of the reports, the SITE Program needs
to resolve its report preparation, review, and publication
difficulties. This would help provide the detailed report
results more promptly to those who make decisions regarding
cleanup solutions.
28
-------
Lack Of Available Data On SITE Technologies Adversely
Impacts Superfund Decision-Makers
Based on our telephone survey, RPMs, OSCs, Superfund cleanup
contractors, and' technology developers believe a lack of
information exists on SITE technologies and corresponding data.
In addition, regional decision-makers responsible for selecting
and recommending technologies to remediate Superfund sites were
not always familiar with the program. We found that
approximately 30 percent of our statistically valid sample of
OSCs and RPMs had never heard of the SITE Program or were not
familiar enough with the program to answer our survey questions.
These users are unable to make intelligent decisions on SITE
technologies in considering them for site cleanups. We were told
that because of a lack of published information and reliable
results, the RPMs, OSCs, and contractors were not willing to rely
on SITE innovative technologies to clean up sites.
EPA's December 1986 SITE Strategy and Program Plan indicates the
importance of disseminating information on technology demonstra-
tions. It states that:
"The dissemination of information on technology
demonstrations or technology transfer will be most
important once results from demonstrations are
available. If alternative technologies are to be
applied more broadly at Superfund sites, Agency
personnel, engineers, and others must have access
to reliable technical information."
In addition, this document statedi
"The shortage of standardized performance and cost
data is inhibiting the commercial development and
use of innovative technologies. The potential user
is unable to judge the incremental technical
benefits and costs of these technologies compared
to conventional technologies because there are no
consistent and uniform testing protocols and no
adequate base line data on performance and cost for
conventional technologies. The major objective of
the demonstration program is to develop reliable
cost and performance information on innovative
alternative technologies so that they can be
adequately considered in Superfund decision
making."
29
-------
During our review, we talked to an extensive sample of RPMs,
OSCs, Superfund cleanup contractors, and technology developers
While most OSCs and RPMs familiar with the program thought it
was a very good idea, to date they had not received sufficient
information about the results of specific demonstrations to
encourage them to use innovative technologies. Few final reports
and the general lack of reliable information inhibited their
willingness to increase the use of innovative technologies to
more permanently clean up Superfund sites. Exhibit D summarizes
the program's strengths and weaknesses pointed out by our survey
respondents. In addition, the following are some comments made
by the RPMs and OSCs.
— There is not enough exposure of the program itself,
resulting in a communication breakdown between the
program and RPMs.
— Many RPMs do not even know what the acronym SITE means.
— SITE lacks an adequate information channel on how
to instruct someone to become a participant in the
program.
— SITE is a fairly new program with risky and unproven
technologies.
* •*fFX»
— The program is limited in the scope of technologies and''
their use on particular sites. In fact, some may be
usable at only one specific site.
— Technologies are not readily available for use.
— The potential risks are high for both the developer and
the OSC.
Our survey of Superfund cleanup contractors disclosed that they
were concerned about the reporting process and said:
— One contractor told us that a weakness is an
"apparent lack of enthusiasm" in getting out the
results of completed demonstrations.
— Two other contractors stated that a weakness of the
program was the delay involved in the dissemination
of SITE demonstration test results to potential users
of the technologies.
— One contractor told us that demonstration results are
not disseminated widely enough for people to know what
is occurring in the program.
30
-------
In addition, we discussed the program with developers who parti-
cipated in it. These developers frequently expressed concerns
that the reporting process took too long. One developer also
indicated he believed that no one in the field (RPMs and OSCs)
knows very much about the SITE technologies.
Our telephone survey also revealed some strengths, which
encourage the continued reporting of the results of the SITE
Program.
— One contractor indicated that SITE generates analysis
reports on completed testings that provide documented
information for decision-makers.
— Another contractor stated that SITE offered a basis for
establishing proof of the capability of new and
innovative processes.
— One developer stated that having an independent
evaluation of his technology and developing credible
data were reasons for participating in the program.
We discussed these results with SITE officials, who believed a
high and rapid turnover rate among OSCs and RPMs caused them to
be unfamiliar with the SITE Program. However, we found that the
OSCs in our telephone survey had been in their positions an
average of about 5 1/2 years, and the RPMs an average of almost
four years. In addition, in our survey there was no correlation
between the OSCs and RPMs who were not familiar with the program
and those who had only been in the position for a short period of
time.
SITE officials also stated that coverage of the SITE Program will
be included in a new seven week training curriculum for OSCs and
RPMs. However, according to officials from OSWER's Office of
Program Management and Technology and Office of Emergency and
Remedial Response, who designed and implemented the new curricu-
lum, it will be only offered to new OSCs and RPMs with less than
six months experience in the position. Because the program was
not scheduled to start until January 1990, OSCs and PRMs who were
hired prior to July 1989, would not be eligible to attend the
training course. Since many of the OSCs and RPMs we surveyed who
were unfamiliar with the SITE Program would not be eligible to
attend, SITE officials need to consider additional ways to
effectively inform them about the program.
We also conducted a telephone survey of five technology
developers who had completed their field demonstrations. All
five developers stated that exposure and evaluation of their
31
-------
technology were the reasons they participated in the Demonstra"-'
tion Program. Two of the five stated that participation in the
Demonstration Program has led to wider uses of their technology.
However, two others believed it was too soon to measure, and the
remaining developer said participation in the program has not led
to any additional uses of their technology.
ORD COMMENTS AND PIG EVALUATION
In his March 12, 1990, memorandum, the Assistant Administrator
for Research and Development generally agreed with our
recommendations (see Appendix 1). The Assistant Administrator
also provided comments regarding our issues. Specifically, while
acknowledging the report timeliness problem, he reemphasized that
ORD has already taken some actions to shorten the report
publication times, and that there are other ways ORD has been
distributing pertinent information. We acknowledged all this
information in our draft report, and we want to again emphasize
that cleanup decision-makers rely on the final published reports
to assist them in their activities. Appendix 2 is our detailed
analysis of ORD's comments.
RECOMMENDATIONS
We recommend that the Assistant Administrator for Research and
Development:
1. Establish formal accountability for SITE management
for meeting reporting milestones.
2. Consider alternatives for performing a detailed
review of the process for preparing and issuing
SITE reports, such as a Management Systems Review.
3. Evaluate publication alternatives to speed up the
actual distribution process.
4. Prepare a video tape or presentation of the Demonstra-
tion Program's goals and objectives to be presented
to all Superfund personnel in the regions and at Head-
quarters .
32
-------
3. IMPROVEMENTS NEEDED IN ENSURING ADEQUATE CONTRACTORS'
PERFORMANCE AND COST CONTROL
The developers in our telephone survey expressed concerns that
SPA's contractors had been slow and inexperienced, and, in some
cases, generated too much data. A recent study by the Congress-
ional Research Service (CRS) has also shown that developers
complained about the performance of the contractors. During
our file review of the nine sample demonstrations, we found
indications of significant EPA concerns regarding contractors'
performance. In our sample we also found frequent and signifi-
cant cost increases and time extensions. In fact, in our sample
virtually all of the contractor requests for funding increases
were granted exactly as requested. SITE officials stated that
while they try to hold down the contractors' costs, their real
priority has been, instead, to issue the reports.
Sound management practices dictate that a certain level of
performance is expected of contractors. Management is to eval-
uate contractor performance in determining whether requested
funding increases and contractual award fees are to be approved.
EPA may not be promoting effective and prudent cost control in
the Demonstration Program because of pressure to issue the
reports. In addition, project managers and SITE management may
be more concerned with the technical quality of the contractors'
work than with their costs.
Concerns About Contractors' Performance
In our telephone survey, RPMs, OSCs and developers made the
following comments about the SITE contractors!
— The demonstration would have run more smoothly if
contractor personnel had more experience dealing
with hazardous waste.
— Contractors have a general lack of knowledge in analyzing
the data on new innovative technologies.
— The contractors required significant amounts of time to
deal with administrative issues, and many contractor
personnel were inexperienced.
— A developer indicated that they had to guide the
contractor in writing the reports.
-- The contractors take too long to write the reports and
are not meeting their timetables.
— Some contractors preparing the reports are unaware of the
SITE Program's overall objectives.
33
-------
A December 20, 1988, CRS study also expressed concerns about
unsatisfactory contractor performance in the SITE Program.
Twenty-eight industry representatives were interviewed, and eight
were dissatisfied overall with the contractors. At the time of
the study, five of these eight developers had demonstrated their
technologies in the SITE Program. The report observed that four
of these five representatives complained about their experience
and were at least somewhat dissatisfied with the EPA contractors.
The report concluded that future demonstrations may be hindered
unless the problems with the SITE contractors are resolved.
Not only the developers, but also EPA employees working on the
Demonstration Program have expressed concerns about the contrac-
tors. For example, the SITE QA/QC officer has repeatedly raised
concerns over their performance. The December 1986.SITE Strategy
and Program Plan discussed the QA/QC program as a critical
element of each SITE demonstration project, because QA/QC
procedures ensure that data collected are of known and acceptable
quality for their intended use. We discussed this with the QA/QC
officer, and he stated that when the SITE contractors were
awarded the contracts, he met with and gave them a comprehensive
instruction package. This package detailed what should be
included in their QA Project Plans (QAPP) for the technology
demonstrations. However, he told us that the contractors did not
always follow the guidance and, as a result, have wasted time.
He told us he plans to publish a pocket guide in fiscal 1990 for
contractors to use in preparing the plans.
During our review of the nine sample technology files, we found
the QA/QC officer expressed numerous specific concerns. For
example, he made this comment in a December 1988, note to the
file on one SITE contractor's draft plansi
"In light of (my) review, I personally feel
that (the contractor) ought to be discouraged
from ever producing a Sampling and Analytical
Plan and a QA Project Plan in the RREL SITE
program without, first, thinking. This could
be accomplished either by having them revise
the planning documents at their own expense
or by not paying them for that which they
have already produced."
In September 1987, the QA/QC officer reviewed a first draft of
another contractor's QAPP, and rated the plan unacceptable. He
indicated that a critical concern existed in a test method, and
that major concerns existed related to analytical methods and to
the lack of important information concerning sample handling.
The plan was subsequently approved after these concerns were
addressed.
34
-------
In a December 1988, review of a draft TER, the QA/QC officer
commented on the contractor's report by stating: "While covering
most of the critical issues, this report is poorly organized and
internally inconsistent." In this same review he expressed
concerns on the validity of test conclusions by stating:
"This . . . casts further doubt on the
ability to predict the time required for site
remediation, also suggested in the report.
While a decrease is evident, clearly it is
impossible to determine from this . . . test
period how long it would take to accomplish
site remediation. Perhaps a longer test
period would give better predictions regard-
ing time required for remediation. Effec-
tiveness of. the technology in various soil
types cannot be adequately addressed by the
data."
A later discussion with the project manager indicated that even
now the technology may not be entirely proven. However, the
project manager did state that the SITE Program has made some
gains with documenting data for this technology, and that the
SITE Program report is the best available evaluation of this
technology.
During the review we found that for this same technology an
external reviewer also evaluated the draft TER in October 1988,
and a subsequent draft in December 1988. In December, the
reviewer indicated that the concerns expressed in October had not
been addressed/ statingt "We believe we performed a quality
review of the original document and subsequently provided
significant comments . . . There was a lack of response to over
80 percent of our comments." The reviewer indicated that if the
comments were not addressed then they would refuse to be
associated with the review of the report. The project manager
told us that the reviewer's comments were later addressed.
During our review, we also saw comments from the SITE economist
in a January 1989, memorandum regarding two different TERs. He
indicated that he believed the information did not fit the
overall format for the criteria established, and that there were
significant problems with both the quality of the analysis and
the accompanying narrative.
Project managers work more closely with the contractors than
either the QA/QC officer or the economist, and they also
expressed concerns about the contractors. For example, one
project manager indicated that he was concerned about the
contractors' report writing, especially getting then to deliver
draft reports which included all the needed information in the
35
-------
required format. Another project manager told us about a serious
problem with one of the subcontractors responsible for analyzing
sample test results. The QA/QC officer determined that the
subcontractor was not accurately following the QAPP, and he
ensured that the subcontractor corrected the problem. The
project manager advised us that while the mistake was corrected,
it could have adversely affected the test results if it had gone
undetected and uncorrected.
Little Emphasis Placed On Controlling Costs
During our review, we found that SITE management focused much
more on performing the demonstrations than on controlling costs.
In fact, according to the Director of the Superfund Technology
Demonstration Division at RREL, the priority in the last few
months has been issuing the reports rather than controlling
costs. SITE project managers we spoke with generally indicated
that the SITE Program is a high priority program with funding
available, and it is not a problem getting the available funds.
However, in our opinion, contractor cost control is especially
important due to the large amount of the Demonstration Program's
resources which go to the contractors. For fiscals 1987 through
1989, these contractors absorbed approximately 80 percent of the
program's $34.6 million budget.
ORD primarily used cost reimbursement contracts to obtain SITE '*
contractor services. Agency guidance states that under these
cost reimbursement contracts the contractor has little incentive
to control costs. Instead, the contractor is only responsible
for making a good faith effort to complete the work at an
estimated cost. The government may be responsible for full
payment even if the final product is unsatisfactory, or it
exceeds the estimated cost. Federal Acquisition Regulation (FAR)
16.301-3 states that cost reimbursement contracts may only be
used when government monitoring of the contractor's work will
provide reasonable assurance that the contractor is using
efficient methods and effective cost controls. Consequently, a
significant amount of contract monitoring and oversight is
needed. Further, the guidance adds that Federal employees are
responsible for monitoring the efforts of contractors in order to
prevent waste of public funds and to obtain the required services
within the budgeted costs.
There are several types of contracts the government typically
uses in research and development efforts. The first two are cost
reimbursement contracts. Both reimburse the contractor for costs
incurred, but they differ in the method used to motivate the
contractor towards superior performance. The Cost-Plus-Award-Fee
(CPAF) contract involves periodic subjective evaluations of
the contractor's performance. FAR 16.404-2 states that CFAF
36
-------
contracts are suitable when, among other things: (l) the work to
be performed is such that it is neither feasible nor effective to
devise predetermined objective incentive targets applicable to
cost, technical performance, or schedule; and (2) any additional
administrative effort and cost required to monitor and evaluate
performance are justified by the expected benefits. The Cost'
Pius-Incentive-Fee (CPIF) contract establishes specific factors
to be objectively evaluated and rewarded, such as cost control,
quality of performance, or adherence to schedule.
Another type of contract used in research and development is the
Fixed-Price-Incentive-Fee (FPIF). In this case, the Government
and the contractor agree upon a fixed price; if the contractor's
costs exceed this amount, the contractor absorbs them. Further,
specific factors are identified for objective evaluation and
reward, as under the CPIF contract type.
One of the most significant differences in these three contract
types is the risk involved and who assumes it. Under the CPAF
contract, the Government assumes virtually all the risk, and the
contractor very little. The CPIF decreases the Government's risk
somewhat; and increases the contractor's, by making the contract-
or responsible for more specific performance. The FPIF places
even more risk on the contractor, by defining a fixed price.
Compared to still other contract types, the CFIF and FPIF are
considered to be relatively evenly balanced, i.e., they place
similar risk on both the Government and the contractor.
The SITE Program awarded two large CPAF support contracts on
November 17, 1987. These two contracts are both for a three-year
period from November 1987 to November 1990. In addition, a third
contractor was used on some of the early technologies. This
contractor was allowed to continue work on some demonstrations,
while on others was replaced by the two newer contractors.
Under the Demonstration Program contracts, ORO issues work
assignments for individual projects. The project managers are
responsible for day-to-day activities related to individual work
assignments. The project managers are the persons most know-
ledgeable about individual technology demonstrations performed
under the contract.
Project managers normally process two work assignments for each
of their demonstration projects. The first work assignment
covers planning the demonstration, and includes the following
activities:
— provision of technical assistance support to all
involved parties;
— development of site selection sampling requirements; and
37
-------
— preparation of the Demonstration Plan, including:
a. Demonstration Test Plan,
b. Quality Assurance Project Plan,
c. Sampling and Analysis Plan, and
d. Health and Safety Plan.
After the developer and EPA review and agree upon the Demonstra-
tion Plan, a second work assignment is necessary to implement it.
The contractor then: oversees the actual demonstration
activities on-site, especially the safety, QA/QC, and monitoring
activities; performs those activities specified in the
Demonstration Plan; and oversees equipment decontamination and
removal at the completion of the demonstration.
We found in our nine sample technologies that EPA granted many
cost increases. Table 5 shows the cost increases and their
percentages. The percentage of increase varied from about 16
percent to about 138 percent.
TABLE S
INCREASES TO ORIGINAL ESTIMATES FOR SITE COSTS
Technology Original Cost Revised
Developer Estimate Increase Cost
Reason? Complete Preparation of Demonstration Plan
TTUSA $119,795 $ 18,706 $138,501
Retech $115,196 $141,341 $256,537
Ultrox $ 99,608 $ 37,138 $136,746
Biotrol $109,693 $ 62,442 $172,135
Silicate $121,183 $155,009 $276,192
Reason; Conduct Demonstration and Write Report
Terra Vac $550,000
Reason: Other
Chemfix I/ $220,000
Chemfix I/ $400,000
Ogden 4/ $425,000
No Increase
IWT
$757,900
$236,200
$ 64,827
$458,662
$195,000
$1,307,900
$1,544,100
$284,827
$858,662
$620,000
% Of
Increase
15.62%
122.70% I/
37.28%
56.92%
127.91%
'137.80%
42.95%
29.47%
114.67%
45.88%
Totals $2,160,475 $2,127,225 $4,287,700
ssaaaasssaa aaaaasasaa aaaaaaaaaaa
98.46%
38
-------
!_/ 95.15% of the increase was due to treatability studies.
2.1 For'the Chemfix demonstration, the increase to complete the
preparation of the Demonstration Plan was due to the project
officer estimating one figure but SITE management believing
it was too high and reducing it. However, the higher figure
was later submitted by ^the contractor as its original
estimate and was approved.
I/ The project manager stated the second increase was due to a
low original estimate. This occurred because the
Demonstration Plan had not been completed yet, and so the
project manager had to estimate the amount to perform the
demonstration without the benefit of a completed plan. A
SITE official stated that the project manager underestimated
the contractor costs. The official indicated he did not
question the increase because he believed: the original
estimate was understated; the funds were available; and
the SITE Program had a high priority in terms of funding.
4.X The increase was due to reasons that pertained to increases
in sampling, waste blending, and permitting problems at the
projected demonstration site.
We realize that some of these problems, such as a change in
site location, are not necessarily related to contractor
performance, but may be due to EPA and/or developer actions.
However, completing the Demonstration Plan and the related
reports, including sampling results, QA/QC concerns, etc., are
to a large extent under the contractors' control. In addition,
during our review we observed and received numerous adverse
comments directed at the contractors' performance.
As previously discussed, the contracts ORD used were of the cost
reimbursement type. This means that as schedule delays occurred
on these demonstration projects, and contractor work efforts
expanded with the changed timeframes, costs invariably grew. In
Findings 1 and 2 of this report, we have discussed two of the
reasons for schedule delayss site matching problems and report-
ing process problems. While these are major factors, we also
found schedule delays due to such things as writing and revising
the Demonstration Plans, starting the fieldwork, conducting and
completing the fieldwork, modifying and replacing equipment, and
revising reports. Although-EPA and the developers frequently
played a role in these delays, it is clear from the documentation
we reviewed that portions of the delays were also due to the
contractors.
39
-------
r
SITE Program officials recognized this in a July 1, 1988,
memorandum, which stated that time delays for contractors'
reporting efforts needed to be addressed. The memorandum
indicated that project managers should meet with contractors to
discuss the project timeframes and deadlines, emphasizing that
these are level-of-effort contracts and that the award fees are
based on contractor performance. However, as previously
discussed, in a March 8, 1989, memorandum, the Director of RAEL
advised the Director of the Office of Environmental Engineering
and Technology Demonstration that there still had been problems
with contractor performance concerning quality of reports.
SITE officials advised ua that the EPA project managers did not
have sufficient time to perform such duties as writing the TERs
and AARs. They believe they must rely on the contractors to
perform the key duties of the program. These officials told us
that the project officers typically handle two to three
demonstration projects, as well as one or two emerging technology
projects, on an ongoing basis. They believe the most efficient
way to get information out of multiple technologies is for the
contractors to prepare reports under the project officers'
direction, not for project officers themselves to write reports.
The SITE officials believe they are providing clear oversight and
guidance to the contractors.
We recognize that giving more attention to cost control and
taking a tougher stand with SITE contractors imposes certain
practical considerations and difficulties. We commend the
efforts the SITE Program has made to date in this area. However,
the degree and scope of the continuing cost increases seem to
indicate that, by not fully challenging contractor cost, EPA may
be in effect telling the contractors that it is willing to
reimburse them for all cost growth. He believe that increased
attention to controlling contractors' expenditures is needed to
convey a strong message that the Government is unwilling to
accept unnecessary costs and time delays. This message is
especially important when one considers that EPA still has two
years remaining under SARA for the SITE Program. And with-the
large number of hazardous waste sites still awaiting cleanups, it
is very possible that this program will be reauthorized. This
further necessitates increased attention to cost control.
In addition, we acknowledge that the research and development
environment is an uncertain and, to some degree, risky one. How-
ever, in our opinion, not all elements of research and develop-
ment carry the exact same degree of risk. For example, the bench
or pilot-scale testing of a mechanical system would seem to
involve more risk than testing a system that had advanced beyond
this stage, to field testing. Further, risk would seem to
decline with increased experience gained in conducting such field
tests. Because of this, we believe that consideration needs to
40
-------
be given to using a contract type that more fairly balances risk
between the SITE contractors and the Government, and.encourages
increased contractor efforts in all key areas: quality,
timeliness, and cost control.
Award Fees Did Not Appear To Be Consistent With The
Increased Costa And Time Delays
We found that during the first 18 months of the contracts, SITE
contractor award fees averaged about 70 percent of the overall
amount available. Further, in our sample the overall program
cost increases averaged about 98 percent, some of which appears
to be due to contractor performance. Finally, none of the
demonstration projects have been completed in accordance with the
original milestones, for a variety of reasons.
The ORD contracts contain an average award fee provision of a
maximum of three percent, which is substantially less than that
allowed by law. The amount of the available award fee paid to a
contractor is based on a subjective evaluation by program
officials. The evaluation is supposed to take into consideration
such factors as the quality, timeliness, and cost effectiveness
of the product. The award fee is decided upon at stated
intervals during the contract, usually every six months. The
overall objective of the award fee is to motivate the contractor
in a positive way to improve poor performance or to continue
satisfactory performance.
Officials at RREL stated the award fees for the first year of the
contract averaged about 70 percent of the maximum. Further, we
reviewed the contractor award fee packages for the latest six-
month period. One contractor received over 69 percent of the
award fee total', and the other contractor received almost 73
percent of the total award fee. In the award fee summary of one
of the contractors, we found the following comment:
"Overall the contractor has done a very good
job in quality assurance. However, a problem
that has been observed is that frequently,
the QA plan was submitted to EPA without
adequate internal contractor review, and in
sane cases, was not forwarded to the contrac-
tor's own QA officer for their review before
submitting it to EPA."
This has been a recurring problem that we found throughout our
review of the technologies. This problem still seems to be
occurring as of the last contract award fee review process.
41
_
-------
Another comment given for one of the contractors was that, "on an
overall basis, the contractor has clearly improved since the last
evaluation and we are very satisfied with their performance,
except for certain items." One of these items was that there was
a lack of coordination between the contractor and the subcontrac-
tors, which caused some sampling problems during the demonstra-
tion. These problems were apparently addressed and corrected.
We noticed in our review of the six-month contractor evaluations
that there was very little mention of cost control and timeli-
ness. Most of the evaluations focused on the technical perfor-
mance of the contractors. Further, the overall award fee of
three percent is relatively small, and the contractors average
receiving about 70 percent (or a total fee of approximately two
percent) of it. The extra one percent fee, compared to signifi-
cant approved cost increases, may provide little incentive to
control costs.
Problems with EPA management of the award fee process were
identified by the Office of Inspector General (016) in a previous
review. Our audit report entitled "Management of Contracts
Supporting the Office of Solid Waste", Audit Report E1XM+8-11-
0047-9100209, dated March 13, 1989, showed that award fee
determinations were not communicated to contractors in a .timely
manner. In that audit, we found that EPA was not effectively
using award fees to motivate contractors. For the four CPAF ,
contracts in the sample, contractors were paid $526,709 in award-
fees during fiscal 1987. The payment of these award fees at
stated intervals during the life of the contract was intended to
motivate the contractor to improve poor performance or continue
satisfactory performance. However, we found that due to the low
priority placed on making award fee determinations, contractors
were not informed of the Agency's award fee decision until an
average of six months after the end of the evaluation period. As
a result, the Agency was not providing contractors with timely
feedback on their performance, so they could make necessary
improvements. More importantly, the Agency was paying contrac-
tors award fees without receiving the full benefit they could
provide in motivating contractors' future performance. Had the
award fee determinations been made timely, contractors may have
provided better quality products.
In addition, the 016 is currently performing a major review of
the management of the award'fee process in contract work for the
Superfund program. When completed, this report will provide a
broader picture of improvements needed in EPA's use of CPAF
contracts.
42
-------
We believe that because of these large cost increases, time
delays and comments made on contractor performance, the current
award fee process needs to be improved. SITE management needs to
ensure that the award fee determinations are adequately used to
communicate performance results and the need for high quality to
the contractors.
ORD COMMENTS AND QIC EVALUATION
In his March 12, 1990, memorandum, the Assistant Administrator
for Research and Development agreed with some of our
recommendations (see Appendix 1). He did not, however, agree
with our recommendation to retain some program functions in-
house, If contractor performance does not improve. We still
believe that the problem of contractors providing inadequate
drafts of reports has significantly contributed to the delays in
issuing the reports, and that, when necessary, performing this
function in-house would speed the process.
The Assistant Administrator also did not agree with the finding
itself. Specifically, while acknowledging that some difficulties
were experienced in the beginning of the program, he maintained
that the contractors in question have been replaced. However,
our reviews of project files, discussions with program officials,
and survey of cleanup personnel showed that the same types of
problems are continuing to occur with the replacement
contractors. Further, the Assistant Administrator reemphasized
some of the reasons for the coat increases discussed in our
report, reasons which we acknowledged in our draft report.
Finally, he stated that the Procurement and Contracts Management
Division believes that the CPAF contract type is the most
appropriate for the SITE Program, and that the award fees in this
program are lower than for other typical EPA contracts of this
type. We still believe, however, that this contracting mechanism
places virtually all of the risk on the Government, and that the
award fees on these two contracts are insignificant compared to
the cost increases realized on eight of the nine projects in our
sample. Appendix 2 is our detailed analysis of ORD's comments.
RECOMMENDATIONS
We recommend that the Assistant Administrator for Research and
Developments
1. Require that adequate justification be provided for
individual funding increases.
2. If contractor performance in key areas, such as report
writing, cannot be improved, retain those functions in-
house on future projects. If necessary, assign
additional staff to perform these functions.
43
_
-------
3. Reinforce to contractors and project managers the
importance of the award fee process and why the award
fees are used.
We recommend that the Assistant Adminstrator for Administration
and Resources Management:
1. Consider for the next contract period a contract type
that would more fairly balance risk between EPA and the
contractors.
44
-------
4- THE SITE DEMONSTRATION PROGRAM DISCOURAGED DISTRIBUTION OF
FEDERAL FUNDS TO DEVELOPERS
The Demonstration Program discouraged distribution of Federal
assistance to technology developers. Although EPA issued SITE
solicitation 001 before SARA was enacted (so no funding was
available), we found that solicitation 002 (issued after SARA)
did not inform technology developers that financial assistance
was available. In .solicitations 003 and 004, the RFP defined the
funding restrictions and required extensive financial information
be provided to EPA. As a result, as of October 1, 1989, no
developer had received funding assistance from the Demonstration
Program. We also found that four of five developers in our
survey believed funding was an impediment to the program, and two
stated, if given the opportunity,.they would not participate in
the program again due to the high cost. Further, lack of funding
may have discouraged developers of other promising technologies
from applying for admission to the program.
Although SARA permitted financial assistance for technology
developers, SITE officials were initially uncertain how to apply
the SARA guidelines. Eventually these officials determined that
a very conservative approach was needed. In addition, SITE
officials were concerned that providing funds to developers would
adversely impact the funding available for other program needs.
SITE Officials Were Initially Uncertain How To Provide
Funding
Although EPA issued the RFP for solicitation 001 prior to SARA'S
enactment, the RFPs for all other solicitations were issued after
the passage of SARA. However, it was not until solicitation 003
that EPA publicized the availability of Federal assistance
funding, due to the initial uncertainty of SITE officials
regarding exactly how to provide this funding.
From the beginning of the program, a number of developers have
expressed an interest in receiving financial assistance. How-
ever, in some cases a lack of funding may have hindered their
admission to the program. For example,-one\ technology developer
(International Technology Corporation) from the second solici-
tation expressed interest in receiving funding. Because the
developer could not fund the company's portion of the demonstra-
tion or obtain a funding commitment from a second party, SITE
officials placed them in a conditionally accepted status. Also,
SITE officials informed the company that EPA had not finalized
the guidelines for Federal assistance. Further, EPA informed the
company that if they were unsuccessful in gaining private fund-
ing, they may want to consider applying for Federal funds, but
that EPA could not guarantee them any funding. This developer
45
-------
never entered the Demonstration Program, and it appears that lack
of funding, including Federal assistance, may have been a contri-
buting factor.
SARA Section 311{b)(S)(J) authorizes the Agency to provide
Federal assistance to any applicant who can demonstrate that it
cannot obtain appropriate private financing on reasonable terms
and conditions sufficient to carry out its demonstration. The
total Federal funds provided to a developer for any full-scale
field demonstration project can not exceed either 50 percent of
the total cost of each project or $3 million. The Agency was
also restricted to spending no more than $10 million for
assistance for all demonstration projects under the program in
any one fiscal year. Further, the December 1986 SITE Strategy
and Program Plan stated that "Since funding is limited at the
outset, the Agency will not provide any Federal assistance for
any part of a full-scale field demonstration project unless the
developer can demonstrate that he cannot obtain appropriate
private financing on reasonable terms and conditions sufficient
to carry out the demonstration project without Federal
assistance."
SITE officials stated that after the passage of SARA, they were
not certain how to implement the requirements for funding tech-w
nology developers, and requested clarification from the Office of
General Counsel (OGC). According to the SITE officials, the
delay in receiving an answer from OGC prevented any offering of
Federal assistance in the first two solicitations. We discussed
this with OGC and were informed that OGC has no record of the
SITE Program's request for advice on funding assistance. Neither
SITE officials nor OGC could provide us any documentation of
these contacts. However, OGC told us that the Grants Administra-
tion Division (GAD) had contacted them to determine under which
regulation SITE funds were to be administered.
We also discussed this issue with GAD officials. They stated
that ORD had a discussion with GAD about the use of cooperative
agreements. According to a GAD official, a cooperative agree-
ment would be the appropriate instrument to\ use to provide funds
when there is substantial applicant work involvement, such as in
the SITE Program. The official further told us that other ORD
research cooperative agreements were awarded in about four
months. However, SITE officials told us they believed the
process for obtaining funding would be from six to nine months.
We believe that the timeframe SITE officials gave to prospective
applicants, i.e., requiring six to nine months for the funding
process, may have discouraged them from applying for assistance.
We believe this is a concern that ORD must clarify with GAD.
46
-------
Program Officials Took A Conservative Approach In Offering
Federal Assistance To Technology Developers
SITE officials stated that they decided to administer funding
very conservatively for several reasons. First, they stated that
Congressional staff had expressed concerns that EPA would be too
liberal in providing funding. We requested documentation of
this, but none was provided. Second, because SARA was not clear
on this issue, SITE officials were unsure how free or restrictive
Congress wanted them to be in providing funding to developers.
These'officials decided to take a more conservative approach.
Finally, they stated they were reluctant to provide research and
development funding to developers, because they believed the
developers might later request continued funding to help make
their technologies operational.
SARA Section 311(b){3) indicates that the Administrator shall, to
the maximum extent possible, enter into appropriate cost sharing
arrangements under this subsection. From this, we believe that
the legislation points out that funding was available and.was
encouraged by Congress.
However, SITE officials finally selected a model for funding
based on the requirements of private venture capital firms, which
emphasize a very conservative approach to lending money. This
model required an extraordinary amount of information to be
provided to EPA. Specifically, SITE officials developed a
financial need evaluation methodology that required applicants
requesting financial assistance to supply the following
information:
1. responses from private financial sources the developer
sought for financing;
2. the application sent to the private financing source,
including audited income statements and balance sheets
for the past three to five years and pro forma
financial projections for three to five years;
3. corporate tax returns for the last three years;
4. list of beneficial owners who have stock that exceed
ten percent of the amount outstanding;
5. financial statements of the identified beneficial
owners;
6. resumes of key company officers covering the last ten
years; and
7. estimates of the anticipated project costs.
47
-------
SITE officials told us this type of detailed information was
required for companies that venture capitalists finance.
However, venture capitalists' interests are different from EPA's.
Their point of view is directed toward profitability, so the
extremely detailed information is necessary for their business
decision, i.e., whether to finance the venture. EPA, on the
other hand, is mainly concerned with testing alternative tech-
nologies to use in cleaning up hazardous waste sites.
After deciding upon this methodology, SITE officials were
concerned that they would have to actually review company
financial records and decide if the company would be eligible to
receive funding. These officials told us that because they did
not believe they were qualified to do this, they did not want to
assume responsibility for deciding which developers would receive
funds and which would not. In addition, the officials acknow-
ledged that the SITE Program does not have any formal procedures
to review a developer's funding application since no developer
has ever gone through the process. However, they told us the
SITS Program would probably set up a panel consisting of the RREL
economist, a grants or contract individual, and other technical
individuals to review the developer's proposal.
In addition to requiring extensive financial information, EPA
required the developers to agree to share future profits with
the Federal Government. Specifically, the cover letter which
accompanied the 003 RFP (and subsequent RFPs) stated that EPA is
entitled to an one to two percent royalty on any future implemen-
tation of a technology which EPA helped finance. SITE officials
did indicate that, depending on each individual situation, they
may be prepared to reduce or remove the requirement for the
royalties or financial return to EPA. This would be negotiated
with each individual applicant.
On January 6, 1988, EPA sent solicitation 003 RFPs to interested
developers. For the first time in the program, this solicitation
informed technology developers that financial assistance was
available to qualified applications. Interested developers were
required to identify their intent to apply for financial'assis-
tance in their proposals. However, the RFP stated that consider-
ation would not be given to any request until after the developer
had been accepted into the program.
SITE officials told us that EPA would be willing to pay specific
developer costs associated with the demonstration, such as
setting up and tearing down the equipment, technicians used by
the developer to get the equipment set up and running, and the
transportation of the equipment to and from the demonstration
site location. The officials stated this would only occur if a
developer has satisfied the financial requirements.
48
-------
The overall conservative funding approach adopted by EPA with
solicitation 003 may have hindered some developers from
participating in the program. For example, one developer
(International Hydronics) from the third solicitation was denied
participation in the SITE Program for reasons which included a
request for funding. Another company from solicitation 003 that
was in our sample of nine technologies, Ultrox International,
initially indicated they were considering applying for funding.
SITE Program officials asked Ultrox to provide more information.
Ultrox later informed the SITE Program officials that they had
decided not to apply for funding assistance. In our telephone
survey, we talked to an Ultrox official who stated that it was
very difficult, especially for a small company, to get funding
assistance, and that the demonstration had been very expensive
for them. This official called lack of readily available funding
assistance one of the weaknesses of the program.
We believe that, while some companies may be concerned about
giving up royalties to the Federal Government, smaller companies
with minimal profit margins may have been completely discouraged
from applying for Federal assistance. And some interested
developers unable to finance their portion of a technology
demonstration may have been deterred from applying for partici-
pation in the Demonstration Program because of the amount and
type of financial information required.
SITE Officials Were Concerned Funding For Developers Would
Adversely Impact Their Program
SITE officials were also concerned that the funds for the
developers would have to come out of the annual Demonstration
Program budget, which they believed was already constrained.
They believed that the overall success of the Demonstration
Program would be adversely affected if the Demonstration Program
provided financial assistance to technology developers out of the
existing appropriated amounts.
SARA authorizes the Agency to spend up to $10 million for Federal
assistance under the program in any fiscal year. Congress
initially appropriated $9.1 million to support the SITE Program
in fiscal 1987. The Demonstration Program received approximately
88 percent ($8 million) of the total SITE Program's budget, and
has received approximately 62 percent of the program's budgeted
funds in fiscals 1988 through 1990, as shown in Table 6.
49
-------
TABLE 6
SITE PROGRAM BUDGET
(dollars in millions)
Total SITE Demonstration Percentage of
Fiscal Year Program Budget Program Budget Total Budget
1987 $ 9.1 $ 8.0* 88%
1988 $19.1 $13.3* 70%
1989 $19.4 $13.3-)- 69%
1990 $25.8 $13.3# 52%
* actual; + estimated; and # projected amounts during each fiscal
year.
At least two other factors served to constrain the funding that
could have been provided to developers. First, the program's
contractors incurred costs representing a very significant
portion of the available funds. These contractors absorbed
approximately 78 percent of the Demonstration Program's budgeted
funds in fiscal 1987, 74 percent in fiscal 1988, and 86 percent
in fiscal 1989. And as discussed in Finding 3, these costs were
greater than originally planned.
A second issue that impacted the SITE Program funding was
the exit program (see Finding 1). As of October 1, 1989, 10
developers had exited the program without doing a demonstra-
tion, costing the program about $1.7 million. In addition,
on December 5, 1989, a SITE official at RREL advised us that
another developer is being asked to exit the program. This
would bring the total to 11 developers at a cost of about $2.6
million. This represents nearly eight percent of the Demonstra-
tion Program's fiscals 1987 through 1989 budgets, for which
little or no benefits accrued to the Agency.
During our review, we found another program within the overall
SITE Program that was actually providing funds to developers.
The Emerging Technologies Program is another component of the
SITE Program which performs laboratory pilot- and bench-scale
evaluations of technologies that are not yet ready for field
demonstration. In September 1988, seven technologies were
selected to participate in the Emerging Technologies Program.
The first year funding provided to developers for these seven
projects totaled approximately $1 million. Another $1 million
was projected to be available for the first year of technologies
selected under the second solicitation issued in July 1988.
50
-------
Even though SARA made no provision for.financial assistance to
technology developers in the Emerging Technologies Program,
developers received funding during the program's first year.
SITE officials responsible for administering funds for the
Emerging Technologies Program apparently had no problems
disbursing funds to the private sector. In contrast, the ORD
flagship program — the Demonstration Program -- struggled for
approximately 16 months to deal with SARA legislation which made
Federal assistance available to qualified technology developers.
Funding Concerns Impact Developer Participation In The
Program
At the forum in Atlanta, Georgia, developers told us that they
believe small companies are not considering participation in the
SITE Program because of the lack of funding and the funding
restrictions. Further, as part of the audit, we conducted a
telephone survey of five technology developers who had completed
field demonstrations. We asked these developers what impediments
they encountered with the Demonstration Program. Four of the
five said funding was an impediment to the program. We also
asked, if given the opportunity, whether they would participate
in the Demonstration Program again. Four of the five said they
would not participate again. Of these four developers, two
attributed their reasons to the high cost.
During our file review, we found that on May 22, 1989, a
Congressman had written to SITE Program officials expressing
concerns about, among other things, the lack of Federal
assistance for a developer in his district. The developer's
proposal under solicitation 004 had been rejected by the SITE
Program, and the Congressman requested this rejection be
reconsidered. The Congressman statedt
"We believe that your agency shares the
commitment by the federal government that
federal grants be made available to all
possible qualified applicants, including
those that are relatively small in terms of
staff and capital, but have the professional
skills and sophistication to do the job."
As of October 1, 1989, the SITE Program's response was still in
.process.
One indicator of developer interest in the program is the volume
of responses to SITE RFPs. During our review of the proposals
received in response to these RFPs, we found that for solicita-
tions 002 and 003, there were increases from the prior year's
volume. However, for solicitation 004, there was approximately a
28 percent decrease in responses received, as shown in Table 7.
51
-------
TABLE 7
RESPONSES TO DEMONSTRATION PROGRAM RFPs
Number Increase (Decrease)
of Proposals From Prior Year
Solicitation Received Number Percentage
001 21 N/A N/A
002 27 6 29%
003 32 5 19%
004 23 (9) (28%)
Based on our work, the lack of available funding and the conser-
vative approach taken by program officials in offering Federal
assistance in solicitation 003 may be part of the reason for this
decrease.
ORD COMMENTS AND PIG EVALUATION
In his March 12, 1990, memorandum, the Assistant Administrator
for Research and Development agreed with some of our
recommendations (see Appendix 1), and indicated that certain
corrective actions are already underway. He did not, however,
agree with our recommendation to reduce the amount of financial
information that is required from prospective developers.
Nevertheless, we still believe that requiring interested
developers to provide EPA with such things as three years of
corporate tax returns and financial statements of beneficial
owners is excessive, and is not specifically required by SARA.
The Assistant Administrator also provided comments regarding our
issues. Specifically, while acknowledging that funding
restrictions and procedures may have discouraged some otherwise
qualified developers from applying for the program, he stated
that these requirements are set by SARA. Further, he stated that
requiring less financial information would have failed to meet
the requirements of SARA. We believe, however, that the
requirements placed on interested developers by the SITE Program
were not set by SARA, but instead reflected the program's
interpretation of the legislative intent. The Assistant
Administrator also stated that EPA has had no trouble attracting
fully qualified developers into the program, and that funding
developers would mean that fewer technologies overall could be
demonstrated. As we discussed in our report, however, we are
concerned that developer interest has declined (based on our
survey of developers and the number of responses to solicitation
004) and that some viable, needed technologies may not be
52
-------
demonstrated due to the high costs incurred by developers.
Appendix 2 is our detailed analysis of ORD's comments.
RECOMMENDATIONS
We recommend that the Assistant Administrator for Research and
Development:
1. Seek further clarification from Congress on the intent
of SARA Section 311{b)(5)(J) concerning funding for
technology developers.
2. Reduce the amount of financial information
that is required from prospective developers.
3. Discuss and formalize with GAD officials an official
Agency position on the amount of time anticipated for
awarding funding for developers, and convey this
information to developers applying under the new
solicitations.
4. Request additional funds be allocated to the SITE
Program to provide for developer funding. Consider
reprogramming budgeted funds, if necessary.
53
-------
EXHIBIT A
Page 1 of 9
Summary of the
Terra Vac, Inc.
SITE Demonstration Project
As of October 1, 1989
Date of Acceptance: January 1987 (Solicitation 001)
Site Selection: The Groveland Wells Superfund site
Grove land, Massachusetts.
Date of Demonstration: February 11 through April 18, 1988
Project ff'""""^^! Terra Vac, Inc., of Dorado, Puerto Rico,
developed the in-situ vacuum extraction process, which removes
volatile organics constituents from the unsaturated zone of
soils. The extracted gases and water proceed to a vapor liquid
separator and an emission control system.
For the demonstration, extraction wells were drilled at four
locations at the edge of the contaminated area. At each
location, two wells were installed, a shallow well and a deep
well, to separately extract volatile organic compounds from above
and below clay soils. The system operated 24 hours per day for
two months. Samples of the soil gas, process gas, stack gas
emissions, and liquid from the vapor/liquid separator were
collected. Initial results indicate that a total of
approximatley 1,300 pounds of volatile organic compounds were
extracted. Some shallow soil gas concentrations were reduced by
more than 95%, and the extraction process seemed to work
successfully even in wet clay soils.
As a result of the demonstration test Terra Vac has been hired by
the site owner to undertake site remediation. The process has
been employed at over 60 sites, including four documented case
studies at three Superfund sites.
The TER is now available. The AAR will be formally distributed
by the end of October. Project costs funded by EPA for the Terra
Vac demonstration through fiscal 1989 have totaled $1,745,000.
-------
EXHIBIT A
Page 2 of 9
Summary of the
Ogden Environmental Services
SITE Demonstration Project
As of October I, 1989
Date of Acceptance: March 1987 (Solicitation 001)
Site Selection: McColl Superfund site in Fullerton,
California
Date of Demonstration: No demonstration has been conducted
Project sftn""MEY» A Circulating Bed Combust or devloped by Ogden
Environmental Services, a San Diego, California, based company,
destroys a variety of waste materials at temperatures below 1560
degrees fahrenheit. The unit employs simultaneous limestone
injection, which captures acid gases and eliminates the need for
a scrubber. Wastes from the Stringfellow and McColl Superfund
sites were initially to be used for the treatability testing at
the Ogden facility in San Diego.
Unexpected delays during the early stages of the project have
hindered the progress of the demonstration. From December 1987
to June 1988, a delay resulted when Ogden was denied a permit
from the city of San Diego to burn wastes from the two sites at
their research facility in San Diego. After a permit was finally
granted to do test burns, another delay occurred from December
1988 to March 1989, when an area agency imposed that a risk
assessment be done on the results from the treatability tests.
The draft report on the results of the treatability study was to
be distributed for comment in the Fall of 1989. The test results
indicate that the preliminary test burns conducted at the Ogden
research facility were mostly successful and support a full-scale
site demonstration at the McColl Superfund site. The
demonstration is tentatively scheduled for January through March
of 1990. Project costs funded by EPA through fiscal 1989 have
totaled $770,000.
55
-------
EXHIBIT A
Page 3 of 9
Summary of the
International Waste Technology
SITE Demonstration Project
As of October 1, 1989
Date of Acceptance: Formally recognized as a SITE participant
in September 1986 (Solicitation 001)
Site Selection: General Electric site in Hialeah, Florida
Date of Demonstration: April 11-16, 1988
Su
International Waste Technology (IWT) of
Wichita, Kansas, developed and markets the HWT series of advanced
chemical fixation products for the treatment of inorganic and
organic content toxic wastes. IWT uses the Geo-Con, Inc., deep
soil mixing system to drill and blend waste material with its
patented bonding agent. This technology can be applied to soils,
sediments, and sludge pond bottoms contaminated with organic
compounds and metals.
in cooperation with General Electric, Inc., IWT demonstrated its
in-situ stabilization/solidification process at a General
Electric service shop site in Hialeah, Florida. The major
objectives of the demonstration were to: evaluate the ability of
the process to immobilize PCBs in the soil; determine the level
of performance and reliability of the mechanical equipment being
used; assess the effectiveness of the process for in-situ
stabilization; and observe the integrity of the solidified soil
over a period of 5 years. After demonstration, preliminary
results show physical properties of the treated soils were
generally satisfactory; unconfined compressive strengths were
high; permeabilities were low; and results of weathering tests
were satisfactory. The results also indicated a potential for
long-term durability. The Geo-Con equipment operated
satisfactorily during demonstration tests.
The TER was published in June 1989. The AAR is currently
undergoing final revision and should be available in early 1990.
The draft long-term monitoring report, with results from sampling
10 months after the demonstration, is undergoing EPA peer review.
This report is scheduled to be completed by the end of December
1989 and be available early 1990. Project costs funded by EPA
through fiscal 1989 have totaled $670,971.
56
-------
EXHIBIT A
Page 4 of 9
Summary of the
Chemfix Technologies, Inc.
Site Demonstration Project
As of October 1, 1989
Date of Acceptance: September 1987 (Solicitation 002)
Site Selection: Portland Equipment Salvage Company Superfund
site, Clackamas, Oregon
Date of Demonstration: March 13-17, 1989
Project ffiimm»-i-yi Chemfix ^Technologies, Inc., of Metairie,
Lousiana, has developed a proprietary*process that stabilizes
high-molecular-weight organics and inorganics constituents in
waste slurries. The process uses soluble silicates, silicate
setting agents, and additives to crosslink with waste components
to produce a stable, solid matrix.
Analyses for all short-term chemical and physical test have been
completed. A decision was made to try a new approach on the
reports for this project. The TER and the AAR may be combined.
If there is a single report, it may be available in the Spring of
1990. Project costs funded by EPA through fiscal 1989 have
totaled $1,143,489.
57
-------
EXHIBIT A
Page 5 of 9
Summary of the
Retech, Inc.
SITE Demonstration Project '
As of October I, 1989
Date of Acceptance: September 1987 (Solicitation 002)
Site Selection: Butte Area Superfund site, Butte, Montana •
Date of Demonstration: No demonstration has been conducted
_ _.._ Retech, Inc., of Ukiah, California, has
developed a thermal treatment centrifugal reactor that uses
plasma heat to decompose organics in admixed solid and liquid ?
feed. The solid components are melted and cast or granulated for $
disposal, while the volatile compounds are vaporized and ^
decomposed in an afterburner.
y
EPA is planning a demonstration in coordination with the
Department of Energy (DOE). DOE, EPA and Retech recently met at
DOE's research facility in Butte, Montana, to discuss the
progress of the project. DOE will begin a series of tests
designed to optimize the process. The unit will be tested on
waste from the Butte Area Superfund site. The demonstration is
tentatively planned for Spring 1990. Project costs funded by EPA
for the Retech demonstration through fiscal 1989 have totaled
$759,357.
58
-------
EXHIBIT A
Page 6 of 9
Summary of the
Silicate Technology Corporation
SITE Demonstration Project
As of October 1, 1989
Date of Acceptance: May 1988 (Solicitation 003)
Site Selection: No official demonstration site has been selected
Date of Demonstration: No demonstration has been conducted
Prolect ff"1*'M<>T* Silicate Technology Corporation (STC) of
Scottsdale, Arizona, has developed a method to stabilize metals
and high-molecular-weight organics in.soils and sludges. This
new technology uses a proprietary reagent, FMS Silicate, to
selectively absorb organic contaminants prior to mixing the waste
with cementing material to form a solid, high strength mass.
In August 1988, EPA tentatively selected the Tacoma Tar Pits
Historical Coal Gasification Superfund site in Tacoma,
Washington, to conduct the demonstration. The demonstration was
initially scheduled to be conducted in mid-Summer of 1989.
However, the two potentially responsible parties for the Tacoma
Tar Pits site cancelled their tentative approval to use the site
and EPA was forced to locate a new site for the demonstration.
Negotiations are currently underway with Kaiser Steel Resources,
near Los Angeles, California, as an alternative site for the
demonstration.
A draft plan for collecting samples and conducting treatability
studies, and a draft demonstration plan, have been prepared for
the Kaiser site. The major concern is the delay which may result
if Region 9 determines that a Research, Development and
Demonstration permit is required in order to conduct the
demonstration. No definite date has been set for the
demonstration. Project costs funded by EPA through fiscal 1989
have totaled $276,192.
59
-------
EXHIBIT A
Page 7 of 9
Summary of the
Ultrox International
SITE Demonstration Project
As of October 1, 1989
Date of Acceptance: May 1988 (Solicitation 003)
Site Selection: Lorentz Barrel and Drum Site
San Jose, California
Date of Demonstration: February 27 through March 10, 1989
r
Project -?"«"»*"""yi The Ultraviolet Radiation and Oxidation System,
developed by Ultrox International of Santa Ana, California, uses
a combination of ultraviolet light radiation, ozone, and hydrogen
peroxide to oxidize organic compounds in the water. To enhance
the oxidation of organic contaminants, various operating
parameters can be adjusted including hydraulic retention time,
influent pH level, oxidant dose, and UV lamp intensity.
During the demonstration, groundwater treated by the system met
the discharge standards for disposal at the 90 percent confidence
level, and no semivolatiles, PCBs, or pesticides were found in
the groundwater at the test site. The system was successful in
achieving removal efficiencies as high as 90 percent for total
vocs and greater than 99 percent for the TCE.
Information from the demonstration is being used by Region 9 in,
their final effort to clean up the groundwater at the Lorentz
Barrel and Drum site. Project costs funded by EPA through fiscal
1989 have totaled $755,425.
Through the coordinating efforts of EPA, the developer, the
contractor, and Region 9, the Ultrox demonstration project was
able to provide a timely 9-month demonstration, to achieve the
objectives of the Demonstration Program as well as those of the
individual Superfund site remediation effort.
60
-------
EXHIBIT A
Page 8 of 9
Summary of the
Biotrol, Inc.
Site Demonstration Project
As of October 1, 1989
Date of Acceptance: May 1988 (Solicitation 003)
Site Selection: MacGillis and Gibbs Superfund
site, New Brighton, Minnesota
Date of Demonstration: The demonstration is ongoing (July 24 -
September 1, 1989) and is scheduled for
completion the last two weeks in October
Project fhimiMTyi Biotrol, Inc., has developed a biological
treatment for organics in aqueous waste.-" This new technology
biodegrades toxic compounds in groundwater, process waters, and
lagoon waters. Contaminants may include PCBs, creosotes, and
inorganic compounds such as nitrates.
The field demonstration of the Biotrol Biological Treatment
System has been in progress at the MacGillis and Gibbs Wood
Preserving site. The system was operated for 6 weeks to complete
3 out of 4 tests. The 4th two-week test of the system will be
conducted in late October. According to preliminary data and
observations, the system operates as expected. Project costs
funded by EPA through fiscal 1989 have totaled $1,151,204.
61
-------
EXHIBIT A
Page 9 of 9
Summary of the
Toxic Treatment, Inc.
SITE Demonstration Project
As of October 1, 1989
Date of Acceptance: May 1988 (Solicitation 003)
Site Selection: Annex Terminal site, San Pedro,
California
Date of Demonstration: September 5-7, 1989
Project giMiiiMrYi Toxic Treatments, Inc., of San Mateo,
California, has developed the Detoxifier, an in-situ method of
removing volatile organics from soil, using steam or air. The
transportable unit uses drills that have been modified to allow
for the expulsion of steam or air through the cutting blades.
The modified drills allow soil to become much more porous so that
the injected, superheated steam and air can uniformly heat and
strip the organic contaminants.
Toxic Treatments demonstrated a full-scale prototype of the
Detoxifier in September 1989. The test was conducted at the
Annex Terminal site in San Pedro, California. In addition to
soil samples taken before and after treatment, EPA collected
samples of treated air and water, and the condensed organics. A
dye was also injected with the steam and air during treatment to
track any downward migration. If early results of testing are
satisfactory, the remainder of the site will be remediated using
this method. Project costs funded by EPA through fiscal 1989
have .totaled $1,114,637.
62
-------
z
o
EXHIBIT B
Page 1 of 2
C/)
L_ >
id O °
0_< ^
n
o
o.
Id
ohu
CC h- L.
Ld (/)
00
O
H
C/J
O
2
LJ
i
o
o
o
(N
o
§
<
£
£
Id
Id
O
O
O
o
Ld
t
X
5
a3d013A3Q JO 3F1VN
63
o
*>
43
o
N
O
2 ui
...
m
2
o D
Z
-------
EXHIBIT B
Paao 2 of 2
CO
^F
Q_
g&
LJ
TO
•Mitt*
I ' " _•' f\jT
z'o_
z0-'
9o
o5
o:
tt: H _
u (/) o
mz
O LiJ
UJ
o
UJ
o
CO
o
H
tr
2
s
0.
o:
i
»•*»*•«
»*****»
»•*****
+
LJ
I
X
X
o
o
UJ
UJ
CO
o
o:
s
m
M3d013A3a JO 3HWN
o
M
« H
^ Z
o "~
° O'
* o:
o kl
m
« i
64
-------
EXHIBIT C
Page 1 of 2
Lo_
O
m o z
D°
I O Q_
I- LU
o
TECH/SHIRC
y
<
X
a. .
OSETOWNSHI
QL
\
0
O
g
X
to
z -
HA2CO
Z
COMBUSTIOI
Z
<
o
a:
LJ
2
<
>-
TECHNOLOG
kl
fc
(/)
1
. _l
H
o
JO 3«VN
65
-------
EXHIBIT C
Page 2 of 2
a3d013A3Q JO 3WVN
66
-------
EXHIBIT D
RESULTS OF SURVEY QUESTIONNAIRE REGARDING THE
STRENGTHS AND WEAKNESSES OF THE SITE DEMONSTRATION PROGRAM
OSCs
SUPERFUNO TECHNOLOGY
RPMs CONTRACTORS DEVELOPERS
Number Contacted in Survey
25
28
15
16
21
14
Strengths
Muter who responded to question
Exposure
Credible Data
Technology Transfer
Full-scale Testing
Other strengths identified by less than 10X of the respondents in each group included
evaluation of technology and meeting CERCLA objectives.
88X
13
38
6
81X
14
43
10
71%
29
50
7
SOX
20
0
0
Weaknesses the Respondent Had Observed
Number who responded to question
Demonstration Cost
Site Matching
Program Policies/Requirements
Opposition to Innovative Technologies
Lengthy Demonstrations
Dissemination of Demonstration Results
Lack of Exposure
Not Channeled towards Emergency
Response
14
OX
20
10X
10
13
15X
31
60%
20
29
0
36
7
0
30
15
25
15
20
23
0
31
31
0
20
20
40
' 40
0
21
Weaknesses Others Had Mentioned
to Respondent
Nunter who responded to question 12 11
Funding
Red Tape
Lengthy Demonstrations ,
Permitting Issues
Treatment Standards
Other weaknesses identified by less than 10X of the respondents in each group included connunity opposition,
liability issues, technologies not readily usable, site matching, and relying on contractors to prepare reports.
ax
42
25
8
0
OX
45
18
36
0
OX
0
0
0
0
SOX
60
0
20
20
67
-------
APPENDIX 1
Page 1 of 50
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON. D.C. 20400
VA? I I
or* ice a*
SUBJECT:
FROM:
TO:
ORD CoBMnts on Draft Audit Report
£ISKF9-ll-0031-Oraft "The SITE
Demonstration Program: Much Proaisa
But Delayed Results"
Erich W. aretthauar
Assistant Adainistr
tor Research and !
(ED-672)
Kenneth A. Konz
Acting Assistant Inspector General Cor Audit (A-109)
In accordance with EPA Directive 2790, I am providing you with a written
response to the subjact audit raport which was datad January 11. 1990. This
response has baan praparad by tha Office of Research and Development with
Input provided by the Offica of Solid Haste and Emergency Response. This
naaoranduB includaa a suMary of our coinanta, and wa hava attached a datalltd
rasponsa to aaeh saction of the report. I understand chat our entire response
will be included In your final report.
See APPENDIX 2
Note 1 -
Note 2
Many of the recoajMtidationi ia the audit are reasonable and helpful.
They are conaistant with our own evaluation of the SITI program that vaa
carried out a* a pare of the Administrator's Kanifsawne Xeviav of die
Superfund tngnm. We already have liplaasntad aaay of the recoajMtidationa or
we are planning to do so. We are concerned, however, that COM of the
findinga leave aa unduly negative and •ialaadlng lapreaalon of the prograa.
We' alia feel that overall balance Is lacking In carma of acknowledging the
substantial accoBplishnenta of the prograsi and ia reeocnixiag the exigencies
associated with applying innovative technologies at conplex, real world,
•itea.
The context in which the SITS Program operates is not adequately
recognised ia the report. The Superfund Innovative Technology Evaluation
(SITE) Prograsi la the anlv coavrebenelve national effort to deswnatrata and
evaluate Innovative hazardous waste treataent technologies to proejote their
use in achieving acre permanent remedies in cleaning up hazardous waste sites.
The SITE Program is aa •innovative* UMD program dealing with the leading edge
of technology. It is unlike any other program that the Agency has undertaken
a true entrepreneurial venture for EPA. The application of innovative.
68
-------
APPENDIX 1
Page 2 of 50
technically sophisticated technologies to eh* complex problem* «t Superfund
site* i* an eabiciou* undertaking - both c«chntc«lly and adminiacracivaly and
Involve* substantial risk caking.
V* understand Chat a key objective of che audit vaa Co identify prograa
deficiencies «nd recommend action* to improve the prograa. But we believe
that the report doe* not recognize the rl*k caking nature of the prograa end
therefore become* critical of re*ult» chac ahould be expected. In firsc time
APPENDIX 2 field application* of innovative technologies, one should expecc budgie
Note 3 modification* and schedule change*. Without Chi* recognition I aa concerned
that che negative tone of the report will Inhlblc responsible entrepreneurial
innovation and rl*k caking In carrying out the prograa.
there are many succe**** of che SITE Prograa chat we believe ihould be
recognized:
o Over the paat three years, 14 Innovative technologies have been
deaonatraced and chair performance and eoata evaluated, the** 14
. demonstrations have included thermal, biological, solidification/
stabilization, chealcal and physical treatment processes.
o Given Che experience we bar* gained, over che next 2 year* we expect
to complete approximately 20 additional demonstration* in a variety of
process categories.
o Thl* range of cechnologle* 1* atgniflcancly enhancing the choices
available Co Chose selecting technologies to clean up site*. At least
Note 4 <*«* of :ch* SITE deaonstrated technologies have been selected or are
being used in permanent remedies at Superfund site*. In FY 88. 721 of
ROD* selected alternative technologic*, compared with 451 in FT 86.
Note 5
We have provided sound technical Information on SITE technologies
through a rang* of aacbsnism* Including an online computerized
database, demonacraclon bulloclns, technical report*, Journal
article*, conference*, sic* demonstration visitor days, meeting* with
Regional office etaff, and other direct contact.
Experience gained through the SITE Prograa enabled EPA to establish a
nationwide Superfund Technology Assistance Re*pon*e Team (START) co
work with OSC* and IMS in identifying technology or combination* of
technology eo remediace site*.
While we agree with many of the audit finding* and recommendation*, we
yr.« with *everal of them. .Specifically, we take strong exception.to che
•negation that limited eapnaai* waa placed on controlling contractor coat*.
Cited coat Increase* can be Justified by a number of factor*, including
problem* in che field that are inherent to any field d*aon*tracton of
N . g innovative technologies, tfe also dl*agrae chac the contracting mechanism u*ed
69
-------
APPENDIX 1
Page 3 of 50
waa laproper or that tha award feaa p*td ware too high. Contracting
aechaniaaa and award feee were aelectad In cloee coordination with and through
the advlca of UA'a Contract* Maaageaent Division. Finally, v« believe that
our approach to providing financial assistance to developers was completely
consistent with th« criteria set forth in tha legislation.
Soae Bid- course corrections have b««n naadad and hav« b*«n aade. This Is
to be expected in a new prograa Involving coop Lax tachnologiaa and sites and
cooperative arrangeaents aaong aany partiaa. But «• hava identified that*
Issues and bagun corracciva actiona. In Juaa 1919 tha Administrator of SPA
subaleted to Congraaa A ffanafeaane Xavlav at ehe Supmrfund Frograa, fulfilling
hi" c«"»i««t« to W«netfy actiona tha Agancy would taka to laprova
. p.rforMne- ef that prograa. Tha raviaw addxaaaad tha application of
Note 7 innovatlva tachnologiaa at Suparfuad aitaa, and includad racoanandationa to
iaprova and anhanca tha SITE Program. Tha raviaw alao raflactad tha kay tola
that SITE haa had and would continua to hava in tha davalopaant of innovativa
tachnologiaa for Suparfund aitaa. Thaaa r»coa»«n<1 • tiona aza now baing
ioplaaantad. Wa ballava that tha analyala rapraaantad by th« Adalniatrator ' a
raviaw and tha ioprovaawnta that ha haa eoawlttad to iaplaatanc rapraaant
important parapactlv* that waa not raflactad in tha OIC raport.
Whila thaaa changaa will ia^rova taa SITE Prograa, wa baliava that oa tha
whola it haa baan on tha sark. Tha Agancy haa a»t virtually avary raquitaMfit
placad on it in tha laglalation that eraatad SITS. Thla includaa tha tiaaly
publication of program eritarla, tha aalaction of at laaat tan tachnologiaa In
Q each of four conaacutiva yaari, and tha conduct of all work within tha budgat
o authori«ad in tha law. In addition, tha technical producta of tha
daax>n*tratlona hava baan high quality, aclantlflcally aound documanta.
Aa part of tha Suparfund prograa, SITE aay auffar froa a problaa of
unraaliatic axpactatlona . aa haa Suparfund. In tha Adalnlatrator'a A
ttanajraaanc Jtavlaw of eh* Sufwrruod Profraa, ha at* tad: *For aa tha aoat
Important finding of tha atudy la tha naad to aat fair and raaliatic
axpactatlona for a prograa of tha ralatlva aovalty and acopa of Suparfund.'
Tha iaaa can ba aaitf for SITt.
Tha 1C' a draft audit raport includaa four aajor critielaaa of tha SITE
Program: (1) Batching tachnologiaa to daaonatratlon aitaa haa baan a alow
procasa, (2) langthy raportlng parloda and dalaya In raporting daaonatratlon
raaulta hava provldad littla datallad inforaation to aaaiat ragional dacialon-
•akara, (3) inauffleiaac attantion haa baan placad on contractor parforaanca
and SITE coata, and (4) tha aatbod aalactad to provida fadaral aaaiatanca
diacouragad technology davalapara. A ayoepaia of tha Agancy 'a raaponaa to
each of thaaa critielaaa la praaaatad balow.
70
-------
APPENDIX 1
Page 4 of 50
A. Finding: Tha 1C report states that the SITE Demonstration Program has
been dalayed because of problem* in locating sita* for the
technology demonstration*.
Sit* selection ha* takan longar then w* had originally planned and v*
agree with *»*t of the recomended action* to shorten thl* process. However.
eh* audit do«* not «d«qutc«ly reflect ewo liportanc eonsldaratiotu. Fir*t, v*
h«v« «lr««dy *ucc««d«d in lignlflcanely reducing sit* ••lection tiae, and Che
average !• now about nln* •oath*. Ue are continuing to work on this inue,
including Icpleaonclng approach*! recowendad in thl» report, and expect
further redxictiona in the future coward our goal of * 90 day lice selection
tio*. Secondly, wa believe the report do«a not recognize •«•• of the
flDDPUHTV 9 inportant r*a»ora that »lte telection can be aubjact to delaye. For exaaple,
MrrbNUlA d. these daaonatrationa are taking place at Superfund site* and therefore are
Note 9 -subject to the general scheduling issua* at thoa* sites. These include legal
actions, concurrence* required by responalble parties and stata*. and public
participation activities. Unfortunately, w* hav* liailted control over these
factors.
1. Review and analyse the reason* why davalopara exit the Dasmwtration
Prograa so that this information could be used in selecting future
technologies.
Respona*:
Note 10 • tf* hav* don* this and suaurized tha analysis in this response.
2. Initlat* SITE tachnology daaonatrations that would includa us* of acre
than on* t*chnology for tha daaonstration.
• On* coabinatlon of tachnologl** vaa daawnatratad by BioTrol (July
through Oetobar 1989). V* hav* requaated such combinations in tha
Note 11 laat solicitation and will continue to pursu* this approach.
3. Poraaliz* tha nuibar of lit* location* at which a davalopar can refuse
to conduct a daaonatratlon.
Note 12 ' W* Pr***r e° puz*u* t^i* informally, sine* a rigid nuabar would b*
arbitrary and could diatourag* d*v*lop*r». It is difficult to
71
-------
APPENDIX 1
Page 5 of 50
s
formalize • site refusal policy, and turd to offer * developer •
•1C* with no contingencies.
t. Add SITE demonstrations to the performance Indicator* or elements of
ch« ORD or SCAP management tracking systems.
See APPENDIX 2
Note 13
The SCAP My not b« appropriate since It, !• an OSVER/reglonal
system for managing workload and resources, but other approaches
*r« undor consideration by OSVER.
5. Explora and d*v»lop och«t Inemtlv** for r«fion*l RPHa and OSC« to
allow chair altaa to b« uaad la SITE tochaoloay daaonatratlom.
• SITE aanagaaant agraaa and will puraua eh*
B. Ptru»n^! Tha 1C raport atataa that tta SITE Progra* naada to address tha
langthy raportlng periods and delays la Issuing the final
reports of the technology daaonacratlona. The report Indicates
that these delays have led to s lack of reported Information and
documented results which has handicapped the success of the SITE
Program.
Note 14
Note 5
He agr«« that publication of ftasl technical reports has taken too long
In some eases. But the audit does not recognise actions that have already
been taken to shorten report publication times. The example* la the report
reflect the early experience, rather than more recent (more favorable)
experience. Second, the report does not adequately consider other ways that
pertinent Information Is being made available la much shorter tlmeframes, auch
as the use of an online database and publication of summary bulletins. Ve
will continue to develop ways of making quality Information available as
quickly as possible. Including addressing recommendations la this report. It
Is Important, however, to maintain strong quality control of the data from
these demonstrations.
1. Establish formal accountability for SITE management for meeting
reporting milestones.
72
-------
APPENDIX 1
Page 6 of 50
See APPENDIX 2
Note 15
• A eysten !• now In place Involving quarterly progress report* and
project schedules/nllestone checklists. The' current tyicca will b*
reviewed and codifications »ay b« considered at the end of the
fiscal year.
2. Consider alternatives for performing • detailed review of th« proc**i
for' proper lac *or •*•* Issuing SIT! roporti, »uch •• •
Sy«t«aj tovlaw.
Note 16
c.
Note 17
• A Maiu(«Miit SyctwM Rovlow Is b«inf ltm*tl|*t«d with « decision
on wh«th«r to proceed by June 1, 1990.
5. Ev»lu*te publiCAtlon elcernetlves to speed up the ectu*l distribution
process .
• tfe iaplestenced soew ehaages in 1969, but will further evaluate
alternatives by Hay 1. 1990.
4. Prepare e videotape or presentation of .the Demonstration Prograa'a
goals and objectives to be presented to all Superfund personnel in the
regions sad at Headquarters.
We are taking several steps to raise awareness, including working
with the OSVBt Office of Technology Innovation, conducting annual
regional SIT! coordinators •eatings, and distributing a videotape
prepared for the 1989 Washington Superfund Conference.
The 1C report state* that l^roveaents are needed to ensure the
quality of contractor's work aad to control contractor costs.
The report indicates that the 1C found that award feea ware not alweyi
consistent via the project tiste delays and increased costs.
The report cites cosMent* from several OSCs, UHa, and developers that
were critical of contractor performance.
We are in substantial disagreement with the findings of the audit in this
area. First, we believe overgeneralixations were •ado about probleas with
73
-------
APPENDIX 1
Page 7 of 50
See APPENDIX
Note 18
Note 8
Notes 6. 51
contractor performance from limited numbers of survey comments. Although some
difficulties were experienced la the beginning of the program, the contractors
Involved were replaced. Generally, contractor* are now performing competently
In our view, but we will continue to push for excellence * applying the
principles of Total Quality Management • and will pursue the recommendations
of the audit to further Improve performance.
We particularly take strong exception to the allegation that Halted
emphasis vas placed on controlling costs. The cost Increases cited In the
audit often resulted fro* OKD's specific attempts to keep costs down by
Initially approving and funding very conservative estimates of potential costs
to cause contractors to economize and then forcing contractors to specifically
Justify additional work and coats. In other cases, Increases resulted from
additional contractor work related to unforeseen performance problems with the
technologies, necessitating additional time la the field. This should be
expected with new technologies. The bottom line Is that the Agency has
successfully Initiated demonstratioas at the mabeir of sites required by
Congress sad within the authorised budget. Moreover, no qualified technology
has been refused due to lack of available ETA funds.
Finally, we dispute the claim that the contracting mechanism used waa
Improper and that award fees paid were toe high. The EPA Procurement sad
Contracts Management Division (PdfD) determined the contracting mechanism to
be used and does not believe that other mechanisms, such as fixed price
contracts, are practical for this program. Award fees are determined by PCMD
based on recommendations from sa Independent performance review board.
Percentage awards are la fact lower for this program than are typical for
other EPA contracts of this typo.
1. Require that adequate justification be provided for Individual funding
Increases.
Note 19
2.
• Current justification documentation has met EPA Contracts
Management Division requirements, but wo will strengthen
requirements, where appropriate.
If contractor performance In key areas, such as report writing, cannot
bo Improved, retain those functions la house on future projects. If
necessary, assign additional staff to perform these functions.
SITE Project Managers have written report sections oa a case by
case basis, but cannot do so routinely. Eocene report format
74
-------
APPENDIX 1
Page 8 of 50
* • and enhanced experience of Project Managers and contractors
should ruulc in quality timely report* .
J. Reinforce to contractor* and project managers the importance of the
award fee process end why the award fees ere used.
• We believe the process LB well understood 4nd correctly
See APPENDIX 2 implemented. However. «• will revisit thU issue «nd take
Note 19 «ppropri»t« Action.
4. Coruld»r for th« Mxt contract period • contract typ« ttut would »or«
f *irly b*l«ne« riik b«cm*n EFA «nd die contractor*.
• Th« EPA Contracts {Un«(«*Bnt Divicioa tue** that ch« cxiicin*; Cost
Plu* Award F«« contract is th« «oat appropriate ••chanis*.
D. FtndtBf : The XG report state* that the Deaonatratlon Prograsj discouraged
distribution of Federal assistance to technology developers.
SITE •anageswnt agrees that funding restrictions and procedures My have
discouraged SOM otherwise qualified developers fro» seeking entry into the
program. These requireswnts however are set by legislated requirtaents in the
Act, not by SPA. Furthermore, SPA baa had no trouble attracting fully
qualified developers Into the program in sufficient numbers to fully utilize
the available funds. Thus, if developers' costs were being funded, fewer
... _. technologies would have been demonstrated. Hie level of information required
NO -6 £\J of developers to demonstrate financial need was also questioned. To have
required less documentation of financial need would have failed to meet the
requirements of toe law, which prohibited providing federal assistance unless
the applicant demonstrated that they could not obtain private financing. The
information required of the applicant is the same Information he would have
supplied in seeking private financing.
1. Seek further clarification from Congress on the intone of SABA Section
311(b)(S)(J) concerning funding for technology developers.
• SITE management believes that the letter and intent of Section
Note 21 niCbXSHJ) la very clear. However, this issue may be eppropriat*
for consideration aa a part of CBCtA reauthoricatlon in 1991.
75
-------
APPENDIX 1
Page 9 of 50
2. Baduee eh* amount of financial information that it required from
prospective developers.
• The information required is the seme as that required by the
APPFMDTy 7 private Mccor (venture capital coap«ai«a). Ttnu, developer*
N t on leeklnj eld would Already have the Information.
3. Dlieuai end formalize wtth GAD officials en official Agency petition
on Che aooune of ciae anticipated for evardinc funding for developers,
and convey thie information to developers applying under ehe new
aolieltationa.
• SITI officials have had discussions with GAD and will develop s
•cacaswnc for future solicitations.
4. Bequest additional funds be allocated to the SITS Frograa to provide
for developer funding. Consider raprograssUng budgeted funds, if
necessary.
The current budget request is at the authorisation limit of SARA
Section lll(n)(l). Thus, this idea would have to be addressed as a
part of CBCLA reauthorUation in 1991.
Attachment
76
-------
APPENDIX 1
Page 10 of 50
1. IMPROVEMENTS NEEDED IN MATCHING TECHNOLOGIES WITH
HAZARDOUS WASTE SITES FOR DEMONSTRATION PURPOSES
See APPENDIX 2
Note 22
Note 9
Note 23
Note 9
The IG-report indicates chit lite selection problems have resulted in lengthy delays in the SITE
Demonstration Program. The findings in (he report include the need for improving the site matching
process for SITE demonstrations.
Site selection has taken longer then we had originally planned and we agree with most of (he
recommended actions to shorten this process. However, the audit does not adequately reflect two
important considerations. First, we have already succeeded in significantly reducing site selection
time, and the average is now about nine months. We are continuing to work on this issue, including
implementing approaches recommended in this report, and expect further reductions in the future
coward our goal of a 90 day site selection time. Secondly, we believe the report does not recognize
some of the important reasons that site selection can be subject to delays. For example, these
demonstrations are taking place at Supeif und sites and therefore are subject to the general scheduling
issues at those sites. These include legal actions, concurrences required by responsible parties and
states, and public participation activities. Unfortunately, we have limited control over these factors.
This section presents the Agency's response to some of the problems and facts mentioned in (he
IG report with regard to site selection, including the:
• Number of active technologies in the SITE Program
* Site selection tine
* Reasons for site selection difficulties.
Number of Active SITE T«ch«alo«l««
The draft audit report states that "... as of September 30, 1989... only 12 of 36 technology
demonstrations were completed* (page 12). Since funding for the program was first received in
mid-1987, 14 of the 38 active innovative treatment technologies have been demonstrated. These 14
demonstration] include Shirco's pilot-scale electric infrared incinerator, Shirco's full-scale electric
infrared incinerator, HAZCON's solidification/stabilization process. Soltditech's
solidification/stabilization process, Caemfix's chemical fixation/stabilization technology, IWTs in-
situ solidification/stabilization process. Tern Vac's in-nni volatilization technology, American
Combustion's pyretran oxygen burner, Ultrox's UV radiation and ozone treatment. Toxic Treatment's
m-titu steam/air stripping process, CF Systems,' solvent extraction technology, BioTrol's soil washing
technology, and BioTrol's fixed-film bkxlegradation process. Ogden completed a pilot test of their
circulating-bed combustor which EPA is processing as a completed demonstration.
SITE Saltation Time
The IG report states that one of the major impediments to demonstrating technologies in the
SITE Program is the excessive time required to match technologies to appropriate demonstration sites.
EPA agrees, but the report does not point out that site selection is an iterative process dependent on
numerous factors such as the technology, site and waste characteristics, the developer, site owner.
etc. The elapsed time from technical acceptance in the SITE Program until a demonstration
l-l
77
-------
APPENDIX 1
Page 11 of 50
evaluation begins is dependent on etch of these variables many of which cannot be controlled by the
Agency. At any particular time there are a certain Dumber of technologies potentially matched to
sites, but this number can change very rapidly due to a variety of factors such as permitting or site
access problems. For any potentially selected site, the developer may decide that its technology is
inappropriate for the waste contaminants, the site characteristics may require the developer to incur
significantly greater costs in conducting the demonstration at that site, state or local authorities may
establish requirements that are not acceptable to the developer, and so forth. There is no credit given
for these interim site selection issues, and the absolute measure of time for site selection unfavorably
underestimates these site selection efforts, in general, it is important to recognize that site selection
is an iterative process that involves coordination among SITE officials, the regions, and the
technology developers in identifying, selecting, and approving the sites. Furthermore, the process
often requires review and analysis of three to four sites before an appropriate site can be selected.
While the 1C report indicates that site selection has varied from 0 to 17 months (for their sample
of nine technologies), the March 16, 1989 SITE Demonstration Progress Report indicates that the
See APPENDIX 2 »v««g« tine for site selection for ait SITE technologies has decreased from 2S2 days (8.4 months)
Note 24 ^or S'TE-001 technologies to 204 days (6.8 months) for SITE-003 technologies. This trend represents
positive movement toward the Agency's goal of an average of 90 days for site selection.
Of the nine developers referenced in the 1C report, some delays in site selection occurred that
were beyond the control of EPA or the SITE contractors. Some of the delays encountered for these
nine technologies were due to delays in implementing a project rather than identifying or selecting
a site. For example, site selection for the FWT demonstration was delayed because the developer had
to perform successful trettabiliry studies before the private site owner would give approval for the
Note 25 , demonstration. In addition, regulatory/permitting issues with the region, state, and county had to
be negotiated. A year elapsed between the time the technology was accepted into the program and
when the private site owner granted permission to conduct the demonstration. EPA had two options
in this case: (l) wait until the sin owner granted permission, or (2) search for an alternative site for
the demonstration. If EPA had elected to pursue the second option, the resources invested up to the
decision point in determining sin appropriateness and developing the demonstration plan would have
been lost. Another factor considered in not electing to change the demonstration site was the fact
that the Hialeah site was a private site and the owner agreed to support some of the IWT
demonstration expenses which was important to the developer.
A demonstration sin for Toxic Treatment! was identified and selected when they entered the
program, but the report indicate* that it took four months for site selection. The time was devoted
to verifying sin eppropriattoe** and resolving specific project issues. The Agency considers the
matching to be completed following the initial meeting with regional and statt representatives,
affected responsible parties, and the developer where agreement is reached to proceed with the
project at the site. Table 1 a a corrected version of the table oa page 13 on the IG report. The
IK. corrected able show* an avenge time for sin selection for these nine technologies of about six
£Xi months. It should also be noted thai several of the nine technologies analyzed in the 1C report had
to be rematched after problems developed with sins which were initially selected. These problems
accounted for several months of delay and were not cited in the 1C report. The problems included
sate permitting issues which negated the initial site match for Retech, Inc., and a change in the
cooperation of the responsible party to demonstrate the Silicate Technology.
1-2
78
-------
APPENDIX 1
Page 12 of 50
TABLE 1. TIME TAKEN IN SITE SELECTION
See APPENDIX 2
Note 26
MONTHS
REQUIRED FOR DEMONSTRATION
TECHNOLOGY DATE ACCEPTED STE SELECTION SITE SELECTION STARTED AS Of
DEVELOPER SOLICITATION INTO PHOQHAli DATE' AS Of MO/M
1. !WT
2. T*rra V»c
3.Ogd«n
"""*
5. Chtmfti
«.Ukmi
7. Toiic
Trc»tnft» 19O.
1-3
79
-------
APPENDIX 1
Page 13 of 50
RCUOBI for SITE Selection Difficulties
See APPENDIX
Note 9
Note 27
Note 28
EPA recognize? that site selection is a critical step in (he demonstration process and that delays
in site matching is a major impediment to the SITE Program. However the 1C report did not provide
a full recognition of the problems underlying site selection. The 1C report identifies five reasons
(page 14) for the site selection difficulties:
• In the past an inventory of sites which could be used for demonstrations was not available.
• Many regional officials were not familiar with the SITE Program and were unwilling co
recommend sites without incentives for them, or were reluctant to participate in the
program due to time constraints in performing cleanup work.
• Technology developers were involved in the approval process and could reject sites they
perceived as not ideal for them.
• 10 (28%) of the 36 technology developers 'exited* the Demonstration Program. (Note that
the IG report only counted technology developers accepted into the SITE Program through
SITE-003.)
• Some technologies were so narrow in scop* of treatment that EPA has had trouble selecting
a suitable site.
The first reiaon attributes site selection difficulties to the fact that there is no inventory of
potential demonstration sites. As acknowledged in che 1C report. EPA is currently developing a
computerized site inventory system, and new site selection procedures have been underway sine
July 1988. Over the IS month period from July 1984 through October 1989 the Agency implement
a number of site selection procedures which were being tested, refined, and updated. Some of these
procedures are discussed below.
i «
•
In a July 1988 memorandum. OSWER identified four actions that were being instituted to
accelerate the site selection process. These four actions were baaed on regional participation in the
site nomination process. However, the IG report highlights the fact that ORD and OSWER officials
requested site nominations from the regions on March 10.1989 and again oa August 31.1989. These
regional requests were planned and expected based on the procedures outlined in the July 1988
memorandum. The Agency does not understand the significance placed on these regional requests
cited in the IG report.
Oa August 9-10, 1989, SITE officials held a two-day meeting with seven of the ten regional
SITE coordinators. At tail meeting, rational sin nomination procedures were reviewed as well as
the status of the completed and planned SITE demonstrations. .Several suggestions to improve
participation of the regions in sin nomination* were identified. One of these suggestions was to
develop a database of potential demonstration sites.
OSWER SITE Program staff have been developing this prototype database to conduct a
preliminary screen of preselected NFL sites for potential application to SITE demonstrations of
specific technologies. The database allows the input of site-specific information and technology
information, with emphasis on waste characteristics. Site information that may be inputted includes
schedules from CERCLB, waste media, contaminants and their concentrations. Technology
information primarily includes the type of media treated and contaminant concentrations.
1-4
80
-------
APPENDIX 1
Page 14 of 50
See APPENDIX
Note 28
Note 29
The database 'a being used to facilitate site selection for technologic! selected under SITE-004.
To date, the database contains information on sites with Records of Decision scheduled for FY 90
and FY 91. The goal is to conduct SITE demonstrations oa those sites early enough in the Rt/FS
process to find the data useful. These sites have been screened for applicability to the technologies
accepted under RFP STTE-004 that require sites for demonstration. The database has allowed the
identification of about 10 to 30 candidate sites for each technology. The next step is to gather
additional information from the regional offices on site characteristics, status, logistics, and regional
interest, so that possible sites may be narrowed down further.
It is important to note that the readily available information on sites for which a ROD has not
been signed is limited to the description provided in background documents when the site is listed
on the NPL. These descriptions usually contain general information on the type of site, waste
sources, waste media, and contaminants. Therefore, the database can serve as a rough screen of sites
(hat may meet technology requirements.
Recognizing that site selection is a critical step in the demonstration process. EPA has instituted
a number of new procedures and developed tools designed to streamline the site selection process.
These include:
• Dissemination of descriptions of newly accepted SITE technologies to SITE regional
coordinators as toon as they an available, along with an informal request for site
nominations •
• Submission of tSUttll requests for SITE nominations to regional management following
the informal request discussed above.
• Cooperation with other agencies, such a* DOE and DOD, in identifying potential
demonstration site*.
The importance of working with other federal agencies was recognized in the Administrator's
Task Force report A HtataftmtM Am'cw of tint Suftrfmd Program which was released in June
1989. The ImpitrngiuatiOH Plan for recommendations from this review that was prepared in
September 1989 identifies a joint EPA/DOE (page 109) demonstration as a milestone toward
conducting additional federal demonstrations of innovative technologies. Currently a demonstration
of the QUAD air treatment process is planned at the DOE facility in Oak Ridge, TN. Efforts are
also underway with the Ui Air Force for evaluation of technologies at two different Air Force
for site selection difficulties cited in the 1C report is the lack of
participation of the ration in identifying appropriate sites. Although regional participation has
been less than that desired by SITE officials, the regions have actively contributed to site
oominatioa despite tremendous work loads and time constraints. For example, for technologies
accepted under the STTE-002 solicitation, nine regional offices nominated 21 sites for consideration.
The 1C report suggests several factors contributing to the lack of regional involvement in site
selection. These factors include (1) time constrains oa cleaning up sites and meeting critical dates
and milestones, (2) little or no incentive for RPMs and OSCs to recommend demonstration sites, and
(3) the regions an dot tracked or held accountable for participation in the SITE Program.
1-5
81
-------
APPENDIX 1
Page 15 of 50
See APPENDIX
Note 30
Note 31
Note 32
Note 33
Note 34
The telephone survey conducted by the 1C indicated that RPMs and OSCs place primary
emphasis oa site cleanup. This is certainly appropriate and SITE management agrees that incentive
Tor SITE participation are not strong. Three of the seven RPM and OSC concerns listed on page
17 of the IG report relate to time constraints. None of the RPMs or OSCs mention lack of incentive
as a reason for not participating in the SITE Program. There seems to be some confusion in the IG
report with respect to the differences between the responsibilities of RPMs and OSCs. RPMs we
under time pressure imposed by the Superf und management process. In contrast, OSCs are under
time pressure to meet the deadline for completing removal actions within one year of initiation, as
required by SARA. In the past, OSCs have demonstrated a willingness to allow SITE demonstrations
to occur at removal sites, provided that the OSCs deemed the demonstrations to be appropriate.
Recently, however, the IG has severely criticized the OSCs involvement in approving the
demonstrations during removal actions at two sites — General Refining, Savannah, GA. and Peak
Oil, Brandon, FL. In their report on these two demonstrations, the IG stated that the technologies
approved for demonstration at those sites had not yet been proven to be effective. Thus, the IG's
criticism has acted as a d«ine«ntiv« to approve SITE demonstrations in the removal program.
The IG report states that 'one cleanup contractor we interviewed reinforced the idea that RPMs
and OSCs do not have an incentive to participate in the SITE Program... (and) this is due to the fact
that little documented information is available on the detailed results of the demonstrations to
convince RPMs and OSCs that it is worthwhile to participate' (page 17). This opinion is attributed
to one cleanup contractor and not to the 16 OSCs and 21 RPMs interviewed. In fact, only one of the
OSCs and three of the RPMs cited disseminatioa of demonstration results as a weakness of the SITE
Program. The survey results indicated that the primary concerns of OSCs and RPMs were the length
of the demonstrations and the program requirements, both of which relate to their time constraints.
Nonetheless, as discussed later, SITE management has made major changes to the technology transfer
program to get information to the OSCs and RPMs at quickly and in at usable a format as possible.
The third reason identified in the IG report for tite selection difficulties was that technology
developers could reject sites they believed wen unfavorable to their technology. Although developed
involvement in the lite selection and approval process hat caused numerous delays, it has also been
justified and beneficial. A number of developers have included potential demonstration sites in their
applications, and with a few exceptions, these demonstrations have proceeded smoothly and more
rapidly than other demonstrations. Involvement of the developer in site selection is essential for two
primary reasons: (1) EPA is requesting that the developer invest its resources in co-funding the
demonstration and specific sin restrictions may affect the con incurred by the developer, and (1)
the developer hat a thorough understanding of the type of wane the technology has been designed
to treat and hat a vested interest in ensuring that the demonstration is a true indication of the
technology's capability.
An excellent example of why a developer hat the right
demonstration. Ogden proposed conducting the SITE
to refuse a site is the Ogden
at the McColl Superf und site.
To conduct the demonstration, Ogden was going to have to set up their incinerator for a short term
test (at a company eon of peanr than $1.000,000). In addition, the state was going to require the
installation of aa earthquake pad for the incinerator (at an estimated cost of $250,000), and to
mechanically enclose or cover the feed material to the incinerator to prevent the release of any
volatile* (at an estimated coat of $250,000). Furthermore, there were state and local permitting
problems at well as public opposition to conducting the demonstration. As a result, Ogden elected
not to conduct the demonstration at the McColl Superfund sin. Overall, there are many factors not
controlled by the Agency or the developer thai can adversely impact site selection making it an
extremely difficult and time consuming step in the demonstration process.
1-6
82
-------
APPENDIX 1
Page 16 of 50
Tt>n fmirth rB«P.n cited ia the 1C report is that 10 developers "exited" the Demonstration
Program. As of October 1989, tU developers (13%) voluntarily withdraw their technologies from
the SITE Program tad four developer) (>%) were filing from the program making a total of 10
AD Drum v iMCtive technologies. It a important to note the distinction between 'withdrawn* technologies ind
AH PC. NUIX Z "exited* technologies. Withdrawn technologies are those that have been removed from the SITE
Note 35 Program at the request of the developer. Exited technologies are those that have been removed by
EPA. This is an important distinction for two reasons: (I) EPA did not initiate and usually has little
control over the withdrawal decisions made by developers, and (2) EPA's exit policy enables the
Agency to remove from the program those technologies that for any of a variety of reasons cannot
initiate the demonstration. Exercise of the exit policy has potentially saved EPA millions of dollars
on technologies that probably would not match their marketing claims and this has released funds
to be applied against other technologies.
The EC report states that *... 11 developers have exited the program without demonstrating their
technology. No benefits have accrued to the Agency, at a cost of approximately $2.6 million"
(page 4). However, only 10 developers have withdrawn or been exited from the SITE Program and
Note 16 the total cost spent by the Agency on these ten technologies totals SI.763.U2 not S2.6 million. The
iiu i.e oo eleventh technology (mentioned on page 41) that was to exit in December 1919 was Ogden. Instead
of exiting the program because the Ogden onsite demonstration at the McColl Superfund site was
cancelled, EPA counted the completed pilot test of their circulating-bed combustor as a completed
demonstration. This pilot scale test used wain from the McColl site during the evaluation.
Beyond budgetary consideration. EPA believe* that benefit! have accrued to die Agency even
though these technologies were never demonstrated. First, treatability studies were conducted on a
two of these withdrawn and exited technologies at a total con of about $250,000. Although the data
generated by a treaability study cannot be used to statistically compare different technologies, the
treatability study results am be useful in assessing the anticipated applicability and performance of
the technology in the treatment of contaminated soils. The treatability study provides an indication
of the types of wastes the technology can successfully treat. The results of these studies can be
compared with existing TCLP/EP data developed under the Best Demonstrated Available Technology
(BOAT) evaluation program, and the treatability data for other SITE technologies. The treatability
studies could also be used to evaluate the relative performance of several different technologies on
similar feeds.
In addition to the reenltt of treatability studies, the Agency has acquired useful information
concerning the readiness of the withdrawn and exited technologies for remedial and response actions.
the limitations of these technologies, and the commitment of the developers in commercializing the
technologies. Data collected by the SITE Program on thermal technologies such as the plasma arc
and electric pyrolyxer, or btotofkal technologies such a* aerobic biodegradation and fluidized bed
reactor systems can be useful to innovative treatment technology decision makers regarding which
technologies an ready for field applications.
l for sit* selection difficulties cited in the IG report is that a technology may
be so narrow in scope of treatment that EPA and the developer may have trouble selecting a suitable
site. While this can delay site selection and ultimately the demonstration, these limited scope
technologies may target an extremely difficult to treat watte or material handling problem that is
plaguing clean-up contractors. The Agency agrees that primary consideration should be given to
evaluating technologies that have broader application or combinations of technologies. EPA
Note 37 recognixed the importance of demonstrating combinations of technologies prior to receipt of the IG
iiu ic o/ report m t{w AaoiaJHrator's Task Force report A Mvutftmtiu Rmtw of ttu Suptrfund Program,
1-7
83
-------
APPENDIX 1
Page 17 of 50
This EPA report sates that"... other expansions htve beta suu«ted by those familiar with the SITE
Prognm. Evaluating combinations of technologies is one example.' The evaluation of t narrow
scope trettment technoloiy esa significantly contribute to solving remediation problems because
permanent remedy selection! normally require combinations of technologies in a cleanup action. The
SITE Prognm may be able to provide data on t narrow scope technology that may be used to pre-
treat a difficult waste contaminant or post-treat a residue that may otherwise require land disposal.
RESPONSE TO RECOMMENDATIONS
V. Review and anoiyze tht reasons why dtvtloptrs exit the Demonstration Program to tHat this
See APPEND IX 2 information could be toed in attaint future ttcnnalofits.
Note 10 SITE management agrees with this recommendations and has completed most of the review.
As of October 1989, six technologies had withdrawn from the SITE Program and four
technologies had been exited from the program (a total of 10 inactive technologies). Of these 10
technologies, two left the program for indemnification reasons, and the remainder for a variety of
technical, marketing, and resource reasons.
Waste-Tech Services withdrew its fluidized bed combnstor front the SITE Program in July 1987
due to indemnification issues. Section 119 of the Comprehensive Environmental Response,
Compensation and Liability Act of 19SO (at amended) allows EPA to provide the same federal
indemnification to SITE developers as Superfund clean up contractors, however state statutes cannot
be preempted. Indemnification is available for amounts twtiliiig S 100,000. Thus, a developer
has a SI00,000 deductible liability during a SITE demonstration evaluation. The Agency recognizes
that SI00,000 may represent a substantial liability to small firms. When this $100,000 liability is
coupled with the potential of fjU state/local liabilities this may be an impediment to many firms
regardless of six*. Although the indemnification limitations may cause some firms, such as Waste-
Tech, to withdraw from the program or prevent other firms from applying to the SITE Program, the
indemnification policy is legislatively mandated and cannot be changed.
The New York State Peeailssent ef lavtrenamtal Coeservattmi withdrew the plasma arc from
the SITE Prognm in May I9U due to contractual problems with the developer snd extensive delays
caused by the developer. Most of EPA's expenditure {$370,500 of 1394,800} was spent in acquiring
the necessary pennies. While the NY Sate Department of Environmental Conservation applied for
acceptance into the SITE Program, the technology was not developed by the state but by a private
firm. EPA was not involved in the contractual problems and was not in a position to mediate the
dispute between the state sad the developer.
Air fiedncls aai Chemicals, Inc. (APCI) withdrew their fluidized bed biological system from
the SITE Prognm in September 19tt doe to indemnification and site selection issues. Their letter
dated September 20,1981 notifying EPA of their decision to withdraw from the program stated that
*... this decision is not based oa a negative evaluation of the SITE Program goals or its execution thus
far, but rather APCTs need to have several criteria met for a candidate demonstration site.* These
criteria included:
(1} Groundwater/teachats contaminants representative of organics encountered such that study
results are pertinent to market needs and support APCTs claimed OXJTRON system
technical features.
1-8
84
-------
APPENDIX 1
Page 18 of 50
(2)
Indemnification of joint ud several liability tt both the state and federal level.
(J) Site a within reasonable proximity to APCI technical support.
An October 6, 1988 memorandum from the SITE Program Manager to the Director of the
Superf und Technology Demonnration Divisioa stated that the Project Manager believed that "the
liability issue is the primary one* (reason) for their decision. *APCI wanted nearly total
indemnification under both state and federal laws. Lacking such indemnification, they have held
back from even submitting an application for a Cooperative Agreement in spite of our assurances
that the application wai not an irrevocable commitment APCTs concern also led (he company to
rejecting any lite in the state of Pennsylvania. ( am not optimistic that their concern could ever have
been allayed."
Corporation withdrew its asphalt-based mknencapsuiatioa technology from the
SITE Program in October I9gg due to the technology's inability to compete economically with other
available technologies. In a letter dated October 3. 19gg, WasteChem notified EPA that "after much
internal introspection and debate, WasteChem Corporation has decided to withdraw from
participation in the EPA SITE Program demonstration. We believe that our decision is in the best
interest of all parties.* WasteChem conducted a detailed in-house review and assessment of the
technology including an analysis of system economics and a comparison to other available alternative
_ technologies, which would be suitable for treating contaminated materials similar to those present
at the Woodland Township site and which produce wane forms that meet the current EPA disposal
criteria. WasteChem stated that * ... our review and assessment clearly indicated that the application
of our asphalt based technology for large hazardous watte sites ... is, at this time, not economically
competitive with other methods and technologies.' Nevertheless, WasteChem supported EPA's efforts
to complete a treaability study on the oxidized asphalt waste form. The waste form samples were
prepared at WasteChem's affiliated laboratory. WasteChem used contaminated soil from the
Woodland Township site and EPA's simulated
RREL.
Weston Service* withdrew its evaporation and i
to prepare waste form samples for analysis at
i stripping process from the SITE Program
in January 1989 due to permit requirements that resulted in long -delays in initiating the
demonstration.
CBI Freeae Tectuotoftes withdraw their freezing crystallization technology from the SITE
Program in September 1989 because the company's management was not convinced that there was
an application for their technology in the hazardous wane field. CBI Freeze's management was also
reluctant to spend! f t&ndf on tfso demonstration n then waa no application.
Wesdngkoeee Electric CetfontlM's electric pyroiyter was exited from the SITE Program in
September 1981 because after two yean of modifying in technology. Westinghouse had not shown
EPA that the technology was ready for demonstration. In a letter to Westinghouse dated August U.
19U. EPA informed Westinghouse of the Agency's intent to exit their electric pyralyzer from the
SITE Program. This letter stated that EPA's acceptance of Westinghouse into the program was based
on their revised proposal submitted on August 14. 1986. In this proposal, under "Current Status."
Westinghouse sated that the unit was complete and ready to be transported for use. EPA informed
Westinghouse that the electric pyralyzer could be returned to active rams once the upcoming DOE
test was successfully completed and the technology was ready for demonstration.
1-9
85
-------
APPENDIX 1
Page.19 of 50
DETOX Industries' aerobic biodegradation process was exited from the SITE Program in
January 1989 because of their extensive delays in performing laboratory treatability tests and
initiating demonstration of their technology. DETOX took samples for treatability testing at the
United Creosoting site in Coaroe. Texas in May 1988. At a project meeting in late September 1988.
DETOX agreed to initiate the treatability testing by early October 1988. which did not occur. In
a letter to DETOX dated December 6, 1988. EPA requested that DETOX provide by December :3,
1988, a firm fixed date that the treatability test would begin. In a letter to DETOX dated
January 10,1989. EPA informed DETOX of the Agency's intent to exit their biodegradation process
from the SITE Program because DETOX had not responded to EPA's December 6, 1988 letter.
Sanitech's ion exchange technology was exited from the SITE Program in September 1989
because of the long delays due to the developer rejecting several proposed demonstration sites. Based
on information supplied by Sanitech on the type of applicable waste streams needed for the
demonstration, EPA presented several potential demonstration sites containing applicable waste
streams to Sanitech for consideration. All of the potential sites were rejected by Sanitech as
inappropriate. la a letter to Saaitech dated September 14. 1989, EPA informed Sanitech of the
Agency's intent to exit their ion exchange technology from the SITE Program. This letter stated that
EPA was taking this action because Sanitech had been in a period of inactivity for over a year and
had not assured EPA that the technology was ready for demonstration. EPA informed Sanitech that
their ion exchange technology could be returned to active status if an appropriate waste stream could
be identified for the demonstration.
WtstloghouM Electric Corporation's pyroplasma system was exited from the SITE Program in
September 1989 due to permitting problems and difficulty in identifying a suitable demonstration
site for the technology. In June 1988, it was decided that the demonstration could take place in t
spring of 1989 at the Waltz Mill Facility as requested by Westinghouse with waste shipped to the uni
from the Southern Maryland Wood Treating site. However, during a meeting in March 1989, it was
discovered that the Westinghouse permit would not allow import of the type of waste found at this
site. In a letter to Westinghouse dated September 14. 1989, EPA informed Westinghouse of the
Agency's intent to exit their pyroplasma system from the SITE Program because no progress has been
made on the project since March 1989. Westinghouse had not been able to suggest a site that was
acceptable to them, and it appeared that the technology was not currently widely applicable to
Superf und sites. EPA informed Westinghouse that the pyroplasma system could be returned to active
status when the unit was ready for field demonstration or a mutually acceptable waste feed was
located for import to the Waltx Mill Facility.
Although it would be difficult, if not impossible, to prevent the withdrawal of some of the
technologies from the SITE Program. SITE officials will renew their efforts to determine the level
ADortinT v o of commitment of the developers during the mmliotion and iat»rvi«w process. The Agency will
APPtNUIX I reiterate the negative financial impact on both EPA and the developer of initiating the SITE
Note 38 demonstration process and later abandoning it. In doing so, EPA will discuss with new applicants
why some developers voluntarily withdrew from the program or were asked to leave. In addition,
EPA will emphasize the negative publicity associated with being exited or withdrawing from the
program (inferring that the technology was inadequate or not ready for demonstration).
1-10
86
-------
APPENDIX i
Page 20 of 50
The criteria for screening developers' applications for acceptance into the SITE Program include
technology factors, such ax
• Description of the technology -- includes concepts upon which the technology is based and
all process steps involved in its use.
• Description of the goiiioment -- the actual treatment unit proposed for the demonstration,
including size and transportability, treatment capability (range of feed volume or
throughput), the availability (i.e.. approximate date) of the treatment unit for 3
demonstration, safety features, and previous use of the unit.
• Advantages over similar technologies -- explain advantages and assert the reasons why the
technology be considered innovative.
The evaluation criteria also includes developer factors, such ax
Cf IMJgBftd ke.V personnel •• relevant experience of key
personnel involved in the demonstration and estimate of the time each person would spend
on the project.
• Comoanv profile — Superfund- related experience of the firm and available internal
support, specifying subcontractor or consultant needs.
• Capability to cotnmercialiie — financial capability and commitment of developer to make
the technology widely commercially available, including sources of capital, marketing
strategy , and per unit cost estimate for the technology.
. SITE officials are responsible for carefully reviewing developers' applications and conducting
interrogatories with developers to ensure that each of these evaluation criteria are met.
While oo developer eaten into the SITE Program believing that they will be exited from the
program, the possibility can serve as aa incentive for cooperating and working closely with EPA and
SITE contractors to ensure that each stage of the demonstration process from site selection through
report preparation proceeds as smoothly as possible. EPA is fully committed to the success of the
SITE Program and will continually review its policies, processes, and procedures to ensure that the
maximum benefit is received for every RAO dollar invested in the program.
i. Initiatt SITE ttekitotogr demonstrations that would inciudt tat of mart thai ont technology
for. tilt demonstration.
M TTiefint combination of technologies in the SrTE Program wts demonstrated from July through
APPENDIX 2 October 1989. Two innovative treatment technologies developed by BioTrol, Inc. were demonstrated
Note 11 at the MacGillis 4 Gibbs Superfund site in New Brighton, MN. The first demonstration involved
a pilot-scale biological treatment system for organics in wastewater. The system was used to treat
groundwater principally contaminated with PCP at concentrations up to 50 ppm.
The second technology, a pilot-scale soil washing system, was used to treat soil contaminated
with 300 ppm and l .000 ppm PCP.
1-11
87
-------
APPENDIX 1
Page 21 of 50
See APPENDIX 2
Note 12
Both BioTrol technologies were accepted into the SITE Program under the SITE-OOJ
solicitation, and they represent the program's first attempt at demonstrating a combination of
technologies.
Recognizing that combinations of technologies are likely to be required to clean up Superfund
sites. EPA emphasized interest in "combinations of unit operations to create treatment trains for
specific wastes" in the SITE-004 solicitation for proposals. The Agency again stressed their interest
in treatment trains designed to handle specific wastes in the SITE-005 solicitation.
Regardless of the Agency's interest in combinations of technologies, there are four obstacles
that may impede EPA's efforts to demonstrate treatment trains. These include: (1) determining the
success or failure of each individual component, (2) identifying developers willing to demonstrate
their technology as part of a treatment train, (3) selecting an appropriate site that adequately
demonstrates the capability of both (or more) technologies, and (4) difficulties in trying to pair
technologies from different developers in an integrated demonstration.
To facilitate future demonstration of combinations of technologies. EPA is taking two actions.
The first is to screen the technologies accepted under the SITE-OOS solicitation to identify possible
technology matches (if none an proposed by developers). Developers of potential matches wiu be
contacted and possible combinations will be investigated. The second action is to modify the STTE-
006 solicitation to require developers to assess their function in a 'combination* approach and require
such possible participation at the direction of EPA.
3. Formalize the number of sttt locations at <*hicn a developer eon refuse to conduct a
demonstration.
With the many factors imputing the selection of demonstration sites, it would be extremely
difficult to formalize the number of sites that a developer could refuse before being exited from the
program. Although the Agency agree* that limiting the number of sites the developer could refuse
would probably decrease the time required for site selection, it is very difficult to offer a developer
a site with no contingencies.
The average site selection time has decreased from 252 days for technologies accepted under
the SITE-001 solicitation to 204 day* for SITE-OOJ technologies. To ensure that the time for site
selection continues to move toward the Agency's goal of 90 days, SITE management will periodically
review site selection progress to ensure that sites are being identified in a timely manner. SITE
officials will also evaluate the reasonableness of developer refusals of sites and will determine when
to employ the exit policy. IB addition, EPA will continue to encourage developers to suggest sites
in their applications, and future SITE solicitations wilt encourage applicants to propose potential sites
in their applications.
4. Add SITE demonstrations to tkt performance indicators or elements of the SPMS or SCAP
management tracking systems.
The Agency generally agrees that matching technologies with sites is a difficult and time
consuming task. The 1C report states that"... site matching can take over a year and longer' (page
12). This actually overstates the site matching problem because the average time for site selection
for Hi technologies in the SITE Program is 26S days (1.9 months). It should be noted that the
average time for site selection for the nine technologies identified in the 1C report is about six
months. Although these average times significantly exceed the Agency's goal of 90 days for site
1-12
-------
APPENDIX 1
Page 22 of 50
See APPENDIX 2
Note 13
Note 13
Note 39
selection, the Agency believes that measure! and procedures implemented for site selection for SITE -
003 technologies have been effective in reducing the time required for site selection.
The Agency accepts the intent to establish management targets for the SITE Program: however.
we propose an alternative to this recommendation. SCAP is primarily an OSWER/regional system
for managing workload and resources. Since ORD ha* the lead responsibility for implementing the
SITE Program. SITE participation would be inappropriate for inclusion in SCAP.
As an Agency-wide system, SPMS may be a more appropriate means to track participation in
the SITE Program, especially if EPA considers adding a 'reporting measure" rather than tracking the
accomplishment of specific regional targets. One potential example of a reporting measure would
be the elapsed time between receipt of a SITE demonstration proposal and the regional response. It
seems likely, however, (hat positive incentives will be of greater value in promoting SITE Program
activity. Along these lines, extending allowable timeframes for completion of some regional
Superfund activities may be a promising way to encourage more regional involvement in SITE
demonstrations. The Agency intends to review all of these options, and others as they are developed.
and take whatever action is deemed most appropriate.
!. Explore and develop other incentives for regional RfMs and OSCs to allow their sites to be used
in SITE technology demonstrations.
The Agency accepts the intent behind this recommendation. However, while it may be
advantageous to explore incentives for the removal program to utilize SITE technologies, the 1C
should recognize that the newer technologies do have higher failure rates than the proven
technologies. Regardless of any potential advantages that would accrue from increased use of (he
newer technologies, they do not always meet initial expectations of their effectiveness. Recently.
OSWER and ORD officials met to discuss more ORD involvement in managing the site identification
and selection process. They have agreed to explore and analyze various incentive measures for
regional participation in the SITE Program and should be prepared to announce these alternatives
when the SITE-005 solicitation technical selection process is completed.
One alternative the Agency may consider would be selecting a site first (such as a wood
preserving site), then advertising for innovative technologies or combinations of technologies that
can manage the wastes at that site. This approach may short circuit the long delays in finding sites.
but could involve delays in actual cleanup operations to accommodate advertisement and
demonstration planning.
1-13
89
-------
APPENDIX 1
Page 23 of 50
See APPENDIX 2
Notes 14, 40
Note 5
Notes 5, 14
Note 41
I, IMPROVEMENTS NEEDED IN REPORTING RESULTS
FOR COMPLETED DEMONSTRATIONS
The IG report states that"... the SITE Program has been adversely impacted by the lengthy
reporting periods and delays in issuing the final Technology Evaluation Reports (TERs) and
Applications Analysis Reports (AARs)... As a result, little reliable detailed information is available
to guide cleanup decision-makers" (page 21}.
We agree that publication of final technical reports has taken too long in some cases. But the
audit does not recognize actions that have already been taken to shorten report publication times
The examples in the report reflect the early experience, rather than more recent (more favorable)
experience. Second, the report does not adequately consider other ways that pertinent information
is ;«ing made available in much shorter timeframes, such as the use of an online database and
publication of summary bulletins. We will continue to develop ways of making quality information
available as quickly as possible, including addressing recommendations in this report. It is important.
however, to maintain strong quality control of die data from these demonstrations.
This section is the Agency's response to the reporting problems enumerated in the 1C report.
including some important issues that were not mentioned in the report. This section addresses the:
• Report preparation process
• Report review process
• Report publication process
• Report distribution process
• ATTIC and other technology transfer mechanisms
• Legitimacy of the conclusions drawn from the telephone survey of RPMs, OSCs, Superf und
contractors, and developers.
SITE Program officials have been concerned over the time taken to prepare, review, and publish
the TERs and AARs aad the impact these delays could nave on the effectiveness of the SITE
Program. Since June 1988, SITE management has taken a very active role in addressing the reporting
problems aad emphasizing the importance of dissemination of demonstration results. Although the
QIC has reviewed the documented evidence of the Agency to improve reporting results from June
1988 through March 1989 the 1C report has failed to acknowledge the tremendous efforts that have
been made, the detailed guidance that has been issued, and the improvements that have resulted from
these efforts.
In an effort to resolve the existing reporting problem aad to prevent furore backlogs, SITE
officials developed a reporting policy in early June 1988. This policy described the approximate
time required for each step in the report preparation, review, and publication process. The time
estimates were based on aa ideal process aad did not take into consideration the many delays that
could be encountered during each phase. The estimated timeframe for completing TERs was 241
days following receipt of the analytical results. The AAR was to be completed within two months
2-1
90
-------
APPENDIX 1
Page 24 of 50
See APPENDIX
Note 42
following the completion of the TER. To implement this policy SITE officials prepared and issued
a guidance document entitled SITE Publication Guidance on August 1, 1988.
This guidance document included brief descriptions of the types of reports prepared for the
SITE Program. It also included a detailed discussion of SITE report review and clearance procedures
for both TERs and AARs. A SITE report procedures timing schedule was also provided in the
guidance document. This schedule tracked the report review and publication process from the day
the SITE Project Manager received the draft report. The draft report was anticipated to be
completed within 90 days after completion of the analytical work. The schedule allowed 241 days
for the review and publication process (a total of 131 days for preparation, review, and publication
of TERs and 391 days for AARs). In addition, the guidance document suggested methods of
avoiding future time delays, such as SITE Project Managers meeting with SITE contractors,
reviewers, and developers to discuss report preparation schedules and deadlines and the importance
of preparing quality reports on time. The guidance also indicated the need for shortening the
clearance time required by ORO and OSWER. and emphasized the importance of the SITE Reports
Tracking System which can be used to track the milestones for each demonstration project from
acceptance into the program through reports publication and dissemination.
SITE management reemphasized their commitment to eliminating the publication backlog in a
September 20,1988 memorandum from the SITE Demonstration and Evaluation Branch Chief to the
Director of the Office of Environmental Engineering and Technology Demonstration (OEETD). The
memorandum included a revised schedule for SITE reports on completed demonstration projects.
The memorandum also projected t schedule for completion of reports on projects for which
demonstrations had not been completed. These were ambitious goals and optimistic deadlines set by
SITE management based oa the importance of getting the reports completed and disseminated as soon
as possible.
The 1C report fails to point out that the length of time required to prepare TERs and AARs
has been decreasing. The longest time required (17 months for a TER and 22 months for an AAR)
are the extremes and occurred for the first TERs and AARs produced under the SITE Program.
Much of the delay in preparing these dm reports was due to ORO and OSWER staff establishing
policy on the best way to present the data and what information would be most helpful to OSCs.
RPMs. and clean-up contractors. The- Agency was slow in determining guidelines for the format and
content of the TERs and AARs, bat SITE officials recognized that the first reports would become
models for subsequent reports and they believed it was vital to prepare quality reports that presented
all of the relevant data in a clear, concise manner.
The March 8. 1989 memorandum from the Director of the Risk Reduction Engineering
Laboratory (RREL) to the Acting Director of OEETD. again stressed SITE officials' concern over
the reporting process. As the 1C report acknowledged, this memorandum established new milestones
to speed up the process and contained a revised milestone chart for reports in prof ress. One of the
key elements mentioned in the memo for reducing the time required to prepare reports was that the
contractor must be required to submit draft reports that do not require extensive changes or rewrites.
The memorandum stated that V.. in the event that reports do require substantive changes, wc wj||
rAvitt them inhouse."
2-2
91
-------
APPENDIX 1
Page 25 of 50
The IG report states that"... one of the major causes of the delays was a long review process
that appeared to go back and forth between EPA. the contractor writing the reports, the developer.
and the various other parties that reviewed the report for the technical details ... We found that
extensive rewrites and changes delayed the reporting process* (page 25). The SITE Publications
Guidance issued by SITE management on August 1, 1988, describes the required review process for
SITE reports. The guidance document requires the following reviews for reports:
• Preliminary review of draft TER by Project Manager
• Preliminary review of draft TER by developer
• Review of draft TER by review panel
• Review of comments by Project Manager
• Review of draft final report by Project Manager
• Review of draft final report by SITE staff, developer, OSWER staff, technical reviewers.
QA/QC officer, economist, and ORD Headquarters
• Clearance of draft final report by ORD aad OSWER aad sign-off by Branch Chief, Division
Director. Laboratory Director, and Laboratory Publications Office.
While this level of review is tedious and time consuming, the SITE reports are very technical
flPPFKinr v 9 and 'nclud* * large volume of data. It is also imperative that extreme care be taken to fairly and
_— HrrtNUIX 2 accurately portray the results of the demonstrations. SITE management devised this review process
Note 43 to ensure that the quality aad accuracy of the reports were such that RPMs and OSCi could rely on
the information and results in these reports to select innovative technologies for remedial actions.
SITE management also recognized that this review process may put undue burdens on Agency staff
and result in delaying the publication of the reports. Therefore, the Director of RREL requested
several changes in the review process. These changes wen outlined in a March 8,1989 memorandum
to the Acting Director of OEETD. One of the procedural changes was the use of a truncated review
tfiSja (for reports on completed demonstrations). Another change noted in the memorandum was
strict adherence to the two week MirB'TOIHHl for EPA to get review comments back to the contractor
on report drafts. The memorandum also requested sole clearance authority for TERs by the Director
of RREL.
Beeart Pahllcatina Ptaceai
The 1C report briefly acknowledges SITE management's efforts to resolve the publications
backlog problems, but indicates that EPA has not done enough to solve the problem. The report
states that *... even with recent increased management emphasis in this area, reports have still been
delayed* (page 21). The report included information on the length of time required to publish seven
TERs and AARs, as well as the publication deadlines proposed in the March 8, 1989 memorandum
from the Director of RREL and the actual publication dates. The 1C report stated that"... *e
reviewed these new dates and found that these intensive report milestones were also missed in most
cases* (page 24). In reality, for the six TERs listed in Table 3 (page 24) of the IG report, projected
M . . , publication dates were met for three damomtrationi and two of the, r^mff'n'n* thrg« publications
nOte 44 were published within one month of the projected deadline. For the six AARs, two were published
within two weeks of the proieeted deadline, and two^""1 ""' —«w:»t««»» within nn» month of the
2-3
92
-------
APPENDIX 1
Page 26 of 50
See APPENDIX 2
Note 44
Notes 5, 45
projected deadline. Only one TER aad two AARs significantly missed the projected publication
deadline. Furthermore, it should be aoted that the projected schedule was ambitious and dependent
on an ideal process. As noted ia (he March 8, 1989 memorandum '... indications are that the
schedule outlined in the SITE Publication Guidance can be met. A key factor will be the ability to
produce only one draft which requires ORD/OSW1R review and then a final report."
Increased SITE management emphasis has been very effective in decreasing the average time
required for preparation, review/ and publication of reports. As reported in the Demonstration
Progress Report dated March 16, 1989. the average time for preparation, review, and publication of
(he AARs has decreased from 469 days (15.3 months) for SITE -001 technologies to 365 days (11
months) for SITE -002 technologies (only Soliditech and CF Systems reports have been completed).
The Agency has set a goal of 315 days for preparation, review, aad publication of reports and a total
of 595 days (20 months) from entry into the SITE Program through publication of (he AAR.
Report DlitrlbutioB
In addition to delays encountered in preparing aad reviewing reports, the IG report notes that
there are delays in distributing the TERs and AARs even after they are published. The report states
that *... the distribution of the actual documents can be months later than the published date shown
on the report" (page 24). Table 4 (page 25) ia the 1C report indicates that distribution usually
follows publication by 2-3 months. SITE management recognized this problem and has taken a
number of actions to eliminate this delay. The March S, 1989 memorandum from the Director of
RREL stated that *... we have had discussions with CERI personnel with respect to our schedule and
urgency on these reports.- By copy of this memorandum I am requesting that CERI provide expedited
publication for the reports that are on completed demonstrations as described ia this schedule." The
Agency has also improved clearance procedures for SITE report coven — submitting them for
clearance prior to completion of the report.
In an effort to reduce the time lag between publication aad distribution, SITE management
established a priority mailing list for SITE reports. As reported ia the meeting summary of the
October 4, 1989 SITE Management Meeting, the SITE mailing list was being revised to identify
certain groups such as key regional personnel (OSCs aad RPMs). ARCS contractors, and state
personnel. This list (totalling about 600 individuals) will be designated as priority recipients and they
will be seat the SITE reports by first class matt. The remaining 5,000-7,000 persons on the mailing
list will receive the reports by fourth clan mail. This revised procedure should reduce the delay in
distributing SITE reports to priority recipients by 3-4 weeks. ORD will continue to work with CERI
on the implementation of aa efficient report distribution process.
ATTIC ••«! Qtti«r T«e
The major concern over the delay ia publishing aad disseminating the SITE reports is that the
reports are the principal means of notif yiag RPMs, OSCs, aad Superf und cleanup contractors about
these innovative technologies. The 1C report indicates that because of the delays ia publishing the
report*, "... little reliable detailed information is available to guide cleanup decision- makers' (page
21). Although the 1C report mentions briefly that there are other products prepared by the SITE
Program to provide 'information quickly, the report cJfViY ayt *|oo|a the importance and intent of
these various technology transfer products.
Recognizing the importance of providing useful data on SITE technologies to OSCs, RPMs. and
Superf und cleanup contractors as sooa as the information is available, SITE management instituted
a variety of mechanisms to make this information available. One of the most diversified means of
2-4
93
-------
APPENDIX 1
Page 27 of 50
disseminating SITE information a the Alternative Treatment Technologies Information Center
(ATTIC). ATTIC became operational ia May 1989. The ATTIC system was then tested in EPA
Region III in June 1989. It became fully operational and online ia November, 1919. ATTIC is a
comprehensive automated information retrieval system that integrates hazardous wmste data into a
unified, searchable source. Users can access ATTIC through direct online access using standard
communications software on any modem-equipped PC or by telephone to the ATTIC system
operator. Based on the report procedures schedule included in the March I, 1989 memorandum from
the Director of RREL, SITE demonstration evaluation data will be loaded onto the ATTIC and
OSWER bulletin board systems by 174 days (5.8 months) following completion of the demonstration.
This is at least 142 days or almost 5 months before the TER is published by CERI.
The demonstration results are also published in 2-4 page Demonstration Bulletins which are
usually released 188 days (6.3 months) after the demonstration has been completed. The
Demonstration Bulletins are disseminated to all of the addressees oa the SITE mailing list. The
Bulletins contain a synopses of the demonstration and preliminary results in a brief easy to use
format. The Demonstration Bulletin also provides the name and number of the EPA Project Manager
who can be contacted for further information. The SITE Project Managers have gained considerable
expertise and knowledge in managing the SITE demonstrations and they have become valuable
sources of timely information and technical assistance.
Beyond summarizing TER results. AAR results are also available on ATTIC approximately 216
days (7.2 months) following completion of the demonstration. This is at least 149 days or about 5
months before the AAR is published by the Center for Environmental Research Information (CERI).
SITE information and demonstration results are also included in several other SITE publications
that are widely disseminated, including the annual Technology Profiles, the semi-annual brochure
reviewing project status, journal articles, project summaries, and the annual SITE Report to
Congress. SITE demonstration results an also presented at technology seminars focusing on
completed demonstration projects. These are held in regional office locations on a periodic basis.
The Agency also produces videotapes of each demonstration. To date videos on more than half
of the 14 completed demonstrations have been prepared and distributed. These videos include a
description of the technology and its benefits, footage of the actual demonstration, and a presentation
of the performance results and cost range, as well as a contact for more information. The videos are
currently available on loan from regional SITE coordinators and SITE Project Managers, and can be
purchased from EPA.
The Agency has been taking an aggressive role in publicizing the SITE Program and making
OSCs, RPMs, states, and Superfond contractors more aware of the mechanisms EPA is using to
disseminate early results. Visitor's days are scheduled for most of the SITE demonstration
evaluations. SITE Project Managers are making presentations on the technology demonstrations at
major hazardous waste symposia and conferences! In June 1989, EPA sponsored a Forum on
Innovative Hazardous Waste Treatment Technologies: Domestic and International; the second forum
is scheduled for May 15-17,1990. In addition, the SITE Program has exhibited a booth at numerous
conferences, providing thousands of copies of SITE publications including Demonstration Bulletins,
TERs. AARs. Technology Profiles, SITE Report to Congress, brochures, fact sheets, and project
summaries. The SITE booth was recently exhibited at the Superfund Conference and Exhibition.
ATTIC has also been exhibited at tea conferences since April 1989, and has been demonstrated to
regional staff in Regioas I. m. IV, and V, at well a« state staff ia California. Florida, and
Washington. ATTIC will soon be capable of accommodating up to 32 simultaneous users.
2-5
94
-------
APPENDIX 1
Page 28 of 50
SITE officials believe that offering demonstration results and information on the capabilities
APPENDIX 2 of innov*tive technologies oo an automated system such as ATTIC can actually be more useful to
Note 46 OSCs. RPMs, and Superfund contractors than published TERs and AAfU. Given the tremendous
time consiniaa HUM individual! fact. ATTIC tllowi them to quickly tutor s search to locate all of
the relevant information oa available innovative technologies that could be used at their particular
sice. Demonstration and applications analysis results are available on ATTIC and can be searched
by waste type, technology, media, demonstration location, etc.
Note 47
Note 48
Note 49
Note 49
Legitimacy of lh« Conclusion* Drawn from the T«ltnhna« Surrey
The 1C report presents jcvt.nl sonclusionj that are bated upon a telephone survey of OSCs,
RPMs, developers, and Superfund contractors that was conducted as pan of the OIG investigation
of the SITE Program. A review of these conclusions and the data upon which they are based raises
several serious questions regarding the survey documentation, sample size, and presentation of data.
Survey Documentation. The IG report does not provide sufficient documentation of this
telephone survey to enable the reader to independently assess the survey's validity and reliability.
For example, were OSCs and RPMs sampled randomly across the country or were they stratified by
region? If the sample happened to overrepresent OSCs and RPMs in Regions VII and VIII, one
would expect a tower level of awareness of the SITE Program because no demonstrations have been
carried out in those regions. The IG Report states "We are 90 percent confident that the results of
our statistical sample represent the total universe of RPMs, OSCs, and Superfund contractors^ page
6)? Is the report using the term in its statistical sense or simply to convey that the sample is
representative? What was the sampling error?
Sample Site. While the IG report states'... we talked to an extensive sample of RPMs, OSCs.
Superfund cleanup contractors, and technology developers" {page 29), the survey is in fact a series
of very small samples. There was no eauivalent proportion?! nmniina (percentage of respondents)
among the four survey groups, i.e., 16 (or less) out of 131 OSCs. 21 (or less) out of 361 RPMs, 14
(or less) out of 32 Superfund contractors and 5 {out of 14 technology developers responded to the
survey question. The sampling varied from less than 6% to more than 40% among these focus groups.
These samples of 16 OSCs, 21 RPMs, 14 Superfund contractors, and 3 technology developers would
"be only marginally adequate to represent the total universe if all respondents were highly
homogeneous. These various groups are, however, quite heterogeneous and are. therefore, poorly
represented by such a small, uostntif ied sample. Problems with sample size were exacerbated by
the fact that significant proportions of potential respondents were screened out, so thai the actual
number of respondents for any given question was even smaller, further compromising the validity
and reliability of the results.
Presentation of DM«. An overall problem with the presentation of data in Exhibit 0 (page 64)
of the IG report is the use of TCfr'^IC? rather than «w number* when the actual numbers are
extremely small. Using 23% to represent three respondents or 40% to represent two respondents tends
more credibility to the conclusions presented in the text than the data merit. What appear to be
significant figures are in fact not significant because they are based on so few responses. While we
are concerned with'any constructive feedback, on the program and will use these anecdotes and
comments to improve the program it is not appropriate to draw general conclusions about the
program from such data. Specific problems with the conclusions of the survey are presented in
Appendix A.
2-6
95
-------
APPENDIX 1
Page 29 of 50
See APPENDIX 2
Note 15
Note 15
RESPONSE TO RECOMMENDATIONS
/. Establish formal accountability for StTE management for nutting reporting milestones.
SITE management agrees and has instituted such a system.
Bawd on the March 1989 memorandum from the Director of RREL to the Acting Director of
OEETD. SITE management currently tracks demonstration program progress on a quarterly basis.
Progress is tracked through the specific schedules generated for each demonstration project and the
SITE Milestone Checklist report. At the initiation of each project, the SITE Project Manager
projects a schedule for each major activity required to complete the project from acceptance into
the program through clearance of the AAR. This schedule is updated quarterly by the Project
Manager and actual dates can be compared with projected dates to determine whether or not the
project can be completed within the 20-month timefra&e. The schedule includes 18 separate
activities, the estimated date of completion, the actual oate of completion, and the source of the
estimate.
The SITE Milestone Checklist report lists all of the active technologies in the program and
identifies the status of each project as of the reporting period. The checklist includes the following
10 major milestones: (1) site selection, (2) demonstration plan, (>) demonstration started,
(4) demonstration completed. (5) demonstration bulletin. (6) bulletin board entry, (?) draft TER
review. (8) draft AAR, (9) final TER, and (10} final AAR. The checklist indicates whether these
milestones have been achieved and which ones were completed during the current reporting period.
While SITE officials believe that the current tracking, system is adequate to ensure
accountability of SITE management, the current system will be reviewed and modifications
considered at the end of the fiscal year.
2. Consider alternatives for performing a detailed review of the process for preparing and issuing
SITE reports. SUCH as a Management Systems Renew.
EPA agrees with the recommendation and although a number of reporting alternatives have
been considered and some instituted, other ideas will be evaluated.
In August 1989, an ORD-OSWER work group re-evaluated the process of preparing, reviewing,
publishing, and disseminating SITE reports. As a result of this review, it was decided that there
would be a pilot-scale test of preparing only one report, a combined TER and AAR. for two
different technologies. Combined reports an being prepared for two completed demonstrations,
Chemfix and BioTroU these reports are expected to be available in the spring and summer of 1990.
respectively. SITE officials will study this process closely to determine if combining the two reports
can decrease the time required for preparation, review, and publication.
A subsequent review of the reporting process will be undertaken by SITE management. This
review will include evaluation of the effectiveness of the 'multiple technical review process,* the
benefits of Level n QA review, and a combination OSWER/ORD administrative review.
EPA recognizes the value of an overall review of the publication process in terms of improving
productivity, timeliness, and efficiency. SITE management will contact OMMSQA to obtain their
assistance in conducting a Management Systems Review of the SITE reporting process as
recommended. SITE staff will work closely with OMMSQA to identify the specific 'bottlenecks' and
2-7
96
-------
APPENDIX 1
Page 30 of 50
inefficiencies in the process tad develop methods to resolve the problems and streamline the process.
A management system review is being investigated with a decision on whether to proceed by June
I, 1990.
j. Evaluate publication alternatives to speed up the actual distribution process.
SITE management is committed to the most rapid report dissemination process possible. A
number of measures to improve the distribution of SITE reports, such as development of a priority
mailing list, have already been instituted by SITE management. However, SITE officials recognize
that additional measures may be required to reach the established goal of 315 days from completion
of the demonstration to publication of the AAR. To identify additional ways to improve the report
distribution process, a series of meetings will be held with the Director of CERI. Within this
SITE/CERI forum, possible short-cue to the current process which are compatible with GPO
guidelines will be explored. At a minimum, the review team may include SITE management
representatives, one SITE Project Manager (experienced with report preparation and review), one
RREL Office of Program Operations representative, and one or more CERI staff. This review win
be completed by May I, 1990.
4. Prepare a videotape or presentation of the Demonstration Program's goals and objectives to be
presented to all Super fund personnel in the regions and at Headquarters.
EPA agrees and has taken steps to implement this recommendation.
At the Superfund '89 conference ia Washington, DC, a videotape on the SITE Program was
shown to conference attendees. This videotape described the SITE Program and included segments
from several of the completed demonstration project videotapes. This videotape will be sent to each
regional SITE coordinator. la addition, OEETD is planning to produce a videotape describing the
ATTIC system and the kinds of information that it can provide. This videotape will be directed
toward OSCs. RPMs, Superfund contractors, aad state agencies involved in site remediation. The
videotape could also include a description of the SITE Program, its goals and objectives, and the
types of information that is generated by the program. The videotape could be useful in publicizing
the SITE Program as well aa informing viewers on how to access the SITE information through
ATTIC. Production of the videotape is expected to begin in spring 1990.
EPA is also undertakiag a Dumber of other activities designed in whole or ia part to improve
information transfer on the SITE Program.
The recent EPA report entitled A Management Review of the Super fund Program, recommended
the appointment of a Technologies Caw* to help remove barriers to the use of innovative treatment
technologies. Ia response to this recommendation, a Technologies Czar" has been appointed as
Director of the newly formed Technology Innovation Office within the Office of Solid Waste and
Emergency Response. This new office will work to improve communication with private technology
developers, trade organizations, engineering firms, non-profit technology evaluation centers, and
other federal agencies to promote the evaluation and use of innovative technologies. This new office
will ensure that information that could be useful to OSCs, RPMs, aad Superfund contractors, such
as information from the SITE Program, is readily available ia the most convenient form.
2-8
97
-------
APPENDIX 1
Page 31 of
50
SITE official] will coordinate existing technology transfer efforts with the new office and will
work closely with the Technology Innovation Office staff to identify aew methods of presenting
SITE Program information and results. Together, they will explore new outreach ideas to ensure
that those who need the information generated by the SITE Program know of its existence.
SITE management have already initiated annual meetings with regional SITE coordinators to
discuss the progress and results of the demonstrations, to encourage participation by the regions, and
discuss ways that would facilitate access of OSCs, RPMs. and Superfund contractors to SITE
information.
2-9
98
-------
APPENDIX 1
Page 32 of 50
3 IMPROVEMENTS NEEDED IN ENSURING ADEQUATE CONTRACTORS'
PERFORMANCE AND COST CONTROL
See APPENDIX 2
Note 17
Note 50
Notes 3, 18
Note 8
Note 20
Note 6
Note 51
Note 17
Note 52
The 1C report indicates that SITE officiab and developers were concerned about the
performance of SITE contractors. The report states that"... the developers in our telephone survey
expressed concerns that EPA's contractors had been slow and inexperienced, and. in some cases.
generated too much data. A recent study by the Congressional Research Service (CRS) has aiso
shown that developers complained about the performance of the contractors* (page 32).
We are in Substantial disagreement with the findings of the audit in this area. First, we believe
overgeneralizations were made about problems with contractor performance from limited numbers
of survey comments. Although some difficulties were experienced in the beginning of the program.
the contractors involved were replaced. Generally, contractors are now performing competently in
our view.'but we will continue to push for excellence - applying the principles of Total Quality
Management - and will pursue the recommendations of the audit to further improve performance.
We particularly take strong exception to the allegation that limited emphasis was placed on
controlling costs. The cost increases cited in the audit often resulted from Olio's specific attempts
to keep costs down by initially approving and funding very conservative estimates of potential costs
to cause contractors to economize aad then forcing contractors to specifically justify additional work
and costs. In other cases, increases resulted from additional contractor work related to unforeseen
performance problems with the technologies, necessitating additional time in the field. This should
be expected with new technologies. The bottom line is that the Agency has successfully initiated
demonstrations at the number of sites required by Congress and within the authorized budget.
Moreover, no qualified technology has been refused due to lack of available EPA funds.
Finally, we dispute the claim that the contracting mechanism used was improper and that award
fees paid were too high. The EPA Procurement and Contracts Management Division (PCMD)
determined the contracting mechanism to be used and does not believe that other mechanisms, such
as fixed price contracts, are practical for this program. Award fees are determined by PCMD based
on recommendations from an independent performance review board. Percentage awards are in fact
tower for this program than are typical for other EPA contracts of this type.
This section presents the Agency's response to:
• SITE contractor performance
• Controlling contractor costs
• Award fees and increased costs and time delays.
SITE Caalractar Perfanuaea
The 1C report includes six comments about SITE contractor performance made by RPMj,
OSCs, and developers during their telephone survey {page 32). The only reference to contractors in
Exhibit D (page 64) that reports the results of the IG's survey sates that '...less than 10% of the
respondents in each group included... relying oa contractors to prepare reports' as a weakness of the
program. Given the number of respondents in each group, only one RPM, one OSC, and/or one
developer could have made that statement. Lacking any supporting documentation of the statements
3-1
99
-------
APPENDIX 1
Page 33 of 50
See APPENDIX
Note 53
Note 54
Note 55
listed above, we were unable to determine whether these concerns were expressed by only one or by
more than one of the developers, RPMs, ud OSCs; whether their comments referred to experiences
early in the program or whether they were more recent; and whether the concerns were applicable
to one or more specific SITE contractor.
While one must not overgeneraliie the statements of a few individuals, we believe that in some
cases these comments are valid, especially during the first demonstrations that were conducted. When
it was initiated, the SITE Program was very different and remains unique from any other program
that EPA sponsored. Neither the Agency nor the developers could predict precisely what would be
required to plan, implement, and evaluate each of these different technologies. The early
demonstration plans, site preparation efforts, demonstrations, sampling activities, analytical efforts.
and report writing exercises were learning experiences for alt parties, including the contractors.
Since that time. SITE Project Managers, RPMs, OSCs. developers, and the SITE contractors have
gained valuable experience with the operation of the SITE Program. Since no discussion was
included in the IG report describing the timeframet applicable to these six concerns, we assume they
relate to the 'start-up' of the SITE Program.
while the IG report provides examples of EPA's QA Officer and Project Managers citing
problems in drafts of plans and reports, it must be noted that these problems were detected by SITE
Program personnel and corrected by program personnel and contractor personnel before testing was
conducted or reports issued. This is the reason that such reviews are conducted and such direct
oversight of the contractors is exercised. Due to its nature the SITE Program is a program of high
technical risk and its testing, sampling, and analytical complexities are such that, even with the most
Qualified contractors, strong technical involvement is required by EPA personnel. AS EPA's SITE
Program personnel and the contractor personnel have gained more experience in the program, such
problems have occurred with less frequency.
The IG report suggests that EPA has paid more attention to implementing demonstrations and
on issuing reports than on controlling costs, and that technical quality appears to be afforded higher
priority than cost control. As an absolute, this statement is probably true. This does noj mean that
cost control is 001 high on the SITE agenda. On the contrary, SITE managers are well aware of the
importance of cost control; we an also all too aware of the difficulties that are involved in predicting
accurately the costs of the SITE demonstrations.
The IG report states that *... contractor cost control is especially important due to the large
amount of the Demonstration Program's resources which go to the contractors. For fiscals 198?
through 1989, these contractors absorbed approximately SO percent of the program's $34.6 million
budger (page 35). SITE officials an concerned with controlling contractor costs and their efforts
coupled with the experience gained by SITE Project Managers and contractors during the past three
yean have resulted in a decrease in the average cost of a demonstration from S1.3S7.473 for SITE-
002 technologies to S959.1M for SITE-003 technologies (as reported in the March 16. 1989
Demonstration Progress Report). This represents a 29% decrease in the average demonstration cost.
This is due primarily to increasing experience on the part of all concerned.
The IG report suggests that a different form of contract mechanism might provide more control
over contractor expenditures. EPA does not agree. Several contracting mechanisms, such as a Cost-
Plus-Fixed-Fee (CPFF) type, provide less control since the contractor's fee is fixed and the only
recourse available to the government in the event of unsatisfactory performance is cancellation of
3-2
100
-------
APPENDIX 1
Page 34 of 50
See APPENDIX 2
Note 56
Note 57
Notes 6, 58
Note 59
Note 60
the contract. The mechanisms suggested by the IG report include Fixed* Price-Incentive-Fee (FPIF1
and Cost-Plus-Incentive-Fee (CPIF) contracts. Both of these types of mechanisms require far more
knowledge of the budget requirements of the tasks than EPA is capable of generating under the SITE
Program. No qualified contractor would perform the rapport required by this program on a fixed
price basis; these are too many variables that are out of the contractor's control and the risk would
be too high. We disagree with the IG's statement that under this type of mechanism the contractor
and the government bear a similar risk (page 36).'
The CPIF mechanism shares the burden of risk between the government and the contractor
more evenly, but it requires that the government be able to establish objective criteria to evaluate
contractor performance. It differs from the Cost-Plus-Award-Fee (CPAF) type of mechanism used
in the SITE Program in (he degree to which the criteria can be defined in advance. Since each
demonstration is different, each work assignment would require negotiation similar to that involved
in initiating the contract. The administrative and contract management time and effort required
for a CPIF mechanism would be prohibitive. We believe that the CPAF contract, which is suitable
when, among other things, *... the work to be performed is inch that it is neither feasible nor
effective to devise predetermined objective incentive targets applicable to cost, technical
performance, or schedule* as noted by the 1C report on page 36 is most appropriate. The award fee
creates an incentive for the contractor and provides an added control for the government. In fact,
SITE management reviewed contracting vehicles with the Agency's contract management staff. X2u
CPAF contract was judged the most appropriate.
In negotiating these CPAF contracts, EPA has tried to incorporate as many specific performance
criteria as possible. The contractors are evaluated on seven criteria of equal importance in the
evaluation of performance. These include: quality of outputs, timeliness, cost effectiveness.
responsiveness to technical direction, ingenuity and' resourcefulness, quality assurance, and
communication. The individuals reviewing contractor performance are senior managers and technical
experts who are substantially involved with the SITE Program and understand all of its requirements.
EPA has found that there is another powerful incentive for contractor performance that has
not been noted by the IG in its report if a contractor performs poorly, its reputation is damaged
and it risks losing this and future contracts. The SITE support contract is time limited and
incumbent contractors must recompete for this work every few yean. Only the most short-sighted
contractor would ignore the effects of poor performance on future work, particularly in a field as
competitive as this and, in fact, the initial contractor was replaced because of less than fuly
satisfactory performance.
The IG report criticize* the ability of SITE Project Managers to control contractor costs.
However, Project Managers an gaining experience and improving their ability in project cost
estimation. We have found that an enormous number of variables affect cost, and that each
demonstration ia different Requirements of the technology, capabilities of the developer to support
site preparation and the demonstration itself, field conditions at the site, sampling requirements, and
other factors vary widely from one demonstration to another. Because many of these technologies
are being demonstrated for the first time at full-scale, even the developers are not always sure of
what is going to be required and how the demonstration will proceed. Another factor that hampers
cost estimates, and leads to cost increases, is the experience of unanticipated problems in the field.
Weather conditions, equipment breakdowns, and other events can drive the contractors' costs up
through no fault of their own. Table 5 (page 37) of the IG report presents data on increases to
original cost estimates. We have included a corrected version of Table 5 of the IG report.
3-3
101
-------
APPENDIX 1
Page 35 of 50
See APPENDIX 2 INCREASES TO ORIGINAL ESTIMATES FOR SITE COSTS
Note 60
Technology Origin*! Coit Rtrii*d H of
Dmnloutt Estimate Incrtan Coit Increase •
TTUSA SI 19,795 $18.706 $138,501 15.62%
* The Demonstration Plan had to be revised due to California regulatory/permitting requirements.
• California regulatory authorities delayed approval of the treatability study for one year.
Retech $115.196 $141.341 $256.537 122.70%
• The increase was not due to the treatability study alone, serious safety problems with the
centrifugal reactor necessitated a Hazards and Operability Analysis.
• Combined waste feed materials at the selected site required additional sampling and analysis.
• The complexity of waste contaminants and reactor operation increased the difficulty in writing
the Demonstration Plan.
Ultrox $99,608 S 37.138 $136,746 37.28%
• Ozone concentrations in the water required special methods evaluations to be planned, and the
air sampling and analysis became more extensive.
• In order to get the unit into the field quickly some Phase II costs (a second work assignment
for the demonstration) such as community relations and site preparation were billed to the
Phase I work assignment (preparation of the Demonstration Plan).
• It should be noted that the Phase H work assignment, not part of the IG report, was estimated
to be $620,000 for a total demonstration cost of about $755,000. Through FY 1989. the total
cost was $85.000 tale* the original EPA estimate.
BloTral $109,693 $62,442 $172,135 56.92%
• Only the ground demonstration was planned in this work assignment.
• Changes in the specific location within the Superfund site for the demonstration evaluation, and
exploratory well drilling for the groundwater plume caused the cost increase.
Silicate $121,183 $155.009 $276,192 127.91%
• Changes in the demonstration site location from Tacoma Tar Pits to Kaiser Steel resulted in
the requirement for a new Demonstration Plan. Six weeks before the demonstration was to
start, the Tacoma Tar Pits owner cancelled site access.
• Both the Tacoma and Kaiser Steel sites required treatability studies to be performed.
3-4
102
-------
APPENDIX 1
Page 36 of 50
See APPENDIX 2 TABLE s
Note 60 Technology Original Co«i Revised % of
Peveloner E»tlig«l« Increase Coit lacrenj
Terr* Vie SI,300,000 J244.IOO $1.544,100 18.78%
• $550,000 was not a cost estimate but the funding given to the SITE contractor to start and
continue working on the project.
• A preliminary cost estimate for $858,000 was prepared by the SITE contractor in October 198"
before Terra Vac requested a minimum of eight weeks of field operation time.
• In November 1987. a cost estimate for SI.300,000 was prepared by the SITE contractor based
on an eight-week demonstration period. The developer requested an eight-week demonstration
period due to the extensive sampling and analysis required by EPA to verify technology
operation.
• Extending the demonstration period caused additional problems, since the extraction wells froze
unexpectedly and extensive insulation had to be added. This caused the system to be stopped,
repairs made and insulation added, and then restarted.
Chemflx S 220,000 164,127 $ 284.827 29.47%
• Preparation of the Demonstration Plan was expanded due to the need for consistency among
SITE solidification/stabilization technologies in the conduct of the demonstration.
Chtmflx S 400,000 $458,662 S 858,662 114.67%
• Costs for conducting the demonstration were increased due to the lack of available information
in estimating demonstration costs. In early 1989, three SITE solidification/stabilization
demonstrations had been completed, but only one of these used a cement-based fixation process
(1WT) and that was an tit-situ not a surface-based application.
Ogdta S 425.000 $195.000 S 620.000 45.88%
• Meeting local permitting requirements and satisfying community relations concerns was more
expensive than originally estimated.
• Extra waste and discharge sampling and analysis was required to complete a risk assessment
requirement; high cost samples for dioxins and hexavalent chromium were involved.
No laeiMie ' '
IWT
TOTALS SJ.910.47S $1,377,225 S4,2«7.700 47.32H
3-5
103
-------
APPENDIX 1
Page 37 of 50
See APPENDIX
Note 18
Note 61
The realities and practicalities of the contracting process also contribute to the propensity for
final project costs to be higher than cost estimates. EPA's SITE Program managers are sufficiently
experienced to understand that approval of a work assignment will more likely $et a floor on cost
rather than a ceiling. Therefore. SITE Project Managers and SITE Program management are inclined
to negotiate work assignment labor estimates that are lower than that which will ultimately be
2 required to complete the effort. It is probable that, in most cases, work assignment completion will
require more funding than originally allocated but less funding than if the initial work assignment
were funded at a level sufficiently high to enable completion of the work. We note that while the
1C report is critical of the cost increases experienced in many of the projects, it expresses no opinion
on whether the ultimate project costs are excessive.
Award Ftei and Increased Costs aod Tine Delays
Note 62
The IG report suggests that EPA has not nude as effective use of the award fee as it could.
in the SITE Program aad in other programs. The report goes on to state that *... the overall award
fee of three percent is relatively small, and the contractors average receiving about 70 percent (or
a total fee of approximately two percent) of it. The extra one percent fee, compared to significant
approved cost increases, may provide little incentive to control costs' (page 41). In discussions
between SITE officials and the Contracts Management Division (CMD), CMD staff strongly
encouraged a tonah scoring system for award fees. However, SITE officials understand that a ?0%
award fee may be low for contracts of this type.
It should be noted that the additional one percent fee on contracts of this size, approximately
SI6 million, is about $160,000 which is a significant amount of profit for even a large company.
SITE officials believe that the award fee serves as a substantial incentive to motivate contractors to
improve performance. In addition, most corporate managers' performance evaluations include
consideration of sales and profit
The 1C report mentions on page 41 that the QIC is conducting a major review of the
management of the award fee process in contract work for the Superfund program. SITE officials
wilt be very interested in the findings of this review, to receive the maximum benefit of CPAF
contracts in motivating contractor performance.
RESPONSE TO RECOMMENDATIONS
/. Rtqian that adequatt justification bt providtd for individual funding increases.
The justification documentation thai has accompanied funding increases has met the
requirements of EPA's Contract Management Division. SITE management believes that the most
effective contractor coat control is achieved by strong contract management and close, frequent
communication between the Project Manager and the contractor. On each demonstration, the Project
Manager is continually in close communication with the contractor regarding the status of the project
and the accomplishments of the work assignments. The Project Manager and the contractor discuss
problems, delays, and potential cost increases long before they become critical to the program. By
the time a request for additional funds is submitted by the contractor, its need has been discussed
with the Project Manager and strong justification is required from the contractor. SITE management
sees no need for additional controls and is unaware of any other options that might be more
effective.
104
-------
APPENDIX 1
Page 38 of 50
^. // contractor performance in key areas, such as report writing, cannot be improved, retain thote
functions inhouse on future projects, [f necessary, assign additional staff to perform these
functions.
In some instances, SITE Project Managers have rewritten various lections of SITE reports. For
example, the American Combustion report was rewritten by SITE staff to adequately address several
key technical issues concerning the operation of the enriched oxygen burner, including correction
of the results. Since preparation of the early reports, contractor performance on report writing has
been much improved. SITE management believes that the recently implemented report format
changes and the enhanced experience of contractors and Project Managers will result in a streamlined
process. EPA is committed to producing high quality reports and will dedicate whatever mix of
contractor and inhouse resources is necessary to achieve chat end. We should note, however, that
Project Managers would not be able to take on writing responsibilities without: (I) a diminution of
time spent in monitoring contractor work and managing contractor work assignments, or (2) the
addition of staff committed to the SITE Program. The first alternative is unacceptable, since it .
would only lead to a compounding of problems in the future. The second is unlikely, given existing
staffing levels. The EPA SITE staff is composed of 16 professionals and support secretaries. While
the number of professional staff remains constant, the number of projects under the SITE
Demonstration and Emerging Technologies Programs continues to expand. The success of both of
these components of the SITE Program relies on the ability of the Project Managers and SITE staff
to manage and perform these projects.
i Reinforce to contractors and project managers the importance of the award fee process and why
the award fees are used.
SITE management agrees with this recommendation and will reinforce and emphasize the award
fee process to our contractors and SITE Project Managers.
We believe thai this is currently an effective incentive for the two existing support contractors.
Even though the amount of the award fee averages three percent of the contract award (this is
negotiated by our Contracting Officer during the award process), our experience is that both
contractors are highly sensitive to the scores and resulting award. Both contractors are graded within
their organization on the award fee basis. In fact, when EPA issued an overall award fee of 66% to
one contractor in June 1989, the Project Manager for that contract informed SITE officials that he
was "in trouble' with his management due to the low score. In fact, one of the contractors scores
much higher on their other EPA CPAF contracts, usually an avenge of 85% to 93%. It is important
to note that the Contract! Management Division in Cincinnati processes the award fee in a timely
manner, and communicates the scores to contractors in a timely and effective manner to ensure that
the Agency receives the full benefit of the impact of award fees on motivating performance.
SITE officials will continue to demand quality performance from the contractors, and in an
effort to support this goat will investigate the feasibility of negotiating smaller base awards and larger
award fee pools for the contracts to be awarded in November 1990. By negotiating smaller base fees
and larger award fee pools, the Agency increases its control over a larger percentage of the total fee
received by the contractors, increases the leverage of award fee as a motivator, and increases the
contractors' risk in-terms of profit (poor performance could result in a much lower fee).
J-7
105
-------
APPENDIX i
Page 39 of 50
See APPENDIX 2 4- Consider for the next contract period a contract type thaivould more fairly balance risk between
Note 63 EPA and the contractors.
In a meeting with SITE management and Contracts Management Division (CMD) staff, CMD
staff stated that a fixed price contract ii not feasible due to the great difficulty in defining the scope
of a technology demonstration prior to actually performing the demonstration planning function.
Frequently, during the formation of the demonstration plan it becomes apparent that treatabtlity
studies for a proposed demonstration lite are necessary. This cannot be determined until many
factors about the technology and the proposed site to conduct the demonstration are known.
Typically, several sites are considered and' evaluated before a match is made for a developer's
technology. With all these variables, a fixed price contract would not be feasible, and few
contractors would accept the tremendous risk associated with such a contract.
The Contracts Management Division staff stated that a CPIF contract is feasible for some well
defined research and development efforts. However, with the variable* mentioned above, it would
be extremely difficult and time-consuming to establish specific factors to be objectively evaluated
and rewarded for each demonstration project. Currently, EPA has issued over 30 work assignments
for each of the two SITE support contractors. Because the types of technologies and support
requirements are not known in advance of issuing the contracts, use of a CPIF contract would force
the Agency to approach contracting on a project by project basis and negotiate each demonstration
as if it were a new contract effort. In attempting to use a CPIF contract. EPA would spend
substantially more time and effort in trying to administer this type of contract and this would
probably cause unmanageable delays in conducting the SITE demonstrations.
In view of the above, SITE management, iupoort«d bv the Contracts Management Division
believes that the Cost-Plus-Award-Fee level of effort contract is the most appropriate type of
contract for supporting the requirements of the SITE Program.
3-g
106
-------
APPENDIX 1
Page 40 of 50
See APPENDIX 2
Note 20
Note 21
Note 64
4. THE SITE DEMONSTRATION PROGRAM DISCOURAGED DISTRIBUTION
OF FEDERAL FUNDS TO DEVELOPERS
The 1C report staid that the *... Demonstration Program discouraged distribution of Federal
assistance to technology developers' (page 43).
SITE management agrees that funding restrictions and procedures may have discouraged some
otherwise qualified developer* from seeking entry into the program. These requirements however
are set by legislated requirements in the Act. not by EPA. Further, EPA has had no trouble
attracting fully qualified developers into the program in sufficient numbers to fully utilize the
available funds. Thus, if developers' costs were being funded, fewer technologies would have been
demonstrated. The level of information required of developers to obtain financial need was also
questioned. To have required less documentation of financial need would have failed to meet the
requirements of the law, which prohibited providing federal assistance unless the applicant
demonstrated that they could not obtain private financing. The information required of the applicant
is the same information he would have supplied in seeking private financing.
This section addresses a number of the issues raised in the IG report including:
• Uncertainty regarding funding for developers
• Approach to offering federal assistance
• Advene impact of developer funding
• Impact of funding concerns oa developer participation.
• The Agency has developed its model baaed on legislative requirements in accordance with
SARA Section 31 l(bXSXJ)- This section states:
The Administrator shall not provide any Federal assistance for any part of a full-scale
field demonstration project under (Ail subsection 10 any applicant unless such applicant cart
demonstrate that it cannot obtain appropriate private financing on reasonable terms and
conditions sufficient to carry out such demonstration projects without suck Federal
assistance,
SITE officials believe the wording of this section of SARA is clear, requiring EPA to confirm
that the developer cannot obtain appropriate private financing on reasonable terns and conditions
before providing the necessary funds for full-scale demonstrations.
The IG report states that "... although EPA issued SITE solicitation 001 before SARA was
enacted (so no funding was available), we found'that solicitation 002 (issued after SARA) did not
inform technology developers that financial assistance was available* (page 43). The IG report fails
to note that the SiTE-002 solicitation was issued on January 15. I9S? just three months after SARA
was passed by Congress oa October 17, 19M. The SITE-002 solicitation had been drafted w«H in
advance of SARA'S enactment and the short time-frame gave SITE officials little opportunity to
develop funding assistance procedures and include them in the solicitation.
4-1
107
-------
APPENDIX 1
Paae 41 of 50
See APPENDIX 2 As correctly sated in the 1C report, the SITE -003 and 004 solicitations included financial
Note 65 assistance information and the information required from financial assistance applicants. Both
solicitations clearly indicate that selection for funding assistance follows technical acceptance into
the SITE Program, and that applying for financial assistance will not affect an applicant's evaluation
under the technical selection criteria.
While the 1C report states that '... lack of funding may have discouraged developers of other
promising technologies from applying for admission to the program* (page 43), the SITE Program
has had no difficulty in attracting adequate numbers of qualified developers to the program. If EPA
were also funding the developers' costs, in whole or in part, then it would be certain that sufficient
Note 66 fundl would not be avaiiible to conduct all of the demonstrations accepted into the system. The
result would certainly be fewer demonstrations. To date there have been no requests for funding or
letters of complaint from potential developers claiming that funding was not available.
Uncertainty Hoarding Fmidlag for
The IG report states that *... SITE officials stated that after the passage of SARA, they were
not certain how to implement the requirements for funding technology developers, and requested
clarification from the Office of General Counsel (OGC). According to the SITE officials, the delay
in receiving an answer from OGC prevented any offering of Federal assistance in the first two
solicitations. We discussed this with OGC and were informed that OGC has no record of the SITE
Program's request for advice on funding assistance* (page 44).
SITE staff met with OGC staff on several occasions in the spring and early summer.pf 1987
to discuss developing procedures for the implementation of CERCLA Section 3ll_(bX5KJ).
£7 Specifically on June 25, 1987 SITE and OGC staff met to resolve some options regarding
o / implementation of this CERCLA section. On June 26, 1987. a memorandum sent to OGC from
OEETD identified sonu of the questions and concerns of the SITE officials and requested
clarification of these issues because the answers could impact the design of the procedures for
implementing funding assistance for SITE demonstrations. OGC did not prepare a written opinion
regarding the June 26 memorandum but provided oral approval in subsequent meetings with SITE
staff. Prior to release of the 'model" referred to in the ~G report, SITE officials submitted
descriptive information on this model to both the OGC and CMS for approval. Both approved the
Note 68 model orally in meetings with SITE staff. This development process for the financial support rules
had nothing to do with the unavailability of financial support for the first two solicitations.
SITE officials also met with Grants Administration Division (GAD) officials to discuss the
processing of competitive cooperative agrMmeno for developers accepted into the SITE Program.
During discussions with GAD personnel, the following time requirements were established for
processing cooperative agreements: (I) approximately two weeks for GAD to process a new
application before sending it to ORD for their review process, (2) the review process conducted by
ORD depends on the complexity of the project and varies in length, normally 10 to 14 weeks, and
(3) three to six weeks (after ORD review) for final processing by GAD before award. This totals
from four to six months, depending on the complexity of the application. This time is for the
competitive cooperative agreement process only (from the time the application is received by GAD
until final award by' GAD) and does not include the time required by the applicant and ORD to
qualify the applicant for federal assistance. It is estimated that this could require from two to four
months. Thus, the estimate of six to nine months cited on nag* 44 of the IG report is realistic, (n
addition, for the 1 5 technologies accepted into the SITE Emerging Technologies Program the average
time for processing the competitive cooperative agreements has beta eight months.
4-2
108
-------
APPENDIX 1
Paoe 42 of 50
The 1C report mentions thit one technology developer responding to the SITE -002 solicitation,
International Technology Corporation, expressed «n interest in receiving financial assistance. The
report sates that "... because the developer could not fund the company's portion of the
demonstration or obtain a funding commitment from a second party, SITE officials placed them in
a conditionally accepted status* (page 43). In reality. EPA sent letters to the 12 conditionally
APPENDIX 2 acceptable developers in the SITE-002 cycle on June 12. 19*7, and responses from the developers
Note 69 were due to the Agency by August IS. International Technology Corporation responded to the letter
by withdrawing their application due to problems in timing with the site they had chosen for the
demonstration. EPA informed International Technology Corporation that if the company was
unsuccessful in obtaining private funding, they could apply for federal funds, but EPA could not
guarantee any funding. International Technology Corporation never applied to the Agency for
funding assistance.
Approach to Offering Federal AiiiatMM
' The IG report states that "... SARA Section 3ll(bK3) indicates that the Administrator shall, to
the maximum extent possible, enter into appropriate cost sharing arrangements under this subsection.
From this, we believe that the legislation points out that funding was available and was encouraged
by 'Congress' (page 45). This subsection of SARA authorizes EPA to enter into contracts and
cooperative agreements with developers and authorize* EPA to the maximum extent possible to enter
into appropriate cost sharing arrangements. EPA's role in demonstration projects was defined in
SARA Section 31 l(bXSXG). This subsection states:
The Administrator shall enter into a written afrtement with each applicant granting the
Administrator the responsibility and authority for testing procedures, quality control,
monitoring, and other measurements necessary to determine and evaluate the results of the
- demonstration project. The Administrator may pay the costs of testing, monitoring, quality
control, and other measurement! required by the Administrator to determine and evaluate
the results of the demonstration project.
SITE officials believe that Congress intended for the Agency to share the cost of demonstrating
SITE technologies by funding the talks described above. SARA Section 3ll(bXJKJ) indicates that
an applicant most demonstrate to EPA that they have made a good-faith effort to obtain funding
from private sources and failed before EPA can consider funding any pan of the demonstration
project. Thu, EPA sboald be Ike flmaaclAi mice of "last resort.'
The 1C report iugg««s that EPA requires extensive financial information in the applications
for funding assistance. The report states that "... SITE officials finally selected a model for funding
based oa the requirements of private venture capital firms, which emphasize a very conservative
approach to lending money. This model required an extraordinary amount of information to be
-------
APPENDIX 1
Page 43 of 50
See APPENDIX 2
Note 20
Note 71
EPA also developed a seven-page document which explained in detail (he requirements for funding
and information on completing the funding application for use by applicants.
The information required by applicants for federal assistance under the SITE Demonstration
Program was developed from information required by venture capital companies of their applicants.
Therefore, all of the information required by EPA for federal assistance would normally be required
of an applicant when seeking financing from private sources. SITE officials did not believe thai
providing this information overburdened (he developers because they would have already provided
such information to venture capital firms when seeking private funding. Thus, the information
required by applicants seeking federal funding assistance would not have to be generated specifically
for the SITE application. There is. therefore, no burden on the developer posed by making the
information available to EPA.
SITE officials believe that this financial information within the model is necessary for three
reasons: (1) the Agency must confirm that the developer has tried and failed to obtain appropriate
private financing on reasonable terms and conditions. (2) the Agency should ensure that the firm is
unable to fund the demonstration with its own resources, and (3) the Agency has an obligation to
ensure that federal funds are invested wisely and prudently in projects that support the goals of the
SITE Program.
The 1C report indicates that the funding approach adopted by EPA with regard to the SITE
Program may have hindered some developers from participating in the program. SITE management
can neither confirm or deay this finding. It is probable that some small developers have chosen not
to pursue a SITE demonstration because of costs and their inability to qualify for the financial
assistance. Within current legislated limits and within available funding levels, the problem cannot
be solved. -:<
The report further states that "... one developer (International Hydronics) from the third
solicitation was denied participation in the SITE Program for reasons, which included a request for
funding* (page 47). EPA's records indicate that International Hydronics was classified as
conditionally acceptable following submission of their proposal in response to the SITE-00 3
solicitation. A meeting was held on June 17, 1988, at which time the status of their application was
discussed. On June 30.1981. SITE officials notified International Hydronics that their proposal was
unacceptable because it appeared they did not have the financial backing to fund the demonstration;
the equipment had not been designed, assembled, or tested as a working unit on any scale; and there
did not appear to be a strong push to commercialize the technology. It must be understood that
International Hydronics did not make a request for federal assistance. Such a request is made only
after a technology is accepted into the program. The rejection of the application was based primarily
on-the fact that their technology was not ready for full-scale field demonstration. In their
application International Hydronics clearly indicated that they lacked organization and resources to
successfully participate in the Demonstration Program. Lack of technical merit and an inability to
even provide the minimal resources for a project were the causes for rejecting the application of
International Hydronics.
International Hydronics also submitted a proposal in response to the SITE-004 solicitation which
the Agency judged to be technically unacceptable. This determination was not affected by the
developer's request fpr financial assistance.
4-4
110
-------
APPENDIX 1
Page 44 of 50
As of October 1, 1989, ao developer had applied for federal financial assistance to conducr a
SITE Program demonstration. Although the Agency has not provided funds directly to the
developers, EPA has agreed to fund as many SITE related activities as possible consistent with the
legislative constraint* (as described in Section 31 KbXSXA) and (bXJXG) of SARA). For all SITE
demonstrations, the Agency bean the cost of publishing the solicitation and evaluating the
applications; developing demonstration plans; supervising demonstration projects and providing
quality assurance for data obtained: sampling and analysing the samples; evaluating the results: and
publishing and disseminating the results.
In addition to these demonstration activities, EPA has agreed to fund for various demonstrations
other activities or services such as treatability studies, site preparation, utility connections and service
charges, and permitting applications and processing. While each of these costs may appear to be
small, their cumulative effect on the cost of a demonstration may indeed by significant. For 1S of
the technologies in the program. EPA has funded over S2.000.000 for treatability studies and site
preparation. For a project-specific example, in the demonstration of CF Systems' solvent extraction
technology. EPA incurred the costs for (1) installation of a cement pad on which the unit could be
placed during operation. (2} installation of additional fencing to exclude public access to the unit.
(3) connection of electricity and phone service and monthly service charges, (4) pre-sarapling of the
site for PCB contamination, and (i) laboratory treatabiliry studies of the harbor sediment for the
demonstration. These additional expenses incurred by the Agency for this one demonstration totalled
approximately $60,000. By paying for these five items that were necessary to conduct the
demonstration, EPA saved the developer approximately $60,000.
Adv«r«« luinict of De»«iantf Fundla*
The IG report identifies three factors that constrained the funding available for developers:
(1) SITE officials believed that the success of the SITE.Program would be adversely affected if the
Demonstration Program provided financial assistance to developers out of the existing appropriated
amounts, (2) the program's contractors incurred costs representing a very significant portion of the
available funds, and (3) the cost incurred for the developers that had exited the program.
EPA acknowledges that the first factor is true. The Agency was required by SARA to initiate
or cause to be initiated at least 10 field demonstration projects in each fiscal year of the program.
At the outset of the SITE Program. SITE officials did not have reliable data for estimating
demonstration costs. Therefore, it was very difficult to determine the funding required by the
Agency to fund their portion of the 10 demonstrations. Only after several demonstrations had been
completed did SITE officials have adequate information to project annual expenditures for their
portion of the demonstrations. The average cost for EPA's portion of a SITE demonstration has been
about SI,136.000. thus to fund 10 demonstrations each year would require SIS,360.000 which
exceeds the SITE Program budget for FY 1987 and is over 85% of this budget for FY 1988 and FY
1989. " * \
The second factor affecting the funding available for developers was that a significant portion
of the funding was received by SITE contractors. The 1C report states that"... contractors absorbed
approximately 78 percent of the Demonstration Program's budgeted funds in fiscal 1987.74 percent
in fiscal 1988. and 86 percent in fiscal 1989 and... these costs wen greater than originally planned'
(page 48). It is true that significant percentages of the Demonstration Program's bud^t were
allocated to SITE contractors. This is because they perform most of the costly work in - "ting
the demonstrations. It should be pointed out, however, that a substantial portion of COP •*$
was devoted to such important technology transfer activities as preparation of
4-S
111
-------
APPENDIX 1
Page 45 or 50
See APPENDIX 2
Note 36
Note 72
Note 73
Note 74
Bulletins, production of videotapes of che demonstrations, and preparation of Technology Profiles,
brochures, and annual Reports to Congress.
The third issue was the funding invested in technologies that had withdrawn or been exited.
from the SITE Program. To date, 6 developers have withdrawn and 4 developers have been exited
from the program. The 10 developers who have left the program cost EPA about SI.7 million, which
is less than 5% (versus the 8% reported in the IG report) of the Demonstration Program's budget
from fiscal years 1987 through 1989. K*.
*.*•
The 1C report highlights the fact that developers in the Emerging Technologies Program •
received funding during the first year it was operational. The report points out that *... SITE
officials responsible for administering funds for the Emerging Technologies Program apparently had
no problems disbursing funds to the private sector. In contrast, the ORO flagship program — the
Demonstration Program -- struggled for approximately 16 months to deal with SARA legislation
which made federal assistance available to qualified technology developers" (page 49). There is a
definite, if subtle, distinction between providing funds to developers in the Emerging Technologies
Program and the Demonstration Program. SARA authorizes and directs EPA to carry out a program
of research, evaluation, testing, development, and demonstration of alternative or innovative
treatment technologies. Section 31l(bX3) authorizes EPA to carry out these programs through
contracts, grants, and cooperative agreements. SITE Program officials discussed this Section with
contracts management and grants administration within EPA to determine possible mechanisms for
funding. A contractual award could not be provided since this would be considered sole source
contracting for individual developers. However, using a competitive cooperative agreement
developers would be eligible for assistance. Section 3ll(bXSXJ) prohibits funding of full-scale
demonstrations unless an applicant can demonstrate that it cannot obtain appropriate private
financing on reasonable terms and conditions. The Emerging Technologies Program component of
the SITE Program consists of research-oriented laboratory pilot- and bench-scale evaluations, not
fuil-icale field demonstrations, and therefore is not subject to the restrictions for federal assistance
as those in the Demonstration Program. Thus, it is possible under the Act to provide funding
assistance to experimental activities under the Emerging Technologies Program, but not possible to
do so under the Demonstration Program.
Imnict of
CoBeenu ost Devetaa
ParticiBatloa
In the process of conducting tail review, the IG interviewed five developers who had completed
SITE demonstrations. Four of the five developers said funding was an impediment to the program
and that they would not participate again given the opportunity. The five developers interviewed
by the OIG represent some of the earliest participants in the SITE Program. Unfortunately, these
developers entered the program while it was in its infancy and encountered many start-up problems.
Policies and procedures had not been established, protocols had not been developed and SITE Project
Managers and contractors had little experience in conducting and evaluating full-scale
demonstration! of innovative technologies- Therefore, then were delays that might have resulted
in additional costs to developer*.
It is also fair to say that the results of the demonstrations and EPA's interpretation of them, do
not always fully comport to advertising claims made by developers. This may tend to color some
developers' views as to how useful the program was to them from a business perspective. It is just
such findings, however, that make the program very useful to EPA's field personnel.
4-6
112
-------
APPENDIX 1
Page 46 of 50
See APPENDIX
Note 75
Note 76
The 1C report refen to • letter dated May 22, 1989 written by a Congressman on behalf of a
developer who bad submitted an application in response to the SITE-004 solicitation. The 1C report
states that'... as of October i, 1919, the SITE Program's response was still in process* (page 49).
EPA'i records indicate that a tetter was sent to the developer on May 24,1919 advising him chat the
RREL staff was reviewing his comments and those of the technical reviewers. A complete
revaluation was made of his proposal and the information provided in his letter. EPA has since
notified the developer that his proposal was found to be technically unacceptable based upon the
information provided to the Agency. Specifically, the technical data submitted by the developer was
found to be unresponsive to the evaluation.criteria set forth ia the RFP and the proposal did not
include the basic pertinent details regarding the technology. Therefore, EPA could not accept this
technology into the SITE Program.
Table 7 in the IG report, Responses to Demonstration Program RFPs (page 50), erroneously
reports the number of proposals received for solicitations 001 through 004. Table 2 is a corrected
version reporting the correct numbers of proposals received.
Table 2. Responses to Draoutratioa Program RFPs
Increase (Decrease)
Solicitation
Number of
Minuter
001
002
003
004
20
29
31
24
N/A
9
2
(7)
N/A
31%
6%
(29%)
Note 20
RESPONSE TO RECOMMENDATIONS
/. Seek further clarification from Congreu on tlu iiutnl of SARA Section lllft»(5)(J) concerning
funding for technology developers.
The legislative requirements for federal assistance seem quite clear. The only question is what
should be required of an applicant to demonstrate that appropriate private financing cannot be
obtained oa reasonable terms and conditions. ORO hai developed to appropriate model for making
decisions for federal assistance for the SITE Demonstration Program. It requests information that
diligent developers would have to have anywmy in order to seek private financing. SITE management
can see no other rational interpretation of Section 31 HbXJXJ)- However, ia keeping with the spirit
of the recommendation, it may be appropriate to consider changes during CERCLA ^authorization
which is scheduled for 1991.
4-7
113
-------
APPENDIX 1
Page 47 of 50
2. Reduce the amount of financial information that is required from prospective developers.
See APPENDIX 2 AJ discussed earlier, the information required by EPA for federal funding is (he same as that
Notes 20, 70 required by venture capital companies and therefore, if aa applicant has made a good-faith effort
to secure private funds, this information should already be available to furnish to EPA. We feel that
the information required is not burdensome to applicants and is necessary for EPA in fulfilling its
responsibilities to assure fairness and proper expenditure of government funds.
J. Discuss and formalize with GAD officials an official Agency position on ike amount of time
anticipated for awarding funding for developers, and convey this information to developers
applying under the new solicitations.
SITE management agrees with this recommendation and will develop a statement regarding the
time anticipated for award of competitive cooperative agreements in the SITE Demonstration
Program which is agreeable to GAD and ORD. This statement will be included in future SITE
solicitations.
«. Keouest additional funds be allocated to the StTE Program to provide for developer funding.
Consider reprogramming budgeted funds, if necessary.
SARA Section lltfnXl) limits the amount of funds for carrying out the applied research.
development, and demonstration program for alternative or innovative technologies to S20.000.000
for each fiscal year from 1987 through 1991. Appropriated funds for each of these fiscal yeanjiaye
been very near this authorisation limit. Thus, there is not much room within the authorization for
Congress to appropriate more funds.
SITE officials believe that this budgetary authority is adequate for the Agency to meet its.
legislative requirements for innovative technology research, development, and demonstration. To
date, no developer has submitted aa application for federal financial assistance and the number of
proposals received in response to SITE solicitations is more than adequate to meet legislative
Note 77 requirements. CERCLA reauthoriation is scheduled for FY 1991 at which time this issue can be
addressed.
4-g
114
-------
APPENDIX 1
Page 48 of 50
APPENDIX A
See APPENDIX 2 INCONSISTENCIES IN ic SURVEY
Note 78
The following specific concerns were identified related to the presentation of data by the 1C
in the draft audit report
• 'Based on our telephone survey (see Exhibit D), RPMs. OSCs, Superfund cleanup
contractor], and technology developers believe a lack of information exists on SITE
technologies and corresponding data* (page 28).
Exhibit D does not support this conclusion, since no item directly relates to this
statement. If the statement is based on the weakness described as 'dissemination of
demonstration results* the number of respondents is too smalt to be reliable -- one OSC.
three RPMs. four contractors, and two developers. It should also be noted that the vast
majority of respondents did not cite this as a weakness. In addition, it should be noted
that the number of respondents who cited strengths in the SITE Program was greater
. than the number of respondents who cited weaknesses (see page 64).
• •...approximately 30% of our statistically valid sample of OSCs and RPMs had never heard
of the SUE Program or were not familiar enough with the program to answer our survey
questions' (pag* 28).
Since the report does not include a copy of the survey questionnaire, we cannot be
certain the question used to screen potential respondents is valid. For example, a
respondent may be aware of the SITE Program and some of in technologies, but may
not feel capable of commenting upon the strengths and weaknesses of the program.
These types of questions can only be addressed by individuals who have been involved
in some way with the program and are therefore familiar with its operation. Further.
it does not appear that the samples of OSCs and RPMs were drawn in a manner that
would permit aggregation of responses across samples to yield the 'approximately 30%"
in the statement.
• '...because of a lack of published information and reliable results, the RPMs, OSCs, and
contractors were not willing to rely on SITE innovative technologies to clean up sites*
(page 28).
How many respondents made this statement? This conclusion is not supported by
Exhibit D. Tat exhibit doe* not address the issue of whether or not the above named
groups an or are not willing to rely on SITE technologies.
• "While most OSCs and RPMs familiar wrth the program thought it was a good idea, to date
they had not received sufficient information about the results of specific demonstrations
to encourage them to use innovative technologies* (page 29).
While the first part of this statement is supported by Exhibit D, tht ueond half it
contradict** by tJW dm* present***. Relatively large numbers of respondents noted
strengths of the program that could be interpreted as *a good idea,* In addition, 93%
of the OSCs and 85% of the RPMs did not view dissemination of demonstration results
as a problem — these respondents appear to be satisfied with the information they had
received. The data presented in the exhibit is insufficient to support or refute the
statement concerning encouragement to use innovative technologies.
115
-------
r
APPENDIX 1
Page 49 of 50
In reporting comments from RPMs and OSCs, the report cites the following (page 29):
• "There is not enough exposure of the program...'
The exhibit does not support this statement, even as an isolated comment, since it is
not mentioned as a weakness. The only data on comments related to "exposure* are
those who cited it as a strength of the program (88% of OSCs. 81% of RPMs. ?!% of
contractors, and 80% of developers).
• "Many RPMs do not even know what the acronym SITE means.'
The number of RPMs who fit this description is not provided. Further, it is important
the RPMs know about innovative treatment technologies; it is less important that they
know the words that comprise the SITE acronym.
• "SITE is a fairly new program with risky and unproven technologies.*
The purpose of the program is to demonstrate these unproven technologies and to
provide information that reduces the risk involved in selecting them for cleanup actions.
• The program is limited in the scope of technologies and their use on particular sites, in
fact, some may be usable at only one specific site.*
While Exhibit O does not present data to support this conclusion, it is true that
technologies hive specific applications. As mentioned in Section 1. technologies of
limited scope can be very effective when used in combination with other technologies.
• Technologies an not readily available for use.*
Exhibit O notes that 'less than 10% of the respondents in each group* identified as a
weakness 'technologies not readily usable." Based on the numbers of potential
respondents, a ««^«ti»niif of one OSC and one RPM could have made this statement,
not a sufficient number upon which to draw a conclusion.
• The potential risks an high for both the developer and die OSC.*
Exhibit D does not present data to support this statement
In reporting comment! from contractors, the report highlights weaknesses cited by four
contractors concerning dissemination of materials (page 30). Nine respondents, or 69%, apparently
believed that dissemination of information is not a problem in the program. On page n, the report
states that one cleanup contractor the !O interviewed reinforced the idea that RPMs and OSCs do
not have an incentive to participate in the SITE Program. Yet in Exhibit D no Superfund contractor
responded to weaknesses others had mentioned (such a RPMs and OSCs) to the respondent.
In reporting comments from developers, the IG report again bases its conclusion on the
responses of a very few individuals. To sat* that these developers 'frequently* expressed concern
about the length of the reporting process (page 30) is an exaggeration; in fact, miiv two developers
oat of fiv« «PPMT to have given that response according to Exhibit D. The opinion of one individual
is also cited, stating that he •hqiiavad (emphasis added) that no one in the field (RPMs and OSCs)
knows very much about the SITE technologies* (page 30). This statement presents the opinion of one
116
-------
APPENDIX i
Page 50 of 50
individual coocernini the knowledge levels of other individuals aad should not imply thit it is *
statement of fact.
Although the developer* aad contractors interviewed during the telephone survey were
concerned about various weaknesses and problems associated with the SITE Program, st least three
(two contractors and one developer) mentioned specific strengths which were cited on page 30 of (he
IG report. The survey results in Exhibit D (page 64) indicated that 16 OSCs. 21 RPMs. M
contractors, and 5 developers identified strengths of the SITE Program.
117
-------
APPENDIX 2
Page 1 of 15
ADDITIONAL PIG COMMENTS TO MARCH 12. 1990. RESPONSE
TO DRAFT AUDIT REPORT
The following notes present the OIG's response to ORO's comments
which were not addressed in the body of the report.
We already addressed in our report the accomplishments of
the program and recognized the problems associated with
applying innovative technologies in a real world situation.
But it should be noted that we reviewed the more
programmatic and managerial aspects of SITE/ not the
technical issues involved. We believe that the findings and
recommendations contained in our report will help strengthen
the SITE Program and insure better results in the future.
On pages 2 and 10 of our report we had already stated that
SITE is the only comprehensive program to demonstrate and
report on innovative technologies. ^ ,
We understand that budget modifications and schedule changes
can occur. However, we are concerned about the major cost
increases and schedule changes that have occurred with such
frequency. We believe that while this program may be the
flagship program of ORD, better management and consistent
project monitoring will result in an overall better program.
We believe these figures are very misleading because we
found no indications that SITE technologies have
significantly contributed to the selection of these
alternative technologies in RODs. While three SITE
technologies may have been selected for cleanups, we believe
from conversations with SITE Program officials that it may
be questionable if some of these technologies were selected
because of the SITE Program. They may very well have been
selected due to the individual selling efforts of the
technology developers. - In addition, some of these
technologies were already in commercial use at the time of
their demonstration in the SITE Program.
We already noted this in our original draft report on page
22. However, as even the Director of RREL noted in a March
1989 memorandum, the AAR is the most important SITE
document. We continue to believe, along with people we
interviewed during the audit, that the two final reports
produced from the demonstrations are the most important SITE
Program documents. All of these other information products
118
-------
APPENDIX 2
Page 2 of 15
produced by SITE officials refer to these two reports for
the full details needed by regional cleanup decision-makers.
6 We never stated or implied that the contracting mechanism
used was improper or that award fees paid were too high. We
stated that consideration should be given to other contract
types which more fairly balance Government/contractor risk;
that the 3-percent award fees were insignificant compared to
the substantial cost overruns; and that award fee
determinations should be adequately used to communicate
performance results and the need for high quality work to
the contractors.
Regarding the contracting mechanism used (CPAF), an official
in the Contracts Management Division (CMD) in Cincinnati,
Ohio, advised us that a CPIF contract type may be useful for
the SITE Program. However, CMD believed they did not have
j, sufficient resources to manage a CPIF contract.
••*
's 7 We did refer to the Administrator's study in our draft
report and the recommendations the study made to help
improve the overall SITE Program. He agree the
recommendations made by the study are needed to improve the
SITE Program's management.
8 OED has asserted that they "met virtually every requirement
placed on it in the legislation that created SITE
(including)... the selection of at least 10 technologies in
each of four consecutive years." However, this is not what
SARA, the Agency's own directives, and the SITE strategy and
Program Plan state. All of these documents require the
initiating of 10 demonstrations each year starting in fiscal
1987, not the selecting of 10 technologies. As we had
previously pointed out, for fiscals 1987-1989 EPA had
completed only 12 demonstrations of the 13 they had started.
Based on-the legislation and their own directives, EPA
should have initiated at least 30 demonstrations during this
same period. In addition, by SITE'S own acknowledgement,
final reports have been extremely slow during the reviewing
and publishing process, voicing comments as recently as
March 1989 that there is a severe report backlog that needs
to be addressed. There is an additional problem here that
we believe impacts SITE. This is the exit program, in which
11 technologies left or were removed from the program
without ever having done a full-scale demonstration. This
represents over 28% of all the technologies admitted to the
program.
ORD has also asserted that it "conduct(ed) all work within
the budget authorized in the law." However, had the program
initiated the required number of demonstrations it is
119
-------
APPENDIX 2
Page 3 of 15
entirely possible it would have exceeded the budget. As we
have pointed out, over 28 percent of the technologies
admitted to the program during the first three fiscal years
exited without doing a field demonstration, and less than
half the number of demonstrations required by SARA were
actually initiated. These two factors minimized the
expenditures of the program during this time period.
.9 In our opinion, to reach the goal of 90 days for site
selection, the SITE Program is going to have to.become more
assertive on setting and meeting timeframes. Further,
unless the recommendations we made are addressed and
implemented, site matching will continue to be! a lengthy
process. While we agree there might be some factors outside
the control of the Agency, our report addressed findings and
made recommendations that are within the Agency's control.
We continue to believe these are critical, time consuming
factors involved in matching sites.
10 While this is a good start, we believe the analysis should
be used to help select future technologies. When over 28%
of developers never demonstrate their technologies, the
selection process needs attention. In addition, we believe
that the Agency should not permit developers to remain in
the program for 3 years before removing them from the
program.
11 While two Biotrol technologies were demonstrated at a single
Superfund site, there was never any indication (in
documentation or interviews) during our audit that these
demonstrations were intended to be done in combination. We
believe these technologies were done basically independently
of each other without the intended purpose of demonstrating
these technologies in combination. It is only now that the
SITE Program is describing them as a "combination."
-&
12 In our opinion this informal approach has allowed this
r problem with site matching to continue; We believe more
~ formal action is necessary. Our report never stated that
I developers should be offered a site with no contingencies.
In reality what we stated was that a formal policy would set
the stage initially with the developer regarding what is
expected in site selection.
13 In our draft report, we recommended that either SPMS or SCAP
might be an appropriate approach to track site matching. In
ORD'~s comments on page 22 of 50, they acknowledge that SPMS
may be more appropriate than SCAP.
120
-------
APPENDIX 2
Page 4 of 15
14 We did recognize actions that SITE officials had taken on
pages 22 and 23 of our draft report, but we continue to
believe that these actions in many cases have been
unsuccessful. Further, in our analysis, we showed all
reports issued by the completion of our review, not
just the early reports.
15 While we agree that ORO has a system, we question the degree
.of accountability. When updates are done quarterly, project
milestones are changed and revised. As we observed, this
happened numerous times and was still occurring at the time
of our review. The question still remains, however,
regarding the effectiveness of the controls used. Simply
revising the milestones each quarter does not establish
formal accountability for meeting these milestones.
16 ORD's detailed response on page 30 of 50 states that the
SITE Program will prepare a videotape on the ATTIC system in
the spring of 1990. We believe that these two videotapes
will be helpful in informing OSCs and RPMs about the
program.
17 Our report never stated that the problems we found with
contractor performance came only from our survey
questionnaire comments. Instead, in every phase of our
audit (interviews with SITE officials, project managers,
developers, contractors not part of our survey, the SITE
quality assurance officer and economist; our file review;
and our review of independent reports), we found problems
with contractors. These problems were not confined to the
earliest contractors, but were across the board. We agree
with the last statement of this paragraph that ORO needs to
pursue the recommendations of this audit to improve
contractors' performance.
18 In our opinion, deliberately approving initial contractor
cost estimates that are below the realistic costs is not an
effective contract management practice. We believe that
this may result in approval of costs that everyone
understands are too low and that everyone knows will
actually be much higher. We believe that rather than
forcing the contractor to submit a cost proposal much lower
than is needed, approving a higher but more realistic
initial cost should be considered. We believe that the way
that SITE contracts are now managed, the contractor and
everyone else involved understands that cost increases will
not' only be forthcoming but will be expected. In our
opinion, this method may encourage the contractors to ask
for and expect cost increases to be routinely approved.
121
-------
APPENDIX 2
Page 5 of 15
A CMD official in Cincinnati, Ohio, agreed with this aspect
of our report and rebuttal. CMD had notified the SITE
contracts project officer of this fact. In addition, CMD
had done a limited review of some of the multitude of
contract increases that occurred due to this type of
contract management. This CMD official indicated that the
cost increases did not contain much justification and
appeared to be routinely approved. Lastly, the official
indicated that CMD was preparing a memorandum to SITE
Program management to request that project managers make
better use of the monthly contractor status reports in
managing and monitoring SITE contractor activities.
* ,*
19 Although the summary of the response is somewhat
inconsistent with the detailed attachment, we have
considered the summary to represent the official ORD
position.
20 The legislation did not specify these requirements.
Instead, SITE management established them. To require
income tax returns and other in-depth financial information
as is presently the case in SITE, is not specified in the
legislation. In addition, whether or not supplying this.
information to ORD is a "burden" is not the issue. Whilea itJ
may not be burdensome (by some definitions) to photocopy•andU
submit existing documents, the issue is the reluctance of
developers to submit all of this detailed information to ORD
when there is not a clear "need to know." ORD has not
demonstrated that anything beyond a valid rejection letter
from one or more venture capitalists is necessary. We
believe the Agency may have misinterpreted the intent of the
legislation, and thus circumvented congress' desire in this
area,
In addition, during our audit we found that lack of actual
funding can discourage developers. For example, in
discussions with developers that have gone though the SITE
process and have actually demonstrated,their technologies, 4
2 of 5 developers stated that funding was a definite
Z impediment to the program. In addition, 2 developers said
that, if given the opportunity, they would not participate
in the program again due to the high costs involved.
21 During our audit, SITE officials stated numerous times that
they believed the intent of SARA was unclear as to the
question of funding. In response to position papers we
issued in October 1989, these officials reiterated the lack
of clarity in SARA'S intent. They suggested at that time
that a clarification would be helpful. Now these same
officials indicate that the intent of SARA is clear as to . -
the funding issue. We are unaware, however, that in the '
122
-------
APPENDIX 2
Page 6 of 15
interim they have obtained any official clarification of
congressional intent.
22 From our file review of the nine technologies we reviewed,
we can not confirm this. In fact,, one of our sample sites,
Silicate, was from solicitation 003, and it had the longest
site selection time of all the ones in our sample. This was
not early in the program but was still in the site selection
process as of 10/1/89.
23 As stated in our.audit report, 12 of 36 technology
demonstrations were completed by 10/1/89. .In addition, a
13th technology; one of the Biotrol technologies (soil
washing), had started the demonstration as of 10/1/89 but
was not completed until the end of October 1989. The 14th
technology that ORD is claiming as having been demonstrated
(Ogden) was never considered a demonstration by SITE
officials during our audit. In fact, during our audit we
visited the McColl Superfund site where the Ogden technology
was to be demonstrated. In March 1989, Ogden had done a
pilot test of its technology. This was not a full-scale
; field test as is required for a demonstration under the SITE
Demonstration Program. During January 1990, we learned that
Ogden was withdrawing from the SITE Program and would not
demonstrate its technology at the McColl site. This was
confirmed in the January 12, 1990, SITE Program Quarterly
Status Report that shows the accomplishments and status of
each SITE project. This status report never mentioned that
the Ogden pilot-scale test was now considered a full-scale
demonstration of the technology. We believe that ORD is
calling this pilot test a full-scale demonstration of the
technology now that Ogden has withdrawn. We believe ORD
officials have misrepresented this information.
Further, in.a discussion within an official of Ogden, we
learned that Ogden was unable to proceed with the
demonstration at the McColl site unless EPA would provide
funding to do the demonstration. This official indicated
that since EPA would not reimburse them, Ogden was unable to
do the demonstration at.the McColl site.
24 At the time of our review, solicitation 003 represented the
most current group of technologies. For two of the four
technologies in solicitation 003 that we reviewed, site
selection took 11 months and 17 months. In fact, even
though solicitation 003 was the latest solicitation we
reviewed during our audit, it contained the technology
(Silicate) with the longest time needed for site selection
in our sample. We still believe that the Agency needs .more
aggressive action to meet its goal of site selection within
90 days.
123
-------
APPENDIX 2
Page 7 of 15
25 This was noted in our draft report. The issue we are
concerned with is that even after SITE officials considered
the site selections a certainty, the private owner of the
site did not, and the site selection was in question for a
whole year. It was not until a year later that the owner
gave permission for the demonstration to occur.
26 In our opinion, the times required for site selection in our
Table 1 are accurate. We determined these dates for site
selection from the individual project managers and from the
project files. In fact, at the completion of our, review, we
went back to the project managers of each site technology to
verify the date of site selection. The ORO response states
"several of the nine technologies analyzed in the 16 report
had to be rematched after problems developed with sites
which were initially selected." This is exactly the point
we are making. In several cases, initial sites selected
were turned down and new sites had to be selected. This is
why we believe that to meet the 90-day goal for site
selection, SITE officials will have to become more
aggressive in meeting milestones. We believe the
recommendations we have made in this report will help
improve the overall process.
27 In our report we discussed the repeated requests SITE
officials made to the regions to nominate sites. We believe
that merely requesting the sites has not worked. That is
why we recommended the Agency develop incentives for
regional RPMs and OSCs to allow sites to be used for
technology demonstrations, and add site demonstrations as a
performance indicator to SPHS or SCAP for management
emphasis. By doing this, SITE officials will not have to
repeatedly request regions to nominate sites.
28 This information was already extensively discussed on page
16 of our draft report. In fact, in many cases ORD's
response repeats almost verbatim the contents of;our draft.
r
-29 We presented this approach of working with Federal agencies
to conduct demonstrations to SITE officials during our
initial recommendation process as a possible course of
action to pursue more aggressively. We believe it can both
enhance the site matching process and assist these other
agencies in their cleanup efforts.
30 Exhibit 0 of our report does not represent the total results
of our telephone survey but only very specific "strengths
and weaknesses", as the title states. We did substantial
work to develop this issue. In addition, ORD "agrees that
incentives for SITE participation are not strong." •
124
-------
i
«i
APPENDIX 2
Page 8 of 15
31 OHO misunderstands the OIG report. The technologies used at
the two sites reviewed were not selected for the SITE
Program until after they were in use at the sites. In its
response to the draft report with respect to Peak Oil, the
region stated, "Region IVs actions at the site were not
intended as a SITE Program demonstration, but were removal
actions." We criticized Region 4 for using unproven
technologies without the kind of controls which exist in the
SITE Program. Our draft recommendation was to "Employ SITE
Program authorization and controls to. test and develop
alternative treatment technologies." Nothing in our report
indicated.that removal sites should not be selected for SITE.
demonstrations.
32 ORD's objection is based essentially on Exhibit D, which
does not show the entire results of our survey (see note
30). We quoted the contractor because he most succinctly
expressed the point, but it was a view shared by the OSCs
and RPMs. In the previous paragraph, ORD agrees that the
incentive for SITE participation is not strong. Finally,
SITE officials indicate they made major changes to get the
information to the OSCs and RPMs as quickly and in as usable
a format as possible.
33 Our report never concluded that developers should not be
involved in the site selection decision. ORD officials have
avoided the issue of allowing developers repeated rejections
while carrying them in the program.for years. See our
previous notes 10 and 12.
34 This is a perfect example of funding problems within the
SITE Demonstration Program. See our previous notes 20 and
23.
35 SITE officials have permitted some technology developers to
remain in the SITE Program for as long as 3 years without
doing a demonstration before asking the developer to
withdraw or exiting them. See our previous note 10.
36 We stand by the information that 11 developers have exited
the program. In the January 12, 1990, SITE Program
Quarterly Status Report, Ogden is shown as having withdrawn
from the SITE Program. See our previous note 23. We did,
however, modify our report to acknowledge that some
relatively minor benefits may have accrued to EPA by these
exited technologies.
37 Our report mentions the Administrator's report and makes the
recommendation that the SITE Program demonstrate
combinations of technologies to more accurately encompass
"real world" cleanup conditions. This issue was also
125
-------
APPENDIX 2
Page 9 of 15
mentioned in the December 1986 SITE Strategy and Program
Flan,. To date, the SITE Program has not demonstrated a
combination of technologies.
38 We are pleased that SITE officials will renew their efforts
to determine the level of developer commitment during the
application phase. We believe this will help ensure that
fewer developers exit from the SITE Program in the future.
39 We are pleased to see that ORD officials have mentioned one
of our suggested incentives to increase regional site
selection for future demonstrations. We proposed&this
incentive of extending the allowable timeframe for
completion of some Superfund activities during meetings with
SITE officials.
40 We strongly disagree. The results presented in our draft
report reflect all SITE reports that were issued as of
9/30/89, and not only the early reporting experience as
claimed by ORD.
41 In our opinion, ORD has confused guidance with results. We
do not question that ORD has issued guidance; however, it
has not produced the results needed. For example, while ORD..
acknowledged the reporting problems in June 1988 and issued -
guidance, in March 1989 they still stated they had reporting
and reviewing problems. We believe this problem still
continues and must be further addressed by ORD.
42 While our report does show some improvement is this effort,
we are still concerned that the reports continue to take so
long. On page 23 of 50 of their response, SITE officials
mention they are "concerned over the time taken to prepare,
review, and publish the TERs and AARs and the impact these
delays could have on the effectiveness of the SITE program."
We believe that our finding and recommendations will help
decrease the length of this process and insure better
results in the future. •'.#•-
..,
i43 We agree with ORD officials as they acknowledge here that
"RPMs and OSCa rslv on the informatjnn ^nd results in these
reports to select innovative technologies for remedial
actions." This is what we have been saying all along in our
report, that these end reports are the products needed by
RPMs and OSCs to select and make their decisions to clean up
sites.
44 This is the information we show in Table 2 on page 24 of our
report. ORD mentioned that a key factor would be producing
only one draft rather than several rewrites. We also
believe that ORD must get a handle on this to improve the
126
-------
APPENDIX 2
Page 10 of 15
report process. We make recommendations in our report that
address this issue.
45 We strongly disagree. We do not overlook the importance of
the various technology transfer products. However, as ORD
acknowledges on page 23 of 50 of their response, the TERs
and AARs are the most critical reports needed by the RPMs
and the OSCs. While all the interim information is helpful,
it is still these end reports that the RPMs and OSCs depend
on to make cleanup decisions.
46 ORD has.asserted that they believe offering demonstration
results and information on an automated system can be more
useful to RPMs, OSCs/ etc., than published TERs and AARs.
While this method is useful in alerting RPMs and OSCs to the
availability of technologies, the TERs and AARs are needed
to fully evaluate the suitability of the technologies to
particular sites.
f *
"47 As a point of clarification, we conducted an audit and not
an investigation.
48 Our report's Scope and Objectives section presents
sufficient detail and information on the nature of our
telephone survey.
49 ORD has misrepresented our sample size and survey. Our
sample size was calculated to give the greatest number of
respondents. Our sample was stratified not by region but by
the type of respondent, i.e. RPM, OSC, Superfund contractor,
and was randomly selected (using interval sampling) from a
nationwide printout provided by the Office of Emergency and
Remedial Response. ORD's number citations given here
apparently relied on our "number who responded" line in
Exhibit D rather than our "number contacted" as given on
page 6 of our report and at the:;top of Exhibit D.
ORD also is concerned here on the percentages given. In our
opinion, any reader will typically covert raw numbers to a
percentage to determine significance of the response. We
did this presentation anticipating the reader's needs. This
is a very typical way of presenting audit results. For
example, GAO often presents information like this in their
reports as well.
Finally, no respondents were "screened out," as ORD alleges.
Receiving an answer to a questionnaire such as "I do not
know" is a valid response, and reflects the lack of
awareness of the SITE Program by many regional personnel.
127
-------
»
f
APPENDIX 2
Page 11 of 15
50 Our file review included all contractors not just the ones
used early in the program.
51 We are concerned that ORD is only mentioning fixed price
contracts. This is only one of the two contract types we
mentioned. ORD appears to have ignored the CPIF contract we
mentioned in our report.
A PCMD official was not certain how SITE officials could say
that the percentage of award fees for their program was
lower than for other EPA contracts of this type. The PCMD
official was not aware of any such "typical" information
available within EPA. ,'
52 Preparing reports is only one part of the situation. In our
report, see finding 3 for further details.
53 These concerns came from a number of developers, RFMs, OSCs,
etc., as well as from our file review and interviews of
project managers, other SITE officials and cleanup
contractors; and represent all three fiscal years reviewed
and all demonstration contractors.
54 These concerns expressed in our report applied to the entire""
time of our audit period not just the "start up" of the SITE^
Program.
55 In our opinion, this is still a problem. For example, see
page 33 of our report. The QA/QC officer stated he plans to
publish a pocket QAPP for fiscal 1990 because contractors
still did not always follow the guidance provided in 1987.
Also, we found that in reviewing award fee performance
indicators, evaluators mentioned that there was still a
problem in the area of quality assurance plans. See pages
40 and 41 of our report. ..':..
56 We strongly disagree with this type of reasoning, especially
for CPIF contracts. We believe that ORD needs to further
review this area.
•v.
57 This is not just our opinion. For example, this is
expressed in the FAR, DOD FAR, and other DOD procurement
regulations. DOD has extensive experience in research and
development.
58 We believe one of the problems here is that EPA uses CPAF
contracts for virtually everything. In addition, the 3-
percent incentive fee mentioned here is an insignificant
incentive compared to the average of nearly 100 percent cost
increases shown in our report for SITE projects. .,
128
-------
APPENDIX 2
Page 12 of 15
59 ORO indicates here that many of these technologies are being
demonstrated for the first time at full-scale. This is a
concern we have, for example, with the Ogden technology. A
pilot-test Ogden did early on is now being claimed by EPA as
a full-scale demonstration. We believe SITE officials have
misrepresented this issue. See our previous note 23.
60 We strongly disagree here with ORD's assertion that our
table is incorrect. The only change ORD makes is for Terra
Vac, and ORD claims this was not a cost estimate or
increase. However, the documentation provided to us clearly
shows that the $550,000 was an ^original estimate and the
$757,900 was a cost increase that was granted because of two
substantial.extensions of the demonstration from 14 days to
56 days. These increases occurred after issuance of the
work assignment by the project manager. Other causes for
the increased project duration and cost included: a longer
than expected installation of the system and delay of
project start up by over a month; increases in field work
cost; longer operating time and drilling costs; and the
contractor using a subcontractor to take samples during the
extended demonstration period. The other $236,200 resulted
from a negotiated EPA-contractor fund increase for field
work costs, operating time, and cost of waste disposal. All
of these reasons ORD officials summarized to us as "Conduct
Demonstration and Write Report."
61 This issue was outside the scope our audit. We would have
had to evaluate each individual technology to determine
this. This is something that ORD itself needs to do.
62 Our file review and interviews do not support the assertion
made by ORD. From our review, it appears that cost
increases were not always known before hand. In addition,
the justification in some cases was very brief and showed
little or no indication of reason. See our previous note .
18.
63 In our opinion,.we still believe that ORD needs to fully
explore the possibilities of using a different contract
type.
64 In our opinion, stating that SITE officials had little
opportunity to develop funding assistance procedures seems
inappropriate. As SITE officials acknowledge, they
anticipated SARA'S enactment. In fact, SITE officials told
us that they were working on the SITE Program for almost a
year before enactment of the SARA legislation. We believe
SITE officials had the opportunity to prepare funding
provisions during this period of time, especially with the
close coordination of EPA and Congress during the
129
-------
APPENDIX 2
Page 13 of 15
preparation of SARA, In addition, EPA has been providing
funding in various ways for many years. We believe that
this issue should npt have been such a time consuming
problem as SITE officials have made it.
65 ORD has asserted that they believe applying for financial
assistance will not affect an applicant's evaluation under
their technical selection criteria. However/ we saw some
indications where it may have ultimately affected the
developer's admission to the program.
f,
66 We believe there are other options available. One-would be
lowering the amount of the contractor's share of the program
costs (74 percent or higher during fiscals 1987-89). Also,
ORD could request additional funding. In addition, we have
definite comments from developers that funding is an
impediment to the program and that, if given a second
chance, they would not participate due to the high cost and
lack of realistic funding. See our previous note 20.
67 We repeatedly asked for documentation such as this during
our review. However, SITE officials could provide us with
no such documentation. ,,
-1-.1
68 During the audit, SITE officials told us that because OGC
did not respond to ORD on their funding question, funding
for solicitation 002 was affected. In fact, this was one of
the reasons why funding was not available until solicitation
003.
69 In our report on pages 43 and 44, we stated that SITE
officials informed the company that EPA had not formalized
the guidelines for federal assistance. In addition, EPA
informed the company that if they were unsuccessful in
getting private funding, the company may want to come back
to EPA for funding, but SITE officials could not guarantee
them any funding. For these reasons, we believe that the
company was discouraged from applying for funds and thus
never did formally apply for funding assistance.
70 We continue to believe, as we stated in our report on page
46, that EPA' s interests in these technologies are different
from a venture capitalist's. Consequently, we do not
believe all the information EPA requires to be submitted is
necessary.
71 in our opinion, a question still remains regarding what SITE
officials presented here. How does anything other than a
rejection letter from a financial institution confirm this?
In addition, how will SITE officials ensure that federal
funds are invested "wisely and prudently?" SITE officials
130
-------
APPENDIX 2
Page 14 of 15
repeatedly told us they were not capable of making such a
determination. In addition, SITE officials admitted to us
that they do not have the procedures needed to consider and
analyze the funding request from a developer.
72 SARA legislation does not even mention providing funding to
the Emerging Technologies Program. Our concern has nothing
to do with whether or not a technology is an "experimental
activity."
73 ORD asserts that these were some of the earliest
participants in the program and this does not happen
anymore. This is not what we found. Two .of the five
developers had demonstrations as late as March 1989. Two of
the five developers were in solicitation 002, and a third
developer was in solicitation 003. Therefore, these
developers were not some of the earliest participants as
SITE officials claim.
74 The comments made by the developers for the most part
involve comments directed at the conduct and management of
the SITE Program -- not how useful the program was to them
from a business perspective, as SITE officials claim. We
believe that these comments made by developers should be
taken by SITE officials in a positive mode to help improve
the SITE Demonstration Program.
75 During the audit SITE officials were unable to provide us an
answer to this issue. We accept the answer given here. One
comment we would make is that SITE officials do not indicate
here that they sent a true response to the Congressman, but
only an acknowledgement of receipt of his letter. We
believe SITE officials should send a letter to the
Congressman explaining their response.
76 We disagree and stand on the information presented in our
report. Our file review and documentation show the
information given in our report to be correct.
77 While EPA may have received enough responses to select 10
technologies for the program each fiscal year, we are
concerned that the response rate is declining (see Table 7
on page 52 of our report). Also, only those developers who
can afford to bear the expense of a demonstration may be
applying. These developers may not necessarily represent
the best, most needed technologies. Smaller developers,
unable to bear the cost of a demonstration, may not be
applying for the program.
78 We strongly disagree with the overall comments made by SITE
officials concerning Exhibit D. We again state that Exhibit
131
-------
'.1
f*
't
APPENDIX 2
Page 15 of 15
D does not, nor did we ever say it did, present all the
results of our survey. Only specific strengths and
weaknesses are mentioned. See our previous notes 30 and 49
132
-------
r
APPENDIX 3
LIST OF ABBREVIATIONS USED IN REPORT
AAR
CERCLA
CPAF
CPIF
CRS
DOE
EPA
FAR
FPIF
GAD
IWT
NASA
NPL
OGC
OIG
OMB
ORD
OSC
OSWER
PCB
QAPP
QA/QC
RFP
RI/FS
ROD
RPM
RREL
SARA
SCAP
SITE
SPMS
STC
TER
TQM
Applications Analysis Report
Comprehensive Environmental Response, Compen-
sation and Liability Act
Cost-Plua-Award-Fee
Cost-Plus-Incentive-Fee
Congressional Research Service
Department of Energy
Environmental Protection Agency
Federal Acquisition Regulation
Fixed-Price-Incentive-Fee
Grants Administration Division
International Waste Technology
National Aeronautics and Space Administration
National Priorities List
Office of General Counsel
Office of Inspector General
Office of Management and Budget
Office of Research and Development
On-Scene Coordinator
Office of Solid Waste and Emergency Response
Polychlorinated Biphenyls
Quality Assurance Project Plan
Quality Assurance/Quality Control
Request for Proposals
Remedial Investigation/Feasibility Study
Record of Decision
Remedial Project Manager
Risk Reduction Engineering Laboratory
Superfund Amendments and Reauthorization Act
Superfund Comprehensive Accomplishments Plan
Superfund Innovative Technology Evaluation
Strategic Planning and Management System
Silicate Technology Corporation
Technology Evaluation Report
Total Quality Management
133
-------
APPENDIX 4
DISTRIBUTION OF REPORT
Assistant Administrator for Research
and Development (RD-672)
Assistant Administrator for Solid Waste
and Emergency Response (OS-100)
Assistant Administrator for Administration
and Resources Management (PM-208)
Associate Administrator for Regional Operations (A-101)
Comptroller (PM-225)
Agency Followup Official (PM-225)
Attn: Director, Resource Management Division
Agency Followup Official (PM-208)
Director, Procurement and Contracts
Management Division (PM-214F)
134
------- |