United States Science Advisory Board EPA-SAB-EEC-97-005
Environmental Washington DC July 1997
Protection Agency
&EPA REVIEW OF THE SUPERFUND
INNOVATIVE TECHNOLOGY
EVALUATION (SITE)
PROGRAM
PREPARED BY THE SITE
SUBCOMMITTEE OF THE
ENVIRONMENTAL ENGINEERING
COMMITTEE (EEC)
-------
July 15, 1997
EPA-SAB-EEC-97-005
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M Street, SW
Washington, DC 20460
Subject: Review of the Superfund Innovative Technology Evaluation (SITE)
Program
Dear Ms. Browner:
At the request of the Office of Research and Development (ORD), the Superfund
Innovative Technology Evaluation (SITE) Subcommittee of the Environmental
Engineering Committee (EEC) of the Science Advisory Board (SAB) reviewed the
Agency's SITE program. The Subcommittee met June 11-13, 1996 at the National Risk
Management Research Laboratory (NRMRL) in Cincinnati, Ohio. The EEC approved
the Subcommittee's report April 2, 1997 and the SAB's Executive Committee approved
this report April 25, 1997.
EPA developed the SITE program in response to the Superfund Amendments
and Reauthorization Act of 1986 (SARA). Participation of technology developers in
SITE is voluntary and since its inception SITE'S primary mission has been the
evaluation of innovative treatment technologies for contaminated site remediation. It is
anticipated that these evaluations will result in commercialization of promising
technologies.
The Risk Reduction Research Laboratory (now part of the National Risk
Management Research Laboratory) and the Environmental Monitoring Systems
Laboratory (now the National Exposure Research Laboratory) directed the SITE
program. ORD requested that the EEC review the technical aspects of the program
including:
a) Establish the extent to which the stated program objectives have been
met;
-------
b) Review the scientific and technical aspects of the SITE program and its
implementation on all levels (bench, pilot, and full), including the
preparation of test, quality assurance (QA), and quality control (QC)
plans, sample collections, and field activities, and procedures for
evaluation and interpretation of results leading to conclusions and
recommendations;
c) Identify impacts and provide recommendations for potential
improvements; and
d) Determine how well the approach taken has supported technology
commercialization.
The Subcommittee was also asked to recommend program improvements and
highlight strengths.
The Subcommittee recognized that decision-makers must consider many factors
in addition to science when determining the existence, size, or form of a program. The
legislative language as well as the legislative history provide the policy directives to the
Agencies in the Executive branch of the federal government. The Subcommittee
restricted its review of the SITE program to the science and engineering context and to
SITE'S impact on the commercialization of innovative technologies. Therefore, its
comments should be understood in that same context.
The Subcommittee found the accomplishments of the SITE program to be
impressive. Stated program objectives have been generally met or exceeded, and the
program has provided credible information on mostly innovative technologies, many of
which are in use today at Superfund or RCRA Corrective Action sites. EPA personnel
have done a good job of conducting self reviews and adjusting the program to improve
its operations.
The need for improved technology to prevent, reduce or remediate
environmental contamination remains a national priority. The Subcommittee believes
that in order to meet this need, the Agency should build upon the success of SITE
either by continuing the existing program with some improvements or by creating a new
technology evaluation program built upon the precepts of the current program.
To help select the most appropriate course of action, the Agency should define
the desired products, the customer base, and plan how best to bring the two together.
-------
Better product definition and improved marketing-including a clear portrayal of the
successes and benefits of the program-will make the program better understood by
and even more valuable to the user community. Interpretive documents summarizing
and comparing the results of the individual technology evaluations would help
practitioners choose wisely from a variety of potentially useful technologies.
The Subcommittee recommends two technical improvements to the evaluations.
Cost information should be collected in a more consistent and complete manner,
because, often times, cost is the pivotal decision criterion. Also, the level of rigor
applied to material balances must be increased to more fully address the Agency's
stated concerns about trans-media migration of hazardous constituents and resultant
concentration data that could be used in future RCRA Risk-Based Cleanup Actions at
actual field sites. SITE should strive for material balances in demonstration projects
providing a high level of closure for hazardous constituents.
The Subcommittee appreciates the opportunity to review this program, and looks
forward to a written response to its recommendations for the SITE program.
Sincerely,
/signed/
Dr. Genevieve M. Matanoski, Chair
Executive Committee
/signed/
Dr. Ishwar P. Murarka, Chair
Environmental Engineering Committee
/signed/
Dr. Wayne M. Kachel, Chair
SITE Subcommittee
-------
NOTICE
This report has been written as part of the activities of the Science Advisory
Board, a public advisory group providing extramural scientific information and advice to
the Administrator and other officials of the Environmental Protection Agency. The
Board is structured to provide balanced, expert assessment of scientific matters related
to problems facing the Agency. This report has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the views
and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or
commercial products constitute a recommendation for use.
-------
ABSTRACT
The Superfund Innovative Technology Evaluation (SITE) Subcommittee
reviewed the technical aspects of EPA's SITE program including: a) whether the stated
program objectives have been met; b) scientific aspects of the program, including the
preparation of test, quality assurance, and quality control plans, sample collections,
and field activities, and procedures for evaluation and interpretation of results leading
to conclusions and recommendations; c) impacts and improvements; and d) whether
the approach taken has supported technology commercialization.
The Subcommittee found the accomplishments of SITE to be impressive
because they generally met or exceeded stated program objectives and because SITE
provided credible information on mostly innovative technologies, many of which are in
use today. The annual Reports to Congress and Associated Technical Profiles have
recorded the growth and development of SITE to a multi-faceted program which
included 77 emerging technologies, 124 demonstrations, 30 monitoring and
measurement methods, and an extensive technology transfer effort. An analysis of
cost savings using innovative technologies for 17 sites in four EPA regions, indicates
that the innovative technologies saved five to two hundred million dollars per region.
Because of the continued need for and the contributions made by the SITE
program, the Subcommittee advises that the Agency build upon the successes of SITE
and continue with either a refinement of SITE or a "SITE Like" program for evaluation of
innovative technologies. The Agency should define the desired products and the
customer base, then develop a plan for how best to provide the product to the
customer. SITE should clearly portray the successes and benefits of the program, and
provide interpretive analyses to facilitate decisions about which technologies are best
suited to their situations.
The Subcommittee also recommends that more complete and consistent cost
data be developed in future demonstration projects and that the rigor of materials
balances be increased to better account for the fate of hazardous constituents.
Keywords: technology, development, demonstration, evaluation, Superfund
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee
SITE Subcommittee
CHAIR
Dr. Wayne M. Kachel, Program Manager, MELE Associates, Brooks AFB, TX
MEMBERS
Dr. George F. Carpenter, Michigan Department of Natural Resources,
Environmental Response Division, Lansing, Ml
Dr. Calvin C. Chien, Environmental Fellow, Corporate Remediation,
E. I. DuPont Company, Wilmington, DE
Dr. Ishwar P. Murarka, Technical Executive, Environmental Group, Electric Power
Research Institute, Palo Alto, CA
Dr. John P. Maney, President, Environmental Measurements Assessment,
5 Whipple Road, Hamilton, MA
Dr. Frederick G. Pohland, Weidlein Chair of Environmental Engineering,
Department of Civil and Environmental Engineering, University of Pittsburgh,
Pittsburgh, PA
SCIENCE ADVISORY BOARD STAFF
Mrs. Kathleen W. Conway, Designated Federal Official, U.S. EPA, Science Advisory
Board (1400), 401 M Street, SW, Washington, DC 20460
Mrs. Dorothy M. Clark, Staff Secretary, U.S. EPA, Science Advisory Board (1400), 401
M Street, SW, Washington, DC 20460
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee (FY96)
CHAIR
Dr. Ishwar P. Murarka, Environmental Group, Electric Power Research Institute, Palo
Alto, CA
MEMBERS
Dr. Calvin C. Chien, Corporate Remediation, E. I. DuPont Company,
Wilmington, DE
Dr. Hilary I. Inyang, Dept. of Civil Engineering, University of Massachusetts,
Lowell, MA
Dr. James H. Johnson, Jr., School of Engineering, Howard University,
Washington, DC
Dr. Wayne Kachel, MELE Associates, Brooks AFB, San Antonio, TX
Dr. Jo Ann Lighty, Department of Chemical and Fuels Engineering,
University of Utah, Salt Lake City, UT
Dr. James W. Mercer, GeoTrans, Inc., Sterling, VA
Dr. Frederick G. Pohland, Department of Civil and Environmental Engineering,
University of Pittsburgh, Pittsburgh, PA
Dr. Robert B. Pojasek, Cambridge Environmental, Inc., Cambridge, MA
Ms. Lynne Preslo, Technical Programs, Earth Tech, Berkeley, CA
Dr. Wm. Randall Seeker, Energy & Environmental Research Corp., Irvine, CA
Science Advisory Board Staff
Mrs. Kathleen W. Conway, Designated Federal Official, U.S. EPA, Science Advisory
Board (1400), 401 M Street, SW., Washington, DC 20460
Mrs. Dorothy M. Clark, Staff Secretary, U.S. EPA, Science Advisory Board (1400), 401
M Street, SW., Washington, DC 20460
iv
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
Science Advisory Board
Environmental Engineering Committee (FY97)
CHAIR
Dr. Ishwar P. Murarka, Environmental Group, Electric Power Research Institute, Palo
Alto, CA
MEMBERS
Dr. Edgar Berkey, President of the Center for Hazardous Materials Research
The National Environmental Technology Applications Center, Pittsburgh, PA
Dr. Calvin C. Chien, Corporate Remediation, E. I. DuPont Company,
Wilmington, DE
Dr. Nina Bergan French, SKY+ , Oakland, CA
Dr. Hilary I. Inyang, Dept. of Civil Engineering, University of Massachusetts,
Lowell, MA
Dr. James H. Johnson, Jr., School of Engineering, Howard University,
Washington, DC
Dr. Jo Ann Lighty, Department of Chemical and Fuels Engineering,
University of Utah, Salt Lake City, UT
Dr. John P. Maney, President, Environmental Measurements Assessment,
Hamilton, MA
Dr. James W. Mercer, GeoTrans, Inc., Sterling, VA
Ms. Lynne Preslo, Technical Programs, Earth Tech, Berkeley, CA
Dr. Wm. Randall Seeker, Energy & Environmental Research Corp., Irvine, CA
Science Advisory Board Staff
Mrs. Kathleen W. Conway, Designated Federal Official, U.S. EPA, Science Advisory
Board (1400), 401 M Street, SW., Washington, DC 20460
Mrs. Dorothy M. Clark, Staff Secretary, U.S. EPA, Science Advisory Board (1400), 401
M Street, SW., Washington, DC 20460
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 3
2.1 Background 3
2.2 Context 4
2.3 The Review and Charge 4
3. RESPONSE TO THE CHARGE 6
3.1 Meeting Objectives 7
3.1.1 Summary of Stated Objectives 7
3.1.2 Subcommittee Findings as They Relate to the Objectives 8
3.1.3 Findings Relating to Unstated Objectives 11
3.1.4 Developing More Demanding Objectives 12
3.2 Scientific/Technical Aspects and Implementation 13
3.2.1 Project Planning 13
3.2.2 Quality Assurance and Quality Control 14
3.2.3 Interpretative Analysis 15
3.2.4 Mass Balances 16
3.2.5 Coordination of MMTP Activities 16
3.3 Impacts and Improvements 17
3.4 Technology Commercialization 18
3.4.1 Factors Affecting Commercialization 18
3.4.2 Timeline 21
3.4.3 Technology Transfer 21
3.4.4 Market Determination 22
3.4.5 Permitting 22
3.4.6 Conclusions About Commercialization 22
3.5 Future Direction 23
4. RECOMMENDATIONS 24
REFERENCES CITED R-1
APPENDIX A - ACRONYMS AND ABBREVIATIONS A-1
APPENDIX B - DOCUMENTS REVIEWED B-1
VI
-------
1. EXECUTIVE SUMMARY
EPA developed the Superfund Innovative Technology Evaluation (SITE)
program in response to the provisions of the Superfund Amendments and
Reauthorization Act of 1986 (SARA). Its primary mission has been to develop,
implement, and commercialize innovative treatment technologies for contaminated
hazardous waste site remediation.
From June 11 through 13, 1996, the SITE Subcommittee met at the National
Risk Management Research Laboratory's (NRMRL) to review the SITE program. The
charge was to review the technical aspects of the program including:
a) Establish to what extent the stated program objectives have been met;
b) Review the scientific and technical aspects of the SITE program and its
implementation on all levels (bench, pilot, and full), including the
preparation of test, QA, and QC plans, sample collections, and field
activities, and procedures for evaluation and interpretation of results
leading to conclusions and recommendations;
c) Identify impacts and provide recommendations for potential
improvements; and
d) Determine how well the approach taken has supported technology
commercialization.
The Subcommittee was also asked to recommend Program improvements and
highlight strengths.
The Subcommittee found the accomplishments of the SITE program to be
impressive. Stated program objectives have been generally met or exceeded, and the
program has provided credible information on innovative technologies, many of which
are in use today at Superfund or Resource Conservation and Recover Act (RCRA)
Corrective Action sites. An analysis of cost savings using innovative technologies for
17 sites in four EPA regions, indicates that the innovative technologies saved five to
two hundred million dollars per region (USEPA, 1996b). EPA personnel have done a
good job of conducting self reviews and adjusting the program to improve its
effectiveness.
-------
The need for improved technology to prevent, reduce or remediate
environmental contamination has not disappeared. Because that need remains, and
because the SITE program has been relatively successful, the Subcommittee-speaking
from a science and engineering viewpoint-strongly recommends that the Agency build
upon the successes of SITE either by continuing the existing program or by creating a
new technology evaluation program built upon the precepts of the current program.
To help select the most appropriate course of action, the Agency should define
the desired products, the customer base, and plan how best to bring the two together.
Better product definition and improved marketing-including a clear portrayal of the
successes and benefits of the program-will make the program better understood by
and even more valuable to the user community. Interpretive documents summarizing
and comparing the results of the individual technology evaluations would be easily
referenced sources of credible information. Such easy-to-use documents would help
practitioners choose among the variety of potentially applicable technologies.
It is now time for the program to develop more substantive objectives with which
to measure the progress of the program. Metrics are needed for the various objectives
to evaluate the program's very real contribution to remediation and technical cultivation.
For purposes of illustration, additional metrics might include: a tabulation of the
technologies that have participated in SITE, the subsequent use of those SITE
technologies in remediation (or elsewhere) and the number of locations where they
were used; some measures of reduced costs for remediation, restoration, measurement
and monitoring; and documentation of cleanups that are cheaper, faster, and better
because of SITE.
The Subcommittee recommends two technical improvements to the evaluations.
Cost information should be collected in a more consistent and complete manner,
because, often times, cost is the pivotal decision criterion in selecting a technology at a
site. Also, the level of rigor applied to material balances must be increased in response
to the Agency's stated concerns about trans-media migration of hazardous
constituents. SITE should strive for material balances that provide a high degree of
certainty as to the fate of hazardous constituents.
-------
2. INTRODUCTION
2.1 Background
EPA developed the Superfund Innovative Technology Evaluation program
(SITE) in response to the provisions of the Superfund Amendments and
Reauthorization Act of 1986 (SARA). Its primary mission has been to evaluate
innovative treatment technologies for contaminated site remediation. The mission is
accomplished through four sub-programs.
a) Demonstration Program (DP) - Conducts and evaluates demonstrations of
promising innovative site cleanup technologies to provide reliable
performance and cost information.
b) Emerging Technology Program (ETP) - Provides funding to developers to
continue research efforts at the bench and pilot-scale levels for the
development of innovative technologies.
c) Monitoring and Measurement Technologies Program (MMTP) - Develops
technologies that detect, monitor, and measure hazardous and toxic
substances to provide better, faster, and more cost-effective methods for
producing real-time data during site characterization and remediation.
d) Technology Transfer Program (TTP) - Disseminates technical information
on innovative technologies obtained via DP, ETP, and TTP.
In 1985, even before the SITE program was established, the Environmental
Engineering Committee (EEC) of the Science Advisory (SAB) sent a resolution to the
EPA Administrator expressing concern, "that enormous expenditures were being made
under Superfund without an adequate technological data base to support rehabilitation
of... sites" (SAB, 1985). In response, the SAB's EEC was asked to review plans for
the SITE program and "found the draft Strategy and Program Plan to be well done, and
a good basis for proceeding." (SAB, 1986). The following year, the SITE Emerging
Technology Program began to foster and accelerate the research and development of
innovative technologies.
At a June 28, 1990 public meeting, the EEC expressed interest in reviewing the
SITE program. A variety of reviews subsequently sharpened the EEC's interest in
-------
innovative environmental technologies and in how to encourage further technology
development. In FY95, agreement was reached with the Office of Research and
Development (ORD) and the Office of Solid Waste and Emergency Response
(OSWER) for the EEC to undertake this review of the SITE program.
2.2 Context
The Subcommittee recognizes that decision-makers must consider many factors
in addition to science when determining the existence, size, or form of a program.
Public values, government policy, various current events, the economy, and competing
needs are part of this larger context. The Subcommittee restricted its review of the
SITE program to the science and engineering context and to SITE'S impact on the
commercialization of innovative technologies. Therefore, its comments should be
understood in that limited context.
The Subcommittee assumes that technology development and evaluation should
and will continue at EPA. Thus, the recommendations herein are intended to improve
that undertaking. Consideration of non-science factors could lead a decision-maker to
alter or end EPA programs that assist with technology development and evaluation.
However, from a science and engineering viewpoint, knowing that the annual market
for remediation in the U.S. alone approaches $10 billion and that decades will be
needed to clean up those locations already identified-and having seen research and
improved technology lower costs, improve performance, and speed clean-up (NETAC,
1995)-the Subcommittee finds a compelling need for continued existence and
improvement of SITE.
An independent technical review of an ongoing program can result in significant
beneficial improvements to the program. In choosing to conduct the SITE review even
during the uncertain budget decisions for FY96-97, the Subcommittee has
acknowledged the fundamental importance of technology development and evaluation
programs in the public sector. Because of SITE'S uniqueness as a prototype for
government/private sector technology innovation programs, documenting the benefits
and shortcomings of SITE could be useful to other programs (particularly for programs
such as the Environmental Technology Verification Centers currently being planned).
2.3 The Review and Charge
On June 11-13, 1996, the SITE Subcommittee of the EEC met at the National
Risk Management Research Laboratory (NRMRL) to review the SITE program. The
-------
Subcommittee was charged to review the technical aspects of the SITE program
methodology, and to:
a) Establish the extent to which the stated program objectives have been
met;
b) Evaluate the scientific and technical aspects of the SITE program and its
implementation on all levels (bench, pilot, and full), including the
preparation of test quality assurance (QA) and quality control (QC) plans,
sample collections, and field activities, and procedures for evaluation and
interpretation of results leading to conclusions and recommendations;
c) Identify impacts and provide recommendations for potential
improvements; and
d) Determine how well the approach taken has supported technology
commercialization.
During the June 11-13, 1996, meeting, the Subcommittee was also asked to
recommend potential future program directions.
A full listing of the review documents provided prior to and during the meeting
can be found in Appendix B.
-------
3. RESPONSE TO THE CHARGE
The Subcommittee finds itself in the unusual position of pointing out that the
SITE program has not received full recognition for its contributions. The SITE program
has met the objectives set for it, has become a model for at least one other technology
evaluation program, EnTICE, and has played a substantial role in joint technology
evaluations with the Departments of Energy and Defense. Because the need for
technology that reduces or remediates environmental contamination remains, the SITE
program should be continued, with some modifications. The Subcommittee's cross-
cutting findings follow immediately. The detailed response to the charge appears in
Sections 3.1 through 3.5.
First, the Subcommittee commends the SITE staff for their careful preparation for
the review, their responsiveness to requests for additional information before and
during the review, and for being forthright in discussions. This enabled the
Subcommittee to reach informed conclusions about the various aspects of the SITE
program. The quality of the interaction indicates that the SITE program is staffed with
professionals who understand the scientific and engineering requirements of
technology evaluation.
Indeed, SITE program staff have recognized the importance and value of using
good science and technology from the inception of the program. The quality
assurance, quality control, and auditing aspects of the program are now well organized
and have resulted in systematic improvements to the implementation of demonstration
projects under SITE. EPA personnel have done a good job of conducting self-reviews
and using the results to improve the SITE program as warranted. As a result, the
program has been very productive.
Second, the SITE program is generating credible, objective evaluations of
individual technologies selected for demonstrations, however, the scientific rigor of the
program-coupled with a natural reluctance to appear to favor any technology or vendor
-has constrained the conclusions which SITE has drawn. The SITE program staff
appear to have a more comprehensive understanding of the limitations and the
applicability of the evaluated technologies than that indicated by the Innovative
Technology Evaluation Reports (ITER) for the Demonstration Program. There appear
to be both data and expertise to support interpretive analyses that could help
developers by guiding future development and could help potential users select the
most appropriate technologies. Such analyses are not currently conducted.
-------
The Subcommittee understands that developers, consulting engineers, local,
state and federal officials making decisions on site remediation, investors, the media
and the lay public all can benefit from the SITE results. The Subcommittee finds that
SITE results are currently reported in a manner suitable for the professional peers of
the researchers and in a manner useful to engineers experienced in location
remediation. Some of the consumers of SITE information may have conflicting needs.
Some developers may not wish their technology to appear with descriptions of other
technologies in an interpretive document where comparisons can more easily be drawn
about which technology is most useful for a particular application. Such documents are
well within the technical capability of the staff and are potentially of great use to less
experienced engineers, decision-makers and the lay public.
Speaking from the research science standpoint, the creation of such interpretive
documents would be a valuable advancement. In the context of regulatory science, it
would appear that such documents should make SITE more accessible and
accountable to a broader range of its potential users.
Third, the program could benefit from an updated strategy developed, in part,
from an improved understanding of SITE'S customer base, the products needed, and
how best to deliver those products to the customers. The updated strategy should
identify metrics for measuring the success of the program as a whole, not just for the
success of individual projects. Examples of such metrics might include: documenting to
what extent SITE has helped commercialization of demonstrated technologies or
documenting reduced remediation costs; follow-up on technologies evaluated to assure
that recommended improvements are widely implemented; and documenting the use of
investigation and monitoring techniques evaluated by SITE.
3.1 Meeting Objectives
Many objectives have been set for SITE. Section 3.1.1 summarizes the stated
objectives and Section 3.1.2 summarizes the Subcommittee's findings in relation to
those objectives. Section 3.1.3 presents the Subcommittee's findings as they relate to
unstated objectives. Section 3.1.4 recommends development of more demanding
objectives.
3.1.1 Summary of Stated Objectives
After more than a decade of operation, the SITE program is well-defined ,with
established objectives and implementing tactics. While few of these objectives were
originally stated in terms of measures of quality or quantity, the Subcommittee finds
-------
substantial quantitative and qualitative evidence that the SITE program has met or
exceeded the objectives as stated in SARA, in the mission statement, and in review
materials.
The Superfund Amendments and Reauthorization Act of 1986 (SARA) required
EPA, "to establish a comprehensive and coordinated Federal program of research . . .
evaluation, testing, development, and demonstration of alternative or innovative
treatment technologies . . . which may be utilized in response actions to achieve more
permanent protection of human health and welfare and the environment." SARA also
required an annual solicitation, demonstration plans, ten demonstrations per year, and
annual reports to Congress.
The SITE program's mission statement reads, "to conduct technology
development and demonstration projects, technology transfer activities, and technical
assistance to increase commercialization of innovative hazardous waste clean-up
technologies" (USEPA, 1996a).
In presentations to the Subcommittee, the following Program objectives were
stated; "facilitate acceptance by regulators, provide a sound scientific basis for
evaluating technology, provide reliable performance information, support the use of
Verified' technologies, pool resources of Federal and private sectors, and reduce the
time it takes to get new technologies into routine use" (USEPA, 1996b).
3.1.2 Subcommittee Findings as They Relate to the Objectives
With respect to Congressional requirements, the SITE program has matured in
response to its broadly defined mission, as directed in SARA, into an operationally
productive and impressive activity. SITE collaborates with and conducts research on a
variety of technologies with other agencies, including the Departments of Commerce,
Defense, Energy, and Interior, various states (including Arizona, Florida, and New
York), and the private sector (USEPA, 1996b). From 1987 to 1994, SITE co-funded 21
emerging technology projects with DOE and 8 with DOD, conducted three
demonstrations with New York State, and developed 72 cooperative agreements with
the private sector (USEPA, 1995a and USEPA, 1996b). SITE has an annual
solicitation for projects and reports annually to Congress as required by SARA
(USEPA, 1995a).
SITE'S Demonstration Program prepares demonstration plans, and conducts and
evaluates demonstrations for ten innovative location cleanup technologies each year.
Subcommittee members are aware from their professional experience that technologies
8
-------
demonstrated in the SITE program are being utilized in response actions to remediate
contaminated locations. Based on these findings, the Subcommittee concludes that the
SITE program has been responsive to the objectives set for it by Congress.
With respect to achieving the objectives of SITE'S mission statement, the
Emerging Technology Program funds research efforts at the bench and pilot-scale
levels for the development of innovative technologies. Of participants in the emerging
technology program, 60% are pursuing commercialization and almost a quarter had
buyers for the technology under development (USEPA, 1995a). The Demonstration
Program conducts and evaluates demonstration of innovative technologies; 117
technologies had been accepted into the program by the end of FY94 (USEPA, 1995a).
The Monitoring and Measurement Technologies Program (MMTP) addresses methods
for detection and monitoring of contaminants and the geophysical characteristics of
Superfund locations. It provides developers with the opportunity for a rigorous
evaluation of their technology's performance; evaluations or demonstrations for 27
technologies have been completed (USEPA, 1994). The Technology Transfer Program
(TTP) undertakes technology transfer activities and provides technical assistance to
increase commercialization of innovative hazardous waste clean-up technologies. Not
all of its work is written; the TTP holds on-location Visitor's Days for local communities
and prepares demonstration videotapes. The TTP made available databases on
treatment technology, treatability studies, underground storage tanks, and spills of
chemicals or oil. In addition to the Annual Report to Congress, the TTP produces SITE
Technology Profiles, Demonstration Bulletins, and Success Stories (Superfund at
Work). SITE personnel have also published in the peer reviewed literature and
provided statistics on the ETP, DP, MMTP, and TT achievements. In FY94, the TTP
distributed almost 325,000 copies of publications to requesters (USEPA, 1995a).
The use of SITE demonstrated technologies for remediation, coupled with
positive statements by developers, implies a positive effect of the program on
commercialization. Three successful developers are quoted in The Annual Report to
Congress 1994 (USEPA, 1995a).
SITE has generally been successful in achieving its mission, "to increase
commercialization." To achieve commercialization of an innovative technology, three
items must exist: a) a market for the technology, b) scientific data verifying the
technology, and c) successful case studies from actual technology applications.
However, some elements of the SITE program create barriers to commercialization in
some situations. Other barriers or needed improvements include: improved timeliness
of reports, better cost estimates, improved performance data interpretation, simplicity
-------
and clarity in summarizing results, and evaluations that aid potential technology users
in determining whether or not a technology is cost-effective in their situation.
Through the SITE program, the Agency has enabled technology innovators to
test and demonstrate technologies under strict protocols and QA/QC requirements,
thereby providing objective performance information. The SITE program has collected
extensive performance data on numerous promising innovative technologies.
The resulting SITE reports have generally delivered reliable scientific
information but limited cost information on the performance of technologies. Most of
the technical reports well describe the technologies and information generated during
testing. In the Subcommittee's experience, successful demonstration of a remediation
technology does facilitate acceptance by regulators, because they tend to support use
of demonstrated technologies.
The Subcommittee found that SITE does make use of the best attributes of both
the Federal and private sector. The government provides credibility, experience in the
evaluation of remediation technologies, access to test locations, visibility, and
technology transfer. The private sector provides creativity, funding, equipment and
related operations. In some sense, SITE is to the user community as Consumer
Reports is to the ordinary consumer. This is a very valuable service to the private
sector.
To measure how much SITE has reduced the time to get new technologies on
line, one would need to know how long it would take similar technologies without SITE.
This is not known. The Subcommittee cannot quantify how much SITE has accelerated
the commercialization process.
However, there are several good reasons to infer that SITE has reduced the time
it takes to get new technologies on line. First, because SITE program projects are
exempt from permits, the SITE program removes the delays associated with obtaining
permits. Second, SITE'S ET program "feeds the pipeline" by encouraging new ideas for
technology to make their way through the "Valley of Death." "The "Valley of Death" for
environmental technology development occurs when the federal funding program
considers a technology 'too applied' for additional funding and industry considers the
technology 'too embryonic' to adopt" (NETAC, 1995). Third, participants have stated
that SITE had assisted them in commercialization-and speed is a critical issue in
commercialization (USEPA, 1995a). Fourth, the SITE program has helped create high
interest in new or practical technologies by users and vendors. Staff within state
agencies as well as within EPA are aware of the technologies being tested in SITE and
10
-------
the results; this may reduce the time and difficulty in getting a technology approved at a
different location.
3.1.3 Findings Relating to Unstated Objectives
In terms of the scientific foundations, the SITE program has appropriately
identified the critical phases of technology development from research to market
penetration and commercialization. These are:
a) Technology research,
b) Pilot development,
c) Field demonstration,
d) Information and Technology Transfer, and
e) Market Penetration/Commercialization (USEPA, 1995a).
The program's research quality is attested to by both the QA/QC documentation
and the acceptance of papers in the peer-reviewed literature (USEPA, 1995a). The
latter is also one of many indicators of relevance. SITE has already been used on at
least one occasion, by the EnTICE program, as a model for other organizations
attempting to evaluate and demonstrate innovative technologies.
With respect to communicating the results and benefits of the SITE program,
there is room for further improvement. Such communication is important for two
reasons. In the Subcommittee's view, SITE is regulatory science-and regulatory
science must be communicated clearly not only to the professional community, but to
other users as well. Also, in the Subcommittee's view, SITE is a relevant and quality
research program worthy of continuation; it is worrisome that some decision makers do
not appear to be aware of SITE'S achievements.
The annual Reports to Congress and Associated Technical Profiles have
recorded the growth and development of SITE to a multi-faceted program which
included 77 emerging technologies, 124 demonstrations, 30 monitoring and
measurement methods, and an extensive technology transfer effort. Almost 325,000
copies of documents were distributed in FY94 alone (USEPA, 1996a; 1995b). Despite
the variety, quantity, and wide distribution of technology transfer products (from on-
location Visitor's Days and videos to a dozen different types of written products), the
11
-------
Subcommittee perceives the availability of such disclosures as somewhat random,
mechanically delivered, and without a concerted effort to highlight those feedback
mechanisms that would serve to sustain and elevate the program to even greater levels
of achievements.
Despite SITE'S communication efforts, it is unfortunately true that SITE'S
accomplishments over this 10-year period have not been as vividly and universally
acknowledged as they may have merited. Indeed, some of the criticism of SITE during
this period may result from the failure to communicate in a clear but scientifically sound
and technically decisive manner, with metrics that indelibly quantify successes as well
as failures. Hence, some skepticism has arisen which brings to question the overall
effectiveness of the program in terms of impact and return on investment.
A more effective mechanism to highlight accomplishments in targeted user
groups could include a more vivid identification of the stakeholder constituencies, their
needs, and how value is tangibly added by the SITE program. Networking with these
beneficiaries should ensure vitality consistent with needs and offer opportunities for
restructuring and refining goals and strategies as the next evolution of SITE unfolds.
3.1.4 Developing More Demanding Objectives
The SITE program has met the original objectives and has developed methods
for evaluating individual projects. Hence, the Subcommittee generally views the
program as a success with an opportunity to grow into an even more productive entity.
It is now time for the program to develop more substantive objectives with which to
measure the progress of the program. Metrics are needed for the various objectives to
evaluate the program's very real contribution to remediation and technology cultivation.
Discussions between SITE program staff and customers could better define the
products needed and the methods of delivery. The Subcommittee believes that such
discussions will identify interpretive analyses as highly useful to the customers.
The SITE program staff already counts the number and variety of technologies
addressed in the subprograms, the progress of technologies to commercialization in
the United States and abroad, the projects co-funded with others, and the distribution of
technology transfer products. For purposes of illustration, additional metrics might
include: a tabulation of the technologies that have participated in SITE, the
subsequent use of those SITE technologies in remediation (or elsewhere) and the
number of locations where they were used; some measures of reduced costs for
remediation, restoration, measurement and monitoring; and documentation of cleanups
that are cheaper, faster, and better because of SITE.
12
-------
Because SITE has generally met the past objectives set for it, the Subcommittee
finds it appropriate to "raise the bar" and hold the program to higher standards. From
this new perspective, the Subcommittee views the lack of such objectives (and metrics)
as an overall program weakness that jeopardizes the desired stability and growth of the
program. Because the SITE program staff have made steady improvements to the
program over the last decade, the Subcommittee is confident that they can strengthen
the program by developing better objectives and metrics in cooperation with their
customers.
The Subcommittee suggests that individual technology evaluations could be
improved by the inclusion of consistent cost information, use of a success matrix, and
more rigorous mass balances.
3.2 Scientific/Technical Aspects and Implementation
In its review of the SITE component programs, the Subcommittee gave the
greatest attention to the Demonstration Program and less attention to the lower-funded
Emerging Technology Program and the Monitoring and Measurement Technology
Program. The proportional funding for DP, ETP and MMTP is, respectively, 28:3:1,
when NRMRL staffing costs are excluded. (The funding for Technology Transfer which
is responsible for dissemination of information regarding DP, ETP and MMTP, is
incorporated into the funding of the other component programs).
3.2.1 Project Planning
SARA requires the Agency to supervise demonstrations and provide quality
assurance for associated data. The SITE planning process includes the developers,
support contractors, Agency personnel, and other stakeholders as participants. To
meet its responsibilities, the Agency employs a Data Quality Objective planning
process that defines critical elements of the planning process and documents them in a
Project Objective Agreement (POA). Planning details are further described in
Implementation Agreements and Quality Assurance Project Plans (QAPP). All
stakeholders, including the Agency or their contractors and those responsible for data
production, sign these plans and agreements .
The Subcommittee reviewed examples of recent planning documents and noted
a level of detail that displays an understanding of the innovative technologies and
proper technology evaluation (USDOE, 1995; USEPA, 1995b; PRC, 1994). The plans
recognize the potential impact of soil, waste, and residual heterogeneity upon precision
and levels of uncertainty. Implementation of these plans should result in data of known
13
-------
quality suitable for technology evaluation. The reviewed documents confirmed the
general consensus of the Subcommittee that SITE addresses most of the technical and
scientific aspects that are key to the evaluation of innovative technology.
NETAC has identified several technical barriers to environmental technology
commercialization. They are inter-related and include: lack of acceptable
demonstration locations; difficulties associated with technical performance and
engineering for scale-up; lack of real performance and cost data, lack of methods to
measure success, lack of site characterization data; and the complexity of both wastes
and contaminated sites (NETAC, 1995). With the exception of the difficulties
associated with technical performance and engineering for scale-up, these barriers to
commercialization are also barriers to evaluation.
The Subcommittee has made suggestions regarding mass balance, cost
comparability and interpretative analysis; if SITE is continued, then planning of future
SITE evaluations and demonstrations should be expanded to accommodate these
features.
3.2.2 Quality Assurance and Quality Control
The Agency's planning process assigns the level of quality assurance according
to the end use of the scientific and technical data. A series of Agency publications
document this structured project planning process (USEPA, 1991a-d). The Agency
employed Level II quality assurance planning for Demonstration Program projects
because data generated by these projects are of high visibility and could be used for
making policy decisions. The Agency implemented a slightly less demanding planning
process (Level III) for Emerging Technology Program projects because the associated
data are used to determine the feasibility of emerging technologies. MMTP projects are
planned according to Agency guidance (USEPA, 1995c).
The Subcommittee reviewed examples of DP, ETP and MMTP Quality
Assurance Project Plans (QAPP), field and laboratory audit reports and corrective
actions. The audit reports, which are referred to as Technical System Reviews, detail
compliance with the QAPP and identify concerns and technical comments. Technical
system reviews are scheduled at the beginning of sampling and analytical efforts to
ensure that concerns are identified and corrected early in the project. The QAPPs
describe the types and frequency of quality control samples, the quality assurance
organization, quality indicators, data handling, management oversight as well as the
appropriate sampling and analytical protocols.
14
-------
In response to an independent management assessment, more recent projects
identify and document their objectives in project POAs. POAs detail quantitative goals
and acceptable levels of decision error. The SITE program recognizes QAPPS and the
issues addressed in POAs are key to project QA/QC and that these planning
documents are in great part responsible for the high quality of sampling and analytical
data now associated with the SITE program.
3.2.3 Interpretative Analysis
Writing from the perspective of science and engineering, the Subcommittee finds
that the usefulness of the SITE program could be improved by the addition of
interpretive analyses, either to the Innovative Technology Evaluation Reports (ITER) or
as stand-alone summary reports. While decision-makers must consider many factors
in addition to science when making decisions about programs and their products, the
Subcommittee believes that such analyses would be very useful to consumers and the
scientific community.
Over the last decade the SITE program has generated significant quantities of
data from DP, ETP and MMTP evaluations. Due to time and costs constraints, these
data cannot capture the effect that all environmental, waste or operating conditions
may have on a technology's efficacy. However, the Subcommittee infers from the
depth, breadth, and quality of its discussions with the SITE program staff at the public
meeting that the program staff have a more comprehensive understanding of the
limitations and the applicability of the evaluated technologies than that which is
routinely documented in Demonstration Program ITERs.
Section 311.(b)(5)(A)(v) of the Comprehensive Environmental Response,
Compensation and Liability Act (CERCLA) requires the Agency to establish a
demonstration program that includes "The evaluation of the results of the alternative
innovative technology demonstration projects and the determination of whether or not
the technologies used are effective and feasible." A discussion of the limitations and
potential applications of a technology including a discussion of the applicable and non-
applicable matrices, wastes, contaminants and operational conditions appears to fall
within this mandate. The Subcommittee believes the Agency can comply with this
aspect of the law within the limitations of the data, and balance this need for
interpretation without displaying bias for a technology. These interpretations and the
bases for these interpretations should be subjected to the Agency's quality assurance
review. While the interpretive analyses may not be a measure of absolute success,
they would at least provide a comparative measure. This interpretive analysis can be
15
-------
of significant help to developers by guiding future development, and to the potential
users who are searching for an appropriate technology.
3.2.4 Mass Balances
Evaluation of innovative technologies requires that destruction or removal
efficiencies be calculated for the contaminants of concern. It is often difficult to
accurately balance inputs and outputs. Heterogeneity of influent, pre-treatment, post-
treatment and in-situ materials are impediments to accurate mass balances, as are the
complexity of the treatment system and the potential presence of sinks and leaks in
treatment equipment. Testing the system for a longer time reduces these difficulties,
but may also add to the cost of the evaluation.
Notwithstanding these limitations, mass balances providing a high degree of
closure are generally expected to determine a technology's performance. During the
public meeting, the Agency made the point that mass balances of 80% or greater are
not always achieved. This problem is not unique to the SITE program. Because staff
understand the difficulties involved and possible solutions, more deliberately sharing
their experiences might benefit the wider technical community. This could be achieved
through publications, a workshop, or some other means.
3.2.5 Coordination of MMTP Activities
Because of the relative expense of evaluating innovative treatment technologies,
the Monitoring and Measurements Technology Program (MMTP) has been funded at a
lower level than the DP and ETP programs. Nonetheless, MMTP is an important part of
the SITE program because characterization and monitoring costs are often a
substantial percentage of remediation costs.
The MMTP is implemented by the National Exposure Research Laboratory's
(NERL), Characterization Research Division located in Las Vegas, NV. Due to its
expertise in monitoring and measurement technology, NERL has also attracted funding
from the Environmental Technology Initiative (ETI) and the Strategic Environmental
Research and Development Program (SERDP). NERL has field-tested characterization
and monitoring technologies and has assessed their applicability and limitations.
The Subcommittee encourages SITE staff to continue to identify those
characterization and monitoring needs important to the Superfund program, and to
work closely with NERL to maintain its integration within the SITE program while
16
-------
leveraging the MMTP to meet the needs of Superfund and other related multi-agency
efforts.
3.3 Impacts and Improvements
First, SITE has developed and tested technologies which have resulted in a
positive impact on the nation's location remediation efforts. SITE has facilitated the
scale-up of some technologies to field trials or demonstrations-a major hurdle in any
development effort. The SITE program has subjected many of the evaluated
technologies to more extensive and rigorous testing and review than is often conducted
by a developer. This permits better decision-making by consumers.
Second, SITE has interested users and vendors in new, improved, or more
practical technologies. By participating in SITE, a new technology or improvement
gains credibility and exposure. Staff within EPA and state agencies are aware of what
technologies SITE is funding, and technical consultants learn from their government
contacts. SITE circulates reports and bulletins to a wide audience, and most of the
technologies that received SITE support are in use. In 1992, more than half of
Superfund Records of Decision (RODs) used innovative technologies versus
conventional technologies (USEPA, 1996b).
Third, an analysis of cost savings using innovative technologies for 17 sites in
four EPA regions, indicates that the innovative technologies saved five to two hundred
million dollars per region (USEPA, 1996b). While data are not available on enough
locations to quantify the total cost savings due to SITE, SITE has reduced technology
costs for site characterization and remediation and has the opportunity to have further
positive impact. SITE financial support, during development and scale-up, reduces the
financial burden on the technology developer. This saving reduces pass-on costs to
first practical applications. When cost reporting for a demonstration project is complete
and credible, then potential technology users will be able to examine base rate
financial data and costs will become less susceptible to vendor manipulation. This, too,
should lead to cost reductions.
It is the Subcommittee's understanding that by reducing barriers to
commercialization of new technologies, collecting and reporting credible and complete
cost and performance data, and including interpretive analyses easily understood by
those who select technologies for location characterization and remediation, the SITE
program will reduce government expenditures at Superfund locations and, in so doing,
reduce costs to the taxpayers.
17
-------
The Subcommittee wishes to emphasize that one of the most critical
expectations of the SITE program is provision of a fair and consistent basis for
evaluation of results and the comparison of competing technologies-to be a
"Consumer Reports" for site remediation. Therefore, one of the most valuable
advancements SITE could make is to improve the consistency and completeness of the
objective measures for evaluation and inter-comparison. Although, SITE has
developed some criteria for comparison, the program currently refrains from making
comparative statements regarding technology success, problems, or failure. And, of
course, not all technologies participate in SITE. As a result, the potential user is still
somewhat vulnerable to marketing claims when making a decision whether or not to
employ a particular technology or to choose between technologies.
3.4 Technology Commercialization
The SITE program mission includes facilitating the commercialization of
innovative technologies. Commercialization can be defined two ways, with most views
on commercialization falling somewhere between the two. Industry usually defines
commercialization as turning a profit, but the term can also be defined in the
environmental arena as commercial scale. Commercial-scale efforts provide reliable
performance, cost, and applicability information to technology developers, industry, and
the public. These data become crucial in enabling industry to market technologies.
Thus, the two definitions are actually integrated and form the basis of the term
"commercialization" in this report.
3.4.1 Factors Affecting Commercialization
To achieve commercialization of an innovative technology, three items must
exist: a) a market for the technology; b) scientific data verifying the technology
performance; and c) successful case studies from actual technology applications.
While SITE generally facilitates commercialization, some elements of the SITE program
create barriers to commercialization. Other needed improvements include: improved
timeliness of reports, better cost estimates, improved performance data interpretation,
simplicity and clarity in summarizing results, and evaluations that aid potential
technology users in determining whether or not a technology is cost-effective in their
situation.
Of the barriers, the biggest, and most important is time. Because of the
numerous and complex barriers associated with environmental technology
development, investors often lack confidence that they will receive an adequate and
timely return on their investment (NETAC, 1995). Often developers rely on the SITE
18
-------
report as a marketing tool. Unfortunately, at times SITE has experienced significant
delays in the review and publishing of reports. Without significant internal financial
resources or outside investors, a small firm may perish in the time it takes the SITE
program to produce a report.
Four problems can limit the ability of the SITE program to gather cost data and
develop useful projections for demonstrated technologies, which are critical in
technology commercialization (Evans, 1990).
a) Each field demonstration represents a mix of unique factors.
Location-specific conditions such as type of media and location geology
largely determine the cost of applying a particular technology at a given
location. Such variability often makes it difficult to translate costs to other
locations having different conditions.
b) The research and development aspects of each demonstration impacts
observed costs. Specifically, this problem relates to the one-time costs
that occur when first demonstrating a technology (e.g., unplanned
equipment modifications, shakedown problems).
c) Each developer is a profit maximizer operating within a competitive
marketplace. By law, all data collected during a demonstration shall be
made available to the public except trade secrets or other proprietary
information. The confidentiality assigned to proprietary information
protects the vendor from having to share information that may affect their
marketing and price competitiveness, but it also makes it more difficult to
develop cost projections.
d) The SITE program creates unique interactions, such as exemptions from
permits, between the public and private sectors. The unique nature of
these interactions may result in a cost under- or overestimate, which
dramatically impacts the market and all stakeholders.
19
-------
To address these concerns, the SITE program developed the following five rules
to govern each SITE cost analysis (Evans, 1990).
a) Provide full disclosure of all assumptions and calculations used in the
base-case analysis.
b) Place each base-case cost analysis within a common framework of 12
cost categories listed in Table 1.
c) Present each base-case cost projection as "order of magnitude" estimates
(+50% and -30%).
d) Identify key operating parameters that are likely to have significant cost
implications beyond the base-case.
e) Offer developers the opportunity to present their own cost analysis.
Table 1
Cost Categories for SITE Base-Case Analyses
1. Site preparation
2. Permitting and regulatory requirements
3. Capital equipment
4. Startup
5. Labor
6. Consumables and supplies
7. Utilities
8. Effluent treatment and disposal
9. Residuals/waste shipping and handling
10. Analytical services
11. Maintenance and modifications
12. Demobilization
(Adapted from Evans, 1990)
The SITE program has assembled vendor supplied cost data for technologies to
aid in commercialization. However, no rigorous evaluation is conducted to investigate
20
-------
how the vendors generated the cost data. To support technology commercialization,
cost data must be presented in a consistent and defensible manner.
3.4.2 Timeline
Long periods for evaluation and reporting are a barrier to commercialization.
The time the SITE program takes for an evaluation varies from one technology to
another, and the time taken to document and report the results also varies. In an
extreme case, a lag time of one year occurred between when the ITER report was final
and when it was published. Smaller companies cannot financially survive the average
three years that it takes to participate in the SITE program.
3.4.3 Technology Transfer
The SITE program has made a conscious effort to provide a variety of users with
information about the program and the technologies evaluated by it. The Agency's
Center for Environmental Research Information (CERI) has distributed almost 325,000
copies of reports documenting innovative technologies in the SITE program (EPA,
1995a). Users include consultants, state and local governments, the Agency, other
federal organizations, universities, industries, and private citizens groups. This
distribution of knowledge provides an informed basis for discussion among users and
technology developers, which, in turn, can mean potential commercialization at other
locations. SITE participants have found the SITE program to be a good marketing tool
that increases visibility. Several stated that SITE was the single most important aspect
of commercializing their technology. In fact, one stated that the technology would not
have become commercial if it had not been for the SITE program.
However, the transfer of knowledge from the SITE program to potential users
outside of the Superfund arena and other government agencies can be improved.
Disagreements between agencies regarding evaluation protocols sometimes result in
lack of commercialization of the technology, despite a favorable report from the SITE
program. If the SITE program proves the technology is effective (through results), then
other government agencies (e.g., DOD) should readily accept these results as valid.
This approach would not only promote commercialization of the technology, it would
save other agencies time and money in testing the technology. A report expected from
the National Research Council's Committee on Innovative Remediation Technologies
in 1997 may be helpful. (National Research Council, in preparation) The report is
expected to address evaluation of protocols for commercializing innovative remediation
technologies for ground-water clean-up and to suggest standard protocols for testing,
evaluating, and verifying these technologies.
21
-------
3.4.4 Market Determination
In the view of private sector environmental firms and potential investors, the
development and demonstration of innovative technology is secondary to the
development of actual markets for these technologies (Commercialization Roundtable,
1993). The NETAC report makes the same point, (p vii)
"... the investment community seeks opportunities for high-growth
businesses, not in technologies. As a result, 'seasoned' entrepreneurial
management which can offer critical business or marketing skills is
essential to the successful development of environmental technologies.
Typically, the developer's emphasis on research is too great, with too little
emphasis on commercialization or technology applications " (NETAC,
1995).
Therefore, to ensure that each demonstration has a purpose, the SITE program
may wish to require that applicants demonstrate in writing the existence of a potential
market prior to selection or it may prove to be just a waste of effort and financial
resources of the involved parties including the Agency.
3.4.5 Permitting
SITE program projects enjoy certain advantages. The Comprehensive
Environmental Response, Compensation and Liability Act (CERCLA) Section 311 (b)
exempts SITE program projects from permits. Also, the projects are covered under
CERCLA Section 107 (b) for liability. Thus, the SITE program inherently removes
many of the barriers associated with obtaining permits (e.g., lengthy delays before
beginning work and costly insurance premiums).
3.4.6 Conclusions About Commercialization
The SITE program has helped a number of technologies become
commercialized. However, the time needed to implement SITE'S procedures and
restrictions has also hindered others from being successful in the commercial market.
Other barriers or needed improvements include: improved timeliness of reports, better
cost estimates, improved performance data interpretation, simplicity and clarity in
summarizing results, and evaluations that aid potential technology users in determining
whether or not a technology is cost-effective in their situation.
22
-------
The SITE program's success in developing and demonstrating innovative
technologies is partly due to the focused strategies and approaches EPA developed to
achieve those ends. However, commercialization brings with it some inherent
difficulties, and EPA's role in the commercialization aspect of the SITE program mission
is unclear. It may be that SITE'S role is simply to provide a unique opportunity to
generate scientifically defensible cost and performance data that allows industry to
simplify its commercialization efforts. In any case, without a clear strategy and
systematic approach, it will be difficult for EPA to focus on commercialization or to
measure success in this area.
3.5 Future Direction
Remediation is a big, expensive problem that won't be solved quickly. The
annual market for remediation in the U.S. alone approaches $10 billion, and decades
will be needed to clean up those locations already identified (NETAC, 1995). Improved
technology lowers costs, improves performance, and speeds results. Therefore, the
Subcommittee finds the need for continuing the SITE program- and for continuing to
improve it-compelling.
SITE can be even bigger and better if the totality of its experiences are
objectively reviewed and measured not only in terms of numbers of "success stories,"
but in terms of a prospective assessment of the needs, directions, and implementing
tactics of today and tomorrow. Mission and objectives should be rethought within a
contemporary perspective, including the possibility of eventually including a
verification/certification element as does the EnTICE program (SAB, 1995).
SITE should embrace the contributions of other complementary activities. Some
examples are: the upcoming report of the National Research Council's Committee on
Innovative Remediation Technologies (NRC, in preparation), the Advanced Applied
Technology Demonstration Facility for Environmental Technology Development at Rice
University, and the DOD Matrix (USEPA/USAF, 1993).
Such an exercise should help to enlist support outside the Agency, and provide
a more convincing argument why SITE should be continued as an EPA-directed
program.
23
-------
4. RECOMMENDATIONS
These recommendations are made from the perspective of scientists and
engineers familiar with environmental problems and methods used to reduce or
eliminate them. Economic, policy, or other considerations could lead a decision-maker
to other conclusions.
Continue SITE: The need for improved technology to prevent, reduce or
remediate environmental contamination has not disappeared. Because that need
remains, and because the SITE program has been successful, the Subcommittee
recommends that the Agency build upon that success either by continuing the existing
program or by creating a different technology development and demonstration program
built upon the precepts of the current program.
Develop a plan: To aid in selecting the most appropriate course of action, the
Agency should develop a plan that defines the desired products (including the
adequacy of the reportable data), the customer base, and how the two will be best
brought together. Better product definition and improved marketing-including the clear
portrayal of the successes and benefits of SITE-will make the program better
understood by and more valuable to the user community. The EEC's "Strategic
Research Planning Commentary" discusses the benefits of such planning and some
approaches to it (SAB, 1994).
The Subcommittee recommends that SITE retain not only the Demonstration
Program, but also the other three subprograms. The Emerging Technology portion of
the program helps to supply the next generation of demonstrations and a primary
means to identify or open new horizons or technical areas. The program should follow-
up on technologies tested to assure improvements recommended get built into
technologies thereby leading to their wider use and the collection of data on their
subsequent performance at additional locations or conditions and associated costs.
Some attention should also be given to minimizing the appearance of conflict of interest
when emerging technologies developed by EPA are evaluated by EPA.
The MMTP should be continued, with SITE management identifying
characterization and monitoring needs important to Superfund and by working closely
with NERL to maintain its integration within the SITE program.
Develop Metrics: Once the plan has been developed, the Subcommittee
recommends that additional program-wide quality metrics be identified. The purpose of
24
-------
such measures may to quantify to what extent SITE has helped commercialization of
demonstrated technologies; document reduced remediation costs; follow-up on
technologies evaluated to assure that recommended improvements are widely
implemented; and document the use of investigation and monitoring techniques
evaluated by SITE.
The measures should be tracked from year to year, and a formal mechanism for
identifying and documenting common problems and successful corrective actions
should be implemented. For example, the proceedings of the April 6, 1995 SITE QA
meeting, contain valuable information regarding program quality (USEPA, 1995d).
Formalization of these types of meetings, their proceedings and documentation of
follow-up actions and quality metrics would allow for the identification of trends while
chronicling and documenting quality for the overall SITE program.
For purposes of illustration, such metrics might be: a tabulation of the
technologies that have been evaluated through SITE, the subsequent use of those
SITE technologies in remediation (or elsewhere) and the number of locations where
used; more complete measures of reduced costs for remediation, restoration,
measurement and monitoring; documentation of cleanups that are cheaper, faster, and
better because of SITE.
Increase Marketing: The Subcommittee recommends increased attention to
three aspects of marketing; selection of technologies for demonstrations, improved
support to SITE customers, and developing support for SITE.
In selecting technologies for demonstration, SITE should require that the vendor
provide evidence of a potential market to ensure that the demonstration will be useful
and the technology will have a market.
In the course of planning improvements to SITE, the staff should develop a
better understanding of who uses the SITE program and its products, sharpen product
definition, and consider how each product may best be delivered to its customers. The
Subcommittee expects that timeliness will be of special importance to developers who
plan to use the SITE report in marketing, and that interpretive analyses will be
important to those who select technologies for location remediation.
Finally, in spite of unstable financial support in recent years, the SITE program
has managed to achieve much more than is recognized by many in the user
community. Clearly stating the program's contribution to commercialization of
demonstrated technologies and to reduced costs of remediation and restoration should
25
-------
help SITE gain the recognition it deserves. Funding stability for the program and
additional opportunities for leveraging resources with other organizations may result.
Develop Success Matrix: The Subcommittee suggests that the concept of
collecting comparative cost data be expanded into a success matrix which reports on
both technical parameters and cost. For example, success in meeting treatment or
capacity goals, reports on the operation history or maintenance needs, identification of
optimal operating ranges or conditions, identification of inappropriate applications or
conditions, reports of problems encountered, or other performance criteria would help
in making a decision to apply a technology.
The Federal Remediation Technologies Roundtable has published a Guide to
Documenting Cost and Performance for Remediation Projects (US Federal Roundtable,
1995). The Guide was written to encourage the use of consistent procedures to
document cost and performance information for projects involving treatment of
contaminated media. While this is not the same task as research to evaluate
innovative technologies, it may be useful in structuring a success matrix for SITE.
Provide interpretive analyses and develop data for intercomparison: Inter-
comparison of evaluations is critical. The SITE program funded many often competitive
projects (for example several thermal desorption technologies and a few luminescence
field investigation techniques), but insufficient objective information was presented to
provide a basis for comparison. Nonetheless, report documents are used to make
technology selection decisions at contaminated sites. The data and information
reported by the program needs to be sufficient to help support these decisions. The
Subcommittee therefore recommends that the program also prepare interpretive
documents that summarize, compare, extend, and extrapolate the results of the
individual evaluations.
Increase the rigor of cost analyses and improve consistency and completeness
of cost data: SITE must improve the collection of cost data. A rigorous evaluation of
cost data should be conducted, and cost data should be presented in a way to evaluate
the technology's engineering value.
The development of the twelve categories for cost reporting was good in concept
but weakly supported or filled out inconsistently thus limiting its value as a tool for
evaluation or inter-comparison. The data need to be reported on some consistent
basis not just total or absolute cost so that inter-comparisons are possible. The data
support calculations and their bases should also be provided.
26
-------
Improve Mass Balances: The ITERS should address the issue of mass
balances, when appropriate, in an interpretive section. If the Agency believes it is not
always possible to achieve mass balances under the constraints of the SITE program,
the rationale for this shortcoming, such as abbreviated test runs or limited sampling of
heterogeneous materials, should be explored and discussed to educate the technical
community. Given the Agency's stated concerns about trans-media migration of
hazardous constituents, SITE should strive for material balances providing a high
degree of closure for hazardous constituents.
Improve Report Clarity: In the reports, vendor claims should be separated from
the reporting of project results and include an Agency or third party reviewer analysis of
technology effectiveness. Reports should provide all the information gathered to help
the user community in decision making.
Decrease Final Report Production Times: Document production should be
accelerated to provide timely information to the marketplace.
27
-------
REFERENCES CITED
Commercialization Roundtable. 1993. Final Report: DOIT.
Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA),
PL-9-510 December 11, 1980.
Evans, G.M., 1990. Estimating Innovative Technology Costs for the SITE Program,
Journal of Air and Waste Management Association, Vol. 40, No. 7, pp. 1047-1051.
NETAC. 1995. Barriers to Commercialization, 400-5408-00, National Environmental
Technology Applications Center. Pittsburgh, PA.
National Research Council, in preparation, Evaluation Protocols for Commercializing
Innovative Remediation Technologies. Committee on Innovative Remediation
Technologies, Washington, DC, National Academy Press.
PRC. 1994. Draft Demonstration Plan for the Evaluation of Field Portable X-Ray
Fluorescence Technologies. Prepared for USEPA/EMSL in partial fulfillment of
Contract No. 68-CO-0047, Work Assignment No. 0-65. PRC Environmental
Management Inc.
SAB. 1985. Superfund Resolution. EPA-SAB-EEC-85-024. USEPA, Science Advisory
Board. Washington, DC
SAB. 1986. Superfund Innovative Technology Evaluation (SITE) Program Draft
Strategy and Program Plan. EPA-SAB-86-017. USEPA, Science Advisory Board.
Washington, DC
SAB. 1994. Commentary on Strategic Research and Development Planning. EPA-
SAB-EEC-COM-94-004. USEPA, Science Advisory Board. Washington, DC
SAB. 1995. EPA's Environmental Technology Innovation and Commercialization
Enhancement Program (EnTICE). EPA-SAB-EEC-95-016. USEPA, Science Advisory
Board. Washington, DC
Superfund Amendments and Reauthorization Act of 1986 (SARA), PL-99-499 October
17, 1986.
R-1
-------
USDOE. 1995. Matrix Photocatalytic Oxidation Technology Demonstration, Final
Quality Assurance Project Plan. U.S. Department of Energy. Oak Ridge Reservation,
Oak Ridge, TN.
USEPA. 1991 a. Preparation Aids for the Development of Category I Quality Assurance
Project Plans. EPA/600/8-91/003. USEPA, Office of Research and Development,
Washington, DC.
USEPA. 1991b. Preparation Aids for the Development of Category II Quality
Assurance Project Plans. EPA/600/8-91/004. USEPA, Office of Research and
Development, Washington, DC.
USEPA. 1991c. Preparation Aids for the Development of Category III Quality
Assurance Project Plans. EPA/600/8-91/005. USEPA, Office of Research and
Development, Washington, DC.
USEPA. 1991d. Preparation Aids for the Development of Category IV Quality
Assurance Project Plans. EPA/600/8-91/006. USEPA, Office of Research and
Development, Washington, DC.
USEPA/USAF. 1993. Remediation Technologies Screening Matrix and Reference
Guide. EPA/542-B-93-005
USEPA. 1994. Superfund Innovative Technology Evaluation Program, Technology
Profiles. Seventh Edition. EPA/540/R-94/526. USEPA, Washington, DC.
USEPA, 1995a. The Superfund Innovative Technology Evaluation Program, Annual
report to Congress. FY1994. EPA/540/R-95/522. USEPA, Washington, DC.
USEPA, 1995b. Project Objective Agreement for General Environmental Corporation,
CURE Electrocoagulation Technology at the Rocky Flats Technology Site. USEPA,
Office of Research and Development Risk Reduction Engineering Laboratory (RREL)
USEPA, 1995c. A Guidance Manual for the Preparation of Site Characterization
Technology Demonstration Plans. USEPA, Office of Research and Development
National Exposure Research Laboratory.
USEPA, 1995d. Record of Meeting for the SITE QA Meeting of April 6, 1995
R-2
-------
USEPA, 1996a. Draft overheads mailed from the National Risk Management Research
Laboratory in advance of the SITE review.
USEPA, 1996b. Science Advisory Board Review Meeting for Site Program, distributed
at the June 11-12, 1996 meeting.
US Federal Roundtable. 1995. Guide to Documenting Cost and Performance for
Remediation Projects. EPA-542-B-95-002. US Federal Remediation Technologies
Roundtable.
R-3
-------
APPENDIX A - ACRONYMS AND ABBREVIATIONS
CERCLA Comprehensive Environmental Response, Compensation and Liability Act
CERI Center for Environmental Research Information
DOD Department of Defense
DOE Department of Energy
DP Demonstration Program
EEC Environmental Engineering Committee
EnTICE EPA's Environmental Technology Innovation and Commercialization
Enhancement Program
ETI Environmental Technology Initiative
ETP Emerging Technologies Program
ITER Innovative Technology Evaluation Report
MMTP Monitoring and Measurements Technologies Program
NERL National Exposure Research Laboratory
NRMRL National Risk Management Research Laboratory
ORD Office of Research and Development
OSW Office of Solid Waste
OSWER Office of Solid Waste and Emergency Response
POA Project Objective Agreement
QA Quality Assurance
QAPP Quality Assurance Program Plans
QC Quality Control
RCRA Resource Conservation and Recovery Act
SAB Science Advisory Board
SARA Superfund Amendments and Reauthorization Act
SERDP Strategic Environmental Research and Development Program
SITE Superfund Innovative Technology Evaluation
TTP Technology Transfer Program
A-1
-------
APPENDIX B - DOCUMENTS REVIEWED
The Following Review and Background/Briefing Materials Were Provided to the
Subcommittee and are listed in the Federal Advisory Committee Act file for this review.
I. May 3, 1996 letter from SAB staff Re: Site Review
II. May 3, 1996 letter from SAB staff Re: Practicalities of SITE Meeting June 11-13,
1996, including:
A. A project description which includes the charge for the review.
B. A draft agenda for the meeting
C. A detailed roster with phones and faxes
D. An "affiliation" roster for use in the final report
E. A copy of the Science Advisory Board's 1986 report on the draft strategy
and program plan for the SITE review.
F. A copy of the Agency's response to that report
III. Materials mailed from the laboratory in advance of the meeting:
A. USEPA 1995, "The Superfund Innovative Technology Evaluation
Program, Annual report to Congress, FY1994," EPA/540/R-95/522
B. "Superfund Innovative Technology Evaluation Program, Technology
Profiles, Seventh Edition." EPA/540/R-94/526, November 1994.
C. "EPA's Process for Technology Demonstration" (This unpublished
document explains what SITE does).
D. Alissa, Robert A., Letter about "RAP SITE-010, Risk Reduction
Engineering Laboratory, U.S. Environmental Protection Agency,"
January 10, 1995.
B-1
-------
E. Request for Preproposals (RFP) SITE-E09 The Emerging Technology
Program (ETP) of the Superfund Innovative Technology Evaluation (SITE)
Program," Ninth Solicitation, Emerging Technology Program, September
6, 1995.
F. Draft overheads and agenda for the SAB meeting June 1995
IV. Materials from Subcommittee Member John Maney
A. May 20, 1996 Survey of Site Developers
B. May 21,1996 Request for Information
V. Materials distributed at the meeting:
A. Science Advisory Board Review Meeting for SITE Program, June 11-13,
1996, Cincinnati, Ohio. This is a bound volume including the annotated
agenda
B. "Estimating Innovative Technology Costs for the SITE Program" by
Gordon M. Evans. J. Air Waste Management Assoc., July 1990
VI. The following materials were distributed to individual members based on
expertise and interest
A. Overview Materials for SITE
1. USEPA 1995, Office of Research and Development National
Exposure Research Laboratory, "A Guidance Manual for the
Preparation of Site Characterization Technology Demonstration
Plans." Interim Final
2. USEPA, 1994, SITE Program: An Engineering Analysis of the
Demonstration Program. EPA/540/R-94/530.
3. OSWER Directive 9380.0-25 Elliot P. Laws, "Initiatives to Promote
Innovative Technology in Waste Management Programs," April 29,
1996
B. Materials relating primarily to QA
B-2
-------
1. USEPA 1991, Office of Research and Development, "Preparation
Aids for the Development of Category I Quality Assurance Project
Plans." EPA/600/8-91/003
2. USEPA 1991, Office of Research and Development, "Preparation
Aids for the Development of Category II Quality Assurance Project
Plans." EPA/600/8-91/004
3. USEPA 1991, Office of Research and Development, "Preparation
Aids for the Development of Category III Quality Assurance Project
Plans." EPA/600/8-91/005
4. USEPA 1991, Office of Research and Development, "Preparation
Aids for the Development of Category IV Quality Assurance Project
Plans." EPA/600/8-91/006
5. Site QA Meeting Summary, February 10, 1993
6. Site QA Meeting Executive Summary, February 11, 1993
7. USEPA 1995, "Record of Meeting for the SITE QA Meeting of April
6, 1995"
8. Technical System Review of the Laboratory Analysis Associated
with the SITE Demonstration of North America Technologies
Group, Inc. SFC 0.5 Oleofication System, June 1994
9. U.S. Department of Energy 1995, "Matrix Photocatalytic Oxidation
Technology Demonstration, Final Quality Assurance Project Plan,"
Oak Ridge Reservation, Oak Ridge, TN.
10. USEPA 1995, Quality Management Plan for the National Risk
Management Research Laboratory, December 1994
C. Project Specific Materials
1. PRC Environmental Management Inc., 1994, "Draft Demonstration
Plan for the Evaluation of Field Portable X-Ray Fluorescence
Technologies," prepared for USEPA/EMSL in partial fulfillment of
Contract No. 68-CO-0047, Work Assignment No. 0-65.
B-3
-------
2. USEPA 1993, SITE Demonstration Bulletin: X*Trax Model 200
Thermal Desorption System, Chemical Waste Management, Inc.,
EPA/540/MR-93/502, February 1993
3. USEPA 1994, "SITE Emerging Technology Summary, Acid
Extraction Treatment System for Treatment of Metal Contaminated
Soils,"EPA/540/SR-94/513
4. USEPA 1994, "SITE Emerging Technology Summary, Handbook
for Constructed Wetlands Receiving Acid Mine Drainage,"
EPA/540/SR-93/523
5. USEPA 1994, "SITE Emerging Technology Summary, Acid
Extraction Treatment System for Treatment of Metal Contaminated
Soils,"EPA/540/SR-94/513
6. USEPA 1994, "SITE Emerging Technology Summary, Handbook
for Constructed Wetlands Receiving Acid Mine Drainage,"
EPA/540/SR-93/523
7. USEPA 1995, "Colloid Polishing Filter Method - Filter Flow
Technology, Inc., Innovative Technology Evaluation Report,"
EPA/540/R-94/501, May 1995
8. USEPA 1995, Office of Research and Development Risk Reduction
Engineering Laboratory (RREL) "Project Objective Agreement for
General Environmental Corporation, CURE Electrocoagulation
Technology at the Rocky Flats Technology Site"
9. USEPA 1995, "J.R. Simplot Ex-Situ Anaerobic Bioremediation
Technology: TNT" EPA 540/R-95/529a, Site Technology Capsule,
September 1995
10. USEPA 1995, "Ex-Situ Anaerobic Bioremediation Technology-TNT"
EPA 540/MR-95 Demonstration Bulletin, August 1995
11. USEPA 1995, "Electrokinetics, Soil Processing," EPA/540/F-
95/504, Emerging Technology Bulleting, March 1995
B-4
-------
12. USEPA 1993, "Metals Treatment at Superfund Sites by Absorptive
Filtration," EPA/540/SR-93/515, Emerging Technology Summary,
September 1995
13. USEPA 1993, "Low Temperature Thermal Aeration (LTTA) Process
Canonie Environmental Services, Inc." EPA/540/AR-93/504,
Application Analysis Report, July 1995
14. Theoretical and Experimental Modeling of Multi-Species Transport
in Soils Under Electric Fields. Cooperative Agreement No. CR
816828-01-1
B-5
-------
DISTRIBUTION LIST
The Administrator
Deputy Administrator
Regional Administrators, Regions 1-10
Assistant Administrators
Directors, Office of Research and Development Laboratories
Director, Office of Air and Radiation
Director, Office of Atmospheric and Indoor Air Programs
Director, Office of Policy Analysis and Review
Director, Office of Environmental Processes and Effects Research
Director, Office of Policy Analysis
Director, Climate Change Division
EPA Headquarters Library
EPA Regional Libraries
National Technical Information Service
Congressional Research Service
Library of Congress
------- |