&EPA
United States
Environmental Protection
Agency
Office of Research and
Development
Washington DC 20460
EPA/600/K-96/003
February 1997
Updated September 1997
Environmental Technology
Verification Program
Verification Strategy
-------
EPA/600/K-96/003
February 1997
Updated September 1997
Environmental Technology
Verification Program
Verification Strategy
Office of Research and Development
U.S. Environmental Protection Agency
Washington, DC 20460
Printed on Recycled Paper
-------
ETV
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
MAR 18
TlEADWNISTRATOH
Honorable Albert Gore, Jr.
The Vice President of the
United States
Washington, DC 20501
Dear Vice President Gore:
Within the next few days I will send the enclosed document. Environmental Technology
Verification Strategy, to the Chairmen and Ranking Minority Members of the House and Senate
Appropriations Subcommittees on Veterans Affairs, Housing and Urban Development, and
Independent Agencies. This document explains the goals, operating principals and program
criteria being used to implement tie EPA Environmental Technology Verification Program
(ETV).
ETV fulfills commitments made by the Agency in the National Pe rformancs Review to
establish a broad-based environmental technology verification program. It executes an initiative
in the President's National Environmental Technology Strategy, Bridge to a Svs, nimble Future.
It also is designed to carry out the intent of Congress as expressed in both the FY96 and FY97
appropriation legislation. I believe the Environmental Technology Verification Strategy will
assist the public and the Congress to understand better the efforts being made by EPA to cany
out this program.
We have made substantial progress in our first year of operation. The program wiU soon
cover all environmental technology areas: Several hundred stakeholders are involved and the first
technology verification statements were issued by EPA two weeks ago. We continue to befievc
that EPA performance verification of innovative technologies can make a major contribution to
more rapid protection of the world's environment at reasonable costs.
Sincerely,
Enclosure
Verification Strategy
-------
Foreword
The U.S. Environmental Protection Agency's (EPA) Office of
Research and Development (ORD) is publishing this document,
Verification Strategy, to report the goals, operating principles, and future
plans of the newly created Environmental Technology Verification
Program (ETV). ETV is intended to expand the environmental technology
choices of public and private decision makers, both in our country and
abroad. The use of new technologies will allow faster and more thorough
pollution prevention, control, cleanup, and monitoring, while bringing the
overall cost of regulatory compliance down.
Evaluations of promising new technologies by ETV should be
useful to a broad array of organizations:
• Buyers of technology, and the consulting engineering community
that advises them, who need an objective source of data to make
informed purchasing decisions.
• Permitters at state, local and federal level who must decide on
which technologies to allow into use.
• The financial community that must determine the level of risk
involved in supporting innovative technology developers.
• Vendors who want to market their products on a level
playing field of objectively acquired data.
The challenges of the next three years will be substantial as we
produce and broadly distribute credible data that are products of ETV.
Meeting these challenges will ensure that the environment, the economy,
and the marketplace benefit from the steady stream of scientific and
technological innovation this program will encourage.
Mr. Henry L. Longest II
Acting Assistant Administrator
Office of Research and Development
U.S. Environmental Protection Agency
Verification Strategy iii
-------
Contents
Page
Foreword iii
Background 1
Important Definitions 2
Operating Principles 3
ETV Pilot Process 7
Selection Criteria for ETV Pilot Projects 9
2001 Report to Congress 11
ETV Program Vision and Projected Development Scenario 11
ETV Web Site Information 14
Verification Strategy iv
-------
Environmental Technology Verification Program
Verification Strategy
Background
Throughout its history, the U.S. Environmental Protection Agency
(EPA) has evaluated technologies to determine their effectiveness in
monitoring, preventing, controlling, and cleaning up pollution. Since the
early 1990s, however, numerous government and private groups have
identified the lack of an organized and ongoing program to produce
independent, credible performance data as a major impediment to the
development and use of innovative environmental technology. Such data
are needed by technology buyers and permitters both at home and abroad
to make informed technology decisions. Because of this broad input, the
President's environmental technology strategy, Bridge to a Sustainable
Future, and the Vice President's National Performance Review, contain
initiatives for an EPA program to accelerate the development of
environmental technology through objective verification and reporting of
technology performance. In 1994, EPA's Office of Research and Develop-
ment formed a workgroup to plan the implementation of the Environmental
Technology Verification Program (ETV). The workgroup produced a
Verification White Paper that guided the initial stages of the program. This
document, Verification Strategy, updates the earlier paper based upon the
evolution of the program over the last two years. It outlines the operating
principles and implementation activities that are shaping the program, as
well as the challenges that are emerging and the decisions that must be
addressed in the future. ETV will continue to be modified through input
from all parties having a stake in environmental technology, through further
operational experience, and through formal evaluation of the program.
The goal of ETV, which remains unchanged, is to verify the
environmental performance characteristics of commercial-ready tech-
nology through the evaluation of objective and quality assured data, so that
potential purchasers and permitters are provided with an independent and
credible assessment of what they are buying and permitting.
Verification Strategy
-------
Important Definitions
A clear definition of the words "evaluate" and "verify", along with the
word "certify", is important to establish at the outset of the program. The
technology development community, the regulated community, and those
charged with executing environmental standards at all levels of
government require a precise understanding of what EPA means and does
not mean by the activities to be undertaken through ETV.
EPA intends to sponsor the evaluation of environmental technologies
through adequate testing and verify that they perform at the levels reported.
By evaluate and verify we mean:
Evaluate/Evaluation
To carefully examine and judge the efficacy of a technology; to
submit technologies for testing under conditions of observation and
analysis; syn., measure, estimate, classify, test.
Verify / Verification
To establish or prove the truth of the performance of a technology
under specific, predetermined criteria or protocols and adequate
data quality assurance procedures; syn., confirm, corroborate,
substantiate, validate.
EPA does not intend to certify that a technology will always, or
under circumstances other than those used in testing, operate at the levels
verified. By certify we mean:
Certify / Certification
To guarantee a technology as meeting a standard or performance
criteria into the future; syn., ensure, warrant, guarantee.
EPA understands that the word certify can have a variety of
meanings, but the Agency believes that the above definition is the one
most commonly understood. Misuse of the term could cause confusion
among the public.
Verification Strategy
-------
Operating Principles
Several important operating principles have defined the basic ETV
program structure and remain fundamental to its operation. These are
briefly outlined below.
1. Performance Evaluation Goal
Under ETV, environmental technologies are evaluated to ascertain and
report their performance characteristics. EPA and its partners will not
seek to determine regulatory compliance; will not rank technologies or
compare their performance; will not label or list technologies as
acceptable or unacceptable; and will not seek to determine "best
available technology" in any form. In general, the Agency will avoid all
potential pathways to picking "winners and losers". The goal of the
program is to make objective performance information available to all of
the actors in the environmental marketplace for their consideration and
decision making.
2. Commercial-ReadyTechnologies
The ETV program is a service of EPA to the domestic and international
marketplace in order to encourage rapid acceptance and
implementation of improved environmental technology. ETV, therefore,
focuses its resources on technologies that are either in, or ready for,
full-scale commercialization. The program does not evaluate
technologies at the pilot or bench scale and does not conduct or
support research. Participation in ETV is completely voluntary.
3. Third-PartyVerification Organizations
ETV leverages the capacity, expertise, and existing facilities of others
through third-party partnerships in order to achieve universal coverage
for all technology types as rapidly as possible. Third-party verification
organizations are chosen from the both the public and private sector,
Verification Strategy
-------
including states, universities, associations, business consortia, private
testing firms, and federal laboratories. EPA designs and conducts
auditing and oversight procedures of these organizations, as
appropriate, to assure the credibility of the process and data. In order to
determine if EPA participation is important to the commercialization
process, ETV is testing the option of one totally unstructured and
independent, private sector pilot in which EPA's role will be solely
fiduciary. In addition, the Agency will continue to publish the results of
commercial-ready technology evaluations that it conducts in the normal
course of its business.
4. PilotPhase
The program will begin with a five year pilot phase to test a wide range
of partner and procedural alternatives, as well as the true market
demand for and response to such a program. Throughout the pilot
period, EPA and its partners will operate in a flexible and creative
manner in order to identify new and efficient methods to verify
environmental technologies, while maintaining the highest credibility
standards. The operational objective will be to actively look for ways to
optimize procedures without compromising quality. The ultimate
objective of the pilot phase is to design and implement a permanent
verification capacity and program within EPA, should the evaluation of
the effectiveness of the program warrant it.
5. PilotTechnologyAreas
ETV has begun with pilots in narrow technology areas in each of the
major environmental media and will expand as appropriate, based on
market forces, availability of resources, and the willingness of the
marketplace to pay for third-party verification. For example, the drinking
water technology pilot has started with a focus on microbial and
particulate contaminants, and disinfection byproducts in small systems
(less than 3300 users), an obvious and very large domestic and
international market with pressing environmental problems. In future
years, the program will be expanded to the wider area of nitrates and
synthetic organic chemicals and pesticides in all drinking water
systems. Success in particular technology areas will allow the program
to have a "pump-priming" effect to bring new technologies to the
marketplace. Selection criteria for ETV pilot programs and other
verification focus areas are discussed in a subsequent section of this
paper.
Verification Strategy
-------
6. Stakeholder Groups
ETV is guided and shaped by using the expertise of appropriate
stakeholder groups in all aspects of the program. These groups consist
of representatives of all verification customer groups: buyers and users
of technology, developers and vendors, and, most importantly,
technology "enablers", i.e., the consulting engineering community that
recommends technology alternatives to purchasers, and the state
permitters and regulators who allow it to be used. Stakeholder groups
must be unique to each technology area in order to capture the
important individual aspects of the different environmental media and to
get buy-in from affected groups. For example, state drinking water
permitters are necessary to participate in development of testing
protocols for cryptosporidium; air pollution regulators are needed to
evaluate innovative compliance monitoring devices; metal production
parts manufacturers need to help design testing procedures for new
coating compounds. In general, the role of stakeholders will be to assist
in the development of procedures and protocols, prioritize types of
technologies to be verified, review all important documents emerging
from the pilot, assist in defining and conducting outreach activities
appropriate to the particular area, and, finally, to serve as information
conduits to the particular constituencies that they represent. As of
September 1997, over 180 individuals are serving in the six
stakeholder groups formed to date.
7. Private Sector Funding
Over the five year pilot phase of the program, the costs of verifying
technologies in many pilots will move from a primarily government
funded effort to a primarily private sector funded effort. At least two
pilots will be vendor supported from the beginning. The original goal, as
articulated in the 1994 strategy, called for complete private sector
sponsorship within three years. A recent review (1995) of the program
by a distinguished panel of outside experts convened by the EPA
Science Advisory Board (SAB) concluded that such a goal was
probably not achievable in so short a time-frame (they suggested five to
eight years) and that some level of government support (10 to 20% of
ongoing costs) would remain necessary to keep the activity viable.
Conclusions on this issue will be reached as data emerge on the
economic value-added of the program and the level of cost that the
private sector is willing to bear in the various technology sectors.
Verification Strategy
-------
8. Pilot Evaluation and Program Decisions
The Agency will collect data on operational parameters, e.g., number of
participants; cost and time required to perform tests and report results,
and on outcomes, e.g., use of data by the states and public; sales
reported by vendors, in order to evaluate all aspects of the program.
EPA will use this information to make long-term recommendations to
the Congress on the future and shape of the program in 2001. Among
the choices at that time will be the formulation of a permanent, broad
scale program; the narrowing of efforts to certain areas in which ETV
appears to be effective; or the discontinuance of verification efforts. The
latter conclusion could be reached either because state regulators/
permit writers and the technology innovation industry are not assisted
by ETV or because the cost of verification proves to be prohibitive.
9. Outreach and Information Diffusion
As was pointed out by the SAB in its 1995 review of ETV, verification
alone will not move better, cheaper, faster technologies to success in
the marketplace. In particular, substantive and substantial interface with
the permitters of environmental technology (primarily at the state level)
will be necessary to have any chance of rapidly implementing
innovative approaches. To date, the outreach activities of the program
have been limited to assuring substantial state representation on the
Stakeholder Groups that are designing the protocols and procedures
for each pilot; developing informational fact sheets about the program;
and placing a Web page on the Internet. In 1998, the Agency intends to
develop an overarching outreach strategy through consultation with all
affected groups and with the help of major organizations in the
technology area, e.g., National Governors Association, Western
Governors Association, Environmental Council of the States, National
Pollution Prevention Roundtable, appropriate corporations, and others.
10. Market Gap Definition
Lastly, EPA will track applications and expressions of interest on the
part of technology developers who come to all parts of the Agency that
do not fit into the present suite of verification activities. This universe
will be characterized during the initial stages of the pilot period and a
strategy to address gaps will be developed.
Verification Strategy
-------
ETV Pilot Process
Although a wide degree of flexibility will characterize the pilot
projects (see above), each ETV pilot generally will go through two periods
of development: an organizational phase and an operational phase (see
Figure 1).
Figure 1. ETV Pilot Process
Organization Phase
Operations Phase
Verification Organization (VO)
Selected
(EPA)
I
Open Solicitation
(VO)
Stakeholder Groups Formulated
(EPA/VO)
Generic/Tailored Test Plans
(Developer/VO)
Technologies Prioritized
and
Generic Protocols Developed
(EPA/VO/Stakeholders)
Testing byVO
OR
Testing by Developer
(With VO Oversight)
I
QA Evaluation
(EPA/VO)
Verification Report/Statement
(EPA/VO)
Information Diffusion
(EPA/Stakeholders)
Verification Strategy
-------
Organizational Phase
During the organizational phase, EPA will select one or more partner
organizations to oversee and conduct verification activities. This
important step will usually occur through an open solicitation process,
although some exceptions will be appropriate. All partner proposals will
be peer reviewed. EPA and its partner(s) will then select approximately
25 appropriate participants for the Stakeholder Group (see #6) that will
guide the progress of the program. EPA may formulate the
Stakeholders Group prior to selecting the partner organization if the
procurement process is protracted. Stakeholder Groups will then begin
the important process of establishing priorities and defining procedures
and protocols appropriate to that particular type of environmental
technology and customer group.
Operational Phase
Once the basic building blocks of the program are in place, actual
verification activities begin (these steps will not always be strictly
sequential). Verification activities in each technology area will be
announced in the Commerce Business Daily and other appropriate
publications to encourage maximal participation by technology
developers, and to assure a level playing field. Test plans will be
developed with the participation of developers and tests conducted by
independent third parties (either the verification organization or other
testing organizations approved by the verification organization).
Appropriate quality assurance procedures will be incorporated into all
aspects of the project and reports will be peer reviewed. Verification
statements of three to five pages, based on the performance data
contained in the reports, will be issued by EPA and appear on the ETV
Internet Web page. Other outreach activities, as defined by the
Stakeholder Group, such as state permitter training, will be conducted.
Verification Strategy
-------
Selection Criteria for ETV Pilot Projects
The appropriate selection of verification pilot programs and other
ETV verification activities to be carried out under the program is critical to
its ultimate success. The ETV pilot programs are designed to meet the
needs of the many stakeholders in environmental technology, while
allowing EPA to experiment with a variety of procedures and partnership
arrangements to determine the optimal form of program implementation.
The following criteria have been used to choose ETV verification pilots
and other verification activities. The first three criteria are applied in
cascading order, i.e., each pilot passes the previous criterion in order to go
on to the next.
1. AddressImportantEnvironmental Needs
All programs conducted by the EPA have environmental improvement
and protection as their ultimate objective. It is particularly important
that this program, which is designed to assist the environmental
technology industry, select pilot verification activities that have clear
and positive environmental benefits. Such benefits may improve the
environment by achieving higher levels of pollution reduction or by
accelerating the rate of technology implementation through lower cost
or simplified operation.
2. Present Substantial Business Opportunities for the Private
Sector
Pilot areas selected have clear market niches, both domestically and
internationally, that present the potential for a substantial increase in
technology sales and use. Although voluntary environmental
improvement technologies, e.g., indoor air filtration systems, and
existing regulatory program technologies, e.g., hazardous waste
monitoring devices, are of interest to the program, particular attention
will be paid to technology areas that stress the Agency's commitment to
pollution prevention or address upcoming regulatory requirements and
Verification Strategy
-------
deadlines that present obvious and major opportunities for innovation
and increased economic activity.
3. Involve Multiple Developers andVendors
To clearly demonstrate the potential of a successful verification
program, the ETV pilots must benefit the largest possible number of
technology developers and users. Pilot programs will focus on areas in
which a number of technologies and companies are active.
The last two criteria are programmatic and allow the Agency to test
operating program parameters in the pilot phase.
4. Address the Full Range of Environmental Media
Assuring that all major environmental media and program areas are
included in the pilots will allow ETV to begin interaction with interested
parties, e.g., technology developers, technology buyers, regulation
writers and state permitters, across a broad range of environmental
areas. This will assist the program to identify particularly fruitful
technology focus areas for the future, spread the word about the EPA
verification function widely, identify regulatory barriers that inhibit the
use of verified technologies, and assist developers across a wide
spectrum of technology.
5. Test a Variety of Verification Organization Types
As described above, EPA is interested in evaluating all possible
verification organization alternatives, including federal government
laboratories, state verification programs, universities, industry
associations, independent testing organizations, and developer
conducted testing (with verification organization or EPA oversight).
Pilot programs selected are expected to cover all of these alternatives.
Having applied the above criteria to a wide variety of potential pilot
candidates, the Agency has selected twelve categories as ETV pilots (see
Table 1).
10 Verification Strategy
-------
Table 1.ETV Pilots
1. Site Characterization & Monitoring Technologies
2. P2, Recycling & Waste Treatment Technologies
3. Drinking Water Systems
4. Indoor Air Products
5. P2 Innovative Coatings and Coating Equipment
6. EvTEC (any technology area)
7. Advanced Monitoring Systems (Air and Water)
8. Air Pollution Control Technology
9. Greenhouse Gas Technology
10. Wet Weather Flow Technologies
11. Source Water Protection Technologies
12. P2 Metal Finishing Technologies
2001 Report to Congress
ETV will prepare a Report to Congress, to be sumitted in 2001. The
questions that will be addressed in the Report include:
1. Does the environmental marketplace need/value an EPA
verification program?
2. Is the need different from one technology area to another?
3. Can we design a credible program that the developer community
can afford?
4. Where is value added from ETV:
• Protocol development?
• Third party QA oversight?
• EPA Verification Statement?
ETV Program Vision and Projected Development
Scenario
Figure 2 lays out the projected ten year development scenario for
the ETV Program in which assumptions are made concerning: (1) the
amount of funding received by the program; (2) the number of viable
technology areas (defined as pilots in the early stages) needed; (3) the
number of EPA staff assigned to quality assurance and technical oversight;
and (4) the number of new technologies that come to the program for
verification. All of these assumptions are projections based upon the
Verification Strategy
11
-------
Figure 2. ETV Program Scenario
ou
45 -
40 -
35 -
30 -
25 -
20 -
15 -
10 -
5 -
i-i
1 1,11
1 JIJJ
i-i
r
i-i
f
i-i
i
-i
{
-
{
-i
I
-
0.
1995 1996 1997 1998 1999 2000 2001 2002 2003
J
-
2004
I
-
2005
• Federal $M Per Year n Technology Areas
• # Verified Technologies Per Year n FTE
experience of technology verification programs in other fields and the level
of resources needed to accomplish the work projected. Changes to any of
the assumptions would, of course, change all of the projections.
1. Federal Funding
The projected budget moves from $7M in 1995 to $1OM for three years
of development and building market acceptance. It then begins a
gradual decline as the private sector pays more and more of the costs.
By 2005 the steady-state cost of the program for all environmental
areas is projected to be $2M.
12
Verification Strategy
-------
2. TechnologyAreas
Starting in 1995, pilot programs gradually expand to cover all
appropriate environmental areas (areas defined primarily by customer
groups). By 1998, twelve areas are defined and implemented and
remain in place throughout the program, although their technology
focus can shift over time, based on market forces.
3. EPA Personnel2
EPA staff necessary to oversee third-party verifiers and maintain the
quality and credibility of the program move from 10 in 1995 to as many
as 30 while the program is in its formative stages. As procedures and
quality assurance measurers become routinized, staff demands
gradually lower to a steady state of 10 FTE by 2003.
4. VerifiedTechnologies
Due to a demand backlog, the number of technologies verified rapidly
rises from five in 1996 to 50 in 2001, and then declines to a steady state
of about 30 to 35 technologies each year.
If executed as projected in this scenario, the Environmental
Technology Verification Program could reasonably be expected to verify
the performance of approximately 300 innovative technologies in a
decade.
2 EPA personnel are assigned to ETV from appropriate media areas (e.g., drinking
water, advanced air monitors) within ORD's national laboratories and centers
based upon programmatic requirements. Six EPA staff are assigned to ETV
directly for management coordination, information dissemination, and oversight.
Verification Strategy 13
-------
ETVWeb Site Information
Accessing the Latest ETV Information
Ongoing information about the progress of the ETV Program,
including all published documents and a list of upcoming events, can
be downloaded from the ETV Web Site at http://www.epa.gov/etv
14
Verification Strategy
------- |