United States Environmental
         Protection Agency
Office of Policy
(1807T)
November 2011
EPA-100-R-11-009
?/EPA
           Evaluation of
           Implementation of the
           Superfund Green
           Remediation Strategy
           Final Report
           Promoting Environmental Results
           i                         >
           Through Evaluation

-------
ACKNOWLEDGEMENTS


This report, Evaluation of Implementation of the Superfund Green Remediation Strategy,
was developed for the U.S. Environmental Protection Agency's Office of Policy (OP)
under Contract EP-W-10-002 between EPA and Industrial Economics, Inc. (IEc) of
Cambridge, MA. The IEc evaluation team included Cynthia Manson, Fauna Samuel, and
Yeshaya Dobrusin.


Carlos Pachon, Suzanne Wells, and Ellen Treimel of EPA's Office of Solid Waste and
Emergency Response, Office of Superfund Remediation and Technology Innovation
(OSRTI) provided critical assistance and background information on the Superfund
Green Remediation Strategy program. John Heffelfinger and Shari Grossarth of OP's
Evaluation Support Division (ESD) provided technical support and advice for the
evaluation.  Special thanks go to EPA staff at Headquarters and the Regions who shared
thoughtful reflections, recommendations, and critiques during the conduct of this
evaluation.

This evaluation was funded jointly by OSRTI and ESD, and was  selected under the
Program Evaluation Competition, sponsored annually by EPA's Office of Policy.
Program evaluation is one of the performance management tools  that EPA uses to assure
itself, the public, and other interested stakeholders that EPA programs are protecting
human health and the environment effectively and efficiently. The information obtained
through  program evaluations can shed light on whether EPA programs are meeting their
goals  and objectives, provide the evidence and road map needed to replicate successes,
and identify those aspects of EPA programs needing improvement. To access copies of
this or other EPA program evaluations, please go to EPA's Evaluation Support Division
website  at http://www.epa.gov/evaluate.

-------
TABLE  OF CONTENTS

EXECUTIVE SUMMARY

CHAPTER 1  |  INTRODUCTION AND PURPOSE
I. I  0\ en ie\\ of the Superfund (irccn Rcsnalknion Siniic^v 2
1.2  Siiperfund (irccn Re/ucJituio/i Sirmc^v Logic Model 5
i .3  H\aluation Questions and Purposes 7
1.4  Report Organization 8

CHAPTER 2  |  METHODOLOGY
2.1  l:\aluation Design 9
2.2  Steps in Conducting the  l:\aluation 14

CHAPTER 3  |  FINDINGS
3,1  L\a!nation Purpose I: Assess LPA I!\penences to Date in Implementing the
    (iR Strategy 22
3.2  L\aination Purpose 2: Determine a Baseline Against Which to Measure LPA
    Progress in Implementing the (iR Sirt//c^v 44
3.3  H\aluation Purpose 3: Determine the Best Metrics for Measuring the Program's
    Success in Implementing (iR Practices 48
3.4  Synthesis of Key l:\aluation Findings 62

CHAPTER 4  |  CONCLUSIONS AND SUGGESTED NEXT STEPS
4,1  L\a!nation Purpose I: Assess LPA I!\penences to Date in Implementing the
    (iR Stniic^y 65
4.2  L\aiiiation I-   „  -           -  !'   -     , -   ist Which to Measure LPA
    Progress in  •  • , • ,   •   •  •  •  •     : 67
4.3  H\aluation '                   ', '  :      •   for Measuring the Program's
    Success in I  ,    •     • 11'  I-    :  68

APPENDIX A: LIST OF INTERVIEWEES
APPENDIX B: INTERVIEW GUIDES
APPENDIX C: REGIONAL SURVEY AND TRACKING DATA
APPENDIX D: EPA CLU-IN USE DATA
APPENDIX E: LIST OF REFERENCES

-------
lEc
                     EXECUTIVE SUMMARY
                     The Superfund program is working to advance greener cleanups at Superfund sites. Green
                     remediation (GR) is defined as the practice of considering all environmental effects of
                     remedy implementation and incorporating options to minimize the environmental
                     footprint of remedies. Central to this effort is the Superfund Green Remediation Strategy,
                     which was published in final form in September, 2010. The strategy outlines 40 action
                     items across three main areas with the ultimate goal of reducing the environmental
                     footprint of cleaning up contaminated sites.
                     As part of the Strategy, the Superfund program is evaluating the implementation of the
                     Strategy itself. The chosen approach is to conduct a "formative" evaluation of the
                     national-level effort. The purpose of the evaluation is to document the Strategy's
                     effectiveness in advancing greener cleanups. This evaluation was guided by nine
                     questions organized under three key purposes:
                     Evaluation Purpose 1: Assess EPA experiences to date in implementing the GR
                     Strategy
                     1.  Does EPA have clearly defined goals and objectives for the GR Strategy! Should they
                        be refined and improved to enhance usefulness (e.g., for management decision
                        making, planning and budgeting, EPA's Strategic Plan)!
                     2.  Which initial activities or initiatives from the GR Strategy have been most effective in
                        increasing awareness, adoption and/or implementation of GR strategies?
                     3.  How do Remedial Project Managers (RPMs) factor the GR Strategy into their
                        approach to planning site cleanup?
                        •   What GR practices are being implemented?
                        •   What percentage of RPMs are  implementing specific GR practices?
                        •   What do RPMs know about the energy usage at the sites they manage?
                        •   What information do RPMs track on other GR core elements?
                     4.  What effect has the GR Strategy had on the practice of using green remediation
                        techniques at Superfund sites?
                     5.  What lessons have been learned as a result of implementing the GR Strategy at sites?
                        •   What factors affect the ability to implement the GR Strategy at sites? (e.g.,
                            technical issues, cost issues, legal issues, management support, contract
                            provisions, or contractor capabilities)
                        •   How is integration of the GR Strategy priorities (e.g., policy guidance, training,
                            and tools) affected by the above factors and experiences to date?
                                                                                                 ES-1

-------
lEc
                     Evaluation Purpose 2:  Determine a baseline against which to measure EPA
                     progress in implementing the GR Strategy
                     6. What options can we identify for developing a baseline?
                        •   What has changed since the implementation of the GR Strategy!
                        •   When did green remediation become important to site cleanup?
                        •   What options are available for quantifying the environmental footprint at sites?
                     Evaluation Purpose 3:  Determine the best metrics for measuring the program's
                     success in implementing GR practices
                     7. What performance measures are appropriate for measuring the effectiveness of the GR
                       Strategy in achieving intended outcomes at a regional or national level?
                     8. What are the best means for measuring the effectiveness of the GR Strategy in
                       reducing the environmental footprint at sites that have implemented GR practices with
                       respect to the five core elements ofGR Strategy?
                        •   What options exist for using qualitative or quantitative measures to assess the
                            five core elements of GR Strategy!
                     9. Where are the primary data gaps and limitations that inhibit a better understanding of
                       the results of implementing the GR Strategy!

                     PROGRAM LOGIC  MODEL
                     An initial step in the evaluation was the development of a program logic model (Exhibit
                     ES-1) to illustrate the various components of the  Superfund Green Remediation Strategy
                     and to inform development of specific evaluation questions.  For this evaluation, the logic
                     model focuses on the outcomes that are most relevant to the Superfund Green
                     Remediation Strategy and the ability to measure its progress. The logic model activities
                     are organized to reflect the three main categories of actions in the GR Strategy, including
                     policy and guidance development, resource development and program implementation,
                     and program evaluation.

                     EVALUATION METHODOLOGY
                     As discussed in Chapter 2 of this report, this evaluation uses a range of data sources and
                     analytic techniques. First lEc conducted a review of existing published  background
                     documents available online and provided by EPA. In addition, lEc reviewed, as relevant,
                     site specific data (e.g., to identify available data to support specific metrics for
                     documenting progress under the GR Strategy on the  core  elements). Finally, this
                     evaluation relies primarily on data collected directly from EPA personnel who are
                     currently involved in the implementation of the GR Strategy. lEc employs a combination
                     of targeted interviews, literature review, and review  of existing survey and site data to
                     ensure high quality data collection and analysis.  Exhibit  ES-2 summarizes the methods
                     and data sources used to address the evaluation questions.
                                                                                                 ES-2

-------
lEc
EXHIBIT  ES-1.
                  SUPERFUND  GREEN REMEDIATION STRATEGY LOGIC  MODEL
                      KCIVICUIM i \\jn S>IKAI t(jY
                                                                                                                             OUTCOMES
  RESOURCES
                J_L
                             ACTIVITIES
Funding
EPA Staff
• OSRTI
•GR Workgroup
• 10 Regional
 Coordinators
Policy
Resources
•EO 13514
• OSWER
 Principles for
 Greener
 Cleanups
•GR Practices at
 State &
 Regional  level

  Peer Network
    (developed
     through
  implementation)

                        Policy and Guidance
                          Development
                        • Evaluate potential
                         ARARs
                        Resource
                          Development
                        • Modify contract
                         language to include GR
                         practices
                        • Define and refine
                         BMPs as appropriate
                        • Briefing contractors
                         and POs
                        • Establish mechanisms
                         for financing GR
                        • Define footprint
                         methodology	
                                                       OUTPUTS
                        Program
                          Implementation
                        • Deliver GR trainings
                         (NARPM)
                        •Communication with
                         RPMs, DDs.and PRPs
                         through briefings,
                         meetings and calls
                       Program Evaluation
                       • Establish program
                         baseline
                       • Measure footprint
                                                   Identify additional
                                                   GR information
                                                   resources
REG Purchase
Agreement
Statement
Issue policy on GR
remedy selection
Define and refine
BMPs
Standard contract
language
• Contracting tool kit
Footprint
methodology
Communications
outputs
• Fact sheets
•Checklists
• Qs & As
• Clu-in website
• Case studies
• Research
• Advertising/
 outreach
                                                   • Baseline & reports
                                                    on environmental
                                                    footprint reductions
                                                                                                    SHORT-TERM
                                    Contextual
                                     Factors
   Cannot be too specific with how to implement GR methods because it limits cleanup options
   Regional access to different GR methods is not equal
   Greener cleanup methods are sometimes more costly than less green methods
                                                                                                                           INTERMEDIATE
                                                                                                                                                    LONG-TERM
                                               Assess consistency
                                               with Regional
                                               Strategies
                                                                                                                         Power purchase
                                                                                                                         from local utility
                                                                                                 Greater awareness
                                                                                                 of GR Strategy and
                                                                                                 BMPs
                                                                       Adopt integrated GR
                                                                       BMPs for each of 5

Awareness of
authority (well
defined)
•Enforcement
•Legal
• Contract
Provide site-specific
assistances.
assistance
mechanisms

Greater awareness
of Resources
•Clu-in
' -Technical
knowhow (skills)
/

>*

• Project managers
communicating to
stakeholders
1
GR reflected in
ROD, where
appropriate
• Consistent
phrasing
t
Use of contracting
tool kit
• Contractor
notification
•RPM request
contractor to
incorporate GR
                                               Greater awareness
                                               of status of
                                               environmental
                                               footprint &
                                               opportunities to
                                               improve (5 core
                                               elements)
Specify., adopt
performance goals,
objectives and
                                                                                                                         measures
                        Full integration of
                        GR Strategy
                        Energy
                        Use of
                        renewables
                        and energy
                        efficiency
                        Air
                        Reduced
                        emissions
                        Water
                        Reduced
                        consumption/
                        increased
                        protection
                        Land/eco-
                        systems
                        Minimize harm
                        and protect
                        land for future
                        use
Materials/
waste
Reduce
material
consumption
and waste
generation
_J

                                                                                                                                                               ES-3

-------
lEc
ES-2.  EVALUATION QUESTIONS AND DATA SOURCES FOR FORMATIVE EVALUATION OF THE CR STRATEGY
EVALUATION QUESTION
PRIMARY DATA AND METHODS
SECONDARY DATA AND METHODS
EVALUATION PURPOSE 1 : ASSESS EPA EXPERIENCES TO DATE IN IMPLEMENTING THE CR STRATEGY
1 . Does EPA have clearly defined goals and
objectives for the GR Strategy? Should they be
refined and improved to enhance usefulness (e.g.
for management decision making, planning and
budgeting, EPA's Strategic Plan)?
2. Which initial activities or initiatives from the
GR Strategy have been most effective in increasing
awareness, adoption and/or implementation of the
GR Strategy?
Interviews:
• Regional Coordinators
• Outside-EPA staff
Interviews:
• Regional Coordinators
Data Review:
• Atlanta Meeting survey
• State and regional strategies
Interviews:
• Regional attorneys
• Front line managers
Data Review:
• GR Strategy
Interviews:
• Regional attorneys
• Front line managers
Data Review:
• GR Strategy
3. How do Remedial Project Managers (RPMs) factor the GR Strategy into their approach to planning site cleanup?
3a) What GR practices are being implemented?
3b) What percentage of RPMs are implementing
specific GR practices?
3c) What do RPMs know about the energy usage at
the sites they manage?
3d) What information do RPMs track on other GR core
elements?
4. What effect has the GR Strategy had on the
practice of using green remediation techniques at
Superfund sites?
Interviews:
• Regional Coordinators
Data Review:
• Atlanta Meeting and regional surveys
• State and regional strategies
Interviews:
• Regional Coordinators
Interviews:
• Regional attorneys
• Front line managers
Data Review:
• GR Strategy
Data Review:
• GR Strategy
• CLU-IN case study site profiles
• Regional surveys
• State and regional strategies
5. What lessons have been learned as a result of implementing the GR Strategy at sites?
5a) What factors affect the ability to implement the
GR Strategy at sites (e.g., technical issues, cost
issues, legal issues, management support, contract
provisions, or contractor capabilities)?
5a) How is integration of the GR Strategy priorities
(e.g., policy guidance, training, and tools) affected
by the above factors and experiences to date?
Interviews:
• Regional Coordinators
Interviews:
• Regional attorneys
• Front line managers
• Outside-EPA staff
Data Review:
• CLU-IN website use data
                                                                                                     ES-4

-------
lEc
EVALUATION QUESTION
PRIMARY DATA AND METHODS
SECONDARY DATA AND METHODS
EVALUATION PURPOSE 2: DETERMINE A BASELINE AGAINST WHICH TO MEASURE EPA PROGRESS IN IMPLEMENTING THE CR STRATEGY
6. What options can we identify for developing a baseline?
6a) What has changed since the implementation of
the CR Strategy?
6b) When did green remediation become important
to site cleanup?
6d) What options are available for quantifying the
environmental footprint at sites?
Data Review:
• GR Strategy
• Footprint methodology
• Published studies
• CLU-IN case studies

Interviews:
• Regional coordinators
• Measurement specialists
• Outside-EPA staff
Data Review:
• Regional tracking data
EVALUATION PURPOSE 3: DETERMINE THE BEST METRICS FOR MEASURING THE PROGRAM'S SUCCESS IN IMPLEMENTING GR PRACTICES
7. What performance measures are appropriate
for measuring the effectiveness of the GR Strategy
in achieving intended outcomes at a regional or
national level?
Data Review:
• Footprint methodology
• Published studies
• Regional tracking data
• Atlanta meeting and regional surveys
Interviews:
• Regional coordinators
• Measurement specialists
• Outside-EPA staff
8. What are the best means for measuring the effectiveness of the GR Strategy in reducing the environmental footprint at sites that have
implemented GR practices with respect to the five core elements of the GR Strategy?
8a) What options exist for using qualitative or
quantitative measures to assess the five core
elements of the GR Strategy?
9. Where are the primary data gaps and
limitations that inhibit a better understanding of
the results of implementing the GR Strategy?
Data Review:
• Published studies
• CLU-IN website use data and case studies
Interviews:
• Regional coordinators
• Measurement specialists
Data Review:
• Published studies
Interviews:
• Regional coordinators
Data Review:
• Atlanta meeting and regional surveys
• Regional tracking data
• Footprint analyses and tools
Data Review:
• Footprint analyses and tools
                                                                     ES-5

-------
lEc
                     EVALUATION FINDINGS

                     Chapter 3 presents the evaluation findings, organized by the evaluation purposes and then
                     the evaluation questions. We provide a short summary below:

                     Evaluation Purpose 1: Assess EPA experiences  to date  in
                     implementing  the CR Strategy
                     Overall, interview respondents were uniformly positive in their opinions of the GR
                     Strategy structure and purpose, though responses identified some differences of opinion
                     in how best to present "goals" and objectives.  Several respondents noted that a more
                     precise goal statement could be useful both in  increasing awareness and focusing further
                     implementation of the GR Strategy.
                     In the strongest finding, EPA and non-EPA interviewees had very positive views of
                     several key products of the GR Strategy, and felt that these tools and products have been
                     a key driver in facilitating an expansion of GR activities.  Respondents felt that awareness
                     of the GR Strategy document was more limited, though publication of the GR Strategy
                     has facilitated the use of GR by raising the national profile of GR.
                     Interview responses from the regions indicate that RPMs typically do not use the GR
                     Strategy directly in their decision-making for GR implementation, though it is clear they
                     use many of the tools and products developed  to support the GR Strategy.  The GR
                     Strategy document was identified to be a more important tool for managers than for
                     RPMs.
                     It is difficult to assess the  distinct contributions of either the national strategy or regional
                     policies separately, since they influence each other. A few regional policies informed the
                     GR Strategy, while others many not have been released without the national focus on GR.
                     Examination of regional data from surveys provides a snapshot of activities underway,
                     and it is apparent that regions have increased emphasis  on GR training and outreach as
                     the GR Strategy has emerged.
                     A range of challenges face the broader implementation  of the GR Strategy, with key
                     concerns including the level of funding and support for GR Strategy personnel and
                     project efforts.  Other hurdles include a concern about policy and liability uncertainty,
                     and limited participation from managers and other key staff.

                     Evaluation Purpose 2:  Determine a  baseline  against which to
                     measure EPA progress in implementing the  CR Strategy
                     Interview responses from the regions indicated that most have not focused to date on
                     developing a baseline. Overall most of the regions (eight often) identified that their
                     current practices represent a fairly accurate baseline before the GR Strategy was released
                     because the implementation of GR efforts is just beginning.  A complexity of the GR
                     Strategy is that it has been implemented as a unifying approach encompassing some
                     existing efforts, and in some cases it clearly post-dates regional activities (e.g., Region 2
                     and 9). Moreover, a key goal of the program is awareness, and in some cases people are
                     "doing" GR without calling it GR. The findings from this evaluation suggest that EPA
                                                                                                ES-6

-------
lEc
                    consider whether one baseline is adequate to support the program. If EPA wishes to
                    document contribution of the GR Strategy generally then a single date-driven baseline
                    may be appropriate. To document attribution, however, use of different regional baselines
                    for site-specific action may be necessary.

                    Evaluation Purpose 3:   Determine the  best metrics for  measuring the
                    program's success  in implementing GR practices
                    A review of the logic model associated with the GR Strategy suggests that a suite of
                    appropriate performance measures for program performance would directly assess the
                    short-term (changes in awareness), intermediate term (changes in behavior), and long-
                    term (changes in site practice and impacts) outcomes of the GR Strategy. Metrics for each
                    type of outcome would also  allow EPA to assess the extent to which the GR Strategy is
                    effectively implemented and successful in integrating GR principles throughout the
                    remediation process. Successful metrics will also likely require only limited data
                    collection and analysis.
                    Review of existing and emerging tools for calculating environmental footprint suggest
                    EPA's efforts to craft and test a footprint methodology to support GR activities at sites
                    provides a comprehensive set of metrics that map to four of the five GR Strategy core
                    elements (excluding land and ecosystems). Several metrics listed in the footprint
                    methodology appear consistent with other sources and may be able to be adopted with
                    limited additional effort. The most successful metrics may be those that HQ can estimate
                    using standardized values and limited regional data.
                    Interview responses suggest that a number of key challenges exist for understanding the
                    impacts of GR.  As was noted in Evaluation Purpose 1, a larger issue that arose from the
                    interview process is the identification of the need for policy-level clarity of the GR
                    Strategy. The lack of clear direction from EPA providing legal and policy justification for
                    incorporating GR techniques at sites seems to have decreased momentum for moving GR
                    forward in some regions. Other limitations that inhibit a better understanding of the
                    results of implementing the GR Strategy include concerns about resource constraints
                    (e.g., time, funding, manpower), concerns that clear legal authority for requiring GR
                    practices is not well defined, and reluctance on the part of EPA staff in many regions to
                    use the methodology to conduct footprint analyses.

                    EVALUATION  CONCLUSIONS AND SUGGESTED NEXT  STEPS

                    Chapter 4 of this report provides conclusions and suggested recommendations for the
                    future implementation of the Superfund Green Remediation Strategy.  In summary, they
                    include recommendations that EPA:

                         •  Focus on clarity of goals and implementation objectives.
                         •  Continue emphasis on practical tools for GR implementation.
                         •  Increase focus on policy and legal information and tools, or on other HQ
                           "signaling."
                                                                                                ES-7

-------
lEc
                           Consider the following as a starting point for establishing two baselines:
                                    o   A region-specific baseline for documenting site-level changes (core
                                        elements) and attributing change to the GR Strategy.
                                    o   A national baseline for documenting integration of GR practices into
                                        EPA cleanup culture.
                           Work with regions and develop guidance on how and when to conduct footprint
                           analyses.
                           Start a dialogue with each of the regions to agree on the best way to leverage case
                           study and other available data to develop an estimation tool or "average" values
                           for GR practices.
                           Select metrics to measure program success based upon appropriate EPA criteria.
                                                                                                  ES-8

-------
lEc
                    CHAPTER 1   |  INTRODUCTION AND  PURPOSE
                    Under the Comprehensive Environmental Response, Compensation, and Liability Act
                    (CERCLA), commonly known as Superfund, the U.S. Environmental Protection Agency
                    (EPA) has, since 1980, investigated and assessed contaminated hazardous waste sites, and
                    undertaken enforcement and remediation activities to ensure protection of human health
                    and the environment. Hazardous waste site investigation and remediation uses resources
                    such as energy, water, and materials, and also creates a physical environmental
                    "footprint" at the site and any related disposal areas. In recent years, EPA has focused on
                    identifying and employing "green remediation" techniques at Superfund sites. Green
                    remediation (GR) is defined as the practice of considering all environmental effects of
                    remedy implementation and incorporating options to minimize the environmental
                    footprint of remedies. A centerpiece of this effort is EPA's Superfund Green Remediation
                    Strategy (the GR Strategy), which was published in draft form in 2009 and in final form
                    in September, 2010. The strategy outlines 40 action items across three main areas:  policy
                    and guidance development; resource development and program implementation; and
                    program evaluation. The central goal of the strategy is to reduce the environmental
                    footprint of cleaning up contaminated sites by focusing on five core elements:
                         • Energy requirements of the treatment system;
                         • Air emissions;
                         • Water requirements and impacts on water resources;
                         • Material consumption and waste generation; and
                         • Land and ecosystem impacts.
                    In addition to the Superfund Green Remediation Strategy, EPA's Office of Superfund
                    Remediation and Technology Innovation (OSRTI) in the Office of Solid Waste and
                    Emergency Response (OSWER) has published a number of fact sheets and a technology
                    primer, Green Remediation: Incorporating Sustainable Environmental Practices into
                    Remediation of Contaminated Sites, to help guide decision-makers about  the options that
                    are most appropriate for specific site circumstances. Moreover, the Superfund site
                    remediation programs in the ten EPA regions have begun to integrate GR practices at
                    some sites. In some cases regional focus on GRpre-dates the publication  of the GR
                    Strategy. As EPA begins to incorporate GR more broadly, it is important  to ensure that
                    the GR Strategy and related efforts are appropriately focused, widely understood and
                    applied where appropriate, and able to demonstrate and measure key impacts related to
                    the five core elements.

-------
lEc
                     As part of pursuing the GR Strategy, the Superfund program is evaluating the
                     implementation of the strategy itself. Industrial Economics, Incorporated (IEc), with the
                     support of EPA's Evaluation Support Division (ESD) in the Office of Policy, is
                     conducting a "formative" evaluation of the national-level effort. A formative evaluation is
                     conducted early in implementation of the program to assess how program activities and
                     priorities are being implemented, and to ensure that program design and objectives are
                     well-aligned. The purpose of the evaluation is to help focus the GR Strategy's future
                     efforts to advance greener cleanups by examining three main parameters:
                         • Insights from EPA experiences to date in implementing the GR Strategy;
                         • Options for developing a baseline against which to measure EPA progress in
                           implementing the GR Strategy;
                         • Options to assist OSRTI in developing the best metrics for measuring the
                           program's successes in implementing GR practices.
                     Nine evaluation questions address different aspects of these evaluation parameters. The
                     evaluation is designed to shed light on cross-cutting issues in implementing the GR
                     Strategy, and to inform future efforts undertaken by OSRTI to integrate GR and reduce
                     environmental footprints at National Priorities List (NPL) sites. As a formative
                     evaluation, this effort does not focus on "performance" in achieving long-term "program"
                     (i.e., GR Strategy) goals. Therefore, we do not attempt to quantify environmental
                     footprint reductions at sites.
                     The Superfund program envisions the eventual integration of GR as standard business
                     practice in site remediation, and anticipates a future  evaluation that will assess the impact
                     of the strategy. To further this effort, IEc has assisted EPA in developing a
                     comprehensive logic model that identifies the key data needed to  support related
                     activities, outputs, and outcomes of the GR Strategy implementation efforts.

              1.1    OVERVIEW OF  THE  SUPERFUND GREEN REMEDIATION STRATEGY
                     In September 2009, EPA issued its first formal strategy on green remediation for public
                     review.  In September 2010, EPA revised and reissued the strategy, after incorporating
                     comments from the public review. The final GR Strategy outlines nine key actions,
                     which describe 40 specific activities that the program intends to implement to promote
                     green remediation. The actions are separated into three categories:
                         • Policy and guidance development;
                         • Resource development and program implementation; and
                         • Program evaluation.
                     One long-term goal of the  GR Strategy is the eventual integration of GR as standard
                     business practice  in site remediation. The ultimate goal is to establish a process that
                     routinely ensures that the environmental footprints of site cleanup actions are minimized
                     to the extent practical. OSRTI plans to treat the Superfund Green Remediation Strategy as

-------
lEc
                    a "living" document and update the GR Strategy as Agency policy progresses, as
                    activities are modified within the key actions and, as green remediation practices develop.
                    A GR Strategy Activity Tracking Chart, published in February 2011, provides an update
                    documenting the implementation of the components of the GR Strategy. Exhibit 1-1
                    summarizes the  40 activities and their status as of February 2011.
      EXHIBIT 1-1.  SUPERFUND GREEN REMEDIATION STRATEGY OVERVIEW
SUPERFUND GR STRATEGY: SUMMARY OF STRATEGIC ACTIONS (AS OF FEBRUARY 201 1 )
POLICY AND GUIDANCE DEVELOPMENT
Key Action #1 : Clarify the role of green remediation in remedy selection and
implementation
1.1
1.2
Develop OSWER policy on green remediation in remedy selection for remedial
and non-time critical removal actions
Evaluate potentially applicable or relevant and appropriate requirements (ARARs)
Status
Under Development
Under Development
RESOURCE DEVELOPMENT AND PROGRAM IMPLEMENTATION
Key Action #2: Develop a compendium of protocols and tools to help project and
program managers integrate green remediation practices
2.1
2.2
2.3
2.4
2.5
2.6
2.7
Identify green remediation resource needs
Identify additional green remediation information resources
Develop technology-specific assessment tools and fact sheets
Develop green remediation Q&A's
Produce green remediation checklists
Deliver or host green remediation training through the Technology Innovation and
Field Services Division's training infrastructure
Provide site-specific assistance and assistance mechanisms
Key Action #3: Identify options that enable use of green remediation practices
3.1
3.2
3.3
3.4
3.5
3.6
3.7
Identify methods to maximize use of renewable energy with a goal of using 100%
renewable energy to power site operations
Identify methods for increasing energy efficiency
Develop a better understanding of the costs or savings associated with use of
green remediation strategies and practices
Develop a fact sheet on using green power for site cleanup
Identify methods to increase use of renewable energy generated onsite for site
remediation at remote locations
Explore and/or establish funding mechanisms to finance green remediation
research, development, and demonstration (RD&D) and initial deployment efforts
at Superfund sites
Participate in development of a national standards and certification process
Key Action #4: Address air pollutant emissions
4.1
4.2
4.3
Key Acti
applicat
5.1
5.2
Develop a fact sheet on clean fuel and emission technologies
Develop cleanup contract requirements for incorporating clean fuel and emission
technologies
Identify opportunities for recovering and using methane gas emitted from
landfills on Superfund sites
on #5: Develop pilot projects to evaluate and demonstrate green remediation
ons
Develop a database of innovative green remediation pilot projects
Develop and pilot test a green remediation analysis template to help collect
information during various phases of the remediation process at any site
Status
Implemented
Implemented
Implemented
Under Development
Under Development
Implemented
Implemented
Status
Under Development
Under Development
Under Development
Implemented
Under Development
Under Development
Under Development
Status
Implemented
Under Development
Under Development
Status
Under Development
Implemented

-------
lEc
SUPERFUND CR STRATEGY: SUMMARY OF STRATEGIC ACTIONS (AS OF FEBRUARY 201 1 )
5.3
5.4
Incorporate green remediation factors into remedy optimization evaluations
Support the Re-Powering America's Land Initiative by identifying Superfund sites
with outstanding or superb renewable energy potential
Key Action #6: Establish opportunities in contracts and assistance agreements to
identify green remediation practices in selected remedies
6.1
6.2
6.3
6.4
6.5
6.6
Modify EPA contract language to include green remediation practices
Modify contract language to require reporting of selected activities
Develop and periodically update a green remediation contracting tool kit
Develop model terms and conditions for assistance agreements and lAs
concerning site cleanup
Explore additional opportunities to use existing federal agreements and establish
new agreements
Explore and promote opportunities to use local expertise in green cleanups
Key Action #7: Communicate and share success stories and lessons learned among
"implementers" across the Program and the public
7.1
7.2
7.3
7.4
Develop a communication plan
Conduct outreach to contractors and industry
Partner with other federal agencies and state organizations to promote national
use of green remediation strategies
Engage local communities in assessing and implementing green remediation
options
Implemented
Implemented
Status
Under Development
Under Development
Implemented
Under Development
Under Development
Under Development
Status
Under Development
Under Development
Implemented
To Be Initiated
PROGRAM EVALUATION
Key Action #8: Establish a roadmap for evaluating the environmental footprint of a
cleanup at a project level
8.1
8.2
8.3
Analyze existing methods and software tools for evaluating the environmental
footprint of a cleanup
Develop an Agency methodology for evaluating the environmental footprint of a
cleanup
Develop evaluation modules for green remediation strategies
Key Action #9: Evaluate the environmental footprints of Superfund cleanups at a
programmatic level
9.1
9.2
9.3
9.4
Estimate a Program baseline for the environmental footprints of Superfund
cleanups
Establish performance goals, objectives, and measures for the Superfund Green
Remediation Strategy
Develop options for addressing possible gaps in measures or metrics
Characterize the state of practice and implications of life cycle assessment on
Program operations
Status
Implemented
Under Development
Under Development
Status
Under Development
Under Development
Under Development
Under Development
                    EPA Regions, Headquarters, and external stakeholders have issued a number of other
                    green remediation policy and guidance documents both prior to the first draft of the GR
                    Strategy that was released in September of 2009, and since the final GR Strategy has  been
                    published. Exhibit 1-2 provides a timeline noting publication of some of the key green
                    remediation policy documents that form the broader context of GR efforts. One focus of
                    this evaluation will be examining the extent to which the GR Strategy builds on, supports,
                    and aligns with existing efforts.

-------
lEc
      EXHIBIT 1-2.  GREEN REMEDIATION STRATEGIES
YEAR
Apr 2008
Mar 2009
Aug 2009
Aug 2009
Aug 2009
Summer 2009
Sept 2009
Sept 2009
Sept 2009
Sept 2009
Nov 2009
Jan 2010
Feb2010
Feb2010
Sept 2010
Feb 201 1
SOURCE
OSWER
Region 2
OSWER
Region 8
Region 10
Sustainable Remediation
Forum(SURF)
OSWER/OSRTI
Region 6
Region 7
Region 9
Region 5
Region 3
Region 1
Region 4
OSWER/OSRTI
OSWER
GREEN REMEDIATION POLICIES
Green Remediation Primer: Incorporating Sustainable Environmental
Practices into Remediation of Contaminated Sites
Clean and Green Policy
Principles for Greener Cleanups
Green Remediation Policy
Clean and Green Policy
SURF White Paper— Integrating Sustainable Principles, Practices,
and Metrics Into Remediation Projects
Superfund Green Remediation Strategy, Public Review Draft
Clean and Green Policy
Interim Green Cleanup Policy
Greener Cleanups Policy
Greener Cleanup Interim Policy
Greener Cleanup and Sustainable Reuse Policy
Clean and Green Policy for Contaminated Sites
Clean and Green Policy
Superfund Green Remediation Strategy
Superfund Green Remediation Strategy, Activity Tracking Chart
             1.2   SUPERFUND GREEN REMEDIATION STRATEGY LOGIC MODEL
                    To illustrate the various components of the Superfund Green Remediation Strategy and to
                    inform development of specific evaluation questions, EPA has developed a logic model
                    (i.e., a graphical representation of the relationships between program inputs, outputs, and
                    intended outcomes). A logic model synthesizes the key activities of a program into a
                    picture of how it is expected to work. A program logic model helps determine the degree
                    to which a program's activities and other related inputs affect the expected outcomes. In
                    addition, the logic model's outputs and outcomes can help identify potential indicators or
                    measures of performance. As shown in Exhibit 1-3, the key components of the model
                    include:
                         •  Resources: basic inputs of funds, staffing and knowledge dedicated to the
                           program.
                         •  Activities: the specific procedures or processes used to achieve program goals.
                         •  Outputs: the immediate products that result from activities and are often used to
                           measure short-term progress.
                         •  Audience: the groups that the program seeks to influence.
                         •  Short-Term Outcomes: the changes in awareness, attitudes, understanding,
                           knowledge, and skills resulting from program outputs.

-------
          lEc
EXHIBIT  1-3.    SUPERFUND GREEN REMEDIATION STRATEGY LOGIC  MODEL
                                  KCIVICUIM I IUN  £> I KM I CO T
                                                                                                                                          OUTCOMES
                                        ACTIVITIES
            Funding
            EPA Staff
            • OSRTI
            • GR Workgroup
            • 10 Regional
             Coordinators
            Policy
            Resources
            •EO 13514
            • OSWER
             Principles for
             Greener
             Cleanups
            •GR Practices at
             State &
             Regional level
              Peer Network
               (developed
                 through
             implementation)
Policy and Guidance
  Development
• Evaluate potential
 ARARs
Resource
  Development
• Modify contract
 language to include GR
 practices
• Define and refine
 BMPs as appropriate
• Briefing contractors
 and PCs
• Establish mechanisms
 for financing GR
• Define footprint
 methodology	
                                                                   OUTPUTS
Program
  Implementation
•Deliver GR trainings
 (NARPM)
• Communication with
 RPMs, DDs, and PRPs
 through briefings,
 meetings and calls
                                   Program Evaluation
                                   •Establish program
                                    baseline
                                   • Measure footprint
                                                               Identify additional
                                                               GR information
                                                               resources
REC Purchase
Ag reement
Statement
Issue policy on GR
remedy selection
Define and refine
BMPs
Standard contract
language
• Contracting tool kit
Footprint
methodology
Communications
outputs
• Fact sheets
•Checklists
• Qs & As
• Clu-in website
• Case studies
• Research
•Advertising/
 outreach
                           • Baseline & reports
                             on environmental
                             footprint reductions
                                                                                                                SHORT-TERM
                                                                                                                                        INTERMEDIATE
                                                                                                                                                                 LONG-TERM
                                                Contextual
                                                 Factors
              Cannot be too specific with how to implement GR methods because it limits cleanup options
              Regional access to different GR methods is not equal
              Greener cleanup methods are sometimes more costlythan less green methods
                                               Assess consistency
                                               with Regional
                                               Strategies
                                                                                                   Power purchase
                                                                                                   from local utility
Greater awareness
of GR Strategy and
, BMPs

Awareness of
authority (well
defined)
•Enforcement
•Legal
• Contract
Provide site-specific
assistances
assistance
mechanisms

Greater awareness
of Resources
•Clu-in
' -Technical
knowhow (skills)
j

-*
^

Adopt integ rated GR
BMPs for each of 5
core areas
•Project managers
communicating to
stakeholders
I
GR reflected in
ROD, where
appropriate
• Consistent
phrasing
t
Use of contracting
tool kit
• Contractor
notification
• RPM req uest
contractorto
incorporate GR
                                               Greater awareness
                                               of status of
                                               environmental
                                               footprint &
                                               opportunities to
                                               improve (5 core
                                               elements)
Specify/adopt
performance goals,
objectives and
                                                                                                                                      measures
                        Full integration of
                        GR Strategy
                        Energy
                        Use of
                        renewables
                        and energy
                        efficiency
                        Air
                        Reduced
                        emissions
                        Water
                        Reduced
                        consumption/'
                        increased
                        protection
                         Land/eco-
                         systems
                         Minimize harm
                         and protect
                         land for future
                         use
Materials/
waste
Reduce
material
consumption
and waste
generation

-------
lEc
                         •  Intermediate Outcomes: the changes in behavior that are broader in scope than
                           short-term outcomes. Intermediate outcomes often build upon the progress
                           achieved in the short-term.
                         •  Long-Term Outcomes: the outcomes that demonstrate the GR Strategy's overall
                           capability to be effective as well as the overall environmental improvements made
                           through the strategy.
                    For this evaluation, the logic model focuses on the outputs that are most relevant to the
                    Superfund Green Remediation Strategy and the ability to measure its progress. The logic
                    model  activities are organized to reflect the three main categories of actions in the GR
                    Strategy, including policy and guidance development, resource development and program
                    implementation, and program evaluation. The resource development and program
                    implementation category was divided into two sections to allow for easier tracking of
                    outputs and intended audiences. The last column of the logic model outlines the two
                    parallel long-term goals of the GR Strategy:  1) measurable reductions in the
                    environmental footprints of site cleanup actions, and 2) the full integration of the GR
                    Strategy itself into EPA decision-making during Superfund site cleanup.

             1.3    EVALUATION QUESTIONS AND PURPOSES
                    lEc conducted an initial data and document review, and coordinated with EPA to finalize
                    the following evaluation purposes and questions:

             1.3.1 Evaluation Purpose  1:  Assess  EPA experiences to date  in
                    implementing the CR  Strategy
                    1.    Does EPA have clearly defined goals and objectives for the GR Strategy! Should
                         they be refined and improved to enhance usefulness (e.g., for management decision
                         making, planning and budgeting, EPA's Strategic Plan)!
                    2.    Which initial activities or initiatives from the GR Strategy have been most effective
                         in increasing awareness, adoption and/or implementation of GR strategies?
                    3.    How do Remedial Project Managers (RPMs) factor the GR Strategy into their
                         approach to planning site cleanup?
                         •  What GR practices are being implemented?
                         •  What percentage of RPMs are  implementing specific GR practices?
                         •  What do RPMs know about the energy usage at the sites they manage?
                         •  What information do RPMs track on other GR core elements?
                    4.  What effect has the GR Strategy had on the practice of using green remediation
                       techniques at Superfund sites?
                    5.  What lessons have been learned as a result of implementing the GR Strategy at sites?

-------
lEc
                         • What factors affect the ability to implement the GR Strategy at sites? (e.g.,
                          technical issues, cost issues, legal issues, management support, contract
                          provisions, or contractor capabilities)
                         • How is integration of the GR Strategy priorities (e.g., policy guidance, training,
                          and tools) affected by the above factors and experiences to date?

             1.3.2 Evaluation Purpose  2: Determine a baseline against which to
                    measure EPA progress in implementing the CR Strategy
                    6. What options can we identify for developing a baseline?
                         • What has changed since the implementation of the GR Strategy!
                         • When did green remediation become important to site cleanup?
                         • What options are available for quantifying the environmental footprint at sites?

             1.3.3 Evaluation Purpose  3: Determine the best metrics for measuring the
                    program's success in implementing CR practices
                    7. What performance measures are appropriate for measuring the effectiveness of the GR
                       Strategy in achieving intended outcomes at a regional or national level?
                    8. What are the best means for measuring the effectiveness of the GR Strategy in
                       reducing the environmental footprint at sites that have implemented GR practices with
                       respect to the five core elements ofGR Strategy?
                         • What options exist for using qualitative or quantitative measures to assess the five
                          core elements of GR Strategy!
                    9. Where are the primary data gaps and limitations that inhibit a better understanding of
                       the results of implementing the GR Strategy!

             1.4    REPORT ORGANIZATION
                    This evaluation report is organized as follows:
                         • Chapter 2 presents the methodology used in this evaluation.
                         • Chapter 3 presents the evaluation findings, organized by the evaluation purpose
                          and evaluation question. The chapter concludes with a summary of key findings.
                         • Chapter 4 presents conclusions and recommendations to ensure the continued
                          success of the GR Strategy.
                    This report also includes appendices with copies of interview guides, regional survey
                    data, CLU-IN website use data, and list of references.

-------
lEc

                     CHAPTER  2  |   METHODOLOGY
              2.1    EVALUATION DESIGN
                     This evaluation seeks to synthesize available information on the implementation of the
                     Superfund Green Remediation Strategy. As a formative evaluation of the national-level
                     effort, this evaluation is primarily a qualitative assessment of how well the GR Strategy
                     program activities and priorities are being implemented, based on information gathered
                     from the early phases of implementation of the GR Strategy.
                     The information needed to support the evaluation reflects a variety of sources, including:
                         • The Superfund Green Remediation Strategy (draft, final and update to the final
                           GR Strategy);
                         • EPA national and regional data such as surveys conducted during the GR Strategy
                           development, data on website and document access, regional policies and data,
                           and methods and case studies for calculating environmental footprints;
                         • Published studies and external data including existing GR literature, State GR
                           strategies, and private sector efforts such as the Sustainable Remediation Forum
                           (SuRF) White Paper;
                         • Superfund records, including site profiles; and
                         • Interviews with EPA and non-EPA staff involved in green remediation
                           implementation efforts.
                     The analytical approach for this evaluation combines content analysis of interview
                     responses with examination of data from surveys, studies, literature, and databases to
                     answer the evaluation questions. Our evaluation design relies principally on new data
                     collection through interviews with key EPA personnel involved in implementing GR
                     techniques and the GR Strategy (e.g., OSRTI staff, Superfund GR Regional Coordinators,
                     the GR Workgroup participants, RPMs, regional managers and attorneys, other Agency
                     representatives), and other federal  agency and state government officials.  In addition, lEc
                     reviewed existing data including GR literature, site-specific data, and documents and
                     publications specific to the GR Strategy prior to the interviews to inform interview guides
                     and also to resolve issues that arose during the interviews.
                     Exhibit 2-1 on the next page provides a summary of the evaluation questions as they link
                     to the key objectives (purpose) of the evaluation. Exhibit 2-1 also includes a brief map of
                     the key data sources that lEc employed in answering each of the questions.

-------
lEc
EXHIBIT 2-1.  EVALUATION QUESTIONS AND DATA SOURCES FOR FORMATIVE EVALUATION OF THE CR STRATEGY
EVALUATION QUESTION
DATA SOURCES
SUPERFUND GR
STRATEGY
EPA REGIONAL
STRATEGIES,
SURVEYS AND
TRACKING DATA
PUBLISHED
STUDIES AND
EXTERNAL
DATA
CLU-IN WEB
USE DATA AND
CASE STUDY
SITE PROFILES
INTERVIEWS
NOTES ON DATA SOURCE(S):
EVALUATION PURPOSE 1 : ASSESS EPA EXPERIENCES TO DATE IN IMPLEMENTING THE CR STRATEGY
1 . Does EPA have clearly defined goals and objectives for the GR Strategy? Should they be refined and improved to enhance usefulness (e.g. for
management decision making, planning and budgeting, EPA's Strategic Plan)?

X



X
Interviews with Regional
Coordinators, non-EPA staff,
evaluation of CR Strategy
2. Which initial activities or initiatives from the GR Strategy have been most effective in increasing awareness, adoption and/or implementation
of the GR Strategy?

X
X


X
Interviews with Regional
Coordinators, regional attorneys,
front line managers, and non-EPA
staff, Atlanta meeting survey, state
and regional strategies
3. How do Remedial Project Managers (RPMs) factor the GR Strategy into their approach to planning site cleanup?
3a) What GR practices are
being implemented?
3b) What percentage of RPMs
are implementing specific GR
practices?
X

X
X


X

X
X
Interviews with Regional
Coordinators (including some
RPMs), evaluation of CR Strategy
(and Update), Region 3 and 4
surveys, Region 9 tracking list,
Atlanta meeting survey, state and
regional strategies, site profiles,
website use data
Interviews with Regional
Coordinators, Region 3 and 4
survey, Atlanta meeting survey
                                                                                                         10

-------
lEc
EVALUATION QUESTION
3c) What do RPMs know about
the energy usage at the sites
they manage?
3d) What information do RPMs
track on other GR core
elements?
DATA SOURCES
SUPERFUND GR
STRATEGY


EPA REGIONAL
STRATEGIES,
SURVEYS AND
TRACKING DATA
X
X
PUBLISHED
STUDIES AND
EXTERNAL
DATA


CLU-IN WEB
USE DATA AND
CASE STUDY
SITE PROFILES


INTERVIEWS
X
X
NOTES ON DATA SOURCE(S):
Interviews with Regional
Coordinators, Region 3 and 4
surveys, Region 2 tracking
database, and Atlanta meeting
survey
Interviews with Regional
Coordinators, Region 3 and 4
surveys, Region 2 tracking
database, and Atlanta meeting
survey
4. What effect has the GR Strategy had on the practice of using green remediation techniques at Superfund sites?



X
X
X
Interviews with Regional
Coordinators , Superfund records,
Region 3 and 4 surveys, and state
and regional strategies
5. What lessons have been learned as a result of implementing the GR Strategy at sites?
5a) What factors affect the
ability to implement the GR
Strategy at sites (e.g.,
technical issues, cost issues,
legal issues, management
support, contract provisions,
or contractor capabilities)?
5a) How is integration of the
GR Strategy priorities (e.g.,
policy guidance, training, and
tools) affected by the above
factors and experiences to
date?


X
X


X
X
X
X
Interviews with RPMs, Regional
Coordinators, regional attorneys,
front line managers, non-EPA staff,
Contract Specialists, and website
use data
Interviews with Regional
Coordinators (Region 2 and Region
9), regional attorneys, front line
managers, non-EPA staff, and
website use data
                                                              11

-------
lEc




EVALUATION QUESTION
DATA SOURCES


SUPERFUND CR
STRATEGY
EPA REGIONAL
STRATEGIES,
SURVEYS AND
TRACKING DATA
PUBLISHED
STUDIES AND
EXTERNAL
DATA
CLU-IN WEB
USE DATA AND
CASE STUDY
SITE PROFILES



INTERVIEWS




NOTES ON DATA SOURCE(S):
EVALUATION PURPOSE 2: DETERMINE A BASELINE AGAINST WHICH TO MEASURE EPA PROGRESS IN IMPLEMENTING THE CR STRATEGY
6. What options can we identify for developing a baseline?
6a) What has changed since
the implementation of the GR
Strategy?

6b) When did green
remediation become
important to site cleanup?

6d) What options are available
for quantifying the
environmental footprint at
sites?




















X



X








X












X



X








X



X


Interviews with Regional
Coordinators, measurement
specialists, evaluation of GR Strategy
(and Update)
Interviews with Regional
Coordinators, measurement
specialists, evaluation of GR Strategy,
and the SURF White Paper
Interviews with Regional
Coordinators, measurement
specialists, Footprint Measurement
Methodology (Tetra Tech), Region 2
Metrics, Region 9, and Case Studies
EVALUATION PURPOSE 3: DETERMINE THE BEST METRICS FOR MEASURING THE PROGRAM'S SUCCESS IN IMPLEMENTING GR PRACTICES
7. What performance measures are appropriate for measuring the effectiveness of the GR Strategy in achieving intended outcomes at regional or
national level?

















X





X











X

Interviews with Regional
Coordinators, measurement
specialists, SURF White Paper, Region
3 and 4 survey, Region 2 tracking
database, Atlanta meeting survey,
and Case Studies
                                                              12

-------
lEc
EVALUATION QUESTION
DATA SOURCES
SUPERFUND GR
STRATEGY
EPA REGIONAL
STRATEGIES,
SURVEYS AND
TRACKING DATA
PUBLISHED
STUDIES AND
EXTERNAL
DATA
CLU-IN WEB
USE DATA AND
CASE STUDY
SITE PROFILES
INTERVIEWS
NOTES ON DATA SOURCE(S):
8. What are the best means for measuring the effectiveness of the GR Strategy in reducing the environmental footprint at sites that have
implemented GR practices with respect to the five core elements of the GR Strategy?
8a) What options exist for
using qualitative or
quantitative measures to
assess the five core elements
of GR Strategy?

X
X
X
X
Interviews with Regional
Coordinators, SURF White Paper,
Region 3 and 4 survey, Region 2
tracking database, Atlanta meeting
survey, footprint analyses and tools,
and Case Studies
9. Where are the primary data gaps and limitations that inhibit a better understanding of the results of implementing the GR Strategy?


X
X

X
Interviews with Regional
Coordinators, measurement
specialists, SURF White Paper,
footprint analyses and tools
                                                              13

-------
EC
            2.2    STEPS IN CONDUCTING  THE EVALUATION
                   Consistent with the purpose of formative evaluations, this effort was designed to be
                   exploratory, and we did not develop a quantitative analysis of measuring progress or
                   assessing effectiveness. In addition, the purpose of this evaluation is to capture a range of
                   insights and ideas to guide next steps for the program. Therefore, we did not implement a
                   statistically robust survey methodology with random sampling and analysis of statistical
                   significance of results. Instead, our approach uses qualitative assessment methods and
                   integrates data sources as follows:
                   The five broad steps for this  evaluation are:
                        1.   Conduct an initial review of existing survey data regarding implementation of
                            the GR Strategy;
                       2.   Conduct in-depth interviews with EPA regional staff who are actively involved
                            in implementing the GR Strategy, including all regional GR coordinators;
                       3.   Validate and expand on interview responses, assess data from regional surveys,
                            tracking efforts, and footprint methods and EPA's Profiles of Green Remediation
                            case studies;
                       4.   Resolve questions raised and obtain detailed technical information to inform
                            questions, conduct a second round of targeted  interviews focusing on specific
                            skill areas, issues, and perspectives (including non-EPA individuals, non-Federal
                            organizations, and regional attorneys and managers); and
                       5.   Report results.
                   In assessing the results of interviews, we used descriptive statistics as appropriate.  We
                   also verify the strength of key conclusions by using  multiple data sources, with a
                   particular focus on any areas where initial data  collection efforts and verification steps
                   appear to provide conflicting results. The remainder of this Chapter describes the
                   approach in more detail.

            2.2.1  COLLECTION  OF DATA FROM EXISTING DATA  SOURCES
                   Evaluation Purpose 1  is informed primarily through new data collected in interviews of
                   key EPA personnel and individuals familiar with GR practices.  Evaluation Purposes 2
                   and 3, however, rely on published literature as well as interviewee input. Additional data
                   sources also support the issues and perspectives identified in the interviews with EPA
                   personnel for Evaluation Purpose 1. These additional sources are organized into the
                   following four categories: Existing Surveys and Tracking Efforts,  Footprint Analyses,
                   CLU-IN web use data, and Literature and EPA  Publications. The subsections below
                   describe these categories of data sources in more detail.
                                                                                               14

-------
EC
                   Analysis of Existing Surveys and Tracking Efforts
                   lEc analyzed the following EPA surveys and tracking efforts:
                        •  Region 3 Green Remediation 2009 Questionnaire:
                          The survey, which included 46 RPMs managing a total of 190 Region 3
                          Superfund sites, was conducted in 2009 prior to the release of the final GR
                          Strategy in September, 2010.1 Responses to the questionnaire provide information
                          on current GR practices being implemented at Superfund sites in Region 3 in
                          2009, and included:
                                  o   General GR status question for each RPM: Do you manage a site that
                                      is using green remediation?
                                  o   Technical questions linked to specific topics (including some of the
                                      GR Strategy's five core elements)
                                         •   Stormwater control;
                                         •   Wetlands;
                                         •   Land use;
                                         •   Recycling;
                                         •   Energy; and,
                                         •   Long-term stewardship.

                        •  Region 4 2010 Superfund Greener Cleanup Survey:
                          The survey, which included responses from 31 Region 4 RPMs, was conducted in
                          early 2010 and the summary of results was released in May 2011.  The survey
                          goal was to identify what Greener Cleanup (using OSWER policy language)
                          activities are occurring in the region and what actions could be taken to help
                          personnel further implement Greener Cleanup techniques (e.g., trainings). The
                          survey responses provide a snapshot of the type of GR activities occurring in
                          Region 4 and which of the five core elements are being addressed. The survey's
                          questions were categorized by five "Principles for Greener Cleanup" that are
                          identical to the Strategy's  five core elements. Questions under each element
                          included:
                                  o   Have you implemented this principle? (not verbatim)
                                  o   Reason for implementing principle?
                                  o   Did the principle play a role in the selection of a remedy?
                                  o   Did the principle affect the way the remedy was implemented?
                                  o   Did the principle affect post construction activities?
                        •  Additional questions at the end of the survey included:
                   1 A draft GR Strategy was released in September, 2009, and development of the final GR Strategy was ongoing and may have
                    influenced the Region 3 survey results.
                                                                                               15

-------
EC
                                  o  Are you familiar with the EPA Region 4 Clean and Green Policy?
                                  o  Are you familiar with the Superfund Green Remediation Strategy!

                          Region 9 List of GR Activities: The Region 9 list reports GR activities taking
                          place at 21 sites in Region 9.2 The list does not address a specified time frame, but
                          the Region 9 respondent indicated that it was originally developed during 2010
                          and was updated prior to being sent to lEc in August 2011. The regional contact
                          noted that the table may not contain the most current activities occurring in the
                          region and that some of the sites are part of the Brownfields program. However,
                          the table provides information on the type of activities occurring in the  region and
                          which of the five core elements are being addressed. The elements that appear to
                          be covered by the listed activities are:
                                  o  Air Emissions;
                                  o  Energy, and;
                                  o  Material Consumption and Waste Generation.

                         Additionally, the list notes two sites that have begun footprint/life cycle analyses.

                          2010 EPA Green Remediation Coordination Atlanta Workshop Surveys: The
                          October 19th-20th, 2010 workshop was intended to ensure consistency and
                          collaboration across all GR efforts (i.e., program specific and regional efforts).
                          EPA conducted two surveys in relation to this workshop in October 2010. The
                          first survey, dated October 15th, polled EPA Cleanup Program representatives
                          from programs and offices implementing GR strategies at contaminated sites (i.e.,
                          Superfund, RCRA, Brownfields) for their opinions, status, and suggestions on GR
                          in general. The second survey, dated October 27th, polled EPA regional
                          representatives (a majority from the Superfund program). The surveys included
                          questions such as:
                                  Survey of Program  Representatives
                                  o  Is there anything in your program that could be leveraged  to help
                                     move GR forward?
                                  o  Are there aspects of your program that could be obstacles  to moving
                                     GR forward?
                                  o  Do you have a GR training program?

                                  Survey of Regional  Representatives
                                  o  What is your region doing as part of implementing its regional "clean
                                     and green" policy?
                   2 There are 22 activities listed in the table, but only 21 sites are included.
                                                                                              16

-------
EC
                                  o  Does your region have a GR implementation plan?
                                  o  Is your region collecting any measures data?
                                  o  Are any of your project managers using any of the environmental
                                     footprint calculator tools?
                                  o  Has your region implemented or conducted any kind of GR training?

                   The findings from these surveys and tracking efforts primarily support the analysis for
                   Evaluation Questions 1 through 5.

                   Footprint Analyses
                   lEc reviewed the following EPA footprint resources:

                       •  EPA's Footprint Methodology: EPA's "Methodology for Understanding and
                          Reducing a Project's Environmental Footprint" was released for public comment
                          on September 16, 2011. The document outlines specific guidelines and required
                          metrics for performing a footprint analysis at a contaminated site. The document
                          is organized by core element as identified in the GR Strategy. The methodology
                          acknowledges that there are other footprint analysis guidelines and tools that can
                          be used and also does not require RPMs to conduct a footprint analysis on all or
                          any sites. The methodology simply provides suggested guidelines and metrics to
                          use if applicable at a site.
                       •  Footprint Methodology Webinar: The "Greener Cleanups - EPA's Methodology
                          for Understanding and Reducing a Project's Environmental Footprint" webinar
                          was held on August 10, 2011. The primary purpose of the webinar was to update
                          the remediation community and other interested parties on the progress and intent
                          of the Footprint Methodology. The webinar discussed how EPA envisioned the
                          methodology would be used and clarified that it was not intended to be applied to
                          all contaminated sites. The presenters continued to walk through  specific
                          examples for quantifying metrics on site and how to use the methodology as a set
                          of guidelines rather than a template  or tool.
                       •  Case Studies: EPA has been developing 28 case studies of contaminated sites
                          where a footprint analysis, or part of a footprint analysis, has been implemented.
                          These case studies, titled "Profiles of Green Remediation" were last updated in
                          July 2011 and include basic information for which core elements were included in
                          the footprint analysis at each site. All of the 28 contaminated sites are not
                          specifically Superfund sites, but provide references for footprint analysis
                          implementation.

                   The findings from these resources were used to evaluate the overall progress of the GR
                   Strategy's footprint initiative and to identify potential metrics that could be used to assess
                                                                                              17

-------
EC
                   the success of the GR Strategy. lEc used these resources primarily to support the analysis
                   for Evaluation Questions 6 through 9.

                   CLU-IN Web Use Data
                   lEc reviewed CLU-IN Web Use Data provided by EPA in September 2011. The data
                   included download statistics for specific GR Strategy resources (e.g., BMP Fact Sheets or
                   Greener Cleanups Contracting and Administrative Toolkit), attendance statistics for
                   trainings sessions and webinars, and general monthly access statistics for the GR CLU-IN
                   homepage ranging from September 2009 to August 2011. The findings from the CLU-IN
                   data primarily support the analysis for Evaluation Questions 1 through 5.

                   Literature and EPA Publications
                   Because implementation of the GR Strategy is ongoing, it was helpful to examine how
                   the Strategy has evolved from the draft issued in September of 2009 to the final GR
                   Strategy in September of 2010, through the updated Activity Tracking Chart released in
                   February of 2011. lEc  collected and evaluated these documents to better understand how
                   the GR Strategy has progressed over this time period.

            2.2.2  NEW DATA COLLECTION
                   After the initial review of key existing program survey data and the GR Strategy, lEc
                   conducted a number of telephone interviews to support this evaluation.  lEc used the
                   interviews in two ways: 1) to provide central information on specific evaluation questions
                   (e.g., Questions 1 through 6); and 2) to investigate specific data questions from the
                   qualitative analysis of existing survey data.
                   The initial interviewees were selected through a targeted consultation process with key
                   OSRTI personnel. This approach ensured that key "thought leaders" and subject matter
                   experts across EPA regions were captured in the initial data collection. To gain an
                   understanding of current issues related to implementation of the GR Strategy, and to
                   begin the identification process for potential interviewees,  lEc attended the 2011 National
                   Association of Remedial Project Managers (NARPM) Training Program held in Kansas
                   City, Missouri. While at NARPM, lEc solicited input from key EPA representatives to
                   make recommendations for interview candidates for the evaluation.
                   In total, lEc interviewed 28 individuals during the conduct of 24 separate interview
                   sessions (some sessions had multiple participants).  lEc used survey data and program
                   information to craft the initial interviews with Regional GR Coordinators, and also to
                   identify data gaps, anomalies, and inconsistencies among responses which became the
                   focus of follow-on interviews. lEc conducted 10 interviews with GR Coordinators at the
                   regional level, and approximately four interviews with key GR Workgroup members. We
                   then conducted eight follow-on interviews. A list of the interviewees and copies of the
                   interview guides are provided in Appendices A and B, respectively.
                   The interviews were focused  roughly as follows:
                                                                                              18

-------
EC
                   EPA Staff
                        •  Superfund Green Remediation Regional Coordinators
                                   o  Interviews were conducted with at least one GR coordinator from
                                      each region; with several respondents from Region 2 and Region 9.
                                   o  Interviews with GR Regional Coordinators addressed many facets of
                                      GR Strategy implementation but focused primarily on Evaluation
                                      Questions 1 through 6.
                        •  Subject Matter Experts and Key Regional Staff
                                   o  Five interviews targeted EPA staff and other contacts with specific
                                      areas of expertise in implementing the GR Strategy. Specifically,
                                      lEc interviewed two measurement specialists (one from Region 9
                                      and a private sector contractor), one contract specialist from
                                      Headquarters, one regional attorney, and one regional front line
                                      manager. These interviews inform Evaluation Questions 5 through
                                      9.
                        •  Green Remediation Workgroup Participants
                                   o  Four interviews with GR Workgroup participants were used to
                                      follow-up and verify findings from analysis of existing survey data
                                      and interviews of the GR Regional Coordinators, particularly to
                                      inform Evaluation Questions 1  through 6.
                   Non-EPA Contacts
                        •  Other Federal Contacts
                                   o  One representative from the Army Corps of Engineers that partners
                                      with EPA in GR efforts was interviewed, primarily to provide
                                      insights into the extent to which the GR Strategy has been raising
                                      awareness and changing practices outside the Agency (e.g.,
                                      Evaluation Questions  1,  2, 4, and 5).
                        •  State Officials
                                   o  Two state officials were  interviewed to supplement data collection
                                      efforts for the Evaluation Questions 1 through 6, where data gaps
                                      exist. Specifically, these interviews focused on the extent to which
                                      the GR Strategy is supporting state GR efforts.

            2.2.3  REVIEW OF PUBLISHED STUDIES AND EXTERNAL DATA
                   In addition to the interview effort,  lEc relied on existing data and published studies as the
                   secondary sources of information to inform responses to several of the evaluation
                   questions.  Specifically, Evaluation Questions 6, 7, 8, and 9 required the review of data
                                                                                               19

-------
EC
                   related to baseline assessments and footprint analysis to draw conclusions about the
                   current focus of the GR Strategy. In addition, development of responses to a number of
                   other questions involved validating or expanding on data collected in interviews by
                   reviewing external, published information about GR activities.  Below we outline a
                   number of key existing data sources and note the evaluation questions that they inform.

                   State and Regional Strategies
                   Many states and Regions have begun to integrate GR practices at some sites—in some
                   cases regional focus on GRpre-dates the publication of the Strategy. lEc collected and
                   analyzed state and regional strategies to better understand how EPA Regions are
                   implementing GR approaches and to understand the role that the GR Strategy is playing
                   in continuing these efforts. The data collected informs Evaluation Questions 2, 3 and 4.

                   SURF White Paper
                   In late 2007, the Sustainable Remediation Forum (SURF) initiated a study to collect,
                   clarify and express the experiences of SURF members on the incorporation of GR
                   principles. The white paper on "Integrating Sustainable Principles, Practices, and Metrics
                   into Remediation Projects" was published in the summer of 2009. Data collected from
                   this report informs Evaluation Questions 6 through 9. Because SURF membership is
                   primarily composed of corporate members in the United States, evaluation of this report
                   provides a private-sector perspective on GR practices.

                   Footprint Methodology and Case  Studies
                   OSRTI has already undertaken a number of activities related to measurement of the
                   impact of GR on site footprints.  The most comprehensive effort is the development of a
                   methodology for performing footprint analyses at NPL sites.  The methodology was
                   released for public comment in September 2011, and is accompanied by 28 case study
                   sites with comprehensive or partial footprint analyses. lEc uses OSRTFs Footprint
                   Methodology and EPA's "Profiles of Green Remediation" case studies as primary data
                   sources to answer Evaluation Questions 6, 7, 8,  and 9, coupled with information from the
                   life cycle assessment (LCA) literature and other published approaches to assessing site
                   footprints.

                   EPA Website Use Data
                   OSRTI currently tracks a number of website use indicators related to the CLU-IN website
                   and specific GR Strategy documents. lEc worked with EPA to review these data. EPA
                   website use data provides external validation for the interviews and primarily informed
                   Evaluation Questions 3 and 5.

                   Additional Data Resources:  Site  Profiles, etc.
                   lEc also performed a targeted review of other existing data sources (e.g., site profiles, and
                   footprint analyses and tools) that interviews indicated would be helpful to the evaluation.
                                                                                             20

-------
EC
            2.2.4  ANALYSIS AND INTEGRATION OF DATA COLLECTION EFFORTS
                   After collecting all of the data and information described above, lEc went through a
                   process to validate findings across multiple sources. The following bullet points outline
                   the data integration process used for each of the three Evaluation Purposes.
                        • Evaluation Purpose 1:
                                   o  Collect primary data - Interviews with Regional GR Coordinators
                                   o  Review external data sources to refine, expand, and validate
                                      findings from the primary data collection.
                                   o  Develop verified findings that integrate multiple data sources.
                        • Evaluation Purposes 2 and 3:
                                   o  Review existing literature and program information.
                                   o  Interview subject matter experts based on information from
                                      literature review and responses from the primary data collection.
                                   o  Validate and expand findings with information from Evaluation
                                      Purpose 1.
                                                                                               21

-------
EC
                   CHAPTER  3  |   FINDINGS
                   This evaluation aims to explore the GR Strategy's progress to date in advancing greener
                   cleanup and to inform the program's priorities going forward. The evaluation considers
                   three main parameters: assessing EPA experiences to date in implementing the GR
                   Strategy; determining a baseline against which to measure EPA progress in implementing
                   the GR Strategy; and determining the best metrics for measuring the program's success in
                   implementing GR practices.  As a formative evaluation of the national-level effort, this
                   evaluation is primarily a qualitative assessment of how well the GR Strategy program
                   activities and priorities are being implemented, based on information gathered from the
                   early phases of implementation of the GR Strategy.
                   This chapter summarizes the results of our analyses as outlined in Chapter 2. We organize
                   our results first by evaluation purpose, and then by the individual evaluation questions
                   pertaining to that evaluation purpose. For each evaluation question, we briefly note key
                   data sources and analytical approaches that are discussed in more detail in Chapter 2. We
                   follow the  description of data sources with a discussion and tabulation of our findings
                   related to the question. The chapter concludes with a brief summary of general findings
                   for each of the three evaluation purposes.

            3.1    EVALUATION PURPOSE 1:   ASSESS EPA EXPERIENCES TO DATE IN
                   IMPLEMENTING THE CR  STRATEGY
                   To assess EPA's experiences in implementing the GR Strategy, lEc relies on data
                   collected through interviews  and other sources focusing on different aspects of the
                   implementation and integration of the GR Strategy across EPA and in the 10 Regions.
                   lEc designed this data collection effort around the following five evaluation questions:
                       • Evaluation Question 1: Does EPA have clearly defined goals and objectives for
                         the GR Strategy! Should they be refined and improved to enhance usefulness
                         (e.g.,  for management decision making, planning and budgeting, EPA's Strategic
                         Plan)!
                       • Evaluation Question 2: Which initial activities or initiatives from the GR
                         Strategy have been most effective in increasing awareness, adoption and/or
                         implementation of the GR Strategy!
                       • Evaluation Question 3: How do Remedial Project Managers (RPMs) factor the
                         GR Strategy into their approach to planning site cleanup?
                       • Evaluation Question 4: What effect has the GR Strategy had on the practice of
                         using green remediation techniques at Superfund sites?
                                                                                             22

-------
EC
                        • Evaluation Question 5: What lessons have been learned as a result of
                          implementing the GR Strategy at sites?
                   As described in more detail in Chapter 2 - Methodology, the primary findings for this
                   purpose reflect insights collected directly from designated GR coordinators and other
                   staff actively focusing on implementation of GR in each of EPA's ten regions. These
                   contacts represent the staff most clearly aware of the GR Strategy and its use.  To verify
                   and expand on the initial findings for each question, we present relevant information
                   obtained from additional interviews with EPA managers, legal staff, and other contacts
                   specializing in certain areas of GR implementation, as well as data tracking the use of the
                   CLU-IN website, several surveys, and a  site characteristic tracking database being
                   developed by Region 2.3

            3.1.1 SUMMARY OF  FINDINGS

                   Overarching findings related to  Evaluation Purpose 1 are as follows:
                        • Respondents are generally positive about the  GR Strategy structure and purpose,
                          though some difference of opinion exists on how best to present "goals" and
                          objectives. Several respondents feel that a more clearly defined statement of goals
                          could be useful both in increasing awareness of the GR Strategy and in focusing
                          GR Strategy implementation.  A key interest is ensuring that GR efforts maintain
                          momentum in the implementation phase.
                        • Interview respondents have strong, positive views of many key products of the GR
                          Strategy, and feel that these tools and products have been important in facilitating
                          an expansion of GR activities. Respondents are  less able to connect the Strategy
                          itself to site-level actions, and report a limited awareness of the document.
                          However, respondents agree that  the release of the GR Strategy has reached many
                          decision-makers and has facilitated GR by raising its national profile.
                        • RPMs typically do not use the GR Strategy directly in GR implementation, though
                          they document use of many of the tools associated with the CLU-IN web site and
                          other GR Strategy products. Respondents consider the GR Strategy document
                          itself as a more important tool for managers than for RPMs.
                        • It is difficult to identify specific changes in practice associated directly with the
                          GR Strategy, due in part to the limited  time that has elapsed since publication, and
                          in part to the fact that other factors such as regional strategies and costs savings
                          also contribute to GR efforts.  The national GR Strategy and regional policies
                          (e.g., "clean and green") influence each other, and it is therefore difficult to
                          attribute influence to each separately. However,  regional data from surveys does
                          provide a snapshot of activities underway, and suggests regions have increased
                          emphasis  on GR training and outreach as the  GR Strategy has emerged.
                    5 Data were provided by Environmental Management Support (EMS) on August 18, 2011.
                                                                                                23

-------
EC
                        • A range of challenges face the broader implementation of the GR Strategy, with
                          key concerns including the level of funding and support for GR Strategy personnel
                          and project efforts.  Other broader hurdles include a concern about policy and
                          liability uncertainty, and limited participation from managers and other key staff.
                          One overall concern raised in different contexts is the need for momentum and
                          Headquarters signaling about commitment to GRto ensure that progress
                          continues.
                    Below we document the more detailed findings for each  of the five questions contributing
                    to Evaluation Purpose 1.

            3.1.2  Evaluation Question  1:  Does  EPA have clearly defined goals  and
                    objectives for  the  CR Strategy? Should they be refined and improved
                    to enhance usefulness  (e.g.,  for management decision making,
                    planning and budgeting,  EPA's  Strateg/c Plan)?
                    This general question presented a challenge for interview respondents because the GR
                    Strategy is organized around three purposes (policy and guidance development, resource
                    development, and program implementation).  Within those purposes are nine key actions
                    and 40 specific actions, but the  GR Strategy does not present a separate, specific set of
                    goals. In addition, interview participants preferred different  definitions of the term "goal"
                    - with some focusing on measurable, timed objectives and others focusing more  broadly
                    on strategic priorities.
                    To encourage open-ended input in the interviews, lEc did not provide a specific definition
                    of GR Strategy goals and objectives for the respondents,  but discussed respondents'
                    perceptions of the  Strategy's overall goals in the context of the purposes and key actions.
                    The following are  brief summations of respondents' answers to the general question of
                    whether EPA has clearly defined goals and objectives for the GR Strategy:
                        • Respondents in four of nine Regions that addressed this question find the GR
                          Strategy goals and objectives to be fairly well-defined. However, three of these
                          respondents voiced concerns about the inclusion of aspirational and unachievable
                          goals, and all of these cited the target of 100 percent renewable energy (RE) on
                          Superfund  sites as an example of an aspirational goal.4  One regional respondent
                          expressed concern with the purchase of renewable energy credits (RECs) as a way
                          to achieve this goal and  noted that it can be difficult to ensure the quality of the
                          RECs being purchased. These respondents explained that aspirational goals,
                          which they believe are clearly unattainable, could deter from the overall
                    4 EPA's Office of Solid Waste and Emergency Response (OSWER) is in the process of making a national bulk REC purchase to
                    achieve the goal of powering site operations with 100 percent renewable energy. A REC is a certificate that represents the
                    generation of one megawatt-hour (MWh) of electricity from an eligible source of renewable energy. The purchase of RECs
                    facilitates the development of the renewable energy market. OSWER believes a bulk REC purchase to be the most
                    immediate and cost-effective way to power all Superfund sites with renewable energy.
                                                                                                24

-------
EC
                          implementation and integration of the Strategy and GR in general. Respondents
                          indicate that there is some difference of opinion whether RECs are equivalent to
                          installation of renewable energy projects at specific sites.
                        •  Respondents from two regions stated that while the GR Strategy goals and
                          objectives are somewhat well defined, most regional staff, including RPMs and
                          front line managers, are not aware of the goals. One respondent suggested that
                          EPA HQ should focus on spreading awareness and educating regional staff on the
                          goals and objectives of the GR Strategy.
                        •  Finally, respondents in three regions felt that the goals and objectives of the GR
                          Strategy should be more clearly indicated and defined. When the question was
                          posed, these respondents were unable to answer with confidence that they were
                          aware of or could clearly identify the goals and objectives of the GR Strategy. All
                          of these respondents felt that this might indicate a challenge for ensuring that the
                          Strategy would be implemented, and that an effort to identify goals and objectives
                          could provide momentum to the program.
                   In addition, some respondents provided the following insights and suggestions for
                   improvement to the GR Strategy goals and objectives:
                        •  Two respondents noted that their lack of familiarity with the goals and status of
                          the GR Strategy  is a result of their relatively "low profile" in contributing to GR
                          Strategy activities since the publication of the document. Both noted that the
                          general purpose  of the GR Strategy and the key actions are well crafted, but they
                          had not remained informed about progress in implementing key actions. They
                          focused their comments on the difficulty in maintaining momentum to implement
                          changes after milestones like GR Strategy publication.
                        •  One respondent suggested that EPA HQ should provide a summary document
                          (fact sheet) laying out the goals and objectives of the GR Strategy in a more
                          concise manner. They felt that people would be more willing to become familiar
                          with a short concise document rather than a long document.  Summarizing key
                          goals and objectives in a more manageable format could help meet a broader
                          method to ensure effective outreach and education. Two  respondents felt that EPA
                          should reconsider and restructure the goals and objectives. Primarily, EPA should
                          reduce the number of unachievable goals and replace them with more realistic
                          objectives. Goals cited most often as particular examples of this are the aim to
                          provide 100 percent renewable energy (RE) on all Superfund sites, and an aim to
                          reduce impacts on land use and ecosystems.
                   To complement the input from the regional coordinators, lEc reviewed written documents
                   on GR Strategy implementation and survey responses from Regions 3 and 4. We also
                   asked technical expert interviewees for feedback on this question. However, only the
                   regional coordinators had any specific knowledge of or insights  into GR Strategy goals.
                                                                                              25

-------
EC
                   Most respondents, including the regional coordinators, did say that the general focus of
                   the Strategy and the key actions identified were appropriate and well-designed. Overall,
                   the lack of awareness of goals does not appear to indicate any negative impressions of the
                   GR Strategy itself but instead appears to suggest a challenge in implementation.
                   Additionally,  an underlying theme identified in responses to Evaluation Question 1 is the
                   need for more momentum surrounding the GR program and GR Strategy specifically.
                   Respondents explained that Headquarters clearly defining the goals of the GR Strategy
                   could reenergize the program and benefit the Strategy's integration into Superfund
                   Program site cleanup culture.

            3.1.3  Evaluation  Question  2:  Which initial activities or initiatives from the
                   CR Strategy have been most  effective in increasing awareness,
                   adoption and/or implementation  of the CR Strategy?
                   Key respondents to this question were the Superfund GR Regional Coordinators. While
                   not asked directly, interviews of non-EPA contacts provided verification of regional
                   responses to this question.  Data on participation in training sessions and on use of
                   resources on the CLU-IN web site also provided a second important source of
                   information.
                   An initial finding of significance is that Regional GR Coordinators focus on different
                   aspects of "awareness" as it relates to the GR Strategy. To some, the question considers
                   visibility of specific products that have been developed through implementation of the
                   GR Strategy (i.e., CLU-IN website and  BMP fact sheets). Others focused on awareness of
                   the GR Strategy as a document  and policy implementation exercise. Finally, one
                   respondent discussed the success of the initial development of the GR Strategy as an
                   exercise that increases awareness. Detailed insights include:

                   Key Products
                   All respondents stated that the most visible and successful manifestation of the GR
                   Strategy is the set of products presented on the CLU-IN web site. Respondents noted five
                   specific products in identifying  effective activities that facilitated GR implementation:
                        •  BMP Fact Sheets
                        •  NARPM Trainings
                        •  Case Studies (any examples of success are helpful)
                        •  CLU-IN Website as a total resource
                        •  "Doughnut of Remedy  Implementation" (as referred to by respondent)5
                   5 The "Doughnut of Remedy Implementation" refers to the graphic of the OSWER five core elements presented on p.2 of the
                    GR Strategy document.
                                                                                              26

-------
lEc
                     Exhibit 3-1 below summarizes graphically, the number of regional respondents that cited
                     each product as being useful to GR awareness and implementation in their region.
                     It is noteworthy that the interview questions did not include or prompt a specific
                     discussion of GR Strategy products. Respondents readily and quickly identified the set of
                     products as a critical resource in expanding the use of GR approaches, and expressed very
                     positive opinions of the quality and focus of the BMPs in particular.
                     We were unable to confirm these responses with the CLU-IN Website use data due to
                     limited available data.6 However, the CLU-IN use data do indicate that the website and
                     products of the GR Strategy are referenced and used by EPA, other government
                     organizations, and third party individuals/companies.7 We summarize the EPA CLU-IN
                     use data in the Exhibit 3-2 below:

      EXHIBIT 3-1.   PRODUCTS IDENTIFIED AS USEFUL IN FURTHERING ADOPTION OF GR PRACTICES
                                       Useful GR Strategy Products
                         E
                         Q.
                         §3
                         01
                         l/l
                         cc
                         (3
                BMP Fact Sheets

               NARPM Trainings

                   Case Studies

                  Cluln Website

Remedy Implementation Doughnut
                                                            0123456789

                                                                Number of Regions Citing Product
                     6 EPA's contractor for documenting CLU-IN use, has explained that the download data do not account for copies of the
                      documents distributed via email or hard copy distribution. Additionally, the EPA training data may not fully capture all
                      participants for online trainings because multiple users could access the training via a single user ID.

                     7 Non-EPA interview responses confirm this.
                                                                                                  27

-------
lEc
      EXHIBIT 3-2.   CLU-IN WEBSITE DOWNLOAD AND PARTICIPATION STATISTICS 8'9
                 o
                 £
                 QC
                            Use of GR Products on CLU-IN Website
                       Total NARPM Training Attendance

                 O
                 gj>    Total BMP Fact Sheet Downloads
                     Total Contracting Toolkit Downloads
k.   Average NARPM Training Attendance
        (Average per Training)
                     Average Downloads of Each BMP Fact
                       Sheet (Average per Fact Sheet)
                                                                   D Other(Non-EPA
                                                                     Participants
                                                                                      EPA
                                             0    100   200   300   400

                                              Number of Participants or Downloads
                     EPA also provided data on downloads of the BMP Fact Sheets' release schedule
                     ("Upcoming Topics" PDF on CLU-IN website). The data is as follows:
                          •  EPA: 14 Downloads
                          •  Other Government: 5 Downloads
                          •  Non-Government: 413 Downloads
                     These data suggest that individuals maintain interest in the other topics that will be
                     covered by future BMP Fact Sheets and actively check the proposed release schedule.
                     In addition to the NARPM trainings, as presented above, EPA provided participation
                     statistics for CLU-IN web seminars and "On-Scene Coordinator (OSC) Readiness"
                     training sessions. Although the data do not support definitive conclusions about total use
                     ofGR Strategy materials, the information does demonstrate that EPA staff are actively
                     participating and seeking out GR Strategy-based information. The most heavily EPA
                     attended CLU-IN web seminars were:
                     8The "total" values in the table are all inclusive (i.e., Total NARPM Training Attendance is the total attendance for all
                     NARPM trainings).
                     9 For NARPM trainings, the "EPA" category represents only RPMs and the "Other" category represents all other participants
                     at the trainings including either EPA staff and non-EPA government and private sector staff.
                                                                                                   28

-------
EC
                        • "Your Role in Green Remediation Implementation and Case Studies in Green
                          Remediation - This Year's Models and Tools"10 - Specifically the first of three
                          sessions held on December 8, 2010 (EPA had 80 identified participants).
                        • "Green Remediation: Applying Strategies in the Field"11 - Specifically the first of
                          three sessions held on October 8, 2009 (EPA had 53 identified participants).
                   The data for NARPM trainings show that the first three GR NARPM trainings had the
                   best in-person RPM attendance. These trainings are:
                        • "Green Remediation: Opening the Door to  Field Use." July 9, 2008 - 43 RPMs
                          were identified at this training.
                        • "Green Remediation - What's Next." June 3, 2009 - 30 RPMs were identified at
                          this training.
                        • "Your Role in Green Remediation Implementation and Case Studies in Green
                          Remediation: This Year's Models and Tools." May 26, 2010 - 35 RPMs were
                          identified at this training.
                   Although the trend in NARPM attendance does not necessarily suggest a decrease in GR
                   momentum, the CLU-IN use data does not show big growth in NARPM attendance that
                   would demonstrate a clear expansion of interest and awareness. The lack of growth in
                   NARPM attendance is difficult to interpret:  it could be indicative of increased regional
                   trainings, an increased saturation of trained RPMs, or that interest in GR is stabilizing.
                   Additionally we note that these values may understate participation at these events
                   because multiple users can register/view a webinar training presentation under a single
                   ID. Regional respondents noted that that some regions held NARPM training webinar
                   sessions in-house and therefore may have used only one registration ID.
                   EPA has not to date tracked use statistics for the  28 case studies (Profiles of Green
                   Remediation) listed on the Superfund GR CLU-IN website, but this could be tracked in
                   the future.
                   Finally, although not a target audience, non-EPA individuals have expressed a positive
                   reaction to the GR Strategy's products. These respondents cited several of the same
                   resources have been useful in informing their organizations' GR initiatives and staff.

                   Awareness of GR Strategy
                   While GR Strategy products are well-known, respondents in three regions noted that the
                   GR Strategy itself is not well known or used by regional staff such as RPMs. Several
                   respondents also noted that regional policies (i.e., GR, Greener Cleanup, or Clean and
                   Green) appear to have higher profiles in triggering  individual  staff interest in exploring
                   greener cleanups.
                     This was a follow-up web seminar to a 2010 NARPM GR session.

                    1 This was a follow-up web seminar to a 2009 NARPM GR session.
                                                                                               29

-------
lEc
                     In a related finding, respondents in all regions emphasized that "awareness" of GR as an
                     option is complicated because GR is often "common sense," and many RPMs would
                     likely implement cost saving actions at their sites even without the national GR Strategy
                     (and possibly without regional policies).  However, respondents felt that the publication
                     of the GR Strategy has generally raised the national profile of GR and contributed to
                     interest in the subject.
                     Interviews with front line managers confirmed these findings and noted that during the
                     development of the GR Strategy, they observed a growing interest in learning to
                     implement GR. However, the managers noted that after the release of the final document,
                     they witnessed a clear decrease in "buzz" surrounding the initiative.
                     Data on the number of visits to the Superfund GR CLU-IN webpage generally confirm
                     the insights shared by front line managers. The data span the time frame from September
                     2009 to August 2011. The following charts (Exhibits 3-3 and 3-4) present trends in the
                     monthly data. Exhibit 3-3 depicts the number of estimated visits per month, while Exhibit
                     3-4 depicts the percentage of visitors making repeat visits. Both charts include a line of
                     best fit to depict the general trend over time. In Exhibit 3-3, the spike in monthly visits in
                     October 2010 corresponds with the release of the final GR Strategy document. Other
                     spikes in the data typically correspond with the release of the different BMP fact sheets.
                     Currently the data do not allow examination of trends of unique visits or visitors over
                     time.12
      EXHIBIT 3-3.   CLU-IN WEBSITE  MONTHLY ESTIMATED VISITS
                                                Estimated Visits
                     12 EPA does not track data by individual, which currently limits the use of this analysis. However, if desired, it may be
                      possible to implement tracking by individual in the future, assuming that no policy or legal restrictions exist.
                                                                                                  30

-------
lEc
      EXIBIT 3-4.
CLU-IN WEBSITE MONTHLY VISITOR REPEAT RATE
                                           Visitor Repeat Rate (%)
                             jS>   •$>   ^   ^
                    Exhibit 3-3 indicates an overall downward trend in monthly visits to the CLU-IN website
                    implying that the website is getting used less over time. This observation echoes the
                    concern raised by interview respondents that the GR Strategy implementation could
                    potentially lose momentum. However, Exhibit 3-4 shows that the estimated monthly
                    visitor repeat rate is trending upward over time. This suggests:

                           •  The population of prior users is increasing over time as GR is more broadly
                              considered, and;

                           •  Individuals that have previously viewed the CLU-IN website are returning to
                              the website, which may imply that it is a valuable resource to users.

                    One caveat of the EPA CLU-IN Website Use Data is that it may not accurately report
                    EPA staff awareness of the GR Strategy and its products (i.e, trainings and BMP Fact
                    Sheets) because the web site is not the only source of information. EPA's Intranet website
                    contains some of the same information found on the CLU-IN website, and the GR
                    workgroup regularly distributes products by e-mail. Although not conclusive in and of
                    itself, the downward trend in monthly CLU-IN website visits, as depicted in Exhibit 3-3,
                    echoes insights from the interviews with Regional GR Coordinators and other EPA staff
                    that a general need for more focus on the program's objectives and goals might help
                    increase GR  awareness and momentum.
                                                                                             31

-------
EC
            3.1.4  Evaluation Question 3: How do Remedial Project Managers (RPMs)
                   factor the CR Strategy into their approach to planning site cleanup?
                   To answer this question lEc relies principally on responses from GR Regional
                   Coordinators, and supplements and verifies that information with data from regional
                   surveys. In general, responses to this question were consistent. Key themes are:
                       • RPMs do not typically use GR Strategy. Respondents from eight regions
                         explained that RPMs do not specifically consider the GR Strategy in their
                         approach to planning a site  cleanup. Respondents noted that this is not equivalent
                         to saying that RPMs do not consider GR in their cleanups. Instead, RPMs often
                         use regional green cleanup policies as a guide to implement GR techniques rather
                         than the GR Strategy itself.  Respondents believe that RPMs, at most, only have a
                         general familiarity with the GR Strategy, unless they were involved it its
                         development through the Superfund GR Workgroup.
                       • GR Strategy is and should be more of a tool for management. Respondents also
                         noted that they do not expect RPMs to take time to consult the GR Strategy during
                         the remediation process. Additionally, three respondents  specifically noted that
                         they view the GR Strategy as a guide  for management and do not think that RPMs
                         need to be familiar with the document itself. This sentiment was also shared by
                         non-EPA GR specialists. Respondents explained that the GR Strategy document is
                         very helpful, but as an internal strategy.  They felt that the document was too
                         technical and prescriptive to be used as a site-level resource for implementing GR
                         practices.
                       • GR Strategy has been successful in raising general awareness among RPMs.
                         Although  respondents do not feel RPMs are factoring the GR Strategy directly
                         into their site cleanup plans, they do believe that the GR Strategy publication has
                         increased  overall awareness and interest in  GR among RPMs. These respondents
                         believe that the development of the GR Strategy, at the least, has resulted in more
                         RPMs considering GR in the remediation process. All respondents said that their
                         regions have facilitated some form of training in connection to the GR Strategy.
                         Although the EPA CLU-IN data cannot inform whether GR activities and
                         techniques are being implemented and used by RPMs, it  does confirm that RPMs
                         and EPA employees have been actively participating in trainings and downloading
                         materials from the CLU-IN website, as discussed in Evaluation Question 2. Based
                         on the NARPM attendance  data, it seems that RPMs have been most interested in
                         learning about basic integration of the GR Strategy and more specifically
                         information on "Pump and Treat" technologies.
                   Responses to Evaluation Question  3 noted that it is still relatively early in the
                   implementation and integration of the GR Strategy. Although RPMs may be currently
                   involved in  GR activities, it is not always the case that they classify their efforts as GR.
                   This suggests that awareness of GR practices is increasing, but as respondents explained,
                                                                                             32

-------
EC
                    individuals are referencing multiple GR resources aside from the GR Strategy and its
                    products. The following sub questions provide further insight into RPMs current level of
                    GR awareness and the type of GR activities currently occurring:

                    What GR practices are being implemented?

                           Interview Data
                           While six, of the nine respondents that addressed this question, were able to
                           provide one or two examples of specific green remediation activities currently
                           occurring at sites in their regions, these respondents indicated that GR practices
                           are not tracked  formally. Several respondents cited specific projects that
                           identified energy or material savings opportunities, but noted that these activities
                           are attributable  to one or more factors other than the GR Strategy (e.g., regional
                           policies, individuals' interest in GR and site optimization, or cost reduction).
                           While most respondents answered this question at the site level, interview
                           responses on other questions revealed that all regions are considering or
                           implementing at least one program-level change to encourage GR, with activities
                           such as training and implementation of contract language specifying GR.
                           In the remaining three regions, some GR activity tracking effort is currently
                           underway. Each tracking effort varies in period of time and level of detail.  The
                           results of these  tracking  efforts are summarized below.

                           Region 4  2010  Survey Data
                           Early 2010 survey results (released in May 2011) of RPMs in Region 4 provide a
                           snapshot of the  activities in use at sites in that region, and reveal that GR-related
                           activities are not uncommon. However, survey responses echo interview
                           responses in noting that the specific role of the GR Strategy is not always clear. A
                           majority of respondents  in Region 4 that reported implementing GR activities at
                           their sites did not feel that explicit consideration of the five core elements had a
                           significant role  in selecting or implementing a remedy. However, 90 percent of
                           the RPMs that responded to the  Region 4 survey were familiar with the GR
                           Strategy.13
                           Although not attributed to the Strategy or the five core elements, some activities
                           that can be considered GR and are being implemented in Region 4 include:
                                   o  Total Energy Use and Renewable Energy Reuse:
                                       (12 of 31 respondents, or 39 percent)
                                           •    Consider use of optimized passive-energy technology
                                           •    Look for energy efficient equipment
                    5 Note that self-selection bias may be significant here, if only interested RPMs chose to respond to the survey.
                                                                                                33

-------
EC
                                         •   Maintain equipment at peak performance
                                         •   Consider installing renewable energy systems
                                  o  Air Pollutants and GHG Emissions:
                                     (14 of 31 respondents, or 45 percent)
                                         •   Minimize use of heavy equipment
                                         •   Use cleaner fuel
                                         •   Minimize dust export of contaminants
                                  o  Water Use and Impacts to Water Resources:
                                     (17 of 31 respondents, or 55 percent)
                                         •   Minimize fresh water consumption
                                         •   Maximize water reuse
                                         •   Prevent impacts to water quality of nearby water bodies
                                                •   Erosion prevention
                                  o  Materials Management and Waste Reduction:
                                     (19 of 31 respondents, or 56 percent)
                                         •   Use technologies to minimize waste generation
                                         •   Reuse materials
                                         •   Recycle waste materials
                                  o  Land Management and Ecosystem Protection:
                                     (16 of 31 respondents, or 52 percent)
                                         •   Use minimally invasive technologies
                                         •   Use passive energy technologies
                                         •   Minimize habitat disturbance

                          Region 3 2009 Survey Data
                          The survey responses were from 46 RPMs from Region 3 about activities at 190
                          sites in that region, and only 2 RPMs reported using GR techniques at some of
                          their sites. However, some techniques reportedly being used by RPMs could be
                          considered GR even  if the RPM did not classify it as such. These activities
                          include, but are not limited to:
                                  o  Stormwater Control - about 50 sites reported some form of
                                     stormwater control with 23 percent using vegetative swales;
                                  o  Minimize Water Use - about 10 percent of sites reported
                                     minimizing water use; and
                                  o  In-Situ Technology - 39 to 45 percent of sites with a ground water
                                     component reported using In-Situ technology rather than traditional
                                     pump and treat.
                                                                                             34

-------
lEc
                            Region  9 List of CR Activities
                            The Region 9 list of GR activities reports the activities for 21 sites in the
                            Region.14 The list does not include dates, but provides a brief description and the
                            current  status of each activity. Exhibit 3-5 below summarizes the information
                            reported in the Region 9 list.
      EXHIBIT 3-5.   SUMMARY OF REGION 9 LIST OF GR ACTIVITIES
GR ACTIVITY
CATEGORY
Renewable Energy
Footprint/Life Cycle
Analysis
Clean Diesel/Bio-
Diesel
Burning Landfill Gas
Other
GR STRATEGY COKE
ELEMENT
Energy
N/A
Air Emissions
Material Reuse and
Waste Generation
N/A
NUMBER OF
SITES
15
3
1
1
2
STATUS OF ACTIVITIES
8 - In Process
7 - Operational
1 - Planning/Not
Feasible
In Process
Completed
Operational
Operational
                     Overall, responses and survey data reveal that most regions have some GR activities
                     occurring. During interviews, however, only two regional respondents (Region 2 and
                     Region 9) identified use of GR as routinely implemented. Additionally, Regions 3 and 4
                     survey data reveal a high level of GR activity taking place, as well. However, the
                     majority of RPMs in these regions are not considering their activities as GR. In the
                     remaining regions GR activities are in developmental stages, and it does not appear that
                     RPMs specifically incorporate the GR Strategy itself into decision-making, though they
                     appear to access materials and projects.

                     What percentage of RPMs are implementing specific GR practices?
                     Regional responses to this question ranged across regions, with respondents estimating
                     that as few as 10 percent to as high as 90 percent of RPMs are aware of GR practices and
                     implement them where they can.15 However, most responses were concentrated at the
                     lower end of this range, indicating that awareness and use of GR practices is still limited
                     in many regions.
                     The 2009 Region 3 survey results show that four percent of the respondents are
                     implementing GR. Note that these results only reflect activities that the respondent
                     considered GR at the time of the survey. Survey results from Region 4 do not lend
                     M One site is listed twice for two separate GR activities.

                     15 Not all sites have opportunities for GR and there may by RPMs that cannot conduct GR because of that.
                                                                                                 35

-------
EC
                    themselves to this computation. However, the "principle" with the highest activity
                    percentage was "Materials Management and Waste Reduction" with approximately 60
                    percent of respondents conducting some related activity in Region 4.
                    Some of the variability across regions could be attributed to respondents' interpretation of
                    GR practices. This issue is prevalent in the Region 3 survey results. As discussed in the
                    prior sub-question, only two RPMs (managing 3 sites) out of 46 (190 total sites) reported
                    implementing GR techniques. However, although RPMs did not classify their
                    remediation activities as such, several reported activities could be considered GR.16
                    It also appears that regional responses could be significantly influenced by the existence
                    of regional green cleanup policies prior to the release of the GR Strategy, and by regional
                    specific characteristics such as the level of senior management involvement in GR. lEc's
                    analysis of the interview data reveals that regions with Greener Cleanup or GR policies
                    established before the national GR Strategy typically reported higher percentage of RPMs
                    involved with GR activities.

                    What  do RPMs  know about  the energy usage at  the sites they manage?
                    All respondents agreed that information on energy use is available for Fund-lead sites,
                    though it is not routinely collected and tracked, except in Region 2. However, for other
                    categories of sites, a majority of regions explained that the availability of information
                    varies by site and the RPM in charge of the site. Respondents from six regions noted that
                    tracking energy use is becoming more important, and several also noted that the national
                    REC purchase policy has increased the profile of energy use. Specifically, some RPMs
                    are voluntarily trying to track their consumption of energy to assist the national effort.
                    At the regional level, interviews for this evaluation and the 2010 Survey of EPA Regional
                    Representatives following the Atlanta GR Coordination Workshop confirm that only one
                    region routinely tracks and measures energy consumption. However,  Region 2 has
                    received funding from HQ to develop a tracking database that could then be used by all
                    RPMs to track energy consumption and the other core elements at their sites.17 The
                    database would require RPMs to report sites' monthly energy usage. If implemented
                    broadly, the database could provide information for national tracking.

                    What  information do RPMs track on other GR  core elements?
                    Respondents from nine of the regions report having the ability to track some information
                    on the other core elements identified in the GR Strategy. However, for the most part, very
                    little is currently tracked and quantified. RPMs most often track energy consumption, as
                    16 Another factor limiting the use of GR is the limited number of sites that are at appropriate stages of remediation for
                     considering GR. This was noted by several respondents but is not directly linked to the GR Strategy itself.

                    17 The database was created to collect and compile data in Region 2 related to the implementation of the regional Clean and
                     Green Policy and not the national GR Strategy. The Region 2 Clean and Green touchstone practices only address three of
                     the five core elements of the GR Strategy (energy, air emissions, materials and waste).
                                                                                                  36

-------
EC
                   mentioned above, as well as material use and recycling, waste management, and air
                   emissions. Only one respondent mentioned that the region tracks water use metrics.
                   Respondents noted that not all sites have opportunities for using GR practices, so limited
                   opportunities exist for tracking implementation at sites.
                   The Region 2 database that is currently under development would track metrics related to
                   the Region 2's 2010 Clean and Green Policy. Although the database, in its current state,
                   does not require RPMs to report information on all five core elements, several site
                   specific data will be available. More specifically, in addition to energy use, the database
                   will require RPMs to report the following information for their sites:
                        •  Clean Diesel:
                                  o  Type of Equipment being used on site
                                  o  Fuel Volume of total fleet of equipment
                                  o  Total number and type of retrofits on equipment
                                  o  Usage rate of equipment
                        •  Material Reuse,  Reduction, and Recycling:18

                                  o  Amount of materials reused
                                  o  Amount of materials reduced
                                  o  Amount of materials recycled
                                  o  Amount of materials landfilled
                                  o  Amount of materials combusted
                                  o  Amount of materials composted
                   Respondents to the 2010 Survey of EPA Regional Representatives following the Atlanta
                   Workshop confirmed that the database was designed to address four core elements, and
                   did not include the land and ecosystems core element.
                   In responding to these subquestions and other evaluation questions, two cross-cutting
                   themes from the interviews emerged. First, respondents from all regions expressed
                   concern that any significant requirements to track GR impacts could potentially reduce
                   interest among RPMs. As discussed in more detail in Question 5, limited resources (e.g.,
                   time, manpower) represent a significant challenge in GR implementation. As an example
                   of this, one respondent emphasized that RPMs  do not require metrics to do their job
                   successfully and therefore it is more difficult to get people to track these sorts of metrics.
                   Respondents recommended developing a simplified universal system to assist the metric
                   tracking effort, which could also be tied to performance reviews.
                    8 Note that this information will be reported by material type (i.e., aluminum cans, glass, food scraps etc.'
                                                                                               37

-------
EC
                    In addition, a review across regional responses reveals that many regions appear to focus
                    on one aspect of the GR Strategy at one time. One respondent remarked that taking this
                    approach to implementation would allow the Strategy to move further without spending
                    time "reinventing the wheel." Respondents from different regions noted identifying and
                    implementing renewable energy efforts (either through the REC purchase or generation
                    on site), waste reduction and management, and reduction of air emissions through the use
                    of newer technology and clean diesel as  specific regional areas of interest. This practice
                    may be important in considering national-level tracking options and coordination efforts,
                    and may limit the practicality of tracking multiple indicators at all sites.

            3.1.5  Evaluation Question 4:  What effect has the CR Strategy had on the
                    practice of  using green remediation techniques at Superfund sites?
                    Consistent with Question 2 responses, most respondents across regions could not identify
                    or attribute specific changes in regional practice to the GR Strategy. In general,
                    respondents thought that the GR Strategy has been most successful in refocusing and
                    defining GR. They explained that it has provided some momentum to the  GR effort and
                    has assisted in spreading awareness of GR, but they noted three factors that complicate
                    the identification of specific activities with the GR Strategy:
                         •  Not enough time has  elapsed: Respondents generally felt that it is "too soon" to
                           see significant impacts of the GR Strategy. They believe it has had a general
                           positive impact, but ongoing site  efforts typically started before its publication.19
                         •  Focus on regional policies: Regions that have a GR policy stated that their
                           RPMs refer to these regional policies rather than the GR Strategy. The BMP fact
                           sheets and CLU-IN website, rather than the GR Strategy itself, have been useful
                           guides for RPMs when attempting to implement GR principals. Respondents from
                           three regions  also asserted that the GR Strategy has not impacted regional practice
                           because the region was involved  in GR activities prior to the final  GR Strategy.
                         •  Attribution poses challenges: Two respondents felt that GR is generally common
                           sense, and it is difficult to attribute any changes in activities to the GR Strategy or
                           to regional strategies because some RPMs would be implementing or exploring
                           GR activities  and opportunities even without the policies in place.
                    The surveys conducted in Region 3 and 4 in 2009 and 2010, respectively,  are consistent
                    with these responses.20 As discussed in Evaluation Question 3, very few RPMs in Region
                    3 report that they are implementing GR activities in the survey responses. However, the
                    19 Due to limited tracking of GR practices within regions it is difficult to assess which activities were implemented prior to,
                     during, and after the development of the draft and final GR Strategy.

                    20 The Region 2 database and Region 9 list of GR activities do not report the reason (i.e., regional policy or national strategy)
                     for implementing specific GR techniques at sites, but Region 2 states specifically that the database is intended to track
                     data related to the series of touchstone practices outlined in their regional Clean and Green Policy. Region 9 does not
                     provide such a distinction.
                                                                                                  38

-------
EC
                   Region 3 survey was completed prior to the publication of the GR Strategy and therefore
                   does not specifically address the role of the Strategy.  Similarly, a majority of Region 4's
                   respondents who are implementing GR activities do not attribute those activities to the
                   GR Strategy or its five core elements. Again, because the survey was completed several
                   months prior to publication of the final GR Strategy, it is not unexpected that impacts
                   would be limited.
                   Responses to the 2010 Survey of Regional Representatives following the GR
                   Coordination Workshop did identify several activities underway to implement GR in their
                   regions. Although the survey responses do not specifically link activities with the GR
                   Strategy, these activities represent an increased awareness and interest in GR practices at
                   the time of publication. Key examples of the activities identified include:
                        •  GR Trainings;
                        •  Focus on site-specific opportunities for renewable energy development;
                        •  Development of RPM guidance documents and checklists;
                        •  Development of websites with links to resources and contact information for
                          experts; and,
                        •  Added GR language to contracts.
                   The CLU-IN website use data generally confirms the statement that the GR Strategy has
                   been successful in increasing overall GR awareness and providing momentum to GR
                   efforts. As was discussed in Evaluation Question 2, a large (over 20 percent) increase in
                   CLU-IN site visits in October 2010 corresponded with the release of the final GR
                   Strategy document. However, this level of activity later dropped back to the August level
                   of visits, which suggests a decrease in momentum.

                   Training and Awareness
                   Interview respondents echoed the Region 4 2010 survey results in the key area of
                   training. While most respondents did not attribute specific changes in practice to the GR
                   Strategy, all regions report taking part in or delivering some form of GR training and/or
                   GR outreach. Regional  GR training is a key action under the GR Strategy, and represents
                   a change in regional practice. The trainings and outreach identified by regions include:
                        •  Development of a regional GR Website
                        •  Brownfields  conference
                        •  Regional facilitated GR trainings
                        •  GR sessions  and speakers during regional meetings and green seminars
                        •  CLU-IN web seminars:  (3,863 participants across 22 sessions, including 623 EPA
                          participants and 1,013 Other Government participants)
                                                                                              39

-------
EC
                        • NARPM webinars and trainings: (323 total participants across six sessions,
                          including 157 RPM participants)
                        • OSC readiness trainings: (105 total participants across three sessions, including 45
                          OSC participants)
                    Although data do not document trends in participation or overall RPM or OSC
                    participation percentages, participation and interest among EPA staff continues.
            3.1.6  Evaluation  Question 5: What lessons  have been  learned as  a  result  of
                    implementing the  CR Strategy at sites?
                    This question had two distinct themes.  Respondents were asked to discuss the general
                    themes and lessons that they've gained from their experiences with GR and specifically
                    with the implementation of the GR Strategy. They were also asked a specific set of sub-
                    questions related to challenges and opportunities in connection to the GR Strategy.
                    Notably, respondents had difficulty drawing broad conclusions from their experiences
                    implementing the GR Strategy. Specifically, seven often respondents could not cite
                    specific  lessons learned from implementation. This limitation appears to reflect the fact
                    that some regions are in the early stages of implementing GR into  site and regional
                    practices. The insight in Evaluation Questions 3 and 4 that it is too early to document GR
                    Strategy impacts is consistent with the reticence in identifying broad lessons.
                    Three respondents with broad insights about GR implementation shared the following:
                        • GR is not always more expensive;
                        • What may work at one site may not work at every site; and,
                        • Working through the challenges and limiting factors has been a learning
                          experience.
                    These insights are echoed and expanded in an Association of State and Territorial Solid
                    Waste Management Officials (ASTSWMO) document that explained state  experiences
                    implementing GR.21 The document provides a short summary of seven common
                    misperceptions about green remediation, along with a rebuttal for each, including: 1) GR
                    is an additional regulatory burden that agencies will have to impose on projects and
                    responsible parties; 2) GR will divert resources from our primary responsibility of
                    protecting human health and the environment from releases of petroleum and hazardous
                    substances; 3) cleanup is already green. There is no need to change our approach; 4) GR
                    will cost more; 5) the benefits or trade-offs of different GR approaches are  too difficult to
                    assess; 6) responsible parties will use GRto argue for doing no remediation at all; and, 7)
                    we don't have the authority to do green remediation.
                    21 In September 2009 as the draft GR Strategy was released for public comment, the Greener Cleanups Task Force of the
                    Association of State and Territorial Solid Waste Management Officials (ASTSWMO) Sustainability Subcommittee released the
                    Green Remediation: Getting Started by Debunking Some Myths document noting a number of lessons learned and popular
                    challenges related to green remediation.
                                                                                                40

-------
lEc
                    In response to the specific questions about issues or factors affecting the implementation
                    and integration of the GR Strategy, and GR in general, in their regions, respondents had
                    much more to discuss. The responses are organized by sub-question below.

                    What factors affect the ability to implement the  GR Strategy at sites (e.g.,
                    technical issues, cost issues,  legal issues, management support, contract
                    provisions, or contractor capabilities)?
                    Each regional respondent cited a different mix of factors affecting their ability to
                    implement the GR Strategy or GR in general. Key factors included limited resources,
                    limited staff time (including lack of dedicated staff time), costs associated with assessing
                    sites, concerns about legal authority, lack of clear policy addressing some circumstances,
                    and limited support from management, legal staff, and contractors.
                    Exhibit 3-6 below identifies the key factors (challenges) mentioned by respondents and
                    the number of respondents that cited each factor.
      EXHIBIT 3-6.  KEY  CHALLENGES TO IMPLEMENTATION OF THE  GR STRATEGY
                                   Challenges Facing GR Strategy
01
OB
01
15
.c
u
                                         Time
                                         Legal
                                         Policy
                                     Resources
                                         Costs
                                      Technical
                             Attitudes/Education
                                      Contracts
                            Management Support
                                                       1234
                                                      Number of Regions Citing Challenge
                    How is integration of the GR Strategy priorities (e.g., policy guidance, training,
                    and tools) affected by the above factors and experiences to date?
                    Interview responses describing each of the factors identified provide some additional
                    insights.
                    Time and resources - Respondents noted that GR efforts in most regions do not involve
                    dedicated staff. Many respondents expressed concern that RPMs, and regional Superfund
                    programs as a whole, are "stretched too thin," and thus GR becomes a "second-tier
                    priority." Several respondents felt that additional resources would be necessary to fully
                    implement and integrate GR techniques into the regional remediation process.
                                                                                              41

-------
EC
                   Costs - Separate from resources, respondents noted that the cost of GR itself is difficult
                   to justify if it exceeds the cost of other options. This is true both for analysis and
                   implementation of GR practices, and a key concern is the ability to justify GR when
                   demanding cost recovery from PRPs. Respondents suggested that HQ and regions
                   improve RPM and front-line manager education about the costs of GR, to ensure that
                   staff accurately measure both short- and long-term costs and benefits and can identify
                   situations where GR is not more expensive than alternatives.
                   Legal authority - Several regional respondents discussed challenges in convincing PRPs
                   to pursue  GR. Respondents noted a lack of clear legal justification and broader
                   Headquarters support for promoting GR activities during remedy selection and design.
                   Separately, respondents also noted that third party liability concerns limit the interest of
                   some renewable energy developers in using Superfund sites for renewable energy (RE)
                   generation.
                   Front line managers also focused specifically on the challenge of addressing third party
                   liability. They discussed recent efforts to clarify operator liability and encourage third
                   party operators to implement GR projects.
                   Policy direction - In  response to this question, three respondents directly identified
                   limited available policy direction as a hurdle to implementing GR.  Specifically,
                   respondents noted that many managers and RPMs are unwilling to undertake GR efforts
                   without clear indication that HQ will support these actions.
                   More broadly, several regional respondents noted that the policy direction and
                   information is "lagging" somewhat compared with the high-quality technical materials
                   associated with the GR Strategy. Respondents noted that draft policy statements related
                   to GR in remedial design have been circulated but expressed a desire for a more visible
                   role by HQ in reaffirming support for GR by clarifying policy where possible. While this
                   issue was most clearly articulated in the responses to this evaluation question, the interest
                   in more evident HQ support emerged in response to several questions.
                   Interviews with other regional staff (attorneys and front line managers) and non-EPA
                   individuals involved in other GR initiatives confirmed that the lack of policy or explicit
                   direction from EPA HQ limits the implementation of the  GR Strategy. However,
                   respondents cite some individuals who have successfully implemented GR techniques
                   without explicit direction, in part through direct negotiation with PRPs and contractors to
                   incorporate GR practices. Many respondents noted that some PRPs can see the value in
                   GR without it being a matter of law, though legal clarity is critical in the event of
                   litigation.
                   Both regional and HQ staff noted that GR should not become a separate "tenth criteria" in
                   considering remedial design, and several respondents noted that HQ should be cautious in
                                                                                               42

-------
EC
                   providing policy direction to avoid being too prescriptive.22  However, clear input from
                   HQ, including completion of the policy portions of the GR Strategy, will assist in the
                   integration of the GR Strategy into regional culture and will provide a key signal that HQ
                   regards GR as a priority.  Both regional and HQ respondents agreed that HQ is a critical
                   driver of momentum to ensure the success of the GR Strategy.  In addition to specific
                   policy clarification, respondents suggested other HQ actions such as inclusion of GR
                   targets in performance agreements and other incentives. Overall, the importance of HQ
                   involvement in GR is a key theme that emerges across questions.
                   In addition to these main issues, respondents identified the following hurdles:
                        • Obtaining legal support - some attorneys hesitate to pursue GR in the absence of
                          clear legal authority.  This  is separate from concerns about development of clear
                          policy, but respondents  agreed that attorney hesitation could reflect limited policy
                          direction and lack of clear EPA HQ support for the GR Strategy.
                        • Management support and direction - One  respondent noted that GR will not
                          happen if management doesn't state it as a priority. Efforts to work with regional
                          managers to pursue GR may therefore be a key part of expanding implementation.
                        • Contractors - Two respondents noted that contractors may  resist suggestions
                          from RPMs to consider GR. Introduction of clear and concise policies related to
                          contracting on the regional and national level could potentially address this issue.
                        • Technical ability - One respondent noted that an aggregated source of
                          information and training on GR technologies could help RPMs identify and
                          compare GR technologies with traditional  approaches. This would support efforts
                          to work with contractors and PRPs.
                   In addition to the obstacles mentioned during the interviews, according to the 2010
                   Survey of EPA Program Representatives, the Superfund representative(s) felt that there
                   was not enough awareness of the goals and objectives of GR; this result provides
                   independent corroboration for the  responses to Evaluation Question 1.
                   Also, although somewhat outside the scope of this evaluation, one non-EPA interviewee
                   expressed interest in having more  coordination of GR efforts across EPA Offices (i.e.,
                   RCRA and Superfund). The respondent explained that EPA and non-EPA organizations
                   alike would benefit from seeing an EPA-wide GR policy and a single  web-based portal
                   consolidating program efforts. Currently GR resources are spread across programs and
                   are difficult to locate.
                   22 This references the Nine National Contingency Plan Evaluation Criteria under CERCLA.
                                                                                               43

-------
EC
            3.2    EVALUATION PURPOSE  2:  DETERMINE A BASELINE AGAINST WHICH TO
                   MEASURE EPA PROGRESS IN IMPLEMENTING THE CR  STRATEGY
                   This evaluation purpose is to create a starting point for measuring the performance of the
                   GR Strategy implementation. Consistent with the purpose of a formative evaluation, EPA
                   is currently considering options for the development of an appropriate baseline against
                   which to measure progress in implementing the GR Strategy. Evaluation Question 6
                   directly addresses this purpose, and includes three sub-questions:
                      •  Evaluation Question 6: What options can we identify for developing a baseline?
                           o What has changed since the implementation of the GR Strategy!
                           o When did green remediation become important to site cleanup?
                           o What options are available for quantifying the environmental footprint at
                            sites?
                   To determine a baseline for the GR Strategy, lEc first interviewed the designated GR
                   Regional Coordinators and RPMs actively attempting to implement GR in the 10 EPA
                   regional offices. In addition, lEc interviewed key measurement specialists that have been
                   involved in the development of the Draft Footprint Analysis for Environmental Cleanup
                   document. Based on this input and a review of pertinent literature, we provide an initial
                   set of options for considering baselines.
            3.2.1  Evaluation Question 6: What Options  Can We Identify For Developing
                   A Baseline?
                   Developing a clear baseline is an important step in effectively measuring outcomes of the
                   GR Strategy. A number of factors are important to consider when developing a baseline.
                   Ideally a program or strategy has clear initial data and a unique set of activities and
                   metrics that are readily tracked.
                   A complexity of the GR Strategy is that it has been implemented as a unifying approach
                   encompassing some existing efforts, and in some cases it clearly post-dates regional
                   activities. Moreover, a key goal of the program is awareness, and in some cases people
                   are "doing" GR without calling it GR. Therefore, in evaluating responses to this question,
                   we consider:
                       • Timing: Have regions begun to implement GR practices (and when)? Many GR
                         activities implemented in Region 2 and Region 9 pre-date the GR Strategy, while
                         some other regions have not yet formally implemented GR practices, and still
                         others are in early stages of implementation at a limited number of sites. The
                         progress of implementing GR practices may be affected by the stage at which site
                         cleanup is in the "Superfund pipeline" (e.g., Remedial Investigation, Record of
                         Decision (ROD) development, remedial design (RD), remedial action (RA),
                         O&M,  or five-year review). Incorporating GR activities into the ROD allows for
                         greatest opportunities for footprint reductions, because the RD/RA phases are
                         based on specifications described in the ROD. Opportunities to optimize the
                                                                                            44

-------
EC
                           remediation through GR practices may not be available again until the five-year
                           review period.23
                         • Multiple baselines: As multiple types of outcomes (awareness, behavior, and
                           condition) are associated with the program, is it helpful to think about different
                           baselines? Are different baselines appropriate for measuring awareness and site
                           activities? Regional respondents noted that a number of current GR activities are
                           not called "GR," suggesting that awareness of GR is at a different level than site
                           activities.
                         • Tracking: Another factor in identifying a clear baseline is collecting data on
                           current GR activities. It is therefore important to assess the data that regions
                           currently and routinely collect and identify data that regions  could collect.
                    Regional respondents provided three different suggestions for determining a baseline for
                    the GR Strategy.
                         • Date-dependent Baseline: In general, responses indicated that most regions have
                           not yet focused on developing a baseline.  Respondents from eight regions stated
                           that current practices represent a fairly accurate "before GR" baseline, because
                           they are just beginning to implement green remediation efforts. They noted that
                           the GR Strategy and accompanying regional efforts are the driving force behind
                           all identified GR activities from this time forward. While some uncertainty would
                           accompany these estimates (e.g., if projects have been underway), the respondents
                           felt the total impact on metrics would be negligible.
                           In Regions 2 and 9, however, it is clear that significant GR activities pre-date the
                           GR Strategy, and it would be difficult to attribute all future activity to the
                           Strategy.
                           Based on these  insights, one option for a baseline is to exclude these regions
                           (Regions 2 and 9) when measuring site-level activities related to the  GR Strategy,
                           except for specific activities that are clearly connected with the GR Strategy (e.g.,
                           energy purchase policies or contract implementation).
                           A second option is to consider all site-level GR activities as  related to the GR
                           Strategy, at least indirectly, but stop short of "attributing" the impacts of these
                           activities to the GR Strategy. This approach would consider  "contribution" and
                           assume that the Strategy contributes to all activities, without asserting that the
                           Strategy is solely responsible.
                     23 The purpose of a five-year review is to evaluate the implementation and performance of a remedy (on a site-by-site basis)
                     in order to determine if the remedy is or will be protective of human health and the environment.  Five-year reviews should
                     be conducted either to meet the statutory mandate under CERCLA §121 (c) or as a matter of EPA policy.  In general, five-
                     year reviews are required whenever a remedial action results in hazardous substances, pollutants, or contaminants
                     remaining on site.
                                                                                                     45

-------
EC
                         •  Three Separate Baselines (Multiple Baselines): A respondent from another
                           region offered three distinct scenarios to describe the current state of GR practices
                           in the regions. If a baseline were to be measured at the present time each of the
                           regions would fall into one of three categories including,
                                    o   Date is prior to all GR activity. Regions in this category have
                                        integrated little to no GR practices at sites. Eight of the ten regions
                                        generally fall into this category;
                                    o   Region is  in the "opportunistic phase" when GR starts to be
                                        considered in  site cleanup. Regions in this category have integrated
                                        energy and cost saving practices, but are not referring to them as
                                        GR. An accurate baseline measurement of GR practices due to the
                                        GR Strategy would include these practices since they are not
                                        attributable to the GR Strategy. This scenario relays the importance
                                        of determining a baseline for changes in awareness and a separate
                                        baseline for changes in behavior. At least one region potentially falls
                                        into this category, and;
                                    o   GR is main stream. Regions in this category already widely
                                        practice GR techniques and had policies in place to address GR
                                        prior to the release of the national GR Strategy. Two regions fall
                                        into this category.
                           This would provide an accurate and informative assessment of different GR
                           Strategy priorities, including integration of GR throughout the remediation
                           process, but requires significant data collection, and could lead to potential
                           confusion as regions transition from one category to another.
                         •  Results from the tracking database: Finally, Region 2 has been implementing
                           GR since before the GR Strategy was developed. In this situation, current
                           measurements may not accurately capture the region's baseline. However, Region
                           2 has received funding to develop a metric tracking database and it may be
                           possible to identify specific activities associated with the GR Strategy using the
                           data from this effort.24
                    Exhibit 3-7 summarizes the number of regions that fall into each of the three categories of
                    baseline measurement as discussed above and provides a summary of responses to the
                    three sub-questions under Evaluation Question 6.
                    24 The database is intended to track data related to the series of touchstone practices of the Region 2 Clean and Green
                     Policy, which addresses the energy, air emissions, and materials and waste, but does not address water, or land and
                     ecosystems core elements.
                                                                                                  46

-------
lEc
     EXHIBIT 3-7.  OVERVIEW OF FINDINGS RELATED TO BASELINE MEASUREMENT




EVALUATION
QUESTION 6 SUB-
QUESTIONS
Number of Regions
Providing this Answer


What has changed
since the
implementation of
the CR Strategy?





When did green
remediation become
important to site
cleanup?






What options are
available for
quantifying the
environmental
footprint at sites?






REGIONAL INSIGHTS ON BASELINE DEFINITION
SINGLE DATE-
DEPENDENT BASELINE:
ASSUME ALL ACTIVITY
AFTER SPECIFIC DATE IS
RELATED TO THE CR
STRATEGY
Eight
Respondents noted that
little has changed "yet"
at the site level since
publication of the
Strategy.
GR activities currently
being implemented
would likely have been
implemented absent
the CR Strategy, but
new activities reflect
increasing awareness.
Responses included
considering, as a
starting point, the
development of the
Strategy in 2009, the
2010 release of the
Strategy, and assertions
that "it is still not
considered important."
Respondents felt that
limited baseline data on
footprints exists, with
the exception of Fund-
lead site energy use.

Survey results from
Regions 3 and 4
generally corroborate
this; data are collected
at specific sites but
footprint data are not
collected in any
standardized way that
would support broad
baseline development.
ESTABLISH THREE
BASELINES FOR REGIONS
DEPENDING ON GR
STATUS AND ASSUME
DIFFERENT GR STRATEGY
IMPACT LEVELS
One

The respondent
explained that current
projects reflect
opportunistic actions,
and felt that the CR
Strategy should identify
time frames for
different types of
impacts in each region
as GR efforts evolve.



Respondent explained
that it was difficult to
pinpoint an exact time
because each effort
evolved differently.







The respondent did not
have any suggestions for
quantifying the
environmental footprint.






TRACK SITE-
SPECIFIC GR
STRATEGY
ACTIVITIES USING
TRACKING
DATABASE
One


The respondent
did not address
this question.





The respondent
did not address
this question, but
the regional GR
policy predates
the CR Strategy.


The respondent
did not have
suggestions for
quantifying site
footprints.
However, the
region is
developing a
metric tracking
database, and
may have
suggestions when
the database is
complete.

                                                                             47

-------
EC
                   Overall, the interview responses to this question and the data review suggest the
                   following:
                       •  Site-level outcomes may require two baselines to clarify attribution. For site-
                          level activities, a single uniform baseline across regions may not be as robust as a
                          dual baseline if EPA wishes to attribute activities specifically to the GR Strategy.
                          This baseline would assume that eight regions are "new" at GR (and all GR
                          activities from this time on can be linked to the GR Strategy), and the remaining
                          two regions can only be considered responding to the GR Strategy if they report
                          significant and specific changes in activity. If EPA considers only a more general
                          "contribution" approach, then a single baseline of 2009 or 2010 might be an
                          appropriate starting point.
                       •  Program-level outcomes may use single  date-driven baseline. For broader
                          changes in awareness and integration of GR tools and techniques throughout the
                          Agency, a single national baseline is more feasible. While interview and survey
                          respondents report undertaking activities that are "GR," the responses to this
                          question and prior questions suggest that broad awareness of GR as a concept and
                          practical approach are clearly linked to the development of the  GR Strategy.
                       •  Regions have not collected baseline footprint data. Because very little
                          information about GR activities is currently tracked across regions, identifying a
                          broad and reliable baseline "site footprint" is difficult. It is possible to measure the
                          environmental effects (for four of the five  core elements) at each site, but
                          resources  (e.g., time, manpower) for conducting measurements are scarce and no
                          standard method for tracking such data currently exists. The exception to this may
                          be the documentation of typical energy use, because data from Fund-lead sites
                          could provide a basis for estimation. However, a measured baseline  footprint
                          might be unnecessary if EPA can develop  reliable metrics that quantify "typical
                          footprint impacts" associated with specific GR practices (see Evaluation Question
                          8 below).

            3.3    EVALUATION PURPOSE 3:  DETERMINE THE  BEST METRICS FOR
                   MEASURING  THE  PROGRAM'S SUCCESS  IN IMPLEMENTING  GR PRACTICES
                   To examine the best metrics for measuring the program's success in implementing GR
                   practices, lEc evaluated a number of existing data sources, and supplemented and
                   corroborated these with interview responses from relevant parties working on GR. To
                   review the metrics for measuring program success, lEc investigated the following
                   questions:
                       •  Evaluation Question 7: What performance measures are appropriate for
                          measuring the effectiveness of the GR Strategy in achieving intended outcomes at
                          a regional or national level?
                                                                                             48

-------
EC
                        •  Evaluation Question 8: What are the best means for measuring the effectiveness
                          of the GR Strategy in reducing the environmental footprint at sites that have
                          implemented GR practices with respect to the five core elements of the GR
                          Strategy!
                        •  Evaluation Question 9: Where are the primary data gaps and limitations that
                          inhibit a better understanding of the results of implementing the GR Strategy!
                   The findings for this evaluation purpose are based primarily on evaluation of the
                   following existing data sources: surveys from Region 3 and Region 4, the survey
                   following the Atlanta Green Remediation Meeting held in October 2010, the Region 2
                   database, the Region 9 GR activity tracking list, the Sustainable Remediation Forum
                   (SURF) White Paper on Integrating Sustainable Principles, Practices and Metrics into
                   Remediation Projects, the draft Footprint Analysis for Environmental Cleanup, and other
                   footprint analyses and tools.
                   In addition, the review of existing data was supplemented with information gathered in a
                   first round of interviews with RPMs and GR Regional Coordinators active in promoting
                   GR practices in each of the 10 EPA Regions, along with a second round of interviews
                   with measurement specialists familiar with metrics and measurement techniques. The
                   second round of interviews also includes other non-EPA professionals working in state or
                   other federal agencies on GR activities.

            3.3.1  SUMMARY  OF  FINDINGS
                   Overarching findings related to this evaluation purpose are as follows:

                        •  Identification of performance measures for the  GR Strategy: The logic model
                          associated with the GR Strategy (Exhibit 1-3) suggests that a suite of appropriate
                          performance measures (metrics) for program performance would include: 1)
                          specific metrics identifying awareness (near-term outcomes), changes in behavior
                          (medium-term outcomes) and changes in site practice and impacts (long-term
                          outcomes) and 2) metrics for each type of outcome that allow EPA to assess the
                          extent to which the GR Strategy is effectively implemented and successful in
                          integrating GR principles throughout the remediation process.  Another key
                          feature of successful metrics is ease of data collection and analysis. A detailed list
                          of possible metrics is presented in the results for Question 7.
                        •  Identification of metrics for assessing site footprints: EPA's efforts to craft
                          and test a footprint methodology to support GR activities at sites provides a
                          comprehensive set of metrics that map to four of the five GR Strategy core
                          elements. A number of these metrics appear consistent with other sources and
                          may be able to be  adopted with limited additional effort.  In other areas,
                          particularly in the area of land and ecosystem protection, practical options may be
                          limited to basic process metrics (e.g., reviewing sites for critical or sensitive
                          habitats) or qualitative descriptions.
                                                                                               49

-------
EC
                        •  Key data gaps and challenges:  Direct responses to this question were very
                          limited, but a number of themes emerged in both data reviews and interview
                          responses to other questions. One concern was the reluctance of EPA staff in
                          many regions to conduct footprint analyses. While some interview responses
                          noted that resources for conducting the analyses were an issue, a broader concern
                          appears to be uncertainty about the defensibility of using these analyses in
                          negotiations.
                          A related, key concern noted by several regional respondents is the need for more
                          clear policy guidance on implementation of GR practices, to serve as both a guide
                          and a "signal" from Headquarters to reticent regional staff.  Interviewees,
                          including two subject-matter experts, also noted HQ  policy guidance is
                          constrained by the statutory authority that is available to require GR practices in
                          cleanups, and by the need to ensure appropriate flexibility for regions to
                          implement the program as they see fit.  Given the constraints, it appears unlikely
                          that HQ will issue guidance as prescriptive as some regional respondents
                          suggested. The challenge for HQ is to determine how to provide information that
                          will continue to enable and motivate regions to implement GR practices at site
                          cleanup, and will provide assurance that HQ continues to consider GR a high
                          priority.
                   It appears that the GR Strategy has been successful in both raising general awareness of
                   GR and in providing specific practical information and tools for implementation, but it
                   has been more difficult to provide information on the appropriate contexts and
                   approaches for different types of GR activities.
                   Below we document the more detailed findings for each of the three questions
                   contributing to this Evaluation Purpose.

            3.3.2  Evaluation  Question 7: What performance measures  are appropriate
                   for  measuring  the effectiveness of the CR Strategy in achieving
                   intended outcomes  at a  regional or national level?
                   To determine the best metrics for measuring the program's success in  implementing the
                   GR Strategy, lEc used two different approaches,  including the collection of new data
                   through interviews and review of existing program evaluation literature and other data
                   sources identifying specific GR metrics. Consistent with the focus of a formative
                   evaluation, we consider a range of outcome (performance) measures that could measure
                   progress toward goals.25
                   As an initial step, lEc reviewed the GR Strategy logic model to identify activities and
                   outcomes that EPA is seeking to measure, and examined literature to identify criteria that
                   would help describe appropriate metrics.
                   25 Handbook of Practical Program Evaluation p. 101.
                                                                                              50

-------
EC
                   To generate potential metrics for consideration, lEc interviewed the Region GR
                   coordinators, as well as key measurement specialists that have been involved in the
                   development of the draft Footprint Analysis for Environmental Cleanup.
                   lEc then used the Handbook of Practical Program Evaluation to assess each of the
                   metrics listed below. Aside from threshold requirements of validity and reliability, the
                   Handbook describes five other criteria for identifying appropriate measures:26

                        • Meaningful and Understandable Measures - Measures focus clearly on the
                          goals and objectives of the evaluated program. Additionally, the program's
                          intended audience should be able to easily understand the suggested metrics.
                        • Timely and Actionable Measures - Measures should be actionable, focus on
                          results that decision makers can have leverage over, and present dimensions that
                          are directly affected by the program's elements. Additionally, measures should be
                          based on fresh data and provide results in a timely manner.
                        • Practical Considerations and Cost - Measures should be cost-effective to
                          document. Metrics that require new data collection systems and procedures may
                          be less optimal.
                        • Balanced and Comprehensive Measures - As a group, measures should provide
                          a balanced and comprehensive picture of the evaluated program.
                        • Goal Displacement - Measures should not cause managers to alter or sacrifice
                          the program's goals (e.g., by focusing on "bean counting" at the expense of best
                          practices).
                   The metrics identified in Exhibit 3-8 below consider the first three criteria described.
                   Potential challenges for measuring these metrics are highlighted in the last column of the
                   table. The last two criteria represent considerations  in identifying an appropriate suite of
                   several metrics. These should be considered throughout the metric selection process and
                   evaluated once all available data are collected.
                   Insights from interview responses: In general, the regional respondents considered this
                   to be a "difficult question" and could not offer many suggestions for appropriate
                   performance measures to assess the  effectiveness of the GR Strategy in achieving national
                   or regional level outcomes. One regional respondent suggests avoiding "green beans" or
                   "counting" green activities as the sole metric to evaluate program success without
                   encouraging the actions with the most impacts.27
                   Responses from the measurement specialists echoed the difficulty in measuring program
                   success. One respondent stressed that tracking the use of the footprint methodology is one
                     Handbook of Practical Program Evaluation p. 110-111.

                   27 Under a system focused on counting green beans, people may be motivated to do as many "green" activities as possible,
                    which may hinder the overall goal of site remediation and protection of human health and the environment.
                                                                                                51

-------
EC
                   metric that could be used to assess implementation of the GR Strategy, but that
                   quantifiable reductions in environmental footprint is the key metric. All respondents
                   remarked that it would be interesting to track the number of sites that complete an
                   environmental footprint analysis and the number that use the Footprint Methodology
                   outlined in the Footprint Analysis for Environmental Cleanup document in order to assess
                   the implementation of the GR Strategy.
                   Insights from literature:  The Sustainable Remediation Forum (SURF) White Paper
                   provides a framework to achieve sustainable remediation, which outlines activities in the
                   following three categories: technical resource integration, cooperative communication,
                   and outreach and recognition. Under each category the paper provides examples of
                   metrics that could be used to assess performance. Technical resource integration includes
                   the development and acceptance of a sustainability framework, technical and regulatory
                   guidance documents, pilot studies and research, lessons learned and case studies, and
                   technical stewardship.
                   These resources track well with the intended outputs and outcomes of the GR Strategy.
                   Cooperative communication at the project and Agency levels can include BMPs, fact
                   sheets and other publications, meetings and trainings, as well as dedicated attention to
                   stakeholder questions and concerns. Finally, the paper notes that outreach and recognition
                   activities will provide momentum for green remediation activities. Examples include
                   publications, participation at conferences, maintenance of a central Web site as a
                   repository for green remediation activities, and the establishment of awards for creative
                   and sustainable projects. Analogous to the metrics included in the SURF White Paper,
                   Exhibit 3-9 contains a list of GR Strategy-related measures that may be used to assess
                   performance.
                                                                                               52

-------
lEc
     EXHIBIT 3-8.   IDENTIFICATION AND REVIEW OF POTENTIAL PERFORMANCE MEASURES

OUTCOMES













LOGIC MODEL
OUTCOMES


Status of regional
policies









POTENTIAL
METRIC
DATA
SOURCES
DATA COLLECTION
REQUIREMENTS
Measure of n . .
. Review and
,. . . Comparison Medium (Would be
coPn°sisCtenc of a" easy to do' but
with the
Strategy over
time
Development of
regional
implementation
plans
Regional
policies and
the Strategy
Regions
Use of CLU-IN
Website and EMS CLU-IN
only goal)
Medium (requires
data collection
form regions)


downloads of Website Use Low; data available
specific Data
materials


Attendance at EMS CLU-IN
NARPM and Website Use Low; data available
other Trainings
Awareness of CR RPM survey
Data



Strategy & BMPs responses
Short-Term: regarding Regional High (would require
Changes in awareness of RPM Surveys survey)
Awareness and
Attitudes
























Strategy over
time
Number of CR


Strategy
actions/product HO
s completed
and published
to web
Use of CLU-IN
Website and
Posted Strategy
Products
Attendance at


EMS CLU-IN
Website Use
Data

EMS CLU-IN


Low; data available
but not direct
indicator of
awareness

Low; data available


NARPM and Website Use Low; data available
Awareness of other Trainings Data
resources and
site specific Number of
ass/stance



phone calls Qr (Depends
placed to .. 3 .. v ., .....
.,. . other on the availability
'^about^S '"formation of a call log or if
.,. on amount the information is
speci ic Q<: requestecj tracked some other
. . assistance way)
and /or "
resources


                                                                              53

-------
lEc

OUTCOMES



















LOGIC MODEL
OUTCOMES








Awareness of
footprint








Interest in GR
POTENTIAL
METRIC
DATA
SOURCES
Use of Pilot
Studies and EMSCLU-IN
DATA COLLECTION
REQUIREMENTS


Footprint Website Use Low; data available
Assessment
CLU-IN Pages
Number of
participants and
Data



organizations
attending CLU-IN List
Footprint of
Methodology Participants
Webinar and
subsequent
trainings

Number of


Case
Studies,





Low; data available





"footprinted" technical Low; data available
sites

GR Workgroup
.-all
experts in
R9
Hn
Coordination .""".- "^
participation


Low, but not direct
awareness indicator
High (Current








Intermediate:
Changes in
Behavior







Adopt BMPs for







GR reflected in
ROD

Use of Contracting
Tool Kit





Number of
times BMPs are
used on sites






Number of RODs
with GR
language
Use or number
of views of the
Contracting
Tool Kit
Regional
Surveys or
interviews
with RPMs
Does CLU-IN
surveys do not ask
about BMPs.
Surveys and
interviews can be
difficult to
coordinate.)
Medium (If there is
,av*j.a . a feedback section,
ee. . ac those results may
... ...... inform this
the BMP
, question.)
page?
Regional
review and
count of
RODs
EMSCLU-IN
Website Use
Data

Number of Review/
Regional master count of
High; requires
review of RODs
Low; data available



contracts that Regional Medium
contain GR master
language I contracts





                                             54

-------
lEc
OUTCOMES

Long-Term:
Changes in
Condition

LOGIC MODEL
OUTCOMES
Specify and adopt
performance goals
Formalize GR
positions in regions
Full integration of
the GR Strategy

Reductions in
environmental
footprint
POTENTIAL
METRIC
Number of
published
implementation
plans and "hard
targets" in
Regional
policies
Changes to
more
permanent
staffing of GR
positions
Number of sites
with completed
footprint
analyses.
Agreement on
integration of
GR in different
pipeline stages

Number of
"non-G/?
Strategy
specific"
webinars and
NARPM trainings
that include GR
in curricula
Inclusion of GR
practices and
targets in
management
and
performance
requirements
Integration of
cross- program
and cross-
Agency GR
strategies
DATA
SOURCES
Regions
Regions
Regions
HQ, Regions

HQ
Regions
HQ, work
group
DATA COLLECTION
REQUIREMENTS
Medium (requires
data collection
from regions)
Low; data available
High; though this
could be a difficult
metric to measure
Low, but requires
significant work to
achieve and
document success

Low- medium; data
collection reviews
of NARPM materials
Medium; would
require data
collection from
Regions
Low, but difficult
to achieve
Refer to Evaluation Question 8 for a more detailed
discussion on available footprint metrics.
                                             55

-------
lEc
             3.3.3  EXAMPLE:  USE  OF CLU-IN WEB DATA

                    To examine the feasibility of one data set for supporting GR Strategy metrics, lEc
                    reviewed a preliminary subset of the CLU-IN Website use data that track the number of
                    participants for CLU-IN web seminars. Exhibit 3-9 tracks the number of CLU-IN web
                    seminars and the number of participants from 2008 to 2011.28

      EXHIBIT 3-9.   REVIEW OF PARTICIPATION IN CLU-IN WEB SEMINARS


Number of Web Seminar Conferences
Average Number of Participants per Conference
Number of EPA Participants
Number of Other Government Participants
Number of Non-Government Participants
Total Number of Participants
YEAR OF CLU-IN WEB SEMINARS
2008
5
186
93
332
507
932
2009
8
173
254
338
795
1387
2010
4
142
110
142
314
566
2011
5
196
166
201
611
978
                    Exhibit 3-9 shows that the two years with the most participants per seminar were 2008
                    and 2011, coinciding with the beginning of the GR Strategy development in 2008 and the
                    period following publication of the final version of the GR Strategy in September 2010.
                    However, the largest number of seminars took place in 2009, during GR Strategy
                    development. These data do not account for the content of the seminars or outreach
                    leading up to each seminar but continued high participation in future years would indicate
                    awareness of GR.
                    It is also interesting to note that the number of non-government participants were
                    consistently higher than the number of EPA and other government participants. This
                    result may reflect a larger population and audience of non-government participants than
                    government, but it may also suggest that EPA's GR Strategy is achieving awareness of
                    GR outside the Agency. Data on the different categories of non-EPA participants might
                    provide insights on the extent to which EPA is reaching targeted audiences.

             3.3.4  Evaluation Question  8:  What are the best means for measuring the
                    effectiveness of the  CR Strategy  in reducing  the  environmental
                    footprint  at sites that have implemented  GR practices with respect to
                    the five core elements  of the  CR  Strategy?
                    To determine the best means for measuring the effectiveness of the GR Strategy in
                    reducing the environmental footprint at sites, lEc first evaluated the Footprint Analysis
                   for Environmental Cleanups document, which was recently released in draft form by
                    ! lEc also examined data on NARPM training and annual OSC Readiness training, but we do not separately discuss results.
                                                                                            56

-------
EC
                     EPA's Office of Superfund Remediation and Technology Innovation GR Team in
                     September 2011. The document is a robust starting place to evaluate the best means for
                     measuring the GR Strategy's ability to reduce the environmental footprint at sites. The
                     document lists a set of established parameters (metrics) to be quantified and a
                     straightforward methodology for quantifying those metrics for four of the five core
                     elements of the GR Strategy (materials & waste, water, energy, and air  quality).29
                     To identify other possible metrics to measure the environmental footprint at sites, lEc
                     mined site case studies (Profiles of Green Remediation), surveys, regional databases,
                     regional metrics, interview responses, literature and other sources, and compared the
                     findings with the metrics list provided by the Footprint Analysis for Environmental
                     Cleanups30 Exhibit 3-10 arrays the possible metrics that were identified by this review.
                     As one source of data, we found that the use of the footprint methodology is not
                     widespread throughout EPA regional offices since it was only recently  released
                     (September 2011). However, Superfund GR Workgroup members developed best
                     management practices (BMPs) for minimizing the environmental footprint of site
                     activities and have implemented at least one of them at 28 case study sites. These BMPs
                     are in place under a range of cleanup  programs including Superfund,  RCRA, federal
                     facility, brownfield, and state voluntary actions. A table of 28 Profiles of Green
                     Remediation case studies available on the CLU-IN website summarizes the use of these
                     BMPs.
                     An analysis of the case studies shows that the most frequently used BMPs are related to
                     air emissions. Over  82 percent of the  28 sites utilized report reducing fossil fuel use over
                     the course of the cleanup to achieve reductions in GHGs and air pollutants.  Energy-
                     focused BMPs (i.e., energy efficiency and renewable energy use) have been documented
                     at 68 percent of the 28 sites. Over 64  percent of case study sites employed recycling or
                     beneficial use BMPs to manage materials and waste generated on site. Just under half (12
                     sites) of the profiles identify strategies to conserve and reuse water treated at sites.
                     Finally, nine sites include land and ecosystems BMPs,  though no formal metrics for such
                     activities were outlined in the footprint methodology.
                     29 The metrics are used to quantify the total environmental effects of a remedy by core element (e.g., total energy use, total
                      air emissions, total water use, total waste generation/material consumption). In this way one or more potential remedy
                      options can be compared across either the total environmental footprint or by a specific core element. The document does
                      not provide guidance on quantitative metrics or a methodology for evaluating land and ecosystems, but rather stated that
                      this core element would be evaluated through qualitative analysis. The details of this type of analysis are under
                      development.

                     30 Specific existing data sources include: Region 3 Green Remediation 2009 Questionnaire, Region 4 2011 Superfund Greener
                      Cleanup Survey, results from the Atlanta Green Remediation Coordination Meeting in October 2010, the SURF White Paper,
                      Region 2 metrics and database of GR practices, the 28 Profiles of Green Remediation case studies available on the CLU-IN
                      Web site, a table of RPM activities that include GR practices from Region 9, and other footprint analyses and tools (e.g.,
                      SiteWise, Sustainable Remediation Tool,  Greener Cleanups Matrix, Green Remediation Evaluation Matrix).
                                                                                                       57

-------
lEc
                    Exhibit 3-10 reflects the results of this data collection effort by identifying potential
                    metrics and the data sources that report their use. The exhibit includes metrics that are
                    currently being tracked or have the potential to be tracked. The exhibit includes interview
                    responses indicating the availability of metric information summarized at core element
                    level. Overall, our analysis identifies some metrics that are currently in use and other
                    metrics that could be tracked rather easily. Other metrics cited in the footprint
                    methodology appear less common or may be more difficult to track.

      EXHIBIT 3-10. REVIEW OF POTENTIAL METRICS FOR CORE ELEMENTS OF THE CR STRATEGY
CORE
ELEMENT
Materials &
Waste
Water
Energy
Air Quality
Land &
Ecosystems
METRICS
Refined materials used (Ibs)
Percent of refined materials from
recycled or waste material
Unrefined materials used (tons)
Percent of unrefined materials from
recycled or waste material
Hazardous waste generated (tons)
Non-hazardous waste generated (tons)
Percent of total potential waste
diverted from landfill disposal
On-site water used, including source,
use, and fate of the used water (gal)
Off-site water used (gal)
Drawdown of the water table 1 00 feet
from pumping location (feet)
Percent reduction in stormwater runoff
Total energy used (MMBtu)
Percent of total energy use from
renewable resources
Scope 1 Criteria Pollutant emissions
(pounds)
Scope 1 HAP emissions (Ibs)
Total greenhouse gas emissions (Ibs)
CO2e)
Total Criteria Pollutant emissions (Ibs)
Total HAP emissions (Ibs)
Number of applications of clean diesel
applied and amount of ULSD used
(convert to CAP emissions reduced)
No metrics. Qualitative analysis.
Percentage of land reforested for
carbon sequestration
Percentage of land used for community
and utility scale solar
FP'
X
X
X
X
X
X
X
X
X
X

X
X
X
X
X
X
X

X


CASE
STUDIES2
x(8
Super-
fund, 10
others)
x(6
Super-
fund, 6
others)
x (12
Super-
fund, 7
others)
x(14
Super-
fund, 9
others)
x(3
Super-
fund, 6
others)
SURVEYS a
DATABASES3

a, b, c

a, c


a, b, c
b
b


b
a, c


a, c


a, b, c



d
d
d
d

INTERVIEWS4
(* OF REGIONS)
x(2)
x(1)
x(6)
x(3)



OTHER
SOURCES5
j, k
k
j, k
k
j
j
i, J
i, j, k, I
i, j, I

g, k
i, k, I
i, k
j
i
i, j, I
j, I

k
i
h
h
                                                                                               58

-------
lEc
CORE
ELEMENT

METRICS
Percentage of land used for community
and utility scale wind
FP'

CASE
STUDIES2

SURVEYS a
DATABASES3


INTERVIEWS4
(* OF REGIONS)

OTHER
SOURCES5
h
        1 Footprint Analysis for Environmental Cleanups Draft, May 2011
        2 Profiles of Green Remediation available online at 
        3 This column includes the following sources. If a source is not listed in the table above it indicates that we have reviewed the source and identified
        no relevant information.
              1 Region 2 database (not publically available) and list of recommended metrics (available online at
              http:/ /www. epo.gov/reg/on02/superfund/green remediation /metrics, html)
              b Region 3 Green Remediation 2009 Questionnaire
              c Atlanta Green Remediation Coordination Meeting in October 2010 survey
              d Region 4 2011 Superfund Greener Cleanup Survey. The survey asks respondents if "calculator used?" for each principle (core element). If
              there were any "Yes" responses recorded, we assume that some metrics corresponding to the specified principle were measured and used in
              some form of analysis. Note that the survey does not provide specific metrics under each principle.  Therefore we can only match Region 4's
              Survey to footprint methodology metrics at the categorical level.
              6 Region 9 list of RPM activities
        4 This column reports the number of regions that noted, during interviews with lEc, that they currently track or could easily track the categories of
        core elements of the OR Strategy. The interviewees did not cite the specific metrics within each category  and therefore we report the results at the
        categorical level. Interviews were conducted with Regional GR coordinators and RPMs actively attempting  GR in the 10 EPA regional offices.
        5The following sources constitute  the "Other" Category. If a source is not listed in the table above it indicates that we have reviewed the source and
        identified no relevant information.
              ' SURF White Paper, 2009
              g EPA Brownfields, Air and Water Quality Impacts of Brown fields Redevelopment, April 2011
              h EPA OSWER, Opportunities to Reduce GHG Emissions through Material and Land Management Practices, Sept 2009
              The following sources are other publicly available footprint analysis tools. We did  our best matching the metrics listed in the following tools
              to the metrics included in the table above. However, the metrics in the following tools do not perfectly sync with the EPA's footprint
              methodology.
              ' SiteWise available at 
              ' California DTSC Green Remediation Evaluation Matrix available at
              
              k Illinois EPA Greener Cleanups Matrix available at
              . The Illinois EPA's Greener Cleanups Matrix is more of a check list of
              activities than a footprint analysis tool or template and does not require specific metrics to be measured. However,  we assume, based on our
              review of the document, that the metrics indicated in the table above would be available if the matrix, and activities outlined in the matrix,
              are implemented.
              'Air Force Center for Engineering and the Environment(AFCEE) Sustainable Remediation Tool (SRT) available at
              
                           The review of metrics proposed by the Footprint Analysis for Environmental Cleanups
                           draft documents shows that other sources typically highlight and track a subset of these
                           markers. This suggests that the Footprint Analysis metrics list is generally
                           comprehensive and highlights some  of the more commonly used metrics that may be
                           most appropriate for program-level examination. Below we briefly provide additional
                           insights from the interview and data collection process, organized by sub-question.
                                                                                                                             59

-------
EC
                   What options exist for using qualitative or quantitative measures to assess the
                   five core elements of CR Strategy?
                   OSRTFs metrics outlined in the Footprint Analysis for Environmental Cleanups draft
                   document represent a clear set of quantitative options for direct measurement of
                   reductions in the environmental footprint at sites. Review of additional sources and
                   practices failed to identify any additional quantitative metrics that would be broadly
                   useful.
                   In implementing these metrics, respondents to the GR Regional Coordinator interviews in
                   nine regions noted that they may have some information or have the ability to track at
                   least one core element (usually energy).  However, no consistent method is currently used
                   to track success across regions or even across sites. The most common elements tracked
                   or for which some information is available are energy use, air emissions and waste
                   management.
                   Respondents noted that the methods exist to track all four of the core elements outlined in
                   the  Footprint Analysis for Environmental Cleanups draft document, but resources
                   necessary to conduct measurements are scarce. Respondents expressed concern about
                   adding tracking requirements to RPM workloads—one regional respondent noted that
                   RPMs do not need metrics to "do their jobs." Several respondents noted a centralized and
                   simplified tracking system would be important in ensuring participation.
                   One option may be to document "typical sites" and "typical activities" based on case
                   study and other existing data sources, and use these data to estimate reductions in
                   environmental footprint resulting from specific GR practices. This option addresses the
                   concern of limited resource availability for data collection and analysis efforts noted in
                   several interviews.  Under this scenario HQ could develop standard values for typical
                   footprint reductions associated with specific activities and assess regional activities with
                   limited regional data tracking. For example, with sufficient data it may be possible to
                   estimate the average energy savings between two different pump-and-treat systems. In
                   this case it would only be necessary to identify the number of sites that use such systems
                   to calculate total savings.

            3.3.5  Evaluation Question  9:  Where are the  primary  data gaps and
                   limitations that inhibit a better understanding of the results  of
                   implementing the CR Strategy?
                   A key source for the response to this evaluation question was lEc's interviews with
                   regional EPA staff, HQ staff, and outside-EPA contacts. Direct responses to this question
                   were very limited, and typically focused on difficulties developing defensible footprint
                   analyses. However, a brief review of insights in response to other questions  (e.g.,
                   Evaluation Question 5's assessment of barriers) indicates three fundamental areas that
                   affect the ability of EPA to fully integrate GR into the remediation process.
                                                                                              60

-------
EC
                   Concerns about use of the footprint methodology.
                   While agreeing that the core elements are well-focused and the quantitative measure of
                   changes in site operation would be an ideal way to document GR success, respondents in
                   several  regions expressed concern about both the cost of conducting footprint analyses,
                   and also the potential consequences if the data and methods are challenged by PRPs.
                   Specific insights included:
                        • The challenge of finding additional resources to undertake (or contract) the
                          analysis.
                        • A need for support and guidance in "triaging" sites to decide when and how to
                          apply the analysis to ensure that results are not able to be misused (e.g., encourage
                          PRPs to argue for selecting a less-prescriptive remedy because it is the "greener"
                          option).31
                        • Clarity about how best to respond to PRPs with "competing analyses."

                   Weed for policy guidance before implementation.
                   A broader concern expressed by respondents in several regions is the need for HQ to
                   reaffirm the GR priorities. This issue arose in responses to Evaluation Purpose 1
                   questions, and was again stressed in response to this question as a "lack of information."
                   Specifically, several respondents noted that a perceived "lack of guidance" confirming
                   the statutory basis for GR is often  cited by Regional staff who are tentative about using
                   GR. Two respondents specifically stated that the  GR Strategy has provided many useful
                   and practical tools for implementation, but without more specific guidance addressing
                   legal and policy questions, many RPMs and regional managers are reluctant to make
                   significant efforts to implement GR. The forthcoming HQ  guidance on GR in remedy
                   selection may address this need to some extent, but a broader reaffirmation of HQ's focus
                   on GR might also be helpful as part of any forward-looking effort to increase momentum.

                   Weed to "revitalize" communication to ensure GR Strategy implementation.
                   Respondents from three regions also noted that the incentives to stay engaged in the GR
                   implementation process are reduced after significant achievements such as the publication
                   of the GR Strategy, and they are not always aware of what aspects of the Strategy are
                   progressing.  The respondents noted that the quality of their own participation was an
                   issue, and emphasized that without dedicated funding for GR staff in the regions, the GR
                   Strategy effort is by definition something that is addressed after other required
                   responsibilities are met.
                   To  supplement these insights, lEc  examined the green remediation literature to see what
                   other approaches had been addressed.  The Sustainable Remediation Forum (SURF)
                   31 These respondents had not yet reviewed the current Footprint Analysis for Environmental Cleanups draft document; some
                    of these concerns may be resolved.
                                                                                               61

-------
EC
                   White Paper, specifically, identified a number of barriers preventing a broad
                   implementation of green remediation practices. These barriers are not specific to a better
                   understanding of the results of implementing GR practices or the GR Strategy, but they
                   echo several issues raised by EPA interviews. These general barriers include a lack of
                   regulatory guidance, insufficiently defined frameworks and metrics, few financial or
                   other incentives [for PRPs to participate in GR efforts], and the lack of regulatory
                   requirements for the incorporation of green remediation practices in remediation
                   assessments. The SURF White Paper also notes that consensus has not yet been reached
                   on specific remedial approaches and tools at sites.

            3.4    SYNTHESIS OF  KEY  EVALUATION FINDINGS
                   This section provides a synthesis of the key findings of this evaluation organized by
                   evaluation purpose.

                   Consistent with the general objective of the evaluation, we have explored the GR
                   Strategy's progress to date in advancing greener cleanup and to inform the program
                   priorities going forward. The evaluation considers three main parameters: assessing EPA
                   experiences to  date in implementing the GR Strategy; determining a baseline against
                   which to measure EPA progress in implementing the GR Strategy; and determining the
                   best metrics  for measuring the program's success in implementing GR practices.

            3.4.1  Evaluation Purpose 1:  Assess EPA experiences to date  in
                   implementing the CR Strategy
                   Overall, interview respondents were uniformly positive in their opinions of the GR
                   Strategy structure and purpose, though responses identified some differences of opinion
                   in how best to present "goals" and objectives.  Several respondents noted that a more
                   precise goal  statement could be useful both in increasing awareness and focusing further
                   implementation of the GR Strategy.
                   In the strongest finding, EPA and non-EPA interviewees had very positive views of
                   several key products of the GR Strategy, and felt that these tools and products have been
                   a key driver  in  facilitating an expansion of GR activities. Respondents felt that awareness
                   of the GR Strategy document was more limited, though publication of the GR Strategy
                   has facilitated the use of GR by raising the national profile of GR.
                   Interview responses from the regions indicate that RPMs typically do not use the GR
                   Strategy directly in their decision-making for GR implementation, though it is clear they
                   use many of the tools and products developed to support the GR Strategy. The GR
                   Strategy document was identified to be a more important tool for managers than for
                   RPMs.
                   It is difficult to assess the distinct contributions of either the national strategy or regional
                   policies separately, since they influence each other.  A few regional policies informed the
                   GR Strategy, while others may not have been released without the national focus on GR.
                   Examination of regional data from surveys provides a snapshot of activities underway,
                                                                                              62

-------
EC
                   and it is apparent that regions have increased emphasis on GR training and outreach as
                   the GR Strategy has emerged.
                   A range of challenges face the broader implementation of the GR Strategy, with key
                   concerns including the level of funding and support for GR Strategy personnel and
                   project efforts.  Other hurdles include a concern about policy and liability uncertainty,
                   and limited participation from managers and other key staff.

            3.4.2  Evaluation  Purpose 2:  Determine  a  baseline against  which to
                   measure EPA progress in implementing the CR Strategy
                   Interview responses from the regions indicated that most have not focused to date on
                   developing a baseline. Overall most of the regions (eight often) identified that their
                   current practices represent a fairly accurate baseline before the GR Strategy was released
                   because the implementation of GR efforts is just beginning. A complexity of the GR
                   Strategy is that it has been implemented as a unifying approach encompassing some
                   existing efforts, and in some cases it clearly post-dates regional activities (e.g., Region 2
                   and 9). Moreover, a key goal of the program is awareness, and in some cases people are
                   "doing" GR without calling it GR. The findings from this evaluation suggest that EPA
                   consider whether one baseline is adequate to support the program. If EPA wishes to
                   document contribution of the GR Strategy generally, then a single date-driven baseline
                   may be appropriate. To document attribution,  however, use of different regional baselines
                   for site-specific action may be necessary.

            3.4.3  Evaluation  Purpose 3:  Determine  the best metrics for measuring the
                   program's  success in implementing GR practices
                   A review of the logic model associated with the GR Strategy suggests that a suite of
                   appropriate performance measures for program performance would directly assess the
                   short-term (changes in awareness), intermediate term (changes in behavior), and long-
                   term (changes in site practice and impacts) outcomes of the GR Strategy. Metrics for each
                   type of outcome would also allow EPA to assess the extent to which the GR Strategy is
                   effectively implemented and successful in integrating GR principles throughout the
                   remediation process. Successful metrics will also likely require only limited data
                   collection and analysis.
                   Review of existing and emerging tools for calculating environmental footprint suggest
                   EPA's efforts to craft and test a footprint methodology to support GR activities at sites
                   provides a comprehensive set of metrics that map four of the five GR Strategy core
                   elements (excluding land and ecosystems).  Several metrics listed in the footprint
                   methodology appear consistent with other sources and may be able to be adopted with
                   limited additional effort.  The most successful metrics may be those that HQ can estimate
                   using standardized values and limited regional data.
                   Interview responses suggest that a number of key challenges exist for understanding the
                   impacts of GR.  As was noted in Evaluation Purpose 1, a larger issue that arose from the
                   interview process is the identification of the need for policy-level clarity of the GR
                                                                                             63

-------
EC
                   Strategy. The lack of clear direction from EPA providing legal and policy justification for
                   incorporating GR techniques at sites seems to have decreased momentum for moving GR
                   forward in some regions. Other limitations that inhibit a better understanding of the
                   results of implementing the GR Strategy include concerns about resource constraints
                   (e.g., time, funding, manpower), concerns that clear legal authority for requiring GR
                   practices is not well defined, and reluctance on the part of EPA staff in many regions to
                   use the methodology to conduct footprint analyses.
                                                                                             64

-------
EC
                   CHAPTER 4   |   CONCLUSIONS AND SUGGESTED NEXT STEPS
                   Based on our analysis of data collected from interviews and from published and EPA
                   internal sources, lEc's evaluation team offers the following conclusions and suggested
                   next steps for consideration. We believe that implementing these suggested next steps
                   could ensure continued momentum for implementation of the Superfund GR Strategy and
                   help ensure the continued integration of GR principles into OSRTI activities.
                   We note that overall feedback from the interviews about experiences with the GR
                   Strategy has been consistently positive, and available data generally support and confirm
                   interview responses. Therefore, lEc's suggested next steps focus on how EPA may be
                   able to strengthen the GR Strategy and its products to support the successful integration
                   and implementation of the Strategy.

            4.1    EVALUATION PURPOSE  1:   ASSESS EPA EXPERIENCES TO DATE IN
                   IMPLEMENTING THE  CR STRATEGY
                   Based on findings from interviews, regional data, and CLU-IN website materials and use
                   data, lEc concludes that the development and publication of the GR Strategy has had
                   some initial success in spreading general awareness about GR concepts and best
                   practices, educating EPA staff, providing tools for implementation of GR practices, and
                   supporting Agency interest for incorporating GR techniques into site cleanups and
                   remedial plans. However, interview responses suggest that the GR Strategy is at an
                   important transitional point, with a need for clear focus to ensure its continued longevity
                   and success. The timing of this formative evaluation is appropriate because the results can
                   be used to help EPA to focus on next steps for the GR Strategy, to ensure that the
                   momentum that was built during the development of the GR Strategy can be maintained
                   to ensure implementation of GR practices.  Specifically, we note the following:
                       • Even among GR Regional Coordinators and other active EPA participants in GR,
                         the general conclusion is that GR Strategy could benefit from more clearly defined
                         goals and objectives. While current documents focus on actions, it is difficult to
                         quickly identify clear and concise goals that could guide implementation.
                       • Users have a very positive reaction to the range of practical tools and products for
                         GR implementation that have been developed as part of the GR Strategy (e.g.,
                         BMP Fact Sheets and the Contracting  Tool Kit). These products  are regarded as
                         user friendly, practical, and well-focused on common issues faced by RPMs. In
                         contrast, the GR Strategy document is not regularly referenced during site cleanup
                         or remedial design and selection,  but is viewed as an internal document from and
                                                                                            65

-------
EC
                          for management that acts as a high-level guide for developing an implementation
                          plan for GR at Superfund sites.
                        •  To date, the GR Strategy has been most successful in refocusing, defining, and
                          raising awareness for GR in general. Changes in practice due specifically to the
                          GR Strategy are still limited since less than a year has passed since publication.
                          Additionally, since regions also have developed separate GR guidance or policies,
                          it is difficult to clearly attribute practice changes wholly to the GR Strategy.
                          However, interviews and data suggest that, assuming momentum and interest are
                          maintained, changes in practice should become more measurable over time, and
                          will likely reflect use of the various GR Strategy-related implementation tools.
                        •  Key factors limiting the implementation of GR practices include limited resources
                          for dedicated GR staff at the regional level, and concerns about ensuring that GR
                          approaches are also cost-saving or cost-neutral. However, the most widely-noted
                          limitation of the current GR Strategy is the absence of published direction or
                          information from HQ addressing several key policy and legal questions facing
                          practitioners who are attempting to incorporate GR techniques at sites. It appears
                          that clarification could provide both practical information and a continued
                          indication of HQ support for GR.  Moreover, it appears that other limiting factors
                          identified, such as  RPM interest and management support, may be able to be
                          addressed in part by more policy-level information from HQ.

            4.1.1  NEXT STEPS
                   The key conclusions above suggest that a number of near-term actions by EPA could
                   improve the direction and effectiveness of the GR Strategy as implementation continues.
                        •  Focus on clarity of goals and implementation objectives. EPA may want to
                          establish a near-term focus on concrete goal setting to ensure implementation of
                          GR Strategy key actions. At a minimum, this could involve clearly identifying
                          and defining the overarching goals and objectives of the GR Strategy in a concise
                          user-friendly document that can be widely used to promote GR principles. In
                          addition, EPA could provide new structure and focus for the GR Regional
                          Coordinators and other key GR participants by considering a more specific
                          implementation plan for key actions. These actions would address the internal
                          need for continued momentum to support  GR,  and could facilitate broader efforts
                          to expand awareness and acceptance of GR among regional staff and managers.
                        •  Continue emphasis on practical tools for GR implementation. A continued
                          focus on development of high-quality, practical products and resources for using
                          GR techniques, especially those with site-specific applicability, will help further
                          implementation of GR practices. Moreover, use of these products provides a clear
                          indication of changes in awareness of GR and implementation of GR practices.
                                                                                              66

-------
EC
                        •  Increase focus on policy and legal information and tools, or on other HQ
                          "signaling." As a complement to the well-received practical implementation
                          tools, EPA may be able to support integration of GR practices by developing
                          materials that address key policy and legal questions. This may include, but is not
                          limited to, completion of the Considering Green Remediation Measures in the
                          Remedy Selection Process Q&A document currently underway.  In concert with
                          other goal-setting priorities, an increased focus on practical assistance for
                          addressing policy questions may be useful both in overcoming regional concerns
                          about both specific policy issues and in reaffirming the general commitment to
                          GR. Finally, expressions of HQ  support for GR as a priority represent signals that
                          are important to many front-line regional staff.

            4.2    EVALUATION PURPOSE  2: DETERMINE A BASELINE AGAINST WHICH TO
                   MEASURE EPA  PROGRESS IN IMPLEMENTING THE CR STRATEGY
                   Development of a clear baseline is a critical step for a new program focusing on
                   measuring its  success.  In the context of the effort to implement GR practices, the recent
                   publication of the GR Strategy itself provides one possible starting point for measuring
                   future changes. However, because the GR Strategy was developed over several years and
                   involved several concurrent regional efforts, it is important to distinguish between "new"
                   activities and those that were under development. Moreover, the GR Strategy has a dual
                   purpose to both physically reduce the environmental footprints at sites, and improve
                   awareness and integration of GR principles throughout the Superfund program.  This dual
                   purpose adds complexity to baseline development process. Below we summarize the
                   conclusions for this evaluation purpose  and discuss suggested next steps.
                        •  Regional interviews confirm that most regions do not currently track GR activities
                          across sites in any way that could be used to develop a reliable baseline "site
                          footprint." With the exception of energy use, little information related to the five
                          core elements is tracked consistently across sites.
                        •  In the absence of existing baseline information, a time-defined baseline is likely to
                          be most appropriate if attribution is a goal.  However, to capture site-level
                          activities, a dual baseline may be more robust than a single uniform baseline
                          across regions.  In eight of the ten regions, GR activities appear to be in beginning
                          stages, and one approach that appears reasonable is to consider all future GR
                          activity as an outcome of the GR Strategy. The remaining two regions, however,
                          have pursued GR implementation since before the development of the GR
                          Strategy; site-level activities in this  regions would only be included if they report
                          specific and significant changes  that are directly attributable to the Strategy.  If
                          EPA prefers to  measure the total change in GR activity without considering the
                          specific role of the GR Strategy  (a contribution analysis) then a national time-
                          defined baseline is appropriate, and likely more feasible to implement.
                                                                                              67

-------
EC
                       • A single national baseline could be used to capture broader changes in awareness
                         and integration of GR tools and techniques throughout EPA. While many regions
                         report undertaking GR without calling it GR, the findings of this evaluation
                         suggest that broad awareness of GR as a concept are clearly linked to the
                         development of the GR Strategy.

            4.2.1  NEXT STEPS
                       • As EPA considers appropriate metrics for measuring program performance, it will
                         likely be necessary to employ two baselines. We suggest that EPA consider the
                         following as a starting point for establishing baselines, assuming that attribution of
                         impacts to the GR Strategy is a focus:
                                  o  A region-specific baseline for documenting site-level changes
                                     (core elements) and attributing change to the GR Strategy: This
                                     baseline would assume that all reductions in site footprints
                                     associated with GR activities since 2010 are related to the GR
                                     Strategy in all regions except Regions 2 and 9, where established
                                     GR efforts pre-dated the GR Strategy development.  Specific
                                     decisions about attribution could include methods for considering
                                     the impact of specific GR Strategy achievements in Regions 2 and 9,
                                     and could also consider the impacts of separate regional efforts in
                                     the remaining regions.  Documenting attribution is challenging,  and
                                     would also require correcting  for other potential influences such as
                                     specific regional strategies, PRP initiatives, or other state or federal
                                     programs.
                                  o   A national baseline for documenting integration of GR
                                     practices into EPA cleanup culture: This baseline would support
                                     measurement of changes in awareness and the integration of GR
                                     policy throughout the EPA cleanup community and among other
                                     audiences, and would assume that all measured changes in
                                     awareness and practice (e.g., use of training opportunities and
                                     implementation of changes such as contracting language) since
                                     September 2010 are related to the  GR Strategy.

            4.3    EVALUATION PURPOSE 3:  DETERMINE THE BEST METRICS FOR
                   MEASURING THE PROGRAM'S SUCCESS  IN  IMPLEMENTING  GR PRACTICES
                   After careful review of the findings of the evaluation we conclude that a number of strong
                   data sources and methodologies exist for measuring the program's success in
                   implementing  GR practices. The CLU-IN Website use data can be used to track
                   awareness of the GR Strategy and related products (e.g., BMPs, Contracting Tool Kit). In
                   addition, the footprint methodology developed by EPA appears to be a promising tool for
                   measuring site-level impacts of GR activities and Region 2 has developed a Clean and
                                                                                            68

-------
EC
                   Green Policy Metrics Tracking Tool database for compiling site-level data, though it is
                   still in the testing phase. Below we summarize the conclusions for this evaluation purpose
                   and discuss suggested next steps.
                        • To address the goal of the GR Strategy of reducing the environmental footprint of
                          cleanup activities, EPA must identify a method for measuring footprint reductions
                          resulting from implementing GR practices.  We conclude that it will be necessary
                          for EPA to either directly measure the environmental gains at each site or develop
                          a set of standardized "average" site-level effects for common GR activities that
                          can be used to estimate national effects. One limitation of using site-level data is
                          that it may be necessary to establish a centralized system for tracking data, which
                          could be very resource intensive to manage, especially across a large number of
                          sites.  Alternatively, if EPA uses case study and other data to develop estimates of
                          average site-level effects for specific GR techniques, the data collection would
                          require only that regions identify the number of sites employing those techniques.
                        • The footprint methodology developed by EPA represents a clear set of
                          quantitative options directly measuring site performance, but interview responses
                          suggest that the methodology is too resource-intensive to use at every site. Review
                          of an array of emerging tools for calculating environmental footprint did not
                          identify any additional metrics or methods that would be more useful than those in
                          EPA's footprint methodology. We conclude that EPA's tool is appropriate for
                          directly measuring site-level gains for four core elements. However, to address
                          resource concerns among regional staff, EPA should examine options for
                          developing average values for specific GR activities that can be used to estimate
                          site progress without direct data collection and measurement.
                        • The Superfund  Green Remediation Strategy Logic Model (Exhibit 1-3) suggests a
                          number of measures that could be used to assess the success of the program.
                          Given that the GR Strategy was created to be a "living" document that is
                          constantly evolving, EPA will have to consider metrics that best align with current
                          objectives of the GR Strategy. While EPA must ultimately select metrics, the
                          selection should include a balanced suite that directly assesses the short-term,
                          intermediate term, and long-term out outcomes of the GR Strategy. Other criteria
                          of the best metrics could include that they: 1) track data that are readily available
                          and easy to collect, 2) clearly address the goals of the program (clearly linked to
                          logic model), 3) are resource- and cost-effective, 4) are easy to understand, and 5)
                          are meaningful  to the intended audiences of the GR Strategy.
                        • Respondents identified a number of key data gaps and challenges related to
                          measuring program success. Many of the challenges focused on the use of the
                          EPA footprint methodology. Many regions noted the  reluctance of EPA staff in
                          their regions to  conduct footprint analyses, the difficulty of implementing
                                                                                               69

-------
EC
                         footprint analyses without clear policy direction, and the limited resources
                         available for conducting such analyses.

            4.3.1  Next Steps
                       • We recommend that EPA work with regions and develop guidance on how and
                         when to conduct footprint analyses and to examine the potential for utilizing such
                         tools to quantify environmental impacts at sites.
                       • EPA may want to start a dialogue with each of the regions to agree on the best
                         way to leverage case study and other available data to develop an estimation tool
                         or "average" values for GR practices that can be used to estimate national impacts
                         based on simple data collection from regions on GR activities underway.
                       • We recommend that EPA select metrics to measure program success based upon
                         the proposed criteria listed above.
                                                                                             70

-------
lEc
                                APPENDICES

-------
lEc
                  APPENDIX A:  LIST OF INTERVIEWEES
                  We would like to thank the individuals who provided information for this evaluation.
NAME
Carol Donna
Robin Anderson
Barbara
McDonough
Ellen Treimel
Carlos Pachon
Suzanne Wells
Ginny Lombardo
John Podgurski
Nicoletta Diforte
Kristin Giacalone
Stephanie Vaughn
Hillary Thorton
Candice Jackson
Don Rigger
Brad Bradley
Raji Josiam
POSITION
Chemical Engineer and Primary
Author of Interim Guidance
Incorporating GR into
Environmental Remediation
Writing Policy Piece for GR
Strategy
Chief of Contracts Management
Branch and Oversees Grants and
Inter-Agency Agreements
Program Analyst, Member of ASTM
Greener Cleanup Standard
Workgroup
Superfund Green Remediation
National Coordinator
Branch Chief
Superfund GR Regional Co-
Coordinator and RPM
Superfund GR Regional Co-
Coordinator
Superfund GR Regional
Coordinator and RPM
Chair of Regional GR Workgroup
and Regional Superfund
Enforcement Coordinator
RPM and member of GR workgroup
and energy forum
RPM and Presenter/Moderator at
NARPM
New Superfund GR Regional
Coordinator and RPM
Branch Chief
Superfund GR Regional
Coordinator and Brownfields
Project Officers' Assessment and
Cleanup Coordinator
Superfund GR Regional
Coordinator, RPM, Co-Lead
National Engineering
Subcommittee on GR, and On-
Scene Coordinator
ORGANIZATION
Army Corps of
Engineers
EPA OSWER
EPA OSWER
EPA OSWER
EPA OSWER
EPA OSWER
EPA Region 1
EPA Region 1
EPA Region 2
EPA Region 2
EPA Region 2
EPA Region 3
EPA Region 4
EPA Region 4
EPA Region 5
EPA Region 6
DATE
INTERVIEWED
September 1 6,
2011
September 28,
2011
August 30, 201 1
September 29,
2011
September 27,
2011
May 17- 18, 2011
July 14, 2011
July 14, 2011
July 11, 2011
July 11, 2011
July 11, 2011
July 12, 2011
July 18, 2011
September 23,
2011
July 11, 2011
July 13, 2011
                                                                                        A-1

-------
lEc
NAME
Clint Sperry
Timothy Rehder
Andria Benner
Rusty Harris-
Bishop
Jeff Dhont
Karen
Scheuermann
Julie Santiago-
Ocasio
Beth Sheldrake
Clifford Villa
Heather NiFong
Kevin Carpenter
Doug Sutton
POSITION
RPM and Co-Chair of Federal
Facilities Forum
Superfund GR Regional
Coordinator and Associate with
Super Fund Brown Fields
RPM
Former Superfund GR Regional
Coordinator (Currently
Communication Coordinator,
Liaison to OPA)
Superfund GR Regional
Coordinator
Author of Superfund GR Strategy
Activity Tracking Chart, Key Action
lead (Also Considered an
Environmental Engineer in Region
9 RCRA)
Former Superfund GR Regional
Coordinator and
Presenter/Moderator at NARPM
Superfund GR Regional
Coordinator and Superfund
Regional Program Management
Unit Manager
Assistant Regional Counsel and
Unit Manager for Staff Attorneys
Program Advisor
Member of Green Standards Work
Group and ASTM Greener
Sustainable Cleanup Team,
ASTSWMO
Technology Staff for the Division
of Environmental Remediation
EPA Consultant (assisted in the
development of the Footprint
Methodology and performs site
optimization analyses)
ORGANIZATION
EPA Region 7
EPA Region 8
EPA Region 9
EPA Region 9
EPA Region 9
EPA Region 9
EPA Region 9
(Formerly at EPA
Region 4)
EPA Region 10
EPA Region 10
Illinois EPA
New York State
Department of
Environmental
Conservation
Tetratech
DATE
INTERVIEWED
July 29, 2011
July 12, 2011
July 19, 2011
July 19, 2011
August 1 , 201 1
August 1 1 , 201 1
July 18, 2011
July 15, 2011
September 9,
2011
August 1 7, 201 1
September 20,
2011
August 8, 201 1
                                                     A-2

-------
EC
                   APPENDIX B: INTERVIEW GUIDES
                   Each interview guide was sent to interviewees with the following cover letter.
                   Superfund Green Remediation Strategy: Implementation Evaluation
                   CONTACT INFORMATION
                          Name:         	
                          Title:          	
                          Email:         	
                          Phone number: 	

                   INTRODUCTION
                   Thank you for participating in this interview process. As you are aware, the Superfund
                   program is working to advance greener cleanups at Superfund sites. Central to this effort
                   is the Superfund Green Remediation (GR) Strategy, which was published in final form in
                   September, 2010. With the support of Industrial Economics, Inc. (IEc) and EPA's
                   Evaluation Support Division (ESD) in the Office of Policy, EPA's Office of Superfund
                   Remediation and Technology Innovation (OSRTI) is evaluating the implementation of
                   the GR Strategy. The evaluation is jointly funded by OSRTI and ESD, and was selected
                   under OP's Program Evaluation Competition, a long-term effort to build capacity for
                   program evaluation among headquarters and regional offices.

                   As part of the GR Strategy, the Superfund program is evaluating the implementation of
                   the GR Strategy itself. The chosen approach is to conduct a "formative" evaluation of the
                   national-level effort. The purpose of the evaluation is to document the GR Strategy's
                   effectiveness in advancing greener cleanups based on three main parameters:
                       •  Assessing EPA experiences to date in implementing the GR Strategy;
                       •  Determining a baseline against which to measure EPA progress in implementing
                          the GR Strategy;
                       •  Determining the best metrics for measuring the program's successes in
                          implementing GR practices.
                   Throughout the interview, we would like to know about the processes with which you are
                   most familiar. We also encourage you to raise any items or topics you think are important
                   to our evaluation.  Information shared during the interviews will be anonymous and
                   summarized thematically, and will  not be attributed to specific individuals in the
                   evaluation report.  In presenting findings from the interviews, IEc may attribute findings
                   to groups of interviewees, (e.g., a regional staffer), but we will not attribute findings or
                   quotes to individuals without first obtaining permission from the respective interviewees.

                   The following interview questions are intended to serve as a guide for our conversation
                   and are provided in advance to spur your thinking and responses. Where possible, please
                   be prepared to provide specific examples. Your responses are important, and we look
                   forward to speaking with you. If you have any questions or would like to provide any
                   additional feedback or information, please contact:
                                                                                                  B-1

-------
lEc
                    Cynthia Manson, Industrial Economics, Inc.
                    cj m(g),indecon .com
                    617-354-0074

                    Interview Guide for Regional GR Coordinators and RPMs

                    INTERVIEW  QUESTIONS

                    Implementation of Superfund Green Remediation Strategy

                    General Information:
                    •   Please briefly describe your position at Region 7.
                    •   What is your role in implementing the GR Strategy!
                    •   How do you use the GR Strategy!
                    Evaluation Questions:
                        1.  Which initial activities  or initiatives from the GR Strategy have been most
                           effective in increasing awareness, adoption and/or implementation of GR
                           activities in your Region?
                               •   How well is the GR Strategy getting used?
                               •   Which activities have been least effective?
                               •   How successful has the GR Strategy been at helping to implement GR
                                  practices?
                               •   What are the main challenges in advancing GR?
                               •   Which audiences have been most easily reached by the GR Strategy!
                                  Which audiences have been more challenging or have not yet been
                                  reached?
                        2.  How do Remedial  Project Managers (RPMs) in your Region factor the  GR
                           Strategy into their approach to planning site cleanup?
                               •   Has your region conducted a survey of GR practices in your Region? If
                                  so, to whom?
                               •   What GR practices are being implemented?
                               •   What percentage of RPMs are implementing specific GR practices?
                               •   What do RPMs know about the energy usage at the sites they manage?
                               •   What information do RPMs track on other GR core elements?
                        3.  What effect has the GR Strategy had on the practice of using green remediation
                           techniques at Superfund sites in your region?
                               •   What practices/activities have you changed as a result of the GR
                                  Strategy?
                               •   Does your region have a GR implementation plan?
                                                                                                 B-2

-------
lEc
                               •   What other policies/strategies do you use for implementing GR
                                   practices?
                               •   Are there other regional programs helping to move GR forward?
                        4.  What lessons have  been learned as a result of implementing the GR Strategy at
                            sites?
                               •   What successes have you had?
                               •   What factors affect the ability to implement the GR Strategy at sites?
                                       i.  Technical issues
                                       ii.  Cost issues
                                      iii.  Legal issues
                                      iv.  Management support
                                       v.  Contract provisions
                                      vi.  Contractor capabilities
                                     vii.  Other?
                               •   How is integration of the  GR Strategy priorities (e.g., policy guidance,
                                   training, and tools) affected by the above factors and experiences to date?
                               •   Has your region implemented or conducted any kind of GR training?
                     Strategic Implementation
                        5.  Review of GR Strategy goals and objectives
                               •   Please tell  us how your work has used or focused on GR Strategy goals
                                   and objectives.
                               •   In your opinion, does EPA have clearly defined goals and objectives for
                                   the GR Strategy!
                               •   Are these objectives and goals well aligned for different  uses?
                                       i.  Management decision-making
                                       ii.  Planning and budgeting
                                      iii.  Strategic planning?
                               •   Do you have suggestions  for refining goals or objectives?
                     Baseline Development
                        6.  What options can we identify for developing a baseline?
                               •   What has changed since the implementation of the GR Strategy in your
                                   Region?
                               •   When did green remediation become important to site cleanup?
                               •   What options are available for quantifying the environmental footprint at
                                   sites?
                     THE FOCUS OF THIS INTERVIEW IS ON THE SIX QUESTIONS  LISTED ABOVE,  BUT
                     ANOTHER PURPOSE OF OUR EVALUATION AIMS  AT EVALUATING THE  BEST METRICS
                     FOR MEASURING THE PROGRAM'S SUCCESS IN IMPLEMENTING GR PRACTICES.
                     PLEASE LET US KNOW IF  YOU HAVE THOUGHTS ON THE  FOLLOWING THREE
                     QUESTIONS RELATED TO THAT TOPIC.
                                                                                                   B-3

-------
lEc
                    Metrics for Measuring Progam Success
                        7.  What performance measures do you think could be used for measuring the
                           effectiveness of the GR Strategy in achieving intended outcomes at a regional or
                           national level?
                        8.  What approaches are useful for measuring the effectiveness of the GR Strategy in
                           reducing the environmental footprint at sites that have implemented GR practices
                           with respect to the five core elements ofGR Strategy!
                               •  What options exist for using qualitative or quantitative measures to
                                  assess the five core elements of GR Strategy!
                               •  Is your region collecting any measurement data related to the use of GR?
                                  What type?
                        9.  Where are the primary data gaps and limitations that inhibit a better
                           understanding of the results of implementing the GR Strategy!

                    Additional  Information
                        10. Please discuss any other issues affecting the efficiency or effectiveness of the GR
                           Strategy. Do you have any other suggestions to improve the GR Strategy, from
                           either the perspective of EPA or the intended audiences?
                    Interview  Guide  for Regional Attorneys

                    INTERVIEW QUESTIONS

                         1.  Please briefly describe your position in your Region.
                         2.  Do you work on green remediation (GR) issues?
                               •  Where in your work have you encountered GR?
                         3.  What is your impression of GR?
                               •  What seems most promising?
                               •  What seems most troubling?
                         4.  In your opinion, what are the challenges for implementing GR practices?
                               •  What factors affect the ability to implement the GR Strategy at sites?
                                         Technical issues
                                         Cost issues
                                      Hi.  Legal issues
                                      iv.  Management support
                                      v.  Contract provisions
                                      vi.  Contractor capabilities
                                     vii.  Other?
                                  Is there currently sufficient policy or legal justification for the legal
                                  department in your Region to support GR activities? Does  the legal
                                  department support RPMs' requests/orders to contractors or PRPs to
                                  consider GR activities?
                                                                                                  B-4

-------
lEc
                         5.  What experience have you had with Superfund Green Remediation (GR)
                            Strategy?
                               •  In your experience, what aspects of the GR Strategy work well?
                               •  What aspects don't work well?
                               •  Do you have suggestions for improvements to the GR Strategy?
                         6.  What effect has the GR Strategy had on the practice of using GR techniques at
                            Superfund sites in your region?
                               •  In what way has the GR Strategy shaped policy in your Region?
                               •  How has the GR Strategy been helpful to your work?
                               •  How has the GR Strategy made your job more difficult?
                         1.  How do you coordinate with Headquarters in your GR work?
                               •  Is there anything that HQ could provide to make your work with  GR
                                  easier?
                               •  Are there specific people that you work with frequently?
                               •  Which topics do you typically address?
                               •  Are there other people or resources you consult with outside ofHQ?
                         8.  Please discuss any other issues affecting the efficiency or effectiveness of the
                            GR Strategy. Do you have any other suggestions to improve the GR Strategy?
                    Interview Guide  for Regional Supervisors

                    INTERVIEW  QUESTIONS

                        1.  Please briefly describe your position in your Region.
                        2.  Do you work on green remediation (GR) issues?
                               •  How do you encounter GR?
                               •  Where in your work do you use GR?
                        3.  What experience have you had with Superfund Green Remediation (GR)
                           Strategy?
                               •  In your experience, what aspects of the GR Strategy work well?
                               •  What aspects don't work well?
                               •  Do you have suggestions for improvements to the GR Strategy?
                        4.  In your opinion, what are the challenges for implementing GR practices?
                               •  What factors affect the ability to implement the GR Strategy at sites?
                                       i.   Technical issues
                                      ii.   Cost issues
                                     Hi.  Legal issues
                                     iv.  Management support
                                      v.   Contract provisions
                                     vi.   Contractor capabilities
                                                                                                  B-5

-------
lEc
                                      vii.  Other?
                            What effect has the GR Strategy had on the practice of using GR techniques at
                            Superfund sites in your region?
                                •   Has the GR Strategy been helpful to this effort?
                            Please discuss any other issues affecting the efficiency or effectiveness of the GR
                            Strategy. Do you have any other suggestions to improve the GR Strategy?
                                                                                                     B-6

-------
lEc
                     Interview Guide for EPA Contract Specialists

                     INTERVIEW QUESTIONS

                        1.   Please briefly describe your position as it relates to the Green Remediation
                            Strategy.
                               •  How are you involved with the GR Strategy?
                               •  In what capacity does your work promote the GR Strategy?
                        2.   Please briefly describe the process of incorporating GR clauses into Superfund
                            contracts.
                               •   What does your work entail?
                               •  How does putting GR into a contract work?
                               •  Which types of contracts have GR clauses been incorporated into?
                                       /'.  Removal action
                                       ii.  Remedial response
                                      Hi.  Support services
                                      iv.  Technical enforcement support
                                       v.  Policy, program management, and administrative services
                                      vi.  Other contract venues
                               •  Are the GR clauses in contracts sufficient to promote GR practices?
                                       i.  Are they followed or ignored?
                               •  You are listed as a contributor to the Greener Cleanups Contracting and
                                  Administrative Toolkit. Do you participate in the periodic updates of this
                                  document?
                                       i.  Who else is involved?
                                       ii.  How are updates communicated to the intended audiences?
                        3.   With whom do you work in the Regions? At HQ?
                               •   What level of staff do you work with in the Regions?
                               •  Do you work with Regional Coordinators or RPMs directly?
                        4.   What are your goals for incorporating GR in Superfund contracts?
                               •  Are there metrics that you track related to GR in contracts?
                                       i.  If not,  what could be tracked?
                                       ii.  Do you have a list of all contracts that incorporate GR?
                               •  How would you measure the success of what your work is trying to
                                  accomplish?
                                       /'.  What successes have you had?
                               •  What challenges have you faced in incorporating GR into contracts?
                                       /'.  What factors affect the ability to incorporate GR into contracts?
                                                                                                  B-7

-------
lEc
                        5.  Please discuss any other issues affecting your work as it relates to the GR
                           Strategy. Do you have any other suggestions to improve the GR Strategy, from
                           either the perspective of EPA or the intended audiences?
                    Interview Guide for EPA GR Strategy Evaluation  Work Group
                    Members

                    INTERVIEW  QUESTIONS

                        1.  What do you see as the next steps for the strategy?
                               •   What things are you working on now?
                        2.  What type of feedback have you received about the strategy?
                               •   What type of feedback from EPA HQ?
                               •   What type of feedback from regional staff (e.g., RPMs, front line
                                  managers, attorneys)?
                        3.  What do you see as areas of the strategy that need the most work/attention?
                               •   Which areas are most pressing?
                        4.  What do you see as the next steps for the footprint methodology?
                               •   Are you planning to issue any further guidance on using the
                                  methodology?
                        5.  What are the challenges facing the integration and implementation of the
                           strategy?
                        6.  What is the status of the Region 2 database for tracking site level data?
                               •   Is that something that is intended to be shared with all the regions?
                        7.  How do you intend to use EMS and their data in the future?
                               •   In addition to what has been tracked and collected,  are there other metrics
                                  you have or will request?
                        8.  We received feedback that EPA should be more coordinated with other programs.
                           Are there any plans to do this in the future?
                               •   How would this happen?
                               •   For outside EPA contacts what do you think the coordination with EPA
                                  on GR should be?
                    Interview Guide for EPA GR Strategy Policy Expert

                    INTERVIEW  QUESTIONS

                        1.  What is your role in the GR Strategy?
                        2.  Tell us about the current piece you are working on.
                                                                                                 B-8

-------
lEc
                        3.  What is next for you?
                        4.  What are the next steps for the GR Strategy as a whole?
                               •   What are the most pressing?
                        5.  What other policy guidance to you see coming in the future?
                               •   What else is in the works?
                    Interview Guide for Non-EPA Federal Agencies

                    INTERVIEW QUESTIONS

                        1.  Please briefly describe your position as it relates to green remediation.
                        2.  What experience have you had with the Superfund Green Remediation (GR)
                           Strategy?
                               •   In your experience, what aspects of the GR Strategy work well?
                               •   What aspects don't work well?
                               •   Do you have suggestions for improvements to the GR Strategy?
                        3.  How does the GR Strategy help with your green remediation efforts?
                               •   Does the GR Strategy create any problems?
                               •   How does the GR Strategy effort differ from yours? Is it complementary?
                        4.  How do you coordinate with the Regional and national (Headquarters) EPA
                           offices in your green remediation work?
                        5.  How does your coordination with EPA compare with your coordination with
                           other federal agencies?
                        6.  Has the GR Strategy had any effect on the practice of using green remediation
                           techniques at remediation sites in your Agency?
                               •   What practices/activities have you changed as a result of the GR
                                  Strategy?
                               •   What other policies/strategies do you use for implementing GR
                                  practices?
                        1.  Please discuss any other issues affecting the efficiency or effectiveness of the GR
                           Strategy. Do you have any other suggestions to improve the GR Strategy!
                    Interview Guide for State Programs

                    INTERVIEW QUESTIONS

                        1.  Please briefly describe your position as it relates to green remediation.
                        2.  What experience have you had with the Superfund Green Remediation (GR)
                           Strategy?
                                                                                                B-9

-------
lEc
                               •   In your experience, what aspects of the GR Strategy work well?
                               •   What aspects don't work well?
                               •   Do you have suggestions for improvements to the GR Strategy?
                        3.  How does the GR Strategy help with your state-level green remediation efforts?
                               •   Does the GR Strategy create any problems?
                               •   How does the GR Strategy effort differ from yours? Is it complementary?
                        4.  How do you coordinate with the Regional and national (Headquarters) EPA
                            offices in your green remediation work?
                        5.  How does your coordination with EPA compare with your coordination with
                            other federal agencies?
                        6.  Has the GR Strategy had any effect on the practice of using green remediation
                            techniques at Superfund sites in your state?
                               •   What practices/activities have you changed as a result of the GR
                                   Strategy?
                               •   What other policies/strategies do you use for implementing GR
                                   practices?
                        7.  Please discuss any other issues affecting the efficiency or effectiveness of the GR
                            Strategy. Do you have any other suggestions to improve the GR Strategy!
                     Interview Guide for  Measurement Specialists

                     INTERVIEW  QUESTIONS

                        1.  Please briefly describe your position as it relates to the Green Remediation
                            Strategy.

                               •   How are you involved with the GR Strategy?
                               •   In what capacity does your work promote the GR Strategy?

                        2.  Please give us an update of the status of the footprint methodology.

                               •   How will this be rolled out and integrated in the Regions?
                               •   What are your plans for how people will use this?
                                       i.   Will there be guidance on how Regions can overcome some of
                                          the limiting factors mentioned in the first round interview s? (i.e.,
                                          time, money, resources,  availability of a universal measurement
                                          system, and educating contractors on the methodology)
                               •   How will it be determined which sites to footprint?
                                       i.   Will the EPA eventually want the analysis conducted on every
                                          Superfund site?
                                                                                                   B-10

-------
lEc
                               •   Who will provide the resources and funding to conduct the footprint
                                   analysis?
                               •   Some Regions raised the issue of PRPs coming back with competing
                                   analyses. How is the footprint methodology designed to limit this
                                   problem?
                               •   How does the footprint methodology address land use?
                     Baseline Development
                        3.    What ideas do you have for measuring a baseline?
                               •   What options are available for quantifying the environmental footprint at
                                   sites?

                     Metrics  for Measuring  Progam Success
                        4.     What performance measures do you think could be used for measuring the
                              effectiveness of the GR Strategy in achieving intended outcomes at a regional
                              or national level?
                        5.     What approaches are useful for measuring the effectiveness of the GR Strategy
                              in reducing the environmental footprint at sites that have implemented GR
                              practices with  respect to the five core elements of GR Strategy!
                               •   What options exist for using qualitative or quantitative measures to
                                   assess the five core elements of GR Strategy!
                               •   What types of measurement data are being collected related to the use of
                                   GR in the Regions?
                        6.     Where are the  primary data gaps and limitations that inhibit a better
                              understanding of the results of implementing the GR Strategy!

                     Additional Information
                        7.     Please discuss any other issues affecting the efficiency or effectiveness of the
                              GR Strategy. Do you have any other suggestions to improve the GR Strategy,
                              from either the perspective of EPA or the intended audiences?
                                                                                                   B-11

-------
lEc
                    APPENDIX  C:  REGIONAL SURVEY AND TRACKING DATA
                    REGION 3 GREEN REMEDIATION 2009 QUESTIONNAIRE:
                    The Region 3 Green Remediation 2009 Questionnaire results could not be formatted
                    effectively to include in the appendices. Instead, we include the actual questionnaire that
                    was sent to RPMs in the region to offer some context for the information available in the
                    results.
                                                                                             C-1

-------
lEc
                                        Green Remediation 2009 Questionnaire
                       Site Name      Location:
                       QPrivate QArmy QNavy QAir Force
                       List RPM of the site Name
WATER
Stortnwater Management:
1) Have vim installed any
of the following landscape features to control stormwater:
A) bioretention cells
B) vegetative swales
C) porous pavement/stones
D) trees
E) biosocks
F) coconut logs



la) What is the area?
Water Usage:
2) Did vou minimize water use on site?
2a) How?

3) Did you store site generated water or storm water for reuse on site?
(i.e. rainwater for dust suppression)
3a) How is the water stored and how much?
4) Did you recycle water for use on site?
4a) How is it used?

5) Have you created open space Q to filter water Q other explain:
I | storm water control
1 1 create wetlands
B\es no
ves no
{_ yes no
[_| \ es 	 no
, D yes _ no
LJ \esL no


D >cs [H no

tU yes D no

[ D yes D no


Wetlands as water management features :
6) Have you constructed wetlands for:
a) Stormwater control
b) act as a filter before entering surface water
c) treat contaminants
d) enhancement
e) replacement


D yes D no
O yes O no
n yes D no
D yes D no
O yes E] no

                                                                                          C-2

-------
lEc
                                       6a) How many acres? I f not known
                                         where could we find this
                                          information?
                                       Green (Jrountlwater remedy:
                                    7) Is there a groundwater or surface water component at your site?
                                       a) monitoring only
                                       b) pump and treat
                                       c) in-situ treatment:
                                       d) Was the in-situ treatment part of a pump and treat optimization?
                                       e) other
                                      Explain
                         Q yes [H no
                         D yes D no
                         O yes d no
                         n yes G no
                         n yes n no
                                    8) What was the reagent used to treat the groundwater contaminant(s):

                                         C] molasses    Q vegetable oil   [D chemical compound like potassium permanganate  Q other
                                    9) Have you installed a permeable reactive barrier as a remedy rather than
                                    groundwater pump and treat system?
                         O yes LJno

                        1	
                                    9a) Was this part of the original remedy or a post-   j %) If post-ROD why?
                                        ROD modification?
                                    10) Have you performed an optimization study on your site? What were the results?
                             yes LJ no
                                       Is the site currently_being_reu3cd?
                                          1 la) When did you start planning for reuse?
                                       D Pre ROD	d_ROD__      D Post ROD
                                          lib) What was the type of re-use;
                           j yes I	I no
                                                          nercial   d ecological     Q residential     d municipal    d other explain
                                    12) What type of revegctation did you perform at your
                                    site?
Qflandfill cover     O replacement
O wetland
d enhance existing land
D other:
                                                                                                                                    C-3

-------
lEc
12a) What type of species
1 2b) How many acres were planted?
O native gr
O trees/shr
Q_wctland

                                                                                                                      Dothe
                                     13) What was the purpose of the revegetation on your site?
                                        a) Stormwater control
                                        b) Create, enhance, or preserve wetlands
                                        c) Treatment of contaminants
                                        d) Reduce green house gases
                                        e) Enhance ecological habitat
                                        f) Replacement of destroyed vegetation
                                        Explain:	                 	
                                     14) Did the remedy include plants or trees (phytoremediation) to remediate
                                     contamination?
yes
yes
yes
                                                   CD yes IU no
                                                   [H yes Z] no
                                                   D yes Q no
                                                             no
                                                            ] no
                                                             no
                                           Soil Q yes D no
                                           Ground water
                                           CD yes CD no
                                           Surface Water/Springs
                                            n yes n no
                                       14a) What type of plants were
                                     used, how many acres were planted or
                                     number of plants?
                                     IS)  Did you reduce or minimize
                                     destruction of ecologically
                                     significant habitats?
     [JyesLjno

     Amount:

     Explain:
                                       SCML:
                                     16) Did you treat contaminated soils on-site and reuse the soils for cover or fill rather
                                     than dig/haul contaminated soils for off-site disposal?
                                                      J yes LJ no
                                     1?) Did you minimize off-site disposal via onsitc containment and/or treatment?
                                                    [~] yes M no
                                        17a) How much soil,
                                             sediment, other waste?
                                             Describe materials and
                                             quantify.
USoil
I  | Sediment
Q Other waste
Total:
Explain:
                                     18) Were any amendments and/or plants used to improve soil quality?
                                          18a) What type of amendment?    Example: pine bark, bio-solid
                                                                                                                                        C-4

-------
lEc
1
18h) What quantity of soils were treated?

1 8c) Do you have information about the impacts of soil amendments have on |_J yes |_| no
sequestering green house gasses?
                                   RECYCLING
                                  I 19) Were recycled materials used to construct anything on site i.e. buildings, roads,
                                   parking lots or pavement?
                                  i Explain:
                                   20) Did you separate materials for re-use or recycling at your site?
                                         20a) What type of material?


                                    D soil for cover  Q wood     Q metal     D oil     D other:         D tire     D Construction &
                                                                                                              Demolition Debris
                                    [  | green concrete

                                         20b) How did you reuse on-site materials?
                                             Amount:

                                             Explain:
                                   21) How did you minimize waste generation at the site? Explain.
                                      Secondary Questions:
                                   22) Did you use waste streams from other sources in your remedy? For example,
                                   biosolids for a cover, pickle liquor for groundwatcr treatment.
                                         22a) What type of waste?
LJyeslLJno
                                   ENERGY
                                       Priot
                                   23) Do any of the components of your remedy uses low energy?
                                   (i.e. monitoringgroundwater)
LJ yes U
                                                                                                                                  C-5

-------
lEc
Did you design the treatment system with optimum efficiency?
24a) Did you perform an optimization study of the groundwater pump and treat
system and as a result did you find energy efficiency and/or cost savings?
IDyesDno
i n yes n no
                                             study, how did you alter the
                                             groundwater pump and treat system?
                                         24c) Have you tracked your energy usage
                                             or cost over time? What figures do you
                                             have? Where are they reported?
                                         24d) Are you using energy efficient equipment (pumps, lights) at your site?
                                         24e) Are you using Energy Star equipment like computers at your site?
                                   25) Is the power used at the site from a renewable energy source?
                                         a) Are you using solar energy, hydro energy or wind power for energy at your site?
                                         b) Has methane gas from a landfill at your site been used for energy?
                                         c) Did you purchase energy from a renewable energy source?
                                   26) Did you use field techniques to minimize energy used? (i.e. XRF, TAGA Van,
                                   TRIAD)** of samples
O Yes [Jno
   yes 	 no
    es [I] no
LJ yes LJ no
  J yes I	| no
                                   27) Did you use low energy remedy such as bioremediation, reactive barrier or
                                   phytoremediation?
                                   AIR QUALITY AND GREEN HOUSE GASSES
                                       Priority Questions	
                                   28) Did you minimize the generation of air emissions and greenhouse gases? i.e. reduce
                                   equipment and transportation usage
                                         28a) If so, how?
                                                                                                                                   C-6

-------
lEc
28b) Did you include additional treatment on an air stripper to reduce air emissions?
29) Have you used or plan to use alternative fuels such as biodiesel, electricity, ethanol,
hydrogen, natural gas, propane in the equipment on the site?
29a) Did you try to reduce the
usage of equipment i.e.
hand excavate?
29b) Did you repair an existing
engine or equipment to
make it more efficient?
Q yes Q no
D yes D n°
i
LJyesLjno Explain: ;
CD yes O no Explain:
29c) Did you install emission reduction 1 LJ yes (_j no Explain:
technology on equipment such as
clean diesel equipment?
30) Are plans in place to reduce truck EH yes [jTno Explain: j
transport either to dispose of waste or bring i
supplies to the site?
LONG TERM STEWARD
Priority Questions

31) Have you evaluated operation and maintenance procedures in order to increase
efficiency or reduce energy usage?
3 la) For example, did you reduce monitoring frequency (number of wells,
contaminants) while still gathering sufficient data to monitor the remedy?
If yes, explain.
31b) For example, have you modified landfill vegetation so there is less maintenance?
If yes, explain.
3 Ic) For example, have you looked at composite sampling versus discrete sampling or
different levels of validation for sampling? If yes, explain.
32) Have you installed a new technology to make the remedy more efficient?
LJ yes LJ no
LJ >'es LJ no
O yes Lj] no
D yes D no
n yes n no
32a) For example, have you installed a computer based monitoring system? | 	 | yes | 	 | no
                                                     C-7

-------
lEc
32b) For example, have you installed energy efficient equipment for the operating Q yes O no
systems?
CLOSING QUESTIONS
33) What training or resources would
help you use green remediation?
34) What are the greenest aspects of your
remedy? Or planned green remedy?
Can any other KI'M's who had the site
elaborate on green aspects of the
remedy?
35) What type of documentation do you
have on the green component of the
remedy including reasons why it was
implemented and cost and where could
we find this information?
36) Would you be interested in attending
the Green Cleanup Symposium
scheduled for February lO11* & II1*,
2010? Or present a case study at the
Green Cleanup Symposium?




                                                     C-8

-------
EC
                     REGION 4 2010 GREENER CLEANUP SURVEY:
                     MEMORANDUM

                     United States Environmental Protection Agency   B&V Project 049049
                     TO 49 - Green Remediation     May 26, 2011

                     To:         Julie Santiago-Ocasio, U.S. Environmental Protection Agency

                     Subject:     Greener Cleanup Survey Summary Memorandum
                                 Document Control Number 49049-0872-02-E-00833RO

                     This memorandum summarizes the results of the U.S. Environmental Protection Agency (EPA)
                     Region 4 Superfund Greener Cleanup Survey. Region 4 conducted this survey to assess progress
                     in implementing Greener Cleanup practices at Superfund sites across Region 4.  The information
                     obtained through this survey will help to identify training needs and opportunities to aid personnel
                     in implementing Greener Cleanup practices at their sites.  This information will also be utilized to
                     develop tools and case studies to help Remedial Project Managers (RPMs) learn from successes
                     and/or failures of the implementation process.

                     This memorandum has been prepared by Black & Veatch Special Projects Corp. (Black & Veatch)
                     under Contract Number EP-S4-09-02 with EPA Region 4 and under specific authorization of EPA
                     Region 4 through the Task Order (TO) Number 049.

                     The results of the survey are presented below. For reference, the definitions of the five Principles
                     for Greener Cleanups are included.

                     Principle 1:  Total Energy Use and Renewable Energy Use
                         •   Minimize energy consumption (e.g. use energy efficient equipment).
                         •   Power cleanup equipment through onsite renewable energy sources.
                         •   Purchase commercial energy from renewable resources.

                     Principle 2:  Air Pollutants and Greenhouse Gas Emissions
                         •   Minimize the generation of greenhouse gases.
                         •   Minimize generation and transport of airborne  contaminants and dust.
                         •   Use heavy equipment efficiently (e.g. diesel emission reduction plan).
                         •   Maximize use of machinery equipped with advanced emission controls.
                         •   Use cleaner fuels to power machinery and auxiliary equipment.

                     Principle 3:  Water Use and Impacts to Water Resources
                         •   Minimize water use and depletion of natural water resources.
                         •   Capture, reclaim and store water for reuse (i.e., recharge aquifer, drinking water
                             irrigation).
                         •   Minimize water demand for revegetation (i.e., native species).
                         •   Employ best management practices for stormwater.

                     Principle 4: Materials Management and Waste Reduction
                         •   Minimize consumption of virgin materials.
                                                                                                           C-9

-------
lEc
                           •    Minimize waste generation.
                           •    Use recycled products and local materials.
                           •    Beneficially reuse waste materials (e.g., concrete made with coal combustion products
                               replacing a portion of the Portland cement).
                           •    Segregate and reuse, or recycle materials, products, and infrastructure (e.g. soil,
                               construction and demolition debris, buildings).

                       Principle 5: Land Management and Ecosystem Protection
                           •    Minimize areas requiring activity or use limitations (e.g., destroy or remove contaminant
                               sources).
                           •    Minimize unnecessary soil and habitat disturbance or destruction.
                           •    Use native species to support habitat.
                           •    Minimize noise and lighting disturbance.


                       General

                       The Greener Cleanup Survey was sent to 63 EPA Remedial Project Managers (RPMs). A total of
                       34 RPMs, or 54%, responded to the survey; however, only 31 respondents answered all of the
                       questions.  The following is a list of all EPA RPMs who participated in the study:

                       Femi Akindele         Bill Denman            Leigh Lattimore           Michael Taylor
                       Cathy  Amoroso        Carl Froede Jr.           Lila Llamas              Michelle Thornton
                       Jim Barksdale          Rachel Hall             McKenzie Mallary        Peter Thorpe
                       Martha Berry          Corey Hendrix           Keriema Newman         Samantha Urquhart-Foster
                       Jon Bornholm          Ralph Howard           Rob Pope                Debbie Vaughn-Wright
                       Randy Bryant          Candice Jackson         Jon Richards              Tim Woolheater
                       Carolyn  Callihan       Constance Jones         Carmen Santiago-Ocasio   Craig Zeller
                       Loften Carr            Yvonne Jones           Pam Scully
                       Peter Dao              Rusty Kestle             Erik Spalvins
                       Type of Site
                                   •    Superfund - 20
                                   •    Superfund with Federal Facilities Sites - 13
                                   •    Both Superfund & Superfund with Federal Facilities Sites - 1
                                                                                                             C-10

-------
lEc
                       Number of sites managed by each of the responding RPMs
Number of Sites
1
2
o
J
4
5
6
7
8
9
10
11
15
20
Number of RPMs
8
5
1
3
3
4
3
1
1
2
1
1
1
                       Principles for Greener Cleanups

                       Principle 1:  Total Energy Use and Renewable Energy Use
                                  •   12 RPMs, or 35% of the respondents, have implemented this principle.
                                  •   Specific elements include:
                                          o  Consider use of optimized passive-energy technologies (with little or no
                                              demand for external utility power) that enable all or  part of the
                                              remediation objectives to be met - 8
                                          o  Look for energy efficient equipment and maintain equipment at peak
                                              performance to maximize efficiency - 6
                                          o  Periodically evaluate and optimize energy efficiency of equipment with
                                              high energy demands - 2
                                          o  Consider installing renewable energy systems  to replace or offset
                                              electricity requirements  otherwise met by the utility (i.e., solar powered
                                              sampling devices) - 5
                                          o  Other - solar powered surface water monitoring system
                                  •   Reason for implementing the principle
                                          o  Technical  -10
                                          o  Financial - 5
                                          o  To comply with Region 4 Clean and Green Policy - 3
                                          o  To comply with Greener Cleanup language added in an enforcement
                                              document - 0
                                          o  Other: No onsite power source.
                                  •   Did the principle play a role in the selection of a remedy?
                                          o  Yes -1
                                          o  No  -10

                                  •   Did the principle affect the way the remedy was implemented?
                                          o  Yes - 3
                                          o  No  - 6
                                                                                                            C-11

-------
lEc
                                        o   Not applicable - 2
                                 •   Did the principle affect post construction activities?
                                        o   Yes - 3
                                        o   No-5
                                        o   Not applicable - 3
                                 •   Sites where Principle 1 has been implemented
Site Name
Marshall Space Flight Center
P Area OU 94
Landia Chemical
Savannah River Site
Sanford Gasification
M Area OU
Cape Fear
Savannah River Site
Barite Hill
Nevada Goldfields
Redstone
Oak Ridge Reservation
East TN Technology Park
Calculator
Used?
No
No
Yes
No
No
No
No
No
No
No
No
No
No
Phase
Investigation, Remedy Selection,
Remedial Design, and Implementation
Remedy Implementation
Remedy Implementation
Remedy Implementation
Remedy Implementation
Remedy Implementation
Post-Construction Complete
Remedy Implementation
Investigation
Investigation
Investigation
Remedial Design and Implementation
Remedial Design and Implementation
                                                                                                       C-12

-------
lEc
                       Principle 2: Air Pollutants and Greenhouse Gas Emissions
                                   •    14 RPMs, or 41% of the respondents, have implemented this principle.
                                   •    Specific elements include:
                                          o   Minimize use of heavy equipment requiring high volumes of fuel - 4
                                          o   Use cleaner fuels  and  retrofit  diesel  engines  to  operate  heavy
                                              equipment, when possible - 2
                                          o   Reduce atmospheric release of toxic or priority  pollutants (ozone,
                                              paniculate  matter, carbon monoxide, nitrogen dioxide, sulfur dioxide,
                                              and lead) (i.e., use of anti-idling policies) - 6
                                          o   Minimize dust export of contaminants -12
                                          o   Other - minimization of personnel traveling to the  site
                                   •    Reason for implementing the principle
                                          o   Technical -10
                                          o   Financial - 2
                                          o   To comply  with Region 4 Clean and Green Policy - 3
                                          o   To comply  with Greener Cleanup language added  in an enforcement
                                              document - 0
                                          o   Other - This is a standard operating procedure at SRS, To control
                                              fugitive dust emissions, Generally good practice.
                                   •    Did the principle play a role in the selection of a remedy?
                                          o   Yes - 2
                                          o   No-11
                                   •    Did the principle affect the way the remedy was implemented?
                                          o   Yes - 3
                                          o   No-9
                                          o   Not applicable -1
                                   •    Did the principle affect post construction activities?
                                          o   Yes - 0
                                          o   No-8
                                          o   Not applicable - 5
                                                                                                             C-13

-------
lEc
                            •  Sites where Principle 2 has been implemented
Site Name
East TN Technology Park
Oak Ridge National Lab
Savannah River Site
Landia Chemical
Aqua Tech
T Area OU
Sanford Gasification
TVA Kingston
Cape Fear
Barber Orchard
Barite Hill
NAS Jacksonville
Redstone Arsenal
Oak Ridge Reservation
East TN Technology Park
Smokey Mountain Smelters
Sprague Electric
Calculator
Used?
No
No
No
Yes
No
No
Yes
Yes
No
No
No
No
No
No
No
No
No
Phase
Treatability Study
Remedy Implementation
Remedy Implementation
Remedy Implementation
Remedy Implementation
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Post-Construction Complete
Remedy Implementation
Investigation
Investigation, Remedy Selection,
Implementation, and Post-Construction
Complete
Investigation, Remedy Selection,
Remedial Design, and Implementation
Remedy Implementation
Remedy Implementation
Investigation & time critical remedy
Remedial Design
                                                                                        C-14

-------
lEc
                       Principle 3:  Water Use and Impacts to Water Resources
                                  •    17 RPMs, or 50% of the respondents, have implemented this principle.
                                  •    Specific elements include:
                                          o  Minimize fresh water consumption and maximize water reuse during
                                              daily operations and treatment processes - 6
                                          o  Reclaim treated water for beneficial use such as irrigation - 3
                                          o  Use native vegetation requiring little or no irrigation - 7
                                          o  Prevent impacts such as nutrient loading on water  quality  in nearby
                                              water bodies (i.e., erosion prevention methods like silt fences) -14
                                          o  Other - Recycling treated  groundwater - 2 and using straw to retain
                                              water in soils being revegetated
                                  •    Reason for implementing the principle
                                          o  Technical -14
                                          o  Financial - 7
                                          o  To comply with Region 4 Clean and Green Policy - 2
                                          o  To comply with Greener Cleanup language added in an enforcement
                                              document - 0
                                          o  Other - Standard  DOE policy, Request of FWS,  Common  sense,
                                              Generally good practice
                                  •    Did the principle play a role in the selection of a remedy?
                                          o  Yes - 4
                                          o  No -12
                                  •    Did the principle affect the way the remedy was implemented?
                                          o  Yes - 4
                                          o  No-9
                                          o  Not applicable - 3
                                  •    Did the principle affect post construction activities?
                                          o  Yes - 4
                                          o  No-7
                                          o  Not applicable - 5
                                                                                                             C-15

-------
lEc
                               Sites where Principle 3 has been implemented
Site Name
Oak Ridge National Lab
Anniston PCB
Marshall Space Flight Center
Savannah River Site
Escambia Treating Company
Landia Chemical
Aqua Tech
Lexington County Landfill
Holtrachem
Reasor Chemical
Shuron
Sanford Gasification
TVA Kingston
Cape Fear
Geigy Chemical
Redstone Arsenal
Site 12 Jericho Island
Site 3 Causeway Island
Site 45 Former Drycleaner
ET Cover
Oak Ridge Reservation
East TN Technology Park
Sprague Electric
Gurley Pesticide
Stauffer Chemical Tarpon Springs
Calculator
Used?
No
No
No
No
No
Yes
No
No
No
No
No
Yes
Yes
No
No
No
No
No
No
No
No
No
No
No
No
Phase
Remedy Implementation
Remedy Selection/Feasibility Study
Investigation, Remedy Selection,
Remedial Design, and Implementation
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Remedy Implementation
Remedy Implementation and Post-
Construction Complete
Investigation
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Remedy Implementation
Post-Construction Complete
Post-Construction Complete
Remedy Implementation
Post-Construction Complete
Post-Construction Complete
Remedy Selection/Feasibility Study
and Maintenance Action
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Remedial Design
Soil Remedial Action Complete
Remedy Implementation
                                                                                        C-16

-------
lEc
                      Principle 4: Materials Management and Waste Reduction
                                  •   19 RPMs, or 56% of the respondents, have implemented this principle.
                                  •   Specific elements include:
                                          o   Use technologies designed to minimize waste generation - 8
                                          o   Reuse materials whenever possible -10
                                          o   Recycle materials generated at or removed from the site whenever
                                              possible -15
                                          o   Minimize natural resource extraction and disposal - 2
                                          o   Use passive sampling devices producing minimal waste, where feasible
                                              (i.e., use of baroball,  remote controlled sampling devices, etc.) - 7
                                          o   Other - autonomous  sampling and monitoring system reduces the
                                              generation of waste, use of local products, resale/salvage/re-use of SVE
                                              blower unit
                                  •   Reason for implementing the principle
                                          o   Technical  -14
                                          o   Financial -13
                                          o   To comply with Region 4 Clean and Green Policy - 3
                                          o   To comply with Greener Cleanup language added in an enforcement
                                              document - 0
                                          o   Other - Because it's  smart to recycle, Generally good practice
                                  •   Did the principle play a role in the selection of a remedy?
                                          o   Yes - 0
                                          o   No -19
                                  •   Did the principle affect the way the remedy was implemented?
                                          o   Yes - 8
                                          o   No-7
                                          o   Not applicable - 4
                                  •   Did the principle affect post construction activities?
                                          o   Yes - 5
                                          o   No - 10
                                          o   Not applicable - 4
                                                                                                            C-17

-------
lEc
                               Sites where Principle 4 has been implemented
Site Name
American Creosote Works
National Southwire Aluminum
Marshall Space Flight Center
National Electric Coil
Savannah River Site
Escambia Treating Company
ITT Thompson Institute
Solitron Microwave
Solitron Devices
Aqua Tech
Lexington County Landfill
Medley Farm Drum Dump
Staley PCE
Sheet Metal Restoration
Holtrachem
Oak Ridge
Sanford Gasification
Aberdeen Pesticide Dump
Barber Orchard
Blue Ridge Plating
Cape Fear
Chemtronics
National Stach & Chemical Co
Savannah River Site - T Area GW
-EOS
Barite Hill
Nevada Goldfields
Redstone Arsenal
Calculator
Used?
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
Yes
No
No
No
No
No
No
No
No
No
No
Phase
Post-Construction Complete
Post-Construction Complete
Investigation, Remedy Selection,
Remedial Design, and Implementation
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Remedy Implementation
Post-Construction Complete
Post-Construction Complete
Remedy Implementation
Remedy Implementation and Post-
Construction Complete
Remedy Implementation
Investigation
Investigation
Investigation
Remedial Design and Implementation
Remedy Implementation
Post-Construction Complete
Remedy Implementation
Post-Construction Complete
Post-Construction Complete
Remedy Selection/Feasibility Study
and Post-Construction Complete
Post-Construction Complete
Remedy Implementation
Investigation
Investigation
Remedy Implementation
                                                                                        C-18

-------
lEc
Site 12 Jericho Island
ET Cover
Oak Ridge Reservation
East TN Technology Park
Smokey Mountain Smelters
No
No
No
No
No
Post-Construction Complete
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Time critical removal action
                       Principle 5: Land Management and Ecosystem Protection
                                  •    16 RPMs, or 47% of the respondents, have implemented this principle.
                                  •    Specific elements include:
                                          o   Use minimally invasive in situ technologies - 7
                                          o   Use passive energy technologies such as bioremediation and
                                              phytoremediation as primary remedies or "finishing steps," where
                                              possible and effective - 7
                                          o   Minimize soil and habitat disturbance -13
                                          o   Minimize bioavailability  of contaminants through adequate source
                                              control/containment - 8
                                          o   Reduce noise and lighting disturbance - 4
                                          o   Other - Use of native plants where new planting was required - 2 and
                                              early soil source removal
                                  •    Reason for implementing the principle
                                          o   Technical -13
                                          o   Financial - 6
                                          o   To comply with Region 4 Clean and Green Policy - 2
                                          o   To comply with Greener  Cleanup language added in an enforcement
                                              document - 0
                                          o   Other - Better for the environment, To keep citizens happy, Good
                                              practice - especially for the site environment
                                  •    Did the principle play a role in the selection of a remedy?
                                          o   Yes - 5
                                          o   No-11
                                  •    Did the principle affect the way the remedy was implemented?
                                          o   Yes - 5
                                          o   No-11
                                          o   Not applicable - 0
                                                                                                             C-19

-------
lEc
                                    Did the principle affect post construction activities?
                                       o  Yes - 3
                                       o  No-5
                                       o  Not applicable - 8
                                    Sites  where Principle 5 has been implemented
Site Name
Oak Ridge National Lab
National Southwire Aluminum
Marshall Space Flight Center
Savannah River Site
Escambia Treating Company
ITT Thompson Institute
GMH Electronics
Murray Ohio Dump
M Area OU
Cape Fear
Savannah River Site
Barite Hill
Nevada Goldfields
Redstone Arsenal
M Area OU
Aberdeen Pesticide Dump
Barber Orchard
Blue Ridge Plating
Cape Fear
Chemtronics
National Stach & Chemical Co
Geigy Chemical
Redstone Arsenal
Site 1 Incenirator
Calculator
Used?
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
No
Phase
Remedy Implementation
Post-Construction Complete
Investigation, Remedy Selection,
Remedial Design, and Implementation
Remedy Implementation
Remedy Implementation
Remedy Implementation
Remedy Selection/Feasibility Study
and Remedy Implementation
Remedial Design
Remedy Implementation
Post-Construction Complete
Remedy Implementation
Investigation
Investigation
Investigation
Remedy Implementation
Post-Construction Complete
Remedy Implementation
Post-Construction Complete
Post-Construction Complete
Remedy Selection/Feasibility Study
and Post-Construction Complete
Post-Construction Complete
Post-Construction Complete
Remedy Implementation
Post-Construction Complete
                                                                                                      C-20

-------
lEc
Site 5 Jericho Causeway Landfill
Site 45 Former Drycleaner
ET Cover
Oak Ridge Reservation
East TN Technology Park
Sprague Electric
No
No
No
No
No
No
Post-Construction Complete
Remedy Selection/Feasibility Study
and Maintenance Action
Post-Construction Complete
Remedy Implementation
Remedy Implementation
Remedial Design
                      Other Greener Cleanup Practices

                      PRP Lead Sites

                      Do you have any PRP-Lead sites?
                             22-YES
                             9-NO

                      Is there an enforcement agreement in place for any of your PRP-Lead sites?
                             20-YES
                             2-NO

                      Was there Greener Cleanup Language in any of the agreements?
                             0-YES
                             20-NO

                      Did you have any PRPs that were willing to incorporate aspects of your regional Clean and Green
                      policy or other Greener Cleanup elements or best management practices?
                             16-YES
                             6-NO

                      Did any PRPs perform actions above and beyond the minimum Greener Cleanup requirements?
                             4-YES
                             6-NO
                             11 - Not applicable

                      Additional Questions

                      Are you familiar with the EPA Region 4 Clean and Green policy?
                             26-YES
                             5-NO

                      Are you familiar with the Superfund Green Remediation Strategy?
                             28 - YES       3 - NO
                                                                                                       C-21

-------
lEc
                    REGION 9 LIST OF GR ACTIVITIES:




                    Region 9 Superfund Green Remediation Projects
Site
Aerojet
Apache Powder
Camp Pendelton
Marine Base
Del Amo
Frontier Fertilizer
Frontier Fertilizer
Iron Mountain Mine
Lawrence
Livermore Lab
Leviathan
McColl
Newmark
Ohlone
College/Newark
Center
Pemaco
Selma Treating
Company
Stringfellow
Alameda NAS
Travis AFB
Tucson
International
Airport
Apache Powder
McKinley Mine,
Navajo Nation
Nineteenth Avenue
Landfill
Operating Industries
Type
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Brownfields
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
Superfund
(none -closed
coal mine-no
substantive
EPA program)
Superfund
Superfund
Project
3.6MWPV
Wind and 1.4 KW PV system for constructed wetlands
to conduct bioremediation
Clean diesel/biodiesel/DPF for non-road equipment
during earthmoving operation
Oxygen injection to reduce time (and energy use) of
soil vapor extraction system
5.7 KW PV system for groundwater treatment
500 KW PV system for in-ground heating
Ran transmission line to use local hydro power instead
on on-site diesel generators
PV to power groundwater treatment
NREL renewable energy evaluation
PV panel to power sump pump
PV powered monitoring equipment at 50 wells
600 KW PV system — 45% of campus electricity need
3.4KWPVsystem
Molasses injection into groundwater plume reduced
cleanup time 30-50 years/energy use
NREL renewable energy evaluation
Life Cycle Analysis
Life Cycle Analysis
Life Cycle Analysis (HQ support)
Technical support for solar farm development
Technical support for renewable energy development
Support for Phoenix and ADEQ for possible solar
development
Six-70 KW microturbines burn landfill gas
Status
Operational;
additional 2.4
MW in planning
Operational
Completed
Operational
Operational
Planning/not
feasible
Operational
Operational
In process
Operational
Operational
Operational
Operational
Operational
In process
In process
In process
In process
In process
In process
In process
Operational
                                                                                             C-22

-------
 lEc
APPENDIX  D:  EPA  CLU-IN  USE DATA
                           GR Focus Stats
                           E«tract«d from monthly CUMN rtports;

                                                   23,4   133
                                     Total Requests for GR Focus Pages
                              ss sss * ss
                                          Direct Vlsltofs (%)
                                                                                    Estimated Visits
                           >v >w sss sss *
                                                                                 Average Visit Length (min)
                                                                         s^s^r
                                                                                                                           Visitor Repeat Rate (%)
ss sss * ss sss
                                                                                                                             Total User Hours
                           Notable
                           rnethodology d
                           methodology- a
                           UST fact sheet
                           mtrofactsheet(>4>datB)
                           reri«wable enera- Fact sheet
                           contracting toolkit
                           Superfurd GR Strategy
                           dean fuels/emissions fact sheet
                           SVE&«fact sheet
                           biorernedation
                           P&T
                           site invesogaCiorr
                           excavation 8. restoration
September 16, 2011
^uly 7, 2011
June M, 2011
tutey 3A20U
Ap-il 13, 2011
February 1, 2011
September 2ft 2010
August 31. 2010
March 24 2010
March 24, 2010
January 18, 2010
Ifirn.lr,- I :, ;-- -M
January 23, 2009
                                                                                                                                                                               D-1

-------
Superfund Green Remediation Strategy: strategic action progress based on use of online products and training
Detailed CLU-INGR Focus slots as of August 1, 2011, unless noted otherwise
Action 2.3 Develop technology-specific assessment tools and fact sheets
An Introduction (May 11, 2011, update * original); http://www.duin.crg/greenremedi3tiw/dccs/GR_Quick_Ref_FS_lntro.pdf
Excavation and Surface Restoration (EPA 542-F-08-012, December 2008);
http^/www. cluin.org/grpenremediati on/doc s/GRJ3uick_Ref_FS_exc_rest.pdf

Site Investigation {£PA 542-F-09-OD4, December 2009); http://www.cluin.org/greenremediation/docs/GR_Fact_Sheet_SI_12-31-20Q9.pdf
Pump and Treat Technologies (EPA 542-F-09-C05, December 2009J; http://www.cluin.org/greenremed!ation/docs/GR_Fact_Sheet_P&T_12-31-
2009.pdf
Bioremediation (EPA542-F-10-006, March 2010);httpv'/www.cluin.CTB/greenremediation/docs/QR_fact^eet_biorem_B2410.pdf
Soil Vapor Extraction & Air Sparging (EPA 542-F-10-007, March 2010);
http;//www. cluin.org/greenremedJati on/doc s/GR_factsheet_SVE_AS_032410.pdf
Sfteswith Leaking Underground Storage Tank Systems (EPA 542-F-ll-QOS, June 2011);
http://www.duin.orB/greenremediati on/docs/ US T_GR_fact_sheet.pdf
Total
                                                                                                                                    S of Views
                                                                                                                                                     EPA   other gov   non gov  ft of Downloads    EPA    otliei gov   non gov
280
2,659
1,190
2,120
1,842
430
10,238
0
10
31
37
24
0
US
3
8
33
31
47
112
350
272
2,641
1,126
2,052
1,771
318
9,760
Action 2.6 Deliver or host green remediation training through the Technology Innovation and Field Services Division's Training
Infrastructure (as of August 17,2011)
CLU IN Web Seminars (open to the public)
Greener Cleanups - EPA's Methodology for Understanding and Reducing a Project's Environmental Footprint. August 10, 2011.
NARPM Presents: Ecological Revitalization: Turning Contaminated Propertiesfnto Communfty Assets. March 15, 2011.
US and EU Perspectives on Green and Sustainable Remediation (Part 2 of ConSoil 2010 follow-up). March 15, 2011.
Your Rote in Green Remediation  Implementation and Case Studies In Green Remediation - This Year's Models and Tools (2010 NARPM green
remediation session follow-on Webinar series )
     Session 1; Decembers, 2010
     Session 2; January 11, 2011
     Session 3; February 10, 2011
Identifying & Evaluating Ecosystem Services at Contaminated Sites Prior to Remediation. August IS, 2010.
US and EU Perspectives on Green and Sustainable Remediation. July 12, 2010.
Energy for the Future: Exploring Methane Gas-to-Energy Projects at Superfund Sites- May 6, 2010.
Green Remediatfon: Applying Strategies in the Field {2009 NARPM green remediation session foltlow-on Webinar series}
     Session 1; October 8, 2009
     Session 2; November 12, 2009
     Session 3; December 15, 2009
Understanding  Life Cycle Assessment: Applications for OSWEffs Land and Materials Management September 23, 2009.
Solar Energy on Closed Landfills: Regulatory and Technical Considerations. September 9, 2009.
Tackling the Carbon Footprint at Pump and Treat Projects: A Case Study in Energy Efficiency.  March 10, 2009.
Green Remediation Voluntary Standards Initiatfve.  March 4, 2009
Green Remediation: Opening the Door to Field Use (2OO8 NARPM gresn remediation sessianfolltow-on Webinar series}
     Introduction and Carbon Calculus: A RCRA Case Study. Session A: November 24, 2008
     Green Remediation Toolsand Examples. Session B: December 16, 2008
     Green Remediation Toolsand Examples. Session C: January 13, 2009
USEPA-ILEPA Green Remediation Update. Decembers, 2008
ConSoil 2008Speci3l Session: Green Remediation. June 5, 2008
Earth Day 200S Green Remediation Panel Session; green remediation, soil amendments for impaired lands, and renewable energy on
contaminated lands. April 2008.
Total

Training Courses (in-person sessions open to the public, based on OSRTI tally of August 10, 2011)

Best Management Practices for Green Remediation Footprint Reduction; August 29, 2011; Westin New Orleans Canal Place, New Orleans, LA
Best Management Practices for Green Remedfation Footprint Reduction; November 15, 2011; U.S. EPA Region 2, New York, NY
S of Participants
359
76
125

199
200
218
119
174
74
]"-•
160
157
in?
191
202
190
200
205
189
267
28
232
3,863

near future
near future
1 IV
40
24
13

80
45
45
9
4
17
53
36
26
33
M
17
>.i
32
29
45
12
3
17
-I-.
\ IV,


olliei gov
37
19
1"

39
33
43
44
49
10
.-,i
35
24
A':
,,u
41
40
43
37
48
179
4
69
1,013
ortiei gov


non gov
232
33
94

80
122
130
66
121
47
100
89
107
31
: 'is
144
120
125
as
96
76
21
146
2,227
rion gov


Hours
2
2
2
2
2
2.25
2.25
1.5
2
2
2
2
2
1.5
1
1
2
2
2
2
2
1.5

39.00

8
.-:
                                                                                                                                                                                                                                               D-2

-------
Total
Total number of participants and training hours in public venues

Annual N AH I'M Training Conference (based on OSRTI tally of August 10,2011)
ISth; Green Remediation: Opening the Door to Field Use. July 9, 2008
19th: Green Remediation - What's Next. June 3, 2009
20th: Your Role in Green Remediation Implementation and Case Studies in Green Remediation: This Year's Models and Tools. May 26, 2010
21st: Greener Cleanups
     An RPM's Primer. May 17, 2011
     Case Studies and Discussion. May 18, 2011
     Addressing Your Project's Environmental Footprint • Theory and Practice. May 18, 2011
Total
Annual OSC Readiness (based on OSRTI tally of August 10,2011)
12th: Greening Response Actions I. February 12, 2009
13th: Greening Response Actions II. February 1, 2010
14th: Greening Response Actions III. January 31, 2011
Total
Total number of RPMs/OSCs trained and training hours delivered
Total number of others trained at NARPM and OSC Readiness

Action 3.4 Develop a fact sheet on using green power for site cleanup
Using input from RPMs experienced in applying onsite renewable energy, OSRTI issued a fact sheet entitled: Green Remediation Best
Management Practices: integrating Renewable Energy into Site Cleanup (EPA S42-F-11-006) in March 2011,
nttp://www.c lu in. o i g/gieenremetl ia lion/docs/I ntegrating_RE_into_site_cleanup_factsheet.pdf

Action 4.1 Develop a fact sheet on dean fuel and emission technologies
With input from OTAQ and regional offices, OSRTI issued a fact sheet entitled: Green Remediation Best Management Practices: Clean Fuel &
Emission Technologies for Site Cleanup (EPA 542-F-10-008) in August 2010.
http^/www.cluin.org/greenremediation/doc5/dean_Fuel Emis_GR_fact_sheet_S-31-10.pdf

Action 4.3 Identify opportunities for recovering and using methane gas emitted from landfills on Superfund sites
Using a screening tool developed by OSRTI, the Agency evaluated feasibility of methane recovery at six NPL sites, as described in the Superfund
Landfill Merhane-to-Energy Pilot Project (OSWER 9200.081, December 2010) released in June 2011.
http-y/www.cluin.org/greenremediation/docVLand fill_Methane_Rnal_Report_051031.pdf

Action 5.2 Develop and pilot test a green remediation analysis template to help collect information during various phases of the
remediation process at any site
     Ronnie Environmental Technologies Corporation facility in Palo Alto, CA; http^/www.cluin.org/greenremediation/romic/index.cfm,'
     hftp^/www.cluin.org/greenremediation/romic/docs/r omic_report_rev_051110.pdf
     Former BP Products North America, Inc. Facility in Wood River, IL; http://www.cluin.org/greenremediation/bpwoodriver/Index.cfm;
     httpy/www.cluin.org/6reenremediat(on/bpwoodrIver/docs/final_8P_report_111510.pdf
     Travis Air Force Base
Total

Action 5.3 Incorporate green remediation factors Into remedy optimization evaluations
As part of remedy optimization studies at Superfund sites, OSRTI has evaluated various options for reducing the environmental  footprint of cleanup
at Shepley's Hill Landfill in Devens, MA. http://www.clu-
in.org/greenremediation/docs/Rnal%20Shepleys%20Hill%20Landfiliyo20RSEyo20082109.pdf

Action 6.3 Develop and periodically update a green remediation contracting toolkit
In June 2009, OSRTI initially released the Green Response and Remedial Action Contracting and Administrative Toolkit, which was updated in
October 2010. Injanuary 2011, OSRTI issued the (re-titled) document, Greener Cleanups Contracting and AdminhtratMe Toolkit,http://www.clu-
in.org/gfeenremediathn/docs/Greener__Cleanups_Contracting_and_Adminhtrative_Tootkit,pdf

Action 8.1 Analyze existing methods and software tools for evaluating the environmental footprint of a cleanup
            M                                            16
         3,863    623       1,013      2,227             55.00
tt of Partipcants
85
55
H
47
S3
46
123

31
46
is
105
202
zje
RPMs
43
v
•'.'.
17
13
19
157







other
42
25
22
•.'i
20
27
166
OSCs other
10 21
22 24
13 15
45 60


Hours
S
:-,
:-:
4
4
4


4
4
4

48

S of Views   EPA   other gov   non gov   ft of Downloads    EPA   other gov   non gov


                                                    332      0           5      327

tt of Views   EPA   other gov   non gov   It of Downloads    EPA   other gov   non gov


                                                    347      4           6      337

tt of Views   EPA   other gov   non gov   tt of Downloads    EPA   other gov   non gov


                                                     46      0           0        46


0 of Views   EPA   other gov   non gov   tt of Downloads    EPA   other gov   non gov

     1,037     96         10       931              146      14           2      130

                                                     50      3           0        47

                                                    196

tt of Views   EPA   other gov   non gov   tt of Downloads    EPA   other gov   non gov
          270     19           2       249
unava'lable 8/1
         1,307
                                                        268      S          3       257

   tt of Views    EPA   other gov  non gov   tt of Downloads    EPA   other gov   non gov


                                                      1,227     71         21      1,135

   ttofViews    EPA   other gov  nongov   tt of Downloads    EPA   other gov   non gov
                                                                                                                                                                                                                                    D-3

-------
OSRTI has identified more than 50 Web-based calculators, software models, and other tools that can help quantify environmental outcomes
associated with one or more core element of green remediation. Four additional, no-cost online tool shave been developed by other federal
agencies or states to specifically address green and sustainable site cleanup. http://www.clu-m.crg/greenremediation/subtabj33.cfrn                            543    276         121       151

Action 8.2 Develop an Agency methodology for evaluating the environmental footprint of a cleanup                                       8 of Views    EPA   other gov  nongov   # of Downloads    EPA   other gov   non gov
OSRTI released Footprint Analysis foi Environmental Cleanups                                                                                 near future
     Footprint Assessment: Methodology [www.cluin.org/greenremediation/rnethodologypage to be activated Septembers]                                                                          near future
     Footprint Analysis for Environmental Cleanups [PDF to be posted September Son "metnodolcgy" page]
                                                                                                                                                                                                                                          D-4

-------
lEc
                     APPENDIX E: LIST OF REFERENCES

                     2010 Green Remediation Coordination Meeting Surveys. Oct. 2010. Atlanta, GA.

                     Air Force Center for Engineering and Environment. "Sustainable Remediation Tool
                            (SRT)."
                            (http://www.afcee.af.mil/resources/technologytransfer/programsandinitiatives/sus
                            tainableremediation/srt/index.asp).

                     Association of State and Territorial Solid Waste Management Officials (ASTSWMO).
                            "Greener Cleanups Task Force (GCTF) Incentives for Greener Cleanups." June
                            2009..(http://www.astswmo.org/Files/Policies_and_Publications/Sustainability/G
                            reener_Cleanups/GCTF_Incentives_Paper_6-25 -09 .pdf).

                     ASTSWMO. "Green Remediation: Getting Started by Debunking Some Myths." August
                            2009.
                            (http://www.astswmo.org/Files/Policies_and_Publications/Sustainability/Greener
                            _Cleanups/Green-Remediation-Myth-Busters_August-2009 .pdf).

                     California EPA Department Toxic Substances Control (DTSC). "DTSC's Interim
                            Advisory for Green Remediation." December 2009.
                            (http://www.dtsc.ca.gov/OMF/Grn_Remediation.cfm).

                     California EPA DTSC. "Green Remediation Evaluation Matrix (GREM)."
                            (http://www.dtsc.ca.gov/OMF/Grn_Remediation.cfm)

                     EMS (an EPA contractor). U.S. EPA Superfund CLU-IN Website Use and Download
                            Data. August-September 2011.

                     Illinois EPA Greener Cleanups Webpage (http://www.epa.state.il.us/land/greener-
                            cleanups/)

                     Illinois EPA. "Greener Cleanups: How to Maximize the Environmental Benefits of Site
                            Remediation (Greener Cleanups Matrix)."
                            (http://www.epa.state.il.us/land/greener-cleanups/matrix.pdf).

                     GeoTrans, Inc. "Environmental Footprint Analysis of Three Potential Remedies: BP
                            Wood River, Wood River, IL." November 15, 2010. (http://www.clu-
                            in.org/greenremediation/bpwoodriver/index.cfm).

                     GeoTrans, Inc. "Environmental Footprint Analysis of Three Potential Remedies: Former
                            Romic Environmental Technologies Corporation Facility, East Palo Alto,
                            California." May 11, 2011. (http://www.clu-
                            in.org/greenremediation/romic/index.cfm).

                     Geo Trans, Inc. "Shepley's Hill Landfill RSE & GR Evaluation." August 21, 2009.
                            (http://www.clu-in.org/greenremediation/subtab_b3.cfm).

                     Massachusetts Department of Environmental Protection (MassDEP). "Sustainable
                            Remediation - FAQs."
                            (http://www.mass.gov/dep/cleanup/priorities/susrem.htm).

                     Minnesota Pollution Control Agency. "Greener Practices for Business, Site Development,
                            and Site Cleanups: A Toolkit." Last modified April  2010.
                            (http://www.pca.state.mn.us/index.php/topics/preventing-waste-and-
                                                                                                   E-1

-------
lEc
                            pollution/sustainability/greener-practices-toolkit/greener-practices-for-business-
                            site-development-and-site-cleanups-a-toolkit.html).

                     National Association of Remedial Project Managers (NARPM). "The 21st Annual
                            NARPM Training Program." Kansas City, MO. May 16-20, 2011.
                            (http://www.epanarpm.org/2011/).

                     National Association of Remedial Project Managers (NARPM). "The 20th Annual
                            NARPM Training Program." Arlington, VA. May 24-28 , 2010.
                            (http://www.ttregistration.com/narpm2010/conference/index.cfm).

                     National Association of Remedial Project Managers (NARPM). "2009 NARPM Annual
                            Training Conference." Atlanta, GA. June 2-5, 2009.
                            (http://www.epanarpm.org/2009/home.htm).

                     New York State Department of Environmental Conservation (NYSDEC). "DER-
                            3 I/Green Remediation." Revised January 2011.
                            (http://www.dec.ny.gov/docs/remediation hudson_pdf/der31 .pdf).

                     Oregon Department of Environmental Quality (DEQ). "Draft DEQ Green Remediation
                            Policy." October 2010. (http://www.deq.state.or.us/lq/cu/greenremediation.htm).

                     Sustainable Remediation Forum (SURF)."Sustainable Remediation White Paper—
                            Integrating Sustainable Principles, Practices, and Metrics Into Remediation
                            Projects."  September 2009. (http://www.sustainableremediation.org/librarv/issue-
                            papers/).

                     Tetra Tech GEO (formerly GeoTrans, Inc.) "Environmental Footprint Analysis of Four
                            Potential Remedies: Site DP039, Travis Air Force Base, Fairfield, California."
                            July 15, 2001. (http://www.clu-in.org/greenremediation/travis/index.cfm).

                     U.S. Department of Defense, Office of the Under Secretary of Defense. "Consideration of
                            Green and Sustainable Remediation Practices in the Defense Environmental
                            Restoration Program." August 10, 2009.

                     U.S. EPA CLU-IN Internet Seminar. "Greener Cleanups - EPA's Methodology for
                            Understanding and Reducing a Project's Environmental Footprint." August 10,
                            2011. (http://www.clu-in.org/greenremediation/).

                     U.S. EPA. "Superfund & Green Remediation." US Environmental Protection Agency.
                            Last updated September 22, 2011.
                            (http://www.epa.gov/superfund/greenremediation/).

                     U.S. EPA Office of Solid Waste and Emergency Response. "Green Remediation:
                            Incorporating Sustainable Environmental Practices  into Remediation of
                            Contaminated Sites (EPA 542-R-08-002)." April 2008. (http://www.clu-
                            in.org/greenremediation/docs/Green-Remediation-Primer.pdf).

                     U.S. EPA Office of Solid Waste and Emergency Response. "Green Remediation Best
                            Management Practices (BMP) Fact Sheet." September 2011. (http://www.clu-
                            in.org/greenremediation/docs/GRfactsheet topics  update.pdf).

                     U.S. EPA Office of Solid Waste and Emergency Response. "Principles for Greener
                            Cleanups." August 27, 2009.
                            (http://www.epa.gov/oswer/greencleanups/pdfs/oswerjgreencleanup_principles.p
                            df).
                                                                                                    E-2

-------
lEc
                     U.S. EPA Office of Solid Waste and Emergency Response. "Superfund Green
                            Remediation Strategy Activity Tracking Chart." February 16, 2011.

                     U.S. EPA Office of Solid Waste and Emergency Response - Office of Superfund
                            Remediation and Technology Innovation. "Contaminated Site Cleanup
                            Information (CLU-IN)." Last updated September 18, 2011. (http://www.clu-
                            in. org/greenremediation/).

                     U.S. EPA Office of Solid Waste and Emergency Response - Office of Superfund
                            Remediation and Technology Innovation. "Greener Cleanups Contracting and
                            Administrative Toolkit, (Update)." January 2011.
                            (http://www.cluin.org/greenremediation/docs/Greener_Cleanups_Contracting_an
                            d_Administrative_Toolkit.pdf).

                     U.S. EPA Office of Solid Waste and Emergency Response - Office of Superfund
                            Remediation and Technology Innovation. "Methodology for Understanding and
                            Reducing a Project's Environmental Footprint." September 16, 2011.
                            (http://www.clu-
                            in.org/greenremediation/methodology/docs/GC  draft EPA Footprint Methodol
                            ogy 091611.pdf).

                     U.S. EPA Office of Solid Waste and Emergency Response - Office of Superfund
                            Remediation and Technology Innovation. "Profiles of Green Remediation." Last
                            updated July 12, 2011.  (http://www.cluin.org/greenremediation/tab_d.cfm).

                     U.S. EPA Office of Solid Waste and Emergency Response - Office of Superfund
                            Remediation and Technology Innovation. "Superfund Green Remediation
                            Strategy." September 2010. (http://www.epa.gov/superfund/greenremediation/sf-
                            gr-strategy.pdf).

                     U.S. EPA Office of Superfund Remediation and Technology Innovation. "Energy
                            Consumption and Carbon Dioxide Emissions at Superfund Cleanups." May 2008.
                            (http://clu-in.org/greenremediation/docs/SF Energy  Carbon Footprint.pdf).

                     U.S. EPA Region 1 Clean and Green Policy for Contaminated Sites. February 2010.
                            (http://www.epa.gov/regionl/brownfields/pdfs/CleanGreenPolicy.pdf).

                     U.S. EPA Region 2 Clean and Green Policy. March 2009.
                            (http://epa.gov/region2/superfund/green remediation/clean  and green_policy.pd
                            f).
                     U.S. EPA Region 2 Superfund.  "Green Remediation Measurements/Evaluation Subgroup
                            - Metrics." Last updated October 5, 2010.
                            (http://www.epa.gov/region02/superfund/green_remediation/metrics.html).

                     U.S. EPA Region 2 Metric Tracking Database. (Not publicly available)

                     U.S. EPA Region 3 Greener Cleanup and Sustainable Reuse Policy. January 2010.
                            (http://www.epa.gov/reg3hscd/bf-
                            Ir/whatsnew/EPA  R3 Greener Cleanup Sustainable Reuse%20Policy.PDF).

                     U.S. EPA Region 3 Green Remediation 2009 Questionnaire.

                     U.S. EPA Region 4 Clean and Green Policy. February 2010.  (http://clu-
                            in.org/greenremediation/docs/R4GRPolicy.pdf).

                     U.S. EPA Region 4 Greener Cleanups Survey Data, January 2010.
                                                                                                   E-3

-------
lEc
                    U.S. EPA Region 5 Interim Greener Cleanup Policy. November 2009.
                            (http://www.epa.gov/r5brownfields/pdf/gcinterimpolicyr05signedcopvll09.pdf).
                    U.S. EPA Region 6 Clean and Green Policy. September 2009 .(http://clu-
                            in.org/greenremediation/docs/R6GRPolicy.pdf).
                    U.S. EPA Region 7 Interim Green Cleanup Policy. September 2009.
                            (http://www.epa.gov/region7/cleanup/pdf/r7 green cleanup_policy.pdf).
                    U.S. EPA Region 8 Green Remediation Policy. August 2009. (http://clu-
                            in.org/greenremediation/docs/R8GRPolicv  revl.pdf).
                    U.S. EPA Region 9 Greener Cleanups Policy. September 2009.
                            (http://www.epa.gov/region9/climatechange/pdfs/greener-cleanups-policy.pdf).
                    U.S. EPA Region 9 List of Green Remediation Activities.

                    U.S. EPA Region 10 Clean and Green Policy. August 2009.
                            (http://vosemite.epa.gOv/R10/EXTAFF.NSF/0/bee5cf8b41felfdl8825761c006bb
                            9fb/$FILE/ATTBM7LK/clean and  green policv%20R10%208%2013%2009.pd
                            f).
                    U.S. Navy. "SiteWise GSR Tool." (http://www.ert2.org/t2gsrportal/SiteWise.aspx).

                    Wisconsin Department of Natural Resources (DNR). "Wisconsin's Initiative for
                            Sustainable Remediation and Redevelopment (WISRR) (formerly known as
                            WISC)." (http://ua.dnr.wi.gov/org/aw/rr/cleanup/wisc.htm).

                    Wholey, Joseph S., Harry P. Hatry, and Kathryn E. Newcomer. Handbook of Practical
                            Program Evaluation. Second ed. San Francisco, CA: Jossey-Bass, 2004.
                                                                                                  E-4

-------