&EPA
Formative Evaluation of the
OSWER Community
Engagement Initiative
Fact Sheet
http://www.epa.gov/evaluate
For more information on
completed evaluations at
EPA or the Evaluation
Support Division, visit the
above link.
November 2013
Introduction
In December 2009, EPA's Office of Solid Waste and Emergency Response (OSWER) announced
the Community Engagement Initiative (CEI) to enhance OSWER headquarters and Regional
offices' engagement with local communities and other stakeholders, and to help stakeholders
meaningfully participate in decision-making processes related to the cleanup and reuse of
contaminated sites. OSWER envisions highly effective community engagement becoming a
standard business practice across all of its programs.
OSWER requested a formative evaluation for the CEI. The CEI is comprised of 16 actions that
affect many aspects of OSWER's work. The evaluation focuses on Action 7 and Action 13A,
which address improving technical assistance (TA) and information dissemination (ID),
respectively. Furthermore, this evaluation focuses on Superfund, RCRA Corrective Action (CA),
and Brownfields programs within OSWER. The goals of the evaluation were to establish a
baseline of current practice, to establish a baseline of current community satisfaction with
EPA's TA and ID practices, and to consider the feasibility of potential measures for tracking TA
and ID moving forward.
Evaluation Questions
1. What are the requirements and drivers for community involvement within the Superfund,
RCRA Corrective Action, and Brownfields programs?
2. What is the baseline of current OSWER TA and ID activities, with respect to:
a. Frequency of practice and program-to program variability?
b. Proportion of communities that receive formal assistance through Technical Assistance
Grants (TAG), the Technical Assistance Services for Communities (TASC) contract, or
other formal program?
c. Selection criteria and on-the-ground process for accessing TAG and TASC (for
Superfund and RCRA CA only)?
d. Community assistance that is provided outside of formal programs (including helping
to set up community advisory groups and providing ad-hoc assistance)?
e. Areas of unmet TA or ID need?
3. How can Superfund community involvement plans (CIPs) be used to improve technical
assistance and delivery of information?
a. Does every site have a CIP?
b. What information is available on the implementation of CIPs?
c. Are CIPs revised over time?
d. Do CIPs, as they are currently used, ensure effective TA and information delivery
throughout the life of the project? Why or why not?
-------
4. What is the baseline of customer satisfaction with OSWER TA and ID activities?
5. What measures can be used to assess the effectiveness and tangible outcomes of OSWER TA and ID activities across
the lifecycle of site planning, remediation, and reuse?
6. How can these measures be used to improve OSWER TA and ID activities?
Evaluation Methods
IEc reviewed existing documentation from the following sources: documentation of community engagement
requirements; site inventory data for cleanup programs; performance measurement resources (for Questions 5 and 6);
and existing satisfaction interviews.
lEc conducted 46 interviews with EPA HQ and regional contacts from Superfund, Brownfields, RCRA CA, as well as state
RCRA CA contacts in Arkansas, California, Illinois, Missouri, New York, and South Carolina. lEc also conducted
satisfaction interviews with contacts from community groups that had received technical assistance through the
Superfund TAG program, and analyzed these data in conjunction with previous satisfaction interviews of service recipients
conducted for the TASC contract. lEc conducted a thematic analysis of interview data, and synthesized interview data
with information gleaned from the document review as applicable, to develop evaluation findings.
Key Findings
High-level findings from the evaluation include the following:
• The EPA Superfund and Brownfields programs appear to have robust systems in place for delivering technical
assistance and information to communities, including clear mandates and guidance, formal programs and
mechanisms for delivering technical assistance, and adequate levels of EPA staffing and resources.
• RCRA CA appears to be meeting its mandate of providing information dissemination to community groups as
required. However, in comparison to the Superfund and Brownfields programs, RCRA CA lacks many inputs
helpful in ensuring the delivery of technical assistance to communities, including a lack of: regulatory mandates;
adequate resources and staffing at EPA and state agencies; and up-to-date, program-specific guidance. Unmet
community needs appear to be higher within the RCRA CA program than other programs, and satisfaction with
technical assistance provided by states to communities under RCRA CA was rated lower on average by
interviewees than for the other two programs.
• Compared to the other programs, it is more difficult to characterize the needs of RCRA CA communities
nationally, and to track progress in meeting those needs, because the program is largely delegated to states, and
EPA currently lacks mechanisms for collecting community engagement data from states. Thus, if EPA were to
conduct regular tracking of measures of unmet needs and customer satisfaction, the Agency would need to work
with states to implement a data collection system. In contrast, existing data collection systems employed by
Superfund and Brownfields could potentially be augmented to track suggested measures.
Contact(s)
• Michelle Mandolia, EPA Evaluation Support Division, mandolia.michelle@eDa.QOv
• Ellen Manges, EPA Office of Solid Waste and Emergency Response, manges.ellen@epa.QOv
• Jackie Harwood, EPA Office of Solid Waste and Emergency Response, harwood.jackie@epa.gov
Report Link: httD://www.eDa.Qov/evaluate/reDorts/index.htm
Date Completed: November 2013
Publication Number: 100-F-15-003
------- |