SEPA Improving EPA's Performance
with Program Evaluation
An Evaluation of Partnerships to Address
Environmental Justice Issues in Communities
Series No. 9A
By continuously evaluating its programs, EPA is able to capitalize on lessons learned and incorporate that
experience into other programs. This enables the Agency to streamline and modernize its operations while promoting
continuous improvement and supporting innovation. This series of short sheets on program evaluation is intended
to share both the results and benefits of evaluations conducted across the Agency, and share lessons learned
about evaluation methodologies in this evolving discipline. For more information contact EPA's Evaluation Support
Division at www.epa.gov/evaluate.
At a Glance
Evaluation Purpose
To identify the value of using collaboration to
address environmental justice issues in distressed
communities.
Evaluation Type
Process/Outcome Evaluation
Publication Date
January 2003
Partners
Federal Interagency Working Group on
Environmental Justice, Office of Environmental
Justice, Office of Policy, Economics and
Innovation
Contact
Charles Lee, OEJ (202) 564-2597
Background: Why was an evaluation
performed?
An evaluation was conducted to assess the value of using
collaborative projects to address environmental justice issues in
predominantly low-income or minority communities. The
evaluation is built upon six case studies that were written primarily
between December 2001 and July 2002. These projects are part
of the Federal Interagency Working Group (IWG) on
Environmental Justice's national demonstration projects announced
in June 2000. These projects are representative of the IWG's effort
to build "dynamic and proactive partnerships among Federal
agencies to benefit environmentally and economically distressed
communities." In 2001, EPA's Office of Policy, Economics and
Innovation (OPEI), with the support of the IWG and EPA's Office
of Environmental Justice (OEJ), agreed to conduct case studies
and a program evaluation of six demonstration projects. The case
studies describe collaborative projects in (1) Annette Island,
Alaska; (2) East St. Louis/St. Glair County, Illinois; (3) New Madrid
County, Missouri; (4) San Diego, California; (5) Spartanburg, South
Carolina; and (6) Washington, DC.
NCEI
NATIONAL CENTER FOR
ENVIRONMENTAL INNOVATION
-------
Basic Evaluation Approach: How
did they do it?
The case studies and evaluation report were developed
using roughly fourteen steps, which are outlined below.
Step I: Develop guiding principles for the evaluation.
Step II: Develop key evaluative questions.
Step III: Develop an evaluation strategy.
Step IV: Gather input on the evaluation strategy
from a range of participants in a
facilitated national conference call.
Step V: Prepare a basic interview guide.
Step VI: Hold conference calls with project leaders
to discuss the evaluation strategy and gain
acceptance for the evaluative effort.
Step VII: Review pertinent project background material.
Step VIM: Develop a list of potential project
interviewees.
Step IX: Conduct interviews with project
participants.
Step X: Analyze interviewee data and develop
draft case studies.
Step XI: Distribute draft case studies to
interviewees for their review.
Step XII: Analyze case studies to develop the
evaluation report.
Step XIII: Distribute the evaluation report and case
studies to interviewees and
representatives of the academic
community for their review.
Step XIV: Complete the evaluation report and case
studies.
Evaluation Results: What was
learned?
Evaluation findings indicate that the collaborative
projects are producing a variety of important results,
including: (1) the improved opportunity for local
residents and community organizations to have a
genuine say in efforts to revitalize their communities;
(2) the enhancement of relationships among
stakeholders; (3) the implementation of environmental
protection and other programs; and (4) the improved
delivery of community assistance by public service
organizations. In regard to the overall value of
collaboration, most interviewees indicated that the
issues facing the affected communities either would
not have been addressed, or would not have been
addressed to the same extent, if at all, without the use
of a collaborative approach. Interviewees also saw
federal involvement in these efforts as critical. In
addition to the many positive points voiced,
interviewees also noted that the partnerships are facing
some challenges, including difficulties associated with
partnership maintenance and operational support, and
the implementation of partnership-specific initiatives.
Despite these and other challenges thatwere expressed,
most interviewees voiced very favorable impressions
of the partnerships with which they were associated.
Overall, evidence from the evaluation report suggests
that use of a multi-stakeholder approach, as
demonstrated within these projects, can be a very
effective means of addressing environmental justice
issues in communities.
Evaluation Outcomes: What
happened as a result?
The evaluation produced ten core findings and nine core
recommendations, including specific findings related to:
(1) project process, activities, and outcomes; (2) key
factors influencing project success and progress; (3)
organizational styles, policies, and procedures influencing
project success and progress; (4) the value of collaboration
to address environmental justice issues; and (5) the value
of federal agency involvement in these efforts. The IWG
has indicated that the evaluation report has clarified what
is and is not working well with the projects, and that the
report's findings will enable the IWG to develop stronger
projects in the future, and to further articulate a generic
collaborative model that other distressed communities can
use as a guide for their own collaborative efforts.
United States
Environmental Protection
Agency
Office of Policy,
Economics and Innovation
(1807T)
June 2003
EPA-100-F-03-013
------- |