SEPA     Improving EPA's Performance
                   with Program Evaluation
                   An Evaluation of Partnerships to Address
                   Environmental Justice Issues in Communities
                                                                           Series No. 9A
By continuously evaluating its programs, EPA is able to capitalize on lessons learned and incorporate that
experience into other programs. This enables the Agency to streamline and modernize its operations while promoting
continuous improvement and supporting innovation. This series of short sheets on program evaluation is intended
to share both the results and benefits of evaluations conducted across the Agency, and share lessons learned
about evaluation methodologies in this evolving discipline.  For more information contact EPA's Evaluation Support
Division at www.epa.gov/evaluate.
           At a Glance
Evaluation Purpose
To identify the value of using collaboration to
address environmental justice issues in distressed
communities.
Evaluation Type
Process/Outcome Evaluation
Publication Date
January 2003
Partners
Federal Interagency Working Group on
Environmental Justice, Office of Environmental
Justice, Office of Policy, Economics and
Innovation
Contact
Charles Lee, OEJ (202) 564-2597
Background: Why was an evaluation
performed?

An evaluation was conducted to assess the value of using
collaborative projects to address environmental justice issues in
predominantly low-income or minority communities.  The
evaluation is built upon six case studies that were written primarily
between December 2001 and July 2002. These projects are part
of the  Federal  Interagency  Working Group  (IWG) on
Environmental Justice's national demonstration projects announced
in June 2000. These projects are representative of the IWG's effort
to build "dynamic and proactive partnerships among Federal
agencies to benefit environmentally and economically distressed
communities."  In 2001, EPA's Office of Policy, Economics and
Innovation (OPEI), with the support of the IWG and EPA's Office
of Environmental Justice (OEJ), agreed to conduct case studies
and a program evaluation of six demonstration projects. The case
studies describe collaborative projects in (1) Annette Island,
Alaska; (2) East St. Louis/St. Glair County, Illinois; (3) New Madrid
County, Missouri; (4) San Diego, California; (5) Spartanburg, South
Carolina; and (6) Washington, DC.
                                                       NCEI

                                                       NATIONAL CENTER FOR
                                                       ENVIRONMENTAL INNOVATION

-------
Basic Evaluation Approach:  How
did they do it?
The case studies and evaluation report were developed
using roughly fourteen steps, which are outlined below.
Step I:     Develop guiding principles for the evaluation.
Step II:    Develop key evaluative questions.
Step III:    Develop an  evaluation strategy.
Step IV:    Gather input on the  evaluation strategy
            from a range of participants  in a
            facilitated national conference call.
Step V:    Prepare a basic interview guide.
Step VI:    Hold conference calls with project leaders
            to discuss the evaluation strategy and gain
            acceptance for the evaluative effort.
Step VII:   Review pertinent project background material.
Step VIM:   Develop a  list of potential project
            interviewees.
Step IX:    Conduct  interviews   with  project
            participants.
Step X:    Analyze interviewee  data and develop
            draft case studies.
Step XI:    Distribute  draft  case  studies  to
            interviewees for their review.
Step XII:   Analyze  case  studies to develop the
            evaluation report.
Step XIII:   Distribute the evaluation report and case
            studies   to    interviewees    and
            representatives  of  the   academic
            community for their review.
Step XIV:  Complete the evaluation report and case
            studies.

Evaluation Results:  What was
learned?
Evaluation findings indicate that the collaborative
projects are producing a variety of important results,
including: (1)  the improved opportunity for  local
             residents and community organizations to have  a
             genuine say in efforts to revitalize their communities;
             (2)  the enhancement  of relationships  among
             stakeholders; (3) the implementation of environmental
             protection  and other programs; and (4)  the improved
             delivery of community assistance by public service
             organizations.  In regard to  the overall value of
             collaboration, most  interviewees indicated that the
             issues facing the affected  communities either would
             not have been addressed, or would not have been
             addressed to the same extent, if at all, without the use
             of a collaborative approach.  Interviewees also saw
             federal involvement in these efforts as critical. In
             addition to the many  positive  points voiced,
             interviewees also noted that the partnerships are facing
             some challenges, including difficulties associated with
             partnership maintenance and operational support, and
             the implementation of partnership-specific initiatives.
             Despite these and other challenges thatwere expressed,
             most interviewees voiced  very favorable impressions
             of the partnerships with which they were associated.
             Overall, evidence from the evaluation report suggests
             that use  of a multi-stakeholder approach, as
             demonstrated within these projects, can  be  a very
             effective means of addressing environmental justice
             issues in communities.

             Evaluation Outcomes:  What
             happened  as a result?

             The evaluation produced ten core findings and nine core
             recommendations, including specific findings related to:
             (1) project process, activities, and outcomes; (2) key
             factors influencing project success and progress; (3)
             organizational styles, policies, and procedures influencing
             project success and progress; (4) the value of collaboration
             to address environmental justice issues; and (5) the value
             of federal agency involvement in these efforts. The IWG
             has indicated that the evaluation report has clarified what
             is and is not working well with the projects, and that the
             report's findings will enable the IWG to develop stronger
             projects in the future, and to further articulate a generic
             collaborative model that other distressed communities can
             use as a guide for their own collaborative efforts.
United States
Environmental Protection
Agency
Office of Policy,
Economics and Innovation
(1807T)
      June 2003
EPA-100-F-03-013

-------