Improving EPA's Performance
with Program Evaluation
Partnerships to Address Environmental Justice
Issues in Communities: Case Studies
Series No. 9B
By continuously evaluating its programs, EPA is able to capitalize on lessons learned and incorporate that
experience into other programs. This enables the Agency to streamline and modernize its operations while promoting
continuous improvement and supporting innovation. This series of short sheets on program evaluation is intended
to share both the results and benefits of evaluations conducted across the Agency, and share lessons learned
about evaluation methodologies in this evolving discipline. For more information contact EPA's Evaluation Support
Division at www.epa.gov/evaluate.
At a Glance
Evaluation Purpose
To better understand six national environmental
justice demonstration projects and their effect on
assisting distressed communities.
Evaluation Type
Case Studies
Publication Date
January 2003
Partners
Federal Interagency Working Group on
Environmental Justice, Office of Environmental
Justice, Office of Policy, Economics and
Innovation
Contact
Charles Lee, OEJ (202) 564-2597
Background: Why was an evaluation
performed?
Case studies were developed to help the Federal Interagency
Working Group (IWG) on Environmental Justice better understand
six of its national environmental justice demonstration projects.
These projects are representative of the IWG's effort to build
"dynamic and proactive partnerships among Federal agencies to
benefit environmentally and economically distressed communities."
In 2000, EPA's Office of Policy, Economics and Innovation
(OPEI), with the support of the IWG and EPA's Office of
Environmental Justice (OEJ), agreed to conduct the case studies
and a program evaluation of the six projects. The case studies,
written primarily between December 2001 and July 2002, highlight
the following projects:
• a partnership based primarily in an inner city community near
downtown San Diego that is addressing health concerns brought
about by incompatible land uses;
• a partnership focused on Southeast and Southwest
Washington, DC., and championed by the Washington Navy
Yard, that is seeking to ensure that local redevelopment
efforts benefit local residents;
NCEI
NATIONAL CENTER FOR
ENVIRONMENTAL INNOVATION
-------
• a collaboration between a Tribal community in
Alaska and several agencies that is working to ensure
cleanup of more than 80 contaminated sites on the
community's home island;
• a partnership between agencies and several
organizations based in East St. Louis and
surrounding communities that is taking a
comprehensive approach to reducing local threats
from lead poisoning;
• a partnership among three rural communities,
agencies, and other organizations in southern
Missouri that is taking a structured approach to
addressing local asthma, lead, and water issues; and
• a partnership consisting of numerous groups and
agencies and driven by a grassroots group in
Spartanburg, South Carolina, that is seeking to clean
up contaminated and abandoned sites and revitalize
the nearby neighborhoods.
Basic Evaluation Approach: How
did they do it?
The case studies and evaluation report were developed
using roughly fourteen steps, which are outlined below.
Step I: Develop guiding principles for the
evaluation.
Step II: Develop key evaluative questions.
Step III: Develop an evaluation strategy.
Step IV: Gather input on the evaluation strategy
from a range of participants in a
facilitated national conference call.
Step V: Prepare a basic interview guide.
Step VI: Hold conference calls with project leaders
to discuss the evaluation strategy and gain
acceptance for the evaluative effort.
Step VII: Review pertinent project background
material.
Step VIM: Develop a list of potential project
interviewees.
Step IX: Conduct interviews with project
participants.
Step X: Analyze interviewee data and develop
draft case studies.
Step XI: Distribute draft case studies to
interviewees for their review.
Step XII: Analyze case studies to develop the
evaluation report.
Step XIII: Distribute the evaluation report and case
studies to interviewees and
representatives of the academic
community for their review.
Step XIV: Complete the evaluation report and case
studies.
Case Study Results: What was
learned?
Generally, partnership mechanisms for involvement
have been effective at allowing partners to be
adequately involved in partnership decision-making
processes. Most of the interviewees addressing the
topic were satisfied with the outcomes of their
partnership activities so far. Interviewees cited the
formation and operation of their partnerships as their
most significant success, and the maintenance and
operation of their partnerships as their most significant
challenge. Interviewees also indicated that their
partnerships aided in addressing environmental justice
issues in the affected communities by improving the
sharing of information, resources, and/or expertise
between organizations; creating efficiencies; securing
and leveraging additional resources; and helping
organizations better understand the needs of the
affected communities. Of those addressing the
question, nearly 80 percent of interviewees (52 of 66)
indicated that the issues facing the affected
communities either would not have been addressed,
or would not have been addressed to the same extent,
if at all, without use of a partnership approach.
Regarding the value of federal involvement,
interviewees indicated that federal agencies provide
or enhance the credibility, legitimacy, and trust
surrounding their partnership efforts; and provide
resources and/or expertise to the projects.
-------
Evaluation Outcome: What
happened as a result?
Six case studies were produced that provided
information on each partnership project's community
history, background, goals and processes, and activities.
In addition, each case study described interviewees'
perceptions of their partnership projects related to
measuring success, successes and challenges,
recommendations for improvement, lessons for other
communities, and the value of federal involvement in
these efforts. Five to eight specific findings for each
case study were also developed.
• Plan visits to the field to observe the subject of the
case study (e.g., partnership project) in action.
Interviewing stakeholders in-person can be helpful;
however, it may be more important to prioritize
observation of program activities than to conduct
interviews in the field. If face-to-face interviews
cannot be conducted while in the field, make plans
to conduct them over the phone at a later date.
• Prioritize the type of documents used in the
development of a case study. For these case studies,
minutes of the partnership meetings proved to be
some of the most helpful documents.
• Consider having individuals, such as academics, who
are not affiliated with the projects in question review
the case studies.
United States
Environmental Protection
Agency
Office of Policy,
Economics and Innovation
(1807T)
June 2003
EPA-100-F-03-014
------- |