SEPA Improving EPA's Performance with Program Evaluation An Evaluation of Partnerships to Address Environmental Justice Issues in Communities Series No. 9A By continuously evaluating its programs, EPA is able to capitalize on lessons learned and incorporate that experience into other programs. This enables the Agency to streamline and modernize its operations while promoting continuous improvement and supporting innovation. This series of short sheets on program evaluation is intended to share both the results and benefits of evaluations conducted across the Agency, and share lessons learned about evaluation methodologies in this evolving discipline. For more information contact EPA's Evaluation Support Division at www.epa.gov/evaluate. At a Glance Evaluation Purpose To identify the value of using collaboration to address environmental justice issues in distressed communities. Evaluation Type Process/Outcome Evaluation Publication Date January 2003 Partners Federal Interagency Working Group on Environmental Justice, Office of Environmental Justice, Office of Policy, Economics and Innovation Contact Charles Lee, OEJ (202) 564-2597 Background: Why was an evaluation performed? An evaluation was conducted to assess the value of using collaborative projects to address environmental justice issues in predominantly low-income or minority communities. The evaluation is built upon six case studies that were written primarily between December 2001 and July 2002. These projects are part of the Federal Interagency Working Group (IWG) on Environmental Justice's national demonstration projects announced in June 2000. These projects are representative of the IWG's effort to build "dynamic and proactive partnerships among Federal agencies to benefit environmentally and economically distressed communities." In 2001, EPA's Office of Policy, Economics and Innovation (OPEI), with the support of the IWG and EPA's Office of Environmental Justice (OEJ), agreed to conduct case studies and a program evaluation of six demonstration projects. The case studies describe collaborative projects in (1) Annette Island, Alaska; (2) East St. Louis/St. Glair County, Illinois; (3) New Madrid County, Missouri; (4) San Diego, California; (5) Spartanburg, South Carolina; and (6) Washington, DC. NCEI NATIONAL CENTER FOR ENVIRONMENTAL INNOVATION ------- Basic Evaluation Approach: How did they do it? The case studies and evaluation report were developed using roughly fourteen steps, which are outlined below. Step I: Develop guiding principles for the evaluation. Step II: Develop key evaluative questions. Step III: Develop an evaluation strategy. Step IV: Gather input on the evaluation strategy from a range of participants in a facilitated national conference call. Step V: Prepare a basic interview guide. Step VI: Hold conference calls with project leaders to discuss the evaluation strategy and gain acceptance for the evaluative effort. Step VII: Review pertinent project background material. Step VIM: Develop a list of potential project interviewees. Step IX: Conduct interviews with project participants. Step X: Analyze interviewee data and develop draft case studies. Step XI: Distribute draft case studies to interviewees for their review. Step XII: Analyze case studies to develop the evaluation report. Step XIII: Distribute the evaluation report and case studies to interviewees and representatives of the academic community for their review. Step XIV: Complete the evaluation report and case studies. Evaluation Results: What was learned? Evaluation findings indicate that the collaborative projects are producing a variety of important results, including: (1) the improved opportunity for local residents and community organizations to have a genuine say in efforts to revitalize their communities; (2) the enhancement of relationships among stakeholders; (3) the implementation of environmental protection and other programs; and (4) the improved delivery of community assistance by public service organizations. In regard to the overall value of collaboration, most interviewees indicated that the issues facing the affected communities either would not have been addressed, or would not have been addressed to the same extent, if at all, without the use of a collaborative approach. Interviewees also saw federal involvement in these efforts as critical. In addition to the many positive points voiced, interviewees also noted that the partnerships are facing some challenges, including difficulties associated with partnership maintenance and operational support, and the implementation of partnership-specific initiatives. Despite these and other challenges thatwere expressed, most interviewees voiced very favorable impressions of the partnerships with which they were associated. Overall, evidence from the evaluation report suggests that use of a multi-stakeholder approach, as demonstrated within these projects, can be a very effective means of addressing environmental justice issues in communities. Evaluation Outcomes: What happened as a result? The evaluation produced ten core findings and nine core recommendations, including specific findings related to: (1) project process, activities, and outcomes; (2) key factors influencing project success and progress; (3) organizational styles, policies, and procedures influencing project success and progress; (4) the value of collaboration to address environmental justice issues; and (5) the value of federal agency involvement in these efforts. The IWG has indicated that the evaluation report has clarified what is and is not working well with the projects, and that the report's findings will enable the IWG to develop stronger projects in the future, and to further articulate a generic collaborative model that other distressed communities can use as a guide for their own collaborative efforts. United States Environmental Protection Agency Office of Policy, Economics and Innovation (1807T) June 2003 EPA-100-F-03-013 ------- |