United States
Environmental Protection
Agency
Office of Policy,
Economics and Innovation
(1807T)
November 2009
EPA-100-F-10-009
Evaluation of the Effectiveness of
an Ethanol Compliance Manual
Fact Sheet
http://www.epa.Qov/evaluate
For more information on
completed evaluations at EPA
or the Evaluation Support
Division, visit the above link.
Introduction
In response to a rapid increase in newly permitted ethanol facilities in U.S. mid-western
states, EPA Region 7 staff developed a compliance assistance manual for ethanol facilities
entitled Environmental Laws Applicable to Construction and Operation of Ethanol Plants.
After the manual was published in November 2007, Region 7 conducted significant outreach
on the manual with ethanol facilities, ethanol contractors, and other interested stakeholders.
In 2009, EPA undertook an evaluation to assess the success of the manual in improving
industry compliance with relevant rules and regulations and to gather information on the
manual's readability, quality of information, and overall usefulness as a compliance
assistance tool to ethanol facilities in the Region. In addition, Region 7 wanted to identify
ways to improve facility satisfaction with the manual as a compliance assistance tool and to
determine what other compliance assistance tools and materials might be helpful to ethanol
facilities. The following information describes the questions that structured the evaluation,
the overall evaluation approach, and the evaluation's key findings and recommendations.
Overarching Evaluation Questions
The following topics and questions formed the structure of the evaluation:
1. Topic: Outreach and Awareness. Question: How effective were Region 7's efforts to
advertise the availability of and distribute the manual to ethanol facilities and other
intended audiences?
2. Topic: Effectiveness of the Manual as an Information Tool. Question: How
effective was the manual as a tool for transferring compliance (and related facility
technical support) information to ethanol facilities?
3. Topic: Changes in Understanding and Behavior. Question: Has the existence and
use of the manual led to increased understanding and behavior change to support
improved compliance?
4. Topic: Improving the Manual. Question: How could the manual be improved to
provide effective compliance assistance to ethanol facilities?
5. Topic: Other Compliance Assistance. Question: What approaches other than the
manual would provide effective compliance assistance to the ethanol facilities?
Evaluation Approach
To answer the overarching evaluation questions, evaluators (1) undertook interviews with
ethanol facility environmental managers, ethanol industry contractors, state compliance staff,
and EPA staff; (2) developed a customer satisfaction survey and distributed the survey to
ethanol facilities in Region 7 and ethanol industry contractors known to work in the Region;
and (3) conducted research on ethanol plant compliance to try to answer the overarching
evaluation question on the effect the manual had on compliance.
-------
Evaluation Findings and Recommendations
Findings
Overall, those that participated in the evaluation found the manual to be very informative. Participants
appreciated the manual's existence and the effort that EPA Region 7 undertook to develop it. Below are
the major findings of the evaluation, organized by the overarching evaluation topic areas.
Outreach and Awareness: Awareness of the manual amongst ethanol facilities and contractors was
not as prevalent as it could be, despite extensive outreach efforts by Region 7 staff. Further, the timing
of the manual's publication occurred after most new plants had commenced or completed construction.
Nonetheless, most of these facilities were receptive to reviewing the manual as a part of this evaluation,
and feedback indicated that non-Region 7 ethanol plants might also find the information useful.
Effectiveness of the Manual as an Information Tool: Evaluation participants said the manual was
well-organized, readable, and an appropriate tool for conveying compliance information. Having this
material, including appropriate contacts, organized in one place provided for an easy reference.
Changes in Understanding and Behavior: All evaluation participants said the manual was helpful for
increasing or solidifying their understanding of environmental compliance requirements. However, this
evaluation cannot conclusively say that the manual has led to any significant changes that would affect
facility compliance. With only a few exceptions, evaluation respondents were not able to identify specific
instances where use of the manual resulted in increased awareness or improved compliance.
Improving the Manual: Respondents to this evaluation had a number of suggestions on how future
versions of the manual could be improved, with suggestions centering around additional content, such as
a section on cellulosic ethanol, or around functional content, such as checklists and case studies.
Other Compliance Assistance: Evaluation participants indicated that venues such as conferences, EPA
workshops, interactive Web sites, and direct EPA interaction were also valuable approaches for providing
compliance assistance to ethanol facilities.
Recommendations
Based on the evaluation findings, the evaluators provided a number of recommendations to Region 7 for
consideration when making changes to the manual and conducting other sector outreach.
To improve outreach to facilities and contractors, the evaluation recommended that Region 7 consider
additional outreach strategies, such as developing a biofuels-centric email list, and considering ethanol
plant contractors as both an audience for the manual and as a communication conduit to the plants. In
addition, as much of the content is relevant to all U.S. ethanol plants, Region 7 could consider ways to
tailor content and distribute the manual nationally.
Transferring the manual to an interactive, Web-based format would improve access and the ability to
keep it up-to-date for its audiences. Future versions of the manual could also include new relevant
regulatory information; an executive summary; more "at a glance" information, such as lists of resources
and checklists; more specific examples; and related regulatory information about topics of interest to
ethanol plants, such as cellulosic ethanol production.
Finally, the evaluators worked with Region 7 staff to identify options for applying lessons from this
evaluation to the Region's other compliance assistance and beyond compliance efforts. A first step for
carrying these ideas forward would be to identify a small group in the Region to explore strategic and
practical options for enhancing the Region's other compliance assistance activities.
Page 2
------- |