United States
                            Environmental Protection
                            Office of Policy,
                            Economics and Innovation
  November 2009
                                 Evaluation  of the  Effectiveness of
                                 an  Ethanol  Compliance  Manual
Fact Sheet

For more information on
completed evaluations at EPA
or the Evaluation Support
Division, visit the above link.

In response to a rapid increase in newly permitted ethanol facilities in U.S. mid-western
states, EPA Region 7 staff developed a compliance assistance manual for ethanol facilities
entitled Environmental Laws Applicable to Construction and Operation of Ethanol Plants.
After the manual was published in November 2007, Region 7 conducted significant outreach
on the manual with ethanol facilities, ethanol contractors, and other interested stakeholders.
In 2009, EPA undertook an  evaluation to assess the success of the manual in improving
industry compliance with relevant rules and regulations and to gather information on the
manual's readability, quality of information, and overall usefulness as a  compliance
assistance tool to ethanol facilities in the Region. In addition, Region 7 wanted to identify
ways to improve facility satisfaction  with the manual as a compliance assistance tool and to
determine what other compliance assistance tools and materials might be helpful to ethanol
facilities. The following  information describes the questions that structured the evaluation,
the overall evaluation approach, and the evaluation's key findings and recommendations.

Overarching Evaluation Questions

The following topics and questions formed  the structure of the evaluation:
1. Topic: Outreach and  Awareness. Question: How effective were  Region 7's efforts to
   advertise the availability of and  distribute the manual to ethanol facilities and other
   intended audiences?
2. Topic: Effectiveness of the Manual as an Information Tool. Question: How
   effective was the  manual as a tool for transferring compliance (and related facility
   technical support) information to ethanol facilities?
3. Topic: Changes in  Understanding and Behavior. Question:  Has the existence and
   use of the manual led to increased understanding and behavior change to support
   improved compliance?
4. Topic: Improving  the Manual. Question: How could the manual  be improved to
   provide effective compliance assistance to ethanol facilities?
5. Topic: Other Compliance Assistance. Question:  What approaches other than the
   manual would provide effective  compliance assistance to the ethanol facilities?

Evaluation Approach

To answer the overarching evaluation questions, evaluators (1) undertook interviews with
ethanol facility environmental managers, ethanol industry contractors, state compliance staff,
and EPA staff; (2) developed a customer satisfaction survey and distributed the survey to
ethanol facilities in Region 7 and ethanol industry contractors known  to work in the Region;
and (3) conducted research on ethanol  plant compliance to try to answer the overarching
evaluation question on the effect the manual had on compliance.

Evaluation Findings and Recommendations

Overall, those that participated in the evaluation found the manual to be very informative. Participants
appreciated the manual's existence and the effort that EPA Region 7 undertook to develop it.  Below are
the major findings of the evaluation, organized by the overarching evaluation topic areas.
Outreach and Awareness: Awareness of the manual amongst ethanol facilities and contractors was
not as prevalent as it could be, despite extensive outreach efforts  by Region 7 staff.  Further, the timing
of the manual's publication occurred after most new plants had commenced or completed construction.
Nonetheless, most of these facilities were receptive to reviewing the manual as a part of this evaluation,
and feedback indicated that non-Region 7 ethanol plants might also find the information useful.
Effectiveness of the Manual as an Information Tool: Evaluation participants said the manual was
well-organized, readable, and an appropriate tool for conveying compliance information. Having this
material, including appropriate contacts, organized in one place provided for an easy reference.
Changes in Understanding and Behavior: All evaluation participants said the manual was helpful for
increasing or solidifying their understanding of environmental compliance requirements.  However, this
evaluation cannot conclusively say that the manual  has led to any significant changes that would affect
facility compliance. With only a few exceptions, evaluation respondents were not able to identify specific
instances where use of the manual resulted in increased awareness or improved compliance.
Improving the Manual: Respondents to this evaluation had a number of suggestions on how future
versions of the manual could be improved, with suggestions  centering around additional content, such as
a section on cellulosic ethanol, or around functional content, such  as checklists and case studies.
Other Compliance Assistance: Evaluation participants indicated that venues such as conferences, EPA
workshops, interactive Web sites, and direct EPA interaction  were  also valuable approaches for providing
compliance assistance to ethanol facilities.

Based on the evaluation findings, the evaluators provided a number of recommendations to Region 7 for
consideration when making changes to the manual and conducting other sector outreach.
To improve outreach to facilities and contractors, the evaluation recommended that Region 7 consider
additional outreach strategies, such as developing a biofuels-centric email list, and considering ethanol
plant contractors as both an audience for the manual and as a communication conduit to the plants. In
addition, as much of the content is relevant to all U.S. ethanol plants, Region 7 could consider ways to
tailor content and distribute the manual nationally.
Transferring the manual to an interactive, Web-based format would improve access and the ability to
keep it up-to-date for its audiences. Future versions of the manual could also include new relevant
regulatory information; an executive summary; more "at a glance" information, such as lists of resources
and checklists; more specific examples; and related regulatory information  about topics of interest to
ethanol  plants, such as cellulosic ethanol production.
Finally, the evaluators worked with Region 7 staff to identify options for applying lessons from this
evaluation to the Region's other compliance assistance and beyond compliance efforts.  A first step for
carrying these  ideas forward would be to identify a small group in  the Region to explore strategic and
practical options for enhancing the Region's other compliance assistance activities.

                                            Page 2