Ron M. Beech
Amy F. Dake
Designing an Effective
Communication Program
A Blueprint for Success
University of Michigan
School of Natural Resources and Environment
Prepared for
U.S. Environmental Protection Agency, Region 5
-------
Acknowledgments
We would like to thank the following people:
Nancy Phillips (EPA) - for trusting us and guiding us when we needed it;
Raymond DeYoung (School of Natural Resources and Environment) - for coffee and
getting us back on track;
Bob Steelquist (Puget Sound Water Quality Authority) - for letting Amy rifle
through his project files;
the folks at the Alliance for the Chesapeake Bay, for sending us a ton of great
information; and most of all,
John Weise (independent graphic designer, Ann Arbor, Michigan) - for transforming
dull-looking words on a page into the much more friendly layout you will see in this
handbook. Without his help, we doubt you would have ever picked up this hand-
book!
We would appreciate your comments on this handbook. Have you used it? Did
you find it helpful? What didn't work and what did? What is missing? Please direct
your comments to...
Ms. Nancy Phillips
U.S. Environmental Protection Agency
Region V
77 West Jackson Blvd.
Chicago, IL 60604-3590
Disclaimer
This report was developed through the U.S. EPA National Network for Environmental
Management Studies (NNEMS) program. The program provides funding and opportunities
for graduate students to investigate topics of particular interest to U.S. EPA. This project was
administered from U.S. EPA Region 5 in Chicago, Dlinois, and monitored by Ms. Nancy
Phillips. The report has been reviewed and approved for copying and dissemination. The
contents and views expressed in this document are those of the authors and do not necessarily
reflect the policies or positions of the U.S. Environmental Protection Agency or other organiza-
tions named in this report, nor does the mention of trade names for products or their software
constitute their endorsement.
-------
Designing an Effective Communication Program
A Blueprint for Success
Ron M. Beech
University of Michigan
Amy F. Dake School of Natural Resources and Environment
Prepared for
U.S. Environmental Protection Agency
Region 5
September 1992
-------
Designing an Effective Communication Program
A Blueprint for Success
Chapter 1 A Blueprint for Success
Introduction 7
A Review of Your Surroundings 8
How to Research the Questions About Your Surroundings 10
Goals and Objectives 11
Target Audiences 13
Learning More About Your Target Audience 17
Develop Specific Strategies and Plan Program Implementation 20
Tasks 21
Media Formats and Channels 23
Planning Your Evaluation 30
Recording the Plan 31
Preparing Your Materials 32
Planning Worksheet 33
A Checklist for Planning an Effective Communication Program 34
Chapter 2 Tips for Material Design
Introduction 39
Useful, Relevant Information 40
Concrete Language 42
Vivid Language 44
Format 46
Page Layout and Type 48
Use of Graphics 50
Titles and Headings 52
White Space 54
A Checklist for Effective Communication Materials 56
2
-------
Chapter 3 Understanding Your Evaluation
Introduction 59
Understanding the Relationship of Evaluation to the Program 60
Step 1 Decide on the Purpose of the Evaluation 64
Step 2 Identify Internal and External Constraints 66
Step 3 Decide at What Stages to Do Evaluation 68
Step 4 Select a Type of Evaluation 72
Step 5 Decide What Type of Evidence You Would Like to Collect 76
Step 6 Select Evaluation Techniques 78
Step 7 Decide How Often You Will Use the Technique 80
A Checklist for Developing Your Evaluation Program 82
Chapter 4 Evaluation Techniques
Introduction 85
Proofreading Software 86
SMOG Readability Test 90
Expert Reviews 92
Signal Stopping 94
Monitoring Audience Interactions 96
Community/ Regional/ State Interactions 100
Organizational Activities 102
Program Activities 104
Polling 108
Focus Groups 112
Observations 116
Self-administered Questionnaires 120
Appendices
Appendix A Chesapeake Bay Information and Education Materials 123
Appendix B Puget Sound Public Information and Education Fund Programs 129
Appendix C How to Test for Readability 132
-------
The Blueprint...
o
Evaluate
T
Reviewing Your Surroundings
Setting Goals and Objectives
Identifying Target Audiences
Developing Strategies
Organizing Tasks
Selecting Media Formats and Channels
f
Preparing Your Materials
Understanding Your Evaluation and
Steps for Performing Effective Evaluation
Evaluation Techniques
This diagram summarizes the contents of the
book. Starting at the left-hand side of the page, you
notice the chapters one through four. The horizontal
rectangles list the most important subjects included
in each chapter.
Chapters one and two provide information for
developing an effective program. However, unless
you evaluate the steps of the process, you will not
know if your efforts are effective. The grey shaded
areas represent this idea. Chapter three provides the
fundamental knowledge needed to understand
evaluation and chapter four explains the techniques
used to accomplish evaluation.
On the rig^t-hand side of the page, you see arrows
that represent the interactive nature of evaluation.
Once die techniques are used, they can provide
feedback that will improve the program. We hope
that the information in this book will be your blue-
print for success.
4
-------
Using This Handbook |
Using This Handbook
Introduction
So you're going to design an environmental
communication program. Congratulations! It should
be a lot of fun, and hopefully, won't give you too
many headaches. This handbook is designed to help
you get through the tough parts and avoid many
common mistakes.
Specifically, this handbook will guide you through
each crucial step involved in designing an environ-
mental communication program. It will help you
design your entire program, from setting your goals
and objectives to evaluating your successes. It will
also provide you with many tips that have been
shown to increase the effectiveness of communication
materials and programs.
Chapter 1 A Blueprint for Success
Chapter 1 describes each critical step in the
planning process for a communication program. It
takes you through analyzing your own surround-
ings, setting your goals and objectives, identifying
your target audience, and choosing your media
formats and communication channels. Each step in
the planning process is crucial to the success of your
program; this chapter guides you painlessly through
the process.
Chapter 2 Tips for Material Design
Chapter 2 gives you tips on designing effective
communication materials, by doing such things as
creating white space on a page and using graphics
effectively. While some of these tips may seem
obvious to you, they are often quickly forgotten once
your program begins. However, these tips are vital—
if you don't use them, people may not even read your
materials or listen to your presentation! The tips
included in this section have been shown to increase
die effectiveness of communication materials by
increasing attention and memory, or persuading
people to change their behavior.
Chapter 3 Conducting an Evaluation
Chapter 3 introduces you to the most important
concepts of evaluation. It explains how you can relate
these ideas to your evaluation design. Evaluation is a
crucial step in your program—if you don't do some
evaluation, how will you ever know whether you are
accomplishing anything? While evaluation is a very
important step, it is often skipped because people
think that it will be too complicated, expensive, or
time-consuming. This does not have to be true, as is
shown in Chapter 4.
Chapter 4 Evaluation Techniques
Chapter 4 provides a description of different
evaluation techniques, most of which are fairly easy
and inexpensive to complete. For each technique,
advantages and disadvantages, procedures, and
estimates of staff time and resources are provided. By
using some of these techniques, you should be able to
conduct a good evaluation of your program and your
materials. Doing this will enable you to know what is
working and what is not, and then to change your
program accordingly.
Throughout this handbook, we give you refer-
ences that you can go to to get more information. We
also provide checklists and worksheets to help you
design and implement each critical step in your
communication process.
Appendices
Appendices A and B provide you with a list of
sample projects that have been done under EPA's
Chesapeake Bay Program and Puget Sound Water
Quality Authority's Public Information and Educa-
tion Fund. Appendix C describes in detail the steps to
do one of the evaluation techniques.
We hope that using this handbook will help you
conduct a successful environmental communication
program, thereby increasing awareness, bringing
about changes in human behavior, and ultimately,
improving environmental conditions in your area.
Good luck!
5
-------
Chapter 1
Introduction 7
A Review of Your Surroundings 8
Social Surroundings 8
Political Surroundings 9
Economic Surroundings 9
Organizational Surroundings 9
How to Research the Questions About Your Surroundings 10
Sources of Secondary Research 10
Goals and Objectives 11
Goals 11
Objectives 12
Target Audiences 13
Exchange Concept 13
Breaking Down Your Audience 13
Examples of Target Audience Descriptions from Water Quality Programs 15
Learning More About Your Target Audience 17
Methods Used to Find Out More About a Target Audience 17
Where to Find More Information 19
Develop Specific Strategies and Plan Program Implementation 20
Tasks 21
Media Formats and Channels 23
Examples of Media Formats 23
Message 24
Your Audience 24
Examples of Channels 24
Budget, Time, and Other Resources 25
Choosing Channels 26
Characteristics of Some Media 27
Things to Consider When You are Choosing Media Formats and Channels 28
Where to Find More Information 29
Planning Your Evaluation 30
Recording the Plan 31
Preparing Your Materials 32
Planning Worksheet 33
A Checklist for Planning an Effective Communication Program 34
Where to Find More information 36
6
-------
Introduction
A Blueprint for Success
Introduction
Would you want a construction contractor to build
your home without blueprints? Assuming you
answered "no," then why would you spend money
on communication programs without drafting a
plan? Although communication plans are compli-
cated and subject to many changes, it is still necessary
to develop a plan.
This chapter will explain the important steps in
planning a communication program. The following
lists includes similarities between communication
plans and blueprints.
Blueprints reveal how electrical, plumbing and
mechanical systems are interconnected with the
structure. Similarly, communication plans show the
relationships between the different program objec-
tives and tasks and how organizational resources will
be used to meet those needs.
Without knowing the details of a design, a house
cannot be built. In the same way, program plans lay
out the details of how, when and where a communi-
cation program will achieve its goals.
Good blueprints allow the contractor to estimate
the quantity of supplies, number of construction
workers and time needed to complete the building.
Without a good communication plan you will lack
the essential information needed to plan financial
budgets and personnel assignments. Embarrassing
mistakes can result that may tarnish the reputation of
your organization.
Most importantly, blueprints communicate the
plans to all the parties that are involved and allows
them to coordinate their work. For example, a
plumber may need to complete a task before the
electricians can do their job. Effective communication
programs are also dependent upon plans because
they allow staff members and volunteers to coordi-
nate their work.
Many other analogies could be developed, but it is
most important to realize that communication
programs deserve die same rigorous planning
process as other worthwhile endeavors. Every step
of a plan should be spelled out, from the first goal to
the final task. This is not something that one person
does in his or her head; it must be written down and
formulated with the help of all people involved. This
chapter will help you plan a successful communica-
tion program. It will explain step by step how to
develop your program. If you follow it carefully, you
will be prepared to achieve your goals.
7
-------
A Review of Your Surroundings
Let's assume that your organization has identified
an environmental problem that is affecting the
community. You know the causes of the problem and
the preventive measures needed to improve the
situation. To solve this problem, you want to develop
an outreach effort that will affect the community in a
positive way and provide a solution. However,
having a great idea or a solution is rarely enough. It is
critical to research and carefully construct a detailed
plan of how you will make your ideas a reality. This
cannot be successfully accomplished by just one
person. It requires exchanging information among
everyone within the organization and being aware of
the external surroundings in which your organiza-
tion operates.
Before an organization can begin any program
that will involve interaction with people or other
organizations, it is imperative that the program
managers and staff ask themselves some very
important questions about their surroundings. These
questions fall into four categories: social, political,
economic and organizational. Three categories
involve examining the external environment and the
other requires looking inside your organization. Each
question should be examined from two time frames.
1 The current conditions in which the
organization operates,
2 The expected future conditions in which
the organization will operate once the
program begins.
Write down your responses to the questions that
follow and share them with others in your organiza-
tion. It is doubtful that good answers can be obtained
without some research, so the next section offers
several resources that should be consulted before
answering the questions. After providing realistic
answers to these questions and others that may be
appropriate, the organization must formulate realistic
goals, objectives, strategies and tasks to carry out the
program.
Social Surroundings
Following is a list of questions an organization
should examine regarding its social surroundings.
Remember to answer these questions from both the
current and future points-of-view.
> Is the community agenda focused on your
issue or are there many other concerns on
the horizon?
> Is there a positive or negative demand for
your ideas?
»- Are dtizens aware of the problems and the
solutions?
>¦ What are the existing activities that address
, the problem?
>- How does the problem affect the commu-
nity?
*• Has the media focused on the issue?
> Why has the situation not improved? Is
there a lack of knowledge; or do attitudes
and behaviors need to change?
> Which agencies or other organizations are
currently involved with the issues?
8 Chapter 1 A Blueprint for Success pages 6-36
-------
A Review of Your Surroundings pages 8-9 1
Political Surroundings
An organization must also ask itself questions
about its political surroundings.
>- What are die existing federal, state and
local laws and rules that are influencing the
situation?
>• Have you Identified and properly defined
all the important political forces?
>• Who will support your ideas in the political
structure?
>- What will be die reaction of those who do
not support your program?
> Do you understand what motivates both
allies and critics?
»• How will your goals and objectives be
influenced by both allies and critics?
>- What relationship does the board of
directors or management team have with
those in the political structure?
Economic Surroundings
In addition, you must examine your organization's
economic surroundings.
»- Are factors such as high unemployment or
inflation affecting the quality of life in your
community?
>¦ What economic impact will your program
have on businesses and other profit-
making enterprises?
> Will the target audience be asked to give up
time or money if it follows your advice?
> Does your program provide the maximum
benefits for the minimum costs?
>¦ Have all the social benefits and costs that
do not have market values been included in
your analysis?
>¦ What additional benefits can be offered to
satisfy community concerns?
> Have you considered the economic impact
of your program over different time
frames?
Organizational Surroundings
Finally, you must analyze your organizational
surroundings.
»- Have your organization's resources been
inventoried? Is there enough staff and are
they properly trained? Also, is there
enough money and equipment to manage
the project?
> Is the staff trained to interact with the target
audience?
»- Are incentives present for staff to accom-
plish ihegoals, objectives and tasks of a
new program?
»• Is your organizational structure designed to
interact with the intended audience?
>- Does your organization's mission inter-
twine with die proposed program?
»¦ Have previous programs prepared your
organization for new programs?
> How will the standard method of doing
work change in your organization?
> Will departments or staff work together in
new ways?
>- Will the program require staff to work
closely with others outside your organiza-
tion?
> How will your organization's financial and
political supporters respond to a new
program?
> Are management and the board of direc-
tors prepared for this effort?
»• What systems are in place to handle the
new program: accounting, filing, communi-
cation, computers?
>¦ How is your organization's image per-
ceived in the community?
>- Does your organization have the proper
image to be responsible few this program?
9
-------
How to Research the Questions About
Your Surroundings
Reviewing secondary data that has already been
compiled by others is a good place to begin answer-
ing the questions about your surroundings. Usually it
is inexpensive and can quickly provide the clues
needed to begin forming program goals and objec-
tives. However, it is important to remember that
secondary data may not provide specific answers to
your questions and the quality will vary. However,
the positive benefits usually outweigfri the negative
aspects of these resources. Following is a list of
possible resources that can be used to answer the
questions about your surroundings.
Where to Find More Information
Stewart, D. 1984. Secondary Research: Information Sources and
Methods. Newbury Park, CA. Sage Publications.
Sources of Secondary Research
>- Permit records from local, county, slate and
federal agencies
>- Voting records
>• Census data from US. Census Bureau
>¦ County and regional planning agencies
>- Professional associations
> Trade associations
> Community service groups
>• Religious institutions
»- Citizen-action nonprofit groups
>- Organization correspondence files
»• Existing organizational data collected from
previous programs
»~ Supporters list
>¦ Newspapers
»• Public opinion polls
> Local, state and university libraries
>- Demographic publications
»¦ Advertising agencies
^Corporations
^Foundations
>* Polling companies
> Nonprofit organizations
»¦ Clearinghouses
and county directories
> Chamber of commerce fists
> Yellow pages
»• Tradejoumab
hapter 1 A Blueprint for Success pages 6-36
-------
Goals and Objectives pages 11 -12
Goals and Objectives
Now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings.
you are ready to identify goals and define
objectives.
The program's goals and objectives will form the
foundation to develop programing efforts. They are
die prerequisites for moving forward with planning a
program. Once goals and objectives have been
established, the next crucial step is to develop
strategies and tasks.
Z3
Goals
Program goals are usually general, broad and all-
encompassing, and should include the very essence
of a program's purpose. However, they also should
include some measure of the expected outcome. This
can relate to what die organization wants to change
or what should remain the same. They should not be
in conflict with other organizational goals and
objectives.
Usually they are developed for different functional
areas of a program. There may be goals related to
fund-raising, management of staff and volunteers,
program advertising, marketing, etc. Most impor-
tantly, goals:
»• Provide tihe staff with something to strive
toward;
>¦ Supply guidance to the program's stiff
when making decisions;
>- Are internally consistent;
»¦ Communicate the purpose of the program
to the public and entities external to your
organization;
>- Provide the basis for developing objectives,
strategies and tasks; and
»- Unite the program staff toward the
achievement of a common purpose.
An Example of a Goal
SB
m Ob|f;(;tive
To protect the Puget Sound and its watershed by
linking and supporting citizens and organizations
who are working locally for protection of wetlands
and water quality.
Staff Involvement with Goals and Objectives
Before these goals can be accomplished, everyone
involved in the program must have input and "buy
into" the ideas. Otherwise goals and objectives will
mean little to the staff and will not be seriously
considered when carrying out the program. Also,
goals may mean different things to different people
in the organization. It is essential to develop the goals
jointly with the staff through a meeting or other
forums that are appropriate in your organization.
11
k
-------
Goals and Objectives pages 11 -12
Objectives
Now that you have:
Used secondary research to answer
questions about your social, political,
economicand organizational surroundings;
and
Identified reasonable goals in consultation
with your staff;
you are ready for the next vital step in program
development - defining objectives.
In contrast to goals, objectives should be specific
about what will be done from a program-operations
perspective. They narrow down the very broad
vision of the goal to something that can be accom-
plished through organizational resources (e.g., staff,
money, ideas.)
Objectives should be:
>- Prioritized;
»- Important to die organization;
>- internally consistent;
>¦ Measurable or quantifiable; and should
have:
»- A strong action verb that describes what
organizational action is necessary to
achieve a goal;
>¦ A specific purpose (several objectives could
be included for one goal)
>- A single outcome
>¦ A date of completion
Each of these components is essential because it
lays the groundwork for developing specific strate-
gies and tasks, die next crucial step. Also, these
components of objectives provide the benchmarks to
evaluate whether the program has achieved the
desired results.
An Example of an Objective
To publish and distribute four 12 -16 page wetland
journals by June 1,1993.
To protect the Puget Sound and its watershed by
linking and supporting citizens and organizations
who are working locally for protection of wetlands
and water quality.
i A Blueprint for Success pages 6-36
-------
Target Audiences pages 13-19
Target Audiences
Ok, now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals in consultation
with your staff; and
Defined specific objectives;
it is necessary to identify, understand and break
doom your target audience into smaller groups
before continuing with strategies and tasks.
Who will you reach and in what ways will you
achieve your goals and objectives? Defining goals
and objectives is necessary, but is not specific enough
to answer the previous question. Next, it is essential
that you identify your target audience and then
develop strategies and tasks to reach this group.
Many programs are unsuccessful because they do not
put any effort into these vital steps in the planning
process.
Target audiences are groups that have common
characteristics. These characteristics could range from
education levels to attitudes, behaviors or needs.
Often, several of these characteristics will overlap. In
a sense, the target audience is the consumer of your
message and program. Your audience must be
treated as special as customers that purchase prod-
ucts or services from for-profit businesses. Your
internal organization design and programing
decisions should be a reflection of this fact All of
your organization's decisions should be designed to
meet the needs of the target audience - not made for
the convenience of your organization.
You may ask why this target audience stuff is so
important? The explanation is simple; if you do not
understand the target audience, it will be impossible
to develop an effective outreach campaign. Therefore,
you will not achieve your goals and objectives. If the
goals and objectives of your project are not achieved,
your organization's reputation may be tarnished and
it may be difficult to obtain funding for future
projects.
Exchange Concept
Often there is an exchange that occurs between
the organization and the target audiences. It
might be as simple as providing information
about proper lawn fertilizing. In exchange, people
may take steps to protect their groundwater or
ensure that their streams are free of extra nutri-
ents that cause algal growth.
By researching the target audience, enough
knowledge can be obtained to understand what is
important to your audience. Also, it helps your
organization define what you have to offer this
group. However, before this exchange can occur,
it is important to know whether there are sub-
groups within the target audience.
Breaking Down Your Audience
To reach your target audience, you must know
specifically who is in it and what they are like.
This involves breaking groups down into sub-
groups that exhibit similar characteristics or traits.
Some possible ways of breaking audiences down
include: demographics, language, lifestyle,
geography, products or services purchased,
attitudes, distribution channels used to purchase
products or services, benefits derived, or commu-
nication channels used.
Breaking down your target audience is critical.
This step allows you to develop messages and
programs in a way that will reach and influence
these subgroups. However, it is important to note
that often these groups do overlap. Different
audiences may live in the same geographic area,
speak the same language or share many other
characteristics.
13
-------
Target Audiences pages 13 -19
The following examples are characteristics that
can be used to break down your audience.
Demographics
level of education, geographic location, occupa-
tion, number of people in household, type of
household (single unit, duplex, apartment), number
of rooms, owner or renter, income, age, martial
status, sex, cultural characteristics, value of owned
units, rents paid for rental units, distance from work,
etc.
Behavior
types of media (e.gv section of newspaper people
read or type of radio station they listen to), member-
ship in organizations, activities that affect the
environment in positive or negative ways, sports,
hobbies, etc.
Attitudes
perspectives on environmental issues, beliefs
about an individual's impact on the environment,
the role of government in environmental protection,
etc.
Here is an example of how a target audience
can be broken into different groups. Let's say
that your outreach campaign has targeted
those who purchase fertilizer used on lawns
and gardens. In reality, this includes several
subgroups.
Apartments
An employee of the management company or
owner may make the decisions concerning fertilizer,
purchases.
Condominiums
Most of these developments have committees that
are made up of the unit owners. Perhaps the land-
scaping committee is involved in the decision-
making.
Lawn care companies
These companies may apply fertilizer to multi-
family housing units, businesses or to individual
homes.
Homeowners
Many homeowners apply their own fertilizer.
Usually they purchase this from a local hardware or
lawn and garden store.
Government property
A grounds manager for a local, county or state
government must make decisions based upon local
ordinances or purchasing policies. Managers of
property owned by the federal government may also
have similiar constraints that affect what products
are used and how they are used.
Gardeners
Flower and vegetable gardeners also purchase
fertilizer from hardware and lawn and garden stores.
Farmers
Farmers may order large quantities of fertilizer
from co-ops or directly from manufacturing
represent! tives.
14 Chapter 1 A Blueprint for Success pages 6-36
-------
Target Audiences pages 13-19
As you can see, what seemed to be a
homogeneous group suddenly became divided.
Other variables could also segment these
audiences, such as:
Source
Managers of multifamily housing units, farmers,
and lawn care companies are likely to purchase
larger quantities from wholesalers or the factory.
Methods/Equipment Used
Each subgroup may use different equipment to
spread the fertilizer. For example, some may use
liquid fertilizer instead of solid forms.
Communication Gunnels
A farmer would be exposed to trade journals and
other sources of information specific to farming,
while a multifamily unit manager would be inter-
ested in real estate management journals.
Geography
Many people who grow flowers may live in
urban areas. In contrast, farmers are more often
located in rural areas.
In summary you want to define the target audi-
ences in a way that separates them from the general
population of your region. A description should
include all the information that is available from the
sources presented in the next section. A target
audience should include only one group. For ex-
amples farmers and fishermen are very different
groups and should be considered different target
audiences. It should be very, very specific and
describe the audience from many different perspec-
tives (e.g., social, economic, demographic characteris-
tics). Simply stating that your program is designed
for the residents of your county is a very poor descrip-
tion of a target audience. However, a good target
audience would be: married men between the ages of
20-45 who change their own oil, own homes, sub-
scribe to automobile magazines and buy their oil
from small auto-parts stores. Unfortunately, this type
of information cannot be obtained without research.
Examples of GREAT target audience descriptions
from water quality programs:
>¦ Minority and learning disabled 6th graders
from Coe Elementary and North Kitsap
Middle Schools.
>- Children between the ages of 6 and 13
enrolled in the Eastside Day Camp.
»- 250 members of a professional graphic art
association that includes commercial graphic
designers, illustrators, photographers.
Examples of OK target audience descriptions from
water quality programs;
>- Citizen activists and community groups in
North Mason County.
>- Sixth grade through high school students in
the Green River Valley.
>- The 900,000 Puget Sound residents who
visit the Woodland Park Zoo each year.
>¦ Construction contractors in King, Pierce,
Snobol and Thurston counties.
> Gardeners and small farmers in King
County.
»¦ Retired citizens, tribal members and 4-H
youth in the Sequim Bay watershed.
> Auto shops in Pierce County.
»- Auto body and repair shops, small boat
yards, photo labs in Snohomish, King,
Pierce and Thurston counties.
Continued...
15
-------
Target Audiences pages 13 -19
Examples of POOR target audience descriptions
from water quality programs:
>• Citizens in Pierce, King, Kitsap, Jefferson,
Island and Mason Counties.
>¦ Anyone eight years of age and older in the
Puget Sound Region.
>- Resource management professionals,
concerned individuals, and state and local
government officials.
> Snohomish County citizens interested in
aquatic resources.
>• Residents of Whatcome County.
>¦ Residents of Chimacum, Quilcene, and
Bririnon watersheds.
>- Farmers, fishermen, native groups, and
individual communities around Boundary
Bay and school children on both sides of
the border.
Chapter 1 A Blueprint for Success pages 6-36
-------
Target Audiences pages 13-19
Learning More About Your Target
Audience
Before you begin developing messages, it is a
prerequisite to find out what your target audience
knows about the issue that is included in your
program. Does the target audience have misconcep-
tions, or misinformation? By gathering this type of
information, you can then work backwards and
develop a program and messages that start from the
end-users perspective.
Often, it is important to gain an understanding
about the intended behavior change and how the
program can affect the behavior. Through target
audience research you can decide what factors
influence the behavior and which aspects could be
changed through your program. Also, it is a good
idea to ask questions about your audience's beliefs,
attitudes and current skill level. For example, mem-
bers of a target audience may be damaging the
environment because they believe that their activities
have no impact. Also, their knowledge about envi-
ronmental issues could be inaccurate. Or they may
have the attitude that protecting the environment is
too time-consuming and expensive. Another possi-
bility is that they know they are having an impact
but do not know how to change their current behav-
ior.
Also, it is important to find out what communica-
tion channels are used by the target audience and the
credibility of those channels. For example, how much
time does the target audience invest in listening to
the radio and television? What shows do they watch
on these sources? Do they read newspapers and
magazines? Which ones do they read and what
sections are most often read? How do they respond
to other sources such as billboards, direct mail,
exhibits at public events, posters and bumper
stickers? Do they rely on organizations for informa-
tion? Perhaps they respect the leaders of these groups
or they rely on the newsletters, attend meetings or
volunteer their time or money.
Methods used to find out more about a target
audience:
Existing External Sources
>- Conduct literature searches about stories
related to topic.
>- Consult special libraries and information
centers.
>- Review books that list sources of marketing
research.
>~ Contact professional associations involved
in marketing, public relations, non-profit
management and communications.
>- Consult computerized databases, which are
available from many on-line data services.
Sometimes these services are available at
libraries.
»• Consult bibliographic databases, which are
available at most college libraries and many
public libraries. These resources can be
used to search thousands of publications.
> Consult trade associations, which often
gather information on specific topics. There
is an association that represents almost any
topic. Libraries usually carry books that list
these organizations.
> Review existing public polls and review
publications like The Galhtp PoU Monthly
and American Demographics.
Continued...
17
-------
Target Audiences pages 13-19
>• Review surveys that monitor consumer
product purchases. These services annually
publish books that cross-reference the
purchase of products with lists of demo-
graphic, social, and economic information,
they include purchases of motor oil,
fertilizer and other products that could
have a negative impact on water quality-
Some examples of items tracked include:
age, education, income, sex, household size,
and occupation. Also, information about .
the use of channels is included, such as
type of radio stations, TV shows, newspa-
pers and magazines read.
> Contact both state and federal agencies
which constantly Collect data on economics
and social characteristics. Often this
information is published regularly in
reports and is available in different geo-
graphic formats.
»• Contact the Census Bureau which has
demographic information in easy-to-read
formats. The 1990census can be down^
loaded into spreadsheet programs and then
manipulated, Many universi ties have this
Capability- Also the data is in formats that
can be loaded into geographic information
systems.
»• Contact marketing research consulting
firms which often prepare original research
and sell this information for a fee. They can
manipulate census data and other informa*
tian to provide unique perspectives. Some
services combine lifestyles and values with
census data to gain an unique view of a
specific geographic region. For example, a
report could include information about
everyone that lives within five miles of a
stream or other body of water.
»- Review other programs that targeted the
same audience.
Original Research
>- Conduct focus group interviews.
>- Conduct telephone, mail and personal
interviews.
: »• Interview community leaders.
*- Interview staff in agencies or organizations
that have contact with the target audience.
>• Interview experts.
>- Observe behavior by systematically
recording information at random locations.
>• Use omnibus surveys, which are often
conducted by universities and other
groups. The survey costs are shared by a
number of clients keeping the costs per
question very low.
>• Contact college professors who teach
courses in marketing and social research.
They are often looking for projects and will
complete market research for a small fee.
>¦ Contact MBA programs that have dub#
that wiU do consulting on target audience
research.
>- Interview organizational leaders.
»- Establish a contest requiring entrants to
answer a few simple questions about your
existing or future program.
>- Includes questionnaire with a newsletter
or other correspondence from theorganiza-
tfon.
> BtabUshapanelaradvisoryrerwewcommit-
tees.
Example of Original Research
Before a professional graphic art association
designed a water quality education program for
their members, they developed a questionnaire. The
questionnaire was designed to determine the depth
of awareness of water quality issues, their existing
office practices and product specification habits.
Chapter i A Blueprint for Success pages a-38
-------
Target Audiences pages 13-19
Where to Find More Information
Ambry, M. 1990. Consumer Power-How Americans Spend Their Money. Ithaca, NY: New Stratagist
Publications,
American Marketing Association, New York Chapter. 1992. International Directory of Marketing
Research Companies and Services. Hartsdale, NY: AMA Green book.
Bogart, L. 1989. Press and Public: Who Beads What, When, Where, and Why in American Newspapers.
Hillsdale, NT: Erlbaum Associates.
Burrelle's Media Directories. Burrelle's Special Groups Media Directory. Livingston, NJ: Buirelle's
Media Directories, (updated regularly)
CACI Marketing Systems. 1992. The Source Book of County Demographics.
Crispell, D. eds. 1991. The Insiders Guide to Demographic Know-How. American Demographic Press.
Crunig, J. 1983. Communication Behaviours and Attitudes of Environmental Publics: Two Studies.
Columbia, SC: Association for Education in Journalism and Mass Communication.
Direct Marketing Association. 1990. Statistical Fact Book-Curreni Information About Direct
Marketing. Direct Marketing Association, Inc.
Frank, N. 1983. Data Sources for Business and Market Analysis. Metuchen, NJ: The Scarecrow Press,
Inc.
Gallup, G., Newport, F. 1991. "Baby-Boomers Seek More Family Time." The Gallup Poll Monthly.
April: 31-42. (Monthly publication includes excellent polls on social trends)
Gannett Newspapers, Media Market Guide, New York: Gannett Newspapers, (updated regularly)
Goldstucker, J. eds. 1987. Marketing Information: A Professional Reference Guide. Atlanta, GA:
Georgia State University, College of Business Administration.
Henion, K. and T. Kinnear. 1975. Ecological Marketing. Chicago, IL: American Marketing Associa-
tion.
Hueber, G. 1991. "Americans Report High Levels of Environmental Concern, Activity." The
Gallup Poll Monthly. April: 6-12. (Monthly publication includes excellent polls on social trends)
Lesko, M. 1990. The Federal Database Finder. Keningston, MD: Information USA Inc.
Lesko, M. 1989. State Data and Database Finder. Kermingston, MD: Information USA Inc.
Mediamark Research. 1991. Study of Media and Markets. Mediamark Research, Inc.
Miller, T, and J. Schwartz. 1991. "The Earth's Best Friends." American Demographics. February: 25-
35. (Monthly publication includes excellent articles on social trends)
Simmon's Marketing Research Bureau. 1991.1990 Study of Media and Markets. New York:
Simmons Market Research Bureau, Inc. (updated annually)
Tull, D. and D. Hawkins. 1987. Marketing Research Measurement and Method. New York:
Macmillan Publishing, Co.
U. S. Department of Commerce. 1980. Social Indicators 111 Selected Data on Social Conditions and
Trends in the United States.
19
-------
Develop Specific Strategies and Plan
Program Implementation
OK now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals in consultation
with your staff;
Defined specific objectives;
Identified and broken down your target
audience; and
Learned more about your audience;
you are ready to develop the strategies that will
be used to reach each of these groups.
Strategies are simply the plans by which the
organization's objectives will be accomplished.
Where objectives focus on what will be done, the
strategies focus on how it will be done. Again this step
is vital to a successful program. If you do not spend
the time planning your communication strategies,
chances are, you will spend more time and money later.
Strategies should:
> Achieve organizational objectives;
>¦ Describe actions to be taken;
What is accomplished at this level is often depen-
dent upon the resources of your organization. It is
important to ask whether your organization can
afford to reach out to each subgroup of your target
audience. In many cases, the list may need to be
narrowed down. Many subgroups in the target
audience may require an unique program. Each
program can require a range of different skills and
organizational resources. Thus, it is important to do a
thorough job and pick only those subgroups that you
can effectively reach.
Strategies can be added to our previous example of
the wetlands journal.
Include an article describing how citizens can get
involved in the public process and discussing
strategies for wetlands protection through the local
planning process.
fAs you begin to develop specific strategies, you
should simultaneously begin to identify media formats
and channels you want to use. This step is described
starting on page 23, but in reality, these steps must be
• done in unison.
>¦ Consider the external environmental
threat* and opportunities;
»¦ Respond to changes in internal and exter-
nal surroundings; and
»• Guide daily operation decisions and tasks.
Chapter 1 A Blueprint for Success pages 6-36
-------
Tasks pages 21-22
1 Write and edit articles.
2 Select printer and paper type.
3 Develop mailing lists.
Tasks
Now that you have:
Used secondary research to answer
questions about your social/ political,
economic and organizational surroundings;
Identified reasonable goals in consultation
with your staff;
Defined specific objectives;
Identified and broken down your target
audience; and
Learned more about your audience;
Developed specific strategies; and
Selected media formats and channels
(see page 23);
you are fiitally ready to plan the tasks that will
be used to reach the target audience. Do not skip
steps to get here! If you have not completed the
steps listed above, go back and do them. Your
time is valuable, don't waste it on projects that
are ineffective and lack a planning agenda.
Tasks usually involve using machines and people
to accomplish the goals of the project. Examples of
tasks are included in our previous example of the
wetlands journal. The example to the right briefly
summarizes the entire process beginning with goals
and ending with tasks.
nmihhhh
To protect the Puget Sound and its watershed by
linking and supporting citizens and organizations
who are working locally for protection of wetlands
and water quality.
21
-------
Tasks pages 21 - 22
The timing of tasks is very important. Some tasks
can be completed simultaneously, while others must
occur in order. The following worksheet will help
you organize the tasks and communicate them to
your staff. For example, if you were going to produce
a brochure about nonpoint source pollution, before
you could print the brochure, you would have to
write and edit the text, design the layout, and choose
a printer. However, you could select a printer while
the other steps are being performed. Also, planning
the timing of tasks will help you plan the commit-
ment of your resources. If you have budgeted 100
hours of work for August and only have one staff
person available for thirty hours, the schedule must
be modified.
Major Tasks
January
February
March
Develop Journal Concept
X
X
Expenses
Salary
$1,340
$670
Office
$100
$400 mm
Travel
$25
$25
Materials
$48
$30 SB.
Staff Assigned
Jay Ooe
Jay Ooe
Staff Hours
80
40
Organizational Resources
j
Computer
X
Car
X
Slide Projector
X
Secretary
X
Manager in Charge
Jane Doe
Jane Doe
Chapter 1 A Blueprint for Success pages 6-36
-------
Media Formats and Channels pages 23-29 1
Media Formats and Channels
Ok, now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals in consultation
with your staff;
Defined specific objectives;
Identified and broken down your target
audience;
Learned more about your target audience;
and
Started developing specific strategies;
you are ready to start thinking about how you
want to get your message out to your audience.
Do you want to use printed materials, such as
fact sheets or newsletters? How will you
distribute these? Or would you rather prepare a
videotape? Where would people see it? What
about workshops? Radio? Television? What
would be most appropriate?
Media format refers to the type of material you
produce, such as a newsletter, fact sheet, video tape,
slide show, etc. Channel refers to the way that you
distribute your material. For instance, you might
choose to produce a video and show it at community
group meetings. The video would be the media
format you have chosen, and the community group
meetings would be the channel.
There are many media formats and channels-and
there are no set rules for which ones work best
Choosing the right media formats and channels
really depends on a number of factors. These include:
»• your message—its complexify, sensitivity,
style, purpose, content, and length;
»¦ your audience— its size, distribution, level
of awareness, and preference for certain
media formats andchannelsjand
>• your available resources—money, time,
staff availability, staff skills and knowledge.
The examples on the following pages list some
common media formats and channels. The pros and
cons of using each are compared on page 27.
Examples of Media Formats
>• TV public service
announcement
»- TV news story
>- radio public service
announcement
»• radio news story
»- newspaper story
>¦ newspaper
advertisement
>- newsletter
»- fact sheet
»- pamphlet
>¦ report
»- magazines
>- magazine
advertisement
»- magazine article
>- books
>- billboard
>- transit cards
(i.e., in buses)
>- "freebies"
(i.e., bumper stickers,
magnets, etc.)
>- t-shirts, hats, etc.
»• displays
23
-------
Madia Formats and Channels pages 23 - 29
How do these factors affect your decision?
Message
Consider whether your message is long, technical,
complex, or sensitive. Consider what you want
people to do with your message. For example, do
you want them to just read it and remember it? Or
use it, and refer to it frequently? Or share it with their
friends and neighbors? These considerations will
help you to decide what media format and channel to
use.
If your message is fairly long and/or technical,
you will probably want to send out print material,
such as a fact sheet or newsletter. But be careful! Even
with print material, you want to keep your message
as brief and non-technical as possible. TV and radio
are not usually adequate for conveying technical
information. However, videotape can be very useful
for demonstrating a behavior that you would like
your audience to adopt.
If your message is very sensitive (for instance, you
want to discuss the negative effects of runoff from
farms, but you don't want to offend or blame the
farmers) a more personal channel would likely be
more effective. You might still choose to prepare a
pamphlet describing the more technical aspects, but
instead of mailing it, you might choose a more
personal channel such as a workshop, community
meeting or personal visit.
If you want people to simply read your informa-
tion, print it on recycled, recyclable paper that is
easily disposed. If you would prefer that people keep
and refer to your material, you might choose a
heavier, glossier media format, more like a magazine,
that could be kept on a bookshelf or coffee table. (You
can still use recycled paper to produce this type of
material.) Or, depending cm the content of your
message, it might be more appropriate to produce a
magnet for the kitchen refrigerator.
Your Audience
Consider your audience. You should already
know a lot about them, having done your target
audience research. How big is your audience? How
widely distributed are they (such as in a rural area)?
Examples of Channels
»• workshops
>- community meetings
»- church meetings
»- school meetings
»- one-on-one personal contact
»- opinion leaders
(i.e., community leaders,
parents, teachers, friends, etc.)
»¦ "event" days
>¦ fairs
»¦ libraries
How much do they already know about the issue?
What types of media formats do they use and to what
channels do they respond? How likely are they to
change their behaviors?
In order to reach a large target audience, you may
choose to send out newsletters or other print mate-
rial. You may be able to print a large number of
newsletters (in fact, printing costs go down with
larger "runs"). However, make sure that you can
afford to mail them, if that is the distribution channel
that you choose.
To reach a large audience, you may also choose to
run public service announcements (PSAs) on TV or
radio. PSAs are free announcements; they may be issued
by government agencies or non-profit organizations. TV
has the largest general audience. However, PSAs are
often run late at night, when viewers may not be
watching TV. Running a radio PSA allows you to
choose stations to matrii your target audience. Be aware
that both TV and radio receive many requests to run
PSAs; you may be in competition with otters.
Other ways to reach a large audience include
displays at public places such as libraries, fairs, and
shopping centers; "transit cards" on public buses and
subways; billboards; and "freebies" such as bumper
stickers and buttons.
Chapter 1 A Blueprint for Success pages 6-36
-------
Media Formats and Channels pages 23 - 29
Make sure that your audience responds to the
media format and channel that you choose. For
instance, if you are dealing with an area that has a
high rate of illiteracy, newsletters would not be your
best bet. Workshops would probably not be most
effective in a very rural area, where people would
have to travel long distances to attend. Some people
are very skeptical about things that they see on TV;
therefore, television PSAs would not work best for
them.
Perhaps most importantly, make sure that you are
communicating in the language that your target
audience uses! While this seems obvious, neverthe-
less many materials are sent out printed only in
English, when the target audience speaks another
language. Communicate in your target audience's
language, as an organization in the Puget Sound area
did—they printed pamphlets for dry cleaners in both
Korean and English.
Budget, Time, and Other Resources
Consider your available resources. How much
money do you have to spend on production and
distribution? How much time do you have? How
many staff people are available and what are their
skills and expertise?
Producing your communication materials will
likely be a major cost of your program. Make sure
that you have enough money to produce sufficient
quantities of your materials, and that you can afford to
distribute them. It is no use to have 5000 wonderful
pamphlets sitting in boxes because you can't afford to
mail them out As one coordinator of a Puget Sound
project stated "people should be aware that any time
you print something, it costs a bundle to mail it. 1
ended up driving the materials around, but you can't
do that forever."
Consider the number of people that you want to
reach as a function of the amount of money you will
be able to spend. You can easily calculate a "cost per
person" by dividing the total cost of production by
the number of people that will be reached. This will
allow you to compare different media formats and
channels on a cost basis. A video, for instance, might
have seemed cost-prohibitive at first, but if you can
reach a large number of people with it, it might be
worth the cost. Keep in mind, however, that reaching
people is not die same as changing their attitudes or
behaviors. You may be able to reach a large number
of people through one media format, but have little
or no impact on them. Whereas, by using another
media format, you may reach fewer people, but get
your message across much more effectively.
If you are a non-profit organization, here's a hint
on reducing your production costs... get others to
pay! Ask local companies, printers, graphics agencies,
paper merchants, and typesetters to contribute work
or materials for free (or at least, at a discount) in
exchange for credit in the publication. You can also
get volunteer labor from scouting troops, church
groups, community organizations, schools, and other
volunteer organizations. For example, one organiza-
tion in the Puget Sound area was able to get the
following donated: printing costs and paper donation
(from a local paper mill); additional printing (from a
printing company); use of a mail permit (from the
town); displays, videotapes, and awards (from
various businesses participating in an "event day");
volunteer labor (from a Boy Scout troop); and stencils
for storm drains (from a regional graphics firm.)
Allow enough time to prepare your materials.
Writing and production almost always take longer
than you think they will! If you need to get informa-
tion out to your audience in a hurry, consider a press
release (if your audience is very large) or personal
visits (if your audience is very small). Above all, plan
ahead!
Match your media formats and channels to the
skills and expertise of your staff. If your staff consists
of highly-skilled scientists who understand very
technical issues, but are scared stiff in front of an
audience, it probably isn't a good idea to send diem
out to conduct workshops with community groups.
Finally, consider the timing of the release of your
materials with whatever else is going on in the
community. For instance, you might not want to
release a pamphlet in the mail during local election-
time, when people tend to receive a great deal of
similar information in the mail. You might not want
to hold a workshop on the same day as another
significant event Remember that whatever you are
trying to do, you are competing for the time and
attention of your audience.
25
-------
Madia Formats and Channels pages 23 -29
Choosing Channels
Even with all the factors described above under
consideration, there are still a few things to think
about when choosing channels. Channels fall into
several categories: mass media, personal contact, and
community channels. Each type of channel has
different characteristics that may make it more or less
suitable to your audience.
Mass Media
Mass media is the best way to raise your
audience's awareness of an issue in a general sense,
mainly because it reaches a large audience, and is
seen as a credible source of information. Most people
rely on the mass media (e.g., television, radio,
newspaper) as their primary source of information.
However, the mass media are less effective at
motivating people to change their behaviors.
Personal Contact
More personal contact has been shown to work
better in persuading people to change their behav-
iors. Information from peers, friends, and family
usually is regarded as more useful or more credible
than information that comes from an "expert"
People are often more likely to believe information
that comes from people they know than information
from a stranger, especially if that stranger is consid-
ered an "outsider" to the community. You may be
most effective if you are known and respected within
the community. If not, get to know the community's
leaders, who may then be able to pass the message
along. Try channels that offer more personal, infor-
mal contact, such as workshops and staffed exhibits.
An organization in the Puget Sound area found
that telephone calls worked better than sending
flyers. According to the coordinator of the project,
"just sending out flyers is not very effective at all.
Newspaper articles are good. But the best method is
to start a word-of-mouth campaign with people you
know are interested. Calling people on the phone is
good. It's personal. They feel like you really want
them to help. Then they'll call people they know."
Another organization, also in the Puget Sound
area, met with dairy farmers in "kitchen meetings" to
discuss water-conscious methods of managing farm
animals. Staff members showed a videotape that had
been prepared using a narrator familiar to the target
audience—a prominent farm broadcaster with a
dairy radio talk show. They then started an informal
discussion of the issues with the farmers.
Community Channels
Community channels fall somewhere between the
media and personal contact. Examples of community
channels are schools, companies, community organi-
zations, and religious organizations. These organiza-
tions often offer open channels, such as meetings,
where you may make presentations and conduct
workshops. Even better, the activities of the organiza-
tions may reinforce and expand upon your message.
For instance, if you give a presentation on local water
quality, the organization with which you are working
might decide to do a stream-walk or start a citizen
monitoring program.
How Many Channels?
Using more than one channel may increase the
effectiveness of your message. People are involved in
many activities, and you can probably reach them
through more than one channel. For instance, people
may go to work during the day, and then come home
and watch the evening news and read their mail, and
then go to a community meeting in the evening. All
of these activities offer channels through which you
may reach them. Reaching people through more Sun
one channel increases their exposure to the message,
and may increase their chances of remembering it or
even acting on it. As an example, one group in the
Puget Sound region hand-delivered a brochure to
peoples' homes. Of those who read it, the majority
found it very effective. However, only half of the
audience even remembered receiving it. This organi-
zation would have benefitted by using another
channel in addition to hand-delivering the brochure.
However, be careful if you decide to use more
than one channel. Don't spread your resources too
thin. It will not be more effective to use more than
one channel if you can't use any of them well, and if
you can't sustain your program. If you are con-
strained by resources, choose one channel and use it
well. If you have more resources to spread around,
then consider using more channels.
!6 Chapter 1 A Blueprint for Success pages 6-36
-------
Madia Formata and Channel# pages 23 - 29
Characteristics of Some Media
MEDIA FORMAT
CHANNEL
PROS
CONS
Newalettera
mail, handout
can reach a large audience
printing/mailing is costly
can be more technical
staff time
passive, not participatory
Videotape
workshops
can reach a large audience
relatively expensive
mail
visually pleasing
must be done well
cable TV
more participatory
can show behavior
Public Service
TV
free
sometimes aired at night
Announcamenta
radio
can reach a large audience
competition for air time
can target audience
very passive
difficult to evaluate
Mass Media
TV
can reach a large audience
constrained by time, space
radio
good for raising awareness
must be "newsworthy"
newspapers
usually considered credible
can't explain complex issues
bad for persuasion
Preeentationa
workshops
can be participatory
read) smaller audience
conferences
good for persuasion
staff time
group meetings
can show behavior
can be too technical
more personal
people may not attend
Exhibits
libraries
can reach a large audience
staff time
malls
visually pleasing
must be durable
fairs
Fraebiea
fairs
increases awareness
very short message
i.s., bumper stickers,
"event" days
inexpensive
weak on persuasion
buttons, magnets,
easy to produce
hats, ate.
27
-------
Media Formats and Channels pages 23 - 29
Things to consider when you are choosing
media formats and channels
Is your message:
long?
sensitive?
technical?
Do you want people to:
read or listen to it?
remember it?
act on it?
save it?
share it?
refer to if?
Do you want your message to:
raise awareness?
inform people how to take action?
give feedback onaction?
is your audience:
— large?
spread out geographically?
well-informed?
active?
responsive to messages like yours?
responsive to the media format you
Do you have:
enough money to produce and
distribute your materials?
skilled staff capable of carrying out
your program?
to devote to your program?
enough time to produce and distribute
your materials?
Chapter 1 A Blueprint lor Success pages 6-36
-------
Media Formats and Channel*
Where to Find More Information
Creighton, J. 1981. The Public Involvement Manual. Cambridge, MA: Abt Books.
Fazio, J. R. and D. L. Gilbert. 1986. Public Relations and Communications for Natural Resource
Managers (2nd ed.) Dubuque, IA: Kendally/Hunt Publishing Company.
National Cancer Institute. 1988. Making Health Communication Programs Work: A Planning Guide.
Washington, D.C.: National Institutes of Health.
Puget Sound Water Quality Authority. 1991. Public Involvement and Education Model Projects Fund;
47 Success Stories from Puget Sound. Olympia, WA: Puget Sound Water Quality Authority.
-------
Planning Your Evaluation
Now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals in consultation
with your staff;
Defined specific objectives;
Identified and broken down your target
audience;
Learned more about your audience;
Developed specific strategies;
Selected media formats and channels
(see page 23);
Harmed the tasks and prepared a timeline;
you are ready to plan your evaluation.
Evaluation is a critical element of any program. If
you don't do evaluation, how will you know whether
you have accomplished anything you set out to do?
Because it is an essential part of any communication
program, we have devoted two chapters of this
handbook to die topic. Chapter 3 describes die
planning process for evaluation and Chapter 4
describes specific evaluation techniques you can
choose.
30 Chapter 1 A Blueprint tor Success pages 6-36
-------
Recording the Plan
Now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals in consultation
with your staff;
Defined specific objectives;
Identified arid broken down your target
audience;
___ Learned more about your audience;
Developed specific strategies;
. Selected media formats arid channels
(see page 23);
Planned the tasks and prepared a timeline;
and
Decided on an evaluation plan:
you are ready to summarize your program.
At this point, you should have a good idea of how
your project will be organized. The planning
worksheet on page 33 will summarize your program.
This approach requires that you investigate the
operational details of the program and estimate the
resources required to complete the tasks. Finally, it
communicates a lot of information about the project
to staff members, management, board members and
supporters outside the organization. With this
information you can move to chapter two, which
discusses how to develop effective messages.
-------
Preparing Your Materials
Now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals In consultation
with your staff;
Defined specific objectives;
Identified and broken down your target
audience;
Learned more about your audience;
Developed specific strategies;
_ Selected media formats and channels
(see page 23);
Planned the tasks and prepared a timeline;
_ Decided on an evaluation plan;
you are ready to start preparing your materials.
Chapter 2 gives you tips on designing effective
materials, ranging from format and design to using
graphics to enhance your materials.
32 Chapter 1 A Blueprint for Siiccms pages 6-36
-------
Planning Worksheet
Planning Worksheet
Program Project Manager
Relevant Goals
Relevant Objectives
Relevant Strategies
Target Audiences
Planning Variables
Planning
Implementation
Follow Up
1. Source of funding
2. Budgeted expenses
Salary
Office Expenses
Travel
Materials
3. Proposed date of completion
4. Evaluation techniques
5. Staff members assigned
6. Budgeted staff time
7. Organizational resources needed
8. Interactions with other departments
9. Interactions with outside agencies or
subcontractors (also contact person)
10. Managers in charge and their date
of approval
11. Possible problems and proposed
strategy for a solution
33
-------
A Checklist for Planning an Effective Communication Program
Reviewing the Surroundings
Have you analyzed the social surroundings and how
they affect your program ?
What are the political forces that will
affect your program? What components
of your program will be hindered and
improved by these forces?
How will your program affect the
economy erf your community?
Is your organization prepared for this
program? How can you Change the
organization or the program to improve
the results?
Have you answered these questions
about your surroundings based upon
current conditions and future conditions
anticipated once the program begins?
Have you relied on the sources of second-
ary research to answer your questions?
Objectives
Have you developed objectives that:
Are prioritized;
Are important to the organization;
Are internally consistent;
Are measurable or quantifiable;
to achieve a goal;
Have a specific purpose (several objec-
tives could be inciuded for one goal)
Have a single outcome; and
Have a date of completion?
Target Audiences
Have you identified your target audience?
Did you conduct research to identify
unique characteristics about the target
audience?
Have you identified the exchange that
will occur with the target audience?
Have you broken down the target
audience into subgroups?
Do you understand what the target
audience's current level of knowledge is
about the issues included in your pro-
gram?
. Do you know what communication
channels are used by the target audience
and whetherthey have credibility?
Goals
Have you developed goals that:
Are general, broad and all-encompass-
ing, but not conflicting;
Provide the staff with something to strive
toward;
Supply guidance to the program's staff
when making decisions;
Are internally consistent;
Communicate the purpose of the pro-
gram to the public and entities external to
the organization;
Provide the basis for developing objec-
tives, strategies and tasks; and
Unites the program staff toward the
achievement of a common purpose?
34 Chapter i A Blueprint for Success pages 6-36
-------
A Checklist for Planning an Effective Communication Program pages 34-35
Strategies
Have you developed strategies that:
Adiieve organizational objectives;
Describe actions to be token;
Consider the external environmental
threats and opportunities;
Respond to changes In internal and
external surroundings; and
„ Guide daily operation decisions and tasks?
Tasks
Hiwe you:
Described the tasks that must occur to
reach your target audience?
HUed in the planning sheet that summa-
rises your goals, objectives, strategies and
tasks?
_ I^ed in the nmthly planning sheet that
surmnarizes organisational resources
devoted to the completion of each major
task?
Media Formats and Channels
Have you:
Considered how your message, audi-
ence, and available resources affect what
inediaformats and channels you choose?
Used the table on page 27 and the
checklist on page 28 to help you dxx»e
media formate and channels?
Evaluation
Have you:
Read Chapters 3 and 4 and selected
evaluation techniques?
Recording the Plan
Did you :
___ Summarize infonnationabout the project
using the summary planning sheet?
35
-------
Where to Find More Information
Fine, S. 1990. Social Marketing. Needham Heights, MA: Allyn and Bacon.
Gaedeke, R. 1977. Marketing in Private and Public Nonprofit Organizations. Santa Monica, CA:
Goodyear Publishing Company, Inc.
Kotler,P. and A. Andrea sen. 1991. Strategic Marketing far Nonprofit Organizations. Englewood
Cliffs, NJ: Prentice-Hall.
Kotler, Pand O. Ferrell. 1987. Strategic Marketing far Nonprofit Organizations Cases and Readings.
Englewood Cliffs, NJ: Prentice-Hall.
Lovelock, C. and C. Weinberg. 1984. Marketing for Public and Nonprofit Managers. New York: John
Wiley & Sons.
Lovelock, G, and C. Weinberg. 1978. Readings in Public and Nonprofit Marketing. Scientific Press.
McCroady, G. 1982. Marketing Tactics Master Guide for Small Business. Englewood Cliffs, NJ:
Prentice-Hall, Inc.
Rados, D. 1981. Marketing For Nonprofit Organizations. Boston, MA: Auburn House Publishing Co.
Rice, R. and K. C. Atkin (Eds.) 1989. Public Communication Campaigns. Newbury Park, CA: Sage
Publications, Inc.
Stein, J., 1986. "The Cancer Information Service: Marketing a Large-scale National Information
Program Through die Media." in Leathar, D., Hasting, G., O'Reilly, K., Da vies, J. (Eds.) Health
Education and the Media II. Elmsford, NY: Pergamon Press.
36 Chapter 1 A Blueprint for Sucoms pages 6-36
-------
Where to Find More Information
37
-------
Chapter 2
Introduction
39
Useful, Relevant Information ,
What to Do 40
Where to Find More Information 41
40
Concrete Language
What to Do 42
Where to Find More Information 43
42
Vivid Language
44
How To Create Vivid Language 44
Where to Find More Information 45
Format
How to Format 46
Where to Find More Information 46
46
Page Layout and Type
48
Tips for Effective Page Layout 48
Type 48
Columns 48
Other Attention-grabbers 48
Where to Find More Information 48
Use of Graphics 50
Graphics Can Be Great 50
Tips on Using Graphics Effectively 51
Where to Find More Information 51
Titles and Headings 52
Titles 52
Headings 52
Creating Good Titles and Headings 52
Use Headings in Speeches Too 53
Where to Find More Information 53
Whits Space
Creating White Space 54
Where to Find More Information 55
A Checklist for Effective Communication Materials.,
54
56
38
-------
Introduction
Tips for Material Design
Ok, now that you have:
Used secondary research to answer
questions about your social, political,
economic and organizational surroundings;
Identified reasonable goals in conjunction
with pur staff;
Defined specific objectives;
Identified and broken down your target
audience;
Learned more about your target audience;
Developed specific strategies;
Selected media formats and channels;
Planned the tasks and prepared a timeline;
and
Decided on an evaluation plan;
you are ready to begin actually designing your
materials!
Introduction
This chapter gives you tips on designing effective
communication materials. "Effective" materials are
ones that people pay attention to, remember, and use.
Effective materials should persuade people to change
their behaviors to more environmentally-friendly
ones, and perhaps, go out and influence others to
change their behaviors too.
Each tip includes a box called "Link with" that
relates it to another tip, as well as a box called
"Media" identifying the appropriate media formats.
In addition, we have provided examples illustrating
these tips from materials produced for EPA's Chesa-
peake Bay Program and fine Puget Sound Water
Quality Authority's Public Involvement and Educa-
tion (PIE) Fund.
The tips provided in this chapter are vital—if you
don't use them, people may not even read your
materials or listen to your presentation! These tips
have been shown to increase the effectiveness of
communication materials by increasing peoples'
attention and memory, or persuading people to
change their behavior.
These tips should help you create appealing,
interesting materials that people will pay attention to,
remember, and act upon, ultimately improving
environmental conditions in your area.
39
-------
Useful, Relevant Information
Unk with
Madia
All people crave information; that's part of what
makes us human. Information helps us understand
the world around us and helps us function in that
world. However, there is a lot of information out
there—too much for most of us to be able to deal
with it all. Because of this, people tend to ignore
information that doesn't seem like it will be useful to
them. If we, as environmental communicators, want
to give some information to other people, then, it is
very important to give them information that is
useful to them, that is relevant to their lives.
Most of us who write about environmental issues
probably think that we are writing important,
relevant, useful stuff that people should pay attention
to. But too often, environmental literature talks about
the problems—the greenhouse effect, eutrophication,
erosion and runoff, etc. Solutions are rarely discussed
(with the exception of books such as 50 Simple Things
You Con Do to Save the Earth.) Knowing more about
the problems isn't useful to people; knowing more
about solutions is. Therefore, if you can talk about the
problems and suggest ways that people can tackle the
problems, the chances that your audience will pay
attention are likely to increase dramatically.
You would think that at least "educating" people
about die problems would help. But it usually
doesn't. Study after study has shown that simply
providing people with information doesn't usually
make them change their attitudes, and rarely makes
them change their behavior. Think about it; most
people know that recycling saves resources. But not
everyone recycles. One study has shown that the
main reason people don't recycle is that they just
don't know how. In this case, people need informa-
tion that tells them how to take action.
Just talking about the problems may raise aware-
ness, but is not likely to lead to changes in behavior
that might help improve the situation. Talking about
the problems is just giving people more informa-
tion—information that may very well be ignored
because people don't find it useful. People need to
know more about the solutions and about actions
that they can take. This information is more useful
and relevant to them.
What to Do
To make information more useful and relevant to
people, explain what they should do with it Provide
solutions. For instance, rather than just talking about
the problems associated with runoff from suburban
lawns, explain what people can do and how to go
about doing it. If possible, go a step further and
explain how people can know whether their action
worked. And if the action didn't work, what should
be done next.
To make your message even more useful, show
solutions. Use simple graphics. In a video or slide
show, you can show pictures of people "doing the
right thing." In a workshop or classroom, for in-
stance, you could demonstrate how to make alterna-
tive household cleaners in order to reduce the
amount of household hazardous waste.
Giving people this more detailed information
about what they can do provides a mental link
between the problem and personal action that leads
to a solution. It is more likely to lead to actual
changes in behavior than simply describing the
problem. And changes in people's behaviors are
more likely to lead to a cleaner environment.
- Concrete Language
- Use of Graphics
- All
(newsletters, video, workshops, etc.)
40 Chapter 2 Tips for Material Design pages 38 -57
-------
Useful, Relevant Information pages 40-41
Example of Useful, Relevant Information
What You Can Do
As important as federal, state and county
efforts for cleaner water are, concerned people
are still the Severn's primary guardians. The
Severn River Association, for example, a group of
concerned private citizens, has kept watch over
the river since 1903, making it one of the nation's
oldest river protection organizations. You can
help keep the Severn beautiful.
If your property has a private septic system,
be sure it is cleaned and maintained regularly.
Whenever possible, replace solid paving with
gravel or vegetation that will trap stormwater.
Limit fertilizers and, if your land is on a steep
grade, take steps to prevent erosion by encourag-
ing the growth of shrubs and other dense ground
cover. Make sure that you properly maintain and
operate engines and sanitation devices on your
boat. Properly dispose of toxic and hazardous
substances including oil, paint, anti-freeze, etc.,
instead of pouring them on die land or into
stormdrains that go directly into the River.
Finally, let your representatives in the govern-
ment of the city of Annapolis, Anne Arundel
County and the state of Maryland know that you
and others require that Maryland's Scenic Capital
River be restored to good health and sustained
for the future.
Where to Find More Information
De Young, R. 1989. "Exploring the difference between
recyclers and non-recyclers: The role of information."
Journal ofEnznronmental Systems. 18:341-351.
Dervin, B. 1989. "Audience as Listener and Learner, Teacher
and Confidante: The Sense-Making Approach" in Rice, R.
and C. K. Atkin (eds.) Public Communication Campaigns.
Newbury Park, CA: Sage Publishers.
Earthworks Group. 1989.50 Simple Things You Can Do to Save
the Earth. Berkeley, CA: Earthworks Press.
Funkhouser, G. R. and N. Maccoby. 1974. "An Experimental
Study on Communicating Specialized Science Information
to a Lay Public" Communications Research. 1:110-128.
Kaplan, S. and R Kaplan. 1982. Cognition and Environment:
Functioning in an uncertain world. New York: Praeger.
McA lister, A., A. G. Ramirez, C. Galavotti, and K. J. Gallion.
1989. "Antismoking Campaigns: Progress in the Applica-
tion of Social Learning Theory" in Rice, R. E. and C. K.
Atkin (eds.) Public Communication Campaigns. Newbury
Park, CA: Sage Publishers.
Simmons, D. A. 1983. The Use of National Park Planning
Workbooks for Citizen Participation: Enhancing Effective
Communication. Doctoral dissertation: University of
Michigan.
Sims, J. H. and D. D. Baumann. 1983. "Educational Programs
and Human Response to Natural Hazards" Environment
and Behavior. 15(2): 165-189.
Teigen, K. H. 1985. "The Novel and the Familiar: Sources of
Interest in Verbal Information" Current Psychological
Research & Reviews. Fall 1985:224-238.
Wang, G. 1977. "Information Utility as a Predictor of News-
paper Readership" Journalism Quarterly. 54:791-797.
Young, C. F. 1989. Evaluating the Effectiveness of Gypsy Moth
Educational Brochures. Doctoral dissertation: University of
Michigan.
— from The Severn River, a fact sheet produced by Alliance for
Ike Cheupaake Bay.
41
-------
Concrete Language
link with
- Useful, Relevant Information
- Use of Graphics
Media
- All
Language that is more concrete is more under-
standable and more likely to be remembered,
especially if it is easy to imagine. People tend to skip
over more abstract language, but concrete language
forms pictures in people's minds that are better
remembered—"cemented" in memory.
For example, "pollution" is an abstract term
and is difficult to picture. Try it. You are probably
picturing a specific type of pollution. "Shiny, oily
muck covering the shoreline" might be a more
concrete, descriptive phrase for what you are trying
to say. If you want people to read (or listen to) and
remember your message, make your language
concrete. Give people something they can grab onto.
Cement your message into their minds.
What to Do
Here are some tricks to help you use more con-
crete language:
>- Stay away from jargon.
>• Use examples.
> Use analogies.
> Use descriptive adjectives and adverbs.
>- Use active verbs.
v Avoid technical, scientific language.
»- Try to picture what you are saying.
>¦ Use graphics to illustrate what you are
saying.
Example of Concrete Language
"Earth and water continue to
compete for this territory,
redrawing the shoreline as land is
built up in some areas and lost
beneath the waves in other places."
— from The Chooptoke Bay, a fact sheet produced by the
Alliance for Mm Chesapeake Bay.
Another Example of Concrete language
"The variety of conditions supports
some 2,700 species. All are linked in
a complex, interdependent web of
producers and consumers. From
the eagle's huge nest high in a
wetland tree to the worms in the
Bay's bottom sediments, from the
microscopic free-floating plants to
the pine trees along the
Shenandoah, all have a part in
maintaining this system's balance."
— from The Chaapnke Bay, a fact iheet produced by the
Alliance fortheCheaapeake Bay.
Chapter 2 Tips for Material Doslgn pages 38 -57
-------
Where to Find More Information
Brans ford, J. 1979. Human Cognition: learning, understanding,
and remembering. Belmont, CA: Wadsworth Publishing
Company.
Funkhouser, G. R. and N. Maccoby. 1971. "Communicating
Specialized Science Information to a Lay Public" Journal of
Communication. 21:58-71.
Funkhouser, G. R. and N. Maccoby. 1974. "An Experimental
Study on Communicating Specialized Science Information
to a Lay Public" Communications Research. 1:110-128.
Nisbett, R. and L. Ross. 1980. Human Inference• strategies and
shortcomings cf social judgment. Englewood Cliffs, NJ:
Prentice-Hall, Inc.
Paivio, A. 1971. Imagery and Verbal Processes. New York: Holt,
Rinehart and Winston, Inc.
Paivio, A. 1991. Images in Mind: The Evolution of a Theory.
London: Harvester Wheatsheaf.
Sharpe, G. W. 1982. Interpreting the Environment (2nd ed.)
New York: John Wiley and Sons, Inc.
Sherer, M. and R. W. Rogers. 1984. "The Role of Vivid
Information in Fear Appeals and Attitude Change" Journal
of Research in Personality. 18321-334.
Simmons, D. A. 1983. The Use of National Park Planning
Workbooks for Citizen Participation: Enhancing Effective
Communication. Doctoral dissertation: University of
Michigan.
Young, C. F. 1989. Evaluating the Effectiveness of Gypsy Moth
Educational Brochures. Doctoral dissertation: University of
Michigan.
-------
Vivid Language
Unkwith
- Concrete Language
Media
- All
Just as concrete language gives the reader some-
thing firm to grasp onto, vivid language allows
people to paint a picture in their minds of a scene or
situation. Vivid language draws attention and
increase peoples' ability to remember the situation
being described. By writing vividly, then, you can
attract your audiences' attention and increase their
chances of remembering what you've said.
How To Create Vivid Language
»¦ Tell a story to draw your reader in and add
more ""human interest."
> Personalise the information, by giving
examples that are closely related to your
audience, or by describing people that are
similar to those in your audience.
> Describe consequences of action (or no
action) in terms of an individual or family
rather than a mass such as "the public."
v Use colorful, descriptive adjectives and
adverbs.
»• Describe something unusual or striking.
»¦ Use emotional or fear appeals. (However,
be careful when using fear appeals, as they
can also cause your reader to become
overwhelmed. You should provide your
reader with a way to handle the fiear, for
instance, by describing actions the reader
can take to avoid the fearful situation.)
>» Use anecdotes and examples to add color,
action, life and interest
>¦ Encourage your audience to actively
participate (even if they are reading} by
saying things like "imagine-." or "try to
picture./'
Example of Vivid Language
"If you were a Susquehannock
Indian living in the early 1600s, the
Bay was a mystery—both a source
of edible delights and a pathway
for your adversary, the
Piscataways. If you were a soldier
in the War of 1812, you might have
fought some bloody battles on its
waters. And if you were a coffee
merchant in Baltimore in the early
1900s, you very likely depended on
the arrival of The Josephine for your
livelihood."
— from The Chmpmkt Buy, a fact iheet produced by the
Alliance lor the Chaaapealce Bay.
14 Chapter 2 Tip* for Material Dealgn pages 38-57
-------
Vlvkl Language
Where to Find More information
Gick, J. W. and R. N. Baiid. 1986. Magazine Editing and
Production (4th edition) Dubuque, IA: Wm. C. Brown
Publishers.
Nisbett, R. and L. Ross. 1980. Human Inference: strategies and
shortcomings of social judgment Englewood Cliffs, NJ:
Prentice-Hall, Inc.
Paivio, A. 1971. Imagery and Verbal Processes. New York: Holt,
Rinehart and Winston, Inc.
Paivio, A. 1991. Images in Mind: The Evolution of a Theory.
London: Harvester Wheatsheaf.
Sherer, M. and R. W. Rogers. 1984. "The Role of Vivid
Information in Fear Appeals and Attitude Change" Journal
of Research in Personality, 18:321-334.
-------
Format
link with
- Use of Titles and Headings
- White Space
- Use of Graphics
- Page Layout and Type
Madia
- Written materials
It is important that you create a clear and consis-
tent format in order to help your reader find desired
information quickly and easily. An audience that is
confused or overwhelmed by the format will not read
the material. However, if you are consistent with
your format, you will help your reader learn how to
use your document, and will greatly increase the
chances that your materials will be read. Think about
it; how many times have you put a book or article
down, even though you know the information you
want is in there, because you just can't find that
information quickly and easily?
Where to Find More information
Click, ]. W. and R. N. Baiid. 1986.Magazine Editing and
Production (4th edition) Dubuque, LA: Wm. C. Brown
Publishers.
Simmons, D. A. 1983. The Use of National Park Planning
Workbooks Jbr Citizen Participation: Enhancing Effective
Communication. Doctoral dissertation: University of
Michigan.
How to Format
Formatting can consist of almost anything, as long
as you are consistent about it and it easy to identify.
For instance, some documents show different
chapters with different colors or with inked edges.
You can set apart titles and headings with lines and
blocks. Begin a new section on a new page. Set aside
a block on each page that provides a certain type of
information, such as a list of "things you can do" or
"where to find more information."
Be creative in the way that you format your
document Make it neat and easy to follow, and
above all, be consistent. Good formatting essentially
allows you to lead your reader by the hand through
your document.
-------
Format pages46 - 47 |
Example of Formatting
ft /a otor oil. battery
| acid, gasoline, car
-+J, ,, wax. engine
«. W I cleaners, antifreeze,
degreasers, radiator (lushes,
and rust preventative* are ex-
amples of automotive products
containing toxic chemicals.
Some car owners do their own
maintenance work: 25 percent
change their car's oil, and
many of these people pour the
used oil down the norm drain.
One quart of oil can contami-
nate up to two million gallons
of drinking water. The oil from
one engine—four to six quarts
—can produce an eight-acre oil
slick.
The only recommended
way to dispose of used oil Is to
put it into a sturdy container,
like a plastic milk jug. and take
it to your neighborhood garage
or oil recycling center. Mary-
land. Pennsylvania. Virginia,
and the District of Columbia all
have a number you can call to
find an oil recycling station
near your home.
Disposing of antifreeze is
also a problem. Antifreeze con-
tains ethylene glycol, which is
poisonous to people, fish, and
wildlife Many cats and dogs
have died after drinking sweet
tasting puddles of antifreeze
they find on driveways in the
winter.
Instead of pouring anti-
freeze down the drain or wash-
ing it into storm drains, ask
your local service station to add
the liquid to their used anti-
freeze storage drum. You can
also dilute the antifreeze and
pour the mixture into a gravel
pit or any area with good drain-
age. This method takes advan-
tage of the soil's natural filter-
ing capacity.
c' Hcuier, o':i
iCXiCi
F' he kinds of household
toxics described in this
chapter should not be
»•' disposed of "down the
Oil con be ricyctaD UsedoHono
anlitou* should to >ok$n to your
local service skjhon lot recycling
New hose on or onmreest down mo
your stotmdiom
drain." Ybur drain leads either
to a home septic system or a
municipal treatment plant, nei-
ther of which is designed to
completely remove toxic chem-
icals from wastewater. At least
some of the toxics pass through
the treatment process and end
up in a stream, river, or
groundwater. Read the section
in this guide on septic systems
for further cautions.
Effective sewage treat-
ment is essential for treating
the large volume of wastewater
that comes from our homes
Improving municipal treatment
plants is a part of the strategy
to clean up Chesapeake Bay.
Well-run treatment plants can
remove some nutrients, or-
ganic materials, and heavy
metals from wastewater. The
chlorine used to disinfect can
also be removed by a process
called dechlorination. Individ-
uals and communities should
insist that the publicly-owned
treatment plants thai serve
them are maintained and oper-
ated at peak efficiency. This
may mean added cost to con-
sumers. but is essential tn good
water quality in our rivers and
Bay
The products described in
this chapter should also never
be poured on the ground or
into gutters where they will
eventually enter storm sewers,
which generally lead directly in
a nearby stream.
In many areas, the only
available disposal method is
the local landfill. While proba-
bly better than flushing a toxic
chemical down the drain, land
Alls are not a good long-term
solution to our waste disposal
problems. New energy needs to
go into Hnding better solutions.
Where household hazard-
ous wastes must be sent to a
landfill, a couple of steps can
be taken to reduce the environ-
mental risk. First, wrap the
product in its original container
In newspaper, and then wrap
in an old plastic bag. Liquids
can be poured into containers
filled with absorbent kitty litter,
then wrapped in plastic.
Some states are dealing
with the problem of hazardous
household wastes by sponsor-
ing amnesty days. On amnesty
day. small quantities of your
unwanted household chemi-
cals and pesticides are col-
lected and disposed of in an
approved facility. The actual
collection and disposal of the
waste should be performed by
technicians who know which
chemicals should not be mixed
together. Amnesty days are de-
signed to educate the general
public about the potential haz-
ards of improper use and dis-
posal of consumer products
that contain toxic chemicals.
Check with your state or county
government to And out about
amnesty days in your area.
; ' ™i i?. i» -t: aC:
s*:** -.".nwr ?«¦ V:siafc*. r
,i*altui«s«f
"i irtiidTWlKmi "•*•
jattnflal'
it siisi; V
* 9 3lOK|nQuc»]n>r•
7.
Jatf— ^
v»-whan«ver
' "I "¦
i-llteiip
...
i &
-if!!'
uasiii "
"M
iiiiptiiiiatei'
v'ziwMwwmm*'
'l >"i"l H f ¦ i 1 iHflHtM ¦"» ItfV*
M WSgr II
for^esx commit
*ky. -
' *< >K ; _ jfgt?
;;5;ijsi K
j o
il
U S E H 0
D
CHEMICALS
from Bttybook produced by the Allium for the Chesapeake Bay.
47
-------
Format
Link with
- Use of Titles and Headings
- White Space
- Use of Graphics
- Page Layout and Type
Madia
- Written materials
It is important that you create a clear and consis-
tent format in order to help your reader find desired
information quickly and easily. An audience that is
confused or overwhelmed by the format will not read
the material. However, if you are consistent with
your format, you will help your reader learn how to
use your document, and will greatly increase the
chances that your materials will be read. Think about
it; how many times have you put a book or article
down, even though you know the information you
want is in there, because you just can't find that
information quickly and easily?
Where to Find More Information
Click, J. W. and R N. Baird. 1986. Magazine Editing and
Production (4th edition) Dubuque, IA: Wm. C. Brown
Publishers.
Simmons, D. A. 1983. The Use of National Park Planning
Workbooks for Citizen Participation: Enhancing Effective
Communication. Doctoral dissertation: University of
Michigan.
How to Format
Formatting can consist of almost anything, as long
as you are consistent about it, and it easy to identify.
For instance, some documents show different
chapters with different colors or with inked edges.
You can set apart titles and headings with lines and
blocks. Begin a new section on a new page. Set aside
a block on each page that provides a certain type of
information, such as a list of "things you can do" or
"where to find more information."
Be creative in the way that you format your
document. Make it neat and easy to follow, and
above all, be consistent Good formatting essentially
allows you to lead your reader by the hand through
your document.
(6 Chapter 2 Tips for Material DMign pages 38-57
-------
Format pages 46-47
Example of Formatting
i.a: lo.v
/'¦I otor oil. battery
m J t acid. gasoline, car
l» if «• wa*. engine
Ł * * cleaners, antifreeze,
degreasers. radiator flushes
and rust prcvcniauves are ex-
amples of automotive products
containing toxic chemicals.
Some car owners do their own
maintenance work: 25 percent
change their car's oil. and
many of these people pour the
used oil down the storm drain
One quart of oil can contami-
nate up to two million gallons
of drinking water The oil from
one engine—four to six quarts
—can produce an eight-acre oil
slick.
The only recommended
way to dispose of used oil is to
put it into a sturdy container,
like a plastic milk jug. and take
It tn your neighborhood garage
or oil recycling center. Mary-
land. Pennsylvania. Virginia,
and the District of Columbia all
have a number you can call to
find an oil recycling station
near your home
Disposing of antifreeze is
also a problem Antifreeze con-
tains ethylene glycol, which is
poisonous to people, fish, and
wildlife Many cats and dogs
have died after drinking sweet
tasting puddles of antifreeze
they find on driveways in the
winter.
Instead of pouring anti-
freeze down the drain or wash-
ing it into storm drains, ask
your local service station to add
the liquid to their used anti-
freeze storage drum. You can
also dilute the antifreeze and
pour the mixture into a gravel
pit or any area with good drain-
age. This method takes advan-
tage of the soil's natural filter-
ing capacity
Disposing c'Hcu&tioU
iOXlCi
TP* he kinds of household
I* toxics described in this
' chapter should not be
5! disposed of "down the
Oil con be recycled Used oil ana
onMntue should be token to vou!
local service station lot recycling
Never hose oil or ontitreese down into
your stoimdroin
f\ fWSSt, -
V'.sSrfri'i;;*
drain * YOur drain leads either
to a home septic system or a
municipal treatment plant, nei-
ther of which is designed to
completely remove toxic chem-
icals from wastewater. At least
some of the toxics pass through
the treatment process and end
up in a stream, river, or
groundwater. Read the section
in this guide on septic systems
for further cauuons.
Effective sewage treat-
ment is essential for treating
the large volume of wastewater
that comes from our homes
Improving municipal treatment
plants is a part of the strategy
to clean up Chesapeake day.
Well-run treatment plants can
remove some nutrients, or-
ganic materials, and heavy
metals from wastewater. The
chlorine used to disinfect can
also be removed by a process
called dechlorination. Individ-
uals and communities should
insist that the publicly-owm-d
treatment plants that serve
them are maintained and oper-
ated at peak efficiency. This
may mean added cost to con-
sumers. hut is essential tn good
water quality in our nvers and
Bay.
The products described in
this chapter should also never
be poured on the ground or
into gutters where they will
eventually enter storm sewers,
which generally lead directly to
a nearbv stream.
In many areas, the only
available disposal method is
the local landfill. While proba-
bly better than flushing a toxic
chemical down the drain, land
fills are not a good long-term
solution to our waste disposal
problems New energy needs to
go into rinding better soiuuons.
Where household hazard-
ous wastes must be sent to a
landfill, a couple of steps can
be taken to reduce the environ-
mental risk. First, wrap the
product in its original container
in newspaper, and then wrap
in an old plastic bag. Liquids
can be poured into containers
filled with absorbent kitty litter,
then wrapped in plastic.
Some states are dealing
with the problem of hazardous
household wastes by sponsor-
ing amnesty days On amnesty
day. small quantities of your
unwanted household chemi
cals and pesricides are col-
lected and disposed of in an
approved facility The actual
rollecrmn and disposal of the
waste should be performed by
technicians who know which
chemicals should not be mixed
together. Amnesty days are de-
signed to educate the general
public about the potential haz-
ards of improper use and dis
posal of consumer products
that contain toxic chemicals.
Check with your state or county
government to find out about
amnesty days in your area
asSiiiiSiSUeSfirtRyS^Tt
WHAT YOU' %
: -mi*?
K!2K* ssl rfciJj&sr
ere-ate .iiyvT
wroe gerv-
era! rules of
handing and dispos-
ing or household
chemicals*
• *oadlhelabflJ-^;P
know .what you
am buying and
Ł:tv> whauhe potential m
Aasaidsar* i?,- -.
(heir original con- :
tamers go the
: tenetftowhein*-
1:=:;!: e^r thepfOduct is
used.
-a Use alternative. Ł'.!j
(cm hairnMI prod- .
vets whenever '
possible (for ex-.
Sjg ample, boric acld||' :
|s very effective tn
controlling ' j
roaches)
a Use Uneieast toxic . .
¦ prod iict you can r
.'. fimland.never '
buy mote than
you need
% Mapoae.ofj^rriisitj
hold chemicals in. ¦
sanitary landfills. .
Four liquids tuch >
at cleaning fitilds
into a plasuc con-
,j, uiner that is filled
with kitty litter or
stuffed with news-
paper Allow It w
dry outdoors be-
lore taking it ff
the landfill.
• Take used motor i j$;
< oil end antifreeze
to a gas station
with an oli recy-
cling program. ...
hwist on effogive
sewage treatment
for yourcommu- >i
tiiiy.
O
11
W
o
H
r\
w
r*
U
H E M I C A L S
— from Baybaak produced by the Alliance for the Chesapeake Bay.
47
-------
Page Layout and Type
Unk with
- Use of Titles and Headings
- White Space
- Use of Graphics
- Format
Media
- Written materials
Like formatting, page layout and your choice of
type can determine whether or not your audience
will read your material. An appealing layout and
easy-to-read type will greatly increase the chances
that a person will become a reader. Crowded pages
with small type and little white space will probably
not be read.
Tips for Effective Pege Layout
Type
> Use a simple typeface, such as Roman,
whidtis the es^est to read. There should
be variations in stroke width, but no serifs,
> Use "normal" type for most of your text
rather than bold or italics. Use those only
for tides, headings, and occasionally, for
extra emphasis. People will not generally
read more than one or two lines in italics.
»- Use a medium-weight type. Avoid delicate
types especially if your document is lobe
printed on textured paper, such assome
recycled paper. Avoid heavy type espe-
cially if your material is to be produced on
a copier.
»• Use large enough type, generaliy lOjpoint
or above. If your type is too smalV people
simply won'tread it
»- Don't mix type-faces; it is distracting to the
reader.
»> Use black type rather than color type. Use
cotor type emfy for titles, headings, or
bullets.
Columns
>• Usea two- or three-column format An
easy rule-of-thumb for dhoosing line width
istochoose the type style and size you will
use, and then set your line width at no
narrower than one lowercase alphabet of
type, and ro wider than two lowercase
alphabets of type.
Other Attention-grabbers
> Use bullets.
>¦ Use boxes and side-bars.
>¦ Use shading or other methods of
highlighting.
Where to Find More Information
Gick, J. W. and R. N. Baiid. 1986. Magazine Editing and
Production (4th edition) Dubuque, IA: Wm. C. Brown
Publishers.
Wilson, K. C. 1986. "How to Create Persuasive Marketing
Tools" in C. Degen (ed) Communicators' Guide to Marketing.
New York: Longman.
48 Chapter 2 Tips for Material DMign pages 38-57
-------
Page Layout and Type pages 48-49
Example of Page Layout and Type
Maryland
VirQ!f)»a
"C
;< ^.s.
-:r!i
-R?
IV
sros
• » r* ir*'*T 1/
' -
THE ANACOSTIA AND
THE CHESAPEAKE BAY
The Chesapeake Bay la a body of
water and, like * human body. Hi
health depend* on what goes Into It.
The Chesapeake is fed by 10 major
riven and numerous imaller streams
that flow out of Virginia, Maryland,
Pennsylvania, and Delaware. These
riven bring the nutrients and fresh
water that when mixed with the
ocean's aah water, nuke Chesapeake
Bay the most productive estuary in
the United States.
But, as everyone knows, the
Chesapeake is not as healthy as it
once was The problems stem. In
part, (mm the declining quality of
the rivers that feed the Bay. This
fact sheet explains how the Anacos-
tia River enntnbutea to the Bay and
outline* some ways to help you
keep your river dean.
THE BASIN
The Anacostia proper begins at
Bladensburg, Maryland, where the
two principal branches of the Ana-
costia—the Northwest and Northeast
branches—meet. The many smaller
tributaries of the branches form a
broad, fan-shaped drainage basin of
170 square miles. Just below Bladens-
burg, the Anacostia drop* to near
tea level and changes from a free-
flowing river into a tidal freshwater
embayment of the Potomac estuary.
The Anacnstia provides a safe
harbor for many recreational boaters
in the D.C. area. Much of the shore-
line of the Anacostia is publicly
owned and there ia a stream park in
almost every portion of the baain
Within the District of Columbia's
borders arc the Kenilwofth Park
and Aquatic Gardens, Fort Dupont
Park, and Anaco*Ua Park. The Na-
tional Arboretum also Is adjacent to
the Anacostia. These natural arras
support an array of plant and
animal communities and bring a
host of outdoor opportunities
within reach of many city residents
Creol •
Ssovwdcm
Cn
Carrier oI Coukk
ANACOSTIA RMV
iano use swrtsncs
Cultivated
Ssad 4 Gravel
Upmnom
The sluggish waters of the tidal
Anacostia slow and trap sediment
and other pollutants that wash into
the river. Thus, whatever gets Into
the Anacostia stays there
What gets into a river depends a
lot on how the land around the
river Is used. Within the Anacostia
basin, land use takes many forms.
Outside the Capitol Beltway, rural
snd suburban areas dominate,
while inside the Bcltwsy the land-
scape becomes increasingly urban
Each type of land use contributes
characteristic pollutants to the
Anacostia.
ANACOSTIA:
TROUBLED WATERS
The Anacostia has been called a lot
of dirty names. Words like noxious,
foul and contaminated are frequently
applied to this main tributary of the
Potomac, a river with headwaters In
rural Prince George's and Mont-
gomery counties in Maryland and a
main channel in the densely popu-
lated sections of Washington, D.C.
The Anarostu's troubles go back
a long way. During the eighteenth
and nineteenth centuries, large por-
tions of the river's basui were cleared
for tobacco, corn, and cotton. Severe
erosion followed and sediment
accumulated in the Anacostia. (The
Anacostis was prone to sedimenta-
tion even in Its natural stale, because
of the composition of the surround-
ing soils.) By the mid- nineteenth
century, it wss impossible fur ocean-
going ships to navigate up river to the
once thriving port of Bladensburg.
Suspended sediment in the Ana-
costia limits the river's sbility to
support game fish and other aquatk
life. Other conditions, including
high levels of ammonia and harmful
bacteria, low amounts of dissolved
oxygen, and excessive amounts of
trace elements such as iron, lead,
and zinc add to the Anacostu's
share of troubles.
— from TV Anacotfia River, a fact sheet produced by the Alliance for the Chesapeake Bay.
49
-------
Use of Graphics
Link with
- What Kind of Information
- Concrete Language
Media
- fact sheets, newsletters, workshops,
speeches, video, slides
Graphics Can Be Great
Graphics tend to capture our attention. Photo-
graphs, drawings, maps, and graphs will generally
attract our eyes on a page before the words do. It is
important to attract the attention of your audience in
order to get your message across, and graphics can be
a useful way to accomplish this.
Graphics can also be emotionally appealing, thus
increasing your reader's enjoyment of your materials.
Beautiful photographs serve this function well.
Graphics also serve an informational function, by
visually simplifying your point Graphics can
increase the chances that your reader will understand
and remember the text.
But Be Careful
However, be careful when using graphics. "The
more, the better" is not necessarily true. Too many
graphics and too many colors can produce "over-
load" and overwhelm us. Complex graphs can
confuse us. Graphics can capture attention too well,
drawing attention away from the text How many
people, for example, have never actually read a
"National Geographic" article, but have only looked
at the pictures? The attention-grabbing nature of
graphics is fine if your message is conveyed in your
graphics, but is less useful if you are trying to say
something important in the text as well. Use graph-
ics... but don't overdo it A good rule-of-thumb is to
keep it simple.
Color graphics can also be expensive when
producing a written document. Black-and-white
photographs and simple line drawings can be nearly
as attractive as more elaborate, color graphics, and
are generally much cheaper to produce. You might
also consider producing your document on colored
(and, of course, recycled) paper stock instead of white
paper to make it more eye-catching.
Unusually-shaped graphics have been shown to
actually annoy readers. Make your graphics regu-
larly-shaped, such as square or rectangular, to fit
your page well.
Finally, be careful about what your graphics say!
Let graphics work for you by reinforcing your
message. If you are trying to get people to dispose of
used motor oil properly, don't show someone pour-
ing it down the storm sewer, even if the text says this
is die wrong way to do it. Instead, show a picture of a
person taking used motor oil to a garage that clearly
recycles used motor oil.
— from the The Puget Soundbook produced by the Marin* Science Society of the Padfic Noithwot Uied with pertniMiort
50 Chapter 2 Tips for Material DMign paget 38-57
-------
Tips on Using Graphics Effectively
>- Large illustrations are better than small
ones.
>• Photographs are more effective than
sketches.
>- If sketches are used, simple, clear, realistic
ones are better than cartoons or more
abstract drawings.
>- A large photo at the beginning of an article
draws the reader iri.
>- Bright colors jtftract our attention, but we
react to their "temperatures." Reds and
oranges are considered "hot" while blues
and greens are considered "cool."
>- Pictures grouped together have greater
interest than pictures scattered throughout
an article. They can also then be used to
"tell a story"
>- Graphics are especially useful for showing
"how to" type Wbnnatkm.
Where to Find More Information
Click, J. W. and G. H. Stempel, HI. 1976. "Reader Response to
Front Pages with Four-Color Halftones" journalism
Quarterly, 53:736-738.
Click, J. W. and R N. Baird. 1986. Magazine Editing and
Production (4th edition) Dubuque, IA: Wm. C. Brown
Publishers.
Simmons, D. A. 1983. The Use of National Park Planning
Workbooks for Citizen Participation: Enhancing Effective
Communication. Doctoral dissertation: University of
Michigan.
Stewart, A. 1986. "The Design of Print for Health Education
—Principles for Communication" in DS. Leather, C.B.
Hastings, K. O'Reilly, and JJC. Davies (eds.) Health
Education and the Media 11. Oxford: Pergamon Press.
Taylor, ]. G. and T. C. Daniel. 1984. "Prescribed Fire: Public
Education and Perception" Journal of Forestry. 82(6): 361-
365.
Wilson, K. C. 1986. "How to Create Persuasive Marketing
Tools" in C. Degen (ed) Communicators' Guide to Marketing.
New York: Longman.
-------
Titles and Headings
Link with
- Format
- Graphics
Channels
- written materials, speeches,
workshops
Titles
Like graphics, titles serve a very important role in
getting the attention of your audience. The purpose
of a tide is to capture your audience, to get their
attention, and to persuade them to read what follows.
There are two types of titles: descriptive titles,
which tell your reader what the article is about, and
suspenseful titles, which tease your reader and
arouse curiosity by withholding information that the
text will reveal. Descriptive titles are probably used
more frequently and are used under the assumption
that the subject of the text is interesting enough to
entice the reader.
Headings
Your audience is much more likely to read your
material if it is laid out clearly and logically, so that
the reader is able to navigate through the material.
Just as a person travelling in a strange new place
looks for landmarks and signs as clues to help
understand the environment, a reader looks for
organizational clues to help understand the structure
of the material and the types of information that are
to be found. Headings contribute greatly to this
effort, serving as "landmarks" for the reader. Your
audience is much more likely to read your material if
they can just glance at it and get an understanding of
how it is organized and what is to be found within it.
And, as we've stated before, getting your audience to
read your material is the first step in getting them to
change their behaviors.
Judging from the research that has been done, it
doesn't seem to matter whether your headings are
placed within the text or are placed to the side of the
text in the left margin. However, it has been shown
that headings help people recall, search, and retrieve
information from your material. So, by using head-
ings, you can not only gpt your audience to read your
material, you can also help them to remember and
use it And chances are, this is exactly what you are
trying to accomplish.
Creating Good Titles and Headings
>-3Make sure ttiat yourtitle Is specific ettou^i
that it could not easily fit another article.
»-1)^ a very basic typeface {such as floman);
people don't Hke fancy, unfamiliar fonts.
>~ Make your tifles and headings a little
bigger than your text so that they stand out.
> Use bokiface or italics (but do not use italics
for more than one line of text; studies have
shown that people do not like to read
italics*)
>*Setyour tides and headings off from your
text with some spa*® above and below
them. By doing this, you make your titles
and headings stand out more and you
increase the axnoontof white space on your
pag*
>• Use rhyme, alliteration, puns, figures of
speech, and other literary tricks to mate
your titles more "catchy."
>¦ Use subtitles to create a "bridge" from your
title to your text
Chapter 2 Hps for Material Design pages 38-57
-------
Example of an Eye-catching Title
and heavy metals remain in the water
for many years and end up in the fish
and shellfish we eat and the water our
families drink.
Cleaning up our polluted waters and
shoreline can cost millions of dollars
which we all pay for eventually. But by
property disposing of used motor oil you
can be part of the solution and prevent
water pollution.
HERE'S THE
GOOD NEWS!
Luckily, used motor oil newer wears
out. Dirty motor oil can be reprocessed
and then recycled and used again
because it never loses its lubricating
properties.
To recycle your used motor oil use
the attached coupon at arty of the retail
CAR.* Oil Committee companies listed
below for a discount on the purchase of
a reusable oil recycling kit The people in
each store can tell you how to collect
your used motor oil and where to take
it for recycling
— from a brochure prepared by the Change and Recycle
(CA.H) Oil Committee.
.
Titles and Heading* pages 52-53 |
Use Headings in Speeches Too
Even if your audience will be listening to a speech
or workshop, or viewing a videotape, it is important
to provide them with a "map" of your presentation
by enabling them to understand the structure of your
presentation. People should be able to identify the
headings in your presentation, even if they aren't
always spoken out loud.
Another trick to helping your audience under-
stand the organization of an oral presentation is to
hand out an outline with the headings on it Be
careful though; you don't want to have so much
writing on your outline that your audience is reading
instead of listening to your talk. Use brief phrases or
bullets—just enough to allow the audience to follow
along easily.
Where to Find More Information
Click, ]. W. and R. N. Baird. 1986. Magazine Editing and
Production (4th edition) Dubuque, IA: Wm. C. Brown
Publishers.
Kaplan, S. and R Kaplan. 1982. Cognition and Environment:
Functioning in an uncertain ivorld. New York: Praeger.
Simmons, D. A. 1983. The Use of National Park Planning
Workbooks fir Citizen Participation: Enhancing Effective
Communication. Doctoral dissertation: University of
Michigan.
Stewart, A. 1986. "The Design of Print for Health Education
—Principles for Communication" in D.S. Leather, C.B.
Hastings, K. O'Reilly, and J JC. Davies (eds.) Health
Education and the Media U. Oxford: Pergamon Press.
53
-------
White Space
Link with
- Format and Page Layout
Madia
- written materials
When preparing your newsletters, fact sheets, or
brochures, chances are that you will have a lot of
information that you want to share. But you probably
also have a page limit BEWARE! Don't just "pack it
all in" by reducing type size or margins. People are
easily turned off by a busy page that looks like it has
too much information.
"White space" refers to the empty space around
text and graphics. While it may seem like a trivial
detail (after all, what you have to say is important!)
keep in mind that the amount of white space on a
page has been shown to affect whether or not people
keep reading and whether they remember what they
have read. If you want people to read and remember
your document, make sure you have plenty of white
space on die page!
There isn't any rule-of-thumb for deciding what
the "right" amount of white space is. But since you
will most likely err on the side of too little white
space rather than too much, keep this in mind, and
make a concerted effort to create white space.
Graphic designers are generally experienced at
judging white space; consult one if you can.
Creating White Space
While judging white space is somewhat intuitive,
there are some easy ways to create more white space
on a page.
>> Usewide margins.
>> Use a two-column format
>• Leave spacearound your graphics; don't let
your text overlap your jpaphks.
>~ Use shorter words, which creates more
spaces between words.
>¦ Leave a line between paragraphs.
> Indent paragraphs, rather than blocking
them.
>~ Leave at least «ie line «Ik»v« and below
headings or titles.
>• Don't wrap text around graphics.
>- End paragraphs toward the beginning of a
new line, which creates white space by
leaving the rest of the line blank.
But be careful!
Don't create too much white space toward the
inside of your layout It tends to make your layout
seem less unified. Put much more white space
towaid the outside of your layout, for instance, by
using wide margins, to create a better feeling of space
and to unify the text
In addition, if you decide to use another paper
color besides white, be sure that there is still sufficient
contrast between the ink color and die paper color.
54 Chapter 2 Hps for Material Dasign pages 38-57
-------
White Space pages 54-55
Example of White Space
We all use Puget Sound
The quality of our lives Puget Sound's wealth of habitats provides food
dapends greatly on the and shelter for more than wildlife: humans have
quality of the environ- shared the abundance of the Sound for more than
ment In Puget Sound. 10,000 years. The Sound's first residents built
their homes, harvested shellfish, caught salmon,
hunted coastal birds and mammals and used na-
k; iŁaiii!
X . Y~-<\
'.rj
fi tive plants in ways that barely altered the natural
| order of the Sound.
1 Puget Sound continues to support a wide range
I of human endeavors. While all of these activities
I contribute greatly to the economy of the state,
they also present new challenges to Puget
I Sound.
— from Pugtl Sound: Our Heritage at Risk produced by the Paget Sound Witer Quality Authority.
Where to Find More Information
Bostian, L. R.. 1970. "Effect of Line Width on Reading Speed
and Comprehension." Journalism Quarterly. 53:328-330.
Click, J. W. and R. N. Baird. 1986. Magazine Editing and
Production (4th edition) Dubuque, IA: Wm. C. Brown
Publishers.
Smith, J. M. and M. E. McCombs. 1971. "The Graphics of
Prose" journalism Quarterly. 48:134-6.
Swartz, D. 1987. "The Newsletter" in Media Resource Guide.
Los Angeles: Foundation for American Communications
(FACS).
55
-------
A Checklist for Effective Material Design
Have you.
used concrete language by:
avoiding jargon?
^ using examples?
„— using analogies?
using descriptive adjectives and adverbs?
using active verbs?
_ avoiding technical, scientific language?
picturing what you are trying to say?
using graphics to illustrate what you are
saying?
provided useful, relevant information by:
„ explaining specifically what your
audience can do to help solve a problem,
in addition to describing the problem?
showing solutions through graphics or
presentations?
created an easy-to-follow and appealing format by:
using colors or shaded areas?
using lines or blocks?
used vivid language by :
^ telling a story?
personalizing the information to your
audience?
describing consequences in terms of an
individual orftmifly?
describing sGmething unusual or
striking?
participate?
used an tffective page layout and type by:
titles, headings, and occasional emphasis?
using a medium-weight type?
using large enough Jype?
avoiding: a mix of types?
— using a two- or three-column format?
56 Chapter 2 Tips for Material Daaign p«ge» 38-57
-------
A Checklist for Effective Material Design pages 56-57
Have You...
used graphics effectively by:
using a large picture to draw your
audience in?
using photographs?
using bright colors that evoke the right
"temperature"?
using simple line drawings?
avoiding overload?
avoiding complex graphs?
avoiding unusually-shaped graphics?
using your graphics to show what you
are saying?
created an adequate amount of white space by:
using wide margins?
, using a two- or three-column format?
leaving space around your graphics, titles
and headings?
using shorter words?
leaving a line between paragraphs?
indenting paragraphs?
used titles and headings effectively by:
creating a title that is specific?
using a basic typeface, like Roman?
making your tides and headings larger
than your text?
_ using boldface or italics?
h leaving white space around your titles
and headings?
using rhyme, alliteration, or other literary
tricks?
m using subtitles to create a bridge from
your title to your text?
constructing your oral presentation so
that headings can be clearly understood?
prepared an outline to hand out for your
oral presentation?
57
-------
Chapter 3
Introduction
Understanding the Relationship of Evaluation to the Program 60
Understanding the Types of Evaluation 62
Linking Evaluation Techniques to the Steps of the Hierarchy 63
Step 1 Decide on the Purpose of the Evaluation 64
Step 2 Identify Internal and External Constraints 66
Internal Constraints 66
External Constraints 67
Step 3 Decide at What Stages to Do Evaluation 68
External Influences 68
Cost and Evaluation Expertise 69
Evaluation Flexibility 70
Overall Effectiveness 70
Resistance to Evaluation 71
Upper End Evaluation 71
Step 4 Select a Type of Evaluation 72
Formative Evaluation 73
Process Evaluation 74
Outcome Evaluation 75
Impact Evaluation 75
Step 5 Decide What Type of Evidence You Would Like to Collect 76
Step 6 Select Evaluation Techniques 78
Step 7 Decide How Often You Will Use the Technique 80
A Checklist for Developing Your Evaluation Program 82
58
-------
Introduction
Understanding Your Evaluation
Introduction
Evaluation is as critical as the goals and objectives
of your program. You should begin formulating your
evaluation when the program begins; it is part of the
planning process. Without an evaluation system you
will be wasting precious time and money.
If you are ever hesitant to use evaluation in a
program, ask yourself the following question: would
you invest your own money knowing that you
would not get any feedback on the performance of
the investment? If your answer is "no," then why
would you bother to spend thousands of dollars and
many hours working on an outreach program
without knowing whether it has achieved its goals
and objectives?
Unfortunately most projects do not use evaluation.
Those that do use evaluation often make many
misleading assumptions.
Others think that evaluation is expensive and
requires much expertise. Sometimes this is true, but
there are easier and less expensive methods that can
be used. If you understand the tradeoffs between
using less complicated evaluation methods, then it is
easier to make a decision and justify your choices.
This section will give you a better understanding
of evaluation and its role in your program and it will
explain the four types of evaluation. Also, you can
expect to learn about the trade-offs that occur when
you select different evaluation methods. Keep in
mind that Chapter 3 presents an overview of evalua-
tion. Please consult the references listed at the end of
the chapter before you begin the evaluatioa In the
next chapter, you will be exposed to specific evalua-
tion techniques.
For Example
A $39,350 grant was used to explain wetland
functions to real estate developers. The success of the
project was based on requests for workbooks, slide
shows or additional information and evaluation
forms from workshops. Although these are good
components of evaluation, die requests for informa-
tion cannot be directly related to a developer's
understanding of the material or a change in attitude
or behavior.
59
-------
Understanding the Relationship of
Evaluation to the Program
Before deciding on the evaluation techniques you
want to use, it is important to think about the compo-
nents of most outreach programs. Typically, pro-
grams follow a natural flow of development. The end
results of the programs are often obtained if the
following events occur:
>- End results are the ultimate objectives/ and
are reached by„.
> Changes in behavior, which result from.-
> Changes in knowledge, attitudes, skills
and/or aspirations (KASA) influenced
by,..
>¦ Reactions to the program, which are a
functioned.
> Taiget audience involvement, which
happens due to the,,.
>- Educational activities offered as a result
of...
>- Inputs into the program of time and other
resources.
This list, called "Bennett's Hierarchy of Evidence
for Program Evaluation," is a good description of
how many programs should transpire. The same list
is exhibited below in a summary format.
End Results
U Behavior Change
H KASA Change
U Reactions ~
St Target Audience Involvement |
It is important that you understand what is
included in each of the Hierachy's components. The
following descriptions give a brief summary of each
step.
1 Inputs - Program inputs are the resources
that are used to make the program work.
They could include money, staff, volunteers, office
space and supplies.
2 Activities - Activities can include internal
events such as planning and the external
events that involve the target audience.
3 Target Audience Involvement - The target
audience should be involved in the program
activities. They could participate in a survey, read an
article or do water sampling.
4 Reactions - The target audience members will
have some type of reaction to the activities.
Reactions could range from "I really don't care about
your program!" to "That was interesting; how do I
find out more?"
5 KASA Change - The program may change
either knowledge, attitudes, skills or aspira-
tions of the participants. For example, a member of
the target audience may learn how to recycle motor
oil safely.
6 Changes in Behavior - Hopefully, as a result
of the activities, there will be some sort of
change in the behavior. For example, participants are
now recycling their oil instead of pouring it into the
sewer.
7 End Result - Results should closely reflect the
program's overall goals and objectives
developed when the program was planned.
Activities
J Inputs
60 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Understanding the Relationship of Evaluation to the Program pages 60-61
Try to think of the components of your program
that would fit into these steps and then list each
component on a worksheet. With this knowledge you
can begin to think about the types of evaluation that
you will use to evaluate the different steps of the
program.
Hierarchy Steps
Your Program Components
7. End Results
6. Behavior Change
5. KASA Change
4. Reactions
3. Target Audience
2. Activities
1. Inputs
61
-------
Understanding the Types of Evaluation
Now that you have:
Related your program components to
Bennett's Hierarchy;
it is imperative that you learn about the four
types of evaluation, each of them focusing on
unique parts of the program.
Evaluations can be done at the beginning or end of
a program, as well as anywhere between these
points. Evaluation methods have been designed for
each of these periods. The four types of evaluation
and examples from the Puget Sound (Washington)
program follow.
Formative Evaluation (before) Outcome Evaluation (afterwards)
A formative evaluation is undertaken to test
materials and ideas and to understand target
audiences before a project is started.
Example
Materials for an elementary-school wetlands
program were selected by an advisory group of
experienced teachers.
Short-term results of the program can be mea-
sured using outcome evaluation.
Example
An evaluation form was attached to a handbook
produced about the Skagit River Watershed. The
questionnaire asked questions about behavior
change and a general understanding about the
handbook.
Process Evaluation (during)
Using process evaluation, you can monitor
activities of the program and provide timely infor-
mation to staff and sponsors. Process evaluation
allows you to change your program in response to
feedback.
The City of Bellevue, Washington, wanted to
reduce nonpoint pollution by developing workshops
that targeted businesses. They monitored the
number of businesses attending workshops and
requests for technical assistance.
Impact Evaluation (much later)
Longer-term results may vary from the short-
term, so impact evaluation can be used to measure
the ultimate outcome of the program.
Example
An education program designed to educate septic
system pumpers was established by the Thurston
County Health Department To measure the success
of a certification course, the County continuously
monitors the number of reports of illegal sewage
dumping and improperly pumped tanks.
62 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Understanding the Types of Evaluation pages 62 - 63
Linking Evaluation Techniques to the Steps of the Hierarchy
Now we can add another column to the previous planning worksheet. Vou must
decide which evaluation techniques will be used at various steps in the program.
However, this is not an easy step in the process, because there may be more than one
evaluation method at any given step and there are tradeoffs when deciding when to
do evaluation.
Hierarchy Steps
Your Program Components
Type of Evaluation Selected
7. End Results
6. Behavior Change
5. KASA Change
4. Reactions
3. Target Audience
2. Activities
1. Inputs
Since this can be a very complicated process, the following table can be used as a
general guide. The table shows which types of evaluation are appropriate for each
step in the hierarchy.
Type of Evaluation
Hierarchy Steps
Formative
Process
Outcome
Impact
7. End Results
X
X
X
6. Behavior Change
X
X
X
5. KASA Change
X
X
X
4. Reactions
X
X
X
3. Target Audience
X
X
X
2. Activities
X
X
1. Inputs
X
X
Once you know the possible types of evaluation, you can begin the more compli-
cated process of selecting specific techniques. In the next section, you will learn about
the specific steps that must be taken to develop an evaluation. Also you will be
exposed to several of the tradeoffs that exist when you are selecting a technique.
63
-------
Step 1 of Developing Your Evaluation
Now that you have:
Related your program components to
Bennett's Hierarchy; and
learned about the four types of evaluation;
the purpose of the evaluation must be defined.
This will allow you to begin planning which
types of evaluation will be used in your
program.
Decide on the purpose of the evaluation.
Usually the criterion for this decision is based on
the program's goals and objectives. First, turn each of
the objectives into questions. To answer the ques-
tions, decide upon a type of evaluation. Finally,
specific evaluation techniques must be selected to
gather the evidence.
Here is an example of how to apply this to your
project
Program Goal
Protect surface and groundwater quality by
developing programs that influence the target
audiences.
chapter 3 Understanding Your Evaluation pages 58-83
-------
Step 1 of Developing Your Evaluation pages 64-65
This goal could result in the following objectives at
each level in the hierarchy.
Hierarchy Steps
Program Objectives
7. End Results
Improve the average water quality levels in the watershed by 4%, before
June 1( 1993.
6. Behavior Change
Encourage the target audience to increase the volume of used motor oil
that is recycled, at the three recycling centers, by 50% before June 1,1993.
5. KASA Change
Ensure that 20% of the target population has the knowledge needed to
dispose of used motor oil by April 1,1993.
4. Reactions
Eighty percent of the members in the target audience should think that the
program brochures and presentations are understandable by the end of the
first month of activities.
3. Target Audience
At least 60% of the program participants should exhibit the characteristics
of the target audience by the end of the first month of the program.
2. Activities
Staff will decide, through market research, who is most likely to change his
or her own motor oil at home and how many people are in this group. This
will occur by September 1,1990.
1. Inputs
Staff members will recruit two volunteers that can contribute 300 hours
over the life of the project, before July 1,1990.
For this example we will focus on step six of the
Hierarchy. Perhaps your program wants to change
the behavior of people who perform maintenance on
their cars at home. The program could seek to ensure
that used motor oil is properly disposed at collection
centers that recycle the oil.
Next, turn the objective into a question and
develop the framework for the evaluation:
Hierarchy Step
Program Objective
Objective as a Question
6. Behavior Change
Encourage the target audience to
increase the volume of used motor
oil that is recycled, at the three
recycling centers, by 50% before
June 1,1993.
Has the volume of used motor oil
increased by 50% at the designated
centers before the deadline?
Did the target audience cause the
increase of 50%?
65
-------
Step 2 of Developing Your Evaluation
Now that you have:
Related your program components to
Bennett's Hierarchy;
Learned about the four types of evaluation;
Defined the purpose of the evaluation; and
Turned your objectives Into questions that
must be answered in the evaluation;
you are ready to go on to Step 2.
Step 2
Identify internal and external constraints.
Many internal and external factors will influence
your ability to select evaluation techniques. These
constraints must be identified and their impact on the
evaluation must be understood. These internal and
external factors will affect:
1 At what stage of the program an evaluation
will occur.
2 What techniques will be used to complete
the evaluation.
Next, ask yourself several questions that will help
you evaluate the internal and external issues affecting
your evaluation.
Internal Constraints
»- How much money is available to complete
an evaluation?
> Does the staff have any experience with
doing evaluations?
>- Are experts available and is money present
to pay for these services?
»- Are there enough computers and support
staff to manage the information collected
through evaluation?
»* Do you need quick and regular feedback
from the evaluation?
> What is the time frame for the program?
> Has it been difficult to develop common
goats and objectives (hat everyone could
agree upon?
> Would the full impact of the program be
noticed before die funding ends?
> Can the evaluation method be applied if
the program changes?
>¦ Is the purpose of the evaluation to focus on
program improvement while the program
is going on?
> Is the evaluation designed to learn about
the overall effectiveness of the program?
»- Does die Staff or management feel threat-
ened by having their program evaluated?
>• Are the staff and management prepared to
make program dhanges based upon
evaluation results?
»- Are organizational policies in place that
promote program change based upon
critical feedback?
»¦ What w01 be done with the results of the
evaluation?
> Will the results be used by program staff to
improve the program?
66 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Stap 2 of Developing Your Evaluation pages 66-67 |
External Constraints
>¦ Are there external factors that could affect
the results of the evaluation?
>- Is fiie program accountable to the public,
other organizations or to funding sources?
»¦ Will the results be used by contributors or
management to decide whether the
program achieved the expected outcomes?
>• Are you dependent on external parties to
complete the evaluation?
Answers to these important questions will help
you decide at what levels of Bennett's Hierarchy you
should evaluate. Information included in the follow-
ing step will assist you with this decision.
67
-------
Step 3 of Developing Your Evaluation
Now that you have:
Related your program components to
Bennett's Hierarchy;
Learned about the four types of evaluation;
Defined the purpose of the evaluation;
Turned your objectives into questions; and
Identified internal and externa) constraints;
you are ready to go on.
Decide at what stages of the program
you want to do evaluation based upon
the internal and external constraints.
Since your organization's resources may be
limited, this is a critical decision. Of course, there are
tradeoffs when deciding when to do the evaluation,
but if you have good information you can make
intelligent decisions. Also, your organization can
explain the weaknesses and strengths of selecting
various techniques to sponsors and critics that are
interested in your program.
The points included in this section are not de-
signed to discourage you from selecting steps to
evaluate. Instead, they are included to help you plan
and make rational decisions. Just think of these
comments as words of wisdom. They could save you
time, money and frustration. You will be able to
avoid mistakes and false starts if you rely on these
ideas. However, there are exceptions to every one of
these rules, so if you are certain that it does not apply
to your program, then please, disregard it!
External Influences
The higher up the hierarchy that you go, the
greater probability that the program results will be
influenced by external factors. Therefore, it is more
difficult to design evaluations that can deflect those
problems.
End Results
|3 Bahai/for Change"
m kasa Change
! Reactions
Target Audience Involvement
|{| Activities
P Inputs ~
Time
The world does not stop as your program
progresses. Hie ultimate results of the program may
be influenced by many factors along the way. For
example, people ate becoming more aware of
environmental issues because of news coverage etc.,
so they may change their behavior or receive new
knowledge even if the program did not exist. Also,
local zoning ordinances or state and federal laws
could change, requiring higher levels of water
quality. Again these changes could influence any of
the expected outcomes in the program. Finally, it is
hard to control who enters the program, when they
enter and when they decide to drop out of the
program. Therefore, it is often difficult to associate
results, which may occur long after program activi-
ties, with the efforts of the program.
66 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Cost and Evaluation Expertise
As you move up the hierarchy, it is usually more
expensive to collect data and requires more time to
obtain results. Also, more expertise is required to
design evaluations, analyze data and provide
feedback.
End Results
Behavior Change
KASA Change
Reactions
Target Audience Involvement
Activities
Inputs
Time
0
«
'€
&
UJ
e
.2
%
3
"3
>
UJ
End Results
Behavior Change
| KASA Change
Reactions
Target Audience Involvement
Activities
Inputs
Time
Evaluating higher levels, such as program out-
comes, can be much more difficult than evaluating
lower levels of the Hierarchy. This is usually true
because the results are often far removed from the
program activities. The sources of evidence linking
die program activities to the results are scattered.
Sometimes it is not clear whether the program
activities caused the results. Evaluation techniques
require more expertise to separate external influences
from the actual program. Therefore, the cost of
collecting data increases.
-------
Step 3 of Developing Your Evaluation pages 68-71
Evaluation Flexibility
The program is likely to change as it develops.
Program evaluation techniques that are selected for
higher levels of the hierarchy must be flexible enough
to respond to change.
Your program will start off with a master plan, but
those plans may have to change. If you select an
evaluation technique that is rigid, you may have
trouble using the approach. Simple studies that
question the before-and-after results of a program
may not be useful if the program content changes.
Many evaluation techniques are not flexible enough
for change. Often they require systematic compari-
sons of the conditions before the program began to
determine the impacts of the program.
Ł
jo
J
kS
End Results
Z]
I Behavior Change
H KASA Change^
Reactions
~
Target Audience Involvement
U Activities
HI lnpm«
Change from Original Plana
Overall Effectiveness
If the main focus of the evaluation is to increase
the program's performance, it is important to apply g 1
more evaluation techniques at die lower levels of the c
hierarchy. Overall program effectiveness requires Ł
evaluations to be conducted at die upper levels of the
hierarchy. uj
"5
If you are going to improve a program, the «
evaluation information must be collected early in die 5
program. Changes can then be made to die program
and results monitored. However, if the focus is to
H End Results
El Behavior Change
El KASA Change
El Reactions
El Target Audience Involvement
H Activities
El Inputs
evaluate whether die overall program was effective,
then behavior change or end results may be more
important Of course, a program could include both
objectives.
Improve Program Performance
70 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Step 3 of Developing Your Evaluation pages 68-71
Resistance to Evaluation
Selecting techniques to evaluate the lower end of
: the hierarchy would be better if your staff or manage-
; ment is resistant to change or feedback.
End Results
Behavior Change
KASA Change
Reactions
Target Audience Involvement
Activities
Inputs
Resistance to Program Feedback
Many organizational structures seem to be
surrounded by concrete. Although they may try a
new program, often they do not change their ways.
Their organizations are not motivated to improve or
make changes. This can be caused by upper manage-
ment or by lower level employees. Or it can be
caused by policies, rules or regulations. Most evalua-
tion activities that focus on the higher levels of the
Hierarchy are more involved and require greater
levels of commitment by the organization. Staff may
resent these activities and ignore the results.
In contrast, evaluation at lower levels is often less
threatening and can involve the staff and manage
ment in positive ways. Usually these evaluations
require fewer resources and occur over a shorter
period. Also, they usually require less technical
knowledge.
Additional benefits of this type of approach are:
>- the staff can accept ownership for the ideas;
>- opponents will discount their impact
because they are small changes;
>¦ many of these evaluation tools require less
change in employee routine;and
>- large program improvements can come
from small changes.
Upper End Evaluation
If your organization is accountable to external
sources, there may be more concern with evaluation
at the upper end of the hierarchy. However, inter-
nally, evaluating the lower end of the hierarchy often
has a greater impact on the success of the program.
End Results
S ~ Behavior Change
E
3
O
o
KASA Change
Reactions
| H Target Audience Involvement
T: E3 Activities
Inputs
External Accountability
Evaluations that pretest ideas and messages are
most important internally. These methods can
provide the feedback that allows the program
managers to make appropriate changes. In contrast,
forces outside the organization are more likely to
want information about the end result. It is important
to balance these goals and ensure that there are
evaluations occurring at levels of the program that
are beneficial to those inside and outside the organi-
zation.
71
-------
Step 4 of Developing Your Evaluation
Now that you have:
Related your program components to
Bennett's Hierarchy;
Learned about the four types of evaluation;
Defined the purpose of the evaluation;
Turned your objectives Into questions;
Identified internal and external constraints;
and
Considered the tradeoffs Involved with
deciding what levels of the Hierarchy to
evaluate;
you are ready to continue.
Step 4
Select a type of evaluation to be used at
the different levels of the hierarchy.
Based upon the previous section, you should have
some idea of the tradeoffs of conducting evaluation at
different stages in the program. Now you need to
decide what types of evaluations you want to
complete. As mentioned at the beginning of this
section, there are four categories of evaluation. Each
type has unique applications at different steps of the
program.
Overall evaluation results will be improved by
doing evaluations at several different points in the
program and by applying more than one type of
evaluation. For example, it would be better to apply
formative and process evaluation instead of only
using process evaluation.
Type of Evaluation
Hierarchy Steps
Formative
Process
Outcome
Impact
7. End Results
X
X
X
6. Behavior Change
X
X
X
5. KASA Change
X
X
X
4. Reactions
X
X
X
3. Target Audience
X
X
X
2. Activities
x
X
1. Inputs
X
X
72 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Step 4 of Developing Your Evaluation pages 72 - 75
Formative Evaluation
Formative evaluation involves pretesting ideas
and/or messages or other program materials. There
are many techniques that can be used at this level of
program development
1 - Confirming Target Audience
Formative evaluation techniques may be used to
confirm that the target audience was defined prop-
erly and to find out more about the audience. Impor-
tant subgroups can be identified and their character-
istics confirmed. Review the target audience section
of chapter 1 to find out more.
2 - Concept development
The staff of the organization may have some ideas
for developing a program. Also, perhaps they have a
good understanding of the target audience. But often
it is still difficult to narrow down the choices. An
example of some choices are listed below:
>• Should there be a spokesperson used?
>- Should the appeal be based on humor, fear
or factual information?
»• What channels should be used to distribute
the message?
»• Should one message or several messages be
developed?
Using formative research can result in new ideas
or more refined ideas from the target audience.
3 - Pretesting Materials
Once specific materials have been produced it is
essential to test their appeal to the target audience.
There is no reason Id print and distribute 100,000
brochures unless you know that it meets the
program's goals and objectives. Pretesting is manda-
tory if you are really concerned about achieving
results. The factors that should be evaluated in
pretesting include:
1 Attention and Recall - If the audience is
attracted to the message and remembers the
content, it will be more effective.
2 Comprehension - The messages must be
understood by the audience before they can
respond.
3 Personal Relevance - People will be more
interested in a message if they feel like it was
designed specifically for them.
4 Sensitive or Controversial - Is the message
considered offensive by certain segments of
the audience? (However, sometimes controversial
messages may have positive benefits.)
5 Strong and Weak Points - There may be
certain parts of a message that mean more to
the audience. Also, parts of die message could be
confusing or misinterpreted.
6 Attractive - Is the message visually
appealing? Does it attract your audience's
attention?
Examples of Formative Evaluation
When the Metropolitan Park District of Tacoma,
Washington, produced materials for a wetlands
curriculum, they conducted field test with students
before a final design was selected.
A program designed to educate theater profes-
sional about local and global ecology issues pro-
duced a handbook. The handbook discussed ways to
reduce and control toxic and hazardous materials
and waste. The publication was written in dose
cooperation with theater personnel. During develop-
ment, theater personnel were regularly asked about
the clarity, usefulness and thoroughness of the
handbook.
73
-------
Step 4 of Developing Your Evaluation pages 72 - 75
Process Evaluation
Process evaluation occurs during your program; it
is an evaluation of the ongoing process. It can
encompass every aspect of program evaluation from
the development of the goals and objectives, until the
last evaluation report is presented. It provides the
links to bring the program together and monitor all
the activities. It can summarize everything from the
documentation of events to participant surveys.
However, it does not have to include everything. It
can include only those things important to program
sponsors. There are three main emphases of process
evaluation.
1 Monitoring interactions with the target
audience.
2 Monitoring the surroundings within whkh
to the program operates.
3 Monitoring the organization activities.
The information collected from a process evalua-
tion is usually recorded through a record-keeping
system. It can be done manually, but it is better to use
a personal computer and a popular database or
spreadsheet program. Using this system, data can be
combined, cross-referenced and analyzed in many
different combinations.
Examples of Process Evaluation
The City of Bellevue, Washington, presented a
series of water quality workshops to the business
community. They monitored the number of busi-
nesses attending workshops and requests for
technical assistance.
A brief report, traveling mural and slide show
was produced on the condition of the Dungeness
River. The project staff tracked the number of reports
distributed and presentations given.
An environmental park education program was
established in South Mason County. To monitor the
program, the number of people using the park were
tracked and the volume of litter was monitored.
Media coverage of program activities were tracked.
What can process evaluation accomplish?
Ilt monitors the program activities by record-
ing activities systematically. These can
include such things as how many hours are spent by
staff on different tasks, goals and objectives. Also, it
can include information about interactions with the
target audience, financial information or any other
aspect of the program. It can be as simple as written
records summarizing decisions and program steps or
as complicated as a large computer database.
2 These documented activities can serve many
purposes. The ultimate result is to inform,
change ineffective approaches and affirm effective
approaches to the staff and supporters of the pro-
gram.
3 Based upon timely information, changes can
be made in the strategies used to develop
messages and to carry out programs. The immediate
impact of the changes can then be evaluated.
4 The information collected allows the program
managers to evaluate cost effectiveness at
every stage of the program.
5 Once information systems are developed,
they can provide those that fund the program
and other external supporters with timely feedback
on the program's progress.
6 If the program is judged successful, it can be
replicated easily by others. This is possible
because information that describes how the program
was operated is recorded.
7 In an environment where program staff are
working at a fast pace, it is important to have
a method of ensuring that everyone is aware of the
program and sees the "big picture." Process evalua-
tion provides a formal channel of communication
between program staff and can ensure that everyone
is exposed to program information.
In summary, the list of items that can be tracked
using process evaluation may seem overwhelming.
However, it should be noted that you are not re-
quired to monitor everything. Select the items that
are important to your program. Also, be innovative
and creative; develop other items to track that meet
die needs of your organization.
74 Chapter 3 Undaratanding Your Evaluation pages 58-83
-------
Step 4 of Developing Your Evaluation pages 12 - 75
Outcome Evaluation
No evaluation is complete unless there is an
emphasis on the outcomes of the program. Outcome
evaluation is used to measure the short-term results
of the program as opposed to long-term changes.
There are several issues to consider when using
outcome evaluation.
Ilt could be applied to measure changes in
knowledge, attitudes, awareness, skills,
aspirations or behavior. Usually the time frame for
this monitoring begins after the initial contact with
the target audience and can continue for sue months.
2 Does a particular outreach program work
with everyone or does it work best with a
certain audience?
3 Does the change occur after the first time a
person is exposed to the program's compo-
nents or does it take several exposures? Or do certain
components of the program work after one exposure
and others take repeated exposures?
4 Will the changes go beyond the desired
effect? Do the program participants experi-
ence changes in other areas of their lives?
Factors to Evaluate
The most common type of outcome evaluation
compares the before-and-after results of a project
Usually this involves comparing data collected before
the project begins to data collected after the program
ends. When possible, it is very important to select
appropriate outcome indicators that truly reflect the
goals and objectives of the project. Sometimes this is
difficult due to the constraints of time and money, so
it is critical that you select the best possible outcome
indicators available.
If the program changes greatly during implemen-
tation, it may not be possible to rely on a before-and-
after study. Often this occurs with new programs that
are innovative. Data collected before the program
begins may not be relevant to the results. In this case it is
appropriate to rely on studies that just focus on results.
Examples of Outcome Evaluation
The Metropolitan Park District of Tacoma,
Washington, produced a homeowners watershed
protection guide. To measure the results, the staff
surveyed those that received the guide to determine
their response. Also, inflows of phosphorous and
nitrogen were monitored in the watershed and
compared to baseline data.
A mobile interpretive center was used to educate
farmers and other groups in the Boundary Bay
region of Washington. To determine the success of
the project, questionnaires were circulated to groups
exposed to the program.
The City of Olympia, Washington, conducted
family day camps and seminars to reduce nonpoint
source pollution. Day camp participants received
pre- and post-surveys to measure the change in
knowledge, attitudes and behaviors. Also, they
conducted a survey after a six-month interval
through a mailed questionnaire.
Impact Evaluation
Impact evaluation is the most difficult type of
evaluation to complete. It measures the long-term
impacts of programs.
The durability of the program's goals and
objectives is very important Does the program have
to reinforce the change often or does it work over
long periods of time? Also, what is the degree of the
change? Does the quality of the change decrease over
time or does it actually improve? This is a formidable
task to accomplish, since many factors can influence
the results. Impact evaluation must be designed to
filter a host of external variables that could sway the
outcome of the program.
For example, let us assume that your program was
designed to protect water quality. An evaluation
would have to be designed very carefully. If water
quality measurements, made one year after die
program ended, were used as an indicator of success,
changes in land use, industry, or environmental laws
could have induced the change. Usually this type of
evaluation requires the help of a trained professional
and it often involves social science measurement
techniques. Therefore, this type of evaluation will not
be discussed any further in the handbook.
75
-------
Step 5 of Developing Your Evaluation
Now that you have:
Related your program components to
Dennett's Hierarchy;
___ Learned about the four types of evaluation;
Defined the purpose of the evaluation;
Turned your objectives into questions;
Identified internal and external constraints;
Considered the tradeoffs involved with
deciding what levels of the Hierarchy to
evaluate; and
5eiected the type of evaluation to use at
each level erf the Hierarchy;
you are ready to continue.
Decide what type of evidence you would
like to collect.
By turning objectives into questions you can
decide upon the types of evidence that are available.
But before you can evaluate the evidence that can be
collected, it is important to record your alternatives
using the worksheet shown previously. This can be
accomplished by listing the objectives that were
turned to questions and then recording sources of
evidence that could be used to collect the data.
Once you have listed some sources of evidence, it
is important to evaluate the evidence. Often there are
many types of evidence that can be collected during
evaluations. For example, there could be statistically-
valid studies completed based on professionally-
developed methodology, which can be very expen-
sive. Or you can just ask the staff to record their
observations about the program outcomes, which
costs only staff time and a few pennies for some
paper. Deciding between these types of choices is not
always easy.
Hierarchy Step
Objective as a Question
Evidence of Outcome
6. Behavior Change
Has die volume of used motor oil
Record the voltim# of motor oil
increased by 50% at the designated
collected weekly and oompare to
centers before the deadline?
past collections.
Did die target audience cause the
Must know whether die people
increase of 50%?
who brought theoil to the center
meet the criteria of the target
audience.
76 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Step 5 of Developing Your Evaluation pages 76 - 77
Evaluations that are based upon valid information
are more useful in decision-making.
Some evaluation methods may answer your
questions, but they may not reflect the true opinions,
attitudes and perspectives of the people who were
evaluated. For example, if you observed a group of
participants in an activity, you might draw conclu-
sions that could be included in an evaluation.
However, if you asked the people what they thought
about the activity, their responses may be very
different from your conclusions. In a sense, the
survey would be "hard data" and the observation
would be "soft data."
Even with the survey, there are many factors that
could influence the results, such as
1 The number of people surveyed could be
too small.
2 The survey answers may not be reliable
because several people asked questions in
slightly different ways.
3 Respondents are usually aware that the
program is being evaluated, so that may
influence their answers.
However, surveys can be designed to avoid many
of these problems. You may wish to consult someone
experienced in survey design.
Generally, a sample of people are surveyed,
instead of interacting with every possible member of
the target audience. Therefore, it is important to ask
whether the results of evaluation represent the target
audience. Someone involved with your program or
an evaluation consultant needs to make this judg-
ment Usually there is not a simple answer, but it is
important to consider. You should carefully weigh
the pros and cons of selecting different methods of
collecting data from the target audience.
Hard data is generally more "comfortable" when
used to make program decisions. However, hard
data is usually more expensive and time-consuming
to acquire, especially at the upper end of Bennett's
Hierarchy.
Often evaluation programs must rely on many
forms of data; some are objective and quantified and
others are not.
Examples of hard data include;
>- The amount of money spent to print
brochures.
>- The number of brochures that were handed
out at an activity.
>• The change in dissolved oxygen levels in a stream.
>¦ Results of a statistically-valid random
sample survey on knowledge and attitudes.
>• Observation of an increase in the volume of
motor oil recycled.
In contrast, soft data examples could include:
>- Staff recording their own observations
about the audience response to a program.
»¦ Asking the staff, or several experts, their
opinions about how the target audience
might respond to an educational program.
>- Relying on information tracked in newspa-
per stories to develop new programs.
>- Using a small group of people from the target
audience to evaluate educational materials.
Often hard data is too difficult to collect and
consumes too much time and money. Therefore,
many evaluation efforts with small budgets must rely
on soft data. In fact, many examples of evaluation
techniques in this publication will be from the soft
data category. Soft data can give you an unique
perspective on the program. However, some deci-
sions may warrant hard data. For example, if your
program is going to be duplicated on a larger scale in
several areas around the county, then it would be a
good idea to spend the extra time and money to gather
hard data. If the program was really ineffective or
had some major flaws, it would be very important to
identify those flaws before they were repeated.
Evaluation information that truly reflects the target
audience is more useful in decision-making.
77
-------
Step 6 of Developing Your Evaluation
Now that you have-.
Related your program components to
Bennett's Hierarchy;
Learned about the four types of evaluation;
, Defined the purpose of the evaluation;
Turned your objectives into questions;
Identified interna! and external constraints;
Considered the tradeoffs involved with
deciding what levels of the Hierardiyto
evaluate;
Selected the type of evaluation to use at
each level of the Hierarchy; and
Decided to collect hard or soft data (or both);
you are ready to continue.
Select evaluation techniques to use to
collect evidence.
Specific evaluation techniques will be discussed in
the next chapter. However, to help plan this process,
you can take the previous worksheet and add
another column. The third column will list the
evaluation method that could be used to collect the
evidence listed in the second column. The worksheet
allows you to select evaluation techniques early in
the planning process when goals and objectives are
formulated. This allows the staff to evaluate the pros
and cons of the project early in the planning stages
before the program begins.
78 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Step 6 of Developing Your Evaluation pages 78 - 79
Hierarchy Step Objective as a Question Evidence of Outcome Evaluation Techniques
6. Behavior Change
Has the volume of used
motor oil increased by 50%
at the designated centers
before the deadline?
Did the target audience
cause the increase of 50%?
Record the volume of
motor oil collected weekly
and compare to past
collections.
Must know whether the
people who brought the oil
to the center meet the
criteria of the target
audience.
Review the invoices from
the vendor that collects oil
and note the volume
collected.
Collect information from
everyone that drops off oil
(or collect information on a
sample basis.) Decide
whether they are in the
target audience and
whether they are recycling
more oil than before the
program began.
Now it is time to turn your own objectives into
questions. Decide what type of evidence to collect
and then read chapter 4 to gain an understanding of
evaluation techniques.
79
-------
Step 7 of Developing Your Evaluation
Now that you have:
Related your program components to
Bennett's Hierarchy;
Learned about die four types of evaluation;
_ Defined the purpose of the evaluation;
Turned your objectives into questions;
Identified Internal and external constraints;
Considered the tradeoffs involved with
deciding what levels of the Hierarchy to
: ¦ -evaluate;: -
Selected the type of evaluation to use at
each level of the Hierarchy; and
Decided to collect hard or soft data {or both);
and
Started fo coraider evaluation techniques;
you are ready to continue.
Determine how you will use the
evaluation technique.
One more topic must be discused before you can
begin using the evaluation techniques. It is important
to understand how the techniques can be used. There
are three common ways of employing the techniques
presented in this handbook.
1 Before-and-After
2 Single Measure
3 Repeated Measures
Before you can decide on a technqiue you must
decide how it will be used in your evaluation.
Before-and-After Studies
The title of this approach simply explains its
function. Data is collected before the outreach efforts
begin and is compared to the information collected
after the program or steps in the program are com-
pleted. For example, a program was developed to
educate children about wetlands. To evaluate the
program, the children were asked to draw a wetland
before the program began. After the education was
completed they were asked to again draw pictures of
a wetland. In comparison, the drawings were
remarkably different; the drawing completed
afterwards included a lot more detail about wetlands.
This approach can be applied in the formative,
process and outcome stages of the evaluation. An
outcome evaluation that did not complete a before-
and-after study would not be credible. Also, it may
be used with other techniques that are not included
in this handbook. For example, you could monitor
the chemical characteristics of stream water quality
before and after a program. The innate weakness to
this approach is often that it is difficult to separate
out program effects from outside influences. If there
is a long time lag between the before-and-after
studies this can be a very serious concern.
Repeated Measures
With this method, information is collected at
various points throughout the program. By recording
information at different points in time, the staff can
begin to understand how the program is affecting the
target audience. Process evaluation is best suited to
this approach, but it can also be applied with forma-
tive and outcome evaluation. Many of the techniques
discussed in this handbook will allow you to carry
out this method of study. For example, each month a
small number of questionnaires could be mailed to
participants. Over time, changes might be seen in the
responses. The changes could be compared to
program components and assumptioms made about
the results.
80 Chapter 3 Understanding Your Evaluation pages 58-83
-------
Stop 7 of Developing Your Evaluation pages 80-81 |
Single Use
Some evaluation techniques may only be used
once during the program. Techniques that fall under
the formative type of evaluation are most likely to be
used only once. Fbr example, the signal stopping
technique may be used to improve the writing in a
brochure. Once the test is completed, and die bro-
chure is modified to include feedback, die technique
may not be used again. In process and outcome
evaluation it is more important to employ an evalua-
tion technique more than once.
81
-------
A Checklist for Developing Your Evaluation Program
Step 3 of Developing Your Evaluation
Decide at what stages of the program you want to do
evaluation based upon the internal and external
constraints.
Are you familiar with the tradeoffs that
occur with evaluation at different stages
of the program?
Have you decided what issues are most
important to your organization when
makir^tradeoffs?
Step 2 of Developing Your Evaluation
Identify internal and external constraints.
Did you take the time to gather answers
to the questions about the internal and
external evaluation constraints?
How wiU these constraints affect your
evaluation design?
Step 1 of Developing Your Evaluation
Decide on the purpose of the evaluation.
Have you decided on the purpose of
your evaluation?
Were objectives placed in Bennett's
Hierachy?
Did you develop articulate questions that
spell out what you are trying to evaluate?
Understanding the Relationship of
Evaluation to the Program
Do you understand the components of Bennett's
Hierarchy?
Have you related components of your
program to Bennett's Hierachy?
Step 4 of Developing Your Evaluation
Select a type of evaluation to be used at the different
levels of the hierarchy.
Do you have a good understanding of
the four types of evaluation available and
have you made appropriate selections?
Step 5 of Developing Your Evaluation
Decide what type of evidence you would like to collect.
_ Do you understand how the quality of
evidence can impact your evaluation?
Can you justified your choices of
evidence?
Step 6 of Developing Your Evaluation
Select evaluation techniques to use to collect evidence.
— Have you wad chapter 4 to gain an
undemanding of selected evaluation
techniques?
Which evaluation techniques will you
use and why have you choosenthem?
Are you aware erf the good and bad
points about these techniques?
Step 7 of Developing Your Evaluation
Decide how often you should use the evaluation
technique.
Have you decided when the evaluation
techniques will be used?
82 Chapter 3 Understanding Your Evaluation pages 58- 83
-------
A Checklist tor Developing Your Evaluation Program pages 82 - 83
Where to Find More Information
Alkin, M. 1985. A Guide for Evaluation Decision Makers.
Newbury Park, CA: Sage Publications, Inc.
Atkin C., Rice, R. Eds. 1989. Public Communication Campaigns.
Newbury Park, CA: Sage Publications, Inc.
Bennett, C. 1976. Analysing Impacts of Extension Programs.
Agricultural Cooperative Extension. Report 511.
Carey, R., Posavac, E. 1980. Program Evaluation Methods and
Case Studies. Englewood Cliffs, NJ: Prentice-Hall, Inc.
Franklin, J., Thrasher, J. 1976. An Introduction to Program
Evaluation. New York: John Wiley and Sons.
Gibbon, F., Taylor, C. 1978. How to Design a Program Evalua-
tion. Newbury Park, CA: Sage Publications, Inc.
Grabe, S. 1983. Evaluation Manual. Paris: United Nations
Educational, Scientific and Cultural Organization.
Green, L., Lewis F. 1986. Measurement and Evaluation in Health
Education and Health Promotion. Palo Alto, CA: Mayfield.
Herman, J. 1987. Evaluator's Handbook. Newbury Park, CA:
Sage Publications, Inc.
King, J. 1987. How to Access Program Implementation. Newbury
Park, CA: Sage Publications, Inc.
Kosecoff, J. 1982. Evaluation Basics: A Practitioners Manual.
Newbury Park, CA: Sage Publications, Inc.
National Cancer Institute. 1992. Making Health Communication
Programs Work: A Planning Guide. Washington, D.C.:
National Institute of Health.
Patton, M. 1987. Creative Evaluation. Newbury Park, CA: Sage
Publications, Inc.
Rossi, P., Freeman H. 1990. Evaluation: A Systematic Approach.
Newbury Park, CA: Sage Publications.
Rutman, L. 1983. Understanding Program Evaluation. Newbury
Park, CA: Sage Publications.
Selnow, G., Crano, W. 1987. Planning, Implementing and
Evaluating Targeted Communication Programs: A Manual for
Business Communicators. New York: Quorum.
Smith, M. 1989. Evaluability Assessment: A Practical Approach.
Nomell, MA: Kluwer Academic Publishers.
United Nations Educational, Scientific and Cultural Organi-
zation. 1980. Evaluating Social Action Projects: Principles,
Methodological Aspects and Selected Examples.
US. Environmental Protection Agency. 1990. Communicating
Environmental Risks: A Guide to Practical Evaluations. Report
230-01-91-001.
Wimmer, R., Dominick, J. 1983. Mass Media Research: An
Introduction. Belmont, CA: Wadsworth Publishing Co.
Winsdor, R, Baranowski, T. 1984. Evaluation of Health
Promotion and Education Programs. Palo Alto, CA: Mayfield.
83
-------
Chapter 4
Introduction 85
Proofreading Software 86
SMOG Readability Test 90
Expert Reviews 92
Signal Stopping 94
Monitoring Audience Interactions 96
Community/ Regional/ State Interactions 100
Organizational Activities 102
Program Activities 104
Polling 108
Focus Groups 112
Observations 116
Self-administered Questionnaires 120
84
-------
Introduction
Evaluation Techniques
Introduction
This chapter describes techniques that can be used
in program evaluation. They range from simple to
difficult and cover three different types of evaluation.
Many techniques are available to evaluators, this
chapter only covers a few examples of options that
could be used with your program. When appropri-
ate, consider variations of these techniques and refer
to the references included in Chapter 3 and 4 for
other options. However, tradeoffs are always present
when selecting a technique, it is important to explic-
itly consider the advantages and disadvantages.
At the beginning of each description, in this
chapter, you will find a summary of information that
will help you decide whether the technique will be
appropriate for your organization. Followed by this
summary, you will find a description of the tech-
nique, the procedures you should follow, the advan-
tages and disadvantages of the technique, and a list
of sources you can review for more information.
Below is a summary of the techniques that are
covered in the chapter listed by type of evaluation.
Type of Evaluation
Evaluation Technique
Formative
Process
Outcome
Proofreading Software
X
SMOG Test
X
Expert Reviews
X
X
Signal Stopping
X
Interaction Monitoring
X
Surroundings Monitoring
X
Organizational Monitoring
X
Program Activities
X
Polls
X
X
X
Focus Groups
X
X
X
Observations
X
X
X
Self-administered Questionnaires
X
X
X
85
-------
Proofreading Software
SUMMARY
Evaluation Categories
Formative
Evaluation Objectives
Comprehension
Overview
A computer program that reviews word processing files for grammar mistakes,
style, usage, punctuation, spelling and readability.
Low
Medium
High
| Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
Types of
Personal computer: Apple or IBM compatible, printer, word-processing
Resources Required
program.
Interaction with
None
Target Audience
Introduction
Proofreading technology is probably one of the
most innovative uses of computer software. Proof-
reading software can ensure that wording is not too
complex, thus reducing audience comprehension. It
also ensures that all communication is free of gram-
mar, spelling, style and usage mistakes. Most users
will find the program feedback less threatening than
discussing mistakes with another person and it saves
a manager's valuable time. Also, custom styles can be
designed by the users, which ensures that a consis-
tent image is presented. For example, you can easily
program the software to replace common abbrevia-
tions like "DO" with "dissolved oxygen." These
programs can perform thousands of tests on a
document within a few minutes and require minimal
experience to use.
Variables Checked by the Program
A fundamental component of all of the programs
is the evaluation of the rules of grammar and the
mechanics of sentence structure. Beyond these
factors, there are many variables of communication
that can be checked by the program. Below is a
summary of the variables.
Reading Comprehension
Several scales have been developed that allow the
user to calculate the education level needed to under-
stand written communication. Most of the scales are
calculated based upon the number of words in
sentences and syllables per word. If the audience that
you have targeted consists of scientists or engineers,
then your writing may include more complex words
and sentences. However, if your audience is more
general, it may be appropriate to keep the writing at a
sixth-to-ninth grade level. The program will allow
you to monitor the grade level of your writing.
Chapter 4 Evaluation Techniques pages 84-121
-------
Proofreading Software pages 86 - 89
Document Statistics
A summary of document components is very
useful. It helps you to prepare a user-friendly
communication that is useful and effective.
Several components can be tracked in documents:
>¦ Number of paragraphs
>• Average number of sentences per para-
graph
»¦ Number of sentences
>- Average number of words
Number of sentences ending with T or T
*~ Number of words in document
>- Average syllables per word
>- Average word length
Word Usage
A summary of all the words used in a document
can be produced. It can help to identify words that
are used too often.
Writing Style
The styles of writing should vary depending upon
the audience. For example, a letter written to a friend
may be very different from one written to a mayor or
governor.
Proofreading software allows you to adjust the style
based upon many variables including:
>• Passive voice
»• Long sentences
>- Split infinitives
»- Gender specific
> Consecutive nouns and prepositional
phrases
»¦ Vague adverbs
Procedure
1 Install the word-proofing software and
become familiar with its functions.
2 Complete your written communication with
a word-processing program and save it in a file.
3 Open die word-proofing program and select
the word-processing file that was saved.
4 Make changes to the file as necessary. This
should be based upon your understanding of
the target audience and program goals and objec-
tives. The programs allow interactive feedback. So
you can change things as you view the text or the
entire document can be checked by the program and
reviewed later. The user has almost complete control
over criteria used to examine the document. There-
fore, only criteria that are important to the user are
reviewed by the program. Changes are always
presented as suggestions, which gives the user
complete editorial authority.
Advantages
+ Easy to install and operate, which results in
immediate feedback.
+ Program flexibility allows the user to control
and create the parameters used to check tine
text
+ Allows the organization to maintain consis-
tency in all external communications.
+ Through on-screen suggestions, the programs
can improve the user's writing ability.
+ The programs can be used for all word-
processing applications.
+ Writers with technical backgrounds are
required to evaluate critically their writing
styles to ensure that others can understand the
text
+ These programs save time and money. They
can track and monitor many aspects of a
document's contents and quickly calculate the
education level needed to understand the
document.
Continued...
87
-------
Proofreading Software pages 86 - 89
Disadvantages
- There is no direct contact with the target
audience.
- Organizational acceptance could be a problem
unless the program is introduced properly.
Some users may resent their work being
checked by a computer, particularly their
grammar. Avoid this problem by focusing on
the goals and objectives of your communica-
tions' project If users understand how the
program could be useful in monitoring the
readability and style of communications, they
may be more likely to accept the assistance.
- The program only applies mechanical rules
and does not consider contents or the meaning
of the words. Therefore, it is important to
realize that it cannot take the place of an
editor.
- The programs do not analyze the organization
of the writing.
How to Find More Information
Most proofreading programs are available at
stores where software is sold. Righwriter, Correct
Grammar and Grammatik are three popular pro-
grams. Be sure that the software is compatible with
your computer's hardware and word processing
program.
88 Chapter 4 Evaluation TachniquM pages 84-121
-------
Proofreading Software pages
Chapter 4 Continues...
-------
SMOG Readability Test
SUMMARY
Evaluation Categories
Formative
Evaluation Objectives
Comprehension
Overview
A simple test called the SMOG (Simple Measure of Gobbledygook) Readability
Grading Formula involves counting words and sentences. The result of the test is
an estimation of the education level needed to understand the text.
Low
Medium
High
f Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
Types of
Resources Required
Calculator, pencil, paper.
Interaction with
Target Audience
None
Introduction
Testing a document's readability can be completed
by using a computer program (see previous section)
or calculated manually. The major difference between
the SMOG tests and a computer program is that with
the SMOG tests, readability can be calculated without
anything more complicated than a calculator. The
underlying theme of this approach is that longer
sentences and words, containing three or more
syllables, require a higher level of education to
comprehend. To understand the education level
needed to read a text, die score is then compared on a
simple chart. The chart was developed through
statistical studies.
For general audiences, most writing should be on
a sixth-to-ninth grade reading level. However, at
times it may be impossible to use simple words and
short sentences. For example, it may be simpler to
use the word "oligotrophic" instead of writing "a
lake with little nutrients and scant vegetation." Also,
there are many one syllable words that are not
necessarily familiar, so the resulting score must be
evaluated carefully.
Chapter 4 Evaluation Techniques pages 84-121
-------
SMOG Readability Test pages 90-91
Procedure
Appendix C includes a detailed worksheet of how
to use this approach.
Advantages
+ This approach is quick and easy to use and
requires no special training.
+ Inexpensive to use, the approach requires no
special equipment—only a pencil, paper and
calculator.
+ Staff can quickly leam that it is important to
use simple words and short sentences when
communicating with most audiences.
Disadvantages
— There is no direct contact with the target
audience.
- The program only applies two mechanical
rules and does not consider text content,
organization, tone, sentence structure, sen-
tence flow or the meaning of the words.
Therefore, it is important to realize this is just
one of several variables to consider when
deciding on text organization and content.
Where to Find More Information
Doak, C. 1985. Teaching Patients urith Low Literacy Skills.
Philadelphia: J.B. Lippincott Co.
McLaughlin, H. 1969. "SMOG Grading: A New Readability
Formula." Journal of Reading. 22:4,639-46.
National Cancer Institute. 1992. Making Health Communication
Programs Work: A Planning Guide. Washington, IX:
National Institutes of Health.
91
-------
Expert Reviews
SUMMARY
Evaluation Categories
Formative
Process
Evaluation Objectives
Concept Development
Confirming Audiences
Attention Recall
Comprehension
Personal Relevance
Sensitivity/Controversial
Strong/Weak Points
Attractiveness
Audience Interactions
Program Activities
Organizational Activities
Overview
This approach involves outside experts. Their views can provide valuable insight,
throughout a program, on how to improve your communications efforts.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
Types of
Resources Required
Questionnaire, meeting place, facilitator.
interaction with
Target Audience
None
Introduction
Involving community experts from the fields of
education, communications, advertising, marketing
and public relations is important. Often they are not
on staff and an organization may lack the resources
needed to hire these experts. To avoid such limitations,
it is useful to form an advisory board that involves
experts. They can review many aspects of a program
from early stage development until a review of the final
evaluation. If they meet every two or three months they
can monitor progress throughout the program.
Consider including people that may have contact
with the target audience, their involvement may
provide some useful insight Use their input in
conjunction with the experts.
Also, members of your advisory board may
control the communication channels needed to
distribute your message. Without their cooperation, it
may be difficult to distribute the information.
92 Chapter 4 Evaluation Techniques pages 84-121
-------
Expert Reviews pages 92 - 93
Procedures
There is not a specific system for working with a
panel of experts. They could meet once a month on a
formal basis and discuss the project with the staff. Or
the panel could be available to answer questions as
they arise.
Although the way the group operates is flexible,
there are some issues that should be addressed in
spite of the organizational style in the group.
1 Select people for the committee who have
experience working in a group setting and can
work with their peers.
2 Ensure that the members have a clear under-
standing of the project's goals and objectives.
Then, through interaction with the members, be
prepared to develop the goals and objectives of the
committee. Clearly define their role and decision-
making authority within the project.
3 Ensure that the staff has "bought into" the
idea of having an external party review its
work. Also, review the organization's structure and
incentives to ensure that the committee can fit into
the picture.
4 Provide regular feedback to the committee
members on the progress of the project and
their suggestions.
^ Say thank-you in many ways and often.
Advantages
+ Using expert panels is an inexpensive way to
gain professional help, at minimal costs.
+ After spending time creating materials, it is
often difficult for staff to step back and
objectively evaluate their own work. The
panel can provide good feedback from a third-
party perspective.
+ The committee members may have many
contacts with people in the area. Their rela-
tionships with others may provide access to
resources that would not be available other-
wise.
Disadvantages
- There is no direct contact with the target
audience.
- It may be difficult to organize a committee
that can work together and is truly committed
to attending meetings and participating in the
review of materials.
- Staff may resent a critique of their work.
- At least one staff member must be familiar
with facilitating meetings and must serve as a
contact person to manage interactions with
the committee members. Depending upon
how often the committee meets, much staff
time could be required.
- The organization must be prepared to share its
control over the program with outsiders or the
committee's presence will be worth much less.
Where to Find More Information
Creighton,J. 1981. "Establishing an Advisory Group " The
Public Involvement Manual. Cambridge, MA: Abt Books.
93
-------
Signal Stopping
SUMMARY
Evaluation Categories
Formative
Evaluation Objectives
Comprehension
Personal Relevance
Sensitivity/Controversial
Strong/Weak Points
Overview
Written materials are presented to members of the target audience. They are
asked to read and evaluate the text by marking it with symbols.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
TVpea of
Resources Required
Written text, meeting place, list of target audience members, trained staff who can
explain the process.
Interaction with
Target Audience
Yes
Introduction
When people stop reading, it is usually because
they are confused, interested, agreeing, or disagree-
ing with the information. Using this assumption, the
Signal Stopping Technique was developed. Target
audiences are asked to respond to a written message.
They can record their reactions to the material by
marking the text with predefined symbols, every
time they stop reading.
94 Chapter 4 Evaluation Techniques pages 84-121
-------
Signal Stopping pages 94-95
Procedures
IThe staff must select a number of people from
the target audience to participate. Also, it is
important to collect data on the readers to ensure that
they typify the target audience.
2 Select a location to administer the test and
train the staff.
3 Readers from the target audience are asked to
write down information about the written
material whenever they stop reading.
Symbols such as those presented below are used to
identify why they stopped reading:
T - Think more about the idea
U - Understood the idea, but want to think
more
C - Confused by the text
R - Re-read the idea because it was difficult
to follow
? - The reader had a question about what
was written
A - Agreed with die writer
D - Disagreed with the writer
Other symbols can be created by the user for
specific purposes.
4 Incorporate the information collected into
improving the program materials.
Advantages
+ The approach is simple to use and can be
administered with little preparation.
+ The target audience is involved.
+ It encourages staff to evaluate their writing
and make modifications if the text is not clear
or has other problems.
Disadvantages
— Staff must be prepared to understand and
interpret beyond the actual response.
— Time and money must be spent finding
people who are part of the target audience.
— The conditions under which the document is
tested may be very different from the actual
reading conditions.
Where to Find More Information
Grunig, J., and Hunt, T. 1984. Managing Public Relations. New
York: Holt, Rinehart, and Winston.
Grunig, J., S. Ramsey, and L. Schneider. 1985. "An Axiomatic
Theory of Cognition and Writing." journal of Technical
Writing and Communication. 15(2): 95-130.
Kline, M., C. Chess and P. Sandman. 1989. Evaluating Risk
Communication Programs: A Catalogue of Quick and Easy
Feedback Methods. New Brunswick, NJ: Rutgers University.
95
-------
Monitoring Audience Interactions
SUMMARY
Evaluation Categories
Process
Evaluation Objectives
Understand audience and
interactions
Overview
When the program begins, it is important to collect information about who
participates and why they participated. Using a computer database or pencil and
paper, information about the target audience can be monitored from the first until
the last interaction.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
Manual System
Computer
Database
Staff Time
Computer
Database
Manual System
Staff Skill
Manual System
Computer
Database
Costs
Manual System
Computer
Database
Typoaof
Resources Required
A database software program and personal computer or a pencil and paper
record-keeping system, forms used to record information.
Interaction with
Target Audience
Yes
Introduction
If you have read Chapter 1 of this handbook you
are aware of how important the target audience is to
the success of a communication program. Interac-
tions with this audience should be monitored
because it is important to determine who is actually
receiving your message and participating in pro-
grams. Information that is collected can help you
evaluate whether you have accurately defined the
target audience. Also, you may find that die audience
changes in composition as program phases are
completed. A variety of information could be col-
lected about the target audience. There are no specific
lists of information that are essential, but often
demographic information can be helpful (i.e., age,
education). The method of collection limits the
amount and type of data that is gathered. Most
people will not stop and complete lengthy question-
naires, so it is better to decide what information is
essential to your program.
96 Chapter 4 Evaluation Techniques pages 84 -121
-------
Monitoring Audience Interactions pages 96-99
Procedures
1 Decide what information is important in
relation to program goals and objectives.
Attributes that could be tracked include:
>¦ Date and time of interaction
> Reason for target audience interests
»- Subject o/inquiry
>" Age
>¦ Level of education
>~ Occupation
>• Source of information where participant
found out about the program.
»- Address
>~ Phone number (if possible)
»- Owner or renter
>- Married/single
>- Ghildren at home
The type of information collected is limited only
by your imagination and the participants' willingness
to provide the information.
2 Decide when information will be recorded
(e.g., at meetings, when your organization
receives phone calls, workshops etc.)
3 Design a method for collecting the informa-
tion that will be convenient for the target
audience.
4 Select a method of recording and analyzing
data. Using a computer system will be better
in the long run because the data can be manipulated
easily and aggregated quickly. Manual systems may
work only if you are tracking two or three variables
or the target audience is very small.
5 Use the data to provide feedback to your staff
and improve program performance. Based
upon interactions, consider modifying program
goals, objectives, strategies or tasks.
Advantages
+ You can confirm whether your messages are
reaching the intended audiences. If you
targeted homeowners in a high income
bracket, but the participants are renters with
low incomes, this information should help
guide an evaluation of the messages and
channels used to distribute the messages.
+ The program staff can relate the timing of
messages to program response. By monitoring
audience interactions you can understand
why people do not respond until the fifth or
sixth time they hear a message?
Also, they can decide whether certain commu-
nication channels are more effective. Perhaps
no erne responds to your newsletter, but you
receive many responses from radio public
service announcements. It is important to
know what communication channels are
working and why. The feedback can be used
to modify the program in the early stages.
+ Program participation can be tracked over
time. Some participants may leave the pro-
gram after the first interaction and others may
continue. If you track who is involved, you
may learn that the program design is more
attractive to people who have certain charac-
teristics.
+ If you track information about the target
audience you will have a list of participants
available. Addresses and phone numbers
could be collected, so that staff can complete
follow-up evaluations to measure the out-
comes of the program.
+ The information collected may be a resource
for finding volunteers and supporters. These
people could be added to your mailing lists
and sent newsletters or other communication
about your organization.
Continued.
97
-------
Monitoring Audience Interactions pages 96-99
Disadvantage
— Avoid data overload. If you collect too much
data, it may be difficult to use the information.
If you use a manual system this could occur
very easily. Only collect data that is truly
important to the program. Don't feel that you
are limited by the examples listed in this
section. You can track any information about
interactions with the target audience that will
help in program evaluation and management.
Where to Find More Information
Dignan, M. B. 1991. Use of Process Evaluation to Guide
Health Education in Forsyth County's Project to Prevent
Cervical Cancer." Public Health Reports. 106:73-77.
McGraw, S., S. McKinlay, L. McClements, T. Lasater, A. Assaf
and R. Carle ton. 1989. "Methods in Program Evaluation:
The Process Evaluation System of the Pawtucket Heart
Health Program." Evaluation Review. October. 459-483.
National Cancer Institute. 1988. Making Health Communication
Programs Work. A Planning Guide. Washington, DC:
National Institutes of Health.
Rossi, P., and H. Freeman. 1990. Evaluation A Systematic
Approach. Newbury Park, CA: Sage.
>8 Chapter 4 Evaluation Technique# pages 84-121
-------
Monitoring Audience Interaction* pages 96-99 \
Chapter 4 Continue*...
99
-------
Community/Regional/State Interactions
SUMMARY
Evaluation Catagoiiaa
Process
Evaluation Objactivaa
Understand outside
influences that affect the
program.
Overview
Programs do not operate in vacuums. Usually there are interactions with a
variety of entities that are not directly involved in the programs, but could affect
the results. Therefore, it is important to monitor these interactions.
Amount of Organizational
Reaources Required
Low
Madium
High
Preparation Time
Manual System
Computer
Database
Staff Time
Computer
Database
Manual System
Staff Skill
Manual System
Computer
Database
Costs
Manual System
Computer
Database
Typaaof
Resource* Required
A database software program and personal computer or a pencil and paper
record-keeping system, forms used to record information, contact sheets to
record information, filing system to collect information.
Interaction with
Target Audience
No
Introduction
Many interactions will occur outside of the
program that may have a significant impact on the
program results. Sometimes, die staff may be in-
volved with governmental committees or in other
community leadership roles. If topics are discussed
that relate to the goals and objectives of the program,
it would be important to monitor these activities.
Often organizations unrelated to your program may
make decisions that directly or indirectly affect your
program. It is important to monitor these situations.
With good information-gathering techniques you
can inform the staff and change strategies and tasks
to respond to the surroundings.
100 Chapter 4 Evaluation Tachniquas pages 84-121
-------
Community/ Regional/ State Interactions pages 100-101 |
Procedure
1 Decide what information is important in
relation to program goals and objectives.
Attributes that could be tracked include:
> Newspaper or magazine articles, radio or
television stories
v Reports published by federal, state; county
and local officials.
»- Organizational contacts
»¦ Elected official contacts
»- Interactions with news media
»¦ Meetings attended by program staff
^ Decide when information will be recorded.
3 Design a method for collecting the informa-
tion. A contact sheet should be completed by
program staff when they receive information. Files
must be set up to collect newspaper articles and
related information.
4 Select a method of tracking contact sheets and
information files. Using a computer system
will be better in the long run, especially if you are
recording other process data. Manual systems may
work if you are tracking two or three variables.
5 Use the data to provide feedback to your staff
and improve program performance. This may
include modifying program goals, objectives, strate-
gies or tasks.
Advantages
+ By recording information and communicat-
ing with staff members, everyone can make
better decisions and contribute to the
management of die project
+ Information is the key to good decisions.
Keeping a library of information about
issues related to your program would be
useful in the future. It can help you plan
new programs and anticipate future
problems with present programs.
+ It allows the organization to constantly
monitor its social, economic and political
surroundings. Answers to the questions
that were asked in the first chapter of this
handbook can be updated on a regular
basis.
+ Monitoring this information keeps every-
one informed. Therefore, communication
between staff members is enhanced.
Disadvantage
- Avoid data overload. If you collect too
much data it may be difficult to use the
information. If you use a manual system
this could occur very easily. Only collect
data that is truly important to the program.
Don't feel that you are limited by the
examples listed in this section. You can
track any information about interactions
with the target audience that will help in
program evaluation and management.
Where to Find More information
Dignan, M. B. 1991. "Use of Process Evaluation to Guide
Health Education in Forsyth County's Project to Prevent
Cervical Cancer." Public Health Reports. 106:73-77.
McGraw, S., S. McKinlay, L. McClements, T. Lasater, A. Assaf
and R. Carleton. 1989. "Methods in Program Evaluation:
The Process Evaluation System of the Paw tucket Heart
Health Program." Evaluation Review. October 459-483.
National Cancer Institute. 1988. Making Health Communication
Programs Work: A Planning Guide. Washington, DC:
National Institutes of Health.
Rossi, P., and H. Freeman. 1990. Evaluation A Systematic
Approach. Newbury Park, CA: Sage.
101
-------
Organizational Activities
SUMMARY
Evaluation Catagories
Process
Evaluation Objectives
Monitor organizational
activities.
Overview
Organizational activities are the heartbeat of the program. Unless you monitor
these activities it is easy to overlook potential problems. A monitoring system will
make it much easier to catch problems early in program development.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
Manual System
Computer
Database
Staff Time
Computer
Database
Manual System
Staff Skill
Manual System
Computer
Database
Costs
Manual System
Computer
Database
Types of
Resources Required
A database software program and personal computer or a pencil and paper
record-keeping system, input sheets used to record information.
interaction with
Target Audience
No
Introduction
The daily operations of the program are very
important to monitor. When tracking how money
and time are spent, a trail of program documentation
is left that can be used to manage the program. By
comparing these inputs to the program's goals,
objectives and budgeted amounts you can ensure
that the program is managed properly. In many
situations this information is tracked in other record-
keeping systems, but it is important to monitor the
information within the context of the program's goals
and objectives.
Procedures
1 Decide what information is important in
relation to program goals and objectives.
These may include:
wmmm
Keep a constant time line, record who is doing
work and record expenses based upon tasks, goals
and objectives.
102 Chapter 4 Evaluation Techniques pages 84-121
-------
Organizational Activities pages 102 -103 I
Budgeted Amounts, Goals and Objectives
On a regular basis, compare budgeted amounts
for each program area to actual results and track
activities concerning goals and objectives. Activities
could include staff and volunteer hours, money
spent, number of presentations made, literature
distributed, meetings and conferences attended by
staff, or planned interactions with the community.
Time
All information should be tracked relative to time.
This provides a common point for aggregating results,
spotting relationships and tracking performance.
A brief summary of major decisions and reasons
for those decisions could be documented and
included in the program records. Also, changes in
policy or program focus should be explained in
writing because others can then benefit from the
knowledge.
2 Decide when information will be recorded
(i.e., daily, weekly or monthly).
3 Design forms and systems to record the
information.
4 Select a method of tracking and analyzing
information. Using a computer system will be
better in the long run, especially if you are recording
other program data. Manual systems may work only
if you are tracking a few variables.
5 Use the data to provide feedback to your staff
and improve program performance. This may
include modifying program goals, objectives, strate-
gies or tasks.
Advantages
+ By keeping time lines and tracking activities
systematically, the staff will be able to tell
whether they are moving away from the planned
agenda. Often there are good reasons to move
away from the original plan, but evaluation
forces everyone to discuss these issues. There-
fore, it can enhance communication and ensure
that everyone understands the change in plans.
+ Documenting program information can be
useful for future efforts. Knowing what it cost
to carry out certain components of a program
will help you plan future efforts.
+ The information collected can be used to
compare the cost-effectiveness and efficiency
of various activities.
+ The staff can monitor time and money spent
in relation to goals and objectives of the
project. This information will allow you to
regularly review the program from a "big
picture" perspective.
+ Program sponsors are accountable for the
money they spend to fund your project.
Information collection efforts will help them
meet their goals and strengthen your relation-
ship.
Disadvantage
— Avoid data overload. If you collect too much
data it may be difficult to use the information.
If you use a manual system this could occur
very easily. Only collect data that is truly
important to the program. Don't feel that you
are limited by the examples listed in this
section. You can track any information about
the organization that will help in program
evaluation and management.
Where to Find More Information
Dignan, M. B. 1991. "Use of Process Evaluation to Guide
Health Education in Forsyth County's Project to Prevent
Cervical Cancer." Public Health Reports. 106:73-77.
McGraw, S., S. McKinlay, L. McClements, T. Lasater, A. Assaf
and R. Carieton. 1989. "Methods in Program Evaluation:
The Process Evaluation System of the Pawtucket Heart
Health Program." Evaluation Review. October 459-483.
National Cancer Institute. 1988. Making Health Communication
Programs Work A Planning Guide. Washington, DC:
National Institutes of Health.
Rossi, P., and H. Freeman. 1990. Evaluation A Systematic
Approach. Newbury Park, CA: Sage.
103
-------
Program Activities
SUMMARY
Evaluation Catagories
Process
Evaluation Objectives
Monitoring program
activities.
Overview
Program activities are made up of many components. Many of the variables can
be tracked over time and used in conjunction with other information to monitor
the success of the program.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
Manual System
Computer
Database
Staff Time
Computer
Database
Manual System
Staff Skill
Manual System
Computer
Database
Costs
Manual System
Computer
Database
Types of
Resources Required
A database software program and personal computer or a pencil and paper
record-keeping system, input sheets used to record information.
Interaction with
Target Audience
Yes
Introduction
Many of the items included in this section involve
quantifying the components of the program activi-
ties. It is important to keep records on the dates of
activities and the location and nature of the activity
(i.e., school presentation, town council meeting,
volunteer training session.) In addition, you could
quantify many of the program outcomes including
the number of participants and copies of literature
distributed. Although none of these factors will
provide you with absolute answers about program
success, they do provide valuable information when
combined with other information sources.
104 Chapter 4 Evaluation Techniques pages 84-121
-------
Program Activities pages 104 -107
Procedures
1 Decide what information is important in
relation to program goals and objectives.
Activities to track include:
»• Channels of interaction
>* Estimated number or people readied
through channels
>¦ Type of presentations made
>- Presentation attendance
»- Number of participants
Nwnber of request for literature
»• Conferences attended
>- Meeting attended
>• Volume of telephone and mail inquires
>• Response time to inquiries
»- Inventory of materials
>¦ Articles produced
>• Timing and number of television and radio
announcements
»- Other activities that involve interactions
with the target audience
All of this information can be combined in a
database to provide feedback on who was involved
in the activities and why. For example, you could-
cross-reference the volume of telephone and mail
inquiries with the timing of radio and TV announce-
ments (consider using graphs to show comparisons).
^ Decide when information will be recorded.
3 Design an input sheet to record the informa-
tion.
4 Select a method of tracking and analyzing
data. Using a computer system will be better
in the long run because the data can be manipulated
easily and aggregated quickly. Manual systems may
work only if you are tracking two or three variables
or the target audience is very small.
5 Use the data to provide feedback to your staff
and improve program performance. Based
upon interactions, consider modifying program
goals, objectives, strategies or tasks.
Advantages
+ The cost effectiveness of various programs can
be tracked. If one type of program cost less,
but attracts more people, it may be more cost-
effective.
+ It allows the program staff to receive feedback
on aspects of the program activities. They can
monitor information and change ineffective
approaches and affirm effective approaches.
+ Information about activities, such as the
number of people participating in an activity,
can be an indicator of program success.
Without this type of information you would
have one less evaluation tool available.
+ Everyone associated with the program needs
to be aware of how the program is proceed-
ing. Information about program activities
helps contribute to good decision-making at
all levels of the organization.
Continued...
105
-------
Program Activities pages 104 -107
Disadvantages
- Avoid data overload. If you collect too much
data it may be difficult to use the information.
If you use a manual system this could occur
very easily. Only collect data that is truly
important to the program. Don't feel that you
are limited by the examples listed in this
section. You can track any information about
the organization that will help in program
evaluation and management
- Do not misinterpret data. Just because a lot of
people are participating in program activities
it does not mean that the program is a success.
Where to Find More Information
Dignan, M. B. 1991. "Use of Process Evaluation to Guide
Health Education in Forsyth County's Project to Prevent
Cervical Cancer." Public Health Reports. 106:73-77.
McGraw, 5., S. McKinlay, L. McClements, T. Lasater, A. Assaf
and R. Carleton. 1989. "Methods in Program Evaluation:
The Process Evaluation System of die Pawtucket Heart
Health Program." Evaluation Review. October 459-483.
National Cancer Institute. 1988. Making Health Communication
Programs Work / Planning Guide. Washington, DC:
National Institutes of Health.
Rossi, P., and H. Freeman. 1990. Evaluation A Systematic
Approach. Newbury Park, CA: Sage.
U.S. Environmental Protection Agency. 1990. Communicating
Environmental Risks: A Guide to Practical Evaluations. Report
230-01-91-001.
106 Chapter 4 EvaluationTachntquM pages 84-121
-------
Program Activities pages 104-107
Chapter 4 Continues...
107
-------
Polling
SUMMARY
Evaluation Catagoriaa
EvaluatlonObjectivea
Formative
Concept Development
Confirming Audiences
Attention Recall
Comprehension
Personal Relevance
Sensitivity/Controversial
Strong/Weak Points
Attractiveness
Overview
Amount of Organizational
Resources Required
Type* of
Reaourcea Required
Interaction with
Target Audience
Procaaa
Audience Interactions
Program Activities
Outcome
Change in behavior,
attitudes, skills,
knowledge, aspirations.
Increase in Awareness
Polls can be used by large and small organizations. If completed with statistically
valid methods, they allow die user to project the results to a larger population.
Polls can uncover who is in a target audience, as well as the best approaches to
develop a program, and what your audience learned from the program. Also they
can find relationships among various polling questions and uncover the
respondents' attitudes, behavior or beliefs.
Polls can be conducted three different ways - by telephone, mail or personal
interview -all of which use varying amounts of time and resources. This is
summarized in the following table.
Low
Medium
High
Preparation Time
Telephone
Personal
Interview Mail
Staff Time
Mail
Telephone
Personal
Inverview
Staff Skill
Mail
Personal
Inverview Phone
Costs
Mail
Telephone
Personal
Inverview
Staff or volunteers to conduct polls, personal computer and spreadsheet or
survey program, copies of questionnaires.
Depends upon objectives of the poll. Polls could be used to identify a target
audience. Also, they are used before, during or after a communication campaign
that was directed at a target audience.
108 Chapter 4 Evaluation Tachnkjuaa pages M-121
-------
Polling pages 108-HI
Introduction
Often organizations overlook polling for evaluat-
ing materials and obtaining an understanding of
public attitudes, behavior, beliefs, or attributes.
Usually this occurs because polling is seen as a
complicated, costly method of collecting data that
involves statistics. However, it is a very powerful
method of finding what the majority of the target
population thinks, while only talking with a small
percentage of that group.
Polling is often misunderstood because people
lump all polls into one category. In reality, there are
at least four types of polling and three different ways
of doing polls. Each method has advantages and
disadvantages. First there are die typical short-
answer polls that last from 10-15 minutes. In-depth
surveys can last up to 60 minutes. A tracking poll
asks a few questions and is conducted in a short
period of time. Finally, a panel poll can be used to
compare the response of the same people over a
period of time.
Polling can be used at any stage of the program
from developing concepts to the final evaluation after
the program ends. Since its applications are flexible, it
can fit into many areas of program development.
Polling can be used to identify the target audience or
to verily trends that were noticed using other evalua-
tion techniques. Probably the most complicated
aspect of polling is analyzing die results, but the
Montana Alliance for Progressive Policy has devel-
oped a computer program called Pollstart, that will
help analyze data. Other programs are also available;
check with your local computer software store.
Procedures
1 Decide how polls fit into the programs goals
and objectives.
2 Decide whether the poll will be conducted by
mail, telephone or personal interviews. Next,
determine the type of poll: will the poll be a few short
questions or an in-depth survey? A poll could be
done on a regular basis to track opinion. Also, a panel
review looks at the before-and-after impact of the
program.
3 Obtain a list of those people that will be
polled or find a location to conduct a personal
interview poll.
4 Organize staff to design the questionnaire,
draw samples and recruit interviewers.
5 Test the questionnaire by asking the questions
and discuss the sample design with experts
(usually university professors are a good choice). Be
prepared to revise the questions or design.
6 Prepare copies of the final questionnaire and
materials needed by those conducting
interviews. Train interviewers and begin the inter-
view process.
7 Code responses and input data into a per-
sonal computer. A personal computer pro-
gram like Pollstart, which is designed for surveys,
will make this process easier to manage.
8 Analyze the data and tabulate results. Then
program staff may use the results for the
purposes outlined in the initial goals and objectives
of the project.
Continued...
109
-------
Polling pages 108-111
Advantages
+ Once the staff of the organization is trained,
you will find that their knowledge will extend
beyond a specific program. Information from
polls can be used in die media and can
influence public officials or contributors. Also,
staff members can use their skills to analyze
critically the results of other polls.
+ The resources presented at the end of this
section will be valuable for other reasons,
because the staff will learn how to develop
questionnaires and how to do interviews,
which are valuable skills that can be used for
gathering information.
+ If you choose to use statistics, the results of a
poll can be projected to a larger population
than was polled.
+ The target audience is involved.
+ The information collected can help you design
better programs.
+ Respondents can remain anonymous.
+ The results from polls, if statistically valid, can
positively reinforce your cause.
Disadvantages
- Staff members or program managers may be
afraid of using this method because most
people perceive it to be too difficult.
- The initial start-up costs and learning curve
related to polls may be higher than with other
methods discussed in this handbook. How-
ever, once the staff is trained and is aware of
the basic components of developing polls,
they will become much more at ease with each
application.
- In some cases, die resources presented in this
section may not provide enough details about
polls. If this occurs, it would be important to
seek the advice of an expert. Often colleges or
universities are a good source of inexpensive
help.
- To make tine results of a poll more reliable,
there must be more people surveyed. How-
ever, this leads to a more expensive poll, so
there is always a trade-off between organiza-
tional resources and the quality of the results.
Where to Find More Information
Hessler, R. 1992. Social Research Methods. St. Paul, MN: West
Publishing Company.
Hsia, H. 1988. Mass Communications Research Methods: A Step-
in/Step Approach. Hillsdale, NJ: Lawrence Erlbaum
Associates.
Lake, C. 1987. Public Opinion Polling: A Handbook far Public
Interest and Citizen Advocacy Groups. Washington, DC:
Island Press.
Miller, T. 1991. Citizen Surveys: How to Do Them, How to Use
Them, What They Mean. Washington, DC: International
City Management Association.
Survey Research Center. 1983. Interviewers' Manual. Ann
Artx>r, MI: University of Michigan Institute for Social
Research.
110 Chapter 4 Evaluation Technique pages 84 -121
-------
Polling pages 108-111
Chapter 4 Continues...
111
-------
Focus Groups
SUMMARY
Evaluation Categories
Formative
Process
Outcome
Evaluation Objectives
Concept Development
Confirming Audiences
Attention and Recall
Comprehension
Personal Relevance
Sensitivity/Controversial
Strong/Weak Points
Attractiveness
Audience Interactions
Program Activities
Change in behavior,
attitudes, skills,
knowledge, aspirations.
Overview
Focus groups are an interactive discussion with a small group of people from a
target audience. It goes beyond other forms of research and tries to answer
questions about the "why", instead of "what" or "where" or "how often."
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
Types of
Resources Required
Moderator, meeting room, video and/or audio equipment.
Interaction with
Target Audience
Yes
Introduction
A focus group is an underutilized tool in develop-
ing communication and outreach programs. Focus
groups are often overlooked because a high level of
skill is required to moderate die discussions, but the
results are uniquely different from any other evalua-
tion technique.
As opposed to questionnaires, surveys or other
research techniques, the focus group answers are not
predefined by "choice A, B or C." Instead, six to ten
people sit in a room with a moderator and have an
in-depth discussion about a few topics. Normal
survey questions may ask whether an idea is appeal-
ing. In contrast, focus groups explore why the person
thinks an idea is appealing and what the factors are
that helped form the decision. The researcher is able
to experience what the audience is thinking in a
personal way, unlike other evaluation techniques.
112 Chapter 4 Evaluation Techniques pages 84-121
-------
Focus Groups pages 112-115
Focus groups are useful at several stages of a
program, beginning with target audience definition.
When program materials are developed, they can be
used to obtain reactions to those materials. After
surveys have been completed, they can be used to
verify the results and to understand the thoughts
behind those results. Finally, focus groups can be
used during the program to understand reactions or
afterwards to understand the impact of the program.
Procedure
1 Define your target audience clearly. For
example, knowing that your audience consists
of males between the ages of 18-45 is not detailed
enough. However, focus groups can assist in defining
some characteristics of the target audience.
2 Decide how focus groups fit into the
program's goals and objectives.
3 Recruit members for the focus group. This is
usually accomplished by calling people and
then screening them for participation, by asking
several questions. The questions will help determine
whether they exhibit die characteristics of the target
audience. People that pass the screening are then
invited to attend a one-to-two hour focus group
discussioa Participants are usually paid a small fee.
Usually meals are provided if the discussion occurs
in the evening.
4 Select facilities that offer a comfortable
setting for participants. Also, it is important to
tape the discussion.
5 Develop a guide, to be used by the facilitator,
that will provide the outline for the discus-
sion. Many questions will be open-ended.
A few examples of facilitator questions:
What do you think about t
Do vou have questions about ?
What would you do if ?
These types of questions can provide the facilitator
with the opportunity to probe for responses and look
for responses and reasons behind the answers.
6 Conduct the focus group meeting. During the
discussion, which usually lasts from one to
two hours, the facilitator will guide the group.
Facilitators are required to carry out the following
activities: move the discussion along, keep die
discussion from moving off the subject, successfully
probe beyond short answers for more important
information and involve quiet group members.
7 Analyze the results of the focus group. The
product of a focus group is more than a
description of who said what. Instead, it is a detailed
analysis and synthesis of comments that occurred
through-out the focus group session. The words are
only one component of the analysis; other compo-
nents include context, body language, intensity of
comments, and reaction to others' comments.
Conclusions and recommendations are developed
from the analysis.
Usually it will take four to six focus groups to gain
a consistent insight If you have just two groups, you
may find differences. To overcome those differences,
it is important to obtain enough information to
observe trends in the results.
Advantages
+ Interviewing six to ten people at once is more
cost- and time-efficient than conducting in-
depth interviews with each person.
+ Focus groups are the only research method
that is appropriate for certain situations. The
approach can provide information about
beliefs, behavior and attitudes in a format that
cannot be matched by other methods.
+ Unique relationships may be uncovered by
the facilitator that may be missed by using
other methods of research.
+ The staff of the organization may find many
uses for die focus group methods once they
become familiar with the format. Focus
groups could be used to discover perceptions
of employees, volunteers and organization
supporters.
Continued...
113
-------
| Focus Groups pages 112-115
+ A group setting often allows participants to
recall events and emotions that would be
missed in individual interviews or through
pencil and paper responses.
+ When there is little information known about
a problem or target audience's attitudes,
beliefs or behaviors, using focus groups is a
good method to gain information.
+ The technique is directed at the target audi-
ence.
Disadvantages
- The information collected is qualitative, not
quantitative and cannot be projected to the
larger population using a statistically-valid
approaches.
- Focus groups do not necessarily answer all the
questions about a program (nor does any
other method.) It does not provide absolute
yes and no answers, but can help narrow
down choices and provide a direction for
future research.
- Focus groups can be expensive. The cost for a
group can range from a few hundred dollars
to $3000. Often this is dependent upon the
location, recording costs, incentives paid to
participants, who moderates die group and
how much involvement the moderator has in
developing the program. Sometimes, non-
profit organizations can obtain reduced rates
from moderators of focus groups. Staff
members could manage the group, but only if
they have spent time researching the issue and
investigating how to handle group dynamics.
If you have never used focus groups, it would
be a good idea to receive guidance or training
from an expert.
- The participants may not provide honest
answers because of the possibility of "group
think." Sometimes group members may not
express the same answers as in a private
interview or when completing an anonymous
questionnaire.
Where to Find More Information
To find consultants who can help develop a focus
group format, provide training or moderate the
sessions, call:
American Marketing Association (312) 648-0536
Marketing Research Association (312) 644-6610
Other Sources
American Marketing Association. 1989. Focus Group Inter-
views: A Reader. Chicago, IL: American Marketing Associa-
tion.
Basch, C. 1987. 'Toeus Group Interview: An Underutilized
Research Technique for Improving Theory and Practice in
Health Education." Health Education Quarterly. Winter. 411-
448.
Byers, Y., and J. Wilcox. 1991. "Focus Groups: A Qualitative
Opportunity for Researchers." Journal of Business Communi-
cation. Winter: 63-78.
Goldman, A. and S. McDonald. 1985. The Group Depth
Interview, Principles and Practice. Englewood Cliffs, NJ:
Prentice-Hall.
Merton, R. 1990. The Focused Interview: A Manual of Problems
and Procedures. New York: Free Press.
National Cancer Institute. 1992. Making Health Communication
Programs Work: A Planning Guide. Washington, DC:
National Institutes of Health.
Qualitative Research Council. 1985. Focus Groups: Issues and
Approaches. New York: Advertising Research Foundation.
Stewart, D. 1990. Focus Groups: Theory and Practice. Newbury
Park, CA: Sage.
US. Environmental Protection Agency. 1990. Communicating
Environmental Risks: A Guide to Practical Evaluations. Report
230-01-91-001.
114 Chapter 4 Evaluation Techniques pages 84-121
-------
Focus Groups pages 112-115
Chapter 4 Continues...
115
-------
Observations
SUMMARY
Evaluation Categories
Formative
Process
Outcome
Evaluation Objectives
Concept Development
Confirming Audiences
Audience Interactions
Program Activities
Surrounding Interactions
Change in behavior, skills.
Overview
Observations can provide a unique perspective on program activities. Observers
can experience things that may be missed in a questionnaire or similar technique.
However, don't rely only on observations for evaluation because everyone
records information through a maze of personal judgments and biases.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
Types of
Resources Required
Pen and paper to record observations, an audio or video recorder to assist in
recording observations.
Introduction
Observation usually involves monitoring the
participants' response to various aspects of a pro-
gram. In some projects, the staff are asked to keep a
logbook.
This approach can be useful if the observations are
not obtrusive and have little impact on the program's
personnel and participants. The observations can
occur while being totally removed from the program
environment or the staff member could fully partici-
pate as a member of the target audience. Many
observations are a combination of these practices.
Observations can provide insight that questionnaires
and other quantitative techniques do not offer. They
can explain what happened and why it affected die
program. For example, responses to a questionnaire
completed after a program may indicate a negative
reaction. However, it may be difficult to determine
what factors contributed to that outcome.
Whenever observations are used, they must be
made systematically and observers should be trained.
Objective information such as the number of people
present, race, sex, time, weather conditions can be
recorded. Also, subjective observations can provide
descriptions and explanation. Regardless of how
observations are used, it is important to have a clear
set of criteria on which to base observations. Observa-
tions are more useful in the early stages of a program,
especially when materials are being tested. They have
116 Chapter 4 Evaluation Techniques pages 84-121
-------
Observations pages 116-119
less value when determining the outcomes of
programs, because there are so many factors that
influence results. Also, it is difficult to summarize
and analyze the results of observations.
Procedure
1 Decide how this evaluation method fits into
the goals and objectives of the program.
2 Clearly define what information you want to
collect and the advantages and disadvantages
of collecting the data.
3 Decide on the setting where you want to
collect information and gain access to the site.
4 Decide how information will be recorded.
There are three common methods for record-
ing data and each method may be appropriate for
certain situations.
Narrative Method
There is little direction in this approach. The
observer records events and provides as much detail
as possible.
Data Guide
An observer is asked to answer specific questions
about what occurred during the observation.
Rating Method
This approach provides the observer with a
numeric rating on the program contents or asks for
the percentage of time spent on program compo-
nents.
Advantages
+ The technique involves the target audience.
+ When the technique is used in the formative
stages of the program, the results can be very
helpful. Background information about the
audience and problem can be obtained, which
can lead to the development of better theories
to test with other evaluation techniques and
improved program design.
+ Some target audiences, especially those
opposed to your program, may not be willing
to complete questionnaires. However, through
an observer, they may be able to gain their
cooperation.
+ This technique is inexpensive and can be
applied in all phases of the evaluation pro-
gram.
+ Observations often occur in the actual envi-
ronment where die program will take place.
Since the natural setting is observed, it allows
for genuine reactions from program partici-
pants.
+ The technique offers insights into why the
results of the activity were good or bad.
Disadvantages
- The information collected in a field observa-
tion is not easily summarized as other more
quantifiable evaluation techniques. It is much
harder to make absolute decisions based upon
narrative comments.
- Staff members that perform observations may
be biased by their personal interests in the
program outcomes, judgments and percep-
tions. It is much harder to develop conditions
in which this bias can be controlled. To reduce
this problem, it is better to have more than one
person observing the same environment If
different conclusions are reached, more
analysis should occur.
4 Train the observers about how to record
information and make sure they clearly
understand the criteria used in the evaluation
technique.
Decide how often data will be collected.
6 Develop a filing system to manage the
information that is collected.
7 Analyze the results of the evaluation and use
the information to improve the program.
Continued...
117
-------
Observations pages 116-119
- The externa] validity of observations is less
certain. If your program is accountable to
external sources you may find it hard to
defend observations, especially if you are
trying to evaluate the outcomes of the pro-
gram.
- Since it would be difficult to include statistical
sampling and analysis, it would be inaccurate
to utilize the observations beyond the environ-
ment that was analyzed. To mitigate this
problem, sometimes observation periods are
chosen on a random basis or participants are
selected at random to observe.
- Unless the observation is constructed care
fully, the participants' interactions and
behavior may be influenced by the observer.
Where to Find More Information
Bogdan, R, and S. Taylor, 1975. Introduction to Qualitative
Research Methods. New York: Wiley.
Hessler, R. 1992. Social Research Methods. St. Paul, MN: West
Publishing Company.
Jorgensen, D. 1989. Participant Observation: A Methodology For
Human Studies. Newbury Park, CA: Sage Publications.
Rossi, P., and H. Freeman. 1990. Evaluation A Systematic
Approach. Newbury Park CA: Sage.
Wimmer, R., and J. Dominick. 1983. Mass Media Research: An
Introduction. Belmont, CA: Wadsworth Publishing Co.
118 Chapter 4 Evaluation Techniques pages 84-121
-------
Observations pagesll6-119
Chapter 4 Continues...
119
-------
Self-administered Questionnaires
Process
Outcome
Evaluation Categories
Evaluation Objectives
Concept Development
Confirming Audiences
Attention and Recall
Comprehension
Personal Relevance
Sensitivity/Controversial
Strong/Weak Points
Attractiveness
Audience Interactions
Change in behavior,
attitudes, skills,
knowledge, aspirations.
Increase in Awareness
Overview
Self-administered questionnaires are a great way of collecting data about your
program. Once questions are developed, the respondents complete the
questionnaires. It can be a cost-effective way of collecting information and
improving your program.
Low
Medium
High
Amount of Organizational
Resources Required
Preparation Time
X
Staff Time
X
Staff Skill
X
Costs
X
Types of
Resources Required
Questionnaire copies, system to record and summarize results.
Interaction with
Target Audience
Yes
Introduction
Self-administered questionnaires can be a great
way of carrying out evaluations. After questions are
developed, the forms can be handed out whenever
there is contact with die target audience. The contact
could be with a staff member or with materials that
are produced by the program. They can be used to
pretest materials or to monitor responses to many
aspects of a communication campaign. For example,
to pretest a handbook the contents could be mailed to
members of the target audience along with a ques-
tionnaire. The input could be used to improve many
components of the handbook.
Question design can include closed- and/or open-
ended questions. Results from closed-ended ques-
tions can be easily summarized. Open-ended results
120 Chapter 4 Evaluation Techniques pages 84-121
-------
Self-admin istered Questionnaires pages 120 -121
often provide insightful information, but are much
harder to summarize. After spending some time
testing the questions with a small number of respon-
dents you will be ready to begin using this powerful
evaluation tool.
Procedure
1 Decide how this evaluation method fits into
the goals and objectives of the program.
2 Clearly define what information you want to
collect and the advantages and disadvantages
of collecting the data.
3 Decide on die setting where you want to
collect information (e.g., meeting, back of a
handbook, mail with materials to pretest).
4 Design a questionnaire, deciding which ques-
tions will be open-ended and which will be
closed-ended. Both types can provide valuable
information. Develop instructions that are dear and
concise. Test the questionnaire with a small sample
from the target audience to find questions that could
be confusing or misinterpreted.
Decide how often data will be collected.
6 Develop a filing system to manage the
information that is collected.
7 Analyze the results of the evaluation and use
the information to improve the program.
Advantages
+ The technique involves the target audience.
+ Detailed information can be collected from
people that are very busy and may not have
time to participate in other interview formats.
+ Respondents can remain anonymous.
+ There is a great amount of flexibility in the use
of the self-administered questionnaire. It can
be mailed along with materials to be used in
formative or outcome evaluation Also, it can
be handed out at meetings or at any point in
the program where there is an interaction with
the target audience.
+ With die exception of mailing expenses, it is a
cost-effective way of completing evaluations.
It takes little staff time to collect the informa-
tion, since it is completed by the respondent.
+ If you require more information, a questionnaire
can be combined with the statistical power of
polling. The results can be projected to a much
larger population and have greater validity.
+ Questionnaires provide you with a way of
collecting demographic data and other
important attributes that can be used in the
analysis, along with answers to questions.
Disadvantages
- You may need to follow up with respondents
to increase the response rate.
- The data collection effort will require more
time than other methods because of delays in
responses.
- There is a bias against those respondents who
have reading and writing limitations.
- Those that return a questionnaire may have
different responses than those that do not
complete the form. A bias is introduced unless
you follow up with those that did not respond.
- Usually respondents do not have a chance to
ask questions about the wording of questions.
Therefore, people may interpret the questions
in different ways than you expected. You
must pretest the questions with the target
audience to avoid this problem.
When to Find More Information
Header, R. 1992. Social Research Methods. St. Paul, MN: West
Publishing Company.
Lake, C. 1987. Public Opinion Polling: A Handbook fir Public Interest
and Citaen Advocacy Groups. Washington, DC: Island Press.
National Cancer Institute. 1992. Making Health Communication
Programs Wort. A Planning Guide. Washington, DC:
National Institutes of Health.
Rice, R., and C. Atkin (Eds.) 1989. Public Communication
Campaigns. Newbury Park, CA: Sage Publications, Inc.
Wimmer, R., and J. Dominick. 1963. Mass Media Research: An
Introduction. Belmont, CA: Wadsworth Publishing Co.
121
-------
122
-------
Appendix A
Chesapeake Bay Information and Education Materials
The Chesapeake Bay Program is conducted primarily by the EPA, in conjunction
with other federal and state agencies and nonprofit organizations. The EPA gives
grants for public information and education to the Alliance for the Chesapeake Bay, a
federation of citizen organizations, businesses, scientists, user groups and citizens. The
Alliance for the Chesapeake Bay has traditionally served as the "public outreach arm"
of EPA's Chesapeake Bay Program.
For more information, contact the following:
Elliot Finkelstein
US. Environmental Protection Agency
Chesapeake Bay Program
410 Severn Avenue
Annapolis, MD 21403
1-800-523-2281
Francis Flanigan, Executive Director
Alliance for the Chesapeake Bay
6600 York Road, Suite 100
Baltimore, MD 21212
410-377-6270
The Alliance for the Chesapeake Bay also operates the Chesapeake Regional
Information Service (CRIS) at 1-800-662-CRIS. CRIS provides the public with
information about meetings and publications regarding Bay activities.
Following is a sample of the materials produced by the Alliance for the Chesapeake
Bay, some of which were completed under grants from the EPA.
Bay Journal
A monthly newspaper describing the Chesapeake Bay cleanup and Bay-related
activities and publications.
Baybook: A Guide to Reducing Water Pollution at Home
A colorful 32-page guide with cartoon-like drawings, full of useful tips on pesticides,
household chemicals, erosion, water conservation and more.
Chesapeake Bay Fact Sheet
A 2-page overview of the Bay, its history, heritage, problems, and ongoing restoration
effects.
Continued...
-------
Appendix A pages 123-127
River Fact Sheets
A series of 2-page summaries about the major rivers feeding the Chesapeake Bay,
providing an overview of the history, unique characteristics, and problems facing
some Bay tributaries.
>- York River (VA)
>• Rappahannock River (VA)
> Elizabeth River (V A)
»¦ James River (VA)
>• Piankatank River (VA)
>¦ Potomac River (VA, MD, DC, WV)
>• Anacostia River (MD, DC)
> Gunpowder River (MD)
>¦ Patuxent River (MD)
>-Chester River (MD)
> Choptank River (MD)
>¦ Severn River (MD)
>- Susquehanna River (PA, MD, NY)
»¦ Spring Creek (PA)
>- Codorus Creek (PA)
»-Conestoga River (PA)
Conodoguinet Creek (PA)
Chesapeake Citizen Directory: A Guide to Agencies and Organizations
A comprehensive directory listing hundreds of citizen groups, research institutions,
government agencies and others involved with the Bay.
White Papers
A series of in-depth technical reports focusing on issues affecting die Bay.
>¦ "Managing Growth in the Chesapeake
Region: A Policy Perspective"
>- 'Toxic Pollution and the Chesapeake Bay"
»• "Fisheries Management and the Chesapeake
Bay"
Bay Business
A quarterly newsletter for businesses and corporations in the Bay region, focusing on
issues such as pollution prevention.
124 Appandicas pages 123-133
-------
Appendix A pages 123-127
The Volunteer Monitor
A quarterly newsletter for those involved in volunteer citizen monitoring, intended to
provide a timely look at what's new in die world of citizen monitoring and to stimu-
late an exchange of ideas among those involved in various citizen monitoring pro-
grams.
Water Pollution Fact Sheets
A series of fact sheets showing how individuals and families affect the Bay by the
products used in the home and on the lawn, as well as by water use.
In addition to the public information and education materials produced by the
Alliance for the Chesapeake Bay, other agencies and organizations have produced
materials, many of which the Alliance distributes through the CRIS hotline. Listed
below are a sample of materials produced by the U.S. Fish and Wildlife Service
(USFWS), the Chesapeake Bay Foundation (CBF) and the National Aquarium in
Baltimore.
Fact Sheets (USFWS)
A series of fact sheets describing the ecology of the Bay, including: submerged aquatic
vegetation, the alewife and blueback herring, the canvasback duck, the great blue
heron, the blue crab, the peregrine falcon, the bald eagle, the osprey, the black duck,
shad, and striped bass.
A Chesapeake Bay Primer (USFWS)
A guide describing the components of the Bay that make it an estuarine ecosystem, as
well as the fish and wildlife that live there. The primer also explains the problems that
the Bay faces, including declining habitat and pollution by nutrients and toxic chemi-
cals.
Chesapeake Wetlands (USFWS)
A fact sheet describing the different types of wetlands and their environmental,
economic, and recreational benefits. The fact sheet also gives tips on what private
citizens can do to protect wetlands.
Chesapeake Bay: Its Beauty and Bounty (USFWS)
A full-color brochure containing photographs, illustrations and text highlighting fish
and wildlife of the Chesapeake Bay and the need to protect their habitats. The bro-
chure folds out to a poster of fish and wildlife of the Bay.
Chesapeake Bay Wetlands (USFWS)
A large poster map illustrating the general extent and distribution of major wetlands
and deepwater habitats in the Chesapeake Bay region.
125
-------
Appendix A pages 123-127
Bay News 2020 (USFWS and CBF)
A futuristic newspaper presenting two possible, but contrasting views of the Bay in
the year 2020. Under one scenario, the Bay has recovered due to the interest and
dedication of the public and the strong leadership of government. The other headline
outlines a doomsday scenario—the Bay is practically dead as a result of public apathy
and government inaction.
Wetlands Newspaper (USFWS)
A mini-newspaper for elementary-school students focusing on wetlands, with stories,
games, and puzzles. The newspaper describes both freshwater and saltwater wet-
lands and explains their many values to humans, fish and wildlife.
The Changing Chesapeake (USFWS and the National Aquarium in Baltimore)
A reader and activity book discussing the natural and cultural history of the Bay. The
book emphasizes changes in the Bay, its animals and plants, and the surrounding land
in the last 15,000 years. The book is designed to provide readily usable lessons for
teachers of grades 4-6.
Bay BC's (USFWS and the National Aquarium in Baltimore)
A multidisciplinary approach to teaching about the Chesapeake Bay. The curriculum
includes background material and lesson plans for teachers of grades K-3, with songs,
games, and stories for students.
The Bay Activity Kit (USFWS)
A kit containing activity cards intended to introduce middle school through adult
audiences to the major issues affecting die Bay and to lead them to additional sources
of information. Each card in the activity kit highlights an important Bay issue or
problem, and describes an activity that an individual or group can undertake to better
understand or help solve the problem.
Chessie: A Chesapeake Bay Story and Chessie Returns (USFWS)
A reader and coloring book and its sequel, written in Dr. Seuss-style rhyme. The
books educate children about Bay animals and the problems in their habitats, as well
as how human development creates problems for the Bay. Intended for students in
grades K-5.
126 App«ndic«S pages 123-133
-------
Appendix A pages 123-127
In addition to the resources listed above, the Virginia Council on the Environment
has produced a document called Chesapeake Bay Resources Directory: A Guide to
Informational and Educational Resources for Virginia's Chesapeake Bay and Its
Watershed (March, 1990). You may obtain a copy of this guide by calling CRIS at
1-800-662-CRIS or by contacting:
Council on the Environment
Commonwealth of Virginia
202 N. Ninth Street, Suite 900
Richmond, VA 23219
1-804-786-4500
127
-------
128 Appendices pages 123-133
-------
Appendix B
Puget Sound Public Information and Education (PIE) Fund
Programs
Unlike EPA's Chesapeake Bay Program, which awards grants for public informa-
tion and education primarily to one organization, Puget Sound's program spreads
grant monies out to numerous organizations throughout the region. As part of the 1987
Puget Sound Water Quality Management Plan, the Public Involvement and Education
Model Projects Fund (or PIE-Fund) was created to fund "projects that could serve as
models for public involvement and education, community cleanup, or citizen monitor-
ing of water quality of biological resources." Under the PIE-Fund, public, private, and
nonprofit organizations may submit proposals for projects. You may request a copy of
Public Involvement and Education Model Projects Fund: 47 Success Stories from
Puget Sound, which describes projects and lessons learned in the first three rounds of
grants.For more information, contact:
Puget Sound Water Quality Authority
Mail Stop PV-15
Olympia, Washington 98504
206-493-9300 or 1-800-54-SOUND
Following is a description of just a sample of the projects that have been completed.
Water Quality and Construction Practices
(Sponsor: Associated General Contractors of Washington)
Information for construction contractors about how their work on construction sites
affects water quality. A manual presenting appropriate construction practices and state
regulations pertaining to work at construction sites, and a poster highlighting proper
waste disposal and erosion control methods were produced.
Household Hazardous Waste Collection Project
(Sponsor: Metrocenter YMC A)
A program designed to educate householders on the hazards of household hazardous
waste, and to provide for collection of this waste. Flyers about each collection, a model
contract for governments and processing companies responsible for collection sites,
and a booklet on how to develop a hazard-free community were produced.
Education Project for Southeast Asian Refugees and Immigrants
(Sponsor: Nisqually Reach Nature Center)
A program designed to educate Southeast Asian refugees on shellfish and seaweed
harvesting regulations and on the dangers of eating contaminated shellfish. Work-
shops were held with both fisheries enforcement officials and refugees, and a hand-
book and slide show were produced.
-------
Appendix B pages 129-131
Soundwatch: A Travelling Exhibit on Marine Mammals
(Sponsor The Whale Museum)
A travelling exhibit, a poster, and a window decal were developed to involve more
people in safeguarding the health of marine mammals.
Water Quality Monitoring Project
(Sponsor: Green River Community College)
A student monitoring program was created, using college students to teach elemen-
tary and secondary students how to do water quality monitoring. Videotapes demon-
strating testing techniques were produced.
Lake Stevens Watershed Awareness
(Sponsor City of Lake Stevens)
A video, signs along a creek, and mailings to residents were produced in order to
develop a water quality ethic and a sense of personal responsibility for maintaining or
improving water quality in the city.
A Native American Perspective on a Clean Puget Sound
(Sponsor Squaxin Island Indian Tribe)
A mobile display board depicting past and present Squaxin fisheries and a slide/tape
presentation depicting historical and contemporary uses of the Sound by the tribe
were produced. The purpose was to provide the surrounding community with a
Native American perspective on a clean Puget Sound.
Puget Sound Citizens Network: Timber/Fish/Wildlife Implementation
(Sponsor Washington Environmental Council)
Environmental representatives were recruited and trained to work in the field along-
side other professionals in forest management. Forest practices were modified and a
process for citizen involvement in resources issues was established, all in an effort to
protect water resources from harmful forest practices.
Children of the Sound
(Sponsor The Resource Institute)
A slide show for learning-disabled children and children-of-color, a series of informa-
tion sheets for parents, and a curriculum for children with special needs was prepared
to encourage these children to become advocates for clean water.
Sound Experience
(Sponsor The Marine Science Society of the Pacific Northwest)
A floating environmental education program was established, using a shipboard
classroom to teach an appreciation of Puget Sound. A pre-trip packet was prepared
for teachers, arid a program using learning stations aboard the ship were designed.
This project resulted in the creation of a new nonprofit environmental education
organization.
Business Partners for Qean Water
(Sponsor Bellevue Storm and Surface Water Utility)
Workshops were used to educate business on reducing sources of water pollution.
130 Appendices pages 123-133
-------
Environmental Radio
(Sponsor Environmental Perspectives)
A series of half-hour radio programs on water quality issues was created.
Canal Cleaners
(Sponsor Mason County Department of Health Services)
A house-to-house travelling water quality education workshop for shoreline and
upland homeowners was created.
Boundary Bay
(Sponsor. Point Roberts heron Preservation Committee)
Environmental education activities related to the Boundary Bay ecosystem were
created for schools.
-------
Appendix C
How to Test for Readability
Tht SMOG Readability Formula
To calculate the SMOG reading grade level,
begin with the entire written work that is being
assessed, and follow these four steps:
1. Count off 10 consecutive sentences near the
beginning, in the middle, and near the end of
the text.
2. From this sample of 30 sentences, circle all
of the words containing three or more
syllables (polysyllabic), including repetitions of
the same word, and total the number of words
circled.
3. Estimate the square root of the total number
of polysyllabic words counted. This is done by
finding the nearest perfect square, and taking
its square root.
4. Finally, add a constant of three to the square
root. This number gives the SMOG grade, or
the reading grade level that a person must
have reached if he or she is to fully under-
stand the text being assessed.
A few additional guidelines will help to clarify
these directions:
• A sentence is defined as a string of words
punctuated with a period (.), an exclamation
point (!) or a question mark (?).
• Hyphenated words are considered as one
word.
• Numbers which are written out should also be
considered, and if in numeric form in the text,
they should be pronounced to determine if
they are polysyllabic.
• Proper nouns, if polysyllabic, should be
counted, too.
• Abbreviations should be read as
unabbreviated to determine if they are
polysyllabic.
Not all pamphlets, fact sheets, or other
printed materials contain 30 sentences. To test a
text that has fewer than 30 sentences:
1. Count all of the polysyllabic words in the text.
2. Count the number of sentences.
3. Find the average number of polysyllabic
words per sentence as follows:
«"»«•=^imrfwords
4. Multiply that average by the number of
sentences short of 30.
5. Add that figure on to the total number of
polysyllabic words.
6. Find the square root and add the constant of
3.
Perhaps the quickest way to administer the
SMOG grading test is by using the SMOG
conversion table. Simply count the number of
polysyllabic words in your chain of 30 sentences
and look up the approximate grade level on the
chart.
An example of how to use the SMOG
Readability Formula and the SMOG Conversion
Table is provided on the following page.
132 Appendices pages 123-133
-------
We have calculated the reading grade level
for this example. Compare your results to ours,
then check both with the SMOG conversion
table:
Readability Test Calculations
lotal Number of Fblysyllabic Words =38
Nearest Perfect Square =36
Square Root = 6
Constant = 3
SMOG Reading Grade bevel = 9
SMOQ Conversion Ifcble*
Total Polysyllabic Approxlmata Grada
Word Counts Laval (*1.5 Grades)
0-2 4
3-6 5
7-12 6
13-20 7
21-30 8
31-42 9
43-56 10
57-72 11
73-90 12
91-110 13
111-132 14
133-156 15
157-182 16
183-210 17
211-240 18
'Developed by. Harold C. McGraw, Office of Educational Research,
Baltimore County Schools, Towson. Maryland.
------- |