xvEPA
United States
Environmental Protection
Agency
EPA 842-B-94-009
November 1994
Office Of Water
(4504F)
Measuring Progress Of
Estuary Programs
Highlights
Recycled/Recyclable
Printed with Soy/Canola Ink on paper that
contains at lecst SQ% rocydod fiber
-------
-------
IVJLeasuring
Progress of
Estuary Programs
HIGHLIGHTS
-------
-------
.STUARY
An Estuary is a coastal area where fresh water
from rivers and streams mixes with salt water
from the ocean. Many bays, sounds, and lagoons
along the coasts are estuaries. Portions of rivers
and streams connected to estuaries are also con-
sidered part of the estuary. The land area from
which fresh water drains into the estuary is its
watershed and affects the health'of the estuary.
o
utcome
An Outcome is an action or occurrence that hap-
pens outside the estuary protection program but
that is likely to have occurred at least in part
because of an estuary protection activity.
I
ndicator
An Indicator is a particular characteristic or refer-
ence marker used to measure whether an out-
come is being achieved.
-------
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
JAN 1 7 1995
THE ADMINISTRATOR
Dear Colleagues:
Estuaries -- the coastal areas where fresh water from rivers
and_streams mixes with salt water from the ocean -- are a major
national resource and economic asset in this country. The
protection of the 100 estuaries in the U.S. is a vital national
concern.
Through the National Estuary Program (NE-P), the U.S.
Environmental Protection Agency brings together citizens,
business representatives, scientists, state and local officials,
and environmental organizations to arrive at common-sense plans
for protecting 21 of these vital coastal areas. These plans,
called Comprehensive Conservation and Management Plans, recommend
actions to achieve specific goals.
This manual is intended to help managers in the National
Estuary Program to assess environmental progress and communicate
to citizens about efforts undertaken and successes achieved. An
informed and involved citizenry is essential to the success of
environmental protection.
Protecting estuaries means controlling pollution from a
great variety of sources --in the air, in the water, and on
land. For this reason, the National Estuary Program provides an
important model as our nation moves beyond piecemeal, pollutant-
by-pollutant regulation to more a comprehensive "ecosystem"
approach to environmental protection.
The procedures presented in this manual for assessing
environmental trends and communicating with the public have been
developed and field-tested by the Urban Institute of Washington,
D.C. We expect they will be useful both within the National
Estuary Program and for managers of watershed protection and
other environmental efforts outside the NEP.
Sincerely,
Prlntedvn Recycled Paper
-------
-------
A
cknowledgments
The procedures described in this manual were devel-
oped under Cooperative Agreement #CX819149-02-3
with the Urban Institute, Washington, D.C. We wish to
thank Harry Hatry, Blaine Liner, and Shelli Rossman of the
Urban Institute for bringing their expertise and knowledge in
the field of program evaluation to the National Estuary Pro-
gram (NEP). The Urban Institute worked with the Buzzards
Bay and Tampa Bay National Estuary Programs to pilot test
these procedures. For their help in ensuring the procedures
will be most relevant to other estuary programs special thanks
is extended to Dr. Joseph Costa, Director of the Buzzards Bay
NEP and Richard Eckenrod, Director of the Tampa Bay NEP.
At each pilot location, local advisory groups of NEP staff and
advisory committee members assisted in the development and
testing of the procedures. All together, probably well over 100
-------
persons were involved in the preparation of this manual.
While space does not permit listing all of them, their important
roles are acknowledged, with our thanks. In addition, a na-
tional steering committee representing state and local agen-
cies, academia, industry, and public interest organizations,
helped guide the development of the procedures and re-
viewed the draft manual. Following is a list of the members of
our steering committee whom we thank for their guidance:
Committee Members
Mr. Robert Bendick, New York State Department of Envi-
ronmental Conservation
Mr. Jeff Benoit, U.S. Department of Commerce
Mr. Hal Bickings, Aquaculture Natural Resources
Dr. Rick Burroughs, University of Rhode Island
Dr. Joseph Costa, Buzzards Bay National Estuary Pro-
gram
Mr. John Costlow, Duke University
Mr. Richard Eckenrod, Tampa Bay National Estuary Pro-
gram
Mr. BUI Eichbaum, World Wildlife Fund
Ms. Eugenia Flatow, Coalition for the Bight, New York
Mr. Kevin P. Gildart, Bath Iron Works, Maine
Mr. Andy Manus, Delaware Department of Natural Re-
sources and Environmental Control
Ms. Nancy McKay, Puget Sound Water Quality Author-
ity
Mr. Steve Ritchie, California Regional Water Quality
Control Board
Ms. Dixie Sansom, Former Florida State Representative
Ms. Amy Zimpfer, U.S. Environmental Protection
Agency
-------
Ex-officio
Jim Burgess, U.S. Department of Commerce
Elizabeth Jester-Fellows, U.S. Environmental Protection
Agency
Marian Mlay, U.S. Environmental Protection Agency
Finally, we extend our gratitude to the Richard King Mel-
lon Foundation of Pittsburgh, Pennsylvania, and to the Mary
Flagler Gary Charitable Trust, for their generous siupport of
the Agency's efforts to help its estuary programs evaluate their
progress.
-------
-------
c
ontents
Identifying Meaningful Outcome Indicators 3
Tracking the Actions of Governments: The
Government Action Checklist 4
Tracking the Estuary Protection Behavior of
Businesses 11
Tracking the Environmental Behavior of Households
and Boaters 13
How to Conduct Household Surveys about
Environmental Behavior 14
Developing a Bay Quality Index 17
Cost 21
-------
Analyzing and Reporting Outcome Data 21
Caveats 23
Exhibits
Estuary Protection Chain of Events 4
Illustrative Outcome Indicators 5
Illustrative Industry Questionnaire 12
Illustrative List of Potential Household and Boater
Behavior Problems to be Covered in Surveys 14
Illustrative Report Format for Household Survey
Responses (Hypothetical Data) 16
Summary of Survey Findings: Tampa Bay Household
Environment Survey, 1992/1993 17
Examples of Components for a Bay Quality Index
Focused on Condition of Living Resources (A
Fourth-Order Outcome) 18
Examples of BQI Components and Their Weights 19
Bay Quality Index by Section of Tampa 20
Illustrative Outcome Indicators: Overall Summary 22
-------
M,
.easurmg
Progress of
Estuary Programs
HIGHLIGHTS
The roughly 100 estuaries in this country are a major
national resource and economic asset and their protec-
tion is a vital national concern. Organizations in many
of these estuaries have established programs aimed at their
protection. Twenty-one estuaries now participate in the U.S.
Environmental Protection Agency's National Estuary Program.
The 1990s have become the decade of "accountability."
Federal and state governments have been required by legisla-
tion to monitor and report on program results or progress.
Common sense dictates that estuary protection officials regu-
-------
larly track the results or progress of their efforts at improving
and maintaining the quality of estuary waters. To the extent
that many estuary programs use similar procedures for assess-
ing their progress, such information can also provide a national
perspective on progress in estuary protection.
This report is a companion document to "Measuring Pro-
gress of Estuary Programs: A Manual" and provides a sum-
mary of how to develop and implement such an outcome
monitoring process. Specifically, we examine procedures for
regularly tracking progress of estuary protection efforts un-
dertaken by governments, businesses, households, and boat-
ers. We also present procedures for translating information on
actual bay quality into a "Bay Quality Index" developed from
available data on environmental monitoring and other sources
such as closures to shellfishing and swimming.
Estuary program officials need information to help deter-
mine where improvements are needed and whether improve-
ment efforts have led to desired results. Such information will
increase the accountability of estuary protection programs to
elected officials and the public, help develop and justify
budget requests, and help communicate to citizens and the
media the progress being made.
Monitoring the status of an estuary is a complex undertak-
ing. Measuring water and living resource quality at all times,
in all locations, and at all depths is infeasible. However, a
desire for perfection and scientific precision should not deter
the use of outcome indicators and data collection procedures
that provide "roughly right" information. We emphasize that,
for estuary program officials, partial knowledge about pro-
gress is better than no knowledge.
The material presented here is based to a large extent on
pilot efforts undertaken by the National Estuary Programs in
Buzzards Bay, Massachusetts and in Tampa Bay, Florida in
1992 and 1993. The outcome monitoring procedures described
were developed over a 14-month period by Urban Institute
staff and personnel from both estuary programs.
2 MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
For estuaries participating in the National Estuary Pro-
gram, the required Comprehensive Conservation and Man-
agement Plan defines the purpose and component parts of the
estuary protection program and the aims of monitoring its
programs. Other estuaries likely have, or will have, a manage-
ment plan to which the monitoring system described in the
manual can be related.
Below, a discussion of how to identify meaningful outcome
indicators is presented, and we set forth a way to track pro-
gress of estuary protection efforts of governments, businesses,
households, and boaters. We then suggest how to analyze and
report the outcome data collected, and caution readers about
the limitations of the outcome monitoring process outlined in
this report.
IDENTIFYING MEANINGFUL OUTCOME
INDICATORS
The word "outcome" is used throughout the manual to refer
to an action or occurrence that happens outside the estuary pro-
tection program but that is likely to have occurred at least in
part because of an estuary protection activity. When designing
an outcome monitoring process, the first step is to identify
outcome indicators, being careful to divide them into interme-
diate and end outcomes. Intermediate indicators typically
mark some action or behavior undertaken by persons or or-
ganizations that can affect bay quality but do not represent the
end results, such as the condition of living resources in the bay.
As shown in the chart on the next page, which maps the
estuary protection chain of events, four categories or orders of
outcome indicators are especially relevant to estuary protec-
tion activities.
Information about the change in behavior on the part of
governments, businesses, households, and boaters can be ob-
tained through the Government Action Checklist and the
business, household, and boater surveys described below.
Highlights 3
-------
Eptuary Protection Chain of Events
FIRST ORDER
Implementation ol
Estuary Protection
Actions (by Gov-
ernments, House- —
holds, Boaters.
and Businesses)
Organizational and
Individual Outcomes
INTERMEDIATE
OUTCOMES
SECOND ORDER
Reduction of
Discharges (into
Soil)
Pollution Reduction
Outcomes
THIRD ORDER
Improvements
in Water and
Quality
Bay Quality
Outcomes
< END 1
OUTCOMES
FOURTH ORDER
Improved Health
of Living Re-
•* sources and of
Quality of Life
Living Resource
and Environmental
Use Outcomes
»
Data on the health of living resources—the extent to which the
health of fish, other wildlife, habitat, and vegetation has
changed—are of particular concern to the public because they
indicate whether or not the estuary can support individual
uses such as swimming and shellfishing.
The table on the following page presents an illustrative set
of outcome indicators that estuary protection officials can use
to measure progress in estuary protection efforts.
TRACKING THE ACTIONS OF GOVERNMENTS:
THE GOVERNMENT ACTION CHECKLIST
The Government Action Checklist (GAC) is a vehicle for an-
nually monitoring the overall progress that local governments
are making in their efforts to protect an estuary. The Tampa
Bay National Estuary Program (NEP) is using the GAC proc-
ess to develop a schedule for its Comprehensive Conservation
and Management Plan (CCMP); having already completed its
CCMP, the Buzzards Bay Project is using the GAC process to
help measure progress in meeting the goals of the CCMP.
MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
«s
1
«
u
^
^
w
s
1
o
p>
•42
^
*^A
s
>•— i
%
g
0
en
Q
O
w
§g
H
£ ^ as 'P 01
O ^% ^>> _ *r* ^
*r*^ cu ^>- y) ,_j o
S 3 ^ 1" | 8, §| § u
C r^" O 3 en PH (0 -M* a3 ^
i^cnSS fi^'s2§
O ffi t§ pa < U U
i-i T-H r-H (M CN| CO •rf
. 43
§ "O T3 CD
W S S "S 2
| 111 1^ 1
1 III II £
ID
£ en
1 o I
0 ? %
^^ TO t . ?
5 ^5 o °
0) 0) -c< T3
«) .g Jl > H
£« •*-* r* a3 ro
O d S ^ N
Overall Summary Outcome Indicat
Governments scoring 75% success o:
Action Checklist"
Households with at least one proble
Boaters with at least one problem be
Businesses reporting reduction in hz
discharged
Amounts of pollutants discharged
Overall water quality
Overall bay quality
& ar sr
t t t
3 S S
en cn cn
-------
1
| Illustrative Outcome Indicators (contin
Wl
U
0
en
1
0
M^ m^
O p
£-< co
Z u3
^ S
INDICATOR
S
Q
FJ5
>! w
H §
^ &C) 43 43 42
10 ^ "e3 Q Q Q
.S SI) S u ]y gj
S "S 43 42 43
pa [£< en en en
i— i eg eg ^ •<*
« «
43 43
-------
>>
^ t x £
> CD > Q «
3 tn 3 ,— i J?
CD 0) CD ni Q
r \ G r \ "
QJ .3 QJ ft) OJ
< 3 < "g |
U pa O fi< CD
i— i i— i i— i fV) CM
in
pa
O3 O -^ vj_,
E-1 pa E— ' D o
"S "o "S "o S3
ix i_( ^ ^ f>
QJ QJ QJ QJ c in
11 ll||
J3 J3 0 J3 I^H (^j
22 22 U
•S
in CD
tn t-t >-i »>
•*- SH fl) S
•i o — ^
Managing Sewage and Septage
Nitrogen-loading limits added to NPDES perm
Businesses adopting pre-treatment and pollutit
prevention programs
Towns with program for licensing septage hau
Days businesses did not meet npdes limits and
substantial non-compliance
Untreated sewage discharged directiy into bay
in
1
1/3 $
>, X * X &
0 QJ cS QJ c^
5 £3 W) 5 t>0
CD CB .£ CS E-i E-i E-i "3
'o tS "o "o ^
)-l ^ l_ i-, if
QJ CL) d) QJ CD
<— i f^ f^ f*> r*^
£ £ £ £ 1
33 3 33
22 222
1
0 C
^? ^^ (73
Managing Stormwater and Agricultural Rune
Remedial Stormwater BMPS in place
Educational measures for farmers developed tc
agricultural runoff
Bylaw changes for new developments (subdivi
regulations)
Remediation of illegal septic tie-ins
Sewer overflows
x
3 3 CD 3 3
CD CD »-, CD CD
U U -g U U
< < § •< <
O U pa O O
r-l r—l *— 1 iH r-H
tn in in in
C C C G
£ & & £
o o o o
HE- [-1 E-i
*S "tS §J) 'o "o
S3 S3 5 S3 S3
11111
3 3 Si 3 3
^ ^ S3 "^ ^
Z Z PH Z Z
m
Managing Boat-related Wastes
Identification of no-discharge areas
Availability of pump-out facilities0
Boaters who report using pump-outs regularly
jxeguiauon of MSD chemical additives
Enforcement actions taken for illegal discharge;
^
•I
Js
Highlights
-------
43 bo
O O w
CO
seagras
a
-------
There are six basic steps to formulating a Government
Action Checklist:
1. Select the items to be assessed, that is, the set of activi-
ties that governments should undertake to maintain
and improve the estuarine environment. Then group
the items by useful categories of protection.
For instance, the Buzzards Bay Project used these categories:
• Nitrogen Action Plan,
• Protecting and Enhancing Shellfish Resources,
• Controlling Stormwater Runoff,
• Managing Sewage From Boats,
• Managing On-Site Wastewater Disposal Systems,
• Preventing Oil Pollution,
• Protecting Wetlands and Coastal Habitat,
• Planning for a Shifting Shoreline,
• Managing Sewage Treatment Facilities,
• Reducing Toxic Pollution,
• Conducting Pollution Remediation Projects in
New Bedford.
The Tampa Bay NEP used these categories:
• Water Quality,
• Land Use,
• Septic Systems,
• Municipal Waste,
• Hazardous Materials and Hazardous Wastes,
• Stormwater,
• Conservation Measures,
• Boater Use and Boater Discharges,
• Oil Pollution and Emergency Response,
• Agricultural Industry,
• Other Industry,
• Shellfishing,
• Public Information/Technical Assistance,
Highlights 9
-------
• Intergovernmental Coordination,
• Miscellaneous.
2. Determine which communities will be included in the
regular GAC assessments, that is, which local govern-
ments and at what level (city, county, etc.). For example,
the Tampa Bay NEP decided to start with three major
counties and three large cities, recognizing that some
counties encompassed as many as 22 municipalities—a
number thought to be unwieldy for the GAC process.
Moreover, many of the counties were thought to have
small impacts on bay quality. A number of the items on
the Tampa Bay GAC refer to county-level activities that
are not applicable to the towns. In contrast, the Buz-
zards Bay Project has targeted GAC monitoring for
each of the communities that were signatories to the
CCMP. The procedure currently covers 12 local govern-
ments. The Coalition for Buzzards Bay developed three
versions of the GAC: one for coastal towns, one for inland
towns, and a separate checklist for New Bedford—the
only large city in the local area and its industrial manu-
facturing hub.
3. Operationally define checklist items for assessment pur-
poses, and develop a rating scale for each item that
captures whether local governments have achieved the
intended activity/action (e.g., achieved fully, partially,
or not at all).
4. Select the organization that will administer the ratings
and develop the summary reports. The organization
should be one that is perceived as being reasonably
objective by the governments being rated and by the
public. A regional council of governments, planning
association, or university are probably appropriate.
5. Pilot test the assessment procedures. Rarely will the
first version of the GAC survive unmodified as it un-
dergoes a pilot test among selected local communities.
10 MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
6. Determine the data analysis process and reporting for-
mats.
TRACKING THE ESTUARY PROTECTION
BEHAVIOR OF BUSINESSES
Businesses and industries, including agriculture, are often
major contributors of pollutants to an estuary. Thus, they
provide major opportunities for pollutant reduction. Progress in
encouraging business and industry to adjust their operations in
order to produce less hazardous polluting materials should be
tracked, as should the actual magnitude of pollutants emitted.
Three ways to track progress in business-related estuary-
protection efforts include monitoring the data on pollutant
loadings, conducting periodic surveys of businesses concern-
ing their efforts to reduce hazardous waste, and—by using a
Government Action Checklist—looking at the extent to which
local governments urge businesses to use alternatives to haz-
ardous materials, sponsor industrial pre-treatment programs,
and implement agriculture pesticide/fertilizer best manage-
ment practice (BMP) programs.
Before conducting a survey of businesses, estuary program
officials need to resolve the following questions: Which busi-
nesses should be included? Which geographic areas should be
covered? What topics should the surveys cover? Should local
business groups be asked to help shape the questionnaire?
Should different versions of the questionnaire go out to different
types of businesses? Should the questionnaire be administered
in person, by phone, or by mail? How many businesses should
be surveyed and how can a high response rate be ensured?
Questionnaires should be short, accompanied by a letter
introducing the purpose of the survey, and should not be
unduly complex or intrusive. Response rates can be increased
by preceding the questionnaire with a substantial educational
effort. The sample questionnaire on the next page was adapted
Highlights
11
-------
Illustrative Industry Questionnaire
(Based on one tested in New Bedford, Mass)
1 . Does your firm have a current toxic material/
waste reduction plan?
2. Does your firm have specific toxic material/
waste reduction targets?
3. Do you believe that you and your firm have suffi-
cient and dear information about the following:
a. Toxic waste requirements?
b. Technical information on the toxic
materials that your firm uses?
If yes to either (a) or (b), what information
do you need?
4. In the past 12 months have you done any of the
following:
a. Sent personnel to workshops or training
sessions that contained a significant toxins
reduction component?
If yes, approximately how many employee-
days were spent?
b. Organized employee teams to work on
toxins/hazardous waste reduction?
c. Conducted an in-plant audit or assessment
that contained a significant toxins reduc-
tion component?
If yes, who did the audit/assessment
(check all that apply):
Q Your own staff
Q A consultant paid by your company
Q State personnel
D Others Cplease specify)
d. Initiated a recovery and reuse program for
any item containing toxic/hazardous
material?
If yes, what toxic/hazardous material?
a Yes
a Yes
a Yes
Q Yes
a Yes
Q Yes
a Yes
Q Yes
a NO
a NO
a NO
a NO
a NO
a NO
a NO
a NO
12
MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
e. Eliminated or replaced the use of any item
that contained toxic/hazardous material?
f. Changed the way you handle toxic
hazardous wastes (such as by a pre-treat-
ment or disposal method) so as to reduce
the amount of waste discharged into the
water and air?
5. Overall, have any of the above activities, or simi-
lar ones that you have used, actually enabled you
over the past 12 months to actively reduce your
company's:
a. Use of toxic/hazardous materials?
b. Amount of toxic/hazardous materials
discharged into the water and air?
6. How have these toxic/hazardous waste reduction
strategies over the past 12 months affected your
company's overall costs per unit of product
(considering both operating and capital cost)?
Please check one:
Q Increased overall cost per unit of product
produced
Q Decreased overall cost per unit of product
produced
Q Neither
Q Don't know
7. What training and technical assistance, if any,
would you like for your firm?
Q Yes
Q Yes
Q Yes
Q Yes
Q No
a NO
a NO
a NO
from one prepared by the Buzzards Bay Project/Urban Insti-
tute team after much research and input from EPA staff and
local officials.
TRACKING THE ENVIRONMENTAL BEHAVIOR
OF HOUSEHOLDS AND BOATERS
Households and boaters play major roles in estuary protec-
tion. Their actions can hurt or help the estuary's condition.
Highlights
13
-------
Households affect estuary quality by their use of fertilizers and
pesticides, disposal of contaminants such as paint and cleaning
materials, and upkeep of private septic systems. Boaters affect
estuary quality by dumping waste, mishandling fuel, and
damaging submerged aquatic vegetation with their propellers.
Improvements in behavior can be expected to lead to re-
duced amounts of contaminants discharged into the waters of
the estuary and subsequently to improved water quality and
life conditions for animals and vegetation. The best way to
assess the status of, and changes in, household and boater
estuary protection behavior is to survey them periodically
about their environmental protection-related activities.
When examining changes in household and boater behavior
from year to year, small observed differences are not likely to
be as important to estuary program officials as large differ-
ences. This means that large household and boater samples
yielding a high degree of precision are not likely to be needed.
Illustrative List of Potential Household and Boater
Behavior Problems to be Covered in Surveys
Handling of fertilizers;
Handling of pesticides and herbicides;
Handling of used oil, paint, and household chemicals;
Lawn-watering;
Handling of waste while boating;
Handling of pet wastes;
Type of mulch used;
Cleaning of on-site septic systems;
Storm water runoff control;
Water use.
How to Conduct Household Surveys about
Environmental Behavior
1. Prepare questionnaire to be administered to house-
holds and consider hiring a local survey organization
14 MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
to conduct the survey. Pre-test the questionnaire on a
small number of households or boaters representing a
cross-section of households to whom the questionnaire
will be administered. In its household survey, the Tampa
Bay NEP sought information on household composi-
tion, whether respondents owned or rented their homes,
county of residence, and whether respondents lived in
an apartment, condominium, or mobile home as dis-
tinct from a house—in order to exclude that group for
questions relating to lawn fertilizers, pesticides, mate-
rials used for walk areas, and so on. i
2. Identify appropriate listings of the household popula-
tion and draw a random sample of households;
3. Consider ways to increase survey completion rates.
These include keeping the questionnaire or telephone
interview short, designing a user-friendly question-
naire, attaching a cover letter to the questionnaire that
explains its purpose and is signed by a community
person with favorable name recognition, using second
and third mailings as well as follow-up phone calls, and
offering an incentive such as a free publication for
completing and returning the questionnaire.
4. Administer questionnaire by a combination of mail and
telephone;
5. Track returns and carry out a second mailing or phone
call to non-respondents about three weeks after the first
mailing;
6. Conduct telephone reminders or phone interviews to
non-respondents in order to achieve the targeted re-
sponse rate (we suggest 50 percent);
7. Tabulate responses;
8. Prepare report on findings, including both the numeri-
cal results and a summary of the main findings. An
illustrative report format for household survey responses,
using hypothetical data appears on the following page.
Highlights
15
-------
Illustrative Report Format for Household
Survey Responses (Hypothetical Data)
Percentage of respondents that reported disposal, in the past 12
months, of motor oil, paints, or chemicals at (a) county collec-
tion sites; (b) local service stations; (c) into sewer or storm
drains.
Total Number of
Respondents
Percentage of
Total
County A
County B
County C
One-Person
Household
Two-Person
Household
Three-Person
Household
Owners
Renters
Number of
Respondents
200
100%
60
75
65
35
65
100
120
80
DISPOSITION
County
Collection
Sites
90
45%
67%
27%
31%
43%
38%
50%
67%
13%
Local
Service
Stations
70
35%
25%
53%
38%
43%
39%
30%
25%
50%
Sewer or
Storm
Drains
40
20%
8%
20%
31%
14%
23%
20%
8%
37%
An example of how the Tampa Bay NEP summarized its
main household survey findings appears on the following
page. This format can also be used for reporting the findings
of the business survey.
16
MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
Summary of Survey Findings: Tampa Bay
Household Environment Survey, 1992/1993
Forty-two percent either do not have or do not know if they have
water flow saving devices.
Forty-five percent do not have a displacement device in their toilet;
another 11 percent do not know if they do or not.
Twenty-five percent of those surveyed who had disposed of oil, paint,
or chemicals in the past 12 months reported disposing of them in
their sewer or outside storm drain.
Seventy-two percent are separating their trash for recycling; 25 per-
cent are not.
Twenty-four percent of dog owners never pick up their animal
wastes; 14 percent do it "sometimes."
While only 9 percent do not use slow-release fertilizers, 30 percent
do not know if they do or not.
Forty-eight percent use pesticides to control yard pests; 50 percent
use pruning.
Seventy-three percent have mostly, or some, native or drought resis-
tant plants and trees; 17 percent are not sure if they do or not.
Regarding new foliage planted, 21 percent said they are not drought
resistant varieties; another 29 percent said they didn't know if they
were or not.
Twenty-six percent said they do not group plants according to fertil-
izer/watering needs; another 31 percent did not know if they did or not
Although only 7 percent have septic systems, of these, 67 percent pump
out residues later than the recommended 3 to 5 years. Additionally, 67
percent use cleaning compounds and 34 percent use a garbage disposal.
DEVELOPING A BAY QUALITY INDEX
The purpose of a Bay Quality Index (BQI) is to provide a
sound, reliable composite indicator of an estuary's overall
health. The index should be clear and useful to public and
private officials and to the public at large. It should permit the
monitoring of trends in estuary quality over time, help identify
aspects of bay quality that need extra attention, and enhance
the ability of estuary program officials to communicate needs
and progress to citizens and the media. A BQI is not intended
as a scientific tool but as a management and policymaking tool.
Highlights
17
-------
Federal and state governments are already using a variety of
environmental indices. For example, the State of Ohio has devel-
oped an "Index of Biotic Integrity." This uses data on the number
of different types of species and the total number of species to
produce an overall biological assessment of the quality of
streams. The Hillsborough County Environmental Protection
Commission in Florida has been annually updating a water
quality index. It was used as a starting point for both the Buzzards
Bay Project and Tampa Bay NEP to develop their Bay Quality
Indices. The Hillsborough index also serves as a model for some
of the procedures discussed in this report.
A BQI is likely to re-
ceive special attention
from the media because
of its apparent simplic-
ity/just as overall air pol-
lution indices in cities
have achieved consider-
able acceptance by the me-
dia, the public, and even
the technical community.
Four steps are needed
to develop a BQI:
1. Select the compo-
nents to include in
the index. Exam-
ples of compo-
nents for a BQI
focusing on condi-
tion of living re-
sources are pre-
sented to the right.
2. Determine the way
in which these
components are to
be combined into
Examples of Components
for a Bay Quality Index
Focused on Condition
of Living Resources
(A Fourth-Order Outcome)
Extent of shellfish bed closures or
restricted use/consumption advi-
sories;
Extent of beaches closed to bathing
or having other restrictions due to
unnatural causes;
Acreage of seagrasses;
Number of fish kills;
Fish populations;
Populations of selected mammals
and birds;
Extent of algae blooms;
Extent of offensive noxious odors;
Prevalence of fish abnormalities due
to unnatural causes, based on tissue
samples;
Number of cases of human illness
due to consumption of contami-
nated seafood;
Citizen ratings of estuary attrac-
tiveness.
18
MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
the index. This involves transforming the values for
each component into a common unit of measurement,
combining these normalized values into an overall index
value, and applying "weighting factors" that reflect each
parameter's importance as an indicator of the health of
the bay. The respective weights can then be multiplied
by each of the "normalized" values of the BQI compo-
nents and added together to produce the latest value of
the BQI. The weights used in the Hillsborough County
index are as follows:
Example of BQI Components and Their Weights
PARAMETER
Percentage of Saturation Dissolved Oxygen
Chlorophyll "a"
Total Coliform
Effective Light Penetration
Total Phosphorus
Total Kjeldahl Nitrogen
Biochemical Oxygen Demand
Sum of Parameter Weights
WEIGHT
.212
.167
.167
.111
.111
.111
.111
1.000
Source: Richard Boler, ed. 1992. "Surface Water Quality, Hillsborough
County, Florida, 1990-1991." Tampa, Fla.: Hillsborough County
Environmental Protection Commission (September), p. A-2.
3. Identify those values of the index that represent the
various levels of bay quality such as excellent, good,
fair, or poor.
4. Select the sub-indices likely to be useful. These include
sub-indices for various geographic segments of the
estuary and for groups of components such as living
resources, support of desired uses of the bay, and water
quality. While an overall index for the estuary is of
interest to elected officials and the public, the sub-indi-
ces and individual environmental indicators are more
useful to government and private sector personnel. The
Highlights
19
-------
exhibit below illustrates an attractive way to report
geographical information. Such a map can also include
data on sub-indices such as the condition of fish and
support for desired uses.
Bay Quality Index by Section of Tampa
\
GENERAL WATER
QUAUTY INDEX
1991
EXCELLENT
GOOD
> FAIR
O POOR
• UNDESIRABLE
20 MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
Cost
Environmental monitoring can be very expensive for a local
program. Estuary programs will need to concentrate scarce
resources on those index components, locations, and times
that represent key "pressure points" for the estuary.
The ultimate test for determining the worth of the time and
cost required is whether estuary protection personnel are able
to use the information to help them more effectively allocate
resources for estuary protection and to communicate bay qual-
ity outcomes to the public.
ANALYZING AND REPORTING OUTCOME! DATA
Outcome data that have taken time, effort, and resources to
collect are useless unless appropriately analyzed and effec-
tively presented. In the manual we suggest specific ways to
analyze and report the findings from each data collection effort
described—the Government Action Checklist, and the busi-
ness, household, and boater surveys. Estuary protection pro-
grams will want to combine the data obtained from these
efforts by using a format like the one shown below, and
compare the findings from the various outcome indicators
against the goals identified in the estuary program's long-
range plan.
A "Summary of Survey Findings," similar in format to the
summaries recommended for each separate data collection
effort and drawn from those summaries, should do two things.
It should report the extent to which targets set by the estuary
program have been achieved during the current reporting
period, and it should compare the outcome information for the
current reporting period to that for the previous period(s).
Estuary program and community officials will likely find
the outcome information more useful if the Summary of Sur-
vey Findings is accompanied by explanatory information to
help identify factors believed to have contributed significantly
to the main findings.
Highlights
21
-------
£
s
g
a
CD
5
Ci
O
e Indicators:
S
0
u
4«i
Illustrative On
P^ fr"H
ft|g
-P
EG w
P O 5
M S "5
£%*
w
p
§
UNIT OF
MEASURE
INDICATOR
I-(
Number of
Governments
"5 in
<4H IS
ments That Have Met at Least 75% c
s on the "Government Action Ched
F
O *"*
U
^
Percent of
Respondents
i
olds With at Least One Problem Bet
*
in
S
PC
t-H
Percent of
Respondents
,_,
With at Least One Problem Behavio
a
S
S
pa
eg
Number
in
3
ses Reporting Reduction in Hazardc
e Discharged
gj •&
pa
CM
Various Units
fi
.5
ts of Various Pollutants Discharged
ay
C (O
I1
CO
Water Quality
Index
Water Quality
;=j
O
"*
Bay Quality
Index
Bay Quality
£3
6
nts/Explanatory Information:
r*
a
22
MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
CAVEATS
Readers should be aware of several limitations to the outcome
monitoring process outlined in the manual. These are listed
below.
Lack of Causal Information. Outcome data provide a "score-
card" but do not generally provide a sound basis for1 praising
or criticizing estuary protection activities. Outcome informa-
tion is vital for understanding what has been happening and
what progress is being made, and for helping to identify where
future attention and resources are needed. But to obtain infor-
mation on why the outcomes are as they are, or why changes
over the last reporting period have occurred, estuary person-
nel will need to conduct more in-depth examinations.
The Need for Periodic Outcome Information. One year of
information will serve as baseline data for comparisons for
future years. But it will be of somewhat limited value by itself
in identifying trends, unless the estuary program has been fortu-
nate enough to have prior-year data on outcome indicators.
The Delayed Effects of Program Actions. Some estuary protec-
tion actions cannot be expected to lead to quick, substantive
changes. For example, major problems caused by the presence
of toxins or excessive nitrogen may take years to correct. Both
the source of the problem as well as accumulated damages
must be corrected.
Cost. Establishing an outcome monitoring process requires
staff time and time on the part of many advisory groups
representing citizens, technical experts, and special interests
in the community. However, once the procedures become
routine, the effort required should be reduced considerably.
Implementing an outcome monitoring process is hard work
and is not something that an estuary program can do over-
night. It will minimally take a year to establish such a. process,
and probably two to three years to fully implement it. Estuary
Highlights
23
-------
protection, however, is a long-run proposition. Doing it well
requires continuous, long-term attention.
Ultimately, an outcome monitoring process needs to stand
the test of cost-effectiveness. If the information is not used to
help estuary program officials improve their work, or if it does
not provide funders or the public with relevant information
that they can understand and utilize, the results will not be
worth the costs. It is therefore imperative that estuary program
staff design and implement an outcome monitoring process
with great care, keeping in mind at all times that the purpose
of the endeavor is to provide officials and the public with clear
and meaningful information that they can use to improve
estuary protection.
24 MEASURING PROGRESS OF ESTUARY PROGRAMS
-------
-------
s P
0.5
•n w
o> .^
o> S
(O >
r
------- |