OOOR83101
The Model Implementation
Program
Lessons Learned From Agricultural
Water Quality Projects
Prepared by
The National Water Quality Evaluation Project at
North Carolina State University
and
Harbridge House, Inc.
for
The U.S. Department of Agriculture and
The U.S. Environmental Protection Agency
February 28, 1983
-------
The Model Implementation Program
Lessons Learned From Agricultural Water Quality Projects
This document was printed for the project
RURAL NON-POINT SOURCE CONTROL WATER QUALITY
EVALUATION AND TECHNICAL ASSISTANCE
USDA Cooperative Agreement 12-05-300-472
EPA Interagency Agreement AD-12-F-0-037-0
Biological & Agricultural Engineering Dept.
North Carolina State University
Raleigh, North Carolina 27650
and
Harbridge House, Inc.
1301 Pennsylvania Ave., N.W.
Suite 1200
Washington, D.C. 20004
This report represents the conclusions and views of North Carolina State University and Harbridge House.
Inc The report does not represent the official policy or views of the U S Department of Agriculture or the
U S Environmental Protection Agency Additional copies of this executive summary may be obtained
from
Public Information Office
Soil Conservation Service
U S. Department of Agriculture
Washington. D C
February 28, 1983
U.S. Environmental Protection
ftsgion 5, Library (PL-12J)
H West Jackson Boulevard, 12th Floor
Chicago, IL 60604-3590
-------
The Model Implementation Program
Lessons Learned from Agricultural Water Quality Projects
BACKGROUND
Introduction
The following is an executive summary of a study sponsored by the U.S. Department of Agriculture
(USDA) and the U S. Environmental Protection Agency (EPA) to evaluate the results of seven nation-
wide Model Implementation Program projects (MlPs) conducted between 1978 and 1982 * The central pur-
pose of the study was to extract "lessons learned" from the seven MIPs. In order to provide an independent.
objective review, North Carolina State University (NCSU) and Harbridge House, Inc . were commis-
sioned (1) to perform field surveys and data analyses and (2) to prepare a detailed report that would
serve as the basis for the executive summary
This summary focuses on various highlights of the full report. These highlights include problem
descriptions or findings concerning MIP activities, the demonstrable consequence of such problems, and
recommendations to address these problems The information presented is designed to add to the limited
body of knowledge on how government can more effectively manage ongoing and future agricultural
non-point source pollution control projects.
MIP Background
In early 1977, USDA and EPA examined several alternative methods for implementing the agri-
cultural and silvicultural non-point source pollution portion of water quality management plans devel-
oped under Section 208 of the Clean Water Act. These 208 plans were designed to address water quality
problems caused by non-point sources of pollution, in the case of agriculture, a voluntary approach
based on farmer participation was selected. However, no mechanisms existed for implementing these
abatement plans Monies for ensuring USDA and EPA participation in cleanup efforts had not been
appropriated; nor had cost-share funds been earmarked to assist in installing needed conservation and
management practices (called Best Management Practices) to solve water quality problems in specific
areas.
The Model Implementation Program was designed to demonstrate and study a concerted attempt
by USDA and EPA to address agricultural non-point source water quality problems by using existing
program authorities. There were many unknowns in conducting such a program However, one clear
goal was to learn from the experience of the MIPs
"This summary is abstracted from a report entitled "An Evaluation of the Management and Water Quality Aspects of the Model
Implementation Program
-------
The program was initiated and conducted primarily by the USD A workgroup on water quality.
Participating agencies included the Agricultural Stabilization and Conservation Service, the Soil Conser-
vation Service, the Cooperative Extension Service, the Economic Research Service, the Agricultural
Research Service, the Forest Service, and the Farmers Home Administration of USDA, and the Imple-
mentation Branch, Water Planning Division, of EPA
In September 1977. USDA and EPA entered into a Memorandum of Understanding to conduct
a Model Implementation Program using existing USDA and EPA programs and activities Projects from
seven states (Indiana, Nebraska, New York, Oklahoma, South Carolina, South Dakota, and Wash-
ington) were selected in December 1977. The projects were originally scheduled to terminate in
December 1980 but were later extended, with final reports on all project activities due in December
1982
Evaluation Study Methodology
The methodological approach used in this study is somewhat unusual Typically, studies are con-
ducted from either a management perspective or a scientific perspective, however, this study employed
both The National Water Quality Evaluation Project at NCSU supplied the water quality analytical
skills (scientific perspective) and Haibndge House supplied the management perspective
The seven projects were highly diversified in terms of geographic conditions, water quality prob-
lems, and farm operations Moreover, the projects were not designed, financed, or administered as
rigorous, highly controlled field experiments. In order to accommodate these limitations, the NCSU and
Harbndge House team used a "reasonable person" approach to reviewing the projects That is, the team
looked at the various phases of project planning and implementation from scientific, management.
institutional, and assistance (technical, financial, and educational) viewpoints and asked, "What would
one reasonably expect the projects to have done if they were going to efficiently achieve their objectives?"
It should be noted that the procedure followed throughout this study was that each point was eval-
uated independently, without reference to preceding points In other words, each observation was eval-
uated as though all other activities were conducted according to an ideal model for project direction.
-------
STUDY RESULTS
Introduction
The following discussion provides a summary of Model Implementation Program assessments and a
series of observations, outcomes, and recommendations grouped under seven major headings, program
structure, project selection, problem definition, goals and objectives, plan of action, implementation, and
results Major recommendations for program refinement are also presented The following is taken from an
evaluation of project activities and suggestions by project personnel on how to improve future and ongoing
agricultural water quality programs
Program Structure
This section assesses how the overall program was developed and conducted by USDA and EPA head
quarters staff The major points are summarized below
• The Model Implementation Program demonstrated that USDA and EPA could work together to
develop a program structure for addressing agricultural non-point sources through a project ap-
proach This coordinated effort also focused national attention on the agricultural water pollution
problem
• USDA and EPA did not state program goals and objectives in precise terms No mechanisms were
set up to measure progress in meeting goals As a result, the success of the Model Implementation
Program is difficult to assess In future agricultural non-point source control programs, goals and
objectives should be well thought out and carefully stated Ways to measure progress should be bu'lt
into the program from the start
• Specific tasks, responsible persons, and reporting formats were not established for project conduct.
especially for the water quality component As a result, many vital program and project activities did
not receive adequate attention In future programs, tasks should be clearly stated, definite responsi-
bilities assigned, and progress reports submitted. Adequate resources should be made available and
results assessed on a continuing basis
• No one person or agency had overall responsibility for directing the program Clear and complete
direction to the seven projects was lacking Therefore, when projects needed guidance, there was no
central source for the projects to consult This lack of direction caused problems for nearly all
projects, particularly in the early stages Future programs should designate a program director with
authority to act on behalf of all agencies in directing project activities This person should have
management experience as well as water quality expertise
Project Selection
This section assesses the way USDA and EPA selected the seven projects from among the 50 applica-
tions prepared at the state level The major points are summarized below
• State and Local Preparation of MIP Applications The 50 applications received within the 30-day
time period underscore the program's success in heightening interest in addressing non-point
source water quality problems. Many applications were received from states with no previous
experience in developing non-point source project proposals
-------
• Headquarters' Role in Selecting MIPs The selection criteria established by USDA and EPA were not
closely related to program goals. A good procedure used by USDA and EPA was to numerically
rank project applications based upon uniform criteria However, in specific cases these rankings were
superseded by other considerations in final project selection As a result, three lower-ranked projects
were selected, and several with high scores were rejected Future project candidates should meet all
technical criteria before other factors are used in the selection process
• Weight Giuen to an Identified Agricultural Water Quality Problem Following USDA and EPA head-
quarters guidance, states were asked to select MIP candidates on the basis of several factors How-
ever, MIPs varied in the attention given to identifying and documenting a solvable, agriculturally
related water quality problem Four projects adequately addressed this matter Future programs
should devote greater attention to the identified agricultural water quality problem if water quality
improvement is a primary goal
(MIP PROJECTS)
SUBSTANTIAL WEIGHT GIVEN TO WATER QUALITY PROBLEM
• Complexify of the Water Quality Problem One of the MIP selection criteria was the diversity of
sources contributing to the water quality problem This implies a complex problem and a corre-
sponding difficulty in obtaining water quality improvement Rather than tackling the most complex
problems. MlP-type projects should be limited to manageably sized project areas where water quality
problems are caused primarily by agricultural non-point sources Anticipating this observation, the
Rural Clean Water Program (RCWP) regulations required this more simplified focus
Problem Definition
This section covers the problem definition phase of a project, which was defined as the process of iden-
tifying and documenting the water quality issues to be addressed Water quality expertise is necessary to pro-
vide technically sound leadership in these important work areas, as noted below.
• The 208 Water Quality Plans In four cases, 208 plans were unfinished or inadequate, yet, immedi-
ate implementation was still expected As a result of improper planning, resources were spent in-
appropriately or inefficiently In future programs, a logical and detailed action plan should be re-
viewed and approved by headquarters staff before implementation begins If necessary, additional
time and resources should be given to projects to prepare an action plan with a supporting data base
YES
NO
USABLE 208 PLAN
-------
Intended Use Efficiency dictates that cleanup efforts focus on waters that have an impaired use,
this first requires a definition of the intended (and actual) uses Four of the MlPs did undertake a
precise evaluation of the intended use of the affected waters. All future projects should identify the
value of beneficial uses for benefit-cost analyses
YES
NO
INTENDED USES IDENTIFIED
Use Impairment After the intended uses are defined, the impairment should be identified This
must be a clear, documented statement of the water quality problem Four projects did document
use impairment In the other projects, efforts were not focused on a clear-cut problem Proper
identification and documentation of use impairment are necessary for project justification and
specification of corrective actions Future pollution abatement projects should not be funded unless
a clear use impairment is documented
YES
NO
USE IMPAIRMENT DOCUMENTED
• Identification of the Pollutant(s) Closely allied with use impairment is the question of which
pollutants are causing the impairments Three MIPs provided clear definitions of the pollutants and
linked them to the perceived impaired uses Others developed a "shopping list" of pollutants with
no discernible linkage to impaired uses Thus, some projects addressed pollutants that did not relate
to use impairment Because success in addressing impaired uses depends on aiming control strate-
gies at the pollutants, future efforts should be quite explicit in this regard
YES
NO
POLLUTANT PRECISELY IDENTIFIED
• Identification of Pollutant(s) Sources Although the pollutants causing the use impairment were not
always properly identified, four of the MIPs still had sufficient data to identify pollutant sources in a
precise manner This accurate source identification facilitated formulation of effective abatement
-------
strategies If sufficient background data are not available at project start-up, data for source identifi-
cation should be collected immediately following identification of the pollutants
YES
NO
POLLUTANT SOURCES DEFINED
Goals and Objectives
This section covers project goals and objectives; that is, the process of defining the projected overall
achievements (goals) and the necessary steps (objectives) to reach the desired outcomes. This process is de-
scribed below.
• Statement of Goals and Objectives A primary purpose (goal) of the MIPs was water quality im-
provement, but only two of the projects stated quantitative objectives for abating the water use
impairment. Since quantified goals make project performance easier to measure, future projects
would benefit from more attention to this issue In addition, headquarters agencies should critically
review project goals and objectives to ensure their consistency with the overall program goals and
objectives.
YES
NO
CLEARLY STATED WATER QUALITY GOALS
Use of Operational Goals Four projects did not establish operational goals Project goals need an
"operational orientation", that is, a precise definition of goals, the translation of these goals into
measurable objectives, the development of strategies to meet objectives effectively, a continuous
monitoring of progress in achieving objectives, and an evaluation of success at project conclusion
Projects that devoted little attention to goals lacked a solid basis for progress evaluation Future
projects can benefit by having objectives more rigorously guide actions throughout the project
OPERATIONAL GOALS
-------
Measurabihfy of Objectives. Although projects tracked the Best Management Practices (BMPs)
installed, objectives such as "effectiveness of the USDA delivery system" and other institutional
areas that may be non-quantifiable were given little treatment other than in general statements like
"agency personnel worked well together " Future projects could benefit from assistance in using
potentially applicable measuring techniques, as well as guidance as to what should be measured
Plan of Action
This section assesses how the project developed a work plan, including a statement of the roles
and responsibilities of participating agencies, schedules, budgets, work products, and other require-
ments to accomplish the project goals and objectives
• Project Coordinator All seven MlPs noted the need for a full-time project coordinator from
project inception Ideally, this person should have both water quality and project management
expertise Four of the projects had a coordinator, and project personnel stated that this was an
important factor There is no evidence from the MlPs as to whether the coordinator should
come from a cooperating agency or other organization Those projects without a full-time co-
ordinator experienced problems such as lack of interagency communication and confusion
over responsibilities All MIPs recommended that future projects should have an active, en-
thusiastic, and capable project coordinator who is involved at the outset of project planning
and is employed full-time in this capacity
YES
NO
FULL-TIME PROJECT COORDINATOR
Pre-lmp!ementatton Planning The MlPs unanimously noted that they had insufficient time for
planning prior to BMP installation Project personnel felt pressured to obligate Agncultural
Conservation Program (ACP) funds as quickly as possible This meant the funds were obligated
before planning, including baseline data gathering, could be comj^leted Such a practice not
only disrupts baseline data requirements, but also risks inefficient resoutce allocation Future
projects should provide a specific jjenod for pre-imj.ilementation planning, followed by the allo-
cation of cost-sharing money when needed for implementation
NO
SUFFICIENT TIME FOR PROJECT PLANNING
-------
• Planning Activities. The MIPs evidenced a range of approaches to planning. One project pre-
pared a detailed plan of action that linked project activities to objectives, a workload analysis, a
plan of work for monitoring and research, and a priority rating system for farms to target, with
annual revisions. At the other extreme, three projects relied on their initial statement of goals
and milestones, with few formal, documented revisions as the project proceeded. Greater
attention to planning as an ongoing project responsibility could be of value to future activities
by enabling the redirection of project resources as experience is gained.
• Definition o/ Responsibilities It is important that the responsibilities of each agency be clearly
stated, agreed to, and executed. In the Model Implementation Program, there was some con-
fusion as to which agency would perform specific duties. As a result, certain tasks were either
not completed or received inadequate attention. Future programs should more clearly state the
responsibility of each agency and incorporate mechanisms to ensure that assigned tasks are
completed.
• Resource Allocation It was clearly stated at the start of the program that no new congressional
appropriations would be available, but that existing resources should be redirected to assigned
tasks. Some agencies were successful in shifting manpower and funds to project activities, but
others could not or did not provide necessary resources. Future programs should incorporate
mechanisms to ensure that adequate resources are allocated by or given to each agency to
complete assigned tasks
• Pollution Source Priontization. Very little was done in the project planning stages of the Model
Implementation Program to estimate non-point source pollution resulting from various land
uses. The result was that resources were inefficiently allocated in some projects Future
projects would benefit from making such estimates, which provide the basis for developing ef-
fective abatement strategies It should be understood that such estimates may need updating as
new information is developed; projects should respond with modifications in abatement
strategies.
• Critical Area Definition. From a water quality perspective, critical areas are those lands that are
disproportionately responsible for water use impairments. Four MIPs did not define critical
areas, this resulted in less efficient abatement strategies for these projects Because treating all
the land in a project area is usually economically unfeasible, it is essential to identify critical
areas within the watershed. Critical area identification should be performed jointly by agricul-
tural and water quality experts
YES
NO
CRITICAL AREAS DEFINED
• BMP Selection MIPs generally evidenced a basic understanding of methods to control various
agricultural pollutants, and many projects attempted to prioritize BMPs. However, all of the
projects approved some BMPs that were of questionable value in addressing the identified
water quality problem. USD A and water quality personnel at the project level should work
-------
more closely together in future BMP selection and pnoritization. It is also recommended that all
BMP lists be accompanied by a project-specific water quality rationale
YES
MOST BMPs REASONABLY SELECTED
• Cost-Effective Solution Identification For the most part, projects relied on structural practices (such
as terracing) to control pollutants Less expensive management practices (such as conservation
tillage) were not widely used in the MIPs. Future projects should more systematically evaluate the
trade-offs between structural and non-structural practices
Implementation
This section deals with aspects of the work performed in carrying out a plan of action The highlights of
this process are described below
• Techniques for Obtaining Voluntary Farmer Participation All seven MIPs noted that one-on-one
contact between project personnel and farmers was the key method for getting farmers to become
cooperators Agency personnel promoted BMPs based on their benefits to the farmer (e.g , drier
barnyards or soil conservation) as well as water quality improvement. Project personnel uniformly
noted the value of demonstration farms, whereby a farmer could see BMPs installed on neighbors'
fields. The MIP experience contributes to the hypothesis that high cost-sharing rates (e.g , 80
percent to 90 percent) may be more critical in early project stages to get demonstrations going
After that, cost-sharing rates may be reduced
• Information and Education As techniques to obtain voluntary participation, information and edu-
cation (I&E) efforts deserve special mention Concerted l&E efforts were made in each of the MIPs
by various agencies, including newsletters, radio spots, meetings, and personal contacts. However,
the majority of project personnel noted the need for greater attention in these areas The l&E
project component should publicize the project, clarify the roles and responsibilities of farmers, en
courage farmer participation, identify sources of assistance, and communicate project results
NO
YES
EXPRESSED NEED FOR GREATER I&E
Delivery of Technical Assistance In all seven MIPs, technical assistance was emphasized and addi-
tional resources were allocated This resulted in timely assistance to farmers in the project areas and
promoted farmer participation However, the proper BMPs were not always put in the right places
because of any combination of the following improper pollutant identification, poor BMP selection,
and inadequate critical area definitions Future projects should carefully determine the BMPs
needed (and where) to maximize the efficiency of the technical assistance delivery system
-------
• Delivery of Financial Assistance Five of the MIPs did not select farms within a project area for
priority financial assistance, even though the critical area concept and efficiency of resource alloca-
tion would argue for such an approach Instead, funds and assistance were delivered in the tradi-
tional USD A method of "first-come-first-served," subject to the applicant being qualified and
assistance being available Future project personnel will need guidance in developing techniques to
concentrate assistance in an acceptable manner, particularly if financial assistance is limited.
YES
NO
FINANCIAL ASSISTANCE PRIORITIZED
• Implementation Monitoring In general, the standard USDA reporting formats used as the primary
land use data collection instrument by most projects did not track BMP application with enough
frequency and in enough detail to establish quantitative land use-water quality relationships
Future projects should design a tracking system based more on pollutant control than on acres
served or cost-share monies spent,
• Water Qualify Monitoring Because of the diverse nature of the MIPs, unique water quality moni-
toring systems were designed for each project Projects that relied more heavily on a scientific
approach in the problem definition and solution stages were generally able to design and implement
water quality monitoring systems capable of determining changes in the water quality
Project Results
This section addresses some of the major accomplishments and deficiencies in the MIPs. These project
results are described below
• Time Frame The program's time frame was adequate for gaining farmer participation, installing
BMPs, and implementing water quality monitoring programs However, the planning, background
data collection, and data analysis and interpretation steps were cut short. From a scientific perspec-
tive, water quality benefits to receiving bodies could not be documented for the majority of project
situations within this period of time.
• BMPs Installed All seven MIPs were successful in accelerating the level of BMPs installed This
reflects well on the ability of SCS, ASCS, and other agencies to get farmers to adopt conservation
practices However, many practices were not selected and applied from a water quality perspec-
tive Better guidance and direction for field personnel are needed if perceived water quality prob-
lems are to be solved Recognition of this need during the RCWP resulted in training for USDA and
EPA staff members and workshops for project personnel Updated and cooperative training should
be provided for this continuing need.
YES
PRACTICE INSTALLATION ACCELERATED
10
-------
• Farmer Support Generally, farmers supported the MIPs through participation in meetings
(Conservation District Board meetings and ASC County Committee meetings) and by install-
ing BMPs. Compared with pre-MIP rates, farmer participation increased during the MIP period
for every project. One project got nearly all of 275 active farms to participate (96 percent).
while another project got 528 individuals or groups out of 1,247 to install one or more prac-
tices
• Water Quality Results Of the four projects from which final reports have been received, one
has demonstrated statistically significant progress in achieving water quality improvement
Status reports from the other three projects indicate that none has demonstrated statistically
significant progress in achieving water quality improvement. However, five projects have
demonstrated the water quality benefits of BMPs from field and plot studies
YES
NO
DOCUMENTED WATER QUALITY IMPROVEMENT
YES
NO
PLOT/FIELD STUDY RESULTS
Although the limited MIP time frame hindered documentation of receiving body water quality
improvements, more success could have been achieved in reaching this principal goal if the
program had more effectively addressed all steps outlined in this report
Value oj the Project Approach MIP personnel believe that the project approach was bene-
ficial in addressing non-point source water quality issues Specifically, cost-sharing was in-
creased, funds were obligated, and BMP installation was accelerated Quantified data on the
benefits of water quality improvement as well as other project benefits were not requested of
the projects and have not been forthcoming This lack of data makes it difficult to conclude
whether or not the projects were economically efficient (i.e . benefits exceeded costs)
USDA Delivery System The USDA's national, state, and local organizational system and pro-
fessional staff capabilities are well established in the agricultural community USDA employees
know farmer attitudes, soils, crops, erosion problems, and farm conservation practices The
MIPs demonstrated that this substantial technical competence can be deployed in non-point
source water pollution control efforts. As has been noted, a need exists to supplement this
capability with water quality expertise. Workshops and training (such as that conducted during
the RCWP) should be continued The water quality training could be most effectively handled
by both in-house and extramural water quality experts.
11
-------
RECOMMENDATIONS
This evaluation of the Model Implementation Program has revealed both strong and weak elements
in each of the projects. It must be remembered that this was an experimental effort; learning how to
better conduct agricultural non-point source control projects was thus a program goal. Grouped below
are major recommendations worthy of further scrutiny. These recommendations, which are drawn
from the preceding evaluation, should help future programs and projects
• Future water quality programs should operate within more realistic time frames. Several years
are generally required to document water quality improvements due to agricultural non-point
source control efforts. The recently established RCWP is based on a more realistic time period.
• Future programs should provide sufficient time for pre-implementation planning. Such plan-
ning should include, water quality problem documentation; abatement strategies, monitoring
plans; estimates of time required to document water quality changes; and definition of data
analysis and evaluation procedures.
• Future programs should designate a program director with authority to ensure that assigned
agency tasks are carried out and program goals are achieved. This person should have both project
managment and water quality expertise.
• Mechanisms should be developed at the headquarters level to ensure that all important pro-
gram elements are clearly assigned to specific agencies; that resources are adequate for the
task; that accountability measures are established; that reporting requirements are met; and
that the evaluation of progress in meeting program goals is periodically updated.
• Future water quality program managers should structure more explicit goals and objectives at
the headquarters level and develop guidelines to assist project personnel in their efforts. In
addition, feedback mechanisms should be set up to disseminate successful (or unsuccessful)
methods for program enhancement. This information should be shared among all project
participants and other interested groups.
• Future programs should place greater emphasis on identifying the most current knowledge in
the evolving field of agricultural non-point source control. Mechanisms should be established to
transfer state-of-the-art technology to projects and ensure that it is appropriately utilized
• Future water quality programs should adopt the scientific method as a management model,
especially in the areas of problem definition, design of abatement strategies, plan implementa-
tion, monitoring, and evaluation and reporting of results.
• All of the recommendations stated above for the program structure at the headquarters level
apply equally to state and local entities at the project level.
12
-------
AGRICULTURAL WATER QUALITY MODEL
KEY STEPS FOR PROGRAM/PROJECT MANAGEMENT
PROGRAM DIRECTOR
PROJECT DIRECTOR
MEASURABLE GOALS AND OBJECTIVES
i MONITORING AND IMPLEMENTATION
PROGRAM PROGRESS
TECHNICAL GUIDANCE
DEVELOP APPLICATIONS
WATER QUALITY PROBLEM
• Intended Use
• Pollutants
• Pollutant Sources
GOALS AND OBJECTIVES
SELECT PROJECTS
TECHNICAL CRITERIA
O
01
DC
NOTIFY
SELECTED PROJECTS
WORK PLAN REVISIONS
AND APPROVAL
DEVELOP WORK PLAN
PRE-IMPLEMENTATION PLANNING
• Pnontization
• Critical Areas
• BMPs
WATER QUALITY MONITORING
LAND TREATMENT MONITORING
INFORMATION AND EDUCATION
OPERATIONAL GOALS AND OBJECTIVES
MONITORING AND
IMPLEMENTATION
ANNUAL REPORTS
DATA ANALYSES
TENTATIVE ASSESSMENTS
------- |