United States
Environmental Protection
Agency
Office of the
Comptroller
(3304)
EPA 205-B-96-001
March 1996
 Management Integrity
 at EPA
A Managers "How To" Guide
for Program Reviews:
Seeing the Forest and the Trees

-------
TABLE OF CONTENTS
Page
Introduction 1
Purpose 2
EPA ’s Management Integrity Principles 4
Basic Steps to Conducting Program Reviews 7
Step 1: Identify Potential Problems 9
Step 2: Examine the Details 13
Step 3: Analyze Data/Define Problem 28
Step 4: Document the Issue 32
Step 5: Develop A Corrective Action Strategy 33
Step 6: Implement the Plan 35
Step 7: Assess Results 38
Integrate Results with Policy and Decision Systems 40
Your Good Judgment is Essential 41

-------
INTRODUCTION
Understanding a concept like management integrity requires a little
imagination. Too often concepts get buried in technical jargon or are
burdened by past associations. In complying with the Federal Managers’
Financial Integrity Act [ P.L. 97-255 ], many Federal managers historically
never saw the “Big Picture.” Most focused on filling out checklists and
performing other routine compliance tests, rather than considering
management controls in light of broader program issues and EPA’s overall
mission. In short, they got lost in the trees and never saw the forest!
Federal programs cannot afford the time or money spent getting lost in
the trees. In fact, the public and Congress want less bureaucracy, more
customer focus, and improved management. One way of achieving these goals
is to enhance the flexibility of management oversight while strengthening
accountabi1it The White House and Congress have worked together in
recent years to implement legislation that provides for both.
The June 1995 revision of the Office of Management and Budget Circular
A-123, which implements the Integrity Act, strengthens management
accountability by linking management controls with planning and budgeting
decisions. In fact, the change in the Circular’s title from internal Gontrols to
Management Accountability and Gont’rol speaks volumes! Managers are now
responsible for making both accountability and control an integral part of
day-to-day program management.
I

-------
PURPOSE
Effective EPA program managers appreciate both the broad mission and
goals and the operational mechanics of their programs. In effect, such
managers are capable of seeing both the forest and the trees as they manage
for results. The purpose of this Guide is to show how managers can monitor
and manage both overall program progress (the forest) and the effectiveness
of day-to-day operations (the trees).
This Guide will:
+ Provide you with a systematic approach and framework for
conducting program reviews that address Integrity Act issues
‘• Promote your consistent attention to EPA’s Management Integrity
Principles and best management practices in performing program
reviews
+ Support your effective implementation of EPA ’s Integrity Act policy
But first, we present EPA’s Management Integrity Principles as the
essential guideposts for performing a good program review. Using this
framework will promote more effective integration of the Integrity Act
across EPA, and will lead to successful results-oriented program
management.
We have developed a little fable to help illustrate the steps to a good
program review. We hope you enjoy the story of Howell the Owl and the
Mystery at Sunnyside Stream, as you read through this Guide.
2

-------
ql radition portrays the owl as the wisest of ailforest animals,
always exercising sound reasoning and good judgment. Given its
reputation for intelligence and good judgment, lets follow a wise old
owl named Howell during a day with his forest friends. During this busy
day Howell will identify analyze, report on, and help to solve a serious
problem in theforest. Along the way this Guide describes a parallel
approach to performing program reviews, including tools and techniques
useful at each stage of the review
3

-------
EPA’s MANAGEMENT INTEGRITY PRINCIPLES:
Guideposts for a Good Program Review
EPA’s integrity policy advocates that managers incorporate EPA ’s
Management Integrity Principles into existing management processes,
program strategies, and guidance to strengthen program operations. Use
these Principles to guide your program review and in developing your review
objectives and methodology The Principles are the desired condition that
you want to verify or meet
“The Guidance Principle” — Develop written strategies,
policies, guidance, procedures, and performance measures to
achieve EPA’s mission and safeguard programs and resources
against waste, loss, unauthorized use, and misappropriation.
“The Accountability Principle” — Establish an organizational
structure and delegate authority, responsibility, and
accountability in accordance with Agency guidelines to
achieve the mission of the organization.
L r “The Feedback Principle” — Carry out program activities,
r consistent with established policies, strategies, guidance and
_____ procedures, and report significant, emerging management
-t problems through the chain of command to the appropriate
national program manager for action.
‘The Competency Principle” — Demonstrate personal
integrity, provide quality supervision, and sustain a level of
professional competence to accomplish assignments and to
ensure that management objectives are achieved.
4

-------
“The Quality Data Principle” — Collect and assure the
quality of data and other information necessary to manage
environmental programs and continuously improve the basis
for the Agency’s scientific, technical, legal, enforcement, or
management decisions.
“The Separation Principle” Separate key duties and
responsibilities in authorizing, reviewing and approving
payment, and maintain individual accountability for the
custody and use of resources.
“The Comparison Principle” Periodically compare written
records of actual and planned activities for budget
expenditures, program operations, property inventory, and
staffing levels, to identify discrepancies and take appropriate
action, where vulnerabilities exist.
“The Identjfication Principle” Use all available
information sources to identify and routinely assess program
areas that are vulnerable to fraud, mismanagement, and
noncompliance with law.
“The Review Principle” — Develop and carry out a systematic
review strategy, comprised of internal program reviews, Office
of Inspector General (OIG) audits, and General Accounting
Office (GAO) studies, to assess the effectiveness of program
guidance and procedures, and revise, as necessary.
“The Correction Principle” — Promptly determine, and carry
out management actions to correct, within established time
frames, significant problems identified by internal program,
OIG, and GAO reviews.
5

-------
Howell the Owl and the Mystery at Sunnyside
Stream
i ile flying through the forest one da Howell the Owl noticed
that the water in. Sunnyside Stream appeared unusually low.
Infac4 he could see many rocks and pebbles that were normally
covered by water He also saw a lot of debris, including tree limbs and twigs,
strewn. on the banks of the stream. And he noticed sereralfish half
submerged, struggling to get to the few remaining pools. Then he saw a
raccoon standing on the shore feasting on a trout. The raccoon admitted
that I/ i c stream bed was a ufuiiy dr but said he liked (lie easy pickings!
Howell knew that probably only the raccoon and his cousins would
appreciate this drastic change at Sunnys ide Stream. and even they wouidn ‘t
like it after all the easy pickings were gone...
So. thinking back on all of the problems he had encountered in the forest
over the years, Howell decided to approach this puzzle much as he had
others. His basic approach followed seven steps, each of which had served
him well in the past...
6

-------
B&sic STEPS TO CONDUCTING PitoGiwvI
REVIEWS
The following steps are basic to a good program review. In some cases,
there are a variety of alternative approaches to performing a step. For
example, in STEP 2, there are several tools and techniques you can use to
examine details.
STEP 1 IDENTIFY POTENTIAL PROBLEMS: Survey the scene for
information, sources, develop key questions, and ident y red
flags.
STEP 2 EXAMINE THE DETAILS: Collect high quality information by
observing, inquiring, inspecting, re-performing, and walking
through activities.
STEP 3 ANALYZE DATA/DEFINE PROBLEM: Assess progress in achieving
goals according to laws, regulations, and administrative
procedures.
STEP 4 DOCUMENT THE IssuE: Describe the identified problem in
writing.
STEP 5 DEVELOP A CORRECTiVE AnioN STRATEGY: Describe the
necessary actions to correct the problem.
STEP 6 IMPLEMENT ThE PlAt : Set the corrective action plan in motion.
STEP 7 ASSESS REsutrs: Verify that corrective actions achieve results.
7

-------
H owell wanted to get a “hands on” too/c at the changes at
Sunnyside Stream. So he flew down to the stream bed and he
walked around a bit picking up some stones and looking
underneath them to get some idea of the extent of the problem.
But Howell also strongly believes in getting the whole story So, in order to
invest igatefull he decided toflyfurther upstream, survey the scene, and
ask afew questions.
S

-------
STEP I IDENTIFY POTENTIAL PROBLEMS:
Survey the Scene for Information Sources
EPA’s integrity policy advocates that managers tap a variety of information
sources when reviewing programs. Consider the following information in
determining top problems, and where to focus your review:
+ Knowledge gained from the daily operations of EPA programs and systems
+ Feedback from state, county, municipal or tribal customers, and /or the
regulated community
+ Annual strategic plans, performance plans and reports prepared in
conformance with the Government Performance and Results Act
(GPRA) of 1993
4• Reports and other information provided by the Congressional
committees of jurisdiction
+ Office of Inspector General (OIG) and General Accounting Office
(GAO) reports, including audits, inspections, reviews, investigations,
and hotline complaints
+ Audits of financial statements conducted pursuant to the Chief
Financial Officers (CFO) Act of 1990
+ Any other internal or external reviews or reports relating to EPA
operations
4• Reviews of financial systems which consider whether the
requirements of 0MB Circular A-12 7-Revised, Financial Management
Systems, are being met
+ Reviews of systems and applications conducted in accordance with the
Computer Security Act of 1987 and 0MB Circular A-130, Management
of Federal information Resources
9

-------
STEP 1 IDENTIFY POTENTIAL PROBLEMS:
Develop Key Questions
In addition to the specific objectives of your review, use EPA ’s
Management Integrity Principles to formulate questions to sharpen the issues.
For example,
4 Are the requirements for program participants reasonable, or are they
so complicated and cumbersome that they are difficult to understand
and carry out? (Guidance Principle)
+ Does the activity have many complex decision or approval points or
transactions? These may increase the likelihood of errors or
irregularities. (Separation Principle)
+ Have important activities/programs been contracted out with
appropriate systems of checks and balances? (Accountability
Principle)
+ Has the recent Agency Integrity Act report included material
weaknesses pertaining to the activity? Such information may indicate
a high degree of susceptibility (IdentWcation Principle)
4 Does the activity involve liquid assets that are readily marketable
(e.g., cash or securities) or could be misappropriated for personal use
(e.g., tools or computers)? (Guidance and Separation Principles)
4 Does the program contain financial or other inherent incentives that
may be conducive to fraud, waste, abuse or misappropriation?
(Guidance and Separation Principles)
10

-------
STEP 1 IDENTIFY POTENTIAL PROBLEMS:
Identify Red Flags
Be alert to any “red flags,” that may set the stage for future problems:
4• Unclear program mission and goals (Guidance Principle)
4• Complex procedures and programs (Accountability Principle)
4• Poorly defined and documented program guidance, policies, and
procedures (Guidance Principle)
4 Lack of specific performance measures for the program/activity
(Guidance and Quality Data Principles)
+ Lack of a systematic framework to establish priorities (Guidance,
Accountability Feedback, and Review Principles)
4 Duplicate, inefficient, poorly designed, and/or ineffective systems
(Separation and Quality Data Principles)
4 A high rate of personnel turnover in key positions (competency and
Separation Principles)
+ A history of material management control weaknesses described in
annual AgencyintegiityAct reports (Identification, Review, anti
Correction Principles)
11

-------
A s he flew upstream, Howell came upon afamily of rabbits huddled
near the mouth of its burroic The baby bunnies were clearly upset
and Howell soon found out why With the water level so low in
Sunnyside Stream, all of the tender shoots of grass along the stream bank
were dying away And the tougher weeds left behind were hard on their little
bunny tu nun ies!
— -
We14 thought Howell, let me look at this in more detail...
12

-------
STEP 2 Ex&irnr’a THE DETAILS By:
Collecting High Quality Information
Based on your understanding of the problem areas identified in Step 1,
determine relevant key activities and processes, and specify program review
objectives. Within your review scope and objectives, collect information in
various ways. For example, observe people doing their work or ask them about
their job responsibilities. Inspect files or reports. Let your judgment dictate
the best methods.
Obtain evidence directly as it provides a higher level of assurance than
evidence obtained indirectly. For this reason, at a minimum, observe and ask
questions of EPA managers and staff. Try to collect information that has
relevance, objectivity, persuasiveness, and freedom from bias:
4• Relevance: Evidence must support the procedure being tested
4• Objectivity: Two or more reviewers should reach essentially the same
conclusion based on the same evidence
+ Persuasiveness: Evidence must be adequate to form a conclusion
+ Freedom from Bias: Evidence should not unfairly support one
alternative conclusion over another
Depending on the quality of evidence—the results of testing—derived
from data gathering techniques, the availability of different types of evidence,
the nature and purpose of the test, and the significance of the problem being
examined, you can determine which type of testing, and how much, is most
appropriate to your review.
13

-------
L eaving the rabbit family Howeilfiew off to find a deer because a deer’s
range is much wider than most of the animals in the forest. Finding
a deer not too far away Howell asked the deer what he knew about the
problems at Sunnyside Stream. The deer reported that he had been away
visiting fri ends in another forest, and had not been to the stream for several
days. But he said that some of the animals had mentioned a beau4ful new
pond that had recently appeared in the northern part of the woods.
“Mmmm,” thought Howell, “I think I ’ll go see this pond for myself walk
around a bit, and ask afew more questions...”
14

-------
15

-------
STEP 2- ExAMINE ThE DETAILS By:
Observing
Description
See first hand. If appropriate, watch people doing their job assignments.
Try to get a sense of the organizational culture through direct observation.
Look at the physical space — are the desks occupied? Are people open? Do
they speak with candor? Observation, like inquiry, is an essential review
technique. If you do nothing else, at least observe operations and ask
questions of EPA managers and staff responsible for the operations.
Attributes
4. Provides highly reliable evidence at a point in time
+ Provides no evidence about any other time
+ Requires supplemental testing (e.g., inquiry) of operations at other
times
16

-------
Sample Tests
Review EPA’s Management Integrity Principles and identify those
Principles relevant to your review methodology and objectives. Here are some
examples for you to consider:
The Comparison Principle
+ Observe grantees’ actual on-board employees,
equipment, etc., compared to their grant agreement.
The Separation Principle
+ Observe that controlled custody exists over critical
forms (e.g., blank checks, purchase orders, signature
plates, master file change forms, vouchers, check
requests).
The Quality Data Principle
+ Observe whether the laboratory is complying with
environmental health and safety standards to support
sound scientific research and data collection methods.
17

-------
STEP 2 EXAMINE THE DETAILS By:
Inquiring
Description
Managers and staff involved in day-to-day operations are the best
resources available. They are also the personnel who must incorporate
management’s best practices and EPA ’s Management Integrity Principles into
their daily work. Ask EPA personnel, either in person or in writing, about how
they perform their activities and responsibilities. This is a basic, but
essential review technique.
Attributes
+ Provides least reliable type of evidence
+ Typically requires open-ended questions
+ Reliability depends on competence, experience, knowledge,
independence, and integrity of the personnel questioned
+ Evidence corroborates best through observation and/or inspection
18

-------
Sample Tests
Hear it for yourself. Ask people about their job responsibilities. Are
people open? Do they speak with candor? Inquiry, like observation, is an
essential review technique. Again, if you do nothing else, at least observe
operations and ask questions of EPA managers and staff responsible for the
operations. Remember to review EPA ’s Management Integrity Principles.
Here are some examples for you to consider:
The Accountability Principle
+ Interview senior managers and staff regarding the
organization’s methods of assigning authority and
responsibility. Determine how authority, responsibility
and accountability are delegated to address
organizational goals and objectives, operating
functions, and other requirements.
The Competency Principle
+ Interview various program staff regarding training
policies and procedures. Assess how staff are trained
to meet their job responsibilities.
The Guidance Principle
+ Interview key managers regarding guidance, rules or
regulations developed to achieve EPA ’s mission and the
program’s objectives. Is the guidance clear? Easily
understood? Is it effective in meeting the program’s
goals and objectives? Should it be modified or
changed to be more effective?
19

-------
STEP 2 ExMuriE ThE DETAILS By:
Inspecting
Description
Inspection is an optional review technique. Use inspections to
supplement observations and inquiries. View, examine, read, and compare
information and data contained in documents and records with actual
procedures performed, and operating policies and guidance.
Attributes
4 Is generally a reliable procedure
+ Can be performed at any time
4• Does not provide information on effectiveness of action taken
4. Works best when supplemented with observation and/or inquiry of
persons performing the activity
20

-------
Sample Tests
Examine it yourself. Handle relevant documents and analyze the data
contained therein. Use the Integrity Principles. Here are some examples for
you to consider:
The Accountability Principle
+ Review functional statements to determine whether
they are current, in writing, and consistent with actual
operations.
The Correction Principle
+ Determine how management identifies significant
problems, and tracks corrective actions. Inspect the
system to ensure that it is complete, current, and
reasonable.
The Feedback Principle
4• Determine how the results of operations are
systematically monitored against plans and budgets,
and variances investigated. Examine an example of a
variance and its resolution.
a
21

-------
STEP 2 ExMIINE THE DETAILS By:
Re-performing
1M cripIion
Use re-performance of selected procedures and activities as an optional
review technique to supplement other review work. Re-performance may be
particularly helpful for activities of a computational nature.
Attributes
+ Provides evidence that the expected outcome resulted
+ Does not provide evidence of how the procedure was actually applied
+ Should be used in conjunction with other procedures
22

-------
Sample Tests
Examples for you to consider are provided below:
The Quality Data Principle
•. Recalculate unit cost extensions in an inventory list,
foot the list, and trace the total to the general ledger
amount.
The Separation Principle
4• Select a sample of invoices. Compare invoice terms
with agreed-upon prices. Confirm that authorized
individuals have approved the invoices. Validate that
exceptions were reviewed by someone authorized to
approve them. Also, confirm that invoices were
processed by designated staff who are independent of
the authorizing, reviewing, approving, purchasing, and
receiving functions.
The Comparison Principle
+ Test the periodic physical inventory counts made by
Agency persormel by re-performing the counts
yourseLf. Relatively heavier test counting should be
performed on high value areas of inventory. Compare
your counts with the inventory listing.
23

-------
Sm 2 Ex 1 nii THE DETAILS By:
“Walking Through”
Description
Follow transactions from initiation to completion by “walking through”
them. Combine observations and inquiries with optional inspections and
re-performance of selected activities. If appropriate, actually observe a
transaction being processed by the respective personnel. Discuss with these
staff their responsibilities in applying key procedures in the selected
activities. Gain, through inquiry, management’s perspective of these
transactions. Inspect the documents to determine how they have been
modified as they have been processed. Re-perform key procedures, as
necessary.
Attributes
+ Serves as a comprehensive test procedure
+ Is usually more time consuming than other independent review
procedures
24

-------
Sample Tests
See hear, handle, analyze, and re-do selected procedures. Pick activities
and/or processes and determine the key procedures to evaluate. Use the
Integrity Principles and develop relevant tests. Listed below are some
examples for you to consider:
The Accountability Principle
+ Determine whether staff job descriptions delineate
specific duties, reporting relationships, and
constraints , Review selected personnel position
descriptions. Interview the respective employees to
determine whether the job descriptions are
descriptive of the jobs actually performed. If
appropriate, observe the employees performing their
duties, or review selected work products that are
substantially representative of the employee’s duties.
The Separation Principle
+ Ascertain how policies and procedures for
authorization of transactions are established.
Determine whether all processed invoices and
supporting documents have been approved by
designated employees before payment. Review
selected invoices for proper approval. Observe
processing of a sample of invoices.
The Comparison Principle
4. Determine how the organizational unit is organized to
achieve its goals and objectives, inspect a unit’s
organization chart. Observe managers and staff and
compare them with the organization chart.
25

-------
F lyingfurt her upstream, Howell came upon a very big pond that
he had never seen before. Sure enough, afamily of eager beavers
was busily putting its finishing touches on a dam right on the
stream! Howell introduced himself to the beaver family and described all
that was happening downstream from the pond. The beavers were very
friendly and said they were sorry qtheir dam had become a problem for the
forest animals living downstream.
Howell thanked the beavers for their time. The wise old owl decided to
contemplate all that he had seen and heard in the forest that day and then
d.eterinine the best way of achieving his goaL
26

-------
27

-------
STEP 3 ANALYZE DATA/DEFINE PROBLEM:
Assess Progress in Achieving Goals
As indicated by the Identification Principle, use all available information
sources to collect information. Interpret the data and information developed
while examining details. You may apply several criteria in this process,
including :
4• Assess compliance with laws and regulations
+ Assess compliance with administrative procedures
Interpret these criteria broadly. For instance, when evaluating
procedures for achieving goals, consider a variety of elements, such as
customer service, timeliness, and availability of resources.
Consider whether timely, accurate information is available for
management’s use, and whether the procedures are cost-effective for the
benefits received. Determine whether other procedures may be more
cost-effective. Focus on the substance rather than the form of management’s
policies, procedures, and actions.
28

-------
Assess compliance with administrative and programmatic laws and
regulations. You should also reassess the quality of applicable strategies and
guidance to determine their effectiveness in achieving mission goals and
guarding against fraud, waste, abuse, and misappropriation.
When evaluating achievement of mission goals, consider the following
questions:
4• Are the policies and procedures readily understandable or are they
vague, complex, or seemingly contradictory? Those that are clear,
understandable, and consistent with laws and regulations are easier to
follow than those lacking these characteristics.
+ Are policies and procedures consistent with current requirements of
laws or regulations? is the program carrying out the requirements
according to law? Are there management systems in place to measure
this?
Determine whether administrative strategies and guidance were followed
in achieving program objectives. For example, you could determine whether
proper contracting practices are being followed. Proper attention to
administrative requirements supports the achievement of program goals and
protects the integrity of the program.
29

-------
A fter collecting what he believed was enough information, Howell
decided it was time to address the problems at Sunnyside Stream.
Fortunately he had taken good notes of all that he had seen and done
so far After a14 he wanted to be quite convincing when it came to describing
the problem to the other animals, and to getting help with a good solution!
30

-------
31

-------
STEP 4 DOCUMENT THE ISSUE:
Describe the Identified Problem in Writing
Consistent with the Guidance Principle, ensure that written review
results and recommendations are prepared and provided to senior
management. Written reports should clearly summarize review results and
present a plan of action to correct identified problems. You should also file
and maintain supporting documentation for review results and
recommendations. Include the following fundamental elements in your
report:
+ Scope of the work conducted: why areas were selected for review; the
organization that performed the work; and how the review was
conducted (who was interviewed; what records were reviewed; and the
extent and type of sampling conducted)
+ Results of review: what was found, including details and explanations
of deficiencies
+ Recommendations: how the problem affects the decision-making
system; changes that are needed within an office or in EPA-wide
systems
L
4 1-
32

-------
STEP 5 DEVELOP A CORRECTiVE ACTION
STRATEGY:
Describe the Necessary Actions to Correct the
Problem
Develop your overall strategy to correct the problem identified in your
review, and how you will check to ensure that your corrective actions will
solve the problem. Corrective actions state what an organization should do to
accomplish beneficial results. Corrective actions should be action-oriented,
convincing, well-supported, and effective. To be effective, corrective actions
must identify a specific course of action that will correct problems or cause
significant improvements. Consider the following when developing your
corrective action strategy and milestones:
4• Describe how the corrective action strategy will successfully solve the
identified problem
4 Ensure a logical flow from the review evidence to conclusions
•• Address the underlying causes that allowed the situation to arise
4 State specifically what actions are to be taken, by whom, and in what
time frame
+ Direct corroborative action to those who have authority and responsibifity
to act
4 State prescribed actions positively and constructively
4 Call for corrective actions consistent with findings and conclusions
• Emphasize awareness of costs and risks
• Consider practical constraints (financial and other priorities)
4• Present alternative solutions when possible
+ Avoid recommending additional studies
33

-------
owell politely asked the beavers why they needed such a large pond,
whether the height of the dam co uki be lowered to allow more water
to pass, or whether a better location for the darn could be found. The
beavers e rp1ained that they needed such a big pond to float new limbs in
from the surrounding forest. And since they had jnst finished building their
new lodge, they would be happy to release more water over the darn again!
They apologized for the problems their dam had caused for the other forest
animals and said they had no idea the impact downstream would be so
great.
They then invited Howell and some of his friends to help remove some of the
limbs from the dam. What a relief it was to see how quickly the water came
pouring into Sunnyside Stream again!
-
-
34

-------
STEP 6 IMPLEMENT THE Pin:
Set the Corrective Action Plan In Motion
Take timely and effective action to correct deficiencies identified during
the course of reviews — precisely the intent of the Correction Principle.
Consider the correction of deficiencies an integral part of management
accountability. Treat the development of corrective action plans as a priority.
Depending on the severity of the identified problems, reporting and tracking
of deficiencies may vary. To the extent that corrective actions require
resources, give them high-level consideration in the program planning and
budgeting process.
If progress on proposed corrective actions is slow, promptly consider
additional steps. Elevation of outstanding issues to progressively higher levels
of Agency management may be appropriate. Agency management at the
highest levels should be made aware of the significance of the corrective
action and the need for prompt action.
If possible, put systems and procedures into effect to prevent problems
from occurring in the first place. Sound systems and procedures assist
management in identifying problems early, and help prevent problems from
developing into major weaknesses that may undermine EPA’s mission.
35

-------
A couple of days after lowering the beaver dam, Howellfl out to
nw/ce sure that Sunnyside Stream was flowing smoothly and u’as
providing ample water for all the forest animal& He was very happy
to find lots of coo4 clear water throughout Sunnyside Stream. All the fish
were swimming freely again and all the forest anintals could enjoy a nice
refreshing drink.
36

-------
37

-------
STEP 7 ASSESS RESULTS:
Verify that corrective Actions Achieve Results
Answer the following two questions to validate the extent to which corrective
actions remedy identified problems:
+ What improvements were made as a result of review work?
+ Did those improvements achieve the desired result?
lithe answers to these questions are “none” and “no” respectively, reassess
your findings and corrective action plan.
Using the Feedback Principle, try to incorporate validation systems in your
corrective action strategy. For example, consider piloting solutions to
identified problems, then applying lessons learned from the pilot to a wider
solution to the problem. Experience derived from a pilot program can be
instrumental in confirming that a proposed solution to a problem actually
works in practice.
38

-------
Your corrective action monitoring and follow-up system should help
accomplish improvements. Incorporate the following basic elements of
successful monitoring and follow-up in your system:
+ Obtain a clear commitment to achieving action
+ Identify hasic ground rules and minimum required actions
+ Define individual responsibility, accountability, and authority
+ Identify what each recommendation is expected to accomplish
+ Determine continuously whether progress is being made
— Review implementation plans and timeframes for consistency with
recommended actions
+ Take actions to foster effective implementation
— Adjust corrective action strategy, plans, and milestones as
necessary to achieve the objectives of the program
39

-------
INTEGRATE RESULTS WITH Pouc DECISION
SYSTEMS
Remember the importance of seeing both the forest and the trees. Do not
conduct your program reviews in a vacuum. Channel program review results
into EPA decision making systems. For example, make sure that EPA policies
reflect the results of your program reviews. Assess the impact of reviews on
your program procedures. Elevate issues that are complex, cross cutting,
and/or outside of your span of control. Ensure that program review results are
part of your continuous program improvement cycle.
40

-------
Yon GOOD JUDGMENT Is ESSENTIAL
Every decision, regardless of its nature, requires judgment and involves
considering options, weighing the pros and the cons of each option, and
deciding which option is best. It is important to realize that you should always
exercise your own good judgment in assessing your program. This Guide
provides you with tools and techniques but you must decide how best to use
them.
In these times of fiscal austerity, you must conduct work in the most
economical and efficient manner. That’s why you must discern the most
pressing program priorities and problems. Address and solve them in the
most expedient and effective manner. Obtain a level of reasonable—not
absolute—assurance that problems have been solved!
41

-------
We hope that this Guide is useful to you and your staff in reviewing your
programs and operations and using EPA ’s Management Integrity Principles as
part of your daily work. And we hope you have enjoyed the story of Howell the
Owl and the Mystery at Sunnyside Stream. Please contact your office’s
Senior Resource Official or management integrity staff for assistance. You
may also contact EPA’s Office of the Comptroller in Headquarters for further
guidance, or additional copies of this Guide.
42 1).S. Government Printhg Office: 1996- 719-126’82716

-------
43

-------