I
ry
                                  DRAFT
                                                      U.S. EPA Headquarters Library
                                                           Mail code 3201
                                                      1200 Pennsylvania Avenue NW
                                                        Washington DC 20460
                               1990 Strategy Paper:

                       Regional Oversight of State  Programs
                                 October  9,  1980
,,,                     '•-! Agency
**•*& • •    - • .  -i»-i FM-311-A
401 M Street. S.W.
        l, DC
                                                         Bill  Garetz
                                                         Joan  LaRock
                                                         Stan  Meiburg
                                                         Tom Kern
                                                         Martin Geitz

-------
         -00/o
                             CONTENTS
Executive Summary

Introduction
Page

E-l

i
Part Is        Assessing the Current State of the Art

               Chapter 1:  How Oversight is Now Conducted	1

               Chapter 2:  The Evolution of State Programs.... 10
                             — Phases I, II and III

               Chapter 3:  Analysis of Alternative	14
                             Oversight Procedures
                             for Phases I and II

Part II:       Advancing the State of th_e_A.rt

               Chapter 4:  Shifting from Oversight	19
                             of Transactions and
                             Procedures to Oversight
                             of "Performance"

               Chapter 5:  Increasing the "Scope" of	26
                             Regional Oversight —
                             from Activity Management
                             to Project and Program
                             Management

               Chapter 6:  Monitoring "Performance" at	33
                             the Project and Program
                             Level

               Chapter 7:  Proposed Oversight Procedures	35
                             for Phases III, IV, V and VI

Part III:      Tying It All Together

               Chapter 8:  The Proposed Role of the	39
                             Regions and of EPA
                             Headquarters

               Chapter 9:  Concise Summary of Recommended	44
                             Oversight Procedures for Phases
                             I through VI

     Appendix A:  Alternative Approaches for Improving	A—1
                    Project-Level Review by the States

     Appendix B:  A Possible Experimental Approach for	B-l
                    Assessing a State Program's Impact
                    on Water Quality

     Background Paper:  Current Regional Oversight Methods
                        (Region-by-Region Descriptions and
                        Summaries) — [not attached]

-------
                     1990 Strategy Paper:
             Regional Oversight of State Programs
                      EXECUTIVE SUMMARY
     EPA is now delegating substantial elements of the
construction grants program to State agencies, in accordance
with Congress' mandate in Section 205 (g) of the .Clean Water
Act.  With the advent of delegation, Regions have had to
shift from directly operating the construction grants program
to overseeing how the States run it.  This shift has resulted
in the use of a wide variety of oversight practices in diffe-
rent Regional offices.  This paper attempts to: 1) catalogue
the different practices which Regional offices are now using
in construction grants; 2) analyze these practices and suggest
a pattern which seems to be evolving across Regions; and 3)
prescribe how Regional offices might proceed in the future of
full assumption by States of responsibility for the construction
grants program.

CURRENT OVERSIGHT PROCEDURES

     At this time, Regions currently rely on both formal and
informal methods of oversight.  The principal formal methods
now in use include:

     o   Review of Current Transactions;  In most Regions,
         this is a routine review of State documents requiring
         Regional approval.  Every Region receives a summary
         or checklist of what is contained in the package
         for EPA approval, detailing all of the significant
         supporting documents in a form previously agreed
         to.  In addition, two Regions carry out a selective,
         in-depth review of specific State actions {e.g.,
         facility plan review) as they are received in the
         Regional Office.

     o   Paperwork sampling:  Quarterly, most Regional Offices
         review, at the Regional Office, 10% of the paperwork
         received for Regional approval during the previous
         quarter.  In general, the sample is not totally
         random; Regions usually emphasize projects that
         are large, difficult, environmentally sensitive, or
         troubled.  A few Regions instead conduct an intensive
         semi-annual review at the State Office of 10% of
         recent paperwork.

     Despite the use of this formal oversight, informal over-
sight methods are the primary sources of information about
the State operations during the years immediately following
delegation.  These methods include:

-------
                             E-2
     o   Day-to-Day Communication;  Most Regional  Staff are
         in daily communication with staff in delegated States.
         The Regions consider such contact to be the primary
         means by which they become aware of problems within
         the State program.

     o   Regularly Scheduled Meetings with_ Staff Program Managers
         These meetings can be weekly, bi-weekly or monthly,
         and aid in problem resolution, discussion of new
         policy and information exchange.  During the first
         1-2 years after delegation, the Regions consider
         these meetings to be the second most important
         source of information about State problems with
         delegation.

REGIONAL OVERSIGHT PROCEDURES ARE CHANGING

     At present, most Regions use the formal and informal
oversight methods to review specific activities which States
perform (e.g., review of plans and specifications; promotion
of minority business enterprises; review of environmental
assessments).  Some Regions have moved beyond this, however,
to reviews which look not at activities themselves but at the
procedures which States use to conduct these activities in
specific cases.

     One Region, for example, conducts an in-depth review of
one or more projects in a particular State with an emphasis
on what the State could do to improve its procedures so that
".everything fits together" for better management of the
project.  Another Region reviews 10% of all on-going projects
and compares expenditures and costs to the project budget.

     Developments in these three Regions reflect a pattern
which we see occurring, over time, in other Regions and States
as well.  We feel that three distinct phases of oversight can
be discerned in the first two years since the signing of the
first delegation agreement (Phases I, II and III).  However,
most Regions are still operating in the first or second of
these phases, and are just beginning to look ahead to how
their activities should change as State capabilities improve.

     If EPA can succeed in developing its oversight relation-
ship further, we may anticipate three additional phases,
based on current trends and the program's ultimate objectives
(Phases IV., V and VI).  These phases of oversight should
evolve in conjunction with improvements in the State's ability
to carry out the constructions grants program.  Table E-l
gives a brief description of the six phases, and the report
describes each in detail.

-------
                             E-3
WE RECOMMEND THAT REGIONS USE OVERSIGHT TO HELP
STATE PROGRAMS GROW

     We believe that Regional offices should structure their
oversight methods in such a way that they encourage growth
in State capabilities.  Based on our survey of the Regions,
we would hope for the following desirable changes in Regional
oversight methods:

     o   In Phases I and II, Regions should oversee how
         States manage specific activities.  That is, they
         should focus on the management not of projects,
         but of distinct "activities" and procedures for
         carrying out these activities.  This would include
         making sure that States have adequate staff to do
         a realistic job in performing a given activity.
         By the end of Phase II, the State should have
         achieved such high quality and consistency in
         its management of activities that the Region.can
         reduce its formal paperwork sample from 10% to 5%
         of recent actions in each activity.

     o   In Phase III, Regional oversight should begin to
         emphasize coordination among activities to help
         ensure sound project-level management.  Success
         in Phase III would lead to Phase IV, in which
         the State and Region would focus on improving the
         management of the municipal wastewter program as
         a whole.  They would do so by emphasizing improved
         priority setting among projects and better long-
         term scheduling.  By the end of Phase IV,  the role.
         of the Region should have to have shifted from that
         of a "supervisor" to that of a "franchiser" or
         indirect manager, concerned not with specific
         details but with sound overall program management.

     o   In Phase V,  the State and Region would strive for
         full consolidation of the municipal wastewater
         program into a broader program of integrated water
         quality management.  Regional oversight would
         encourage States to coordinate the scheduling and
         design of municipal wastewater facilities with other
         point and non-point sources control efforts.  In
         Phase VI, the State should have achieved integrated
         water quality management.  Regional oversight, to
         the extent that is is still required,  would focus
         on identifying new methods or procedures which
         would help the State achieve greater efficiency in
         its operations.

-------
                             E-4

     These differing patterns of Regional oversight, based on
evolutionary changes in the quality of State programs, should
directly affect the kind of information EPA uses to oversee
State operations.  While at the beginning of Phase I, the Region
oversees State operations solely through the review of current
transactions, (i.e., by reviewing documents for quality after
the State review is completed), this should not remain the
dominant pattern of Regional oversight for long.

     By the end of Phase II, Regional oversight should have
initiated the review of State procedures and start to de-
emphasize the review of individual State transactions.  For
example, with regard to MBE, the Region should focus more
on how the State goes about ensuring adequate MBE partici-
pation, and spend relatively less time reviewing individual
MBE documents.  By the end of Phase III, the Region should
emphasize oversight through direct monitoring of State
performance.  To return to the MBE example, a Region might
assess a State's MBE performance directly by determining
what portion of project dollars have been awarded to MBE
firms rather than by reviewing MBE documents or procedures.
In later phases, the Region would increase its use of direct
measures of performance in overseeing State performance.
Ultimately, they might make use of indicators of environmental
quality to provide feedback on a State's management performance.

     Even in the most advanced stages of State delegation,
when the Regions are relying primarly on direct measures of
performance for overseeing State programs, we still believe
that Regions should continue a 2-5% sample review of indi-
vidual transactions (i.e., documents) for quality control
reasons, and should be prepared to review procedures when
performance (as measured directly) fails to meet expectations.
Nevertheless, we strongly urge Regions to move from oversight
based primarily on review of documents to oversight based
mainly on review of procedures, and finally to oversight
based primarily on direct measurement of performance, as
rapidly as the growth of a State's capability will allow.
This will encourage the Regional role to evolve from
"supervisor" to "franchiser" or indirect manager by the end
of Phase IV.  By Phase VI, the Region's role will approach
that of a true environmental manager.

-------
^ —
^^B *T1
m 35
w >
cn
w

^
M



O
03 g|
Oi CO
ft
fo cn
Oi ft
^ ft

ft
(0 y
1 P
CO
03
C P
0) O
p- P'
ft CO
K <
(B
3 a
^^^. 0J
A 3 p-
^V P 3
CD CD
3 1
CO
3
ft

£
3
(0
P
CO

pj
0*
o

0












j^
•
r
V
p*
5*
cn


^




0
tQ ft fD i-3
C 0 3 =r
P 4! O CD

P-il C CO
ft QJ 1 ft
>•< co P P
03 ft
3 P* CD CD
9) 3 3
3 ft CD —
P fD 3 £
03 03 ft p-
fD H — ' ft
3 (D 3-
(D ft D*
3 CD  ft
ft Q.03 3*
P- (D

P CO 53
ft fD
(D ftU3
1 O P-
0
£ 3
*1 CO
yif




cn ^3 co
p o P
3 <0 3
13 CO CD
CD Ml (1)
»l to
*-> o
H 3 P
(D O*
< Ui O

CD (D
O CD
0 X
MI p< n
CD
QJ totj
O dP ft
O
c ^rt ^T
3 P 3*
(0 13 fO
3 (D

CO ^ fD
*— ' 0 iQ
1 P-
yf o
3
CO

^d
IT]
J5
CO
M

H
<^


0
ft (D 13
0 3 31
£ O CD
(D 0
^ C CO
QJ 1 ft
co P (a
03 rt
^0 (D (D
W bj
O CD -^
03 3 £
M ft p-
P — ft
3 3*
1 tf
M CD ft
CD 03 31
< P" CD
CD 3
M CO p3
CD
3 ft 03
0) O P-
3 O

(0 1 CO
3 IT
fD
3
ft

O O 0

'•V tfP Mi S1 1 (Tel 3
fy 0 C^ * (U O
i"r3 ^^ j,,^ ^ ^ ^g u^ f^j
(DP'^lD^'t* C fD
H 3 O & * h &>
£ ^rt O W U^ (D QJ
Q U_* /ft ^p ^ 3 ^Q ^
^ [0 QJ Q
fQ O & CO M O
p- fl CO Mi Mi
CO C P- O 0
— 3 cn MI »i Q*
co 3 P-
O 3 O P- fl» 1
0 ft 3 3 3 ID
3 CO *O O O
rt n n CD ft
P- (DO
3 < <
C P- CD
(D CD QJ



*T3
CC
ft
CO
t'd

H
M
IH

0
2 ft CD H|
P O 3 31
D C n fD
P P O
03 i c cn
CO QJ ^1 ft
(B 03 ft
3 ft (D CO
ft 4 3
C (D —
CD 3 «
ft P-
*TJ '*-+ ft
hj y
0 0*
VJ. (O ft
CD 03 31
O P- CD
ft 3
1 CO Jd
M CD
CO rtiH
< 0 P-
CD O

1 CO
??*




000

p- QJ r1 o n o cn
O O CD Mi O Mi 0
<*> O co 3 3
C (Q *O ft *O (D
ft 3 M H- CD
O CD CD O 3 n C
3 3 0 C Mi cn
tn rt H3 fD CO 0 fD
<*P co 3* QJ Q. 11
. di c 30
13 CO n (D (H Ml
p p- CO 3 3
^J 50 to CO H3 O QJ
CD CO 3* CD P-
*1 O3 O P 1
SP- 3 CO CD
O P- O
•I 3 "1 CO ft
7f CD
03 < O 3
W 0 P- 3 CO
P CD (D P
3 co $ n co
13 CD C
H Ml O < 1
(D n Mi P- CD
o co co
3 *


^
a

CO
M

1-1
M


0
3 O
Q. ft
p-
o <
< p-
CO ft
*1 *^
CO 1
p-p-
03 CD
ft CO
§3
H P -
^ 3
P
03
CD
, CD
3
ft






O O

K. P- 0 O
CD (D Ml 1
< CO CD
P- CO U P
CO *1 ft
< CD 0 CD
Mi O 11
O Mi  ^J
0 CD 31
O 03 CO P
CO ffl
3 CD P-
CD CO (6
3
ft P- O
CO 3 3
ft
O 11 .
CO
ft <
31 P-
CD CD
^



•t)
X
^*
CO
td

H




o
p >
3 0
QJ ft
P-
0 <
<; P-
CD ft
1 "^
CO I
p- p<
03 CO
3* <
ft fl>
'S«
H^ (jU
»< 3
P
03
(0
1
3
ft






O 0

ffff
3 <
CO P'
CD
1 C
< O
P- Ml
CD
O O
Mi C
3
^O (D
O ft
O CO
CD
flit?
C h
fl P-
2 3

11
P-
P1

DT3
CO 3*
^ P
CD CO
P* CD
O
13 0
U i.
CD
3
ft
P O
3 0
Qj*O
CD
O =
Ml
O
yd MI
CD
03 cn
p- ft
O P
3 ft
P CD
H
O P
< 3
CD P
1 O3
n (o
p- 3
03 CD
y 3
ft ft





1


ft Q ^
y CD p-
CO CD 3
Qj Pj
CO
ft Mi O
P O Ml
ft H
O 1 1
^ CD 3
O CD P
03 CD ft
n P- P-
P 3 O
3 ua 3




1
1




-------
               Construction Grants Issue Paper:

             Regional Oversight of State Programs


     An earlier paper on the role of EPA in the construction
grants program identified three distinct Roles that EPA could
play in a program whose activities have been largely delegated
to the States.  These roles can serve as models around which
the Agency's managers can more clearly define the goal of
full program delegation which we hope to achieve by 1990.

     The purpose of this paper is to develop these roles in
more detail by outlining, in a preliminary way, how the
Regions might go about making the transition from the first
role, that of a "supervisor," to the second role, that of a
"franchiser" or indirect manager.  It also shows how to begin
to make a further transition to the role of environmental
manager.  (For a brief description of the three possible
roles, see Table 1.)

     Specifically, the paper attempts to:  1) catalog the
different oversight practices which Regional offices are now
using in construction grants? 2) analyze these practices and
suggest a pattern which seems to be evolving across Regions;
and 3) prescribe how Regional offices might proceed in the
future to make oversight both more effective and more supportive
of full assumption by States of responsibility for the
construction grants program.  The description and analysis of
current practices forms Part I of the paper; Part II suggests
directions for the future, and Part III recapitulates and
summarizes the conclusions and recommendations reached in the
first two parts.

PART I;  ASSESSING THE CURRENT STATE OF THE ART

     The paper begins by summarizing — in Chapter I — the
methods now used by the ten Regional Offices to oversee State
programs.  This summary is based on a background paper
(distributed separately and not a part of this package) which
contains relatively complete though brief descriptions of how
oversight is now conducted in each of the Regional Offices,
and one- or two-page summaries of current oversight methods.*
* The information present in the Background Paper forms the
factual basis for the overview of current procedures in
Chapters 1 and 2 and for the analysis and recommendations in
Chapters 3-8.

-------
 6
 id
 fc
Oi

-P
 c
O
-P
 o
 3
 fc
•P
 co
 C
 0
u

•a
 CD
-P
 CD

 CD
O

 (0

 C
 fc
 o
 01
 CD
rH

£

 CD
 CO
 CO
 o
 04


 CD
 CD
 fc



rf
&
W

MH
O ••
.p
to 3
0) 0
r-l
O *O
Pi *H
id
CD rH-
ff,

- fc
CD
T> 3
CD
H CO
4J CD
•rl rH
•P O
fc

c tn
id c
-H
^i 3
tn o
CD H
•P rH
id O
fc MH
jj
CO CD
rj
0 *J

en :
iH -
E
CD Id

4J tn
O
fc fc
O Cn
to
'O -P
CD C
fc id
id fc
ftU
CD
fc C
ft O

fc -p
CD O
ft 3
ft -P
to
-P C
MH O
id U
fc
•C CD
id -P
c c
r-C -H
O
fc
+J
c
o
o

Jj
c
CD
g
CD
tn
i
CD

-P

CO
id

CO
CD
-P
TO
^J
CO
fc
CD

0









.jj
U
CD
fc
•H
•O

E
0

44

to
CD
T3
CD
O
CD
ij

rfj
0,
pa

jg
o
•H
c*
3

C
•H


•*
fc
CD
tn
id
c
id
g

4J
O
CD
fc
•rH
TJ
n
•H

S_l
0
E
fc
CD
to
•H
r;
U
G
id
fc
En
"

0






4J UH
fc O

ft C
ft O
3 -H
CO +J
id
CD fc
> 4J
.rH CO
» -H
C C
(!) -rl
•P £
X -O
CD Id

£, *O
4J C
•H id
u
..
co tn
CD C
CD -H
4J g
C C
id id
fc H


•0 H
C id
id o
CO
CO -H
CD HH
-P
id -
JrJ JS,,
CO +J
•H
tQ rH
CD (d
•C 3
*H D"1
>
O rH
fc Id
ft o
-H
-P C
3 ,C
Xt 0
- +J
C
O CD
•H >
to O
•H fc
> a
fc E
CD -H
ft
3 0
CO 4->









t7i
C
•H
fc
0
•H
C
0
E

;
r-l
O
fc
4J
C
O
o

>^
4J
-H
f—i
id
3
JJ1







C
CD
^

C
O
•iH
-P
id
fc
-P
CO
•H
C
•H
g
-s
(d

E
id
fc
tn
0
fc






























•







r-f
id

CD
-P
id
1 \
V3

CD
,c
4J

0
4.)

CO
CD

id
tn
CD
rH
CD
ft*O T3
-P
U
3
T3
c
o
o

0


CO
CD
3
C
•H
fi
O
u

<;
Cu
W

•
E
id
fc
tn
0
fc
ft
CD

^






c


to
CD
C
CD

fc
CD
-P
C
•H

T3
C

O
o
to
*H
"0

CD
fc
id

CO
CD
•H
O
C
CD
-H
O
•H
MH
CD

-P
C
0}
u
-H
4H
•H
C
tn
•rH
CO






f3?
cx
fr3

Xi
O
•rl
X


c
-H

*
*
• *
fc
CD
tn
td
c
id
s
rH
id
.p
c
CD


o
fc
•H
>
c
H

0








c
H
id
fc
CD
u

4J
/d

4J

C
o
•H
4J
,
4J
•H
fc
O
.P
3
id












CD
.C CD
&* -H

CD
• 0
•P C
CD Id
E E

CD 0
& MH
fc
CO CD
•P ft
CD
tn CD
fc -P
id ^
-P CD
•H rH
rH ,£
Id Id
3 4J
O1 ft
CD
rH O
Id O
4J fQ
c
CD tn
g c
C -H
0 fc
fc 3
•H tO
> C
C CD
CD
fc
fc 0
O , to
•P E
•H CO
O -H
Id C,
ft id
i
g fc
4-» id
id g
CD •<-!
fc fc
•P ft

















CD
tn
fc •
id cu
rC H
0 3
CO ,
O id
CM -P
CO
-P
CO O
C -P
•H
(d tn
tn c
id -H
H
C -H
o id
•wl tlj
w^ ^*<
ij
id 0

4-> CO
c c
CD O
E -H
CD -P
O 
-------
                              11

     In Chapter 2 we describe how oversight tends to evolve
during the first two to three years after delegation.  We
introduce the concept of phased development of State programs
and describe Phases I and II.

     In Chapter 3 we then analyze the oversight procedures
currently in use and identify what we have found to be the
best procedures for use in Phases I and II, the period
immediately after delegation.  In this period, EPA is still
clearly operating in the role of "supervisor."  This analysis
completes our assessment of current oversight procedures.

PART II;  ADVANCING THE STATE OF THE ART

     Part II builds on the description and analysis in Part I
with specific recommendations for changes in Regional oversight
methods.  Chapter 4 looks at the current efforts of the
Regions to move from oversight based on the review of individual
State transactions to oversight based on the review of State
procedures.  We endorse these efforts and urge that the
Regions move a step further and begin to conduct direct
monitoring of State performance.  We give meaning to the
concept "direct monitoring of performance" by citing specific
examples of how "performance" might be measured.

     Chapter 5 continues the analysis of possible new direct-
ions in oversight by examining the potential "scope" of
Regional oversight.  We begin by looking at current Regional
efforts to encourage States to move from activity management
to true project management.  Again, we .endorse these efforts
and recommend that Regions continue to strive for a further
shift in the States from project management to program manage-
ment , that is, balanced management of the municipal wastewater
program as a whole.  We also recommend that, where possible,
Regions and States move to incorporate the construction grants
program more fully into a broader program of integrated water-
quality management.

     Chapter 6 ties together the distinct concepts and
objectives presented in Chapters 4 and 5 by suggesting how
the Regions might go about overseeing State performance
(rather than merely overseeing State transactions or pro-
cedures) once the State program is able to carry out true
project management.  Appendix B to this paper contains a
possible experimental approach for measuring State perfor-
mance in Phase IV and beyond when a State has begun to
achieve full program management.

     Chapter 7 follows this up by describing Phases III
through VI in some detail.  This description includes a
projection of how a Region might go about overseeing a State
program during an extended period after delegation, as the
State shifts first to project management and then to full
program management.  Chapter 7 also completes our analysis

-------
                              Ill

of how the  Regions might go  about  shifting  from  the  initial
role of direct  "supervisor"  of  State activities  to the  second
role of "franchiser" or indirect manager, pointing out  that
Regions should  achieve the role of franchiser by the end  of
Phase IV.   When a State reaches Phase VI, the Regional  role
should begin to approach that of an environmental manager.

PART III;   TYING IT ALL TOGETHER

     Part III summarizes the conclusions and recommendations
arrived at  in the previous two  parts of the paper.   In  Chapter
8, we review the responsibilities  and objectives both of  the
Regions and of  EPA Headquarters under the blueprint  for program
development and oversight developed in Part II.  Chapter  9
concludes the paper by summarizing the recommended methods for
each of the six phases of oversight which we have identified
in the course of the paper.   Table 3 contained in Chapter 9
presents a  detailed but concise summary of these recommendations
     Virtually all of the analysis contained in this paper is
based on - interviews with Regional staff.  In five Regions
(Regions I, IV, VIII and IX), the study team conducted fairly
extensive fact-to-face interviews over several days with
numerous Regional and State officials, and with officials in
grantee communities and in the Corps of Engineers.  In another
Region (Region III), one team member conducted limited face-
to-fact interviews.  In all six of these Regions, there were
extensive follow-up telephone interviews with one or more
Regional staff members.  In the remaining four Regions (Regions
II, VI, VII and X), we relied solely on telephone interviews,
typically with a single member of the Regional staff.  We also
used Larry Walker's written analysis of Regions and States
where these analyses were available (Regions IV, V, VI and
VII).

     Since this draft is based primarily on interviews with
Regional staff, it should be considered highly preliminary.
As we receive comments and additional information from the
Regions and from other participants in the program (i.e.,
headquarters staff, State staff, A/E firms, public interest
groups, environmentalists and others) we will make any
necessary revisions to ensure that the paper presents a fully
balanced analysis and reflects the insights and perspectives
of all participants in the construction grants program.
Comments should be forward to:

          Bill Garetz,
          Program Evaluation Division (PM-222),
          401 M Street, S.W.,
          Washington, DC  20460
          (tel: (202) 755-3977;
                (FTS) 755-3977).

Handwritten comments including "markups" of this paper are
welcome.   Telephone comments are also welcome.

-------
PART I:  ASSESSING THE CURRENT STATE OF  THE ART

-------
         CHAPTER 1:  HOW OVERSIGHT IS NOW CONDUCTED
     All Regions are now already overseeing State
construction grants programs.  They are doing so under general
guidance from the Office of Water Program Operations (OWPO),
but each Region has developed its own specific oversight
methods.

     This chapter briefly describes the specific approaches
used in each Region.  For a more detailed survey, the reader
should refer to our separate background paper, "Current
Regional Oversight Methods."

THE REGIONS HOW USE A NUMBER OF APPROACHES

     No two Regions use exactly the same methods of overseeing
construction grants programs in delegated States.  This
apparent diversity, however, masks a number of common threads
which appear to run through most existing Regional oversight
programs.

The Basic OversightProceduresAre The Same

     Each Region uses a slightly different combinations of a
limited set of basic approaches.  Although specific implementation
varies somewhat from Region to Region, the similarities are
quite striking.  Since the name applied to a given approach
varies from Region to Region, we have taken the liberty of
assigning a single name to each approach to minimize confusion.

     We have grouped the Regions'  oversight methods into two
categories — formal and informal methods.  Table 2 contains
a list-of all these methods, which are described in more
detail below.

FORMAL OVERSIGHT METHODS
     Formal oversight methods are those which the Regional
office and the State have agreed to as "standard, operating
procedures."  Regions now carry out two such types of oversight
and focusing on a) specific activities, or b) individual
projects.

Activity—level Reviews

     Most existing formal oversight methods now focus on the
State's management of specific "activities."  Examples of

-------
                             -2-
Table 2:   Summary of Oversight Procedures Now Used by ths
           Regional Offices
FORMAL OVERSIGHT PROCEDURES

Activity - Level Reviews

1)   Review of Current Transactions

     o   Cursory review of all incoming packages
     o   Selective in-depth review of incoming packages

2)   Review of Recent Transactions: "Paperwork Sampling"

     o   Paperwork sampling at the Regional Office
     o   Paperwork sampling at the State Office

Project-level^ Reviews

1)   Project-level review of transactions and procedures

2)   Project-level performance review

     o   Cost review
     o   Timeliness review

Program-level Reviews

Review of the State's Internal Administration

1)   Review of administrative procedures

2)   Review of administrative performance — monthly
     "report cards"
INFORMAL OVERSIGHT PROCEDURES
1)   Day-to-day communications and feedback

2}   Regular meetings with State program managers

-------
                              -3-
such activities include State reviews of MBE actions. Environ-
mental Assessments, and review of plans and specifications.

1)   Review of current transactions;  This consists of a
routine review of documents and document packages submitted
by the State for EPA approval.  It can take one of two forms,
a cursory review or an in-depth review.

     o   Cursory review of all incoming packages.  Virtually
         every Region reviews a State checklist before accepting
         the State's evaluation of material requiring EPA
         approval.  In at least one Region (Region I), this
         "cursory review" is somewhat more elaborate, but is still
         quite distinct from the "in-depth review" described
         below.  Region I examines not only the State checklist,
         but also the State approval letter and and applicable
         cost figures to ensure that they are reasonable.

     o   Selective in-depthreview of incomingpackages.
         Some Regions(e.g.,RegionsI and II)conduct a
         more detailed review, on a selective basis, of
         specific activities whose results are contained
         in State-submitted grant packages.  In these
         Regions, the 'frequency of in-depth reviews depends
         on:

         -  the importance of the activity;

         -  how long it "has been delegated ;

            the adequacy of State review (as based on the
            quality of recent transactions); and

         -  the existence of any State requests for
            assistance.

     If a State's conduct of a specific activity is found to be
     inadequate,  one Region (Region I) responds with the following
     corrective actions:

         -  preparing a sample review for State reference
            (e.g., if the environmental assessment for a
            project is inadequate,  the Regional staff will
            re-draft it properly so that State staff can use
            it as a sample for future reference);

         -  going to the State office and conducting a review
            side-by-side with the State staff member; or

         -  providing more formal training for the State
            staff member.

-------
                             -4-
     The Region will also monitor subsequent State actions in
     this activity closely until quality improves, and stay in
     contact with the State to help solve continuing problems.

2)    Review of Recent Transactions — "PaperworkSampling";

     "Paperwork sampling" refers to an in-depth review of
     a sample of transactions received by the Regional
     office in the previous 3 to 6 months.  This paperwork
     sample can take one of two distinct forms.

     o   Paperwork Sampling at the Regional Office

         Most Regions conduct this paperwork review in the
         Regional Office on a quarterly basis,  relying
         solely on material the State has submitted.  The
         Region selects a 10% sample of the State's actions
         in each activity for a detailed check.  In general,
         the sample selected is not totally random.  Most
         Regions see to it that the sample a) includes some
         small projects, but b) gives more than proportional
         emphasis to projects which are large,  environmentally
         sensitive, unique or otherwise "difficult," or
         believed to be "troubled."

         The review is fairly mechanical and is concerned with:

         -  inspection of documents to make sure that the
            State has filled them out correctly;

         -  review of forms and checklists to make sure the
            State had updated them to incorporate recent
            changes in regulations or procedures; and

         -  review of the document packages to ensure that
            the State has submitted the proper support
            doucments and checklists.

         In some Regions, this review may take as much as two
         months or more to complete.

     o   Paperwork Sampling at the State Office

         Several Regions now sample paperwork at the State
         office, for the most part on a semi-annual basis.
         The activity is similar to sampling at the Regional
         office, except that Regional staff go to the State
         office to draw on the State files, and do not rely
         on material sent to the region to conduct the review.
         Two Regions (Regions I and X) often go to individual
         project sites as well.  Another difference is that
         Regions normally complete reviews at the State Office

-------
         within 2 to 5 days, rather that spreading them out
         over weeks or months.  Two Regions  (Regions V and IX)
         stagger their visits, looking at only a handful of
         activities on each visit, but visiting the State
         Office monthly so that all activities are reviewed
         every six months.*

The paperwork sample closes with an exit interview with State
officials in which Regional staff discuss any problems they
have found.  The Region then follows up with a formal report
to the State which focuses on major problems.  One Region sends
this final report within  two weeks of the review, while another
does not normally send it for two months.

Project-Level Reviews

      Project-level reviews focus on the State's "project
management." "Project management" here refers to the State's
success in ensuring that  each project as a whole is technically
adequate, environmentally sound, cost-effective and carried
out in a timely manner.   There are two types of project-level
reviews:  reviews of transactions and procedures,
and reviews of performance.

1)   Project-level review of transactionsand procedures.
     One Region(Region IV) hasjust begun to conduct such
     reviews, and calls them "program management evaluations"
     or "program oversight reviews."  (We will use this term
     later to cover a different and somewhat broader concept
     of oversight.)  In its first such review, conducted in
     May 1980,  Region IV selected a single project and had
     specialized staff examine the State's handling of each
     distinct activity in that project.  The review took
     one week,  with Regional staff meeting each evening to
     compare notes and identify coordination problems within
     the State office.   The Region then met on the last day
     with key State staff to present the problems they had
     discovered.  The emphasis was on what the State could
     do to improve its procedures so that "everything fits
     together"  on any given project.
* One Region(Region X)varies its level of review project-by-
  project based on the Regional staff's judgment as to the
  likelihood of problems.

  Region X's levels of review are:

         -  checklist review - has the checklist been properly
            followed and completed?

         -  content review - has the activity been performed properly?

         - . in-depth review - EPA duplicates the State review to
            determine whether it was conducted properly.

-------
                             -6-
2)   Project-level performance review;

     "Performance reviews" focus not on individual transactions
     per se, but on whether individual grant projects as a whole
     are proceeding satisfactorily.  Only one Region now conducts
     them,* though one other Region plans to adopt them in
     the future.  So far, two different types of "performance
     reviews" have been conducted:  a "cost review" and a
     "progress review."

     o   Cost Review

         One Region (Region VI) conducts an annual review
         which focuses on identifying and correcting problems
         by examining project costs.  The Region selects 10%
         of all on-going projects and compares costs to
         the project budget.  Unlike the paperwork sample,
         this review covers a sample of all on-going, active
         projects; the paperwork sample is limited to projects
         on which State actions have been taken during the
         previous 3 to 6 months.  Another Region (Region IV)
         now plans to conduct the same kind of review on an
         annual basis.

     o   T ime1i ness Rev Jew

         In conjunction with its planned cost review, one
         Region (Region IV) also plans to conduct a time-
         liness review (which it calls a "progress review")
         to determine whether projects are proceeding on
         schedule.  Region IV will conduct both reviews on
         the same sample of on-going projects.

Program-1eve1 Reviews

     Several Regions (Regions IV, V, VII,  and X) stated that
they wish to modify the nature of their semi-annual reviews
to emphasize "program management" rather than just "project
management."  These Regions, however, are just beginning to
think through what a program management review might consist
of? they are not actually conducting such reviews at this
time.
*~"Ohe Region (Region IV) has taken an initial step in this
  direction with the project-level review of transactions and
  procedures described above.

-------
                             -7-


Reviewof the States InternalAdministration

     Two Regions are now conducting such reviews:  one Region
does so by examining the State's administrative procedures,
another does so by looking at the State's administrative
performance.

1.   Review of administrative procedures

     One Region (Region X) conducts such a review two weeks
     prior to its paperwork sample at the State Office.  This
     earlier review (also conducted at the State Office)
     examines the adequacy of the State's operations in certain
     key administrative areas:  staffing, training, grant
     processing, and payments.  The findings are reported  along
     with those from the Paperwork Sample.

     Region II is proposing to conduct a similar review at
     the State Office prior to a semi-annual Paperwork Sample.
     Both Region X and Region II call the review of the
     State's internal administration a "functional review."

2.   Review of administrative performance —monthly "report card"

     Several Regions have spoken of their desire to issue
     monthly "report cards" on State performance.  None,
     however, has been specific about how it would go about
     generating such reports.  One Headquarters staff member
     suggested that what the Regions probably have in mind
     is a State report card similar to the current "Regional
     report card", which evaluates several aspects of the
     performance, including the Region's:

     o   progress to date in obligating funds

     o   time delays in reviewing current transactions, etc.

INFORMAL OVERSIGHT PROCEDURES

     In addition to the formal procedures described above,
all Regions rely heavily on two highly informal methods for
overseeing State operations.  Most Regions concede that these
informal procedures are their primary souce of information
about State operations and operational problems during the'
period (lasting two years or more) immediately after delegation.

Day-to-day communications and feedback

      Most Regions are still in day-to-day telephone contact
with State staff to discuss the conduct of delegated activities.
These Regions consider such contact to be the primary means
by which they are alerted to problems in the State program.
In most cases,  Regional staff attempt to correct such problems
by giving immediate feedback and guidance over the phone.

-------
                             -8-
Regular Meetings with StateProgram Managers

      In many Regions, EPA's senior staff meet  frequently
with the State Program Director and his key staff.  The
purpose of these meetings is to bring the State program
managers up to date on new requirements and to provide a
forum for exchange of information.  According to the Regions,
they are also an extremely important informal source of
information regarding difficulties the State is having in
implementing delegated activities.

      Although the "EPA senior staff" who take part in these
meetings varies from Region to Region, it typically includes
one or more branch chiefs, the section chief responsible for
the State and the 205(g) coordinator.  In many cases, these
meetings take place monthly.  In other Regions, they take
place weekly or every two weeks.

      In most cases face-to-face meetings are preferred,
although conference calls are often substituted in order to
conserve travel money.  Meetings may take place either in the
State office or in the Regional office.  Top Region "IX staff,
for example, meet monthly at the State office with the managers
of the California program.  In Region V, the State program
managers meet monthly with the Regional staff at the Regional
office.

WHY ARE THERE DIFFERENCES AMONG REGIONS?

      Although for the most part no single oversight procedure
is unique to any one Region,* there are significant differences
between Regions in the mix of procedures used and in the relative
emphasis given to those procedures.  It has been suggested that
the differences might be due to three factors:

     o   the elapsed time since an activity was delegated;

     o   the adequacy of State staffing, and

     o   philosophical differences between Regions

      Surprisingly, we found that philosophical differences
between Regions appeajr^to be a relatively minor factor in
determining how a Region approaches oversj.jght.  Although such
differencesstrongly affect_how a Region describes its
intentions/when the^ jx:tual^conduct of oversight is examined,
the differences in practice turn out to be much less pronounced.
* The current exceptions are the use of "Project-level Reviews
of Transactions and Procedures" in Region IV, the use of
"Timeliness Reviews" in Region VI and the use of Review of
the State's Internal Administrative Procedures in Region X.

-------
                             -9-
     Instead, the principal factor governing how a Region
conducts oversight appears to be the time elapsed since the
State began to accept delegated activities.  Indeed, this
tendency is so pronounced that it is possible, based on the
testimony of Regional Staff, to discern a natural evolution
of oversight practices over time.  In the first two years
after delegation, we have identified at least three distinct
"phases" of State development and Regional oversight.  These
three phases will be described in considerable detail in the
next chapter.

-------
CHAPTER 2;  THE EVOLUTION OF STATE PROGRAMS —
            PHASES I, II, AND III

      In examining the descriptions of State programs and of
oversight methods used by the Regions during the first two to
three years after delegation, we were able to identify three
distinct "phases" of development.  Briefly, they are:

     o   Phase I — the initial phase — which normally
         encompasses the first 12 to 24 months after delegation.

     o   Phase II — in which the focus is on improved
         activity-level management by the State — and which
         normally encompasses the next 12 to 24 months after the
         completion of Phase I.

     o   Phase III — in which the State begins to undertake
         and the Region begins to promote true project-level
         management.

Later in Part II of the paper, we will discuss stages of
development beyond Phase III.  No State or Regional office
has yet entered these later stages.  We should note here, however,
that we do not regard Phase III as an endpoint for Regional/
State development, but as a milestone on the road to further
progress.

THE THREE INITIAL PHASES OF DEVELOPMENT OF A STATE PROGRAM
  AFTER DELEGATION

     We now present a composite picture of existing State-
Regional relations and of Regional oversight procedures during
the first three phases of State program development.  This
composite picture is based primarily on the testimony of
selected Regional staff.  The picture is most complete for
events in Phase I? Phases II and III are necessarily more
conjectural since fewer States and Regions have progressed to
this point.

Phase I — The Initial Phase — the First 12-24 Months After
  Delegation

     After EPA and a State conclude and sign a 205(g) delegation
agreement, Regional offices begin the process of disengaging
from the direct review of grant documents and begin to transfer
this responsibility to the State on an activity-by-activity
basis.  States then begin to conduct these reviews themselves.
When the States completes its review of a particular document

-------
                             -11-
package, it submits the package to the Regional office for
approval.  Accompanying the package is a "checklist" which
sets forth, in outline form, the requirements which the
documents had to meet and specifies whether or not these
requirements were in fact met.  The packages themselves
normally include all required back-up documentation.

      Immediately after the delegation of an activity, the
Region reviews each package carefully.  Any problems result
in an immediate phone call to the State staff member who
prepared the package.  During this period State staff also
frequently phone their Regional counterparts for advice and
guidance.

      For those activities which a State consistently performs
well, the intensity of the Regional review gradually decreases.
After a few months with no problems, the responsible Regional
Staff member feels confident enough in the State staff member's
performance that a perfunctory review of the checklist is the
only review necessary.  The intensity of review for -an activity,
increases, however, whenever there is a change in the State
staff performing that activity.  In other words, in Phase I
the Regional staff doesn't trust the State's performance as
much as it trusts the performance of the individua1 member of
the staff.

      No formal paperwork sampling is necessary at the be-
ginning of Phase I since effectively a 100% sample is subject
to in-depth review.  Later in the year, or early in the
second year after delegation, the Region begins to conduct
formalized paperwork sampling in the Regional Office for
activities in which trust has developed.  The paperwork
sample focuses on problem areas and problem projects as
revealed through daily telephone communications.

     Typically, there are no formal reviews  conducted by the
Regional Staff in the State Office before the end of this
first year.  This is because the Regional and State staff
are in close communication and feel they are aware of each
other's problems and concerns.

     By the end of the first year, calls from the State staff
for guidance have diminished considerably.   Also, Regional
staff are relying solely on a checklist review of incoming
documents for most activities.  State performance has reached
the point where the number of calls initiated by Regional
staff to inform State staff of problems has diminished markedly.
Most calls from the Region are either to inform the State of
changes in regulations or procedures, or to follow-up on
specific projects that are particularly troublesome.  Never-
theless, the State is still playing "catch-up" throughout this

-------
                                                                          I
                             -12-
phase.  Most management attention in the State and in the
Region is devoted to the initial problems of implementing
delegation, rather than to ensuring smooth coordination of
non-crisis activities.

     This phase continues until all delegable activities have
been assumed by the State and all are performed with "reasonable"
adequacy and consistency.

Phase II — Striving for Consistent, High Quality Activity
Management —the 12-24 Months After the Completion of Phase I

     With the significant decrease in daily telephone contact
between State and Regional staff, formal review procedures take
on much greater importance.  Formal paperwork samples are
conducted regularly; the Regional staff begins to conduct formal
reviews at the State office on a regular schedule (every six
months).  Although telephone conversations with State staff
diminish, they are still the principal source of information
about problem areas.

     At this time, the State begins to take charge of its
managerial responsibilities and takes steps to ensure consis-
tently high quality performance of each activity it reviews
(i.e., MBE activities, change orders, plans and specifications
preparation, etc.).  By the end of Phase II, this objective
has been achieved:  allState activity-level reviews are of
uniformly high quality,especially those of basic administrative
and engineering activities (see Table2 at the end of Chapter 5)•
areof uniformly high quality.  This does not mean that they
are error-free, but that they have achieved a quality and
consistency more or less equivalent to that achieved prior
to delegation by EPA's Regional Offices.

     At the end of Phase II,  the quality of State activity
reviews should be so high that the Regional Office can review
a 5% sample of paperwork in place of the 10% sample originally
used, without jeopardizing adequate quality control of the
State's operations.

Phase III — Striving for Effective Project Management — the
18-24 Months after the Completion of Phase II."

     Phase III begins when the Region's confidence in the State's
performance of all activities, especially basic administrative
and engineering activities, reaches the point that the Region can
rely on a 5% paperwork sample.  The Region then begins to focus
on the State's ability to review and assess projects as a whole

-------
rather than just as a collection of activities.  Meanwhile,
the State is also beginning to focus on improving coordination
among separate activities.  The State thus starts to draw
independent, uncoordinated "activity reviews" into more
coordinated "project reviews."  Yet, while the Region begins
to review the effectiveness of the State's overall project
management, it still continues, on a less intensive basis,
to review the quality of the State's conduct of specific
activities.

     By the end of this period, the Regional staff has con-
siderable confidence in the State's ability to carry out
its delegated responsibilities at two levels:  project review
as well as activity review.  There is now trust in the
competence and effectiveness of the State's operation as a
whole as well as in that of specific individuals on the
State staff.

FURTHER PHASES OF DEVELOPMENT

     Our descriptions of Phases I, II and III, complete those
phases of development which we have actually observed to date, in
however rudimentary a fashion.  We believe, as noted above, that
the process of delegation is still in an early stage, and that
States and Regional offices should pass through three more phases
as the process continues.  Part II of this paper (Chapters
4-7) describes what these further phases might look like.
     We have discerned this general pattern of development of
Region/State relations despite the current differences in
oversight procedures within and between Regions.  One factor,
however, can frequently prevent this normal pattern from
being followed.  This factor is inadequate numbers of State
staff.  Inadequate staffing can keep even a very strong,
capable state like New York from conforming to the rate of
development we would expect to see based on our observation
of the experiences of other States.  One Region {Region VIII)
in particular expects to continue operating for an extended
period in Phases I and II in most of its States because of
the great likelihood of continued inadequate State staffing
levels.  The issue of adequate State staffing clearly requires
resolution very early in the delegation process.  It is the
sine qua non of further progress by both the State and the
Region; without it, neither States nor Regions can expect to
make much headway.

-------
                             -14-
CHAPTER 3:  ANALYSIS OF ALTERNATIVE OVERSIGHT PROCEDURES FOR
            PHASES I AND II

     To be useful, any generalized oversight model must reflect
both the varying stages of development of Region/State relations
and the prospect of continued inadequate staffing levels in many
States.  We therefore propose that EPA develop guidance which
formalizes the lessons learned in the last few years in dealing
with such conditions.  In particular, we propose that oversight
procedures be tailored to the phases of development described
in the previous chapter.

     In this chapter we will examine the situation of States
with less fully developed programs {i.e., States in Phase I,
Phase II or in the early part of Phase III) and propose
standard oversight procedures for use in such States.  Since
most Regions now have had at least some experience in .over-
seeing the activities of States in Phases I and II, we have
drawn heavily on the experience and the insights of staff in
the Regional offices in proposing oversight procedures for
these two phases.

PHASE I -~ HELPING THE STATE GET UP TO SPEED

     Regional staff paint a consistent picture of how State
staff react to delegation.  While prior to delegation the State
staff had anticipated having more control over the program in
their State and had confidence in their ability to take over
the activities previously carried out by the Regional Office,
on the day of delegation some of the State's working staff may
suddenly be overwhelmed.

     Some State staff may be overwhelmed at first by what they
perceive as the great personal responsibility they now bear
for managing their portion of a multi-billion dollar program.
Some may also be overwhelmed by the complexity of the projects
and the intricacy of the review requirements with which they
must deal.  They may be overwhelmed as well by the variety of
unexpected problems and situations which accompany delegation.
Faced with this sudden burden, they know they need assistance
from "those who have been there before" and they readily call
on the Regional staff for advice.  They normally find that
Regional staff are quite responsive to requests for advice
or assistance.

     At the same time, delegation tends to be a wrenching
experience for Regional staff as well.  Where they had
previously borne considerable personal responsibility for a
major Federal program and had extensive personal involvement
with the "nuts and bolts" of that program,  now many feel as
their role changes though they have suddenly become mere

-------
                             -15-
"observers".  As a result, key Regional staff often seek
other jobs.  The sudden loss of these staff represents the
loss of valuable experience and insight that could be of.
great benefit to the State staff now responsible for carrying
out the program, especially during the first 12-24 months
after delegation.

Informal Contacts Are Important

     Given these circumstances, it seems appropriate to soften
the impact of delegation both on the State staff and on the
Regional staff by encouraging free interaction between them.
Regional managers should especially encourage the natural tendency
toward steady — normally daily — interaction in the form of
frequent telephone calls in both directions.  State staff should
want Regional staff to phone at once when the Regions becomes
aware of specific problems in the way the State is carrying out
a certain activity.  Indeed, a common complaint of the States
is that they don't want the Region to wait for a formal quarterly
or semiannual review to tell them about deficiencies in their
operations.  On the contrary, if there's a problem, the State
staff wants to know about it i mmed iate1y, so they can take
corrective action at once.

     Similarly, close communication between the State and the
Region at the supervisory level is also desirable during the
first phase of delegation.  Most Regions have stated that
they have found it useful for the Region's program managers
(at the branch and section chief level) to meet with the
State's program managers at least monthly throughout this
period, and preferably every two weeks during the first six
to twelve months.  The purpose of these meetings is to discuss
and begin to resolve current problems, as well as to allow
the State program managers to draw on Regional experience in
dealing with unusual or especially difficult-projects.  Since
travel funds are always limited, some Regions have substituted
conference calls for many of these meetings.  Still, many
Regions have found that frequent face-to-face meetings are
highly desirable and should be held at least once every
month or two.

Formal Contacts Should Be Regular Too

      The same principles hold for formal as well as informal
aspects of Region/State interaction.

      Review of Current Transactions;  Immediately after an
activity has been delegated, the Region staff should conduct
an in-depth review {essentially a duplicative review) of
State document packages related to that activity that are
submitted for Regional approval.  These in-depth reviews can
serve as "models" for the State, and should continue until
the State staff is performing that activity properly.  Any
problems discovered should be brought to the attention of the
responsible State staff member at once by telephone.

-------
                             -16-
      Initial in-depth reviews and rapid telephone feedback
should result in early correction of most deficiencies in
State staff performance.  Once initial problems are cleared
up, the intensity of Regional review should (and normally
does) ease off considerably.  (This has apparently been the
case in Regions I and II.)  As it becomes clear that the
State's performance of a certain activity is consistently of
good quality, the Region should reduce its review of current
transactions to inspection of the attached checklist only.

      Paperwork Sampling;  During the first quarter or two,
no formal paperwork sampling may be necessary since the
Regional staff is still carefully scrutinizing most State
packages coming to the Region.  Later, as State performance
improves and the Regional staff begins to rely more on check-
list reviews, formal paperwork sampling should begin.

      There are essentially two approaches now in use among
the Regions for conducting paperwork sampling:

      o   Quarterly reviews within the Regional Office of a
          sample of packages received the previous quarter.

      o   Semi-annual reviews within the State Office of
          State file copies of packages sent to the Region
          during the previous six months.

      Although each approach has its advantages, we recommend
that Regions conductthese reviews quarterly within their own
offices.  We make this recommendation for the following reasons

      o   Because of limited travel funds, the cost of
          conducting extended reviews in the State office for
          all States in a Region, even on a semi-annual
          basis, appears to be prohibitive for most Regions.

      o   Since this is simply a document review,  there is no
          need for access to State staff.   It therefore seems
          more reasonable to ship the documents to the Regional
          staff than to "ship" the Regional staff to the
          documents.

      o   Several Regions have described the paperwork sample
          as a rather "mechanical" kind of review.   If this
          review occurs in the Regional Office, the semi-annual
          visits to the State Office can be devoted to other
          kinds of review that require access either to State
          files or to State staff.

      Semi-Annual Visits to the State Office;   There has been
a tendency for Regions to dispense with formal semi-annual
reviews or visits to the State Office during the first year of

-------
                             -17-
delegation on the grounds that there is already sufficient
review of State activities and sufficient Region/State
communication.  It does, however, seem desirable to establish
the precedent of formal, scheduled semi-annual "reviews"
which can increase in content over time.  During the first
year of delegation, these visits could well be devoted simply
to a formalized general review of problems encountered over
the previous six months and of the adequacy of the State's
response.  This is in contrast to the more informal bi-weekly
or monthly meetings which are, for the most part, devoted
only to the problems-of-the-moment.

PHASE II — TRANSITION TO LESS FREQUENT, MORE FORMALIZED OVERSIGHT

      By the end of Phase I, the Region is relying almost
solely on a checklist review before signing off on incoming
documents.  Considerable mutual trust and confidence should
have developed between State and Regional staff.  The focus
of both the Region and the State is on helping to attain
consistently high quality State review of all delegated
activities, especially basic administrative and engineering
activities.

Formal Reviews Become More Important in Phase II

      Phase II is a period of less frequent communications
between State and Regional staff.  This calls for greater
Regional reliance on formalized oversight mechanisms.  The
quarterly paperwork sample is initially the principal review
mechanism, but throughout Phase II, the semi-annual review at
the State office increases in importance until, by the end of
Phase II, it overshadows the Paperwork Sample in importance.

      This shift occurs because the quality and consistency
of State "activity-level reviews" continues to improve
throughout Phase II.  Early in the Phase, Regional paperwork
sampling is still discovering many deficiencies in State
reviews.  By the end of Phase II, the Region should find so
few deficiencies that it is able to reduce its sample of
paperwork from 10% to 5%,

Regions Can Introduce "Performance Reviews" Here

      At the end of Phase II, the Region might add to the
semi-annual visit a "timeliness review" of current projects
as described in Chapter 1.  By quantifying one aspect of
State performance (timeliness of projects), the Region can
introduce the notion of "direct measurement of performance"
to its oversight methods.  It will thereby begin to augment
its "oversight through review of transactions and through
review of procedures," with "direct measurement of performance."

-------
                             -18-
PHASE III — FOCUS ON IMPROVED PROJECT MANAGEMENT

     Phase III formally begins when the Region acknowledges
that the quality of State performance is so high and so
consistent that the Region requires only a 5% Paperwork
Sample for basic quality control.

      The Region and State both begin to focus on the State's
ability to go beyond compartmentalized activity-level reviews
to conduct coordinated project-level reviews.  In a true
project-level review, each project is viewed not as a collection
of disjointed administrative and engineering activities, but
as a unique, multi-faceted whole.

      Since only a few States have entered Phase III, we
cannot readily draw on State and Regional experience to date
in analyzing possible oversight procedures for this Phase.
Instead, we cover this task in Part II of this paper, which
attempts to determine the desirable future course of the
delegation process and to develop oversight procedures
appropriate for use in those circumstances.  For convenience,
the recommended approaches for conducting oversight in Phase
I and Phase II, developed in this chapter, are summarized in
Chapter 9 along with the approaches we recommend for later
stages of development.

-------
PART II:  ADVANCING  THE STATE OF THE ART

-------
                             -19-

Chapter4;  Shifting fromOversight of Transactions
             and Procedures to Oversight ofPerformance

     In this chapter we will    focus on certain trends that
are apparent in the way Regions are modifying the manner in
which they oversee State operations.

     This chapter (Chapter 4) will be devoted specifically
to an analysis of the k ind of information the Regions use
to conduct such oversight.  In other words, do the Regions
evaluate State operations based on the review of individual
transactions, based on the review of the State's admini-
strative procedures or based on direct measures of the State's
performance?

     In the next chapter (Chapter 5) we will look at the
"scope" of the State's operations and of the Region's over-
sight activities.  By scope, we mean the level of perspective
of both the State and Region.  Do they still focus only on
the quality of individual activities, or have they begun to
focus at a higher level, i.e., on each project as a whole,
on the municipal construction program as a whole or on the
larger water quality program as a whole in order to evaluate
the State's performance?

     Then in Chapter 6, we examine whether direct measures of
performance will continue to be available as the management
scope moves from the activity level to project level and
beyond.
     We will now examine the kinds of information that can
be used to oversee State operations that are still somewhat
narrow in scope, i.e., State operations that still focus for
the most part on "activity management."  We will argue that
Regions should strive for oversight based on direct measure-
ment of performance in preference to oversight of transactions
or of State procedures.

OVERSEEING STATE OPERATIONS AT THE ACTIVITY LEVEL

     In conducting such oversight, the Regions have until now
tended to engage in reviews of:

     o   State transactions.  That is, the Regions have
         examined the quality of the final documents gene-
         rated after State review (e.g.,  How good is the
         environmental assessment?  Are the plans arid
         specifications biddable and constructible?)

-------
                             -20-
     o   State procedures.  That is, the Regions have attempted
         to determine the adequacy of the approach the State
         is taking in carrying out that activity (e.g., does it
         follow the checklist?, are the necessary documents
         filed?, is all the required "boilerplate" included?,
         does the State follow the proper bidding procedures?).

     Only to a very limited extent have- some Regions begun to
develop direct measures of:

     o  State performance.  That is, the Regions have attempted
        to develop and use quantitative measures of the effective-
        ness of the State's operations (e.g., Are projects being
        conducted within budget?  Are projects moving forward
        on a timely basis?).

The Current Shift is towards Review of Procedures and Direct
  Measurement;of Performance'!

     There is already a clear tendency for the Regions to
shift substantially from review of individual State, trans-
actions or documents to review of State procedures.  This
trend was noted earlier in Chapters 1 and 2,  At the begin-
ning of Phase I of delegation, the Region looks closely at
virtually every individual transaction requiring Regional
signoff.  The Regional staff will then react to problems in
individual transactions.  But throughout Phase I and Phase
II, the Region begins to focus more and more on the proce-
dures the State staff follows in carrying on each activity.
As the Regional staff become convinced of the soundness of
the State's procedures as reflected in the quality of
individual transactions, their attention shifts primarily
to the consistency with which the procedures are followed.
As a consequence, while the Regional staff conducted an
in-depth review of virtually 100% of current transactions at
the beginning of Phase I; by the beginning of Phase II,  a
mere checklist review of each current transaction is con-
sidered adequate.  At the beginning of Phase III, the Regional
staff, feel comfortable giving only 5% of these transactions
a closer look.

     This tendency to shift substantially from review of
individual transactions to review of general procedures is
desirable since it requires less Regional staff time.
Furthermore,  it does not increase the  risk of a deterioration
in State performance because even a 5% paperwork sample of
recent transactions gives the Region enough quality control
information to detect any areas of deteriorating quality.

-------
                             -21-
     At the same time that the Regions have been shifting
from reviews of transactions to reviews of procedures, a
second shift has begun to occur as well.  One or two Regions
have begun to experiment on a very limited basis with the
use of direct measures of performance for overseeing State
operations.  For example, the "cost review" carried out by
Region VI seeks to determine the adequacy of the State's
cost related activities not by looking at the apparent
adequacy of individual transactions or of general procedures»
but by looking at the "bottom line."  Region VI looks to see
how the actual project cost compares with the original project
budget.  If a project is going significantly over budget,
this is clear evidence that State procedures or handling of
individual transactions for cost-related activities are
inadequate.  Indeed, such actual cost performance information
is the best and most meaningful evidence possible of how well
the State is carrying out its financial responsibilities.
While the quality of individual transactions or of general
procedures gives an indication of the probable quality of
the State's performance, direct measurement of that perfor-
mance tells the Region about the actual quality of that
performance.

DIRECT MEASURES OF PERFORMANCE AND HOW TO USE THEM

     In many cases, reviews of State transactions (i.e., the
quality of the documents generated) or of State procedures
are virtually the only tools available for assessing State
performance.  In other cases, however, more direct,  often
quantitative measures of State performance are available.
Here are two examples:

     Change Orders;

     o   The Region could detect most unwarranted use of
         change orders by monitoring the percent increase
         in project cost due to change orders.

     Minority Business Enterprise:

     o   The Region could determine the ultimate effectiveness
         of the State's MBE-related activities by monitoring
         the fraction of project dollars awarded to minority
         business enterprises.

     For activities for which direct measures of performance
can be developed, it is possible to make use of a streamlined
approach to oversight.  Briefly, the streamlined approach
consists of monitoring the State's conduct of that activity
by gathering the direct performance data (e.g., the actual
cost increases due to change orders).  If that performance
data indicated that there is no problem, then no further

-------
                             -22.-

review is necessary (other than a review of 5% of recent
change orders in the Paperwork Sample to ensure adequate
quality control).  If, however, the performance data indi-
cates that there is a problem, then there should be follow-up
assessment of the State's procedures for that activity (e.g.,
of the State's change order review procedures).

     Since performance data is quantitative, the adequacy
or inadequacy of the State's performance must be assessed
initially against pre-specified expectations or "performance
targets."  These performance targets should be agreed on
jointly by the State and the Regional Office and should
reflect realistic expectations as to what is "do-able" in
that State.  The MBE performance targets,        for example,
might be set much lower in New Hampshire,  where there is a
very small minority population, than in California, where the
minority population is much larger.

     Here, step-by-step, is how an oversight system based on
direct measurement of performance might work:

     o  Agree on Activities to be Subject to Direct Monitoring
          of Performance

        -  First, the State and Region agree which State
           activities will be subject to direct measures of
           performance during the coming year.  They also
           agree on the specific measures  of performance to
           be used.

     o  Agree on the Appropriate Performance Targets

        -  The State and Region then negotiate performance
           targets for each of these activities for the coming
           year.

        -  Such targets can be revised annually based in part
           on experience gained in previous years as to what
           target levels are feasible and appropriate.

     o  Oversee these activities on the basis of measured
          performance

        -  During the year, the Region samples projects
           underway to determine actual performance.

           If actual performance falls within the agreed-
           upon norms,  then the State's performance is
           satisfactory and no further review is necessary.

-------
                             -23-

           If actual performance falls outside the agreed-
           upon norms, then the Region should examine the
           State's review procedures to determine the probable
           reasons for the deficiencies and should negotiate
           changes in those procedures believed likely to
           improve performance.

There isGreat Opportunity for Regionaland State Initiative
  in Developing and Testing Additional Direct Measures Q.f
  Performance

     While it is clear how performance norms can be applied
to such factors as degree of MBE participation and extent of
cost overruns due to change orders, innovative Regions might
also seek ways to specify performance for other aspects of
the program.  For example, some Regions and States may wish
to experiment with the use of additional performance measures
as such as:

     o  Reasonableness of Contract Cost.
        For each class of facility to be built using
        off-the-shelf technology, the State and Region
        might agree on normal cost ranges and average
        cost.

        The Region would then periodically review contracts
        awarded to determine:

        -   what percentage of each class of contract
            falls outside the specified range,

            the extent of deviation of the actual average
            cost from the expected average cost.

        If the performance measured in this way appears
        satisfactory, there will be no further review.
        If the actual performance seems to reflect un-
        expectedly high costs, the Region would conduct
        an in-depth review of the State's cost estimation,
        bidding and bid selection procedures to determine
        whether these procedures are contributing to the
        the unexpectedly high costs.

     Thus, the Region would again be applying the principle
that if the State's measured performance meets agreed upon
targets, then no detailed review of procedures for that
activity is necessary.

     Many other applications of this approach are possible,
and the Regions and States should be encouraged to develop
and use appropriate performance measures in place of con-
ventional transactions or procedural reviews for as many
aspects of the program as possible.

-------
                             -24-

     It should be emphasized that performance measures are
suitable not only for monitoring State performance at the
"activity-review" level but also at the "project-review"
level and potentially at the "program management" level as
well.  Possible measures of performance at these levels are
presented in Chapter 6 and Appendix B.

THE POTENTIAL BENEFITS TO BE DERIVED FROM USE OF DIRECT
  MEASURES OF PERFORMANCE

     For the two specific activities cited earlier as lending
themselves to direct measures of performance, the following
impact can be seen.

MBE.   Monitoring the percent of contract dollars awarded to
MBE firms would substitute for detailed reviews of the MBE
aspects of bid solicitation and contract awarding procedures.

Change orders.   Monitoring cost overruns due to change orders
as a percentage of total project dollars would substitute for
detailed scrutiny of the State's change order review procedures
(procedure review) not to mention detailed Regional review of
each proposed change .order (transaction review.)

     Such changes in the manner in which Regional oversight
is conducted would bring the following benefits:

     o  Reduction in RegionalReviewTime.  Regional over-
        sight based on direct, monitoring of performance
        would in virtually every case be less time consuming
        th2Ji Regional review of individual transactions and
        would, in many cases, be less time consuming than
        Regional review of State procedures.

     o  Increase in State Flexibility.  Use of direct measures
        of performance would potentially allow States to modify
        their review procedures as they see fit as long as the
        agreed-upon performance targets are met.  Thus, it
        would allow for the gradual evolution of the program
        from one of "uniform" State programs that are required
        to conform to pre-determined procedures to one of
        ."equivalent" State programs in which State procedures
        are allowed to vary as long as the results meet reason-
        able expectations.

     o  More Meaningful Indicators of Impact.  States are now
        required to perform certain specified procedures not
        for the sake of the procedure itself or of procedural
        uniformity, but because EPA believes that use of those
        procedures will tend to produce a certain outcome, for
        example:

-------
                             -25-
        -  MBE;  a reasonable level of MBE participation in
           current projects

        -  Change Orders;  prevention of abuse of change
           orders by contractors to inflate project costs or
           to reduce project quality.

        EPA can best ensure that these outcomes are being
        achieved by measuring the outcomes themselves (i.e.,
        amount of MBE participation, degree of overrun due
        to change orders) rather than by verifying that
        specified procedures have been followed.  In many
        cases, in fact, it may turn out that a State may
        follow the specified MBE or change order proce-
        dures and still have inadequate MBE participation
        or excessive change-order related cost increases.
        The monitoring of procedures entails monitoring
        something one step away from EPA's real concern;
        the monitoring of performance entails monitoring
        the thing of concern itself.

     As we have demonstrated, the impact on State/Regional
relations of adopting and using direct measures of performance
will be substantial.  Indeed, the implications are so great
and so far-reaching that it can reasonably be said that a
Region that relies primarily on direct measures of performance
has shifted from the role of "supervisor" to "franchiser" or
"indirect manager."
     In this chapter, we have endorsed the current shift in
the Regions from oversight based on review of individual
transactions to oversight based on review of State procedures.
We have also taken note of some initial efforts by the Regions
to develop and use direct measures of performance for over-
seeing State operations.  We have given strong encouragement
to these efforts and have urged that additional direct measures
of performance be adopted as the primary tools for overseeing
State operations at the activity level, thereby facilitating
the shift of the Regional Office from the role of a "supervisor"
to that of a "franchiser."

-------
                             -26-

Chapter 5:  Increasing the"Scope" of Regional Oversight _--
             from Activity Management to Project, and
           •  Program Management
THE MANAGEMENT SCOPE IN A STATE PROGRAM

     In the earlier descriptions, in Chapter 2, of the phases
through Which a State program develops, three distinct kinds
of changes over time can be discerned.   The most obvious
changes are improvements in quality.  At the end of Phase II,
for example, a State is simply doing a better job of reviewing
grant applications, plans and specifications, etc.  than it
was at the beginning of that Phase.

     A second change, highlighted in the preceding chapter, is
the change in the kind of information used to oversee   State
programs.  During the course of Phase I and II, there is a
pronounced shift from oversight based on the review of
individual transactions to oversight based on the review of
procedures.  We are also beginning to see a second, even-more-
desirable shift to the use of direct measures of performance
as a major oversight tool.

     At the same time a third and more subtle kind of change
is occurring.  In Phases I and II, the State has its hands
full trying to ensure that it is adequately carrying out
specific reviews at the activity-level (e.g., reviews of
environmental assessments, reviews of MBE provisions, reviews
of plans and specifications).  When a State enters Phase III,
however, it is relatively confident of its ability to conduct
reviews at the activity-level, and can begin to increase the
"scope" of its reviews by focusing on conducting adequately
coordinated reviews at the project-level.  We expect to see
further shifts in time from project-level management to
program-level management and finally to integrated water
quality management.

Anticipated Shifts in Management Scope

     When a State enters Phase III, it begins a pronounced
shift from mere activity management to true project manage-
ment.  In Phases I and II, the State tended to view each
project as a disjointed set of activities to be reviewed.
In Phase III, it begins to look beyond those activities
and to see the project as a whole.  While in Phase III the
State must necessarily continue to conduct activity-level
reviews, it now strives hard to focus on each project as a
single, though complex, entity.  And in doing so, it begins
to ask broader questions:

-------
                             -27-


     o   is the project as a whole technically sound?

     o   is the project as a whole environmentally beneficial?

     o   is it being built in a timely manner?

     o   is it being built at a reasonable cost?

     There are several ways a State might go about improving
its effectiveness in performing "project-level reviews."
These are discussed more fully in the next section and in
Appendix A.

     Once a State has succeeded in achieving true project-
level review, it might be said to be leaving Phase III and
entering a new Phase — "Phase IV."

     In Phase IV we might expect to see a similar change in
the level of review.  By the beginning of Phase IV, a State
should be reasonably proficient in conducting effective
project-level reviews.  It can therefore begin to focus on
better coordination among projects in order to improve its
overall management of the construction grants program.
Improved municipal wastewater program management might, for
example, include more thorough consideration of potential
water quality benefits when ranking municipal wastewater
projects on the priority list.  It might also include
efforts to identify communities with significant water
quality problems where facilities are greatly needed but
that have not yet applied for grants.  The State could then
work with such communities and assist them, if necessary, in
preparing grant applications.  By "beating the bushes" in
this way, the State can help ensure that enough projects are
proposed that it can pick those with the most merit to go on
the priority list.*  It is now often the case that a State,
in seeking to avoid re-allotment of its grant funds, will
include projects of less merit on its priority list simply
because they are the only additional ones available that
are "ready to go."

     Similarly, we might expect some States,  after having
achieved effective management of the construction grants
program as described above, to advance to a new "Phase V" in
which it begins to focus more on "integrated water quality
management."  At this point, the State program would closely
coordinate its municipal construction efforts in specific
communities with other pollution control efforts there.  At
* As noted below, this strategy is desirable even after a
  "date certain" several years in the future has been set
  for terminating the award of additional construction grants

-------
this point, strong efforts might,  for instance, be made to
ensure that the scheduling of construction grants projects
in each locality is fully consistent with the scheduling of
industrial source and non-point source control efforts there.
Furthermore, the State might begin to take major steps towards
careful coordination with any groundwater related activities
planned for that area.  At this point a State program will
have achieved the ultimate in meaningful, effective program
management.
     Having now described briefly the four distinct levels at
which a State program can operate, we will now explore in more
detail what is involved in conducting each level of review.
We will begin our discussion with Phase III and project-level
reviews.

THE PHASE III PROBLEM — PROMOTING PRQJECT-LEVEL MANAGEMENT

     In Phase III we are dealing with a State program that
has begun to demonstrate its proficiency in carrying out
the mechanics of performing each delegated activity.
Although the Region should continue to monitor the State
performance in reviewing each activity in order to ensure
that the quality remains consistently high, the Region does
not, at this point, really expect many serious problems to
be discovered.

     The Region can therefore begin to focus on the quality
of the State program's operations as viewed from a broader
perspective — project-level management.  Specifically, the
Region can begin to ask:  how well is the State doing in
ensuring that each project as a whole is technically sound,
environmentally beneficial and is built in a timely manner
at reasonable cost?

     One criticism of current activity-oriented reviews heard
often in discussions with Regional staff is that the States
are called upon to conduct too many separate reviews,  each of
which is conducted independently without a clear understanding
of the role it plays in promoting the ultimate goal of sound,
efficient wastewater treatment.  It was with this concern in
mind that Region IV developed the concept for the Project-Level
Management Review it recently conducted in Georgia.

     Region IV 's procedure was to conduct an intensive review
of the whole range of State activities for a single project in
order to highlight how each activity contributed or failed to
contribute to the success of the project as a whole.  The
review was designed also to uncover gaps or deficiencies in

-------
                             -29-

the State's review process that allowed problems to arise
that detracted from the project's quality or timeliness or
that added to its cost.

     The intended result of such a review by the Region is
improved coordination between different aspects of a State's
activities and the elimination of gaps or deficiencies among
such activities so that the State will be better able
to assess a project as a whole.

     Ideally, the State should propose specific changes in its
review procedures or internal coordination to minimize the
likelihood that such deficiencies will recur.  In some cases,
the changes might alter the manner in which the State, the
grantees, or the consultants carry out their responsibilities.
For example, it might mean taking steps to ensure better
coordination between two or more separate staffs in the State
Office.  In other cases, it might be determined that there
should be a consolidation of several staff groups to ensure
greater integration of closely related activities.  In pro-
moting effective project-level management, the State and
Region should probably focus on the integration of certain
key activities in Step 1 that have a strong impact on basic
project design.  These key activities are the "analyses of
alternatives" associated with the alternative/innovative
design review, the environmental assessment activities,
initial financial planning efforts conducted in the early
stages of the the facility planning process and any other
analysis of alternatives in the facility plan.*

     Once the available options have been fully explored,
the State and the Region should reach agreement on the
corrective actions to be taken.  This will often necessitate
changes in the corresponding checklists.  In its next program
review (6 to 12 months later),  the Region would then look to
see how effective these changes have been in eliminating the
deficiencies seen earlier.   If necessary, further changes in
State procedures should be negotiated in this and other
subsequent reviews.

     The kinds of corrective action that a State will normally
propose to improve coordination among closely related activities
are the following:
  Indeed, there is a growing consensus both in the Regions
  and at headquarters that such "up-front" planning activities
  in Step 1 are the most crucial single element in ensuring
  that each municipal project is both technically and
  environmentally sound and is cost effective.

-------
                             -30-

     o  Require multiple, concurrent reviews of certain
        documents "by different staffs with related functions.

     o  Institute "project review" meetings, with partici-
        pation by each staff with distinct responsibilities.

     o  Create new "integrated review groups."

The relative advantages and disadvantages of these three
approaches for improving the degree of project-level review
in a State program are discussed at some length in Appendix A.

     As noted above,  there is a growing consensus that the
facility planning activities are the key to ensuring techni-
cally and environmentally sound, cost-effective projects.
Those activities leading to a sound ultimate decision as to
what kind of facility to build are therefore the "Tcey
activities" for ensuring good project-1eye1 management.  The
key activities are, of course, the activities that help
identify alternative approaches for controlling municipal
wastewater and help define the advantages and disadvantages
of each alternative.   The key activities for project-level
management and for each subsequent higher level or "scope"
of management are summarized in Table 3 at the end of this
Chapter   and are discussed again briefly in Chapter 7.

THE PHASE IV PROBLEM — PROMOTING IMPROVED PROGRAM MANAGEMENT

     Since in Phase III the Region and State will be empha-
sizing improved project-level review, by the end of Phase III,
the State will have developed a considerable degree of
proficiency in ensuring the soundness of each project as a
whole.  Therefore, in Phase IV, the Region and State can then
begin to focus on improvement of the State's management of the
municipal wastewater program as a whole.

     As was noted earlier, improved program management would
result in improved priority-setting and coordination among
projects.  Specifically,  the Region might, in Phase IV, begin
to give considerable attention to the State's performance
with regard to the relative priority assigned to municipal
wastewater projects in light of anticipated water quality
benefits.

     The Region might also examine what steps the State is
taking to determine what additional communities are in need
of improved treatment facilities and to assist them in pre-
paring an appropriate grant application.  As was noted earlier,
such action on the part of the State would increase the
number of potential projects with significant water quality
benefits.  It would thereby reduce the need of the State to

-------
                              -31-

 include projects of relatively less merit on its priority
 list in a given year simply because they are "ready-to-go."*

      Activities such as these would consitute the "key
 activities" for ensuring adequate program management.  By
 focusing on the State's efforts to upgrade its performance
 in such areas, the Region can help promote State program
 management that is focused on maximizing the water quality
 benefits derived from the State's construction grants program
 as a whole.
 PHASE V — PROMOTING INTEGRATED WATER QUALITY MANAGEMENT

      Many Regions may be satisfied with a State program
 that has met the Phase IV goal of having established a
 balanced, water-quality-oriented construction grants program.
 And they would have good reason to be satisfied since it is
 likely that because of staffing limitations or for other
 reasons, some States may never reach this goal.

      Nevertheless, we should anticipate that some Regions and
 States may wish to go one step further, to enter a '"Phase V"
 in which the objective would be to integrate construction
 grants activities fully into the State's broader program of
 water quality management.'  At this point, relative project
 priorities would be set not just on the basis of the relative
 merits of the projects themselves but on the basis of how
 they fit into the larger context of water quality management
 efforts as a whole.  The State program's scheduling of
 projects and guidance on facility planning would now fully
 consider the nature and extent of other activities underway
 or projected, including industrial and non-point source control
 efforts and groundwater control needs.  This would mean the
 full incorporation of municipal facility planning and
 construction into a broader framework of meaningful, ongoing
 areawide planning and State-wide planning.**  It would reflect
 the full maturation of the construction grants program as
 a key element in balanced environmental planning at the State
 and National level.
 * Even if a date certain is adopted for ending the awarding
   of Federal construction grants, encouragement of additional
   Step 1 grants where really needed is still desirable.  This
   will help ensure that the limited funds remaining to be
   awarded can and will be spent where they will do the most
   good in terms of improving water quality.

** Clearly the "key activities" for achieving water quality
   management are areawide planning activities at the local
   and State level.  (See Table 3.)

-------
                             -32-
PHASE VI — A FULLY-DEVELOPED STATE PROGRAM

     In the previous phase — Phase V — a State would begin
to strive for incorporation of its municipal wastewater
program into a integrated water quality management.  Once that
goal is achieved, we can consider the State to have entered
the final phase of development.  States that reach this
final phase — Phase VI — will have achieved the ultimate
in balanced, fully water quality oriented program management.
At this point, the State and Region should jointly work for
greater efficiency in State operations and in Regional over-
sight procedures as long this does not jeopardize the quality
or scope of the State's operations.
     We believe that the four increasing levels of "management
scope" described in this chapter provide a useful framework
within which to assess State programs and help promote their
further development in order to ensure the most effective
possible management of the municipal wastewater program.

-------
          Proposed Classification of State Activities
BASIC ENGINEERING AND ADMINISTRATIVE ACTIVITIES

     o   Review of Change Orders
     o   Review of MBE Activities
     o   Review of User Charge and Industrial Cost Recovery
     o   Review of Plans and Specifications
     o   etc. —  includes all other State activities not
         included in the list of "Key Activities" below
"KEY ACTIVITIES" (See Chapter 5 for discussion)


1 .   Key activities for effective project.-level management

     o   Review the grantee's "analysis of alternatives"

         -  Identification and technical analysis of possible
            alternative and innovative designs

            Identification and technical analysis of other
            design alternatives in the facility plan

            Assessment of the environmental impacts of each
            viable design alternative

         -  Identification of the costs and possible financing
            arrangements for each viable alternative
     Key activities for effective program-level management

     o   Needs - related and priority - setting activities:

         -  Complete Statewide assessment of municipal treatment
            needs

         -  Incentives for communities with significant needs
            to apply for Step 1 Grants

         -  Assistance to communities with significant needs
            in preparing Step 1 applications

            Priority list preparation with priorities based on
            (true) relative need in terms of water quality

-------
Table 3:  (cont'd)
3 .    Key activities for effective integrated water quality
       management

     o   Meaningful, comprehensive areawide water quality
         planning:

            at the  local level
            at the  state level

     o   Joint scheduling of all water quality improvement
         efforts in each locality including:

            municipal control efforts
         -  industrial point-source control efforts
            non-point source control efforts
            groundwater protection efforts

-------
                             -33-

Chapter J5j_j4onitor ing "Performance" at the Project
             and Program Level

     In Chapter 4 we reached the following conclusion about the
type of information that should be used in overseeing State
operations at the activity-level:  whenever it is possible
to do so, direct monitoring of performance is preferrable to
reviewing procedures or reviewing individual transactions.

     Then, in Chapter 5, we concluded that the Regions and
States should continue to work for an ever broader management
scope in State operations.  The States, we concluded, should
move from an activity-level focus to a project-level focus, and
then to a program-level focus.

     The question we will now address briefly is whether it
is possible to continue making use of direct measures of
performance as we move from activity-level oversight to
project-level and program-level oversight.
PROJECT-LEVEL MEASURES OF PERFORMANCE

     In Chapter 4, we presented two direct measures of perfor-
mance that operate at the activity-level.  One measures the
State's performance in ensuring MBE participation in projects;
the other measures the State's performance in ensuring that
change orders are not used to inflate project costs.

     But as a State enters Phase III and begins to focus on
"projects" rather than "activities," is oversight based on
the direct measurement of performance still possible?  In
other words, is it possible to identify project-level measures
of performance?  The answer, quite simply, is yes.  In fact,
the performance measures already developed by Regions IV and
VI are really project-level measures.  Here are slightly
refined variants of the performance measures developed by
these two Regions:

     o   Timeliness.  Monitoring total time to completion
         after approval of the facility plan for projects
         of a conventional nature would give one indication
         of the State's performance in expediting the
         completion of current projects.  (This is a slight
         variation on the performance measure Region IV
         intends to use in its "Progress Reviews.")

     o   Cost.  Monitoring project cost (both cost at award
         and cost upon final completion) against expected
         cost would give an indication of the State's overall
         effectiveness in applying bid and award requirements.
         (This is a slight variation on the performance measure
         Region VI uses in its "Cost Reviews.")

-------
                             -34-

     Again individual Regions and States should be encouraged
to develop and test other project-level measures of perfor-
mance.  Innovative measures of performance that are shown to
be both workable and useful can then be adopted by other
Regions as well.
PROGRAM-LEVEL MEASURES OF PERFORMANCE

     It is clearly more difficult to develop direct measures
of performance that would work at a level higher than project-
level.  Nevertheless, the prospects for developing program
level measures of performance are not hopeless.  On the
contrary, performance measures can be envisioned that would
help monitor the effectiveness of the State's program manage-
ment as a whole.

     The ultimate performance measures of this kind would
quantify the impact of the State program in improving water
quality.  Such measures can be expected to be rather diffi-
cult to develop and apply.  Furthermore, they are not suitable
for routine oversight of the State's program because- of the
extended time frames necessary for water quality impact to
be seen.  Nevertheless, some Regions may wish to experiment
by attempting to use them for periodic assessment of the
State program's overall performance.  Once successfully
demonstrated, they would then be available for use by other
States and Regions.  One possible approach that some Regions
and States may be interested in exploring at some point in
the future is presented in Appendix B.
     In this chapter and in Appendix B we have shown that
direct measures of performance are not only available for
oversight at the activity-level, they are also available for
project-level oversight and are potentially available for
program-level oversight as well.

-------
                             -35-

Chapter 7;  Proposed Oversight Procedures for
         ~~"   Phases III, IV, V and VI
     In Chapter 3, we recommended specific oversight proce-
dures for use in Phases I and II based on an analysis of
procedures currently in use and Regional experience to date
in applying them.

     In this chapter, we will recommend oversight procedures
for use in Phases III, IV, V and VI.   Since there is little
Regional experience to date in overseeing State programs that
have reached this stage of development, we will have to draw
not on Regional experience but rather on the analysis in
Chapters 4, 5 and 6 in making our recommendations.

PHASE III — STRIVING FOR PROJECT-LEVEL MANAGEMENT

     A State enters Phase III when, in the judgment of the
Regional Office, State operations in carrying out delegated
activities are of such high quality and consistency that a
10% sample of recent transactions is no longer necessary.
The Region now believes that a 5% sample will be adequate
to ensure good quality control.

     The focus on both the Region and the State can therefore
shift to project-level management.  The State strives to
ensure that activity-level reviews come together in such a
way that the State has a fully comprehensive project-level
review of the quality, environmental soundness and cost-
effectiveness of each project.  To achieve this objective,
the State focuses on those activities that are essential to.
effective project-level management.  These "key activities" are
the analyses of alternatives in Step 1, which should result
in the identification of all feasible design alternatives and
their analysis to determine the single most cost-effective
and environmentally sound alternative.   These analyses include:

     o  the identification and technical analysis of possible
        alternative and innovative designs,

     o  the identification and technical analysis of other
        design alternatives in the facility plan,

     o  the environmental assessment — that is, the analysis
        of the relative environmental effects of each viable
        alternative

     o  the financial plan — which should contain both an
        analysis of the cost of each viable alternative and
        an analysis of the possible methods of financing for
        each alternative.

-------
                            -36-

     Once these analyses have been completed in a thorough
manner, the grantee is then in a position to select the
"best" of the available alternatives, subject to State
approval.  Thus, if these analyses have been conducted
properly, the decision at the end of Step 1 will be the
selection of the best possible project design.  All that
is then required in Steps 2 and 3 is to ensure that the
basic decisions made in Step 1 are faithfully carried
out and that they are carried out in a cost-effective and
timely manner.

     In addition to encouraging the State to strive for true
project management by focusing on these key activities, the
Region should, in Phase III, also begin to introduce direct
measures of performance for use in overseeing the State's
operations.  These would begin to substitute for oversight
based on review of individual State transactions and over-
sight based on review of State procedures.  Nevertheless,
since the Region is now introducing direct measures of
performance for the first time, it will have to continue
to rely primarily on reviews of individual transactions
and of State procedures in overseeing the State program.
PHASE IV—STRIVING FOR PROGRAM-LEVEL MANAGEMENT

     Once a State has achieved a reasonable level of proficiency
in project-level management, it can then begin to strive for
effective management of the municipal wastewater program as a
whole.  This objective can be said to have been achieved when the
State achieves a balanced multi-year schedule with the highest
priority projects in terms of water quality needs receiving the
highest priority in terms of funding and scheduling.  The key
activities for achieving these objectives relate primarily to
pre-Step 1 actions:

     o   Needs identification

     o   Providing incentives to communities to take needed
         control measures

     o   Assisting communities in need of facilities to apply
         for Step 1 grants

     o   Water-quality oriented priority setting by the State
         when preparing the State's Priority List.

-------
                            -37-

     Carefully carrying out these key activities will ensure a
balanced plan for construction of needed municipal facilities.*

     In Phase IV, the Region should make increased  use of
direct measures of performance.   As it does so, it can de-
emphasize oversight based on the review of State procedures.
When a direct measure of performance is adopted, review of
procedures can be reserved as a back up approach for use only
when performance, as measured directly, is found to be deficient,
At the same time, the Region should continue to de-emphasize
oversight through review of individual transactions until,
by the end of Phase IV, the Region agrees that 2% sample of
transactions (rather than the 5% sample used in Phases III
and IV) will be adequate for quality control purposes.

     A Region that works vigorously to substitute oversight
based on measures of performance for oversight based on
review of individual transactions on of State procedures
can reasonably be said to have made the transition from the
role of a "supervisor" to that of a "franchiser" or indirect
manager by the end of Phase IV .

PHASE V — STRIVING FOR INTEGRATED WATER QUALITY MANAGEMENT

     Phase V begins when the State has achieved consistent,
well-coordinated program management of the municipal waste-
water program.  The State and Region can then begin to
focus on the full integration of municipal wastewater control
efforts into a broader program of integrated water quality
management.  This objective can be achieved by going beyond
careful priority setting and scheduling of municipal wastewater
projects to careful priority setting and joint scheduling of
all water quality management efforts.  Thus,  priorities would
now be set after looking at all pollution control needs in
each locality:  industrial point source control needs, non-
point source control needs and groundwater protection needs
     *Note that even if it is agreed to cut off Construction
grants at some date certain in the future (e.g., in 1990),
it is still important for the States to focus on all of these
key activities including encouraging new Step 1 applications.
Indeed, if a cut off date is set, these activities become even
more important since if only limited funds remain available,
it is crucial that they be spent on the most important projects
that are needed, and are not just spent on the most important
project that are "ready to go."  In other words, there may be
some greatly needed projects that are not on the priority list.
It is the most needed projects that should be funded and the
States should therefore continue to promote new Step 1 appli-
cations for highly needed projects that have not yet entered
the funding pipeline.

-------
                            -38-

as well as municipal control needs.  Furthermore, the
scheduling of municipal projects would also consider and be
consistent with the scheduling of control actions for the
other classes of water quality problems.

     It is clear that the "key activities" for ensuring
effective integrated water quality mangement are comprehensive
water quality assessment and planning activities.  In
Phase V, the State and Region should therefore upgrade and
more greatly emphasize areawide planning activities at
both the local and State level.

     At the same time, the Region should continue to develop
and use additional direct measures of performance.  In doing
so, it can continue to de-emphasize oversight through review
of procedures and oversight through review of individual
transactions.

PHASE VI — OPERATING IN A FULLY-DEVELOPED,
    FULLY-DELEGATED MODE

     Once the State achieves full integration of its municipal
wastewater program into its broader program of integrated
water quality management, it will have reached the ultimate
in "management scope" for construction grants activities.  The
principal objective of the Region and the State should then
be to maintain this level of operation.  A secondary objective
should be to achieve greater efficiencies in program operations
(at the State level) and in oversight methods (at the Regional
level) without jeopardizing the scope or quality of State
operations.   Some increased efficiency in Regional operations
can continue to be made by introducing new direct measures of
performance wherever possible.

     Once the State enters Phase VI, the Regional role has begun
to shift from that of "franchiser" or indirect manager to that
of "regulator" or environmental manager.
     This completes our analysis and recommendations with
regard to the improvement over time in the scope of State
management and Regional oversight and in the kinds of over-
sight mechanisms used.  These recommendations are summarized
briefly in the beginning of the next chapter (Chapter 8) and
are summarized Phase-by-Phase in more detail in Chapfer 9.

-------
PART III:  TYING IT ALL TOGETHER

-------
                            -39-

Chapter 8;  The Proposed Role for the Regions
~             and of EPA Headquarters


     We will now briefly discuss the steps that the Regions
and EPA's Office of Water Program Operations at Headquarters
might take to speed both the development of State programs
and the transformation of State/EPA relations to those
appropriate for a fully delegated construction grants program.

THE REGIONS WILL CONTINUE TO PLAY THE KEY ROLE

     Since we have already addressed in considerable detail
the actions that the Regions can take, we will merely summarize
them here:

     The Regions should continue to ensure that each State is
carrying out its delegated functions with reasonable proficiency
and consistency.  The amount of oversight necessary to achieve
this objective will decrease over time.  The Regions should,
therefore, regularly reassess the State program's stage of
development to ensure that the oversight procedures in effect
for a given State are still consistent with current conditions.
As the performance of a State improves, the closeness of the
Region's scrutiny should be relaxed accordingly.  In reviewing
State performance, the Regions should continue to use reasonable
standards.  The benchmark should not be "zero errors" or "no
decisions that the Regional staff disagree with" since such a
standard can never be met.  After all, mistakes are unavoidable,
and no two individuals will ever agree on every judgment call
that has to be made.  Instead, the benchmark should be perfor-
mance of reasonable quality and consistency and decisions
based on a sound, .thoughtful, balanced analysis of the
available alternatives.

     The Regions should continue to encourage each State to
enhance the "scope" at which the State is carrying out its
responsibilities.  Once a State becomes reasonably proficient
at carrying out "activity-level reviews," the Region should
begin to focus on how it can restructure its oversight proce-
dures to encourage the State to begin conducting "integrated
project-level reviews."  Once the State is reasonably proficient
at project-level reviews, the Region should adjust its over-
sight procedures to encourage the States to achieve full program
management of the municipal wastewater program, and finally,
integrated water quality managment.

     The Regions should, wherever possible, replace oversight
procedures that rely on reviewing State procedures or on
reviewing the quality of individual transactions (i.e.,  of
documentssubmitted to the Region after State review) with
oversight based on the use of direct measures of performance.

-------
                            -40-

In conjunction with the use of quantitative measures of
performance, the Regions should begin to negotiate with
the State to arrive at performance targets against which
the State's actual performance can be measured.  Whenever
performance targets are met, detailed scrutiny of the
corresponding State review procedures and of the related
project documents* can be dispensed with.  In this way,
the Region can further reduce the extent of EPA's direct
"hands-on" involvement in State operations, without
sacrificing program quality.

     The implications for State/Regional relation of con-
ducting oversight using direct measures of performance in
place of oversight based on review of individual transactions
or reviews of State procedures are far-reaching.  Indeed,
if a Region moves vigorously to adopt direct measures of
performance during Phases III and IV, it can reasonably
claim to have shifted from the role of a "supervisor" to
that of a "franchiser" by the end of Phase IV.  When the.
State reaches Phase VI, the Region will have begun to approach
the role of "regulator" or environmental manager.
    .The three paragraphs above briefly but rather fully
characterize the steps each Region can take to promote the
full development of the construction grants program operating
in a delegated mode.
EPA HEADQUARTERS ALSO PLAYS A CRUCIAL ROLE

     Until now, the Office of Water Program Operations (OWPO)
has chosen to play a fairly limited role in overseeing the
delegation of program responsibilities to the States and
overseeing the operation of the program in its delegated
mode.  OWPO's primary actions to date have included issuing
some general advice to the Regions on how to oversee program
activities once delegated, and establishing a small staff to
monitor and coordinate delegation activities nationally.

     There are several other key areas in which OWPO could
profitably play a strong role thereby expediting the successful
development of delegated State programs:

     OWPO shouldprovide formal guidance to the Regions on
how to utilize specific oversight methods.  For Phases I and
II, there is already sufficient Regional experience that
OWPO can now prepare detailed guidance on "how to do it."
For Phase III and beyond, the guidance must necessarily be
considerably less detailed; nevertheless, it is still possible
  Other than a small sample review for quality control purposes.

-------
                            -41-

to prepare reasonably specific guidance.  As some Regions gain
more experience operating in Phase III and later phases,
this guidance can be revised periodically to be more speci-
fic and, as necessary, modified to reflect experience gained
regarding what works and what doesn't.  OWPO should neverthe-
less remain permissive in its attitude toward the guidance
that has been prepared.  Regions should be allowed to depart
from the guidance in order to experiment with alternative
oversight methods (especially for the later Phases).  The
results of such "experiments," when successful, can then be
reflected in subsequent revisions of the guidance.

     OWPO should review Regional determinations as to each
State's stage of development.  In this way, OWPO can help
ensure reasonable national consistency in assessing where
each State stands in terms of "phase" of development to
ensure that:

     o   a Region is not holding a State back by underrating
         its degree of development,

     o   a Region is not establishing overambitious goals
         for a State as a result of overrating its degree
         of development.

     OWPO should review direct measures of performance adopted
by States and Regionsand should carefully reviewthe nego-
tiated performance targets to ensure reasonable national
consistency where appropriate.  Considerable latitude should
be allowed at first in the definition of performance measures
to allow needed experimentation and testing of alternative
approaches.  But as certain specific performance measures
are demonstrated to be preferable to others,  less latitude
should be allowed.  Similarly, for performance targets, only
reasonable variations from State to State should be allowed
once there is enough experience to determine what levels are
reasonably achievable.  For some aspects of performance,
considerable State-to-State variation in performance targets
may be desirable.   For example,  New Hampshire,  with a very
small minority population, might appropriately have a much
lower MBE performance target than California, which has a
relatively large minority population.

     OWPO should prepare detailed case studies of States
that have success f u 13.y^ ajchi e ved pr o j e ct-1 e ve 1 management,
program management or integrated water quality management.
This case study information would than be available
to provide guidance on how to increase the"scope"
of State operations and of Regional oversight methods
elsewhere.

-------
                            -42-

     OWPOshould provide guidance to the Regions and States
on staffing patterns and staffing levels.  More than one
Regional program manager has remarked that program operation
in a delegated mode calls for a new mix of expertise in both
the Regions and States.   One in particular asserted that
whereas States tend to be presently staffed with 80% engi-
neers and 20% administrative personnel, the ideal mix for
full delegation should be more like 50% of each.

     Other Regional managers have made similar remarks about
the need to change the skill mix in the Regional Office.
(Some seem to feel that few engineers are needed in a Regional
Office once operating in a fully delegated mode.)  OWPO might
well analyze the situation and provide guidance on "target
staffing patterns."  Such guidance might also indicate the
number of staff needed in the State Office to conduct various
delegable activities at various levels of effort.  This is
merely recommended staffing; levels also could be proposed
based on expected workload measured as total number of- dollars
and total number of positions.

     OWPO should also prepare material thatanswers "the
questions State staff most often ask." OWPO could begin by
polling the Regional staff to determine, for the program
as a whole and for each specific delegated activity, what
questions the State staff most often ask.  OWPO could then
prepare standard responses and distribute this material in
question-and-answer form to all' the Regions and States.
This would reduce the need for Regional staff to have to
keep answering the same questions over and over as new
States assume delegation of activities or as new State
staff come on board.

     OWPOshould prepare standard training packages or otherwise
arrange to make training available to State staff and Regional
staff on how they should function in a fully-delegated program.
The current situation appears to be that each Region is making
its own arrangements to train State staff, and there appears
to be no training of Regional staff at all on how to deal with
delegation.  Such a situation is clearly inefficient in that
each Region must take the effort to set up its own training
procedures or programs, and the Regional staff who must do
this are generally inexperienced in designing and operating
training programs.

     OWPO should arrange for new distribution mechanisms for
ensuring that State agencies and State staff responsible for
carrying out specific activities receive, directly, and on a
timely basis, notice of new regulations or other requirements
thataffect the performance of their responsibilities.   Some
States have indicated that they are dissatisfied with the

-------
                                      -43-

          mechanisras now used to inform them of -new requirements and proce-
          dures.   They frequently become aware of new requirements only
          through staff-level contacts with the Regional Office.  In
          other cases they discover that a new requirement has gone
          into effect at a formal review point several months later.
          The State is then told that everything  they've done in that
          period has to be redone since it failed to reflect the
          new requirement.

               To prevent such problems from recurring,  OWPO should
          develop a distribution mechanism that results in automatic
          notification of State Program Managers  of all new require-
          ments or procedures that affect their operations.   Such
          notification might be timed to reach the States two to three
          weeks after the Regional Offices are informed.   Such timing
          would help ensure that the Regions will be prepared to
          respond to State inquiries regarding the new requirements
          as  soon as the State officials receive  the formal  notice,
          and would allow the Regions reasonable  time to prepare'
          supplementary guidance when the Regions feel that  this is
          necessary.
L

-------
Chapter 9;  Concise Summary of RecommendedOversight
            Methods for each of Phases I throughVI


     The specific oversight, mechanisms recommended for Phases
I and II in Chapter 3 and recommended for Phases III, IV, V
and VI in Chapter 7 are summarized in Table 4«

     The remainder of this Chapter will present "checklists"
of oversight methods to be used in each of the six phases
of development of a State program.


Phase I Oversight--Helping the State Get up to Speed.

     In Phase I:

     1.  The Region encourages inquiries from State staff.
         Direct contacts between State staff and their
         Regional counterparts are accepted as the norm.

     2.  Regional staff are encouraged to call their State
         counterparts directly to inform them of any problems
         in the way an activity is being conducted.

     3.  Informal communications are expected to serve as
         the primary source of information about potential
         problem areas.

     4.  Document packages from the States requiring Regional
         signoff will be given in-depth (essentially duplicative)
         review by Regional staff until they are satisfied
         with the quality and consistency of State performance
         for that activity,

     5.  The Regional staff gives an in-depth review to selected
         State document packages based on the criteria
         described earlier:

             o   the importance of the activity,
             o   how long it has been delegated,
             o   the adequacy and consistency of recent State
                 actions for that activity,  and
             o   the extent of State requests for assistance
                 for that activity.

     6.  The Regional staff attempts to correct any problems
         discovered by successively applying the following
         corrective measures:
                 Provide telephone feedback to the State staff
                 member responsible,  noting the exact nature
                 of the deficiency.

-------
CO
t!
(1)
14-1
1
i!
4J

f
2

I
•8
 "8
 I
     is
> EH  H E"1 ^ S
H M  > rf O S
E-" >  H gj H 0j


< £  S O § W
              en

           H < g CO
                  O »
                            to
                          H) -H
                          ^ 4J
                         4-> O

                         .3*
                            S
                          O C
                         •H -H
                          ra 51
                            in
                    03 Ot -P

                    CO 0 M
                       ^4 <1>
                               <4-t
                                <0
                                                co
                                            32
                                           i  «j -H

                                             In  
                                        O  c
                                       •H -H
                                        in  tr.
                                     {


                                          S
                                            0) fO
                                                 C H

                                        *r      -H QJ
                                        CN      +J OJ
                                           co     W
                                        M E  «}  0 0
                       SCO ra
                          (U







                     4J -H
                                     'H
     0) «O ^   (ti

  U> £ CO 4J  >i

  ^^aiS-^
I  -H 4J -H 4-> H
I  4J C w  tQ  ro

'  S i 01 '»  &
                                              fd
                                                 M
                                              BU  H
                                                 M
                                                                                           CO
                                                                                          -S
tion
staff
ega
                                                    nS

                                                    ^"S

                                                    8*°
                                                       w
                                                                                                   in
                                                                                                a O
                                                                                           0) -H
                                                                                           4J  g


                                                                                           W|S




                                                                                           03
i
H
«
 (0
 i  cc
 5  5)
 ffl -H
 C0.4J
 rOl-H
 >i-H
                                                                                                 i—'    H
l
ct
a
fo
                                                                                                   w
                                                                                                   *
                                                                                                           M-)
s
                                                                                                                      id

                                                                                                                      £
                                                                                                           ffl W &

                                                                                                           O 0) -H
                                                                                                           •!l.2 b
                                                                                                                •sa
                                                                                                                     *
                                                                                                                   4J  O
                                                                                                                   a-
4J 4J
 1
T->.
1
i-H


!
3
                                                                                                        3$    *S
                                                                                                                 en
                                                                     a
                                                                     «
                                                                     *

-------
•o
•**

 cu
                          0 0
                        CN
                            OP
                            o
                            CD
                          ir,
                          •H
                        CN  „ 2. .O 3.
                        fe°
                        itig  .
                     S a-3Z
                        03  (U
                     SfUS
                     se!
                                                                        CO
                                                                        0)    C
                                                                       •H JJ  O
                                                                       +J C -H
                                                                       •H Q) -U
                                                                        ^ g  
               CO W
            a, c -H
            f O J
            4J -H
                                                                        ^
              ^^

              s-d
            E r^ O

            iftj
            K) (0 (X
                                                                          O  O
8
       S
                                                                                    8
                                                                                    in
                                                                        OrH  O


                                                                        &    £
                                                                        C S "5 S  HJ S
                                                                        O ••-» ® -H  01 -H
                                                                         O. >  C >
                                                                        SM  TP    i
                                                                        -p > -H   •>-!    
-------
t§      ^ 2 W
M CO  _ 03 H tO
6-1 w O 53   s
H u !Z E-i S E



PiisSl
•O
 Q)
       &
         l-l
         2
                CO
       ©
         ggi
         lie
                     4J ..



                       •H
                         CO 0)
                            s  s
                                 Cfi
                         if—
                       -*J it) 4.)  w -j-1
                       -H 0 (0  Q O
                       i— I 0 4->  0 
                          O O  0
4J  "

§  C    4-> CD
(Q H O n-t •-» co  g —
                     -n  a) <*p    <8 *
                     •H  > O
                     H  G) {N
                                    »—^
                                    5

                           &
                           mm  0 m H CJ
                           y ^* r< — ' 1 "^
                            •H
                            U  h-H
                            •H  0) fl)
                                    •s
                                    S
   5) dc
-,-4 > O
r-l 
              > *
  V-i  fl) ro  O
  0)    40   <#P
                 (DO   O C O
                 U f-   (N O rH


                 I  ^ §  j ti j
                 W  u) *U  w (v Q/
                 O -H 0) -H W -H
                 m  > Q  > C >

                 01  i^ !h  )-< >4 >-l
                 pt    p1    g-i
     U   JJ
   s 3 s o
   (D 
                    CO 0>


                 2 cl
                                                       'C C  *0 S  ®
                                                       C O  4-" 5) *C
                                                       -H <-(  cn k-i  0}
                          CO C 6
                          d) 0 ft
                        4J > -S Ji   CO
                        C -H 4-> DI   n3
                      In  CTl
                             C 1 fl)
                                    4J rH
                                                 a s T5
                                                 -H J-l  Ifl
                                                 
                             O
                                                          Sl..
                                                          »w  J-i (0
                                                          0
                                                     s
                                1
                                a
                                ^
                                a
                                g
                                *
                                                                                J-!
                                                                                0)
                                                                               •O

                                                                                CO
                                                                               -r4


                                                                               |

-------
                       -45-
        o   Perform a sample {or model) review that
            demonstrates to the State staff how the
            activity should be carried out.

        o   Go to the State Office to conduct a sample
            review jointly with the State staff member.

        o   Provide or arrange for additional training
            for the State staff member.

        Such actions are to be followed by:

        o   careful monitoring of subsequent State
            actions of this kind to determine whether
            the quality has improved, and

        o   remaining in continuous contact with the
            State staff to provide feedback on perfor-
            mance and to provide additional guidance
            as needed.

7.  Program Managers in the Region (at the Branch and
    Section Chief level) meet regularly with State
    Program Managers to discover and review current
    problems and to inform them of any changes in
    regulations or procedures.  These meetings can be
    conducted in person or by conference call, with
    face-to-face meetings at least every month or
    two.

8.  Pending development of a system for direct noti-
    fication by OWPO, the Region informs both the
    State Director and appropriate State staff in
    writing of changes in regulations or procedures
    that affect their operations.

9.  Later in Phase I, the Region begins to conduct
    Paperwork Sampling at the Regional Office on a
    quarterly basis for any activities that have
    evolved to the point that the Region is conducting
    only a checklist review.  The sample consists of
    10% of the actions in each activity during the
    previous quarter (or more than 10% if the number
    of actions is less than twenty).   The sample
    should include both large and small projects but
    may give somewhat greater emphasis to:

        o   large projects
        o   environmentally sensitive projects
        o   unique or particularly troublesome projects

    The Region prepares a formal report to the State
    on the results of the Paperwork Sample.

-------
                       -46-

10. The Regional staff conduct formal "visits" to the
    State Office at least twice per year:

        o   Early in Phase I, these visits should be
            used to review the nature of the operational
            problems discovered during the previous
            six months and the adequacy of the response
            by the State staff.

        o   Later in Phase I, these visits should also
            incorporate a formal review of State proce-
            dures for activities in which the State's
            actions have been less than adequate.

Phase II Oversight — PromotingConsistent,
   High Quality Activity-Level Management

The State enters Phase II when:

    o   all activities to be delegated have been
        assumed by the State and

    o   all are being performed with reasonable
        quality and consistency.

In Phase II;

1.  The Region normally conducts only checklist reviews
    of incoming documents.

2.  The Region conducts formal Paperwork Samples at
    the Regional Office as described above (Phase I,
    Item 9).

3.  The Regional staff continues to alert their State
    staff counterparts of problems and inadequacies
    on an informal basis as they are discovered.

4.  The Regional managers continue to meet on a
    monthly basis with State managers to discuss
    current problems and changes in regulations and
    procedures.

5.  The Regional Staff conducts semi-annual reviews at
    the State Offices.  These reviews emphasize the
    State procedures for carrying out specific activi-
    ties, focusing on those activities shown in the
    Paperwork Sample to be resulting in products that
    are not consistently of high quality.

-------
                             _47-

Phase III — Promoting Project-Level Management and
   Introducing Direct Measures of Performance

     Phase III begins when:

     o   all delegable activities have been assumed by
         the State and

     o   all activities are being performed with a con-
         sistent high quality.* The Region inaugurates
         Phase III when it finds the consistency and
         quality of State activity-level operations to
         be high enough that it is willing to go from
         a 10% to a 5% sample in conducting the quarterly
         Paperwork Samples.

      In Phase III;

     1.   The State has attained*high quality in its activity-
         level operations that the Region and State begin
         to work towards project-level management and
         oversight.

     2.   There is now substantially less need for informal
         contact between the State and Regional staffs.
         Most oversight information is now obtained through
         formal ovrsight methods.

     3.   Paperwork samples are conducted quarterly in the
         Regional Office using a 5% sample.

     4.   The Region begins to introduce some direct monitoring
         of State performance as part of its oversight methods
         The first performance measures introduced include:

         o   Activity-level measures of performance
             -  MBE performance
             -  Change order cost increases

         o   Project level measures of performance
             -  Actual project cost as compared with
                project budget
             -  Project timeliness as compared with
                "reasonable expectations"

         Note, however, that performance measures are adopted
         only after negotiation with the State on the specific
         measures to be used.  The Region and State also
         negotiate "performance targets" for each aspect of
         performance to be monitored directly during the
         following year.
*This does not,  however,  mean that there will be a zero
 error rate.

-------
                             -4*-

         The Region conducts semi-annual reviews at the
         State Office.  The Region uses these visits
         primarily to review State procedures.  These
         reviews of procedures are conducted for the
         following reasons:

         -   to determine the degree and adequacy of
             project-level management (i.e., the degree
             of project-level coordination between
             ativities).  In doing so, the Regions
             should focus on State procedures for the
             "key activities" for effective project-
             level management (i.e., activities related
             to the analysis of alternatives).

             to follow up on State procedures for acti-
             vities that have been found to be deficient
             through direct measures of performance.

         -   to check on State procedures for those
             activities for which no direct measures of
             performance were available.
Phase IV — Promoting Program-level Management and Increasing
            the Use of Direct Measures of Performance

     This phase begins when the Region determines that the
State has achieved consistent and effective project-level
management.

     The oversight methods used in Phase IV are the same as
in Phase III with the following exceptions:

     o   Direct measures of performance are adopted for
         additional aspects of State performance.  (i.e.,
         by agreement with the State, the Region continues
         to substitute oversight through direct monitoring
         of performance for oversight through review of
         State procedures.)

     o   The Region and State focus on promoting program-
         level management by the State.  The Region therefore
         focuses more on the "key activities" for effective
         program management in its semi-reviews at the State
         Office.  These "key activities" include:

         -   improved assessment of needs

             incentives for communities with strong needs
             (based on water quality considerations) to
             apply for Step 1 grants

-------
                            -49-

             assistance to communities with strong needs
             in preparing Step 1 grant applications

             improved priority setting among projects based
             on water quality consideration
Phase V -- Promoting Integrated Water Quality Management
           and Increasing the Use of Direct Measures of
           Performance

     This phase begins when the Region determines that the
State has achieved consistent and effective program-level
management.

     The oversight methods used in Phase V are the same as
those in Phase IV with the following exceptions:

     o   State operations are of such high quality that the
         Region deems it appropriate to drop from a 5% sample
         to a 2% sample in conducting its quarterly Paperwork
         Sample.

     o   The Region, with the State's concurrence, continues
         to substitute oversight based on direct measures
         of State performance for oversight based on review
         of State procedures.

     o   The Region and State focus on promoting integration
         of the program into a broader program of integrated
         water quality management.  They do so, in part, by
         placing more emphasis on the "key activities" for
         achieving this objective:

         -   comprehensive areawide water quality planning
             at the local level,

         -   comprehensive areawide water quality planning
             at the State level, and

             joint scheduling of all water quality improve-
             ment efforts in each locality (including municipal
             control efforts, industrial point source control
             efforts, non-point source control efforts and
             groundwater protection efforts).
Phase VI — Overseeing a, Fully-Developed, Fully-Delegated
  Program

     This phase begins when the Region determines that the
State has achieved consistent and effective management of
the program as an integral part of an integrated water quality
management program.

-------
                             -50-

     The oversight procedures in Phase VI are the same as
those in Phase V with the following exceptions:

     o   The State, with Regional encouragement and assis-
         tance, continues to seek ways to improve the
         efficiency of State operations without sacrificing
         the quality or "scope" of the State program.

     o   The Region continues to develop and adopt direct
         measures of performance for overseeing the State
         program.  Use of these direct performance measures
         replaces oversight based on review of procedures.

-------
                            A-l
Appendix A;  Alternative ApproachesforImproving
             Project-level Review by the States
     In Phases I and II, the State staff tends to approach
each project as a disjointed set of activities to be re-
viewed.  In Phase III, the State begins to review projects
as a whole.  The Region attempts to promote this change in
management scope by assessing the State's project-level
performance.  When deficiencies in the effectiveness of
project-level reviews are discovered, the Region and State
negotiate changes in the State's review procedures to
correct these deficiencies.

     In negotiating with the States on possible improve-
ments in procedures to promote integrated project level
reviews, the Region should bear in mind the advantages and
disadvantages of the three distinct procedural changes
the State might propose.  The three alternatives that the
State might propose are:

     o   Require multiple concurrent reviews of certain
         documents by different Staffs with related
         functions.

     o   Institute "project review meetings," with
         participation by each staff with distinct
         responsibilities

     o   Create new integrated review groups.

THE RELATIVE COST AND RELATIVE EFFECTIVENESS OF THE THREE
  APPROACHES DIFFER GREATLY

     Require Multiple^ Reviews.  While requiring multiple
reviews of various documents by different staffs is the
solution most often proposed, these multiple reviews are
the least desirable means of achieving increased coordi-
nation.  For example, it may be discovered that deficiencies
frequently arise in projets because of inadequate facility
plan review.  A major reason for this is that the State
fails to discover fundamental flaws in the basic analysis
of alternatives because that analysis appears in four
separate documents.  Typically,  we might find that:

     o   A member of one staff group is responsible for
         reviewing the alternatives analysis incorporated
         in the environmental assessment (EA),

     o   A member of another group looks at the manner in
         which the grantee and its consultant consider
         alternative and innovative (A/I) technologies, and

-------
                            A-2

     o   A member of a third group looks at the remainder
         of the alternative analysis contained in the
         facility plan.

     o   A member of a fourth group reviews the financial
         aspects of the project (the cost analysis of
         possible alternatives, the proposed method of
         financing the local share, and the user charge,
         industrial cost recovery — UC/ICR — proposal*).

     To ensure that its staff begins to consider the quality
of the analysis of alternatives for the project as a whole,
the State might simply propose that each of three groups
look at and comment on all four aspects of the alternative
analysis (i.e., the EA, the A/I analysis, and the remainding
analysis of alternatives contained in the facility plan, and
the financial plan).  If any group raised objections to any
portion of a project's alternatives analysis, the project
would be "flagged" and a joint critique would be prepared.

     Although such a solution is easy to implement — since
no change in organization or functional responsibilities is
required to implement it — it is usually not very satis-
factory because:

     o   the staff, who are already devoting full time to
         reviewing their individual pieces of the alter-
         natives analysis, resent having to "shuffle more
         paper" than before.  Furthermore, they can in fact
         be overwhelmed by the total volume of material
         they are now expected to review.  In a real sense,
         this approach increases total workload by requiring
         multiple reviews of the same material.

     o   As a result, total review time increases, bringing
         with it increased project delays.

     o   Also, the overworked staff does a less adequate job
         in conducting each review.

     The only way to make "increased coordination" work is to
increase the size of each of the affected staff groups so that
each staff member has fewer projects for which he or she is
responsible.  But with current staffing limitations in the
States, such a course of action is not normally available.

     Institute Project Review Meetings.   An alternative
solution that avoids some of the negative aspects of simple
"increased coordination" is to have each staff group remain
responsible for reviewing only that portion of the project
for which it was originally responsible.  Members of each
staff group then meet periodically as a project review team
  The situation is complicated by the fact that the UC/ICR
  work is not normally conducted until well  after the completion
  of Step 1 planning activities.

-------
                            A-3

to "compare notes."  By having each representative discuss
the strengths and weaknesses of the material he or she has
reviewed, it is possible for the team as a whole to discover
deficiencies in the project as a whole that were not uncovered
in the individual reviews.

     The advantages of such an approach are:

     o   the burden on each group does not increase except
         to the extent that time is consumed in meetings of
         the "project review team>"

    , o   each group can retain a specific expertise, yet
         the project team (and therefore the project) can
         benefit from that expertise when appropriate.

     The approach has the disadvantage that:

     o   either considerable time is consumed in meetings
         of the project review team, or

     o   the project review team meetings are so-hurried
         and discussion is so limited that there is a
         significant likelihood that major problems in
         project consistency will go undetected.

     Create Integrated Review Groups.  A third possible
approach is to assign closely related review functions to a
single group.  A single staff member, for example, would
then be responsible for reviewing all aspects of the analysis
of alternatives for a single project.  By consolidating
the separate staffs that formerly conducted separate pieces
of the alternatives analysis (i.e., the EA, A/I and facility
plans and financial planning staffs*),  enough staff will
be so available that the number of projects assigned to each
staff member can be significantly reduced.  In this approach,
"coordination" among different staff members or staff groups
that perform related functions is no longer needed because
all of the functions to be coordinated have been "integrated"
and assigned to a single individual.

     Such an approach has the following advantages:

     o   It does not increase the total workload.

     o   By assigning full responsibility for all aspects
         of related reviews  to a single individual, this
         approach minimizes  the need for coordination
   Note that most States do not have distinct financial
   planning staffs — indeed, little financial analysis is
   now conducted at either the State or Regional level.

-------
                            A-4

         between staff (and the attendant possibility of
         failing to discover problems because of faulty
         communication).

     o   It is likely, in the long run, to increase staff
         satisfaction since each staff member is given a
         larger, more significant aspect of each project
         to be responsible for.  (In other words, the
         staff would tend to feel less like "workers on
         an assembly line" and more like responsible
         professionals.)

     It has the following disadvantages:

     o   Normally, only one "mind" is brought to bear on
         major aspects of each project.

     o   It requires a significant reorganization, including
         a realignment of functions and the training of staff
         to perform the new functions that are being added
         to the ones they had before.
     In general, the creation of integrated review groups
where feasible appears to be the best approach for promoting
"project level review."*  Nevertheless, there is a limit to
the number of project activities that a single person can
be expected to be able to review adequately.  Some coordi-
nation between staff and staff groups will therefore always
be necessary when reviewing individual projects.  It is there-
fore inevitable that several individuals will be responsible
for reviewing any given project, and establishing "project
teams" appears to be the best mechanism for promoting adequate
coordination between these individuals.  Such teams would
meet regularly to discuss ongoing projects for which they are
responsible and identify possible problems with the project
as a whole.
*While we have recommended the creation by States of integrated
 review groups in order to improve the degree to which it
 conducts effective project-level reviews, we will now
 for illustrative purposes describe the effective use of
 such groups use in a Regional Office.

 One Region (Region IV) has already begun to make use of inte-
 grated review groups in its own internal operations.  To ensure
 the fullest coordination of Facility Plans with Environmental
 Assessment and with 208 Plans, the responsibility for reviewing
 Facility Plans and preparing Environmental Assessments has
 recently been assigned to the Region's Water Quality Management
 Branch, which also has responsibility for 208 Plans.

-------
Appendix B:
A Possible Experimental Approach
for Assessing a State Program's
Impact on Water Quality
     As stated in Chapter 6, the approach described here may
not be suitable for regular oversight of a State's performance.
It might, however, prove to be useful for periodic assessment
of the extent of the problem being addressed and the extent
of the program's beneficial impact.  Application of this
approach presents sufficient technical problems that its use
should not now be required.  At most we might expect that one or
two Regions may determine that its use would bring sufficient
benefits that they might, at some point in the future, attempt
to apply it on an experimental basis.

Assessing the Extent of the Problem

     The basic purpose of the Municipal Wastewater Program is
to eliminate violations of water quality criteria due to dis-
charges of inadequately treated municipal wastewater.  The
extent of this problem in a given State can be quantified
as follows using reasonable estimates based on available
monitoring data:

     o  number of stream miles violating water quality
        criteria — solely or in great part — because
        of inadequately treated municipal discharges

     o  amount of lake surface acreage violating criteria
        because of municipal discharges

     o  amount of bay and coastal water acerage violating
        criteria because of such discharges.

     With a periodic reassessment of the extent of the problem
(e.g.,  every 1  to 3 years), it is possible to determine both
the extent to which water quality adversely affected by
municipal discharges is improving (or deteriorating)  and to
quantify the extent of the benefits derived from projects
completed since the last assessment.   As a result of such a
survey, it might be discovered,  for example that "37  stream
miles that violated water quality criteria as a result of
inadequate treated municipal wastewater no longer do so
because of treatment facilities  that have come on line in
the last two years.  There are now only 123 stream miles in
this State that violate water quality criteria wholly or
substantially as a result of inadequately treated municipal
discharges."

     This approach could also be applied to projecting the
water quality impact of projects now underway by using reason-
able estimates or projections.    For example:  "Treatment
projects initiated this year (i.e.,  new Step  3  awards)  will
result in 55 additional stream miles  that do  not now meet
water quality criteria because of municipal discharges to
come into conformance with those criteria."

-------