virortmental Protection
Office of
Admintstrij 1 ion
401 M Street, SW
Washington DC 20460
tPA.ADF'-Auriit.82-02
&'|>ti;mbi.'r 1982
ADP Audit Report
Storage and Retrieval
of Water Related Data
The STORE! System
-------
-------
em*
rt- I.
i, 3W
Washington, DC 20460
United States Environmental Protection Agency
Office of Administration
401 M Streets, S, W.
Washington, D. C. 20460
EPA-ADP-Audit-82-02
information Resources Center
US ERft (3404)
401 M Street S
Washington, DC
ADP AUDIT REPORT
STORAGE AND RETRIEVAL OF WATER RELATED DATA
THE STORET SYSTEM
ADP AUDIT TEAM
OFFICE OF MANAGEMENT INFORMATIONAND SUPPORT SERVICES
September 30, 1982
CD
CD
o
OJ
HEADQUARTERS LIBRARY
ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460.
-------
W6 ซJ^;ปfc
-------
TABLE CF 03NTENTS
Page
MANAGEMENT SLMvlARY
I. Introduction 1
II. Background
III. Audit Results 12
IV. Surmary of Conclusions and Reccnmendations
56
Appendix A - Analytical Plan
Appendix S - Individuals Interviewed in the STORE! Audit
-------
-------
MANAGEMENT SUMMARY
This report contains the. results of an audit of the Environmental Protec-
tion Agency's STORET system. The goals of the audit were to evaluate EPA's
management of STORET in three areas: (1) timesharing budgeting and billing;
(2) system management and operations; and (3) user support.
Since this audit was one of the first performed as part of EPA's system
audit and evaluation program, it was intended to serve as a prototype manage-
ment audit. A draft version of this report was reviewed by the Office of Water
Regulations and Standards (OWRS), the Office of Drinking Water, by the National
Computer Center (NCC), and by representatives from two Regions, four states,
and the Corps of Engineers. The comments did not concern basic policy issues
and have been incorporated into this final report.
Background , .
.STORET is used to store, -retrieve, and analyze data on the quality of the
nation's waterways. It is one of EPA's largest and most, widely used computer
systems. STORET is'used by personnel in EPA headquarters, EPA regional offices
and laboratories, states and interstate commissions, and other Federal agen-
cies. Use of the system (including data- entry) is completely voluntary and
almost all system operations are user-controlled. EPA provides timesharing
funds and other support for STORET to states to encourage data entry and.use of
the system, thus .improving water quality analysis and providing EPA with access
to a data base that would otherwise not be available.
STORET runs on the IBM 370 computer at EPA's National Computer Center
(NCC). -The Data Processing and User Assistance Branch (DP/UA) of the Office of
Water Regulations and Standards' Monitoring and Data Support Division (MDSD) is
responsible for user training, documentation, and system maintenance, opera-
tions, and enhancements. ..._..
-------
Summary of Conclusions and Recommendations
Exhibit MS-1 summarizes our major conclusions and recommendations for each
audit area. Our analysis revealed that STORET is generally well managed. Our
audit did identify a number of problems that EPA should investigate and elimi-
nate; in other areas we have recommended that some part of EPA analyze a topic
in more detail in order to define more precisely the costs and benefits of
various actions or changes to existing procedures. Overall, however, we iden-
tified no critical problems in EPA's management of STORET and are proposing no
major changes in procedures.
-------
. I
i-M
03-
o
"
ซc
Q
"ZZ
UJ
o
0
yj
o
'on
-o
t/5
_j
o
z:
o
^^
o:
s
to
"
VI
1
ฃ
g
5C
i/i
3
^
UJ
<
^
i-
L
i X
o ฃ
ซl ซQ
u *"" .
^ X vt
^ w ^
Ol 4? "
i. ^ ป
W 4} VI
3 O>
tj vป ซป
ป ""* vป
*"* 5 .
X O) <
3^5
*J >Q
"* a *
tJ -a"
ป *"
ซ 5 **
*-'ฃ
< * ง
"s ~-
o ~ o
*
^
UJ
O s*
C* C
ซ* T5 O! X
* C W
IJ s (fl. k.
a a
V. Oi
a ^4
ซ i/i W
7 ซ * *"
a * > vi
5.^ 3 i 5
c 3 SI
^ 3 j: > ป,
ซ ~ - ฃ a
"fSgs
= 5 i^ซ
!ฐ r a
s*
^
a
5 c
si
I 1
- 1
1 1
1 I
I-. *
j ' '
1
1 S
1*
O
u "2
at *^
L. u
u a
o c.
w
o
C **
<3 g
w w
gn^o
3 c
> 3
.* J
2S
3 >fl
"51* i
a a
fhl
*
Vป
*^ V Ol
s*.. as -
*Q U VI u 41
C W k. V) 1 U
9 Ol V W vi 3
(J 'L.
.H-=ฃ> "" s ฐ *
ป ,
W W 4J 3 ^ -fl
5 3 --j 4> *J **
^S'S- ง w*- s
c "^ aj > vt a oi
-o e t/t ^ c -^
* *
e
4)
?
'^
' U
Of **
U
a *
w c
_c -9
2 ?
*r
-s
I
i
i
01
c
o
I/I
o>
c
Of
c c
a a
or ซ
u *J
13 U
C G.
iป X .
X
If'
> V
fl
^ a"
2 ฃ
0)
vป o
b *
O V
ซ
l/(
l/f
a
c
vi o
c a.
S
-> =
ซ 1
1 =
"a ซ
a <
i i
s |
i/) ^
O)
^3 C 1/1 VI
W TJ ซป X
2 S^-a .
s -5 5 s>
T3 S S -
ป * *3
*O L, VI
a c a
am x
.Z VI Cl
wl X*^
*- o ซ
c ^
o w a. 13
ซ -^ 2 's
^o|i
's o
- y gi_
S O wi
= |1|
2 0 2 0
** C1 "^ a_
w 5 ซi
O a S o^
;_ i =
Jj S i S
"
c
S -^ a ^ :
M a a
O 01
' VI
vi J vi ^ ifl Oi
w o **i u e.^
ง71 *"t
^3 01 - O
=. ^ 3 5
c 3 w ' a. a
3 X w U
'- c 5 3
w ^ --* e; ^ ซ9
-o^I C^J s .'
i-3 | |^<
*- 01 L X*^
ป U
^ * - - ^ 51
a 3 oi
J ^ -3 T3 > 3 ^
VI '- ^ 4t VI iซ
O -* U ^ U '
C * 3 V U V vi
ซ- u a* v vioisa
< J - CL = k- ^ CL.
* *
VI
1
IS
i '
UJ - *
O
* *
'
'
4*
5
*
VI
01
* W
11
*c ?
^ X
i.
X Of
3 41
vi -C
31 5
> 5" -
3 j< a
a^ 4 -n
a a
3 3"
"3 4) nfl
U "^ -
3 -* 5 '
V O
3 S 3
3 ^
'i. 1_ 1.
a. ia 10
-
Ol
0 .
4)
eg.
' *
3*
_ X .
.? ':
3
W '
a*
UJ " '
2ฃ ,
'O
ll/n
t
IP 1
M. ^ **
ซ ft.*' g
SC JE ฃ
u v* ,a
^ 2f ^ us
ป P 01 _2
5 3 i- ^j
251 si
^ ai ^B ? >; '
vt 3 C - VI Of
< "3 Sir < 5
"*s, 3 U Vป "^ 'r~
aT u a w =. a
3 Q.T3 3 ซ*-
*
ฃ
a 3
o3 ' -1
-I ii
"c 3 a^
*_f a -
u c a i
(ta 5 v *J vi
r w 33=.
v> a v -3 o a
'-23- งซI
_ J s ~ tj a.
w cJปฃ *= 3" O
i s ~ 1 s "i"
4J ฃ 4J U ซ
S Oj u 0 J 0
o c '
"" 1 * 3-53!-
Ill i^3
OT U W .itf
3 ul O It U
-------
a 1
yj O
r3 o
2: LU
i_ 0ฃ
Z Q
O Z
o ป
S to
I I
ii O
x;
LU
O
CJ
O
>-
c;
i >, ฃ *
5 ซ. u*
- ^3
~ 5 3
_ 2 ^*
"3 o is i
-C w-l ^
irt y< v4 >
7s . 3^2
4 y ^ Ol 4j
*z* 5;fฎ
O -ซ 5 4? O wป
- -^ C. S
3 5 ci o w.
tsl. :UF
M *u tn *" ^ ฃ1.
i 2? a ฐ
n> ^( C C >
-is s>-ง
ง o w a J i.
!'" !ฐ3
t
o
1* cl
4j ฃj
Of V *"" *J
"C Si > i*
;ป t: "
V'a* = "
V 3 "S X
B^ ?^
E S, -* *
ai ซc s >
t: 3 ปซ _i; LM
7^| Y"^S
ฃ ง S SLi/i
"0 X > w
- ฃ - 01 ^
-^ -w. < 13 *-
*
9
I
S
MJ
S
4 j
*
P
c |
1 k. i. vn
1 4f ^
3 ^- C J
e > 2 "
b. i ^ Oป
c w cr.
OJ <9 =
*^ W S w 01
i^ wi O 41
ฃw ^ 0 S
SO 4 w ^
U ' 3 wx 41
Si * ซ ^ 3
< *rf 3 H-
*-<** O UJ
L, 4*
ฑL ^ ซ ^
^ O * -o
J43 01 O C
u w e
52 5ll
S- ฃ-5o
ซ>
*>
f ^.
Is!
^- 3
- U 0
o 5 ซ*
ฃ Kt
*ฃ 3
i O
fli O "5
ฃฃ -is a
w 4ป
3 ซ* S*
5- us
er %*ป *" c
4> 41 W S
JT tf* U V
1- 3 W *1
*
"^ !?
| |
L.
01
^C ซ
= *
01
c
a "
4ft
^a
01
a
*"
ง
-a
i
(j
T) X
uZ!
a 3
U
01 O
4, -a
t"
ง
*
e
I
SJ
0
"
1 = =
^ ^ < ^,2 |ง
O C & O.ซ/1 = k
55 5 a. "5 a 2
.^3 % Ol *J Jf
V C O ฑ9 ซ
*" "3. T? 5 S > 5
w "a 3 O
M ita. "* ** C*i ^
-fl O t. X *! >*
3 01 U C trt
> i O ^ it **
^3 O ฐc^งS*?*
ซ- 4>^4J*J5**3
Cl v ^
j"IS ,^| plf
St. c L. u a -as
q i u ^s w ซ M-
*
11
ฃ
1 j
aj
=
"
-------
. I. INTRODUCTION.
This report presents the results of an audit of the Environmental Protec-
tion Agency's STORET system, which is used to store, retrieve, and analyze in-
formation about the quality of water in the nation's streams, rivers, and
lakes. STORET operates on an IBM 370/168 MP computer at EPA's National Compu-
ter Center (NCC) in Research Triangle Park, North Carolina. The system is used
by EPA personnel at headquarters and in field laboratories and regional of-
fices, by other Federal organizations (such as the U.S. Forest Service and the
Array Corps o.f Engineers), and by states and interstate commissions.
By intent, the STORET. audit was tightly focused. Briefly, the. effort was
intended to produce a management audit of the system, as opposed to a mission
audit or a. technical audit[l] . In other words, primary attention was given to
the efficiency and effectiveness . with which EPA manages an existing system..
Issues such as the fundamental need for STORET, the system's overall costs and
benefits, the quality and processing efficiency, of its programs, and -the ade-
quacy of its data base .design were not considered to be within the scope of our
evaluation. ' Many of these issues were-discussed in more detail in an earlier;
report (An Interim Report on the Audit of. the Storage'.and Retrieval of Water
Related Data System)and may-deserve additional attention' at'a later time.
Within the broad framework of a management audit, attention should logical-
ly be focused on those areas that appear to offer the greatest potential bene-
fits and improvements relative to' the effort required for investigation and
analysis. After collecting a wide variety of information about STORฃT in gen-
eral, a. decision was made to evaluate three aspects of STORET management in
more detail'. . . . "' . ' '
{!] As part of Us overall effort to implement a system audit and evaluation
program, ' EPA is attempting to define a typology, of audits accordin.g ' to
purpose, and content. The distinctions between technical, management, and
mission audits are explained in EPA's system audit program planning docu-
ments. The audit of STORET was the Agency's, prototype management au-
dit. . ' . ' "
-------
o First, how effective are EPA's procedures for allocating timeshar-
ing budgets to states that use 5TORET, adequately informing users
of the financial status of their computer system accounts, and
billing^ for and collecting amounts owed by non-Agency organiza-
tions ?
o Second, is STORET well operated and maintained? This area includes
topics such as system reliability, backup and recovery procedures,
system documentation, system enhancements, and system maintenance
and operations.
o Third, how well is EPA supporting STQRET users? The overall area
of user support was subdivided into user training, user documenta-
tion, and EPA's user assistance functions.
After the scope of the audit was defined, an analytical plan was prepared
to specify more precisely the questions to be answered and the sources from
which relevant information was .to be collected. Since this audit was one of
the first performed as part of EPA's system audit and evaluation program, it
was intended to serve as a prototype management audit. The audit team's ac-
tivities were guided by the analytical plan because no EPA system audit stand-
ards were available. A draft version of this report was reviewed by the Office
of Water Regulations and Standards (OWRS), the Office of Drinking Water, by the
National Computer Center (NCC), and by representatives from two Regions, four
states, and the Corps of Engineers. The comments did not concern basic policy
issues and have been incorporated into this final report.
The audit was conducted by a project team consisting of personnel from
EPA's Management Information and Data Systems Division (MIDSD) and American
Management Systems (AMS). Work first began early in calendar 1982 with the
collection and review by MIDSD of information on STORET itself and on the per-
ceptions and opinions of system users. Among the activities completed in this
phase of the project were a review of some STORET documentation, attendance at
a STORET training class, the completion of two group interviews with STORET
users from EPA and states, and preliminary interviews with EPA personnel re-
sponsible for supporting STORET. A work group of individuals familiar with
-------
STORET was formed to facilitate information collection, and discussion of poten-
tial issues and problems. The work group included representatives' from EPA
headquarters, two EPA regional offices, two states, and two other Federal users
of the system. Data collection, analysis, and audit review for the areas iden-
tified above were performed largely in August and September of 1982.
The first phase of the STORET audit culminated in the selection of issues
for more detailed analysis, the preparation of the final analytical plan, and
the production fay MIDSD of An Interim Report on the Audit of the Storage and
Retrieval of Water Related Data System (Donald C. Rosene, Project Director; ADP
Audit Team, Office of Management Information and Support Services, U.S. En-
vironmental Protection Agency; August 9, 1982). The Interim Report summarized
the information collected about STORET in the first phase of. the effort and
identified possible actions that might be taken to address some of the problems
mentioned by system users. It is available through the Office of Management
Information and Support Services (OMISS).
This report represents the results of the second phase of the STORET audit.
The balance of the document contains three chapters:
. o Chapter II, "Background, " provides an overview of the STORET system
and outlines the audit methodology;
o Chapter III, "Audit Results," describes the audit objectives, find-
ings, conclusions, and recommendations for each of the three main
audit areas; and
o Chapter IV, "Summary of Conclusions and Recommendations," inte-
grates and restates the most important results of the evaluation.
The complete analytical plan and a list of personnel interviewed as part of the
project are included as appendicies.
-------
II. BACKGROUND
A. Overview of STORET
1. Svstetn Historv and Use
The fundamental concept for STORET was developed in the early 1960's by
the U. S, Public Health Service's Division of Water Supply and Pollution Cont-
rol. Briefly, the emergence of increasingly powerful data processing capabili-
ties made feasible an automated system that would store water quality data from
a variety of sources in a common format and allow users to retrieve and manipu-
late that data to meet their own needs. -In addition to offering savings in
system development and operations, such a system would facilitate data sharing
among users. STORET began operations in 1964 with information from approxi-
mately 140 sampling locations.
Over time, STORET's functional and technical characteristics have been
greatly enhanced. STORET came under EPA!s responsibility when che agency was
first formed. The system now contains about 32 million individual water quali-
ty observations from more than 500,000 surface water collection points. Ap-
proximately 6,000 different water quality parameters can be recorded in the
system.
STORET is provided to users as a common utility and is used by organiza-
tions at EPA headquarters and regions, by states and'interstate commissions and
.by other Federal agencies. The STORET users mailing list contains over 600
entries; STORET support personnel estimate that there are approximately 250
active users. STORET is used to support Federal, state, and local efforts in
areas such as water quality monitoring, pollution control, basin planning, and
water quality reporting. Use of the system is completely voluntary. EPA has
encouraged states to use STORET for data entry and water quality analysis by
providing free timesharing, training, documentation', and other assistance. Ob-
taining high participation and satisfaction rate on the part of states has long
been an important goal of system managers.
-------
2, System Structure and Operations
Exhibit II-l depicts the structure of STORET. Conceptually./ the system
consists' of two basic functions: (1) data entry and (2) data retrieval' and
analysis. '
o Data entry is initiated by system users, who enter water quali-
ty information from sampling stations in their states or loca-
lities. Data entry can be accomplished entirely through STORET
or by.STORET in conjunction with user-supplied software. There
is no required data entry schedule. After entry into STORET,
data is subjected, to a series of preliminary edits and. (if
valid) is stored, in a transaction file. Once a week, the file' .
of valid transactions is used to update the main STORET'water
quality file. Any errors detected 'by the system's edit proce-
dures are flagged and. returned in automated form to the user -
'. for correction and resubmission. Each data record . identifies .
;-. .- ' "the relevant sampling station and- contains information 'about ' ;
r the date-and-nature of the sample and values for water quality - '
- parameters. Users also enter information- to describe new. sam-
pling sites. . ' . .
' ' -O Data retrieval and analysis are controlled by' STORET users. A
'' ''wide array' of on-line processing capabilities is available.
Briefly, users enter on-line commands that determine what data
is to.be retrieved from the. water quality file and how that
' data is to be manipulated'. Among the output options available
" ' .' to users are data listings, statistical analyses, .plots, and"- ;'
. .tables. .Users'can also retrieve data into their own files for .
processing by custom-developed programs. .
Thus, STORET is almost completely user-driven, with centralized control
of processing being limited to the data base update job, file verification, and
system backup activities. . . -. ,
-------
EXHIBIT II-l
OVERVIEW OF STORE! OPERATIONS
USER DATA
ENTRY
USER
3ATA
CORRECTIONS
Performed by
Each User on
an As Required
basis
USER ANALYSIS.
REQUEST
ANALYSIS
PROGRAM
Maintained by
> STORET Software
Group
-------
In addition to the water quality capabilities described above, STORET
can also be used to store and retrieve data on fish' kills, effluents, and
selected other water pollution-related topics. Because the water quality file
is by far the largest and most commonly used part of STORET, we focused our
analysis on it and did not address the system's other features.
3. Organizational Responsibilities
STORET is maintained and supported by the Data Processing and User
Assistance Branch (DP/UA) .of the Monitoring and Data Support Division in EPA's
Office of Water Regulations and Standards. DP/UA contains two sections: the
Information Access Section, which is responsible for system maintenance and
operations, and the User Assistance Section,. which is responsible for training,
documentation, and providing technical assistance to users with questions or
problems. Exhibit U-2 illustrates the relevant organization structure.
The system resides on an IBM computer "operated by the .Management Infor-
mation and Data Systems Division (MIDSD) at EPA's National Computer Center-at
Research Triangle Park, North Carolina. Thus, MIDSD is responsible.for provid-
1
ing the computer resources necessary to run STORET,'and is also involved'in
'timesharing cost reporting and billing.
4.' STORET Documentation . ' . . .
The following, materials provide additional information on the struc-
ture, operations, and use of STORET:
o Handbook, Water Quality Control'Information System (STORET,).
Volumes I and II (U.S. Environmental Protection Agency); . '
o Manager's Guide to STORET (U.S. Environmental Protection Agen-
cy) ; and .
o ' STORET program documentation (maintained by DP/UA's Data Access
Section). . . . .
-------
8
EXHIBIT II-2
MONITORING AND DATA SUPPORT DIVISION ORGANIZATION STRUCTURE
MONITORING AND
DATA SUPPORT
DIVISION
DATA PROCESSING
AND USER
ASSISTANCE
BRANCH
WATER QUALITY
ANALYSIS BRANCH
MONITORING
BRANCH
INFORMATION
ACC&SS
SECTION
USER
ASSISTANCE
SECTION
-------
B. Audit Methodology
This section, describes the audit methodology. The audit consisted of four
basic steps: (1) audit planning; (2) information collection; (3) analysis; and
(4) audit review and documentation. The purpose and activities of each of
these steps are described, separately below. In practice, work in various areas
tended to overlap. For example, some information collection and analysis tasks
.were performed iteratively, with additional data gathering being used to con-
firm or illuminate a preliminary conclusion or recommendation. Appendix B con-
tains a list of individuals interviewed as part of the project.
Overall, audit planning was performed beginning early in calendar 1982
through July 1982. Information collection, analysis, and audit review and
documentation were then performed, at a higher level of effort, in August and
September of 1982. The audit absorbed approximately 60 professional person-
days within MDSD.
Implementation of audit findings could be considered as a .separate step in
the audit process, but cannot be discussed as part, of this paper..
1. Step 1: Audit Planning
o Purpose of this step; define the objectives and scope of the
audit and prepare a plan for data collection
o Activities performed in this step:
.- preliminary review of relevant documents
- formation of a STORET audit work group to facilitate the ef-
' .fort
coordination of the audit with efforts in the Office of
Standards and Regulations in. the Office of Administration
- initial interviews with users of STORET and system opera-
tions and support personnel
- - preparation of the Final Analytical Plan for the STQRET
Audit (July 27, 1982)(included in this report as Appendix
. -A ). .
-------
10
preparation of An Interim Report on the Audit of the Storage
and Retrieval ofWater Related Data System.
2. Step 2; Information Collection
o Purpose of this step; gather all data necessary to answer the
questions listed in the analytical plan
o Activities performed in this step;
- interviews with personnel in the Management Information and
Data Systems Division (MIDSD) and the Data Processing and
User Assistance Branch
telephone interviews with a total of ten system users in EPA
headquarters, EPA regions, states, and other Federal agen-
cies (the users interviewed represent a variety of different
kinds of users, but were not selected to represent a statis-
tically valid sample of all users)
telephone interviews with EPA ADP coordinators and other
relevant personnel (List of individuals interviewed in Ap-
pendix B)
- collection and review of STORET documentation (e.g., users
manuals, system documentation)
- collection and review of other relevant documentation and
materials (e.g, Informatics1 audit of STORET, timesharing
budget data)
3. Sjep 3; Analysis
o Purpose of this step; synthesize all information collected and
develop preliminary conclusions and recommendations
o Activities performed in this step;
- analysis and review of all available data
-------
11
- formulation of major conclusions, recommendations, and other
significant audit results '
- preparation of informal listing of major findings and con-
clusions
4. Step 4; Audit Review and Documentation
o Purpose of this steo: review audit results with all concerned
parties and prepare final audit report
. . o Activities performed in this step: -
- review of- preliminary listing of audit results by -various
organizations in EPA
- revision of preliminary listing and formal presentation to
EPA. work group
preparation or draft audit report .
- -review of draft audit report.by STORET audit work'-group
' - preparation of final'audit report ' ' - _ .
-------
12
III. AUDIT RESULTS
A. Timesharing Budgeting and Billing
1 . Audit Objectives for Timesharing Budgeting and Billi
EPA provides funds for STORET timesharing costs to states and inter-
state commissions in order to encourage water pollution control by supporting
the entry of data into the system and its use for water quality monitoring and
analysis. These funds are first budgeted at EPA headquarters and then allo-
cated to EPA Regional Offices, which in turn divide their allocations among
their states. To encourage accountability and control, each state is allocated
a given amount of funds and can be required to reimburse EPA for any use of
computer services in excess of its ceiling. In addition, many other Federal
organizations (such as the U.S. Forest Service, the Army Corps of Engineers,
and the Department of Energy) use STORET through Interagency Agreements
(LAGs). Under an IAG, EPA distributes invoices and receives payments for its
costs of providing services.
Our evaluation focused on the process by which funds are allocated to
states and on the EPA's "billing administration" function (e.g., the timeliness
ป
and accuracy of invoices and cost reports, financial control procedures, and
invoicing and, collection procedures). Overall, this part of the audit concen-
trated on STORET users outside of EPA.
The rest of this section contains our findings, conclusions, and recommen-
dations for timesharing budgeting and billing .
2. Timesharing Budgeting
a . A udit Findings for Timesharing Budgeting
o Before fiscal 1976, state users were not charged for the
use of STORET. To encourage states to begin using
-------
13
- the system, EPA would provide training to potential users
and loan a terminal to a state on a temporary basis.
States that decided to use STORET permanently were required
to provide their own equipment and were. given a regular
computer account number. Use of the system was essentially
open-ended.
o Beginning with fiscal 1976, EPA instituted a slightly dif-
ferent set of policies concerning states' use of STORET.
Those, policies are basically still in effect. In a June,
1975, memorandum from EPA's Assistant Administrators for
Water and Hazardous Materials and for Planning and Manage-
ment to EPA regions and STORET points of contact (the
"Agee/Alm memo"), EPA reaffirmed its commitment to making
STORET available to states for valid water quality uses.
The memo also required regions to define a timesharing
budget ceiling for each state receiving an allocation of
funds for 'STQRET use and to identify how each state would
pay for computer services used in excess 'of its ceiling. A
state- receiving STORET allocations does not actually re-
ceive money from EPA. Instead, the financial balance of
its timeshare account is credited by the amount of the al-
locations; the balance is reduced as computer services are
used under that account. Funds not used in one year cannot
be carried over into the next year.
o .Exhibit III-l depicts the way that hands for state use of
STORET have been'allocated among EPA's regions over time.
As the exhibit illustrates, the absolute and relative al-
locations to many regions have remained -fairly stable, al-
though a few regions have experienced significant changes.
The total amount allocated to states was reduced, for fiscal
.1982 and again for fiscal 1983. Fiscal 1983 allocations
-are expected to total $480 ,000 .
-------
14
'^
~3
O
00
co .=r
ua cs)
PS, CD
LT.
m
La oo
Cs4 La
oo ua .3- ua
.t ,3- OO -=T
10
ua csj oo cn
ha <i csi >
O
CO
g
CSI
en
cn
LfJ
CxJ
u
(Q
3
OJ
OJ
4-ป CSI
o
en (
-------
15
0 Responsibility for allocating funds to states is divided
among several parts of EPA as follows:
- The total amount of STORET timesharing funds to be allo-
cated to states is determined in EPA's overall budgeting
process. Accordingly, it is part of a budget request
that is reviewed in the Office of Water, by EPA's budget
personnel, and by the Office of Management, and Budget.
Final responsibility for setting . the total timesharing
funds to be allocated to states for use of STORET lies
with. EPA's Assistant Administrator for Water.
- Once the total amount of funds to be allocated' to states
is determined, it is allocated among EPA's ten regional
offices by the Data Processing and User Assistance
Branch of the Monitoring and Data Support Division. The
allocation, for fiscal 1982 was the first performed' by
DP/UA; previous allocations .were made byMLDSD. '
Once a region has received its "allocation of timesharing-
funds for state ' use of STORET,' it divides its total
among its states- This function is usually performed by
the. Regional ADP Coordinator, often with assistance from
. . the Regional STORET Point of Contact. .
o Once allocated to the regions, funds earmarked for state
use .'of STORET are intended to be passed directly on to the
. region's states -and not diverted to other users. This
policy is defined by the A gee/Aim memo, although state
. STORET timesharing funds are technically under the full
control of the regions. Each state that uses STORET is
supposed to sign an agreement with its region that de-
scribes the way the; system will be used. -States are ex-
.,- pected -to use their STORET allocation only for valid
water quality related functions. There is no rigorous
-------
16
enforcement of how STORET allocations are used, although
there is no evidence of abuse by states.
o The primary factor used in allocating states' STORET time-
sharing funds for the upcoming fiscal year is actual spend-
ing in the current fiscal year. DP/UA receives budget re-
quests from EPA regions for timesharing funds, but uses
current year spending as the primary determinant in allo-
cating funds. Similarly, the state and EPA regions that we
interviewed said that states are usually contacted by their
EPA region to discuss their allocation, but that actual use
of the current allocation is the major factor that influ-
ences whether that allocation will increase or decrease.
The most common shift in funding allocations is to reduce
the budget of a state or region that is spending signifi-
cantly less than its ceiling and to increase the allocation
of a state or region that is actually using its full allo-
cation and seems able to make good use of additional funds.
The states that we interviewed seemed reasonably satisfied
with the budgeting process.
*
The existence of a ceiling encourages efficient use of
STORET by those states that tend to use all or most of
their timesharing allocations. DP/UA also demonstrates
and encourages cost-effective use of STORET through its
training and user assistance activities. There appears to
be no easy, low-cost way of providing additional incentives
for cost-effective system use by states through the
existing budgeting procedures.
o States usually are notified of their allocations in Septem-
ber, October, or November of each "year, which is adequate
for their planning purposes under current conditions. As
long as their allocations do not change- dramatically,
states have no difficulties in planning their use of
-------
17
STORET, However, if allocations were to change more
significantly (e-.g., 20% or more), state users expressed a
desire to receive earlier notification of .their timesharing '
budgets for STORET so that they could take effective action
to adjust their use of the system or to obtain additional
funding.
o The total timesharing budget for all states, combined has
never been exceeded. In fiscal -1982, 33,000 was shifted
from Region IV to Region I during the fiscal year to sup-
port additional use in states in Region I. In only two
. ' cases have-states actually reimbursed EPA for computer ser-
vices used above their STORET ceilings. When a state ex-
pects to exceed its ceiling, it typically contacts its
regional office, which is usually able to shift funds from.
a .state that is using substantially less than its ceiling
to cover for the. excess use. . No state makes regular ar-
' rangments-to use STORET for more 'than its ceiling.
b -Conclusions and Recommendations for Timesharing Budgeting _
* ป
' o Taken as a whole, EPA's procedures for allocating STORET
; time-sharing funds to regions and states are sound and ap-
. propriate; The state representatives that we interviewed
had no major complaints about the budgeting process'. The
budget allocation procedures are simple, efficient, and
.-. flexible, and are. largely performed by personnel with a.
good knowledge of how states are actually. using STORET;
' The pattern of allocation to states and regions has changed
at the margin to meet shifting needs and requirements; at
the same . time, allocations are usually fairly stable and
.'-.' the- pattern of funding' now being used is still generally
. 'similar- to -that, established for fiscal 1976, the first year
in which. EPA imposed ceilings on states' use of STORET.
Although the-annual' budgeting process could be made more.
formal and elaborate (and more expensive to administer) , it
-------
18
is questionable whether such changes would increase signif-
icantly the total benefits accruing to EPA and the states
from STORET.
o The fiscal 1976 allocation of STORET timesharing funds to
states was based on the volume of system use by states in
previous years. Because states had not until then been
charged for or restricted in their use of STORET, the orig-
inal budget allocations would be expected to have allowed
existing users to obtain maximum benefits from STORET use.
However, since the budget allocations to states for STORET
timesharing have changed fairly slowly, the current pattern
of funding may not be optimal. Accordingly, we' recommend
that the Data Processing and User Assistance Branch vali-
date the current allocation of state STORET timesharing
funds to assess its adequacy in terms of factors such as
states' needs, benefits to EPA (such as entrv into the svs-
tern of water-quality data), and equity.
In conducting such a study, it should be recognized that
each state consumes an average of only about 310,000 of
timesharing resources for STORET each year. Therefore,
DP/UA's analysis should be designed to consume a limited
number of resources. Factors that might be considered for
each state include the amount of data entered into STORET,
the number and types of retrievals and analyses performed,
the nature of its water quality planning and analysis pro-
gram, the role of the state's region in helping it use
STORET, the state's population and industrial make-up,
and the extent of its navigable waterways. Assuming that
information could be. collected from easily available
sources, an initial review could probably be conducted for
about three to six weeks of effort. The methodology and
results of the study should be reviewed by the Director of
the Monitoring and Data Support Division (MDSD). Based
on the results of the analysis, DP/UA and MDSD could
-------
19
identify any desirable shifts in the current pattern of
budget allocations or (if necessary) define areas for fur-
ther investigation. As a side benefit, the study might
help shed light on the question of whether the total
STORET timesharing budget for states is set at an appro-
priate level. Given the current budgeting procedures and
levels, such an evaluation should probably be conducted by
DP/UA every two to four years.
3. Cost Reporting and Billing Administration
a. Audit Findings for Cost Reporting and Billing Administration
The reports generated by NCC for distribution to system
account holders are more than adequate to meet the.typical
needs of account holders for' information about the costs
incurred for NCC computer services. Reports are generated
'by the Timesharing Service Management System (TSSMS.), which
collects and processes information . from a variety of
single-purpose usage accounting systems (including the tape
management system, the IBM and UNIVAC computer accounting
systems, a system that tracks use of a on-line storage, and
so on). STORET users also receive reports from the IBM
accounting system directly. Exhibit III-2 describes the
reports produced by TSSMS and the IBM accounting system on
a regular basis for distribution to system account holders.
Most (but not all) of the users that we interviewed were
satisfied with the level of detail on their reports. Addi-
tional information on the costs of each terminal session or
batch job is available from NCC, but is seldom requested
by users. TSSMS currently provides no information on the
budgets associated with each individual account, but it is
being enhanced to include actual-to-budget comparisons; the
new reports are- scheduled to be available in November of
1982. ' .'. . . .
-------
EXHIBIT III-2
SELECTED SYSTEM COST REPORTS AVAILABLE FOR REGULAR DISTRIBUTION
20
REPORT NUMBER AND/OR NAME
REPORT DESCRIPTION
Account Information
Report
EPA Computer Costs by
Supplier, Organization,
Decision Unit, and
Account
IBM System Monthly
Statement
IBM System Fiscal Year
To Date Statement
IBM 370/168 Account Job
Summary
IBM 370/168 Userid Job
Cost Summary
Lists the name, address, telephone number,-and other data
for the manager and all authorized users of an EPA computer
system account. Each user's official user-id is given, as
is information on the status of the account as a whole.
By account, displays spending in the current month,
average monthly spending for the year to date, and total
spending for the year to date.
For each account, presents cost incurred in the previous
month in total and subdivided by major cost category .
(e.g., public on-line disk, tapes, etc.).
Identical to the monthly statement, but spending is shown
for the year-to-date rather than for a single month.
Displays the number of jobs or sessions in batch, TSO,
and WYLBUR for an account, showing resources used and the
corresponding charges for major categories (e.g., CPU
time, connect time).
Identical to the account job summary, but breaks usage
down by userid.
-------
21
o The users that -we interviewed had never had any problems
with the' accuracy of their system cost reports, although
none performs a careful check. Any complaints or questions
about cost accounting information are investigated at NCC
and can usually be resolved within a day. To support such
investigations, a wide array of back-up reports are avail-
able from TSSMS and TSSMS's feeder systems. These reports
provide, in essence, an audit trail of each component of
user costs so that any problems can be accurately resolved.
If a legitimate error is discovered, NCC corrects its re-
cords either by re-executing its cost accounting programs
for the periods in .question or by generating appropriate
debits or credits to be applied to system accounts in the
future. . .
o A number of other Federal agencies use STORET through
.Interagency Agreements (IAGs). The largest of these users
are the U.S. Forest Service and the Army Corps of Engineers
According to TSSMS reports, use of NCC .services 'under
STORET IAGs totalled about S4QQ.QQO in fiscal 1980 and
about 5420,000 in fiscal 1981. An estimate of the total
amount of IAG timesharing activity is included in NCC's
budget, and. NCC uses those budgeted funds for activities
such as maintenance and operations. As a matter of sound
financial management, NCC spends- funds included in its
budget as a result of lAGs only after costs have actually .
. been incurred in providing . services to IAG system
accounts. . , '' . '
o 'Exhibit III-3 depicts EPA's formal procedures covering the
flow of computer system cost reports and billing informa~
tion. Those procedures are described below. . .
Cost reports are produced at-NCC by TSSMS every month
. for every computer; system account. 'NCC uses the system
account as the primary unit of cost recordkeeping,
-------
22
a:
2
o
CJ
<
2:
in
O
LU
ro
i
ca ^
= ฃ
rTi ci
a
yj
O
o en
S. L, Cl O ~
ni .I t_ _i '"*
X
u w
i
"x
_!
V5
as
iivi S
i-i o ^
5 ฑ*
Vป t/> ฃฃ
"" ฃ2
X
ฃ
9
-_j-
w
53
gto <
S2 S
,i/1 ซ ป
3 <ฃ h
^Q:&: <
pป uj UJ r*>j
t_j. a- *
^-ast:
sslii
en *J crt
ฃป- II
T3 4 4J W*
IJJ ^3
2 ^K C
ซ V/1 4 *.*-ซ.
wt . >/ซ V)
2 ?S "Si
o o> -e **
^s-s ***
LJ W cc *n
o wi *"
ls< .
Uj ^ ^
s^l
OH-
-------
23
although a variety of data is also available by user-id.
There can . be several user-ids within a single system
account.
Detailed cost reports by system account are distributed
to EPA's ADP Coordinators. ADP Coordinators-have been
designated for each regional office and for all major
headquarters offices, and' have oversight responsibili-
ties for all computer system accounts within their orga-
nizations. According to formal EPA procedures, each ADP
Coordinator should review the cost, reports received and
distribute them to the appropriate system account hold-
ers. However, as described in more detail below, ADP
Coordinators do not all distribute these reports as reg-
ularly as they should. Cost reports for IAG accounts
and for states using STORET are treated in the same
manner as normal EPA accounts. Cost reports for state
users of STORET are sent to the ADP' Coordinator in the
relevant regional offices; reports for IAG accounts are
distributed through the regional ADP Coordinator (if the
IAG holder is a field office of another Federal agency\
or through MIDSD in headquarters or at NCC (if the IAG
is executed at the headquarters level with the other
agency).
. Cost reports by system account are also received by
MIDSD's Information Resources Management Branch (IRMB).
For each system account, ,IRMB compares actual costs to
budget in order to identify situations in which spending
is likely to exceed the corresponding timesharing bud-
get. The comparison is now done by hand, but will be
performed automatically when TSSMS's enhancements become
available. If IRMB. judges that use of a system account
seems to be on its way-to exceeding budgeted amounts,
IRMB contacts the account holder's ADP Coordinator, who
is responsible for . investigating the situation in more
-------
24
detail and working with IRMB to resolve any problems.
Beginning in fiscal 1983, MIDSD's stated policy will be
to terminate the system account of any account holders
that exceed their timesharing budgets and are unable to
provide additional funds to cover continued system use.
- IRMB also receives cost reports for ail IAG system ac-
counts. Based on those cost reports, IRMB completes
copies of EPA Form 2550-3 (Report of Reimbursable Ser-
vices Rendered) and forwards them to EPA's Accounting
Office in Cincinnati, where bills are prepared and
mailed. Most Interagency Agreements call for quarterly
billing. At Cincinnati, EPA prepares and distributes
the bills themselves and actively follows a set of pro-
cedures designed to ensure that all amounts due are ac-
tually received. Reminder bills are distributed for un-
paid invoices after 30 days, 60 days, and 90 days, and
the IAG holder is contacted by telephone if an invoice
has not been paid in 90 days. The Cincinnati Accounting
Office performs invoicing and collection activities for
approximately 90% of all lAGs. Except for Region V and
VII, lAGs executed by EPA regional offices are billed by
the region.
- NCC uses certain summary reports from TSSMS to analyze
the overall pattern of timesharing spending within EPA.
NCC's review is performed every quarter for large or-
ganizational groupings, and is intended to help top man-
agement in MIDSD and EPA as a whole understand and, if
appropriate, take action on the Agency's use of time-
sharing resources. This analysis provides a useful com-
pliment to the more detailed reviews performed by
. IRMB.
o Most of the STORET users that we interviewed said that they
do not receive cost reports on a monthly basis. For states
-------
25
using STORET, lack of current cost reports hampers finan-
cial control and is a potentially serious problem since EPA
could require payment for timesharing resources used in
excess of budget ceilings. IAG holders who fail to receive
cost reports regularly find it difficult to monitor the
current level of their spending. The ADP Coordinators that
we interviewed consider the distribution of computer system
cost reports as a time-consuming task of low priority rela-
tive to their other assignments, particularly for system
account holders within EPA who are within budget or as long
as there are no meaningful sanctions for exceeding budgeted
amounts.
o Of the two IAG holders we interviewed, one was not receiv-
ing invoices regularly. Furthermore, personnel in the Data
Processing and User Assistance Branch and the Accounting
Office in Cincinnati stated that they believed that IAG ac-
counts are not always billed as regularly or frequently as
required. .Any "problems in distributing' invoices to IAG
holders could result in a reduction in the funds actually
received by EPA and delays in receiving those funds that
are paid. Several initiatives already begun within EPA may
help to address this problem. For example, the Accounting
Office in Cincinnati is considering a proposal that it be
made responsible for all IAG billing. In addition, en-
hancements being made to TSSMS may improve EPA's ability
to bill for the use" of IAG system accounts.' However, these
efforts do not seem to be part of an overall strategy to
eliminate the problems encountered by many STORET IAG
holders. ....
o By typing a sequence of WYLBUR commands, users can obtain a
summary of each month's timesharing spending for the cur-
rent year by account number. However, personnel at NCC
-------
26
said that users often, enter the necessary commands impro-
perly and, as a result, obtain erroneous or confusing out-
puts. DP/UA has implemented a similar feature under T5O
that requires users to enter only a single command. The
command ("YTD") is documented in an on-line data set main-
tained by DP/UA but available to all users. These on-line
cost reporting features are useful, but are not intended to
meet the full needs of system account holders for detailed
cost information broken down by resources used or by user-
id.
b. Conclusions and Recommendations for Cost Reporting and Billing
Administration
o Our evaluation indicates that many STORET users do not re-
ceive cost reports as regularly as called for by EPA's pro-
cedures. Similarly, some IAG accounts may not be billed as
promptly or systematically as required. It is important
that these problems be eliminated, particularly if they are
not confined to STORET users. It should be emphasized,
however, that these problems 'occur in areas not unique to
STORET. Furthermore, the results of our analysis may not
be fully representative of the experience of other computer
users and systems at EPA. MIDSD, in conjunction with other
affected EPA offices (such as the Accounting Office in
Cincinnati) , should investigate these problems to define
more precisely their nature, scooe, and importance. Based
on the results of that investigation, alternative solutions
can be identified, evaluated, and implemented. It should
be possible to begin implementation of the selected alter-
native by the middle of fiscal 1983. Any actions taken
should probably be coordinated with the efforts already
begun. It is not possible to present precise final recom-
mendations on the basis 'of _the current study. However, it
seems likely that the problems can be remedied fairly easi-
ly at a relatively low cost to EPA.
-------
27
Because of their value to users,, NCC should retain the
existing on-line cost reporting capabilities and, if rea-
sonable, Investigate how they can be further publicized
and/or made easier to use. For example, the capabilities
of SUPERWYLBUR might be used to simplify the procedures
that allow users to display summary spending data and DP/
UA's TSO capability might be incorporated into the NCC
users manual or publicized through a logon message.
-------
28
B. System Operations and Management
1. Audit Objectives for System Operation and Management
STORET is a system which supports the maintenance and retrieval of
water quality information on a nationwide basis. The use of this system is
voluntary. However, in order for STORET to be most effective, it is important
that all those involved in water quality analysis supply timely and accurate
data to the system.
In addition to insuring the reliability, integrity and availability of
STORET data, the operations and management of STORET has a very important im-
pact on the attitudes of current and potential system users. Effective opera-
tions and system management will encourage users to rely on STORET and thus
enhance the system's value for all users. On the other hand, continuous STORET
operating problems will discourage its use and eventually render the system
less effective.
The objective of this portion of the audit is to review the procedures
and controls that insure smooth operation of the system. Although a portion of
this review included operations at NCC, the extent of the data center audit was
limited only to those aspects which affect STORET and did not address the
broader aspects of NCC operations which affect all systems. The analysis
focuses on the following attributes:
o The reliability of system operations and its consistency with
user requirements.
o The capability through system backup and recovery procedures to
restore the system with minimum user impact should problems
arise.
o The quality of system documentation and procedures to support
maintenance and enhancements.
-------
29
. The rest of this section contains our findings, conclusions, and recom-
mendations in the area of STORET operations and management.
2. System Availability and Responsiveness
a. Audit Findings for System Availability and Responsiveness ,.
o Telecommunications availability is adequate to meet user
requirements. The following modes are available for STORET
users to logon to the NCC system;
- Direct RJE line. High volume users of STORET and other
EPA systems have a direct dedicated line to NCC. There
are currently 64 direct line modems and there is capa-
' * i
city for expansion.
. . , "- The Tymnet Network. Many users have dial-up access to .
: , ' ' . this nationwide -network through a local call. -There are
, - approximately 256 ports available at NCC for this net- . .
work. - . '
- WATS service. Between 30 and 40 ports are available for
direct toll-free dial-up to NCC. These lines are gen-
erally reserved for those users who do not have an RJE
line or access to Tymnet. However others may use this
service if they have problems with their normal mode of
communication. .
, '."'- Statistical Multiplexer' at EPA headquarters.' The 84
--.'' ' 'ports currently available to EPA headquarters ; users . are
frequently almost all in use. Two steps are being taken
; to remedy . this situation. A direct line will be in- .
".' stalled by November,, 1982 for DP/UA which will initially
'. ;- handle 6 terminals and. can be expanded to 16 terminals.
. - .In addition the number'of total ports available-through "
-------
30
the statistical multiplexer will be increased before the
end of 1982.
If a user experiences a problem with any of these modes or
communication, he can contact the Telecommunications Main-
tenance Group at NCC directly. This group has the person-
nel and equipment to diagnose reported problems and identi-
fy alternate -communications modes while problems are being
resolved. The availability of Telecommunication Main-
tenance (shown in Exhibit III-4) is consistent with the NCC
computer operations schedule.
o System availability for NCC and in turn for STQRET has
averaged 99.2% for Fiscal Year 1982. STORET users depend
on the availability of the NCC computer to process jobs and
on the TSO/WYLBUR services -to enter data and submit jobs.
Exhibit III-4 presents the NCC'hours of operation and Ex-
hibit III-5 presents actual availability statistics. As
these exhibits show, the NCC services have been readily
available
o Job turnaround times have been steadily increasing but a
capacity expansion plan is being implemented to remedy
this. The turnaround time for every class and job priority
has more than doubled since April, 1982. This is a func-
tion of the overall usage of the NCC computer and not of
STORET. This situation has been recognized by NCC manage-
ment and the installation of an Amdahl V6 is underway to
increase computer capacity by 15-30% by the end of 1982.
The impact of increased turnaround time on STORET users
appears to be more a matter of convenience than a major
deterent from using the system.
o The response time for T5O is well within the limits of a
good on-line system. STORET users are officially encour-
aged to use TSO as the facility for logging on to the NCC
-------
31
O)
OJ
(O
T3
3
C/1
f^
ซ
>
fO
C
3
unavall
o
cu
0
(J
a
O)
(/i
0
^ซ
u
unavall
o
01
in
Q
U
*
i^
i
^^
ซ
><
tu
>
(Q
-o
4->
ซ
oo
o
1
r~
Cฃ >i
III (Q
Q_ . -O
O ' <-
s_
u. ' u_
0 ,
CO
OS >i
ZD ' ^
1 '. 1
i
z
S E
. 0 0
n PO
* ซ*
i i
< ' <
r-> r-.
'
s: z
-iH
o>
j;
-5
f
S .
Z. 5
cs: a
va r-
i-
tu
*->
3
CL
C
O
o '
o .
in
0)
u
>
S- '
.
Qฃ
3
CD
_J
>- '
3
O
0)
4->
C'
Crt
p-
a
c
o
+j
CT
C
r
^:
to
rtment
C
O
<_>
o
-------
32
to
CQ
31
x
03
UJ (/>
z
^
a.
o
h-
O
3
i
a.
>.
ca C
i x
ป-*
uj a.
1/1
UJ
ol i
>- 1 . UJ
> CQ
r
3Z
ง
to
u ^
00
>-* X
o
o =>
os o
UJ
1- i
>- i UJ
> CQ
>
3EI _JOฃ
oc > ca o
a. ซ x
UJ > CO
ZU.Q:
a: x 0
a- co x
g
o
CO
ON
m
*
0
oo
in
m
m
ON
vO
O
o
o
CM
t<
ao
u
o
CN
ON
ON
ON
00
CN
rป.
vฃ
00
CN
cn
ON
ON
cn
CO
CM
CN
o
cn
CN
CN
CO
i-
m
CN
cn
m
00
in
*
ON
ON
VO
00
o
CN
CN
QO
a.
ON
ON
ON
00
O
o
ON
-------
33
computer and executing STORET jobs. As can be seen in Ex-
hibit III-6, average TSO response time between 9:00 and
10:00 AM has varied between 2 and 4 seconds during 1981 and
1982. This is within the generally accepted limit of 3-5
seconds, at which point user interest and satisfaction
begin to decline.
b. Conclusions and Recommendations for System Availability and Re-
sponsiveness
o Overall STORET availability is above average relative to
user perceptions and requirements. DP/UA is supporting
STORET in such a way that STORET's availability does not
deviate from the availability of the NCC computer. In
addition* NCC maintains a very high level of system
availability and appears to identify and respond to
long-run problems, as evidenced by the upgrading of
telecommunications with EPA headquarters and the machine
. . capacity expansion projects now in progress.
3. STORET Operations
a. Audit Findings for STORET Operations .
o STORET data input, editing, data retrieval and analysis
pjrograms are run by the STORET users. These programs
have been relatively stable over the last few years, and
- users have encountered very few. bugs in attempting to
'run them. The user, documentation, discussed in Section
III..C of this report, provides thorough and sufficient
operations information. If the user does encounter
problems running a STORET program his -first point of
contact should be his regional point of contact; how-
.' ever, most users call DP/UA whether the problem is a
direct STORET problem or an NCC operations problem.
If necessary, the STORET User Assistance Section will
-------
34
CD
X
a
CO
ซ!>!
ซg
UJ
cu
oujooonj
zo=>conj
o 01-0001
u.uioaconj
rxrzooru
-53-JCO
-53200^
-------
35
coordinate the problem resolution through the appropri-
ate NCC user ' assistance personnel. Periodically
throughout each day the User Assistance Section moni-
tors the backlog of STORET jobs awaiting processing in
order to anticipate potential job execution problems
and be prepared to respond "to user inquiries in a time-
ly fashion.
o Direct updates to the STORET data base are performed on
a weekly basis by STORฃT Data Processing and User As-
sistance . This is the only major job not executed by
users. The job (approximately 30 programs in one job
stream) is submitted for processing each Friday for
.execution overnight. The job steps are then reviewed
on Saturday to verify successful completion. If a.
problem occurs during the execution of this job, every
effort is 'made to correct the problem and complete.
execution by'Monday .morning. There-is no known in-
.' .stance in-the past year when this schedule has not been
maintained. . .
o . Foreign tapes are controlled bv the NCC I/O Control
Group. The procedures for logging foreign tapes into
and out of NCC are well documented in the NCC Users
Guide. Strict control is maintained over foreign input
tapes to insure that all appropriate identification
information is obtained- and. an NCC tape number is as-
signed. The normal procedure then; calls for the for-
eign tape to be copied to a Tape Management. System
- (TMS) tape which can then be automatically tracked by
TMS. At this point the foreign tape is mailed back to
,: the user. ' Output tapes requested by the user must be
copied., from- a TMS tape to an assigned foreign tape
which' is then sent to the user. Mailings of foreign
f
" tapes from NCC to the user are done on a daily.basis as
-part of-a standard output distribution procedure. ..
-------
36
o Hardcopy STORET output may be printed at the user's
site, at NCC, or__at__E_PA's Washington Computer Center.
If the listing is short or the user has his own RJE
printer, printing at the user's site is usually quicker
and easier. Listings printed at NCC and WCC are sorted
by user-id and mailed on a daily basis as part of the
standard I/O Control procedures. The mailing labels
for these listings are produced automatically based on
the job's user-id. It usually takes about a week for
the user to receive these listings, which are distri-
buted by mail.
o Most STORET plots are printed bv STORET User Assistance
in Washington and mailed to the user. Users have ex-
perienced delays of 1-3 weeks in receiving plotted out-
gut. A review of the procedures surrounding the pro-
duction of STORZT plots is summarized by the following
points:
- The plot submitted by the user will usually execute
and be despooled to a tape at WCC by the NCC staff
by 6:00 am of the following morning. In addition to
the tape, a hardcopy listing is produced at WCC that
includes a mailing label for each plot.
- STORET user assistance receives the tape by 8:00 am
and begins printing the plots shortly ' thereafter.
Occasionally the tape is corrupt and must be re-
covered from a backup tape at NCC. Recovery only
takes 20 minutes.
- By the end of the day that a plot is printed, DP/UA
will process the plots by inserting them into mail-
ing tubes, attaching the mailing labels and submit-
ting them to the EPA mailroom.
-------
37
- The STORET User Assistance Section maintains a de-
tailed log of this process which identifies for each
plot the date the job was run,' the date the plot was
printed, and the date the plot was mailed.
Based on these findings, the delay that users experi-
ence in receiving printed plots appears to be related
to the procedures in the EPA headquarters mailroom and/
or the U.S. Post Office. Some users have recently pur--
chased their own plotting equipment, in part to elimi-
nate these delays.
b. Conclusions and Recommendations for STORET Operations
6 STORET job submission and execution is well controlled
and understood by users. The operation-of STORET jobs
is reliable .and the schedule for all -aspects of ".the
system- (except the data base update) is under the
users' control. 'The weekly -schedule 'for updating the
' . STORET data base appears to be appropriate and is re-
lied upon as part of the users' normal routine. The
procedures for handling any STORET operations problem
are adequately provided through the STORET User Assis-
tance Section which will coordinate with NCC. user as-
sistance if necessary.
o EPA's' Office of Administration should determine how
'best to speed the delivery of STORET plots to the user.
. ' A delay of 1-3 weeks before receiving a plot is not
satisfactory to the user. Further investigation of the
. procedures for handling' plots after' the STORET User
Assistance. Section submits them to the EPA mailroom. is
necessary. Alternative methods for delivering plots
to the users should be-reviewed again if current, proce-
dures cannot provide more prompt delivery.
-------
38
STORET Security. Backup and Recovery
a. Audit Findings for STORET Security, Backup and Recovery
o First level security is provided by NCC's User-Id and Pass-
word logon procedures. In order to access NCC's system, a
user must open an account and be assigned a User-Id and
Password. The passwords are changed every 90 days, in an
attempt to prevent unauthorized use of the system.
o Second level security for STORET is provided by RACF, an
IBM proprietary file protection system used by DP/UA. All
All STORET programs and TSO CLISTS are protected from
modification or deletion by STORET users through the
security features of RACF.
o The STORET data base and update files are backed .up_on_a_
weekly basis by the STORET Data Processing and User As~
sistance Branch. As part of the weekly data base update
job stream, a complete backup of the STORET data base is
performed. In addition, tape files are produced containing
the data base update transactions submitted by the users
and two intermediate update transaction files. These files
are retained for 1 year and provide the capability to re-
store the data base to a known point and recover forward
without rerunning the entire job.
o The process for modifying STORET programs is well con-
trolled and includes a backup of the program source and
load libraries. These backups are performed by DP/UA prior
to installing any program changes into the production li-
brary. As a result, recovery to the last stable version of
the system can be performed should a problem be encountered
with the most recent modifications.
-------
39
o NCC backup procedures include certain standard STORET
disks. These backups are performed according to the
following schedule:
- Daily. All incremental data set changes (32 day reten-
tion)
- Weekly. Full pack backup of all specified packs (5 week
retention)
- Monthly. Full pack backup of all specified packs (IS
month retention)
- Users may request special retention of specific backups
for up to 7 years
o The Data Processing and User Assistance Branch is develop-
ing a data base verification program. Once this 'program is
completed, it will allow DP/UA to verify 'the data ba'se on
Monday mornings before it is accessed by the users. If
..data base problems-are identified, recovery procedures can'
then be started with minimum impact on the user.
ฐ All backup tapes are controlled under the Tape Management
System. If recovery is required the latest backup tapes
are available from the TMS tape log. A. hardcopy listing of
this log is produced monthly. In addition, the log is
maintained in an on-line accessable data set.
b. Conclusions and Recommendations for STORET Security. Backup and
Recovery ' , .
o The level of security is appropriate for a nationwide data
base system such as STORET. The STORET data base is
maintained by STORET users for their own use and to support
other water- quality analysis requirements. Therefore the
' 'logon .security is.'appropriate to support the general level
-------
40
of data availability desired. The further use of RACF ap-
propriately protects against the inadvertaint destruction
of the STORET data access routines and insures their inte-
grity to all users.
o The level of STORET system backup is more than adequate to
insure that there is minimum impact on the user from system
problems* The schedule for the weekly update program
leaves sufficient time to recover should a problem be en-
countered during data base maintenance. In addition, the
data base can be recovered from. a total loss through the
restoration of a backup version and the reapplication of
all update transactions with minimum processing effort.
5. STORET System Documentation
a. Audit Findings for STORET System Documentation
o The program documentation, supplemented bv the commented
source code, is more than sufficient to support any main-
tenance requirements. The procedure for documenting pro-
gram changes is to comment the source code heavily for
minor changes and to rewrite the appropriate portion of the
documentation for major changes. The documentation main-
tained for each program includes:
- A narrative description of the program's purpose
- An overview flowchart of the program's files
- A narrative describing each subroutine
- A data element dictionary
- An heirarchical function diagram
- A mapping of program line numbers to functions
- Flowcharts for any complex logic
Detail program notes as necessary
-------
41
o A library of program documentation is maintained and the
latest production version of program source code is avail-
able through a security protected on-line data set. A sin-
gle set of book cases in the office of'the STORET Data Pro-
cessing and User Assistance Branch contains the bound vol-
umes of program documentation. Anyone removing a volume is
expected to sign it out. Listings of the latest version of
program source "code may be produced from the on-line data
set CWT.SOURCE. 'This data set is secured against changes
through RACF and backups are taken each time it is updated.
o Operations documentation does not exist for the non-user
exec ut ed STORฃT jaja base update pro grams. The STORET
User Handbooks document the operation of all data entry,
retrieval, and analysis programs. The weekly data base up-
date and backup jobs, however, are run by DP/UA and no
documentation exists on how to run the jobs or how to veri-
fy that they ran successfully. Currently two members of
the staff are very familiar with how to run these jobs and
one other member of the staff could run the jobs after one
day's preparation. This is a potentially vulnerable, area
of the system.
b. Conclusions and Recommandations for STORET Systems Documentation
o The program documentation is sufficient to support system
maintenance. However DP/JJA should investigate procedures
to insure that the documentation remains consistent with
the programs. Currently each programmer is responsible for
" -commenting the code and updating the documentation after he
modifies a program. .In' the context of the size, of .the pro~
gramming 'staff and the stability of STORET, this informali-
ty with respect to program documentation does not have any
severe impact on the system. .However, should the environ-
ment .change through staff turnover or .a- higher level of
program, changes, more formal procedures should be intro-
-------
42
duced to insure that all program documentation is kept-up-
to-date and consistent.
DP/UA should prepare operations documentation for the
STORฃT data base update programs. The STORET data base
represents millions of dollars worth of water quality ob-
servations. The lack of documentation on how to run the
jobs which maintain and backup this data base is a poten-
tially serious problem should none of the three people
familiar with this job be available. At a minimum, the
documentation should identify job flow and sequence for the
programs, how to verify the successful execution of each
job step, and the procedures for recovering should a job
step not execute successfully. In addition, . a list of
known potential or typical problems that might be encoun-
tered for each job step and how to resolve them would be
beneficial.
6. STORET Maintenance
a. Audit Findings for STORET Maintenance
o The responsibilities for maintaining STORET are divided
between the two sections of the Data Processing and User
Assistance Branch; the User Assistance Section and the
Information Access Section. User Assistance interacts with
the users who identify a problem, gathering a description
of the problem and as much supporting information as possi-
ble including listings, jobs, files, etc. The User Assis-
tance Section then takes the problem to member of the In-
formation Access Section, who will investigate, modify the
programs, and test as necessary. Additional information
required by the Information Access Section is usually re-
quested from the user through User Assistance. The user is
notified of the problem resolution by the User Assistance
Section. This division of DP/UA into two sections is very
-------
43
effective in isolating those involved with actual program-
. ming and debugging from user interruption.
o There is no significant backlog of major system maintenance
tasks. The STORฃT system has been relatively stable over
the past few years. As a result, most system problems have
already been resolved and relatively few new problems are
encountered. In this environment, the assignment of prob-
lems to the staff and the tracking of problem resolution is
done informally. There is no cause to alter this procedure
unless major enhancements are made to the system.
o A formal procedure exists for modifying a STORET program
and moving it into production. The programmer, responsible
for modifying- the program will take a copy of it from CWT.
SOURCE into his own library, modify it, compile it into a
test library, . and unit test it. If related to a user prob-
lem, that user will be requested to run his own. tests.
against the program from the test library. -Once user test-
'ing is complete, the' production library is . backed, up and
' -then the-modified program is moved into the production'sys-
tem. If any problems are encountered, the original produc-
tion.library is restored. .
o Periodic changes to'the system' and procedures at NCC are a
source of modifications to STORET. When these- changes are
defined, NCC communicates a warning to all users through an
on-Ene data set describing the change at least 30 days in
advance. If a user will be affected by a change, he has an
opportunity to 'contact NCC user, assistance' and, if it is
serious, possibly affect the- nature of the 'change and "its
schedule. DP/UA. handles all such communications with NCC
' for STORET and modifies the STORET system as necessary to
accommodate the changes. There have been past instances
when DP/UA indicated that the planning and implementation of
NCC changes could have been better coordinated.
-------
I
b. Conclusions and Recommendations for STORET Maintenance
o If the level of STORET problems and maintenance tasks in-
creases, a more formal mechanism should be developed by DP/
UA for tracking the status of problem resolution. One ap-
proach is to maintain a log of problems referred to the In-
formation Access Section for resolution and, on a regular
basis, identify which problems are actively being worked
on, who is responsible for resolution, and an estimated
completion date.
o Increased interaction between NCC and the STORET Data Pro-
cessing and User Assistance Branch could lessen the impact
of NCC systems changes. STORET users are among the largest
group of users of NCC. Although it would be inefficient to
incorporate users into the NCC's detailed planning process,
certain system changes that affect a majority of users can
be identified. If the STORET User Assistance Section
were to be notified of the nature of such changes, it might
be able to contribute information that would help NCC- de-
. fine the change and its schedule to have minimum impact on
a majority of users. In addition, STORET users would be
willing to provide testing for system changes during NCC
specified test periods to insure system reliability before
the changes are put into production.
7. STORET Enhancements
44
a. Audit Findings for STORET Enhancements
o There is not a significant backlog of STORET enhancements.
Certain enhancements currently in progress were requested
by the users during conversations with STORET User Assis-
tance either at. a training course or during a telephone in-
quiry. A number of enhancements have been requested by the
Director of EPA's Monitoring and Data Support Division.
-------
45
In addition, .members of the-Information Access Section have
identified certain program changes which could improve sys-
tem- efficiency and maintainability.
o Only an informal__mechanism exists for users- to communicate
requests for system enhancements. Prior to 1979, the annu-
al STORET Users Meeting served this purpose and gave the
users a chance to interact with one another to provide DP/-
UA with a prioritized list of the enhancements that they
would.find beneficial. Since 1979, no such formal mechan-
ism has existed. As a result two situations have evolved.
First, fewer user enhancement requests have been communi-
cated to the STORET staff. Secondly, many users have been
developing and . maintaining their own related routines
around the STORET system. This second situation means that
there are a number of- potentially useful -routines .to the
STORET .user community which are, hot .generally known or
available.
o Enhancement priorities are set through informal discussions
within the STORET Data Processing and ' User Assistance
Branch. These priorities are based on perceptions of use-
fulness, magnitude of the task, and -staff availability. At
the current level of enhancement requests, this mechanism
for setting priorities is effective. Once enhancements
. have been completed, DP/UA contacts by telephone any users
who were involved in. requesting the enhancement. Other.
users are' notified via an enhancement description in an on-
line data sets and by the- distribution of new documenta-
tion.
b, Conclusions and Recommendations for STORET Enhancements.
"o- DP/UA' should institute a procedure for user input into the
enhancement identification and, priority process. The- most
effective mechanism would-be to reintroduce the annual
-------
46
STORET Users Meeting. This meeting would provide a forum
for users to communicate with one another on how they use
the system and what improvements are desired. They could
. then work with the STORET staff to develop a consensus pri-
ority list of enhancements to be worked on over the next
year. Other effective mechanisms for accomplishing this
goal might include regional STORET user meetings, a peri-
odic STORET newsletter, and an annual user questionnaire.
o MDSD should evaluate the costs and benefits of having DP/
UA perform maintenance and documentation activities for
selected user developed enhancements. . A number of enhance-"
ments that are potentially useful to the user group in gen-
eral have been developed by users over the past few years.
The annual User Meeting would provide a good forum for
presenting the nature of these routines, identifying which
are generally useful, and adding them to the enhancement
list. The STORET staff could then gather the information
it needed from the developer, generalize the modules, add
them to the STORET libraries, and prepare the user docu-
mentation .
o If the backlog of enhancements increases, DP/UA should de-
velop a more formal mechanism for monitoring programmer
progress. Currently progress is informally monitored
through manager inquiry. The mechanism could be enhanced
with task plans identifying programmer assignments, level
of effort estimates and projected completion dates, as well
as weekly status reports.
-------
47
C. STORET User Support
1. Audit Objectives for STORET User Support
The objective of evaluating STORET user support, as expressed in the
analytical plan for phase two of the STORET audit, is to determine the overall
effectiveness of STORET user support as a combination of (1) STORET training;
(2) STORET user documentation; and (3) STORET user assistance. STORET is a
large, powerful, and flexible system with two unusual environmental character-
istics: use of STORET is voluntary, and the users of STORET are, on average,
less sophisticated in ADP than the users of other large systems. As stated in
the analytical plan, if STORET appears to users to be difficult to use, users
are free to employ other means of storing, retrieving, and analyzing water
quality data. Thus, effective user support is essential to assuring that users
will continue to use STORET, and thus in turn continue to provide EPA with a
base of data it could not otherwise afford to collect.
Our evaluation focused on examining-the procedures and. controls used in
support to users. We asked questions and gathered documentation from
both the Data Processing and User Assistance Branch and from users in the
field.
This section presents findings and recommendations in three parts":' (1)
STORET training; (2) STORET user documentation; and (3) STORET user
assistance.
2.. STORET Training . - . , .
.a. Audit Findings for STORET Training
o STORET User Assistance Section's (SUAS) formal and informal
training 'procedures are very- goody within, "the limits of
available resources. Formal training - is provided by a
Basic Seminar and an Analytical Applications Seminar.
-------
48
The Basic Seminar includes the basics of a terminal command
language (TSO) and computer access procedures, as a service
to STORET users who otherwise would have no local access to
training in. these procedures. The one complaint users have
about the seminars is insufficient hands-on terminal time.
SUAS is limited to the facilities available at each train-
ing site.
o SUAS is f-unded and staffed to be able to conduct about
twelve Basic and three Analytical 'Applications Seminars per
year, on request. Users would like to have more training
than available resources permit.
Textual materials are stored as on-line data sets, and are
updated before each Seminar. The materials for the Basic
Seminar contain step-by-step examples for common STORET
operations, and include -a good number of practical
exercises. Students rate these materials as good, and
carry them away to use as prompts and reminders.
Economic pressures are forcing more states to restrict
state employee travel. SUAS's budget will not support
conducting seminars in each state whose employees cannot
travel across state boundaries. In time, this may lead to
real degradation of STORET operations in travel-restricted
states. This is a predictable result of trained staff
being promoted or leaving and their replacements not able
to get training.
The process of problem resolution through telephone con-
sultation often amounts to, in effect, informal training.
This coaching is extremely valuable, and is highly rated by
users. This medium could become saturated if formal train-
ing of users can't keep pace with the turnover rate of
users.
-------
49
STORET broadcast messages on the system are not used for
training, because there is no assurance that a given user
would log on frequently enough to receive a given
broadcast.
o Another training source is on-the-job training from peers
or supervisors in the users' agencies. On-the-job training
is not regarded well by users interviewed, for example, in
Region V. First, supervisors and peers feel overloaded
with their regular assigned work; second, they feel unqua-
lified to do a good job training someone else.
o Self-study is workable only for someone with a good ground-
ing in ADP and water quality programs, and is rarely used.
o One of the most effective training vehicles, the annual
meeting of STORET users, was cut from the bud get'after
1979. Users ask that it- be restored. Users who partici-
pated in these meetings rate them highly, because the con-
tent was heavily loaded with practical "lessons learned"
and suggestions for creative' ways to use STORET, and
because of the opportunity users had to. reach consensus on
improvements and their priorities. Staff and users both
benefitted from the brainstorming done at these meetings,
and from the consensus reached on needed enhancements. The
budget for the last meeting in 1979 was. about $15,000; one
staff member in DP/UA Branch (DP/UA) estimates that a
budget'of $20/000 would provide the same level of support
today. Most of''this'money is used for- air fare for state
users and the money comes from EPA travel funds.
-------
50
b. Conclusions and Recommendations for STORET Training
o We recommend that the Office of Water Regulations and
Standards provide resources for the reestablishment of the
annual STORET users meeting.
o The Data Processing and User Assitance Branch should write
a proposal which describes alternative methods and costs
for providing seminar-type training to regional and state
users, whose travel is becoming more restricted. Alterna-
tives might be a videotaped set of lessons and use of a
"tutor-text".
3, STORET User Documentation
a. Audit Findings for STORET User Documentation
o The STORET User Assistance Section (SUAS) prepares the
documentation for new or improved STORET capabilities be-
fore these capabilities are announced. The announcement is
contained in the forwarding letter which distributes the
documentation. In this way. STORET documentation is al-
ways up to date. An iterative procedure is used to devel-
op, test, and document STORET changes. There is a formal
procedure using a mail-in card found in each manual for
reporting user-discovered errors in the documentation;
however, glitches are usually reported by phone to SUAS.
o The STORET documentation-holder mailing list is culled once
a year. Addressees are automatically mailed all changes
with an updated change log. New users are given complete
sets of user documentation with all change pages already
inserted. New sets of documentation are issued only in
response to a specific request from a specific user; this
avoid duplication of documentation in an agency, and thus
saves money.
-------
'51
o Documentation which is subject to frequent change is kept
in an on-line data set, and not printed as part of user
documentation. Users are told how to call .up these data
sets as needed. The data set for parameter codes allows a
user to ask for only those changes which have been made
after a given date, to eliminate' the need to scan the full
set looking for changes,
o Users rate STORET user documentation from "adequate" to
"good" according to SUA5. Users find the current page
numbering scheme hard to understand at first. Once under-
stood, it is easy to use. User .documentation is not in-
tended to provide all of the details and s'tep-by-step
prompts which would be needed for it to be a stand-alone
course in how to use STORET. For most users, the majority
of whom have limited ADP background, the fastest and
easiest way to use the system for unusual operations . or
sophisticated retrievals is to telephone for help from
SUAS. In terms of readability, ease of use, accuracy, and
currentness, most users rate STORET user documentation
quality, high.
b. Conclusions and Recommendations for STORET User Documentation
o The quality and comprehensiveness of STQRET user documen-
tation are good. The documentation meets users' needs for
written references describing how'to use STORET.
-------
52
4. STORET User Assistance
a. Availability and Utilization
i. Audit Findings for STORET -User Assistance Availability
and Utilization
o STORET users know that helo is available from SUAS .
In fact, most users try to get all the help they need
from SUAS or regional STORET points of contact,
even help which should be available from NCC user
assistance. Users take advantage of STORET's ser-
vice .
o SUAS estimates that about 50% of calls for help come
from state agencies. Calls range from routine re-
quests for documentation to major suggestions for new
capabilities. Approximately 35 to 90% of questions
can be answered during the initial call. The 10 to
15% balance are the types of questions or issues which
require time to staff and evaluate. These figures
represent the User Assistance Section's best esti-
mates ; noformal, written records are kept concerning
the number, nature, or sources of calls received.
Such records might be useful for management purposes
and for providing precise feedback on the exact nature
of SUAS activities. For example, such records might
indicate what questions are most commonly asked in
order to permit DP/UA to implement appropriate im-
provements in documentation and training.
o SUAS would like to have at least one person in each
Regional Office, referred to as the STORET point of
contact, able to provide support equal to that provid-
ed from SUAS. Each region has a designated STORET
-------
point of contact, although the level of .support actu-
ally provided varies widely. These points of contact
must be designated by' the regions and the individuals
so designated may have other duties as well. The
STORET points of contact often must rely on on-the-
job training and a lot of coaching from SUAS to learn
their business. Historically, they are promoted just
about the time they become proficient in providing
technical assistance. This means that many users con-
tinue to have to call SUAS with requests for all
types of1 assistance, even relatively routine ques-
tions . .
o The User Assistance Section does more than provide
consultation and coaching by phone, and present the
seminars. For example, staff are responsible for
maintaining user documentation,- developing . system
enhancements, contributing to feasibility studies for
other Office of Water systems, and providing general
staff support to DP/UA and MDSD. ..
Cross-training is used in SUAS. Each "staff member
stays . continually apprised' of ' current problems and
SUAS solutions by informal discussions. Staff members
have specialties, but aH issues may be assigned to
any staff member if the "speciaEst" is. not available.
The User Assistance Section. Chief distributes action
items to a staff member. DP/UA is working to ensure
that routine requests for documentation and similar
questions are- handled by clerical personnel rather
than, being referred to SUAS personnel.
-------
54
ii. Conclusions and Recommendations for 5TOR.ET User Assis-
tance Availability and Utilization.
o The Office of Water should work with the regions to
develop a policy to promptly train a. replacement when-
ever any Regional Office point of contact leaves, and
to ensure that this person be dedicated (at least 75%
of his or her time) to addressing state problems
promptly, in coordination with DP/UA.
o In the first half of fiscal 1983, the DP/UA should
examine the feasibility of establishing a log of
requests for user assistance. MDSD should examine
alternative designs and costs and develop the most
feasible option, or should substantiate the in*
feasibility of maintaining a log, especially with
regard to costs.
b. Quality of STORฃT User Assistance
i. Audit Findines for Qualitv of STORET User Assistance
The sample of users asked give SO AS ratings from
"good" to "the best,.." Users frequently commented on
the friendliness and patience of SUAS. Users also
expressed appreciation for the fact that SUAS staff
talk in their language, and not in computerspeak. The
staff in SUAS have worked there for an average of
eight years. They are respected for being honest and
straightforward, and very resourceful and tenacious in
getting answers to problems.
-------
55
ii-. Conclusions and Recommendations for Quality of STORET
User Assistance
o The quality of STORET user assistance is very high.
.Users' questions are answered promptly and correctly
.by SUAS personnel.
-------
56
IV. SUMMARY OF CONCLUSIONS AND RECOMMENDATIONS
STORET is one of EPA's largest and most widely used computer systems.
The system is provided as a set of common utility programs to a dispersed and
varied group ' users in EPA, other Federal agencies, and states. The system,
while simple in its broad conceptual outline, contains a large number of indi-
vidual capabilities, some of which are complex and sophisticated. STORET runs
on the IBM computer at NCC and is supported and maintained by the Data Process-
ing and User Assistance Branch in the Monitoring and Data Support Division of
the Office of Water Regulations and Standards.
STORET represents a large investment, in terms both of its monetary
costs and of the value of its data base of water quality information. In addi-
tion, system maintenance, operations, and support require a reasonably large
level of yearly spending. According to EPA's ADP Resources Management System,
STORET timesharing spending (exclusive of lAGs) will total more than $1,700,000
in fiscal 1982. By managing STORET well, EPA can help ensure that the system
yields maximum benefits.
Overall, our analysis revealed ^hat STORET is well managed. Although
we identified deficiencies in some areas and believe that it is important that
EPA investigate and correct a number of problems, we are not proposing major
changes in the way STORET is managed. In addition, most of our recommendations
can be implemented fairly quickly and at moderate cost. STORET has been op-
erating in a reasonably stable way for a number of years and any major problems
in system management would be expected to have been addressed already. In many
areas management procedures are fairly informal, but operate effectively and
are reasonably suited to current circumstances of steady operations
Exhibit IV-I summarizes our major conclusions and recommendations for
each of the three major audit areas: (1) timesharing budgeting and billing;
(2) system management and operations; and (3) user support.- Our audit results
are outlined briefly below.
-------
57
t/)
O
l_ซ
O
UJ
y
o
o
UJ
T Q
> Z?
!- '00
CO O
) * N^
Z 00
>; rs
^ d
. ง
CJ
u_
o
>-
a
^
2
S
i-
ง
ซ
55
1
3
. 5
3
^
i
^
g'
x'
, vป 3
*_ s
0 '*. 0
V
s: * ts
i- _ c
5|| .
C SP C
W -SI -
a ** <ฃ
tj i.rt +*
ซ.ป no
"^ ฃ *
^"-31*
3 vป w
vi 3
a c -J
Q O
^* fl C
0*^*0
*
ง w
i/l Of A
g vป a
C 5 ,3 vi
. SIS^
J3~i
^ c *-
>ป
< * k. O
"**.-- ^ C U
. 2 g. X X
mis
0- ป 1
=
^,
a
1 ?'.-.
f 'S
Of 3 '
' f " ' '
2- i
TI ' "vi
fl s>
ซ>
V)
1 "
F- *
.ฃ ฃ2
a. c
Lป Vป
II
U QJ
I.
"e
* o
41 U
MB Uป
3 e
> **
5 ^
13 ป
Jv*
o .
-ซf
2 IE
VI
x i- U
01 Ok-
*ฃฃ S* , S
3ป ซ 1* V VI 3
C 3^*ป ** W X1^
"w 5 o ^ S
JV l_l L> O vป O
^ ^.t*J ฃ 4 k.
vi as w c.
gj -OO > ta U
I iDi "Si.ฐS
b> X W W L. >,
ฃ Si' v.2.2^
2 >ป 3 3 t.
vi 3 "ป *q o o* a
^ (j o 5 a* *^
U * V ** If s
, C U U vt ^
^ v) <0 X
152 "U5
CT. ff*j e * * g
e -S S ? vป o S
ซo e i^t * c '"
c
w
c
OJ
oe us
<->
0
u s
53
'S-S
*
i^ a
* i
or
ฃ
e
V
*vป *
II.
VI W
ฃ3
S 1
01
>,
11
A Kl
=1
IT'S overall ava
users' requirein
1!
VI
VI
OJ
U
i"
C
vi O
c ง.
O vi
^ 4>
^s K
t .1
* I
1 =
^ A
c ป
3 C O ^
3 tj M
^ VI ซ
^ tM O U
O ** Gw *ซ,
3 ฃ _.ฃ
to *t3 LM ฃ
* """ oซ^
wi o ^- 41
5 " 2
< ซ
** -O L.
o 5 ||
_u O e p
O O = 31
-"--
^ ซ. O '
a. g u <9
w ฃ a. ฃ
'
1> 3 X
> -o o *-
.-S ซE
u O t. Of O.^ '
3^1 3 ป t
3 01 - 0
53- 2-e
s-*; 5 a ฐ
W 01 ^ O ^
>q Q -* ~
w* *rf Ot _ ** trt
M O ** C. C
es 3 u o
3 u u -c y ป
1/1 3 e - e v. ?~
:i~S 23-; s
'r^1^ oo,"3"
J "3 J3 T3 > 0
^OtUW W*QJ3O
ซ U 1- Q. => i. 13 CL
'
M
O
2
01
c.
o
ae ~ *
o
I
*
VI
or ง
O Of
^ u
1/1 O
'k.
w a.
' U
X ซ
.-*" o
3 OJ
W W
41
: 1U 4
3. '
> 3 *
O .* U *
'_ U
3, 4 4 '
CS 3
2 -I
^ 01 *
U -*
vป ra s
3_2.
MO.-'
oi a
&. fa. 0)
= 0.1-
0 3. 3
ฃฃฃ
Q, ซ 4
X
V
3
U
5
3 _t
*J
U - -
3-
u
UJ
ง*'
^
' *
5/
a = -3 c
% it) O 9
I ^ y 3
^ 3* 4* ^ "
255 5^
^ "C **
c 3 s 2
3 ,i 3 "S.
Ill y
!u "* vi 2
o o> ^c a.
'v< S s "v~n
ฃ C 5] > "^
O V ^ 3*
3 =
O -> 3 O Jf
i ฃ ซ os
i^ 3 = vt u
11 s's tt
a. u a -J e.^
' *
*ป JI tfป
L, - -*
O O -i
a. L, i
3*" C ป2 ^
wi O 31
U *J 3
5 K *^
U C C V
01 C
'a 3 " ' S "3 =
v* O "3 C ^ซ
-*-.. O ftt O
JfQ . ป!.'_
a -a O.
-2 .5 31...
- 4> ^ ? 3 .2,
w - s ซ *ซ
= fl 3 ซrt ฃ
U ~ V U 4
if 5 ^^i
a vi vi -* -^
:ซi.-j*i
w = -0 ^ x n
3tซป u s. .*
0
2 .
1
^
>i
S
O .
(/t
1
. j
1
i
1
' *
(
c ' ซ
3 e**-
"5 u ซ O
งio" u M. i
S 41 _ O
19 ^ & CO
Ol 3 V
- c- > , s c
a or -~ - ป> o
^ T3 ^ ฃ
-- ^ * 1 ^ :
w u a W ^
13 C O 01 -J :
'ง1?' '?1
' s x a/ ^ ซ>
^- s
-------
58
1/5
Q S
LU O
=3 O
Z LU
ii t_3
ca TH
S O
X
5/1
1
i c
I >, -ป 0)
JU w ./>
If |l
"*"
. b u! S ซi
7 a .. 34E2
33= J! *ซ-
0 " g - i. S
it o s f i. <
v 2 ** = = "?
Ciฃ 2*|
.> g c -S 4i
S jซ *A > *c
3 ^O *" 3 5 '1
a. c o w IA
-a o <ซ ฃ o a
* " !
u.
5
sa = =-
lA ,A 1
rq * U.
W ซ i U
^ "5* s 3
ซ = uj"
>ซ- w Gt
o a*
" > ? J'Sg '
41 3 3 (!*ฃป
jป "3 IA c-yn
^ w -g Sป
** ^ w o ป -
c ซ; Q. -i
S c ป ** a/
U ftj ** 9 S- C
111 !c ซ S 2
S3 3t t/ป
O. %- T3
-s ni
|5 i^.5
5 ฐ "3 5
aT ฃi ป-
O XI i ui O
M
^
g, 7
i -S
ฃ O '
3
ifl- Oj O
ill
A 41
o
"" O
LU Vt
cc a, i
a ^ c
^ 9 V U
A Wป U OJ
t- 3 u IA
1. B* -
i .5
& c
J? ;
S *
V
i
a
VT
3
*"
1
I
1 ป,
b.
3 ง
C ^
.A 0*
X ป
tt O
flj ^)
J= C
^ "fl
*
i
w
2
S
u
-
es
(U i/t 1
U Ol 5 Nป ฃ>
OS .O. 3,1/1 3 b>
w3 ^ ฃ"5 S 0
^T ฃ ^ a 5 ^ *"*
ซi u "u ฃ: = -*
01 O 1 1- >, u
w^ a c ซ
a. 13 iA IQ > ซ
"w o w *o **
i^ :;.=4ฐ
3 _ ?
SO 4P -J C Ot O ซซ
C O <9 ^
viu k. w ,u> lA > flj 5;
** i, b. i. u w e 5
u'o o> ซ *-* e a, o u
'T, i^350 =
ปV kC 3CJ}iA^^^
c *ฐ c .3 ซป '31 E 21
vft3- w4J^ SSJJ
vi 3 O* 4l-a*-C
fl>* O>ปtA ^ .
^ *J |