^'
   Environmental Protection Agency
        Management Information &
             Data Systems Division

       Data Management and
     Standardization  Program
              Feasibility Study

                      Final Report
                     June 5, 1979
U.S. Environmental Protection Agency
Library, Room 2404 FM-211-A
401 M Street, S.W.
Washington, DC  20460

-------
ARTHUR  YOUNG  S COMPANY
                                            IO2S CONNECTICUT AVENUE. N. W.'

                                                 WASHINGTON. O. C. 2OO3S


                                                 June 5, 1979
 Ms. Mary Lou Melley
 MIDSD  (PM-218)
 Environmental Protection Agency
 Room M3101
 401 M Street, S.W.
 Washington, O.C.
 Reference:
Contract No. 68-01-4640
Task Order No. 68-01-4640-18
 Subject:    Final Report

 Dear Ms. Melley:.

      Arthur Young & Company is pleased to submit this final report
 on the Data Management and Standardization Program Feasibility
 Study for the EPA.  This report reflects our response to the
 comments and suggestions received from you during the past
 several months.  This deliverable will be followed in the near
 future by the Executive Summary which provides a synopsis of our
 recommendations for this feasibility study.

      We greatly appreciate the active participation and cooperation
 which our project team has received from both you and other EPA
 personnel during this engagement.

      Should you have any questions, please contact Mr. Philip
 Snyder or myself at (202) 828-7000.
                                        Very truly yours,

                                        ARTHUR YOUNG & COMPANY
                                     By:
                                                 -          I
                                        Gerald Mendenhall  L
                                        Partner
                                                                         .

-------
                         TABLE OF CONTENTS
       BACKGROUND S SCOPE
       1. BACKGROUND
       2. METHODOLOGY
                                          PAGE

                                          1-1
                                          1-1
                                          1-3
II.    REQUIREMENTS ANALYSIS
       1. CURRENT TRENDS IN DATA MANAGEMENT PROGRAMS
       2. SPA CURRENT STATUS
       3. OVERALL EPA REQUIREMENTS
                                           II-l
                                           II-l
                                           II-8
                                           11-12
III.   ALTERNATIVE ANALYSIS
       1. ORGANIZATIONAL STRUCTURE
       2. DATA MANAGEMENT TOOLS
       3. POLICIES AND PROCEDURES
       4. RECOMMENDATIONS
                                          III-l
                                          III-l
                                          III-6
                                          III-8.
                                          III-8
iv.    IMPLEMENTATION^ PLAN
       1.. IMPLEMENTATION PLAN
       2. PROGRAM COST ESTIMATES
                                           IV-1
                                           IV-1
                                           iv-6
       APPENDIX A
       APPENDIX B
DATA
CODING SCHEMES
A-l
B-l

-------
                          LIST OP EXHIBITS
EXHIBIT          .                        "                       FOLLOWS
NUMBER                      •                                      PAGE
1-1       STUDY METHODOLOGY                             .        1-3

II-1      DATA MANAGEMENT PROGRAM COMPONENTS                    II-l

II-2      THE DATA ELEMENT DICTIONARY/DIRECTORY AS A            II-5
          DATA MANAGEMENT TOOL

11-3      DATA QUALITY ASSURANCE                                11-8

II-4      SAMPLE SYSTEMS COMPARISON MATRIX                      11-10

II-5      SUMMARY OP FINDINGS                                   11-10
                     -N

III-l     ALTERNATIVE ORGANIZATIONS                             III-2

III-2     DECENTRALIZED ALTERNATIVE INTERSYSTEM LIPE            III-3
          CYCLE DATA MANAGEMENT

III-3     CENTRALIZED ALTERNATIVE INTERSYSTEM LIFE              III-4
          CYCLE' DATA MANAGEMENT

III-4     HIERARCHICAL ALTERNATIVE INTERSYSTEM LIFE             III-5
          CYCLE DATA MANAGEMENT

III-5     ALTERNATIVE COMPOSITION OP THE TOOLS AND              III-6
          POLICY AND PROCEDURE RESPONSIBILITY

III-6     HIERARCHICAL ORGANIZATION STRUCTURE                   III-9
                              ;
IV-1      EPA DATA MANAGEMENT AND STANDARDIZATION       "        IV-1
          PROGRAM LIPE CYCLE

IV-2      EPA DATA MANAGEMENT AND STANDARDIZATION               IV-6
          PROGRAM IMPLEMENTATION SCHEDULE

IV-3      COST ESTIMATE MATRIX FISCAL YEAR 78-79                IV-8

IV-4      COST ESTIMATE MATRIX FISCAL YEAR 79-80                IV-8

IV-5      COST ESTIMATE MATRIX FISCAL YEAR 80-81                IV-8

-------
                      LIST OF EXHIBITS (cont'd)
EXHIBIT
NUMBER
                                                      FOLLOWS
                                                       PAGE
IV-6

IV-7

IV-8
COST ESTIMATE MATRIX FISCAL YEAR 81-82

COST ESTIMATE MATRIX FISCAL YEAR 82-83

COST ESTIMATE MATRIX COST SUMMARY -
FIVE YEAR PLAN
IV-8

IV-8

IV-8
IV-9
FIVE-YEAR COST AFTER FULL IMPLEMENTATION
IV-8

-------
Background & Scope

-------
I.   BACKGROUND & SCOPE

-------
                         BACKGROUND & SCOPE
     This document  is the  final  report  for  the  Data  Management and
Standardization Program Feasibility Study prepared in response to the
Environmental Protection Agency Directive of Work Number:  68-01-4640-
18 under Contract Number 68-01-4640.

     In this chapter Arthur Young & Company describes  the background
behind the study initiation and  resulting project scope,  and the
methodology employed throughout  the study effort.

1.   BACKGROUND
     The impetus for this project came  from  several  diverse  sources
and factors within EPA.  These factors included:

          Desire for a common facility  identifier,

          Realization that information  is a  resource that can and
          should be managed,

          Perception that -EPA is in the  third  stage  of ADP growth,  and

          Need for an implementation vehicle for  IRLG recommendations.

     In the following paragraphs we describe these motivational
factors and attempt to show how they came together for the initiation
of the project.

     (1)  Common facility identifier

          For a long period  of time there has been active support from
     diverse areas of EPA for the development  of  a common facility
     identifier so that data collected  and used  in one program can
     also be utilized by other programs.  Primary advocates  for  this
     feature were Region 3, Office of Enforcement, and Office of Water
     Program Operations (OWPO).  Prior to this study OWPO had initiated
     their own project to develop a long-range ADP plan.   A prominent
     feature of this plan -is the recommendation for a common facility
     identifier for the Wastewater Treatment (WWT) Program.

     (2)  Management of Information

          In both the public and private sectors  there is growing
     realization that information is a  vital resource to  an
     organization that can and should be managed  in  the  same manner
     as other resources, such as money,  personnel, or inventories.
     Information is a tool for the management of the other resources,
     but there are aspects of information that must  be managed in


                                 1-1

-------
     order  to assure  its quality  and  its  effective  use  for  the total
     user community.  These include:

          Location  and  responsibility  for  the  data

          Data definitions

          Data coding schemes

          Data collection and processing.

          MIDSD, as well as several program offices and Regions, felt
     that the information in the  Agency was not  being effectively
     managed.  For information to be effectively managed the active
     support of top management is required. These advocates of
     informaton management wanted a third  party  to  evaluate  and
     document the status of information management in EPA in the hope
     that an objective  opinion would help  gain top  management  support.

     (3)  Stage 3 of Data Processing Growth

          In the "stage  hypothesis" of data processing growth advanced
     by Richard L. Nolan Ph.D, Stage 3  is  defined  as  the  control stage
     where emphasis has moved from management  of computers  to
     management of data.  Some individuals perceive the Agency  as
     moving into  Stage  3 in many  areas, and a  study is underway to
     assess the stage of each major program and  functional  area.  A
     key element  in this stage of growth  is the  development  and
     application of formalized controls.  The initiation of this study,
     therefore, is compatible with the Stage 3 assumption,!/

     (4)  Interagency Rggulatgry  Lia.ison Group (IRLG) Support

          Douglas M. Costle, Administrator of EPA,  has stated on several
     occasions, including the August 9, 1978, IRLG Common- Codes Project
     Steering Committee meeting, that the Interagency Regulatory
     Liason Group  (IRLG) effort to coordinate  the regulatory and
     enforcement efforts dealing with chemical substances of the four
     primary Federal chemical regulation agencies is a  top priority
     item.  The IRLG,  comprised  of EPA, Consumer  Product  Safety
     Commission, Food and Drug  Administration,  and  the Occupational
     Safety and Health  Administration, has several projects ongoing
     to accomplish this coordination.  One of  the projects is  the
     identification and  implementation of common coding schemes and
i/  Since  the delivery of the draft version of  this  report in December,
1978, a study has been completed by wolan, Nortan & Company assessing
the effectiveness of EPA's automated information processing
capabilities.  The study was performed in relation to  Dr. Nolan's stage
hypothesis." The result, in part, was a conformation of the assumption
that EPA was in Stage 3.


                                 1-2

-------
definitions for data elements vital  to  the  sharing  of  chemical
information.  It was recognized early in the conceptual stages of the
total IRLG effort that  to be able to implement the coding requirements
will require a coordinated, Agency-wide data management program.

     These factors and forces came together  in  the  initiation of the
Data Management and Standardization Program Feasibility Study. During
the initial Advisory Committee Meeting  held  March 17, 1978, it was
decided by the Committee that the project should concentrate  on two
aspects or phases.  The first  was the definition and documentation of
the need for a data management and standardization  program based on
the current status of data management in EPA  as reflected in a specific
sample of typical, major systems.  The second phase would be predicated
on the results of  Phase I.   If it was determined that there is a need
for a data management program,  than Phase II would address the policies,
procedures and tools that would be required to implement the program.
It was decided by  the Committee  that  the project would  not address
standardization of specific data definitions or  coding  schemes  since
that is a major effort  in itself  to be undertaken on a selective  basis
as part of an operational data management program.  The methodology
used in performing the phases is presented  in the following section.

2.   METHODOLOGY
     The Data Management and Standardization Program Feasibility Study
was performed in two analytical phases:

          a requirements analysis phase to identify and determine EPA's
          requirements  for data management and  standardization;  and

          program development phase to define and evalute  alternative
          strategies for the development, organization, and
          implementation of a Data. Management and  Standardization
          Program in EPA.

A briefing of the findings and recommendations  was presented  to  the
EPA Project.Advisory Committee and interested EPA  personnel at the
completion of each phase.  This report is composed of the  information
presented at the briefings and the feedback  obtained from EPA on the
briefings. The methodology utilized in the performance of the  project
is depicted graphically in Exhibit 1-1.

     Phase I, the development and documentation of EPA's requirements
for a Data Management and Standardization Program, was approached from
both a quantitative and qualitative perspective.

     (1)  Quantitative  Approach

     The objective of the quantitative approach was to  identify
     concrete examples  of EPA's need for data management  and
     standardization through a review of 15  representative systems.
     The systems were intially defined at the- first advisory committee

-------
      II
      s
                                           EXHIBIT 1-1
u

2
o
a
g

I


a''

i
3
2


5
<
2

u
      I

      i
         I*
;
-

«
!
j1
a
»
t *
                    t
                        Jl
                        ill!
                        IJti
                         t
                            I
                            II
1,
-I
                             w a
                                i

-------
meeting.  Their selection was based primarily "on the committees'
evaluation of potential of sharing data across the systems.  Other
considerations included the systems visibility  in EPA  and  that
each of the Assistant  Administrator's areas was  represented.
During the course of the project/ systems were added and deleted
from the sample group  for such 'reasons as:

     Identified systems did not  contain  the  specified  data
     categories

     Systems were still in- initial development  stages

     Documentation on  the systems were inadequate for  complete
     analysis.

The systems included in the final analysis are  identified  both
in the methodology exhibit and  in Appendix A,  the data dictionary
prepared as a deliverable for Phase I.

     At the first  meeting  it was  also decided  that for the review
to be meaningful given the  time and  resources available for this
effort, the analyses of the  representative systems would be
focused on five data categories.  The data categories were
selected by the commmittee based on their potential  for being
used to interface files and  their current relevance to  the  major
issues facing the Agency.   The data categories selected- for
detailed review in the sample systems were:

     F aci1itytIden t i fi e rs - The  unique name or  number  used in
     identifying any type of facility or authority.

     Monitoring S ample Station•Site - The name  or  identifying
     number of a site where monitoring or sample data is actually
     collected.

     Geographical Location - A location  name  or  code which
     identifies the geographic  location of a facility or sampling
     station.

     Parameter Unit Identifiers  - A code on .the  file indicating
     the unit of measure for a specific  field.

     Quality Assurance Codes — A code indicating the level of
     confidence in the data,  or when the  information was updated.

     The systems review consisted of documentation  review  and
supporting interviews with the  appropriate EPA system's  managers.
The result of this process was an approximation of EPA's current
level of data standardization and the need for  further
standardization.  As a by-product/ a manual Data Element
Dictionary was prepared.  The  Data Dictionary, consisting of
definitions,  coding  schemes, and  field descriptions for  the five


                           1-4

-------
     data categories and the sample systems, is presented in Appendix
     A.

     (2)  Qualitative Approach

          The objective of the qualitative  approach  was  to  determine
     perceived needs for an overall EPA data management  and
     standardization program,  and potential  benefits  EPA would derive
     from the program.  This approach included the following  steps:

          Interviews with cognizant individuals in EPA; and interviews
          with external organizations such  as  the National  Bureau of
          Standards and the Department of Defense, which currently
          maintain successful data management  programs.

          A literature review to  determine  the state-of-the-art in
          information management.                '

          As a result of this effort, the components and benefits of
     a data management and standardization  program were  defined for
     further evaluation in terms  of their appropriateness  for
     application in meeting EPA's requirements.

     The information obtained during the  quantitative and qualitative
phases' of the requirements analysis were synthesized  and a  briefing
was given to the  EPA Advisory Committee on July 7, 1978.  This briefing
was also video taped for distribution to EPA regions.  A draft  data
dictionary was also presented prior to the  requirements  analysis
briefing to facilitate EPA verification of-the quantitative results
of system review, and  to stimulate concerns or ideas over the
difficulties which have or could  be encountered due  to lack of data
standardization.

     The EPA Advisory Committee members were extremely helpful during
the formulation stages for the approaches to requirements analysis,
and also provided valuable comments and feedback related to the
findings from system review,  draft data dictionary,  and contents of
the requirements  analysis.  Following the briefing, direction was given
to proceed with the second phase  of the study.

     In September,  1978, this project was temporarily suspended for
one month to facilitate coordination with other EPA initiatives related
to data management and standardization concepts.  The initiatives
included the IRLG Common Codes Project, OWPO/WWT Long-Range Plan, and
Region 2 Facility File Pilot Project.  Review of these other related
activities further defined EPA's pressing requirements for data
management and standardization, and" provided insights into program
implementation strategies which appeared most  feasible for  use  from
the perspective of the EPA current organizational structure.

     After establishment of EPA's requirement  for data  management and
standardization,  alternative  implementation strategies for a related


                                 1-5

-------
program were developed.  These strategies were organized in terms of
three program components which included:

          required organization structure for program administration
          and oversight,

          data management and standardization tools, and

     .     policies and procedures to govern program operation.

     These strategies were presented to the  EPA  Advisory Committee on
October 18,  1978,  and  comments were  reviewed and incorporated.  The
effectiveness of program alternatives were assessed weighing
considerations of benefits, disadvantages, impacts  in  the EPA
environment, and implementation  issues.   A recommended program
alternative was then selected, and estimated costs  for program
development, implementation and  operations  were  developed.   An
implementation plan and strategies were defined  for the  recommended
alternative program strategy.

     The program development phase concluded with  the delivery of
the draft final report on December 4, 1978, which  synthesized all
study efforts to date, and presented recommendations for the
implementation of a Data Management and Standardization Program
in EPA.  The EPA Advisory Committee decided to  delay the formal
release of their comments on the draft report,  and thus the
finalization of this report, until the results  of  the Nolan,
Nortan & Company study, referenced earlier, were delivered.   The
results of the Nolan, Nortan & Company study concurred  with the
earlier findings and recommendations of  this study and  thus did
not effect the Advisory Committees comments..  The  suggestions
and recommendations made by the Advisory Committee have been
incorporated into this final yersion of  this report.
                                 1-6

-------
Requirements Analysis

-------
II.   REQUIREMENTS  ANALYSIS

-------
                   II.  REQUIREMENTS ANALYSIS
     In this chapter Arthur Young & Company presents  the  results  of
the requirements analysis phase of the EPA data management  and
standardization program feasibility study.  Requirements analysis
activities and findings are presented in terms of:

          Current Trends in Data Management Programs

          EPA Current Status
1.
     EPA Requirements.

CURRENT TRENDS IN DATA MANAGEMENT PROGRAMS
     The components of an effective data management program  are
presented at this point to provide a framework within which  the  rest
of the report should be viewed.  An effective data management program
must consist of a dynamic program structure which actively interfaces
with program participants.  Our research has shown that current
effective information management programs  consist of, three primary
components:

          Organization structure which ,supports  the program;

          Data management tools which produce the program products;
          and

          Policies and procedures which govern program operations.

Exhibit II-l presents an overview of the relationship of  data
management program components.  The  functions of each of these three
entities are further discussed in the paragraphs below.

     (1)  Organizational"Structure

          An effective data management program requires a defined
     organizational structure to  implement  the program, administer and
     monitor ongoing program operations, and initiate and implement
     program updates.  The data management, administration entity is
     also responsible for obtaining programmatic or user  support,
     resolving program related issues, evaluating the ongoing  benefits
     of program use, and seeking  new areas for program application.

          The actual structure for a data management program is, for
     the most part, dependent on the characteristics of the
     organization into which the program is.to be  implemented. Several

                                II-l

-------
                                                       EXHIBIT II-1
c/a
O


o
u
O
es


H

yj
<


<

<
Q
                                                                            .

-------
I
t
 t
factors must therefore be considered in determining  what  type of
organizational configuration is the best strategy for successful
implementation of a data  management and standardization program.
These factors include the following:

     Current structure of the organization including
     responsibilities and authorities?

     Authority levels required for successful program
     implementation and  administration  in a given organization;

     Expertise required  for program implementation  and operation;

    "Defined program requirements;

Evaluation of these factors will lead  to a determination of the
proper positioning of data management and standardization program
responsibilities  in the  organization;  required functional
assignments; coordination and communications mechanisms; and
approval flows for program activities.

     Alternative organizational structures include: a
decentralized approach placing data management and
standardization programs within specific units of an
organization; a highly centralized program approach using a Data
Administrator and  staff  concept which  organizationally reports
to senior management;  or  a combined or hierarchical approach.  In
a hierarchical approach  a central  policy making body, either a
Data Administrator or committee, will set policies and
organizational standards.  The staff for this agency-wide
committee will monitor and enforce the  standards set by  the
Committee.  The implementation of the program is delegated to
specific units of  the organization.  These alternative will be
discussed in greater detail later  in Chapter III.   Each of  these
alternatives must be evaluated in terms of the factors identified
above.

(2)  Data Management Tools

     The primary tools which can be  employed to effectively manage
data as a resource can be categorized  as  follows:

     Standards

     Data Element  Dictionary/Directory

    . Feasibility  Studies

     Quality Assurance Programs.

     The use of  these tools is prescribed, implemented, and
maintained by the organizational component of the data management
program through  use of program policies  and  procedures.

                           II-2

-------
I
t
 t
     The following paragraphs contain a more detailed explanation
of these four primary data management tools.

(a)   Standards

     Three major types of standards can be developed to  facilitate
     data management.  These  include standards for:  data
     elements, system design  and documentation,  and data
     acquisition techniques.

     Data Element Standards - the objectives of the use of  the
     dataTelement standards are to:

          Improve data accessibility

          Facilitate timely data transfer and exchange

     -    Enhance support for management decisions

          Reduce the reporting burden

          Improve data effectiveness

     -    Increase data re-utilization.

     Data element standardization  involves the use of common  data
     definitions, data use, coding  schemes, and naming
     conventions.  For similar  data elements which are found in
     various system files, data element standardization is.
     necessary for the sharing  and utilization of  information in
     the files.  Data transfer  and exchange efforts are
     facilitated, and requirements for  data element conversion
     to other forms or interpretations before utilization are
     reduced.  This results in  improved information access time
     and data quality.  In addition, data standardization
     simplifies the efforts required for implementation and
     maintenance of a data element dictionary/directory, as  well
     as system design and development efforts.

     System design and documentation standards —• involve the use
     of:

          Standardized methods for system design  and
          documentation throughout the system development life
          cycle;

          Milestones :and control points for quality review  of
          system development and documentation efforts;

          Acceptance criteria to determine the adequacy of  the
          system;
                                          II-3

-------
t
t
 t
    , System implementation procedures review to measure the
     feasibility of implementation strategies, user impacts,
     and impacts on current computer resource  utilization;

     A system change approval process.

Use of these types of standards provides an  essential
mechanism for controlling the quality and compatibility of
system development and change.  System design and
documentation standards result in  the establishment  of a
detailed record of system  related  decisions, activities, and
developments. This record can be utilized during the interim
stages of system development as an  information  exchange
medium for both system design teams and potential  users,
thereby maximizing use of  personnel resources.  The  detailed
documentation in a standardized format also-minimizes  the
efforts required for system change by providing a meaningful
and complete reference source for  both programming  and  user
personnel.   .Implementation procedures and standards  reduce
the risk in system development and  implementation  through
application of proven techniques.   In particular,  they
facilitate effective utilization of contractor support.  The
system change approval process assists in maintaining  the
integrity of both the system itself as well  as  any
standardized data elements contained in system  files.

Data;acquisition techniques standards - involve the  use of:

     Data collection request and  approval procedures to
     reduce duplicate data collection

     Forms design and instruction  writing guidelines to
     assure uniformity and clarity in data collection forms
     and definition.

- .   Document tracking procedures to provide an audit trail
     for locating stalled or lost  source data

     Key verification techniques to control  the accuracy of
     data entry

     Control and hash  totals to assure complete and  accurate
     processing of source data

These standards are intended to provide a control  on the
acquisition of data and the procedures used  to  handle  the
data prior  to storage  on a computer, micrographics,  or other
media.   The  objective  of  the control is to reduce  the
reporting burden on the suppliers  of the  source data  and to
improve the quality of the data received and stored.  The
reduction in reporting burden is accomplished  in two ways.
The first is by reducing or eliminating the  duplicate
                                          11-4

-------
t
t
 t
     reporting of data.  The second is by assuring that when data
     is requested from an  entity the  data definitions  and coding
     schemes are consistent.  This eliminates the  need to recode
     information to  fit a new scheme.   Consistency in definitions
     and coding schemes will help improve the accuracy of the
     data reported as well.  The procedures,  such as the document
     tracking, key verification, and  control and  hash  total will
     help improve quality by assuring  that the data that is stored
     is the same as what was reported.

(fa)   Data Element Dictionary/Directory

     A data element  dictionary/directory(DED/D)  is a software or
     firmware tool that is used to control and manage data
     elements in a uniform manner..  It will  provide  a central
     repository of information about each data element in related
     systems in order to facilitate access and control of the
     data bases.  This tool  does not  manage  the  actual content
     of the data, but manages the descriptive characteristics of
     that data (metadata).   Metadata  includes  such physical
     properties as length, value range, types of  admissible  data,
     and validation criteria.   Of more importance to  top
     management is the identification of the individuals
     responsible for the quality and  dissemination of data in
     the specific systems.  .When a  request  is  received for
     specific information from other  EPA Offices, Congress, GAO,
     or the public,  management  can  access the  dictionary to""—~=-
     determine if the information is  available,  and,  if  itu.is,
     they can direct the query to the  proper  individual.  Exn"ibi-t.,..
     II-2 presents the basic characteristics of a DED/D and their"*
     uses.

     The development of the DED/D will also provide a focal point
    . for the development of the much needed standard  data
     defintions and coding schemes.   The DED/D will  continue  to
    .be the primary tool for monitoring the adherence to the
     standards.  This will  be  accomplished  by  reviews of new
     systems and elements that are being added to the DED/D.

     It is possible  for  a. DED/D  to be manual'using either a list
     or card file.  Typically, because of the access and updating
     requirements, the dictionary is maintained via computer. The
     look-up media may be either hardcopy, via listings  or
     micrographics,  or through  on-line query capabilities.  A
     recent development in DED/D is the  developmment of "active"
     DED/D systems.   In  an  active  system the DED/D dynamically
     interfaces with the application  systems,  in either  a free-
     standing or DBMS environment,, to automatically and
     simultaneously update the dictionary/directory and the
     application programs.  For example, if a field definition or
     coding scheme was changed in an  active DED/D system the
     change would only have to be made once.  The system would


                           II-5


-------
                                                                      EXHIBIT  II-2
I
t
                <

                cr>





                O

                u
— o
a o
                UJ



C*
7*
t^
•Ml
!•£•
2
,<••
m*

<
^
POTEN'I





,_


£4
F"
U
j
ft*
M*



















Cft
U
• E?
CO
as:
ca
^_
illARAC',
>M









o
> i3
*"i ^
^ «3
y T3
0
U4 4)

O JJ

f"jj vu
= 0
_i
Simple
contro
! 04
§°
a
-4 4J
f-4 a
(3 01
a

a u

to a
Cf^ 3
c o
JJ U
CD (3
**4 ^
^4
^
•o o
4>
Ui ca
4) 4.
•a -H
U Ul
O jJ
•

%
* tn
c u
Q -*
— t jj
JJ CO
(3 -*
U Ul
-t 4)
14 JJ
-i 0
JJ (3
C U
4) 18
•O JS
•H O
3 
(.|
a


e
O

a.
••4
U
0
a
£
e a i^
3 >• fl JJ
r s e
«J O.JJ 4)
n >u
e o

13 jj
JJ 4)
C CO
C
0 <3
rti fli
JJ F4
4)
(3
<9
*a jj
C Q
(3 T3

1 8!
01 —
Ui tn

<3 5
JJ U
c —

rg •••
l^j M
0 C
c
c
0 ><
-4' U
Reduct
dundani
x
•H
^4
--4
#2
j-j

5^
c

7j
5

41

4J

1
to
ta
0
u
U
*
4)
O —
JJ JJ

CO «4
jj o
G
^
o e
-^ 41
O G
O
a s
tn y
a
*2 3
"3
CTJ
o e
-4 Q
jj
18 U
4J >C
Ui JJ
O
0}
3 ~
Q 0
«« *f
VI 4]














>•
U
4)






















O
^
U
o
a
a
^i

*o
c
13

CO
•g
(8
cf
U
C.
o»
3
a
Ul
JJ

5
4)

33
>
ta


^
^T
^j
tt

< . .
»%

• i
.—
i
3
Enforci
dardfi



<3

S
Q
U
*M

4)
S
 0
u'c
s sour
inatio
4) JJ
-4 ca
Ma a
•*4 ^3
Q
qj T"j
«•* —
01 (3
*















1
4)
• jj
4)
»-4
A
i*
p


•o
G


























0>
C
>»—»
U
e
4)

CB
••4
n
o
u
*
•
i
c
a
•s
3
TJ
4)


^
a
<*4
a
jtf
u
u
1
n
4}
JJ
13
13

•*«)
(3
>
.tt
u
4>

tj

tn
4)
e





















ca

-*i
O
JJ
•H
U
3
O
QJ
n

Ifc4
0

JJ
C 0)
4> -a
S u
4) iQ
. Cnforc
safegu
t
(3
4>
. Ut
1
41 ta
e c
•H C
.C — <
U 4J

3 "c

Q 49
•c
C
Q 13
•*•) ^
JJ (3
ic -a
4) 4)
G .-»
4) J3
a «
•



10
a j=
jj jj

a o
t3 jj
u
13 >
S JJ

4) •-!
a
>r (Q
JJ O
-4 O
u u
3 e
u
C3 33
O J=
a
e ^
•M 0
<3 U
JJ JJ
e e
c c
a a
13
jj
 «
COS
T3 4J
tn jj
(3 *"j m
G >•
CD <3 ta
jj
elemen
design
of new
i
c
o
~»t
JJ
u
««•)
/•».

(3
jj
13
a

0

G
a
—*
^j
u
13
X Ul
ij 13
•












4)
(3
CT
C
1
a
|
o
u

<3

tn
>«*

^j
O
li
O ^.4
0 O
v
(•5
C .U

'"J
.>•»
o w
St2
jj
Cons is
tation
ments







ta
jj
i^
O
4?
U

4)
jj
^4
a
i
1
CJ
•

^4
£
a
a.
13


^
a

41
1-1
u
jj
0
u
•a
15
e
Ul
o
a
s) a
Ui Cl
•IB*
0} JJ
1^ •«"(
z —I

-------
t
t
     automatically modify  the application programs.  In a passive
     system the  change would have  to be coded into the DED/D and
     each application program  affected by the change.

(c)   Feasibility Studies

     The  objectives of coordinated feasibility studies in a total
     data management program are to facilitate review of intended
     program development so that redundant efforts and data
     inconsistency can be  identified and efforts to improve
     coordination and reduce costs can be employed.

     Feasibility studies are therefore required as a basis for
     determining and  justifying  the application of data
     management  concepts within  an organization's  operational
     units and systems.  During  both the  initial  implementation
     and  operational  stages of  a data  management program,
     feasibility studies should  be utilized to determine and
     justify appropriate levels of data standardization, required
     data element dictionary/directory (DED/D) contents and level
     of detail/  and program policy and  procedural  requirements
     and  impacts.   Through the ongoing  operations  of the data
     management  program, feasibility studies  should  be  utilized
     to determine and justify  new system development or system
     change efforts.  Feasibility  studies will  not only correct
     data errors but assist in  finding  and correcting the reasons
     for  the. errors.  These studies are an  essential component
     of an ongoing data  management program.  Once  study
     methodology is defined, future efforts will  be  simpler  to
     adminisister.

(d)   Quality Assurance Program

     A common problem is a general lack of confidence in the
     quality of  the data contained in  an organization's systems.
     Sometimes this lack of confidence is not well founded,  but
     is simply a matter of perception.   In  this circumstance an
     effective remedy is to educate the users on quality control
     measures being applied, and perhaps  to publish  statistics —
     from time to time based on  data quality  audits.

     However,  in other cases, the  lack of confidence is well
     founded and data contained  in the systems is incomplete,
     inaccurate, or out of  date.  Reasons for poor data can usually
     be traced to the data entry point and  such causes as:

          Lack of incentive on  the part of those entering  the
          data

          Confusion as to  what data is required - content  or form
                                          II-6

-------
t
t
 §
     Poor data collection/entry forms design or  poor
     procedures

     Lack of effective software edit capabilities.

In order to restore confidence in the data,  and  improve its
quality, it is necessary not only to clean up data already
in system files, but also to address the causes, such as
those listed above.

Because of the variety of causes, any one or more of which
may apply to a particular system, an audit is required to
detect the problems.  The objective of the audit is to
identify the completeness and accuracy of the data  in  the
files,  and then to analyze procedures for data capture and
processing to isolate the causes..  The current files must
be updated with good data and the capture and processing
procedures modified to assure the future quality of the data.
The  steps that can be taken to improve  the quality  of the
data include:

     Inclusion of effective edit/update software in  systems
     — proper edits will identify obvious  errors such as
     data out of allowable ranges,letters in numeric fields,
     etc.

     Standardization of Data Definitions — standard
     deflations eliminate confusion on data to be collected.

     Revised Forms and Procedures — forms  and procedures
     need to be easy to fill out and follow. Any ambiguity
     or. unnecessary complexity should be eliminated.

     New Incentives — The data collector must be given an
     incentive to collect data.  The best method  is to offer
     reports or data access techniques that are  of  use to
     the collector.  If  this cannot be done, then it is
     necessary to explore alternative means for  collecting
     the data.

     Contractor .Support — It may be impossible  to provide
     incentives to the current collectors for all or part
     of the collection and data entry process.  If this
     occurs an alternative is to contract  the work to private
     organizations. This  is often done for data  entry where
     incentive fees or penalties can provide the  motive for
     accurate data entry.

The Quality Assurance Program must be an ongoing function
with periodic audits conducted and improvements  made.  The
Quality Assurance function is an essential part of effective
information management,  for without reliable data and  the


                      II-7  •

-------
t
I
          2.
          confidence of its users, the most efficient and powerful
          system is worthless.  Highlights of a data quality assurance
          program are shown in Exhibit II-3.

          The quality assurance audit is also an essential tool for
          monitoring the utilization of other data management tools.
          Periodic review of existing systems and  new system
          initiatives will aid in identifying:

                    Inaccurate data or data deficiencies

               .    Poor procedures and lack of edit functions

                    Non-compliance to data management policies and
                    procedures.

          Existing data standards, DED/D  output, and program policies
          and procedures can serve as the basis for development of
          quality assurance criteria.

     (3)   Policies and Procedures

          Policies are the rules  set forth  by the organizational unit
     responsible for the data management program..  They include
     regulations on program responsibility assignments, approval
     flows,  tools to be utilized,  and  quality  assurance  criteria.

          Procedures describe the processes through which data
     management policies can be executed in a consistent manner. These
     include documented mechanisms for:

          Setting and promulgating data standards,

          Evaluating and approving requests,

          Auditing systems, and

          Updating data management tools.

     Policies and procedures serve as a major  data  management program
     component since they provide the operational.link between the
     organizational component and the program participants; and
     program participants, and the data management  tools.

     SPA CURRENT STATUS
 I
     This section presents the current status of data management  in
SPA.   The  current status  is discussed  in terms of existing policies
and procedures for data standards, and the findings from review of a
cross section of representative systems.
                                          II-8

-------
                                                             EXHIBIT II-3
                                    '•*
a  S
                                           (S
                                           c
                                 S  I  i  5
                                 e  *  01  u
                                 <  G  u  <

t
2

5r
                   S
                   a
                                    <
   Ml •
   M
   IU

   2  >
   H  U
   m •<
   a  =
   a.  3
   2  u

   3  3
r
L..
      a

      i

      I*
   |  §2
   P  IU o
   swg|

   111~" < 2!
                                    > 2
                                    « «
                                      vu iu •
                                      rf o a
                                      p s a

                                      a o 2
                                      " 
-------
t
t
(1)  Policies and Procedures

     There currently exists in EPA an ADP Manual  which documents
policies and procedures related to the utilization of data
standards and approval processes for  feasibility studies  and
system design and development.  The Management Information and
Data Systems Division (MIDSD)  is recognized in the  ADP Manual  as
the central area for coordination of  data management efforts.
The Manual states that,"All organizational elements in EPA, their
contractors, and or grantees will  promote the full utilization
of Federal and Agency standard data elements  and  representations
in the design and development of information  systems".  In
addition any data elements and codes  that are already in  use  by
the Agency are to be adopted by the Agency as standards wherever
it is practicable.  Candidates for standardization can be
recommended by any component organization of the Agency. Approved
PIPS or Agency data  standards  are to be published  and promulgated
in the EPA Data  Standards Catalog.  Data element name, definition,
item, coding scheme and  code and abbreviation constitute
prescribed catalog contents. Although an EPA-wide data dictionary
initiative is currently underway, an  EPA Data Standards Catalog
does not yet exist.

     In the current EPA Environment,  the Management Information
and Data Systems Division is  to forward proposed standards to
the Data Standards Coordinators in other  organizational units
for clearance.  The Data Standards Coordinators are to coordinate
standardization proposals within their areas  and submit their
comments on the standard to MIDSD.  MIDSD then can resolve any
conflicts in the proposed standard prior  to  publication of the
approved data standard for EPA use.  Initiation of potential data
standards for review and approval  is  currently performed  on a
limited basis.

          It is apparent from  interviews  with cognizant
individuals within EPA and a review of the current  level of data
management and  standardization activities that, although data
management policies existed  and are documented, they are not being
actively implemented.  There are many factors contributing to
this circumstance including:

     Decentralized management of data  and systems

     Little high level management awareness of the  need  for data
     management

     Limited resources in MIDSD

     The policies and procedures are  not  currently accompanied
by a dynamic program structure nor adequate  tools for effective
implementation  and operation.  This is further demonstrated by
the system findings described  in the  next section..


                           11-9

-------
t
t
i
 (2)  System F indings

     Data standardization is a major data management  tool  which
aids in improving the quality of data, increases the capability
for data re-utilization, and simplifies system design  and change
effort's.  Data  re-utilization is the use of data collected for
one system by a second  system for perhaps a different purpose.
The effect can  be  to reduce  the redundant collection of data thus
reducing the reporting burden on the public and private sectors.
If system interfaces are developed, that can also lead to  the
reduction of storage of data in multiple locations.

     EPA's need for  data standardization was established on the
basis of review of a cross  section  of fifteen  active EPA systems
which were selected by the  EPA Advisory Committee.  This review
resulted in the determination of the types of  data elements
contained within sample systems.  This analysis also identified
the potential for information exchange or data re-utilization
based on commonalities  in naming conventions and coding schemes
for data elements which were representative of the five data
categories reviewed.

     Exhibit II-4 presents a detailed systems category comparison
matrix which lists data element name by category and by sample
system.  This exhibit shows  the  commonality of data elements in
sample systems as well as the differences in naming conventions
which impact data interpretations and  transfer  capabilities.

     Exhibit II-5 presents  a summary of findings for each  data
category.   These  findings are presented in terms of occurrences
in systems reviewed,  number  of different elements, and number of
different coding schemes for each data category. It is apparent
from this more detailed perspective, that while data category
commonalities provide candidates for data sharing, the
proliferation of naming conventions and coding  schemes prohibits
or at least limits this potential.

     The review of sample systems reinforced the concerns
expressed by interview contacts in terms of limitations in the
current environment relative to the  application  of data standards,
limited application of  system design and documentation standards,
capabilities for data transfer, and  current level of data quality.
As a result of sample systems review, EPA's  present  status  with
regard to these issues can  be summarized as follows:

(a)  Data Standards

     Elements with the same name having different definitions
     and different coding schemes are common throughout EPA
     Systems.   Naming conventions for similar data elements also
     vary considerably.  For example, in  the Grants  Information
     and Control System (GICS) the city name refers to the name


                            11-10

-------
Sample Systems Comparison Matrix
                                  EXHIBIT II-4.
1. FACILITY IDENTIFIERS
COMPANY/AUTHORITY
NAMB
case
FACILITY MAME
FACILITY 10 NUMSSfl
It. MONITORING SAW US
STATION SITE

SITS MAMS
srra coos
ill. GEOGRAPHICAL
LOCATION
SSGIGN COOB

STATE COOt

STATS NAMB

CCUNTY COOS

COUNTY NAME
CITY CODE
CITY NAME

AOOBESS

ZIP CODE
OTHER CEO.
IV. PARAMETER UNtT

IOW ii™»*™*

UNIT
V. QUALITY ASSUAANCS
COOES

OA.





X
x




X


x

V



X

X
X
X

X

X
X








x .

I
!
X j X j
x . x ; x :
xx ;
x x : • x



: t
i
x : :
f
I i
x : x \ | x
t i '.
X f X ; j X
* i
X i ! i -
i I 1
x ; i i
: i
X ; ! x
X ! ; i
X • X '• | X
! i •
X • X • i X
: | ;
X : X i ; X
X i X ' X
: ' .

i j
[ 1
X X j |
1 j
' i
! ; '
: | . i



X

x
X



X



x

X



X



X

X

X
X




X



x





X
X




X




X



X

**
X
X




X




X

.

X



X i
X ,
X X
X : X

1
r
i

!
1

X
i
X I X

!

X

x
'
X j X

•: x
1
X >
X ; X








X
1




x •
X







X

X



X

X
X
X




X




X







X
X

x







X

X

..





X

X

X













i

X .
X



i „
i
X

!
x
I
x : x



X X

x
X X
X

X X


X X

t


X


j







X




X




*



X


X





X




X





-

X
X
X
X







X

x



X

X
X
X

X

X









X


-------
                                          EXHIBIT II-5
09
a
2
a

ut
uu
a


' NOTES
Ul
e
ui M
a u
u. —
o 
-------
     of the  city in which the headquarters of the applicant is
     located, however, in the NEEDS system the city name  is the
     name of the city or town in which  the  facility is physically
     located.

     The National  Bureau of Standards  (NBS)  has been committed
     to a policy of adopting voluntary ADP standards developed
     by the American National Standards  Institute  for use in the
     federal government.  There is not  uniform adherence to these
     standards  in  EPA  systems.

     EPA is currently  engaged in  several projects that deal with
     the development and implementation  of data standards.  The
     Inter-agency  Regulatory Liaison Group (IRLG)  Common Codes
     Project  is a  good example.   The objective of  this project
     is to  identify specific data elements within selected key
     data element  categories, and recommend preferred naming
     conventions and coding schemes for  the elements.  The
     proposed  system of common  codes would be applied to new as
     well as existing  information systems  which are of common
     use to  the four member IRLG agencies.

     Data standardization is also being  undertaken at a
     programmatic  level.  The Wastewater Treatment {WWT)  Program
     for the Of-fice of Water Program Operations (OWPO)  is
     currently  engaged in this  type of initiative. The WWT  data
     management and standardization program will utilize tools
     such-as cross reference files,  standard  definitions  and  a
     data element  dictionary/directory.

     While  data standardization attempts can  be facilitated by
     aggregating and analyzing  the attributes  of data components
     in systems, there  is currently  no central location  for
     metadata within EPA to afford this capability. Some offices
     maintain their  own data  inventories, but  there is no central
     data reference source  such as a data  element
     dictionary/directory.  There is, however, a current study in
     which  a common facility identification coding scheme for
     facilities monitored by the EPA is  being developed.  The
     resulting  facility file can serve as  input to an EPA-wide
     data dictionary/directory  system  which is currently under
     consideration.

(b)   System Design and Documentation Standards

     On an EPA-wide basis,limited consideration has historically
     been given to the  need for data management or standardization
     with respect  to systems design and  documentation for
     proposed systems.  Some offices have'established and enforce
     system design and documentation standards on their  own,
     however, the  Agency  as  a whole  does not  strictly enforce
     established standards.   The  impending potential for massive


                           11-11

-------
          conversion following the 1980's  ADPE  procurement has
          emphasized the need for documentation standards.  There is
          an effort by MIDSD currently underway to develop  new Agency-
          wide standards for system documentation.   Upon preliminary
          review, this appears to be a particularly practical approach
          in that it relates the rigor  of the standard required to
          the size, life  expectancy and scope of the system.

     (c)   Data Transfer

          There are two  forces in EPA that are leading to increased
          interest  in the sharing of data between offices and among
          Headquarters, Regions and States.  The first is the increased
          interaction between programs  (e.g.  construction  grants  and"
          NPDES permits;  water enforcement and WWT facility operation
          and maintenance;  and multi-media new source tracking.)  The
          second force  is the increased delegation of programs to
          States by Regions.  Current lack of  standardization presents
          difficulties  in sharing  data  across systems, as well  as data
          interpretation and data quality.   Improved mechanisms are
          required  to aid EPA. Headquarters and Regions in  the effective
          management and utilization of data.

     (d)   Data-Quality

          Interview contacts consistently expressed a lack of
          confidence in  the quality of available data.  Audits of
          individual system files provide documented evidence of
       •   incomplete data,  and data inaccuracies.  Not only does poor
          quality impose a manual burden on generation of major reports -in
       -   assuring  accuracy and completeness, but there  is the
          potential for  reporting inaccurate data to other Federal
          regulatory agencies or to the  public.  There is not currently
          an active Agency-wide quality assurance program  so that any
          activity  is left up to the initiative of individual system
          managers  or program offices.  When activities are undertaken, •
          they are  usually crisis oriented directed  at merely cleaning
          up existing data rather than correcting the causes for poor
        .  quality.   There is currently a Blue Ribbon Monitoring Group
          in EPA, chaired by Richard Dowd, which is studying the many
          aspects of monitoring data.  One of the major considerations cl
          this group is the quality of this monitoring data and the
          confidence level of the data that is available.

3.    OVERALL EPA REQUIREMENTS

     A data management and standardization-program for EPA  is directed
at several objectives which will facilitate management of
environmental programs and help to maintain a positive relationship
between the Agency  and the public.  Key among these objectives are:

                                11-12

-------
          Improved data quality to provide a basis for sound decisions,
          to minimize the use of scarce personnel resources for data
          collection and validation, and to permit efficient use of
          automated capabilities

          Facilitated data transfer  to  permit effective communication
          between related programs,  accurate aggregation of multi-
          media data, and  sharing of capabilities to reduce redundant
          system development

          Reduced reporting burden on  industry, the states, and the
          Agency's own regions so that  a positive relationship  can  be
          maintained and resources of all entities can be applied more
          efficiently.

          Development of a focal point  for  answering  inquiries to
          respond to the increased visibility of the more mature
          programs in the Agency.  This will help to assure accurate
          and complete response to requests for information.

     Current capabilities in the Agency do  not adequately  support
these objectives.  It is concluded  therefore, that  a need does  exist
for a data management and standardization program for EPA.  This
conclusion is based on the fact that both perceived and  documented
benefits of data management and  standardization provide"potential for
resolving current data accessibility and information exchange problems
experienced by EPA.  Since there is a need to share data across systems,
it is important that these systems adhere  to the same standard data
definitions, coding schemes, and documentation techniques.  The systems
that currently support the programs in  EPA  are under  the control of
the various offices which support programmatic functions. Enforcement
of standards across these systems therefore requires coordination
between Programmatic areas.   A  data management program can provide
the mechanism to permit the required data  sharing.  This program can
also provide a medium for the coordination  which is required to plan
for new systems or enhancements to existing ones.

     Because data management involves  all  aspects of data, from its
collection to its final transformation  into information  products,  it
offers the prospect of enhancing those products  in several ways.  First,
improvements in the validity,  reliability and timeliness of data will
be directly reflected in the corresponding  information products.
Second, information products viewed apart from their respective systems
may become candidates for standardization, and  standardization may
make the information  both more meaningful and more desirable.  Finally,
when information products are viewed across the Agency,  there  arises
the possibility of .two or more such products being recognized  as
complementary.  Actions may then be  taken to enrich the products in
some manner of combination.

     As described  in Section 1,  the  component of a data management
program are policies and procedures, tools, and a  data management


                                11-13

-------
t
         organizational structure.   Alternative approaches for policy and
         organization are described in Chapter III.  Highlights of the .tools
         required for EPA are described in the paragraphs which follow.

              (1)   Standards

                   The development, maintenance and rigorous application of
              standards are required for:

              .     data elements

                   data acquisition

                   system design and documentation.

                   There is a clear need to standardize data elements to permit
            .  the  necessary sharing of data between related  programs and to
              help improve data  quality.  The  standardization  may  be  limited
              to  key  data elements  which  serve as linkages to  related systems,
              or  could be targeted  for the  entire Agency base  of data.  A high
              priority candidate for standardization,  for  example, is the
              facility or authority identification  data category which is a
              common  element in  the  majority of the Agency's  larger systems.  A
              primary mechanism  for effecting  data  element standardization is
              the  data element dictionary/directory described  in Section 1.

                   There is also a  current  need in  EFA to enhance  data
              acquisition procedures.  Enhancements  to  the  current  approval
              process for reporting requirements could include:

                   Verification  of  standard names,  definitions, and coding
                   schemes,

                   Combination and  reduction of the number of  data collection
                   forms,

                   Re-utilization of data already collected

                   Verification  of  the inclusion of quality  assurance elements,

                   Verification  of  proper economic  analysis  to measure the
                   cost-effectiveness of  data  collection techniques.

              Data acquisition processes  should involve  the Regions as well as
              Headquarters.  Since the Agency requires an increasing amount of
              data from the States  and Regions, it  is  necessary to establish
              and  enforce standard  methodologies and procedures for  data
              acquisition.  This will help to assure quality  and compatability
              of  the  data received  at headquarters.
                                         11-14

-------
     Standard procedures should be developed  and  followed  for
all stages in systems development.  The approval process for
system development must include an in-depth look  at  the  system
software in order to evaluate the possibilities of using
standardized packages or subroutines.  Any improvements or
modifications to the existing hardware or software should  be
approved through the data management program.  Without such
approval, systems would be modified using non-standardized
methodologies resulting in little or no possibility  for  data
transfer or interface.

     Policies and Procedures for the standardization of  system
documentation also need enhancement and enforcement.  These
policies should specify the types of attribute  data to be included
as well as basic formats for the documentation, which should be
user-oriented and easy to understand.

     Standard programs and subroutines, such as conversion from
English to metric units of measure, should be catalogued during
the approval process. This would provide an inventory of programs,
and would reduce any programming duplication.  These programs
could be catalogued in a Data Element Dictionary/Directory
(DED/D)which would serve as a centralized systems reference
source.  The  OED/D would use  its capability  as a copy library  in
cataloging these standardized programs and  subroutines.

(2)  Data Element Dictionary/Directory

     Due to the increased visibility of EPA,  an increasing number
of inquiries are coming in from various entities  such as:

     Congress

     States

     Public (FOI)

Thus, an increasing amount of resources is needed to respond to
this ever-increasing amount of inquiries.

     Currently there is no mechanism available in EPA that
facilitates timely reference of available data resources.  EPA
therefore needs a central repository of information  that can  be
used to locate data relevant to any program within the Agency.
This requirement can be satisfied by establishing a  centralized
Data Element Dictionary/Directory.  The Dictionary should  contain
a list of all key elements relevant to the  EPA programs, along
with a definition of each element including any special  coding
schemes.  The Data Dictionary can  be supplemented by a Data
Directory which will meet identified requirements to establish
a mechanism which provides information on how data can be
accessed.  The Data Directory will identify location of data by


                           11-15

-------
     systems.  This combination of a data dictionary and directory
     will not only provide a focal point for locating data, but provides
     a consolidated reference source which can be accessed to determine
     standards utilization.  This will aid in the identification of
     standardization candidates, or  enforcement of existing standards
     use.

     (3)  Quality Assurance

          Quality assurance is another  important  tool  in  a data
     management program.  The  prevalent lack  of confidence in  data  in
     Agency systems clearly demonstrates the need for such a program.
     A quality assurance program must  be directed at not only cleaning
     up existing data but also  at removing  the causes of poor quality.
     These causes can be  identified through system and  procedural
     audits, and  some can be removed  through improved data entry forms,
     comprehensive data input software edits, and clarification of
     data element definitions.  However, a major factor in poor  quality
     in the Agency i's the circumstances which  require  the  Regions  to
     report data to Headquarters which is  not  data which  the  Regions
     require for their own operations,  when this circumstance truly
     exists, imaginative alternatives to acquiring data must be
     considered.  For example:

          The Regions may be helped  to discover the  usefulness of the
          data to themselves as well  as to Headquarters

          Contractors may be used to  enter the  data  under  direct
          responsibility of Headquarters

          Sample data collection may  be determined  to  be  sufficient
          rather than Agency-wide data collection on a  recurring basis.

          Since an effective quality  assurance program  does require
     significant funds, an efficient plan  for addressing the program
     which emphasizes long-lasting  solutions and  self-correcting
     procedures must be developed and management  commitment obtained.

     These requirements, form a basis  for  the data management  and
•standardization program for EPA.  The next chapter describes
alternative strategies  for  statisfying  these requirements.
                                 11-16

-------
Alternative Analysis

-------
III.  ALTERNATIVES  ANALYSIS

-------
                   III.  ALTERNATIVES ANALYSIS
     In this Chapter, Arthur Young & Company presents  the  analysis of
alternatives for the major  components of a  Data Management  and
Standardization Program in EPA.  An effective EPA-wide data management
and standardization program must  consist of three major components:

          an organization . structure  which supports program
          implementation and administration;

          data management tools such as standards, data element
          dictionary/directory (DED/D), feasibility studies, and
          quality assurance programs which  produce program  products;
          and

          policies and procedures to govern  program operations.

Each of these components is discussed in the  following sections.  Where
alternative approaches exist, the  discussion will focus on  the
alternatives'  attributes  and potential effects.

1.   ORGANIZATIONAL'STRUCTURE
     EPA requires a somewhat different approach to the  development of
a Data Management and Standardization. Program  than  is  advocated  in
the text books.  The advocated  approach  is  to define  the requirements
for the tools, develop the policies and procedures that will best
implement and control the tools and then define the  organization to
administer the policies and procedures and operate  the tools.  The
formal organization in EPA requires a modification  to  this  approach
because certain alternatives for a data management program  framework
are not possible.  Specifically, the multi-faceted nature  of the
Agency's programs as well  as the  line-staff relationship between
Headquarters, Regions, and Laboratories mandates division  of
responsibility that allows only coordination, not~dictation of efforts.
The effects of the Agency structure on each alternative will be
discussed along with the merits of each alternative.

     Traditionally AOP management has looked at the  alternatives for
control as being either centralized, decentralized or distributed.
These concepts have been primarily applied to  computer management.
The concept in favor has been dependent on the state-of-the-art  in
hardware.  Large computers with advances in telecommunications and
remote access equipment  led  the move from decentralized  to centralized
computing facilities and control.   Recent advent of mini and micro
processors with further  advances  in telecommunications  is  leading the
move towards distributive systems.
                                III-l

-------
     The principle behind each control concept are:

          Centralized - authority and responsibilty, being placed at
          the highest common point in the organization to effect
          maximum control

          Decentralization - authority and responsibility being
          located as close to the level  of the  organization  where  the
          actual work takes place to speed the decision-making process.

          Distributive - locate the authority and responsibility  at
          the level in the organization that is best suited to perform
          the specific task.

     The same trends are being followed in the recently recognized
area of information resource management.  The concepts are the same:
decentralized, centralized, and a  form of  distributive that  we  have
named hierarchical.  Our discussion of each concept is placed in terms
of the SPA structure.   Exhibit III-l  presents a graphic overview of
the three structures as they might apply to EPA.

     Included in the discussion of the alternatives  is the evaluation
of the alternatives potential and impact in EPA.  In preforming the
evaluations we have considered the inherent ability of each alternative
to obtain:

          Top management commitment

          Programmatic participation

          Balance of authority and responsibility

          Control and coordination of EPA information resources.

It should be noted that,  in  the  discussions of the concepts  of
Decentralized and Centralized Organizations,  we are  looking  at the
extremes of the spectrum.  Actual implementation is  typically less
stringent.

     (!)  Decentralized Organization

          In the decentralized organization structure, data management
     responsibility would be assigned at the programmatic level under
     the control of  the Deputy Assistant Administrator who has primary
     responsibility for a particular program.  Program management would
     be controlled by an Oversight Committee that would coordinate
     with ADP support  staffs  for the program components to assure that
     data management program  concepts were properly applied throughout
     programmatic level operations.  The Oversight Committee would be
     responsible for the setting and promulgating of program policies?
     review of feasibility studies, system designs and
     implementations;  oversight audits to assure standards compliance;


                               IIX-2  -

-------
                                                   EXHIBIT  III-]
       3



       I
N
z

s
I
i
       Itt
       
-------
and system coordination to assure that programmatic  requirements
are met.

     The ADP support staff, which may be one centralized group
for the program or separate staffs for each system  supporting
the program, would be responsible for the performance of
feasibility studies and systems design and  implementation.  In
addition, they would be responsible for  updating data element •
detail in individual systems catalogues and/or DED/D.  Systems
groups would also be responsible  for performing  systems  audits
to assure data quality/ and system evaluation to assure that
system requirements were being met.

     As can be seen in Exhibit III-2, an overview of the
intersystem life cycle data management for  this  alternative
organization structure/ no EPA-wide management of the data
resource is included in this alternative.  With  the  inclusion of
some MIDSD oversight of the systems, this  alternative  approximates
the current status of information processing  in  most of  EPA.

     This alternative does not support EPA-wide  requirements for
data management and standardization since program activity and
control is confined  to the programmatic level.  While  improvements
to data quality and accessibility may be obtained on a
programmatic basis/  the benefits to be  accrued from data
management and standardization are limited  to the degree of
program implementation which is chosen to meet  the  requirements
of individual programmatic units. As an organization, EPA's need
to share data, minimize system development and change efforts,
improve overall data quality, reduce reporting burdens, and meet
external reporting or information exchange  requirements  cannot
be met. This alternative is therefore infeasible for use  because
it does not meet the defined criteria for an  effective EPA data
management and standardization program.

(2)  Centralized"Organization

     In the centralized organization  structure, all EPA-wide data
management responsibility would-be assigned to  a Data
Administrator  (DA) and staff which reports  directly to the
Administrator.  The  Data Administrator  would be  responsible for
the setting and promulgating of program policies and the review
of feasibility studies, and system design and implementation. The
Data Administrator's staff would be responsible  for monitoring
the use of program policies; performing feasibility studies and
system design  and implementation, or managing contractors who
perform the projects; updating the EPA-wide data element
dictionary/directory; auditing systems for  standards compliance
and data quality; and conducting  system evaluations to assure
that both EPA-wide and programmatic requirements are met.
                           III-3

-------
                                       EXHIBIT III-2
< «
/*» *•"
9 a

as

ii
en
    il
    to IS
     s

     Ul

     §
                    I!
                    JTo

-------
     Under this alternative organization structure, the
responsibilities of the programmatic staff would be limited  to
performance of system operations-.  Exhibit III-3 on the  following
page presents an overview of the intersystera life  cycle data
management for this alternative organization structure.

     This alternative supports almost all the EPA-wide
requirements for a data management and standardization program.
It does not, however,  support  the need  to encourage active
participation by programmatic areas.  In fact,  current
programmatic level data management responsibility  is reduced to
maintenance of ongoing system operations.  This alternative
prescribes a complete reorganization of EPA-wide data control
responsibilities and the creation of an additional Data
Administrator position and supporting staff.  The newly formed
data administration function requires high-level, Administrator
staff positioning to give legitimate power  to the strong role of
the Data Administrator required in this alternative.

     The logistics for the implementation and ongoing operation
of this alternative program strategy are both complex and contrary
to EPA's  current  programmatic  oriented  organization structure in
both the Regions and Headquarters.   The effect  is intensified by
the relative independence of the Regions. Since programmatic  and
Regional support of a data management and  standardization program
is essential to program success, an alternative which does not
facilitate and encourage such support may never  achieve its
potential benefits due to lack of participation and cooperation.
This alternative is therefore infeasible for use since it does
not have  the potential for effective  implementation in the current
SPA organizational environment.

(3)  Hierarchical Organization

     The hierarchical organization structure provides  for both
an ADP Oversight Committee which reports to the  Administrator,
and individual oversight committees for each program..  The ADP
Oversight Committee which reports to the Administrator would be
responsible for the  setting and promulgating of program policies.
A Data Administrator and staff would assist the  ADF Oversight
Committee in the documentation of policies and development of
supporting procedures.  The  Data Administrator  and staff would
also support the ADP Oversight Committee by performing review
functions for feasibility studies,  and system design and
implementation; updating key elements in the EPA-wide
dictionary/directory based on input from programmatic areas;
perform system audits  to assure standards compliance; and conduct
system evaluations to assure that EPA-wide requirements were
being met.

     The Programmatic Staff would retain current responsibilities
for feasibility studies, system design and  implementation, and


                           III-4

-------
                                 EXHIBIT HI -3
iu Z

11
^ IM
Sd
i
"5
 ut
,
 a
ss
3°
 SH,1
 Hi
 3 w

 II

 
-------
system operations.  In addition,  they would be responsible for
maintaining a detailed program specific DED/D and providing  the
OA's staff with the metadata needed for the Agency-wide
dictionary/directory.  (Details on the DED/D  alternative  follows
in the Tools Alternative  section.)  Programmatic areas would also
be responsibile for auditing data quality  and performing  system
evaluation from the perspective of programmatic requirements.

     An oversight committee would be formed for each programmatic
area to monitor the adherence to programmatic level  data
management concepts and provide valuable input to the EPA-wide
ADP oversight committee in  terms of additional policy and
procedures requirements and data management  program  operations.
Exhibit III-4 on the following page presents an overview  of  the
intersystem life cycle data management for this alternative
organization structure.

     This- alternative supports the EPA-wide  requirements  for  a
data management and standardization program.  Responsibilities
for data management are assigned on a hierarchical  basis  so that
program objectives are properly addressed  by the levels of EPA
management which can best assure and control that these objectives
are met.  Participation is encouraged from programmatic areas
through oversight committees which are represented on the EPA-
wide ADP Oversight Committee.  The ADP Oversight Committee then
reports on program activities to th Administrator.  A Data
Administrator and staff assist with required EPA-wide review
processes, and monitor  the administrative aspects of the program.

     In this alternative strategy, both EPA-wide, programmatic,
and regional level data management and standardization needs  are
accomplished with limited changes to  current organization
structure or responsibility assignments.  This alternative will
also provide one focal point for initial contact with external
agencies.  In this  context, the program will provide an effective
mechanism for interagency information exchange such  as that
anticipated by  the  IRLG common codes effort. Already the OWPO/WWT
long-range plan is establishing the first  programmatic/office
sub-groups which provides a pilot implementation of  data
management concepts for the agency which can effectively  feed
into an EPA-wide program implementation.

     This alternative program implementation strategy therefore
appears to provide the most effective data management program
implementation strategy for the current EPA  organizational
environment.  In some areas EPA is already developing
organizations which incorporate many  of the  data management  and
standardization concepts contained in the  above alternative
program implementation strategies.  What is needed, however,  is
a coordinated program  implementation  effort.  How this
coordination can be achieved,  to what  degree  it must be employed,
and the speed at which it can be done can  be further assessed


                           III-5

-------


H
li
Ui

i


§ Ii
II
< S
N. ii
u >
s £j
O UJ
< = i
« -S -:
U4 < T •
« • M "2 i
g al
U4
5.
| ]


1
!



'
.




!

»







,





































-










-1
£
H
tl

























1
A
• -8
1
1



1
S
i











i


S
h
J
.



i'
«

i
|
s

3
|


i
i
I
ii
A b



-












, *







X Z
5 u
i!
- k
if
t

1
T
ill
"i5
4 k
3
J
a
£

S
1

t ,


|








|
C













EXHIBIT III-4


-------
     based on findings of the Nolan, Norton and Company study to
     determine EPA's current position in the organization-wide  system
     development life cycle.

2.    DATA MANAGEMENT TOOLS
    .In Chapter III we identified the major tools and  their
characteristics that are required for an effective data management
program in EPA.  These tools already exist  in  some  form in the  Agency
or are currently under development.  The alternatives in regard  to the
tools are primarily in the  delegation of responsibility and authority
for the different phases of the tools' development and application.
The degree of delegation is dependent on the organization structure
selected. Exhibit III-5 presents the recommended degree of delegation
for each structure.

     One set of alternatives that should be addressed  with regard to
EPA data management tools is the approach for the Data Element
Dictionary/Directory  (DED/D).  The pertinent alternatives deal with:

          Centralized vs Hierarchical DEO/0

          Active vs. Passive DED/D

The considerations for each of the alternatives are discussed  in the
following paragraphs.

     (1)  Centralized vs. Hierarchical DED/D

          There are a number of. data dictionaries and
     dictionary/directories currently in use in  EPA.  Most of  them are
     systems oriented like  GICS and STORET.  There are  several program
     or multiple system dictionaries available  such as AERQS in  RTP.
     This needs to be a consideration in evaluating  the alternatives.
     The first option is to retain these dictionaries  with their
     various formats, contents, and media and  to overlay an Agency-
     wide directory.  The Agency directory would be organized by  data
     category and contain only key identification data, or metadata,
     on data elements in the programatic data bases.  The  type of
     information that might be carried  in the Agency directory would
     include:

          Data element name with brief definition

          Programmatic DED/Ds that have detailed metadata  on element

          Systems on which the element  is carried

          Contact name and  phone  number  for  both relevant  syste
          DED/Ds.                                     .       "
                                III-6

-------
                                        EXHIBIT III-5

= 0
p e.
Ess
o88

z =
o =
E =
U U
S
bU
UJ



RCHICAL
<
e
UJ
/

a
Ul
N
3
<
e
IU
u


O
IU
rjj
<
"1
3
S
ui /
p" /

« /
iu /
*j /

^
2
a
a

al-
5«3 2
III
•• EPA-WidaSl
byADPOwai
• Eolorced by
•M <9
< 5
i S<
i s®
I a 1*
« OS'S
fill
ill!
w i ;
.1
g 2
? WI
||
Pi
£ 5.1
ui ca iu


OS
i
1
' -S



? G
•"* <
- s -
.331
s « .a as
| »< 0 £2
Illlrli
|li!li«°
^w *^ Z *^
iu a i £ a i
i
^ Ut
1 s 1
ill
:°!
o **
| 
£ <*
|i



,
2 »
a u
I 11
I r, 2
s > «>
U « 3
** 3 ^
1 ft
s . S §
sf « «
£l2!

M
i
f
1
J
<
a



a
I i
II

•
Ul
^1
II
0|




-------
The objective would be to use the Agency directory 'as  a  central
location to direct inquiries to the  individuals and  system  that
can best answer the questions.  For example, if a  congressman
requested the number and dollar  amount of waste treatment plants
under construction in his district, the Agency directory would     •
direct the inquiry to the proper person or  persons in  the OWPO,    I
Grants Administration and/or the Regions who, according  to  the
directory, would have the information. Each systems/program would
be responsible  for maintaining their  own DED/D  and supplying
relevant metadata  to the group charged with maintaining the Agency
directory.

     The second alternative  is to develop a  completely integrated
Agency-wide DED/D.  MIDSD has initiated a pilot project,  with
contractor support, to test  this  concept.   The  advantage  of  this
approach is the central  location of  all metadata  that  facilitates
the quick location of information, the monitoring of data definitic
and coding scheme standards, and the uniformity DED/D  content.   Th«
disadvantage is that a Agency-wide DED/D will be massive  in  size
which will require a major effort to  maintain.  Thus, potential
for problems is compounded when the maintenance  is done  by people
not familiar with the actual systems.  In addition,  under this
approach the programmatic areas lose  proximity  to the  DED/D.
Experience has  shown that the program areas are more committed
to capabilities which are directly under their  control.  The
effect may be that changes in systems  that require  DED/D updating
will not be reported.

(2)  Active vs; Passive DED/D

     The primary difference between  an active  and passive DED/D
is that the active DED/D  interfaces with the programs to simplify
the maintenance of both  the DED/D and the  application  programs.
A change in either field  description  or coding schemes is  entered
into the DED/D  then the  various application programs that use
the file and the coding schemes are automatically updated.  This
is a sophisticated firmware package  concept that  will  require
significant modifications to all current programs.  The advantages
of the active DED/D in reduction  of maintenance- and assurance of
standards in system development are  such that  it  should  be
seriously evaluated for  inclusion in  the 1980's Procurement. We
do not recommend  the active DSD/D at  this  time  because of the
massive effort  that would be required to modify the  application
programs to permit the use of an active DED/D.  In addition, for
an active system  to be fully useful, standard data definitions
and coding schemes need to be in effect. The 1980 ADPE  Procurement
affords an excellent opportunity for  implementation  of both
comprehensive data standards and  an active  DED/D because  a major
conversion effort is anticipated to operationalize the current
programs on the new hardware.  While  the programs are being
modified for the new operating system, the   incremental  cost  for
modifying or including source code to effect standards and  the


                          III-7

-------
     active DED/D will probably be more  than offset  by  the  benefits
     to be gained by each.  A study should be performed to  test this
     assumption.

          Our recommendation for  the DED/D is to  implement  a  passive
     dictionary/directory on a hierarchical basis.  This allows the
     utilization of the current DED/Ds in operation  in  EPA  with  the
     development of a centralized directory to be  used  to point  to
     specific data elements  in the detailed programmatic/system DED/D.
     Standards should be set for the development of additional DED/D.
     The current MIDSD effort to build a DED/D for selected systems
     is the vehicle for not  only setting  standards but for eventually
     providing DED/D capabilities to the other programmatic areas.
     Finally, we recommend a detailed feasibility study for the
     implementation of, an active DED/D in conjunction with  the 1980
     ADFE Procurement.  This study should be conducted relatively  soon
     so that the ADP specification for  supporting an  active  DED/D can
     be included in the RFD for the Procurement, if the study deems
     the system feasible.

3.   POLICIES AND PROCEDURES
     As stated earlier, the policies and procedures needed to implement
and operate a data management program are dependent on  the  level  of
information management desired, the delegation of responsibility and
authority throughout the agency, and the  exact nature  of the tools to
be implemented.  The development and implementation of policies and
procedures must be of primary priority to the ADP Oversight Committee.

4.   RECOMMENDATION
          The specific recommendations in regards to  the  tools  used
in the Data Management Program are not new to SPA.  Most, if not all,
are. in one stage or another of development or operation  currently in
sections of EPA.  They'range from pilot efforts that have been initiated
by EPA since the inception of this project to fully active  programs
that have been successfully in existence  for years. What  this project
has accomplished is to  identify these  isolated programs and to prepare
a framework for coordinating and promulgating these efforts throughout
SPA.

     (1)  Organizational"Structure

          The organizational structure for the data management
     program, with  its corresponding responsibilities and  authorities,
     is the basis for pulling together an effective program in  EPA.
     In performing the evaluation of  the alternative  organizational
     structures, we had to consider  the inherent ability  of each
     alternative to obtain:

          Top management commitment,
                                II1-8

-------
      Programmatic  participation,

      Balance of authority and responsibility,  and

      Control and coordination of Agency-wide information
      resources.

      Our  recommendation  is a  hybrid of the classical centralized
 and  decentralized  management  approaches that we call
 hierarchical.  The basic  philosophy is analogeous to distributive
 processing  in computers  - locate the  work at the level in the
 organization best  suited to perform the specific tasks.

      The  hierarchical  organization structure provides for both
 an ADP Oversight Committee which reports to the Administrator,
 and  individual oversight committees for each program. This
'concept  is  presented graphically in Exhibit III-6.   The  ADP
 Oversight Commitee which reports to the Administrator would be
 responsible for  the setting and promulgating of program policies.
 A Data Administrator (DA) and  staff would assist the ADP Oversight
 Committee in the documentation of policies and development of
 supporting  procedures.   The Data  Admnistrator  and staff  would
 also support the ADP Oversight Committee by performing review
 functions for feasibility studies,  and  system  design and
 implementation;  updating key  elements in the EPA-wide data
 element  dictionary/directory  based on input from programmatic
 areas; perform system  audits  to assure standards compliance; and
 conduct  systems evaluations to assure that EPA-wide requirements
 were being  met.

      The  Programmatic Staff would retain current responsibilities
 for  feasibility studies, system design  and  implementation, and
 system operations. In addition, they would be responsible for
 maintaining a detailed program specific data element
 dictionary/directory and providing the  DA's.staff with the data
.needed for  the Agency-wide dictionary/directory.   (The data   .
 element  dictionary/directory  concept  is described in Section 3.
 Details  on  the DED/D alternative which  follows.)  Programmatic
 areas would also be responsible for auditing data quality and
 performing  system  evaluation from the  perspective of programmatic
 requirements.

      An oversight  committee would be formed  for each programmatic
 area to  monitor the adherence to programmatic level data
 management  concepts and  provide valuable input to the EPA-wide '
 ADP  oversight committee  in terms of additional policy and
 procedures  requirements  and data management program operations.
 Such a programmatic oversight committee is being formed by OWPO
 to  support  the Wastewater Treatment (WWT) Facility program.   We
 recommend that the chairperson of the programmatic committee be
 a member  of the Agency Committee.
                           III-9

-------
                                EXHIBIT III-6
Hierarchical Organization Structure

-------
                (2)   Data Management Tools

                     We recommend that priority and support be given to the
*  ^.,            current projects that are developing individual tools for
                specific programs in EPA.  These projects should be viewed as
                pilots upon which an Agency-wide program  can be built. With this
                perspective it is necessary that the projects be monitored to
                assure that they meet the guidelines and  objections of the tools
                as described in Chapter II.  The specific action that should be
                taken for each tool is discussed in the  following paragraphs in
                the context of the recommended organization.

                {a)   Standards

                     There are efforts currently underway in each area of
                standards that must be supported.   They need to be coordinated
                such that they are ready for Agency-wide implementation before
                or during the 1980's ADPE Procurement conversion.

                     Data Standards - There are a number of efforts going on in
                     EPA dealing  with data definitions,  coding schemes, and naming
                     conventions.  They include:

                          MIDSD Data Element Dictionary/Directory Project

                          IRLG Common Codes Project

                     -    Interagency Toxic Substances Data Committee

                          Region 2 Facility File Pilot Project

                          OWPO Long-Range ADP/IS Plans

                     They need to be coordinated to assure that the codes and
                     definitions established in one effort are compatible with
                     the other projects results.  The IRLG,  Region  2  and OWPO
                     projects are.all dealing with  facility/authority/establishment
                     identifiers which must result  in  the same or compatible
                     coding schemes.

                     Our sample of systems and projects  in EPA was not exhaustive.
                     It is very likely that there are other efforts going on to
                     develop special coding schemes and definitions  for specific
                     program areas.   A study needs to be  instituted to  identify
                     all these projects and to identify those projects developing
                     codes and definitions for the same data categories.  Efforts
                     should then be made to coordinate the development.   In
                     addition, other data categories not  currently being
                     addressed, but  that  should  be standardized need to  be
                     identified and 'work begun on their  standarization so that
                     they are ready  for the 1980  Procurement  Conversion. As noted
                     in Chapter II, not all data  elements  used to be standardized.


                                           111-10

-------
What may be needed is a method of relating data, such as
cross-reference indices or conversion tables for measurement
units.

System Design and Documentation Standards - An effort to
prepare new standards is being conducted by MIDSD in
preparation for the 1980 Procurment.  We did not review this
project in detail, but the concept of having three (3) levels
of detail documentation dependent on the use and complexity
of the system is  an excellent idea that has  worked extremely
well in other agencies. This effort should be
enthusiastically supported by top management with each AA
assuring that his area meets the standards by the 1980
Procurement Conversion. Not only  is  this good on an Agency
level but it will help minimize the description and potential
errors from ADPE  conversion that could seriously impact the
operations of the programs under each AA,  It is in their
own best interest to assure that the standards are met by
conversion time.

Data Acquisition Standards  - This is the area where we found
the least activity.  There  are isolated efforts addressing
some aspects of data acquisition.   For example:

     Forms redesign - Operations and Maintenance in OWPO

-    Review of reporting impact of new requests on Regions
     - Program Reports Division

     Coordination of OMB approval for new forms - Program
     Reports Division

     Evaluation of document handling and tracking - OTS and
     OPP

     Data Transfer between COE and Regions, Regions and
     Headquarters, WWT program and permits - OWPO Long-Range
     ADP/IS Plan.

These efforts are addressing individual  areas that are a
problem to the specific groups.  A coordinated effort is
needed to address all areas of data acquisition in a
systematic fashion.  As stated in  Chapter II,  these areas
include:

     Data collection request and approval procedures to
     reduce duplicate data collection

     Forms design and instruction writing guidelines to
     assure uniformity and  clarity in data collection forms
     and definitions
                      III-11

-------
          Document tracking procedures to provide an audit trail
          for locating stalled or lost source data

          Key verification techniques to control the accuracy of
          data entry

          Control and hash totals to assure complete and accurate
          processing of source data

(b)  Data Element Dictionary/Directory

     Our recommendation is to implement a passive data element
dictionary/directory (DED/D) on a hierarchical basis.  This allows
the utilizaiton of the current DED/Ds in operation  in  EPA with
the development of a centralized directory to be used  to point
to specific data elements  in the detailed programmatic/system
DEO/D.  Standards should be set for  the development  of additional
DED/D.  The  current MIDSD effort to build a DED/D for  selected
systems using IDMS is the vehicle for  not only setting  standards
but for eventually providing DED/D capabilities to  the other
programmatic areas. Finally, we  recommend a detailed feasibility
study for the implementation of an active DED/D in  conjunction
with the 1980 ADPE Procurement.   This study should be  conducted
relatively soon so that the ADP specification for  supporting  an
active DED/D can be included in the RPQ for the Procurement,  if
the study deems the system feasible.

(c)  Feasibility Studies

     The current procedures in the ADP Manual and those  practiced
by MIDSD appear to be quite effective.  The proposed Mini-computer
Review Group comprised of the Regional and Laboratory  site ADP
Managers can maintain effective control on standards for the
Regions provided sufficient interaction with national  level
personnel is maintained.  Our recommendation in this area is that
the Agency ADP Oversight Committee  endorse the review process  by
MIDSD and that the Committee members from each programmatic area
personally see that the procedures are followed for  their
programmatic systems.

(d)  Quality Assurance Program

     We have addressed the problem of poor quality  and lack of
confidence in even some data that appears to be good.  There are
numerous efforts going on  in Headquarters and the Regions  to
"clean up" the various data bases.   These include the GICS  and
NEEDS systems in the Wastewater Treatment (WWT) program.
Reconciliation and monitoring in the Financial Management System
and numerous other efforts. What is needed is a coordinated
effort supported  by top management to first get all systems groups
to initiate such system audit activities and second  to assure
that the programs do not focus just  on "clearing  up" the current


                           111-12

-------
t
files but address and correct the causes that allowed  bad  data
to get out in the files in the first'place.  As identified in
Chapter II the reasons can usually be traced to one of the-
following:

     Lack of incentive on the part of those entering the data

     Confusion as to what data is required - content or  form

     Lack of effective software edit capabilities.

     The steps that can be taken to  improve the quality of the
data include:

     Inclusion of Effective Software in Systems — edit and update
     procedures are the final line of defense against  bad  data
     being stored on files.   They must be well  conceived and, at
     times,  imaginative.

     Standardization of Data Definitions — standard definitions
     eliminate confusion on data to be  collected.

     Revision of  Forms and Procedures — forms and procedures
     need to be easy to fill out and follow.  Any ambiguity or
     unnecessary  complexity should be eliminated.

     Introduction of New Incentives —  The data collector  must
     be given an  incentive to collect data.  The best method is
     to offer reports or data access techniques that are of use
     to the  collector.  If this cannot be done then it is  necessary
     to explore alternative means for collecting the data.

     Acquisition  of Contractor Support  ~ It may be impossible
     to provide incentives to the current collectors for all or
     part of the  collection and data entry process.  If  this
     occurs an alternative is to contract the work to  private
     organizations.  This is often done for data entry where
     incentive fees can provide the motive for accurate  data
     entry.   This method has been used  in the NEEDS SURVEY, for
     the initial  collection of the data.

     Periodic audits-should be performed and mechanisms should
be built into the systems to identify problems.  For example, a
counter in the edit program for  a specific  error  on a particular
data element may help identify a definition or  procedure that  is
ambiguous and needing revision.  Consistency checks between
similar items or  the same data on two separate files can be
performed.  If there are significant discrepancies then  a  full
audit, with  the  appropriate  analysis  and correction cycles  would
be mandated.  These  steps need to be formalized  into policies and
procedures that must be included in  any revised set of Agency
ADP/IS Policy and Procedures Manual.


                           111-13

-------
 (3)  Policy and Procedures

     The basic policies and procedures with regard  to data
management documents in the current ADP Manual indicate  that the
basis for control and an effective program are present, but are
not currently implemented.   There  are  many factors  contributing
to their not being implemented.  These include:

     Decentralized management of data and systems

     Little high level mangement awareness of  the need  for  data
     management

     Limited resources in MIDSD

     We believe that the organization structure and  tools
recommended will provide the dynamic program  structure'and
effective tools that will make the policies  and procedures work.
What is needed  is a thorough review of the ADP manual to  evaluate
the appropriateness of the policies and procedures in the context
of the other recommendations.  Where changes are necessary they
must be made.   This can either be done by delegating this
responsibility to the group establishing the  new documentation
standards or by establishing a separate  task  force.  The latter
approach would probably be preferred because  of the  combined
workload.  Finally, top  level  management commitment  to  the
enforcement of these new policies and procedures must be obtained.
The vehicle for this approval is the ADP Oversight  Committee.

 (4)  Conclusion

     The key to the successful implementation  of this Data
Management and Standardization Program  is the  combined  support
of top level management and the programmatic  areas.  This is
normally a difficult request,but we believe that it  is  possible
because of the potential benefits to both.  These potential
benefits include:

.    Facilitates sharing and re-utilization of data  to  ease  the
     effects of potential resource reductions  in some programs,
     and to reduce the  reporting burden on the public and private
     sectors

     Provides a focal point for locating answers to Congressional,
     OMB, GAO,  and FOI  inquiries

     Provides Standards that Regions and States will be  required
     to follow as programs are delegated

     Eases potential conversion efforts  for the 1980 ADPE
     Procurement
                           111-14

-------
»
          Provides an implementation program for the IRLG Common Codes
          Project.

     The next chapter presents a plan for implementing the recommended
program and provides cost estimates  for the  implementation.
 t
                                111-15

-------
t
t
 I
Implementation Plan

-------
*
                               IV.  IMPLEMENTATION PLAN
S

-------
t
                               IV. IMPLEMENTATION PLAN
               Implementation of  the proposed EPA Data  Management  and
          Standardization Program will  require  a nine  step  process which is
          graphically depicted in Exhibit IV-1.  The first phase of  this exhibit,
          Data Management and Standardization Program Plan, is shaded to indicate
          that it has been completed with the presentation of this  report.  Thus
          far the requirement for an EPA-wide Data Management and Standardization
          Program has been established;  and program components have  been defined.
          This information will serve as a starting point for  the next phase of
          implementation, establishment of a data management task  force.

               There are  two primary facets  of  the  implementation  plan:

                    the implementation  plan  itself  which  includes  activities,
                    schedules, and control points;  and

                    supporting costs.

               These two  factors  are further discussed  in the sections below.
f
          1.
     IMPLEMENTATION PLAN
     To further present an expanded overview of the activities which
are supportive to program implementation, this section has been
organized into the following areas:

     .     Recommended Approach to Program Implementation

          Implementation Activties

          Discussion of Milestones and Control Points.

These aspects of the implementation plan are further discussed below.

     (1)  Recommended Approach to Program Implementation

          Arthur Young & Company recommends a phased approach using
     phased program implementations of data management and
     standardization program concepts.  Use  of phased program
     implementations will:

          Spread the resource requirements for implementation over a
          longer period of time, thereby balancing the use of personnel
          and funds
                                          IV-1

-------
S
t
             E
             D
S

^

Q

•Q


O 0)

   O
O ** a a
                              w 2 ff •
                              <•« 3 a
                              s ~ 4 S
                              Q. IM Stk
                                                     I
                                                     a

                                                     ^
        ill
I*  HI

11=111
5< 3 S i S
u a 01 £ -iu

-------
t
f
.     Permit appropriate lead time for  interfacing programmatic
     areas to prepare for program implementation

     Enable the proposed ADP Oversight Committee and  supporting
     Data Administrator and staff to concentrate on the more
     detailed aspects of interfacing specific  programmatic  areas
     as program participants.

     The activities which must be undertaken  to establish  the
organization structure necessary to control phased program
implementation and required implementation steps are  further
discussed in the section below.

(2)  Implementation Activities

     Implementation activities include those phases specified in
Exhibit IV-1,  these are the activities that follow the--prog ram
plan to effect a full program implementation.   The objectives and
activities of the phases are discussed below:

(a)  Establish Data Management Task Force

     The first step EPA must take to develop  a Data Management
and Standardization Program is to establish a Data Management
Task Force.  The  functions  of  this  task force  will include:

.     Identification and nomination of members for the ADP
     Oversight Committee and supporting oversight committees
     throughout EPA

     Prepare draft procedures for day-to-day  program  operations

     Recommend the levels of administrative authority and
     responsibility for day-to-day program operations and  the
     method for delegation throughout the organization.

     The Data Management Task Force will provide the  basis for
formation of the required hierarchy of oversight committees. This
group will also prepare the preliminary program procedures for
ADP Oversight Committee evaluation.  A final  report should be
produced to document all task force activities,.

(b)  Establish Organization Structure to Support the  Program

     Based on task force recommendations the  ADP Oversight
Committee and programmatic level oversight committees will be
formed.  The defined role of the ADP oversight committee should
include:

     Establishment of data management program policies and
     procedures
                                          IV-2

-------
S
I
     Specification and control of the use of data management
     tools
     Review and activate standards for:
          System documentation
          Data element definitions
          Data element coding schemes
     Development, review and revision of long-range plans for
     data management and standardization program application
     Coordination of the data management activities of
     programmatic level oversight committees.
     The defined role of the programmatic level oversight
committees should include:
     Establishment of more detailed data management policy  and
     procedures for their area which are consistent with the  EFA-
     wide policies and procedures.
     Review and activate standards for:
     -    System Documentation
     -    Data Element Definitions
          Data Element Coding Schemes
     Develop,  review, and  revise  programmatic level Long-Range
     Plan for Data Management
     Prioritize a yearly plan for the development of new
     information systems and enhancement, of current systems
     Review and endorse the information systems annual  budget
     requests.
Both levels of the oversight committees will be required to
monitor the adherence to their decisions through appropriate
support staff.
     The first task of these committees will be to agree on their
working and meeting procedures  and to identify  the authority and
responsibility which will be delegated to committee support
staffs. The ADP Oversight Committee must also evaluate  the skill
levels for the Data Administrator and support  staff positions
and initiate activities to obtain the required personnel
resources.  Draft procedures for  day-to-day program
                           IV-3

-------
S
administration must also be reviewed, modified, and finally
adopted.

     After the operational aspects of the ADP Oversight Committee
and programmatic oversight committees are. determined, a  plan  and
schedule for phased program implementations must be developed
and approved.  A list of priorities to be considered  during phased
program implementations must also be developed  and approved.
Programmatic level oversight committees can contribute  valuable
support for this effort.

     At the conclusion of this  effort the  EPA-wide  ADP Oversight
Committee,  and  supporting Data  Administrator and staff should be
organized and capable of functioning.  Programmatic level
oversight committees should be identified and committee
recommendations and operations  formally documented for activation
prior to initiation of pilot program implementations in that area.
All related components of the designated organization structures
should be formally documented.

(c)  Set Policies and Procedures

     In this phase; the ADP  Oversight Committee will formalize
the data management and standardization policies and procedures
which will govern program operations.  These policies and
procedures will be passed to the programmatic level oversight
committees to serve as a basis for planning pilot program
implementations, actual implementation,  and  ongoing program
operations. The  ADP Data Management  and Standardization Program
Policies and Procedural Manual  being prepared  by MIDSD should be
finalized.

(d)  PhasedApproachOsingPilotImplementations

     Based on the program implementation schedule  approved in  .
the previous phase by the ADP Oversight Committee, implementation
of the data management program can be initiated.  The programmatic
level oversight committee, responsible for the new program
participant area, should first be activated.  The role of  required
committee support staff should be defined and staff composition
determined.  This role will  be  dependent on the implementation
priorities established by the  oversight committee  and the  level
of authority and responsibility delegated to the oversight
committee by the ADP oversight committee.  If  necessary,
individuals will have to  be  transferred, hired  and/or trained to
meet the.requirements. Once the staff is assembled work on
installation of the tools for data management and standardization,
and appropriate user training  can begin.  The  plans for pilot
program implementation should  be formally documented for review
by the ADP oversight committee and responsible  programmatic
oversight committee.  This will  be highly dependent on the current
pilot projects in development.   The phased approach comes into


                           IV-4

-------
S
I
play by adding other programmatic areas to the particular  tool
in a sequential rather than a one-shot implementation effort. -

(e)   Implement Use of Tools

     The data management tools to be installed during pilot
program implementations include:

     Application of the standards established by the ADP
     Oversight Committee for system development and
     documentation, data  elements, and data acquisition
     techniques.

     Development of an Agency-wide data directory and individual
     program DED/D.

     Coordination procedures for feasibility studies for system
     design, system changes, and  data standardization efforts

     Initiation of quality assurance* audits which will  identify
     the completeness and accuracy of system data, and  analyze
     procedures for data capture and processing to isolate the'
     causes of the errors.

     Details of the composition of the specific tools are
presented in Chapter III.

(f)   User Training

     An essential activity during phase of program implementation
is the training of data management program participants.  EPA
personnel must be trained to accept program operations in a manner
that will encourage active participation  and promote the
maintenance of a highly reliable program.  Appropriate  training
materials related to EPA-wide data management and standardization
concepts should be prepared by the Data Administrator support
staff and approved by the ADP oversight committee. These  training
guides can then be distributed through programmatic level
oversight committees to the committee support staff responsible
for user training.  This  support staff  should also prepare
additional training materials related to  the more detailed data
management activities of  their programmatic area. After  approval
by the responsible oversight committee, a combined training
program can be initiated.   User  training manuals should also be
produced for use as training guidelines and to document  the
training methodology employed.

(g)   Full Program Implementation

     At the conclusion of the final phased program..implementation,
all SPA organizational components should  be participating  in an
active, organization-wide  Data  Management and Standardization


                           IV-5

-------
 S
I
     Program.   Due  to the large scale  nature of  this activity  the ADP
     Oversight Committee may be.required to reassess and revise
     original working and meeting  procedures, and delegated authority
     and responsibility.  In addition,  this committee should begin  to
     plan additional information management activities over a
     succeeding five year time period.

     (h)  Post-Implementation Evaluation

          Post-implementation evaluation involves the determination
     of whether the fully implemented Data Management and
     Standardization Program meets EPA's needs  and performs
     efficiently.   This  activity should be  performed several months
     after the final pilot program implementation.  This time lag is
     required so that EPA can form an opinion as to the program's
     effectiveness and efficiency.

          A Post-Implementation Evaluation report should include an
     evaluation of program performance/ operational costs,  areas  for
     improvements,  and a determination of further  enhancement areas.

     (3)  Milestones and Control Points

          Each phase of the program implementation results in the
     production of a document which relates the activities performed
     during that phase.   This document.should  be submitted  to EPA
     management to facilitate review of activities to date, and
     adherence to schedules so that approval can be obtained  for
     progression to the  next phase of the implementation.  Exhibit IV-
     2,  on the following page, presents  an approximate  schedule  for
     the implementation plan described  in this chapter.

          Throughout the program implementation considerations of
     costs, and availability of personnel resources may impact the
     implementation schedule presented in Exhibit IV-2. The evaluation
     of the entire implementation  task at the completion of the
     activities of each  phase provides milestones which will  assist
     EPA management in their evaluation of potential impact on the
     program implementation effort and time schedule as a whole.

2.    PROGHAM COST ESTIMATES
               This section describes the methodology utilized by Arthur Young
          & Company to develop the cost estimates for the development and annual
          operations of the proposed EPA Data Management and Standardization
          Program.  Program costs  are discussed  in terms of specific assumptions
          which provided the basis for cost algorithms; and a cost estimate
          matrix which  summarizes the  results of detailed cost computations and
          permits analysis from  varying  perspectives.  The specific assumptions
          and cost estimates for the recommended program are further discussed
          below.
                                          IV-6

-------
                                                                     EXHIBIT IV-2
t
                 2
                 o
                 o
                 oT

                 i
                 ±i £
                    Q
                    eo

                   "S.
                    £
                 §
                            s
SH
                                       1
                                      •*



                                                           =3   5

-------
i
 (1)  Specific Assumptions

     The rationale for  the inclusion of specific cost assumptions
are presented below:

     SPA Clerical Personnel - EPA clerical personnel will
     primarily be required to provide support  services  for  the
     organizational component of the program structure.  (ADP
     oversight committee, programmatic  oversight committees, and
     data administrator and staff).  This category does not
     include the costs for the preparation of documentation
     required to document each implementation phase. EPA  clerical
     rates were costed at an average GS-7 Step  5 annual salary
     of $13,980.  Hourly  rates were calculated  utilizing a  260
     day work year comprised of 2080 hours.

     E_PA Professional Personnel - EPA professional personnel will
     be required to actively participate in all phases  of the
     program life cycle and annual operations.   This  category
     includes the EPA management  personnel  who  will serve on the
     ADP oversight committee, programmatic  oversight committees,
     or act as Data Administrator.   Professional personnel will
     also attend user training sessions. Throughout  annual
     program operations,  professional personnel  will be  required
     for program administration and enforcement purposes.   EPA
     professional personnel rates were costed at an average  of
     GS-14 step 5 annual salary of $34,850.  Hourly rates were
     calculated utilizing a 260 day work year comprised of  2080
     hours.

.-    EPA Systems Personnel - EPA systems personnel will support
     all phases of the  program life cycle and annual operations.
     Primarily this category consists of programmatic level
     committee members and support staff, Data  Administrator
     support staff,  and Data Coordinators.

     On an ongoing operational basis, EPA systems personnel
     resources will support program operations related  to the
     operational aspects of data  management tool-s, participation
     in programmatic level oversight committees, and  aspects of
     the Data Coordinator role which pertain to program
     activities or EPA passive data dictionary maintenance  and
     use.

     The'program-related costs for  this  type of personnel do not
     include the salaries for  all systems personnel  in EPA.   The
     intent is to show only those costs which relate to
     utilization of systems personnel resources for data
     management and standardization program related activities.
     EPA systems personnel rates  were costed at an average GS-
     12 Step 5 annual salary of $24,799.  Hourly rates were
     calculated utilizing a 260 day work year  comprised of  2080
     hours.

                           IV-7

-------
     Documentation - At the conclusion  of each  phase  of the
     program life cycle, various forms of documentation will be
     •produced.   This  cost  category includes  expenses which would
     be incurred for the  typing, graphic arts, and reproduction
     aspects of program documentation.  Typing costs were  based
     on 4 pages per hour at.the EPA clerical rate of $7 per hour.
     Graphic arts preparation was based  on 45 minutes per graphic
     at the EPA clerical rate  of $7 per hour.  Reproduction costs
     were based on $.05 per page, per  copy produced.

     Other Assumptions^ - For the purpose of costing pilot program
     implementation ^.nd the resulting increases  in annual
     operations costs,  the following assumptions were made:

     -    Approximately 15 pilot program implementations  are
          required.  Currently, there  are approximately 22 EPA
          Deputy Assistant Administrators (DAA's) who could
          install programmatic  level oversight committees to
          control data  management activities in their areas.  Each
          of these DAA's may not, however, have sufficient systems
          activity to warrant program implementation.  In
          addition, the systems activity for certain areas may .
          fall within the scope  of other programmatic .level
          oversight committees.  Consideration should also be
          given to the fact that some DAA's may wish to jointly
          participate in the program through one common oversight
          committee due to responsibilities  for  highly  related
          systems.  It  was therefore determined  that at  least  11
          DAA's would implement their  own program, while the
          remaining 11 would jointly share  in 4  other oversight
          committees.

          Pilot program implementation would be  undertaken based
          on the following schedule:  0 in  fiscal  year  78-79;
          three in fiscal year 79-80;  and four each in  fiscal
          years 80-81,  81-82,  and 82-83.

(2)  Cost Matrix

     To present the cost estimates for  the  implementation and
annual operations of an EPA Data Management and Standardization
Program, cost estimate  matrices were utilized.  Since  the
recommended implementation plan  for this program  includes a
phased approach consisting of pilot program  implementations over
a five-year period, cost estimate matrices have been prepared for
each year of the five-year implementation plan.  These annual
cost estimate matrices are shown as Exhibits.IV-3  through IV-7.
In addition, total  costs for  the five-year  implementation plan
are shown on Exhibit IV-8.  Exhibit IV-9 shows the total annual
operations costs for the program after  full  implementation, for
an additional five-year period.  A  7% cost escalation  factor was
added to all cost estimates for each successive year beyond fiscal


                           IV-8

-------
     Documentation - At the conclusion of  each  phase  of the
     program life cycle, various forms of documentation  will be
     produced.  This  cost  category includes expenses which would
     be incurred for the  typing, graphic arts, and reproduction
     aspects of program documentation.  Typing costs were  based
     on 4 pages per hour at the EPA clerical rate  of $7 per hour.
     Graphic arts preparation was based on 45 minutes per graphic
     at the EPA clerical rate  of $7  per hour.  Reproduction costs
     were based on $.05 per page, per copy  produced.

     Other Assumptions - For  the purpose of costing pilot program
     implementation and the resulting increases  in  annual
     operations costs, the following assumptions were made:

          Approximately 15 pilot program implementations are
          required.  Currently, there are approximately 22 EPA
          Deputy Assistant Administrators  (DAA's) who could
          install programmatic  level oversight committees to
          control data management activities in their areas.  Each
          of these DAA's may not, however, have sufficient systems
          activity to warrant  program implementation.  In
          addition, the systems activity for certain areas may
          fall within the scope of  other programmatic .level
          oversight committees.  Consideration should also be
          given to the fact that some DAA's may wish to  jointly
          participate in the program through one common oversight
          committee due to responsibilities for  highly  related
          systems.  It was therefore determined  that at  least 11
          DAA's would implement their own program, while the
          remaining 11 would jointly share  in 4  other oversight
          committees.

     -    Pilot program implementation would be  undertaken based
          on the following schedule:  0 in  fiscal year  78-79;
          three in fiscal year  79-80; and  four each in  fiscal
          years 80-81,  81-82,  and  82-83.

(2)  Cost Matrix

     To present the cost estimates  for the  implementation and
annual operations of an EPA Data Management and Standardization
Program,  cost estimate matrices were utilized.  Since  the
recommended implementation plan  for  this program  includes a
phased approach consisting of pilot program implementations over
a five-year period, cost estimate matrices have been prepared for
each year of the five-year implementation plan.  These annual
cost estimate matrices are shown as Exhibits. IV-3 through IV-7.
In addition, total  costs  for  the five-year   implementation plan
are shown on Exhibit IV-8.  Exhibit IV-9 shows the total annual
operations costs for the program after full  implementation, for
an additional five-year period.  A  7% cost  escalation  factor was
added to all cost estimates for each  successive year beyond fiscal


                           IV-8

-------
                                                         EXHIBIT  IV-3
a
a
i

"S
o


§  2 s
eo  ^


1  *
D  te



a — *~
O.ZO
0.3U
01—
co
Ul
tt> IPH 4*J
QUIZ
1 ^£ Ul
H»«J a.
30X
O 0-U1
I
5
^
2
vu
2
21
u
i






IU
u
et
i
IU
Q£

!
Ul
0.




' X»
<4Q Ul
a»^-
U4CO
^H
W

|
O
<4S Cl9
O.U1
Ulttl
a

5
a.te
tuui



(19
H.S
cnui
ox
-UJ
U Ul
U.£



9
a
»
^^
^

«.

a
a
Ul

«.

^
9
c*

9
9
• a

ft


(*
o
9
d
pj

Ok

9
a
;
»

9.'

**
,_

ESTABLISH
ATA MANAGEME
TASK FORCE
a

a
9
^»
^
VS

v>

g
a
PI
^
«.

_
9
p-t

9
9

• «
r-t


V>
9
9
9

C4
S

9
9
(H
n
«>

a
9
w>


ESTABLISH
ORGANIZATION
STRUCTURE

1
9
3
39
a
9
^
«. j

9
s
9
*^
<»

9
a
*4

o
o
^
*
<••*


<*
9
9 •
cn

t
—i cn
ET AND PROMUI
ATE POLICIES
NO PROCEDURE:
« a<

^-
t- z
Z u
ui ^2 o
^c ^c o ^^ ^*
< < KLUQ
O S &CU


a
a :
% i
Ul

rt I
1

a
9
*
a
~

«>




1
1
1




1
1
1




•
i
'

9
9
a
•*
^


u>
o
Q
^4
CN
M
£

o
rt
9
«.

9
5
°"


§
^
UJ
>
LU
I






t
1
1





1
I
1



t
t



1
*"


MPLEHENT USE
OF TOOLS
^


'
1
1
t ;
1




1
t

i


I
1




I
t
i





1
i




1



t
I



USER
TRAINING

S
^
! 5
• ISs
O Q- to
csso
3, — O
1




i
i
i

I


i
i
i

t



i




i
i
i





i
i
i



i
i



i



1
S S

o >— •
c~ S
11



a
a
O

lift
«P)
rt
«>

a
9
9
r>

Vf

9
9
a
I

0
a
C9
•
^


**
9
9
^^
M
ft
ft
o>

9
9
n
9
<»

9
9
VO
**


IIEVELOPHEIIT
ill Oil COSTS
« ^
s'S

a
a
jgj

r»
o
«
0

a
a
a

v>

g
(N
«*

9
a
«M
•
C*4


<*
^ O
o
^
^r
(^
«>

O
"i*
£
«>

• a
a

-------
i
                               IV.   IMPLEMENTATION PLAN
f

-------
t
                          implementation Plan


-------
                                                                        EXHIBIT  IV-4
t
                 2
                 o
                1
                 o
                •a
                CO
                       «r»

                 0
                 O
                 a
                 a
                 a

                 I








CA
_l
^
3







^ ^J
si—
Ul O
Q




1 >
O —ol
Q.ZO
o»-u
Ul
OUIZ
H-Sal
OO.U1
i
^
1^
^c
1—
i
u
§




01
Ul
u
OS
Ul
rt*

J
s
Q
u*.
£**>
ut