,
          Environmental Protection Agency  $ «» \
             Office of Pesticide Programs
                                     °

    Detailed Design for an Information

  Processing System for the Pesticides

                  Monitoring Program
                         VL Operations Manual
                            December, 1978
                       Contract No. 68-01-3833
ARTHUR YOUNG & COMPANY

-------
                    OPP MONITORING SYSTEM

               DETAILED DESIGN DOCUMENTATION

     The Detailed Design Documentation for the EPA/OPP pesticides
Monitoring System consists of 7 Chapters vrhich are bound in 6
volumes.  These chapters are:

            Chapter                      Title

                I                  System Overview   Bound Together
               II                  Implementation Plan

              III                  Data Base Design

               IV                  Input/Output Specifications

                V                  Functional Module Specifications

               VI                  Operations Manual

              VII                  User's Manual

Each of these chapters is largely self-contained but may refer to
information contained in other chapters as necessary.

-------
                        TABLE OF CONTENTS






           TOPIC                                         PAGE






1.  DATA BASE GENERATION                                 VI-1




2.  APPLICATION PROGRAM SUPPORT                          VI-16




3.  DATA BASE LOADING                                    VI-23




4.  DATA BASE UPDATING                                   VI-26




5.  BACK-UP AND RECOVERY                                 VI-27




6.  DATA BASE MAINTENANCE                                VI-41




7.  SECURITY PROCEDURES                                  VI-44




8.  ARCHIVAL DATA                                        VI-46

-------
VI.   OPERATIONS MANUAL

-------
                     VI.   OPERATIONS MANUAL
INTRODUCTION

     The OPP Pesticides Monitoring System Operations Manual integrates
PMS specific topics such as database  update  and  archival  procedures
with standard IDMS database system procedures.  The information
pertaining directly to IDMS routines,  syntax and  Job Control Language
has been abstracted from Cullinane Corporation IDMS manuals and
tailored to OPP's specific  needs.  The manuals referenced are Database
Design and Definition  and  Installation  and Operations Guide.

     In order to tailor the JCL to OPP's operating environment, the
following names have been  used throughout the manual.

     SYS2.IDMS.OPPMON.DIRECTRY - the DSNAME  for the  IDMS data directory.

     SYS2.IDMS.LOAD - the DSNAME for the  library containing IDMS system
                     modules  as delivered  by Cullinane;  resides on
                     APPL01.

     SYS2.IDMS.OPPMON.SUBSDMCL - the DSNAME for the library containing
                                Subschema and DMCL load modules which
                                belong  to the OPP monitoring data
                                base; resides on  APPL02.

     SYS2.IDMS.OPPMON.DBPROC - the  DSNAME for the  library  containing
                              Data Base Procedure load modules which
                              are used  by the OPP monitoring datbase;
                              resides on APPL02.

     SYS2.IDMS.OPPMON.PROG  - the DSNAME  for  the library containing
                            application programs  which use the OPP
                            monitoring  data  base;  resides on APPL02.
                                VI-1

-------
1.    DATA BASE GENERATION

     (1)   Data Directory Generation

          Functional Description

          The IDMS Data Directory System is a communications tool
          designed to provide database control and documentation. All
          of the IDMS compiler and processor system components operate
          in conjunction with the IDMS Data Directory file  (Figure
          VI-1).   This  file  (which itself  is an  IDMS  database)
          represents a definitional resource in  which IDMS stores all
          information concerning the user's database  environment.
          Since all major system components, including the Schema,
          Subschema,  and DMCL compilers, the  Data Directory  Utility
          Program  (IDMSCLUC), and the  PL/I,  and  DML-COBOL  processors,
          post information into the Directory, it becomes  a centralized
          information resource  for database administration.

          For this purpose,  the  Data Directory  Reports Program
          (IDMSRPTS)  produces reports  which are defined in detail in
          2.  Application Program Support.

          Allocation

          The Data Directory is allocated as an OS file prior to the
          initialization step.   The allocation  should be done in a
          separate step in  order to avoid changing the JCL for  the
          initialization step prior to re-compiling  the schema.   The
          Data Directory is  allocated  using the following JCL.

          //   EXEC PGM=IEFBR14

          //   DD     DSNOPPMON, UNIT=3330-1,VOL=SER=USRVOL,

          //   SPACE = (960,1000),DISP=(NEW,CATLG)

          OPPMON = Data set  name of Data Directory

          USRVOL = serial number of disk containing  Data Directory

          Initialization -  IDMSINIT

          Since the Data Directory is itself an IDMS database,IDMSINIT
          must be  the first  program run prior to processing.  This
          step is  mandatory,  since each  page of  the  data directory
          must be  formatted  prior to storing records.  Consequently,
          IDMSINIT performs  the following initializing actions:
                                VI-2

-------
    IOMS OOL
   COMPILERS
    1DMS OML
   PROCESSORS
                                 IOMS
                                 DATA
                               DIRECTORY
                                             IDMS
                                           DIRECTORY
                                            REPORTS
                                          I QMS/CULPRIT
FIGURE vi-i
-,= IOMS  DATA  OISECTCRY  FILE

-------
          Clears and formats direct access blocks (one per  page)
          for the Data Directory

          Produces a parameter listing

     General information on the IDMSINIT  function can  be found
     in the IDMS Database Design and Definition manual, page  11-
     4.

     The following card formats are specific to  the
     Initialization of the Data Directory:

          PROCESS = TOTAL,  DMCL = IDMSBASE

     The DMCL name that provides access to the entire  database
     directory network is always IDMSBASE.  This formatting
     operation must be performed prior to the execution of the
     IDMS Schema Compiler.   Figure VI-2 illustrates  the
     initialization of the IDMS Data Directory.

     The JCL for performing the operation is as  follows:

     //          EXEC    PGM=IDMSINIT

     //STEPLIB   DD      DSN=SYS2.IDMS.LOAD,DISP=SHR

     //SYSLST    DD      SYSOUT=A

     //SYSDDL    DD      DSN=SYS 2.IDMS.OPPMON.DIRECTRY,DISP=OLD

     //SYSIPT    DD      *

     PROCESS =TOTAL,DMCL=IDMS BASE

     /*

     SYS2.IDMS.LOAD = data  set  name  of  IDMS  load  library
     SYSOUT=A = printed output file
     SYSDDL = DD name of Data Directory
     SYS2.IDMS.OPPMON.DIRECTRY = data set name of Data  Directory

(2)   Schema Compilation

     Functional Description

     A schema is a  'global'  definition of the database,  including
     descriptions of  all files, areas,  records, and sets in the
     database environment.  Defined by the DBA staff, the purpose
     of  the schema  is two-fold:

          To provide  commonality of data descriptions and data
          relationships among all user programs accessing the
          database

                           VI-3

-------
                                                  pagers/
      •OOLDML*AflEA
                             LOGICAL PAGE
                             NUMBERS (LPN'S)
                                        *OOLDM(. IS THE NAME OP Trig
                                         IQMS DATA OlflECTOaY AflEA
                       RELATIVE BUOCX
                       NUMBEHS (RBN'S)
FIGURE  vi-2    INITIALIZING  THE  DATA  DIRECTORY

-------
          To maintain data-program  independence

     When the user has coded all necessary Schema DDL statements
     for his database description, he is ready for actual
     execution of the IDMS Schema Compiler.  Figure  VI-3 shows
     the components of the schema phase of  IDMS  data description.

     Syntax and Semantics

     The following paragraphs describe  the  general  function,
     format/order, and rules governing SCHEMA DESCRIPTION
     statements.  Specific information for  each  statement  can  be
     found in the IDMS Database Design  and  Definition manual, 6.3.

          General Function

          SCHEMA DESCRIPTION statements identify the schema
          according to:

               Schema Name, Author,  Date, Installation, and
               Remarks

               File Description

               Area Description

               Record Description

               Set Description.

The statement groups must appear  in  the sequence shown above,
each describing one aspect of the whole schema,  and  each relying
on the content of O-he description statements from the prior
group(s).

Each statement group begins with a descriptive paragraph  name.
Processing of statements for a specific group begins with  the
associated paragraph name and ends with the recognition of the
paragraph name of the next group or  the end of the  statement
input stream.

At least one entry in  each statement group,  except the set  group,
is required.

     JCL

                         IDMSCHEM

     //          EXEC    PGM=IDMSCHEM,REGION=180K
     //STEPLIB   DD      DSN=SYS2.IDMS.LOAD,DISP=SHR
     //SYSDDL    DD      DSN=OPPMON,DISP= (OLD,PASS)
                           VI-4

-------
SCHEMA
SOURCE
1DMSCHEM
                                   DATA
                                 DIRECTORY
                               ERRORS  &
                               WARNINGS
                               INPUT
                             STATEMENT
                               UST1NG
                     FIGURE vi-3 DEFINING SCHEMA

-------
     //SYSJRNL   DD      UNIT=TAPE62,VOL=OPPDIR,DSN=OPPJRNL,DISP=MOD
     //SYSOUT    DD      SYSOUT=A
     //SYSLST   DD       SYSOUT=A,DCB=BLKSIZE=133
     //SYSIPT    DD      *

     Schema DDL source statements (omit if SYSIPT is other than
     *)

     SYS2.  IDMS. LOAD = data set name of IDMS load library
     SYSDDL = DD name of  Data  Directory
     OPPMON = data set name of Data  Directory
     SYSJRNL = DD name of Journal file
     SYSOUT=A = printed output files
     *  = input unit

(3)   DMCL  Compilation

     Functional Description

     The Device-Media Control  Language  (DMCL) is used to:

          Control those data sets that  are on-line

          Regulate user page (block)  size  for the database

          Specify the buffer pool space used by IDMS at execution
          time.

     The DMCL Compiler cannot  be executed  until  an error-free
     schema compilation has been achieved and the associated Data
     Directory File (as built  by the Schema Compiler)  is
     available.  An object DMCL module encompasses all or selected
     areas of the user's  database  and provides:

          Necessary file  control block information (DTF's, DCB's)
          to permit IDMS  run-time access to the  user database
          files on disk and to write the system  journal (log)
          file

          Mapping tables  to allow  IDMS to translate logical page
          addresses to proper  relative  block assignments within
          the secondary storage file(s)

          Information for establishment of system buffer pools
          (in OS,  main  storage  is  dynamically acquired  by  IDMS
          at execution time)

     Figure VI-4 traces the significant steps in executing the
     DMCL  phase of data definition.

     Syntax and Semantics
                           VI-5

-------
The DMCL consists of the following  three  statement  groups:

     Device-Media Description statements

     Buffer Section statements

     Area Section statements

     Journal Section statements

The statement groups must be coded  in the order shown:

DEVICE-MEDIA DESCRIPTION statements identify the DMCL Name,
Author, Date, Installation, and Remarks

The BUFFER SECTION statements are used to assign a  name  to
each buffer pool, declare the  page size (block size)  of pages
in the pool, and declare the number of pages  (blocks)  in the
pool.

The AREA SECTION statements specify the logical areas
defined in the Schema DDL that are  to be available  to the
DBMS through this DMCL module.

The JOURNAL section provides information about the  journal
file.   It  is mandatory for disk journaling, and optional for
tape journaling.   If the JOURNAL section is  present  in the
DMCL,  the  journal file  need not  have been described  in the
schema.  If it  is missing from  the DMCL, tape journaling  as
described in the schema will be used.  If journal-file
specifications are included both in  the DMCL and the schema,
those  in the DMCL override the schema description.
                                        &•
Specific rules concerning each section statement can be
found  in the IDMS Database Design & Definition manual.

Execution Instructions;  To define  a DMCL module,  execute
the program IDMSDMCL followed by the assembler and  linkage
editor. Input to IDMSDMCL consists of a  source deck written
in DMCL DDL.   Output is  as  follows:

     A description of the DMCL module stored in the  Data
     Directory

     An assembler source program and device-media input
     listing

Input  to the assembler  consists of  the source program
produced by IDMSDMCL.   Output  is a  relocatable  program and
an assembler listing.
                      VI-6

-------
     Input to the linkage editor consists  of  the relocatable
     program  produced by the assembler.   Output  is  an  object
     program and a linkedit map.

     The  following JCL is reflected  in  Figure VI-4:

                         IDMSDMCL
     //STEP1
     //STEPLIB
     //SYSDDL
     //SYSJRNL
     //SYSOUT
     //SYSPCH

     //SYSIPT
     DMCL SOURCE
     //STEP2
     //SYS PRINT
     //SYSLIB
     //SYSUT1
     //SYSUT2
     //SYSUT3
     //SYS PUNCH
     //SYSIN
     //STEPS
 EXECPGM=IDMSDMCL,REGION=160K
 DD   DSN=SYS2.IDMS.LOAD,  DISP=SHR
 DD   DSN=SYS2.IDMS.OPPMON.DIRECTRY,DISP= (OLD,PASS)
 DD   UNIT=TAPE62,VOL=OPPDIR,DSN=OPPJRNL,DISP=MOD
 DD   SYSOUT=A
 DD   DSN=&&DMCL,UNIT=SYSDA,DISP= (NEW, PASS),
      SPACE = (80,(400,40)),DCB=BLKSIZE=80
 DD   *
STATEMENTS OMIT  IF SYSIPT is other than*)
 EXEC PGM =IEUASM,PARM='NOLOAD,DECK',REGION=80K
 DD   SYSOUT=A
 DD   DSN=SYS1.MACLIB,DISP=SHR
 DD   UNIT=SYSDA,SPACE=(CYL,(2,2))
 DD   UNIT=SYSDA,SPACE=(CYL,(2,2))
 DD   UNIT=SYSDA,SPACE=(CYL,(2,2))
 DD   DSN=&&ASMBL,UNIT=SYSDA,DISP= (NEW,PASS)
      SPACE=(80,(400,40))

 DD   DSN=SSDMCL,DISP= (OLD,DELETE)
 EXEC PGM=IEWL,PARM='SREF,LIST,LET,SIZE=(96K,12K)1,
      REGION=96K
     //SYS PRINT   DD
     //SYSLIN      DD
     //SYSUT1      DD
     //SYSLMOD    DD
      SYSOUT=A
      DSN=&&ASMBL,DISP= (OLD/DELETE)
      UNIT=SYSDA,SPACE=(TRK,(20,5) )
      DSN=SYS2.IDMS.OPPMON.SUBSDMCL(DMCL),DISP=OLD
     SYS2.IDMS.LOAD  =  data  set  name of IDMS load library
     SYSDDL = DD name of Data  Directory
     SYSJRNL = DD name of Journal  File
     SYSOUT=A = printed output  files
     &&DMCL = data set name of  temporary data set for DMCL output
     *  = input unit
     &&ASMBL = data set name of  temporary  data  set for assembler
     output
     DMCL = name for DMCL module
(4)   Subschema Compilation

     Functional Description
     The subschema,
     definition,  is
   a logical subset of the schema  database
   designed primarily to maintain  program-data
                           VI-7

-------
DMCJ. SOURCS
         ASSEMBLE
           UST1NG
           UNKEDIT
             MAP
                                       STEP 1
IOMSOMCL
                                     T STEP 2
ASSEMBLER
                                       STEP 3
 LINKAGE
  EDITOR
DEVICE-MEDIA
   INPUT
   USTING
              OBJECT
               DMCL
              MODULE
                     FIGURE vi-4 DEFINING DiMCL MODULE

-------
independence.  Containing only those data elements to be
referenced by the application program, the subschema
definition remains largely  independent of the  schema  or
"global" definition.   Consequently,  changes  may be made  to
a specific subschema without necessitating  schema
recompilation or extensive  database  reprogramming.

The following points are  important  to an  understanding of
the relationship between  the schema  and subschema:

     The schema is established independently of  any user
     program or any subschema.

     A user program invokes a subschema, i.e., sees the
     database in terms of a  particular subschema's view.  No
     user-written program can have  access to an  IDMS
     database without use of one  (and only one)  subschema
     view.

     Only the areas,  records,  data-items,  and sets included
     in the subschema  invoked  by a program may be  referenced
     by that program.

     Only the database management functions permitted within
     its subschema view are executable by the  application
     program in the run-time environment.  (For example, in
     terms of one particular subschema view, customer
     records may be OBTAINED but not ERASED.)

     The declaration of a subschema  has no effect on  the
     declaration of any other subschema,  and  subschemas may
     overlap one another  in their 'views'  of  the  database.

     The same subschema may be invoked by any  number  of
     programs.

     Any number of subschemas may be declared  on  the  basis
     of a given schema

     Object versions of subschemas  are compiled
     independently of  any user program and stored in a 'load1
     or 'core-image1 library.

At least one user-established DMCL module must have been
compiled into the IDMS Data Directory before the  Subschema
Compiler can be executed. The subschema compile  phase will
generate system control tables which describe  a  specific
program view containing particular

     Areas
                      VI-8

-------
     Records (all or specified portions)

     Sets

In addition,  the  subschema  tables  also  reflect privacy  lock
restrictions limiting the programs using this view of  the
database to designated subsets of DML commands.

No user program may request the services of IDMS at  run-
time without reference to a subschema.  In such  an execution-
time environment  no  run-unit can successfully request  access
to a portion of the database not  included in its  subschema.
It also cannot request a function for which it is not
authorized according to that subschema's privacy locks.

Figure VI-5 outlines the important steps comprising  the
subschema phase of IDMS data definition.  These  are described
below:

     The Subschema Compiler is executed  (1), accepting as
     input user-coded subschema descriptor statements, (2)
     defining a particular program view of the database.
     The Data Directory file is used as a resource to
     establish knowledge of the schema description of  the
     database and ensure that each subschema DDL statement
     is consistent with that description.  The presence of
     any area,  data-item,  record, or set included in  the
     subschema is verified against the schema description.
     The compiler registers in the Directory all data
     pertaining to this particular subschema description,
     replacing any previous such information associated  with
     the same subschema name.

     IDMSUBSC generates several output listings (3) during
     the course of  its processing:

          The 'Subschema  DDL Processor  Input Listing1
          displays all user-entered card images into the
          compile run,  along with  any diagnostics as
          appropriate.

          The 'Subschema  Data  Records'  listing shows  the
          make-up of each record type included in this
          subschema, as it would be visible to an application
          program.   These  data  descriptions are  in  COBOL
          data division format.

          The 'Definition  Entries' listing  has three  parts:

               Record - the names of all records included
               in this subschema view.  These names are shown
               in COBOL data division format.
                      VI-9

-------
                 SUBSCHEMA
                  SOURCE
                 STATEMENTS
                                             IDMSUBSC
                                             COMPILES
                 SUBSCHEMA
                  DDL INPUT
                   USTING
                              IDMS
                              DATA
                            DIRECTORY
                                             ASSEMBLES
                                               SOURCE

ASSEMBLES
USTINGIS)
^ 	


V
ASSEMBLES
                   UNK EDIT
                     MAP
LINK EDIT
                                                  *  (3
(
                                                OBJECT
                                               SUBSCHEMA
                                                MODULE
               FIGURE  vi-5   SUBSCHEMA  COMPILE PHASE

-------
               Set - the names of all sets included in  this
               subschema view

               Area - the names  of all areas  so  included

          A fourth type of listing, the 'DBMS  Bind Calls,1
          indicates all BIND statements  (in COBOL  CALL
          format) that an application program  would have  to
          issue  in the run-time  environment to make
          available all record  types  in this subschema view.

          Since  the IDMS DML processors automatically
          provide these BIND source  statements at  compile
          time and since the call form these  requests  take
          is dependent upon compile  mode, this listing is
          not referenced as frequently by the data
          administrator as it once was.

     The Subschema Compiler also produces an  assembler
     source file  (4) which is used as  input to a  standard
     OS or DOS assembly (5),  which  in turn  is  followed  by a
     link edit (7).  The  listings coming out of these steps
     are the normal assembler listing(s)  and link edit  maps
     with which  the user is familar.

     The object subschema module  (9) which results is  stored
     in the user  core-image or  load library. The  name under
     which it is  available must be the same as the 'subschema
     name1  assigned  in  the  source subschema statements for
     the subschema view.

The Subschema Compiler can be executed any number  of times
in conjunction with a given schema description.  Thus,  as
many program views as the DBA thinks  are necessary can be
established.  Each resultant  subschema module  can be  shared
by as many application programs  as required.

Syntax and Semantics

The Subschema DDL (Data Definition Language)  defines the
user's  view of the overall  database.   It  consists of  two
major statement groups:

     SUBSCHEMA IDENTIFICATION DIVISION

     SUBSCHEMA DATA DIVISION

The statement groups must appear in  the sequence shown.  Each
describes a specific aspect of the subschema and begins  with
a descriptive paragraph name.
                      VI-10

-------
The SUBSCHEMA IDENTIFICATION DIVISION  statements  identify
the subschema name, Device-media, name,  author, date,
installation, and remarks.

The SUBSCHEMA DATA DIVISION statements specify the area(s),
record(s), and set(s)  that are to be included in the
subschema view being defined.

Specific  information concerning  each Subschema  Division
statement can be found  in the IDMS Database Design and
Definition Guide, Section 8.1.

Execution Instructions

To define a subschema, execute the program IDMSUBSC followed
by the assembler and linkage editor.  Input to IDMSUBSC
consists of a source deck written in subschema DDL.  Output
is as follows:

     A description of the subschema stored  in the  Data
     Directory

     An assembler source program

     A subschema DDL input listing, a subschema data records
     listing, and a definition entries listing

Input to  the assembler  consists  of the  source program
produced  by the assembler.  Output is as follows:

     A relocatable program

     An assembler listing

Input to  the linkage editor consists of  the relocatable
program produced by the assembler.  Output is as follows:

     An object program

     A linkedit map

Figure VI-5 illustrates these operations.

The JCL for performing  the operations  is as follows:

                         IDMSUBSC
//STEP1        EXEC PGM=IDMSUBSC,REGION=130K
//STEPLIB      DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
//SYSDDL       DD   DSN=SYS 2.IDMS.OPPMON.DIRECTRY,DISP= (OLD,PASS)
                      VI-11

-------
//SYSJRNL
//SYSLST
//SYS OUT
//SYSPCH
//SYSIPT
subschema
//STEP2
//SYS PRINT
//SYSLIB
//SYSUT1
//SYSUT2
//SYSUT3
//SYS PCH

//SYS IN
//STEP3

//SYS PRINT
//SYSLIN
//SYSUT1
//SYSLMOD
                    DD   UNIT=TAPE62,VOL=OPPDIR,DSN=OPPJRNL,DISP=MOD
                    DD   SYSOUT=A,DCB=BLKSIZE=133
                    DD   SYSOUT=A
                    DD   DSN=&&SUBSC,DISP=(NEW,PASS),UNIT=SYSDA,
                         SPACE=(CYL,(5,1)),
                         DCB=(RECFM=FB,LRECL=80,BLKSIZE=1500)
                    DD   *
               DDL source statements  (omit  if  SYSIPT is other than *)
                    EXEC PGM=IEUASM,PARM='NOLOAD,DECK',REGION=80K
                    DD   SYSOUT=A
                    DD   DSN=SYS1,MACLIB,DISP=SHR
                    DD   UNIT=SYSDA,SPACE = (CYL,(2,2))
                    DD   UNIT=SYSDA,SPACE=(CYL,(2,2))
                    DD   UNIT=SYSDA,SPACE=(CYL,(2,2))
                    DD   DSN=&&ASMBL,UNIT=SYSDA,DISP= (NEW,PASS),
                         SPACE=(80,(400,40))
                    DD   DSN=&&SUBSC,DISP= (OLD,DELETE)
                    EXEC PGM=IEWL,PARM='XREF,LIST,LET,SIZE = (96K,12K)
                         REGION=96K
                    DD   SYSOUT=A
                    DD   DSN=&&ASMBL,DISP= (OLD,DELETE)
                    DD   UNIT=SYSDA,SPACE = (TRK, (20,5))
                    DD   DSN=SYS2.IDMS.OPPMON.SUBSDMCL(SUBSC) ,DISP=OLD
     SYS2.IDMS.LOAD  =  data  set  name  of  IDMS load library
     SYSDDL = DD name of Data  Directory
     SYSJRNL = DD name of Journal File

     SYSOUT=A = printed output
     &&SUBSC = name of temporary data  set  for  subschema output
     * = input file
     &&ASMBL =name of temporary data set for assembler output
     SUBSC = name for subschema.

(5)   Data Base Procedure

     Functional Description

     Database procedures are routines associated with a specified
     DML function for a particular  record  type.  Defined  in  the
     schema record description statements, these procedures
     intercept and process service  requests between  the
     application program and DBMS.   Database procedures, which
     are normally written by the database  administrator,  are
     invisible to the application program.

     In general, database procedures increase  the  number of data
     management capabilities available  to the DBA.  Some specific
     procedure functions include the ability to:

          Modify a schema record on its way in  and out of the
          database
                           VI-12

-------
Compress/expand data-items and records to save disk
storage space

Validate the data-items and establish data  integrity

Encode/decode data-items and records to ensure physical
data security

Compute data-items based upon the value of other data-
items

Perform search algorithms and gather statistics

Ensure data privacy at the record occurrence level as
described below:

An important feature concerning database procedures is
that procedures have access to the schema record
occurrence, while  the application program, of course,
has access only to the segmented subschema  record.

This fact allows the DBA to implement privacy locks
within database procedures that will intercept and
protect sensitive data  (i.e., those few occurrences of
a given record type that may not be viewed  by an
application).  For example, the DBA may write a
procedure to allow a program access to all  personnel
records except those with account numbers less than
1000.   If  the  record  fails the procedure's range check,
access to the record is denied,  and a user error  code
is returned to the program,  indicating violation  of a
DBA-implemented privacy lock.

Thus,  database procedures can determine not  only which
record types may be  seen by an application program, but
also which occurrences  of a given type may be accessed.
The end result is that the DBA can provide  record
occurrence privacy through database procedures.

Service application database requests from  non-IDMS
tape or disk files.   Using  procedures, the DBA can
include non-IDMS files  in his configuration and process
application database requests from those files.   The
application program remains unaware of either the
procedure's presence or the  source of the logical
records returned to working storage.   Normal
application processing continues despite the switch
from IDMS to non-IDMS mode.   The  application program
still sees only the segmented subschema record.  Thus,
the advantages of  data independence are extended to
non-IDMS records and files.
                 VI-13

-------
Syntax and Semantics

A database procedure,  which  is defined  in the schema record
description statements and associated with one or more DML
functions, is  invoked  using  the  following CALL command:

                                                  CONNECT
                              BEFORE              MODIFY
CALL database-procedure-name  AFTER               DISCONNECT
                              ON ERROR DURING     ERASE
                                                  FIND
                                                  GET

This statement is formatted according to the following
rules:

     Multiple procedures may be associated with the same
     record type by specifying separate CALL clauses for
     each procedure.   However, the  following options apply
     to any procedure:

          One or more DML entrance times (BEFORE,  ON  ERROR
          DURING, AFTER)  may be  specified for  the  same
          procedure.

          One or more DML functions may be specified for
          the same procedure.

          The same procedure may be specified in different
          CALL clauses,  and  thus  may  be associated with
          different record types.

     Procedure entrance time is controlled by specifying
     one of the following options:

          BEFORE - the procedure is entered immediately
          prior to the performance of the DML function(s).

          AFTER - the  procedure is entered immediately after
          the DML function and error procedures, if any, have
          been performed.

          ON ERROR DURING -  this  is a special  form of AFTER
          procedure which is entered  each time DBMS detects
          an error condition (i.e., error status indicator
          not equal to zeros) during  the performance of the
          DML function.

     Procedures are executed only on explicit DML functions.
     For example, if there is an  implied CONNECT on a STORE
     (set membership  is  automatic), a procedure declared as
     CALL . . . BEFORE CONNECT would not be invoked at the
     time a STORE is  executed.

                      VI-14

-------
Coding Data Base Procedures

A Data Base Procedure is coded as a subroutine employing
standard OS calling sequences.  It is  invoked by IDMS on
behalf of an application program.  When invoked,  program
control is passed along with the  following  information
blocks:

     Procedure Control Block
     Application Control Block
     Application Procedure Communication data
     Record Control Block
     User Record Occurrence

The Procedure Control  Block provides information  reflecting
the general conditions under which the procedure is being
invoked.   The information passed is the same for any
procedure invoked by any DML function.

The Application Control Block is  a second fixed  block of
information that reflects conditions  in the application
program at procedure execution time.  This  data  includes  the
name of the subschema associated with the application
program that caused the  procedure to be  invoked followed by
a copy of  the  first  200 bytes of  the user IDMS Communcations
Block.  This is the first  portion of the same area used  for
all communication between IDMS and application programs.

The third block of  information passed by IDMS is the
Application-Procedure Communication data.   The  contents of
this block are determined completely by the user and include
any data  to be passed between the procedure and  the
application.  Consequently, the area may consist  of any
number of user-defined data fields.  Normally, however,  the
procedure will operate invisible  to the application program.
In these  instances, the  null  indicator is  set by  IDMS to an
alphanumeric value of  'NULL1,  indicating that no information
is being  passed between the application and procedure.

The Record Control Block gives the procedure information
regarding the specific record type upon which it is
operating.  This information, which consists of data specific
to record-level procedures, is essential, since any procedure
may be invoked for more than one record type.

The fifth argument passed by IDMS is  the actual  record
occurrence as described in the schema, including  all  data-
items within the record regardless of the application's
subschema view.

A detailed description of the layout  and contents of these
blocks is found in the IDMS Database  Design and Definition
                      VI-15

-------
Guide, 10.5-10-10.

Link-Editing Database Procedures

Data-base Procedures are compiled, link-edited and stored
into a load library which contains all IDMS data  base
procedures.
                      VI-16

-------
2.    APPLICATION  PROGRAM SUPPORT

     (1)   Data Directory Reports

          Report  Description

          An integral part of the IDMS  system is a group of ten Data
          Directory reports provided by the IDMSRPTS program.   These
          reports furnish  system documentation for use by programmers,
          analysts, database  administration staff, data processing
          management, and  user department personnel.  Figure VI-6
          illustrates the  reports available from the Data  Directory.

          The  reports and  brief  descriptions  are listed below:

               Range  rtap - ranges of each area  within  the  schema and
               each record type  within  each area

               Device-Media Control  Listing - physical  file and
               corresponding area mapping  information  as defined in
               the schema

               Program  Cross References - each  host language program
               (COBOL or PL/I) written under each subschema, including
               the most recent compilation  date

               Schema/Subschema  Record  Descriptions -  schema and
               subschema-view record specifications

               Schema/Subschema  Set Descriptions - schema and program-
               view set specifications

               Subschema Data Dictionary Listing - specifications for
               records  and all component data items, particularly for
               use by non-EDP personnel in  conjunction  with the
               IDMS/CULPRIT retrieval system

               Procedure/COPY Listing - record  descriptions and
               procedure statements  categorized by language or  by mode

               Protocol Listing  - specifications of user  environment
               'calling  sequences' and formats.

          Several of  the Data Directory reports are designed for use
          by management and user department personnel,  as well  as
          programmers and DBAs.   In general,  the  Data Directory Reports
          Program (IDMSRPTS) communicates all database  specifications
          and  constraints  to the user.
                                VI-17

-------
   REPORT
   PARAMETERS
DATA
DIRECTORY
REPORT
PROGRAM
DATA
DIRECTOR?
FILE
                     SCHEMA
                     SETS
                     (SE70ES)
   3POGRAM
   CROSS-REF
   (PGMLSTj
                           ATS
                          D I RECTOR?
                          •_ISTING
                          {OATQIRj
FIGURE vi-6   OQCUMENTATION  FROM THE  DATA  DIRECTORY SYSTEM

-------
Input Parameters

To provide maximum flexibility  in  selecting  reports, the
IDMSRPTS control parameter syntax  permits  the  user  to
request reports for any number  of  schemas  and  subschemas.
IDMSRPTS syntax follow:
SCHEMA =
           (schema-name-1  [schema-name-2]  . .  .]_

           HIGHEST
VERSION =  ALL
           (schema-version-1  [schema-version-2]  . . .]_

SUBSCHEMA  =  ALL
              (subschema-name-1  [subschema-name-2]  . . .)_

ALLRPT
     RNGMAP
     DMCLST
     PGMLST
     RECDES
     SETDES
     SUBREC
     SUBSET
     DAIDIK
     MODLST
     PRTLST

Specific rules for each statement can be  found  in  the  IDMS
Database Design and Definition Manual Section 9.

JCL

To generate Data Directory reports, execute the program
IDMSRPTS.   Input to IDMSRPTS consists of parameter cards and
the information contained in  the  Data Directory.  Output can
be any of  the following as requested on the parameter  cards:

     Range map (RNGMAP)

     Device-media control listing  (DMCLST)

     Program cross-reference  listing (PGMLST)

     Schema record descriptions  (RECDES)

     Schema set descriptions  (SETDES)

     Module/copy listing  (MODLST)

     Protocol listing  (PRTLST)
                      VI-13

-------
          Subschema record descriptions (SUBREC)

          Subschema set descriptions (SUBSET)

          Data  dictionary listing.  (DATDIR)

     The  JCL for performing the operation is as follows:

                              IDMSRPTS
     //STEPLIB
     //SYSDDL
     //SYSJRNL
     //SYSOUT
     //SYSLST
     //SYSIPT
     parameter  cards
EXEC PGM=IDMSRPTS,REGION=180K
DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
DD   DSN=SYS2.IDMS.OPPMON.DIRECTRY,DISP=(OLD,PASS)
DD   DUMMY
DD   SYSOUT=A
DD   SYSOUT=A,DCB=BLKSIZE=133
DD   *
     SYSDDL  = DD name of Data Directory
     SYSJRNL = DD name of Journal File
     SYSOUT=A = printed output

(2)   DML  Protocols

     Application Program Usage of Protocols

     The  use of various DML protocols controls the translation
     of DML  statements into  the appropriate programming language
     source  code.  The  first  level of control is the programming
     language itself, that is identification of the  correct syntax
     for  this program.   The  second  function  is  to  vary the
     translation of  the DML statements as it applies to various
     modes of operation  (e.g., batch vs teleprocessing). The third
     major  function  is to identify programs into which the DML
     processor is to copy additional data descriptions or code
     sections.

     Utility Description

     The  IDMS Data Directory Utility (IDMSCLUC) establishes,
     updates, and deletes various  controls for the DML processors.
     Specifically, IDMSCLUC  permits  the  user  to:

         Build non-IDMS data structures into the
         Dictionary/Directory file for subsequent COPY by the
         DMLC  or DMLP processors

         Store and  COPY module  source code  (e.g., IDMS-STATUS
         SECTION)
                           VI-19

-------
     Establish protocol  information  needed  by the DMLC and
     DMLP processors  in  generating  "calling sequences'
     between application programs and  IDMS

     Register programs in  the  Directory  file  to  restrict
     program compilation against specific subschemas

     Delete schema descriptions  from the Directory

     Transfer program activity statistics between subschema
     and schema versions

     Establish a system-wide quote character.

A Full Description of the  IDMSCLUC control  options along
with information on individual action codes can be found in
the IDMS Database Design and Definition Manual,  Section  12.

Utility Execution

Prior to executing the DML processors,  the  IDMSCLUC utility
must be executed.  Input to IDMSCLUC consists of  a  source
deck of CLUC statements.  These CLUC statements can be found
on the IDMS, version 5.0 installation  tape in  file  3  (SOURCE),
member = IDMSPROT.  Input only the source needed  by OPP
(COBOL, BATCH).   Output  is as  follows:

     Protocol and other  information  stored  in the Data
     Directory

     A CLUC source listing.

Figure VI-7 illustrates  this operation.  The  JCL  for
performing this operation  is as  follows:

                          IDMSCLUC
//             EXEC PGM=IDMSCLUC,REGION=180K
//STEPLIB      DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
//SYSDDL       DD   DSN=SYS 2.IDMS.OPPMON.DIRECTRY,DISP= (OLD,PASS)
//SYSJRNL      DD   UNIT^TAPE 62,VOL=OPPDIR,DSN=OPPJRNL,DISP=MOD

//SYSOUT       DD   SYSOUT=A
//SYSLST       DD   SYSOUT=A
//SYSIPT       DD   *
CLUC source statements  (omit  if SYSIPT  is  other  than  *)
SYSDDL = DD name of Data Directory
SYSJRNL = DD name of Journal  File
SYSOUT=A = printed output
                      VI-20

-------
CLUC  CONTROL
 STATEMENTS
IOMSCLUC
  DATA
DIRECTORY
                                SOURCE
                                UST1NG
      HGURE VI-7STORING PROTOCOL AND OTHER INFORMATION

-------
(3)   Program Compilation and Link-Edit Procedures

     JCL  for COBOL Program

     COBOL  programs are preprocessed by  the  DMLC  compiler to
     convert DML statements into standard COBOL.

     Output is as follows:

         A source COBOL program

         A DML source listing and diagnostics

     Input  to the ANS COBOL compiler consists of  the  source
     program produced by IDMSDMLC.   Output is as  follows:

         A relocatable program

         COBOL listings

     Input  to the linkage editor consists of the  relocatable
     program.   Output is  as follows:

         An object program

         A link-edit map

     Figure VI-8 illustrates this operation.

     The  JCL to perform the preprocessing and compilation is as
     follows:

     //STEP1        EXEC PGM=IDMSDMLC,REGION=160K
     //STEPLIB      DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
     //SYSDDL       DD   DSN=SYS2.IDMS.OPPMON.DIRECTRY,DISP= (OLD, PASS)
     //SYSJRNL      DD   UNIT=TAPE62,VOL=OPPDIR,DSN=OPPJRNL,DISP=MOD
     //SYSLST       DD   SYSOUT=A,DCB=BLKSIZE=133
     //SYSPCH        DD   DSN=&&DMLC,DISP=(NEW,PASS),UNIT=SYSDA,
     //                   SPACE=(TRK,(5,5)),
     //                   DCB = (RECFM=FB,BLKSIZE=3120,LRECL=80)

                source statements (omit  if SYSIPT is  other than *)
                    EXEC PGM=IKFCBLOO,REGION=160K
                         PARM='DECK,NOLOAD,NOLIB,BUF=50000,SIZE=150K'
                         SYSOUT=A
                         DSN=SYS1.COBLIB,DISP=SHR
                         UNIT=S YSDA,SPACE = (TRK, (10, 5) )
                         UNIT=SYSDA,SPACE=(TRK, (10,5))
                         UNIT=SYSDA,SPACE=(TRK, (10,5))
                         UNIT=SYSDA,SPACE=(TRK, (10,5))
                         DSN=&&COB,DISP=(NEW,PASS),UNIT=SYSDA,
//SYSIPT
COBOL/DMLC
//STEP
//SYS PR I NT
//SYS LIB
//SYSUT1
//SYSUT2
//SYSUT3
//SYSUT4
//SYS PUNCH
DD
source
EX)
DD
DD
DD
DD
DD
DD
DD
                           VI-21

-------
 SOURCE
PROGRAM
         COBOL
       LISTINGS £r
      DIAGNOSTICS
         UNKMAP
                                   STEP 1
IOMSDMLC
                                    STe? 2
ANS COBOL
 COMPILER
                                    STE? 3
  LINKAGE
  EDITOR
                     OPTIONAL
                      SOURCE
                      LISTING
                  DIAGNOSTICS
 OBJECT
PROGRAM
                 FIGURE vi-SCOMPILJNG COBOL PROGRAM
                               IOMS

                    1ISE3 PROGRAM
                     RUN-UNIT
           FIGURE VI~9 EXECUTING A PROGR.-O1 IN LOCAL MODE

-------
     //                   SPACE=(80,(400,40))
     //SYS IN         DD   DSN=&&DMLC,DISP= (OLD, DELETE)
     //STEP3         EXEC PGM=IEWL,PARM='XREF,LIST,LET,SIZE = (95K, 12K)
     //                   REGION=96K
     //SYSUT1        DD   UNIT=SYSDA,SPACE=(TRK,(20,5))
     //LIB           DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
     //SYSLMOD       DD   DSN=SYS2.IDMS.OPPMON.PROG. (USERPROG ) ,DISP=OLD
     //SYSLIN        DD   DSN=&&COB,DISP=(OLD,DELETE)
     //              DD   *
     /INCLUDE LIB (IDMS,IDMSCANC)
     //SYSLIB        DD   DSN=SYS1.COBLIB.DISP=SHR

       SYSDDL = DD  name  of Data Directory
       SYSJRNL = DD name of Journal File
       SYSOUT=A = printed output
       &&DMLC = name of  temporary data set for DMLC  output
       SYSl.COBLIB  =  data  set  name of  ANS COBOL Library
       &&COB = name of temporary data set for COBOL  output
       SYS2.IDMS.OPPMON.PROG (USERPROG) = name of  user
        applications program

(4)  Execution Instructions

     In local mode,  the  applications  program is linked to  IDMS,
the batch interface module.  The program is started as a job step
and the interface module loads the appropriate subschema and a
copy of the  DBMS.  The DBMS then loads the appropriate  DMCL module.
Figure VI-9 illustrates  database access via a local  DBMS.  The
JCL to start a program using IDMS in local mode  is as  follows:

                              LOCAL MODE
     //STEPLIB
     //IDMSJRNL

     //DATABASE
     //CODEFILE
     //INDEX
     //IDMSJRNL 9
     //SYS OUT
     //SYSUDUMP

       USERPROG
       IDMS. LOAD
   EXEC  PGM=USERPROG
   DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
   DD   DSN=SYS2.IDMS.OPPMON.PROG,DISP=SHR
   DD   DSN=SYS2.DMS.OPPMON.SUBSDMCL,DISP=SHR
   DD   DSN=SYS2.IDMS.OPPMON.DBRROC,DISP=SHR
        DD   DSN=IDMS.JRNLFILE,DISP=MOD,UNIT=TAPE62,
             VOL=PRIVATE
        DD   DSN=DBFILE1,DISP=OLD
        DD   DNS=DBFILE2,DISP=OLD
   DD   DSN=DBFILES3,DISP=OLD
        DD   DSN=IDMS.JRNLFILE,DISP=MOD,UNIT=TAPE62,
        VOL=PRIVATE additional JCL for
            applications program, as required
   DD   SYSOUT=A
   DD   SYSOUT=A

 name of user's  applications program
! data set  name  of  IDMS  load library
                            VI-22

-------
IDMS.JRNLFILE = data  set  name  of  Journal  File
DBFILE1 = data set name of database
DBFILE2 = data set name of code file
DBFILE3 = data set name of index file
SYSOUT  = A = printed output
                    VI-23

-------
3.    DATA BASE LOADING

     (1)   Data Base Initialization

          File Allocation

          The OPP Monitoring Data Base will be allocated  as  three  OS
          files.   The  largest file contains the main data base and
          encompasses  the STATE area and the RESIDUE area.  The file
          will be allocated initially to be 70% full. The  second file
          contains codes found in the data base along with their
          associated meanings.  This file  requires 3 data base pages
          or  1 track.  The  third  file holds the indices for the
          sequential processing facility.   Prior  to allocating this
          file, the  utility  described in the  Data  Base  Design volume
          (page 52)  must be run to obtain  the size parameters.  The
          following JCL  will be used for the allocation.

          //   EXEC       PGM=IEFBR14
          //DATABASE     DD   DSN=DBFILE1,UNIT=3 330-1,VOL=SER=USRVOL
          //    SPACE=(3156,20575),DISP=(NEW,CATLG)
          //CODEFILE     DD   DSN=DBFILE2,UNIT=3 330-1,VOL=SER=USRVOL
          //    SPACE = (3156,4),DISP=(NEW CATLG)
          //INDEX        DD   DSN=DBFILE3,UNIT=3330-1,VOL=SER=USRVOL,
          //    SPACE=(1180,numpg) ,DISP= (NEW,CATLG)

          DBFILE1 = data set name for DATABASE file
          DBFILE2 = data set name for CODEFILE file
          DBFILE3 = data set name for INDEX file

          Area Initialization

          IDMSINIT must  be  the first program run against  any  IDMS
          database prior to processing.  This step  is mandatory, since
          each page of the  user's  database must be  formatted  prior  to
          storing records in the  database.   Consequently,  IDMSINIT
          performs the following  initializing actions:

              Writes  out direct  access  blocks  (one block  = one page)
               to the  database

               Formats each page   (e.g., writes page numbers)

              Reformats all or sections of the database  following
              device  (e.g., head  crash)  or program failure

          Depending on user specification,  IDMSINIT initializes the
          database in  its entirety or by FILE.  In  addition, IDMSINIT
          may be  used  to re-initialize the database by AREA or FILE.
                                VI-24

-------
     In order to initialize the data base  in  its  entirety  the
     following cards must be used:

          PROCESS = TOTAL,DMCL=OPPDMCL

     To initialize or reinitialize the database, execute the
     program^IDMSINIT.  Input consists of  parameter cards. Output
     is as follows:

          Formatted or reformatted pages  in the database

          A parameter listing

          Figure VI-10 illustrates this  operation.  The JCL for
          performing the operation is as  follows:

                              IDMSINIT


     //             EXEC PGM=IDMSINIT
     //STEPLIB      DD
     //SYSLST       DD
     //DATABASE     DD
     //CODEFILE     DD
     //INDEX        DD
     //SYSIPT       DD
     parameter cards
DSN=SYS2.IDMS.LOAD,DISP=SHR
SYSOUT=A
DSN=DBFILE1,DISP=OLD
DSN=DBFILE2,DISP=OLD
DSN=DBFILE3,DISP=OLD
     SYSOUT=A = printed output
     DBFILE1 = data set name of database  file
     DBFILE2 = data set name of code file
     DBFILE3 = data set name of index file

(2)  Initialization of CODE Area

     Prior to loading the main database, the CODE record  must be
initialized and loaded.   An application program  (CODEINIT) must
be run which will store blanks in the CODE  field  and  20  in  the
length field.  Then  a second application program  (CODELOAD)  must
be run which will input a file containing the CODE, NAME
combinations and store the values into  the  database CODE record.

(3)  Data Base Loading

     Loading of  the  main data base requires  running two programs.
The first program (RESLOAD) will input  a  file containing the
residue codes,  residue  class, names, and  system warning levels
and store these values into the data base RESIDUE record.  The
second program (DBLOAD)  will input files  containing information
on states,  sites, and samples and will  load  the  rest of the  data
base for all monitoring systems. The input files should be sorted
into state,  site, sample date order prior to running DBLOAD.
                           VI-25

-------
PARAMETER
  CARO(S)
IOMSINIT
                                 DATABASE
                                PARAMETER
                                  UST1NG
            FIGURE VI-10   INITIALIZING THE DATABASE

-------
4.    DATA BASE UPDATING

     (1)   Input File Creation

          The OPP Monitoring Data Base will be updated via  input
     transactions processed by  the  Edit/Update Subsystem.  Three  input
     files can be used  depending on the type of Edit/Update being run.
     These files are as follows:

          "New" input transactions

          Previously rejected Humans transactions

          Previously rejected non-Humans transactions.

     The  latter two files are created as output by a previous run of
     the  edit/update subsystem. These  two  files  will  initially  be
     empty the first time the subsystem is run and also when no
     transactions are rejected  during the previous run.

          The new input transaction file is created by keying the
     input forms provided to SSB  by FMB.

     (2)   Edit/Update Cycle

          The users are to provide the type of cycle to be  run.   Two
     modes are possible. These are an edit only mode where transactions
     are  edited with no attempt to update,  and edit  and update mode
     where transactions are edited and if clean an attempt  to update
     is made.   The  mode  must be provided  as input  to the subsystem by
     the  programmer responsible for system  operation.   In  either mode
     of operation, reject files  are  created. Additional output include
     an error list providing details of errors in rejected
     transactions.   If  in  the edit  only mode,  all  transactions are
     rejected to the reject files.

     (3)   Correction Cycles

          The error list is sorted by transport mode and  should be
     referred to EMB with the input forms.  The  Project Officer  will
     then make the required corrections to the transactions by
     accessing the appropriate  reject  files via ALPHA.  The  next time
     the  update program is run, the  new rejection files are used as
     input along with any new input transaction file created since
     the  last update run.
                                VI-26

-------
5.    BACK-UP AND RECOVERY

     (1)   Database Back-Up

          Backup simply means keeping a second copy of the database.
     Two  backup media  are  provided under IDMS:

     Dump Pile       By dumping the database,  the  user  obtains  a
                    complete picture  of the database as it existed at
                    a  point  in time.

     Journal File   By recording  checkpoints  and  data updates as they
                    are made,  the user maintains a running picture of
                    the database.

          Database Dump Description

          The purpose  of the IDMSDUMP program is  two-fold:

               To copy all or specified AREAS of  an IDMS  database to
               tape for backup security

               To provide  a  database  statistics report

          IDMSDUMP performs  the following functions:

               IDMSDUMP reads a PROCESS card  to determine the  user's
               run time requirements.

               If PROCESS=AREA is specified,  additional cards  follow
               the first,  giving  the  names of  the AREAS to be  copied.

               If REPORT=YES or ONLY  is specified,  IDMSDUMP will
               collect data  and compile reports about  the current
               content and status of  the database,  including space
               utilization and record distribution  data.   If
               PROCESS=AREA  is designated the reports  are generated
               by AREA;  for  PROCESS=TOTAL the  report covers all AREAS
               known to the  DMCL  tables entered.

          Database Dump Syntax

          First Card

                                                                YES
               PROCESS = T2IJL, DMCL=device-media-name , REPORTS=NO
                                VI-27

-------
Subsequent Cards . .  .

     AREA=area-name

Specific information on each statement can be  found  in  the
Database Design & Definition manual, Section 11-34.

IDMSDUMP Output Format

IDMSDUMP writes the dump tape in a  fixed format such  that
block sizes are equal to the page size of the database.   In
the first block of the file,  the program writes the  following
information only:

     Date the dump is taken - MM/DD/YY - 8 bytes

     Names of AREAS dumped - 16 bytes

     The literal 'END1 to specify end of AREA names.  One or
     more blocks may be written, depending on the  number of
     AREA names which must be stored.  Following these
     blocks,  the  database  is written beginning with the
     lowest numbered page to be copied.  Data is dumped  in
     ascending order by logical page number.

IDMSDUMP optionally produces a Database Utilization  Report
consisting of the following three sections:

     Section I is a summary report  of the area being  dumped
     and contains the following information:

          Gross page size

          Number of pages dumped

          AREAS dumped  (as known to  the DMCL tables)

          Space used (i.e., number of characters occupied)

          Space remaining (i.e., number of characters  of
          storage available)

          File utilization expressed as a percentage  of
          space used vs. space  available  rounded up to  the
          next integer.

     Section II is a space distribution table, indicating
     how the occupied space is allocated among each  record
     type.   The  table, which can keep track of  up  to  1000
     record types,  contains  the  following information:
                      VI-28

-------
           Record  type  listed  by  record ID assigned in the
           schema

           Number  of occurrences  of  each  record type found
           in the  dumped  area

           Total space  used -  length multiplied by number of
           occurrences

           Percentage of  space used  -  the percentage of the
           total space  occupied by  this record type

     Section III  is a  second  space  distribution table,
     indicating:

           Occupancy range - the  number of characters per
           page in multiples of 100  through 1,599

           Number  of pages in  each  range.

If PROCESS=TOTAL  is specified  on the IDMSDUMP input PROCESS
card, the utility produces area as well  as global  total
reports.

Database Dump JCL

Figure  VI-11 illustrates the  operation.   Use  the  following
JCL  to  run  IDMSDUMP:

//         EXEC PGM=IDMSDUMP
//STEPLIB      DD  DSN=SYS2.IDMS.LOAD,DISP=SHR
//SYS001       DD  DSN=IDMS.DUMP,DISP=(NEW,KEEP),UNIT=TAPE62,
//                  DCB=BLKSIZE=3156
//DATABASE     DD   DSN=DBFILE1,DISP=OLD
//CODEFILE     DD   DSN=DBFILE2,DISP=OLD
//INDEX        DD   DSN=DBFILE3,DISP=OLD
//SYSLST       DD   SYSOUT=A
//SYSIPT       DD   *
parameter  cards

IDMS.DUMP = data  set name of dump tape
DBFILE1 =  data set name  of data  base
DBFILE2 =  data set name  of CODE  file
DBFILE3 = Data Set Name  of INDEX file
SYSOUT=A =  printed output

Database Dump Frequency

The frequency of  data base dumps  is a variable of the amount
of updating and the required time for recovery in the event
of a system crash. Since the OPP Monitoring system is batch
and the level of data  base updates  is  relatively  low, a
weekly dump is recommended.

                       VI-29

-------
            PARAMETER
             CARO(S)
             IDMSDUMP
                                            DATABASE
FIGURE v-i-llDUMPING THE DATABASE

-------
Journaling

The journal tape provides a running record of all data base
changes which have taken place  since the last data base dump
was taken. A new tape (or series of tapes) should be started
following each back-up data base dump.  The  combined use of
these two sets of tapes  for recovery  purposes will  be
discussed in the next section.

The journal tape must have a standard  OS tape label.  If the
IDMS data base sustains heavy usage, a dedicated tape drive
is desirable.  Oherwise,  the journal tape  must  be kept in
the operations area and remounted each  time an IDMS job runs.

The IDMS journal file contains each run-unit's  identity,
starting time,  ending time, and  any database changes effected
during program execution.  As  each program signs onto  the
DBMS,  a beginning-of-the-job checkpoint is written  to  the
journal tape.  The checkpoint  contains various  program-id,
starting time,  and area usage mode  information.   If a program
READIES an area in an UPDATE usage mode, 'before1 and 'after1
page images are written to the  journal  for  each page changed.
'Before1 images provide a 'snapshot1 of  a database page prior
to modification(s).  'After'  images provide a page image that
includes all  changes made while the page  was in the  buffer
pool.   When a run-unit issues  a FINISH for a given area, an
end-of-job checkpoint is logged on the journal.  This
checkpoint contains program-id and ending time  information
as well as run-unit statistics.  These statistics include
the number of pages read from  and written to the  database,
as well as other information.   If  the  run-unit  terminates
abnormally, an ABRT  checkpoint that does not contain  these
access statistics is written to the journal  file.

The DBA may use the contents of the journal  tape  in
conjunction with the IDMSJFIX, IDMSRBCK,  and IDMSRFWD
utilities to  recover the database following a system failure.
The utilities will restore the database as  follows:

     IDMS Rollback erases a run-unit's effect by reapplying
     'before1  page images  to a  specified  BGIN checkpoint.

     IDMS Rollforward restores the database to  a  specified
     time by  applying 'after1  images to all  or  selected
     database areas.

A Journal File is created by assigning a  file to  an
application program  (local mode)  or CAMP (Central Version).
IDMS automatically writes the following checkpoints and data
images on the Journal File:
                      VI-30

-------
          BGIN Checkpoint - Written when an application program
          becomes active.   An  AREA  checkpoint  is sometimes used
          for overflow information  from a BGIN checkpoint.

          ENDJ Checkpoint - Written when a program terminates
          normally.

          ABRT Checkpoint - Written when program accessing the
          database via CAMP terminates abnormally.   An  ABRT
          checkpoint cannot be written for a local program that
          terminates abnormally.

          DATA Before  Image - Written when a DML command  is issued
          that will  change data in  the database.  The affected
          database page is written  to  the Journal File  before
          the change is made.

          DATA After Image - Written when data  is changed  in the
          database.  The affected database page is written  to the
          Journal File after  the change is made, but before the
          page is written to  the database.

(2)   Recovery

     Recovery Techniques

     When a hardware or software failure causes a program(s)
     using the database to fail or  causes data  on the disk  to be
     physically lost,  a recovery operation  is  necessary.  The
     procedures required for  the recovery depend on the nature
     of  the crash:

          Program Running in Local Mode Fails -  Before the program
          can be  restarted, all  changes  made to the database  by
          the aborted  program  must  be  nullified; i.e.,  the  data
          as it existed at the start of the program must be
          restored.

          System  Fails - If the operating system fails,  it  is
          necessary  first to close out  the Journal File  properly:
          ABRT checkpoints for the  programs that were  active  at
          the time of  the  crash and an end-of-file marker  must
          be written.   The  database can  then be restored to a
          point where  no programs are  operating in update  mode
          and all affected programs can be  restarted.

          Disk Fails - On a disk failure, data is physically  lost.
          A new database must  be  initialized and loaded using
          the latest Dump  File.  The Journal File tapes  are then
          used to restore  the database to the latest point  where
          no programs  are  running in update mode.   Programs
          running at the time  of  the crash  can then be  restarted.
                          VI-31

-------
Each of the above causes of a system crash  are  explained  in
detail in the following paragraphs.

Figure VI-12 illustrates the crash  and  recovery  of  a  local
applications program.  The steps involved after  the crash
is detected are as follows:

     Execute the Program IDMSRBCK with  the operation
     PROCESS=TOTAL.  If more than one program has used the
     Journal File,  specify the date and time of  the last
     (most recent)  BGIN checkpoint.

     Restart the applications program.  Usually  it is best
     to use a new file for the Journal  File.

When a program running under the Central Version fails, a
console message appears indicating  that either:

     The program will be recovered  automatically.  This is
     possible if no programs are operating in  UPDATE  mode.

     The program cannot be recovered automatically  due to
     a usage mode conflict.

Figure VI-13 illustrates the recovery of the  database  after
a system crash.   The  steps  involved are as  follows:

     Recover the operating system.

     Execute the program IDMSJFIX.   This produces (1)  a new
     Journal File with ABRT checkpoints for  the  programs
     that were active at the time of the crash, plus an EOT
     marker,  and  (2)  PROCESS=TOTAL  and date/time cards for
     IDMSRBCK.

     Execute the  program IDMSRBCK using the  new Journal File
     and the parameter cards  produced by IDMSJFIX.  IDMSRBCK
     will recover the database to the most recent quiesce
     level-0 point.

     Restart all programs affected  by the IDMSRBCK
     operation.

Figure VI-14 illustrates the recovery of the  database  after
data is physically lost due to a disk failure.  The steps
involved after the problem is detected are as  follows:

     Abort local applications programs

     Initialize all or the  affected part of  the database on
     a new disk(s)  with the program IDMSINIT.
                      VI-32

-------
/

/

SGIN
PGM-A
3EFORE
INV « 2022
PGM-A
AFTER
INV = 2073
PGM-A
;
(
(
I
1
BEFORE
INV = 2073
PGM -A
AFTER
INV = 2096
PGM-A
(
)
1
\
3EFORE
INV = 2096
PGM-A
AFTER
INV - 3001
PGM-A

(
j
              DATABASE
FIGURE VI-12EXAMPLE OF RECOVERING DATABASE FOR LOCAL PROGRAM.

-------
BGIN
PGM-A
3EFORE
INV * 2022
PGM-A
AFTER
INV « 2073
PGM-A
3G1N
PGM-3
BEFORE
INV • 2073
PGM-S
AFTcR
INV - 2063
PGM-3
BEFORE
I.NV » 2063
PGM -A
AFTER
INV - 2086
PGM-A
A8RT
?GM-A
A3RT
PGM -3
              OATASASe
FIGURE VI-13  EXAMPLE OF RECOVERING DATABASE AFTER SYSTEM FAILURE.

-------
              IOMSHFWD
               USAGE
               REPORT
                                IOMSINIT
                               IDMSHSTR
                                           IOMSRFWD
                                        WOSTRECEMTCUIESCS
                                        LEVEL -0 POINT
FIGURE VI-LI EXAMPLE OF RECOVERING DATABASE AFTER DISK FAILURE.

-------
     Restore the database from the most recent Dump File  to
     the new disk(s) using the utility IDMSRSTR

     Using the most recent Journal File tape, execute the
     utility IDMSRFWD with the option  PROCESS=REPORTS

     Determine from the usage report produced by IDMSRFWD
     the date and time of the most recent 3GIN checkpoint
     where the quiesce level is 0 - i.e., the most  recent
     point where no programs were running in update mode

     Using all Journal Files taken since the dump,  execute
     the utility IDMSRFWD with the options PROCESS=TOTAL,
     and RESTORE=YES,  specifying the  date  and time  of the
     most recent quiesce level-0 point.  Restart all  programs
     that began after the most recent quiesce level-0 point.

Journal Tape Maintenance

IDMSJFIX (Journal Fix) is used to close properly a  journal
tape that does not contain a normal end-of-file mark.  This
abnormal condition can occur for several reasons -  hardware
crash,  power  failure,  or operating system malfunction.

Program functions are as follows:

     IDMSJFIX reads a syntax card to identify the  user's
     DMCL tables

     The utility is a tape-to-tape copy program that  reads
     from external  file name SYS001  (input)  and writes to
     external file name SYS002  (output).

     IDMSJFIX continues to read and write tape blocks  until
     it encounters a wrong block length error or a  date and
     time earlier than the date and time of  the previous
     block written.  When neither of these conditions occurs,
     IDMSJFIX assumes that it has reached end-of-file.

     At end-of-file IDMSJFIX:

          Writes ABRT (abort) checkpoints for all  active
          run-units

          Prints a parameter and checkpoint  listing  (see
          "Report  Format1  below)

          Generates two parameter cards to  be used  as  input
          to IDMSRBCK:

               PROCESS card
                      VI-33

-------
               RESTORE, PRINT,  DATE,  and  TIME  parameter card

          Under OS, sets a condition code indicating whether
          or not the recovery  cards  were generated

          Prints a  message  stating whether  or  not  database
          recovery  is  necessary

          Generates cards to rollback  database to  last
          quiescent point.

     Writes a normal end-of-file mark  to close out the
     journal tape.

In order to generate an ABRT checkpoint  and create the
punched card input  required by IDMSRBCK  to  roll back the
database to a quiesce  point, IDMSJFIX must  have access  to
the beginning-of-job checkpoint.  This requirement  becomes
especially important if a long-running batch update program
(with more than one reel of journal  file output) were
operating when the  system went down.  In this instance, the
user must begin execution of the IDMSJFIX utility  with the
journal file positioned to  the beginning of tape for the
first volume (the one  with  the beginning-of-job checkpoint
on it).  The  Journal Fix program will copy all reels of the
file,  despite the  fact  that intervening  volumes do  not
require an EOF mark.

To write a proper Journal File when  the  original Journal
File does not terminate at  normal end-of-file,  execute  the
program IDMSJFIX.   Figure VI-15 illustrates the operation.
The JCL to run the  above operation is:
//STEPLIB
//SYS 001
//
//SYS 002
DD
//SYSOUT       DD
//SYSIN        DD
parameter cards
PGM=IDMSJFIX
DSN=SYS2.IDMS.LOAD,DISP=SHR
DSN=IDMS.JRNFILE,DISP= (OLD,PASS),UNIT=TAPE62,
VOL=PRIVATE,DCB=BLKSIZE=3200
DSN=IDMS.JRNLFILE,DISP= (NEW,PASS),UNIT=TAPE62,
VOL=PRIVATE,DCB=BLKSIZE=3200
SYSOUT=A
IDMS.JRNLFILE = data set name of bad Journal File  (SYS001)
                and fixed Journal File (SYS002)
3200 = size of database page +  44 bytes
SYSOUT=A = printed output

Database Restore

The IDMS Security Restore Program (IDMSRSTR) uses  the  data
contained on a dump tape produced by  the  IDMSDUMP  program
to restore all or specified AREAS of  the  database.

                      VI-34

-------
              PARAMETER
                CARDS
BAD \
F1L£ I
IOMSJFIX
f FIXED
V FILE
V <
                USTING
TIGURL Vl-Ib  FIXING A JOURNAL FILE

-------
Program operation can be described  as  follows:

     IDMSRSTR reads a PROCESS card to determine the type of
     restore required and  the name  of  the  device-media
     control (DMCL) tables  for  the  run.

     If PROCESS=TOTAL is specified, no  additional cards are
     required.   All AREAS represented on the input dump tape
     (and known to the DMCL tables) will  be  restored.   If
     PROCESS=AREA, additional cards follow the first to
     specify the AREAS to  be restored.

     Initialize must be executed  if restore  will  reference
     a different  (new) portion.

To restore all  or portions  of the  database  from a dump tape,
execute the program IDMSRSTR.   Figure VI-16 illustrates the
operation.  Use the following JCL to run IDMSRSTR:

                         IDMSRSTR
//             EXEC IDMSRSTR
//STEPLIB      DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
//SYS001       DD   DSN=IDMS.DUMP,DISP=(OLD,PASS),UNIT=TAPE62,
//                  DCB=BLKSIZE=3156
//DATABASE     DD   DSN=DBFILE1,DISP=OLD
//CODEFILE     DD   DSN=DBFILE2,DISP=OLD
//INDEX        DD   DSN=DBFILE3,DISP=OLD
//SYSLST       DD   SYSO(JT=A
//SYSIPT       DD   *
parameter cards

SYS001 = DD name ofS-dump  tape
IDMS.DUMP = data  set name of dumpt tape
TAPE62 = symbolic device  name of dump  tape
3156 = tape block size; must be multiple  of  page  size
DBFILE1 = data set name of database
DBFILE2 = data set name of CODE file
DBILFE3 = dta set name of INDEX file
SYSOUT=A = printed output

Roll-Forward

IDMSRFWD is a diagnostic  and recovery  tool  that performs
the following tasks:

     Prints page images from the journal  file

     Restores 'after'  images  to  the  database
                      VI-35

-------
                       PARAMETER
                         CARO(S)
                        IDMSRSTH
                                                       0ATA3AS£
FIGURZ VI-16 RESTORING THE DATABASE FROM A DUMP TAPE
                       PARAMETER
                        CARD(S)
                        IDMSRFWO
                                                       DATABASE
                         U STING
        FIGURE VI-17 RESTORING .AFTER IMAGES

-------
     Prints a run-unit/area usage report

These three functions may be specified  independently  of one
another.   It  should  be noted that unlike IDMSRBCK,  which
restores the database from its final to its initial  (prior
to processing)  status, IDMSRFWD builds  up the database from
initial to a determined status with respect to all changes
contained on the journal file(s).

Program functions are:

     IDMSRFWD reads a PROCESS card to  determine whether
     processing will  be TOTAL or by AREA.   TOTAL indicates
     that 'after1  images will  be  applied to  the  entire
     database to a specified point in  time.  AREA indicates
     that 'after' images will  be applied only to pages  within
     the AREA specified on the parameter card.

     IDMSRFWD reads the journal tape forward and:

          Optionally  rebuilds the database  (RESTORE=YES or
          RESTORE=NO); the  utility will restore the database
          in whole or in part, according to whether
          PROCESS=TOTAL or PROCESS=AREA is  specified

          Optionally prints 'before1 and 'after'  page  images
          depending on whether or not  the PRINT option is
          specified

          Optionally  prints the run-unit/area usage  report
          only,  without  restoring the  database

     If the PRINT option is specified  (i.e., PRINT=BOTH,
     DECIMAL,  or HEXADECIMAL), the following information is
     printed:

          Program name and unique main memory address on
          each 'before1  and  'after'  page image

          Hexadecimal page images (PRINT=BOTH or
          PRINT=HEXADECIMAL) with page headers and footers

          Decimal page images (PRINT=BOTH or PRINT=DECIMAL)
          with page headers and footers and a single line
          documenting the type of change and line number of
          the record  last changed on the page

     Even if the PRINT option is specified  as NO, IDMSRFWD
     always produces:

          Parameter listing
                      VI-36

-------
          A single-line job checkpoint  listing

          End-of-job statistics  for  each  run-unit

Finally, since the utility  processes  the journal one  volume
at a time, the user must ensure that the volumes are  supplied
in the correct order.

IDMSRFWD maintains a 'quiesce level1 counter for each program
on the journal tape.  The counter mechanism works as follows:

     A BGIN (beginning-of-job) checkpoint in  any type  of
     UPDATE usage-mode  increments the counter by one.

     An ENDJ  (end-of-job)  or ABRT (abort)  checkpoint  in any
     type of  UPDATE usage-mode decrements the counter  by
     one.

     A counter level of zero indicates that  the  database is
     in a quiesce  (non-UPDATE) state at that  point.

Normally the  user  restores the database to a  designated
quiesce point  (i.e., a point at which no update  run units
were in execution).  However, it is possible  to roll forward
to a non-quiesce point  (level number  greater than zero), but
the user should be aware that one or more programs were at
that point potentially  changing  the  database.  Thus,  there
is no guarantee of 'logical integrity1  in  the  database if
IDMSRFWD is executed to a  non-zero checkpoint.

'Physical integrity,1 the assurance that there are no  broken
chains or related  structural problems in the  database,  is
guaranteed whenever the database is  rolled forward to  any
IDMS-created  journal checkpoint  (quiesce or non-quiesce).
This is true  since all  system buffers are written to the
journal file  and  the database whenever  a run-unit  issues  a
FINISH command.

To restore after  images from the Journal File to the
database, print page images from the  Journal File,  and/or
print a usage report, execute  the program IDMSRFWD.   Figure
VI-17 illustrates  the operation.   Use the following JCL to
run IDMSRFWD:

                    IDMSRFWD
//           EXEC PGM=IDMSRFWD
//STEPLIB    DD   DSN=SYS2.IDMS.LOAD,DSP=SHR
//SYS 009     DD   DSN=IDMS.JRNLFILE,DISP=(OLD,PASS),UNIT=TAPE62,
//                DCB=BLKSIZE=3200
                            VI-37

-------
//DATABASE   DD   DSN=DBFILE1,DISP=OLD
//CODEFILE   DD   DSN=DBFILE2,DISP=OLD
//INDEX      DD   DSN=DBFILE3,DI5P=OLD
//SYSLST     DD   SYSOUT=A
//SYSIPT     DD   *
parameter cards

SYS009 = DD name of Journal File
IDMS.JRNLFILE = data set name  of Journal File
3200 = size of database page +  44
DBFILE1 = data set name of database
DBFILE2 = data set name of CODE file
DBFILE3 = data set name of INDEX file
SYSOUT=A = printed outpout

Roll-back

IDMSRBCK is a diagnostic and recovery program that performs
the following tasks;

     Prints page images from the journal file

     Restores 'before1  image to  the  database

The print and restore functions may be specified
independently of one another.

Program functions are:

     IDMSRBCK reads a PROCESS card  to determine whether
     processing will be TOTAL or SELECT.  TOTAL indicates
     that the effects of all run-units will  be erased  to  a
     specified point in time.   SELECT designates that the
     effects of specified run-units  will be  erased according
     to console input.

     IDMSRBCK reads the journal tape backwards and:

          Optionally restores the database in whole or in
          part, according  to whether PROCESS=TOTAL or
          PROCESS=SELECT is selected

          Optionally prints before  and after  images,
          according to whether or not the  PRINT option is
          specified

     If PROCESS=SELECT is specified, the database  is
     restored on a run-unit by  run-unit basis, depending
     upon specifications entered from the  console.

     The utility always produces a  parameter  listing that
     contains the contents of  the input control cards  (see
     below,  'Report  Format1).

                      VI-38

-------
     The utility always produces a parameter  listing  that
     contains the contents of the input control  cards  (see
     below, 'Report  Format').

     In addition, the utility always produces a job
     checkpoint  listing, including for each run-unit
     encountered:

           Program ID

           Time and  date of activity

           Run-unit  statistics contained within each end-of-
           job checkpoint

IDMSRBCK maints  a 'quiesce level1  counter  for  each program
on the journal tape.  The  counter mechanism works as follows;

     An ENDJ (end-of-job)  or ABRT (abort)  checkpoint in any
     type  of UPDATE mode  increments the counter  by  one.

     A BGIN  (beginning-of-job) checkpoint decrements  the
     counter by  one.

     A counter level of zero indicates that the database is
     in a  quiesce (non-UPDATE) state at that  time.

Normally,  the user  rolls back the database to a designated
quiesce point (i.e., a point at which no update  run-units
were in execution).   It is possible  to force IDMS to roll
back to a non-quiesce point (level number greater than zero),
but the user should  be aware that one  or more  programs  were
at that point potentially changing the database.  Thus,  there
is no guarantee  of  'logical  integrity1  in  the  database if
roll back  is executed to a non-zero level checkpoint.

'Physical  integrity,' is guaranteed whenever the database is
rolled back to any  IDMS-created journal checkpoint.  This
is true since all system buffers  are  written to the  journal
file and database whenever a  run-unit  issues  a  FINISH
command.

To restore before images from the Journal File  to  the
database and/or  print page images from the Journal  File,
execute the program  IDMSRBCK.  Figure VI-18  illustrates the
operation.

Use the following JCL to run  IDMSRBCK:

//                  EXEC PGM=IDMSRBCK
//STEPLIB       DD   DSN=SYS2.IDMS.LOAD,DISP=SHR
                      VI-39

-------
               PARAMETER
                 CARO(S)
                IDMSR8CK
                 USTING
                                                DATABASE
riGURL VI-1U RESTORING BEFORE IMAGES

-------
//SYS009       DD    DSN=IDMS.JRNLFILE,DISP= (OLD, PASS), UNIT=TAPS62,
//                   DCB=BLKSIZE=3200
//DATABASE     DD    DSN=DBFILE1,DISP=OLD
//CODEFILE     DD    DSN=DBFILE2,DISP=OLD
//INDEX        DD    DSN=DBFILE3,DISP=OLD
//SYSLST  DD   SYSOUT=A
//SYSIPT       DD    *
  parameter cards

  IDMS.LOAD = data set name of IDMS  load  library
  SYS 009 = DD name of Journal  File
  IDMS.JRNLFILE = data set name of Journal  File
  TAPE = symbolic device  name  of  Journal  File
  3200 = size of database page •*-  44  bytes
  SYS010 = DD name of database
  DATABASE = data set name of  data  base
  SYSOUT = A = printed output
  DBFILE1 = data set  name of data base
  DBFILE2 = data set  name of CODE fileput
  DBFILE3 = data set  name of INDEX  file
                      VI-40

-------
6.    DATABASE  MAINTENANCE

     (1)   Error Conditions

          In the course of running programs against the database,
     conditions will arise which cause the database to be in error.
     The  conditions will include a bug in a program,  a program  which
     aborted without recovering the database properly, or a  bad
     transmission of data.   In  the  data  base, this could  result  in
     broken set chains,  improperly  stored data  or  non-retrievable data.

     (2)   Diagnosis

          The  presence of any of these errors can cause subsequently
     run  programs to executed incorrectly or  to abort with mysterious
     error codes.   If  a  set  chain  is  broken,  frequently the program
     error will be "no current of set-name".  Garbled data may  lead
     to a "record not  found"  or  a set which appears to be  out  of  sort.
     It  should also be remembered that a broken set chain will not
     cause all accesses to a record to fail,  but  only those  accesses
     which use the set in error. Therefore,  the error may exist  for
     some time before  it  is discovered, making it virtually impossible
     to  pinpoint the cause of the error.  This makes it impracticable
     to  use the journal tape to roll  back the program which  caused
     the  error.   The major  tool  for diagnosis and  correction  is
     IDMSPFIX.

          Data Base Page Dumps - IDMSPFIX

          IDMSPFIX is a program debugging tool  that allows the user
          to print and,  optionally, alter data in a database page and
          write the altered data back to the database.

          Program functions are:

               IDMSPFIX reads a PROCESS  card to determine the input
               source - i.e., cards or console.

               The utility accepts and processes  one  function at a
               time.  A  function is equivalent  to  a group of  one or
               more cards or lines  from  the console.

               The utility prints pages  in decimal, hexadecimal,  or
               both formats,  depending on PRINT parameter
               specification (both  Decimal/Hexadecimal).  In addition,
               pages may be printed singly or by  page number  range.

               IDMSPFIX locates  and prints CALC  records,  using:
                                VI-41

-------
               Area name into which the record has  been
               randomized or low and high page limit  of CALC
               range

               CALC key value

          Optionally, additional pages  following  the CALC'ed page
          are printed.

          IDMSPFIX alters data and writes  it back to the database
          using the PATCH,  VERIFY,  and  REPLACE  parameters.   The
          user is urged to exercise caution when taking advantage
          of these powerful options.

General syntax requirements are listed below.  For specific
parameter information  refer to Database Design & Definition
Manual, Section  11-43.

     first card
     INPUT=  CARDS  ,DMCL=device-media-name
             CONSOLE

             subsequent cards or console entries

                  (printing database pages)

          PRINT = BOTH
                  DECIMAL, Area-name
                  HEXADECIMAL

                  (printing page from calc key)

                  AREA = Area name


                  CALC = Key-value  [,PAGES= page-count]

                  (end of job)

                  END    The process goes to end of  job.

To print pages  from  the database and/or make patches,  execute  the
program IDMSPFIX.   Figure VI- 19 i llustrates the operation.

     Use the following JCL to run IDMSPFIX:

     //             EXEC PGM=IDMSPFIX
     //STEPLIB      DD   DSN=IDMS.LOAD,DISP=SHR
                           VI-42

-------
PARAMETER
  CARDS
IDMSPFIX
                               DATABASE
                               USTING
           FIGURE  VI-19  ALTERING AND PRINTING PAGES

-------
     //DATABASE     DD   DSN=DBFILE1,DISP=OLD
     //CODEFILE     DD   DSN=DBFILE2,DISP=OLD
     //INDEX        DD   DSN=DBFILE3,DISP=OLD
     //SYSLST       DD   SYSOUT=A
     //SYSIPT       DD   *
     parameter  cards

     SYS 2.IDMS. LOAD = data set name of IDMS load library
     DBFILE1 =  data  set name of  database
     DBFILE2 =  data  set name of  CODE file
     DBFILE3 =  data  set name of  INDEX file
     SYSOUT=A = printed output

(3)   Error  Correction

     Operational Precautions

     Error  Correction using the  IDMSPFIX utility should be
     undertaken with the greatest precautions  possible.   Misuse
     of  the tool will lead to a  database disaster.   None  of  the
     normal safety features (journaling)  will  be in effect to
     protect against careless errors.  If  misuse becomes  a
     problem, the  utility  should be removed from the library and
     kept in the DBA's  desk.  A log should be set up to record
     all  changes (much like a log by recording IDMS system changes
     sent by CULLINANE).

     Database Page Fixes

     The  program IDMSPFIX has been previously  described in (2)
     Diagnosis,  Database  Page  Dumps.  In order to alter pages of
     the  database replace the print parameters  of  the IDMSPFIX
     with the following:

     PATCH      = n
     VERIFY    = aaaa  pppp
     REPLACE   = aaaa  pppp
     FINISHED
                           VI-43

-------
7.    SECURITY PROCEDURES

     (1)   On-line Data Base Protection

          In order to protect the data base as a whole from
     unauthorized usage, OPP  must  rely on  facilities within the
     operating system.   The first  operating standard which should be
     employed is to put a disposition of "OLD" on all database files
     in all IDMS application  programs and utilities.  Not only will
     this have the affect of  locking out unauthorized users most of
     the  time but it  will  force all IDMS jobs to be executed serially,
     thus insuring data base  integrity in a non-CAMP environment.
     Additional protection can  be  obtained  by  using  OS file passwords
     on the data base files.

     The  data base files are  the  most valuable component of the OPP
     Monitoring systems and,  as such, should receive maximum physical
     security.   No  unauthorized person should  be  allowed  to remove
     them or destroy  them.  Furthermore, the same degree of protection
     should be given  to the IDMS  dump tapes and journal tapes.

     (2)   Subschema Protection

          For those programs and their users running under  the auspices
     of IDMS,  the subschema provides  maximum security to  prevent
     unauthorized access to data  within the database.  The major
     security consideration is to protect the subschemas themselves
     from unauthorized changes.   Since the  ultimate  protection would
     be deletion of the  subschema compiler from the  IDMS load  library,
     which would prove inconvenient for those authorized users, the
     DBA  should simply make frequent checks on the subschemas using
     the  IDMSRPTS utility.

          Subschema Usage

          The Subschema DDL enables the DBA to define  a subset of the
          'global'  or schema database  definition.  This definitional
          subset provides security and privacy in three ways:

     (i)   The Subschema DDL allows the DBA to establish a 'program
          view,1 consisting of only those database areas,  records, and
          sets that are  relevant to the application.  The program view
          forms the basis for data privacy in the IDMS environment.

     (ii)  The Subschema DDL permits the DBA to 'segment' database
          records;  i.e.,  define portions of records that are to be made
          available to the application excluding data-items not
          pertinent to the application.
                                VI-44

-------
(iii) The  Subschema  DDL  allows the DBA to define privacy locks
     that restrict  specified  area usage  modes and explicit DML
     functions.  For instance, if  an area usage mode is specified
     as having  a  privacy  lock of  'NO1 (see Section 8, 'Subschema
     DDL  Preparation1),  the pertinent database area may not be
     READIED  with  that  usage  mode.  These  locks  permit  the DBA
     to govern  the  manner in  which the  database resource  is to
     be used. For example, the DBA may indicate that the CUSTOMER-
     REGION  cannot  be opened  in EXCLUSIVE  UPDATE mode.   This
     specification  ensures that concurrent run-units may access
     the  CUSTOMER-REGION,  but  only in RETRIEVAL  usage mode.

     A second type  of privacy lock restricts explicit DML
     functions  against  specified  record  and set types.   For
     example, by defining  'PRIVACY LOCK FOR CONNECT IS NO,1 the
     DBA  may prohibit this function  from being executed against
     specific records and sets.   However, the user should  note
     that implied functions are not prohibited.  Thus, the privacy
     lock for CONNECT does not preclude  the use of  the  STORE
     command, with  its  associated implied  CONNECTS.  Refer  to
     Section 8  of  the IDMS Programmer's  Reference Guide  for a
     discussion of  subschema  privacy  locks.

     The  subschema  compile phase  generates control  tables that
 reflect the  program view,  data  segmentation, and privacy lock
 restrictions  described  here.  The  compiler posts this information
 into the  Directory  for  subsequent use by the DMLC and DMLP
 processors.  At execution-time, these tables will provide
 additional security

 (3)  Unauthorized Programs

     Compilation Protection

     Prior to execution of the IDMSDMLC  or DMLP processors, the
     Data Directory Utility (IDMSCLUC)  stores program control
     information  in the Directory file.  These  controls  define
     which programs may invoke  a  given subschema, and, therefore,
     constitute important compile-time  checks.   Two  IDMSCLUC
     action  codes  require that a  program be registered  in the
     Directory  prior to compilation:

     +_ (ADD/DELETE  'any-program1  option)  - The -A code specifies
     that a  program name  must  be  registered  in the Directory for
     a given  subschema  prior  to compilation.

     +G  (ADD/DELETE 'program-name') - The +G code registers the
     program  name  in the  Directory.

     At compile-time, the  DMLC  and DMLP  processors check  for
     proper  program registration. If the  program name  is  not
     registered for the subschema, and the  subschema's 'any-
     program' option is turned  'off,'  compilation is  prohibited.

                            VI-45

-------
          The DBA should note that when the "any-program"  option is
     turned 'on1  (the  default), the DMLC and DMLP processors will
     execute,  regardless of  whether  the  program  name  has been  entered
     via the +G command.  In addition, only  COBOL  and PL/I  programs
     may be registered  in the Directory.  Assembler and FORTRAN
     programs are not subject to IDMSCLUC controls, since these
     programs are not pre-processed by  the DML  compilers.

          Additional Authorization Checks

          Database procedures are special DBA-coded routines that may
          be associated with a particular DML function for a specified
          record type.  These modules intercept  functions between the
          DBMS and the  user program, enabling the DBA to modify data
          on its way into and out of the database, perform search
          algorithms,  gather statistics, and  execute  various special
          functions.   Database  procedures also have several uses in
          the context of data privacy and security, including:

          Encoding/decoding of data-items and records to ensure
          physical data security

          Compression/decompression of data to  save disk space  and
          provide controls against data theft.

          Validation of data-items to establish data  integrity

          Enforcement of data privacy at the  record occurrence  level

     (4)  Protection of Input Files

               Within the OPP Monitoring System, the intermediate
          input files which have been created from the survey forms
          and are sitting on the disk prior to being added to the data
          base represent the least secure aspect of  the system.  It
          is therefore  recommended that  these files be protected with
          OS file passwords.

8.    ARCHIVAL DATA
     The DBUNLOAD program can be used to either partially or  fully
unload the database.  The data is stored into site,  master,  residue,
and code files.  Parameters  to the  program are stored in a parameter
file and allow the removal of all the data or only  that data  which
pertains to a sample taken prior to the specified date.
                                VI-46

-------