University Information Technology Services

Database Administration Team

DB2 Standards - Operations, Procedures, and Recovery

Introduction and Overview

Operations Procedures

• Start DB2
• Stop DB2
• Cancel DB2
• Terminate DB2 Utilities

Backup and Recovery Procedures

A. System Procedures

• Copy Catalog and Directory

• Copy BSDS (Boot Strap Dataset) and Logs
• Remove Entries from SYSCOPY / SYSLGRNG
• Create a System Point of Consistency (POC)
• Restore to a System Point of Consistency (POC)
• Recover DB2 Catalog and Directory
• Catalog / Directory out of Space

B. User Data Procedures

• Create User Data Point of Consistency (POC)

• Restore User Data to Point of Consistency (POC)

Disaster Procedures

A. Introduction and Overview

1. Introduction

This part of Database Administration Standards and Procedures deals with the operation
of DB2, and the recovery procedures necessary to restore both system and user data. The
first part of this section, Introduction and Overview, describes how the Operations and
Recovery Standards chapter is organized. The second part of this section, Overview of
DB2 Operations and Recovery, gives a brief introduction to the subject of operations and
recovery in the DB2 environment.

Sections B through D document the various procedures -- Operations/Restart,

Backup/Recovery, and Disaster -- necessary to effectively run DB2. For each procedure
documented, you will find step-by-step instructions, and some procedures include a
flowchart. Sample JCL for DB2 utilities can be found in the PDS DBT1.DB2.UTILLIB.
All of the Operations and Restart procedures in section B are labeled "Onn", where nn
stands for two digits; in section C system recovery procedures are labeled "Snn", user
data recovery procedures, "Unn". Where appropriate, each procedure will contain
STANDARD, GUIDELINE, and INFORMATION sections to inform the reader of what is
required, what is good practice, and what is useful to know. A list of the procedures
documented in each section can be found in the introduction to that section.

Complete information on DB2 operations and recovery can be found in IBM Database 2
Version 2 Administration Guide, Section 6 Operations and Recovery (SC26-4374).

2. Overview of DB2 Operations and Recovery

DB2, like most online systems, uses logging (applying all changes in data to a log
dataset) to facilitate recovery of data. The basic steps in the recovery method for online
systems are:

• Make backup copies of user datasets.

• Restore the dataset using the backup copy.
• Read the log dataset and apply all changes made to the user dataset since the
backup copy was taken. This is known as forward recovery.
• Backward recovery is the technique by which changes made to data are removed.
The log stores not only changes to the data, but also an image of the data before it
was changed. Thus, backward recovery applies "undo" images to data.

Within DB2 specifically, recovery is implemented through the following system

resources:

• DB2 log datasets

• DB2 bootstrap data set (BSDS)
• DB2 Utilities
• Backup copies of user data
• Backup copies of system resources

-- DB2 libraries
-- DB2 Catalog and Directory
First we will present an overview of DB2 recovery, and then discuss the system
components DB2 utilizes in the recovery process.

DB2 Recovery

Units of Recovery

A unit of recovery is the work, done by DB2 for an application, that affects the state of
DB2 data from one point of consistency to another. A point of consistency, (also knows
as sync point or commit point) is a point in time when all recoverable data an application
program accesses is consistent with other data accessed by that same program. For
example, a bank transaction might transfer funds from account A to account B. The
program first subtracts an amount from account A and adds that amount to account B.
After the subtraction, the two accounts are inconsistent not until the amount subtracted
from account A is added to account B are they consistent again. Once both steps are
complete, the program has created a point of consistency. Normal termination of a
program, the SQL COMMIT statement, and CICS program requests all cause a point of
consistency to be established.

Rolling back work

If failure occurs within a unit of recovery, DB2 removes any changes to data, returning
the data to its state at the start of the unit of recovery. The work is undone. The SQL
statement ROLLBACK will also cause this same sequence of events (the "undo"
sequence) to occur.

Undo and Redo Records

When a change is made to a database, DB2 logs an undo/redo record that describes the
change. Redo information is required if work is committed and later must be recovered.
Undo information is used to back out work that has not been committed. If changes to the
database must be rolled back, the undo/redo record is used to remove the change. At the
same time, a new "redo/undo" record is created that contains information, called
compensation information, that reverses the change. For example, if a value of 3 is
changed to 5, redo compensation information changes that value back to 3.

If work must be recovered, DB2 scans the log forward and applies the redo/undo log
records, and the redo portions of compensation records, without keeping track of whether
the unit of recovery was committed or rolled back. If the work is to be rolled back, redo
records restore the data.

Database Recovery

The above processes describe recovery initiated by incorrect results of either interactive
or programmatic SQL, in which case the DB2 log completely recovers the data back to a
point of consistency. The programmer controls this recovery through the use of SQL
COMMIT and ROLLBACK statements. But assume, for a moment, that a SQL
COMMIT statement successfully ended a unit of recovery, writing the updated data back
to the database. After this COMMIT has been issued it is discovered that an application
program, functioning incorrectly, inserted bad data into a table. Or assume that the disk
pack which physically contains a DB2 table incurs an I/O error, and data in the table is
lost. In either of these instances, the only recourse is to restore the table from a backup
copy, and, if necessary, apply the log to update the restored table with changes made after
the backup was created. In the first instance, the table would be restored to a previous
point of consistency, i.e., a time when the data in the table was correct, and updates
captured by the log would not be applied to the table. In the second instance the table
would be restored to the current time, using both the backup and the log. Thus, a database
can be restored to a previous point in time (restore to a backup copy), or to the current
time (i.e., the last log RBA for that database). The point to remember is that if an entire
database, or table, must be recovered, DB2 must have a backup copy of each table space
that is to be restored.

Flow of Recovery in DB2

Changes made to DB2 tables (system and application) are recorded in the active log
datasets. When an active log is filled, DB2 initiates an archival process (called
"offloading") which copies the active log to an archive log. During this archival
procedure DB2 creates an entry in the Bootstrap Dataset (BSDS) that describes the
archive log dataset, including its dataset name and the range of log records it contains.
The DB2 Catalog table SYSCOPY contains records that describe image copy (i.e.,
backup copy) datasets for each table space (image copies are created by the DB2 COPY
utility). The DB2 Directory table SYSLGRNG contains records that tell what log records
apply to each table space for each table space, the starting and ending RBA of a log
record is recorded.

DB2 tables and databases are restored by running the RECOVER utility against a table
space, initiating the recovery process. This flow runs as follows: DB2 first reads the
Catalog table SYSCOPY to find the most recent image copy for a table space, and uses
that copy dataset as the base from which to begin the recovery; next reading the Directory
table SYSLGRNG, DB2 determines which units of recovery apply to a particular table
space; finally, reading first the BSDS to determine which log datasets (both active and
archive) contain applicable units of recovery, it applies changes recorded by the log to the
table space being recovered.

This overview, though simplified, should give the reader a basic understanding of how
DB2 implements recovery. Those wishing more detailed knowledge of this process may
wish to consult IBM Database 2 Version 2 Administration Guide, Vol. II (SC26-4374).
We will now discuss these components of the recovery process in greater detail.

DB2 Log Datasets

The DB2 log contains the information necessary to recover the DB2 subsystem and user
databases, and to recover from incorrect results of program execution. Each byte in the
log is addressable by its offset from the beginning of the log this is known as the log
RBA. A record"s log RBA uniquely identifies its position within the log, and allows for
retrieval of individual log records. DB2 writes three kinds of records to the log -- "Unit of
Recovery", "Checkpoint", and "Database Page Set" -- and each record on the log is
identifiable, and thus accessible, by its log RBA. Changes made to DB2 data (both
system and user) are captured by "Unit of Recovery" log records, and are used to recover
data. The other two types of log records, as well as complete information on DB2
recovery, are documented in IBM Database 2 Version 2 Administration Guide, Vol. II
(SC26-4374).

How logging is implemented

Log records, when first created, are formatted and placed into log buffers. When these
buffers become full, or when an SQL COMMIT is issued, log records in the log buffer
are written to the active log, which is a VSAM LDS. Active log datasets are written
(offloaded) to an archive log, a sequential dataset, when an active log becomes full. DB2
is capable of employing dual-logging, which means two copies of each active log are
always being written. During recovery DB2 uses both the active and archive logs to
restore a table space.

DB2 Bootstrap Dataset (BSDS)

The BSDS is a VSAM KSDS that contains information about all of the DB2 log datasets
and the records they include. The names of all current active log datasets are placed in the
BSDS when DB2 is first installed. When an archive log is dynamically allocated during
the offload process, its dataset name is placed in the BSDS, and a copy of the BSDS is
placed on the same volume as the archive log (this provides a backup should the active
BSDS be destroyed). Every time an active log is offloaded to an archive log, a record
describing the new archive log is added to the BSDS.

DB2 Utilities

DB2 utilities are standard MVS batch jobs that perform functions on DB2 objects, both
system and application. See Section VII, DB2 Utilities, for a discussion of this topic.

Backup Copies of System Resources

To be able to recover the DB2 subsystem, there must be backup copies of the system
resources used by DB2. Each of these resources is briefly described.

DB2 Libraries: The DB2 execution and installation libraries are backed up in the event
they need to be restored.
DB2 Catalog and Directory: Two of the most important components of DB2 are the
Catalog and Directory. The Catalog contains tables that describe every object defined to
the DB2. Whenever you CREATE, ALTER, or DROP any object, DB2 inserts, updates,
or deletes rows from tables within the Catalog that both describe an object and tell how it
is related to other objects. For example, when the COPY utility is run to back up a table
space, a row describing the image copy dataset is inserted into the Catalog table
SYSCOPY. This information is then used by DB2 for recovering that table space.

The Directory contains four tables that are used to start and run DB2. The table
SYSLGRNG, for example, contains open/close log RBAs for each table space. It speeds
up recovery by limiting the log information that must be scanned to apply changes to a
table space being recovered. The Catalog and Directory should be considered as one unit
for purposes of backup and recovery. For more information on DB2 system resources,
please consult IBM Database 2 Version 2 Administration Guide, Vol. I (SC26-4374).

Backup Copies of User Data

As stated above, image copies are made of table spaces, not tables. In order to implement
a successful recovery strategy for user data, user table spaces must be backed up on a
regular basis using the COPY utility. The type of data in each table will determine how
often the table space should be backed up. For example, is the data updated?, and if so,
how often? How critical is the data to the mission of the organization? If the data is for
inquiry only, can the table be simply loaded again from the original source of the data
(e.g., VSAM or sequential file)? Do quiesce points need to be taken between creation of
image copies? These questions need to be addressed before any user table is put into
production.

B. Operations Procedures

The procedures in this section document how to run DB2, start DB2, stop DB2, cancel
DB2, and so on. These procedures will be performed by staff of Technical Services and
Operations. There is a different subsystem recognition for the test and production
systems: + for test, and - for production. Some of the commands given in this section
require the subsystem recognition character when working at the MVS console. These
procedures use the production system as an example, but when issuing commands against
the test DB2 system use the plus sign (+) as the DB2 subsystem character. Some of the
basic DB2 commands, such as START and STOP, have been given CA-OPERA aliases
(e.g., the command to stop the production DB2 subsystem can be entered as "P
DB2PROD", rather than "-STOP DB2"). This makes it less likely that the production and
test subsystems will be confused. This section gives the CA-OPERA alias for a DB2
command whenever they can be substituted.

• O01 - Start DB2

• O02 - Stop DB2
• O03 - Cancel DB2
• O04 - Terminate DB2 Utilities
Procedure O01 -- Start DB2

STANDARD

This procedure will be executed only by staff of Operations and Technical Services.

INFORMATION

The DB2 system consists of three started tasks: DBT1MSTR, DBT1DBM1, and
DBT1IRLM (for production, these started tasks are DBP1MSTR, DBP1DBM1, and
DBP1IRLM). DB2 should be left up between IPLs. Tape mounts will be requested
intermittently for logging purposes. This command can be entered only from an MVS
console with the START command capability.

Procedure O01

1. At the Master Console type:

S DB2PROD (for test, type S DB2TEST)

The response to the start DB2 command will be the message

DSNY001I SUBSYSTEM STARTING

2. Check for message DSN9022I, which signals that DB2 has started successfully. If you
do not receive this message, then

3. Notify Technical Services

Procedure O02 -- Stop DB2

STANDARD

This procedure will be executed only by staff of Operations and Technical Services.

INFORMATION

This command must be entered from the MVS console. The default for stopping DB2 is
MODE(QUIESCE), which means that all currently active programs are allowed to
complete.

Procedure O02

1. At the Master Console type

P DB2PROD (for test, type P DB2TEST)

The response to the stop command is the message

DSNY002I SUBSYSTEM STOPPING

2. Message DSN9022I tells you that the DB2 stop command was successful.

Message DSN3100I tells you that DB2 has been successfully stopped and is ready for the
next start command.

3. If message DSN9022I is not displayed, check for outstanding requests on the MVS
console.

4. If message DSNJ008E is outstanding, DB2 cannot terminate until it has been

answered. It was issued because at least one active log data set was ready for archiving,
and the installation parameter DSNZPARM specifies WTOR=YES.

5. Reply "N" to DSNJ008E to allow DB2 termination. But you may want to reply "Y" to
DSNJ008E if you previously received the following message

DSNJ110E LAST COPYn ACTIVE LOG DATA SET IS nn PERCENT FULL

which tells you that the last active log dataset is seventy-five percent or more full. If you
reply "Y" to DSNJ008E, DB2 will wait for offload completion before terminating.

6. If you arrive here, DB2 could not quiesce normally, and you should use

Procedure O03 - Cancel DB2.

Procedure O03 -- Cancel DB2

STANDARD

This procedure will be executed only by staff of Operations and Technical Services.

INFORMATION

Canceling DB2 will probably cause in-doubt situations.

Procedure O03

1. If DB2 does not terminate after a "STOP DB2" command (e.g., P DB2PROD), it may
be because there are still threads active. Use the commands

-DISPLAY THREAD(*)

-DISPLAY UTILITY(*)
from the MVS console or from the DB2 commands panel (#7 in DB2I) to determine
whether there are still active threads. Remember that the default when stopping DB2 is
"MODE(QUIESCE)", which allows all active programs to complete before DB2 stops.

To stop DB2 immediately (i.e., cancel DB2), enter this command:

-STOP DB2 MODE (FORCE)

from the MVS console or DB2 commands panel. DB2 should respond with the message:

DSNY002I SUBSYSTEM STOPPING

Active DB2I users receive the following message:

THE DB2 OPERATOR IS STOPPING THE SUBSYSTEM. PLEASE END YOUR DSN
SESSION.

Once DB2 has been successfully canceled, you will receive the following message:

DSN3100I DSN3EC00 - SUBSYSTEM ssnm READY FOR -START COMMAND

2. Check for message DSN3100I.

3. If you do not receive message DSN3100I, that is, if you have been unable to cancel
DB2, contact Technical Services. (Note: Technical Services can authorize the operator to
stop the IRLMPROC, which will immediately bring DB2 down. This decision will be left
up to Technical Services.)

Procedure O04 -- Terminate Utilities

STANDARD

This procedure will be executed only by staff of DBA, Operations, and Technical
Services.

INFORMATION

The DB2 "-TERM UTILITY" command terminates execution of a utility job step and
releases all resources associated with that step. This command can be entered from an
MVS console, from the DSN command processor, the DB2I commands panel, or from a
CICS terminal.

Procedure O04

To see which DB2 utilities are currently executing, type:

-DISPLAY UTILITY(*)

For each utility that is stopped or running, information will be displayed. For example,

UTILID = CG2040CG
PROCESSING UTILITY STATEMENT 2
UTILITY = RUNSTATS
PHASE = RUNSTATS COUNT = 0
STATUS = ACTIVE

Notice the UTILID specification.

To terminate a particular utility, enter the following command:

-TERM UTILITY(utility-id)

Where (utility-id) is the utility identifier, or UID parameter, used when creating the utility
job step.

Example: -TERM UTILITY (DB2050TS)

Example: -TERM UTILITY (DB2050*)

(*) denotes all utility job steps. This parameter will terminate every utility job step known
to DB2.

Example: -TERM UTILITY (*)

C. Backup and Recovery Procedures

The procedures in this section document those procedures necessary to insure the
integrity of DB2. The procedures will be carried out by staff of DBA, Technical Services,
and Production Services. The backup and recovery procedures documented in this section
are divided between system data and user data. System data procedures will be designed
and implemented by DBA and Technical Services. Note, however, that user data
recovery procedures will be designed by both DBA and developers. User data procedures
will be regular production jobstreams, and will follow DBA naming standards. A
description of these jobstreams, as well as their respective naming standards, are given in
Section VII, DB2 Utilities.

1. System Data Procedures

• S01 - Copy Catalog and Directory

• S02 - Copy BSDS and Logs
• S03 - Remove Entries from SYSCOPY/SYSLGRNG
• S04 - Create a System Point of Consistency (POC)
• S05 - Restore to a System Point of Consistency (POC)
 • S06 - Recover DB2 Catalog and Directory
• S07 - Catalog/Directory Out of Space

The above procedures utilize the jobstreams described below. These jobstreams are for
DB2 system datasets only, and as such are run under CA-7, or are submitted by a person
with appropriate authority -- a staff member of DBA or Technical Services. For both test
and production the jobstreams begin with 'DB', but production jobstreams end with the
suffix 'DB', whereas test jobstreams end with suffix 'DT'. For example, the jobstream to
back up the DB2 Catalog and Directory is DB2510DB for production, and DB2510DT
for test.

• DB2510DB -- Back up Catalog/Directory (runs nightly)

• DB2520DB -- Back up BSDS and Logs (runs weekly)
• DB2525DB -- Print Log Map/BSDS (utility DSNJU004)
• DB2530DB -- Remove Old Entries from SYSCOPY/SYSLGRNG
• DB2535DB -- List DB2 ICF Alias
• DB2540DB -- Recover Catalog/Directory
• DB2545DB -- Recover Catalog/Directory to a POC
• DB2550DB -- STOSPACE Utility
• DB2555DB -- Change Log Inventory (DSNJU003)
• DB2570DB -- Recover BSDS and Logs
• DB2575DB -- Recover BSDS and Logs to POC
• DB2580DB -- Delete and Define BSDS and Logs Datasets
• DB2590DB -- Delete and Define Catalog/Directory Datasets
• DB2595DB -- Initialize SYSUTIL

Procedure S01 - Copy Catalog and Directory

STANDARD

• The naming standard for system image copy datasets will be as follows:

yyyyyyy.dddddddd.tttttttt .Annn .OFS .GxxxxVxx, where

yyyyyyy = high-level qualifier (for Test, DBT1 for Prod, DBP1)

dddddddd = eight-character database name
tttttttt = eight-character table space name
Annn (optional) = partition number if copy is by partition
OFS (optional) = designates copy stored offsite
GxxxxVxx = generation dataset group number

Example DBP1.DSNDB06.SYSCOPY.G0205V00

• The naming standard for this production jobstream will be DB2510DB

• The Catalog and Directory will be backed up nightly.
GUIDELINE

• The Catalog and Directory should be copied at the same time to insure
consistency between them. This backup jobstream should be run at a point of very
low system activity.

Procedure S01

1. Execute jobstream to back up Catalog and Directory

2. Was return code "00"? If not
3. Determine the cause of the problem, resolve the problem, and resubmit the job.

Procedure S02 -- Copy BSDS and Logs

STANDARD

• This procedure must be run while DB2 is down.

• The naming standard for this production jobstream will be DB2520DB

INFORMATION

• The BSDS, together with active logs, the Catalog, and the Directory, are the most
important sources of information DB2 needs to operate.

Procedure S02

1. Run the DSNJU004 utility to print the contents of the BSDS.

2. From looking at the report produced in step 1, determine whether there are any
stopped or truncated active logs.
3. If there are stopped or truncated active logs, new data sets will have to be defined
for them. The new active log datasets you define will also have to be recorded in
the BSDS, which can be done using the utility DSNJU003 (Change Log
Inventory).
4. Copy the bootstrap datasets using IDCAMS REPRO. Check to make sure the
return code from this job step is "00".
5. If the return code from step 4 of this procedure is not "00", determine the cause of
the problem, resolve the problem, and resubmit the job.
6. Copy all active log datasets using IDCAMS EXPORT. Make sure the return code
from this job step is "00".
7. If the return code from step 4 of this procedure is not "00", determine the cause of
the problem, resolve the problem, and resubmit the job.

Procedure S03 -- Removing Entries From SYSCOPY/SYSLGRNG

GUIDELINE
The decision on what entries to remove from SYSCOPY and SYSLGRNG, and when to
remove them, is governed by the recovery strategy. Recovery information should be kept
for as many levels (i.e., generations) of data as are required by the recovery strategy. Four
types of backups should be kept:

1. Image copies of table spaces for recovery to the current point in time.
2. Point of consistency backups for user data.
3. Point of consistency backups for the entire DB2 subsystem.
4. Volume dumps of all DB2 volumes for disaster recovery.

Recommended backup intervals are:

• Daily, weekly, or monthly (depending on the application) for user table spaces
and points of consistency.
• Weekly for DB2 subsystem point of consistency backups.
• Weekly for volume dumps.

Based on these recommendations, dates when point of consistency backups were taken
(for both system and user data) can be determined, and outdated entries containing
information on these backups can be removed from SYSCOPY and SYSLGRNG. At the
same time that this is done, information about archive log datasets no longer used can be
removed from the BSDS.

INFORMATION

DB2 does not interact with TMS, or other external management systems, and thus does
not know if a particular backup tape has been scratched, or a disk dataset deleted.
Therefore, it is important to keep DB2 (i.e., the system tables SYSCOPY and
SYSLGRNG) in sync with whatever external storage management system is used for
backing up datasets. Datasets that have been removed from the appropriate DB2 table
should be physically deleted, and vice-versa.

Procedure S03

1. Determine the date(s) for which entries from SYSCOPY and SYSLGRNG should
be removed. If datasets are going to be removed by POC, then simply use the date
on which the POC was created. If the dates are not known, the Catalog table
SYSCOPY can be queried to obtain them. There should also be copies of the
BSDS and active logs for that date.
2. If the datasets for the image copies recorded in SYSCOPY are to be deleted, their
dataset names can be obtained from SYSCOPY. Note that defining a retention
period for a dataset on tape will automatically cause the dataset to be deleted at
the end of the retention period.
3. Use IDCAMS or TSO to physically delete and uncatalog the datasets.
4. Execute the MODIFY utility to delete entries from SYSCOPY and SYSLGRNG.
Remember that MODIFY only deletes records older than the date you specify.
 5. If you are discarding a POC created by procedure S04, delete the bootstrap dataset
and active log dataset backups created when you executed that procedure.
6. Decide whether to remove information from the BSDS. Remember that entries in
the BSDS are added in wrap-around mode. When the BSDS fills up, the oldest
entry is used for the next record. Before removing information from the BSDS,
run utility DSNUJ004 and examine its output to obtain the dataset names of those
archive log datasets that are older than the date chosen in step 1 (The date from
the BSDS will be in YY.DDD format.) Avoid removing information from the
BSDS that might still be needed for a recovery.
7. Run utility DSNJU003 (Change Log Inventory), jobstream DB2555DB, to
remove the names of archive log datasets from the BSDS. Note that DB2 must be
down to run this utility. DSNJU003 requires that you specify a dataset name, not
date. These names were obtained in step 9.
8. Physically delete and uncatalog the archive log datasets whose names were
removed from the BSDS.

Procedure S04 -- Create a System Point of Consistency (POC)

STANDARD

• A system POC should be created once a week.

• System POC creation dates will be maintained for the purpose of recovering to a
POC.

INFORMATION

• By creating a point of consistency (POC) for the DB2 subsystem, you are able to
recover the entire system to a point in time where all the data (both system and
user) is consistent. In carrying out this procedure, there are several points to
remember:
• There is no consistency with resources outside DB2 (e.g., TMS). It is up to the
installation to synchronize these resources with DB2.
• This procedure may not be carried out while DB2 is running.
• If there is a recovery to a POC, any changes made to data after the POC was
created will be lost.
• A POC is established by taking image copies of all table spaces (both system and
user data), and by backing up the BSDS and active logs. SYSADM authority is
required to execute this procedure.

Procedure S04

1. Use procedure O02 to stop DB2.

2. Restrict access to the DB2 subsystem to those USERIDs defined as SYSADM or

SYSOPR by entering the following command :
F DB2PROD (for test, type F DB2TEST)

3. Make sure the system is in a consistent state before creating the POC. Check the status
of:

• Databases and table spaces

• Utilities
• Threads

To check the status of databases and table spaces, enter the following command at the
MVS Console or at panel 7 ("Commands") of DB2I

-DISPLAY DATABASE(*) SPACENAM(*) LIMIT(*) RESTRICT

You should receive this message for all table spaces

DSNT365I - ******** NO DATABASES FOUND

To determine whether there are utilities in the system, enter

-DISPLAY UTILITY(*)

You should receive this message

DSNU112I - NO AUTHORIZED UTILITY FOUND FOR UTILITY = *

To find any active threads, enter

-DISPLAY THREAD(*)

The message DSNV402I - ACTIVE THREAD should list only your thread

4. Create image copies for all user table spaces. For large tables, it may be more efficient
to take an incremental image copy, and later, after this entire procedure has been
completed, run MERGECOPY to combine the copies.

5. Use procedure 'S01' to create full image copies for all system table spaces (Catalog and
Directory).

6. Use procedure 'O02' to stop DB2. Step 7 cannot be executed until DB2 has been
stopped.

7. Use procedure 'S02' to copy the BSDS and active logs.

8. Execute jobstream DB2535DB to get a listing of the ICF catalog. This listing will be
used in procedure 'S05', step 7 to determine which table and index spaces were created
after this POC.

9. Jobstream DB2545DB has been set up to recover the Catalog/Directory to a POC. Edit
this jobtream to put in the appropriate backup dataset names from which recovery will
take place. Using procedure 'U04' as an example, prepare a RECOVER utility job for all
user table spaces and indexes. In preparing the recover job for the Catalog, Directory, and
user table spaces, remember that you will be restoring not to the current point in time, but
rather to a POC. Therefore, when you use procedure 'S05' to restore the DB2 system to a
POC, you will be restoring to the backup copies created in steps 4 and 5 of this
procedure. Note: though step 9 could be considered optional, it is best to prepare these
recover jobs now when you have the time to do so, instead of when the system must be
recovered quickly.

10. Make copies of DB2 libraries. This will avoid inconsistencies should DB2 be restored
to the POC. (Note: this may be accomplished by the standard volume backups executed
by Production Services.)

11. DB2 is now available. Use procedure 'S01' to start DB2.

12. Delete information for an older POC. These steps include:

• Deleting copies of the BSDS.

• Deleting copies of the active logs.
• Deleting copies of table spaces if they are not required for recovery procedures
other than POC.

Procedure S05 -- Restore to a System Point of Consistency

STANDARD

• The DBA will determine whether it is necessary to run this procedure.

INFORMATION

Recovering to a system point of consistency (POC) brings the DB2 subsystem back to
consistent point for all DB2. This point in time is created by procedure 'S04 - Create a
System Point of Consistency'. There will be no inconsistencies left in DB2 (Directory,
Catalog, and user data) after this procedure has been executed. Points to remember when
executing this procedure are:

• Data that was created or modified after the date the POC was created will be lost.
• All DDL (Data Definition Language) executed after the creation of the POC will
have to be rerun.
• All BINDs run after the creation of the POC will have to be rerun.
 • A recovery to a prior POC is performed by restoring backup copies of the BSDS
and active logs, and by restoring all system and user table spaces.
• SYSADM authority is required to perform this procedure.

Procedure S05

1. Determine whether DB2 is running.

2. If DB2 is running, use procedure "O02" to shut down DB2.

3. If the BSDS is not available, restore it from the latest archive log.

4. Print the BSDS using DB2 utility DSNJU004, and compare the listing with the one
created in procedure "S04", step 7. This will give you information about archive log
datasets created after the POC you are restoring to, and will be used in step 9 of this
procedure to delete these archive log datasets.

This listing will also tell you if there have been changes in the active log since the POC.
Such changes would have been recorded in the current BSDS, but not in the one that will
be restored, and thus should be reversed. For example, the active log datasets could have
been extended since the POC was taken.

5. Restore the BSDS from the backup created in procedure "S04", step 7, using jobstream
DB2575DB. If the BSDS was defined with the IDCAMS parameter REUSE, the backup
can be copied directly into the BSDS. If not, the BSDS will have to be deleted and
defined using IDCAMS. Check the return code on the REPRO step to make sure it is
"00".

6. Use jobstream DB2575DB to restore the active logs using the backups created in
procedure "S04", step 7. Check the return code on this job step to make sure it is "00".

7. Run jobstream DB2535DB to obtain a listing of the ICF catalog. Compare this listing
with the one created in procedure "S04", step 8, to determine the names of all table and
index spaces created after the POC you are restoring to.

Image copies and archive log datasets written to DASD will also show up on the listing
Remember that the off-loading process creates two datasets, one for the archive log, and
one for the BSDS. The naming convention for these datasets is:

xxxx.ARCHLOG1.Axxxxxxx

xxxx.ARCHLOG1.Bxxxxxxx

8. Using IDCAMS DELETE, delete all table and index spaces created after the POC.
9. If step 4 revealed archive log datasets created after the POC, delete them using TSO. If
step 7 revealed image copies created after the POC, delete them using TSO.

10. Restore program libraries. This step is optional and should be executed only if
changes have been made to the DB2 libraries.

11. Execute jobstream DB2545DB to recover the DB2 Catalog and Directory to the
appropriate point in time. In procedure "S04", step 9 this jobstream was edited to reflect
the correct backup datasets to be used.

12. Recover all user table spaces and indexes, using procedure "U04" as an example. The
utility job to execute this restore was created in procedure "S04", step 9.

Procedure S06 -- Recover DB2 Catalog and Directory

STANDARD

• The DBA will determine whether it is necessary to run this procedure.

• The naming standard for this production jobstream will be DB2540DB

GUIDELINE

The utility job to recover the Catalog and Directory should be prepared and ready to run
before it is required. There won't be time to prepare it when it is needed. It is assumed
that if you are running this procedure, there has either been damage to the Catalog or
Directory, or DB2 is being recovered to a point of consistency.

INFORMATION

• When recovering the Catalog and Directory, remember that these two tables are
the most important in the DB2 system. Always, for purposes of recovery, consider
the Catalog and Directory as one unit -- recover them both to the same point in
time. And always recover all Catalog and Directory table spaces and indexes.
• The first choice for recovery should be to the current time. If that is not possible,
the second choice would be to recover to a point of consistency (procedure S05
describes how to recover the DB2 subsystem to a POC). The last choice would be
to restore to a point in time, which is the most difficult and risky of the three
approaches.
• The Directory table spaces, tables, and indexes are not recorded in the Catalog.
Refer to IBM Database 2 Version II Administration Guide Volume II, Section 6
Operations and Recovery for a list of the objects in the Directory to be recovered.
• SYSADM authority is required to execute this procedure.

Procedure S06
1. Determine, through VTOC listings, whether the Catalog and Directory are available on
DASD.

2. If any Catalog or Directory table is not available, delete and define all DB2 Catalog
and Directory clusters. Note that it is possible to bring DB2 up even if the Catalog and
Directory have been lost. At startup, DB2 needs

• BSDS
• Active log dataset
• Load library (prefix.DSNLOAD)
• Early code library (prefix.DSNLINK)

3. Start DB2 in maintenance mode with the following command

F DB2PROD (for test, type F DB2TEST)

This limits DB2 access to SYSADM and SYSOPR.

4. Execute jobstream DB2540DB to recover table space SYSUTIL. This is the first step
in this jobstream. If this step fails, the remaining steps (which recover the rest of the
Catalog/Directory) should not execute.

5. Determine whether SYSUTIL was successfully recovered.

6. If SYSUTIL was not successfully recovered, initialize it by executing jobstream

DB2595DB. This may cause the status in SYSUTIL to not match that of the DB2
subsystem. Steps 11 and 12 of this procedure will deal with these possible
inconsistencies.

7. Execute the remaining steps in jobstream DB2540DB to recover all table spaces and
indexes for the Catalog and Directory. The order of recovery of these objects is essential.
Therefore, execute the steps of this jobstream in sequence.

If you are going to recover to a POC, refer to procedure 'S05' for information on the
entire POC recovery process. The procedure documented here ('S06 - Recover DB2
Catalog and Directory') is used by procedure S05 to recover the Catalog and Directory
during a complete POC recovery.

8. Determine whether the recovery was successful by querying the Catalog with SQL
statements. If a resource (a Catalog or Directory table or index space) is unavailable, it
means you probably forgot to recover it. Recover the appropriate object using the
RECOVER utility.

9. If a recovery to the current time was unsuccessful, use procedure 'S05' to recover to a
point of consistency.
10. If SYSUTIL was initialized, check SYSLOG for message DSNR007I to determine
whether there were any active utilities at the time of failure.

11. If there were utilities active, you may have to:

• rerun utilities
• keep datasets that were being used by utilities to be able to recover the data
• delete datasets that are no longer needed
• remove restrictions

12. Restrictions will be shown when the following command is issued against the table
space being used by the utility

-DIS DATABASE(database-name) SPACENAM(tablespace-name) LOCKS

Restrictions (UTRO,UTRW,UTUT) may be removed by entering the following

command. This command also removes information from SYSUTIL.

-START DATABASE(database-name) SPACENAM(tablespace-name) ACCESS(FORCE)

13. Use procedure 'O02' to stop DB2.

14. Use procedure 'O01' to start DB2. This will establish normal operation mode.

Procedure S07 -- Catalog/Directory Out of Space

STANDARD

Only DBA staff will execute this procedure.

GUIDELINE

Use this procedure to resize a Catalog or Directory dataset, whether required because of
planned maintenance or because a dataset out of space caused a DB2 failure.

INFORMATION

Catalog and Directory table spaces and index spaces are VSAM linear datasets, and
therefore are subject to VSAM space limitations. From time to time, or in the case of
emergency, it will be necessary to reallocate space for these datasets. The proper
procedure for doing this, as outlined in the steps below, is:

(a) take DB2 down,

(b) delete and redefine the appropriate dataset(s),
(c) bring DB2 up in maintenance mode,
(d) recover the Catalog and Directory, and
(e) restart DB2 in normal mode.

Procedure S07

1. Determine which Catalog or Directory dataset needs to be resized. Note: the ouput
from jobstream DB2510DB (Backup Catalog/Directory) contains a LISTCAT of all
Catalog and Directory datasets.

2. Use Procedure O02 to stop DB2.

3. Restrict access to the DB2 subsystem to those USERIDs defined as SYSADM or

SYSOPR by entering the following command

F DB2PROD (for test, type F DB2TEST)

4. Execute jobstream DB2580DB to delete and redefine the appropriate Catalog or

Directory dataset(s). Make sure the return code from this jobstream is '00'.

5. Execute jobstream DB2540DB to recover the Catalog and Directory.

6. Query the Catalog to insure that the recovery executed in step 5 was successful. If the
recovery was successful, determine the problem, correct it, and rerun the recovery
jobstream (DB2540DB).

7. Use Procedure O02 to stop DB2.

8. Use Procedure O01 to start DB2 in normal mode.

2. User Data Procedures

The basic user data procedures are those described in the section on DB2 utilities. These
procedures invoke DB2 utilities to LOAD, CHECK, COPY, and RECOVER data, as well
as perform other tasks. For information on these utilities, see "Section VII DB2 Utilities".

U01 - Create User Data Point of Consistency (POC)

U02 - Restore User Data to Point of Consistency (POC)

Procedure U01 -- Create User Data Point of Consistency (POC)

STANDARD

• The naming standard for a production create user POC jobstream will be
aa2200tt, where:

aa = two-character database prefix assigned by DBA

tt = two-character table space identifier assigned by DBA

• A point of consistency will be established for only those tables that (a) are
updated and (b) are referentially related to other tables.
• A POC jobstream should always create a full image copy.

GUIDELINE

• A data POC jobstream should be set up in production for all table space sets that
permit insertions, updates, and deletes. This jobstream should follow the
execution of the CHECK utility against all tables in the table space set.
• An alternative method to the one given below is to use the QUIESCE utility to
quiesce all table spaces in a table space set.

INFORMATION

• Tables that are not updated cannot have inconsistencies introduced into them
through standard SQL INSERT, UPDATE, and DELETE statements. Tables that
are not referentially related to other tables can have their integrity insured by a
simple COPY or QUIESCE, and do not face the problem of data anomalies across
tables. Therefore, only tables that are updated and that belong to a table space set
need have a POC created for them.

Procedure "U02" is used to restore user data to a POC.

Procedure U01

1 Enter the following DB2 Command for all table spaces in the table space set

-STOP DATABASE(database-name) SPACENAM(tablespace1,tablespace2,...)

2 Then start the table spaces in the table space set for utility access

-START DATABASE(database-name)

SPACENAM(tablespace1,tablespace2,...) ACCESS(UT)

3 Execute the COPY jobstream that will create a full image copy of each table space in
the table space set

4 If problems were encountered during the COPY jobstream, determine the problem,
correct it, and resubmit the job.

5 Once the COPY jobstream has completed successfully, enter the following commands

-START DATABASE(database-name)
SPACENAM(tablespace1,tablespace2,...) ACCESS(RW)

Procedure U02 -- Restore User Data to Point of Consistency (POC)

STANDARD

~The naming standard for a production restore to user POC jobstream will be aa2210tt

Where

aa = two-character database prefix assigned by DBA tt = two-character table space

identifier assigned by DBA GUIDELINE

A jobstream to restore user data to a POC should be set up in production for all table
space sets that permit insertions, updates, and deletes. This jobstream should be used only
when it is necessary to recover a table space set to a previous point in time. Always
recover the indexes for all tables in the table space set after the tables have been
recovered. An alternative method to the one given below POC is to recover all table
spaces in a table space set to a RBA created by the QUIESCE utility. INFORMATION

Tables that are recovered to a POC will lose all insertions, updates, and deletes executed
after the creation of that POC. Procedure 'U01' is used to create a POC for user data.

Procedure U02

1 Enter the following DB2 Command for all table spaces in the table space set

-STOP DATABASE(database-name) SPACENAM(tablespace1,tablespace2,...)

2 Then start the table spaces in the table space set for utility access

-START DATABASE(database-name)

SPACENAM(tablespace1,tablespace2,...) ACCESS(UT)

3 Execute the RECOVER jobstream that will each table space in the table space set to the
most recent full image copy created by procedure U07.

4 If problems were encountered during the restore jobstream, determine the problem,
correct it, and resubmit the job.

5 Once the restore to POC jobstream has completed successfully, enter the following
commands

-START DATABASE(database-name)
SPACENAM(tablespace1,tablespace2,...) ACCESS(RW)

D. Disaster Procedures

STANDARD

Computing Systems will be responsible for designing and implementing disaster

procedures for DB2.

Last revised 05/08/2003 20:57:24

http://www.indiana.edu/~dbateam/databases/db2/standards/recover.html
Comments: dbateam@indiana.edu
The Trustees of Indiana University
Copyright 1999