MVS TOOLS AND TRICKS OF THE TRADE February 1995 Sam Golob MVS Systems Programmer sbgolob@cbttape.org Sam Golob is a Senior Systems Programmer ON DATASET RECOVERY - INTRODUCTION We systems programmers are sometimes confronted with a situation where a previously good dataset has gotten broken. It becomes our job to create some sort of recovery. Either the objective is to restore from a backup copy, fix the dataset, or short of that, we'd like to recover as much useful data as possible. Of course, every situation is different, but the cases do fall into certain known categories much of the time. This month, our concentration will be on introducing the tools needed to fix non-VSAM datasets. As we shall eventually see, there are enough of those types of problems to keep us quite busy studying. Next time, we'll talk specifically about broken PDS'es (partitioned datasets) and broken sequential datasets. And we'll try to develop an organized approach to fixing them. Today, we'll introduce the basis for that discussion. GETTING STARTED WITH BROKEN DATASETS When a dataset has something wrong with it, where do you start? Although I have my own experience, I often ask for a second opinion, and I asked my friend Rick Fochtman what he thought on this subject. He told me three questions he always asks. They are: "What does it look like?" "How bad is it?" "What can we do?" There is a lot of depth to Rick's approach, which seems more simple than it actually is. Let's look at these three questions in the context of a real problem. "What does it look like?" The first aspect of diagnosing a broken dataset is to form a picture in your mind of the "shape" of the dataset and the place where it seems to be broken. For example, if a disk sequential dataset has an I/O error somewhere in the middle, you can picture a long row of data blocks containing records. Each data block has its own TTR (relative location from the beginning of the dataset by Track and Record Count). Then you can superimpose some damaged TTR locations into that picture. This will give you a mental image of the problem situation to start with. There are "good" TTR locations before the damaged place, and there are good records afterwards. In this simple situation, we have answered the first question by creating an initial mental picture. In a real problem, we would probably have to continually refine this picture as new evidence accumulates. "How bad is it?" Let's go back to our simple example. We can ask: Is there only one I/O error, or are there several? Does the dataset have a proper EOF marker? Do we have a backup from before? How current is the backup--will several days' work be lost if we restore from the backup? Is the disk location of the error in a place where there is old data, or is the error in a place containing new data from after the backup was taken? All these questions and more, have to be asked before we go the the third step, which is deciding what we can do toward a remedy. "What can we do?" Here, we have to make some decisions, use some ingenuity, and marshal whatever tools are needed. The safest alternative is usually to try and restore the dataset from a backup copy. If that option is not open, either because no backup exists, or the data has changed too much since the backup was taken, then we might try to create another disk copy of the bad dataset which contains as much "good data" as we can recover. Finally, if this is possible, we have to find a way of getting the good data copied into the new dataset, in such a way as to allow the data to be usable. For all that, we need one or more tools. Sometimes, especially when there is corrupted data, there is a DCB blocksize problem. Such "details" must also be handled. (Editors: Please keep the quotation marks in the above section, as they have technical connotations. Thanks. ) TOOLS TO USE I am an advocate of the "whatever works" approach. When dealing with bad non-VSAM datasets, we will need a means of changing VTOC entries, especially FORMAT 1 entries. We will need a means of dumping or looking at bad data and good data. Finally, we will have to be able to copy good records from a damaged dataset to the "recovery" dataset. When dealing with partitioned datasets, we will need a means of restoring directory entries for "deleted" members. This is even more necessary when a PDS directory has been ruined, or it has been overlaid by sequential data. In that case, all of the old directory entries may have to be reconstructed. IBM has given us many of these tools. However, some of my favorite ones are not from IBM. That is because these non-IBM tools are often easier to use, leave less room for user error, and you can see what you're doing better. I will not write here about any tools that your shop has to buy. The non-IBM tools mentioned here are all obtainable from the CBT MVS Utilities tape that can be ordered from NaSPA. The CBT Tape may be freely copied, but it is frequently updated, so it pays to get a new one every so often. What are some IBM tools available? There is one that we'll always have to use. That one is ICKDSF, which we'll need to convert indexed VTOCs to OSVTOC format, so we can zap the VTOCs effectively. After our manipulations to the VTOC in OSVTOC format, ICKDSF is used again to convert the volume back to using the VTOC index. A bit more cleverness may be required if the volume is SMS managed and can't be easily un-indexed. See Figure 2 for JCL to un-index and re-index a disk volume. For looking at VTOCs, you can use IEHLIST with LISTVTOC DUMP. This will produce a CCHHR dump of all VTOC records. Then, to change them, you can use AMASPZAP (IBM's superzap) with its CCHHR control cards to do the VER's and the REP's. The dataset name will be "FORMAT4.DSCB", pointing to the unit and the volume serial. As awkward as this is, this technique with IBM utilities is very powerful and far reaching. A variation of it can even be used to make small adjustments to PDS directory entries, using different control cards for IEHLIST. However, user-friendliness is non-existent with this method. You have to thoroughly know what you are doing. To find out more about these IBM tools, look in the "Service Aids" manual and the "DFP Utilities" manual for almost any level of the MVS operating system. The "IBM approach" cannot be used without further knowledge of the layout of FORMAT 1 VTOC entries, and possibly, the knowledge of PDS directory entries. I am a strong advocate of using system macros from SYS1.MODGEN (or AMODGEN) and SYS1.MACLIB, along with the "Data Areas" manuals. The system macro IECSDSL1 from SYS1.AMODGEN will show the layout of all VTOC formats. See Figure 1 for how to use this macro. The IBM macro IHAPDS, also in SYS1.AMODGEN, will help you to recognize PDS directory entry fields. The layouts of PDS directory entries have been discussed in previous columns of mine (September thru November 1991). All of my previous columns are found on File 120 of the CBT MVS Tape, so you don't need back copies of this magazine. Non-IBM tools from the CBT MVS Tape are: (fullscreen) ZAP and REVIEW TSO commands from File 134, the CDSCB (change the DSCB) TSO command from File 300, and the "PDS" TSO command Version 8.4 from File 182. We'll elaborate on these four TSO commands shortly. The IEHMAP batch program on File 083 of the CBT Tape will produce a track map of an entire volume. IEHMAP conveniently supplies CCHHR values for the beginning and end of all dataset extents on a disk pack, which will greatly aid us in the VTOC zapping we'll have to do. These five non-IBM tools should cover us for the rest of this month's talk. (Editors: Please respect the selection of caps in the above paragraph. Thanks. ) The ZAP command, with keywords 'FORMAT4.DSCB' and VOL(volser) will allow you to look at and change VTOC entries on the screen. ZAP should be carefully restricted to systems personnel only, but we're fixing broken datasets here, and we'll need it. REVIEW is a full screen browse program with enormous power. Using REVIEW's NEWTOP subcommand, you can position the logical "top of data" just previous to an I/O error, or just subsequent to one. The newest versions of REVIEW (after version 379 of the CBT Tape) have CUT and ADD subcommands to copy records from data being looked at, to another dataset. This allows for recovery of sequential data records before and after an I/O error. The CDSCB command, which has to run authorized, allows the user to change DCB attributes of a dataset without having to know the exact layout of the FORMAT 1 VTOC entry. See Figure 3 for more information. Finally, the PDS command (Version 8.4) is a multi-purpose dataset utility that can be used to easily dump the FORMAT 1 DSCB layout of a dataset (using its "USAGE ALL" subcommand). "PDS" (with its RESTORE subcommand) can be used to "undelete" pds members, while with its DIRENTRY and ATTRIB subcommands, "PDS" can be used to view and change directory entry fields. I do not recommend using PDS's FIXPDS subcommand to change the DCB attributes of a broken dataset. This is because the FIXPDS DCB manipulations open the dataset for output, and they could ruin the setting of DS1LSTAR (the dataset high-water mark in the FORMAT 1 DSCB). Rather use CDSCB, which only opens and changes the FORMAT 1 VTOC entry itself. With CDSCB, the data in the dataset is not altered at all; only the VTOC entry for the dataset is altered. EXAMPLES. We've used most of our space for this month, and I'll have to be brief here. We'll continue and elaborate on this phase next time. Let's see what to do about the sequential dataset with one I/O error in the middle, assuming we can't use a tape backup of the volume. One way to recover is to REVIEW the dataset from the top, and find the location of the I/O error by issuing a FIND subcommand in REVIEW for a non-existent string. This will force REVIEW to search through all the data until it hits the I/O error. When you hit the I/O error, back up 20 records or so, and go DOWN one record at a time. The exact record count until the error position will be apparent. Mark this count down. Then allocate a duplicate copy of the dataset to ddname SYSUT2. Go to the top of the data, and CUT the required number of records. This will copy those records to the beginning of the SYSUT2 dataset. Then NEWTOP to a CCHHR considerably past the I/O error where you already have good data again. Slowly NEWTOP back, one block at a time, until you hit bad data. NEWTOP to the first block of good data, and ADD a large number of records, so that you copy the rest of the records, with DISP=MOD effectively, to the end of the SYSUT2 dataset. That's it folks. You've recovered most, or all, of the good records. Another approach would be to allocate a new dataset with the same DCB attributes as the old dataset. Then, with the VTOC un-indexed, ZAP the two FORMAT 1 DSCB entries so that the CCHHR extents of the original dataset only go up to the I/O error, and the other dataset continues from after the end of the I/O error to the end of the original dataset's extents. Thus we have two datasets, which together, contain recoverable data. You get the general idea. Next time, we'll talk about specific cases more. This kind of work helps you "feel like you are earning your pay as a systems programmer". Good luck and good zapping (only if necessary). See you next month. * * * * * * * * * * * * * * * * * * * * * * * Figure 1. How to obtain the layout of all VTOC record formats. Each entry in a VTOC is called a DSCB (Dataset Control Block). The Format 1 DSCB provides most (or all) of the information connected with individual datasets. If the dataset has more than 3 extents, you will also need a Format 3 DSCB entry. * * * * V T O C D E S C R I P T I O N * * * * ASSEMBLING THIS GIVES VTOC FORMATS FOR ALL TYPES OF * VTOC ENTRIES. (SYS1.AMODGEN) IECSDSL1 1 , FORMAT 1 DSCB IECSDSL1 2 , FORMAT 2 DSCB IECSDSL1 3 , FORMAT 3 DSCB IECSDSL1 4 , FORMAT 4 DSCB IECSDSL1 5 , FORMAT 5 DSCB IECSDSL1 6 , FORMAT 6 DSCB END , * * * * * * * * * * * * * * * * * * * * * * * Figure 2. Jobs to un-index the VTOC and re-index the VTOC of a pack. It is assumed that the pack was originally indexed, and we do not want to use the index dataset while zapping VTOC entries. When the VTOC is un-indexed, only the VTOC itself has meaningful information about the datasets on a pack. The index is then not used. Re-indexing the pack will have the effect of reorganizing the VTOC index and recalculating the pack's free space. To recalculate the free space of an OSVTOC volume, you have to turn on the "DIRF" bit of the FORMAT 4 DSCB, allocate a dummy dataset on the volume, and later delete the dummy dataset to clean up. //* * * * O S V T O C * * * //TSTBSP2N JOB (rest of job card) //*******************************************************************// //* NOTE..... THIS WILL ASK OPERS FOR A REPLY - YOU NEED A "U" IF OK. //*******************************************************************// //* CONVERT IXVTOC TO OSVTOC: //****************************************// //S1 EXEC PGM=ICKDSF,TIME=1439 //SYSPRINT DD SYSOUT=* //VPDKZ01 DD VOL=SER=PDKZ01,UNIT=SYSDA,DISP=SHR, <=== CHANGE ALL // DSN=SYS1.VTOCIX.VPDKZ01 <=== OCCURRENCES OF VOLSER NAME. BUILDIX DDNAME(VPDKZ01) OSVTOC /* // //* * * * I X V T O C * * * //*******************************************************************// //* NOTE..... THIS WILL ASK OPERS FOR A REPLY. "U" IF OK. **** //*******************************************************************// //* REBUILD INDEXED VTOC: //****************************************// //START EXEC PGM=ICKDSF,TIME=1439 //SYSPRINT DD SYSOUT=* //VPDKZ01 DD VOL=SER=PDKZ01,UNIT=SYSDA,DISP=SHR, <=== CHANGE ALL // DSN=SYS1.VTOCIX.VPDKZ01 <=== OCCURRENCES OF VOLSER NAME TO YOURS BUILDIX DDNAME(VPDKZ01) IXVTOC /* // * * * * * * * * * * * * * * * * * * * * * * * Figure 3. This is the Help member for the CDSCB TSO command, that can change any field in a dataset's Format 1 DSCB. CDSCB must run authorized, because any program which directly changes VTOC fields must have APF authorization under the MVS operating system. )F FUNCTION - THE CDSCB (CHANGE DSCB) COMMAND MODIFIES A DATA SET'S FORMAT-1 DSCB IN A VTOC. SINCE THE FORMAT-1 DSCB CONTAINS INFORMATION CRUCIAL TO A DATA SETS' SECURITY AND INTEGRITY, (AND IN FACT TO THE WHOLE SYSTEM'S SECURITY AND INTEGRITY), THIS COMMAND MUST BE RESTRICTED TO SYSTEMS SUPPORT PERSONNEL. NOTE: THIS IS AN APF-AUTHORIZED COMMAND, AND THEREFORE WILL NOT RUN PROPERLY UNDER SPF. YOU MUST EXIT SPF TO 'READY' MODE BEFORE USING THIS COMMAND. )X SYNTAX - CDSCB 'DSNAME' EXPDT('YYDDD') SHR VOL(VOLUME) UNIT(UNIT) CREATE('YYDDD') REFDT('YYDDD') DSORG(XX) RECFM(XX) LRECL(XX) BLKSIZE(XX) ALLOC(TR/CYL/BL) SPACE(SECONDARY-AMOUNT) PWR/PWW/NOP/RACF/NORACF ZAP(OFFSET VERDATA REPDATA) REQUIRED - 'DSNAME' DEFAULTS - NOTHING WILL HAPPEN IF NO CHANGES ARE SPECIFIED. ALIAS - NONE