MVS TOOLS AND TRICKS OF THE TRADE July 1993 Sam Golob MVS Systems Programmer sbgolob@cbttape.org Sam Golob is a Senior Systems Programmer MAKING SEQUENTIAL FILES OUT OF PDS'ES - AND VICE-VERSA. It's good to be back by popular demand. Thank you all very much for making yourselves heard. My hope is to be able to continue sharing, helping, and being helped by all of you out there. Today's column is about fixed and variable blocked pds'es. Sometimes it isn't efficient or easy to process them member by member. There are times when it is better to deal with a single sequential file constructed from a partitioned dataset, rather than with the partitioned dataset as it is, member by member. We shall talk about creating sequential files from partitioned datasets, and about reconstructing the partitioned datasets from these sequential files. Why should we want to do this? One obvious purpose is to download a pds to a PC file. It's a lot easier to download one file to one PC file, than to download hundreds of members of a pds, member by member, into separate PC files in a subdirectory. The individual members can later be separated on the PC side by using a file editor such as SPFPC, KEDIT or DOS EDIT. Some of you out there might have better methods of separating the members on the PC. If so, I'd like to hear about them. Please contact me via Naspa headquarters. I'll give you another example. Arnold Casinghino (the founder and longtime proprietor of the CBT MVS tape) once called me with the following problem. He was the ACF2 administrator at his shop, and he had to deal with a 2000 member pds, LRECL 130. This was some kind of ACF2 database, consisting of access rules. Arnie had to run some CLIST processing globally against all the members, and it took about an hour to run because each member had to be accessed separately. Arnie felt that if he could convert this pds into a sequential dataset, IEBUPDTE style with "./ ADD NAME=memname" cards as member headers, his CLIST would run in a much shorter time. The problem was that IEBUPDTE doesn't work for datasets with LRECL 130, only with record lengths of 80. Two programs on the CBT tape that did this processing, OFFLOAD and PDSLOAD, also (at that time) only handled datasets with an LRECL of 80. What could Arnie do? I set out to solve Arnie's problem. I didn't want to work directly with IEBUBDTE because it was IBM's program, but I looked at the source code to the OFFLOAD and PDSLOAD programs, which are public domain and which we had available from the CBT tape. After searching for all references to the DCB LRECL in the source, I found that I could convert the LRECL from a hard-coded value of 80 to a global conditional assembly variable, at all occurrences in both programs. Then I could assemble both programs for any LRECL that I needed at the time. This jury-rigged solution worked well for Arnie's problem, and his CLIST took only two minutes to run against the sequential file he created from his ACF2 pds. The two changed programs, PDSLOAD and OFFLOAD, were then placed on File 93 of the CBT tape so they could be used by the public. Greg Price of Melbourne, Australia has since fixed PDSLOAD so it doesn't need conditional assembly, but will automatically handle record lengths for fixed blocked datasets with any LRECL from 1 to 256. Greg's version of PDSLOAD is the one on File 93 of the CBT tape currently. This process has since gone much farther. File 93 now contains more software than those two original programs. At one point in time, I discovered the existence of the SHARE PL/I tape, which is now distributed by SPLA at the University of Miami in Florida (telephone 305-284-6257). It's nice to have this tape, especially if you're a PL/I buff. Since interest in PL/I is not necessarily limited to MVS only, the format of that tape's files had to be made available to the other operating systems. The format therefore had to be as close to a flat file as possible. IEBCOPY-unloaded pds'es are mostly unacceptable on the SHARE PL/I tape. The PL/I tape solution is related to our concerns here. That tape supplies two programs, called UNUPDTE and UPDTE, which convert both fixed blocked and variable blocked pds'es to sequential datasets using "./ ADD NAME=memname" headers. UNUPDTE converts a pds to a sequential dataset, and UPDTE reloads the pds from the sequential dataset. Most files on the SHARE PL/I tape are pds'es converted into sequential files by UNUPDTE. The advantages of UPDTE and UNUPDTE over PDSLOAD and OFFLOAD are that UPDTE and UNUPDTE can handle variable blocked files also. PDSLOAD and OFFLOAD will only handle fixed blocked files. UPDTE and UNUPDTE will deal with record lengths of up to 32760. I'm not sure if PDSLOAD and OFFLOAD are safe above record lengths of 255. Anyway, when I saw the UPDTE and UNUPDTE programs, I called Bob Styma, proprietor of the SHARE PL/I tape, to ask his permission to add them to File 93 of the CBT tape. They are now included there. Both UPDTE and UNUPDTE use JCL that resembles the corresponding JCL which IEBUPDTE would use in a similar situation. See Figure 1 for an illustration of their JCL. UNUPDTE and UPDTE are extremely flexible in the types of datasets they will process. But they do have one disadvantage in an MVS environment, which we'll now discuss. ISPF STATISTICS. When IEBUPDTE unloads a pds to a sequential file, the ISPF statistics for the individual members are lost. That is because the (./ ADD NAME= ) IEBUPDTE member headers, as defined by IBM, do not contain control parameters to preserve ISPF statistics for each member. However, this problem has been solved in a standard way by a number of programs which are on the CBT MVS tape, and by the vendor product called PDSTOOLS (from Serena International) which handles all this processing with its COMBINE and SEPARATE subcommands. These programs modify the standard IEBUPDTE member header to include the values of the ISPF statistics for each pds member. This is done in a standard format. See Figure 2. The modified member headers can either be processed by IEBUPDTE, with the statistics lost, or by PDSLOAD and other programs, with the statistics preserved in the new loaded members. Two free programs which preserve ISPF statistics in IEBUPDTE- format member headers are: LISTPDS from File 316 of the CBT MVS Tape (see Figure 2 for its JCL), and Greg Price's version of REVIEW from File 134 of the CBT Tape. LISTPDS is a batch program, very nicely written, but it only handles pds'es with 80-byte record length. REVIEW will only work interactively as a TSO command. REVIEW is normally a browsing program for partitioned datasets, members, and VSAM files. But if you REVIEW a pds, you will get a member list with an 8-byte command area at the top (see Figure 3). If you had previously allocated a suitable sequential output dataset under ddname SYSUT2, you can enter the expression "=OFFLOAD" in the command area of the REVIEW member list screen. REVIEW will then dump the contents of the pds into the SYSUT2 dataset in IEBUPDTE format, preserving any ISPF statistics present. This works for FB and VB datasets, and for any LRECL up to 32760. Quite nice. REVIEW will also selectively dump some members, but not all. To do that, you must have previously selected those members from the member list by placing an "O" next to each of them, before issuing your final =OFFLOAD command. If your installation is lucky enough to have the product PDSTOOLS (formerly PDS/E SysProg Utilities), that product will do all this work with just two of its many subcommands. The COMBINE subcommand of PDSTOOLS will string together any desired group of pds members into a single file. You may optionally including IEBUPDTE-type headers in between, having LISTPDS-format ISPF stats. The SEPARATE subcommand of PDSTOOLS will separate a COMBINEd file into individual members again, stowing ISPF stats back if they were there before. PDSTOOLS will work with any FB or VB pds, up to LRECL 32760 (VB to 32756). This product does our job best of all. One final note. Sometimes the data in a pds contains control cards that are already in IEBUPDTE format. For example, the data might contain "./ CHANGE" cards. In that case, the LISTPDS, OFFLOAD, UNUPDTE and REVIEW programs change the "./" string to another string, usually "><", so that an IEBUPDTE reload of the pds would not get confused. If PDSLOAD reloads the pds, it will automatically restore the "><" string to "./" when each member is reconstructed. That process is controlled by a parm, so that you can substitute an arbitrary two-character string instead of "><". SUMMARY. I have found many occasions in the course of my career, when it was inconvenient to deal with a partitioned dataset as a library. It would have been far more convienient to treat the data as a single sequential file. All of you have undoubtedly encountered numerous times when you wished you could conveniently turn a pds into a sequential file, do some processing, and turn the sequential file back into a pds with no loss. Now you have some tools for this. My aim here has been to show you the existence of the technique and the means to accomplish the result. Get a CBT Tape (NaSPA distributes it) and explore the programs we've described here. You'll have a few more tools in your box. Good luck in everything you do. See you next time. * * * * * * * * * * * * * * * * * * * * * * * Figure 1. JCL to execute the UNUPDTE and UPDTE programs from the SHARE PL/I tape, and from File 93 of the CBT MVS Tape. Notice how this JCL corresponds closely to IEBUPDTE JCL which does the equivalent function. The difference is that UPDTE and UNUPDTE will handle FB and VB datasets of any LRECL, as opposed to IEBUPDTE, which only handles FB datasets with record length equal to 80. //UNUPDTE JOB (TS,8400),'TECH SUPP-SAM GOLOB',CLASS=M,NOTIFY=&SYSUID, // MSGLEVEL=(1,1),MSGCLASS=V //* //***************************************************************// //* UNUPDTE EXECUTION. CREATE A SEQUENTIAL FILE FROM A PDS *// //***************************************************************// //UNUPDTE EXEC PGM=UNUPDTE,REGION=1000K //STEPLIB DD DISP=SHR,DSN=MY.USER.LOADLIB //SYSPRINT DD SYSOUT=* //SYSUT1 DD DISP=SHR,DSN=ABE.DOCSUB <=== partitioned dataset //SYSUT2 DD DISP=SHR,DSN=PL1MOD.FILE18 <=== sequential file (already allocated) //UPDTE JOB (TS,8400),'TECH SUPP-SAM GOLOB',CLASS=M,NOTIFY=&SYSUID, // MSGLEVEL=(1,1),MSGCLASS=V //* //***************************************************************// //* UPDTE EXECUTION. LOAD A PDS FROM THE SEQUENTIAL FILE *// //***************************************************************// //DOUPDTE EXEC PGM=UPDTE,REGION=1000K //STEPLIB DD DISP=SHR,DSN=MY.USER.LOADLIB //SYSPRINT DD SYSOUT=* //SYSIN DD DISP=SHR,DSN=PL1MOD.FILE18 <=== sequential file //SYSUT2 DD DISP=SHR,DSN=ABE.DOCSUB <=== partitioned dataset (already allocated) * * * * * * * * * * * * * * * * * * * * * * * Figure 2. JCL to run the LISTPDS program from File 316 of the CBT MVS Utilities Tape. This program preserves ISPF statistics in a standard form that can be processed by the PDSLOAD program to reload the statistics into a new pds. The LISTPDS program will only sequentialize pds'es with LRECL 80. Greg Price's version of the REVIEW program (File 134 of the CBT tape) will perform this "unload" service from TSO interactively for FB and VB datasets of any LRECL. See Figure 3. //SEQUNLD EXEC PGM=LISTPDS,REGION=2000K, <=== from File 316 of CBT // PARM='UPDTE(><),SPF,NOLIST,DECK,NOSEL' //STEPLIB DD DISP=SHR,DSN=SYS1.USER.LOADLIB //SYSPRINT DD SYSOUT=* //SYSLIB DD DISP=SHR,DSN=THE.SOURCE.PDS //SYSPUNCH DD DISP=(NEW,CATLG,DELETE),UNIT=SYSDA, SPACE=(TRK,(15,15),RLSE), DSN=THE.OUTPUT.SEQNTIAL.FILE, DCB=(RECFM=FB,LRECL=80,BLKSIZE=23440) Format of the IEBUPDTE-type header created by the LISTPDS program or by =OFFLOAD processing of Greg Price's version of REVIEW. This header may be processed by IEBUPDTE or by PDSLOAD. If processed by PDSLOAD, the ISPF statistics are stowed for the new member created. ----+----1----+----2----+----3----+----4----+----5----+----6----+----7-- ./ ADD NAME=SAMPMEMB 0100-92332-92335-2005-00034-00023-00001-MYUSRID (field format): vvmm-crdat-modat-hhmm-size -init - mod -userid * * * * * * * * * * * * * * * * * * * * * * * Figure 3. Illustration of sequentialization of a pds, using Greg Price's verion of the REVIEW command from CBT Tape File 134. The REVIEW command has been issued against the pds SYS1.PARMLIB. An output dataset has been previously allocated to ddname SYSUT2, and the "=OFFLOAD" command has been issued in the REVIEW command area next to the word, "LOCATE". The bottom part of this figure shows the top of the screen when the offloading operation has been completed. SYS1.PARMLIB -------------------------------------------- 177 MEMBERS ON ESACAP LOCATE ===> =offload 1-TOP 2-BOT 3-END 4-SD 6-SN 7-UP 8-DN 9-SSZ 10-STTR 11-SID NAME TTR VV.MM CREATED LAST MODIFIED SIZE INIT MOD ID ACCMDTT 001407 01.62 91/09/19 92/01/26 08:46 311 288 0 TIMMY ADYSET00 00002A 01.01 90/05/21 90/07/15 17:11 15 15 0 IBM ADYSET01 00002C 01.01 91/09/19 92/01/26 15:23 11 11 0 IBM ALLOC00 000401 01.01 91/07/22 91/07/26 11:03 99 99 0 IBM ARCCMD00 006808 01.18 93/03/19 93/04/30 09:53 372 366 0 CAROL BLSCECT 00050B CLOCK00 001912 01.02 93/01/15 93/01/17 08:11 4 5 0 JOEY CNLENU00 000405 01.00 91/07/26 92/07/26 12:46 55 55 0 IBM COFDLF00 000408 01.00 91/07/26 92/07/26 13:23 26 26 0 IBM COFIPCSP 00040A 01.00 91/07/26 92/07/26 12:39 73 73 1 BORIS COFVLF00 002706 01.01 92/12/22 93/03/03 13:08 55 56 0 BORIS COMMND00 00110B 01.03 92/07/31 92/08/23 06:09 30 46 0 BORIS COMMND01 001106 01.01 92/08/19 92/08/19 15:24 17 38 0 BORIS OFFLOAD COMPLETE - SYSUT2 CLOSED ------------------------ 177 MEMBERS ON ESACAP LOCATE ===> =offload 1-TOP 2-BOT 3-END 4-SD 6-SN 7-UP 8-DN 9-SSZ 10-STTR 11-SID NAME TTR VV.MM CREATED LAST MODIFIED SIZE INIT MOD ID ACCMDTT 001407 01.62 91/09/19 92/01/26 08:46 311 288 0 TIMMY . . . . . . . . . etc.