MVS TOOLS AND TRICKS OF THE TRADE NOVEMBER 2007 Sam Golob MVS Systems Programmer P.O. Box 906 Tallman, New York 10982 Sam Golob is a Senior Systems Programmer. He also participates in library tours and book signings with his wife, author Courtney Taylor. Sam can be contacted at sbgolob@cbttape.org. Information about the CBT MVS Tapes can be found on the web, at http://www.cbttape.org. DISPLAYING MVS SYSTEM VALUES - Part 1 There's a general question when dealing with any operating system, and especially when dealing with MVS in any of its forms (z/OS, OS/390, and further back). That is: We know there is much information about the general well-being and the operating particulars of the system stored in the many control blocks and control areas of the system. But that information remains hidden from us, unless there's some way for us to display it clearly. How can we get to the information? And how can we use it to see what the system is doing? The answer, in the case of MVS, is that even if we see many of the system values, it still takes skill and knowledge to interpret them. But we must be able to see them first, and that requires a program (or many programs) to use for displaying them. When we logon to TSO and see the standard screen and the ISPF entry panel, those give us a warm and fuzzy feeling because they are very familiar to us. The entry TSO screens do not vary much, when going from one MVS instance or MVS installation to the other. But the entry screens give us very little visible indication about what is going on in OUR PARTICULAR SYSTEM, and about how THIS SYSTEM is behaving. When we first log on, we have absolutely no indication about whether our system is "sick" or "healthy". And we don't see a thing, about what kind of DASD drives are attached to the system, or about what processes and tasks are actually running there. We need more tools, obviously. Various program packages have been written to display the values in many system control blocks. These are the big monitoring tools, such as Omegamon (TM) and Resolve (TM) and such general utility packages. Those tools are supposed to help an administrator to regulate and watch the general health and operation of the system. And there are many more specific utilities and lesser-known monitors, such as Greg Price's IMON monitor, as well, which can do this kind of job. That's from a "general administration" point of view. This kind of work can be done by a skilled computer operator. But we, as the "system doctors", have to do (and know) much more. In our position, we are required to gain as much knowledge about the innards of the various MVS component parts as possible. And we should know the possible correct operating parameters of most of the system components, so that they operate efficiently, and run the way we want them to run. So how do we display these parameters and quantities, in such a way that we can use our skills to follow how our own MVS systems are doing? Fortunately, there is a free tool available for this, with source code accessible too. This tool is called SHOWMVS (originally from Gilbert Saint-flour), which in its more modern form, is called SHOWzOS. SHOWzOS is now maintained by Roland Schiradin. You can obtain both of these programs from CBT Tape File 492 (at www.cbttape.org) by looking at the Updates page. SHOWzOS performs the valuable function of making the MVS control information (which is hidden in the system innards) DISPLAYABLE. How SHOWMVS and SHOWzOS do this, is the subject of today's article. WHY SHOWzOS IN PARTICULAR? SHOWMVS and SHOWzOS are available in source code, which we can study ourselves. Looking at this source code can prove very useful in our learning process. Other similar tools, such as MXI from Rob Scott (CBT Files 409 and 410), and TASID from Doug Nadel of IBM (www.sillysot.com/mvs/tasid.htm) do not have source code with them, so while they are very useful to us, it is much harder for us to learn about the actual paths to the MVS innards from them. Even though they display the contents of many of those innards, MXI and TASID won't show you directly, where (in the system) their information came from. From studying SHOWMVS and SHOWzOS, we can learn how to do control block displays ourselves, in our own programs. SHOWMVS and SHOWzOS are written in Assembler language. We can look at the SHOWzOS Assembler code and figure out the direct paths to each control block which SHOWzOS can display. It's not too easy, just from looking at the code, but the knowledge is worth the effort. Once we know how to get to a piece of system data, Assembler code, or even REXX, can be used to display it. The display tool in SHOWzOS is packaged as a macro (invented by Gilbert Saint-flour or by members of his group) called STRING. In SHOWzOS, the 64-bit version of STRING is called STRING64. Using the STRING or STRING64 macro, a whole line of data, built up from various fields and character strings mixed together, can be displayed by one macro instruction. That makes it easy for SHOWMVS or SHOWzOS to produce its complicated display lines. If you write your own display code for some control blocks, you don't have to use the STRING macro, but you can form your display lines in some more traditional manner. I'll try to write more about this subject next time. Meanwhile, look at Figure 2 to see some sample SHOWzOS output. As for using REXX instead of Assembler, if REXX is easier for you, there exist wonderful examples of the use of REXX to display system control quantities. Mark Zelden's examples (in CBT Tape File 434 - be sure to use the Updates page of www.cbttape.org) are among the best I've seen. Especially look at Mark's marvelous IPLINFO REXX exec. You can learn a lot from looking at the IPLINFO REXX source, and from Mark Zelden's stuff on CBT File 434, in general. Again, check the Updates page on www.cbttape.org first. HOW TO FIND AND DISPLAY MVS CONTROL INFORMATION Every component of MVS has to store its control information somewhere. After the initial control information has been stored away, at IPL time or during the later initialization of a component, the system has to be able to find it or change it, when it needs it. So there have to be well designed and well documented paths that get you to each particular piece of information. Once you know these paths, you can get to the information. And once you get to the information, you have to figure out how to display it and make it visible. Otherwise you'd have to interpret all the raw hexadecimal bits and bytes, which is much more difficult than just seeing the correct quantities displayed plainly in front of you, together with their appropriate titles. The layouts of most MVS control blocks are described by MACROS, at least in Assembler, and also in IBM's proprietary system development language, PL/X. IBM uses PL/X (which is very much like PL/I) to write many of the MVS system components. IBM releases the contents of many of its macros to the public, in its big macro libraries SYS1.MACLIB and SYS1.MODGEN, although some of the IBM macros are distributed in small macro libraries that are specialized to the individual components. A key example of this is HLASM, the High Level Assembler, whose macros are available in macro libraries HLA.SASMMAC1 and HLA.SASMMAC2, and not in SYS1.MACLIB or SYS1.MODGEN. When IBM does NOT want to publish the contents of a control block at all, it does NOT release the descriptive macro (which shows the layout of the control block) to the public. For sure, such a macro does exist within the confines of IBM. Those macros which IBM uses for its own development purposes are marked "for Internal Use Only", and are not released to the public. So, in that case, if any "interested users" would want to know what is going on, they have to research that control block, try to figure out how it is used, and manufacture a descriptive macro for themselves. I have found the LOOK program (CBT Tape File 264) which we described last month, to be very useful in doing this research. The idea is to try and document the contents of the unknown control block, so we can write a program which can then display its current values. Since it is the job of the SHOWzOS program to display as many critical system values as possible, the developers of SHOWzOS have done much of the necessary research to create their own descriptive macros that show the layout of many undocumented IBM control blocks. You can be the beneficiary of their exhaustive and painstaking research. Therefore, take a look at CBT File 492 (use the www.cbttape.org Updates page) member SHOWMACS. SHOWMACS is a collection of user-developed macros which attempt to describe many MVS control areas that IBM doesn't especially want you to know about. The member SHOWMACS is really an unloaded pds in IEBUPDTE unload format. The SHOWMACS library on CBT File 492 (again, use the Updates page if possible) can form the basis of much of your initial research into unknown MVS control blocks. If you are interested enough in seeing an in-core version of some of the areas which these macros describe, you can actually assemble them into the CBMACS component of the LOOK TSO command which we wrote about last month (CBT File 264), and you can use the macros from the SHOWMACS library to format the (proprietary) control areas of your choice, while you are viewing system storage on the screen! SOME SUGGESTIONS Please look at Figure 1, which summarizes the kinds of MVS control information that SHOWzOS can display. As you can see, it is a huge list. If you are interested in making your own displays of similar pieces of system information, you can get your start by examining the particular pieces of SHOWzOS code which access the areas you are interested in looking at. Also take a look at Figure 2, which shows some of the top level information which SHOWzOS displays in a modern system. Thus, you can get a small idea about the way SHOWzOS displays its information. My advice is: Try it. You'll like it. If you want load modules, or (for some reason) you can't cleanly assemble the SHOWzOS source code, the CBT Tape collection has a load module library file on File 614 (again try and use the Updates page). Various levels of SHOWMVS and SHOWzOS load libraries, in TSO XMIT format, are members of the File 614 pds. The reason why these load module libraries are in a separate location from the source code, is because some of the source code is z/OS release dependent, and the load module for SHOWzOS should PREFERABLY be exactly matched to the level of system macros your installation has, for all installed components. That's why we can't create ONE load module to exactly fit all the components in your system's particular mix. But File 614 is there for you to use, so you can fire up SHOWzOS quickly on your system. If there are any runtime errors, you can always reassemble SHOWzOS again later. This topic will be continued next month. I hope that you have gotten a bit wiser by reading this article. At least, if it makes your mind work and "gets the wheels turning" a bit, I feel that I've done my job. I wish all of you the best of everything, and I'm looking forward to seeing all of you here again, next time. * * * * * * * * * * * * * * * * * * * * * * * Figure 1. A summary of SHOWzOS functions, taken from the actual source code. Many of the SHOWzOS displays can be created, even if the SHOWzOS command is not being run APF-authorized. Which particular displays need APF authorization to run, are indicated below. SSSS H H OOOOO W W ZZZZZ OOOOO SSSS S H H O O W W ZZ O O S SSS HHHHH O O W W W Z O O SSS S H H O O WW WW ZZ O O S SSSS H H OOOOO W W ZZZZZ OOOOO SSSS This program displays information obtained from various control blocks of the MVS system on which it is run. The information can be displayed under ISPF using EDIF or BRIF, written to the TSO terminal using PUTLINE or to a data set using QSAM. SHOWzOS displays two different types of data: 1. Data related to the MVS operating system: MVS LEVEL, DFP LEVEL, OSLVL FLAGS, JES2 LEVEL IPL DATE, TIME, VOLSER, NUC-ID, CLPA, CVTUSER Date and time of last CLPA (auth) Time Zone, Primary sub-system name and type Software Level: TSO/E, ISPF, DSS, HSM, RACF, VTAM, SORT DSF, DB2, CICS, MQ Series, TCP/IP, LE, IMS (v9 and higher) Run-time Environments (CBL,PLI,FORTRAN,C,LE,REXX,SAS/C RTLS) SORT LE runtime options (CEEDOPT, CEECOPT, CELQDOPT) UNIX System Services Settings HFS Statistics Files PID IPC ISV software base CSRCTABL ICSF configuration (Crypto) SMF Information: SID, JWT, data sets, exits SMS Configuration: SCDS, system names, last update dataclas, mgmtclas and so on HSM information SDM information GRS Configuration Hardware configuration Central Processing Complex Node Description (CPC ND) On-line CPUs and storage Performance (SU/sec, estimated MIPS) Service Processor data CSRSI support Other CPU related attributes I/O configuration Definition (IODF) LPAR configuration VM host information (auth) VIRTUAL STORAGE MAP (CSA, SQA, LPA, etc) Subpool attributes Global Cellpools Subpool Usage Common Storage Usage Dataspaces (auth) 64-bit memory object and MEMLIMIT CSM Communication Storage Manager XCF Configuration (auth) Path data (auth) Couple Datasets (auth) Coupling facility (auth) External timer (auth) Resource Manager WLM data (MODE, etc) SMF data for each ASID (enclaves, zAAP, zIIP) (auth) System Logger (IXGLOGR) (auth) Resource Recovery Manager (RRS) Automatic Restart Manager (ARM) TCP/IP statistics, connections, listener (auth) OPEN catalogs PAGE data sets in use Dump data sets status and dump options Dump data sets date, time and title (auth) Automatically-allocated dump data sets (auth) DAE Parameters (auth) SLIP data (auth) GTF status and options DIAG settings Initialization Parameters (IPA) IPL-Statistic Health Checker (auth) Master JCL (IEEMSJCL) Sub-system vector table with functions processed Primary Sub-system JCL (auth) TCAS parameters (auth) TSO PARMLIB data (IKJTSOxx) TSO Exits and Tables RACF options, data sets and tables UNICODE data Address space usage: Batch Jobs TSO users Started tasks USS users CICS regions STEPLIB/DFHRPL (&VERBOSE_CICS switch) (auth) TCP/IP users JES2 Initiators and corresponding jobs JES2 Dynamic proclib (auth) JES Monitor and Job class information (auth) Link-list data sets, with creation date LPA-list data sets, with creation date List of authorized libraries Dynamic Exit Facility (auth) Static system symbols LLA parameters and managed libraries (auth) ACTIVE LPA QUEUE SVC Table with name of the corresponding module T1, T2, T3 and T6 ESR tables Linkage Index (LX) Table (auth) Auth Index (LX) Table (auth) Cross Memory Connection (XMS) (auth) Memory Delete Queue (MDQ) (auth) PC usage (ETE) (auth) ENF Listener (ENF) (auth) Timer Queue Elements (only DIE) (TQE) (auth) Program Properties Table (PPT) (auth) I/O Appendage Table Resource Manager List (IEAVTRML) Products Information Device Classes and corresponding unit names On-line devices, with unit-name, VOLSER, owning job, use attribute, storage group Channel Measurement (ECMB) (auth) Config Data Record (CDR) (auth) PAV info (PAV) (auth) Channel Path information Channel Path Measurement Facility System consoles, with status & Routcde list EMCS-Consoles (auth) CMDS (auth) Consol Query (auth) Master Trace Table (MTT) see &MTTDATA (auth) Message Processing Facility (MPF) Command Prefix Table (CPFT) Name/Token information Device Allocation Defaults (ALLOCxx) Addresses of selected global control blocks 2. Data related to the current address space JCL information for current JOB/STEP RACF profile (from ACEE) TSO profile (from PSCB & UPT) ISPF Tso Command table (ISPTCM) REXX environments, host cmd tables and func pkg directories Allocated Data sets (from TIOT, SWA, TCT) TCB tree and PRB chain Attention Routines Enhanced view of the JPAQ and Load-lists Local Cellpools Recovery exits and timers Local Name/Token Addresses of selected local control blocks * * * * * * * * * * * * * * * * * * * * * * * Figure 2. Top few lines of a modern SHOWzOS Display. On a small system, the display can be well over 7000 lines long! BROWSE GSF Utilities - SHOWzOS R715 Line 00000000 Col 001 080 Command ===> Scroll ===> CSR ********************************* Top of Data ********************************** SHOWMVS is running authorized >Operating System: z/OS 01.08.00 FMID: HBB7730 CVTOSLVL: FF FF FF BF EB 60 00 00 DFSMS 1.8.0 Feature : FF 6E 73 7F A0 30 DFSMS Loader Fork Exit is present DFSMS RVA SnapShot is available DFSMS RVA SnapShot API is available JES2 Level: z/OS 1.8 NJE Node: N1 DSNID: 01 SHOWzOS REL=R715 20070503 20.30 MVS=708 SMS=03010800 HLASM=1.5.0 OSREL=01010800 STRING=R515 Switches=E2C03FC000000000 Auth=Yes,ESR=IGX00011 >Last IPL: Date: Monday 2007-10-24 (Today) Time: 01.29 Julian: 2007.297 From: S8RES1/0A80 NUC Id: 1 Type: Warm Start CVTUSER: 00000000 Last Cold Start (CLPA) Date: 2007-10-22 Time: 13.27.14 Last Quick Start (CVIO) Date: 2007-10-22 Time: 13.27.14 SYSPLEX name: ADCDPL SYSPLEX ID: 1A Sysname: ADCD Lparname: Timezone: W 04.00.00 Leap Seconds: 0000000000000000 >System Software: TSO/E Level: 3.8.0 ISPF Level: 5.8 PDF 5.8 RACF Level: 7.73.0 ICKDSF Level: 1.17.0 VTAM Level: 6.1.8 VE618 00C7D008 LE Version: 1.8.0 MQ Series: 5655L8200 SSCTSNAM=CSQ1 Inactive MQ Series: 5655L8200 SSCTSNAM=CSQ2 Inactive MQ Series: 5655L8200 SSCTSNAM=CSQ3 Inactive TCP/IP: 05.363 5655-HAL MVPTASK Tseb SI Proc Ver Tsdb Tsdx Asid TraceOpts Status 107D5040 01 TCPIP 06.18 107BA000 107BA0C8 0017 00000000 Active * * * and much more....