MVS TOOLS AND TRICKS OF THE TRADE OCTOBER 2001 Sam Golob MVS Systems Programmer P.O. Box 906 Tallman, New York 10982 Sam Golob is a Senior Systems Programmer. He also participates in library tours and book signings with his wife, author Courtney Taylor. Sam can be contacted at sbgolob@cbttape.org. Information about the CBT MVS Tapes can be found on the web, at http://www.cbttape.org. LEARNING ABOUT MVS FROM THE SMF BOOK Ask any of us the question: "Do you know ALL about MVS"? We'll all answer, "no". Ask us again: "So what DO you do?" And most of us will answer, in one way or another: "We keep on learning more about MVS, but it's 'way too big' for us to learn ALL about it." That's what we're going to talk about today. How does one "keep on learning more" about MVS? It's important to find some angles and approaches to MVS knowledge, that are useful and productive to follow. Today, I'll show you one unusual approach. Some of us have doubtless already thought about this idea, but I'm quite sure that not everybody has followed through on such a method of learning MVS, in everyday practice. So I think the method is worth talking about. Also, it's somewhat original, but it makes a lot of sense when you think about it. The idea is, to learn as much as you can about MVS, by studying the contents of the various types of SMF records, and by looking in the current IBM SMF manual for the kinds of material that SMF tries to collect. Since the purpose of SMF is to collect the history of your MVS system, it will pay for you to know what type of information that the MVS developers believe constitutes "your system's history". And thus you can gather more "know how" about how your MVS system actually does work. I'll explain further why this will help you learn about MVS. SMF, or the MVS "System Management Facilities", is a way of recording for posterity, the many transitory phenomena which tell you how an MVS system is performing. Most significant actions on an MVS system, will cause an "SMF record" to be written to the "active SMF dataset" (a special sequential VSAM file) every once in a while. Each "type of action" will write a different "type of SMF record" to the active SMF dataset. And the SMF dataset will accumulate many of these SMF records, of many different types, over the course of time. Now if you want to know what your MVS system was doing, the information you save, has to be indicative of all the different kinds of actions which the various parts of the MVS are constantly performing. The MVS developers, when designing the format of each type of SMF record, studied each part of MVS, trying to figure out what information was worth saving. It has to be facts and quantities that concern the inner workings of MVS. So what easier way to learn the inner workings of MVS, than to look at which facts and measurements these developers have decided to save, in each appropriate type of SMF record? Right now, we won't concentrate on the usual job of how to SAVE the physical SMF records for posterity. That's what we often think about, when it comes to SMF records. But today, we'll rather concern ourselves with trying to understand the content and format of each type of SMF record, so we can grasp the workings of the MVS component which created it. I have used this approach, and have learned a lot from it. Now I'll tell you some more specifics about this idea. SMF RECORD TYPES There are 256 valid "types of SMF record", and the numbering goes from 0 thru 255. These correspond to the bit contents of one byte. IBM tries to assign low numbers (less than 120 or so) to SMF records that they produce themselves. The other vendors generally assign high SMF record type numbers, or more usually, they ask the installation to choose their own number. Most installations usually choose high numbers--in the 200s, for user-written or vendor-written SMF records. Additionally, an SMF Record Type may contain several subtypes. The designation of the subtype also occupies one byte of the SMF record. Here's one illustration about how subtypes of SMF records are used. For example, job-related and jobstep-related accounting information is contained in SMF Type 30 records. But information gathered at job initiation time, gets written to subtype 1. Similar information concerning job step termination gets written to subtype 4. And information concerning job termination gets written to subtype 5. That's just one case, illustrating how subtypes of SMF record types can be used. A list of the formats of the various "IBM-assigned SMF Record Types" may be found in the current IBM SMF Manual, usually entitled: "MVS System Management Facilities (SMF)". The manual number for the z/OS version of the SMF manual is SA22-7630. The OS/390 manual number for this book is GC28-1783. This list of formats of SMF records, is what I'm suggesting that you spend some time studying. GETTING A GRASP OF SMF RECORD TYPES I'd suggest that the first thing to do, when it comes to studying SMF records, is to read the first couple of chapters in the SMF manual. That'll give you an introduction to what SMF records are for. After that, I'd suggest that you go straight to the beginning of the chapter about the individual SMF record formats, which is entitled, "SMF Records". Once there, take a slow look at the "one-liners" which say what each SMF record type is about. You'll discover that there is a pattern to the assignment of the SMF record numbers. Each component of MVS is assigned a certain range of SMF Record Type numbers, or a certain single number which has several subtypes. For example, SMF record types which report on VSAM-related topics, are all assigned SMF type numbers from 60 thru 69. SMF record types that reflect RMF statistics that were collected, all have numbers from 70 thru 79. RACF-related SMF records have type numbers from 80 thru 83. DB2-related SMF records are assigned Types 100 thru 102. On the other hand, all CICS-related records are in Type 110 only. If you're running CICS, you'll probably find a lot of these. And DFSORT statistics are only collected in Type 16 records. While we're here, I'll give you a hint on how to look at the SMF records that are being collected on your own system. The REVIEW TSO command from File 134 of the CBT MVS Utilities collection, can be pointed at any SMF dataset, to browse its contents. So why is REVIEW better than ISPF BROWSE? Because REVIEW has an "SMF" subcommand that formats the timestamps and the type and subtype numbers of all the SMF records. REVIEW also has a "FINDSMF" or "FS" subcommand, which will get you the next SMF record of a certain type. For instance, if you say, "FS 14", REVIEW will position your screen to the next "Type 14" SMF record, and will highlight it. This facility of REVIEW is extremely handy. REVIEW can also look at SMF records on a tape, if you give your TSO session the MOUNT attribute, using the CPSCB command from CBT Tape File 300 (or if your session has the MOUNT attribute already). When looking at an SMF dataset using REVIEW, one invocation of the SMF subcommand will format the records, and a second invocation will toggle back to looking at the raw data. REVIEW is an extremely handy tool for looking at the real SMF records on your system, while you're reading about the SMF record formats in the SMF book. Now, let's get back to studying the SMF record formats. After you've done your overview of all the types, I'd suggest starting to study the RMF record types in detail: Types 70 thru 79. Since RMF collects performance data from many MVS components, you'll gain a ton of knowledge from studying the fields of RMF-generated SMF records. I'll answer another question before you ask it. What if you have an RMF-substitute package, such as CMF from BMC software, and you don't run RMF? There's still a simple answer. CMF does produce its own SMF records. Perhaps your installation has assigned Type 240 for them. But if you REVIEW the records in the SMF datasets, you'll still see Types 7x being generated, and just afterwards, there'll be the Type 240 records too. CMF seems to try to achieve some compatibility with RMF, so if you're converting from RMF to CMF, I guess the reporting stats won't be too different. But CMF's own specific extra reporting will be in the Type 240 records that come afterwards. Now here's an example of what you can learn: If you have an SMF book handy, turn to the format of Type 71 records. There are fields in that record type, which account for every different type of SWAP reason code. By looking in the book, you'll see that there are at least 17 different reasons, why an address space might be swapped out. For instance, a TSO address space might be waiting for a command from a terminal. That's called a "terminal input wait". You can look in the SMF book to see the names of the 16 other SWAP reasons, and to try to understand what they mean. By doing this, you'll learn an awful lot about address space swapping in MVS. So the plan would be, to dip into the SMF book on a fairly regular basis, and try to understand the "reasons behind the fields". You should especially concentrate on SMF record types which concern the parts of MVS you're dealing with, in your regular MVS work. SUMMARY Let's just sum up this approach to learning about MVS. SMF records capture the history of what's going on in your MVS system. The information gathered, will reflect the inner workings of many MVS components. Therefore, if you look in the IBM SMF manual, and vendor manuals, which describe the fields of the SMF records being produced, you will learn much more about the inner workings of that component. The REVIEW command from File 134 of the CBT MVS Utilities Tape collection, can view and format real SMF records on your system, and on tape. Using the REVIEW command, you can see, on your own system, the information you're been learning about from the SMF book. I only have space to give you a hint at this "different" approach to learning more about MVS. The job of "following through" will have to be yours. If you want to exploit this enormous opportunity to increase your MVS knowledge, it lies before you. Take a stroll on the path, and enjoy.... That's all for now. I hope to see all of you, next month!