MVS TOOLS AND TRICKS OF THE TRADE
                         FEBRUARY 2004

                                      Sam Golob
                                      MVS Systems Programmer
                                      P.O. Box 906
                                      Tallman, New York 10982

Sam Golob is a Senior Systems Programmer.  He also participates in
library tours and book signings with his wife, author Courtney Taylor.
Sam can be contacted at sbgolob@cbttape.org.  Information about the CBT
MVS Tapes can be found on the web, at http://www.cbttape.org.


MODULARIZING AN MVS SYSTEM - PART 1

      Today I want to start a subject that should be an essential part
of the MVS systems programmer's knowledge.  The idea is to know which
parts of an MVS system can be separated from the other parts on DASD,
and which parts can't be.  Another way to put it, is to call it "MVS
DASD planning for the system components themselves".  There are many
subtleties to this knowledge.  As soon as you think you "know
something", you get caught by something else.  The proof of success is
to remove the "non-essential" disk pack, IPL the system, and see that
nothing breaks.  Most often, something does.

      The idea of MVS system DASD planning is to make it possible to
divide the "system" disk packs of an MVS system so some disk packs can
run part of the system while the other disk packs are not present.
Although you can run an MVS system in a mish-mash, with all the
components scattered on all the system disks, it is more prudent to set
up MVS so that its different components and add-ons are physically
separated and the system disks are modularized as much as possible.
There are many reasons for that, but one reason is that when you have
your MVS system "modularized", it makes the shop's backup and recovery
scenarios very much easier.  Another advantage is that migration to new
releases of MVS and its components is also very much simplified.

      The capability and experience to set this up, while not being an
absolutely essential part of the new sysprog's duties, becomes extremely
important as one's MVS responsibilities grow.  A senior MVS person must
be very knowledgeable in this area.  I feel that this ability is an
absolute requirement, to be considered a "senior" MVS practitioner.

      This is not to say that a less experienced sysprog can't do a
very good job of organizing the DASD pack structure of an MVS system.
It'll just take a lot more effort, and it will involve making a lot more
mistakes, but that learning process is extremely valuable.  And such
experience will make a junior person more "senior" in much less time.

      Why is this subject so difficult?  (Try it and see!)  The reason
is that in order to physically separate certain parts of MVS from other
parts, you simply have to know the minimum of what is necessary to make
each piece of MVS run, and sometimes you need considerably more than
that minimum to satisfy your shop's needs.  It is very helpful to know
how to set up each of the MVS system parts, and this knowledge only
comes from much exposure to MVS.  A senior-level person can make much
shorter work of doing system DASD planning, because a senior person has
already acquired the knowledge of what each component needs, to be set
up to run properly.

      Many good MVS people do not have direct exposure to what I'll term
"the shop's DASD organization", because they are working in an already
established place, where somebody else did the initial planning.  But
the truth is that when a shop is older, and a lot of people have worked
there over the years, some sloppiness often tends to creep into the
installing of new products.  You might find that many of the disk packs
running the actual production MVS system are too dependent on each
other.  Somebody installing a new software package may have scattered
its various pieces among too many separate disk packs, and therefore
those packs are now hard to separate.  If you try to remove one pack
from the configuration, you'll find that one or more products might
suddenly break.

      I feel that this subject is too big to cover in one month's
column, so we'll start it here, and we'll continue speaking about it
next month as well.  Let's begin.

      I believe that a very good shortcut to learning MVS DASD
organization skills, is to learn about the handy disaster recovery
tool that has come to be known as an "MVS Rescue System".


THE "MVS RESCUE SYSTEM"

      Another term for an "MVS Rescue System" is a "one-pack IPL-able
MVS sysres", although actually such a system could occupy more than one
physical or logical disk pack.  The idea of the one-pack system is to
have a compact, easily backed-up MVS system that can be used to rescue a
broken production MVS system easily.  Setting up a minimal MVS system
yourself, is probably the easiest way to start getting a handle on
understanding the complex structure of your big MVS production system.

      How does the setting up of a one-pack rescue system help you to
learn DASD dataset organization?  It's simple.  If you have to set up
all the ingredients of an MVS component on one disk pack, you'll be
forced to learn about all the ingredients necessary for that component
to work.  And once you've seen the ingredients of that component on the
rescue pack, you'll be able to take a new look at the production system
and recognize all those corresponding pieces of that component there,
too.

      Now let's talk about the rescue system itself.  First, we have to
ask the question about why we should have one in the first place.
Putting it another way, we can ask:  "Why it is handy, and sometimes
downright essential, to have an easily IPL-able skeleton MVS system
always available?"  To grasp the answer, I have to talk about a
peculiarity of MVS.  That is, you can't build a new MVS system without
first having an old MVS system to start with.

      Sorry about that.  The fact is that it takes an already running
MVS system to fix or build another MVS system.  So if you can't IPL your
production MVS system, you can't easily fix it.  You need to have
another MVS system available to access the datasets on the broken
system's disk packs.  So if a shop has only one working MVS system
available, it is vulnerable to extreme disaster.  You need to have at
least two working MVS systems in your shop, to be reasonably safe, and
it is far better to have several more than that.

      So most shops run with at least an "MVS Production system" and an
"MVS Test system" or "the A-system" and "the B-system" which they
alternate.  But the extra one-pack system which can fix either of these,
is an extra measure of protection, and it is even better if the one pack
MVS sysres has been backed up several times with DFDSS or FDR, to tape.
After the unthinkable, you might just have to make a standalone restore
of your "rescue system", so you can start bootstrapping the rest of your
MVS installation back up.  At least you'll have a good TSO to be working
with.

      Therefore, if your shop doesn't already have one of these mini-MVS
systems, it would be prudent for you to make one.  Besides the obvious
advantage of having a shop that is better prepared to face emergencies,
you would also be getting some very essential experience while doing
this.  That's because the act of making a one-pack MVS system would give
you exactly the type of exercise necessary for doing MVS system "DASD
planning".  Putting all the essential parts of MVS in one place, will
make you become very aware of "what belongs where".  And in a larger
system like the production system, which you'll work on later, you'll be
able to physically separate your MVS system into several smaller pieces
without any unwanted dependencies remaining.


WHAT BELONGS ON A RESCUE SYSTEM?

      When you build an MVS rescue system, you have a conflict.  Ideally
you'd like everything to be there that's in your production system, but
it just doesn't fit, so you have to plan what to leave out, and what to
make smaller.  It essentially boils down to having enough of the system
available, so you can fix another system or run a piece of the
production if really necessary.  Almost always, you don't use it for
production.  You just use it to fix the actual production system, and
then you run your production jobs on the real production system.

      As for my personal preference, I'd say that on my own rescue
system, I want to have the full complement of my MVS system analysis and
data manipulation tools already installed.  In other words, I personally
would not be satisfied with only the vanilla IBM tools being installed
there.  I want my personal tools there too.

      What are they?  For myself, I want Fullscreen ZAP and the PDS
command package, and TAPEMAP, TAPESCAN, CDSCB, SHOWMVS, MXI, together
with a host of others, already installed.  These can fit on a couple of
load libraries, and in a few panel, table and skeleton libraries.  All
together, they do not take up a lot of DASD space, and they will all fit
on my rescue pack.  There should also be an available TSO logon proc
that will help me access all these tools easily.  I use a logon proc
with an authorized STEPLIB and a logon CLIST that gives me instant
access to all of my TSO tools, so I don't have to grope for them.
Usually when I'm actually using my MVS rescue pack, I am in the middle
of fixing some problem, so I need my tools.

      But the main story is about how to copy or initialize the actual
MVS components on this pack, so it will actually IPL all the way up.
Many people have created jobstreams to do that work, but for
definiteness, I'll refer you to the CBT Tape File 657, where Kevin Mitts
has a good set of jobstreams that have been tried on z/OS 1.4.  You can
access the pds containing these jobstreams at:

  http://www.cbttape.org/ftp/cbt/CBT657.zip    or possibly
  http://www.cbttape.org/ftp/updates/CBT657.zip

      When MVS initializes, it has to refer to a bootstrap program, and
that is loaded into track 0 of the new one-pack MVS system residence
pack.  You need an ICKDSF job for that, and this job is Kevin's first,
in member RESCUE01 of his pds.  IBM ships the object decks of the
current release of its bootstrap programs in SYS1.SAMPLIB of the MVS
system.  Next, you need SYS1.IPLPARM with a LOADxx member initialized.
Or alternatively, just having SYS1.PARMLIB with the LOADxx member will
do.  Kevin's second job in member RESCUE02 of his pds, allocates all the
non-VSAM system libraries, including SYS1.PARMLIB.  SYS1.PARMLIB will be
loaded with the minimal essential materials in a later jobstream.  Then
of course, the RESCUE pack will need an MVS master catalog.  This is
created in member RESCUE03 as a usercatalog of the driving system, and
it will be disconnected later.

      Then you have to copy all the members of the MVS system libraries
from your production system onto the corresponding system libraries on
the rescue pack.  This is in member RESCUE04.  The job uses ADRDSSU, and
it is an efficient way to get all the copying done.   Next, you need to
define the PLPA, COMMON, and LOCAL page spaces on the rescue pack.  This
is done by the job in member RESCUE05.  Then you have to catalog all the
nonVSAM system libraries you created earlier, into your new MVS master
catalog, so the IPL of the RESCUE system will later find them.  This is
done in member RESCUE06.

      Next, an IEBUPDTE job is run on the driving system, to load the
new SYS1.PARMLIB of the rescue system with its necessary members.
Essential PROCLIB members are loaded similarly into the new SYS1.PROCLIB
by job RESCUE08.  To get TSO on the new system, you'll need VTAM, so the
new SYS1.VTAMLST library is loaded using IEBUPDTE by job RESCUE09.  Job
RESCUE10 assembles a couple of necessary RACF customization modules,
ICHRDSNT and ICHRIN03, as well as creating the RACF database itself.
Job RESCUE11 runs TSO ACCOUNT to create the IBMUSER TSO userid, and it
also initializes SYS1.LOGREC.  Job RESCUE12 copies over the old IODF
dataset to the rescue pack, so you have an I/O device configuration
defined there.  RESCUE13 is a job to EXPORT DISCONNECT the new rescue
MVS master catalog from the driving MVS system.  And finally RESCUE14
creates a tape, so you can do a standalone ICKDSF, a standalone DFDSS,
and you have a DFDSS backup of the entire new RESCUE pack.

      Obviously, you can modify all these jobs in any way you want, to
create an MVS rescue system that is either bigger, or is more practical
for your own use.  But it's nice to have a starting set of jobs that
work, with parameters that have already been tested.  As we said before,
just going through this exercise of customizing Kevin Mitts' jobs (or
somebody else's) to create a working MVS skeleton system, is of immense
value for learning the MVS system DASD structure.  And as we will
continue to explain next time, these skills will help you very much, in
your efforts to get your shop's MVS DASD organization under better
control.  Until then, I wish you much success and an excellent month.
See you soon!