Introduction to
S/370 Principles of Operation



In terms of our introduction to POPs, very little of Chapter Four, Control, is of interest. Most of the contents of this chapter deal with more advanced concepts. The bulk of the facilities discussed in this chapter are only used by the authors of an operating system, such as MVS.

However, there are a few fundamental concepts which we can benefit from learning, such as the PSW and to a lesser extent Control Registers.

Here are the main sections of this chapter:

  • Stopped, Operating, Load, and Check-Stop States
  • Program-Status Word
  • Control Registers
  • DAS Tracing
  • Program-Event Recording
  • Direct Control
  • Timing
  • Externally Initiated Functions
  • Multiprocessing
  • CPU Signaling and Response
  • Channel-Set Switching

  • This chapter describes in detail the facilities for controlling, measuring, and recording the operation of one or more CPUs.

    CPU states

    As the "STOPPED, OPERATING, LOAD, AND CHECK-STOP STATES" section discusses, each CPU can be in only one of these four states at a time. Go ahead and read as much of this section as you can; it's short and you can skim the parts that don't make sense at this stage. However, it's important that you make this effort, as it directly bears on the quality of the "mental model" you have of the S/370 CPU.

    In terms of operating the Hercules emulator, the stop command places the CPU in the stopped state (normally just called "stopped").

    The Hercules start command normally transitions the CPU from the stopped state to the operating state (normally just called "running").

    The Hercules ipl command places the CPU in the load state (normally called "an IPL", an abbreviation of Initial Program Load).

    The final state, CHECK-STOP state, will normally never be entered by the Hercules emulator. This state is normally entered when the CPU detects certain conditions relating to the system hardware.


    This section discusses the PSW (Program Status Word). As you might recall from our storage discussion, a word is four bytes; yet the PSW is eight bytes. So perhaps our Imaginary S/370 Design might have some grain of truth; maybe the PSW started out as four bytes. Regardless of how the PSW's contents evolved, it's called what it's called: the PSW.

    Each CPU installed in the S/370 system has its own PSW, from which you might gather each CPU can be executing different programs located in main memory. When we get to Chapter 5, Program Execution, we'll see just how that happens.

    Before we get too far into this section, let's indulge in a little history of the PSW. When IBM introduced the S/360 architecture (the S/370's predecessor), the PSW didn't have a "mode"; it was just the PSW. Each of the bits in the PSW always meant one and only one thing. When IBM began designing the S/370, it wanted to maintain as much backwards compatibility as possible but still be able to enhance the S/370 system.

    The method they chose to perform this enhancement was to recycle the definition of one of the bits in the PSW. IBM polled their customer base to determine which PSW bits were being used, and bit 12 turned out to be lightly referenced, hence redefining the bit disturbed the fewest IBM customers.

    Here's a portion of the text from the S/360 Principles of Operation for PSW bit 12:

    ASCII(A): When bit 12 of the PSW is one, the codes preferred for the USASCII-8 code are generated for decimal results. When bit 12 is zero, the codes preferred for the extended binary-coded-decimal interchange codes are generated.

    Once IBM selected the PSW bit, they redefined it to specify the mode in which the PSW operated: BC (Basic-Control, for S/360 operating systems), and EC (Extended-Control, for S/370 operating systems). The PSW format is decidedly different between the two modes. Unless you plan on writing S/360 operating system level code, you can immediately forget about BC mode PSWs.

    So now that you know the historical background of the two PSW modes, read the PROGRAM-STATUS WORD section, skimming the parts that don't make sense.

    We'll delve into the complete contents of the EC PSW a bit later, but for now we'll just show how it's laid out and note a few of the individual fields. As usual, the bits are numbered using S/370 notation; since two decimal digits are required to number some of the bits, when a two digit number is shown, it refers to the left-most bit of the nibble.

    0R00 0TIE KKKK CMWP 00CC MMMM 0000 0000
    0123 4567 8    12   16   20   24   28
    32   36   40   44   48   52   56   60

    Bits 0-7 represent the "system mask", which the operating system sets depending on whether it wishes to recognize certain classes of interrupts. There's also a bit here which controls whether the CPU should map virtual addresses to real addresses (T for Translation). Virtual and real addreses are a subject for another time; as we begin learning S/370 assembler, we'll almost always be dealing with virtual addresses.

    Bits 8-11 represent the storage key, which the operating system sets in such a manner to protect the system and users from altering each other's memory. Storage key zero has a special meaning.

    Bits 12-15 is a hodge-podge of bits for various purposes, and is probably the most interesting set of bits in the PSW.

    Bits 16-17 are binary zeroes.

    Bits 18-19 are the Condition Code, which we recall the SLR instruction sets. The condition code describes the result of some of the S/370 instructions. The REFSUM identifies these instructions with (c) to flag this attribute.

    Bits 20-23 represent the "program mask", which the user program can set to control whether certain categories of exceptional circumstances cause the program to be interrupted. In essence, the settings of these bits describe how well prepared the user program is to handle certain unusual results.

    Bits 24-39 are binary zeroes; when IBM learns a lesson about leaving reserved bits in the PSW, it really learns its lesson.

    Bits 40-63 are the Instruction Address, which is the byte address of the beginning of the next instruction which the CPU is to execute when in the operating state. Many programmers get confused here, so remember it's the NEXT instruction.

    Before we leave the PSW, let's briefly examine those interesting bits 12-15.

    Bit 12 (C), as we said, was recycled from S/360 to indicate whether the PSW is in BC or EC mode; an EC mode PSW has bit 12 set to one. Our level of MVS will always run in EC mode.

    Bit 13 (M), when one, allows the CPU to recognize machine check interruptions. We'll say almost nothing about machine checks since you'll never really have to deal with them (and the MVS source we have is missing pieces of the code so it's a bit difficult to understand what's happening there). Suffice it to say that a machine check occurs when the hardware detects a problem with the hardware. Most supplemental storage bits are set to binary one to indicate "enabled", for those facilities which can be programmatically controlled.

    Bit 14 (W) controls whether the CPU will execute instructions, or wait. When this bit is one, the CPU is waiting rather than executing instructions. Exactly what is the CPU waiting for? An interrupt. The S/370 architecture uses interrupts to indicate a change of state somewhere in the system; in the case of one special instruction (the SVC instruction for SuperVisor Call), the interrupt frequently indicates a request for a change of state, such as from problem state to supervisor state, so the operating system can provide some service that requires greater privilege than the user possesses.

    Bit 15 (P) is a system programmer's favorite bit in the PSW. It indicates whether the CPU is in the supervisor or problem program state.

    Finally, no discussion of the PSW would be complete without mentioning another pivotal instruction, the Load PSW instruction (LPSW). The LPSW instruction is used by the operating system, or user extensions thereof, to replace the entire contents of the PSW with new contents. This one instruction allows a program to atomically update the bulk of the system's status. For this reason, the LPSW instruction requires supervisor state.

    Curiously (at least to me), there is no Store PSW instruction.

    Instead, there are two alternative methods which provide a similar ability: instructions and interrupts. There are privileged instructions (instructions that require supervisor state) that can modify, and in some cases retrieve, bits in the PSW.

    The other mechanism, interrupts, stores the entire PSW in an architecturally-defined storage location then loads an entirely new PSW from another architecturally-defined storage location. Sometimes this activity is called "swapping PSWs".

    If you want to get the most out of the PROGRAM-STATUS WORD section of POPs, you might want to re-read it now that you've read this part of the tutorial. Understanding the PSW is one of the key areas that divides the real assembler programmer from the casual assembler programmer.


    Just as there are sixteen GPRs (General Purpose Registers), so are there sixteen CRs (Control Registers). Unlike GPRs, CRs are highly specialized; you cannot perform mathematical or logical operations upon them from your program. However, the CPU itself can internally use them in strictly limited operations based on the individual control register definition.

    At this early stage of understanding, we'll have very little exposure to control registers. At the level of MVS we're using, manipulating a control register requires supervisor state, and is almost always only done by the operating system itself. Higher levels of MVS support the Program Call (PC) instruction, which POPs reminds me can also manipulate control register contents. POPs is like that; just when you think you've got something conceptualized, it reminds you about something you've either never heard of, or forgotten.

    We'll pause briefly here to mention just one of the CRs: CR1. This CR is pivotal in causing the S/370 hardware to recognize virtual storage addresses. Sometimes CR1 is called the Segment Table Origin Register (STOR), because part of CR1 points at a data area defined by the architecture that describes virtual storage (the Segment Table). MVS saves the value to be placed in CR1 in a field (ASCBSTOR) in each user's Address Space Control Block (ASCB); this ASCB is responsible for representing each users' Address Space. We'll say more about Address Spaces when we look more closely at how storage really works in a S/370 system; for now, think of an Address Space (AS) as a separate process in MVS.

    Go ahead and read the CONTROL REGISTERS section; as always, skimming it when you reach something you don't understand. Just this little bit of reading will give you an excellent introduction into some of the many advanced functions the S/370 provides, even if you don't quite understand what they all are, or what they do.


    DAS Tracing is obviously tracing DAS. However, it's less than obvious what DAS is or does. DAS stands for Dual Address Space, which is a technique the S/370 offers that allows programs the ability to manipulate storage in more than one Address Space.

    This is an advanced technique, which our level of MVS doesn't really support. Really, really clever programmers like Greg Price and a few others have figured out ways to use DAS in our level of MVS, but it's akin to magic. These hardy souls have had to directly manipulate the Control Registers to do it, something not usually contemplated except by hardcore assembler programmers.

    My advice is that beginning assembler programmers skip this section entirely.


    Program Event Recording (abbreviated PER) is an awesome debugging facility. The VM operating system began offering a PER command somewhere back in the early 1980s, and it really allows you to slice and dice a S/370 virtual machine.

    It wasn't too long before the MVS developers picked up on PER, and the MVS operator command SLIP provides somewhat analogous features.

    If you want to get a bit of exposure to PER, read this section.


    The S/370 Direct Control facilities are provided for systems that contain more than one CPU, called multi-cpu systems. Read the section if you want, or skip it.


    There are four timing facilities in a S/370 system:

  • Time Of Day (TOD) clock
  • Clock comparator
  • CPU timer
  • Interval timer
  • Of these facilities, the one most programmers will encounter is the TOD clock. Most frequently, they will store the TOD clock into main storage using the Store Clock (STCK) instruction, to determine the current time.

    Many Hercules users specify the SYSEPOCH 1928 command in their configuration file to keep the TOD clock from exposing Y2K bugs in older S/370 code.

    Glance through this section, but it will be some time before you need to know the material.


    The authors of POPs chose to lump three subjects into this section: resets, IPL, and Store Status. I won't mind if beginning assembler programmers skip the reset portions.

    Initial Program Loading is a fascinating subject, which really deserves its own tutorial. It's what happens when you issue the Hercules ipl command, which starts the MVS system. Unfortunately, you're not going to understand much of this section since it deals quite a bit with I/O, which we haven't covered yet. Read and skim.

    Store Status is provided by the Hercules store command, and it is usually only used immediately before taking a standalone dump to capture the current contents of the PSW, timers, and registers into low storage. It's a short section, take a quick glance.


    It means more than one CPU. I don't have anything to say about it right now. Skim it if you like.


    I don't have anything to say about it right now. Skim it if you like.


    It's optional I/O stuff, which not very many S/370s used. I don't have anything to say about it right now. Skim it if you like.