SHARING THE LOAD (c) Copyright 2010 by Sam Golob. All rights reserved. INTRODUCTION - COLLECTING YOUR OWN FREE SOFTWARE TOOLS Many years ago, when I broke into this field, there existed a considerable number of (then) grizzled (read "40-year-old") veterans who had broken in 10 or more years earlier, during or before the mid-60's. These folks were very schooled in Assembler language, because (number one) they had gone to school for it, and (number two) not many shops were then programming in higher level languages, even in COBOL. So you had to be fluent in Assembler if you would be doing any programming at all, even applications programming. The "systems types" among these people were still around in relatively large numbers when I broke in. After I found out about the CBT Tape and other free software collections (belatedly in 1985), I asked some of these people if they would like to have copies of the tapes for themselves. Many of them answered (to me they seemed rather cocky) that if they needed any tools, they could write their own whenever they needed them, so they did not have to acquire someone else's free tools. I do not like to disparage any opinions, because I generally have a lot of respect for people, especially knowledgeable people. But upon hearing these statements, I did smell a rat. Were these people really that good? Did they really know that much? Would the tools they came up with (assuming that they actually did write them) be good tools for EVERYONE to use? Right now, I'm here to express my doubts, and to add my own opinions on that subject (25 years later). I also might add another thing. The MVS (aka z/OS or whatever) system nowadays is more complicated than it used to be. I have a copy of an OS/360 Principles of Operation manual, and it is barely more than 1/2 inch thick. That's a laugh nowadays. But IBM has always been very good about upward compatibility, and some 40-year-old Assembler programs written for those systems, still work today. Nevertheless, IBM has been adding on more and more components to z/OS. Doing the requisite research to write a tool, while still being quite straightforward in many cases, can often be more time-consuming than it used to be. Therefore, it isn't easy to be a jack-of-all-trades (i.e. "I'll write anything I want") person today. Truth be told, I think there really existed a VERY small group of people who could live up to these claims. I heard of one man who was contracted to produce a shared SPOOL package (before one was available from IBM), and he produced 7000 lines of error-free code in one weekend (which ran smoothly for 12 years at that installation). This is a truly phenomenal feat, and he really did it (in the one weekend). But most people (even very very good ones) are more ordinary. Especially nowadays, when most of us are not so well-trained in Assembler, and the system is much more complicated, this is very much the exception, and not at all, the rule. So I think that if most people would try and write a tool, quickly, at the time they needed it, most of these tools would probably have several shortcomings. WHAT CAN GO WRONG WHILE WHIPPING UP A TOOL FOR YOURSELF? It's easy to throw a bit, but are you throwing the RIGHT bit, ALL the time? If you write a tool for yourself, and you assume you are going to be the only user of it, then (usually) you ALSO assume that the tool is not going to be used incorrectly. For example, if the program needs a PARM, you might assume that the parm is going to be the correct one, the one you planned on using. You sometimes don't program for the person who will make a mistake with the PARM field, for example, making it non-numeric when it is supposed to be numeric, or making the number too large or too small. So if you write for yourself only, you might leave out checks for the following things, and many more. 1. You might let the system ABEND if there is a mistake, instead of trapping the mistakes and putting out an error message that advises the user how to correct the mistakes. 2. You might not test to see if the user is APF-authorized where the tool requires it. It is better to test for authorization with the TESTAUTH macro and intercept the errors. 3. If the tool affects system storage, it might not ABEND, but it could possibly alter the wrong storage and damage the system dangerously, when it is not used properly, the way it was planned by the writer. 4. You might not test for incorrect data. 5. You might not see that your code is accessing incorrect data. 6. And so forth...... When you're writing a tool for others, especially for others who might bumble, you must add safeguards to your tools, and safeguards mean that your tools will get more complicated, harder to test, and you can't just whip one up in five minutes when you feel you need it. You might need to spend considerable time testing the tool, to make sure it does not damage anything. After all, IT IS YOUR JOB to keep the installation running safely, and staying up. Sometimes a bad tool is way worse than having no tool at all. The bottom line is that when you are writing a tool that affects the system, you must do it CAREFULLY, just like when you are driving a car. You don't want to kill or hurt, anything or anybody. This takes time, patience, and a lot of testing. It usually cannot be done in five minutes, in an emergency, only at the time you need it. SHARING THE LOAD I think I've come up with a better plan. So now it makes sense that if someone already wrote a tool, carefully, lovingly, and with a lot of testing and trial, it just might pay FOR OTHER PEOPLE to use that tool profitably as well. Therefore, IF A LOT OF PEOPLE will EACH write a FEW tools, but they do a good job at them, and they SHARE their work with other people, and each one profits from the careful work of the others, then a lot of ground can be covered. That is what we are about. OPEN-SOURCE PRODUCTS VERSUS COMMERCIAL PRODUCTS I have been convinced that open-source products are often (but not always) better than commercial products that do the same thing. As usual, the answer to that question is: "It depends." If the commercial software house is big, generous-minded, and they afford ample resources for their developers, and they aren't restrictive, money-wise or policy wise, and they practice firm but pleasant quality control, then it is possible that they can produce a top-notch product. (Not so many of them are that good. Some are.) But if the commercial software house has constraints in money, time, or mind-set of management, there can be an inflexibility in that place, which effectively paralyzes development and progress. For example, I've seen an open-source project whose product surpassed and exceeded its commercial counterpart in just a few years' time, overcoming the commercial product's ten-year head start in just 3 or 4 years. It's quite true. I've used both of these myself. The free product wins the contest, hands down! If you've ever worked for IBM, "at IBM", or at a commercial software house, you'll know something about why this can be so. Being "inside" is far different than "looking in from the outside". As with most things in life, there are many sides to the matter, and (depending on the circumstances) the quality issues can go either way. Either the open-source product is better, or the commercial product is better. Sometimes they both can be equally good. Now let's look at some of the factors which affect the quality of commercial products vis-a-vis the quality of open source products. "PROFESSIONAL" VS "DOING IT FOR LOVE" A professional person, in whatever field, has to develop a pride and expertise and self-respect. After all, he or she has to warrant the salary and professional respect which is given to them, in exchange for the quality of their work. A professional cannot be slipshod. A professional person must produce a quality product. A professional person's work must warrant the the salary and benefits that are paid to him or her. But in any company, money (or the lack of it) talks, and money matters dictate much of the eventual result. The professional programmer is often tied down by management, and can't bring to fruition all of the creativity that is in his our her soul. On the other hand, open source projects are made by people with "a passion for the project". They don't get paid. But they are driven by a passion for the result, and sometimes, by the pride of being members of the team that produces the result. If someone is enthusiastic to make an enhancement, no one stops him or her because of monetary or time constraints. The person writes the implementation, with all the bells or whistles desired. Other people in the group make suggestions, the component is tested, and it is integrated into the whole product. Nothing is left out because some manager thinks so, or because of a monetary or time constraint. The resulting product is much richer and more robust because of that. And it gets done sooner. I have produced both free products and commercial products. If you haven't produced a commercial product, you won't realize how much testing has to be done. The product has to be foolproof, as much as possible, because it is guaranteed in writing, and money is being paid for it. IT HAD BETTER BE CORRECT and function properly. There are legal matters involved. Nevertheless, I've still seen open-source products that are better, by far, than their commercial counterparts. The reason is because of the lack of restriction due to monetary constraints, or the manager's ego, or legal factors, or for reasons like the company doesn't want to conflict with something that another of its acquired products does, and so forth. As always, "it depends". SO WHAT'S IN IT FOR US? Finally, we are coming to the point of this article. It is now possible for everyone to accumulate large collections of free tools and to learn how to deploy them and use them. You don't have to write them all, yourself. You can take full advantage of other people's knowledge, you have their blessing and encouragement, and you can use their tools, too. So if you have happened to have a particular expertise or need, and you have written your own tools to cope with that need, then you can contribute to the public good and share all the tools, yours and the others', for your own use and for everyone else's. That's how we all benefit. No ONE of us has time to do EVERYTHING, but ALL of us TOGETHER have had time to do A LOT. And that is how it all gets done. For example, my particular expertise, where I had to spend a lot of time, is with magnetic tapes and their AWS and FLEX tape images, the SYS1.BRODCAST dataset, and TSO control blocks (many of them undocumented outside of IBM) having to do with the authorization of commands and programs under TSO. When I was doing my "TSO Authorization Table" work, I noticed that in the SHOWzOS monitor (CBT Tape File 492), the coverage for the TSO control blocks was not adequate. So since that was my specialty, I beefed up the TSO Auth Table coverage there, and I added the coverage for the number of Global Notices that are made, when you do an ACCOUNT/SYNC to format a new Broadcast Dataset. When I was doing my magnetic tape work, I converted all of the tape mapping and tape copying programs on the CBT Tape, to look at 64K blocks instead of 32K blocks, whenever I could. (There was one exception - in that case, I couldn't penetrate the complexities of his BSAM code.) Therefore as a result, everyone else could benefit from what I was spending MY time on. One major vehicle for sharing is the free "CBT Tape" collection of tools which you can access (no userid or password needed) at www.cbttape.org. You can use the "already developed and tested" tools that are there, as well as contributing your own. I have found that many of our system programming jobs are done more easily using these tools, which were made "for the sysprogs, by the sysprogs". Everyone has their own style and expertise, making for a large diversity of subjects and MVS components covered. www.cbttape.org now has support rights for most of the Xephon code from the Xephon magazines, as well. Now you can use, and get ideas from, Xephon magazine code, without feeling as if you've violated a whole bunch of copyrights. BTW, please read the Disclaimer at the www.cbttape.org web site. But notwithstanding the disclaimer, most of the code has been tested a lot. I myself have bounced around to a bunch of shops, and I can testify that having my own set of tools, ready to go, and not needing ANY licensing, really enhances my ability to get real work done on a moment's notice, without elaborate setting up. For a lot of my work, I don't have to rely on any vendor's tools at all, except for IBM's. I can mostly make up for IBM's inadequacies with my collection of FREE tools. And as for most of my tools, I didn't write them myself (though I HAVE written SOME of them). The ones I have written, I've tried to take my time with, and do a thorough job of testing. I didn't write them in five minutes. My own feeling is that I am very happy to get help wherever I can find it. And I try to lend a hand to others, whenever they need it. Collecting free software encourages cooperation among folks. It makes me happy, and it makes many other people happy too. Feelings count for a lot in life! I just have to note here, that most of my professional friends did not work in the same shop with me. Therefore, there was no chance of rivalry for the boss' attention, backbiting, or petty jealously. Everybody was out for the other guy's good, and the other guy's good ONLY! That's the way it SHOULD be, and that's the way it is. So to conclude, I think that in today's climate, it's best to pool our knowledge and our research and development skills to create a few good MVS tools and share them with a lot of other people. We can't, and shouldn't, try and write everything ourselves. We should use other people's work too. That's the best way to get all our own work done, quickly, efficiently, and effectively.