The Computer Journal, Issue 38
Z-System Corner
© Jay Sage
Reproduced with permission of author and publisher.
True, February is a short month, but somehow I don't think that explains why the beginning of February (the due date for this column) came quicker than usual. It is probably a good thing that I have convinced several others to start contributing regular columns to TCJ. That way, Art Carlson may be busy enough not to notice that the deadline passed without my column. Because it is late and because we now have quite a few prolific writers joining the TCJ ranks, I'm going to keep my column shorter than usual this time (I know I have said that before, but this time I think it really will be true).
For this issue I will be catching up on some correspondence, informing you of the imminent release of the first high level language for Z-System programming, bringing you up to date on Z-Node developments (there is a new Z-Node Central), and beginning a discussion of issues related to bringing up a remote access system (so that more of you might decide to set up a Z-Node).
Letters
Despite our requests for letters with your comments and suggestions, we don't get very many. Recently, however, Art forwarded to me two lengthy and thoughtful letters from James Ott. I would like to begin by addressing some of his comments and questions.
The ZCPR33 Programmer's Reference Manual
First, Mr. Ott asked about the "Programmer's Reference Manual" to which I made reference in my ZCPR33 User's Guide. Well, the truth is that after writing the user's guide, I really didn't have energy left for another major manual. Instead, I started to release programming notes one at a time as files on the Z-Nodes. Even at that, I only got to three of them. The files have names of the form Z33PNOTE.###, where "###" is a sequence number. Note 001 deals with the command status flag in the message buffer and the extensions in its use introduced with ZCPR33. Note 002 discusses proper coding techniques for shells under ZCPR33 and later. The third note covers the parsing of files by the Z33 and Z34 command processors. I am tempted to reproduce some of that material here, but then I would surely fail in my resolve to keep this column to a reasonable length. So you will just have to look for them at the Z-Node in your neighborhood. If it is not there, see if the sysop will get a copy from Z-Node #3.
Shells and ZEX
Mr. Ott continued: "The User's Guide mentions an addition to shell coding necessary to ensure the shell pushes its name onto the shell stack when it is called by ZEX." I think there is some misunderstanding here. It was under previous versions of ZCPR3 that special code was required in shells to deal with ZEX (and it never had to do with pushing anything onto the shell stack). Under Z33 and later, this code can (and, to make the programs smaller, should) be removed. It has now been quite some time since the release of ZCPR33, and I think that most shells are now coded in the more efficient way.
As we have discussed in previous columns, a shell command comes into play when the command line buffer becomes empty. In that case, if there is a command on the shell stack, the command processor invokes that command instead of asking the user for the next command line. In this way, the shell takes over the function of the command processor.
ZEX's function is similar to that of SUBMIT and XSUB together. While SUBMIT stores its script data in a disk file, ZEX keeps it in memory. This enables ZEX to run much faster, but it reduces the memory available to programs. A simple SUBMIT script feeds a series of commands to the command processor, thus doing under CP/M what the multiple command line of ZCPR does. SUBMIT can be useful even under ZCPR, however, because it can supply a longer string of commands than could fit into the command line buffer.
XSUB running under SUBMIT allows characters in the script to be fed not only to the command processor but also to programs as they run. This is what is called input/output (I/O) redirection. In this case it is input redirection; the operating system is made to take its characters not from its normal source - the keyboard - but from a disk file (SUBMIT) or a memory buffer (ZEX). This is both extraordinarily useful and extraordinarily tricky. Long ago I promised to discuss this subject at length in this column, but I have never gotten around to it. Bridger Mitchell, Joe Wright, and I are now engaged in a joint project to perform a major upgrading of ZEX, and I am sure it will be the subject of TCJ columns by one or more of us.
The ZCPR33 command processor observes the following hierarchy for acquiring new commands, where step 1 has the highest priority and step 5 the lowest. The way this hierarchy functions is described in more detail in the ZCPR33 User's Guide.
- Commands pending in the multiple command line buffer
- Commands from a ZEX script
- Commands from a SUBMIT script
- A shell command
- User input
Under ZCPR30, ZEX did not appear explicitly in this hierarchy; it came into play only by virtue of its ability to redirect input at step 5. This posed a serious problem when a ZEX command was issued under a shell. Although the user intended the script to be performed as commands, the shell would take it as input to the shell instead. To avoid this, rather lengthy code had to be included in every shell to see if ZEX was running and, if so, to feed its input directly to the multiple command line buffer. This resulted in completely useless loads of the shell code for each line of the ZEX script. Execution was so slow and annoying that for all practical purposes ZEX scripts could not be run under shells.
The ZCPR33 command processor was redesigned to deal with ZEX explicitly just as ZCPR30 always did with SUBMIT. ZEX was placed above SUBMIT in the hierarchy so that ZEX scripts would function like arbitrarily long command lines and could be invoked from SUBMIT scripts.
The New Libraries
Mr. Ott had several questions about the assembly-language libraries that support the Z-System. I will not comment extensively here, but I would like to announce that new versions of the libraries, which have been in use by a number of us for many months already, will soon be made available to the public. I had expected them to be a commercial product, but, for several reasons, it now appears that the code will be made available at no charge. Only the manual, which will be quite a large book, will be sold. This is good news.
One of Mr. Ott's suggestions was that the Z3LIB routine called PRGLOAD, which is used for chaining from one program to another, be updated to allow for type-3 programs. This probably could be done without a great deal of difficulty, but I am not sure that it is worth the trouble. What about type-4 programs? Loading them is much more complex because of the need to compute the relocation to a run-time address. It seems to me that a better way for programs to chain to other programs is via the multiple command line buffer. If anyone can suggest any advantages of a direct load, I would be interested in hearing them.
The Command Line Tail
One of the mistakes in ZCPR30 was its failure to check for command line tails that would not fit in the buffer from 80H to FFH. CP/M had no problem with this because the whole command line was not long enough to allow this to occur. With a 200-or-more-character command line buffer, there is nothing to stop a user from entering a command with a tail longer than 128 bytes. Under Z30 this caused a catastrophe, because the tail was copied into the buffer after the program was loaded, and then the tail could overwrite the beginning of the code. Z33 and later monitor the length of the tail and stop copying before this can happen.
With type-3 programs that load at addresses higher than 100H, longer tails could be copied without damaging the code, and Mr. Ott requested that this be implemented in future versions of the command processor. I think this would be a very bad idea. One of the reasons for using type-3 programs is so that any code residing at 100H can be run again later using the GO command. If the type-3 program's tail overwrote the TPA, then trouble would occur when the GO command was executed.
Moreover, there is absolutely no need for such a feature. If a program wants to support a tail longer than 126 bytes (it doesn't even have to be a type-3 program), it can simply read its arguments not from the buffer at 80H but directly from the multiple command line buffer. As an aside, I have thought of making the command processor not convert the command line buffer to upper case. Then programs could process lower case input directly (as in MS-DOS) without the special symbols to indicate case shifts as in the ECHO and IF INPUT commands. The command tail buffer at 80H must be converted to upper case for compatibility with CP/M programs, which assume that that will be done. Besides the fact that there might be a significant code penalty (the command tail buffer and the command file control block would both have to be individually case shifted), I worry that there may be a number of Z-System programs that either rely on the command line being in upper case or, worse, convert it to upper case. I'd be interested in hearing any opinions on this topic.
Still More on Z3 vs. Z2 Shells
Mr. Ott sent me a lengthy letter with some interesting examples of the use of the shell stack. I think his is the first response I have received that pointed out an aspect of the shell stack that I had not previously appreciated.
I don't think about shell command lines with more than one command, so the following distinction never occurred to me. One can think of the shell stack as containing not just, say, four commands but rather four groups of commands. The shell stack not only holds these commands; it also provides the mechanism for grouping them, for providing parentheses, if you will. The Z2 approach to shelling would have a very hard time doing this. For example, if the first element on the shell stack contains the command line "CMD1A;CMD1B" and the next lower element the line "CMD2A;CMD2B;CMD2C", the equivalent Z2 situation would have the command line
CMD1A;CMD1B;CMD2A;CMD2B;CMD2C
The SHCTRL POP command removes an entire Z3 shell entry, containing possible multiple commands. How could we implement this function with a Z2-style shell? Even if each command were marked with a "/s" token, one could still not tell which ones were grouped with which. There might be a solution to this problem, but the Z3 shell approach certainly handles it nicely.
Mr. Ott's letter included a very interesting application example. I've made a few changes that, I hope, do not harm its essence. When the computer is booted, the STARTUP alias loads the command sequence "FIRSTSH;MENU", where FIRSTSH is a special utility that places the following command sequence onto the shell stack:
PARK;VID RST.MSG;STOP
When this sequence runs, it will park the heads on the disk drive, display a message to shut off the computer, and run a program called STOP that locks the machine. Of course, this multiple-command shell could easily be replaced by an alias SHUTDOWN.
This termination shell does not run immediately because of the command MENU following it in the command sequence. This loads a second shell onto the shell stack. The user then lives inside the MENU shell for some time. At some point, the user may make a selection that generates the command "CD WORK:". This would log into the WORK: directory and issue an automatic ST command there. The ST alias might have the script
LDR WORK.NDR;SHCTRL POP;ZFILER
This would install a new set of named directories, pop MENU off the shell stack, and replace it with the ZFILER shell. The user could then do some work using ZFILER. From ZFILER, a macro command might make another shell change by popping the shell stack and installing another shell.
The above process would continue until the user made an exit selection. This would pop the shell stack without installing a new shell. As a result, that first line we put on the shell stack would become active, forcing the computer to be shut down in an orderly, preplanned way. Of course, this approach is not totally foolproof if the user has access to the power switch or power cord. But it certainly helps.
Mr. Ott worried that this example could not be achieved using a Z2 shell system. I think it could, though not with quite the same ease and elegance. The STARTUP alias would contain the line
MENU;SHUTDOWN
When MENU ran, it would substitute for itself in the command line the desired command line plus its own reinvocation command with the flag "/S" to mark it as a shell. Thus the command line would become
USERCMD;MENU /S;SHUTDOWN
For the user command CD WORK:, the command buffer would evolve as follows:
CD WORK:;MENU /S;SHUTDOWN
ST;MENU /S;SHUTDOWN
LDR WORK.NDR;SHCTRL POP;ZFILER;MENU /S;SHUTDOWN
When the SHCTRL command ran, it would scan the command line for the next shell command as marked by the "/S" flag and remove it from the command line. This would leave one with
ZFILER;SHUTDOWN
When ZFILER generates a macro command, the command buffer would have
MACROCMD;ZFILER /S;SHUTDOWN
This would continue until one cancelled the shell. Then the SHUTDOWN alias would run.
Mr. Ott was concerned that the command line would grow longer and longer as old shell commands piled up. Indeed, this would happen if there were no 'popping' mechanism. The "/S" marker I proposed in my previous columns is critical here, since without it there would be no way to tell which command to pop. This approach, I think, would fail with aliases as shell commands. The Z3-type shell really helps us in this case.
High-Level-Language Support for Z-System
I have some especially exciting news about the first high level language specially designed for Z-System. In late January, I got a phone call from Leor Zolman, author of BDS C. He was interested in bringing out a new version of his C compiler for the modern 8-bit market. After some consultation with me on what was needed to support Z-System, Leor went to work on the run-time code, and in less than two weeks he had a version ready for beta testing
He came over to my house to demonstrate the result, and I was amazed to see how easily one could write a utility, complete with named-directory support, even including password protection. A little fine-tuning is still needed, but I am already excited about the impact that the availability of this quality high level language will have on Z-System development. Marketing details have not been worked out at this point, but you can expect the new BDS C to be available at an attractive price through Z Systems Associates (ZSA) channels (Plu*Perfect Systems, Sage Microsystems East, Z-Nodes, and Z-Plan user groups).
The New Z-Node Central
Ron Bardarson, who had operated Z-Node Central after David McCord retired as the sysop, decided to change the focus of his system and his node designation from Z-Node to X-Node. This was to indicate its experimental nature and its focus on software and systems development rather than on the distribution of Z-System software. Its number remains 408-432-0821 (CASJO on PC-Pursuit).
To maintain a center of support for Z-System users and for Z-Node sysops, we have established a new Z-Node Central. Richard Jacobson, whose Lillipute Z-Node in Chicago has long been, in my opinion, the premier Z-Node in the country, has agreed to take on this new role. He will continue to use the delightfully ironic Gulliverian name (at over 100 Mbytes, his is the least Lilliputian of the Z-Nodes), but his node number now drops from 15 to 1.
This new function is added to several special services that Lilliputealready provides. It is the official bulletin board and remote access system for both the North American One Eighty Group (NAOG) and TCJ. The full system, comprising two independent computers, is available on a subscription basis. All users who provide registration information will get 15 minutes per day of free access. Unlimited access to both computers (within reason) is available at a rate of $25 for six months or $40 for a full year. TCJ subscribers and NAOG members get free access to limited areas and can upgrade to full subscriber privileges at $5 less than the standard rates. Z-Node sysops will have free access to the full system. The phone numbers are 312-649-1730 and 312-664-1730, accessible via the ILCHI outdial of PC-Pursuit.
System Security Under Z-System
As part of my plan to build up the network of Z-Nodes, I would like to begin a series on issues related to setting up a remote access system (RAS) using Z-System. Many people do not realize it, but NZCOM and Z3PLUS (the automatic Z-Systems for CP/M-2 and CP/M-Plus, respectively) create systems with the full security capability necessary for a RAS. It just takes a few simple operations to engage it.
These issues have been brought to my attention recently because I have been in the process of setting up a second remote access system at my house. This one is for the BOSKUG group of the Boston Computer Society. It will be a two-line system running on a very powerful 16 MHz Kaypro 286 computer with multitasking software. For all this power, however, I was struck by the tremendous complexity of setting up a remote system on such a computer, all because of the lack of a secure operating system.
With MS-DOS, a remote system absolutely cannot allow a caller to gain direct access to the operating system command prompt, because once he has that access, there is no way to limit what he can do. It made me realize how fortunate we are with Z-System to have an operating system with enough security to permit callers on a remote system to run the system more or less as if they were sitting at the keyboard of their personal machine. They don't have to have an elaborate apparatus standing between them and the system.
There are two main aspects of that security. One is the wheel byte. This system flag is tested by many Z-System programs to determine whether certain operations should be permitted or denied. Commands for doing things like erasing, renaming, or copying files typically require that wheel status be in force. Other commands will allow some operations to non-wheel users but deny other operations. For example, some directory programs allow writing an image of the directory to disk or to the printer. These options are (or should be) restricted to 'wheel' users. The wheel byte itself is set and cleared by special commands, such as the WHL command of the RCP. Obviously, a password must be entered correctly before WHL will set the wheel byte.
The second and more complex security feature in Z-System concerns the facilities for limiting the disk areas which a user can access. Many users are unaware of these features, and even those who are aware of them often do not understand them fully and clearly. I will cover the major points here.
With version 3.3 of ZCPR, I introduced extensive and significant changes in the way directory references and security are handled. These changes made understanding security more complex for the system implementor but much easier and less intrusive for the user.
ZCPR3 supports two basic forms of directory reference, the disk/user or DU form and the named directory or DIR form. We will assume that the reader is already somewhat familiar with the basic concepts. The DU form is native to CP/M, which knows about disk drives from 'A' to 'P' and user numbers from 0 to 31 (though there are restrictions on user numbers above 15). The DU form of directory reference is basically physical in nature. Drive letters are associated with real physical devices, and the files in all user areas associated with a given drive letter are stored on the same physical device. One can think of this directory structure as spanning a flat, two-dimensional space (in contrast to the hierarchical tree-structured directories of Unix or MS-DOS).
While the DU directories are basically physical, named directories are purely logical constructs. The named directory register (NDR) module in a Z-System contains a mapping of directory names to drive/user values. The user can load different sets of directory associations at different times. Thus, unlike the static (fixed in time) directory structure of Unix and MS-DOS, the directory structure of Z-System can be dynamic (changing in time).
When the DIR form is used, the command processor or a Z-System program looks for the name in the NDR and substitutes the drive and user values. Only drive/user values are used in the actual file references processed by the disk operating system (DOS). Named directories provide two different and important functions. One is convenience. It is much easier to remember that one's assembly language tools are in ASM and one's wordprocessing files in TEXT than it is to remember that the directories are A7 and B13. The second purpose of named directories is to provide security.
Access to directory areas in a Z-System is controlled in both the DU and DIR domains. Under Z30 these two control mechanisms were completely independent; under Z33 and later, as we shall see, they are very closely coupled. The limits in the DU domain are set by values in the environment descriptor (ENV) called max-drive and max-user. They define a rectangular area of allowed directories in the flat directory space, with drive values ranging from 'A' to the drive specified by max-drive and user numbers ranging from 0 to the number specified by max-user. The smallest space still includes the boot directory A0.
Named directories offer a more flexible means of controlling access to areas on a system. The user can access a named directory even if it refers to a DU area that is beyond the bounds defined above. Each directory name can have an optional password associated with it. Whenever a reference is made to such a directory, the password is (should be) asked for and access granted only if it is entered correctly.
There are a number of important exceptions to the two security limits described above. One concerns the command search path specified in the Z-System PATH module. No restrictions whatever are imposed on the DU areas specified there. If the user was able to place a directory into the path, then it will always be scanned as necessary by the command processor, even if it would no longer be explicitly accessible.
Another general exception occurs for the standard (but optional) configuration of the command processor called WPASS. With this option, when the wheel byte is set, directory passwords are ignored. The user can then freely make reference to any directories either within the specified DU range or associated with a named directory (with or without a password).
Versions of the command processor since Z33 also make the assumption that if a user is in a given directory at the present time he must have had the authority to get there. Therefore, the command processor will always accept any reference to the currently logged directory, even if the DU is out of range and it has no unprotected directory name.
Another set of exceptions relates to the interplay of the DU and DIR limits. Recent versions of the command process act on the principle that if reference to a directory in one form (DU or DIR) would be allowed then references using the alternative form should be equally allowed. For example, suppose that the max DU limit is set to B3 and that directory C4 has the name DIRNAME with no password. Z30 would have refused to accept a reference to C4:, even though it would have no complaints about DIRNAME:. Under Z34, either would work. Conversely, if directory area A3 in the example had a directory name PRIVATE with password SECRET, Z30 would allow a reference to A3: but would insist on correct password entry if the reference was made as PRIVATE:. Again, Z34 always checks the alternate form of reference, and if either meets the security restrictions, then both are accepted.
The standard version of Z-System created by NZCOM or Z3PLUS has the max DU limit set to P31. Thus all directories are allowed using the DU format. As a result, even when named directories with passwords are created, they are accepted freely. In order to create a secure system, the values of max-drive and max-user must be reduced. Also, since password checking is bypassed when the wheel byte is set, the wheel byte must be cleared before the security limits imposed by directory passwords will take effect. Once those two changes have been made, your NZCOM/Z3PLUS system is ready to serve as a remote access system.
[This article was originally published in issue 38 of The Computer Journal, P.O. Box 12, South Plainfield, NJ 07080-0012 and is reproduced with the permission of the author and the publisher. Further reproduction for non-commercial purposes is authorized. This copyright notice must be retained. (c) Copyright 1989, 1991 Socrates Press and respective authors]