The Computer Journal, Issue 41
Z-System Corner
© Jay Sage
Reproduced with permission of author and publisher.
By the time you read this, summer vacation will probably be just a fond memory for you, but it is August as I write this, and I have just returned from three weeks in Israel. This was a total vacation. I didn't touch or even think about computers the whole time I was away, except at the very end when my mind started to refocus on the responsibilities that awaited me at home, including this TCJ column. It was so nice to get "computer compulsion" out of my system that I have not been all that eager to get back immediately to my old routine... but I know it will happen soon enough.
Having not thought about computing for a month, I had to work to recall the things I was excited about before I left and planned to discuss in this issue. Fortunately, I left myself some notes. One item was the continuation of the BYE discussion, this time covering the extended DOS functions implemented in BYE. The truth is I have neither the energy nor the time to take up that subject now. Instead, I am going to come back once again to my favorite subject: aliases and ARUNZ.
I think ARUNZ is the one thing that keeps me hooked on Z-System and not eager to follow after the MS-DOS crowd. Although I have looked hard, I have not found anything with the combined simplicity and power of ARUNZ for MS-DOS. The mainframe batch processing language REXX, which has been ported by Mansfield Software to DOS machines, is far more powerful, and some day I hope to port a greatly simplified version to Z-System. The DOS version of REXX, you see, takes over 200K of memory while it is running! That often does not leave enough memory, even on a 640K machine, for application programs to run. I think I actually have more problems running out of TPA on my Compaq 386 than I do on my SB180!
Anyway, for this column I am going to begin with a very brief report on a rather dramatic change in the works for ARUNZ and the Z-System as a whole. Then I am going to describe two ARUNZ applications that I recently developed for my own use. I think they illustrate some interesting general principles, and you may even find them useful as they are.
The Extended Multiple Command Line
Most people who use ARUNZ aliases - or even standard aliases - sooner or later run into a situation where the command line overflows and the whole process comes to a crashing halt (well, not really a crash, but a sudden stoppage). The standard Z-System configuration supports a multiple command line buffer (MCL) that can accommodate 203 characters. The largest size possible is 255 characters. Either way, there comes a time when aliases are invoked from command lines that already contain additional commands, and the combined command line is too long to fit in the MCL. People like Rick Charnes would have this happen constantly if they did not adopt a strategy to avoid the problem (more about that in one of our examples later).
I have long been intrigued by the possibility of having a much longer command line. The command processor (CPR) has always used a word-size (16-bit) pointer into the command line, and so, without any change in the ZCPR34 code, the CPR could handle a command line as big as the address space of the Z80.
To verify this, I performed a simple experiment. I configured a Z-System with free memory after the MCL, and then, using the memory utility program MU3, I manually filled in the command line with a very long multiple command line sequence terminated, as required, by a null character (binary zero). Sure enough, after exiting from MU3, the huge command line ran without a hitch.
The next step was to write a special new version of ARUNZ that could be configured to recognize an oversized MCL. Richard Conn set up the environment descriptor (ENV) with a one-byte value for the length of the command line that the MCL buffer could contain. Thus there is presently no way to support an extended MCL (XMCL) in a system-invariant way, that is, in a way that allows programs to determine at run time how big the MCL is. We are working on that problem right now, and, by the time you are reading this, there will almost certainly be a new ENV type defined (81H) that uses one of the remaining spare bytes in the type-80H ENV to report the size of XMCL to programs smart enough to check for it.
The original, single-byte MCL size value in the ENV has to remain as is and contain a value no larger (by definition) than 255 (0FFH). That value is used by the command processor when new user input is being requested. There is no way for the CPR to allow users to type in command lines longer than 255 characters without adding a vast amount of code to perform the line-input function now so conveniently and efficiently provided by the DOS. A shell could be written that included such code, but I really can't imagine anyone typing in such long command lines. If they do, it probably shows that they are not making proper use of aliases.
I have decided to use only one of the spare ENV bytes for the XMCL size and to let that value represent the size - in paragraphs - of the total MCL memory buffer allocated, including the five bytes used by the address pointer, size bytes, and terminating null. The term 'paragraph' as a unit of memory is not often used in the Z80 world. I believe it was introduced with the 8086 processor, where the segment registers represent addresses that are shifted four bits right. Each unit in the segment register is, therefore, 16 bytes and is called a paragraph. With this system, the XMCL buffer can be as large as 255 * 16 = 4080, which allows command lines with up to 4075 characters. Rich Charnes, do you think you can live with that without cramping your style too much?!
Most people will not want to allocate that much memory to the operating system, and I would never have considered this step before the new dynamic versions of Z-System were available. While I might be willing to allocate 1K to the XMCL most of the time, I certainly would want to be able to reclaim that memory when I need it. I'm not sure whether NZCOM or Z3PLUS can be cajoled into handling this kind of flexibility yet; new versions may be needed at some time in the future.
I put the new version of ARUNZ out for beta test, and it worked just fine, and one could write very long alias scripts. Rick Charnes, however, quickly identified a problem. Suppose a conventional alias appeared in the command sequence. After expanding itself and constructing the new command line, the alias would find that, as far as it knew, there was not enough room for it in the MCL. In a nutshell, the hard part with going to the XMCL is that it is not enough to have an advanced ARUNZ; all programs that operate on the MCL must be upgraded. We hope to have new versions of the library routines in Z3LIB that perform these functions. Then, if we are lucky, most of the utility programs can be upgraded simply by relinking. I'm sure it won't be quite that easy, of course!
A MEX Alias
For those who are not familiar with it, MEX (Modem EXecutive) is an advanced telecommunications program written by Ron Fowler of NightOwl Software. Early versions were released for free to the public (up to version 1.14), while the most advanced versions (called MEX-Plus) are commercial products. I use version 1.65, and some of the specifics in my example apply to that version. I am pretty sure that the technique I describe can be applied to the free version as well.
Rather than being a telecommunications program, MEX should probably be considered a telecommunications programming language. It supports a very wide range of internal commands for managing telecommunications tasks, and it even has a script language for automating complex sequences of operations.
The MEX command line allows multiple commands to be entered just as in Z-System, and a MEX command allows the user to define the command separator. Although I depend on aliases to generate complex Z-System commands and MEX script files to automate complex MEX command sequences, I still frequently make use of simple, manually entered multiple commands.
Being accustomed as I am to entering Z-System commands separated by semicolons, I naturally set up my version of MEX to use the semicolon as its separator, too. Now I can comfortably work in both environments. However, I also frequently like to invoke MEX with some initial commands, which MEX allows one to include in the command tail. Here's a simple example.
B11:TEMP>mex read mnp on
This command invokes MEX and tells it to run the script file MNP.MEX with the parameter "ON". This script causes a string to be sent to my modem which engages the MNP error correcting mode (yes, when I purchased my most recent modem - replacing a USR Password - I decided to spend the extra money for MNP, although at the time there weren't many systems that supported it; now I'm glad I did).
That command line works fine. But often I want to do more, and so I always wanted to enter something like:
B11:TEMP>mex read mnp on;call zitel
This would start out by doing what the first example did but would then continue by placing a call to the ZITEL BBS. [If you can keep a secret, I'll tell you that the ZITEL BBS is the MS-DOS system that I run for the ZI/TEL Group of the Boston Computer Society. ZI/TEL comes from the letters in Zilog and Intel, and it symbolizes the fact that we support the two main operating systems run on chips from those companies: CP/M and MS-DOS. The BBS machine, a Kaypro 286/16, is sitting in the other room (you don't think I'd allow it in the same room with the Z-Node, do you?), and it has an HST 9600 bps modem with MNP error correction. If you want to contact me there, by the way, the number is 617-965-7046.]
An on-the-ball reader already realized that the above command will not work, because the semicolon separator before the CALL command, which I intended as the MEX separator, will be interpreted by the CPR as its separator, and it will terminate the MEX command. What can we do about this?
Some compromise here is inescapable, and I was willing to accept - from the CPR command line only - a MEX separator other than semicolon. Thus the following form would be acceptable
B11:TEMP>mex read mnp on!call zitel
with an exclamation point as the separator as in CP/M-Plus. But for years I could not figure out how to accomplish this.
At first I thought there was a very simple solution. When MEX starts up, it can be set up to automatically run an initialization script file INI.MEX. So, I created a version of MEX (the MEX "CLONE" command makes it easy to create new versions) that used "!" as the separator, and I created an INI.MEX file with the command
STAT SEP ";"
Thus, as soon as MEX was running, the separator would be set back to a semicolon. Unfortunately, to my chagrin, I learned that MEX invokes the INI.MEX script only when no commands are included on the command line. With the ZITEL command line shown earlier, MEX would be left with the exclamation point as the separator.
Here is what I thought of next (at least momentarily). Rename MEX.COM to MEX!.COM and set up a MEX alias in ALIAS.CMD with the definition
MEX mex:mex! $*!stat sep ";"
The idea here is that the user's MEX commands from the command line (separated by "!") will be passed in by the $* parameter and will have the STAT command added. Thus our earlier example will turn into the command line
B11:TEMP>mex:mex! read mnp on!call zitel!stat sep ";"
This, of course, fails for the same reason that I could not just enter commands with semicolons in the first place. The trick to get around this is to use a command that for some reason Ron Fowler does not document in the MEX manual: POKE. It works like the Z-System command of the same name and places a byte of data into a specified memory address.
I knew the value I wanted to poke: 3BH, the hex value for the semicolon character. The question was, where should it go? To find out, I took a version of MEX set up with semicolon as the separator and the version with exclamation point as the separator and ran the utility DIFF on them (in verbose mode to show all the differences). Then I looked for the place where the former has a semicolon and the latter an exclamation point. For MEX-Plus this turned out to be 0D18H so that the MEX poke command would be
POKE $0D18 $3B
Note that MEX uses a dollar sign to designate hex numbers. The alias now read
MEX mex:mex! $*!poke $$0d18 $$3b
Observe that a double dollar sign is needed to get a single dollar sign character into a command. A lot of people forget this and end up with scripts that don't do what they're supposed to.
I tested this, and it works splendidly - but with one possible 'gotcha'. The commands passed to MEX must not invoke any script files that depend on the command separator being a semicolon (because it will be exclamation point until the final poke command runs); nor may the read files change the command separator (because the rest of the command sequence still assumes it is the exclamation point). For this reason, it is prudent to write all script files with only one command per line so that no separator is needed. I haven't been doing this, but I will from now on!
One final word on the script. I actually did not do this exactly as I have described. Instead, I left my MEX.COM set up with the semicolon separator, and I created a distinct ARUNZ alias called MEX! so that I would be reminded of the separator. This alias script reads
MEX! get 100 mex:mex.com;poke d18 "!;go $*!poke $$0d18 $$3b
This uses the famous poke&go technique originated by Bruce Morgen. MEX.COM is loaded into memory by the GET command, and then the Z-System POKE command sets "!" as the command separator. Then the modified loaded code is run by the GO command. The rest is as described previously.
A Spell-Check Alias
I try to remember to put all my writing through The Word Plus spelling checker that came with WordStar Release 4 so that as many typos as possible will be caught. The procedure for doing that on a Z-System is a bit complicated because the text file is generally not in the same user area as the spelling check program. While writing my last TCJ column, I finally got fed up with the complexity and automated the whole process using a set of aliases.
I wanted to support the following syntax:
C1:TCJ>spell filename.typ dictname
If just the file name was given, the alias would prompt for the name of the special dictionary to use, and if not even a file name was given, then the alias would prompt for both names. A special version of the command, ZSPELL, would take only the file name and would automatically use ZSYSTEM as the name of the special dictionary (it knows about mnemonics like ZCPR, MCL, and RCP, and about all those special words like debugger, relocatable, and modem). We'll describe the general alias set first. In listing the aliases, we will write them in multiline format for easy reading;in the ALIAS.CMD file the scripts have to be on a single line (though I hope that will change soon).
The user-interface alias, SPELL, deals only with the matter of how many parameters the user has provided. It reads as follows:
SPELL
if nu $1;
/TW0;
else;
if nu $2;
/TW1 $1;
else;
/TW2 $1 $2;
fi;
fi
If no parameters at all are provided (IF NULL $1), then the secondary script TW0 is run. The leading slash signals ZCPR34 that the command should be directed immediately to the extended command processor. If a first parameter but no second parameter is present (IF NULL $2), then the secondary script TW1 is run. Finally, if both parameter are provided, then script TW2 is run.
The script TW1 includes a prompt only for the name of the special dictionary file:
TW1
$"Name of special dictionary: "
/TW2 $1 $'e1
The first token in any user response to the first prompt ($'E1 - when working with ARUNZ you should have a printout of the parameter DOC file) is used along with the file name that was already given, and both are passed to TW2.
The script TW0 includes prompts for both the file name and the special dictionary:
TW0
$"Name of file to check: "
$"Name of special dictionary: "
if ~nu $'e1
/TW2 $'e1 $'e2
fi
The first tokens in the responses to the prompts are passed to script TW2. If no file is specified for checking, the alias simply terminates.
Before we look at TW2, which does the real work, let me ask a rhetorical question: why do we break this process up into so many separate aliases. There are two main reasons. The first is that the command line buffer would overflow if all these smaller scripts were merged into a single big script. The extended MCL we discussed earlier could overcome this problem, but for another reason we would still have to use separate aliases.
As I have discussed in past columns, ARUNZ cannot know at the time it expands a script what the results of conditional tests will be later when the IF commands are run. Thus ARUNZ must process all user input prompts that appear in the script. This would mean asking for a file name and special dictionary even when the names were given on the command line. The solution to this problem is to put the prompts in separate scripts that get invoked only when the information requested in those prompts is actually needed.
Now let's look at the script TW2.
TW2
path /d=tw:;
$td1$tu1:;
tw:tw $tf1 $tn2.cmp;
path /d=;
/twend $tn2;
$hb:
This is simpler than what you expected, no? Well, there is still a lot of work imbedded in the subroutine script TWEND, which we will cover later. Here we broke up the script solely to prevent MCL overflow.
The first command makes use of the ZSDOS file search path (see the articles by Hal Bower and Cam Cotrill on ZSDOS in TCJ issues 37 and 38). Although there was an attempt to update WordStar Release 4 to include some Z-System support, no such attempt was made with The Word Plus spell checker. In general, the file to be spell-checked will be in one directory and The Word files in another directory. The main program TW.COM could be located by the command processor using its search path, but TW.COM needs a number of auxiliary files, such as the dictionary files. How can the system be made to find all of these files at the same time. ZSDOS provides the answer.
I have replaced the standard Z-System PATH command with the ZSDOS utility ZPATH (renamed to PATH). The first command in TW2 defines the DOS search path to include the directory TW:, which is where I keep all the files that are part of The Word Plus spell-checking package. Once that directory is on the DOS path, all files in it will more-or-less appear to be in the current directory. Very handy! If you use ZDDOS, the search path is not available. I will not show it here, but you can accomplish the same thing using only public files. It's just not quite as neat and straightforward. I am willing to pay the small memory penalty to get the nice extra features of ZSDOS over ZDDOS.
The second command logs us into the directory where the file to be checked resides. If we did not include a DIR: prefix, we were already there, but the extra command does not hurt, and it is nice to know that a directory can be specified explicitly (in either DU: or DIR: form) for the file to be checked. There could be a problem if the file is in a user area above 15, since you may not be able to log into that area. My configuration of Z34 allows this, but when I run BGii I lose this feature (and I sure miss it). If you can't log into those areas, then you should not keep files there that you want to spell-check.
The third line actually runs the spell checker (you knew that had to happen some time!). Notice that even if the user specified a file type for the special dictionary, type CMP is used. Only the name ($TN2) without the type is taken from the user. As the master program TW.COM is run, it will find its component program files (e.g., SPELL.COM, LOOKUP.COM, MARKFIX.COM) and the various dictionaries in the TW: directory thanks to ZSDOS, and it will find the text file in the current directory. As it works through the text, if there are any questionable words, it will write out a file ERRWORDS.TXT to the current directory. If any words are added to the special or UPDATE dictionaries, then the modified dictionaries will be read from TW: but written out to the current directory. You must understand these facts in order to understand the rest of the script.
Once the spell-checking is complete, the ZSDOS path is set back to null (unless I have a special need to have the DOS perform a search, I leave it this way to avoid surprises). Then the ending script TWEND is run, and finally the original directory ($HB:) is restored as the current directory.
Now let's look at TWEND. As it is invoked, the name of the special dictionary is passed to it. TWEND's job is to clean up scratch files and to take care of any updated dictionaries. It reads
TWEND
if ex errwords.txt;
era errwords.txt;
fi;
/dupd $1;
/dupd updict
For efficiency and to prevent MCL overflow, the dictionary updating is performed by yet another subroutine script, DUPD. It gets called twice, once with the special dictionary (if any) and once with the update dictionary. It reads as follows:
DUPD
if ex $tn1.cmp;
mcopy tw:=$tn1.cmp /ex;
fi
If an updated version of the specified dictionary exists in the current directory, then it is copied to the TW: directory, overwriting any existing file of that name (MCOPY option E). The source file is then erased (MCOPY option X). Oh yes, I almost forgot; the MCOPY here is my renamed version of the COPY program supplied with ZSDOS.
That is it except for showing you the special ZSPELL version of the alias. Notice that I make the "ELL" part of the command optional by inserting the comma in front of that part of the alias name. I also allow the script to be invoked under the name ZTW. The main SPELL script actually has the name "TW=SP,ELL" on my system. Since TW: is not on my command search path, the command "TW" will invoke the ARUNZ script unless I am in the TW: directory at the time.
ZTW=ZSP,ELL
if nu $1;
/ZTW1;
else;
/TW2 $1 zsystem.cmp;
fi
ZTW1
$"Name fo file to check: "
if ~nu $'e1
/TW2 $'e1 zsystem.cmp
fi
I hope you find these alias examples useful and instructive. That's all for this time. See you again in two months.
[This article was originally published in issue 41 of The Computer Journal, P.O. Box 12, South Plainfield, NJ 07080-0012 and is reproduced with the permission of the author and the publisher. Further reproduction for non-commercial purposes is authorized. This copyright notice must be retained. (c) Copyright 1989, 1991 Socrates Press and respective authors]