The Computer Journal, Issue 34
Z-System Corner
© Jay Sage
Reproduced with permission of author and publisher.
For this issue I am going to live up to a long-standing tradition: once again I am not going to cover the material that I said I was. Last time I presented half the new material on ARUNZ and said I would cover the second half this time. Well, I am not going to. ARUNZ is now up to version 'N' (it was 'J' last time). Until it settles down a bit, it is probably futile to try to describe it. Besides that, I am getting a little bored with the subject (though obviously not with the program), and perhaps you are, too.
Though living up to tradition, I am going to reverse a trend. For some time now my columns have been getting longer and longer. This time I really am going to write a short one. Besides the fact that because of me Art Carlson is apparently running low on printer's ink, I am just about written out, having just completed the manuals for NZ-COM and Z3PLUS.
NZ-COM and Z3PLUS
Those manuals started out being simple affairs, but I just don't seem to be able to get my obsession with thoroughness and completeness under control, and they soon turned into full-fledged books about the respective product and Z-System in general. I've been burning the midnight (actually, 2 am) oil for the past two or three months. Each manual now runs about 80 pages! No wonder I don't have many words left in my system at this point.
Though somewhat reluctant to indulge in self-praise, I have to say that the manuals are really quite good, and the products (NZ-COM and Z3PLUS) are absolutely fantastic. Joe Wright (NZ-COM), Bridger Mitchell (Z3PLUS), and I have had a very enjoyable and highly productive partnership in this effort. I sincerely urge you all to buy the automatic, universal, dynamic Z-System appropriate for your computer: NZ-COM for CP/M-2.2 computers and Z3PLUS for CP/M-Plus computers. Both are $69.95 from Sage Microsystems East, Plu*Perfect Systems, or Alpha Systems (see ads in TCJ).
After the experience bringing these products to market, I will no longer laugh so heartily when I hear stories about Borland or Lotus or Ashton-Tate not delivering their products on the promised dates. Hopefully you have lost your TCJ issue #32 and have forgotten that I wrote there, and I quote: "By the time you read this, they will definitely be available." In issue #33 I said, "the two new dynamic Z-Systems that will have been released by the time you read this." That almost made a double liar out of me. Luckily, issue #33 was delivered just enough behind schedule to let that statement squeak through - barely.
With respect to NZ-COM, Joe Wright and I have agreed to publicly blame each other for the delay. Actually, following common practice, we originally both agreed to blame Bridger Mitchell, though he, of course, had nothing to do with the NZ-COM delay (Z3PLUS is another story). However, now that Bridger has a TCJ column, too, we worried that such a slander might not go unanswered. Anyway, Joe can blame me for not getting the manual done on time, and I can blame Joe for not writing the code to conform to my description of it in the manual! You can readily see that no one should ever get involved in a product development all by himself. Always make sure there is someone else to blame.
The truth of the matter is that we really thought the coding was complete by early April and that a simple manual could be knocked off in a week (naive!!). In fact, as I alluded to above, the scope of the manual kept expanding. At the same time, as we attempted to describe the programs very precisely, we discovered a number of deficiencies in the code. Some coding limitations that we thought we would accept in the current version of NZ-COM and would upgrade later just didn't seem acceptable any more once we wrote them down on paper. As a result, we have really skipped version 1 of NZ-COM and have gone directly to version 2. There are quite a few exciting new features beyond what I described in issue #32; many will appeal especially to those with a penchant for the unconventional (but I won't say any more about them now).
PRL Files and Type-4 Programs
One of the new features introduced with ZCPR34 is the type-4 program. A number of questions have been appearing in messages on Z-Nodes, so I thought I would say a few words on this subject.
Just to refresh your memory, ordinary CP/M program files are loaded beginning at an address of 100H. This was also true of Z-System programs. They differ from standard CP/M programs in that the code starts with a special header. One item in the header is the text string 'Z3ENV' beginning with the fourth byte of the program. This string is used to identify the program as a Z-System program.
After the text string comes a number, now called the program type identifier. If the number is 2, then the so-called environment descriptor is included in the first page of the program file. These type-2 programs are rarely seen today. If the number is 1, then the program was designed to run in a ZCPR3 system with a permanent operating system memory buffer containing the environment descriptor. The program only has to store the address of that descriptor, and it can then adapt to any system in which it is run.
The environment or ENV address is stored in the two bytes immediately following the program type byte. Prior to ZCPR version 3.3, the address had to be installed into programs using a utility like Z3INS before they could be used. Starting with ZCPR33, the command processor installs the value as part of the process of loading the file from disk.
With ZCPR33, the type-3 program was introduced. These programs are not limited to execution at 100H, as all previous programs had been. The two bytes after the ENV address contain the address at which the code is designed to run. The Z33 command processor examines the program type byte, and if it is 3, it reads the load address from the header and proceeds to load the program to the designated address and execute it there.
The type-3 program made it possible to load and run programs at addresses other than 100H, but the address at which any given program file would run was still fixed. In his column in the last issue, Bridger Mitchell described a fascinating and remarkable program structure that allows a program to run at whatever address it is loaded to. That same idea is the basis for the new type-4 program. Bridger's ANYWHERE program could be loaded manually to any address and then executed. Type-4 programs are loaded automatically by the command processor to the highest address in memory at which they can run without overwriting the command processor or any resident system extension (RSX) that is present. I would like to provide some additional details on how type-4 programs work and how they are constructed.
As with ANYWHERE, type-4 programs are derived from so-called (and, as Bridger pointed out, mis-named) page-relocatable or PRL files. Bridger defined and described those files in his column in the last issue, but another shot at it probably won't hurt. I will approach the subject somewhat differently - with a concrete example. Consider the short and simple program in Listing 1. It is set up for a starting address of 100H. Fig. 1 shows the binary image of the sort one would see if the program were loaded with a debugger (e.g., DDT) and displayed.
If we change the argument of the ORG directive from 100H to 200H and assemble again, we get the results shown in Listing 2 and Fig. 2. You should examine those results and note the things that have stayed the same and the things that have changed. Note in particular that only three bytes in the code have actually changed. One is the high order byte of the address of the initial jump instruction. The destination of that jump is in the program, and, since the program has moved up by 100H, the jump address has increased by an identical amount. The second change is in the data word containing the entry point address. Obviously that address changes when the rogram origin is changed. The third change is in the value loaded into the DE register pair. It is the address of the message string, which is likewise a part of the program.
Note that the argument of the jump to DOS has not changed. It is an absolute address outside the program. Therefore, it does not change.
Now let's look at a PRL file for the same program. I am not aware of any assemblers that can produce a PRL file directly. The usual procedure is to remove the ORG statement from the source code, assemble the program to a REL (normal relocatable format), and then use a linker to generate the PRL file from the REL file. Fig. 3 shows the binary image of a PRL file produced using the SLR virtual-memory linker SLRNK+. Unfortunately, inexpensive linkers, such as SLRNK and ZLINK, are not able to produce PRL files. Later we will show you a method, though somewhat tedious, that allows you to construct a reasonable approximation to a PRL file using an ordinary assembler (no linker at all).
SLRNK+ actually cannot produce a PRL directly using the source code as listed. The SLR manual discusses in a somewhat opaque way the technique for generating a correct PRL file. The problem is that the one-page nearly empty header at the beginning of the program is not generated. Joe Wright invented the trick of linking in the file SLR.REL derived by assembling source code with the sole statement
DS 256
This allocates one page of memory. The PRL file is produced by the linker command
SLRNK+ TEST/K,SLR,TEST,/E
The term "TEST/K" defines the output file, the term "SLR" allocates the empty header, and the term "TEST" links in the actual program code.
You should notice in Fig. 3 the following things. First, the PRL file begins with a one-page header, which is entirely zero except for a word at address 101H (you can't tell from this example that it is a word, but it is). This word is the length of the code, 001BH or 27 decimal in this example. The program code itself begins on the next page (200H) and is the same as the code in Fig. 1.
The other new bytes in the PRL file are those that follow the last byte of the program code. These bytes comprise the relocation bitmap that Bridger Mitchell described in his column in the previous issue of TCJ. The first byte is 20H, which expanded to binary is 00100000. This means that the third byte in the program code is the high byte of an address that must be relocated to make the program code execute at an address other than 100H. Indeed, the third byte is the address to which the JP instruction jumps. The second byte in the bitmap is 08H or 00001000 binary. This tells us that the 13th byte in the program code is an address that has to be relocated when the program is relocated. Indeed, this is the address of the start of the program in the Z-header. The third byte in the bitmap is 01H or 00000001 binary. It tells us that byte 23 is an address. If we look carefully, this is the address part of the "LD DE,MSG" instruction.
How Does ZCPR34 Load and Execute a Type-4
Bridger Mitchell explained last time in some detail how a PRL file can be relocated to run at any address. It really is not necessary to understand all the details. The basic idea is that the bitmap tells a loader which bytes in the code to adjust.
The Z34 command processor has a special mechanism for processing type-4 programs. After the command processor has located a transient program, it loads the first record of the file into the default buffer at 80h. Here it can examine it to see what kind of program it is. If it is a standard CP/M program or a type-1 or type-2 Z-System program, it sets up the load address as 100H and proceeds to load the entire file into memory, starting over with the first record. If it is a type-3 program, the same procedure is followed except that the load address is extracted from the program header.
With the type-4 program things are not so simple, because the load address has to be calculated and the code has to be relocated. Z34 gets these tasks accomplished in a very clever and tricky way. It could have done all the work itself, but that would have added a lot of code to the command processor. Instead, we took advantage of the fact that a PRL file has that two-record header with almost nothing in it. To make a type-4 program, we overlay onto this header a special loader program. Z34 executes the code there to calculate the load address and then to perform the code relocation.
The loader is available in HEX format (TYP4LDR.HEX) and can be grafted onto the PRL file using the command
MLOAD file=file.PRL,TYP4LDR
where 'file' is the name of the PRL file that you want to convert to a type-4 executable file.
By putting the loader code in the program rather than in the command processor, we provide additional flexibility. TYP4LDR calculates the highest address in the TPA to which a program can be loaded, but other loaders could return the address of the RCP or FCP and make possible self-installing modules. Clever users will undoubtedly come up with some other interesting applications that use special header code.
How Do We Make a PRL File
The easiest way to make a PRL file and from that a type-4 program is with a capable linker like LINK from Digital Research or SLRNK+ from SLR Systems. LINK came with my CP/M-Plus computer; I do not know how much it costs or how to obtain it otherwise. SLRNK+, which offers many very useful and powerful features besides the ability to make PRL files, costs $195. For someone who wants to experiment casually with type-4 programming, this is probably too much money to spend. If you are not going to do it very often and don't mind a little work, you can hand craft a PRL file using a debugger like DDT. I will take you through the procedure using our sample program from Listing 1.
Making the bitmap is the hard part of the procedure. You should key in the program called MAKEPRL.Z80 in Listing 3 and assemble it to a HEX file. We will use that code in the debugger first to make a "byte-map" and then to convert the byte-map into a bitmap. We assume that we have already assembled versions of the program with ORGs of 100H and 200H.
To construct the PRL file, we invoke the debugger (assumed to be DDT) and issue the commands shown in Fig. 4. The first pair of commands loads the utility program MAKEPRL. The next two lines load the version of our program that was assembled to run at 100H. At this point we have to note the "next load address" reported by the debugger (I suggest you write it down). Now we load the version of the program assembled to run at 200H so that it follows right on the end of the 100H version. To do this, we use an offset value in the "R" command that is 100H lower than the "next address" that was reported a moment ago.
There is one other very important step we need to perform at this point. MAKEPRL has to be told the address at which the second program image was loaded. The value is patched in at address 10EH using the commands shown in Fig. 4 starting with "S10E". For our example program, the next address is reported as 0280. Therefore, low-nextaddr is 80 and the high-nextaddr is 02.
Now we let MAKEPRL do the hard part by running it with the "G" command. When it is finished, we need to examine the value in the HL register, since it tells us the next address after the bitmap. After leaving DDT we have to save the code image from 100H up to but not including that address. For the example program, the value in HL is reported to be 290H. Since we are presumably running Z34 and have the type-4 SAVE program, we save the result using the command
SAVE 100-28F PRLTEST.COM
If you do not have the type-4 SAVE,you will have to calculate the number of sectors to save.
Fig. 4 lists one DDT command that we did not discuss. The "F103,1FF,0" command fills the part of the header after the code size word with zeros. This makes the file look prettier, but it is not absolutely necessary, especially if you are later going to overlay the type-4 loader as described below.
The PRL files made this way can be used to make type-4 programs, and they can be used in Bridger Mitchell's ANYWHERE program. However, we should point out that these PRL files are not as efficient as those produced by a linker. We assumed that the code in the COM files extended to the end of the last record in the file.
Perhaps you can build on my simple method and figure out how to extend it to produce an optimal PRL file just like the one from SLRNK+. It would also not be too difficult to write a program using routines in SYSLIB to read in the pair of COM files and generate a PRL file from them completely automatically. The most elegant method for doing this would use random-record writes. I invite readers to send me such a program.
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
----------------------------------------------------
0100 C3 13 01 5A 33 45 4E 56 03 00 00 00 01 48 65 6C
0110 6C 6F 24 0E 09 11 0D 01 C3 05 00Fig. 1.
Binary image of the sample program in Listing 1.
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
----------------------------------------------------
0100 C3 13 02 5A 33 45 4E 56 03 00 00 00 02 48 65 6C
0110 6C 6F 24 0E 09 11 0D 02 C3 05 00Fig. 2.
Binary image of the sample program in Listing 1 when linked to run at a starting address of 200H and loaded at 100H.
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
----------------------------------------------------
0100 00 1B 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0200 C3 13 01 5A 33 45 4E 56 03 00 00 00 01 48 65 6C
0210 6C 6F 24 0E 09 11 0D 01 C3 05 00 20 08 01 00Fig. 3.
Binary image of PRL file produced for the same test program and loaded at address 100H. Some memory regions containing bytes of 00 have been omitted from the display here.
IMAKEPRL.HEX<cr> ; Ready to load PRL maker routine
R,<cr> ; Load it to address 100h
ITEST100.COM<cr> ; Ready to load program ORGed for 100h
R100<cr> ; Load it to address 200h (offset 100)
ITEST200.COM<cr> ; Ready to load program ORGed for 200h
R<nextaddr-100><cr> ; Load it to proper offset
s10E<cr> ; Ready to patch in code size
low-nextaddr<cr> ; Low byte of code size
high-nextaddr<cr> ; High byte of code size
.<cr> ; End patch with period
G<cr> ; Run MAKEPRL code at 100h
X<cr> ; Display registers - note value of HL
F103,1FF,0<cr> ; Clean up the header area
G0<cr> ; Exit from DDTFigure 4.
Commands issued to DDT to produce a PRL file from two COM files assembled to run at addresses of 100h and 200h.
Z80ASM SuperFast Relocating Macro Assembler Z80ASM 1.31 Page 1
PRLTEST Z80
1 0100 org 100h
2
3 0100 entry:
4 0100 C3 0113 jp start
5
6 0103 5A 33 45 4E db 'Z3ENV'
7 0108 03 db 3
8 0109 0000 dw 0
9 010B 0100 dw entry
10
11 010D 48 65 6C 6C msg: db 'Hello','$'
12
13 0113 start:
14 0113 0E 09 ld c,9
15 0115 11 010D ld de,msg
16 0118 C3 0005 jp 0005h
17
18 end
0 Error(s) Detected.
27 Absolute Bytes. 3 Symbols Detected.Listing 1.
Simple example program assembled for a load address of 100H.
Z80ASM SuperFast Relocating Macro Assembler Z80ASM 1.31 Page 1
PRLTEST Z80
1 0200 org 200h
2
3 0200 entry:
4 0200 C3 0213 jp start
5
6 0203 5A 33 45 4E db 'Z3ENV'
7 0208 03 db 3
8 0209 0000 dw 0
9 020B 0200 dw entry
10
11 020D 48 65 6C 6C msg: db 'Hello','$'
12
13 0213 start:
14 0213 0E 09 ld c,9
15 0215 11 020D ld de,msg
16 0218 C3 0005 jp 0005h
17
18 end
0 Error(s) Detected.
27 Absolute Bytes. 3 Symbols Detected.Listing 2.
Simple example program assembled for a load address of 200H.
; This code, which assists in the generation of a PRL file from a pair of COM
; files assembled for execution at 100H and 200H, is by no means optimized for
; speed or size. I have tried to optimize it for clarity!
org 100h
db 0 ; Standard PRL header (and NOP)
size: dw 0 ; PRL file size (filled in by code)
jp start
db 'CODE200:' ; Identification string
c200:
dw 0 ; Patch to address of code linked to 200h
start:
; The first step is to compute the size of the code and store the value at
; address 101h as required for a PRL file. We also put this value in BC.
; We set up DE to point to the code assembled for 200H and HL to point to
; the code assembled for 100H.
ld hl,(c200) ; Start of code for 200h
ld de,200h ; Start of code for 100h
xor a
sbc hl,de ; Difference is assumed size of code
ld (size),hl ; Store in proper place for PRL file
ld b,h ; ..and in BC
ld c,l
ld hl,(c200)
ex de,hl ; DE -> code for 200h, HL -> code for 100h
; Now we subtract the code for 100h from the code for 200h to generate the map
; of bytes that are addresses that have to be relocated. There will be a byte
; of 01 corresponding to each byte in the code that is the high order byte of
; an address that must be relocated. There will be bytes of 00 everywhere
; else.
bytemap:
ld a,(de) ; Get byte from 200h version
sub (hl) ; Subtract byte from 100h version
ld (de),a ; Replace 200h code with byte map
inc hl ; Point to next bytes
inc de
dec bc ; Any more to do?
ld a,b
or c
jr nz,bytemap ; Loop until done
; Now we have to compress the byte map into a bit map, taking each 8 bytes of
; the byte map and packing the values into a single byte in the bit map. The
; result is written immediately following the code (i.e., at the location of
; the code linked to 200h).
ld hl,(size) ; Get number of bytes in byte map
ld b,3 ; Divide by 8 (2 to the 3rd power)
divide:
xor a ; Clear carry
rr h ; Rotate H right, low bit to carry
rr l ; Rotate L right, carry into high bit
djnz divide ; Repeat 3 times
ld (mapsize),hl ; Save the value
ld de,(c200) ; Point to byte map
ld hl,(c200) ; Point to bit map (same place!)
makemap:
ld b,8 ; Process block of 8 bytes into 1 byte
ld c,0 ; Initial value for byte in bit map
makebyte:
ld a,(de) ; Get byte from byte map
inc de ; Advance pointer
rr a ; Move relocation bit into carry flag
rl c ; Move carry flag into byte in C
djnz makebyte ; Repeat 8 times
ld (hl),c ; Put result into bit map
inc hl ; ..and advance its pointer
push hl
ld hl,(mapsize) ; See if we are done
dec hl
ld (mapsize),hl
ld a,h
or l
pop hl
jr nz,makemap
rst 38h ; Breakpoint to end program
mapsize:
ds 2 ; Scratch area
endListing 3.
Utility program to perform the hard part of making a PRL file using a debugger.
[This article was originally published in issue 34 of The Computer Journal, P.O. Box 12, South Plainfield, NJ 07080-0012 and is reproduced with the permission of the author and the publisher. Further reproduction for non-commercial purposes is authorized. This copyright notice must be retained. (c) Copyright 1988, 1991 Socrates Press and respective authors]