11 Jul
2024
11 Jul
'24
2:05 a.m.
On Wed, Jul 10, 2024 at 9:54 PM John R Levine <johnl(a)taugh.com> wrote:
Well, sure, except in odd cases like the Vax compiler and reoptimizer
someone mentioned a few messages back.
I think the point about the VAX compiler is that it's an actual
compiler and that the VAX MACRO-32 _language_ is treated as a "high
level" programming language, rather than as a macro assembly language.
That's not doing binary->binary translation, that's doing
source->binary compilation. It's just that, in this case, the source
language happens to look like assembler for an obsolete computer.
- Dan C.
On Wed, 10 Jul 2024, Dan Cross wrote:
Sure. But we're trying to date the topic here; my point is that
JITing was well known, and simulation was similarly well known; we
know when work on those books started; it doesn't seem that odd to me
that combining the two would be known around that time as well.
It's not clear to me why you suggest with
such evident authority that
Knuth was referring only to serialized instruction emulation and not
something like JIT'ed code; true, he doesn't specify one way or the
other, but I find it specious to conclude that that implies the
technique wasn't already in use, or at least known.
The code on pages 205 to 211 shows an instruction by instruction
interpreter. I assume Knuth knew about JIT compiling since Lisp systems
had been doing it since the 1960s, but that's not what this section of the
book is about. One of the later volumes of TAOCP was supposed to be
about
compiling, but it seems unlikely he'll have time to write it.
Yes; volumes 5, 6 and 7 are to cover parsing, languages, and compilers
(more or less respectively). Sadly, I suspect you are right that it's
unlikely he will have time to write them.
We've been discussing batch or JIT translation of
code which gives
much better performance without a lot of hardware help.
JIT'd performance of binary transliteration is certainly going to be
_better_ than strict emulation, but it is unlikely to be _as good_ as
native code. 
11 Jul
11 Jul
4:50 p.m.
New subject: [TUHS] Re: machine code translation,as mental architecture models
On Wed, Jul 10, 2024 at 10:12 PM Dan Cross <crossd(a)gmail.com> wrote:
I think the point about the VAX compiler is that
it's an actual
compiler and that the VAX MACRO-32 _language_ is treated as a "high
level" programming language, rather than as a macro assembly language.
That's not doing binary->binary translation, that's doing
source->binary compilation. It's just that, in this case, the source
language happens to look like assembler for an obsolete computer.
Yes, that was precisely my point, and thank you for stating it more clearly
and concisely than I did. The VAX MACRO compiler takes in VAX assembly
source code, not binary VAX instructions.
The discrete transistor -> integrated circuit transition (2nd generation->
3rd generation) eventually resulted in enough of a speedup that microcode
implementations of instruction sets such as System/360 offered acceptable
performance at a low manufacturing cost. IIRC System/360 models 75 and up
were done completely in hardware--no microcode. S/360 models 67 and down
were microcoded. The lowest end S/360, the model 25, was actually a 16-bit
machine. All the S/360s from the 65 and down had microcoded emulators for
2nd generation IBM architectures such as the 1400. The emulators usually
ran faster than the real hardware.
The emulators were there to accommodate data centers that had lost their
source code for mission critical applications. There was also translation
software available that would translate 2nd generation machine code into
COBOL. I remember their ads appearing weekly in ComputerWorld. It showed
a data center manager smacking his forehead. The headline of the ad read,
"1400 emulation? In your 19xx budget?" I never had personal experience
with the translator, but I'm told that the resulting COBOL was hard to
maintain.
Another example of using instruction set emulation was the IBM 5100
Portable Computer. Released in 1975, this was a desktop-sized machine with
an attached keyboard and small CRT screen. One interacted with it in
either APL or BASIC, the language selectable by a toggle switch. 64K of
memory was available to the user. Internally the machine had a CPU
codenamed Palm that was used in many of IBM's peripheral controllers. The
hardware had two 64K banks of ROM and one 64K bank of RAM. One of the ROM
banks held the BASIC interpreter and the other bank the APL interpreter.
The front panel toggle switch selected which ROM bank was in use.
The BASIC interpreter was implemented in native Palm code. But the APL
interpreter was APL\360, in S/360 machine code. There was a S/360
instruction set emulator written in Palm code that interpreted the APL\360
interpreter. This sort of situation delivers horrible performance.
During grad school I interned for a few years at IBM's Cambridge Scientific
Center (CSC) in Cambridge, MA, the birthplace of CP-67, the virtual machine
software system, and CMS (Cambridge Monitor System), an interactive OS that
ran in CP-67 virtual machines. The CSC folks got hold of a 5100 and were
fascinated in its S/360 instruction set emulation facility. The first
versions of CMS ran in less than 64K on a S/360 model 40. CSC had a
project to take modern CMS, put it on a bit of a diet, and then run it on
the 5100, thus creating an interactive, desktop S/370. Project programming
was done on a large S/370 mainframe. The problem was getting the code onto
the 5100. They eventually wrote an OS/VS link editor in APL that ran on
the 5100 and linked OS/VS object files into executable images stored on the
5100's floppy disk. They wrote a APL shared variable module called
delta-S360 that caused the Palm processor's S/360 emulator to stop
emulating the APL interpreter and instead emulate instructions loaded into
an APL variable passed as a parameter to delta-S360.
The APL link editor ran as slow as death. It processed one object file
card image every 10 seconds or so and took hours to link a program. I was
given the task of writing an OS/360 link editor that could run standalone
on the 5100's S/360 emulator. It was invoked from the APL interpreter by a
delta-S360 call. With the APL interpreter out of the picture, my link
editor ran the floppy disk as fast as it could go and reduced link time
from hours to minutes.
Knuth was right about multiple levels of emulation/interpretaton.
-Paul W.
8:32 p.m.
New subject: [TUHS] Re: machine code translation,as mental architecture models
On Thu, 11 Jul 2024, Paul Winalski wrote:
The lowest end S/360, the model 25, was actually a
16-bit machine.
Wouldn't that be the /20? 16-bit, 8 registers, no FP (but did have a HALT
instruction), etc...
-- Dave
12 Jul
12 Jul
4:23 p.m.
New subject: [TUHS] Re: machine code translation,as mental architecture models
On Thu, 11 Jul 2024, Paul Winalski wrote:
Yes, that was precisely my point, and thank you for
stating it more clearly
and concisely than I did. The VAX MACRO compiler takes in VAX assembly
source code, not binary VAX instructions.
Does anyone know how extensively they used the macro facilities? You can
write much higher level stuff as macros than as single instructions, which
makes it a lot easier to do efficient translation. For example, on OS/360
you'd write a GET macro to retrieve the next record (or a pointer to it)
from a file, which was a lot easier to figure out than the control blocks
and subroutine calls the macros expanded into.
were done completely in hardware--no microcode. S/360
models 67 and down
were microcoded. The lowest end S/360, the model 25, was actually a 16-bit
machine.
The /25 and /30 were 8 bits internally and as slow as you would expect,
but were still full implementations of S/360. IBM sold a lot of them.
The /40 was 16 bits, /50 32 bits, and /65 64 bits. The later /85 was
roughly a /65 reimplemented in faster logic with a cache and 128 bit
memory, making it as fast as the more expensive /91 for programs that
didn't use a lot of floating point. The /85 was microcoded and could
emulate a 7094.
All the S/360s from the 65 and down had microcoded emulators for
2nd generation IBM architectures such as the 1400.
The emulators usually
ran faster than the real hardware.
They always ran faster, company policy. That's why you needed a 360/65 to
emulate a 7094.
The emulators were there to accommodate data centers
that had lost their
source code for mission critical applications.
Partly that, more that the emulators allowed the customers to spread the
conversion work out partly before they got the new machine, partly after.
Remember that the new machine was faster and had better peripherals.
Since it was built with more modern components it may well have been
cheaper to rent.
Knuth was right about multiple levels of
emulation/interpretaton.
The 5100 was indeed a marvel, but it was a very slow one.
Regards,
John Levine, johnl(a)taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly
4:51 p.m.
New subject: machine code translation,as mental architecture models
On Fri, Jul 12, 2024 at 12:23 PM John R Levine <johnl(a)taugh.com> wrote:
On Thu, 11 Jul 2024, Paul Winalski wrote:
and in
the operating system. All of the low-level system calls for user
programs were implemented and documented as macros. The OS assembly code
made heavy use of macros as well.
Outside the VAX/VMS development group, BLISS was DEC's standard
implementation language. In the development organizations I worked in
(software development tools and compilers), we did almost zero programming
in assembly code. The only assembly-level programming I ever did was the
innermost loop of my VAX/VMS Mandelbrot set program and a device driver to
implement Unix-style pipes on VMS. I did the latter in assembly code only
because there was no documented HLL interface for writing VAX/VMS device
drivers.
As Clem Cole observed, Dave Cutler hated BLISS with a passion. All of his
work on the VAX/VMS operating system was in assembly code. Because of this
there is still a large amount of OpenVMS still written in VAX MACRO. But
the RMS (Record Management Services, the VMS file system user interface)
group wrote all of their code in BLISS.
After leaving the VMS group Dave went on to design and implement a common
compiler back end for VAX, the VAX Code Generator (VCG), which was the
optimizer and back end for VAX PL/I, VAX C, and VAX Ada. The earlier
generations of VAX/VMS compilers (VAX Fortran, VAX Pascal, VAX COBOL) each
had their own back ends. Cutler later did get HLL religion. His real-time
operating system for the VAX, VAXeln, was written almost entirely in Pascal.
-Paul W.
Yes, that was precisely my point, and thank you
for stating it more
clearly
and concisely than I did. The VAX MACRO compiler
takes in VAX assembly
source code, not binary VAX instructions.
Does anyone know how extensively they used the macro facilities? You can
write much higher level stuff as macros than as single instructions, which
makes it a lot easier to do efficient translation. For example, on OS/360
you'd write a GET macro to retrieve the next record (or a pointer to it)
from a file, which was a lot easier to figure out than the control blocks
and subroutine calls the macros expanded into.
Macros were used very extensively in VAX MACRO, both for user programming
5:02 p.m.
New subject: machine code translation,as mental architecture models
On Fri, 12 Jul 2024, Paul Winalski wrote:
Oh, no wonder the translator worked so well.
Macros were
used very extensively in VAX MACRO, both for user programming
and in the operating
system. All of the low-level system calls for user
programs were implemented and documented as macros. The OS assembly code
made heavy use of macros as well. Outside the VAX/VMS development group, BLISS was
DEC's standard
implementation language. In the development organizations I worked in
(software development tools and compilers), we did almost zero programming
in assembly code. ...
When I was at Yale we did a fair amount of programming in BLISS-36 which
was a pretty nice language once you wrapped your brain around some of its
quirks like needing a dot for every memory reference.
Our Vaxes ran Unix so it was all C other than a few things like tracking
down a bug in the 11/750's microcode that broke an instruction in the
inner loop of printf(). I had managed to get a cross-compiling
environment working an a PDP-11 but Bill Joy found the bug at the same
time so we used his patched version.
Regards,
John Levine, johnl(a)taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly
8:52 p.m.
New subject: machine code translation,as mental architecture models
On Sat, 12 Jul 2024, John R Levine wrote:
Our Vaxes ran Unix so it was all C other than a few
things like tracking
down a bug in the 11/750's microcode that broke an instruction in the
inner loop of printf(). [...]
Do tell...
-- Dave
10:03 p.m.
New subject: machine code translation,as mental architecture models
It appears that Dave Horsfall <dave(a)horsfall.org> said:
On Sat, 12 Jul 2024, John R Levine wrote:
The details are a litle dim after 45 years, but there was a MOVTUC
instruction in the inner loop of printf that scanned for the null at
the end of the string. The /750 had a microcode bug that didn't
matter for the way DEC's software used it but broke the libc and
I think also the kernel version. MOVTUC sets six registers and
we probably used one they didn't.
Bill replaced it with a few simpler instructions and the comment
; Comet sucks
R's,
John
PS: For you young folks, Comet was DEC's internal project name for the /750.
Our Vaxes ran Unix so it was all C other than a
few things like tracking
down a bug in the 11/750's microcode that broke an instruction in the
inner loop of printf(). [...]
Do tell... 
13 Jul
13 Jul
12:25 a.m.
New subject: machine code translation,as mental architecture models
I think my work was in the summer of 1979 or possibly 1980, before the
FRS in Oct. 1980. People were happy because mass production of the
ucode ROMs hadn't started yet. I was using one of the 2~4 11/750
prototypes which were on concrete blocks in a lab, and the 11/730
breadboard was standing up on a table (no box) with its boards fanned
out. And it was during the gasoline shortage. Maybe I'll find
something about it later.
- Aron
On 7/12/24 18:03, John Levine wrote:
It appears that Dave Horsfall
<dave(a)horsfall.org> said:
On Sat, 12 Jul 2024, John R Levine wrote:
The details are a litle dim after 45 years, but there was a MOVTUC
instruction in the inner loop of printf that scanned for the null at
the end of the string. The /750 had a microcode bug that didn't
matter for the way DEC's software used it but broke the libc and
I think also the kernel version. MOVTUC sets six registers and
we probably used one they didn't.
Bill replaced it with a few simpler instructions and the comment
; Comet sucks
R's,
John
PS: For you young folks, Comet was DEC's internal project name for the /750.
Our Vaxes ran Unix so it was all C other than a
few things like tracking
down a bug in the 11/750's microcode that broke an instruction in the
inner loop of printf(). [...]
Do tell...
12:27 a.m.
New subject: machine code translation,as mental architecture models
On Sat, 12 Jul 2024, John Levine wrote:
The details are a litle dim after 45 years, but there
was a MOVTUC
instruction in the inner loop of printf that scanned for the null at the
end of the string. The /750 had a microcode bug that didn't matter for
the way DEC's software used it but broke the libc and I think also the
kernel version. MOVTUC sets six registers and we probably used one they
didn't.
I am reminded of the time when overlapped seeks on the RK-11 were
implemented in Unix, per the Peripherals Handbook.
They didn't work.
DEC responded with "Well, you were running Unix!"...
We then ran "DECEX" (their own diagnostic tool), which indeed failed
on overlapped seekson the RK-11.
It turned out that DEC never used overlapped seeks in their own stuff...
-- Dave
1:19 a.m.
New subject: machine code translation,as mental architecture models
On Fri, Jul 12, 2024 at 6:10 PM John Levine <johnl(a)taugh.com> wrote:
PS: For you young folks, Comet was DEC's internal project name for the
/750.
The first generation of VAXen were the 11-780 (codenamed Star), the 11/750
(Comet) and the 11/730 (Nebula).
The 11/780's successor was to be implemented in ECL and was codenamed
Venus. Along with Venus were a two-board VAX implementation (Gemini) and a
one-board VAX (Scorpio). Gemini was cancelled. Venus had lots of schedule
slippage but was eventually released, but under the new 4-digit VAX
branding system (it was originally slated to be called the 11-790).
Scorpio became the VAX 8000.
-Paul W
9:57 a.m.
New subject: machine code translation, as mental architecture models
Hi,
John and Paul wrote:
Macros were used very extensively in VAX MACRO, both for user
programming and in the operating system. All of the low-level
system calls for user programs were implemented and documented as
macros. The OS assembly code made heavy use of macros as well.
Oh, no wonder the translator worked so well.
Well, doesn't it depend on whether VAX MACRO kept the macros as
high-level entities when translating them, or if it processed macros in
the familiar way into instructions that sat at the same level as
hand-written ‘assembler’. I don't think this thread has made that clear
so far.
--
Cheers, Ralph.
The VAX MACRO
compiler takes in VAX assembly source code, not
binary VAX instructions.
Does anyone know how extensively they used the macro facilities?
You can write much higher level stuff as macros than as single
instructions, which makes it a lot easier to do efficient
translation.
2:25 p.m.
New subject: machine code translation, as mental architecture models
On Sat, Jul 13, 2024 at 5:57 AM Ralph Corderoy <ralph(a)inputplus.co.uk> wrote:
John and Paul wrote:
Oh, no wonder the translator worked so well.
Well, doesn't it depend on whether VAX MACRO kept the macros as
high-level entities when translating them, or if it processed macros in
the familiar way into instructions that sat at the same level as
hand-written ‘assembler’. I don't think this thread has made that clear
so far.
The non-VAX compilers for Macro-32 do expansion. That is, they don't
just recognize macros and treat them as intrinsics or primitives that
are specially optimized. Macro is not C, where you can treat "library"
functions defined in the standard specially. John Reagan, of VSI, has
posted about XMACRO (the Macro compiler for x86) in various places.
For instance, here:
https://comp.os.vms.narkive.com/F05qmMaD/x86-cross-tools-kit#post4
This post, and others, suggests to me that XMACRO is definitely doing
macro expansion before compilation proper.
- Dan C.
> The VAX MACRO compiler takes in VAX assembly
source code, not
> binary VAX instructions.
Does anyone know how extensively they used the macro facilities?
You can write much higher level stuff as macros than as single
instructions, which makes it a lot easier to do efficient
translation.
Macros were used very extensively in VAX MACRO, both for user
programming and in the operating system. All of the low-level
system calls for user programs were implemented and documented as
macros. The OS assembly code made heavy use of macros as well.
5:27 p.m.
New subject: machine code translation, as mental architecture models
On 7/13/24 10:25, Dan Cross wrote:
On Sat, Jul 13, 2024 at 5:57 AM Ralph
Corderoy<ralph(a)inputplus.co.uk> wrote:
> John and Paul wrote:
> Well, doesn't it depend on whether VAX MACRO kept the macros as
> high-level entities when translating them, or if it processed macros in
> the familiar way into instructions that sat at the same level as
> hand-written ‘assembler’. I don't think this thread has made that clear
> so far.
The DEC assemblers [sic] for the PDP-11 and VAX-11 did macro expansion
as text substitution.
AA-V027A-TC PDP-11 MACRO-11 Language Reference Manual
Section 7.1 (PDF file p 115, original document p 7-1) says
"Macro expansion is the insertion of the macro source lines into the
main program."
Note: This manual also discusses concatenation with macro arguments and
(on p 120 or p 7-6) macros that define other macros.
AA-D032B-TE VAX-11 MACRO Language Reference Manual
The beginning of chapter 6 (p 119 or p 6-1) says the same thing.
At least in those assemblers, macros were not inline procedures.
Google can find manuals for other assemblers. For even earlier history, see
https://github.com/PDP-10/its/issues/2032
http://www.bitsavers.org/pdf/mit/rle_pdp1/memos/PDP-1_MIDAS.pdf
12 Jul
12 Jul
5:03 p.m.
New subject: machine code translation,as mental architecture models
On 2024-07-12 12:51, Paul Winalski wrote (in part):
Macros were used very extensively in VAX MACRO, both for user
programming and in the operating system. All of the low-level system
calls for user programs were implemented and documented as macros. The
OS assembly code made heavy use of macros as well.
I recall one place that used macros that were hundreds of lines long.
They were a nightmare to maintain as they were so rigid -- not the right
way write macros.
S.
6:01 p.m.
New subject: machine code translation,as mental architecture models
On Fri, Jul 12, 2024 at 1:03 PM Stuff Received <stuff(a)riddermarkfarm.ca>
wrote:
I recall one place that used macros that were hundreds of lines long.
They were a nightmare to maintain as they were so rigid -- not the right
way write macros.
DEC BLISS has the most powerful macro facility that I've ever seen in any
programming language. Ward Clark of DEC's Development Methods and Software
Tools Group wrote a set of macros called "Portable BLISS" that provided a
uniform means to access basic operating system services such as sequential
file I/O and heap memory management. There were Portable BLISS
implementations for much of the vast zoo of DEC operating systems (RSX-11,
RSTS, RT-11, TOPS-10, TOPS-20, VMS). Some of the Portable BLISS macros
were over a thousand lines long. They were all obscure and hard to
maintain.
Unix users will be familiar with the Obfuscated C Contest. Within DEC
Engineering there was an Obfuscated BLISS Contest one year. The winner was
Stan Rabinowitz, who used macros to produce a text that looked nothing like
BLISS syntax. Stan said it would compile cleanly except for one semantic
error and challenged us to figure out what that was. It turned out to be
an illegal uplevel reference.
-Paul W.
8:35 p.m.
New subject: machine code translation,as mental architecture models
It appears that Paul Winalski <paul.winalski(a)gmail.com> said:
Take a look at PL/I. Its preprocessor lets you use a large subset of
PL/I including if, goto, and do loops and most of the string and
arithmetic operators to write functions that run at compile time and
put their results into the source file.
And don't forget the IBM macro assembler. In macros along with all of
the usual loops and local and global variables and subscripted
parameters and conditional expressions and string processing, it had
the AREAD op which read the next line from the source file and put it
in a macro-time variable, thereby letting the macro define its own
syntax from scratch if you wanted. There was also PUNCH to write a
line directly to the object file.
DEC BLISS has
the most powerful macro facility that I've ever seen in any
programming
language. ...
13 Jul
13 Jul
9:26 a.m.
New subject: machine code translation, as mental architecture models
Hi,
John Levine wrote:
Take a look at PL/I. Its preprocessor lets you use a
large subset of
PL/I including if, goto, and do loops and most of the string and
arithmetic operators to write functions that run at compile time and
put their results into the source file.
Another approach is for the language to offer evaluation at compile
time.
- FORTH had COMPILE and IMMEDIATE to switch modes; there's POSTPONE as
well now.
- Zig has comptime.
https://ziglang.org/documentation/master/#comptime
https://ziglang.org/documentation/master/#Case-Study-print-in-Zig
--
Cheers, Ralph.
12 Jul
12 Jul
6:54 p.m.
New subject: machine code translation,as mental architecture models
Outside of product development, the internal DECSIM multi-level logic
simulator and its CLI and compilers were also written in BLISS. Most of
these started out in BLISS-36 but they were all brought together with a
fault simulation kernel on the VAX to take advantage of the larger
address space.
Some other internal VLSI CAD tools were also written in BLISS, although
a few were written in more common languages. The first (AFAIK)
retargetable microcode assembler at DEC, MICRO, was written in MACRO-10;
sources are on the net. The next generation MICRO2 was written in BLISS
and AFAIK the binary is available but the sources have sadly been lost.
<Insert obvious plea here.>
Re: Paul's comment about the portable I/O library in BLISS: When I first
encountered C, I used the Unix system calls to do I/O. The "standard
I/O" library was a very important addition to C's usability. Similarly,
when I first encountered BLISS-10, I had to type in a package from the
DECUS documentation to call TOPS-10 to do I/O. Getting the "XPORT" I/O
(etc.) library for Common BLISS was invaluable for the CAD tools.
People have repeatedly omitted I/O from the initial design of a language
and relegated it to a library, but even if the language is intended to
be used for a kernel, people learning the language ("Hello, world!") or
using the language above the kernel for utilities will need to do I/O,
so this library should be included in the language design from the very
beginning. I think this lesson has been learned by now. Its fallout is
recognized in Stroustrup's quote of a Bell Labs axiom, "Library design
is language design, and vice versa."
- Aron
On 7/12/24 12:51, Paul Winalski wrote:
Outside the VAX/VMS development group, BLISS was
DEC's standard
implementation language. In the development organizations I worked in
(software development tools and compilers), we did almost zero
programming in assembly code.
9:49 p.m.
New subject: machine code translation,as mental architecture models
On Fri, 12 Jul 2024, Aron Insinga wrote:
Re: Paul's comment about the portable I/O library
in BLISS: When I first
encountered C, I used the Unix system calls to do I/O. The "standard
I/O" library was a very important addition to C's usability. [...]
As was the "Portable I/O Library" preceding it.
-- Dave
13 Jul
13 Jul
6:50 p.m.
New subject: machine code translation, as mental architecture models
According to Ralph Corderoy <ralph(a)inputplus.co.uk>:
It was a macro assembler. The macros generated assembler statements
that got assembled the normal way. I agree with the person that every
macro assembler I've ever seen did that. A semi-exception is the IBM
assembler that also had a PUNCH statement that put records into the
object file but I think that was only used to pass commands to the
linker.
The more relevant question is how they used the macros. If the macros
were used consistently for semantically higher level things, the
translator can use the semantics of the macros when translating.
--
Regards,
John Levine, johnl(a)taugh.com, Primary Perpetrator of "The Internet for
Dummies",
Please consider the environment before reading this e-mail. https://jl.ly
Oh, no wonder
the translator worked so well.
Well, doesn't it depend on whether VAX MACRO kept the macros as
high-level entities when translating them, or if it processed macros in
the familiar way into instructions that sat at the same level as
hand-written ‘assembler’. I don't think this thread has made that clear
so far.
7:27 p.m.
New subject: machine code translation, as mental architecture models
On Sat, Jul 13, 2024 at 2:50 PM John Levine <johnl(a)taugh.com> wrote:
The more relevant question is how they used the
macros. If the macros
were used consistently for semantically higher level things, the
translator can use the semantics of the macros when translating.
There is an assembly macro for each of the user-mode VMS system services
These are
wrappers around the RTL subroutines. So, for example, the $QIO
(queue I/O) macro is a wrapper around a call to the SYS$QIO runtime library
routine. There is a similar set of macros for the RMS (record management
services--the VMS file I/O system) data structures and routines.
The VAX MACRO compiler could perhaps use the semantics of the macros when
translating, but it doesn't. It has to support semantics-free macro
expansion since users can write their own macros, so there is little point
in building semantic knowledge into the translator.
-Paul W.
7:42 p.m.
New subject: machine code translation, as mental architecture models
On Sat, Jul 13, 2024 at 2:50 PM John Levine <johnl(a)taugh.com> wrote:
According to Ralph Corderoy
<ralph(a)inputplus.co.uk>:
It was a macro assembler. The macros generated assembler statements
that got assembled the normal way. I agree with the person that every
macro assembler I've ever seen did that. A semi-exception is the IBM
assembler that also had a PUNCH statement that put records into the
object file but I think that was only used to pass commands to the
linker.
This is conflating two different things over the life of the MACRO-32
language. It certainly started out as a macro assembler, but with the
introduction of the Alpha, DEC turned it into a true compiler (where
the source language happens to be VAX assembly language) that
generated native code for whatever machine it was compiled for: VAX or
Alpha; using the GEM backends.
When Compaq started the port to Itanium, they employed the same
technique; this was maintained by HP.
VSI has shifted to using LLVM as their compiler backend for the x86_64
port, with ELF as the executable file format (and presumably for
object files as well). As I understand the current state of affairs,
there is a GEM to LLVM layer that interfaces between the still-GEM
output of compiler frontends and the LLVM backend, that performs
native code generation. Macro, in particular, bypasses most of the
LLVM optimization layer.
Oh, no
wonder the translator worked so well.
Well, doesn't it depend on whether VAX MACRO kept the macros as
high-level entities when translating them, or if it processed macros in
the familiar way into instructions that sat at the same level as
hand-written ‘assembler’. I don't think this thread has made that clear
so far. The more relevant question is how they used the
macros. If the macros
were used consistently for semantically higher level things, the
translator can use the semantics of the macros when translating.
As has been mentioned, they don't do this. The output of fully
expanded MACRO-32 is basically VAX assembly language, which is then
compiled in a manner similar to how one would compile preprocessed C.
- Dan C.
14 Jul
14 Jul
1:09 a.m.
New subject: machine code translation, as mental architecture models
Back to topic of machine code translation, I stumbled over this paper
describing DEC's migration from VAX (VMS) or MIPS (Ultrix) to Alpha (VMS
or OSF/1, respectively) for user-mode applications. The paper states
that this included hand-coded assembly language applications.
https://web.stanford.edu/class/cs343/resources/binary-translation.pdf
1:46 a.m.
New subject: machine code translation, as mental architecture models
On 7/13/24 15:42, Dan Cross wrote:
The output of fully expanded MACRO-32 is basically VAX
assembly language, which is then
compiled in a manner similar to how one would compile preprocessed C.
- Dan C.
Although, a minor point, the macro definition & expansion, conditional
assembly, etc. are all done on the fly as the assembly language source
file is read. There's no separate macro processing pass like on Unix
when, if the first character of the .c file was '#', cpp read the .c
file and generated a .i file to pass to the compiler proper (which let
cpp be used on Pascal, assembler, and who knows what else).
- Aron
4:16 p.m.
New subject: machine code translation, as mental architecture models
On Sat, Jul 13, 2024 at 4:00 PM Dan Cross <crossd(a)gmail.com> wrote:
This is conflating two different things over the life of the MACRO-32
language. It certainly started out as a macro assembler, but with the
introduction of the Alpha, DEC turned it into a true compiler (where
the source language happens to be VAX assembly language) that
generated native code for whatever machine it was compiled for: VAX or
Alpha; using the GEM backends.
The confusion arises because the name VAX MACRO was used for two
completely
independent and very different software tools.
The first VAX MACRO was the conventional assembler for VAX/VMS, developed
in the mid-1970s. It had conventional macroinstruction capabilities, hence
the name.
An inconveniently large amount of the VAX/VMS operating system code was
written in VAX assembly language. Dave Cutler didn't believe in using
higher level languages for system code, and he despised BLISS (DEC's
standard HLL language for software product development) in particular. To
migrate VMS to Alpha it was decided that the quickest and least error-prone
path was to write a compiler front end that takes VAX MACRO syntax and
generates expanded intermediate language (EIL) for the GEM compiler back
end. GEM then generates the Alpha code and writes it into an object file.
This work was done circa 1990. The name "VAX MACRO" was used for this
compiler as well as for the previous assembler.
When Compaq started the port to Itanium, they employed
the same
technique; this was maintained by HP.
No further work needed to be done in the VAX MACRO compiler front end to
support
Itanium. All that needed to be done was to add Itanium code
generation capability to the GEM back end. After that all GEM-based
compilers acquired Itanium code generation capability. That is the big
advantage of having a common back end for all compilers. Over its lifetime
GEM could generate object files for the MIPS, Alpha, and IA-32 machine
architectures and for the VMS, Ultrix, OSF-1, and Windows operating
systems.
VSI has shifted to using LLVM as their compiler backend for the x86_64
port, with ELF as the executable file format (and
presumably for
object files as well). As I understand the current state of affairs,
there is a GEM to LLVM layer that interfaces between the still-GEM
output of compiler frontends and the LLVM backend, that performs
native code generation. Macro, in particular, bypasses most of the
LLVM optimization layer.
VAX MACRO also bypassed most of GEM's optimization layer as well. And
yes,
most (if not all) of the legacy OpenVMS compilers use a GEM-to-LLVM IL
translator. VSI has more than enough on their plate without taking on a
wholesale rewrite of the IL generators in the compiler front ends.
-Paul W.
12 Jul
12 Jul
4:54 p.m.
New subject: [TUHS] Re: history of machine code translation,as mental architecture models
On Wed, 10 Jul 2024, Dan Cross wrote:
< [[ Knuth described instruction by instruction
simulation ]]
Sure. But we're trying to date the topic here; my point is that
JITing was well known, and simulation was similarly well known; we
know when work on those books started; it doesn't seem that odd to me
that combining the two would be known around that time as well.
Simulators that interpreted a pseudo-machine code date from the early
1950s. They were popular before compilers existed. Fortran and
Flow-matic arrived in 1954, translating languages that were somewhat
familiar to people into machine code, mathematical formulas for Fortran,
business English for Flow-matic. But it would be rather a stretch to
assert that anyone was doing machine code translation in 1954. It was a
while until enough machine code existed for it to be worth thinking about,
and the jump from tranlating from human-ish languages to translating from
machine languages was likely far less obvious at the time than it is in
retrosoect.
On the other hand, there was UNCOL. In the 1950s people quckly realized
that building a compiler for every language for every machine would be a
lot of work, particularly in an era when there was such extreme variation
in machine architecture. So someone came up with UNCOL which was supposed
to be a UNiversal Computer Oriented Language, or what we would call a
universal intermediate language. All of the compilers would translate the
original language to UNCOL, and there wold be an UNCOL to machine language
translator for each machine. This reduced the NxM compiler problem to N+M
if it worked, which of course it didn't because the languages and the
machines were both too varied. But UNCOL to machine code is in the same
ballpark as machine to machine.
CACM articles on UNCOL here:
https://dl.acm.org/doi/10.1145/368892.368915
https://dl.acm.org/doi/10.1145/368919.3165711
I would still love to hear if there are any actual reports or papers or
anecdotes of people doing machine code translation before the 1967 one
that started this discussion.
R's,
John
13 Jul
13 Jul
2:25 a.m.
New subject: machine code translation,as mental architecture models
According to Dave Horsfall <dave(a)horsfall.org>:
I am reminded of the time when overlapped seeks on the
RK-11 were
implemented in Unix, per the Peripherals Handbook.
They didn't work.
Yup, I recall looking at the disk drivers which had a comment saying
that overlapped seeks on RK disks didn't work.
At Yale our PDP-11 had a pair of 2314-style RP drives. Its controller
did work and I stayed up all night one weekend debugging a driver that
queued and sorted requests for the two drives separately and did
separate seeks. It made a lot of difference, something like 30% more
transfers/minute.
--
Regards,
John Levine, johnl(a)taugh.com, Primary Perpetrator of "The Internet for
Dummies",
Please consider the environment before reading this e-mail. https://jl.ly
55
days inactive
58
days old
27 comments
8 participants
participants (8)
-
Aron Insinga -
Dan Cross -
Dave Horsfall -
John Levine -
John R Levine
-
Paul Winalski -
Ralph Corderoy -
Stuff Received