Having done the SysIII 64-bit port to a recent Risc-V chip, I realised that whilst it is an interesting exercise bij itself -- and maybe even useful to students and educators around the world -- it is not ideal as a research tool for analysing Unix from the early 80’s. The address size difference adds some superfluous porting, and the 100x speed difference can hide critical algorithm constraints. Also the complex IO devices are out of character.
For a Risc-V 32 bit target I’ve settled on an FPGA implementation from the University of Tokyo. I’ve somewhat modified the system to work with the open source Yosys/NextPNR tool chain. It now implements a Linux-capable SoC with a full MMU, a 4-way cache and SD card driver in less than 4,000 lines of plain Verilog (compiling to about 14K LUTs). In a way, the code has a 6th edition feel to it: it covers a real and usable system and the code can be understood in a couple of days -- or a semester for a student who is new to the concepts involved.
https://gitlab.com/r1809/rvsoc/-/tree/main/doc
So far I have Linux and XV6 (https://gitlab.com/r1809/xv6-rv32) running, but have not started on SysIII yet.
Usefully for my use case this system is not very fast, completing an instruction in on average 10 clocks. Still, when running at 40MHz it is about 2 or 3 times as fast as a VAX11/780, which is similar to the systems of the mid-80’s. Even at this speed, a single user console Linux is surprisingly usable. By the way, funny to realise that ‘Unix/Linux capable’ has been a marketing slogan for system capability for 40 years now.
There is a short video clip with a demonstration (but running at 100MHz) here: https://youtu.be/Kt_iXVAjXcQ
Due to its simple design, the main CPU only uses some 30% of the cache memory bandwidth and it should not be all that hard to add a second CPU to the system (the CPU already supports the Risc-V atomic operations), and this could be a nice target for studying the early Unix multi-processor designs (e.g. VAX/BSD & 3B2/SVR3).
I find it an intriguing thought that the chip technology of the early 80’s (let’s say the technology used for the Bellmac-32 or M68K) would probably have sufficed to build a CPU similar to the one used in this FPGA design.
As the topic of this post is on a tangent from the focus of this list, I would recommend that any follow-ups not related to the history of Unix are sent off list.
Porting the SysIII kernel to a D1 board (https://www.aliexpress.us/item/3256803408560538.html) began with a port of XV6, in order to test the tool chain and to get comfortable with this target. Michael Engel had already ported XV6 to the D1 chip a few months before (https://www.uni-bamberg.de/fileadmin/sysnap/slides/xv6-riscv.pdf) giving a ready base to work with.
The main new effort was to add code to initialise the DRAM controller and the SD Card interface, and to have a simple boot loader. Such code is available from the manufacturer board support package (BSP), although in this case the DRAM controller code was only available as assembler compiler output and had to be reverse engineered back into C. In general I was surprised to see how big and unwieldy the BSP code is; maybe the code just looks sloppy because it has to deal with all kinds of edge cases - but I can also imagine that it accumulates cruft as it is ported from SoC to SoC by the manufacturer.
The resulting XV6 source tree is here: https://gitlab.com/pnru/xv6-d1
This version automatically boots from the SD Card on the board.
With that out of the way, the ancient Unix port was relatively easy. It would seem to me that the SysIII code base has a lot of clean-up work in it that still pays off today. The code compiles to a 64-bit target with minimal updates, which I think is a compliment to the engineers that worked on it. Probably using a LLP64 compiler also helped. In order to bring something up quickly, I modified the kernel to load ELF binaries, so that I could use proven material from the XV6 port (such as a minimalistic init and shell).
Initially, I just replaced VAX memory management with page table code taken from XV6 (i.e. no VM or swapping). Working with Risc-V page tables gives much simpler code, but I have a deeper appreciation of the VAX paging design now: for the type of code that was run in 1980, the VAX design enables very small page tables with just a few dozen entries. In contrast, for the 3-level page tables of 64-bit Risc-V I end up with 7 pages of page table of 4KB each, or 28KB -- that is larger than the memory image of many SysIII programs. If I move the ‘trampoline' to just above the stack in virtual memory it could be 5 pages instead of 7, but the overall picture remains the same. The 68020 or ‘030 MMU could be configured to have various page sizes -- this looked byzantine to me when I first saw it, but it makes more sense now.
Next I replaced the VAX scatter paging / partial swapping code, keeping the same methodology. I noticed that there is still confusion over memory management in 32V and SysIII (and implicitly SVR1,R2). The original 32V as described in the London/Reiser paper used V7 style swapping. This code can be found as ‘slowsys’ in the surviving source (https://www.tuhs.org/cgi-bin/utree.pl?file=32V/usr/src/slowsys) It was quickly (Mar vs. Jan 1979) replaced by the scatter loading / partial swapping design already hinted at in the paper (source is in 32V/usr/src/sys). Unfortunately, the “32V uses V7 swapping” meme lives on.
In scatter paging, the pages are no longer allocated continuous in physical memory but new pages are taken from a free list and expansion swaps are not usually needed. Also, when a process is swapped out, it is not fully swapped out, but just enough pages to make room for the new process. When it is readied to run again, only the partial set needs to be reloaded. In the VAX context, scatter paging and partial swapping are quite effective and I think competitive with demand paging for the 25-100KB processes that were in use at the time. As I mentioned in the post on the toolchain, the Plan 9 C compiler can easily use 1MB of memory and in a 4MB of core context, this trashes the algorithm; it starts to behave much like traditional swapping. The reason for this is that the entire process must be in memory in order to be able to run and the algorithm cannot recognise that a much smaller working set is needed. The implicit assumption of small processes can also be seen in the choice to limit partial swaps to 4KB per iteration (8 VAX pages).
For handling processes with a large memory footprint but a small working set a true demand paged VM approach is needed. The simplest such approach appears to be Richard Miller’s work for SVR1 (see June 1984 Usenix conference proceedings, "A Demand Paging Virtual Memory Manager for System V"). This is a very light touch implementation of demand paging and it seems that enough bits and pieces survive to recreate it.
The journey through the memory code made it clear again that in SysIII and before, the memory code is scattered over several locations and not so easy to fathom at first glance. It would seem that in SysV/68 an attempt was made to organise the code into separate files and with a more defined API. It does not seem to have carried through. Maybe this was because the MMU’s of the 1980-1985 era were all too different to be efficiently abstracted into a single framework.
Beyond SysV/68, were there any other attempts in the early 80’s to organise and abstract the kernel memory management code (outside BSD)?
After initially gearing up to use the Motorola 68020 or 68030 as a porting target for a study of Unix in the 1980-1985 era, I reconsidered and used Risc-V as a target instead. As the original RISC and MIPS projects were contemporaneous with early 32-bit Unix (32V, BSD, SysIII and SVr1,r2) it seems appropriate and there is currently considerable interest (hype?) around Risc-V.
From a programming perspective, the Risk-V ISA does not feel (at least to me) all that different from what was current in the early 80’s — the number of WTFs/min is low. The modularity is a pleasant surprise, as is the observation that the 32-bit and 64-bit instruction sets are almost identical and that compressed instructions mingle nicely with full size ones. The MMU design appears straightforward. Maybe this is because the ISA is still relatively new and has not acquired much historical baggage at this point in its lifespan, but it also seems to be a good synthesis of insights gained over the last 4 decades and applied with a sense of minimalism.
At first I was thinking to create a toolchain based on pcc or pcc2 for the SysIII porting effort, based on some preparation I had done when I was still thinking about 68030 as a target (the surviving Blit code includes a pcc-based 68000 compiler and the SysV/68 source archive contains a pcc2-based compiler). Before I got underway with that, I came across a presentation Richard Miller had done about his Risc-V compiler:
https://riscv.org/news/2020/10/a-plan-9-c-compiler-for-rv32gc-and-rv64gc/
Richard was kind enough to share the source code for his Risc-V back-end. The first complication was that the source code assumes that it will be running inside a Plan-9 environment, whereas I was looking for a Unix/Posix environment. Luckily somebody already had assembled the libraries needed for this:
https://github.com/aryx/fork-kencc
I’m not sure where it came from, but I would assume it has some roots in the "Plan-9 from user space" effort. From this work I extracted the minimum needed to get the C compiler working and to build from scratch. The libraries mostly just worked. The compiler was a bit harder: the source code assumes a LLP64 model in a few places and compiling this with clang (which uses a LP64 model) introduces issues in a handful of places. Other than this initial hurdle, the compiler and tools have worked flawlessly, both for 64-bit code and for 32-bit code, and have been a joy to use. One particular nicety is that Plan 9 style "abstract assembler" source for 64-bit code is even more identical to its 32-bit variant than with the mainstream Risc-V assembler syntax. My repo for the tool chain is here:
https://gitlab.com/pnru/riscv-kencc
Initially, my expectation was that I could only use these compilers as cross-compilers and that I would need to do a pcc2 version for native compilation at some point. However, when I saw how small and fast the tools were, I experimented with using them on SysIII. Much to my surprise the effort required was absolutely minimal, all that was needed was adding a dozen simple standard functions in libc (see here: https://gitlab.com/pnru/SysIII_rv64/-/tree/master/libc/compat) and adding the ‘dup2' sys call. I had not expected that SysIII was so close to the Unix systems of the 1990’s in this regard. This result inspires ideas for future projects: as I plan to add an 8th edition style file system switch anyway, maybe it will not be all that hard to make the Plan-9 debugger work on this “SysIII+” as well.
Another observation has been that the code size of binaries compiled for Risc-V by this tool chain is almost the same as those compiled for the VAX by pcc (the Risc-V ones are maybe 10-20% larger). This is using the compressed instructions where possible. This is I think another indication that both the Risc-V ISA and the tool chain are quite well done.
The one less positive surprise has been the memory use of the compiler. Even on a relatively simple program file it will quickly use 1 megabyte or more of ram. I understood from Richard that this is because the compiler only malloc()’s and never free()’s by design. This has been a mixed blessing. Such large memory images don’t work all that well with the "scatter paging + partial swapping" memory management of SysIII when memory is constrained to say 4MB of core to mimic the systems of the era. On the other hand, parallel compiling the kernel on SysIII itself heavily exercises the partial swapping code and has been a great test case for debugging.
Many thanks to Ken, Rob, Richard and all the others who created this fine tool chain!
> James Johnston:
>> Yeah, but Rob, where was Fred? I was there in Acoustics Research (not
>> 127!) then, using R70 for UCDS.
> Rob Pike:
> Not in (1)127 yet. He was transferred in some time after I arrived. Not
> sure quite when. Mid-80s maybe.
> =====
> Early-to-mid 1980s. ftg was already there when I interviewed in early 1984.
> Norman Wilson
In 1980 Fred was a stalwart of the computer center. There he exhibited
great creativity, including the invention of "quest" for sniffing out
security lapses throughout the BTL computer network. His findings
underpinned the headline claim of the Labs' first computer-security
task force (1982), "It is easy and not very risky to pilfer data from
Bell Labs computers".
Doug
James Johnston:
> Yeah, but Rob, where was Fred? I was there in Acoustics Research (not
> 127!) then, using R70 for UCDS.
Rob Pike:
Not in (1)127 yet. He was transferred in some time after I arrived. Not
sure quite when. Mid-80s maybe.
=====
Early-to-mid 1980s. ftg was already there when I interviewed in early 1984.
Norman Wilson
Toronto ON
From [1]:
The X11 Conservancy Project (X11CP) pulls together the disparate set of programs which were being written between the very late 80s, and early 90s -- usually for Unix and Linux.
…snip…
As the Internet expanded and Linux distributions became established, certain FTP sites were largely used to host some of the more established programs, as well as those found in the LSM.
…snip…
But the early dawn of free software, especially around applications written for X11, using Motif and XT and other widget libraries has now mostly been consigned to obscurity.
With X11 itself now under threat of no longer being developed in favour of Wayland, these applications are going to be harder to run and be discovered.
Hence, the X11CP is designed to be a central place for hosting the sources of these applications, and to showcase their unique history and properties. In keeping this software active, it will help keep an important historical point alive.
[1] https://x11cp.org/
— Michelangelo
> From: Phil Budne
> The cover page has:
> ...
> Upper right corner:
> PA-1C300-01
> Section 1
> Issue 1, January 1976
> AT&TCo SPCS
I have a very similar manual; I got it a long time ago, and no longer recall
how I came by it. Minor difference: mine is for PD-1C301-01, and at the
bottom of the page, it says "ISSUE 1 1/30/76", followed by a prominent trade
secret notice.
TUHS has a copy of this version, here:
https://www.tuhs.org/Archive/Distributions/USDL/unix_program_description_ja…
The README file in that directory:
https://www.tuhs.org/Archive/Distributions/USDL/README
speculates that "this is PWB/1.0" but admits "this has not yet been
confirmed". It's not PWB1, it's stock V6. If you look at the writeup of
sys1$exec(), on pg. 39 of the PDF, you'll see it describing how arguments are
copied into a disk buffer; that right there is the tip-off. In PWB1 (whose
source we do have):
https://minnie.tuhs.org/cgi-bin/utree.pl?file=PWB1/sys/sys/os/sys1.c
you'll see that PWB1 accumulates the arguments in a chunk of swap space.
V6 _does_ use a disk buffer for this:
https://minnie.tuhs.org/cgi-bin/utree.pl?file=V6/usr/sys/ken/sys1.c
So this is for V6.
Noel
The October 1984 BSTJ article by Felton, Miller and Milner
https://www.bell-labs.com/usr/dmr/www/otherports/ibm.pdf
Describes an AT&T port of UNIX to System/370 using TSS/370
underpinnings as the "Resident System Supervisor" and used as the 5ESS
switching system development environment.
I also found mention at http://www.columbia.edu/~rh120/ch106.x09
chapter 9 of http://www.columbia.edu/~rh120/ with footnote 96:
Ian Johnstone, who had been the tutor at University of New
South Wales working with Professor John Lions, was one of the
researchers invited to Bell Labs. He managed the completion at
AT&T Bell Labs of the port of Unix to the IBM 370 computer. See
"Unix on Big Iron" by Ian Johnstone and Steve Rosenthal, UNIX
Review, October, 1984, p. 26. Johnstone also led the group that did
the port to the AT&T 2B20A multiprocessor system.
I found
https://ia902801.us.archive.org/3/items/Unix_Review_1984_Oct.pdf/Unix_Revie…
"BIG UNIX: The Whys and Wherefores" (pdf p.24), which only offers rationale.
Also:
"IBM's own involvement in Unix can be dated to 1979, when it
assisted Bell Labs in doing its own Unix port to the 370 (to
be used as a build host for the 5ESS switch's software). In
the process, IBM made modifications to the TSS/370 hypervisor
to better support Unix.[12]"
at https://en.wikipedia.org/wiki/IBM_AIX#cite_ref-att-s370-unix_12-0
Is there any other surviving documentation about the system?
Any recall of what branch of AT&T UNIX it was based on?
Thanks!
Phil
> From: Bakul Shah
> There is a further para:
> Reducing external memory fragmentation to zero by utilizing the VAX-
> 11/780 memory mapping hardware for scatter loading is high on the list
> of things to do in the second implementation pass.
I'm curious as to exactly what is meant by "external memory"? They must mean
memory on the Synchronous Backplane Interconnect:
http://gunkies.org/wiki/Synchronous_Backplane_Interconnect
I.e. what most of us would call 'main memory'.
If this code didn't even allocate main memory by pages, instead of in
process-size blocks, it sounds like it's much like 32V (or is it 32V that's
being discussed; I thought this thread had moved on to the Reiser demand
paging version - my apologies it I've gotten lost).
Also, this note:
http://gunkies.org/wiki/Talk:CB-UNIX
from Dale DeJager (which he kindly gave me permission to post) gives a fair
amount of detail on the relationship between the Research and CB/UNIX
versions, with a brief mention of USG - precisely the era, and relationships,
that are so poorly documented. Interestingly, he indicates that the early
versions of what later became CB/UNIX used something in the V1/V3 range (V4
was the first one in C), so it dates back earlier than most people apparently
assume.
If anyone else has any first-hand notes (i.e.from people who were there at the
time), about the relationship between all the early systems, for which the
author has given permiosssion to post it, please send it to me and I will
add it to the appropriate article on the CHWiki.
Probably the most needed is more about the roots of USG; Dale has filled in
CB/UNIX, and the roots of PWB are covered fairly well in the BSTJ article
on it:
https://archive.org/details/bstj57-6-2177
at least, for PWB1. Anything that covers the later PWBs would likewise be
gratefully receied.
I suppose I should also write up the relationships of the later UNIXen - 32V
and its descendants too - any material sent to me about them will be most
gratefully received. (If anyone want a CHWiki account, to write it up
themselves, please let me know).
Noel
