The /bin/sh stuff made me think of an interview question I had for engineers,
that a surprisingly few could pass:
"Tell me about something you wrote that was entirely you, the docs, the
tests, the source, the installer, everything. It doesn't have to be a
big thing, but it has to have been successfully used by at least 10
people who had no contact with you (other than to say thanks)."
Most people fail this. I think the people who pass might look
positively on the v7 sh stuff. But who knows?
As mentioned in the first post on SysIII porting, I was surprised to see how much code was needed to initialise modern hardware and to load an OS. Of course, modern devices are much more capable than the ones of 40 years ago, so maybe my surprise is misplaced. It did raise an interest in the history of Unix system configuration though.
It would seem that 5th Edition already contained a configuration program that generated a few system tables and the ‘low.s’ file with interrupt vectors and alike. Although it steadily grew in sophistication, the approach appears still the same in SysIII. I suppose this is all in line with common practice of the era, with OS’s typically having a ’system generation kit' to combine the pre-linked OS kernel with device drivers and system tables.
SysIII also introduces the "var struct" and the “v” kernel variable that summarises some of the system configuration. I’m not sure whether it has roots in earlier Unix systems, it does not seem to originate from Research. I’m not sure what the point of this ‘v’ kernel variable was. Does anybody remember?
One could argue that one of the drivers of the success of CP/M in the 1970’s was due to its clear separation between the boot rom, BIOS and BDOS components. As far as I am aware, Unix prior to 1985 did never attempt to separate the device drivers from the other kernel code. I am not very familiar with early Xenix, it could be that Microsoft had both the skill and the interest to separate Xenix in a standard binary (i.e. BDOS part) and a device driver binary (i.e. BIOS part). Maybe the differences in MMU for the machines of the early 80’s were such that a standard binary could not be done anyway and separating out the device drivers would serve no purpose. Once the PC became dominant, maybe the point became moot for MS.
It would seem that the next step for Unix in the area of boot, config and device drivers came with Sun’s OpenBoot in 1988 or so. This also appears to be the first appearance of device trees to describe the hardware to the bios and the kernel. Moreover, it would seem to me that OpenBoot is a spiritual ancestor of the modern Risc-V SBI specification. Maybe by 1988 the IO hardware had become sufficiently complex and/or diverse to warrant a break from tradition?
Was there any other notable Unix work on better organising the boot process and the device drivers prior to OpenBoot?
> "Originally the idea of adding command line editing to ksh was
> rejected in the hope that line editing would move into the terminal
> driver." [2]
> I have always wondered, what such a central terminal driver driven
> history/line-editing would have felt like.
You can get a feel for it in Rob's "sam" editor, which works that way.
Doug
at the risk of making a fool of myself - there are several people far better qualified here, however…
my memory is that the plan9 linker could be easily rebuilt to use malloc and free in the traditional style, reducing its memory footprint - though making it much slower.
-Steve
Adam Thorton wrote:
> I mean all I really want for Christmas is a 64-bit v7 with TCP/IP support, a screen editor, and SMP support.
>
> The third one is a solved problem. The second one would not be that hard to adapt, say, uIP 0.9, to v7. That first one would require some work with C type sizes, but getting larger is easier than the reverse. It's that last one.
>
> Having said that...maybe what I really want is 64-bit 4.3 BSD?
>
> I mean, just a Unix, without all the cruft of a modern Linux, but which can actually take advantage of the resources of a modern machine. I don't care about a desktop, or even a graphical environment, I don't care about all the strange syscalls that are there to support particular databases, I don't care much about being a virtualization host.
Luther Johnson wrote:
> I'm in the process of building a system like that for myself, but
> perhaps a little smaller - mine will be based on an embedded
> microprocessor I've developed (so much work still yet to do ! at least a
> year out).
Earlier this year I ported VAX System III to Risc-V, to a simple Allwinner D1 based SBC. This is RV64GC. Just ported to the console terminal.
It turned out that porting Sys III to 64 bit was surprisingly easy, most of the kernel and user land appears to be 64 bit clean. It helps that I am using a LLP64 compiler, though. Apart from networking Sys III also feels surprisingly modern (for an ancient Unix) - its should get more attention than it does. The hardest work was in porting the VAX memory code to Risc-V page tables (and to a lesser extent, updating libc for the different FP formats).
The code is currently in an ugly state (with debug stuff in commented-out blocks, a mix of ansi and K&R styles, an incoherent kludgy build system, etc.) and the shame stopped me from putting it out on gitlab until I found enough time to clean this up. As there seems to be some interest now, I’ll put it up anyway in the next week or so. There you go Adam, half your wish comes true.
The kernel is about 60KB and most binaries are quite close in size to the VAX equivalents.
My next goals for it are to re-implement the Reiser demand paging (I think I have a good enough view of how that worked, but the proof of the pudding will be in the eating), and to add TCP/IP networking, probably the BBN stack. Making it work on RV32 and exploring early SMP work is also on my interest list.
===
David Arnold wrote:
> I think xv6 does SMP? (param.h says NCPU = 8, at least).
>
> You’d need to add a network stack and a userland, but there are options for those …
For the above, making xv6 work on the D1 board was my first stepping stone, to proof the tool chain and to get the basics right (hardware init, low-level I/O, etc.).
As an educational tool, I am sure that xv6 hits all the right spots, and it certainly does SMP (the D1 is single hart, so I have not tried that myself). I like it a lot in that context. However, as a simple Unix it is useless: from a user-land view it is less capable than LSX. At minimum it needs fixes to make the file system less constrained.
In my view, for SMP Keith Kelleman’s work for Sys-V is probably a better place to start.
Having done the SysIII 64-bit port to a recent Risc-V chip, I realised that whilst it is an interesting exercise bij itself -- and maybe even useful to students and educators around the world -- it is not ideal as a research tool for analysing Unix from the early 80’s. The address size difference adds some superfluous porting, and the 100x speed difference can hide critical algorithm constraints. Also the complex IO devices are out of character.
For a Risc-V 32 bit target I’ve settled on an FPGA implementation from the University of Tokyo. I’ve somewhat modified the system to work with the open source Yosys/NextPNR tool chain. It now implements a Linux-capable SoC with a full MMU, a 4-way cache and SD card driver in less than 4,000 lines of plain Verilog (compiling to about 14K LUTs). In a way, the code has a 6th edition feel to it: it covers a real and usable system and the code can be understood in a couple of days -- or a semester for a student who is new to the concepts involved.
https://gitlab.com/r1809/rvsoc/-/tree/main/doc
So far I have Linux and XV6 (https://gitlab.com/r1809/xv6-rv32) running, but have not started on SysIII yet.
Usefully for my use case this system is not very fast, completing an instruction in on average 10 clocks. Still, when running at 40MHz it is about 2 or 3 times as fast as a VAX11/780, which is similar to the systems of the mid-80’s. Even at this speed, a single user console Linux is surprisingly usable. By the way, funny to realise that ‘Unix/Linux capable’ has been a marketing slogan for system capability for 40 years now.
There is a short video clip with a demonstration (but running at 100MHz) here: https://youtu.be/Kt_iXVAjXcQ
Due to its simple design, the main CPU only uses some 30% of the cache memory bandwidth and it should not be all that hard to add a second CPU to the system (the CPU already supports the Risc-V atomic operations), and this could be a nice target for studying the early Unix multi-processor designs (e.g. VAX/BSD & 3B2/SVR3).
I find it an intriguing thought that the chip technology of the early 80’s (let’s say the technology used for the Bellmac-32 or M68K) would probably have sufficed to build a CPU similar to the one used in this FPGA design.
As the topic of this post is on a tangent from the focus of this list, I would recommend that any follow-ups not related to the history of Unix are sent off list.
Porting the SysIII kernel to a D1 board (https://www.aliexpress.us/item/3256803408560538.html) began with a port of XV6, in order to test the tool chain and to get comfortable with this target. Michael Engel had already ported XV6 to the D1 chip a few months before (https://www.uni-bamberg.de/fileadmin/sysnap/slides/xv6-riscv.pdf) giving a ready base to work with.
The main new effort was to add code to initialise the DRAM controller and the SD Card interface, and to have a simple boot loader. Such code is available from the manufacturer board support package (BSP), although in this case the DRAM controller code was only available as assembler compiler output and had to be reverse engineered back into C. In general I was surprised to see how big and unwieldy the BSP code is; maybe the code just looks sloppy because it has to deal with all kinds of edge cases - but I can also imagine that it accumulates cruft as it is ported from SoC to SoC by the manufacturer.
The resulting XV6 source tree is here: https://gitlab.com/pnru/xv6-d1
This version automatically boots from the SD Card on the board.
With that out of the way, the ancient Unix port was relatively easy. It would seem to me that the SysIII code base has a lot of clean-up work in it that still pays off today. The code compiles to a 64-bit target with minimal updates, which I think is a compliment to the engineers that worked on it. Probably using a LLP64 compiler also helped. In order to bring something up quickly, I modified the kernel to load ELF binaries, so that I could use proven material from the XV6 port (such as a minimalistic init and shell).
Initially, I just replaced VAX memory management with page table code taken from XV6 (i.e. no VM or swapping). Working with Risc-V page tables gives much simpler code, but I have a deeper appreciation of the VAX paging design now: for the type of code that was run in 1980, the VAX design enables very small page tables with just a few dozen entries. In contrast, for the 3-level page tables of 64-bit Risc-V I end up with 7 pages of page table of 4KB each, or 28KB -- that is larger than the memory image of many SysIII programs. If I move the ‘trampoline' to just above the stack in virtual memory it could be 5 pages instead of 7, but the overall picture remains the same. The 68020 or ‘030 MMU could be configured to have various page sizes -- this looked byzantine to me when I first saw it, but it makes more sense now.
Next I replaced the VAX scatter paging / partial swapping code, keeping the same methodology. I noticed that there is still confusion over memory management in 32V and SysIII (and implicitly SVR1,R2). The original 32V as described in the London/Reiser paper used V7 style swapping. This code can be found as ‘slowsys’ in the surviving source (https://www.tuhs.org/cgi-bin/utree.pl?file=32V/usr/src/slowsys) It was quickly (Mar vs. Jan 1979) replaced by the scatter loading / partial swapping design already hinted at in the paper (source is in 32V/usr/src/sys). Unfortunately, the “32V uses V7 swapping” meme lives on.
In scatter paging, the pages are no longer allocated continuous in physical memory but new pages are taken from a free list and expansion swaps are not usually needed. Also, when a process is swapped out, it is not fully swapped out, but just enough pages to make room for the new process. When it is readied to run again, only the partial set needs to be reloaded. In the VAX context, scatter paging and partial swapping are quite effective and I think competitive with demand paging for the 25-100KB processes that were in use at the time. As I mentioned in the post on the toolchain, the Plan 9 C compiler can easily use 1MB of memory and in a 4MB of core context, this trashes the algorithm; it starts to behave much like traditional swapping. The reason for this is that the entire process must be in memory in order to be able to run and the algorithm cannot recognise that a much smaller working set is needed. The implicit assumption of small processes can also be seen in the choice to limit partial swaps to 4KB per iteration (8 VAX pages).
For handling processes with a large memory footprint but a small working set a true demand paged VM approach is needed. The simplest such approach appears to be Richard Miller’s work for SVR1 (see June 1984 Usenix conference proceedings, "A Demand Paging Virtual Memory Manager for System V"). This is a very light touch implementation of demand paging and it seems that enough bits and pieces survive to recreate it.
The journey through the memory code made it clear again that in SysIII and before, the memory code is scattered over several locations and not so easy to fathom at first glance. It would seem that in SysV/68 an attempt was made to organise the code into separate files and with a more defined API. It does not seem to have carried through. Maybe this was because the MMU’s of the 1980-1985 era were all too different to be efficiently abstracted into a single framework.
Beyond SysV/68, were there any other attempts in the early 80’s to organise and abstract the kernel memory management code (outside BSD)?
After initially gearing up to use the Motorola 68020 or 68030 as a porting target for a study of Unix in the 1980-1985 era, I reconsidered and used Risc-V as a target instead. As the original RISC and MIPS projects were contemporaneous with early 32-bit Unix (32V, BSD, SysIII and SVr1,r2) it seems appropriate and there is currently considerable interest (hype?) around Risc-V.
From a programming perspective, the Risk-V ISA does not feel (at least to me) all that different from what was current in the early 80’s — the number of WTFs/min is low. The modularity is a pleasant surprise, as is the observation that the 32-bit and 64-bit instruction sets are almost identical and that compressed instructions mingle nicely with full size ones. The MMU design appears straightforward. Maybe this is because the ISA is still relatively new and has not acquired much historical baggage at this point in its lifespan, but it also seems to be a good synthesis of insights gained over the last 4 decades and applied with a sense of minimalism.
At first I was thinking to create a toolchain based on pcc or pcc2 for the SysIII porting effort, based on some preparation I had done when I was still thinking about 68030 as a target (the surviving Blit code includes a pcc-based 68000 compiler and the SysV/68 source archive contains a pcc2-based compiler). Before I got underway with that, I came across a presentation Richard Miller had done about his Risc-V compiler:
https://riscv.org/news/2020/10/a-plan-9-c-compiler-for-rv32gc-and-rv64gc/
Richard was kind enough to share the source code for his Risc-V back-end. The first complication was that the source code assumes that it will be running inside a Plan-9 environment, whereas I was looking for a Unix/Posix environment. Luckily somebody already had assembled the libraries needed for this:
https://github.com/aryx/fork-kencc
I’m not sure where it came from, but I would assume it has some roots in the "Plan-9 from user space" effort. From this work I extracted the minimum needed to get the C compiler working and to build from scratch. The libraries mostly just worked. The compiler was a bit harder: the source code assumes a LLP64 model in a few places and compiling this with clang (which uses a LP64 model) introduces issues in a handful of places. Other than this initial hurdle, the compiler and tools have worked flawlessly, both for 64-bit code and for 32-bit code, and have been a joy to use. One particular nicety is that Plan 9 style "abstract assembler" source for 64-bit code is even more identical to its 32-bit variant than with the mainstream Risc-V assembler syntax. My repo for the tool chain is here:
https://gitlab.com/pnru/riscv-kencc
Initially, my expectation was that I could only use these compilers as cross-compilers and that I would need to do a pcc2 version for native compilation at some point. However, when I saw how small and fast the tools were, I experimented with using them on SysIII. Much to my surprise the effort required was absolutely minimal, all that was needed was adding a dozen simple standard functions in libc (see here: https://gitlab.com/pnru/SysIII_rv64/-/tree/master/libc/compat) and adding the ‘dup2' sys call. I had not expected that SysIII was so close to the Unix systems of the 1990’s in this regard. This result inspires ideas for future projects: as I plan to add an 8th edition style file system switch anyway, maybe it will not be all that hard to make the Plan-9 debugger work on this “SysIII+” as well.
Another observation has been that the code size of binaries compiled for Risc-V by this tool chain is almost the same as those compiled for the VAX by pcc (the Risc-V ones are maybe 10-20% larger). This is using the compressed instructions where possible. This is I think another indication that both the Risc-V ISA and the tool chain are quite well done.
The one less positive surprise has been the memory use of the compiler. Even on a relatively simple program file it will quickly use 1 megabyte or more of ram. I understood from Richard that this is because the compiler only malloc()’s and never free()’s by design. This has been a mixed blessing. Such large memory images don’t work all that well with the "scatter paging + partial swapping" memory management of SysIII when memory is constrained to say 4MB of core to mimic the systems of the era. On the other hand, parallel compiling the kernel on SysIII itself heavily exercises the partial swapping code and has been a great test case for debugging.
Many thanks to Ken, Rob, Richard and all the others who created this fine tool chain!
> James Johnston:
>> Yeah, but Rob, where was Fred? I was there in Acoustics Research (not
>> 127!) then, using R70 for UCDS.
> Rob Pike:
> Not in (1)127 yet. He was transferred in some time after I arrived. Not
> sure quite when. Mid-80s maybe.
> =====
> Early-to-mid 1980s. ftg was already there when I interviewed in early 1984.
> Norman Wilson
In 1980 Fred was a stalwart of the computer center. There he exhibited
great creativity, including the invention of "quest" for sniffing out
security lapses throughout the BTL computer network. His findings
underpinned the headline claim of the Labs' first computer-security
task force (1982), "It is easy and not very risky to pilfer data from
Bell Labs computers".
Doug
