TUHS September 2021

tuhs@tuhs.org

43 participants
25 discussions

by Warner Losh

Greetings, What's the canonical source for patches to 2.9BSD and 2.11BSD? I see we have 2.11BSD patch 469 dated last month in the archive. Where does it come from? Has anybody climbed the hill to import all the patches into a git repo? I've found some mirrors, but moe.2bsd.org has been down for me for ages... How does Warren keep things up to date? I also have a (maybe faulty) memory of a similar series of patches to 2.9BSD because it was the last BSD to support non-split I&D space machines. yet a quick google search turns up nothing other than a set of patches dated August 1985 (also in our archive) and some changes for variants of hardware (pro, mscp). Is that it? Warner

1 year, 7 months

Old UNIX file system formats

by norman＠nose.cita.utoronto.ca

I've assembled some notes from old manuals and other sources on the formats used for on-disk file systems through the Seventh Edition: http://www.cita.utoronto.ca/~norman/old-unix/old-fs.html Additional notes, comments on style, and whatnot are welcome. (It may be sensible to send anything in the last two categories directly to me, rather than to the whole list.)

2 years, 6 months

ksh88 source code?

by Warren Toomey

Hi all, I received an e-mail looking for the ksh-88 source code. A quick search for it on-line doesn't reveal it. Does anybody have a copy? Cheers, Warren Original e-mail: I recently built a PiDP11 and have been enjoying going back in time to 2.11BSD.. I was at UC Davis in the the early 1980's and we had a few PDP-11/70's running 2.8/2.9 BSD. Back then we reached out to David Korn and he sent us the source for KSH -- this would have been in 1985ish if I remember, and we compiled it for 2.9 & 4.1BSD, Xenix, and some other variants that used K&R C. It may have been what was later called ksh88. I wish I still had the files from then.. I was wondering if you might know if there's an older version like this or one that's been ported for 2.11BSD? Many thanks, Joe

2 years, 10 months

mmap origin (was Systematic approach to command-line interfaces)

by Paul Ruizendaal

On Wed, Sep 29, 2021 at 09:40:23AM -0700, Greg A. Woods wrote: > I think perhaps the problem was that mmap() came too soon in a narrow > sub-set of the Unix implementations that were around at the time, when > many couldn't support it well (especially on 32-bit systems -- it really > only becomes universally useful with either segments or 64-bit and > larger address spaces). The fracturing of "unix" standards at the time > didn't help either. > > Perhaps these "add-on hack" problems are the reason so many people think > fondly of the good old Unix versions where everything was still coming > from a few good minds that could work together to build a cohesive > design. The add-ons were poorly done, not widely implemented, and > usually incompatible with each other when they were adopted by > additional implementations. mmap() did come from those days and minds. The first appearance of mmap() was in 32V R3, done by John Reiser in 1981. This is the version of 32V with full demand paging; it implemented a unified buffer cache. According to John, that version of mmap() already had the modern 6 argument API. John added mmap() because he worked with Tenex a lot during his PhD days and missed PMAP. He needed some 6 months to design, implement and debug this version of 32V as a skunkworks project. I am trying to revert early VAX SVr1/r2 code to get a better view of what 32V R3 looked like, but unfortunately I did not have much time for this effort in last couple of months. It would seem that 32V R3 assumed that disk blocks and memory pages were the same size (true on a 1980 VAX) and with that assumption a unified buffer cache comes natural in this code base. For 4.2BSD initially Joy cs. had a different approach to memory mapped files in mind (see the 1981 tech report #4 from CSRG). By the time of 4.2BSD’s release the manual defined a mmap() system call, but it was not implemented and it appears to have been largely forgotten until SunOS 4 and dynamic libraries six years later. In the SysV lineage it is less clear. For sure mmap() is not there, but the first implementation of the shmem IPC feature might derive from the 32V R3 code. On the inside, SVr2 virtual memory appears to implement the segments (now called regions) that Joy envisaged for 4.2BSD but did not implement. CB Unix had a precursor to shmem as well, where a portion of system core was reserved for shared memory purposes and could be accessed either via the /dev/mem device or could be mapped into the PDP-11 address space (using 1 of the 8 segment registers for each map). Here too the device and the map were unified. So far, I have not come across any shared library implementations or precursors in early Unix prior to SunOS 4. Paul

3 years, 1 month

Systematic approach to command-line interfaces

by Michael Siegel

Hello, I've recently started to implement a set of helper functions and procedures for parsing Unix-like command-line interfaces (i.e., POSIX + GNU-style long options, in this case) in Ada. While doing that, I learned that there is a better way to approach this problem – beyond using getopt(s) (which never really made sense to me) and having to write case statements in loops every time: Define a grammar, let a pre-built parser do the work, and have the parser provide the results to the program. Now, defining such a grammar requires a thoroughly systematic approach to the design of command-line interfaces. One problem with that is whether that grammar should allow for sub-commands. And that leads to the question of how task-specific tool sets should be designed. These seem to be a relatively new phenomenon in Unix-like systems that POSIX doesn't say anything about, as far as I can see. So, I've prepared a bit of a write-up, pondering on the pros and cons of two different ways of having task-specific tool sets (non-hierarchical command sets vs. sub-commands) that is available at https://www.msiism.org/files/doc/unix-like_command-line_interfaces.html I tend to think the sub-command approach is better. But I'm neither a UI nor a Unix expert and have no formal training in computer things. So, I thought this would be a good place to ask for comment (and get some historical perspective). This is all just my pro-hobbyist attempt to make some people's lives easier, especially mine. I mean, currently, the "Unix" command line is quite a zoo, and not in a positive sense. Also, the number of well-thought-out command-line interfaces doesn't seem to be a growing one. But I guess that could be changed by providing truly easy ways to make good interfaces. -- Michael

3 years, 1 month

Re: [TUHS] Systematic approach to command-line interfaces

by Douglas McIlroy

> one other thing that SLS breaks, for data files, is the whole Unix 'pipe' > abstraction, which is at the heart of the whole Unix tools paradigm. Multics had an IO system with an inherent notion of redirectable data streams. Pipes could have--and eventually did (circa 1987)--fit into that framework. I presume a pipe DIM (device interface manager) was not hard to build once it was proposed and accepted. Doug

3 years, 1 month

Re: [TUHS] Systematic approach to command-line interfaces

by jnc＠mercury.lcs.mit.edu

> From: Larry McVoy > If you read(2) a page and mmap()ed it and then did a write(2) to the > page, the mapped page is the same physical memory as the write()ed > page. Zero coherency issues. Now I'm confused; read() and write() semantically include a copy operation (so there are then two copies of that data chunk, and possible consistency issues between them), and the copied item is not necessarily page-sized (so you can't ensure consistency between the original+copy by mapping it in). So when one does a read(file, &buffer, 1), one gets a _copy of just that byte_ in the process' address space (and similar for write()). Yes, there's no coherency issue between the contents of an mmap()'d page, and the system's idea of what's in that page of the file, but that's a _different_ coherency issue. Or am I confused? PS: > From: "Greg A. Woods" > I now struggle with liking the the Unix concept of "everything is a > file" -- especially with respect to actual data files. Multics also got > it right to use single-level storage -- that's the right abstraction Oh, one other thing that SLS breaks, for data files, is the whole Unix 'pipe' abstraction, which is at the heart of the whole Unix tools paradigm. So no more 'cmd | wc' et al. And since SLS doesn't have the 'make a copy' semantics of pipe output, it would be hard to trivially work around it. Yes, one could build up a similar framework, but each command would have to specify an input file and an output file (no more 'standard in' and 'out'), and then the command interpreter would have to i) take command A's output file and feed it to command B, and ii) delete A's output file when the whole works was done. Yes, the user could do it manually, but compare: cmd aaa | wc and cmd aaa bbb wc bbb rm bbb If bbb is huge, one might run out of room, but with today's 'light my cigar with disk blocks' life, not a problem - but it would involve more disk traffic, as bbb would have to be written out in its entirety, not just have a mall piece kept in the disk cache as with a pipe. Noel

3 years, 1 month

Re: [TUHS] Systematic approach to command-line interfaces

by jnc＠mercury.lcs.mit.edu

> From: "Greg A. Woods" > the elegance of fork() is incredible! That's because in PDP-11 Unix, they didn't have the _room_ to create a huge mess. Try reading the exec() code in V6 or so. (I'm in a bit of a foul mood today; my laptop sorta locked up when a _single_ Edge browser window process grew to almost _2GB_ in size. Are you effing kidding me? If I had any idea what today would look like, back when I was 20 - especially the massive excrement pile that the Internet has turned into - I never would have gone into computers - cabinetwork, or something, would have been an infinitely superior career choice.) > I now struggle with liking the the Unix concept of "everything is a > file" -- especially with respect to actual data files. Multics also got > it right to use single-level storage -- that's the right abstraction Well, files a la Unix, instead of the SLS, are OK for a _lot_ of data storage - pretty much everything except less-common cases like concurrent access to a shared database, etc. Where the SLS really shines is _code_ - being able to just do a subroutine call to interact with something else has incredible bang/buck ratio - although I concede doing it all securely is hard (although they did make a lot of progress there). Noel

3 years, 1 month

Thompson trojan put into practice

by Douglas McIlroy

>> > It's part of my academic project to work on provable compiler security. >> > I tried to do it according to the "Reflections on Trusting Trust" by Ken >> > Thompson, not only to show a compiler Trojan horse but also to prove that >> > we can discover it. >> >> Of course it can be discovered if you look for it. What was impressive about >> the folks who got Thompson's compiler at PWB is that they found the horse >> even though they weren't looking for it. > I had not heard this story. Can you elaborate, please? My impression from having > read the paper (a long time ago now) is that Ken did the experiment locally only. Ken did it locally, but a vigilant person at PWB noticed there was an experimental compiler on the research machine and grabbed it. While they weren't looking for hidden stuff, they probably were trying to find what was new in the compiler. Ken may know details about what they had in the way of source and binary. Doug

3 years, 1 month

Thompson trojan put into practice

by Douglas McIlroy

> It's part of my academic project to work on provable compiler security. > I tried to do it according to the "Reflections on Trusting Trust" by Ken > Thompson, not only to show a compiler Trojan horse but also to prove that > we can discover it. Of course it can be discovered if you look for it. What was impressive about the folks who got Thompson's compiler at PWB is that they found the horse even though they weren't looking for it. Then there was the first time Jim Reeds and I turned on integrity control in IX, our multilevel-security version of Research Unix. When it reported a security violation during startup we were sure it was a bug. But no, it had snagged Tom Duff's virus in the act of replication. It surprised Tom as much as it did us, because he thought he'd eradicated it. Doug

3 years, 1 month

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

1996

1995

1994

1993

1992

1991

1990

TUHS September 2021