TUHS June 2020

tuhs@tuhs.org

68 participants
61 discussions

non-blocking IO (morphing to "The serial I/O loop")

by Paul Ruizendaal

> On 5/31/2020 9:46 AM, Warner Losh wrote: > > Sorry to top post, but LSX or Miniunix had non blocking I/O as well. > > It was in one of the documents that Clem scanned in the last year. It > > specifically was an experiment into how to do it. > > > > Warner > I did add a few new features to LSX to deal with contiguous files > and to handle asynchronous read/write's for real time applications. > They are described in the LSX paper in the 1978 BSTJ on the > UNIX Time-Sharing System. > > Heinz Thanks for highlighting this! The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic. For others, the paper that you refer to can also be found here: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNI… If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost. From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-… - - - When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped. I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-…) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive? Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-…) This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network. Can you tell me more about the serial I/O loop and its use cases?

4 years, 1 month

Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)

by Bakul Shah

> Would you be so kind to explain a bit about the hm version of MH. Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my Fortune box until I retired it. It provided a 2D interface. Later I tried xmh but didn't like it. The early history of MH is covered in some detail in Willis Ware's "RAND and the Information Evolution" book from page 128 onward. hm & Guyton get a paragraph on page 136. https://www.rand.org/content/dam/rand/pubs/corporate_pubs/2008/RAND_CP537.p…

4 years, 1 month

DEC RL01/RL02 RX01/RX02 Disk Image Creator

by Paul Riley

Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley* Mo: +86 186 8227 8332 Email: paul(a)rileyriot.com

4 years, 1 month

non-blocking IO

by Paul Ruizendaal

> > when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous > Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it. I did. There is capac() support also for the IMP interface: https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-V6/dmr/imp11a.c (see bottom two functions) BBN took the same approach as Research: with capac() or select() one can prevent blocking on read() and write().

4 years, 1 month

Re: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator

by jnc＠mercury.lcs.mit.edu

> From: Ronald Natalie > The V6 file system was limited to 2^24 blocks No, 2^16; from filsys.h: int s_fsize; /* size in blocks of entire volume */ and of course on an -11 an int is 16 bits. Maybe you're thinking of the file size, which was 2^24 bytes (max). Noel

4 years, 1 month

Re: [TUHS] non-blocking IO

by jnc＠mercury.lcs.mit.edu

> when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it. (With the NCP as well, that did some amount of demultiplexing in the kernel, and probably had buffering there, so, if so, in theory capac() could have been done there. Of course, with the ARPANET link being only 100Kbit/sec maximum - although only to a host on the same IMP - the overhead of copying buffered data made kernel buffering more 'affordable'.) Noel

4 years, 1 month

Re: [TUHS] non-blocking IO

by jnc＠mercury.lcs.mit.edu

> From: Paul Ruizendaal > This time looking into non-blocking file access. (... right now my > scope is 'communication files' (tty's, pipes, network connections). > ... > First appearance of non-blocking behaviour seems to have been with > Chesson's multiplexed files ... in 1979. At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well. I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too). There were two main generations of code; I don't recall the second one well, and I'm too lazy to go look, but I can tell you off the top of my head a bit about how the first one worked. Open/read/write all looked standard to the user in the process (the latter two were oriented to packets, a bit like raw disks being blocks); multiple operations could be queued in each direction. (There was only one user allowed at a time for the network device; no input demultiplexing.) Whenever an I/O operation completed, the process was sent a signal. Since the read/write call had long since returned, it had to do a getty() to get info about that operation - the size of the packet, error indications, etc. One complication was that for a variety of reasons (we wanted to avoid having to copy data, and the interface did not have packet buffers) we did DMA directly to/from the user's memory; this meant the process has to be locked in place while I/O was pending. (I didn't realize it at the time, but we dodged a bullet there; a comment in xalloc(), which I only absorbed recently, explains the problem. More here: https://gunkies.org/wiki/UNIX_V6_internals#exec()_and_pure-text_images if anyone wants the gory details.) That all (the queing, signals for I/O completion, locking the process to a fixed location in memory while it continued to run) etc all worked well, as I recall (although I guess it couldn't do an sbrk() while locked), but one complication was the UNIBUS map on the -11/70. The DSSR/RTS group at LCS wanted to have a ring interface, but their machine was a /70 (ours, the one the driver was initially done on/for, was a /40), so with DMA we had to use the UNIBUS map. The stock V6 code had mapalloc(), (and mapfree(), both called on all DMA operations), but... it allocated the whole map to whatever I/O operation asked for the map. Clearly, if you're about to start a network input operation, and wait a packet to show up, you don't want the disk controller to have to sit and wait for for a packet to show up so _it_ can have the map. Luckily, mapalloc() was called with a pointer to the buffer header (which had all the info about the xfer), so I added a 'ubmap' array, and called the existing malloc() on it, to allocate only a big enough chunk of the UNIBUS map for the I/O operation defined by the header. Since there was 248KB of map space, and the largest single DMA transfer possible in V6 was about 64KB (maybe a little more, for a max-sized process with its 'user' block), there was never a problem with contention for the map, and we didn't have to touch any of the other drivers at all. That was dandy, and only a couple of lines of extra code, but I somehow made a math error in my changes, and as I recall I had to debug it with a printf() in mapalloc(). I was not popular that day! Luckily, the error was quickly obvious, a fix was applied, and we were on our way. Noel

4 years, 1 month

DEC RL01/RL02 RX01/RX02 Disk Image Creator

by Paul Ruizendaal

> Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley* It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and dskutil https://github.com/sergev/bkunix/tree/master/fsutil https://github.com/sergev/bkunix/tree/master/diskutil The first generates a Unix 6th edition file system from a local directory tree. The result is a binary file with the 512 byte disk blocks stored in sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH. The second appears to be a tool to take the a file generated by the fsutil tool and split that into sectors and tracks. I’m not familiar with this tool, but it looks like you might need something similar (I assume that you have some way to hook up a 8” drive to your PC?). Sector interleaving may be an issue to look out for. When preparing a LSX system disk, you will need to think carefully about the layout: Presumably the disk works with 128 byte sectors and 4 sectors are grouped together to create a 512 byte unix block. Check out the disk driver code for details: https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c Block 0 contains bootstrap code which is loaded/called from the monitor rom (or from a short code sequence keyed in on a “blinkenpanel” console). The filesystem itself starts at block 1 and runs up to a block N (you need to tell fsutil how big the filesystem needs to be). After block N up to the end of the disk is space to contain 2 swapped out programs plus 1 block for the return code of the third (default LSX has a maximum of 3 processes). You have to figure out how many 512 blocks are on your floppy and subtract out the swap space to arrive at a figure for N. In the LSX source code ’N’ is known as the define SWPLO, see param.h for details: https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h Be careful with size units in the source code. Often sizes are expressed in words (2 bytes). Where memory is concerned it is often expressed in ‘clicks’, 64 bytes.

4 years, 1 month

LSX on the PDP-11/03 (LSI-11)

by Paul Riley

I've managed to acquire a PDP-11/03 with twin floppy drives (Sykes Datatronics RX01 or RX02 equivalents, not sure yet which). I've stumbled across LSX, and I have it running on SimH. I'm quite inexperienced with Unix, but it's something I want to learn well, having brushed against it at university in the '80s, and having played with Linux somewhat. I have some interest in Forth, but I don't like the block system of early forths such as FigForth, and I plan to create a new Forth based on FigForth, but supporting external source files. Anyway, I've tried compiling Hello World on LSX, and I get "1: External definition syntax" error. Some help would be nice, but more generally, is anyone on this list more than vaguely familiar with LSX, or 6th Edition itself? void main () { printf("Hello World!"); } It seems that the 7th Edition was the beginning of the standard library in C, and that this is missing in LSX. I'm not sure if printf is an intrinsic function in (6th Edition) C, or if it's from a library. My questions are a bit random, but looking to converse with others with LSX experience. Paul *Paul Riley* Email: paul(a)rileyriot.com

4 years, 1 month

DEC RL01/RL02 RX01/RX02 Disk Image Creator

by Paul Ruizendaal

> > In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h Oops: 77 tracks by 26 sectors of 128 bytes, makes 250.25KB, so it should work as configured.

4 years, 1 month

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

1996

1995

1994

1993

1992

1991

1990

TUHS June 2020