TUHS

tuhs@tuhs.org

6532 discussions

by A. P. Garcia

Hi, The Hacker's Dictionary says that daemons were so named in CTSS. I'm guessing then that Ken Thompson brought them into Unix? I've noticed that more recent implementations of init have shunned the traditional terminology in favor of the more prosaic word "services". For example, Solaris now has SMF, the Service Management Facility, and systemd, the linux replacement for init, has services as well. It makes me a little sad, because it feels like some of the imaginativeness, fancifulness, and playfulness that imbue the Unix spirit are being lost.

7 years, 3 months

Fwd: long lived programs (was Re: RIP John Backus

by Clem Cole

[try-II] On Fri, Mar 23, 2018 at 6:43 AM, Tim Bradshaw <tfb(a)tfeb.org> wrote: > On 22 Mar 2018, at 21:05, Bakul Shah <bakul(a)bitblocks.com> wrote: > > > I was thinking about a similar issue after reading Bradshaw's > message about FORTRAN programs being critical to his country's > security. What happens in 50-100 years when such programs have > been in use for a long time but none of the original authors > may be alive? The world may have moved on to newer languages > and there may be very few people who study "ancient" computer > languages and even they won't have in-depth experience to > understand the nuances & traps of these languages well enough. > No guarantee that FORTRAN will be in much use then! Will it be > like in science fiction where ancient spaceships continue > working but no one knows what to do when they break? > > > My experience of large systems like this is that this isn't how they work > at all. The program I deal with (which is around 5 million lines naively > (counting a lot of stuff which probably is not source but is in the source > tree)) is looked after by probably several hundred people. It's been > through several major changes in its essential guts and in the next ten > years or so it will be entirely replaced by a new version of itself to deal > with scaling problems inherent in the current implementation. We get a new > machine every few years onto which it needs to be ported, and those > machines have not all been just faster versions of the previous one, and > will probably never be so. > > What it doesn't do is to just sit there as some sacred artifact which > no-one understands, and it's unlikely ever to do so. The requirements for > it to become like that would be at least that the technology of large-scale > computers was entirely stable, compilers, libraries and operating systems > had become entirely stable and people had stopped caring about making it do > what it does better. None of those things seems very likely to me. > > (Just to be clear: this thing isn't simulating bombs: it's forecasting the > weather.) > +1 - exactly my point. We have drifted a bit from pure UNIX, but I actually do think this is relevant to UNIX history. Once UNIX started to run on systems targeting HPC loads where Fortran was the dominate programming language, UNIX quickly displaced custom OSs and became the dominant target even if at the beginning of that transition as the 'flavor' of UNIX did vary (we probably can and should discuss how that happened and why independently -- although I will point out the UNIX/Linux implementation running at say LLNL != the version running at say Nasa Moffitt). And the truth is today, for small experiments you probably run Fortran on Windows on your desktop. But for 'production' - the primary OS for Fortran is a UNIX flavor of some type and has been that way since the mid-1980s - really starting with the UNIX wars of that time. As I also have said here and elsewhere, while HPC and very much its lubricant, Fortran, are not something 'academic CS types' like to study these days - even though Fortran (HPC) pays my and many of our salar ies . Yet it runs on the system the those same academic types all prefer - *i.e.* Ken and Dennis' ideas. The primary difference is the type of program the users are running. But Ken and Dennis ideas work well for almost all users and spans specific application market s. Here is a picture I did a few years ago for a number of Intel exec's. At the time I was trying to explain to them that HPC is not a single style of application and also help them understand that there two types of value - the code itself and the data. Some markets ( *e.g.* Financial) use public data but the methods they use to crunch it ( *i.e.* the code s ) are private, while others market segments might have private data (*e.g.* oil and gas) but different customers use the same or similar codes to crunch it. F or this discussion, think about how much of the code I sho w below is complex arithmetics - while much of it is searching google style , but a lot is just plain nasty math. The 'nasty math' that has not changed over the years and thus those codes are dominated by Fortran. [Note Steve has pointed out that with AI maybe the math could change in the future - but certainly so far, history of these markets is basically differential equations solvers]. As Tim says, I really can not see that changing and the reason (I believe) is I do not see any compelling economic reason to do so. Clem ᐧ ᐧ

7 years, 3 months

1978-03-25 - 2018-03-25: 40 years BSD Mail

by Steffen Nurpmeso

On this day in 1978 Kurt Shoens placed the following comment in def.h (the fork i maintain keeps it in nail.h): /* * Mail -- a mail program * * Author: Kurt Shoens (UCB) March 25, 1978 */ --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)

7 years, 3 months

Re: [TUHS] long lived programs (was Re: RIP John Backus

by Doug McIlroy

"The only thing I can think of is to use have programs that translate programs in todays languages to a common but very simple universal language for their "long term storage". May be something like Lamport's TLA+? A very tough job. " Maybe not so hard. An existence proof is Brenda Baker's "struct", which was in v7. It converted Fortran to Ratfor (which of course turned it back to Fortran). Interestingly, authors found their completely reorganized code easier to read than what they had written in the first place. Her big discovery was a canonical form--it was not a matter of taste or choice how the code got rearranged. It would be harder to convert the code to say, Matlab, because then you'd have to unravel COMMON statements and format strings. It's easy to cook up nasty examples, like getting away with writing behyond the end of an array, but such things are rare in working code. Doug

7 years, 3 months

Re: [TUHS] long lived programs

by Ron Natalie

A core package in a lot of the geospatial applications is a old piece of mathematical code originally written in Fortran (probably in the sixties). Someone probably in the 80's recoded the thing pretty much line for line (maintaining the horrendous F66 variable names etc.) into C. It's probably ripe for a jump to something else now. We've been through four major generations of the software. The original was all VAX based with specialized hardware (don't know what it was written in). We followed that on with a portable UNIX (but mostly Suns, but ours worked on SGI, Ardent, Stellar, various IBM AIX platofrms, Apollo DN1000's, HP, DEC Alphas). This was primarily a C application. Then right about the year 2000, we jumped to C++ on Windows. Subsequently it got back ported to Linux. Yes there are some modules that have been unchanged for decades, but the system on the whole has been maintained. The bigger issue than software getting obsoleted is that the platform needed to run it goes away.

7 years, 3 months

Re: [TUHS] Comments in early Unix systems

by Doug McIlroy

Thanks for the noweb story. A reward for straying off topic! Doug

7 years, 3 months

Re: [TUHS] daemons are not to be exorcised

by jnc＠mercury.lcs.mit.edu

> From: Larry McVoy <lm(a)mcvoy.com> > Going forward, I wish that people tried to be simple as they tackle the > more complicated problems we have. I have a couple of relevant quotations on my 'Some Computer-Related Lines' page: "Deliberate complexity is the mark of an amateur. Elegant simplicity is the mark of a master." -- Unknown, quoted by Robert A. Crawford "Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it." -- Alan Perlis "The most reliable components are the ones you leave out." -- Gordon Bell (For software, the latter needs to be read as 'The most bug-free lines of codqe are the ones you leave out', of course.) I remember watching the people building the LISP machine, and thinking 'Wow, that system is complex'. I eventually decided the problem was that they were _too_ smart. They could understand, and retain in their minds, all that complexity. Noel

7 years, 3 months

Masscomp - Real Time Unix (RTU) an early UNIX based system, the first commercial multiprocessor and was realtime

by Clem Cole

On Wed, Mar 21, 2018 at 7:50 PM, Larry McVoy <lm(a)mcvoy.com> wrote: > I was sys admin for a Masscomp with a 40MB disk Right an early MC-500/DP system - although I think the minimum was a 20M [ST-506 based] disk. On Wed, Mar 21, 2018 at 8:55 PM, Mike Markowski <mike.ab3ap(a)gmail.com> wrote: > I remember Masscomp ... it allowed data acquisition to not be swapped >> out. >> > Actually, not quite right in how it worked, but in practice you got the desired behavior. More in a minute. On Wed, Mar 21, 2018 at 9:00 PM, Larry McVoy <lm(a)mcvoy.com> wrote: > I remember the Masscomps we had fondly. The ones we had were 68000 based > and they had two of those CPUs running in lock step Not lock step ... 'executor' and 'fixor' -- actually a solution Forest Basket proposed at Stanford. The two implementations that actually did this were the Masscomp MC-500 family and the original Apollo. > ... because the > 68K didn't do VM right. I can't remember the details, it was something > like they didn't handle faults right so they ran two CPUs and when the > fault happened the 2nd CPU somehow got involved. > Right, it the original 68000 did not save the faulting address properly when it took the exception, so there was not enough information to roll back the from the fault and restart. The microcode in the 68010 corrected this flaw. > > Clem, what model was that? We called the dual 68000 board the CPU board and the 68010/68000 board the MPU. The difference was which chip was in the 'executor' socket and some small changes in some PALs on the board to allow the 68010 to actually take the exception. Either way, the memory fault was processed by the 'fixor'. BTW: the original MC-500/DP was a dual processor system, so it has 4 68000 just for the 2 'cpu boards -- i.e. an executor and fixor on each board. The later MC-5000 family could handle up to 16 processor boards depending the model (MC-5000/300 was dual, MC-5000/500 up to 4 and the MC-5000/700 up to 16 processor boards). Also for the record, the MC-500/DP and MC-5000/700 predate the multiprocessor 'Symmetry System' that Sequent would produce by a number of years). The original RTU ran master/slave (Purdue VAX style) for the first generation (certainly through RT 2.x and maybe 3.x). That restriction was removed and a fully symmetric OS was created, as we did the 700 development. I've forgotten which OS release brought that out. A few years later Stellix, while not in direct source development a child, had the same Texieria/Cole team as RTU - was always fully symmetric - i.e. lesson learned. > And can you provide the real version of what I was trying say? > Sure ... soon after Motorola released the 68000, Stanford's Forest Baskett in a paper I have sadly lost, proposed that the solution the 68000's issue of not saving enough information when an illegal memory exception occured, was to instead of allowing the memory exception, return 'wait states' to the processor - thus never letting it fault. More specifically, the original microprocessor designs had a fixed time that the memory system needed to respond on a read or write cycle that was defined by the system clock. I don't have either Motorola 68000 or MOS 6502 processor books handy, but as a for instance IIRC on a 1 Mhz MOS 6502 you had 100 nS for this operation. Because EPROMs of the day could not respond that fast (IIRC 400ns for a read), the CPU chip designers created a special 'WAIT' signal that would tell the processor, to look for the data in on the next clock tick (and the wait signal could be repeated for each tick for an indefinite wait if need be). *i.e. *in effect, when running from ROM a 6502 would run at .25MHz if it was doing direct fetches from something like an Intel 1702 ROM on every cycle. Similarly, early dynamic RAM, while faster that ROM, had access issues with needing to ensure the refresh cycles and the access cycles aligned. Static RAMs which were fast, did not have these problems and could directly interface to processor, but static RAMs of the day were the quite expensive chips (5-10 times) in comparison to DRAM cost, so small caches were built to front end the memory system. Hence, the HW 'wait state' feature became standard (and I think is still supported in todays processors), since it allows the memory system speed and the processor to work at differing rates. i*.e.* the difference in speed could be 'biased' to each side -> processor vs memory. In his paper, Forest Basket observed that if a HW designer used the wait state feature, a memory system could be built that refreshed the cache using another processor, as long as the second processor that was 'fixing up' the exception ( i.e. refilled the cache with proper data) could be ensured it never had a memory fault. Basket's idea was exactly how both the original Apollo and Masscomp CPU boards were designed. On the Masscomp CPU board, there was a 68000 that always ran from a static ram based cache (which we called the 'executor.' If it tried to access a memory location that was not yet in cache, or if it was a legal memory access, the memory system sent wait states until the cache was refilled as you might expect. If the location was illegal, the memory system also return a error exception as expected. However, it was legal address but not yet in memory, the second 6800 (the 'fixor') was notified of desire to access that specific memory location and the fixor ran the paging code to put the page into the live memory. Once the cache was properly set up, then the executor could be released from wait state and instruction allowed to complete. When Motorola released the 68010 with the internal fixes that would allow an faulting instruction to be restarted, Masscomp revised the CPU board (creating the MPU board) to install a 68010 in the executor socket and changed a few PALs in the memory system on the board. If a memory address was discovered to be legal, but not in memory, the executor (now a 68010) was allowed to returned a paging exception and stack was saved appropriately, and the executor did a context switch to different process - allow the executor to do something useful while the fixor processed the fault. Although we now had to add kernel code for the executor processor to return from restart at the fault instruction on a context switch back to the original process. The original fixor code did not need to be changed, other than to remove need to clearing of the 'waiting flop' for that restarted the executor. [RTU would detect which board was plugged into a specific system automatically so it was an invisible change to the end user - other than you got a few percent performance back if there was a lot of page faults in your application since the executor never wait stated]. As for the way real time and analog I/O worked that Mike commented upon. Yes, RTU - Real Time Unix supported features that could guarantee that I/O would not be lost from the DMA front end. For those not familiar with the system, it was a 'federated system' with a number of subsystems: the primary UNIX portion that I just described and then a number of other specialized processors that surrounded it for specific tasks. This architecture was chosen because the market we were attacking was a scientific laboratory and in particular real-time oriented - for instance, some uses were Mass General in the Cardiac ward, on board ever EWACS plane to collect all and retain all signals during any sortie [very interesting application], NASA used 75 of them in Mission Control to be the 'consoles' you see on TV, *etc *[ which have only recently be replaced by PCs, as some were still in use at least 2 years ago when I was last in Houston]. So besides the 2/4 68000s in the main UNIX system, their was another dedicated 68000 running a graphics system on the graphics board, another 80186 running TCP/IP, and an AM2900 bit slices processor called Data Acquisition and Control Processor (DA/CP) - which Mike hinted, as well as 29000's for the FP unit and Array processor functionality. Using ther DA/CP, in 1984, and MC-500 system could handle multiple 1MHz analog 16 bit signals - just by sampling them -> in fact, a cute demo at the Computer Museum in Boston just connected a TV camera to the DA/CP and without any special HW, it could 'read' the video signal (if you normally have a connection a TV camera, it usually takes a bunch of special logic to interface to the TV signals -- in this case it was just SW in the DA/CP). The resultant OS was RTU, were we had modified a combined 4.1BSD and System III 'unices' (later up dated to 4.2 and V.2] to support RT behaviors, with pre-emption, ASTs [which I still miss - a great idea but nicer than traditional UNIX signals], async I/O, *etc*.. Another interesting idea was the concept of 'universes' that allowed on a per user basis, to see a BSD view of the system or an System V view]. One of the important things we did in RTU was rethink the file system (although only a little then). Besides all the file types and storage schemes you have with UFS, Masscomp added support for an pre-allocated extent based files (like VMS and RSX) and then created a new file type called contiguous file that used it [by the time of the Stellar and Stellix, we just made all files extent based and got rid of the special file type - *i.e.* looked like UFS to the user, but under the covers was fully extend based]. So under RTU, once we had preallocated files that we knew were on contiguous blocks, we could make all sorts of speed ups and guarantees that traditional UNIX can not because of the randomized location of the disk blocks. So the feature that Mike was using, was actually less a UNIX change and did not actually use the preemption and other types of RT features. We had added the ability for the DA/CP to write/read information directly from the disk without OS getting in the way - (we called it Direct to Disk IO). An RTU application created a contiguous file large enough on disk to store the data the DA/CP might receive - many megabytes typically - but the data itself was never passed thru or stored in the UNIX buffer cache *etc*. The microcode of the DA/CP and the RTU disk driver could/would cooperate and all kernel or user processing on the UNIX side was by-passed. Thus, when the real time data started to be produced by the DA/CP (say the EWACS started getting radio signals on one of those 16 bit analog converters) the digital representation of those analog signals were stored in the users file system independent of what RTU was doing. The same idea, by the way was why we chose to run IP/TCP on a co-processor. So much of network I/O is 'unexpected,' we could potentially lose the ability to make real-time guarantees. But keeping all the protocol work outside of UNIX, the network I/O just operated on 'final bits' at the rate that was right for it. As an interesting aside, this worked until we switched to a Motorola 68020 at 16 Mhz or 20Mhz (whatever it was I've forgotten now) in the MC-5000 system. The host ended up be so much faster than the 80186 in the ethernet coprocessor board. We ended up offering a mode were they swapping roles and just used the '186 board as a very heavily buffered ethernet controller, so we could keep protocol processing very low priority compared to other tasks. However if we did that, it did mean some of the system guarantees had to be removed. My recollection is that only a couple of die hard RT style customers, ended up continuing to use the original networking configuration. Clem

7 years, 3 months

daemons are not to be exorcised/Comments in code

by David

> From: Dave Horsfall <dave(a)horsfall.org> > Yep, and I'm glad that I had bosses to whom I didn't have to explain why > my comments were so voluminous. > > And who first said "Write your code as though the next person to maintain > it is a psychotic axe-wielding murderer who knows where you live"? I've > often thought that way (as the murderer I mean, not the murderee). > > I'd name names, but he might be on this list... > I’ve always said that the person was a ‘reformed’ axe-wielding murderer who knows where you live. Please, a little decorum. W.R.T. comments in code, and a bit of Unix, when I taught Unix Systems programming at UCSD one of my students wanted to use his personal computer for the programming. I didn’t care as long as it would pass the tests I ran after the fact. After his second homework was turned in, I stopped looking at his code or even running it, as the comment blocks were just so well done. Nary a comment in the function itself, just a block before each one very clearly explaining what was happening. David

7 years, 3 months

Re: [TUHS] Comments in early Unix systems

by Doug McIlroy

"I was told in school (in 1985) that if I was ever lucky enough to have access to the unix source, I'd find there were no comments. The reason, I was told at the time, was that comments would make the source code useful, and selling software was something AT&T couldn't do due to some consent decree." I can't speak for SYS V, but no such idea was ever mentioned in Research circles. Aside from copyright notices, the licensing folks took no interest in comments. Within rsearch there was tacit recognition of good coding style--Ken's cut-to-the-bone code was universally admired. This cultural understanding did not extend to comments. There was disparagement for the bad, but not honor for the good. Whatever comments you find in the code were put there at the author's whim. My own commenting style is consistent within a project, but wildly inconsistent across projects, and not well correlated with my perception of the audience I might be writing for. Why? I'm still groping for the answer. For imoortant code, custom is to describe it in a separate paper, which is of course not maintained in parallel with the code. In fact, comments are often out of date, too. Knuth offered the remedy of "literate programming", which might help in academic circles. In business, probably not. Yet think of the possibility of a "literate spec", where the code grows organically with the understanding of what has to be done. Doug

7 years, 3 months

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

1996

1995

1994

1993

1992

1991

1990

TUHS