On Thu, Sep 2, 2021 at 5:01 AM Tony Finch <dot(a)dotat.at> wrote: As the saying has it: "What Andy giveth, Bill taketh away."

On Thu, Sep 02, 2021 at 11:24:37PM -0400, Douglas McIlroy wrote: I'm really not convinced trying to build distributed computing into the OS ala Plan 9 is viable. The moment the OS has to span multiple TCB's (Trusted Computing Bases), you have to make some very opinionated decisions on a number of issues for which we do not have consensus after decades of trial and error: * What kind of directory service do you use? x.500/LDAP? Yellow Pages? Project Athena's Hesiod? * What kind of distributed authentication do you use? Kerboers? Trust on first use authentication ala ssh? .rhosts style "trust the network" style authentication? * What kind of distributed authorization service do you use? Unix-style numeric user-id/group-id's? X.500 Distinguished Names in ACL's? Windows-style Security ID's? * Do you assume that all of the machines in your distributed computation system belong to the same administrative domain? What if individuals owning their own workstations want to have system administrator privs on their system? Or is your distributed OS a niche system which only works when you have clusters of machines that are all centrally and administratively owned? * What scale should the distributed system work at? 10's of machines in a cluster? 100's of machines? 1000's of machines? Tens of thousands of machines? Distributed systems that work well on football-sized data centers may not work that well when you only have a few racks in colo facility. The "I forgot how to count that low" challenge is a real one.... There have been many, many proposals in the distributed computing arena which all try to answer these questions differently. Solaris had an answer with Yellow Pages, NFS, etc. OSF/DCE had an answer involving Kerberos, DCE/RPC, DCE/DFS, etc. More recently we have Docker's Swarm and Kubernetes, etc. None have achieved dominance, and that should tell us something. The advantage of trying push all of these questions into the OS is that you can try to provide the illusion that there is no difference between local and remote resources. But that either means that you have a toy (sorry, "research") system which ignores all of the ways in which remote computation which extends to a different node that may or may not be up, which may or may not have belong to a different administration domain, which may or may not have an adversary on the network between you and the remote node, etc. OR, you have to make access to local resources just as painful as access to remote resources. Furthermore, since supporting access remote resources is going to have more overhead, the illusion that access to local and remote resources can be the same can't be comfortably sustained in any case. When you add to that the complexities of building an OS that tries to do a really good job supporting local resources --- see all of the observations in Rob Pike's Systems Software Research is Dead slides about why this is hard --- it seems to me the solution of trying to build a hard dividing line between the Local OS and Distributed Computation infrastructure is the right one. There is a huge difference between creating a local OS that can live on a single developer's machine in their house --- and a distributed OS which requires setting up a directory server, and an authentication server, and a secure distributed time server, etc., before you set up the first useful node that can actually run user workloads. You can try to do both under a single source tree, but it's going to result in a huge amount of bloat, and a huge amount of maintenance burden to keep it all working. By keeping the local node OS and the distributed computation system separate, it can help control complexity, and that's a big part of computer science, isn't it? - Ted

On Fri, Sep 3, 2021 at 9:23 AM Theodore Ts'o &lt;tytso(a)mit.edu&gt; wrote: It seems like plan9 itself is an existence proof that this is possible. What it did not present was an existence proof of its scalability and it wasn't successful commercially. It probably bears mentioning that that wasn't really the point of plan9, though; it was a research system. I'll try to address the plan9 specific bits below. The moment the OS has to span multiple Interestingly, plan9 did make opinionated decisions about all of the things mentioned below. Largely, those decisions worked out pretty well. * What kind of directory service do you use? x.500/LDAP? Yellow Pages? In the plan9 world, the directory services were provided by the filesystem and a user-level library that consumed databases of information resident in the filesystem itself (ndb(6) -- https://9p.io/magic/man2html/6/ndb) It also provided DNS services for interacting with the larger world. The connection server was an idea that was ahead of it's time (ndb(8) -- https://9p.io/magic/man2html/8/ndb) Also, https://9p.io/sys/doc/net/net.html. * What kind of distributed authentication do you use? Kerboers? Plan 9 specifically used a Kerberos-like model, but did not use the Kerberos protocol. I say "Kerberos-like" in that there was a trusted agent on the network that provided authentication services using a protocol based on shared secrets. * What kind of distributed authorization service do you use? Unix-style User and group names were simple strings. There were no numeric UIDs associated with processes, though the original file server had a user<->integer mapping for directory entries. * Do you assume that all of the machines in your distributed Not necessarily, no. The model is one of resource sharing, rather than remote access. You usually pull resources into your local namespace, and those can come from anywhere you have access to take them from. They may come from a different "administrative domain". For example, towards the end of the BTL reign, there was a file server at Bell Labs that some folks had accounts on and that one could "import" into one's namespace. That was the main distribution point for the larger community. What if individuals owning their own workstations want to have When a user logs into a Plan 9 terminal, they become the "host owner" of that terminal. The host owner is distinguished only in owning the hardware resources of the hosting machine; they have no other special privileges, nor is there an administrative user like `root`. It's slightly unusual, though not unheard of, for a terminal to have a local disk; the disk device is implicitly owned by the host owner. If the user puts a filesystem on that device (say, to cache a dataset locally or something), that's on them, though host owner status doesn't really give any special privileges over the permissions on the files on that filesystem, modulo them going through the raw disk device, of course. That is, the uid==0 implies you bypass all access permission checking is gone in Plan 9. CPU servers have a mechanism where a remote user can start a process on the server that becomes owned by the calling user; this is similar to remote login, except that the user brings their environment with them; the model is more of importing the CPU server's computational resources into the local environment than, say, ssh'ing into a machine. Or is your I'm not sure how to parse that; I mean, arguably networks of Unix machines associated with any given organization are "centrally owned and administered"? To get access to any given plan9 network, someone would have to create an account on the file and auth servers, but the system was also installable on a standalone machine with a local filesystem. If folks wanted to connect in from other types of systems, there were mechanisms for doing so: `ssh` and `telnet` servers were distributed and could be used, though the experience for an interactive user was pretty anemic. It was more typical to use a program called `drawterm` that runs as an application on e.g. a Mac or Unix machine and emulates enough of a Plan 9 terminal kernel that a user can effectively `cpu` to a plan9 CPU server. Once logged in via drawterm, one can run an environment including a window system and all the graphical stuff from there. Perhaps the aforementioned Bell Labs file server example clarifies things a bit? * What scale should the distributed system work at? 10's of machines This is, I think, where one gets to the crux of the problem. Plan 9 worked _really_ well for small clusters of machines (10s) and well enough for larger clusters (up to 100s or 1000s). Distributed systems that work And how. Plan 9 _was_ eventually ported to football-field sized machines (the BlueGene port for DoE was on that scale); Ron can be able to speak to that in more detail, if he is so inclined. In fairness, I do think that required significant effort and it was, of course, highly specialized to HPC applications. My subjective impression was that any given plan9 network would break down at the scale of single-digit thousands of machines and, perhaps, tens of thousands of users. Growing beyond that for general use would probably require some pretty fundamental changes; for example, 9P (the file protocol) includes a client-chosen "FID" in transactions related to any open file, so that file servers must keep track of client state to associate fids to actual files, whether those file refer to disk-resident stable storage or software synthesized "files" for other things (IPC end points; process memory; whatever). There have been many, many proposals in the distributed computing Is that the case, or is it that system designers try to provide uniform access to different classes of resources? Unix treats socket descriptors very similar to file descriptors very similar to pipes; why shouldn't named resources be handled in similar ways? But that either means that you ...or some other way, which we'll never know about because no one thinks to ask the question, "how could we do this differently?" I think that's the crux of Mothy's point. Plan 9, as just one example, asked a lot of questions about the issues you mentioned above 30 years ago. They came up with _a_ set of answers; that set did evolve over time as things progressed. That doesn't mean that those questions were resolved definitively, just that there was a group of researchers who came up with an approach to them that worked for that group. What's changed is that we now take for granted that Linux is there, and we've stopped asking questions about anything outside of that model. When you add to that the complexities of building an OS that tries to I agree with the first part of this paragraph, but then we're talking about researchers, not necessarily unaffiliated open-source developers. Hopefully researchers have some organizational and infrastructure support! By keeping the local node OS and the distributed computation system I don't know. It seems like this whole idea of distributed systems built on networks of loosely coupled, miniature timesharing systems has led to enormous amounts of inescapable complexity. I'll bet the Kubernetes by itself is larger than all of plan9. - Dan C.

On Thu, Sep 16, 2021 at 08:34:52PM -0400, Theodore Ts'o wrote: Linux runs on _everything_. From your phone to the top 500 HPC clusters. Unix, for all that it did, never had that level of success. I credit Unix for a lot, including showing Linux what it should be, but Linux took that model and ran with it. Plan 9 is very cool but I am channeling my inner Clem, Plan 9 didn't meet Clem's law. It was never compelling enough to make the masses love it. Linux was good enough. We can argue about if that is a good thing or not, I've watched Linux become more complex and seen docker et al react to that.

On Thu, Sep 16, 2021 at 8:34 PM Theodore Ts'o &lt;tytso(a)mit.edu&gt; wrote: Ah, I take your meaning. Yes, I can see that being a problem. But we've had similar problems before: "we only buy IBM", or, "does it integrate into our VAXcluster?" Put another way, _every_ system has opinions about how to do things. I suppose the distinction you're making is that we can paper over so many of those by building abstractions on top of the "node" OS. But the node OS is already forcing a shape onto our solutions. Folks working on the Go runtime have told me painful stories about detection of blocking system calls using timers and signals: wouldn't it be easier if the system provided real asynchronous abstractions? But the system call model in Unix/Linux/plan9 etc is highly synchronous. If `open` takes a while for whatever reason (say, blocking on reading directory entries looking up a name?) there's no async IO interface for that, hence shenanigans. But that's what the local node gives me; c'est la vie. Of course, this doesn't matter if you don't care if anyone uses it I suspect most researchers don't expect the actual research artifacts to make it directly into products, but that the ideas will hopefully have some impact. Interestingly, Unix seems to have been an exception to this in that Unix itself did make it into industry. True, but they aren't. I suspect there are a number of confounding factors at play here; certainly, the breadth and size of the standards they have to implement is an issue, but so is lack of documentation. No one is seriously looking at new system architectures, though. Yup. That is the presentation I meant when I mentioned Rob Pike lamenting the situation 20 years ago in the previous message and earlier in the thread. An interesting thing here is that we assume that we have to redo _all_ of that, though. A lot of the software out there is just code that does something interesting, but actually touches the system in a pretty small way. gvisor is an interesting example of this; it provides something that looks an awful lot like Linux to an application, and a lot of stuff can run under it. But the number of system calls _it_ in turn makes to the underlying system is much smaller. a) There's a lot more Linux in the world than there ever was Unix. b) There are more computers now than there were when Unix was popular. c) computers are significantly more complex now than they were when Unix was written. But to be clear, I don't think this trend started with Linux; I get the impression that by the 1980s, a lot of research focused on a Unix-like model to the exclusion of other architectures. The PDP-10 was basically dead by 1981, and we haven't seen a system like TOPS-20 since the 70s. Or is the argument that it's Linux's fault that Plan 9 has apparently It's hard to make that argument when Linux borrowed so many of plan9's ideas: /proc, per-process namespaces, etc. That wasn't the argument. There are a number of reasons why plan9 failed to achieve commercial success relative to Unix; most of them have little to do with technology. In many ways, AT&T strangled the baby by holding it too tightly to its chest, fearful of losing control the way they "lost" control of Unix (ironically, something that allowed Unix to flourish and become wildly successful). Incompatibility with the rest of the world was likely an issue, but inaccessibility and overly restrictive licensing in the early 90s practically made it a foregone conclusion. Also, it's a little bit of an aside, but I think we often undercount the impact of individual preference on systems. In so many ways, Linux succeeded because, simply put, people liked working on Linux more than they liked working on other systems. You've mentioned yourself that it was more fun to hack on Linux without having to appease some of the big personalities in the BSD world. - Dan C.

Kevin, I think that's a great framing of why this talk actually seemed inverted in its focus for me, and a good identification of why the presenter might see OS development stalling out and ossifying around Linux. I come from the opposite side of the presenter here: my frustration as a backend dev and user has been that modern OSs still think presenting an abstraction over my resources means making it easy to use one single machine (or, as the presenter brings up, a subset of the machine). Instead, my resources are spread out among many machines and a number of remote web services, for which I'd like to have one seamless interface - both for development and use. From an OS perspective, Plan 9 and its daughter systems have come the closest I've seen to addressing this by intentionally thinking about the problem and creating an API system for representing resources that reaches across networks, and a mutable namespace for using and manipulating those APIs. Despite pulling other ideas from 9, the importance of having an opinion on the distributed nature of modern computing seems to have been missed by prominent operating systems today. As a result, their development has been relegated to what they do: be a platform for things that actually provide an abstraction for my resources. And userspace systems have filled the demand for abstracting distributed resource usage to demonstrable business success, if questionable architectural success (as in, they can still be a confusing pain in the buns and require excess work sometimes). As a dev, the systems that have come the closest to presenting one unified abstraction over my resources are the meta-services offered by Google, MS and Amazon such as Azure and AWS. I think the distributed nature of things today is also potentially why the focus of the conference is on distributed systems now, as lamented by the presenter. Granted that I'm not the sharpest bulb in the drawer, but I can't think of a way an OS taking more direct control of the internal hardware of an individual computer would impact me beyond the security issues mentioned in the talk. However, I can think of a number of ways an OS being opinionated about working with networked machines would greatly improve my situation. Boy, it would be great to just spin up a cluster of machines, install one OS on all of them, and treat them as one resource. That's the dream the k8s mentality promises, and MS and Amazon are already walking towards being this sort of one-stop shop: "want cluster computing? Press a button to spin up a cluster with ECS, and store your containers in ECR. Want to run a program or twelve somewhere on the cluster? Just tell us which one and how many. Worried about storage? Just tell us what size storage it needs. We've got you covered!" None of it is perfect, but it shows that there's heavy demand for a system where users don't have to think about how to architect and maintain arbitrary groupings of their resources as necessitated by how OSs think of their job now, and instead just want to feel as if they're writing and running programs on one big 'thing'. So I think the ossification around Linux mentioned in the talk might be that unless operating systems start doing something more than being a host for the tools that actually provide an abstraction over all my resources, there's no real reason to make them do anything else. If you're not making it easier to use my resources than k8s or Azure, why would I want you? Cheers, Marshall On Thu, Sep 2, 2021 at 11:42 AM Kevin Bowling &lt;kevin.bowling(a)kev009.com&gt; wrote:

Much of the problem, I think, is that: 1) an idealized PDP-11 (I absolutely take Warner's point that that idealization never really existed) is a sufficiently simple model that a Bear Of Little Brain, such as myself, can reason about what's going to happen in response to a particular sequence of instructions, and get fairly proficient in instructing the machine to do so in a non-geological timeframe. 2) a modern CPU? Let alone SoC? Fuggedaboutit unless you're way, way smarter than I am. (I mean, I do realize that this particular venue has a lot of those people in it...but, really, those are people with extraordinary minds.) There are enough people in the world capable of doing 1 and not 2 that we can write software that usually mostly kinda works and often gets stuff done before collapsing in a puddle of nasty-smelling goo. There aren't many people at all capable of 2, and as the complexity of systems increases, that number shrinks. In short, this ends up being the same argument that comes around every so often, "why are you people still pretending that the computer is a PDP-11 when it clearly isn't?" Because, as with the keys and the streetlight, that's what we have available to us. Only a grossly oversimplified model fits into our heads. Adam On Fri, Sep 3, 2021 at 8:57 AM Warner Losh &lt;imp(a)bsdimp.com&gt; wrote:

When I took Computer Architecture, "reasoning" about out-of-order execution involved 30-page worksheets where we could track the state of the Tomasulo algorithm through each iteration. It was ludicrously slow work, and wouldn't be a lot of fun even if you had a computerized tool to help step through things instead. If you're talking about a modern Intel CPU where your compiler emits CISC instructions which are actually implemented in RISC instructions in the microcode, which in turn get rewritten and reordered internally by the CPU... it's hard to fault programmers for thinking at the level of the instruction set that's presented to them, even if it looks like a PDP-11. The above should not be read as an endorsement of the CPU status quo, of course :) john ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Friday, September 3rd, 2021 at 10:28 AM, Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote:

Larry - it's the compiler (code generator) folks that I really feel bad for. They had to deal with the realities of the ISA in many ways more than we do and for them, it's getting worse and worse. BTW - there was a misstatement in a previous message. A current CISC system like the INTEL*64 is not implemented as a RISC µcode, nor are current more RISCy machines like the SPARX and Alpha much less the StrongARM and its followers. What they are internally are *data flow machines *which is why you getting a mixing of instruction ordering, scoreboarding, and all sorts of complexities that blows our mind. At least at the OS we have been used to doing things in parallel, exceptions and interrupts occurring and we have reasoned our ways through things. Butler Lampson and Leslie Lamport gave a parallel calculus to help verify things (although Butler once observed at an old SOSP talk that the problem with parallel is what does 'single step the processor mean anymore.' ). So the idea while the processor is not a PDP-10 or PDP-11 much less a 360/91 or a CDC-6600, we build a model in our heads that does simplify the machine(s) as much as possible. We ensure at least that is correct and then, build up more complexity from there. To me, the problem is that we too often do a poor job of what should be the simple stuff and we continue to make it too complicated. Not to pick on any one group/code base, but Jon's recent observation about the Linux kernel FS interface is a prime point. It's not the processor that was made complex, it's the SW wanting to be all things to all people. To me what Unix started and succeed at its time, and clearly Plan9 was attempted in its time (but failed commercially) was to mask if not toss out as much of the complexity of the HW and get to a couple of simple and common ideas and all programs could agree. Going back to the idea of the bear of the 'slittle brain and try to expose the simplest way to computation. Two of the best Unix talks/papers ever, Rob's "cat -v is a bad idea" and Tom's "All Chips that Fit" has morphed into "I have 64-bits of address space I can link anything into my framework" and "what I power and cool in my current process technology" [a SoC is not different that the board level products that some of us lived]. I recently read a suggestion that the best way to teach begging students to be "good programmers" was to "introduce them to as many frameworks as possible and teach as little theory as they need." I nearly lost my dinner. Is this what programming has come to? Framework/Access Methods/Smart Objects .... To be fair, my own employer is betting on DPC++ and believing OneAPI as the one ring to rule them all. There is a lot to be said of "small is beautiful." How did we get from Sixth Edition UNIX with K&R1 to today? One transistor and one line a code at a time. ᐧ On Fri, Sep 3, 2021 at 1:29 PM Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote:

On Thu, Sep 16, 2021 at 12:34:15PM -0700, Jon Steinhart wrote: Amen. I think it is that the newer kids are less willing to understand stuff. So they build something on top that they understand. I agree that they will hit problems and likely build "safe deposit boxes" because the containers are "too complex". Oh, and get off my lawn!

I believe that Docker would not have been nearly as successful if shared libraries had not taken over the Unix world. Docker would have been helpful for putting together Python and Java installations, with all their ancillary bits, but less so for C binaries. Now, though, we run an OS on the machine, put a VM on that, install a container on that, and run another VM (or JVM) inside the container, then maybe we're at the level where we can execute actual code. And that misses all the extra pieces that are distributed and layered and indirected and virtualized for management, logging, security, data warehousing, ... It's a wonder any of it works at all. Here's a map to guide you: https://landscape.cncf.io/ -rob On Fri, Sep 17, 2021 at 9:15 AM Marshall Conover &lt;marzhall.o(a)gmail.com&gt; wrote:

Marshall Conover writes: Wow. I think that it would be more accurate to say that people writing business logic today are targeting the browser because other people are going through the trouble of porting it to different platforms. Doesn't seem to me the best use of resources given that browsers are more complex than operating systems. And I've had many an experience with things that are not portable among browsers. Of course, given that firefox is trying to die by regularly alienating users and that to a first approximation much everything else is chrome or chrome based, you're effectively saying that people are targeting a single operating system even though we don't call a brower an OS. And while there's no going back, I think that browsers suck. Doesn't seem that anybody had the foresight in the early days to realize that they were really building a virtual machine, so the one that we have ended up with is a Rube Goldberg contraption. CSS is one of the brower components that I find especially appalling. I understand its genesis and all that. Would be lovely to be able to make stuff just work without having to "program". Despite revision after revision it's still impossible to lay out things as desired without reverting to JavaScript. While it didn't start that way, at this point there are so many properties with twisty and often undocumented interactions that it would be better to toss it and just write programs. Of course, programming is "hard" and it's supposedly easier to pore though online forums looking for answers to questions like "I think that this can be done but I have no idea how so can someone please share an incantation with me. I personally prefer a language that has a small number of primitives that can be combined in an understandable manner than a huge number of poorly documented primitives that no one person fully understands. And don't tell me that JavaScript is the answer; while it has some good, it suffers from being the dumping ground for people who were never able to get their favorite feature into other languages; it's an incoherent mess. I know that Larry and Clem and I agree on the value of past work. I was a proofreader for Alan Wirf-Brock's 20 years of JavaScript article. I was busy with other stuff when JavaScript began so wasn't familiar with some of the history. Kind of shook my head reading about Eich's big week-long sprint to get the parser up and running. Though to myself that it would have only been a half-day sprint at most had he used existing tools such as lex and yacc, and had he done so we wouldn't still be suffering from the optional semicolon problem 20 years later. Don't mean to offend anybody here, all just my opinion. Jon

Larry McVoy writes: Like usual, we agree on this sort of stuff. A conundrum for me is that this stuff that "they understand" is in my opinion way more complicated than understanding computer hardware and/or an operating system. So I'm not sure where the win is. Jon

On Thu, Sep 16, 2021 at 8:13 PM Al Kossow &lt;aek(a)bitsavers.org&gt; wrote: Y'know, the guy who invented that jargon is a younker of 70 (and from the Labs at that).

On Thu, Sep 16, 2021 at 04:45:15PM -0700, Jon Steinhart wrote: Someone, sorry, I suck at names, I think he is in aerospace or similar, had a pretty rational view on why docker made things easier. It was today so should be easy to find. The part that I don't understand is why it seems so hard to deploy stuff today. We supported the same application, a pretty complicated one, 636K lines of code, on every Unix variant, Linux {32,64} {every arch including IBM 360}, MacOS {PPC, x86, and I'm working on M1}, Windows {XP..} and it wasn't that hard. Granted, of the core team, I'm the least intelligent so I hired well, but still.

David Arnold writes: I have to respectfully disagree which is a challenge because being disagreeable comes more naturally than being respectful :-) We already have this. I kind of wonder what actual value could have been created with the resources that went into supporting the dozen languages, hundred protocols, and so on. Is there value to me that a library exists that lets me to something in python that is identical to the library that lets me do that same thing in perl or the one that lets me do it in php or the one that lets me do it in ...? No. You made a big assumption that I was suggesting tossing prior work and API specs which I wasn't. Would you have wanted to have the 32 bit system call API frozen because it worked and not wanted 64 bit versions? History shows plenty of good work going into compatibility when the underlying technology evolves. Don't know how much time you spend with "the kids" these days. I make it a point to do so when I can; SIGCOVID has cut into that unfortunately. One can get a CS degree without ever taking an OS course at many respectable institutions. Many are not so much making a choice as doing what they can which is inadequate in my opinion. Was discussing this with someone the other day. I'm glad that I have an engineering degree instead of a computer science degree. And I'm also glad that I prevailed with one of my mentees to double major in EE and CS when he wanted to bail on the EE. While it's a generalization, as an engineeer I was educated on how the universe worked - chemistry, physics, and so on. It was up to me to figure out how to apply that knowledge - I wasn't taught how to lay out a circuit board or to use a CAD package or to write an app. A modern CS degree at many institutions is vocational training in programming. It's not the same thing. Jon

Larry McVoy writes: I'm the opposite. Would have double-majored in EE and CS but my school wouldn't allow double majoring unless you stayed for 9 semesters. I finished my EE in 7 semesters and was 2 classes shy of the CS degree and couldn't justify another year just for 2 classes. In hindsight I'm not sure that that was the correct choice because I would have partied my ass off.

Larry McVoy writes: OK, we're getting pretty far afield from UNIX. Probably wasn't using the word "party" in the traditional sense - of the drugs, sex, and rock-n-roll college was consumed by the last two. BSD wasn't a thing when I was in school; I do have a memory of coming in to work at BTL after dinner one evening when Ken was walking out the door with mag tapes under his arm on his way to Berkeley for sabbatical. I got to play with computers and UNIX at BTL, college was building and operating radio stations. Jon

On Sep 16, 2021, at 4:54 PM, David Arnold &lt;davida(a)pobox.com&gt; wrote: I have mixed feelings about this. Unix didn't "throw away" the mainframe world of computing. It simply created a new ecosystem, more suited for the microprocessor age. For IBM it was perhaps the classic Innovator's Dilemma. Similarly now we have (mostly) the Linux ecosystem, while the actual hardware has diverged a lot from the C memory model. There are security issues. There is firmware running on these system about which the OS knows nothing. We have processors like Esperanto Tech's 1088 64 bit Risc-V cores, each with its own vector/tensor unit, 160MB onchip sram and 23.8B transistors but can take only limited advantage of it. We have super performant GPUs but programming them is vendor dependent and a pain. If someone can see a clear path through all this, and create a new software system, they will simply generate a new ecosystem and not worry about 50 years worth of work.

On Fri, Sep 17, 2021 at 10:51:38AM -0700, Jon Steinhart wrote: Apple, much to my surprise, has been thinking about this stuff. The M1 is one of the better thought out chips in a long time. I especially liked the idea of slow (and low battery draw) processors for some apps, and fast ones for apps that need fast. It's an obvious idea in retrospect but I hadn't thought to do it that way. Clever.

On Fri, Sep 17, 2021 at 10:03:42PM +0100, Derek Fawcus wrote: Credit to ARM, but M1 was where I saw it first. The M1 performance is pretty impressive as well. Reminds me of LED bulbs: "Wait, these are brighter, use less power, and cost less? How is that a thing?"

On Sep 17, 2021, at 10:51 AM, Jon Steinhart &lt;jon(a)fourwinds.com&gt; wrote: If you look at the chip architecture, it is pretty regular. https://www.esperanto.ai/technology/ and very low power (0.01W/core as opposed to 7W/core on X86-64) https://www.servethehome.com/esperanto-et-soc-1-1092-risc-v-ai-accelerator-… The equivalent of turrets and porticos and columns and dormers are IO ports like USB, ethernet, and various GPIOs etc. but they use only a small portion of the available gates. IMHO the real issue is that the software folks are *not* providing and *can no*t provide any sort of guidance for general purpose computing, as the memory underlying modern programming languages is so far removed from reality. The situation is sorta like what happens with people with newly acquired incredible wealth but no background in how to spend or manage it wisely (and I don't mean *investing* to get more wealth). At least in that case there are people who can help you and a tremendous need. Here we can put billions and billions of gates on a chip and even do wafer scale integration but these gates are not fungible like money.

On Sep 17, 2021, at 11:56 AM, Jon Steinhart &lt;jon(a)fourwinds.com&gt; wrote: Let me ask you. You have a budget of 24 Billion transistors (& a much more limited power budget). How would you spend it to maximize general purpose computing? Note that Esperanto Tech. was founded by Dave Ditzel, not exactly a h/w newbie. [EE/CS, worked with Dave Patterson, CTO @ Sun, Transmeta founder etc. etc.]. Now it is entirely possible that Esperanto folks are building such processors for special purpose applications like Machine learning but the fact is we simply have an incredible riches of transistors that we don't how to spend wisely, [My opinion, of course]

On Sep 16, 2021, at 12:34 PM, Jon Steinhart &lt;jon(a)fourwinds.com&gt; wrote: Recall that previously sysadmins used programs such as ghost to image a system. A completely operational system with all the required software could be created fairly quickly with minimum configuration. If your h/w crashed, you can get up and running fairly quickly on a new machine (provided your unique bits were backed up & restored). The same thing could be done for server machines. By minimizing differences you can apply security patches or put new machines in service quickly. A server machine needs much more than the main service program before it can be put in real service but machines providing the same service need pretty much the same things. When VMs and containers started getting used, the same model could be used for provisioning them. The docker folks simplified this further. Now you can spin up new servers almost trivially (even if later tooling via Kubernetes and such got quite complicated). Seems to me, this provisioning of whole systems is what users of technologies such as jail never quite got it. A couple of points on this: 1) I think this can be simplified even further if one can rely on a fast control plane connection by basically lazily pulling in the contents of each container. 2) If the underlying system provides a capability architecture, you can probably achieve the exact same functionality without containers as the required "many worlds" functionality is already built in. -- Bakul