yeah, I was on a rant that day, it actually made it into the plan 9 fortunes 'You want to make your way in the CS field? Simple. Calculate rough time of amnesia (hell, 10 years is plenty, probably 10 months is plenty), go to the dusty archives, dig out something fun, and go for it. It's worked for many people, and it can work for you. - Ron Minnich ' I'm sorry to say, it still seems to work that way. I just saw a talk from red hat about putting linux in flash to use as a boot loader, for example, and the authors somehow ignored the last 25 years and acted like they invented it. Amazing. Turns out 10 minutes is enough. On Sun, Jun 16, 2024 at 6:05 PM Larry McVoy <lm(a)mcvoy.com> wrote:

On Jun 16, 2024, at 8:57 PM, ron minnich <rminnich(a)gmail.com> wrote: I am not a fan of systemd (or linux) and don't follow their excesses/adventures but I am not a fan of how BSD does initialization & brings up services either. They don't quite get all the dependencies right for all the possible combinations of devices etc. Its /etc/rc.d/* system is pretty clunky -- I tend to think any time you are repeating more or less the same boilerplate code in many files, something worth abstracting is hiding in there. I like how launchd treats a service as an object (more than just a program but also the auxiliary files and scripts). For me it was a lightbulb moment (like realizing how a function operates in an environment!). Though I'd probably use s-expr or a simpler config format, not xml (as in launchd plist/SMF manifest). At the other extreme of complexity we have things like Kubernetes. Not a fan. What I want is to be able to map all my computers and compute clusters into a single virtual machine -- where storage, IO and computing resources may be added / removed without taking the whole VM down, and where each display/input user interface is a window on the same underlying VM and all sharing is under my control. Plan9 does a bit of this but that experiment ended too early. Apple is tending in this direction though not cleanly (+ I don't want to rely on a faceless behemoth corp that may trample on my data without even meaning to).

On Mon, Jun 17, 2024 at 1:51 AM Bakul Shah via TUHS <tuhs(a)tuhs.org> wrote: Jerry and Bruce's book is the complete reference: https://www.amazon.com/Distributed-System-Architecture-Computer-Systems/dp/… There were basically 3/4 versions... the original version of the PDP 11 which is the SOSP paper, which morphed to include a VAX at UCLA; IBM's AIX/370 and AIX/PS2 which included TCF (Transparent Computing Facility), and LCC's TNC Transparent Networking Computing "product" which were the 14 core technologies used to built it. Part of them landed in other systems from Tru64, HPUX, the Paragon and even a later a Linux implementation (which sadly was done on the V2 kernel so was lost when Linus did not understand it). What worked well was different flavors of the DFS and the later core idea of the VPROCS layer which I sorely miss, which allowed process migration - which w worked well and boy did I miss later in my career. Admin of a Locus based system was a dream because it was just one system for up to 4096 nodes in a Paragon. It also means you could migrate processes off a node, take the node down, reboot/change and bring it back. Very cool. After the first system was installed, adding a node was trivial, by the way. You booted the node, "joined" the cluster, and were up. AIX used file replication to then build the local disks as needed. BTW: "checkpointing" was a freebie -- you just migrated the file to a disk. Mixing ISA like the 370 and PS/2 was a mixed bag -- I'll let Charlie comment. With TNC we redid that model a bit, I'm not sure we ever got it 100% right. The HP-UX version was probably the best. The biggest implementation issue is that UNIX has too many different namespaces with all sorts of rules that are particular to each. For all of the concept of "everything is a file," - when you start to try to bring it together, you discover new and werid^H^H^H^H^Hintersting name spaces from System V IPC to signals to FIFOs and Name Pipes (similar but different). It seemed like everything we looked, we would find another NS we needed to handle, and when we started to try to look at non-UNIX process layers, it got even stranger. The original UNIX protection model is a tad weak, but most people had started to add ACLs, and POSIX was in the throughs of standardizing them -- so we based it on an early POSIX proposal (mostly based on HP-UX since they had them before the others did). To be more specific, the virtual process layer (VPROC) attempted to do what VFS had done for the FS layer to the core kernel. If you look at both the original 2 Locus schemes, process control was ad hoc and thus very messy. LCC realized if we were going to succeed, we needed to make that cleaner. But that still took major surgery - although, like the CFS layer, things were a lot clearer once done. Bruce, Roman, and I came up with VPROCs. BTW: one of the cool parts of VPROC is like VFS. It conceptually made it possible to have other process models. We did a prototype for OS/2 running inside of the OSF uK and were trying to get a contract from DEC to do it to Tru64 and adding VMS before we got sold (we had already developed CFS for DEC as part of Tru64 - which TNC's Cluster File System). Truth is, cheap VMs killed the need for this idea, but it worked fairly well. After the core VPROCs layer, the hardest thing was distributed shared memory (DSM) and the distributed lock manager (DLM). DSM was an example that offered pure transparency in operation, *i.e.,* test and set worked (operationally) correctly across the DSM, but it was not "speed transparent." But if you rewrote to use DLM, then you could get full transparency and speed. The DLM is one of the TNC technology which lives on today. It ended up in a number of systems - Oracle wrote their own based on the specs for the DEC DLM we built for the CFS for Tru64 (which is from TNC). I believe a few other folks used it. It was in OSF's DCE, and ISTR Microsoft picked it up. So a good question is if TNC was so cool, why did Beowulf (a real hack in comparison) stick around and TNC die? Well, a few things. LCC/HP did not open-source the code until it was too late. So Beowulf, which was around, was what folks (like me) used to build big scientific clusters. And while Popek was "right," -- it takes something like Locus/TNC to make a cluster fully transparent. Beowulf ignored the seams and i the end, that was "good enough." But it makes setup and admin a PITA, and the program needs to be careful -- the dragons are all over the place. So, when I went to Intel, I was the Architect of Cluster Ready, which defined away many of those seams and then provided tools to test for them and help you admin. Tools like the Cluster Checker and the whole ClusterReady program would not be needed if TNC had "stuck," and I think clusters, in general, a cluster of small computers on a LAN, not just clusters on a high-speed/special interconnect like a supercomputer, would be more available today. Clem ᐧ

Clem suggests I comment on mixing ISA. I'm not sure how to respond. I saw Bruce and Jerry demo process migration many times, particularly during our dramatic Santa Monica meetings in October 1987, coincident with the Whittier earthquake. However, I never got a chance to work with this myself. (During the strongest aftershocks, Bruce and I would just stare and hold on to our chairs. Having us Austin IBM folks in Santa Monica to try to resolve the Austin/LCC disagreements seemed historic, but probably not the cause of the October 19 Wall Street crash.) In general, I was always impressed by what Bruce and Jerry did, but the assertions that LCC could do everything exacerbated the ongoing political challenges within IBM. To repeat from https://notes.technologists.com/notes/2017/03/08/lets-start-at-the-very-beg…: o "The former LCC person has mentioned that IBM then seemed like N competing companies. Actually, it was more like M_n competing factions within N competing companies." o "The traditional product organizations, e.g., those associated with the 370 and the System 3x, saw little need for UNIX or a new hardware architecture. The renegade but surprisingly successful PC organizations looked askance for their own reasons. Even the Yorktown partners were partly detrimental because of disdain for UNIX." [To amplify on this, in 1984 CEO John Akers told a gathering of Austin IBM managers that he questioned the need for RISC processors and UNIX.] o "Besides our technical concerns about distributed system issues, the implicit question seemed an all or nothing proposition of continuing AIX vs. IBM depending on LCC for UNIX." And we could dwell on OSF, DCE, etc. On the day OSF was announced, with Akers on stage with Ken Olsen, Akers flew across the country to an awards event, where Glenn Henry, Larry Loucks and I received substantial checks in recognition of AIX. When Akers shook my hand, he told me how proud he was of what had happened that day. When I saw the Register article, I knew that systemd folks hadn't boasted '42% less Unix philosophy', that it was really someone on mas.to, but I felt like stirring up discussion. Seems to have worked... Charlie On 6/17/2024 11:00 AM, Clem Cole wrote: -- voice: +1.512.784.7526 e-mail:sauer@technologists.com fax: +1.512.346.5240 Web:https://technologists.com/sauer/ Facebook/Google/LinkedIn/Twitter: CharlesHSauer

Bakul Shah via TUHS wrote in &lt;653E15D7-DD66-414C-94F3-A74B4EE3DD10(a)iitbombay.org&gt;: |On Jun 16, 2024, at 8:57 PM, ron minnich &lt;rminnich(a)gmail.com&gt; wrote: |> I'm curious, as to the original topic of this discussion: can anyone \ |> justify systemd-homed and how it works? Does that even look like \ |> 0% of a unix idea? | |I am not a fan of systemd (or linux) and don't follow their excesses/adv\ |entures but I am not a fan of how BSD does initialization & brings \ |up services either. They don't quite get all the dependencies right \ |for all the possible combinations of devices etc. Its /etc/rc.d/* \ |system is pretty clunky -- I tend to think any time you are repeating \ Now even more since they started to add a "jail-this-service" variable, ie containerization by setting a variable. |more or less the same boilerplate code in many files, something worth \ |abstracting is hiding in there. | |I like how launchd treats a service as an object (more than just a \ |program but also the auxiliary files and scripts). For me it was a \ |lightbulb moment (like realizing how a function operates in an environme\ To me -- it turned off my light as i tried to "do something", but could not figure out how; i somehow managed to create the XML file necessary. I am happy to have forgotten what it was about. Ah, wait, voila: $ v ~/arena/misc/macosx-plist-use.txt <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.bell-labs.plan9.u9fs</string> <key>Program</key> <string>/usr/bin/u9fs</string> <key>ProgramArguments</key> <array> <string>u9fs</string> <string>-l</string> <string>/var/log/u9fs.log</string> <string>-a</string> <string>p9any</string> <string>/opt/plan9</string> </array> <key>Sockets</key> <dict> <key>Listeners</key> <dict> <key>SockServiceName</key> <string>9pfs</string> </dict> </dict> <key>inetdCompatibility</key> <dict> <key>Wait</key> <false/> </dict> </dict> </plist> To cause this to be run on system start, this should be installed as /Library/LaunchDaemons/9pfs.plist. Installing instead in /Library/LaunchAgents will cause it to be run only when a user is logged in, while $HOME/Library/LaunchAgents will cause it to be run only when that particular user is logged in. In order to start the listner it must first be ``loaded'' $ sudo launchctrl load /path/to/9pfs.plist If you are running the Mac OS X firewall you will need to add an entry pass the 9pfs protocol in: SystemPreferences->Sharing->Firewall I give you ten points for configuration lightbulb moments! So nice and easy, also for human consumption once written. ... |What I want is to be able to map all my computers and compute clusters \ |into a single virtual machine -- where storage, IO and computing resources \ |may be added / removed without taking the whole VM down, and where \ |each display/input user interface is a window on the same underlying \ |VM and all sharing is under my control. Plan9 does a bit of this but \ |that experiment ended too early. Apple is tending in this direction \ |though not cleanly (+ I don't want to rely on a faceless behemoth corp \ |that may trample on my data without even meaning to). I had that dream somewhen spoken out in a FreeBSD IRC channel a few years back. It *could* be that the new per-service-jails do it a bit like that, via nullfs mounting, that deep i have not looked into it yet. But my idea was that the base system is mounted partially, and you would specify the PKGs you want in the jail, and that only the files of the given pkgs are actually visible in the jail. I use something a bit similar for some boxed things here on Linux, with overlayfs; however, after mount -n -t overlay -o upperdir=${rundir}/storage,lowerdir=/,workdir=${rundir}/work \ overlayfs ${rundir}/root || exit 21 i then start rm(1) removal of some files, eg rm -rf \ ${rundir}/root/boot \ ${rundir}/root/home \ ${rundir}/root/media \ ${rundir}/root/opt \ ${rundir}/root/root \ ${rundir}/root/run \ ${rundir}/root/var \ plus over-mounting things like dev # devtmpfs fully populates instead, including log socket etc!! #mount -n -t devtmpfs dev ${rundir}/root/dev || exit 50 mount -n -t tmpfs -o nosuid,noexec dev ${rundir}/root/dev || exit 50 etc etc etc. That is *not* what i meant. That idea is old, i have not yet managed it to create a shareable root system, which is then *per-se* overlay mounted, even by the system itself. (That is: the base is shared by the base system and containers, which then add onto what they want and need. It could even be mounted as a base for real VMs.) Regarding systemd my only hope is that Linux remains usable without it. It seems more and more require the systemd infrastructure in order to function, i have heard. That "super / sudo / doas / [BSD] su -c" replacement that systemd just recently added, somehow i followed a link to the github or so issue where this was discussed, and i still hear the lead developer of AlpineLinux ask for a separate udev, a part that anyone needs, i think he did not even get an answer. Which deterred me further. (I think the AlpineLinux lead developer's name is pretty well-known in that society.) Hmm. It *could* be that it was in fact in another issue that turned fixed linked-in libraries like compressors like xz (the backdoor there, you know) into dlopen()-managed mess, via kinda marshalling. There. Whatever. Btw i did not sent out another email last week Jim Capp wrote in <1403506.1536.1718310415450.JavaMail.root@zimbraanteil>: |https://nosystemd.org/ It is the pride and ignorance of the billion dollars that pay lots of developers on the Linux front that makes me sad. For example, on ossec-, we repeatedly see an OpenSUSE employee, who seems to get paid for doing security audits, publish security advisories. That is (mostly seems to be) a one man show. Of course, many things happen behind the scenes, in bug trackers etc. But i track fulldisclosure and ossec- for way over a decade. And "the same is true" for the boot and daemon monitoring environment. On FreeBSD, for example, one programmer is working to integrate "jail"ing (FreeBSD jails: twenty+ years ago it was a precondition in some system calls, looking for "is-jailed", plus dedicated network stack etc; usually (i hate it) also with its dedicated file system aka mounts etc. etc) into daemon startup aka the rc system, so you (that seems to be the idea) just set a rc.conf variable and the service is "boxed" in a jail automatically. If you look at all the rotten data in the Linux login etc (with PAM or not, etc etc), just a few programs in the startup process, but it is too much to keep them in line Yeah! [.] with modern achievements like PR_SET_CHILD_SUBREAPER to move entire process hierarchies to dedicated zombie collectors and such that is to say, with capabilities and all that, which systemd makes easy, via easy text config files. Ie, namespace containment (aka network stack etc isolation), at your fingertips. But per se a stacked call like cd / ip netns exec ${netns} \ /usr/bin/env -i AUTHDISPLAY=${AUTHDISPLAY} DISPLAY=${DISPLAY} TERM=${TERM} XAUTHORITY=${XAUTHORITY} \ /usr/bin/unshare --ipc --uts --pid --fork --mount --mount-proc ${kill_child} ${rooter} ${prog} & pid=${!} [ -d /sys/fs/cgroup/_box_web ] && printf '%s\n' ${pid} > /sys/fs/cgroup/_box_web/cgroup.procs # if [ ${netns} = secweb ] || [ ${netns} = priwse ]; then wait $pid cleanup_setup # fi exec 7>&- can (add capabilities) do the very same thing. Btw there is an init-on-steroids [1] from the guy who took maintainership of sysklogd quite some years ago (used by my Linux distro ever since), it can also supervise, use cgroups, etc. I plan to try it out for years, but since i realized i have to go second line i have written some scripts, and then just call in via SysV or OpenRC, or what. I mean, i have no problem with the notification stuff of systemd, it is now even included in OpenSSH (optionally, in openbsd-compat/port-linux.c), but i mean the PID file things are used for decades, and with "daemons and zombies are reparented to a configured subreaper process" (instead of PID 1), and with the fully capable but not systemd integrated udev (all "looser" Linux distributions "have to use" eudev). Ah! Talking too much!! [1] https://github.com/troglobit/finit --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)