21 Feb
2021
21 Feb
'21
5:09 a.m.
- separation of code and data using read-only and
read/write file systems
I'll bite. How do you install code in a read-only file system? And
where does a.out go?
My guess is that /bin is in a file system of its own. Executables from
/letc and /lib are probably there too. On the other hand, I guess
users' personal code is still read/write.
I agree that such an arrangement is prudent. I don't see a way,
though, to update bin without disrupting most running programs.
Doug
21 Feb
21 Feb
2:15 p.m.
On Sat, Feb 20, 2021 at 06:09:42PM -0500, M Douglas McIlroy wrote:
I always wonder how to distunguish data and programs when people want
to separate them. One person's data is another person's program and
vice versa. Think scripting, config files, grammar definitions,
postscript files, exectuables to be fed to emulators, compilers,
linkers, code analysis tools, the examples are endless.
Turing already saw that form the theoretical point of view, others
(like Von Neumann) more from the practical persppective.
Data = Programs.
-Otto
- separation
of code and data using read-only and read/write file systems
I'll bite. How do you install code in a read-only file system? And
where does a.out go?
My guess is that /bin is in a file system of its own. Executables from
/letc and /lib are probably there too. On the other hand, I guess
users' personal code is still read/write.
I agree that such an arrangement is prudent. I don't see a way,
though, to update bin without disrupting most running programs.
Doug 
5:08 p.m.
On Feb 20, 2021, at 15:09, M Douglas McIlroy
<m.douglas.mcilroy(a)dartmouth.edu> wrote:
Disclaimer: I haven't actually used Nerves myself, just watched some presentations,
read various web pages, etc. So anything I say about it is quite unreliable. And,
although that item was (sort of) true, it was obviously rather misleading if interpreted
too broadly. So, I'll try to provide some context to explain what I meant by it.
As I understand it, Nerves is intended as a build and delivery mechanism for IoT system
software. It's supposed to be possible to upgrade a deployed device without blowing
away its persistent saved state. And, if the upgrade fails, to back down to the previous
version. Also, the running code on the device should not be able to trash the system
software.
To support this, they use multiple file systems, with various updating attributes. For
example, they might have two file systems for the system software and a third one for the
persistent saved state. This lets a developer upload and boot a new copy of the system
software, but fall back to the old version if something goes wrong.
-r
- separation of code and data using read-only and
read/write file systems
I'll bite. How do you install code in a read-only file system? 
22 Feb
22 Feb
4:40 a.m.
On Sat, 20 Feb 2021, M Douglas McIlroy wrote:
I once worked for a place who reckoned that /bin and /lib etc ought to be
in an EEPROM; I reckon that he was right (Penguin/OS dumps everything
under /usr/bin, for example).
- separation
of code and data using read-only and read/write file
systems
I'll bite. How do you install code in a read-only file system? And where
does a.out go? My guess is that /bin is in a file system of its own.
Executables from
/letc and /lib are probably there too. On the other hand, I guess users'
personal code is still read/write.
That's how we ran our RK-05 11/40s since Ed 5... Good fun writing a DJ-11
driver from the DH-11 source; even more fun when I wrote a UT-200 driver
from the manual alone (I'm sure that "ei.c" is Out There Somewhere),
junking IanJ's driver.
The war stories that I could tell...
I agree that such an arrangement is prudent. I
don't see a way, though,
to update bin without disrupting most running programs.
Change is inevitable; the trick is to minimise the disruption.
-- Dave, who carried RK-05s all over the UNSW campus
5:01 a.m.
On Mon, 22 Feb 2021, Dave Horsfall wrote:
I once worked for a place who reckoned that /bin and
/lib etc ought to be in
an EEPROM; I reckon that he was right (Penguin/OS dumps everything under
/usr/bin, for example).
I have used distributions in the past that maintained the traditional
distinction.
While I've been stuck regarding bringing up a kernel, C compiler and libc
all together, (keeping in mind my desire to avoid gcc and glibc for the
project) the conceptual distribution I've been working on for some time
uses more or less the same abstraction as the BSDs, with distinct /bin and
/sbin vs. /usr/bin and /usr/sbin as I personally believe it should be,
that the stuff in /bin should be enough to bring up and/or run diagnostics
on a system, and everything else go in /usr.
-uso.
23 Feb
23 Feb
9:31 a.m.
On 2/21/21, Steve Nickolas <usotsuki(a)buric.co> wrote:
While I've been stuck regarding bringing up a kernel, C compiler and libc
all together, (keeping in mind my desire to avoid gcc and glibc for the
project) the conceptual distribution I've been working on for some time
uses more or less the same abstraction as the BSDs, with distinct /bin and
/sbin vs. /usr/bin and /usr/sbin as I personally believe it should be,
that the stuff in /bin should be enough to bring up and/or run diagnostics
on a system, and everything else go in /usr.
I don't see much of a point in maintaining the separation these days.
/bin and /usr/bin were originally separated because it wasn't possible
to fit everything on one disk, and (AFAIK) the separation was mostly
maintained after that to reduce the chance of filesystem corruption
rendering the system unbootable (which is much less of a problem
nowadays because of journalled and log-structured filesystems).
Under UX/RT, the OS I'm writing, all commands (administrative or
otherwise) will appear to be in /bin, and all daemons will appear to
be in /sbin (with corresponding symlinks in /usr). The separation into
administrative and regular commands will be meaningless since the
traditional root/non-root security model will be completely eliminated
in favor of role-based access control. The / and /usr separation will
be useless since it will be impossible to have a separate /usr
partition (the contents of the root will be dynamically bound from a
collection of individual package directories, and won't correspond to
the root of the system volume).
11:29 p.m.
New subject: /usr separation (was: Abstractions)
At Mon, 22 Feb 2021 20:31:49 -0700, Andrew Warkentin <andreww591(a)gmail.com> wrote:
Subject: Re: [TUHS] Abstractions
On 2/21/21, Steve Nickolas <usotsuki(a)buric.co> wrote:
Maybe there isn't any impetus to _create_ a separate /usr these days of
large software but even larger disks.
However I think there are at least two good reasons to _maintain_ a
separate /usr. At least for ostensibly POSIX and Unix compatible
systems, that is.
For one there's a huge amount of deeply embedded lore, human (finger and
brain) memory, actual code, documentation, and widespread practices that
use this separation and rely on it, effectively making it a requirement.
As Steve mentions above there's also the concept of knowing the minimum
requirements for bringing up a system capable of the most basic tasks.
Of course there's likely going to be some variance in what any given
person might define as "most basic tasks", but that's most a separate
issue. However I will give one example of why this might be a good
thing to know and preserved: it is highly useful for those creating
"embedded" systems, or application specific systems. They can start
with just the minimal root filesystem, and then know exactly what they
have to add in order to meet their application's requirements precisely.
(and the reasons for doing that can be much wider than many might assume)
Also the basic idea of having a root filesystem that contains just and
only what's necessary for the system to boot and run, and putting
everything else that makes the system usable to users into /usr, is also
still a worthwhile concept even just on its own.
The maintenance of an illusion of a separate /usr can of course be
easily done with a farm of symlinks, thus preserving any dependencies in
anyone's memory, documentation, or code.
However the reality of maintaining a separate minimal toolset for system
bring-up is that it cannot be reliably done without constant and
pervasive testing; and the very best (and perhaps only) way to achieve
this, especially in any smaller open-source project, is for everyone to
use it that way as much of the time as possible. I say this from
decades long experience of slowly moving systems to having just one
partition for both root and /usr and then on occasion testing with
separate root and /usr, and every time I do this testing I find
dependencies have crept in on something in /usr for basic booting. (and
that's even when I base my system on a platform that still tries hard to
maintain this separation of root and /usr!)
BTW, I think it was Sun that first did some of this merging of root and
/usr a very long time ago.
--
Greg A. Woods <gwoods(a)acm.org>
Kelowna, BC +1 250 762-7675 RoboHack <woods(a)robohack.ca>
Planix, Inc. <woods(a)planix.com> Avoncote Farms <woods(a)avoncote.ca>
While I've been stuck regarding bringing up a kernel, C compiler and libc
all together, (keeping in mind my desire to avoid gcc and glibc for the
project) the conceptual distribution I've been working on for some time
uses more or less the same abstraction as the BSDs, with distinct /bin and
/sbin vs. /usr/bin and /usr/sbin as I personally believe it should be,
that the stuff in /bin should be enough to bring up and/or run diagnostics
on a system, and everything else go in /usr.
I don't see much of a point in maintaining the separation these days.
/bin and /usr/bin were originally separated because it wasn't possible
to fit everything on one disk, and (AFAIK) the separation was mostly
maintained after that to reduce the chance of filesystem corruption
rendering the system unbootable (which is much less of a problem
nowadays because of journalled and log-structured filesystems). 
24 Feb
24 Feb
12:28 a.m.
New subject: /usr separation
On 2/23/21 10:29 AM, Greg A. Woods wrote:
Maybe there isn't any impetus to _create_ a
separate /usr these days
of large software but even larger disks.
I'm undecided. Part of me likes the / (root) and /usr split. But
another part of me questions /if/ and (if so) /why/ it is (still) /needed/.
However I think there are at least two good reasons to
_maintain_
a separate /usr. At least for ostensibly POSIX and Unix compatible
systems, that is.
Does /usr actually /need/ to be a /separate/ file system? Or would a
wholesale link from /usr to / (root) suffice? Or perhaps a collection
of sym-links from /usr/<foo> to /<foo> suffice?
For one there's a huge amount of deeply embedded
lore, human
(finger and brain) memory, actual code, documentation, and widespread
practices that use this separation and rely on it, effectively making
it a requirement.
Are they relying on the /separation/ of separate file systems? Or are
they simply relying on wrote memory for the path? Ergo sym-links could
fulfill the perceived need?
As Steve mentions above there's also the concept
of knowing the
minimum requirements for bringing up a system capable of the most
basic tasks.
The pat response to this in the Linux community is "That's what the
initrd / initramfs is for!"
What that fails to take into account is if the system actually uses an
initrd / initramfs or not. Many of the systems I maintain do /not/ use
an initrd / initramfs. Thus the systems have /some/ actual /need/ to be
able to bring up a minimal system to repair file system problems. Even
if the so called problem is simply that the extent file system needs an
fsck with human interaction (time since last check and / or maximum
number of mounts).
If you do use an initrd / initramfs, then you can reasonably safely lump
everything* in the / (root) file system.
*/boot still tends to be it's own file system on Linux, mostly because
that's where the initrd / initramfs image live which contain drivers for
more fancy things (software RAID, LVM, ZFS, SAN, etc.) which are needed
to bring up / (root).
Of course there's likely going to be some
variance in what any
given person might define as "most basic tasks", but that's most a
separate issue.
Agreed.
However, I posit that "most basic tasks" be what is necessary to
transition from single user mode to multi-user mode. Including any and
all utilities required to fix file systems, work with logical volumes,
SAN, etc.
However I will give one example of why this might be a
good thing to
know and preserved: it is highly useful for those creating "embedded"
systems, or application specific systems. They can start with just the
minimal root filesystem, and then know exactly what they have to add
in order to meet their application's requirements precisely. (and the
reasons for doing that can be much wider than many might assume)
Please elaborate on what that has to do with the / (root) vs /usr split?
I feel like you're differentiating between a minimal install vs a
kitchen sink install. Which seems to me to be independent of how the
underlying file system(s) is (are) arranged.
Also the basic idea of having a root filesystem that
contains just
and only what's necessary for the system to boot and run, and putting
everything else that makes the system usable to users into /usr,
is also still a worthwhile concept even just on its own.
Many in the Linux community think this is the job of the initrd /
initramfs. I personally believe that this is the job of the / (root)
file system.
Aside: In the event that /usr is on the / (root) file system, then the
system should still be able to come up as if /usr didn't exist b/c it
had been renamed or was on a separate file system.
The maintenance of an illusion of a separate /usr can
of course be
easily done with a farm of symlinks, thus preserving any dependencies
in anyone's memory, documentation, or code.
Agreed. With things like bind mounts, we don't even need to use
sym-links. }:-)
Though, one potential danger is that people see duplication between
/bin/<foo> and /usr/bin/<foo> and decide to remove one of them. Doing
so will ultimately remove both and cause someone to have a not good day.
Aside: Perhaps these not good days are not something to be avoided, but
instead something to be treated as a learning opportunity. Much like
young kids need to learn that fire is hot for themselves.
However the reality of maintaining a separate minimal
toolset for
system bring-up is that it cannot be reliably done without constant
and pervasive testing; and the very best (and perhaps only) way to
achieve this, especially in any smaller open-source project, is for
everyone to use it that way as much of the time as possible. I say
this from decades long experience of slowly moving systems to having
just one partition for both root and /usr and then on occasion testing
with separate root and /usr, and every time I do this testing I find
dependencies have crept in on something in /usr for basic booting.
(and that's even when I base my system on a platform that still tries
hard to maintain this separation of root and /usr!)
I have a different conundrum regarding */bin. Why do I need nine
different (s)bin directories in my path? I -- possibly naively --
believe that we have the technology to have all commands in /one/
directory, namely /bin.
Quickly after that thought, I realize that I want different things in my
path than other people do. So I end up with custom /bin directories.
Which usually ends up with sym-links that reference variables or custom
mounts (possibly via auto-mount applying some logic).
BTW, I think it was Sun that first did some of this
merging of root
and /usr a very long time ago.
Agreed. Though I'm far from authoritative.
--
Grant. . . .
unix || die
12:57 a.m.
New subject: /usr separation
On Tue, Feb 23, 2021 at 11:28:13AM -0700, Grant Taylor via TUHS wrote:
What that fails to take into account is if the system actually uses an
initrd / initramfs or not. Many of the systems I maintain do /not/ use an
initrd / initramfs. Thus the systems have /some/ actual /need/ to be able
to bring up a minimal system to repair file system problems. Even if the so
called problem is simply that the extent file system needs an fsck with
human interaction (time since last check and / or maximum number of mounts).
There are two reasons why you might want to have an initramfs. One is
you are using a distribution-provided generic kernel, in which case
the device driver / kernel modules needed to access the root file
system needed to be loaded from *somewhere*, and that's the in-memory
initramfs/initrd.
The other reason is how you run fsck on the root file system. That
won't be needed if hardware is perfect, the kernel is bug-free(tm),
and the root file system has journalling support, as all modern file
systems tend to have. However, if it is needed, there are two ways to
do this. One is the traditional way, which is to mount the root file
system read/only, repair the file system, and if any changes were
required to the root file system, force a reboot; otherwise, remount
the root file system read-write, and proceed. The other way of doing
this is to include the fsck program in the initrams, and run fsck on
the root file system before it is mounted. Now you never have to
worry about rebooting if any chances were made, since the root file
system wasn't mounted and so there is no danger of invalid metadata
being cached in memory.
That being said, it's certainly possible to skip using an initramfs;
it's geenrally not required, and if you're building your own kernel,
with the device drivers you need for your hardwaer compiled into the
kernel, most distributions will support skipping the initramfs.
(Debian certainly does, in any case.)
*/boot still tends to be it's own file system on
Linux, mostly because
that's where the initrd / initramfs image live which contain drivers for
more fancy things (software RAID, LVM, ZFS, SAN, etc.) which are needed to
bring up / (root).
/boot needs to exist due to limitations to the firmware and/or boot
loader being used. If the boot loader is using the legacy PC Bios
interfaces to read the kernel and initial ramdisk/file system, then
those files need to be in a low-numbered LBA disk space, due to legacy
BIOS/firmware limitations. It could also be a concern if you are
using some exotic file system (say, ZFS), and the bootloader doesn't
support that file system due to copyright licensing incompatibilities,
or the boot loader just not supporting that bleeding-edge file system.
In that case, you might have to keep /boot as an ext4 file system.
Other than that, there is no reason why /boot needs to be its own file
system, except that most installers will create one just because it's
simpler to use the same approach for all cases, even if it's not
needed for a particular use case.
- Ted
P.S. Oh, and if you are using UEFI, you might need to have yet
another file system which is a Microsoft FAT file system, typically
mounted as /boot/efi, to keep the UEFI firmware happy....
2:29 a.m.
New subject: /usr separation
On 2/23/21 11:57 AM, Theodore Ts'o wrote:
There are two reasons why you might want to have an
initramfs.
Rather than getting into a tit for tat debate, I'll agree that we
have
both proposed reasons why you /might/ want to use an initramfs. The
operative words are "you" and "might". Each person probably wants
slightly different things. It's far from one size fits all.
The other reason is how you run fsck on the root file
system.
The same way that it's been done for years. Root is mounted read only
and you run fsck to repair damage. If it's severe damage, you will
likely need to boot off of something else. I've had both situations
happen multiple times.
The quintessential max mount count / max days since last check have
happily been fixed while root was mounted read only.
That won't be needed if hardware is perfect, the
kernel is
bug-free(tm), and the root file system has journalling support,
as all modern file systems tend to have.
I wouldn't bet on that. I've had to run fsck on journalling file
systems at boot / mount time multiple times.
However, if it is needed, there are two ways to do
this. One is the
traditional way, which is to mount the root file system read/only,
repair the file system, and if any changes were required to the root
file system, force a reboot; otherwise, remount the root file system
read-write, and proceed.
This is what happened in /most/ of the cases that I've needed to
interact with fsck of a root file system.
The other way of doing this is to include the fsck
program in the
initrams, and run fsck on the root file system before it is mounted.
Now you never have to worry about rebooting if any chances were made,
since the root file system wasn't mounted and so there is no danger
of invalid metadata being cached in memory.
Oh ... I would definitely *NOT* say /never/. There are ways that a file
system can get corrupted that will cause fsck to stop and require manual
intervention.
That being said, it's certainly possible to skip
using an initramfs;
it's geenrally not required, and if you're building your own kernel,
with the device drivers you need for your hardwaer compiled into
the kernel, most distributions will support skipping the initramfs.
(Debian certainly does, in any case.)
And if you're building a minimal kernel, removing support for modules
and what's required for swing-root saves space. ;-)
/boot needs to exist due to limitations to the
firmware and/or boot
loader being used.
Not necessarily. E.g. one single partition containing /boot and / (root).
If the boot loader is using the legacy PC Bios
interfaces to read the
kernel and initial ramdisk/file system, then those files need to be in
a low-numbered LBA disk space, due to legacy BIOS/firmware limitations.
So make sure said /boot & / (root) partition stays within that
limitation. I don't recall exactly what that is. I think it's ~8 GB.
But it's definitely possible to have small installations in that space.
It could also be a concern if you are using some
exotic file system
(say, ZFS), and the bootloader doesn't support that file system due
to copyright licensing incompatibilities, or the boot loader just not
supporting that bleeding-edge file system. In that case, you might
have to keep /boot as an ext4 file system.
That scenario is definitely a possibility. Though such scenarios are
not a requirement and tend to be antithetical to minimal installations,
like the type that would be used in embedded devices and possibly copied
to ROM as indicated in a different post.
Other than that, there is no reason why /boot needs to
be its own
file system, except that most installers will create one just because
it's simpler to use the same approach for all cases, even if it's
not needed for a particular use case.
As Steve Gibson is famous for saying; The tyranny of the default.
P.S. Oh, and if you are using UEFI, you might need to
have yet
another file system which is a Microsoft FAT file system, typically
mounted as /boot/efi, to keep the UEFI firmware happy....
Yes, the file system needs to exist. But that's part of the firmware,
not the operating system. I also question if that FAT file system needs
to be mounted or not. -- I don't know how GRUB et al. deal with a
non-mounted UEFI file system.
But even if it does need to be mounted, you can still get away with two
partitions; / (root) and /boot/efi. I suspect UEFI does away with the
LBA issue you mentioned.
--
Grant. . . .
unix || die
8:14 p.m.
New subject: /usr separation
On Tue, Feb 23, 2021 at 01:29:11PM -0700, Grant Taylor via TUHS wrote:
Sorry, I should have written, "/boot MAY need to exist".
I wouldn't say that; I'd rather say that if you have a huge
combination of configurations that you have to test, those
configurations which aren't regularly tested will tend to bitrot, or
have odd failures in various error cases. The more corners that you
have, the more corner cases.
And this is where it's all about *who* gets to pay, either via money,
or via their labor, to support these various cases. Weren't people
just complaining, in other TUHS threads, of "bloat" in Linux?
Well, this is how you get bloat. It's just that if it's a feature
*you* want, then it's not bloat, but an essential feature, and if it's
not provided, you whine mightily. And when you have a large number of
enterprise customers paying $$$ to enterprise distribution vendors,
each with their own set of essential features, and where *binary*
backwards compatibility is considered an essential feature, then
that's how you get what others will called "bloat". I would call this
the "Tyrany of Gold", as in the reformulated Golden Rule, "The ones
with the Gold, makes the Rules".
GRUB doesn't care. But various system administration utilities that
want to manage to UEFI boot menu (as distinct from the GRUB boot
menu), they need to modify the files that are read by the UEFI
firmware. So it's convenient if it's mounted *somewhere*. Also, even
if it's not mounted, it's still a partition that has to be around, and
one reason to keep it mounted is to avoid a system administrator from
saying, "hmmm, what's this unused /dev/sda1 partition? I guess I can
use it as an extra swap partition!" And then the system won't boot,
and then they call the enterprise distro's help desk, and unnecessary
calls into the help desk costs $$$, and distro's tend to optimize for
unnecessary cost. (Plus lots of unhappy customers who are down, even
if it is there own d*mned fault, is not good for business.)
On 2/23/21 11:57 AM, Theodore Ts'o wrote:
Sure, I was trying to enumerate the reasons why initramfs, for some
combinations of hardware / configurations, might be necessary.
There are two reasons why you might want to have
an initramfs.
Rather than getting into a tit for tat debate, I'll agree that we have both
proposed reasons why you /might/ want to use an initramfs. The operative
words are "you" and "might". Each person probably wants slightly
different
things. It's far from one size fits all. /boot needs to
exist due to limitations to the firmware and/or boot
loader being used.
Not necessarily. E.g. one single partition containing /boot and / (root). Other than
that, there is no reason why /boot needs to be its own file
system, except that most installers will create one just because it's
simpler to use the same approach for all cases, even if it's not needed
for a particular use case.
As Steve Gibson is famous for saying; The tyranny of the default. P.S. Oh, and
if you are using UEFI, you might need to have yet another
file system which is a Microsoft FAT file system, typically mounted as
/boot/efi, to keep the UEFI firmware happy....
Yes, the file system needs to exist. But that's part of the firmware, not
the operating system. I also question if that FAT file system needs to be
mounted or not. -- I don't know how GRUB et al. deal with a non-mounted
UEFI file system. But even if it does need to be mounted, you can still
get away with two
partitions; / (root) and /boot/efi. I suspect UEFI does away with the LBA
issue you mentioned.
Yes, in another 5 or 10 years, we can probably completely deprecate
the MBR-based boot sequence. At which point there will be another
series of whiners on TUHS ala the complaint that distributions are
dropping support for i386....
But since most TUHS posters aren't paying $$$ to enterprise
distributions, most enterpise distro engineers are going to give
precisely zero f*cks. But hey, if you want to volunteer to provide
the hard work for supporting these configurations to the community
distribution, like Debian, those distros will be happy to accept the
volunteer help. :-)
- Ted
11:50 p.m.
New subject: /usr separation
On 2/24/21 7:14 AM, Theodore Ts'o wrote:
I wouldn't say that; I'd rather say that if
you have a huge combination
of configurations that you have to test, those configurations which
aren't regularly tested will tend to bitrot, or have odd failures
in various error cases. The more corners that you have, the more
corner cases.
Fair enough.
I would call this the "Tyrany of Gold", as
in the reformulated Golden
Rule, "The ones with the Gold, makes the Rules".
Being a fan of the golden rule, I would not make, much less use, that
derivation. I think it completely changes the meaning of the spirit
behind the golden rule.
I don't fault your logic. I just dislike where it ended up.
GRUB doesn't care. But various system
administration utilities that
want to manage to UEFI boot menu (as distinct from the GRUB boot menu),
they need to modify the files that are read by the UEFI firmware.
Valid distinction.
So it's convenient if it's mounted
*somewhere*. Also, even if it's not
mounted, it's still a partition that has to be around, and one reason
to keep it mounted is to avoid a system administrator from saying,
"hmmm, what's this unused /dev/sda1 partition? I guess I can use it
as an extra swap partition!"
I seem to recall hearing about a problem where a rogue rm could
accidentally wipe out part of the UEFI. Maybe it was the contents of
the /boot/efi partition. So, I'd suggest a happy medium of mounting it
Read-Only. That way it's known to be used /and/ it's protected from a
simple rogue rm. It can relatively easily be re-mounted as Read-Write
when necessary. As well as subsequently re-mounted back to Read-Only.
Yes, in another 5 or 10 years, we can probably
completely deprecate
the MBR-based boot sequence. At which point there will be another
series of whiners on TUHS ala the complaint that distributions are
dropping support for i386....
I feel like we've already abandoned i386 as in 80386 (or compatible)
architecture. I think we now require Pentium (586?) or better. At some
point, we'll completely remove 32-bit support from mainstream Linux
distributions, thus requiring something from the 21st century.
But since most TUHS posters aren't paying $$$ to
enterprise
distributions, most enterpise distro engineers are going to give
precisely zero f*cks. But hey, if you want to volunteer to provide
the hard work for supporting these configurations to the community
distribution, like Debian, those distros will be happy to accept the
volunteer help. :-)
~chuckle~
--
Grant. . . .
unix || die
25 Feb
25 Feb
12:37 a.m.
New subject: /usr separation
On Wed, Feb 24, 2021 at 10:50:03AM -0700, Grant Taylor via TUHS wrote:
Oh, sure. I agree completely that it's 180 degrees from the original
golden rule; it had intended to be a joke. Unfortunately, years of
living in a country whre the ones with the Gold really do make all of
the Rules has gotten me to the point where if I don't laugh at it, I
would have to cry....
I would call
this the "Tyrany of Gold", as in the reformulated Golden
Rule, "The ones with the Gold, makes the Rules".
Being a fan of the golden rule, I would not make, much less use, that
derivation. I think it completely changes the meaning of the spirit behind
the golden rule. I seem to recall hearing about a problem where a rogue
rm could accidentally
wipe out part of the UEFI. Maybe it was the contents of the /boot/efi
partition. So, I'd suggest a happy medium of mounting it Read-Only. That
way it's known to be used /and/ it's protected from a simple rogue rm. It
can relatively easily be re-mounted as Read-Write when necessary. As well
as subsequently re-mounted back to Read-Only.
So technically it doesn't wipe out UEFI; it just will destroy the
ability to boot the system. (e.g., this is where Grub lives, and if
you delete it, UEFI will no longer be able to launch Grub, and hence,
not boot Linux.) Fortunately, if you have a rescue CD / USB Thumb
drive, it's relatively easy to recover from this.
A rogue rm which deletes /bin (even if /bin is a symlink to /usr/bin,
all of the shell scripts and /etc/passwd entries probably still refer
to /bin/sh) is going to make the system similarly unbootable.
As far as making a system more robust against rogue rm's, I really
like scheme used by ChromeOS, where the entire file system is not only
read-only, but protected by a cryptographic Merkle Tree such that if
malware attempts to modify it, the system will crash. This is
combined with firmware which will only load a kernel with a valid
digital signature, and the user data is stored on an encrypted file
system mounted on /mnt/stateful_partition and it is the only file
system mounted read/write on a ChromeOS system. It violates a lot of
expectations about where files should live on a "normal" Unix or Linux
system, but it's defnitely way more safe and secure.
I feel like we've already abandoned i386 as in
80386 (or compatible)
architecture. I think we now require Pentium (586?) or better. At some
point, we'll completely remove 32-bit support from mainstream Linux
distributions, thus requiring something from the 21st century.
For now, as far as I know, Debian still supports a 486 (or i386 with
an i387 co-processor, which was my first Linux system). But yes, it
is very likely, absent people showing up to volunteer to support
32-bit userspace at Debian (e.g., ongoing security updates, support
for the i386 build farm, reporting and triaging build failures of
packages on i386, etc.), that the i386 arch will probably get dropped
after Debian Bullseye release (which will probably happpen sometime in
mid-2021 if I had to guess).
I'm not sure there are any 486's around any more, and it's likely most
uses of systems with i386 binaries are on 64-bit processors running in
32-bit mode, so 486 vs 586 is probably not all that important in the
grand scheme of things.
- Ted
12:48 a.m.
New subject: /usr separation
On 2/24/21 11:37 AM, Theodore Ts'o wrote:
Oh, sure. I agree completely that it's 180
degrees from the original
golden rule; it had intended to be a joke. Unfortunately, years of
living in a country whre the ones with the Gold really do make all
of the Rules has gotten me to the point where if I don't laugh at it,
I would have to cry....
When colleagues would say "you would think" or "I've been
thinking" or
the likes, with "We don't do that! The logo does it for us!" when
dealing with IBM shenanigans. Again, laugh, lest I cry.
So technically it doesn't wipe out UEFI; it just
will destroy the
ability to boot the system. (e.g., this is where Grub lives, and if
you delete it, UEFI will no longer be able to launch Grub, and hence,
not boot Linux.)
ACK
Either way, it causes someone to have a Bad Day™.
Fortunately, if you have a rescue CD / USB Thumb
drive, it's relatively
easy to recover from this.
And now we're back towards the start of this (sub)thread of a system
being able to boot strap itself or not.
A rogue rm which deletes /bin (even if /bin is a
symlink to /usr/bin,
all of the shell scripts and /etc/passwd entries probably still refer
to /bin/sh) is going to make the system similarly unbootable.
Agreed.
Though I think there is a difference in containing the damage to the OS
vs going beyond it and damaging the firmware configuration.
As far as making a system more robust against rogue
rm's, I really
like scheme used by ChromeOS, where the entire file system is
not only read-only, but protected by a cryptographic Merkle Tree
such that if malware attempts to modify it, the system will crash.
This is combined with firmware which will only load a kernel with a
valid digital signature, and the user data is stored on an encrypted
file system mounted on /mnt/stateful_partition and it is the only
file system mounted read/write on a ChromeOS system. It violates
a lot of expectations about where files should live on a "normal"
Unix or Linux system, but it's defnitely way more safe and secure.
I've not looked at Chrome OS or how it does things because of my dislike
for actually /using/ it. However, it sounds like it's worth popping the
hood and looking at things.
For now, as far as I know, Debian still supports a 486
(or i386 with
an i387 co-processor, which was my first Linux system). But yes,
it is very likely, absent people showing up to volunteer to support
32-bit userspace at Debian (e.g., ongoing security updates, support
for the i386 build farm, reporting and triaging build failures of
packages on i386, etc.), that the i386 arch will probably get dropped
after Debian Bullseye release (which will probably happpen sometime
in mid-2021 if I had to guess).
I don't know how quickly 32-bit will disappear. I think the embedded
market and other non-i386 32-bit platforms will likely keep 32-bit code
around for a while yet. At least user space application code. Maybe
the i386 kernel code will languish ~> bit rot. Or worse, get in the way
of maintaining 64-bit code and thereby be ejected.
I'm not sure there are any 486's around any
more, and it's likely most
uses of systems with i386 binaries are on 64-bit processors running
in 32-bit mode, so 486 vs 586 is probably not all that important in
the grand scheme of things.
¯\_(ツ)_/¯
--
Grant. . . .
unix || die
9:38 a.m.
New subject: /usr separation
On Wed, Feb 24, 2021 at 11:48:28AM -0700, Grant Taylor via TUHS wrote:
I've not looked at Chrome OS or how it does
things because of my dislike for
actually /using/ it. However, it sounds like it's worth popping the hood
and looking at things.
If you don't like using Chromebooks, the same scheme is used for
Google's Container Optimized OS (intended for use in cloud VM's
running docker images):
Container-Optimized OS is an operating system image for your
Compute Engine VMs that is optimized for running Docker
containers. With Container-Optimized OS, you can bring up your
Docker containers on Google Cloud Platform quickly, efficiently,
and securely. Container-Optimized OS is maintained by Google and
is based on the open source Chromium OS project.
https://cloud.google.com/container-optimized-os/docs
Cheers,
- Ted
2:25 a.m.
New subject: /usr separation
On Wed, 24 Feb 2021, Theodore Ts'o wrote:
On Wed, Feb 24, 2021 at 10:50:03AM -0700, Grant Taylor
via TUHS wrote:
I first heard this form used in the movie "Aladdin" (the 1992 Disney one,
with Robin Williams).
Being a fan of the golden rule, I would not make,
much less use, that
derivation. I think it completely changes the meaning of the spirit behind
the golden rule.
Oh, sure. I agree completely that it's 180 degrees from the original
golden rule; it had intended to be a joke. Unfortunately, years of
living in a country whre the ones with the Gold really do make all of
the Rules has gotten me to the point where if I don't laugh at it, I
would have to cry.... > I seem to recall hearing about a problem where a
rogue rm could accidentally
> wipe out part of the UEFI. Maybe it was the contents of the /boot/efi
> partition. So, I'd suggest a happy medium of mounting it Read-Only. That
> way it's known to be used /and/ it's protected from a simple rogue rm. It
> can relatively easily be re-mounted as Read-Write when necessary. As well
> as subsequently re-mounted back to Read-Only.
<snip>
As far as making a system more robust against rogue
rm's, I really
like scheme used by ChromeOS, where the entire file system is not only
read-only, but protected by a cryptographic Merkle Tree such that if
malware attempts to modify it, the system will crash. This is
combined with firmware which will only load a kernel with a valid
digital signature, and the user data is stored on an encrypted file
system mounted on /mnt/stateful_partition and it is the only file
system mounted read/write on a ChromeOS system. It violates a lot of
expectations about where files should live on a "normal" Unix or Linux
system, but it's defnitely way more safe and secure.
It may not be as much of a protection, but I replaced the system rm on my
Debian with one based on 4.4BSD (since I already had the code lying
around) to which I added a bit of protection against attempts to "rm -rf
/" after a worm got in and ran an obfuscated version of that...thankfully
it didn't run as the superuser.
I do get occasional "invalid switch" errors from it while using apt, so it
probably uses a gnuism (since afaict, the code I used was strictly
conformant to Posix). Otherwise, it hasn't caused any issues.
-uso.
4:08 a.m.
New subject: /usr separation
Steve Nickolas wrote in
<alpine.DEB.2.21.2102241520550.15020(a)sd-119843.dedibox.fr>:
|On Wed, 24 Feb 2021, Theodore Ts'o wrote:
...
|> As far as making a system more robust against rogue rm's, I really
|> like scheme used by ChromeOS, where the entire file system is not only
|> read-only, but protected by a cryptographic Merkle Tree such that if
|> malware attempts to modify it, the system will crash. This is
|> combined with firmware which will only load a kernel with a valid
|> digital signature, and the user data is stored on an encrypted file
|> system mounted on /mnt/stateful_partition and it is the only file
|> system mounted read/write on a ChromeOS system. It violates a lot of
|> expectations about where files should live on a "normal" Unix or Linux
|> system, but it's defnitely way more safe and secure.
|
|It may not be as much of a protection, but I replaced the system rm on my
|Debian with one based on 4.4BSD (since I already had the code lying
|around) to which I added a bit of protection against attempts to "rm -rf
|/" after a worm got in and ran an obfuscated version of that...thankfully
|it didn't run as the superuser.
|
|I do get occasional "invalid switch" errors from it while using apt, so it
|probably uses a gnuism (since afaict, the code I used was strictly
|conformant to Posix). Otherwise, it hasn't caused any issues.
Just this week i finished my move from BSD compatibility to plain
Linux-only (which you seem to run) for my "web" and my "web with
credentials" user accounts; the accounts are gone now, instead
i as "i" execute according overlays. pstree for example now say
[sudo..]
box-browse.sh---box-browse.sh---unshare---
su---.box-browse-gui-+-firefox-bin-+-Web Content
when i browse totally boxed and unprivileged. (Still not CPU and
memory restricted, but other than that.)
The / root is the low level of an overlayfs, the upper level is
a tmpfs that may not use more than 5 percent of RAM. It has its
own minimal /dev (with audio even) and has read/write access to
one shared folder. Ditto with credentials, but that runs in the
global network namespace, whereas the unprivileged even runs
isolated from that.
It is a bit messy if you want to be portable to Linux
distributions which use busybox unshare etc., because there you
need to use chroot(1) yourself, and therefore mount /proc also
yourself thereafter (ie unshare(1)s --mount-proc is effectively
useless). Also it would be nice to be able to execute a few
commands before you switch aka map user and group IDs in the
containment (if you do so). But for open source software the
answer there usually is "shut up and hack", thus.
Of course with this approach the containers need to live in the
same X11 session, therefore the one mounts only /tmp/.X11-unix (it
is tremendous that Linux can "mount" a normal directory now!), the
other just the plain /tmp.
So an rm -f could destroy the shared folder (it lives on
a filesystem with snapshot support though). For the credential
account it could even wipe /tmp/ and the --bind mounted .mozilla
encfs that is in the containment there (but ditto, plus specific
backups). I have not looked at overlayfs code, but i think the
whiteouts of the upper layer will be saved in the "work" layer, so
an rm -rf / could possibly even not finish because 5 percent RAM
could exceed earlier? Happy to (un)share ~150 lines sh(1) script.
Yes Mr. Cole, thanks for this working on overlay (less so union)
filesystems, it is tremendous!
(P.S.: that .box-browse-gui.sh is a condome to prevent that
firefox-bin as compiled by Mozilla locks me out of the system.
Have seen this twice already when browsing serious German and
Austrian magazines, needing a reboot. So i now have my browser
session been protected by a shell guard, and my window manager
menu has a "TOUCH" entry. If the timestamp of the touch file
becomes older than 300 seconds i hear the first gong of Big Ben
and need to touch .. after the fifth a "kill -TERM -1" happens.)
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
24 Feb
24 Feb
9:12 a.m.
New subject: /usr separation (was: Abstractions)
On 2/23/21, Greg A. Woods <woods(a)robohack.ca> wrote:
For one there's a huge amount of deeply embedded lore, human (finger and
brain) memory, actual code, documentation, and widespread practices that
use this separation and rely on it, effectively making it a requirement.
That is only a justification for keeping the /usr hierarchy around
(and using symlinks/binding to make stuff appear in both places), not
for arbitrarily separating programs and libraries between the two.
However the reality of maintaining a separate minimal toolset for system
bring-up is that it cannot be reliably done without constant and
pervasive testing; and the very best (and perhaps only) way to achieve
this, especially in any smaller open-source project, is for everyone to
use it that way as much of the time as possible. I say this from
decades long experience of slowly moving systems to having just one
partition for both root and /usr and then on occasion testing with
separate root and /usr, and every time I do this testing I find
dependencies have crept in on something in /usr for basic booting. (and
that's even when I base my system on a platform that still tries hard to
maintain this separation of root and /usr!)
With a system-wide package manger a set of basic packages can be
maintained without having an arbitrary separation into root and usr.
The reference distribution of UX/RT will have several nested sets of
packages rather than a separation of binaries between root and usr.
The smallest will be what is included in the supervisor image (the
equivalent of a kernel image and initramfs combined into one), which
will be what is required to mount the system volume. Above that will
be the minimal system, which will be the set of packages required to
boot to a multi-user login. All of this will be in the base system
repository, along with a few other optional groups of packages
(including a full desktop
environment). Most optional third-party application packages will be
in a separate repository (like ports or pkgsrc under BSD, but using
the same package manager as the base system and available by default
without any special configuration).
On 2/23/21, Theodore Ts'o <tytso(a)mit.edu> wrote:
/boot needs to exist due to limitations to the firmware and/or boot
loader being used. If the boot loader is using the legacy PC Bios
interfaces to read the kernel and initial ramdisk/file system, then
those files need to be in a low-numbered LBA disk space, due to legacy
BIOS/firmware limitations. It could also be a concern if you are
using some exotic file system (say, ZFS), and the bootloader doesn't
support that file system due to copyright licensing incompatibilities,
or the boot loader just not supporting that bleeding-edge file system.
In that case, you might have to keep /boot as an ext4 file system.
The BIOS addressing limitations only happen with CHS-only BIOSes,
which haven't really been a thing since the mid-to-late 90s. The only
reason to have a separate /boot partition for anything newer than that
is because of bootloader limitations.
On 2/23/21, Grant Taylor via TUHS <tuhs(a)minnie.tuhs.org> wrote:
I have a different conundrum regarding */bin. Why do I need nine
different (s)bin directories in my path? I -- possibly naively --
believe that we have the technology to have all commands in /one/
directory, namely /bin.
Quickly after that thought, I realize that I want different things in my
path than other people do. So I end up with custom /bin directories.
Which usually ends up with sym-links that reference variables or custom
mounts (possibly via auto-mount applying some logic).
UX/RT will solve the issue of different sets of programs in the path
in different user or application contexts with per-process and
per-user namespaces (since fine-grained security will be deeply
integrated into the system and neither on-disk device files nor setuid
binaries will exist, there shouldn't be any security concerns with
letting regular users bind and mount stuff for themselves). $PATH will
just be set to "/bin" in the vast majority of cases.
22 Feb
22 Feb
6:13 a.m.
On Mon, Feb 22, 2021 at 09:40:57AM +1100, Dave Horsfall wrote:
That's how we ran our RK-05 11/40s since Ed 5...
Good fun writing a DJ-11
driver from the DH-11 source; even more fun when I wrote a UT-200 driver
from the manual alone (I'm sure that "ei.c" is Out There Somewhere),
junking
IanJ's driver.
https://minnie.tuhs.org/cgi-bin/utree.pl?file=AUSAM/sys/dmr/ei.c
Cheers, Warren
27 Feb
27 Feb
8:47 a.m.
On Mon, 22 Feb 2021, Warren Toomey wrote:
Nah; that's the IanJ rubbish. Mine was written from scratch (and actually
worked) but I don't think that it ever left UNSW.
-- Dave
That's
how we ran our RK-05 11/40s since Ed 5... Good fun writing a
DJ-11 driver from the DH-11 source; even more fun when I wrote a UT-200
driver from the manual alone (I'm sure that "ei.c" is Out There
Somewhere), junking IanJ's driver.
https://minnie.tuhs.org/cgi-bin/utree.pl?file=AUSAM/sys/dmr/ei.c 
23 Feb
23 Feb
6:25 a.m.
I've just checked Slackware 14.* and it's still got a few binaries in
/bin, unlike the RedHat* group which has indeed sent them all to
/usr/bin. I don't know about the Debian* group, or if the Mandrake*
group have gone with the RedHat* or not. Let alone all the other
distros.
Wesley Parish
On 2/22/21, Dave Horsfall <dave(a)horsfall.org> wrote:
On Sat, 20 Feb 2021, M Douglas McIlroy wrote:
I once worked for a place who reckoned that /bin and /lib etc ought to be
in an EEPROM; I reckon that he was right (Penguin/OS dumps everything
under /usr/bin, for example).
-
separation of code and data using read-only and read/write file
systems
I'll bite. How do you install code in a read-only file system? And where
does a.out go? My guess is that /bin is in a file system of its
own. Executables from
/letc and /lib are probably there too. On the other hand, I guess users'
personal code is still read/write.
That's how we ran our RK-05 11/40s since Ed 5... Good fun writing a DJ-11
driver from the DH-11 source; even more fun when I wrote a UT-200 driver
from the manual alone (I'm sure that "ei.c" is Out There Somewhere),
junking IanJ's driver.
The war stories that I could tell...
I agree that such an arrangement is prudent. I
don't see a way, though,
to update bin without disrupting most running programs.
Change is inevitable; the trick is to minimise the disruption.
-- Dave, who carried RK-05s all over the UNSW campus
6:38 a.m.
On Tue, 23 Feb 2021, Wesley Parish wrote:
I've just checked Slackware 14.* and it's
still got a few binaries in
/bin, unlike the RedHat* group which has indeed sent them all to
/usr/bin. I don't know about the Debian* group, or if the Mandrake*
group have gone with the RedHat* or not. Let alone all the other
distros.
Debian links /bin to /usr/bin.
-uso.
8:50 a.m.
On Mon, Feb 22, 2021 at 07:38:21PM -0500, Steve Nickolas wrote:
On Tue, 23 Feb 2021, Wesley Parish wrote:
New installs of Debian will use a /usr merged configuration. However,
for pre-existing installations, we are not yet forcing, or even
strongly recommending, system administrators to install the usrmerge
package which will transition an legacy directory hierarchy to be /usr
merged. So at the moment, Debian packages need to support both merged
and non-merged configurations, which is not ideal from a pacakge
maintainer's POV.
- Ted
I've just checked Slackware 14.* and
it's still got a few binaries in
/bin, unlike the RedHat* group which has indeed sent them all to
/usr/bin. I don't know about the Debian* group, or if the Mandrake*
group have gone with the RedHat* or not. Let alone all the other
distros.
Debian links /bin to /usr/bin. 
9:19 a.m.
On Mon, Feb 22, 2021, 7:50 PM Theodore Ts'o <tytso(a)mit.edu> wrote:
On Mon, Feb 22, 2021 at 07:38:21PM -0500, Steve
Nickolas wrote:
I anticipate needing a /usr/bin/bash soon on my FreeBSD system for the same
reason I have a /bin/bash pointing at /usr/local/bin/bash. Progress :)
Warner
On Tue, 23 Feb 2021, Wesley Parish wrote:
New installs of Debian will use a /usr merged configuration. However,
for pre-existing installations, we are not yet forcing, or even
strongly recommending, system administrators to install the usrmerge
package which will transition an legacy directory hierarchy to be /usr
merged. So at the moment, Debian packages need to support both merged
and non-merged configurations, which is not ideal
I've just checked Slackware 14.* and
it's still got a few binaries in
/bin, unlike the RedHat* group which has indeed sent them all to
/usr/bin. I don't know about the Debian* group, or if the Mandrake*
group have gone with the RedHat* or not. Let alone all the other
distros.
Debian links /bin to /usr/bin.
7:47 a.m.
On Tue, Feb 23, 2021 at 01:25:51PM +1300, Wesley Parish wrote:
I've just checked Slackware 14.* and it's
still got a few binaries in
/bin, unlike the RedHat* group which has indeed sent them all to
/usr/bin. I don't know about the Debian* group, or if the Mandrake*
group have gone with the RedHat* or not. Let alone all the other
distros.
More information about the /usr migration, can be found at:
* https://wiki.debian.org/UsrMerge
* https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
One of the interesting points made in the above is that merging /bin
and /usr/bin, et. al., was first done by Solaris 11 (ten years ago)
and so one of the arguments for Linux distributions for proceeding
with the /usr merge was to improve cross compatibility with legacy
commercial Unix systems.
So obviously, like so many other things, it's all Oracle's fault. :-)
- Ted
22 Feb
22 Feb
4:54 a.m.
On Sat, Feb 20, 2021 at 6:10 PM M Douglas McIlroy <
m.douglas.mcilroy(a)dartmouth.edu> wrote:
The best way I have seen this done is with overlay and union file system
support. The 'writeable' versions are the file in /bin are overlayed as
needed. To do this properly you need the stackable file system stuff we
worked on at LCC and Sun. If you can interpose at the inode level it's
very cool and flexible (Sun played with - but makes the Sun symlink
nightmare seem like an easy night at the movies), at the filesystem switch
layer (Locus and UCLA - scheme that was in BSD at one point - easier to
manage/admin).
ᐧ
- separation
of code and data using read-only and read/write file
systems
I'll bite. How do you install code in a read-only file system? And
where does a.out go?
1393
days inactive
1400
days old
25 comments
14 participants
participants (14)
-
Andrew Warkentin -
Clem Cole -
Dave Horsfall -
Grant Taylor -
Greg A. Woods -
M Douglas McIlroy -
Otto Moerbeek -
Rich Morin -
Steffen Nurpmeso -
Steve Nickolas -
Theodore Ts'o -
Warner Losh -
Warren Toomey -
Wesley Parish