30 Jan
2021
30 Jan
'21
8:01 p.m.
For sure, I've seen at least two interesting changes:
- market forces have pushed fast iteration and fast prototyping into the
mainstream in the form of Silicon valley "fail fast" culture and the
"agile" culture. This, over the disastrous "waterfall" style, has led
to a
momentous improvement in overall productivity improvements.
- As coders get pulled away from the machine and performance is less and
less in coders hands, engineers aren't sucked into (premature) optimization
as much.
Tyler
On Sat, Jan 30, 2021 at 6:10 AM M Douglas McIlroy <
m.douglas.mcilroy(a)dartmouth.edu> wrote:
Have you spotted an evolutionary trend toward better,
more productive
programmers? Or has programmer productivity risen across the board due to
better tools? Arguably what's happened is that principle has been
self-obsoleting, for we have cut back on the demand for unskilled (i.e.
less capable) programmers. A broad moral principle may be in play:
programmers should work to put themselves out of business, i.e. it is wrong
to be doing the same kind of work (or working in the same way) tomorrowas
yesterday.
Doug
On Tue, Jan 26, 2021 at 5:23 AM Tyler Adams <coppero1237(a)gmail.com> wrote:
> Looking at the 1978 list, the last one really stands out:
>
> "Use tools in preference to unskilled help to lighten a programming task"
> -- The concept of unskilled help for a programming task...doesn't really
> exist in 2020. The only special case is doing unskilled labor yourself.
> What unskilled tasks did people used to do back in the day?
>
> Tyler
>
>
> On Tue, Jan 26, 2021 at 4:07 AM M Douglas McIlroy <
> m.douglas.mcilroy(a)dartmouth.edu> wrote:
>
>> It might be interesting to compare your final list with the two lists in
>> the 1978 special issue of the BSTJ--one in the Foreword, the other in the
>> revised version of the Ritchi/Thompson article from the CACM. How have
>> perceptions or values changed over time?
>>
>> Doug
>>
>>
>> On Mon, Jan 25, 2021 at 7:32 AM Steve Nickolas <usotsuki(a)buric.co>
>> wrote:
>>
>>> On Mon, 25 Jan 2021, Tyler Adams wrote:
>>>
>>> > I'm writing about my 5 favorite unix design principles on my blog
this
>>> > week, and it got me wondering what others' favorite unix design
>>> principles
>>> > are? For reference, mine are:
>>> >
>>> > - Rule of Separation (from TAOUP <
>>> http://catb.org/~esr/writings/taoup/html/>
>>> > )
>>> > - Let the Machine Do the Dirty Work (from Elements of Programming
>>> Style)
>>> > - Rule of Silence (from TAOUP <
>>> http://catb.org/~esr/writings/taoup/html/>)
>>> > - Data Dominates (Rob Pike #5)
>>> > - The SPOT (Single Point of Truth) Rule (from TAOUP
>>> > <http://catb.org/~esr/writings/taoup/html/>)
>>> >
>>> > Tyler
>>> >
>>>
>>> 1. Pipes
>>> 2. Text as the preferred format for input and output
>>> 3. 'Most everything as a file
>>> 4. The idea of simple tools that are optimized for a single task
>>> 5. A powerful scripting language built into the system that, combined
>>> with
>>> 1-4, makes writing new tools heaps easier.
>>>
>>> -uso.
>>>
>>
30 Jan
30 Jan
8:50 p.m.
New subject: Favorite unix design principles?
Tyler Adams writes:
For sure, I've seen at least two interesting changes:
- market forces have pushed fast iteration and fast prototyping into the
mainstream in the form of Silicon valley "fail fast" culture and the
"agile" culture. This, over the disastrous "waterfall" style, has led
to a
momentous improvement in overall productivity improvements.
- As coders get pulled away from the machine and performance is less and
less in coders hands, engineers aren't sucked into (premature) optimization
as much.
It's interesting in more than one way.
The "fail fast" culture seems to result in a lot more failure than I find
acceptable.
As performance is less in coders hands, performance is getting worse. I
haven't seen less premature optimization, I've just seen more premature
optimization that didn't optimize anything.
My take is that the above changes have resulted in less reliable products
with poor performance being delivered more quickly. I'm just kind of weird
in that I'd prefer better products delivered more slowly. Especially since
much of what counts as a product these days is just churn to keep people
buying, not to provide things that are actually useful.
9:06 p.m.
New subject: Favorite unix design principles?
Really? Except for one particularly incompetent team, I cannot recall
working with nor reviewing code that sacrificed clarity for performance.
Tyler
On Sat, Jan 30, 2021 at 9:51 PM Jon Steinhart <jon(a)fourwinds.com> wrote:
Tyler Adams writes:
For sure, I've seen at least two interesting changes:
- market forces have pushed fast iteration and fast prototyping into the
mainstream in the form of Silicon valley "fail fast" culture and the
"agile" culture. This, over the disastrous "waterfall" style, has led
to
a
momentous improvement in overall productivity
improvements.
- As coders get pulled away from the machine and performance is less and
less in coders hands, engineers aren't sucked into (premature)
optimization
as much.
It's interesting in more than one way.
The "fail fast" culture seems to result in a lot more failure than I find
acceptable.
As performance is less in coders hands, performance is getting worse. I
haven't seen less premature optimization, I've just seen more premature
optimization that didn't optimize anything.
My take is that the above changes have resulted in less reliable products
with poor performance being delivered more quickly. I'm just kind of weird
in that I'd prefer better products delivered more slowly. Especially since
much of what counts as a product these days is just churn to keep people
buying, not to provide things that are actually useful.
10:28 p.m.
New subject: Favorite unix design principles?
Tyler - I'm with Jon on this. I'll pick on Apple here. It used to be a
huge difference between MSFT SW and MacOS was that the systems folks at
Apple really tested the system and the result was that Mac OS with way
really stable. My system never panic'ed except when I ran Windows under
parallels. After 3-4 years ago, that stopped being true. Crashes occur,
just like Windows BSD. It's not unusual for my Mac to panic just letting
it run overnight - which is just backups and the like. Yes, I have a
multiple monitors, a zillion windows open etc..
I come downstairs and the screen is blank (it should be, I have it turn off
after no activity), but I move the mouse or try to type something --
nothing wakes the system up again. I've chased it to Mac OS running out of
memory and not gracefully handling the lowe memory situation. Sad, I have
16G of RAM a 1T SSD and many TB of memory on Thunderbolt 3 connectivity.
Look I grew up with a 256K byte RAM Unix V6 system on an 11/34, 3 RK05s and
an RK07 for storage. We swapped. Yeah, I never ran a window manager, but
he had a number of 9600K terminals on DH11's and we were happy. You could
see it swapping like mad, but that system never crashed. It just ran and
ran and ran.
IMO, this is what I think Jon is referring. Those systems were stable
because we tested them and found and fixed the issue. These days, Apple
no longer cares about Mac OS because iOS is where they now put their
effort, although I'm not super impressed there either, but I also don't
push it like I do Mac OS. Sad really. If I could get the day-2-day
applications that I need to work on FreeBSD, I suspect I would be there in
a heartbeat.
Clem
ᐧ
On Sat, Jan 30, 2021 at 3:07 PM Tyler Adams <coppero1237(a)gmail.com> wrote:
Really? Except for one particularly incompetent team,
I cannot recall
working with nor reviewing code that sacrificed clarity for performance.
Tyler
On Sat, Jan 30, 2021 at 9:51 PM Jon Steinhart <jon(a)fourwinds.com> wrote:
Tyler Adams writes:
For sure, I've seen at least two interesting changes:
- market forces have pushed fast iteration and fast prototyping into the
mainstream in the form of Silicon valley "fail fast" culture and the
"agile" culture. This, over the disastrous "waterfall" style, has led
to a
momentous improvement in overall productivity
improvements.
- As coders get pulled away from the machine and performance is less and
less in coders hands, engineers aren't sucked into (premature)
optimization
as much.
It's interesting in more than one way.
The "fail fast" culture seems to result in a lot more failure than I find
acceptable.
As performance is less in coders hands, performance is getting worse. I
haven't seen less premature optimization, I've just seen more premature
optimization that didn't optimize anything.
My take is that the above changes have resulted in less reliable products
with poor performance being delivered more quickly. I'm just kind of
weird
in that I'd prefer better products delivered more slowly. Especially
since
much of what counts as a product these days is just churn to keep people
buying, not to provide things that are actually useful.
10:42 p.m.
New subject: Favorite unix design principles?
On Sat, 30 Jan 2021, Clem Cole wrote:
Look I grew up with a 256K byte RAM Unix V6 system on
an 11/34, 3 RK05s
and an RK07 for storage. We swapped. Yeah, I never ran a window
manager, but he had a number of 9600K terminals on DH11's and we were
happy. You could see it swapping like mad, but that system never
crashed. It just ran and ran and ran.
You had *three* RK05s, an RK07, *and* a DH11? We had just two, and a
crappy DJ-11 on our /40s. Pretty much stable; any bugs were fixed on the
spot because we had the source code after all.
(They were configured that way to run RSX-11D, then that issue of the BSTJ
appeared; the rest is history.)
-- Dave
10:45 p.m.
New subject: Favorite unix design principles?
Fair points about macOS, I hated using my mac in 2018 (thankfully I've
lived in linux bliss since), but I wouldn't say macOS is representative of
the software industry as a whole. Especially as you said iOS is Apple's
real cash cow and apple's been focusing on better *mac hardware *and from
what I hear that M1 delivers*.*
The modern software industry is mostly over the browser (or an app), and
honestly almost every site I use these days is pretty fast and stable.
Unless it has too many ads.
Tyler
On Sat, Jan 30, 2021 at 11:28 PM Clem Cole <clemc(a)ccc.com> wrote:
Tyler - I'm with Jon on this. I'll pick on
Apple here. It used to be a
huge difference between MSFT SW and MacOS was that the systems folks at
Apple really tested the system and the result was that Mac OS with way
really stable. My system never panic'ed except when I ran Windows under
parallels. After 3-4 years ago, that stopped being true. Crashes occur,
just like Windows BSD. It's not unusual for my Mac to panic just letting
it run overnight - which is just backups and the like. Yes, I have a
multiple monitors, a zillion windows open etc..
I come downstairs and the screen is blank (it should be, I have it turn
off after no activity), but I move the mouse or try to type something --
nothing wakes the system up again. I've chased it to Mac OS running out of
memory and not gracefully handling the lowe memory situation. Sad, I have
16G of RAM a 1T SSD and many TB of memory on Thunderbolt 3 connectivity.
Look I grew up with a 256K byte RAM Unix V6 system on an 11/34, 3 RK05s
and an RK07 for storage. We swapped. Yeah, I never ran a window manager,
but he had a number of 9600K terminals on DH11's and we were happy. You
could see it swapping like mad, but that system never crashed. It just ran
and ran and ran.
IMO, this is what I think Jon is referring. Those systems were stable
because we tested them and found and fixed the issue. These days, Apple
no longer cares about Mac OS because iOS is where they now put their
effort, although I'm not super impressed there either, but I also don't
push it like I do Mac OS. Sad really. If I could get the day-2-day
applications that I need to work on FreeBSD, I suspect I would be there in
a heartbeat.
Clem
ᐧ
On Sat, Jan 30, 2021 at 3:07 PM Tyler Adams <coppero1237(a)gmail.com> wrote:
> Really? Except for one particularly incompetent team, I cannot recall
> working with nor reviewing code that sacrificed clarity for performance.
>
> Tyler
>
>
> On Sat, Jan 30, 2021 at 9:51 PM Jon Steinhart <jon(a)fourwinds.com> wrote:
>
>> Tyler Adams writes:
>> >
>> > For sure, I've seen at least two interesting changes:
>> > - market forces have pushed fast iteration and fast prototyping into
>> the
>> > mainstream in the form of Silicon valley "fail fast" culture and
the
>> > "agile" culture. This, over the disastrous "waterfall"
style, has led
>> to a
>> > momentous improvement in overall productivity improvements.
>> > - As coders get pulled away from the machine and performance is less
>> and
>> > less in coders hands, engineers aren't sucked into (premature)
>> optimization
>> > as much.
>>
>> It's interesting in more than one way.
>>
>> The "fail fast" culture seems to result in a lot more failure than I
find
>> acceptable.
>>
>> As performance is less in coders hands, performance is getting worse. I
>> haven't seen less premature optimization, I've just seen more
premature
>> optimization that didn't optimize anything.
>>
>> My take is that the above changes have resulted in less reliable products
>> with poor performance being delivered more quickly. I'm just kind of
>> weird
>> in that I'd prefer better products delivered more slowly. Especially
>> since
>> much of what counts as a product these days is just churn to keep people
>> buying, not to provide things that are actually useful.
>>
>
11:31 p.m.
New subject: Favorite unix design principles?
On Sat, Jan 30, 2021 at 11:45:38PM +0200, Tyler Adams wrote:
Fair points about macOS, I hated using my mac in 2018
(thankfully I've
lived in linux bliss since), but I wouldn't say macOS is representative of
the software industry as a whole. Especially as you said iOS is Apple's
real cash cow and apple's been focusing on better *mac hardware *and from
what I hear that M1 delivers*.*
M1 is a stunning pile of systems work. Warms my heart. Seems to be
faster, uses less power, and cheaper than Intel's best. Kudos to Apple
for that.
The modern software industry is mostly over the
browser (or an app), and
honestly almost every site I use these days is pretty fast and stable.
Unless it has too many ads.
Ad blocker to the rescue, I use ublock origin and it works great.
--lm
11:28 p.m.
New subject: Favorite unix design principles?
On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
If I could get the day-2-day
applications that I need to work on FreeBSD, I suspect I would be there in
a heartbeat.
I dunno about that. I tried out FreeBSD a couple of years ago when
Netflix was flirting with me. The installer hasn't seen any loving in
30 years it would seem. The disk setup tool sucks just as bad as it
did back in 1988.
I remember when Linux was this bad in the .90ish releases. A long time
ago. Now their install is painless, it's every bit as good as Windows
and maybe better. And it got that way fast, I remember doing an install
on some machine around 1998 or 1999, I didn't have a mouse plugged in,
no worries, you could just move around with the keyboard. X11 came up
as part of the install, the entire install was graphical and seamless.
FreeBSD is stuck in the 1990's in terms of user interface. They've done
some good stuff in the kernel but it's not an end user system, I'd argue
that the install is much worse than any of the Sun installers and I
started with BSD.
31 Jan
31 Jan
12:11 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole
wrote:
You could be right there, for some value of 1988 (FreeBSD came into
being in 1992). The tools work without being good. But how often do
you use them? I've been using FreeBSD since the beginning, and I
can't recall when I last used the disk partitioning tool, though I'm
sure that when I did I overrode a lot of (all?) the suggestions.
If I could get the day-2-day
applications that I need to work on FreeBSD, I suspect I would be there in
a heartbeat.
I dunno about that. I tried out FreeBSD a couple of years ago when
Netflix was flirting with me. The installer hasn't seen any loving in
30 years it would seem. The disk setup tool sucks just as bad as it
did back in 1988. I remember when Linux was this bad in the .90ish
releases. A long
time ago. Now their install is painless, it's every bit as good as
Windows and maybe better.
FWIW, I find Microsoft "Windows" installation terminally confusing
(that's what you were talking about, right?). And I've run into
serious problems with various Linux installations too. That doesn't
make the FreeBSD tools better, but maybe it relativizes it.
And it got that way fast, I remember doing an install
on some
machine around 1998 or 1999, I didn't have a mouse plugged in, no
worries, you could just move around with the keyboard. X11 came up
as part of the install, the entire install was graphical and
seamless.
The FreeBSD installer *does* install X if you select it.
FreeBSD is stuck in the 1990's in terms of user
interface.
You're still talking about the installer, aren't you? The normal user
interface is via the shell, which hasn't changed, and for a good
reason.
They've done some good stuff in the kernel but
it's not an end user
system,
There I have to agree with you. A little TLC would go a long way.
But I hope that you're not advocating the "change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have. One of the reasons I don't use Linux is because every time I
try, the interface has changed.
Greg
--
Sent from my desktop computer.
Finger grog(a)lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed. If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA
12:17 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sun, Jan 31, 2021 at 10:11:19AM +1100, Greg 'groggy' Lehey wrote:
Um, my mother could install any Linux system today and 10-20 years ago.
There is not the slightest chance that she could install FreeBSD.
Yup. If FreeBSD wants anyone to use it, fix that installer. 99.99%
of people would give up after seeing that, you'd never get them to
userland.
Try xubuntu, that's what I use. Pretty light weight UI but all the
parts are there and it doesn't change much.
I remember
when Linux was this bad in the .90ish releases. A long
time ago. Now their install is painless, it's every bit as good as
Windows and maybe better.
FWIW, I find Microsoft "Windows" installation terminally confusing
(that's what you were talking about, right?). And I've run into
serious problems with various Linux installations too. That doesn't
make the FreeBSD tools better, but maybe it relativizes it. The FreeBSD installer *does* install X if you select
it.
Linux installers start in X. No "select it" required.
FreeBSD is
stuck in the 1990's in terms of user interface.
You're still talking about the installer, aren't you? They've
done some good stuff in the kernel but it's not an end user
system,
There I have to agree with you. A little TLC would go a long way.
But I hope that you're not advocating the "change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have. One of the reasons I don't use Linux is because every time I
try, the interface has changed. 
12:22 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sat, Jan 30, 2021 at 4:19 PM Larry McVoy <lm(a)mcvoy.com> wrote:
Yea. Once upon a time, this was super dangerous. These days it's kinda
required.
Yup. If FreeBSD wants anyone to use it, fix that installer. 99.99%
of people would give up after seeing that, you'd never get them to
userland.
No argument there... Part of the problem is that, up until relatively
lately, the whole X experience sucked really badly on FreeBSD. Now that it
doesn't suck, it's time for a re-evaluation...
Try xubuntu, that's what I use. Pretty light weight UI but all the
parts are there and it doesn't change much.
But yet it's not stuck?
Warner
On Sun, Jan 31, 2021 at 10:11:19AM +1100, Greg
'groggy' Lehey wrote:
Um, my mother could install any Linux system today and 10-20 years ago.
There is not the slightest chance that she could install FreeBSD.
I find that hard to believe. The defaults just work on the vast majority of
systems, even if the interface is text-based and not a fancy GUI...
I
remember when Linux was this bad in the .90ish releases. A long
time ago. Now their install is painless, it's every bit as good as
Windows and maybe better.
FWIW, I find Microsoft "Windows" installation terminally confusing
(that's what you were talking about, right?). And I've run into
serious problems with various Linux installations too. That doesn't
make the FreeBSD tools better, but maybe it relativizes it. The FreeBSD
installer *does* install X if you select it.
Linux installers start in X. No "select it" required.
FreeBSD is stuck in the 1990's in terms of user
interface.
You're still talking about the installer, aren't you? They've done some good stuff in the kernel but
it's not an end user
system,
There I have to agree with you. A little TLC would go a long way.
But I hope that you're not advocating the "change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have. One of the reasons I don't use Linux is because every time I
try, the interface has changed.
12:31 a.m.
New subject: [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sat, Jan 30, 2021 at 04:22:49PM -0700, Warner Losh wrote:
Yup. If FreeBSD wants anyone to use it, fix that installer. 99.99%
of people would give up after seeing that, you'd never get them to
userland.
No argument there... Part of the problem is that, up until relatively
lately, the whole X experience sucked really badly on FreeBSD. Now that it
doesn't suck, it's time for a re-evaluation...
It's 20 years past the time for that re-evaluation. Seriously, Linux
distros have been installing in X for at least 20 years, I think maybe
more. If they can do it....
Try xubuntu, that's what I use. Pretty light weight UI but all the
parts are there and it doesn't change much.
But yet it's not stuck?
Nope, it's not remotely stuck. I get it, you like FreeBSD. That's fine
but be honest about where it is. How many of the developers with a commit
bit actually run FreeBSD on their desktop and laptops? As their daily
platform?
On Sat, Jan 30, 2021 at 4:19 PM Larry McVoy
<lm(a)mcvoy.com> wrote:
I speak from experience of trying to install FreeBSD on a netflix server
a couple of years back. It wasn't pleasant and it seems pretty identical
to the installer I used decades ago.
On Sun, Jan 31, 2021 at 10:11:19AM +1100, Greg
'groggy' Lehey wrote:
Um, my mother could install any Linux system today and 10-20 years ago.
There is not the slightest chance that she could install FreeBSD.
I find that hard to believe. The defaults just work on the vast majority of
systems, even if the interface is text-based and not a fancy GUI... I
remember when Linux was this bad in the .90ish releases. A long
time ago. Now their install is painless, it's every bit as good as
Windows and maybe better.
FWIW, I find Microsoft "Windows" installation terminally confusing
(that's what you were talking about, right?). And I've run into
serious problems with various Linux installations too. That doesn't
make the FreeBSD tools better, but maybe it relativizes it. FreeBSD is
stuck in the 1990's in terms of user interface.
You're still talking about the installer, aren't you? They've
done some good stuff in the kernel but it's not an end user
system,
There I have to agree with you. A little TLC would go a long way.
But I hope that you're not advocating the "change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have. One of the reasons I don't use Linux is because every time I
try, the interface has changed.
12:37 a.m.
New subject: [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
Larry McVoy writes:
It's 20 years past the time for that
re-evaluation. Seriously, Linux
distros have been installing in X for at least 20 years, I think maybe
more. If they can do it....
That doesn't mean that it's great. It's only easy to use once one learns
the quirks, like it appearing to hang with no feedback when you hit buttons,
and the way one has to click "done" when you're not actually done. The
number of options is so small that I think that it would be better to just
have the whole configuration on a single screen so that newbies wouldn't
have to wonder where things were done and whether or not they'd be able to
"go back" without trying it.
Nope, it's not remotely stuck. I get it, you
like FreeBSD. That's fine
but be honest about where it is. How many of the developers with a commit
bit actually run FreeBSD on their desktop and laptops? As their daily
platform?
Huh, I thought that most of the linux developers these days ran on Macs.
12:54 a.m.
New subject: [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sat, Jan 30, 2021 at 03:37:29PM -0800, Jon Steinhart wrote:
Larry McVoy writes:
It's on the same level as Windows and has been for a long time. In fact,
it has been better than Windows for at least 10 years and probably longer.
It takes me way longer to install Windows and then go get this driver,
and then go get that one, whoops, that's the wrong one, they wanted
32 bit emulation mode blah blah blah, go get that one.
Linux, other than closed source drivers and even that is pretty pleasant
these days, just works.
It's 20 years past the time for that
re-evaluation. Seriously, Linux
distros have been installing in X for at least 20 years, I think maybe
more. If they can do it....
That doesn't mean that it's great. Huh, I thought that most of the linux developers these
days ran on Macs.
I can imagine that, Macs have nice screens. But they run Linux on their
Macs.
Look, this is turning into a Linux vs BSD pissing contest. You do
realize that I started in the best BSD ever, SunOS, right? I was a BSD
guy long before I was a Linux guy, UW-Madison was all BSD on the vaxen
and I loved it. But I didn't have to install it. So it wasn't until
later I got to see the hell that is the FreeBSD installer.
You like FreeBSD, cool, it's fine. Nobody will ever know because nobody
is going to put up with a 1990's era installer. Nobody. It's 2020, shit
is just supposed to work, it's supposed do so in a graphical environment
let works with or without a mouse. Anyone who doesn't see that isn't
going to get any market share.
1:23 p.m.
New subject: [SPAM] Re: FreeBSD behind the times?
On 30/01/2021 23:54, Larry McVoy wrote:
You like FreeBSD, cool, it's fine. Nobody will
ever know because nobody
is going to put up with a 1990's era installer. Nobody. It's 2020, shit
is just supposed to work, it's supposed do so in a graphical environment
let works with or without a mouse. Anyone who doesn't see that isn't
going to get any market share.
It's "horses for courses". I run a mix of FreeBSD and Linux servers. As
they're headless, the last thing I want is X on there. Ideally, I'd
strip out the automount crap as well, to say nothing of the horror that
is systemd. But yeah, those people with desktops want a simple GUI
install and they want their USB stick to appear as a "folder". I run
Ubuntu on my desktop for vaguely those reasons (and Dropbox!).
I think the biggest threat to FreeBSD is docker/k8s and things like
libvirt. Unfortunately, most open source (and closed source) software
runs de facto on Linux (if it runs on Unix-esque systems), and the ports
to FreeBSD are slow in coming. Last time I looked, there was no native
Dropbox client.
As a server though, if you don't need to use it as a hypervisor or a
container system, it gets my vote. But again it's down to preference.
Linux has the breeze now, and is streaking ahead of the *BSDs. Sad, but
true.
- Der
--
Dermot Tynan
dtynan(a)kalopa.com
1 a.m.
New subject: [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
On Jan 30, 2021, at 3:37 PM, Jon Steinhart <jon(a)fourwinds.com> wrote:
Larry McVoy writes:
I vastly prefer FreeBSD over Linux but Larry has a point. Presenting *any*
configuration choice to a newbie can be confusing as they often lack the
knowledge to make an "informed choice". I do think your idea of a single
screen config is great, but with defaults chosen! The installer should
proceed after a time if the user doesn't change anything. Perhaps this can
be a useful GSOC project?
But we should bring this back to design principles! Ease of use should
certainly be one of them.
It's 20 years past the time for that
re-evaluation. Seriously, Linux
distros have been installing in X for at least 20 years, I think maybe
more. If they can do it....
That doesn't mean that it's great. It's only easy to use once one learns
the quirks, like it appearing to hang with no feedback when you hit buttons,
and the way one has to click "done" when you're not actually done. The
number of options is so small that I think that it would be better to just
have the whole configuration on a single screen so that newbies wouldn't
have to wonder where things were done and whether or not they'd be able to
"go back" without trying it. 
9 Feb
9 Feb
3:15 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 1/30/21 5:22 PM, Warner Losh wrote some pretty good stuff and I
needed a hook back into the thread...
Today, I found out that on Jan 12, distrowatch transitioned to FreeBSD.
I guess it's not so far behind the times to scare off the folks that
track the bleeding edge :)
3:16 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 2/8/21 8:15 PM, Will Senn wrote:
On 1/30/21 5:22 PM, Warner Losh wrote some pretty good
stuff and I
needed a hook back into the thread...
Today, I found out that on Jan 12, distrowatch transitioned to
FreeBSD. I guess it's not so far behind the times to scare off the
folks that track the bleeding edge :)
OK. So it was Jan 12, of 2020 - sue me :)
3:30 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Monday, 8 February 2021 at 20:16:50 -0600, Will Senn wrote:
On 2/8/21 8:15 PM, Will Senn wrote:
No, just proof that we're behind the times :-)
Greg
--
Sent from my desktop computer.
Finger grog(a)lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed. If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA
On 1/30/21 5:22 PM, Warner Losh wrote some pretty
good stuff and I
needed a hook back into the thread...
Today, I found out that on Jan 12, distrowatch transitioned to
FreeBSD. I guess it's not so far behind the times to scare off the
folks that track the bleeding edge :)
OK. So it was Jan 12, of 2020 - sue me :) 
31 Jan
31 Jan
1:39 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sun, 31 Jan 2021, Greg 'groggy' Lehey wrote:
But I hope that you're not advocating the
"change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have. One of the reasons I don't use Linux is because every time I
try, the interface has changed.
I'm still using the same GUI I've used for the last decade, although its
name changed.
Personally, I think a lot of Linux people who haven't come from a Unix
background don't understand Linux, because they don't understand Unix. And
because they don't understand Linux, they're trying to mutilate it into
being something other than what it was meant to be.
I thought of creating a system built on top of Linux that represented the
system as I visualized it. And I've come a fair way, but I don't yet have
the important parts (the ability to self-host, including a working kernel,
C compiler and libc) up and running, but there's pretty much your basic
stuff in /bin, /usr/bin etc., and there's a curses library, and there's ed
and vi (more or less)...so once those are available there'll be enough to
get the rest of it functional. Of course, as I visualized it will seem
remarkably antiquated to most people!
-uso.
2:47 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has
NEVER let me down and I’ve done my share of stupid user error. Now that Linux has ZFS, it
doesn’t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of my
Linux instances. We are soooo off topic, I think :). But, I’m always up for talking up
FBSD. I use it in my classes, too and the system is much more coherent for my systems
programming classes than linux.
Will
Sent from my iPhone
On Jan 30, 2021, at 5:11 PM, Greg 'groggy'
Lehey <grog(a)lemis.com> wrote:
On Saturday, 30 January 2021 at 14:28:54 -0800,
Larry McVoy wrote:
You could be right there, for some value of 1988 (FreeBSD came into
being in 1992). The tools work without being good. But how often do
you use them? I've been using FreeBSD since the beginning, and I
can't recall when I last used the disk partitioning tool, though I'm
sure that when I did I overrode a lot of (all?) the suggestions.
On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem
Cole wrote:
If I could get the day-2-day
applications that I need to work on FreeBSD, I suspect I would be there in
a heartbeat.
I dunno about that. I tried out FreeBSD a couple of years ago when
Netflix was flirting with me. The installer hasn't seen any loving in
30 years it would seem. The disk setup tool sucks just as bad as it
did back in 1988. I remember when Linux was this bad in the .90ish
releases. A long
time ago. Now their install is painless, it's every bit as good as
Windows and maybe better.
FWIW, I find Microsoft "Windows" installation terminally confusing
(that's what you were talking about, right?). And I've run into
serious problems with various Linux installations too. That doesn't
make the FreeBSD tools better, but maybe it relativizes it.
And it got that way fast, I remember doing an
install on some
machine around 1998 or 1999, I didn't have a mouse plugged in, no
worries, you could just move around with the keyboard. X11 came up
as part of the install, the entire install was graphical and
seamless.
The FreeBSD installer *does* install X if you select it.
FreeBSD is stuck in the 1990's in terms of
user interface.
You're still talking about the installer, aren't you? The normal user
interface is via the shell, which hasn't changed, and for a good
reason.
They've done some good stuff in the kernel
but it's not an end user
system,
There I have to agree with you. A little TLC would go a long way.
But I hope that you're not advocating the "change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have. One of the reasons I don't use Linux is because every time I
try, the interface has changed.
Greg
--
Sent from my desktop computer.
Finger grog(a)lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed. If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA
3:25 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
If you like ZFS you don't understand operating systems design. I do.
Jeff Bonwick was a stats student at Stanford when he took my OS class,
I convinced him to come to Sun. Bill Moore worked for me. That's the two
main ZFS guys and I thought I had taught them well but they let me down.
ZFS doesn't use the page cache, they said it was too hard because ZFS
is compressed. A typical file system just has block numbers, a compressed
one needs another int per block, it's the int that says these many bytes
are a block uncompressed. It's not that hard, it is 2 ints instead of 1.
In case I'm not being clear, the page cache is what everyone else uses
but ZFS has its own cache. So if you want to mmap() a ZFS file, ZFS
has to bcopy() the data into the page cache and then spend a shit ton
of code to make sure that the page cache data is in sync with the ZFS
cache data.
SunOS came from BSD but SunOS added mmap. Which had the same problem,
the BSD buffer cache was exactly the same as the ZFS cache, Sun spent
years of effort to get rid of the buffer cache, everything is in the
page cache. So ZFS was a HUGE step backwards in systems design. Might
be the best file system ever (it is not) but it was not a good player
in the OS world.
Those guys said that it was too hard to make a compressed file fit in
the page cache. BitKeeper has that code and proves that it can be done.
Be happy to walk anyone who cares through that code, I didn't write that,
Wayne Scott did, but it's some of the best written code I've ever seen.
Up there with Mojo's work on the SunOS VM system. (I'll bet that noone
takes me up on this offer, people love to argue but most don't want to
learn. Prove me wrong, please).
So good on you that you like ZFS and FreeBSD. I don't and I don't for
really good reasons.
Let's try it this way. Get back to me when you can show me 40 people
who have installed FreeBSD on their own, with no help. In the same
time, I can show you 40,000 people who have installed Linux on their
own, with no help. Probably 400,000.
Technology is great, ease of use is what gets you users. ZFS is
great but doesn't play nice with the OS.
That's my oh brother.
On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
Oh brother. I use FreeBSD all the time. I prefer it
for its stability and ZFS which has NEVER let me down and I???ve done my share of stupid
user error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages, but
uptime on my fbsd instance is 10x any of my Linux instances. We are soooo off topic, I
think :). But, I???m always up for talking up FBSD. I use it in my classes, too and the
system is much more coherent for my systems programming classes than linux.
Will
Sent from my iPhone
> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog(a)lemis.com>
wrote:
>
>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>> If I could get the day-2-day
>>> applications that I need to work on FreeBSD, I suspect I would be there in
>>> a heartbeat.
>>
>> I dunno about that. I tried out FreeBSD a couple of years ago when
>> Netflix was flirting with me. The installer hasn't seen any loving in
>> 30 years it would seem. The disk setup tool sucks just as bad as it
>> did back in 1988.
>
> You could be right there, for some value of 1988 (FreeBSD came into
> being in 1992). The tools work without being good. But how often do
> you use them? I've been using FreeBSD since the beginning, and I
> can't recall when I last used the disk partitioning tool, though I'm
> sure that when I did I overrode a lot of (all?) the suggestions.
>
>> I remember when Linux was this bad in the .90ish releases. A long
>> time ago. Now their install is painless, it's every bit as good as
>> Windows and maybe better.
>
> FWIW, I find Microsoft "Windows" installation terminally confusing
> (that's what you were talking about, right?). And I've run into
> serious problems with various Linux installations too. That doesn't
> make the FreeBSD tools better, but maybe it relativizes it.
>
>> And it got that way fast, I remember doing an install on some
>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>> worries, you could just move around with the keyboard. X11 came up
>> as part of the install, the entire install was graphical and
>> seamless.
>
> The FreeBSD installer *does* install X if you select it.
>
>> FreeBSD is stuck in the 1990's in terms of user interface.
>
> You're still talking about the installer, aren't you? The normal user
> interface is via the shell, which hasn't changed, and for a good
> reason.
>
>> They've done some good stuff in the kernel but it's not an end user
>> system,
>
> There I have to agree with you. A little TLC would go a long way.
> But I hope that you're not advocating the "change your GUI with your
> underwear" attitude that Microsoft, Apple and many Linux distros
> have. One of the reasons I don't use Linux is because every time I
> try, the interface has changed.
>
> Greg
> --
> Sent from my desktop computer.
> Finger grog(a)lemis.com for PGP public key.
> See complete headers for address and phone numbers.
> This message is digitally signed. If your Microsoft mail program
> reports problems, please read http://lemis.com/broken-MUA
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
3:52 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Ha. Zfs may not be the be all and end all, but like I said, it’s never failed me. Whereas
extX and btrfs, and, and, and have many times. Please don’t denigrate my knowledge, as so
far as I know, we’ve never met, and nothing I said warrants such. The installer reminds me
of Redhat’s old anaconda installer, I’ll grant you it’s dated. However, I typically
install a new linux distro every week and there are many, many installers that are far
more confusing - Open Suse and Fedora are two that come to mind, Debian as well. I would
hazard to guess your favorite Linux is based on a distro that lacks a decent installer
(Ubuntu and Mint are Debian based).
Will
Sent from my iPhone
On Jan 30, 2021, at 8:25 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
If you like ZFS you don't understand operating systems design. I do.
Jeff Bonwick was a stats student at Stanford when he took my OS class,
I convinced him to come to Sun. Bill Moore worked for me. That's the two
main ZFS guys and I thought I had taught them well but they let me down.
ZFS doesn't use the page cache, they said it was too hard because ZFS
is compressed. A typical file system just has block numbers, a compressed
one needs another int per block, it's the int that says these many bytes
are a block uncompressed. It's not that hard, it is 2 ints instead of 1.
In case I'm not being clear, the page cache is what everyone else uses
but ZFS has its own cache. So if you want to mmap() a ZFS file, ZFS
has to bcopy() the data into the page cache and then spend a shit ton
of code to make sure that the page cache data is in sync with the ZFS
cache data.
SunOS came from BSD but SunOS added mmap. Which had the same problem,
the BSD buffer cache was exactly the same as the ZFS cache, Sun spent
years of effort to get rid of the buffer cache, everything is in the
page cache. So ZFS was a HUGE step backwards in systems design. Might
be the best file system ever (it is not) but it was not a good player
in the OS world.
Those guys said that it was too hard to make a compressed file fit in
the page cache. BitKeeper has that code and proves that it can be done.
Be happy to walk anyone who cares through that code, I didn't write that,
Wayne Scott did, but it's some of the best written code I've ever seen.
Up there with Mojo's work on the SunOS VM system. (I'll bet that noone
takes me up on this offer, people love to argue but most don't want to
learn. Prove me wrong, please).
So good on you that you like ZFS and FreeBSD. I don't and I don't for
really good reasons.
Let's try it this way. Get back to me when you can show me 40 people
who have installed FreeBSD on their own, with no help. In the same
time, I can show you 40,000 people who have installed Linux on their
own, with no help. Probably 400,000.
Technology is great, ease of use is what gets you users. ZFS is
great but doesn't play nice with the OS.
That's my oh brother.
On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will
Senn wrote:
Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has
NEVER let me down and I???ve done my share of stupid user error. Now that Linux has ZFS,
it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of
my Linux instances. We are soooo off topic, I think :). But, I???m always up for talking
up FBSD. I use it in my classes, too and the system is much more coherent for my systems
programming classes than linux.
Will
Sent from my iPhone
>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog(a)lemis.com>
wrote:
>>
>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>> If I could get the day-2-day
>>> applications that I need to work on FreeBSD, I suspect I would be there in
>>> a heartbeat.
>>
>> I dunno about that. I tried out FreeBSD a couple of years ago when
>> Netflix was flirting with me. The installer hasn't seen any loving in
>> 30 years it would seem. The disk setup tool sucks just as bad as it
>> did back in 1988.
>
> You could be right there, for some value of 1988 (FreeBSD came into
> being in 1992). The tools work without being good. But how often do
> you use them? I've been using FreeBSD since the beginning, and I
> can't recall when I last used the disk partitioning tool, though I'm
> sure that when I did I overrode a lot of (all?) the suggestions.
>
>> I remember when Linux was this bad in the .90ish releases. A long
>> time ago. Now their install is painless, it's every bit as good as
>> Windows and maybe better.
>
> FWIW, I find Microsoft "Windows" installation terminally confusing
> (that's what you were talking about, right?). And I've run into
> serious problems with various Linux installations too. That doesn't
> make the FreeBSD tools better, but maybe it relativizes it.
>
>> And it got that way fast, I remember doing an install on some
>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>> worries, you could just move around with the keyboard. X11 came up
>> as part of the install, the entire install was graphical and
>> seamless.
>
> The FreeBSD installer *does* install X if you select it.
>
>> FreeBSD is stuck in the 1990's in terms of user interface.
>
> You're still talking about the installer, aren't you? The normal user
> interface is via the shell, which hasn't changed, and for a good
> reason.
>
>> They've done some good stuff in the kernel but it's not an end user
>> system,
>
> There I have to agree with you. A little TLC would go a long way.
> But I hope that you're not advocating the "change your GUI with your
> underwear" attitude that Microsoft, Apple and many Linux distros
> have. One of the reasons I don't use Linux is because every time I
> try, the interface has changed.
>
> Greg
> --
> Sent from my desktop computer.
> Finger grog(a)lemis.com for PGP public key.
> See complete headers for address and phone numbers.
> This message is digitally signed. If your Microsoft mail program
> reports problems, please read http://lemis.com/broken-MUA
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
4 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
I think this has gone on along enough. Don't want it to be personal, not
my intent.
Like I said it seems like a Linux vs FreeBSD thing. Don't want that.
You can search the archives about Ted talking about how ext was not
all that (I'm a fan).
On Sat, Jan 30, 2021 at 08:52:09PM -0600, Will Senn wrote:
Ha. Zfs may not be the be all and end all, but like I
said, it???s never failed me. Whereas extX and btrfs, and, and, and have many times.
Please don???t denigrate my knowledge, as so far as I know, we???ve never met, and nothing
I said warrants such. The installer reminds me of Redhat???s old anaconda installer,
I???ll grant you it???s dated. However, I typically install a new linux distro every week
and there are many, many installers that are far more confusing - Open Suse and Fedora are
two that come to mind, Debian as well. I would hazard to guess your favorite Linux is
based on a distro that lacks a decent installer (Ubuntu and Mint are Debian based).
Will
Sent from my iPhone
> On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
>
> If you like ZFS you don't understand operating systems design. I do.
> Jeff Bonwick was a stats student at Stanford when he took my OS class,
> I convinced him to come to Sun. Bill Moore worked for me. That's the two
> main ZFS guys and I thought I had taught them well but they let me down.
>
> ZFS doesn't use the page cache, they said it was too hard because ZFS
> is compressed. A typical file system just has block numbers, a compressed
> one needs another int per block, it's the int that says these many bytes
> are a block uncompressed. It's not that hard, it is 2 ints instead of 1.
>
> In case I'm not being clear, the page cache is what everyone else uses
> but ZFS has its own cache. So if you want to mmap() a ZFS file, ZFS
> has to bcopy() the data into the page cache and then spend a shit ton
> of code to make sure that the page cache data is in sync with the ZFS
> cache data.
>
> SunOS came from BSD but SunOS added mmap. Which had the same problem,
> the BSD buffer cache was exactly the same as the ZFS cache, Sun spent
> years of effort to get rid of the buffer cache, everything is in the
> page cache. So ZFS was a HUGE step backwards in systems design. Might
> be the best file system ever (it is not) but it was not a good player
> in the OS world.
>
> Those guys said that it was too hard to make a compressed file fit in
> the page cache. BitKeeper has that code and proves that it can be done.
> Be happy to walk anyone who cares through that code, I didn't write that,
> Wayne Scott did, but it's some of the best written code I've ever seen.
> Up there with Mojo's work on the SunOS VM system. (I'll bet that noone
> takes me up on this offer, people love to argue but most don't want to
> learn. Prove me wrong, please).
>
> So good on you that you like ZFS and FreeBSD. I don't and I don't for
> really good reasons.
>
> Let's try it this way. Get back to me when you can show me 40 people
> who have installed FreeBSD on their own, with no help. In the same
> time, I can show you 40,000 people who have installed Linux on their
> own, with no help. Probably 400,000.
>
> Technology is great, ease of use is what gets you users. ZFS is
> great but doesn't play nice with the OS.
>
> That's my oh brother.
>
>> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
>> Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS
which has NEVER let me down and I???ve done my share of stupid user error. Now that Linux
has ZFS, it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is
10x any of my Linux instances. We are soooo off topic, I think :). But, I???m always up
for talking up FBSD. I use it in my classes, too and the system is much more coherent for
my systems programming classes than linux.
>>
>> Will
>>
>> Sent from my iPhone
>>
>>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey
<grog(a)lemis.com> wrote:
>>>>
>>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>>>> If I could get the day-2-day
>>>>> applications that I need to work on FreeBSD, I suspect I would be
there in
>>>>> a heartbeat.
>>>>
>>>> I dunno about that. I tried out FreeBSD a couple of years ago when
>>>> Netflix was flirting with me. The installer hasn't seen any loving
in
>>>> 30 years it would seem. The disk setup tool sucks just as bad as it
>>>> did back in 1988.
>>>
>>> You could be right there, for some value of 1988 (FreeBSD came into
>>> being in 1992). The tools work without being good. But how often do
>>> you use them? I've been using FreeBSD since the beginning, and I
>>> can't recall when I last used the disk partitioning tool, though
I'm
>>> sure that when I did I overrode a lot of (all?) the suggestions.
>>>
>>>> I remember when Linux was this bad in the .90ish releases. A long
>>>> time ago. Now their install is painless, it's every bit as good
as
>>>> Windows and maybe better.
>>>
>>> FWIW, I find Microsoft "Windows" installation terminally
confusing
>>> (that's what you were talking about, right?). And I've run into
>>> serious problems with various Linux installations too. That doesn't
>>> make the FreeBSD tools better, but maybe it relativizes it.
>>>
>>>> And it got that way fast, I remember doing an install on some
>>>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>>>> worries, you could just move around with the keyboard. X11 came up
>>>> as part of the install, the entire install was graphical and
>>>> seamless.
>>>
>>> The FreeBSD installer *does* install X if you select it.
>>>
>>>> FreeBSD is stuck in the 1990's in terms of user interface.
>>>
>>> You're still talking about the installer, aren't you? The normal
user
>>> interface is via the shell, which hasn't changed, and for a good
>>> reason.
>>>
>>>> They've done some good stuff in the kernel but it's not an end
user
>>>> system,
>>>
>>> There I have to agree with you. A little TLC would go a long way.
>>> But I hope that you're not advocating the "change your GUI with
your
>>> underwear" attitude that Microsoft, Apple and many Linux distros
>>> have. One of the reasons I don't use Linux is because every time I
>>> try, the interface has changed.
>>>
>>> Greg
>>> --
>>> Sent from my desktop computer.
>>> Finger grog(a)lemis.com for PGP public key.
>>> See complete headers for address and phone numbers.
>>> This message is digitally signed. If your Microsoft mail program
>>> reports problems, please read http://lemis.com/broken-MUA
>
> --
> ---
> Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
4:06 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
No worries. Maybe it’s time for a vi emacs discussion... totally kidding :)
Sent from my iPhone
On Jan 30, 2021, at 9:00 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
I think this has gone on along enough. Don't want it to be personal, not
my intent.
Like I said it seems like a Linux vs FreeBSD thing. Don't want that.
You can search the archives about Ted talking about how ext was not
all that (I'm a fan).
On Sat, Jan 30, 2021 at 08:52:09PM -0600, Will
Senn wrote:
Ha. Zfs may not be the be all and end all, but like I said, it???s never failed me.
Whereas extX and btrfs, and, and, and have many times. Please don???t denigrate my
knowledge, as so far as I know, we???ve never met, and nothing I said warrants such. The
installer reminds me of Redhat???s old anaconda installer, I???ll grant you it???s dated.
However, I typically install a new linux distro every week and there are many, many
installers that are far more confusing - Open Suse and Fedora are two that come to mind,
Debian as well. I would hazard to guess your favorite Linux is based on a distro that
lacks a decent installer (Ubuntu and Mint are Debian based).
Will
Sent from my iPhone
> On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
>
> If you like ZFS you don't understand operating systems design. I do.
> Jeff Bonwick was a stats student at Stanford when he took my OS class,
> I convinced him to come to Sun. Bill Moore worked for me. That's the two
> main ZFS guys and I thought I had taught them well but they let me down.
>
> ZFS doesn't use the page cache, they said it was too hard because ZFS
> is compressed. A typical file system just has block numbers, a compressed
> one needs another int per block, it's the int that says these many bytes
> are a block uncompressed. It's not that hard, it is 2 ints instead of 1.
>
> In case I'm not being clear, the page cache is what everyone else uses
> but ZFS has its own cache. So if you want to mmap() a ZFS file, ZFS
> has to bcopy() the data into the page cache and then spend a shit ton
> of code to make sure that the page cache data is in sync with the ZFS
> cache data.
>
> SunOS came from BSD but SunOS added mmap. Which had the same problem,
> the BSD buffer cache was exactly the same as the ZFS cache, Sun spent
> years of effort to get rid of the buffer cache, everything is in the
> page cache. So ZFS was a HUGE step backwards in systems design. Might
> be the best file system ever (it is not) but it was not a good player
> in the OS world.
>
> Those guys said that it was too hard to make a compressed file fit in
> the page cache. BitKeeper has that code and proves that it can be done.
> Be happy to walk anyone who cares through that code, I didn't write that,
> Wayne Scott did, but it's some of the best written code I've ever seen.
> Up there with Mojo's work on the SunOS VM system. (I'll bet that noone
> takes me up on this offer, people love to argue but most don't want to
> learn. Prove me wrong, please).
>
> So good on you that you like ZFS and FreeBSD. I don't and I don't for
> really good reasons.
>
> Let's try it this way. Get back to me when you can show me 40 people
> who have installed FreeBSD on their own, with no help. In the same
> time, I can show you 40,000 people who have installed Linux on their
> own, with no help. Probably 400,000.
>
> Technology is great, ease of use is what gets you users. ZFS is
> great but doesn't play nice with the OS.
>
> That's my oh brother.
>
>> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
>> Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS
which has NEVER let me down and I???ve done my share of stupid user error. Now that Linux
has ZFS, it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is
10x any of my Linux instances. We are soooo off topic, I think :). But, I???m always up
for talking up FBSD. I use it in my classes, too and the system is much more coherent for
my systems programming classes than linux.
>>
>> Will
>>
>> Sent from my iPhone
>>
>>>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey
<grog(a)lemis.com> wrote:
>>>>>
>>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>>>> If I could get the day-2-day
>>>>> applications that I need to work on FreeBSD, I suspect I would be
there in
>>>>> a heartbeat.
>>>>
>>>> I dunno about that. I tried out FreeBSD a couple of years ago when
>>>> Netflix was flirting with me. The installer hasn't seen any loving
in
>>>> 30 years it would seem. The disk setup tool sucks just as bad as it
>>>> did back in 1988.
>>>
>>> You could be right there, for some value of 1988 (FreeBSD came into
>>> being in 1992). The tools work without being good. But how often do
>>> you use them? I've been using FreeBSD since the beginning, and I
>>> can't recall when I last used the disk partitioning tool, though
I'm
>>> sure that when I did I overrode a lot of (all?) the suggestions.
>>>
>>>> I remember when Linux was this bad in the .90ish releases. A long
>>>> time ago. Now their install is painless, it's every bit as good as
>>>> Windows and maybe better.
>>>
>>> FWIW, I find Microsoft "Windows" installation terminally confusing
>>> (that's what you were talking about, right?). And I've run into
>>> serious problems with various Linux installations too. That doesn't
>>> make the FreeBSD tools better, but maybe it relativizes it.
>>>
>>>> And it got that way fast, I remember doing an install on some
>>>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>>>> worries, you could just move around with the keyboard. X11 came up
>>>> as part of the install, the entire install was graphical and
>>>> seamless.
>>>
>>> The FreeBSD installer *does* install X if you select it.
>>>
>>>> FreeBSD is stuck in the 1990's in terms of user interface.
>>>
>>> You're still talking about the installer, aren't you? The normal
user
>>> interface is via the shell, which hasn't changed, and for a good
>>> reason.
>>>
>>>> They've done some good stuff in the kernel but it's not an end
user
>>>> system,
>>>
>>> There I have to agree with you. A little TLC would go a long way.
>>> But I hope that you're not advocating the "change your GUI with
your
>>> underwear" attitude that Microsoft, Apple and many Linux distros
>>> have. One of the reasons I don't use Linux is because every time I
>>> try, the interface has changed.
>>>
>>> Greg
>>> --
>>> Sent from my desktop computer.
>>> Finger grog(a)lemis.com for PGP public key.
>>> See complete headers for address and phone numbers.
>>> This message is digitally signed. If your Microsoft mail program
>>> reports problems, please read http://lemis.com/broken-MUA
>
> --
> ---
> Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
4:32 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Since, I am an ex(1) troglodyte, I'll be happy to umpire. (I know ed(1) is
the standard editor, but I'm willing to trade a little standardosity for
more convenience.)
On Sat, Jan 30, 2021 at 10:06 PM Will Senn <will.senn(a)gmail.com> wrote:
No worries. Maybe it’s time for a vi emacs
discussion... totally kidding :)
Sent from my iPhone
On Jan 30, 2021, at 9:00 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
I think this has gone on along enough. Don't want it to be personal, not
my intent.
Like I said it seems like a Linux vs FreeBSD thing. Don't want that.
You can search the archives about Ted talking about how ext was not
all that (I'm a fan).
> On Sat, Jan 30, 2021 at 08:52:09PM -0600, Will Senn wrote:
> Ha. Zfs may not be the be all and end all, but like I said, it???s
never
failed me. Whereas extX and btrfs, and, and, and have many times.
Please don???t denigrate my knowledge, as so far as I know, we???ve never
met, and nothing I said warrants such. The installer reminds me of
Redhat???s old anaconda installer, I???ll grant you it???s dated. However,
I typically install a new linux distro every week and there are many, many
installers that are far more confusing - Open Suse and Fedora are two that
come to mind, Debian as well. I would hazard to guess your favorite Linux
is based on a distro that lacks a decent installer (Ubuntu and Mint are
Debian based).
>
> Will
>
> Sent from my iPhone
>
>> On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
>>
>> If you like ZFS you don't understand operating systems design. I do.
>> Jeff Bonwick was a stats student at Stanford when he took my OS class,
>> I convinced him to come to Sun. Bill Moore worked for me. That's the
two
>> main ZFS guys and I thought I had taught
them well but they let me
down.
>>
>> ZFS doesn't use the page cache, they said it was too hard because ZFS
>> is compressed. A typical file system just has block numbers, a
compressed
>> one needs another int per block,
it's the int that says these many
bytes
>> are a block uncompressed. It's not
that hard, it is 2 ints instead of
1.
>>
>> In case I'm not being clear, the page cache is what everyone else uses
>> but ZFS has its own cache. So if you want to mmap() a ZFS file, ZFS
>> has to bcopy() the data into the page cache and then spend a shit ton
>> of code to make sure that the page cache data is in sync with the ZFS
>> cache data.
>>
>> SunOS came from BSD but SunOS added mmap. Which had the same problem,
>> the BSD buffer cache was exactly the same as the ZFS cache, Sun spent
>> years of effort to get rid of the buffer cache, everything is in the
>> page cache. So ZFS was a HUGE step backwards in systems design. Might
>> be the best file system ever (it is not) but it was not a good player
>> in the OS world.
>>
>> Those guys said that it was too hard to make a compressed file fit in
>> the page cache. BitKeeper has that code and proves that it can be
done.
>> Be happy to walk anyone who cares through
that code, I didn't write
that,
>> Wayne Scott did, but it's some of
the best written code I've ever seen.
>> Up there with Mojo's work on the SunOS VM system. (I'll bet that
noone
>> takes me up on this offer, people love to argue but most don't want to
>> learn. Prove me wrong, please).
>>
>> So good on you that you like ZFS and FreeBSD. I don't and I don't for
>> really good reasons.
>>
>> Let's try it this way. Get back to me when you can show me 40 people
>> who have installed FreeBSD on their own, with no help. In the same
>> time, I can show you 40,000 people who have installed Linux on their
>> own, with no help. Probably 400,000.
>>
>> Technology is great, ease of use is what gets you users. ZFS is
>> great but doesn't play nice with the OS.
>>
>> That's my oh brother.
>>
>>> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
>>> Oh brother. I use FreeBSD all the time. I prefer it for its stability
and ZFS which has NEVER let me down and I???ve done my share of stupid user
error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages,
but uptime on my fbsd instance is 10x any of my Linux instances. We are
soooo off topic, I think :). But, I???m always up for talking up FBSD. I
use it in my classes, too and the system is much more coherent for my
systems programming classes than linux.
>>>
>>> Will
>>>
>>> Sent from my iPhone
>>>
>>>>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey
<grog(a)lemis.com>
wrote:
>>>>>>
>>>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy
wrote:
>>>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>>>>> If I could get the day-2-day
>>>>>> applications that I need to work on FreeBSD, I suspect I would be
there in
>>>>>> a heartbeat.
>>>>>
>>>>> I dunno about that. I tried out FreeBSD a couple of years ago when
>>>>> Netflix was flirting with me. The installer hasn't seen any
loving
in
>>>>> 30 years it would seem. The
disk setup tool sucks just as bad as it
>>>>> did back in 1988.
>>>>
>>>> You could be right there, for some value of 1988 (FreeBSD came into
>>>> being in 1992). The tools work without being good. But how often do
>>>> you use them? I've been using FreeBSD since the beginning, and I
>>>> can't recall when I last used the disk partitioning tool, though
I'm
>>>> sure that when I did I overrode a lot of (all?) the suggestions.
>>>>
>>>>> I remember when Linux was this bad in the .90ish releases. A long
>>>>> time ago. Now their install is painless, it's every bit as good
as
>>>>> Windows and maybe better.
>>>>
>>>> FWIW, I find Microsoft "Windows" installation terminally
confusing
>>>> (that's what you were talking about, right?). And I've run
into
>>>> serious problems with various Linux installations too. That
doesn't
>>>> make the FreeBSD tools better, but maybe it relativizes it.
>>>>
>>>>> And it got that way fast, I remember doing an install on some
>>>>> machine around 1998 or 1999, I didn't have a mouse plugged in,
no
>>>>> worries, you could just move around with the keyboard. X11 came up
>>>>> as part of the install, the entire install was graphical and
>>>>> seamless.
>>>>
>>>> The FreeBSD installer *does* install X if you select it.
>>>>
>>>>> FreeBSD is stuck in the 1990's in terms of user interface.
>>>>
>>>> You're still talking about the installer, aren't you? The
normal
user
>>>> interface is via the shell, which
hasn't changed, and for a good
>>>> reason.
>>>>
>>>>> They've done some good stuff in the kernel but it's not an
end user
>>>>> system,
>>>>
>>>> There I have to agree with you. A little TLC would go a long way.
>>>> But I hope that you're not advocating the "change your GUI with
your
>>>> underwear" attitude that Microsoft, Apple and many Linux distros
>>>> have. One of the reasons I don't use Linux is because every time I
>>>> try, the interface has changed.
>>>>
>>>> Greg
>>>> --
>>>> Sent from my desktop computer.
>>>> Finger grog(a)lemis.com for PGP public key.
>>>> See complete headers for address and phone numbers.
>>>> This message is digitally signed. If your Microsoft mail program
>>>> reports problems, please read http://lemis.com/broken-MUA
>>
>> --
>> ---
>> Larry McVoy lm at mcvoy.com
http://www.mcvoy.com/lm
--
---
Larry McVoy lm at mcvoy.com
http://www.mcvoy.com/lm
4 Feb
4 Feb
6:43 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sat, 30 Jan 2021, Larry McVoy wrote:
[ Usual insightful... insights ]
If you like ZFS you don't understand operating
systems design. I do.
Indeed...
Jeff Bonwick was a stats student at Stanford when he
took my OS class, I
convinced him to come to Sun. Bill Moore worked for me. That's the two
main ZFS guys and I thought I had taught them well but they let me down.
{ ... ]
There's no way that I'd use ZFS; lose a block in an ordinary file, well,
you now have a hole (but not in the file-system sense); lose a block in a
compressed system, well...
Or perhaps I'm becoming conservative in my old age; I remember when I once
rewrote utilities that when writing a zero block merely did a seek instead
(or something like that; you had to remember to actually write out the
last block). I wouldn't try it these days, as Unix file systems were
simple back then.
[ ... ]
Let's try it this way. Get back to me when you
can show me 40 people
who have installed FreeBSD on their own, with no help. In the same
time, I can show you 40,000 people who have installed Linux on their
own, with no help. Probably 400,000.
Well, I did (but without ZFS) on several boxes, with zero help. Having
had SunOS experience (4.4 was the best) helped :-)
I can't stand Penguin/OS; it looks too much like Windoze for my liking
(and does its best to be almost-Unix-but-not-quite).
-- Dave, a grey-beard
7:10 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Not entirely sure as it's been a while since iI have used it but last time
I remember TrueOS which used FreeBSD was easy to use for a newbie. There
are other FreeBSD distro's like GhostBSD,etc that have an installer like
Linux.
My 2c is that FreeBSD is not trying to get people in that are newbies, it's
for the server environment and it works extremely well.
Kind Regards,
Angus Robinson
On Thu, Feb 4, 2021 at 7:45 AM Dave Horsfall <dave(a)horsfall.org> wrote:
On Sat, 30 Jan 2021, Larry McVoy wrote:
[ Usual insightful... insights ]
If you like ZFS you don't understand
operating systems design. I do.
Indeed...
Jeff Bonwick was a stats student at Stanford when
he took my OS class, I
convinced him to come to Sun. Bill Moore worked for me. That's the two
main ZFS guys and I thought I had taught them well but they let me down.
{ ... ]
There's no way that I'd use ZFS; lose a block in an ordinary file, well,
you now have a hole (but not in the file-system sense); lose a block in a
compressed system, well...
Or perhaps I'm becoming conservative in my old age; I remember when I once
rewrote utilities that when writing a zero block merely did a seek instead
(or something like that; you had to remember to actually write out the
last block). I wouldn't try it these days, as Unix file systems were
simple back then.
[ ... ]
Let's try it this way. Get back to me when
you can show me 40 people
who have installed FreeBSD on their own, with no help. In the same
time, I can show you 40,000 people who have installed Linux on their
own, with no help. Probably 400,000.
Well, I did (but without ZFS) on several boxes, with zero help. Having
had SunOS experience (4.4 was the best) helped :-)
I can't stand Penguin/OS; it looks too much like Windoze for my liking
(and does its best to be almost-Unix-but-not-quite).
-- Dave, a grey-beard
8:46 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 2/4/21, Angus Robinson <angus(a)fairhaven.za.net> wrote:
it's for the server environment and it works
extremely well.
Well...20 years ago it was a catchy phrase when FreeBSD was still in
stiff competition with Linux. Now 20 years later we all see who won
the Unix server wars. Literally every corporate entity is using some
form of commercial Red Hat Enterprise Linux now. It is running
everywhere.
There are some individual companies that still utilizes FreeBSD like
Netflix, but the overall majority of server systems are based on
Linux. I am not saying it is better; I am just pointing out the
facts. I generally despise all modern Unix: be it Linux with systemd
or FreeBSD as being too bloated and complex. My personal opinion is
that Linux peaked around late 90s-early/mid 2000s. It was still
pretty logical and simple, but excelled in performance. It went
downhill since then...
--Andy
11:25 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Thu, 4 Feb 2021, Angus Robinson wrote:
Not entirely sure as it's been a while since iI
have used it but last
time I remember TrueOS which used FreeBSD was easy to use for a newbie.
There are other FreeBSD distro's like GhostBSD,etc that have an
installer like Linux. My 2c is that FreeBSD is not trying to get
people in that are newbies, it's for the server environment and it works
extremely well.
Exactly; FreeBSD is not for newbies: I'll leave Penguin/OS to that market.
Mind you, OpenBSD was a bit of a bugger; all services are off by default.
It's perfect for a firewall :-)
-- Dave
4:45 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 2/3/21 11:43 PM, Dave Horsfall wrote:
On Sat, 30 Jan 2021, Larry McVoy wrote:
[ Usual insightful... insights ]
ZFS needn't be compressed, and I don't generally do compression or
encryption unless required by law, so I can't speak from personal
experience on those use cases (others, far more experienced can). I do
know that it's truly a pain to recover from issues with either.
In response to the negative vibes around ZFS. I've never lost a file (or
a piece of a file) in 10+ years of using ZFS. I get the feeling we may
not be talking about the same ZFS. My experience is with the ZFS FreeBSD
comes with, not the version that Oracle owns. Perhaps the info is a
little out of date for the naysayers. In my experience, using ZFS is
fairly transparent and simple to use - no partitioning to deal with, no
need to worry about generating filesystems, none of that - add your
disks to a pool, choose your RAID levels and it gets mounted, no fuss.
I've lost plenty of disks along the way, but ZFS just keeps on chugging
along nicely until I replace them and then rebuilds the arrays, again,
no fuss other than replacing the hardware. In terms of massive system
updates and such, I just snapshot the environment (a near instantaneous
operation) before making significant changes to my system, that might
break things and when they do break (and they do, more often than I'd
like), I just rollback. man bectl. Painless (and I mean painless,
hundreds of times, or mor). I'm sure it all sounds scifi, but it's my
experience along with plenty of other folks, and this ZFS sucks thread
seems to be FUD to me - ala Microsoft vs Linux, or at best informed
hypothetical speculations - reminds me of an if statement conversation I
had online in the early 1990's where one group of folks claimed that
braces worked a certain way, based on the then current standard, and
another group of folks (I'd be on this side of things), tested the
theory with a host of compilers, observed the functions effects, shook
their heads and wondered why it didn't match up with the theory, and
said it worked another. Who was right? I'm still not entirely sure, from
a philosophical perspective, but I have since coded my if statements
according to my environment, not the standards.
As I mentioned in the prior thread, I've lost my share of files and file
systems (many, many times since 1993 when I started with linux - 0.9
kernel, slackware, then redhat, then debian, now mint) with ext3/4, and
btrfs, though, and the only recovery was backup (a time intensive
process). I really don't see the logic behind the negative arguments.
Don't like it, fine, say it and live it. Claim it sucks? Then, back it
up with a real-world, current experience and I'll cede the point - I'll
keep using ZFS though :).
I want to be clear, I don't dislike Linux. I don't think FreeBSD is
superior. I like both. I use both... daily. With enough prep and
planning, my linux environment is similarly recoverable, but with
freebsd, the prep and planning requires a lot less time and effort.
Personally, I heart linux Mint - it's based on Debian and Ubuntu - is a
straightforward install, works well, has zfs (not yet on boot), has
timeshift (lovely piece of software), and can be quite pretty.
Vive la difference.
Will
If you like ZFS you don't understand
operating systems design. I do.
...
There's no way that I'd use ZFS; lose a block in an ordinary file,
well, you now have a hole (but not in the file-system sense); lose a
block in a compressed system, well...
5:03 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
I don't really know enough about filesystem internals to comment on the
design one way or another. I can say that Oberlin College Computer Science
switched to using ZFS with snapshots in 2008 or so (on Solaris 10) and it
greatly simplified restoring from backups. Home directories had weekly
snapshots taken, rolling over a month or two, and it was a godsend when a
student (or a professor...) accidentally deleted something important. No
more trawling through backup tapes by the admin to restore the file you
wanted, it could easily be taken from an on-disk snapshot. Obviously it
required a certain greater amount of disk space, but I contend that it was
worth it.
-Henry
On Thu, 4 Feb 2021 at 10:47, Will Senn <will.senn(a)gmail.com> wrote:
On 2/3/21 11:43 PM, Dave Horsfall wrote:
On Sat, 30 Jan 2021, Larry McVoy wrote:
[ Usual insightful... insights ]
ZFS needn't be compressed, and I don't generally do compression or
encryption unless required by law, so I can't speak from personal
experience on those use cases (others, far more experienced can). I do
know that it's truly a pain to recover from issues with either.
In response to the negative vibes around ZFS. I've never lost a file (or
a piece of a file) in 10+ years of using ZFS. I get the feeling we may
not be talking about the same ZFS. My experience is with the ZFS FreeBSD
comes with, not the version that Oracle owns. Perhaps the info is a
little out of date for the naysayers. In my experience, using ZFS is
fairly transparent and simple to use - no partitioning to deal with, no
need to worry about generating filesystems, none of that - add your
disks to a pool, choose your RAID levels and it gets mounted, no fuss.
I've lost plenty of disks along the way, but ZFS just keeps on chugging
along nicely until I replace them and then rebuilds the arrays, again,
no fuss other than replacing the hardware. In terms of massive system
updates and such, I just snapshot the environment (a near instantaneous
operation) before making significant changes to my system, that might
break things and when they do break (and they do, more often than I'd
like), I just rollback. man bectl. Painless (and I mean painless,
hundreds of times, or mor). I'm sure it all sounds scifi, but it's my
experience along with plenty of other folks, and this ZFS sucks thread
seems to be FUD to me - ala Microsoft vs Linux, or at best informed
hypothetical speculations - reminds me of an if statement conversation I
had online in the early 1990's where one group of folks claimed that
braces worked a certain way, based on the then current standard, and
another group of folks (I'd be on this side of things), tested the
theory with a host of compilers, observed the functions effects, shook
their heads and wondered why it didn't match up with the theory, and
said it worked another. Who was right? I'm still not entirely sure, from
a philosophical perspective, but I have since coded my if statements
according to my environment, not the standards.
As I mentioned in the prior thread, I've lost my share of files and file
systems (many, many times since 1993 when I started with linux - 0.9
kernel, slackware, then redhat, then debian, now mint) with ext3/4, and
btrfs, though, and the only recovery was backup (a time intensive
process). I really don't see the logic behind the negative arguments.
Don't like it, fine, say it and live it. Claim it sucks? Then, back it
up with a real-world, current experience and I'll cede the point - I'll
keep using ZFS though :).
I want to be clear, I don't dislike Linux. I don't think FreeBSD is
superior. I like both. I use both... daily. With enough prep and
planning, my linux environment is similarly recoverable, but with
freebsd, the prep and planning requires a lot less time and effort.
Personally, I heart linux Mint - it's based on Debian and Ubuntu - is a
straightforward install, works well, has zfs (not yet on boot), has
timeshift (lovely piece of software), and can be quite pretty.
Vive la difference.
Will
If you like ZFS you don't understand
operating systems design. I do.
...
There's no way that I'd use ZFS; lose a block in an ordinary file,
well, you now have a hole (but not in the file-system sense); lose a
block in a compressed system, well...
5:32 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Thu, Feb 4, 2021 at 10:47 AM Will Senn <will.senn(a)gmail.com> wrote:
[snip]
In response to the negative vibes around ZFS. [snip]
I think the discordance is around the semantics ZFS's implementation
implies. Larry's point about mmap() vs a buffer cache is entirely valid; it
took lots of people heroic amounts of work worthy of Greek sagas to bridge
the difference between the original buffer and VM page caches, but ZFS
says, "meh. too much work; not worth it." The practical implication of that
is that memory mapped IO (via `mmap`) is no longer coherent with file IO
(via `open`/`close`/`read`/`write`) without lots of work that both degrades
performance and add complexity.
The question that a lot of folks who use ZFS regularly ask is, "does that
matter?" And perhaps it doesn't: if I've got a file server sitting there
serving NFS, do I care what it's kernel is doing? As long as it's
saturating the network and disks, and it's reliable...not really.
(Incidentally, that was kind of the philosophy behind the original plan9
file server kernel...as I heard the story, the rate of change of the plan9
kernel proper was too high, so Ken split off the file server portion into
its own, special-purpose kernel, and it stayed like that for ~20 years).
Similarly, if I'm on the local machine and the required coherence code is
there and largely works, then again, perhaps as a consumer of the
filesystem, I just don't care. After all, one can still get work done, and
ZFS has a bunch of other features that make it very attractive, right? In
particular, it's very good at NOT losing my data, kernel purity be damned.
On the other hand, if we're discussing OS design and implementation,
(re)splitting the VM and buffer caches is a poor decision. One might well
ask, "why?" and the answer may be, "because it adds significant complexity
to the kernel." This to me seems like the crux of the disagreement.
Satisfied users of ZFS might legitimately ask, "who cares?" and one might
respond, "kernel maintainers." If the kernel is mostly transparent as far
as a particular use case goes, though, then I can see why one would bulk at
the suggestion that this matters. If one is concerned with the design and
implementation of kernels, I could see why one would care very much.
Like many things, it's a matter of perspective.
- Dan C.
5:49 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 2/4/21 10:32 AM, Dan Cross wrote:
On Thu, Feb 4, 2021 at 10:47 AM Will Senn
<will.senn(a)gmail.com
<mailto:will.senn@gmail.com>> wrote:
[snip]
In response to the negative vibes around ZFS. [snip]
I think the discordance is around the semantics ZFS's implementation
implies. Larry's point about mmap() vs a buffer cache is entirely
valid; it took lots of people heroic amounts of work worthy of Greek
sagas to bridge the difference between the original buffer and VM page
caches, but ZFS says, "meh. too much work; not worth it." The
practical implication of that is that memory mapped IO (via `mmap`) is
no longer coherent with file IO (via `open`/`close`/`read`/`write`)
without lots of work that both degrades performance and add complexity.
The question that a lot of folks who use ZFS regularly ask is, "does
that matter?" And perhaps it doesn't: if I've got a file server
sitting there serving NFS, do I care what it's kernel is doing? As
long as it's saturating the network and disks, and it's reliable...not
really. (Incidentally, that was kind of the philosophy behind the
original plan9 file server kernel...as I heard the story, the rate of
change of the plan9 kernel proper was too high, so Ken split off the
file server portion into its own, special-purpose kernel, and it
stayed like that for ~20 years). Similarly, if I'm on the local
machine and the required coherence code is there and largely works,
then again, perhaps as a consumer of the filesystem, I just don't
care. After all, one can still get work done, and ZFS has a bunch of
other features that make it very attractive, right? In particular,
it's very good at NOT losing my data, kernel purity be damned.
On the other hand, if we're discussing OS design and implementation,
(re)splitting the VM and buffer caches is a poor decision. One might
well ask, "why?" and the answer may be, "because it adds significant
complexity to the kernel." This to me seems like the crux of the
disagreement. Satisfied users of ZFS might legitimately ask, "who
cares?" and one might respond, "kernel maintainers." If the kernel is
mostly transparent as far as a particular use case goes, though, then
I can see why one would bulk at the suggestion that this matters. If
one is concerned with the design and implementation of kernels, I
could see why one would care very much.
Like many things, it's a matter of perspective.
- Dan C.
Thanks for the comments, Dan. I see your point. I was thinking as a
user/admin. In this light, I'll admit that I'm not an expert on the
internals and say that I can only imagine the breadth of design
tradeoffs that were contemplated and the many decisions that were made
when coming up with ZFS. I'm glad somebody thought through them, and
worked through them, though. So I could consume their work,
Frankensteinian though it may be. Now, if somebody would only get it
working properly in Linux (boot environments included), or even get
BTRFS to be more realiable, I'd be a happy camper, at least for a while :).
6:46 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
+1 to everything Dan said, including the "who cares if it works?".
I was definitely coming at it from the kernel perspective, I joined Sun
just after SunOS 4.0 came out; that was the release that made mmap/page
cache be a coherent thing, the buffer cache was almost completely gone,
I think it was still used for inodes and directories but no user data
was in there.
The amount of work that went into that was huge but the resulting kernel
was actually smaller (I think) and much easier to understand and maintain
(I know). And the kernel team was quite proud of that work, with good
reason.
Which is why it blows my mind that the same organization that saw the
value of have one way to do things, did a 180 and said the page cache
isn't that important. When I talked to the ZFS guys about it, their
plan was that ZFS was going to be the only local file system so it sort
of didn't matter. But that's nonsense, you are going to have tmpfs,
nfs, vfat, ntfs, whatever apple uses, etc. So you are going to have a
page cache for all of those, which means the page cache is never going
away, which means that mmap() wants pages (remember, you can mmap stuff
that isn't in memory, you need the hardware to fault for you so you can
go get it). The ZFS guys said making that work is too hard, we did it
in BitKeeper, it works fine.
So the combination of going back to something anyone with OS smarts
knows is a bad idea plus the fact that I know from personal experience
that they could have done in the page cache and still have compression
and XOR, yeah, I lost a lot of respect for those guys.
ZFS is very useful, it could have been that and played nice with the
page cache.
On Thu, Feb 04, 2021 at 11:32:31AM -0500, Dan Cross wrote:
On Thu, Feb 4, 2021 at 10:47 AM Will Senn
<will.senn(a)gmail.com> wrote:
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
[snip]
In response to the negative vibes around ZFS. [snip]
I think the discordance is around the semantics ZFS's implementation
implies. Larry's point about mmap() vs a buffer cache is entirely valid; it
took lots of people heroic amounts of work worthy of Greek sagas to bridge
the difference between the original buffer and VM page caches, but ZFS
says, "meh. too much work; not worth it." The practical implication of that
is that memory mapped IO (via `mmap`) is no longer coherent with file IO
(via `open`/`close`/`read`/`write`) without lots of work that both degrades
performance and add complexity.
The question that a lot of folks who use ZFS regularly ask is, "does that
matter?" And perhaps it doesn't: if I've got a file server sitting there
serving NFS, do I care what it's kernel is doing? As long as it's
saturating the network and disks, and it's reliable...not really.
(Incidentally, that was kind of the philosophy behind the original plan9
file server kernel...as I heard the story, the rate of change of the plan9
kernel proper was too high, so Ken split off the file server portion into
its own, special-purpose kernel, and it stayed like that for ~20 years).
Similarly, if I'm on the local machine and the required coherence code is
there and largely works, then again, perhaps as a consumer of the
filesystem, I just don't care. After all, one can still get work done, and
ZFS has a bunch of other features that make it very attractive, right? In
particular, it's very good at NOT losing my data, kernel purity be damned.
On the other hand, if we're discussing OS design and implementation,
(re)splitting the VM and buffer caches is a poor decision. One might well
ask, "why?" and the answer may be, "because it adds significant
complexity
to the kernel." This to me seems like the crux of the disagreement.
Satisfied users of ZFS might legitimately ask, "who cares?" and one might
respond, "kernel maintainers." If the kernel is mostly transparent as far
as a particular use case goes, though, then I can see why one would bulk at
the suggestion that this matters. If one is concerned with the design and
implementation of kernels, I could see why one would care very much.
Like many things, it's a matter of perspective.
- Dan C.
7:41 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Feb 4, 2021, at 8:34 AM, Dan Cross <crossd(a)gmail.com> wrote:
On the other hand, if we're discussing OS design and implementation, (re)splitting
the VM and buffer caches is a poor decision. One might well ask, "why?" and the
answer may be, "because it adds significant complexity to the kernel." This to
me seems like the crux of the disagreement. Satisfied users of ZFS might legitimately ask,
"who cares?" and one might respond, "kernel maintainers." If the
kernel is mostly transparent as far as a particular use case goes, though, then I can see
why one would bulk at the suggestion that this matters. If one is concerned with the
design and implementation of kernels, I could see why one would care very much.
Largely agree; though the complexity battle has long been lost. On multiple fronts. Many
of us are happy to use such complex systems for their ease of use or their feature set but
wouldn’t want to maintain these systems!
I have used ZFS since 2005 and largely happy with it. Replaced all the disks twice. Moved
the same set of disks to a new machine. etc. Features: cheap and fast snapshots,
send/receive, clone, adding disks, checksummed blocks, redundancy etc. The dedup impl. is
suboptimal so I don't use it. No idea if they considered using a bloom filter and a
cache to reduce memory use. If a new FS came along with a similar set of features and a
simpler, better integrated implementation, I'd switch.
11:28 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
snapshots of FS state existed for years before ZFS/BTRFS did it,
because FS journaling existed. The underlying model of COW forking of
the inodes and a journal were all there.
What ZFS did, and what Docker does, and snap does, and flatpack does,
is package things in a way which make modalities for use for modern
sysadmin "just work"
I don't recall seeing a backup model built on UFS+journalling with an
integrated command tooling to do what zfs snapshot; zfs send | zfs
receive does as an incremental copy mechanism. I'm not trying to
disrespect the prior art, if there was a good packaging, I probably
missed it. ZFS made this "yea, I get it now" for a lot of people.
I think ZFS is fine. I have adopted it wholesale in the work context
for the last 15+ years and at home for about a year. The FUD around
minimum memory for ARC is a misunderstanding of an Oracle document
"back then" and you can run ZFS on rPi class systems fine, if you can
accept some lumpy paging and VM behaviour, and if you don't enable
compression which burns the ARC. (you need the extent of memory to
deal with things a lot more) -And there are modern OpenZFS docs which
patiently explain this stuff.
Zsys is going to get better. This means rollback from systems update
under Ubuntu snaps and things, will get better. It will be a lot less
risky to do significant upgrade on systems. -Think about Android and
the two-root model of updating the passive root in the background, so
the new OS is live across a reboot without having the spinning
beachball of upgrade on your phone, or analogues of the "keep this
setting or go back" model WIndows does, and Windows has had OS
snapshots for a long time now, and offers "undo that major update"
models to users: people expect this.
I also think ZFS was a godsend for getting over the smart RAID card
mistakes of the past. I HAVE lost data with these puppies. I've moved
multi TB fs between FreeBSD and Linux and back again (with care) under
ZFS, and you can't do that with Apples FS, or EXT3 with anything like
the same confidence.
Really? I like UUID. UUID are a godsend, for making things have
unique, but asynchronously generated identity, so when you move them
and mix them, you can stop worrying about device/bus order and simply
re-create them as they were defined semantically. ZFS does this too:
zfs import is leveraging what UUID does for you.
Basically Larry, I think you are kindof wrong. These alumni of yours
did what all kids should do: they ran ahead. Did they scrape their
knees doing it? Sure. But if they don't try things their teachers say
are bad, how do they advance the art? If we'd listened to Eddy
Dijkstra, we'd never have got BGP: He said it couldn't scale, even
though it was based on his own work.
-G
On Fri, Feb 5, 2021 at 4:41 AM Bakul Shah <bakul(a)iitbombay.org> wrote:
On Feb 4, 2021, at 8:34 AM, Dan Cross <crossd(a)gmail.com> wrote:
On the other hand, if we're discussing OS design and implementation, (re)splitting
the VM and buffer caches is a poor decision. One might well ask, "why?" and the
answer may be, "because it adds significant complexity to the kernel." This to
me seems like the crux of the disagreement. Satisfied users of ZFS might legitimately ask,
"who cares?" and one might respond, "kernel maintainers." If the
kernel is mostly transparent as far as a particular use case goes, though, then I can see
why one would bulk at the suggestion that this matters. If one is concerned with the
design and implementation of kernels, I could see why one would care very much.
Largely agree; though the complexity battle has long been lost. On multiple fronts. Many
of us are happy to use such complex systems for their ease of use or their feature set but
wouldn’t want to maintain these systems!
I have used ZFS since 2005 and largely happy with it. Replaced all the disks twice. Moved
the same set of disks to a new machine. etc. Features: cheap and fast snapshots,
send/receive, clone, adding disks, checksummed blocks, redundancy etc. The dedup impl. is
suboptimal so I don't use it. No idea if they considered using a bloom filter and a
cache to reduce memory use. If a new FS came along with a similar set of features and a
simpler, better integrated implementation, I'd switch.
11:41 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Feb 4, 2021, at 2:28 PM, George Michaelson <ggm(a)algebras.org> wrote:
Basically Larry, I think you are kindof wrong. These alumni of yours
did what all kids should do: they ran ahead. Did they scrape their
knees doing it? Sure. But if they don't try things their teachers say
are bad, how do they advance the art? If we'd listened to Eddy
Dijkstra, we'd never have got BGP: He said it couldn't scale, even
though it was based on his own work.
I think Larry is talking about ZFS *implementation* design choices;
which we, as ZFS users, mostly don't care about!
It was OSPF, not BGP, that used Dijkstra's SPF algorithm. The last
I knew BGP was all about "policy" -- why random bad actors can sling
half of the internet traffic through China or Vanuatu or whatever.
5 Feb
5 Feb
1:33 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Fri, Feb 05, 2021 at 08:28:12AM +1000, George Michaelson wrote:
What ZFS did, and what Docker does, and snap does, and
flatpack does,
is package things in a way which make modalities for use for modern
sysadmin "just work"
No argument there.
Basically Larry, I think you are kindof wrong. These
alumni of yours
did what all kids should do: they ran ahead. Did they scrape their
knees doing it? Sure. But if they don't try things their teachers say
are bad, how do they advance the art?
Before I show you I'm not wrong, if you are saying (and I think you
are) that you like ZFS and find it useful, I have no disagreement with
that. I'm in no way arguing that ZFS isn't useful. If you are just
an end user and you don't run into any of the coherency problems,
it's great.
I'm arguing from the point of view of how a kernel is supposed to work.
What ZFS did is a gross violation of how the kernel is supposed to work
and both Bonwick and Moore have admitted that, they just thought it was
too hard to do it right. There is a body of code in BitKeeper that does
the exact part that they thought was too hard, a layer that takes a page
fault and fills in the page from a compressed and xor-ed data source.
Works great, one guy did it in a few months or so. It's not that hard.
So why is what ZFS did so wrong?
Ignoring the page cache and make their own cache has big problems.
You can mmap() ZFS files and doing so means that when a page is referenced
it is copied from the ZFS cache to the page cache. That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
You can get around it, I know, because I've written the coherency code
for SGI's IRIX when I did the bulk data server that went around the page
cache and made NFS go at 60MB/sec for a single stream (and many times
that for multiple streams). So I'm not talking out of my ass, I know
what coherency means when you have the same data in two different places,
I know it is possible to make it work, I've done that, and I don't think
it is a good idea (it was OK in SGI's case, it was for O_DIRECT which
exists to completely bypass the page cache; so a special case that wasn't
so bad and wasn't general).
It's messy. You could remove the data from the ZFS cache when you put
it in the page cache but ZFS is compresses so it's not going line up on
page boundaries like you'd want it to. That means you're removing more
than your page which sort of sucks.
You could never map the pages writable, take a fault every time someone
wants to write the page and then do the write back to the ZFS cache.
That doesn't really work because you take the fault before the write
completes, not after. You can make it work, the write fault has to
get an exclusive lock on the data in the ZFS cache, then return, then
the page gets modified, now someone has to wake up and copy that data
from the page cache to ZFS. It's messy and it performs really poorly,
nobody would do it this way.
You could lock the data in the ZFS cache, making it read only. That
doesn't work because you can write via mmap() and read via ZFS and you
get old data.
All of these sorts of problems, which are solvable, I've solved them,
Sun solved them, are why you don't really want what ZFS did. It's non
ending case of wack a mole as the code evolves and someone slips in
something that makes the page cache and the ZFS cache incoherent again.
There isn't a pleasant way to make this stuff work, that's exactly
why Sun made everything live in the page cache, there was only one
copy of any chunk of data.
Which makes it baffling to me that Sun would allow ZFS into the kernel
but I guess the benefits were perceived to outweigh the ongoing work to
make the caches coherent. Personally, I think just doing it right is
way easier.
--lm
6:17 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
Ignoring the page cache and make their own cache has big problems.
You can mmap() ZFS files and doing so means that when a page is referenced
it is copied from the ZFS cache to the page cache. That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing. And you can keep track of
mapped pages and read/write from them if necessary even if you have
a separate cache for any compressed pages. I haven’t read zfs code
but this doesn’t seem like a tricky problem.
3:18 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
On Feb 4, 2021, at 4:33 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
The entire point of the SunOS 4.0 VM system was that the page you
saw via mmap(2) is the exact same page you saw via read(2). It's
the page cache, it has page sized chunks of memory that cache
file,offset pairs.
There is one, and only one, copy of the truth. Doesn't matter how
you get at it, there is only one "it".
ZFS broke that contract and that was a step backwards in terms of
OS design.
Ignoring the page cache and make their own cache has big problems.
You can mmap() ZFS files and doing so means that when a page is referenced
it is copied from the ZFS cache to the page cache. That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing.
7:16 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Fri, Feb 5, 2021, 7:19 AM Larry McVoy <lm(a)mcvoy.com> wrote:
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah
wrote:
The entire point of the SunOS 4.0 VM system was that the page you
saw via mmap(2) is the exact same page you saw via read(2). It's
the page cache, it has page sized chunks of memory that cache
file,offset pairs.
There is one, and only one, copy of the truth. Doesn't matter how
you get at it, there is only one "it".
ZFS broke that contract and that was a step backwards in terms of
OS design.
The double copy is the primary reason we don't use it to store videos we
serve. It's a performance bottleneck as well.
And fixing it is... rather involved... possible, but a lot of work to teach
the ARC about the buffer cache or the buffer cache about the ARC...
But for everything else I do, I accept the imperfect design because of all
the other features it unlocks.
Warner
On Feb 4, 2021, at 4:33 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
> Ignoring the page cache and make their own
cache has big problems.
> You can mmap() ZFS files and doing so means that when a page is
referenced
it is
copied from the ZFS cache to the page cache. That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing. 
7:21 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
I think hearing *why* they did that would be interesting.
On Fri, Feb 5, 2021 at 6:19 AM Larry McVoy <lm(a)mcvoy.com> wrote:
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
On Feb 4, 2021, at 4:33 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
The entire point of the SunOS 4.0 VM system was that the page you
saw via mmap(2) is the exact same page you saw via read(2). It's
the page cache, it has page sized chunks of memory that cache
file,offset pairs.
There is one, and only one, copy of the truth. Doesn't matter how
you get at it, there is only one "it".
ZFS broke that contract and that was a step backwards in terms of
OS design.
Ignoring the page cache and make their own cache has big problems.
You can mmap() ZFS files and doing so means that when a page is referenced
it is copied from the ZFS cache to the page cache. That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing.
6 Feb
6 Feb
1:03 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Feb 5, 2021, at 6:18 AM, Larry McVoy <lm(a)mcvoy.com> wrote:
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
Let me repeat a part of my response you cut out:
And you can keep track of mapped pages and read/write from them if
necessary even if you have a separate cache for any compressed pages.
In essence you pass the ownership of a page's data from a compressed
page cache to the mapped page. Just like in processor cache coherence
algorithms there is one source of truth: the current owner of a cached
unit (line or page or whatever). In other words, the you see via mmap(2)
will be the exact same page you will see via read(2). Not having actually
tried this I may have missed corner cases + any practical considerations
complicating things but *conceptually* this doesn't seem hard.
Warner mentions not using ZFS for its double copying. May be omething
like the above can a step in the direction of integrating the caches?
As Ron says, I too would like to hear what the authors of ZFS have to say....
On Feb 4, 2021, at 4:33 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
The entire point of the SunOS 4.0 VM system was that the page you
saw via mmap(2) is the exact same page you saw via read(2). It's
the page cache, it has page sized chunks of memory that cache
file,offset pairs.
There is one, and only one, copy of the truth. Doesn't matter how
you get at it, there is only one "it".
ZFS broke that contract and that was a step backwards in terms of
OS design.
Ignoring the page cache and make their own cache has big problems.
You can mmap() ZFS files and doing so means that when a page is referenced
it is copied from the ZFS cache to the page cache. That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing.
3:06 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Fri, Feb 5, 2021 at 7:04 PM Bakul Shah <bakul(a)iitbombay.org> wrote:
On Feb 5, 2021, at 6:18 AM, Larry McVoy
<lm(a)mcvoy.com> wrote:
The entire point of the SunOS 4.0 VM system was that the page you
saw via mmap(2) is the exact same page you saw via read(2). It's
the page cache, it has page sized chunks of memory that cache
file,offset pairs.
There is one, and only one, copy of the truth. Doesn't matter how
you get at it, there is only one "it".
ZFS broke that contract and that was a step backwards in terms of
OS design.
Let me repeat a part of my response you cut out:
And you can keep track of mapped pages and read/write from them if
necessary even if you have a separate cache for any compressed pages.
In essence you pass the ownership of a page's data from a compressed
page cache to the mapped page. Just like in processor cache coherence
algorithms there is one source of truth: the current owner of a cached
unit (line or page or whatever). In other words, the you see via mmap(2)
will be the exact same page you will see via read(2). Not having actually
tried this I may have missed corner cases + any practical considerations
complicating things but *conceptually* this doesn't seem hard.
In essence, that's what the merged page/buffer cache is all about: file IO
(read/write) operations are satisfied from the same memory cache that backs
up VM objects. I agree that conceptually it's not that complex; but that's
not what ZFS does.
Of course the original Unix buffer cache didn't do that either, because no
one was mmap'ing files on the PDP-11, let alone the PDP-7. A RAM-based
buffer cache for blocks as the nexus around which the system serialized
access to the disc-resident filesystem sufficed. When virtual address
spaces got bigger (starting on the VAX) and folks wanted to start being
more clever with marrying virtual memory and IO, you had an impedance
mismatch with a fairly large extant kernel that had developed not taking
into consideration memory-mapped IO to/from files. Sun fixed this, at what
I take to be great expense (I followed keenly the same path of development
in *BSD and Linux at the time and saw how long that took, so I believe
this). But then the same Sun broke it again.
Warner mentions not using ZFS for its double copying. May be omething
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
> On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
>>
>> Ignoring the page cache and make their own cache has big problems.
>> You can mmap() ZFS files and doing so means that when a page is
referenced
it is copied from the ZFS cache to the page cache.
That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.
Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing. like the above can a step in the direction of
integrating the caches?
But the cache was integrated! Until it wasn't again....
As Ron says, I too would like to hear what the authors of ZFS have to
say....
Sounds like they thought it was too hard because compression means the
place on storage where an offset in a file lands is no longer a linear
function of the file's contents. Presumably the compressed contents are not
kept in RAM in any of the caches (aside from a temporary buffer to which
the compressed contents are read or something).
- Dan C.
4:01 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Feb 5, 2021, at 6:06 PM, Dan Cross
<crossd(a)gmail.com> wrote:
On Fri, Feb 5, 2021 at 7:04 PM Bakul Shah <bakul(a)iitbombay.org
<mailto:bakul@iitbombay.org>> wrote:
And you can keep track of mapped pages and read/write from them if
necessary even if you have a separate cache for any compressed pages.
In essence you pass the ownership of a page's data from a compressed
page cache to the mapped page. Just like in processor cache coherence
algorithms there is one source of truth: the current owner of a cached
unit (line or page or whatever). In other words, the you see via mmap(2)
will be the exact same page you will see via read(2). Not having actually
tried this I may have missed corner cases + any practical considerations
complicating things but *conceptually* this doesn't seem hard.
In essence, that's what the merged page/buffer cache is all about: file IO
(read/write) operations are satisfied from the same memory cache that backs up VM objects.
I agree that conceptually it's not that complex; but that's not what ZFS does.
Good that we agree on that at least! This is why it would be good to hear from
Bonwick/Moore.
2:18 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
Write(2)ing to a mapped page sounds pretty dodgy.
Likely to get you
in trouble in any case. Similarly read(2)ing.
Uh, no. You misunderstand completely.
The purpose of the kernel is to provide a reliable interface to system
facilities, that lets processes NOT DEPEND on what other processes are
doing.
The decision about whether Tool X uses mmap() versus read() to access a
file, or mmap() versus write() to change one, is a decision that DOES
NOT DEPEND on what Tool Y is doing. Tools X and Y may have been written
by different groups in different decades. Tool X may have been written
to use stdio, which used read(). Three years later, stdio got rewritten
to use mmap() for speed, but that's invisible to the author of Tool X.
And maybe an end user in 2025 decides to use both Tool X and Tool Y on
the same file. So only much later will any malign interactions between
read/write and mmap actually be noticed by end users. And the fix is
not to create new dependencies between Tool X, stdio, and Tool Y. It is
to fix the kernel so they do not depend on each other!
Here is a real-life example from my own experience.
There is a long-standing bug in the Linux kernel, in which the inotify()
system call simply didn't work on nested file systems. This caused a
long-standing bug in Ubuntu, which I reported in 2012 here:
https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847
The symptom was that after booting from a LiveCD image, "apt-get
install" for system services (in my case an NFS client package) wouldn't
work. Turned out the system startup scripts used inotify() to notice
and start newly installed system services. The root cause was that
inotify failed because the root file system was an "overlayfs" that
overlaid a RAMdisk on top of the read-only LiveCD file system. The
people who implemented "overlayfs" didn't think inotify() was important,
or they thought it would be too much work to make it actually meet its
specs, so they just made it ignore changes to the files in the overlaid
file system. So the startup daemon's inotify() would never report the
creation of new files about the new services, because those files were
in the overlaying RAM disk, and so it would not start them and the user
would notice the error.
The underlying overlayfs bug was reported in 2011 here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147
As far as I know it has never been fixed. (The bug report was
closed in 2019 for one of the usual bogus reasons.)
The problem came because real tools (like systemd, or the tail command)
actually started using inotify, assuming that as a well documented
kernel interface, it would actually meet its specs. And because a
completely unrelated other real tool (like the LiveCD installer)
actually started using overlayfs, assuming that as a well documented
kernel interface, it too would actually meet its specs. And then one
day somebody tried to use both those tools together and they failed.
That's why telling people "Don't use mmap() on the same file that you
use read() on" is an invalid attitude for a Real Kernel Maintainer.
Props to Larry McVoy for caring about this. Boos to the Linux
maintainers of overlayfs who didn't give a shit.
John
2:43 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
When I was working for a big chip testing company, a STDIO vs MMAP problem came up.
The tester was at it’s heart a SPARC VME system running Solaris. The tester read in
‘patterns’
from disk, it it literally took hours to read in all the test patterns. At the scale of
the large chip vendors,
every minute you can’t test because the tester is booting, etc means dollars are lost.
We wrote a bunch of macros that replaced the STDIO file system I/O calls with the
equivalent
mmap calls. . It turns out STDIO does a lot of prefetching and has some assumptions that
you’re
going to read a file linearly from beginning to end, whereas we wanted to jump around a
lot in the pattern files.
Pattern loading went from 4 hours to 30 minutes. Our customer was ecstatic.
Joe McGuckin
ViaNet Communications
joe(a)via.net
650-207-0372 cell
650-213-1302 office
650-969-2124 fax
On Feb 5, 2021, at 5:18 PM, John Gilmore
<gnu(a)toad.com> wrote:
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
Write(2)ing to a mapped page sounds pretty dodgy.
Likely to get you
in trouble in any case. Similarly read(2)ing.
Uh, no. You misunderstand completely.
The purpose of the kernel is to provide a reliable interface to system
facilities, that lets processes NOT DEPEND on what other processes are
doing.
The decision about whether Tool X uses mmap() versus read() to access a
file, or mmap() versus write() to change one, is a decision that DOES
NOT DEPEND on what Tool Y is doing. Tools X and Y may have been written
by different groups in different decades. Tool X may have been written
to use stdio, which used read(). Three years later, stdio got rewritten
to use mmap() for speed, but that's invisible to the author of Tool X.
And maybe an end user in 2025 decides to use both Tool X and Tool Y on
the same file. So only much later will any malign interactions between
read/write and mmap actually be noticed by end users. And the fix is
not to create new dependencies between Tool X, stdio, and Tool Y. It is
to fix the kernel so they do not depend on each other!
Here is a real-life example from my own experience.
There is a long-standing bug in the Linux kernel, in which the inotify()
system call simply didn't work on nested file systems. This caused a
long-standing bug in Ubuntu, which I reported in 2012 here:
https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847
The symptom was that after booting from a LiveCD image, "apt-get
install" for system services (in my case an NFS client package) wouldn't
work. Turned out the system startup scripts used inotify() to notice
and start newly installed system services. The root cause was that
inotify failed because the root file system was an "overlayfs" that
overlaid a RAMdisk on top of the read-only LiveCD file system. The
people who implemented "overlayfs" didn't think inotify() was important,
or they thought it would be too much work to make it actually meet its
specs, so they just made it ignore changes to the files in the overlaid
file system. So the startup daemon's inotify() would never report the
creation of new files about the new services, because those files were
in the overlaying RAM disk, and so it would not start them and the user
would notice the error.
The underlying overlayfs bug was reported in 2011 here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147
As far as I know it has never been fixed. (The bug report was
closed in 2019 for one of the usual bogus reasons.)
The problem came because real tools (like systemd, or the tail command)
actually started using inotify, assuming that as a well documented
kernel interface, it would actually meet its specs. And because a
completely unrelated other real tool (like the LiveCD installer)
actually started using overlayfs, assuming that as a well documented
kernel interface, it too would actually meet its specs. And then one
day somebody tried to use both those tools together and they failed.
That's why telling people "Don't use mmap() on the same file that you
use read() on" is an invalid attitude for a Real Kernel Maintainer.
Props to Larry McVoy for caring about this. Boos to the Linux
maintainers of overlayfs who didn't give a shit.
John
2:55 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Please see my followup message. My fault for mixing two separate things
(what a user should not do vs how the kernel can still provide coherence).
On Feb 5, 2021, at 5:18 PM, John Gilmore
<gnu(a)toad.com> wrote:
On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
Write(2)ing to a mapped page sounds pretty dodgy.
Likely to get you
in trouble in any case. Similarly read(2)ing.
Uh, no. You misunderstand completely.
The purpose of the kernel is to provide a reliable interface to system
facilities, that lets processes NOT DEPEND on what other processes are
doing.
The decision about whether Tool X uses mmap() versus read() to access a
file, or mmap() versus write() to change one, is a decision that DOES
NOT DEPEND on what Tool Y is doing. Tools X and Y may have been written
by different groups in different decades. Tool X may have been written
to use stdio, which used read(). Three years later, stdio got rewritten
to use mmap() for speed, but that's invisible to the author of Tool X.
And maybe an end user in 2025 decides to use both Tool X and Tool Y on
the same file. So only much later will any malign interactions between
read/write and mmap actually be noticed by end users. And the fix is
not to create new dependencies between Tool X, stdio, and Tool Y. It is
to fix the kernel so they do not depend on each other!
Here is a real-life example from my own experience.
There is a long-standing bug in the Linux kernel, in which the inotify()
system call simply didn't work on nested file systems. This caused a
long-standing bug in Ubuntu, which I reported in 2012 here:
https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847
The symptom was that after booting from a LiveCD image, "apt-get
install" for system services (in my case an NFS client package) wouldn't
work. Turned out the system startup scripts used inotify() to notice
and start newly installed system services. The root cause was that
inotify failed because the root file system was an "overlayfs" that
overlaid a RAMdisk on top of the read-only LiveCD file system. The
people who implemented "overlayfs" didn't think inotify() was important,
or they thought it would be too much work to make it actually meet its
specs, so they just made it ignore changes to the files in the overlaid
file system. So the startup daemon's inotify() would never report the
creation of new files about the new services, because those files were
in the overlaying RAM disk, and so it would not start them and the user
would notice the error.
The underlying overlayfs bug was reported in 2011 here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147
As far as I know it has never been fixed. (The bug report was
closed in 2019 for one of the usual bogus reasons.)
The problem came because real tools (like systemd, or the tail command)
actually started using inotify, assuming that as a well documented
kernel interface, it would actually meet its specs. And because a
completely unrelated other real tool (like the LiveCD installer)
actually started using overlayfs, assuming that as a well documented
kernel interface, it too would actually meet its specs. And then one
day somebody tried to use both those tools together and they failed.
That's why telling people "Don't use mmap() on the same file that you
use read() on" is an invalid attitude for a Real Kernel Maintainer.
Props to Larry McVoy for caring about this. Boos to the Linux
maintainers of overlayfs who didn't give a shit.
John
5 Feb
5 Feb
9:50 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Thu, 4 Feb 2021, Will Senn wrote:
ZFS needn't be compressed, and I don't
generally do compression or
encryption unless required by law, so I can't speak from personal
experience on those use cases (others, far more experienced can). I do
know that it's truly a pain to recover from issues with either.
[...]
Thanks; I'd heard that ZFS was a compressed file system, so I stopped
right there (I had lots of experience in recovering from corrupted RK05s,
and didn't need any more trouble).
Ah, the RK05 - evil incarnate. I mean, a disk drive exposed to the air?
Out There Somewhere [tm] is a picture of a human hair compared with the
head clearance; yikes!
Once a month, our DEC ginger-beer[*] would PM our 40s, and I was most
perturbed to learn that the sole job of the internal NiCd battery was to
drag those heads back in the event of a power failure; what, you were
writing at the time? Too bad...
[*]
His surname was "Roth", so naturally his nickname was "Portnoy".
-- Dave
6 Feb
6 Feb
1:21 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Dave Horsfall <dave(a)horsfall.org> writes:
On Thu, 4 Feb 2021, Will Senn wrote:
[snip]
I have some real world experience with ZFS and bad blocks. As mentioned
by others, compression is not required, but it would not matter too
much. ZFS is somewhat odd in that you can mount and use a damaged
filesystem without too much trouble and recover anything possible. A
real example (and to keep it a TUHS topic), we had an older Solaris 10
Sparc at the $DAYJOB about two years ago that developed bad blocks on
the drive (ZFS detected this for us). Not a surprise given the age of
the system, being 6 to 8 years old at the time, we replaced the drive
without issue and started a re-silver (ZFS's version of fsck and raid
rebuild) and during that re-silver another drive developed block errors.
This was a RAIDZ1 set up, so having two drives go out at the same time
would cause something to be lost. If it were another RAID variant, it
would probably have been fatal. What ZFS did was told us exactly which
file was effected by the bad block and the system kept going, no
unmounts, reboots or down time. We replaced the second drive and the
only problem we had was a single lost file. Compression would not have
changed this situation much, except perhaps making the problem a bit
bigger, we may have lost two files or something. I have another example
with a Sun NFS appliance that runs under the covers Solaris 11 that
developed multiple drive fails. In the end, we estimated that around
40% or so of the multi terabyte filesystem was damaged. It was possible
to still use the appliance and we were able to get a lot off of it as it
continued to function as best as it could you just could not access the
files that had bad blocks in them. ZFS has honestly been amazing in the
face of the problems I have seen.
--
Brad Spencer - brad(a)anduin.eldar.org - KC8VKS - http://anduin.eldar.org
ZFS needn't be compressed, and I don't
generally do compression or
encryption unless required by law, so I can't speak from personal
experience on those use cases (others, far more experienced can). I do
know that it's truly a pain to recover from issues with either.
[...]
Thanks; I'd heard that ZFS was a compressed file system, so I stopped
right there (I had lots of experience in recovering from corrupted RK05s,
and didn't need any more trouble). 
3:22 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Fri, Feb 5, 2021 at 12:51 PM Dave Horsfall <dave(a)horsfall.org> wrote:
[...]
Thanks; I'd heard that ZFS was a compressed file system, so I stopped
right there (I had lots of experience in recovering from corrupted RK05s,
and didn't need any more trouble).
That's funny, for me this is the main reason to use ZFS... What really sets
ZFS apart from everything else is the lack of trouble and its resilience to
failures. We used to have lots and lots of ZFS filesystems at work, and
I've been using ZFS exclusively at home ever since. I have run into a
non-importable ZFS file system (all drives are there, but it's corrupt and
won't import) only once, and was able to fix it with zdb.
ZFS compression is completely optional, and not even on by default. I've
only tried it once and found it cost too much performance on something
that's not very fast to begin with, but I don't think it affects data
recovery much (the way ZFS stripes data makes traditional data recovery
tools pretty useless anyway).
I personally don't care about purity of implementation, because everything
is a trade-off. The argument really reminds me of Tanenbaums criticism of
the Linux Monokernel (was Tanenbaum right? Maybe, but who cares, because
Linux took over the world, and Minix didn't, so from a practical point of
view, Linus was right). The other one it reminds me of is the criticism of
TCPs "blatant layering violation" (vs OSI). But IMHO the critics were just
jealous of the cool things they couldn't do because they needed to respect
the division of labor along those pesky layers.
I remember reading on one of the Sun engineers blogs (remember when Sun
allowed their engineers to keep blogs about Solaris development? Good
times!) about the heated discussions they had over the ARC and bypassing
the page cache. I don't remember the actual arguments for it, but it was
certainly not a decision that was made out of laziness.
Performance wise, ZFS is not the best, and if that's all you care about,
there are better options. It needs a lot of tuning to just reach
"acceptable" and it definitely does not play well with doing other stuff on
the same machine (it pretty much assumes that your storage appliance is
dedicated). It has particularly abysmal performance when you do lots of
small random writes and then try to read that back in order, but if you
care about not losing your data, it's 2nd to none.
In $JOB-1 (almost 15 years ago), we spent a few weeks stress testing ZFS.
The setup was 24x4TB SATA drives, divided into 2 12 drive raidz2 vdevs or
something like that. All tests were done while it was busy reading/writing
checksummed test files at full speed, 1GB/s or so (see? Performance was not
impressive. We definitely got a lot more out of that with UFS). What was
absolutely stunning was the fact that in all our tests it never served one
bit of corrupted data. It either had it, or it returned an error.
We tortured the storage in any way we could imagine. Wiggled the cables,
yanked out drives, used dd to overwrite random parts or entire drives,
smashed a drive with a hammer and put it back in, put in drives of the
wrong size, put in known bad drives, yanked out drives while it was
resilvering, put drives back into a different slot, overwrote stuff while
it was resilvering. Unplugged the entire storage, plugged the storage into
another machine and imported it, plugged the drives back into the first
machine in a different order. We even did things like "copy a drive onto a
spare with dd, remove 3 drives, and then substitute the spare drive for the
removed one" (this led to some data loss because making the copy was not
atomic, but most of the data was recoverable). And no matter what we did,
it just kept going unless the data was simply not there, and even then, it
kept serving the files (or parts of files) that were available, and
indicated exactly which files were affected by data loss. And when you put
the drives back (or restored the overwritten parts), it would continue as
if nothing had ever happened.
If you've ever wrestled a hardware RAID controller, or VxFS/JFS/HPFS, or
mdadm, you know that none of that can be taken for granted, and that doing
any of the stupid things mentioned above would most likely lead to complete
data loss and/or serving lots of random corrupted data and no way to tell
what had been corrupted.
I remember some performance issues with mmap, but I don't remember how we
fixed it. Probably just sucked it up. Using ZFS was not for maximum
performance.
3:55 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Fri, Feb 05, 2021 at 06:22:32PM -0800, Rico Pajarola wrote:
On Fri, Feb 5, 2021 at 12:51 PM Dave Horsfall
<dave(a)horsfall.org> wrote:
I'm gonna call Bill tomorrow and get his take again, that's Bill Moore
one of the two main guys who did ZFS.
This whole thread is sort of silly. There are the users of ZFS who love
it for what it does for them. I have no argument with them. Then there
are the much smaller, depressingly so, group of people who care about OS
design that think ZFS took a step backwards.
I think Dennis might have stepped in here, if he was still with us, and
had some words.
I think Dennis would have brought us back to lets talk about the kernel
and what is right. ZFS is useful, no doubt, but it is not right from
a kernel guy's point of view.
I miss Dennis.
Thanks; I'd heard that ZFS was a compressed
file system, so I stopped
right there (I had lots of experience in recovering from corrupted RK05s,
and didn't need any more trouble).
That's funny, for me this is the main reason to use ZFS... What really sets
ZFS apart from everything else is the lack of trouble and its resilience to
failures.
4:07 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 2/5/21 8:55 PM, Larry McVoy wrote:
I'm gonna call Bill tomorrow and get his take
again, that's Bill Moore
one of the two main guys who did ZFS.
This whole thread is sort of silly. There are the users of ZFS who love
it for what it does for them. I have no argument with them. Then there
are the much smaller, depressingly so, group of people who care about OS
design that think ZFS took a step backwards.
I think Dennis might have stepped in here, if he was still with us, and
had some words.
I think Dennis would have brought us back to lets talk about the kernel
and what is right. ZFS is useful, no doubt, but it is not right from
a kernel guy's point of view.
I miss Dennis.
Larry,
Now, after the last 50 emails or so on this topic, I get it :). At
least, I understand that technical decision were made in creating ZFS
that were likely ill considered, the impact of those changes dubbed
insignificant, or even possibly sound design principles ignored. It's
debatable whether or not these decisions were deliberately contrary to
good OS design, but I appreciate your and the other experts hanging in
there and explaining your perspectives. I'm a systems guy, so a lot of
the detailed OS discussions go over my head, but after enough of them,
it kinda makes sense. At least, now, I'll pay closer attention to the
ZFS developer list discussions :).
Will
27 Feb
27 Feb
9:54 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
Hi Larry et al,
Just curious about this: was there any feedback from Jeff Bonwick and/or
Bill Moore re the ARC -vs- page cache?
Or would any of the design notes document the reasoning behind the decision?
Surely it must have come up and been justified or got an exception in the
Solaris architecture review (SARC "20 Q's", wasn't it called?) Since
AFAICS
it affected Solaris O/S interface (former-)guarantees. Although those notes
are probably lost / inaccessible now...
There's also the monthly OpenZFS leadership meeting, Matt Ahrens et al are
in there: I wonder if they would have access to some of the original
reasoning; how it was justified / why it was permitted.
Dave, btw: check out the high-level structure of ZFS metadata -- every
block is checksummed, and the checksum kept in the parent block (i.e. *not*
kept together), applicable for both data and metadata blocks, and at least
two copies are kept of metadata (but you can request more depending on your
paranoia, see also "ditto" blocks). Compression is optional at the
filesystem level (not held at the pool aka volume level; a pool may contain
multiple filesystems), when compression is enabled if affects future
created files, same if unset or changed to another algorithm; the
filesystem handles a mix of files (blocks, even; I forget offhand) existing
with various or no compression.
Rgds, Stuart.
On Sat, 6 Feb 2021 at 10:56, Larry McVoy <lm(a)mcvoy.com> wrote:
On Fri, Feb 05, 2021 at 06:22:32PM -0800, Rico
Pajarola wrote:
sets
On Fri, Feb 5, 2021 at 12:51 PM Dave Horsfall
<dave(a)horsfall.org> wrote:
> Thanks; I'd heard that ZFS was a compressed file system, so I stopped
> right there (I had lots of experience in recovering from corrupted
RK05s,
and
didn't need any more trouble).
That's funny, for me this is the main reason to use ZFS... What really ZFS apart from everything else is the lack of
trouble and its resilience
to
failures.
I'm gonna call Bill tomorrow and get his take again, that's Bill Moore
one of the two main guys who did ZFS.
This whole thread is sort of silly. There are the users of ZFS who love
it for what it does for them. I have no argument with them. Then there
are the much smaller, depressingly so, group of people who care about OS
design that think ZFS took a step backwards.
I think Dennis might have stepped in here, if he was still with us, and
had some words.
I think Dennis would have brought us back to lets talk about the kernel
and what is right. ZFS is useful, no doubt, but it is not right from
a kernel guy's point of view.
I miss Dennis.
6 Feb
6 Feb
5:55 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Fri, Feb 5, 2021 at 3:50 PM Dave Horsfall <dave(a)horsfall.org> wrote:
Ah, the RK05 - evil incarnate. I mean, a disk drive
exposed to the air?
Out There Somewhere [tm] is a picture of a human hair compared with the
head clearance; yikes!
IIRC, the picture showed that even a smoke particle or a fingerprint is
bigger than the gap, never mind the hair.
Once I had to take a PDP-11 accounting application from $EMPLOYER's NJ
office to a customer in KC. It fit on a single RK, so I took two identical
disks just in case. This was in the late 70s. They were already X-raying
carry-on luggage then, and I had a Bad Feeling about what might happen, so
I said no.
"No problem. Does it open up?"
"Well, yes, but dust will ruin it."
"Well, what should we do?"
"Trust me?"
"Well -- okay. Go ahead."
Of course when I got there both RKs were scrambled completely, so I took
the next plane home. The next weekend, a colleague took the application on
the same flight. On floppies. Floppy floppies. Lots and lots of them.
It installed fine. Of course it took him all day and much of the following
day.
John Cowan http://vrici.lojban.org/~cowan cowan(a)ccil.org
You cannot enter here. Go back to the abyss prepared for you! Go back!
Fall into the nothingness that awaits you and your Master. Go! --Gandalf
4 Feb
4 Feb
8:46 a.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
For what it's worth, you don't *have* to use compression on ZFS.
But everything still goes through the ARC, which is ... messy.
And eats memory for breakfast and then more memory for snacks and
lunch and more snacks and so on. Fortunately memory is cheap, if
you have a modern box. Unfortunately, I still don't, yet.
ZFS has a ton of stuff in it. That, also, is messy, and not a
great thing in terms of kernel size and security and alacrity and
so on. But it has some really cool ideas in it. I'm perfectly
happy to use it, or will be once I build a new box (still haven't
made the jump to an AMD system with ECC).
Chris
4:47 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On 2/4/21 1:46 AM, Chris Torek wrote:
For what it's worth, you don't *have* to use
compression on ZFS.
But everything still goes through the ARC, which is ... messy.
And eats memory for breakfast and then more memory for snacks and
lunch and more snacks and so on. Fortunately memory is cheap, if
you have a modern box. Unfortunately, I still don't, yet.
ZFS has a ton of stuff in it. That, also, is messy, and not a
great thing in terms of kernel size and security and alacrity and
so on. But it has some really cool ideas in it. I'm perfectly
happy to use it, or will be once I build a new box (still haven't
made the jump to an AMD system with ECC).
Chris
OMG! It's a memory hog, for sure, and messy :)
11 Feb
11 Feb
10:01 p.m.
New subject: FreeBSD behind the times? (was: Favorite unix design principles?)
On Sat, Jan 30, 2021 at 06:25:00PM -0800, Larry McVoy wrote:
BitKeeper has that code and proves that it can be
done.
I'm still using BitKeeper and enjoing it a lot.
I like it much better than Git. :-)
So good on you that you like ZFS and FreeBSD. I
don't and I don't for
really good reasons.
I like FreeBSD, and I like ZFS (as a user, without knowing the
internals), but I like Btrfs much better. I wish it was supported by
the BSD's, mainly OpenBSD.
What are your views on BTRFS, Larry? I'd like to know. :-)
Regards.
Ángel
31 Jan
31 Jan
12:09 a.m.
New subject: Favorite unix design principles?
On Sat, Jan 30, 2021 at 4:29 PM Clem Cole <clemc(a)ccc.com> wrote:
These days, Apple no longer cares about Mac OS because
iOS is where they
now put their effort, although I'm not super impressed there either, but I
also don't push it like I do Mac OS. Sad really. If I could get the
day-2-day applications that I need to work on FreeBSD, I suspect I would be
there in a heartbeat.
Decades ago I worked out the lifecycle of tech companies:
1) Engineering-driven: the goal is to make and sell high-quality products.
DEC and HP were like this for a long time.
2) Sales-driven: the goal is to sell products, high-quality or not. Too
many examples to specify.
3) Finance-driven: the goal is to make money, whether you sell products or
not. The classic case here is Carnegie Steel. When Carnegie told his
direct reports they were going into the railroad business and they
protested that the company knew nothing about it, he said "Carnegie Steel
isn't about making steel, it's about making money, and anyone who forgets
that is fired."
4) Survival-driven: the goal is to keep the company going whether you make
money or not. Auto companies just after bailouts are in this step. In
particular, Chrysler was bailed out in 1979 and again in 2009: see Tom
Paxton's song "I'm Changing My Name To Chrysler" (covered by Arlo and
Pete
on _Precious Friends_).
On Sat, Jan 30, 2021 at 4:43 PM Dave Horsfall <dave(a)horsfall.org> wrote:
You had *three* RK05s, an RK07, *and* a DH11? We had just two, and a
crappy DJ-11 on our /40s. Pretty much stable; any
bugs were fixed on the
spot because we had the source code after all.
I cut my teeth on a PDP-8 with a single TU58 DECtape and a TD8/E controller
(no interrupts, no data break cycles; I/O was more or less analogous to
terminal I/O). Uphill both ways in the snow.
John Cowan http://vrici.lojban.org/~cowan cowan(a)ccil.org
So they play that [tune] on their fascist banjos, eh?
--Great-Souled Sam
12:22 a.m.
New subject: Favorite unix design principles?
John Cowan writes:
Decades ago I worked out the lifecycle of tech companies:
1) Engineering-driven: the goal is to make and sell high-quality products.
DEC and HP were like this for a long time.
2) Sales-driven: the goal is to sell products, high-quality or not. Too
many examples to specify.
3) Finance-driven: the goal is to make money, whether you sell products or
not. The classic case here is Carnegie Steel. When Carnegie told his
direct reports they were going into the railroad business and they
protested that the company knew nothing about it, he said "Carnegie Steel
isn't about making steel, it's about making money, and anyone who forgets
that is fired."
4) Survival-driven: the goal is to keep the company going whether you make
money or not. Auto companies just after bailouts are in this step. In
particular, Chrysler was bailed out in 1979 and again in 2009: see Tom
Paxton's song "I'm Changing My Name To Chrysler" (covered by Arlo and
Pete
on _Precious Friends_).
Don't want to get too off topic here, but I consider myself fortunate
to have spent the prime of my career working on #1. I always believed
that if one made high quality-products that met a customer's need that
the rest would take care of itself. Yeah, I'm naive. I see 2-4 as
indicative of Silicon Valley today where I don't think that I could work.
It's no longer about making great things, it's about making money, if if
absolutely necessary, making something to support that goal. If you took
the trouble to read the Microsoft antitrust trial depositions, you'd see
that this was exactly Bill Gate's MO. He would try to tank competition
through any means necessary except making a better product; that only
happened when everything else failed.
I've probably ranted about this before, but to me this is emblematic of
how America has lost its way. My big evil villians are Jack Welch and
Bill Gates. When I was growing up and kids were asked "What do you want
to be when you grow up?", the answer was always astronaut first, then
doctor, lawyer, engineer, and so on. Post Microsoft, the answer was "I
want to be as rich as Bill Gates." People just want the outcome without
having to do any of the work, and just want the results without the joy
of actual accomplishment. I find it sad.
Jon
1359
days inactive
1387
days old
60 comments
25 participants
participants (25)
-
Andy Kosela -
Angel M Alganza -
Angus Robinson -
Bakul Shah -
Brad Spencer -
Chris Torek -
Clem Cole -
Dan Cross -
Dave Horsfall -
Dermot Tynan -
George Michaelson -
Greg 'groggy' Lehey -
Henry Bent -
joe mcguckin -
John Cowan -
John Gilmore -
Jon Steinhart -
Larry McVoy -
Rico Pajarola -
ron minnich -
Steve Nickolas -
Stuart Remphrey -
Tyler Adams -
Warner Losh -
Will Senn