I got to thinking about the file system sync vs. order
argument as a
result of this interesting discussion.
[...]
What we discovered, when we added the Linux cluster, was that building
power was really terrible. We had not realized it with the freebsd
cluster because it tended to cleanly recover from unplanned nasty power
cycles. But the Linux cluster tended to always have a small number of
nodes that did not get through fsck.
Soft updates was available in 1998. Were you running that?
Surprised it hasn't come up already in this discussion; I've been waiting
for someone to mention it.
http://www.mckusick.com/softdep/
There was some licensing issue with it in the early days: it was freely
available and free to use, but McKusick wanted a chance to try to sell it
to a commercial unix vendor as well. But he ended up BSD-licensing it a
couple years later.
The original README from the FreeBSD source tree:
https://svnweb.freebsd.org/base/release/3.0.0/contrib/sys/softupdates/READM…
Anyway, it solved the on-disk consistency problem and boosted performance
as well. I enabled it on all my systems.
-Jason