I got to thinking about the file system sync vs. order argument as a result of this interesting discussion.
Back in NJ in 1998 I had a 144-node PC cluster. It started with 80 FreeBSD nodes, and a few months later we add 64 Linux nodes. It was DARPA funded and we were looking at issues around clustering.
What we discovered, when we added the Linux cluster, was that building power was really terrible. We had not realized it with the freebsd cluster because it tended to cleanly recover from unplanned nasty power cycles. But the Linux cluster tended to always have a small number of nodes that did not get through fsck.
yeah, it's not that good a data point maybe, but we found it interesting.