gzipped_tar's Journal: Data safety is too important to be entrusted to FS alone

When I clicked on the "Write in Journal" link to write this ramblings down here, I noticed the pun in it, because this story is inspired by the recent posts on the EXT4 data losing matter.

There was a huge amount of finger-pointing, FUD, and groupthink-catering in the comments and I don't wish to repeat them here. Just some random points of interest I gathered by reading them.

• Does fsync() cripple the system?

  I expect fsync() to be a heavy operation for all filesystems except tmpfs because, well, it's side effects include (usually) the spinning up of disks. The performance (more accurately, temporal performance) is expected to be affected. However, "performance" is too vague a point to be discussed in this manner, and in real world programmers are paid because they are supposed to make reasonable, case-dependent trade-offs between stability and speed (i.e. the "performance" you hear a lot among the comments).

  To be specific, since fsync() gains points in data safety by sacrificing speed, and this is almost the case for all FSes, the reasonable and cross-platform tactic should be separating out the "precious, need fsync() whenever possible" data away from the "non-essential, don't have to worry about crashes in the middle" data, and use whatever heavy operations to ensure the former's safety (fsync, transactional database, or whatever shiny gadgets out there) while leaving the rest to be taken care of by the FS. By nature, you don't write to the "precious" data's storage very often like you do with other data, so the performance hit could be limited within the minimum necessary.

  In a word, using fsync != peppering the code with unnecessary fsync, and if you find yourself in a situation where you have to fsync() a lot, don't worry, it's time to show off your optimization skills ;)

• Are the KDE guys, and millions of app developers at fault?

  Probably yes, because their approach to the safety of the configuration file was not FS-independent, relying on undocumented features.

  However, to look at things the glass-half-full way, they stumbled upon this caveat just because they are trying to ensure data safety. Had they not even bothered with this "petty safety issue" from the beginning, they wouldn't be able to make this mistake, er, I mean discovery. After this affair we can expect the overall safety of user data under KDE improved, not degraded.

• Are the EXT4 guys, particularly TyTso, at fault?

  Probably not, though I'm not sure.

  I admit I may be biased on this point. I'm a user of both EXT3 and EXT4, and I tend to defend my choice. I think TyTso and other EXT4 developers are brilliant guys to which I own a lot.

  I think the solutin from Tso is sound, albeit not perfect. They are offering an upcoming patch which provides backward-compatibility. It's not perfect because it's an ad-hoc plug in the leak, but well, I don't know better than they do so it's not up to me to comment further.

• Is the POSIX standard at fault?

  I don't think so. The POSIX standard doesn't address this issue. Want filesystem transaction? Probably time for something else in parallel to POSIX (as long as it's an open standard not affected by corporate greed a la OOXML).

  Which, I think, renders Tso's "POSIX defence" more or less Chewbacca-like, but he did point out a technical problem which is taken for granted by many, and should not. By the "POSIX defence" he can't prove that EXT4 is feature-complete (but was he even trying to prove that?) while pointing out the problem of relying on non-standard features (see the KDE section above).

• Is this the death bell of EXT4? Is ZFS putting the final nail into the coffin of Linux?

  No and no. Heck, how did I even bother to answer this one?

• You don't have any bones. You are just keeping it moderate by using weasel words and a lot of "probably".

  Probably ;) but it doesn't matter that much as data safety ;)