First Journaling FS for Linux

wendyW writes "LinuxPR has the press release from Namesys, announcing the stable release of the journaling version of ReiserFS. According to the press release, journaling wound up making it even faster than it already was. "

XFS, reiserfs, ext3fs

It looks like there are a lot of questions about other journalling filesystems. I'm no expert on these things, but I have spent quite a bit of time following all three projects and I've read through all available documents on the three filesystems. Here's what I understand of the three.

XFS
Originally made by SGI for their IRIX OS, XFS is one awesome filesystem. Read this white paper (http://www.sgi.com/Technology/xfs -whitepaper.html [sgi.com]). This white paper describes all of its cool features. The main features of XFS make it a super scalable, very reliable, ultra fast journalling filesystem utilizing many cool FS technologies like B-trees and other cool stuff.

Unfortunately, it seems that currently there are many problems with the Linux implementation of XFS. I don't know any details of this, but I guess it is safe to say that XFS will some day become available for Linux. This would be great.

ext3fs
I've only read about this in the linux mailing lists. ext3 appears to be a standard ext2fs implementation with journalling data, allowing backward compatibility with ext2, although one of the authors hinted that they may not make it backwards compatible in some later version. It is currently in super early alpha testing and definately not anywhere close to usable, stable and reliable.

In my opinion this project is very new, and holds much promise. From their README, they appear to be done basic journalling code, and what remains to be done is error handling contingencies, metadata only journalling, performance tuning and lots of other coding. As a result, it may take some time but this could hold much promise and give another viable option for a journalling FS for Linux. Choices are always good.

Ext3 Site - ftp://ftp.linux.org.uk/pub/linux/sct/f s/jfs/ [linux.org.uk]

Reiserfs - http://devlinux.com/namesys/ [devlinux.com]
I've been following reiserfs for a few months now. Its actually been available for quite some time now as a very stable, reliable and quick filesystem for Linux, but it was only recently when journalling was added to the code. Apparently this new addition is supposed to make it faster.

In "releasing" reiserfs, SuSE doesn't mean that it is the first journalling filesystem for Linux. It is the first journalling FS for Linux to be dubbed reliable and suitable for normal use. This is great as journalling has long been a stumbling block for enterprise adoption of Linux. Alan Cox hinted that he may include reiserfs in the standard kernels soon. Excellent =)

Warren Togami
[e-mail] [mailto]

Hmmm... What about the *BSDs?

I noticed that this code is released under the GPL. That means that the *BSD folks can't just take the code and incorporate it into their OS's.

There is a clause in the license that states that if you contact them, they will let you use it under a different license. But I can't imagine them putting it under the BSD license. It sounds like they want to control who can use it, and they've decided that GNU projects and commercial entities who pay are their target market. If they ever release it under a BSD license, then commercial entities could just grab the BSD-released copy and work from there.

Will the BSD's simply miss out on this nice new filesystem?

99 little bugs in the code, 99 bugs in the code,
fix one bug, compile it again...

Journaling File System

This would be a huge boon to those of us trying to truly break free of the commercial unices. I've had to put together quotes for enterprise quality database solutions before and there have always been a couple of hurdles to get past when considering an Intel/linux based system.

PostgreSQL works wonderfully with large data sets, but lacks the ability to do hot restores. I'm eagerly awaiting that one... Now that it does a much better job with concurrant locks, that's my only real hesitation at this point.

SMP has come a long way in a short time with linux, but is still a bit lacking. This makes it difficult to settle on Intel hardware - sometimes, you just need Raw Horsepower. I'd like to get there without having to buckle down and buy a Sun or HP box. I'm not worried about this one - things are coming along quite nicely...

Now, my last concern was journaling filesystems - and it looks like it's coming at long last! I was excited when the initial announcement was made, but now that the code is out (and Alan is even considering merging into the stable branch!), I'm all gushy inside! Let's hear it for our team!

I've watched this whole linux thing start out as a 'hobby OS' and develop through adolescence into what is becoming a damned serious contender with the big boys. Sure, they're baby steps at the moment, but at this pace, they add up right quick. God, I love this industry - never know what's gunna happen next. Who knows - maybe the government will sue Microsoft for anti-trust violations next. Oh... right...

Selling GPL Exceptions

This is not the first time software has been released under this model. My understanding is that this is how RT Linux was released.

[The idea of RT Linux is to put a small real time kernel underneath Linux. This kernel handles the real time tasks, and schedules Linux when a real time task doesn't require it. It also provides a communication mechanism between Linux processes and real time tasks.]

So the RT linux kernel could, in theory, be used without Linux (perhaps with another OS instead) to provide real time services. The author has carefully retained the copyright to his code, so he can sell it under a non-GPL license if someone wishes to incorporate it into a commercial project.

I'm not aware of any non-GPL licenses for RT Linux, but the model is there.

The main thing that helps make this model work is that the copyright holder controls the distribution. That means that in order to get your changes into the official releases, you have to resolve any copyright issues. It only breaks down if there is a significant dispute and someone is willing to go to the effort to start a separate distribution. Of course, if they get the file system into the main Linux distribution, that action will trigger a fork in development.

Wasn't ext3 first?

The ext3 journaling filesystem has its first beta a few months ago. It does't require you to reformat your existing ext2 partitions to convert to ext3. And an ext3 filesystem can still be used as an ext2 filesystem, you just need to update the journaling information if you go back to ext3 after using it as ext2. Read more about it at Stephen Tweedie's ext3 site [linux.org.uk].

Re:sgi's xfs?

The difference between the two are more of an academical than user-related issue, as it is basically in the way they are built up. As far as journaling goes, they are both up to the task.
I do not know if ReiserFS is a true 64 bit one, handling the files as big as the XFS does, but a quick and dirty look at the two FS's homepages should yield a lot more info on this.
XFS and ReiserFS is not going to replace ext2. Actually, ext3 is, which will, when released, also be a journaling FS (from what I heard).
Maybe someone could provide the right urls or more info on this than I can. I believe in time, they will all be included into the kernel, and you can choose your preference based on your needs. In the meantime, make a small partition, insmod the module and mount the drive and play with it I guess :)

Re:Deletion times

There is a semi recent benchmark vs ext2fs at http://name sys.botik.ru/~yura/benchmarks/journal_227/ext2_vs_ jour9.html [botik.ru]

Chris has the office next to mine and has been showing me these benchmarks just about every day - they improve just about every day.

-Jeff

And Suse is funding this.. cool

So Redhat pays for Alan (and Gnome?), Corel supports WINE, and Suse pays for file systems.

Open Source has always been good at producing excellent, relatively small and self-contained components. We haven't been so great (with a few very notable exceptions, the kernel being one) at producing large projects. If it's a lot of effort with no quick return, the coders get tired of it.

Now the commercial companies are funding the big stuff in an attempt to gain mindshare ("we must know what we're doing, we've got Alan"). This really complements the existing strengths of Open Source.

Re:What IS a journaling file system?

This sort of covers it:

http://collective.cpoint.net/lfs/ what_lfs_is.html [cpoint.net]

Aladdin Ghostscript vs ReiserFS

RMS had no problems with this business model, . . .

Actually, I hear that he's not thrilled with it. Indeed, one of the biggest problems that I can see is that there is very little incentive for people to improve the existing GPL version of Ghostscript when they know that Aladdin has (a) already improved Ghostscript in the current commercial version, and (b) will be releasing their changes 'soon' (after one year). This interview with Ghostscript's author Peter Deutsch [linuxcare.com] sheds more light on the situation, including Stallman's thoughts.

One result is that the GPL community is almost guaranteed to always be one year behind the latest in Ghostscript technology, unless someone gets up enough nerve to fork Ghostscript development and try to get ahead of Aladdin.

With Ghostscript the GPL was not restrictive enough. Proprietary software would simply call the gs executable in a separate process.

Part of the problem here is that the Aladdin folks try to license their code to printer manufacturers, etc. The printer folks aren't too keen on having to ship Ghostscript on demand to anyone who buys a printer. Also, if the printer folks make any platform specific changes (which undoubtedly they will, such as specific driver technology for running the print engine), they'd have to distribute those changes, and most aren't willing to do so.

Also, more importantly, Peter Deutsch doesn't seem too keen on having people ship Postscript-enabled printers by using his work for free (as in gratis).

The upshot: Aladdin offers their latest and greatest Ghostscript with a commercial license.

With ReiserFS, I'm sure a similar but not identical set of considerations exist. People building embedded or mission critical systems on an otherwise proprietary base might license ReiserFS for their application without introducing any questions as to the effects of GPL. At the same time, a GPL version is available for everyone.

The difference here is a bit subtle but important. Namesys appears to be releasing the latest and greatest ReiserFS under GPL, rather than imposing an artificial delay. (Whether or not this changes in the future is unclear, but for now it is an important distinction.) In this case, the commercial license seems to be a means for companies to buy an "unencumbered" version of ReiserFS for their own purposes. (By "unencumbered", I mean free of the implications of GPL.) I see this potentially as a way to keep both camps happy. Maybe. (Except, of course, RMS.)

--Joe
--

Important, but likely not for DBMSes

I hate to take issue with a well-spoken posting, but journalling is not of primary usefulness for helping support High Availability RDBMS systems.

The main effect of journalling, the thing that is really important about it, is that it guarantees that metadata updates are kept consistent. That is, journalling is primarily supportive of making sure that filenames, directory structures, permissions, and such are kept consistent even when moderately catastrophic things happen.

This is a really good thing when supporting file serving activities, as that indeed tends to involve lots of manipulations of files as users shift them around.

I've been on the ReiserFS mailing list since '97; have been running a personal news spool on a small ReiserFS partition for probably 6 months. I can't tell for sure if the journalling now available is metadata-only, or if it also journals normal data updates. It looks rather more like metadata-only, which is useful for file-server work, but not so much for RDBMSes.

Databases behave in quite different ways from file servers in terms of the way they do file access.

If you look at most RDBMSes, they create a few files, and do lots of manipulations on top of them. Informix SE is a counterexample, basically using Informix C-ISAM underneath, but is unusual in that regard. If you look at the database partitions, you get one of two things:

• Partitions containing a few very large files.

  Note that for these, the metadata is very static which means that journalling of metadata is of relatively little importance.

• Partitions containing no filesystem, but rather raw data being managed by the RDBMS.

  Don't just believe me; I am not the ultimate authority on this. Transaction Processing : Concepts and Techniques [amazon.com] is a rather definitive reference; it discusses methods of managing transactions in the context of database management systems, and goes into considerable detail discussing transaction logging, which bears striking (and not merely coincidental) resemblance to journalling.

  The critical point here is that it is the database manager that wants to manage the logging/journalling; Oracle and Sybase and IBM and Informix will be loathe to pass on responsibility for this to Hans Reiser, wonderful guy though he is.

Conclusions

1. Sorry, I have to disagree with you on ReiserFS being of fundamental importance to those doing serious database work.

   What will be of fundamental importance will be when Stephen Tweedy's Raw Device Support [lwn.net] gets integrated into the "production" kernels. That is what Oracle is looking for (consider: Oracle has pumped some funds into RHAT, and RHAT is paying Stephen Tweedie... Could there be some connection?)

2. Journalling IS important for sorts of applications that manipulate lots of files, which includes things like dynamic web serving and file serving.

   Even if this isn't such a boon to those doing serious RDBMS work, it can still be a boon to lots of other folks...

Not Even Close

I've got a filesystem that has been using ReiserFS for probably 6-8 months now, and Hans has been working on it since at least July 1997.

"Who was first" isn't all that important; it should be noted that there is considerable communication between the development groups, and there are conscious efforts ongoing to make sure they build facilities that will be useful across the board:

• The ReiserFS folks have been doing BTree "stuff," and intend to provide some code that should be usable by anyone wanting to do B-Trees at the kernel level, whether that be with ReiserFS, ext3, "ext4," or (and this has been explicitly mentioned) SGI's XFS.
• Considerable discussion has taken place in trying to coordinate needed modifications to kernel code in terms of:
  • VFS
  • Buffer management
  • Cache management
  It often enough turns out that what one group needs another finds that they also need.