Forgot your password?
typodupeerror

Practical File System Design with the Be File System 258

Posted by timothy
from the online-knowledge dept.
erikharrison writes "Dominic Giampaolo's Practical File System Design with the Be Filesystem has been around since 1999 - not exactly a new book. The book has been out of print for a time now, however, so Dominic made the book available in PDF form on his website. With this public release of the book, and the BeOS rising to join the ranks of OSs that won't die (hi Amiga!) it makes sense to take a look at what the book has to offer us today." Read on for the rest of Harrison's review below to see just what that is -- it covers a surprisingly broad range.
Practical File System Design with the Be File System
author Dominic Giampaolo
pages 227 pp
publisher MORGAN KAUFMANN PUBLISHERS, INC.
rating 8.5 of 10
reviewer erikharrison
ISBN 1558604979
summary Discusses implemeting a file system, using the Be file system as example

Table of Contents

  • Chapter 1 Introduction to the BeOS and BFS
  • Chapter 2 What Is a File System?
  • Chapter 3 Other File Systems
  • Chapter 4 The Data Structures of BFS
  • Chapter 5 Attributes, Indexing, and Queries
  • Chapter 6 Allocation Policies
  • Chapter 7 Journaling
  • Chapter 8 The Disk Block Cache
  • Chapter 9 File System Performance
  • Chapter 10 The Vnode Layer
  • Chapter 11 User-Level API
  • Chapter 12 Testing
  • Appendix A File System Construction Kit

First thing to note is that Giampaolo is not a great writer, nor is he a bad one. He does not have the gift that some tech writers have of making both an interesting technical document and a fun read. His style is very straightforward - introduce idea, explicate idea, summarize idea. On the other hand, he knows his topic inside and out, and has an obvious enthusiasm for the material, and a real talent for saying things simply without dumbing it down, and his occasional dry wit makes the book a surprisingly easy read.

Giampaolo is doing two things - discussing designing filesystems in general and documenting the Be filesystem. He does both well. BeFS has some advanced features - arbitrary metadata, attribute queries, and indexing. The desire to support these features influences the overall design of the system, but Giampaolo shows how changes to that design change implementation details. The result is a good overview of how a file system works, the trade-offs in optimizing for a particular usage pattern, and how to design one yourself.

The book can be roughly divided into three sections: the first is an overview of how filesystems work and some of the concepts that you encounter - extents, inodes, B-trees, superblocks, and the other standard pieces of a filesystem. Included in this early section is a good high-level overview of the design of five other file systems: BSD FFS, Linux's ext2, Macintosh HFS, Irix XFS, and Windows NT's NTFS. The coverage here strikes a proper balance between too much and too little information. Giampaolo prefers to show rather than to tell, and these filesystem overviews make the connection between design, performance, and features perfectly clear, and provide a solid background to talk about a specific implementation in detail - namely BeFS.

The second section is the bulk of the book - how to implement a filesystem from the ground up, leaning heavily on the BeFS implementation for examples. This is the most straightforward part of the book. Giampaolo covers a single issue in design and implementation in a "Here's the problem, here's and overview of possible solutions and their drawbacks, here's how I did it, now lets summarize" manner. Again, Giampaolo's style makes this an easy if somewhat dry read. As a filesystem and kernel ignoramus, I would have appreciated a slightly more detailed coverage of how all of the various data structures get to disk - how are they serialized, whether endianess is an issue, etc. The BeOS was pretty portable, running at one time or another on the AT&T Hobbit processor, PowerPC, and x86 - I would have liked to have seen portability issues discussed, however, BeFS wasn't written until after the move from the Hobbit to PowerPC, and the book was written prior to the move to x86, so the lack of coverage is reasonable.

Even considering the plain Jane style of this middle section, there are a few gems. The coverage of journaling is excellent, and while I've long understood journaling from a 10,000 foot perspective, this really made me understand the underlying concepts, combined with simple code snippets that helped understand implementation. The Allocation Policies chapter showed in clear terms that disk access is a major bottleneck, and filesystems have become very sophisticated in their optimizations.

The third section of the book deals with some of the more indirect concerns in implementing a file system; specifically, interacting with the kernel, designing a user level API and the major role of testing in filesystem development. This is the one place Giampaolo's writing shines. He really is a good teacher, and this section affords him the chance to talk about the broader perspective of OS design, and even recount a few war stories. For example, in terms of parentage, the BeOS has BSD and classic MacOS as its father and mother. In a few places, such as the Storage Kit API covered in chapter 11, this heritage shows some signs of less-than-seamless integration, and this offers Giampalo a chance to wax philosophical on the nature of OS design, company politics, and the pressure of shipping dates.

In short, the book lives up to it's title. The author is a pragmatist, and offers a clear roadmap for those who have a need to work with low level filesystem implementation. His emphasis on testing, careful optimization, and data structure protection not only helps to show the pitfalls of filesystem work, but also offers a Swiss army knife of techniques to dodge them. The book concludes with a short appendix which covers a file system construction kit, allowing a would-be implementor to begin work on his own filesystem safely without worrying about killing his hard disk. All in all, a solid read.


Here's a link to Practical File System Design with the Be File System as a PDF; you can also look for a used copy at Barnes & Noble. Slashdot welcomes readers' book reviews -- to see your own review here, carefully read the book review guidelines, then visit the submission page.

This discussion has been archived. No new comments can be posted.

Practical File System Design with the Be File System

Comments Filter:
  • by leandrod (17766) <l@@@dutras...org> on Monday May 10, 2004 @04:48PM (#9110519) Homepage Journal
    One more backwards-looking text... it simply ignores that the future is relational databases as a filesystem.
  • by Timesprout (579035) on Monday May 10, 2004 @04:50PM (#9110543)
    He does not have the gift that some tech writers have of making both an interesting technical document and a fun read.

    But for most people this is seriously dull subject matter. Oscar Wilde would stuggle to get a chuckle out of this stuff.
  • Mirrors: (Score:3, Insightful)

    by Anonymous Coward on Monday May 10, 2004 @04:51PM (#9110552)
    Mirrors of the PDF:

    http:/beos.spb.ru/program/105/practical-file-sys te m-design.pdf.zip

    http://users.aber.ac.uk/mmb9/data/practical-file -s ystem-design.pdf

    http://www.funtech.org/downloads/Temp/practical- fi le-system-design.pdf
  • by happyfrogcow (708359) on Monday May 10, 2004 @04:54PM (#9110574)
    and just where are you going to store that database? on another database acting as a filesystem? or will you have a db built into the OS, right next to your flight simulator, and supercolliding monkey accelerator?

  • by Anonymous Coward on Monday May 10, 2004 @04:57PM (#9110609)
    All you need is one!

    Surely all you need is love?

  • by spellraiser (764337) on Monday May 10, 2004 @05:03PM (#9110673) Journal

    Aren't there enough filesystems? All you need is one!

    Yes, and 640K is enough for anyone, and everything worth inventing has already been invented.

    But seriously, what's wrong with creating new things? A new filesystem just might be better than the current ones. Stranger things have happened. Much stranger things, in fact. Especially in Computer Science. Remember hierarchical databases? No? Well, they used to be all the rage back in the old days, before relational databases took over. These days, the idea of using any other database model seems ludicrous. However, even those might be superseded at some point in the future.

    It never hurts to keep an open mind you know...

  • Interesting (Score:3, Insightful)

    by 0xC0FFEE (763100) on Monday May 10, 2004 @05:07PM (#9110710)
    This seems like quite an interesting read. But what I wonder is if BeFS is encumbered by any patent... If so, who own them now, do they plan on enforcing it, etc. Would be oh so cool to have an open source BeFS implementation.
  • I love BeOS (Score:1, Insightful)

    by helf (759423) on Monday May 10, 2004 @05:07PM (#9110712)
    I actually use it as my main OS. I still have windows 2k loaded on a second partition incase i want to play some games. But I use BeOS for 99% of my computer needs.
  • Like a warm hug. (Score:2, Insightful)

    by hedon_elite (559044) on Monday May 10, 2004 @05:07PM (#9110716)
    Amiga is like being curled up in a rocking chair with your grandmother when you're 5 years old. Total comfort and contentment. Btw, the Amiga website just got Slashdotted. I bet its been a long time since that happened. Ray Akey's gonna be p*ssed =)
  • by Short Circuit (52384) <mikemol@gmail.com> on Monday May 10, 2004 @05:12PM (#9110749) Homepage Journal
    You know, I hope nobody ever tries a "one size fits all" approach with filesystems.

    Some are good for a specific purpose, like ISO9660. It's got error correction in it, so it's good for write-once removable media.

    Others are extremely widespread, like FAT. That's good for interoperability.

    Others support advanced permissions like ACLs. That's good for system security.
  • by leandrod (17766) <l@@@dutras...org> on Monday May 10, 2004 @05:19PM (#9110818) Homepage Journal
    >
    its metadata indexing capabilities (which are the beginnings of a relational model)

    No, they could be the beginnings of an implementation of the relational model.

    >
    the BeOS designers agree with you

    If they did they would have gone full ahead instead of taking half measures.

    >
    It's also interesting that the author spends quite a while discussing how difficult it is to do well (particularly performance-wise) and how they almost left it out (IIRC) and/or had to limit its scope.

    No one ever said data was easy. But implementors usually do this particular blunder because they never understood the relational model, confounding it with SQL.

    >
    I'll believe people who have actually tried to implement the technology in question over people who say others should do so

    No, I am not talking about their technology. I am talking about the relational model.

  • by Anonymous Coward on Monday May 10, 2004 @05:31PM (#9110911)
    How about Ext2 for /boot and ReiserFS for everything else on the primary hard drive? :)
  • by leandrod (17766) <l@@@dutras...org> on Tuesday May 11, 2004 @01:58PM (#9118766) Homepage Journal
    >
    'Relational Database' typically refers to something like DB2 or MSSQL that support features like foreign keys and referential integrity, allowing one to create relations between tables.

    There are some misunderstandings here.

    First, both IBM DB2 and MS SQL Server are based on the SQL standard -- actually IBM DB2 defined the standard --, and SQL is in frontal violation of serveral fundamental characteristics of the relational model. It does, however, incorporates some relational ideas and is generally saner than alternatives like XML, MV, OO DBs.

    Second, foreign keys are constraints to enforce referential integrity. This does not 'create relations between tables', it documents and enforces some relationships, and does that non-exclusively. The user is free to query the data and find any other relationships.

    Third, relation has a very definite mathematical meaning. It is the defining feature of the relational model, and corresponds to what SQL calls tables -- only that relations are sets, while tables are bags.

    >
    I don't understand how you are suggesting it be organized

    One can still have a hierarchy, but it would be just one way of accessing files, not the way. The position(s) of a file in the hierarchy would be just yet another attribute of it.

Consultants are mystical people who ask a company for a number and then give it back to them.

Working...