Submission + - Dreaming in Code
AdamBa writes: "Scott Rosenberg's new book Dreaming in Code chronicles the attempt by Mitch Kapor's Open Source Applications Foundation to produce a new Personal Information Manager, code-named Chandler. Beginning in the spring of 2002, Kapor gathered programmers together with the somewhat vague goal of producing a new piece of software inspired by Agenda, a Lotus product from the late 1980s. The new product would be cross-platform and open source; the other details were still to be determined.
Rosenberg's book sports the obligatory back cover blurb comparison to "The Soul of a New Machine". It's the latest in a long line that purport to be the worthy successor to Tracy Kidder's Pulitzer Prize winning 1979 tale of a computer project at Data General. Although the book is worth reading both for programmers and the rest of the world, it falls short of the master in a couple of ways.
First of all, unlike Kidder's heroes who eventually triumph over long odds, Chandler doesn't get finished (disclaimer: I work at Microsoft, which produces Outlook, a potential competitor to Chandler). It progresses at an excruciatingly slow pace, until after 3 years Rosenberg gives up and publishes the book. The product is still shuffling towards a 1.0 release; one of the best aspects of the book is that the participants are still out there, blogging and coding away. Chandler may well mirror the Mozilla project, getting bogged down at the start before gradually moving out of the "large pile of interesting code" phase at which the book bids it adieu, and eventually maturing to take over a part of the world; but right now it's unclear what the future holds.
More importantly for the story, Rosenberg does not spend much time delving into the motivations and inner feelings of his characters. Kidder's big advantage at Data General was that he was actually present during the project, rather than relying on the after-the-fact interviews that guide many similar books. Rosenberg was also there for the duration, sitting in meetings and watching the team work. But aside from some quotes from blog posts at the time, we never get a sense for what the characters are thinking. They wind up very two-dimensional, identified mostly by their previous work experience and area of Chandler that they own: there's the old Mac guy working on a prototype of the UI, the Next manager trying to architect the system, the former Lisp programmer designing the repository, the two dudes from Netscape thinking about network protocols, and so on (somewhat strangely, but presumably intentionally, Rosenberg rarely even describes anybody's physical appearance). This is an important oversight because it is apparent that the participants are not completely aligned in how they would implement Chandler; the slow pace is partly due to doubts about the technologies, algorithms, and features that have been chosen. The result reminds you of a basketball team composed of All-Stars who haven't practiced together much, with each one grumbling that if they were only given the ball more, the team would win. Drawing out some of this internal dialogue would have emphasized its impact on the project. It also would have demonstrated the somewhat artificial nature of the open-sourceness of the project; people were initially brought together and told to work out their differences, rather than being able to evaluate the current state of the project and decide if they wanted to participate (one notable battle is between the team members who want to get some code working and then improve it, and those who want to do more up-front design: the fight continues until those who advocate the former plan give up and abandon the project one-by-one).
At one point Rosenberg actually writes, "By now, I know, any software developer reading this volume has likely thrown it across the room in despair, thinking, 'Stop the madness! They're making every mistake in the book!'" (luckily, he said it about 10 pages before I was about to do just that). Certainly the Chandler team did err in a variety of ways, but to my mind they made one overarching mistake. They ignored the obvious "elephant in the room" problem with their plan: trying to make a distributed PIM that synchronizes without a server. Instead of figuring out an algorithm that would work for that, they spend their time debating over details of visuals and network protocols. It's somewhat puzzling why such an individually experienced team would have such a collective blind spot. In the end it takes a new arrival on the team to point out that this problem is extremely hard, and eventually redirect the project towards a server-based approach. This new person is also the first one with prior experience working at Microsoft, a fact that you can interpret as you wish (to me, as a Microsoft employee, it made the mild potshots that Rosenberg directs at the company seem silly).
The other decision that caused problems, but is never questioned by the team or the book, is the plan to produce an application that runs on multiple platforms (WIndows, Mac, and Linux). The fact that Microsoft writes almost all of its software for a single platform is an often-overlooked technical advantage, which makes it simpler for the company to develop applications, and lessens the test burden significantly. The decision to go cross-platform leads to a dependency on a cross-platform GUI builder, which leads to problems when it doesn't support features that the team needs. Although I understand the motivation to go cross-platform, the team also missed the fact that for an open-source project, cross-platform porting is a perfect task for someone outside the core team. Get your code working great on one platform, induce extreme amounts of envy in someone who runs a different platform, and let them scratch that itch until you are cross-platform.
Rosenberg takes a 70-page break in the middle of the book to look back on the history of software development in general, trying to understand if there is a better way. He presents some fascinating research from the early days of programming, making you realize that all the problems we have today were perfectly anticipated by programmers 30 years ago (unfortunately, they didn't have any better answers than we do). I was hoping that I would get some good advice from this section, but his overview of development practices is somewhat cursory, and he gives roughly equal time to the important (test-driven development), the futuristic (Jaron Lanier has resurfaced after his virtual reality heyday and is now somewhere beyond the gravitational pull of the inner planets), and the incomprehensible (Charles Simonyi and his thing-that-Charles-Simonyi-is-working-on-that-nobo dy-groks). Rosenberg admits that he doesn't fully understand all this stuff, and we have to cut him some slack: we don't fully understand all this stuff either.
A better place to gain wisdom for your own software project is simply to play fly-on-the-wall as the team — despite the best of intentions, no financial concerns, and very smart people — makes the same kinds of mistakes that we always read about, but "know" that we would never make ourselves. They plan too many features, don't make realistic schedules, can't decide on goals for their interim releases, etc. They even have the advantage of a brilliant (to my mind) usability design insight, that the program should present the user with a model based on David Allen's "Getting Things Done" plan of a single to-do list. This is the direction in which Outlook is moving, but the Chandler team nevers takes much advantage of this vision. In the end the project is a great example of what not to do, and Rosenberg does a great job of telling that story. As Kapor muses at the end, "I do think that organizationally and personally we've learned an enormous amount about how to develop software. We've kind of reinvented the wheel. These are things that other people know, so it's taken us a little longer to learn those things. But having learned them, that's a kind of intellectual capital, and I'm absolutely firmly intent on reapplying it and staying the course." It's not clear why the Chandler team needed to reinvent so much that others already knew; but hopefully people who read the book will realize thay they are not immune to problems either, and do a little bit more of whatever it is that you need to do more of, and a little bit less of whatever it is you need to do less of. The question of just what those things are won't be any clearer when you are done reading Dreaming in Code, but at least you will know that you aren't alone."
Rosenberg's book sports the obligatory back cover blurb comparison to "The Soul of a New Machine". It's the latest in a long line that purport to be the worthy successor to Tracy Kidder's Pulitzer Prize winning 1979 tale of a computer project at Data General. Although the book is worth reading both for programmers and the rest of the world, it falls short of the master in a couple of ways.
First of all, unlike Kidder's heroes who eventually triumph over long odds, Chandler doesn't get finished (disclaimer: I work at Microsoft, which produces Outlook, a potential competitor to Chandler). It progresses at an excruciatingly slow pace, until after 3 years Rosenberg gives up and publishes the book. The product is still shuffling towards a 1.0 release; one of the best aspects of the book is that the participants are still out there, blogging and coding away. Chandler may well mirror the Mozilla project, getting bogged down at the start before gradually moving out of the "large pile of interesting code" phase at which the book bids it adieu, and eventually maturing to take over a part of the world; but right now it's unclear what the future holds.
More importantly for the story, Rosenberg does not spend much time delving into the motivations and inner feelings of his characters. Kidder's big advantage at Data General was that he was actually present during the project, rather than relying on the after-the-fact interviews that guide many similar books. Rosenberg was also there for the duration, sitting in meetings and watching the team work. But aside from some quotes from blog posts at the time, we never get a sense for what the characters are thinking. They wind up very two-dimensional, identified mostly by their previous work experience and area of Chandler that they own: there's the old Mac guy working on a prototype of the UI, the Next manager trying to architect the system, the former Lisp programmer designing the repository, the two dudes from Netscape thinking about network protocols, and so on (somewhat strangely, but presumably intentionally, Rosenberg rarely even describes anybody's physical appearance). This is an important oversight because it is apparent that the participants are not completely aligned in how they would implement Chandler; the slow pace is partly due to doubts about the technologies, algorithms, and features that have been chosen. The result reminds you of a basketball team composed of All-Stars who haven't practiced together much, with each one grumbling that if they were only given the ball more, the team would win. Drawing out some of this internal dialogue would have emphasized its impact on the project. It also would have demonstrated the somewhat artificial nature of the open-sourceness of the project; people were initially brought together and told to work out their differences, rather than being able to evaluate the current state of the project and decide if they wanted to participate (one notable battle is between the team members who want to get some code working and then improve it, and those who want to do more up-front design: the fight continues until those who advocate the former plan give up and abandon the project one-by-one).
At one point Rosenberg actually writes, "By now, I know, any software developer reading this volume has likely thrown it across the room in despair, thinking, 'Stop the madness! They're making every mistake in the book!'" (luckily, he said it about 10 pages before I was about to do just that). Certainly the Chandler team did err in a variety of ways, but to my mind they made one overarching mistake. They ignored the obvious "elephant in the room" problem with their plan: trying to make a distributed PIM that synchronizes without a server. Instead of figuring out an algorithm that would work for that, they spend their time debating over details of visuals and network protocols. It's somewhat puzzling why such an individually experienced team would have such a collective blind spot. In the end it takes a new arrival on the team to point out that this problem is extremely hard, and eventually redirect the project towards a server-based approach. This new person is also the first one with prior experience working at Microsoft, a fact that you can interpret as you wish (to me, as a Microsoft employee, it made the mild potshots that Rosenberg directs at the company seem silly).
The other decision that caused problems, but is never questioned by the team or the book, is the plan to produce an application that runs on multiple platforms (WIndows, Mac, and Linux). The fact that Microsoft writes almost all of its software for a single platform is an often-overlooked technical advantage, which makes it simpler for the company to develop applications, and lessens the test burden significantly. The decision to go cross-platform leads to a dependency on a cross-platform GUI builder, which leads to problems when it doesn't support features that the team needs. Although I understand the motivation to go cross-platform, the team also missed the fact that for an open-source project, cross-platform porting is a perfect task for someone outside the core team. Get your code working great on one platform, induce extreme amounts of envy in someone who runs a different platform, and let them scratch that itch until you are cross-platform.
Rosenberg takes a 70-page break in the middle of the book to look back on the history of software development in general, trying to understand if there is a better way. He presents some fascinating research from the early days of programming, making you realize that all the problems we have today were perfectly anticipated by programmers 30 years ago (unfortunately, they didn't have any better answers than we do). I was hoping that I would get some good advice from this section, but his overview of development practices is somewhat cursory, and he gives roughly equal time to the important (test-driven development), the futuristic (Jaron Lanier has resurfaced after his virtual reality heyday and is now somewhere beyond the gravitational pull of the inner planets), and the incomprehensible (Charles Simonyi and his thing-that-Charles-Simonyi-is-working-on-that-nob
A better place to gain wisdom for your own software project is simply to play fly-on-the-wall as the team — despite the best of intentions, no financial concerns, and very smart people — makes the same kinds of mistakes that we always read about, but "know" that we would never make ourselves. They plan too many features, don't make realistic schedules, can't decide on goals for their interim releases, etc. They even have the advantage of a brilliant (to my mind) usability design insight, that the program should present the user with a model based on David Allen's "Getting Things Done" plan of a single to-do list. This is the direction in which Outlook is moving, but the Chandler team nevers takes much advantage of this vision. In the end the project is a great example of what not to do, and Rosenberg does a great job of telling that story. As Kapor muses at the end, "I do think that organizationally and personally we've learned an enormous amount about how to develop software. We've kind of reinvented the wheel. These are things that other people know, so it's taken us a little longer to learn those things. But having learned them, that's a kind of intellectual capital, and I'm absolutely firmly intent on reapplying it and staying the course." It's not clear why the Chandler team needed to reinvent so much that others already knew; but hopefully people who read the book will realize thay they are not immune to problems either, and do a little bit more of whatever it is that you need to do more of, and a little bit less of whatever it is you need to do less of. The question of just what those things are won't be any clearer when you are done reading Dreaming in Code, but at least you will know that you aren't alone."
