Big Ball Of Mud Development Model

Lightborn writes: "The Big Ball of Mud Development Model examines exactly why so many projects (software and otherwise) end up looking like a bowl of spaghetti. A good list of things not to do when developing a project."

Reading code ...

Open source is massively contributing to better source, if it were only because more students now not only get the opportunity to read real-life code.

Even stronger, learning to read code has become more important than churning out vast amounts by yourself.

Inadvertedly, open source addresses a serious flaw in computer science education: the fact that students should, in the first place, learn to read, use, alter and inspire themselves from existing bodies of code, instead of churning out unrealisticly small mickey mouse examples.

Well-written code reads like poetry.

Re:The Achilles' Heel of OSS

Yes. The problem with OSS is that much of it is evolved code, not designed code. This leads to robust and secure systems, but not ones where it is easy to add features or hunt down new bugs. Consider the example of an AI neural net (or the real thing, if you like squishy things). You've trained the thing to give mostly right answers by propagating feedback through the net in complicated ways, but have sacrificed any possibility of understanding why it gives the right answers. It is like a roof made from patches with no actual roof left. It keeps the rain out, but has lost all structure and is harder and harder to deal with. (Too many analogies in that paragraph).

I've finally adopted a very game-programmer oriented philosophy towards development. Code should be written so that it is the specification, with appropriate inline comments documenting it and really clear variable names. Programmers should be extremely vigilant, and continuously roam their own code making sure that it actually reflects the current state of assumptions about the system. Whenever a change is made to the system, anything remotely affected should be proactively rewritten to reflect the change. This is pretty much how Abrash describes himself and Carmack working on Doom and Quake, and it is really successful. You keep performance up, stay in touch with your code, and never accumulate cruft. Bugs are immediately ferreted out and the programmer must never fear diving into code to tackle a big cleanup job, and can never allow pieces of code to exist that she (or he) doesn't understand.

Of course, you need massive automated tests to make sure your rewrites don't screw anything up. Designs must be extremely abstraction oriented, with a close eye to strong interfaces and bootstrapping, otherwise you will end up with so much code that it is impossible to manage the continual cleaning. And you need really dedicated programmers.

When I look at the Doom and Quake source, and the code that my own dev. team has produced, I see that the results are worthwhile. Each routine is beautifully crafted and works flawlessly. The codebase is a fraction of the size you would expect because so much effort has been put into doing everything the right way and eliminating broken or excessive code. And no bugs...

magic

Re:Some Good Software

One example of some very well designed software is the Shuttle OS that powers NASA's Space Shuttle.

Yep. However, I read some years ago that the Space Shuttle code costs $1000 per source line to develop (for the whole shebang, analysis, design, implementation, testing, maintenance, documentation, etc.) That's one thousand dollars per line (if I got paid that much, I'd be long retired :-). This only applies to manned missions though, software for unmanned missions costs about $100 per SLOC.

[cynical] I fear this is not because they value human life so much, it's more that loss of human life leads to huge costs in terms of publicity, scrutiny, congressional investigations, freezing of funding, halting of programs, etc. The Challenger tragedy cost far more than the $2 billion that the spacecraft costed, the halting of the launches and the redesign of the shuttle was far more expensive.[/cynical]

Anyway, as I should not have to point out, testing and bug finding is hampered by the law of diminishing returns. It is extremely costly to get the last 1% of the bugs out of the software. In the case of NASA and the space shuttle, there is an economic incentive to hunt down the last of the bugs. In the case of commercial software, stuff riddled with bugs is released because the cost of delaying the release outweighs the cost of leaving bugs in (dare I say that it often makes economic sense to leave bugs in? Just release the bug fixes as an upgrade...).

Anyhow, I think that Open Source has an advantage over commercial software in the sense that the developer(s) are motivated by something else than a paycheck. It is often a sense of pride that makes them strive for clean, bug-free code. Ever notice that the quality of software seems proportional to the inverse of the cost? I'm only half joking here....

Re:The Achilles' Heal of OSS

Freaky! I was just reading one if Brian Foote's, papers: Metadata and Active Object Models [laputan.org]

Pilot Light? Check!

One thing that gets me about the OSS community is the over-reliance on C.

Petrol(UK->US Translation - Gasoline)? Check!

I mean look at Gnome and GTK+, it based on some ugly C struct kludge to enable pseudo-object-orientation!

Flame-throwers are go!!

And then there is KDE 2.0 based on even uglier preprocessor commands.

I mean WTF is going on. The method of production in OSS is innovative, but the resulting programs end up being MS ripoffs. We need some true innovation regarding what we develop and the tools we use to do so. Because lets face it, most app level programming would really benefit from C++, or even something like Eiffel and the concepts of design by contract. Using pre/post conditions and invariants as in B notation, one can almost guarantee the correctness of one's program. (Eiffel and B were both in part developed by Betrand Meyer, he also played a hand in Z notation).

There is also an interesting project called EDMA [gnu.org] That is trying to create an enviroment in which objects can be inherited from after they are built, a bit like CORBA, but IMO better.

void SelfPromotion {

I am in the early stages (i.e. thinking a lot and getting myself confused) of developing a Dynamic Object Enviroment to support reflection and better models of code reuse through selective "pilfering" of code and structure from other objects (Classes don't exist, only instances, although instances may share code). No links, or anything much to speak of as yet.

}

Anyway I digress, we should start thinking about the tools we are using, and ensure that they are suited to the job. For most things, the performance benefits of C, are not really crucial WRT anything outside the Kernel.

Cheers,

faichai

It took me a while to figure this out...

For a long time I couldn't figure out why others had such a hard time fixing bugs and changing their programs, while I could do it without any problems (no I'm not trying to be arrogant or pretentious).

Then a coworker made this remark: "Ray how can you write such well-organized code in one pass?" At first I couldn't understand what she was talking about, after all doesn't everybody constantly review their code? Doesn't everybody constantly rearrange functions and classes, rename variables, redefine protocols? Apparently not.

Correct me if I'm wrong, but it looks to me like most programmers write something once then spend the rest of the time trying to get that working. They never go back and rewrite the code, they just keep adding fixes to it. How can this ever work smoothly?

I've also seen and heard a lot about processes to make a program, or anything else, come out right the first time. I don't get that either! To me, the only objective when writing programs is to make it easy to change. Period. If it's easy change, it's easy to fix bugs, it's easy to enhance, and it's easy to rearrange and redesign with hindsight.

If you want to have a good time programming, do yourself a favour: learn the tools to make global changes to your code quickly, then spend a _lot_ of effort rearranging your code and renaming things as your program evolves.

XP might offer the ultimate solution

Is there anyone out there who's into extreme programming?
I'm trying to convince my manager that this is the right approach for us but with little success.

I work for a small software house with just six developers on two main projects. What I see happening though is emergance of a structure that closely follows the XP guidelines. Our key developer left last month and left us with well over half a million lines of code to maintain and extend.
Naturally the first outcome was an outbreak of panic (particularly among the management) but us programmers didn't have time for lamenting. What happened though is that once we lost our mastermind of coding and had to rely on ourselves even with the most critical parts of the system people started sharing knowledge much more freely and the environment became extremely productive. I found myself often pairing with other developers to help them understand some parts of the code and vice versa.
It seems that with the old state of things (one developer churning out masses of code and the rest almost idle) while it felt comfortable it left many people fearing that they can't perform on their own (one of my colleagues wanted to resign straight away as soon as she learned that the guru is leaving).

The moral is that what pair programming prescribes turned out to be our life jacket and ultimately boosted the productivity of people who were previously intimidated by the presence of the 'main man'.

Personally I find coding in pairs a brilliant idea. I find myself producing much higher quality code when I have someone looking at what I'm typing. The bugs are fewer and more people know what's going on in the system. It works much better than peer reviews and documentation projects that we had foisted upon us by the management. It's not the official company policy but we do it anyway. For us XP works (or at least the parts we adopted).

Re:The Achilles' Heal of OSS

I agree with you in part, but pigeonholing all of OO as obscurantist and overcomplicated is as erroneous as overhyping it as if it were the Second Coming. There is nothing wrong with the motivation behind OO itself; the problem lies mainly in implementation. Specifically, what we now consider to be "OO programming languages" and "OO design practices" goes far beyond the original concept of both, and much of it is indeed obscurantist nonsense, which induces a huge amount of needless overhead, both of the conceptual kind on the designer and of the practical kind on the implementer. This is especially true in the case of small to medium software projects, even more so because often designer and implementer are one and the same.

Case study 1: C++. An extensive critique of C++ as an OO language for production systems, from the point of view of an Eiffel-cheerer, can be found here [uts.edu.au]; in my opinion, it suffices to say that, given its status as just one step up from a C add-on, and given that, when building on such a shoddy conceptual infrastructure as C's, it's hard to conceive how one could do any better, C++ should be considered to be outside the scope of this discussion.

Case study 2: Java. Now, Java is built from head to toe for maximum OO. This is incredibly intrusive to anyone who wants to do some real work using it, as opposed to just drawing nice schemes and writing UML models. Java is built to enforce those styles and concepts of programming which the designer felt to be correct. It's languages like this which give OO a bad name, and they should be shunned.

Case study 3: Perl. Perl was built to be a scripting language - in Osterhout's original conception, a "glue" language. Thus, practicality being the most important goal in it, it's easy to understand why Perl's OO is as it is. Specifically, it doesn't exist per se; no special syntax or semantics is enforced for OO programming, in fact all of it is built upon simpler, pre-existing constructs - specifically, taking advantage of an isomorphy between modules and classes, objects and references (via abuse of the bless() and ref() functions), methods and namespace-local subs. This makes a transition to OO practices easier as a project grows. It also allows one to implement the concept of an object as he sees fit - usually the slot approach is used, using hashrefs, but there other approaches for specialised cases - including objects as indices into class-wide property arrays, an approach described in "Advanced Programming with Perl" and which is useful for when you need many objects and creating a hash table for each would be a waste of space. The discussion of OO in Perl could be extended further, but it suffices to say that, in true Perl form, it restrains from imposing a paradigm on the programmer, trusting instead that he knows better.

Case study 4: Smalltalk. Smalltalk is widely considered to be the godfather of modern OO (yes, Modula had something called "OO" before Smalltalk, but a quick glance at both languages will make it clear right away that most of what we call OO today was fathered by Smalltalk); this, combined with the widespread availability of "OO software design tools" for the environment, could lead to some people blaming it unfairly for their current issues with the paradigm. In reality, when using Squeak, a computing environment integrated with a derivative of Smalltalk, I've found the use of OO in programming the system to be perfectly natural, in contrast to the uncomfortable feeling that you get from using, e.g., Java. Part of this comes from language design itself, which makes the concept ubiquitous in a very straightforward and graspable way, but most of it comes from the environment, which is fully built on objects. In the Morphic system, you can "see" and "touch" - inspect, manipulate, delete - all objects alike. The user- and programmer-levels are intertwined, and so instead of programs, methods are the elementary user-level executable unit; this removes one unnecessary level of encapsulation, leaving all objects free to talk to each other, without being first streamlined into the procedural mold enforced by the "program" concept. All of this, plus the elegance of the Smalltalk language, makes for a system which is very easy to program, and which leaves relatively little to be desired. Thus, I consider Squeak to be a paradigm of well-used OO.

Hell, I think I've said more than I set out to... I hope at least some of it is of any use.

Spaghetti code follows from spaghetti data

One thing I believe I've learned in 25 years of coding is that your data structures are really important. I've written my share of spaghetti code, and it always results from badly organized data. Too much global data, failure to use reasonable structures, using the wrong structures, etc. Fortunately, much of the code I write these days is used for exploratory purposes and I get a chance to rewrite some of the bad parts. And the only times that the code ever really improves is when I restructure the data.

One of the good things about OO languages (and I'm not particularly fond of OO) is that they make you think about your data more. OO is not a silver bullet, though, since it's certainly possible to use one to organize your data badly. No language is a substitute for an experienced developer with some talent for organizing data in the right way for the particular project.

Of course, this is not a new concept. Fred Brooks said it nicely in The Mythical Man-Month, a book which should be required reading for everyone who does software development, and more so for people who manage development efforts.

The Achilles' Heal of OSS

I'm really glad to see this. In my experience, the great flaw in the OSS model is the quality of the code. Can we be honest? The vast majority of it is complete crap, developed by amateurs with absolutely no clue how develop to professional standards.

The OSS community needs to establish some quality standards. Linux code is relatively new, but this is going to bite everyone in the butt as the code gets modified more and more, and software rot starts to rear its ugly head.

Unfortunately, the vast majority of OSS developers are not very old (less than 25), and don't have the perspective to appreciate trying to maintain 10 year old code that has been modified 20 zillion times.

Mark my words: Unless coding standards get real important soon, OSS is going to collapse under its own weight. "As long as it works" is not good enough.

--

Not Unique to OpenSource

Why do I get the feeling this problem isn't just found in OpenSource projects? Zillions of programms, both free and commercial, are badly designed from the start. Many more could be well designed if only they didn't have to worry about backward compatibility. (probably one of the biggest problems for Windows right now...) The Big Ball of Mud architecture isn't uncommon by any means. And it's not a problem that only OpenSource faces.

Re:The Achilles' Heal of OSS

The vast majority *is* crap. But the stuff that is important, libc, the Linux kernel, GCC etc. isn't. If you look at Windows there is lots of third party software that is complete junk. The case is the same with Linux. But the "big stuff" is all big *because* it's good, well designed software. OpenSource can produce crap, anyone armed with a compiler (standard on most UNIX's) can produce utter junk. But will that junk be used? Will anyone even know it exists? Of course not.

Both OpenSource and Closed Source development can produce junk software. And both can produce great software.

Re:The Achilles' Heal of OSS

This is *not* just an OSS problem; it's a problem with *all* software. Even stuff which is well-designed from the beginning, and reasonably well-written, degenerates over time; and with high engineer turnover, even stuff written four or five years ago becomes painful to maintain.

For the OSS community *in isolation* to seek to solve this problem would be unfortunate; the industry as a whole needs to find a way to address it.

Some Good Software

One example of some very well designed software is the Shuttle OS that powers NASA's Space Shuttle. In 420k lines of code each revision has only had 1 bug each that wasn't caught by testing. If we *really* want to make some good software it wouldn't be a bad idea to take these lessons to heart. OpenSource software is already good, lets make it better. Full artical here [fastcompany.com].

slashdotted DOS attack, Google mirror

Here [google.com] is the obligitory mirror courtesy of Google.

I thought the comparisons to Design Patterns were hilarious but way too true.

Re:Anti Patterns Re:Positive patterns

Anti Patterns: Refactoring Software, Archetectures and Projects in Crisis
ISBN 0-471-19713-0

Excellent book, I would highly recommend getting your bosses a copy or get a copy if you are running a project, its concise, incisive and useful, personally I'd rate it up there with The Mythical Man Month and helps when you need to point out the company is making a classic mistake.

I code like I write

I was writing philosophy long before I was coding (PERL mostly), so it doesn't come as much of a surprise to me that I code using the same method that I use to write.

When I write, I go along a line of argumentation until something starts feeling wrong, like I've strayed too far. That's when I go back, read all that I have written, fix it up, and then continue writing until I have to stop again.

When I code, I do exactly the same thing: code until it feels too messy, go back, rework, continue to code anew, get stuck, etc.

The result has been fairly decent code that isn't too bad to alter over time. However, sometimes I get tempted to overhaul code when it really isn't necessary, because some minor issues are bothering me. (This happened with GeekPress [www.geekpress] when I was just a few days of programming away from launch, but thankfully my husband helped me get over my fussiness!)

Since I've always completely coded my own projects (even when working within a company), I have no idea whether others code in a similar fashion or not. (I'm sure that my situation is greatly simplified by the fact that I don't have co-programmers. That seems like a nightmare to me!)

-- Diana Hsieh

Re:It took me a while to figure this out...

While I agree with everything you said, a reply to your post's title is probably worth a comment.

It took you a long time to figure out. You were probably doing it instinctively before figuring out whatever everyone else's problem with maintenance was. Fair enough. I, on the other hand, learned proper maintenance through a gradual progression, and during that journey learned to recognize the shortcomings (and strong points) on others when it came to maintenance and design.

However, I think the significance of this paper is that it actually addresses some solutions to problems rather than just explaining the problems, and thereby, hopefully, promoting and explaining useful maintenance and design techniques.

This article is one of the few times that I've seen this. For example, I've read "Anti-Paterns", which is sited in the article, and quite frankly, I was rather less than impressed. It points out problems, but it's only suggestion, usually, is "rewrite". Not much of a help to the newcomer to programming. On the other hand, anyone that has been coding for any length of time, will find the symptoms that this book points out, and (obvious) solutions, as a waste of reading time.

The posted article, however, I thot made some useful points through analogy that I think might actually help some newcomers to programming understand some of the issues. I particularly was impressed by the section on "shearing". The project I'm currently working on deals with this issue particularly well, and I've used the concept since (unfortunately, I didnt design the system, and therefore cant take credit for it's chameleon-like design). This article relates the issue of shearing quite well.

So, in summary, [0] you are right, [1] this article is important because it actually shows *why* you are right.

Now, if I can just get management to read this and agree. (yeah, right)

Cheers.

The timing on this is perfect

Just the other day I was in talking with the lead developer for one of the projects I was on over the last 9 months, explaining to him how the last 7 years of code that had been written on the project was completely underdesigned and thrown together with duct tape and bailing wire. I cannot even begin to describe it. And I am supposed to be the buildmaster for this, and we are looking to have offsite developers go to town on this stuff. He looked at me, and listened to me bitch and then said "Well, we don't want to invest a lot of time into this, or money." WHat??!?!?!? He wants me to restructure the last seven years of work in a slap dash manner, not even fixing the problem, or making it worse! It was exactly this kind of thinking that got me to the point where I had to say this has got to stop. I may be just ranting, but what the hell else am I going to do? I am supposed to be buildmaster and tech support. Well, because it is such a bear to install, we have like 3 users. Guess what? I don't have anything to do but fix it! SO what am I going to do? I am going to come in on weekends and work when he is not looking and make it go, and do it right, because I am _offended_ by the crappiness of the hierarchy and code design. Now, I am not a total lone wolf here, I am discussing the structure with them, but people need to realize that a crappy design leads to a crappy project.

Really Good Software Architects

It is hard to find really good Software Architects. Lately the tendency has being to produce code at the speed of rabbit procreation - 10 times a week. The problem has gotten worse over time due to the popular rapid development tools such as Visual Basic for Windows and Object Oriented approaches such as Java for JVM. The bad news is that this deters new good programmers to appear instead of old ones. The old programming school did not have to face such issues as handling millions of users and huge databases or creating user interfaces accessible to a novice user, they were mostly concerned with the speed. Ability to hack together some brilliant Assembly code was the primary concern, I admit it is cool.

Today most so-called Microsoft Certified 'Engineers' have no clue what 'Assembly' stands for but they still don't know how to handle millions of users terrabytes of data or create decent user interfaces. The problem is that computer science became popular among those people who have no real call in their lives and who regard their work as simply a way of getting their salary. Large salaries of IT department does not help too much, they create an unhealthy attitude towards the profession.

Working on a large project that is supposed to be scalable to millions of customers, supposed to handle multiple user interfaces of various wireless devices (PDAs, Cell Phones etc) over time I had to design various components of the system. In the beginning there was only an idea which later became basically a large collection of various components. I have never before had to design and build such a complex piece of software and I am just happy that my current formal education allows me to make sound judgements about network traffic averages and variances, the speed of code in terms of iterations (big-O, big-Omega, big-Theta, they are usefull after all), being able to handle various datastructures and even creating my own new tree designs.

Nevertheless, all the way through I've felt the need for an experienced software architect. My company did not have one and we still don't and I think it is very difficult to find one with really good experience and skills.

Once I have seing a real software architect at work (he was in his forties) he was giving a presentation of his design and it was jus WOW. I mean even after working professionally for three years and handling hundreds of different programming and design problems, I don't think I could produced such a thoughtfull design that goes into details and goes over every possible issue with all the computations and considerations. It was beautiful.
I wish we all could learn from the best.