How To Make Software Projects Fail

Bob Abooey writes: "SoftwareMarketSolution has an interesting interview of Joel Spolsky, of Joel on Software fame. Joel, a former programmer at Microsoft, discusses some of the reasons he thinks some very popular software companies or projects fail, including Netscape, Lotus 123, Borland, etc." This interview brings out some mild boiler-room stories which sound like they could be the basis of a good book, along the lines of Soul of a New Machine .

Re:Isn't it obvious?

Actually, reading this interview shows how there where serious blunders performed by NS, Borland, etc. In each case, while MS improved their software, the other companies rewrote their software.

Re:Isn't it obvious?

It is funny how every company he talks about lost to MS. Seriously though, one of the things he does say is:

Fortunately for Microsoft, they did this with parallel teams, and had never stopped working on the old code base, so they had something to ship, making it merely a financial disaster, not a strategic one.

IOW, have more money than God and throw it at any problem you're having trouble with. The minnows in the pond get beaten up by the 800lb gorilla (or something).

Re:Isn't it obvious?

> IOW, have more money than God and throw it at any problem you're having trouble with

Didn't work for IBM in the early 90's, didn't work for Detroit in the late 70's and early 80's, still doesn't work for the government.

Re:Isn't it obvious?

Show me a government owned power company in the USA.

OK, I'll mention a few: TVA (i.e., Tennesse Valley Authority, which lifted an entire region out of utter abject poverty during the Depression, SoCal's DWP (which not only distributes water and generates power, but also manages to generate power while distributing water), Sacramento's Municipal Utility District (MUD) which generates and distributes most of the power in north-central CA, and finally the BPA (i.e., Bonneville Power Administration) which built and still operates most of the hydroelectric power generation and transmission in the Columbia watershed. The Northwest has a lower cost of living partly due to low power costs (though it isn't guaranteed and it has been rising) and low water costs (likely to continue given near term global warming effects). Water, Power (and soon, Broadband) are _exactly_ the infrastructure investments that our government does well and should control. Private utilities are very vulnerable to economic fluctuations where their executives' self-interest leads them to try foolish deals and daft accounting tricks in search of short-term performance, while government can weather tough quarters (and years) without worrying about the stock analysts.

In case you hadn't noticed, the major private power utilities in California are in bankruptcy and are desperately beseeching the State to bail them out (and might yet stick the tax and rate paying citizens after all, given how cozy their lobbyists have been with the CA PUC, Legislature, and Executive branch fixers, just about forever). One can only hope that the CA government and regulators now realize that the public is watching with interest and will nail them if they screw it up further, so they might fix it properly.

Of course, private utility executives and board members never do get held accountable, nor do their government co-conspirators, but if things were to be really just, there'd be a few of them hung from lamp-posts in San Francisco before this is over. Screwing the public for private gain is just the sort of thing that deserves "extreme prejudice."

Government utilities are a good thing, mostly (WPPS notwithstanding, but that was a _private_ boondoggle admittedly triggered by a BPA error). Private utilities are simply disasters waiting to ripen, explode, and be discovered, unless they are regulated into castrated quasi-governmental entities. The term "private utility" really is an oxymoron.

Perhaps you should read the article

Before you jump to false conclusions. A lot of companies that Microsoft "drove out of business" were driven out of business because they made stupid mistakes. Yes, MS's money and marketing helped, but some of the stupid things these companies do is their own damned fault.

You may want to check out this article by Robert Cringely: Microsoft's C# Language Might Be the Death of Java, but Sun's the One to Blame [pbs.org].

A lot of truth in that...

Re:Perhaps you should read the article

The interesting thing is that Microsoft made plenty of stupid mistakes too, but since they were powered by monopoly profits in OSes (and earlier on by licences for BASIC in ROMs), they could afford to wait out their mistakes and just keep throwing money at the problems until they straightened them out. As long as a company is successful in the long run, people forget most if not all of their stupid mistakes, but if the stupid mistakes take down the company then people remember it. The history books are written by the victorious (in the short term at least), so I would take a lot of this guy's story with a grain of salt, as it is certainly far from unbiased.

It is a pretty bad situation for a market to be in if any one company is so big that all they have to do is wait for their competitors to make a mistake in order to be able to crush them. When any one company wields so much power, it makes it nearly impossible to sustain any sort of competition. Not to mention that when a market is ruled by an 800lb gorilla, all of the smaller players are pretty much forced to take more risks and make other decisions differently than companies do in a market where there are at least two or three players splitting up significant chunks of market share. Sometimes those risks pay off brilliantly, sometimes they are stupid mistakes.

Re:Perhaps you should read the article

Microsoft has had a monopoly position for a long time. Most people don't remember when there wasn't a such thing as an IBM PC or MS-DOS. Microsoft got into a monopoly position with MS-DOS pretty quickly after its release in 1981. Within a couple of years it had killed off CP/M and the 8 bit 65xx OSes. So certainly since at least 1984 or so Microsoft has basically had a monopoly position in OSes.

But wait... that wasn't Microsoft's first monopoly position. Even before MS-DOS they basically had a lock-hold on BASIC interpreters, which were one of the most critical parts of the pre-IBM PC desktop. Apple's BASIC, Commodore's, and Radio Shack's were all licensed from Microsoft. Most CP/M machines also bundled Microsoft BASIC. In fact a strong case could be made that the MS-DOS monopoly which grew into the Windows monopoly was itself leveraged from the BASIC monopoly.

There is a difference between pointing to monopoly power as the primary reason for Microsoft's success than the only reason. Saying that Microsoft's monopoly power had nothing to do with the failures of other companies is as wrong as saying that those other companies made no mistakes.

As for Microsoft's marketing, I am not so sure I would call it 'shrewd' as pervasive and persistant. They've outspent just about everyone else for years, with the possible exception of IBM, but that is easy when you have monopoly profits to fall back on. It would be hard for a startup to outspend Microsoft on advertising, even in a niche.

Good point

He says:

"My theory is that this happens because it's harder to read code than to write it."

He couldn't be more right. I've recently been asked to port some code from another group in the company. Upon first reading it, I found global variables being referenced from everywhere, and it looked terrible.

The more I looked at it though, the easier it got to read, and having an existing code base to work from made things much easier.

Plus, when I have problems with it, I can blame it on a "design error" by the previous programmers!

Re:Good point

This is why it's important to force your developers to (gasp) comment their code. Of course, 99 times out of 100, this won't happen because either (1) the boss thinks that'll slow you down and you'll miss your release date or (2) your boss has never written a line of code in his life and doesn't even know you can comment on that computer codes thing.

Re:Good point

Good code speaks for itself about what it does, but not about WHY it does something and that's were comments come in handy.

Re:Good point

I couldn't agree more. In a similar vein, I removed the turn signals from my car. I get .0000047% improved performance and, after all, what good are signals? I know where I'm going.

Re:Good point

If you think no comment is ever useful on well written code, you shouldn't be a developer. If someone new has to come into your project (especially a jr. engineer), well placed comments will greatly decrease the time required to understand the overall function of your code. By assuming your code speaks for itself, you're assuming everyone is exactly as familiar with the code as you. This basically forces a newcomer to study your code down to every nook and cranny before it even makes sense. Not commenting your code is nothing other than lack of discipline and a sign of indifference towards the overall success of the project.

or (3)..

or (3), incessantly repeated nerdisms such as "if it was hard to write it should be hard to read" instill an improper sense into young, impressionable programmers.

Re:Good point

Good code is not just code that compiles and runs efficiently. Good code also has the following properties:

• Clear, Consistant Formatting - This code complies with the company standard for writing code. Indents are properly nested, Functions are named consistantly, variables use Hungarian Notation or some other standard. Any programmer should be able to look at code by another programmer and pick up on it very quickly, without shaking their head and saying "What the hell were they thinking?"
• Copious Comments - Lots of comments, clearly written and explanatory. What does this function do? Put a block at the beginning explaining it. How does this algorithm work briefly? Write a paragraph if you have to. The best comment I heard was from a friend about a former coworkers code: "It's English with some C++ thrown inbetween the comments."
• Documentation - Anyone who shrugs this off is an idiot. You always have time for documentation. And it's not just for the instance where a programmer gets "hit by a bus." It's for people who leave behind code when they quit, or go to a new project. It's for the new hires, so they can understand and study and learn good design, good techniques, and developer rationale. It forces developers to explain themselves. And it allows non-techies to understand what they're doing. Imagine you had to get through 12 years of grade school with no books. Pretty frightening, eh? Documentation is good. Write it.

Coders who follow these rules truly are an asset to their company. Geeks who hack, write unreadable code, and utter geek credos about enforcing obfuscation and being purposefully vague have no place in a business environment.

Hubris, laziness, and impatience

Let's compare:

for (i = 0; i array_size; i++)
free(array[i]);
free(array);

and now let's look at:

// get rid of the array
for (i = 0; i array_size; i++)
free(array[i]);
free(array);

Has your life *really* been so harmed? Is this *really* so terrible? Comments should not be written with the thought that your university professor would know what everything else means. Comments should be written so that all of those folks without a PhD in CompSci. know what it means.

What if the next joe to hit your code doesn't have a degree? What if the recently-hired intern was just handed a "C in 21 days" book and told by the manager to "fix it" because the programming team is snowed in (or similarly unavailable) and the customer is screaming? (Yeah, try and tell me that's never happened...)

A fine use of comments is (for example) every ten lines to say, in general, what is going on. One thing I used to do is write a comment at least every 10-15 lines. Why? When the next joe who comes along has to read/edit my code, scanning through some periodically placed comments will *always* be quicker and easier than reading the code. ...assuming they are English speakers of course. Proof? I have programmed (I don't count the BASIC years) for ~10 years. I have been writing/speaking English for over twice that long. Which do you think I'm better at?

The code effectively shows my implementation, but may not show my intention. I have coded for years. I started dreaming in code several years ago. Shortly thereafter, the code actually worked when I typed it in the next morning. That isn't the point. How good a coder you are isn't the point.

When you have a hundred thousand lines of code to go through, comments become like "Cliff's Notes." For the quick patch (probably the majority of code being written by most people), comments are invaluable. Who cares if I didn't read Moby Dick if I can still pass the pop quiz? If I need to make an indepth study, I can still do this, but thank god for the "Cliff's Notes."

Now then, on to the "proper" use of comments.

1. Write out what you are planning to do in English. (or whatever else may be the dominant language in your development group) Fill in every step in the problem. This is NOT psuedo code. This is akin to: Find out who www.yahoo.com is, open a connection, ask for the main page, and check to see if our cache is still valid. If the cache is stale (the yahoo page has been updated), get a new copy of the main page. If the cache is still valid, pull the page from cache instead. Drop the page into the "ready" bin and send a message to the user that the page is here.

2. Make a copy and label it "documentation."

3. Go back to the original, fill in all of the logic in whatever programming language at the appropriate points in your "documentation," and label it "source file."

This means that your documentation is done, your code is adequately commented, and your algorithm and intent(!) are clearly defined for both your co-workers (and yourself when you have to fix something ten months from now). If you can't spell out the problem and the solution in your primary native language, you sure as hell better not be trying to spell it out in a programming language that members of your team have only been using for two years.

The only excuse not to do the above is laziness. For some people, laziness is not considered a bad thing. It was noted as being one of the main virtues of a hacker -- hubris, laziness, and impatience. Hell, according to this measure, I myself am lazy from time to time. But cut the bravado, the beating of the chest, the battle cries of "I'm smart enough to figure this out, so should you be," and call a spade a spade. Avoiding comments means that you are being lazy.

Re:Good point

From a purely intrinsic point of view, I agree with you StaticEngine. From a purely practical point of view, I couldn't disagree more.

Let me explain myself. I have been the type of programmer you speak of. I have written copiously commented code. I have properly formatted my code and used standardized function names and such. After all, I was taught in college to write and comment my code so that any programmer could walk in off the street and understand it easily; that made it easy to replace me and I was.

It seems that when you follow good programming practice, you end up destroying your job security; and as silly as it sounds... it appears to be sooth.
Jaded in a realistic world.

Job security?

Dude, think about what you are saying. Do you want to keep maintaining your old crappy code or pass that job onto someone else? Or do you want to go write some new code?

Your perspective assumes your company requires a fixed amount of software. Think more imaginatively.

Better documentation means you can shove maintenance to a more junior programmer with less pushback.

Also, without good documentation, its a b*tch to try to outsource/handoff pieces of the code you don't want to bother writing.

Besides, I don't care how well documented your code is, you should always be able to convince a boss that its more efficient for you to make changes to it (even at higher salary) than some cheaper guy who has never seen the code before.

--LP

Re:Good point NOT

Coders who follow these rules produce what? 3000-6000 lines of code a year? Ain't gonna get product out the door that way. What you get is code with lots of comments about what it is supposed to do, but doesn't because of all the time spent on documentation instead of design and debugging.

Excellent point. My philosophy about commercial software is this:

Never forget why you're writing the code to begin with. The point is to get working, stable code out the door as fast as possible. Anything that does not accomplish this is a waste of time.

Doing your architecture work is fine, but do it on a whiteboard in your cube with your co-workers. Don't waste time holding formal design meetings and drafting useless documentation and diagrams because, honestly, nobody will ever read them.

Modularize/componentize your code as much as possible. Document what the module as a whole does and what data it requires and what data it returns . You shouldn't have to waste time commenting every single peice of code. If you''ve modularized and documented what the module does, any decent programmer can figure out the rest.

In addition to not hiring idiots, don't hire people who love to wallow in bureaucracy and process. Besides not getting jack shit done, they impede everybody else.

If you want to comment and spend hours drawing Visio diagrams, fine. Wait until after the product is released to do this. These do not accomplish the goal (see point #1).

Chris

Hungarian notation considered harmful

Clear, Consistant Formatting: [...] variables use Hungarian Notation or some other standard

I find Hungarian notation much harder to read than not using it. For example, I find the Unix man page for strcpy which looks like this:
char *strcpy(char *dest, const char *src);
much easier to read than the Windows-style Help which is full of stuff like "LPCSTR lpBuf" and suchlike. The idea which is commonly called "Hungarian Notation" says that a variable name should include the type of the variable as a prefixed abbreviation in front of the name. This leads to stuff like:
byte[] baBuf;
whereas without Hungarian, it might be called:
byte[] message;
which would be much more meaningful.

Especially in object-oriented programming, the type of a variable is the least important piece of information about the variable, and has no place being abbreviated and prefixed to the name. The most important thing about a variable is what the programmer is using the variable for, and that information should be what the name of the variable tells another programmer. If somebody really wants to know the type of a variable, then their editor or IDE should tell them what it is. If it doesn't tell them automatically, then they should look at the variable declaration, which will state exactly what type the variable is. If programmers want the variable name to tell them the type, then what is the point of declarations? And why bother putting a comment near the declaration saying what the variable is for, because people aren't going to read the declaration or comment anyway, because they are just going to look at the Hungarian warts.

The argument that Hungarian notation reduces the possibility of assigning variables of different type to each other is long dead with compilers well capable of throwing errors if any incompatible type assignments are attended. I think that Hungarian notation is dead, or at least should be.

How To Make Software Projects Fail:

Step 1: Hire my boss (God, please hire him away!).
Step 2: Put him in charge of software development.
Step 3: Do nothing as priorities change weekly and deadlines slip away.
Step 4: Do nothing to stem exodus of clued-in employees to less-screwed companies.
Step 5: Force remaining employees to work 15 hour days. Provide subtle reminders that there's a recession out there.
Step 6: Do nothing as even non-clued-in employees flee.
Step 7: Hire a sweatshop in China to crank out code; present this sound like a good idea.

There, that was pretty easy. And, to be honest, everything beyond Step 1 pretty much happens on its own.

Re:How To Make Software Projects Fail:

Ah, the true sign of a clueless boss: A non-AC post about how clueless your boss is, with no fear he'll read /. and find it ;)

Soul of a New Machine (slightly OT)

I'd HIGHLY suggest reading a SoaNM. One of my favorite books (fiction or non).

problem = clueless management/directors/execs etc.

Alot of .com's I've worked at in the past 2-3 years always wanted to "lowball" developers and engineers, while lining the pockets of resource managers, implementation managers, marketing people, etc.

Then a skilled/talented developer and/or engineer wants more money. The employer does nothing to retain them - thus the skilled/talented employee leaves.

Now who maintains the code?

The other problem is bringing in short term consultants for long term projects. The non-technical people who make these executive decisions don't seem to see the feasability of KEEPING their code maintained by the talented/skilled person who BEGAN the development on it.

I know alot of consultants read /. - so I'll probably take a big hit on the karma - but I just was the casualty of another dotcom failure - and this was a seriousl problem.

Another problem is hiring non-technical managers to manage technical people. At my last job we had a manager off of an automobile manufacturing production line quit his job at the auto company to take a job as the manager of a group of Unix admins. This "bumper jockey" had NO CLUE what we did for a living, and treated us like a bunch of unionized UAW slobs, and not like professionals.

How can a non-technical boss effectively manage technical people???

Also - how about all the Ceo, Cio, Cto, eieio - types with their big salaries, catered lunches, etc... Alot of them have NO programming or hands-on technical experience. Hell - I've had the CTO come up to me and tell me that "The Internet was broken" when he knocked the dongle out of the side of his laptop - severing the network connection. And this guy is our Chief Technology Officer???? *lmao*

I'm not saying that only technological people can make technology companies work - but I do feel that managers should take some sort of hands-on classes to learn some basic programming and internet skills so they have SOME SORT OF CLUE about what WE all do for a living!

Death by Engineers

I'm in the alternate situation: Too many of my execs (except, for some reason, the VP of Development) are engineers.

This leads to a whole host of problems:

Many of them tend think they're smarter than people in non-engineering roles.

Pursuant to this, they don't think PR and marketing and sales are "hard" or really even "important".

Again after #1, they're always right when in disagreement with marketing or sales guys.

Most of them haven't developed in a decade+, so now they know just enough to be dangerous -- make micromanaging decisions about detailed subjects things they don't understand well enough, chase unnecessarily after bleeding edge tech, etc.

Fail to understand that not everyone wants to always work 14 hours a day.

Laugh off meetings, so that eventually nobody in the company knows whats going on.

As a result, nobody's heard of us (no marketing budget, no trade shows, no nothing) and nobody's buying our products (engineers tend to make lousy sales guys; despite what they might believe, nobody wants to listen to a 3-hour ridiculously detailed presentation on your product).

There's got to be a happy medium someplace.

Rewrite vs compatability

Joel: Hmm. That reminds me that Microsoft learned the "no rewrite" lesson the hard way.

Obviously, MS biggest problem though is that they don't know when to give up and actually rewrite. For instance, it seems that the windows series of operating systems are all made with the intent of being backwards compatible and reusing core parts back to early DOS systems. Backwards compatability and code reuse is nice and all, but there is a limit to it and a time to give up.

It will however be interesting to see what comes out of the "total rewrite" of IIS.