Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
×
Linux Software

Notes From the Cathedral 209

A reader wrote to us about "Notes From The Cathedral", which is sort of a tell-all from a former developer of a "major software company". An interesting, if somewhat sad, read.
This discussion has been archived. No new comments can be posted.

Notes From the Cathedral

Comments Filter:
  • Well, myself, I'll choose to work at the place that pays me $55/hr and let me use the microsoft toolset and produce actual results. Proprietary I suppose but OS religion isn't much of a concern with capitalists.
  • You've read A Deepness In The Sky by Vernor Vinge, right?

    Heh. I had exactly the same response to his comment! It creeps me out every time I hear someone mention focus at work.

    If you haven't read A Deepness In The Sky [amazon.com] get it now.

  • by Mark F. Komarinski ( 97174 ) on Friday August 18, 2000 @09:25AM (#844422) Homepage
    For all the hollering about the US Govt, the Dept. of VA (worked for a few years for them) did things the "right" way:

    1) Big 'ol binder that had coding style, approved APIs, things like that
    2) All the functions must declare inputs, outputs, and globals used before you even start coding. These declarations also make it into the code.
    3) That pseudocode goes through a peer review with coders from other facilities.
    4) Once the code is written, it goes through another peer review, and there are usually a few rewrites after that review.

    The reason for much of this is *this* is the code that runs 170+ hospitals around the country. Don't want rogue code screwing up the systems.

    The upshot out of all this is that the code is extremely easy to update, easy to debug, and the support people have an easier job.

  • On the point that cathedral projects have focus:

    Most commercial projects of 300+ people fail to complete. The failure rate goes up exponentially as the size increases.

    Very few 300 person projects would take only 6 months - you just cannot ramp up that quickly and maintain any sort of coherence.

    A good example would be MSFT Word for Windows 1.0. Took over 5 years but the plan never said it was more than 1 year from completion. ("Rapid Development" by Steve McConnell has the details).

    Most commercial products are abandonded by their vendors. A key decision factor when buying a product is - will this vendor be around in 12 months? The idea that in buying a commercial product you are buying an assurance of future upgrades would be laughable were it not so serious.

    Even within one company I have had the same company offer me three different and incompatible products in three successive years for the same problem.

  • by GregWebb ( 26123 ) on Friday August 18, 2000 @12:26PM (#844424)
    Sorry, no.

    That might be a (loose) specification, it might be a partial prototype. If that's what you'd think of as a design then I'd rather not work with you if that's OK.

    Design needs to be many things which that simply cannot be. It needs to go over the requirements that have been established and properly sort out what needs to be done to sort these out. The structure needs working out, the interfaces need specification. If all you have is a vague description of the issue and a code fragment you really can't expect to sort out the design with any serious chance of success. After a while of designing the system that way you can expect to hit problems as what is a fundamental feature of one component's implementation may become a block to another working. Whereas if it's been properly designed and analysed the chance of this happening is substantially lower.

    The other side is that, when it comes to maintaining the software, your approach to design means there's no concrete explanation of exactly why something does what it does - or even what's there. By forcing the maintainers to work out the code with no substantial, clear documentation to explain what's going on, you end up with a much larger effort and a lower probability of success as the maintainers aren't necessarily going to pick up on the subtelties. After all, look back at code you wrote a while back. How well can you understand it at first reading, undocumented? And you wrote it.

    You _really_ need more documentation and design than that if you're going to do a seriously good job, long-term.
  • It's true that Cathedral gets the product out the door often times, but I think the real fundamental difference in Cathedral vs Bazaar is that 'getting it out the door' is not the focus in the Bazaar - quality is.

    Commercial development says you pick some features, set a time-frame, and come hell or high water, you will make that release (you can miss, but only with grave consequences.)

    OSS development says pikc some features, come up with a ballpark as to when each feature might be done, and start working. Gradually make release as features are implemented, allowing time for the system to evolve and improve. New features migh creep in because of consequences of the development, but eventually you have a 'stable release'.

    Now what's the difference? Besides the quality of the implementation, which OSS has proven can be quite high in comparison to Commercial software, the real difference is release schedules and targets.

    Commercial development has a single goal: The Release. OSS development has multiple goals: getting features to work well. Release early and release often means that you don't really have The Release. You just release 'snapshots' as the code evolves. One reason it works because by the time you get to The Release, you have committed to a number of architecture and design decisions that might interfere with implementation down the road. However, frequent, incremental releases allow the code to mature faster, because things are fine grained.

    Thus, it's true that the Cathedral gets something out the door, and its often of lower quality, because that is the goal - getting it out the door. But the OSS world is different - The Release is really a concept that doesn't exist; the focus is just on getting features correct and then making a release.
  • According to the article:
    • In this paper, I will relate my experiences in the Cathedral to attempt to explain why the Open Source model produces better software.
    Since the author is trying to make a useful comparison between the two models, I assumed that the situations were similar. The Cathedral model usually entails others using the product (so the author can make money).

    Bazaar = loud group of people consuming and participating in things that interest them

    Also, code reviews were mentioned in the article. They don't seem very useful if you're the only one who's going to use or maintain the program.

  • Someday, the moderators will actually figure out what's really going on. That day is not today.

    Jeremiah, you missed, that's a link to part #2...he wanted part #3. Don't worry, I've made silly mistakes like that too. And it's not your fault you were modded up.

    -David T. C.

  • by aphr0 ( 7423 )
    Here's a mirror of the article..

    Open source rocks. Closed source sucks. Any other argument is mere bigotry and ignorance.
  • by Junks Jerzey ( 54586 ) on Friday August 18, 2000 @12:50PM (#844429)
    I know it isn't politically correct, but I can't help but regard that article as so much tripe. With all the "Rah, rah, we're right and you're not" nonsense you hear from Open Source advocates, you'd expect the wins to be much larger and more obvious. The Linux kernel is a nice piece of work. Ditto for Perl, Python, and Apache. Other free software is serviceable but of questionable overall quality and design, including gcc and others that I won't mention for fear of starting tangential flame wars. And there are lots and lots of Open Source projects that are complete and utter garbage, handily beat out by many commercial offerings. The evidence isn't completely in favor of either side.

    So why hasn't Open Source shown itself to be an across the board win? I can think of a few reasons:

    1. While some figures, most notably Linus Torvalds, have proven themselves to be software engineers of the highest caliber, there's a decided lack of experience among open source programmers. There's much enthusiasm, yes, but there's a pervasive reliance in staying up all night and writing lots and lots of code to fix problems. As such, there's just as much code and feature bloat in the open source world as in this so-called cathedral, including the lack of reliability that comes with such practices. It's not like thousands of people are wanting to sift through voluminous and poorly written code to fix bugs.

    2. The "scratch an itch" philosophy applied to programming tools, but it doesn't seem to be applying to other applications. There's a definite "We've gotta out do Microsoft!" mantra, resulting in people working on UIs who know nothing about UI design, and students working on applications without understanding the intended user base.

    3. For reasons that still aren't clear, there's a startling lack of creativity in the open source world. There's a lot of copying existing commercial software, but when that isn't the goal there's much floundering about. Look at the attempts to write free games. You certainly could write a game that isn't Tetris, Asteroids, Missile Command, Arkanoid, Tron lightcyles, etc., but there's no desire. The desire is to have an engineering problem to work on, not to create.
  • Although I tend to agree that deadline pressures and the like that are found in industry can put a huge strain on coders and add a lot of bugs to the process, I don't think open source is quite the panacea that he makes it out to be.

    For starters, it does suffer from exactly the same shortage of coders as industry; probably the shortage is even more acute, since industry pays so much more in most cases. We gotta eat, remember! I'd expect that even the ratio of incompetent coders to good ones is just as bad, and that the difference is that no one uses really bad open source software. It is likely also mitigated by the lack of deadline pressures, though, so do mark that as a small advantage for open development.

    With regards to fixes and features, I think that there isn't really any big advantage there for open source. Getting a patch into a release is much simpler than in the closed commercial world, particularly if dealing with software from external suppliers, but what of the features covered? They have to be ones that scratch the itch of someone who writes code. Why has it taken so long to get even the inklings of a decent open source word processor (and lets face it, StarOffice-under-new-name is more bloated but less featureful than MS Office)? I rarely write long documents, and when I do, I'm happy using LaTeX, a programming language, to write them. That just isn't an itch that we get very often. The difference is even bigger when considering stuff like special software for musicians.

    Ok, that's that, and please don't take this as a rant against the open source development model. It obviously works well, often. So does closed commercial development. Both models have big problems, and neither is vastly superior; the two can exist in parallel. Perhaps the can even have a symbiotic relation, who knows?

  • It's amazing how many otherwise rational adults don't understand that you can't gain immense productivity today without accepting significant costs in the future. I'm not opposed to working on internet time for V1.0 of a product, as long as customers, marketing, etc. realize that the cost of "internet time" is that the cycle time for V2.0 will be about four times as long, as we fix all the mistakes from the first release.

    OK, I lied - I personally would be opposed to working like that, because I'd rather do things right once than half-right n times, but as long as you're not easily bored it might work for you :)

    The embedded marketplace (where I'm at) is a little nicer right now - since you're more closely tied to the HW which doesn't change instantly, there's not as much push to turn the accompanying SW on a dime. Really, until the world at large understands that building quality SW requires the same sort of overhead as building a bridge, we'll never really have the time to do things right. Software is magic to most people, and magic just takes a wave of the wand, right?

  • Well, this may be true, but if those same 1000 monkeys were given infinate time, they could write the entire works of Microsoft. I take that back-- they only need a few hours. :-)


  • by Golias ( 176380 ) on Friday August 18, 2000 @10:40AM (#844433)
    Some unpleasant thoughts on the divide between the techies and the non-techies...

    Computers are gadgets that require specific instructions to reach specific objectives.

    People who know how to give specific instructions and objectives often become programmers, because they are well-suited to the work.

    Non-programmers are often people incapable of expressing ideas with the mathematical accuracy that computers require. That's why they hire programmers.

    If you are a programmer working directly under a non-programmer boss, it is your job to translate between their fuzzy ideas and the computer's concrete logic.

    Programmers who can make this difficult transition are the ones who get paid huge salaries to be project leaders.

    Programmers who can't do it are probably better off working for one of those project leaders, so they will be safely isolated from the sort of people who picked on them in Junior High School.

  • There are no 9-to-5ers in the Open Source community

    the code should do the talking not the hours
    it is about working smarter not harder!
  • I suggest you all read this powerfull Edito [freshmeat.net] from freshmeat [freshmeat.net].

    dudle

  • When I resigned from my position at my previous company, I needed to write a letter of resignation. The WWW is such a great thing, I even found this link [kissmyfreckledassbye.com].

    Enjoy!

    PS: After a month, I am still looking for a job though!
  • In this respect, Open Source takes advantage of the same forces that capitalism does in order to produce and innovate. In capitalism, the selfish pursuit of individual desires and needs results in a system with high efficiency at producing these things and exchanging them between those who would benefit by the trade. On the other hand, planned economies suffer from having to guess what the market desires and are in a position where they aren't free to innovate or try new things because they effect too many people. Also, they suffer from inefficiency due to the required bureaucracy and administrative structure.

    Similarly, the difficulty with commercial software is based on bureaucratic inefficiency, poor incentives to make radical changes to software, and the need to try and gauge the needs and desires of a large audience. Open Source, on the other hand, is driven by numerous individuals coding for their own selfish needs and tends to produce much more innovative software, more quickly and efficiently. Also, the same complaints are levelled at capitalism and Open Source: they are strict meritocracies where the incompetant suffer and humanitarian concerns (like feeding the hungry and getting computer illiterates online) can fall by the wayside.

    And who said Open Source was communistics? :)

    Eric Christian Berg

  • Really? All my users hate the paper clip dude with a passion. And they are by no means techies. Many of them think they are turning the computer off when they switch of their monitor. :)

    "Free your mind and your ass will follow"

  • by happystink ( 204158 ) on Friday August 18, 2000 @09:25AM (#844439)
    "Technology Prostitute: A technology prostitute is a software coder who will program anything for money. Programming language, operating system or morality of the company who wants the coded project are of minor importance compared to the paycheck."

    I'm sorry, but come on. The way that is worded is so silly.. "You don't care what operating system you program for???".. I thought it was called being a versatile programmer, not a whore. It's only being a prostitue if you maintain a really sad, religious devotion to your OS of choice.

    And noone tell me "it was a joke", becuase if you read the article, it isn't. That thing isn't funny at all. This guy is the definition of an overzealous open source nut.

    sig:

  • by dolo666 ( 195584 ) on Friday August 18, 2000 @09:26AM (#844440) Journal
    In most cases these "mandatory" code reviews turn into farcical meetings where the topic is about how code reviews are a waste of time. It becomes a self-fulfilling prophecy.

    Code reviews can only be a waste of time, if the staff are anal. You have to be willing to give up some privacy to put the software ahead of personal interests. Currently the number one enemy of open source is self importance. Look at your company, and you'll see why! It's not security... it's the damn fools who are afraid to stand up and be recognized for their controbutions, or the guys who are afraid the rest of the staff are copying.

    This does not apply in a rich social environment.

    If you are lucky enough to find yourself working for a company that permits open source, then you may find yourself walking a short plank to corporate security repromands. Be sure you know the limitations of what can and can not be made public, and where such data can be published, and get it in writing from your boss.

    There's more. What the open source movement means to me, is a freedom from reinventing the wheel, with knowledge as a tradeoff. No matter how plain your source is, someone in some place can use it to make themselves look good by copying your style.

    Take the open source movement a step further; involve Napster and we have a real use for the technology that has every record label peeved.

    Sorcery -- a program for finding source code on the internet.

    Spread the love!

    /d o0 {somehow I think mp3s would get mixed in with the search}

  • While some figures, most notably Linus Torvalds, have proven themselves to be software engineers of the highest caliber, there's a decided lack of experience among open source programmers.

    Linus has proven himself to be a competent coder but a less-than-average software engineer since the latter involves much more than mere coding. Linus's design skills and particularly his attitudes toward other elements of correct process would surely earn him negative marks at any reputable software company.

    That said, there are many great programmers in OSS. RMS, ESR, Alan Cox, and Larry Wall all produce fantastic amounts of code while maintaining quality that's above average (though sometimes only slightly so). My favorite is Guido van Rossum, who also manages to maintain truly excellent quality in his work. The thing is, though, that there are also very many excellent programmers in the commercial world. Kernighan, Ritchie, Thompson, Pike et al. did a lot of pretty good work at AT&T. James Gosling has done some cool stuff at Sun. There are hundreds more well-known examples, but none of us have time for the whole list. For every well-known example, too, there are a dozen totally unknown examples. Many companies have one or more gurus whose abilities can match any of the Big Names. Everyone in the company knows who they are, but more often than not they're utterly unknown to outsiders (unless you happen to meet them at a conference, which is why conferences are often worthwhile).

    It's a fallacy to say there are more good programmers in open-source that in close-source just because you see more. It may have a lot more to do with where you're looking, where you are unable to look, etc. than with the actual distribution of talent.

  • and, if you are a decent coder, you usually get something like:

    Hey Michelangelo, could you re-paint the ceiling? Use whatever colors you want, just get it done by next week...ok? I'll let you know what kind of faces we'd like after the progress meeting next Monday.

  • While these arguements seem valid when you read them, where are the results in practice? Why is Mozilla so buggy? Why is GNOME so bloated? Why do so many OSS programs feel hacked together and unpolished? Sure there is generally less utter crap in the OSS world but the best commercial software is usually better than the best OSS software. Theoretically OSS should be great, but where are these advantages in practice?
  • The space shuttle group is almost an extreme case, but they are a great example. They have development methodologies...and they follow them. They design before code...always. They have code reviews...where they actually review the code. They work with the singular purpose of never delivering a defect in their *ever*. Now not everyone gets to have a client like that, but you should at least appreciate their results. In an interview with the team I remember them making a comment to the effects of "We had a bug once...". That's it, they have had exactly one defect occur in their product in its lifetime, and that's good because lives could depend on it (in this case it was minor, but they were still pretty embarassed about it). -Aron My code does not run on the space shuttle...and I'm pretty happy about that.
  • Whoops. Sorry. Forgot to mention the beta and testing periods done by facilities willing to do testing. Support during the beta period is handled by the support group, and the beta testers are the actual ones who will be using the software once it's released into the field.

    APIs were designed by the kernel group (we were using MUMPS, if you know what that is) and there was a central kernel that was really a combination of what we could think of as libc (common routines and functions) and the kernel itself (other processes that were running that tied it all together). The idea behind the coding style was so that the written code could work on anything from a 20-year-old VAX to a state-of-the-art Alpha machine or even an x86 box, with M implementations that ran all over the place.

    Yes, it's least comon denomonator code, but it works, which is somewhat important if you're running a hospital, and want easy support and maintenance.
  • by Anonymous Coward

    That is something I never understood. You got some person that goes to business school or 4-6 years (if they even went to school in some cases) that didn't take 1 single computer corse in college. Yet these are the people making the biggest and most crictical judgements that will effect the software the most.

    The biggest thing IMHO is to have a solid well thought out, logical designed program on paper before you hit the code.

    My theogry is if you get the "master hackers" give them a program specs and then change them every 2 weeks, the software is going to come out a peice of junk.

    But if you take a group of "alright coders" and give them solid program specs, then the program is going to come out decent. (proved they come up with a though out design)

    Why don't big companies fire a lot of managers? Seriously, what good are they, they just end up fsck everything up. Why not replace management (or even the CEO!) with the "head programmer" that knowns what needs to be done for a quality product.

    If you look at say idsoftware, isn't that mostly run by hackers (wheather programer, artist, modeler, etc). They produced one of the most visually stunning games these year, made a decent amount of cash off of it (are there stock holders bitching? are they public?). Why couldn't this type of envoirment exist within other companeis?

    Seriously, why the hell do they need THAT MANY managers in a company, screw em, fire em and hire more IT staff.

  • Every time I see the term 'best practices' used, the project it refers to is usually suit-heavy and tech light. Because of that, I'd be surprised if it has any real meaning beyond sounding good.

    SEI/CMM can be different, though. CMM is a way to categorize a project environment from no process (CMM level 0) to a cross-department consistant and complete process (CMM level 5). This covers all stages of a project, and while it is used by technical groups, it is really a project management tool.

    CMM does *not* dictate how the process works, or how detailed it is, only that a process exists and that it is documented (CMM level 1 and above). The process may be audited, but it does cover all areas in a project except for customer relations -- from design through development, dba, test, and deployment as well as maintenance.

    Unfortunately, CMM levels 1 and above can be achieved as goals in themselves...and not to increase actual quality. CMM is usually left as an exercise for the QA/VV&T/Test group to go through...and then usually as an after thought.

    Primarily it acts as a check box on a contract -- and people don't get paid if they can't show any documentation. This isn't necessarily a bad thing -- it can save some projects and increase quality -- but it is usually just make-work and has no real value. The only thing that can make it worth something is care and attention.

    Just like ISO 9000, you can have a good process and a bad project and bisa-versa.

  • Why should I care what I do, as long as I get paid? I get paid very, very, very well.

    So if I were to offer you a competetive salary to write and maintain an accounting package that automatically maintained a double set of books -- the real set, and the one you show to the film/music artists when they demand a royalty audit -- you'd do it? Without even a moment's hesitation?

    How about an email spamming package? Would you do that?

    Schwab

  • Damn...I'm always wanting people give me *real* specs, or have them sit down and *discuss* design issues with me. But here any pseudo-formal design methodology is anathema, "code review" is an obscene word, specs are laughed at, "design issues" are solved fifteen minutes to 5:00 on Friday, and testing is logging in from home on weekends to fix bugs.

    I always thought I was lazy or anal trying to design and picture and rationalize all my decisions before coding.

    Sheesh. I wish people here would understand that a semi-formal (standardized even!) design process is not contrary to being a hacker.
  • # WRT the impossibility of gettin one's code reviewed
    Hire a contractor. Seriously.
    Temps are often used for development (IMHO foolish) but one thing that they can do with relatively little preparation is design reviews. They come in with no other commitments, no agendas, and plenty of focus: they are there to review, period.
  • by staplin ( 78853 ) on Friday August 18, 2000 @10:47AM (#844459) Homepage Journal
    I agree with you totally. I started out working for a large well established government contracting company. The whole system was slow and process bound. Code reviews were project milestones, as were such things as functional specs, high level designs, detailed designs, and test plans. The paperwork was occasionally a pain in the ass that kept you from doing any coding.

    Now I'm working for a small, relatively young company, just starting to ramp up from when it was just a coder or two. I was looking for change, and I found it. Now, code reviews are few and far between, the customer may see a design spec (if they are technically saavy), and test plans are usually incomplete or so vague you could pass them with "HelloWorld" if you squint.

    And I find myself missing a lot of the benefits of having a real process, despite the increased paperwork. As we ramp up, we're trying to instill the same kind of discipline that we've seen work at other companies, without sacrificing that ever so important "internet-time" project scope. It may seem to save time to drop all that overhead, but in the long run, the code you write is higher quality, more maintainable, and has fewer bugs than the stuff you crank out blindly...

    I read "The Mythical Man-Month" last spring, and was shocked at how much of it was almost directly applicable to many of the small companies I've seen. Too much work for too few people, poorly documented and only margionally tested.

    But to get back to the topic, OSS isn't always the solution. Many of the projects I've worked on couldn't be open sourced... too many proprietary issues, government content, or built to work with expensive, proprietary, third-party systems almost no OSS developer would have experience with, let alone have a personal copy of to test the software.

    And when OSS won't work for you, the important thing to do is identify the pieces that OSS has that you don't, like lots of peer review, and institute a way to make use of those things.
  • Acting civilized pays off in the long term.

    Or, as I like to say, "Time wounds all heels."
  • This article is posted at "themestream", which coincidentally says:

    Articles receive 10-cents per viewing (limited time offer)
    Exposure to Millions of Readers
    Complete Editorial Freedom
    Develop a Portfolio of All Your Writing

    Question: Should /. readers subsidize an author that repeats a well-worn topic around here? There is nothing new in this author's rambling... and it's an incomplete article at that!

    Is Hemos getting a cut of the dimes we're throwing the author's way? =)
  • by Malcontent ( 40834 ) on Friday August 18, 2000 @01:32PM (#844468)
    If your life belongs to you that's one thing. If you are married and have children you have a responsibility to them that is much more then the responsibility you have to your boss. In the long run you gain nothing by cheating your children out of a father and cheating your wife out of a husband. When you are old or sick your boss is not going to give damn about you.

    IMHO if you bought a child into this world it better be your top priority in life to make sure it gets raised properly, responsibly, and lovingly. Anything else has got to be pretty far down on your list of things to do.

    A Dick and a Bush .. You know somebody's gonna get screwed.

  • by DunkPonch ( 215121 ) on Friday August 18, 2000 @09:08AM (#844469) Homepage
    "There are no 9-to-5ers in the Open Source Community."

    Well, there are if you mean 9 p.m. to 5 a.m. :)
  • by antibryce ( 124264 ) on Friday August 18, 2000 @09:36AM (#844471)

    like this originally ended with "For these reason, I QUIT!"

    :)

  • by mickwd ( 196449 ) on Friday August 18, 2000 @09:39AM (#844473)
    I've always felt that a good indicator of whether a (commercial) project will succeed or not is the ratio of talented techies to technical also-rans. (Of course, there are many other factors as well, such as realistic goals and timescales, and good management).

    A project will usually succeed if there are enough skillful hackers to "carry" the rest of the team. Why are there "also-rans" writing commercial software ? Well, it's a job, and often highly-paid too. Often people's circumstances change - keen hackers can lose the urge if other things take over in their lives - having kids, for example, or other personal issues. And bad management can be a hell of a demotivation. Also, let's face it, there are plenty of bullsh*tters in our industry.

    But why is OSS better ? Because there are no (or at least much fewer also-rans). Many (most?) people write OSS software because it's enjoyable. People who enjoy coding are usually good at it. Are they good at it because they enjoy it, or do they enjoy it because they're good at it ? I don't know. But OSS, almost by its very nature, has attracted, and will continue to attract very good technical brains.

    -

    Linux - the Unix defragmentation tool.

  • I am developing a *VERY* large and far reaching application as far as what it will do. We are writing the core system, Yesterday I was explaning arrays to my boss (Its a Web Application). I am the only one with any real experience and or practice doing things like algorithm analysis but he is a very smart guy with a lot of good ideas. But..

    More over the entire *look* of the application is controlled by our 'marketing/PR/graphics arm'.

    I am vaguely aware of how a properly structured software development firm should run, but it seems there are so few people out there who run big projects who still write quality software to teach. I have no problem learning and participating in these type of reviews. I think they are good I have even bugged my boss to do so a couple of times. Every morning we just come back in and start writing code. We are already behind schedule since he does many meetings so we have put off several meetings already in lieu of completeing more code (bad idea :-(

    We were going to have an entire review and more in depth planning of our next set of components however that just has not happend its been abbreivated to sit down, sketh out the notes we talk about them for like 30 minutes and then I go hack some more... we had a week scheduled for planning.

    I cant forcibly make code reviews happen and there is no really strong CS backgrounded people here to help... Someone please give some advice on how to rescue this project before it falls into the depths of unmaintainablility and I am a disgruntled hacker.. I have strong influence right now its just me and one other developer...

    Jeremy
  • Hey, Dolly Parton was in Nine to Five [imdb.com]! She might not be an opensource hacker, but there are plenty of people in the OSS community with big jugs... too bad none of them are women!
    (duck)

    --

  • by TheDullBlade ( 28998 ) on Friday August 18, 2000 @01:39PM (#844479)
    I find that in the design stage, there is no useful distinction between "at work" and "at home". You're working 24 hours, with ideas popping up from all over the place. Might as well leave at 5 and be more comfortable while you think; for that matter, you might as well leave at noon, and sit in your back yard with a beer and a pad of paper close at hand, staring at clouds and contemplating data structures.

    However, in the coding stage, I find that immediately dropping everything and leaving at five works great, as long as things are rolling along when you leave. The best time to quit is when you're getting lots done and you know exactly what you're going to do next. Tomorrow morning, you'll know exactly what you've got to start with, and once you do that, you're in the flow again.

    If you just keep coding untill you can't get any further, you'll come in the next day, tired out and wondering where to begin. You'll spend hours just getting rolling again.

    That's been my personal experience, anyway. I've done some pretty neat stuff on 40-hour coding sprees, but usually the project died there. Sure, I'd get a week's work done in a day, maybe more, but somehow I'd never seem to get anywhere else with the hack-attack code. Eventually, I'd end up rewriting the project from scratch.

    ---
    Despite rumors to the contrary, I am not a turnip.
  • SAP is the German company that makes the R/3 business software. The company you are describing is probably SAS institute, who makes statistics/ data analysis software.
  • Yes, but no master craftsman would ever touch Visual Basic.
  • I have a question:

    Watt Humphrey of Carnegie Mellon's Software Engineering Institute [cmu.edu] pushes the Capability Maturity Model [cmu.edu], which he claims marries Total Quality Management (made famous by the Japanese) with software engineering. It's sounds a bit like Scientology to me, with PSP/TSP, reviews, and lot of acronyms, but I have seen rave reviews [accu.org] around the web.

    Could some please let me know if the CMM and the "Best Practices" being discussed have anything to do with each other?

    Thanks.

    Moderators: this isn't intended to be offtopic - I am trying to get at what "best practices" are driving the discussion
  • It's true, we could turn around our product three times as fast without all of the process overhead, and I think sometimes management is tempted to go that way.

    No you wouldn't. Without the detailed upfront stuff, test suites, full specs, etc. you'd be adding an extra day of coding, two days of integration, and four of testing for every upfront day you saved. If you were lucky, that is.

    Been there, done that. In almost thirty years in this business the fastest projects I've ever been involved with were always the ones with the most religious observance of Best Practices. The latest and more over-budget ones were the ones that cut corners, and the time & money sinks were always in the "Gosh, maybe we should have paid more attention to that up front" category.
  • Thank you!

    I get so sick of this assumption that just because you code under certain OSes to make a living, you must be some sort of bad person.

    Sorry, but an OS is a tool, and while we can all certainly bitch about the tools we have to work with, does it really make sense to take a lower salary to work at a place with better tools? I mean, what would we say about a construction guy who refuses to work at a site because the hammers suck?

    Personally, I think the worst thing you can do is only ever use one OS. That makes you a one dimensional coder. A good coder is someone who can make quality software in any environment. And your never going to get there unless you've had some experience with crappy environemnts.

    When I'm at work, I work with one set of tools, chosen by my employers for business reasons. They are not the tools I'd choose, but then, the tools I'd choose won't work for their app and their business model. It is pretty hard to develop Windows GUI software under Linux. When I go home, I develop on a Linux box because that has the tools I enjoy the most. Am I a whore? Should I instead refuse to work with anything but the best, mortgage be damned? That, to me, is a fairly idiotic idea.

  • I am a begining perl codie. and if it wasn't for open source i would not have made all my deadlines. Finding scripts that almost do what i need them to and fixing them to what i need is probabley one of the biggest productiviy kicks i have had wile working.
    I have heard family talking about how they think open source is bad, there arguments are usualy the following "people can see your code and find ways to hack in" but i just mention peer review and then they understand.
    sorry about my spelling, i just got the warm fuzzies and needed to speak :)

    Gentleman, you can't fight in here, this is the war room..
  • by ethereal ( 13958 ) on Friday August 18, 2000 @09:45AM (#844498) Journal

    I work for probably the most process-heavy, Dilbert-inspired corporate monolith around. We do have an intricate process involving frequent peer reviews of code, documentation, and architecture, combined with gate-keeping by senior designers who determine what is good enough to go into the final product. And y'know what, correct application of the correct process really does make our lives as software engineers easier. Reviews of requirements and architecture up front reduce the amount of re-coding we have to do at coding time, so much so that our time breakdown is something like 40% up-front documentation (including test cases), only 10-20% coding, and we have the rest of the time left to exhaustively test the thing. Sure, things aren't always so rosy for every single project and feature, but overall my impression is that you can get a lot of gain out of your process if you invest in it up front.

    Some of the gains aren't evident right away, but full test case suites, requirements, and architecture documents make your results more maintainable. We still have 10 year old code in our product; because it was documented, we know that it still works, and we have tests to prove it. We're not in a situation of "don't change this magic code or the product will never work again".

    The most important thing, though, is management support. It's true, we could turn around our product three times as fast without all of the process overhead, and I think sometimes management is tempted to go that way. But without documentation, etc., we wouldn't be able to keep up that pace for 10-15 years on a single product. Our release dates would gradually slip more and more, as the combination of all of our earlier shortcuts came back to hurt us. I see this happening to a lot of PC software companies (I'm in embedded systems) and it's painful to watch.

    You really have to have a corporate-wide committment to software quality, up-front design and documentation, and a repeatable process, but if you're really in the software business to stay, I think it's worth it. SW engineering process is the stuff that was both tremendously boring and incredibly obvious in school, but it does work well if you take the time to use it.

  • One of the things that is more common in the Open Source world is clear statements of requirements and design documents. Either that, or the vague and incomprehensible ones disappear rather than being completed and archived by mandate. The reason they are clear is that if they exist at all, it is because they were written with the deliberate intention to communicate with other developers. They aren't written because the process says we have to have them. They aren't written to communicate with a vaguely understood audience. They aren't written to adhere to a formal template. And, in a sense, the working code serves as documentation for the design. If you want collaborators, you'd better make it comprehensible.

    When you post a note to a mailing list or newsgroup that starts with "Hey, I want [insert program name here] to do [insert feature here]" and ends with "Here's a patch that does about a third of what I'm trying to accomplish," that is a design in a real sense. The reason that source code is human readable is that it serves as a means of communication. It documents what the program does, for other developers and for the original author across the gulf of time. Open source projects are often written with an explicit understanding of that.
  • >>No. People write them because they are unable to think
    >>of anything else. I don't know why open source programmers
    >>are devoid of creativity, but, sadly, it seems to be the case.
    >
    >Bzzzt. The reason those games are coded is because people
    >grew up with them, period. No one would play Missile Command
    >instead of DOOM otherwise. They deliberately have the same
    >graphics and everything. In fact, many of them are copyright
    >violations, or would be if the company cared.

    No, you are completely wrong here. What I'm saying is that Linux game sites are filled with endless clones of old arcade games, and almost all of those games don't even live up to the originals. You would expect the free availability of Linux to foster the next Sid Meier or Shigeru Miyamoto or Dani Bunten, but it isn't happening. Linux game developers are coders, not designers.
  • Congratulations on your business, keep up the hard work.

    When I think of the years I wasted sitting in classrooms listening to some stuffed shirt drone on about things I already knew, I wish I'd had the courage to drop out at junior high. School is nothing but an attempt at brainwashing to create perfect corporate wage slaves. Any learning is purely coincidental

    This I highly disagree with... college is the one chance you have to truly think, to exercise parts of your mind that the corporate does not tolerate. I am in business for myself too, and turned down several corporate jobs that many would have taken... like you, I do miss the people at school, but unlike you I also miss the cerebral part of it. I guess you're right in part, lectures can be boring and unstimulating (and there is ever so much busywork)... I dunno, though, you still get to think about things more abstractly than you do in the real world, and you are surrounded by smart people. I think that's what I miss most, being surrounded by intelligence. The only problem with the real world is that the average person has... average intelligence! :-)

    Regardless, I'd recommend that you finish school... I know it may seem like a waste now, but I'd wager you'd be glad you did it down the road... I have some friends now who stopped school mid-way through and are trying to get back into while working full time... that is a hard and long process.

  • You're right; I forgot that. Too enamoured with the idea of only working 9 to 5 to remember ;)
  • So if I were to offer you a competetive salary to write and maintain an accounting package that automatically maintained a double set of books -- the real set, and the one you show to the film/music artists when they demand a royalty audit -- you'd do it?

    Just how competitive are we talking?

    Without even a moment's hesitation?

    Well, obviously I'd hesitate long enough to make sure that you were offering the maximum I could get out of you...
  • I can only say, "read the book"! This sort of argument is dealt with in great detail.

    ... and it *is* a funny name, I agree ;)

  • by Greyfox ( 87712 ) on Friday August 18, 2000 @02:58PM (#844519) Homepage Journal
    My experience with commercial software coding is that many developers simply don't care. I've seen coding atrocities committed which will never see the light of day. I've had to maintain some of them.

    You don't get stuff fixed, code is usually an unmaintainable time bomb waiting to go off, and you see constructs that make you wonder if the coders ever went through basic CS. Data storage is rarely thought out and structures resemble rats nests. God help you if you have to deal with C++ code.

    In short, the commercial development world is a Mongolian Clusterfuck of immense proportions.

  • the difference in open source is sharing of course. the final product is shared. in a sense open source is like academia - the competing drives to produce code for oneself, plus the need to share said code with peers. its neither capitalism nor communism but something that takes the best of both and merges them. the whole is greater than the sum of the parts etc etc. i guess its the exact same thing as the university model which neither makes a profit nor a loss but contributes to the average population nonetheless.
  • The idea that large corps don't review their code explains why their click-wrap licenses claim they can't be held accountable for thebugs.


    The article and this comment remind me of how pleasant my current boss' attitude is. As far as he is concerned, code reviews aren't about the formal process. Reviewing code is something programmers do. If you are going to work on it, you have to review it to understand it. As a result, he encourages us to raise issues with possible bugs and ways to improve it. But then, it's nice to be able to say, "My boss writes good code."
  • by ethereal ( 13958 ) on Friday August 18, 2000 @10:09AM (#844528) Journal

    Some random thoughts:

    • You need management support pronto, because what you have to do is get enough time to understand what you already have done, look for big architectural flaws that will force you to work around them for the rest of your life, and think about where you're going. If you don't control your schedule (and it sounds like you don't) you have to convince the folks who do that there is a benefit to proper planning.
    • If this is more than a one-off project, if somebody will have to maintain it, you should try to get some more time to document your architecture and design decisions and write some tests. You can get tools that make generating test & test drivers a lot easier. You can save time now by not documenting and not producing test cases, but you will lose two or three times that much time in the long run. If you know there will never be any maintenance on your product, then you can't make as good of an argument for these things.
    • Tailor your process. You don't have to do line-by-line Fagan reviews of your code if the extra attention to detail isn't finding more bugs, for example.
    • Contrariwise, if you find that you're using code to paper over serious architecture flaws, then you know to have more complete architecture reviews next time. But your time in peer reviews has to be well spent - if you're on a deadline, have everyone review ahead of time and just hit the biggest issues at a meeting, with minor issues discussed over email. If you make peer reviews a hell to conduct and attend, you won't gain anything from them. If you make reviews easy to attend, you will generally get good comments. Invite subject matter experts if you have 'em; even if they aren't on your exact project (or sometimes because they aren't) they often bring up vital issues that you'd never think of.
    • Document how you've tailored things, so that everyone knows what steps they have to take their code through to get it in. Enforce some sort of policy so that there is at least one peer review of everyone's code before it goes into the product, because once it's in you may never come back to look at it until it's too late to fix it.
    • You really need management support or ownership of your schedule in order to make time to do these things. Once you have time to do these things you will find yourself worrying less about bits of code and more about architectural issues up front. This is good because anybody can write code, but not everybody can take the larger view of the product as a software system and pinpoint the weak links or missed requirements in your plans.
    • Marketing owning the user interface isn't necessarily a bad thing. What's bad is if they are allowed to dictate your schedule so that you end up using poor design or bad code to make up the time. Again, you need management support so that you can tell marketing there will be a cost for any new/changed requirements, and the cost will include time to fully document those requirements (including peer review if possible), time to implement them, and time to test them. Marketing isn't automatically bad, they just don't have any SW engineering experience to know what is possible and what the costs of things are. You have to reiterate this to them until they realize that their actions have consequences. It's sort of like training a puppy.
    • If you're spending most of your time coding, trying to get it to compile, or debugging, then you didn't spend enough time up front. Aim to spend most of your time testing, because you'll need it. The worse your development process, the more test time you'll need unless you want to compromise product quality. This is another good topic to bring up to management.
    • Adding staff to a late project makes it later - but not always. If you divide up responsibilities, you can have new staff do coding from your design documents while the more experienced staff write test cases or look into design/architecture problems. It sounds like your project might be in that sort of situation - you can get coders fairly cheaply (in terms of time, although maybe not in terms of salary) but people who understand the SW system as a whole and are conversant with the design philosophy have better things to do than generate code or test.
    • If you have really good requirements and/or test cases, you can hand off testing to an outside group entirely, but that requires more time up front and more budget. It is a good way to find a lot of bugs quickly, though.
    • Wow, that got long. Hopefully some of it was useful.

  • by NaughtyEddie ( 140998 ) on Friday August 18, 2000 @10:10AM (#844531)
    Anyone heard of XP (eXtreme Progamming)? See e.g. "eXtreme Programming eXplained" by Kent Beck. One of XP's ten "rules" are "only work 9 to 5".

    I acknowledge many of the problems this guy is talking about, and XP is a promising way of making this work in the real-world of software schedules. Also, some organizations (e.g. Data Connection Ltd, a former employer of mine) have amazing setups. Code review is mandatory - they don't try and tart it up as "objectives" either, it's part of the discipline of working professionally. It confounds me how many paid programmers don't even understand the meaning of the words "professional" and "discipline"!

    Incidentally, I don't hold with his notion that a thousand developers adding features willy-nilly is better than a tight team with a coherent design philosophy, but - hey - you can hide all that crap behind a user interface and no-one's any the wiser. Hmm. Didn't Microsoft try something similar?

    [The opinions stated in this article are not endorsed in any way by Data Connection Ltd. Just thought I'd make that clear!]

  • Do you think the company cares as much about clean code as the developer does? They just want the product to get out and make them money.
    Commercial=revenue, OSS=great code
    You take one road or the other. That's it. As of now there is no middle. Try to tell the average Office Suite user that OSS is better and they'll most likely reply with, "I don't care."

    Even the samurai
    have teddy bears,
    and even the teddy bears

  • Did whoever moderated this post even *READ* what I wrote??? This is exactly what the person who wrote the article was talking about.

    Projects that sacrifice delivery time with proper development, its happening I was hoping to provoke a little discussion and gain some insight into the problem, instead I am modded down. Just GREAT I hate it when people moderate without reading the damn articles.
  • Fred Brooks wrote about this in The Mythical Man-Month. I believe he called it the "second system effect". It does indeed exist inside Cathedrals as well.

    One of the things that mitigates its effects in open source is that few projects get started without a working prototype. A lone programmer scratches an itch first, and then goes looking for a pool of users/co-developers to find and fix bugs and extend the concept.
  • No offense, but this article didn't really have anything new to add to the discussion of why it's better to have access to source code. For the record, I'm in favor of access to source, and I don't particularly care what license it's under as long as I can use it legally to learn something new and interesting (which is why I refer to it as source access rather than the politicalized terms "free software" or "open source"). I do want to reply to two comments he made, though:

    In the hands of countless developers, what can be done with an Open Source project is unlimited. It simply takes time to add whatever functionality you (or the boss, customer, etc.) wish were there.

    Not every free software developer can work on every free software project. I see a lot of postings on /. whenever somebody complains about missing functionality in an application, or a utility that doesn't work on their system. Invariably there are several comments along the lines of "you have the source, fix it yourself." That's great if the problem lies within my skillset. Should a Perlmonk need to learn OpenGL to fix a graphic utility he or she uses? Should a GNOME guru need to learn m68k assembly to write an installer for OpenBSD? OK, those are sort of contrived. But the point is that not everybody knows everything about everything, and sometimes you just want to get work done. Apple didn't sell 4 million iMacs on their looks alone, at least not the ones in use where I work [nih.gov]. Some very smart people use them to do all kinds of smart scientific things (perhaps you've heard of one or two?), and some of those people do write their own code because nothing else does what they need it to do. But by and large, they're using off the shelf products to get their real work done. Nothing wrong with that.

    Contrast this bureaucratic model with Open Source. If I want to add a feature to an Open Source project, I simply download the source code and add the feature. (Actually, it's a little more complicated than that, but you get the idea).

    This might be true for smaller projects with only a few coders and easy-to-navigate license terms, but it doesn't hold up for projects like Mozilla, StarOffice, or the Linux kernel. Many projects have specific coding rules, accept changes from specific users only, and are simply too damn big to just go in and add one or two features. I'm not saying it's impossible; clearly it's not. Just look at how far Mozilla came in two years (a relatively short time for such a large project, one that had to start over from scratch no less). I just feel it's important to stress to newcomers to the various source access movements that it's more than just downloading the code, rewriting it, and making history. There's publishing it afterwards (depending on the license/project), integrating it with other code, testing it (optional, sadly, in too many cases), and the like. Getting involved in a medium-to-large project can be a serious investment of braintrust, just as it is with sequestered-source projects.

    I can definitely agree about the tech-support thing, though. It's what I do for money while I try to go back to school to learn all the things I should have been paying attention to before going out into the real world, and let me tell you: he was lucky to be considered a step above the slugs....

  • Which market?

    That's the whole thing with open source. The code is not written for any nebulous market. It is written for the coder that wrote it. Thus, in the bazaar the code satisfies the target `market' exactly. So what if that market consists of 1 person? If the next person to use the code isn't 100% satisfied, what's preventing the addition of a feature and an option switch?

    Sorry, but while what you say is basicly true, it is totally irrelevant.

    Bill - aka taniwha
    --

  • It was then that I realized I could never tolerate the corporate environment, and stopped pursuing the university degree I'd need for such a job

    So what are you doing now? Don't give up your schooling, that's a decision you'll regret later in life...

  • If so then you have a ton of major problems in your code and just don't know it yet. I wish I was kidding. :-(

    A site I think does a pretty good job of helping Perl newbies learn is Perl Monks [perlmonks.org]. OTOH I like to answer questions there so I am biased. :-)

    Cheers,
    Ben
  • I've tried to explain my position before (I like Linux, btw). From the article:
    Open Source simply replaces one pre-approval process (focus groups) with another, more inclusive, pre-approval process. In order to add a feature to an Open Source product, you must be able to write code.

    This is how I interpret the above line: Only features that serious, hardcore programmers deem valuable will ever be written into the software. By definition, things like the Office Paper Clip will never make it. GREAT! I can hear thousands of Linux hackers going crazy. Listen, everytime I sit down at some computer neophytes' desk to show them how to do something in Word/Excel/Outlook, that little dude pops up, I hide him and say "I hate that &*($!," they say "you're kidding. I love it."
    ---

  • Mass market busking [boswa.com]

    "Here, free software! Give me money, and other people will give you free software because they see I got money, so they think you might give them some." It's that simple.

    Acting civilized pays off in the long term.

    ---
    Despite rumors to the contrary, I am not a turnip.
  • Hold on a second...

    He argues that a product is more functional if you can do more with it. Clearly the implication of benefit is for the users.

    That's all fine and dandy, except that his reason for why OSS wins is that developers can do limitless things with the software! I'm sorry, but users don't give a damn if limitless development/developers could happen to a product. They want to use it now.

    He almost touched upon my biggest issue with OSS when he mentioned that development is done to "satisfy a developer's itch." I agree with that, which is where the problem comes from. Feature bloat. Everyone wants to be the guy to add the coolest new gadget/widget/feature. Not too many people like fixing bugs/memory leaks. Likewise, I'd rather write my own code then peer-review your feature (which I never liked anyway).

    It's the same old argument:

    #1: OSS is better because if there's something wrong you can fix it. Unfortunately the average user can't. He's just as dependant on someone else - only this time they're anonymous.

    #2 OSS is better because everyone can examine the code and make sure it's "up to par" in terms of security, efficiency, privacy etc. Unfortunately, you have to people with strange interests to scour those things for free.

    He hasn't revealed any insight, except that he was frustrated by not getting his own features into his company's software.

    Cry me a river.

  • ...according to the spec I was given in a few days, and went to explain it to the guy in charge.

    He took one look at my documentation, said "This'll never work." and apparently got stuck on that setting. I explained that it was done, and it did work already, but that didn't faze him. Eventually, he got me to throw away all my work and rewrite it so it didn't meet the original spec, ran slower, and used more file space.

    The reason? He couldn't understand anything with a data structure more complicated than an array, and didn't believe in using dynamic allocation or pointers. Ever. Seriously.

    Hundreds of globals, gotos in every function, 20,000 line undocumented source files, hidden arbitrary limits to data size: all that was okay, but no linked lists or trees. Certainly not something as complex and mysterious as a hash table! Stuff like that causes bugs!

    I won't mention where this was, but if you ever wonder why an automated telephone response system is so screwed up and useless, maybe it came out of this shop.

    It was then that I realized I could never tolerate the corporate environment, and stopped pursuing the university degree I'd need for such a job.

    ---
    Despite rumors to the contrary, I am not a turnip.
  • No doubt.
    • Boss: I've got a new project for you. It involves VB and MS-Windows.
      Employee: No, I won't do it. I'm morally opposed to using VB or MS-Windows.
      Boss: Okay......
    There's... not being able to use a language or OS because you don't have enough knowledge... or asking your boss to reconsider because you think they're the wrong tools to use in that situation... but this is entirely different.
  • marketing SHOULD set the *look* of the product. That's their job (along with some help of UI experts hopefully). They will, of course, change their mind 20 times. That is also their job. None of your business logic should be in the UI anyway, so that shouldn't matter much.
    ---
  • It's no wonder that almost all software projects are over budget and late, the budget and timeframe were unrealistic to begin with and the requirements and specifications never existed!

    This isn't just a coding issue. I run into this in other fields. There is a real tendancy for people (particularly managers) that anything they don't know how to do a) is easy and b) takes very little time. The managers look at you working and think "all they're doing is sitting there typing - that's not very hard" because nothing they do on a computer is very hard.

    I used to have managers hit me with extra task after extra task, claiming "It'll just take fifteen minutes", because any task they didn't do themelves must be trivial. I developed a much used mantra "Nothing, ever takes 'just fifteen minutes'".

    The shame is, people managing projects rarely understand what they want or what the job entails, only that they've been told to produce a project. And, since their bosses don't have to deal with it, they've been given the same too-short deadline, too-vague instruction problem they're passing on to you.

  • Actually, in case anyone cares, this group was spun off from the mother company several years ago and is no longer an IBM dept.

    Another interesting fact about this group is that the Capability Maturity Model was built around the processes that this group implemented, and they are the only software organization to hold a CMM of five for a sustained period of time.

    However, their supreme reliability comes at the price of a cost-per-line almost double that of your average delivered software project. I guess you could say that their process is overkill for projects that don't require this kind of reliability.
  • by goingware ( 85213 ) on Friday August 18, 2000 @06:06PM (#844564) Homepage
    I've been working as a programmer for 13 years now, and for most of my career, the way I described it is this:

    I've spent most of my career fixing somebody else's broken code.

    This is not to say that my own has always been of the very highest quality, but in fact I decided early on to try to come to a fundamental understanding of what was wrong with software development, to get very good at debugging (I say that debugging is a specialty on my homepage [goingware.com]) and to learn to write better code.

    In my early years I was initially very shocked at what I'd discover in production use at companies. Over the years I just learned that that's standard practice, in commercial software, in-house software, and even in scientific software (where I have become convinced, because of my experiences with high-energy physics data analysis, that many scientific papers are published with erroneous measurements because of software bugs).

    Early on I read that something like 90% of software development is spent doing maintenance programming. Some of this is doing legitimate upgrading, but a lot of it is just spent fixing bugs, and even a lot of time spent doing upgrades would be more productive if the code were of better quality.

    After reading this figure and having so many experiences with software bugs, both other people's and my own, I decided very early on to get very good at debugging.

    One of the first things I did was adopt the regular use of "lint" for checking my C code. I would integrate lint targets into my makefiles and after editing a source file I would type "make lint" before compiling to objects and lint would check all the files that were out of date with the object modules. Pretty quickly I got to where I could write code that was nearly always lint-clean - but the existing code I worked on would make lint scream with hundreds if not thousands of complaints, often serious things like variables being used before they are initialized.

    One of the first solid clues I got about software quality came from Robert Ward's book "Debugging C" - now out of print, it predated the common use of source code debuggers and talked about how to write your own stack crawls and other tools.

    Ward emphasized the use of the Scientific Method in debugging, and because I was trained in physics, this came very naturally to me; before that I'd mostly floundered and used printf a lot.

    I've gotten very good at debugging and have even worked full-time as a debugger at Apple Computer [apple.com] where I was a "Debug Meister" and my business card gave my title as "Cybernetic Entomologist".

    I can easily get highly paid consulting work doing debugging for companies desperate to ship a product (and have in the past) but I don't really like to do it for various reasons, some of the same reasons I quit my debugging job at Apple.

    One is that if I only do debugging I don't have something to point to at the end of the day and say "I wrote that". I could say "that works because of me", but sadly there's usually lots of bugs left that I didn't have the time to find so I don't really feel proud of the result. The other problem is that the bugs are usually not there because of something interesting, it's not like the code is mostly good but there's some subtle flaws, rather the code is a heap of dung and I can go in with a pitchfork and do debugging wholesale until I get tired of it and the client or manager decides the rate new bugs are being found is low enough they can feel OK about shipping it.

    I don't feel good about contributing to such shoddiness. If a company is not good enough to support quality in their corporate culture I don't want to come in and put on a band-aid for them. It would be an entirely different thing if a company hired me to restructure their development process so that quality was a goal that was achieved through direct application of process but gee whiz no one has ever asked me to do that for them.

    I do have to say though that the best thing that ever happened to me is that I became a "technology prostitute" as the author of the original article puts it. One benefit of this is that the process is entirely of my own creation, and almost all of the work I've been given has been to write entirely new products from scratch, so I can engineer in quality from the beginning.

    Here's a few recommendations I have. Get good tools. Besides a compiler, editor and debugger, you need a static code checker and you also need dynamic testers. The ones I know about are (I haven't used them all yet):

    • PC-Lint [gimpel.com] static code checking for C and C++. It runs on Windows but Flexe-Lint comes as shrouded source code and is highly portable.
    • Spotlight [onyx-tech.com] dynamic tester for Mac PowerPC - I use this every day and recommend it highly
    • BoundsChecker [numega.com] dynamic tester for Windows
    • Purify [rational.com] dynamic tester for Unix (but apparently not Linux) and Windows NT
    • Optimizeit [optimizeit.com] dynamic tester for Java - do you know many Java programs have memory leaks? Can you understand why? Not just Java but any garbage collected program.
    There's also memprof and electric fence, which I think run on Linux. It would be possible to modify gcc so that you program when built with debugging flags and linked with a special library would have all memory references boundschecked before allowing the access. This could work with both read and write access and would catch overruns by as little as one byte - Spotlight does this by patching the executable powerpc code and rewriting the debugging symbol table.

    Finally, to really come to understand the software quality problem in the industry and what you can do about it, read The Forum on Risks to the Public in Computers and Related Systems [ncl.ac.uk] also available on the Usenet News as comp.risks [comp.risks]. The book The Software Conspiracy [softwareconspiracy.com] exposes the complete disregard the commercial software industry has for serving the consumer by providing quality products - I haven't read it yet but it looks interesting.

    A very interesting methodology that emphasizes personal responsibility and puts the fun back into programming as well as maintaining quality from the very start is Extreme Programming [c2.com]. I'm starting to adopt extreme programming (the the extent a one man operation can - I can't work in pairs :-/ ) and find it a tremendous benefit.

  • by dmccarty ( 152630 ) on Friday August 18, 2000 @11:43AM (#844569)
    If Open Source was 1000 monkeys typing randomly in diff files, I agree that it would be doomed to fail.

    Oh no! We're doomed!
    --

  • DON'T FEEL GUILTY! The eight-hour day was designed for labor with minimal mental effort.

    Most people are mentally productive and efficient about two hours a day on a regular basis. The rest of the time at work for a white-collar worker consists of doing their core job at less than full competence, completing the mundania necessary to the job, and slacking.

    And of those things, the slacking is the most conducive to keeping you mentally rested enough to be able to have those productive hours each day. Working hard at mental work eight hours a day is a recipie for producing eight hours of low-quality mental work each day.

    Now, this is also why enjoying your work is valuable. If you truly enjoy your work, you will get some of the benefits of relaxation from the work itself -- in that case you'll probably have no trouble putting in four hours of work per day and will desire to put in those hours anyway.

    Anyway, they're not paying you for eight hours of work; that's just the traditional way of accounting the pay. They're paying you for the two hours of productivity, even if they don't understand it.

    Steven E. Ehrbar
  • >gcc? GCC? GCC? Are you insane? gcc is one of the best compilers out there.

    Have you ever looked at the gcc source? It's amazing that it hasn't collapsed under its own weight. It's very scary that so many people are reliant on it.

    >>While some figures, most notably Linus Torvalds, have proven
    >>themselves to be software engineers of the highest caliber,
    >there's a decided lack of experience among open source programmers.
    >
    >Sadly, you are right. Also sadly, however, it is the same in commerial shops.

    You usually don't find commercial shops run by a bunch of newbie programmers. There are old hands with an eye on things. I find it's pretty typical for new programmers to jump onto a project with all sorts of great ideas...ideas that show they don't really understand the architecture, or that show they aren't thinking about reliability. Most open source projects are completely lacking the experience factor.

    >And, may I point out, the games you listed are classic games.
    >People like them because they grew up with them.

    No. People write them because they are unable to think of anything else. I don't know why open source programmers are devoid of creativity, but, sadly, it seems to be the case.
  • by sfbanutt ( 116292 ) on Friday August 18, 2000 @09:16AM (#844574) Homepage
    I've been a professional software developer for nearly 20 years now and everything he writes rings very true. It's actually worse than that in many cases. I can count on the fingers of one hand the number of times I've actually been given a specification for a project, or even a reasonable list of requirements. And the established deadlines are typically even worse, you end up doubling or even tripling your estimates for project length simply because you know that whatever time you ask for will be cut in half (or more). Code reviews are something you read about in software engineering texts, if you actually know someone who is able to do them, you feel privileged. It's no wonder that almost all software projects are over budget and late, the budget and timeframe were unrealistic to begin with and the requirements and specifications never existed!
  • I guess Stephen King's experiment has taken off.

    Maybe he will only publish the third atricle if a certain number of employees of said "major software company" quit.

    :-)
  • In the cathedral Engineering decision are often made by people who have no idea how to develop software

    In the bazaar, design decisions are often made by people who have no idea what the market wants.


    Sorry, I'm not trying to be incredibly inflammatory, but my statement seems to be about as true as yours is.

  • by Anonymous Coward on Friday August 18, 2000 @09:18AM (#844592)
    The IBM group which writes code for the space shuttle works 9am to 5pm almost exclusively, and they produce very high quality code. There's nothing wrong with coders who work 9-5, theres nothing wrong with coders who have lives outside the office. Sometimes, they're even more productive and predictable than those who put in, and brag about, their 36 hour coding stretches.
  • by remande ( 31154 ) <remande@bigfoot. c o m> on Friday August 18, 2000 @09:13PM (#844593) Homepage
    There are at least two types of code out there: money-critical (failures cost money) and life-critical (failures cost lives). The Shuttle code lives clearly in the second category; most of us are programming in the first category.

    Building life-critical software is fundamentally different from making money-critical software. This is because, for life-critical software, people will trade off time-to-market and feature base to get quality (absence of bugs).

    Open Source doesn't even provide this. Think about it. If you needed a pacemaker (life-critical), would you want it to run on Linux? I wouldn't. Linux has uptimes of over a year; I need uptimes measured in decades. Frankly, no top 10 OS is qualified for life-critical stuff. I'll trust Linux with my data, even my money, but not with my pulse, my shuttle, or my airliner avionics.

    When you build life-critical software, you do it anal-retentively, line-by-line, and work at a snail's pace. The result is something that does one thing (or a few things) very well under incredible circumstances.

    Try working that way in the commercial world of money-critical software, and you will be beat to market every time. And not by a little. I bet that it would take decades to build a word processor to compete with today's stuff, if it had to have life-critical reliability.

    The code is different, the process is different, the people are different.

  • by Alternity ( 16492 ) on Friday August 18, 2000 @09:19AM (#844594)
    In the cathedral Engineering decision are often made by people who have no idea how to develop software, do not care how to develop software and would by no mean consider it appropriate to consult the coders before making decision.

    Thus you end with administrative request about a near impossible feature to be implemented in an astoundingly short delay withouth enough resources. That added with the fact that a lot of programmers in the Cathedral are only waiting 5pm to go back home (and hack in code they really like??) forces people to cut around the corners and take shortcuts. After all the holy priests of the cathedrals do not care if the program is working fine... all that matters is if the feature is implemented and if the software has enough eye-candy to impress other priests in other cathedrals who will pay for our stuff.

    Of course it might not be that way in all cathedrals... but the more I move from catehdral to cathedral the more I become atheist :o)
  • That is something I never understood. You got some person that goes to business school or 4-6 years (if they even went to school in some cases) that didn't take 1 single computer corse in college. Yet these are the people making the biggest and most crictical judgements that will effect the software the most.

    If the business types are competent, they don't have to know a thing about how to make software. They do need to know what features will sell, and how well. That is, they need to grok the customer. That's their job.

    If the techies are competent, they don't have to know a thing about the customer. They have to understand what the business types are telling them to build, and to build it to their satisfaction.

    You don't want techies telling the business people what features need to go in (unless the software is to be consumed by techies, perhaps), and you don't want business people telling techies how to put the features in. Once that happens, people are stepping on each others' toes.

    Sure, the spec is going to change. The business reality is going to change. What's going to happen in three months in this biz? I don't know, and I'm a Slashdotter! There are several ways to handle this, and every good software engineering methodology, from extreme programming to Rational Unified Process, takes that into account.

    Now for the dirty little secret. If you have competent businesspeople and competent techies, this will work. If you have incompetent businesspeople, or incompetent techies, the product will be a hard sell. The nasty part is, unlike a McDonald's, no amount of process or other idiotproofing measures can change this. I've seen process go out of hand to try to reign in incompetent businesspeople and incompetent techies: it just doesn't work. Any process heavy enough to keep an incompetent from doing damage (to say nothing about making progress) is so heavy that the competent people must ignore it in order to get anywhere.

    There are exactly two ways to handle incompetent businesspeople and techies on a software project: training and severence pay.

  • The entire point is to get the product out and making money. If making it clean doesn't help that, don't make it clean!

    Unfortunately, the company often doesn't realize that keeping it clean helps you work faster, and that gets the product out the door faster. Getting the product out the door faster is something they can understand. Clean code, to people who read code as line noise, sounds like some esoteric virtue, so they don't care about it.

    The solution is often to make the code clean as your personal version of "I'm working as fast as I can". I've fallen into the trap of trying to speed up by sacrificing cleanliness. Unless you are doing a sprint measurable in hours, this doesn't work; you end up sloshing through mud.

    The analogy I've heard comes from house painting. Ask a company to come in and paint a room, and the first thing they do is come in with drop cloths, blue tape, and do a lot of things that are not painting. This is patently unneccesary; if they are careful, they won't get paint where it's not wanted. but once they do all this prep work (the equivalent of cleaning code), they can paint fast-fast-fast. The prep work actually saves time.

    Similarly, cleaning code saves time, usually immediately. It makes you faster at coding, reduces the debug cycles, reduces the amount of time you have to listen to QA/management/customers rant on about bugs.

  • by tylerh ( 137246 ) on Friday August 18, 2000 @09:21AM (#844598)
    A nice piece, but he missing one of ESRs key points: Economic Viability.

    His discussion of peer approval being crucial reinforces ESRs insight [tuxedo.org] that a "gift economy" lies at the heart of Open Source culture, but the author gives no hint of how his (or anyone elses) open source coding would be compatible with long term economic self-interest

    ESR does so in the Magic Cauldron [tuxedo.org]
  • Usually, customers aren't individuals. They are usually large outfits, buying in bulk.

    Sure, they can't fix things when they break. But they can hire somebody.

    If I am a dot-com, I have a commercial web server, and it crashes under certain circumstances, I can only work with the vendor. If they want to fix it and can, I'm in luck; otherwise, I'm hosed. If I have an OSS web server, and it crashes under certain circumstances, I can hire people to fix it. I probably won't actually put them on my payroll; I'll just look for a contract.

    The fact that developers have so much freedom with an OSS product directly implies that users with money can get said developers to push the software in their preferred direction. In the commercial market, the customer is stuck with the prayer method of software support.

  • Nope. Here [themestream.com] it is.
  • Egos are amazing. More than once, I have sent email out to my co-workers noting that I wrote the bug that was causing the day's grief (usually with "Mea Culpa" as the subject line) Every time, people come back to me and are amazed that I am one of the few developers willing to admit that I wrote a bug.

    If enough developers can get over themselves, and admit that they make mistakes once in a while, we might get a little further...

  • by dmccarty ( 152630 ) on Friday August 18, 2000 @11:55AM (#844604)
    The Cathedral does have a very good ingredient in its mixture that OSS programmers don't have, and that is focus. Corporations have the ability to assign teams of 50, 100, 300 programmers to an enormous project and finish it in 6 months to 1 year, something that would be daunting to the Linux community (look how long the 2.4 kernel update has taken).

    Now I know that people will want to jump in and say "But look at all the bugs! Look at the poor quality! 2.4 will be released as better software because of the delay!" but the fact remains that the Cathedral gets products out the door while most OSSers live in the near future. "Just wait till the next version," we say, "when it will have [X] feature."

    To a large degree, when we buy a piece of software, like Word 2.0, we don't just buy Word 2.0. We also buy into the idea that we bought a piece of software that works decently now, and the creator is committed to eventually upgrading the software until it's the word processor to end all word processors. And so we're at Word 9 today, and the cycle continues. When software is looked at this way, getting a product out the door becomes more important than waiting for people to finish it in their free time. That's why Cathedrals (read: corporations) will make money and that's why they'll stay in business.

    I'll add two caveats to the above:
    1: This doesn't necessarily apply to security products. Thank God Microsoft didn't introduce us to sockets.
    2: Another reason why the Cathedral will always exist is because of competition. If you have highly sensitive secrets it doesn't make sense to publish your source. Security through obscurity might not be a good thing, but that doesn't mean I go around giving everyone copies of my door key and daring them to find out where I live.
    --

  • I work for a big company that works for the government.

    We are a contracting firm, so just because things are good where I work doesn't mean they aren't bad somewhere else or glorious in a third place in the same company.

    I find the processes we have to be pretty thorough and not too excessive.

    The funny thing is that people always complain about 'government inefficiency.' My contract, like many government contractors, gets an 'award fee,' which is the only money we get over cost. If we drop code into operations, and there's some big bug, guess what happens? The company loses money. So, there's a huge investment in documentation and code quality, because that's how we actually make our profit (well, and hardware support quality, etc., but who cares about that? ;) )

    One enormously bad thing : since we work for the government, we essentially have 275 million 'managers.' The president made an executive order at one point that means I have to do this one particularly annoying thing everytime I do documentation that cannot be done automatically with current technology. Stupid politicians.

    Also, for what's it's worth, even though I don't think it's *that* good, we were rated on one of those job sites/magazines in the top 20 places to work in america.

  • by Hard_Code ( 49548 ) on Friday August 18, 2000 @10:31AM (#844609)
    Somebody please tell me they work nine to five. I feel *guilty* when I do. I almost wish I could be paid on some sort of contractual basis, because 90% of my work is done in sporadic 20% of my time, and the rest of my time is spent lazing about reading slashdot and feeling guilty.
  • According to an alternate-management-styles documentary I saw a while back, the company that makes SAP, in NC, has a fairly strict 9-5 schedule.

    According to the show, they have some rather nice policies:
    - 9-5; no late-night coding sessions. The Pres. believes tired coders write bad code, and spend even more time fixing their bugs the next day

    - On-site physician, so employees can get to a doctor more easily, keeping them happy, and increasing productivity

    - full-time landscaping staff. The company is located on a few 10s of acres that are maintained year round.

    - half the management is women; many married with children. Meeting times are generally flexible, to accomodate those who need to leave at odd times to pick up kids from school, go to a little league game, etc.

    - I think they have on-site daycare

    - Generally open, comfortable, attractive workspaces for employees.

    The pres. also said something like, 'We spend and extra $5 million a year to keep our employees very happy, and save $15 million by increased productivity, and low-turnover."
    -----
    http://movies.shoutingman.com
  • Firing all managers would be a BAD thing.
    Anywhere in the real world, a certain level of BS is /required/ to carry on buisness. Yeah, it's a nasty thought, but it's there. My boss, the CIO, has to dig through a new buisness plan. I sure as all hell don't want to swim though something like that. I want to code.
    Generally, programers think along those lines. Very few people are able to talk to clients and pull something reasonable from them. That's where the managers belong - an interface layer, if you will.
    iD didn't have customers, per say. They were making something that they thought was cool, and it just so happened that a lot of other people thought it was cool and bought it. In most of the rest of the buisness world, it's not like that. You do what the customer wants you to do, not just something that you like.
    Now, if we're talking about something as small as iD, even in a more "professional" situation, what you propose will work. But this is also a world of giant corporations (IBM, for instance) that simply cannot operate without a large governing body of managment.
    Some companies will have far too many managers and too many BS rules, sure. That can't really be stopped.

    BTW, read the "Tao of Programming". It's, shall I say, enlightening. I beleive one section specifically addresses what you raised.
  • Also, for what it's worth, most (if not all) of our visible management were at one time grunts, who just decided instead of being programmers that they'd rather be managers of programmers. People complain about the management, but it's not like "they don't understand" complain, it's like "I can't believe they are making me this thorough" complain.

    I don't complain (about the management) much.

  • You're trying to compare two types programmers and two different types of environments like they are interchangable.

    According to the Space Shuttle software article on Slashdot few months back, the team working on this and their environment are nothing like anywhere else in the world. The programmers do not have any freedom. NONE. They cannot work after 5, it's not allowed. They cannot add a useful feature. Not allowed. They program with a manager looking over their shoulder the entire 8 hours.

    However, I agree wholeheartedly with the paragraph that Brian wrote that upset you. I have seen programmers come and go for many years. And the ones who "swim" become nothing short of gods while the ones who leave at 5:00 end up quiting never to be heard from again.

    Personally, I would hate working in an environment like this "IBM group". I love to innovate. I live for implementing ideas, no matter how late into the night it takes me. And as for myself not having a life, balony.

    Ozwald
  • by Alternity ( 16492 ) on Friday August 18, 2000 @10:35AM (#844620)
    Agreed. What is wrong IMHO is a coder who doesn't care about the code he has to produce for his company. It happened to me a few times that I was working on something I found so interesting that I did not care to stay a few extra hours at work. What is wrong is to see a programmer that spend his whole day looking at code only wishing the day could end so he can stop looking at that aweful code he has to work on.

    Programmers working from 9 to 5 each day? I have no problems with that...
    Programmers who cannot wait 5 to run away from the code they're working on? Now there's a problem... Those need to realize that they're are lots of other jobs with projects they might like more out there.

    Of course everything up there was IMHO
  • by Ded Bob ( 67043 ) on Friday August 18, 2000 @09:22AM (#844624) Homepage
    Exhausted workers make more mistakes. I have seen that even in my code.
  • As a decent software developer, I relish the thought of someone else questioning my techniques.

    I'm returning to school after working at a large company.... I just finished introducing my replacement to the code I've spent the last few months writing/refining. I've never seen so many mistakes in my life - and it was my code!

    The idea that large corps don't review their code explains why their click-wrap licenses claim they can't be held accountable for thebugs.

It's a naive, domestic operating system without any breeding, but I think you'll be amused by its presumption.

Working...