Explaining Complexity in Software Development?

BMazurek asks: "I'm stumped by how to explain software development complexity (not theoretical big-O notation, that's easy) to non-developers. When it comes to people who aren't in the code, my explanations fall flat. It's not that the people I'm talking to are stupid, they're quite honestly people at the top of their respective (non-tech) fields. How do -you- explain software development complexity to non-developers? What analogies do you use?"

"I often try the famous Fred Brooks, Jr. quote (seldom to much success):

'Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.'

I don't (anymore)

[yagu]

Spanning 20+ years in computer programming I've stopped trying to explain what it is like and how it is done. Heck, we can't even clearly explain to peer programmers why vi is better than emacs, XP is better than Linux, Gimp is better than Photoshop (NOTE: these do not necessarily reflect my opinion, FTR, I prefer: vim; Linux; and Photoshop)

I shrug my shoulders and explain it's mostly dull. It's kind of like doing Math homework, except I have to do it every day for my job. It's always about solving and debugging, and people's eyes glaze when I begin poking programming nuance with a ten-foot pole.

Fortunately I find most people don't really care. (Also for those who would "get it", my experience has been they are ones who dabble and experiment in it anyway already.)

(Aside: as for the original poster, congratulations on being able to explain "big O notation". I sometimes suspect my girlfriend of faking it.)

My favorite: A Christmas Carol

[Marxist Hacker 42]

Easiest way I've found- though it's begining to get a bit outdated thanks to bloatware. Charles Dicken's famous novel is about 100k. This makes it easy to estimate source code in terms of number of copies of that novel. The quarter-million lines of code project I'm currently working on takes about 25 MB to store- 25000k, or 250 copies of A Christmas Carol.

Depends on the audience

[nosredna]

As with explaining the complexity of anything, you have to try thinking either in terms that are completely universal, or in terms that the person you're explaining it to will understand from their field (assuming you know enough about their field to make an analogy, of course).

I usually end up rambling on until the listener's eyes glaze over, but I've had some success with demonstrating some relatively simple things with a deck of cards... sort and random algorithms are especially well suited to this type of explanation, for obvious reasons.

Anything that doesn't have an easy analogy in common knowledge, I don't generally worry about explaining beyond some noncommital answer involving a basic description of the task, then asking the listener to think about breaking it down into simple parts, with directions that a six-year old could follow. Generally works, and gives an idea for the complexity of a program, at least sufficient to give somebody who doesn't really need to know everything about the programming side an idea of the work involved.

My analogy

[Rosco P. Coltrane]

When it comes to people who aren't in the code, my explanations fall flat.

Before I changed line of work (I'm not a computer professional anymore thank goodness), I used to explain my work like this:

See, what I do is programming. Programming is like writing a cooking recipe, only slightly more complex but not much more. But it's a very large recipe, and it relies upon many more recipes to make an actual dish (the program). Many cooks write recipes in different languages separately, and all we cooks have to coordinate to prepare the final dish. So we need chief cooks (managers) that call meetings and prepare Gantt charts to do that. Then sometimes a cook writes a recipe that has the wrong combination of ingredients, or that make no sense to describe how to prepare food (bugs), so we need tasters (QA) to tell us busy cooks if the overall result will be pleasing to the restaurant's customers. In the end, you get a huge dish that has some nasty morsels in it, as well as tasty ones. We then have to refine the dish so the nastyness goes away and more goodness goes in (new versions). etc etc...

The cooking recipe analogy has always worked great to explain what I did.

it's like serving food at a restaurant

[silvermorph]

If you're looking for the perfect 4-course meal at a fancy restaurant, you need to match flavors, textures, and colors from a lengthy menu, weigh the transitions from appetizer to main course(s) to desert, and fit in the perfect glass of wine and the ideally sized, shaped, and sweetened dessert.

Only it's a 100-course meal, the menu is 1000 items long, there's a cellar full of wines (most of them are unpronounceable) and it takes months to find a sequence that people will even TRY to eat all the way through.

And even then, they probably won't be satisfied enough to tip you very well, so you keep making slight tweaks to the courses until someone finally gives you the tip you want. At that point it's good enough, but everyone knows it's never perfect. (And they call you on it, too.)

As soon as someone notices that you're making a decent tip, the owner invariably decides they're going to serve Chinese food instead, so you start over.

Why is software complex?

[maxume]

Because it interfaces the most complex product of evolution(people) with the most complex creations of those people(computers and software). Computers are utterly rigid in what they do, and humans are nearly infinitely variable. Dealing with the differences, with a comparitivly small amount of experience, is hard.

Some software is pretty simple, because it solves simple problems. Is it any wonder that it often takes a complicated system to solve a complex problem?

Re:I like legal analogies

[mOdQuArK!]

I thought it would be interesting if it were required that all legal documents be written in an unambiguous computer-parseable language.

Decision points where ambiguity had to be resolved would be referred to human actors (judges, government officials, etc) who would have to provide their decisions in terms of data that was unambiguously interpreted (in terms of the legal language).

Couple of benefits that I can think of (aside from reducing the number of arguments about what the laws actually mean):

1) Trials could be reduced to a "legal" computer, where the prosecutors & defense lawyers write their cases in terms of the legal language, feed it into the computer, the computer asks the judge & juries to make decisions at key points, then renders the verdict.

The more interesting possibility was being able to "test" the effects of new laws by running through large population "simulators", and try to forecast how they might affect your society.

I discussed my brilliant idea with my brother (a lawyer), who laughed at me and told me to get a life. *sigh*

Easy.

[Kaenneth]

"The main processing unit in a computer has Millions of transistors, which are like little switches, turning on and off. These switches flip automatically Billions of times a second. Every second Quadrillions of switches flip, and if a single one goes wrong the computer 'crashes'. That's not even counting the Memory, or the Graphics card, or all the extra devices from Hard Drives to Tempurature sensors that are hooked in. All of these switches are controlled by programs that were written by the combined efforts of thousands of people, most of whom have never met each other. I'm continually amazed every second that a modern computer keeps running."

I say..

[Anonymous Coward]

Imagine a very complicated machine, like the Boeing 777 airplane, which took years to design and build, and has millions (? actually not sure) of parts.

Imagine that each of those parts can directly affect 2-3 other parts.. the ones that it is connected to physically or electrically, etc. That gives you a huge set of possibilities for parts to fit wrong, or wear out in unexpected ways, or otherwise fail.

That's a pretty complex system, right? In fact, it's almost amazing that human minds can conceive, let alone build and deploy, such a complex system.

Now.. imagine that you've got millions of parts, and each part can potentially interact with EVERY OTHER PART. Also, imagine, because it's such a young field, there really isn't much common vocabulary or theory, and incompetent practitioners pretty much have the same power as seasoned veterans (perhaps more so in some cases). So things are designed and built in an ad-hoc manner rather than with careful engineering discipline.

That's what programming is like. So don't complain when Windows crashes or you can't get to your favorite web site.

Explain by analogy/example

[(Score.5, Interestin]

If I've got 5-10 minutes, I use a simple exercise of getting them to sketch out a program for a humanoid robot to set the table, i.e. to carry cutlery and food from the kitchen to the dining table. Before they begin, they consider it a trivial task. After about 5-10 minutes, they accept that even this "trivial" task is close to impossible. Here's an example: First, you have to get the thing to walk from the kitchen to the table. So you have to teach it to walk. Then you have to teach it to avoid obstacles. I tell them that their robot has just crushed the cat (it's not a chair or table, which was their understanding of an obstacle). So they modify it to avoid small furry objects... and kill the dog, which is large and furry. So they modify it again, and find that it's frozen in front of a throw rug, which is small and furry. So they modify it again, and find that it's been halted by a sleeping cat. Then you throw in exception conditions, e.g. a fire, or even just the phone ringing so the robot has to clear the way for someone to get the phone... it's fairly easy to demonstrate that even the apparently most trivial task is in fact incredibly complex once you have take all the different conditions into account, and depending on how much time you have (and how long they take to convince) you can keep throwing hurdles at them almost indefinitely.

Try web development

[SmallFurryCreature]

It is very hard to get across the difference between a properly coded site and one made by some kid who knows some scripting language.

The simplest is browser compatability. Does the site owner care? Well yeah if you can get their head around the fact that if the site is IE only they will be losing a percentage of customers. It is very odd. Supermarkets understand they need an extra wide register for people with special needs. They know they need staff on hand to help disabled people because no supermarket can afford to turn that tiny percentage of people away who need help. Most brick and mortar stores would be horrified to learn that their to narrow door would loose them say 5% of visitors. So why is it acceptable on the net?

Then their is the proper use of standards. It ain't just that you got to use CSS but more that you got to choose a standard way of doing things. Use font tags if you want BUT then ALWAYS use them. Not some horrid combo of font tags, inline css, external css and included css. Makes it a pain to figure out what the fuck is supposed to affect what.

Finally there is the coding itself. Every damn time I get called into to fix a site I find that some kiddie has hacked it together and totally forget to do any kind of error checking let alone proper security. Most just take values straight from the user and insert them into sql queries. And why does anyone create a login page for the site CMS without https being enforced?

I seen code that would make you weep. Yet the site works doesn't it. Well no that is why I am called in but it then is very hard to explain why you need a week to upgrade the site before you can even start fixing things. (Note that the sites are generally so poorly designed that the original builders are no longer willing to support it wich is when I get involved).

But how do you explain to the customer about the difference between bad code and good code.

Recent page had a simple form for the usual add, update and deleting of a record. The code contained no functions and had the most horribly arranged IF/ELSE statements you could imagine (IF(user_logged_in) {ALL THE CODE} else {error_msg} EWH!!!!!

Worse it had the HTML/FORM 3 times for each type of action. Except that they were practically the same. Just an empty form for add, a filled in form for update and again a filled in form for delete. 3 Times the same HTML all because the idiot never grasped the concept of functions. (He didn't use them anywhere in the code).

Yet how do you make this clear to the customer. If you know the answer please tell me.

Re:It's not horribly complex.

[Kjella]

Well, part of it is that there are 25 millions line of code, you just don't interact with them. Recovering from an error is like a red light flashing in a 747, except you need to build in the intelligence to handle it. And while you have multiple redundant systems for physical failure, you only have one application (although it can have multiple threads etc) so there's no "ignore and continue", unless it's a specificly handled condition.

The second part is that the computer will do *exactly* what you tell it to do, which you have to specify exactly. That is much harder than people think, because even a dog has more basic intelligence (or at least self-preservation instincts) than a computer, not to mention a basic understanding of objects (ball, stick etc), impossible states (no, I can't fetch the ball, it's on the roof) and so on. For a computer, nothing exists that hasn't been defined to exist.