but you can't equate that level of programming with modern software development.
Why not? There are just a few simple concepts you need to understand. It's why so many pre-teens were able to easily 'graduate' from BASIC to Assembly back in the 80's -- they already had all the necessary skills. The rest, like any skill, improves with practice.
Hey, let's face it: modern software is a disaster. A few easy examples: I've got a simple Pac-Man game I found for my phone that weighs in at nearly 30mb. I'm not even sure how that's possible. (Give it a shot. Over the weekend, write a pac-man clone. I'll bet that yours will be significantly smaller, even if you try to bulk it up by adding extras like flashy graphics and effects.) This isn't an anomaly, it's disturbingly common.
A $30,000/seat bit of industry specific software the folks at the office across from mine uses actually includes the full QuickBooks SDK as part of the installer -- not the redistributable, mind you, the bit intended for developers download and use. I don't recall exactly, but the program is well over a gig installed and took more than a minute to start. The kicker? It's a simple database and reporting app. The DOS app that it replaced fit on a floppy.
Modern software is getting bigger and more complex, but isn't actually *doing* a whole lot more. (The app mentioned earlier has two new features lacking in the DOS version: the ability for multiple terminals to use the same DB and the ability to electronically submit a few reports.) Sometimes, as you well know, it even does less.
I have no explanation for that other than lazy, apathetic and unmotivated developers working in an industry driven by fads promising quick and easy development. How else do you explain the countless articles in the form of "Use Framework-X with Does-It-All-Library-Y to do Simple-Thing"? It's pure madness!
Do you honestly believe that such a proportion of people who make their living developing software are that bad at it purely because they're lazy, apathetic or unmotivated?
Yes. It's why most people are bad at things, be it their profession or otherwise.
For example, I'm a third-rate cook. (I look upon Charlie Brown with envy.) The reason being of course, is that I have no interest in cooking. I do it only occasionally out of necessity. However, given enough interest and time, I have no doubt that I (or anyone else, for that matter) could learn to cook as well as my wife.
Why are there so many second-rate professionals in software? I can only offer speculation, but I think there may be some truth to it: Trade-schools and colleges have been churning out programmers at an alarming rate. A good number of those students 'liked computers' and, lacking other serious interests, elected to study CS, MIS, or whatever was offered. When they found out that CS is really math and that programming is a dull, tedious, activity, they plodded along anyway just to graduate. Having no other relevant skills, and the promise of easy money, they started off their careers in software. (I'd be willing to bet that a lot of students in CS programs in the mid to late 90's were lured in by the promise of easy employment and high salaries, without even a passing interest in computers.)
For the obligatory car analogy, most people are probably capable of learning to swap to a spare tyre, change the oil, or top up the radiator (learning some simple scripting). Most people are probably not capable of learning to design high-flow intake manifolds or variable valve timing mechanisms (useful commercial software development).
You've got quite the gap there! Anyone can learn basic auto maintenance, just as anyone can learn to become a competent mechanic. ('Competent mechanic' is on-par with 'good developer'. I'd even go so far as to say that the necessary skills for each are similar. The main difference, of course, is that auto mechanics, as a profession, isn't driven primarily by fads and folk-wisdom.)
If you really want to take that further, yes, anyone with sufficient interest and resources can become a skilled engineer capable of designing such systems. Programming, of course, is about as far from engineering as you can get. I consider it offensive to compare programming to engineering.