You've got three broad brush categories, though of course people are often some of each.
- Programmers code stuff. Whatever you tell them to. Sometimes you can give them a problem and they come up with a solution, but it doesn't keep the entire ecosystem in mind. Often don't understand what they're really doing. Drives me batty when they're called Software Engineers.
- Hackers solve problems using whatever means necessary, however ugly it is.
- Engineers are solving problems keeping all the tradeoffs in mind and considering that this is a /product/, even if only an internal product.
Here's an example: This week a customer asked me for a feature in some PC software that generates files for processing by the embedded system. I know the entire ecosystem, not just the PC part of it, so I was able to tell them it was doable, but would have these negative effects - I can get you something working in the short term with these negative side effects, but we can get rid of them in the long run if we do this and that. They thought it was more important just to have the feature in the short term to show off in the short term and would ask me to make the other changes later if the tradeoffs got onerous. Maybe they'll never need it. This was a good deal for them in terms of what was delivered for what they're paying for my time.
You mention testing, but that's not really engineering per se, just one of the tools in the belt. A good programmer would (and should) be able to employ testing where necessary. In this case I did only cursory testing because they needed it /right now/ and the demo files I provided worked and were sufficient to show off the feature. And of course I told them this. Now that there's some breathing room I'll add better testing, and more if they decide to move forward on it.
It's all about knowing your /entire/ system enough that you can weigh the risks and requirements properly. Sometimes we blow a risk assessment or the requirement is wrong or something else goes wrong (customers are often very bad at providing real requirements or information) but at least you started from an educated guess.
You're a hacker, that's a good start since you're focused on solving the problem, and that's crucial - programmers are often (though certainly not all) bad at that. You are almost certainly more creative than some engineers. But now you need to consider that the requirements and the environment may substantially change how you choose to solve the problem. That thing you did may work, but is it maintainable and sustainable, and will it survive foreseeable new requirements?
Another example: there's a place in one of our codebases where sometimes you're looking for a string in an array in user time (someone typed something). We don't bother to sort that. Who cares? It's 'instantaneous' for the user either way. Why waste the time and code and complexity sorting it? There's another case where we're constantly looking things up, on the order of 5x a millisecond, so that one is sorted. But not cached since 5x/msec isn't /that/ much of a hit in context
I went a bit long, but I hope this makes sense. It's all mindset. Engineering is learning everything you can that even indirectly effects your system and solving problems based on ALL the tradeoffs. Realistically you can't know them all, but you can try. Iteration helps, as does time and budget.
