"You might be amazed how much "code" that is not web-pages still interfaces with people."
I don't need to be amazed, because I'm an architect and have worked on many such projects, but the people they're interfacing with are technically competent enough to know what they need - there's no real formula to always get a front end system right first time, but there are plenty of well defined ways of doing back end systems right first time.
"If libraries and compilers aren't used by at least 100x as many programmers as the ones required to write and maintain them, I'd call "you're doing it wrong" on the libraries and compilers."
The people writing them are themselves end users though, so (if they're half way competent) they don't need an outside subject matter expert because they are one - at least they should be if they're writing the library and compiler in the first place. Besides, many people don't even use compilers directly now, they use them indirectly through an IDE, and it's really the IDE that matters. The idea of a compiler as just being this individual thing that is only used in one way seems naive, compilers are used in a variety of circumstances even beyond this where users don't even realise compilation is occurring. Case in point we're building right now a system for defining workflows in a visual manner that generates code and compiles it to bytecode so that it performs incredibly fast when executed - the only other people even knowing anything about that compiler will be those of us working on it.
"I'll grant that "large back end systems" require a lot of effort, and maintenance, but in this case your "customers" are the programmers that interface to the system - and, in a way, the specifications of those interfaces."
But this doesn't require daily interaction, it's the sort of thing that can be typically well defined up front in a waterfall style way. It's not like a user focused application where you need regular iterative cycles with user feedback to refine it. I've trained people in Agile methods (scrum specifically) so it's not like I'm some old school guy who can't move on, but I do realise that there are vast differences between different projects and some techniques work better than others depending on those circumstances.
"If a coding problem is clearly defined, unchanging, in other words: academic"
This is nonsense and I'd frankly question your experience if you think only academic problems fulfil this criteria. When we build systems for major financial institutions it's typical for them to have these systems in place for 10, 20, even 30 years. They don't need anything other than an agreed interface to access the system from the outset and from there on out we simply build it. Some projects simply necessitate (especially given the sums of money involved) that we put in the effort to get it right, and get it stable from the outset. If we built these systems in an Agile manner we'd simply lose all respect and they'd have no confidence, when they're talking about something with a long life time they don't want it to look like we're winging it and figuring it out as we go. We do however sometimes offer additional services, including building user focused front ends that interface with these systems, and in those cases we do use Agile methods, we do constantly get feedback and so on and so forth, these systems aren't going to stay static like the back end for an extended period, they'll either change, or just be outright replaced on a much shorter cycle because front end technology tends to change much more frequently than back end.
There's a time and place for each and every tool, but you're sounding an awful lot like one of those guys who believes you can and should throw agile at absolutely everything. It's not a silver bullet, in fact, there is no silver bullet. Don't criticise people because you assume they're working on the same sorts of project as you and doing it wrong, there's every chance they're doing it exactly right and that it's merely your assumptions that are wrong.