"Unfortunately the only technique I ever found (and I've forgotten what it was at this point) generated a text file listing *every* package installed on the machine"
Unfortunately I fail to see where's the problem.
"a list nigh guaranteed to bork a machine if I tried to import it all on a different OS version"
Not my experience.
Now, my experience:
1) Debian-based: I don't reinstall that often (now that I remember, my current setup goes in time about 10 years or maybe more).
2) Debian-based when cloning a machine: when it's been the same release, no problem at all. When the receptor is a different version (newer) I installed a minimal system and then applied the package list. It might fail on some package disappearing or changing names (usually only a few) and then it's a matter to see what failed and act accordingly. Worst case scenario, I had to extract a list of the (partial) setup on the new machine and diff old/new.
3) Red-Hat based: yes, they are not so great at upgrading in place so I had to resort to the trick in point two. It was a bit longer and required more than one iteration but far from a drama.
"And good luck sorting out the 10% of user software from the umpteen dozen pages of semi-cryptically named packages."
From time to time (I mean months or even years, here) I spend no more than an hour looking at the installed package list. I know what most of the packages do, for the minority I don't know, I read its description as provided by the package manager. If still no clue, I try to unistall it and see what reverse-dependencies are going to be unistalled, which always made clear what was happening. Not a big problem either.
Oh! by the way, a few seconds of google search showed me how to list manually installed packages both for debian-based and redhat-based systems so it seems your concern was not so much a problem even for you as to expend even a minute looking for a solution.