You bring up some good points in your post. But I have to disagree on one thing. Good quality music reproduction is today more accessible than ever. There was a time when you had to get horn speakers or at least speakers as big as cabinets, class A amplification -solid state or tubes, and a really hard to setup vinyl turntable. Then there was room treatment, speaker placement, and all those shenanigans.
Not saying this is still not relevant. But today, you can get a decent pair of headphones (sennheiser, audio technical, akg, grado, fostex/MrSpeakers, etc), a decent DAC and amp (Schiit, Audio GD, etc), and good quality source and good quality digital (hi res or even redbook) - all at even a college dorm budget, and similarly compact.
I remember the days of the walkman and audio cassettes, and for sure, the progress has been dramatic. The only irony being that the single most important piece - the quality of mastering and quality of recording - has largely gone for a toss. Today, it is all about loudness wars and auto tune. But that is a different matter.
When people pursuing any hobby go beyond a certain expense level, they make purchasing decisions for most things other than money. Why is there no Slashdot argument about people paying $3 million for a vintage Ferrari or a Jag? Is there any basis to that price! Is the buyer, no matter how much an auto enthusiast, ever going to take his or her vintage Jag for a really rough spin that could risk damaging the car?
Maybe the analogy is not accurate. Fair enough. But a lot of audiophiles with really high end systems do find a difference in sound even with trivial component swaps. They will even claim that placement of certain objects in the room alters the sound.
But before dismissing them as twats, it might be worth thinking about how idiosyncratic and bizarre other people are who are equally immersed in their hobby or pursuit. The guy who is cooling his Intel CPU in liquid nitro to get the last bit of over clock - really, what practical purpose did he serve? And he probably spent a bunch of money on his rig too.
The strangest thing of all is that music is one of those strange beasts that changes quality with every trivial change in component, room, source, you name it. That is what gets audiophiles hooked. Maybe and probably it is psychoacoustics. But if you can hear the difference, it is there, right?
Now how much tweaking and money you want to throw at this pursuit, that is a very subjective thing. But dissing it and ridiculing it is also wrong. It is only one of the many things that continue to fascinate us as a species. And music is indeed very very special to most of us. We just don't pay enough attention to this sense.