At higher bit rates I'm not bothered by lossy compression. I can be bothered by the results at lower bit rates and if I am aware how the track is supposed to sound.
Back in early 2000, I ripped much of my earlier collection using 224 kbps ABR. I was a big Maximum PC reader, and one of their multimedia issues recommended using VBR for MP3 encoding. Not understanding too much about encoding, I used ABR for compatibility, and "stereo" as I found "joint stereo" butchered cassette rips. I played these tracks mostly through my PC & laptop so I didn't notice any issues. I used CD's on my main system anyways and never used headphones.
When I purchased a Grado headset for a new (and first) iPod, I found differences in many of my CD rips. This bothered me to no end. For example, Thievery Corporation albums had distorted flaws in echo decay, and highs were harsh. Strings in some classical music seemed butchered while piano had detectable warbliness. High hats seemed wrong in my rock recordings. Choral music vocals sounded harsh. Similar experience when iTunes finally came to Canada, I bought a couple of Iggy Pop tracks. They were aac's encoded at 128kbps. The tracks were clean but the guitars and cymbals were so harsh, I had to stop listening after only a short while. Once I bought the "New Values" album on CD, I didn't experience the fatigue with the same tracks.
Most of the issues mentioned above dropped with properly set command line in Lame with significantly higher bit rates. I do notice a difference. Once I set up a media server all the old rips had to go. I re-ripped my collection. I notice very minor differences with Lame V0 tracks, and 320 kbps CBR, compared to CD's but they don't prevent me from enjoying the music. If I buy tracks from eMusic, ignorance to the original recording is bliss. At their low prices and with tracks ripped mostly with Lame V0 to V2, I can also accept the cost/quality trade off.
I have a couple of DVD-A and SACD discs. Aside from a slightly better sound stage, I would probably fail an A/B comparison test. Either my 40 year old hearing or my equipment would fail me. If I'm being charged iTunes prices, I would still opt for the CD or FLAC equivalent for my music, and hope sound engineers trend back to recording quality.