I can here the difference on good headphones and certain songs. I've done listening tests on 3 headphones. Using Sennheiser CX300's ($30), I could barely tell the difference between FLAC and V0 MP3. In a blind test, I might not be able to tell them apart at all. Using Sennheiser IE6's (~$120) it is easier to tell the difference, and I can tell the difference on certain songs. Using my friend's Audio Technica ATH-M50's (~$120), the difference is obvious on many songs. All of this is listening coming straight out of a PC sound card. With the ATH-M50's I will sometimes use a headphone amp because my sound card can't drive them loud enough to hear every detail.
When I say certain songs, I like to pick songs that the V0 encoder had trouble with, or in other words V0 encoded songs with the highest average bitrate. When there are many instruments and voices at the same time, there is more information that has to be represented, or more than may be lost in compression.
The difference is most noticeable in the high end, particularly in the percussion. I can hear up to 19.5KHz. If you can't hear as high, you will have more trouble distinguishing FLAC vs MP3, bu it isn't impossible. Percussion, especially cymbals, sound clearer and sharper in FLAC, but clearness or sharpness alone aren't enough for me to distinguish the difference in back to back listening. I always listen for flaws in the sound of cymbals-- instead of sounding like a real instrument, they will sound digital, shimmery, or wrong. It's hard to describe without making up what sounds like audiophile BS, but the difference is there.
I agree for casual listening FLAC is unnecessary. If there is even the slightest noise coming from the environment (in the car, air conditioning running at home, etc) it becomes hard to distinguish the difference, and if I am not devoting my full attention to the music, I don't hear anything wrong MP3 encoded music.