If I'm familiar with the song, then it typically takes a bitrate of 320Kbps or higher before I cannot hear a notable difference.
However, the often-ignored problem with lossy file formats is what happens when you attempt to edit the audio you've licensed/purchased. For example, say you want to re-encode a lower bitrate version of a song for a mobile device, or maybe adjust the volume, or trim a song down in length for personal listening preferences. (I do this quite often, actually.) Trying to re-compress lossy source material again after editing just makes things sound far worse.
This remains another reason I try to get lossless audio files whenever possible.