Downloading music or movies without a license has always been copyright infringement, just like uploading or sharing them. However, the labels only go after uploaders for a few different reasons:
First, technically, it's difficult to identify a leecher or someone who only downloads - due to the nature of file transfers and how the various protocols work, you can easily discover uploaders and download complete copies from them (e.g. by finding Napster hosts, bittorrent seeders, etc., and then blocking transfers from everyone except your 'target'). To discover a leecher, however, you have to be
a seeder or host and wait for them to download the file from you. And with so many other seeders out there for any file, that doesn't happen often.
Second, even if you did manage to get someone to download the file from you, if you're the copyright owner or acting on the copyright owner's behalf, you put the file online for public distribution!
So the downloader can easily argue that they have at least an implied license from you, and they actually obtained a legal copy. Ooops.
Third, even if somehow you get over those two hurdles, a leecher actually can use the "a download only costs 99 cents, so the actual damages due to my infringement are tiny" argument to mitigate the label's giant statutory damage request. This doesn't work for uploaders who are distributing
copies, as a distribution license typically costs tens or hundreds of thousands of dollars, depending on the work (Michael Jackson paid about $45k each for the distribution licenses for several hundred Beatles songs back in the 1980s, for example). But a mere downloader isn't distributing to anyone.
And finally, though it would be illegal and unethical, if you were accused of downloading something, you could rush out and buy a copy of it with cash, and then claim you were just legally format shifting (albeit by proxy). Maybe your proxy that you got it from is liable for infringement due to their distribution, but if you
can legally rip your own CDs for archival purposes, then simply using someone else's drive (and computer, and network connection) to do it shouldn't create liability for you.
So, yeah, could a label go after a mere downloader for infringement? Absolutely, and that's always been true. Are they going to do so, and potentially spend millions knowing they're going to run into those four potentially insurmountable barriers? Hell, no. Not when moviebuff6969 is seeding 50 films on bittorrent.
Disclaimer: I am an IP lawyer, but I'm not your IP lawyer. This is not legal advice, but is purely for (my own) entertainment purposes.