Is the recipient of a mix CD a copyright infringer? If not, it doesn't make any sense that a downloader would be either.
Your argument relies on some sort of distinction between "who makes the copy." In the mix CD case, where it's given to you, yes, you obviously didn't make a copy.
However, if you load up your torrent manager and say "download please!" you are making your own copy, which is then stored locally, just like pushing the button on a copy machine.
The one who started out in possession of the media, made and distributed a copy of it, is violating the right to control copying and distribution, i.e. copyright.
To continue the analogy, it's like a library places a book on a public shelf. You are the one choosing to take it off that shelf, walk over to the copy machine, push the button, and then take the photocopy home with you.
It seems you may also be trying to make the argument that the person who originally ripped the copy or whatever was infringing, but you're not by making a copy of that copy. Except that doesn't work in the analogy either. If you go to an office where somebody has made an illegal photocopy of a book, and you take that photocopy and make your own photocopy, you're still violating copyright.
If that's weren't true, I could just download a (legal) PDF that was made from a print journal from a library, and then place that PDF on my own public website for anyone else to download, and I wouldn't be guilty of infringement. After all, I didn't make the PDF myself -- I didn't "rip" the media, so why should I be guilty of anything?
Someone who started out with nothing, copied nothing, distributed nothing, but ends up in possession of something that someone else illegally copied and distributed, has done what exactly that violates what law?
You are correct that you "distributed nothing," which is why GP argues that the case is harder to make, and excessive damages are harder to justify. But you're wrong about the fact that you "copied nothing," since you ordered your computer to do precisely that, just as if you'd press the "copy" button on a copy machine.