While the USB memory key (in this example) could have low level software to snoop your data, how are they going to get it? Is the USB key going to open a TCP/IP or UDP connection back to their servers without tripping my firewall that a new application is trying to connect? Is my virus scanner going to get tripped that something suspicious is coming out of the key without my interaction?
Just because the cases are not obvious doesn't mean there is no potential for exploit.
Keyboards get a lot of raw sensitive data: usernames and passwords, often even accompanied with the direct URLs where the credentials apply. Now, the keyboard obviously wouldn't be able to open a TCP/IP or UDP connection to upload the data, but it could sneak time-encoded hints about pre-recorded data into your typing. While you type, the keyboard firmware could impose miniature delays that would go unnoticed by the human eye, but would in turn influence the timing of packets sent by an SSH session. Such an attack wouldn't necessitate decrypting the SSH session and it would go completely unnoticed through all your Intrusion Detection Systems and firewalls. The practicality of such an attack can be questioned, but it demonstrates non-obvious applications.
The closest equivalent I can think of for a USB memory dongle would be firmware that could recognize, say, JPEG images in FAT file systems. Any information the firmware recognizes as interesting could be steganographically watermarked into your images by the time you pull them off the dongle. In such a case, any image you upload online that came from that dongle could contain sensitive information and you'd have no idea you uploaded it.