then you may as well just give the server the AES key and ask it to decrypt the file
But in that model, if "the server" has the key, wouldn't Dropbox have the key? I thought that was the whole thing people were freaking out about.
No, you'd have the key. If you wanted to share the file publicly, then there's no point in keeping it encrypted, so you'd provide the server with the key and it would decrypt, saving you the cost of downloading and reencrypting.
I understand what you (and the AC) are saying about storing an encrypted key on the server, and then re-encrypting the key for each new user you'd want to share with. That's a clever arrangement and I admit that I hadn't thought of it, but it still seems like it has the potential to create more complexity than most people want to deal with. It still means you need to manage various encryption keys, and we (Internet culture) seem intent on not developing a coherent system for managing encryption keys.
The client just needs one key, the RSA (or equivalent) public key. You'd need to copy this between devices, but it's relatively small (under 1KB). It's small enough to fit in a version 40 QR code quite easily, so you could set up mobile devices by displaying the QR code on your laptop screen and point the mobile device's camera at it, if you don't have any sensible way of transferring files between devices. The client then has to download the file and the associated key, decrypt the key with the locally-stored key, and then decrypt the file, but that's not something that's exposed to the user.