then you need two copies of every shared library that they use to be mapped at the same time.
No, you don't. The libraries will be paged in on-demand. If you never use more than 4 GB address space within a single program, the amd64 libraries will never be paged in. Typically, I-cache lines will be driven by LRU, so you'll have them filled also on a on-demand basis.
Except you've trashed your cache by needing two copies of every standard library
You should learn how virtual memory and cache works before spouting this bullshit.
As you suggest, pretty much everyone has enough RAM to run just about any normal application with 64-bit pointers.
Most users will never need 64-bit pointers. Only applications which require more than 4 GB of addressable memory within that single program will ever use this. Examples of such applications include in-core scientific computing and very large media file editing.
The finest eloquence is that which gets things done.