If none of your processes require more than 4Gb of virtual memory, there is no reason â" other than the developers' laziness â" to go 64-bit.
First, addresses/pointers aren't normally the largest chunk of code or data memory usage, so the include in RAM usage is far less than double.
Also, in the specific case of the Intel x86 architecture (which is what this is about, not general 32 bits vs. 64 bits), there is a significant reason to move from i386 to x86_64. The i386 architecture has a very small CPU register set, compared to most modern CPU architectures (and some instructions can only use certain registers). That means lots more things require memory loads/stores, which is bad for performance. When AMD created x86_64, they added a bunch of registers (and got rid of most of the usage restrictions), so 64 bit code performance is better.