ARMv8 supports both AArch32 (32-bit ISA) and AArch64 (64-bit ISA), similar to how AMD (and now Intel) CPUs support both x86 and amd64 ISAs.
Meaning, you can run a 32-bit OS on a 64-bit chip, and get access to all the improvements to the architecture, and it will run like a faster 32-bit chip.
Or, you can run a 64-bit OS on the 64-bit chip, and still run 32-bit apps, and get access to all the improvements to the architecture, and it will run like a 32-bit chip with access to a full 64-bit address space (for the OS, the apps are still limited to 4 GB each).
Or, you can run a 64-bit OS on the 64-bit chip and run 64-bit apps and get access to all the improvements to the architecture, including access to the full 64-bit address space within each app.
Or, you can mix and match the last two as needed. Which is what Apple is doing with their A7 SoC (64-bit CPU, 64-bit OS, mix of 32-bit and 64-bit apps).
There's a lot more to the ARMv8 architecture than just 64-bit-ness. There's a lot more memory bandwidth, there's a lot more registers, there's a lot of clean-up to the ISA, etc, etc, etc.
You don't need more than 4 GB of RAM to get improvements from running a 64-bit SoC. Just like you don't need 4 GB of RAM on the desktop to get improvements from running an AMD CPU in 64-bit mode with a 64-bit OS.