Because in 1981 or so, everybody was pretty sure that this fairly obscure educational network would *never* need more than about 4 billion addresses... and they were *obviously right*.
The discussion about grains of sand or atoms is pretty silly. The reality is that the idea of 1 item, 1 address is already hogwash. It's very typical for one address to host *many somethings* (EG: websites, NAT, etc) and the opposite is also equally true: it's very typical for one something to respond to many addresses.
There are many applications that we likely can't even consider due to today's limitations that may well depend on or benefit from a large address space. IPv6 is a definite step in the right direction, but having seen the transition from 8 -> 16 bit computers, 16 -> 32 bit computers, and the transition from 32 -> 64 bit computers, the reality is that **growth is exponential**.
When 2% of your address space is consumed, you are just over 6 doublings away consumption. Even if you assume an entire decade per doubling, that's less than an average lifetime before you're doing it all over again.
IMHO: what needs to happen next is to have a 16 bit packet header to indicate the size of the address in use. This makes the address space not only dynamic, but MASSIVE without requiring all hardware on the face of the Earth to be updated any time the address space runs out.