I doubt that using 64 bit or 128 bit addresses would have increased the complexity for users that much. It just would have been a couple of extra characters to type in. IPX used 12 byte protocol addresses and it wasn't that big of a deal.
For developers, the biggest hassle would have been switching from a single dword to an array of some sort to hold the protocol address on 32 bit systems. No more simple register compares. But next to the logic for handling fragmentation/reassembly and IP options, it still would have been simple.
It would have been nice to have had something like IPSec transport mode from the start, but only if it were an optional component and if it didn't hard-code encryption or integrity algorithms. Also, for lightweight and low-end systems during the '80s and '90s, mandatory encryption support in the stack would have been overkill. Hobbyists would have preferred a smaller memory footprint.
I kinda wish that IPv4 would have made IP options a separate header, as they are in IPv6. Variable length IP headers are a little bit of a pain to work with.