Do you watch soccer? 4k resolution means a player's head is about 14 pixels high, not enough to make out much beyond a blob of color; their jersey is 60 pixels high, enough to make out the number but not much more. Doubling the vertical resolution (i.e. going to 8k) would likely be enough to let you make out similar detail to what you'd see in real life. (Frame rate is another issue: HDMI 2.0 allows 4k at 60hz which is too slow when panning in a soccer game; HDMI 2.1 allows 4k at 120hz which is probably enough). I think that 16k is probably the right bandwidth to get soccer looking good.
Do you do VR? 4k per eye isn't good enough for VR yet. It's possible that 16k will be, but we might still need more.
Do you watch the gorgeous film classics like Lawrence of Arabia? One of the (many) things that make it look great is that it was shot on 65mm, equivalent to about 12k resolution.