Because Civ is a single player game. It isn't meant for multiplayer, and multiplayer has always been a terrible experience. I'd prefer if they dropped it entirely and spent more time on polishing the AI or released it earlier. Because they shove in a half baked multiplayer we get a worse game.
I've had lots of fun playing Civilization 4 over LANs, so please allow me to disagree, for the most part, with your statement. That said, the AI in Civ 5 is rather flawed.