The problem with the electoral college is not that it exists, it's that it's being used improperly as a flawed proxy for the popular vote instead of as it was originally intended, which was to reflect the will of the individual states, not the people.
Are you sure that's how it was intended? The EC has state-apportioned representatives because the constituents of the Continental Congress and later the Convention were colonies, later called states. Neither the US constitution, nor does any commentary I'm aware of, state that electors are pledged to represent the interests of their state.
Of course, at every crucial point in history prior to the 1860s, somebody suggests reducing the power of states in favor of either democratic populism (Jackson) of federal power (Hamilton, Washington...), and the argument against goes something like, "You're just trying to abolish slavery!" American federalism was invented as a pretext to sustain slavery in the colonies where it was economically entrenched.