Holy Heck, as a Non-American I thought your claim that there are states that ban atheists from being elected was probably an exaggeration or simply an interpretation of how it is hard in many places to be elected if one is a self-declared atheist. Hit a google search and figured i'd put this in there in case others were thinking like I was but don't bother to follow up.
Arkansas, Article 19, Section 1:
No person who denies the being of a God shall hold any office in the civil departments of this State, nor be competent to testify as a witness in any Court.
Maryland, Article 37:
That no religious test ought ever to be required as a qualification for any office of profit or trust in this State, other than a declaration of belief in the existence of God; nor shall the Legislature prescribe any other oath of office than the oath prescribed by this Constitution.
Mississippi, Article 14, Section 265:
No person who denies the existence of a Supreme Being shall hold any office in this state.
North Carolina, Article 6, Section 8
The following persons shall be disqualified for office: Any person who shall deny the being of Almighty God.
South Carolina, Article 17, Section 4:
No person who denies the existence of a Supreme Being shall hold any office under this Constitution.
Tennessee, Article 9, Section 2:
No person who denies the being of God, or a future state of rewards and punishments, shall hold any office in the civil department of this state.
Texas, Article 1, Section 4:
No religious test shall ever be required as a qualification to any office, or public trust, in this State; nor shall any one be excluded from holding office on account of his religious sentiments, provided he acknowledge the existence of a Supreme Being.
If i had more time i'd look to see if there are cases of anyone actually being denied in the recent past or unseated after the fact but the fact that these provisions even exist to begin with is pretty awful.