The key isn't the batteries. The Tesla battery pack is over a thousand pounds, making the Tesla S weigh as much as a SUV. That hasn't hindered it. The key is recharging the battery. The current 30-minute minimum charge time is the primary hindrance to wide-scale adoption (purchase price is too, but I'm pretty sure the government would offer incentives to get us off of oil imports). If you can get the recharge time down to 5 minutes (whether by better charging technology or simply swapping battery packs), then most people's apprehension over having an electric as their only car disappears. Right now, unless you drive very little or are incredibly conscientious about the environment, only people who can afford 2+ cars are getting EVs.
The other problem is going to be operating cost. EV advocates are doing rosy predictions based on current electricity prices, or even current overnight electricity prices. If EVs become mass-market, the huge number of them being charged overnight will flatten out the electrical consumption curve and the overnight prices will no longer be much lower than daytime prices. And if it leaks over into higher consumption during the day, overall electricity prices are going to increase as well. This is going to impact the price of everything that uses electricity, not just operating EVs. Normally the market would simply build more power plants in response, but the same people advocating EVs are blocking the most effective ways to generate power. They insist that new power generation be wind whose inconsistency would require a massive overhaul of our electrical system, or solar whose price would make the EV about the same cost to operate as a gas vehicle.