The point is actually that the poll averages are reasonably likely to be wrong, because some polls are designed much better than others. Most news outlets just average the polls. Nate Silver weights them in an attempt to give more weight to accurate ones. So, the simple averages of polls are right in most cases, but in a handful of states are sufficiently skewed by biased polls to give an incorrect prediction. Nate Silvers' weighting of polls, on the other hand, got all 50 states correct--and in many so-called "contested" states actually nailed the Romney v Obama share perfectly to 0.1%!
I don't think a safe call can be made that Nate Silver's method (which not only uses poll weighting, but also "state fundamentals") is actually superior compared to simple poll averaging as done by the Princeton people or by Andrew Tanenbaum. While Silver correctly predicted all states in the presidential election, with Florida being sheer luck, he missed the Senate races in MT and ND quite badly (in the latter he claimed a 92% probability for an R win).