Martin has stated that the first thing he wrote about when he put pen to paper was the image of Bran and company in the snow, around the dead direwolf -- which is quite an ominous portent in light of what happens in the series, and a somewhat fantastical one.
He has said that he had considered having _no_ overt fantasy elements whatever, but rejected this -- instead he chose to gradually increase the magic in the series. He wants to keep it rather mysterious, as in Tolkien, although it's clear that it is getting a bit more prevalent. Still, I doubt we'll ever see wizardly duels.