Some podcasters might have their shows available on their website. But you shouldn't confuse podcasting with a website linking to mp3s.
A podcast is a RSS feed with enclosures linking to the show. You use an aggregator to subscribe to the shows. The aggregator will automagically download the shows you subscribe to your harddrive. The aggregator will also automaticaly copy the shows over to your mp3 player (assuming you have one).