Submission + - DreamHost blocks Google Bot on Shared Hosting - Ho
jsbi writes: DreamHost is supposedly touted as one of the Best web hosting providers by some of it's users[including a few people I know personally of]. But it now seems that it is requiring users to edit the .htaccess file to block GoogleBot from indexing their pages. Are you wondering what the GoogleBot is all about — well it is the Google web crawler that indexes web pages for them to appear on a Google search.
It seems that the explanation given by DreamHost is that GoogleBot is a "high memory usage and load on the server" and blocking it seems only the way out. The company has already dispatched off email's to most of it's customers, asking them to make their content unsearchable for spiders and crawlers. I guess the entire move by DreamHost is rather so STUPID!! Instead of them to upgrade its systems to make them more powerful, stable and able to support the crawl process of Googlebot — they ask you shut it down completely.
http://jsbi.blogspot.com/2007/05/dreamhost-blocks- google-bot-on-shared.html
