Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror

Submission Summary: 0 pending, 1 declined, 0 accepted (1 total, 0.00% accepted)

DEAL: For $25 - Add A Second Phone Number To Your Smartphone for life! Use promo code SLASHDOT25. Also, Slashdot's Facebook page has a chat bot now. Message it for stories and more. Check out the new SourceForge HTML5 Internet speed test! ×
Google

Submission + - Google News gets a separate web crawler and index

markkezner writes: In response to complaints by Rupert Murdoch's News Corp. and others, Google has given Google News its own web crawler. The service had previously relied on the same web crawler that is used to index pages for regular web search. This change allows publishers to use robots.txt to finely control how Google News indexes their content without affecting how it is indexed for normal Google web searches. Before this change occurred, publishers could have contacted Google to specifically de-index content from Google News.

Slashdot Top Deals

"We shall reach greater and greater platitudes of achievement." -- Richard J. Daley

Working...