Applebot is the web crawler for Apple, used by products including Siri and Spotlight Suggestions. It respects customary robots.txt rules and robots meta tags. It originates in the 22.214.171.124 net block.
User-agent strings will contain “Applebot” together with additional agent information. For example:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)
If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions.
Will AppleSearch be 'a feature Apple users want', 'a Google Killer', or 'the Rise of Siri'? You decide...
Link to Original Source