I scrape AA.com too... (Score 2, Informative)

I scrape AA.com too, but in a very different way technically and for a different reason.

I've developed a free program called MileTracker that consolidates users' frequent flier accounts. One of the accounts that I pull information from is of course at AA.com. I haven't received any complaints yet, but there are several reasons why I can think of that:

• I do the scraping from the client. There is no central server involved at any point.
• There are only a few hundred users at this point, each generating at most 2-3 requests a day. More likely they're only generating 1 request every few days.
• I've been very careful to simulate an actual human user using a Web browser. There are some steps that I skip for performance reasons (such as downloading images which could be cached anyway), but I make sure to handle all cookies, redirects, Javascript, etc. I also throw in some delays and randomization.

Someone looking through logs would probably have a lot of trouble telling the difference between a real user and my "ghost" user.

The application really isn't harmful to AA. In fact, there's a good chance that I actually increase real traffic to AA.com amongst their loyal customers. When there is a change in a user's frequent flier account balance, I provide an easy way for them to actually login to AA.com so that they can view the details. If AA complains, I'd probably obey since I have more to lose than I do to gain. It's AA's customers that use MileTracker that would suffer.