Re:I discovered this the hard way (Score 1)

not really.

ya really

in order to "cause" the "attack" the website must come up in a search.

No they have to come up as a potential link on any page the person visits. The connectivity of the internet is high enough that this is a very high ratio to the actual number of site visits, if you have a site that is highly connected to other sites, it has a particularly high cost (e.g. the "digg this" link on so many news sites). Assuming google uses the refcount as its sole ranking, this means that the highest google hits get mashed up when any of the sites that reference them are viewed.

all this does is "pre-crawl" the pages in a search result to look for malware.

"All this does" is follow every link on every page you view on the off-chance that you view it, and then fail to database the results. The reason people say this is skeevy is that they don't collect the results that each client generates and perform simple DB queries instead of hammering servers. Of course, this would be an "invasion of privacy" since they would be sending browse information across to a remote host to check or update the database for a specific site... The net cost of this rudimentary solution is borne by the customer in performance costs and by the most interconnected sites on the internet in the form of traffic and the resulting load.

so unless everyone is searching google for the same thing, it really doesn't do a ton.

No, you don't know what you are talking about.

unless of course you run some pos server and have somehow gotten your result for whatever to be top ranked and of course it's a popular search string.

Now you are just sounding stupid, did you write this thing? I bet you don't even think it's a big deal to use BUBBLE SORT for large data sets with very short keys!

but then, i would blame the company, not avg, since they've gone to the trouble to probably cause this themselves.

The cost of running servers to handle even 25% more traffic than required is not a one-time payout, it comes in the form of maintenance and facilities bills every month. There is no reasonable justification for this cost, and it is the result of a faulty design. If they are going to prescan, they should prescan when a new page would be loaded instead of scanning the links of the current page without user action. Let the user/app take the brunt of the cost, not the internet.