A public act by an organization ignoring robots.txt will only lead to the justification of other organizations ignoring robots.txt.
So what? When DoubleClick argues that they ought to have the same advantages as Archive.org, they'll only manage to look like douchebags reaching their filthy hands into a cookie jar.
It's not always a bad thing to set up douchebag-honeypot moral exemption, even if it does depend on the mass audience (mostly) managing to find two sticks to rub together.
The real solution here is to make the directives in robots.txt more explicit concerning the predatory/non-predatory use cases.