Absolutely! robots.txt applies at the time, not retroactively. I see nothing in the robots.txt standard that implies it should apply retroactively. It's basically a do/don't crawl *now*, it says nothing of what was the case before. So, if it was allowed per robots.txt when the pages were crawled, those are fair game for archiving ... period. archive.org or anyone else deeming themselves the exception regarding what robots.txt specified, and crawling where it says not to - that's wrong direction to go - period. But what was allowed and crawled earlier, perfectly fine, and nothing in change of robots.txt should imply it should alter availability or presentation of what was crawled when robots.txt earlier allowed it to be crawled at that time. End of story. :-) "Make things as simple as possible, but not simpler." - Einsein

At the point of entry to the clean-room, use RFID + biometric (and possibly also PIN or password). That effectively reasserts RFID in possession of the authenticated person upon entry to clean-room. Policy should enforce that RFID is to be on the person from entry to exit of clean-room. Then just use the RFID until they exit the clean-room. If any operations in the clean-room are so crucial as to require additional authentication/audit beyond that, add cameras+recording and/or additional authentication where reasonably called for, but don't overburden every step and authentication within the clean-room - after all that's part of the issue you're trying to solve. RFID should mostly suffice with sufficient controls upon entry and suitable policy and enforcement thereof.

The DMCA sucks. It may be useful to also pursue additional legal means. False/unwarranted takedown notices are a form of harassment, particularly when there's a pattern of such, and it's not "mere relatively rare accident". Sue the party submitting the takedown notice and the copyright holder authorizing the party to submit such, get a restraining order against them issuing any such unwarranted takedown notices against any of your content, and if/when they violate such restraining order, notify law enforcement - violation of restraining order is generally a felony. Can probably also sue them in small claims court for nuisance or the like.

Debian 3 CDs - do you have a good copy of debian-30r0-i386-binary-1.iso ? If so, I'd like to get a copy of that (or at least the blocks I'm missing), or perhaps you'd have the additional needed .deb files to create that ISO. Though the Debian Archive does have back into 3.0 "Woody", they don't have all the way back to 3.0r0. They do have jigo files for building the ISOs, but alas, they don't have all the necessary constituent .deb files to create those ISOs from the jigdo files. I do have good verified copies of all the other CD ISOs in that set (debian-30r0-i386-binary-[2-7].iso). Anyway, I'd be interested in obtaining/assembling a good debian-30r0-i386-binary-1.iso, and Debian itself may then also be interested in getting from the debian-30r0-i386-binary-[1-7].iso set, the missing needed .deb files for the archives, so the ISOs could then be reconstructed by anyone desiring to with the jigdo files they do have available.