2229491 comment Comment Use the Archive's crawler (Score 2, Informative) 29 by Danton on Wednesday December 31, 2003 @08:22PM (#7849878) Attached to: Do-It-Yourself Internet Archiving? How about using Heritrix, the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler?