Heritrix: Internet Archive Web Crawler The archive-crawler project is building a flexible, extensible, robust, and scalable web crawler capable of fetching, archiving, and analyzing the full diversity and breadth of in

Heritrix是一个爬虫框架，可加如入一些可互换的组件。 它的执行是递归进行的，主要有以下几步： 1。在预定的URI中选择一个。 2。获取URI 3。分析，归档结果 4。选择已经发现的感兴趣的URI。加入预定队列。 5。标记已经处理过的URI -Heritrix is a fr a mework for reptiles, such as income may be a number of interchangeable com