heritrix是一种开源的网络爬虫/网络蜘蛛，heritrix目的是能够跟踪页面的url进行扩展的抓取，最后为搜索引擎提供广泛的数据来源。-heritrix is an open source network reptiles/Web Spiders, heritrix purpose is to track the page url to the expansion of the crawl, and finally for the search engine provides a wide