Package us.codecraft.webmagic.scheduler

Class Summary

Class	Description
BloomFilterDuplicateRemover	BloomFilterDuplicateRemover for huge number of urls.
FileCacheQueueScheduler	Store urls and cursor in files so that a Spider can resume the status when shutdown.
RedisPriorityScheduler	the redis scheduler with priority
RedisScheduler	Use Redis as url scheduler for distributed crawlers.