Package us.codecraft.webmagic.recover

Class DuplicateStorageRemover

  • All Implemented Interfaces:
    us.codecraft.webmagic.scheduler.component.DuplicateRemover
    public class DuplicateStorageRemover
extends java.lang.Object
implements us.codecraft.webmagic.scheduler.component.DuplicateRemover
    Author:
    ：linweisen

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int getTotalRequestsCount​(us.codecraft.webmagic.Task task)  
      boolean isDuplicate​(us.codecraft.webmagic.Request request, us.codecraft.webmagic.Task task)  
      void resetDuplicateCheck​(us.codecraft.webmagic.Task task)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • DuplicateStorageRemover

        public DuplicateStorageRemover​(java.lang.String path)

    • Method Detail

      • isDuplicate

        public boolean isDuplicate​(us.codecraft.webmagic.Request request,
                           us.codecraft.webmagic.Task task)
        Specified by:
        isDuplicate in interface us.codecraft.webmagic.scheduler.component.DuplicateRemover

      • resetDuplicateCheck

        public void resetDuplicateCheck​(us.codecraft.webmagic.Task task)
        Specified by:
        resetDuplicateCheck in interface us.codecraft.webmagic.scheduler.component.DuplicateRemover

      • getTotalRequestsCount

        public int getTotalRequestsCount​(us.codecraft.webmagic.Task task)
        Specified by:
        getTotalRequestsCount in interface us.codecraft.webmagic.scheduler.component.DuplicateRemover