Package us.codecraft.webmagic.recover

Class DuplicateStorageRemover

java.lang.Object
us.codecraft.webmagic.recover.DuplicateStorageRemover
All Implemented Interfaces:
us.codecraft.webmagic.scheduler.component.DuplicateRemover
public class DuplicateStorageRemover extends Object implements us.codecraft.webmagic.scheduler.component.DuplicateRemover
Author:
：linweisen

  • Constructor Details

    • DuplicateStorageRemover

      public DuplicateStorageRemover(String path)

  • Method Details

    • isDuplicate

      public boolean isDuplicate(us.codecraft.webmagic.Request request, us.codecraft.webmagic.Task task)
      Specified by:
      isDuplicate in interface us.codecraft.webmagic.scheduler.component.DuplicateRemover

    • resetDuplicateCheck

      public void resetDuplicateCheck(us.codecraft.webmagic.Task task)
      Specified by:
      resetDuplicateCheck in interface us.codecraft.webmagic.scheduler.component.DuplicateRemover

    • getTotalRequestsCount

      public int getTotalRequestsCount(us.codecraft.webmagic.Task task)
      Specified by:
      getTotalRequestsCount in interface us.codecraft.webmagic.scheduler.component.DuplicateRemover