Class HashSetDuplicateRemover
- java.lang.Object
-
- us.codecraft.webmagic.scheduler.component.HashSetDuplicateRemover
-
- All Implemented Interfaces:
DuplicateRemover
public class HashSetDuplicateRemover extends java.lang.Object implements DuplicateRemover
- Author:
- code4crafer@gmail.com
-
-
Constructor Summary
Constructors Constructor Description HashSetDuplicateRemover()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description int
getTotalRequestsCount(Task task)
Get TotalRequestsCount for monitor.protected java.lang.String
getUrl(Request request)
boolean
isDuplicate(Request request, Task task)
Check whether the request is duplicate.void
resetDuplicateCheck(Task task)
Reset duplicate check.
-
-
-
Method Detail
-
isDuplicate
public boolean isDuplicate(Request request, Task task)
Description copied from interface:DuplicateRemover
Check whether the request is duplicate.- Specified by:
isDuplicate
in interfaceDuplicateRemover
- Parameters:
request
- requesttask
- task- Returns:
- true if is duplicate
-
getUrl
protected java.lang.String getUrl(Request request)
-
resetDuplicateCheck
public void resetDuplicateCheck(Task task)
Description copied from interface:DuplicateRemover
Reset duplicate check.- Specified by:
resetDuplicateCheck
in interfaceDuplicateRemover
- Parameters:
task
- task
-
getTotalRequestsCount
public int getTotalRequestsCount(Task task)
Description copied from interface:DuplicateRemover
Get TotalRequestsCount for monitor.- Specified by:
getTotalRequestsCount
in interfaceDuplicateRemover
- Parameters:
task
- task- Returns:
- number of total request
-
-