Package us.codecraft.webmagic.samples.scheduler

Class DelayQueueScheduler

java.lang.Object

us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

us.codecraft.webmagic.scheduler.PriorityScheduler

us.codecraft.webmagic.samples.scheduler.DelayQueueScheduler

All Implemented Interfaces:

MonitorableScheduler, Scheduler

public class DelayQueueScheduler
extends PriorityScheduler

Author:

code4crafter@gmail.com

Field Summary

Fields inherited from class us.codecraft.webmagic.scheduler.PriorityScheduler

INITIAL_CAPACITY

Fields inherited from class us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

logger

Constructor Summary

Constructors

Constructor	Description
DelayQueueScheduler(long time, java.util.concurrent.TimeUnit timeUnit)

Method Summary

Modifier and Type	Method	Description
Request	poll(Task task)	get an url to crawl
void	push(Request request, Task task)	add a url to fetch

Methods inherited from class us.codecraft.webmagic.scheduler.PriorityScheduler

getLeftRequestsCount, getTotalRequestsCount, pushWhenNoDuplicate

Methods inherited from class us.codecraft.webmagic.scheduler.DuplicateRemovedScheduler

getDuplicateRemover, noNeedToRemoveDuplicate, setDuplicateRemover, shouldReserved

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DelayQueueScheduler

public DelayQueueScheduler(long time,
                           java.util.concurrent.TimeUnit timeUnit)

Method Detail

push

public void push(Request request,
                 Task task)

Description copied from interface: Scheduler

add a url to fetch

Specified by:

push in interface Scheduler

Overrides:

push in class DuplicateRemovedScheduler

Parameters:

request - request

task - task

poll

public Request poll(Task task)

Description copied from interface: Scheduler

get an url to crawl

Specified by:

poll in interface Scheduler

Overrides:

poll in class PriorityScheduler

Parameters:

task - the task of spider

Returns:

the url to crawl