Package us.codecraft.webmagic.downloader.selenium

Class SeleniumDownloader

java.lang.Object

us.codecraft.webmagic.downloader.AbstractDownloader

us.codecraft.webmagic.downloader.selenium.SeleniumDownloader

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, Downloader

public class SeleniumDownloader
extends AbstractDownloader
implements java.io.Closeable

使用Selenium调用浏览器进行渲染。目前仅支持chrome。
需要下载Selenium driver支持。

Author:

code4crafter@gmail.com
Date: 13-7-26
Time: 下午1:37

Constructor Summary

Constructors

Constructor	Description
SeleniumDownloader()	Constructor without any filed.
SeleniumDownloader(java.lang.String chromeDriverPath)	新建

Method Summary

Modifier and Type	Method	Description
void	close()
Page	download(Request request, Task task)	Downloads web pages and store in Page object.
SeleniumDownloader	setSleepTime(int sleepTime)	set sleep time to wait until load success
void	setThread(int thread)	Tell the downloader how many threads the spider used.

Methods inherited from class us.codecraft.webmagic.downloader.AbstractDownloader

download, download, onError, onError, onError, onSuccess, onSuccess, onSuccess

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SeleniumDownloader

public SeleniumDownloader(java.lang.String chromeDriverPath)

新建

Parameters:

chromeDriverPath - chromeDriverPath

SeleniumDownloader

public SeleniumDownloader()

Constructor without any filed. Construct PhantomJS browser

Method Detail

setSleepTime

public SeleniumDownloader setSleepTime(int sleepTime)

set sleep time to wait until load success

Parameters:

sleepTime - sleepTime

Returns:

this

download

public Page download(Request request,
                     Task task)

Description copied from interface: Downloader

Downloads web pages and store in Page object.

Specified by:

download in interface Downloader

Parameters:

request - request

task - task

Returns:

page

setThread

public void setThread(int thread)

Description copied from interface: Downloader

Tell the downloader how many threads the spider used.

Specified by:

setThread in interface Downloader

Parameters:

thread - number of threads

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Throws:

java.io.IOException