Package us.codecraft.webmagic.downloader
Class HttpClientDownloader
- java.lang.Object
-
- us.codecraft.webmagic.downloader.AbstractDownloader
-
- us.codecraft.webmagic.downloader.HttpClientDownloader
-
- All Implemented Interfaces:
Downloader
public class HttpClientDownloader extends AbstractDownloader
The http downloader based on HttpClient.- Since:
- 0.1.0
- Author:
- code4crafter@gmail.com
-
-
Constructor Summary
Constructors Constructor Description HttpClientDownloader()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Page
download(Request request, Task task)
Downloads web pages and store in Page object.protected Page
handleResponse(Request request, java.lang.String charset, org.apache.http.HttpResponse httpResponse, Task task)
void
setHttpUriRequestConverter(HttpUriRequestConverter httpUriRequestConverter)
void
setProxyProvider(ProxyProvider proxyProvider)
void
setThread(int thread)
Tell the downloader how many threads the spider used.
-
-
-
Method Detail
-
setHttpUriRequestConverter
public void setHttpUriRequestConverter(HttpUriRequestConverter httpUriRequestConverter)
-
setProxyProvider
public void setProxyProvider(ProxyProvider proxyProvider)
-
download
public Page download(Request request, Task task)
Description copied from interface:Downloader
Downloads web pages and store in Page object.- Parameters:
request
- requesttask
- task- Returns:
- page
-
setThread
public void setThread(int thread)
Description copied from interface:Downloader
Tell the downloader how many threads the spider used.- Parameters:
thread
- number of threads
-
-