Uses of Class
us.codecraft.webmagic.Page
-
Packages that use Page Package Description us.codecraft.webmagic Main class "Spider" and models.us.codecraft.webmagic.downloader Downloader is the part that downloads web pages and store in Page object.us.codecraft.webmagic.processor PageProcessor custom part of a crawler for specific site.us.codecraft.webmagic.processor.example us.codecraft.webmagic.proxy -
-
Uses of Page in us.codecraft.webmagic
Methods in us.codecraft.webmagic that return Page Modifier and Type Method Description static Page
Page. fail()
Deprecated.Usefail(Request)
instead.static Page
Page. fail(Request request)
Page
Page. setRawText(java.lang.String rawText)
Page
Page. setSkip(boolean skip)
Methods in us.codecraft.webmagic with parameters of type Page Modifier and Type Method Description protected void
Spider. extractAndAddRequests(Page page, boolean spawnUrl)
-
Uses of Page in us.codecraft.webmagic.downloader
Methods in us.codecraft.webmagic.downloader that return Page Modifier and Type Method Description Page
Downloader. download(Request request, Task task)
Downloads web pages and store in Page object.Page
HttpClientDownloader. download(Request request, Task task)
protected Page
HttpClientDownloader. handleResponse(Request request, java.lang.String charset, org.apache.http.HttpResponse httpResponse, Task task)
Methods in us.codecraft.webmagic.downloader with parameters of type Page Modifier and Type Method Description protected void
AbstractDownloader. onError(Page page, Task task, java.lang.Throwable e)
protected void
AbstractDownloader. onSuccess(Page page, Task task)
-
Uses of Page in us.codecraft.webmagic.processor
Methods in us.codecraft.webmagic.processor with parameters of type Page Modifier and Type Method Description void
PageProcessor. process(Page page)
Processes the page, extract URLs to fetch, extract the data and store.void
SimplePageProcessor. process(Page page)
-
Uses of Page in us.codecraft.webmagic.processor.example
Methods in us.codecraft.webmagic.processor.example with parameters of type Page Modifier and Type Method Description void
BaiduBaikePageProcessor. process(Page page)
void
GithubRepoPageProcessor. process(Page page)
void
ZhihuPageProcessor. process(Page page)
-
Uses of Page in us.codecraft.webmagic.proxy
Methods in us.codecraft.webmagic.proxy with parameters of type Page Modifier and Type Method Description void
ProxyProvider. returnProxy(Proxy proxy, Page page, Task task)
Return proxy to Provider when complete a download.void
SimpleProxyProvider. returnProxy(Proxy proxy, Page page, Task task)
-