Package us.codecraft.webmagic

Class Page

java.lang.Object
us.codecraft.webmagic.Page
public class Page extends Object
Object storing extracted result and urls to fetch.
Not thread safe.
Main method：
getUrl() get url of current page
getHtml() get content of current page
putField(String, Object) save extracted result
getResultItems() get extract results to be used in Pipeline
addTargetRequests(Iterable) addTargetRequest(String) add urls to fetch
Since:
0.1.0
Author:
code4crafter@gmail.com
See Also:

  • Constructor Details

    • Page

      public Page()

  • Method Details

    • ofSuccess

      public static Page ofSuccess(Request request)
      Returns a Page with downloadSuccess is true, and request is specified.
      Parameters:
      request - the request.
      Since:
      1.0.2

    • ofFailure

      public static Page ofFailure(Request request)
      Returns a Page with downloadSuccess is true, and request is specified.
      Parameters:
      request - the request.
      Since:
      1.0.2

    • fail

      @Deprecated public static Page fail()
      Deprecated.
      Use fail(Request) instead.
      Returns a Page with downloadSuccess is false.
      Returns:
      the page.

    • fail

      @Deprecated(since="1.0.2", forRemoval=true) public static Page fail(Request request)
      Deprecated, for removal: This API element is subject to removal in a future version.
      Use ofFailure(Request) instead.
      Returns a Page with downloadSuccess is false, and request is specified.
      Parameters:
      request - the Request.
      Returns:
      the page.
      Since:
      0.10.0

    • setSkip

      public Page setSkip(boolean skip)

    • putField

      public void putField(String key, Object field)
      store extract results
      Parameters:
      key - key
      field - field

    • getHtml

      public Html getHtml()
      get html content of page
      Returns:
      html

    • getJson

      public Json getJson()
      get json content of page
      Returns:
      json
      Since:
      0.5.0

    • setHtml

      @Deprecated public void setHtml(Html html)
      Deprecated.
      since 0.4.0 The html is parse just when first time of calling getHtml(), so use setRawText(String) instead.
      Parameters:
      html - html

    • getTargetRequests

      public List<Request> getTargetRequests()

    • addTargetRequests

      public void addTargetRequests(Iterable<String> requests)
      add urls to fetch
      Parameters:
      requests - requests

    • addTargetRequests

      public void addTargetRequests(Iterable<String> requests, long priority)
      add urls to fetch
      Parameters:
      requests - requests
      priority - priority

    • addTargetRequest

      public void addTargetRequest(String requestString)
      add url to fetch
      Parameters:
      requestString - requestString

    • addTargetRequest

      public void addTargetRequest(Request request)
      add requests to fetch
      Parameters:
      request - request

    • getUrl

      public Selectable getUrl()
      get url of current page
      Returns:
      url of current page

    • setUrl

      public void setUrl(Selectable url)

    • getRequest

      public Request getRequest()
      get request of current page
      Returns:
      request

    • setRequest

      public void setRequest(Request request)

    • getResultItems

      public ResultItems getResultItems()

    • getStatusCode

      public int getStatusCode()

    • setStatusCode

      public void setStatusCode(int statusCode)

    • getRawText

      public String getRawText()

    • setRawText

      public Page setRawText(String rawText)

    • getHeaders

      public Map<String,List<String>> getHeaders()

    • setHeaders

      public void setHeaders(Map<String,List<String>> headers)

    • isDownloadSuccess

      public boolean isDownloadSuccess()

    • setDownloadSuccess

      public void setDownloadSuccess(boolean downloadSuccess)

    • getBytes

      public byte[] getBytes()

    • setBytes

      public void setBytes(byte[] bytes)

    • getCharset

      public String getCharset()

    • setCharset

      public void setCharset(String charset)

    • toString

      public String toString()
      Overrides:
      toString in class Object