Package us.codecraft.webmagic.pipeline

Class JsonFilePageModelPipeline

java.lang.Object
us.codecraft.webmagic.utils.FilePersistentBase
us.codecraft.webmagic.pipeline.JsonFilePageModelPipeline
All Implemented Interfaces:
PageModelPipeline
public class JsonFilePageModelPipeline extends us.codecraft.webmagic.utils.FilePersistentBase implements PageModelPipeline
Store results objects (page models) to files in JSON format.
Use model.getKey() as file name if the model implements HasKey.
Otherwise use SHA1 as file name.
Since:
0.2.0
Author:
code4crafter@gmail.com

  • Field Summary

    Fields inherited from class us.codecraft.webmagic.utils.FilePersistentBase

    path, PATH_SEPERATOR

  • Constructor Summary

    Constructors
    Constructor
    Description
    JsonFilePageModelPipeline()
    new JsonFilePageModelPipeline with default path "/data/webmagic/"
    JsonFilePageModelPipeline(String path)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    process(Object o, us.codecraft.webmagic.Task task)
     

    Methods inherited from class us.codecraft.webmagic.utils.FilePersistentBase

    checkAndMakeParentDirecotry, getFile, getPath, setPath

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • JsonFilePageModelPipeline

      public JsonFilePageModelPipeline()
      new JsonFilePageModelPipeline with default path "/data/webmagic/"

    • JsonFilePageModelPipeline

      public JsonFilePageModelPipeline(String path)

  • Method Details