Class JsonFilePageModelPipeline

  • All Implemented Interfaces:
    PageModelPipeline

    public class JsonFilePageModelPipeline
    extends us.codecraft.webmagic.utils.FilePersistentBase
    implements PageModelPipeline
    Store results objects (page models) to files in JSON format.
    Use model.getKey() as file name if the model implements HasKey.
    Otherwise use SHA1 as file name.
    Since:
    0.2.0
    Author:
    code4crafter@gmail.com
    • Field Summary

      • Fields inherited from class us.codecraft.webmagic.utils.FilePersistentBase

        path, PATH_SEPERATOR
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void process​(java.lang.Object o, us.codecraft.webmagic.Task task)  
      • Methods inherited from class us.codecraft.webmagic.utils.FilePersistentBase

        checkAndMakeParentDirecotry, getFile, getPath, setPath
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • JsonFilePageModelPipeline

        public JsonFilePageModelPipeline()
        new JsonFilePageModelPipeline with default path "/data/webmagic/"
      • JsonFilePageModelPipeline

        public JsonFilePageModelPipeline​(java.lang.String path)
    • Method Detail

      • process

        public void process​(java.lang.Object o,
                            us.codecraft.webmagic.Task task)
        Specified by:
        process in interface PageModelPipeline