Class FilePageModelPipeline

  • All Implemented Interfaces:
    PageModelPipeline

    public class FilePageModelPipeline
    extends us.codecraft.webmagic.utils.FilePersistentBase
    implements PageModelPipeline
    Store results objects (page models) to files in plain format.
    Use model.getKey() as file name if the model implements HasKey.
    Otherwise use SHA1 as file name.
    Since:
    0.3.0
    Author:
    code4crafter@gmail.com
    • Field Summary

      • Fields inherited from class us.codecraft.webmagic.utils.FilePersistentBase

        path, PATH_SEPERATOR
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void process​(java.lang.Object o, us.codecraft.webmagic.Task task)  
      • Methods inherited from class us.codecraft.webmagic.utils.FilePersistentBase

        checkAndMakeParentDirecotry, getFile, getPath, setPath
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • FilePageModelPipeline

        public FilePageModelPipeline()
        new JsonFilePageModelPipeline with default path "/data/webmagic/"
      • FilePageModelPipeline

        public FilePageModelPipeline​(java.lang.String path)
    • Method Detail

      • process

        public void process​(java.lang.Object o,
                            us.codecraft.webmagic.Task task)
        Specified by:
        process in interface PageModelPipeline