All Packages

Package Summary
Package
Description
Main class "Spider" and models.
 
Downloader is the part that downloads web pages and store in Page object.
 
 
 
 
Page model and annotations used to customize a crawler.
Annotations for defining a extractor.
 
 
 
 
 
Pipeline is the persistent and offline process part of crawler.
PageProcessor custom part of a crawler for specific site.
 
 
 
 
 
 
 
Scheduler is the part of url management.
Component of scheduler.
 
 
 
Selectors for page extraction.
 
Static utils of webmagic.