├── Logger.py ├── README.md ├── Utility.py ├── base.py ├── crawl ├── downloader.py ├── maps.cfg ├── pipeline.py ├── scheduler.py ├── settings.py └── test.py /Logger.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/Logger.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/README.md -------------------------------------------------------------------------------- /Utility.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/Utility.py -------------------------------------------------------------------------------- /base.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/base.py -------------------------------------------------------------------------------- /crawl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/crawl -------------------------------------------------------------------------------- /downloader.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/downloader.py -------------------------------------------------------------------------------- /maps.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/maps.cfg -------------------------------------------------------------------------------- /pipeline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/pipeline.py -------------------------------------------------------------------------------- /scheduler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/scheduler.py -------------------------------------------------------------------------------- /settings.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/settings.py -------------------------------------------------------------------------------- /test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/weizetao/spider-roach/HEAD/test.py --------------------------------------------------------------------------------