├── README.md ├── config_load.py ├── global_value.py ├── log.py ├── mini_spider.py ├── spider.conf ├── test ├── test_config_load.py └── test_webpage_utils.py ├── url_item.py ├── urls └── webpage_utils.py /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/README.md -------------------------------------------------------------------------------- /config_load.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/config_load.py -------------------------------------------------------------------------------- /global_value.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/global_value.py -------------------------------------------------------------------------------- /log.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/log.py -------------------------------------------------------------------------------- /mini_spider.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/mini_spider.py -------------------------------------------------------------------------------- /spider.conf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/spider.conf -------------------------------------------------------------------------------- /test/test_config_load.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/test/test_config_load.py -------------------------------------------------------------------------------- /test/test_webpage_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/test/test_webpage_utils.py -------------------------------------------------------------------------------- /url_item.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/url_item.py -------------------------------------------------------------------------------- /urls: -------------------------------------------------------------------------------- 1 | http://www.qq.com 2 | -------------------------------------------------------------------------------- /webpage_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suliangxd/multithreading-spider/HEAD/webpage_utils.py --------------------------------------------------------------------------------