├── .gitignore ├── README.md ├── img ├── ptt_console_view.png ├── ptt_page_view.png ├── ptt_post_related.png ├── ptt_search_author.png ├── ptt_search_kw.png ├── ptt_search_recommend.png ├── ptt_search_title.png ├── ptt_source_parse_query.png ├── ptt_source_tree.png ├── ptt_source_tree_page_control.png ├── step3_result_rich_table.png └── step4_result_rich_tables.png ├── ptt-parser ├── README.md ├── main.py ├── ptt │ ├── __init__.py │ ├── cli.py │ ├── core.py │ ├── model.py │ └── parser.py ├── setup.cfg ├── setup.py └── test │ ├── __init__.py │ ├── test_core.py │ ├── test_example.py │ └── test_search.py └── src ├── _soup-based ├── crawler_5.py ├── crawler_6.py ├── crawler_7.py └── crawler_8.py └── basic_crawler.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/README.md -------------------------------------------------------------------------------- /img/ptt_console_view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_console_view.png -------------------------------------------------------------------------------- /img/ptt_page_view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_page_view.png -------------------------------------------------------------------------------- /img/ptt_post_related.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_post_related.png -------------------------------------------------------------------------------- /img/ptt_search_author.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_search_author.png -------------------------------------------------------------------------------- /img/ptt_search_kw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_search_kw.png -------------------------------------------------------------------------------- /img/ptt_search_recommend.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_search_recommend.png -------------------------------------------------------------------------------- /img/ptt_search_title.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_search_title.png -------------------------------------------------------------------------------- /img/ptt_source_parse_query.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_source_parse_query.png -------------------------------------------------------------------------------- /img/ptt_source_tree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_source_tree.png -------------------------------------------------------------------------------- /img/ptt_source_tree_page_control.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/ptt_source_tree_page_control.png -------------------------------------------------------------------------------- /img/step3_result_rich_table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/step3_result_rich_table.png -------------------------------------------------------------------------------- /img/step4_result_rich_tables.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/img/step4_result_rich_tables.png -------------------------------------------------------------------------------- /ptt-parser/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/README.md -------------------------------------------------------------------------------- /ptt-parser/main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/main.py -------------------------------------------------------------------------------- /ptt-parser/ptt/__init__.py: -------------------------------------------------------------------------------- 1 | from ptt.core import Ptt, Board # noqa 2 | 3 | __VERSION__ = '0.1.0' 4 | -------------------------------------------------------------------------------- /ptt-parser/ptt/cli.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/ptt/cli.py -------------------------------------------------------------------------------- /ptt-parser/ptt/core.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/ptt/core.py -------------------------------------------------------------------------------- /ptt-parser/ptt/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/ptt/model.py -------------------------------------------------------------------------------- /ptt-parser/ptt/parser.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/ptt/parser.py -------------------------------------------------------------------------------- /ptt-parser/setup.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/setup.cfg -------------------------------------------------------------------------------- /ptt-parser/setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/setup.py -------------------------------------------------------------------------------- /ptt-parser/test/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ptt-parser/test/test_core.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/test/test_core.py -------------------------------------------------------------------------------- /ptt-parser/test/test_example.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/test/test_example.py -------------------------------------------------------------------------------- /ptt-parser/test/test_search.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/ptt-parser/test/test_search.py -------------------------------------------------------------------------------- /src/_soup-based/crawler_5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/src/_soup-based/crawler_5.py -------------------------------------------------------------------------------- /src/_soup-based/crawler_6.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/src/_soup-based/crawler_6.py -------------------------------------------------------------------------------- /src/_soup-based/crawler_7.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/src/_soup-based/crawler_7.py -------------------------------------------------------------------------------- /src/_soup-based/crawler_8.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/src/_soup-based/crawler_8.py -------------------------------------------------------------------------------- /src/basic_crawler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/leVirve/CrawlerTutorial/HEAD/src/basic_crawler.py --------------------------------------------------------------------------------