├── README.md ├── data ├── README.md ├── id_info.json ├── issues_message.json ├── message_detail_text.json ├── message_info.json ├── minhash_dict.pickle ├── name2fakeid.json ├── 微信公众号聚合平台_按公众号区分.md └── 微信公众号聚合平台_按时间区分.md ├── figures └── blog_preview.png ├── main.py ├── requirements.txt ├── scripts └── daily_update.sh └── src ├── __init__.py ├── crawler ├── __init__.py └── wechat_request.py ├── processor ├── __init__.py ├── deduplication.py └── message_converter.py ├── utils ├── __init__.py ├── data_manager.py └── helpers.py └── web ├── __init__.py └── blog_generator.py /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/README.md -------------------------------------------------------------------------------- /data/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/README.md -------------------------------------------------------------------------------- /data/id_info.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/id_info.json -------------------------------------------------------------------------------- /data/issues_message.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/issues_message.json -------------------------------------------------------------------------------- /data/message_detail_text.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/message_detail_text.json -------------------------------------------------------------------------------- /data/message_info.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/message_info.json -------------------------------------------------------------------------------- /data/minhash_dict.pickle: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/minhash_dict.pickle -------------------------------------------------------------------------------- /data/name2fakeid.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/name2fakeid.json -------------------------------------------------------------------------------- /data/微信公众号聚合平台_按公众号区分.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/微信公众号聚合平台_按公众号区分.md -------------------------------------------------------------------------------- /data/微信公众号聚合平台_按时间区分.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/data/微信公众号聚合平台_按时间区分.md -------------------------------------------------------------------------------- /figures/blog_preview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/figures/blog_preview.png -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/main.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | DrissionPage 2 | lxml 3 | requests 4 | tqdm 5 | datasketch -------------------------------------------------------------------------------- /scripts/daily_update.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/scripts/daily_update.sh -------------------------------------------------------------------------------- /src/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/__init__.py -------------------------------------------------------------------------------- /src/crawler/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/crawler/__init__.py -------------------------------------------------------------------------------- /src/crawler/wechat_request.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/crawler/wechat_request.py -------------------------------------------------------------------------------- /src/processor/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/processor/__init__.py -------------------------------------------------------------------------------- /src/processor/deduplication.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/processor/deduplication.py -------------------------------------------------------------------------------- /src/processor/message_converter.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/processor/message_converter.py -------------------------------------------------------------------------------- /src/utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/utils/__init__.py -------------------------------------------------------------------------------- /src/utils/data_manager.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/utils/data_manager.py -------------------------------------------------------------------------------- /src/utils/helpers.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/utils/helpers.py -------------------------------------------------------------------------------- /src/web/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/web/__init__.py -------------------------------------------------------------------------------- /src/web/blog_generator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ZejunCao/WeChatOA_Aggregation/HEAD/src/web/blog_generator.py --------------------------------------------------------------------------------