├── .gitattributes ├── .gitignore ├── .idea ├── .gitignore ├── encodings.xml ├── inspectionProfiles │ ├── Project_Default.xml │ └── profiles_settings.xml ├── misc.xml ├── modules.xml ├── vcs.xml └── 中国大学生计算机设计大赛_抓取.iml ├── LICENSE ├── README.md ├── config.py ├── database ├── cnjsj_base.csv ├── error_log.txt ├── fake_useragent_0.1.11.json ├── id_pool.xlsx ├── id_set_01.csv ├── psar │ ├── ASH.json │ ├── 华中科技大学_分析报告.json │ └── 海南大学_分析报告.json └── tpd │ ├── class_name.txt │ ├── get_school_211.txt │ └── get_school_985.txt ├── main.py ├── middleware ├── app.py ├── chromedriver.exe ├── cmp_data.py ├── proxy_key.py └── raw_data_load.py ├── requirements.txt └── spiders ├── cnjsj.py ├── ppy_flow.py └── sns_project.py /.gitattributes: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.gitattributes -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.gitignore -------------------------------------------------------------------------------- /.idea/.gitignore: -------------------------------------------------------------------------------- 1 | # Default ignored files 2 | /shelf/ 3 | /workspace.xml 4 | -------------------------------------------------------------------------------- /.idea/encodings.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/encodings.xml -------------------------------------------------------------------------------- /.idea/inspectionProfiles/Project_Default.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/inspectionProfiles/Project_Default.xml -------------------------------------------------------------------------------- /.idea/inspectionProfiles/profiles_settings.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/inspectionProfiles/profiles_settings.xml -------------------------------------------------------------------------------- /.idea/misc.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/misc.xml -------------------------------------------------------------------------------- /.idea/modules.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/modules.xml -------------------------------------------------------------------------------- /.idea/vcs.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/vcs.xml -------------------------------------------------------------------------------- /.idea/中国大学生计算机设计大赛_抓取.iml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/.idea/中国大学生计算机设计大赛_抓取.iml -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/README.md -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/config.py -------------------------------------------------------------------------------- /database/cnjsj_base.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/cnjsj_base.csv -------------------------------------------------------------------------------- /database/error_log.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/error_log.txt -------------------------------------------------------------------------------- /database/fake_useragent_0.1.11.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/fake_useragent_0.1.11.json -------------------------------------------------------------------------------- /database/id_pool.xlsx: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /database/id_set_01.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/id_set_01.csv -------------------------------------------------------------------------------- /database/psar/ASH.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/psar/ASH.json -------------------------------------------------------------------------------- /database/psar/华中科技大学_分析报告.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/psar/华中科技大学_分析报告.json -------------------------------------------------------------------------------- /database/psar/海南大学_分析报告.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/psar/海南大学_分析报告.json -------------------------------------------------------------------------------- /database/tpd/class_name.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/tpd/class_name.txt -------------------------------------------------------------------------------- /database/tpd/get_school_211.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/tpd/get_school_211.txt -------------------------------------------------------------------------------- /database/tpd/get_school_985.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/database/tpd/get_school_985.txt -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/main.py -------------------------------------------------------------------------------- /middleware/app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/middleware/app.py -------------------------------------------------------------------------------- /middleware/chromedriver.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/middleware/chromedriver.exe -------------------------------------------------------------------------------- /middleware/cmp_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/middleware/cmp_data.py -------------------------------------------------------------------------------- /middleware/proxy_key.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/middleware/proxy_key.py -------------------------------------------------------------------------------- /middleware/raw_data_load.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/middleware/raw_data_load.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | openpyxl 2 | gevent 3 | requests 4 | selenium 5 | bs4 6 | retrying -------------------------------------------------------------------------------- /spiders/cnjsj.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/spiders/cnjsj.py -------------------------------------------------------------------------------- /spiders/ppy_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/spiders/ppy_flow.py -------------------------------------------------------------------------------- /spiders/sns_project.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/QIN2DIM/4C-ExcellentProject/HEAD/spiders/sns_project.py --------------------------------------------------------------------------------