├── .gitignore ├── README.md ├── cookiespool ├── __init__.py ├── api.py ├── config.py ├── db.py ├── error.py ├── generator.py ├── importer.py ├── scheduler.py ├── tester.py └── verify.py ├── importer.py ├── requirements.txt ├── run.py └── tests └── login.py /.gitignore: -------------------------------------------------------------------------------- 1 | /.idea 2 | *.pyc 3 | ghostdriver.log -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CookiesPool / Cookies池 2 | 3 | 可扩展的Cookies池，目前对接了新浪微博，可自行扩展其他站点 4 | 5 | 优化版(推荐)：[https://github.com/Python3WebSpider/CookiesPool](https://github.com/Python3WebSpider/CookiesPool) 6 | 7 | ## 安装 8 | 9 | ``` 10 | pip3 install -r requirements.txt 11 | ``` 12 | 13 | ## 基础配置 14 | 15 | 修改cookiespool/config.py 16 | 17 | ### 数据库配置 18 | 19 | account:weibo:账号 20 | 21 | cookies:weibo:账号 22 | 23 | Value分别为密码和Cookies 24 | 25 | 账号自行某宝购买 26 | 27 | Redis连接信息到cookiespool/config文件修改 28 | 29 | ### 云打码平台配置 30 | 31 | 到yundama.com注册开发者和普通用户。 32 | 33 | 开发者申请应用ID和KEY，普通用户用于充值登录。 34 | 35 | 配置信息到cookiespool/config文件修改 36 | 37 | 38 | ### 进程开关 39 | 40 | 配置信息到cookiespool/config文件修改 41 | 42 | ## 运行 43 | 44 | ``` 45 | python3 run.py 46 | ``` 47 | 48 | ## 批量导入 49 | 50 | ``` 51 | python3 importer.py 52 | ``` 53 | 54 | ``` 55 | 请输入账号密码组, 输入exit退出读入 56 | 18459748505----astvar3647 57 | 14760253606----gmidy8470 58 | 14760253607----uoyuic8427 59 | 18459749258----rktfye8937 60 | 账号 18459748505 密码 astvar3647 61 | 录入成功 62 | 账号 14760253606 密码 gmidy8470 63 | 录入成功 64 | 账号 14760253607 密码 uoyuic8427 65 | 录入成功 66 | 账号 18459749258 密码 rktfye8937 67 | 录入成功 68 | exit 69 | ``` 70 | 71 | ## 运行效果 72 | 73 | 开启Generator、API，关闭Tester为例： 74 | 75 | ``` 76 | Generating Cookies 77 | * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) 78 | Getting 526 accounts from Redis 79 | Getting Cookies of weibo uva68312006hao@163.com 1nr0nkwes7 80 | Generating Cookies of uva68312006hao@163.com 81 | 出现验证码，开始识别验证码 82 | Retrying: 1400009488 Count: 1 83 | {'cid': 1400009488, 'method': 'result'} 84 | {'cid': 1400009488, 'ret': -3002, 'text': ''} 85 | 云打码验证码还在识别 86 | Retrying: 1400009488 Count: 2 87 | {'cid': 1400009488, 'method': 'result'} 88 | {'cid': 1400009488, 'ret': -3002, 'text': ''} 89 | 云打码验证码还在识别 90 | Retrying: 1400009488 Count: 3 91 | {'cid': 1400009488, 'method': 'result'} 92 | {'cid': 1400009488, 'ret': -3002, 'text': ''} 93 | 云打码验证码还在识别 94 | Retrying: 1400009488 Count: 4 95 | {'cid': 1400009488, 'method': 'result'} 96 | {'cid': 1400009488, 'ret': -3002, 'text': ''} 97 | 云打码验证码还在识别 98 | Retrying: 1400009488 Count: 5 99 | {'cid': 1400009488, 'method': 'result'} 100 | {'cid': 1400009488, 'ret': 0, 'text': '4uh4h'} 101 | 登录成功 102 | [{'expiry': 1525092087.375918, 'secure': False, 'value': '0cS3YKcgWrDRsV', 'httpOnly': False, 'name': 'SUHB', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1525092087.374765, 'secure': False, 'value': '0033WrSXqPxfM725Ws9jqgMF55529P9D9W5H8du8O9J-.X.2Zd-6LbEK5JpX5o2p5NHD95QfS0ecSKe7e0n4Ws4Dqcj.i--fiK.7iKn4i--ci-z7i-zRi--fi-2fiK.ci--4i-2pi-i2', 'httpOnly': False, 'name': 'SUBP', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1525092087.373734, 'secure': False, 'value': '_2A250AasnDeThGeNJ6FQU8y7PwzWIHXVXDTVvrDV6PUJbktBeLRfukW0wFLR2922NnApkDOZb_eNrRfNBZQ..', 'httpOnly': True, 'name': 'SUB', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1496148087.677672, 'secure': False, 'value': 'b2d2d84c5aafcc02c0dac2faa4acc43f', 'httpOnly': True, 'name': '_T_WM', 'path': '/', 'domain': '.weibo.cn'}, {'secure': False, 'value': '1493556087', 'httpOnly': False, 'name': 'SSOLoginState', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1808916087.372439, 'secure': False, 'value': 'AnPRL-vKWx_-0LWAO4Mk79GSyD6FxNqsFzhSNFOZI1MxKni8Ok7C3thzDOuIdEdh-0SvYb25zbF-znxYY_Z5hK4.', 'httpOnly': True, 'name': 'SCF', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1496148083.263736, 'secure': False, 'value': '1496148083', 'httpOnly': False, 'name': 'ALF', 'path': '/', 'domain': '.weibo.cn'}] 103 | {'SUHB': '0cS3YKcgWrDRsV', 'SUB': '_2A250AasnDeThGeNJ6FQU8y7PwzWIHXVXDTVvrDV6PUJbktBeLRfukW0wFLR2922NnApkDOZb_eNrRfNBZQ..', 'SUBP': '0033WrSXqPxfM725Ws9jqgMF55529P9D9W5H8du8O9J-.X.2Zd-6LbEK5JpX5o2p5NHD95QfS0ecSKe7e0n4Ws4Dqcj.i--fiK.7iKn4i--ci-z7i-zRi--fi-2fiK.ci--4i-2pi-i2', 'SCF': 'AnPRL-vKWx_-0LWAO4Mk79GSyD6FxNqsFzhSNFOZI1MxKni8Ok7C3thzDOuIdEdh-0SvYb25zbF-znxYY_Z5hK4.', 'SSOLoginState': '1493556087', '_T_WM': 'b2d2d84c5aafcc02c0dac2faa4acc43f', 'ALF': '1496148083'} 104 | 成功获取到Cookies 105 | Saving Cookies to Redis uva68312006hao@163.com {"SUHB": "0cS3YKcgWrDRsV", "SUB": "_2A250AasnDeThGeNJ6FQU8y7PwzWIHXVXDTVvrDV6PUJbktBeLRfukW0wFLR2922NnApkDOZb_eNrRfNBZQ..", "SUBP": "0033WrSXqPxfM725Ws9jqgMF55529P9D9W5H8du8O9J-.X.2Zd-6LbEK5JpX5o2p5NHD95QfS0ecSKe7e0n4Ws4Dqcj.i--fiK.7iKn4i--ci-z7i-zRi--fi-2fiK.ci--4i-2pi-i2", "SCF": "AnPRL-vKWx_-0LWAO4Mk79GSyD6FxNqsFzhSNFOZI1MxKni8Ok7C3thzDOuIdEdh-0SvYb25zbF-znxYY_Z5hK4.", "SSOLoginState": "1493556087", "_T_WM": "b2d2d84c5aafcc02c0dac2faa4acc43f", "ALF": "1496148083"} 106 | ``` 107 | -------------------------------------------------------------------------------- /cookiespool/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Germey/CookiesPool/baf1989fcbe56634907ce9c923d74a022740ee87/cookiespool/__init__.py -------------------------------------------------------------------------------- /cookiespool/api.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, g 2 | 3 | from cookiespool.config import * 4 | from cookiespool.db import * 5 | 6 | __all__ = ['app'] 7 | 8 | app = Flask(__name__) 9 | 10 | 11 | @app.route('/') 12 | def index(): 13 | return '

Welcome to Cookie Pool System

' 14 | 15 | 16 | def get_conn(): 17 | """ 18 | 获取 19 | :return: 20 | """ 21 | for name in GENERATOR_MAP: 22 | print(name) 23 | if not hasattr(g, name): 24 | setattr(g, name + '_cookies', eval('CookiesRedisClient' + '(name="' + name + '")')) 25 | setattr(g, name + '_account', eval('AccountRedisClient' + '(name="' + name + '")')) 26 | return g 27 | 28 | 29 | @app.route('//random') 30 | def random(name): 31 | """ 32 | 获取随机的Cookie, 访问地址如 /weibo/random 33 | :return: 随机Cookie 34 | """ 35 | g = get_conn() 36 | cookies = getattr(g, name + '_cookies').random() 37 | return cookies 38 | 39 | @app.route('//add//') 40 | def add(name, username, password): 41 | """ 42 | 添加用户, 访问地址如 /weibo/add/user/password 43 | """ 44 | g = get_conn() 45 | result = getattr(g, name + '_account').set(username, password) 46 | return result 47 | 48 | 49 | @app.route('//count') 50 | def count(name): 51 | """ 52 | 获取Cookies总数 53 | """ 54 | g = get_conn() 55 | count = getattr(g, name + '_cookies').count() 56 | return str(int) if isinstance(count, int) else count 57 | 58 | 59 | if __name__ == '__main__': 60 | app.run(host='0.0.0.0') 61 | -------------------------------------------------------------------------------- /cookiespool/config.py: -------------------------------------------------------------------------------- 1 | # Redis数据库地址 2 | REDIS_HOST = '' 3 | 4 | # Redis端口 5 | REDIS_PORT = 6379 6 | 7 | # Redis密码，如无填None 8 | REDIS_PASSWORD = '' 9 | 10 | # 配置信息，无需修改 11 | REDIS_DOMAIN = '*' 12 | REDIS_NAME = '*' 13 | 14 | # 云打码相关配置到yundama.com申请注册 15 | YUNDAMA_USERNAME = '' 16 | YUNDAMA_PASSWORD = '' 17 | YUNDAMA_APP_ID = '3372' 18 | YUNDAMA_APP_KEY = '1b586a30bfda5c7fa71c881075ba49d0' 19 | 20 | YUNDAMA_API_URL = 'http://api.yundama.com/api.php' 21 | 22 | # 云打码最大尝试次数 23 | YUNDAMA_MAX_RETRY = 20 24 | 25 | # 产生器默认使用的浏览器 26 | DEFAULT_BROWSER = 'Chrome' 27 | 28 | # 产生器类，如扩展其他站点，请在此配置 29 | GENERATOR_MAP = { 30 | 'weibo': 'WeiboCookiesGenerator' 31 | } 32 | 33 | # 测试类，如扩展其他站点，请在此配置 34 | TESTER_MAP = { 35 | 'weibo': 'WeiboValidTester' 36 | } 37 | 38 | # 产生器和验证器循环周期 39 | CYCLE = 120 40 | 41 | # API地址和端口 42 | API_HOST = '127.0.0.1' 43 | API_PORT = 5000 44 | 45 | # 进程开关 46 | # 产生器，模拟登录添加Cookies 47 | GENERATOR_PROCESS = True 48 | # 验证器，循环检测数据库中Cookies是否可用，不可用删除 49 | VALID_PROCESS = False 50 | # API接口服务 51 | API_PROCESS = True 52 | -------------------------------------------------------------------------------- /cookiespool/db.py: -------------------------------------------------------------------------------- 1 | import random 2 | 3 | import redis 4 | 5 | from cookiespool.config import * 6 | from cookiespool.error import * 7 | 8 | 9 | class RedisClient(object): 10 | def __init__(self, host=REDIS_HOST, port=REDIS_PORT, password=REDIS_PASSWORD): 11 | """ 12 | 初始化Redis连接 13 | :param host: 地址 14 | :param port: 端口 15 | :param password: 密码 16 | """ 17 | if password: 18 | self._db = redis.Redis(host=host, port=port, password=password) 19 | else: 20 | self._db = redis.Redis(host=host, port=port) 21 | self.domain = REDIS_DOMAIN 22 | self.name = REDIS_NAME 23 | 24 | def _key(self, key): 25 | """ 26 | 得到格式化的key 27 | :param key: 最后一个参数key 28 | :return: 29 | """ 30 | return "{domain}:{name}:{key}".format(domain=self.domain, name=self.name, key=key) 31 | 32 | def set(self, key, value): 33 | """ 34 | 设置键值对 35 | :param key: 36 | :param value: 37 | :return: 38 | """ 39 | raise NotImplementedError 40 | 41 | def get(self, key): 42 | """ 43 | 根据键名获取键值 44 | :param key: 45 | :return: 46 | """ 47 | raise NotImplementedError 48 | 49 | def delete(self, key): 50 | """ 51 | 根据键名删除键值对 52 | :param key: 53 | :return: 54 | """ 55 | raise NotImplementedError 56 | 57 | def keys(self): 58 | """ 59 | 得到所有的键名 60 | :return: 61 | """ 62 | return self._db.keys('{domain}:{name}:*'.format(domain=self.domain, name=self.name)) 63 | 64 | def flush(self): 65 | """ 66 | 清空数据库, 慎用 67 | :return: 68 | """ 69 | self._db.flushall() 70 | 71 | 72 | class CookiesRedisClient(RedisClient): 73 | def __init__(self, host=REDIS_HOST, port=REDIS_PORT, password=REDIS_PASSWORD, domain='cookies', name='default'): 74 | """ 75 | 管理Cookies的对象 76 | :param host: 地址 77 | :param port: 端口 78 | :param password: 密码 79 | :param domain: 域, 如cookies, account等 80 | :param name: 名称, 一般为站点名, 如 weibo, 默认 default 81 | """ 82 | RedisClient.__init__(self, host, port, password) 83 | self.domain = domain 84 | self.name = name 85 | 86 | def set(self, key, value): 87 | try: 88 | self._db.set(self._key(key), value) 89 | except: 90 | raise SetCookieError 91 | 92 | def get(self, key): 93 | try: 94 | return self._db.get(self._key(key)).decode('utf-8') 95 | except: 96 | return None 97 | 98 | def delete(self, key): 99 | try: 100 | print('Delete', key) 101 | return self._db.delete(self._key(key)) 102 | except: 103 | raise DeleteCookieError 104 | 105 | def random(self): 106 | """ 107 | 随机得到一Cookies 108 | :return: 109 | """ 110 | try: 111 | keys = self.keys() 112 | return self._db.get(random.choice(keys)) 113 | except: 114 | raise GetRandomCookieError 115 | 116 | def all(self): 117 | """ 118 | 获取所有账户, 以字典形式返回 119 | :return: 120 | """ 121 | try: 122 | for key in self._db.keys('{domain}:{name}:*'.format(domain=self.domain, name=self.name)): 123 | group = key.decode('utf-8').split(':') 124 | if len(group) == 3: 125 | username = group[2] 126 | yield { 127 | 'username': username, 128 | 'cookies': self.get(username) 129 | } 130 | except Exception as e: 131 | print(e.args) 132 | raise GetAllCookieError 133 | 134 | def count(self): 135 | """ 136 | 获取当前Cookies数目 137 | :return: 数目 138 | """ 139 | return len(self.keys()) 140 | 141 | 142 | 143 | class AccountRedisClient(RedisClient): 144 | def __init__(self, host=REDIS_HOST, port=REDIS_PORT, password=REDIS_PASSWORD, domain='account', name='default'): 145 | RedisClient.__init__(self, host, port, password) 146 | self.domain = domain 147 | self.name = name 148 | 149 | def set(self, key, value): 150 | try: 151 | return self._db.set(self._key(key), value) 152 | except: 153 | raise SetAccountError 154 | 155 | def get(self, key): 156 | try: 157 | return self._db.get(self._key(key)).decode('utf-8') 158 | except: 159 | raise GetAccountError 160 | 161 | def all(self): 162 | """ 163 | 获取所有账户, 以字典形式返回 164 | :return: 165 | """ 166 | try: 167 | for key in self._db.keys('{domain}:{name}:*'.format(domain=self.domain, name=self.name)): 168 | group = key.decode('utf-8').split(':') 169 | if len(group) == 3: 170 | username = group[2] 171 | yield { 172 | 'username': username, 173 | 'password': self.get(username) 174 | } 175 | except Exception as e: 176 | print(e.args) 177 | raise GetAllAccountError 178 | 179 | def delete(self, key): 180 | """ 181 | 通过用户名删除用户 182 | :param key: 183 | :return: 184 | """ 185 | try: 186 | return self._db.delete(self._key(key)) 187 | except: 188 | raise DeleteAccountError 189 | 190 | 191 | if __name__ == '__main__': 192 | """ 193 | conn = CookiesRedisClient() 194 | conn.set('name', 'Mike') 195 | conn.set('name2', 'Bob') 196 | conn.set('name3', 'Amy') 197 | print(conn.get('name')) 198 | conn.delete('name') 199 | print(conn.keys()) 200 | print(conn.random()) 201 | """ 202 | # 测试 203 | conn = AccountRedisClient(name='weibo') 204 | conn2 = AccountRedisClient(name='mweibo') 205 | 206 | 207 | accounts = conn.all() 208 | for account in accounts: 209 | conn2.set(account['username'], account['password']) 210 | -------------------------------------------------------------------------------- /cookiespool/error.py: -------------------------------------------------------------------------------- 1 | class CookiePoolError(Exception): 2 | def __str__(self): 3 | return repr('Cookie Pool Error') 4 | 5 | 6 | class SetCookieError(CookiePoolError): 7 | def __str__(self): 8 | return repr('Set Cookie Error') 9 | 10 | 11 | class GetCookieError(CookiePoolError): 12 | def __str__(self): 13 | return repr('Get Cookie Error') 14 | 15 | 16 | class DeleteCookieError(CookiePoolError): 17 | def __str__(self): 18 | return repr('Delete Cookie Error') 19 | 20 | 21 | class GetRandomCookieError(CookiePoolError): 22 | def __str__(self): 23 | return repr('Get Random Cookie Error') 24 | 25 | 26 | class GetAllCookieError(CookiePoolError): 27 | def __str__(self): 28 | return repr('Get All Cookie Error') 29 | 30 | 31 | class SetAccountError(CookiePoolError): 32 | def __str__(self): 33 | return repr('Set Account Error') 34 | 35 | 36 | class DeleteAccountError(CookiePoolError): 37 | def __str__(self): 38 | return repr('Delete Account Error') 39 | 40 | 41 | class GetAccountError(CookiePoolError): 42 | def __str__(self): 43 | return repr('Get Account Error') 44 | 45 | 46 | class GetAllAccountError(CookiePoolError): 47 | def __str__(self): 48 | return repr('Get All Account Error') 49 | -------------------------------------------------------------------------------- /cookiespool/generator.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | import requests 4 | import time 5 | from selenium import webdriver 6 | from selenium.common.exceptions import WebDriverException, TimeoutException 7 | from selenium.webdriver import DesiredCapabilities 8 | from selenium.webdriver.common.by import By 9 | from selenium.webdriver.support import expected_conditions as EC 10 | from selenium.webdriver.support.ui import WebDriverWait 11 | 12 | from cookiespool.config import * 13 | from cookiespool.db import CookiesRedisClient, AccountRedisClient 14 | from cookiespool.verify import Yundama 15 | 16 | 17 | class CookiesGenerator(object): 18 | def __init__(self, name='default', browser_type=DEFAULT_BROWSER): 19 | """ 20 | 父类, 初始化一些对象 21 | :param name: 名称 22 | :param browser: 浏览器, 若不使用浏览器则可设置为 None 23 | """ 24 | self.name = name 25 | self.cookies_db = CookiesRedisClient(name=self.name) 26 | self.account_db = AccountRedisClient(name=self.name) 27 | self.browser_type = browser_type 28 | 29 | def _init_browser(self, browser_type): 30 | """ 31 | 通过browser参数初始化全局浏览器供模拟登录使用 32 | :param browser: 浏览器 PhantomJS/ Chrome 33 | :return: 34 | """ 35 | if browser_type == 'PhantomJS': 36 | caps = DesiredCapabilities.PHANTOMJS 37 | caps[ 38 | "phantomjs.page.settings.userAgent"] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36' 39 | self.browser = webdriver.PhantomJS(desired_capabilities=caps) 40 | self.browser.set_window_size(1400, 500) 41 | elif browser_type == 'Chrome': 42 | self.browser = webdriver.Chrome() 43 | 44 | def new_cookies(self, username, password): 45 | raise NotImplementedError 46 | 47 | def set_cookies(self, account): 48 | """ 49 | 根据账户设置新的Cookies 50 | :param account: 51 | :return: 52 | """ 53 | results = self.new_cookies(account.get('username'), account.get('password')) 54 | if results: 55 | username, cookies = results 56 | print('Saving Cookies to Redis', username, cookies) 57 | self.cookies_db.set(username, cookies) 58 | 59 | 60 | def run(self): 61 | """ 62 | 运行, 得到所有账户, 然后顺次模拟登录 63 | :return: 64 | """ 65 | accounts = self.account_db.all() 66 | cookies = self.cookies_db.all() 67 | # Account 中对应的用户 68 | accounts = list(accounts) 69 | # Cookies中对应的用户 70 | valid_users = [cookie.get('username') for cookie in cookies] 71 | print('Getting', len(accounts), 'accounts from Redis') 72 | if len(accounts): 73 | self._init_browser(browser_type=self.browser_type) 74 | for account in accounts: 75 | if not account.get('username') in valid_users: 76 | print('Getting Cookies of ', self.name, account.get('username'), account.get('password')) 77 | self.set_cookies(account) 78 | print('Generator Run Finished') 79 | 80 | def close(self): 81 | try: 82 | print('Closing Browser') 83 | self.browser.close() 84 | del self.browser 85 | except TypeError: 86 | print('Browser not opened') 87 | 88 | 89 | class WeiboCookiesGenerator(CookiesGenerator): 90 | def __init__(self, name='weibo', browser_type=DEFAULT_BROWSER): 91 | """ 92 | 初始化操作, 微博需要声明一个云打码引用 93 | :param name: 名称微博 94 | :param browser: 使用的浏览器 95 | """ 96 | CookiesGenerator.__init__(self, name, browser_type) 97 | self.name = name 98 | self.ydm = Yundama(YUNDAMA_USERNAME, YUNDAMA_PASSWORD, YUNDAMA_APP_ID, YUNDAMA_APP_KEY) 99 | 100 | def _success(self, username): 101 | wait = WebDriverWait(self.browser, 5) 102 | success = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'me_portrait_w'))) 103 | if success: 104 | print('登录成功') 105 | self.browser.get('http://weibo.cn/') 106 | 107 | if "我的首页" in self.browser.title: 108 | print(self.browser.get_cookies()) 109 | cookies = {} 110 | for cookie in self.browser.get_cookies(): 111 | cookies[cookie["name"]] = cookie["value"] 112 | print(cookies) 113 | print('成功获取到Cookies') 114 | return (username, json.dumps(cookies)) 115 | 116 | def new_cookies(self, username, password): 117 | """ 118 | 生成Cookies 119 | :param username: 用户名 120 | :param password: 密码 121 | :return: 用户名和Cookies 122 | """ 123 | print('Generating Cookies of', username) 124 | self.browser.delete_all_cookies() 125 | self.browser.get('http://my.sina.com.cn/profile/unlogin') 126 | wait = WebDriverWait(self.browser, 20) 127 | 128 | try: 129 | login = wait.until(EC.visibility_of_element_located((By.ID, 'hd_login'))) 130 | login.click() 131 | user = wait.until( 132 | EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginformlist input[name="loginname"]'))) 133 | user.send_keys(username) 134 | psd = wait.until( 135 | EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginformlist input[name="password"]'))) 136 | psd.send_keys(password) 137 | submit = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.login_btn'))) 138 | submit.click() 139 | try: 140 | result = self._success(username) 141 | if result: 142 | return result 143 | except TimeoutException: 144 | print('出现验证码，开始识别验证码') 145 | yzm = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginform_yzm .yzm'))) 146 | url = yzm.get_attribute('src') 147 | cookies = self.browser.get_cookies() 148 | cookies_dict = {} 149 | for cookie in cookies: 150 | cookies_dict[cookie.get('name')] = cookie.get('value') 151 | response = requests.get(url, cookies=cookies_dict) 152 | result = self.ydm.identify(stream=response.content) 153 | if not result: 154 | print('验证码识别失败, 跳过识别') 155 | return 156 | door = wait.until( 157 | EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginform_yzm input[name="door"]'))) 158 | door.send_keys(result) 159 | submit.click() 160 | result = self._success(username) 161 | if result: 162 | return result 163 | except WebDriverException as e: 164 | print(e.args) 165 | 166 | 167 | class MWeiboCookiesGenerator(CookiesGenerator): 168 | def __init__(self, name='weibo', browser_type=DEFAULT_BROWSER): 169 | """ 170 | 初始化操作, 微博需要声明一个云打码引用 171 | :param name: 名称微博 172 | :param browser: 使用的浏览器 173 | """ 174 | CookiesGenerator.__init__(self, name, browser_type) 175 | self.name = name 176 | self.ydm = Yundama(YUNDAMA_USERNAME, YUNDAMA_PASSWORD, YUNDAMA_APP_ID, YUNDAMA_APP_KEY) 177 | 178 | def _success(self, username): 179 | wait = WebDriverWait(self.browser, 5) 180 | success = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'me_portrait_w'))) 181 | 182 | if success: 183 | print('登录成功') 184 | self.browser.get('http://m.weibo.cn/') 185 | 186 | if "微博" in self.browser.title: 187 | print(self.browser.get_cookies()) 188 | cookies = {} 189 | for cookie in self.browser.get_cookies(): 190 | cookies[cookie["name"]] = cookie["value"] 191 | print(cookies) 192 | print('成功获取到Cookies') 193 | return (username, json.dumps(cookies)) 194 | 195 | def new_cookies(self, username, password): 196 | """ 197 | 生成Cookies 198 | :param username: 用户名 199 | :param password: 密码 200 | :return: 用户名和Cookies 201 | """ 202 | print('Generating Cookies of', username) 203 | self.browser.delete_all_cookies() 204 | self.browser.get('http://my.sina.com.cn/profile/unlogin') 205 | wait = WebDriverWait(self.browser, 20) 206 | 207 | try: 208 | login = wait.until(EC.visibility_of_element_located((By.ID, 'hd_login'))) 209 | login.click() 210 | 211 | user = wait.until( 212 | EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginformlist input[name="loginname"]'))) 213 | user.send_keys(username) 214 | psd = wait.until( 215 | EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginformlist input[name="password"]'))) 216 | psd.send_keys(password) 217 | submit = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.login_btn'))) 218 | submit.click() 219 | try: 220 | result = self._success(username) 221 | if result: 222 | return result 223 | except TimeoutException: 224 | print('出现验证码，开始识别验证码') 225 | yzm = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginform_yzm .yzm'))) 226 | url = yzm.get_attribute('src') 227 | cookies = self.browser.get_cookies() 228 | 229 | cookies_dict = {} 230 | for cookie in cookies: 231 | cookies_dict[cookie.get('name')] = cookie.get('value') 232 | response = requests.get(url, cookies=cookies_dict) 233 | result = self.ydm.identify(stream=response.content) 234 | if not result: 235 | print('验证码识别失败, 跳过识别') 236 | return 237 | door = wait.until( 238 | EC.visibility_of_element_located((By.CSS_SELECTOR, '.loginform_yzm input[name="door"]'))) 239 | door.send_keys(result) 240 | submit.click() 241 | result = self._success(username) 242 | if result: 243 | return result 244 | except WebDriverException as e: 245 | pass 246 | 247 | 248 | if __name__ == '__main__': 249 | generator = WeiboCookiesGenerator() 250 | generator._init_browser('Chrome') 251 | generator.new_cookies('15197170054', 'gmwkms222') 252 | -------------------------------------------------------------------------------- /cookiespool/importer.py: -------------------------------------------------------------------------------- 1 | import requests 2 | 3 | from cookiespool.db import AccountRedisClient 4 | 5 | conn = AccountRedisClient(name='weibo') 6 | 7 | def set(account, sep='----'): 8 | username, password = account.split(sep) 9 | result = conn.set(username, password) 10 | print('账号', username, '密码', password) 11 | print('录入成功' if result else '录入失败') 12 | 13 | 14 | def scan(): 15 | print('请输入账号密码组, 输入exit退出读入') 16 | while True: 17 | account = input() 18 | if account == 'exit': 19 | break 20 | set(account) 21 | 22 | 23 | if __name__ == '__main__': 24 | scan() -------------------------------------------------------------------------------- /cookiespool/scheduler.py: -------------------------------------------------------------------------------- 1 | import time 2 | from multiprocessing import Process 3 | 4 | from cookiespool.api import app 5 | from cookiespool.config import * 6 | from cookiespool.generator import * 7 | from cookiespool.tester import * 8 | 9 | class Scheduler(object): 10 | @staticmethod 11 | def valid_cookie(cycle=CYCLE): 12 | while True: 13 | print('Checking Cookies') 14 | try: 15 | for name, cls in TESTER_MAP.items(): 16 | tester = eval(cls + '(name="' + name + '")') 17 | tester.run() 18 | print('Tester Finished') 19 | del tester 20 | time.sleep(cycle) 21 | except Exception as e: 22 | print(e.args) 23 | 24 | @staticmethod 25 | def generate_cookie(cycle=CYCLE): 26 | while True: 27 | print('Generating Cookies') 28 | try: 29 | for name, cls in GENERATOR_MAP.items(): 30 | generator = eval(cls + '(name="' + name + '")') 31 | generator.run() 32 | print('Generator Finished') 33 | generator.close() 34 | print('Deleted Generator') 35 | time.sleep(cycle) 36 | except Exception as e: 37 | print(e.args) 38 | 39 | @staticmethod 40 | def api(): 41 | app.run(host=API_HOST, port=API_PORT) 42 | 43 | def run(self): 44 | if GENERATOR_PROCESS: 45 | generate_process = Process(target=Scheduler.generate_cookie) 46 | generate_process.start() 47 | 48 | if VALID_PROCESS: 49 | valid_process = Process(target=Scheduler.valid_cookie) 50 | valid_process.start() 51 | 52 | if API_PROCESS: 53 | api_process = Process(target=Scheduler.api) 54 | api_process.start() 55 | 56 | -------------------------------------------------------------------------------- /cookiespool/tester.py: -------------------------------------------------------------------------------- 1 | import json 2 | from bs4 import BeautifulSoup 3 | import requests 4 | from requests.exceptions import ConnectionError 5 | from cookiespool.db import * 6 | from cookiespool.generator import WeiboCookiesGenerator 7 | 8 | 9 | class ValidTester(object): 10 | def __init__(self, name='default'): 11 | self.name = name 12 | self.cookies_db = CookiesRedisClient(name=self.name) 13 | self.account_db = AccountRedisClient(name=self.name) 14 | 15 | def test(self, account, cookies): 16 | raise NotImplementedError 17 | 18 | def run(self): 19 | accounts = self.cookies_db.all() 20 | for account in accounts: 21 | username = account.get('username') 22 | cookies = self.cookies_db.get(username) 23 | self.test(account, cookies) 24 | 25 | 26 | class WeiboValidTester(ValidTester): 27 | def __init__(self, name='weibo'): 28 | ValidTester.__init__(self, name) 29 | 30 | def test(self, account, cookies): 31 | print('Testing Account', account.get('username')) 32 | try: 33 | cookies = json.loads(cookies) 34 | except TypeError: 35 | # Cookie 格式不正确 36 | print('Invalid Cookies Value', account.get('username')) 37 | self.cookies_db.delete(account.get('username')) 38 | print('Deleted User', account.get('username')) 39 | return None 40 | try: 41 | response = requests.get('http://weibo.cn', cookies=cookies) 42 | if response.status_code == 200: 43 | html = response.text 44 | soup = BeautifulSoup(html, 'lxml') 45 | title = soup.title.string 46 | if title == '我的首页': 47 | print('Valid Cookies', account.get('username')) 48 | else: 49 | print('Title is', title) 50 | # Cookie已失效 51 | print('Invalid Cookies', account.get('username')) 52 | self.cookies_db.delete(account.get('username')) 53 | print('Deleted User', account.get('username')) 54 | except ConnectionError as e: 55 | print('Error', e.args) 56 | print('Invalid Cookies', account.get('username')) 57 | 58 | 59 | class MWeiboValidTester(ValidTester): 60 | def __init__(self, name='weibo'): 61 | ValidTester.__init__(self, name) 62 | 63 | def test(self, account, cookies): 64 | print('Testing Account', account.get('username')) 65 | try: 66 | cookies = json.loads(cookies) 67 | except TypeError: 68 | # Cookie 格式不正确 69 | print('Invalid Cookies Value', account.get('username')) 70 | self.cookies_db.delete(account.get('username')) 71 | print('Deleted User', account.get('username')) 72 | return None 73 | try: 74 | test_url = 'http://m.weibo.cn/api/container/getIndex?uid=1804544030&type=uid&page=1&containerid=1076031804544030' 75 | response = requests.get(test_url, cookies=cookies, timeout=5, allow_redirects=False) 76 | if response.status_code == 200: 77 | print('Valid Cookies', account.get('username')) 78 | else: 79 | print(response.status_code, response.headers) 80 | print('Invalid Cookies', account.get('username')) 81 | self.cookies_db.delete(account.get('username')) 82 | print('Deleted User', account.get('username')) 83 | except ConnectionError as e: 84 | print('Error', e.args) 85 | print('Invalid Cookies', account.get('username')) 86 | 87 | if __name__ == '__main__': 88 | tester = WeiboValidTester() 89 | tester.run() 90 | -------------------------------------------------------------------------------- /cookiespool/verify.py: -------------------------------------------------------------------------------- 1 | import time 2 | 3 | import requests 4 | from requests.exceptions import ConnectionError 5 | 6 | from cookiespool.config import * 7 | 8 | 9 | class Yundama(): 10 | def __init__(self, username, password, app_id, app_key, api_url=YUNDAMA_API_URL): 11 | self.username = username 12 | self.password = password 13 | self.app_id = str(app_id) if not isinstance(app_id, str) else app_id 14 | self.app_key = app_key 15 | self.api_url = api_url 16 | 17 | def login(self): 18 | """ 19 | 登录云打码账户 20 | :return: 21 | """ 22 | try: 23 | data = {'method': 'login', 'username': self.username, 'password': self.password, 'appid': self.app_id, 24 | 'appkey': self.app_key} 25 | response = requests.post(self.api_url, data=data) 26 | if response.status_code == 200: 27 | result = response.json() 28 | print(result) 29 | if 'ret' in result.keys() and result.get('ret') < 0: 30 | return self.error(result.get('ret')) 31 | else: 32 | return result 33 | return None 34 | except ConnectionError: 35 | return None 36 | 37 | def upload(self, files, timeout, code_type): 38 | """ 39 | 上传验证码得到识别结果 40 | :param files: 41 | :param timeout: 42 | :param code_type: 43 | :return: 44 | """ 45 | try: 46 | data = {'method': 'upload', 'username': self.username, 'password': self.password, 'appid': self.app_id, 47 | 'appkey': self.app_key, 'codetype': str(code_type), 'timeout': str(timeout)} 48 | response = requests.post(self.api_url, data=data, files=files) 49 | if response.status_code == 200: 50 | return response.json() 51 | return None 52 | except ConnectionError: 53 | return None 54 | 55 | def retry(self, cid, try_count=1): 56 | """ 57 | 临时识别不出, 传入cid重试 58 | :param cid: 验证码ID 59 | :param try_count: 重试次数 60 | :return: 验证码结果 61 | """ 62 | if try_count >= YUNDAMA_MAX_RETRY: 63 | return None 64 | print('Retrying: ', cid, 'Count: ', try_count) 65 | time.sleep(2) 66 | try: 67 | data = {'method': 'result', 'cid': cid} 68 | print(data) 69 | response = requests.post(self.api_url, data=data) 70 | if response.status_code == 200: 71 | result = response.json() 72 | print(result) 73 | if 'ret' in result.keys() and result.get('ret') < 0: 74 | print(self.error(result.get('ret'))) 75 | if result.get('ret') == 0 and 'text' in result.keys(): 76 | return result.get('text') 77 | else: 78 | return self.retry(cid, try_count + 1) 79 | return None 80 | except ConnectionError: 81 | return None 82 | 83 | def identify(self, file=None, stream=None, timeout=60, code_type=5000): 84 | """ 85 | 主函数 86 | :param file: 文件名 87 | :param stream: 文件流, 优先于文件名 88 | :param timeout: 超时时间 89 | :param code_type: 验证码类型 90 | :return: 识别结果 91 | """ 92 | if stream: 93 | files = {'file': stream} 94 | elif file: 95 | files = {'file': open(file, 'rb')} 96 | else: 97 | return None 98 | result = self.upload(files, timeout, code_type) 99 | if 'ret' in result.keys() and result.get('ret') < 0: 100 | print(self.error(result.get('ret'))) 101 | if result.get('text'): 102 | print('验证码识别成功', result.get('text')) 103 | return result.get('text') 104 | else: 105 | return self.retry(result.get('cid')) 106 | 107 | def error(self, code): 108 | """ 109 | 报错原因 110 | :param code: 错误码 111 | :return: 错误原因 112 | """ 113 | map = { 114 | -1001: '密码错误', 115 | -1002: '软件ID/密钥有误', 116 | -1003: '用户被封', 117 | -1004: 'IP被封', 118 | -1005: '软件被封', 119 | -1006: '登录IP与绑定的区域不匹配', 120 | -1007: '账号余额为零', 121 | -2001: '验证码类型有误', 122 | -2002: '验证码图片太大', 123 | -2003: '验证码图片损坏', 124 | -2004: '上传验证码图片失败', 125 | -3001: '验证码ID不存在 ', 126 | -3002: '验证码还在识别', 127 | -3003: '验证码识别超时', 128 | -3004: '验证码看不清', 129 | -3005: '验证码报错失败', 130 | -4001: '充值卡号不正确或已使用', 131 | -5001: '注册用户失败' 132 | } 133 | return '云打码' + map.get(code) 134 | 135 | 136 | if __name__ == '__main__': 137 | ydm = Yundama(YUNDAMA_USERNAME, YUNDAMA_PASSWORD, YUNDAMA_APP_ID, YUNDAMA_APP_KEY) 138 | result = ydm.identify(file='getimage.jpg') 139 | print(result) 140 | -------------------------------------------------------------------------------- /importer.py: -------------------------------------------------------------------------------- 1 | from cookiespool.importer import scan 2 | 3 | if __name__ == '__main__': 4 | scan() -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests>=2.13.0 2 | selenium>=3.4.0 3 | redis>=2.10.5 4 | Flask>=0.12.1 -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- 1 | from cookiespool.scheduler import Scheduler 2 | 3 | def main(): 4 | s = Scheduler() 5 | s.run() 6 | 7 | if __name__ == '__main__': 8 | main() -------------------------------------------------------------------------------- /tests/login.py: -------------------------------------------------------------------------------- 1 | import requests 2 | from bs4 import BeautifulSoup 3 | 4 | cookies = { 5 | 'SSOLoginState': '1493555070', 6 | 'SCF': 'ArrMd41qtHmW87eTIsI-sT1IjDG8oncB9A0HbSmyDw1FwO5sbI_j6_ZellQQ07ZjTXTIBrM3Y_tpKym39f1tYWs.', 7 | 'SUB': '_2A250AacuDeRhGeRG6FIX9ybIzDiIHXVXDclmrDV6PUJbktBeLVfCkW18FWjD8r3ddYXy2abmqSauclujaw..', 8 | '_T_WM': '5e9c698a350ddeba1c5d77e1958af21b', 'ALF': '1496147067', 'SUHB': '0G0vWR88D2VokZ', 9 | 'SUBP': '0033WrSXqPxfM725Ws9jqgMF55529P9D9W5APD86CuQDllBusA-6OZaq5JpX5o2p5NHD95QE1he7SoMRShMXWs4Dqcjci--fi-zXiK.Xi--fi-iWiKnci--ciKn4iKy2i--Xi-zRi-2Ri--4iKL2iK.4i--Ri-2NiKnf' 10 | } 11 | 12 | response = requests.get('http://weibo.cn', cookies=cookies) 13 | if response.status_code == 200: 14 | html = response.text 15 | soup = BeautifulSoup(html, 'lxml') 16 | title = soup.title.string 17 | print(title) 18 | --------------------------------------------------------------------------------