├── .gitignore
├── README.md
├── README.ru.md
├── darklibria.png
├── darklibria.py
├── lostfilm.png
└── lostfilm.py


/.gitignore:
--------------------------------------------------------------------------------
 1 | *.json
 2 | *.log
 3 | *.bat
 4 | 
 5 | helpers.py
 6 | novaprinter.py
 7 | socks.py
 8 | 
 9 | __pycache__
10 | .vscode
11 | install
12 | proxycheck
13 | venv


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # qBittorrent search plugins
 2 | 
 3 | [Readme in russian](https://github.com/bugsbringer/qbit-plugins/blob/master/README.ru.md)
 4 | 
 5 | Plugins
 6 | -------
 7 | 
 8 | * LostFilm.TV
 9 | * dark-libria.it
10 | 
11 | Installation
12 | ------------
13 | 
14 | 1.For private torrent-trackers(LostFilm):
15 | 
16 | * Save *.py file in any convenient directory
17 | * Open this file using **notepad** or any other **text editor**
18 | * Replace text in rows **EMAIL** and **PASSWORD** with **your login data**<br>
19 | Example:
20 | 
21 |         EMAIL = "example123@gmail.com"
22 |         PASSWORD = "qwerty345"
23 | 
24 | 2.Follow the **official tutorial**: [Install search plugins](https://github.com/qbittorrent/search-plugins/wiki/Install-search-plugins)
25 | 
26 | LostFilm plugin features
27 | ------------------------
28 | 
29 | * Information about seeders and leechers in search results. (Reduces search speed)<br>
30 |         You can disable this functionality in lostfilm.py file:
31 | 
32 |         ENABLE_PEERS_INFO = False
33 | 
34 | * Additional search queries:<br>
35 | *just enter it in search field*
36 |   * Favorites serials:
37 | 
38 |         @fav
39 | 
40 |     * New episodes in the last 7 days:
41 | 
42 |           @new
43 | 
44 |       * Among favorites:
45 | 
46 |             @new:fav
47 | 
48 | * <strong><em>New in version 0.21<br>
49 | Now you can change site root url, to solve the blocking issies: <br>
50 | </em></strong>
51 | 
52 |       SITE_URL = "https://www.lostfilm.tv"
53 |       
54 | Errors
55 | ------
56 | 
57 | ### Captcha requested
58 | 
59 | * You need to **logout**:
60 |   * Go to <https://www.lostfilm.tv/my_logout>
61 |     * Then aprove logout
62 | * **Login** again:
63 |   * Go to <https://www.lostfilm.tv/login>
64 |   * Enter your login data and captcha
65 |   * And finally Log in
66 | 
67 | ### Connection failed
68 | 
69 | * Could not connect to server
70 | 
71 | ### Incorrect login data
72 | 
73 | * Most likely you incorrectly filled in the authorization data
74 | 


--------------------------------------------------------------------------------
/README.ru.md:
--------------------------------------------------------------------------------
 1 | # qBittorrent плагины
 2 | 
 3 | Плагины
 4 | -------
 5 | 
 6 | * LostFilm.TV
 7 | * dark-libria.it
 8 | 
 9 | Установка
10 | ---------
11 | 
12 | 1.Для приватных торрент-трекеров(LostFilm):
13 | 
14 | * Сохраните файл *.py в любую удобную директорию
15 | * Откройте файл при помощи **блокнота** или другого **текстового редактора**
16 | * Замените текст в строках **EMAIL** и **PASSWORD**  **своими данными для входа на сайт**<br>
17 | Пример:
18 | 
19 |         EMAIL = "example123@gmail.com"
20 |         PASSWORD = "qwerty345"
21 | 
22 | 2.Следуйте **официальному руководству**: [Install search plugins](https://github.com/qbittorrent/search-plugins/wiki/Install-search-plugins)
23 | 
24 | Особенности плагина LostFilm
25 | ----------------------------
26 | 
27 | * Информация о сидах и личах в результатах поиска. (Замедляет поиск)<br>
28 |         Вы можете отключить эту функцию в lostfilm.py файле:
29 | 
30 |         ENABLE_PEERS_INFO = False
31 | 
32 | * Дополнительные поисковые запросы:<br>
33 | *Просто введите в поле поиска*
34 |   * Избранные сериалы:
35 | 
36 |             @fav
37 | 
38 |     * Новые эпизоды:
39 |       * Среди всех или среди избранных(за последние 7 дней):
40 | 
41 |                 @new
42 |                 @new:fav
43 | 
44 | <strong><em>New in version 0.21<br>
45 | * Теперь вы можете изменить корневой адрес сайта, чтобы решить проблемы с блокировкой: <br>
46 | </em></strong>
47 | 
48 |       SITE_URL = "https://www.lostfilm.tv"
49 |       
50 | Ошибки
51 | ------
52 | 
53 | ### Captcha requested
54 | 
55 | * Вам нужно **выйти** из своего аккаунта на сайте:
56 |   * Перейдите по ссылке <https://www.lostfilm.tv/my_logout>
57 |   * Подтвердите выход
58 | * **Войти** опять:
59 |   * Перейдите по ссылке <https://www.lostfilm.tv/login>
60 |   * Введи ваши данные для входа и капчу
61 |   * Войдите :)
62 | 
63 | ### Connection failed
64 | 
65 | * Не удалось подключиться к серверу
66 | 
67 | ### Incorrect login data
68 | 
69 | * Похоже что поля **EMAIL** и **PASSWORD** заполнены не верно
70 | 


--------------------------------------------------------------------------------
/darklibria.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bugsbringer/qbit-plugins/0ee6392ded7953cb50ac913a29dd839b1e53bdda/darklibria.png


--------------------------------------------------------------------------------
/darklibria.py:
--------------------------------------------------------------------------------
  1 | #VERSION: 0.13
  2 | #AUTHORS: Bugsbringer (dastins193@gmail.com)
  3 | 
  4 | 
  5 | SITE_URL = 'https://darklibria.it/'
  6 | 
  7 | 
  8 | import logging
  9 | import os
 10 | from concurrent.futures import ThreadPoolExecutor
 11 | from datetime import datetime
 12 | from html.parser import HTMLParser
 13 | from math import ceil
 14 | from re import compile as re_compile
 15 | from time import mktime
 16 | from urllib import parse
 17 | 
 18 | from helpers import retrieve_url
 19 | from novaprinter import prettyPrinter
 20 | 
 21 | LOG_FORMAT = '[%(asctime)s] %(levelname)s:%(name)s:%(funcName)s - %(message)s'
 22 | LOG_DT_FORMAT = '%d-%b-%y %H:%M:%S'
 23 | 
 24 | 
 25 | class darklibria:
 26 |     url = SITE_URL
 27 |     name = 'dark-libria'
 28 |     supported_categories = {'all': '0'}
 29 | 
 30 |     units_dict = {"Тб": "TB", "Гб": "GB", "Мб": "MB", "Кб": "KB", "б": "B"}
 31 |     page_search_url_pattern = SITE_URL + 'search?page={page}&find={what}'
 32 |     dt_regex = re_compile('\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}')
 33 | 
 34 |     def __init__(self, output=True):
 35 |         self.output = output
 36 | 
 37 |     def search(self, what, cat='all'):
 38 |         self.torrents_count = 0
 39 |         what = parse.quote(parse.unquote(what))
 40 |         logger.info(parse.unquote(what))
 41 |         self.set_search_data(self.handle_page(what, 1))
 42 |         with ThreadPoolExecutor() as executor:
 43 |             for page in range(2, self.pages_count + 1):
 44 |                 executor.submit(self.handle_page, what, page)
 45 |         logger.info('%s torrents', self.torrents_count)
 46 | 
 47 |     def handle_page(self, what, page):
 48 |         url = self.page_search_url_pattern.format(page=page, what=what)
 49 |         data = self.request_get(url)
 50 |         if not data:
 51 |             return
 52 |         parser = Parser(data)
 53 |         serials = parser.find_all('tbody', {'style': 'vertical-align: center'})
 54 |         with ThreadPoolExecutor() as executor:
 55 |             for serial in serials:
 56 |                 executor.submit(self.handle_serial, serial.a['href'])
 57 |         return parser
 58 | 
 59 |     def handle_serial(self, url):
 60 |         data = self.request_get(url)
 61 |         if not data:
 62 |             return
 63 |         parser = Parser(data)
 64 |         name = parser.find(attrs={'id': 'russian_name'}).text
 65 |         for torrent_row in parser.find_all('tr', {'class': 'torrent'}):
 66 |             self.handle_torrent_row(torrent_row, name, url)
 67 | 
 68 |     def handle_torrent_row(self, torrent_row, name, url):
 69 |         type, quality, size_data, date_time, download, seeds, leech, *_ = torrent_row.children
 70 |         self.pretty_printer({
 71 |             'link': self.get_link(download),
 72 |             'name': self.get_name(name, quality, type, date_time),
 73 |             'size': self.get_size(size_data),
 74 |             'seeds': int(seeds.text),
 75 |             'leech': int(leech.text),
 76 |             'engine_url': self.url,
 77 |             'desc_link': url
 78 |         })
 79 |         self.torrents_count += 1
 80 | 
 81 |     def get_link(self, download):
 82 |         return download.find(attrs={'title': 'Magnet-ссылка'})['href'] \
 83 |             or download.find(attrs={'title': 'Скачать торрент'})['href']
 84 |             
 85 |     def get_name(self, name, quality, type, date_time):
 86 |         return '[{}] {} [{}] {}'.format(
 87 |             self.get_date(date_time),
 88 |             name,
 89 |             type.text,
 90 |             quality.text
 91 |         )
 92 | 
 93 |     def get_date(self, date_time):
 94 |         utc_dt_string = self.dt_regex.search(date_time.text).group()
 95 |         utc = datetime.strptime(utc_dt_string, '%Y-%m-%d %H:%M:%S')
 96 |         return str(utc2local(utc))
 97 | 
 98 |     def get_size(self, size_data):
 99 |         size, unit = size_data.text.split()
100 |         return size + ' ' + self.units_dict[unit]
101 | 
102 |     def request_get(self, url):
103 |         try:
104 |             return retrieve_url(url)
105 |         except Exception as exp:
106 |             logger.error(exp)
107 |             self.pretty_printer({
108 |                 'link': 'Error',
109 |                 'name': 'Connection failed',
110 |                 'size': "0",
111 |                 'seeds': -1,
112 |                 'leech': -1,
113 |                 'engine_url': self.url,
114 |                 'desc_link': self.url
115 |             })
116 | 
117 |     def pretty_printer(self, dictionary):
118 |         logger.debug(str(dictionary))
119 |         if self.output:
120 |             prettyPrinter(dictionary)
121 | 
122 |     def set_search_data(self, parser):
123 |         results = parser.find('span', {'class': 'text text-light mt-0'})
124 |         if results:
125 |             parts = results.text.split()
126 |             items_count = int(parts[4])
127 |             items_on_page = int(parts[2].split('-')[1])
128 |             self.pages_count = ceil(items_count / items_on_page)
129 | 
130 |             logger.info('%s animes', items_count)
131 |         else:
132 |             self.pages_count = 0
133 | 
134 |         logger.info('%s pages', self.pages_count)
135 | 
136 | 
137 | class Tag:
138 |     def __init__(self, tag=None, attrs=(), is_self_closing=None):
139 |         self.type = tag
140 |         self.is_self_closing = is_self_closing
141 |         self._attrs = tuple(attrs)
142 |         self._content = tuple()
143 | 
144 |     @property
145 |     def attrs(self):
146 |         """returns dict of Tag's attrs"""
147 |         return dict(self._attrs)
148 | 
149 |     @property
150 |     def text(self):
151 |         """returns str of all contained text"""
152 |         return ''.join(c if isinstance(c, str) else c.text for c in self._content)
153 | 
154 |     def _add_content(self, obj):
155 |         if isinstance(obj, (Tag, str)):
156 |             self._content += (obj,)
157 |         else:
158 |             raise TypeError('Argument must be str or %s, not %s' %
159 |                             (self.__class__, obj.__class__))
160 | 
161 |     def find(self, tag=None, attrs=None):
162 |         """returns Tag or None"""
163 |         return next(self._find_all(tag, attrs), None)
164 | 
165 |     def find_all(self, tag=None, attrs=None):
166 |         """returns list"""
167 |         return list(self._find_all(tag, attrs))
168 | 
169 |     def _find_all(self, tag_type=None, attrs=None):
170 |         """returns generator"""
171 |         if not (isinstance(tag_type, (str, Tag)) or tag_type is None):
172 |             raise TypeError(
173 |                 'tag_type argument must be str or Tag, not %s' % (tag_type.__class__))
174 | 
175 |         if not (isinstance(attrs, dict) or attrs is None):
176 |             raise TypeError('attrs argument must be dict, not %s' %
177 |                             (self.__class__))
178 | 
179 |         # get tags-descendants generator
180 |         results = self.descendants
181 | 
182 |         # filter by Tag.type
183 |         if tag_type:
184 |             if isinstance(tag_type, Tag):
185 |                 tag_type, attrs = tag_type.type, (
186 |                     attrs if attrs else tag_type.attrs)
187 | 
188 |             results = filter(lambda t: t.type == tag_type, results)
189 | 
190 |         # filter by Tag.attrs
191 |         if attrs:
192 |             # remove Tags without attrs
193 |             results = filter(lambda t: t._attrs, results)
194 | 
195 |             def filter_func(tag):
196 |                 for key in attrs.keys():
197 |                     if attrs[key] not in tag.attrs.get(key, ()):
198 |                         return False
199 |                 return True
200 | 
201 |             # filter by attrs
202 |             results = filter(filter_func, results)
203 | 
204 |         yield from results
205 | 
206 |     @property
207 |     def children(self):
208 |         """returns generator of tags-children"""
209 |         return (obj for obj in self._content if isinstance(obj, Tag))
210 | 
211 |     @property
212 |     def descendants(self):
213 |         """returns generator of tags-descendants"""
214 |         for child_tag in self.children:
215 |             yield child_tag
216 |             yield from child_tag.descendants
217 | 
218 |     def __getitem__(self, key):
219 |         return self.attrs[key]
220 | 
221 |     def __getattr__(self, attr):
222 |         if not attr.startswith("__"):
223 |             return self.find(tag=attr)
224 | 
225 |     def __repr__(self):
226 |         attrs = ' '.join(str(k) if v is None else '{}="{}"'.format(k, v)
227 |                          for k, v in self._attrs)
228 |         starttag = ' '.join((self.type, attrs)) if attrs else self.type
229 | 
230 |         if self.is_self_closing:
231 |             return '<{starttag}>\n'.format(starttag=starttag)
232 |         else:
233 |             nested = '\n' * bool(next(self.children, None)) + \
234 |                 ''.join(map(str, self._content))
235 |             return '<{}>{}</{}>\n'.format(starttag, nested, self.type)
236 | 
237 | 
238 | class Parser(HTMLParser):
239 |     def __init__(self, html_code, *args, **kwargs):
240 |         super().__init__(*args, **kwargs)
241 | 
242 |         self._root = Tag('_root')
243 |         self._path = [self._root]
244 | 
245 |         self.feed(''.join(map(str.strip, html_code.splitlines())))
246 |         self.handle_endtag(self._root.type)
247 |         self.close()
248 | 
249 |         self.find = self._root.find
250 |         self.find_all = self._root.find_all
251 | 
252 |     @property
253 |     def attrs(self):
254 |         return self._root.attrs
255 | 
256 |     @property
257 |     def text(self):
258 |         return self._root.text
259 | 
260 |     def handle_starttag(self, tag, attrs):
261 |         self._path.append(Tag(tag=tag, attrs=attrs))
262 | 
263 |     def handle_endtag(self, tag_type):
264 |         for pos, tag in tuple(enumerate(self._path))[::-1]:
265 |             if isinstance(tag, Tag) and tag.type == tag_type and tag.is_self_closing is None:
266 |                 tag.is_self_closing = False
267 | 
268 |                 for obj in self._path[pos + 1:]:
269 |                     if isinstance(obj, Tag) and obj.is_self_closing is None:
270 |                         obj.is_self_closing = True
271 | 
272 |                     tag._add_content(obj)
273 | 
274 |                 self._path = self._path[:pos + 1]
275 | 
276 |                 break
277 | 
278 |     def handle_startendtag(self, tag, attrs):
279 |         self._path.append(Tag(tag=tag, attrs=attrs, is_self_closing=True))
280 | 
281 |     def handle_decl(self, decl):
282 |         self._path.append(Tag(tag='!'+decl, is_self_closing=True))
283 | 
284 |     def handle_data(self, text):
285 |         self._path.append(text)
286 | 
287 |     def __getitem__(self, key):
288 |         return self.attrs[key]
289 | 
290 |     def __getattr__(self, attr):
291 |         if not attr.startswith("__"):
292 |             return getattr(self._root, attr)
293 | 
294 |     def __repr__(self):
295 |         return ''.join(str(c) for c in self._root._content)
296 | 
297 | 
298 | def utc2local(utc):
299 |     epoch = mktime(utc.timetuple())
300 |     offset = datetime.fromtimestamp(epoch) - datetime.utcfromtimestamp(epoch)
301 |     return utc + offset
302 | 
303 | 
304 | is_main = __name__ == '__main__'
305 | STORAGE = os.path.abspath(os.path.dirname(__file__))
306 | log_config = {
307 |     'level': logging.INFO if is_main else logging.WARNING,
308 |     'filename': None if is_main else os.path.join(STORAGE, 'darklibria.log'),
309 |     'format': LOG_FORMAT,
310 |     'datefmt': LOG_DT_FORMAT
311 | }
312 | logging.basicConfig(**log_config)
313 | logger = logging.getLogger('darklibria')
314 | 
315 | if is_main:
316 |     import sys
317 |     darklibria(output=False).search(sys.argv[-1])
318 | 


--------------------------------------------------------------------------------
/lostfilm.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bugsbringer/qbit-plugins/0ee6392ded7953cb50ac913a29dd839b1e53bdda/lostfilm.png


--------------------------------------------------------------------------------
/lostfilm.py:
--------------------------------------------------------------------------------
  1 | #VERSION: 0.22
  2 | #AUTHORS: Bugsbringer (dastins193@gmail.com)
  3 | 
  4 | 
  5 | EMAIL = "YOUR_EMAIL"
  6 | PASSWORD = "YOUR_PASSWORD"
  7 | ENABLE_PEERS_INFO = True
  8 | SITE_URL = "https://www.lostfilm.tv"
  9 | 
 10 | proxy = {
 11 |     'enable': False,
 12 | 
 13 |     'proxy_urls': {
 14 |         'http': 'ip:port',
 15 |         'https': 'ip:port'
 16 |     },
 17 | 
 18 |     'auth': False,
 19 |     'username': '',
 20 |     'password': ''
 21 | }
 22 | 
 23 | import concurrent.futures
 24 | import hashlib
 25 | import json
 26 | import logging
 27 | import os
 28 | import re
 29 | from collections import OrderedDict
 30 | from datetime import datetime
 31 | from html.parser import HTMLParser
 32 | from http.cookiejar import CookieJar
 33 | from io import BytesIO
 34 | from time import time
 35 | from urllib import parse, request
 36 | 
 37 | from novaprinter import prettyPrinter
 38 | 
 39 | STORAGE = os.path.abspath(os.path.dirname(__file__))
 40 | is_main = __name__ == '__main__'
 41 | 
 42 | # logging
 43 | log_config = {
 44 |     'level': 'DEBUG' if is_main else 'ERROR',
 45 |     'format': '[%(asctime)s] %(levelname)s:%(name)s:%(funcName)s - %(message)s',
 46 |     'datefmt': '%d-%b-%y %H:%M:%S'
 47 | }
 48 | 
 49 | if not is_main:
 50 |     log_config.update({'filename': os.path.join(STORAGE, 'lostfilm.log')})
 51 | 
 52 | logging.basicConfig(**log_config)
 53 | logger = logging.getLogger('lostfilm')
 54 | logger.setLevel(logging.WARNING)
 55 | 
 56 | 
 57 | class lostfilm:
 58 |     url = SITE_URL
 59 |     name = 'LostFilm'
 60 |     supported_categories = {'all': '0'}
 61 | 
 62 |     search_url_pattern = SITE_URL + '/search/?q={what}'
 63 |     serial_url_pattern = SITE_URL + '{href}/seasons'
 64 |     download_url_pattern = SITE_URL + '/v_search.php?a={code}'
 65 |     season_url_pattern = SITE_URL + '{href}/season_{season}'
 66 |     episode_url_pattern = SITE_URL + '{href}/season_{season}/episode_{episode}/'
 67 |     additional_url_pattern = SITE_URL + '{href}/additional/episode_{episode}/'
 68 |     new_url_pattern = SITE_URL + '/new/page_{page}/type_{type}'
 69 | 
 70 |     additional_season = 999
 71 |     all_episodes = 999
 72 |     peer_id = '-PC0001-' + str(time()).replace('.', '')[-12:]
 73 | 
 74 |     datetime_format = '%d.%m.%Y'
 75 |     units_dict = {"ТБ": "TB", "ГБ": "GB", "МБ": "MB", "КБ": "KB", "Б": "B"}
 76 | 
 77 |     def __init__(self, output=True):
 78 |         self.output = output
 79 |         self.session = Session()
 80 |         
 81 |     def search(self, what, cat='all'):
 82 |         logger.info(what)
 83 | 
 84 |         self.torrents_count = 0
 85 |         self.prevs = set()
 86 |         self.old_seasons = dict()
 87 | 
 88 |         if not self.session.is_actual: 
 89 |             self.pretty_printer({
 90 |                 'link': 'Error',
 91 |                 'name': self.session.error,
 92 |                 'size': "0",
 93 |                 'seeds': -1,
 94 |                 'leech': -1,
 95 |                 'engine_url': self.url,
 96 |                 'desc_link': self.url
 97 |             })
 98 | 
 99 |             return False
100 | 
101 |         if parse.unquote(what).startswith('@'): 
102 |             params = parse.unquote(what)[1:].split(':')
103 |             
104 |             if params:
105 |                 if params[0] == 'fav':
106 |                     self.get_fav()
107 | 
108 |                 elif params[0] == 'new':
109 |                     if len(params) == 1:
110 |                         self.get_new()
111 | 
112 |                     elif len(params) == 2 and params[1] == 'fav':
113 |                         self.get_new(fav=True)
114 |         else:
115 |             try:
116 |                 url = self.search_url_pattern.format(what=request.quote(what))
117 |                 search_result = self.session.request(url)
118 |             except Exception as exp:
119 |                 logger.error(exp)
120 | 
121 |             else:
122 |                 serials_tags = Parser(search_result).find_all('div', {'class': 'row-search'})
123 |                 if serials_tags:
124 |                     with concurrent.futures.ThreadPoolExecutor() as executor:
125 |                         for serial_href in (serial.a['href'] for serial in serials_tags):
126 |                             logger.debug(serial_href)
127 | 
128 |                             executor.submit(self.get_episodes, serial_href)
129 |         
130 |         logger.info('%s torrents', self.torrents_count)
131 | 
132 |     def get_new(self, fav=False, days=7):
133 |         type = 99 if fav else 0
134 |         today = datetime.now().date()
135 |         self.dates = {}
136 | 
137 |         with concurrent.futures.ThreadPoolExecutor() as executor:
138 |             page_number = 1
139 |             while True:
140 |                 url = self.new_url_pattern.format(page=page_number, type=type)
141 |                 page = self.session.request(url)
142 | 
143 |                 rows = Parser(page).find_all('div', {'class': 'row'})
144 | 
145 |                 if not rows:
146 |                     break
147 | 
148 |                 for row in rows:
149 |                     
150 |                     release_date_str = row.find_all('div', {'class': 'alpha'})[1].text
151 |                     release_date_str = re.search(r'\d{2}.\d{2}.\d{4}', release_date_str)[0]
152 |                     release_date = datetime.strptime(release_date_str, self.datetime_format).date()
153 | 
154 |                     date_delta = today - release_date
155 | 
156 |                     if date_delta.days > days:
157 |                         return
158 | 
159 |                     href = '/'.join(row.a['href'].split('/')[:3])
160 | 
161 |                     haveseen_btn = row.find('div', {'onclick': 'markEpisodeAsWatched(this);'})
162 |                     episode_code = haveseen_btn['data-episode'].rjust(9, '0')
163 | 
164 |                     self.dates[episode_code] = release_date_str
165 |                     
166 |                     executor.submit(self.get_torrents, href, episode_code, True)
167 |                 
168 |                 page_number += 1
169 | 
170 |     def get_fav(self):
171 |         page = self.session.request(SITE_URL + '/my/type_1')
172 | 
173 |         with concurrent.futures.ThreadPoolExecutor() as executor:
174 |             for serial in Parser(page).find_all('div', {'class': 'serial-box'}):
175 |                 href = serial.find('a', {'class': 'body'})['href']
176 |                 executor.submit(self.get_episodes, href)
177 | 
178 |     def get_episodes(self, serial_href):
179 |         self.old_seasons.setdefault(serial_href, 0)
180 | 
181 |         serial_page = self.session.request(self.serial_url_pattern.format(href=serial_href))
182 |         with concurrent.futures.ThreadPoolExecutor() as executor:
183 |             for button in Parser(serial_page).find_all('div', {'class': 'external-btn'}):
184 |                 item_button = button.attrs.get('onclick')
185 | 
186 |                 if item_button:
187 |                     episode_code = re.search(r'\d{7,9}', item_button)[0].rjust(9, '0')
188 |                     logger.debug('episode_code = %s', episode_code)
189 |                     executor.submit(self.get_torrents, serial_href, episode_code)
190 | 
191 |     def get_torrents(self, href, code, new_episodes=False):
192 |         season, episode = int(code[3:6]), int(code[6:])
193 |         
194 |         if not any((
195 | 			season > self.old_seasons.get(href, -1),
196 | 			episode == self.all_episodes,
197 | 			season == self.additional_season,
198 | 			new_episodes
199 |         )):
200 |             return
201 | 
202 |         redir_page = self.session.request(self.download_url_pattern.format(code=code))
203 |         torrent_page_url = re.search(r'(?<=location.replace\(").+(?="\);)', redir_page)
204 | 
205 |         if not torrent_page_url:
206 |             return
207 |         
208 |         torrent_page = self.session.request(torrent_page_url[0])
209 |         date = '' if not new_episodes else '[' + self.dates.pop(code, '') + ']'
210 |         desc_link = self.get_description_url(href, code)
211 | 
212 |         logger.debug('desc_link = %s', desc_link)
213 | 
214 |         with concurrent.futures.ThreadPoolExecutor() as executor:
215 |             for torrent_tag in Parser(torrent_page).find_all('div', {'class': 'inner-box--item'}):
216 |                 main = torrent_tag.find('div', {'class': 'inner-box--link main'}).a
217 |                 link, name = main['href'], ' '.join((main.text.replace('\n', ' '), date))
218 | 
219 |                 if not new_episodes:
220 |                     if link in self.prevs:
221 |                         self.old_seasons[href] = max(self.old_seasons.get(href, 0), season)
222 |                         break
223 |                     
224 |                     self.prevs.add(link)
225 | 
226 |                 size, unit = re.search(
227 |                     r'\d+.\d+ \w\w(?=\.)', 
228 |                     torrent_tag.find('div', {'class': 'inner-box--desc'}).text
229 |                 )[0].split()
230 | 
231 |                 torrent_dict = {
232 |                     'link': link,
233 |                     'name': name,
234 |                     'size': ' '.join((size, self.units_dict.get(unit, ''))),
235 |                     'seeds': -1,
236 |                     'leech': -1,
237 |                     'engine_url': self.url,
238 |                     'desc_link': desc_link
239 |                 }
240 | 
241 |                 if ENABLE_PEERS_INFO:
242 |                     future = executor.submit(self.get_torrent_info, torrent_dict)
243 |                     future.add_done_callback(lambda f: self.pretty_printer(f.result()))
244 |                 else:
245 |                     self.pretty_printer(torrent_dict)
246 | 
247 |     def get_description_url(self, href, code):
248 |         season, episode = int(code[3:6]), int(code[6:])
249 | 
250 |         if season == self.additional_season:
251 |             return self.additional_url_pattern.format(href=href, episode=episode)
252 | 
253 |         elif episode == self.all_episodes:
254 |             return self.season_url_pattern.format(href=href, season=season)
255 | 
256 |         else:
257 |             return self.episode_url_pattern.format(href=href, season=season, episode=episode)
258 | 
259 |     def get_torrent_info(self, tdict):
260 |         response = self.session.request(tdict['link'], decode=False)
261 |         if not response:
262 |             return tdict
263 | 
264 |         torrent = self.decode_data(response)
265 |         torrent_info = self.encode_obj(torrent.get(b'info'))
266 |         if not torrent_info:
267 |             return tdict
268 | 
269 |         info_hash = hashlib.sha1(torrent_info).digest()
270 | 
271 |         params = {
272 |             'peer_id': self.peer_id,
273 |             'info_hash': info_hash,
274 |             'port': 6881,
275 |             'left': 0,
276 |             'downloaded': 0,
277 |             'uploaded': 0,
278 |             'compact': 1
279 |         }
280 |         url = torrent[b'announce'].decode('utf-8') + '?' + parse.urlencode(params)
281 |         response = self.session.request(url, decode=False)
282 |         if response:
283 |             data = self.decode_data(response)
284 |             tdict['seeds'] = data.get(b'complete', 0) - 1
285 |             tdict['leech'] = data.get(b'incomplete', -1)
286 |         
287 |         return tdict
288 | 
289 |     def decode_data(self, data):
290 |         try:
291 |             return bdecode(data)
292 |         except Exception as e:
293 |             logger.error(e)
294 |             return dict()
295 | 
296 |     def encode_obj(self, obj):
297 |         try:
298 |             return bencode(obj)
299 |         except Exception as e:
300 |             logger.error(e)
301 |             return b''
302 |     
303 |     def pretty_printer(self, dictionary):
304 |         if dictionary['link'] == 'Error':
305 |             logger.error(dictionary)
306 |         else:
307 |             logger.debug(dictionary)
308 |             self.torrents_count += 1
309 | 
310 |         if self.output:
311 |             try:
312 |                 prettyPrinter(dictionary.copy())
313 |             except OSError as e:
314 |                 logger.error('error %s on printing %s', e, dictionary)
315 | 
316 | 
317 | class Session:
318 |     site_name = 'lostfilm'
319 |     file_name = 'lostfilm.json'
320 |     datetime_format = '%m-%d-%y %H:%M:%S'
321 | 
322 |     token = None
323 |     time = None
324 |     _error = None
325 | 
326 |     @property
327 |     def error(self):
328 |         return 'Error: {info}.'.format(info=self._error)
329 | 
330 |     @property
331 |     def file_path(self):
332 |         """path to file with session data"""
333 |         return os.path.join(STORAGE, self.file_name)
334 | 
335 |     @property
336 |     def is_actual(self):
337 |         """Checks the relevance of the token"""
338 | 
339 |         if self.token and self.time and not self._error:
340 |             delta = datetime.now() - self.time
341 |             return delta.days < 1
342 | 
343 |         return False
344 | 
345 |     @property
346 |     def cookies(self):
347 |         if not self.is_actual:
348 |             self.create_new()
349 | 
350 |         return {'lf_session': self.token}
351 | 
352 |     def __init__(self):
353 |         self.load_data()
354 |             
355 |         if not self.is_actual:
356 |             if self.create_new():
357 |                 self.save_data()
358 | 
359 |     def request(self, url, params=None, decode=True):
360 |         args = [url]
361 | 
362 |         try:
363 |             if proxy['enable'] and self.site_name in url:
364 |                 opener = request.build_opener(
365 |                     request.ProxyBasicAuthHandler(),
366 |                     request.ProxyHandler(proxy['proxy_urls'])
367 |                 )
368 | 
369 |                 logger.info('proxy used for "%s"', url)
370 |             else:
371 |                 opener = request.build_opener()
372 | 
373 |             # use cookies only for lostfilm site urls
374 |             if self.site_name in url:
375 |                 if not params:
376 |                     params = self.cookies
377 |                 else:
378 |                     params.update(self.cookies)
379 | 
380 |             if params:
381 |                 args.append(parse.urlencode(params).encode('utf-8'))
382 | 
383 |             result = opener.open(*args).read()
384 | 
385 |             return result if not decode else result.decode('utf-8')
386 | 
387 |         except Exception as e:
388 |             logger.error('%s url="%s" params="%s"' % (e, url, params))
389 | 
390 |     def load_data(self):
391 |         if not os.path.exists(self.file_path):
392 |             return
393 | 
394 |         with open(self.file_path, 'r') as file:
395 |             result = json.load(file)
396 | 
397 |         if result.get('token') and result.get('time'):
398 |             self.token = result['token']
399 |             self.time = self.datetime_from_string(result['time'])
400 | 
401 |             logger.info('%s %s', self.token, self.time)
402 | 
403 |     def create_new(self):
404 |         self._error = None
405 | 
406 |         if not (EMAIL and PASSWORD):
407 |             self._error = 'Incorrect login data'
408 |             logger.error(self._error)
409 |             return False
410 | 
411 |         login_data = {
412 |             "act": "users",
413 |             "type": "login",
414 |             "mail": EMAIL,
415 |             "pass": PASSWORD,
416 |             "need_captcha": "",
417 |             "captcha": "",
418 |             "rem": 1
419 |         }
420 |         
421 |         url = SITE_URL + '/ajaxik.php?'
422 |         params = parse.urlencode(login_data).encode('utf-8')
423 |         
424 |         cjar = CookieJar()
425 |         if proxy['enable']:
426 |             opener = request.build_opener(
427 |                 request.ProxyHandler(proxy['proxy_urls']),
428 |                 request.HTTPCookieProcessor(cjar)
429 |             )
430 |             logger.debug('proxy used')
431 |         else:
432 |             opener = request.build_opener(request.HTTPCookieProcessor(cjar))
433 | 
434 |         try:
435 |             response = opener.open(url, params).read().decode('utf-8')
436 |         except Exception as e:
437 |             self._error = 'Connection failed'
438 |             logger.error('%s %s', self._error, e)
439 |             return False
440 | 
441 |         result = json.loads(response)
442 |         
443 |         if 'error' in result:
444 |             self._error = 'Incorrect login data'
445 | 
446 |         elif 'need_captcha' in result:
447 |             self._error = 'Captcha requested'
448 | 
449 |         else:
450 |             for cookie in cjar:
451 |                 if cookie.name == 'lf_session':
452 |                     self.time = datetime.now()
453 |                     self.token = cookie.value
454 |                     
455 |                     logger.info('%s %s', self.token, self.time)
456 | 
457 |                     return True
458 | 
459 |             else:
460 |                 self._error = 'Token problem'
461 | 
462 |         logger.error(self._error)
463 | 
464 |         return False
465 | 
466 |     def save_data(self):
467 |         data = {
468 |             "token": self.token,
469 |             "time": None if not self.time else self.datetime_to_string(self.time)
470 |         }
471 | 
472 |         logger.info(data)
473 | 
474 |         with open(self.file_path, 'w') as file:
475 |             json.dump(data, file)
476 | 
477 |     def datetime_to_string(self, dt_obj):
478 |         if isinstance(dt_obj, datetime):
479 |             return dt_obj.strftime(self.datetime_format)
480 | 
481 |         else:
482 |             raise TypeError('argument must be datetime, not %s' % (type(dt_obj)))
483 | 
484 |     def datetime_from_string(self, dt_string):
485 |         if isinstance(dt_string, str):
486 |             return datetime.strptime(dt_string, self.datetime_format)
487 | 
488 |         else:
489 |             raise TypeError('argument must be str, not %s' % (type(dt_string)))
490 | 
491 | 
492 | class Tag:
493 |     def __init__(self, tag=None, attrs=(), is_self_closing=None):
494 |         self.type = tag
495 |         self.is_self_closing = is_self_closing
496 |         self._attrs = tuple(attrs)
497 |         self._content = tuple()
498 |         
499 |     @property
500 |     def attrs(self):
501 |         """returns dict of Tag's attrs"""
502 |         return dict(self._attrs)
503 | 
504 |     @property
505 |     def text(self):
506 |         """returns str of all contained text"""
507 |         return ''.join(c if isinstance(c, str) else c.text for c in self._content)
508 | 
509 |     def _add_content(self, obj):
510 |         if isinstance(obj, (Tag, str)):
511 |             self._content += (obj,)
512 |         else:
513 |             raise TypeError('Argument must be str or %s, not %s' % (self.__class__, obj.__class__))
514 | 
515 |     def find(self, tag=None, attrs=None):
516 |         """returns Tag or None"""
517 | 
518 |         return next(self._find_all(tag, attrs), None)
519 | 
520 |     def find_all(self, tag=None, attrs=None):
521 |         """returns list"""
522 | 
523 |         return list(self._find_all(tag, attrs))
524 |         
525 |     def _find_all(self, tag_type=None, attrs=None):
526 |         """returns generator"""
527 | 
528 |         if not (isinstance(tag_type, (str, Tag)) or tag_type is None):
529 |             raise TypeError('tag_type argument must be str or Tag, not %s' % (tag_type.__class__))
530 | 
531 |         if not (isinstance(attrs, dict) or attrs is None):
532 |             raise TypeError('attrs argument must be dict, not %s' % (self.__class__))
533 | 
534 |         # get tags-descendants generator
535 |         results = self.descendants
536 | 
537 |         # filter by Tag.type
538 |         if tag_type:
539 |             if isinstance(tag_type, Tag):
540 |                 tag_type, attrs = tag_type.type, (attrs if attrs else tag_type.attrs)
541 | 
542 |             results = filter(lambda t: t.type == tag_type, results)
543 | 
544 |         # filter by Tag.attrs
545 |         if attrs:
546 |             # remove Tags without attrs
547 |             results = filter(lambda t: t._attrs, results)
548 | 
549 |             def filter_func(tag):
550 |                 for key in attrs.keys():
551 |                     if attrs[key] not in tag.attrs.get(key, ()):
552 |                         return False
553 |                 return True
554 |             
555 |             # filter by attrs
556 |             results = filter(filter_func, results)
557 |         
558 |         yield from results
559 | 
560 |     @property
561 |     def children(self):
562 |         """returns generator of tags-children"""
563 | 
564 |         return (obj for obj in self._content if isinstance(obj, Tag))
565 | 
566 |     @property
567 |     def descendants(self):
568 |         """returns generator of tags-descendants"""
569 | 
570 |         for child_tag in self.children:
571 |             yield child_tag
572 |             yield from child_tag.descendants
573 | 
574 |     def __getitem__(self, key):
575 |         return self.attrs[key]
576 | 
577 |     def __getattr__(self, attr):
578 |         if not attr.startswith("__"):
579 |             return self.find(tag=attr)
580 | 
581 |     def __repr__(self):
582 |         attrs = ' '.join(str(k) if v is None else '{}="{}"'.format(k, v) for k, v in self._attrs)
583 |         starttag = ' '.join((self.type, attrs)) if attrs else self.type
584 | 
585 |         if self.is_self_closing:
586 |             return '<{}>\n'.format(starttag)
587 |         else:
588 |             nested = '\n' * bool(next(self.children, None)) + ''.join(map(str, self._content))
589 |             return '<{}>{}</{}>\n'.format(starttag, nested, self.type)
590 | 
591 |             
592 | class Parser(HTMLParser):
593 |     def __init__(self, html_code, *args, **kwargs):
594 |         super().__init__(*args, **kwargs)
595 | 
596 |         self._root = Tag('_root')
597 |         self._path = [self._root]
598 |         
599 |         self.feed(''.join(map(str.strip, html_code.splitlines())))
600 |         self.handle_endtag(self._root.type)
601 |         self.close()
602 | 
603 |         self.find = self._root.find
604 |         self.find_all = self._root.find_all
605 | 
606 |     @property
607 |     def attrs(self):
608 |         return self._root.attrs
609 | 
610 |     @property
611 |     def text(self):
612 |         return self._root.text
613 | 
614 |     def handle_starttag(self, tag, attrs):
615 |         self._path.append(Tag(tag=tag, attrs=attrs))
616 | 
617 |     def handle_endtag(self, tag_type):
618 |         for pos, tag in tuple(enumerate(self._path))[::-1]:
619 |             if isinstance(tag, Tag) and tag.type == tag_type and tag.is_self_closing is None:
620 |                 tag.is_self_closing = False
621 | 
622 |                 for obj in self._path[pos + 1:]:
623 |                     if isinstance(obj, Tag) and obj.is_self_closing is None:
624 |                         obj.is_self_closing = True
625 | 
626 |                     tag._add_content(obj)
627 | 
628 |                 self._path = self._path[:pos + 1]
629 | 
630 |                 break
631 | 
632 |     def handle_startendtag(self, tag, attrs):
633 |         self._path.append(Tag(tag=tag, attrs=attrs, is_self_closing=True))
634 | 
635 |     def handle_decl(self, decl):
636 |         self._path.append(Tag(tag='!'+decl, is_self_closing=True))
637 | 
638 |     def handle_data(self, text):
639 |         self._path.append(text)
640 | 
641 |     def __getitem__(self, key):
642 |         return self.attrs[key]
643 | 
644 |     def __getattr__(self, attr):
645 |         if not attr.startswith("__"):
646 |             return getattr(self._root, attr)
647 | 
648 |     def __repr__(self):
649 |         return ''.join(str(c) for c in self._root._content)
650 | 
651 | 
652 | def bencode(value):
653 |     if isinstance(value, dict):
654 |         return b'd%be' % b''.join([bencode(k) + bencode(v) for k, v in value.items()])
655 |     if isinstance(value, list) or isinstance(value, tuple):
656 |         return b'l%be' % b''.join([bencode(v) for v in value])
657 |     if isinstance(value, int):
658 |         return b'i%ie' % value
659 |     if isinstance(value, bytes):
660 |         return b'%i:%b' % (len(value), value)
661 | 
662 |     raise ValueError("Only int, bytes, list or dict can be encoded, got %s" % type(value).__name__)
663 | 
664 | 
665 | def bdecode(data):
666 |     class InvalidBencode(Exception):
667 |         @classmethod
668 |         def at_position(cls, error, position):
669 |             logger.error("%s at position %i" % (error, position))
670 |             return cls("%s at position %i" % (error, position))
671 | 
672 |         @classmethod
673 |         def eof(cls):
674 |             logger.error("EOF reached while parsing")
675 |             return cls("EOF reached while parsing")
676 |             
677 |     def decode_from_io(f):
678 |         char = f.read(1)
679 |         if char == b'd':
680 |             dict_ = OrderedDict()
681 |             while True:
682 |                 position = f.tell()
683 |                 char = f.read(1)
684 |                 if char == b'e':
685 |                     return dict_
686 |                 if char == b'':
687 |                     raise InvalidBencode.eof()
688 | 
689 |                 f.seek(position)
690 |                 key = decode_from_io(f)
691 |                 dict_[key] = decode_from_io(f)
692 | 
693 |         if char == b'l':
694 |             list_ = []
695 |             while True:
696 |                 position = f.tell()
697 |                 char = f.read(1)
698 |                 if char == b'e':
699 |                     return list_
700 |                 if char == b'':
701 |                     raise InvalidBencode.eof()
702 |                 f.seek(position)
703 |                 list_.append(decode_from_io(f))
704 | 
705 |         if char == b'i':
706 |             digits = b''
707 |             while True:
708 |                 char = f.read(1)
709 |                 if char == b'e':
710 |                     break
711 |                 if char == b'':
712 |                     raise InvalidBencode.eof()
713 |                 if not char.isdigit():
714 |                     raise InvalidBencode.at_position('Expected int, got %s' % str(char), f.tell())
715 |                 digits += char
716 |             return int(digits)
717 | 
718 |         if char.isdigit():
719 |             digits = char
720 |             while True:
721 |                 char = f.read(1)
722 |                 if char == b':':
723 |                     break
724 |                 if char == b'':
725 |                     raise InvalidBencode
726 |                 digits += char
727 |             length = int(digits)
728 |             string = f.read(length)
729 |             return string
730 | 
731 |         raise InvalidBencode.at_position('Unknown type : %s' % char, f.tell())
732 | 
733 |     return decode_from_io(BytesIO(data))
734 | 
735 | 
736 | if __name__ == '__main__':
737 |     import sys
738 |     
739 |     if 1 < len(sys.argv) < 4:
740 | 
741 |         if len(sys.argv) == 3:
742 |             if sys.argv[1] == '-d':
743 |                 logger.setLevel(logging.DEBUG)
744 |             
745 |             else:
746 |                 print('%s [-d] "search query"' % (__file__))
747 |                 exit()
748 |         else:
749 |             logger.setLevel(logging.INFO)
750 |         
751 |         lostfilm(True).search(sys.argv[-1])
752 | 


--------------------------------------------------------------------------------