├── .gitignore
├── LICENSE
├── README.md
├── funnel.png
├── setup.py
├── tests
    └── test_funnel.py
└── video_funnel
    ├── __init__.py
    ├── __main__.py
    ├── funnel.py
    ├── index.dev.html
    ├── index.html
    ├── server.py
    └── utils.py


/.gitignore:
--------------------------------------------------------------------------------
  1 | # Byte-compiled / optimized / DLL files
  2 | __pycache__/
  3 | *.py[cod]
  4 | *$py.class
  5 | 
  6 | # C extensions
  7 | *.so
  8 | 
  9 | # Distribution / packaging
 10 | .Python
 11 | build/
 12 | develop-eggs/
 13 | dist/
 14 | downloads/
 15 | eggs/
 16 | .eggs/
 17 | lib/
 18 | lib64/
 19 | parts/
 20 | sdist/
 21 | var/
 22 | wheels/
 23 | *.egg-info/
 24 | .installed.cfg
 25 | *.egg
 26 | MANIFEST
 27 | 
 28 | # PyInstaller
 29 | #  Usually these files are written by a python script from a template
 30 | #  before PyInstaller builds the exe, so as to inject date/other infos into it.
 31 | *.manifest
 32 | *.spec
 33 | 
 34 | # Installer logs
 35 | pip-log.txt
 36 | pip-delete-this-directory.txt
 37 | 
 38 | # Unit test / coverage reports
 39 | htmlcov/
 40 | .tox/
 41 | .coverage
 42 | .coverage.*
 43 | .cache
 44 | nosetests.xml
 45 | coverage.xml
 46 | *.cover
 47 | .hypothesis/
 48 | 
 49 | # Translations
 50 | *.mo
 51 | *.pot
 52 | 
 53 | # Django stuff:
 54 | *.log
 55 | .static_storage/
 56 | .media/
 57 | local_settings.py
 58 | 
 59 | # Flask stuff:
 60 | instance/
 61 | .webassets-cache
 62 | 
 63 | # Scrapy stuff:
 64 | .scrapy
 65 | 
 66 | # Sphinx documentation
 67 | docs/_build/
 68 | 
 69 | # PyBuilder
 70 | target/
 71 | 
 72 | # Jupyter Notebook
 73 | .ipynb_checkpoints
 74 | 
 75 | # pyenv
 76 | .python-version
 77 | 
 78 | # celery beat schedule file
 79 | celerybeat-schedule
 80 | 
 81 | # SageMath parsed files
 82 | *.sage.py
 83 | 
 84 | # Environments
 85 | .env
 86 | .venv
 87 | env/
 88 | venv/
 89 | ENV/
 90 | env.bak/
 91 | venv.bak/
 92 | 
 93 | # Spyder project settings
 94 | .spyderproject
 95 | .spyproject
 96 | 
 97 | # Rope project settings
 98 | .ropeproject
 99 | 
100 | # mkdocs documentation
101 | /site
102 | 
103 | # mypy
104 | .mypy_cache/
105 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2017 Chen Shuaimin
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | <p align="center"><img src="funnel.png" /></p>
 2 | <h1 align="center">Video Funnel</h1>
 3 | <p align="center"><a href="https://badge.fury.io/py/video-funnel"><img src="https://badge.fury.io/py/video-funnel.svg" alt="PyPI version" height="18"></a></p> 
 4 | <p align="center">让你在线看视频也能达到多线程下载的速度</p>
 5 | 
 6 | #### 马上使用：
 7 | 
 8 | 1. 从 [PyPI](https://pypi.python.org/pypi/video_funnel) 安装：
 9 | ```bash
10 | $ pip(3) install --user video_funnel
11 | # or
12 | $ sudo pip(3) install video_funnel
13 | ```
14 | 
15 | 2. 启动 `video_funnel` 的服务器：
16 | ```console
17 | $ vf -u http://pcs.baidu.com/... &
18 | * Listening at port 8080 ...
19 | ```
20 | 百度网盘的直链有很多油猴脚本可以获取，比如[这个](https://github.com/syhyz1990/baiduyun)，注意加上 Cookie 参数。
21 | 
22 | 3. 用 `vlc` 播放：
23 | ```bash
24 | $ vlc http://localhost:8080
25 | ```
26 | `mpv` 播放时会出现 `Seek failed` 的错误，原因未知（如果有路过的大神遇见过类似情况，请一定给我解释下～） #2
27 | 
28 | 另外 @Zxilly 贡献了个 Web UI，启动 vf 时不加 `-u` 参数即可使用。
29 | 
30 | #### 动机：
31 | 
32 | 众所周知，百度网盘之类产品的视频在线播放非常模糊，下载吧又限速，于是我写了 [aiodl](https://github.com/cshuaimin/aiodl) 这个下载器，通过 [EX-百度云盘](https://github.com/gxvv/ex-baiduyunpan/) 获取的直链来“多线程”下载。可是每次都要下载完才能看又十分不爽，直接用 mpv 之类的播放器播放直链又因为限速的原因根本没法看，遂有了本项目。
33 | 
34 | #### 实现思路：
35 | 
36 | 1. 先将视频按照一定大小分块。块的大小根据视频的清晰度而异，可通过命令行参数 `--block-size/-b` 来指定，默认为 4MB 。
37 | 2. 对于上一步中的一个块，再次分块——为区别改叫切片，启动多个协程来下载这些切片，以实现“多线程”提速的目的。块和切片大小一起决定了有多少个连接在同时下载。切片的大小通过 `--piece-size/-p` 来指定，默认为 1MB 。即默认有 4 个连接同时下载。
38 | 3. 一个块中的切片全部下载完后，就可以将数据传给播放器了。当播放器播放这一块的时候，回到第 2 步下载下一块数据。为节省内存，设置了在内存中最多存在 2 个下载完而又没有传给播放器的块。
39 | 4. 该如何把数据传给播放器呢？我最初的设想是通过标准输出，这样简单好写。但 stdio 是无法 seek 的，这就意味着你只能从视频的开头看起，无法快进 :P 现在的解决方案是用 HTTP 协议与播放器传输数据。需要快进的时候播放器发送 HTTP Range 请求，video_funnel 将请求中的范围经过分块、切片后“多线程”下载。
40 | 


--------------------------------------------------------------------------------
/funnel.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/cshuaimin/video-funnel/1535f8c75f6c3ced2bc3a1b4eb2a49bc283e26de/funnel.png


--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
 1 | from setuptools import setup
 2 | 
 3 | version = '0.3.0'
 4 | 
 5 | setup(
 6 |     name='video_funnel',
 7 |     packages=['video_funnel'],
 8 |     version=version,
 9 |     description='Use multiple connections to request the video, then feed the combined data to the player.',
10 |     author='Chen Shuaimin',
11 |     author_email='chen_shuaimin@outlook.com',
12 |     url='https://github.com/cshuaimin/video-funnel',
13 |     python_requires='>=3.6',
14 |     install_requires=['aiohttp == 3.5.4', 'argparse', 'tqdm', 'browsercookie', 'pycookiecheat'],
15 |     package_data={'video_funnel': ['index.html']},
16 | 
17 |     classifiers=[
18 |         'Development Status :: 3 - Alpha',
19 | 
20 |         'License :: OSI Approved :: MIT License',
21 | 
22 |         'Programming Language :: Python :: 3.6',
23 |     ],
24 |     license="MIT",
25 |     keywords='Online multi-threaded video play',
26 |     entry_points={
27 |         'console_scripts': [
28 |             'vf = video_funnel.__main__:main'
29 |         ]
30 |     }
31 | )
32 | 


--------------------------------------------------------------------------------
/tests/test_funnel.py:
--------------------------------------------------------------------------------
 1 | import asyncio
 2 | from io import BytesIO
 3 | from random import getrandbits
 4 | 
 5 | from aiohttp import web
 6 | 
 7 | from video_funnel.funnel import Funnel
 8 | from video_funnel.utils import HttpRange
 9 | 
10 | 
11 | async def test_funnel(tmp_path, aiohttp_client):
12 |     tmp_file = tmp_path / 'test'
13 |     data = bytes(getrandbits(8) for _ in range(16))
14 |     tmp_file.write_bytes(data)
15 | 
16 |     async def serve_file(request):
17 |         # FileResponse supports range requests.
18 |         return web.FileResponse(tmp_file)
19 | 
20 |     app = web.Application()
21 |     app.router.add_get('/', serve_file)
22 |     session = await aiohttp_client(app)
23 | 
24 |     async def test(block_size, piece_size):
25 |         r = HttpRange(0, len(data) - 1)
26 |         buf = BytesIO()
27 |         async with Funnel(
28 |                 url='/',
29 |                 range=r,
30 |                 session=session,
31 |                 block_size=block_size,
32 |                 piece_size=piece_size) as funnel:
33 |             async for block in funnel:
34 |                 buf.write(block)
35 |         assert buf.getvalue() == data
36 | 
37 |     tests = []
38 |     for block_size in range(1, len(data) + 1):
39 |         for piece_size in range(1, block_size + 1):
40 |             tests.append(asyncio.create_task(test(block_size, piece_size)))
41 |     await asyncio.gather(*tests)
42 | 


--------------------------------------------------------------------------------
/video_funnel/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/cshuaimin/video-funnel/1535f8c75f6c3ced2bc3a1b4eb2a49bc283e26de/video_funnel/__init__.py


--------------------------------------------------------------------------------
/video_funnel/__main__.py:
--------------------------------------------------------------------------------
 1 | from argparse import ArgumentParser
 2 | 
 3 | from aiohttp import web
 4 | 
 5 | from .server import make_app
 6 | 
 7 | 
 8 | def make_args():
 9 |     ap = ArgumentParser(
10 |         description='Video Funnel -- Use multiple connections to request the '
11 |         'video, then feed the combined data to the player.')
12 | 
13 |     ap.add_argument(
14 |         '--url',
15 |         '-u',
16 |         help='the video url, omitting it to run VF in WebUI mode '
17 |         '(in this mode, http://127.0.0.1:8080 returns the WebUI '
18 |         'instead of the video stream)')
19 |     ap.add_argument('--port', type=int, default=8080, help='port to listen')
20 |     ap.add_argument(
21 |         '--block-size',
22 |         '-b',
23 |         metavar='N',
24 |         default='4M',
25 |         help='size of one block')
26 |     ap.add_argument(
27 |         '--piece-size',
28 |         '-p',
29 |         metavar='N',
30 |         default='1M',
31 |         help='size of one piece')
32 |     ap.add_argument(
33 |         '--cookies-from',
34 |         '-c',
35 |         choices=['chrome', 'chromium', 'firefox'],
36 |         help='load browser cookies')
37 |     ap.add_argument(
38 |         '--use-original-url',
39 |         '-g',
40 |         action='store_true',
41 |         help='always use the original URL '
42 |         '(no optimization for 3XX response code)')
43 | 
44 |     return ap.parse_args()
45 | 
46 | 
47 | def main():
48 |     args = make_args()
49 |     print(f'* Listening at port {args.port} ...')
50 |     web.run_app(make_app(args), print=None, port=args.port)
51 | 
52 | 
53 | if __name__ == '__main__':
54 |     main()
55 | 


--------------------------------------------------------------------------------
/video_funnel/funnel.py:
--------------------------------------------------------------------------------
 1 | import asyncio
 2 | from io import BytesIO
 3 | 
 4 | import aiohttp
 5 | from tqdm import tqdm
 6 | 
 7 | from .utils import RangeNotSupportedError, hook_print, retry
 8 | 
 9 | 
10 | class Funnel:
11 |     def __init__(self, url, range, session, block_size, piece_size):
12 |         self.url = url
13 |         self.range = range
14 |         self.session = session
15 |         self.block_size = block_size
16 |         self.piece_size = piece_size
17 |         self.blocks = asyncio.Queue(maxsize=2)
18 | 
19 |     async def __aenter__(self):
20 |         self.producer = asyncio.ensure_future(self.produce_blocks())
21 |         return self
22 | 
23 |     async def __aexit__(self, type, value, tb):
24 |         self.producer.cancel()
25 |         while not self.blocks.empty():
26 |             self.blocks.get_nowait()
27 |         await self.producer
28 | 
29 |     # needs Python 3.6
30 |     async def __aiter__(self):
31 |         while not (self.producer.done() and self.blocks.empty()):
32 |             chunk = await self.blocks.get()
33 |             if isinstance(chunk, Exception):
34 |                 raise chunk
35 |             yield chunk
36 | 
37 |     @retry
38 |     async def request_piece(self, range, block_begin, bar):
39 |         headers = {'Range': 'bytes={0.begin}-{0.end}'.format(range)}
40 |         async with self.session.get(self.url, headers=headers) as resp:
41 |             if resp.status != 206:
42 |                 raise RangeNotSupportedError
43 |             async for chunk in resp.content.iter_any():
44 |                 # Here multiple request_piece() coroutines collaborate on
45 |                 # this buffer to generate the *block*, so we need to seek to
46 |                 # the correct position before writing.
47 |                 # The range parameter is the offset in the *entire file*,
48 |                 # so we need to convert it to an offset relative to the block.
49 |                 self.buffer.seek(range.begin - block_begin)
50 |                 self.buffer.write(chunk)
51 |                 bar.update(len(chunk))
52 |                 range.begin += len(chunk)
53 | 
54 |     async def produce_blocks(self):
55 |         for nr, block in enumerate(self.range.subranges(self.block_size)):
56 |             with tqdm(
57 |                     desc=f'Block #{nr}',
58 |                     leave=False,
59 |                     dynamic_ncols=True,
60 |                     total=block.size(),
61 |                     unit='B',
62 |                     unit_scale=True,
63 |                     unit_divisor=1024) as bar, hook_print(bar.write):
64 | 
65 |                 self.buffer = BytesIO()
66 |                 futures = [
67 |                     asyncio.ensure_future(
68 |                         self.request_piece(r, block.begin, bar))
69 |                     for r in block.subranges(self.piece_size)
70 |                 ]
71 |                 try:
72 |                     await asyncio.gather(*futures)
73 |                     await self.blocks.put(self.buffer.getvalue())
74 |                 except (asyncio.CancelledError, aiohttp.ClientError) as exc:
75 |                     for f in futures:
76 |                         f.cancel()
77 |                     #  Notify the consumer to leave
78 |                     #  -- which is waiting at the end of this queue!
79 |                     await self.blocks.put(exc)
80 |                     return
81 | 


--------------------------------------------------------------------------------
/video_funnel/index.dev.html:
--------------------------------------------------------------------------------
 1 | <!DOCTYPE html>
 2 | <html>
 3 | <head>
 4 | <title>Video Funnel</title>
 5 | <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 6 | <link rel="stylesheet" href="https://cdnjs.loli.net/ajax/libs/mdui/0.4.2/css/mdui.min.css" integrity="sha256-ciZjlE3mLkmwd+Xel9t1pR7alsPIJj9yLb0lXcKjt+U=" crossorigin="anonymous" />
 7 | <script src="https://cdnjs.loli.net/ajax/libs/mdui/0.4.2/js/mdui.js" integrity="sha256-2kEEmZa8zf1k6TtViyoY3su5HJEnd926SXdqN02Crr4=" crossorigin="anonymous"></script>
 8 | <!--	
 9 | <script src="https://cdn.bootcss.com/clipboard.js/2.0.4/clipboard.min.js"></script>
10 | -->
11 | <script>
12 | var $$ = mdui.JQ;
13 | function play()
14 | {
15 | var url = $$("#url").val();if (url =="" ||url ==null) {mdui.snackbar({message:'未输入URL！'});return;}
16 | var blocksize = $$("#blocksize").val();if (blocksize =="" ||blocksize ==null) {mdui.snackbar({message:'未输入Block Size！'});return;}
17 | var piecesize = $$("#piecesize").val();if (piecesize =="" ||piecesize ==null) {mdui.snackbar({message:'未输入Piece Size！'});return;}
18 | location.href = "api?url=" + encodeURIComponent(url) + "&block_size=" + blocksize + "&piece_size=" + piecesize;
19 | };
20 | </script>
21 | </head>
22 | <body class="mdui-theme-primary-indigo">
23 | 
24 | <div class="mdui-appbar">
25 | <div class="mdui-toolbar mdui-color-theme mdui-text-color-dark-text mdui-text-color-white-icon">
26 |   <i class="mdui-icon material-icons">av_timer</i>
27 |   <span class="mdui-typo-title">Video Funnel</span>
28 |   <div class="mdui-toolbar-spacer"></div>
29 |    <a href="https://github.com/cshuaimin/video-funnel" target="_blank" class="mdui-btn mdui-btn-icon mdui-ripple mdui-ripple-white" mdui-tooltip="{content: '查看 Github'}">
30 |       <svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 36 36" enable-background="new 0 0 36 36" xml:space="preserve" class="mdui-icon" style="width: 24px;height:24px;">
31 |         <path fill-rule="evenodd" clip-rule="evenodd" fill="#ffffff" d="M18,1.4C9,1.4,1.7,8.7,1.7,17.7c0,7.2,4.7,13.3,11.1,15.5
32 | 	c0.8,0.1,1.1-0.4,1.1-0.8c0-0.4,0-1.4,0-2.8c-4.5,1-5.5-2.2-5.5-2.2c-0.7-1.9-1.8-2.4-1.8-2.4c-1.5-1,0.1-1,0.1-1
33 | 	c1.6,0.1,2.5,1.7,2.5,1.7c1.5,2.5,3.8,1.8,4.7,1.4c0.1-1.1,0.6-1.8,1-2.2c-3.6-0.4-7.4-1.8-7.4-8.1c0-1.8,0.6-3.2,1.7-4.4
34 | 	c-0.2-0.4-0.7-2.1,0.2-4.3c0,0,1.4-0.4,4.5,1.7c1.3-0.4,2.7-0.5,4.1-0.5c1.4,0,2.8,0.2,4.1,0.5c3.1-2.1,4.5-1.7,4.5-1.7
35 | 	c0.9,2.2,0.3,3.9,0.2,4.3c1,1.1,1.7,2.6,1.7,4.4c0,6.3-3.8,7.6-7.4,8c0.6,0.5,1.1,1.5,1.1,3c0,2.2,0,3.9,0,4.5
36 | 	c0,0.4,0.3,0.9,1.1,0.8c6.5-2.2,11.1-8.3,11.1-15.5C34.3,8.7,27,1.4,18,1.4z"></path>
37 |       </svg>
38 |     </a>
39 | </div>
40 | </div>
41 | 
42 | <div class="mdui-container-fluid" style="margin-top:70px;min-width:320px;max-width:600px;">
43 | <div class="mdui-center mdui-text-color-grey-800 mdui-text-center" style="float: left;width:100%;font-size: 35px;">
44 | Video Funnel
45 | </div>
46 | <div class="mdui-center" style="width: 100%;float: left">
47 | <div class="mdui-textfield mdui-textfield-floating-label">
48 | <label class="mdui-textfield-label">URL</label>
49 | <input id="url" class="mdui-textfield-input" type="url" AUTOCOMPLETE="off"/>
50 | </div>
51 | <div class="mdui-textfield mdui-textfield-floating-label">
52 | <label class="mdui-textfield-label">Block Size</label>
53 | <input id="blocksize" class="mdui-textfield-input" type="text" AUTOCOMPLETE="off" value="10M"/>
54 | </div><div class="mdui-textfield mdui-textfield-floating-label">
55 | <label class="mdui-textfield-label">Piece Size</label>
56 | <input id="piecesize" class="mdui-textfield-input" type="text" AUTOCOMPLETE="off" value="2M"/>
57 | </div>
58 | <button id="submitButton" type="submit" onclick="play()" class=" mdui-ripple mdui-color-indigo-400 submitButton mdui-btn mdui-btn-raised  mdui-center" style="margin-top:20px">
59 | 播放
60 | </button>
61 | </div>
62 | </div>
63 | </body>
64 | </html>
65 | 


--------------------------------------------------------------------------------
/video_funnel/index.html:
--------------------------------------------------------------------------------
1 | <!DOCTYPE html><html><head><title>Video Funnel</title><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /><link rel="stylesheet" href="https://cdnjs.loli.net/ajax/libs/mdui/0.4.2/css/mdui.min.css" integrity="sha256-ciZjlE3mLkmwd+Xel9t1pR7alsPIJj9yLb0lXcKjt+U=" crossorigin="anonymous" /><script src="https://cdnjs.loli.net/ajax/libs/mdui/0.4.2/js/mdui.js" integrity="sha256-2kEEmZa8zf1k6TtViyoY3su5HJEnd926SXdqN02Crr4=" crossorigin="anonymous"></script><script>var $$ = mdui.JQ;function play(){var url = $$("#url").val();if (url =="" ||url ==null) {mdui.snackbar({message:'未输入URL！'});return;}var blocksize = $$("#blocksize").val();if (blocksize =="" ||blocksize ==null) {mdui.snackbar({message:'未输入Block Size！'});return;}var piecesize = $$("#piecesize").val();if (piecesize =="" ||piecesize ==null) {mdui.snackbar({message:'未输入Piece Size！'});return;}location.href = "api?url=" + encodeURIComponent(url) + "&block_size=" + blocksize + "&piece_size=" + piecesize;};</script></head><body class="mdui-theme-primary-indigo"><div class="mdui-appbar"><div class="mdui-toolbar mdui-color-theme mdui-text-color-dark-text mdui-text-color-white-icon"><i class="mdui-icon material-icons">av_timer</i><span class="mdui-typo-title">Video Funnel</span><div class="mdui-toolbar-spacer"></div><a href="https://github.com/cshuaimin/video-funnel" target="_blank" class="mdui-btn mdui-btn-icon mdui-ripple mdui-ripple-white" mdui-tooltip="{content: '查看 Github'}"><svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 36 36" enable-background="new 0 0 36 36" xml:space="preserve" class="mdui-icon" style="width: 24px;height:24px;"><path fill-rule="evenodd" clip-rule="evenodd" fill="#ffffff" d="M18,1.4C9,1.4,1.7,8.7,1.7,17.7c0,7.2,4.7,13.3,11.1,15.5	c0.8,0.1,1.1-0.4,1.1-0.8c0-0.4,0-1.4,0-2.8c-4.5,1-5.5-2.2-5.5-2.2c-0.7-1.9-1.8-2.4-1.8-2.4c-1.5-1,0.1-1,0.1-1	c1.6,0.1,2.5,1.7,2.5,1.7c1.5,2.5,3.8,1.8,4.7,1.4c0.1-1.1,0.6-1.8,1-2.2c-3.6-0.4-7.4-1.8-7.4-8.1c0-1.8,0.6-3.2,1.7-4.4	c-0.2-0.4-0.7-2.1,0.2-4.3c0,0,1.4-0.4,4.5,1.7c1.3-0.4,2.7-0.5,4.1-0.5c1.4,0,2.8,0.2,4.1,0.5c3.1-2.1,4.5-1.7,4.5-1.7	c0.9,2.2,0.3,3.9,0.2,4.3c1,1.1,1.7,2.6,1.7,4.4c0,6.3-3.8,7.6-7.4,8c0.6,0.5,1.1,1.5,1.1,3c0,2.2,0,3.9,0,4.5	c0,0.4,0.3,0.9,1.1,0.8c6.5-2.2,11.1-8.3,11.1-15.5C34.3,8.7,27,1.4,18,1.4z"></path></svg></a></div></div><div class="mdui-container-fluid" style="margin-top:70px;min-width:320px;max-width:600px;"><div class="mdui-center mdui-text-color-grey-800 mdui-text-center" style="float: left;width:100%;font-size: 35px;">Video Funnel</div><div class="mdui-center" style="width: 100%;float: left"><div class="mdui-textfield mdui-textfield-floating-label"><label class="mdui-textfield-label">URL</label><input id="url" class="mdui-textfield-input" type="url" AUTOCOMPLETE="off"/></div><div class="mdui-textfield mdui-textfield-floating-label"><label class="mdui-textfield-label">Block Size</label><input id="blocksize" class="mdui-textfield-input" type="text" AUTOCOMPLETE="off" value="10M"/></div><div class="mdui-textfield mdui-textfield-floating-label"><label class="mdui-textfield-label">Piece Size</label><input id="piecesize" class="mdui-textfield-input" type="text" AUTOCOMPLETE="off" value="2M"/></div><button id="submitButton" type="submit" onclick="play()" class=" mdui-ripple mdui-color-indigo-400 submitButton mdui-btn mdui-btn-raised  mdui-center" style="margin-top:20px">播放</button></div></div></body></html>
2 | 


--------------------------------------------------------------------------------
/video_funnel/server.py:
--------------------------------------------------------------------------------
  1 | import asyncio
  2 | from pathlib import Path
  3 | 
  4 | import aiohttp
  5 | from aiohttp import web
  6 | 
  7 | from .funnel import Funnel
  8 | from .utils import (
  9 |     HttpRange,
 10 |     RangeNotSupportedError,
 11 |     convert_unit,
 12 |     load_browser_cookies,
 13 |     retry,
 14 | )
 15 | 
 16 | 
 17 | async def make_response(request, url, block_size, piece_size, cookies_from,
 18 |                         use_original_url):
 19 |     session = request.app['session']
 20 |     if cookies_from:
 21 |         session.cookie_jar.update_cookies(
 22 |             load_browser_cookies(cookies_from, url))
 23 | 
 24 |     @retry
 25 |     async def get_info():
 26 |         nonlocal url
 27 |         async with session.head(url, allow_redirects=True) as resp:
 28 |             if resp.headers.get('Accept-Ranges') != 'bytes':
 29 |                 raise RangeNotSupportedError
 30 |             if not use_original_url:
 31 |                 url = resp.url
 32 |             return resp.content_length, resp.content_type
 33 | 
 34 |     try:
 35 |         content_length, content_type = await get_info()
 36 |     except RangeNotSupportedError as exc:
 37 |         msg = str(exc)
 38 |         print(msg)
 39 |         return web.Response(status=501, text=msg)
 40 |     except aiohttp.ClientError as exc:
 41 |         print(exc)
 42 |         return web.Response(status=exc.status)
 43 | 
 44 |     range = request.headers.get('Range')
 45 |     if range is None:
 46 |         # not a Range request - the whole file
 47 |         range = HttpRange(0, content_length - 1)
 48 |         resp = web.StreamResponse(
 49 |             status=200,
 50 |             headers={
 51 |                 'Content-Length': str(content_length),
 52 |                 'Content-Type': content_type,
 53 |                 'Accept-Ranges': 'bytes'
 54 |             })
 55 |     else:
 56 |         try:
 57 |             range = HttpRange.from_str(range, content_length)
 58 |         except ValueError:
 59 |             return web.Response(
 60 |                 status=416, headers={'Content-Range': f'*/{content_length}'})
 61 |         else:
 62 |             resp = web.StreamResponse(
 63 |                 status=206,
 64 |                 headers={
 65 |                     'Content-Type':
 66 |                     content_type,
 67 |                     'Content-Range':
 68 |                     f'bytes {range.begin}-{range.end}/{content_length}'
 69 |                 })
 70 | 
 71 |     if request.method == 'HEAD':
 72 |         return resp
 73 | 
 74 |     await resp.prepare(request)
 75 |     async with Funnel(
 76 |             url,
 77 |             range,
 78 |             session,
 79 |             block_size,
 80 |             piece_size,
 81 |     ) as funnel:
 82 |         try:
 83 |             async for chunk in funnel:
 84 |                 await resp.write(chunk)
 85 |             return resp
 86 |         except (aiohttp.ClientError, RangeNotSupportedError) as exc:
 87 |             print(exc)
 88 |             return web.Response(status=exc.status)
 89 |         except asyncio.CancelledError:
 90 |             raise
 91 | 
 92 | 
 93 | ROOT = Path(__file__).parent
 94 | 
 95 | 
 96 | async def index(request):
 97 |     return web.FileResponse(ROOT / 'index.html')
 98 | 
 99 | 
100 | async def cli(request):
101 |     args = request.app['args']
102 |     url = request.raw_path[1:] or args.url
103 |     return await make_response(
104 |         request,
105 |         url,
106 |         convert_unit(args.block_size),
107 |         convert_unit(args.piece_size),
108 |         args.cookies_from,
109 |         args.use_original_url,
110 |     )
111 | 
112 | 
113 | async def api(request):
114 |     args = request.app['args']
115 |     query = request.query
116 |     block_size = convert_unit(query.get('block_size', args.block_size))
117 |     piece_size = convert_unit(query.get('piece_size', args.piece_size))
118 |     return await make_response(
119 |         request,
120 |         query.get('url', args.url),
121 |         block_size,
122 |         piece_size,
123 |         query.get('cookies_from', args.cookies_from),
124 |         query.get('use_original_url', args.use_original_url),
125 |     )
126 | 
127 | 
128 | async def make_app(args):
129 |     app = web.Application()
130 |     app['args'] = args
131 |     if args.url is None:
132 |         app.router.add_get('/', index)
133 |         #  app.router.add_static('/static', ROOT / 'static')
134 |         app.router.add_get('/api', api)
135 |         app.router.add_get('/{_:https?://.+}', cli)
136 |     else:
137 |         app.router.add_get('/', cli)
138 |         app.router.add_get('/{_:https?://.+}', cli)
139 | 
140 |     async def session(app):
141 |         app['session'] = aiohttp.ClientSession(raise_for_status=True)
142 |         yield
143 |         await app['session'].close()
144 | 
145 |     app.cleanup_ctx.append(session)
146 | 
147 |     return app
148 | 


--------------------------------------------------------------------------------
/video_funnel/utils.py:
--------------------------------------------------------------------------------
  1 | import asyncio
  2 | import builtins
  3 | import re
  4 | import socket
  5 | import sys
  6 | import urllib.parse
  7 | from contextlib import contextmanager
  8 | from functools import wraps
  9 | 
 10 | import aiohttp
 11 | 
 12 | max_tries = 10
 13 | 
 14 | 
 15 | def convert_unit(s):
 16 |     """Convert sizes like 1M, 10k.
 17 | 
 18 |     >>> convert_unit('1')
 19 |     1
 20 |     >>> convert_unit('1B')
 21 |     1
 22 |     >>> convert_unit('10k')
 23 |     10240
 24 |     >>> convert_unit('1M') == 1024 * 1024
 25 |     True
 26 |     """
 27 |     num, unit = re.match(r'(\d+)([BKMG]?)', s, re.I).groups()
 28 |     units = {'B': 1, 'K': 1024, 'M': 1024 * 1024, 'G': 1024 * 1024 * 1024}
 29 |     return int(num) * units.get(unit.upper(), 1)
 30 | 
 31 | 
 32 | class HttpRange:
 33 |     """Class for iterating subrange.
 34 | 
 35 |     >>> r = HttpRange(0, 5)
 36 |     >>> r
 37 |     [0, 5]
 38 |     >>> list(r.iter_subrange(2))
 39 |     [[0, 1], [2, 3], [4, 5]]
 40 |     >>> list(HttpRange(0, 4).iter_subrange(2))
 41 |     [[0, 1], [2, 3], [4, 4]]
 42 |     >>> list(HttpRange(0, 1).iter_subrange(1))
 43 |     [[0, 0], [1, 1]]
 44 |     >>> list(HttpRange(1, 1).iter_subrange(1))
 45 |     [[1, 1]]
 46 |     >>> HttpRange.from_str('bytes=12-34', 35)
 47 |     [12, 34]
 48 |     >>> HttpRange.from_str('bytes=12-', 35)
 49 |     [12, 34]
 50 |     """
 51 |     pattern = re.compile(r'bytes=(\d+)-(\d*)')
 52 | 
 53 |     def __init__(self, begin, end):
 54 |         self.begin = begin
 55 |         self.end = end
 56 | 
 57 |     def __repr__(self):
 58 |         return '[{0.begin}, {0.end}]'.format(self)
 59 | 
 60 |     def size(self):
 61 |         return self.end - self.begin + 1
 62 | 
 63 |     @classmethod
 64 |     def from_str(cls, range_str, content_length):
 65 |         match = cls.pattern.match(range_str)
 66 |         if not match:
 67 |             raise ValueError
 68 |         begin, end = match.groups()
 69 |         begin = int(begin)
 70 |         end = int(end) if end else content_length - 1
 71 |         if begin > end:
 72 |             raise ValueError
 73 |         if end >= content_length:
 74 |             end = content_length - 1
 75 |         return cls(begin, end)
 76 | 
 77 |     def subranges(self, size):
 78 |         begin = self.begin
 79 |         end = begin + size - 1
 80 |         while begin <= self.end:
 81 |             if end >= self.end:
 82 |                 end = self.end
 83 |             yield self.__class__(begin, end)
 84 |             begin = end + 1
 85 |             end += size
 86 | 
 87 | 
 88 | class RangeNotSupportedError(Exception):
 89 |     def __init__(self):
 90 |         # BaseException.__str__ returns the first argument
 91 |         # passed to BaseException.__init__.
 92 |         super().__init__('Range requests are not supported by the server.')
 93 | 
 94 | 
 95 | def retry(coro_func):
 96 |     @wraps(coro_func)
 97 |     async def wrapper(*args, **kwargs):
 98 |         tried = 0
 99 |         while True:
100 |             tried += 1
101 |             try:
102 |                 return await coro_func(*args, **kwargs)
103 |             except (aiohttp.ClientError, socket.gaierror) as exc:
104 |                 try:
105 |                     msg = f'{exc.status} {exc.message}'
106 |                     # For 4xx client errors, it's no use to try again :)
107 |                     if 400 <= exc.status < 500:
108 |                         print(msg)
109 |                         raise
110 |                 except AttributeError:
111 |                     msg = str(exc) or exc.__class__.__name__
112 | 
113 |                 if tried <= max_tries:
114 |                     sec = tried / 2
115 |                     print(f'{coro_func.__name__}() failed: {msg}, retry in '
116 |                           f'{sec:.1f} seconds ({tried}/{max_tries})')
117 |                     await asyncio.sleep(sec)
118 |                 else:
119 |                     print(f'{coro_func.__name__}() failed after '
120 |                           f'{max_tries} tries: {msg}')
121 |                     raise
122 | 
123 |             except asyncio.TimeoutError:
124 |                 # Usually server has a fixed TCP timeout to clean dead
125 |                 # connections, so you can see a lot of timeouts appear
126 |                 # at the same time. I don't think this is an error,
127 |                 # So retry it without checking the max retries.
128 |                 print(f'{coro_func.__name__}() timed out, retry in 1 second')
129 |                 await asyncio.sleep(1)
130 | 
131 |     return wrapper
132 | 
133 | 
134 | @contextmanager
135 | def hook_print(print):
136 |     save = builtins.print
137 |     builtins.print = print
138 |     try:
139 |         yield
140 |     finally:
141 |         builtins.print = save
142 | 
143 | 
144 | def load_browser_cookies(browser, url):
145 |     if browser is None:
146 |         return None
147 | 
148 |     # browsercookie can't get Chrome's cookie on Linux
149 |     if sys.platform.startswith('linux') and (browser == 'chrome'
150 |                                              or browser == 'chromium'):
151 |         from pycookiecheat import chrome_cookies
152 |         return chrome_cookies(url, browser=browser)
153 |     else:
154 |         import browsercookie
155 |         from aiohttp.cookiejar import CookieJar
156 | 
157 |         def _is_domain_match(domain, hostname):
158 |             # In aiohttp, this is done in previous steps.
159 |             if domain.startswith('.'):
160 |                 domain = domain[1:]
161 |             return CookieJar._is_domain_match(domain, hostname)
162 | 
163 |         with hook_print(lambda *_: None):
164 |             jar = getattr(browsercookie, browser)()
165 |         host = urllib.parse.urlsplit(url).netloc
166 |         return {
167 |             cookie.name: cookie.value
168 |             for cookie in jar if _is_domain_match(cookie.domain, host)
169 |         }
170 | 


--------------------------------------------------------------------------------