├── .gitignore ├── LICENSE ├── README.md ├── biligrab.py ├── biligrablite.py ├── danmaku2ass2.py └── danmaku2ass3.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # C extensions 6 | *.so 7 | 8 | # Distribution / packaging 9 | .Python 10 | env/ 11 | bin/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | eggs/ 16 | lib/ 17 | lib64/ 18 | parts/ 19 | sdist/ 20 | var/ 21 | *.egg-info/ 22 | .installed.cfg 23 | *.egg 24 | 25 | # Installer logs 26 | pip-log.txt 27 | pip-delete-this-directory.txt 28 | 29 | # Unit test / coverage reports 30 | htmlcov/ 31 | .tox/ 32 | .coverage 33 | .cache 34 | nosetests.xml 35 | coverage.xml 36 | 37 | # Translations 38 | *.mo 39 | 40 | # Mr Developer 41 | .mr.developer.cfg 42 | .project 43 | .pydevproject 44 | 45 | # Rope 46 | .ropeproject 47 | 48 | # Django stuff: 49 | *.log 50 | *.pot 51 | 52 | # Sphinx documentation 53 | docs/_build/ 54 | 55 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 David.Zhuang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Biligrab 2 | ======== 3 | 4 | Yet another automatic/semi-automatic/manual danmaku and video file downloader of Bilibili. 5 | 6 | This grabber is integrated with most of the "black techs". Good to bypass some copyright and 7 | geolocation restrictions: 8 | 9 | * 7 independent ways to parse source(s)! Now with BilibiliPr and you-get! And fake IP! 10 | * Auto concatenates and converts to MP4 (or FLV, even nothing, if not possible) file(s) via direct integration with [Mukioplayer-Py-Mac](https://github.com/cnbeining/Mukioplayer-Py-Mac) (the Flash danmaku playing solution) and [ABPlayer-HTML5-Mac](https://github.com/cnbeining/ABPlayerHTML5-Py--nix) (the HTML5 playing solution, preferred). 11 | * Interactive and command line mode for different use-cases. And also silent mode. 12 | * Process single or multiple videos with ease. Now you can use Bilibili *mylist*s. 13 | * Convert danmaku to ASS subtitles with ease using m13253's [Danmaku2ass](https://github.com/m13253/danmaku2ass) (GPLv2). 14 | Both py2 and master (py3) branches available for better danmaku handling. 15 | * Damaku-only exports 16 | * M3U exports -- play danmaku without waiting with players like MPlayer, MPC or VLC, etc. 17 | * Built-in multi-part download, useful to reduce slow overheads. 18 | 19 | Usage 20 | ----- 21 | 22 | If you have a Bilibili account, setting the cookie with [dantmnf/biliupload/getcookie.py](https://github.com/dantmnf/biliupload/blob/master/getcookie.py) will help you to download some of the restricted videos. You can do this by hand too. 23 | 24 | The file should look like: 25 | 26 | DedeUserID=123456;DedeUserID__ckMd5=****************;SESSDATA=******************* 27 | 28 | Interactive mode (Note some functionalities are not available in this mode): 29 | 30 | $ python2 biligrab.py 31 | 32 | Or command line mode: 33 | 34 | $ python2 biligrab.py -h 35 | python biligrab.py (-h) (-a) (-p) (-s) (-c) (-d) (-v) (-l) (-e) (-b) (-m) (-n) (-u) (-t) (-q) (-r) (-g) 36 | 37 | -h: Default: None 38 | Print this usage file. 39 | 40 | -a: Default: None 41 | The av number. 42 | If not set, Biligrab will use the fallback interactive mode. 43 | Support "~", "," and mix use. 44 | Examples: 45 | Input Output 46 | 1 [1] 47 | 1,2 [1, 2] 48 | 1~3 [1, 2, 3] 49 | 1,2~3 [1, 2, 3] 50 | 51 | -p: Default: 0 52 | The part number. 53 | Able to use the same syntax as "-a". 54 | If set to 0, Biligrab will download all the available parts in the video. 55 | 56 | -s: Default: 0 57 | Source to download. 58 | 0: The original API source, can be Letv backup, 59 | and can fail if the original video is not available(e.g., deleted) 60 | 1: The CDN API source, "oversea accelerate". 61 | Can be MINICDN backup in Mainland China or oversea. 62 | Good to bypass some bangumi's restrictions. 63 | 2: Force to use the original source. 64 | Use Flvcd to parse the video, but would fail if 65 | 1) The original source DNE, e.g., some old videos 66 | 2) The original source is Letvcloud itself. 67 | 3) Other unknown reason(s) that stops Flvcd from parsing the video. 68 | For any video that failed to parse, Biligrab will try to use Flvcd. 69 | (Mainly for oversea users regarding to copyright-restricted bangumies.) 70 | If the API is blocked, Biligrab would fake the UA. 71 | 3: (Not stable) Use the HTML5 API. 72 | This works for downloading some cached Letvcloud videos, but is slow, and would fail for no reason sometimes. 73 | Will retry if unavailable. 74 | 4: Use Flvcd. 75 | Good to fight with oversea and copyright restriction, but not working with iQiyi. 76 | May retrieve better quality video, especially for Youku. 77 | 5: Use BilibiliPr. 78 | Good to fight with some copyright restriction that BilibiliPr can fix. 79 | Not always working though. 80 | 6: Use You-get (https://github.com/soimort/you-get). 81 | You need a you-get callable directly like "you-get -u blahblah". 82 | 83 | -c: Default: ./bilicookies 84 | The path of cookies. 85 | Use cookies to visit member-only videos. 86 | 87 | -d: Default: None 88 | Set the desired download software. 89 | Biligrab supports aria2c(16 threads), axel(20 threads), wget and curl by far. 90 | If not set, Biligrab will detect an available one; 91 | If none of those is available, Biligrab will quit. 92 | For more software support, please open an issue at https://github.com/cnbeining/Biligrab/issues/ 93 | 94 | -v: Default:None 95 | Set the desired concatenate software. 96 | Biligrab supports ffmpeg by far. 97 | If not set, Biligrab will detect an available one; 98 | If none of those is available, Biligrab will quit. 99 | For more software support, please open an issue at https://github.com/cnbeining/Biligrab/issues/ 100 | Make sure you include a *working* command line example of this software! 101 | 102 | -l: Default: INFO 103 | Dump the log of the output for better debugging. 104 | Can be set to debug. 105 | 106 | -e: Default: 1 107 | Export Danmaku to ASS file. 108 | Fulfilled with danmaku2ass(https://github.com/m13253/danmaku2ass/tree/py2), 109 | Author: @m13253, GPLv3 License. 110 | *For issue with this function, if you think the problem lies on the danmaku2ass side, 111 | please open the issue at both projects.* 112 | If set to 1 or 2, Biligrab will use Danmaku2ass's py2 branch. 113 | If set to 3, Biligrab will use Danmaku2ass's master branch, which would require 114 | a python3 callable via 'python3'. 115 | If python3 not callable or danmaku2ass2/3 DNE, Biligrab will ask for action. 116 | 117 | -b: Default: None 118 | Set the probe software. 119 | Biligrab supports Mediainfo and FFprobe. 120 | If not set, Biligrab will detect an available one; 121 | If none of those is available, Biligrab will quit. 122 | For more software support, please open an issue at https://github.com/cnbeining/Biligrab/issues/ 123 | Make sure you include a *working* command line example of this software! 124 | 125 | -m: Default: 0 126 | Only download the danmaku. 127 | 128 | -n: Default: 0 129 | Silent Mode. 130 | Biligrab will not ask any question. 131 | 132 | -u: Default: 0 133 | Export video link to .m3u file, which can be used with MPlayer, mpc, VLC, etc. 134 | Biligrab will export a m3u8 instead of downloading any video(s). 135 | Can be broken with sources other than 0 or 1. 136 | 137 | -t: Default: None 138 | The number of Mylist. 139 | Biligrab will process all the videos in this list. 140 | 141 | -q: Default: 3 142 | The thread number for downloading. 143 | Good to fix overhead problem. 144 | 145 | -r: Default: -1 146 | Select video quality. 147 | Only works with Source 0 or 1. 148 | Range: 0~4, higher for better quality. 149 | 150 | -g: Default: 6 151 | Threads for downloading every part. 152 | Works with aria2 and axel. 153 | 154 | -i: Default: None 155 | Fake IP address. 156 | 157 | Requirements 158 | ------------ 159 | 160 | - Python 2.7 161 | - curl + None/aria2c/wget/axel 162 | - ffmpeg 163 | - mediainfo/ffprobe (for danmaku2ass) 164 | - Python 3.x (for danmaku2ass's python3 mode, or you-get) 165 | - you-get (See https://github.com/soimort/you-get for mode 6 info.) 166 | 167 | Author 168 | ----- 169 | Beining, http://www.cnbeining.com/ 170 | 171 | License 172 | ------- 173 | 174 | MIT license. 175 | 176 | The Danmaku2ass(master) and Danmaku2ass(py2) part belongs to @m13253, GPLv3 license. Used under the authorization of the original author. 177 | 178 | This program is provided **as is**, with absolutely no warranty. 179 | 180 | Contributing 181 | ------------ 182 | 183 | Any contribution are welcome. 184 | 185 | For issues, it would be better to include the log output, which can be enabled by `-l`. 186 | 187 | MAKE SURE YOU DELETE ANY SENSITIVE INFORMATION THAT YOU DO NOT WANT TO SHARE PUBLICLY (E.G., IP ADDRESS, USERNAME, ETC.) BEFORE YOU POST ANYTHING! 188 | 189 | *You can still send me the info privately via my email. PGP public key available at http://www.cnbeining.com/about/* 190 | 191 | Any donation is welcome as well. Please get in touch with me: cnbeining[at]gmail.com . 192 | 193 | Release History 194 | --------------- 195 | 0.98.95: Use the miniloader's key. 196 | 197 | 0.98.92: Again change API key set. 198 | 199 | 0.98.91: Change API key set. 200 | 201 | 0.98.86: Emergency update: Fake UA to download. 202 | 203 | 0.98.85: Rewrite url get function, also add IP faking. 204 | 205 | 0.98.81: Update as API domain update. Thanks for @m13253 's help. 206 | 207 | 0.98.8: Add download via you-get; Fix traceback printing. 208 | 209 | 0.98.72: Add thread control, as in #15 . 210 | 211 | 0.98.7: Add some quality selection. 212 | 213 | 0.98.6: Add BilibiliPr's API. 214 | 215 | 0.98.5: Add testing multi-part download, Change UA, Change error handling, Fix #13, #14. 216 | 217 | 0.98.4: Change UA, change to use best source, rewrite HTML5 API as in #11. 218 | 219 | 0.98.39: Change to download high quality Youku video, rewrite the error report and logging, retry if failed to fetch video, rewrite arguments. 220 | 221 | 0.98.3: Change default probe software to ffprobe; Fix error at resolution with danmaku export only; Better debugging output; Fix error with danmaku2ass(py2)'s float problem; Beautify code. 222 | 223 | 0.98.29: Add ver. number: Fix: do not fail to fake UA when use normal api #9, thanks to @arition 's help. 224 | 225 | 0.98.28: Fix typo; Fix error with CLI mode. 226 | 227 | 0.98.27: Update as #7. 228 | 229 | 0.98.26: Add more support of broken sources. 230 | 231 | 0.98.25: Add **some** support of M3U export of Sina source and source that was broken. 232 | 233 | 0.98.2: Add **some** support of M3U export of non-Sina source. 234 | 235 | 0.98.1: Add mylist download. 236 | 237 | 0.98: Fix error with special characters in filename; Add export to M3U file to use players like MPlayer, VLC, etc.; Rewrite video URL API logic; Fix error with Danmaku2ASS(main); Error handling with ffprobe 238 | 239 | 0.97.9: Rewrite URL retrieve logic; Divide URL retrieve to functions; Change to ```.format()``` style; Add HTML5 API; Directly use Flvcd; Beautify ERROR logging. 240 | 241 | 0.97.5: Add (auto) download all the pages; Auto PEP-8. 242 | 243 | 0.97: Silent mode; Multiple video mode; Functions beautify; More error handling. 244 | 245 | 0.96.2: Merge pull request #6, #7: Optimize Danmaku2ASS parameters and exception handling, thanks to @m13253's help; Fix error when cookie does not exist, thanks to @m13253's report. 246 | 247 | 0.96.1: Add exception handling regarding to Danmaku2ass2; Fix vid guessing. 248 | 249 | 0.96: Add danmaku2ass(py2) to handle ass convertion without Python 3; Add the danmaku2ass dependencies check mode for safety; Add "danmaku only" mode to depreciate BiligrabLite; Change the default live time to 8 sec as the update from upstream; Update the license info. 250 | 251 | 0.95: Add danmaku2ass, able to convert danmaku to ass file; Fix axel error. 252 | 253 | 0.94: Add faking UA to bypass blocking; Add auto-generate UA; Rewrite API logic. 254 | 255 | 0.93: Fix error when handling filenames containing ```/\&```, thanks to @solimot 's report; Add log mode, which can be enabled by ```-l 1 ```; Clean multiple headers; Rearrange global variables. 256 | 257 | 0.92: Fix wrongly exit when downloading multiple parts. 258 | 259 | 0.91: Add support to axel, wget, curl and easy way to add more support; Add easy way to add more concat support; Able to select desired software and auto detect; Change dependencies check; Code beauty. 260 | 261 | 0.90: Fix if cannot get download URL for some reason(geo location, or API server error), try to use Flvcd to download video. 262 | 263 | 0.89: Fix #4, force declare the variable, and set the path if not assigned. 264 | 265 | 0.88: Fix #3, 2 typos. 266 | 267 | 0.87: Able to edit cookie path. Fix cannot read cookie. 268 | 269 | 0.86: Add non-interact mode, change API domain, fix #2. 270 | 271 | 0.81: Fix Flvcd module; When failed to concat, try to concat to flv; If failed, leave the original file; Delete some lines to make it easier to intregrate; Fix domain name 272 | 273 | 0.8: Fix the most recent change with APIs, with player biliInterface-201407302359. Use own key instead of AcDown's. 274 | 275 | For history before V0.74, visit http://www.cnbeining.com/ , or check the code at https://gist.github.com/cnbeining/9605757/revisions . 276 | -------------------------------------------------------------------------------- /biligrab.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # Author: Beining -- 4 | # Purpose: Yet another danmaku and video file downloader of Bilibili. 5 | # Created: 11/06/2013 6 | # 7 | # Biligrab is licensed under MIT license (https://github.com/cnbeining/Biligrab/blob/master/LICENSE) 8 | # 9 | # Copyright (c) 2013-2015 10 | 11 | ''' 12 | Biligrab 13 | Beining@ACICFG 14 | cnbeining[at]gmail.com 15 | http://www.cnbeining.com 16 | https://github.com/cnbeining/Biligrab 17 | MIT license 18 | ''' 19 | 20 | from ast import literal_eval 21 | import sys 22 | import os 23 | from StringIO import StringIO 24 | import gzip 25 | import urllib 26 | import urllib2 27 | import math 28 | import json 29 | import commands 30 | import subprocess 31 | import hashlib 32 | import getopt 33 | import logging 34 | import traceback 35 | import threading 36 | import Queue 37 | from time import time 38 | 39 | from xml.dom.minidom import parseString 40 | 41 | try: 42 | from danmaku2ass2 import * 43 | except Exception: 44 | pass 45 | 46 | global vid, cid, partname, title, videourl, part_now, is_first_run, APPKEY, SECRETKEY, LOG_LEVEL, VER, LOCATION_DIR, VIDEO_FORMAT, convert_ass, is_export, IS_SLIENT, pages, IS_M3U, FFPROBE_USABLE, QUALITY, IS_FAKE_IP, FAKE_IP 47 | 48 | cookies, VIDEO_FORMAT = '', '' 49 | LOG_LEVEL, pages, FFPROBE_USABLE = 0, 0, 0 50 | APPKEY = '6f90a59ac58a4123' 51 | SECRETKEY = 'b78be1fef78c3e7fdc7633e5fd5eee90' 52 | SECRETKEY_MINILOADER = '1c15888dc316e05a15fdd0a02ed6584f' 53 | VER = '0.98.95' 54 | FAKE_UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.52 Safari/537.36' 55 | FAKE_HEADER = { 56 | 'User-Agent': FAKE_UA, 57 | 'Cache-Control': 'no-cache', 58 | 'Pragma': 'no-cache', 59 | 'pianhao': '%7B%22qing%22%3A%22super%22%2C%22qtudou%22%3A%22real%22%2C%22qyouku%22%3A%22super%22%2C%22q56%22%3A%22super%22%2C%22qcntv%22%3A%22super%22%2C%22qletv%22%3A%22super2%22%2C%22qqiyi%22%3A%22real%22%2C%22qsohu%22%3A%22real%22%2C%22qqq%22%3A%22real%22%2C%22qhunantv%22%3A%22super%22%2C%22qku6%22%3A%22super%22%2C%22qyinyuetai%22%3A%22super%22%2C%22qtangdou%22%3A%22super%22%2C%22qxunlei%22%3A%22super%22%2C%22qsina%22%3A%22high%22%2C%22qpptv%22%3A%22super%22%2C%22qpps%22%3A%22high%22%2C%22qm1905%22%3A%22high%22%2C%22qbokecc%22%3A%22super%22%2C%22q17173%22%3A%22super%22%2C%22qcuctv%22%3A%22super%22%2C%22q163%22%3A%22super%22%2C%22q51cto%22%3A%22high%22%2C%22xia%22%3A%22auto%22%2C%22pop%22%3A%22no%22%2C%22open%22%3A%22no%22%7D'} 60 | LOCATION_DIR = os.path.dirname(os.path.realpath(__file__)) 61 | 62 | #---------------------------------------------------------------------- 63 | def list_del_repeat(list): 64 | """delete repeated items in a list, and keep the order. 65 | http://www.cnblogs.com/infim/archive/2011/03/10/1979615.html""" 66 | l2 = [] 67 | [l2.append(i) for i in list if not i in l2] 68 | return(l2) 69 | 70 | #---------------------------------------------------------------------- 71 | def logging_level_reader(LOG_LEVEL): 72 | """str->int 73 | Logging level.""" 74 | return { 75 | 'INFO': logging.INFO, 76 | 'DEBUG': logging.DEBUG, 77 | 'WARNING': logging.WARNING, 78 | 'FATAL': logging.FATAL 79 | }.get(LOG_LEVEL) 80 | 81 | #---------------------------------------------------------------------- 82 | def calc_sign(string): 83 | """str/any->str 84 | return MD5.""" 85 | return str(hashlib.md5(str(string).encode('utf-8')).hexdigest()) 86 | 87 | #---------------------------------------------------------------------- 88 | def read_cookie(cookiepath): 89 | """str->list 90 | Original target: set the cookie 91 | Target now: Set the global header""" 92 | global BILIGRAB_HEADER 93 | try: 94 | cookies_file = open(cookiepath, 'r') 95 | cookies = cookies_file.readlines() 96 | cookies_file.close() 97 | # print(cookies) 98 | return cookies 99 | except Exception: 100 | logging.warning('Cannot read cookie, may affect some videos...') 101 | return [''] 102 | 103 | #---------------------------------------------------------------------- 104 | def clean_name(name): 105 | """str->str 106 | delete all the dramas in the filename.""" 107 | return (str(name).strip().replace('\\',' ').replace('/', ' ').replace('&', ' ')).replace('-', ' ') 108 | 109 | #---------------------------------------------------------------------- 110 | def send_request(url, header, is_fake_ip): 111 | """str,dict,int->str 112 | Send request, and return answer.""" 113 | global IS_FAKE_IP 114 | data = '' 115 | if IS_FAKE_IP == 1: 116 | header['X-Forwarded-For'] = FAKE_IP 117 | header['Client-IP'] = FAKE_IP 118 | header['X-Real-IP'] = FAKE_IP 119 | try: 120 | #logging.debug(header) 121 | request = urllib2.Request(url, headers=header) 122 | response = urllib2.urlopen(request) 123 | data = response.read() 124 | except urllib2.HTTPError: 125 | logging.info('ERROR!') 126 | return '' 127 | if response.info().get('Content-Encoding') == 'gzip': 128 | buf = StringIO(response.read()) 129 | f = gzip.GzipFile(fileobj=buf) 130 | data = f.read() 131 | #except Exception: 132 | #raise URLOpenException('Cannot open URL! Raw output:\n\n{output}'.format(output = command_result[1])) 133 | #print(request.headers) 134 | logging.debug(data) 135 | return data 136 | 137 | #---------------------------------------------------------------------- 138 | def mylist_to_aid_list(mylist): 139 | """str/int->list""" 140 | data = send_request('http://www.bilibili.com/mylist/mylist-{mylist}.js'.format(mylist = mylist), FAKE_HEADER, IS_FAKE_IP) 141 | #request = urllib2.Request('http://www.bilibili.com/mylist/mylist-{mylist}.js'.format(mylist = mylist), headers = FAKE_HEADER) 142 | #response = urllib2.urlopen(request) 143 | aid_list = [] 144 | #data = response.read() 145 | for i in data.split('\n')[-3].split(','): 146 | if 'aid' in i: 147 | aid_list.append(i.split(':')[1]) 148 | return aid_list 149 | 150 | 151 | 152 | #---------------------------------------------------------------------- 153 | def find_cid_api(vid, p, cookies): 154 | """find cid and print video detail 155 | str,int?,str->str,str,str,str 156 | TODO: Use json.""" 157 | global cid, partname, title, videourl, pages 158 | cid = 0 159 | title , partname , pages, = '', '', '' 160 | if str(p) is '0' or str(p) is '1': 161 | #str2Hash = 'appkey={APPKEY}&id={vid}&type=xml{SECRETKEY}'.format(APPKEY = APPKEY, vid = vid, SECRETKEY = SECRETKEY) 162 | #biliurl = 'https://api.bilibili.com/view?appkey={APPKEY}&id={vid}&type=xml&sign={sign}'.format(APPKEY = APPKEY, vid = vid, SECRETKEY = SECRETKEY, sign = calc_sign(str2Hash)) 163 | biliurl = 'https://api.bilibili.com/view?appkey={APPKEY}&id={vid}&type=xml'.format(APPKEY = '8e9fc618fbd41e28', vid = vid, SECRETKEY = SECRETKEY) 164 | 165 | else: 166 | #str2Hash = 'appkey={APPKEY}&id={vid}&page={p}&type=xml{SECRETKEY}'.format(APPKEY = APPKEY, vid = vid, p = p, SECRETKEY = SECRETKEY) 167 | #biliurl = 'https://api.bilibili.com/view?appkey={APPKEY}&id={vid}&page={p}&type=xml&sign={sign}'.format(APPKEY = APPKEY, vid = vid, SECRETKEY = SECRETKEY, p = p, sign = calc_sign(str2Hash)) 168 | biliurl = 'https://api.bilibili.com/view?appkey={APPKEY}&id={vid}&page={p}&type=xml'.format(APPKEY = '8e9fc618fbd41e28', vid = vid, SECRETKEY = SECRETKEY, p = p) 169 | logging.debug('BiliURL: ' + biliurl) 170 | videourl = 'http://www.bilibili.com/video/av{vid}/index_{p}.html'.format(vid = vid, p = p) 171 | logging.info('Fetching api to read video info...') 172 | data = '' 173 | try: 174 | #request = urllib2.Request(biliurl, headers=BILIGRAB_HEADER) 175 | #response = urllib2.urlopen(request) 176 | #data = response.read() 177 | data = send_request(biliurl, BILIGRAB_HEADER, IS_FAKE_IP) 178 | logging.debug('Bilibili API: ' + data) 179 | dom = parseString(data) 180 | for node in dom.getElementsByTagName('cid'): 181 | if node.parentNode.tagName == "info": 182 | cid = node.toxml()[5:-6] 183 | logging.info('cid is ' + cid) 184 | break 185 | for node in dom.getElementsByTagName('partname'): 186 | if node.parentNode.tagName == "info": 187 | partname = clean_name(str(node.toxml()[10:-11])) 188 | logging.info('partname is ' + partname)# no more /\ drama 189 | break 190 | for node in dom.getElementsByTagName('title'): 191 | if node.parentNode.tagName == "info": 192 | title = clean_name(str(node.toxml()[7:-8])).decode("utf-8") 193 | logging.info((u'Title is ' + title).encode(sys.stdout.encoding)) 194 | for node in dom.getElementsByTagName('pages'): 195 | if node.parentNode.tagName == "info": 196 | pages = clean_name(str(node.toxml()[7:-8])) 197 | logging.info('Total pages is ' + str(pages)) 198 | return [cid, partname, title, pages] 199 | except Exception: # If API failed 200 | logging.warning('Cannot connect to API server! \nIf you think this is wrong, please open an issue at \nhttps://github.com/cnbeining/Biligrab/issues with *ALL* the screen output, \nas well as your IP address and basic system info.\nYou can get these data via "-l".') 201 | logging.debug('API Data: ' + data) 202 | return ['', '', '', ''] 203 | 204 | #---------------------------------------------------------------------- 205 | def find_cid_flvcd(videourl): 206 | """str->None 207 | set cid.""" 208 | global vid, cid, partname, title 209 | logging.info('Fetching webpage with raw page...') 210 | #request = urllib2.Request(videourl, headers=FAKE_HEADER) 211 | data = send_request(videourl, FAKE_HEADER, IS_FAKE_IP) 212 | #request.add_header('Accept-encoding', 'gzip') 213 | #try: 214 | #response = urllib2.urlopen(request) 215 | #except urllib2.HTTPError: 216 | #logging.info('ERROR!') 217 | #return '' 218 | #if response.info().get('Content-Encoding') == 'gzip': 219 | #buf = StringIO(response.read()) 220 | #f = gzip.GzipFile(fileobj=buf) 221 | #data = f.read() 222 | data_list = data.split('\n') 223 | logging.debug(data) 224 | # Todo: read title 225 | for lines in data_list: 226 | if 'cid=' in lines: 227 | cid = lines.split('&') 228 | cid = cid[0].split('=') 229 | cid = cid[-1] 230 | logging.info('cid is ' + str(cid)) 231 | break 232 | 233 | #---------------------------------------------------------------------- 234 | def check_dependencies(download_software, concat_software, probe_software): 235 | """None->str,str,str 236 | Will give softwares for concat, download and probe. 237 | The detection of Python3 is located at the end of Main function.""" 238 | concat_software_list = ['ffmpeg', 'avconv'] 239 | download_software_list = ['aria2c', 'axel', 'wget', 'curl'] 240 | probe_software_list = ['ffprobe', 'mediainfo'] 241 | name_list = [[concat_software, 242 | concat_software_list], 243 | [download_software, 244 | download_software_list], 245 | [probe_software, 246 | probe_software_list]] 247 | for name in name_list: 248 | if name[0].strip().lower() not in name[1]: # Unsupported software 249 | # Set a Unsupported software, not blank 250 | if len(name[0].strip()) != 0: 251 | logging.warning('Requested Software not supported!\n Biligrab only support these following software(s):\n ' + str(name[1]) + '\n Trying to find available one...') 252 | for software in name[1]: 253 | output = commands.getstatusoutput(software + ' --help') 254 | if str(output[0]) != '32512': # If exist 255 | name[0] = software 256 | break 257 | if name[0] == '': 258 | logging.fatal('Cannot find software in ' + str(name[1]) + ' !') 259 | exit() 260 | return name_list[0][0], name_list[1][0], name_list[2][0] 261 | 262 | #---------------------------------------------------------------------- 263 | def download_video_link(part_number, download_software, video_link, thread_single_download): 264 | """set->str""" 265 | logging.info('Downloading #{part_number}...'.format(part_number = part_number)) 266 | if download_software == 'aria2c': 267 | cmd = 'aria2c -c -U "{FAKE_UA}" -s{thread_single_download} -x{thread_single_download} -k1M --out {part_number}.flv "{video_link}"' 268 | elif download_software == 'wget': 269 | cmd = 'wget -c -A "{FAKE_UA}" -O {part_number}.flv "{video_link}"' 270 | elif download_software == 'curl': 271 | cmd = 'curl -L -C - -A "{FAKE_UA}" -o {part_number}.flv "{video_link}"' 272 | elif download_software == 'axel': 273 | cmd = 'axel -U "{FAKE_UA}" -n {thread_single_download} -o {part_number}.flv "{video_link}"' 274 | cmd = cmd.format(part_number = part_number, video_link = video_link, thread_single_download = thread_single_download, FAKE_UA = FAKE_UA) 275 | logging.debug(cmd) 276 | return cmd 277 | 278 | #---------------------------------------------------------------------- 279 | def execute_cmd(cmd): 280 | """""" 281 | return_code = subprocess.call(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 282 | if return_code != 0: 283 | logging.warning('ERROR') 284 | return return_code 285 | 286 | def execute_sysencode_cmd(command): 287 | """execute cmd with sysencoding""" 288 | os.system(command.decode("utf-8").encode(sys.stdout.encoding)) 289 | 290 | #---------------------------------------------------------------------- 291 | def concat_videos(concat_software, vid_num, filename): 292 | """str,str->None""" 293 | global VIDEO_FORMAT,title 294 | if concat_software == 'ffmpeg': 295 | f = open('ff.txt', 'w') 296 | ff = '' 297 | cwd = os.getcwd() 298 | for i in range(vid_num): 299 | ff += 'file \'{cwd}/{i}.flv\'\n'.format(cwd = cwd, i = i) 300 | # ff = ff.encode("utf8") 301 | f.write(ff) 302 | f.close() 303 | logging.debug(ff) 304 | logging.info('Concating videos...') 305 | 306 | execute_sysencode_cmd('ffmpeg -f concat -i ff.txt -c copy "' + filename + '".mp4') 307 | VIDEO_FORMAT = 'mp4' 308 | if os.path.isfile((str(i) + '.mp4').decode("utf-8")): 309 | try: 310 | # os.remove('ff.txt') 311 | print((str(i) + '.flv').decode("utf-8")) 312 | os.remove((str(i) + '.flv').decode("utf-8")) 313 | for i in range(vid_num): 314 | os.remove((str(i) + '.flv').decode("utf-8")) 315 | #execute_sysencode_cmd('rm -r ' + str(i) + '.flv') 316 | logging.info('Done, enjoy yourself!') 317 | except Exception: 318 | logging.warning('Cannot delete temporary files!') 319 | return [''] 320 | else: 321 | print('ERROR: Cannot concatenate files, trying to make flv...') 322 | execute_sysencode_cmd('ffmpeg -f concat -i ff.txt -c copy "' + filename + '".flv') 323 | VIDEO_FORMAT = 'flv' 324 | if os.path.isfile((str(i) + '.flv').decode("utf-8")): 325 | logging.warning('FLV file made. Not possible to mux to MP4, highly likely due to audio format.') 326 | #execute_sysencode_cmd('rm -r ff.txt') 327 | # os.remove('ff.txt') 328 | print(('ff.txt').decode("utf-8")) 329 | os.remove(('ff.txt').decode("utf-8")) 330 | for i in range(vid_num): 331 | #execute_sysencode_cmd('rm -r ' + str(i) + '.flv') 332 | os.remove((str(i) + '.flv').decode("utf-8")) 333 | else: 334 | logging.error('Cannot concatenate files!') 335 | elif concat_software == 'avconv': 336 | pass 337 | 338 | #---------------------------------------------------------------------- 339 | def process_m3u8(url): 340 | """str->list 341 | Only Youku.""" 342 | url_list = [] 343 | data = send_request(url, FAKE_HEADER, IS_FAKE_IP) 344 | if data == '': 345 | logging.error('Cannot download required m3u8!') 346 | return [] 347 | #request = urllib2.Request(url, headers=BILIGRAB_HEADER) 348 | #try: 349 | #response = urllib2.urlopen(request) 350 | #except Exception: 351 | #logging.error('Cannot download required m3u8!') 352 | #return [] 353 | #data = response.read() 354 | #logging.debug(data) 355 | data = data.split() 356 | if 'youku' in url: 357 | return [data[4].split('?')[0]] 358 | 359 | #---------------------------------------------------------------------- 360 | def make_m3u8(video_list): 361 | """list->str 362 | list: 363 | [(VIDEO_URL, TIME_IN_SEC), ...]""" 364 | TARGETDURATION = int(max([i[1] for i in video_list])) + 1 365 | line = '#EXTM3U\n#EXT-X-TARGETDURATION:{TARGETDURATION}\n#EXT-X-VERSION:2\n'.format(TARGETDURATION = TARGETDURATION) 366 | for i in video_list: 367 | line += '#EXTINF:{time}\n{url}\n'.format(time = str(i[1]), url = i[0]) 368 | line += '#EXT-X-ENDLIST' 369 | logging.debug('m3u8: ' + line) 370 | return line 371 | 372 | #---------------------------------------------------------------------- 373 | def find_video_address_html5(vid, p, header): 374 | """str,str,dict->list 375 | Method #3.""" 376 | api_url = 'http://www.bilibili.com/m/html5?aid={vid}&page={p}'.format(vid = vid, p = p) 377 | data = send_request(api_url, header, IS_FAKE_IP) 378 | if data == '': 379 | logging.error('Cannot connect to HTML5 API!') 380 | return [] 381 | #request = urllib2.Request(api_url, headers=header) 382 | #url_list = [] 383 | #try: 384 | #response = urllib2.urlopen(request) 385 | #except Exception: 386 | #logging.error('Cannot connect to HTML5 API!') 387 | #return [] 388 | #data = response.read() 389 | #Fix #13 390 | #if response.info().get('Content-Encoding') == 'gzip': 391 | #data = gzip.GzipFile(fileobj=StringIO(data), mode="r").read() 392 | #logging.debug(data) 393 | info = json.loads(data.decode('utf-8')) 394 | raw_url = info['src'] 395 | if 'error.mp4' in raw_url: 396 | logging.error('HTML5 API returned ERROR or not available!') 397 | return [] #As in #11 398 | if 'm3u8' in raw_url: 399 | logging.info('Found m3u8, processing...') 400 | return process_m3u8(raw_url) 401 | return [raw_url] 402 | 403 | #---------------------------------------------------------------------- 404 | def find_video_address_force_original(cid, header): 405 | """str,str->str 406 | Give the original URL, if possible. 407 | Method #2.""" 408 | # Force get oriurl 409 | #sign_this = calc_sign('appkey={APPKEY}&cid={cid}{SECRETKEY}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY = SECRETKEY)) 410 | api_url = 'http://interface.bilibili.com/player?' 411 | #data = send_request(api_url + 'appkey={APPKEY}&cid={cid}&sign={sign_this}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY = SECRETKEY, sign_this = sign_this), header, IS_FAKE_IP) 412 | data = send_request(api_url + 'appkey={APPKEY}&cid={cid}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY = SECRETKEY), header, IS_FAKE_IP) 413 | #request = urllib2.Request(api_url + 'appkey={APPKEY}&cid={cid}&sign={sign_this}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY = SECRETKEY, sign_this = sign_this), headers=header) 414 | #response = urllib2.urlopen(request) 415 | #data = response.read() 416 | #logging.debug('interface responce: ' + data) 417 | data = data.split('\n') 418 | for l in data: 419 | if 'oriurl' in l: 420 | originalurl = str(l[8:-9]) 421 | logging.info('Original URL is ' + originalurl) 422 | return originalurl 423 | logging.warning('Cannot get original URL! Chances are it does not exist.') 424 | return '' 425 | 426 | #---------------------------------------------------------------------- 427 | def find_link_flvcd(videourl): 428 | """str->list 429 | Used in method 2 and 5.""" 430 | logging.info('Finding link via Flvcd...') 431 | data = send_request('http://www.flvcd.com/parse.php?' + urllib.urlencode([('kw', videourl)]) + '&format=super', FAKE_HEADER, IS_FAKE_IP) 432 | 433 | #request = urllib2.Request('http://www.flvcd.com/parse.php?' + 434 | #urllib.urlencode([('kw', videourl)]) + '&format=super', headers=FAKE_HEADER) 435 | #request.add_header('Accept-encoding', 'gzip') 436 | #response = urllib2.urlopen(request) 437 | #data = response.read() 438 | #if response.info().get('Content-Encoding') == 'gzip': 439 | #buf = StringIO(data) 440 | #f = gzip.GzipFile(fileobj=buf) 441 | #data = f.read() 442 | data_list = data.split('\n') 443 | #logging.debug(data) 444 | for items in data_list: 445 | if 'name' in items and 'inf' in items and 'input' in items: 446 | c = items 447 | rawurlflvcd = c[59:-5] 448 | rawurlflvcd = rawurlflvcd.split('|') 449 | return rawurlflvcd 450 | 451 | #---------------------------------------------------------------------- 452 | def find_video_address_pr(cid, quality, header): 453 | """str,str->list 454 | The API provided by BilibiliPr.""" 455 | logging.info('Finding link via BilibiliPr...') 456 | api_url = 'http://pr.lolly.cc/P{quality}?cid={cid}'.format(quality = quality, cid = cid) 457 | data = send_request(api_url, header, IS_FAKE_IP) 458 | 459 | #request = urllib2.Request(api_url, headers=header) 460 | #try: 461 | #response = urllib2.urlopen(request, timeout=3) 462 | #data = response.read() 463 | #except Exception: 464 | #logging.warning('No response!') 465 | #return ['ERROR'] 466 | #logging.debug('BilibiliPr API: ' + data) 467 | if '!' in data[0:2]: 468 | logging.warning('API returned 404!') 469 | return ['ERROR'] 470 | else: 471 | rawurl = [] 472 | originalurl = '' 473 | dom = parseString(data) 474 | for node in dom.getElementsByTagName('durl'): 475 | url = node.getElementsByTagName('url')[0] 476 | rawurl.append(url.childNodes[0].data) 477 | return rawurl 478 | 479 | #---------------------------------------------------------------------- 480 | def find_video_address_normal_api(cid, header, method, convert_m3u = False): 481 | """str,str,str->list 482 | Change in 0.98: Return the file list directly. 483 | Method: 484 | 0: Original API 485 | 1: CDN API 486 | 2: Original URL API - Divided in another function 487 | 3: Mobile API - Divided in another function 488 | 4: Flvcd - Divided in another function 489 | 5: BilibiliPr 490 | [(VIDEO_URL, TIME_IN_SEC), ...] 491 | """ 492 | if method == '1': 493 | api_url = 'http://interface.bilibili.com/v_cdn_play?' 494 | else: #Method 0 or other 495 | api_url = 'http://interface.bilibili.com/playurl?' 496 | if QUALITY == -1: 497 | sign_this = calc_sign('cid={cid}&from=miniplay&player=1{SECRETKEY_MINILOADER}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY_MINILOADER = SECRETKEY_MINILOADER)) 498 | interface_url = api_url + 'cid={cid}&from=miniplay&player=1&sign={sign_this}'.format(cid = cid, sign_this = sign_this) 499 | #interface_url = api_url + 'appkey={APPKEY}&cid={cid}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY = SECRETKEY) 500 | else: 501 | sign_this = calc_sign('cid={cid}&from=miniplay&player=1&quality={QUALITY}{SECRETKEY_MINILOADER}'.format(APPKEY = APPKEY, cid = cid, SECRETKEY_MINILOADER = SECRETKEY_MINILOADER, QUALITY = QUALITY)) 502 | interface_url = api_url + 'cid={cid}&from=miniplay&player=1&quality={QUALITY}&sign={sign_this}'.format(cid = cid, sign_this = sign_this, QUALITY = QUALITY) 503 | 504 | logging.info(interface_url) 505 | data = send_request(interface_url, header, IS_FAKE_IP) 506 | #request = urllib2.Request(interface_url, headers=header) 507 | #logging.debug('Interface: ' + interface_url) 508 | #response = urllib2.urlopen(request) 509 | #data = response.read() 510 | #logging.debug('interface API: ' + data) 511 | for l in data.split('\n'): # In case shit happens 512 | if 'error.mp4' in l or 'copyright.mp4' in l: 513 | logging.warning('API header may be blocked!') 514 | return ['API_BLOCKED'] 515 | rawurl = [] 516 | originalurl = '' 517 | dom = parseString(data) 518 | if convert_m3u: 519 | for node in dom.getElementsByTagName('durl'): 520 | length = node.getElementsByTagName('length')[0] 521 | url = node.getElementsByTagName('url')[0] 522 | rawurl.append((url.childNodes[0].data, int(int(length.childNodes[0].data) / 1000) + 1)) 523 | else: 524 | for node in dom.getElementsByTagName('durl'): 525 | url = node.getElementsByTagName('url')[0] 526 | rawurl.append(url.childNodes[0].data) 527 | return rawurl 528 | 529 | #---------------------------------------------------------------------- 530 | def find_link_you_get(videourl): 531 | """str->list 532 | Extract urls with you-get.""" 533 | command_result = commands.getstatusoutput('you-get -u {videourl}'.format(videourl = videourl)) 534 | logging.debug(command_result) 535 | if command_result[0] != 0: 536 | raise YougetURLException('You-get failed somehow! Raw output:\n\n{output}'.format(output = command_result[1])) 537 | else: 538 | url_list = command_result[1].split('\n') 539 | for k, v in enumerate(url_list): 540 | if v.startswith('http'): 541 | url_list = url_list[k:] 542 | break 543 | #url_list = literal_eval(url_list_str) 544 | logging.debug('URL_LIST:{url_list}'.format(url_list = url_list)) 545 | return list(url_list) 546 | 547 | #---------------------------------------------------------------------- 548 | def get_video(oversea, convert_m3u = False): 549 | """str->list 550 | A full parser for getting video. 551 | convert_m3u: [(URL, time_in_sec)] 552 | else: [url,url]""" 553 | rawurl = [] 554 | if oversea == '2': 555 | raw_link = find_video_address_force_original(cid, BILIGRAB_HEADER) 556 | rawurl = find_link_flvcd(raw_link) 557 | elif oversea == '3': 558 | rawurl = find_video_address_html5(vid, p, BILIGRAB_HEADER) 559 | if rawurl == []: #As in #11 560 | rawurl = find_video_address_html5(vid, p, FAKE_HEADER) 561 | elif oversea == '4': 562 | rawurl = find_link_flvcd(videourl) 563 | elif oversea == '5': 564 | rawurl = find_video_address_pr(cid, 1080, BILIGRAB_HEADER) 565 | if '404' in rawurl[0]: 566 | logging.info('Using lower quality...') 567 | rawurl = find_video_address_pr(cid, 720, BILIGRAB_HEADER) 568 | if '404' in rawurl[0]: 569 | logging.error('Failed!') 570 | rawurl = [] 571 | else: 572 | pass 573 | elif 'ERROR' in rawurl[0]: 574 | logging.info('Wait a little bit...') 575 | time.sleep(5) 576 | rawurl = find_video_address_pr(cid, 1080, BILIGRAB_HEADER) 577 | elif oversea == '6': 578 | raw_link = find_video_address_force_original(cid, BILIGRAB_HEADER) 579 | rawurl = find_link_you_get(raw_link) 580 | else: 581 | rawurl = find_video_address_normal_api(cid, BILIGRAB_HEADER, oversea, convert_m3u) 582 | if 'API_BLOCKED' in rawurl[0]: 583 | logging.warning('API header may be blocked! Using fake one instead...') 584 | rawurl = find_video_address_normal_api(cid, FAKE_HEADER, oversea, convert_m3u) 585 | return rawurl 586 | 587 | #---------------------------------------------------------------------- 588 | def get_resolution(filename, probe_software): 589 | """str,str->list""" 590 | resolution = [] 591 | filename = filename + '.' + VIDEO_FORMAT 592 | try: 593 | if probe_software == 'mediainfo': 594 | resolution = get_resolution_mediainfo(filename) 595 | if probe_software == 'ffprobe': 596 | resolution = get_resolution_ffprobe(filename) 597 | logging.debug('Software: {probe_software}, resolution {resolution}'.format(probe_software = probe_software, resolution = resolution)) 598 | return resolution 599 | except Exception: # magic number 600 | return[1280, 720] 601 | 602 | #---------------------------------------------------------------------- 603 | def get_resolution_mediainfo(filename): 604 | """str->list 605 | [640,360] 606 | path to dimention""" 607 | resolution = str(os.popen('mediainfo \'--Inform=Video;%Width%x%Height%\' "' +filename +'"').read()).strip().split('x') 608 | return [int(resolution[0]), int(resolution[1])] 609 | 610 | #---------------------------------------------------------------------- 611 | def get_resolution_ffprobe(filename): 612 | '''str->list 613 | [640,360]''' 614 | width = '' 615 | height = '' 616 | cmnd = ['ffprobe', '-show_format', '-show_streams', '-pretty', '-loglevel', 'quiet', filename] 617 | p = subprocess.Popen(cmnd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 618 | # print filename 619 | out, err = p.communicate() 620 | if err: 621 | print err 622 | return None 623 | try: 624 | for line in out.split(): 625 | if 'width=' in line: 626 | width = line.split('=')[1] 627 | if 'height=' in line: 628 | height = line.split('=')[1] 629 | except Exception: 630 | return None 631 | # return width + 'x' + height 632 | return [int(width), int(height)] 633 | 634 | #---------------------------------------------------------------------- 635 | def get_url_size(url): 636 | """str->int 637 | Get remote URL size by reading Content-Length. 638 | In bytes.""" 639 | site = urllib.urlopen(url) 640 | meta = site.info() 641 | return int(meta.getheaders("Content-Length")[0]) 642 | 643 | #---------------------------------------------------------------------- 644 | def getvideosize(url, verbose=False): 645 | try: 646 | if url.startswith('http:') or url.startswith('https:'): 647 | ffprobe_command = ['ffprobe', '-icy', '0', '-loglevel', 'repeat+warning' if verbose else 'repeat+error', '-print_format', 'json', '-select_streams', 'v', '-show_format', '-show_streams', '-timeout', '60000000', '-user-agent', BILIGRAB_UA, url] 648 | else: 649 | ffprobe_command = ['ffprobe', '-loglevel', 'repeat+warning' if verbose else 'repeat+error', '-print_format', 'json', '-select_streams', 'v', '-show_streams', url] 650 | logcommand(ffprobe_command) 651 | ffprobe_process = subprocess.Popen(ffprobe_command, stdout=subprocess.PIPE) 652 | try: 653 | ffprobe_output = json.loads(ffprobe_process.communicate()[0].decode('utf-8', 'replace')) 654 | except KeyboardInterrupt: 655 | logging.warning('Cancelling getting video size, press Ctrl-C again to terminate.') 656 | ffprobe_process.terminate() 657 | return 0, 0 658 | width, height, widthxheight, duration, total_bitrate = 0, 0, 0, 0, 0 659 | try: 660 | if dict.get(ffprobe_output, 'format')['duration'] > duration: 661 | duration = dict.get(ffprobe_output, 'format')['duration'] 662 | except Exception: 663 | pass 664 | for stream in dict.get(ffprobe_output, 'streams', []): 665 | try: 666 | if duration == 0 and (dict.get(stream, 'duration') > duration): 667 | duration = dict.get(stream, 'duration') 668 | if dict.get(stream, 'width')*dict.get(stream, 'height') > widthxheight: 669 | width, height = dict.get(stream, 'width'), dict.get(stream, 'height') 670 | if dict.get(stream, 'bit_rate') > total_bitrate: 671 | total_bitrate += int(dict.get(stream, 'bit_rate')) 672 | except Exception: 673 | pass 674 | if duration == 0: 675 | duration = int(get_url_size(url) * 8 / total_bitrate) 676 | return [[int(width), int(height)], int(float(duration))+1] 677 | except Exception as e: 678 | logorraise(e) 679 | return [[0, 0], 0] 680 | 681 | #---------------------------------------------------------------------- 682 | def convert_ass_py3(filename, probe_software, resolution = [0, 0]): 683 | """str,str->None 684 | With danmaku2ass, branch master. 685 | https://github.com/m13253/danmaku2ass/ 686 | Author: @m13253 687 | GPLv3 688 | A simple way to do that. 689 | resolution_str:1920x1080""" 690 | xml_name = os.path.abspath(filename + '.xml') 691 | ass_name = filename + '.ass' 692 | logging.info('Converting danmaku to ASS file with danmaku2ass(main)...') 693 | logging.info('Resolution is %dx%d' % (resolution[0], resolution[1])) 694 | if resolution == [0, 0]: 695 | logging.info('Trying to get resolution...') 696 | resolution = get_resolution(filename, probe_software) 697 | logging.info('Resolution is %dx%d' % (resolution[0], resolution[1])) 698 | if execute_sysencode_cmd('python3 %s/danmaku2ass3.py -o %s -s %dx%d -fs %d -a 0.8 -dm 8 %s' % (LOCATION_DIR, ass_name, resolution[0], resolution[1], int(math.ceil(resolution[1] / 21.6)), xml_name)) == 0: 699 | logging.info('The ASS file should be ready!') 700 | else: 701 | logging.error('''Danmaku2ASS failed. 702 | Head to https://github.com/m13253/danmaku2ass/issues to complain about this.''') 703 | 704 | #---------------------------------------------------------------------- 705 | def convert_ass_py2(filename, probe_software, resolution = [0, 0]): 706 | """str,str->None 707 | With danmaku2ass, branch py2. 708 | https://github.com/m13253/danmaku2ass/tree/py2 709 | Author: @m13253 710 | GPLv3""" 711 | logging.info('Converting danmaku to ASS file with danmaku2ass(py2)...') 712 | xml_name = filename + '.xml' 713 | if resolution == [0, 0]: 714 | logging.info('Trying to get resolution...') 715 | resolution = get_resolution(filename, probe_software) 716 | logging.info('Resolution is {width}x{height}'.format(width = resolution[0], height = resolution[1])) 717 | #convert_ass(xml_name, filename + '.ass', resolution) 718 | try: 719 | Danmaku2ASS(xml_name, filename + '.ass', resolution[0], resolution[1], 720 | font_size = int(math.ceil(resolution[1] / 21.6)), text_opacity=0.8, duration_marquee=8.0) 721 | logging.info('INFO: The ASS file should be ready!') 722 | except Exception as e: 723 | logging.error('''Danmaku2ASS failed: %s 724 | Head to https://github.com/m13253/danmaku2ass/issues to complain about this.'''% e) 725 | logging.debug(traceback.print_exc()) 726 | pass #Or it may stop leaving lots of lines unprocessed 727 | 728 | #---------------------------------------------------------------------- 729 | def download_danmaku(cid, filename): 730 | """str,str,int->None 731 | Download XML file, and convert to ASS(if required) 732 | Used to be in main(), but replaced due to the merge of -m (BiligrabLite). 733 | If danmaku only, will see whether need to export ASS.""" 734 | logging.info('Fetching XML...') 735 | execute_sysencode_cmd('curl -o "{filename}.xml" --compressed http://comment.bilibili.com/{cid}.xml'.format(filename = filename, cid = cid)) 736 | #execute_sysencode_cmd('gzip -d '+cid+'.xml.gz') 737 | logging.info('The XML file, {filename}.xml should be ready...enjoy!'.format(filename = filename.decode("utf-8").encode(sys.stdout.encoding))) 738 | 739 | #---------------------------------------------------------------------- 740 | def logcommand(command_line): 741 | logging.debug('Executing: '+' '.join('\''+i+'\'' if ' ' in i or '&' in i or '"' in i else i for i in command_line)) 742 | 743 | #---------------------------------------------------------------------- 744 | def logorraise(message, debug=False): 745 | if debug: 746 | raise message 747 | else: 748 | logging.error(str(message)) 749 | 750 | ######################################################################## 751 | class DanmakuOnlyException(Exception): 752 | 753 | '''Deal with DanmakuOnly to stop the main() function.''' 754 | #---------------------------------------------------------------------- 755 | 756 | def __init__(self, value): 757 | self.value = value 758 | #---------------------------------------------------------------------- 759 | 760 | def __str__(self): 761 | return repr(self.value) 762 | 763 | ######################################################################## 764 | class Danmaku2Ass2Exception(Exception): 765 | 766 | '''Deal with Danmaku2ASS2 to stop the main() function.''' 767 | #---------------------------------------------------------------------- 768 | 769 | def __init__(self, value): 770 | self.value = value 771 | #---------------------------------------------------------------------- 772 | 773 | def __str__(self): 774 | return repr(self.value) 775 | 776 | ######################################################################## 777 | class NoCidException(Exception): 778 | 779 | '''Deal with no cid to stop the main() function.''' 780 | #---------------------------------------------------------------------- 781 | 782 | def __init__(self, value): 783 | self.value = value 784 | #---------------------------------------------------------------------- 785 | 786 | def __str__(self): 787 | return repr(self.value) 788 | 789 | ######################################################################## 790 | class NoVideoURLException(Exception): 791 | 792 | '''Deal with no video URL to stop the main() function.''' 793 | #---------------------------------------------------------------------- 794 | 795 | def __init__(self, value): 796 | self.value = value 797 | #---------------------------------------------------------------------- 798 | 799 | def __str__(self): 800 | return repr(self.value) 801 | 802 | ######################################################################## 803 | class ExportM3UException(Exception): 804 | 805 | '''Deal with export to m3u to stop the main() function.''' 806 | #---------------------------------------------------------------------- 807 | 808 | def __init__(self, value): 809 | self.value = value 810 | #---------------------------------------------------------------------- 811 | 812 | def __str__(self): 813 | return repr(self.value) 814 | 815 | ######################################################################## 816 | class YougetURLException(Exception): 817 | 818 | '''you-get cannot get URL somehow''' 819 | #---------------------------------------------------------------------- 820 | 821 | def __init__(self, value): 822 | self.value = value 823 | #---------------------------------------------------------------------- 824 | 825 | def __str__(self): 826 | return repr(self.value) 827 | 828 | ######################################################################## 829 | class URLOpenException(Exception): 830 | 831 | '''cannot get URL somehow''' 832 | #---------------------------------------------------------------------- 833 | 834 | def __init__(self, value): 835 | self.value = value 836 | #---------------------------------------------------------------------- 837 | 838 | def __str__(self): 839 | return repr(self.value) 840 | 841 | 842 | ######################################################################## 843 | class DownloadVideo(threading.Thread): 844 | """Threaded Download Video""" 845 | #---------------------------------------------------------------------- 846 | def __init__(self, queue): 847 | threading.Thread.__init__(self) 848 | self.queue = queue 849 | #---------------------------------------------------------------------- 850 | def run(self): 851 | while True: 852 | #grabs start time from queue 853 | down_set = self.queue.get() 854 | #return_value = download_video(down_set) 855 | cmd = download_video_link(*down_set) 856 | return_value = execute_cmd(cmd) 857 | self.queue.task_done() 858 | 859 | #---------------------------------------------------------------------- 860 | def main_threading(download_thread = 3, video_list = [], thread_single_download = 16): 861 | """""" 862 | command_pool = [(video_list.index(url_this), download_software, url_this, thread_single_download) for url_this in video_list] 863 | #spawn a pool of threads, and pass them queue instance 864 | for i in range(int(download_thread)): 865 | t = DownloadVideo(queue) 866 | t.setDaemon(True) 867 | t.start() 868 | #populate queue with data 869 | for command_single in command_pool: 870 | queue.put(command_single) 871 | #wait on the queue until everything has been processed 872 | queue.join() 873 | 874 | #---------------------------------------------------------------------- 875 | def main(vid, p, oversea, cookies, download_software, concat_software, is_export, probe_software, danmaku_only, time_fetch=5, download_thread= 16, thread_single_download= 16): 876 | global cid, partname, title, videourl, is_first_run 877 | videourl = 'http://www.bilibili.com/video/av{vid}/index_{p}.html'.format(vid = vid, p = p) 878 | # Check both software 879 | logging.debug(concat_software + ', ' + download_software) 880 | # Start to find cid, api 881 | cid, partname, title, pages = find_cid_api(vid, p, cookies) 882 | #if cid is 0: 883 | #logging.warning('Cannot find cid, trying to do it brutely...') 884 | #find_cid_flvcd(videourl) 885 | if cid is 0: 886 | if IS_SLIENT == 0: 887 | logging.warning('Strange, still cannot find cid... ') 888 | is_black3 = str(raw_input('Type y for trying the unpredictable way, or input the cid by yourself; Press ENTER to quit.')) 889 | else: 890 | is_black3 = 'y' 891 | if 'y' in str(is_black3): 892 | vid = str(int(vid) - 1) 893 | p = 1 894 | find_cid_api(int(vid) - 1, p) 895 | cid = cid + 1 896 | elif str(is_black3) is '': 897 | raise NoCidException('FATAL: Cannot get cid anyway!') 898 | else: 899 | cid = str(is_black3) 900 | # start to make folders... 901 | if title is not '': 902 | folder = title 903 | else: 904 | folder = cid 905 | if len(partname) is not 0: 906 | filename = partname 907 | elif title is not '': 908 | filename = title 909 | else: 910 | filename = cid 911 | #In case cannot find which s which 912 | filename = str(p) + ' - ' + filename 913 | # In case make too much folders 914 | folder_to_make = os.getcwd() + '/' + folder 915 | if is_first_run == 0: 916 | if not os.path.exists(folder_to_make): 917 | os.makedirs(folder_to_make) 918 | is_first_run = 1 919 | os.chdir(folder_to_make) 920 | # Download Danmaku 921 | download_danmaku(cid, filename) 922 | if is_export >= 1 and IS_M3U != 1 and danmaku_only == 1: 923 | rawurl = get_video(oversea, convert_m3u = True) 924 | check_dependencies_remote_resolution('ffprobe') 925 | resolution = getvideosize(rawurl[0])[0] 926 | convert_ass(filename, probe_software, resolution = resolution) 927 | if IS_M3U == 1: 928 | rawurl = [] 929 | #M3U export, then stop 930 | if oversea in {'0', '1'}: 931 | rawurl = get_video(oversea, convert_m3u = True) 932 | else: 933 | duration_list = [] 934 | rawurl = get_video(oversea, convert_m3u = False) 935 | for url in rawurl: 936 | duration_list.append(getvideosize(url)[1]) 937 | rawurl = map(lambda x,y: (x, y), rawurl, duration_list) 938 | #print(rawurl) 939 | resolution = getvideosize(rawurl[0][0])[0] 940 | m3u_file = make_m3u8(rawurl) 941 | f = open(filename + '.m3u', 'w') 942 | cwd = os.getcwd() 943 | m3u_file = m3u_file.encode("utf8") 944 | f.write(m3u_file) 945 | f.close() 946 | convert_ass(filename, probe_software, resolution = resolution) 947 | logging.debug(m3u_file) 948 | raise ExportM3UException('INFO: Export to M3U') 949 | if danmaku_only == 1: 950 | raise DanmakuOnlyException('INFO: Danmaku only') 951 | # Find video location 952 | logging.info('Finding video location...') 953 | # try api 954 | # flvcd 955 | url_flag = 1 956 | rawurl = [] 957 | logging.info('Trying to get download URL...') 958 | rawurl = get_video(oversea, convert_m3u = False) 959 | if len(rawurl) == 0 and oversea != '4': # hope this never happen 960 | logging.warning('API failed, using falloff plan...') 961 | rawurl = find_link_flvcd(videourl) 962 | vid_num = len(rawurl) 963 | if IS_SLIENT == 0 and vid_num == 0: 964 | logging.warning('Cannot get download URL!') 965 | rawurl = list(str(raw_input('If you know the url, please enter it now: URL1|URL2...'))).split('|') 966 | vid_num = len(rawurl) 967 | if vid_num is 0: # shit really hit the fan 968 | raise NoVIdeoURLException('FATAL: Cannot get video URL anyway!') 969 | logging.info('{vid_num} videos in part {part_now} to download, fetch yourself a cup of coffee...'.format(vid_num = vid_num, part_now = part_now)) 970 | #Multi thread 971 | if len(rawurl) == 1: 972 | cmd = download_video_link(0,download_software,rawurl[0], thread_single_download) 973 | execute_sysencode_cmd(cmd) 974 | else: 975 | global queue 976 | queue = Queue.Queue() 977 | main_threading(download_thread, rawurl, thread_single_download) 978 | queue.join() 979 | concat_videos(concat_software, vid_num, filename) 980 | if is_export >= 1: 981 | try: 982 | convert_ass(filename, probe_software) 983 | except Exception: 984 | logging.warning('Problem with ASS conversion!') 985 | pass 986 | logging.info('Part Done!') 987 | 988 | #---------------------------------------------------------------------- 989 | def get_full_p(p_raw): 990 | """str->list""" 991 | p_list = [] 992 | p_raw = p_raw.split(',') 993 | for item in p_raw: 994 | if '~' in item: 995 | # print(item) 996 | lower = 0 997 | higher = 0 998 | item = item.split('~') 999 | part_now = '0' 1000 | try: 1001 | lower = int(item[0]) 1002 | except Exception: 1003 | logging.warning('Cannot read lower!') 1004 | try: 1005 | higher = int(item[1]) 1006 | except Exception: 1007 | logging.warning('Cannot read higher!') 1008 | if lower == 0 or higher == 0: 1009 | if lower == 0 and higher != 0: 1010 | lower = higher 1011 | elif lower != 0 and higher == 0: 1012 | higher = lower 1013 | else: 1014 | logging.warning('Cannot find any higher or lower, ignoring...') 1015 | # break 1016 | mid = 0 1017 | if higher < lower: 1018 | mid = higher 1019 | higher = lower 1020 | lower = mid 1021 | p_list.append(lower) 1022 | while lower < higher: 1023 | lower = lower + 1 1024 | p_list.append(lower) 1025 | # break 1026 | else: 1027 | try: 1028 | p_list.append(int(item)) 1029 | except Exception: 1030 | logging.warning('Cannot read "{item}", abandon it.'.format(item = item)) 1031 | # break 1032 | p_list = list_del_repeat(p_list) 1033 | return p_list 1034 | 1035 | #---------------------------------------------------------------------- 1036 | def check_dependencies_remote_resolution(software): 1037 | """""" 1038 | if 'ffprobe' in software: 1039 | output = commands.getstatusoutput('ffprobe --help') 1040 | if str(output[0]) == '32512': 1041 | FFPROBE_USABLE = 0 1042 | else: 1043 | FFPROBE_USABLE = 1 1044 | 1045 | #---------------------------------------------------------------------- 1046 | def check_dependencies_exportm3u(IS_M3U): 1047 | """int,str->int,str""" 1048 | if IS_M3U == 1: 1049 | output = commands.getstatusoutput('ffprobe --help') 1050 | if str(output[0]) == '32512': 1051 | logging.error('ffprobe DNE, python3 does not exist or not callable!') 1052 | err_input = str(raw_input('Do you want to exit, ignore or stop the conversion?(e/i/s)')) 1053 | if err_input == 'e': 1054 | exit() 1055 | elif err_input == '2': 1056 | FFPROBE_USABLE = 0 1057 | elif err_input == 's': 1058 | IS_M3U = 0 1059 | else: 1060 | logging.warning('Cannot read input, stop the conversion!') 1061 | IS_M3U = 0 1062 | else: 1063 | FFPROBE_USABLE = 1 1064 | return IS_M3U 1065 | 1066 | #---------------------------------------------------------------------- 1067 | def check_dependencies_danmaku2ass(is_export): 1068 | """int,str->int,str""" 1069 | if is_export == 3: 1070 | convert_ass = convert_ass_py3 1071 | output = commands.getstatusoutput('python3 --help') 1072 | if str(output[0]) == '32512' or not os.path.exists(os.path.join(LOCATION_DIR, 'danmaku2ass3.py')): 1073 | logging.warning('danmaku2ass3.py DNE, python3 does not exist or not callable!') 1074 | err_input = str(raw_input('Do you want to exit, use Python 2.x or stop the conversion?(e/2/s)')) 1075 | if err_input == 'e': 1076 | exit() 1077 | elif err_input == '2': 1078 | convert_ass = convert_ass_py2 1079 | is_export = 2 1080 | elif err_input == 's': 1081 | is_export = 0 1082 | else: 1083 | logging.warning('Cannot read input, stop the conversion!') 1084 | is_export = 0 1085 | elif is_export == 2 or is_export == 1: 1086 | convert_ass = convert_ass_py2 1087 | if not os.path.exists(os.path.join(LOCATION_DIR, 'danmaku2ass2.py')): 1088 | logging.warning('danmaku2ass2.py DNE!') 1089 | err_input = str(raw_input('Do you want to exit, use Python 3.x or stop the conversion?(e/3/s)')) 1090 | if err_input == 'e': 1091 | exit() 1092 | elif err_input == '3': 1093 | convert_ass = convert_ass_py3 1094 | is_export = 3 1095 | elif err_input == 's': 1096 | is_export = 0 1097 | else: 1098 | logging.warning('Cannot read input, stop the conversion!') 1099 | is_export = 0 1100 | else: 1101 | convert_ass = convert_ass_py2 1102 | return is_export, convert_ass 1103 | 1104 | #---------------------------------------------------------------------- 1105 | def usage(): 1106 | """""" 1107 | print(''' 1108 | Biligrab 1109 | 1110 | https://github.com/cnbeining/Biligrab 1111 | http://www.cnbeining.com/ 1112 | 1113 | Beining@ACICFG 1114 | 1115 | 1116 | 1117 | Usage: 1118 | 1119 | python biligrab.py (-h) (-a) (-p) (-s) (-c) (-d) (-v) (-l) (-e) (-b) (-m) (-n) (-u) (-t) (-q) (-r) (-g) 1120 | 1121 | -h: Default: None 1122 | Print this usage file. 1123 | 1124 | -a: Default: None 1125 | The av number. 1126 | If not set, Biligrab will use the fallback interactive mode. 1127 | Support "~", "," and mix use. 1128 | Examples: 1129 | Input Output 1130 | 1 [1] 1131 | 1,2 [1, 2] 1132 | 1~3 [1, 2, 3] 1133 | 1,2~3 [1, 2, 3] 1134 | 1135 | -p: Default: 0 1136 | The part number. 1137 | Able to use the same syntax as "-a". 1138 | If set to 0, Biligrab will download all the available parts in the video. 1139 | 1140 | -s: Default: 0 1141 | Source to download. 1142 | 0: The original API source, can be Letv backup, 1143 | and can fail if the original video is not available(e.g., deleted) 1144 | 1: The CDN API source, "oversea accelerate". 1145 | Can be MINICDN backup in Mainland China or oversea. 1146 | Good to bypass some bangumi's restrictions. 1147 | 2: Force to use the original source. 1148 | Use Flvcd to parse the video, but would fail if 1149 | 1) The original source DNE, e.g., some old videos 1150 | 2) The original source is Letvcloud itself. 1151 | 3) Other unknown reason(s) that stops Flvcd from parsing the video. 1152 | For any video that failed to parse, Biligrab will try to use Flvcd. 1153 | (Mainly for oversea users regarding to copyright-restricted bangumies.) 1154 | If the API is blocked, Biligrab would fake the UA. 1155 | 3: (Not stable) Use the HTML5 API. 1156 | This works for downloading some cached Letvcloud videos, but is slow, and would fail for no reason sometimes. 1157 | Will retry if unavailable. 1158 | 4: Use Flvcd. 1159 | Good to fight with oversea and copyright restriction, but not working with iQiyi. 1160 | May retrieve better quality video, especially for Youku. 1161 | 5: Use BilibiliPr. 1162 | Good to fight with some copyright restriction that BilibiliPr can fix. 1163 | Not always working though. 1164 | 6: Use You-get (https://github.com/soimort/you-get). 1165 | You need a you-get callable directly like "you-get -u blahblah". 1166 | 1167 | -c: Default: ./bilicookies 1168 | The path of cookies. 1169 | Use cookies to visit member-only videos. 1170 | 1171 | -d: Default: None 1172 | Set the desired download software. 1173 | Biligrab supports aria2c(16 threads), axel(20 threads), wget and curl by far. 1174 | If not set, Biligrab will detect an available one; 1175 | If none of those is available, Biligrab will quit. 1176 | For more software support, please open an issue at https://github.com/cnbeining/Biligrab/issues/ 1177 | 1178 | -v: Default:None 1179 | Set the desired concatenate software. 1180 | Biligrab supports ffmpeg by far. 1181 | If not set, Biligrab will detect an available one; 1182 | If none of those is available, Biligrab will quit. 1183 | For more software support, please open an issue at https://github.com/cnbeining/Biligrab/issues/ 1184 | Make sure you include a *working* command line example of this software! 1185 | 1186 | -l: Default: INFO 1187 | Dump the log of the output for better debugging. 1188 | Can be set to debug. 1189 | 1190 | -e: Default: 1 1191 | Export Danmaku to ASS file. 1192 | Fulfilled with danmaku2ass(https://github.com/m13253/danmaku2ass/tree/py2), 1193 | Author: @m13253, GPLv3 License. 1194 | *For issue with this function, if you think the problem lies on the danmaku2ass side, 1195 | please open the issue at both projects.* 1196 | If set to 1 or 2, Biligrab will use Danmaku2ass's py2 branch. 1197 | If set to 3, Biligrab will use Danmaku2ass's master branch, which would require 1198 | a python3 callable via 'python3'. 1199 | If python3 not callable or danmaku2ass2/3 DNE, Biligrab will ask for action. 1200 | 1201 | -b: Default: None 1202 | Set the probe software. 1203 | Biligrab supports Mediainfo and FFprobe. 1204 | If not set, Biligrab will detect an available one; 1205 | If none of those is available, Biligrab will quit. 1206 | For more software support, please open an issue at https://github.com/cnbeining/Biligrab/issues/ 1207 | Make sure you include a *working* command line example of this software! 1208 | 1209 | -m: Default: 0 1210 | Only download the danmaku. 1211 | 1212 | -n: Default: 0 1213 | Silent Mode. 1214 | Biligrab will not ask any question. 1215 | 1216 | -u: Default: 0 1217 | Export video link to .m3u file, which can be used with MPlayer, mpc, VLC, etc. 1218 | Biligrab will export a m3u8 instead of downloading any video(s). 1219 | Can be broken with sources other than 0 or 1. 1220 | 1221 | -t: Default: None 1222 | The number of Mylist. 1223 | Biligrab will process all the videos in this list. 1224 | 1225 | -q: Default: 3 1226 | The thread number for downloading. 1227 | Good to fix overhead problem. 1228 | 1229 | -r: Default: -1 1230 | Select video quality. 1231 | Only works with Source 0 or 1. 1232 | Range: 0~4, higher for better quality. 1233 | 1234 | -g: Default: 6 1235 | Threads for downloading every part. 1236 | Works with aria2 and axel. 1237 | 1238 | -i: Default: None 1239 | Fake IP address. 1240 | ''') 1241 | 1242 | #---------------------------------------------------------------------- 1243 | if __name__ == '__main__': 1244 | is_first_run, is_export, danmaku_only, IS_SLIENT, IS_M3U, mylist, time_fetch, download_thread, QUALITY, thread_single_download = 0, 1, 0, 0, 0, 0, 5, 16, -1, 16 1245 | argv_list,av_list = [], [] 1246 | argv_list = sys.argv[1:] 1247 | p_raw, vid, oversea, cookiepath, download_software, concat_software, probe_software, vid_raw, LOG_LEVEL, FAKE_IP, IS_FAKE_IP = '', '', '', '', '', '', '', '', 'INFO', '', 0 1248 | convert_ass = convert_ass_py2 1249 | try: 1250 | opts, args = getopt.getopt(argv_list, "ha:p:s:c:d:v:l:e:b:m:n:u:t:q:r:g:i:", 1251 | ['help', "av=", 'part=', 'source=', 'cookie=', 'download=', 'concat=', 'log=', 'export=', 'probe=', 'danmaku=', 'slient=', 'm3u=', 'mylist=', 'thread=', 'quality=', 'thread_single=', 'fake-ip=']) 1252 | except getopt.GetoptError: 1253 | usage() 1254 | exit() 1255 | for o, a in opts: 1256 | if o in ('-h', '--help'): 1257 | usage() 1258 | exit() 1259 | if o in ('-a', '--av'): 1260 | vid_raw = a 1261 | if o in ('-p', '--part'): 1262 | p_raw = a 1263 | if o in ('-s', '--source'): 1264 | oversea = a 1265 | if o in ('-c', '--cookie'): 1266 | cookiepath = a 1267 | if cookiepath == '': 1268 | logging.warning('No cookie path set, use default: ./bilicookies') 1269 | cookiepath = './bilicookies' 1270 | if o in ('-d', '--download'): 1271 | download_software = a 1272 | if o in ('-v', '--concat'): 1273 | concat_software = a 1274 | if o in ('-l', '--log'): 1275 | try: 1276 | LOG_LEVEL = str(a) 1277 | except Exception: 1278 | LOG_LEVEL = 'INFO' 1279 | if o in ('-e', '--export'): 1280 | is_export = int(a) 1281 | if o in ('-b', '--probe'): 1282 | probe_software = a 1283 | if o in ('-m', '--danmaku'): 1284 | danmaku_only = int(a) 1285 | if o in ('-n', '--slient'): 1286 | IS_SLIENT = int(a) 1287 | if o in ('-u', '--m3u'): 1288 | IS_M3U = int(a) 1289 | if o in ('-t', '--mylist'): 1290 | mylist = a 1291 | if o in ('-q', '--thread'): 1292 | download_thread = int(a) 1293 | if o in ('-r', '--quality'): 1294 | QUALITY = int(a) 1295 | if o in ('-g', '--thread_single'): 1296 | thread_single_download = int(a) 1297 | if o in ('-i', '--fake-ip'): 1298 | FAKE_IP = a 1299 | IS_FAKE_IP = 1 1300 | if len(vid_raw) == 0: 1301 | vid_raw = str(raw_input('av')) 1302 | p_raw = str(raw_input('P')) 1303 | oversea = str(raw_input('Source?')) 1304 | cookiepath = './bilicookies' 1305 | logging.basicConfig(level = logging_level_reader(LOG_LEVEL)) 1306 | logging.debug('FAKE IP: ' + str(IS_FAKE_IP) + ' ' + FAKE_IP) 1307 | av_list = get_full_p(vid_raw) 1308 | if mylist != 0: 1309 | av_list += mylist_to_aid_list(mylist) 1310 | logging.debug('av_list') 1311 | if len(cookiepath) == 0: 1312 | cookiepath = './bilicookies' 1313 | if len(p_raw) == 0: 1314 | logging.info('No part number set, download all the parts.') 1315 | p_raw = '0' 1316 | if len(oversea) == 0: 1317 | oversea = '0' 1318 | logging.info('Oversea not set, use original API(methon 0).') 1319 | IS_M3U = check_dependencies_exportm3u(IS_M3U) 1320 | if IS_M3U == 1 and oversea not in {'0', '1'}: 1321 | # See issue #8 1322 | logging.info('M3U exporting with source other than 0 or 1 can be broken, and lead to wrong duration!') 1323 | if IS_SLIENT == 0: 1324 | input_raw = str(raw_input('Enter "q" to quit, or enter the source you want.')) 1325 | if input_raw == 'q': 1326 | exit() 1327 | else: 1328 | oversea = input_raw 1329 | concat_software, download_software, probe_software = check_dependencies(download_software, concat_software, probe_software) 1330 | p_list = get_full_p(p_raw) 1331 | if len(av_list) > 1 and len(p_list) > 1: 1332 | logging.warning('You are downloading multi parts from multiple videos! This may result in unpredictable outputs!') 1333 | if IS_SLIENT == 0: 1334 | input_raw = str(raw_input('Enter "y" to continue, "n" to only download the first part, "q" to quit, or enter the part number you want.')) 1335 | if input_raw == 'y': 1336 | pass 1337 | elif input_raw == 'n': 1338 | p_list = ['1'] 1339 | elif input_raw == 'q': 1340 | exit() 1341 | else: 1342 | p_list = get_full_p(input_raw) 1343 | cookies = read_cookie(cookiepath) 1344 | global BILIGRAB_HEADER, BILIGRAB_UA 1345 | # deal with danmaku2ass's drama / Twice in case someone failed to check dependencies 1346 | is_export, convert_ass = check_dependencies_danmaku2ass(is_export) 1347 | is_export, convert_ass = check_dependencies_danmaku2ass(is_export) 1348 | python_ver_str = '.'.join([str(i) for i in sys.version_info[:2]]) 1349 | BILIGRAB_UA = 'Biligrab/{VER} (cnbeining@gmail.com) (Python-urllib/{python_ver_str}, like libcurl/1.0 NSS-Mozilla/2.0)'.format(VER = VER, python_ver_str = python_ver_str) 1350 | 1351 | #BILIGRAB_UA = 'Biligrab / ' + str(VER) + ' (cnbeining@gmail.com) (like )' 1352 | BILIGRAB_HEADER = {'User-Agent': BILIGRAB_UA, 'Cache-Control': 'no-cache', 'Pragma': 'no-cache', 'Cookie': cookies[0]} 1353 | if LOG_LEVEL == 'DEBUG': 1354 | logging.debug('!!!!!!!!!!!!!!!!!!!!!!!\nWARNING: This log contains some sensitive data. You may want to delete some part of the data before you post it publicly!\n!!!!!!!!!!!!!!!!!!!!!!!') 1355 | logging.debug('BILIGRAB_HEADER') 1356 | try: 1357 | request = urllib2.Request('http://ipinfo.io/json', headers=FAKE_HEADER) 1358 | response = urllib2.urlopen(request) 1359 | data = response.read() 1360 | print('!!!!!!!!!!!!!!!!!!!!!!!\nWARNING: This log contains some sensitive data. You may want to delete some part of the data before you post it publicly!\n!!!!!!!!!!!!!!!!!!!!!!!') 1361 | print('=======================DUMP DATA==================') 1362 | print(data) 1363 | print('========================DATA END==================') 1364 | print('DEBUG: ' + str(av_list)) 1365 | except Exception: 1366 | print('WARNING: Cannot connect to IP-geo database server!') 1367 | pass 1368 | for av in av_list: 1369 | vid = str(av) 1370 | if str(p_raw) == '0': 1371 | logging.info('You are downloading all the parts in this video...') 1372 | try: 1373 | p_raw = str('1~' + find_cid_api(vid, p_raw, cookies)[3]) 1374 | p_list = get_full_p(p_raw) 1375 | except Exception: 1376 | logging.info('Error when reading all the parts!') 1377 | if IS_SLIENT == 0: 1378 | input_raw = str(raw_input('Enter the part number you want, or "q" to quit.')) 1379 | if input_raw == '0': 1380 | print('ERROR: Cannot use all the parts!') 1381 | exit() 1382 | elif input_raw == 'q': 1383 | exit() 1384 | else: 1385 | p_list = get_full_p(input_raw) 1386 | else: 1387 | logging.info('Download the first part of the video...') 1388 | p_raw = '1' 1389 | p_list = [1] 1390 | logging.info('Your target download is av{vid}, part {p_raw}, from source {oversea}'.format(vid = vid, p_raw = p_raw, oversea = oversea)) 1391 | for p in p_list: 1392 | reload(sys) 1393 | sys.setdefaultencoding('utf-8') 1394 | part_now = str(p) 1395 | try: 1396 | logging.info('Downloading part {p} ...'.format(p = p)) 1397 | main(vid, p, oversea, cookies, download_software, concat_software, is_export, probe_software, danmaku_only, time_fetch, download_thread, thread_single_download) 1398 | except DanmakuOnlyException: 1399 | pass 1400 | except ExportM3UException: 1401 | pass 1402 | except Exception as e: 1403 | print('ERROR: Biligrab failed: %s' % e) 1404 | print(' If you think this should not happen, please dump your log using "-l", and open a issue at https://github.com/cnbeining/Biligrab/issues .') 1405 | print(' Make sure you delete all the sensitive data before you post it publicly.') 1406 | traceback.print_exc() 1407 | exit() -------------------------------------------------------------------------------- /biligrablite.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Biligrab Lite 0.21 3 | Beining@ACICFG 4 | cnbeining[at]gmail.com 5 | http://www.cnbeining.com 6 | MIT licence 7 | ''' 8 | 9 | import sys 10 | import os 11 | from StringIO import StringIO 12 | import gzip 13 | import urllib2 14 | import hashlib 15 | 16 | from xml.dom.minidom import parseString 17 | 18 | global vid 19 | global cid 20 | global partname 21 | global title 22 | global videourl 23 | global part_now 24 | 25 | global appkey 26 | global secretkey 27 | appkey='c1b107428d337928'; 28 | secretkey = 'ea85624dfcf12d7cc7b2b3a94fac1f2c' 29 | 30 | def list_del_repeat(list): 31 | """delete repeating items in a list, and keep the order. 32 | http://www.cnblogs.com/infim/archive/2011/03/10/1979615.html""" 33 | l2 = [] 34 | [l2.append(i) for i in list if not i in l2] 35 | return(l2) 36 | 37 | #---------------------------------------------------------------------- 38 | def find_cid_api(vid, p): 39 | """find cid and print video detail""" 40 | global cid 41 | global partname 42 | global title 43 | global videourl 44 | cookiepath = './bilicookies' 45 | try: 46 | cookies = open(cookiepath, 'r').readline() 47 | #print(cookies) 48 | except: 49 | print('Cannot read cookie, may affect some videos...') 50 | cookies = '' 51 | cid = 0 52 | title = '' 53 | partname = '' 54 | if str(p) is '0' or str(p) is '1': 55 | str2Hash = 'appkey=85eb6835b0a1034e&id=' + str(vid) + '&type=xml2ad42749773c441109bdc0191257a664' 56 | sign_this = hashlib.md5(str2Hash.encode('utf-8')).hexdigest() 57 | biliurl = 'https://api.bilibili.com/view?appkey=85eb6835b0a1034e&id=' + str(vid) + '&type=xml&sign=' + sign_this 58 | else: 59 | str2Hash = 'appkey=85eb6835b0a1034e&id=' + str(vid) + '&page=' + str(p) + '&type=xml2ad42749773c441109bdc0191257a664' 60 | sign_this = hashlib.md5(str2Hash.encode('utf-8')).hexdigest() 61 | biliurl = 'https://api.bilibili.com/view?appkey=85eb6835b0a1034e&id=' + str(vid) + '&page=' + str(p) + '&type=xml&sign=' + sign_this 62 | #print(biliurl) 63 | videourl = 'http://www.bilibili.tv/video/av'+ str(vid)+'/index_'+ str(p)+'.html' 64 | print('Fetching webpage...') 65 | print(biliurl) 66 | try: 67 | request = urllib2.Request(biliurl, headers={ 'User-Agent' : 'Biligrab /0.8 (cnbeining@gmail.com)', 'Cache-Control': 'no-cache', 'Pragma': 'no-cache' , 'Cookie': cookies}) 68 | response = urllib2.urlopen(request) 69 | data = response.read() 70 | dom = parseString(data) 71 | for node in dom.getElementsByTagName('cid'): 72 | if node.parentNode.tagName == "info": 73 | cid = node.toxml()[5:-6] 74 | print('cid is ' + cid) 75 | break 76 | for node in dom.getElementsByTagName('partname'): 77 | if node.parentNode.tagName == "info": 78 | partname = node.toxml()[10:-11].strip() 79 | print('partname is ' + partname) 80 | break 81 | for node in dom.getElementsByTagName('title'): 82 | if node.parentNode.tagName == "info": 83 | title = node.toxml()[7:-8].strip() 84 | print('Title is ' + title.decode("utf-8")) 85 | except: #If API failed 86 | print('ERROR: Cannot connect to API server!') 87 | 88 | 89 | #---------------------------------------------------------------------- 90 | def find_cid_flvcd(videourl): 91 | """""" 92 | global vid 93 | global cid 94 | global partname 95 | global title 96 | print('Fetching webpage via Flvcd...') 97 | request = urllib2.Request(videourl, headers={ 'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36', 'Cache-Control': 'no-cache', 'Pragma': 'no-cache' }) 98 | request.add_header('Accept-encoding', 'gzip') 99 | response = urllib2.urlopen(request) 100 | if response.info().get('Content-Encoding') == 'gzip': 101 | buf = StringIO( response.read()) 102 | f = gzip.GzipFile(fileobj=buf) 103 | data = f.read() 104 | data_list = data.split('\n') 105 | #Todo: read title 106 | for lines in data_list: 107 | if 'cid=' in lines: 108 | cid = lines.split('&') 109 | cid = cid[0].split('=') 110 | cid = cid[-1] 111 | print('cid is ' + str(cid)) 112 | break 113 | 114 | 115 | #---------------------------------------------------------------------- 116 | def main(vid, p, oversea): 117 | global cid 118 | global partname 119 | global title 120 | global videourl 121 | global is_first_run 122 | videourl = 'http://www.bilibili.tv/video/av'+ str(vid)+'/index_'+ str(p)+'.html' 123 | 124 | find_cid_api(vid, p) 125 | global cid 126 | if cid is 0: 127 | print('Cannot find cid, trying to do it brutely...') 128 | find_cid_flvcd(videourl) 129 | 130 | if cid is 0: 131 | is_black3 = str(raw_input('Strange, still cannot find cid... Type y for trying the unpredictable way, or input the cid by yourself, press ENTER to quit.')) 132 | if 'y' in str(is_black3): 133 | vid = vid - 1 134 | p = 1 135 | find_cid_api(vid-1, p) 136 | cid = cid + 1 137 | elif str(is_black3) is '': 138 | print('Cannot get cid anyway! Quit.') 139 | exit() 140 | else: 141 | cid = str(is_black3) 142 | if len(partname) is not 0: 143 | filename = partname 144 | elif title is not '': 145 | filename = title 146 | else: 147 | filename = cid 148 | print('Fetching XML...') 149 | os.system((u'curl -o "'+filename+u'.xml" --compressed http://comment.bilibili.cn/'+cid+u'.xml').encode(sys.stdout.encoding)) 150 | os.system((u'gzip -d '+cid+u'.xml.gz').encode(sys.stdout.encoding)) 151 | print(u'The XML file, ' + filename + u'.xml should be ready...enjoy!') 152 | #try api 153 | # 154 | 155 | 156 | vid = str(raw_input('av')) 157 | p_raw = str(raw_input('P')) 158 | if p_raw == '': 159 | p_raw = '1' 160 | oversea = '0' 161 | 162 | p_list = [] 163 | p_raw = p_raw.split(',') 164 | 165 | for item in p_raw: 166 | if '~' in item: 167 | #print(item) 168 | lower = 0 169 | higher = 0 170 | item = item.split('~') 171 | try: 172 | lower = int(item[0]) 173 | except: 174 | print('Cannot read lower!') 175 | try: 176 | higher = int(item[1]) 177 | except: 178 | print('Cannot read higher!') 179 | if lower == 0 or higher == 0: 180 | if lower == 0 and higher != 0: 181 | lower = higher 182 | elif lower != 0 and higher == 0: 183 | higher = lower 184 | else: 185 | print('Cannot find any higher or lower, ignoring...') 186 | #break 187 | mid = 0 188 | if higher < lower: 189 | mid = higher 190 | higher = lower 191 | lower = mid 192 | p_list.append(lower) 193 | while lower < higher: 194 | lower = lower + 1 195 | p_list.append(lower) 196 | #break 197 | else: 198 | try: 199 | p_list.append(int(item)) 200 | except: 201 | print('Cannot read "'+str(item)+'", abandon it.') 202 | #break 203 | 204 | 205 | p_list = list_del_repeat(p_list) 206 | 207 | global is_first_run 208 | is_first_run = 0 209 | 210 | part_now = '0' 211 | print(p_list) 212 | for p in p_list: 213 | print part_now 214 | part_now = str(p) 215 | main(vid, p, oversea) 216 | exit() 217 | -------------------------------------------------------------------------------- /danmaku2ass2.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | # -*- coding: utf-8 -*- 3 | 4 | # The original author of this program, Danmaku2ASS, is StarBrilliant. 5 | # This file is released under General Public License version 3. 6 | # You should have received a copy of General Public License text alongside with 7 | # this program. If not, you can obtain it at http://gnu.org/copyleft/gpl.html . 8 | # This program comes with no warranty, the author will not be resopnsible for 9 | # any damage or problems caused by this program. 10 | 11 | # You can obtain a latest copy of Danmaku2ASS at: 12 | # https://github.com/m13253/danmaku2ass 13 | # Please update to the latest version before complaining. 14 | 15 | from __future__ import unicode_literals 16 | from __future__ import with_statement 17 | from __future__ import division 18 | import argparse 19 | import calendar 20 | import gettext 21 | import io 22 | import json 23 | import logging 24 | import math 25 | import os 26 | import random 27 | import re 28 | import sys 29 | import time 30 | import xml.dom.minidom 31 | from itertools import imap 32 | from io import open 33 | 34 | 35 | if not ((2, 7) <= sys.version_info < (3,)): 36 | raise RuntimeError(u'this version of Danmaku2ASS only works on Python 2.7, please switch to the original version of Danmaku2ASS') 37 | 38 | bytes, str = str, unicode 39 | 40 | gettext.install('danmaku2ass', os.path.join(os.path.dirname(os.path.abspath(os.path.realpath(sys.argv[0] or 'locale'))), 'locale')) 41 | 42 | 43 | def SeekZero(function): 44 | def decorated_function(file_): 45 | file_.seek(0) 46 | try: 47 | return function(file_) 48 | finally: 49 | file_.seek(0) 50 | return decorated_function 51 | 52 | 53 | def EOFAsNone(function): 54 | def decorated_function(*args, **kwargs): 55 | try: 56 | return function(*args, **kwargs) 57 | except EOFError: 58 | return None 59 | return decorated_function 60 | 61 | 62 | @SeekZero 63 | @EOFAsNone 64 | def ProbeCommentFormat(f): 65 | tmp = f.read(1) 66 | if tmp == '[': 67 | return 'Acfun' 68 | # It is unwise to wrap a JSON object in an array! 69 | # See this: http://haacked.com/archive/2008/11/20/anatomy-of-a-subtle-json-vulnerability.aspx/ 70 | # Do never follow what Acfun developers did! 71 | elif tmp == '{': 72 | tmp = f.read(14) 73 | if tmp == '"status_code":': 74 | return 'Tudou' 75 | elif tmp == '"root":{"total': 76 | return 'sH5V' 77 | elif tmp == '<': 78 | tmp = f.read(1) 79 | if tmp == '?': 80 | tmp = f.read(38) 81 | if tmp == 'xml version="1.0" encoding="UTF-8"?>\n<': 88 | return 'Bilibili' # Komica, with the same file format as Bilibili 89 | elif tmp == 'xml version="1.0" encoding="UTF-8"?>\n<': 90 | return 'MioMio' 91 | elif tmp == 'p': 92 | return 'Niconico' # Himawari Douga, with the same file format as Niconico Douga 93 | 94 | 95 | # 96 | # ReadComments**** protocol 97 | # 98 | # Input: 99 | # f: Input file 100 | # fontsize: Default font size 101 | # 102 | # Output: 103 | # yield a tuple: 104 | # (timeline, timestamp, no, comment, pos, color, size, height, width) 105 | # timeline: The position when the comment is replayed 106 | # timestamp: The UNIX timestamp when the comment is submitted 107 | # no: A sequence of 1, 2, 3, ..., used for sorting 108 | # comment: The content of the comment 109 | # pos: 0 for regular moving comment, 110 | # 1 for bottom centered comment, 111 | # 2 for top centered comment, 112 | # 3 for reversed moving comment 113 | # color: Font color represented in 0xRRGGBB, 114 | # e.g. 0xffffff for white 115 | # size: Font size 116 | # height: The estimated height in pixels 117 | # i.e. (comment.count('\n')+1)*size 118 | # width: The estimated width in pixels 119 | # i.e. CalculateLength(comment)*size 120 | # 121 | # After implementing ReadComments****, make sure to update ProbeCommentFormat 122 | # and CommentFormatMap. 123 | # 124 | 125 | 126 | def ReadCommentsNiconico(f, fontsize): 127 | NiconicoColorMap = {'red': 0xff0000, 'pink': 0xff8080, 'orange': 0xffcc00, 'yellow': 0xffff00, 'green': 0x00ff00, 'cyan': 0x00ffff, 'blue': 0x0000ff, 'purple': 0xc000ff, 'black': 0x000000, 'niconicowhite': 0xcccc99, 'white2': 0xcccc99, 'truered': 0xcc0033, 'red2': 0xcc0033, 'passionorange': 0xff6600, 'orange2': 0xff6600, 'madyellow': 0x999900, 'yellow2': 0x999900, 'elementalgreen': 0x00cc66, 'green2': 0x00cc66, 'marineblue': 0x33ffcc, 'blue2': 0x33ffcc, 'nobleviolet': 0x6633cc, 'purple2': 0x6633cc} 128 | dom = xml.dom.minidom.parseString(f.read().encode('utf-8', 'replace')) 129 | comment_element = dom.getElementsByTagName('chat') 130 | for comment in comment_element: 131 | try: 132 | c = str(comment.childNodes[0].wholeText) 133 | if c.startswith('/'): 134 | continue # ignore advanced comments 135 | pos = 0 136 | color = 0xffffff 137 | size = fontsize 138 | for mailstyle in str(comment.getAttribute('mail')).split(): 139 | if mailstyle == 'ue': 140 | pos = 1 141 | elif mailstyle == 'shita': 142 | pos = 2 143 | elif mailstyle == 'big': 144 | size = fontsize*1.44 145 | elif mailstyle == 'small': 146 | size = fontsize*0.64 147 | elif mailstyle in NiconicoColorMap: 148 | color = NiconicoColorMap[mailstyle] 149 | yield (max(int(comment.getAttribute('vpos')), 0)*0.01, int(comment.getAttribute('date')), int(comment.getAttribute('no')), c, pos, color, size, (c.count('\n')+1)*size, CalculateLength(c)*size) 150 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 151 | logging.warning(_('Invalid comment: %s') % comment.toxml()) 152 | continue 153 | 154 | 155 | def ReadCommentsAcfun(f, fontsize): 156 | comment_element = json.load(f) 157 | for i, comment in enumerate(comment_element): 158 | try: 159 | p = str(comment['c']).split(',') 160 | assert len(p) >= 6 161 | assert p[2] in ('1', '2', '4', '5', '7') 162 | size = int(p[3])*fontsize/25.0 163 | if p[2] != '7': 164 | c = str(comment['m']).replace('\\r', '\n').replace('\r', '\n') 165 | yield (float(p[0]), int(p[5]), i, c, {'1': 0, '2': 0, '4': 2, '5': 1}[p[2]], int(p[1]), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 166 | else: 167 | c = dict(json.loads(comment['m'])) 168 | yield (float(p[0]), int(p[5]), i, c, 'acfunpos', int(p[1]), size, 0, 0) 169 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 170 | logging.warning(_('Invalid comment: %r') % comment) 171 | continue 172 | 173 | 174 | def ReadCommentsBilibili(f, fontsize): 175 | dom = xml.dom.minidom.parseString(f.read().encode('utf-8', 'replace')) 176 | comment_element = dom.getElementsByTagName('d') 177 | for i, comment in enumerate(comment_element): 178 | try: 179 | p = str(comment.getAttribute('p')).split(',') 180 | assert len(p) >= 5 181 | assert p[1] in ('1', '4', '5', '6', '7') 182 | if p[1] != '7': 183 | c = str(comment.childNodes[0].wholeText).replace('/n', '\n') 184 | size = int(p[2])*fontsize/25.0 185 | yield (float(p[0]), int(p[4]), i, c, {'1': 0, '4': 2, '5': 1, '6': 3}[p[1]], int(p[3]), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 186 | else: # positioned comment 187 | c = str(comment.childNodes[0].wholeText) 188 | yield (float(p[0]), int(p[4]), i, c, 'bilipos', int(p[3]), int(p[2]), 0, 0) 189 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 190 | logging.warning(_('Invalid comment: %s') % comment.toxml()) 191 | continue 192 | 193 | 194 | def ReadCommentsTudou(f, fontsize): 195 | comment_element = json.load(f) 196 | for i, comment in enumerate(comment_element['comment_list']): 197 | try: 198 | assert comment['pos'] in (3, 4, 6) 199 | c = str(comment['data']) 200 | assert comment['size'] in (0, 1, 2) 201 | size = {0: 0.64, 1: 1, 2: 1.44}[comment['size']]*fontsize 202 | yield (int(comment['replay_time']*0.001), int(comment['commit_time']), i, c, {3: 0, 4: 2, 6: 1}[comment['pos']], int(comment['color']), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 203 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 204 | logging.warning(_('Invalid comment: %r') % comment) 205 | continue 206 | 207 | 208 | def ReadCommentsMioMio(f, fontsize): 209 | NiconicoColorMap = {'red': 0xff0000, 'pink': 0xff8080, 'orange': 0xffc000, 'yellow': 0xffff00, 'green': 0x00ff00, 'cyan': 0x00ffff, 'blue': 0x0000ff, 'purple': 0xc000ff, 'black': 0x000000} 210 | dom = xml.dom.minidom.parseString(f.read().encode('utf-8', 'replace')) 211 | comment_element = dom.getElementsByTagName('data') 212 | for i, comment in enumerate(comment_element): 213 | try: 214 | message = comment.getElementsByTagName('message')[0] 215 | c = str(message.childNodes[0].wholeText) 216 | pos = 0 217 | size = int(message.getAttribute('fontsize'))*fontsize/25.0 218 | yield (float(comment.getElementsByTagName('playTime')[0].childNodes[0].wholeText), int(calendar.timegm(time.strptime(comment.getElementsByTagName('times')[0].childNodes[0].wholeText, '%Y-%m-%d %H:%M:%S')))-28800, i, c, {'1': 0, '4': 2, '5': 1}[message.getAttribute('mode')], int(message.getAttribute('color')), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 219 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 220 | logging.warning(_('Invalid comment: %s') % comment.toxml()) 221 | continue 222 | 223 | 224 | def ReadCommentsSH5V(f, fontsize): 225 | comment_element = json.load(f) 226 | for i, comment in enumerate(comment_element["root"]["bgs"]): 227 | try: 228 | c_at = str(comment['at']) 229 | c_type = str(comment['type']) 230 | c_date = str(comment['timestamp']) 231 | c_color = str(comment['color']) 232 | c = str(comment['text']) 233 | size = fontsize 234 | if c_type != '7': 235 | yield (float(c_at), int(c_date), i, c, {'0': 0, '1': 0, '4': 2, '5': 1}[c_type], int(c_color[1:], 16), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 236 | else: 237 | c_x = float(comment['x']) 238 | c_y = float(comment['y']) 239 | size = int(comment['size']) 240 | dur = int(comment['dur']) 241 | data1 = float(comment['data1']) 242 | data2 = float(comment['data2']) 243 | data3 = int(comment['data3']) 244 | data4 = int(comment['data4']) 245 | yield (float(c_at), int(c_date), i, c, 'sH5Vpos', int(c_color[1:], 16), size, 0, 0, c_x, c_y, dur, data1, data2, data3, data4) 246 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 247 | logging.warning(_('Invalid comment: %r') % comment) 248 | continue 249 | 250 | 251 | CommentFormatMap = {None: None, 'Niconico': ReadCommentsNiconico, 'Acfun': ReadCommentsAcfun, 'Bilibili': ReadCommentsBilibili, 'Tudou': ReadCommentsTudou, 'MioMio': ReadCommentsMioMio, 'sH5V': ReadCommentsSH5V} 252 | 253 | 254 | def WriteCommentBilibiliPositioned(f, c, width, height, styleid): 255 | #BiliPlayerSize = (512, 384) # Bilibili player version 2010 256 | #BiliPlayerSize = (540, 384) # Bilibili player version 2012 257 | BiliPlayerSize = (672, 438) # Bilibili player version 2014 258 | ZoomFactor = GetZoomFactor(BiliPlayerSize, (width, height)) 259 | 260 | def GetPosition(InputPos, isHeight): 261 | isHeight = int(isHeight) # True -> 1 262 | if isinstance(InputPos, int): 263 | return ZoomFactor[0]*InputPos+ZoomFactor[isHeight+1] 264 | elif isinstance(InputPos, float): 265 | if InputPos > 1: 266 | return ZoomFactor[0]*InputPos+ZoomFactor[isHeight+1] 267 | else: 268 | return BiliPlayerSize[isHeight]*ZoomFactor[0]*InputPos+ZoomFactor[isHeight+1] 269 | else: 270 | try: 271 | InputPos = int(InputPos) 272 | except ValueError: 273 | InputPos = float(InputPos) 274 | return GetPosition(InputPos, isHeight) 275 | 276 | try: 277 | comment_args = safe_list(json.loads(c[3])) 278 | text = ASSEscape(str(comment_args[4]).replace('/n', '\n')) 279 | from_x = comment_args.get(0, 0) 280 | from_y = comment_args.get(1, 0) 281 | to_x = comment_args.get(7, from_x) 282 | to_y = comment_args.get(8, from_y) 283 | from_x = GetPosition(from_x, False) 284 | from_y = GetPosition(from_y, True) 285 | to_x = GetPosition(to_x, False) 286 | to_y = GetPosition(to_y, True) 287 | alpha = safe_list(str(comment_args.get(2, '1')).split('-')) 288 | from_alpha = float(alpha.get(0, 1)) 289 | to_alpha = float(alpha.get(1, from_alpha)) 290 | from_alpha = 255-round(from_alpha*255) 291 | to_alpha = 255-round(to_alpha*255) 292 | rotate_z = int(comment_args.get(5, 0)) 293 | rotate_y = int(comment_args.get(6, 0)) 294 | lifetime = float(comment_args.get(3, 4500)) 295 | duration = int(comment_args.get(9, lifetime*1000)) 296 | delay = int(comment_args.get(10, 0)) 297 | fontface = comment_args.get(12) 298 | isborder = comment_args.get(11, 'true') 299 | from_rotarg = ConvertFlashRotation(rotate_y, rotate_z, from_x, from_y, width, height) 300 | to_rotarg = ConvertFlashRotation(rotate_y, rotate_z, to_x, to_y, width, height) 301 | styles = ['\\org(%d, %d)' % (width/2, height/2)] 302 | if from_rotarg[0:2] == to_rotarg[0:2]: 303 | styles.append('\\pos(%.0f, %.0f)' % (from_rotarg[0:2])) 304 | else: 305 | styles.append('\\move(%.0f, %.0f, %.0f, %.0f, %.0f, %.0f)' % (from_rotarg[0:2]+to_rotarg[0:2]+(delay, delay+duration))) 306 | styles.append('\\frx%.0f\\fry%.0f\\frz%.0f\\fscx%.0f\\fscy%.0f' % (from_rotarg[2:7])) 307 | if (from_x, from_y) != (to_x, to_y): 308 | styles.append('\\t(%d, %d, ' % (delay, delay+duration)) 309 | styles.append('\\frx%.0f\\fry%.0f\\frz%.0f\\fscx%.0f\\fscy%.0f' % (to_rotarg[2:7])) 310 | styles.append(')') 311 | if fontface: 312 | styles.append('\\fn%s' % ASSEscape(fontface)) 313 | styles.append('\\fs%.0f' % (c[6]*ZoomFactor[0])) 314 | if c[5] != 0xffffff: 315 | styles.append('\\c&H%s&' % ConvertColor(c[5])) 316 | if c[5] == 0x000000: 317 | styles.append('\\3c&HFFFFFF&') 318 | if from_alpha == to_alpha: 319 | styles.append('\\alpha&H%02X' % from_alpha) 320 | elif (from_alpha, to_alpha) == (255, 0): 321 | styles.append('\\fad(%.0f,0)' % (lifetime*1000)) 322 | elif (from_alpha, to_alpha) == (0, 255): 323 | styles.append('\\fad(0, %.0f)' % (lifetime*1000)) 324 | else: 325 | styles.append('\\fade(%(from_alpha)d, %(to_alpha)d, %(to_alpha)d, 0, %(end_time).0f, %(end_time).0f, %(end_time).0f)' % {'from_alpha': from_alpha, 'to_alpha': to_alpha, 'end_time': lifetime*1000}) 326 | if isborder == 'false': 327 | styles.append('\\bord0') 328 | f.write('Dialogue: -1,%(start)s,%(end)s,%(styleid)s,,0,0,0,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(c[0]), 'end': ConvertTimestamp(c[0]+lifetime), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 329 | except (IndexError, ValueError), e: 330 | try: 331 | logging.warning(_('Invalid comment: %r') % c[3]) 332 | except IndexError: 333 | logging.warning(_('Invalid comment: %r') % c) 334 | 335 | 336 | def WriteCommentAcfunPositioned(f, c, width, height, styleid): 337 | AcfunPlayerSize = (560, 400) 338 | ZoomFactor = GetZoomFactor(AcfunPlayerSize, (width, height)) 339 | 340 | def GetPosition(InputPos, isHeight): 341 | isHeight = int(isHeight) # True -> 1 342 | return AcfunPlayerSize[isHeight]*ZoomFactor[0]*InputPos*0.001+ZoomFactor[isHeight+1] 343 | 344 | def GetTransformStyles(x=None, y=None, scale_x=None, scale_y=None, rotate_z=None, rotate_y=None, color=None, alpha=None): 345 | styles = [] 346 | out_x, out_y = x, y 347 | if rotate_z is not None and rotate_y is not None: 348 | assert x is not None 349 | assert y is not None 350 | rotarg = ConvertFlashRotation(rotate_y, rotate_z, x, y, width, height) 351 | out_x, out_y = rotarg[0:2] 352 | if scale_x is None: 353 | scale_x = 1 354 | if scale_y is None: 355 | scale_y = 1 356 | styles.append('\\frx%.0f\\fry%.0f\\frz%.0f\\fscx%.0f\\fscy%.0f' % (rotarg[2:5]+(rotarg[5]*scale_x, rotarg[6]*scale_y))) 357 | else: 358 | if scale_x is not None: 359 | styles.append('\\fscx%.0f' % (scale_x*100)) 360 | if scale_y is not None: 361 | styles.append('\\fscy%.0f' % (scale_y*100)) 362 | if color is not None: 363 | styles.append('\\c&H%s&' % ConvertColor(color)) 364 | if color == 0x000000: 365 | styles.append('\\3c&HFFFFFF&') 366 | if alpha is not None: 367 | alpha = 255-round(alpha*255) 368 | styles.append('\\alpha&H%02X' % alpha) 369 | return out_x, out_y, styles 370 | 371 | def FlushCommentLine(f, text, styles, start_time, end_time, styleid): 372 | if end_time > start_time: 373 | f.write('Dialogue: -1,%(start)s,%(end)s,%(styleid)s,,0,0,0,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(start_time), 'end': ConvertTimestamp(end_time), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 374 | 375 | try: 376 | comment_args = c[3] 377 | text = ASSEscape(str(comment_args['n']).replace('\r', '\n')) 378 | common_styles = ['\org(%d, %d)' % (width/2, height/2)] 379 | anchor = {0: 7, 1: 8, 2: 9, 3: 4, 4: 5, 5: 6, 6: 1, 7: 2, 8: 3}.get(comment_args.get('c', 0), 7) 380 | if anchor != 7: 381 | common_styles.append('\\an%s' % anchor) 382 | font = comment_args.get('w') 383 | if font: 384 | font = dict(font) 385 | fontface = font.get('f') 386 | if fontface: 387 | common_styles.append('\\fn%s' % ASSEscape(str(fontface))) 388 | fontbold = bool(font.get('b')) 389 | if fontbold: 390 | common_styles.append('\\b1') 391 | common_styles.append('\\fs%.0f' % (c[6]*ZoomFactor[0])) 392 | isborder = bool(comment_args.get('b', True)) 393 | if not isborder: 394 | common_styles.append('\\bord0') 395 | to_pos = dict(comment_args.get('p', {'x': 0, 'y': 0})) 396 | to_x = round(GetPosition(int(to_pos.get('x', 0)), False)) 397 | to_y = round(GetPosition(int(to_pos.get('y', 0)), True)) 398 | to_scale_x = float(comment_args.get('e', 1.0)) 399 | to_scale_y = float(comment_args.get('f', 1.0)) 400 | to_rotate_z = float(comment_args.get('r', 0.0)) 401 | to_rotate_y = float(comment_args.get('k', 0.0)) 402 | to_color = c[5] 403 | to_alpha = float(comment_args.get('a', 1.0)) 404 | from_time = float(comment_args.get('t', 0.0)) 405 | action_time = float(comment_args.get('l', 3.0)) 406 | actions = list(comment_args.get('z', [])) 407 | to_out_x, to_out_y, transform_styles = GetTransformStyles(to_x, to_y, to_scale_x, to_scale_y, to_rotate_z, to_rotate_y, to_color, to_alpha) 408 | FlushCommentLine(f, text, common_styles+['\\pos(%.0f, %.0f)' % (to_out_x, to_out_y)]+transform_styles, c[0]+from_time, c[0]+from_time+action_time, styleid) 409 | action_styles = transform_styles 410 | for action in actions: 411 | action = dict(action) 412 | from_x, from_y = to_x, to_y 413 | from_out_x, from_out_y = to_out_x, to_out_y 414 | from_scale_x, from_scale_y = to_scale_x, to_scale_y 415 | from_rotate_z, from_rotate_y = to_rotate_z, to_rotate_y 416 | from_color, from_alpha = to_color, to_alpha 417 | transform_styles, action_styles = action_styles, [] 418 | from_time += action_time 419 | action_time = float(action.get('l', 0.0)) 420 | if 'x' in action: 421 | to_x = round(GetPosition(int(action['x']), False)) 422 | if 'y' in action: 423 | to_y = round(GetPosition(int(action['y']), True)) 424 | if 'f' in action: 425 | to_scale_x = float(action['f']) 426 | if 'g' in action: 427 | to_scale_y = float(action['g']) 428 | if 'c' in action: 429 | to_color = int(action['c']) 430 | if 't' in action: 431 | to_alpha = float(action['t']) 432 | if 'd' in action: 433 | to_rotate_z = float(action['d']) 434 | if 'e' in action: 435 | to_rotate_y = float(action['e']) 436 | to_out_x, to_out_y, action_styles = GetTransformStyles(to_x, to_y, from_scale_x, from_scale_y, to_rotate_z, to_rotate_y, from_color, from_alpha) 437 | if (from_out_x, from_out_y) == (to_out_x, to_out_y): 438 | pos_style = '\\pos(%.0f, %.0f)' % (to_out_x, to_out_y) 439 | else: 440 | pos_style = '\\move(%.0f, %.0f, %.0f, %.0f)' % (from_out_x, from_out_y, to_out_x, to_out_y) 441 | styles = common_styles+transform_styles 442 | styles.append(pos_style) 443 | if action_styles: 444 | styles.append('\\t(%s)' % (''.join(action_styles))) 445 | FlushCommentLine(f, text, styles, c[0]+from_time, c[0]+from_time+action_time, styleid) 446 | except (IndexError, ValueError) as e: 447 | logging.warning(_('Invalid comment: %r') % c[3]) 448 | 449 | 450 | def WriteCommentSH5VPositioned(f, c, width, height, styleid): 451 | 452 | def GetTransformStyles(x=None, y=None, fsize=None, rotate_z=None, rotate_y=None, color=None, alpha=None): 453 | styles = [] 454 | if x is not None and y is not None: 455 | styles.append('\\pos(%.0f, %.0f)' % (x, y)) 456 | if fsize is not None: 457 | styles.append('\\fs%.0f' % fsize) 458 | if rotate_y is not None and rotate_z is not None: 459 | styles.append('\\frz%.0f' % rotate_z) 460 | styles.append('\\fry%.0f' % rotate_y) 461 | if color is not None: 462 | styles.append('\\c&H%s&' % ConvertColor(color)) 463 | if color == 0x000000: 464 | styles.append('\\3c&HFFFFFF&') 465 | if alpha is not None: 466 | alpha = 255-round(alpha*255) 467 | styles.append('\\alpha&H%02X' % alpha) 468 | return styles 469 | 470 | def FlushCommentLine(f, text, styles, start_time, end_time, styleid): 471 | if end_time > start_time: 472 | f.write('Dialogue: -1,%(start)s,%(end)s,%(styleid)s,,0,0,0,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(start_time), 'end': ConvertTimestamp(end_time), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 473 | 474 | try: 475 | text = ASSEscape(str(c[3])) 476 | to_x = float(c[9])*width 477 | to_y = float(c[10])*height 478 | to_rotate_z = -int(c[14]) 479 | to_rotate_y = -int(c[15]) 480 | to_color = c[5] 481 | to_alpha = float(c[12]) 482 | # Note: Alpha transition hasn't been worked out yet. 483 | to_size = int(c[6])*math.sqrt(width*height/307200) 484 | # Note: Because sH5V's data is the absolute size of font,temporarily solve by it at present.[*math.sqrt(width/640*height/480)] 485 | # But it seems to be working fine... 486 | from_time = float(c[0]) 487 | action_time = float(c[11])/1000 488 | transform_styles = GetTransformStyles(to_x, to_y, to_size, to_rotate_z, to_rotate_y, to_color, to_alpha) 489 | FlushCommentLine(f, text, transform_styles, from_time, from_time+action_time, styleid) 490 | except (IndexError, ValueError) as e: 491 | logging.warning(_('Invalid comment: %r') % c[3]) 492 | 493 | 494 | # Result: (f, dx, dy) 495 | # To convert: NewX = f*x+dx, NewY = f*y+dy 496 | def GetZoomFactor(SourceSize, TargetSize): 497 | try: 498 | if (SourceSize, TargetSize) == GetZoomFactor.Cached_Size: 499 | return GetZoomFactor.Cached_Result 500 | except AttributeError: 501 | pass 502 | GetZoomFactor.Cached_Size = (SourceSize, TargetSize) 503 | try: 504 | SourceAspect = SourceSize[0]/SourceSize[1] 505 | TargetAspect = TargetSize[0]/TargetSize[1] 506 | if TargetAspect < SourceAspect: # narrower 507 | ScaleFactor = TargetSize[0]/SourceSize[0] 508 | GetZoomFactor.Cached_Result = (ScaleFactor, 0, (TargetSize[1]-TargetSize[0]/SourceAspect)/2) 509 | elif TargetAspect > SourceAspect: # wider 510 | ScaleFactor = TargetSize[1]/SourceSize[1] 511 | GetZoomFactor.Cached_Result = (ScaleFactor, (TargetSize[0]-TargetSize[1]*SourceAspect)/2, 0) 512 | else: 513 | GetZoomFactor.Cached_Result = (TargetSize[0]/SourceSize[0], 0, 0) 514 | return GetZoomFactor.Cached_Result 515 | except ZeroDivisionError: 516 | GetZoomFactor.Cached_Result = (1, 0, 0) 517 | return GetZoomFactor.Cached_Result 518 | 519 | 520 | # Calculation is based on https://github.com/jabbany/CommentCoreLibrary/issues/5#issuecomment-40087282 521 | # and https://github.com/m13253/danmaku2ass/issues/7#issuecomment-41489422 522 | # ASS FOV = width*4/3.0 523 | # But Flash FOV = width/math.tan(100*math.pi/360.0)/2 will be used instead 524 | # Result: (transX, transY, rotX, rotY, rotZ, scaleX, scaleY) 525 | def ConvertFlashRotation(rotY, rotZ, X, Y, width, height): 526 | def WrapAngle(deg): 527 | return 180-((180-deg) % 360) 528 | rotY = WrapAngle(rotY) 529 | rotZ = WrapAngle(rotZ) 530 | if rotY in (90, -90): 531 | rotY -= 1 532 | if rotY == 0 or rotZ == 0: 533 | outX = 0 534 | outY = -rotY # Positive value means clockwise in Flash 535 | outZ = -rotZ 536 | rotY *= math.pi/180.0 537 | rotZ *= math.pi/180.0 538 | else: 539 | rotY *= math.pi/180.0 540 | rotZ *= math.pi/180.0 541 | outY = math.atan2(-math.sin(rotY)*math.cos(rotZ), math.cos(rotY))*180/math.pi 542 | outZ = math.atan2(-math.cos(rotY)*math.sin(rotZ), math.cos(rotZ))*180/math.pi 543 | outX = math.asin(math.sin(rotY)*math.sin(rotZ))*180/math.pi 544 | trX = (X*math.cos(rotZ)+Y*math.sin(rotZ))/math.cos(rotY)+(1-math.cos(rotZ)/math.cos(rotY))*width/2-math.sin(rotZ)/math.cos(rotY)*height/2 545 | trY = Y*math.cos(rotZ)-X*math.sin(rotZ)+math.sin(rotZ)*width/2+(1-math.cos(rotZ))*height/2 546 | trZ = (trX-width/2)*math.sin(rotY) 547 | FOV = width*math.tan(2*math.pi/9.0)/2 548 | try: 549 | scaleXY = FOV/(FOV+trZ) 550 | except ZeroDivisionError: 551 | logging.error('Rotation makes object behind the camera: trZ == %.0f' % trZ) 552 | scaleXY = 1 553 | trX = (trX-width/2)*scaleXY+width/2 554 | trY = (trY-height/2)*scaleXY+height/2 555 | if scaleXY < 0: 556 | scaleXY = -scaleXY 557 | outX += 180 558 | outY += 180 559 | logging.error('Rotation makes object behind the camera: trZ == %.0f < %.0f' % (trZ, FOV)) 560 | return (trX, trY, WrapAngle(outX), WrapAngle(outY), WrapAngle(outZ), scaleXY*100, scaleXY*100) 561 | 562 | 563 | def ProcessComments(comments, f, width, height, bottomReserved, fontface, fontsize, alpha, duration_marquee, duration_still, reduced, progress_callback): 564 | styleid = 'Danmaku2ASS_%04x' % random.randint(0, 0xffff) 565 | WriteASSHead(f, width, height, fontface, fontsize, alpha, styleid) 566 | rows = [[None]*(height-bottomReserved+1) for i in xrange(4)] 567 | for idx, i in enumerate(comments): 568 | if progress_callback and idx % 1000 == 0: 569 | progress_callback(idx, len(comments)) 570 | if isinstance(i[4], int): 571 | row = 0 572 | rowmax = height-bottomReserved-i[7] 573 | while row <= rowmax: 574 | freerows = TestFreeRows(rows, i, row, width, height, bottomReserved, duration_marquee, duration_still) 575 | if freerows >= i[7]: 576 | MarkCommentRow(rows, i, row) 577 | WriteComment(f, i, row, width, height, bottomReserved, fontsize, duration_marquee, duration_still, styleid) 578 | break 579 | else: 580 | row += freerows or 1 581 | else: 582 | if not reduced: 583 | row = FindAlternativeRow(rows, i, height, bottomReserved) 584 | MarkCommentRow(rows, i, row) 585 | WriteComment(f, i, row, width, height, bottomReserved, fontsize, duration_marquee, duration_still, styleid) 586 | elif i[4] == 'bilipos': 587 | WriteCommentBilibiliPositioned(f, i, width, height, styleid) 588 | elif i[4] == 'acfunpos': 589 | WriteCommentAcfunPositioned(f, i, width, height, styleid) 590 | elif i[4] == 'sH5Vpos': 591 | WriteCommentSH5VPositioned(f, i, width, height, styleid) 592 | else: 593 | logging.warning(_('Invalid comment: %r') % i[3]) 594 | if progress_callback: 595 | progress_callback(len(comments), len(comments)) 596 | 597 | 598 | def TestFreeRows(rows, c, row, width, height, bottomReserved, duration_marquee, duration_still): 599 | res = 0 600 | rowmax = height-bottomReserved 601 | targetRow = None 602 | if c[4] in (1, 2): 603 | while row < rowmax and res < c[7]: 604 | if targetRow != rows[c[4]][row]: 605 | targetRow = rows[c[4]][row] 606 | if targetRow and targetRow[0]+duration_still > c[0]: 607 | break 608 | row += 1 609 | res += 1 610 | else: 611 | try: 612 | thresholdTime = c[0]-duration_marquee*(1-width/(c[8]+width)) 613 | except ZeroDivisionError: 614 | thresholdTime = c[0]-duration_marquee 615 | while row < rowmax and res < c[7]: 616 | if targetRow != rows[c[4]][row]: 617 | targetRow = rows[c[4]][row] 618 | try: 619 | if targetRow and (targetRow[0] > thresholdTime or targetRow[0]+targetRow[8]*duration_marquee/(targetRow[8]+width) > c[0]): 620 | break 621 | except ZeroDivisionError: 622 | pass 623 | row += 1 624 | res += 1 625 | return res 626 | 627 | 628 | def FindAlternativeRow(rows, c, height, bottomReserved): 629 | res = 0 630 | for row in xrange(height-bottomReserved-int(math.ceil(c[7]))): 631 | if not rows[c[4]][row]: 632 | return row 633 | elif rows[c[4]][row][0] < rows[c[4]][res][0]: 634 | res = row 635 | return res 636 | 637 | 638 | def MarkCommentRow(rows, c, row): 639 | try: 640 | for i in xrange(row, row+int(math.ceil(c[7]))): 641 | rows[c[4]][i] = c 642 | except IndexError: 643 | pass 644 | 645 | 646 | def WriteASSHead(f, width, height, fontface, fontsize, alpha, styleid): 647 | f.write( 648 | ''' 649 | [Script Info] 650 | ; Script generated by Danmaku2ASS 651 | ; https://github.com/m13253/danmaku2ass 652 | Script Updated By: Danmaku2ASS (https://github.com/m13253/danmaku2ass) 653 | ScriptType: v4.00+ 654 | PlayResX: %(width)d 655 | PlayResY: %(height)d 656 | Aspect Ratio: %(width)d:%(height)d 657 | Collisions: Normal 658 | WrapStyle: 2 659 | ScaledBorderAndShadow: yes 660 | YCbCr Matrix: TV.601 661 | 662 | [V4+ Styles] 663 | Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding 664 | Style: %(styleid)s, %(fontface)s, %(fontsize).0f, &H%(alpha)02XFFFFFF, &H%(alpha)02XFFFFFF, &H%(alpha)02X000000, &H%(alpha)02X000000, 0, 0, 0, 0, 100, 100, 0.00, 0.00, 1, %(outline).0f, 0, 7, 0, 0, 0, 0 665 | 666 | [Events] 667 | Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text 668 | ''' % {'width': width, 'height': height, 'fontface': fontface, 'fontsize': fontsize, 'alpha': 255-round(alpha*255), 'outline': max(fontsize/25.0, 1), 'styleid': styleid} 669 | ) 670 | 671 | 672 | def WriteComment(f, c, row, width, height, bottomReserved, fontsize, duration_marquee, duration_still, styleid): 673 | text = ASSEscape(c[3]) 674 | styles = [] 675 | if c[4] == 1: 676 | styles.append('\\an8\\pos(%(halfwidth)d, %(row)d)' % {'halfwidth': width/2, 'row': row}) 677 | duration = duration_still 678 | elif c[4] == 2: 679 | styles.append('\\an2\\pos(%(halfwidth)d, %(row)d)' % {'halfwidth': width/2, 'row': ConvertType2(row, height, bottomReserved)}) 680 | duration = duration_still 681 | elif c[4] == 3: 682 | styles.append('\\move(%(neglen)d, %(row)d, %(width)d, %(row)d)' % {'width': width, 'row': row, 'neglen': -math.ceil(c[8])}) 683 | duration = duration_marquee 684 | else: 685 | styles.append('\\move(%(width)d, %(row)d, %(neglen)d, %(row)d)' % {'width': width, 'row': row, 'neglen': -math.ceil(c[8])}) 686 | duration = duration_marquee 687 | if not (-1 < c[6]-fontsize < 1): 688 | styles.append('\\fs%.0f' % c[6]) 689 | if c[5] != 0xffffff: 690 | styles.append('\\c&H%s&' % ConvertColor(c[5])) 691 | if c[5] == 0x000000: 692 | styles.append('\\3c&HFFFFFF&') 693 | f.write(u'Dialogue: 2,%(start)s,%(end)s,%(styleid)s,,0000,0000,0000,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(c[0]), 'end': ConvertTimestamp(c[0]+duration), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 694 | 695 | 696 | def ASSEscape(s): 697 | def ReplaceLeadingSpace(s): 698 | sstrip = s.strip(' ') 699 | slen = len(s) 700 | if slen == len(sstrip): 701 | return s 702 | else: 703 | llen = slen-len(s.lstrip(' ')) 704 | rlen = slen-len(s.rstrip(' ')) 705 | return ''.join(('\u2007'*llen, sstrip, '\u2007'*rlen)) 706 | return '\\N'.join((ReplaceLeadingSpace(i) or ' ' for i in str(s).replace('\\', '\\\\').replace('{', '\\{').replace('}', '\\}').split('\n'))) 707 | 708 | 709 | def CalculateLength(s): 710 | return max(imap(len, s.split('\n'))) # May not be accurate 711 | 712 | 713 | def ConvertTimestamp(timestamp): 714 | timestamp = round(timestamp*100.0) 715 | hour, minute = divmod(timestamp, 360000) 716 | minute, second = divmod(minute, 6000) 717 | second, centsecond = divmod(second, 100) 718 | return '%d:%02d:%02d.%02d' % (int(hour), int(minute), int(second), int(centsecond)) 719 | 720 | 721 | def ConvertColor(RGB, width=1280, height=576): 722 | if RGB == 0x000000: 723 | return '000000' 724 | elif RGB == 0xffffff: 725 | return 'FFFFFF' 726 | R = (RGB >> 16) & 0xff 727 | G = (RGB >> 8) & 0xff 728 | B = RGB & 0xff 729 | if width < 1280 and height < 576: 730 | return '%02X%02X%02X' % (B, G, R) 731 | else: # VobSub always uses BT.601 colorspace, convert to BT.709 732 | ClipByte = lambda x: 255 if x > 255 else 0 if x < 0 else round(x) 733 | return '%02X%02X%02X' % ( 734 | ClipByte(R*0.00956384088080656+G*0.03217254540203729+B*0.95826361371715607), 735 | ClipByte(R*-0.10493933142075390+G*1.17231478191855154+B*-0.06737545049779757), 736 | ClipByte(R*0.91348912373987645+G*0.07858536372532510+B*0.00792551253479842) 737 | ) 738 | 739 | 740 | def ConvertType2(row, height, bottomReserved): 741 | return height-bottomReserved-row 742 | 743 | 744 | def ConvertToFile(filename_or_file, *args, **kwargs): 745 | if isinstance(filename_or_file, bytes): 746 | filename_or_file = str(bytes(filename_or_file).decode('utf-8', 'replace')) 747 | if isinstance(filename_or_file, str): 748 | return open(filename_or_file, *args, **kwargs) 749 | else: 750 | return filename_or_file 751 | 752 | 753 | def FilterBadChars(f): 754 | s = f.read() 755 | s = re.sub('[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f]', '\ufffd', s) 756 | return io.StringIO(s) 757 | 758 | 759 | class safe_list(list): 760 | def get(self, index, default=None): 761 | try: 762 | return self[index] 763 | except IndexError: 764 | return default 765 | 766 | 767 | def export(func): 768 | global __all__ 769 | try: 770 | __all__.append(func.__name__) 771 | except NameError: 772 | __all__ = [func.__name__] 773 | return func 774 | 775 | 776 | @export 777 | def Danmaku2ASS(input_files, output_file, stage_width, stage_height, reserve_blank=0, font_face=_('(FONT) sans-serif')[7:], font_size=25.0, text_opacity=1.0, duration_marquee=5.0, duration_still=5.0, is_reduce_comments=False, progress_callback=None): 778 | fo = None 779 | comments = ReadComments(input_files, font_size) 780 | try: 781 | if output_file: 782 | fo = ConvertToFile(output_file, 'w', encoding='utf-8-sig', errors='replace', newline='\r\n') 783 | else: 784 | fo = sys.stdout 785 | ProcessComments(comments, fo, stage_width, stage_height, reserve_blank, font_face, font_size, text_opacity, duration_marquee, duration_still, is_reduce_comments, progress_callback) 786 | finally: 787 | if output_file and fo != output_file: 788 | fo.close() 789 | 790 | 791 | @export 792 | def ReadComments(input_files, font_size=25.0, progress_callback=None): 793 | if isinstance(input_files, bytes): 794 | input_files = str(bytes(input_files).decode('utf-8', 'replace')) 795 | if isinstance(input_files, str): 796 | input_files = [input_files] 797 | else: 798 | input_files = list(input_files) 799 | comments = [] 800 | for idx, i in enumerate(input_files): 801 | if progress_callback: 802 | progress_callback(idx, len(input_files)) 803 | with ConvertToFile(i, 'r', encoding='utf-8', errors='replace') as f: 804 | CommentProcessor = GetCommentProcessor(f) 805 | if not CommentProcessor: 806 | raise ValueError(_('Unknown comment file format: %s') % i) 807 | comments.extend(CommentProcessor(FilterBadChars(f), font_size)) 808 | if progress_callback: 809 | progress_callback(len(input_files), len(input_files)) 810 | comments.sort() 811 | return comments 812 | 813 | 814 | @export 815 | def GetCommentProcessor(input_file): 816 | return CommentFormatMap[ProbeCommentFormat(input_file)] 817 | 818 | def turn_List_Unicode(List): 819 | Files = [] 820 | for _file in List: 821 | Files.append(_file.decode(sys.stdin.encoding)) 822 | return Files 823 | 824 | def main(): 825 | logging.basicConfig(format='%(levelname)s: %(message)s') 826 | if len(sys.argv) == 1: 827 | sys.argv.append('--help') 828 | parser = argparse.ArgumentParser() 829 | parser.add_argument(b'-o', b'--output', metavar=_(b'OUTPUT'), help=_(b'Output file')) 830 | parser.add_argument(b'-s', b'--size', metavar=_(b'WIDTHxHEIGHT'), required=True, help=_(b'Stage size in pixels')) 831 | parser.add_argument(b'-fn', b'--font', metavar=_(b'FONT'), help=_(b'Specify font face [default: %s]') % _(b'(FONT) sans-serif')[7:], default=_('(FONT) sans-serif')[7:]) 832 | parser.add_argument(b'-fs', b'--fontsize', metavar=_(b'SIZE'), help=(_(b'Default font size [default: %s]') % 25), type=float, default=25.0) 833 | parser.add_argument(b'-a', b'--alpha', metavar=_(b'ALPHA'), help=_(b'Text opacity'), type=float, default=1.0) 834 | parser.add_argument(b'-dm', b'--duration-marquee', metavar=_(b'SECONDS'), help=_(b'Duration of scrolling comment display [default: %s]') % 5, type=float, default=5.0) 835 | parser.add_argument(b'-ds', b'--duration-still', metavar=_(b'SECONDS'), help=_(b'Duration of still comment display [default: %s]') % 5, type=float, default=5.0) 836 | parser.add_argument(b'-p', b'--protect', metavar=_(b'HEIGHT'), help=_(b'Reserve blank on the bottom of the stage'), type=int, default=0) 837 | parser.add_argument(b'-r', b'--reduce', action=b'store_true', help=_(b'Reduce the amount of comments if stage is full')) 838 | parser.add_argument(b'file', metavar=_(b'FILE'), nargs=b'+', help=_(b'Comment file to be processed')) 839 | args = parser.parse_args() 840 | try: 841 | width, height = bytes(args.size).decode('utf-8', 'replace').split('x', 1) 842 | width = int(width) 843 | height = int(height) 844 | args.file = turn_List_Unicode(args.file) 845 | except ValueError: 846 | raise ValueError(_('Invalid stage size: %r') % args.size) 847 | Danmaku2ASS(args.file, args.output, width, height, args.protect, args.font, args.fontsize, args.alpha, args.duration_marquee, args.duration_still, args.reduce) 848 | 849 | 850 | if __name__ == '__main__': 851 | main() -------------------------------------------------------------------------------- /danmaku2ass3.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # The original author of this program, Danmaku2ASS, is StarBrilliant. 4 | # This file is released under General Public License version 3. 5 | # You should have received a copy of General Public License text alongside with 6 | # this program. If not, you can obtain it at http://gnu.org/copyleft/gpl.html . 7 | # This program comes with no warranty, the author will not be resopnsible for 8 | # any damage or problems caused by this program. 9 | 10 | # You can obtain a latest copy of Danmaku2ASS at: 11 | # https://github.com/m13253/danmaku2ass 12 | # Please update to the latest version before complaining. 13 | 14 | import argparse 15 | import calendar 16 | import gettext 17 | import io 18 | import json 19 | import logging 20 | import math 21 | import os 22 | import random 23 | import re 24 | import sys 25 | import time 26 | import xml.dom.minidom 27 | 28 | 29 | if sys.version_info < (3,): 30 | raise RuntimeError('at least Python 3.0 is required') 31 | 32 | gettext.install('danmaku2ass', os.path.join(os.path.dirname(os.path.abspath(os.path.realpath(sys.argv[0] or 'locale'))), 'locale')) 33 | 34 | 35 | def SeekZero(function): 36 | def decorated_function(file_): 37 | file_.seek(0) 38 | try: 39 | return function(file_) 40 | finally: 41 | file_.seek(0) 42 | return decorated_function 43 | 44 | 45 | def EOFAsNone(function): 46 | def decorated_function(*args, **kwargs): 47 | try: 48 | return function(*args, **kwargs) 49 | except EOFError: 50 | return None 51 | return decorated_function 52 | 53 | 54 | @SeekZero 55 | @EOFAsNone 56 | def ProbeCommentFormat(f): 57 | tmp = f.read(1) 58 | if tmp == '[': 59 | return 'Acfun' 60 | # It is unwise to wrap a JSON object in an array! 61 | # See this: http://haacked.com/archive/2008/11/20/anatomy-of-a-subtle-json-vulnerability.aspx/ 62 | # Do never follow what Acfun developers did! 63 | elif tmp == '{': 64 | tmp = f.read(14) 65 | if tmp == '"status_code":': 66 | return 'Tudou' 67 | elif tmp == '"root":{"total': 68 | return 'sH5V' 69 | elif tmp == '<': 70 | tmp = f.read(1) 71 | if tmp == '?': 72 | tmp = f.read(38) 73 | if tmp == 'xml version="1.0" encoding="UTF-8"?>\n<': 80 | return 'Bilibili' # Komica, with the same file format as Bilibili 81 | elif tmp == 'xml version="1.0" encoding="UTF-8"?>\n<': 82 | return 'MioMio' 83 | elif tmp == 'p': 84 | return 'Niconico' # Himawari Douga, with the same file format as Niconico Douga 85 | 86 | 87 | # 88 | # ReadComments**** protocol 89 | # 90 | # Input: 91 | # f: Input file 92 | # fontsize: Default font size 93 | # 94 | # Output: 95 | # yield a tuple: 96 | # (timeline, timestamp, no, comment, pos, color, size, height, width) 97 | # timeline: The position when the comment is replayed 98 | # timestamp: The UNIX timestamp when the comment is submitted 99 | # no: A sequence of 1, 2, 3, ..., used for sorting 100 | # comment: The content of the comment 101 | # pos: 0 for regular moving comment, 102 | # 1 for bottom centered comment, 103 | # 2 for top centered comment, 104 | # 3 for reversed moving comment 105 | # color: Font color represented in 0xRRGGBB, 106 | # e.g. 0xffffff for white 107 | # size: Font size 108 | # height: The estimated height in pixels 109 | # i.e. (comment.count('\n')+1)*size 110 | # width: The estimated width in pixels 111 | # i.e. CalculateLength(comment)*size 112 | # 113 | # After implementing ReadComments****, make sure to update ProbeCommentFormat 114 | # and CommentFormatMap. 115 | # 116 | 117 | 118 | def ReadCommentsNiconico(f, fontsize): 119 | NiconicoColorMap = {'red': 0xff0000, 'pink': 0xff8080, 'orange': 0xffcc00, 'yellow': 0xffff00, 'green': 0x00ff00, 'cyan': 0x00ffff, 'blue': 0x0000ff, 'purple': 0xc000ff, 'black': 0x000000, 'niconicowhite': 0xcccc99, 'white2': 0xcccc99, 'truered': 0xcc0033, 'red2': 0xcc0033, 'passionorange': 0xff6600, 'orange2': 0xff6600, 'madyellow': 0x999900, 'yellow2': 0x999900, 'elementalgreen': 0x00cc66, 'green2': 0x00cc66, 'marineblue': 0x33ffcc, 'blue2': 0x33ffcc, 'nobleviolet': 0x6633cc, 'purple2': 0x6633cc} 120 | dom = xml.dom.minidom.parse(f) 121 | comment_element = dom.getElementsByTagName('chat') 122 | for comment in comment_element: 123 | try: 124 | c = str(comment.childNodes[0].wholeText) 125 | if c.startswith('/'): 126 | continue # ignore advanced comments 127 | pos = 0 128 | color = 0xffffff 129 | size = fontsize 130 | for mailstyle in str(comment.getAttribute('mail')).split(): 131 | if mailstyle == 'ue': 132 | pos = 1 133 | elif mailstyle == 'shita': 134 | pos = 2 135 | elif mailstyle == 'big': 136 | size = fontsize*1.44 137 | elif mailstyle == 'small': 138 | size = fontsize*0.64 139 | elif mailstyle in NiconicoColorMap: 140 | color = NiconicoColorMap[mailstyle] 141 | yield (max(int(comment.getAttribute('vpos')), 0)*0.01, int(comment.getAttribute('date')), int(comment.getAttribute('no')), c, pos, color, size, (c.count('\n')+1)*size, CalculateLength(c)*size) 142 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 143 | logging.warning(_('Invalid comment: %s') % comment.toxml()) 144 | continue 145 | 146 | 147 | def ReadCommentsAcfun(f, fontsize): 148 | comment_element = json.load(f) 149 | for i, comment in enumerate(comment_element): 150 | try: 151 | p = str(comment['c']).split(',') 152 | assert len(p) >= 6 153 | assert p[2] in ('1', '2', '4', '5', '7') 154 | size = int(p[3])*fontsize/25.0 155 | if p[2] != '7': 156 | c = str(comment['m']).replace('\\r', '\n').replace('\r', '\n') 157 | yield (float(p[0]), int(p[5]), i, c, {'1': 0, '2': 0, '4': 2, '5': 1}[p[2]], int(p[1]), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 158 | else: 159 | c = dict(json.loads(comment['m'])) 160 | yield (float(p[0]), int(p[5]), i, c, 'acfunpos', int(p[1]), size, 0, 0) 161 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 162 | logging.warning(_('Invalid comment: %r') % comment) 163 | continue 164 | 165 | 166 | def ReadCommentsBilibili(f, fontsize): 167 | dom = xml.dom.minidom.parse(f) 168 | comment_element = dom.getElementsByTagName('d') 169 | for i, comment in enumerate(comment_element): 170 | try: 171 | p = str(comment.getAttribute('p')).split(',') 172 | assert len(p) >= 5 173 | assert p[1] in ('1', '4', '5', '6', '7') 174 | if p[1] != '7': 175 | c = str(comment.childNodes[0].wholeText).replace('/n', '\n') 176 | size = int(p[2])*fontsize/25.0 177 | yield (float(p[0]), int(p[4]), i, c, {'1': 0, '4': 2, '5': 1, '6': 3}[p[1]], int(p[3]), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 178 | else: # positioned comment 179 | c = str(comment.childNodes[0].wholeText) 180 | yield (float(p[0]), int(p[4]), i, c, 'bilipos', int(p[3]), int(p[2]), 0, 0) 181 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 182 | logging.warning(_('Invalid comment: %s') % comment.toxml()) 183 | continue 184 | 185 | 186 | def ReadCommentsTudou(f, fontsize): 187 | comment_element = json.load(f) 188 | for i, comment in enumerate(comment_element['comment_list']): 189 | try: 190 | assert comment['pos'] in (3, 4, 6) 191 | c = str(comment['data']) 192 | assert comment['size'] in (0, 1, 2) 193 | size = {0: 0.64, 1: 1, 2: 1.44}[comment['size']]*fontsize 194 | yield (int(comment['replay_time']*0.001), int(comment['commit_time']), i, c, {3: 0, 4: 2, 6: 1}[comment['pos']], int(comment['color']), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 195 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 196 | logging.warning(_('Invalid comment: %r') % comment) 197 | continue 198 | 199 | 200 | def ReadCommentsMioMio(f, fontsize): 201 | NiconicoColorMap = {'red': 0xff0000, 'pink': 0xff8080, 'orange': 0xffc000, 'yellow': 0xffff00, 'green': 0x00ff00, 'cyan': 0x00ffff, 'blue': 0x0000ff, 'purple': 0xc000ff, 'black': 0x000000} 202 | dom = xml.dom.minidom.parse(f) 203 | comment_element = dom.getElementsByTagName('data') 204 | for i, comment in enumerate(comment_element): 205 | try: 206 | message = comment.getElementsByTagName('message')[0] 207 | c = str(message.childNodes[0].wholeText) 208 | pos = 0 209 | size = int(message.getAttribute('fontsize'))*fontsize/25.0 210 | yield (float(comment.getElementsByTagName('playTime')[0].childNodes[0].wholeText), int(calendar.timegm(time.strptime(comment.getElementsByTagName('times')[0].childNodes[0].wholeText, '%Y-%m-%d %H:%M:%S')))-28800, i, c, {'1': 0, '4': 2, '5': 1}[message.getAttribute('mode')], int(message.getAttribute('color')), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 211 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 212 | logging.warning(_('Invalid comment: %s') % comment.toxml()) 213 | continue 214 | 215 | 216 | def ReadCommentsSH5V(f, fontsize): 217 | comment_element = json.load(f) 218 | for i, comment in enumerate(comment_element["root"]["bgs"]): 219 | try: 220 | c_at = str(comment['at']) 221 | c_type = str(comment['type']) 222 | c_date = str(comment['timestamp']) 223 | c_color = str(comment['color']) 224 | c = str(comment['text']) 225 | size = fontsize 226 | if c_type != '7': 227 | yield (float(c_at), int(c_date), i, c, {'0': 0, '1': 0, '4': 2, '5': 1}[c_type], int(c_color[1:], 16), size, (c.count('\n')+1)*size, CalculateLength(c)*size) 228 | else: 229 | c_x = float(comment['x']) 230 | c_y = float(comment['y']) 231 | size = int(comment['size']) 232 | dur = int(comment['dur']) 233 | data1 = float(comment['data1']) 234 | data2 = float(comment['data2']) 235 | data3 = int(comment['data3']) 236 | data4 = int(comment['data4']) 237 | yield (float(c_at), int(c_date), i, c, 'sH5Vpos', int(c_color[1:], 16), size, 0, 0, c_x, c_y, dur, data1, data2, data3, data4) 238 | except (AssertionError, AttributeError, IndexError, TypeError, ValueError): 239 | logging.warning(_('Invalid comment: %r') % comment) 240 | continue 241 | 242 | 243 | CommentFormatMap = {None: None, 'Niconico': ReadCommentsNiconico, 'Acfun': ReadCommentsAcfun, 'Bilibili': ReadCommentsBilibili, 'Tudou': ReadCommentsTudou, 'MioMio': ReadCommentsMioMio, 'sH5V': ReadCommentsSH5V} 244 | 245 | 246 | def WriteCommentBilibiliPositioned(f, c, width, height, styleid): 247 | #BiliPlayerSize = (512, 384) # Bilibili player version 2010 248 | #BiliPlayerSize = (540, 384) # Bilibili player version 2012 249 | BiliPlayerSize = (672, 438) # Bilibili player version 2014 250 | ZoomFactor = GetZoomFactor(BiliPlayerSize, (width, height)) 251 | 252 | def GetPosition(InputPos, isHeight): 253 | isHeight = int(isHeight) # True -> 1 254 | if isinstance(InputPos, int): 255 | return ZoomFactor[0]*InputPos+ZoomFactor[isHeight+1] 256 | elif isinstance(InputPos, float): 257 | if InputPos > 1: 258 | return ZoomFactor[0]*InputPos+ZoomFactor[isHeight+1] 259 | else: 260 | return BiliPlayerSize[isHeight]*ZoomFactor[0]*InputPos+ZoomFactor[isHeight+1] 261 | else: 262 | try: 263 | InputPos = int(InputPos) 264 | except ValueError: 265 | InputPos = float(InputPos) 266 | return GetPosition(InputPos, isHeight) 267 | 268 | try: 269 | comment_args = safe_list(json.loads(c[3])) 270 | text = ASSEscape(str(comment_args[4]).replace('/n', '\n')) 271 | from_x = comment_args.get(0, 0) 272 | from_y = comment_args.get(1, 0) 273 | to_x = comment_args.get(7, from_x) 274 | to_y = comment_args.get(8, from_y) 275 | from_x = GetPosition(from_x, False) 276 | from_y = GetPosition(from_y, True) 277 | to_x = GetPosition(to_x, False) 278 | to_y = GetPosition(to_y, True) 279 | alpha = safe_list(str(comment_args.get(2, '1')).split('-')) 280 | from_alpha = float(alpha.get(0, 1)) 281 | to_alpha = float(alpha.get(1, from_alpha)) 282 | from_alpha = 255-round(from_alpha*255) 283 | to_alpha = 255-round(to_alpha*255) 284 | rotate_z = int(comment_args.get(5, 0)) 285 | rotate_y = int(comment_args.get(6, 0)) 286 | lifetime = float(comment_args.get(3, 4500)) 287 | duration = int(comment_args.get(9, lifetime*1000)) 288 | delay = int(comment_args.get(10, 0)) 289 | fontface = comment_args.get(12) 290 | isborder = comment_args.get(11, 'true') 291 | from_rotarg = ConvertFlashRotation(rotate_y, rotate_z, from_x, from_y, width, height) 292 | to_rotarg = ConvertFlashRotation(rotate_y, rotate_z, to_x, to_y, width, height) 293 | styles = ['\\org(%d, %d)' % (width/2, height/2)] 294 | if from_rotarg[0:2] == to_rotarg[0:2]: 295 | styles.append('\\pos(%.0f, %.0f)' % (from_rotarg[0:2])) 296 | else: 297 | styles.append('\\move(%.0f, %.0f, %.0f, %.0f, %.0f, %.0f)' % (from_rotarg[0:2]+to_rotarg[0:2]+(delay, delay+duration))) 298 | styles.append('\\frx%.0f\\fry%.0f\\frz%.0f\\fscx%.0f\\fscy%.0f' % (from_rotarg[2:7])) 299 | if (from_x, from_y) != (to_x, to_y): 300 | styles.append('\\t(%d, %d, ' % (delay, delay+duration)) 301 | styles.append('\\frx%.0f\\fry%.0f\\frz%.0f\\fscx%.0f\\fscy%.0f' % (to_rotarg[2:7])) 302 | styles.append(')') 303 | if fontface: 304 | styles.append('\\fn%s' % ASSEscape(fontface)) 305 | styles.append('\\fs%.0f' % (c[6]*ZoomFactor[0])) 306 | if c[5] != 0xffffff: 307 | styles.append('\\c&H%s&' % ConvertColor(c[5])) 308 | if c[5] == 0x000000: 309 | styles.append('\\3c&HFFFFFF&') 310 | if from_alpha == to_alpha: 311 | styles.append('\\alpha&H%02X' % from_alpha) 312 | elif (from_alpha, to_alpha) == (255, 0): 313 | styles.append('\\fad(%.0f,0)' % (lifetime*1000)) 314 | elif (from_alpha, to_alpha) == (0, 255): 315 | styles.append('\\fad(0, %.0f)' % (lifetime*1000)) 316 | else: 317 | styles.append('\\fade(%(from_alpha)d, %(to_alpha)d, %(to_alpha)d, 0, %(end_time).0f, %(end_time).0f, %(end_time).0f)' % {'from_alpha': from_alpha, 'to_alpha': to_alpha, 'end_time': lifetime*1000}) 318 | if isborder == 'false': 319 | styles.append('\\bord0') 320 | f.write('Dialogue: -1,%(start)s,%(end)s,%(styleid)s,,0,0,0,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(c[0]), 'end': ConvertTimestamp(c[0]+lifetime), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 321 | except (IndexError, ValueError) as e: 322 | try: 323 | logging.warning(_('Invalid comment: %r') % c[3]) 324 | except IndexError: 325 | logging.warning(_('Invalid comment: %r') % c) 326 | 327 | 328 | def WriteCommentAcfunPositioned(f, c, width, height, styleid): 329 | AcfunPlayerSize = (560, 400) 330 | ZoomFactor = GetZoomFactor(AcfunPlayerSize, (width, height)) 331 | 332 | def GetPosition(InputPos, isHeight): 333 | isHeight = int(isHeight) # True -> 1 334 | return AcfunPlayerSize[isHeight]*ZoomFactor[0]*InputPos*0.001+ZoomFactor[isHeight+1] 335 | 336 | def GetTransformStyles(x=None, y=None, scale_x=None, scale_y=None, rotate_z=None, rotate_y=None, color=None, alpha=None): 337 | styles = [] 338 | out_x, out_y = x, y 339 | if rotate_z is not None and rotate_y is not None: 340 | assert x is not None 341 | assert y is not None 342 | rotarg = ConvertFlashRotation(rotate_y, rotate_z, x, y, width, height) 343 | out_x, out_y = rotarg[0:2] 344 | if scale_x is None: 345 | scale_x = 1 346 | if scale_y is None: 347 | scale_y = 1 348 | styles.append('\\frx%.0f\\fry%.0f\\frz%.0f\\fscx%.0f\\fscy%.0f' % (rotarg[2:5]+(rotarg[5]*scale_x, rotarg[6]*scale_y))) 349 | else: 350 | if scale_x is not None: 351 | styles.append('\\fscx%.0f' % (scale_x*100)) 352 | if scale_y is not None: 353 | styles.append('\\fscy%.0f' % (scale_y*100)) 354 | if color is not None: 355 | styles.append('\\c&H%s&' % ConvertColor(color)) 356 | if color == 0x000000: 357 | styles.append('\\3c&HFFFFFF&') 358 | if alpha is not None: 359 | alpha = 255-round(alpha*255) 360 | styles.append('\\alpha&H%02X' % alpha) 361 | return out_x, out_y, styles 362 | 363 | def FlushCommentLine(f, text, styles, start_time, end_time, styleid): 364 | if end_time > start_time: 365 | f.write('Dialogue: -1,%(start)s,%(end)s,%(styleid)s,,0,0,0,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(start_time), 'end': ConvertTimestamp(end_time), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 366 | 367 | try: 368 | comment_args = c[3] 369 | text = ASSEscape(str(comment_args['n']).replace('\r', '\n')) 370 | common_styles = ['\org(%d, %d)' % (width/2, height/2)] 371 | anchor = {0: 7, 1: 8, 2: 9, 3: 4, 4: 5, 5: 6, 6: 1, 7: 2, 8: 3}.get(comment_args.get('c', 0), 7) 372 | if anchor != 7: 373 | common_styles.append('\\an%s' % anchor) 374 | font = comment_args.get('w') 375 | if font: 376 | font = dict(font) 377 | fontface = font.get('f') 378 | if fontface: 379 | common_styles.append('\\fn%s' % ASSEscape(str(fontface))) 380 | fontbold = bool(font.get('b')) 381 | if fontbold: 382 | common_styles.append('\\b1') 383 | common_styles.append('\\fs%.0f' % (c[6]*ZoomFactor[0])) 384 | isborder = bool(comment_args.get('b', True)) 385 | if not isborder: 386 | common_styles.append('\\bord0') 387 | to_pos = dict(comment_args.get('p', {'x': 0, 'y': 0})) 388 | to_x = round(GetPosition(int(to_pos.get('x', 0)), False)) 389 | to_y = round(GetPosition(int(to_pos.get('y', 0)), True)) 390 | to_scale_x = float(comment_args.get('e', 1.0)) 391 | to_scale_y = float(comment_args.get('f', 1.0)) 392 | to_rotate_z = float(comment_args.get('r', 0.0)) 393 | to_rotate_y = float(comment_args.get('k', 0.0)) 394 | to_color = c[5] 395 | to_alpha = float(comment_args.get('a', 1.0)) 396 | from_time = float(comment_args.get('t', 0.0)) 397 | action_time = float(comment_args.get('l', 3.0)) 398 | actions = list(comment_args.get('z', [])) 399 | to_out_x, to_out_y, transform_styles = GetTransformStyles(to_x, to_y, to_scale_x, to_scale_y, to_rotate_z, to_rotate_y, to_color, to_alpha) 400 | FlushCommentLine(f, text, common_styles+['\\pos(%.0f, %.0f)' % (to_out_x, to_out_y)]+transform_styles, c[0]+from_time, c[0]+from_time+action_time, styleid) 401 | action_styles = transform_styles 402 | for action in actions: 403 | action = dict(action) 404 | from_x, from_y = to_x, to_y 405 | from_out_x, from_out_y = to_out_x, to_out_y 406 | from_scale_x, from_scale_y = to_scale_x, to_scale_y 407 | from_rotate_z, from_rotate_y = to_rotate_z, to_rotate_y 408 | from_color, from_alpha = to_color, to_alpha 409 | transform_styles, action_styles = action_styles, [] 410 | from_time += action_time 411 | action_time = float(action.get('l', 0.0)) 412 | if 'x' in action: 413 | to_x = round(GetPosition(int(action['x']), False)) 414 | if 'y' in action: 415 | to_y = round(GetPosition(int(action['y']), True)) 416 | if 'f' in action: 417 | to_scale_x = float(action['f']) 418 | if 'g' in action: 419 | to_scale_y = float(action['g']) 420 | if 'c' in action: 421 | to_color = int(action['c']) 422 | if 't' in action: 423 | to_alpha = float(action['t']) 424 | if 'd' in action: 425 | to_rotate_z = float(action['d']) 426 | if 'e' in action: 427 | to_rotate_y = float(action['e']) 428 | to_out_x, to_out_y, action_styles = GetTransformStyles(to_x, to_y, from_scale_x, from_scale_y, to_rotate_z, to_rotate_y, from_color, from_alpha) 429 | if (from_out_x, from_out_y) == (to_out_x, to_out_y): 430 | pos_style = '\\pos(%.0f, %.0f)' % (to_out_x, to_out_y) 431 | else: 432 | pos_style = '\\move(%.0f, %.0f, %.0f, %.0f)' % (from_out_x, from_out_y, to_out_x, to_out_y) 433 | styles = common_styles+transform_styles 434 | styles.append(pos_style) 435 | if action_styles: 436 | styles.append('\\t(%s)' % (''.join(action_styles))) 437 | FlushCommentLine(f, text, styles, c[0]+from_time, c[0]+from_time+action_time, styleid) 438 | except (IndexError, ValueError) as e: 439 | logging.warning(_('Invalid comment: %r') % c[3]) 440 | 441 | 442 | def WriteCommentSH5VPositioned(f, c, width, height, styleid): 443 | 444 | def GetTransformStyles(x=None, y=None, fsize=None, rotate_z=None, rotate_y=None, color=None, alpha=None): 445 | styles = [] 446 | if x is not None and y is not None: 447 | styles.append('\\pos(%.0f, %.0f)' % (x, y)) 448 | if fsize is not None: 449 | styles.append('\\fs%.0f' % fsize) 450 | if rotate_y is not None and rotate_z is not None: 451 | styles.append('\\frz%.0f' % rotate_z) 452 | styles.append('\\fry%.0f' % rotate_y) 453 | if color is not None: 454 | styles.append('\\c&H%s&' % ConvertColor(color)) 455 | if color == 0x000000: 456 | styles.append('\\3c&HFFFFFF&') 457 | if alpha is not None: 458 | alpha = 255-round(alpha*255) 459 | styles.append('\\alpha&H%02X' % alpha) 460 | return styles 461 | 462 | def FlushCommentLine(f, text, styles, start_time, end_time, styleid): 463 | if end_time > start_time: 464 | f.write('Dialogue: -1,%(start)s,%(end)s,%(styleid)s,,0,0,0,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(start_time), 'end': ConvertTimestamp(end_time), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 465 | 466 | try: 467 | text = ASSEscape(str(c[3])) 468 | to_x = float(c[9])*width 469 | to_y = float(c[10])*height 470 | to_rotate_z = -int(c[14]) 471 | to_rotate_y = -int(c[15]) 472 | to_color = c[5] 473 | to_alpha = float(c[12]) 474 | # Note: Alpha transition hasn't been worked out yet. 475 | to_size = int(c[6])*math.sqrt(width*height/307200) 476 | # Note: Because sH5V's data is the absolute size of font,temporarily solve by it at present.[*math.sqrt(width/640*height/480)] 477 | # But it seems to be working fine... 478 | from_time = float(c[0]) 479 | action_time = float(c[11])/1000 480 | transform_styles = GetTransformStyles(to_x, to_y, to_size, to_rotate_z, to_rotate_y, to_color, to_alpha) 481 | FlushCommentLine(f, text, transform_styles, from_time, from_time+action_time, styleid) 482 | except (IndexError, ValueError) as e: 483 | logging.warning(_('Invalid comment: %r') % c[3]) 484 | 485 | 486 | # Result: (f, dx, dy) 487 | # To convert: NewX = f*x+dx, NewY = f*y+dy 488 | def GetZoomFactor(SourceSize, TargetSize): 489 | try: 490 | if (SourceSize, TargetSize) == GetZoomFactor.Cached_Size: 491 | return GetZoomFactor.Cached_Result 492 | except AttributeError: 493 | pass 494 | GetZoomFactor.Cached_Size = (SourceSize, TargetSize) 495 | try: 496 | SourceAspect = SourceSize[0]/SourceSize[1] 497 | TargetAspect = TargetSize[0]/TargetSize[1] 498 | if TargetAspect < SourceAspect: # narrower 499 | ScaleFactor = TargetSize[0]/SourceSize[0] 500 | GetZoomFactor.Cached_Result = (ScaleFactor, 0, (TargetSize[1]-TargetSize[0]/SourceAspect)/2) 501 | elif TargetAspect > SourceAspect: # wider 502 | ScaleFactor = TargetSize[1]/SourceSize[1] 503 | GetZoomFactor.Cached_Result = (ScaleFactor, (TargetSize[0]-TargetSize[1]*SourceAspect)/2, 0) 504 | else: 505 | GetZoomFactor.Cached_Result = (TargetSize[0]/SourceSize[0], 0, 0) 506 | return GetZoomFactor.Cached_Result 507 | except ZeroDivisionError: 508 | GetZoomFactor.Cached_Result = (1, 0, 0) 509 | return GetZoomFactor.Cached_Result 510 | 511 | 512 | # Calculation is based on https://github.com/jabbany/CommentCoreLibrary/issues/5#issuecomment-40087282 513 | # and https://github.com/m13253/danmaku2ass/issues/7#issuecomment-41489422 514 | # ASS FOV = width*4/3.0 515 | # But Flash FOV = width/math.tan(100*math.pi/360.0)/2 will be used instead 516 | # Result: (transX, transY, rotX, rotY, rotZ, scaleX, scaleY) 517 | def ConvertFlashRotation(rotY, rotZ, X, Y, width, height): 518 | def WrapAngle(deg): 519 | return 180-((180-deg) % 360) 520 | rotY = WrapAngle(rotY) 521 | rotZ = WrapAngle(rotZ) 522 | if rotY in (90, -90): 523 | rotY -= 1 524 | if rotY == 0 or rotZ == 0: 525 | outX = 0 526 | outY = -rotY # Positive value means clockwise in Flash 527 | outZ = -rotZ 528 | rotY *= math.pi/180.0 529 | rotZ *= math.pi/180.0 530 | else: 531 | rotY *= math.pi/180.0 532 | rotZ *= math.pi/180.0 533 | outY = math.atan2(-math.sin(rotY)*math.cos(rotZ), math.cos(rotY))*180/math.pi 534 | outZ = math.atan2(-math.cos(rotY)*math.sin(rotZ), math.cos(rotZ))*180/math.pi 535 | outX = math.asin(math.sin(rotY)*math.sin(rotZ))*180/math.pi 536 | trX = (X*math.cos(rotZ)+Y*math.sin(rotZ))/math.cos(rotY)+(1-math.cos(rotZ)/math.cos(rotY))*width/2-math.sin(rotZ)/math.cos(rotY)*height/2 537 | trY = Y*math.cos(rotZ)-X*math.sin(rotZ)+math.sin(rotZ)*width/2+(1-math.cos(rotZ))*height/2 538 | trZ = (trX-width/2)*math.sin(rotY) 539 | FOV = width*math.tan(2*math.pi/9.0)/2 540 | try: 541 | scaleXY = FOV/(FOV+trZ) 542 | except ZeroDivisionError: 543 | logging.error('Rotation makes object behind the camera: trZ == %.0f' % trZ) 544 | scaleXY = 1 545 | trX = (trX-width/2)*scaleXY+width/2 546 | trY = (trY-height/2)*scaleXY+height/2 547 | if scaleXY < 0: 548 | scaleXY = -scaleXY 549 | outX += 180 550 | outY += 180 551 | logging.error('Rotation makes object behind the camera: trZ == %.0f < %.0f' % (trZ, FOV)) 552 | return (trX, trY, WrapAngle(outX), WrapAngle(outY), WrapAngle(outZ), scaleXY*100, scaleXY*100) 553 | 554 | 555 | def ProcessComments(comments, f, width, height, bottomReserved, fontface, fontsize, alpha, duration_marquee, duration_still, reduced, progress_callback): 556 | styleid = 'Danmaku2ASS_%04x' % random.randint(0, 0xffff) 557 | WriteASSHead(f, width, height, fontface, fontsize, alpha, styleid) 558 | rows = [[None]*(height-bottomReserved+1) for i in range(4)] 559 | for idx, i in enumerate(comments): 560 | if progress_callback and idx % 1000 == 0: 561 | progress_callback(idx, len(comments)) 562 | if isinstance(i[4], int): 563 | row = 0 564 | rowmax = height-bottomReserved-i[7] 565 | while row <= rowmax: 566 | freerows = TestFreeRows(rows, i, row, width, height, bottomReserved, duration_marquee, duration_still) 567 | if freerows >= i[7]: 568 | MarkCommentRow(rows, i, row) 569 | WriteComment(f, i, row, width, height, bottomReserved, fontsize, duration_marquee, duration_still, styleid) 570 | break 571 | else: 572 | row += freerows or 1 573 | else: 574 | if not reduced: 575 | row = FindAlternativeRow(rows, i, height, bottomReserved) 576 | MarkCommentRow(rows, i, row) 577 | WriteComment(f, i, row, width, height, bottomReserved, fontsize, duration_marquee, duration_still, styleid) 578 | elif i[4] == 'bilipos': 579 | WriteCommentBilibiliPositioned(f, i, width, height, styleid) 580 | elif i[4] == 'acfunpos': 581 | WriteCommentAcfunPositioned(f, i, width, height, styleid) 582 | elif i[4] == 'sH5Vpos': 583 | WriteCommentSH5VPositioned(f, i, width, height, styleid) 584 | else: 585 | logging.warning(_('Invalid comment: %r') % i[3]) 586 | if progress_callback: 587 | progress_callback(len(comments), len(comments)) 588 | 589 | 590 | def TestFreeRows(rows, c, row, width, height, bottomReserved, duration_marquee, duration_still): 591 | res = 0 592 | rowmax = height-bottomReserved 593 | targetRow = None 594 | if c[4] in (1, 2): 595 | while row < rowmax and res < c[7]: 596 | if targetRow != rows[c[4]][row]: 597 | targetRow = rows[c[4]][row] 598 | if targetRow and targetRow[0]+duration_still > c[0]: 599 | break 600 | row += 1 601 | res += 1 602 | else: 603 | try: 604 | thresholdTime = c[0]-duration_marquee*(1-width/(c[8]+width)) 605 | except ZeroDivisionError: 606 | thresholdTime = c[0]-duration_marquee 607 | while row < rowmax and res < c[7]: 608 | if targetRow != rows[c[4]][row]: 609 | targetRow = rows[c[4]][row] 610 | try: 611 | if targetRow and (targetRow[0] > thresholdTime or targetRow[0]+targetRow[8]*duration_marquee/(targetRow[8]+width) > c[0]): 612 | break 613 | except ZeroDivisionError: 614 | pass 615 | row += 1 616 | res += 1 617 | return res 618 | 619 | 620 | def FindAlternativeRow(rows, c, height, bottomReserved): 621 | res = 0 622 | for row in range(height-bottomReserved-math.ceil(c[7])): 623 | if not rows[c[4]][row]: 624 | return row 625 | elif rows[c[4]][row][0] < rows[c[4]][res][0]: 626 | res = row 627 | return res 628 | 629 | 630 | def MarkCommentRow(rows, c, row): 631 | try: 632 | for i in range(row, row+math.ceil(c[7])): 633 | rows[c[4]][i] = c 634 | except IndexError: 635 | pass 636 | 637 | 638 | def WriteASSHead(f, width, height, fontface, fontsize, alpha, styleid): 639 | f.write( 640 | ''' 641 | [Script Info] 642 | ; Script generated by Danmaku2ASS 643 | ; https://github.com/m13253/danmaku2ass 644 | Script Updated By: Danmaku2ASS (https://github.com/m13253/danmaku2ass) 645 | ScriptType: v4.00+ 646 | PlayResX: %(width)d 647 | PlayResY: %(height)d 648 | Aspect Ratio: %(width)d:%(height)d 649 | Collisions: Normal 650 | WrapStyle: 2 651 | ScaledBorderAndShadow: yes 652 | YCbCr Matrix: TV.601 653 | 654 | [V4+ Styles] 655 | Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding 656 | Style: %(styleid)s, %(fontface)s, %(fontsize).0f, &H%(alpha)02XFFFFFF, &H%(alpha)02XFFFFFF, &H%(alpha)02X000000, &H%(alpha)02X000000, 0, 0, 0, 0, 100, 100, 0.00, 0.00, 1, %(outline).0f, 0, 7, 0, 0, 0, 0 657 | 658 | [Events] 659 | Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text 660 | ''' % {'width': width, 'height': height, 'fontface': fontface, 'fontsize': fontsize, 'alpha': 255-round(alpha*255), 'outline': max(fontsize/25.0, 1), 'styleid': styleid} 661 | ) 662 | 663 | 664 | def WriteComment(f, c, row, width, height, bottomReserved, fontsize, duration_marquee, duration_still, styleid): 665 | text = ASSEscape(c[3]) 666 | styles = [] 667 | if c[4] == 1: 668 | styles.append('\\an8\\pos(%(halfwidth)d, %(row)d)' % {'halfwidth': width/2, 'row': row}) 669 | duration = duration_still 670 | elif c[4] == 2: 671 | styles.append('\\an2\\pos(%(halfwidth)d, %(row)d)' % {'halfwidth': width/2, 'row': ConvertType2(row, height, bottomReserved)}) 672 | duration = duration_still 673 | elif c[4] == 3: 674 | styles.append('\\move(%(neglen)d, %(row)d, %(width)d, %(row)d)' % {'width': width, 'row': row, 'neglen': -math.ceil(c[8])}) 675 | duration = duration_marquee 676 | else: 677 | styles.append('\\move(%(width)d, %(row)d, %(neglen)d, %(row)d)' % {'width': width, 'row': row, 'neglen': -math.ceil(c[8])}) 678 | duration = duration_marquee 679 | if not (-1 < c[6]-fontsize < 1): 680 | styles.append('\\fs%.0f' % c[6]) 681 | if c[5] != 0xffffff: 682 | styles.append('\\c&H%s&' % ConvertColor(c[5])) 683 | if c[5] == 0x000000: 684 | styles.append('\\3c&HFFFFFF&') 685 | f.write('Dialogue: 2,%(start)s,%(end)s,%(styleid)s,,0000,0000,0000,,{%(styles)s}%(text)s\n' % {'start': ConvertTimestamp(c[0]), 'end': ConvertTimestamp(c[0]+duration), 'styles': ''.join(styles), 'text': text, 'styleid': styleid}) 686 | 687 | 688 | def ASSEscape(s): 689 | def ReplaceLeadingSpace(s): 690 | sstrip = s.strip(' ') 691 | slen = len(s) 692 | if slen == len(sstrip): 693 | return s 694 | else: 695 | llen = slen-len(s.lstrip(' ')) 696 | rlen = slen-len(s.rstrip(' ')) 697 | return ''.join(('\u2007'*llen, sstrip, '\u2007'*rlen)) 698 | return '\\N'.join((ReplaceLeadingSpace(i) or ' ' for i in str(s).replace('\\', '\\\\').replace('{', '\\{').replace('}', '\\}').split('\n'))) 699 | 700 | 701 | def CalculateLength(s): 702 | return max(map(len, s.split('\n'))) # May not be accurate 703 | 704 | 705 | def ConvertTimestamp(timestamp): 706 | timestamp = round(timestamp*100.0) 707 | hour, minute = divmod(timestamp, 360000) 708 | minute, second = divmod(minute, 6000) 709 | second, centsecond = divmod(second, 100) 710 | return '%d:%02d:%02d.%02d' % (int(hour), int(minute), int(second), int(centsecond)) 711 | 712 | 713 | def ConvertColor(RGB, width=1280, height=576): 714 | if RGB == 0x000000: 715 | return '000000' 716 | elif RGB == 0xffffff: 717 | return 'FFFFFF' 718 | R = (RGB >> 16) & 0xff 719 | G = (RGB >> 8) & 0xff 720 | B = RGB & 0xff 721 | if width < 1280 and height < 576: 722 | return '%02X%02X%02X' % (B, G, R) 723 | else: # VobSub always uses BT.601 colorspace, convert to BT.709 724 | ClipByte = lambda x: 255 if x > 255 else 0 if x < 0 else round(x) 725 | return '%02X%02X%02X' % ( 726 | ClipByte(R*0.00956384088080656+G*0.03217254540203729+B*0.95826361371715607), 727 | ClipByte(R*-0.10493933142075390+G*1.17231478191855154+B*-0.06737545049779757), 728 | ClipByte(R*0.91348912373987645+G*0.07858536372532510+B*0.00792551253479842) 729 | ) 730 | 731 | 732 | def ConvertType2(row, height, bottomReserved): 733 | return height-bottomReserved-row 734 | 735 | 736 | def ConvertToFile(filename_or_file, *args, **kwargs): 737 | if isinstance(filename_or_file, bytes): 738 | filename_or_file = str(bytes(filename_or_file).decode('utf-8', 'replace')) 739 | if isinstance(filename_or_file, str): 740 | return open(filename_or_file, *args, **kwargs) 741 | else: 742 | return filename_or_file 743 | 744 | 745 | def FilterBadChars(f): 746 | s = f.read() 747 | s = re.sub('[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f]', '\ufffd', s) 748 | return io.StringIO(s) 749 | 750 | 751 | class safe_list(list): 752 | def get(self, index, default=None): 753 | try: 754 | return self[index] 755 | except IndexError: 756 | return default 757 | 758 | 759 | def export(func): 760 | global __all__ 761 | try: 762 | __all__.append(func.__name__) 763 | except NameError: 764 | __all__ = [func.__name__] 765 | return func 766 | 767 | 768 | @export 769 | def Danmaku2ASS(input_files, output_file, stage_width, stage_height, reserve_blank=0, font_face=_('(FONT) sans-serif')[7:], font_size=25.0, text_opacity=1.0, duration_marquee=5.0, duration_still=5.0, is_reduce_comments=False, progress_callback=None): 770 | fo = None 771 | comments = ReadComments(input_files, font_size) 772 | try: 773 | if output_file: 774 | fo = ConvertToFile(output_file, 'w', encoding='utf-8-sig', errors='replace', newline='\r\n') 775 | else: 776 | fo = sys.stdout 777 | ProcessComments(comments, fo, stage_width, stage_height, reserve_blank, font_face, font_size, text_opacity, duration_marquee, duration_still, is_reduce_comments, progress_callback) 778 | finally: 779 | if output_file and fo != output_file: 780 | fo.close() 781 | 782 | 783 | @export 784 | def ReadComments(input_files, font_size=25.0, progress_callback=None): 785 | if isinstance(input_files, bytes): 786 | input_files = str(bytes(input_files).decode('utf-8', 'replace')) 787 | if isinstance(input_files, str): 788 | input_files = [input_files] 789 | else: 790 | input_files = list(input_files) 791 | comments = [] 792 | for idx, i in enumerate(input_files): 793 | if progress_callback: 794 | progress_callback(idx, len(input_files)) 795 | with ConvertToFile(i, 'r', encoding='utf-8', errors='replace') as f: 796 | CommentProcessor = GetCommentProcessor(f) 797 | if not CommentProcessor: 798 | raise ValueError(_('Unknown comment file format: %s') % i) 799 | comments.extend(CommentProcessor(FilterBadChars(f), font_size)) 800 | if progress_callback: 801 | progress_callback(len(input_files), len(input_files)) 802 | comments.sort() 803 | return comments 804 | 805 | 806 | @export 807 | def GetCommentProcessor(input_file): 808 | return CommentFormatMap[ProbeCommentFormat(input_file)] 809 | 810 | 811 | def main(): 812 | logging.basicConfig(format='%(levelname)s: %(message)s') 813 | if len(sys.argv) == 1: 814 | sys.argv.append('--help') 815 | parser = argparse.ArgumentParser() 816 | parser.add_argument('-o', '--output', metavar=_('OUTPUT'), help=_('Output file')) 817 | parser.add_argument('-s', '--size', metavar=_('WIDTHxHEIGHT'), required=True, help=_('Stage size in pixels')) 818 | parser.add_argument('-fn', '--font', metavar=_('FONT'), help=_('Specify font face [default: %s]') % _('(FONT) sans-serif')[7:], default=_('(FONT) sans-serif')[7:]) 819 | parser.add_argument('-fs', '--fontsize', metavar=_('SIZE'), help=(_('Default font size [default: %s]') % 25), type=float, default=25.0) 820 | parser.add_argument('-a', '--alpha', metavar=_('ALPHA'), help=_('Text opacity'), type=float, default=1.0) 821 | parser.add_argument('-dm', '--duration-marquee', metavar=_('SECONDS'), help=_('Duration of scrolling comment display [default: %s]') % 5, type=float, default=5.0) 822 | parser.add_argument('-ds', '--duration-still', metavar=_('SECONDS'), help=_('Duration of still comment display [default: %s]') % 5, type=float, default=5.0) 823 | parser.add_argument('-p', '--protect', metavar=_('HEIGHT'), help=_('Reserve blank on the bottom of the stage'), type=int, default=0) 824 | parser.add_argument('-r', '--reduce', action='store_true', help=_('Reduce the amount of comments if stage is full')) 825 | parser.add_argument('file', metavar=_('FILE'), nargs='+', help=_('Comment file to be processed')) 826 | args = parser.parse_args() 827 | try: 828 | width, height = str(args.size).split('x', 1) 829 | width = int(width) 830 | height = int(height) 831 | except ValueError: 832 | raise ValueError(_('Invalid stage size: %r') % args.size) 833 | Danmaku2ASS(args.file, args.output, width, height, args.protect, args.font, args.fontsize, args.alpha, args.duration_marquee, args.duration_still, args.reduce) 834 | 835 | 836 | if __name__ == '__main__': 837 | main() --------------------------------------------------------------------------------