├── .gitignore ├── README.md ├── modules ├── __init__.py └── example │ ├── __init__.py │ └── interface.py ├── moduletesting.py ├── orpheus.py ├── orpheus ├── __init__.py ├── core.py ├── housekeeping.py ├── music_downloader.py └── tagging.py ├── requirements.txt └── utils ├── __init__.py ├── exceptions.py ├── models.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | venv 3 | .vscode 4 | .DS_Store 5 | .idea/ 6 | config/ 7 | downloads/ 8 | temp/ 9 | .python-version 10 | modules/ 11 | extensions/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | OrpheusDL 6 | ========= 7 | 8 | A modular music archival program 9 | 10 | [Report Bug](https://github.com/OrfiTeam/OrpheusDL/issues) 11 | · 12 | [Request Feature](https://github.com/OrfiTeam/OrpheusDL/issues) 13 | 14 | 15 | ## Table of content 16 | 17 | - [About OrpheusDL](#about-orpheusdl) 18 | - [Getting Started](#getting-started) 19 | - [Prerequisites](#prerequisites) 20 | - [Installation](#installation) 21 | - [Usage](#usage) 22 | - [Configuration](#configuration) 23 | - [Global/Formatting](#globalformatting) 24 | - [Format variables](#format-variables) 25 | - [Contact](#contact) 26 | - [Acknowledgements](#acknowledgements) 27 | 28 | 29 | 30 | 31 | ## About OrpheusDL 32 | 33 | OrpheusDL is a modular music archival tool written in Python which allows archiving from multiple different services. 34 | 35 | 36 | 37 | ## Getting Started 38 | 39 | Follow these steps to get a local copy of Orpheus up and running: 40 | 41 | ### Prerequisites 42 | 43 | * Python 3.7+ (due to the requirement of dataclasses), though Python 3.9 is highly recommended 44 | 45 | ### Installation 46 | 47 | 1. Clone the repo 48 | ```shell 49 | git clone https://github.com/OrfiTeam/OrpheusDL.git && cd OrpheusDL 50 | ``` 51 | 2. Install all requirements 52 | ```shell 53 | pip install -r requirements.txt 54 | ``` 55 | 3. Run the program at least once, or use this command to create the settings file 56 | ```shell 57 | python3 orpheus.py settings refresh 58 | ``` 59 | 4. Enter your credentials in `config/settings.json` 60 | 61 | 62 | ## Usage 63 | 64 | Just call `orpheus.py` with any link you want to archive, for example Qobuz: 65 | ```shell 66 | python3 orpheus.py https://open.qobuz.com/album/c9wsrrjh49ftb 67 | ``` 68 | 69 | Alternatively do a search (luckysearch to automatically select the first option): 70 | ```shell 71 | python3 orpheus.py search qobuz track darkside alan walker 72 | ``` 73 | 74 | Or if you have the ID of what you want to download, use: 75 | ```shell 76 | python3 orpheus.py download qobuz track 52151405 77 | ``` 78 | 79 | 80 | ## Configuration 81 | 82 | You can customize every module from Orpheus individually and also set general/global settings which are active in every 83 | loaded module. You'll find the configuration file here: `config/settings.json` 84 | 85 | ### Global/General 86 | ```json5 87 | { 88 | "download_path": "./downloads/", 89 | "download_quality": "hifi", 90 | "search_limit": 10 91 | } 92 | ``` 93 | 94 | `download_path`: Set the absolute or relative output path with `/` as the delimiter 95 | 96 | `download_quality`: Choose one of the following settings: 97 | * "hifi": FLAC higher than 44.1/16 if available 98 | * "lossless": FLAC with 44.1/16 if available 99 | * "high": lossy codecs such as MP3, AAC, ... in a higher bitrate 100 | * "medium": lossy codecs such as MP3, AAC, ... in a medium bitrate 101 | * "low": lossy codecs such as MP3, AAC, ... in a lower bitrate 102 | 103 | **NOTE: The `download_quality` really depends on the used modules, so check out the modules README.md** 104 | 105 | `search_limit`: How many search results are shown 106 | 107 | 108 | ### Global/Formatting: 109 | 110 | ```json5 111 | { 112 | "album_format": "{name}{explicit}", 113 | "playlist_format": "{name}{explicit}", 114 | "track_filename_format": "{track_number}. {name}", 115 | "single_full_path_format": "{name}", 116 | "enable_zfill": true, 117 | "force_album_format": false 118 | } 119 | ``` 120 | 121 | `track_filename_format`: How tracks are formatted in albums and playlists. The relevant extension is appended to the end. 122 | 123 | `album_format`, `playlist_format`, `artist_format`: Base directories for their respective formats - tracks and cover 124 | art are stored here. May have slashes in it, for instance {artist}/{album}. 125 | 126 | `single_full_path_format`: How singles are handled, which is separate to how the above work. 127 | Instead, this has both the folder's name and the track's name. 128 | 129 | `enable_zfill`: Enables zero padding for `track_number`, `total_tracks`, `disc_number`, `total_discs` if the 130 | corresponding number has more than 2 digits 131 | 132 | `force_album_format`: Forces the `album_format` for tracks instead of the `single_full_path_format` and also 133 | uses `album_format` in the `playlist_format` folder 134 | 135 | 136 | #### Format variables 137 | 138 | `track_filename_format` variables are `{name}`, `{album}`, `{album_artist}`, `{album_id}`, `{track_number}`, 139 | `{total_tracks}`, `{disc_number}`, `{total_discs}`, `{release_date}`, `{release_year}`, `{artist_id}`, `{isrc}`, 140 | `{upc}`, `{explicit}`, `{copyright}`, `{codec}`, `{sample_rate}`, `{bit_depth}`. 141 | 142 | `album_format` variables are `{name}`, `{id}`, `{artist}`, `{artist_id}`, `{release_year}`, `{upc}`, `{explicit}`, 143 | `{quality}`, `{artist_initials}`. 144 | 145 | `playlist_format` variables are `{name}`, `{creator}`, `{tracks}`, `{release_year}`, `{explicit}`, `{creator_id}` 146 | 147 | * `{quality}` will add 148 | ``` 149 | [Dolby Atmos] 150 | [96kHz 24bit] 151 | [M] 152 | ``` 153 | to the corresponding path (depending on the module) 154 | * `{explicit}` will add 155 | ``` 156 | [E] 157 | ``` 158 | to the corresponding path 159 | 160 | ### Global/Covers 161 | 162 | ```json5 163 | { 164 | "embed_cover": true, 165 | "main_compression": "high", 166 | "main_resolution": 1400, 167 | "save_external": false, 168 | "external_format": "png", 169 | "external_compression": "low", 170 | "external_resolution": 3000, 171 | "save_animated_cover": true 172 | } 173 | ``` 174 | 175 | | Option | Info | 176 | |----------------------|------------------------------------------------------------------------------------------| 177 | | embed_cover | Enable it to embed the album cover inside every track | 178 | | main_compression | Compression of the main cover | 179 | | main_resolution | Resolution (in pixels) of the cover of the module used | 180 | | save_external | Enable it to save the cover from a third party cover module | 181 | | external_format | Format of the third party cover, supported values: `jpg`, `png`, `webp` | 182 | | external_compression | Compression of the third party cover, supported values: `low`, `high` | 183 | | external_resolution | Resolution (in pixels) of the third party cover | 184 | | save_animated_cover | Enable saving the animated cover when supported from the module (often in MPEG-4 format) | 185 | 186 | ### Global/Codecs 187 | 188 | ```json5 189 | { 190 | "proprietary_codecs": false, 191 | "spatial_codecs": true 192 | } 193 | ``` 194 | 195 | `proprietary_codecs`: Enable it to allow `MQA`, `E-AC-3 JOC` or `AC-4 IMS` 196 | 197 | `spatial_codecs`: Enable it to allow `MPEG-H 3D`, `E-AC-3 JOC` or `AC-4 IMS` 198 | 199 | **Note: `spatial_codecs` has priority over `proprietary_codecs` when deciding if a codec is enabled** 200 | 201 | ### Global/Module_defaults 202 | 203 | ```json5 204 | { 205 | "lyrics": "default", 206 | "covers": "default", 207 | "credits": "default" 208 | } 209 | ``` 210 | 211 | Change `default` to the module name under `/modules` in order to retrieve `lyrics`, `covers` or `credits` from the 212 | selected module 213 | 214 | ### Global/Lyrics 215 | ```json5 216 | { 217 | "embed_lyrics": true, 218 | "embed_synced_lyrics": false, 219 | "save_synced_lyrics": true 220 | } 221 | ``` 222 | 223 | | Option | Info | 224 | |---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------| 225 | | embed_lyrics | Embeds the (unsynced) lyrics inside every track | 226 | | embed_synced_lyrics | Embeds the synced lyrics inside every track (needs `embed_lyrics` to be enabled) (required for [Roon](https://community.roonlabs.com/t/1-7-lyrics-tag-guide/85182)) | 227 | | save_synced_lyrics | Saves the synced lyrics inside a `.lrc` file in the same directory as the track with the same `track_format` variables | 228 | 229 | 230 | ## Contact 231 | 232 | OrfiDev (Project Lead) - [@OrfiDev](https://github.com/OrfiTeam) 233 | 234 | Dniel97 (Current Lead Developer) - [@Dniel97](https://github.com/Dniel97) 235 | 236 | Project Link: [Orpheus Public GitHub Repository](https://github.com/OrfiTeam/OrpheusDL) 237 | 238 | 239 | 240 | 241 | ## Acknowledgements 242 | * Chimera by Aesir - the inspiration to the project 243 | * [Icon modified from a freepik image](https://www.freepik.com/) 244 | -------------------------------------------------------------------------------- /modules/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OrfiTeam/OrpheusDL/a45ff47913508d4c09971bdb847d5845984f1e64/modules/__init__.py -------------------------------------------------------------------------------- /modules/example/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OrfiTeam/OrpheusDL/a45ff47913508d4c09971bdb847d5845984f1e64/modules/example/__init__.py -------------------------------------------------------------------------------- /modules/example/interface.py: -------------------------------------------------------------------------------- 1 | from utils.models import * 2 | from utils.utils import create_temp_filename 3 | 4 | 5 | module_information = ModuleInformation( # Only service_name and module_supported_modes are mandatory 6 | service_name = 'Example', 7 | module_supported_modes = ModuleModes.download | ModuleModes.lyrics | ModuleModes.covers | ModuleModes.credits, 8 | flags = ModuleFlags.hidden, 9 | # Flags: 10 | # startup_load: load module on startup 11 | # hidden: hides module from CLI help options 12 | # jwt_system_enable: handles bearer and refresh tokens automatically, though currently untested 13 | # private: override any public modules, only enabled with the -p/--private argument, currently broken 14 | global_settings = {}, 15 | global_storage_variables = [], 16 | session_settings = {}, 17 | session_storage_variables = ['access_token'], 18 | netlocation_constant = 'example', 19 | test_url = 'https://player.example.com/track/idhere', 20 | url_constants = { # This is the default if no url_constants is given. Unused if custom_url_parsing is flagged 21 | 'track': DownloadTypeEnum.track, 22 | 'album': DownloadTypeEnum.album, 23 | 'playlist': DownloadTypeEnum.playlist, 24 | 'artist': DownloadTypeEnum.artist 25 | }, # How this works: if '/track/' is detected in the URL, then track downloading is triggered 26 | login_behaviour = ManualEnum.manual, # setting to ManualEnum.manual disables Orpheus automatically calling login() when needed 27 | url_decoding = ManualEnum.orpheus # setting to ManualEnum.manual disables Orpheus' automatic url decoding which works as follows: 28 | # taking the url_constants dict as a list of constants to check for in the url's segments, and the final part of the URL as the ID 29 | ) 30 | 31 | 32 | class ModuleInterface: 33 | def __init__(self, module_controller: ModuleController): 34 | settings = module_controller.module_settings 35 | self.session = (settings['app_id'], settings['app_secret']) # API class goes here 36 | self.session.auth_token = module_controller.temporary_settings_controller.read('access_token') 37 | self.module_controller = module_controller 38 | 39 | self.quality_parse = { 40 | QualityEnum.MINIMUM: 0, 41 | QualityEnum.LOW: 1, 42 | QualityEnum.MEDIUM: 2, 43 | QualityEnum.HIGH: 3, 44 | QualityEnum.LOSSLESS: 4, 45 | QualityEnum.HIFI: 5 46 | } 47 | if not module_controller.orpheus_options.disable_subscription_check and (self.quality_parse[module_controller.orpheus_options.quality_tier] > self.session.get_user_tier()): 48 | print('Example: quality set in the settings is not accessible by the current subscription') 49 | 50 | def login(self, email: str, password: str): # Called automatically by Orpheus when standard_login is flagged, otherwise optional 51 | token = self.session.login(email, password) 52 | self.session.auth_token = token 53 | self.module_controller.temporary_settings_controller.set('token', token) 54 | 55 | def get_track_info(self, track_id: str, quality_tier: QualityEnum, codec_options: CodecOptions, data={}) -> TrackInfo: # Mandatory 56 | quality_tier = self.quality_parse[quality_tier] 57 | track_data = data[track_id] if data and track_id in data else self.session.get_track(track_id) 58 | 59 | tags = Tags( # every single one of these is optional 60 | album_artist = '', 61 | composer = '', 62 | track_number = 1, 63 | total_tracks = 1, 64 | copyright = '', 65 | isrc = '', 66 | upc = '', 67 | disc_number = 1, # None/0/1 if no discs 68 | total_discs = 1, # None/0/1 if no discs 69 | replay_gain = 0.0, 70 | replay_peak = 0.0, 71 | genres = [], 72 | release_date = '1969-09-06' # Format: YYYY-MM-DD 73 | ) 74 | 75 | return TrackInfo( 76 | name = '', 77 | album_id = '', 78 | album = '', 79 | artists = [''], 80 | tags = tags, 81 | codec = CodecEnum.FLAC, 82 | cover_url = '', # make sure to check module_controller.orpheus_options.default_cover_options 83 | release_year = 2021, 84 | explicit = False, 85 | artist_id = '', # optional 86 | animated_cover_url = '', # optional 87 | description = '', # optional 88 | bit_depth = 16, # optional 89 | sample_rate = 44.1, # optional 90 | bitrate = 1411, # optional 91 | download_extra_kwargs = {'file_url': '', 'codec': ''}, # optional only if download_type isn't DownloadEnum.TEMP_FILE_PATH, whatever you want 92 | cover_extra_kwargs = {'data': {track_id: ''}}, # optional, whatever you want, but be very careful 93 | credits_extra_kwargs = {'data': {track_id: ''}}, # optional, whatever you want, but be very careful 94 | lyrics_extra_kwargs = {'data': {track_id: ''}}, # optional, whatever you want, but be very careful 95 | error = '' # only use if there is an error 96 | ) 97 | 98 | def get_track_download(self, file_url, codec): 99 | track_location = create_temp_filename() 100 | # Do magic here 101 | return TrackDownloadInfo( 102 | download_type = DownloadEnum.URL, 103 | file_url = '', # optional only if download_type isn't DownloadEnum.URL 104 | file_url_headers = {}, # optional 105 | temp_file_path = track_location 106 | ) 107 | 108 | def get_album_info(self, album_id: str, data={}) -> Optional[AlbumInfo]: # Mandatory if ModuleModes.download 109 | album_data = data[album_id] if album_id in data else self.session.get_album(album_id) 110 | 111 | return AlbumInfo( 112 | name = '', 113 | artist = '', 114 | tracks = [], 115 | release_year = '', 116 | explicit = False, 117 | artist_id = '', # optional 118 | booklet_url = '', # optional 119 | cover_url = '', # optional 120 | cover_type = ImageFileTypeEnum.jpg, # optional 121 | all_track_cover_jpg_url = '', # technically optional, but HIGHLY recommended 122 | animated_cover_url = '', # optional 123 | description = '', # optional 124 | track_extra_kwargs = {'data': ''} # optional, whatever you want 125 | ) 126 | 127 | def get_playlist_info(self, playlist_id: str, data={}) -> PlaylistInfo: # Mandatory if either ModuleModes.download or ModuleModes.playlist 128 | playlist_data = data[playlist_id] if playlist_id in data else self.session.get_playlist(playlist_id) 129 | 130 | return PlaylistInfo( 131 | name = '', 132 | creator = '', 133 | tracks = [], 134 | release_year = '', 135 | explicit = False, 136 | creator_id = '', # optional 137 | cover_url = '', # optional 138 | cover_type = ImageFileTypeEnum.jpg, # optional 139 | animated_cover_url = '', # optional 140 | description = '', # optional 141 | track_extra_kwargs = {'data': ''} # optional, whatever you want 142 | ) 143 | 144 | def get_artist_info(self, artist_id: str, get_credited_albums: bool) -> ArtistInfo: # Mandatory if ModuleModes.download 145 | # get_credited_albums means stuff like remix compilations the artist was part of 146 | artist_data = self.session.get_artist(artist_id) 147 | 148 | return ArtistInfo( 149 | name = '', 150 | albums = [], # optional 151 | album_extra_kwargs = {'data': ''}, # optional, whatever you want 152 | tracks = [], # optional 153 | track_extra_kwargs = {'data': ''} # optional, whatever you want 154 | ) 155 | 156 | def get_track_credits(self, track_id: str, data={}): # Mandatory if ModuleModes.credits 157 | track_data = data[track_id] if track_id in data else self.session.get_track(track_id) 158 | credits = track_data['credits'] 159 | credits_dict = {} 160 | return [CreditsInfo(k, v) for k, v in credits_dict.items()] 161 | 162 | def get_track_cover(self, track_id: str, cover_options: CoverOptions, data={}) -> CoverInfo: # Mandatory if ModuleModes.covers 163 | track_data = data[track_id] if track_id in data else self.session.get_track(track_id) 164 | cover_info = track_data['cover'] 165 | return CoverInfo(url='', file_type=ImageFileTypeEnum.jpg) 166 | 167 | def get_track_lyrics(self, track_id: str, data={}) -> LyricsInfo: # Mandatory if ModuleModes.lyrics 168 | track_data = data[track_id] if track_id in data else self.session.get_track(track_id) 169 | lyrics = track_data['lyrics'] 170 | return LyricsInfo(embedded='', synced='') # both optional if not found 171 | 172 | def search(self, query_type: DownloadTypeEnum, query: str, track_info: TrackInfo = None, limit: int = 10): # Mandatory 173 | results = {} 174 | if track_info and track_info.tags.isrc: 175 | results = self.session.search(query_type.name, track_info.tags.isrc, limit) 176 | if not results: 177 | results = self.session.search(query_type.name, query, limit) 178 | 179 | return [SearchResult( 180 | result_id = '', 181 | name = '', # optional only if a lyrics/covers only module 182 | artists = [], # optional only if a lyrics/covers only module or an artist search 183 | year = '', # optional 184 | explicit = False, # optional 185 | additional = [], # optional, used to convey more info when using orpheus.py search (not luckysearch, for obvious reasons) 186 | extra_kwargs = {'data': {i['id']: i}} # optional, whatever you want. NOTE: BE CAREFUL! this can be given to: 187 | # get_track_info, get_album_info, get_artist_info with normal search results, and 188 | # get_track_credits, get_track_cover, get_track_lyrics in the case of other modules using this module just for those. 189 | # therefore, it's recommended to choose something generic like 'data' rather than specifics like 'cover_info' 190 | # or, you could use both, keeping a data field just in case track data is given, while keeping the specifics, but that's overcomplicated 191 | ) for i in results] 192 | -------------------------------------------------------------------------------- /moduletesting.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import argparse, cProfile, pstats 4 | from orpheus.core import Orpheus 5 | 6 | def main(): 7 | parser = argparse.ArgumentParser(description='Orpheus Module Testing Tool') 8 | parser.add_argument('-pr', '--private', action='store_true', help='Enable private modules') 9 | parser.add_argument('-sp', '--save_profile', action='store_true', help='Save profiling for use with SnakeViz') 10 | parser.add_argument('-pp', '--print_profile', action='store_true', help='Print profiling (long output)') 11 | parser.add_argument('module') 12 | parser.add_argument('function') 13 | parser.add_argument('arguments', nargs='*') 14 | parsed_args = parser.parse_args() 15 | 16 | try: 17 | with cProfile.Profile() as pr: 18 | orpheus = Orpheus(parsed_args.private) 19 | if parsed_args.module.lower() not in orpheus.module_list: 20 | raise Exception(f'Module {parsed_args.module} either does not exist or mismatches private mode') 21 | module_instance = orpheus.load_module(parsed_args.module.lower()) 22 | requested_function = getattr(module_instance, parsed_args.function.lower(), None) 23 | if not requested_function: 24 | raise Exception(f'Function {parsed_args.function} does not exist') 25 | 26 | args, kwargs = [], {} 27 | for i in parsed_args.arguments: 28 | if '=' in i: 29 | item, value = i.split('=') 30 | kwargs[item] = value 31 | else: 32 | args.append(i) 33 | requested_function(*args, **kwargs) 34 | finally: 35 | stats = pstats.Stats(pr) 36 | stats.sort_stats(pstats.SortKey.TIME) 37 | stats.dump_stats(filename='orpheus_profiling.prof') if parsed_args.save_profile else None 38 | stats.print_stats() if parsed_args.print_profile else None 39 | 40 | if __name__ == "__main__": 41 | try: 42 | main() 43 | except KeyboardInterrupt: 44 | print('\n\t^C pressed - abort') 45 | exit() -------------------------------------------------------------------------------- /orpheus.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import argparse 4 | import re 5 | from urllib.parse import urlparse 6 | 7 | from orpheus.core import * 8 | from orpheus.music_downloader import beauty_format_seconds 9 | 10 | 11 | def main(): 12 | print(r''' 13 | ____ _ _____ _ 14 | / __ \ | | | __ \| | 15 | | | | |_ __ _ __ | |__ ___ _ _ ___| | | | | 16 | | | | | '__| '_ \| '_ \ / _ \ | | / __| | | | | 17 | | |__| | | | |_) | | | | __/ |_| \__ \ |__| | |____ 18 | \____/|_| | .__/|_| |_|\___|\__,_|___/_____/|______| 19 | | | 20 | |_| 21 | 22 | ''') 23 | 24 | help_ = 'Use "settings [option]" for orpheus controls (coreupdate, fullupdate, modinstall), "settings [module]' \ 25 | '[option]" for module specific options (update, test, setup), searching by "[search/luckysearch] [module]' \ 26 | '[track/artist/playlist/album] [query]", or just putting in urls. (you may need to wrap the URLs in double' \ 27 | 'quotes if you have issues downloading)' 28 | parser = argparse.ArgumentParser(description='Orpheus: modular music archival') 29 | parser.add_argument('-p', '--private', action='store_true', help=argparse.SUPPRESS) 30 | parser.add_argument('-o', '--output', help='Select a download output path. Default is the provided download path in config/settings.py') 31 | parser.add_argument('-lr', '--lyrics', default='default', help='Set module to get lyrics from') 32 | parser.add_argument('-cv', '--covers', default='default', help='Override module to get covers from') 33 | parser.add_argument('-cr', '--credits', default='default', help='Override module to get credits from') 34 | parser.add_argument('-sd', '--separatedownload', default='default', help='Select a different module that will download the playlist instead of the main module. Only for playlists.') 35 | parser.add_argument('arguments', nargs='*', help=help_) 36 | args = parser.parse_args() 37 | 38 | orpheus = Orpheus(args.private) 39 | if not args.arguments: 40 | parser.print_help() 41 | exit() 42 | 43 | orpheus_mode = args.arguments[0].lower() 44 | if orpheus_mode == 'settings': # These should call functions in a separate py file, that does not yet exist 45 | setting = args.arguments[1].lower() 46 | if setting == 'refresh': 47 | print('settings.json has been refreshed successfully.') 48 | return # Actually the only one that should genuinely return here after doing nothing 49 | elif setting == 'core_update': # Updates only Orpheus 50 | return # TODO 51 | elif setting == 'full_update': # Updates Orpheus and all modules 52 | return # TODO 53 | orpheus.update_setting_storage() 54 | elif setting == 'module_install': # Installs a module with git 55 | return # TODO 56 | orpheus.update_setting_storage() 57 | elif setting == 'test_modules': 58 | return # TODO 59 | elif setting in orpheus.module_list: 60 | orpheus.load_module(setting) 61 | modulesetting = args.arguments[2].lower() 62 | if modulesetting == 'update': 63 | return # TODO 64 | orpheus.update_setting_storage() 65 | elif modulesetting == 'setup': 66 | return # TODO 67 | elif modulesetting == 'adjust_setting': 68 | return # TODO 69 | #elif modulesetting in [custom settings function list] TODO (here so test can be replaced) 70 | elif modulesetting == 'test': # Almost equivalent to sessions test 71 | return # TODO 72 | else: 73 | raise Exception(f'Unknown setting "{modulesetting}" for module "{setting}"') 74 | else: 75 | raise Exception(f'Unknown setting: "{setting}"') 76 | elif orpheus_mode == 'sessions': 77 | module = args.arguments[1].lower() 78 | if module in orpheus.module_list: 79 | option = args.arguments[2].lower() 80 | if option == 'add': 81 | return # TODO 82 | elif option == 'delete': 83 | return # TODO 84 | elif option == 'list': 85 | return # TODO 86 | elif option == 'test': 87 | session_name = args.arguments[3].lower() 88 | if session_name == 'all': 89 | return # TODO 90 | else: 91 | return # TODO, will also have a check for if the requested session actually exists, obviously 92 | else: 93 | raise Exception(f'Unknown option {option}, choose add/delete/list/test') 94 | else: 95 | raise Exception(f'Unknown module {module}') # TODO: replace with InvalidModuleError 96 | else: 97 | path = args.output if args.output else orpheus.settings['global']['general']['download_path'] 98 | if path[-1] == '/': path = path[:-1] # removes '/' from end if it exists 99 | os.makedirs(path, exist_ok=True) 100 | 101 | media_types = '/'.join(i.name for i in DownloadTypeEnum) 102 | 103 | if orpheus_mode == 'search' or orpheus_mode == 'luckysearch': 104 | if len(args.arguments) > 3: 105 | modulename = args.arguments[1].lower() 106 | if modulename in orpheus.module_list: 107 | try: 108 | query_type = DownloadTypeEnum[args.arguments[2].lower()] 109 | except KeyError: 110 | raise Exception(f'{args.arguments[2].lower()} is not a valid search type! Choose {media_types}') 111 | lucky_mode = True if orpheus_mode == 'luckysearch' else False 112 | 113 | query = ' '.join(args.arguments[3:]) 114 | module = orpheus.load_module(modulename) 115 | items = module.search(query_type, query, limit = (1 if lucky_mode else orpheus.settings['global']['general']['search_limit'])) 116 | if len(items) == 0: 117 | raise Exception(f'No search results for {query_type.name}: {query}') 118 | 119 | if lucky_mode: 120 | selection = 0 121 | else: 122 | for index, item in enumerate(items, start=1): 123 | additional_details = '[E] ' if item.explicit else '' 124 | additional_details += f'[{beauty_format_seconds(item.duration)}] ' if item.duration else '' 125 | additional_details += f'[{item.year}] ' if item.year else '' 126 | additional_details += ' '.join([f'[{i}]' for i in item.additional]) if item.additional else '' 127 | if query_type is not DownloadTypeEnum.artist: 128 | artists = ', '.join(item.artists) if item.artists is list else item.artists 129 | print(f'{str(index)}. {item.name} - {", ".join(artists)} {additional_details}') 130 | else: 131 | print(f'{str(index)}. {item.name} {additional_details}') 132 | 133 | selection_input = input('Selection: ') 134 | if selection_input.lower() in ['e', 'q', 'x', 'exit', 'quit']: exit() 135 | if not selection_input.isdigit(): raise Exception('Input a number') 136 | selection = int(selection_input)-1 137 | if selection < 0 or selection >= len(items): raise Exception('Invalid selection') 138 | print() 139 | selected_item: SearchResult = items[selection] 140 | media_to_download = {modulename: [MediaIdentification(media_type=query_type, media_id=selected_item.result_id, extra_kwargs=selected_item.extra_kwargs)]} 141 | elif modulename == 'multi': 142 | return # TODO 143 | else: 144 | modules = [i for i in orpheus.module_list if ModuleFlags.hidden not in orpheus.module_settings[i].flags] 145 | raise Exception(f'Unknown module name "{modulename}". Must select from: {", ".join(modules)}') # TODO: replace with InvalidModuleError 146 | else: 147 | print(f'Search must be done as orpheus.py [search/luckysearch] [module] [{media_types}] [query]') 148 | exit() # TODO: replace with InvalidInput 149 | elif orpheus_mode == 'download': 150 | if len(args.arguments) > 3: 151 | modulename = args.arguments[1].lower() 152 | if modulename in orpheus.module_list: 153 | try: 154 | media_type = DownloadTypeEnum[args.arguments[2].lower()] 155 | except KeyError: 156 | raise Exception(f'{args.arguments[2].lower()} is not a valid download type! Choose {media_types}') 157 | media_to_download = {modulename: [MediaIdentification(media_type=media_type, media_id=i) for i in args.arguments[3:]]} 158 | else: 159 | modules = [i for i in orpheus.module_list if ModuleFlags.hidden not in orpheus.module_settings[i].flags] 160 | raise Exception(f'Unknown module name "{modulename}". Must select from: {", ".join(modules)}') # TODO: replace with InvalidModuleError 161 | else: 162 | print(f'Download must be done as orpheus.py [download] [module] [{media_types}] [media ID 1] [media ID 2] ...') 163 | exit() # TODO: replace with InvalidInput 164 | else: # if no specific modes are detected, parse as urls, but first try loading as a list of URLs 165 | arguments = tuple(open(args.arguments[0], 'r')) if len(args.arguments) == 1 and os.path.exists(args.arguments[0]) else args.arguments 166 | media_to_download = {} 167 | for link in arguments: 168 | if link.startswith('http'): 169 | url = urlparse(link) 170 | components = url.path.split('/') 171 | 172 | service_name = None 173 | for i in orpheus.module_netloc_constants: 174 | if re.findall(i, url.netloc): service_name = orpheus.module_netloc_constants[i] 175 | if not service_name: 176 | raise Exception(f'URL location "{url.netloc}" is not found in modules!') 177 | if service_name not in media_to_download: media_to_download[service_name] = [] 178 | 179 | if orpheus.module_settings[service_name].url_decoding is ManualEnum.manual: 180 | module = orpheus.load_module(service_name) 181 | media_to_download[service_name].append(module.custom_url_parse(link)) 182 | else: 183 | if not components or len(components) <= 2: 184 | print(f'\tInvalid URL: "{link}"') 185 | exit() # TODO: replace with InvalidInput 186 | 187 | url_constants = orpheus.module_settings[service_name].url_constants 188 | if not url_constants: 189 | url_constants = { 190 | 'track': DownloadTypeEnum.track, 191 | 'album': DownloadTypeEnum.album, 192 | 'playlist': DownloadTypeEnum.playlist, 193 | 'artist': DownloadTypeEnum.artist 194 | } 195 | 196 | type_matches = [media_type for url_check, media_type in url_constants.items() if url_check in components] 197 | 198 | if not type_matches: 199 | print(f'Invalid URL: "{link}"') 200 | exit() 201 | 202 | media_to_download[service_name].append(MediaIdentification(media_type=type_matches[-1], media_id=components[-1])) 203 | else: 204 | raise Exception(f'Invalid argument: "{link}"') 205 | 206 | # Prepare the third-party modules similar to above 207 | tpm = {ModuleModes.covers: '', ModuleModes.lyrics: '', ModuleModes.credits: ''} 208 | for i in tpm: 209 | moduleselected = getattr(args, i.name).lower() 210 | if moduleselected == 'default': 211 | moduleselected = orpheus.settings['global']['module_defaults'][i.name] 212 | if moduleselected == 'default': 213 | moduleselected = None 214 | tpm[i] = moduleselected 215 | sdm = args.separatedownload.lower() 216 | 217 | if not media_to_download: 218 | print('No links given') 219 | 220 | orpheus_core_download(orpheus, media_to_download, tpm, sdm, path) 221 | 222 | 223 | if __name__ == "__main__": 224 | try: 225 | main() 226 | except KeyboardInterrupt: 227 | print('\n\t^C pressed - abort') 228 | exit() 229 | -------------------------------------------------------------------------------- /orpheus/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OrfiTeam/OrpheusDL/a45ff47913508d4c09971bdb847d5845984f1e64/orpheus/__init__.py -------------------------------------------------------------------------------- /orpheus/core.py: -------------------------------------------------------------------------------- 1 | import importlib, json, logging, os, pickle, requests, urllib3, base64, shutil 2 | from datetime import datetime 3 | 4 | from orpheus.music_downloader import Downloader 5 | from utils.models import * 6 | from utils.utils import * 7 | from utils.exceptions import * 8 | 9 | os.environ['CURL_CA_BUNDLE'] = '' # Hack to disable SSL errors for requests module for easier debugging 10 | urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) # Make SSL warnings hidden 11 | 12 | # try: 13 | # time_request = requests.get('https://github.com') # to be replaced with something useful, like an Orpheus updates json 14 | # except: 15 | # print('Could not reach the internet, quitting') 16 | # exit() 17 | 18 | # timestamp_correction_term = int(datetime.strptime(time_request.headers['Date'], '%a, %d %b %Y %H:%M:%S GMT').timestamp() - datetime.utcnow().timestamp()) 19 | # if abs(timestamp_correction_term) > 60*60*24: 20 | # print('System time is incorrect, using online time to correct it for subscription expiry checks') 21 | 22 | timestamp_correction_term = 0 23 | # Use the same Oprinter instance wherever it's needed 24 | oprinter = Oprinter() 25 | 26 | 27 | def true_current_utc_timestamp(): 28 | return int(datetime.utcnow().timestamp()) + timestamp_correction_term 29 | 30 | 31 | class Orpheus: 32 | def __init__(self, private_mode=False): 33 | self.extensions, self.extension_list, self.module_list, self.module_settings, self.module_netloc_constants, self.loaded_modules = {}, set(), set(), {}, {}, {} 34 | 35 | self.default_global_settings = { 36 | "general": { 37 | "download_path": "./downloads/", 38 | "download_quality": "hifi", 39 | "search_limit": 10 40 | }, 41 | "artist_downloading":{ 42 | "return_credited_albums": True, 43 | "separate_tracks_skip_downloaded": True 44 | }, 45 | "formatting": { 46 | "album_format": "{name}{explicit}", 47 | "playlist_format": "{name}{explicit}", 48 | "track_filename_format": "{track_number}. {name}", 49 | "single_full_path_format": "{name}", 50 | "enable_zfill": True, 51 | "force_album_format": False 52 | }, 53 | "codecs": { 54 | "proprietary_codecs": False, 55 | "spatial_codecs": True 56 | }, 57 | "module_defaults": { 58 | "lyrics": "default", 59 | "covers": "default", 60 | "credits": "default" 61 | }, 62 | "lyrics": { 63 | "embed_lyrics": True, 64 | "embed_synced_lyrics": False, 65 | "save_synced_lyrics": True 66 | }, 67 | "covers": { 68 | "embed_cover": True, 69 | "main_compression": "high", 70 | "main_resolution": 1400, 71 | "save_external": False, 72 | "external_format": 'png', 73 | "external_compression": "low", 74 | "external_resolution": 3000, 75 | "save_animated_cover": True 76 | }, 77 | "playlist": { 78 | "save_m3u": True, 79 | "paths_m3u": "absolute", 80 | "extended_m3u": True 81 | }, 82 | "advanced": { 83 | "advanced_login_system": False, 84 | "codec_conversions": { 85 | "alac": "flac", 86 | "wav": "flac" 87 | }, 88 | "conversion_flags": { 89 | "flac": { 90 | "compression_level": "5" 91 | } 92 | }, 93 | "conversion_keep_original": False, 94 | "cover_variance_threshold": 8, 95 | "debug_mode": False, 96 | "disable_subscription_checks": False, 97 | "enable_undesirable_conversions": False, 98 | "ignore_existing_files": False, 99 | "ignore_different_artists": True 100 | } 101 | } 102 | 103 | self.data_folder_base = 'config' 104 | self.settings_location = os.path.join(self.data_folder_base, 'settings.json') 105 | self.session_storage_location = os.path.join(self.data_folder_base, 'loginstorage.bin') 106 | 107 | os.makedirs('config', exist_ok=True) 108 | self.settings = json.loads(open(self.settings_location, 'r').read()) if os.path.exists(self.settings_location) else {} 109 | 110 | try: 111 | if self.settings['global']['advanced']['debug_mode']: logging.basicConfig(level=logging.DEBUG) 112 | except KeyError: 113 | pass 114 | 115 | os.makedirs('extensions', exist_ok=True) 116 | for extension in os.listdir('extensions'): # Loading extensions 117 | if os.path.isdir(f'extensions/{extension}') and os.path.exists(f'extensions/{extension}/interface.py'): 118 | class_ = getattr(importlib.import_module(f'extensions.{extension}.interface'), 'OrpheusExtension', None) 119 | if class_: 120 | self.extension_list.add(extension) 121 | logging.debug(f'Orpheus: {extension} extension detected') 122 | else: 123 | raise Exception('Error loading extension: "{extension}"') 124 | 125 | # Module preparation (not loaded yet for performance purposes) 126 | os.makedirs('modules', exist_ok=True) 127 | module_list = [module.lower() for module in os.listdir('modules') if os.path.exists(f'modules/{module}/interface.py')] 128 | if not module_list or module_list == ['example']: 129 | print('No modules are installed, quitting') 130 | exit() 131 | logging.debug('Orpheus: Modules detected: ' + ", ".join(module_list)) 132 | 133 | for module in module_list: # Loading module information into module_settings 134 | module_information: ModuleInformation = getattr(importlib.import_module(f'modules.{module}.interface'), 'module_information', None) 135 | if module_information and not ModuleFlags.private in module_information.flags and not private_mode: 136 | self.module_list.add(module) 137 | self.module_settings[module] = module_information 138 | logging.debug(f'Orpheus: {module} added as a module') 139 | else: 140 | raise Exception(f'Error loading module information from module: "{module}"') # TODO: replace with InvalidModuleError 141 | 142 | duplicates = set() 143 | for module in self.module_list: # Detecting duplicate url constants 144 | module_info: ModuleInformation = self.module_settings[module] 145 | url_constants = module_info.netlocation_constant 146 | if not isinstance(url_constants, list): url_constants = [str(url_constants)] 147 | for constant in url_constants: 148 | if constant.startswith('setting.'): 149 | if self.settings.get('modules') and self.settings['modules'].get(module): 150 | constant = self.settings['modules'][module][constant.split('setting.')[1]] 151 | else: 152 | constant = None 153 | 154 | if constant: 155 | if constant not in self.module_netloc_constants: 156 | self.module_netloc_constants[constant] = module 157 | elif ModuleFlags.private in module_info.flags: # Replacing public modules with private ones 158 | if ModuleFlags.private in self.module_settings[constant].flags: duplicates.add(constant) 159 | else: 160 | duplicates.add(sorted([module, self.module_netloc_constants[constant]])) 161 | if duplicates: raise Exception('Multiple modules installed that connect to the same service names: ' + ', '.join(' and '.join(duplicates))) 162 | 163 | self.update_module_storage() 164 | 165 | for i in self.extension_list: 166 | extension_settings: ExtensionInformation = getattr(importlib.import_module(f'extensions.{i}.interface'), 'extension_settings', None) 167 | settings = self.settings['extensions'][extension_settings.extension_type][extension] \ 168 | if extension_settings.extension_type in self.settings['extensions'] \ 169 | and extension in self.settings['extensions'][extension_settings.extension_type] else extension_settings.settings 170 | extension_type = extension_settings.extension_type 171 | self.extensions[extension_type] = self.extensions[extension_type] if extension_type in self.extensions else {} 172 | self.extensions[extension_type][extension] = class_(settings) 173 | 174 | [self.load_module(module) for module in self.module_list if ModuleFlags.startup_load in self.module_settings[module].flags] 175 | 176 | self.module_controls = {'module_list': self.module_list, 'module_settings': self.module_settings, 177 | 'loaded_modules': self.loaded_modules, 'module_loader': self.load_module} 178 | 179 | def load_module(self, module: str): 180 | module = module.lower() 181 | if module not in self.module_list: 182 | raise Exception(f'"{module}" does not exist in modules.') # TODO: replace with InvalidModuleError 183 | if module not in self.loaded_modules: 184 | class_ = getattr(importlib.import_module(f'modules.{module}.interface'), 'ModuleInterface', None) 185 | if class_: 186 | class ModuleError(Exception): # TODO: get rid of this, as it is deprecated 187 | def __init__(self, message): 188 | super().__init__(module + ' --> ' + str(message)) 189 | 190 | module_controller = ModuleController( 191 | module_settings = self.settings['modules'][module] if module in self.settings['modules'] else {}, 192 | data_folder = os.path.join(self.data_folder_base, 'modules', module), 193 | extensions = self.extensions, 194 | temporary_settings_controller = TemporarySettingsController(module, self.session_storage_location), 195 | module_error = ModuleError, # DEPRECATED 196 | get_current_timestamp = true_current_utc_timestamp, 197 | printer_controller = oprinter, 198 | orpheus_options = OrpheusOptions( 199 | debug_mode = self.settings['global']['advanced']['debug_mode'], 200 | quality_tier = QualityEnum[self.settings['global']['general']['download_quality'].upper()], 201 | disable_subscription_check = self.settings['global']['advanced']['disable_subscription_checks'], 202 | default_cover_options = CoverOptions( 203 | file_type = ImageFileTypeEnum[self.settings['global']['covers']['external_format']], 204 | resolution = self.settings['global']['covers']['main_resolution'], 205 | compression = CoverCompressionEnum[self.settings['global']['covers']['main_compression']] 206 | ) 207 | ) 208 | ) 209 | 210 | loaded_module = class_(module_controller) 211 | self.loaded_modules[module] = loaded_module 212 | 213 | # Check if module has settings 214 | settings = self.settings['modules'][module] if module in self.settings['modules'] else {} 215 | temporary_session = read_temporary_setting(self.session_storage_location, module) 216 | if self.module_settings[module].login_behaviour is ManualEnum.orpheus: 217 | # Login if simple mode, username login and requested by update_setting_storage 218 | if temporary_session and temporary_session['clear_session'] and not self.settings['global']['advanced']['advanced_login_system']: 219 | hashes = {k: hash_string(str(v)) for k, v in settings.items()} 220 | if not temporary_session.get('hashes') or \ 221 | any(k not in hashes or hashes[k] != v for k,v in temporary_session['hashes'].items() if k in self.module_settings[module].session_settings): 222 | print('Logging into ' + self.module_settings[module].service_name) 223 | try: 224 | loaded_module.login(settings['email'] if 'email' in settings else settings['username'], settings['password']) 225 | except: 226 | set_temporary_setting(self.session_storage_location, module, 'hashes', None, {}) 227 | raise 228 | set_temporary_setting(self.session_storage_location, module, 'hashes', None, hashes) 229 | if ModuleFlags.enable_jwt_system in self.module_settings[module].flags and temporary_session and \ 230 | temporary_session['refresh'] and not temporary_session['bearer']: 231 | loaded_module.refresh_login() 232 | 233 | data_folder = os.path.join(self.data_folder_base, 'modules', module) 234 | if ModuleFlags.uses_data in self.module_settings[module].flags and not os.path.exists(data_folder): os.makedirs(data_folder) 235 | 236 | logging.debug(f'Orpheus: {module} module has been loaded') 237 | return loaded_module 238 | else: 239 | raise Exception(f'Error loading module: "{module}"') # TODO: replace with InvalidModuleError 240 | else: 241 | return self.loaded_modules[module] 242 | 243 | def update_module_storage(self): # Should be refactored eventually 244 | ## Settings 245 | old_settings, new_settings, global_settings, extension_settings, module_settings, new_setting_detected = {}, {}, {}, {}, {}, False 246 | 247 | for i in ['global', 'extensions', 'modules']: 248 | old_settings[i] = self.settings[i] if i in self.settings else {} 249 | 250 | for setting_type in self.default_global_settings: 251 | if setting_type in old_settings['global']: 252 | global_settings[setting_type] = {} 253 | for setting in self.default_global_settings[setting_type]: 254 | # Also check if the type is identical 255 | if (setting in old_settings['global'][setting_type] and 256 | isinstance(self.default_global_settings[setting_type][setting], 257 | type(old_settings['global'][setting_type][setting]))): 258 | global_settings[setting_type][setting] = old_settings['global'][setting_type][setting] 259 | else: 260 | global_settings[setting_type][setting] = self.default_global_settings[setting_type][setting] 261 | new_setting_detected = True 262 | else: 263 | global_settings[setting_type] = self.default_global_settings[setting_type] 264 | new_setting_detected = True 265 | 266 | for i in self.extension_list: 267 | extension_information: ExtensionInformation = getattr(importlib.import_module(f'extensions.{i}.interface'), 'extension_settings', None) 268 | extension_type = extension_information.extension_type 269 | extension_settings[extension_type] = {} if 'extension_type' not in extension_settings else extension_settings[extension_type] 270 | old_settings['extensions'][extension_type] = {} if extension_type not in old_settings['extensions'] else old_settings['extensions'][extension_type] 271 | extension_settings[extension_type][i] = {} # This code regenerates the settings 272 | for j in extension_information.settings: 273 | if i in old_settings['extensions'][extension_type] and j in old_settings['extensions'][extension_type][i]: 274 | extension_settings[extension_type][i][j] = old_settings['extensions'][extension_type][i][j] 275 | else: 276 | extension_settings[extension_type][i][j] = extension_information.settings[j] 277 | new_setting_detected = True 278 | 279 | advanced_login_mode = global_settings['advanced']['advanced_login_system'] 280 | for i in self.module_list: 281 | module_settings[i] = {} # This code regenerates the settings 282 | if advanced_login_mode: 283 | settings_to_parse = self.module_settings[i].global_settings 284 | else: 285 | settings_to_parse = {**self.module_settings[i].global_settings, **self.module_settings[i].session_settings} 286 | if settings_to_parse: 287 | for j in settings_to_parse: 288 | if i in old_settings['modules'] and j in old_settings['modules'][i]: 289 | module_settings[i][j] = old_settings['modules'][i][j] 290 | else: 291 | module_settings[i][j] = settings_to_parse[j] 292 | new_setting_detected = True 293 | else: 294 | module_settings.pop(i) 295 | 296 | new_settings['global'] = global_settings 297 | new_settings['extensions'] = extension_settings 298 | new_settings['modules'] = module_settings 299 | 300 | ## Sessions 301 | sessions = pickle.load(open(self.session_storage_location, 'rb')) if os.path.exists(self.session_storage_location) else {} 302 | 303 | if not ('advancedmode' in sessions and 'modules' in sessions and sessions['advancedmode'] == advanced_login_mode): 304 | sessions = {'advancedmode': advanced_login_mode, 'modules':{}} 305 | 306 | # in format {advancedmode, modules: {modulename: {default, type, custom_data, sessions: [sessionname: {##}]}}} 307 | # where ## is 'custom_session' plus if jwt 'access, refresh' (+ emailhash in simple) 308 | # in the special case of simple mode, session is always called default 309 | new_module_sessions = {} 310 | for i in self.module_list: 311 | # Clear storage if type changed 312 | new_module_sessions[i] = sessions['modules'][i] if i in sessions['modules'] else {'selected':'default', 'sessions':{'default':{}}} 313 | 314 | if self.module_settings[i].global_storage_variables: new_module_sessions[i]['custom_data'] = \ 315 | {j:new_module_sessions[i]['custom_data'][j] for j in self.module_settings[i].global_storage_variables \ 316 | if 'custom_data' in new_module_sessions[i] and j in new_module_sessions[i]['custom_data']} 317 | 318 | for current_session in new_module_sessions[i]['sessions'].values(): 319 | # For simple login type only, as it does not apply to advanced login 320 | if self.module_settings[i].login_behaviour is ManualEnum.orpheus and not advanced_login_mode: 321 | hashes = {k:hash_string(str(v)) for k,v in module_settings[i].items()} 322 | if current_session.get('hashes'): 323 | clear_session = any(k not in hashes or hashes[k] != v for k,v in current_session['hashes'].items() if k in self.module_settings[i].session_settings) 324 | else: 325 | clear_session = True 326 | else: 327 | clear_session = False 328 | current_session['clear_session'] = clear_session 329 | 330 | if ModuleFlags.enable_jwt_system in self.module_settings[i].flags: 331 | if 'bearer' in current_session and current_session['bearer'] and not clear_session: 332 | # Clears bearer token if it's expired 333 | try: 334 | time_left_until_refresh = json.loads(base64.b64decode(current_session['bearer'].split('.')[0]))['exp'] - true_current_utc_timestamp() 335 | current_session['bearer'] = current_session['bearer'] if time_left_until_refresh > 0 else '' 336 | except: 337 | pass 338 | else: 339 | current_session['bearer'] = '' 340 | current_session['refresh'] = '' 341 | else: 342 | if 'bearer' in current_session: current_session.pop('bearer') 343 | if 'refresh' in current_session: current_session.pop('refresh') 344 | 345 | if self.module_settings[i].session_storage_variables: current_session['custom_data'] = \ 346 | {j:current_session['custom_data'][j] for j in self.module_settings[i].session_storage_variables \ 347 | if 'custom_data' in current_session and j in current_session['custom_data'] and not clear_session} 348 | elif 'custom_data' in current_session: current_session.pop('custom_data') 349 | 350 | pickle.dump({'advancedmode': advanced_login_mode, 'modules': new_module_sessions}, open(self.session_storage_location, 'wb')) 351 | open(self.settings_location, 'w').write(json.dumps(new_settings, indent = 4, sort_keys = False)) 352 | 353 | if new_setting_detected: 354 | print('New settings detected, or the configuration has been reset. Please update settings.json') 355 | exit() 356 | 357 | 358 | def orpheus_core_download(orpheus_session: Orpheus, media_to_download, third_party_modules, separate_download_module, output_path): 359 | downloader = Downloader(orpheus_session.settings['global'], orpheus_session.module_controls, oprinter, output_path) 360 | os.makedirs('temp', exist_ok=True) 361 | 362 | for mainmodule, items in media_to_download.items(): 363 | for media in items: 364 | if ModuleModes.download not in orpheus_session.module_settings[mainmodule].module_supported_modes: 365 | raise Exception(f'{mainmodule} does not support track downloading') # TODO: replace with ModuleDoesNotSupportAbility 366 | 367 | # Load and prepare module 368 | music = orpheus_session.load_module(mainmodule) 369 | downloader.service = music 370 | downloader.service_name = mainmodule 371 | 372 | for i in third_party_modules: 373 | moduleselected = third_party_modules[i] 374 | if moduleselected: 375 | if moduleselected not in orpheus_session.module_list: 376 | raise Exception(f'{moduleselected} does not exist in modules.') # TODO: replace with InvalidModuleError 377 | elif i not in orpheus_session.module_settings[moduleselected].module_supported_modes: 378 | raise Exception(f'Module {moduleselected} does not support {i}') # TODO: replace with ModuleDoesNotSupportAbility 379 | else: 380 | # If all checks pass, load up the selected module 381 | orpheus_session.load_module(moduleselected) 382 | 383 | downloader.third_party_modules = third_party_modules 384 | 385 | mediatype = media.media_type 386 | media_id = media.media_id 387 | 388 | downloader.download_mode = mediatype 389 | 390 | # Mode to download playlist using other service 391 | if separate_download_module != 'default' and separate_download_module != mainmodule: 392 | if mediatype is not DownloadTypeEnum.playlist: 393 | raise Exception('The separate download module option is only for playlists.') # TODO: replace with ModuleDoesNotSupportAbility 394 | downloader.download_playlist(media_id, custom_module=separate_download_module, extra_kwargs=media.extra_kwargs) 395 | else: # Standard download modes 396 | if mediatype is DownloadTypeEnum.album: 397 | downloader.download_album(media_id, extra_kwargs=media.extra_kwargs) 398 | elif mediatype is DownloadTypeEnum.track: 399 | downloader.download_track(media_id, extra_kwargs=media.extra_kwargs) 400 | elif mediatype is DownloadTypeEnum.playlist: 401 | downloader.download_playlist(media_id, extra_kwargs=media.extra_kwargs) 402 | elif mediatype is DownloadTypeEnum.artist: 403 | downloader.download_artist(media_id, extra_kwargs=media.extra_kwargs) 404 | else: 405 | raise Exception(f'\tUnknown media type "{mediatype}"') 406 | 407 | if os.path.exists('temp'): shutil.rmtree('temp') -------------------------------------------------------------------------------- /orpheus/housekeeping.py: -------------------------------------------------------------------------------- 1 | # Will be used for CLI commands in orpheus.py -------------------------------------------------------------------------------- /orpheus/music_downloader.py: -------------------------------------------------------------------------------- 1 | import logging, os, ffmpeg, sys 2 | import shutil 3 | import unicodedata 4 | from dataclasses import asdict 5 | from time import strftime, gmtime 6 | 7 | from ffmpeg import Error 8 | 9 | from orpheus.tagging import tag_file 10 | from utils.models import * 11 | from utils.utils import * 12 | from utils.exceptions import * 13 | 14 | 15 | def beauty_format_seconds(seconds: int) -> str: 16 | time_data = gmtime(seconds) 17 | 18 | time_format = "%Mm:%Ss" 19 | # if seconds are higher than 3600s also add the hour format 20 | if time_data.tm_hour > 0: 21 | time_format = "%Hh:" + time_format 22 | # TODO: also add days to time_format if hours > 24? 23 | 24 | # return the formatted time string 25 | return strftime(time_format, time_data) 26 | 27 | 28 | class Downloader: 29 | def __init__(self, settings, module_controls, oprinter, path): 30 | self.path = path if path.endswith('/') else path + '/' 31 | self.third_party_modules = None 32 | self.download_mode = None 33 | self.service = None 34 | self.service_name = None 35 | self.module_list = module_controls['module_list'] 36 | self.module_settings = module_controls['module_settings'] 37 | self.loaded_modules = module_controls['loaded_modules'] 38 | self.load_module = module_controls['module_loader'] 39 | self.global_settings = settings 40 | 41 | self.oprinter = oprinter 42 | self.print = self.oprinter.oprint 43 | self.set_indent_number = self.oprinter.set_indent_number 44 | 45 | def search_by_tags(self, module_name, track_info: TrackInfo): 46 | return self.loaded_modules[module_name].search(DownloadTypeEnum.track, f'{track_info.name} {" ".join(track_info.artists)}', track_info=track_info) 47 | 48 | def _add_track_m3u_playlist(self, m3u_playlist: str, track_info: TrackInfo, track_location: str): 49 | if self.global_settings['playlist']['extended_m3u']: 50 | with open(m3u_playlist, 'a', encoding='utf-8') as f: 51 | # if no duration exists default to -1 52 | duration = track_info.duration if track_info.duration else -1 53 | # write the extended track header 54 | f.write(f'#EXTINF:{duration}, {track_info.artists[0]} - {track_info.name}\n') 55 | 56 | with open(m3u_playlist, 'a', encoding='utf-8') as f: 57 | if self.global_settings['playlist']['paths_m3u'] == "absolute": 58 | # add the absolute paths to the playlist 59 | f.write(f'{os.path.abspath(track_location)}\n') 60 | else: 61 | # add the relative paths to the playlist by subtracting the track_location with the m3u_path 62 | f.write(f'{os.path.relpath(track_location, os.path.dirname(m3u_playlist))}\n') 63 | 64 | # add an extra new line to the extended format 65 | f.write('\n') if self.global_settings['playlist']['extended_m3u'] else None 66 | 67 | def download_playlist(self, playlist_id, custom_module=None, extra_kwargs={}): 68 | self.set_indent_number(1) 69 | 70 | playlist_info: PlaylistInfo = self.service.get_playlist_info(playlist_id, **extra_kwargs) 71 | self.print(f'=== Downloading playlist {playlist_info.name} ({playlist_id}) ===', drop_level=1) 72 | self.print(f'Playlist creator: {playlist_info.creator}' + (f' ({playlist_info.creator_id})' if playlist_info.creator_id else '')) 73 | if playlist_info.release_year: self.print(f'Playlist creation year: {playlist_info.release_year}') 74 | if playlist_info.duration: self.print(f'Duration: {beauty_format_seconds(playlist_info.duration)}') 75 | number_of_tracks = len(playlist_info.tracks) 76 | self.print(f'Number of tracks: {number_of_tracks!s}') 77 | self.print(f'Service: {self.module_settings[self.service_name].service_name}') 78 | 79 | playlist_tags = {k: sanitise_name(v) for k, v in asdict(playlist_info).items()} 80 | playlist_tags['explicit'] = ' [E]' if playlist_info.explicit else '' 81 | playlist_path = self.path + self.global_settings['formatting']['playlist_format'].format(**playlist_tags) 82 | # fix path byte limit 83 | playlist_path = fix_byte_limit(playlist_path) + '/' 84 | os.makedirs(playlist_path, exist_ok=True) 85 | 86 | if playlist_info.cover_url: 87 | self.print('Downloading playlist cover') 88 | download_file(playlist_info.cover_url, f'{playlist_path}cover.{playlist_info.cover_type.name}', artwork_settings=self._get_artwork_settings()) 89 | 90 | if playlist_info.animated_cover_url and self.global_settings['covers']['save_animated_cover']: 91 | self.print('Downloading animated playlist cover') 92 | download_file(playlist_info.animated_cover_url, playlist_path + 'cover.mp4', enable_progress_bar=True) 93 | 94 | if playlist_info.description: 95 | with open(playlist_path + 'description.txt', 'w', encoding='utf-8') as f: f.write(playlist_info.description) 96 | 97 | m3u_playlist_path = None 98 | if self.global_settings['playlist']['save_m3u']: 99 | if self.global_settings['playlist']['paths_m3u'] not in {"absolute", "relative"}: 100 | raise ValueError(f'Invalid value for paths_m3u: "{self.global_settings["playlist"]["paths_m3u"]}",' 101 | f' must be either "absolute" or "relative"') 102 | 103 | m3u_playlist_path = playlist_path + f'{playlist_tags["name"]}.m3u' 104 | 105 | # create empty file 106 | with open(m3u_playlist_path, 'w', encoding='utf-8') as f: 107 | f.write('') 108 | 109 | # if extended format add the header 110 | if self.global_settings['playlist']['extended_m3u']: 111 | with open(m3u_playlist_path, 'a', encoding='utf-8') as f: 112 | f.write('#EXTM3U\n\n') 113 | 114 | tracks_errored = set() 115 | if custom_module: 116 | supported_modes = self.module_settings[custom_module].module_supported_modes 117 | if ModuleModes.download not in supported_modes and ModuleModes.playlist not in supported_modes: 118 | raise Exception(f'Module "{custom_module}" cannot be used to download a playlist') # TODO: replace with ModuleDoesNotSupportAbility 119 | self.print(f'Service used for downloading: {self.module_settings[custom_module].service_name}') 120 | original_service = str(self.service_name) 121 | self.load_module(custom_module) 122 | for index, track_id in enumerate(playlist_info.tracks, start=1): 123 | self.set_indent_number(2) 124 | print() 125 | self.print(f'Track {index}/{number_of_tracks}', drop_level=1) 126 | quality_tier = QualityEnum[self.global_settings['general']['download_quality'].upper()] 127 | codec_options = CodecOptions( 128 | spatial_codecs = self.global_settings['codecs']['spatial_codecs'], 129 | proprietary_codecs = self.global_settings['codecs']['proprietary_codecs'], 130 | ) 131 | track_info: TrackInfo = self.loaded_modules[original_service].get_track_info(track_id, quality_tier, codec_options, **playlist_info.track_extra_kwargs) 132 | 133 | self.service = self.loaded_modules[custom_module] 134 | self.service_name = custom_module 135 | results = self.search_by_tags(custom_module, track_info) 136 | track_id_new = results[0].result_id if len(results) else None 137 | 138 | if track_id_new: 139 | self.download_track(track_id_new, album_location=playlist_path, track_index=index, number_of_tracks=number_of_tracks, indent_level=2, m3u_playlist=m3u_playlist_path, extra_kwargs=results[0].extra_kwargs) 140 | else: 141 | tracks_errored.add(f'{track_info.name} - {track_info.artists[0]}') 142 | if ModuleModes.download in self.module_settings[original_service].module_supported_modes: 143 | self.service = self.loaded_modules[original_service] 144 | self.service_name = original_service 145 | self.print(f'Track {track_info.name} not found, using the original service as a fallback', drop_level=1) 146 | self.download_track(track_id, album_location=playlist_path, track_index=index, number_of_tracks=number_of_tracks, indent_level=2, m3u_playlist=m3u_playlist_path, extra_kwargs=playlist_info.track_extra_kwargs) 147 | else: 148 | self.print(f'Track {track_info.name} not found, skipping') 149 | else: 150 | for index, track_id in enumerate(playlist_info.tracks, start=1): 151 | self.set_indent_number(2) 152 | print() 153 | self.print(f'Track {index}/{number_of_tracks}', drop_level=1) 154 | self.download_track(track_id, album_location=playlist_path, track_index=index, number_of_tracks=number_of_tracks, indent_level=2, m3u_playlist=m3u_playlist_path, extra_kwargs=playlist_info.track_extra_kwargs) 155 | 156 | self.set_indent_number(1) 157 | self.print(f'=== Playlist {playlist_info.name} downloaded ===', drop_level=1) 158 | 159 | if tracks_errored: logging.debug('Failed tracks: ' + ', '.join(tracks_errored)) 160 | 161 | @staticmethod 162 | def _get_artist_initials_from_name(album_info: AlbumInfo) -> str: 163 | # Remove "the" from the inital string 164 | initial = album_info.artist.lower() 165 | if album_info.artist.lower().startswith('the'): 166 | initial = initial.replace('the ', '')[0].upper() 167 | 168 | # Unicode fix 169 | initial = unicodedata.normalize('NFKD', initial[0]).encode('ascii', 'ignore').decode('utf-8') 170 | 171 | # Make the initial upper if it's alpha 172 | initial = initial.upper() if initial.isalpha() else '#' 173 | 174 | return initial 175 | 176 | def _create_album_location(self, path: str, album_id: str, album_info: AlbumInfo) -> str: 177 | # Clean up album tags and add special explicit and additional formats 178 | album_tags = {k: sanitise_name(v) for k, v in asdict(album_info).items()} 179 | album_tags['id'] = str(album_id) 180 | album_tags['quality'] = f' [{album_info.quality}]' if album_info.quality else '' 181 | album_tags['explicit'] = ' [E]' if album_info.explicit else '' 182 | album_tags['artist_initials'] = self._get_artist_initials_from_name(album_info) 183 | 184 | album_path = path + self.global_settings['formatting']['album_format'].format(**album_tags) 185 | # fix path byte limit 186 | album_path = fix_byte_limit(album_path) + '/' 187 | os.makedirs(album_path, exist_ok=True) 188 | 189 | return album_path 190 | 191 | def _download_album_files(self, album_path: str, album_info: AlbumInfo): 192 | if album_info.cover_url: 193 | self.print('Downloading album cover') 194 | download_file(album_info.cover_url, f'{album_path}cover.{album_info.cover_type.name}', artwork_settings=self._get_artwork_settings()) 195 | 196 | if album_info.animated_cover_url and self.global_settings['covers']['save_animated_cover']: 197 | self.print('Downloading animated album cover') 198 | download_file(album_info.animated_cover_url, album_path + 'cover.mp4', enable_progress_bar=True) 199 | 200 | if album_info.description: 201 | with open(album_path + 'description.txt', 'w', encoding='utf-8') as f: 202 | f.write(album_info.description) # Also add support for this with singles maybe? 203 | 204 | def download_album(self, album_id, artist_name='', path=None, indent_level=1, extra_kwargs={}): 205 | self.set_indent_number(indent_level) 206 | 207 | album_info: AlbumInfo = self.service.get_album_info(album_id, **extra_kwargs) 208 | if not album_info: 209 | return 210 | number_of_tracks = len(album_info.tracks) 211 | path = self.path if not path else path 212 | 213 | if number_of_tracks > 1 or self.global_settings['formatting']['force_album_format']: 214 | # Creates the album_location folders 215 | album_path = self._create_album_location(path, album_id, album_info) 216 | 217 | if self.download_mode is DownloadTypeEnum.album: 218 | self.set_indent_number(1) 219 | elif self.download_mode is DownloadTypeEnum.artist: 220 | self.set_indent_number(2) 221 | 222 | self.print(f'=== Downloading album {album_info.name} ({album_id}) ===', drop_level=1) 223 | self.print(f'Artist: {album_info.artist} ({album_info.artist_id})') 224 | if album_info.release_year: self.print(f'Year: {album_info.release_year}') 225 | if album_info.duration: self.print(f'Duration: {beauty_format_seconds(album_info.duration)}') 226 | self.print(f'Number of tracks: {number_of_tracks!s}') 227 | self.print(f'Service: {self.module_settings[self.service_name].service_name}') 228 | 229 | if album_info.booklet_url and not os.path.exists(album_path + 'Booklet.pdf'): 230 | self.print('Downloading booklet') 231 | download_file(album_info.booklet_url, album_path + 'Booklet.pdf') 232 | 233 | cover_temp_location = download_to_temp(album_info.all_track_cover_jpg_url) if album_info.all_track_cover_jpg_url else '' 234 | 235 | # Download booklet, animated album cover and album cover if present 236 | self._download_album_files(album_path, album_info) 237 | 238 | for index, track_id in enumerate(album_info.tracks, start=1): 239 | self.set_indent_number(indent_level + 1) 240 | print() 241 | self.print(f'Track {index}/{number_of_tracks}', drop_level=1) 242 | self.download_track(track_id, album_location=album_path, track_index=index, number_of_tracks=number_of_tracks, main_artist=artist_name, cover_temp_location=cover_temp_location, indent_level=indent_level+1, extra_kwargs=album_info.track_extra_kwargs) 243 | 244 | self.set_indent_number(indent_level) 245 | self.print(f'=== Album {album_info.name} downloaded ===', drop_level=1) 246 | if cover_temp_location: silentremove(cover_temp_location) 247 | elif number_of_tracks == 1: 248 | self.download_track(album_info.tracks[0], album_location=path, number_of_tracks=1, main_artist=artist_name, indent_level=indent_level, extra_kwargs=album_info.track_extra_kwargs) 249 | 250 | return album_info.tracks 251 | 252 | def download_artist(self, artist_id, extra_kwargs={}): 253 | artist_info: ArtistInfo = self.service.get_artist_info(artist_id, self.global_settings['artist_downloading']['return_credited_albums'], **extra_kwargs) 254 | artist_name = artist_info.name 255 | 256 | self.set_indent_number(1) 257 | 258 | number_of_albums = len(artist_info.albums) 259 | number_of_tracks = len(artist_info.tracks) 260 | 261 | self.print(f'=== Downloading artist {artist_name} ({artist_id}) ===', drop_level=1) 262 | if number_of_albums: self.print(f'Number of albums: {number_of_albums!s}') 263 | if number_of_tracks: self.print(f'Number of tracks: {number_of_tracks!s}') 264 | self.print(f'Service: {self.module_settings[self.service_name].service_name}') 265 | artist_path = self.path + sanitise_name(artist_name) + '/' 266 | 267 | self.set_indent_number(2) 268 | tracks_downloaded = [] 269 | for index, album_id in enumerate(artist_info.albums, start=1): 270 | print() 271 | self.print(f'Album {index}/{number_of_albums}', drop_level=1) 272 | tracks_downloaded += self.download_album(album_id, artist_name=artist_name, path=artist_path, indent_level=2, extra_kwargs=artist_info.album_extra_kwargs) 273 | 274 | self.set_indent_number(2) 275 | skip_tracks = self.global_settings['artist_downloading']['separate_tracks_skip_downloaded'] 276 | tracks_to_download = [i for i in artist_info.tracks if (i not in tracks_downloaded and skip_tracks) or not skip_tracks] 277 | number_of_tracks_new = len(tracks_to_download) 278 | for index, track_id in enumerate(tracks_to_download, start=1): 279 | print() 280 | self.print(f'Track {index}/{number_of_tracks_new}', drop_level=1) 281 | self.download_track(track_id, album_location=artist_path, main_artist=artist_name, number_of_tracks=1, indent_level=2, extra_kwargs=artist_info.track_extra_kwargs) 282 | 283 | self.set_indent_number(1) 284 | tracks_skipped = number_of_tracks - number_of_tracks_new 285 | if tracks_skipped > 0: self.print(f'Tracks skipped: {tracks_skipped!s}', drop_level=1) 286 | self.print(f'=== Artist {artist_name} downloaded ===', drop_level=1) 287 | 288 | def download_track(self, track_id, album_location='', main_artist='', track_index=0, number_of_tracks=0, cover_temp_location='', indent_level=1, m3u_playlist=None, extra_kwargs={}): 289 | quality_tier = QualityEnum[self.global_settings['general']['download_quality'].upper()] 290 | codec_options = CodecOptions( 291 | spatial_codecs = self.global_settings['codecs']['spatial_codecs'], 292 | proprietary_codecs = self.global_settings['codecs']['proprietary_codecs'], 293 | ) 294 | track_info: TrackInfo = self.service.get_track_info(track_id, quality_tier, codec_options, **extra_kwargs) 295 | 296 | if main_artist.lower() not in [i.lower() for i in track_info.artists] and self.global_settings['advanced']['ignore_different_artists'] and self.download_mode is DownloadTypeEnum.artist: 297 | self.print('Track is not from the correct artist, skipping', drop_level=1) 298 | return 299 | 300 | if not self.global_settings['formatting']['force_album_format']: 301 | if track_index: 302 | track_info.tags.track_number = track_index 303 | if number_of_tracks: 304 | track_info.tags.total_tracks = number_of_tracks 305 | zfill_number = len(str(track_info.tags.total_tracks)) if self.download_mode is not DownloadTypeEnum.track else 1 306 | zfill_lambda = lambda input : sanitise_name(str(input)).zfill(zfill_number) if input is not None else None 307 | 308 | # Separate copy of tags for formatting purposes 309 | zfill_enabled, zfill_list = self.global_settings['formatting']['enable_zfill'], ['track_number', 'total_tracks', 'disc_number', 'total_discs'] 310 | track_tags = {k: (zfill_lambda(v) if zfill_enabled and k in zfill_list else sanitise_name(v)) for k, v in {**asdict(track_info.tags), **asdict(track_info)}.items()} 311 | track_tags['explicit'] = ' [E]' if track_info.explicit else '' 312 | track_tags['artist'] = sanitise_name(track_info.artists[0]) # if len(track_info.artists) == 1 else 'Various Artists' 313 | codec = track_info.codec 314 | 315 | self.set_indent_number(indent_level) 316 | self.print(f'=== Downloading track {track_info.name} ({track_id}) ===', drop_level=1) 317 | 318 | if self.download_mode is not DownloadTypeEnum.album and track_info.album: self.print(f'Album: {track_info.album} ({track_info.album_id})') 319 | if self.download_mode is not DownloadTypeEnum.artist: self.print(f'Artists: {", ".join(track_info.artists)} ({track_info.artist_id})') 320 | if track_info.release_year: self.print(f'Release year: {track_info.release_year!s}') 321 | if track_info.duration: self.print(f'Duration: {beauty_format_seconds(track_info.duration)}') 322 | if self.download_mode is DownloadTypeEnum.track: self.print(f'Service: {self.module_settings[self.service_name].service_name}') 323 | 324 | to_print = 'Codec: ' + codec_data[codec].pretty_name 325 | if track_info.bitrate: to_print += f', bitrate: {track_info.bitrate!s}kbps' 326 | if track_info.bit_depth: to_print += f', bit depth: {track_info.bit_depth!s}bit' 327 | if track_info.sample_rate: to_print += f', sample rate: {track_info.sample_rate!s}kHz' 328 | self.print(to_print) 329 | 330 | # Check if track_info returns error, display it and return this function to not download the track 331 | if track_info.error: 332 | self.print(track_info.error) 333 | self.print(f'=== Track {track_id} failed ===', drop_level=1) 334 | return 335 | 336 | album_location = album_location.replace('\\', '/') 337 | 338 | # Ignores "single_full_path_format" and just downloads every track as an album 339 | if self.global_settings['formatting']['force_album_format'] and self.download_mode in { 340 | DownloadTypeEnum.track, DownloadTypeEnum.playlist}: 341 | # Fetch every needed album_info tag and create an album_location 342 | album_info: AlbumInfo = self.service.get_album_info(track_info.album_id) 343 | # Save the playlist path to save all the albums in the playlist path 344 | path = self.path if album_location == '' else album_location 345 | album_location = self._create_album_location(path, track_info.album_id, album_info) 346 | album_location = album_location.replace('\\', '/') 347 | 348 | # Download booklet, animated album cover and album cover if present 349 | self._download_album_files(album_location, album_info) 350 | 351 | if self.download_mode is DownloadTypeEnum.track and not self.global_settings['formatting']['force_album_format']: # Python 3.10 can't become popular sooner, ugh 352 | track_location_name = self.path + self.global_settings['formatting']['single_full_path_format'].format(**track_tags) 353 | elif track_info.tags.total_tracks == 1 and not self.global_settings['formatting']['force_album_format']: 354 | track_location_name = album_location + self.global_settings['formatting']['single_full_path_format'].format(**track_tags) 355 | else: 356 | if track_info.tags.total_discs and track_info.tags.total_discs > 1: album_location += f'CD {track_info.tags.disc_number!s}/' 357 | track_location_name = album_location + self.global_settings['formatting']['track_filename_format'].format(**track_tags) 358 | # fix file byte limit 359 | track_location_name = fix_byte_limit(track_location_name) 360 | os.makedirs(track_location_name[:track_location_name.rfind('/')], exist_ok=True) 361 | 362 | try: 363 | conversions = {CodecEnum[k.upper()]: CodecEnum[v.upper()] for k, v in self.global_settings['advanced']['codec_conversions'].items()} 364 | except: 365 | conversions = {} 366 | self.print('Warning: codec_conversions setting is invalid!') 367 | 368 | container = codec_data[codec].container 369 | track_location = f'{track_location_name}.{container.name}' 370 | 371 | check_codec = conversions[track_info.codec] if track_info.codec in conversions else track_info.codec 372 | check_location = f'{track_location_name}.{codec_data[check_codec].container.name}' 373 | 374 | if os.path.isfile(check_location) and not self.global_settings['advanced']['ignore_existing_files']: 375 | self.print('Track file already exists') 376 | 377 | # also make sure to add already existing tracks to the m3u playlist 378 | if m3u_playlist: 379 | self._add_track_m3u_playlist(m3u_playlist, track_info, track_location) 380 | 381 | self.print(f'=== Track {track_id} skipped ===', drop_level=1) 382 | return 383 | 384 | if track_info.description: 385 | with open(track_location_name + '.txt', 'w', encoding='utf-8') as f: f.write(track_info.description) 386 | 387 | # Begin process 388 | print() 389 | self.print("Downloading track file") 390 | try: 391 | download_info: TrackDownloadInfo = self.service.get_track_download(**track_info.download_extra_kwargs) 392 | download_file(download_info.file_url, track_location, headers=download_info.file_url_headers, enable_progress_bar=True, indent_level=self.oprinter.indent_number) \ 393 | if download_info.download_type is DownloadEnum.URL else shutil.move(download_info.temp_file_path, track_location) 394 | 395 | # check if get_track_download returns a different codec, for example ffmpeg failed 396 | if download_info.different_codec: 397 | # overwrite the old known codec with the new 398 | codec = download_info.different_codec 399 | container = codec_data[codec].container 400 | old_track_location = track_location 401 | # create the new track_location and move the old file to the new location 402 | track_location = f'{track_location_name}.{container.name}' 403 | shutil.move(old_track_location, track_location) 404 | except KeyboardInterrupt: 405 | self.print('^C pressed, exiting') 406 | sys.exit(0) 407 | except Exception: 408 | if self.global_settings['advanced']['debug_mode']: raise 409 | self.print('Warning: Track download failed: ' + str(sys.exc_info()[1])) 410 | self.print(f'=== Track {track_id} failed ===', drop_level=1) 411 | return 412 | 413 | delete_cover = False 414 | if not cover_temp_location: 415 | cover_temp_location = create_temp_filename() 416 | delete_cover = True 417 | covers_module_name = self.third_party_modules[ModuleModes.covers] 418 | covers_module_name = covers_module_name if covers_module_name != self.service_name else None 419 | if covers_module_name: print() 420 | self.print('Downloading artwork' + ((' with ' + covers_module_name) if covers_module_name else '')) 421 | 422 | jpg_cover_options = CoverOptions(file_type=ImageFileTypeEnum.jpg, resolution=self.global_settings['covers']['main_resolution'], \ 423 | compression=CoverCompressionEnum[self.global_settings['covers']['main_compression'].lower()]) 424 | ext_cover_options = CoverOptions(file_type=ImageFileTypeEnum[self.global_settings['covers']['external_format']], \ 425 | resolution=self.global_settings['covers']['external_resolution'], \ 426 | compression=CoverCompressionEnum[self.global_settings['covers']['external_compression'].lower()]) 427 | 428 | if covers_module_name: 429 | default_temp = download_to_temp(track_info.cover_url) 430 | test_cover_options = CoverOptions(file_type=ImageFileTypeEnum.jpg, resolution=get_image_resolution(default_temp), compression=CoverCompressionEnum.high) 431 | cover_module = self.loaded_modules[covers_module_name] 432 | rms_threshold = self.global_settings['advanced']['cover_variance_threshold'] 433 | 434 | results: list[SearchResult] = self.search_by_tags(covers_module_name, track_info) 435 | self.print('Covers to test: ' + str(len(results))) 436 | attempted_urls = [] 437 | for i, r in enumerate(results, start=1): 438 | test_cover_info: CoverInfo = cover_module.get_track_cover(r.result_id, test_cover_options, **r.extra_kwargs) 439 | if test_cover_info.url not in attempted_urls: 440 | attempted_urls.append(test_cover_info.url) 441 | test_temp = download_to_temp(test_cover_info.url) 442 | rms = compare_images(default_temp, test_temp) 443 | silentremove(test_temp) 444 | self.print(f'Attempt {i} RMS: {rms!s}') # The smaller the root mean square, the closer the image is to the desired one 445 | if rms < rms_threshold: 446 | self.print('Match found below threshold ' + str(rms_threshold)) 447 | jpg_cover_info: CoverInfo = cover_module.get_track_cover(r.result_id, jpg_cover_options, **r.extra_kwargs) 448 | download_file(jpg_cover_info.url, cover_temp_location, artwork_settings=self._get_artwork_settings(covers_module_name)) 449 | silentremove(default_temp) 450 | if self.global_settings['covers']['save_external']: 451 | ext_cover_info: CoverInfo = cover_module.get_track_cover(r.result_id, ext_cover_options, **r.extra_kwargs) 452 | download_file(ext_cover_info.url, f'{track_location_name}.{ext_cover_info.file_type.name}', artwork_settings=self._get_artwork_settings(covers_module_name, is_external=True)) 453 | break 454 | else: 455 | self.print('Third-party module could not find cover, using fallback') 456 | shutil.move(default_temp, cover_temp_location) 457 | else: 458 | download_file(track_info.cover_url, cover_temp_location, artwork_settings=self._get_artwork_settings()) 459 | if self.global_settings['covers']['save_external'] and ModuleModes.covers in self.module_settings[self.service_name].module_supported_modes: 460 | ext_cover_info: CoverInfo = self.service.get_track_cover(track_id, ext_cover_options, **track_info.cover_extra_kwargs) 461 | download_file(ext_cover_info.url, f'{track_location_name}.{ext_cover_info.file_type.name}', artwork_settings=self._get_artwork_settings(is_external=True)) 462 | 463 | if track_info.animated_cover_url and self.global_settings['covers']['save_animated_cover']: 464 | self.print('Downloading animated cover') 465 | download_file(track_info.animated_cover_url, track_location_name + '_cover.mp4', enable_progress_bar=True) 466 | 467 | # Get lyrics 468 | embedded_lyrics = '' 469 | if self.global_settings['lyrics']['embed_lyrics'] or self.global_settings['lyrics']['save_synced_lyrics']: 470 | lyrics_info = LyricsInfo() 471 | if self.third_party_modules[ModuleModes.lyrics] and self.third_party_modules[ModuleModes.lyrics] != self.service_name: 472 | lyrics_module_name = self.third_party_modules[ModuleModes.lyrics] 473 | self.print('Retrieving lyrics with ' + lyrics_module_name) 474 | lyrics_module = self.loaded_modules[lyrics_module_name] 475 | 476 | if lyrics_module_name != self.service_name: 477 | results: list[SearchResult] = self.search_by_tags(lyrics_module_name, track_info) 478 | lyrics_track_id = results[0].result_id if len(results) else None 479 | extra_kwargs = results[0].extra_kwargs if len(results) else None 480 | else: 481 | lyrics_track_id = track_id 482 | extra_kwargs = {} 483 | 484 | if lyrics_track_id: 485 | lyrics_info: LyricsInfo = lyrics_module.get_track_lyrics(lyrics_track_id, **extra_kwargs) 486 | # if lyrics_info.embedded or lyrics_info.synced: 487 | # self.print('Lyrics retrieved') 488 | # else: 489 | # self.print('Lyrics module could not find any lyrics.') 490 | else: 491 | self.print('Lyrics module could not find any lyrics.') 492 | elif ModuleModes.lyrics in self.module_settings[self.service_name].module_supported_modes: 493 | lyrics_info: LyricsInfo = self.service.get_track_lyrics(track_id, **track_info.lyrics_extra_kwargs) 494 | # if lyrics_info.embedded or lyrics_info.synced: 495 | # self.print('Lyrics retrieved') 496 | # else: 497 | # self.print('No lyrics available') 498 | 499 | if lyrics_info.embedded and self.global_settings['lyrics']['embed_lyrics']: 500 | embedded_lyrics = lyrics_info.embedded 501 | # embed the synced lyrics (f.e. Roon) if they are available 502 | if lyrics_info.synced and self.global_settings['lyrics']['embed_lyrics'] and \ 503 | self.global_settings['lyrics']['embed_synced_lyrics']: 504 | embedded_lyrics = lyrics_info.synced 505 | if lyrics_info.synced and self.global_settings['lyrics']['save_synced_lyrics']: 506 | lrc_location = f'{track_location_name}.lrc' 507 | if not os.path.isfile(lrc_location): 508 | with open(lrc_location, 'w', encoding='utf-8') as f: 509 | f.write(lyrics_info.synced) 510 | 511 | # Get credits 512 | credits_list = [] 513 | if self.third_party_modules[ModuleModes.credits] and self.third_party_modules[ModuleModes.credits] != self.service_name: 514 | credits_module_name = self.third_party_modules[ModuleModes.credits] 515 | self.print('Retrieving credits with ' + credits_module_name) 516 | credits_module = self.loaded_modules[credits_module_name] 517 | 518 | if credits_module_name != self.service_name: 519 | results: list[SearchResult] = self.search_by_tags(credits_module_name, track_info) 520 | credits_track_id = results[0].result_id if len(results) else None 521 | extra_kwargs = results[0].extra_kwargs if len(results) else None 522 | else: 523 | credits_track_id = track_id 524 | extra_kwargs = {} 525 | 526 | if credits_track_id: 527 | credits_list = credits_module.get_track_credits(credits_track_id, **extra_kwargs) 528 | # if credits_list: 529 | # self.print('Credits retrieved') 530 | # else: 531 | # self.print('Credits module could not find any credits.') 532 | # else: 533 | # self.print('Credits module could not find any credits.') 534 | elif ModuleModes.credits in self.module_settings[self.service_name].module_supported_modes: 535 | self.print('Retrieving credits') 536 | credits_list = self.service.get_track_credits(track_id, **track_info.credits_extra_kwargs) 537 | # if credits_list: 538 | # self.print('Credits retrieved') 539 | # else: 540 | # self.print('No credits available') 541 | 542 | # Do conversions 543 | old_track_location, old_container = None, None 544 | if codec in conversions: 545 | old_codec_data = codec_data[codec] 546 | new_codec = conversions[codec] 547 | new_codec_data = codec_data[new_codec] 548 | self.print(f'Converting to {new_codec_data.pretty_name}') 549 | 550 | if old_codec_data.spatial or new_codec_data.spatial: 551 | self.print('Warning: converting spacial formats is not allowed, skipping') 552 | elif not old_codec_data.lossless and new_codec_data.lossless and not self.global_settings['advanced']['enable_undesirable_conversions']: 553 | self.print('Warning: Undesirable lossy-to-lossless conversion detected, skipping') 554 | elif not old_codec_data and not self.global_settings['advanced']['enable_undesirable_conversions']: 555 | self.print('Warning: Undesirable lossy-to-lossy conversion detected, skipping') 556 | else: 557 | if not old_codec_data.lossless and new_codec_data.lossless: 558 | self.print('Warning: Undesirable lossy-to-lossless conversion') 559 | elif not old_codec_data: 560 | self.print('Warning: Undesirable lossy-to-lossy conversion') 561 | 562 | try: 563 | conversion_flags = {CodecEnum[k.upper()]:v for k,v in self.global_settings['advanced']['conversion_flags'].items()} 564 | except: 565 | conversion_flags = {} 566 | self.print('Warning: conversion_flags setting is invalid, using defaults') 567 | 568 | conv_flags = conversion_flags[new_codec] if new_codec in conversion_flags else {} 569 | temp_track_location = f'{create_temp_filename()}.{new_codec_data.container.name}' 570 | new_track_location = f'{track_location_name}.{new_codec_data.container.name}' 571 | 572 | stream: ffmpeg = ffmpeg.input(track_location, hide_banner=None, y=None) 573 | # capture_stderr is required for the error output to be captured 574 | try: 575 | # capture_stderr is required for the error output to be captured 576 | stream.output( 577 | temp_track_location, 578 | acodec=new_codec.name.lower(), 579 | **conv_flags, 580 | loglevel='error' 581 | ).run(capture_stdout=True, capture_stderr=True) 582 | except Error as e: 583 | error_msg = e.stderr.decode('utf-8') 584 | # get the error message from ffmpeg and search foe the non-experimental encoder 585 | encoder = re.search(r"(?<=non experimental encoder ')[^']+", error_msg) 586 | if encoder: 587 | self.print(f'Encoder {new_codec.name.lower()} is experimental, trying {encoder.group(0)}') 588 | # try to use the non-experimental encoder 589 | stream.output( 590 | temp_track_location, 591 | acodec=encoder.group(0), 592 | **conv_flags, 593 | loglevel='error' 594 | ).run() 595 | else: 596 | # raise any other occurring error 597 | raise Exception(f'ffmpeg error converting to {new_codec.name.lower()}:\n{error_msg}') 598 | 599 | # remove file if it requires an overwrite, maybe os.replace would work too? 600 | if track_location == new_track_location: 601 | silentremove(track_location) 602 | # just needed so it won't get deleted 603 | track_location = temp_track_location 604 | 605 | # move temp_file to new_track_location and delete temp file 606 | shutil.move(temp_track_location, new_track_location) 607 | silentremove(temp_track_location) 608 | 609 | if self.global_settings['advanced']['conversion_keep_original']: 610 | old_track_location = track_location 611 | old_container = container 612 | else: 613 | silentremove(track_location) 614 | 615 | container = new_codec_data.container 616 | track_location = new_track_location 617 | 618 | # Add the playlist track to the m3u playlist 619 | if m3u_playlist: 620 | self._add_track_m3u_playlist(m3u_playlist, track_info, track_location) 621 | 622 | # Finally tag file 623 | self.print('Tagging file') 624 | try: 625 | tag_file(track_location, cover_temp_location if self.global_settings['covers']['embed_cover'] else None, 626 | track_info, credits_list, embedded_lyrics, container) 627 | if old_track_location: 628 | tag_file(old_track_location, cover_temp_location if self.global_settings['covers']['embed_cover'] else None, 629 | track_info, credits_list, embedded_lyrics, old_container) 630 | except TagSavingFailure: 631 | self.print('Tagging failed, tags saved to text file') 632 | if delete_cover: 633 | silentremove(cover_temp_location) 634 | 635 | self.print(f'=== Track {track_id} downloaded ===', drop_level=1) 636 | 637 | def _get_artwork_settings(self, module_name = None, is_external = False): 638 | if not module_name: 639 | module_name = self.service_name 640 | return { 641 | 'should_resize': ModuleFlags.needs_cover_resize in self.module_settings[module_name].flags, 642 | 'resolution': self.global_settings['covers']['external_resolution'] if is_external else self.global_settings['covers']['main_resolution'], 643 | 'compression': self.global_settings['covers']['external_compression'] if is_external else self.global_settings['covers']['main_compression'], 644 | 'format': self.global_settings['covers']['external_format'] if is_external else 'jpg' 645 | } 646 | -------------------------------------------------------------------------------- /orpheus/tagging.py: -------------------------------------------------------------------------------- 1 | import base64 2 | import logging 3 | from dataclasses import asdict 4 | 5 | from PIL import Image 6 | from mutagen.easyid3 import EasyID3 7 | from mutagen.easymp4 import EasyMP4 8 | from mutagen.flac import FLAC, Picture 9 | from mutagen.id3 import PictureType, APIC, USLT, TDAT, COMM, TPUB 10 | from mutagen.mp3 import EasyMP3 11 | from mutagen.mp4 import MP4Cover 12 | from mutagen.mp4 import MP4Tags 13 | from mutagen.oggopus import OggOpus 14 | from mutagen.oggvorbis import OggVorbis 15 | 16 | from utils.exceptions import * 17 | from utils.models import ContainerEnum, TrackInfo 18 | 19 | # Needed for Windows tagging support 20 | MP4Tags._padding = 0 21 | 22 | 23 | def tag_file(file_path: str, image_path: str, track_info: TrackInfo, credits_list: list, embedded_lyrics: str, container: ContainerEnum): 24 | if container == ContainerEnum.flac: 25 | tagger = FLAC(file_path) 26 | elif container == ContainerEnum.opus: 27 | tagger = OggOpus(file_path) 28 | elif container == ContainerEnum.ogg: 29 | tagger = OggVorbis(file_path) 30 | elif container == ContainerEnum.mp3: 31 | tagger = EasyMP3(file_path) 32 | 33 | if tagger.tags is None: 34 | tagger.tags = EasyID3() # Add EasyID3 tags if none are present 35 | 36 | # Register encoded, rating, barcode, compatible_brands, major_brand and minor_version 37 | tagger.tags.RegisterTextKey('encoded', 'TSSE') 38 | tagger.tags.RegisterTXXXKey('compatible_brands', 'compatible_brands') 39 | tagger.tags.RegisterTXXXKey('major_brand', 'major_brand') 40 | tagger.tags.RegisterTXXXKey('minor_version', 'minor_version') 41 | tagger.tags.RegisterTXXXKey('Rating', 'Rating') 42 | tagger.tags.RegisterTXXXKey('upc', 'BARCODE') 43 | 44 | tagger.tags.pop('encoded', None) 45 | elif container == ContainerEnum.m4a: 46 | tagger = EasyMP4(file_path) 47 | 48 | # Register ISRC, lyrics, cover and explicit tags 49 | tagger.RegisterTextKey('isrc', '----:com.apple.itunes:ISRC') 50 | tagger.RegisterTextKey('upc', '----:com.apple.itunes:UPC') 51 | tagger.RegisterTextKey('explicit', 'rtng') if track_info.explicit is not None else None 52 | tagger.RegisterTextKey('covr', 'covr') 53 | tagger.RegisterTextKey('lyrics', '\xa9lyr') if embedded_lyrics else None 54 | else: 55 | raise Exception('Unknown container for tagging') 56 | 57 | # Remove all useless MPEG-DASH ffmpeg tags 58 | if tagger.tags is not None: 59 | if 'major_brand' in tagger.tags: 60 | del tagger.tags['major_brand'] 61 | if 'minor_version' in tagger.tags: 62 | del tagger.tags['minor_version'] 63 | if 'compatible_brands' in tagger.tags: 64 | del tagger.tags['compatible_brands'] 65 | if 'encoder' in tagger.tags: 66 | del tagger.tags['encoder'] 67 | 68 | tagger['title'] = track_info.name 69 | if track_info.album: tagger['album'] = track_info.album 70 | if track_info.tags.album_artist: tagger['albumartist'] = track_info.tags.album_artist 71 | 72 | tagger['artist'] = track_info.artists 73 | 74 | if container == ContainerEnum.m4a or container == ContainerEnum.mp3: 75 | if track_info.tags.track_number and track_info.tags.total_tracks: 76 | tagger['tracknumber'] = str(track_info.tags.track_number) + '/' + str(track_info.tags.total_tracks) 77 | elif track_info.tags.track_number: 78 | tagger['tracknumber'] = str(track_info.tags.track_number) 79 | if track_info.tags.disc_number and track_info.tags.total_discs: 80 | tagger['discnumber'] = str(track_info.tags.disc_number) + '/' + str(track_info.tags.total_discs) 81 | elif track_info.tags.disc_number: 82 | tagger['discnumber'] = str(track_info.tags.disc_number) 83 | else: 84 | if track_info.tags.track_number: tagger['tracknumber'] = str(track_info.tags.track_number) 85 | if track_info.tags.disc_number: tagger['discnumber'] = str(track_info.tags.disc_number) 86 | if track_info.tags.total_tracks: tagger['totaltracks'] = str(track_info.tags.total_tracks) 87 | if track_info.tags.total_discs: tagger['totaldiscs'] = str(track_info.tags.total_discs) 88 | 89 | if track_info.tags.release_date: 90 | if container == ContainerEnum.mp3: 91 | # Never access protected attributes, too bad! Only works on ID3v2.4, disabled for now! 92 | # tagger.tags._EasyID3__id3._DictProxy__dict['TDRL'] = TDRL(encoding=3, text=track_info.tags.release_date) 93 | # Use YYYY-MM-DD for consistency and convert it to DDMM 94 | release_dd_mm = f'{track_info.tags.release_date[8:10]}{track_info.tags.release_date[5:7]}' 95 | tagger.tags._EasyID3__id3._DictProxy__dict['TDAT'] = TDAT(encoding=3, text=release_dd_mm) 96 | # Now add the year tag 97 | tagger['date'] = str(track_info.release_year) 98 | else: 99 | tagger['date'] = track_info.tags.release_date 100 | else: 101 | tagger['date'] = str(track_info.release_year) 102 | 103 | if track_info.tags.copyright:tagger['copyright'] = track_info.tags.copyright 104 | 105 | if track_info.explicit is not None: 106 | if container == ContainerEnum.m4a: 107 | tagger['explicit'] = b'\x01' if track_info.explicit else b'\x02' 108 | elif container == ContainerEnum.mp3: 109 | tagger['Rating'] = 'Explicit' if track_info.explicit else 'Clean' 110 | else: 111 | tagger['Rating'] = 'Explicit' if track_info.explicit else 'Clean' 112 | 113 | if track_info.tags.genres: tagger['genre'] = track_info.tags.genres 114 | if track_info.tags.isrc: tagger['isrc'] = track_info.tags.isrc.encode() if container == ContainerEnum.m4a else track_info.tags.isrc 115 | if track_info.tags.upc: tagger['UPC'] = track_info.tags.upc.encode() if container == ContainerEnum.m4a else track_info.tags.upc 116 | 117 | # add the label tag 118 | if track_info.tags.label: 119 | if container in {ContainerEnum.flac, ContainerEnum.ogg}: 120 | tagger['Label'] = track_info.tags.label 121 | elif container == ContainerEnum.mp3: 122 | tagger.tags._EasyID3__id3._DictProxy__dict['TPUB'] = TPUB( 123 | encoding=3, 124 | text=track_info.tags.label 125 | ) 126 | elif container == ContainerEnum.m4a: 127 | # only works with MP3TAG? https://docs.mp3tag.de/mapping/ 128 | tagger.RegisterTextKey('label', '\xa9pub') 129 | tagger['label'] = track_info.tags.label 130 | 131 | # add the description tag 132 | if track_info.tags.description and container == ContainerEnum.m4a: 133 | tagger.RegisterTextKey('desc', 'description') 134 | tagger['description'] = track_info.tags.description 135 | 136 | # add comment tag 137 | if track_info.tags.comment: 138 | if container == ContainerEnum.m4a: 139 | tagger.RegisterTextKey('comment', '\xa9cmt') 140 | tagger['comment'] = track_info.tags.comment 141 | elif container == ContainerEnum.mp3: 142 | tagger.tags._EasyID3__id3._DictProxy__dict['COMM'] = COMM( 143 | encoding=3, 144 | lang=u'eng', 145 | desc=u'', 146 | text=track_info.tags.description 147 | ) 148 | 149 | # add all extra_kwargs key value pairs to the (FLAC, Vorbis) file 150 | if container in {ContainerEnum.flac, ContainerEnum.ogg}: 151 | for key, value in track_info.tags.extra_tags.items(): 152 | tagger[key] = value 153 | elif container is ContainerEnum.m4a: 154 | for key, value in track_info.tags.extra_tags.items(): 155 | # Create a new freeform atom and set the extra_tags in bytes 156 | tagger.RegisterTextKey(key, '----:com.apple.itunes:' + key) 157 | tagger[key] = str(value).encode() 158 | 159 | # Need to change to merge duplicate credits automatically, or switch to plain dicts instead of list[dataclass] 160 | if credits_list: 161 | if container == ContainerEnum.m4a: 162 | for credit in credits_list: 163 | # Create a new freeform atom and set the contributors in bytes 164 | tagger.RegisterTextKey(credit.type, '----:com.apple.itunes:' + credit.type) 165 | tagger[credit.type] = [con.encode() for con in credit.names] 166 | elif container == ContainerEnum.mp3: 167 | for credit in credits_list: 168 | # Create a new user-defined text frame key 169 | tagger.tags.RegisterTXXXKey(credit.type.upper(), credit.type) 170 | tagger[credit.type] = credit.names 171 | else: 172 | for credit in credits_list: 173 | try: 174 | tagger.tags[credit.type] = credit.names 175 | except: 176 | pass 177 | 178 | if embedded_lyrics: 179 | if container == ContainerEnum.mp3: 180 | # Never access protected attributes, too bad! I hope I never have to write ID3 code again 181 | tagger.tags._EasyID3__id3._DictProxy__dict['USLT'] = USLT( 182 | encoding=3, 183 | lang=u'eng', # don't assume? 184 | text=embedded_lyrics 185 | ) 186 | else: 187 | tagger['lyrics'] = embedded_lyrics 188 | 189 | if track_info.tags.replay_gain and track_info.tags.replay_peak and container != ContainerEnum.m4a: 190 | tagger['REPLAYGAIN_TRACK_GAIN'] = str(track_info.tags.replay_gain) 191 | tagger['REPLAYGAIN_TRACK_PEAK'] = str(track_info.tags.replay_peak) 192 | 193 | # only embed the cover when embed_cover is set to True 194 | if image_path: 195 | with open(image_path, 'rb') as c: 196 | data = c.read() 197 | picture = Picture() 198 | picture.data = data 199 | 200 | # Check if cover is smaller than 16MB 201 | if len(picture.data) < picture._MAX_SIZE: 202 | if container == ContainerEnum.flac: 203 | picture.type = PictureType.COVER_FRONT 204 | picture.mime = u'image/jpeg' 205 | tagger.add_picture(picture) 206 | elif container == ContainerEnum.m4a: 207 | tagger['covr'] = [MP4Cover(data, imageformat=MP4Cover.FORMAT_JPEG)] 208 | elif container == ContainerEnum.mp3: 209 | # Never access protected attributes, too bad! 210 | tagger.tags._EasyID3__id3._DictProxy__dict['APIC'] = APIC( 211 | encoding=3, # UTF-8 212 | mime='image/jpeg', 213 | type=3, # album art 214 | desc='Cover', # name 215 | data=data 216 | ) 217 | # If you want to have a cover in only a few applications, then this technically works for Opus 218 | elif container in {ContainerEnum.ogg, ContainerEnum.opus}: 219 | im = Image.open(image_path) 220 | width, height = im.size 221 | picture.type = 17 222 | picture.desc = u'Cover Art' 223 | picture.mime = u'image/jpeg' 224 | picture.width = width 225 | picture.height = height 226 | picture.depth = 24 227 | encoded_data = base64.b64encode(picture.write()) 228 | tagger['metadata_block_picture'] = [encoded_data.decode('ascii')] 229 | else: 230 | print(f'\tCover file size is too large, only {(picture._MAX_SIZE / 1024 ** 2):.2f}MB are allowed. Track ' 231 | f'will not have cover saved.') 232 | 233 | try: 234 | tagger.save(file_path, v1=2, v2_version=3, v23_sep=None) if container == ContainerEnum.mp3 else tagger.save() 235 | except: 236 | logging.debug('Tagging failed.') 237 | tag_text = '\n'.join((f'{k}: {v}' for k, v in asdict(track_info.tags).items() if v and k != 'credits' and k != 'lyrics')) 238 | tag_text += '\n\ncredits:\n ' + '\n '.join(f'{credit.type}: {", ".join(credit.names)}' for credit in credits_list if credit.names) if credits_list else '' 239 | tag_text += '\n\nlyrics:\n ' + '\n '.join(embedded_lyrics.split('\n')) if embedded_lyrics else '' 240 | open(file_path.rsplit('.', 1)[0] + '_tags.txt', 'w', encoding='utf-8').write(tag_text) 241 | raise TagSavingFailure 242 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | defusedxml>=0.7.1 2 | protobuf==3.15.8 3 | pycryptodomex>=3.10.1 4 | requests>=2.25.1 5 | Pillow>=8.2.0 6 | tqdm>=4.60.0 7 | mutagen>=1.45.1 8 | ffmpeg-python>=0.2.0 9 | m3u8>=2.0.0 -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OrfiTeam/OrpheusDL/a45ff47913508d4c09971bdb847d5845984f1e64/utils/__init__.py -------------------------------------------------------------------------------- /utils/exceptions.py: -------------------------------------------------------------------------------- 1 | import inspect 2 | from re import L 3 | 4 | get_module_name = lambda : next((s.filename for s in inspect.stack() if 'interface.py' in s.filename), None).split('/')[-2] 5 | 6 | class ModuleAuthError(Exception): 7 | def __init__(self): 8 | super().__init__(f'Invalid login details for the {get_module_name()} module') 9 | 10 | class ModuleAPIError(Exception): 11 | def __init__(self, error_code: int, error_message: str, api_endpoint: str) -> None: 12 | __class__.__name__ = get_module_name() + 'ModuleAPIError' 13 | return super().__init__(f'Error {error_code!s}: {error_message} using endpoint {api_endpoint}') 14 | 15 | class ModuleGeneralError(Exception): 16 | def __init__(self, *args, **qwargs) -> None: 17 | __class__.__name__ = get_module_name() + 'ModuleGeneralError' 18 | return super().__init__(*args, **qwargs) 19 | 20 | class InvalidInput(Exception): 21 | pass # TODO: fill out with custom message, will not be in error format 22 | 23 | class InvalidModuleError(Exception): 24 | pass # TODO: fill out with custom message, will be in error format 25 | 26 | class ModuleDoesNotSupportAbility(Exception): 27 | pass # TODO: fill out with custom message, will be in error format 28 | 29 | class ModuleSettingsNotSet(Exception): 30 | pass # TODO: will either tell you to add settings for a specific module in simple sessions mode, or the command needed to set a setting in advanced sessions mode 31 | 32 | class TagSavingFailure(Exception): 33 | pass -------------------------------------------------------------------------------- /utils/models.py: -------------------------------------------------------------------------------- 1 | import os 2 | from dataclasses import dataclass, field 3 | from enum import Flag, auto 4 | from types import ClassMethodDescriptorType, FunctionType 5 | from typing import Optional 6 | 7 | from utils.utils import read_temporary_setting, set_temporary_setting 8 | 9 | 10 | class Oprinter: # Could change to inherit from print class instead, but this is fine 11 | def __init__(self): 12 | self.indent_number = 1 13 | self.printing_enabled = True 14 | self.multiplier = 8 15 | 16 | def set_indent_number(self, number: int): 17 | try: 18 | size = os.get_terminal_size().columns 19 | if 60 < size < 80: 20 | self.multiplier = int((size - 60)/2.5) 21 | elif size < 60: 22 | self.multiplier = 0 23 | else: 24 | self.multiplier = 8 25 | except: 26 | self.multiplier = 8 27 | 28 | self.indent_number = number * self.multiplier 29 | 30 | def oprint(self, inp: str, drop_level: int = 0): 31 | if self.printing_enabled: 32 | print(' ' * (self.indent_number - drop_level * self.multiplier) + inp) 33 | 34 | 35 | class CodecEnum(Flag): 36 | FLAC = auto() # Lossless, free 37 | ALAC = auto() # Lossless, free, useless 38 | WAV = auto() # Lossless (uncompressed), free, useless 39 | MQA = auto() # Lossy, proprietary, terrible 40 | OPUS = auto() # Lossy, free 41 | VORBIS = auto() # Lossy, free 42 | MP3 = auto() # Lossy, not fully free 43 | AAC = auto() # Lossy, requires license 44 | HEAAC = auto() # Lossy, requires license 45 | MHA1 = auto() # Lossy, requires license, spatial 46 | MHM1 = auto() # Lossy, requires license, spatial 47 | EAC3 = auto() # Specifically E-AC-3 JOC # Lossy, proprietary, spatial 48 | AC4 = auto() # Specifically AC-4 IMS # Lossy, proprietary, spatial 49 | AC3 = auto() # Lossy, proprietary, spatial kinda 50 | NONE = auto() # No codec 51 | 52 | 53 | class ContainerEnum(Flag): 54 | flac = auto() 55 | wav = auto() 56 | opus = auto() 57 | ogg = auto() 58 | m4a = auto() 59 | mp3 = auto() 60 | 61 | 62 | @dataclass 63 | class SearchResult: 64 | result_id: str 65 | name: Optional[str] = None 66 | artists: Optional[list] = None 67 | year: Optional[str] = None 68 | explicit: Optional[bool] = False 69 | duration: Optional[int] = None # Duration in whole seconds 70 | additional: Optional[list] = None 71 | extra_kwargs: Optional[dict] = field(default_factory=dict) 72 | 73 | 74 | @dataclass 75 | class CodecData: 76 | pretty_name: str 77 | container: ContainerEnum 78 | lossless: bool 79 | spatial: bool 80 | proprietary: bool 81 | 82 | 83 | codec_data = { 84 | CodecEnum.FLAC: CodecData(pretty_name='FLAC', container=ContainerEnum.flac, lossless=True, spatial=False, proprietary=False), 85 | CodecEnum.ALAC: CodecData(pretty_name='ALAC', container=ContainerEnum.m4a, lossless=True, spatial=False, proprietary=False), 86 | CodecEnum.WAV: CodecData(pretty_name='WAVE', container=ContainerEnum.wav, lossless=True, spatial=False, proprietary=False), 87 | CodecEnum.MQA: CodecData(pretty_name='MQA', container=ContainerEnum.flac, lossless=False, spatial=False, proprietary=True), 88 | CodecEnum.OPUS: CodecData(pretty_name='Opus', container=ContainerEnum.opus, lossless=False, spatial=False, proprietary=False), 89 | CodecEnum.VORBIS: CodecData(pretty_name='Vorbis', container=ContainerEnum.ogg, lossless=False, spatial=False, proprietary=False), 90 | CodecEnum.MP3: CodecData(pretty_name='MP3', container=ContainerEnum.mp3, lossless=False, spatial=False, proprietary=False), 91 | CodecEnum.AAC: CodecData(pretty_name='AAC-LC', container=ContainerEnum.m4a, lossless=False, spatial=False, proprietary=False), 92 | CodecEnum.HEAAC: CodecData(pretty_name='HE-AAC', container=ContainerEnum.m4a, lossless=False, spatial=False, proprietary=False), 93 | CodecEnum.MHA1: CodecData(pretty_name='MPEG-H 3D (MHA1)', container=ContainerEnum.m4a, lossless=False, spatial=True, proprietary=False), 94 | CodecEnum.MHM1: CodecData(pretty_name='MPEG-H 3D (MHM1)', container=ContainerEnum.m4a, lossless=False, spatial=True, proprietary=False), 95 | CodecEnum.EAC3: CodecData(pretty_name='E-AC-3 JOC', container=ContainerEnum.m4a, lossless=False, spatial=True, proprietary=True), 96 | CodecEnum.AC4: CodecData(pretty_name='AC-4 IMS', container=ContainerEnum.m4a, lossless=False, spatial=True, proprietary=True), 97 | CodecEnum.AC3: CodecData(pretty_name='Dolby Digital', container=ContainerEnum.m4a, lossless=False, spatial=True, proprietary=True), 98 | CodecEnum.NONE: CodecData(pretty_name='Error', container=ContainerEnum.m4a, lossless=False, spatial=False, proprietary=False) 99 | } # Note: spatial has priority over proprietary when deciding if a codec is enabled 100 | 101 | 102 | class DownloadEnum(Flag): 103 | URL = auto() 104 | TEMP_FILE_PATH = auto() # Specifically designed for use with protected streams 105 | MPD = auto() 106 | 107 | 108 | class TemporarySettingsController: 109 | def __init__(self, module: str, settings_location: str): 110 | self.module = module 111 | self.settings_location = settings_location 112 | 113 | def read(self, setting: str, setting_type='custom'): 114 | if setting_type == 'custom': 115 | return read_temporary_setting(self.settings_location, self.module, 'custom_data', setting) 116 | elif setting_type == 'global': 117 | return read_temporary_setting(self.settings_location, self.module, 'custom_data', setting, global_mode=True) 118 | elif setting_type == 'jwt' and (setting == 'bearer' or setting == 'refresh'): 119 | return read_temporary_setting(self.settings_location, self.module, setting, None) 120 | else: 121 | raise Exception('Invalid temporary setting requested') 122 | 123 | def set(self, setting: str, value: str or object, setting_type='custom'): 124 | if setting_type == 'custom': 125 | set_temporary_setting(self.settings_location, self.module, 'custom_data', setting, value) 126 | elif setting_type == 'global': 127 | set_temporary_setting(self.settings_location, self.module, 'custom_data', setting, value, global_mode=True) 128 | elif setting_type == 'jwt' and (setting == 'bearer' or setting == 'refresh'): 129 | set_temporary_setting(self.settings_location, self.module, setting, None, value) 130 | else: 131 | raise Exception('Invalid temporary setting requested') 132 | 133 | 134 | class ModuleFlags(Flag): 135 | startup_load = auto() 136 | hidden = auto() 137 | enable_jwt_system = auto() 138 | private = auto() 139 | uses_data = auto() 140 | needs_cover_resize = auto() 141 | 142 | 143 | class ModuleModes(Flag): 144 | download = auto() 145 | playlist = auto() 146 | lyrics = auto() 147 | credits = auto() 148 | covers = auto() 149 | 150 | 151 | class ManualEnum(Flag): 152 | orpheus = auto() 153 | manual = auto() 154 | 155 | 156 | @dataclass 157 | class ModuleInformation: 158 | service_name: str 159 | module_supported_modes: ModuleModes 160 | global_settings: Optional[dict] = field(default_factory=dict) 161 | global_storage_variables: Optional[list] = None 162 | session_settings: Optional[dict] = field(default_factory=dict) 163 | session_storage_variables: Optional[list] = None 164 | flags: Optional[ModuleFlags] = field(default_factory=dict) 165 | netlocation_constant: Optional[str] or Optional[list] = field(default_factory=list) # not sure if this works with python 3.7/3.8 166 | # note that by setting netlocation_constant to setting.X, it will use that setting instead 167 | url_constants: Optional[dict] = None 168 | test_url: Optional[str] = None 169 | url_decoding: Optional[ManualEnum] = ManualEnum.orpheus 170 | login_behaviour: Optional[ManualEnum] = ManualEnum.orpheus 171 | 172 | 173 | @dataclass 174 | class ExtensionInformation: 175 | extension_type: str 176 | settings: dict 177 | 178 | 179 | class DownloadTypeEnum(Flag): 180 | track = auto() 181 | playlist = auto() 182 | artist = auto() 183 | album = auto() 184 | 185 | 186 | @dataclass 187 | class MediaIdentification: 188 | media_type: DownloadTypeEnum 189 | media_id: str 190 | extra_kwargs: Optional[dict] = field(default_factory=dict) 191 | 192 | 193 | class QualityEnum(Flag): 194 | MINIMUM = auto() 195 | LOW = auto() 196 | MEDIUM = auto() 197 | HIGH = auto() 198 | LOSSLESS = auto() 199 | HIFI = auto() 200 | 201 | 202 | @dataclass 203 | class CodecOptions: 204 | proprietary_codecs: bool 205 | spatial_codecs: bool 206 | 207 | 208 | class ImageFileTypeEnum(Flag): 209 | jpg = auto() 210 | png = auto() 211 | webp = auto() 212 | 213 | 214 | class CoverCompressionEnum(Flag): 215 | low = auto() 216 | high = auto() 217 | 218 | 219 | @dataclass 220 | class CoverOptions: 221 | file_type: ImageFileTypeEnum 222 | resolution: int 223 | compression: CoverCompressionEnum 224 | 225 | 226 | @dataclass 227 | class OrpheusOptions: 228 | debug_mode: bool 229 | disable_subscription_check: bool 230 | quality_tier: QualityEnum # Here because of subscription checking 231 | default_cover_options: CoverOptions 232 | 233 | 234 | @dataclass 235 | class ModuleController: 236 | module_settings: dict 237 | data_folder: str 238 | extensions: dict 239 | temporary_settings_controller: TemporarySettingsController 240 | orpheus_options: OrpheusOptions 241 | get_current_timestamp: FunctionType 242 | printer_controller: Oprinter 243 | module_error: ClassMethodDescriptorType # Will eventually be deprecated *sigh* 244 | 245 | 246 | @dataclass 247 | class Tags: 248 | album_artist: Optional[str] = None 249 | composer: Optional[str] = None 250 | track_number: Optional[int] = None 251 | total_tracks: Optional[int] = None 252 | copyright: Optional[str] = None 253 | isrc: Optional[str] = None 254 | upc: Optional[str] = None 255 | disc_number: Optional[int] = None 256 | total_discs: Optional[int] = None 257 | replay_gain: Optional[float] = None 258 | replay_peak: Optional[float] = None 259 | genres: Optional[list] = None 260 | release_date: Optional[str] = None # Format: YYYY-MM-DD 261 | description: Optional[str] = None 262 | comment: Optional[str] = None 263 | label: Optional[str] = None 264 | extra_tags: Optional[dict] = field(default_factory=dict) 265 | 266 | 267 | @dataclass 268 | class CoverInfo: 269 | url: str 270 | file_type: ImageFileTypeEnum 271 | 272 | 273 | @dataclass 274 | class LyricsInfo: 275 | embedded: Optional[str] = None 276 | synced: Optional[str] = None 277 | 278 | 279 | # TODO: get rid of CreditsInfo! 280 | @dataclass 281 | class CreditsInfo: 282 | type: str 283 | names: list 284 | 285 | 286 | @dataclass 287 | class AlbumInfo: 288 | name: str 289 | artist: str 290 | tracks: list 291 | release_year: int 292 | duration: Optional[int] = None # Duration in whole seconds 293 | explicit: Optional[bool] = False 294 | artist_id: Optional[str] = None 295 | quality: Optional[str] = None 296 | booklet_url: Optional[str] = None 297 | cover_url: Optional[str] = None 298 | upc: Optional[str] = None 299 | cover_type: Optional[ImageFileTypeEnum] = ImageFileTypeEnum.jpg 300 | all_track_cover_jpg_url: Optional[str] = None 301 | animated_cover_url: Optional[str] = None 302 | description: Optional[str] = None 303 | track_extra_kwargs: Optional[dict] = field(default_factory=dict) 304 | 305 | 306 | @dataclass 307 | class ArtistInfo: 308 | name: str 309 | albums: Optional[list] = field(default_factory=list) 310 | album_extra_kwargs: Optional[dict] = field(default_factory=dict) 311 | tracks: Optional[list] = field(default_factory=list) 312 | track_extra_kwargs: Optional[dict] = field(default_factory=dict) 313 | 314 | 315 | @dataclass 316 | class PlaylistInfo: 317 | name: str 318 | creator: str 319 | tracks: list 320 | release_year: int 321 | duration: Optional[int] = None # Duration in whole seconds 322 | explicit: Optional[bool] = False 323 | creator_id: Optional[str] = None 324 | cover_url: Optional[str] = None 325 | cover_type: Optional[ImageFileTypeEnum] = ImageFileTypeEnum.jpg 326 | animated_cover_url: Optional[str] = None 327 | description: Optional[str] = None 328 | track_extra_kwargs: Optional[dict] = field(default_factory=dict) 329 | 330 | 331 | @dataclass 332 | class TrackInfo: 333 | name: str 334 | album: str 335 | album_id: str 336 | artists: list 337 | tags: Tags 338 | codec: CodecEnum 339 | cover_url: str 340 | release_year: int 341 | duration: Optional[int] = None # Duration in whole seconds 342 | explicit: Optional[bool] = None 343 | artist_id: Optional[str] = None 344 | animated_cover_url: Optional[str] = None 345 | description: Optional[str] = None 346 | bit_depth: Optional[int] = 16 347 | sample_rate: Optional[float] = 44.1 348 | bitrate: Optional[int] = None 349 | download_extra_kwargs: Optional[dict] = field(default_factory=dict) 350 | cover_extra_kwargs: Optional[dict] = field(default_factory=dict) 351 | credits_extra_kwargs: Optional[dict] = field(default_factory=dict) 352 | lyrics_extra_kwargs: Optional[dict] = field(default_factory=dict) 353 | error: Optional[str] = None 354 | 355 | 356 | @dataclass 357 | class TrackDownloadInfo: 358 | download_type: DownloadEnum 359 | file_url: Optional[str] = None 360 | file_url_headers: Optional[dict] = None 361 | temp_file_path: Optional[str] = None 362 | different_codec: Optional[CodecEnum] = None 363 | -------------------------------------------------------------------------------- /utils/utils.py: -------------------------------------------------------------------------------- 1 | import pickle, requests, errno, hashlib, math, os, re, operator 2 | from tqdm import tqdm 3 | from PIL import Image, ImageChops 4 | from requests.adapters import HTTPAdapter 5 | from urllib3.util.retry import Retry 6 | from functools import reduce 7 | 8 | 9 | def hash_string(input_str: str, hash_type: str = 'MD5'): 10 | if hash_type == 'MD5': 11 | return hashlib.md5(input_str.encode("utf-8")).hexdigest() 12 | else: 13 | raise Exception('Invalid hash type selected') 14 | 15 | def create_requests_session(): 16 | session_ = requests.Session() 17 | retries = Retry(total=10, backoff_factor=0.4, status_forcelist=[429, 500, 502, 503, 504]) 18 | session_.mount('http://', HTTPAdapter(max_retries=retries)) 19 | session_.mount('https://', HTTPAdapter(max_retries=retries)) 20 | return session_ 21 | 22 | sanitise_name = lambda name : re.sub(r'[:]', ' - ', re.sub(r'[\\/*?"<>|$]', '', re.sub(r'[ \t]+$', '', str(name).rstrip()))) if name else '' 23 | 24 | 25 | def fix_byte_limit(path: str, byte_limit=250): 26 | # only needs the relative path, the abspath uses already existing folders 27 | rel_path = os.path.relpath(path).replace('\\', '/') 28 | 29 | # split path into directory and filename 30 | directory, filename = os.path.split(rel_path) 31 | 32 | # truncate filename if its byte size exceeds the byte_limit 33 | filename_bytes = filename.encode('utf-8') 34 | fixed_bytes = filename_bytes[:byte_limit] 35 | fixed_filename = fixed_bytes.decode('utf-8', 'ignore') 36 | 37 | # join the directory and truncated filename together 38 | return directory + '/' + fixed_filename 39 | 40 | 41 | r_session = create_requests_session() 42 | 43 | def download_file(url, file_location, headers={}, enable_progress_bar=False, indent_level=0, artwork_settings=None): 44 | if os.path.isfile(file_location): 45 | return None 46 | 47 | r = r_session.get(url, stream=True, headers=headers, verify=False) 48 | 49 | total = None 50 | if 'content-length' in r.headers: 51 | total = int(r.headers['content-length']) 52 | 53 | try: 54 | with open(file_location, 'wb') as f: 55 | if enable_progress_bar and total: 56 | try: 57 | columns = os.get_terminal_size().columns 58 | if os.name == 'nt': 59 | bar = tqdm(total=total, unit='B', unit_scale=True, unit_divisor=1024, initial=0, miniters=1, ncols=(columns-indent_level), bar_format=' '*indent_level + '{l_bar}{bar}{r_bar}') 60 | else: 61 | raise 62 | except: 63 | bar = tqdm(total=total, unit='B', unit_scale=True, unit_divisor=1024, initial=0, miniters=1, bar_format=' '*indent_level + '{l_bar}{bar}{r_bar}') 64 | # bar.set_description(' '*indent_level) 65 | for chunk in r.iter_content(chunk_size=1024): 66 | if chunk: # filter out keep-alive new chunks 67 | f.write(chunk) 68 | bar.update(len(chunk)) 69 | bar.close() 70 | else: 71 | [f.write(chunk) for chunk in r.iter_content(chunk_size=1024) if chunk] 72 | if artwork_settings and artwork_settings.get('should_resize', False): 73 | new_resolution = artwork_settings.get('resolution', 1400) 74 | new_format = artwork_settings.get('format', 'jpeg') 75 | if new_format == 'jpg': new_format = 'jpeg' 76 | new_compression = artwork_settings.get('compression', 'low') 77 | if new_compression == 'low': 78 | new_compression = 90 79 | elif new_compression == 'high': 80 | new_compression = 70 81 | if new_format == 'png': new_compression = None 82 | with Image.open(file_location) as im: 83 | im = im.resize((new_resolution, new_resolution), Image.Resampling.BICUBIC) 84 | im.save(file_location, new_format, quality=new_compression) 85 | except KeyboardInterrupt: 86 | if os.path.isfile(file_location): 87 | print(f'\tDeleting partially downloaded file "{str(file_location)}"') 88 | silentremove(file_location) 89 | raise KeyboardInterrupt 90 | 91 | # root mean square code by Charlie Clark: https://code.activestate.com/recipes/577630-comparing-two-images/ 92 | def compare_images(image_1, image_2): 93 | with Image.open(image_1) as im1, Image.open(image_2) as im2: 94 | h = ImageChops.difference(im1, im2).convert('L').histogram() 95 | return math.sqrt(reduce(operator.add, map(lambda h, i: h*(i**2), h, range(256))) / (float(im1.size[0]) * im1.size[1])) 96 | 97 | # TODO: check if not closing the files causes issues, and see if there's a way to use the context manager with lambda expressions 98 | get_image_resolution = lambda image_location : Image.open(image_location).size[0] 99 | 100 | def silentremove(filename): 101 | try: 102 | os.remove(filename) 103 | except OSError as e: 104 | if e.errno != errno.ENOENT: 105 | raise 106 | 107 | def read_temporary_setting(settings_location, module, root_setting=None, setting=None, global_mode=False): 108 | temporary_settings = pickle.load(open(settings_location, 'rb')) 109 | module_settings = temporary_settings['modules'][module] if module in temporary_settings['modules'] else None 110 | 111 | if module_settings: 112 | if global_mode: 113 | session = module_settings 114 | else: 115 | session = module_settings['sessions'][module_settings['selected']] 116 | else: 117 | session = None 118 | 119 | if session and root_setting: 120 | if setting: 121 | return session[root_setting][setting] if root_setting in session and setting in session[root_setting] else None 122 | else: 123 | return session[root_setting] if root_setting in session else None 124 | elif root_setting and not session: 125 | raise Exception('Module does not use temporary settings') 126 | else: 127 | return session 128 | 129 | def set_temporary_setting(settings_location, module, root_setting, setting=None, value=None, global_mode=False): 130 | temporary_settings = pickle.load(open(settings_location, 'rb')) 131 | module_settings = temporary_settings['modules'][module] if module in temporary_settings['modules'] else None 132 | 133 | if module_settings: 134 | if global_mode: 135 | session = module_settings 136 | else: 137 | session = module_settings['sessions'][module_settings['selected']] 138 | else: 139 | session = None 140 | 141 | if not session: 142 | raise Exception('Module does not use temporary settings') 143 | if setting: 144 | session[root_setting][setting] = value 145 | else: 146 | session[root_setting] = value 147 | pickle.dump(temporary_settings, open(settings_location, 'wb')) 148 | 149 | create_temp_filename = lambda : f'temp/{os.urandom(16).hex()}' 150 | 151 | def save_to_temp(input: bytes): 152 | location = create_temp_filename() 153 | open(location, 'wb').write(input) 154 | return location 155 | 156 | def download_to_temp(url, headers={}, extension='', enable_progress_bar=False, indent_level=0): 157 | location = create_temp_filename() + (('.' + extension) if extension else '') 158 | download_file(url, location, headers=headers, enable_progress_bar=enable_progress_bar, indent_level=indent_level) 159 | return location 160 | --------------------------------------------------------------------------------