├── .gitignore ├── LICENSE.txt ├── README.rst ├── config-template.yml ├── plexlibrary ├── __init__.py ├── __main__.py ├── config.py ├── imdbutils.py ├── logs.py ├── plexlibrary.py ├── plexutils.py ├── recipe.py ├── recipes.py ├── tmdb.py ├── traktutils.py ├── tvdb.py └── utils.py ├── recipes └── examples │ ├── movies_1001_movies_you_must_see_before_you_die.yml │ ├── movies_imdb_top_250.yml │ ├── movies_recommended.yml │ ├── movies_trending.yml │ └── tv_trending.yml └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | syntax: glob 2 | *.db 3 | *.egg-info 4 | *.log 5 | *.orig 6 | *.pickle 7 | *.pyc 8 | *.sublime-* 9 | *.swp 10 | *__pycache__* 11 | .cache/ 12 | .idea/ 13 | config.yml 14 | recipes/*.yml 15 | venv/ 16 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2017, Adam Göthe 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without modification, 5 | are permitted provided that the following conditions are met: 6 | 7 | * Redistributions of source code must retain the above copyright notice, 8 | this list of conditions and the following disclaimer. 9 | * Redistributions in binary form must reproduce the above copyright notice, 10 | this list of conditions and the following disclaimer in the documentation 11 | and/or other materials provided with the distribution. 12 | * Neither the name python-plexlibrary nor the names of its contributors 13 | may be used to endorse or promote products derived from this software without 14 | specific prior written permission. 15 | 16 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 17 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 18 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 19 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR 20 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 21 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 22 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 23 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 24 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 25 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Python-PlexLibrary 2 | ================== 3 | 4 | Python command line utility for creating and maintaining dynamic Plex 5 | libraries and playlists based on "recipes". 6 | 7 | E.g. Create a library or playlist consisting of all movies or tv shows in a Trakt_ list or 8 | on an IMDb_ chart that exist in your main library, and set the sort titles 9 | accordingly (sort only available for libraries). 10 | 11 | .. _Trakt: https://trakt.tv/ 12 | .. _IMDb: https://imdb.com/ 13 | 14 | 15 | Disclaimer 16 | ---------- 17 | This is still a work in progress, so major changes may occur in new versions. 18 | 19 | Requirements 20 | ------------ 21 | 22 | * Python 3 23 | 24 | * You need a trakt.tv account and an API app: https://trakt.tv/oauth/applications/new 25 | 26 | * (optional) The Movie Database API 27 | 28 | * https://developers.themoviedb.org/3/getting-started 29 | 30 | * Required for fetching scores, release dates etcetera, for weighted sorting 31 | 32 | * Required for matching any library items that use the TMDb agent with the items from the lists (if those items do not include a TMDb ID) 33 | 34 | * Shouldn't be necessary for Trakt, as those usually all have TMDb IDs. 35 | 36 | * Required for matching movies and some TV shows sourced from IMDb 37 | 38 | * (optional) TheTVDB API 39 | 40 | * https://www.thetvdb.com/?tab=apiregister 41 | 42 | * Required for matching any library items that use the TheTVDB agent with the items from the lists (if those items do not include a TheTVDB ID) 43 | 44 | * Shouldn't be necessary for Trakt, as those usually all have TVDB IDs. 45 | 46 | * Required for matching TV shows sourced from IMDb 47 | 48 | Getting started 49 | --------------- 50 | 51 | 1. Clone or download this repo. 52 | 53 | 2. Install Python and pip if you haven't already. 54 | 55 | 3. Install the requirements: 56 | 57 | .. code-block:: shell 58 | 59 | pip install -r requirements.txt 60 | 61 | 4. Copy config-template.yml to config.yml and edit it with your information. 62 | 63 | * Here's a guide if you're unfamiliar with YAML syntax. **Most notably you need to use spaces instead of tabs!** http://docs.ansible.com/ansible/latest/YAMLSyntax.html 64 | 65 | 5. Check out the recipe examples under recipes/examples. Copy an example to recipes/ and edit it with the appropriate information. 66 | 67 | Usage 68 | ----- 69 | In the base directory, run: 70 | 71 | .. code-block:: shell 72 | 73 | python3 plexlibrary -h 74 | 75 | for details on how to use the utility. 76 | 77 | .. code-block:: shell 78 | 79 | python3 plexlibrary -l 80 | 81 | lists available recipes. 82 | 83 | To run a recipe named "movies_trending", run: 84 | 85 | .. code-block:: shell 86 | 87 | python3 plexlibrary movies_trending 88 | 89 | **(If you're on Windows, you might have to run as admin)** 90 | 91 | When you're happy with the results, automate the recipe in cron_ or equivalent (automated tasks in Windows https://technet.microsoft.com/en-us/library/cc748993(v=ws.11).aspx). 92 | 93 | .. _cron: https://code.tutsplus.com/tutorials/scheduling-tasks-with-cron-jobs--net-8800 94 | 95 | **Pro tip!** Edit the new library and uncheck *"Include in dashboard"*. Othewise if you start watching something that exists in multiple libraries, all items will show up on the On Deck. This makes it so that only the item in your main library shows up. 96 | 97 | Planned features 98 | ---------------- 99 | See issues. 100 | 101 | Credit 102 | ------ 103 | Original functionality is based on https://gist.github.com/JonnyWong16/b1aa2c0f604ed92b9b3afaa6db18e5fd 104 | 105 | -------------------------------------------------------------------------------- /config-template.yml: -------------------------------------------------------------------------------- 1 | guid_cache_file: '/tmp/plex_guid_cache.json' 2 | 3 | # Plex server details 4 | # * Defaults to plexapi config 5 | plex: 6 | baseurl: 'http://localhost:32400' 7 | token: '' # https://support.plex.tv/hc/en-us/articles/204059436-Finding-an-authentication-token-X-Plex-Token 8 | 9 | # trakt.tv API details 10 | # * Required for fetching trakt lists 11 | # Create a Trakt.tv account, then create an API app here: 12 | # https://trakt.tv/oauth/applications/new 13 | trakt: 14 | username: '' 15 | client_id: '' 16 | client_secret: '' 17 | oauth_token: '' # Filled in later depending on recipe 18 | 19 | # The Movie Database details 20 | # * Required for fetching scores, release dates etc for weighted sorting 21 | # * Required for matching any library items that use the TMDb agent with the items from the lists 22 | # (if those items do not include a TMDb ID) 23 | tmdb: 24 | api_key: '' 25 | cache_file: '/tmp/tmdb_details.shelve' 26 | 27 | # TheTVDB details 28 | # * Required for matching any library items that use the TheTVDB agent with the items from the lists 29 | # (if those items do not include a TheTVDB ID) 30 | tvdb: 31 | username: '' 32 | api_key: '' 33 | user_key: '' 34 | -------------------------------------------------------------------------------- /plexlibrary/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adamgot/python-plexlibrary/6525ea73e1d5b6713e085f5a417ab831cf1c13bb/plexlibrary/__init__.py -------------------------------------------------------------------------------- /plexlibrary/__main__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from plexlibrary import main 3 | import logging 4 | 5 | logging.basicConfig(format='%(asctime)s %(levelname)s:%(message)s', 6 | level=logging.INFO) 7 | 8 | main() 9 | -------------------------------------------------------------------------------- /plexlibrary/config.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import os 3 | 4 | from utils import YAMLBase 5 | 6 | 7 | class ConfigParser(YAMLBase): 8 | def __init__(self, filepath=None): 9 | if not filepath: 10 | # FIXME? 11 | parent_dir = (os.path.abspath(os.path.join( 12 | os.path.dirname(__file__), os.path.pardir))) 13 | filepath = os.path.join(parent_dir, 'config.yml') 14 | 15 | super(ConfigParser, self).__init__(filepath) 16 | 17 | def validate(self): 18 | if not self.get('plex'): 19 | raise Exception("Missing 'plex' in config") 20 | else: 21 | if 'baseurl' not in self['plex']: 22 | raise Exception("Missing 'baseurl' in 'plex'") 23 | if 'token' not in self['plex']: 24 | raise Exception("Missing 'token' in 'plex'") 25 | 26 | if not self.get('trakt'): 27 | raise Exception("Missing 'trakt' in config") 28 | else: 29 | if 'username' not in self['trakt']: 30 | raise Exception("Missing 'username' in 'trakt'") 31 | if 'client_id' not in self['trakt']: 32 | raise Exception("Missing 'client_id' in 'trakt'") 33 | if 'client_secret' not in self['trakt']: 34 | raise Exception("Missing 'client_secret' in 'trakt'") 35 | 36 | return True 37 | -------------------------------------------------------------------------------- /plexlibrary/imdbutils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import datetime 3 | 4 | import requests 5 | from lxml import html 6 | 7 | import logs 8 | from utils import add_years 9 | 10 | 11 | class IMDb(object): 12 | def __init__(self, tmdb, tvdb): 13 | self.tmdb = tmdb 14 | self.tvdb = tvdb 15 | 16 | def _handle_request(self, url): 17 | """Stolen from Automated IMDB Top 250 Plex library script 18 | by /u/SwiftPanda16 19 | """ 20 | r = requests.get(url) 21 | tree = html.fromstring(r.content) 22 | 23 | # Dict of the IMDB top 250 ids in order 24 | titles = tree.xpath("//table[contains(@class, 'chart')]" 25 | "//td[@class='titleColumn']/a/text()") 26 | years = tree.xpath("//table[contains(@class, 'chart')]" 27 | "//td[@class='titleColumn']/span/text()") 28 | ids = tree.xpath("//table[contains(@class, 'chart')]" 29 | "//td[@class='ratingColumn']/div//@data-titleid") 30 | 31 | return ids, titles, years 32 | 33 | def add_movies(self, url, movie_list=None, movie_ids=None, max_age=0): 34 | if not movie_list: 35 | movie_list = [] 36 | if not movie_ids: 37 | movie_ids = [] 38 | max_date = add_years(max_age * -1) 39 | logs.info(u"Retrieving the IMDB list: {}".format(url)) 40 | 41 | (imdb_ids, imdb_titles, imdb_years) = self._handle_request(url) 42 | for i, imdb_id in enumerate(imdb_ids): 43 | # Skip already added movies 44 | if imdb_id in movie_ids: 45 | continue 46 | 47 | if self.tmdb: 48 | tmdb_data = self.tmdb.get_tmdb_from_imdb(imdb_id, 'movie') 49 | 50 | if tmdb_data and tmdb_data['release_date']: 51 | date = datetime.datetime.strptime(tmdb_data['release_date'], 52 | '%Y-%m-%d') 53 | elif imdb_years[i]: 54 | date = datetime.datetime(int(str(imdb_years[i]).strip("()")), 55 | 12, 31) 56 | else: 57 | date = datetime.date.today() 58 | 59 | # Skip old movies 60 | if max_age != 0 and (max_date > date): 61 | continue 62 | movie_list.append({ 63 | 'id': imdb_id, 64 | 'tmdb_id': tmdb_data['id'] if tmdb_data else None, 65 | 'title': tmdb_data['title'] if tmdb_data else imdb_titles[i], 66 | 'year': date.year, 67 | }) 68 | movie_ids.append(imdb_id) 69 | if tmdb_data and tmdb_data['id']: 70 | movie_ids.append('tmdb' + str(tmdb_data['id'])) 71 | 72 | return movie_list, movie_ids 73 | 74 | def add_shows(self, url, show_list=None, show_ids=None, max_age=0): 75 | if not show_list: 76 | show_list = [] 77 | if not show_ids: 78 | show_ids = [] 79 | curyear = datetime.datetime.now().year 80 | logs.info(u"Retrieving the IMDb list: {}".format(url)) 81 | data = {} 82 | if max_age != 0: 83 | data['extended'] = 'full' 84 | (imdb_ids, imdb_titles, imdb_years) = self._handle_request(url) 85 | for i, imdb_id in enumerate(imdb_ids): 86 | # Skip already added shows 87 | if imdb_id in show_ids: 88 | continue 89 | 90 | if self.tvdb: 91 | tvdb_data = self.tvdb.get_tvdb_from_imdb(imdb_id) 92 | 93 | if self.tmdb: 94 | tmdb_data = self.tmdb.get_tmdb_from_imdb(imdb_id, 'tv') 95 | 96 | if tvdb_data and tvdb_data['firstAired'] != "": 97 | year = datetime.datetime.strptime(tvdb_data['firstAired'], 98 | '%Y-%m-%d').year 99 | elif tmdb_data and tmdb_data['first_air_date'] != "": 100 | year = datetime.datetime.strptime(tmdb_data['first_air_date'], 101 | '%Y-%m-%d').year 102 | elif imdb_years[i]: 103 | year = str(imdb_years[i]).strip("()") 104 | else: 105 | year = datetime.date.today().year 106 | 107 | # Skip old shows 108 | if max_age != 0 \ 109 | and (curyear - (max_age - 1)) > year: 110 | continue 111 | 112 | if tvdb_data: 113 | title = tvdb_data['seriesName'] 114 | else: 115 | title = tmdb_data['name'] if tmdb_data else imdb_titles[i] 116 | 117 | show_list.append({ 118 | 'id': imdb_id, 119 | 'tvdb_id': tvdb_data['id'] if tvdb_data else None, 120 | 'tmdb_id': tmdb_data['id'] if tmdb_data else None, 121 | 'title': title, 122 | 'year': year, 123 | }) 124 | show_ids.append(imdb_id) 125 | if tmdb_data and tmdb_data['id']: 126 | show_ids.append('tmdb' + str(tmdb_data['id'])) 127 | if tvdb_data and tvdb_data['id']: 128 | show_ids.append('tvdb' + str(tvdb_data['id'])) 129 | 130 | return show_list, show_ids 131 | 132 | def add_items(self, item_type, url, item_list=None, item_ids=None, 133 | max_age=0): 134 | if item_type == 'movie': 135 | return self.add_movies(url, movie_list=item_list, 136 | movie_ids=item_ids, max_age=max_age) 137 | elif item_type == 'tv': 138 | return self.add_shows(url, show_list=item_list, 139 | show_ids=item_ids, max_age=max_age) 140 | -------------------------------------------------------------------------------- /plexlibrary/logs.py: -------------------------------------------------------------------------------- 1 | import logging 2 | 3 | 4 | def info(message): 5 | logging.info(msg=message) 6 | 7 | 8 | def error(message): 9 | logging.error(msg=message) 10 | 11 | 12 | def warning(message): 13 | logging.warning(msg=message) 14 | -------------------------------------------------------------------------------- /plexlibrary/plexlibrary.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | """Automated Plex library utility 4 | 5 | This utility creates or maintains a Plex library 6 | based on a configuration recipe. 7 | 8 | Disclaimer: 9 | Use at your own risk! I am not responsible 10 | for damages to your Plex server or libraries. 11 | 12 | Credit: 13 | Originally based on 14 | https://gist.github.com/JonnyWong16/f5b9af386ea58e19bf18c09f2681df23 15 | by /u/SwiftPanda16 16 | """ 17 | 18 | import argparse 19 | import sys 20 | 21 | import recipes 22 | from recipe import Recipe 23 | 24 | 25 | def list_recipes(directory=None): 26 | print("Available recipes:") 27 | for name in recipes.get_recipes(directory): 28 | print(" {}".format(name)) 29 | 30 | 31 | def main(): 32 | parser = argparse.ArgumentParser( 33 | prog='plexlibrary', 34 | description=("This utility creates or maintains a Plex library " 35 | "based on a configuration recipe."), 36 | usage='%(prog)s [options] []', 37 | ) 38 | parser.add_argument('recipe', nargs='?', 39 | help='Create a library using this recipe') 40 | parser.add_argument( 41 | '-l', '--list-recipes', action='store_true', 42 | help='list available recipes') 43 | parser.add_argument( 44 | '-s', '--sort-only', action='store_true', help='only sort the library') 45 | parser.add_argument( 46 | '-p', '--playlists', action='store_true', help='make playlists rather than libraries' 47 | ) 48 | parser.add_argument( 49 | '-e', '--everyone', action='store_true', help='share playlist with all users (overrides settings in recipe)' 50 | ) 51 | 52 | if len(sys.argv) == 1: 53 | parser.print_help() 54 | sys.exit(1) 55 | 56 | args = parser.parse_args() 57 | if args.list_recipes: 58 | list_recipes() 59 | sys.exit(0) 60 | 61 | if args.recipe not in recipes.get_recipes(): 62 | print("Error: No such recipe") 63 | list_recipes() 64 | sys.exit(1) 65 | 66 | r = Recipe(recipe_name=args.recipe, use_playlists=args.playlists) 67 | r.run(sort_only=args.sort_only, share_playlist_to_all=args.everyone) 68 | 69 | print("Done!") 70 | 71 | 72 | if __name__ == "__main__": 73 | main() 74 | -------------------------------------------------------------------------------- /plexlibrary/plexutils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import plexapi.server 3 | import plexapi.media 4 | import requests 5 | import time 6 | from typing import List 7 | 8 | import logs 9 | 10 | 11 | class Plex(object): 12 | def __init__(self, baseurl, token): 13 | self.baseurl = baseurl 14 | self.token = token 15 | try: 16 | self.server = plexapi.server.PlexServer( 17 | baseurl=baseurl, token=token) 18 | except: 19 | raise Exception("No Plex server found at: {base_url}".format( 20 | base_url=baseurl)) 21 | 22 | def create_new_library(self, name, folder, library_type='movie'): 23 | headers = {"X-Plex-Token": self.token} 24 | params = { 25 | 'name': name, 26 | 'language': 'en', 27 | 'location': folder, 28 | } 29 | if library_type == 'movie': 30 | params['type'] = 'movie' 31 | params['agent'] = 'com.plexapp.agents.imdb' 32 | params['scanner'] = 'Plex Movie Scanner' 33 | elif library_type == 'tv': 34 | params['type'] = 'show' 35 | params['agent'] = 'com.plexapp.agents.thetvdb' 36 | params['scanner'] = 'Plex Series Scanner' 37 | else: 38 | raise Exception("Library type should be 'movie' or 'tv'") 39 | 40 | url = '{base_url}/library/sections'.format(base_url=self.baseurl) 41 | requests.post(url, headers=headers, params=params) 42 | 43 | def _user_has_access(self, user): 44 | for server in user.servers: 45 | if server.machineIdentifier == self.server.machineIdentifier: 46 | return True 47 | return False 48 | 49 | def _get_plex_instance_for_user(self, user): 50 | if self._user_has_access(user): 51 | return Plex(baseurl=self.baseurl, token=self.server.myPlexAccount().user(user.username).get_token( 52 | self.server.machineIdentifier)) 53 | return None 54 | 55 | def _get_all_users(self): 56 | return self.server.myPlexAccount().users() 57 | 58 | def _get_specific_users(self, user_names: List): 59 | users = [] 60 | for user in self._get_all_users(): 61 | if user.username in user_names: 62 | users.append(user) 63 | return users 64 | 65 | def _create_new_playlist(self, playlist_name, items: List[plexapi.media.Media]): 66 | self.server.createPlaylist(title=playlist_name, items=items) 67 | 68 | def _get_existing_playlist(self, playlist_name, user_name: str = None): 69 | if user_name: 70 | users = self._get_specific_users(user_names=[user_name]) 71 | if users: 72 | user_server = self._get_plex_instance_for_user(user=users[0]) 73 | if user_server: 74 | return user_server._get_existing_playlist(playlist_name=playlist_name) 75 | else: 76 | logs.info(f"{user_name} does not have access to your server.") 77 | else: 78 | for playlist in self.server.playlists(): 79 | if playlist.title == playlist_name: 80 | return playlist 81 | return None 82 | 83 | def get_playlist_items(self, playlist_name, user_name: str = None): 84 | playlist = self._get_existing_playlist(playlist_name=playlist_name, user_name=user_name) 85 | if playlist: 86 | return playlist.items() 87 | return [] 88 | 89 | def add_to_playlist_for_users(self, playlist_name, items: List[plexapi.media.Media], user_names: List = None, 90 | all_users: bool = False): 91 | users = [] 92 | if all_users: 93 | users = self._get_all_users() 94 | elif user_names: 95 | users = self._get_specific_users(user_names=user_names) 96 | # add on admin account 97 | self.add_to_playlist(playlist_name=playlist_name, items=items) 98 | # add for all other users 99 | for user in users: 100 | logs.info("Adding items to {user_name}'s {list_name} playlist".format(user_name=user.username, 101 | list_name=playlist_name)) 102 | user_server = self._get_plex_instance_for_user(user=user) 103 | if user_server: 104 | user_server.add_to_playlist(playlist_name=playlist_name, items=items) 105 | else: 106 | logs.info(f"{user.username} does not have access to your server.") 107 | 108 | def add_to_playlist(self, playlist_name, items: List[plexapi.media.Media]): 109 | playlist = self._get_existing_playlist(playlist_name=playlist_name) 110 | if playlist: 111 | playlist.addItems(items=items) 112 | else: 113 | self._create_new_playlist(playlist_name=playlist_name, items=items) 114 | 115 | def remove_from_playlist_for_users(self, playlist_name, items: List[plexapi.media.Media], user_names: List = None, 116 | all_users: bool = False): 117 | users = [] 118 | if all_users: 119 | users = self._get_all_users() 120 | elif user_names: 121 | users = self._get_specific_users(user_names=user_names) 122 | # remove on admin account 123 | self.remove_from_playlist(playlist_name=playlist_name, items=items) 124 | # remove for all other users 125 | for user in users: 126 | logs.info("Removing items from {user_name}'s {list_name} playlist".format(user_name=user.username, 127 | list_name=playlist_name)) 128 | user_server = self._get_plex_instance_for_user(user=user) 129 | if user_server: 130 | user_server.remove_from_playlist(playlist_name=playlist_name, items=items) 131 | else: 132 | logs.info(f"{user.username} does not have access to your server.") 133 | 134 | 135 | def remove_from_playlist(self, playlist_name, items: List[plexapi.media.Media]): 136 | playlist = self._get_existing_playlist(playlist_name=playlist_name) 137 | if playlist: 138 | for item in items: 139 | playlist.removeItem(item=item) 140 | 141 | def reset_playlist(self, playlist_name, new_items: List[plexapi.media.Media], user_names: List = None, 142 | all_users: bool = False): 143 | """ 144 | Delete old playlist and remake it with new items 145 | :param user_names: Make change for specific users, ["name", "name2", "name3"] 146 | :param all_users: Make change for all users 147 | :param new_items: list of Media objects 148 | :param playlist_name: 149 | :return: 150 | """ 151 | users = [] 152 | if all_users: 153 | users = self._get_all_users() 154 | elif user_names: 155 | users = self._get_specific_users(user_names=user_names) 156 | if users: # recursively reset for self and for each user 157 | self.reset_playlist(playlist_name=playlist_name, new_items=new_items) 158 | for user in users: 159 | logs.info("Resetting {list_name} playlist for {user_name}".format(list_name=playlist_name, 160 | user_name=user.username)) 161 | user_server = self._get_plex_instance_for_user(user=user) 162 | if user_server: 163 | user_server.reset_playlist(playlist_name=playlist_name, new_items=new_items) 164 | else: 165 | logs.info(f"{user.username} does not have access to your server.") 166 | 167 | else: 168 | playlist = self._get_existing_playlist(playlist_name=playlist_name) 169 | if playlist: 170 | playlist.delete() 171 | self._create_new_playlist(playlist_name=playlist_name, items=new_items) 172 | 173 | def _get_section_by_name(self, section_name): 174 | try: 175 | return self.server.library.section(title=section_name) 176 | except: 177 | pass 178 | return None 179 | 180 | def get_library_paths(self, library_name): 181 | section = self._get_section_by_name(section_name=library_name) 182 | if section: 183 | return section.locations 184 | return [] 185 | 186 | def set_sort_title(self, library_key, rating_key, number, title, 187 | library_type, title_format, visible=False): 188 | headers = {'X-Plex-Token': self.token} 189 | if library_type == 'movie': 190 | search_type = 1 191 | elif library_type == 'tv': 192 | search_type = 2 193 | params = { 194 | 'type': search_type, 195 | 'id': rating_key, 196 | 'titleSort.value': title_format.format( 197 | number=str(number).zfill(6), title=title), 198 | 'titleSort.locked': 1, 199 | } 200 | 201 | if visible: 202 | params['title.value'] = title_format.format( 203 | number=str(number), title=title) 204 | params['title.locked'] = 1 205 | else: 206 | params['title.value'] = title 207 | params['title.locked'] = 0 208 | 209 | url = "{base_url}/library/sections/{library}/all".format( 210 | base_url=self.baseurl, library=library_key) 211 | requests.put(url, headers=headers, params=params) 212 | -------------------------------------------------------------------------------- /plexlibrary/recipe.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """recipe 3 | """ 4 | 5 | import datetime 6 | import errno 7 | import os 8 | import random 9 | import subprocess 10 | import sys 11 | import time 12 | import logs 13 | import json 14 | 15 | import plexapi 16 | 17 | import plexutils 18 | import tmdb 19 | import traktutils 20 | import imdbutils 21 | import tvdb 22 | from config import ConfigParser 23 | from recipes import RecipeParser 24 | from utils import Colors, add_years 25 | 26 | 27 | class IdMap(): 28 | def __init__(self, matching_only=False, cache_file=None, 29 | match_imdb=None, match_tmdb=None, match_tvdb=None): 30 | self.items = set() 31 | self.imdb = {} 32 | self.tmdb = {} 33 | self.tvdb = {} 34 | self.matching_only = matching_only 35 | if cache_file: 36 | self.cache_file = cache_file 37 | else: 38 | self.cache_file = 'plex_guid_cache.json' 39 | self.cache = None 40 | self.cache_section = None 41 | self.cache_section_id = None 42 | if matching_only: 43 | self.match_imdb = match_imdb or [] 44 | self.match_tmdb = match_tmdb or [] 45 | self.match_tvdb = match_tvdb or [] 46 | 47 | def add_libraries(self, libraries): 48 | for library in libraries: 49 | self.add_items(library.all()) 50 | 51 | def add_items(self, items): 52 | while items: 53 | self.add_item(items.pop(0)) # Pop to save on memory 54 | 55 | def add_item(self, item): 56 | if item.guid.startswith('plex'): 57 | guids = self._get_guids(item) 58 | for guid in guids: 59 | self._add_id(guid, item) 60 | else: 61 | self._add_id(item.guid, item) 62 | 63 | def get(self, imdb=None, tmdb=None, tvdb=None): 64 | item = None 65 | if imdb: 66 | item = self.imdb.get(imdb) 67 | if not item and tmdb: 68 | item = self.tmdb.get(tmdb) 69 | if not item and tvdb: 70 | item = self.tvdb.get(tvdb) 71 | return item if item in self.items else None 72 | 73 | def pop_item(self, item): 74 | return self.pop(item=item) 75 | 76 | def pop(self, imdb=None, tmdb=None, tvdb=None, item=None): 77 | if imdb: 78 | item = self.imdb.pop(imdb, None) 79 | if not item and tmdb: 80 | item = self.tmdb.pop(tmdb, None) 81 | if not item and tvdb: 82 | item = self.tvdb.pop(tvdb, None) 83 | if not item: 84 | return None 85 | self._popall(item) 86 | try: 87 | self.items.remove(item) 88 | except KeyError: 89 | logs.warning("Item didn't exist in map set, collision?") 90 | return item 91 | 92 | def _get_guids(self, item): 93 | self._load_cache(str(item.librarySectionID)) 94 | guids = [] 95 | try: 96 | ts = item.updatedAt.timestamp() 97 | except AttributeError: 98 | logs.warning("Missing updatedAt timestamp for {} ({})".format( 99 | item.title, item.year)) 100 | ts = 0 101 | try: 102 | if (item.guid in self.cache and 103 | self.cache[item.guid]['updatedAt'] >= ts): 104 | guids = self.cache[item.guid]['guids'] 105 | except (KeyError, TypeError): 106 | logs.warning("Cache error, overwriting") 107 | guids = self.cache[item.guid] 108 | self.cache[item.guid] = { 109 | 'guids': guids, 110 | 'updatedAt': ts 111 | } 112 | self._save_cache() 113 | if not guids: 114 | guids = [guid.id for guid in item.guids] 115 | if guids: 116 | self.cache[item.guid] = { 117 | 'guids': guids, 118 | 'updatedAt': ts 119 | } 120 | self._save_cache() 121 | return guids 122 | 123 | def _load_cache(self, section_id): 124 | if self.cache and self.cache_section_id == section_id: 125 | return 126 | if not os.path.isfile(self.cache_file): 127 | with open(self.cache_file, 'w') as f: 128 | json.dump(dict(), f) 129 | with open(self.cache_file, 'r') as f: 130 | try: 131 | self._cache = json.load(f) 132 | except Exception as e: 133 | logs.warning("Unable to read cache, recreating ({})".format(e)) 134 | self._cache = dict() 135 | self.cache = self._cache.get(section_id, dict()) 136 | 137 | self.cache_section_id = section_id 138 | 139 | def _save_cache(self): 140 | self._cache[self.cache_section_id] = self.cache 141 | with open(self.cache_file, 'w') as f: 142 | json.dump(self._cache, f) 143 | 144 | def _add_id(self, guid, item): 145 | try: 146 | source, id_ = guid.split('://', 1) 147 | except ValueError: 148 | logs.warning(f"Unknown guid: {guid}") 149 | return 150 | id_ = id_.split('?')[0] 151 | if 'imdb' in source: 152 | if '/' in id_: 153 | id_ = id_.split('/')[-2] 154 | if self.matching_only and not id_ in self.match_imdb: 155 | return 156 | self.imdb[id_] = item 157 | elif 'tmdb' in source or 'themoviedb' in source: 158 | if self.matching_only and not id_ in self.match_tmdb: 159 | return 160 | self.tmdb[id_] = item 161 | elif 'tvdb' in source or 'thetvdb' in source: 162 | if '/' in id_: 163 | id_ = id_.split('/')[-2] 164 | if self.matching_only and not id_ in self.match_tvdb: 165 | return 166 | self.tvdb[id_] = item 167 | else: 168 | logs.warning(f"Unknown guid: {guid}. " 169 | f"Possibly unmatched: {item.title} ({item.year})") 170 | self.items.add(item) 171 | 172 | def _popall(self, item): 173 | items = [] 174 | for d in (self.imdb, self.tmdb, self.tvdb): 175 | keys = [] 176 | for k, v in d.items(): 177 | if v is item: 178 | keys.append(k) 179 | for k in keys: 180 | items.append(d.pop(k)) 181 | return items 182 | 183 | 184 | class Recipe(): 185 | plex = None 186 | trakt = None 187 | tmdb = None 188 | tvdb = None 189 | 190 | def __init__(self, recipe_name, sort_only=False, config_file=None, use_playlists=False): 191 | self.recipe_name = recipe_name 192 | self.use_playlists = use_playlists 193 | 194 | self.config = ConfigParser(config_file) 195 | self.recipe = RecipeParser(recipe_name) 196 | 197 | if not self.config.validate(): 198 | raise Exception("Error(s) in config") 199 | 200 | if not self.recipe.validate(use_playlists=use_playlists): 201 | raise Exception("Error(s) in recipe") 202 | 203 | if self.recipe['library_type'].lower().startswith('movie'): 204 | self.library_type = 'movie' 205 | elif self.recipe['library_type'].lower().startswith('tv'): 206 | self.library_type = 'tv' 207 | else: 208 | raise Exception("Library type should be 'movie' or 'tv'") 209 | 210 | self.source_library_config = self.recipe['source_libraries'] 211 | 212 | self.plex = plexutils.Plex(self.config['plex']['baseurl'], 213 | self.config['plex']['token']) 214 | 215 | if self.config['trakt']['username']: 216 | self.trakt = traktutils.Trakt( 217 | self.config['trakt']['username'], 218 | client_id=self.config['trakt']['client_id'], 219 | client_secret=self.config['trakt']['client_secret'], 220 | oauth_token=self.config['trakt'].get('oauth_token', ''), 221 | oauth=self.recipe.get('trakt_oauth', False), 222 | config=self.config) 223 | if self.trakt.oauth_token: 224 | self.config['trakt']['oauth_token'] = self.trakt.oauth_token 225 | 226 | if self.config['tmdb']['api_key']: 227 | self.tmdb = tmdb.TMDb( 228 | self.config['tmdb']['api_key'], 229 | cache_file=self.config['tmdb']['cache_file']) 230 | 231 | if self.config['tvdb']['username']: 232 | self.tvdb = tvdb.TheTVDB(self.config['tvdb']['username'], 233 | self.config['tvdb']['api_key'], 234 | self.config['tvdb']['user_key']) 235 | 236 | self.imdb = imdbutils.IMDb(self.tmdb, self.tvdb) 237 | 238 | self.source_map = IdMap(matching_only=True, 239 | cache_file=self.config.get('guid_cache_file')) 240 | self.dest_map = IdMap(cache_file=self.config.get('guid_cache_file')) 241 | 242 | 243 | 244 | def _get_trakt_lists(self): 245 | item_list = [] # TODO Replace with dict, scrap item_ids? 246 | item_ids = [] 247 | 248 | for url in self.recipe['source_list_urls']: 249 | max_age = (self.recipe['new_playlist'].get('max_age', 0) if self.use_playlists 250 | else self.recipe['new_library'].get('max_age', 0)) 251 | if 'api.trakt.tv' in url: 252 | (item_list, item_ids) = self.trakt.add_items( 253 | self.library_type, url, item_list, item_ids, 254 | max_age or 0) 255 | elif 'imdb.com/chart' in url: 256 | (item_list, item_ids) = self.imdb.add_items( 257 | self.library_type, url, item_list, item_ids, 258 | max_age or 0) 259 | else: 260 | raise Exception("Unsupported source list: {url}".format( 261 | url=url)) 262 | 263 | if self.recipe['weighted_sorting']['enabled']: 264 | if self.config['tmdb']['api_key']: 265 | logs.info(u"Getting data from TMDb to add weighted sorting...") 266 | item_list = self.weighted_sorting(item_list) 267 | else: 268 | logs.warning(u"Warning: TMDd API key is required " 269 | u"for weighted sorting") 270 | return item_list, item_ids 271 | 272 | def _get_plex_libraries(self): 273 | source_libraries = [] 274 | for library_config in self.source_library_config: 275 | logs.info(u"Trying to match with items from the '{}' library ".format( 276 | library_config['name'])) 277 | try: 278 | source_library = self.plex.server.library.section( 279 | library_config['name']) 280 | except: # FIXME 281 | raise Exception("The '{}' library does not exist".format( 282 | library_config['name'])) 283 | 284 | source_libraries.append(source_library) 285 | return source_libraries 286 | 287 | def _get_matching_items(self, source_libraries, item_list): 288 | matching_items = [] 289 | missing_items = [] 290 | matching_total = 0 291 | nonmatching_idx = [] 292 | max_count = (self.recipe['new_playlist'].get('max_count', 0) if self.use_playlists 293 | else self.recipe['new_library'].get('max_count', 0)) 294 | 295 | for i, item in enumerate(item_list): 296 | if 0 < max_count <= matching_total: 297 | nonmatching_idx.append(i) 298 | continue 299 | res = self.source_map.get(item.get('id'), item.get('tmdb_id'), 300 | item.get('tvdb_id')) 301 | 302 | if not res: 303 | missing_items.append((i, item)) 304 | nonmatching_idx.append(i) 305 | continue 306 | 307 | matching_total += 1 308 | matching_items += res 309 | 310 | if not self.use_playlists and self.recipe['new_library']['sort_title']['absolute']: 311 | logs.info(u"{} {} ({})".format( 312 | i + 1, item['title'], item['year'])) 313 | else: 314 | logs.info(u"{} {} ({})".format( 315 | matching_total, item['title'], item['year'])) 316 | 317 | if not self.use_playlists and not self.recipe['new_library']['sort_title']['absolute']: 318 | for i in reversed(nonmatching_idx): 319 | del item_list[i] 320 | 321 | return matching_items, missing_items, matching_total, nonmatching_idx, max_count 322 | 323 | def _create_symbolic_links(self, matching_items, matching_total): 324 | logs.info(u"Creating symlinks for {count} matching items in the " 325 | u"library...".format(count=matching_total)) 326 | 327 | try: 328 | if not os.path.exists(self.recipe['new_library']['folder']): 329 | os.mkdir(self.recipe['new_library']['folder']) 330 | except: 331 | logs.error(u"Unable to create the new library folder " 332 | u"'{folder}'.".format(folder=self.recipe['new_library']['folder'])) 333 | logs.info(u"Exiting script.") 334 | return 0 335 | 336 | count = 0 337 | updated_paths = [] 338 | new_items = [] 339 | if self.library_type == 'movie': 340 | for movie in matching_items: 341 | for part in movie.iterParts(): 342 | old_path_file = part.file 343 | old_path, file_name = os.path.split(old_path_file) 344 | 345 | folder_name = '' 346 | for library_config in self.source_library_config: 347 | for f in self.plex.get_library_paths(library_name=library_config['name']): 348 | f = os.path.abspath(f) 349 | if old_path.lower().startswith(f.lower()): 350 | folder_name = os.path.relpath(old_path, f) 351 | break 352 | else: 353 | continue 354 | 355 | if folder_name == '.': 356 | new_path = os.path.join( 357 | self.recipe['new_library']['folder'], 358 | file_name) 359 | dir = False 360 | else: 361 | new_path = os.path.join( 362 | self.recipe['new_library']['folder'], 363 | folder_name) 364 | dir = True 365 | parent_path = os.path.dirname( 366 | os.path.abspath(new_path)) 367 | if not os.path.exists(parent_path): 368 | try: 369 | os.makedirs(parent_path) 370 | except OSError as e: 371 | if e.errno == errno.EEXIST \ 372 | and os.path.isdir(parent_path): 373 | pass 374 | else: 375 | raise 376 | # Clean up old, empty directories 377 | if os.path.exists(new_path) \ 378 | and not os.listdir(new_path): 379 | os.rmdir(new_path) 380 | 381 | if (dir and not os.path.exists(new_path)) \ 382 | or not dir and not os.path.isfile(new_path): 383 | try: 384 | if os.name == 'nt': 385 | if dir: 386 | subprocess.call(['mklink', '/D', 387 | new_path, old_path], 388 | shell=True) 389 | else: 390 | subprocess.call(['mklink', new_path, 391 | old_path_file], 392 | shell=True) 393 | else: 394 | if dir: 395 | os.symlink(old_path, new_path) 396 | else: 397 | os.symlink(old_path_file, new_path) 398 | count += 1 399 | new_items.append(movie) 400 | updated_paths.append(new_path) 401 | except Exception as e: 402 | logs.error(u"Symlink failed for {path}: {e}".format( 403 | path=new_path, e=e)) 404 | else: 405 | for tv_show in matching_items: 406 | done = False 407 | if done: 408 | continue 409 | for episode in tv_show.episodes(): 410 | if done: 411 | break 412 | for part in episode.iterParts(): 413 | old_path_file = part.file 414 | old_path, file_name = os.path.split(old_path_file) 415 | 416 | folder_name = '' 417 | for library_config in self.source_library_config: 418 | for f in self.plex.get_library_paths(library_name=library_config['name']): 419 | if old_path.lower().startswith(f.lower()): 420 | old_path = os.path.join(f, 421 | old_path.replace( 422 | f, '').strip( 423 | os.sep).split( 424 | os.sep)[0]) 425 | folder_name = os.path.relpath(old_path, f) 426 | break 427 | else: 428 | continue 429 | 430 | new_path = os.path.join( 431 | self.recipe['new_library']['folder'], 432 | folder_name) 433 | 434 | if not os.path.exists(new_path): 435 | try: 436 | if os.name == 'nt': 437 | subprocess.call(['mklink', '/D', 438 | new_path, old_path], 439 | shell=True) 440 | else: 441 | os.symlink(old_path, new_path) 442 | count += 1 443 | new_items.append(tv_show) 444 | updated_paths.append(new_path) 445 | done = True 446 | break 447 | except Exception as e: 448 | logs.error(u"Symlink failed for {path}: {e}" 449 | .format(path=new_path, e=e)) 450 | else: 451 | done = True 452 | break 453 | 454 | logs.info(u"Created symlinks for {count} new items:".format(count=count)) 455 | for item in new_items: 456 | logs.info(u"{title} ({year})".format(title=item.title, year=getattr(item, 'year', None))) 457 | 458 | def _verify_new_library_and_get_items(self, create_if_not_found=False): 459 | # Check if the new library exists in Plex 460 | try: 461 | new_library = self.plex.server.library.section( 462 | self.recipe['new_library']['name']) 463 | logs.info(u"Library already exists in Plex. Scanning the library...") 464 | 465 | new_library.update() 466 | except plexapi.exceptions.NotFound: 467 | if create_if_not_found: 468 | self.plex.create_new_library( 469 | self.recipe['new_library']['name'], 470 | self.recipe['new_library']['folder'], 471 | self.library_type) 472 | new_library = self.plex.server.library.section( 473 | self.recipe['new_library']['name']) 474 | else: 475 | raise Exception("Library '{library}' does not exist".format( 476 | library=self.recipe['new_library']['name'])) 477 | 478 | # Wait for metadata to finish downloading before continuing 479 | logs.info(u"Waiting for metadata to finish downloading...") 480 | new_library = self.plex.server.library.section( 481 | self.recipe['new_library']['name']) 482 | while new_library.refreshing: 483 | time.sleep(5) 484 | new_library = self.plex.server.library.section( 485 | self.recipe['new_library']['name']) 486 | 487 | # Retrieve a list of items from the new library 488 | logs.info(u"Retrieving a list of items from the '{library}' library in " 489 | u"Plex...".format(library=self.recipe['new_library']['name'])) 490 | return new_library, new_library.all() 491 | 492 | def _modify_sort_titles_and_cleanup(self, item_list, new_library, 493 | sort_only=False): 494 | if self.recipe['new_library']['sort']: 495 | logs.info(u"Setting the sort titles for the '{}' library".format( 496 | self.recipe['new_library']['name'])) 497 | if self.recipe['new_library']['sort_title']['absolute']: 498 | for i, m in enumerate(item_list): 499 | item = self.dest_map.pop(m.get('id'), m.get('tmdb_id'), m.get('tvdb_id')) 500 | if item and self.recipe['new_library']['sort']: 501 | self.plex.set_sort_title( 502 | new_library.key, item.ratingKey, i + 1, m['title'], 503 | self.library_type, 504 | self.recipe['new_library']['sort_title']['format'], 505 | self.recipe['new_library']['sort_title']['visible'] 506 | ) 507 | else: 508 | i = 0 509 | for m in item_list: 510 | i += 1 511 | item = self.dest_map.pop(m.get('id'), m.get('tmdb_id'), m.get('tvdb_id')) 512 | if item and self.recipe['new_library']['sort']: 513 | self.plex.set_sort_title( 514 | new_library.key, item.ratingKey, i, m['title'], 515 | self.library_type, 516 | self.recipe['new_library']['sort_title']['format'], 517 | self.recipe['new_library']['sort_title']['visible'] 518 | ) 519 | unmatched_items = list(self.dest_map.items) 520 | if not sort_only and ( 521 | self.recipe['new_library']['remove_from_library'] or 522 | self.recipe['new_library'].get('remove_old', False)): 523 | # Remove old items that no longer qualify 524 | self._remove_old_items_from_library(unmatched_items) 525 | elif sort_only: 526 | return True 527 | if self.recipe['new_library']['sort'] and \ 528 | not self.recipe['new_library']['remove_from_library']: 529 | unmatched_items.sort(key=lambda x: x.titleSort) 530 | while unmatched_items: 531 | item = unmatched_items.pop(0) 532 | i += 1 533 | logs.info(u"{} {} ({})".format(i, item.title, item.year)) 534 | self.plex.set_sort_title( 535 | new_library.key, item.ratingKey, i, item.title, 536 | self.library_type, 537 | self.recipe['new_library']['sort_title']['format'], 538 | self.recipe['new_library']['sort_title']['visible']) 539 | all_new_items = self._cleanup_new_library(new_library=new_library) 540 | return all_new_items 541 | 542 | def _remove_old_items_from_library(self, unmatched_items): 543 | logs.info(u"Removing symlinks for items " 544 | "which no longer qualify ".format(library=self.recipe['new_library']['name'])) 545 | count = 0 546 | updated_paths = [] 547 | deleted_items = [] 548 | max_date = add_years( 549 | (self.recipe['new_library']['max_age'] or 0) * -1) 550 | if self.library_type == 'movie': 551 | for movie in unmatched_items: 552 | if not self.recipe['new_library']['remove_from_library']: 553 | # Only remove older than max_age 554 | if not self.recipe['new_library']['max_age'] \ 555 | or (movie.originallyAvailableAt and 556 | max_date < movie.originallyAvailableAt): 557 | continue 558 | 559 | for part in movie.iterParts(): 560 | old_path_file = part.file 561 | old_path, file_name = os.path.split(old_path_file) 562 | 563 | folder_name = os.path.relpath( 564 | old_path, self.recipe['new_library']['folder']) 565 | 566 | if folder_name == '.': 567 | new_path = os.path.join( 568 | self.recipe['new_library']['folder'], 569 | file_name) 570 | dir = False 571 | else: 572 | new_path = os.path.join( 573 | self.recipe['new_library']['folder'], 574 | folder_name) 575 | dir = True 576 | 577 | if (dir and os.path.exists(new_path)) or ( 578 | not dir and os.path.isfile(new_path)): 579 | try: 580 | if os.name == 'nt': 581 | # Python 3.2+ only 582 | if sys.version_info < (3, 2): 583 | assert os.path.islink(new_path) 584 | if dir: 585 | os.rmdir(new_path) 586 | else: 587 | os.remove(new_path) 588 | else: 589 | assert os.path.islink(new_path) 590 | os.unlink(new_path) 591 | count += 1 592 | deleted_items.append(movie) 593 | updated_paths.append(new_path) 594 | except Exception as e: 595 | logs.error(u"Remove symlink failed for " 596 | "{path}: {e}".format(path=new_path, e=e)) 597 | else: 598 | for tv_show in unmatched_items: 599 | done = False 600 | if done: 601 | continue 602 | for episode in tv_show.episodes(): 603 | if done: 604 | break 605 | for part in episode.iterParts(): 606 | if done: 607 | break 608 | old_path_file = part.file 609 | old_path, file_name = os.path.split(old_path_file) 610 | 611 | folder_name = '' 612 | new_library_folder = \ 613 | self.recipe['new_library']['folder'] 614 | old_path = os.path.join( 615 | new_library_folder, 616 | old_path.replace(new_library_folder, '').strip( 617 | os.sep).split(os.sep)[0]) 618 | folder_name = os.path.relpath(old_path, 619 | new_library_folder) 620 | 621 | new_path = os.path.join( 622 | self.recipe['new_library']['folder'], 623 | folder_name) 624 | if os.path.exists(new_path): 625 | try: 626 | if os.name == 'nt': 627 | # Python 3.2+ only 628 | if sys.version_info < (3, 2): 629 | assert os.path.islink(new_path) 630 | os.rmdir(new_path) 631 | else: 632 | assert os.path.islink(new_path) 633 | os.unlink(new_path) 634 | count += 1 635 | deleted_items.append(tv_show) 636 | updated_paths.append(new_path) 637 | done = True 638 | break 639 | except Exception as e: 640 | logs.error(u"Remove symlink failed for " 641 | "{path}: {e}".format(path=new_path, 642 | e=e)) 643 | else: 644 | done = True 645 | break 646 | 647 | logs.info(u"Removed symlinks for {count} items.".format(count=count)) 648 | for item in deleted_items: 649 | logs.info(u"{title} ({year})".format(title=item.title, 650 | year=item.year)) 651 | 652 | def _cleanup_new_library(self, new_library): 653 | # Scan the library to clean up the deleted items 654 | logs.info(u"Scanning the '{library}' library...".format( 655 | library=self.recipe['new_library']['name'])) 656 | new_library.update() 657 | time.sleep(10) 658 | new_library = self.plex.server.library.section( 659 | self.recipe['new_library']['name']) 660 | while new_library.refreshing: 661 | time.sleep(5) 662 | new_library = self.plex.server.library.section( 663 | self.recipe['new_library']['name']) 664 | new_library.emptyTrash() 665 | return new_library.all() 666 | 667 | def _run(self, share_playlist_to_all=False): 668 | # Get the trakt lists 669 | item_list, item_ids = self._get_trakt_lists() 670 | force_imdb_id_match = False 671 | 672 | # Get list of items from the Plex server 673 | source_libraries = self._get_plex_libraries() 674 | 675 | # Populate source library guid map 676 | for item in item_list: 677 | if item.get('id'): 678 | self.source_map.match_imdb.append(item['id']) 679 | if item.get('tmdb_id'): 680 | self.source_map.match_tmdb.append(item['tmdb_id']) 681 | if item.get('tvdb_id'): 682 | self.source_map.match_tvdb.append(item['tvdb_id']) 683 | self.source_map.add_libraries(source_libraries) 684 | 685 | # Create a list of matching items 686 | matching_items, missing_items, matching_total, nonmatching_idx, max_count = self._get_matching_items( 687 | source_libraries, item_list) 688 | 689 | if self.use_playlists: 690 | # Start playlist process 691 | if self.recipe['new_playlist']['remove_from_playlist'] or self.recipe['new_playlist'].get('remove_old', 692 | False): 693 | # Start playlist over again 694 | self.plex.reset_playlist(playlist_name=self.recipe['new_playlist']['name'], new_items=matching_items, 695 | user_names=self.recipe['new_playlist'].get('share_to_users', []), 696 | all_users=(share_playlist_to_all if share_playlist_to_all else 697 | self.recipe['new_playlist'].get('share_to_all', False))) 698 | else: 699 | # Keep existing items 700 | self.plex.add_to_playlist_for_users(playlist_name=self.recipe['new_playlist']['name'], 701 | items=matching_items, 702 | user_names=self.recipe['new_playlist'].get('share_to_users', []), 703 | all_users=(share_playlist_to_all if share_playlist_to_all else 704 | self.recipe['new_playlist'].get('share_to_all', False))) 705 | playlist_items = self.plex.get_playlist_items(playlist_name=self.recipe['new_playlist']['name']) 706 | return missing_items, (len(playlist_items) if playlist_items else 0) 707 | else: 708 | # Start library process 709 | # Create symlinks for all items in your library on the trakt watched 710 | self._create_symbolic_links(matching_items=matching_items, matching_total=matching_total) 711 | # Post-process new library 712 | logs.info(u"Creating the '{}' library in Plex...".format( 713 | self.recipe['new_library']['name'])) 714 | new_library, all_new_items = self._verify_new_library_and_get_items(create_if_not_found=True) 715 | self.dest_map.add_items(all_new_items) 716 | # Modify the sort titles 717 | all_new_items = self._modify_sort_titles_and_cleanup( 718 | item_list, new_library, sort_only=False) 719 | return missing_items, len(all_new_items) 720 | 721 | def _run_sort_only(self): 722 | item_list, item_ids = self._get_trakt_lists() 723 | force_imdb_id_match = False 724 | 725 | # Get existing library and its items 726 | new_library, all_new_items = self._verify_new_library_and_get_items(create_if_not_found=False) 727 | self.dest_map.add_items(all_new_items) 728 | # Modify the sort titles 729 | self._modify_sort_titles_and_cleanup(item_list, new_library, 730 | sort_only=True) 731 | return len(all_new_items) 732 | 733 | def run(self, sort_only=False, share_playlist_to_all=False): 734 | if sort_only: 735 | logs.info(u"Running the recipe '{}', sorting only".format( 736 | self.recipe_name)) 737 | list_count = self._run_sort_only() 738 | logs.info(u"Number of items in the new {library_or_playlist}: {count}".format( 739 | count=list_count, library_or_playlist=('playlist' if self.use_playlists else 'library'))) 740 | else: 741 | logs.info(u"Running the recipe '{}'".format(self.recipe_name)) 742 | missing_items, list_count = self._run(share_playlist_to_all=share_playlist_to_all) 743 | logs.info(u"Number of items in the new {library_or_playlist}: {count}".format( 744 | count=list_count, library_or_playlist=('playlist' if self.use_playlists else 'library'))) 745 | logs.info(u"Number of missing items: {count}".format( 746 | count=len(missing_items))) 747 | for idx, item in missing_items: 748 | logs.info(u"{idx}\t{release}\t{imdb_id}\t{title} ({year})".format( 749 | idx=idx + 1, release=item.get('release_date', ''), 750 | imdb_id=item['id'], title=item['title'], 751 | year=item['year'])) 752 | 753 | def weighted_sorting(self, item_list): 754 | def _get_non_theatrical_release(release_dates): 755 | # Returns earliest release date that is not theatrical 756 | # TODO PREDB 757 | types = {} 758 | for country in release_dates.get('results', []): 759 | # FIXME Look at others too? 760 | if country['iso_3166_1'] != 'US': 761 | continue 762 | for d in country['release_dates']: 763 | if d['type'] in (4, 5, 6): 764 | # 4: Digital, 5: Physical, 6: TV 765 | types[str(d['type'])] = datetime.datetime.strptime( 766 | d['release_date'], '%Y-%m-%dT%H:%M:%S.%fZ').date() 767 | break 768 | 769 | release_date = None 770 | for t, d in types.items(): 771 | if not release_date or d < release_date: 772 | release_date = d 773 | 774 | return release_date 775 | 776 | def _get_age_weight(days): 777 | if self.library_type == 'movie': 778 | # Everything younger than this will get 1 779 | min_days = 180 780 | # Everything older than this will get 0 781 | max_days = (float(self.recipe['new_library']['max_age']) 782 | / 4.0 * 365.25 or 360) 783 | else: 784 | min_days = 14 785 | max_days = (float(self.recipe['new_library']['max_age']) 786 | / 4.0 * 365.25 or 180) 787 | if days <= min_days: 788 | return 1 789 | elif days >= max_days: 790 | return 0 791 | else: 792 | return 1 - (days - min_days) / (max_days - min_days) 793 | 794 | total_items = len(item_list) 795 | 796 | weights = self.recipe['weighted_sorting']['weights'] 797 | 798 | # TMDB details 799 | today = datetime.date.today() 800 | total_tmdb_vote = 0.0 801 | tmdb_votes = [] 802 | for i, m in enumerate(item_list): 803 | m['original_idx'] = i + 1 804 | details = self.tmdb.get_details(m['tmdb_id'], self.library_type) 805 | if not details: 806 | logs.warning(u"Warning: No TMDb data for {}".format(m['title'])) 807 | continue 808 | m['tmdb_popularity'] = float(details['popularity']) 809 | m['tmdb_vote'] = float(details['vote_average']) 810 | m['tmdb_vote_count'] = int(details['vote_count']) 811 | if self.library_type == 'movie': 812 | if self.recipe['weighted_sorting']['better_release_date']: 813 | m['release_date'] = _get_non_theatrical_release( 814 | details['release_dates']) or \ 815 | datetime.datetime.strptime( 816 | details['release_date'], 817 | '%Y-%m-%d').date() 818 | else: 819 | m['release_date'] = datetime.datetime.strptime( 820 | details['release_date'], '%Y-%m-%d').date() 821 | item_age_td = today - m['release_date'] 822 | elif self.library_type == 'tv': 823 | try: 824 | m['last_air_date'] = datetime.datetime.strptime( 825 | details['last_air_date'], '%Y-%m-%d').date() 826 | except TypeError: 827 | m['last_air_date'] = today 828 | item_age_td = today - m['last_air_date'] 829 | m['genres'] = [g['name'].lower() for g in details['genres']] 830 | m['age'] = item_age_td.days 831 | if (self.library_type == 'tv' or m['tmdb_vote_count'] > 150 or 832 | m['age'] > 50): 833 | tmdb_votes.append(m['tmdb_vote']) 834 | total_tmdb_vote += m['tmdb_vote'] 835 | item_list[i] = m 836 | 837 | tmdb_votes.sort() 838 | 839 | for i, m in enumerate(item_list): 840 | # Distribute all weights evenly from 0 to 1 (times global factor) 841 | # More weight means it'll go higher in the final list 842 | index_weight = float(total_items - i) / float(total_items) 843 | m['index_weight'] = index_weight * weights['index'] 844 | if m.get('tmdb_popularity'): 845 | if (self.library_type == 'tv' or 846 | m.get('tmdb_vote_count') > 150 or m['age'] > 50): 847 | vote_weight = ((tmdb_votes.index(m['tmdb_vote']) + 1) 848 | / float(len(tmdb_votes))) 849 | else: 850 | # Assume below average rating for new/less voted items 851 | vote_weight = 0.25 852 | age_weight = _get_age_weight(float(m['age'])) 853 | 854 | if weights.get('random'): 855 | random_weight = random.random() 856 | m['random_weight'] = random_weight * weights['random'] 857 | else: 858 | m['random_weight'] = 0.0 859 | 860 | m['vote_weight'] = vote_weight * weights['vote'] 861 | m['age_weight'] = age_weight * weights['age'] 862 | 863 | weight = (m['index_weight'] + m['vote_weight'] 864 | + m['age_weight'] + m['random_weight']) 865 | for genre, value in weights['genre_bias'].items(): 866 | if genre.lower() in m['genres']: 867 | weight *= value 868 | 869 | m['weight'] = weight 870 | else: 871 | m['vote_weight'] = 0.0 872 | m['age_weight'] = 0.0 873 | m['weight'] = index_weight 874 | item_list[i] = m 875 | 876 | item_list.sort(key=lambda m: m['weight'], reverse=True) 877 | 878 | for i, m in enumerate(item_list): 879 | if (i + 1) < m['original_idx']: 880 | net = Colors.GREEN + u'↑' 881 | elif (i + 1) > m['original_idx']: 882 | net = Colors.RED + u'↓' 883 | else: 884 | net = u' ' 885 | net += str(abs(i + 1 - m['original_idx'])).rjust(3) 886 | try: 887 | # TODO 888 | logs.info(u"{} {:>3}: trnd:{:>3}, w_trnd:{:0<5}; vote:{}, " 889 | "w_vote:{:0<5}; age:{:>4}, w_age:{:0<5}; w_rnd:{:0<5}; " 890 | "w_cmb:{:0<5}; {} {}{}" 891 | .format(net, i + 1, m['original_idx'], 892 | round(m['index_weight'], 3), 893 | m.get('tmdb_vote', 0.0), 894 | round(m['vote_weight'], 3), m.get('age', 0), 895 | round(m['age_weight'], 3), 896 | round(m.get('random_weight', 0), 3), 897 | round(m['weight'], 3), str(m['title']), 898 | str(m['year']), Colors.RESET)) 899 | except UnicodeEncodeError: 900 | pass 901 | 902 | return item_list 903 | -------------------------------------------------------------------------------- /plexlibrary/recipes.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import glob 3 | import os 4 | 5 | import logs 6 | 7 | from utils import YAMLBase 8 | 9 | 10 | class RecipeParser(YAMLBase): 11 | def __init__(self, name, directory=None): 12 | # TODO accept filename 13 | self.name = os.path.splitext(name)[0] 14 | recipe_file = self.name + '.yml' # TODO support .yaml 15 | # FIXME? 16 | if not directory: 17 | parent_dir = os.path.abspath( 18 | os.path.join(os.path.dirname(__file__), os.path.pardir)) 19 | directory = os.path.join(parent_dir, 'recipes') 20 | 21 | filepath = os.path.join(directory, recipe_file) 22 | 23 | super(RecipeParser, self).__init__(filepath) 24 | 25 | def dump(self): 26 | logs.info(self.data) 27 | 28 | def validate(self, use_playlists: bool = False): 29 | if not self.get('library_type'): 30 | raise Exception("Missing 'library_type' in recipe") 31 | else: 32 | if not self['library_type'].lower().startswith('movie') \ 33 | and not self['library_type'].lower().startswith('tv'): 34 | raise Exception("'library_type' should be 'movie' or 'tv'") 35 | 36 | if not self.get('source_list_urls'): 37 | raise Exception("Missing 'source_list_urls' in recipe") 38 | 39 | if not self.get('source_libraries'): 40 | raise Exception("Missing 'source_libraries' in recipe") 41 | else: 42 | for i in self['source_libraries']: 43 | if 'name' not in i: 44 | raise Exception("Missing 'name' in 'source_libraries'") 45 | 46 | if use_playlists: # check new_playlist section 47 | if not self.get('new_playlist'): 48 | raise Exception("Missing 'new_playlist' in recipe") 49 | else: 50 | if not self['new_playlist'].get('name'): 51 | raise Exception("Missing 'name' in 'new_playlist'") 52 | 53 | else: # check new_library section 54 | if not self.get('new_library'): 55 | raise Exception("Missing 'new_library' in recipe") 56 | else: 57 | if not self['new_library'].get('name'): 58 | raise Exception("Missing 'name' in 'new_library'") 59 | if not self['new_library'].get('folder'): 60 | raise Exception("Missing 'folder' in 'new_library'") 61 | if self['new_library'].get('sort_title'): 62 | if 'format' not in self['new_library']['sort_title']: 63 | raise Exception("Missing 'format' in 'sort_title'") 64 | if 'visible' not in self['new_library']['sort_title']: 65 | raise Exception("Missing 'visible' in 'sort_title'") 66 | if 'absolute' not in self['new_library']['sort_title']: 67 | raise Exception("Missing 'absolute' in 'sort_title'") 68 | 69 | if not self.get('weighted_sorting'): 70 | raise Exception("Missing 'weighted_sorting' in recipe") 71 | else: 72 | if 'enabled' not in self['weighted_sorting']: 73 | raise Exception("Missing 'enabled' in 'weighted_sorting'") 74 | else: 75 | if 'better_release_date' not in self['weighted_sorting']: 76 | raise Exception("Missing 'better_release_date' in 'weighted_sorting'") 77 | if 'weights' not in self['weighted_sorting']: 78 | raise Exception("Missing 'weights' in 'weighted_sorting'") 79 | else: 80 | if 'index' not in self['weighted_sorting']['weights']: 81 | raise Exception("Missing 'index' in 'weights'") 82 | if 'vote' not in self['weighted_sorting']['weights']: 83 | raise Exception("Missing 'vote' in 'weights'") 84 | if 'age' not in self['weighted_sorting']['weights']: 85 | raise Exception("Missing 'age' in 'weights'") 86 | if 'random' not in self['weighted_sorting']['weights']: 87 | raise Exception("Missing 'random' in 'weights'") 88 | if 'genre_bias' not in self['weighted_sorting']['weights']: 89 | raise Exception("Missing 'genre_bias' in 'weights'") 90 | 91 | return True 92 | 93 | 94 | def get_recipes(directory=None): 95 | if not directory: 96 | parent_dir = os.path.abspath( 97 | os.path.join(os.path.dirname(__file__), os.path.pardir)) 98 | directory = os.path.join(parent_dir, 'recipes') 99 | 100 | recipes = [] 101 | for path in glob.glob(os.path.join(directory, '*.yml')): 102 | d, filename = os.path.split(path) 103 | recipe_name = os.path.splitext(filename)[0] 104 | recipes.append(recipe_name) 105 | recipes.sort() 106 | 107 | return recipes 108 | -------------------------------------------------------------------------------- /plexlibrary/tmdb.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import json 3 | import shelve 4 | import time 5 | try: 6 | from cPickle import UnpicklingError 7 | except ImportError: 8 | from pickle import UnpicklingError 9 | 10 | import requests 11 | 12 | import logs 13 | 14 | 15 | class TMDb(object): 16 | api_key = None 17 | cache_file = None 18 | request_count = 0 19 | 20 | def __init__(self, api_key, cache_file=None): 21 | self.api_key = api_key 22 | if cache_file: 23 | self.cache_file = cache_file 24 | else: 25 | self.cache_file = 'tmdb_details.shelve' 26 | 27 | def get_imdb_id(self, tmdb_id, library_type='movie'): 28 | if library_type not in ('movie', 'tv'): 29 | raise Exception("Library type should be 'movie' or 'tv'") 30 | 31 | # Use cache 32 | cache = shelve.open(self.cache_file) 33 | if str(tmdb_id) in cache: 34 | try: 35 | cache_item = cache[str(tmdb_id)] 36 | except (EOFError, UnpicklingError): 37 | # Cache file error, clear 38 | cache.close() 39 | cache = shelve.open(self.cache_file, 'n') 40 | else: 41 | if (cache_item['cached'] + 3600 * 24) > int(time.time()): 42 | cache.close() 43 | return cache_item.get('imdb_id') 44 | 45 | # Wait 10 seconds for the TMDb rate limit 46 | if self.request_count >= 40: 47 | logs.info(u"Waiting 10 seconds for the TMDb rate limit...") 48 | time.sleep(10) 49 | self.request_count = 0 50 | 51 | params = { 52 | 'api_key': self.api_key, 53 | } 54 | 55 | if library_type == 'movie': 56 | url = "https://api.themoviedb.org/3/movie/{tmdb_id}".format( 57 | tmdb_id=tmdb_id) 58 | else: 59 | url = ("https://api.themoviedb.org/3/tv/{tmdb_id}/external_ids" 60 | .format(tmdb_id=tmdb_id)) 61 | r = requests.get(url, params=params) 62 | 63 | self.request_count += 1 64 | 65 | if r.status_code == 200: 66 | item = json.loads(r.text) 67 | item['cached'] = int(time.time()) 68 | cache[str(tmdb_id)] = item 69 | cache.close() 70 | return item.get('imdb_id') 71 | else: 72 | return None 73 | 74 | def get_details(self, tmdb_id, library_type='movie'): 75 | if library_type not in ('movie', 'tv'): 76 | raise Exception("Library type should be 'movie' or 'tv'") 77 | 78 | # Use cache 79 | cache = shelve.open(self.cache_file) 80 | if str(tmdb_id) in cache: 81 | try: 82 | cache_item = cache[str(tmdb_id)] 83 | except (EOFError, UnpicklingError): 84 | # Cache file error, clear 85 | cache.close() 86 | cache = shelve.open(self.cache_file, 'n') 87 | except: 88 | # Unknown cache file error, clear 89 | logs.error(u"Error in loading cache: {}".format(e)) 90 | cache.close() 91 | cache = shelve.open(self.cache_file, 'n') 92 | else: 93 | if (cache_item['cached'] + 3600 * 24) > int(time.time()): 94 | cache.close() 95 | return cache_item 96 | 97 | # Wait 10 seconds for the TMDb rate limit 98 | if self.request_count >= 40: 99 | logs.info(u"Waiting 10 seconds for the TMDb rate limit...") 100 | time.sleep(10) 101 | self.request_count = 0 102 | 103 | params = { 104 | 'api_key': self.api_key, 105 | } 106 | 107 | if library_type == 'movie': 108 | params['append_to_response'] = 'release_dates' 109 | url = "https://api.themoviedb.org/3/movie/{tmdb_id}".format( 110 | tmdb_id=tmdb_id) 111 | else: 112 | url = "https://api.themoviedb.org/3/tv/{tmdb_id}".format( 113 | tmdb_id=tmdb_id) 114 | r = requests.get(url, params=params) 115 | 116 | self.request_count += 1 117 | 118 | if r.status_code == 200: 119 | item = json.loads(r.text) 120 | item['cached'] = int(time.time()) 121 | cache[str(tmdb_id)] = item 122 | cache.close() 123 | return item 124 | else: 125 | return None 126 | 127 | def get_tmdb_from_imdb(self, imdb_id, library_type): 128 | if library_type not in ('movie', 'tv'): 129 | raise Exception("Library type should be 'movie' or 'tv'") 130 | 131 | # Use cache 132 | cache = shelve.open(self.cache_file) 133 | if str(imdb_id) in cache: 134 | try: 135 | cache_item = cache[str(imdb_id)] 136 | except (EOFError, UnpicklingError): 137 | # Cache file error, clear 138 | cache.close() 139 | cache = shelve.open(self.cache_file, 'n') 140 | else: 141 | if (cache_item['cached'] + 3600 * 24) > int(time.time()): 142 | cache.close() 143 | return cache_item 144 | 145 | # Wait 10 seconds for the TMDb rate limit 146 | if self.request_count >= 40: 147 | logs.info(u"Waiting 10 seconds for the TMDb rate limit...") 148 | time.sleep(10) 149 | self.request_count = 0 150 | 151 | params = { 152 | 'api_key': self.api_key, 153 | 'external_source': 'imdb_id' 154 | } 155 | 156 | url = "https://api.themoviedb.org/3/find/{imdb_id}".format( 157 | imdb_id=imdb_id) 158 | 159 | r = requests.get(url, params=params) 160 | 161 | self.request_count += 1 162 | 163 | media_result = None 164 | 165 | if r.status_code == 200: 166 | item = json.loads(r.text) 167 | 168 | if library_type == 'movie': 169 | if item and item.get('movie_results'): 170 | media_result = item.get('movie_results')[0] 171 | else: 172 | if item and item.get('tv_results'): 173 | media_result = item.get('tv_results')[0] 174 | 175 | if media_result: 176 | media_result['cached'] = int(time.time()) 177 | cache[str(imdb_id)] = media_result 178 | 179 | cache.close() 180 | return media_result 181 | -------------------------------------------------------------------------------- /plexlibrary/traktutils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import datetime 3 | import json 4 | 5 | import requests 6 | import trakt 7 | import logs 8 | 9 | from utils import add_years 10 | 11 | 12 | class Trakt(object): 13 | def __init__(self, username, client_id='', client_secret='', 14 | oauth_token='', oauth=False, config=None): 15 | self.config = config 16 | self.username = username 17 | self.client_id = client_id 18 | self.client_secret = client_secret 19 | self.oauth_token = oauth_token 20 | self.oauth = oauth 21 | if oauth: 22 | if not self.oauth_token: 23 | self.oauth_auth() 24 | else: 25 | trakt.core.pin_auth(username, client_id=client_id, 26 | client_secret=client_secret) 27 | self.trakt = trakt 28 | self.trakt_core = trakt.core.Core() 29 | 30 | def oauth_auth(self): 31 | store = True 32 | self.oauth_token = trakt.core.oauth_auth( 33 | self.username, client_id=self.client_id, 34 | client_secret=self.client_secret, store=store) 35 | # Write to the file 36 | if self.config: 37 | self.config['trakt']['oauth_token'] = self.oauth_token 38 | self.config.save() 39 | logs.info(u"Added new OAuth token to the config file under trakt:") 40 | logs.info(u" oauth_token: '{}'".format(self.oauth_token)) 41 | 42 | def _handle_request(self, method, url, data=None): 43 | """Stolen from trakt.core to support optional OAUTH operations 44 | :todo: Fix trakt 45 | """ 46 | headers = {'Content-Type': 'application/json', 47 | 'trakt-api-version': '2'} 48 | # self.logger.debug('%s: %s', method, url) 49 | headers['trakt-api-key'] = self.client_id 50 | if self.oauth: 51 | headers['Authorization'] = 'Bearer {0}'.format(self.oauth_token) 52 | # self.logger.debug('headers: %s', str(headers)) 53 | # self.logger.debug('method, url :: %s, %s', method, url) 54 | if method == 'get': # GETs need to pass data as params, not body 55 | response = requests.request(method, url, params=data, 56 | headers=headers) 57 | else: 58 | response = requests.request(method, url, data=json.dumps(data), 59 | headers=headers) 60 | # self.logger.debug('RESPONSE [%s] (%s): %s', 61 | # method, url, str(response)) 62 | if response.status_code in self.trakt_core.error_map: 63 | if response.status_code == trakt.core.errors.OAuthException.http_code: 64 | # OAuth token probably expired 65 | logs.warning(u"Trakt OAuth token invalid/expired") 66 | self.oauth_auth() 67 | return self._handle_request(method, url, data) 68 | raise self.trakt_core.error_map[response.status_code]() 69 | elif response.status_code == 204: # HTTP no content 70 | return None 71 | json_data = json.loads(response.content.decode('UTF-8', 'ignore')) 72 | return json_data 73 | 74 | def add_movies(self, url, movie_list=None, movie_ids=None, max_age=0): 75 | if not movie_list: 76 | movie_list = [] 77 | if not movie_ids: 78 | movie_ids = [] 79 | max_date = add_years(max_age * -1) 80 | logs.info(u"Retrieving the trakt list: {}".format(url)) 81 | data = {} 82 | if max_age != 0: 83 | data['extended'] = 'full' 84 | movie_data = self._handle_request('get', url, data=data) 85 | for m in movie_data: 86 | if 'movie' not in m: 87 | m['movie'] = m 88 | # Skip already added movies 89 | if m['movie']['ids']['imdb'] in movie_ids: 90 | continue 91 | if not m['movie']['year']: # TODO: Handle this better? 92 | continue 93 | # Skip old movies 94 | if max_age != 0 \ 95 | and (max_date > datetime.datetime.strptime( 96 | m['movie']['released'], '%Y-%m-%d')): 97 | continue 98 | movie_list.append({ 99 | 'id': m['movie']['ids']['imdb'], 100 | 'tmdb_id': str(m['movie']['ids'].get('tmdb', '')), 101 | 'title': m['movie']['title'], 102 | 'year': m['movie']['year'], 103 | }) 104 | movie_ids.append(m['movie']['ids']['imdb']) 105 | if m['movie']['ids'].get('tmdb'): 106 | movie_ids.append('tmdb' + str(m['movie']['ids']['tmdb'])) 107 | 108 | return movie_list, movie_ids 109 | 110 | def add_shows(self, url, show_list=None, show_ids=None, max_age=0): 111 | if not show_list: 112 | show_list = [] 113 | if not show_ids: 114 | show_ids = [] 115 | curyear = datetime.datetime.now().year 116 | logs.info(u"Retrieving the trakt list: {}".format(url)) 117 | data = {} 118 | if max_age != 0: 119 | data['extended'] = 'full' 120 | show_data = self._handle_request('get', url, data=data) 121 | for m in show_data: 122 | if 'show' not in m: 123 | m['show'] = m 124 | # Skip already added shows 125 | if m['show']['ids']['imdb'] in show_ids: 126 | continue 127 | if not m['show']['year']: 128 | continue 129 | # Skip old shows 130 | if max_age != 0 \ 131 | and (curyear - (max_age - 1)) > int(m['show']['year']): 132 | continue 133 | show_list.append({ 134 | 'id': m['show']['ids']['imdb'], 135 | 'tmdb_id': str(m['show']['ids'].get('tmdb', '')), 136 | 'tvdb_id': str(m['show']['ids'].get('tvdb', '')), 137 | 'title': m['show']['title'], 138 | 'year': m['show']['year'], 139 | }) 140 | show_ids.append(m['show']['ids']['imdb']) 141 | if m['show']['ids'].get('tmdb'): 142 | show_ids.append('tmdb' + str(m['show']['ids']['tmdb'])) 143 | if m['show']['ids'].get('tvdb'): 144 | show_ids.append('tvdb' + str(m['show']['ids']['tvdb'])) 145 | 146 | return show_list, show_ids 147 | 148 | def add_items(self, item_type, url, item_list=None, item_ids=None, 149 | max_age=0): 150 | if item_type == 'movie': 151 | return self.add_movies(url, movie_list=item_list, 152 | movie_ids=item_ids, max_age=max_age) 153 | elif item_type == 'tv': 154 | return self.add_shows(url, show_list=item_list, 155 | show_ids=item_ids, max_age=max_age) 156 | -------------------------------------------------------------------------------- /plexlibrary/tvdb.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import requests 3 | import json 4 | 5 | 6 | class TheTVDB(object): 7 | token = None 8 | 9 | def __init__(self, username, api_key, user_key): 10 | self.username = username 11 | self.api_key = api_key 12 | self.user_key = user_key 13 | 14 | def get_imdb_id(self, tvdb_id): 15 | # TODO Cache 16 | if not self.token: 17 | self._refresh_token() 18 | 19 | url = "https://api.thetvdb.com/series/{tvdb_id}".format( 20 | tvdb_id=tvdb_id) 21 | headers = { 22 | 'Authorization': 'Bearer {token}'.format(token=self.token) 23 | } 24 | r = requests.get(url, headers=headers) 25 | 26 | if r.status_code == 200: 27 | tv_show = r.json() 28 | return tv_show['data']['imdbId'] 29 | else: 30 | return None 31 | 32 | def _refresh_token(self): 33 | data = { 34 | 'apikey': self.api_key, 35 | 'userkey': self.user_key, 36 | 'username': self.username, 37 | } 38 | 39 | url = "https://api.thetvdb.com/login" 40 | r = requests.post(url, json=data) 41 | 42 | if r.status_code == 200: 43 | result = r.json() 44 | self.token = result['token'] 45 | else: 46 | return None 47 | 48 | def get_tvdb_from_imdb(self, imdb_id): 49 | # TODO Cache 50 | if not self.token: 51 | self._refresh_token() 52 | 53 | params = { 54 | 'imdbId': imdb_id 55 | } 56 | 57 | url = "https://api.thetvdb.com/search/series" 58 | headers = { 59 | 'Authorization': 'Bearer {token}'.format(token=self.token) 60 | } 61 | r = requests.get(url, headers=headers, params=params) 62 | 63 | if r.status_code == 200: 64 | item = json.loads(r.text) 65 | 66 | return item.get('data')[0] if item and item.get('data') else None 67 | else: 68 | return None 69 | -------------------------------------------------------------------------------- /plexlibrary/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from datetime import datetime 3 | 4 | import ruamel.yaml 5 | 6 | 7 | class Colors(object): 8 | RED = u'\033[1;31m' 9 | BLUE = u'\033[1;34m' 10 | CYAN = u'\033[1;36m' 11 | GREEN = u'\033[0;32m' 12 | RESET = u'\033[0;0m' 13 | BOLD = u'\033[;1m' 14 | REVERSE = u'\033[;7m' 15 | 16 | 17 | class YAMLBase(object): 18 | def __init__(self, filename): 19 | self.filename = filename 20 | 21 | yaml = ruamel.yaml.YAML() 22 | yaml.preserve_quotes = True 23 | with open(self.filename, 'r') as f: 24 | try: 25 | self.data = yaml.load(f) 26 | except ruamel.yaml.YAMLError as e: 27 | raise e 28 | 29 | def __getitem__(self, k): 30 | return self.data[k] 31 | 32 | def __iter__(self, k): 33 | return self.data.itervalues() 34 | 35 | def __setitem__(self, k, v): 36 | self.data[k] = v 37 | 38 | def get(self, k, default=None): 39 | if k in self.data: 40 | return self.data[k] 41 | else: 42 | return default 43 | 44 | def save(self): 45 | yaml = ruamel.yaml.YAML() 46 | with open(self.filename, 'w') as f: 47 | yaml.dump(self.data, f) 48 | 49 | 50 | def add_years(years, from_date=None): 51 | if from_date is None: 52 | from_date = datetime.now() 53 | try: 54 | return from_date.replace(year=from_date.year + years) 55 | except ValueError: 56 | # Must be 2/29! 57 | return from_date.replace(month=2, day=28, 58 | year=from_date.year + years) 59 | -------------------------------------------------------------------------------- /recipes/examples/movies_1001_movies_you_must_see_before_you_die.yml: -------------------------------------------------------------------------------- 1 | # Supported types: movie, tv 2 | library_type: 'movie' 3 | 4 | # Source list(s) 5 | source_list_urls: 6 | - 'https://api.trakt.tv/movies/trending?limit=50' 7 | 8 | # Source library details 9 | source_libraries: 10 | - name: 'Movies' 11 | folders: 12 | - '/path/to/Movies' 13 | - '/path/to/More Movies' 14 | - name: 'Different Movies' 15 | folders: 16 | - '/path/to/Different Movies' 17 | 18 | # New playlist details 19 | new_playlist: 20 | name: 'Movies - 1001 Movies You Must See Before You Die' 21 | # Limit the age (in years) of items to be considered 22 | # * 0 for no limit 23 | max_age: 0 24 | # Maximum number of items to keep in the library 25 | max_count: 250 26 | # Remove items that no longer exist in the source lists 27 | remove_from_playlist: true 28 | share_to_users: 29 | - username1 30 | - username2 31 | - username3 32 | share_to_all: false 33 | 34 | # New library details 35 | new_library: 36 | name: 'Movies - 1001 Movies You Must See Before You Die' 37 | folder: '/path/to/symlink/supporting/filesystem/Movies - 1001 Must/' 38 | sort: true 39 | sort_title: 40 | format: '{number}. {title}' 41 | visible: true 42 | absolute: true # Skips numbers for missing items 43 | # Limit the age (in years) of items to be considered 44 | # * 0 for no limit 45 | max_age: 0 46 | # Maximum number of items to keep in the library 47 | max_count: 0 48 | # Remove items that no longer exist in the source lists 49 | remove_from_library: true 50 | 51 | # Weighted sorting (requires TMDb API) 52 | weighted_sorting: 53 | enabled: false 54 | better_release_date: false 55 | weights: 56 | # Think of these as percentages, 57 | # but they don't have to add up to 1.0 58 | # * Additive 59 | # * Higher value -> more important 60 | index: 0.0 61 | vote: 0.0 62 | age: 0.0 63 | random: 0.0 64 | # Penalize (<0) or reward (>0) certain (TMDb) genres 65 | # * Final weight is multipled by these values 66 | genre_bias: 67 | 'TV Movie': 0.7 68 | 'Animation': 0.95 69 | 70 | -------------------------------------------------------------------------------- /recipes/examples/movies_imdb_top_250.yml: -------------------------------------------------------------------------------- 1 | # Supported types: movie, tv 2 | library_type: 'movie' 3 | 4 | # Source list(s) 5 | source_list_urls: 6 | - 'http://www.imdb.com/chart/top' 7 | - 'https://api.trakt.tv/users/justin/lists/imdb-top-rated-movies/items/movies' 8 | 9 | # Source library details 10 | source_libraries: 11 | - name: 'Movies' 12 | folders: 13 | - '/path/to/Movies' 14 | - '/path/to/More Movies' 15 | - name: 'Different Movies' 16 | folders: 17 | - '/path/to/Different Movies' 18 | 19 | # New playlist details 20 | new_playlist: 21 | name: 'Movies - IMDB Top 250' 22 | # Limit the age (in years) of items to be considered 23 | # * 0 for no limit 24 | max_age: 0 25 | # Maximum number of items to keep in the library 26 | max_count: 250 27 | # Remove items that no longer exist in the source lists 28 | remove_from_playlist: true 29 | share_to_users: 30 | - username1 31 | - username2 32 | - username3 33 | share_to_all: false 34 | 35 | # New library details 36 | new_library: 37 | name: 'Movies - IMDB Top 250' 38 | folder: '/path/to/symlink/supporting/filesystem/Movies - IMDB Top 250/' 39 | sort: true 40 | sort_title: 41 | format: '{number}. {title}' 42 | visible: true 43 | absolute: true # Skips numbers for missing items 44 | # Limit the age (in years) of items to be considered 45 | # * 0 for no limit 46 | max_age: 0 47 | # Maximum number of items to keep in the library 48 | max_count: 250 49 | # Remove items that no longer exist in the source lists 50 | remove_from_library: true 51 | 52 | # Weighted sorting (requires TMDb API) 53 | weighted_sorting: 54 | enabled: false 55 | better_release_date: false 56 | weights: 57 | # Think of these as percentages, 58 | # but they don't have to add up to 1.0 59 | # * Additive 60 | # * Higher value -> more important 61 | index: 0.0 62 | vote: 0.0 63 | age: 0.0 64 | random: 0.0 65 | # Penalize (<0) or reward (>0) certain (TMDb) genres 66 | # * Final weight is multipled by these values 67 | genre_bias: 68 | 'TV Movie': 0.7 69 | 'Animation': 0.95 70 | 71 | -------------------------------------------------------------------------------- /recipes/examples/movies_recommended.yml: -------------------------------------------------------------------------------- 1 | # Supported types: movie, tv 2 | library_type: 'movie' 3 | trakt_oauth: true # Note: has to be run manually for the first time to generate a token which needs to be added to the config file 4 | 5 | # Source list(s) 6 | source_list_urls: 7 | - 'https://api.trakt.tv/recommendations/movies?limit=100' 8 | 9 | # Source library details 10 | source_libraries: 11 | - name: 'Movies' 12 | folders: 13 | - '/path/to/Movies' 14 | - '/path/to/More Movies' 15 | - name: 'Different Movies' 16 | folders: 17 | - '/path/to/Different Movies' 18 | 19 | # New playlist details 20 | new_playlist: 21 | name: 'Movies - Recommended' 22 | # Limit the age (in years) of items to be considered 23 | # * 0 for no limit 24 | max_age: 0 25 | # Maximum number of items to keep in the library 26 | max_count: 100 27 | # Remove items that no longer exist in the source lists 28 | remove_from_playlist: true 29 | share_to_users: 30 | - username1 31 | - username2 32 | - username3 33 | share_to_all: false 34 | 35 | # New library details 36 | new_library: 37 | name: 'Movies - Recommended' 38 | folder: '/path/to/symlink/supporting/filesystem/Movies - Recommended/' 39 | sort: true 40 | sort_title: 41 | format: '{number}. {title}' 42 | visible: false 43 | absolute: false 44 | max_age: 0 45 | max_count: 100 46 | remove_from_library: true 47 | 48 | # Weighted sorting (requires TMDb API) 49 | weighted_sorting: 50 | enabled: false 51 | better_release_date: false 52 | weights: 53 | index: 0.0 54 | vote: 0.0 55 | age: 0.0 56 | random: 0.0 57 | genre_bias: 58 | 'TV Movie': 0.0 59 | 'Animation': 0.0 60 | 'Family': 0.0 61 | 62 | -------------------------------------------------------------------------------- /recipes/examples/movies_trending.yml: -------------------------------------------------------------------------------- 1 | # Supported types: movie, tv 2 | library_type: 'movie' 3 | 4 | # Source list(s) 5 | # * Experiment with the limits and order of the URLs below 6 | # to get a different balance. 7 | source_list_urls: 8 | - 'https://api.trakt.tv/movies/trending?limit=2' 9 | - 'https://api.trakt.tv/movies/watched/weekly?limit=100' 10 | - 'https://api.trakt.tv/movies/trending?limit=50' 11 | - 'https://api.trakt.tv/movies/watched/monthly?limit=150' 12 | - 'https://api.trakt.tv/movies/watched/yearly?limit=500' 13 | 14 | # Source library details 15 | source_libraries: 16 | - name: 'Movies' 17 | folders: 18 | - '/path/to/Movies' 19 | - '/path/to/More Movies' 20 | - name: 'Different Movies' 21 | folders: 22 | - '/path/to/Different Movies' 23 | 24 | # New playlist details 25 | new_playlist: 26 | name: 'Movies - Trending' 27 | # Limit the age (in years) of items to be considered 28 | # * 0 for no limit 29 | max_age: 3 30 | # Maximum number of items to keep in the library 31 | max_count: 250 32 | # Remove items that no longer exist in the source lists 33 | remove_from_playlist: true 34 | share_to_users: 35 | - username1 36 | - username2 37 | - username3 38 | share_to_all: false 39 | 40 | # New library details 41 | new_library: 42 | name: 'Movies - Trending' 43 | folder: '/path/to/symlink/supporting/filesystem/Movies - Trending/' 44 | sort: true 45 | sort_title: 46 | format: '{number}. {title}' 47 | visible: false 48 | absolute: false # Skips numbers for missing items 49 | # Limit the age (in years) of items to be considered 50 | # * 0 for no limit 51 | max_age: 3 52 | # Maximum number of items to keep in the library 53 | max_count: 250 54 | # Remove items that no longer exist in the source lists 55 | remove_from_library: true 56 | 57 | # Weighted sorting (requires TMDb API) 58 | weighted_sorting: 59 | enabled: true 60 | better_release_date: false 61 | weights: 62 | # Think of these as percentages, 63 | # but they don't have to add up to 1.0 64 | # * Additive 65 | # * Higher value -> more important 66 | index: 0.70 67 | vote: 0.10 68 | age: 0.15 69 | random: 0.05 70 | # Penalize (<0) or reward (>0) certain (TMDb) genres 71 | # * Final weight is multipled by these values 72 | genre_bias: 73 | 'TV Movie': 0.7 74 | 'Animation': 0.95 75 | 76 | -------------------------------------------------------------------------------- /recipes/examples/tv_trending.yml: -------------------------------------------------------------------------------- 1 | # Supported types: movie, tv 2 | library_type: 'tv' 3 | 4 | # Source list(s) 5 | # * Experiment with the limits and order of the URLs below 6 | # to get a different balance. 7 | source_list_urls: 8 | - 'https://api.trakt.tv/shows/trending?limit=5' 9 | - 'https://api.trakt.tv/shows/watched/weekly?limit=20' 10 | - 'https://api.trakt.tv/shows/watched/monthly?limit=50' 11 | - 'https://api.trakt.tv/shows/watched/yearly?limit=100' 12 | 13 | # Source library details 14 | source_libraries: 15 | - name: 'TV Shows' 16 | folders: 17 | - '/path/to/TV Shows' 18 | - '/path/to/More TV Shows' 19 | - name: 'Different Shows' 20 | folders: 21 | - '/path/to/Different Shows' 22 | 23 | # New playlist details 24 | new_playlist: 25 | name: 'TV - Trending' 26 | # Limit the age (in years) of items to be considered 27 | # * 0 for no limit 28 | max_age: 0 29 | # Maximum number of items to keep in the library 30 | max_count: 250 31 | # Remove items that no longer exist in the source lists 32 | remove_from_playlist: false 33 | share_to_users: 34 | - username1 35 | - username2 36 | - username3 37 | share_to_all: false 38 | 39 | # New library details 40 | new_library: 41 | name: 'TV - Trending' 42 | folder: '/path/to/symlink/supporting/filesystem/TV - Trending/' 43 | sort: true 44 | sort_title: 45 | format: '{number}. {title}' 46 | visible: false 47 | absolute: false # Skips numbers for missing items 48 | # Limit the age (in years) of items to be considered 49 | # * 0 for no limit 50 | max_age: 0 51 | # Maximum number of items to keep in the library 52 | max_count: 250 53 | # Remove items that no longer exist in the source lists 54 | remove_from_library: false 55 | 56 | # Weighted sorting (requires TMDb API) 57 | weighted_sorting: 58 | enabled: true 59 | better_release_date: false 60 | weights: 61 | # Think of these as percentages, 62 | # but they don't have to add up to 1.0 63 | # * Additive 64 | # * Higher value -> more important 65 | index: 0.75 66 | vote: 0.10 67 | age: 0.15 68 | random: 0.0 69 | # Penalize (<0) or reward (>0) certain (TMDb) genres 70 | # * Final weight is multipled by these values 71 | genre_bias: 72 | 'Comedy': 0.9 73 | 74 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | # pip install -r requirements.txt 2 | plexapi 3 | requests 4 | trakt 5 | ruamel.yaml 6 | lxml 7 | --------------------------------------------------------------------------------