├── .gitignore ├── Dockerfile ├── README.md ├── entrypoint.sh ├── mreg.py ├── mreg.sh ├── requirements.txt ├── server.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | venv/ 2 | mreg.egg-info/ 3 | mreg.pyc 4 | autodl.cfg 5 | autodl.cfg.old 6 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.7 2 | 3 | COPY . /app 4 | 5 | WORKDIR /app 6 | RUN pip3 install -r requirements.txt 7 | RUN pip3 install . 8 | 9 | ENTRYPOINT ["/app/entrypoint.sh"] 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mreg (match releases expression generator) 2 | 3 | ## How to use 4 | 5 | ``` 6 | $ git clone https://github.com/Igglybuff/mreg.git 7 | $ cd mreg 8 | ``` 9 | 10 | ### Bash script: 11 | 12 | The bash script will dump the expression as a single string to STDOUT. 13 | 14 | ``` 15 | $ ./mreg.sh 16 | Bohemian?Rhapsody*,The?House?with?a?Clock?in?its?Walls*,Venom*,Bad?Times?at?the?El?Royale*,Fantastic?Beasts:?The?Crimes?of?Grindelwald*,Assassination?Nation*,The?Grinch*,Halloween*,Spider-Man:?Into?the?Spider-Verse*,First?Man*,Creed?II*,Hunter?Killer*,A?Star?Is?Born*,Life?Itself*,Ralph?Breaks?the?Internet*,Fahrenheit*,The?Predator*,Instant?Family*,The?Nutcracker?and?the?Four?Realms*,Johnny?English*,Strikes?Again*,Overlord*,McQueen*,A?Simple?Favor*,Peppermint* 17 | ``` 18 | 19 | ### Python Flask app: 20 | 21 | The flask app will run an HTTP server exposing the expression as text for use with `curl`. It doesn't do any caching, so every time you refresh it will hit dvdsreleasedates.com again. 22 | 23 | ``` 24 | $ pip3 install -r requirements.txt 25 | $ python3 server.py 26 | * Serving Flask app "server" (lazy loading) 27 | * Environment: production 28 | WARNING: Do not use the development server in a production environment. 29 | Use a production WSGI server instead. 30 | * Debug mode: off 31 | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) 32 | ``` 33 | 34 | The application will be exposed on `0.0.0.0` at port `5000`. Access it by navigating to http://127.0.0.1:5000/ in your web browser. 35 | 36 | ### Python CLI tool: 37 | 38 | ``` 39 | $ pip3 install -r requirements.txt 40 | $ pip install . 41 | $ mreg --help 42 | Usage: mreg [OPTIONS] 43 | 44 | Options: 45 | -c, --autodlcfg TEXT The path to your autodl.cfg file. [default: 46 | ~/.autodl/autodl.cfg] 47 | -f, --filter TEXT The name of your autodl-irssi filter for movies. 48 | [required] 49 | --help Show this message and exit. 50 | $ mreg --autodlcfg "/home/rtorrent/.autodl/autodl.cfg" --filter "filter 1080p/720p Movies IPT" 51 | Your filter was updated successfully! 52 | ``` 53 | 54 | You can use environment variables instead of command line options instead if you prefer: 55 | 56 | ``` 57 | export MREG_AUTODLCFG_PATH=/home/rtorrent/.autodl/autodl.cfg 58 | export MREG_FILTER_NAME="filter 1080p/720p Movies IPT" 59 | ``` 60 | 61 | ### Docker 62 | 63 | `mreg` can run in a docker container and trigger at your desired frequency. The frequency interval is configured with the `INTERVAL` environment variable, e.g. `-e INTERVAL=21600` would update your `autodl.cfg` every six hours. The default is 7200 seconds (2 hours). 64 | 65 | You must map your `autodl.cfg` file into the container with `-v /path/to/your/autodl.cfg:/root/.autodl/autodl.cfg`. If you want to put `autodl.cfg` somewhere else inside the container, make sure you set the `MREG_AUTODLCFG_PATH` environment variable too with `-e MREG_AUTODLCFG_PATH=/path/to/container/autodl.cfg`. 66 | 67 | #### `docker run` 68 | 69 | `$ docker run -it --rm --name mreg -v /path/to/your/autodl.cfg:/root/.autodl/autodl.cfg -e MREG_FILTER_NAME="filter 1080p/720p Movies IPT" -e MREG_AUTODLCFG_PATH=/root/.autodl/autodl.cfg wigglytuff/mreg:latest` 70 | 71 | Replace `-it --rm` with `-d` when you are confident it is working. 72 | 73 | #### `docker-compose` 74 | 75 | ``` 76 | version: '3' 77 | 78 | services: 79 | mreg: 80 | container_name: mreg 81 | image: "wigglytuff/mreg:latest" 82 | volumes: 83 | - /path/to/your/autodl.cfg:/root/.autodl/autodl.cfg 84 | environment: 85 | - MREG_FILTER_NAME="filter 1080p/720p Movies IPT" 86 | restart: always 87 | ``` -------------------------------------------------------------------------------- /entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -euo pipefail 4 | 5 | INTERVAL="${INTERVAL:-7200}" 6 | 7 | if [ -z "${MREG_FILTER_NAME}" ]; then 8 | echo "ERROR - you have not specified a filter name!" 9 | exit 1 10 | fi 11 | 12 | while true; do 13 | echo "Running mreg..." 14 | mreg || echo "Something went wrong!" 15 | sleep "${INTERVAL}" 16 | done -------------------------------------------------------------------------------- /mreg.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from flask import Flask, Response 4 | import requests 5 | import configparser 6 | from bs4 import BeautifulSoup 7 | import click 8 | from pathlib import Path 9 | import re 10 | import sys 11 | 12 | 13 | def scrape_releases(): 14 | headers = { 15 | 'Connection': 'keep-alive', 16 | 'Cache-Control': 'max-age=0', 17 | 'Upgrade-Insecure-Requests': '1', 18 | 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36', 19 | 'DNT': '1', 20 | 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 21 | 'Accept-Encoding': 'gzip, deflate, br', 22 | 'Accept-Language': 'en-GB,en;q=0.9,en-US;q=0.8,pt;q=0.7', 23 | } 24 | 25 | page = requests.get('https://www.dvdsreleasedates.com/', headers=headers) 26 | soup = BeautifulSoup(page.text, 'html.parser') 27 | requested_table = soup.find("div", {"id": 'requested'}) 28 | requested_hrefs = requested_table.find_all('a') 29 | movie_names = [ movie.contents[0].replace(' ', '?') + "*," for movie in requested_hrefs ] 30 | 31 | return movie_names 32 | 33 | 34 | def get_abs_path(file): 35 | file = Path(file) 36 | return file.resolve(strict=True) 37 | 38 | 39 | def update_autodl_cfg(expression, autodlcfg, filter_name): 40 | autodlcfg_abs = get_abs_path(autodlcfg) 41 | new_expression = ''.join(expression) 42 | config = configparser.ConfigParser() 43 | config.read(get_abs_path(autodlcfg_abs)) 44 | config[filter_name]['match-releases'] = new_expression 45 | with open(get_abs_path(autodlcfg_abs), 'w') as configfile: 46 | config.write(configfile) 47 | 48 | 49 | def fix_filter(filter_name): 50 | if filter_name.startswith("filter"): 51 | return filter_name 52 | else: 53 | return 'filter ' + filter_name 54 | 55 | 56 | def check_config(autodlcfg, filter_name): 57 | config = configparser.ConfigParser() 58 | config.read(get_abs_path(autodlcfg)) 59 | if not filter_name in config: 60 | raise configparser.Error('Unable to find filter "' + filter_name + '" in ' + autodlcfg) 61 | 62 | 63 | def check_regex_validity(expression): 64 | new_expression = ''.join(expression) 65 | try: 66 | re.compile(new_expression) 67 | except re.error: 68 | click.echo('ERROR - mreg generated an invalid regex.') 69 | sys.exit(1) 70 | 71 | 72 | @click.command() 73 | @click.option('-c', '--autodlcfg', 'autodlcfg', envvar='MREG_AUTODLCFG_PATH', default='~/.autodl/autodl.cfg', show_default=True, help='The path to your autodl.cfg file.') 74 | @click.option('-f', '--filter', 'filter', envvar='MREG_FILTER_NAME', required=True, help='The name of your autodl-irssi filter for movies.') 75 | def mreg(autodlcfg, filter): 76 | expression = scrape_releases() 77 | check_regex_validity(expression) 78 | filter = fix_filter(filter) 79 | check_config(autodlcfg, filter) 80 | update_autodl_cfg(expression, autodlcfg, filter) 81 | click.echo('Your filter was updated successfully!') 82 | 83 | # TODO: add "live" mode which triggers every x minutes 84 | -------------------------------------------------------------------------------- /mreg.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | URL="https://www.dvdsreleasedates.com/" 4 | 5 | MATCH_EXPRESSION=$(lynx -dump "${URL}" | \ 6 | grep -i "Most Requested DVD Release Dates" -A 15 | \ 7 | grep -i -v "Most Requested DVD Release Dates" | \ 8 | tr '.' '\n' | \ 9 | sed -e 's/[0-9]*//g' -e 's/\[//g' \ 10 | -e 's/\]//g' -e '/^[[:space:]]*$/d' \ 11 | -e 's/\///g' -e 's/^[[:space:]]*//' \ 12 | -e 's/[[:space:]]*$//' -e 's/$/\*/' | \ 13 | tr '\n' ',' | \ 14 | tr ' ' '?') 15 | 16 | echo ${MATCH_EXPRESSION::-1} -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests 2 | flask 3 | bs4 4 | configparser 5 | click 6 | pathlib -------------------------------------------------------------------------------- /server.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from flask import Flask, Response 4 | import requests 5 | from bs4 import BeautifulSoup 6 | 7 | app = Flask(__name__) 8 | 9 | 10 | @app.route("/") 11 | def index(): 12 | resp = scrape_releases() 13 | return Response(resp, mimetype='text/plain') 14 | 15 | 16 | def scrape_releases(): 17 | headers = { 18 | 'Connection': 'keep-alive', 19 | 'Cache-Control': 'max-age=0', 20 | 'Upgrade-Insecure-Requests': '1', 21 | 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36', 22 | 'DNT': '1', 23 | 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 24 | 'Accept-Encoding': 'gzip, deflate, br', 25 | 'Accept-Language': 'en-GB,en;q=0.9,en-US;q=0.8,pt;q=0.7', 26 | } 27 | 28 | page = requests.get('https://www.dvdsreleasedates.com/', headers=headers) 29 | soup = BeautifulSoup(page.text, 'html.parser') 30 | requested_table = soup.find("div", {"id": 'requested'}) 31 | requested_hrefs = requested_table.find_all('a') 32 | movie_names = [ movie.contents[0].replace(' ', '?') + "*," for movie in requested_hrefs ] 33 | 34 | return movie_names 35 | 36 | 37 | if __name__ == "__main__": 38 | app.run(host="0.0.0.0", debug=False) 39 | 40 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='mreg', 5 | version='0.1', 6 | py_modules=['mreg'], 7 | install_requires=[ 8 | 'Click', 9 | 'requests', 10 | 'flask', 11 | 'bs4', 12 | 'configparser', 13 | 'pathlib', 14 | ], 15 | entry_points=''' 16 | [console_scripts] 17 | mreg=mreg:mreg 18 | ''', 19 | ) --------------------------------------------------------------------------------