├── .env.example ├── .gitignore ├── Dockerfile ├── LICENSE ├── README.md ├── bot ├── bot.py ├── config.py ├── core.py └── strings.py ├── docker-compose.yml ├── requirements.txt ├── sql └── feeds.sql └── websites.txt /.env.example: -------------------------------------------------------------------------------- 1 | # Copy to .env and fill entries below: 2 | 3 | NEWS_CHANNEL= # ID of the Telegram channel where the messages will get sent. 4 | DEBUG_CHANNEL= # Channel/Group ID for DEBUG. 5 | GDC_MAINTAINER= # ID of the maintainer to get notified about critical errors. 6 | PG_LINK= # DB Link to be used. 7 | GDC_TOKEN= # Telegram token. 8 | GDC_BUFFER= # Time between start of websites.txt parsing and message sending. 9 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # SageMath parsed files 80 | *.sage.py 81 | 82 | # dotenv 83 | .env 84 | 85 | # virtualenv 86 | .venv 87 | venv/ 88 | ENV/ 89 | 90 | # Spyder project settings 91 | .spyderproject 92 | .spyproject 93 | 94 | # Rope project settings 95 | .ropeproject 96 | 97 | # mkdocs documentation 98 | /site 99 | 100 | # mypy 101 | .mypy_cache/ 102 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.7 2 | LABEL author="artur@barichello.me" 3 | 4 | COPY . /gdc 5 | COPY ./sql/feeds.sql /docker-entrypoint-initdb.d/ 6 | WORKDIR /gdc 7 | RUN pip install -r requirements.txt 8 | 9 | CMD ["python3", "bot/bot.py"] 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Barichello 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # gamedev_channel_bot 2 | 3 | GameDev channel on Telegram, updated hourly with the best gamedev websites content. 4 | 5 | # Channel 6 | Click the image below to preview the channel on your browser. 7 | 8 | 9 | 10 | 11 | 12 | # Bot 13 | This bot can be repurposed to parse any type of RSS feeds you want, just modify the `websites.txt` file and environment variables in `config.py`. 14 | 15 | # Timing 16 | This bot was designed to collect new RSS posts from the last hour and post them together at the minute 0, to prevent the updates from flooding your Telegram and to group them at nice hourly updates.
17 | To accomplish that there are two `python-telegram-bot`'s jobs. The first job is the parse_job, it runs minutes prior (currently 3) to buffer the new entries.
18 | The message_job is scheduled after the parse, it runs after it. This delay is governed by the env var GDC_BUFFER.
19 | Current settings are minute 57 for the parse job and 3 minutes (GDC_BUFFER=180) for the message_job.
20 | 21 | # Contributing 22 | To include new RSS feeds to the channel open a pull request and edit the `bot/websites.txt` file located in the repository. 23 | At the Pull Request description include examples of great material that the site has produced and why it is a great gamedev resource. 24 | -------------------------------------------------------------------------------- /bot/bot.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import datetime 3 | from telegram.ext import Updater, CommandHandler 4 | 5 | import time 6 | import core 7 | import config 8 | 9 | 10 | def main(): 11 | logging.basicConfig(format='%(asctime)s - %(levelname)s | %(message)s', level=logging.ERROR) 12 | logger = logging.getLogger() 13 | logger.setLevel(logging.INFO) 14 | 15 | def error_callback(bot, update, error): 16 | logging.error(error) 17 | 18 | updater = Updater(token=config.GDC_TOKEN) 19 | job = updater.job_queue 20 | dp = updater.dispatcher 21 | 22 | dp.add_error_handler(error_callback) 23 | dp.add_handler(CommandHandler('start', core.start)) 24 | dp.add_handler(CommandHandler('help', core.get_help)) 25 | dp.add_handler(CommandHandler('next', core.print_jobs)) 26 | 27 | while True: 28 | if datetime.datetime.now().minute == 57: 29 | job.run_repeating(core.parse, interval=3600, first=0, name='parse_job') 30 | break 31 | else: 32 | time.sleep(1) 33 | 34 | updater.start_polling() 35 | updater.idle() 36 | 37 | 38 | if __name__ == '__main__': 39 | main() 40 | -------------------------------------------------------------------------------- /bot/config.py: -------------------------------------------------------------------------------- 1 | from os import environ 2 | 3 | 4 | GDC_TOKEN = environ['GDC_TOKEN'] 5 | GDC_MAINTAINER = environ['GDC_MAINTAINER'] 6 | GDC_BUFFER = int(environ['GDC_BUFFER']) 7 | PG_LINK = environ['PG_LINK'] 8 | NEWS_CHANNEL = environ['NEWS_CHANNEL'] 9 | DEBUG_CHANNEL = environ['DEBUG_CHANNEL'] 10 | -------------------------------------------------------------------------------- /bot/core.py: -------------------------------------------------------------------------------- 1 | import feedparser 2 | import dataset 3 | import logging 4 | from datetime import datetime, timedelta 5 | from telegram import ParseMode, InlineKeyboardButton, InlineKeyboardMarkup 6 | 7 | import strings 8 | import config 9 | 10 | 11 | buffer = [] 12 | MAX_UPDATES_PER_HOUR = 5 13 | db = dataset.connect(config.PG_LINK, row_type=dict) 14 | 15 | 16 | def start(bot, update): 17 | update.message.reply_text(strings.GREETING_TEXT) 18 | 19 | 20 | def parse(bot, job): 21 | logging.info('-- Starting buffer') 22 | start_time = datetime.now() 23 | 24 | with open('websites.txt', 'r') as websites: 25 | for line in websites: 26 | page = feedparser.parse(line) 27 | 28 | if line.startswith('#'): # websites.txt supports comments 29 | continue 30 | 31 | if len(buffer) >= MAX_UPDATES_PER_HOUR: 32 | logging.info('Reached max updates per hour') 33 | break 34 | 35 | if page.status >= 400: 36 | msg = 'Could not reach feed:\n' + str(page.bozo_exception) 37 | logging.error(msg) 38 | report_to_maintainer(bot, msg) 39 | continue 40 | 41 | feed_title = page.feed.title 42 | post_title = page.entries[0].title 43 | url = page.entries[0].link 44 | logging.info(f'-- Parsing: {feed_title}') 45 | 46 | # Fixes for inconsistent element naming in some RSS feeds 47 | if 'published' in page.feed: 48 | published = page.entries[0].published 49 | else: 50 | published = page.entries[0].updated 51 | 52 | table = db['feeds'] 53 | if not table.find_one(feed_title=feed_title, post_title=post_title): 54 | info = {'url': url, 'feed_title': feed_title, 'post_title': post_title} 55 | buffer.append(info) 56 | 57 | table.insert({ 58 | 'feed_title': feed_title, 59 | 'post_title': post_title, 60 | 'url': url, 61 | 'published': published 62 | }) 63 | logging.info(f'Buffered {post_title}') 64 | 65 | # Report time taken to buffer 66 | end_time = datetime.now() 67 | total_time = (end_time - start_time).total_seconds() 68 | total_time_str = f'Buffering took {total_time} seconds with {len(buffer)} elements.' 69 | logging.info(total_time_str) 70 | report_to_maintainer(bot, total_time_str) 71 | 72 | # Schedule next job according to env GDC_BUFFER 73 | next_job = config.GDC_BUFFER - total_time 74 | logging.info(f'-- Next job scheduled to run in {next_job} seconds') 75 | job.job_queue.run_once(send_messages_from_buffer, when=next_job, name='message_job') 76 | 77 | 78 | def send_messages_from_buffer(bot, job): 79 | logging.info('-- Sending messages from buffer') 80 | for element in buffer: 81 | send_to_channel(bot, element) 82 | logging.info('-- Empty buffer') 83 | buffer.clear() 84 | 85 | 86 | def send_to_channel(bot, info): 87 | feed_title = info['feed_title'] 88 | if len(feed_title) > 30: # Strip long feed titles 89 | feed_title = feed_title[0:30] + '...' 90 | keyboard = [ 91 | [InlineKeyboardButton(feed_title, url=info['url'])] 92 | ] 93 | reply_markup = InlineKeyboardMarkup(keyboard) 94 | 95 | bot.send_message( 96 | chat_id=config.NEWS_CHANNEL, 97 | text=f' {info["post_title"]}', 98 | parse_mode=ParseMode.HTML, 99 | reply_markup=reply_markup 100 | ) 101 | 102 | 103 | def print_jobs(bot, update, job): 104 | update.message.reply_text(str(job.next_job)) 105 | 106 | 107 | def report_to_maintainer(bot, message): 108 | bot.send_message(chat_id=config.GDC_MAINTAINER, text=message) 109 | 110 | 111 | def get_help(bot, update): 112 | update.message.reply_text(strings.HELP_STRING) 113 | -------------------------------------------------------------------------------- /bot/strings.py: -------------------------------------------------------------------------------- 1 | WORLD = u'\U0001F30E' 2 | GREETING_TEXT = 'Hi! I am the bot that fuels the @gamedev_channel! GitHub: \ 3 | https://github.com/aBARICHELLO/gamedev_channel_bot' 4 | HELP_STRING = (""" 5 | /help - Shows this helpful block of text 6 | """) 7 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | 3 | services: 4 | db: 5 | image: postgres:10-alpine 6 | ports: 7 | - "5432:5432" 8 | environment: 9 | POSTGRES_USER: ${PGUSER} 10 | POSTGRES_PASSWORD: ${PGPASSWORD} 11 | POSTGRES_DB: ${PGDB} 12 | 13 | bot: 14 | build: . 15 | ports: 16 | - "30:30" 17 | env_file: .env 18 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | telegram==0.0.1 2 | python_telegram_bot==8.0 3 | feedparser==5.2.1 4 | dataset==1.1.2 5 | psycopg2-binary -------------------------------------------------------------------------------- /sql/feeds.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE IF NOT EXISTS feeds ( 2 | id SERIAL PRIMARY KEY DEFAULT nextval('id_seq'), 3 | feed_title TEXT NOT NULL, 4 | post_title TEXT NOT NULL, 5 | url TEXT NOT NULL, 6 | published TEXT NOT NULL, 7 | added TIMESTAMP DEFAULT now() 8 | ); 9 | -------------------------------------------------------------------------------- /websites.txt: -------------------------------------------------------------------------------- 1 | # Gaming News Websites 2 | http://www.eurogamer.net/?format=rss&type=article 3 | http://feeds.feedburner.com/GamasutraFeatureArticles/ 4 | http://www.gamesauce.biz/feed/ 5 | http://feeds.feedburner.com/GamesBrief 6 | # Reverse Engineering / Game Hacking / Game Network 7 | http://dsasmblr.com/feed/ 8 | https://guidedhacking.com/forums/game-hacking-tutorials.37/index.rss 9 | https://www.youtube.com/feeds/videos.xml?channel_id=UCqfqH-wq12WOm4QG4KiRisw 10 | https://www.youtube.com/feeds/videos.xml?channel_id=UCxuUEVD4bQqT3twlaeW9usQ 11 | https://www.youtube.com/feeds/videos.xml?channel_id=UCP7QY6L5pvmm0-stL-pNFrw 12 | https://nickcano.com/rss/ 13 | https://www.youtube.com/feeds/videos.xml?channel_id=UCRuevX5KUt960aHuWqxj-bQ 14 | https://www.youtube.com/feeds/videos.xml?channel_id=UCYDnJiF0_RqSjkjvjRbG1tA 15 | https://www.youtube.com/feeds/videos.xml?channel_id=UCHTnEwQKNwm49CQeCVZogMw 16 | # Tech Talks 17 | https://www.youtube.com/feeds/videos.xml?channel_id=UC0JB7TSe49lg56u6qH8y_MQ 18 | # Game Engine Blogs / Engine Tech / Graphics 19 | https://www.youtube.com/feeds/videos.xml?channel_id=UCCuoqzrsHlwv1YyPKLuMDUQ 20 | https://forum.unity.com/forums/9/index.rss 21 | https://godotengine.org/rss.xml 22 | http://the-witness.net/news/feed/ 23 | http://www.adriancourreges.com/atom.xml 24 | # Documentaries / Interviews 25 | https://www.youtube.com/feeds/videos.xml?channel_id=UC0fDG3byEcMtbOqPMymDNbw 26 | https://www.youtube.com/feeds/videos.xml?channel_id=UCSuhUzpdXg9jme6eN6HA_IA 27 | https://www.youtube.com/feeds/videos.xml?playlist_id=PLKeyR9bVVLDgdC1OzYxIsCCfV-TKtO30L 28 | # Gaming/Developer specific blogs 29 | https://www.factorio.com/blog/rss 30 | https://engineering.riotgames.com/news/feed 31 | https://rocketsnail.com/blog?format=RSS 32 | https://aras-p.info/atom.xml 33 | # Game Design 34 | http://www.lostgarden.com/feeds/posts/default?alt=rss 35 | https://itch.io/blog.rss 36 | https://www.youtube.com/feeds/videos.xml?channel_id=UCqJ-Xo29CKyLTjn6z2XwYAw 37 | https://www.youtube.com/feeds/videos.xml?channel_id=UC8P_raHQ4EoWTSH2GMESMQA 38 | --------------------------------------------------------------------------------