├── .gitignore
├── LICENSE
├── README.md
├── config_example.py
├── create_db.sql
├── dbhelper.py
├── sample_custom_post.py
└── sample_hourly.py


/.gitignore:
--------------------------------------------------------------------------------
1 | .git/
2 | __pycache__/
3 | .idea/
4 | config.py
5 | storage.db
6 | static/atom.xml
7 | static/rss.xml
8 | test_feedgen.py


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | The MIT License (MIT)
 2 | 
 3 | Copyright (c) 2016-2019 Evgeny Petrov
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # Create RSS/Atom feed for Telegram channel
 2 | 
 3 | <code> ⚠️ Warning! This code was made a long time ago, lots of API changes happened since then, making some parts of this code useless. For example, bots can already read messages in channels.
 4 | 
 5 | Update 16 May 2019: It seems that Telegram now allows to read public channels in browser (https://t.me/tgbeta/3618) even without Telegram account. So this repo now goes read-only and just for historical purposes. So long and thanks for all the fish!
 6 | </code>
 7 | 
 8 | This code allows you to form an RSS feed when you post messages to channel via bot. It works fine for me, but there can be some bugs.  
 9 | In theory, devs can use this to form RSS/Atom feed for their channels to make content available outside Telegram.
10 | 
11 | ## Prerequisites
12 | * Python 3;
13 | * [pyTelegramBotAPI](https://github.com/eternnoir/pyTelegramBotAPI/) lib to work with Telegram [Bot API](https://core.telegram.org/bots/api);
14 | * [pytz](http://pytz.sourceforge.net/) library to work with timezones;
15 | * [feedgen](https://github.com/lkiesow/python-feedgen) library to create RSS/Atom feeds;
16 | * [CherryPy](http://www.cherrypy.org/) Micro web-server (to use webhooks);
17 | * sqlite3;
18 | * (optional) [nginx](http://nginx.org/) as reverse-proxy
19 | 
20 | ## Installation
21 | To install feedgen you need 1024+ megabytes of RAM, also you need to install some libs first:  
22 | 
23 | ```
24 | sudo apt-get install sqlite3 libxml2 libxml2-dev libxslt1.1 libxslt1.1-dev  
25 | pip3 install feedgen pytz pytelegrambotapi cherrypy  
26 | sqlite3 storage.db < create_db.sql
27 | ```
28 | 
29 | 1. Rename `config_example.py` to `config.py` (**Important**)
30 | 2. Fill in the necessary fields in `config.py` file
31 | 3. Launch bot using webhooks with self-signed certificate (`sample_custom_post.py`) or check sample auto-posting bot (`sample_hourly.py`)
32 | 
33 | Check that your bot is set as admin in your channel!  
34 | If using custom poster, open chat with your bot and write a message to it. You should see that message in both channel and RSS/Atom feed.
35 | 
36 | ## Notes and restrictions
37 | 
38 | * It's still unknown, why some RSS Parsers take too long to find updates in feed (though [QuiteRSS](https://quiterss.org/en/node) for Windows finds updates instantly)
39 | * In this version of code, you can only remove one entry from feed at a time. No way to also remove message from channel, only manually with `/remlast` command (Restriction of Bot API).
40 | * Only text messages are supported.
41 | * `parse_mode` argument is set to "HTML" by default (to be compatible with real RSS readers). You can disable it in code or change to "Markdown".
42 | 
43 | ## Any questions?
44 | The code provided here should be self-explanatory, but in case you get stuck somewhere, feel free to e-mail me: groosha @ protonmail.com (remove extra spaces)
45 | 


--------------------------------------------------------------------------------
/config_example.py:
--------------------------------------------------------------------------------
 1 | # -*- coding: utf-8 -*-
 2 | 
 3 | channel_id = "@yourchannel" # @username for public channels / chat_id for private channels
 4 | token = "token"  # Your bot's token from @BotFather
 5 | db_name = "storage.db"  # Name of SQL Database to use
 6 | number_of_entries = 15  # How many entries will be included in RSS
 7 | admin_ids = [111111, 2222222]  # Allow only people with these IDs to post messages to channel via bot.
 8 | webhook_url = "https://example.com/mywebhook/" # URL to set webhook
 9 | 
10 | # --- RSS Config values ---
11 | author_name = "Mr Jack"  # Your name
12 | author_email = "jack@example.com"  # Your e-mail
13 | timezone = 'GMT' # Find yours from here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
14 | bot_link = "https://telegram.me/yourchannel"  # Using your bot URL as ID and Link
15 | feed_link = "https://example.com/rss/atom.xml"  # Self link to RSS/Atom
16 | feed_description = "Your Description"
17 | feed_title = "Your title"
18 | feed_language = "en"  # Optional feed language value


--------------------------------------------------------------------------------
/create_db.sql:
--------------------------------------------------------------------------------
1 | DROP TABLE IF EXISTS texts;
2 | 
3 | CREATE TABLE texts(
4 |     id INTEGER PRIMARY KEY AUTOINCREMENT,
5 |     post_text TEXT,
6 | 	post_date TEXT,
7 | 	post_id TEXT
8 | );


--------------------------------------------------------------------------------
/dbhelper.py:
--------------------------------------------------------------------------------
 1 | # -*- coding: utf-8 -*-
 2 | 
 3 | import sqlite3
 4 | from config import db_name, number_of_entries
 5 | from time import time
 6 | 
 7 | def insert(text, date, pid):
 8 |     with sqlite3.connect(db_name) as connection:
 9 |         connection.execute("INSERT INTO texts(post_text, post_date, post_id) values (?, ?, ?)", (text, date, pid,))
10 |         
11 |  
12 | def get_latest_entries():
13 |     with sqlite3.connect(db_name) as connection:
14 |         cursor = connection.cursor()
15 |         entries = [dict(ptext=row[0], pdate=row[1], pid=row[2]) for row in cursor.execute("SELECT post_text, post_date, post_id FROM texts ORDER BY id DESC LIMIT ?", (number_of_entries,)).fetchall()]
16 |         return entries
17 |         
18 | def remove_last_entry():
19 |     with sqlite3.connect(db_name) as connection:
20 |         connection.execute("DELETE from texts WHERE id in (SELECT id from texts ORDER BY id DESC LIMIT 1);")    
21 | 
22 | 
23 | def remove_old_entries():
24 |     """
25 |     Removes the oldest entries except the number set in config
26 |     """
27 |     with sqlite3.connect(db_name) as connection:
28 |         connection.execute("DELETE FROM texts WHERE id IN (SELECT id FROM texts ORDER BY id DESC LIMIT -1 OFFSET ? )",(number_of_entries,))
29 | 
30 | 
31 | 


--------------------------------------------------------------------------------
/sample_custom_post.py:
--------------------------------------------------------------------------------
  1 | #!/usr/bin/python3
  2 | # -*- coding: utf-8 -*-
  3 | 
  4 | """
  5 | This is an example of using Feedgen and pyTelegramBotAPI to send messages to channel
  6 | as well as creating an RSS Feed.
  7 | Note that to add new items to feed, you need to post via bot (not directly to channel).
  8 | 
  9 | This example uses Webhooks method.
 10 | """
 11 | 
 12 | import cherrypy
 13 | import pytz
 14 | from datetime import datetime
 15 | from feedgen.feed import FeedGenerator
 16 | from email.utils import formatdate
 17 | import random
 18 | import telebot
 19 | import config
 20 | import dbhelper
 21 | from cherrypy.lib.static import serve_file
 22 | 
 23 | bot = telebot.TeleBot(config.token)
 24 | 
 25 | 
 26 | WEBHOOK_HOST = '<ip/host where the bot is running>'
 27 | WEBHOOK_PORT = 8443  # 443, 80, 88 or 8443 (port need to be 'open')
 28 | WEBHOOK_LISTEN = '0.0.0.0'  # In some VPS you may need to put here the IP addr
 29 | 
 30 | WEBHOOK_SSL_CERT = './webhook_cert.pem'  # Path to the ssl certificate
 31 | WEBHOOK_SSL_PRIV = './webhook_pkey.pem'  # Path to the ssl private key
 32 | 
 33 | """
 34 | Quick'n'dirty SSL certificate generation:
 35 | 
 36 | openssl genrsa -out webhook_pkey.pem 2048
 37 | openssl req -new -x509 -days 3650 -key webhook_pkey.pem -out webhook_cert.pem
 38 | 
 39 | When asked for "Common Name (e.g. server FQDN or YOUR name)" you should reply
 40 | with the same value in you put in WEBHOOK_HOST
 41 | """
 42 | 
 43 | WEBHOOK_URL_BASE = "https://%s:%s" % (WEBHOOK_HOST, WEBHOOK_PORT)
 44 | WEBHOOK_URL_PATH = "/%s/" % (API_TOKEN)
 45 | 
 46 | 
 47 | @bot.message_handler(commands=["remlast"])
 48 | def remove_last_entry(message):
 49 |     if message.chat.id in config.admin_ids:
 50 |         dbhelper.remove_last_entry()
 51 |         generate_feed()
 52 | 
 53 |         
 54 | # Handle only text messages
 55 | @bot.message_handler(func=lambda message: message.chat.id in config.admin_ids, content_types=["text"])
 56 | def my_text(message):
 57 |     # We don't need this crap in channel
 58 |     if message.text == "/start":
 59 |         return
 60 |     tz = pytz.timezone(config.timezone)
 61 |     msg_to_channel = bot.send_message(config.channel_id, message.text, parse_mode="HTML")
 62 |     dbhelper.insert(message.text.replace("\n","<br />"), formatdate(datetime.timestamp(datetime.now(tz)), localtime=True), msg_to_channel.message_id)
 63 |     generate_feed()
 64 | 
 65 |     
 66 | class WebhookServer(object):
 67 |     @cherrypy.expose
 68 |     def index(self):
 69 |         length = int(cherrypy.request.headers['content-length'])
 70 |         json_string = cherrypy.request.body.read(length)
 71 |         json_string = json_string.decode("utf-8")
 72 |         update = telebot.types.Update.de_json(json_string)
 73 |         if update.message:
 74 |             bot.process_new_messages([update.message])
 75 |         if update.inline_query:
 76 |             bot.process_new_inline_query([update.inline_query])
 77 |             
 78 |     @cherrypy.expose
 79 |     def rss(self):
 80 |         # Return static file with ATOM feed
 81 |         return serve_file("/bots/telegram-feedgen-bot/static/atom.xml")
 82 |     
 83 |     
 84 | def generate_feed():
 85 |     tz = pytz.timezone(config.timezone)
 86 |     # Get latest X entries from database
 87 |     entries = dbhelper.get_latest_entries()
 88 | 
 89 |     fg = FeedGenerator()
 90 |     # Feed id
 91 |     fg.id(config.bot_link)
 92 |     # Creator info (for Atom)
 93 |     fg.author(name=config.author_name, email=config.author_email, replace=True )
 94 |     # Self link to the feed
 95 |     fg.link(href=config.feed_link, rel='self')
 96 |     # Set description of your feed
 97 |     fg.description(config.feed_description)
 98 |     # Last time feed updated (use system time with timezone)
 99 |     fg.lastBuildDate(formatdate(datetime.timestamp(datetime.now(tz)), localtime=True))
100 |     fg.title(config.feed_title)
101 |     # Set time-to-live (I really don't know why set this)
102 |     fg.ttl(5)
103 |     # Does this parameter mean anything?
104 |     fg.language(config.feed_language)
105 |     
106 |     for entry in entries:
107 |         item = fg.add_entry()
108 |         # Use message id to form valid URL (new feature in Telegram since Feb 2016)
109 |         item.id("{!s}".format(entry["pid"]))
110 |         item.link(href="{!s}/{!s}".format(config.bot_link, entry["pid"]), rel="alternate")
111 |         # Set title and content from message text
112 |         item.title(entry["ptext"])
113 |         item.content(entry["ptext"])
114 |         # Set publish/update datetime
115 |         item.pubdate(entry["pdate"])
116 |         item.updated(entry["pdate"])
117 | 
118 |     # Write RSS/Atom feed to file
119 |     # It's preferred to have only one type at a time (or just create two functions)
120 |     fg.atom_file('static/atom.xml')
121 |     # fg.rss_file('static/rss.xml')
122 |     
123 | if __name__ == '__main__':
124 | 
125 |     # Remove webhook, it fails sometimes the set if there is a previous webhook
126 |     bot.remove_webhook()
127 | 
128 |     # Set webhook
129 |     bot.set_webhook(url=WEBHOOK_URL_BASE+WEBHOOK_URL_PATH,
130 |                 certificate=open(WEBHOOK_SSL_CERT, 'r'))
131 |                 
132 |     print("Webhook set")            
133 | 
134 |     # Start cherrypy server
135 |     cherrypy.config.update({
136 |         'server.socket_host': WEBHOOK_LISTEN,
137 |         'server.socket_port': WEBHOOK_PORT,
138 |         'server.ssl_module': 'builtin',
139 |         'server.ssl_certificate': WEBHOOK_SSL_CERT,
140 |         'server.ssl_private_key': WEBHOOK_SSL_PRIV,
141 |         'engine.autoreload.on': False
142 |     })
143 | 
144 |     cherrypy.quickstart(WebhookServer(), '/', {'/': {}})
145 | 


--------------------------------------------------------------------------------
/sample_hourly.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/python3
 2 | # -*- coding: utf-8 -*-
 3 | 
 4 | """
 5 | This is an example of using Feedgen and pyTelegramBotAPI to send messages to channel
 6 | as well as creating an RSS Feed.
 7 | 
 8 | Automatically posts current time every hour to your channel and updates RSS.
 9 | You can set this script to a cron: 0 * * * * cd /path/to/your/bot/ && python3 sample_hourly.py
10 | """
11 | 
12 | from time import gmtime
13 | import telebot
14 | import config
15 | import pytz
16 | from datetime import datetime
17 | from feedgen.feed import FeedGenerator
18 | from email.utils import formatdate
19 | import dbhelper
20 | 
21 | bot = telebot.TeleBot(config.token)
22 |     
23 |     
24 | def generate_feed():
25 |     tz = pytz.timezone(config.timezone)
26 |     # Get latest X entries from database
27 |     entries = dbhelper.get_latest_entries()
28 | 
29 |     fg = FeedGenerator()
30 |     # Feed id
31 |     fg.id(config.bot_link)
32 |     # Creator info (for Atom)
33 |     fg.author(name=config.author_name, email=config.author_email, replace=True )
34 |     # Self link to the feed
35 |     fg.link(href=config.feed_link, rel='self')
36 |     # Set description of your feed
37 |     fg.description(config.feed_description)
38 |     # Last time feed updated (use system time with timezone)
39 |     fg.lastBuildDate(formatdate(datetime.timestamp(datetime.now(tz)), localtime=True))
40 |     fg.title(config.feed_title)
41 |     # Set time-to-live (I really don't know why set this)
42 |     fg.ttl(5)
43 |     # Does this parameter mean anything?
44 |     fg.language(config.feed_language)
45 |     
46 |     for entry in entries:
47 |         item = fg.add_entry()
48 |         # Use message id to form valid URL (new feature in Telegram since Feb 2016)
49 |         item.id("{!s}".format(entry["pid"]))
50 |         item.link(href="{!s}/{!s}".format(config.bot_link, entry["pid"]), rel="alternate")
51 |         # Set title and content from message text
52 |         item.title(entry["ptext"])
53 |         item.content(entry["ptext"])
54 |         # Set publish/update datetime
55 |         item.pubdate(entry["pdate"])
56 |         item.updated(entry["pdate"])
57 | 
58 |     # Write RSS/Atom feed to file
59 |     # It's preferred to have only one type at a time (or just create two functions)
60 |     fg.atom_file('static/atom.xml')
61 |     # fg.rss_file('static/rss.xml')
62 | 
63 | if __name__ == '__main__':
64 |     message = "It\'s <b>{!s}</b> o\'clock! (GMT)".format(gmtime().tm_hour)
65 |     msg = bot.send_message(config.channel_id, message, disable_notification=True, parse_mode="HTML")
66 |     tz = pytz.timezone(config.timezone)
67 |     dbhelper.insert(message, formatdate(datetime.timestamp(datetime.now(tz)), localtime=True), msg.message_id)
68 |     generate_feed()
69 |     


--------------------------------------------------------------------------------