├── exports └── .gitkeep ├── pages └── .gitkeep ├── templates ├── banner.png ├── resume.pdf ├── Exo2-Regular.ttf ├── iosevka-fixed-extended.ttf ├── collection.html └── homepage.html ├── run.sh ├── requirements.txt ├── LICENSE ├── README.md ├── .gitignore ├── cms └── exporter.py └── main.py /exports/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /pages/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /templates/banner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Groverkss/Notion-CMS-Blog/HEAD/templates/banner.png -------------------------------------------------------------------------------- /templates/resume.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Groverkss/Notion-CMS-Blog/HEAD/templates/resume.pdf -------------------------------------------------------------------------------- /templates/Exo2-Regular.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Groverkss/Notion-CMS-Blog/HEAD/templates/Exo2-Regular.ttf -------------------------------------------------------------------------------- /templates/iosevka-fixed-extended.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Groverkss/Notion-CMS-Blog/HEAD/templates/iosevka-fixed-extended.ttf -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Export environment variables`` 4 | [ -e .secrets ] && . ./.secrets 5 | 6 | # Run script 7 | python3 main.py 8 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | beautifulsoup4==4.9.3 2 | bs4==0.0.1 3 | cached-property==1.5.2 4 | certifi==2020.11.8 5 | chardet==3.0.4 6 | commonmark==0.9.1 7 | dictdiffer==0.8.1 8 | idna==2.10 9 | Jinja2==2.11.2 10 | MarkupSafe==1.1.1 11 | notion==0.0.27 12 | python-slugify==4.0.1 13 | pytz==2020.4 14 | requests==2.25.0 15 | soupsieve==2.0.1 16 | text-unidecode==1.3 17 | tqdm==4.54.0 18 | tzlocal==2.1 19 | urllib3==1.26.2 20 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Kunwar Shaanjeet Singh Grover 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Notion-CMS-Blog 2 | A blog with Notion as content management system. Imports content from notion and publishes it. 3 | 4 | ## Installation 5 | 6 | > :warning: Versions of Python older than 3.6 may not work 7 | 8 | 1. Create a virtual environment using `venv`. For systems using `apt`: 9 | 10 | ``` 11 | sudo apt install python3-venv 12 | python3 -m venv .env 13 | ``` 14 | 15 | 2. Source the virtual environment and install 16 | dependencies from `requirements.txt` 17 | 18 | ``` 19 | source .env/bin/activate 20 | pip3 install -r requirements.txt 21 | ``` 22 | 23 | 3. You need to create a `.secrets` file with the following contents: 24 | 25 | ``` 26 | #!/bin/bash 27 | 28 | export TOKEN_V2="" 29 | export SPACE_ID="" 30 | ``` 31 | 32 | - Token can be obtained from cookies as `token_v2` after logging in Notion. 33 | 34 | - To obtain workspace id, login to Notion, use developer tools in browser. 35 | Go to Networks tab and press `Ctrl+R` to reload the page. 36 | Look for `getSpaces` object. Click on the object and go to 37 | **Preview** tab. Expand and look for id in `spaces` value 38 | corresponding to the required workspace. 39 | 40 | Give execute permissions to this file: 41 | 42 | ``` 43 | chmod +700 .secrets 44 | ``` 45 | 46 | 4. Remove lines 5-7 in `.gitignore` 47 | 48 | ## Runnning 49 | 50 | After creating the `.secrets` file, run the program as follows: 51 | 52 | ``` 53 | bash run.sh 54 | ``` 55 | 56 | ## Customizing 57 | 58 | To customize the homepage or collection page, edit the corresponding `html` 59 | templates in `templates` directory. 60 | 61 | ## Features 62 | 63 | - Pages/Collections are only downloaded if they have been changed. Deleted 64 | objects are removed. 65 | - Currently it only supports pages and collections of pages. I do not know how to 66 | export pages inside pages without sending an email to the members. 67 | 68 | ## TODO 69 | 70 | - Remove redundant zips and check availability straight from HTML pages 71 | 72 | ## Contributing 73 | 74 | Feel free to open issues/pull requests. Format code with `black` formatter before 75 | submitting please. 76 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Secrets 2 | .secrets 3 | 4 | # Exports 5 | /exports/* 6 | /pages/* 7 | index.html 8 | !*.gitkeep 9 | 10 | # Byte-compiled / optimized / DLL files 11 | __pycache__/ 12 | *.py[cod] 13 | *$py.class 14 | 15 | # C extensions 16 | *.so 17 | 18 | # Distribution / packaging 19 | .Python 20 | build/ 21 | develop-eggs/ 22 | dist/ 23 | downloads/ 24 | eggs/ 25 | .eggs/ 26 | lib/ 27 | lib64/ 28 | parts/ 29 | sdist/ 30 | var/ 31 | wheels/ 32 | pip-wheel-metadata/ 33 | share/python-wheels/ 34 | *.egg-info/ 35 | .installed.cfg 36 | *.egg 37 | MANIFEST 38 | 39 | # PyInstaller 40 | # Usually these files are written by a python script from a template 41 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 42 | *.manifest 43 | *.spec 44 | 45 | # Installer logs 46 | pip-log.txt 47 | pip-delete-this-directory.txt 48 | 49 | # Unit test / coverage reports 50 | htmlcov/ 51 | .tox/ 52 | .nox/ 53 | .coverage 54 | .coverage.* 55 | .cache 56 | nosetests.xml 57 | coverage.xml 58 | *.cover 59 | *.py,cover 60 | .hypothesis/ 61 | .pytest_cache/ 62 | 63 | # Translations 64 | *.mo 65 | *.pot 66 | 67 | # Django stuff: 68 | *.log 69 | local_settings.py 70 | db.sqlite3 71 | db.sqlite3-journal 72 | 73 | # Flask stuff: 74 | instance/ 75 | .webassets-cache 76 | 77 | # Scrapy stuff: 78 | .scrapy 79 | 80 | # Sphinx documentation 81 | docs/_build/ 82 | 83 | # PyBuilder 84 | target/ 85 | 86 | # Jupyter Notebook 87 | .ipynb_checkpoints 88 | 89 | # IPython 90 | profile_default/ 91 | ipython_config.py 92 | 93 | # pyenv 94 | .python-version 95 | 96 | # pipenv 97 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 98 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 99 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 100 | # install all needed dependencies. 101 | #Pipfile.lock 102 | 103 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 104 | __pypackages__/ 105 | 106 | # Celery stuff 107 | celerybeat-schedule 108 | celerybeat.pid 109 | 110 | # SageMath parsed files 111 | *.sage.py 112 | 113 | # Environments 114 | .env 115 | .venv 116 | env/ 117 | venv/ 118 | ENV/ 119 | env.bak/ 120 | venv.bak/ 121 | 122 | # Spyder project settings 123 | .spyderproject 124 | .spyproject 125 | 126 | # Rope project settings 127 | .ropeproject 128 | 129 | # mkdocs documentation 130 | /site 131 | 132 | # mypy 133 | .mypy_cache/ 134 | .dmypy.json 135 | dmypy.json 136 | 137 | # Pyre type checker 138 | .pyre/ 139 | -------------------------------------------------------------------------------- /cms/exporter.py: -------------------------------------------------------------------------------- 1 | import requests 2 | from tqdm import tqdm 3 | from time import sleep 4 | 5 | BLOCK_SIZE = 1024 6 | STATUS_WAIT_TIME = 3 7 | N_TRAILS = 5 8 | FAIL_SLEEP_TIME = 1 9 | 10 | 11 | class Exporter: 12 | def __init__(self, client): 13 | self.client = client 14 | 15 | def download_file(self, url, export_file): 16 | with requests.get(url, stream=True, allow_redirects=True) as response: 17 | total_size = int(response.headers.get("content-length", 0)) 18 | tqdm_bar = tqdm(total=total_size, unit="iB", unit_scale=True) 19 | with export_file.open("wb") as export_file_handle: 20 | for data in response.iter_content(BLOCK_SIZE): 21 | tqdm_bar.update(len(data)) 22 | export_file_handle.write(data) 23 | tqdm_bar.close() 24 | 25 | def get_task_status(self, task_id): 26 | task_statuses = self.client.post("getTasks", {"taskIds": [task_id]}).json()[ 27 | "results" 28 | ] 29 | 30 | return list( 31 | filter(lambda task_status: task_status["id"] == task_id, task_statuses) 32 | )[0] 33 | 34 | def launch_page_export(self, block_id): 35 | data = { 36 | "task": { 37 | "eventName": "exportBlock", 38 | "request": { 39 | "blockId": block_id, 40 | "recursive": False, 41 | "exportOptions": { 42 | "exportType": "html", 43 | "timeZone": "Asia/Calcutta", 44 | "locale": "en", 45 | }, 46 | }, 47 | } 48 | } 49 | response = self.client.post("enqueueTask", data) 50 | return response.json()["taskId"] 51 | 52 | def clean_pages(self, path): 53 | """Takes a Path class and removes all non hidden files and dirs from it""" 54 | for entry in path.iterdir(): 55 | if entry.name.startswith("."): 56 | continue 57 | elif entry.is_file(): 58 | entry.unlink() 59 | else: 60 | self.clean_pages(path / entry.name) 61 | entry.rmdir() 62 | 63 | def clean_exports(self, path, preserve): 64 | """Deletes all non hidden files of a folder which are not present in 65 | preserve""" 66 | for entry in path.iterdir(): 67 | if ( 68 | entry.is_file() 69 | and entry not in preserve 70 | and not entry.name.startswith(".") 71 | ): 72 | entry.unlink() 73 | 74 | def download_page(self, page_id, title, output_dir_path): 75 | """Downloads a page with id: page_id, title: title and zipfile path: 76 | output_dir_path""" 77 | task_id = self.launch_page_export(page_id) 78 | done = 0 79 | while done < N_TRAILS: 80 | try: 81 | while True: 82 | task_status = self.get_task_status(task_id) 83 | if task_status["status"]["type"] == "complete": 84 | break 85 | print( 86 | f"...Export still in progress, waiting for {STATUS_WAIT_TIME} seconds" 87 | ) 88 | sleep(STATUS_WAIT_TIME) 89 | print("Export task is finished") 90 | export_link = task_status["status"]["exportURL"] 91 | done = N_TRAILS + 1 92 | except: 93 | print(f"Problem downloading {title} on Trial {done + 1}") 94 | done += 1 95 | if done < N_TRAILS: 96 | print( 97 | f"Sleeping for {FAIL_SLEEP_TIME} seconds to prevent rate limiting" 98 | ) 99 | sleep(FAIL_SLEEP_TIME) 100 | 101 | if done == N_TRAILS: 102 | print(f"Failed all trails of downloading {title}") 103 | return None 104 | 105 | print(f"Downloading zip for {title}") 106 | self.download_file(export_link, output_dir_path) 107 | return output_dir_path 108 | -------------------------------------------------------------------------------- /templates/collection.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | {{title}} 7 | 158 | 159 | 160 | 161 |
162 |

163 | {{title}} 164 |

165 | 166 | {% if description is not none %} 167 |

168 | {{description}} 169 |

170 | {% endif %} 171 | 172 |
    173 | {% for link, title in pages %} 174 |
  1. {{title}}
    2. 175 | {% endfor %} 176 |
177 | 178 | 179 | 180 | -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | import notion 2 | from zipfile import ZipFile 3 | from notion.client import NotionClient 4 | from os import environ 5 | from pathlib import Path 6 | from cms.exporter import Exporter 7 | from jinja2 import Template 8 | 9 | token = environ.get("TOKEN_V2") 10 | client = NotionClient(token_v2=token) 11 | 12 | exporter = Exporter(client) 13 | new_exports = [] 14 | 15 | 16 | def get_block(block_id): 17 | """Gets a notion block""" 18 | block = client.get_block(block_id) 19 | return block 20 | 21 | 22 | def build_page(page_block): 23 | """Exports a page from notion. Does not export the page if 24 | the page is already available in exports. Returns a Path class 25 | for the corresponding htmk file""" 26 | 27 | title = page_block.title 28 | page_id = page_block.id 29 | 30 | page_info = page_block.get() 31 | last_edited_time = page_info["last_edited_time"] 32 | 33 | output_dir_path = Path("exports/") 34 | export_file_name = f"{page_id}|{last_edited_time}.zip" 35 | output_dir_path /= export_file_name 36 | 37 | if output_dir_path.is_file(): 38 | print(f"No changes in {title}") 39 | else: 40 | print(f"Downloading {title}") 41 | output_dir_path = exporter.download_page(page_id, title, output_dir_path) 42 | 43 | if output_dir_path is None: 44 | return None 45 | new_exports.append(output_dir_path) 46 | 47 | outpath = Path("pages/") 48 | with ZipFile(output_dir_path, "r") as zip_ref: 49 | print(f"Unziping {title}") 50 | zip_ref.extractall(outpath) 51 | zip_files = zip_ref.infolist() 52 | 53 | # Get HTML file name from zip 54 | zip_files = [ 55 | zip_file.filename 56 | for zip_file in zip_files 57 | if zip_file.filename.endswith(".html") 58 | ] 59 | return outpath / zip_files[0], title 60 | 61 | 62 | def build_collection(collection_block): 63 | """Traverses through a collection and builds its pages.""" 64 | collection = collection_block.collection 65 | collection_info = collection.get() 66 | 67 | try: 68 | description = collection_info["description"][0][0] 69 | except: 70 | description = None 71 | 72 | title = collection_info["name"][0][0] 73 | collection_id = collection_info["id"] 74 | 75 | pages = [] 76 | for row in collection.get_rows(): 77 | pages.append(build_page(row)) 78 | pages = [page for page in pages if page is not None] 79 | 80 | # Build collection page 81 | with open("./templates/collection.html") as home_template: 82 | home_contents = home_template.read() 83 | 84 | collect_t = Template(home_contents) 85 | collect_t = collect_t.render(title=title, description=description, pages=pages) 86 | 87 | outpath = Path(f"pages/{title} {collection_id}.html") 88 | outpath.write_text(collect_t) 89 | 90 | # Return collection page path,title 91 | return outpath, title 92 | 93 | 94 | def build_solopage(page_block): 95 | """Builds a single page""" 96 | page = build_page(page_block) 97 | 98 | # Return solopage path, title 99 | return page 100 | 101 | 102 | def build_space(space_id): 103 | """Traverses through a space and builds all collections and pages""" 104 | workspace = client.get_space(space_id) 105 | page_ids = workspace.get()["pages"] 106 | 107 | pages = [] 108 | for page_id in page_ids: 109 | page_block = get_block(page_id) 110 | if page_block is None: 111 | continue 112 | if page_block.type == "page": 113 | pages.append(build_solopage(page_block)) 114 | else: 115 | pages.append(build_collection(page_block)) 116 | 117 | # Build main page 118 | with open("./templates/homepage.html") as home_template: 119 | home_contents = home_template.read() 120 | 121 | collect_t = Template(home_contents) 122 | collect_t = collect_t.render(pages=pages) 123 | 124 | outpath = Path(f"index.html") 125 | outpath.write_text(collect_t) 126 | 127 | 128 | def main(): 129 | # Clean old pages 130 | print("Cleaning old pages") 131 | pages_path = Path("pages/") 132 | exporter.clean_pages(pages_path) 133 | 134 | # Build new pages 135 | print("Building Space") 136 | space_id = environ.get("SPACE_ID") 137 | build_space(space_id) 138 | 139 | # Clean redundent zips 140 | print("Removing Redundent Zips") 141 | export_path = Path("exports/") 142 | exporter.clean_exports(export_path, new_exports) 143 | 144 | 145 | if __name__ == "__main__": 146 | main() 147 | -------------------------------------------------------------------------------- /templates/homepage.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Homepage 7 | 158 | 159 | 160 | 161 |
162 |

163 | 164 | Homepage 165 |

166 |

Kunwar Shaanjeet Singh Grover

167 | 174 | 175 | :pensive: 176 | 177 |
    178 | {% for link, title in pages %} 179 |
  1. {{title}}
    2. 180 | {% endfor %} 181 |
182 | 183 | 184 | 185 | 186 | --------------------------------------------------------------------------------