├── data └── .gitkeep ├── requirements.txt ├── docker-compose.yml ├── LICENSE ├── Dockerfile ├── README.md ├── .gitignore └── kosync.py /data/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | fastapi==0.111.0 2 | tinydb==4.8.0 3 | uvicorn==0.29.0 4 | python-dotenv==1.0.1 5 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | kosync: 3 | image: kosync:latest 4 | container_name: kosync 5 | ports: 6 | - 8081:8081 7 | volumes: 8 | - ./data:/app/data 9 | environment: 10 | - RECEIVE_RANDOM_DEVICE_ID=False 11 | - OPEN_REGISTRATIONS=True 12 | restart: unless-stopped 13 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 B1N4RYJ4N 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.11-slim AS builder 2 | 3 | RUN apt-get update && apt-get install -y \ 4 | build-essential \ 5 | libpq-dev \ 6 | && apt-get clean \ 7 | && rm -rf /var/lib/apt/lists/* 8 | 9 | RUN mkdir /app 10 | RUN mkdir /app/data 11 | 12 | WORKDIR /app 13 | 14 | COPY requirements.txt . 15 | 16 | ENV PYTHONDONTWRITEBYTECODE 1 17 | 18 | RUN python3 -m pip install --user --no-cache-dir --upgrade \ 19 | pip \ 20 | setuptools \ 21 | wheel 22 | 23 | RUN python3 -m pip install --user --no-cache-dir \ 24 | -r requirements.txt 25 | 26 | FROM python:3.11-slim 27 | 28 | RUN apt-get update && apt-get install -y \ 29 | curl \ 30 | && apt-get clean \ 31 | && rm -rf /var/lib/apt/lists/* 32 | 33 | ENV PYTHONUNBUFFERED 1 34 | 35 | WORKDIR /app 36 | 37 | COPY --from=builder /root/.local /root/.local 38 | 39 | COPY kosync.py . 40 | 41 | EXPOSE 8081 42 | 43 | VOLUME ["/app/data"] 44 | 45 | ENV PATH=/root/.local/bin:$PATH 46 | 47 | HEALTHCHECK --interval=30s --timeout=10s CMD curl --fail http://localhost:8081/healthstatus || exit 1 48 | 49 | CMD ["uvicorn", "kosync:app", "--host", "0.0.0.0", "--port", "8081"] 50 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Koreader position sync server 2 | 3 | ## Description 4 | 5 | This is a simple implementation of the KOReader (https://github.com/koreader/koreader) position sync server for self-hosting at home which has docker support for arm and amd64 :) _This is a fork of https://github.com/myelsukov/koreader-sync but with a complete code rewrite._ 6 | 7 | ## Dependencies 8 | 9 | * FastAPI : https://github.com/tiangolo/fastapi 10 | * TinyDB: https://github.com/msiemens/tinydb 11 | * Uvicorn: https://www.uvicorn.org/ 12 | * Python-dotenv: https://saurabh-kumar.com/python-dotenv/ 13 | 14 | ## Install and run 15 | 16 | ```bash 17 | > pip install -r requirements.txt 18 | 19 | > uvicorn kosync:app --host 0.0.0.0 --port 8081 20 | 21 | ``` 22 | 23 | ## Or via Docker 24 | 25 | ```bash 26 | > docker build --rm=true --tag=kosync:latest . 27 | 28 | > docker compose up -d 29 | 30 | ``` 31 | 32 | ## Environment Variables 33 | 34 | * RECEIVE_RANDOM_DEVICE_ID ("True"|"False") 35 | 36 | Set it true to retrieve always a random device id to force a progress sync. 37 | This is usefull if you only sync your progress from one device and 38 | usually delete the *.sdr files with some cleaning tools. 39 | 40 | * OPEN_REGISTRATIONS ("True"|"False") 41 | 42 | Enable/disable new registrations to the server. Useful if you want to run a private server for a few users, although it doesn't necessarily improve security by itself. 43 | Set to True (enabled) by default. 44 | 45 | ## Dockerhub 46 | 47 | There is also a dockerhub image available if you are not able to build yourself the image. 48 | 49 | For linux/amd64 you can use `docker pull b1n4ryj4n/koreader-sync` and for linux/arm `docker pull b1n4ryj4n/koreader-sync:arm` 50 | 51 | ## Connection 52 | 53 | * Use http://IP:8081 as custom sync server 54 | * Recommendation: Setup a reverse proxy for example with Nginx Proxy Manager (https://nginxproxymanager.com/) to connect with https 55 | 56 | ## Changelog 57 | 58 | ## [0.0.5] - 2024-05-24 59 | ### Added 60 | - Merged ["Option to disable registration of new user accounts by env var."](https://github.com/b1n4ryj4n/koreader-sync/pull/5) 61 | 62 | ## [0.0.4] - 2023-10-29 63 | ### Added 64 | - Added the HEALTHCHECK command (also accessible via http://IP:8081/healthstatus) 65 | 66 | ## [0.0.3] - 2022-03-20 67 | ### Added 68 | - Added an environment variable option to receive always a random device id 69 | 70 | ## [0.0.2] - 2022-02-21 71 | ### Added 72 | - Merged ["Dockerfile: use multi-stage build to optimize image size"](https://github.com/b1n4ryj4n/koreader-sync/pull/3) 73 | 74 | ## [0.0.1] - 2021-09-15 75 | ### Added 76 | - First version 77 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | share/python-wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | MANIFEST 28 | 29 | # PyInstaller 30 | # Usually these files are written by a python script from a template 31 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 32 | *.manifest 33 | *.spec 34 | 35 | # Installer logs 36 | pip-log.txt 37 | pip-delete-this-directory.txt 38 | 39 | # Unit test / coverage reports 40 | htmlcov/ 41 | .tox/ 42 | .nox/ 43 | .coverage 44 | .coverage.* 45 | .cache 46 | nosetests.xml 47 | coverage.xml 48 | *.cover 49 | *.py,cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | cover/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | .pybuilder/ 76 | target/ 77 | 78 | # Jupyter Notebook 79 | .ipynb_checkpoints 80 | 81 | # IPython 82 | profile_default/ 83 | ipython_config.py 84 | 85 | # pyenv 86 | # For a library or package, you might want to ignore these files since the code is 87 | # intended to run in multiple environments; otherwise, check them in: 88 | # .python-version 89 | 90 | # pipenv 91 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 92 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 93 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 94 | # install all needed dependencies. 95 | #Pipfile.lock 96 | 97 | # poetry 98 | # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. 99 | # This is especially recommended for binary packages to ensure reproducibility, and is more 100 | # commonly ignored for libraries. 101 | # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control 102 | #poetry.lock 103 | 104 | # pdm 105 | # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. 106 | #pdm.lock 107 | # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it 108 | # in version control. 109 | # https://pdm.fming.dev/#use-with-ide 110 | .pdm.toml 111 | 112 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm 113 | __pypackages__/ 114 | 115 | # Celery stuff 116 | celerybeat-schedule 117 | celerybeat.pid 118 | 119 | # SageMath parsed files 120 | *.sage.py 121 | 122 | # Environments 123 | .env 124 | .venv 125 | env/ 126 | venv/ 127 | ENV/ 128 | env.bak/ 129 | venv.bak/ 130 | 131 | # Spyder project settings 132 | .spyderproject 133 | .spyproject 134 | 135 | # Rope project settings 136 | .ropeproject 137 | 138 | # mkdocs documentation 139 | /site 140 | 141 | # mypy 142 | .mypy_cache/ 143 | .dmypy.json 144 | dmypy.json 145 | 146 | # Pyre type checker 147 | .pyre/ 148 | 149 | # pytype static type analyzer 150 | .pytype/ 151 | 152 | # Cython debug symbols 153 | cython_debug/ 154 | 155 | # PyCharm 156 | # JetBrains specific template is maintained in a separate JetBrains.gitignore that can 157 | # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore 158 | # and can be added to the global gitignore or merged into this file. For a more nuclear 159 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. 160 | #.idea/ 161 | 162 | # koreader-sync 163 | data/db.json -------------------------------------------------------------------------------- /kosync.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import time 3 | import uuid 4 | from distutils.util import strtobool 5 | from os import getenv 6 | from typing import Optional 7 | 8 | from dotenv import load_dotenv 9 | from fastapi import FastAPI, Header 10 | from fastapi.responses import JSONResponse 11 | from pydantic import BaseModel 12 | from tinydb import Query, TinyDB 13 | 14 | app = FastAPI(openapi_url=None, redoc_url=None) 15 | db = TinyDB("data/db.json") 16 | users = db.table("users") 17 | documents = db.table("documents") 18 | load_dotenv() 19 | 20 | class KosyncUser(BaseModel): 21 | username: Optional[str] = None 22 | password: Optional[str] = None 23 | 24 | 25 | class KosyncDocument(BaseModel): 26 | document: Optional[str] = None 27 | progress: Optional[str] = None 28 | percentage: Optional[float] = None 29 | device: Optional[str] = None 30 | device_id: Optional[str] = None 31 | 32 | 33 | @app.post("/users/create") 34 | def register(kosync_user: KosyncUser): 35 | # Check whether new registrations are allowed on this server based on the OPEN_REGISTRATIONS environment variable. 36 | # By default registrations are enabled. 37 | registrations_allowed = bool(strtobool(getenv("OPEN_REGISTRATIONS", "True"))) 38 | if registrations_allowed: 39 | # check if username or password is missing 40 | if kosync_user.username is None or kosync_user.password is None: 41 | return JSONResponse(status_code=400, content={"message": f"Invalid request"}) 42 | # check if user already exists 43 | QUser = Query() 44 | if users.contains(QUser.username == kosync_user.username): 45 | return JSONResponse(status_code=409, content="Username is already registered.") 46 | # register new user 47 | if users.insert({'username': kosync_user.username, 'password': kosync_user.password}): 48 | return JSONResponse(status_code=201, content={"username": kosync_user.username}) 49 | # if something went wrong 50 | return JSONResponse(status_code=500, content="Unknown server error") 51 | else: 52 | return JSONResponse(status_code=403, content="This server is currently not accepting new registrations.") 53 | 54 | 55 | @app.get("/users/auth") 56 | def authorize(x_auth_user: Optional[str] = Header(None), x_auth_key: Optional[str] = Header(None)): 57 | # check if username or password is missing 58 | if x_auth_user is None or x_auth_key is None: 59 | return JSONResponse(status_code=401, content={"message": f"Unauthorized"}) 60 | # check if username is in database 61 | QUser = Query() 62 | # check username and password combination 63 | if users.contains(QUser.username == x_auth_user): 64 | if users.contains((QUser.username == x_auth_user) & (QUser.password == x_auth_key)): 65 | return JSONResponse(status_code=200, content={"authorized": f"OK"}) 66 | else: 67 | return JSONResponse(status_code=401, content={"message": f"Unauthorized"}) 68 | return JSONResponse(status_code=403, content={"message": f"Forbidden"}) 69 | 70 | 71 | @app.put("/syncs/progress") 72 | def update_progress(kosync_document: KosyncDocument, x_auth_user: Optional[str] = Header(None), 73 | x_auth_key: Optional[str] = Header(None)): 74 | # check if username or password is missing 75 | if x_auth_user is None or x_auth_key is None: 76 | return JSONResponse(status_code=401, content={"message": f"Unauthorized"}) 77 | QUser = Query() 78 | QDocument = Query() 79 | # check if username is in database 80 | if not users.contains(QUser.username == x_auth_user): 81 | return JSONResponse(status_code=403, content={"message": f"Forbidden"}) 82 | # check username and password combination before put data in database 83 | if users.contains((QUser.username == x_auth_user) & (QUser.password == x_auth_key)): 84 | # add new document progress 85 | timestamp = int(time.time()) 86 | if kosync_document.document is None or kosync_document.progress is None or kosync_document.percentage is None \ 87 | or kosync_document.device is None or kosync_document.device_id is None: 88 | return JSONResponse(status_code=500, content="Unknown server error") 89 | else: 90 | if documents.upsert({'username': x_auth_user, 'document': kosync_document.document, 91 | 'progress': kosync_document.progress, 'percentage': kosync_document.percentage, 92 | 'device': kosync_document.device, 'device_id': kosync_document.device_id, 93 | 'timestamp': timestamp}, (QDocument.username == x_auth_user) & 94 | (QDocument.document == kosync_document.document)): 95 | return JSONResponse(status_code=200, 96 | content={"document": kosync_document.document, "timestamp": timestamp}) 97 | else: 98 | return JSONResponse(status_code=401, content={"message": f"Unauthorized"}) 99 | 100 | 101 | @app.get("/syncs/progress/{document}") 102 | def get_progress(document: Optional[str] = None, x_auth_user: Optional[str] = Header(None), 103 | x_auth_key: Optional[str] = Header(None)): 104 | # check if username or password is missing 105 | if x_auth_user is None or x_auth_key is None: 106 | return JSONResponse(status_code=401, content={"message": f"Unauthorized"}) 107 | # check if document parameter exists 108 | if document is None: 109 | return JSONResponse(status_code=500, content="Unknown server error") 110 | 111 | QUser = Query() 112 | QDocument = Query() 113 | 114 | # check if username is in database 115 | if not users.contains(QUser.username == x_auth_user): 116 | return JSONResponse(status_code=403, content={"message": f"Forbidden"}) 117 | 118 | # check username and password combination before get progress data 119 | if users.contains((QUser.username == x_auth_user) & (QUser.password == x_auth_key)): 120 | # get document progress if user has the document 121 | result = documents.get((QDocument.username == x_auth_user) & (QDocument.document == document)) 122 | if result: 123 | rrdi = bool(strtobool(getenv("RECEIVE_RANDOM_DEVICE_ID", "False"))) 124 | if rrdi == False: 125 | device_id = result["device_id"] 126 | else: 127 | device_id = uuid.uuid1() 128 | device_id = str(device_id.hex).upper() 129 | return JSONResponse(status_code=200, 130 | content={'username': x_auth_user, 'document': result["document"], 131 | 'progress': result["progress"], 'percentage': result["percentage"], 132 | 'device': result["device"], 'device_id': device_id, 133 | 'timestamp': result["timestamp"]}) 134 | else: 135 | return JSONResponse(status_code=401, content={"message": f"Unauthorized"}) 136 | 137 | @app.get("/healthstatus") 138 | def get_healthstatus(): 139 | return JSONResponse(status_code=200, content={"message": f"healthy"}) 140 | --------------------------------------------------------------------------------