├── .gitignore ├── LICENCE ├── README.rst ├── remarkable_fs ├── __init__.py ├── connection.py ├── documents.py ├── fs.py └── rM2svg.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | build 2 | *.egg-info 3 | *.pyc 4 | *.pyo 5 | -------------------------------------------------------------------------------- /LICENCE: -------------------------------------------------------------------------------- 1 | All files except remarkable_fs/rM2svg.py are covered by the following 2 | copyright licence: 3 | 4 | Copyright (c) 2018 Nick Smallbone 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a copy 7 | of this software and associated documentation files (the "Software"), to deal 8 | in the Software without restriction, including without limitation the rights 9 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | copies of the Software, and to permit persons to whom the Software is 11 | furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in all 14 | copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 21 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 22 | SOFTWARE. 23 | 24 | remarkable_fs/rM2svg.py is licensed under the GNU Lesser General 25 | Public License, version 3. 26 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | A FUSE filesystem driver for reMarkable 2 | ======================================= 3 | 4 | ``remarkable-fs`` allows you to see the contents of your reMarkable as a normal 5 | folder on your computer. All your documents and folders are visible; you can 6 | copy documents to and from the reMarkable, create folders, and move and delete 7 | folders and files. You cannot, however, edit documents that are 8 | already on the reMarkable (apart from moving or deleting them). 9 | 10 | It supports PDF files, EPUB files, and handwritten notes, but does not 11 | yet support annotated PDFs (they are exported without annotations). 12 | 13 | *This software is in an early stage of development. I take no responsibility if 14 | it deletes your files or bricks your reMarkable! Do not use it if you are 15 | unwilling to lose all your documents. Use at your own risk!* 16 | 17 | Installation 18 | ------------ 19 | 20 | ``remarkable-fs`` works on both Linux and macOS. Support for macOS may 21 | be a bit patchy as I myself run Linux, but at least the basics should 22 | work. To install it, you will need to have first installed: 23 | 24 | - FUSE. If on macOS, get this from the `Fuse for macOS`_ project. If 25 | on Linux, your package manager should have it. 26 | - ``pip``, the Python package installer. You can install this by running 27 | ``sudo easy_install pip`` in a terminal. 28 | 29 | .. _Fuse for macOS: https://osxfuse.github.io/ 30 | 31 | Then, to install ``remarkable-fs``, run the following command in a 32 | terminal: 33 | 34 | sudo pip install remarkable-fs 35 | 36 | Note for Linux users: ``remarkable-fs`` supports both Python 2 and 3, 37 | but for handwritten notes it works much faster on *Python 2*. 38 | 39 | Running 40 | ------- 41 | 42 | First make sure that your reMarkable is plugged into your computer. 43 | 44 | Make an empty directory on your computer. This directory is where your 45 | documents will appear. Then run ``remarkable-fs``. You will be 46 | prompted for the path to this directory; type it in. (On macOS, you 47 | can instead drag the directory to the terminal window at this point.) 48 | 49 | You will then be prompted for the root password of your reMarkable. 50 | You can find this by opening the settings menu of the reMarkable, 51 | choosing "About", scrolling to the very bottom, and finding the 52 | paragraph beginning: "To do so, this device acts as a USB ethernet 53 | device..." (If you don't want to have to type in the root password 54 | every time you run ``remarkable-fs``, follow the instructions on 55 | passwordless login from the `reMarkable wiki`_.) 56 | 57 | .. _reMarkable wiki: http://remarkablewiki.com/index.php?title=Methods_of_access#Setting_up_ssh-keys 58 | 59 | If all goes well, you will find all your documents in the directory 60 | you chose. You can copy PDF or EPUB files there, and they will appear 61 | on the reMarkable; you can also move files around. Go wild! 62 | 63 | (On macOS, your reMarkable will also appear in the finder under the 64 | menu Go -> Computer.) 65 | 66 | When you are finished, you can stop ``remarkable-fs`` by pressing ctrl-C. 67 | 68 | Note that your reMarkable will be unresponsive for the time you have 69 | ``remarkable-fs`` running. It should start responding as soon as you close 70 | ``remarkable-fs`` or unplug the USB cable. If for some reason it doesn't, you 71 | can force your reMarkable to restart by holding down the power button for five 72 | seconds, letting go, and then pressing the power button for another second. 73 | 74 | Copyright 75 | --------- 76 | 77 | The software is licensed under the MIT licence, apart from the file 78 | remarkable_fs/rM2svg.py. That file is is licensed under the GNU LGPL 79 | version 3, and is taken from https://github.com/phil777/maxio. 80 | -------------------------------------------------------------------------------- /remarkable_fs/__init__.py: -------------------------------------------------------------------------------- 1 | from remarkable_fs.connection import connect 2 | from remarkable_fs.documents import DocumentRoot 3 | from remarkable_fs.fs import mount 4 | import sys 5 | import fuse 6 | 7 | try: 8 | import __builtin__ 9 | raw_input = __builtin__.raw_input 10 | except: 11 | raw_input = input 12 | 13 | def main(argv = sys.argv): 14 | if len(argv) >= 2: 15 | mount_point = argv[1] 16 | else: 17 | mount_point = raw_input("Directory: ") 18 | 19 | print("Connecting to reMarkable...") 20 | with connect(*argv[2:]) as conn: 21 | root = DocumentRoot(conn) 22 | print("Now serving documents at " + mount_point) 23 | kwargs={} 24 | if fuse.system() == "Darwin": 25 | kwargs["volname"] = "reMarkable" 26 | mount(mount_point, root, **kwargs) 27 | -------------------------------------------------------------------------------- /remarkable_fs/connection.py: -------------------------------------------------------------------------------- 1 | """Handles maintaining a connection to the reMarkable.""" 2 | 3 | from contextlib import contextmanager 4 | from collections import namedtuple 5 | from paramiko.client import SSHClient, AutoAddPolicy 6 | from paramiko.sftp_client import SFTPClient 7 | from paramiko.ssh_exception import SSHException, AuthenticationException 8 | from getpass import getpass 9 | from signal import signal, SIGTERM, SIGHUP 10 | 11 | Connection = namedtuple('Connection', 'ssh sftp') 12 | 13 | @contextmanager 14 | def connect(addr=None): 15 | """Connect to the remarkable. Yields a Connection object. 16 | 17 | The sftp field of the connection object has as its working directory the 18 | data directory of xochitl.""" 19 | 20 | default_addr = "10.11.99.1" 21 | with SSHClient() as ssh: 22 | ssh.load_system_host_keys() 23 | ssh.set_missing_host_key_policy(AutoAddPolicy) 24 | try: 25 | ssh.connect(addr or default_addr, username="root") 26 | except (SSHException, AuthenticationException): 27 | print("Please enter the root password of your reMarkable.") 28 | print("To find out the password, follow the instructions at:") 29 | print("http://remarkablewiki.com/index.php?title=Methods_of_access#Connecting_via_ssh") 30 | password = getpass() 31 | ssh.connect(addr or default_addr, username="root", password=password, look_for_keys=False) 32 | 33 | # Stop xochitl but restart it again if the connection drops 34 | on_start = "systemctl stop xochitl" 35 | on_finish = "systemctl restart xochitl" 36 | # We know USB was disconnected when the power supply drops. 37 | # We also kill the SSH connection so that the information 38 | # in FUSE is not out of date. 39 | ssh.exec_command(on_start) 40 | if addr is None: 41 | # Only do this if we are plugged in to the device 42 | ssh.exec_command("while udevadm info -p /devices/soc0/soc/2100000.aips-bus/2184000.usb/power_supply/imx_usb_charger | grep -q POWER_SUPPLY_ONLINE=1; do sleep 1; done; %s; kill $PPID" % on_finish) 43 | 44 | try: 45 | def raise_exception(*args): 46 | raise RuntimeError("Process terminated") 47 | signal(SIGTERM, raise_exception) 48 | signal(SIGHUP, raise_exception) 49 | with ssh.open_sftp() as sftp: 50 | sftp.chdir("/home/root/.local/share/remarkable/xochitl") 51 | yield Connection(ssh, sftp) 52 | 53 | finally: 54 | # Closing stdin triggers on_finish to run, so only do it now 55 | try: 56 | ssh.exec_command(on_finish) 57 | except: 58 | pass 59 | -------------------------------------------------------------------------------- /remarkable_fs/documents.py: -------------------------------------------------------------------------------- 1 | """Classes for reading and creating reMarkable documents. The main entry point 2 | to this module is the DocumentRoot class. 3 | 4 | Minimal example: 5 | 6 | >>> from remarkable_fs.connection import connect 7 | >>> from remarkable_fs.documents import DocumentRoot 8 | >>> 9 | >>> with connect() as conn: 10 | >>> root = DocumentRoot(conn) 11 | >>> print root["foo.pdf"].read(0, 4096) # prints first 4KB of foo.pdf""" 12 | 13 | import fnmatch 14 | import json 15 | import time 16 | import os.path 17 | import itertools 18 | import traceback 19 | from tempfile import NamedTemporaryFile 20 | from uuid import uuid4 21 | from lazy import lazy 22 | from progress.bar import Bar 23 | from io import BytesIO 24 | import remarkable_fs.rM2svg 25 | 26 | try: 27 | from json import JSONDecodeError 28 | except ImportError: 29 | JSONDecodeError = ValueError 30 | 31 | class Node(object): 32 | """A document or collection on the reMarkable. 33 | 34 | Most of the properties below are read-write, but to persist any changes to 35 | the reMarkable, you must call save().""" 36 | 37 | def __init__(self, root, id, metadata): 38 | """Create a new Node object. Unless you are hacking on this module, you 39 | do not want to do this. Instead, you can get existing documents by 40 | indexing into the DocumentRoot, or create new ones with 41 | Collection.new_document or Collection.new_collection. 42 | 43 | For those hacking on this module, creating a node object requires the 44 | following steps: 45 | * The correct class must be chosen for the node (either Collection, 46 | Document or NewDocument). 47 | * An object of that class is created, passing in the DocumentRoot and 48 | the node's id and metadata (which is stored as a dict). 49 | A new id can be created using new_id(). 50 | Metadata can be read using DocumentRoot.read_metadata, 51 | or created using initial_metadata. 52 | * The node is registered using DocumentRoot.register_node(). 53 | * The node is added to the directory hierarchy by calling link(). 54 | This can only be called if the parent node has been created, 55 | which is why it is not done automatically. 56 | The easiest way to create a node is to write the metadata to the 57 | reMarkable, and then call DocumentRoot.load_node() to load it. 58 | That is what Collection.new_collection() does, for example.""" 59 | 60 | self.root = root 61 | self.id = id 62 | self.metadata = metadata 63 | self.modified = False 64 | if metadata is not None: 65 | self.file_name = self.name 66 | 67 | def __repr__(self): 68 | return "%s(%s, %s)" % \ 69 | (type(self).__name__, 70 | self.id, 71 | self.name) 72 | 73 | def link(self): 74 | """Add a node to the directory hierarchy. May only be called after the 75 | parent node has been loaded. Unless you are hacking on this module, you 76 | do not want this function.""" 77 | 78 | self.parent = self.root.find_node(self.metadata["parent"]) 79 | if self.parent is not None: 80 | self.parent.add_child(self) 81 | 82 | def _rw(name, doc): 83 | """Creates a property which references a metadata field. On update, the 84 | metadata is marked as being modified.""" 85 | 86 | def get(self): 87 | return self.metadata[name] 88 | def set(self, val): 89 | self.metadata["synced"] = False 90 | self.metadata["metadatamodified"] = True 91 | self.metadata["version"] += 1 92 | self.metadata[name] = val 93 | self.modified = True 94 | return property(fget=get, fset=set, doc=doc) 95 | 96 | name = _rw("visibleName", "The document name.") 97 | deleted = _rw("deleted", "True if the document has been deleted.") 98 | data_modified = _rw("modified", "True if the data in the document has been modified.") 99 | pinned = _rw("pinned", "True if the document is bookmarked.") 100 | 101 | @property 102 | def metadata_modified(self): 103 | """True if the metadata of the document has been modified.""" 104 | return self.metadata["metadatamodified"] 105 | 106 | @property 107 | def size(self): 108 | """The approximate size of the document in bytes.""" 109 | return 0 110 | 111 | @property 112 | def mtime(self): 113 | """The modification time of the document.""" 114 | if self.metadata is None: 115 | return time.time() 116 | else: 117 | return int(int(self.metadata["lastModified"])/1000) 118 | 119 | def save(self): 120 | """Write the document to the reMarkable if it has been changed.""" 121 | if self.modified: 122 | self.root.write_metadata(self.id, self.metadata) 123 | self.modified = False 124 | 125 | def rename(self, parent, file_name): 126 | """Rename the document. Arguments are the parent node and the new filename.""" 127 | 128 | self.parent.remove_child(self) 129 | self.name = strip_extension(file_name) 130 | self.file_name = file_name 131 | self.metadata["parent"] = parent.id 132 | self.parent = parent 133 | self.parent.add_child(self) 134 | self.save() 135 | 136 | def delete(self): 137 | """Delete the document.""" 138 | self.parent.remove_child(self) 139 | self.deleted = True 140 | self.save() 141 | 142 | class Collection(Node): 143 | """A reMarkable collection. 144 | 145 | You can index into a Collection as if it was a dict. The keys are filenames 146 | and the values are nodes.""" 147 | 148 | def __init__(self, root, id, metadata): 149 | super(Collection, self).__init__(root, id, metadata) 150 | self.children = {} 151 | self.children_pathnames = {} 152 | 153 | def new_collection(self, name): 154 | """Create a new collection. Returns the created node.""" 155 | 156 | id = new_id() 157 | metadata = initial_metadata(Collection.node_type(), name, self.id) 158 | self.root.write_metadata(id, metadata) 159 | self.root.write_content(id, {}) 160 | return self.root.load_node(id) 161 | 162 | def new_document(self, name): 163 | """Create a new document. Returns the created node. 164 | 165 | The document will not be written to the reMarkable until the node's 166 | save() method is called.""" 167 | 168 | id = new_id() 169 | metadata = initial_metadata(Document.node_type(), strip_extension(name), self.id) 170 | node = NewDocument(self.root, id, metadata, name) 171 | self.root.register_node(node) 172 | node.link() 173 | return node 174 | 175 | def add_child(self, child): 176 | """Add a node to this collection. Called by link(). 177 | Unless you are hacking on this module, you do not want this function.""" 178 | 179 | # Remove invalid chars 180 | name = child.file_name.replace("/", "-") 181 | 182 | # Disambiguate duplicate names e.g. Foo/bar, Foo-bar 183 | if name in self.children: 184 | for n in itertools.count(2): 185 | x = "%s (%d)" % (name, n) 186 | if x not in self.children: 187 | name = x 188 | break 189 | 190 | self.children[name] = child 191 | self.children_pathnames[child] = name 192 | 193 | def remove_child(self, child): 194 | """Remove a node from this collection. Called by rename() and delete(). 195 | Unless you are hacking on this module, you do not want this function.""" 196 | 197 | name = self.children_pathnames[child] 198 | del self.children[name] 199 | del self.children_pathnames[child] 200 | 201 | def get(self, file): 202 | """Look up a file in the collection. Returns a node, or None if not found.""" 203 | return self.children.get(file) 204 | 205 | def items(self): 206 | """Find all nodes in the collection.""" 207 | return self.children.items() 208 | 209 | def __repr__(self): 210 | return "%s(%s, %s, %s)" % \ 211 | (type(self).__name__, 212 | self.id, 213 | self.name, 214 | self.children) 215 | 216 | def __getitem__(self, key): 217 | return self.children[key] 218 | 219 | def __iter__(self): 220 | return iter(self.children) 221 | 222 | def __contains__(self, item): 223 | return item in self.children 224 | 225 | @staticmethod 226 | def node_type(): 227 | return "CollectionType" 228 | 229 | class DocumentRoot(Collection): 230 | """A collection representing the root of the reMarkable directory tree. 231 | 232 | Creating one of these will read in all metadata and construct the directory hierarchy. 233 | 234 | You can index into a DocumentRoot as if it was a dict. The keys are 235 | filenames and the values are nodes. You can also use find_node() to look up 236 | a node by id.""" 237 | 238 | def __init__(self, connection): 239 | """connection - a Connection object returned by remarkable_fs.connection.connect().""" 240 | 241 | super(DocumentRoot, self).__init__(self, "", None) 242 | self.nodes = {"": self} 243 | self.sftp = connection.sftp 244 | self.templates = {} 245 | 246 | paths = fnmatch.filter(self.sftp.listdir(), '*.metadata') 247 | bar = Bar("Reading document information", max=len(paths)) 248 | for path in paths: 249 | id, _ = os.path.splitext(path) 250 | self.load_node_without_linking(id) 251 | bar.next() 252 | 253 | for node in self.nodes.values(): 254 | node.link() 255 | 256 | bar.finish() 257 | 258 | def find_node(self, id): 259 | """Find a node by id. Returns None if not found.""" 260 | return self.nodes.get(id) 261 | 262 | @property 263 | def name(self): 264 | return "ROOT" 265 | 266 | def link(self): 267 | pass 268 | 269 | def load_node(self, id): 270 | """Read a node from the reMarkable and link it into the tree. 271 | Unless you are hacking on this module, you do not want this function.""" 272 | 273 | node = self.load_node_without_linking(id) 274 | if node is not None: node.link() 275 | return node 276 | 277 | def load_node_without_linking(self, id): 278 | """Read a node from the reMarkable, without linking it into the tree. 279 | Unless you are hacking on this module, you do not want this function.""" 280 | 281 | classes = [Document, Collection] 282 | classes_dict = {cls.node_type(): cls for cls in classes} 283 | 284 | metadata = json.loads(self.read_file(id + ".metadata").decode("utf-8")) 285 | try: 286 | cls = classes_dict[metadata["type"]] 287 | except KeyError: 288 | cls = Node 289 | 290 | try: 291 | node = cls(self, id, metadata) 292 | if not node.deleted: 293 | self.register_node(node) 294 | return node 295 | except NoContents: 296 | pass 297 | except (IOError, JSONDecodeError): 298 | traceback.print_exc() 299 | 300 | def register_node(self, node): 301 | """Register a node object. Unless you are hacking on this module, you do 302 | not want this function.""" 303 | self.nodes[node.id] = node 304 | 305 | def read_file(self, file): 306 | """Read a file from SFTP.""" 307 | return self.sftp.open(file, "rb").read() 308 | 309 | def write_file(self, file, data): 310 | """Write a file to SFTP.""" 311 | f = self.sftp.open(file, "wb") 312 | f.set_pipelined() 313 | f.write(memoryview(data)) 314 | 315 | def read_json(self, file): 316 | """Read a JSON file from SFTP and convert to a dict.""" 317 | return json.loads(self.read_file(file).decode("utf-8")) 318 | 319 | def write_json(self, file, value): 320 | """Write a JSON file from SFTP, given as a dict.""" 321 | self.write_file(file, json.dumps(value).encode("utf-8")) 322 | 323 | def read_metadata(self, id): 324 | """Read the metadata for a given id and convert to a dict.""" 325 | return self.read_json(id + ".metadata") 326 | 327 | def write_metadata(self, id, metadata): 328 | """Write the metadata for a given id, given as a dict.""" 329 | self.write_json(id + ".metadata", metadata) 330 | 331 | def read_content(self, id): 332 | """Read the .content file for a given id and convert to a dict.""" 333 | return self.read_json(id + ".content") 334 | 335 | def write_content(self, id, content): 336 | """Write the .content file for a given id, given as a dict.""" 337 | self.write_json(id + ".content", content) 338 | 339 | def read_template(self, name): 340 | """Read a particular template file. Returns a local filename.""" 341 | file = self.templates.get(name) 342 | if file is None: 343 | data = self.read_file("/usr/share/remarkable/templates/%s.png" % name) 344 | file = NamedTemporaryFile(suffix = ".png") 345 | file.write(data) 346 | file.flush() 347 | self.templates[name] = file 348 | 349 | return file.name 350 | 351 | class NoContents(Exception): 352 | """An exception that indicates that a document only has notes and no PDF or EPUB file.""" 353 | pass 354 | 355 | class Document(Node): 356 | """A single document on the reMarkable.""" 357 | 358 | def __init__(self, root, id, metadata): 359 | super(Document, self).__init__(root, id, metadata) 360 | self.content = self.root.read_content(id) 361 | if self.file_type() == "": 362 | raise NoContents() 363 | self.file_name = self.name + "." + self.file_type() 364 | self._size = self.root.sftp.stat(self.id + "." + self.file_type()).st_size 365 | 366 | def file_type(self): 367 | """Return the type of file.""" 368 | return self.content["fileType"] 369 | 370 | @lazy 371 | def file(self): 372 | """A file handle to the file contents itself.""" 373 | ext = self.file_type() 374 | return self.root.sftp.open(self.id + "." + ext, "rb") 375 | 376 | @property 377 | def size(self): 378 | return self._size 379 | 380 | def read(self, offset, length): 381 | """Read length bytes from position offset.""" 382 | self.file.seek(offset) 383 | return self.file.read(length) 384 | 385 | @staticmethod 386 | def node_type(): 387 | return "DocumentType" 388 | 389 | class NewDocument(Node): 390 | """A newly-created document, which (unlike an object of class Document) can 391 | be both read and written. 392 | 393 | On calling save(), the document is converted to PDF or EPUB (if necessary) 394 | and written to the remarkable. If the document could not be converted. 395 | an IOError is thrown. 396 | 397 | File names starting with a dot are not written to the reMarkable 398 | (they are treated as temporary files).""" 399 | 400 | def __init__(self, root, id, metadata, filename): 401 | super(NewDocument, self).__init__(root, id, metadata) 402 | self.modified = True 403 | self.buf = BytesIO() 404 | self.file_name = filename 405 | 406 | @property 407 | def size(self): 408 | return len(self.buf.getvalue()) 409 | 410 | def read(self, offset, length): 411 | """Read length bytes from position offset.""" 412 | return self.buf.getvalue()[offset:offset+length] 413 | 414 | def write(self, offset, data): 415 | """Read data to position offset.""" 416 | self.buf.seek(offset) 417 | self.buf.write(data) 418 | 419 | def truncate(self, length): 420 | """Truncate the file to a certain length.""" 421 | self.buf.truncate(length) 422 | 423 | def save(self): 424 | if not self.file_name.startswith(".") and not self.deleted: 425 | self.really_save() 426 | 427 | def really_save(self): 428 | # Don't save zero-size files - Finder creates them 429 | if self.modified and len(self.buf.getvalue()) > 0: 430 | try: 431 | (filetype, data) = convert_document(self.buf.getvalue()) 432 | except IOError: 433 | self.delete() 434 | raise 435 | content = { 436 | "extraMetadata": {}, 437 | "fileType": filetype, 438 | "fontName": "", 439 | "lastOpenedPage": 0, 440 | "lineHeight": -1, 441 | "margins": 100, 442 | "orientation": "portrait", 443 | "pageCount": 1, 444 | "textScale": 1, 445 | "transform": { 446 | "m11": 1, 447 | "m12": 0, 448 | "m13": 0, 449 | "m21": 0, 450 | "m22": 1, 451 | "m23": 0, 452 | "m31": 0, 453 | "m32": 0, 454 | "m33": 1 455 | } 456 | } 457 | self.root.write_content(self.id, content) 458 | self.root.write_file(self.id + "." + filetype, data) 459 | super(NewDocument, self).save() 460 | 461 | def rename(self, parent, file_name): 462 | # If this file starts with a dot and now we want to rename it so it 463 | # doesn't, we can no longer treat it as a temporary file. 464 | if not file_name.startswith("."): 465 | self.really_save() 466 | super(NewDocument, self).rename(parent, file_name) 467 | 468 | def new_id(): 469 | """Generate a new document id.""" 470 | 471 | return str(uuid4()) 472 | 473 | def strip_extension(filename): 474 | """Remove the extension from a filename, if it is a recognised document type.""" 475 | 476 | name, ext = os.path.splitext(filename) 477 | if ext in [".pdf", ".djvu", ".ps", ".epub"]: 478 | return name 479 | return filename 480 | 481 | def initial_metadata(node_type, name, parent): 482 | """The .metadata for a newly-created node. 483 | 484 | node_type - value of 'type' field of .metadata file 485 | name - node name 486 | parent - parent id (not node object)""" 487 | 488 | return { 489 | "deleted": False, 490 | "lastModified": str(int(time.time()*1000)), 491 | "metadatamodified": True, 492 | "modified": True, 493 | "parent": parent, 494 | "pinned": False, 495 | "synced": False, 496 | "type": node_type, 497 | "version": 1, 498 | "visibleName": name 499 | } 500 | 501 | def convert_document(data): 502 | """Convert a document to PDF or EPUB. 503 | 504 | Input is the document contents as a 'bytes'. 505 | 506 | Returns (filetype, converted contents) where filetype is either "pdf" or 507 | "epub" and converted contents is a 'bytes'. 508 | 509 | Raises IOError if the file could not be converted.""" 510 | 511 | convert = None 512 | if data.startswith(b"%PDF"): 513 | filetype = "pdf" 514 | elif data.startswith(b"AT&TFORM"): 515 | filetype = "pdf" 516 | suffix = ".djvu" 517 | convert = "ddjvu --format=pdf" 518 | elif data.startswith(b"%!PS-Adobe"): 519 | filetype = "pdf" 520 | suffix = ".ps" 521 | convert = "ps2pdf" 522 | elif data.startswith(b"PK"): 523 | filetype = "epub" 524 | else: 525 | raise IOError("Only PDF, epub, djvu and ps format files supported") 526 | 527 | if convert is not None: 528 | infile = NamedTemporaryFile(suffix = suffix) 529 | outfile = NamedTemporaryFile(suffix = ".pdf") 530 | infile.write(data) 531 | infile.flush() 532 | res = os.system("%s %s %s" % (convert, infile.name, outfile.name)) 533 | if res != 0: 534 | raise IOError("Could not run %s" % convert) 535 | data = outfile.read() 536 | 537 | return (filetype, data) 538 | -------------------------------------------------------------------------------- /remarkable_fs/fs.py: -------------------------------------------------------------------------------- 1 | """A FUSE filesystem wrapper for the reMarkable.""" 2 | 3 | import os 4 | import sys 5 | from errno import * 6 | from posix import O_WRONLY, O_RDWR 7 | import stat 8 | from fuse import FUSE, FuseOSError, Operations 9 | from remarkable_fs.documents import Collection, Document, NewDocument 10 | from io import BytesIO 11 | import traceback 12 | 13 | class FileHandles(object): 14 | """Keeps track of the mapping between file handles and files.""" 15 | def __init__(self): 16 | # All open files 17 | self.file_handles = {} 18 | # File handles which have been closed and can be re-used 19 | self.free_file_handles = [] 20 | # The next unused (never allocated) file handle 21 | self.next_file_handle = 0 22 | 23 | def new(self, file): 24 | """Allocate a new file descriptor and return it.""" 25 | 26 | # Find a free file handle 27 | if self.free_file_handles: 28 | fd = self.free_file_handles.pop() 29 | else: 30 | fd = self.next_file_handle 31 | self.next_file_handle += 1 32 | 33 | # Record that the file is open 34 | self.file_handles[fd] = file 35 | return fd 36 | 37 | def close(self, fd): 38 | """Close a file descriptor.""" 39 | 40 | del self.file_handles[fd] 41 | self.free_file_handles.append(fd) 42 | 43 | def get(self, fd): 44 | """Look up a file descriptor. The file descriptor must be valid.""" 45 | return self.file_handles[fd] 46 | 47 | class Remarkable(Operations): 48 | """The main filesystem implementation.""" 49 | 50 | def __init__(self, documents): 51 | """documents - a remarkable_fs.documents.DocumentRoot object.""" 52 | 53 | self.documents = documents 54 | self.fds = FileHandles() 55 | 56 | def node(self, path): 57 | """Find a node in the filesystem. 58 | 59 | Raises ENOENT if the file does not exist.""" 60 | 61 | path = os.path.normpath(path) 62 | if path == '/' or path == '.': 63 | return self.documents 64 | else: 65 | dir, file = os.path.split(path) 66 | node = self.node(dir) 67 | try: 68 | return node[file] 69 | except KeyError: 70 | raise FuseOSError(ENOENT) 71 | 72 | def parent(self, path): 73 | """Find the parent node of a path in the filesystem. The path does not 74 | have to exist but its parent directory should. Generally used when 75 | creating a new file. 76 | 77 | Returns (parent node, basename). Raises ENOENT if the parent directory 78 | does not exist and EBUSY if the path is the root directory.""" 79 | 80 | path = os.path.normpath(path) 81 | dir, file = os.path.split(path) 82 | 83 | if file == '': 84 | # Root directory - cannot be moved/created/deleted 85 | raise FuseOSError(EBUSY) 86 | 87 | return (self.node(dir), file) 88 | 89 | # 90 | # Opening and closing files 91 | # 92 | 93 | def open(self, path, flags): 94 | node = self.node(path) 95 | 96 | # Don't allow overwriting existing files 97 | # (changing this needs more code in documents.py) 98 | if flags & O_WRONLY or flags & O_RDWR and not isinstance(node, NewDocument): 99 | raise FuseOSError(EPERM) 100 | 101 | return self.fds.new(self.node(path)) 102 | 103 | def create(self, path, flags): 104 | parent, name = self.parent(path) 105 | 106 | # Don't allow overwriting existing files, for paranoia 107 | if name in parent: 108 | raise FuseOSError(EEXIST) 109 | 110 | return self.fds.new(parent.new_document(name)) 111 | 112 | def flush(self, path, fd): 113 | try: 114 | self.fds.get(fd).save() 115 | except IOError: 116 | # File conversion error 117 | traceback.print_exc() 118 | raise FuseOSError(EIO) 119 | 120 | def release(self, path, fd): 121 | self.fds.close(fd) 122 | 123 | # 124 | # Reading and writing files 125 | # 126 | 127 | def read(self, path, size, offset, fd): 128 | node = self.fds.get(fd) 129 | if isinstance(node, Collection): 130 | raise FuseOSError(EISDIR) 131 | return self.fds.get(fd).read(offset, size) 132 | 133 | def write(self, path, data, offset, fd): 134 | node = self.fds.get(fd) 135 | if isinstance(node, Collection): 136 | raise FuseOSError(EISDIR) 137 | node.write(offset, data) 138 | 139 | return len(data) 140 | 141 | def truncate(self, path, length, fd=None): 142 | if fd is None: 143 | node = self.node(path) 144 | else: 145 | node = self.fds.get(fd) 146 | 147 | # Don't allow overwriting existing files 148 | # (changing this needs more code in documents.py) 149 | if hasattr(node, "truncate"): 150 | node.truncate(length) 151 | else: 152 | raise FuseOSError(EPERM) 153 | 154 | # 155 | # Creating directories, moving, deleting 156 | # 157 | 158 | def mkdir(self, path, mode): 159 | parent, name = self.parent(path) 160 | if name in parent: 161 | raise FuseOSError(EEXIST) 162 | parent.new_collection(name) 163 | 164 | def rmdir(self, path): 165 | node = self.node(path) 166 | if not isinstance(node, Collection): 167 | raise FuseOSError(ENOTDIR) 168 | if len(node.items()) > 0: 169 | raise FuseOSError(ENOTEMPTY) 170 | node.delete() 171 | 172 | def rename(self, old, new): 173 | old_node = self.node(old) 174 | new_dir, new_file = self.parent(new) 175 | new_node = new_dir.get(new_file) 176 | 177 | try: 178 | if new_node is None: 179 | # It's a move with a filename 180 | old_node.rename(new_dir, new_file) 181 | elif isinstance(new_node, Collection): 182 | # It's a move into a directory 183 | old_node.rename(new_node, old_node.name) 184 | else: 185 | # It's overwriting a file. 186 | # Don't allow this because it might be an editor doing 187 | # a rename to overwrite the file with a new version. 188 | # This would lose all handwritten notes associated with the file. 189 | raise FuseOSError(EEXIST) 190 | except IOError: 191 | # File conversion error 192 | traceback.print_exc() 193 | raise FuseOSError(EIO) 194 | 195 | def unlink(self, path): 196 | node = self.node(path) 197 | if isinstance(node, Collection): 198 | raise FuseOSError(EISDIR) 199 | node.delete() 200 | 201 | # 202 | # Reading directories and statting files 203 | # 204 | 205 | def opendir(self, path): 206 | return self.fds.new(self.node(path)) 207 | 208 | def releasedir(self, path, fd): 209 | self.fds.close(fd) 210 | 211 | def readdir(self, path, fd): 212 | node = self.fds.get(fd) 213 | if isinstance(node, Collection): 214 | yield "." 215 | yield ".." 216 | for file in node: 217 | yield file 218 | else: 219 | raise FuseOSError(ENOTDIR) 220 | 221 | def getattr(self, path, fd=None): 222 | if fd is None: 223 | node = self.node(path) 224 | else: 225 | node = self.fds.get(fd) 226 | 227 | mode = stat.S_IRUSR + stat.S_IRGRP + stat.S_IROTH 228 | if isinstance(node, Collection): 229 | mode += stat.S_IFDIR + stat.S_IXUSR + stat.S_IXGRP + stat.S_IXOTH 230 | else: 231 | mode += stat.S_IFREG 232 | mtime = node.mtime 233 | 234 | return { 235 | "st_mode": mode, "st_uid": os.getuid(), "st_gid": os.getgid(), 236 | "st_atime": mtime, "st_mtime": mtime, "st_ctime": mtime, 237 | "st_size": node.size } 238 | 239 | def statfs(self, path): 240 | # Just invent free space info (macOS finder seems to want them) 241 | bsize=512 242 | total=int(8192*1024*1024/bsize) 243 | return {"f_bsize": bsize, "f_blocks": total, "f_bavail": total, "f_bfree": total} 244 | 245 | # 246 | # chmod and chown - ignored (the default implementation raises an error - 247 | # this makes programs like cp complain) 248 | # 249 | 250 | def chmod(self, path, mode): 251 | pass 252 | 253 | def chown(self, path, uid, gid): 254 | pass 255 | 256 | # 257 | # Extended attributes: only 'bookmarked' supported, which sets whether the 258 | # document appears in the reMarkable bookmarks list 259 | # 260 | 261 | def listxattr(self, path): 262 | return ["user.bookmarked"] 263 | 264 | def getxattr(self, path, name, position=0): 265 | if name != "user.bookmarked": 266 | return b"" 267 | 268 | node = self.node(path) 269 | if node.pinned: 270 | return b"yes" 271 | else: 272 | return b"no" 273 | 274 | def setxattr(self, path, name, value, options, position=0): 275 | if name != "user.bookmarked": 276 | return 277 | 278 | if value == "yes" or value == "true" or value == "1": 279 | pinned = True 280 | elif value == "no" or value == "false" or value == "0": 281 | pinned = False 282 | else: 283 | raise FuseOSError(ENOTSUP) 284 | 285 | node = self.node(path) 286 | node.pinned = pinned 287 | node.save() 288 | 289 | def mount(mountpoint, documents, **kwargs): 290 | """Mount the FUSE filesystem. 291 | 292 | mountpoint - directory name of mount point 293 | documents - remarkable_fs.documents.DocumentRoot object""" 294 | 295 | FUSE(Remarkable(documents), mountpoint, nothreads=True, foreground=True, big_writes=True, max_write=1048576, **kwargs) 296 | -------------------------------------------------------------------------------- /remarkable_fs/rM2svg.py: -------------------------------------------------------------------------------- 1 | # This code copied from https://github.com/phil777/maxio 2 | # and licensed under the LGPL, version 3. 3 | 4 | import sys 5 | import struct 6 | import os.path 7 | import argparse 8 | from fpdf import FPDF 9 | 10 | 11 | __prog_name__ = "rM2svg" 12 | __version__ = "0.0.1beta" 13 | 14 | 15 | # Size 16 | x_width = 1404 17 | y_width = 1872 18 | 19 | # Mappings 20 | stroke_colour={ 21 | 0 : (0,0,0), 22 | 1 : (128,128,128), 23 | 2 : (255,255,255), 24 | } 25 | '''stroke_width={ 26 | 0x3ff00000 : 2, 27 | 0x40000000 : 4, 28 | 0x40080000 : 8, 29 | }''' 30 | 31 | 32 | def main(): 33 | parser = argparse.ArgumentParser(prog=__prog_name__) 34 | parser.add_argument("-i", 35 | "--input", 36 | help=".lines input file", 37 | required=True, 38 | metavar="FILENAME", 39 | #type=argparse.FileType('r') 40 | ) 41 | parser.add_argument("-p", 42 | "--pdf", 43 | help=".pdf input file", 44 | required=False, 45 | metavar="PDF") 46 | parser.add_argument("-o", 47 | "--output", 48 | help="prefix for output files", 49 | required=True, 50 | metavar="NAME", 51 | #type=argparse.FileType('w') 52 | ) 53 | parser.add_argument('--version', 54 | action='version', 55 | version='%(prog)s {version}'.format(version=__version__)) 56 | args = parser.parse_args() 57 | 58 | if not os.path.exists(args.input): 59 | parser.error('The file "{}" does not exist!'.format(args.input)) 60 | 61 | lines2cairo(args.input, args.output, args.pdf) 62 | 63 | 64 | def abort(msg): 65 | print(msg) 66 | sys.exit(1) 67 | 68 | class FPDFPlus(FPDF): 69 | """Adds alpha support to FPDF. 70 | 71 | Ported from http://www.fpdf.org/en/script/script74.php.""" 72 | 73 | def __init__(self, *args, **kwargs): 74 | super(FPDFPlus, self).__init__(*args, **kwargs) 75 | self.ext_gs_states = {} 76 | self.ext_gs_objs = {} 77 | self.next = 0 78 | 79 | if self.pdf_version < '1.4': 80 | self.pdf_version = '1.4' 81 | 82 | def set_alpha(self, alpha, blend_mode="Normal"): 83 | state = "/ca %.3f /CA %.3f /BM /%s" % (alpha, alpha, blend_mode) 84 | n = self.ext_gs_states.get(state) 85 | if n is None: 86 | n = self.next + 1 87 | self.next += 1 88 | self.ext_gs_states[state] = n 89 | self._out("/GS%d gs" % n) 90 | 91 | def _putresources(self): 92 | for (x, i) in self.ext_gs_states.items(): 93 | self._newobj() 94 | self.ext_gs_objs[x] = self.n 95 | self._out("<> endobj" % x) 96 | super(FPDFPlus, self)._putresources() 97 | 98 | def _putresourcedict(self): 99 | super(FPDFPlus, self)._putresourcedict() 100 | self._out("/ExtGState <<") 101 | for (x, i) in self.ext_gs_states.items(): 102 | self._out("/GS%d %d 0 R" % (i, self.ext_gs_objs[x])) 103 | self._out(">>") 104 | 105 | def lines2cairo(input_file, output_name, templates): 106 | # Read the file in memory. Consider optimising by reading chunks. 107 | #with open(input_file, 'rb') as f: 108 | # data = f.read() 109 | data = input_file.read() 110 | offset = 0 111 | 112 | # Is this a reMarkable .lines file? 113 | expected_header=b'reMarkable lines with selections and layers' 114 | if len(data) < len(expected_header) + 4: 115 | abort('File too short to be a valid file') 116 | 117 | fmt = '<{}sI'.format(len(expected_header)) 118 | header, npages = struct.unpack_from(fmt, data, offset); offset += struct.calcsize(fmt) 119 | if header != expected_header or npages < 1: 120 | abort('Not a valid reMarkable file: '.format(header, npages)) 121 | 122 | 123 | 124 | #pdfdoc = poppler.document_new_from_file("file://"+os.path.realpath(pdf_name),"") if pdf_name else None 125 | pdfdoc = None 126 | 127 | if pdfdoc: 128 | pdfpage = pdfdoc.get_page(0) 129 | pdfx,pdfy = pdfpage.get_size() 130 | print("page %.2f %.2f" % (pdfx,pdfy)) 131 | xfactor = pdfx/x_width 132 | yfactor = pdfy/y_width 133 | xfactor = yfactor = max(xfactor, yfactor) 134 | else: 135 | pdfy = 600.0 136 | pdfx = pdfy*x_width/y_width 137 | yfactor = pdfy/y_width 138 | xfactor = pdfx/x_width 139 | 140 | pdf = FPDFPlus(unit = 'pt', format=(pdfx, pdfy)) 141 | 142 | # Iterate through pages (There is at least one) 143 | for page in range(npages): 144 | pdf.add_page() 145 | 146 | template = None 147 | if templates: 148 | template = templates.pop(0) 149 | 150 | if template is not None: 151 | pdf.image(template, 0, 0, pdfx, pdfy) 152 | 153 | fmt = '