├── LICENSE ├── README.md ├── apply-image-transform.py ├── apply-svg-transform.py ├── is-transformed.py └── rm-dl-annotated.sh /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Owen Trueblood 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # rm-dl-annotated 2 | 3 | Export annotated PDFs from [ReMarkable tablets](https://remarkable.com/). 4 | 5 | I read lots of papers on my RM tablet. It's super cool to be able to scribble 6 | notes and highlight them, but later I want to go back and review the notes and 7 | unfortunately the interface on the actual RM sucks for that. I made this simple 8 | utility so I could scroll through the PDFs on my laptop and see my highlights 9 | and notes at a glance during review. 10 | 11 | It also works for notebooks. 12 | 13 | Tested with reMarkable tablet software version 2.3.0.16. 14 | 15 | ## Example 16 | 17 | ``` 18 | rm-dl-annotated.sh "/Super Cool Research Paper I Scribbled All Over" 19 | ``` 20 | 21 | Generates `./"Super Cool Research Paper (exported).pdf"` with the scribbles on top of the original PDF. 22 | 23 | ## Dependencies 24 | 25 | All these things need to be on your path, and you need to have given `rmapi` access to your ReMarkable Cloud account: 26 | 27 | * python 28 | * ImageMagick (`convert`) 29 | * pdfinfo (from poppler-utils) 30 | * pdfunite (from poppler-utils) 31 | * qpdf 32 | * [rmapi](https://github.com/juruen/rmapi) 33 | * [svgexport](https://github.com/shakiba/svgexport) 34 | * [rM2svg](https://github.com/delaere/maxio/blob/master/tools/rM2svg) 35 | 36 | If any of your PDFs have been cropped on your ReMarkable then you will also need: 37 | 38 | * pdftoppm (from poppler-utils) 39 | 40 | And the following Python libraries: 41 | 42 | * [opencv-python](https://pypi.org/project/opencv-python/) 43 | * [numpy](https://numpy.org/) 44 | 45 | As of this writing [the reHackable rM2svg](https://github.com/reHackable/maxio/blob/a0a9d8291bd034a0114919bbf334973bbdd6a218/tools/rM2svg) 46 | hasn't been updated to support new versions of the .lines file format for new 47 | version of the ReMarkable tablet OS, so I suggest using [the fork by delaere](https://github.com/delaere/maxio/blob/master/tools/rM2svg). 48 | 49 | ## Installation 50 | 51 | On Ubuntu: 52 | 53 | ``` 54 | sudo apt install imagemagick poppler-utils qpdf 55 | pip install opencv-python numpy 56 | ``` 57 | 58 | Follow the installation instructions on the project pages for 59 | [rmapi](https://github.com/juruen/rmapi), 60 | [rM2svg](https://github.com/delaere/maxio/blob/master/tools/rM2svg) 61 | (download the script and put it in a directory on your PATH like /usr/bin), and 62 | [svgexport](https://github.com/shakiba/svgexport). 63 | 64 | -------------------------------------------------------------------------------- /apply-image-transform.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import sys 4 | import os 5 | import json 6 | import cv2 7 | import numpy as np 8 | 9 | def load_content_file(content_filepath): 10 | if not os.path.isfile(content_filepath): 11 | raise Exception('Content file "{}" does not exist'.format(content_filepath)) 12 | 13 | with open(content_filepath, 'r') as content_file: 14 | return json.loads(content_file.read()) 15 | 16 | def transform_by_content_data(image, content_data): 17 | rows, cols, _ = image.shape 18 | transform = content_data['transform'] 19 | affine_matrix = np.float32([ 20 | [transform['m11'], transform['m21'], cols * transform['m31']], 21 | [transform['m12'], transform['m22'], rows * transform['m32']] 22 | ]) 23 | 24 | return cv2.warpAffine(image, affine_matrix, (cols, rows), None, 0, cv2.BORDER_CONSTANT, (255, 255, 255)) 25 | 26 | def load_image(image_filepath): 27 | if not os.path.isfile(image_filepath): 28 | raise Exception('Image file "{}" does not exist'.format(image_filepath)) 29 | 30 | return cv2.imread(image_filepath) 31 | 32 | def save_image(filename, image): 33 | cv2.imwrite(filename, image) 34 | 35 | def main(): 36 | if not len(sys.argv) == 4: 37 | print('Usage: {} image content_file'.format(sys.argv[0])) 38 | sys.exit(1) 39 | 40 | image = load_image(sys.argv[1]) 41 | content_data = load_content_file(sys.argv[2]) 42 | 43 | transformed_image = transform_by_content_data(image, content_data) 44 | 45 | save_image(sys.argv[3], transformed_image) 46 | 47 | main() 48 | -------------------------------------------------------------------------------- /apply-svg-transform.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import sys 4 | import os 5 | import json 6 | import xml.etree.ElementTree 7 | 8 | def load_content_file(content_filepath): 9 | if not os.path.isfile(content_filepath): 10 | raise Exception('Content file "{}" does not exist'.format(content_filepath)) 11 | 12 | with open(content_filepath, 'r') as content_file: 13 | return json.loads(content_file.read()) 14 | 15 | def transform_by_content_data(svg, content_data): 16 | root = svg.getroot() 17 | 18 | width = int(root.attrib['width']) 19 | height = int(root.attrib['height']) 20 | 21 | transform = content_data['transform'] 22 | affine_matrix = [ 23 | transform['m11'], transform['m12'], transform['m21'], 24 | transform['m22'], width * transform['m31'], height * transform['m32'], 25 | ] 26 | 27 | transform_group_element = xml.etree.ElementTree.Element('g') 28 | transform_group_element.attrib['transform'] = 'matrix({}, {}, {}, {}, {}, {})'.format(*affine_matrix) 29 | 30 | children_to_remove = [] 31 | for child in root: 32 | transform_group_element.append(child) 33 | children_to_remove.append(child) 34 | 35 | for child in children_to_remove: 36 | root.remove(child) 37 | 38 | root.append(transform_group_element) 39 | 40 | return svg 41 | 42 | def load_image(svg_filepath): 43 | if not os.path.isfile(svg_filepath): 44 | raise Exception('Image file "{}" does not exist'.format(svg_filepath)) 45 | 46 | xml.etree.ElementTree.register_namespace('', 'http://www.w3.org/2000/svg') 47 | return xml.etree.ElementTree.parse(svg_filepath) 48 | 49 | def save_image(filename, svg): 50 | svg.write(filename) 51 | 52 | def main(): 53 | if not len(sys.argv) == 4: 54 | print('Usage: {} svg_image content_file'.format(sys.argv[0])) 55 | sys.exit(1) 56 | 57 | image = load_image(sys.argv[1]) 58 | content_data = load_content_file(sys.argv[2]) 59 | 60 | transformed_image = transform_by_content_data(image, content_data) 61 | 62 | save_image(sys.argv[3], transformed_image) 63 | 64 | main() 65 | -------------------------------------------------------------------------------- /is-transformed.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import sys 4 | import json 5 | 6 | unit_transform = { 7 | 'm11': 1, 8 | 'm12': 0, 9 | 'm13': 0, 10 | 'm21': 0, 11 | 'm22': 1, 12 | 'm23': 0, 13 | 'm31': 0, 14 | 'm32': 0, 15 | 'm33': 1, 16 | } 17 | 18 | def main(): 19 | if not len(sys.argv) == 2: 20 | print('Usage: {} contentfile'.format(sys.argv[0])) 21 | sys.exit(1) 22 | 23 | content_filepath = sys.argv[1] 24 | 25 | with open(content_filepath, 'r') as content_file: 26 | content_data = json.loads(content_file.read()) 27 | transform = content_data['transform'] 28 | 29 | for cellname in unit_transform: 30 | if not unit_transform[cellname] == transform[cellname]: 31 | print('Yes') 32 | sys.exit(0) 33 | 34 | print('No') 35 | sys.exit(0) 36 | 37 | # Something went wrong, don't know what 38 | sys.exit(1) 39 | 40 | main() 41 | -------------------------------------------------------------------------------- /rm-dl-annotated.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Example: rm-dl-annotated.sh "/Super Cool Research Paper I Scribbled All Over" 4 | # Generates: ./"Super Cool Research Paper (annotated).pdf" with the scribbles on top of the original PDF 5 | 6 | # Dependencies: 7 | # * rmapi https://github.com/juruen/rmapi 8 | # * rM2svg https://github.com/reHackable/maxio/blob/master/tools/rM2svg 9 | # * svgexport https://github.com/shakiba/svgexport 10 | # * pdfinfo 11 | # * pdfunite 12 | # * qpdf 13 | # * pdftoppm (if you use cropping) 14 | 15 | set -o errexit 16 | 17 | # ReMarkable's screen size in pixels 18 | RM_WIDTH=1404 19 | RM_HEIGHT=1872 20 | 21 | function print_help() { 22 | echo "rm-dl-annotated.sh [-v] [--help | -h] path/to/cloud/PDF" 23 | echo "where: 24 | -h or --help show this help text 25 | --keep keep work directory (for debugging) 26 | -v verbose mode" 27 | } 28 | 29 | function die_with_usage() { 30 | print_help 31 | exit 1 32 | } 33 | 34 | # Check arguments 35 | 36 | VERBOSE= 37 | KEEP_WORK= 38 | REMOTE_PATH= 39 | 40 | while test $# -gt 0 41 | do 42 | case "$1" in 43 | -v) VERBOSE=yes 44 | ;; 45 | --keep) KEEP_WORK=yes 46 | ;; 47 | -h) print_help 48 | exit 0 49 | ;; 50 | --help) print_help 51 | exit 0 52 | ;; 53 | *) REMOTE_PATH="$1" 54 | ;; 55 | esac 56 | shift 57 | done 58 | 59 | if [ -z "$REMOTE_PATH" ]; then 60 | die_with_usage 61 | exit 1 62 | fi 63 | 64 | # Do our work in a temporary directory 65 | 66 | # https://stackoverflow.com/a/246128 67 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 68 | WORK_DIR=$(mktemp -d) 69 | OBJECT_NAME=$(basename "$REMOTE_PATH") 70 | 71 | pushd "$WORK_DIR" >/dev/null 72 | 73 | if [ "$VERBOSE" = "yes" ] || [ "$KEEP_WORK" = "yes" ]; then 74 | echo "Created temporary directory \"$WORK_DIR\"" 75 | fi 76 | 77 | # Download the given document using the ReMarkable Cloud API 78 | 79 | rmapi get "$REMOTE_PATH" >/dev/null 80 | unzip "$OBJECT_NAME.zip" >/dev/null 81 | 82 | UUID=$(basename "$(ls ./*.content)" .content) 83 | 84 | if [ ! -d "./$UUID" ]; then 85 | echo "Document is not annotated. Exiting." 86 | 87 | if [ -z "$KEEP_WORK" ]; then 88 | rm -r "$WORK_DIR" 89 | fi 90 | 91 | exit 0 92 | fi 93 | 94 | CONTENT_FILE=$WORK_DIR/$UUID.content 95 | 96 | IS_NOTEBOOK= 97 | if [ ! -f "$UUID".pdf ]; then 98 | IS_NOTEBOOK=yes 99 | fi 100 | 101 | # Check if the document has been cropped 102 | IS_TRANSFORMED=false 103 | if [ "$("$SCRIPT_DIR"/is-transformed.py "$CONTENT_FILE")" = "Yes" ]; then 104 | IS_TRANSFORMED=true 105 | fi 106 | 107 | # Convert the lines file containing our scribbles to SVGs and then a PDF 108 | 109 | OUT_WIDTH="$RM_WIDTH" 110 | OUT_HEIGHT="$RM_HEIGHT" 111 | 112 | if [ -z "$IS_NOTEBOOK" ]; then 113 | PDF_DIMS=$(pdfinfo "$UUID".pdf | grep "Page size" | grep -Eo '[-+]?[0-9]*\.?[0-9]+' | tr '\n' ' ') 114 | OUT_WIDTH=$(echo $PDF_DIMS | awk -v height=$RM_HEIGHT '{print $1 / $2 * height}') 115 | NUM_PAGES=$(pdfinfo "$UUID".pdf |grep Pages | awk '{print $2}') 116 | fi 117 | 118 | for i in $(seq 0 $NUM_PAGES ); do 119 | lines_file="$UUID/$i.rm" 120 | svg_file=$(basename "$lines_file" .rm).svg 121 | if test -f "./$lines_file"; then 122 | rM2svg --width=$OUT_WIDTH --coloured_annotations -i "./$lines_file" -o "./$svg_file" 123 | else 124 | cat <> "./$svg_file" 125 | 126 | 127 | 128 | EOT 129 | fi 130 | done 131 | 132 | if [ $IS_TRANSFORMED = true ]; then 133 | echo "Document is cropped." 134 | echo "Applying crop to SVGs..." 135 | fi 136 | 137 | PAGES=() 138 | for svg in ./*.svg; do 139 | if [ $IS_TRANSFORMED = true ]; then 140 | # Apply the transformation (crop) to the SVGs so the annotations end up in the right places 141 | "$SCRIPT_DIR/apply-svg-transform.py" "$svg" "$UUID.content" "$svg" 142 | fi 143 | 144 | # Convert SVG to PDF 145 | PAGE_NAME=$(basename "$svg" .svg) 146 | 147 | svgexport "$PAGE_NAME".svg "$PAGE_NAME".png 148 | convert "$PAGE_NAME".png "$PAGE_NAME".pdf 149 | 150 | PAGES+=("$PAGE_NAME".pdf) 151 | done 152 | 153 | # Sort the pages by number 154 | SORTED_PAGES=( $( printf "%s\n" "${PAGES[@]}" | sort -n ) ) 155 | 156 | PDF_ANNOTATIONS="$UUID"_annotations.pdf 157 | pdfunite "${SORTED_PAGES[@]}" "$PDF_ANNOTATIONS" 158 | 159 | # Transform (crop) the original PDF if necessary 160 | if [ -z "$IS_NOTEBOOK" ] && [ $IS_TRANSFORMED = true ]; then 161 | echo "Applying crop to PDF..." 162 | 163 | IMAGE_DIR=pdf_images 164 | mkdir "$IMAGE_DIR" 165 | pdftoppm "$UUID".pdf "$IMAGE_DIR/$UUID" -png 166 | 167 | PAGES=() 168 | for pdf_image in "$IMAGE_DIR"/*.png; do 169 | "$SCRIPT_DIR/apply-image-transform.py" "$pdf_image" "$UUID".content "$pdf_image" 170 | PAGES+=("$pdf_image") 171 | done 172 | 173 | convert "${PAGES[@]}" "$UUID".pdf 174 | fi 175 | 176 | 177 | OUTPUT_PDF="$OBJECT_NAME (exported).pdf" 178 | if [ -z "$IS_NOTEBOOK" ]; then 179 | # Layer the annotations onto the original PDF 180 | qpdf "$UUID".pdf --overlay "$PDF_ANNOTATIONS" -- "$OUTPUT_PDF" 181 | else 182 | cp "$PDF_ANNOTATIONS" "$OUTPUT_PDF" 183 | fi 184 | 185 | popd >/dev/null 186 | cp "$WORK_DIR"/"$OUTPUT_PDF" . 187 | 188 | if [ -z "$KEEP_WORK" ]; then 189 | rm -r "$WORK_DIR" 190 | fi 191 | 192 | echo Generated "\"$OUTPUT_PDF\"" 193 | --------------------------------------------------------------------------------