├── go.mod ├── .gitignore ├── version.txt ├── TODO.md ├── LICENSE ├── match_files.go ├── safe_filename.go ├── CHANGELOG.md ├── Makefile ├── download_pdf.go ├── do_download.go ├── download_rmdoc.go ├── do_list.go ├── read_files.go ├── main.go └── README.md /go.mod: -------------------------------------------------------------------------------- 1 | module kg4zow/rmweb 2 | 3 | go 1.21 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | ._* 3 | *.pdf 4 | out/ 5 | /rmweb 6 | -------------------------------------------------------------------------------- /version.txt: -------------------------------------------------------------------------------- 1 | 0.8 2 | 3 | The first line of this file is used as the "version number" in the resulting 4 | executable. The rest of the file (such as this comment) is ignored. 5 | -------------------------------------------------------------------------------- /TODO.md: -------------------------------------------------------------------------------- 1 | # TODO 2 | 3 | * Option to download `.rmdoc` files 4 | * only works with 3.10 and later, how to check? 5 | * if first request succeeds, check the output file to see if it's actually a ZIP file or not 6 | * download `.rmn` files? (download `.rmdoc` then convert) 7 | 8 | * Upload PDF, EPUB, and `.rmdoc` files 9 | * how to detect input file type 10 | * does golang have a file-type identifier module? 11 | * if not, write one? 12 | * require that each filename ends with `.pdf`, `.epub`, or `.rmdoc`? 13 | * option to specify parent folder (by UUID or name) 14 | * upload `.rmn` files (convert to `.rmdoc` first), for [drawj2d](https://drawj2d.sourceforge.io/) users) 15 | 16 | ## Requested by others 17 | 18 | * Match: all files in a given folder 19 | * [requested](https://old.reddit.com/r/RemarkableTablet/comments/1e2ea01/bulk_exporting_documents/ldgge6f/) 20 | 21 | * Option to skip existing files, rather than overwriting or renaming 22 | * [requested](https://old.reddit.com/r/RemarkableTablet/comments/1e2ea01/bulk_exporting_documents/ldgge6f/) 23 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 John Simpson 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /match_files.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/match_files.go 4 | // John Simpson 2023-12-22 5 | 6 | package main 7 | 8 | import ( 9 | "fmt" 10 | "regexp" 11 | "strings" 12 | ) 13 | 14 | /////////////////////////////////////////////////////////////////////////////// 15 | // 16 | // Return a list of UUIDs which match a given pattern 17 | // - if the pattern is a UUID, and that UUID exists, return that UUID 18 | // - otherwise match against the files' "find_by" value 19 | 20 | func match_files( the_files map[string]DocInfo , look_for string ) []string { 21 | 22 | if flag_debug { 23 | fmt.Printf( "match_files: looking for '%s'\n" , look_for ) 24 | } 25 | 26 | rv := make( []string , 0 , len( the_files ) ) 27 | 28 | //////////////////////////////////////// 29 | // If we're looking for a UUID, either it exists or it doesnt. 30 | 31 | re_uuid := "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$" 32 | is_uuid := regexp.MustCompile( re_uuid ) 33 | 34 | if is_uuid.Match( []byte( look_for ) ) { 35 | if v,present := the_files[look_for] ; present { 36 | if ! v.folder { 37 | rv = append( rv , look_for ) 38 | 39 | if flag_debug { 40 | fmt.Printf( "match_files: found by UUID '%s'\n" , look_for ) 41 | } 42 | } 43 | } 44 | } else { 45 | 46 | //////////////////////////////////////// 47 | // Otherwise, search for matching strings in the_files[].find_by 48 | 49 | for k,_ := range the_files { 50 | if strings.Contains( the_files[k].find_by , look_for ) { 51 | rv = append( rv , k ) 52 | 53 | if flag_debug { 54 | fmt.Printf( "match_files: found by name '%s' '%s'\n" , 55 | k , the_files[k].full_name ) 56 | } 57 | } 58 | } 59 | } 60 | 61 | //////////////////////////////////////// 62 | // Done 63 | 64 | if flag_debug { 65 | fmt.Printf( "match_files: returning %d items\n" , len( rv ) ) 66 | } 67 | 68 | return rv 69 | } 70 | -------------------------------------------------------------------------------- /safe_filename.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/safe_filename.go 4 | // John Simpson 2023-12-18 5 | // 6 | // Given a filename, return a possibly modified filename which doesn't 7 | // already exist. 8 | 9 | package main 10 | 11 | import ( 12 | "fmt" 13 | "log" 14 | "os" 15 | "strings" 16 | ) 17 | 18 | 19 | /////////////////////////////////////////////////////////////////////////////// 20 | // 21 | // Does a file exist or not? 22 | 23 | func file_exists( name string ) bool { 24 | _,err := os.Stat( name ) 25 | if err == nil { 26 | return true 27 | } 28 | if os.IsNotExist( err ) { 29 | return false 30 | } 31 | 32 | log.Fatal( fmt.Sprintf( "file_exists('%s'): %v\n" , name , err ) ) 33 | os.Exit( 1 ) 34 | 35 | return false 36 | } 37 | 38 | /////////////////////////////////////////////////////////////////////////////// 39 | // 40 | // Return a possibly modified filename which doesn't already exist 41 | 42 | func safe_filename( name string ) string { 43 | 44 | if flag_debug { 45 | fmt.Printf( "safe_filename('%s') starting\n" , name ) 46 | } 47 | 48 | rv := name 49 | 50 | //////////////////////////////////////// 51 | // Find extension in filename 52 | 53 | base := rv 54 | ext := "" 55 | 56 | dotp := strings.LastIndex( rv , "." ) 57 | if dotp >= 0 { 58 | base = rv[:dotp] 59 | ext = rv[dotp:] 60 | } 61 | 62 | if flag_debug { 63 | fmt.Printf( "safe_filename() base='%s' ext='%s'\n" , base , ext ) 64 | } 65 | 66 | //////////////////////////////////////// 67 | // Try numbers until we find one which doesn't exist yet 68 | 69 | if file_exists( rv ) { 70 | n := 1 71 | x := fmt.Sprintf( "%s-%d%s" , base , n , ext ) 72 | 73 | if flag_debug { 74 | fmt.Printf( "safe_filename() x='%s'\n" , x ) 75 | } 76 | 77 | for file_exists( x ) { 78 | n ++ 79 | x = fmt.Sprintf( "%s-%d%s" , base , n , ext ) 80 | 81 | if flag_debug { 82 | fmt.Printf( "safe_filename() x='%s'\n" , x ) 83 | } 84 | } 85 | 86 | rv = x 87 | } 88 | 89 | //////////////////////////////////////// 90 | // Return what we found 91 | 92 | if flag_debug { 93 | fmt.Printf( "safe_filename('%s') returning '%s'\n" , name , rv ) 94 | } 95 | 96 | return rv 97 | } 98 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # CHANGELOG 2 | 3 | ## v0.8 - 2024-09-23 4 | 5 | Handle directory list JSON not containing file size or page count. 6 | (Looks like reMarkable removed this between 3.10 and 3.14 software.) 7 | https://github.com/kg4zow/rmweb/issues/1 8 | 9 | * `read_files()` check for JSON keys before trying to read them. 10 | * `do_list()` Dondt show size or page count if the directory list JSON 11 | didn't contain them. 12 | 13 | ## v0.7 - 2024-09-07 14 | 15 | Added "download rmdoc" functionality 16 | 17 | * added download_rmdoc() 18 | * replaced '/', '\', and ':' in visible names, with underscores 19 | * moved PassThru() (prints byte counts while reading data from a file) 20 | from download_pdf.go to do_download.go, so other download_xxx() can 21 | use it 22 | * included '-D' in usage() message 23 | 24 | ## v0.06 - 2023-12-22 25 | 26 | * Added `download` command 27 | * by UUID or by substring match in "VissibleName" 28 | * handles multiple substring match patterns 29 | * same "-1" handling for duplicate output files 30 | * Added `-c` option to "flatten" folder structure 31 | * Added "pattern searching" (UUID or filename substring) to `list` command 32 | * Deprecated `backup` command, use `download` with no pattern instead 33 | * Updated `list` command to do the same UUID/filename search as `download` 34 | * Added `-I` option to set tablet IP address 35 | * only useful for tablets which have been "hacked" to make the web 36 | interface available over wifi or other interfaces 37 | 38 | ## v0.05 - 2023-12-19 39 | 40 | * Set things up to automate "publishing" new binaries to Keybase 41 | * `Makefile`: added `push` target to "publish" new executables 42 | * Handle duplicate output filenames by adding "-1", "-2", etc. 43 | * added `-f` option to skip this and overwrite existing files 44 | * `download_pdf()` now creates directories as needed 45 | * removed `download_backup_all.go` (forgot to do this in v0.03) 46 | * Updated `README.md` 47 | * updated info about where/how to download 48 | * updated examples with new `list` output format 49 | 50 | ## v0.04 - 2023-12-18 51 | 52 | Re-thinking how the executables are distributed. 53 | 54 | * Started a new git repo to remove the executables from the repo. 55 | * Changed targets from `out/GOOS-GOARCH/rmweb` to `out/rmweb-GOOS-GOARCH` 56 | * Updated format of `version` message to show the new executable names. 57 | 58 | Executables can be downloaded from: 59 | 60 | * /keybase/public/jms1/rmweb/ 61 | * https://jms1.pub/rmweb/ 62 | 63 | ## v0.03 - 2023-12-18 64 | 65 | * Added `TODO.md` 66 | * Renamed `download` command to `backup` 67 | * Changed `-V` option to `version` command 68 | * Updated `list`: add file size, page count 69 | 70 | ## v0.02 - 2023-12-17 71 | 72 | * Updated `README.md` file 73 | * Fix program name in all files 74 | 75 | ## v0.01 - 2023-12-17 76 | 77 | * Initial commit, seems to be working 78 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | ######################################## 2 | # Filename of the final executable 3 | 4 | NAME := rmweb 5 | 6 | ######################################## 7 | # Pre-calculate values used in 'version' output so that if building multiple 8 | # arches, they all have the same version/time/etc. 9 | 10 | NOW := $(shell date -u +%Y-%m-%dT%H:%M:%SZ ) 11 | VERSION := $(shell head -1 version.txt ) 12 | HASH := $(shell git rev-parse --short HEAD 2>/dev/null || true ) 13 | DESC := $(shell git describe --dirty --broken --long 2>/dev/null || true ) 14 | 15 | ######################################## 16 | # Other automatic variables 17 | 18 | SOURCES := $(shell find . -name "*.go" ) 19 | MYOS := $(shell go env GOOS ) 20 | MYARCH := $(shell go env GOARCH ) 21 | 22 | ######################################## 23 | # Where binaries will be published 24 | 25 | PUBDIR := "/keybase/public/jms1/rmweb" 26 | 27 | ######################################## 28 | # Which OS/ARCH combinations will be built by 'make all' 29 | # - run 'go tool dist list' to see all available combinations 30 | 31 | ALL_ARCHES := darwin/amd64 darwin/arm64 \ 32 | linux/386 linux/amd64 \ 33 | windows/386 windows/amd64 34 | 35 | ############################################################################### 36 | # 37 | # First/default target: build for *this* machine's OS/ARCH 38 | 39 | $(NAME): out/$(NAME)-$(MYOS)-$(MYARCH) 40 | ln -sf "out/$(NAME)-$(MYOS)-$(MYARCH)" "$(NAME)" 41 | 42 | ######################################## 43 | # Build OS-ARCH/NAME for all combinations listed in ARCHES above 44 | # - if OS is "windows", add ".exe" to the name 45 | 46 | all: $(foreach A,$(ALL_ARCHES),out/$(NAME)-$(subst /,-,$(A))$(if $(A:windows%=),,.exe)) 47 | ln -sf "out/$(NAME)-$(MYOS)-$(MYARCH)" "$(NAME)" 48 | 49 | ######################################## 50 | # Remove all previously compiled binaries and symlinks 51 | 52 | clean: 53 | rm -rf $(NAME) out 54 | 55 | ############################################################################### 56 | # 57 | # How to build OS-ARCH/NAME for any OS/ARCH combination 58 | 59 | out/$(NAME)-%: go.mod Makefile version.txt $(SOURCES) 60 | GOOS=$(shell echo "$@" | awk -F '[\-/]' '{print $$3}' ) \ 61 | GOARCH=$(shell echo "$@" | awk -F '[\-\./]' '{print $$4}' ) \ 62 | go build -o $@ \ 63 | -ldflags "-X main.prog_name=$(NAME) \ 64 | -X main.prog_version=$(VERSION) \ 65 | -X main.prog_date=$(NOW) \ 66 | -X main.prog_hash=$(HASH) \ 67 | -X main.prog_desc=$(DESC)" 68 | 69 | ######################################## 70 | # Specific rule for reMarkable 2 tablet ... linux/arm with GOARM=7 71 | # - add "linux/arm" to ALL_ARCHES to build this 72 | 73 | out/$(NAME)-linux-arm: go.mod Makefile version.txt $(SOURCES) 74 | GOOS=linux GOARCH=arm GOARM=7 \ 75 | go build -o $@ \ 76 | -ldflags "-X main.prog_name=$(NAME) \ 77 | -X main.prog_version=$(VERSION) \ 78 | -X main.prog_date=$(NOW) \ 79 | -X main.prog_hash=$(HASH) \ 80 | -X main.prog_desc=$(DESC)" 81 | 82 | ############################################################################### 83 | # 84 | # Push (publish) executables to Keybase (old) 85 | 86 | push: all 87 | TAG="v$$( head -1 version.txt )" ; \ 88 | if [[ -e $(PUBDIR)/$$TAG ]] ; \ 89 | then \ 90 | echo "ERROR: $(PUBDIR)/$$TAG already exists" ; \ 91 | exit 1 ; \ 92 | else \ 93 | mkdir $(PUBDIR)/$$TAG && \ 94 | cp -v out/* $(PUBDIR)/$$TAG/ && \ 95 | chmod -x $(PUBDIR)/$$TAG/* && \ 96 | ( cd $(PUBDIR)/ && ./.reindex ) ; \ 97 | fi 98 | -------------------------------------------------------------------------------- /download_pdf.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/download_pdf.go 4 | // John Simpson 2023-12-17 5 | // 6 | // Download a PDF file from a reMarkable tablet 7 | 8 | package main 9 | 10 | import ( 11 | "fmt" 12 | "io" 13 | "io/fs" 14 | "net/http" 15 | "os" 16 | ) 17 | 18 | /////////////////////////////////////////////////////////////////////////////// 19 | // 20 | // Download a PDF file 21 | 22 | func download_pdf( uuid string , localfile string ) { 23 | 24 | //////////////////////////////////////////////////////////// 25 | // If the output filename contains any directory names, 26 | // make sure any necessary directories exist. 27 | 28 | for n := 1 ; n < len( localfile ) ; n ++ { 29 | if localfile[n] == '/' { 30 | dir := localfile[:n] 31 | 32 | if flag_debug { 33 | fmt.Printf( "checking dir='%s'\n" , dir ) 34 | } 35 | 36 | //////////////////////////////////////// 37 | // Check the directory 38 | 39 | s,err := os.Stat( dir ) 40 | if os.IsNotExist( err ) { 41 | //////////////////////////////////////// 42 | // doesn't exist yet - create it 43 | 44 | fmt.Printf( "Creating '%s' ..." , dir ) 45 | 46 | err := os.Mkdir( dir , 0755 ) 47 | if err != nil { 48 | fmt.Printf( "ERROR: %v\n" , err ) 49 | os.Exit( 1 ) 50 | } 51 | 52 | fmt.Println( " ok" ) 53 | 54 | } else if err != nil { 55 | //////////////////////////////////////// 56 | // os.Stat() had some other error 57 | 58 | fmt.Printf( "ERROR: os.Stat('%s'): %v\n" , dir , err ) 59 | os.Exit( 1 ) 60 | 61 | } else if ( ( s.Mode() & fs.ModeDir ) == 0 ) { 62 | //////////////////////////////////////// 63 | // exists and is not a directory 64 | 65 | fmt.Printf( "ERROR: '%s' exists and is not a directory\n" , dir ) 66 | os.Exit( 1 ) 67 | } 68 | 69 | } // if localfile[n] == '/' 70 | } // for n 71 | 72 | //////////////////////////////////////////////////////////// 73 | // Download the file 74 | 75 | fmt.Printf( "Downloading '%s' ... " , localfile ) 76 | 77 | //////////////////////////////////////// 78 | // Request the file 79 | 80 | url := "http://" + tablet_addr + "/download/" + uuid + "/placeholder" 81 | 82 | resp, err := http.Get( url ) 83 | if err != nil { 84 | fmt.Printf( "ERROR: %v" , err ) 85 | os.Exit( 1 ) 86 | } 87 | 88 | defer resp.Body.Close() 89 | 90 | //////////////////////////////////////// 91 | // Create output file 92 | 93 | dest, err := os.Create( localfile ) 94 | if err != nil { 95 | fmt.Printf( "ERROR: os.Create('%s'): %v" , localfile , err ) 96 | os.Exit( 1 ) 97 | } 98 | 99 | defer dest.Close() 100 | 101 | //////////////////////////////////////// 102 | // Copy the output to the file 103 | 104 | var src io.Reader = &PassThru{ Reader: resp.Body } 105 | 106 | total, err := io.Copy( dest , src ) 107 | if err != nil { 108 | fmt.Printf( "ERROR: os.Copy(): %v" , err ) 109 | os.Exit( 1 ) 110 | } 111 | 112 | //////////////////////////////////////// 113 | // done 114 | 115 | fmt.Printf( "%d ... ok\n" , total ) 116 | } 117 | -------------------------------------------------------------------------------- /do_download.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/do_download.go 4 | // John Simpson 2023-12-22 5 | 6 | package main 7 | 8 | import ( 9 | "fmt" 10 | "io" 11 | "os" 12 | "sort" 13 | "strings" 14 | ) 15 | 16 | /////////////////////////////////////////////////////////////////////////////// 17 | // 18 | // Passthru wrapper for io.Reader, prints total bytes while reading 19 | // used by download_xxx() functions 20 | 21 | type PassThru struct { 22 | io.Reader 23 | total int64 24 | } 25 | 26 | func (pt *PassThru) Read( p []byte ) ( int , error ) { 27 | n, err := pt.Reader.Read( p ) 28 | pt.total += int64( n ) 29 | 30 | if err == nil { 31 | x := fmt.Sprintf( "%d" , pt.total ) 32 | b := fmt.Sprintf( strings.Repeat( "\b" , len( x ) ) ) 33 | 34 | fmt.Print( x , b ) 35 | } 36 | 37 | return n , err 38 | } 39 | 40 | /////////////////////////////////////////////////////////////////////////////// 41 | // 42 | // Select and download one or more files 43 | 44 | func do_download( args ...string ) { 45 | 46 | //////////////////////////////////////// 47 | // Read the contents of the tablet 48 | 49 | the_files := read_files() 50 | 51 | //////////////////////////////////////////////////////////// 52 | // Figure out which UUIDs we'll be downloading 53 | 54 | get_uuids := make( map[string]bool , len( the_files ) ) 55 | 56 | //////////////////////////////////////// 57 | // If no pattern, include every UUID 58 | 59 | if len( args ) < 1 { 60 | for uuid,_ := range the_files { 61 | get_uuids[uuid] = true 62 | } 63 | 64 | if flag_debug { 65 | fmt.Printf( "do_list: including all UUIDs\n" ) 66 | } 67 | 68 | //////////////////////////////////////// 69 | // Otherwise, process each pattern 70 | 71 | } else { 72 | for _,pattern := range args { 73 | look_for := strings.ToLower( pattern ) 74 | 75 | //////////////////////////////////////// 76 | // Figure out which items match the current pattern 77 | 78 | this_match := match_files( the_files , look_for ) 79 | 80 | if len( this_match ) > 0 { 81 | for _,x := range this_match { 82 | get_uuids[x] = true 83 | } 84 | } else { 85 | fmt.Printf( "no matching items found for '%s'\n" , pattern ) 86 | } 87 | } 88 | } 89 | 90 | //////////////////////////////////////// 91 | // Make sure we found *something* 92 | 93 | if len( get_uuids ) < 1 { 94 | fmt.Println( "ERROR: nothing to search for" ) 95 | os.Exit( 1 ) 96 | } 97 | 98 | //////////////////////////////////////////////////////////// 99 | // Build and sort a list of filenames 100 | 101 | var get_names []string 102 | 103 | for uuid,_ := range get_uuids { 104 | get_names = append( get_names , uuid ) 105 | } 106 | 107 | sortby_name := func( a int , b int ) bool { 108 | a_name := the_files[get_names[a]].full_name 109 | b_name := the_files[get_names[b]].full_name 110 | return a_name < b_name 111 | } 112 | sort.SliceStable( get_names , sortby_name ) 113 | 114 | //////////////////////////////////////////////////////////// 115 | // Process entries 116 | 117 | for _,uuid := range get_names { 118 | if ! the_files[uuid].folder { 119 | 120 | //////////////////////////////////////// 121 | // Download the file 122 | 123 | lname_pdf := the_files[uuid].full_name + ".pdf" 124 | lname_rmdoc := the_files[uuid].full_name + ".rmdoc" 125 | 126 | if ! flag_overwrite { 127 | lname_pdf = safe_filename( lname_pdf ) 128 | lname_rmdoc = safe_filename( lname_rmdoc ) 129 | } 130 | 131 | if flag_dl_pdf { 132 | download_pdf( uuid , lname_pdf ) 133 | } 134 | 135 | if flag_dl_rmdoc { 136 | download_rmdoc( uuid , lname_rmdoc ) 137 | } 138 | } 139 | } 140 | 141 | } 142 | -------------------------------------------------------------------------------- /download_rmdoc.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/download_rmdoc.go 4 | // John Simpson 2024-09-07 5 | // 6 | // Download an RMDOC file from a reMarkable tablet 7 | 8 | package main 9 | 10 | import ( 11 | "fmt" 12 | "io" 13 | "io/fs" 14 | "net/http" 15 | "os" 16 | ) 17 | 18 | /////////////////////////////////////////////////////////////////////////////// 19 | // 20 | // Is a file a ZIP file or not? 21 | 22 | func is_zipfile( filename string ) bool { 23 | 24 | f, err := os.Open( filename ) 25 | if err != nil { 26 | fmt.Printf( "ERROR: can't read %s: %v\n" , filename , err ) 27 | os.Exit( 1 ) 28 | } 29 | defer f.Close() 30 | 31 | b := make( []byte , 4 ) 32 | nr, err := f.Read( b ) 33 | if err != nil { 34 | fmt.Sprintf( "ERROR: can't read 4 bytes from %s: %v\n" , filename , err ) 35 | os.Exit( 1 ) 36 | } 37 | 38 | if nr != 4 { 39 | fmt.Printf( "ERROR: expected 4 bytes from %s, only got %d" , filename , nr ) 40 | os.Exit( 1 ) 41 | } 42 | return ( b[0] == 0x50 && b[1] == 0x4b && b[2] == 0x03 && b[3] == 0x04 ) 43 | } 44 | 45 | /////////////////////////////////////////////////////////////////////////////// 46 | // 47 | // Download an RMDOC file 48 | 49 | var know_can_rmdoc bool = false 50 | var can_rmdoc bool = false 51 | 52 | func download_rmdoc( uuid string , localfile string ) { 53 | 54 | //////////////////////////////////////////////////////////// 55 | // If the output filename contains any directory names, 56 | // make sure any necessary directories exist. 57 | 58 | for n := 1 ; n < len( localfile ) ; n ++ { 59 | if localfile[n] == '/' { 60 | dir := localfile[:n] 61 | 62 | if flag_debug { 63 | fmt.Printf( "checking dir='%s'\n" , dir ) 64 | } 65 | 66 | //////////////////////////////////////// 67 | // Check the directory 68 | 69 | s,err := os.Stat( dir ) 70 | if os.IsNotExist( err ) { 71 | //////////////////////////////////////// 72 | // doesn't exist yet - create it 73 | 74 | fmt.Printf( "Creating '%s' ..." , dir ) 75 | 76 | err := os.Mkdir( dir , 0755 ) 77 | if err != nil { 78 | fmt.Printf( "ERROR: %v\n" , err ) 79 | os.Exit( 1 ) 80 | } 81 | 82 | fmt.Println( " ok" ) 83 | 84 | } else if err != nil { 85 | //////////////////////////////////////// 86 | // os.Stat() had some other error 87 | 88 | fmt.Printf( "ERROR: os.Stat('%s'): %v\n" , dir , err ) 89 | os.Exit( 1 ) 90 | 91 | } else if ( ( s.Mode() & fs.ModeDir ) == 0 ) { 92 | //////////////////////////////////////// 93 | // exists and is not a directory 94 | 95 | fmt.Printf( "ERROR: '%s' exists and is not a directory\n" , dir ) 96 | os.Exit( 1 ) 97 | } 98 | 99 | } // if localfile[n] == '/' 100 | } // for n 101 | 102 | //////////////////////////////////////////////////////////// 103 | // Download the file 104 | 105 | fmt.Printf( "Downloading '%s' ... " , localfile ) 106 | 107 | //////////////////////////////////////// 108 | // Request the file 109 | 110 | url := "http://" + tablet_addr + "/download/" + uuid + "/rmdoc" 111 | 112 | resp, err := http.Get( url ) 113 | if err != nil { 114 | fmt.Printf( "ERROR: can't download '%s': %v" , url , err ) 115 | os.Exit( 1 ) 116 | } 117 | 118 | //////////////////////////////////////// 119 | // Create output file 120 | 121 | dest, err := os.Create( localfile ) 122 | if err != nil { 123 | fmt.Printf( "ERROR: can't create '%s': %v\n" , localfile , err ) 124 | } 125 | 126 | //////////////////////////////////////// 127 | // Copy the output to the file 128 | 129 | var src io.Reader = &PassThru{ Reader: resp.Body } 130 | 131 | total, err := io.Copy( dest , src ) 132 | if err != nil { 133 | fmt.Printf( "ERROR: os.Copy(): %v" , err ) 134 | os.Exit( 1 ) 135 | } 136 | 137 | //////////////////////////////////////// 138 | // Finished with transfer 139 | 140 | dest.Close() 141 | resp.Body.Close() 142 | fmt.Printf( "%d ... ok\n" , total ) 143 | 144 | //////////////////////////////////////// 145 | // Check whether the output file is a ZIP file 146 | 147 | if ! know_can_rmdoc { 148 | know_can_rmdoc = true 149 | 150 | if is_zipfile( localfile ) { 151 | can_rmdoc = true 152 | } 153 | 154 | if ! can_rmdoc { 155 | //////////////////////////////////////// 156 | // Remove the file we just downloaded (it isn't an RMDOC file) 157 | 158 | os.Remove( localfile ) 159 | 160 | //////////////////////////////////////// 161 | // Tell the user what's going on 162 | 163 | if ! flag_dl_pdf { 164 | fmt.Println( "FATAL: this tablet's software cannot download .rmdoc files, cannot continue" ) 165 | os.Exit( 1 ) 166 | } else { 167 | fmt.Printf( "WARNING: this tablet's software cannot download .rmdoc files\n" ) 168 | flag_dl_rmdoc = false 169 | } 170 | } 171 | } 172 | 173 | } 174 | -------------------------------------------------------------------------------- /do_list.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/do_list.go 4 | // John Simpson 2023-12-17 5 | 6 | package main 7 | 8 | import ( 9 | "fmt" 10 | "os" 11 | "sort" 12 | "strconv" 13 | "strings" 14 | ) 15 | 16 | var list_size bool = false 17 | var list_pages bool = false 18 | 19 | /////////////////////////////////////////////////////////////////////////////// 20 | // 21 | // do_list 22 | 23 | func do_list( args ...string ) { 24 | 25 | the_files := read_files() 26 | 27 | //////////////////////////////////////////////////////////// 28 | // Select which UUIDs to show 29 | 30 | show_uuids := make( map[string]bool , len( the_files ) ) 31 | 32 | //////////////////////////////////////// 33 | // If no pattern, include every UUID 34 | 35 | if len( args ) < 1 { 36 | for uuid,_ := range the_files { 37 | show_uuids[uuid] = true 38 | } 39 | 40 | if flag_debug { 41 | fmt.Printf( "do_list: including all UUIDs\n" ) 42 | } 43 | 44 | //////////////////////////////////////// 45 | // Otherwise, build a list of matching UUIDs 46 | 47 | } else { 48 | for _,pattern := range args { 49 | if flag_debug { 50 | fmt.Printf( "do_list: pattern '%s'\n" ) 51 | } 52 | 53 | look_for := strings.ToLower( pattern ) 54 | 55 | //////////////////////////////////////// 56 | // Figure out which items match the current pattern 57 | 58 | this_match := match_files( the_files , look_for ) 59 | 60 | if len( this_match ) > 0 { 61 | for _,x := range this_match { 62 | show_uuids[x] = true 63 | 64 | if flag_debug { 65 | fmt.Printf( "do_list: found '%s' '%s'\n" , x , 66 | the_files[x].full_name ) 67 | } 68 | } 69 | } else { 70 | fmt.Printf( "no matching items found for '%s'\n" , pattern ) 71 | } 72 | } 73 | } 74 | 75 | //////////////////////////////////////// 76 | // Make sure we found *something* 77 | 78 | if len( show_uuids ) < 1 { 79 | fmt.Println( "ERROR: nothing to search for" ) 80 | os.Exit( 1 ) 81 | } 82 | 83 | //////////////////////////////////////////////////////////// 84 | // Build a list of filenames 85 | // - the keys in the_files are UUIDs 86 | 87 | var l_name int = 4 // length of "Name" header 88 | var l_size int = 4 // length of "Size" header 89 | var l_pages int = 5 // length of "Pages" header 90 | 91 | show_names := make( []string , 0 , len( show_uuids ) ) 92 | for uuid := range show_uuids { 93 | show_names = append( show_names , uuid ) 94 | 95 | //////////////////////////////////////// 96 | // Find the length of the longest full_name 97 | 98 | l := len( the_files[uuid].full_name ) 99 | if the_files[uuid].folder { 100 | l ++ 101 | } 102 | 103 | if l > l_name { 104 | l_name = l 105 | } 106 | 107 | //////////////////////////////////////// 108 | // Find the length of the longest "size" 109 | 110 | l = len( fmt.Sprintf( "%d" , the_files[uuid].size ) ) 111 | if l > l_size { 112 | l_size = l 113 | } 114 | 115 | //////////////////////////////////////// 116 | // Find the length of the longest page count 117 | 118 | l = len( fmt.Sprintf( "%d" , the_files[uuid].pages ) ) 119 | if l > l_pages { 120 | l_pages = l 121 | } 122 | 123 | } 124 | 125 | //////////////////////////////////////// 126 | // Sort the list by fullname 127 | 128 | sortby_name := func( a int , b int ) bool { 129 | a_name := the_files[show_names[a]].full_name 130 | b_name := the_files[show_names[b]].full_name 131 | return a_name < b_name 132 | } 133 | sort.SliceStable( show_names , sortby_name ) 134 | 135 | //////////////////////////////////////////////////////////// 136 | // Print entries 137 | 138 | fmt.Printf( "%-36s %*s %*s %s\n" , 139 | "UUID" , 140 | l_size , "Size" , 141 | l_pages , "Pages" , 142 | "Name" ) 143 | fmt.Printf( "%s %s %s %s\n" , 144 | strings.Repeat( "-" , 36 ) , 145 | strings.Repeat( "-" , l_size ) , 146 | strings.Repeat( "-" , l_pages ) , 147 | strings.Repeat( "-" , l_name ) ) 148 | 149 | for _,uuid := range show_names { 150 | if the_files[uuid].folder { 151 | fmt.Printf( "%-36s %*s %*s %s/\n" , 152 | uuid , 153 | l_size , "" , 154 | l_pages , "" , 155 | the_files[uuid].full_name ) 156 | 157 | } else { 158 | var d_size string = "" 159 | var d_pages string = "" 160 | 161 | if list_size { 162 | d_size = strconv.FormatInt( the_files[uuid].size , 10 ) 163 | } 164 | 165 | if list_pages { 166 | d_pages = strconv.FormatInt( the_files[uuid].pages , 10 ) 167 | } 168 | 169 | fmt.Printf( "%-36s %*s %*s %s\n" , 170 | uuid , 171 | l_size , d_size , 172 | l_pages , d_pages , 173 | the_files[uuid].full_name ) 174 | } 175 | } 176 | 177 | } 178 | -------------------------------------------------------------------------------- /read_files.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/read_files.go 4 | // John Simpson 2023-12-16 5 | // 6 | // List files on a reMarkable tablet 7 | 8 | package main 9 | 10 | import ( 11 | "bytes" 12 | "encoding/json" 13 | "fmt" 14 | "io" 15 | "log" 16 | "net/http" 17 | "strings" 18 | ) 19 | 20 | /////////////////////////////////////////////////////////////////////////////// 21 | // 22 | // Replace problematic characters in documents' visible names 23 | 24 | var name_cleaner = strings.NewReplacer( "/" , "_" , "\\" , "_" , ":" , "_" ) 25 | 26 | /////////////////////////////////////////////////////////////////////////////// 27 | // 28 | // read_files 29 | // 30 | // Send a series of "POST http://10.11.99.1/documents/" requests to retrieve 31 | // the documents and containers in the tablet. 32 | // 33 | // Returns a map of IDs pointing to DocInfo structures. 34 | 35 | func read_files() ( map[string]DocInfo ) { 36 | 37 | //////////////////////////////////////// 38 | // Start map to store file/dir info 39 | 40 | rv := make( map [string]DocInfo ) 41 | 42 | //////////////////////////////////////////////////////////// 43 | // Process directories until there are no more 44 | 45 | l_dirs := []string{ "" } 46 | for len( l_dirs ) > 0 { 47 | 48 | //////////////////////////////////////// 49 | // Get the first directory name from the array 50 | 51 | this_dir := l_dirs[0] 52 | l_dirs = l_dirs[1:] 53 | 54 | //////////////////////////////////////// 55 | // Request info about this directory 56 | 57 | url := "http://" + tablet_addr + "/documents" + this_dir + "/" 58 | content_type := "application/json" 59 | buf := bytes.NewBufferString( "" ) 60 | 61 | if flag_debug { 62 | fmt.Println( "/========================================" ) 63 | fmt.Println( "POST " + url ) 64 | } 65 | 66 | resp, err := http.Post( url , content_type , buf ) 67 | if err != nil { 68 | log.Fatal( err ) 69 | } 70 | 71 | defer resp.Body.Close() 72 | 73 | //////////////////////////////////////// 74 | // Read the response into memory 75 | 76 | resp_bytes,err := io.ReadAll( resp.Body ) 77 | if ( err != nil ) { 78 | log.Fatal( err ) 79 | } 80 | 81 | if flag_debug { 82 | fmt.Print( string( resp_bytes[:] ) ) 83 | fmt.Println( "\\========================================" ) 84 | } 85 | 86 | //////////////////////////////////////// 87 | // Parse the response 88 | 89 | var data []map[string]interface{} 90 | 91 | err = json.Unmarshal( resp_bytes , &data ) 92 | if err != nil { 93 | log.Fatal(err) 94 | } 95 | 96 | //////////////////////////////////////// 97 | // process items within response 98 | 99 | for _,v := range data { 100 | 101 | //////////////////////////////////////// 102 | // Get info about this item 103 | 104 | var size int64 105 | var pages int 106 | 107 | id := v["ID"].(string) 108 | parent := v["Parent"].(string) 109 | folder := bool( v["Type"].(string) == "CollectionType" ) 110 | vis_name := v["VissibleName"].(string) 111 | 112 | //////////////////////////////////////// 113 | // Convert all '/', '\', and ':' with underscores 114 | 115 | name := name_cleaner.Replace( vis_name ) 116 | 117 | //////////////////////////////////////// 118 | // Get size and page count from data 119 | 120 | if ! folder { 121 | if _,ok := v["sizeInBytes"] ; ok { 122 | fmt.Sscan( v["sizeInBytes"].(string) , &size ) 123 | list_size = true 124 | } 125 | 126 | if _,ok := v["pageCount"] ; ok { 127 | pages = int( v["pageCount"].(float64) ) 128 | list_pages = true 129 | } 130 | } 131 | 132 | if flag_debug { 133 | fmt.Printf( "%s %-5t %s\n" , id , folder , name ) 134 | } 135 | 136 | //////////////////////////////////////// 137 | // Build user-facing name for this item 138 | 139 | parent_name := "" 140 | if parent != "" { 141 | parent_name = rv[parent].full_name 142 | } 143 | 144 | full_name := name 145 | 146 | if ! flag_collapse { 147 | full_name = parent_name + "/" + name 148 | } 149 | 150 | if full_name[0] == '/' { 151 | full_name = full_name[1:] 152 | } 153 | 154 | //////////////////////////////////////// 155 | // Remember this item 156 | 157 | var f DocInfo 158 | 159 | f.id = id 160 | f.parent = parent 161 | f.folder = folder 162 | f.name = name 163 | f.full_name = full_name 164 | f.size = int64( size ) 165 | f.pages = int64( pages ) 166 | f.find_by = strings.ToLower( name ) 167 | 168 | rv[f.id] = f 169 | 170 | //////////////////////////////////////// 171 | // If this item is a folder, add it to the list 172 | // so it also gets scanned 173 | 174 | if folder { 175 | l_dirs = append( l_dirs , string( this_dir + "/" + id ) ) 176 | } 177 | 178 | } // for range data 179 | } // for len( l_dirs ) > 0 180 | 181 | //////////////////////////////////////// 182 | // Return the files and directories 183 | 184 | return rv 185 | } 186 | -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | /////////////////////////////////////////////////////////////////////////////// 2 | // 3 | // rmweb/main.go 4 | // John Simpson 2023-12-16 5 | 6 | package main 7 | 8 | import ( 9 | "flag" 10 | "fmt" 11 | "os" 12 | "runtime" 13 | ) 14 | 15 | /////////////////////////////////////////////////////////////////////////////// 16 | // 17 | 18 | var ( 19 | // Actual values will be filled by -X options in Makefile, these values 20 | // are here in case somebody uses 'go run .'. 21 | 22 | prog_name string = "rmweb" 23 | prog_version string = "(unset)" 24 | prog_date string = "(unset)" 25 | prog_hash string = "" 26 | prog_desc string = "" 27 | 28 | // Hard-coded, not set by 'make' 29 | 30 | prog_url string = "https://github.com/kg4zow/rmweb/" 31 | ) 32 | 33 | /////////////////////////////////////////////////////////////////////////////// 34 | // 35 | // Values will be set by command line options 36 | // 37 | // NOTE: if new 'flag_dl_xxx' items are needed to support new file types, 38 | // be sure to update the check at the end of download_rmdoc() to check for 39 | // the new file types. 40 | 41 | var flag_debug bool = false 42 | var flag_overwrite bool = false 43 | var flag_collapse bool = false 44 | var tablet_addr string = "10.11.99.1" 45 | var flag_dl_pdf bool = false // default is set in main() below 46 | var flag_dl_rmdoc bool = false // default is set in main() below 47 | 48 | //////////////////////////////////////// 49 | // All files and directories on the tablet 50 | 51 | type DocInfo struct { 52 | id string 53 | parent string 54 | folder bool 55 | name string 56 | full_name string 57 | size int64 58 | pages int64 59 | find_by string 60 | } 61 | 62 | /////////////////////////////////////////////////////////////////////////////// 63 | // 64 | // usage 65 | 66 | func usage( ) { 67 | 68 | msg := `%s [options] COMMAND [...] 69 | 70 | Download files from a reMarkable tablet. 71 | 72 | Commands 73 | 74 | list ___ List all files on tablet. 75 | download ___ Download one or more documents to PDF file(s). 76 | 77 | version Show the program's version info 78 | help Show this help message. 79 | 80 | Options 81 | 82 | -p Download PDF files. 83 | 84 | -d Download RMDOC files. This requires that the tablet have 85 | software version version 3.10 or later. 86 | 87 | -a Download all available file types. 88 | 89 | -c Collapse filenames, i.e. don't create any sub-directories. 90 | All files will be written to the current directory. 91 | 92 | -f Overwrite existing files. 93 | 94 | -I ___ Specify the tablet's IP address. Default is '10.11.99.1', 95 | which the tablet uses when connected via USB cable. Note that 96 | unless you've "hacked" your tablet, the web interface is not 97 | available via any interface other than the USB cable. 98 | 99 | -D Show debugging messages. 100 | 101 | -h Show this help message. 102 | 103 | If no file types are explicitly requested (i.e. no '-a', '-p' or '-d' 104 | options are used), the program will download PDF files only by default. 105 | 106 | Commands with "___" after them allow you to specify one or more patterns 107 | to search for. Only matching documents will be (listed, downloaded, etc.) 108 | If a UUID is specified, that *exact* document will be selected. Otherwise, 109 | all documents whose names (as seen in the tablet's UI) contain the pattern 110 | will be selected. 111 | 112 | ` 113 | 114 | fmt.Printf( msg , prog_name ) 115 | 116 | os.Exit( 0 ) 117 | } 118 | 119 | /////////////////////////////////////////////////////////////////////////////// 120 | // 121 | // Show version info 122 | 123 | func do_version( args ...string ) { 124 | fmt.Printf( "%s-%s-%s version %s\n" , 125 | prog_name , runtime.GOOS , runtime.GOARCH , prog_version ) 126 | 127 | if prog_desc != "" { 128 | fmt.Printf( "Built %s from %s\n" , prog_date , prog_desc ) 129 | } else if prog_hash != "" { 130 | fmt.Printf( "Built %s from commit %s\n" , prog_date , prog_hash ) 131 | } else { 132 | fmt.Printf( "Built %s\n" , prog_date ) 133 | } 134 | 135 | fmt.Println( prog_url ) 136 | } 137 | 138 | /////////////////////////////////////////////////////////////////////////////// 139 | // 140 | // Show deprecation messages 141 | 142 | func do_backup() { 143 | msg := `The 'backup' command has been deprecated. 144 | Please use 'download' instead. 145 | ` 146 | 147 | fmt.Print( msg ) 148 | os.Exit( 1 ) 149 | } 150 | 151 | /////////////////////////////////////////////////////////////////////////////// 152 | /////////////////////////////////////////////////////////////////////////////// 153 | /////////////////////////////////////////////////////////////////////////////// 154 | 155 | func main() { 156 | 157 | //////////////////////////////////////// 158 | // Parse command line options 159 | 160 | var helpme bool = false 161 | var want_pdf bool = false 162 | var want_rmdoc bool = false 163 | var want_all bool = false 164 | 165 | flag.Usage = usage 166 | flag.BoolVar ( &helpme , "h" , helpme , "" ) 167 | flag.BoolVar ( &flag_debug , "D" , flag_debug , "" ) 168 | flag.BoolVar ( &flag_overwrite , "f" , flag_overwrite , "" ) 169 | flag.BoolVar ( &flag_collapse , "c" , flag_collapse , "" ) 170 | flag.StringVar( &tablet_addr , "I" , tablet_addr , "" ) 171 | flag.BoolVar ( &want_pdf , "p" , want_pdf , "" ) 172 | flag.BoolVar ( &want_rmdoc , "d" , want_rmdoc , "" ) 173 | flag.BoolVar ( &want_all , "a" , want_all , "" ) 174 | flag.Parse() 175 | 176 | //////////////////////////////////////// 177 | // If they used '-h', show usage 178 | 179 | if ( helpme ) { 180 | usage() 181 | } 182 | 183 | //////////////////////////////////////// 184 | // Figure out which file type options were requested 185 | 186 | if want_all { 187 | flag_dl_pdf = true 188 | flag_dl_rmdoc = true 189 | } else if want_pdf || want_rmdoc { 190 | flag_dl_pdf = want_pdf 191 | flag_dl_rmdoc = want_rmdoc 192 | } else { 193 | flag_dl_pdf = true 194 | flag_dl_rmdoc = false 195 | } 196 | 197 | //////////////////////////////////////// 198 | // Figure out which command we're being asked to run 199 | 200 | if len( flag.Args() ) > 0 { 201 | switch flag.Args()[0] { 202 | case "help" : usage() 203 | case "version" : do_version() 204 | case "backup" : do_backup() 205 | case "list" : do_list ( flag.Args()[1:]... ) 206 | case "download" : do_download ( flag.Args()[1:]... ) 207 | default : usage() 208 | } 209 | } else { 210 | usage() 211 | } 212 | 213 | } 214 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 🛑 **"Cloud Archive" files** 2 | 3 | The reMarkable cloud has a feature, available to customers with a paid account, which lets you "archive" documents to the cloud. This means that the document's files are in the cloud, but only the documents *metadata* is stored on your tablet. 4 | 5 | At first glance, the web API's "list files" request doesn't seem to provide any way to tell if a document is "archived to the cloud" or not. I don't use the reMarkable cloud, so at the moment I have no way to *create* any "cloud archive" files to experiment with. 6 | 7 | In addition, attempting to *download* such a document via the web interface can cause the tablet's internal web server to lock up. 8 | 9 | If you have a paid cloud account, make sure all of your documents are on your tablet (i.e. not "archived to the cloud") before using this program. 10 | 11 | --- 12 | 13 | # `rmweb` 14 | 15 | John Simpson `` 2023-12-17 16 | 17 | Last updated 2024-09-12 18 | 19 | This program lists all documents on a reMarakble tablet, or downloads them all as PDF files, using the tablet's built-in web interface. 20 | 21 | I threw this together after reading [this question](https://www.reddit.com/r/RemarkableTablet/comments/18js4wo/any_way_to_transfer_all_my_files_to_an_ipad_app/) on Reddit. 22 | 23 | # Background Information 24 | 25 | I figured out the correct web requests by using [Wireshark](https://www.wireshark.org/) to sniff the web traffic between my computer and three different reMarkable tablets (both rM1 and rM2, running a mix of 3.8 and 3.9 software.) 26 | 27 | ### Go (or Golang) 28 | 29 | [Go](https://go.dev/) (or "Golang") is a programming language from Google. I used this language for a few reasons ... 30 | 31 | * It's compiled rather than interpreted, which means programs tend to start and run more quickly. 32 | 33 | * Users don't have to install any additional libraries or "runtime" packages in order to run the program. All they need is an executable file for the platform where they're going to run it. 34 | 35 | * The same code can be compiled for multiple platforms, without having to figure out how to install and configure cross-compiler toolchains. (If you don't know what this means, count yourself lucky that you've never had to think about it.) 36 | 37 | * Go is a relatively new language for me, so this was a way to "get some practice" with it. 38 | 39 | One of the ideas on my list is a more "general" command line utility, written in Go, which will be able to list, download, upload, and make backups of reMarkable tablets. I'm planning to make this other program use SSH instead of the web interface, however I was going to have to use the web API to download PDF files anyway, so when I get ready to write that other program, I'll be able to copy a few of the functions from this program. 40 | 41 | ### File Formats 42 | 43 | The program will download PDF files by default. These can be viewed and printed on a computer, but any pen strokes you may have written will be "burned into" the file, and cannot be edited if you upload them back to the tablet (or to a different tablet). 44 | 45 | The program *can* download `.rmdoc` files, which reMarkable calls "Archive files". Your computer can't do a whole lot with these files, but if you upload them to the tablet (or to a different tablet), they will be edit-able, just like the original document. 46 | 47 | Note that reMarkable software versions prior to 3.10 were not capable of downloading "Archive files". (There was a bug in 3.9 where the web interface had an option to download Archive files, but the option didn't work - the files it would download were actually PDF files.) 48 | 49 | Because of this. when the program tries to download the first `.rmdoc` file, it will check the first few bytes in the downloaded file to be sure it *is* a valid `.rmdoc` file. If not, it will stop trying to download `.rmdoc` files, and if no other file formats were requested, the program will stop. 50 | 51 | # Installing the Program 52 | 53 | ## Download 54 | 55 | I'm using Github's "releases" mechanism. There should be a "Latest release" section to the right, if you're not reading this through the Github web interface, click [here](https://github.com/kg4zow/rmweb/releases). 56 | 57 | Download the appropriate executable for the machine where you plan to run the program. Store it in a directory in your `PATH`, and make sure its permissions are set to be executable. I also recommend renaming it to `rmweb`. 58 | 59 | ## Compiling the Program 60 | 61 | If you want to compile the program from source ... 62 | 63 | * [Install Go](https://go.dev/doc/install). 64 | 65 | * Clone the source code. 66 | 67 | ``` 68 | $ cd ~/git/ 69 | $ git clone https://github.com/kg4zow/rmweb 70 | ``` 71 | 72 | I use `~/git/` as a container for the git repos I have cloned on my machines. Obviously feel free to clone the repo wherever it makes sense for your machine. 73 | 74 | * Run `make` in the cloned repo. 75 | 76 | ``` 77 | $ cd ~/git/rmweb/ 78 | $ make 79 | ``` 80 | 81 | This will build the correct binary for your computer, under the `out/` directory. It will also create `rmweb` in the current dirctory, as a symbolic link to that binary. 82 | 83 | Note that you could also run "`make all`" to build binaries for a list of architectures. (This is how I build the executables when creating a release.) The list of architectures is set in the "`ALL_ARCHES :=`" line in `Makefile`, and currently includes the following: 84 | 85 | * `darwin/amd64` (Apple Intel 64-bit) 86 | * `darwin/arm64` (Apple M1/M2) 87 | * `linux/386` (Linux Intel 32-bit) 88 | * `linux/amd64` (Linux Intel 64-bit) 89 | * `windows/386` (windows 32-bit) 90 | * `windows/amd64` (windows 64-bit) 91 | 92 | You can see a list of all *possible* `GOOS`/`GOARCH` combinations in your installed copy of `go` by running "`go tool dist list`". 93 | 94 | # Set up the tablet 95 | 96 | ### Connect the tablet via USB cable 97 | 98 | The program uses the web interface to talk to the tablet, and reMarkable sets things up so that the web interface is only available over the USB interface. 99 | 100 | > ℹ️ If your tablet is "hacked" and the web interface is available over some other IP, you can use the "`-I`" (uppercase "i") option to specify that IP, like so: 101 | > 102 | > ``` 103 | > $ rmweb -I 192.0.2.7 list 104 | > ``` 105 | 106 | ### Make sure the tablet is not sleeping 107 | 108 | Fairly obvious. 109 | 110 | ### Make sure the web interface is enabled 111 | 112 | See "Settings → Storage" on the tablet. Note that you won't be able to turn the web interface on unless the tablet is connected to a computer via the USB cable. 113 | 114 | # Running the program 115 | 116 | The examples below assume that when you installed the executable, you renamed it to "`rmweb`". If not, you'll need to adjust the commands below to use whatever name you gave it. 117 | 118 | ## Available Options 119 | 120 | If you use the `-h` option, or run the program without specifying a command, it will show a quick explanation of how to run the program, along with lists of the commands and options it supports. 121 | 122 | ``` 123 | $ rmweb -h 124 | rmweb [options] COMMAND [...] 125 | 126 | Download files from a reMarkable tablet. 127 | 128 | Commands 129 | 130 | list ___ List all files on tablet. 131 | download ___ Download one or more documents to PDF file(s). 132 | 133 | version Show the program's version info 134 | help Show this help message. 135 | 136 | Options 137 | 138 | -p Download PDF files. 139 | 140 | -d Download RMDOC files. This requires that the tablet have 141 | software version version 3.10 or later. 142 | 143 | -a Download all available file types. 144 | 145 | -c Collapse filenames, i.e. don't create any sub-directories. 146 | All files will be written to the current directory. 147 | 148 | -f Overwrite existing files. 149 | 150 | -I ___ Specify the tablet's IP address. Default is '10.11.99.1', 151 | which the tablet uses when connected via USB cable. Note that 152 | unless you've "hacked" your tablet, the web interface is not 153 | available via any interface other than the USB cable. 154 | 155 | -D Show debugging messages. 156 | 157 | -h Show this help message. 158 | 159 | If no file types are explicitly requested (i.e. no '-a', '-p' or '-d' 160 | options are used), the program will download PDF files only by default. 161 | 162 | Commands with "___" after them allow you to specify one or more patterns 163 | to search for. Only matching documents will be (listed, downloaded, etc.) 164 | If a UUID is specified, that *exact* document will be selected. Otherwise, 165 | all documents whose names (as seen in the tablet's UI) contain the pattern 166 | will be selected. 167 | ``` 168 | 169 | This example is from v0.07. You may see different output if you're using a different version. 170 | 171 | ## Check the version 172 | 173 | The "`rmweb version`" command will show you the version number, along with information about the specific code it was built from in the git repo. 174 | 175 | ``` 176 | $ rmweb version 177 | rmweb-darwin-arm64 version 0.04 178 | Built 2023-12-18T17:44:17Z from v0.04-0-g17c101b 179 | https://github.com/kg4zow/rmweb/ 180 | ``` 181 | 182 | ## List Documents on the Tablet 183 | 184 | To list the files on the tablet, use "`rmweb list`". 185 | 186 | ``` 187 | $ rmweb list 188 | UUID Size Pages Name 189 | ------------------------------------ --------- ----- ------------------------------------------ 190 | 22e6d931-c205-4d86-b022-04b6a0527b67 Amateur Radio/ 191 | ec1989b1-bc41-40d7-a8f7-768b5be42bfd 38048 1 Amateur Radio/GMRS 192 | f7acd9d5-6c98-475d-b7bf-5cbb31501720 95486 1 Amateur Radio/Icom ID-5100 193 | 3314ef15-3b23-49dc-86f4-c89696963076 114720 2 Amateur Radio/Quansheng UV-K5(8) aka UV-K6 194 | 9d5198b0-ce76-4f58-ba0a-a1a058678695 277447 1 Amateur Radio/RTL-SDR 195 | f67c74d2-7d23-4587-95bd-7a6e8ebaed2c Ebooks/ 196 | 8efcdb0a-891f-40cb-901f-7c5bc0df7ad1 78249189 541 Ebooks/A City on Mars 197 | 9800e36b-5a0c-4eb7-aedc-72c1845d2816 5615420 264 Ebooks/Chokepoint Capitalism 198 | cc2135a3-08ea-4ae5-be77-8f455b039451 8025642 684 Ebooks/The Art of Unix Programming 199 | 702ef913-16a0-47b1-806e-1769f251b06b 4241185 306 Ebooks/The Cathedral & the Bazaar 200 | ... 201 | 24f6b013-054e-4706-9248-3d3d97d0d268 606562 8 Quick sheets 202 | 225a451f-61c9-4ffb-96ad-cbe7b7bb530c 987530 9 TODO 203 | 015deb02-0589-462a-bc98-3034d7d23628 Work/ 204 | 6f3d00ae-925b-48a0-83fb-963644bd7747 19915627 380 Work/2024 Daily 205 | ... 206 | ``` 207 | 208 | The UUID values are the internal identifiers for each document. The files within the tablet that make up each document, all have this as part of their filename. If you're curious, [this page](https://remarkable.jms1.info/info/filesystem.html) has a lot more detail. 209 | 210 | The size of each document is calculated by the tablet itself. It *looks like* it's the total of the sizes of all files which make up that document, including pen strokes and page thumbnail images. This is not the size you can expect the downloaded files to be. From what I've seen ... 211 | 212 | * PDF files can be anywhere from half to five times this size. 213 | * `.rmdoc` files can be anywhere from 0.5 to about 1.1 times this size. 214 | 215 | ### Patterns 216 | 217 | You can specify one or more patterns when listing or downloading files. If you do, any documents whose names match one or more of the patterns will be downloaded. 218 | 219 | ``` 220 | $ rmweb list quick unix 221 | UUID Size Pages Name 222 | ------------------------------------ ------- ----- ---------------------------------- 223 | cc2135a3-08ea-4ae5-be77-8f455b039451 8025642 684 Ebooks/The Art of Unix Programming 224 | 24f6b013-054e-4706-9248-3d3d97d0d268 606562 8 Quick sheets 225 | ``` 226 | 227 | #### Notes 228 | 229 | * If you need to specify a pattern containing spaces, you should quote it. For example ... 230 | 231 | ``` 232 | $ rmweb list quick brown fox 233 | ``` 234 | 235 | This command would list any documents with "quick" in the name, OR with "brown" in the name, OR with "fox" in the name. 236 | 237 | ``` 238 | $ rmweb list "quick brown fox" 239 | ``` 240 | 241 | This command would only download documents with the string "quick brown fox" in their name. 242 | 243 | * Filename searches are case-*in*sensitive, i.e. "`test`", "`Test`", "`TEST`", and "`tEsT`" all match each other. 244 | 245 | * If a pattern *looks like* a UUID (i.e. has the "8-4-4-4-12 hex digits" structure), the program will look it up by UUID rather than searching for it by filename. 246 | 247 | * If no patterns are specified (i.e. just "`rmweb list`" or "`rmweb download`" by itself), the program will list or download ALL documents. 248 | 249 | ## Download Documents 250 | 251 | To download one or more individual documents, first `cd` into the directory where you want to download the files. 252 | 253 | ``` 254 | $ mkdir ~/rm2-backup 255 | $ cd ~/rm2-backup/ 256 | ``` 257 | 258 | Then, run the appropriate "`rmweb download`" command. 259 | 260 | ### Selecting file formats 261 | 262 | * The `-p` option tells the program to download PDF files. 263 | 264 | ``` 265 | $ rmweb -p download 266 | ``` 267 | 268 | * The `-d` option tells the program to download RMDOC files. 269 | 270 | ``` 271 | $ rmweb -d download 272 | ``` 273 | 274 | * The `-a` option tells the program to download all available file formats. Currently (for v0.07) this means PDF and RMDOC, but if more file formats are added in the future, this option will include those new formats. 275 | 276 | ``` 277 | $ rmweb -a download 278 | ``` 279 | 280 | * If none of these options are used, the program will download just PDF files. 281 | 282 | ### Selecting which files to download 283 | 284 | The `rmweb download` command supports the same pattern-matching options as the `rmweb list` command. The notes in the Patterns section above, apply here as well. 285 | 286 | Each pattern can be either a UUID ... 287 | 288 | ``` 289 | $ rmweb download 24f6b013-054e-4706-9248-3d3d97d0d268 290 | Downloading 'Quick sheets.pdf' ... 2577411 ... ok 291 | ``` 292 | 293 | ... or a portion of the filename ... 294 | 295 | ``` 296 | $ rmweb -d download quick unix 297 | Creating 'Ebooks' ...ok 298 | Downloading 'Ebooks/The Art of Unix Programming.rmdoc' ... 5804725 ... ok 299 | Downloading 'Quick sheets.rmdoc' ... 347002 ... ok 300 | ``` 301 | 302 | ... or nothing at all, in which case it will download all documents. 303 | 304 | ``` 305 | $ rmweb -a download 306 | Creating 'Amateur Radio' ...ok 307 | Downloading 'Amateur Radio/GMRS.pdf' ... 27201 ... ok 308 | Downloading 'Amateur Radio/GMRS.rmdoc' ... 31160 ... ok 309 | Downloading 'Amateur Radio/Icom ID-5100.pdf' ... 48021 ... ok 310 | Downloading 'Amateur Radio/Icom ID-5100.rmdoc' ... 55268 ... ok 311 | Downloading 'Amateur Radio/Quansheng UV-K5(8) aka UV-K6.pdf' ... 69276 ... ok 312 | Downloading 'Amateur Radio/Quansheng UV-K5(8) aka UV-K6.rmdoc' ... 78329 ... ok 313 | Downloading 'Amateur Radio/RTL-SDR.pdf' ... 120784 ... ok 314 | Downloading 'Amateur Radio/RTL-SDR.rmdoc' ... 151231 ... ok 315 | Creating 'Ebooks' ...ok 316 | Downloading 'Ebooks/42.pdf' ... 53960677 ... ok 317 | Downloading 'Ebooks/42.rmdoc' ... 52533573 ... ok 318 | Downloading 'Ebooks/A City on Mars.pdf' ... 78018531 ... ok 319 | Downloading 'Ebooks/A City on Mars.rmdoc' ... 60687232 ... ok 320 | Downloading 'Ebooks/Chokepoint Capitalism.pdf' ... 4259286 ... ok 321 | Downloading 'Ebooks/Chokepoint Capitalism.rmdoc' ... 3732748 ... ok 322 | Downloading 'Ebooks/The Art of Unix Programming.pdf' ... 7856878 ... ok 323 | Downloading 'Ebooks/The Art of Unix Programming.rmdoc' ... 5804725 ... ok 324 | Downloading 'Ebooks/The Cathedral & the Bazaar.pdf' ... 1382013 ... ok 325 | Downloading 'Ebooks/The Cathedral & the Bazaar.rmdoc' ... 3962084 ... ok 326 | ... 327 | Downloading 'Quick sheets.pdf' ... 309012 ... ok 328 | Downloading 'Quick sheets.rmdoc' ... 347002 ... ok 329 | Downloading 'TODO.pdf' ... 502225 ... ok 330 | Downloading 'TODO.rmdoc' ... 569832 ... ok 331 | Creating 'Work' ...ok 332 | Downloading 'Work/2024 Daily.pdf' ... 12172136 ... ok 333 | Downloading 'Work/2024 Daily.rmdoc' ... 7485556 ... ok 334 | ... 335 | ``` 336 | 337 | ### Other notes 338 | 339 | * As each file downloads, the program shows a counter of how many bytes have been read from the tablet. For larger files you'll notice a time delay before this counter starts. This delay is when the tablet is building the file. 340 | 341 | [This video](https://jms1.pub/reMarkable/rmweb-time-difference.mov) shows this time delay. along with the difference between how long it takes to generate a PDF file (~25 seconds for this 380-page PDF-backed document) and how long it takes to build an `.rmdoc` file (~5 seconds for the same document). 342 | 343 | * If an output file already exists, the program will add "`-1`", "`-2`", etc. to the filename until it finds a name which doesn't already exist. If you want to *overwrite* existing files, use the "`-f`" option. 344 | 345 | ``` 346 | $ rmweb -f download quick 347 | Downloading 'Quick sheets.pdf' ... 2577411 ... ok 348 | ``` 349 | --------------------------------------------------------------------------------