├── LICENSE ├── dropboxsetup.py ├── README.md └── DropboxSync.py /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Dr. R. Matthew Ward, http://rmward.com 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /dropboxsetup.py: -------------------------------------------------------------------------------- 1 | import dropbox, os, webbrowser 2 | 3 | # A generic Dropbox module to create a token and login. 4 | # Michelle L. Gill, 2014/01/06 5 | 6 | # To use: 7 | # import dropboxsetup 8 | # sess, client = dropboxsetup.init(TOKEN_FILENAME, APP_KEY, APP_SECRET) 9 | # TOKEN_DIRECTORY can be set to store tokens in a folder, set to "Tokens" by default 10 | 11 | # requires a dropbox app key and secret, which can be created on dropbox's developer website 12 | # most of this script was shamelessly copied from https://gist.github.com/ctaloi/4156185 13 | 14 | def configure_token(sess, TOKEN_FILENAME): 15 | 16 | # read token if it exists, otherwise create a new one 17 | if os.path.exists(TOKEN_FILENAME): 18 | token_file = open(TOKEN_FILENAME) 19 | token_key, token_secret = token_file.read().split('|') 20 | token_file.close() 21 | sess.set_token(token_key,token_secret) 22 | else: 23 | first_access(sess, TOKEN_FILENAME) 24 | return 25 | 26 | 27 | def first_access(sess, TOKEN_FILENAME): 28 | request_token = sess.obtain_request_token() 29 | url = sess.build_authorize_url(request_token) 30 | 31 | # make the user sign in and authorize this token 32 | print "url:", url 33 | print "Please visit this website and press the 'Allow' button, then hit 'Enter' here." 34 | webbrowser.open(url) 35 | raw_input() 36 | 37 | # this will fail if the user didn't visit the above URL and hit 'Allow' 38 | access_token = sess.obtain_access_token(request_token) 39 | 40 | # save the token file 41 | token_file = open(TOKEN_FILENAME,'w') 42 | token_file.write("%s|%s" % (access_token.key,access_token.secret) ) 43 | token_file.close() 44 | return 45 | 46 | 47 | def init(TOKEN_FILENAME, APP_KEY, APP_SECRET, ACCESS_TYPE='app_folder'): 48 | # create the Dropbox session and client for file interaction 49 | sess = dropbox.session.DropboxSession(APP_KEY, APP_SECRET, ACCESS_TYPE) 50 | configure_token(sess, TOKEN_FILENAME) 51 | client = dropbox.client.DropboxClient(sess) 52 | 53 | return sess, client 54 | 55 | if __name__ == "__main__": 56 | token_filename = raw_input('Enter token filename:').strip() 57 | app_key = raw_input('Enter app key:').strip() 58 | app_secret = raw_input('Enter app secret:').strip() 59 | sess, client = init(token_filename, app_key, app_secret) -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # pythonista-dropbox-sync 2 | 3 | A Pythonista script and supporting module to sync [Pythonista](http://omz-software.com/pythonista/) files with [Dropbox](http://dropbox.com), based on and [original script](https://gist.github.com/mlgill/8311088) and [module](https://gist.github.com/mlgill/8311046) by Michelle Gill, but with my own improvements. 4 | 5 | ## Motivation 6 | 7 | Pythonista does not include any sync or backup functionality. This script makes a little hacky progress toward adding that functionality. 8 | 9 | In particular, I view this script as a first step in linking my scripts in Pythonista to projects in GitHub. In fact, it's how this script was developed. 10 | 11 | ## Installation and use 12 | 13 | Go to [Dropbox's developer page](https://www.dropbox.com/developers/apps/create) to register a new app. Be sure to: 14 | 15 | - Make it a "Dropbox API app". 16 | - Tell Dropbox that your app needs access to "Files and datastores". 17 | - Tell Dropbox that the app can be limited to its own folder. 18 | - Give it a unique name (for example, "Pythonista Sync YOURINITIALS"). 19 | - Note your app key. 20 | - Note your app secret. 21 | 22 | Otherwise, default setting should be fine. 23 | 24 | Make sure DropboxSync.py and dropboxsetup.py are stored in the same directory, and that it's the directory in Pythonista you want to sync—currently you'll probably want this to be Pythonista's root directory. Then open DropboxSync.py and run it. 25 | 26 | On first run, DropboxSync will request that you enter a name for the dropbox token file (I recommend that it start with "." so that it's hidden on many systems). It will then request that you enter your app key and app secret. The app then reads the current Pythonista directory and subdirectories, the target Dropbox directory, and ensures that the latest version of each file (based on time stamp) is stored in both locations. 27 | 28 | This input is stored so it isn't needed subsequently. The script just reads the stored data and performs the sync without requiring interaction. 29 | 30 | One note on usage: currently I have individual Dropbox files in the DropboxSync app folder hard-linked to their counterparts in my local git repo. This way changes made in Pythonista will propagate to the copies in the repo, and vice versa. Given my lack of knowledge of how Dropbox treats hard links, this seems iffy, but it works OK so far with one caveat—updating a file in the repo doesn't cause Dropbox to rescan its counterpart in the app folder. There are a vareity of ways to trigger a manual rescan, but I'd like to have it work automatically. As it is, this still makes integrating Pythonista with GitHub much easier. 31 | 32 | ## Improvements over original script 33 | 34 | - The original "dropboxsetup" module created a directory by default to store the Dropbox token. I've removed all support for this. 35 | - The original "DropboxSync" script required that multiple things be hardcoded into the script: the app key, app secret, and Dropbox token file name. Not only is this not very user friendly, but it means that I couldn't directly put a working version of this script on GitHub. The script now requests these items interactively. 36 | - Added support to store these inputs in the "pickled" state file. 37 | 38 | ## Future improvements 39 | 40 | - Currently, for this script to backup all of your Pythonista files, it must reside in the Pythonista root directory. If it's possible, I'd like to be able to store it in its own subdirectory. 41 | - I'd like to reintroduce the option to use a subdirectory to store the Python session file in a way that makes it optional. 42 | - The script relies on modification date/times to decide which version of a file is newer. I need to do some debugging to determine whether these times are universal or time zone dpendent—I wouldn't want a change in timezone on a device to cause the wrong file to be kept. 43 | - I'd like to have both a "sync" mode and an "archive" mode. 44 | - I need to clean up the code and comments to change it to my "house style"; currently it still looks very much like Michelle's original script. 45 | 46 | 47 | ## License 48 | 49 | See included license file. In addition to crediting me, you should probably also credit Michelle, the original author. Though she didn't specifically provide a license for the code, she provided a very nice base for this and deserves credit. . 50 | -------------------------------------------------------------------------------- /DropboxSync.py: -------------------------------------------------------------------------------- 1 | import os, sys, pickle, console, re 2 | 3 | sys.path += ['lib'] 4 | import dropboxsetup 5 | 6 | # dropbox_sync 7 | # by Michelle L. Gill, michelle@michellelynngill.com 8 | # requires my dropboxsetup module (https://gist.github.com/8311046) 9 | 10 | # Change log 11 | # 2013/01/07: initial version 12 | # 2013/01/09: added regex filtering 13 | 14 | ################################################################## 15 | # # 16 | # User specified parameters # 17 | # # 18 | ################################################################## 19 | 20 | ####### REGEX FILTERS NOT FULLY TESTED YET. USE DEBUG FIRST ####### 21 | # only files which match regexes in this list will be considered 22 | # leave empty to consider all files 23 | include_list = [] 24 | 25 | # then files which match regular expressions in this list will be removed 26 | # leave empty for no exclusions 27 | # empty directories are not uploaded 28 | exclude_list = [] 29 | 30 | # set this to true to test regular expressions instead of uploading 31 | debug_regex = False 32 | 33 | ################################################################## 34 | # # 35 | # Probably don't need to change anything below this point # 36 | # # 37 | ################################################################## 38 | 39 | 40 | STATE_FILE = '.dropbox_state' 41 | 42 | class dropbox_state: 43 | def __init__(self): 44 | self.cursor = None 45 | self.local_files = {} 46 | self.remote_files = {} 47 | self.app_key="" 48 | self.app_secret="" 49 | self.token_file_name="" 50 | 51 | def get_app_key(self): 52 | return self.app_key 53 | 54 | def set_app_key(self, name): 55 | self.app_key=name 56 | 57 | def get_app_secret(self): 58 | return self.app_secret 59 | 60 | def set_app_secret(self, name): 61 | self.app_secret=name 62 | 63 | def get_token_file_name(self): 64 | return self.token_file_name 65 | 66 | def set_token_file_name(self, name): 67 | self.token_file_name=name 68 | 69 | # use ignore_path to prevent download of recently uploaded files 70 | def execute_delta(self, client, ignore_path = None): 71 | delta = client.delta(self.cursor) 72 | self.cursor = delta['cursor'] 73 | 74 | for entry in delta['entries']: 75 | path = entry[0][1:] 76 | meta = entry[1] 77 | 78 | # this skips the path if we just uploaded it 79 | if path != ignore_path: 80 | if meta != None: 81 | path = meta['path'][1:] # caps sensitive 82 | if meta['is_dir']: 83 | print '\n\tMaking Directory:',path 84 | self.makedir_local(path) 85 | elif path not in self.remote_files: 86 | print '\n\tNot in local' 87 | self.download(client, path) 88 | elif meta['rev'] != self.remote_files[path]['rev']: 89 | print '\n\tOutdated revision' 90 | self.download(client, path) 91 | # remove file or directory 92 | else: 93 | if os.path.isdir(path): 94 | print '\n\tRemoving Directory:', path 95 | os.removedirs(path) 96 | elif os.path.isfile(path): 97 | print '\n\tRemoving File:', path 98 | os.remove(path) 99 | 100 | del self.local_files[path] 101 | del self.remote_files[path] 102 | else: 103 | pass # file already doesn't exist localy 104 | 105 | # makes dirs if necessary, downloads, and adds to local state data 106 | def download(self, client, path): 107 | print '\tDownloading:', path 108 | # TODO: what if there is a folder there...? 109 | head, tail = os.path.split(path) 110 | # make the folder if it doesn't exist yet 111 | if not os.path.exists(head) and head != '': 112 | os.makedirs(head) 113 | #open file to write 114 | local = open(path,'w') 115 | remote, meta = client.get_file_and_metadata(os.path.join('/',path)) 116 | local.write(remote.read()) 117 | #clean up 118 | remote.close() 119 | local.close() 120 | # add to local repository 121 | self.local_files[path] = {'modified': os.path.getmtime(path)} 122 | self.remote_files[path] = meta 123 | 124 | def upload(self, client, path): 125 | print '\tUploading:', path 126 | local = open(path,'r') 127 | meta = client.put_file(os.path.join('/',path), local, True) 128 | local.close() 129 | 130 | self.local_files[path] = {'modified': os.path.getmtime(path)} 131 | self.remote_files[path] = meta 132 | 133 | # clean out the delta for the file upload 134 | self.execute_delta(client, ignore_path=meta['path']) 135 | 136 | def delete(self, client, path): 137 | print '\tFile deleted locally. Deleting on Dropbox:',path 138 | try: 139 | client.file_delete(path) 140 | except: 141 | # file was probably already deleted 142 | print '\tFile already removed from Dropbox' 143 | 144 | del self.local_files[path] 145 | del self.remote_files[path] 146 | 147 | # safely makes local dir 148 | def makedir_local(self,path): 149 | if not os.path.exists(path): # no need to make a dir that exists 150 | os.makedirs(path) 151 | elif os.path.isfile(path): # if there is a file there ditch it 152 | os.remove(path) 153 | del self.files[path] 154 | 155 | os.makedir(path) 156 | 157 | # recursively list files on dropbox 158 | def _listfiles(self, client, path = '/'): 159 | meta = client.metadata(path) 160 | filelist = [] 161 | 162 | for item in meta['contents']: 163 | if item['is_dir']: 164 | filelist += self._listfiles(client,item['path']) 165 | else: 166 | filelist.append(item['path']) 167 | return filelist 168 | 169 | def download_all(self, client, path = '/'): 170 | filelist = self._listfiles(client) 171 | for file in filelist: 172 | self.download(client, file[1:]) # trim root slash 173 | 174 | def check_state(self, client, path): 175 | # lets see if we've seen it before 176 | if path not in self.local_files: 177 | # upload it! 178 | self.upload(client, path) 179 | elif os.path.getmtime(path) > self.local_files[path]['modified']: 180 | # newer file than last sync 181 | self.upload(client, path) 182 | else: 183 | pass # looks like everything is good 184 | 185 | def loadstate(): 186 | fyle = open(STATE_FILE,'r') 187 | state = pickle.load(fyle) 188 | fyle.close() 189 | 190 | return state 191 | 192 | def savestate(state): 193 | fyle = open(STATE_FILE,'w') 194 | pickle.dump(state,fyle) 195 | fyle.close() 196 | 197 | if __name__ == '__main__': 198 | console.show_activity() 199 | 200 | print """ 201 | **************************************** 202 | * Dropbox File Syncronization * 203 | ****************************************""" 204 | 205 | print '\nLoading local state' 206 | # lets see if we can unpickle 207 | try: 208 | state = loadstate() 209 | _, client = dropboxsetup.init(state.get_token_filename(), state.get_app_key(), state.get_app_secret()) 210 | except: 211 | print '\nCannot find state file. ***Making new local state***\n' 212 | # Aaaah, we have nothing, probably first run 213 | state = dropbox_state() 214 | 215 | print("We need a few setup details.\n") 216 | print("What do you want to call the Dropbox token file?") 217 | temp_input=raw_input() 218 | state.set_token_file_name(temp_input) 219 | print("What's your Dropbox app key?") 220 | temp_input=raw_input() 221 | state.set_app_key(temp_input) 222 | print("What's your Dropbox app secret?") 223 | temp_input=raw_input() 224 | state.set_app_secret(temp_input) 225 | print("Cool. We're ready to proceed.\n\n") 226 | 227 | _, client = dropboxsetup.init(state.get_token_file_name(), state.get_app_key(), state.get_app_secret()) 228 | 229 | print '\nDownloading everything from Dropbox' 230 | # no way to check what we have locally is newer, gratuitous dl 231 | state.download_all(client) 232 | 233 | print '\nUpdating state from Dropbox' 234 | state.execute_delta(client) 235 | 236 | 237 | print '\nChecking for new or updated local files' 238 | # back to business, lets see if there is anything new or changed localy 239 | dir_match=re.search(r'(/private/var/mobile/Applications/[0-9A-Z\\-]+/Documents)/.*', os.getcwd()) 240 | pythonista_root=dir_match.group(1) 241 | 242 | filelist = [] 243 | 244 | for root, dirnames, filenames in os.walk(pythonista_root): 245 | for filename in filenames: 246 | if filename != STATE_FILE: 247 | filelist.append( os.path.join(root, filename)) 248 | 249 | ####### REGEX FILTERS NOT FULLY TESTED YET ####### 250 | filelist = set(filelist) 251 | if debug_regex: 252 | allfiles = filelist 253 | 254 | # remove all files not in the includes list 255 | if len(include_list) >= 1: 256 | filelistinc = set() 257 | for reg in include_list: 258 | regc = re.compile(reg) 259 | filelistinc = filelistinc | set([x for x in filelist if re.search(regc,x)]) 260 | 261 | filelist = filelistinc 262 | 263 | if debug_regex: 264 | incfiles_keep = filelist 265 | incfiles_filt = allfiles - incfiles_keep 266 | 267 | # remove the excludes 268 | for reg in exclude_list: 269 | regc = re.compile(reg) 270 | filelist = filelist - set([x for x in filelist if re.search(regc,x)]) 271 | 272 | if debug_regex: 273 | excfiles_keep = filelist 274 | excfiles_filt = incfiles_keep - excfiles_keep 275 | 276 | print '\nThe following files were removed from sync list because they did not match include filters:\n' 277 | print incfiles_filt 278 | print '\nThe following files were then removed from sync list because they matched exclude filters:\n' 279 | print excfiles_filt 280 | print '\n The following files remain in the sync list after filtering:\n' 281 | print filelist 282 | 283 | else: 284 | filelist = list(filelist) 285 | 286 | for file in filelist: 287 | state.check_state(client,file) 288 | 289 | print '\nChecking for deleted local files' 290 | old_list = state.local_files.keys() 291 | for file in old_list: 292 | if file not in filelist: 293 | state.delete(client, file) 294 | 295 | print '\nSaving local state' 296 | savestate(state) 297 | 298 | print '\nSync complete' 299 | 300 | --------------------------------------------------------------------------------