├── README.md └── uploader ├── OpenScanUploader.zip ├── WindowsUploader.py └── uploader.py /README.md: -------------------------------------------------------------------------------- 1 | # OpenScanCloud 2 | Photogrammetry Web API 3 | 4 | ## Overview / Outline: 5 | The OpenScan Cloud is intended to be a decentralized, open and free photogrammetry web API. 6 | 7 | The API can be used as a great addition to existing photogrammetry rigs like the OpenScan Mini or Classic but also any other rig. The only things needed to start a reconstruction are a user-specific token and of course an image set (preferably as a zip file). 8 | 9 | Note that the application is totally free (and hopefully I can keep it free/donation-based in the future). Your data will be transferred through Dropbox and stored/processed on my local servers. I will use those image sets and resulting 3d models for further research, but none of your data will be published without your explicit consent! 10 | 11 | If you like the project, feel free to support my work on [Patreon](https://www.patreon.com/bePatron?u=51974655) 12 | 13 | Thank you :) 14 | 15 | ## Current functionality / Web Uploader through the browser 16 | 17 | ![image](https://github.com/user-attachments/assets/d6619cd9-f54c-4795-8ee4-a6e16f038743) 18 | 19 | Upload up to 2GB of photos through the Web interface without the need for any additional app! 20 | 21 | ## Current functionality / desktop uploader for Windows ( [Download ZIP](https://github.com/OpenScanEu/OpenScanCloud/raw/main/uploader/OpenScanUploader.zip)) 22 | 23 | * updated 2022-05-12 24 | 25 | ![image](https://user-images.githubusercontent.com/57842400/168089502-314cec43-0555-49e3-9043-06a5b5068906.png) 26 | 27 | Enter your token 28 | 29 | ![image](https://user-images.githubusercontent.com/57842400/168090093-0900defb-6f92-4978-b2ba-e4946e7882d5.png) 30 | 31 | Select your folder and upload the photos to the OpenScanCloud 32 | 33 | 34 | The Uploader is a standalone .exe, which should be able to run on any Windows machine. It allows you to select either a folder containing images and uploading those to the OpenScanCloud. All you need is an individual token (apply by email to cloud@openscan.eu) and some images. Note, that the program will create a temporary folder and an additional .txt file containing the token in the same directory as the .exe file. 35 | 36 | [SourceCode (Python)](https://github.com/OpenScanEu/OpenScanCloud/raw/main/uploader/WindowsUploader.py) 37 | 38 | ## Current functionality / Beta Firmware for OpenScan Classic & Mini 39 | Uploading through the firmware can be done with one click and has been implemented in all recent firmware versions. 40 | See [Github-OpenScan2](https://github.com/OpenScanEu/OpenScan2/) for more details 41 | 42 | ## Current functionality / python uploader (See [Sourecode](https://github.com/OpenScanEu/OpenScanCloud/blob/main/uploader/uploader.py)) 43 | The python script can be a starting point to create your own solution. Currently you only need to change a handful of parameters at the beginning of the file. Note, that the 'requests' module is needed (```pip install requests```) 44 | 45 | ## Current functionality / http endpoints 46 | Please feel free to add your thoughts on the design. Currently I implemented the following http endpoints: 47 | 48 | # Flask API Reference for OpenScan User 49 | 50 | ## Authentication 51 | 52 | This API uses HTTP Basic Authentication. The following credentials are required for access: 53 | 54 | - Username: `openscan` 55 | - Password: `free` 56 | 57 | Unauthorized access will result in a 401 Unauthorized response. 58 | 59 | ## Endpoints 60 | 61 | ### 1. Request Token 62 | 63 | Request a new token for access to the service. 64 | 65 | ``` 66 | GET /requestToken 67 | ``` 68 | 69 | #### Parameters 70 | 71 | | Name | Type | Description | 72 | |----------|--------|--------------------------------| 73 | | mail | string | Email address of the requester | 74 | | forename | string | First name of the requester | 75 | | lastname | string | Last name of the requester | 76 | 77 | #### Responses 78 | 79 | | Status | Description | 80 | |--------|--------------------------------------------| 81 | | 200 | Success. Returns an empty object `{}` | 82 | | 400 | Bad Request. Missing fields or unknown error | 83 | 84 | ### 2. Get Token Info 85 | 86 | Retrieve information about a specific token. 87 | 88 | ``` 89 | GET /getTokenInfo 90 | ``` 91 | 92 | #### Parameters 93 | 94 | | Name | Type | Description | 95 | |-------|--------|------------------------------------| 96 | | token | string | The token to retrieve information for | 97 | 98 | #### Responses 99 | 100 | | Status | Description | 101 | |--------|--------------------------------------------| 102 | | 200 | Success. Returns JSON object with token info | 103 | | 400 | Bad Request. Missing token or token doesn't exist | 104 | 105 | #### Success Response Fields 106 | 107 | - `credit` 108 | - `limit_filesize` 109 | - `limit_photos` 110 | 111 | ### 3. Get Project Info 112 | 113 | Retrieve information about a specific project. 114 | 115 | ``` 116 | GET /getProjectInfo 117 | ``` 118 | 119 | #### Parameters 120 | 121 | | Name | Type | Description | 122 | |---------|--------|---------------------------------| 123 | | token | string | The token associated with the project | 124 | | project | string | The project identifier | 125 | 126 | #### Responses 127 | 128 | | Status | Description | 129 | |--------|--------------------------------------------| 130 | | 200 | Success. Returns JSON object with project info | 131 | | 400 | Bad Request. Missing fields or token doesn't exist | 132 | | 401 | Unauthorized. Project doesn't exist | 133 | 134 | #### Success Response Fields 135 | 136 | - `dlink` 137 | - `status` 138 | - `ulink` 139 | 140 | ### 4. Create Project 141 | 142 | Create a new project associated with a token. 143 | 144 | ``` 145 | GET /createProject 146 | ``` 147 | 148 | #### Parameters 149 | 150 | | Name | Type | Description | 151 | |----------|---------|----------------------------------| 152 | | token | string | The token to associate with the project | 153 | | project | string | The project identifier | 154 | | photos | integer | Number of photos | 155 | | parts | integer | Number of parts | 156 | | filesize | integer | File size | 157 | 158 | #### Responses 159 | 160 | | Status | Description | 161 | |--------|--------------------------------------------| 162 | | 200 | Success. Returns JSON object | 163 | | 400 | Bad Request. Various error conditions | 164 | 165 | #### Success Response Object 166 | 167 | ```json 168 | { 169 | "status": "created", 170 | "ulink": [array of upload links], 171 | "credit": remaining credit 172 | } 173 | ``` 174 | 175 | ### 5. Reset Project 176 | 177 | Reset an existing project. 178 | 179 | ``` 180 | GET /resetProject 181 | ``` 182 | 183 | #### Parameters 184 | 185 | | Name | Type | Description | 186 | |---------|--------|---------------------------------| 187 | | token | string | The token associated with the project | 188 | | project | string | The project identifier | 189 | 190 | #### Responses 191 | 192 | | Status | Description | 193 | |--------|--------------------------------------------| 194 | | 200 | Success. Returns an empty object `{}` | 195 | | 400 | Bad Request. Missing fields or doesn't exist | 196 | 197 | ### 6. Start Project 198 | 199 | Start an existing project. 200 | 201 | ``` 202 | GET /startProject 203 | ``` 204 | 205 | #### Parameters 206 | 207 | | Name | Type | Description | 208 | |---------|--------|---------------------------------| 209 | | token | string | The token associated with the project | 210 | | project | string | The project identifier | 211 | 212 | #### Responses 213 | 214 | | Status | Description | 215 | |--------|--------------------------------------------| 216 | | 200 | Success. Returns JSON object | 217 | | 400 | Bad Request. Various error conditions | 218 | 219 | #### Success Response Object 220 | 221 | ```json 222 | { 223 | "status": "initialized" 224 | } 225 | ``` 226 | 227 | --- 228 | 229 | ## Token and credit system 230 | ### Credit 231 | Credit will be used to monitor the overall usage of processing ressources. The credit value is bound to each token. 232 | 233 | ### Tokens 234 | - Private Token 235 | This token is bound to an individual and certain details (forename and surname and email address) in order to allow additional features, like email alerts and individual support. At the current stage, the image sets submitted will be used for internal research and testing. No images/3d models will be published. 236 | 237 | ## Changelog 238 | - 2022-05-11 changed Readme.md, removed beta firmware (as this has been implemented in the main branch See [Github-OpenScan2](https://github.com/OpenScanEu/OpenScan2/) for more details) 239 | - 2021-12-20 added Texture to the 3d model + improved firmware 240 | - 2021-10-11 added Beta Firmware for OpenScanPi 241 | - 2021-10-08 added a Windows Uploader GUI in /uploader 242 | -------------------------------------------------------------------------------- /uploader/OpenScanUploader.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenScan-org/OpenScanCloud/c2515bcdf63431bf3e916fd52993e931145f40f8/uploader/OpenScanUploader.zip -------------------------------------------------------------------------------- /uploader/WindowsUploader.py: -------------------------------------------------------------------------------- 1 | import tkinter 2 | import os 3 | import requests 4 | import time 5 | from zipfile import ZipFile 6 | import sys 7 | from tkinter import filedialog 8 | import webbrowser 9 | import threading 10 | from sys import exit 11 | 12 | def browse_button_bg(): 13 | threading.Thread(target=browse_button).start() 14 | 15 | 16 | def browse_button(): 17 | folder = filedialog.askdirectory() 18 | if folder == '': 19 | return 20 | folderpath.set(folder) 21 | statustext.set('Selected ' + folder) 22 | list = [] 23 | 24 | for i in os.listdir(folder): 25 | if os.path.splitext(i)[1] in allowed_extensions: 26 | list.append(i) 27 | if len(list) == 0: 28 | statustext.set('No images found, allowed formats:'+ str(allowed_extensions)) 29 | upload['state'] = 'disabled' 30 | return 31 | 32 | filesize = 0 33 | for i in list: 34 | filesize = filesize + os.path.getsize(folder + '/' + i) 35 | 36 | statustext.set('Selected: ' + folder + '\nFound ' + str(len(list)) + ' photos with total filesize of ' + 37 | str(int(filesize/1000000)) + 'MB') 38 | upload['state']='active' 39 | msg['filesize'] = filesize 40 | msg['folder'] = folder + '/' 41 | msg['filelist'] = list 42 | msg['photos'] = len(list) 43 | msg['token'] = token.get() 44 | 45 | 46 | def page1(): 47 | naviUpload.grid_forget() 48 | enterToken.grid_forget() 49 | verifyToken.grid_forget() 50 | browse.grid(row=2, column=0, sticky='e') 51 | upload.grid(row=2, column=1, sticky='w') 52 | 53 | naviSettings.grid(row=4, columnspan=2) 54 | statustext.set('Select the folder containing your photos:') 55 | 56 | 57 | def page2(): 58 | browse.grid_forget() 59 | upload.grid_forget() 60 | naviSettings.grid_forget() 61 | enterToken.grid(row=2, columnspan=2) 62 | naviUpload.grid(row=4, columnspan=2) 63 | verifyToken.grid(row=3, columnspan=2) 64 | if token.get() == '': 65 | statustext.set('Please enter a valid OpenScanCloud token') 66 | naviUpload['state'] = 'disabled' 67 | else: 68 | statustext.set('Your OpenScanCloud token:') 69 | naviUpload['state'] = 'active' 70 | 71 | 72 | def OpenScanCloud(cmd, msg): 73 | r = requests.get(server + cmd, auth=(user, pw), params=msg) 74 | return r 75 | 76 | def verify_bg(): 77 | threading.Thread(target=verify).start() 78 | 79 | def verify(): 80 | verifyToken['state'] = 'disabled' 81 | statustext.set('Verifying Token ...') 82 | msg['token'] = token.get() 83 | if len(token.get()) < 15: 84 | statustext.set('Invalid Token') 85 | verifyToken['state'] = 'active' 86 | return 87 | r = OpenScanCloud('getTokenInfo', msg) 88 | if r.status_code != 200: 89 | statustext.set('Could not verify token, please try again:') 90 | naviUpload['state'] = 'disabled' 91 | verifyToken['state'] = 'active' 92 | return 93 | try: 94 | with open(active_directory + '/token.txt', 'w') as file: 95 | file.write(msg['token']) 96 | credit = round(int(r.json()['credit']) / 1000000000, 2) 97 | filesize = round(int(r.json()['limit_filesize']) / 1000000, 2) 98 | statustext.set('Token verified and saved.\nYou can upload a total of ' + str(credit) + 'GB\nWith a maximum size of '+str(filesize)+'MB per set') 99 | naviUpload['state'] = 'active' 100 | except: 101 | statustext.set('ERROR: Could not save token.') 102 | verifyToken['state'] = 'active' 103 | 104 | 105 | def uploader_bg(): 106 | threading.Thread(target=uploader).start() 107 | 108 | def uploader(): 109 | upload['state'] = 'disabled' 110 | browse['state'] = 'disabled' 111 | 112 | statustext.set('Preparing upload ...') 113 | r = OpenScanCloud('getTokenInfo', msg) 114 | if r.status_code != 200: 115 | statustext.set('Connection failed') 116 | upload['state'] = 'active' 117 | browse['state'] = 'active' 118 | 119 | return 120 | 121 | msg2 = r.json() 122 | 123 | if msg['filesize'] > msg2['credit']: 124 | statustext.set('Not enough credit, please contact cloud@openscan.eu') 125 | upload['state'] = 'active' 126 | browse['state'] = 'active' 127 | return 128 | 129 | if msg['filesize'] > msg2['limit_filesize']: 130 | statustext.set('Filesize limit exceeded') 131 | upload['state'] = 'active' 132 | browse['state'] = 'active' 133 | return 134 | 135 | zipAndSplit() 136 | uploadAndStart() 137 | browse['state'] = 'active' 138 | 139 | def zipAndSplit(): 140 | statustext.set('Creating zip ...') 141 | 142 | dir_tmp = active_directory + '/tmp/' 143 | 144 | if not os.path.isdir(dir_tmp): 145 | os.mkdir(dir_tmp) 146 | 147 | 148 | for i in os.listdir(dir_tmp): 149 | if os.path.isfile(dir_tmp + i): 150 | os.remove(dir_tmp + i) 151 | 152 | projectname = str(int(time.time()*100))+ '-OSC.zip' 153 | file = dir_tmp + projectname 154 | 155 | msg['project'] = projectname 156 | with ZipFile(file, 'w') as zip: 157 | for i in msg['filelist']: 158 | statustext.set('Adding to zip: ' + i) 159 | zip.write(msg['folder'] + i, i) 160 | 161 | msg['filesize'] = os.path.getsize(file) 162 | 163 | msg['partslist'] = [file] 164 | 165 | if os.path.getsize(file) > size_to_split: 166 | msg['partslist'] = [] 167 | number = 1 168 | with open(file, 'rb') as f: 169 | chunk = f.read(size_to_split) 170 | while chunk: 171 | statustext.set('Splitting archive into chunks: ' + str(number)) 172 | with open(file + '_' + str(number), 'wb+') as chunk_file: 173 | chunk_file.write(chunk) 174 | msg['partslist'].append(file + '_' + str(number)) 175 | number += 1 176 | chunk = f.read(size_to_split) 177 | os.remove(file) 178 | msg['parts'] = len(msg['partslist']) 179 | statustext.set('preparing project on the OpenScanCloud server') 180 | r = OpenScanCloud('createProject', msg) 181 | if r.status_code != 200: 182 | statustext.set('ERROR: Could not create project') 183 | return 184 | msg['ulink'] = '' 185 | msg['ulink'] = r.json()['ulink'] 186 | 187 | def uploadAndStart(): 188 | if msg['ulink'] == '': 189 | statustext.set('ERROR: Upload not started') 190 | return 191 | i = 0 192 | 193 | filelist = msg['partslist'] 194 | ulinks = msg['ulink'] 195 | 196 | for file in filelist: 197 | statustext.set('uploading part ' + str(i+1) + ' of ' + str(len(filelist))) 198 | link = ulinks[i] 199 | i = i+1 200 | 201 | data = open(file, 'rb').read() 202 | r = requests.post(url=link, data=data, headers={'Content-type': 'application/octet-stream'}) 203 | if r.status_code != 200: 204 | statustext.set('ERROR: could not upload file' + str(i)) 205 | return 206 | os.remove(file) 207 | 208 | statustext.set('starting project') 209 | r = OpenScanCloud('startProject', msg) 210 | if r.status_code != 200: 211 | statustext.set('ERROR: could not start processing') 212 | statustext.set('processing started ... you will get an email soon') 213 | try: 214 | os.rmdir(active_directory + '/tmp/') 215 | except: 216 | pass 217 | 218 | ## OSC Settings 219 | size_to_split = 200000000 #200MB is the maximum part size (total zip file can be up to 2GB) 220 | limit_filesize = 0 221 | limit_photos = 0 222 | credit = 0 223 | allowed_extensions = ['.jpg', '.jpeg', '.JPG', '.JPEG', '.png', '.PNG'] 224 | user = 'openscan' 225 | pw = 'free' 226 | server = 'http://openscanfeedback.dnsuser.de:1334/' 227 | msg = {} 228 | 229 | active_directory = os.path.dirname(os.path.realpath(sys.argv[0])) 230 | 231 | # TKinter Setup 232 | 233 | window = tkinter.Tk() 234 | window.title('OpenScan Desktop Uploader') 235 | window.geometry('500x220') 236 | window.resizable(0,0) 237 | window.grid_columnconfigure((0, 1), weight=1) 238 | window.grid_rowconfigure((0, 1, 2, 4, 5), weight=1,minsize=30) 239 | 240 | statustext = tkinter.StringVar() 241 | folderpath = tkinter.StringVar() 242 | token = tkinter.StringVar() 243 | 244 | 245 | title = tkinter.Label(window, text="--OpenScan Uploader --", font='Helvetica 18 bold') 246 | status = tkinter.Label(window, textvariable=statustext) 247 | naviUpload = tkinter.Button(text="UPLOAD", command=page1, cursor="hand2") 248 | naviSettings = tkinter.Button(text="SETTINGS", command=page2, cursor="hand2") 249 | 250 | link = tkinter.Button(text="GITHUB", cursor="hand2") 251 | link.bind("", lambda e: webbrowser.open_new("https://github.com/OpenScanEu/OpenScanCloud" 252 | "#current-functionality--desktop-uploader-for-windows--download")) 253 | donate = tkinter.Button(text="DONATE", cursor="hand2") 254 | donate.bind("", lambda e: webbrowser.open_new("https://www.patreon.com/bePatron?u=51974655")) 255 | 256 | browse = tkinter.Button(text="Select folder", command=browse_button_bg) 257 | upload = tkinter.Button(text = 'Upload Photos', command=uploader_bg) 258 | 259 | enterToken = tkinter.Entry(textvariable=token, justify='center') 260 | verifyToken = tkinter.Button(text="Verify and Save Token", cursor="hand2", command=verify_bg) 261 | 262 | title.grid(row=0, columnspan=2) 263 | status.grid(row=1, columnspan=2, sticky="ew") 264 | donate.grid(row=5, column=1, sticky='ew') 265 | link.grid(row=5, column=0, sticky='ew') 266 | upload['state']='disabled' 267 | 268 | 269 | if os.path.isfile(active_directory + '/token.txt'): 270 | with open(active_directory + '/token.txt', 'r') as file: 271 | token.set(file.read()) 272 | page1() 273 | else: 274 | token.set('') 275 | statustext.set('Please go to SETTINGS and enter a valid token') 276 | page2() 277 | 278 | window.mainloop() 279 | 280 | exit() 281 | -------------------------------------------------------------------------------- /uploader/uploader.py: -------------------------------------------------------------------------------- 1 | import os 2 | import requests 3 | import time 4 | from zipfile import ZipFile 5 | 6 | ################ That's all you need to change : ################### 7 | 8 | dir_images = '' # enter the directory of your images 9 | dir_temp = '' # enter your temporary directory 10 | token = '' # enter your token (send me a mail to cloud@openscan.eu with your forename and surname to get a free token) 11 | 12 | ################ No need to change anything below ################## 13 | 14 | size_to_split = 200000000 #200MB is the maximum part size (total zip file can be up to 2GB) 15 | limit_filesize = 0 16 | limit_photos = 0 17 | credit = 0 18 | allowed_extensions = ['.jpg', '.jpeg', '.JPG', '.JPEG', '.png', '.PNG'] #will add more soon 19 | user = 'openscan' 20 | pw = 'free' 21 | server = 'http://openscanfeedback.dnsuser.de:1334/' 22 | msg= {} 23 | msg['token'] = token 24 | 25 | def stop(msg): 26 | print(msg) 27 | while True: 28 | pass 29 | 30 | def OpenScanCloud(cmd, msg): 31 | r = requests.get(server + cmd, auth=(user, pw), params=msg) 32 | return r 33 | 34 | def uploadAndStart(filelist, ulinks): 35 | i = 0 36 | for file in filelist: 37 | print('uploading part ' + str(i+1) + ' of ' + str(len(filelist))) 38 | link = ulinks[i] 39 | i = i+1 40 | 41 | data = open(file, 'rb').read() 42 | r = requests.post(url=link, data=data, headers={'Content-type': 'application/octet-stream'}) 43 | if r.status_code != 200: 44 | stop('ERROR: could not upload file') 45 | 46 | print('starting project') 47 | r = OpenScanCloud('startProject', msg) 48 | if r.status_code != 200: 49 | stop('ERROR: could not start processing') 50 | print('processing started ... you will get an email soon') 51 | stop('thank you for testing OpenScanCloud') 52 | def getAndVerifyToken(): 53 | print('verifying token') 54 | global limit_filesize 55 | global limit_photos 56 | global credit 57 | 58 | tokenInfo = OpenScanCloud('getTokenInfo', msg) 59 | if tokenInfo.status_code != 200: 60 | stop('ERROR: invalid token') 61 | 62 | limit_filesize = tokenInfo.json()['limit_filesize'] 63 | limit_photos = tokenInfo.json()['limit_photos'] 64 | credit = tokenInfo.json()['credit'] 65 | 66 | def prepareSet(): 67 | print('preparing imageset') 68 | # load images: 69 | list = [] 70 | 71 | for i in os.listdir(dir_images): 72 | if os.path.splitext(i)[1] in allowed_extensions: 73 | list.append(i) 74 | if len(list) == 0: 75 | stop('ERROR: no images found in ' + dir_images) 76 | 77 | filesize = 0 78 | for i in list: 79 | filesize = filesize + os.path.getsize(dir_images + i) 80 | 81 | msg['photos'] = len(list) 82 | 83 | if filesize > limit_filesize or len(list) > limit_photos: 84 | stop('ERROR: Limits exceeded') 85 | return list 86 | 87 | def zipAndSplit(imagelist): 88 | print('zipping images') 89 | for i in os.listdir(dir_temp): 90 | os.remove(dir_temp + i) 91 | 92 | projectname = str(int(time.time()*100))+ '-OSC.zip' 93 | file = dir_temp + projectname 94 | 95 | print('projectname: ' + projectname) 96 | msg['project'] = projectname 97 | with ZipFile(file, 'w') as zip: 98 | for i in imagelist: 99 | zip.write(dir_images + i, i) 100 | 101 | msg['filesize'] = os.path.getsize(file) 102 | 103 | msg['partslist'] = [file] 104 | 105 | if os.path.getsize(file) > size_to_split: 106 | msg['partslist'] = [] 107 | number = 1 108 | with open (file, 'rb') as f: 109 | chunk = f.read(size_to_split) 110 | while chunk: 111 | with open(file + '_' + str(number), 'wb+') as chunk_file: 112 | chunk_file.write(chunk) 113 | msg['partslist'].append(file + '_' + str(number)) 114 | number += 1 115 | chunk = f.read(size_to_split) 116 | os.remove(file) 117 | msg['parts'] = len(msg['partslist']) 118 | print('preparing project on the OpenScanCloud server') 119 | r = OpenScanCloud('createProject', msg) 120 | if r.status_code != 200: 121 | stop('ERROR: Could not create project') 122 | msg['ulink'] = r.json()['ulink'] 123 | 124 | getAndVerifyToken() 125 | imagelist = prepareSet() 126 | zipAndSplit(imagelist) 127 | uploadAndStart(msg['partslist'], msg['ulink']) 128 | --------------------------------------------------------------------------------