├── .gitignore ├── LICENSE ├── README.md ├── download_amstertime.py ├── download_eynsham.py ├── download_nordland.py ├── download_san_francisco.py ├── download_sped.py ├── download_st_lucia.py ├── download_svox.py ├── format_mapillary.py ├── format_pitts250k.py ├── format_pitts30k.py ├── format_tokyo247.py ├── map_builder.py ├── requirements.txt └── util.py /.gitignore: -------------------------------------------------------------------------------- 1 | # custom 2 | msls 3 | testing.py 4 | 5 | 6 | z_others 7 | datasets 8 | .spyproject 9 | __pycache__ 10 | map_*.png 11 | tmp 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016-2019 VRG, CTU Prague 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # VPR Datasets Downloader 2 | 3 | This repo is made to automatically download Visual Place Recognition (VPR) datasets in a simple and standardized way. 4 | This is useful because all VPR datasets use different ways of storing the labels (some in csv files, some in matlab files, some in the filename), making it very inconvenient to train and test on different datasets. 5 | Furthermore, some datasets (e.g. Pitts30k and Nordland) require some pre-processing, and small changes in the pre-processing can lead to changes in results. 6 | The goal of this codebase is therefore to ensure that researchers and practitioners in VPR can use the same standardized datasets for their experiments. 7 | 8 | ## Overview 9 | 10 | The datasets are downloaded and formatted using a standard format: for some of the datasets also the maps are automatically created. 11 | The adopted convention is that the names of the files with the images are: 12 | 13 | `@ UTM_east @ UTM_north @ UTM_zone_number @ UTM_zone_letter @ latitude @ longitude @ pano_id @ tile_num @ heading @ pitch @ roll @ height @ timestamp @ note @ extension` 14 | 15 | Note that for many datasets some of these values are empty, and however the only required values for VPR are UTM coordinates (obtained from latitude and longitude). 16 | 17 | The reason for using the character `@` as a separator, is that commonly used characters such as dash `-` or underscore `_` might be used in the fields, for example in the `pano_id` field. 18 | 19 | The directory tree that is generated is as follows: 20 | ``` 21 | . 22 | └── datasets 23 | └── dataset_name 24 | └── images 25 | ├── train 26 | │ ├── database 27 | │ └── queries 28 | ├── val 29 | │ ├── database 30 | │ └── queries 31 | └── test 32 | ├── database 33 | └── queries 34 | ``` 35 | 36 | Most datasets are used only for testing, and therefore do not have a train and validation set. 37 | 38 | ## Available datasets 39 | 40 | The list of datasets that you can download with this codebase is the following: 41 | - Pitts30k* 42 | - Pitts250k* 43 | - Mapillary SLS** 44 | - Eysham - as test set only 45 | - San Francisco - as test set only 46 | - Tokyo 24/7* - as test set only 47 | - St Lucia - as test set only 48 | - SVOX - as test set only 49 | - Nordland - as test set only 50 | - AmsterTime - as test set only 51 | - SPED - as test set only 52 | 53 | To download each dataset, simply run the corresponding python script, that will download, 54 | unpack and format the file according to the structure above. 55 | 56 | *: for Pitts30k, Pitts250k and Tokyo 24/7 the images should be downloaded by asking permission to the respective authors. Then they can be formatted with this codebase 57 | 58 | *\*: for Mapillary SLS, you need to first log in into their website, download it [here](https://www.mapillary.com/dataset/places), 59 | then extract the zip files and run 60 | `$ python format_mapillary.py` 61 | 62 | #### Pitts30k 63 | 64 | For Pitts30k, first download the data under datasets/pitts30k/raw_data, then simply run `$ python format_pitts30k.py` 65 | 66 |
67 | Cite / BibTex 68 | 
 69 | @article{Torii_2015_pitts,
 70 |     author = {A. {Torii} and J. {Sivic} and M. {Okutomi} and T. {Pajdla}},
 71 |     journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
 72 |     title = {Visual Place Recognition with Repetitive Structures}, 
 73 |     year = {2015}
 74 | }
 75 |
76 |
77 | 78 | #### Pitts250k 79 | 80 | For Pitts250k, first download the data under datasets/pitts250k/raw_data, then simply run `$ python format_pitts250k.py` 81 | 82 |
83 | Cite / BibTex 84 |
85 | 
 86 | @article{Torii_2015_pitts,
 87 |     author = {A. {Torii} and J. {Sivic} and M. {Okutomi} and T. {Pajdla}},
 88 |     journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
 89 |     title = {Visual Place Recognition with Repetitive Structures}, 
 90 |     year = {2015}
 91 | }
 92 |
93 |
94 | 95 | #### Mapillary SLS 96 | 97 | For Mapillary SLS, you need to first log in into their website, download it [here](https://www.mapillary.com/dataset/places), 98 | then extract the zip files, and place it in a folder `datasets` inside the repository root and name it 99 | `mapillary_sls`. 100 | Then you can run: 101 | 102 | `$ python format_mapillary.py` 103 | 104 |
105 | Cite / BibTex 106 |
107 | 
108 | @inproceedings{Warburg_2020_msls,
109 |     author={Warburg, Frederik and Hauberg, Soren and Lopez-Antequera, Manuel and Gargallo, Pau and Kuang, Yubin and Civera, Javier},
110 |     title={Mapillary Street-Level Sequences: A Dataset for Lifelong Place Recognition},
111 |     booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
112 |     month={June},
113 |     year={2020}
114 | }
115 |
116 |
117 | 118 | #### Eynsham 119 | 120 | To download Eynsham, simply run `$ python download_eynsham.py` 121 | 122 |
123 | Cite / BibTex 124 |
125 | 
126 | @inproceedings{Cummins_2009_eynsham,
127 |     title={Highly scalable appearance-only SLAM - {FAB-MAP} 2.0},
128 |     author={M. Cummins and P. Newman},
129 |     booktitle={Robotics: Science and Systems},
130 |     year={2009}
131 | }
132 |
133 |
134 | 135 | #### San Francisco 136 | 137 | To download San Francisco, simply run `$ python download_san_francisco.py` 138 | 139 |
140 | Cite / BibTex 141 |
142 | 
143 | @inproceedings{Chen_2011_san_francisco,
144 |     author={D. M. {Chen} and G. {Baatz} and K. {Köser} and S. S. {Tsai} and R. {Vedantham} and T. {Pylvänäinen} and K. {Roimela} and X. {Chen} and J. {Bach} and M. {Pollefeys} and B. {Girod} and R. {Grzeszczuk}},
145 |     booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
146 |     title={City-scale landmark identification on mobile devices}, 
147 |     year={2011},
148 |     pages={737-744},
149 |     doi={10.1109/CVPR.2011.5995610}
150 | }
151 |
152 |
153 | 154 | #### St Lucia 155 | 156 | To download St Lucia, simply run `$ python download_st_lucia.py` 157 | 158 |
159 | Cite / BibTex 160 |
161 | 
162 | @article{Milford_2008_st_lucia,
163 |     title={Mapping a Suburb With a Single Camera Using a Biologically Inspired SLAM System},
164 |     author={Michael Milford and G. Wyeth},
165 |     journal={IEEE Transactions on Robotics},
166 |     year={2008},
167 |     volume={24},
168 |     pages={1038-1053}
169 | }
170 |
171 |
172 | 173 | #### SVOX 174 | 175 | To download SVOX, simply run `$ python download_svox.py` 176 | 177 |
178 | Cite / BibTex 179 |
180 | 
181 | @inproceedings{Berton_2021_svox, 
182 |     author = {Berton, Gabriele and Paolicelli, Valerio and Masone, Carlo and Caputo, Barbara},
183 |     title = {Adaptive-Attentive Geolocalization From Few Queries: A Hybrid Approach},
184 |     booktitle = {IEEE Winter Conference on Applications of Computer Vision},
185 |     month = {January},
186 |     year = {2021},
187 |     pages = {2918-2927}
188 | }
189 |
190 |
191 | 192 | #### Nordland 193 | 194 | To download Nordland, simply run `$ python download_nordland.py` 195 | 196 | The images will be arranged to have GPS/UTM labels compatible with the benchmarking code. More info on it are in the comment on top of the `download_nordland.py` script. We used the splits used by the [Patch-NetVLAD paper](https://arxiv.org/abs/2103.01486). 197 | 198 |
199 | Cite / BibTex 200 |
201 | 
202 | @inproceedings{Sunderhauf_2013_nordland,
203 |     title = {Are we there yet? Challenging {SeqSLAM} on a 3000 km journey across all four seasons},
204 |     author = {N. S{\"u}nderhauf and P. Neubert and P. Protzel},
205 |     booktitle = {Proc. of Workshop on Long-Term Autonomy, }#icra,
206 |     pages = {2013},
207 |     year = {2013}
208 | }
209 |
210 |
211 | 212 | #### Tokyo 24/7 213 | 214 | For Tokyo 24/7, first download the data under datasets/tokyo247/raw_data, then simply run `$ python format_tokyo247.py`. Queries are automatically downloaded. 215 | 216 |
217 | Cite / BibTex 218 |
219 | 
220 | @article{Torii_2018_tokyo247,
221 |     author = {A. {Torii} and R. {Arandjelović} and J. {Sivic} and M. {Okutomi} and T. {Pajdla}},
222 |     journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
223 |     title = {24/7 Place Recognition by View Synthesis}, 
224 |     year = {2018},
225 |     volume = {40},
226 |     number = {2},
227 |     pages = {257-271}
228 | }
229 |
230 |
231 | 232 | ### Other datasets 233 | 234 | Other datasets can be found in the [directory of Anyloc's datasets](https://iiitaphyd-my.sharepoint.com/personal/robotics_iiit_ac_in/_layouts/15/onedrive.aspx?ga=1&id=%2Fpersonal%2Frobotics%5Fiiit%5Fac%5Fin%2FDocuments%2FStudent%20Research%2FAnyLoc%2D2023%2FPublic%2FDatasets%2DAll) and [the directory of SALAD's datasets](https://surfdrive.surf.nl/files/index.php/s/sbZRXzYe3l0v67W). 235 | 236 | 237 | ## Cite / BibTex 238 | If you use this codebase, please cite [our benchmark](https://github.com/gmberton/deep-visual-geo-localization-benchmark) for which this code was built, and the respective paper for the datasets. 239 | ``` 240 | @inProceedings{Berton_CVPR_2022_benchmark, 241 | author = {Berton, Gabriele and Mereu, Riccardo and Trivigno, Gabriele and Masone, Carlo and 242 | Csurka, Gabriela and Sattler, Torsten and Caputo, Barbara}, 243 | title = {Deep Visual Geo-localization Benchmark}, 244 | booktitle = {CVPR}, 245 | month = {June}, 246 | year = {2022}, 247 | } 248 | ``` 249 | -------------------------------------------------------------------------------- /download_amstertime.py: -------------------------------------------------------------------------------- 1 | """ 2 | This script download the AmsterTime dataset, from the paper by Yildiz et al 3 | "AmsterTime: A Visual Place Recognition Benchmark Dataset for Severe Domain Shift" 4 | - https://arxiv.org/abs/2203.16291. 5 | The AmsterTime dataset is organize in pair of images, each pair being one query 6 | and one database image. The query is a historical photo, while the database 7 | image is a modern photo. Because their positions (e.g. GPS) are not available, 8 | we create mock positions for each pair, with images within a pair having the 9 | same position, and consecutive pairs being 100 meters away from each other. 10 | """ 11 | 12 | import os 13 | import shutil 14 | import py3_wget 15 | from glob import glob 16 | from tqdm import tqdm 17 | from PIL import Image 18 | from os.path import join 19 | 20 | 21 | datasets_folder = join(os.curdir, "datasets") 22 | dataset_name = "amstertime" 23 | dataset_folder = join(datasets_folder, dataset_name) 24 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 25 | database_folder = join(dataset_folder, "images", "test", "database") 26 | queries_folder = join(dataset_folder, "images", "test", "queries") 27 | os.makedirs(dataset_folder, exist_ok=True) 28 | os.makedirs(database_folder, exist_ok=True) 29 | os.makedirs(queries_folder, exist_ok=True) 30 | os.makedirs(raw_data_folder, exist_ok=True) 31 | 32 | py3_wget.download_file( 33 | url="https://data.4tu.nl/ndownloader/items/d2ee2551-986a-46bc-8540-b43f5b01ec4d/versions/4", 34 | output_path=join(raw_data_folder, "data.zip") 35 | ) 36 | 37 | shutil.unpack_archive(join(raw_data_folder, "data.zip"), raw_data_folder) 38 | shutil.unpack_archive(join(raw_data_folder, "AmsterTime-v1.0.zip"), raw_data_folder) 39 | 40 | database_paths = sorted(glob(join(raw_data_folder, "new", "*.png"))) 41 | queries_paths = sorted(glob(join(raw_data_folder, "old", "*.jpg"))) 42 | 43 | for db_filepath, q_filepath in zip(tqdm(database_paths), queries_paths): 44 | db_filename = os.path.basename(db_filepath) 45 | q_filename = os.path.basename(q_filepath) 46 | # Query and DB images from the same pair have the same name 47 | assert os.path.splitext(db_filename)[0] == os.path.splitext(q_filename)[0] 48 | pair_name = os.path.splitext(db_filename)[0] 49 | # Simulate a distance of at least 100 meters between any two pairs of non-matching images 50 | mock_utm_east = int(pair_name) * 1000 51 | new_image_name = f"@0@{mock_utm_east}@@@@@{pair_name}@@@@@@@@.jpg" 52 | new_q_path = os.path.join(queries_folder, new_image_name) # queries are in JPEG 53 | _ = shutil.move(q_filepath, new_q_path) 54 | new_db_path = os.path.join(database_folder, new_image_name) 55 | Image.open(db_filepath).convert("RGB").save(new_db_path) # db images are in PNG, so convert to JPEG 56 | os.remove(db_filepath) 57 | 58 | -------------------------------------------------------------------------------- /download_eynsham.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import shutil 4 | import py3_wget 5 | import numpy as np 6 | from tqdm import tqdm 7 | from glob import glob 8 | from PIL import Image 9 | from os.path import join 10 | from datetime import datetime 11 | 12 | import util 13 | import map_builder 14 | 15 | datasets_folder = join(os.curdir, "datasets") 16 | dataset_name = "eynsham" 17 | dataset_folder = join(datasets_folder, dataset_name) 18 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 19 | database_folder = join(dataset_folder, "images", "test", "database") 20 | queries_folder = join(dataset_folder, "images", "test", "queries") 21 | os.makedirs(dataset_folder, exist_ok=True) 22 | os.makedirs(database_folder, exist_ok=True) 23 | os.makedirs(queries_folder, exist_ok=True) 24 | os.makedirs(raw_data_folder, exist_ok=True) 25 | 26 | py3_wget.download_file( 27 | url="https://zenodo.org/record/1243106/files/Eynsham.zip?download=1", 28 | output_path=join(raw_data_folder, "Eynsham.zip") 29 | ) 30 | 31 | shutil.unpack_archive(join(raw_data_folder, "Eynsham.zip"), raw_data_folder) 32 | print("Unzipping, this will take a while") 33 | 34 | with open(join(raw_data_folder, "Eynsham", "Route_map", "Eynsham.kml"), "r") as file: 35 | lines = file.readlines() 36 | 37 | lines = [l.replace("\n", "") for l in lines] 38 | text = lines[11] 39 | splits = text.split("")[1].split("")[0].split(" ")[:-2] 40 | coords = np.array([s.split(",")[:2] for s in splits]).astype(np.float64) 41 | 42 | src_images_paths = sorted(glob(join(raw_data_folder, "Eynsham", "Images", "*.ppm")))[5:] 43 | 44 | for pano_num, (lon, lat) in enumerate(tqdm(coords)): 45 | for tile_num in range(5): 46 | src_image_path = src_images_paths[pano_num*5 + tile_num] 47 | timestamp = src_image_path.split("grab_")[1].split(".")[0] 48 | timestamp = datetime.utcfromtimestamp(int(timestamp)).strftime('%Y%m%d_%H%M%S') 49 | dst_image_name = util.get_dst_image_name(lat, lon, pano_id=f"{pano_num:04d}", 50 | tile_num=tile_num, timestamp=timestamp) 51 | # The first 4787 images correspond to the first sequence, and it is the database 52 | if pano_num < 4787: 53 | Image.open(src_image_path).save(join(database_folder, dst_image_name)) 54 | else: 55 | Image.open(src_image_path).save(join(queries_folder, dst_image_name)) 56 | 57 | map_builder.build_map_from_dataset(dataset_folder) 58 | shutil.rmtree(raw_data_folder) 59 | 60 | -------------------------------------------------------------------------------- /download_nordland.py: -------------------------------------------------------------------------------- 1 | """ 2 | This script downloads the Nordland dataset with the split used in Patch-NetVLAD. 3 | The images are arranged to be compatible with our benchmarking framework, and 4 | also with the CosPlace repository. 5 | In Nordland usually a prediction for a given query is considered correct if it 6 | is within 10 frames from the query (there's a 1-to-1 match between queries and 7 | database images). Given that our codebases rely on UTM coordinates to find 8 | positives and negatives, we create dummy UTM coordinates for all images, 9 | ensuring that images that are within 10 frames are also within 25 meters. 10 | In practice, we organize all images with UTM_east = 0 (i.e. in a straight line) 11 | and set UTM_north to be 2.4 meters apart between consecutive frames. 12 | 13 | NOTE: in some works [1-3] the split between database and queries is flipped, 14 | in the sense that winter images are used for the database and summer images for queries. 15 | [1] Zaffar et al, 2020, VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change 16 | [2] Ali-bey et al, 2024, BoQ: A Place is Worth a Bag of Learnable Queries 17 | [3] Izquierdo et al, 2024, Optimal transport aggregation for visual place recognition 18 | """ 19 | 20 | import os 21 | import shutil 22 | import py3_wget 23 | from tqdm import tqdm 24 | from glob import glob 25 | from PIL import Image 26 | from os.path import join 27 | 28 | import util 29 | 30 | THRESHOLD_METERS = 25 31 | THRESHOLD_FRAMES = 10 32 | DISTANCE_BETWEEN_FRAMES = THRESHOLD_METERS / (THRESHOLD_FRAMES + 0.5) 33 | 34 | datasets_folder = join(os.curdir, "datasets") 35 | dataset_name = "nordland" 36 | dataset_folder = join(datasets_folder, dataset_name) 37 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 38 | database_folder = join(dataset_folder, "images", "test", "database") 39 | queries_folder = join(dataset_folder, "images", "test", "queries") 40 | os.makedirs(dataset_folder, exist_ok=True) 41 | os.makedirs(database_folder, exist_ok=True) 42 | os.makedirs(queries_folder, exist_ok=True) 43 | os.makedirs(raw_data_folder, exist_ok=True) 44 | 45 | print("Downloading tars with the images") 46 | py3_wget.download_file( 47 | url='https://universityofadelaide.app.box.com/index.php?rm=box_download_shared_file&shared_name=zkfk1akpbo5318fzqmtvlpp7030ex4up&file_id=f_1424421870101', 48 | output_path=join(raw_data_folder, "summer.tar.gz") 49 | ) 50 | py3_wget.download_file( 51 | url='https://universityofadelaide.app.box.com/index.php?rm=box_download_shared_file&shared_name=zkfk1akpbo5318fzqmtvlpp7030ex4up&file_id=f_1521702837314', 52 | output_path=join(raw_data_folder, "winter.tar.gz") 53 | ) 54 | py3_wget.download_file( 55 | url="https://universityofadelaide.app.box.com/index.php?rm=box_download_shared_file&shared_name=zkfk1akpbo5318fzqmtvlpp7030ex4up&file_id=f_1424408901067", 56 | output_path=join(raw_data_folder, "cleanImageNames.txt") 57 | ) 58 | 59 | print("Unpacking tars with the images, this will take a few minutes") 60 | shutil.unpack_archive(join(raw_data_folder, "summer.tar.gz"), raw_data_folder) 61 | shutil.unpack_archive(join(raw_data_folder, "winter.tar.gz"), join(raw_data_folder, "winter")) 62 | 63 | with open(join(raw_data_folder, "cleanImageNames.txt")) as file: 64 | selected_images = file.readlines() 65 | selected_images = [i.replace("\n", "") for i in selected_images] 66 | selected_images = set(selected_images) 67 | 68 | database_paths = sorted(glob(join(raw_data_folder, "summer", "*.png"))) 69 | queries_paths = sorted(glob(join(raw_data_folder, "winter", "*.png"))) 70 | 71 | num_image = 0 72 | for path in tqdm(database_paths, desc="Copying DB images to dst"): 73 | if os.path.basename(path) not in selected_images: 74 | continue 75 | utm_north = util.format_coord(num_image*DISTANCE_BETWEEN_FRAMES, 5, 1) 76 | filename = f"@0@{utm_north}@@@@@{num_image}@@@@@@@@.jpg" 77 | new_path = join(database_folder, filename) 78 | Image.open(path).save(new_path) 79 | num_image += 1 80 | 81 | num_image = 0 82 | for path in tqdm(queries_paths, desc="Copying queries to dst"): 83 | if os.path.basename(path) not in selected_images: 84 | continue 85 | utm_north = util.format_coord(num_image*DISTANCE_BETWEEN_FRAMES, 5, 1) 86 | filename = f"@0@{utm_north}@@@@@{num_image}@@@@@@@@.jpg" 87 | new_path = join(queries_folder, filename) 88 | Image.open(path).save(new_path) 89 | num_image += 1 90 | 91 | -------------------------------------------------------------------------------- /download_san_francisco.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import utm 4 | import math 5 | import shutil 6 | import py3_wget 7 | from glob import glob 8 | from tqdm import tqdm 9 | from os.path import join 10 | 11 | import util 12 | import map_builder 13 | 14 | datasets_folder = join(os.curdir, "datasets") 15 | dataset_name = "san_francisco" 16 | dataset_folder = join(datasets_folder, dataset_name) 17 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 18 | os.makedirs(dataset_folder, exist_ok=True) 19 | os.makedirs(raw_data_folder, exist_ok=True) 20 | os.makedirs(join(dataset_folder, "images", "test"), exist_ok=True) 21 | 22 | #### Database 23 | print("Downloading database archives") 24 | filenames = [f"PCIs_{i*1000:08d}_{(i+1)*1000:08d}_3.tar" for i in range(11, 150)] 25 | urls = [f"https://stacks.stanford.edu/file/druid:vn158kj2087/{f}" for f in filenames] 26 | tars_paths = [join(raw_data_folder, f) for f in filenames] 27 | for i, (url, tar_path) in enumerate(zip(urls, tars_paths)): 28 | if os.path.exists(tar_path.replace("PCIs_", "").replace(".tar", "")): 29 | continue 30 | print(f"{i:>3} / {len(filenames)} ) downloading {tar_path}") 31 | py3_wget.download_file(url, tar_path) 32 | try: # Unpacking database archives 33 | shutil.unpack_archive(tar_path, raw_data_folder) 34 | except shutil.ReadError: 35 | pass # Some tars are empty files 36 | 37 | print("Formatting database files") 38 | dst_database_folder = join(dataset_folder, "images", "test", "database") 39 | os.makedirs(dst_database_folder, exist_ok=True) 40 | src_images_paths = sorted(glob(join(raw_data_folder, "**", "*.jpg"), recursive=True)) 41 | for src_image_path in tqdm(src_images_paths): 42 | _, _, pano_id, latitude, longitude, building_id, tile_num, carto_id, heading, pitch = os.path.basename(src_image_path).split("_") 43 | pitch = pitch.replace(".jpg", "") 44 | dst_image_name = util.get_dst_image_name(latitude, longitude, pano_id, tile_num, 45 | heading, pitch, extension=".jpg") 46 | dst_image_path = join(dst_database_folder, dst_image_name) 47 | _ = shutil.move(src_image_path, dst_image_path) 48 | 49 | #### Queries 50 | print("Downloading query archive") 51 | queries_zip_filename = "BuildingQueryImagesCartoIDCorrected-Upright.zip" 52 | url = f"https://stacks.stanford.edu/file/druid:vn158kj2087/{queries_zip_filename}" 53 | queries_zip_path = join(raw_data_folder, queries_zip_filename) 54 | py3_wget.download_file(url, queries_zip_path) 55 | 56 | print("Unpacking query archive") 57 | shutil.unpack_archive(queries_zip_path, raw_data_folder) 58 | 59 | print("Formatting query files") 60 | dst_queries_folder = join(dataset_folder, "images", "test", "queries") 61 | os.makedirs(dst_queries_folder, exist_ok=True) 62 | poses_file_path = join(raw_data_folder, "reference_poses_598.zip") 63 | py3_wget.download_file( 64 | url="http://www.ok.sc.e.titech.ac.jp/~torii/project/vlocalization/icons/reference_poses_598.zip", 65 | output_path=poses_file_path 66 | ) 67 | shutil.unpack_archive(poses_file_path, raw_data_folder) 68 | 69 | with open(join(raw_data_folder, "reference_poses_598", "reference_poses_addTM_all_598.txt"), "r") as file: 70 | lines = file.readlines()[1:] 71 | 72 | for line in lines: 73 | _, image_id, x, y, w, z, utm_east, utm_north, _ = line.split(" ") 74 | latitude, longitude = utm.to_latlon(float(utm_east), float(utm_north), 10, "S") 75 | x, y, w, z = float(x), float(y), float(w), float(z) 76 | yaw = math.atan2(2.0 * (z * x + y * w) , - 1.0 + 2.0 * (x * x + y * y)) 77 | heading = ((((yaw / math.pi) + 1) * 180) + 180) % 360 78 | dst_image_name = util.get_dst_image_name(latitude, longitude, pano_id=image_id, heading=heading) 79 | dst_image_path = join(dst_queries_folder, dst_image_name) 80 | src_image_path = join(raw_data_folder, "BuildingQueryImagesCartoIDCorrected-Upright", f"{image_id}.jpg") 81 | _ = shutil.move(src_image_path, dst_image_path) 82 | 83 | map_builder.build_map_from_dataset(dataset_folder) 84 | shutil.rmtree(raw_data_folder) 85 | 86 | -------------------------------------------------------------------------------- /download_sped.py: -------------------------------------------------------------------------------- 1 | 2 | import shutil 3 | import py3_wget 4 | from pathlib import Path 5 | from zipfile import ZipFile 6 | 7 | 8 | zip_filepath = Path("datasets", "sped", "raw_data", "sped.zip") 9 | zip_filepath.parent.mkdir(exist_ok=True, parents=True) 10 | 11 | py3_wget.download_file( 12 | url="https://surfdrive.surf.nl/files/index.php/s/sbZRXzYe3l0v67W/download?path=%2F&files=SPEDTEST.zip", 13 | output_path=zip_filepath 14 | ) 15 | 16 | zf = ZipFile(zip_filepath, 'r') 17 | zf.extractall(zip_filepath.parent) 18 | zf.close() 19 | 20 | database_paths = sorted((zip_filepath.parent / "SPEDTEST" / "ref").glob("*.jpg")) 21 | queries_paths = sorted((zip_filepath.parent / "SPEDTEST" / "query").glob("*.jpg")) 22 | 23 | database_folder = Path("datasets", "sped", "images", "test", "database") 24 | queries_folder = Path("datasets", "sped", "images", "test", "queries") 25 | database_folder.mkdir(exist_ok=True, parents=True) 26 | queries_folder.mkdir(exist_ok=True, parents=True) 27 | 28 | assert len(database_paths) == len(queries_paths) 29 | for db_path, q_path in zip(database_paths, queries_paths): 30 | db_path = Path(db_path) 31 | q_path = Path(q_path) 32 | assert db_path.name == q_path.name 33 | mock_utm_east = int(db_path.stem) * 1000 34 | new_image_name = f"@0@{mock_utm_east}@@@@@{db_path.stem}@@@@@@@@.jpg" 35 | new_db_path = Path(database_folder, new_image_name) 36 | new_q_path = Path(queries_folder, new_image_name) 37 | _ = shutil.move(db_path, new_db_path) 38 | _ = shutil.move(q_path, new_q_path) 39 | 40 | shutil.rmtree(zip_filepath.parent) 41 | 42 | 43 | -------------------------------------------------------------------------------- /download_st_lucia.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import shutil 4 | from tqdm import tqdm 5 | from mega import Mega 6 | from skimage import io 7 | from os.path import join 8 | 9 | import util 10 | import map_builder 11 | 12 | THRESHOLD_IN_METERS = 5 13 | 14 | datasets_folder = join(os.curdir, "datasets") 15 | dataset_name = "st_lucia" 16 | dataset_folder = join(datasets_folder, dataset_name) 17 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 18 | dst_database_folder = join(dataset_folder, "images", "test", "database") 19 | dst_queries_folder = join(dataset_folder, "images", "test", "queries") 20 | os.makedirs(dataset_folder, exist_ok=True) 21 | os.makedirs(dst_database_folder, exist_ok=True) 22 | os.makedirs(dst_queries_folder, exist_ok=True) 23 | os.makedirs(raw_data_folder, exist_ok=True) 24 | 25 | # Use the first pass for the database, and the last one for the queries 26 | urls = ['https://mega.nz/file/nE4g0LzZ#c8eL_H3ZfXElqEukw38i32p5cjwusTuNJYYeEP1d5Pg', 27 | # 'https://mega.nz/file/TVRXlDZR#WUJad1yQunPLpA38Z0rPqBeXXh4g_jnt4n-ZjDF8hKw', 28 | # 'https://mega.nz/file/LNRkiZAB#LemKJHU7kDunl_9CMQM6xCUdLzVhMYqE3DIZ6QVDOUo', 29 | # 'https://mega.nz/file/PRJy2BpI#TWSaNioftbPxw7T4LFV3CTtVNN08XnI94WH6_SZEJHE', 30 | # 'https://mega.nz/file/OFZngZwJ#L3rF3wU69v0Fdfh-iUcKLzkdrUhD6DI6SSYoRZuJkHE', 31 | # 'https://mega.nz/file/SRJ2wLYQ#EEBH4wCy5je0lbsBNoX9EmGuonYJ2hMothK5w0ehh9U', 32 | # 'https://mega.nz/file/3U4XnQqT#WifPXhWVmkYw5xAUFyIbaHzjEcghcUfjCK7rqBVl-YY', 33 | # 'https://mega.nz/file/WN4jzA7Q#Qlg1DxSuYvylIWuZRGFqwjnL1lYV698rsf_eGE1QWnI', 34 | # 'https://mega.nz/file/fQBmELwa#HHKNU28SreKnQjxIIZ_qUQorWqxZjCraiPT1oiYR4wo', 35 | 'https://mega.nz/file/PAgWSIhD#UeeA6knWL3pDh_IczbYkcA1R1MwSZ2vhEg2DTr1_oNw'] 36 | login = Mega().login() 37 | 38 | for sequence_num, url in enumerate(urls): 39 | print(f"{sequence_num:>2} / {len(urls)} ) downloading {url}") 40 | zip_path = login.download_url(url, raw_data_folder) 41 | zip_path = str(zip_path) 42 | subset_name = os.path.basename(zip_path).replace(".zip", "") 43 | shutil.unpack_archive(zip_path, raw_data_folder) 44 | 45 | vr = util.VideoReader(join(raw_data_folder, subset_name, "webcam_video.avi")) 46 | 47 | with open(join(raw_data_folder, subset_name, "fGPS.txt"), "r") as file: 48 | lines = file.readlines() 49 | 50 | last_coordinates = None 51 | for frame_num, line in zip(tqdm(range(vr.frames_num)), lines): 52 | latitude, longitude = line.split(",") 53 | latitude = "-" + latitude # Given latitude is positive, real latitude is negative (in Australia) 54 | easting, northing = util.format_location_info(latitude, longitude)[:2] 55 | if last_coordinates is None: 56 | last_coordinates = (easting, northing) 57 | else: 58 | distance_in_meters = util.get_distance(last_coordinates, (easting, northing)) 59 | if distance_in_meters < THRESHOLD_IN_METERS: 60 | continue # If this frame is too close to the previous one, skip it 61 | else: 62 | last_coordinates = (easting, northing) 63 | 64 | frame = vr.get_frame_at_frame_num(frame_num) 65 | image_name = util.get_dst_image_name(latitude, longitude, pano_id=f"{subset_name}_{frame_num:05d}") 66 | if sequence_num == 0: # The first sequence is the database 67 | io.imsave(join(dst_database_folder, image_name), frame) 68 | else: 69 | io.imsave(join(dst_queries_folder, image_name), frame) 70 | 71 | map_builder.build_map_from_dataset(dataset_folder) 72 | shutil.rmtree(raw_data_folder) 73 | 74 | -------------------------------------------------------------------------------- /download_svox.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import shutil 4 | 5 | import gdown 6 | 7 | datasets_folder = os.path.join(os.curdir, "datasets") 8 | os.makedirs(datasets_folder, exist_ok=True) 9 | zip_filepath = os.path.join(datasets_folder, 'svox.zip') 10 | gdown.download(id="16iuk8voW65GaywNUQlWAbDt6HZzAJ_t9", 11 | output=zip_filepath, 12 | quiet=False) 13 | 14 | shutil.unpack_archive(zip_filepath, datasets_folder) 15 | os.remove(zip_filepath) 16 | -------------------------------------------------------------------------------- /format_mapillary.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import shutil 4 | from glob import glob 5 | from tqdm import tqdm 6 | from os.path import join 7 | 8 | import util 9 | 10 | # This dictionary is copied from the original code 11 | # https://github.com/mapillary/mapillary_sls/blob/master/mapillary_sls/datasets/msls.py#L16 12 | default_cities = { 13 | 'train': ["trondheim", "london", "boston", "melbourne", "amsterdam","helsinki", 14 | "tokyo","toronto","saopaulo","moscow","zurich","paris","bangkok", 15 | "budapest","austin","berlin","ottawa","phoenix","goa","amman","nairobi","manila"], 16 | 'val': ["cph", "sf"], 17 | 'test': ["miami","athens","buenosaires","stockholm","bengaluru","kampala"] 18 | } 19 | 20 | csv_files_paths = sorted(glob(join("datasets", "mapillary_sls", "*", "*", "*", "postprocessed.csv"), 21 | recursive=True)) 22 | 23 | for csv_file_path in csv_files_paths: 24 | with open(csv_file_path, "r") as file: 25 | postprocessed_lines = file.readlines()[1:] 26 | with open(csv_file_path.replace("postprocessed", "raw"), "r") as file: 27 | raw_lines = file.readlines()[1:] 28 | assert len(raw_lines) == len(postprocessed_lines) 29 | 30 | csv_dir = os.path.dirname(csv_file_path) 31 | city_path, folder = os.path.split(csv_dir) 32 | city = os.path.split(city_path)[1] 33 | 34 | folder = "database" if folder == "database" else "queries" 35 | train_val = "train" if city in default_cities["train"] else "val" 36 | dst_folder = os.path.join('msls', train_val, folder) 37 | 38 | os.makedirs(dst_folder, exist_ok=True) 39 | for postprocessed_line, raw_line in zip(tqdm(postprocessed_lines, desc=city), raw_lines): 40 | _, pano_id, lon, lat, _, timestamp, is_panorama = raw_line.split(",") 41 | if is_panorama == "True\n": 42 | continue 43 | timestamp = timestamp.replace("-", "") 44 | view_direction = postprocessed_line.split(",")[-1].replace("\n", "").lower() 45 | day_night = "day" if postprocessed_line.split(",")[-2] == "False" else "night" 46 | note = f"{day_night}_{view_direction}_{city}" 47 | dst_image_name = util.get_dst_image_name(lat, lon, pano_id, timestamp=timestamp, note=note) 48 | src_image_path = os.path.join(os.path.dirname(csv_file_path), 'images', f'{pano_id}.jpg') 49 | dst_image_path = os.path.join(dst_folder, dst_image_name) 50 | _ = shutil.move(src_image_path, dst_image_path) 51 | 52 | val_path = os.path.join('msls', 'val') 53 | test_path = os.path.join('msls', 'test') 54 | os.symlink(os.path.abspath(val_path), test_path) 55 | -------------------------------------------------------------------------------- /format_pitts250k.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | In datasets/pitts250k/raw_data there should be the following files/folders: 4 | 000, 001, 002, 003, 004, 005, 006, 007, 008, 009, 010, queries_real, datasets 5 | (datasets is a folder that contains files such as pitts250k_train.mat) 6 | """ 7 | 8 | import os 9 | import re 10 | import utm 11 | import shutil 12 | from tqdm import tqdm 13 | from os.path import join 14 | from scipy.io import loadmat 15 | 16 | import util 17 | import map_builder 18 | 19 | datasets_folder = join(os.curdir, "datasets") 20 | dataset_name = "pitts250k" 21 | dataset_folder = join(datasets_folder, dataset_name) 22 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 23 | os.makedirs(dataset_folder, exist_ok=True) 24 | os.makedirs(raw_data_folder, exist_ok=True) 25 | 26 | def move_images(dst_folder, src_images_paths, utms): 27 | os.makedirs(dst_folder, exist_ok=True) 28 | for src_image_path, (utm_east, utm_north) in zip(tqdm(src_images_paths, desc=f"Move images to {dst_folder}"), 29 | utms): 30 | src_image_name = os.path.basename(src_image_path) 31 | latitude, longitude = utm.to_latlon(utm_east, utm_north, 17, "T") 32 | pitch = int(re.findall('pitch(\d+)_', src_image_name)[0])-1 33 | yaw = int(re.findall('yaw(\d+)\.', src_image_name)[0])-1 34 | note = re.findall('_(.+)\.jpg', src_image_name)[0] 35 | tile_num = pitch*24 + yaw 36 | dst_image_name = util.get_dst_image_name(latitude, longitude, pano_id=src_image_name.split("_")[0], 37 | tile_num=tile_num, note=note) 38 | 39 | src_path = os.path.join(dataset_folder, 'raw_data', src_image_path) 40 | dst_path = os.path.join(dst_folder, dst_image_name) 41 | shutil.move(src_path, dst_path) 42 | 43 | 44 | for dataset in ["train", "val", "test"]: 45 | matlab_struct_file_path = os.path.join(dataset_folder, "raw_data", "datasets", f"pitts250k_{dataset}.mat") 46 | mat_struct = loadmat(matlab_struct_file_path)["dbStruct"].item() 47 | # Database 48 | g_images = [f[0].item() for f in mat_struct[1]] 49 | g_utms = mat_struct[2].T 50 | move_images(os.path.join(dataset_folder, 'images', dataset, 'database'), g_images, g_utms) 51 | # Queries 52 | q_images = [os.path.join("queries_real", f"{f[0].item()}") for f in mat_struct[3]] 53 | q_utms = mat_struct[4].T 54 | move_images(os.path.join(dataset_folder, 'images', dataset, 'queries'), q_images, q_utms) 55 | 56 | map_builder.build_map_from_dataset(dataset_folder) 57 | shutil.rmtree(raw_data_folder) 58 | -------------------------------------------------------------------------------- /format_pitts30k.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | In datasets/pitts30k/raw_data there should be the following files/folders: 4 | 000, 001, 002, 003, 004, 005, 006, queries_real, datasets 5 | (datasets is a folder that contains files such as pitts250k_train.mat) 6 | """ 7 | 8 | import os 9 | import re 10 | import utm 11 | import shutil 12 | from tqdm import tqdm 13 | from os.path import join 14 | from scipy.io import loadmat 15 | 16 | import util 17 | import map_builder 18 | 19 | datasets_folder = join(os.curdir, "datasets") 20 | dataset_name = "pitts30k" 21 | dataset_folder = join(datasets_folder, dataset_name) 22 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 23 | os.makedirs(dataset_folder, exist_ok=True) 24 | os.makedirs(raw_data_folder, exist_ok=True) 25 | 26 | def copy_images(dst_folder, src_images_paths, utms): 27 | os.makedirs(dst_folder, exist_ok=True) 28 | for src_image_path, (utm_east, utm_north) in zip(tqdm(src_images_paths, desc=f"Copy to {dst_folder}"), 29 | utms): 30 | src_image_name = os.path.basename(src_image_path) 31 | latitude, longitude = utm.to_latlon(utm_east, utm_north, 17, "T") 32 | pitch = int(re.findall('pitch(\d+)_', src_image_name)[0])-1 33 | yaw = int(re.findall('yaw(\d+)\.', src_image_name)[0])-1 34 | note = re.findall('_(.+)\.jpg', src_image_name)[0] 35 | tile_num = pitch*24 + yaw 36 | dst_image_name = util.get_dst_image_name(latitude, longitude, pano_id=src_image_name.split("_")[0], 37 | tile_num=tile_num, note=note) 38 | src_path = os.path.join(dataset_folder, 'raw_data', src_image_path) 39 | dst_path = os.path.join(dst_folder, dst_image_name) 40 | shutil.move(src_path, dst_path) 41 | 42 | 43 | for dataset in ["train", "val", "test"]: 44 | matlab_struct_file_path = os.path.join(dataset_folder, "raw_data", "datasets", f"pitts30k_{dataset}.mat") 45 | mat_struct = loadmat(matlab_struct_file_path)["dbStruct"].item() 46 | # Database 47 | g_images = [f[0].item() for f in mat_struct[1]] 48 | g_utms = mat_struct[2].T 49 | copy_images(os.path.join(dataset_folder, 'images', dataset, 'database'), g_images, g_utms) 50 | # Queries 51 | q_images = [os.path.join("queries_real", f"{f[0].item()}") for f in mat_struct[3]] 52 | q_utms = mat_struct[4].T 53 | copy_images(os.path.join(dataset_folder, 'images', dataset, 'queries'), q_images, q_utms) 54 | 55 | map_builder.build_map_from_dataset(dataset_folder) 56 | shutil.rmtree(raw_data_folder) 57 | -------------------------------------------------------------------------------- /format_tokyo247.py: -------------------------------------------------------------------------------- 1 | """ 2 | This script allows to format the dataset of Tokyo 24/7 to a common format. 3 | The dataset should be downloaded before running this script (contact its 4 | authors for the link), and the files should be arranged as in the tree below. 5 | Folders like 03814 contain the images, and tokyo247.mat contains the metadata. 6 | Folders like 038XX should be extracted from archive 038XX.tar 7 | The queries are automatically downloaded by this script. 8 | Note that Tokyo 24/7 is a test-only dataset, i.e. there is no train and 9 | validation set. In some works (e.g. NetVLAD) Tokyo TM (Time Machine) has been 10 | used as a training set, but we do not provide the code to download it and 11 | format it. 12 | 13 | . 14 | |-- format_tokyo247.py 15 | |-- map_builder.py 16 | |-- util.py 17 | `-- datasets 18 | `-- tokyo247 19 | `-- raw_data 20 | |-- datasets 21 | | `-- tokyo247.mat 22 | `-- tokyo247 23 | |-- 03814 24 | |-- 03815 25 | |-- 03816 26 | |-- 03817 27 | |-- 03818 28 | |-- 03819 29 | |-- 03820 30 | |-- 03821 31 | |-- 03822 32 | |-- 03823 33 | |-- 03824 34 | |-- 03825 35 | |-- 03826 36 | |-- 03827 37 | |-- 03828 38 | `-- 03829 39 | 40 | """ 41 | 42 | import os 43 | import re 44 | import utm 45 | import shutil 46 | import py3_wget 47 | import torchvision 48 | from glob import glob 49 | from tqdm import tqdm 50 | from PIL import Image 51 | from os.path import join 52 | from scipy.io import loadmat 53 | 54 | import util 55 | import map_builder 56 | 57 | datasets_folder = join(os.curdir, "datasets") 58 | dataset_name = "tokyo247" 59 | dataset_folder = join(datasets_folder, dataset_name) 60 | raw_data_folder = join(datasets_folder, dataset_name, "raw_data") 61 | os.makedirs(dataset_folder, exist_ok=True) 62 | os.makedirs(raw_data_folder, exist_ok=True) 63 | os.makedirs(join(raw_data_folder, "tokyo247"), exist_ok=True) 64 | 65 | #### Database 66 | matlab_struct_file_path = join(dataset_folder, 'raw_data', 'datasets', 'tokyo247.mat') 67 | 68 | mat_struct = loadmat(matlab_struct_file_path)["dbStruct"].item() 69 | db_images = [join('tokyo247', f[0].item().replace('.jpg', '.png')) for f in mat_struct[1]] 70 | 71 | db_utms = mat_struct[2].T 72 | dst_folder = join(dataset_folder, 'images', 'test', 'database') 73 | 74 | os.makedirs(dst_folder, exist_ok=True) 75 | for src_image_path, (utm_east, utm_north) in zip(tqdm(db_images, desc=f"Copy to {dst_folder}"), 76 | db_utms): 77 | src_image_name = os.path.basename(src_image_path) 78 | latitude, longitude = utm.to_latlon(utm_east, utm_north, 54, 'S') 79 | pano_id = src_image_name[:22] 80 | tile_num = int(re.findall('_012_(\d+)\.png', src_image_name)[0])//30 81 | assert 0 <= tile_num < 12 82 | dst_image_name = util.get_dst_image_name(latitude, longitude, pano_id=pano_id, 83 | tile_num=tile_num) 84 | src_image_path = f"{dataset_folder}/raw_data/{src_image_path}" 85 | try: 86 | Image.open(src_image_path).save(f"{dst_folder}/{dst_image_name}") 87 | except OSError as e: 88 | print(f"Exception {e} with file {src_image_path}") 89 | raise e 90 | 91 | #### Queries 92 | filename = "247query_subset_v2.zip" 93 | url = f"https://data.ciirc.cvut.cz/public/projects/2015netVLAD/Tokyo247/queries/{filename}" 94 | file_zip_path = join(raw_data_folder, "tokyo247", filename) 95 | py3_wget.download_file(url, file_zip_path) 96 | shutil.unpack_archive(file_zip_path, join(raw_data_folder, "tokyo247")) 97 | src_queries_folder = file_zip_path.replace(".zip", "") 98 | src_queries_paths = sorted(glob(join(src_queries_folder, "*.jpg"))) 99 | os.makedirs(join(dataset_folder, "images", "test", "queries"), exist_ok=True) 100 | for src_query_path in tqdm(src_queries_paths, desc=f"Copy to {dataset_folder}/images/test/queries"): 101 | csv_path = src_query_path.replace(".jpg", ".csv") 102 | with open(csv_path, "r") as file: 103 | info = file.readline() 104 | pano_id, latitude, longitude = info.split(",")[:3] 105 | pano_id = pano_id.replace(",jpg", "") 106 | dst_image_name = util.get_dst_image_name(latitude, longitude, pano_id=pano_id) 107 | dst_image_path = join(dataset_folder, "images", "test", "queries", dst_image_name) 108 | try: 109 | pil_img = Image.open(src_query_path) 110 | except OSError as e: 111 | print(f"Exception {e} with file {src_query_path}") 112 | raise e 113 | resized_pil_img = torchvision.transforms.Resize(480)(pil_img) 114 | resized_pil_img.save(dst_image_path) 115 | 116 | map_builder.build_map_from_dataset(dataset_folder) 117 | shutil.rmtree(raw_data_folder) 118 | 119 | -------------------------------------------------------------------------------- /map_builder.py: -------------------------------------------------------------------------------- 1 | 2 | import matplotlib 3 | matplotlib.use('Agg') 4 | import os 5 | import cv2 6 | import copy 7 | import math 8 | import numpy as np 9 | from glob import glob 10 | from skimage import io 11 | from os.path import join 12 | import matplotlib.cm as cm 13 | import matplotlib.pyplot as plt 14 | from collections import defaultdict 15 | from staticmap import StaticMap, Polygon 16 | from matplotlib.colors import ListedColormap 17 | 18 | 19 | def _lon_to_x(lon, zoom): 20 | if not (-180 <= lon <= 180): lon = (lon + 180) % 360 - 180 21 | return ((lon + 180.) / 360) * pow(2, zoom) 22 | 23 | 24 | def _lat_to_y(lat, zoom): 25 | if not (-90 <= lat <= 90): lat = (lat + 90) % 180 - 90 26 | return (1 - math.log(math.tan(lat * math.pi / 180) + 1 / math.cos(lat * math.pi / 180)) / math.pi) / 2 * pow(2, zoom) 27 | 28 | 29 | def _download_map_image(min_lat=45.0, min_lon=7.6, max_lat=45.1, max_lon=7.7, size=2000): 30 | """"Download a map of the chosen area as a numpy image""" 31 | mean_lat = (min_lat + max_lat) / 2 32 | mean_lon = (min_lon + max_lon) / 2 33 | static_map = StaticMap(size, size) 34 | static_map.add_polygon( 35 | Polygon(((min_lon, min_lat), (min_lon, max_lat), (max_lon, max_lat), (max_lon, min_lat)), None, '#FFFFFF')) 36 | zoom = static_map._calculate_zoom() 37 | static_map = StaticMap(size, size) 38 | image = static_map.render(zoom, [mean_lon, mean_lat]) 39 | print( 40 | f"You can see the map on Google Maps at this link www.google.com/maps/place/@{mean_lat},{mean_lon},{zoom - 1}z") 41 | min_lat_px, min_lon_px, max_lat_px, max_lon_px = \ 42 | static_map._y_to_px(_lat_to_y(min_lat, zoom)), \ 43 | static_map._x_to_px(_lon_to_x(min_lon, zoom)), \ 44 | static_map._y_to_px(_lat_to_y(max_lat, zoom)), \ 45 | static_map._x_to_px(_lon_to_x(max_lon, zoom)) 46 | assert 0 <= max_lat_px < min_lat_px < size and 0 <= min_lon_px < max_lon_px < size 47 | return np.array(image)[max_lat_px:min_lat_px, min_lon_px:max_lon_px], static_map, zoom 48 | 49 | 50 | def get_edges(coordinates, enlarge=0): 51 | """ 52 | Send the edges of the coordinates, i.e. the most south, west, north and 53 | east coordinates. 54 | :param coordinates: A list of numpy.arrays of shape (Nx2) 55 | :param float enlarge: How much to increase the coordinates, to enlarge 56 | the area included between the points 57 | :return: a tuple with the four float 58 | """ 59 | min_lat, min_lon, max_lat, max_lon = (*np.concatenate(coordinates).min(0), *np.concatenate(coordinates).max(0)) 60 | diff_lat = (max_lat - min_lat) * enlarge 61 | diff_lon = (max_lon - min_lon) * enlarge 62 | inc_min_lat, inc_min_lon, inc_max_lat, inc_max_lon = \ 63 | min_lat - diff_lat, min_lon - diff_lon, max_lat + diff_lat, max_lon + diff_lon 64 | return inc_min_lat, inc_min_lon, inc_max_lat, inc_max_lon 65 | 66 | 67 | def _create_map(coordinates, colors=None, dot_sizes=None, legend_names=None, map_intensity=0.6): 68 | coordinates = copy.deepcopy(coordinates) 69 | dot_sizes = dot_sizes if dot_sizes is not None else [10] * len(coordinates) 70 | colors = colors if colors is not None else ["r"] * len(coordinates) 71 | assert len(coordinates) == len(dot_sizes) == len(colors), \ 72 | f"The number of coordinates must be equals to the number of colors and dot_sizes, but they're " \ 73 | f"{len(coordinates)}, {len(colors)}, {len(dot_sizes)}" 74 | 75 | # Add two dummy points to slightly enlarge the map 76 | min_lat, min_lon, max_lat, max_lon = get_edges(coordinates, enlarge=0.1) 77 | coordinates.append(np.array([[min_lat, min_lon], [max_lat, max_lon]])) 78 | # Download the map of the chosen area 79 | map_img, static_map, zoom = _download_map_image(min_lat, min_lon, max_lat, max_lon) 80 | 81 | scatters = [] 82 | fig = plt.figure(figsize=(map_img.shape[1] / 100, map_img.shape[0] / 100), dpi=1000) 83 | for i, coord in enumerate(coordinates): 84 | for i in range(len(coord)): # Scale latitudes because of earth's curvature 85 | coord[i, 0] = -static_map._y_to_px(_lat_to_y(coord[i, 0], zoom)) 86 | for coord, size, color in zip(coordinates, dot_sizes, colors): 87 | scatters.append(plt.scatter(coord[:, 1], coord[:, 0], s=size, color=color)) 88 | 89 | if legend_names != None: 90 | plt.legend(scatters, legend_names, scatterpoints=10000, loc='lower left', 91 | ncol=1, framealpha=0, prop={"weight": "bold", "size": 30}) 92 | 93 | min_lat, min_lon, max_lat, max_lon = get_edges(coordinates) 94 | plt.ylim(min_lat, max_lat) 95 | plt.xlim(min_lon, max_lon) 96 | fig.subplots_adjust(bottom=0, top=1, left=0, right=1) 97 | fig.canvas.draw() 98 | plot_img = np.array(fig.canvas.renderer._renderer) 99 | plt.close() 100 | 101 | plot_img = cv2.resize(plot_img[:, :, :3], map_img.shape[:2][::-1], interpolation=cv2.INTER_LANCZOS4) 102 | map_img[(map_img.sum(2) < 444)] = 188 # brighten dark pixels 103 | map_img = (((map_img / 255) ** map_intensity) * 255).astype(np.uint8) # fade map 104 | mask = (plot_img.sum(2) == 255 * 3)[:, :, None] # mask of plot, to find white pixels 105 | final_map = map_img * mask + plot_img * (~mask) 106 | return final_map 107 | 108 | 109 | def _get_coordinates_from_dataset(dataset_folder, extension="jpg"): 110 | """ 111 | Takes as input the path of a dataset, such as "datasets/st_lucia/images" 112 | and returns 113 | [("train/database", [[45, 8.1], [45.2, 8.2]]), ("train/queries", [[45, 8.1], [45.2, 8.2]])] 114 | """ 115 | images_paths = glob(join(dataset_folder, "**", f"*.{extension}"), recursive=True) 116 | if len(images_paths) != 0: 117 | print(f"I found {len(images_paths)} images in {dataset_folder}") 118 | else: 119 | raise ValueError(f"I found no images in {dataset_folder} !") 120 | 121 | grouped_gps_coords = defaultdict(list) 122 | 123 | for image_path in images_paths: 124 | full_path = os.path.dirname(image_path) 125 | full_parent_path, parent_dir = os.path.split(full_path) 126 | parent_parent_dir = os.path.split(full_parent_path)[1] 127 | 128 | # folder_name is for example "train - database" 129 | folder_name = " - ".join([parent_parent_dir, parent_dir]) 130 | 131 | gps_coords = image_path.split("@")[5], image_path.split("@")[6] 132 | grouped_gps_coords[folder_name].append(gps_coords) 133 | 134 | grouped_gps_coords = sorted([(k, np.array(v).astype(np.float64)) 135 | for k, v in grouped_gps_coords.items()]) 136 | return grouped_gps_coords 137 | 138 | 139 | def build_map_from_dataset(dataset_folder, dot_sizes=None): 140 | """dataset_folder is the path that contains the 'images' folder.""" 141 | grouped_gps_coords = _get_coordinates_from_dataset(join(dataset_folder, "images")) 142 | SORTED_FOLDERS = ["train - database", "train - queries", "val - database", "val - queries", 143 | "test - database", "test - queries"] 144 | try: 145 | grouped_gps_coords = sorted(grouped_gps_coords, key=lambda x: SORTED_FOLDERS.index(x[0])) 146 | except ValueError: 147 | pass # This dataset has different folder names than the standard train-val-test database-queries. 148 | coordinates = [] 149 | legend_names = [] 150 | for folder_name, coords in grouped_gps_coords: 151 | legend_names.append(f"{folder_name} - {len(coords)}") 152 | coordinates.append(coords) 153 | 154 | colors = cm.rainbow(np.linspace(0, 1, len(legend_names))) 155 | colors = ListedColormap(colors) 156 | colors = colors.colors 157 | if len(legend_names) == 1: 158 | legend_names = None # If there's only one folder, don't show the legend 159 | colors = np.array([[1, 0, 0]]) 160 | 161 | map_img = _create_map(coordinates, colors, dot_sizes, legend_names) 162 | 163 | print(f"Map image resolution: {map_img.shape}") 164 | dataset_name = os.path.basename(os.path.abspath(dataset_folder)) 165 | io.imsave(join(dataset_folder, f"map_{dataset_name}.png"), map_img) 166 | 167 | 168 | if __name__ == "__main__": 169 | coordinates = [ 170 | np.array([[41.8931, 12.4828], [45.4669, 9.1900], [40.8333, 14.2500]]), 171 | np.array([[52.5200, 13.4050], [48.7775, 9.1800], [48.1375, 11.5750]]), 172 | np.array([[48.8567, 2.3522], [43.2964, 5.3700], [45.7600, 4.8400]]) 173 | ] 174 | map_img = _create_map( 175 | coordinates, 176 | colors=["green", "black", "blue"], 177 | dot_sizes=[1000, 1000, 1000], 178 | legend_names=[ 179 | "Main Italian Cities", 180 | "Main German Cities", 181 | "Main French Cities", 182 | ]) 183 | 184 | io.imsave("cities.png", map_img) 185 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | staticmap 2 | tqdm 3 | mega.py 4 | requests 5 | opencv_contrib_python 6 | scikit_image 7 | scipy 8 | matplotlib 9 | numpy 10 | utm 11 | Pillow 12 | protobuf 13 | torchvision 14 | gdown 15 | py3_wget -------------------------------------------------------------------------------- /util.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import re 4 | import utm 5 | import cv2 6 | import math 7 | 8 | 9 | def get_distance(coords_A, coords_B): 10 | return math.sqrt((float(coords_B[0])-float(coords_A[0]))**2 + (float(coords_B[1])-float(coords_A[1]))**2) 11 | 12 | 13 | def is_valid_timestamp(timestamp): 14 | """Return True if it's a valid timestamp, in format YYYYMMDD_hhmmss, 15 | with all fields from left to right optional. 16 | >>> is_valid_timestamp('') 17 | True 18 | >>> is_valid_timestamp('201901') 19 | True 20 | >>> is_valid_timestamp('20190101_123000') 21 | True 22 | """ 23 | return bool(re.match("^(\d{4}(\d{2}(\d{2}(_(\d{2})(\d{2})?(\d{2})?)?)?)?)?$", timestamp)) 24 | 25 | 26 | def format_coord(num, left=2, right=5): 27 | """Return the formatted number as a string with (left) int digits 28 | (including sign '-' for negatives) and (right) float digits. 29 | >>> format_coord2(1.1, 3, 3) 30 | '001.100' 31 | >>> format_coord2(-0.12345, 3, 3) 32 | '-00.123' 33 | >>> format_coord2(-0.123, 5, 5) 34 | '-0000.12300' 35 | """ 36 | return f'{float(num):0={left+right+1}.{right}f}' 37 | 38 | import doctest 39 | doctest.testmod() # Automatically execute unit-test of format_coord() 40 | 41 | 42 | def format_location_info(latitude, longitude): 43 | easting, northing, zone_number, zone_letter = utm.from_latlon(float(latitude), float(longitude)) 44 | easting = format_coord(easting, 7, 2) 45 | northing = format_coord(northing, 7, 2) 46 | latitude = format_coord(latitude, 3, 5) 47 | longitude = format_coord(longitude, 4, 5) 48 | return easting, northing, zone_number, zone_letter, latitude, longitude 49 | 50 | 51 | def get_dst_image_name(latitude, longitude, pano_id=None, tile_num=None, heading=None, 52 | pitch=None, roll=None, height=None, timestamp=None, note=None, extension=".jpg"): 53 | easting, northing, zone_number, zone_letter, latitude, longitude = format_location_info(latitude, longitude) 54 | tile_num = f"{int(float(tile_num)):02d}" if tile_num is not None else "" 55 | heading = f"{int(float(heading)):03d}" if heading is not None else "" 56 | pitch = f"{int(float(pitch)):03d}" if pitch is not None else "" 57 | timestamp = f"{timestamp}" if timestamp is not None else "" 58 | note = f"{note}" if note is not None else "" 59 | assert is_valid_timestamp(timestamp), f"{timestamp} is not in YYYYMMDD_hhmmss format" 60 | if roll is None: roll = "" 61 | else: raise NotImplementedError() 62 | if height is None: height = "" 63 | else: raise NotImplementedError() 64 | 65 | return f"@{easting}@{northing}@{zone_number:02d}@{zone_letter}@{latitude}@{longitude}" + \ 66 | f"@{pano_id}@{tile_num}@{heading}@{pitch}@{roll}@{height}@{timestamp}@{note}@{extension}" 67 | 68 | 69 | class VideoReader: 70 | def __init__(self, video_name, size=None): 71 | if not os.path.exists(video_name): 72 | raise FileNotFoundError(f"{video_name} does not exist") 73 | self.video_name = video_name 74 | self.size = size 75 | self.vc = cv2.VideoCapture(f"{video_name}") 76 | self.frames_per_second = self.vc.get(cv2.CAP_PROP_FPS) 77 | self.frame_duration_millis = 1000 / self.frames_per_second 78 | self.frames_num = int(self.vc.get(cv2.CAP_PROP_FRAME_COUNT)) 79 | self.video_length_in_millis = int(self.frames_num * 1000 / self.frames_per_second) 80 | 81 | def get_time_at_frame(self, frame_num): 82 | return int(self.frame_duration_millis * frame_num) 83 | 84 | def get_frame_num_at_time(self, time): 85 | # time can be str ('21:59') or int in milliseconds 86 | millis = time if type(time) == int else self.str_to_millis(time) 87 | return min(int(millis / self.frame_duration_millis), self.frames_num) 88 | 89 | def get_frame_at_frame_num(self, frame_num): 90 | self.vc.set(cv2.CAP_PROP_POS_FRAMES, frame_num) 91 | frame = self.vc.read()[1] 92 | if frame is None: return None # In case of corrupt videos 93 | if self.size is not None: 94 | frame = cv2.resize(frame, self.size[::-1], cv2.INTER_CUBIC) 95 | frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) 96 | return frame 97 | 98 | @staticmethod 99 | def str_to_millis(time_str): 100 | return (int(time_str.split(":")[0]) * 60 + int(time_str.split(":")[1])) * 1000 101 | 102 | @staticmethod 103 | def millis_to_str(millis): 104 | if millis < 60*60*1000: 105 | return f"{math.floor((millis//1000//60)%60):02d}:{millis//1000%60:02d}" 106 | else: 107 | return f"{math.floor((millis//1000//60//60)%60):02d}:{math.floor((millis//1000//60)%60):02d}:{millis//1000%60:02d}" 108 | 109 | def __repr__(self): 110 | H, W = int(self.vc.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(self.vc.get(cv2.CAP_PROP_FRAME_WIDTH)) 111 | return (f"Video '{self.video_name}' has {self.frames_num} frames, " + 112 | f"with resolution {H}x{W}, " + 113 | f"and lasts {self.video_length_in_millis // 1000} seconds " 114 | f"({self.millis_to_str(self.video_length_in_millis)}), therefore " 115 | f"there's a frame every {int(self.frame_duration_millis)} millis") 116 | 117 | def __del__(self): 118 | self.vc.release() 119 | 120 | --------------------------------------------------------------------------------