├── LICENSE ├── README.md ├── img ├── detections.png ├── images.png ├── images_easy.png ├── images_hard.png ├── patches_easy.png └── patches_hard.png └── references.txt /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 HPatches 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # HPatches: Homography-patches dataset 2 | 3 | This repository contains details about the `HPatches` (Homography 4 | patches) dataset. 5 | 6 | More details can be found on the relevant `CVPR 2017` publication 7 | [[1]](#refs). 8 | 9 | The `HPatches` dataset was used as the basis for the local descriptor 10 | evaluation challenge that was presented in the 11 | [Local Features: State of the Art, Open Problems and Performance Evaluation](http://icvl.ee.ic.ac.uk/DescrWorkshop/index.html) 12 | workshop during ECCV 2016. 13 | 14 | There is a companion 15 | [benchmarking](https://github.com/hpatches/hpatches-benchmark) toolbox 16 | which defines the tasks and implements the `HPatches` evaluation 17 | protocol. Note that the benchmarking code allows to automatically 18 | download all the needed dataset files with a use of an automated 19 | script, and thus is the suggested way to use the dataset for 20 | descriptor evaluation purposes. 21 | 22 | If there is a need to manually get the `HPatches` dataset, download 23 | and untar the following file: 24 | 25 | * [HPatches](https://huggingface.co/datasets/vbalnt/hpatches/resolve/main/hpatches-release.zip) [4.2GB]. 26 | 27 | Sample code for reading the patches in `python` and `matlab` is 28 | provided in the 29 | [hpatches-benchmark](https://github.com/hpatches/hpatches-benchmark) 30 | repository. 31 | 32 | ## Dataset Description 33 | 34 | Patches are extracted from a number of image sequences, where each 35 | sequence contains images of the same scenes. Sequences are organised 36 | in folders depending on the type of transformations between images: 37 | 38 | * `i_X`: patches extracted from image sequences with illumination changes. 39 | * `v_X`: patches extracted from image sequences with viewpoint changes. 40 | 41 | For each image sequence, we provide a set of reference patches 42 | `ref.png` extracted from an image used as reference. For all other 43 | images in the sequence, we provide two more files, `eX.png` and 44 | `hX.png`, containing the "same" (corresponding) patches as found in 45 | the other images. In order to simulate the limitations of common patch 46 | detectors, correspondence are extracted by adding a certain amount of 47 | geometric noise (affine jitter). In particular, the `e` (easy) patches 48 | have little geometric noise and the `h` (hard) patches have more. Each 49 | patch has a size of `65x65` pixels and a single `*.png` file contains 50 | all the patches extracted from an image stacked along a single column. 51 | 52 | ## Patch Extraction Method 53 | 54 | Each image sequence contains a reference image and 5 target images 55 | taken under a different illumination and/or, for a planar scenes, a 56 | different viewpoint. For all images we have the estimated ground truth 57 | homography $H$ with respect to the reference (stored in CSV files 58 | `H_ref_X` where $X=1,...,5$). 59 | 60 | ![Example sequence](img/images.png) 61 | *Image 1: Example image sequence. The leftmost image is the reference image, followed by 5 images with a different viewpoint.* 62 | 63 | Patches are sampled in the reference image using a combination of 64 | local feature extractors (Hessian, Harris and DoG detector). The patch 65 | orientation is estimated using a single major orientation using Lowe's 66 | method. No affine adaptation is used, therefore all patches are square 67 | regions in the reference image. 68 | 69 | Patches are extracted from regions with a scale magnified by a factor 70 | of 5 compared to the original detected feature scale. Only patches for 71 | which this region is fully contained in the image are kept. 72 | 73 | In order to prevent multiple detections at the same location, multiple 74 | detections with ellipse overlap greater than 50% are clustered and a 75 | single ellipse at random is kept. A subset of the detected patches 76 | with their measurement regions is shown in the following image: 77 | 78 | ![Example detections](img/detections.png) 79 | 80 | *Image 2: Example detections in the reference image. Patches locations are visualized as ellipses. The scale of the detected patches (orange) is magnified by factor 5 to obtain the patch measurement region (yellow).* 81 | 82 | In order to extract the patches from a target image, first an affine 83 | jitter is applied. The goal of the affine jitter is to simulate the 84 | geometric repeatability error of typical local features detector. 85 | 86 | For easy jitter, the median ellipse overlap with the original patches 87 | is ~0.85 and for hard jitter it is ~0.72. After jittering, the frames 88 | are reprojected to the target image using the ground truth homography. 89 | 90 | The following images show the reprojected easy/hard patches in the 91 | target image together with the extracted patches. 92 | 93 | ![Reprojected easy patches](img/images_easy.png) 94 | 95 | *Image 3: Visualization of the easy patches locations in the target images.* 96 | 97 | ![Extracted easy patches](img/patches_easy.png) 98 | 99 | *Image 4: Extracted easy patches from the example sequence.* 100 | 101 | ![Reprojected hard patches](img/images_hard.png) 102 | 103 | *Image 5: Visualization of the hard patches locations in the target images.* 104 | 105 | ![Extracted hard patches](img/patches_hard.png) 106 | 107 | *Image 6: Extracted hard patches from the example sequence.* 108 | 109 | 110 | ## Full image sequences 111 | 112 | In addition to the extracted patch-based dataset, we provide the full 113 | image sequences that were used, together with the corresponding 114 | homographies. 115 | 116 | * [HPatches full sequences](https://huggingface.co/datasets/vbalnt/hpatches/resolve/main/hpatches-sequences-release.zip) [1.3GB]. 117 | 118 | For information about relevant citations concerning patches extracted 119 | from sequences that were not originally introduced in this dataset, 120 | please check [references.txt](references.txt). 121 | 122 | Please cite the original sources if you use `HPatches` for your 123 | research. 124 | 125 | ### References 126 | 127 | 128 | [1] *HPatches: A benchmark and evaluation of handcrafted and learned local descriptors*, Vassileios Balntas*, Karel Lenc*, Andrea Vedaldi and Krystian Mikolajczyk, CVPR 2017. [arXiv pdf] (https://arxiv.org/pdf/1704.05939.pdf) 129 | *Authors contributed equally. 130 | 131 | ``` 132 | @InProceedings{hpatches_2017_cvpr, 133 | author={Vassileios Balntas and Karel Lenc and Andrea Vedaldi and Krystian Mikolajczyk}, 134 | title = {HPatches: A benchmark and evaluation of handcrafted and learned local descriptors}, 135 | booktitle = {CVPR}, 136 | year = {2017}} 137 | ``` 138 | -------------------------------------------------------------------------------- /img/detections.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpatches/hpatches-dataset/fd79bca4dc755c9f623a5f129f3452fc663ee305/img/detections.png -------------------------------------------------------------------------------- /img/images.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpatches/hpatches-dataset/fd79bca4dc755c9f623a5f129f3452fc663ee305/img/images.png -------------------------------------------------------------------------------- /img/images_easy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpatches/hpatches-dataset/fd79bca4dc755c9f623a5f129f3452fc663ee305/img/images_easy.png -------------------------------------------------------------------------------- /img/images_hard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpatches/hpatches-dataset/fd79bca4dc755c9f623a5f129f3452fc663ee305/img/images_hard.png -------------------------------------------------------------------------------- /img/patches_easy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpatches/hpatches-dataset/fd79bca4dc755c9f623a5f129f3452fc663ee305/img/patches_easy.png -------------------------------------------------------------------------------- /img/patches_hard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpatches/hpatches-dataset/fd79bca4dc755c9f623a5f129f3452fc663ee305/img/patches_hard.png -------------------------------------------------------------------------------- /references.txt: -------------------------------------------------------------------------------- 1 | References for sequences included in the dataset 2 | 3 | v_abstract - [] 4 | v_adam - [ASIFT] 5 | v_apprentices - [] 6 | v_artisans - [] 7 | v_astronautis - [] 8 | v_azzola - [] 9 | v_bark - [OXF] 10 | v_bees - [] 11 | v_beyus - [] 12 | v_bip - [OXF] 13 | v_bird - [] 14 | v_birdwoman - [] 15 | v_blueprint - [] 16 | v_boat - [OXF] 17 | v_bricks - [OXF] 18 | v_busstop - [] 19 | v_calder - [] 20 | v_cartooncity - [] 21 | v_charing - [] 22 | v_churchill - [] 23 | v_circus - [] 24 | v_coffeehouse - [] 25 | v_colors - [HAN] 26 | v_courses - [] 27 | v_dirtywall - [] 28 | v_dogman - [] 29 | v_eastsouth - [] 30 | v_feast - [] 31 | v_fest - [] 32 | v_gardens - [] 33 | v_grace - [HAN] 34 | v_graffiti - [OXF] 35 | v_home - [] 36 | v_laptop - [OXF] 37 | v_london - [] 38 | v_machines - [] 39 | v_man - [] 40 | v_maskedman - [] 41 | v_pomegranate - [] 42 | v_posters - [HAN] 43 | v_samples - [] 44 | v_soldiers - [] 45 | v_strand - [] 46 | v_sunseason - [] 47 | v_tabletop - [] 48 | v_talent - [] 49 | v_tempera - [] 50 | v_there - [HAN] 51 | v_underground - [HAN] 52 | v_vitro - [] 53 | v_wall - [] 54 | v_wapping - [] 55 | v_war - [] 56 | v_weapons - [] 57 | v_woman - [] 58 | v_wormhole - [] 59 | v_wounded - [] 60 | v_yard - [] 61 | v_yuri - [] 62 | i_ajuntament - [AMOS] 63 | i_autannes - [AMOS] 64 | i_bologna - [AMOS] 65 | i_books - [DTU] 66 | i_boutique - [DTU] 67 | i_bridger - [AMOS] 68 | i_brooklyn - [AMOS] 69 | i_castle - [AMOS] 70 | i_chestnuts - [DTU] 71 | i_contruction - [AMOS] 72 | i_crownday - [AMOS] 73 | i_crownnight - [AMOS] 74 | i_dc - [AMOS[ 75 | i_dome - [AMOS] 76 | i_duda - [PHOS] 77 | i_fenis - [AMOS] 78 | i_fog - [AMOS] 79 | i_fruits - [OXF] 80 | i_gonnenberg - [AMOS] 81 | i_greenhouse - [AMOS] 82 | i_greentea - [DTU] 83 | i_indiana - [AMOS] 84 | i_kions - [AMOS] 85 | i_ktirio - [AMOS] 86 | i_kurhaus - [AMOS] 87 | i_leuven - [OXF] 88 | i_lionday - [AMOS] 89 | i_lionnight - [AMOS] 90 | i_londonbridge - [] 91 | i_melon - [DTU] 92 | i_miniature - [DTU] 93 | i_nescafe - [DTU] 94 | i_nijmegen - [AMOS] 95 | i_nuts - [OXF] 96 | i_objects - [] 97 | i_parking - [AMOS] 98 | i_partyfood - [PHOS] 99 | i_pencils - [] 100 | i_pinard - [DTU] 101 | i_pool - [AMOS] 102 | i_porta - [AMOS] 103 | i_resort - [AMOS] 104 | i_salon - [DTU] 105 | i_santuario - [AMOS] 106 | i_school - [AMOS] 107 | i_ski - [AMOS] 108 | i_smurf - [DTU] 109 | i_steps - [] 110 | i_table - [] 111 | i_tools - [] 112 | i_toy - [OXF] 113 | i_troulos - [AMOS] 114 | i_veggies - [DTU] 115 | i_village - [AMOS] 116 | i_whitebuilding - [AMOS] 117 | i_yellowtent - [DTU] 118 | i_zion - [AMOS] 119 | 120 | [AMOS] - N. Jacobs, N. Roman and R. Pless, Consistent Temporal Variations in Many Outdoor Scenes, CVPR 2007. 121 | [DTU] - H. Aanæs, A. Lindbjerg Dahl and K. Steenstrup Pedersen, Interesting Interest Points, IJCV 2012. 122 | [HAN] - K. Cordes, B. Rosenhahn, and J. Ostermann, Increasing the Accuracy of Feature Evaluation Benchmarks Using Differential Evolution, SDE IEEE, 2011. 123 | [OXF] - K Mikolajczyk and C Schmid, A performance evaluation of local descriptors, PAMI 2005 124 | [PHOS] - V. Vonikakis, D. Chrysostomou, R. Kouskouridas and A. Gasteratos, Improving the Robustness in Feature Detection by Local Contrast Enhancement, IST 2012. 125 | [ASIFT] - Y. Guoshen and J-M. Morel. ASIFT: An algorithm for fully affine invariant comparison, Image Processing Online 2011. 126 | [] - First introduced in this dataset. 127 | --------------------------------------------------------------------------------