├── .gitignore ├── LICENSE ├── README.md ├── aid.py ├── clue.py ├── common.py ├── eval_metrics.py ├── evaluate_query_disambiguation.py ├── extract_features.py ├── mirflickr ├── baby_query.txt ├── baby_r1.txt ├── bird_query.txt ├── bird_r1.txt ├── car_query.txt ├── car_r1.txt ├── clouds_query.txt ├── clouds_r1.txt ├── dog_query.txt ├── dog_r1.txt ├── duplicates.txt ├── female_query.txt ├── female_r1.txt ├── flower_query.txt ├── flower_r1.txt ├── male_query.txt ├── male_r1.txt ├── night_query.txt ├── night_r1.txt ├── people_query.txt ├── people_r1.txt ├── portrait_query.txt ├── portrait_r1.txt ├── river_query.txt ├── river_r1.txt ├── sea_query.txt ├── sea_r1.txt ├── tree_query.txt └── tree_r1.txt ├── model ├── VGG_ILSVRC_16_layers_deploy.prototxt └── image_mean.txt └── utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # SageMath parsed files 80 | *.sage.py 81 | 82 | # dotenv 83 | .env 84 | 85 | # virtualenv 86 | .venv 87 | venv/ 88 | ENV/ 89 | 90 | # Spyder project settings 91 | .spyderproject 92 | .spyproject 93 | 94 | # Rope project settings 95 | .ropeproject 96 | 97 | # mkdocs documentation 98 | /site 99 | 100 | # mypy 101 | .mypy_cache/ 102 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Computer Vision Group Jena 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Automatic Query Image Disambiguation (AID) 2 | ========================================== 3 | 4 | This repository contains the reference implementation of AID and code that can be used 5 | to reproduce the results from the corresponding [paper][4]: 6 | 7 | > Björn Barz and Joachim Denzler. 8 | > "Automatic Query Image Disambiguation for Content-based Image Retrieval." 9 | > International Conference on Computer Vision Theory and Applications (VISAPP), 2018. 10 | 11 | If you use AID, please cite that paper. 12 | 13 | 14 | What is AID? 15 | ------------ 16 | 17 | ![aid-schema](https://user-images.githubusercontent.com/7915048/31986052-52dd0688-b967-11e7-84d0-c778aa129f3d.png) 18 | 19 | AID is a novel recommendation and re-ranking technique for content-based image retrieval (CBIR). 20 | Query images presented to a CBIR system are usually ambiguous, so that user feedback is crucial for 21 | refining the search results towards the actual search objective pursued by the user. 22 | 23 | Instead of asking the user to mark multiple relevant and irrelevant images among the initial search 24 | results one by one, AID automatically discovers different meanings of the query image by clustering 25 | the top search results and then asks the users to simply select the cluster that seems most relevant 26 | to them. This way, the user's effort is minimized. 27 | 28 | Many similar methods restrict the set of refined results to the selected cluster. However, such an 29 | approach is sub-optimal, because the final set of results will be a strict subset of the initial 30 | set used for clustering, though there could be more relevant images not present among the first 31 | top results. AID; in contrast, applies a global re-ranking of all images in the database with 32 | respect to both the cluster selected by the user and the similarity to the initial query image. 33 | 34 | For details, please refer to the paper mentioned above. 35 | 36 | 37 | Dependencies 38 | ------------ 39 | 40 | ##### Mandatory 41 | 42 | - `Python >= 3.3` 43 | - `numpy` 44 | - `scipy` 45 | - `scikit-learn` 46 | 47 | ##### Optional 48 | 49 | - `caffe` & `pycaffe` (required if you want to extract the image features yourself) 50 | - `tqdm` (for progress bars) 51 | - `matplotlib` (if you would like to generate graphs for Precision@k) 52 | 53 | 54 | Reproducing the results from the paper 55 | -------------------------------------- 56 | 57 | ### Getting the features 58 | 59 | Before you can actually run the benchmark of the different query image disambiguation methods, 60 | you need to compute some features for the images in the dataset. You can either just download 61 | a [.npy file with pre-computed features][1] (49 MB) for the MIRFLICKR dataset or you can extract 62 | the features yourself as follows: 63 | 64 | 1. Download the [MIRFLICKR-25K dataset][2] (2.9 GB). 65 | 2. Extract the downloaded file inside of the `mirflickr` directory of this repository, so that you 66 | end up with another `mirflickr` directory inside of the top-level `mirflickr` directory. 67 | 3. Download the [pre-trained weights of the VGG 16 model][3] (528 MB) and store them in the `model` 68 | directory. 69 | 4. From the root directory of the repository, run: `python extract_features.py` 70 | 71 | ### Running the benchmark 72 | 73 | Once you have downloaded or extracted the features of the dataset images, you can run the benchmark 74 | as follows: 75 | 76 | python evaluate_query_disambiguation.py --plot_precision 77 | 78 | See `python evaluate_query_disambiguation.py --help` for the full list of options. 79 | 80 | The result should be similar to the following: 81 | 82 | | AP | P@1 | P@10 | P@50 | P@100 | NDCG | NDCG@100 83 | ---------------------------------------------------------------------------- 84 | Baseline | 0.4201 | 0.6453 | 0.6191 | 0.5932 | 0.5790 | 0.8693 | 0.5869 85 | CLUE | 0.4221 | 0.8301 | 0.7829 | 0.6466 | 0.5978 | 0.8722 | 0.6306 86 | Hard-Select | 0.4231 | 0.8138 | 0.8056 | 0.6773 | 0.6116 | 0.8727 | 0.6450 87 | AID | 0.5188 | 0.8263 | 0.7950 | 0.7454 | 0.7212 | 0.8991 | 0.7351 88 | 89 | The baseline results should match exactly, while deviations may occur in the other rows due to 90 | randomization. 91 | 92 | However, running the benchmark on the entire MIRFLICKR-25K dataset might take about a week and lots of RAM. 93 | If you would like to perform a slightly faster consistency check, you can also run the evaluation on 94 | a set of 70 pre-defined queries (5 for each topic): 95 | 96 | python evaluate_query_disambiguation.py --query_dir mirflickr --rounds 10 --show_sd --plot_precision 97 | 98 | In that case, the results should be similar to: 99 | 100 | | AP | P@1 | P@10 | P@50 | P@100 | NDCG | NDCG@100 101 | ---------------------------------------------------------------------------- 102 | Baseline | 0.3753 | 0.7286 | 0.6800 | 0.6100 | 0.5664 | 0.8223 | 0.5880 103 | CLUE | 0.3810 | 0.9100 | 0.8133 | 0.6462 | 0.5816 | 0.8290 | 0.6232 104 | Hard-Select | 0.3849 | 0.8457 | 0.8469 | 0.6846 | 0.6011 | 0.8314 | 0.6426 105 | AID | 0.4625 | 0.8757 | 0.8206 | 0.7211 | 0.6711 | 0.8531 | 0.6991 106 | 107 | 108 | Standard Deviation: 109 | 110 | | AP | P@1 | P@10 | P@50 | P@100 | NDCG | NDCG@100 111 | ---------------------------------------------------------------------------- 112 | Baseline | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 113 | CLUE | 0.0005 | 0.0239 | 0.0074 | 0.0045 | 0.0033 | 0.0003 | 0.0037 114 | Hard-Select | 0.0006 | 0.0270 | 0.0068 | 0.0072 | 0.0031 | 0.0005 | 0.0039 115 | AID | 0.0053 | 0.0203 | 0.0087 | 0.0085 | 0.0088 | 0.0017 | 0.0075 116 | 117 | 118 | 119 | [1]: http://www.inf-cv.uni-jena.de/dbvmedia/de/Barz/AID/features.npy 120 | [2]: http://press.liacs.nl/mirflickr/mirflickr25k/mirflickr25k.zip 121 | [3]: http://www.robots.ox.ac.uk/%7Evgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel 122 | [4]: http://hera.inf-cv.uni-jena.de:6680/pdf/Barz18:AID.pdf 123 | -------------------------------------------------------------------------------- /aid.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import scipy.linalg 3 | from sklearn.decomposition import PCA 4 | from sklearn.cluster import KMeans 5 | from sklearn.metrics.pairwise import rbf_kernel 6 | from scipy.sparse.csgraph import laplacian as graph_laplacian 7 | 8 | import random 9 | from multiprocessing import Pool 10 | 11 | from common import baseline_retrieval 12 | from utils import tqdm 13 | 14 | 15 | 16 | EPS = np.finfo('float32').resolution 17 | 18 | 19 | 20 | ## AID ## 21 | 22 | 23 | def automatic_image_disambiguation(features, queries, select_clusters, gamma = 1.0, k = 200, n_clusters = None, max_clusters = 10, show_progress = False): 24 | """ Automatic Image Disambiguation (our method) based on clustering of directions and directed boni. 25 | 26 | features - n-by-d matrix containing d-dimensional features of n samples. 27 | 28 | queries - Dictionary mapping query IDs to dictionaries with keys 'relevant' and 'img_id'. 'img_id' gives the ID of the query 29 | image and 'relevant' points to a list of IDs of images relevant for this query. 30 | 31 | select_clusters - Callback function taking a query dictionary with keys 'relevant' and 'img_id' and a list of lists of images 32 | for each cluster as arguments and returning a list of indices of selected clusters. 33 | 34 | gamma - Controls the effect of the cluster selection. For gamma < 1.0, the direction of samples must match the selected direction 35 | more exactly for those samples being adjusted, while for very large gamma, even samples in the orthogonal direction will 36 | be assigned a highly adjusted distance. 37 | 38 | k - The number of baseline retrieval results to be used for the initial clustering step. 39 | 40 | n_clusters - The number of clusters (image senses) to be shown to the user for selection of the relevant clusters. If set to None, 41 | the number of clusters will be determined heuristically. 42 | 43 | max_clusters - Maximum number of clusters. Has only an effect if n_clusters is None. 44 | 45 | show_progress - If True, a progress bar will be shown (requires tqdm). 46 | 47 | Returns: re-ranked retrieval results as dictionary mapping query IDs to tuples consisting of an ordered list of retrieved image IDs 48 | and a corresponding list of adjusted distances to the query. 49 | """ 50 | 51 | # Baseline retrieval 52 | retrievals = baseline_retrieval(features, queries, select_clusters) 53 | 54 | ret_it = tqdm(retrievals.items(), desc = 'AID', total = len(retrievals), leave = False) if show_progress else retrievals.items() 55 | 56 | with Pool(initializer = _init_pool, initargs = (features, queries, select_clusters, gamma, k, n_clusters, max_clusters)) as p: 57 | return dict(p.imap_unordered(_aid_worker, ret_it, 10)) 58 | 59 | 60 | def _init_pool(features, queries, select_clusters, gamma, k, n_clusters, max_clusters): 61 | global _features 62 | global _queries 63 | global _select_clusters 64 | global _gamma 65 | global _k 66 | global _n_clusters 67 | global _max_clusters 68 | _features = features 69 | _queries = queries 70 | _select_clusters = select_clusters 71 | _gamma = gamma 72 | _k = k 73 | _n_clusters = n_clusters 74 | _max_clusters = max_clusters 75 | random.seed() 76 | np.random.seed() 77 | 78 | 79 | def _aid_worker(args): 80 | 81 | global _features 82 | global _queries 83 | global _select_clusters 84 | global _gamma 85 | global _k 86 | global _n_clusters 87 | global _max_clusters 88 | 89 | qid, (ret, distances) = args 90 | 91 | query = _queries[qid] 92 | query_feat = _features[query['img_id']] 93 | 94 | # Compute directions from query to results 95 | directions = _features[ret] - query_feat[None,:] 96 | directions /= np.maximum(np.linalg.norm(directions, axis = -1, keepdims = True), EPS) 97 | 98 | # Cluster directions of top results 99 | nc = _n_clusters if (_n_clusters is not None) and (_n_clusters >= 1) else determine_num_clusters_spectral(directions[:_k, :], max_clusters = _max_clusters) 100 | if nc > 1: 101 | km = KMeans(nc, n_init = 100, max_iter = 1000, n_jobs = 1) 102 | # The KMeans implementation of sklearn <= 0.18.X suffers from numerical precision errors when using float32, 103 | # so we convert the data to float64 for clustering. See: https://github.com/scikit-learn/scikit-learn/issues/7705 104 | cluster_ind = km.fit_predict(directions[:_k, :].astype(np.float64)) 105 | 106 | # Ask user to select relevant clusters 107 | cluster_preview = [[id for id, l in zip(ret, cluster_ind) if l == i] for i in range(nc)] 108 | selected_clusters = _select_clusters(query, cluster_preview) 109 | 110 | # Re-rank results by taking their direction in relation to the selected clusters into account 111 | if (len(selected_clusters) > 0) and (len(selected_clusters) < nc): 112 | distances = adjust_distances(distances, directions, km.cluster_centers_[selected_clusters, :], _gamma) 113 | ind = np.argsort(distances) 114 | return (qid, (ret[ind], distances[ind])) 115 | 116 | return (qid, (ret, distances)) 117 | 118 | 119 | def determine_num_clusters_spectral(X, max_clusters = 10, gamma = None): 120 | """ Determine number of clusters based on Eigengaps of Graph Laplacian. """ 121 | 122 | if gamma is None: 123 | gamma = np.sqrt(X.shape[1]) 124 | 125 | adjacency = rbf_kernel(X, gamma = gamma) 126 | laplacian = graph_laplacian(adjacency, normed = True, return_diag = False) 127 | eig = scipy.linalg.eigh(laplacian, eigvals = (0, min(max_clusters, laplacian.shape[0] - 1)), eigvals_only = True) 128 | 129 | eigengap = eig[1:] - eig[:-1] 130 | return np.argmax(eigengap) + 1 131 | 132 | 133 | def adjust_distances(distances, directions, selected_directions, gamma = 1.0): 134 | """ Reduce distances of samples in the selected directions and increase distances of samples in the opposite directions. 135 | 136 | distances - Vector of length n with distances of samples in the database to the query. 137 | 138 | directions - n-by-d matrix with directions from the query to samples in the database, normalized to unit length. 139 | 140 | selected_directions - m-by-d matrix of relevant directions. 141 | 142 | gamma - Controls the effect of the cluster selection. For gamma < 1.0, the direction of samples must match the selected direction 143 | more exactly for those samples being adjusted, while for very large gamma, even samples in the orthogonal direction will 144 | be assigned a highly adjusted distance. 145 | 146 | Returns: adjusted distances of the samples in the database to the query. 147 | """ 148 | 149 | # Broadcast single direction to matrix 150 | if selected_directions.ndim == 1: 151 | selected_directions = selected_directions[None,:] 152 | 153 | # Normalize directions 154 | directions = directions / np.maximum(np.linalg.norm(directions, axis = -1, keepdims = True), EPS) 155 | selected_directions = selected_directions / np.maximum(np.linalg.norm(selected_directions, axis = -1, keepdims = True), EPS) 156 | 157 | # Compute cosine similarity to most similar direction as dot product (thanks to normalization) 158 | sim = np.dot(directions, selected_directions.T).max(axis = -1) 159 | 160 | # Fuse distance to query and similarity to directions and re-sort results 161 | max_dist = np.max(distances) 162 | return distances - np.sign(sim) * (np.abs(sim) ** gamma) * max_dist 163 | 164 | 165 | 166 | ## Hard Cluster Selection on the same clusters as AID ## 167 | 168 | 169 | def hard_cluster_selection(features, queries, select_clusters, k = 200, n_clusters = None, max_clusters = 10, show_progress = False): 170 | """ Hard Cluster Selection as used by CLUE, but on the clusters determined by AID (our method). """ 171 | 172 | # Baseline retrieval 173 | retrievals = baseline_retrieval(features, queries, select_clusters) 174 | 175 | ret_it = tqdm(retrievals.items(), desc = 'Hard-Select', total = len(retrievals), leave = False) if show_progress else retrievals.items() 176 | 177 | with Pool(initializer = _init_pool, initargs = (features, queries, select_clusters, 1.0, k, n_clusters, max_clusters)) as p: 178 | return dict(p.imap_unordered(_hs_worker, ret_it, 10)) 179 | 180 | 181 | def _hs_worker(args): 182 | 183 | global _features 184 | global _queries 185 | global _select_clusters 186 | global _k 187 | global _n_clusters 188 | global _max_clusters 189 | 190 | qid, (ret, distances) = args 191 | 192 | query = _queries[qid] 193 | query_feat = _features[query['img_id']] 194 | 195 | # Compute directions from query to results 196 | directions = _features[ret] - query_feat[None,:] 197 | directions /= np.maximum(np.linalg.norm(directions, axis = -1, keepdims = True), EPS) 198 | 199 | # Cluster directions of top results 200 | nc = _n_clusters if (_n_clusters is not None) and (_n_clusters >= 1) else determine_num_clusters_spectral(directions[:_k, :], max_clusters = _max_clusters) 201 | if nc > 1: 202 | km = KMeans(nc, n_init = 100, max_iter = 1000, n_jobs = 1) 203 | cluster_ind = km.fit_predict(directions[:_k, :].astype(np.float64)) 204 | 205 | # Ask user to select relevant clusters 206 | cluster_preview = [[id for id, l in zip(ret, cluster_ind) if l == i] for i in range(nc)] 207 | selected_clusters = _select_clusters(query, cluster_preview) 208 | 209 | # Put images from the selected clusters first 210 | return (qid, ( 211 | np.concatenate(([id for i, id in enumerate(ret[:_k]) if cluster_ind[i] in selected_clusters], [id for i, id in enumerate(ret[:_k]) if cluster_ind[i] not in selected_clusters], ret[_k:])), 212 | np.concatenate(([dist for i, dist in enumerate(distances[:_k]) if cluster_ind[i] in selected_clusters], [dist for i, dist in enumerate(distances[:_k]) if cluster_ind[i] not in selected_clusters], distances[_k:])) 213 | )) 214 | 215 | return (qid, (ret, distances)) 216 | -------------------------------------------------------------------------------- /clue.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import scipy.spatial.distance 3 | from sklearn.cluster import SpectralClustering 4 | 5 | import heapq 6 | 7 | from common import baseline_retrieval 8 | from utils import tqdm 9 | 10 | 11 | 12 | ## CLUE ## 13 | 14 | def clue(features, queries, select_clusters, k = 200, max_clusters = 10, T = 0.9, min_cluster_size = 2, show_progress = False): 15 | """ CLUE method for cluster-based relevance feedback in image retrieval. 16 | 17 | Reference: 18 | Chen, Yixin; Wang, James Z.; Krovetz, Robert. 19 | "CLUE: Cluster-Based Retrieval of Images by Unsupervised Learning." 20 | IEEE transactions on Image Processing 14.8, 2005, pp. 1187-1201. 21 | 22 | features - n-by-d matrix containing d-dimensional features of n samples. 23 | 24 | queries - Dictionary mapping query IDs to dictionaries with keys 'relevant' and 'img_id'. 'img_id' gives the ID of the query 25 | image and 'relevant' points to a list of IDs of images relevant for this query. 26 | 27 | select_clusters - Callback function taking a query dictionary with keys 'relevant' and 'img_id' and a list of lists of images 28 | for each cluster as arguments and returning a list of indices of selected clusters. 29 | 30 | k - The number of baseline retrieval results to be used for the initial clustering step. 31 | 32 | max_clusters - Maximum number of clusters. 33 | 34 | T - Threshold for the n-cut value. Nodes with an n-cut value larger than this threshold won't be subdivided any further. 35 | 36 | min_cluster_size - Minimum number of items per cluster. 37 | 38 | show_progress - If True, a progress bar will be shown (requires tqdm). 39 | 40 | Returns: re-ranked retrieval results as dictionary mapping query IDs to tuples consisting of an ordered list of retrieved image IDs 41 | and a corresponding list of adjusted distances to the query. 42 | """ 43 | 44 | # Baseline retrieval 45 | retrievals = baseline_retrieval(features, queries, select_clusters) 46 | 47 | ret_it = tqdm(retrievals.items(), desc = 'CLUE', total = len(retrievals), leave = False) if show_progress else retrievals.items() 48 | for qid, (ret, distances) in ret_it: 49 | 50 | query = queries[qid] 51 | query_feat = features[query['img_id']] 52 | 53 | # Spectral clustering of top results 54 | tree = RecursiveNormalizedCuts(max_clusters, T, min_cluster_size) 55 | tree.fit([(id, features[id]) for id in ret[:k]]) 56 | clusters = tree.clusters() 57 | 58 | # Ask user to select relevant clusters 59 | selected_clusters = select_clusters(query, tree.sort_items_by_centroid_distance()) 60 | 61 | # Put images from the selected clusters first 62 | offset = 0 63 | selected_clusters.sort() # disable cheating through fine-grained relevance ranking 64 | for c in selected_clusters: 65 | ret[offset:offset+len(clusters[c])] = [id for id, _ in clusters[c]] 66 | offset += len(clusters[c]) 67 | 68 | # Add remaining clusters in tree order 69 | for i, c in enumerate(clusters): 70 | if i not in selected_clusters: 71 | ret[offset:offset+len(c)] = [id for id, _ in c] 72 | offset += len(c) 73 | 74 | return retrievals 75 | 76 | 77 | class RecursiveNormalizedCuts(object): 78 | 79 | def __init__(self, max_clusters, T, min_cluster_size = 2): 80 | object.__init__(self) 81 | self.max_clusters = max_clusters 82 | self.T = T 83 | self.min_cluster_size = min_cluster_size 84 | self.tree = { 'depth' : 0, 'height' : 0, 'size' : 0, 'leafs' : 1, 'children' : [], 'parent' : None, 'items' : [], 'affinity' : [] } 85 | 86 | 87 | def fit(self, feat): 88 | 89 | # Compute affinity matrix using RBF kernel on pair-wise distances 90 | affinity = scipy.spatial.distance.pdist(np.array([f for id, f in feat])) 91 | sigma = -2 * np.var(affinity) 92 | affinity = np.exp(scipy.spatial.distance.squareform(affinity) / sigma) 93 | 94 | # Recursive clustering 95 | self.tree = { 'depth' : 0, 'height' : 0, 'size' : 0, 'leafs' : 1, 'children' : [], 'parent' : None, 'items' : feat, 'affinity' : affinity } 96 | queue = [] 97 | heapq.heappush(queue, (-1 * len(self.tree['items']), np.random.rand(), self.tree)) 98 | while (self.tree['leafs'] < self.max_clusters) and (len(queue) > 0): 99 | if len(queue[0][2]['items']) <= self.min_cluster_size: 100 | break 101 | left, right, ncut_value = self.split(heapq.heappop(queue)[2]) 102 | if ncut_value > self.T: 103 | break 104 | if (left is not None) and (right is not None): 105 | heapq.heappush(queue, (-1 * len(left['items']), np.random.rand(), left)) 106 | heapq.heappush(queue, (-1 * len(right['items']), np.random.rand(), right)) 107 | 108 | 109 | def split(self, node): 110 | 111 | # Perform normalized cut 112 | try: 113 | ind = SpectralClustering(2, affinity = 'precomputed', assign_labels = 'discretize').fit_predict(node['affinity']) 114 | except KeyboardInterrupt: 115 | raise 116 | except: 117 | return None, None, 0 118 | 119 | # Create left and right node 120 | mask1, mask2 = (ind == 0), (ind == 1) 121 | if not (np.any(mask1) and np.any(mask2)): 122 | return None, None, 0 123 | left = { 'depth' : node['depth'] + 1, 'height' : 0, 'size' : 0, 'leafs' : 1, 'children' : [], 'parent' : node, 'items' : [f for i, f in enumerate(node['items']) if ind[i] == 0], 'affinity' : node['affinity'][np.ix_(mask1, mask1)] } 124 | right = { 'depth' : node['depth'] + 1, 'height' : 0, 'size' : 0, 'leafs' : 1, 'children' : [], 'parent' : node, 'items' : [f for i, f in enumerate(node['items']) if ind[i] == 1], 'affinity' : node['affinity'][np.ix_(mask2, mask2)] } 125 | 126 | # Force the node with the lower minimum distance to the query to be the left node 127 | if ind[0] == 1: # items are already sorted when passed to fit(), so we just need to look at the first item instead of re-computing all distances 128 | left, right = right, left 129 | 130 | # Modify parent 131 | node['children'] = [left, right] 132 | 133 | # Modify parent chain 134 | parent = node 135 | while parent is not None: 136 | parent['height'] += 1 137 | parent['size'] += 2 138 | parent['leafs'] += 1 139 | parent = parent['parent'] 140 | 141 | return left, right, self.ncut_value(node['affinity'], ind) 142 | 143 | 144 | def clusters(self): 145 | 146 | def _clusters(node): 147 | return sum([_clusters(child) for child in node['children']], []) if len(node['children']) > 0 else [node['items']] 148 | 149 | return _clusters(self.tree) 150 | 151 | 152 | def sort_items_by_centroid_distance(self): 153 | 154 | clusters = self.clusters() 155 | sorted_clusters = [] 156 | for c in clusters: 157 | feat = np.array([f for id, f in c]) 158 | dist = np.linalg.norm(feat - feat.mean(axis = 0), axis = -1) 159 | ind = np.argsort(dist) 160 | sorted_clusters.append([c[i][0] for i in ind]) 161 | return sorted_clusters 162 | 163 | 164 | def ncut_value(self, affinity, lbl): 165 | 166 | mask_a, mask_b = (lbl == 0), (lbl == 1) 167 | cut_a_b = affinity[mask_a,:][:,mask_b].sum() 168 | cut_a_v = affinity[mask_a,:].sum() 169 | cut_b_v = affinity[mask_b,:].sum() 170 | if (cut_a_v == 0) or (cut_b_v == 0): 171 | print(affinity) 172 | print(lbl) 173 | return cut_a_b / cut_a_v + cut_a_b / cut_b_v -------------------------------------------------------------------------------- /common.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from multiprocessing import Pool 3 | from utils import tqdm 4 | 5 | 6 | 7 | def baseline_retrieval(features, queries, select_clusters, show_progress = False): 8 | """ Baseline retrieval without disambiguation. 9 | 10 | features - n-by-d matrix containing d-dimensional features of n samples. 11 | 12 | queries - Dictionary mapping query IDs to dictionaries with keys 'relevant' and 'img_id'. 'img_id' gives the ID of the query 13 | image and 'relevant' points to a list of IDs of images relevant for this query. 14 | 15 | select_clusters - Not used, only present for compatibility with similar functions. 16 | 17 | show_progress - If True, a progress bar will be shown (requires tqdm). 18 | 19 | Returns: Baseline retrieval results as dictionary mapping query IDs to tuples consisting of an ordered list of retrieved image 20 | IDs and a corresponding list of adjusted distances to the query. 21 | """ 22 | 23 | # Build ranked list of retrieval results for each query 24 | query_it = tqdm(queries.keys(), desc = 'Baseline', total = len(queries), leave = False) if show_progress else queries.keys() 25 | with Pool(initializer = _init_pool, initargs = (features, { qid : query['img_id'] for qid, query in queries.items() })) as p: 26 | return dict(p.imap_unordered(_retrieval_worker, query_it, 100)) 27 | 28 | 29 | def _init_pool(features, query_img_ids): 30 | global _feat 31 | global _img_ids 32 | _feat = features 33 | _img_ids = query_img_ids 34 | 35 | def _retrieval_worker(qid): 36 | global _feat 37 | global _img_ids 38 | distances = np.sum((_feat - _feat[[_img_ids[qid]],:]) ** 2, axis = 1) 39 | ranking = np.argsort(distances) 40 | assert(ranking[0] == _img_ids[qid]) 41 | return (qid, (ranking[1:], distances[ranking[1:]])) 42 | -------------------------------------------------------------------------------- /eval_metrics.py: -------------------------------------------------------------------------------- 1 | import math, os 2 | from multiprocessing import Pool 3 | 4 | 5 | 6 | def query_metrics(relevant, retrieved, ignore = [], include_prec = True): 7 | """ Computes several performance metrics for the result of a single query. 8 | 9 | relevant - Ground-Truth: Set of IDs relevant to the query. 10 | 11 | ret - Ranked list of retrieved image identifiers. 12 | 13 | ignore - Optionally, a set of IDs to be ignored, i.e., to be counted neither as true nor as false positive. 14 | 15 | include_prec - If `True`, P@1, P@10, P@50, and P@100 will be included in the resulting performance metrics. 16 | 17 | Returns: dictionary with the following items: 18 | - 'AP' : average precision 19 | - 'P@1' : accuracy regarding the best scoring result 20 | - 'P@10' : precision among the top 10 results 21 | - 'P@50' : precision among the top 50 results 22 | - 'P@100' : precision among the top 100 results 23 | - 'NDCG' : normalized discounted cumulative gain over the entire list of results 24 | - 'NDCG@100' : normalized discounted cumulative gain over the top 100 results 25 | """ 26 | 27 | if include_prec: 28 | 29 | return { 30 | 'AP' : average_precision(relevant, retrieved, ignore), 31 | 'P@1' : precision_at_k(1, relevant, retrieved, ignore), 32 | 'P@10' : precision_at_k(10, relevant, retrieved, ignore), 33 | 'P@50' : precision_at_k(50, relevant, retrieved, ignore), 34 | 'P@100' : precision_at_k(100, relevant, retrieved, ignore), 35 | 'NDCG' : ndcg(relevant, retrieved, ignore), 36 | 'NDCG@100' : ndcg(relevant, retrieved, ignore, 100), 37 | } 38 | 39 | else: 40 | 41 | return { 42 | 'AP' : average_precision(relevant, retrieved, ignore), 43 | 'NDCG' : ndcg(relevant, retrieved, ignore), 44 | 'NDCG@100' : ndcg(relevant, retrieved, ignore, 100), 45 | } 46 | 47 | 48 | 49 | def query_metrics_dict(query): 50 | """ Computes several performance metrics for the result of a single query, given as dictionary. 51 | 52 | query - Dictionary with keys 'relevant' (a set of relevant image IDs) and 'retrieved' (a list of retrieved image IDs). 53 | In addition, the dictionary may also contain an item named 'ignore', which gives a set of 54 | image IDs to be ignored, i.e., to be considered neither a true nor a false positive. 55 | 56 | Returns: dictionary with the following items: 57 | - 'AP' : average precision 58 | - 'NDCG' : normalized discounted cumulative gain over the entire list of results 59 | - 'NDCG@100' : normalized discounted cumulative gain over the top 100 results 60 | """ 61 | 62 | if 'ignore' not in query: 63 | query['ignore'] = [] 64 | 65 | return { 66 | 'AP' : average_precision(query['relevant'], query['retrieved'], query['ignore']), 67 | 'NDCG' : ndcg(query['relevant'], query['retrieved'], query['ignore']), 68 | 'NDCG@100' : ndcg(query['relevant'], query['retrieved'], query['ignore'], 100), 69 | } 70 | 71 | 72 | 73 | def avg_query_metrics(queries): 74 | """ Computes several performance metrics averaged over multiple queries. 75 | 76 | queries - Dictionary mapping query IDs to dictionaries with keys 'relevant' (a set of relevant image IDs) 77 | and 'retrieved' (a list of retrieved image IDs). 78 | In addition, the dictionaries may also contain an item named 'ignore', which gives a set of 79 | image IDs to be ignored, i.e., to be considered neither a true nor a false positive. 80 | 81 | Returns: dictionary with averages of the performance metrics described at `query_metrics_dict` 82 | """ 83 | 84 | sums = dict() 85 | 86 | with Pool() as p: 87 | for single in p.imap_unordered(query_metrics_dict, queries.values(), 100): 88 | for metric, value in single.items(): 89 | if metric not in sums: 90 | sums[metric] = value 91 | else: 92 | sums[metric] += value 93 | 94 | for metric, value in sums.items(): 95 | sums[metric] = value / float(len(queries)) 96 | 97 | return sums 98 | 99 | 100 | 101 | def mean_average_precision(queries): 102 | """ Computes Mean Average Precision for a set of retrieval results for a number of queries. 103 | 104 | queries - Dictionary mapping query IDs to dictionaries with keys 'relevant' (a set of relevant image IDs) 105 | and 'retrieved' (a list of retrieved image IDs). 106 | In addition, the dictionaries may also contain an item named 'ignore', which gives a set of 107 | image IDs to be ignored, i.e., to be considered neither a true nor a false positive. 108 | 109 | Returns: mAP 110 | """ 111 | 112 | return sum(average_precision( 113 | query['relevant'], 114 | query['retrieved'], 115 | query['ignore'] if 'ignore' in query else [] 116 | ) for qid, query in queries.items()) / len(queries) 117 | 118 | 119 | 120 | def average_precision(relevant, retrieved, ignore = []): 121 | """ Computes Average Precision for given retrieval results for a particular query. 122 | 123 | relevant - Ground-Truth: Set of IDs relevant to the query. 124 | retrieved - Ranked list of retrieved IDs. 125 | ignore - Optionally, a set of IDs to be ignored, i.e., to be counted neither as true nor as false positive. 126 | 127 | Returns: Area under the precision-recall curve. 128 | """ 129 | 130 | ignore = set(ignore) 131 | relevant = set(relevant) - ignore 132 | 133 | # Construct list of ranks of true-positive detections 134 | ranks = [] 135 | rank = 0 136 | for rid in retrieved: 137 | if rid not in ignore: 138 | if rid in relevant: 139 | ranks.append(rank) 140 | rank += 1 141 | 142 | return ap_from_ranks(ranks, len(relevant)) 143 | 144 | 145 | 146 | def ap_from_ranks(ranks, nres): 147 | """ Computes Average Precision for retrieval results given by the retrieved ranks of the relevant documents. 148 | 149 | ranks - Ordered list of ranks of true positives (zero-based). 150 | nres = Total number of positives in dataset. 151 | 152 | Returns: Area under the precision-recall curve. 153 | """ 154 | 155 | # accumulate trapezoids in PR-plot 156 | ap = 0.0 157 | 158 | # All have an x-size of: 159 | recall_step = 1.0 / nres 160 | 161 | for ntp, rank in enumerate(ranks): 162 | 163 | # y-size on left side of trapezoid: 164 | # ntp = nb of true positives so far 165 | # rank = nb of retrieved items so far 166 | precision_0 = 1.0 if rank == 0 else ntp / float(rank) 167 | 168 | # y-size on right side of trapezoid: 169 | # ntp and rank are increased by one 170 | precision_1 = (ntp+1) / float(rank+1) 171 | 172 | ap += (precision_1 + precision_0) * recall_step / 2.0 173 | 174 | return ap 175 | 176 | 177 | 178 | def precision_at_k(k, relevant, retrieved, ignore = []): 179 | """ Computes the precision among the top `k` retrieval results for a particular query. 180 | 181 | k - Number of best-scoring results to be considered. Instead of a single number, a list of `k` values may be given 182 | to evaluate precision at. In this case, this function will return a list of corresponding precisions as well. 183 | relevant - Ground-Truth: Set of IDs relevant to the query. 184 | retrieved - Ranked list of retrieved IDs. 185 | ignore - Optionally, a set of IDs to be ignored, i.e., to be counted neither as true nor as false positive. 186 | 187 | Returns: Precision@k, given as a single number or a list, depending on the value of `k` 188 | """ 189 | 190 | ks = [k] if isinstance(k, int) else list(k) 191 | ks.sort() 192 | k_index = 0 193 | 194 | precisions = [] 195 | 196 | rank, tp = 0, 0 197 | for ret in retrieved: 198 | if ret not in ignore: 199 | if ret in relevant: 200 | tp += 1 201 | rank += 1 202 | while (k_index < len(ks)) and (rank >= ks[k_index]): 203 | precisions.append(float(tp) / float(rank)) 204 | k_index += 1 205 | if k_index >= len(ks): 206 | break 207 | return precisions[0] if isinstance(k, int) else precisions 208 | 209 | 210 | 211 | def ndcg(relevant, retrieved, ignore = [], k = None): 212 | """ Computes the Normalized Discounted Cumulative Gain of a list of retrieval results with binary relevance levels for a particular query. 213 | 214 | relevant - Ground-Truth: Set of IDs relevant to the query. 215 | retrieved - Ranked list of retrieved IDs. 216 | ignore - Optionally, a set of IDs to be ignored, i.e., to be counted neither as true nor as false positive. 217 | k - Optionally, the maximum number of best-scoring results to be considered (will compute the NDCG @ k). 218 | If set to `None`, the NDCG of the entire list of retrieval results will be computed. 219 | Instead of a single number, a sorted list of values for various `k` may be given to evaluate NDCG at. 220 | In this case, this function will return a list of corresponding precisions as well. 221 | 222 | Returns: NDCG@k, given as a single number or a list, depending on the value of `k` 223 | """ 224 | 225 | ignore = set(ignore) 226 | max_rank = len(set(retrieved) - ignore) 227 | 228 | ks = [k] if isinstance(k, int) or (k is None) else list(k) 229 | for i in range(len(ks)): 230 | if (ks[i] is None) or (ks[i] > max_rank): 231 | ks[i] = max_rank 232 | ks.sort() 233 | k_index = 0 234 | 235 | ndcgs = [] 236 | 237 | rank, cgain, normalizer = 0, 0.0, 0.0 238 | for ret in retrieved: 239 | if ret not in ignore: 240 | rank += 1 241 | gain = 1.0 / math.log2(rank + 1) 242 | if ret in relevant: 243 | cgain += gain 244 | if rank <= len(relevant): 245 | normalizer += gain 246 | while (k_index < len(ks)) and (rank >= ks[k_index]): 247 | ndcgs.append(cgain / normalizer) 248 | k_index += 1 249 | if k_index >= len(ks): 250 | break 251 | return ndcgs[0] if isinstance(k, int) or (k is None) else ndcgs -------------------------------------------------------------------------------- /evaluate_query_disambiguation.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | import sys 4 | import argparse 5 | import itertools 6 | from collections import OrderedDict 7 | from multiprocessing import Pool 8 | 9 | from common import baseline_retrieval 10 | from aid import automatic_image_disambiguation, hard_cluster_selection 11 | from clue import clue 12 | from utils import get_dataset_queries, print_metrics, ptqdm 13 | import eval_metrics 14 | 15 | 16 | 17 | ## Cluster Selection Oracles ## 18 | 19 | 20 | def select_clusters_by_precision(query, clusters, min_precision = 0.5): 21 | 22 | selection = {} 23 | for i, c in enumerate(clusters): 24 | precision = sum(1 for id in c if id in query['relevant']) / float(len(c)) 25 | if precision >= min_precision: 26 | selection[i] = precision 27 | return sorted(selection.keys(), key = lambda i: selection[i], reverse = True) 28 | 29 | 30 | def select_best_cluster(query, clusters): 31 | 32 | return select_clusters_by_precision(query, clusters, 0.0)[:1] 33 | 34 | 35 | 36 | ## Query Disambiguation Methods ## 37 | 38 | METHODS = OrderedDict(( 39 | ('Baseline', baseline_retrieval), 40 | ('CLUE', clue), 41 | ('Hard-Select', hard_cluster_selection), 42 | ('AID', automatic_image_disambiguation) 43 | )) 44 | 45 | 46 | 47 | if __name__ == '__main__': 48 | 49 | np.random.seed(0) 50 | 51 | # Set up CLI argument parser 52 | parser = argparse.ArgumentParser(description = 'Evaluates the performance of various techniques for automatic image query disambiguation.', formatter_class = argparse.ArgumentDefaultsHelpFormatter) 53 | parser_general = parser.add_argument_group('General') 54 | parser_general.add_argument('methods', type = str, nargs = '*', default = list(METHODS.keys()), help = 'Methods to be tested (choices: {}).'.format(', '.join(list(METHODS.keys())))) 55 | parser_general.add_argument('--num_preview', type = int, default = 10, help = 'Number of images shown to the user for each suggested image sense.') 56 | parser_general.add_argument('--multiple', action = 'store_const', const = True, default = False, help = 'Allow selection of multiple suggested image senses.') 57 | parser_general.add_argument('--rounds', type = int, default = 5, help = 'Number of iterations of the experiment for more robustness against random initializations. Results will be averaged.') 58 | parser_data = parser.add_argument_group('Data') 59 | parser_data.add_argument('--gt_dir', type = str, default = 'mirflickr', help = 'Directory with the ground-truth label files.') 60 | parser_data.add_argument('--query_dir', type = str, default = None, help = 'Directory with the query list files.') 61 | parser_data.add_argument('--dup_file', type = str, default = 'mirflickr/duplicates.txt', help = 'File containing a list of IDs of duplicate images.') 62 | parser_data.add_argument('--feature_dump', type = str, default = 'features.npy', help = 'Path to a dump of the feature matrix of the dataset as created by extract_features.py.') 63 | parser_display = parser.add_argument_group('Results Format') 64 | parser_display.add_argument('--show_sd', action = 'store_const', const = True, default = False, help = 'Show table with standard deviation of performance metrics.') 65 | parser_display.add_argument('--csv', action = 'store_const', const = True, default = False, help = 'Output results as CSV instead of a table.') 66 | parser_display.add_argument('--plot_precision', action = 'store_const', const = True, default = False, help = 'Plot Precision@k versus several values of k.') 67 | parser_aid = parser.add_argument_group('AID') 68 | parser_aid.add_argument('--aid_gamma', type = float, default = None, 69 | help = 'Parameter gamma for distance adjustment function of AID. Larger values will require a more exact match of directions, while smaller values will lead to an adjustment of near-orthogonal samples as well.') 70 | parser_aid.add_argument('--aid_k', type = int, default = None, help = 'Number of top baseline results to be used for clustering by AID.') 71 | parser_aid.add_argument('--aid_n_clusters', type = int, default = None, help = 'Fixes the number of clusters generated by AID.') 72 | parser_aid.add_argument('--aid_max_clusters', type = int, default = 10, help = 'Maximum number of clusters generated by AID.') 73 | parser_clue = parser.add_argument_group('CLUE') 74 | parser_clue.add_argument('--clue_k', type = int, default = None, help = 'Number of top baseline results to be used for clustering by CLUE.') 75 | parser_clue.add_argument('--clue_max_clusters', type = int, default = 10, help = 'Maximum number of clusters generated by CLUE.') 76 | parser_clue.add_argument('--clue_T', type = float, default = None, help = 'N-Cut threshold for CLUE.') 77 | parser_hard_select = parser.add_argument_group('Hard Cluster Selection on the same clusters as AID') 78 | parser_hard_select.add_argument('--hard-select_k', type = int, default = None, help = 'Number of top baseline results to be used for clustering by Hard Cluster Selection.') 79 | parser_hard_select.add_argument('--hard-select_n_clusters', type = int, default = None, help = 'Fixes the number of clusters generated by Hard Cluster Selection.') 80 | parser_hard_select.add_argument('--hard-select_max_clusters', type = int, default = 10, help = 'Maximum number of clusters generated by Hard Cluster Selection.') 81 | args = parser.parse_args() 82 | 83 | # Parse algorithm-specific parameters 84 | algo_params = {} 85 | for arg_name, arg_value in vars(args).items(): 86 | arg_components = arg_name.split('_', 1) 87 | if (len(arg_components) > 1) and (arg_value is not None): 88 | algo_name = arg_components[0].lower() 89 | if algo_name in ('aid', 'clue', 'hard-select'): 90 | if algo_name not in algo_params: 91 | algo_params[algo_name] = {} 92 | algo_params[algo_name][arg_components[1]] = arg_value 93 | 94 | # Load features and queries 95 | queries = get_dataset_queries(args.gt_dir, args.query_dir, args.dup_file) 96 | if len(queries) == 0: 97 | print('Could not find any queries. Have you set --query_dir correctly?') 98 | exit() 99 | features = np.load(args.feature_dump) 100 | 101 | # Run benchmarks 102 | metrics = OrderedDict() 103 | precisions = OrderedDict() 104 | select_clusters_func = select_clusters_by_precision if args.multiple else select_best_cluster 105 | select_clusters = lambda query, clusters: select_clusters_func(query, [c[:args.num_preview] for c in clusters]) 106 | for r in ptqdm(range(args.rounds), desc = 'Rounds', total = args.rounds): 107 | for method in args.methods: 108 | 109 | params = algo_params[method.lower()] if method.lower() in algo_params else {} 110 | retrieved = METHODS[method](features, queries, select_clusters, show_progress = True, **params) 111 | 112 | if method not in precisions: 113 | precisions[method] = [] 114 | with Pool() as p: 115 | precisions[method].append(np.mean(list(p.starmap( 116 | eval_metrics.precision_at_k, 117 | ((range(1, 101), queries[qid]['relevant'], ret, set([queries[qid]['img_id']]) | (queries[qid]['ignore'] if 'ignore' in queries[qid] else set())) for qid, (ret, dist) in retrieved.items()) 118 | )), axis = 0)) 119 | 120 | if method not in metrics: 121 | metrics[method] = [] 122 | metrics[method].append(eval_metrics.avg_query_metrics({ qid : { 123 | 'relevant' : query['relevant'], 124 | 'retrieved' : retrieved[qid][0], 125 | 'ignore' : set([query['img_id']]) | (query['ignore'] if 'ignore' in query else set()) 126 | } for qid, query in queries.items() })) 127 | for pk in (1, 10, 50, 100): 128 | metrics[method][-1]['P@{}'.format(pk)] = precisions[method][-1][pk - 1] 129 | 130 | print_metrics(OrderedDict((method, { metric: np.mean([m[metric] for m in lists]) for metric in lists[0].keys() }) for method, lists in metrics.items()), tabular = not args.csv) 131 | 132 | if args.show_sd: 133 | print('\nStandard Deviation:') 134 | print_metrics(OrderedDict((method, { metric: np.std([m[metric] for m in lists]) for metric in lists[0].keys() }) for method, lists in metrics.items()), tabular = not args.csv) 135 | 136 | if args.plot_precision: 137 | import matplotlib.pyplot as plt 138 | x = np.arange(1, 101) 139 | for (method, prec), col in zip(precisions.items(), itertools.cycle('bgyrcm')): 140 | y = np.mean(prec, axis = 0) 141 | yerr = 1.959964 * np.std(prec, axis = 0) 142 | plt.plot(x, y, linestyle = '-', c = col, label = method) 143 | if yerr.max() > 0.01: 144 | plt.plot(x, y + yerr, linestyle = ':', linewidth = 1, c = col) 145 | plt.plot(x, y - yerr, linestyle = ':', linewidth = 1, c = col) 146 | 147 | plt.xlabel('Retrieved Images') 148 | plt.ylabel('Precision') 149 | plt.grid(linestyle = '--') 150 | plt.legend() 151 | plt.show() 152 | -------------------------------------------------------------------------------- /extract_features.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import caffe 3 | import argparse 4 | 5 | from sklearn.decomposition import PCA 6 | 7 | from utils import get_dataset_images, tqdm 8 | 9 | 10 | 11 | def extract_cnn_features(net, mean, img, layer, blob = None): 12 | """ Extracts image features from a CNN using Caffe. 13 | 14 | net - caffe.Net instance 15 | 16 | mean - A numpy array with the mean values for each channel. 17 | Channels must be in BGR order and the scale of values is assumed to be [0,255]. 18 | 19 | img - The image to extract features from, given either as numpy array in the format 20 | returned by `caffe.io.load_image` or as the filename of the image. 21 | 22 | layer - The name of the layer to forward the net to. 23 | 24 | blob - The name of the blob to extract features from. If set to `None`, the name of the layer will be used. 25 | A list of blob names can be given as well for extracting features from multiple blobs. 26 | 27 | Returns: If `imgs` was given as list, a list of results will be returned, otherwise a single result. 28 | A result is either a list of numpy arrays if `blob` was a list, or a single numpy array otherwise. 29 | """ 30 | 31 | # Check parameters 32 | if blob is None: 33 | blob = layer 34 | blobs = [blob] if isinstance(blob, str) else blob 35 | if layer not in net._layer_names: 36 | raise RuntimeError('Layer not found: {}'.format(layer)) 37 | for b in blobs: 38 | if b not in net.blobs: 39 | raise RuntimeError('Blob not found: {}'.format(b)) 40 | 41 | # Prepare pre-processor 42 | transformer = caffe.io.Transformer({'data' : net.blobs['data'].data.shape}) 43 | if mean is not None: 44 | transformer.set_mean('data', mean) # subtract the mean channel value 45 | transformer.set_transpose('data', (2,0,1)) # move image channels to outermost dimension 46 | transformer.set_raw_scale('data', 255) # rescale float-valued images from [0,1] to [0,255] 47 | transformer.set_channel_swap('data', (2,1,0)) # swap channels from RGB to BGR 48 | 49 | # Load image 50 | img = caffe.io.load_image(img) if isinstance(img, str) else img 51 | 52 | # Convert image to correct data format and copy the result to the input blob 53 | net.blobs['data'].data[...] = transformer.preprocess('data', img) 54 | 55 | # Forward the network and extract features 56 | net._forward(0, list(net._layer_names).index(layer)) 57 | if isinstance(blob, str): 58 | return net.blobs[blob].data[0].astype(np.float32) 59 | else: 60 | return [net.blobs[b].data[0].astype(np.float32) for b in blobs] 61 | 62 | 63 | 64 | if __name__ == '__main__': 65 | 66 | # Set up CLI argument parser 67 | parser = argparse.ArgumentParser(description = 'Extracts fc6 features from VGG16 for all images in the dataset, followed by PCA.', formatter_class = argparse.ArgumentDefaultsHelpFormatter) 68 | parser_general = parser.add_argument_group('General') 69 | parser_general.add_argument('--pca_dim', type = int, default = 512, help = 'Number of resulting feature dimensions after PCA.') 70 | parser_general.add_argument('--feature_dump', type = str, default = 'features.npy', help = 'Filename where the feature matrix will be stored.') 71 | parser_data = parser.add_argument_group('Data') 72 | parser_data.add_argument('--img_dir', type = str, default = 'mirflickr/mirflickr', help = 'Directory with the images in the dataset.') 73 | parser_cnn = parser.add_argument_group('CNN') 74 | parser_cnn.add_argument('--model', type = str, default = 'model/VGG_ILSVRC_16_layers_deploy.prototxt', help = 'Model file.') 75 | parser_cnn.add_argument('--weights', type = str, default = 'model/VGG_ILSVRC_16_layers.caffemodel', help = 'Weights file.') 76 | parser_cnn.add_argument('--mean', type = str, default = 'model/image_mean.txt', help = 'File with channel means.') 77 | parser_cnn.add_argument('--gpu', type = int, default = None, help = 'GPU device to be used.') 78 | args = parser.parse_args() 79 | 80 | # Load model and image list 81 | images = get_dataset_images(args.img_dir) 82 | if len(images) == 0: 83 | print('Could not find any images. Have you set --img_dir correctly?') 84 | exit() 85 | if args.gpu is not None: 86 | caffe.set_mode_gpu() 87 | caffe.set_device(args.gpu) 88 | else: 89 | caffe.set_mode_cpu() 90 | net = caffe.Net(args.model, caffe.TEST, weights = args.weights) 91 | mean = np.loadtxt(args.mean) 92 | 93 | # Extract features 94 | features = np.array([extract_cnn_features(net, mean, img, 'fc6') for img in tqdm(images, desc = 'Extracting CNN features...', dynamic_ncols = True)], dtype = np.float32) 95 | 96 | # Norm, PCA, Norm 97 | features /= np.linalg.norm(features, axis = 1, keepdims = True) 98 | features = PCA(args.pca_dim).fit_transform(features) 99 | features /= np.linalg.norm(features, axis = 1, keepdims = True) 100 | 101 | # Save features 102 | np.save(args.feature_dump, features.astype(np.float32, copy = False)) 103 | -------------------------------------------------------------------------------- /mirflickr/baby_query.txt: -------------------------------------------------------------------------------- 1 | 2075 2 | 22255 3 | 17772 4 | 17666 5 | 8624 6 | -------------------------------------------------------------------------------- /mirflickr/baby_r1.txt: -------------------------------------------------------------------------------- 1 | 140 2 | 185 3 | 229 4 | 231 5 | 640 6 | 681 7 | 780 8 | 1295 9 | 1345 10 | 1719 11 | 2075 12 | 2523 13 | 2853 14 | 2987 15 | 3430 16 | 3661 17 | 3783 18 | 3819 19 | 4228 20 | 4317 21 | 4916 22 | 5102 23 | 5604 24 | 5666 25 | 5969 26 | 6125 27 | 6337 28 | 6355 29 | 6614 30 | 6641 31 | 6720 32 | 6846 33 | 6953 34 | 8565 35 | 8624 36 | 8690 37 | 9000 38 | 9044 39 | 9091 40 | 9192 41 | 9377 42 | 9442 43 | 9468 44 | 9601 45 | 9730 46 | 9981 47 | 10148 48 | 10225 49 | 10612 50 | 10937 51 | 11241 52 | 11501 53 | 11648 54 | 11681 55 | 11736 56 | 13217 57 | 13243 58 | 13442 59 | 14363 60 | 14386 61 | 15053 62 | 15111 63 | 15180 64 | 15191 65 | 15422 66 | 15665 67 | 15734 68 | 15804 69 | 15811 70 | 15816 71 | 15963 72 | 16285 73 | 16295 74 | 16570 75 | 16614 76 | 16721 77 | 16774 78 | 17049 79 | 17410 80 | 17465 81 | 17484 82 | 17666 83 | 17772 84 | 17940 85 | 18011 86 | 18055 87 | 18558 88 | 18788 89 | 18821 90 | 18880 91 | 19061 92 | 19357 93 | 19542 94 | 19568 95 | 19595 96 | 20163 97 | 20242 98 | 20447 99 | 20522 100 | 20710 101 | 20883 102 | 21052 103 | 21338 104 | 21479 105 | 21973 106 | 22171 107 | 22226 108 | 22255 109 | 22328 110 | 22443 111 | 22597 112 | 22601 113 | 22882 114 | 23613 115 | 23860 116 | 24758 117 | -------------------------------------------------------------------------------- /mirflickr/bird_query.txt: -------------------------------------------------------------------------------- 1 | 6775 2 | 21020 3 | 10891 4 | 5851 5 | 14393 6 | -------------------------------------------------------------------------------- /mirflickr/bird_r1.txt: -------------------------------------------------------------------------------- 1 | 49 2 | 97 3 | 159 4 | 237 5 | 254 6 | 296 7 | 317 8 | 409 9 | 453 10 | 454 11 | 459 12 | 468 13 | 618 14 | 702 15 | 750 16 | 752 17 | 781 18 | 787 19 | 849 20 | 893 21 | 1015 22 | 1264 23 | 1386 24 | 1416 25 | 1448 26 | 1457 27 | 1460 28 | 1534 29 | 1617 30 | 1620 31 | 1724 32 | 1771 33 | 1806 34 | 1924 35 | 2111 36 | 2172 37 | 2195 38 | 2204 39 | 2210 40 | 2314 41 | 2404 42 | 2510 43 | 2527 44 | 2564 45 | 2643 46 | 2705 47 | 2796 48 | 2875 49 | 2883 50 | 2908 51 | 2933 52 | 2938 53 | 2981 54 | 3021 55 | 3114 56 | 3138 57 | 3194 58 | 3238 59 | 3334 60 | 3454 61 | 3557 62 | 3578 63 | 3612 64 | 3645 65 | 3654 66 | 3757 67 | 3865 68 | 3869 69 | 3910 70 | 3941 71 | 4061 72 | 4113 73 | 4117 74 | 4133 75 | 4143 76 | 4155 77 | 4167 78 | 4182 79 | 4197 80 | 4222 81 | 4232 82 | 4312 83 | 4313 84 | 4314 85 | 4334 86 | 4349 87 | 4366 88 | 4421 89 | 4434 90 | 4464 91 | 4504 92 | 4507 93 | 4512 94 | 4516 95 | 4523 96 | 4594 97 | 4605 98 | 4727 99 | 4733 100 | 4736 101 | 4748 102 | 4773 103 | 5134 104 | 5147 105 | 5178 106 | 5193 107 | 5218 108 | 5224 109 | 5278 110 | 5283 111 | 5295 112 | 5367 113 | 5372 114 | 5457 115 | 5466 116 | 5472 117 | 5561 118 | 5622 119 | 5637 120 | 5677 121 | 5771 122 | 5800 123 | 5806 124 | 5851 125 | 5866 126 | 5873 127 | 5929 128 | 5933 129 | 5989 130 | 6069 131 | 6075 132 | 6094 133 | 6134 134 | 6166 135 | 6169 136 | 6268 137 | 6313 138 | 6406 139 | 6518 140 | 6612 141 | 6629 142 | 6674 143 | 6707 144 | 6716 145 | 6752 146 | 6775 147 | 6820 148 | 6879 149 | 6889 150 | 6898 151 | 6943 152 | 6949 153 | 7148 154 | 7155 155 | 7196 156 | 7287 157 | 7318 158 | 7320 159 | 7367 160 | 7495 161 | 7528 162 | 7660 163 | 7780 164 | 7799 165 | 7812 166 | 7914 167 | 7967 168 | 7998 169 | 8016 170 | 8020 171 | 8039 172 | 8040 173 | 8067 174 | 8110 175 | 8170 176 | 8172 177 | 8254 178 | 8289 179 | 8330 180 | 8372 181 | 8467 182 | 8502 183 | 8520 184 | 8535 185 | 8674 186 | 8702 187 | 8727 188 | 8817 189 | 8853 190 | 8979 191 | 9020 192 | 9048 193 | 9105 194 | 9108 195 | 9120 196 | 9145 197 | 9151 198 | 9336 199 | 9366 200 | 9393 201 | 9430 202 | 9543 203 | 9638 204 | 9655 205 | 9673 206 | 9805 207 | 9837 208 | 9906 209 | 10019 210 | 10077 211 | 10102 212 | 10165 213 | 10181 214 | 10228 215 | 10261 216 | 10298 217 | 10332 218 | 10444 219 | 10457 220 | 10484 221 | 10529 222 | 10561 223 | 10688 224 | 10758 225 | 10777 226 | 10780 227 | 10817 228 | 10887 229 | 10891 230 | 10963 231 | 10973 232 | 11032 233 | 11050 234 | 11116 235 | 11120 236 | 11235 237 | 11304 238 | 11426 239 | 11664 240 | 11686 241 | 11715 242 | 11780 243 | 11832 244 | 11849 245 | 12018 246 | 12258 247 | 12364 248 | 12505 249 | 12511 250 | 12602 251 | 12844 252 | 12893 253 | 12901 254 | 13033 255 | 13092 256 | 13164 257 | 13189 258 | 13247 259 | 13265 260 | 13420 261 | 13453 262 | 13463 263 | 13490 264 | 13756 265 | 13790 266 | 13840 267 | 13881 268 | 13943 269 | 14105 270 | 14107 271 | 14140 272 | 14150 273 | 14246 274 | 14393 275 | 14435 276 | 14451 277 | 14509 278 | 14512 279 | 14514 280 | 14652 281 | 14756 282 | 14877 283 | 14887 284 | 14920 285 | 15220 286 | 15286 287 | 15357 288 | 15421 289 | 15616 290 | 15779 291 | 15781 292 | 15832 293 | 15849 294 | 15924 295 | 15998 296 | 16039 297 | 16138 298 | 16261 299 | 16382 300 | 16679 301 | 16711 302 | 16791 303 | 16827 304 | 16866 305 | 16871 306 | 16872 307 | 16955 308 | 17063 309 | 17089 310 | 17187 311 | 17233 312 | 17300 313 | 17311 314 | 17378 315 | 17547 316 | 17552 317 | 17712 318 | 17760 319 | 17811 320 | 17946 321 | 18044 322 | 18116 323 | 18141 324 | 18185 325 | 18196 326 | 18274 327 | 18277 328 | 18318 329 | 18319 330 | 18366 331 | 18367 332 | 18417 333 | 18492 334 | 18528 335 | 18536 336 | 18560 337 | 18572 338 | 18619 339 | 18691 340 | 18742 341 | 18782 342 | 18811 343 | 18818 344 | 18989 345 | 19022 346 | 19099 347 | 19101 348 | 19110 349 | 19211 350 | 19342 351 | 19359 352 | 19552 353 | 19602 354 | 19627 355 | 19685 356 | 19705 357 | 19763 358 | 19841 359 | 19939 360 | 19964 361 | 20036 362 | 20080 363 | 20162 364 | 20286 365 | 20323 366 | 20332 367 | 20334 368 | 20339 369 | 20445 370 | 20562 371 | 20609 372 | 20648 373 | 20657 374 | 20694 375 | 20803 376 | 20820 377 | 20950 378 | 21020 379 | 21080 380 | 21099 381 | 21296 382 | 21444 383 | 21490 384 | 21550 385 | 21566 386 | 21636 387 | 21666 388 | 21748 389 | 21752 390 | 21839 391 | 21943 392 | 21981 393 | 21982 394 | 22016 395 | 22022 396 | 22083 397 | 22129 398 | 22245 399 | 22344 400 | 22454 401 | 22485 402 | 22502 403 | 22534 404 | 22589 405 | 22592 406 | 22649 407 | 22722 408 | 22754 409 | 22758 410 | 22805 411 | 22822 412 | 22826 413 | 22884 414 | 22948 415 | 22985 416 | 22993 417 | 23003 418 | 23014 419 | 23061 420 | 23068 421 | 23082 422 | 23084 423 | 23092 424 | 23100 425 | 23152 426 | 23169 427 | 23177 428 | 23178 429 | 23187 430 | 23202 431 | 23236 432 | 23271 433 | 23284 434 | 23294 435 | 23309 436 | 23374 437 | 23525 438 | 23540 439 | 23553 440 | 23602 441 | 23772 442 | 23782 443 | 23787 444 | 23803 445 | 23809 446 | 23892 447 | 23904 448 | 23913 449 | 23916 450 | 23921 451 | 23999 452 | 24017 453 | 24019 454 | 24023 455 | 24025 456 | 24026 457 | 24048 458 | 24060 459 | 24070 460 | 24108 461 | 24154 462 | 24169 463 | 24260 464 | 24281 465 | 24304 466 | 24314 467 | 24383 468 | 24413 469 | 24446 470 | 24559 471 | 24575 472 | 24642 473 | 24647 474 | 24686 475 | 24687 476 | 24765 477 | 24774 478 | 24803 479 | 24853 480 | 24878 481 | 24894 482 | 24896 483 | 24897 484 | 24935 485 | -------------------------------------------------------------------------------- /mirflickr/car_query.txt: -------------------------------------------------------------------------------- 1 | 402 2 | 24845 3 | 16529 4 | 18672 5 | 15928 6 | -------------------------------------------------------------------------------- /mirflickr/car_r1.txt: -------------------------------------------------------------------------------- 1 | 47 2 | 56 3 | 60 4 | 72 5 | 137 6 | 184 7 | 211 8 | 336 9 | 401 10 | 402 11 | 462 12 | 810 13 | 814 14 | 851 15 | 853 16 | 933 17 | 989 18 | 1051 19 | 1130 20 | 1229 21 | 1305 22 | 1344 23 | 1361 24 | 1429 25 | 1475 26 | 1514 27 | 1522 28 | 1527 29 | 1757 30 | 1802 31 | 1869 32 | 1927 33 | 1947 34 | 2264 35 | 2326 36 | 2333 37 | 2382 38 | 2390 39 | 2393 40 | 2557 41 | 2609 42 | 2692 43 | 2822 44 | 2832 45 | 2880 46 | 2915 47 | 2960 48 | 3035 49 | 3218 50 | 3231 51 | 3330 52 | 3348 53 | 3370 54 | 3472 55 | 3513 56 | 3577 57 | 3598 58 | 3717 59 | 3832 60 | 3945 61 | 4018 62 | 4138 63 | 4326 64 | 4373 65 | 4662 66 | 4688 67 | 4705 68 | 4706 69 | 4709 70 | 4835 71 | 4837 72 | 4862 73 | 4881 74 | 4885 75 | 4922 76 | 5007 77 | 5068 78 | 5116 79 | 5129 80 | 5253 81 | 5314 82 | 5439 83 | 5460 84 | 5470 85 | 5495 86 | 5528 87 | 5631 88 | 5726 89 | 5923 90 | 5949 91 | 5999 92 | 6086 93 | 6236 94 | 6389 95 | 6691 96 | 6745 97 | 6754 98 | 6907 99 | 6964 100 | 6982 101 | 6989 102 | 6992 103 | 7096 104 | 7172 105 | 7215 106 | 7354 107 | 7468 108 | 7494 109 | 7523 110 | 7610 111 | 7628 112 | 7643 113 | 7784 114 | 7976 115 | 8024 116 | 8303 117 | 8352 118 | 8399 119 | 8417 120 | 8424 121 | 8568 122 | 8714 123 | 8861 124 | 8916 125 | 9059 126 | 9065 127 | 9166 128 | 9297 129 | 9306 130 | 9331 131 | 9337 132 | 9381 133 | 9395 134 | 9401 135 | 9419 136 | 9489 137 | 9736 138 | 9845 139 | 9848 140 | 10001 141 | 10128 142 | 10168 143 | 10251 144 | 10320 145 | 10350 146 | 10567 147 | 10572 148 | 10684 149 | 10827 150 | 10830 151 | 11031 152 | 11036 153 | 11048 154 | 11110 155 | 11113 156 | 11132 157 | 11138 158 | 11152 159 | 11206 160 | 11213 161 | 11322 162 | 11344 163 | 11518 164 | 11599 165 | 11622 166 | 11651 167 | 11916 168 | 11999 169 | 12009 170 | 12052 171 | 12064 172 | 12069 173 | 12122 174 | 12193 175 | 12245 176 | 12264 177 | 12323 178 | 12361 179 | 12396 180 | 12469 181 | 12482 182 | 12506 183 | 12525 184 | 12556 185 | 12620 186 | 12721 187 | 12734 188 | 12828 189 | 12835 190 | 12863 191 | 12910 192 | 12930 193 | 12947 194 | 12987 195 | 13072 196 | 13081 197 | 13188 198 | 13195 199 | 13215 200 | 13233 201 | 13297 202 | 13323 203 | 13351 204 | 13412 205 | 13473 206 | 13485 207 | 13523 208 | 13587 209 | 13634 210 | 13641 211 | 13661 212 | 13670 213 | 13762 214 | 13811 215 | 13816 216 | 13817 217 | 13824 218 | 13834 219 | 13841 220 | 13911 221 | 13919 222 | 13976 223 | 14037 224 | 14131 225 | 14148 226 | 14183 227 | 14279 228 | 14291 229 | 14304 230 | 14331 231 | 14344 232 | 14452 233 | 14454 234 | 14484 235 | 14492 236 | 14501 237 | 14560 238 | 14600 239 | 14631 240 | 14683 241 | 14741 242 | 14940 243 | 14996 244 | 15087 245 | 15265 246 | 15302 247 | 15375 248 | 15380 249 | 15467 250 | 15487 251 | 15538 252 | 15780 253 | 15917 254 | 15928 255 | 15945 256 | 15948 257 | 16041 258 | 16158 259 | 16254 260 | 16260 261 | 16291 262 | 16366 263 | 16414 264 | 16466 265 | 16504 266 | 16529 267 | 16609 268 | 16638 269 | 16647 270 | 16663 271 | 16671 272 | 16707 273 | 16819 274 | 16862 275 | 16867 276 | 16966 277 | 17102 278 | 17117 279 | 17160 280 | 17198 281 | 17231 282 | 17244 283 | 17292 284 | 17320 285 | 17389 286 | 17589 287 | 17593 288 | 17640 289 | 17672 290 | 17709 291 | 17817 292 | 17944 293 | 17957 294 | 18029 295 | 18048 296 | 18077 297 | 18188 298 | 18358 299 | 18493 300 | 18544 301 | 18646 302 | 18668 303 | 18672 304 | 18693 305 | 18749 306 | 18760 307 | 18875 308 | 18897 309 | 18985 310 | 19052 311 | 19135 312 | 19163 313 | 19332 314 | 19402 315 | 19462 316 | 19489 317 | 19562 318 | 19872 319 | 19913 320 | 20119 321 | 20201 322 | 20250 323 | 20383 324 | 20420 325 | 20427 326 | 20470 327 | 20654 328 | 20669 329 | 20688 330 | 20715 331 | 20766 332 | 20807 333 | 20810 334 | 20861 335 | 20887 336 | 20949 337 | 20952 338 | 21055 339 | 21101 340 | 21121 341 | 21208 342 | 21219 343 | 21272 344 | 21287 345 | 21288 346 | 21310 347 | 21432 348 | 21442 349 | 21587 350 | 21658 351 | 21767 352 | 21830 353 | 21883 354 | 22042 355 | 22149 356 | 22268 357 | 22303 358 | 22322 359 | 22434 360 | 22530 361 | 22467 362 | 22564 363 | 22623 364 | 22710 365 | 22772 366 | 23094 367 | 23119 368 | 23214 369 | 23264 370 | 23514 371 | 23660 372 | 23960 373 | 23964 374 | 24068 375 | 24334 376 | 24509 377 | 24601 378 | 24678 379 | 24816 380 | 24845 381 | -------------------------------------------------------------------------------- /mirflickr/clouds_query.txt: -------------------------------------------------------------------------------- 1 | 15932 2 | 23763 3 | 20791 4 | 22913 5 | 968 6 | -------------------------------------------------------------------------------- /mirflickr/clouds_r1.txt: -------------------------------------------------------------------------------- 1 | 15 2 | 21 3 | 31 4 | 33 5 | 77 6 | 81 7 | 93 8 | 103 9 | 116 10 | 165 11 | 173 12 | 179 13 | 187 14 | 191 15 | 206 16 | 221 17 | 233 18 | 249 19 | 253 20 | 295 21 | 302 22 | 306 23 | 307 24 | 337 25 | 345 26 | 367 27 | 368 28 | 373 29 | 385 30 | 400 31 | 405 32 | 417 33 | 426 34 | 446 35 | 464 36 | 473 37 | 485 38 | 499 39 | 521 40 | 538 41 | 549 42 | 553 43 | 598 44 | 616 45 | 645 46 | 657 47 | 664 48 | 666 49 | 688 50 | 700 51 | 712 52 | 716 53 | 729 54 | 759 55 | 762 56 | 783 57 | 794 58 | 796 59 | 797 60 | 818 61 | 820 62 | 867 63 | 883 64 | 932 65 | 966 66 | 968 67 | 970 68 | 982 69 | 990 70 | 1000 71 | 1009 72 | 1010 73 | 1021 74 | 1042 75 | 1068 76 | 1074 77 | 1089 78 | 1095 79 | 1113 80 | 1143 81 | 1149 82 | 1155 83 | 1157 84 | 1160 85 | 1162 86 | 1195 87 | 1199 88 | 1229 89 | 1242 90 | 1250 91 | 1251 92 | 1279 93 | 1284 94 | 1319 95 | 1347 96 | 1394 97 | 1414 98 | 1437 99 | 1450 100 | 1464 101 | 1466 102 | 1467 103 | 1488 104 | 1493 105 | 1499 106 | 1502 107 | 1506 108 | 1507 109 | 1527 110 | 1544 111 | 1549 112 | 1550 113 | 1559 114 | 1588 115 | 1596 116 | 1608 117 | 1611 118 | 1617 119 | 1627 120 | 1634 121 | 1696 122 | 1711 123 | 1730 124 | 1734 125 | 1743 126 | 1766 127 | 1775 128 | 1782 129 | 1816 130 | 1817 131 | 1904 132 | 1908 133 | 1992 134 | 1999 135 | 2044 136 | 2080 137 | 2092 138 | 2130 139 | 2140 140 | 2231 141 | 2236 142 | 2259 143 | 2283 144 | 2323 145 | 2331 146 | 2337 147 | 2348 148 | 2449 149 | 2452 150 | 2455 151 | 2470 152 | 2500 153 | 2540 154 | 2581 155 | 2603 156 | 2604 157 | 2607 158 | 2610 159 | 2665 160 | 2679 161 | 2686 162 | 2760 163 | 2845 164 | 2862 165 | 2866 166 | 2913 167 | 2985 168 | 3000 169 | 3076 170 | 3124 171 | 3138 172 | 3163 173 | 3216 174 | 3226 175 | 3294 176 | 3296 177 | 3317 178 | 3328 179 | 3341 180 | 3370 181 | 3377 182 | 3382 183 | 3386 184 | 3408 185 | 3460 186 | 3493 187 | 3505 188 | 3509 189 | 3568 190 | 3587 191 | 3625 192 | 3636 193 | 3665 194 | 3719 195 | 3737 196 | 3776 197 | 3839 198 | 3842 199 | 3859 200 | 3909 201 | 3944 202 | 3963 203 | 4017 204 | 4036 205 | 4128 206 | 4156 207 | 4164 208 | 4190 209 | 4251 210 | 4272 211 | 4302 212 | 4303 213 | 4304 214 | 4306 215 | 4390 216 | 4403 217 | 4415 218 | 4429 219 | 4462 220 | 4567 221 | 4573 222 | 4582 223 | 4591 224 | 4602 225 | 4638 226 | 4654 227 | 4658 228 | 4711 229 | 4740 230 | 4792 231 | 4814 232 | 4825 233 | 4905 234 | 4919 235 | 4933 236 | 4941 237 | 4945 238 | 4958 239 | 4993 240 | 5004 241 | 5009 242 | 5019 243 | 5025 244 | 5026 245 | 5038 246 | 5040 247 | 5063 248 | 5068 249 | 5069 250 | 5082 251 | 5097 252 | 5162 253 | 5168 254 | 5177 255 | 5182 256 | 5212 257 | 5221 258 | 5225 259 | 5245 260 | 5281 261 | 5284 262 | 5327 263 | 5345 264 | 5374 265 | 5399 266 | 5434 267 | 5476 268 | 5478 269 | 5515 270 | 5570 271 | 5581 272 | 5583 273 | 5587 274 | 5596 275 | 5625 276 | 5630 277 | 5649 278 | 5657 279 | 5658 280 | 5672 281 | 5682 282 | 5704 283 | 5736 284 | 5753 285 | 5770 286 | 5774 287 | 5778 288 | 5789 289 | 5791 290 | 5792 291 | 5824 292 | 5835 293 | 5855 294 | 5910 295 | 5922 296 | 5955 297 | 5963 298 | 5965 299 | 5981 300 | 5990 301 | 6026 302 | 6030 303 | 6034 304 | 6037 305 | 6040 306 | 6047 307 | 6052 308 | 6101 309 | 6112 310 | 6114 311 | 6115 312 | 6134 313 | 6188 314 | 6202 315 | 6207 316 | 6235 317 | 6277 318 | 6280 319 | 6301 320 | 6315 321 | 6358 322 | 6398 323 | 6408 324 | 6415 325 | 6451 326 | 6478 327 | 6482 328 | 6494 329 | 6501 330 | 6507 331 | 6533 332 | 6545 333 | 6558 334 | 6574 335 | 6585 336 | 6588 337 | 6589 338 | 6591 339 | 6646 340 | 6648 341 | 6652 342 | 6664 343 | 6684 344 | 6728 345 | 6748 346 | 6781 347 | 6795 348 | 6801 349 | 6828 350 | 6853 351 | 6883 352 | 6884 353 | 6887 354 | 6909 355 | 6952 356 | 6959 357 | 6972 358 | 7009 359 | 7024 360 | 7026 361 | 7093 362 | 7104 363 | 7120 364 | 7140 365 | 7158 366 | 7171 367 | 7172 368 | 7176 369 | 7178 370 | 7179 371 | 7191 372 | 7201 373 | 7247 374 | 7250 375 | 7259 376 | 7267 377 | 7284 378 | 7314 379 | 7332 380 | 7337 381 | 7341 382 | 7381 383 | 7423 384 | 7431 385 | 7456 386 | 7458 387 | 7503 388 | 7508 389 | 7532 390 | 7558 391 | 7565 392 | 7588 393 | 7606 394 | 7611 395 | 7623 396 | 7629 397 | 7639 398 | 7656 399 | 7670 400 | 7676 401 | 7695 402 | 7709 403 | 7713 404 | 7747 405 | 7782 406 | 7786 407 | 7798 408 | 7806 409 | 7837 410 | 7865 411 | 7882 412 | 7885 413 | 7909 414 | 7911 415 | 7912 416 | 7950 417 | 7958 418 | 7982 419 | 8138 420 | 8151 421 | 8161 422 | 8166 423 | 8184 424 | 8205 425 | 8269 426 | 8299 427 | 8311 428 | 8317 429 | 8323 430 | 8326 431 | 8354 432 | 8355 433 | 8375 434 | 8378 435 | 8403 436 | 8421 437 | 8439 438 | 8446 439 | 8488 440 | 8495 441 | 8501 442 | 8513 443 | 8541 444 | 8571 445 | 8591 446 | 8621 447 | 8641 448 | 8653 449 | 8656 450 | 8657 451 | 8673 452 | 8740 453 | 8746 454 | 8747 455 | 8753 456 | 8790 457 | 8793 458 | 8800 459 | 8822 460 | 8838 461 | 8850 462 | 8864 463 | 8886 464 | 8893 465 | 8912 466 | 8922 467 | 8952 468 | 8975 469 | 9043 470 | 9047 471 | 9063 472 | 9080 473 | 9100 474 | 9104 475 | 9130 476 | 9194 477 | 9209 478 | 9227 479 | 9229 480 | 9235 481 | 9247 482 | 9274 483 | 9312 484 | 9314 485 | 9319 486 | 9323 487 | 9325 488 | 9367 489 | 9405 490 | 9467 491 | 9475 492 | 9506 493 | 9511 494 | 9526 495 | 9528 496 | 9583 497 | 9594 498 | 9608 499 | 9610 500 | 9624 501 | 9632 502 | 9639 503 | 9641 504 | 9663 505 | 9716 506 | 9788 507 | 9810 508 | 9812 509 | 9822 510 | 9870 511 | 9876 512 | 9891 513 | 9892 514 | 9915 515 | 9926 516 | 9970 517 | 9977 518 | 9991 519 | 10006 520 | 10025 521 | 10036 522 | 10041 523 | 10044 524 | 10045 525 | 10047 526 | 10052 527 | 10080 528 | 10105 529 | 10115 530 | 10124 531 | 10147 532 | 10154 533 | 10158 534 | 10166 535 | 10176 536 | 10230 537 | 10249 538 | 10261 539 | 10266 540 | 10283 541 | 10293 542 | 10295 543 | 10296 544 | 10305 545 | 10312 546 | 10368 547 | 10384 548 | 10388 549 | 10395 550 | 10398 551 | 10404 552 | 10408 553 | 10410 554 | 10465 555 | 10470 556 | 10472 557 | 10479 558 | 10512 559 | 10513 560 | 10526 561 | 10609 562 | 10628 563 | 10678 564 | 10734 565 | 10757 566 | 10759 567 | 10774 568 | 10809 569 | 10841 570 | 10857 571 | 10861 572 | 10874 573 | 10909 574 | 10920 575 | 10943 576 | 10977 577 | 10986 578 | 11063 579 | 11071 580 | 11083 581 | 11102 582 | 11119 583 | 11134 584 | 11147 585 | 11155 586 | 11164 587 | 11175 588 | 11203 589 | 11219 590 | 11239 591 | 11243 592 | 11252 593 | 11287 594 | 11300 595 | 11332 596 | 11373 597 | 11384 598 | 11386 599 | 11412 600 | 11448 601 | 11496 602 | 11544 603 | 11546 604 | 11554 605 | 11570 606 | 11573 607 | 11578 608 | 11650 609 | 11674 610 | 11699 611 | 11740 612 | 11745 613 | 11747 614 | 11749 615 | 11756 616 | 11782 617 | 11815 618 | 11827 619 | 11846 620 | 11847 621 | 11856 622 | 11869 623 | 11887 624 | 11948 625 | 11955 626 | 11968 627 | 12009 628 | 12018 629 | 12066 630 | 12085 631 | 12162 632 | 12166 633 | 12183 634 | 12273 635 | 12276 636 | 12291 637 | 12317 638 | 12329 639 | 12331 640 | 12334 641 | 12339 642 | 12357 643 | 12373 644 | 12398 645 | 12411 646 | 12418 647 | 12450 648 | 12464 649 | 12468 650 | 12482 651 | 12486 652 | 12493 653 | 12506 654 | 12511 655 | 12529 656 | 12533 657 | 12547 658 | 12562 659 | 12580 660 | 12593 661 | 12636 662 | 12647 663 | 12690 664 | 12693 665 | 12709 666 | 12714 667 | 12761 668 | 12766 669 | 12814 670 | 12837 671 | 12842 672 | 12855 673 | 12857 674 | 12889 675 | 12898 676 | 12921 677 | 13001 678 | 13035 679 | 13039 680 | 13061 681 | 13062 682 | 13083 683 | 13094 684 | 13111 685 | 13125 686 | 13138 687 | 13193 688 | 13228 689 | 13246 690 | 13252 691 | 13261 692 | 13310 693 | 13327 694 | 13340 695 | 13341 696 | 13351 697 | 13368 698 | 13369 699 | 13402 700 | 13412 701 | 13417 702 | 13494 703 | 13509 704 | 13540 705 | 13565 706 | 13568 707 | 13574 708 | 13630 709 | 13637 710 | 13638 711 | 13673 712 | 13677 713 | 13700 714 | 13713 715 | 13714 716 | 13715 717 | 13718 718 | 13725 719 | 13727 720 | 13772 721 | 13775 722 | 13791 723 | 13797 724 | 13806 725 | 13823 726 | 13869 727 | 13870 728 | 13898 729 | 13946 730 | 13961 731 | 13969 732 | 13977 733 | 14051 734 | 14083 735 | 14097 736 | 14157 737 | 14158 738 | 14177 739 | 14196 740 | 14204 741 | 14221 742 | 14222 743 | 14227 744 | 14237 745 | 14242 746 | 14243 747 | 14280 748 | 14315 749 | 14319 750 | 14360 751 | 14401 752 | 14402 753 | 14409 754 | 14426 755 | 14433 756 | 14457 757 | 14464 758 | 14480 759 | 14484 760 | 14495 761 | 14541 762 | 14555 763 | 14569 764 | 14606 765 | 14609 766 | 14613 767 | 14648 768 | 14656 769 | 14686 770 | 14719 771 | 14730 772 | 14784 773 | 14786 774 | 14791 775 | 14800 776 | 14830 777 | 14897 778 | 14917 779 | 15014 780 | 15024 781 | 15064 782 | 15097 783 | 15121 784 | 15127 785 | 15129 786 | 15133 787 | 15137 788 | 15203 789 | 15208 790 | 15210 791 | 15268 792 | 15277 793 | 15292 794 | 15307 795 | 15320 796 | 15395 797 | 15404 798 | 15412 799 | 15420 800 | 15423 801 | 15439 802 | 15450 803 | 15459 804 | 15463 805 | 15485 806 | 15511 807 | 15538 808 | 15575 809 | 15587 810 | 15594 811 | 15611 812 | 15614 813 | 15641 814 | 15687 815 | 15693 816 | 15709 817 | 15716 818 | 15737 819 | 15764 820 | 15802 821 | 15809 822 | 15827 823 | 15851 824 | 15854 825 | 15910 826 | 15926 827 | 15932 828 | 15933 829 | 15935 830 | 15948 831 | 15960 832 | 15981 833 | 15995 834 | 16014 835 | 16023 836 | 16037 837 | 16046 838 | 16065 839 | 16066 840 | 16095 841 | 16098 842 | 16107 843 | 16128 844 | 16131 845 | 16149 846 | 16157 847 | 16166 848 | 16170 849 | 16177 850 | 16187 851 | 16188 852 | 16196 853 | 16203 854 | 16219 855 | 16245 856 | 16247 857 | 16249 858 | 16265 859 | 16276 860 | 16367 861 | 16377 862 | 16396 863 | 16432 864 | 16443 865 | 16448 866 | 16449 867 | 16454 868 | 16477 869 | 16501 870 | 16513 871 | 16522 872 | 16524 873 | 16554 874 | 16563 875 | 16574 876 | 16580 877 | 16591 878 | 16627 879 | 16628 880 | 16633 881 | 16638 882 | 16666 883 | 16668 884 | 16680 885 | 16684 886 | 16696 887 | 16698 888 | 16717 889 | 16734 890 | 16780 891 | 16784 892 | 16829 893 | 16831 894 | 16861 895 | 16865 896 | 16881 897 | 16887 898 | 16888 899 | 16895 900 | 16900 901 | 16918 902 | 16943 903 | 16995 904 | 16998 905 | 17002 906 | 17006 907 | 17019 908 | 17031 909 | 17034 910 | 17052 911 | 17054 912 | 17060 913 | 17092 914 | 17110 915 | 17153 916 | 17174 917 | 17183 918 | 17199 919 | 17200 920 | 17215 921 | 17235 922 | 17253 923 | 17255 924 | 17287 925 | 17327 926 | 17330 927 | 17338 928 | 17339 929 | 17342 930 | 17355 931 | 17370 932 | 17375 933 | 17380 934 | 17404 935 | 17445 936 | 17448 937 | 17525 938 | 17526 939 | 17538 940 | 17540 941 | 17544 942 | 17558 943 | 17565 944 | 17576 945 | 17591 946 | 17597 947 | 17625 948 | 17646 949 | 17651 950 | 17652 951 | 17689 952 | 17729 953 | 17735 954 | 17742 955 | 17747 956 | 17750 957 | 17783 958 | 17802 959 | 17815 960 | 17827 961 | 17837 962 | 17847 963 | 17954 964 | 17959 965 | 17983 966 | 17990 967 | 17994 968 | 18039 969 | 18056 970 | 18061 971 | 18070 972 | 18076 973 | 18104 974 | 18108 975 | 18127 976 | 18132 977 | 18142 978 | 18149 979 | 18157 980 | 18166 981 | 18180 982 | 18183 983 | 18208 984 | 18228 985 | 18231 986 | 18233 987 | 18240 988 | 18251 989 | 18264 990 | 18289 991 | 18291 992 | 18306 993 | 18310 994 | 18311 995 | 18319 996 | 18330 997 | 18338 998 | 18342 999 | 18346 1000 | 18356 1001 | 18369 1002 | 18371 1003 | 18444 1004 | 18447 1005 | 18467 1006 | 18472 1007 | 18474 1008 | 18487 1009 | 18527 1010 | 18541 1011 | 18549 1012 | 18550 1013 | 18567 1014 | 18570 1015 | 18590 1016 | 18596 1017 | 18599 1018 | 18603 1019 | 18606 1020 | 18607 1021 | 18619 1022 | 18622 1023 | 18670 1024 | 18698 1025 | 18711 1026 | 18725 1027 | 18759 1028 | 18761 1029 | 18762 1030 | 18766 1031 | 18842 1032 | 18853 1033 | 18863 1034 | 18885 1035 | 18903 1036 | 18905 1037 | 18922 1038 | 18926 1039 | 18939 1040 | 18940 1041 | 18941 1042 | 18947 1043 | 18948 1044 | 18955 1045 | 18969 1046 | 18974 1047 | 18990 1048 | 18993 1049 | 18995 1050 | 18997 1051 | 19004 1052 | 19020 1053 | 19035 1054 | 19036 1055 | 19087 1056 | 19124 1057 | 19132 1058 | 19143 1059 | 19160 1060 | 19205 1061 | 19256 1062 | 19257 1063 | 19290 1064 | 19302 1065 | 19352 1066 | 19373 1067 | 19388 1068 | 19443 1069 | 19453 1070 | 19455 1071 | 19466 1072 | 19490 1073 | 19496 1074 | 19509 1075 | 19521 1076 | 19566 1077 | 19576 1078 | 19586 1079 | 19699 1080 | 19710 1081 | 19724 1082 | 19735 1083 | 19759 1084 | 19772 1085 | 19791 1086 | 19793 1087 | 19863 1088 | 19866 1089 | 19877 1090 | 19935 1091 | 19937 1092 | 19957 1093 | 19966 1094 | 19973 1095 | 19996 1096 | 20015 1097 | 20023 1098 | 20055 1099 | 20095 1100 | 20111 1101 | 20154 1102 | 20155 1103 | 20195 1104 | 20207 1105 | 20259 1106 | 20298 1107 | 20319 1108 | 20347 1109 | 20389 1110 | 20402 1111 | 20441 1112 | 20486 1113 | 20499 1114 | 20523 1115 | 20569 1116 | 20579 1117 | 20590 1118 | 20593 1119 | 20613 1120 | 20649 1121 | 20716 1122 | 20727 1123 | 20791 1124 | 20796 1125 | 20800 1126 | 20811 1127 | 20912 1128 | 20931 1129 | 20942 1130 | 20969 1131 | 20994 1132 | 21011 1133 | 21018 1134 | 21032 1135 | 21068 1136 | 21092 1137 | 21104 1138 | 21119 1139 | 21126 1140 | 21195 1141 | 21231 1142 | 21281 1143 | 21300 1144 | 21353 1145 | 21433 1146 | 21473 1147 | 21499 1148 | 21547 1149 | 21559 1150 | 21601 1151 | 21616 1152 | 21622 1153 | 21654 1154 | 21678 1155 | 21682 1156 | 21698 1157 | 21701 1158 | 21771 1159 | 21777 1160 | 21797 1161 | 21804 1162 | 21814 1163 | 21820 1164 | 21835 1165 | 21839 1166 | 21846 1167 | 21866 1168 | 21878 1169 | 21881 1170 | 21887 1171 | 21897 1172 | 21911 1173 | 21916 1174 | 21933 1175 | 21941 1176 | 21971 1177 | 21993 1178 | 22015 1179 | 22033 1180 | 22070 1181 | 22127 1182 | 22148 1183 | 22150 1184 | 22161 1185 | 22168 1186 | 22199 1187 | 22208 1188 | 22240 1189 | 22258 1190 | 22304 1191 | 22331 1192 | 22358 1193 | 22435 1194 | 22447 1195 | 22448 1196 | 22453 1197 | 22469 1198 | 22528 1199 | 22536 1200 | 22552 1201 | 22555 1202 | 22569 1203 | 22581 1204 | 22633 1205 | 22704 1206 | 22728 1207 | 22740 1208 | 22751 1209 | 22775 1210 | 22782 1211 | 22783 1212 | 22857 1213 | 22863 1214 | 22903 1215 | 22906 1216 | 22912 1217 | 22913 1218 | 22924 1219 | 22926 1220 | 22939 1221 | 22951 1222 | 22956 1223 | 22984 1224 | 23000 1225 | 23006 1226 | 23018 1227 | 23045 1228 | 23070 1229 | 23099 1230 | 23112 1231 | 23114 1232 | 23128 1233 | 23159 1234 | 23165 1235 | 23176 1236 | 23184 1237 | 23186 1238 | 23197 1239 | 23199 1240 | 23210 1241 | 23241 1242 | 23247 1243 | 23259 1244 | 23262 1245 | 23289 1246 | 23292 1247 | 23369 1248 | 23384 1249 | 23388 1250 | 23396 1251 | 23404 1252 | 23408 1253 | 23416 1254 | 23420 1255 | 23431 1256 | 23497 1257 | 23583 1258 | 23584 1259 | 23592 1260 | 23621 1261 | 23622 1262 | 23626 1263 | 23637 1264 | 23686 1265 | 23697 1266 | 23706 1267 | 23719 1268 | 23757 1269 | 23763 1270 | 23776 1271 | 23839 1272 | 23847 1273 | 23852 1274 | 23855 1275 | 23859 1276 | 23881 1277 | 23885 1278 | 23902 1279 | 23914 1280 | 23937 1281 | 23954 1282 | 23955 1283 | 23976 1284 | 23980 1285 | 23993 1286 | 23995 1287 | 24054 1288 | 24072 1289 | 24095 1290 | 24104 1291 | 24125 1292 | 24148 1293 | 24157 1294 | 24170 1295 | 24184 1296 | 24191 1297 | 24219 1298 | 24231 1299 | 24242 1300 | 24259 1301 | 24284 1302 | 24288 1303 | 24330 1304 | 24343 1305 | 24364 1306 | 24368 1307 | 24391 1308 | 24396 1309 | 24410 1310 | 24429 1311 | 24435 1312 | 24439 1313 | 24458 1314 | 24462 1315 | 24473 1316 | 24489 1317 | 24498 1318 | 24585 1319 | 24600 1320 | 24605 1321 | 24606 1322 | 24648 1323 | 24672 1324 | 24685 1325 | 24702 1326 | 24711 1327 | 24713 1328 | 24729 1329 | 24730 1330 | 24735 1331 | 24741 1332 | 24743 1333 | 24776 1334 | 24777 1335 | 24789 1336 | 24795 1337 | 24836 1338 | 24848 1339 | 24879 1340 | 24882 1341 | 24894 1342 | 24913 1343 | 24927 1344 | 24931 1345 | 24933 1346 | 24939 1347 | 24951 1348 | 24963 1349 | 24985 1350 | 24998 1351 | -------------------------------------------------------------------------------- /mirflickr/dog_query.txt: -------------------------------------------------------------------------------- 1 | 18179 2 | 12713 3 | 2460 4 | 12810 5 | 15264 6 | -------------------------------------------------------------------------------- /mirflickr/dog_r1.txt: -------------------------------------------------------------------------------- 1 | 121 2 | 129 3 | 309 4 | 327 5 | 343 6 | 410 7 | 443 8 | 505 9 | 566 10 | 743 11 | 833 12 | 909 13 | 1037 14 | 1082 15 | 1098 16 | 1112 17 | 1115 18 | 1184 19 | 1237 20 | 1244 21 | 1291 22 | 1314 23 | 1335 24 | 1350 25 | 1352 26 | 1399 27 | 1470 28 | 1548 29 | 1636 30 | 1647 31 | 1737 32 | 1798 33 | 1811 34 | 1814 35 | 1953 36 | 1969 37 | 1983 38 | 2007 39 | 2049 40 | 2108 41 | 2118 42 | 2170 43 | 2177 44 | 2178 45 | 2226 46 | 2293 47 | 2302 48 | 2318 49 | 2332 50 | 2341 51 | 2343 52 | 2406 53 | 2410 54 | 2411 55 | 2412 56 | 2414 57 | 2421 58 | 2460 59 | 2606 60 | 2608 61 | 2616 62 | 2639 63 | 2677 64 | 2721 65 | 2722 66 | 2770 67 | 2851 68 | 2858 69 | 2935 70 | 2979 71 | 2992 72 | 3074 73 | 3079 74 | 3113 75 | 3119 76 | 3124 77 | 3132 78 | 3150 79 | 3183 80 | 3204 81 | 3361 82 | 3367 83 | 3372 84 | 3415 85 | 3416 86 | 3483 87 | 3542 88 | 3563 89 | 3691 90 | 3694 91 | 3718 92 | 3760 93 | 3764 94 | 3853 95 | 3854 96 | 3929 97 | 3965 98 | 3976 99 | 4014 100 | 4027 101 | 4042 102 | 4068 103 | 4121 104 | 4184 105 | 4209 106 | 4244 107 | 4305 108 | 4311 109 | 4318 110 | 4395 111 | 4402 112 | 4447 113 | 4498 114 | 4546 115 | 4632 116 | 4677 117 | 4684 118 | 4685 119 | 4693 120 | 4706 121 | 4923 122 | 4930 123 | 4984 124 | 5012 125 | 5035 126 | 5061 127 | 5084 128 | 5113 129 | 5117 130 | 5237 131 | 5286 132 | 5333 133 | 5388 134 | 5418 135 | 5589 136 | 5702 137 | 5714 138 | 5717 139 | 5755 140 | 5810 141 | 5888 142 | 5982 143 | 6033 144 | 6038 145 | 6048 146 | 6064 147 | 6089 148 | 6187 149 | 6190 150 | 6266 151 | 6335 152 | 6349 153 | 6368 154 | 6373 155 | 6444 156 | 6450 157 | 6483 158 | 6510 159 | 6658 160 | 6733 161 | 6756 162 | 6768 163 | 6776 164 | 6796 165 | 6853 166 | 6905 167 | 6908 168 | 6967 169 | 6969 170 | 6973 171 | 6994 172 | 7064 173 | 7206 174 | 7238 175 | 7285 176 | 7299 177 | 7323 178 | 7360 179 | 7374 180 | 7482 181 | 7507 182 | 7534 183 | 7555 184 | 7638 185 | 7654 186 | 7718 187 | 7729 188 | 7752 189 | 7796 190 | 7827 191 | 7859 192 | 7890 193 | 7968 194 | 7984 195 | 8027 196 | 8074 197 | 8126 198 | 8258 199 | 8460 200 | 8473 201 | 8482 202 | 8486 203 | 8499 204 | 8529 205 | 8570 206 | 8632 207 | 8637 208 | 8718 209 | 8721 210 | 8723 211 | 8829 212 | 8839 213 | 8865 214 | 8902 215 | 8909 216 | 8987 217 | 9055 218 | 9093 219 | 9106 220 | 9144 221 | 9189 222 | 9207 223 | 9266 224 | 9270 225 | 9303 226 | 9391 227 | 9398 228 | 9404 229 | 9407 230 | 9423 231 | 9446 232 | 9493 233 | 9530 234 | 9532 235 | 9591 236 | 9677 237 | 9678 238 | 9727 239 | 9813 240 | 9845 241 | 9950 242 | 10003 243 | 10081 244 | 10219 245 | 10440 246 | 10480 247 | 10557 248 | 10581 249 | 10601 250 | 10696 251 | 10730 252 | 10736 253 | 10878 254 | 10934 255 | 10972 256 | 11107 257 | 11159 258 | 11228 259 | 11240 260 | 11257 261 | 11306 262 | 11314 263 | 11390 264 | 11492 265 | 11557 266 | 11773 267 | 11808 268 | 11809 269 | 11837 270 | 11841 271 | 11919 272 | 11920 273 | 11923 274 | 12022 275 | 12067 276 | 12090 277 | 12198 278 | 12217 279 | 12229 280 | 12239 281 | 12286 282 | 12332 283 | 12404 284 | 12405 285 | 12553 286 | 12583 287 | 12621 288 | 12639 289 | 12708 290 | 12713 291 | 12744 292 | 12747 293 | 12801 294 | 12810 295 | 12914 296 | 13011 297 | 13040 298 | 13152 299 | 13279 300 | 13361 301 | 13367 302 | 13427 303 | 13517 304 | 13567 305 | 13593 306 | 13645 307 | 13885 308 | 14103 309 | 14108 310 | 14153 311 | 14165 312 | 14313 313 | 14367 314 | 14406 315 | 14500 316 | 14655 317 | 14672 318 | 14709 319 | 14723 320 | 14876 321 | 14948 322 | 14957 323 | 14968 324 | 15095 325 | 15217 326 | 15264 327 | 15293 328 | 15325 329 | 15340 330 | 15382 331 | 15392 332 | 15393 333 | 15427 334 | 15457 335 | 15530 336 | 15565 337 | 15638 338 | 15707 339 | 15777 340 | 15814 341 | 15825 342 | 15830 343 | 15895 344 | 15916 345 | 15959 346 | 16013 347 | 16034 348 | 16054 349 | 16120 350 | 16172 351 | 16175 352 | 16229 353 | 16402 354 | 16428 355 | 16437 356 | 16474 357 | 16527 358 | 16539 359 | 16605 360 | 16640 361 | 16645 362 | 16676 363 | 16731 364 | 16748 365 | 16773 366 | 16892 367 | 16933 368 | 17058 369 | 17072 370 | 17147 371 | 17165 372 | 17237 373 | 17248 374 | 17304 375 | 17334 376 | 17409 377 | 17419 378 | 17474 379 | 17482 380 | 17485 381 | 17497 382 | 17509 383 | 17521 384 | 17562 385 | 17572 386 | 17611 387 | 17653 388 | 17665 389 | 17744 390 | 17763 391 | 17776 392 | 17803 393 | 17836 394 | 17843 395 | 17884 396 | 17890 397 | 17955 398 | 18107 399 | 18179 400 | 18210 401 | 18222 402 | 18250 403 | 18283 404 | 18295 405 | 18343 406 | 18390 407 | 18391 408 | 18402 409 | 18491 410 | 18629 411 | 18640 412 | 18675 413 | 18771 414 | 18843 415 | 18851 416 | 18870 417 | 18962 418 | 18963 419 | 18967 420 | 18981 421 | 19148 422 | 19182 423 | 19186 424 | 19196 425 | 19244 426 | 19247 427 | 19259 428 | 19312 429 | 19347 430 | 19355 431 | 19365 432 | 19366 433 | 19378 434 | 19417 435 | 19536 436 | 19582 437 | 19608 438 | 19633 439 | 19663 440 | 19690 441 | 19694 442 | 19703 443 | 19722 444 | 19770 445 | 19813 446 | 19834 447 | 19936 448 | 19946 449 | 20059 450 | 20099 451 | 20109 452 | 20130 453 | 20401 454 | 20498 455 | 20516 456 | 20543 457 | 20602 458 | 20603 459 | 20622 460 | 20681 461 | 20724 462 | 20755 463 | 20814 464 | 20831 465 | 20847 466 | 20870 467 | 20895 468 | 20922 469 | 20978 470 | 21125 471 | 21178 472 | 21209 473 | 21328 474 | 21391 475 | 21402 476 | 21446 477 | 21465 478 | 21495 479 | 21529 480 | 21590 481 | 21630 482 | 21646 483 | 21758 484 | 21759 485 | 21784 486 | 21793 487 | 21824 488 | 21833 489 | 21844 490 | 21871 491 | 21932 492 | 21938 493 | 21947 494 | 22000 495 | 22061 496 | 22079 497 | 22109 498 | 22114 499 | 22142 500 | 22145 501 | 22174 502 | 22187 503 | 22197 504 | 22207 505 | 22224 506 | 22244 507 | 22298 508 | 22376 509 | 22423 510 | 22430 511 | 22495 512 | 22585 513 | 22595 514 | 22604 515 | 22631 516 | 22645 517 | 22708 518 | 22738 519 | 22860 520 | 22942 521 | 22994 522 | 23048 523 | 23074 524 | 23091 525 | 23147 526 | 23151 527 | 23230 528 | 23253 529 | 23269 530 | 23275 531 | 23276 532 | 23335 533 | 23415 534 | 23428 535 | 23453 536 | 23476 537 | 23494 538 | 23513 539 | 23521 540 | 23557 541 | 23608 542 | 23611 543 | 23638 544 | 23647 545 | 23649 546 | 23656 547 | 23661 548 | 23666 549 | 23764 550 | 23774 551 | 23775 552 | 23840 553 | 23846 554 | 23862 555 | 23910 556 | 23934 557 | 23958 558 | 23969 559 | 23983 560 | 24008 561 | 24092 562 | 24111 563 | 24138 564 | 24152 565 | 24182 566 | 24220 567 | 24252 568 | 24302 569 | 24337 570 | 24338 571 | 24341 572 | 24363 573 | 24372 574 | 24392 575 | 24507 576 | 24521 577 | 24573 578 | 24602 579 | 24608 580 | 24626 581 | 24640 582 | 24644 583 | 24648 584 | 24662 585 | 24709 586 | 24749 587 | 24750 588 | 24863 589 | 24898 590 | 24965 591 | -------------------------------------------------------------------------------- /mirflickr/duplicates.txt: -------------------------------------------------------------------------------- 1 | 6602 6921 -------------------------------------------------------------------------------- /mirflickr/female_query.txt: -------------------------------------------------------------------------------- 1 | 19868 2 | 3967 3 | 12352 4 | 17325 5 | 12234 6 | -------------------------------------------------------------------------------- /mirflickr/flower_query.txt: -------------------------------------------------------------------------------- 1 | 1513 2 | 6247 3 | 17140 4 | 1808 5 | 21715 6 | -------------------------------------------------------------------------------- /mirflickr/flower_r1.txt: -------------------------------------------------------------------------------- 1 | 28 2 | 43 3 | 44 4 | 45 5 | 68 6 | 73 7 | 76 8 | 80 9 | 85 10 | 87 11 | 95 12 | 112 13 | 114 14 | 119 15 | 125 16 | 147 17 | 154 18 | 158 19 | 166 20 | 170 21 | 189 22 | 198 23 | 202 24 | 204 25 | 208 26 | 247 27 | 252 28 | 261 29 | 276 30 | 278 31 | 288 32 | 308 33 | 311 34 | 326 35 | 339 36 | 360 37 | 372 38 | 378 39 | 379 40 | 385 41 | 395 42 | 403 43 | 413 44 | 416 45 | 429 46 | 493 47 | 500 48 | 502 49 | 535 50 | 575 51 | 577 52 | 583 53 | 625 54 | 646 55 | 653 56 | 656 57 | 658 58 | 671 59 | 676 60 | 691 61 | 695 62 | 732 63 | 738 64 | 751 65 | 760 66 | 821 67 | 832 68 | 834 69 | 850 70 | 865 71 | 886 72 | 888 73 | 921 74 | 925 75 | 949 76 | 950 77 | 957 78 | 962 79 | 969 80 | 983 81 | 987 82 | 992 83 | 1004 84 | 1039 85 | 1052 86 | 1075 87 | 1077 88 | 1081 89 | 1091 90 | 1128 91 | 1134 92 | 1144 93 | 1154 94 | 1168 95 | 1171 96 | 1177 97 | 1185 98 | 1208 99 | 1217 100 | 1218 101 | 1273 102 | 1286 103 | 1289 104 | 1307 105 | 1311 106 | 1324 107 | 1328 108 | 1363 109 | 1411 110 | 1419 111 | 1502 112 | 1513 113 | 1515 114 | 1552 115 | 1564 116 | 1566 117 | 1591 118 | 1600 119 | 1603 120 | 1623 121 | 1624 122 | 1632 123 | 1653 124 | 1676 125 | 1686 126 | 1697 127 | 1699 128 | 1703 129 | 1709 130 | 1765 131 | 1791 132 | 1795 133 | 1801 134 | 1803 135 | 1808 136 | 1810 137 | 1815 138 | 1832 139 | 1845 140 | 1854 141 | 1864 142 | 1885 143 | 1892 144 | 1893 145 | 1901 146 | 1905 147 | 1914 148 | 1916 149 | 1918 150 | 1950 151 | 1952 152 | 1994 153 | 1997 154 | 2001 155 | 2012 156 | 2014 157 | 2023 158 | 2027 159 | 2085 160 | 2119 161 | 2144 162 | 2152 163 | 2156 164 | 2160 165 | 2164 166 | 2216 167 | 2220 168 | 2227 169 | 2247 170 | 2297 171 | 2299 172 | 2317 173 | 2335 174 | 2353 175 | 2355 176 | 2362 177 | 2369 178 | 2378 179 | 2381 180 | 2383 181 | 2396 182 | 2431 183 | 2449 184 | 2453 185 | 2459 186 | 2462 187 | 2464 188 | 2476 189 | 2488 190 | 2516 191 | 2531 192 | 2543 193 | 2545 194 | 2585 195 | 2648 196 | 2650 197 | 2683 198 | 2715 199 | 2736 200 | 2752 201 | 2756 202 | 2776 203 | 2807 204 | 2814 205 | 2843 206 | 2844 207 | 2856 208 | 2897 209 | 2963 210 | 2986 211 | 2997 212 | 3002 213 | 3018 214 | 3026 215 | 3031 216 | 3051 217 | 3063 218 | 3111 219 | 3112 220 | 3117 221 | 3135 222 | 3153 223 | 3174 224 | 3197 225 | 3232 226 | 3237 227 | 3249 228 | 3257 229 | 3264 230 | 3297 231 | 3305 232 | 3316 233 | 3323 234 | 3324 235 | 3329 236 | 3333 237 | 3357 238 | 3391 239 | 3397 240 | 3406 241 | 3486 242 | 3506 243 | 3519 244 | 3559 245 | 3619 246 | 3626 247 | 3681 248 | 3743 249 | 3779 250 | 3795 251 | 3802 252 | 3811 253 | 3817 254 | 3829 255 | 3836 256 | 3860 257 | 3882 258 | 3890 259 | 3895 260 | 3897 261 | 3899 262 | 3927 263 | 3998 264 | 4054 265 | 4062 266 | 4070 267 | 4084 268 | 4118 269 | 4120 270 | 4146 271 | 4152 272 | 4157 273 | 4183 274 | 4186 275 | 4229 276 | 4234 277 | 4249 278 | 4359 279 | 4413 280 | 4451 281 | 4543 282 | 4547 283 | 4599 284 | 4609 285 | 4630 286 | 4663 287 | 4692 288 | 4696 289 | 4715 290 | 4747 291 | 4754 292 | 4760 293 | 4761 294 | 4783 295 | 4939 296 | 4948 297 | 5003 298 | 5021 299 | 5032 300 | 5039 301 | 5093 302 | 5110 303 | 5115 304 | 5130 305 | 5131 306 | 5153 307 | 5155 308 | 5167 309 | 5181 310 | 5197 311 | 5213 312 | 5239 313 | 5242 314 | 5306 315 | 5326 316 | 5361 317 | 5365 318 | 5384 319 | 5392 320 | 5394 321 | 5426 322 | 5428 323 | 5437 324 | 5498 325 | 5521 326 | 5535 327 | 5537 328 | 5546 329 | 5576 330 | 5598 331 | 5633 332 | 5653 333 | 5713 334 | 5747 335 | 5748 336 | 5797 337 | 5798 338 | 5829 339 | 5851 340 | 5852 341 | 5859 342 | 5895 343 | 5918 344 | 5948 345 | 6021 346 | 6049 347 | 6074 348 | 6082 349 | 6090 350 | 6127 351 | 6132 352 | 6159 353 | 6164 354 | 6176 355 | 6184 356 | 6206 357 | 6242 358 | 6247 359 | 6334 360 | 6336 361 | 6380 362 | 6385 363 | 6401 364 | 6423 365 | 6440 366 | 6484 367 | 6571 368 | 6602 369 | 6637 370 | 6726 371 | 6798 372 | 6814 373 | 6831 374 | 6874 375 | 6880 376 | 6921 377 | 6938 378 | 6950 379 | 6968 380 | 7006 381 | 7011 382 | 7124 383 | 7131 384 | 7184 385 | 7204 386 | 7213 387 | 7293 388 | 7322 389 | 7328 390 | 7505 391 | 7512 392 | 7515 393 | 7521 394 | 7540 395 | 7586 396 | 7595 397 | 7658 398 | 7667 399 | 7777 400 | 7811 401 | 7818 402 | 7829 403 | 7888 404 | 7892 405 | 7918 406 | 7946 407 | 7961 408 | 8044 409 | 8046 410 | 8047 411 | 8084 412 | 8110 413 | 8131 414 | 8150 415 | 8286 416 | 8411 417 | 8422 418 | 8441 419 | 8477 420 | 8509 421 | 8518 422 | 8532 423 | 8538 424 | 8543 425 | 8572 426 | 8640 427 | 8749 428 | 8836 429 | 8920 430 | 8923 431 | 8940 432 | 8998 433 | 9024 434 | 9034 435 | 9041 436 | 9124 437 | 9188 438 | 9202 439 | 9307 440 | 9483 441 | 9497 442 | 9577 443 | 9669 444 | 9672 445 | 9702 446 | 9896 447 | 9947 448 | 10083 449 | 10145 450 | 10191 451 | 10257 452 | 10263 453 | 10297 454 | 10369 455 | 10423 456 | 10483 457 | 10487 458 | 10554 459 | 10617 460 | 10667 461 | 10740 462 | 10752 463 | 10764 464 | 10790 465 | 10857 466 | 10906 467 | 10958 468 | 10982 469 | 10983 470 | 10998 471 | 11044 472 | 11045 473 | 11062 474 | 11064 475 | 11139 476 | 11185 477 | 11197 478 | 11220 479 | 11452 480 | 11516 481 | 11558 482 | 11612 483 | 11687 484 | 11720 485 | 11850 486 | 11871 487 | 11913 488 | 11983 489 | 12049 490 | 12077 491 | 12091 492 | 12174 493 | 12196 494 | 12247 495 | 12249 496 | 12256 497 | 12422 498 | 12424 499 | 12559 500 | 12604 501 | 12612 502 | 12627 503 | 12635 504 | 12679 505 | 12685 506 | 12689 507 | 12802 508 | 12813 509 | 12819 510 | 12827 511 | 12888 512 | 12948 513 | 12955 514 | 12961 515 | 12990 516 | 13022 517 | 13041 518 | 13044 519 | 13065 520 | 13067 521 | 13146 522 | 13225 523 | 13241 524 | 13250 525 | 13326 526 | 13339 527 | 13418 528 | 13451 529 | 13452 530 | 13522 531 | 13544 532 | 13585 533 | 13609 534 | 13635 535 | 13693 536 | 13724 537 | 13766 538 | 13776 539 | 13803 540 | 13832 541 | 13856 542 | 13861 543 | 13924 544 | 13945 545 | 13959 546 | 13978 547 | 13989 548 | 13993 549 | 13995 550 | 14022 551 | 14026 552 | 14028 553 | 14053 554 | 14072 555 | 14098 556 | 14143 557 | 14159 558 | 14217 559 | 14244 560 | 14247 561 | 14252 562 | 14396 563 | 14415 564 | 14481 565 | 14532 566 | 14577 567 | 14586 568 | 14596 569 | 14603 570 | 14628 571 | 14634 572 | 14682 573 | 14716 574 | 14724 575 | 14738 576 | 14782 577 | 14795 578 | 14903 579 | 14920 580 | 14941 581 | 14995 582 | 15006 583 | 15034 584 | 15073 585 | 15107 586 | 15108 587 | 15231 588 | 15237 589 | 15288 590 | 15298 591 | 15312 592 | 15335 593 | 15354 594 | 15362 595 | 15503 596 | 15544 597 | 15547 598 | 15613 599 | 15620 600 | 15623 601 | 15644 602 | 15715 603 | 15718 604 | 15738 605 | 15749 606 | 15751 607 | 15839 608 | 15845 609 | 15868 610 | 15907 611 | 15965 612 | 15966 613 | 15982 614 | 15990 615 | 16011 616 | 16062 617 | 16110 618 | 16118 619 | 16140 620 | 16156 621 | 16194 622 | 16222 623 | 16225 624 | 16230 625 | 16280 626 | 16282 627 | 16289 628 | 16304 629 | 16320 630 | 16326 631 | 16332 632 | 16333 633 | 16360 634 | 16418 635 | 16425 636 | 16487 637 | 16571 638 | 16578 639 | 16610 640 | 16649 641 | 16662 642 | 16733 643 | 16735 644 | 16788 645 | 16792 646 | 16806 647 | 16843 648 | 16846 649 | 16858 650 | 16860 651 | 16865 652 | 17002 653 | 17017 654 | 17048 655 | 17078 656 | 17113 657 | 17131 658 | 17138 659 | 17140 660 | 17168 661 | 17171 662 | 17175 663 | 17219 664 | 17225 665 | 17229 666 | 17335 667 | 17439 668 | 17446 669 | 17449 670 | 17478 671 | 17486 672 | 17492 673 | 17521 674 | 17560 675 | 17566 676 | 17650 677 | 17665 678 | 17699 679 | 17700 680 | 17740 681 | 17781 682 | 17797 683 | 17898 684 | 17922 685 | 17928 686 | 17941 687 | 17973 688 | 18051 689 | 18060 690 | 18079 691 | 18082 692 | 18189 693 | 18219 694 | 18220 695 | 18312 696 | 18317 697 | 18325 698 | 18327 699 | 18373 700 | 18407 701 | 18442 702 | 18497 703 | 18525 704 | 18553 705 | 18557 706 | 18618 707 | 18667 708 | 18684 709 | 18770 710 | 18799 711 | 18816 712 | 18839 713 | 18841 714 | 18847 715 | 18868 716 | 18902 717 | 18907 718 | 18927 719 | 18932 720 | 18951 721 | 18960 722 | 19025 723 | 19033 724 | 19037 725 | 19109 726 | 19113 727 | 19114 728 | 19141 729 | 19221 730 | 19227 731 | 19249 732 | 19283 733 | 19289 734 | 19296 735 | 19323 736 | 19324 737 | 19341 738 | 19346 739 | 19354 740 | 19407 741 | 19415 742 | 19445 743 | 19526 744 | 19553 745 | 19600 746 | 19623 747 | 19643 748 | 19646 749 | 19721 750 | 19734 751 | 19746 752 | 19764 753 | 19783 754 | 19797 755 | 19805 756 | 19817 757 | 19832 758 | 19843 759 | 19855 760 | 19871 761 | 19991 762 | 20041 763 | 20084 764 | 20127 765 | 20173 766 | 20190 767 | 20221 768 | 20240 769 | 20373 770 | 20400 771 | 20423 772 | 20440 773 | 20444 774 | 20477 775 | 20539 776 | 20559 777 | 20560 778 | 20600 779 | 20632 780 | 20648 781 | 20661 782 | 20667 783 | 20673 784 | 20705 785 | 20722 786 | 20729 787 | 20806 788 | 20819 789 | 20853 790 | 20865 791 | 20932 792 | 20966 793 | 20970 794 | 20977 795 | 20996 796 | 21013 797 | 21041 798 | 21050 799 | 21061 800 | 21106 801 | 21127 802 | 21174 803 | 21188 804 | 21216 805 | 21252 806 | 21263 807 | 21294 808 | 21302 809 | 21384 810 | 21400 811 | 21470 812 | 21526 813 | 21527 814 | 21602 815 | 21626 816 | 21697 817 | 21706 818 | 21715 819 | 21738 820 | 21795 821 | 21834 822 | 21840 823 | 21888 824 | 21898 825 | 21959 826 | 22025 827 | 22036 828 | 22041 829 | 22045 830 | 22047 831 | 22048 832 | 22050 833 | 22052 834 | 22094 835 | 22122 836 | 22160 837 | 22167 838 | 22274 839 | 22311 840 | 22314 841 | 22349 842 | 22362 843 | 22364 844 | 22372 845 | 22373 846 | 22382 847 | 22420 848 | 22422 849 | 22439 850 | 22440 851 | 22445 852 | 22457 853 | 22497 854 | 22506 855 | 22543 856 | 22560 857 | 22570 858 | 22576 859 | 22580 860 | 22596 861 | 22607 862 | 22608 863 | 22611 864 | 22618 865 | 22624 866 | 22639 867 | 22652 868 | 22658 869 | 22666 870 | 22680 871 | 22684 872 | 22695 873 | 22701 874 | 22707 875 | 22713 876 | 22748 877 | 22755 878 | 22768 879 | 22793 880 | 22802 881 | 22827 882 | 22829 883 | 22830 884 | 22851 885 | 22877 886 | 22908 887 | 22915 888 | 22925 889 | 22930 890 | 22932 891 | 22935 892 | 22941 893 | 22962 894 | 22967 895 | 22979 896 | 23013 897 | 23026 898 | 23029 899 | 23030 900 | 23035 901 | 23049 902 | 23060 903 | 23067 904 | 23075 905 | 23076 906 | 23107 907 | 23108 908 | 23127 909 | 23131 910 | 23140 911 | 23171 912 | 23175 913 | 23179 914 | 23190 915 | 23198 916 | 23203 917 | 23219 918 | 23246 919 | 23271 920 | 23274 921 | 23277 922 | 23283 923 | 23295 924 | 23324 925 | 23326 926 | 23353 927 | 23355 928 | 23360 929 | 23376 930 | 23383 931 | 23399 932 | 23405 933 | 23432 934 | 23443 935 | 23452 936 | 23456 937 | 23466 938 | 23468 939 | 23470 940 | 23487 941 | 23498 942 | 23502 943 | 23503 944 | 23505 945 | 23506 946 | 23542 947 | 23544 948 | 23552 949 | 23558 950 | 23590 951 | 23598 952 | 23612 953 | 23618 954 | 23654 955 | 23655 956 | 23661 957 | 23685 958 | 23689 959 | 23695 960 | 23702 961 | 23712 962 | 23744 963 | 23745 964 | 23751 965 | 23755 966 | 23769 967 | 23778 968 | 23785 969 | 23786 970 | 23814 971 | 23837 972 | 23871 973 | 23877 974 | 23888 975 | 23890 976 | 23895 977 | 23907 978 | 23917 979 | 23920 980 | 23942 981 | 23962 982 | 23970 983 | 23971 984 | 23976 985 | 24015 986 | 24020 987 | 24031 988 | 24032 989 | 24036 990 | 24046 991 | 24053 992 | 24057 993 | 24067 994 | 24076 995 | 24100 996 | 24114 997 | 24121 998 | 24133 999 | 24160 1000 | 24198 1001 | 24250 1002 | 24264 1003 | 24265 1004 | 24268 1005 | 24269 1006 | 24271 1007 | 24278 1008 | 24290 1009 | 24298 1010 | 24307 1011 | 24312 1012 | 24315 1013 | 24346 1014 | 24348 1015 | 24354 1016 | 24367 1017 | 24373 1018 | 24381 1019 | 24384 1020 | 24386 1021 | 24387 1022 | 24395 1023 | 24400 1024 | 24404 1025 | 24409 1026 | 24431 1027 | 24433 1028 | 24438 1029 | 24442 1030 | 24461 1031 | 24482 1032 | 24484 1033 | 24490 1034 | 24495 1035 | 24504 1036 | 24544 1037 | 24577 1038 | 24579 1039 | 24581 1040 | 24582 1041 | 24596 1042 | 24607 1043 | 24613 1044 | 24627 1045 | 24629 1046 | 24637 1047 | 24639 1048 | 24652 1049 | 24656 1050 | 24663 1051 | 24689 1052 | 24692 1053 | 24712 1054 | 24718 1055 | 24757 1056 | 24788 1057 | 24791 1058 | 24801 1059 | 24827 1060 | 24831 1061 | 24832 1062 | 24839 1063 | 24857 1064 | 24861 1065 | 24868 1066 | 24871 1067 | 24876 1068 | 24890 1069 | 24892 1070 | 24918 1071 | 24947 1072 | 24957 1073 | 24961 1074 | 24964 1075 | 24977 1076 | 24978 1077 | 24999 1078 | -------------------------------------------------------------------------------- /mirflickr/male_query.txt: -------------------------------------------------------------------------------- 1 | 24350 2 | 14346 3 | 6761 4 | 12736 5 | 3502 6 | -------------------------------------------------------------------------------- /mirflickr/male_r1.txt: -------------------------------------------------------------------------------- 1 | 4 2 | 5 3 | 11 4 | 13 5 | 24 6 | 39 7 | 49 8 | 50 9 | 75 10 | 83 11 | 86 12 | 91 13 | 96 14 | 98 15 | 99 16 | 124 17 | 126 18 | 130 19 | 134 20 | 136 21 | 141 22 | 142 23 | 144 24 | 165 25 | 172 26 | 176 27 | 183 28 | 197 29 | 200 30 | 213 31 | 215 32 | 216 33 | 219 34 | 223 35 | 229 36 | 237 37 | 257 38 | 258 39 | 260 40 | 264 41 | 270 42 | 273 43 | 284 44 | 294 45 | 298 46 | 301 47 | 316 48 | 321 49 | 322 50 | 330 51 | 338 52 | 342 53 | 353 54 | 359 55 | 361 56 | 365 57 | 374 58 | 387 59 | 396 60 | 399 61 | 422 62 | 423 63 | 432 64 | 436 65 | 450 66 | 451 67 | 457 68 | 461 69 | 465 70 | 466 71 | 468 72 | 480 73 | 483 74 | 489 75 | 504 76 | 508 77 | 513 78 | 522 79 | 523 80 | 529 81 | 530 82 | 542 83 | 543 84 | 548 85 | 551 86 | 560 87 | 571 88 | 580 89 | 585 90 | 588 91 | 591 92 | 602 93 | 603 94 | 604 95 | 637 96 | 642 97 | 672 98 | 689 99 | 693 100 | 698 101 | 719 102 | 735 103 | 744 104 | 747 105 | 753 106 | 793 107 | 810 108 | 816 109 | 823 110 | 827 111 | 829 112 | 841 113 | 843 114 | 844 115 | 862 116 | 864 117 | 875 118 | 884 119 | 888 120 | 895 121 | 898 122 | 901 123 | 908 124 | 914 125 | 929 126 | 935 127 | 948 128 | 953 129 | 956 130 | 958 131 | 964 132 | 975 133 | 979 134 | 988 135 | 1006 136 | 1017 137 | 1025 138 | 1028 139 | 1040 140 | 1041 141 | 1043 142 | 1045 143 | 1046 144 | 1049 145 | 1055 146 | 1067 147 | 1076 148 | 1079 149 | 1083 150 | 1087 151 | 1094 152 | 1097 153 | 1105 154 | 1117 155 | 1119 156 | 1120 157 | 1121 158 | 1125 159 | 1132 160 | 1135 161 | 1141 162 | 1142 163 | 1148 164 | 1152 165 | 1153 166 | 1173 167 | 1181 168 | 1189 169 | 1190 170 | 1197 171 | 1201 172 | 1205 173 | 1215 174 | 1222 175 | 1224 176 | 1226 177 | 1233 178 | 1234 179 | 1235 180 | 1239 181 | 1245 182 | 1248 183 | 1258 184 | 1262 185 | 1267 186 | 1274 187 | 1278 188 | 1280 189 | 1281 190 | 1282 191 | 1297 192 | 1310 193 | 1312 194 | 1313 195 | 1318 196 | 1320 197 | 1321 198 | 1322 199 | 1330 200 | 1362 201 | 1367 202 | 1371 203 | 1378 204 | 1383 205 | 1390 206 | 1393 207 | 1396 208 | 1397 209 | 1402 210 | 1403 211 | 1407 212 | 1415 213 | 1421 214 | 1427 215 | 1430 216 | 1433 217 | 1449 218 | 1463 219 | 1466 220 | 1474 221 | 1477 222 | 1482 223 | 1489 224 | 1497 225 | 1500 226 | 1501 227 | 1503 228 | 1508 229 | 1511 230 | 1532 231 | 1536 232 | 1540 233 | 1545 234 | 1550 235 | 1551 236 | 1568 237 | 1569 238 | 1571 239 | 1579 240 | 1582 241 | 1583 242 | 1589 243 | 1595 244 | 1602 245 | 1609 246 | 1610 247 | 1613 248 | 1614 249 | 1615 250 | 1622 251 | 1625 252 | 1641 253 | 1642 254 | 1648 255 | 1650 256 | 1663 257 | 1664 258 | 1665 259 | 1683 260 | 1687 261 | 1688 262 | 1695 263 | 1700 264 | 1701 265 | 1706 266 | 1711 267 | 1717 268 | 1718 269 | 1722 270 | 1723 271 | 1732 272 | 1736 273 | 1741 274 | 1747 275 | 1750 276 | 1753 277 | 1754 278 | 1755 279 | 1756 280 | 1759 281 | 1786 282 | 1788 283 | 1797 284 | 1819 285 | 1821 286 | 1822 287 | 1833 288 | 1838 289 | 1840 290 | 1850 291 | 1858 292 | 1862 293 | 1866 294 | 1874 295 | 1888 296 | 1889 297 | 1891 298 | 1902 299 | 1905 300 | 1911 301 | 1913 302 | 1917 303 | 1925 304 | 1929 305 | 1931 306 | 1932 307 | 1933 308 | 1934 309 | 1936 310 | 1945 311 | 1949 312 | 1954 313 | 1955 314 | 1959 315 | 1973 316 | 1984 317 | 1996 318 | 2001 319 | 2002 320 | 2004 321 | 2020 322 | 2032 323 | 2034 324 | 2057 325 | 2067 326 | 2073 327 | 2079 328 | 2083 329 | 2084 330 | 2086 331 | 2088 332 | 2095 333 | 2097 334 | 2098 335 | 2102 336 | 2116 337 | 2126 338 | 2128 339 | 2137 340 | 2143 341 | 2157 342 | 2169 343 | 2175 344 | 2179 345 | 2180 346 | 2181 347 | 2187 348 | 2193 349 | 2196 350 | 2197 351 | 2199 352 | 2209 353 | 2211 354 | 2215 355 | 2221 356 | 2224 357 | 2241 358 | 2242 359 | 2246 360 | 2258 361 | 2266 362 | 2269 363 | 2276 364 | 2285 365 | 2292 366 | 2298 367 | 2308 368 | 2309 369 | 2313 370 | 2320 371 | 2324 372 | 2328 373 | 2330 374 | 2332 375 | 2339 376 | 2340 377 | 2341 378 | 2346 379 | 2354 380 | 2359 381 | 2360 382 | 2367 383 | 2376 384 | 2380 385 | 2387 386 | 2397 387 | 2399 388 | 2410 389 | 2412 390 | 2421 391 | 2422 392 | 2424 393 | 2427 394 | 2440 395 | 2445 396 | 2446 397 | 2450 398 | 2451 399 | 2455 400 | 2461 401 | 2467 402 | 2477 403 | 2478 404 | 2481 405 | 2484 406 | 2489 407 | 2495 408 | 2502 409 | 2506 410 | 2512 411 | 2519 412 | 2558 413 | 2559 414 | 2560 415 | 2575 416 | 2577 417 | 2578 418 | 2586 419 | 2588 420 | 2593 421 | 2594 422 | 2595 423 | 2598 424 | 2619 425 | 2620 426 | 2622 427 | 2624 428 | 2625 429 | 2646 430 | 2659 431 | 2663 432 | 2666 433 | 2691 434 | 2694 435 | 2699 436 | 2701 437 | 2704 438 | 2706 439 | 2714 440 | 2719 441 | 2734 442 | 2737 443 | 2743 444 | 2754 445 | 2774 446 | 2777 447 | 2780 448 | 2784 449 | 2785 450 | 2793 451 | 2801 452 | 2813 453 | 2841 454 | 2853 455 | 2854 456 | 2860 457 | 2862 458 | 2879 459 | 2884 460 | 2887 461 | 2888 462 | 2892 463 | 2911 464 | 2912 465 | 2922 466 | 2927 467 | 2929 468 | 2935 469 | 2936 470 | 2941 471 | 2944 472 | 2957 473 | 2972 474 | 2974 475 | 2976 476 | 2984 477 | 2994 478 | 2996 479 | 2998 480 | 3022 481 | 3029 482 | 3032 483 | 3039 484 | 3042 485 | 3045 486 | 3061 487 | 3070 488 | 3072 489 | 3075 490 | 3077 491 | 3081 492 | 3085 493 | 3086 494 | 3105 495 | 3110 496 | 3118 497 | 3121 498 | 3126 499 | 3141 500 | 3159 501 | 3160 502 | 3161 503 | 3165 504 | 3168 505 | 3169 506 | 3170 507 | 3171 508 | 3175 509 | 3186 510 | 3188 511 | 3193 512 | 3195 513 | 3209 514 | 3210 515 | 3224 516 | 3225 517 | 3244 518 | 3248 519 | 3266 520 | 3272 521 | 3273 522 | 3289 523 | 3291 524 | 3307 525 | 3308 526 | 3309 527 | 3322 528 | 3331 529 | 3338 530 | 3339 531 | 3345 532 | 3354 533 | 3358 534 | 3359 535 | 3362 536 | 3376 537 | 3380 538 | 3384 539 | 3388 540 | 3392 541 | 3399 542 | 3402 543 | 3404 544 | 3406 545 | 3410 546 | 3413 547 | 3428 548 | 3429 549 | 3430 550 | 3432 551 | 3440 552 | 3442 553 | 3445 554 | 3447 555 | 3450 556 | 3451 557 | 3455 558 | 3463 559 | 3465 560 | 3466 561 | 3472 562 | 3474 563 | 3477 564 | 3492 565 | 3501 566 | 3502 567 | 3503 568 | 3508 569 | 3514 570 | 3520 571 | 3523 572 | 3529 573 | 3530 574 | 3533 575 | 3545 576 | 3549 577 | 3550 578 | 3551 579 | 3553 580 | 3558 581 | 3566 582 | 3568 583 | 3569 584 | 3574 585 | 3576 586 | 3583 587 | 3584 588 | 3591 589 | 3599 590 | 3600 591 | 3602 592 | 3608 593 | 3614 594 | 3615 595 | 3618 596 | 3636 597 | 3641 598 | 3650 599 | 3653 600 | 3659 601 | 3670 602 | 3672 603 | 3680 604 | 3686 605 | 3712 606 | 3713 607 | 3721 608 | 3723 609 | 3747 610 | 3750 611 | 3753 612 | 3758 613 | 3761 614 | 3770 615 | 3774 616 | 3785 617 | 3787 618 | 3788 619 | 3791 620 | 3799 621 | 3801 622 | 3809 623 | 3814 624 | 3819 625 | 3832 626 | 3833 627 | 3843 628 | 3844 629 | 3849 630 | 3875 631 | 3879 632 | 3880 633 | 3909 634 | 3916 635 | 3924 636 | 3926 637 | 3935 638 | 3938 639 | 3952 640 | 3966 641 | 3968 642 | 3969 643 | 3970 644 | 3978 645 | 3980 646 | 3990 647 | 3992 648 | 4007 649 | 4018 650 | 4024 651 | 4030 652 | 4035 653 | 4038 654 | 4041 655 | 4048 656 | 4053 657 | 4060 658 | 4063 659 | 4065 660 | 4069 661 | 4076 662 | 4077 663 | 4090 664 | 4093 665 | 4096 666 | 4102 667 | 4104 668 | 4108 669 | 4111 670 | 4124 671 | 4125 672 | 4127 673 | 4136 674 | 4141 675 | 4142 676 | 4150 677 | 4163 678 | 4168 679 | 4178 680 | 4184 681 | 4192 682 | 4202 683 | 4204 684 | 4215 685 | 4224 686 | 4226 687 | 4227 688 | 4228 689 | 4236 690 | 4245 691 | 4262 692 | 4264 693 | 4268 694 | 4271 695 | 4292 696 | 4297 697 | 4300 698 | 4311 699 | 4322 700 | 4327 701 | 4338 702 | 4339 703 | 4346 704 | 4351 705 | 4352 706 | 4362 707 | 4373 708 | 4377 709 | 4379 710 | 4397 711 | 4402 712 | 4405 713 | 4408 714 | 4409 715 | 4411 716 | 4414 717 | 4417 718 | 4418 719 | 4419 720 | 4420 721 | 4424 722 | 4425 723 | 4437 724 | 4438 725 | 4446 726 | 4448 727 | 4459 728 | 4465 729 | 4467 730 | 4472 731 | 4473 732 | 4478 733 | 4479 734 | 4486 735 | 4488 736 | 4493 737 | 4508 738 | 4509 739 | 4510 740 | 4511 741 | 4531 742 | 4539 743 | 4550 744 | 4562 745 | 4563 746 | 4578 747 | 4584 748 | 4611 749 | 4613 750 | 4614 751 | 4625 752 | 4634 753 | 4635 754 | 4652 755 | 4656 756 | 4657 757 | 4659 758 | 4670 759 | 4671 760 | 4678 761 | 4681 762 | 4683 763 | 4691 764 | 4702 765 | 4703 766 | 4739 767 | 4743 768 | 4744 769 | 4752 770 | 4757 771 | 4763 772 | 4765 773 | 4768 774 | 4775 775 | 4782 776 | 4792 777 | 4795 778 | 4796 779 | 4798 780 | 4802 781 | 4820 782 | 4821 783 | 4824 784 | 4827 785 | 4831 786 | 4834 787 | 4836 788 | 4851 789 | 4861 790 | 4862 791 | 4866 792 | 4867 793 | 4871 794 | 4887 795 | 4888 796 | 4895 797 | 4896 798 | 4897 799 | 4900 800 | 4902 801 | 4918 802 | 4935 803 | 4946 804 | 4950 805 | 4951 806 | 4954 807 | 4965 808 | 4973 809 | 4983 810 | 4986 811 | 4988 812 | 4991 813 | 4996 814 | 5017 815 | 5023 816 | 5024 817 | 5042 818 | 5049 819 | 5051 820 | 5058 821 | 5073 822 | 5077 823 | 5085 824 | 5090 825 | 5096 826 | 5098 827 | 5102 828 | 5108 829 | 5109 830 | 5122 831 | 5126 832 | 5136 833 | 5138 834 | 5166 835 | 5174 836 | 5187 837 | 5189 838 | 5192 839 | 5195 840 | 5202 841 | 5206 842 | 5246 843 | 5247 844 | 5255 845 | 5256 846 | 5257 847 | 5266 848 | 5282 849 | 5287 850 | 5293 851 | 5294 852 | 5317 853 | 5319 854 | 5329 855 | 5332 856 | 5339 857 | 5340 858 | 5346 859 | 5349 860 | 5351 861 | 5356 862 | 5360 863 | 5373 864 | 5379 865 | 5387 866 | 5389 867 | 5390 868 | 5391 869 | 5397 870 | 5410 871 | 5412 872 | 5414 873 | 5415 874 | 5419 875 | 5432 876 | 5436 877 | 5439 878 | 5441 879 | 5442 880 | 5443 881 | 5445 882 | 5446 883 | 5455 884 | 5461 885 | 5486 886 | 5493 887 | 5494 888 | 5501 889 | 5514 890 | 5516 891 | 5523 892 | 5525 893 | 5543 894 | 5544 895 | 5548 896 | 5551 897 | 5559 898 | 5565 899 | 5569 900 | 5600 901 | 5601 902 | 5614 903 | 5626 904 | 5628 905 | 5638 906 | 5640 907 | 5645 908 | 5654 909 | 5655 910 | 5661 911 | 5665 912 | 5666 913 | 5667 914 | 5683 915 | 5686 916 | 5688 917 | 5691 918 | 5692 919 | 5693 920 | 5706 921 | 5715 922 | 5721 923 | 5725 924 | 5739 925 | 5741 926 | 5745 927 | 5750 928 | 5759 929 | 5765 930 | 5789 931 | 5805 932 | 5817 933 | 5822 934 | 5823 935 | 5825 936 | 5826 937 | 5831 938 | 5846 939 | 5847 940 | 5849 941 | 5856 942 | 5860 943 | 5864 944 | 5870 945 | 5871 946 | 5872 947 | 5873 948 | 5876 949 | 5885 950 | 5913 951 | 5932 952 | 5938 953 | 5947 954 | 5959 955 | 5960 956 | 5970 957 | 5971 958 | 5974 959 | 5998 960 | 6007 961 | 6009 962 | 6011 963 | 6014 964 | 6016 965 | 6028 966 | 6030 967 | 6043 968 | 6044 969 | 6071 970 | 6077 971 | 6096 972 | 6098 973 | 6102 974 | 6107 975 | 6110 976 | 6139 977 | 6140 978 | 6142 979 | 6156 980 | 6171 981 | 6179 982 | 6180 983 | 6192 984 | 6214 985 | 6220 986 | 6221 987 | 6226 988 | 6229 989 | 6231 990 | 6236 991 | 6248 992 | 6249 993 | 6255 994 | 6261 995 | 6262 996 | 6269 997 | 6274 998 | 6275 999 | 6292 1000 | 6293 1001 | 6294 1002 | 6295 1003 | 6315 1004 | 6327 1005 | 6329 1006 | 6330 1007 | 6332 1008 | 6342 1009 | 6345 1010 | 6348 1011 | 6350 1012 | 6354 1013 | 6363 1014 | 6371 1015 | 6374 1016 | 6375 1017 | 6391 1018 | 6400 1019 | 6402 1020 | 6407 1021 | 6416 1022 | 6427 1023 | 6431 1024 | 6441 1025 | 6443 1026 | 6451 1027 | 6452 1028 | 6457 1029 | 6471 1030 | 6479 1031 | 6485 1032 | 6508 1033 | 6513 1034 | 6515 1035 | 6520 1036 | 6528 1037 | 6540 1038 | 6555 1039 | 6560 1040 | 6568 1041 | 6569 1042 | 6574 1043 | 6578 1044 | 6582 1045 | 6592 1046 | 6593 1047 | 6595 1048 | 6597 1049 | 6614 1050 | 6625 1051 | 6626 1052 | 6641 1053 | 6655 1054 | 6666 1055 | 6677 1056 | 6686 1057 | 6700 1058 | 6713 1059 | 6731 1060 | 6734 1061 | 6737 1062 | 6739 1063 | 6744 1064 | 6750 1065 | 6755 1066 | 6758 1067 | 6761 1068 | 6772 1069 | 6778 1070 | 6791 1071 | 6800 1072 | 6801 1073 | 6803 1074 | 6806 1075 | 6813 1076 | 6840 1077 | 6844 1078 | 6846 1079 | 6867 1080 | 6870 1081 | 6872 1082 | 6876 1083 | 6878 1084 | 6885 1085 | 6886 1086 | 6891 1087 | 6897 1088 | 6902 1089 | 6907 1090 | 6924 1091 | 6925 1092 | 6932 1093 | 6934 1094 | 6937 1095 | 6943 1096 | 6947 1097 | 6948 1098 | 6966 1099 | 6967 1100 | 6997 1101 | 7002 1102 | 7005 1103 | 7012 1104 | 7020 1105 | 7029 1106 | 7034 1107 | 7041 1108 | 7050 1109 | 7067 1110 | 7080 1111 | 7082 1112 | 7089 1113 | 7091 1114 | 7097 1115 | 7104 1116 | 7110 1117 | 7111 1118 | 7112 1119 | 7123 1120 | 7133 1121 | 7141 1122 | 7153 1123 | 7154 1124 | 7166 1125 | 7177 1126 | 7187 1127 | 7192 1128 | 7197 1129 | 7202 1130 | 7203 1131 | 7207 1132 | 7216 1133 | 7217 1134 | 7231 1135 | 7233 1136 | 7234 1137 | 7242 1138 | 7245 1139 | 7246 1140 | 7257 1141 | 7270 1142 | 7288 1143 | 7289 1144 | 7295 1145 | 7298 1146 | 7307 1147 | 7311 1148 | 7329 1149 | 7332 1150 | 7334 1151 | 7338 1152 | 7341 1153 | 7342 1154 | 7357 1155 | 7358 1156 | 7365 1157 | 7366 1158 | 7368 1159 | 7370 1160 | 7377 1161 | 7383 1162 | 7384 1163 | 7385 1164 | 7388 1165 | 7392 1166 | 7393 1167 | 7403 1168 | 7427 1169 | 7439 1170 | 7444 1171 | 7463 1172 | 7464 1173 | 7498 1174 | 7500 1175 | 7513 1176 | 7524 1177 | 7533 1178 | 7537 1179 | 7549 1180 | 7554 1181 | 7583 1182 | 7587 1183 | 7589 1184 | 7592 1185 | 7618 1186 | 7628 1187 | 7630 1188 | 7634 1189 | 7635 1190 | 7648 1191 | 7651 1192 | 7655 1193 | 7667 1194 | 7677 1195 | 7679 1196 | 7681 1197 | 7684 1198 | 7699 1199 | 7716 1200 | 7748 1201 | 7751 1202 | 7756 1203 | 7758 1204 | 7764 1205 | 7776 1206 | 7790 1207 | 7791 1208 | 7801 1209 | 7815 1210 | 7817 1211 | 7834 1212 | 7848 1213 | 7853 1214 | 7856 1215 | 7857 1216 | 7861 1217 | 7862 1218 | 7875 1219 | 7878 1220 | 7900 1221 | 7917 1222 | 7931 1223 | 7941 1224 | 7944 1225 | 7948 1226 | 7954 1227 | 7959 1228 | 7969 1229 | 7972 1230 | 7974 1231 | 7979 1232 | 8000 1233 | 8003 1234 | 8015 1235 | 8018 1236 | 8019 1237 | 8066 1238 | 8072 1239 | 8074 1240 | 8099 1241 | 8101 1242 | 8104 1243 | 8121 1244 | 8132 1245 | 8137 1246 | 8143 1247 | 8145 1248 | 8157 1249 | 8158 1250 | 8160 1251 | 8164 1252 | 8165 1253 | 8180 1254 | 8196 1255 | 8197 1256 | 8198 1257 | 8202 1258 | 8203 1259 | 8206 1260 | 8209 1261 | 8221 1262 | 8224 1263 | 8225 1264 | 8226 1265 | 8230 1266 | 8245 1267 | 8253 1268 | 8279 1269 | 8281 1270 | 8290 1271 | 8291 1272 | 8300 1273 | 8307 1274 | 8324 1275 | 8337 1276 | 8357 1277 | 8371 1278 | 8384 1279 | 8387 1280 | 8408 1281 | 8414 1282 | 8432 1283 | 8444 1284 | 8452 1285 | 8453 1286 | 8470 1287 | 8487 1288 | 8489 1289 | 8508 1290 | 8511 1291 | 8517 1292 | 8522 1293 | 8527 1294 | 8531 1295 | 8549 1296 | 8553 1297 | 8558 1298 | 8565 1299 | 8587 1300 | 8592 1301 | 8593 1302 | 8598 1303 | 8613 1304 | 8618 1305 | 8624 1306 | 8627 1307 | 8629 1308 | 8633 1309 | 8636 1310 | 8647 1311 | 8657 1312 | 8663 1313 | 8677 1314 | 8685 1315 | 8690 1316 | 8710 1317 | 8712 1318 | 8713 1319 | 8726 1320 | 8734 1321 | 8745 1322 | 8767 1323 | 8779 1324 | 8784 1325 | 8789 1326 | 8797 1327 | 8802 1328 | 8805 1329 | 8812 1330 | 8813 1331 | 8815 1332 | 8823 1333 | 8827 1334 | 8837 1335 | 8842 1336 | 8851 1337 | 8865 1338 | 8868 1339 | 8874 1340 | 8876 1341 | 8896 1342 | 8899 1343 | 8900 1344 | 8903 1345 | 8905 1346 | 8906 1347 | 8907 1348 | 8914 1349 | 8922 1350 | 8928 1351 | 8930 1352 | 8934 1353 | 8939 1354 | 8954 1355 | 8967 1356 | 8969 1357 | 8971 1358 | 8974 1359 | 8978 1360 | 8996 1361 | 9000 1362 | 9008 1363 | 9013 1364 | 9014 1365 | 9018 1366 | 9019 1367 | 9031 1368 | 9036 1369 | 9042 1370 | 9049 1371 | 9051 1372 | 9058 1373 | 9069 1374 | 9070 1375 | 9091 1376 | 9137 1377 | 9143 1378 | 9148 1379 | 9156 1380 | 9158 1381 | 9171 1382 | 9187 1383 | 9192 1384 | 9199 1385 | 9218 1386 | 9226 1387 | 9234 1388 | 9240 1389 | 9241 1390 | 9249 1391 | 9250 1392 | 9251 1393 | 9256 1394 | 9261 1395 | 9264 1396 | 9265 1397 | 9289 1398 | 9291 1399 | 9304 1400 | 9330 1401 | 9357 1402 | 9363 1403 | 9379 1404 | 9384 1405 | 9394 1406 | 9401 1407 | 9406 1408 | 9412 1409 | 9426 1410 | 9429 1411 | 9433 1412 | 9434 1413 | 9441 1414 | 9442 1415 | 9449 1416 | 9451 1417 | 9452 1418 | 9460 1419 | 9464 1420 | 9484 1421 | 9492 1422 | 9494 1423 | 9499 1424 | 9501 1425 | 9506 1426 | 9509 1427 | 9512 1428 | 9522 1429 | 9523 1430 | 9526 1431 | 9527 1432 | 9530 1433 | 9534 1434 | 9535 1435 | 9540 1436 | 9543 1437 | 9550 1438 | 9559 1439 | 9561 1440 | 9564 1441 | 9573 1442 | 9582 1443 | 9583 1444 | 9584 1445 | 9589 1446 | 9611 1447 | 9612 1448 | 9614 1449 | 9629 1450 | 9643 1451 | 9656 1452 | 9658 1453 | 9659 1454 | 9663 1455 | 9681 1456 | 9687 1457 | 9690 1458 | 9700 1459 | 9704 1460 | 9715 1461 | 9717 1462 | 9718 1463 | 9719 1464 | 9724 1465 | 9729 1466 | 9731 1467 | 9737 1468 | 9741 1469 | 9752 1470 | 9765 1471 | 9766 1472 | 9770 1473 | 9775 1474 | 9777 1475 | 9791 1476 | 9792 1477 | 9804 1478 | 9815 1479 | 9816 1480 | 9825 1481 | 9826 1482 | 9832 1483 | 9833 1484 | 9836 1485 | 9837 1486 | 9839 1487 | 9840 1488 | 9844 1489 | 9854 1490 | 9857 1491 | 9881 1492 | 9885 1493 | 9887 1494 | 9892 1495 | 9893 1496 | 9897 1497 | 9915 1498 | 9925 1499 | 9927 1500 | 9930 1501 | 9935 1502 | 9936 1503 | 9938 1504 | 9954 1505 | 9966 1506 | 9969 1507 | 9994 1508 | 9995 1509 | 9996 1510 | 10002 1511 | 10012 1512 | 10017 1513 | 10026 1514 | 10038 1515 | 10042 1516 | 10046 1517 | 10048 1518 | 10055 1519 | 10056 1520 | 10061 1521 | 10063 1522 | 10066 1523 | 10085 1524 | 10092 1525 | 10093 1526 | 10100 1527 | 10106 1528 | 10107 1529 | 10109 1530 | 10116 1531 | 10125 1532 | 10131 1533 | 10137 1534 | 10144 1535 | 10148 1536 | 10149 1537 | 10155 1538 | 10157 1539 | 10158 1540 | 10167 1541 | 10178 1542 | 10184 1543 | 10190 1544 | 10201 1545 | 10206 1546 | 10207 1547 | 10211 1548 | 10213 1549 | 10217 1550 | 10218 1551 | 10220 1552 | 10221 1553 | 10227 1554 | 10232 1555 | 10238 1556 | 10239 1557 | 10244 1558 | 10247 1559 | 10258 1560 | 10259 1561 | 10267 1562 | 10273 1563 | 10276 1564 | 10279 1565 | 10281 1566 | 10285 1567 | 10289 1568 | 10290 1569 | 10300 1570 | 10301 1571 | 10325 1572 | 10337 1573 | 10343 1574 | 10346 1575 | 10347 1576 | 10348 1577 | 10361 1578 | 10371 1579 | 10398 1580 | 10403 1581 | 10406 1582 | 10409 1583 | 10414 1584 | 10416 1585 | 10421 1586 | 10434 1587 | 10435 1588 | 10436 1589 | 10438 1590 | 10439 1591 | 10443 1592 | 10452 1593 | 10463 1594 | 10472 1595 | 10473 1596 | 10475 1597 | 10476 1598 | 10481 1599 | 10486 1600 | 10494 1601 | 10500 1602 | 10504 1603 | 10510 1604 | 10514 1605 | 10519 1606 | 10522 1607 | 10530 1608 | 10533 1609 | 10534 1610 | 10536 1611 | 10541 1612 | 10568 1613 | 10582 1614 | 10584 1615 | 10585 1616 | 10586 1617 | 10587 1618 | 10591 1619 | 10598 1620 | 10605 1621 | 10612 1622 | 10614 1623 | 10615 1624 | 10618 1625 | 10621 1626 | 10625 1627 | 10632 1628 | 10638 1629 | 10643 1630 | 10650 1631 | 10661 1632 | 10667 1633 | 10685 1634 | 10689 1635 | 10692 1636 | 10713 1637 | 10724 1638 | 10743 1639 | 10752 1640 | 10768 1641 | 10776 1642 | 10781 1643 | 10785 1644 | 10787 1645 | 10795 1646 | 10796 1647 | 10798 1648 | 10807 1649 | 10814 1650 | 10818 1651 | 10822 1652 | 10825 1653 | 10835 1654 | 10847 1655 | 10871 1656 | 10882 1657 | 10886 1658 | 10894 1659 | 10896 1660 | 10897 1661 | 10909 1662 | 10911 1663 | 10929 1664 | 10931 1665 | 10940 1666 | 10961 1667 | 10977 1668 | 10979 1669 | 10984 1670 | 10992 1671 | 11008 1672 | 11018 1673 | 11027 1674 | 11028 1675 | 11034 1676 | 11040 1677 | 11043 1678 | 11048 1679 | 11055 1680 | 11068 1681 | 11072 1682 | 11094 1683 | 11096 1684 | 11101 1685 | 11102 1686 | 11103 1687 | 11109 1688 | 11114 1689 | 11128 1690 | 11140 1691 | 11142 1692 | 11144 1693 | 11150 1694 | 11163 1695 | 11165 1696 | 11173 1697 | 11176 1698 | 11178 1699 | 11182 1700 | 11193 1701 | 11204 1702 | 11209 1703 | 11215 1704 | 11224 1705 | 11226 1706 | 11231 1707 | 11249 1708 | 11250 1709 | 11254 1710 | 11271 1711 | 11276 1712 | 11279 1713 | 11282 1714 | 11291 1715 | 11294 1716 | 11296 1717 | 11303 1718 | 11317 1719 | 11324 1720 | 11338 1721 | 11340 1722 | 11347 1723 | 11349 1724 | 11353 1725 | 11369 1726 | 11375 1727 | 11376 1728 | 11383 1729 | 11385 1730 | 11401 1731 | 11403 1732 | 11405 1733 | 11406 1734 | 11418 1735 | 11424 1736 | 11427 1737 | 11432 1738 | 11440 1739 | 11446 1740 | 11455 1741 | 11460 1742 | 11482 1743 | 11488 1744 | 11497 1745 | 11509 1746 | 11533 1747 | 11534 1748 | 11542 1749 | 11545 1750 | 11563 1751 | 11572 1752 | 11580 1753 | 11581 1754 | 11583 1755 | 11584 1756 | 11587 1757 | 11588 1758 | 11596 1759 | 11605 1760 | 11628 1761 | 11629 1762 | 11631 1763 | 11634 1764 | 11635 1765 | 11648 1766 | 11662 1767 | 11691 1768 | 11703 1769 | 11723 1770 | 11729 1771 | 11733 1772 | 11736 1773 | 11737 1774 | 11746 1775 | 11754 1776 | 11768 1777 | 11771 1778 | 11773 1779 | 11774 1780 | 11775 1781 | 11790 1782 | 11810 1783 | 11812 1784 | 11830 1785 | 11845 1786 | 11852 1787 | 11880 1788 | 11888 1789 | 11891 1790 | 11892 1791 | 11897 1792 | 11906 1793 | 11925 1794 | 11928 1795 | 11939 1796 | 11940 1797 | 11942 1798 | 11943 1799 | 11946 1800 | 11947 1801 | 11951 1802 | 11959 1803 | 11963 1804 | 11986 1805 | 11989 1806 | 11998 1807 | 12024 1808 | 12031 1809 | 12078 1810 | 12097 1811 | 12102 1812 | 12124 1813 | 12127 1814 | 12130 1815 | 12134 1816 | 12138 1817 | 12144 1818 | 12151 1819 | 12152 1820 | 12154 1821 | 12155 1822 | 12176 1823 | 12183 1824 | 12188 1825 | 12191 1826 | 12202 1827 | 12209 1828 | 12211 1829 | 12213 1830 | 12216 1831 | 12222 1832 | 12226 1833 | 12229 1834 | 12232 1835 | 12238 1836 | 12263 1837 | 12268 1838 | 12271 1839 | 12292 1840 | 12293 1841 | 12309 1842 | 12311 1843 | 12314 1844 | 12319 1845 | 12324 1846 | 12326 1847 | 12327 1848 | 12333 1849 | 12345 1850 | 12346 1851 | 12347 1852 | 12348 1853 | 12358 1854 | 12377 1855 | 12382 1856 | 12393 1857 | 12408 1858 | 12419 1859 | 12420 1860 | 12425 1861 | 12433 1862 | 12436 1863 | 12445 1864 | 12449 1865 | 12451 1866 | 12453 1867 | 12461 1868 | 12477 1869 | 12480 1870 | 12487 1871 | 12491 1872 | 12494 1873 | 12500 1874 | 12501 1875 | 12517 1876 | 12521 1877 | 12536 1878 | 12540 1879 | 12543 1880 | 12550 1881 | 12555 1882 | 12558 1883 | 12565 1884 | 12576 1885 | 12589 1886 | 12595 1887 | 12598 1888 | 12608 1889 | 12613 1890 | 12617 1891 | 12618 1892 | 12625 1893 | 12634 1894 | 12640 1895 | 12645 1896 | 12656 1897 | 12662 1898 | 12681 1899 | 12693 1900 | 12695 1901 | 12701 1902 | 12702 1903 | 12714 1904 | 12717 1905 | 12723 1906 | 12725 1907 | 12728 1908 | 12730 1909 | 12732 1910 | 12736 1911 | 12740 1912 | 12758 1913 | 12764 1914 | 12775 1915 | 12777 1916 | 12782 1917 | 12798 1918 | 12799 1919 | 12816 1920 | 12829 1921 | 12832 1922 | 12834 1923 | 12840 1924 | 12861 1925 | 12864 1926 | 12867 1927 | 12870 1928 | 12876 1929 | 12881 1930 | 12900 1931 | 12922 1932 | 12926 1933 | 12933 1934 | 12936 1935 | 12946 1936 | 12956 1937 | 12960 1938 | 12962 1939 | 12973 1940 | 12979 1941 | 12989 1942 | 12994 1943 | 13004 1944 | 13008 1945 | 13014 1946 | 13017 1947 | 13023 1948 | 13038 1949 | 13040 1950 | 13043 1951 | 13047 1952 | 13063 1953 | 13069 1954 | 13079 1955 | 13082 1956 | 13083 1957 | 13086 1958 | 13103 1959 | 13113 1960 | 13114 1961 | 13116 1962 | 13131 1963 | 13145 1964 | 13161 1965 | 13166 1966 | 13168 1967 | 13170 1968 | 13185 1969 | 13192 1970 | 13195 1971 | 13200 1972 | 13212 1973 | 13216 1974 | 13217 1975 | 13222 1976 | 13224 1977 | 13230 1978 | 13234 1979 | 13243 1980 | 13244 1981 | 13248 1982 | 13249 1983 | 13257 1984 | 13259 1985 | 13263 1986 | 13271 1987 | 13272 1988 | 13277 1989 | 13286 1990 | 13288 1991 | 13290 1992 | 13291 1993 | 13304 1994 | 13305 1995 | 13317 1996 | 13327 1997 | 13334 1998 | 13347 1999 | 13350 2000 | 13352 2001 | 13366 2002 | 13374 2003 | 13377 2004 | 13386 2005 | 13389 2006 | 13396 2007 | 13406 2008 | 13410 2009 | 13411 2010 | 13425 2011 | 13441 2012 | 13448 2013 | 13454 2014 | 13461 2015 | 13468 2016 | 13498 2017 | 13502 2018 | 13506 2019 | 13519 2020 | 13535 2021 | 13537 2022 | 13539 2023 | 13542 2024 | 13547 2025 | 13557 2026 | 13558 2027 | 13559 2028 | 13560 2029 | 13566 2030 | 13576 2031 | 13583 2032 | 13605 2033 | 13633 2034 | 13636 2035 | 13650 2036 | 13655 2037 | 13674 2038 | 13677 2039 | 13678 2040 | 13680 2041 | 13684 2042 | 13686 2043 | 13694 2044 | 13707 2045 | 13716 2046 | 13719 2047 | 13725 2048 | 13728 2049 | 13733 2050 | 13738 2051 | 13752 2052 | 13753 2053 | 13763 2054 | 13767 2055 | 13783 2056 | 13785 2057 | 13787 2058 | 13788 2059 | 13796 2060 | 13810 2061 | 13811 2062 | 13837 2063 | 13842 2064 | 13852 2065 | 13865 2066 | 13875 2067 | 13884 2068 | 13888 2069 | 13896 2070 | 13899 2071 | 13920 2072 | 13923 2073 | 13929 2074 | 13938 2075 | 13949 2076 | 13951 2077 | 13952 2078 | 13956 2079 | 13963 2080 | 13964 2081 | 13974 2082 | 13975 2083 | 13977 2084 | 13979 2085 | 13980 2086 | 13985 2087 | 13986 2088 | 13991 2089 | 13997 2090 | 14000 2091 | 14001 2092 | 14023 2093 | 14027 2094 | 14029 2095 | 14035 2096 | 14065 2097 | 14073 2098 | 14086 2099 | 14089 2100 | 14091 2101 | 14094 2102 | 14095 2103 | 14106 2104 | 14115 2105 | 14116 2106 | 14121 2107 | 14128 2108 | 14129 2109 | 14137 2110 | 14141 2111 | 14157 2112 | 14160 2113 | 14168 2114 | 14173 2115 | 14180 2116 | 14184 2117 | 14186 2118 | 14189 2119 | 14192 2120 | 14198 2121 | 14204 2122 | 14218 2123 | 14225 2124 | 14243 2125 | 14249 2126 | 14253 2127 | 14255 2128 | 14256 2129 | 14265 2130 | 14269 2131 | 14276 2132 | 14278 2133 | 14283 2134 | 14295 2135 | 14317 2136 | 14333 2137 | 14334 2138 | 14343 2139 | 14346 2140 | 14347 2141 | 14352 2142 | 14356 2143 | 14365 2144 | 14369 2145 | 14370 2146 | 14386 2147 | 14399 2148 | 14400 2149 | 14404 2150 | 14414 2151 | 14416 2152 | 14421 2153 | 14434 2154 | 14436 2155 | 14452 2156 | 14463 2157 | 14468 2158 | 14470 2159 | 14472 2160 | 14476 2161 | 14487 2162 | 14493 2163 | 14497 2164 | 14511 2165 | 14520 2166 | 14525 2167 | 14527 2168 | 14534 2169 | 14535 2170 | 14546 2171 | 14548 2172 | 14571 2173 | 14573 2174 | 14582 2175 | 14584 2176 | 14585 2177 | 14590 2178 | 14602 2179 | 14605 2180 | 14608 2181 | 14617 2182 | 14618 2183 | 14622 2184 | 14625 2185 | 14645 2186 | 14661 2187 | 14671 2188 | 14675 2189 | 14687 2190 | 14693 2191 | 14701 2192 | 14708 2193 | 14721 2194 | 14722 2195 | 14723 2196 | 14727 2197 | 14729 2198 | 14732 2199 | 14739 2200 | 14767 2201 | 14770 2202 | 14796 2203 | 14801 2204 | 14811 2205 | 14815 2206 | 14824 2207 | 14828 2208 | 14830 2209 | 14849 2210 | 14853 2211 | 14863 2212 | 14868 2213 | 14883 2214 | 14892 2215 | 14900 2216 | 14908 2217 | 14911 2218 | 14914 2219 | 14918 2220 | 14926 2221 | 14927 2222 | 14932 2223 | 14945 2224 | 14946 2225 | 14956 2226 | 14962 2227 | 14963 2228 | 14964 2229 | 14966 2230 | 14969 2231 | 14970 2232 | 14973 2233 | 14987 2234 | 14992 2235 | 14994 2236 | 15002 2237 | 15004 2238 | 15023 2239 | 15025 2240 | 15030 2241 | 15037 2242 | 15040 2243 | 15041 2244 | 15045 2245 | 15062 2246 | 15081 2247 | 15083 2248 | 15101 2249 | 15102 2250 | 15122 2251 | 15132 2252 | 15151 2253 | 15157 2254 | 15162 2255 | 15169 2256 | 15177 2257 | 15178 2258 | 15183 2259 | 15191 2260 | 15201 2261 | 15215 2262 | 15222 2263 | 15225 2264 | 15226 2265 | 15229 2266 | 15234 2267 | 15249 2268 | 15270 2269 | 15282 2270 | 15299 2271 | 15308 2272 | 15309 2273 | 15310 2274 | 15314 2275 | 15317 2276 | 15322 2277 | 15348 2278 | 15349 2279 | 15351 2280 | 15359 2281 | 15364 2282 | 15384 2283 | 15387 2284 | 15394 2285 | 15411 2286 | 15422 2287 | 15423 2288 | 15426 2289 | 15431 2290 | 15432 2291 | 15433 2292 | 15435 2293 | 15442 2294 | 15455 2295 | 15458 2296 | 15460 2297 | 15461 2298 | 15464 2299 | 15465 2300 | 15467 2301 | 15485 2302 | 15488 2303 | 15500 2304 | 15502 2305 | 15517 2306 | 15525 2307 | 15526 2308 | 15527 2309 | 15533 2310 | 15543 2311 | 15545 2312 | 15549 2313 | 15552 2314 | 15553 2315 | 15559 2316 | 15560 2317 | 15589 2318 | 15591 2319 | 15592 2320 | 15605 2321 | 15607 2322 | 15608 2323 | 15612 2324 | 15615 2325 | 15627 2326 | 15634 2327 | 15648 2328 | 15652 2329 | 15664 2330 | 15665 2331 | 15668 2332 | 15682 2333 | 15696 2334 | 15697 2335 | 15699 2336 | 15702 2337 | 15708 2338 | 15711 2339 | 15712 2340 | 15717 2341 | 15721 2342 | 15730 2343 | 15731 2344 | 15746 2345 | 15754 2346 | 15774 2347 | 15804 2348 | 15810 2349 | 15819 2350 | 15820 2351 | 15828 2352 | 15833 2353 | 15838 2354 | 15840 2355 | 15843 2356 | 15852 2357 | 15856 2358 | 15858 2359 | 15866 2360 | 15888 2361 | 15892 2362 | 15894 2363 | 15913 2364 | 15930 2365 | 15953 2366 | 15962 2367 | 15968 2368 | 15969 2369 | 15979 2370 | 15992 2371 | 15993 2372 | 16000 2373 | 16002 2374 | 16003 2375 | 16008 2376 | 16012 2377 | 16019 2378 | 16030 2379 | 16033 2380 | 16049 2381 | 16053 2382 | 16077 2383 | 16079 2384 | 16096 2385 | 16103 2386 | 16105 2387 | 16122 2388 | 16145 2389 | 16148 2390 | 16150 2391 | 16157 2392 | 16170 2393 | 16173 2394 | 16174 2395 | 16183 2396 | 16184 2397 | 16193 2398 | 16199 2399 | 16207 2400 | 16208 2401 | 16220 2402 | 16224 2403 | 16227 2404 | 16228 2405 | 16234 2406 | 16238 2407 | 16242 2408 | 16251 2409 | 16257 2410 | 16262 2411 | 16263 2412 | 16264 2413 | 16270 2414 | 16281 2415 | 16283 2416 | 16284 2417 | 16285 2418 | 16295 2419 | 16299 2420 | 16302 2421 | 16303 2422 | 16306 2423 | 16307 2424 | 16327 2425 | 16340 2426 | 16355 2427 | 16359 2428 | 16374 2429 | 16385 2430 | 16391 2431 | 16392 2432 | 16394 2433 | 16395 2434 | 16406 2435 | 16411 2436 | 16419 2437 | 16421 2438 | 16423 2439 | 16431 2440 | 16433 2441 | 16436 2442 | 16438 2443 | 16444 2444 | 16464 2445 | 16469 2446 | 16480 2447 | 16488 2448 | 16491 2449 | 16495 2450 | 16498 2451 | 16521 2452 | 16525 2453 | 16533 2454 | 16541 2455 | 16545 2456 | 16548 2457 | 16549 2458 | 16552 2459 | 16553 2460 | 16556 2461 | 16566 2462 | 16567 2463 | 16570 2464 | 16576 2465 | 16589 2466 | 16590 2467 | 16599 2468 | 16602 2469 | 16618 2470 | 16619 2471 | 16623 2472 | 16625 2473 | 16643 2474 | 16654 2475 | 16657 2476 | 16661 2477 | 16665 2478 | 16697 2479 | 16701 2480 | 16705 2481 | 16707 2482 | 16714 2483 | 16718 2484 | 16730 2485 | 16736 2486 | 16743 2487 | 16748 2488 | 16750 2489 | 16751 2490 | 16756 2491 | 16774 2492 | 16779 2493 | 16785 2494 | 16789 2495 | 16793 2496 | 16797 2497 | 16798 2498 | 16799 2499 | 16800 2500 | 16804 2501 | 16807 2502 | 16808 2503 | 16815 2504 | 16829 2505 | 16833 2506 | 16834 2507 | 16848 2508 | 16857 2509 | 16870 2510 | 16896 2511 | 16906 2512 | 16909 2513 | 16923 2514 | 16925 2515 | 16936 2516 | 16938 2517 | 16940 2518 | 16944 2519 | 16970 2520 | 16979 2521 | 16995 2522 | 17001 2523 | 17005 2524 | 17007 2525 | 17028 2526 | 17029 2527 | 17047 2528 | 17050 2529 | 17055 2530 | 17056 2531 | 17073 2532 | 17078 2533 | 17081 2534 | 17083 2535 | 17088 2536 | 17093 2537 | 17099 2538 | 17100 2539 | 17103 2540 | 17104 2541 | 17107 2542 | 17110 2543 | 17124 2544 | 17125 2545 | 17126 2546 | 17127 2547 | 17128 2548 | 17133 2549 | 17137 2550 | 17144 2551 | 17149 2552 | 17152 2553 | 17156 2554 | 17169 2555 | 17176 2556 | 17178 2557 | 17193 2558 | 17194 2559 | 17205 2560 | 17226 2561 | 17265 2562 | 17266 2563 | 17283 2564 | 17285 2565 | 17289 2566 | 17295 2567 | 17296 2568 | 17312 2569 | 17319 2570 | 17326 2571 | 17328 2572 | 17331 2573 | 17341 2574 | 17347 2575 | 17352 2576 | 17361 2577 | 17366 2578 | 17367 2579 | 17369 2580 | 17373 2581 | 17395 2582 | 17397 2583 | 17409 2584 | 17410 2585 | 17411 2586 | 17421 2587 | 17424 2588 | 17425 2589 | 17429 2590 | 17430 2591 | 17437 2592 | 17438 2593 | 17444 2594 | 17465 2595 | 17469 2596 | 17470 2597 | 17471 2598 | 17481 2599 | 17484 2600 | 17489 2601 | 17495 2602 | 17508 2603 | 17520 2604 | 17537 2605 | 17545 2606 | 17556 2607 | 17564 2608 | 17565 2609 | 17569 2610 | 17573 2611 | 17585 2612 | 17586 2613 | 17587 2614 | 17602 2615 | 17603 2616 | 17605 2617 | 17607 2618 | 17610 2619 | 17619 2620 | 17629 2621 | 17634 2622 | 17638 2623 | 17639 2624 | 17642 2625 | 17644 2626 | 17647 2627 | 17656 2628 | 17657 2629 | 17659 2630 | 17668 2631 | 17688 2632 | 17691 2633 | 17702 2634 | 17705 2635 | 17706 2636 | 17733 2637 | 17736 2638 | 17740 2639 | 17741 2640 | 17743 2641 | 17755 2642 | 17756 2643 | 17765 2644 | 17767 2645 | 17796 2646 | 17803 2647 | 17807 2648 | 17813 2649 | 17816 2650 | 17818 2651 | 17822 2652 | 17824 2653 | 17829 2654 | 17834 2655 | 17839 2656 | 17840 2657 | 17841 2658 | 17844 2659 | 17850 2660 | 17853 2661 | 17854 2662 | 17864 2663 | 17872 2664 | 17875 2665 | 17891 2666 | 17902 2667 | 17906 2668 | 17908 2669 | 17911 2670 | 17915 2671 | 17916 2672 | 17918 2673 | 17940 2674 | 17945 2675 | 17949 2676 | 17956 2677 | 17958 2678 | 17960 2679 | 17961 2680 | 17965 2681 | 17969 2682 | 17984 2683 | 17991 2684 | 17998 2685 | 18007 2686 | 18010 2687 | 18013 2688 | 18027 2689 | 18053 2690 | 18055 2691 | 18056 2692 | 18062 2693 | 18066 2694 | 18067 2695 | 18075 2696 | 18077 2697 | 18084 2698 | 18102 2699 | 18122 2700 | 18131 2701 | 18135 2702 | 18140 2703 | 18147 2704 | 18152 2705 | 18194 2706 | 18201 2707 | 18208 2708 | 18213 2709 | 18214 2710 | 18232 2711 | 18259 2712 | 18269 2713 | 18270 2714 | 18272 2715 | 18284 2716 | 18296 2717 | 18297 2718 | 18305 2719 | 18321 2720 | 18333 2721 | 18348 2722 | 18349 2723 | 18353 2724 | 18354 2725 | 18357 2726 | 18361 2727 | 18376 2728 | 18381 2729 | 18387 2730 | 18390 2731 | 18404 2732 | 18410 2733 | 18419 2734 | 18427 2735 | 18431 2736 | 18449 2737 | 18457 2738 | 18460 2739 | 18482 2740 | 18489 2741 | 18491 2742 | 18496 2743 | 18505 2744 | 18512 2745 | 18513 2746 | 18522 2747 | 18546 2748 | 18552 2749 | 18555 2750 | 18559 2751 | 18561 2752 | 18562 2753 | 18584 2754 | 18592 2755 | 18604 2756 | 18605 2757 | 18629 2758 | 18637 2759 | 18638 2760 | 18640 2761 | 18641 2762 | 18645 2763 | 18650 2764 | 18658 2765 | 18665 2766 | 18672 2767 | 18675 2768 | 18678 2769 | 18700 2770 | 18704 2771 | 18706 2772 | 18717 2773 | 18727 2774 | 18732 2775 | 18733 2776 | 18744 2777 | 18745 2778 | 18754 2779 | 18775 2780 | 18776 2781 | 18777 2782 | 18780 2783 | 18781 2784 | 18786 2785 | 18789 2786 | 18790 2787 | 18800 2788 | 18821 2789 | 18822 2790 | 18829 2791 | 18857 2792 | 18859 2793 | 18862 2794 | 18867 2795 | 18872 2796 | 18874 2797 | 18876 2798 | 18880 2799 | 18885 2800 | 18890 2801 | 18896 2802 | 18900 2803 | 18912 2804 | 18914 2805 | 18915 2806 | 18916 2807 | 18928 2808 | 18938 2809 | 18946 2810 | 18952 2811 | 18953 2812 | 18958 2813 | 18971 2814 | 18973 2815 | 18979 2816 | 18980 2817 | 18985 2818 | 18988 2819 | 18990 2820 | 19000 2821 | 19002 2822 | 19011 2823 | 19013 2824 | 19018 2825 | 19021 2826 | 19024 2827 | 19026 2828 | 19027 2829 | 19038 2830 | 19042 2831 | 19047 2832 | 19049 2833 | 19057 2834 | 19059 2835 | 19061 2836 | 19069 2837 | 19086 2838 | 19089 2839 | 19098 2840 | 19100 2841 | 19106 2842 | 19108 2843 | 19116 2844 | 19121 2845 | 19127 2846 | 19130 2847 | 19136 2848 | 19138 2849 | 19139 2850 | 19153 2851 | 19156 2852 | 19157 2853 | 19173 2854 | 19191 2855 | 19201 2856 | 19204 2857 | 19205 2858 | 19216 2859 | 19220 2860 | 19224 2861 | 19233 2862 | 19236 2863 | 19238 2864 | 19243 2865 | 19251 2866 | 19259 2867 | 19262 2868 | 19265 2869 | 19275 2870 | 19277 2871 | 19278 2872 | 19280 2873 | 19295 2874 | 19297 2875 | 19304 2876 | 19305 2877 | 19312 2878 | 19313 2879 | 19316 2880 | 19336 2881 | 19345 2882 | 19349 2883 | 19352 2884 | 19356 2885 | 19358 2886 | 19369 2887 | 19397 2888 | 19401 2889 | 19406 2890 | 19414 2891 | 19425 2892 | 19436 2893 | 19441 2894 | 19443 2895 | 19468 2896 | 19470 2897 | 19474 2898 | 19482 2899 | 19488 2900 | 19513 2901 | 19515 2902 | 19523 2903 | 19528 2904 | 19530 2905 | 19531 2906 | 19534 2907 | 19542 2908 | 19543 2909 | 19547 2910 | 19552 2911 | 19557 2912 | 19558 2913 | 19568 2914 | 19575 2915 | 19579 2916 | 19585 2917 | 19588 2918 | 19591 2919 | 19595 2920 | 19596 2921 | 19597 2922 | 19599 2923 | 19604 2924 | 19605 2925 | 19606 2926 | 19607 2927 | 19612 2928 | 19631 2929 | 19640 2930 | 19642 2931 | 19648 2932 | 19657 2933 | 19658 2934 | 19659 2935 | 19664 2936 | 19675 2937 | 19677 2938 | 19680 2939 | 19682 2940 | 19714 2941 | 19716 2942 | 19731 2943 | 19733 2944 | 19738 2945 | 19742 2946 | 19761 2947 | 19769 2948 | 19780 2949 | 19782 2950 | 19784 2951 | 19785 2952 | 19787 2953 | 19792 2954 | 19799 2955 | 19803 2956 | 19811 2957 | 19824 2958 | 19827 2959 | 19845 2960 | 19852 2961 | 19857 2962 | 19870 2963 | 19880 2964 | 19881 2965 | 19898 2966 | 19905 2967 | 19914 2968 | 19927 2969 | 19945 2970 | 19953 2971 | 19954 2972 | 19955 2973 | 19963 2974 | 19974 2975 | 19981 2976 | 19984 2977 | 19992 2978 | 19997 2979 | 20000 2980 | 20002 2981 | 20006 2982 | 20014 2983 | 20030 2984 | 20034 2985 | 20043 2986 | 20053 2987 | 20057 2988 | 20058 2989 | 20062 2990 | 20069 2991 | 20088 2992 | 20089 2993 | 20091 2994 | 20094 2995 | 20097 2996 | 20113 2997 | 20120 2998 | 20129 2999 | 20137 3000 | 20140 3001 | 20142 3002 | 20143 3003 | 20153 3004 | 20159 3005 | 20180 3006 | 20181 3007 | 20182 3008 | 20185 3009 | 20198 3010 | 20199 3011 | 20203 3012 | 20208 3013 | 20210 3014 | 20213 3015 | 20218 3016 | 20226 3017 | 20228 3018 | 20232 3019 | 20241 3020 | 20243 3021 | 20245 3022 | 20248 3023 | 20249 3024 | 20260 3025 | 20263 3026 | 20289 3027 | 20291 3028 | 20293 3029 | 20301 3030 | 20305 3031 | 20306 3032 | 20313 3033 | 20317 3034 | 20318 3035 | 20323 3036 | 20335 3037 | 20346 3038 | 20349 3039 | 20351 3040 | 20366 3041 | 20374 3042 | 20394 3043 | 20398 3044 | 20432 3045 | 20433 3046 | 20435 3047 | 20462 3048 | 20465 3049 | 20474 3050 | 20476 3051 | 20479 3052 | 20482 3053 | 20485 3054 | 20505 3055 | 20506 3056 | 20507 3057 | 20510 3058 | 20512 3059 | 20513 3060 | 20515 3061 | 20518 3062 | 20533 3063 | 20552 3064 | 20554 3065 | 20556 3066 | 20561 3067 | 20563 3068 | 20564 3069 | 20569 3070 | 20570 3071 | 20572 3072 | 20573 3073 | 20588 3074 | 20592 3075 | 20597 3076 | 20598 3077 | 20599 3078 | 20604 3079 | 20605 3080 | 20606 3081 | 20615 3082 | 20620 3083 | 20628 3084 | 20637 3085 | 20644 3086 | 20652 3087 | 20662 3088 | 20664 3089 | 20675 3090 | 20684 3091 | 20685 3092 | 20693 3093 | 20703 3094 | 20708 3095 | 20709 3096 | 20714 3097 | 20716 3098 | 20728 3099 | 20739 3100 | 20740 3101 | 20741 3102 | 20743 3103 | 20780 3104 | 20782 3105 | 20804 3106 | 20805 3107 | 20808 3108 | 20821 3109 | 20822 3110 | 20824 3111 | 20829 3112 | 20830 3113 | 20836 3114 | 20839 3115 | 20843 3116 | 20848 3117 | 20876 3118 | 20878 3119 | 20881 3120 | 20882 3121 | 20883 3122 | 20888 3123 | 20889 3124 | 20891 3125 | 20901 3126 | 20903 3127 | 20914 3128 | 20940 3129 | 20960 3130 | 20967 3131 | 20974 3132 | 20981 3133 | 20983 3134 | 20986 3135 | 20987 3136 | 20990 3137 | 20992 3138 | 21001 3139 | 21015 3140 | 21016 3141 | 21023 3142 | 21024 3143 | 21039 3144 | 21043 3145 | 21045 3146 | 21049 3147 | 21052 3148 | 21054 3149 | 21067 3150 | 21069 3151 | 21076 3152 | 21079 3153 | 21088 3154 | 21091 3155 | 21093 3156 | 21100 3157 | 21102 3158 | 21103 3159 | 21108 3160 | 21110 3161 | 21112 3162 | 21113 3163 | 21115 3164 | 21120 3165 | 21123 3166 | 21124 3167 | 21130 3168 | 21134 3169 | 21138 3170 | 21139 3171 | 21143 3172 | 21144 3173 | 21152 3174 | 21153 3175 | 21155 3176 | 21165 3177 | 21176 3178 | 21179 3179 | 21186 3180 | 21195 3181 | 21197 3182 | 21202 3183 | 21213 3184 | 21221 3185 | 21228 3186 | 21245 3187 | 21248 3188 | 21256 3189 | 21257 3190 | 21258 3191 | 21278 3192 | 21292 3193 | 21298 3194 | 21299 3195 | 21315 3196 | 21318 3197 | 21320 3198 | 21335 3199 | 21339 3200 | 21342 3201 | 21355 3202 | 21357 3203 | 21360 3204 | 21363 3205 | 21372 3206 | 21376 3207 | 21381 3208 | 21382 3209 | 21390 3210 | 21393 3211 | 21395 3212 | 21397 3213 | 21413 3214 | 21415 3215 | 21419 3216 | 21421 3217 | 21426 3218 | 21443 3219 | 21454 3220 | 21456 3221 | 21457 3222 | 21469 3223 | 21475 3224 | 21478 3225 | 21479 3226 | 21483 3227 | 21487 3228 | 21491 3229 | 21494 3230 | 21506 3231 | 21518 3232 | 21519 3233 | 21523 3234 | 21524 3235 | 21535 3236 | 21536 3237 | 21545 3238 | 21551 3239 | 21555 3240 | 21571 3241 | 21572 3242 | 21591 3243 | 21592 3244 | 21597 3245 | 21606 3246 | 21610 3247 | 21617 3248 | 21621 3249 | 21622 3250 | 21623 3251 | 21625 3252 | 21629 3253 | 21639 3254 | 21647 3255 | 21649 3256 | 21665 3257 | 21667 3258 | 21687 3259 | 21696 3260 | 21700 3261 | 21708 3262 | 21717 3263 | 21726 3264 | 21735 3265 | 21739 3266 | 21740 3267 | 21754 3268 | 21760 3269 | 21766 3270 | 21778 3271 | 21779 3272 | 21780 3273 | 21789 3274 | 21792 3275 | 21796 3276 | 21798 3277 | 21815 3278 | 21823 3279 | 21845 3280 | 21847 3281 | 21848 3282 | 21852 3283 | 21858 3284 | 21859 3285 | 21869 3286 | 21876 3287 | 21882 3288 | 21885 3289 | 21894 3290 | 21903 3291 | 21910 3292 | 21919 3293 | 21925 3294 | 21948 3295 | 21952 3296 | 21953 3297 | 21961 3298 | 21962 3299 | 21971 3300 | 21986 3301 | 21991 3302 | 21998 3303 | 22004 3304 | 22007 3305 | 22011 3306 | 22020 3307 | 22023 3308 | 22029 3309 | 22030 3310 | 22039 3311 | 22044 3312 | 22055 3313 | 22065 3314 | 22066 3315 | 22077 3316 | 22078 3317 | 22080 3318 | 22088 3319 | 22093 3320 | 22101 3321 | 22106 3322 | 22112 3323 | 22119 3324 | 22121 3325 | 22128 3326 | 22155 3327 | 22157 3328 | 22165 3329 | 22166 3330 | 22169 3331 | 22177 3332 | 22183 3333 | 22185 3334 | 22188 3335 | 22191 3336 | 22194 3337 | 22204 3338 | 22211 3339 | 22222 3340 | 22229 3341 | 22255 3342 | 22283 3343 | 22289 3344 | 22306 3345 | 22310 3346 | 22318 3347 | 22327 3348 | 22328 3349 | 22333 3350 | 22337 3351 | 22340 3352 | 22356 3353 | 22379 3354 | 22389 3355 | 22393 3356 | 22400 3357 | 22401 3358 | 22403 3359 | 22418 3360 | 22431 3361 | 22434 3362 | 22436 3363 | 22441 3364 | 22443 3365 | 22449 3366 | 22452 3367 | 22458 3368 | 22462 3369 | 22471 3370 | 22475 3371 | 22480 3372 | 22486 3373 | 22501 3374 | 22508 3375 | 22510 3376 | 22515 3377 | 22520 3378 | 22526 3379 | 22527 3380 | 22537 3381 | 22542 3382 | 22548 3383 | 22552 3384 | 22577 3385 | 22583 3386 | 22588 3387 | 22597 3388 | 22602 3389 | 22603 3390 | 22609 3391 | 22617 3392 | 22625 3393 | 22632 3394 | 22640 3395 | 22664 3396 | 22676 3397 | 22679 3398 | 22686 3399 | 22687 3400 | 22692 3401 | 22735 3402 | 22750 3403 | 22753 3404 | 22756 3405 | 22759 3406 | 22764 3407 | 22765 3408 | 22780 3409 | 22784 3410 | 22786 3411 | 22795 3412 | 22804 3413 | 22823 3414 | 22824 3415 | 22835 3416 | 22853 3417 | 22865 3418 | 22872 3419 | 22874 3420 | 22878 3421 | 22886 3422 | 22899 3423 | 22900 3424 | 22901 3425 | 22909 3426 | 22916 3427 | 22921 3428 | 22940 3429 | 22942 3430 | 22952 3431 | 22953 3432 | 22960 3433 | 22987 3434 | 22991 3435 | 22995 3436 | 22997 3437 | 22999 3438 | 23011 3439 | 23044 3440 | 23058 3441 | 23064 3442 | 23077 3443 | 23087 3444 | 23088 3445 | 23105 3446 | 23117 3447 | 23125 3448 | 23136 3449 | 23150 3450 | 23155 3451 | 23174 3452 | 23200 3453 | 23206 3454 | 23208 3455 | 23223 3456 | 23224 3457 | 23232 3458 | 23235 3459 | 23237 3460 | 23239 3461 | 23270 3462 | 23288 3463 | 23296 3464 | 23311 3465 | 23314 3466 | 23316 3467 | 23322 3468 | 23332 3469 | 23333 3470 | 23336 3471 | 23342 3472 | 23344 3473 | 23354 3474 | 23358 3475 | 23370 3476 | 23381 3477 | 23385 3478 | 23394 3479 | 23395 3480 | 23401 3481 | 23412 3482 | 23417 3483 | 23433 3484 | 23451 3485 | 23455 3486 | 23460 3487 | 23464 3488 | 23474 3489 | 23491 3490 | 23492 3491 | 23496 3492 | 23509 3493 | 23510 3494 | 23514 3495 | 23526 3496 | 23528 3497 | 23535 3498 | 23547 3499 | 23568 3500 | 23569 3501 | 23573 3502 | 23588 3503 | 23591 3504 | 23599 3505 | 23617 3506 | 23641 3507 | 23644 3508 | 23658 3509 | 23659 3510 | 23669 3511 | 23674 3512 | 23676 3513 | 23692 3514 | 23723 3515 | 23724 3516 | 23737 3517 | 23749 3518 | 23757 3519 | 23762 3520 | 23773 3521 | 23800 3522 | 23807 3523 | 23815 3524 | 23820 3525 | 23825 3526 | 23831 3527 | 23841 3528 | 23848 3529 | 23849 3530 | 23858 3531 | 23867 3532 | 23870 3533 | 23878 3534 | 23882 3535 | 23887 3536 | 23897 3537 | 23931 3538 | 23935 3539 | 23959 3540 | 23961 3541 | 23974 3542 | 23991 3543 | 23997 3544 | 24040 3545 | 24043 3546 | 24054 3547 | 24063 3548 | 24064 3549 | 24072 3550 | 24079 3551 | 24087 3552 | 24088 3553 | 24117 3554 | 24136 3555 | 24146 3556 | 24161 3557 | 24165 3558 | 24183 3559 | 24194 3560 | 24197 3561 | 24203 3562 | 24205 3563 | 24209 3564 | 24213 3565 | 24227 3566 | 24228 3567 | 24234 3568 | 24235 3569 | 24245 3570 | 24247 3571 | 24255 3572 | 24261 3573 | 24266 3574 | 24275 3575 | 24276 3576 | 24277 3577 | 24289 3578 | 24293 3579 | 24308 3580 | 24320 3581 | 24327 3582 | 24332 3583 | 24350 3584 | 24394 3585 | 24414 3586 | 24421 3587 | 24428 3588 | 24447 3589 | 24460 3590 | 24474 3591 | 24476 3592 | 24481 3593 | 24493 3594 | 24500 3595 | 24501 3596 | 24510 3597 | 24511 3598 | 24512 3599 | 24513 3600 | 24516 3601 | 24517 3602 | 24524 3603 | 24533 3604 | 24537 3605 | 24550 3606 | 24555 3607 | 24571 3608 | 24604 3609 | 24620 3610 | 24635 3611 | 24638 3612 | 24650 3613 | 24651 3614 | 24656 3615 | 24673 3616 | 24680 3617 | 24716 3618 | 24736 3619 | 24739 3620 | 24746 3621 | 24756 3622 | 24768 3623 | 24777 3624 | 24784 3625 | 24797 3626 | 24835 3627 | 24844 3628 | 24849 3629 | 24851 3630 | 24854 3631 | 24856 3632 | 24859 3633 | 24860 3634 | 24862 3635 | 24894 3636 | 24903 3637 | 24906 3638 | 24920 3639 | 24923 3640 | 24945 3641 | 24979 3642 | 24980 3643 | 24981 3644 | 24988 3645 | 24989 3646 | 24993 3647 | 25000 3648 | -------------------------------------------------------------------------------- /mirflickr/night_query.txt: -------------------------------------------------------------------------------- 1 | 7046 2 | 5995 3 | 18280 4 | 9588 5 | 6532 6 | -------------------------------------------------------------------------------- /mirflickr/night_r1.txt: -------------------------------------------------------------------------------- 1 | 16 2 | 36 3 | 88 4 | 92 5 | 101 6 | 106 7 | 152 8 | 175 9 | 178 10 | 241 11 | 265 12 | 329 13 | 359 14 | 362 15 | 391 16 | 427 17 | 465 18 | 510 19 | 549 20 | 587 21 | 606 22 | 621 23 | 632 24 | 637 25 | 647 26 | 706 27 | 710 28 | 713 29 | 730 30 | 749 31 | 755 32 | 776 33 | 777 34 | 868 35 | 918 36 | 944 37 | 978 38 | 980 39 | 989 40 | 999 41 | 1012 42 | 1048 43 | 1116 44 | 1130 45 | 1188 46 | 1253 47 | 1261 48 | 1301 49 | 1306 50 | 1320 51 | 1355 52 | 1404 53 | 1422 54 | 1445 55 | 1446 56 | 1484 57 | 1550 58 | 1598 59 | 1599 60 | 1633 61 | 1643 62 | 1727 63 | 1739 64 | 1741 65 | 1752 66 | 1830 67 | 1831 68 | 1872 69 | 1890 70 | 1919 71 | 2037 72 | 2100 73 | 2167 74 | 2171 75 | 2174 76 | 2198 77 | 2203 78 | 2225 79 | 2426 80 | 2430 81 | 2441 82 | 2442 83 | 2467 84 | 2497 85 | 2576 86 | 2609 87 | 2651 88 | 2707 89 | 2717 90 | 2781 91 | 2785 92 | 2799 93 | 2806 94 | 2829 95 | 2842 96 | 2846 97 | 2945 98 | 2964 99 | 2969 100 | 3011 101 | 3013 102 | 3037 103 | 3066 104 | 3088 105 | 3094 106 | 3207 107 | 3250 108 | 3343 109 | 3365 110 | 3423 111 | 3456 112 | 3538 113 | 3548 114 | 3561 115 | 3581 116 | 3590 117 | 3652 118 | 3670 119 | 3677 120 | 3704 121 | 3730 122 | 3756 123 | 3780 124 | 3852 125 | 3871 126 | 3893 127 | 3989 128 | 4072 129 | 4105 130 | 4161 131 | 4205 132 | 4215 133 | 4220 134 | 4260 135 | 4368 136 | 4379 137 | 4380 138 | 4399 139 | 4401 140 | 4410 141 | 4439 142 | 4442 143 | 4495 144 | 4517 145 | 4534 146 | 4580 147 | 4596 148 | 4629 149 | 4679 150 | 4704 151 | 4711 152 | 4713 153 | 4726 154 | 4745 155 | 4846 156 | 4848 157 | 4906 158 | 4919 159 | 4959 160 | 4966 161 | 5029 162 | 5044 163 | 5057 164 | 5099 165 | 5144 166 | 5146 167 | 5170 168 | 5177 169 | 5183 170 | 5219 171 | 5281 172 | 5291 173 | 5296 174 | 5300 175 | 5304 176 | 5358 177 | 5440 178 | 5460 179 | 5470 180 | 5474 181 | 5503 182 | 5558 183 | 5575 184 | 5583 185 | 5595 186 | 5660 187 | 5696 188 | 5699 189 | 5788 190 | 5791 191 | 5794 192 | 5799 193 | 5802 194 | 5813 195 | 5840 196 | 5843 197 | 5867 198 | 5880 199 | 5882 200 | 5910 201 | 5941 202 | 5993 203 | 5995 204 | 6032 205 | 6036 206 | 6086 207 | 6118 208 | 6124 209 | 6147 210 | 6157 211 | 6171 212 | 6235 213 | 6236 214 | 6319 215 | 6413 216 | 6426 217 | 6437 218 | 6453 219 | 6454 220 | 6478 221 | 6508 222 | 6532 223 | 6534 224 | 6550 225 | 6552 226 | 6577 227 | 6604 228 | 6647 229 | 6685 230 | 6688 231 | 6699 232 | 6700 233 | 6701 234 | 6805 235 | 6817 236 | 6824 237 | 6862 238 | 6913 239 | 6930 240 | 6951 241 | 6955 242 | 6965 243 | 6971 244 | 6998 245 | 7046 246 | 7076 247 | 7128 248 | 7168 249 | 7188 250 | 7225 251 | 7236 252 | 7252 253 | 7429 254 | 7530 255 | 7538 256 | 7576 257 | 7632 258 | 7753 259 | 7766 260 | 7767 261 | 7795 262 | 7825 263 | 7839 264 | 7924 265 | 7932 266 | 7941 267 | 7958 268 | 7971 269 | 7972 270 | 7975 271 | 7981 272 | 7985 273 | 8017 274 | 8056 275 | 8149 276 | 8182 277 | 8220 278 | 8234 279 | 8239 280 | 8247 281 | 8249 282 | 8251 283 | 8272 284 | 8285 285 | 8287 286 | 8315 287 | 8320 288 | 8325 289 | 8336 290 | 8339 291 | 8364 292 | 8369 293 | 8370 294 | 8371 295 | 8377 296 | 8386 297 | 8392 298 | 8413 299 | 8521 300 | 8537 301 | 8612 302 | 8615 303 | 8620 304 | 8623 305 | 8659 306 | 8670 307 | 8675 308 | 8676 309 | 8689 310 | 8697 311 | 8699 312 | 8735 313 | 8738 314 | 8739 315 | 8752 316 | 8803 317 | 8822 318 | 8823 319 | 8831 320 | 8834 321 | 8861 322 | 8880 323 | 8881 324 | 8910 325 | 8913 326 | 8949 327 | 8950 328 | 8972 329 | 8991 330 | 9035 331 | 9084 332 | 9119 333 | 9149 334 | 9164 335 | 9181 336 | 9187 337 | 9248 338 | 9267 339 | 9271 340 | 9273 341 | 9277 342 | 9278 343 | 9279 344 | 9341 345 | 9349 346 | 9350 347 | 9371 348 | 9421 349 | 9469 350 | 9533 351 | 9586 352 | 9588 353 | 9619 354 | 9647 355 | 9648 356 | 9650 357 | 9711 358 | 9762 359 | 9771 360 | 9772 361 | 9796 362 | 9864 363 | 9878 364 | 9886 365 | 9899 366 | 9901 367 | 9902 368 | 9961 369 | 9974 370 | 9978 371 | 9987 372 | 10000 373 | 10016 374 | 10026 375 | 10030 376 | 10031 377 | 10059 378 | 10088 379 | 10152 380 | 10159 381 | 10170 382 | 10172 383 | 10177 384 | 10245 385 | 10271 386 | 10274 387 | 10280 388 | 10296 389 | 10347 390 | 10349 391 | 10353 392 | 10355 393 | 10386 394 | 10409 395 | 10417 396 | 10427 397 | 10559 398 | 10616 399 | 10681 400 | 10775 401 | 10810 402 | 10830 403 | 11053 404 | 11066 405 | 11079 406 | 11106 407 | 11112 408 | 11171 409 | 11295 410 | 11313 411 | 11330 412 | 11334 413 | 11392 414 | 11449 415 | 11458 416 | 11475 417 | 11530 418 | 11631 419 | 11643 420 | 11651 421 | 11675 422 | 11690 423 | 11712 424 | 11717 425 | 11719 426 | 11728 427 | 11734 428 | 11763 429 | 11767 430 | 11791 431 | 11960 432 | 11965 433 | 11967 434 | 12041 435 | 12059 436 | 12075 437 | 12120 438 | 12201 439 | 12235 440 | 12243 441 | 12301 442 | 12365 443 | 12381 444 | 12388 445 | 12392 446 | 12412 447 | 12434 448 | 12447 449 | 12477 450 | 12537 451 | 12541 452 | 12556 453 | 12594 454 | 12633 455 | 12694 456 | 12760 457 | 12838 458 | 12957 459 | 12977 460 | 12993 461 | 13031 462 | 13059 463 | 13089 464 | 13111 465 | 13227 466 | 13239 467 | 13249 468 | 13296 469 | 13329 470 | 13381 471 | 13447 472 | 13472 473 | 13563 474 | 13580 475 | 13602 476 | 13615 477 | 13718 478 | 13730 479 | 13751 480 | 13769 481 | 13935 482 | 13940 483 | 14058 484 | 14076 485 | 14084 486 | 14087 487 | 14111 488 | 14123 489 | 14174 490 | 14200 491 | 14214 492 | 14398 493 | 14429 494 | 14432 495 | 14466 496 | 14475 497 | 14649 498 | 14714 499 | 14816 500 | 14869 501 | 14905 502 | 14918 503 | 14928 504 | 14931 505 | 14935 506 | 14936 507 | 14944 508 | 14954 509 | 14959 510 | 14978 511 | 14980 512 | 14981 513 | 14982 514 | 14997 515 | 15020 516 | 15160 517 | 15233 518 | 15270 519 | 15280 520 | 15303 521 | 15327 522 | 15456 523 | 15522 524 | 15550 525 | 15568 526 | 15685 527 | 15692 528 | 15752 529 | 15760 530 | 15767 531 | 15791 532 | 15793 533 | 15794 534 | 15964 535 | 15975 536 | 16018 537 | 16176 538 | 16256 539 | 16361 540 | 16403 541 | 16416 542 | 16435 543 | 16455 544 | 16558 545 | 16559 546 | 16597 547 | 16668 548 | 16692 549 | 16727 550 | 16770 551 | 16772 552 | 16775 553 | 16849 554 | 16936 555 | 16957 556 | 17270 557 | 17301 558 | 17418 559 | 17530 560 | 17601 561 | 17723 562 | 17748 563 | 18032 564 | 18054 565 | 18088 566 | 18113 567 | 18154 568 | 18158 569 | 18161 570 | 18167 571 | 18203 572 | 18280 573 | 18284 574 | 18362 575 | 18499 576 | 18645 577 | 18746 578 | 18806 579 | 18825 580 | 18958 581 | 18961 582 | 18966 583 | 19048 584 | 19062 585 | 19064 586 | 19140 587 | 19184 588 | 19228 589 | 19245 590 | 19343 591 | 19393 592 | 19460 593 | 19491 594 | 19593 595 | 19613 596 | 19778 597 | 19825 598 | 19968 599 | 20044 600 | 20154 601 | 20202 602 | 20341 603 | 20386 604 | 20407 605 | 20414 606 | 20448 607 | 20529 608 | 20574 609 | 20748 610 | 20769 611 | 20832 612 | 20920 613 | 20943 614 | 20982 615 | 21153 616 | 21201 617 | 21306 618 | 21441 619 | 21502 620 | 21530 621 | 21612 622 | 21657 623 | 21755 624 | 21929 625 | 22027 626 | 22031 627 | 22076 628 | 22112 629 | 22280 630 | 22317 631 | 22341 632 | 22374 633 | 22474 634 | 22518 635 | 22653 636 | 22677 637 | 22716 638 | 22729 639 | 22745 640 | 23009 641 | 23097 642 | 23160 643 | 23413 644 | 23424 645 | 23535 646 | 23565 647 | 23626 648 | 23793 649 | 23798 650 | 23918 651 | 23994 652 | 24026 653 | 24075 654 | 24085 655 | 24163 656 | 24189 657 | 24214 658 | 24251 659 | 24356 660 | 24580 661 | 24618 662 | 24619 663 | 24630 664 | 24657 665 | 24690 666 | 24706 667 | 24843 668 | 24884 669 | 24932 670 | -------------------------------------------------------------------------------- /mirflickr/people_query.txt: -------------------------------------------------------------------------------- 1 | 5294 2 | 11059 3 | 20566 4 | 20891 5 | 21292 6 | -------------------------------------------------------------------------------- /mirflickr/portrait_query.txt: -------------------------------------------------------------------------------- 1 | 1928 2 | 21141 3 | 16096 4 | 9614 5 | 9171 6 | -------------------------------------------------------------------------------- /mirflickr/river_query.txt: -------------------------------------------------------------------------------- 1 | 3698 2 | 13857 3 | 14719 4 | 20767 5 | 18699 6 | -------------------------------------------------------------------------------- /mirflickr/river_r1.txt: -------------------------------------------------------------------------------- 1 | 253 2 | 347 3 | 506 4 | 666 5 | 1155 6 | 1157 7 | 1304 8 | 1644 9 | 1782 10 | 1898 11 | 2021 12 | 2147 13 | 2254 14 | 2891 15 | 3698 16 | 3731 17 | 3999 18 | 4010 19 | 4892 20 | 4902 21 | 4971 22 | 5188 23 | 5276 24 | 5682 25 | 5904 26 | 6207 27 | 6250 28 | 6321 29 | 6789 30 | 7045 31 | 7127 32 | 7136 33 | 7331 34 | 7339 35 | 7530 36 | 7567 37 | 7735 38 | 8846 39 | 8919 40 | 8952 41 | 8985 42 | 9007 43 | 9062 44 | 9080 45 | 9135 46 | 9574 47 | 9608 48 | 9613 49 | 9663 50 | 9915 51 | 10312 52 | 10420 53 | 10653 54 | 11150 55 | 11281 56 | 11423 57 | 11484 58 | 11610 59 | 11656 60 | 11673 61 | 11734 62 | 11746 63 | 11749 64 | 11800 65 | 11805 66 | 11858 67 | 11869 68 | 11968 69 | 12021 70 | 12041 71 | 12146 72 | 12170 73 | 12233 74 | 12331 75 | 12761 76 | 13127 77 | 13365 78 | 13443 79 | 13573 80 | 13717 81 | 13857 82 | 13878 83 | 13921 84 | 13942 85 | 14219 86 | 14667 87 | 14699 88 | 14719 89 | 14776 90 | 14800 91 | 15024 92 | 15076 93 | 15105 94 | 15161 95 | 15349 96 | 15575 97 | 15596 98 | 15706 99 | 15827 100 | 15899 101 | 16666 102 | 16986 103 | 17032 104 | 17051 105 | 17060 106 | 17369 107 | 17513 108 | 17525 109 | 17627 110 | 17697 111 | 17750 112 | 17894 113 | 18099 114 | 18348 115 | 18699 116 | 19260 117 | 19413 118 | 19429 119 | 19877 120 | 19911 121 | 20023 122 | 20328 123 | 20365 124 | 20635 125 | 20668 126 | 20767 127 | 21048 128 | 21411 129 | 21547 130 | 22148 131 | 22247 132 | 22381 133 | 22712 134 | 22905 135 | 23010 136 | 23114 137 | 23290 138 | 23317 139 | 23795 140 | 23880 141 | 23885 142 | 24080 143 | 24084 144 | 24143 145 | 24643 146 | 24713 147 | 24817 148 | 24865 149 | 24934 150 | -------------------------------------------------------------------------------- /mirflickr/sea_query.txt: -------------------------------------------------------------------------------- 1 | 2517 2 | 16629 3 | 21018 4 | 1743 5 | 23585 6 | -------------------------------------------------------------------------------- /mirflickr/sea_r1.txt: -------------------------------------------------------------------------------- 1 | 67 2 | 123 3 | 165 4 | 310 5 | 312 6 | 405 7 | 757 8 | 783 9 | 897 10 | 1002 11 | 1021 12 | 1088 13 | 1229 14 | 1424 15 | 1544 16 | 1743 17 | 1873 18 | 2182 19 | 2500 20 | 2517 21 | 2533 22 | 2578 23 | 2631 24 | 2713 25 | 2816 26 | 3059 27 | 3106 28 | 3121 29 | 3192 30 | 3238 31 | 3295 32 | 3404 33 | 3416 34 | 3460 35 | 3470 36 | 3477 37 | 3484 38 | 3498 39 | 3734 40 | 3859 41 | 3880 42 | 3970 43 | 4078 44 | 4128 45 | 4423 46 | 4445 47 | 4471 48 | 4484 49 | 4591 50 | 4611 51 | 4615 52 | 4867 53 | 4907 54 | 4971 55 | 5081 56 | 5251 57 | 5391 58 | 5469 59 | 5579 60 | 5590 61 | 5626 62 | 5672 63 | 5704 64 | 5757 65 | 5884 66 | 5963 67 | 5967 68 | 6001 69 | 6011 70 | 6330 71 | 6405 72 | 6621 73 | 6853 74 | 6884 75 | 6887 76 | 7158 77 | 7164 78 | 7300 79 | 7412 80 | 7431 81 | 7503 82 | 7593 83 | 7639 84 | 7713 85 | 7746 86 | 7899 87 | 7917 88 | 8087 89 | 8151 90 | 8252 91 | 8279 92 | 8324 93 | 8541 94 | 8682 95 | 8765 96 | 8821 97 | 9272 98 | 9323 99 | 9582 100 | 9656 101 | 9694 102 | 9769 103 | 9810 104 | 9822 105 | 9866 106 | 9892 107 | 9931 108 | 10038 109 | 10041 110 | 10237 111 | 10259 112 | 10300 113 | 10319 114 | 10395 115 | 10507 116 | 10641 117 | 10871 118 | 10995 119 | 11083 120 | 11389 121 | 11747 122 | 11936 123 | 11969 124 | 12036 125 | 12383 126 | 12595 127 | 12898 128 | 12941 129 | 12980 130 | 12998 131 | 13509 132 | 13713 133 | 13768 134 | 13797 135 | 13840 136 | 13848 137 | 14206 138 | 14265 139 | 14594 140 | 15205 141 | 15212 142 | 15305 143 | 15307 144 | 15442 145 | 15640 146 | 15648 147 | 15978 148 | 16197 149 | 16265 150 | 16370 151 | 16473 152 | 16484 153 | 16629 154 | 16798 155 | 16876 156 | 17126 157 | 17161 158 | 17178 159 | 17200 160 | 17353 161 | 17423 162 | 17461 163 | 17524 164 | 17703 165 | 18036 166 | 18090 167 | 18116 168 | 18164 169 | 18324 170 | 18591 171 | 18939 172 | 19302 173 | 19952 174 | 20014 175 | 20015 176 | 20092 177 | 20320 178 | 20614 179 | 20656 180 | 21018 181 | 21104 182 | 21215 183 | 21231 184 | 21312 185 | 21546 186 | 21621 187 | 21701 188 | 21881 189 | 21887 190 | 22032 191 | 22170 192 | 22569 193 | 22586 194 | 22704 195 | 22740 196 | 22939 197 | 23585 198 | 23621 199 | 23644 200 | 23763 201 | 23776 202 | 23805 203 | 23858 204 | 23870 205 | 23902 206 | 23939 207 | 23941 208 | 24405 209 | 24439 210 | 24446 211 | 24513 212 | 24833 213 | 24895 214 | 24998 215 | -------------------------------------------------------------------------------- /mirflickr/tree_query.txt: -------------------------------------------------------------------------------- 1 | 16020 2 | 12603 3 | 10154 4 | 24811 5 | 11637 6 | -------------------------------------------------------------------------------- /mirflickr/tree_r1.txt: -------------------------------------------------------------------------------- 1 | 15 2 | 77 3 | 79 4 | 113 5 | 135 6 | 164 7 | 213 8 | 233 9 | 254 10 | 277 11 | 303 12 | 315 13 | 391 14 | 434 15 | 478 16 | 520 17 | 538 18 | 574 19 | 594 20 | 627 21 | 696 22 | 700 23 | 748 24 | 762 25 | 776 26 | 808 27 | 818 28 | 866 29 | 871 30 | 999 31 | 1023 32 | 1025 33 | 1028 34 | 1038 35 | 1060 36 | 1143 37 | 1149 38 | 1157 39 | 1167 40 | 1225 41 | 1253 42 | 1260 43 | 1304 44 | 1333 45 | 1355 46 | 1385 47 | 1456 48 | 1483 49 | 1512 50 | 1529 51 | 1673 52 | 1745 53 | 1794 54 | 1803 55 | 1825 56 | 1834 57 | 1835 58 | 1883 59 | 1898 60 | 1908 61 | 1951 62 | 1964 63 | 1978 64 | 1995 65 | 2005 66 | 2019 67 | 2045 68 | 2136 69 | 2161 70 | 2223 71 | 2249 72 | 2300 73 | 2317 74 | 2326 75 | 2329 76 | 2345 77 | 2357 78 | 2449 79 | 2475 80 | 2491 81 | 2493 82 | 2535 83 | 2604 84 | 2687 85 | 2776 86 | 2933 87 | 2939 88 | 2980 89 | 2993 90 | 3001 91 | 3013 92 | 3076 93 | 3115 94 | 3125 95 | 3163 96 | 3198 97 | 3233 98 | 3235 99 | 3241 100 | 3242 101 | 3289 102 | 3300 103 | 3303 104 | 3317 105 | 3325 106 | 3333 107 | 3334 108 | 3372 109 | 3377 110 | 3506 111 | 3516 112 | 3664 113 | 3689 114 | 3697 115 | 3731 116 | 3737 117 | 3789 118 | 3816 119 | 3829 120 | 3895 121 | 3962 122 | 4013 123 | 4040 124 | 4059 125 | 4083 126 | 4119 127 | 4144 128 | 4212 129 | 4217 130 | 4234 131 | 4235 132 | 4242 133 | 4286 134 | 4320 135 | 4333 136 | 4349 137 | 4380 138 | 4388 139 | 4389 140 | 4456 141 | 4461 142 | 4469 143 | 4488 144 | 4502 145 | 4548 146 | 4557 147 | 4747 148 | 4777 149 | 4813 150 | 4822 151 | 4858 152 | 4863 153 | 4869 154 | 4875 155 | 4890 156 | 4894 157 | 4899 158 | 4942 159 | 4971 160 | 4982 161 | 5003 162 | 5043 163 | 5044 164 | 5060 165 | 5093 166 | 5135 167 | 5203 168 | 5212 169 | 5228 170 | 5245 171 | 5334 172 | 5376 173 | 5464 174 | 5503 175 | 5509 176 | 5595 177 | 5607 178 | 5676 179 | 5705 180 | 5720 181 | 5734 182 | 5771 183 | 5778 184 | 5804 185 | 5835 186 | 5843 187 | 5851 188 | 5879 189 | 5889 190 | 5904 191 | 5908 192 | 5956 193 | 5972 194 | 5976 195 | 5997 196 | 6018 197 | 6026 198 | 6061 199 | 6069 200 | 6095 201 | 6176 202 | 6191 203 | 6206 204 | 6227 205 | 6246 206 | 6268 207 | 6309 208 | 6318 209 | 6320 210 | 6322 211 | 6338 212 | 6442 213 | 6467 214 | 6477 215 | 6489 216 | 6492 217 | 6505 218 | 6602 219 | 6645 220 | 6711 221 | 6723 222 | 6815 223 | 6862 224 | 6874 225 | 6887 226 | 6921 227 | 6991 228 | 6998 229 | 7006 230 | 7016 231 | 7018 232 | 7045 233 | 7155 234 | 7222 235 | 7226 236 | 7237 237 | 7317 238 | 7325 239 | 7347 240 | 7397 241 | 7448 242 | 7461 243 | 7536 244 | 7672 245 | 7771 246 | 7778 247 | 7820 248 | 7824 249 | 7884 250 | 7943 251 | 7953 252 | 8028 253 | 8033 254 | 8108 255 | 8114 256 | 8115 257 | 8140 258 | 8147 259 | 8155 260 | 8183 261 | 8189 262 | 8251 263 | 8265 264 | 8275 265 | 8287 266 | 8302 267 | 8346 268 | 8356 269 | 8368 270 | 8389 271 | 8405 272 | 8469 273 | 8476 274 | 8502 275 | 8536 276 | 8577 277 | 8579 278 | 8589 279 | 8613 280 | 8695 281 | 8696 282 | 8702 283 | 8732 284 | 8763 285 | 8794 286 | 8800 287 | 8816 288 | 8834 289 | 8889 290 | 8940 291 | 8941 292 | 9001 293 | 9020 294 | 9048 295 | 9172 296 | 9178 297 | 9278 298 | 9287 299 | 9295 300 | 9312 301 | 9313 302 | 9322 303 | 9351 304 | 9365 305 | 9376 306 | 9410 307 | 9476 308 | 9521 309 | 9527 310 | 9605 311 | 9635 312 | 9636 313 | 9720 314 | 9754 315 | 9783 316 | 9850 317 | 9893 318 | 9902 319 | 9940 320 | 9974 321 | 9979 322 | 10096 323 | 10154 324 | 10164 325 | 10192 326 | 10198 327 | 10237 328 | 10265 329 | 10275 330 | 10317 331 | 10321 332 | 10342 333 | 10367 334 | 10448 335 | 10461 336 | 10470 337 | 10509 338 | 10633 339 | 10636 340 | 10639 341 | 10688 342 | 10762 343 | 10771 344 | 10794 345 | 10859 346 | 10889 347 | 10924 348 | 10988 349 | 11011 350 | 11017 351 | 11033 352 | 11064 353 | 11089 354 | 11104 355 | 11121 356 | 11130 357 | 11160 358 | 11214 359 | 11237 360 | 11252 361 | 11259 362 | 11346 363 | 11423 364 | 11543 365 | 11578 366 | 11632 367 | 11637 368 | 11656 369 | 11658 370 | 11680 371 | 11685 372 | 11760 373 | 11800 374 | 11802 375 | 11865 376 | 11920 377 | 11971 378 | 12019 379 | 12033 380 | 12120 381 | 12137 382 | 12150 383 | 12165 384 | 12183 385 | 12190 386 | 12207 387 | 12227 388 | 12275 389 | 12282 390 | 12341 391 | 12343 392 | 12346 393 | 12383 394 | 12395 395 | 12433 396 | 12529 397 | 12560 398 | 12578 399 | 12585 400 | 12593 401 | 12603 402 | 12653 403 | 12698 404 | 12718 405 | 12748 406 | 12756 407 | 12765 408 | 12771 409 | 12774 410 | 12790 411 | 12800 412 | 12814 413 | 12815 414 | 12817 415 | 12862 416 | 12889 417 | 12958 418 | 13155 419 | 13193 420 | 13405 421 | 13417 422 | 13420 423 | 13430 424 | 13432 425 | 13503 426 | 13632 427 | 13646 428 | 13655 429 | 13854 430 | 14061 431 | 14063 432 | 14120 433 | 14220 434 | 14314 435 | 14336 436 | 14371 437 | 14378 438 | 14397 439 | 14481 440 | 14491 441 | 14572 442 | 14639 443 | 14768 444 | 14864 445 | 14884 446 | 14950 447 | 15076 448 | 15148 449 | 15182 450 | 15258 451 | 15307 452 | 15326 453 | 15376 454 | 15377 455 | 15398 456 | 15502 457 | 15644 458 | 15698 459 | 15706 460 | 15752 461 | 15849 462 | 15927 463 | 16020 464 | 16074 465 | 16157 466 | 16192 467 | 16233 468 | 16294 469 | 16313 470 | 16601 471 | 16616 472 | 16649 473 | 16685 474 | 16688 475 | 16704 476 | 16741 477 | 16742 478 | 16825 479 | 16847 480 | 16895 481 | 16994 482 | 17074 483 | 17076 484 | 17234 485 | 17317 486 | 17597 487 | 17598 488 | 17752 489 | 17794 490 | 18072 491 | 18186 492 | 18208 493 | 18220 494 | 18227 495 | 18273 496 | 18371 497 | 18374 498 | 18590 499 | 18627 500 | 18644 501 | 18662 502 | 18719 503 | 18774 504 | 18815 505 | 18819 506 | 18825 507 | 18877 508 | 18955 509 | 18995 510 | 19035 511 | 19167 512 | 19234 513 | 19310 514 | 19340 515 | 19387 516 | 19405 517 | 19464 518 | 19466 519 | 19480 520 | 19593 521 | 19625 522 | 19670 523 | 19692 524 | 19776 525 | 19911 526 | 20035 527 | 20074 528 | 20100 529 | 20122 530 | 20135 531 | 20312 532 | 20350 533 | 20511 534 | 20549 535 | 20562 536 | 20567 537 | 20653 538 | 20729 539 | 20834 540 | 20899 541 | 20945 542 | 21023 543 | 21048 544 | 21058 545 | 21187 546 | 21194 547 | 21244 548 | 21284 549 | 21313 550 | 21324 551 | 21379 552 | 21436 553 | 21445 554 | 21567 555 | 21578 556 | 21794 557 | 21797 558 | 21862 559 | 21924 560 | 21974 561 | 21979 562 | 22057 563 | 22076 564 | 22122 565 | 22249 566 | 22256 567 | 22364 568 | 22404 569 | 22442 570 | 22470 571 | 22485 572 | 22533 573 | 22562 574 | 22567 575 | 22578 576 | 22690 577 | 22774 578 | 22848 579 | 22981 580 | 23086 581 | 23113 582 | 23132 583 | 23148 584 | 23153 585 | 23171 586 | 23188 587 | 23233 588 | 23244 589 | 23292 590 | 23300 591 | 23321 592 | 23325 593 | 23338 594 | 23365 595 | 23396 596 | 23398 597 | 23403 598 | 23409 599 | 23421 600 | 23443 601 | 23461 602 | 23486 603 | 23493 604 | 23495 605 | 23511 606 | 23523 607 | 23635 608 | 23657 609 | 23690 610 | 23740 611 | 23767 612 | 23781 613 | 23791 614 | 23814 615 | 23865 616 | 23874 617 | 23896 618 | 23909 619 | 23915 620 | 23929 621 | 23942 622 | 23944 623 | 24014 624 | 24031 625 | 24041 626 | 24048 627 | 24106 628 | 24144 629 | 24155 630 | 24187 631 | 24188 632 | 24213 633 | 24218 634 | 24227 635 | 24254 636 | 24273 637 | 24307 638 | 24348 639 | 24373 640 | 24380 641 | 24386 642 | 24398 643 | 24420 644 | 24432 645 | 24442 646 | 24444 647 | 24448 648 | 24462 649 | 24483 650 | 24515 651 | 24527 652 | 24561 653 | 24628 654 | 24647 655 | 24681 656 | 24683 657 | 24726 658 | 24727 659 | 24811 660 | 24822 661 | 24832 662 | 24866 663 | 24899 664 | 24921 665 | 24966 666 | 24968 667 | 24979 668 | 24999 669 | -------------------------------------------------------------------------------- /model/VGG_ILSVRC_16_layers_deploy.prototxt: -------------------------------------------------------------------------------- 1 | name: "VGG_ILSVRC_16_layers" 2 | input: "data" 3 | input_dim: 10 4 | input_dim: 3 5 | input_dim: 224 6 | input_dim: 224 7 | layers { 8 | bottom: "data" 9 | top: "conv1_1" 10 | name: "conv1_1" 11 | type: CONVOLUTION 12 | convolution_param { 13 | num_output: 64 14 | pad: 1 15 | kernel_size: 3 16 | } 17 | } 18 | layers { 19 | bottom: "conv1_1" 20 | top: "conv1_1" 21 | name: "relu1_1" 22 | type: RELU 23 | } 24 | layers { 25 | bottom: "conv1_1" 26 | top: "conv1_2" 27 | name: "conv1_2" 28 | type: CONVOLUTION 29 | convolution_param { 30 | num_output: 64 31 | pad: 1 32 | kernel_size: 3 33 | } 34 | } 35 | layers { 36 | bottom: "conv1_2" 37 | top: "conv1_2" 38 | name: "relu1_2" 39 | type: RELU 40 | } 41 | layers { 42 | bottom: "conv1_2" 43 | top: "pool1" 44 | name: "pool1" 45 | type: POOLING 46 | pooling_param { 47 | pool: MAX 48 | kernel_size: 2 49 | stride: 2 50 | } 51 | } 52 | layers { 53 | bottom: "pool1" 54 | top: "conv2_1" 55 | name: "conv2_1" 56 | type: CONVOLUTION 57 | convolution_param { 58 | num_output: 128 59 | pad: 1 60 | kernel_size: 3 61 | } 62 | } 63 | layers { 64 | bottom: "conv2_1" 65 | top: "conv2_1" 66 | name: "relu2_1" 67 | type: RELU 68 | } 69 | layers { 70 | bottom: "conv2_1" 71 | top: "conv2_2" 72 | name: "conv2_2" 73 | type: CONVOLUTION 74 | convolution_param { 75 | num_output: 128 76 | pad: 1 77 | kernel_size: 3 78 | } 79 | } 80 | layers { 81 | bottom: "conv2_2" 82 | top: "conv2_2" 83 | name: "relu2_2" 84 | type: RELU 85 | } 86 | layers { 87 | bottom: "conv2_2" 88 | top: "pool2" 89 | name: "pool2" 90 | type: POOLING 91 | pooling_param { 92 | pool: MAX 93 | kernel_size: 2 94 | stride: 2 95 | } 96 | } 97 | layers { 98 | bottom: "pool2" 99 | top: "conv3_1" 100 | name: "conv3_1" 101 | type: CONVOLUTION 102 | convolution_param { 103 | num_output: 256 104 | pad: 1 105 | kernel_size: 3 106 | } 107 | } 108 | layers { 109 | bottom: "conv3_1" 110 | top: "conv3_1" 111 | name: "relu3_1" 112 | type: RELU 113 | } 114 | layers { 115 | bottom: "conv3_1" 116 | top: "conv3_2" 117 | name: "conv3_2" 118 | type: CONVOLUTION 119 | convolution_param { 120 | num_output: 256 121 | pad: 1 122 | kernel_size: 3 123 | } 124 | } 125 | layers { 126 | bottom: "conv3_2" 127 | top: "conv3_2" 128 | name: "relu3_2" 129 | type: RELU 130 | } 131 | layers { 132 | bottom: "conv3_2" 133 | top: "conv3_3" 134 | name: "conv3_3" 135 | type: CONVOLUTION 136 | convolution_param { 137 | num_output: 256 138 | pad: 1 139 | kernel_size: 3 140 | } 141 | } 142 | layers { 143 | bottom: "conv3_3" 144 | top: "conv3_3" 145 | name: "relu3_3" 146 | type: RELU 147 | } 148 | layers { 149 | bottom: "conv3_3" 150 | top: "pool3" 151 | name: "pool3" 152 | type: POOLING 153 | pooling_param { 154 | pool: MAX 155 | kernel_size: 2 156 | stride: 2 157 | } 158 | } 159 | layers { 160 | bottom: "pool3" 161 | top: "conv4_1" 162 | name: "conv4_1" 163 | type: CONVOLUTION 164 | convolution_param { 165 | num_output: 512 166 | pad: 1 167 | kernel_size: 3 168 | } 169 | } 170 | layers { 171 | bottom: "conv4_1" 172 | top: "conv4_1" 173 | name: "relu4_1" 174 | type: RELU 175 | } 176 | layers { 177 | bottom: "conv4_1" 178 | top: "conv4_2" 179 | name: "conv4_2" 180 | type: CONVOLUTION 181 | convolution_param { 182 | num_output: 512 183 | pad: 1 184 | kernel_size: 3 185 | } 186 | } 187 | layers { 188 | bottom: "conv4_2" 189 | top: "conv4_2" 190 | name: "relu4_2" 191 | type: RELU 192 | } 193 | layers { 194 | bottom: "conv4_2" 195 | top: "conv4_3" 196 | name: "conv4_3" 197 | type: CONVOLUTION 198 | convolution_param { 199 | num_output: 512 200 | pad: 1 201 | kernel_size: 3 202 | } 203 | } 204 | layers { 205 | bottom: "conv4_3" 206 | top: "conv4_3" 207 | name: "relu4_3" 208 | type: RELU 209 | } 210 | layers { 211 | bottom: "conv4_3" 212 | top: "pool4" 213 | name: "pool4" 214 | type: POOLING 215 | pooling_param { 216 | pool: MAX 217 | kernel_size: 2 218 | stride: 2 219 | } 220 | } 221 | layers { 222 | bottom: "pool4" 223 | top: "conv5_1" 224 | name: "conv5_1" 225 | type: CONVOLUTION 226 | convolution_param { 227 | num_output: 512 228 | pad: 1 229 | kernel_size: 3 230 | } 231 | } 232 | layers { 233 | bottom: "conv5_1" 234 | top: "conv5_1" 235 | name: "relu5_1" 236 | type: RELU 237 | } 238 | layers { 239 | bottom: "conv5_1" 240 | top: "conv5_2" 241 | name: "conv5_2" 242 | type: CONVOLUTION 243 | convolution_param { 244 | num_output: 512 245 | pad: 1 246 | kernel_size: 3 247 | } 248 | } 249 | layers { 250 | bottom: "conv5_2" 251 | top: "conv5_2" 252 | name: "relu5_2" 253 | type: RELU 254 | } 255 | layers { 256 | bottom: "conv5_2" 257 | top: "conv5_3" 258 | name: "conv5_3" 259 | type: CONVOLUTION 260 | convolution_param { 261 | num_output: 512 262 | pad: 1 263 | kernel_size: 3 264 | } 265 | } 266 | layers { 267 | bottom: "conv5_3" 268 | top: "conv5_3" 269 | name: "relu5_3" 270 | type: RELU 271 | } 272 | layers { 273 | bottom: "conv5_3" 274 | top: "pool5" 275 | name: "pool5" 276 | type: POOLING 277 | pooling_param { 278 | pool: MAX 279 | kernel_size: 2 280 | stride: 2 281 | } 282 | } 283 | layers { 284 | bottom: "pool5" 285 | top: "fc6" 286 | name: "fc6" 287 | type: INNER_PRODUCT 288 | inner_product_param { 289 | num_output: 4096 290 | } 291 | } 292 | layers { 293 | bottom: "fc6" 294 | top: "fc6" 295 | name: "relu6" 296 | type: RELU 297 | } 298 | layers { 299 | bottom: "fc6" 300 | top: "fc6" 301 | name: "drop6" 302 | type: DROPOUT 303 | dropout_param { 304 | dropout_ratio: 0.5 305 | } 306 | } 307 | layers { 308 | bottom: "fc6" 309 | top: "fc7" 310 | name: "fc7" 311 | type: INNER_PRODUCT 312 | inner_product_param { 313 | num_output: 4096 314 | } 315 | } 316 | layers { 317 | bottom: "fc7" 318 | top: "fc7" 319 | name: "relu7" 320 | type: RELU 321 | } 322 | layers { 323 | bottom: "fc7" 324 | top: "fc7" 325 | name: "drop7" 326 | type: DROPOUT 327 | dropout_param { 328 | dropout_ratio: 0.5 329 | } 330 | } 331 | layers { 332 | bottom: "fc7" 333 | top: "fc8" 334 | name: "fc8" 335 | type: INNER_PRODUCT 336 | inner_product_param { 337 | num_output: 1000 338 | } 339 | } 340 | layers { 341 | bottom: "fc8" 342 | top: "prob" 343 | name: "prob" 344 | type: SOFTMAX 345 | } 346 | -------------------------------------------------------------------------------- /model/image_mean.txt: -------------------------------------------------------------------------------- 1 | 103.939 116.779 123.68 2 | -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | from glob import glob 3 | 4 | try: 5 | 6 | from tqdm import tqdm 7 | 8 | def ptqdm(*args, **kwargs): 9 | return tqdm(*args, **kwargs) 10 | 11 | except ImportError: 12 | 13 | def tqdm(it, **kwargs): 14 | return it 15 | 16 | def ptqdm(it, **kwargs): 17 | for i, x in enumerate(it): 18 | print('Round {}'.format(i+1)) 19 | yield x 20 | 21 | 22 | 23 | def get_dataset_images(img_dir): 24 | """ Searches for image files in a given directory. 25 | 26 | Files are expected to be named according to the MIRFLICKR format, starting with "im1.jpg" and counting continously. 27 | 28 | img_dir - Path of the directory containing the images. 29 | 30 | Returns: list of filenames of images found in the directory, sorted by ascending number. 31 | """ 32 | 33 | images = [] 34 | i = 1 35 | while os.path.exists(os.path.join(img_dir, 'im{}.jpg'.format(i))): 36 | images.append(os.path.join(img_dir, 'im{}.jpg'.format(i))) 37 | i += 1 38 | return images 39 | 40 | 41 | def get_dataset_queries(gt_dir, query_dir = None, dup_file = None): 42 | """ Loads pre-defined queries. 43 | 44 | gt_dir - Directory containing ground-truth files, named like "_r1.txt". Each file contains a list of 45 | integer image IDs (counting from 1) that belong to the respective class, one ID per line. 46 | 47 | query_dir - Directory containing query files, named like "_query.txt". Each file contains a list of 48 | integer image IDs (counting from 1) to be used as query for this class, one query per line. 49 | If `None` is given, all images found in the ground-truth list files will be used as queries. 50 | 51 | dup_file - Path to a file containing a list of IDs of duplicate images, one list per line. Duplicates of 52 | the first image on each line will be ignored. 53 | 54 | Returns: a dictionary mapping query IDs to dictionaries with keys 'img_id' and 'relevant'. 'img_id' gives 55 | the ID of the query image and 'relevant' points to a list of IDs of images relevant for this query. 56 | """ 57 | 58 | if query_dir is None: 59 | query_files = glob(os.path.join(gt_dir, '*_r1.txt')) 60 | else: 61 | query_files = glob(os.path.join(query_dir, '*_query.txt')) 62 | 63 | duplicates = {} 64 | if dup_file is not None: 65 | with open(dup_file) as df: 66 | for l in df: 67 | if l.strip() != '': 68 | dup_ids = [int(x) - 1 for x in l.strip().split()] 69 | for di in dup_ids[1:]: 70 | duplicates[di] = dup_ids[0] 71 | 72 | queries = {} 73 | for query_file in query_files: 74 | topic = os.path.basename(query_file)[:-7] if query_dir is None else os.path.basename(query_file)[:-10] 75 | label_file = query_file if query_dir is None else os.path.join(gt_dir, os.path.basename(query_file)[:-10] + '_r1.txt') 76 | if os.path.exists(label_file): 77 | 78 | with open(label_file) as lf: 79 | relevant = set(int(l.strip()) - 1 for l in lf if (l.strip() != '') and ((int(l.strip()) - 1) not in duplicates)) 80 | 81 | with open(query_file) as qf: 82 | query_imgs = set(duplicates[int(l.strip()) - 1] if (int(l.strip()) - 1) in duplicates else int(l.strip()) - 1 for l in qf if l.strip() != '') 83 | 84 | for qid in query_imgs: 85 | queries['{}_{}'.format(topic, qid)] = { 'img_id' : qid, 'relevant' : relevant, 'ignore' : set(duplicates.keys()) } 86 | 87 | return queries 88 | 89 | 90 | def print_metrics(metrics, tabular = True): 91 | """ Prints evaluation results. 92 | 93 | metrics - Dictionary mapping benchmark names to dictionaries mapping metric names to values. 94 | 95 | tabular - If True, results will be printed as table, otherwise in CSV format. 96 | """ 97 | 98 | METRICS = ['AP', 'P@1', 'P@10', 'P@50', 'P@100', 'NDCG', 'NDCG@100'] 99 | 100 | if tabular: 101 | 102 | print() 103 | 104 | # Print header 105 | max_name_len = max(len(bench) for bench in metrics.keys()) 106 | print(' | '.join([' ' * max_name_len] + ['{:^6s}'.format(metric) for metric in METRICS])) 107 | print('-' * (max_name_len + sum(max(len(metric), 6) + 3 for metric in METRICS))) 108 | 109 | # Print result rows 110 | for bench, results in metrics.items(): 111 | print('{:{}s} | {}'.format(bench, max_name_len, ' | '.join('{:>{}.{}f}'.format(results[metric], max(len(metric), 6), 4) for metric in METRICS))) 112 | 113 | print() 114 | 115 | else: 116 | 117 | print(';'.join(['Benchmark'] + METRICS)) 118 | for bench, results in metrics.items(): 119 | print('{};{}'.format(bench, ';'.join('{:.7g}'.format(results[metric]) for metric in METRICS))) 120 | --------------------------------------------------------------------------------