├── LICENSE ├── README.md ├── basic_data_loading.ipynb ├── data ├── annotations_classify_task.pkl ├── annotations_contains_task.pkl ├── image_info.pkl ├── imagenet_label_map.npy ├── imagenet_label_to_wnid.npy ├── model_predictions.pkl └── superclasses.npy └── pipeline.jpg /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Madry Lab 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Fine-grained annotations for the ImageNet validation set 2 | This is the data collected for our paper "From ImageNet to Image Classification: Contextualizing Progress on Benchmarks" ([preprint](https://arxiv.org/abs/2005.11295), [blog](https://gradientscience.org/benchmarks)). 3 | 4 |  5 | 6 | ## Parsing the annotations 7 | Our annotations are available as `pandas` dataframes in `data/annotations_{contains,classify}_task.pkl`. These dataframes included both the raw data collected (after quality control), as well as the aggregate quantities we computed for our analysis. The rest of the files in `data` contain auxiliary information. 8 | 9 | The easiest way to navigate these files is by running the jupyter notebook (`basic_data_loading.ipynb`) which loads all files, providing an explanation for each field. 10 | 11 | ## Citation 12 | 13 | ``` 14 | @inproceedings{tsipras2020imagenet, 15 | title={From ImageNet to Image Classification: Contextualizing Progress on Benchmarks}, 16 | author={Dimitris Tsipras and Shibani Santurkar and Logan Engstrom and Andrew Ilyas and Aleksander Madry}, 17 | booktitle={ArXiv preprint arXiv:2005.11295}, 18 | year={2020} 19 | } 20 | ``` 21 | 22 | -------------------------------------------------------------------------------- /basic_data_loading.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Load and visualize data" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import pandas as pd\n", 17 | "import numpy as np\n", 18 | "from PIL import Image\n", 19 | "import matplotlib.pyplot as plt\n", 20 | "\n", 21 | "%matplotlib inline" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "#### ImageNet label map" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "label_map = np.load('./data/imagenet_label_map.npy', allow_pickle=True).item()\n", 38 | "label_to_wnid = np.load('./data/imagenet_label_to_wnid.npy', allow_pickle=True).item()" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": {}, 44 | "source": [ 45 | "### ImageNet superclasses" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": 3, 51 | "metadata": {}, 52 | "outputs": [ 53 | { 54 | "name": "stdout", 55 | "output_type": "stream", 56 | "text": [ 57 | "Dogs, #ImageNet classes: 130, Sample classes: [151, 152, 153, 154, 155]\n", 58 | "Other mammals, #ImageNet classes: 88, Sample classes: [286, 287, 288, 289, 290]\n", 59 | "Birds, #ImageNet classes: 59, Sample classes: [128, 129, 130, 131, 132]\n", 60 | "Reptiles, fish, amphibians, #ImageNet classes: 60, Sample classes: [33, 34, 35, 36, 37]\n", 61 | "inverterbrates, #ImageNet classes: 61, Sample classes: [300, 301, 302, 303, 304]\n", 62 | "Food, plants, fungi, #ImageNet classes: 63, Sample classes: [992, 993, 994, 995, 996]\n", 63 | "Devices, #ImageNet classes: 172, Sample classes: [513, 517, 527, 530, 531]\n", 64 | "Structures, furnishing, #ImageNet classes: 90, Sample classes: [516, 648, 520, 526, 532]\n", 65 | "Clothes, covering, #ImageNet classes: 92, Sample classes: [643, 515, 518, 775, 903]\n", 66 | "Implements, containers, misc. objects, #ImageNet classes: 117, Sample classes: [512, 644, 521, 523, 909]\n", 67 | "vehicles, #ImageNet classes: 68, Sample classes: [779, 780, 654, 913, 914]\n" 68 | ] 69 | } 70 | ], 71 | "source": [ 72 | "superclasses = np.load('./data/superclasses.npy', allow_pickle=True).item()\n", 73 | "\n", 74 | "for k, v in superclasses.items():\n", 75 | " print(f\"{k}, #ImageNet classes: {len(v)}, Sample classes: {v[:5]}\")" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | "## Model predictions\n", 83 | "\n", 84 | "The dataframe contains the following information:\n", 85 | "\n", 86 | "____________________________________________________________________________________________________\n", 87 | "\n", 88 | "`image_number`: number of image as per an unshuffled loader\n", 89 | "\n", 90 | "`imagenet_label`: ImageNet label\n", 91 | "\n", 92 | "____________________________________________________________________________________________________\n", 93 | "\n", 94 | "`pred_{MODEL_NAME}`: Top-5 predictions of model\n", 95 | "\n", 96 | "`top1_{MODEL_NAME}`: True if label == Top-1 model prediction\n", 97 | "\n", 98 | "`top5_{MODEL_NAME}`: True if label is in Top-5 model predictions" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": 4, 104 | "metadata": {}, 105 | "outputs": [], 106 | "source": [ 107 | "model_preds = pd.read_pickle(\"./data/model_predictions.pkl\")" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 5, 113 | "metadata": {}, 114 | "outputs": [ 115 | { 116 | "data": { 117 | "text/html": [ 118 | "
\n", 136 | " | img_number | \n", 137 | "imagenet_label | \n", 138 | "pred_googlenet | \n", 139 | "pred_alexnet | \n", 140 | "pred_resnet50 | \n", 141 | "pred_resnet101 | \n", 142 | "pred_inception_v3 | \n", 143 | "pred_vgg16 | \n", 144 | "pred_wide_resnet50_2 | \n", 145 | "pred_densenet161 | \n", 146 | "... | \n", 147 | "top5_googlenet | \n", 148 | "top5_alexnet | \n", 149 | "top5_resnet50 | \n", 150 | "top5_resnet101 | \n", 151 | "top5_inception_v3 | \n", 152 | "top5_vgg16 | \n", 153 | "top5_wide_resnet50_2 | \n", 154 | "top5_densenet161 | \n", 155 | "top5_mobilenet_v2 | \n", 156 | "top5_tf_efficientnet_b7 | \n", 157 | "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", 162 | "0 | \n", 163 | "0 | \n", 164 | "[48, 39, 0, 391, 394] | \n", 165 | "[395, 0, 48, 390, 394] | \n", 166 | "[394, 0, 395, 391, 389] | \n", 167 | "[394, 0, 149, 395, 391] | \n", 168 | "[0, 394, 48, 36, 391] | \n", 169 | "[395, 394, 391, 48, 39] | \n", 170 | "[394, 391, 0, 389, 390] | \n", 171 | "[394, 0, 391, 48, 389] | \n", 172 | "... | \n", 173 | "True | \n", 174 | "True | \n", 175 | "True | \n", 176 | "True | \n", 177 | "True | \n", 178 | "False | \n", 179 | "True | \n", 180 | "True | \n", 181 | "True | \n", 182 | "True | \n", 183 | "
1 | \n", 186 | "1 | \n", 187 | "0 | \n", 188 | "[0, 391, 122, 766, 389] | \n", 189 | "[764, 763, 413, 758, 391] | \n", 190 | "[0, 389, 391, 395, 758] | \n", 191 | "[0, 389, 391, 758, 394] | \n", 192 | "[391, 0, 758, 389, 764] | \n", 193 | "[0, 391, 758, 389, 395] | \n", 194 | "[0, 758, 391, 389, 395] | \n", 195 | "[0, 391, 389, 758, 395] | \n", 196 | "... | \n", 197 | "True | \n", 198 | "False | \n", 199 | "True | \n", 200 | "True | \n", 201 | "True | \n", 202 | "True | \n", 203 | "True | \n", 204 | "True | \n", 205 | "True | \n", 206 | "True | \n", 207 | "
2 | \n", 210 | "2 | \n", 211 | "0 | \n", 212 | "[0, 391, 346, 693, 223] | \n", 213 | "[929, 472, 444, 870, 655] | \n", 214 | "[0, 391, 693, 389, 390] | \n", 215 | "[0, 693, 758, 391, 472] | \n", 216 | "[837, 703, 419, 836, 785] | \n", 217 | "[0, 758, 391, 223, 237] | \n", 218 | "[0, 693, 391, 472, 389] | \n", 219 | "[0, 391, 395, 693, 472] | \n", 220 | "... | \n", 221 | "True | \n", 222 | "False | \n", 223 | "True | \n", 224 | "True | \n", 225 | "False | \n", 226 | "True | \n", 227 | "True | \n", 228 | "True | \n", 229 | "True | \n", 230 | "True | \n", 231 | "
3 | \n", 234 | "3 | \n", 235 | "0 | \n", 236 | "[0, 391, 389, 394, 395] | \n", 237 | "[0, 389, 395, 36, 955] | \n", 238 | "[0, 389, 391, 395, 394] | \n", 239 | "[0, 389, 391, 758, 997] | \n", 240 | "[0, 391, 122, 955, 933] | \n", 241 | "[0, 389, 30, 758, 390] | \n", 242 | "[0, 389, 395, 955, 36] | \n", 243 | "[0, 389, 391, 394, 397] | \n", 244 | "... | \n", 245 | "True | \n", 246 | "True | \n", 247 | "True | \n", 248 | "True | \n", 249 | "True | \n", 250 | "True | \n", 251 | "True | \n", 252 | "True | \n", 253 | "True | \n", 254 | "True | \n", 255 | "
4 | \n", 258 | "4 | \n", 259 | "0 | \n", 260 | "[0, 391, 1, 389, 556] | \n", 261 | "[0, 389, 114, 390, 939] | \n", 262 | "[0, 389, 391, 758, 955] | \n", 263 | "[0, 955, 389, 382, 1] | \n", 264 | "[0, 389, 37, 265, 5] | \n", 265 | "[0, 389, 758, 391, 390] | \n", 266 | "[0, 389, 758, 391, 395] | \n", 267 | "[0, 389, 904, 955, 391] | \n", 268 | "... | \n", 269 | "True | \n", 270 | "True | \n", 271 | "True | \n", 272 | "True | \n", 273 | "True | \n", 274 | "True | \n", 275 | "True | \n", 276 | "True | \n", 277 | "True | \n", 278 | "True | \n", 279 | "
... | \n", 282 | "... | \n", 283 | "... | \n", 284 | "... | \n", 285 | "... | \n", 286 | "... | \n", 287 | "... | \n", 288 | "... | \n", 289 | "... | \n", 290 | "... | \n", 291 | "... | \n", 292 | "... | \n", 293 | "... | \n", 294 | "... | \n", 295 | "... | \n", 296 | "... | \n", 297 | "... | \n", 298 | "... | \n", 299 | "... | \n", 300 | "... | \n", 301 | "... | \n", 302 | "... | \n", 303 | "
49995 | \n", 306 | "49995 | \n", 307 | "999 | \n", 308 | "[896, 799, 435, 876, 706] | \n", 309 | "[799, 905, 876, 435, 831] | \n", 310 | "[896, 905, 904, 435, 799] | \n", 311 | "[896, 876, 435, 804, 861] | \n", 312 | "[876, 435, 579, 532, 526] | \n", 313 | "[799, 896, 435, 876, 904] | \n", 314 | "[999, 700, 904, 896, 861] | \n", 315 | "[896, 700, 435, 876, 904] | \n", 316 | "... | \n", 317 | "False | \n", 318 | "False | \n", 319 | "False | \n", 320 | "False | \n", 321 | "False | \n", 322 | "False | \n", 323 | "True | \n", 324 | "False | \n", 325 | "False | \n", 326 | "True | \n", 327 | "
49996 | \n", 330 | "49996 | \n", 331 | "999 | \n", 332 | "[999, 700, 868, 861, 435] | \n", 333 | "[725, 504, 876, 666, 896] | \n", 334 | "[999, 700, 876, 435, 868] | \n", 335 | "[700, 999, 599, 876, 934] | \n", 336 | "[700, 999, 896, 435, 731] | \n", 337 | "[999, 700, 861, 896, 868] | \n", 338 | "[999, 876, 896, 435, 804] | \n", 339 | "[999, 700, 868, 876, 965] | \n", 340 | "... | \n", 341 | "True | \n", 342 | "False | \n", 343 | "True | \n", 344 | "True | \n", 345 | "True | \n", 346 | "True | \n", 347 | "True | \n", 348 | "True | \n", 349 | "True | \n", 350 | "True | \n", 351 | "
49997 | \n", 354 | "49997 | \n", 355 | "999 | \n", 356 | "[762, 532, 495, 896, 857] | \n", 357 | "[470, 857, 532, 854, 406] | \n", 358 | "[854, 406, 851, 492, 762] | \n", 359 | "[521, 505, 968, 899, 859] | \n", 360 | "[857, 854, 406, 498, 762] | \n", 361 | "[406, 745, 762, 470, 896] | \n", 362 | "[495, 532, 725, 505, 968] | \n", 363 | "[406, 982, 607, 470, 846] | \n", 364 | "... | \n", 365 | "False | \n", 366 | "False | \n", 367 | "False | \n", 368 | "False | \n", 369 | "False | \n", 370 | "False | \n", 371 | "False | \n", 372 | "False | \n", 373 | "False | \n", 374 | "True | \n", 375 | "
49998 | \n", 378 | "49998 | \n", 379 | "999 | \n", 380 | "[333, 937, 700, 330, 999] | \n", 381 | "[333, 937, 356, 359, 334] | \n", 382 | "[333, 700, 937, 338, 356] | \n", 383 | "[333, 700, 999, 937, 794] | \n", 384 | "[333, 937, 700, 338, 999] | \n", 385 | "[333, 937, 987, 338, 357] | \n", 386 | "[333, 937, 961, 700, 338] | \n", 387 | "[333, 700, 999, 728, 937] | \n", 388 | "... | \n", 389 | "True | \n", 390 | "False | \n", 391 | "False | \n", 392 | "True | \n", 393 | "True | \n", 394 | "False | \n", 395 | "False | \n", 396 | "True | \n", 397 | "False | \n", 398 | "False | \n", 399 | "
49999 | \n", 402 | "49999 | \n", 403 | "999 | \n", 404 | "[999, 700, 772, 728, 845] | \n", 405 | "[474, 850, 911, 658, 700] | \n", 406 | "[999, 700, 543, 605, 772] | \n", 407 | "[700, 999, 605, 728, 455] | \n", 408 | "[605, 700, 999, 761, 633] | \n", 409 | "[714, 543, 999, 605, 805] | \n", 410 | "[999, 700, 419, 605, 574] | \n", 411 | "[999, 700, 728, 543, 723] | \n", 412 | "... | \n", 413 | "True | \n", 414 | "False | \n", 415 | "True | \n", 416 | "True | \n", 417 | "True | \n", 418 | "True | \n", 419 | "True | \n", 420 | "True | \n", 421 | "True | \n", 422 | "True | \n", 423 | "
50000 rows × 32 columns
\n", 427 | "\n", 612 | " | Unnamed: 0 | \n", 613 | "query | \n", 614 | "selected | \n", 615 | "total | \n", 616 | "image_number | \n", 617 | "imagenet_label | \n", 618 | "num_queries | \n", 619 | "sel_dict | \n", 620 | "
---|---|---|---|---|---|---|---|---|
image | \n", 623 | "\n", 624 | " | \n", 625 | " | \n", 626 | " | \n", 627 | " | \n", 628 | " | \n", 629 | " | \n", 630 | " | \n", 631 | " |
ILSVRC2012_val_00047683.JPEG | \n", 636 | "90113 | \n", 637 | "[392, 310, 322, 712, 321, 325, 946, 309, 326, ... | \n", 638 | "[0, 0, 4, 0, 4, 7, 1, 0, 6, 1, 4, 0] | \n", 639 | "[9, 9, 9, 7, 5, 8, 9, 9, 7, 9, 9, 9] | \n", 640 | "16296 | \n", 641 | "325 | \n", 642 | "12 | \n", 643 | "{392: (0, 9), 310: (0, 9), 322: (4, 9), 712: (... | \n", 644 | "
ILSVRC2012_val_00030990.JPEG | \n", 647 | "46639 | \n", 648 | "[312, 30, 39, 31, 149, 46, 40, 47, 42, 55, 401] | \n", 649 | "[0, 0, 8, 0, 1, 3, 4, 7, 2, 1, 0] | \n", 650 | "[7, 9, 8, 8, 8, 7, 8, 8, 4, 9, 8] | \n", 651 | "2382 | \n", 652 | "47 | \n", 653 | "11 | \n", 654 | "{312: (0, 7), 30: (0, 9), 39: (8, 8), 31: (0, ... | \n", 655 | "
ILSVRC2012_val_00002431.JPEG | \n", 658 | "67588 | \n", 659 | "[870, 444, 670, 826, 778, 685, 63, 880, 597, 4... | \n", 660 | "[1, 0, 0, 0, 1, 0, 0, 2, 0, 0, 1, 7, 1, 0, 0, ... | \n", 661 | "[9, 9, 8, 9, 8, 9, 7, 8, 9, 8, 9, 9, 7, 9, 9, ... | \n", 662 | "35201 | \n", 663 | "704 | \n", 664 | "17 | \n", 665 | "{870: (1, 9), 444: (0, 9), 670: (0, 8), 826: (... | \n", 666 | "
ILSVRC2012_val_00033061.JPEG | \n", 669 | "125421 | \n", 670 | "[497, 442, 448, 494, 884, 698, 406, 425, 663, ... | \n", 671 | "[6, 8, 0, 4, 0, 0, 1, 1, 4, 1, 1] | \n", 672 | "[6, 8, 9, 9, 8, 7, 8, 9, 6, 8, 9] | \n", 673 | "22133 | \n", 674 | "442 | \n", 675 | "11 | \n", 676 | "{497: (6, 6), 442: (8, 8), 448: (0, 9), 494: (... | \n", 677 | "
ILSVRC2012_val_00002735.JPEG | \n", 680 | "126066 | \n", 681 | "[497, 442, 884, 698, 399, 668, 832, 663, 538] | \n", 682 | "[6, 3, 4, 4, 0, 6, 6, 5, 8] | \n", 683 | "[8, 8, 8, 9, 8, 7, 8, 8, 9] | \n", 684 | "33403 | \n", 685 | "668 | \n", 686 | "9 | \n", 687 | "{497: (6, 8), 442: (3, 8), 884: (4, 8), 698: (... | \n", 688 | "
... | \n", 691 | "... | \n", 692 | "... | \n", 693 | "... | \n", 694 | "... | \n", 695 | "... | \n", 696 | "... | \n", 697 | "... | \n", 698 | "... | \n", 699 | "
ILSVRC2012_val_00003714.JPEG | \n", 702 | "143137 | \n", 703 | "[214, 177, 170, 231, 236, 264, 224, 205, 240, ... | \n", 704 | "[3, 1, 2, 1, 2, 1, 1, 2, 4, 1, 1, 1, 1, 1, 0, ... | \n", 705 | "[9, 6, 8, 9, 8, 6, 5, 8, 8, 8, 8, 7, 9, 8, 8, ... | \n", 706 | "10702 | \n", 707 | "214 | \n", 708 | "19 | \n", 709 | "{214: (3, 9), 177: (1, 6), 170: (2, 8), 231: (... | \n", 710 | "
ILSVRC2012_val_00018015.JPEG | \n", 713 | "54180 | \n", 714 | "[843, 671, 892, 871, 814, 826, 778, 915, 880, ... | \n", 715 | "[0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, ... | \n", 716 | "[9, 9, 9, 8, 9, 9, 5, 8, 9, 8, 7, 5, 9, 6, 9, ... | \n", 717 | "33564 | \n", 718 | "671 | \n", 719 | "23 | \n", 720 | "{843: (0, 9), 671: (2, 9), 892: (1, 9), 871: (... | \n", 721 | "
ILSVRC2012_val_00029538.JPEG | \n", 724 | "129790 | \n", 725 | "[610, 714, 566, 745, 771, 515, 592, 650, 420, ... | \n", 726 | "[0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 8, 0, 0] | \n", 727 | "[9, 9, 9, 8, 6, 9, 9, 8, 9, 8, 8, 8, 9] | \n", 728 | "20129 | \n", 729 | "402 | \n", 730 | "13 | \n", 731 | "{610: (0, 9), 714: (0, 9), 566: (0, 9), 745: (... | \n", 732 | "
ILSVRC2012_val_00040262.JPEG | \n", 735 | "96869 | \n", 736 | "[794, 440, 868, 585, 709, 611, 118, 917, 857, ... | \n", 737 | "[0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 1, 0, 6, 0, 0, ... | \n", 738 | "[9, 9, 8, 7, 8, 9, 8, 9, 9, 7, 8, 8, 8, 8, 8, ... | \n", 739 | "46093 | \n", 740 | "921 | \n", 741 | "19 | \n", 742 | "{794: (0, 9), 440: (0, 9), 868: (0, 8), 585: (... | \n", 743 | "
ILSVRC2012_val_00006124.JPEG | \n", 746 | "20850 | \n", 747 | "[758, 726, 589, 618, 606, 666, 910, 828, 813, ... | \n", 748 | "[1, 1, 0, 2, 0, 0, 2, 5, 0, 0, 0, 2, 4, 0, 0, ... | \n", 749 | "[8, 8, 8, 9, 9, 8, 9, 9, 9, 7, 8, 8, 9, 9, 9, ... | \n", 750 | "36454 | \n", 751 | "729 | \n", 752 | "18 | \n", 753 | "{758: (1, 8), 726: (1, 8), 589: (0, 8), 618: (... | \n", 754 | "
10000 rows × 8 columns
\n", 758 | "\n", 913 | " | annotation | \n", 914 | "
---|---|
image | \n", 917 | "\n", 918 | " |
ILSVRC2012_val_00047683.JPEG | \n", 923 | "fu | \n", 924 | "
ILSVRC2012_val_00030990.JPEG | \n", 927 | "fu | \n", 928 | "
ILSVRC2012_val_00002431.JPEG | \n", 931 | "easy | \n", 932 | "
ILSVRC2012_val_00033061.JPEG | \n", 935 | "fu | \n", 936 | "
ILSVRC2012_val_00002735.JPEG | \n", 939 | "fu | \n", 940 | "
... | \n", 943 | "... | \n", 944 | "
ILSVRC2012_val_00003714.JPEG | \n", 947 | "fu | \n", 948 | "
ILSVRC2012_val_00018015.JPEG | \n", 951 | "fu | \n", 952 | "
ILSVRC2012_val_00029538.JPEG | \n", 955 | "easy | \n", 956 | "
ILSVRC2012_val_00040262.JPEG | \n", 959 | "easy | \n", 960 | "
ILSVRC2012_val_00006124.JPEG | \n", 963 | "fu | \n", 964 | "
10000 rows × 1 columns
\n", 968 | "\n", 1073 | " | main | \n", 1074 | "selected | \n", 1075 | "num_workers | \n", 1076 | "image_number | \n", 1077 | "imagenet_label | \n", 1078 | "main_counter | \n", 1079 | "main_top | \n", 1080 | "num_objects | \n", 1081 | "object_counter | \n", 1082 | "objects | \n", 1083 | "main_object | \n", 1084 | "main_object_counter | \n", 1085 | "imagenet_object | \n", 1086 | "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
image | \n", 1089 | "\n", 1090 | " | \n", 1091 | " | \n", 1092 | " | \n", 1093 | " | \n", 1094 | " | \n", 1095 | " | \n", 1096 | " | \n", 1097 | " | \n", 1098 | " | \n", 1099 | " | \n", 1100 | " | \n", 1101 | " | \n", 1102 | " |
ILSVRC2012_val_00047683.JPEG | \n", 1107 | "[325, 322, 322, 325, 322, 322, 325, 324, 322] | \n", 1108 | "[[326, 322, 325], [322], [322], [325], [322], ... | \n", 1109 | "9 | \n", 1110 | "16296 | \n", 1111 | "325 | \n", 1112 | "{325: 3, 322: 5, 324: 1} | \n", 1113 | "322 | \n", 1114 | "1 | \n", 1115 | "{3: 1, 1: 8} | \n", 1116 | "[{326: (0.09090909090909091, 0.111111111111111... | \n", 1117 | "0 | \n", 1118 | "[9] | \n", 1119 | "0.0 | \n", 1120 | "
ILSVRC2012_val_00030990.JPEG | \n", 1123 | "[40, 47, 39, 47, 47, 40, 47, 40, 47] | \n", 1124 | "[[40], [47], [39], [47], [47], [40], [47], [40... | \n", 1125 | "9 | \n", 1126 | "2382 | \n", 1127 | "47 | \n", 1128 | "{40: 3, 47: 5, 39: 1} | \n", 1129 | "47 | \n", 1130 | "1 | \n", 1131 | "{1: 7, 2: 2} | \n", 1132 | "[{40: (0.36363636363636365, 0.4444444444444444... | \n", 1133 | "0 | \n", 1134 | "[9] | \n", 1135 | "0.0 | \n", 1136 | "
ILSVRC2012_val_00033061.JPEG | \n", 1139 | "[497, 663, 497, 442, 497, 497, 497, 497, 497] | \n", 1140 | "[[497, 442], [497, 442, 663], [497, 442], [442... | \n", 1141 | "9 | \n", 1142 | "22133 | \n", 1143 | "442 | \n", 1144 | "{497: 7, 663: 1, 442: 1} | \n", 1145 | "497 | \n", 1146 | "2 | \n", 1147 | "{2: 3, 3: 3, 1: 3} | \n", 1148 | "[{442: (1.0, 0.7777777777777778)}, {497: (0.72... | \n", 1149 | "1 | \n", 1150 | "[1, 8] | \n", 1151 | "0.0 | \n", 1152 | "
ILSVRC2012_val_00002735.JPEG | \n", 1155 | "[497, 832, 663, 668, 832, 668, 538, 668, 668] | \n", 1156 | "[[538, 497], [832, 538, 668, 663, 884, 497], [... | \n", 1157 | "9 | \n", 1158 | "33403 | \n", 1159 | "668 | \n", 1160 | "{497: 1, 832: 2, 663: 1, 668: 4, 538: 1} | \n", 1161 | "668 | \n", 1162 | "2 | \n", 1163 | "{2: 3, 6: 1, 4: 3, 1: 2} | \n", 1164 | "[{884: (0.15384615384615385, 0.222222222222222... | \n", 1165 | "1 | \n", 1166 | "[2, 7] | \n", 1167 | "1.0 | \n", 1168 | "
ILSVRC2012_val_00019033.JPEG | \n", 1171 | "[69, 69, 69, 69, 116, 116, 69, 116, 126] | \n", 1172 | "[[69], [69], [69], [69], [116], [69, 126, 116]... | \n", 1173 | "9 | \n", 1174 | "5818 | \n", 1175 | "116 | \n", 1176 | "{69: 5, 116: 3, 126: 1} | \n", 1177 | "69 | \n", 1178 | "1 | \n", 1179 | "{1: 7, 3: 2} | \n", 1180 | "[{69: (0.46153846153846156, 0.6666666666666666... | \n", 1181 | "0 | \n", 1182 | "[9] | \n", 1183 | "0.0 | \n", 1184 | "
... | \n", 1187 | "... | \n", 1188 | "... | \n", 1189 | "... | \n", 1190 | "... | \n", 1191 | "... | \n", 1192 | "... | \n", 1193 | "... | \n", 1194 | "... | \n", 1195 | "... | \n", 1196 | "... | \n", 1197 | "... | \n", 1198 | "... | \n", 1199 | "... | \n", 1200 | "
ILSVRC2012_val_00025813.JPEG | \n", 1203 | "[947, 947, 947, 947, 947, 947, 947, 947, 947] | \n", 1204 | "[[997, 947], [997, 947], [947], [947], [997, 9... | \n", 1205 | "9 | \n", 1206 | "47377 | \n", 1207 | "947 | \n", 1208 | "{947: 9} | \n", 1209 | "947 | \n", 1210 | "1 | \n", 1211 | "{2: 4, 1: 5} | \n", 1212 | "[{997: (0.3076923076923077, 0.4444444444444444... | \n", 1213 | "0 | \n", 1214 | "[9] | \n", 1215 | "0.0 | \n", 1216 | "
ILSVRC2012_val_00040269.JPEG | \n", 1219 | "[382, 381, 381, 382, 382, 382, 382, 382, 382] | \n", 1220 | "[[382], [381], [381], [382], [382], [382], [38... | \n", 1221 | "9 | \n", 1222 | "19041 | \n", 1223 | "380 | \n", 1224 | "{382: 7, 381: 2} | \n", 1225 | "382 | \n", 1226 | "1 | \n", 1227 | "{1: 8, 2: 1} | \n", 1228 | "[{382: (0.7, 0.7777777777777778), 381: (0.3, 0... | \n", 1229 | "0 | \n", 1230 | "[9] | \n", 1231 | "NaN | \n", 1232 | "
ILSVRC2012_val_00003714.JPEG | \n", 1235 | "[214, 256, 240, 214, 256, 214, 205, 214, 214] | \n", 1236 | "[[214, 240, 256, 236], [256], [240], [214, 256... | \n", 1237 | "9 | \n", 1238 | "10702 | \n", 1239 | "214 | \n", 1240 | "{214: 5, 256: 2, 240: 1, 205: 1} | \n", 1241 | "214 | \n", 1242 | "1 | \n", 1243 | "{4: 1, 1: 7, 2: 1} | \n", 1244 | "[{214: (0.38461538461538464, 0.555555555555555... | \n", 1245 | "0 | \n", 1246 | "[9] | \n", 1247 | "0.0 | \n", 1248 | "
ILSVRC2012_val_00018015.JPEG | \n", 1251 | "[671, 671, 671, 671, 671, 671, 671, 671, 671] | \n", 1252 | "[[535, 671], [671], [671], [671], [671], [671]... | \n", 1253 | "9 | \n", 1254 | "33564 | \n", 1255 | "671 | \n", 1256 | "{671: 9} | \n", 1257 | "671 | \n", 1258 | "1 | \n", 1259 | "{2: 1, 1: 8} | \n", 1260 | "[{535: (0.1, 0.1111111111111111), 671: (0.9, 1... | \n", 1261 | "0 | \n", 1262 | "[9] | \n", 1263 | "0.0 | \n", 1264 | "
ILSVRC2012_val_00006124.JPEG | \n", 1267 | "[729, 729, 729, 729, 729, 729, 729, 729, 729] | \n", 1268 | "[[828, 729, 567], [828, 729], [828, 729], [828... | \n", 1269 | "9 | \n", 1270 | "36454 | \n", 1271 | "729 | \n", 1272 | "{729: 9} | \n", 1273 | "729 | \n", 1274 | "2 | \n", 1275 | "{3: 2, 2: 6, 1: 1} | \n", 1276 | "[{828: (1.0, 0.8888888888888888)}, {729: (0.81... | \n", 1277 | "1 | \n", 1278 | "[0, 9] | \n", 1279 | "1.0 | \n", 1280 | "
6761 rows × 13 columns
\n", 1284 | "