├── Annotations.zip ├── README.md ├── code ├── .ipynb_checkpoints │ ├── analysis-checkpoint.ipynb │ ├── analysis2-checkpoint.ipynb │ ├── easy_hard_imgs-checkpoint.ipynb │ ├── file_check-checkpoint.ipynb │ └── plots-checkpoint.ipynb ├── Classifier-COCO.ipynb ├── Classifier-COCO.py ├── Classifier-New.ipynb ├── Classifier.ipynb ├── Classifier.py ├── SummaryBox.txt ├── SummaryCrop.txt ├── SummaryFull.txt ├── analysis-attack.ipynb ├── analysis-robustness_rob.py ├── analysis-robustness_rob_seg.py ├── analysis.ipynb ├── analysis2.ipynb ├── analysis_Masks.ipynb ├── easy_hard_imgs.ipynb ├── file_check.ipynb ├── imagenet_classes.txt ├── make_imagenet_64_c.py ├── outputs │ ├── AlexNet-Crop.txt │ ├── AlexNet-Full.txt │ ├── AlexNet-box.txt │ ├── GoogLeNet-Crop.txt │ ├── GoogLeNet-Full.txt │ ├── GoogLeNet-box.txt │ ├── Inception3-Crop.txt │ ├── Inception3-Full.txt │ ├── Inception3-box.txt │ ├── MNASNet-Crop.txt │ ├── MNASNet-Full.txt │ ├── MNASNet-box.txt │ ├── ResNet-Crop.txt │ ├── ResNet-Full.txt │ ├── ResNet-box.txt │ ├── VGG-Crop.txt │ ├── VGG-Full.txt │ └── VGG-box.txt ├── overlapped_classes.py ├── plots.ipynb ├── read_rob_txt_files.ipynb └── test_noise_robustness.py └── mappings ├── folder_to_objectnet_label.json ├── imagenet_to_label_2012_v2.txt ├── objectnet_to_imagenet_1k.json └── pytorch_to_imagenet_2012_id.json /Annotations.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aliborji/ObjectNetReanalysis/b27a3efe2a399a21aedcf4c86f3b6666a50fe880/Annotations.zip -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ObjectNet Reanalysis and Annotations 2 | ===================== 3 | 4 | April 1, 2020 5 | 6 | 7 | **News**: Manuscript is avaiable in [Arxiv](https://arxiv.org/pdf/2004.02042.pdf). 8 | 9 | 10 | 11 | 12 | * The code works with **PyTorch 1.0.0 or higher. 13 | 14 | 15 | 16 | 17 | 18 | 19 | This repository provides bounding box annotations of the ObjectNet dataset 20 | by Barbu et al. NIPS 2019. 21 | 22 | 23 | 24 | About the ObjectNet dataset: 25 | Taken from the ObjectNet repository README.md, verbatim: 26 | "ObjectNet is an object recognition dataset developed at MIT in CSAIL & 27 | BCS as well as CBMM (Center for Brains, Minds, and Machines) and at 28 | the MIT-IBM Watson AI Lab. It intentionally adds variation to the 29 | background, viewpoint, and rotation of objects during the image 30 | capturing process while minimizing correlations between these 31 | properties and object class. Removing these biases challenges current 32 | object detectors. We hope that this will lead to new detectors and new 33 | ideas in machine learning as well a more rigorous and scientific 34 | approaches to datasets everywhere. 35 | 36 | ObjectNet has no training set! The ObjectNet license, attached, which 37 | you agree to in order to use this work, EXPLICITLY FORBIDS UPDATING 38 | THE PARAMETERS OF ANY MODEL BASED ON THESE IMAGES. Our goal is for 39 | ObjectNet and datasets like it to produce accuracies that are 40 | predictive of what any user, including you, will see in the real world 41 | when deploying object detectors." 42 | 43 | 44 | 45 | ** passsword for the .zip file: objectnet1234 46 | 47 | 48 | If you use ObjectNet Annotations or found the ideas in the paper useful, please 49 | cite: 50 | 51 | ObjectNet Dataset: Reanalysis and Correction 52 | Ali Borji 53 | 54 | Here is the .bib file: 55 | 56 | ``` 57 | @misc{borji2020objectnet, 58 | title={ObjectNet Dataset: Reanalysis and Correction}, 59 | author={Ali Borji}, 60 | year={2020}, 61 | eprint={2004.02042}, 62 | archivePrefix={arXiv}, 63 | primaryClass={cs.CV} 64 | } 65 | 66 | 67 | @misc{borji2021Contemplating, 68 | title={Contemplating real-world object classification}, 69 | author={Ali Borji}, 70 | year={2021}, 71 | jounral={ICLR}, 72 | primaryClass={cs.CV} 73 | } 74 | 75 | 76 | ``` 77 | 78 | 79 | 80 | Please write to aliborji@gmail.com for further questiones, comments, etc. 81 | 82 | -------------------------------------------------------------------------------- /code/.ipynb_checkpoints/analysis-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [], 3 | "metadata": {}, 4 | "nbformat": 4, 5 | "nbformat_minor": 4 6 | } 7 | -------------------------------------------------------------------------------- /code/.ipynb_checkpoints/analysis2-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 3, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import torch\n", 10 | "import torchvision\n", 11 | "import torchvision.transforms as transforms\n", 12 | "import numpy as np\n", 13 | "import cv2\n", 14 | "from matplotlib import pyplot as plt\n", 15 | "import os\n", 16 | "\n", 17 | "from overlapped_classes import overlapped_classes\n", 18 | "import json\n", 19 | "from PIL import Image\n", 20 | "import scipy" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 65, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "with open(\"../mappings/objectnet_to_imagenet_1k.json\") as f:\n", 30 | " data = json.load(f)\n", 31 | "\n", 32 | "with open(\"../mappings/pytorch_to_imagenet_2012_id.json\") as f:\n", 33 | " idxMap = json.load(f)\n", 34 | " \n", 35 | "with open(\"../mappings/folder_to_objectnet_label.json\") as f:\n", 36 | " folderMap = json.load(f)\n", 37 | "\n", 38 | "with open('imagenet_classes.txt') as f:\n", 39 | "# with open('../mappings/imagenet_to_label_2012_v2.txt') as f: \n", 40 | " labels = [line.strip() for line in f.readlines()] " 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 34, 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "# inverting the folderMap\n", 50 | "dirMap = {}\n", 51 | "for u, v in folderMap.items():\n", 52 | " dirMap[v]= u" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 36, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "# dirMap" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "# help(json.load)\n", 71 | "# for i in overlapped_classes:\n", 72 | "# if not data[i] in labels:\n", 73 | "# print(f\"{i} : {data[i]}, {data[i] in labels}\")" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "# overlapped_classes\n", 83 | "# data['Alarm clock']\n", 84 | "# idxMap" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "metadata": {}, 91 | "outputs": [], 92 | "source": [ 93 | "# for i in overlapped_classes:\n", 94 | "# print(data[i] in labels)\n", 95 | "# print(f\"{i} : {data[i]}\")\n", 96 | "del model" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": 62, 102 | "metadata": {}, 103 | "outputs": [ 104 | { 105 | "data": { 106 | "text/plain": [ 107 | "AlexNet(\n", 108 | " (features): Sequential(\n", 109 | " (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))\n", 110 | " (1): ReLU(inplace=True)\n", 111 | " (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 112 | " (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n", 113 | " (4): ReLU(inplace=True)\n", 114 | " (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 115 | " (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 116 | " (7): ReLU(inplace=True)\n", 117 | " (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 118 | " (9): ReLU(inplace=True)\n", 119 | " (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 120 | " (11): ReLU(inplace=True)\n", 121 | " (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 122 | " )\n", 123 | " (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))\n", 124 | " (classifier): Sequential(\n", 125 | " (0): Dropout(p=0.5, inplace=False)\n", 126 | " (1): Linear(in_features=9216, out_features=4096, bias=True)\n", 127 | " (2): ReLU(inplace=True)\n", 128 | " (3): Dropout(p=0.5, inplace=False)\n", 129 | " (4): Linear(in_features=4096, out_features=4096, bias=True)\n", 130 | " (5): ReLU(inplace=True)\n", 131 | " (6): Linear(in_features=4096, out_features=1000, bias=True)\n", 132 | " )\n", 133 | ")" 134 | ] 135 | }, 136 | "execution_count": 62, 137 | "metadata": {}, 138 | "output_type": "execute_result" 139 | } 140 | ], 141 | "source": [ 142 | "from torchvision import models\n", 143 | "dir(models)\n", 144 | "\n", 145 | "\n", 146 | "\n", 147 | "model = models.alexnet(pretrained=True)\n", 148 | "# model = models.resnet152(pretrained=True)\n", 149 | "# model = models.inception_v3(pretrained=True)\n", 150 | "# model = models.googlenet(pretrained=True)\n", 151 | "\n", 152 | "model.eval()\n" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": 59, 158 | "metadata": {}, 159 | "outputs": [], 160 | "source": [ 161 | "def predict_image(image_path, box, draw=False):\n", 162 | " # Define transformations for the image, should (note that imagenet models are trained with image size 224)\n", 163 | " transform = transforms.Compose([\n", 164 | "# transforms.CenterCrop(224),\n", 165 | " transforms.ToTensor(),\n", 166 | "# transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n", 167 | " transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) \n", 168 | "\n", 169 | " ])\n", 170 | " \n", 171 | " \n", 172 | "# transform = transforms.Compose([ #[1]\n", 173 | "# # transforms.Resize(256), #[2]\n", 174 | "# # transforms.CenterCrop(224), #[3]\n", 175 | "# transforms.ToTensor(), #[4]\n", 176 | "# # transforms.Normalize( #[5]\n", 177 | "# # mean=[0.485, 0.456, 0.406], #[6]\n", 178 | "# # std=[0.229, 0.224, 0.225] #[7]\n", 179 | "# # )\n", 180 | "# transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) \n", 181 | "# ])\n", 182 | " \n", 183 | "# print(\"Prediction in progress\")\n", 184 | " img = Image.open(image_path).convert(\"RGB\")\n", 185 | "\n", 186 | " \n", 187 | " xl, yl, xr, yr = box\n", 188 | " img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) \n", 189 | " img = img.resize((224, 224))\n", 190 | "\n", 191 | " img_t = transform(img).float()\n", 192 | "\n", 193 | "# img_t = img_t.permute((0,1,2))\n", 194 | "# img_t = torch.transpose(img_t, 0, 1) # rotate\n", 195 | " \n", 196 | " if draw:\n", 197 | " plt.imshow(img) #img_t.permute((2,1,0)) )\n", 198 | " # ## plt.imshow(torch.transpose(img_t.permute((1,2,0)), 0, 1))\n", 199 | " plt.show()\n", 200 | "\n", 201 | "# print(img_t.shape)\n", 202 | " # Preprocess the image\n", 203 | "# image_tensor = transformation(image).float()\n", 204 | "\n", 205 | " # Add an extra batch dimension since pytorch treats all images as batches\n", 206 | "# image_tensor = image_tensor.unsqueeze_(0)\n", 207 | " image_tensor = img_t.unsqueeze_(0)\n", 208 | "\n", 209 | " if torch.cuda.is_available():\n", 210 | " image_tensor.cuda()\n", 211 | "\n", 212 | " # Turn the input into a Variable\n", 213 | "# input = Variable(image_tensor)\n", 214 | "\n", 215 | " # Predict the class of the image\n", 216 | " output = model(image_tensor)\n", 217 | "\n", 218 | "# index = output.data.numpy().argmax() # top 1\n", 219 | " _, indices = torch.sort(output.data, descending=True)\n", 220 | "# [(labels[idx], percentage[idx].item()) for idx in indices[0][:5]]\n", 221 | " return indices[0][:5] # index" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": 41, 227 | "metadata": {}, 228 | "outputs": [ 229 | { 230 | "data": { 231 | "text/plain": [ 232 | "'alarm_clock'" 233 | ] 234 | }, 235 | "execution_count": 41, 236 | "metadata": {}, 237 | "output_type": "execute_result" 238 | } 239 | ], 240 | "source": [ 241 | "len(data)\n", 242 | "dirMap[category]\n" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": 63, 248 | "metadata": {}, 249 | "outputs": [ 250 | { 251 | "name": "stdout", 252 | "output_type": "stream", 253 | "text": [ 254 | "0 -> Alarm clock: top 1: 6.25 - top 5: 20.13888888888889 [num imgs: 144]\n", 255 | "1 -> Backpack: top 1: 27.659574468085108 - top 5: 53.61702127659574 [num imgs: 235]\n", 256 | "2 -> Banana: top 1: 44.255319148936174 - top 5: 62.5531914893617 [num imgs: 235]\n", 257 | "3 -> Band Aid: top 1: 20.081967213114755 - top 5: 37.704918032786885 [num imgs: 244]\n", 258 | "4 -> Basket: top 1: 5.524861878453039 - top 5: 23.756906077348066 [num imgs: 181]\n", 259 | "5 -> Bath towel: top 1: 10.06711409395973 - top 5: 26.845637583892618 [num imgs: 149]\n", 260 | "6 -> Beer bottle: top 1: 8.67579908675799 - top 5: 17.80821917808219 [num imgs: 219]\n", 261 | "7 -> Bench: top 1: 0.0 - top 5: 3.0303030303030303 [num imgs: 132]\n", 262 | "8 -> Bicycle: top 1: 20.408163265306122 - top 5: 40.816326530612244 [num imgs: 147]\n", 263 | "9 -> Binder (closed): top 1: 8.241758241758241 - top 5: 25.274725274725274 [num imgs: 182]\n", 264 | "10 -> Bottle cap: top 1: 8.870967741935484 - top 5: 19.35483870967742 [num imgs: 248]\n", 265 | "11 -> Bread loaf: top 1: 2.262443438914027 - top 5: 6.334841628959276 [num imgs: 221]\n", 266 | "12 -> Broom: top 1: 14.655172413793103 - top 5: 35.775862068965516 [num imgs: 232]\n", 267 | "13 -> Bucket: top 1: 10.18867924528302 - top 5: 25.660377358490567 [num imgs: 265]\n", 268 | "14 -> Butcher's knife: top 1: 8.717948717948717 - top 5: 24.615384615384617 [num imgs: 195]\n", 269 | "15 -> Can opener: top 1: 9.313725490196079 - top 5: 27.941176470588236 [num imgs: 204]\n", 270 | "16 -> Candle: top 1: 1.7167381974248928 - top 5: 6.866952789699571 [num imgs: 233]\n", 271 | "18 -> Chair: top 1: 18.636363636363637 - top 5: 41.81818181818182 [num imgs: 220]\n", 272 | "20 -> Coffee/French press: top 1: 2.803738317757009 - top 5: 14.018691588785046 [num imgs: 107]\n", 273 | "21 -> Combination lock: top 1: 24.305555555555557 - top 5: 37.5 [num imgs: 144]\n", 274 | "22 -> Computer mouse: top 1: 29.017857142857142 - top 5: 49.107142857142854 [num imgs: 224]\n", 275 | "23 -> Desk lamp: top 1: 3.5 - top 5: 15.0 [num imgs: 200]\n", 276 | "24 -> Dishrag or hand towel: top 1: 0.0 - top 5: 33.333333333333336 [num imgs: 3]\n", 277 | "68 -> Printer: top 1: 22.4 - top 5: 49.6 [num imgs: 125]\n", 278 | "69 -> Remote control: top 1: 33.333333333333336 - top 5: 49.18032786885246 [num imgs: 183]\n", 279 | "70 -> Ruler: top 1: 23.026315789473685 - top 5: 40.78947368421053 [num imgs: 152]\n" 280 | ] 281 | } 282 | ], 283 | "source": [ 284 | "# category = 'Banana'\n", 285 | "\n", 286 | "res_top1 = []\n", 287 | "res_top5 = []\n", 288 | "\n", 289 | "for n, category in enumerate(data):\n", 290 | "# if n > 1:\n", 291 | "# break\n", 292 | " txtfile = '../' + dirMap[category] + '.txt'\n", 293 | " if not os.path.exists(txtfile):\n", 294 | " continue\n", 295 | " \n", 296 | " with open(txtfile) as f:\n", 297 | " boxes = [line.strip() for line in f.readlines()] \n", 298 | "\n", 299 | "# preds = []\n", 300 | " count = [0, 0]\n", 301 | " lines = 0\n", 302 | " for im in boxes[0:]:\n", 303 | " ss = im.split(' ')\n", 304 | " if len(ss) > 1:\n", 305 | " lines += 1\n", 306 | " coords = (int(i) for i in ss[1:] if i)\n", 307 | " idx = predict_image(os.path.join('../images/' + dirMap[category] + '/', ss[0]), coords, False)\n", 308 | " \n", 309 | " # top 1\n", 310 | " count[0] += 1 if labels[idx[0]] in data[category] else 0 \n", 311 | " \n", 312 | " # top 5\n", 313 | " flag = False\n", 314 | " for i in idx[0:]:\n", 315 | " # print(labels[i], end = ' ')\n", 316 | "# predLabels = labels[idxMap[str(i.tolist())]]\n", 317 | "# print(i, labels[i])\n", 318 | " flag = flag or (labels[i] in data[category])\n", 319 | " count[1] += 1 if flag else 0\n", 320 | "# print('\\n')\n", 321 | "# preds.append(idx)\n", 322 | "\n", 323 | "\n", 324 | "\n", 325 | " accs = np.array(count)*100/lines\n", 326 | " \n", 327 | " print(f\"{n} -> {category}: top 1: {accs[0]} - top 5: {accs[1]} [num imgs: {lines}]\") \n", 328 | "\n", 329 | " res_top1.append(accs[0]) \n", 330 | " res_top5.append(accs[1])\n" 331 | ] 332 | }, 333 | { 334 | "cell_type": "code", 335 | "execution_count": 64, 336 | "metadata": {}, 337 | "outputs": [ 338 | { 339 | "name": "stdout", 340 | "output_type": "stream", 341 | "text": [ 342 | "13.996669092969494\n", 343 | "30.32472008287954\n" 344 | ] 345 | } 346 | ], 347 | "source": [ 348 | "print(sum(res_top1)/len(res_top1))\n", 349 | "print(sum(res_top5)/len(res_top5))\n" 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "# model = alexnet\n", 359 | "# for l in preds:\n", 360 | "# print(labels[l])\n", 361 | "# preds \n" 362 | ] 363 | }, 364 | { 365 | "cell_type": "code", 366 | "execution_count": null, 367 | "metadata": {}, 368 | "outputs": [], 369 | "source": [ 370 | "# \n", 371 | "if ss is ['']:\n", 372 | " print('ss')\n", 373 | "len(ss)" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": null, 379 | "metadata": {}, 380 | "outputs": [], 381 | "source": [ 382 | "# Import Pillow\n", 383 | "\n", 384 | "from PIL import Image\n", 385 | "img = Image.open(\"../images/air_freshener/01d44a4b77b44a0.png\").convert('RGB')\n", 386 | "\n", 387 | "img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) \n", 388 | "img = img.resize((224, 224))\n", 389 | "\n", 390 | "\n", 391 | "img_t = transform(img)#.float()\n", 392 | "\n", 393 | "img_t = img_t.permute((2,1,0))\n", 394 | "\n", 395 | "img_t = torch.transpose(img_t, 0, 1)\n", 396 | "\n", 397 | "# np.clip\n", 398 | "# img_t = torch.unsqueeze(img_t, 0)\n", 399 | "# help(img_t.transpose)\n", 400 | "\n", 401 | "plt.imshow(img_t)\n", 402 | "\n", 403 | "# img = cv2.imread(\"../images/air_freshener/01d44a4b77b44a0.png\", cv2.IMREAD_UNCHANGED)\n", 404 | "# img = cv2cvLoadImage(\"../images/air_freshener/01d44a4b77b44a0.png\", CV_LOAD_IMAGE_UNCHANGED)\n" 405 | ] 406 | }, 407 | { 408 | "cell_type": "code", 409 | "execution_count": null, 410 | "metadata": {}, 411 | "outputs": [], 412 | "source": [ 413 | "np.asarray(img_t).shape" 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": null, 419 | "metadata": {}, 420 | "outputs": [], 421 | "source": [ 422 | "img_t[:10,:10].min()" 423 | ] 424 | }, 425 | { 426 | "cell_type": "code", 427 | "execution_count": null, 428 | "metadata": {}, 429 | "outputs": [], 430 | "source": [ 431 | "help(torch.transpose)" 432 | ] 433 | }, 434 | { 435 | "cell_type": "code", 436 | "execution_count": null, 437 | "metadata": {}, 438 | "outputs": [], 439 | "source": [ 440 | "# plt.show(img)\n", 441 | "%matplotlib inline\n", 442 | "img = np.array(img)\n", 443 | "# img = img[:,:,::-1]\n", 444 | "\n", 445 | "\n", 446 | "\n", 447 | "\n" 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": null, 453 | "metadata": {}, 454 | "outputs": [], 455 | "source": [ 456 | "with open('../air_freshener.txt') as f:\n", 457 | " boxes = [line.strip() for line in f.readlines()] " 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": null, 463 | "metadata": {}, 464 | "outputs": [], 465 | "source": [ 466 | "xl, yl, xr, yr = [int(i) for i in boxes[11].split(' ')[1:] if i]\n", 467 | "img[yl:yr,xl:xr, :]\n", 468 | "\n", 469 | "# std = np.array([0.229, 0.224, 0.225])\n", 470 | "# inp = std * inp + mean\n", 471 | "# inp = np.clip(inp, 0, 1)\n", 472 | "\n", 473 | "img_t = transform().float()\n", 474 | "batch_t = torch.unsqueeze(img, 0)\n", 475 | "\n", 476 | "\n", 477 | "plt.imshow(batch_t)\n", 478 | "# cv2.imshow('t', img[xl:xr, yl:yr,:])\n", 479 | "# # help(cv2.imshow())\n", 480 | "# cv2.waitKey(0)\n", 481 | "\n", 482 | "# Image.crop()" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": {}, 489 | "outputs": [], 490 | "source": [ 491 | "# cv2.destroyAllWindows()\n" 492 | ] 493 | }, 494 | { 495 | "cell_type": "code", 496 | "execution_count": null, 497 | "metadata": {}, 498 | "outputs": [], 499 | "source": [ 500 | "xl, yl, xr, yr" 501 | ] 502 | }, 503 | { 504 | "cell_type": "code", 505 | "execution_count": null, 506 | "metadata": {}, 507 | "outputs": [], 508 | "source": [ 509 | "batch_t.shape" 510 | ] 511 | }, 512 | { 513 | "cell_type": "code", 514 | "execution_count": null, 515 | "metadata": {}, 516 | "outputs": [], 517 | "source": [ 518 | "# resnet = models.resnet101(pretrained=True)\n", 519 | "\n", 520 | "alexnet.eval()\n", 521 | "img_a = img_t.permute((2,1,0))\n", 522 | "batch_t = torch.unsqueeze(img_a, 0)\n", 523 | "out = alexnet(batch_t)\n", 524 | "\n", 525 | "print(out.shape)" 526 | ] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "execution_count": null, 531 | "metadata": {}, 532 | "outputs": [], 533 | "source": [ 534 | "# out\n", 535 | "batch_t.shape" 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": null, 541 | "metadata": {}, 542 | "outputs": [], 543 | "source": [ 544 | "with open('imagenet_classes.txt') as f:\n", 545 | " labels = [line.strip() for line in f.readlines()]" 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": null, 551 | "metadata": {}, 552 | "outputs": [], 553 | "source": [ 554 | "_, index = torch.max(out, 1)\n", 555 | "percentage = torch.nn.functional.softmax(out, dim=1)[0] * 100\n", 556 | "print(labels[index[0]], percentage[index[0]].item())" 557 | ] 558 | }, 559 | { 560 | "cell_type": "code", 561 | "execution_count": null, 562 | "metadata": {}, 563 | "outputs": [], 564 | "source": [ 565 | "_, indices = torch.sort(out, descending=True)\n", 566 | "2\n", 567 | "[(labels[idx], percentage[idx].item()) for idx in indices[0][:5]]" 568 | ] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "execution_count": null, 573 | "metadata": {}, 574 | "outputs": [], 575 | "source": [ 576 | "index" 577 | ] 578 | }, 579 | { 580 | "cell_type": "code", 581 | "execution_count": null, 582 | "metadata": {}, 583 | "outputs": [], 584 | "source": [] 585 | } 586 | ], 587 | "metadata": { 588 | "kernelspec": { 589 | "display_name": "Python 3", 590 | "language": "python", 591 | "name": "python3" 592 | }, 593 | "language_info": { 594 | "codemirror_mode": { 595 | "name": "ipython", 596 | "version": 3 597 | }, 598 | "file_extension": ".py", 599 | "mimetype": "text/x-python", 600 | "name": "python", 601 | "nbconvert_exporter": "python", 602 | "pygments_lexer": "ipython3", 603 | "version": "3.7.6" 604 | } 605 | }, 606 | "nbformat": 4, 607 | "nbformat_minor": 4 608 | } 609 | -------------------------------------------------------------------------------- /code/.ipynb_checkpoints/file_check-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [], 3 | "metadata": {}, 4 | "nbformat": 4, 5 | "nbformat_minor": 4 6 | } 7 | -------------------------------------------------------------------------------- /code/.ipynb_checkpoints/plots-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [], 3 | "metadata": {}, 4 | "nbformat": 4, 5 | "nbformat_minor": 4 6 | } 7 | -------------------------------------------------------------------------------- /code/Classifier-COCO.py: -------------------------------------------------------------------------------- 1 | 2 | import os 3 | import json 4 | from torchvision import datasets 5 | from torch.utils.data import Dataset, DataLoader 6 | from torchvision import transforms, datasets, models 7 | import torch 8 | from PIL import Image, ImageDraw 9 | from torch.optim import lr_scheduler 10 | import tqdm 11 | import numpy as np 12 | from collections import Counter 13 | import math 14 | # get_ipython().run_line_magic('matplotlib', 'inline') 15 | 16 | 17 | # In[21]: 18 | 19 | 20 | import sys 21 | sys.path.insert(1, './MSCOCO-IC') 22 | import dataloader 23 | 24 | ds_trans = transforms.Compose([ 25 | transforms.Scale((224,224)), 26 | transforms.ToTensor(), 27 | # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 28 | transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) 29 | ]) 30 | 31 | 32 | 33 | def train_model(net, mscoco, criterior, optimizer, num_epochs, save_path, kind): 34 | 35 | # device GPU or CPU? 36 | device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") 37 | print("device: {}".format(device)) 38 | 39 | # move network to train on device 40 | net.to(device) 41 | net.train() 42 | # boost network speed on gpu 43 | torch.backends.cudnn.benchmark = True 44 | 45 | phase = 'train' 46 | 47 | batch_size = 100 48 | num_batches = 100 #int(math.ceil(mscoco.num_train/batch_size)) 49 | # import pdb; pdb.set_trace() 50 | 51 | for epoch in range(num_epochs): 52 | print("Epoch {}/{}".format(epoch+1, num_epochs)) 53 | 54 | epoch_loss = 0.0 55 | epoch_corrects = 0 56 | 57 | 58 | for ibatch in range(num_batches): 59 | bstart = ibatch * batch_size 60 | bend = min(bstart + batch_size, mscoco.num_train) 61 | 62 | 63 | 64 | a, b, y_batch = mscoco.get_batch(bstart, bend, mode='train') 65 | x_batch = a if kind == 'box' else b 66 | 67 | x_batch = x_batch.astype('uint8') 68 | inputs = torch.zeros((x_batch.shape[0], x_batch.shape[3], 224,224)) 69 | 70 | for idx,x in enumerate(x_batch): 71 | x = Image.fromarray(x) 72 | inputs[idx] = ds_trans(x)[None] 73 | 74 | labels = torch.tensor(y_batch) 75 | 76 | inputs, labels = inputs.to(device), labels.to(device) 77 | # inputs = inputs.permute(0,3,1,2) 78 | optimizer.zero_grad() 79 | import pdb; pdb.set_trace() 80 | with torch.set_grad_enabled(phase == 'train'): 81 | outputs = net(inputs) 82 | #labels = labels 83 | loss = criterior(outputs, labels) 84 | _, preds = torch.max(outputs, axis=1) 85 | 86 | loss.backward() 87 | optimizer.step() 88 | 89 | epoch_loss += loss.item() * inputs.shape[0] 90 | epoch_corrects += torch.sum(preds==labels.data) 91 | 92 | 93 | epoch_loss = epoch_loss / (num_batches ) 94 | epoch_accuracy = epoch_corrects.double() / (num_batches * batch_size) 95 | 96 | print("{} Loss: {:.4f}, Acc: {:.4f}".format(phase, epoch_loss, epoch_accuracy)) 97 | 98 | torch.save(net.state_dict(), save_path) 99 | 100 | 101 | 102 | 103 | def test_model_clean(net, mscoco, kind): 104 | correct = 0 105 | total = 0 106 | 107 | # device GPU or CPU? 108 | device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") 109 | print("device: {}".format(device)) 110 | 111 | # move network to train on device 112 | net.to(device) 113 | net.eval() 114 | 115 | 116 | batch_size = 100 117 | num_batches = 100 #int(math.ceil(mscoco.num_val/batch_size)) 118 | 119 | 120 | for ibatch in range(num_batches): 121 | bstart = ibatch * batch_size 122 | bend = min(bstart + batch_size, mscoco.num_val) 123 | 124 | 125 | a, b, y_batch = mscoco.get_batch(bstart, bend, mode='val') 126 | x_batch = a if kind == 'box' else b 127 | 128 | x_batch = x_batch.astype('uint8') 129 | inputs = torch.zeros((x_batch.shape[0], x_batch.shape[3], 224,224)) 130 | 131 | for idx,x in enumerate(x_batch): 132 | x = Image.fromarray(x) 133 | inputs[idx] = ds_trans(x)[None] 134 | 135 | labels = torch.tensor(y_batch) 136 | 137 | 138 | inputs, labels = inputs.to(device), labels.to(device) 139 | 140 | outputs = net(inputs) 141 | _, predicted = torch.max(outputs.data, 1) 142 | 143 | total += labels.size(0) 144 | # correct += (predicted == labels.cuda()).sum() 145 | correct += (predicted == labels).sum() 146 | 147 | acc = float(correct) / total #(num_batches * batch_size) 148 | 149 | return acc#, images 150 | 151 | 152 | 153 | def main(): 154 | 155 | mscoco = dataloader.DataLoader(data_dir = './MSCOCO-IC/') 156 | 157 | import sys 158 | 159 | 160 | 161 | kind = sys.argv[1] 162 | 163 | # train 164 | NUM_CLASSES = 10 165 | resnet = models.resnet18(pretrained=True) 166 | num_ftrs = resnet.fc.in_features 167 | resnet.fc = torch.nn.Linear(num_ftrs, NUM_CLASSES) 168 | criterion = torch.nn.CrossEntropyLoss() 169 | optimizer = torch.optim.SGD(resnet.parameters(), lr=0.001, momentum=0.9) 170 | exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1) 171 | 172 | #resnet.load_state_dict(torch.load('./coco_'+kind+'.pth')) 173 | train_model(resnet, mscoco, criterion, optimizer, 20, './coco_'+kind+'.pth', kind) 174 | 175 | 176 | # test 177 | # resnet.load_state_dict(torch.load('./coco_'+kind+'.pth')) 178 | # acc = test_model_clean(resnet, mscoco, kind) 179 | # print(f'accuracy is {acc}') 180 | 181 | 182 | 183 | if __name__ == "__main__": 184 | main() 185 | -------------------------------------------------------------------------------- /code/Classifier.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | # In[2]: 5 | 6 | 7 | import os 8 | import json 9 | from torchvision import datasets 10 | from torch.utils.data import Dataset, DataLoader 11 | from torchvision import transforms, datasets, models 12 | import torch 13 | from PIL import Image, ImageDraw 14 | from matplotlib import pyplot as plt 15 | from torch.optim import lr_scheduler 16 | import tqdm 17 | import numpy as np 18 | from matplotlib import pyplot as plt 19 | # from matplotlib.nxutils import points_inside_poly 20 | from make_imagenet_64_c import * 21 | 22 | 23 | # In[3]: 24 | 25 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 26 | 27 | 28 | 29 | readPath = '/Users/ali/Desktop/objectnet-1.0/images/' 30 | 31 | 32 | # In[4]: 33 | 34 | 35 | objsNames = {'chair':0, 'broom':1, 'candle':2, 'basket':3, 'banana':4, 'iron_for_clothes':5, 'plate':6, 'fan':7, 'toaster':8, 't-shirt':9} 36 | 37 | 38 | # In[5]: 39 | 40 | 41 | objs = list(objsNames.keys()) 42 | 43 | # with open('./models/train_data.npy', 'rb') as f: 44 | # train_data = np.load(f) 45 | 46 | with open('./models/val_data.npy', 'rb') as f: 47 | val_data = np.load(f,allow_pickle=True) 48 | 49 | 50 | 51 | 52 | def crop_mask(img, pts_lst): 53 | img_array = np.asarray(img) 54 | # print(img_array.shape) 55 | # width, height = img.size 56 | # polygon = [(x1,y1),(x2,y2),...] or [x1,y1,x2,y2,...] 57 | # width = ? 58 | # height = ? 59 | 60 | pts = np.array(pts_lst) 61 | pts = pts.astype('int') 62 | # print(pts) 63 | 64 | # read image as RGB (without alpha) 65 | # img = Image.open("crop.jpg").convert("RGB") 66 | 67 | # convert to numpy (for convenience) 68 | # img_array = numpy.asarray(img) 69 | 70 | # create mask 71 | polygon = [(u[0],u[1]) for u in pts] # [(40,203),(623,243),(691,177),(581,26),(482,42)] 72 | # print(polygon) 73 | # create new image ("1-bit pixels, black and white", (width, height), "default color") 74 | mask_img = Image.new('1', (img_array.shape[1], img_array.shape[0]), 0) 75 | 76 | ImageDraw.Draw(mask_img).polygon(polygon, outline=1, fill=1) 77 | mask = np.array(mask_img) 78 | 79 | # assemble new image (uint8: 0-255) 80 | new_img_array = np.ones(img_array.shape, dtype='uint8') 81 | 82 | # copy color values (RGB) 83 | new_img_array[:,:,:3] = img_array[:,:,:3] 84 | 85 | # filtering image by mask 86 | new_img_array[:,:,0] = new_img_array[:,:,0] * mask 87 | new_img_array[:,:,1] = new_img_array[:,:,1] * mask 88 | new_img_array[:,:,2] = new_img_array[:,:,2] * mask 89 | 90 | # back to Image from numpy 91 | img = Image.fromarray(new_img_array, "RGB") 92 | 93 | return img 94 | 95 | 96 | 97 | 98 | import collections 99 | 100 | print('\nUsing ObjectNet data') 101 | 102 | distortions = [ 103 | 'gaussian_noise', 104 | 'shot_noise', 105 | 'impulse_noise', 106 | 'defocus_blur', 107 | 'glass_blur', 108 | 'motion_blur', 109 | 'zoom_blur', 110 | 'snow', 111 | 'frost', 112 | 'fog', 113 | 'brightness', 114 | 'contrast', 115 | 'elastic_transform', 116 | # 'pixelate', 117 | 'jpeg_compression', 118 | # 'speckle_noise', 'gaussian_blur', 'spatter', 'saturate' 119 | ] 120 | transform = transforms.Compose([ 121 | # transforms.CenterCrop(224), 122 | transforms.ToTensor(), 123 | # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 124 | transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) 125 | ]) 126 | 127 | 128 | # In[8]: 129 | 130 | 131 | kind = 'seg' 132 | NUM_CLASSES = 10 133 | resnet = models.resnet18(pretrained=True) 134 | num_ftrs = resnet.fc.in_features 135 | resnet.fc = torch.nn.Linear(num_ftrs, NUM_CLASSES) 136 | resnet.load_state_dict(torch.load('./models/'+kind+'.pth')) 137 | 138 | 139 | # In[ ]: 140 | 141 | 142 | 143 | device = 'cpu' 144 | resnet.eval() 145 | error_rates = dict() 146 | for distortion in distortions: 147 | print(distortion + '\n') 148 | error_rates[distortion] = [] 149 | # error_rates_distortion = dict() 150 | 151 | for severity in range(1,4): 152 | print(str(severity)) 153 | # error_rates_distortion[severity] = [] 154 | 155 | res_top1, res_top5 = [], [] 156 | correct = 0 157 | total = 0 158 | 159 | # for n, category in enumerate(data): 160 | for img_path, polygon, label in val_data: 161 | images = Image.open(img_path).convert('RGB') 162 | # import pdb; pdb.set_trace() 163 | # images, labels = images[0].to(device), labels.to(device) 164 | #images = np.array(img)#.permute(1,2,0) 165 | if kind == 'box': 166 | pts = np.array(polygon) 167 | pts = pts.astype('int') 168 | min_x, max_x, min_y, max_y = pts[:,0].min(), pts[:,0].max(), pts[:,1].min(), pts[:,1].max() 169 | # print([min_x, max_x, min_y, max_y]) 170 | # print(img.width) 171 | images = images.crop([min_x, min_y, max_x, max_y]) 172 | 173 | if kind == 'seg': 174 | images = crop_mask(images, polygon) 175 | pts = np.array(polygon) 176 | pts = pts.astype('int') 177 | min_x, max_x, min_y, max_y = pts[:,0].min(), pts[:,0].max(), pts[:,1].min(), pts[:,1].max() 178 | # print([min_x, max_x, min_y, max_y]) 179 | # print(img.width) 180 | images = images.crop([min_x, min_y, max_x, max_y]) 181 | 182 | 183 | images = images.resize((224, 224)) 184 | 185 | exr = distortion + '(images,severity)' 186 | aa = torch.Tensor(eval(exr)) 187 | images = transform((aa/255).numpy()) # range [0,1] 188 | 189 | # import pdb; pdb.set_trace() 190 | outputs = resnet(images[None]) 191 | _, predicted = torch.max(outputs.data, 1) 192 | 193 | total += 1#label.size(0) 194 | # correct += (predicted == labels.cuda()).sum() 195 | correct += (predicted == label).sum() 196 | 197 | acc = float(correct) / total 198 | 199 | # after severity 200 | 201 | error_rates[distortion].append(acc) 202 | print(error_rates[distortion]) 203 | 204 | -------------------------------------------------------------------------------- /code/analysis-robustness_rob.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | # ## Note: You may need to adjust some paths in the following code. I moved some stuff around last minute! 5 | 6 | # In[3]: 7 | 8 | 9 | import torch 10 | import torchvision 11 | import torchvision.transforms as transforms 12 | import numpy as np 13 | import cv2 14 | from matplotlib import pyplot as plt 15 | import os 16 | 17 | from overlapped_classes import overlapped_classes 18 | import json 19 | from PIL import Image 20 | import scipy 21 | from make_imagenet_64_c import * 22 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 23 | 24 | 25 | # In[4]: 26 | 27 | 28 | with open("../mappings/objectnet_to_imagenet_1k.json") as f: 29 | data = json.load(f) 30 | 31 | with open("../mappings/pytorch_to_imagenet_2012_id.json") as f: 32 | idxMap = json.load(f) 33 | 34 | with open("../mappings/folder_to_objectnet_label.json") as f: 35 | folderMap = json.load(f) 36 | 37 | with open('imagenet_classes.txt') as f: 38 | # with open('../mappings/imagenet_to_label_2012_v2.txt') as f: 39 | labels = [line.strip() for line in f.readlines()] 40 | 41 | 42 | # In[5]: 43 | 44 | 45 | # inverting the folderMap 46 | dirMap = {} 47 | for u, v in folderMap.items(): 48 | dirMap[v]= u 49 | 50 | 51 | # In[6]: 52 | 53 | 54 | from torchvision import models 55 | dir(models) 56 | 57 | # models.googlenet(pretrained=True)] #, 58 | 59 | models_ = [#models.googlenet(pretrained=True), models.alexnet(pretrained=True), 60 | models.vgg19(pretrained=True), 61 | models.resnet152(pretrained=True), 62 | models.inception_v3(pretrained=True)] 63 | #models.mnasnet1_0(pretrained=True)] #, models.resnext101_32x8d(pretrained=True), models.densenet161(pretrained=True)] 64 | 65 | 66 | # In[7]: 67 | 68 | 69 | 70 | # model = models.alexnet(pretrained=True) 71 | # model = models.resnet152(pretrained=True) 72 | # model = models.inception_v3(pretrained=True) 73 | # model = models.googlenet(pretrained=True) 74 | 75 | # for model_name in models_: 76 | # model = eval(model_name) 77 | # model.eval() 78 | 79 | 80 | # In[8]: 81 | 82 | 83 | def predict_image(image_path, box, draw=False, kind='Box', distortion='blur', severity=1): 84 | # Define transformations for the image, should (note that imagenet models are trained with image size 224) 85 | transform = transforms.Compose([ 86 | # transforms.CenterCrop(224), 87 | transforms.ToTensor(), 88 | # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 89 | transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) 90 | ]) 91 | 92 | 93 | if kind == 'Crop': 94 | transform = transforms.Compose([ 95 | transforms.CenterCrop(224), 96 | transforms.ToTensor(), 97 | # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 98 | transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) 99 | ]) 100 | 101 | 102 | img = Image.open(image_path).convert("RGB") 103 | 104 | if kind == 'Box': 105 | xl, yl, xr, yr = box 106 | img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) 107 | img = img.resize((224, 224)) 108 | 109 | 110 | if kind == 'Full': 111 | # xl, yl, xr, yr = box 112 | # img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) 113 | img = img.resize((224, 224)) 114 | 115 | 116 | # apply the distortion 117 | # import pdb; pdb.set_trace() 118 | exr = distortion + '(img,severity)' 119 | aa = torch.Tensor(eval(exr)) 120 | img_t = transform((aa/255).numpy()) # range [0,1] 121 | 122 | 123 | # import pdb; pdb.set_trace() 124 | # img_t = transform(img).float() 125 | 126 | 127 | if draw: 128 | plt.imshow(img) #img_t.permute((2,1,0)) ) 129 | plt.show() 130 | 131 | image_tensor = img_t.unsqueeze_(0) 132 | 133 | if torch.cuda.is_available(): 134 | image_tensor.cuda() 135 | 136 | # Predict the class of the image 137 | # import pdb; pdb.set_trace() 138 | output = model(image_tensor) 139 | 140 | confs, indices = torch.sort(output.data, descending=True) 141 | confs[0] = confs[0]/sum(confs[0]) 142 | return np.abs(confs[0][:5]), indices[0][:5] # index 143 | 144 | 145 | # In[ ]: 146 | 147 | 148 | 149 | 150 | 151 | # In[9]: 152 | 153 | 154 | # /////////////// Display Results /////////////// 155 | import collections 156 | 157 | print('\nUsing ObjectNet data') 158 | 159 | distortions = [ 160 | 'gaussian_noise', 161 | 'shot_noise', 162 | 'impulse_noise', 163 | 'defocus_blur', 'glass_blur', 164 | 'motion_blur', 165 | 'zoom_blur', 166 | 'snow', 167 | 'frost', 168 | 'fog', 169 | 'brightness', 170 | 'contrast', 171 | 'elastic_transform', 172 | # 'pixelate', 173 | 'jpeg_compression', 174 | # 'speckle_noise', 'gaussian_blur', 'spatter', 'saturate' 175 | ] 176 | 177 | 178 | 179 | 180 | # In[10]: 181 | 182 | 183 | # data.items() 184 | 185 | 186 | # In[3]: 187 | 188 | 189 | kind = 'Box' 190 | error_rates = dict() 191 | for distortion in distortions: 192 | print(distortion + '\n') 193 | error_rates[distortion] = [] 194 | error_rates_distortion = dict() 195 | 196 | for severity in range(4,5): #range(1,4): 197 | print(str(severity) + '\n') 198 | error_rates_distortion[severity] = [] 199 | 200 | for model in models_: 201 | model_name = model._get_name() #eval(model_name) 202 | model.eval() 203 | 204 | print(model_name + '\n') 205 | 206 | res_top1 = [] 207 | res_top5 = [] 208 | 209 | save_file = os.path.join('./outputs_robust/', distortion + '_' + str(severity) + '_' + model_name + '.txt') 210 | 211 | for n, category in enumerate(data): 212 | # if n > 1: break 213 | txtfile = os.path.join('../Annotations', dirMap[category] + '.txt') 214 | if not os.path.exists(txtfile): 215 | continue 216 | 217 | with open(txtfile) as f: 218 | boxes = [line.strip() for line in f.readlines()] 219 | 220 | # writing to the file 221 | with open(save_file, 'a+') as ff: 222 | ff.write(f"{category} ---------------------------------------------------------------- \n") 223 | 224 | 225 | count = [0, 0] 226 | lines = 0 227 | for im in boxes[0:10]: 228 | # import pdb; pdb.set_trace() 229 | ss = im.split(' ') 230 | fName = ss[0] 231 | if len(ss) > 1: 232 | lines += 1 233 | coords = (int(i) for i in ss[1:] if i) 234 | confs, idx = predict_image(os.path.join('../images/' + dirMap[category] + '/', fName), coords, False, kind, distortion, severity) 235 | 236 | # top 1 237 | count[0] += 1 if labels[idx[0]] in data[category] else 0 238 | 239 | # top 5 240 | flag = False 241 | for i in idx[0:]: 242 | flag = flag or (labels[i] in data[category]) 243 | count[1] += 1 if flag else 0 244 | 245 | # writing to the file 246 | with open(save_file, 'a') as ff: 247 | ff.write(f"{fName} {int(idx[0])} {int(idx[1])} {int(idx[2])} {int(idx[3])} {int(idx[4])} {confs[0]:.2f} {confs[1]:.2f} {confs[2]:.2f} {confs[3]:.2f} {confs[4]:.2f} \n") 248 | 249 | accs = np.array(count)*100/lines 250 | 251 | print(f"{n} -> {category}: top 1: {accs[0]:.2f} - top 5: {accs[1]:.2f} [num imgs: {lines}]") 252 | 253 | with open(save_file, 'a+') as ff: 254 | ff.write(f"{n} -> {category}: top 1: {accs[0]:.2f} - top 5: {accs[1]:.2f} [num imgs: {lines}] \n") 255 | 256 | res_top1.append(accs[0]) 257 | res_top5.append(accs[1]) 258 | 259 | # avg. over categories 260 | sum1_model = sum(res_top1)/len(res_top1) 261 | sum5_model = sum(res_top5)/len(res_top5) 262 | print(sum1_model) 263 | print(sum5_model) 264 | 265 | # after severity 266 | error_rates_distortion[severity].append((model_name, sum1_model, sum5_model)) 267 | 268 | error_rates[distortion].append(error_rates_distortion) 269 | print(error_rates[distortion]) 270 | 271 | 272 | # In[4]: 273 | 274 | 275 | import pickle 276 | 277 | # a = {'hello': 'world'} 278 | 279 | with open('error_robust_box.pickle', 'wb') as handle: 280 | pickle.dump(error_rates, handle, protocol=pickle.HIGHEST_PROTOCOL) 281 | 282 | # with open('filename.pickle', 'rb') as handle: 283 | # b = pickle.load(handle) 284 | 285 | # print a == b 286 | 287 | 288 | # In[32]: 289 | 290 | 291 | # pip install wand 292 | 293 | 294 | # print(sum(res_top1)/len(res_top1)) 295 | # print(sum(res_top5)/len(res_top5)) 296 | 297 | # In[5]: 298 | 299 | 300 | # # eval(model) 301 | # # int(idx[0]) 302 | # print(f"{1.911333:.2f}") 303 | # # print(f'{val:.2f}') 304 | 305 | # confs, idx = predict_image(os.path.join('../images/' + dirMap[category] + '/', fName), coords, False) 306 | # # confs / sum(confs) 307 | 308 | 309 | # In[6]: 310 | 311 | 312 | # eval(model_name) 313 | 314 | 315 | # In[7]: 316 | 317 | 318 | # models_ = [models.googlenet(pretrained=True), models.alexnet(pretrained=True), models.vgg19(pretrained=True), 319 | # models.resnet152(pretrained=True), models.inception_v3(pretrained=True), 320 | # models.mnasnet(pretrained=True), models.resnext101_32x8d(pretrained=True)] 321 | 322 | 323 | # In[8]: 324 | 325 | 326 | # m = models.mnasnet1_0(pretrained=True) 327 | 328 | 329 | # In[9]: 330 | 331 | 332 | # # dir(models) 333 | # models_[0]._get_name() 334 | 335 | -------------------------------------------------------------------------------- /code/analysis-robustness_rob_seg.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | # ## Note: You may need to adjust some paths in the following code. I moved some stuff around last minute! 5 | 6 | # In[3]: 7 | 8 | 9 | import torch 10 | import torchvision 11 | import torchvision.transforms as transforms 12 | import numpy as np 13 | import cv2 14 | from matplotlib import pyplot as plt 15 | import os 16 | 17 | from overlapped_classes import overlapped_classes 18 | import json 19 | from PIL import Image 20 | import scipy 21 | from make_imagenet_64_c import * 22 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 23 | 24 | 25 | 26 | from torchvision import models 27 | 28 | 29 | model = models.resnet18(pretrained=True) 30 | load_model(model, load_path) 31 | model.to(device) 32 | model.eval() 33 | 34 | 35 | # In[7]: 36 | 37 | 38 | 39 | # model = models.alexnet(pretrained=True) 40 | 41 | # model = models.inception_v3(pretrained=True) 42 | # model = models.googlenet(pretrained=True) 43 | 44 | # for model_name in models_: 45 | # model = eval(model_name) 46 | # model.eval() 47 | 48 | 49 | # In[8]: 50 | 51 | 52 | def predict_image(image_path, box, draw=False, kind='Box', distortion='blur', severity=1): 53 | # Define transformations for the image, should (note that imagenet models are trained with image size 224) 54 | transform = transforms.Compose([ 55 | # transforms.CenterCrop(224), 56 | transforms.ToTensor(), 57 | # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 58 | transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) 59 | ]) 60 | 61 | 62 | # if kind == 'Crop': 63 | # transform = transforms.Compose([ 64 | # transforms.CenterCrop(224), 65 | # transforms.ToTensor(), 66 | # # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) 67 | # transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) 68 | # ]) 69 | 70 | 71 | img = Image.open(image_path).convert("RGB") 72 | 73 | if kind == 'Box': 74 | xl, yl, xr, yr = box 75 | img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) 76 | img = img.resize((224, 224)) 77 | 78 | 79 | if kind == 'Full': 80 | # xl, yl, xr, yr = box 81 | # img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) 82 | img = img.resize((224, 224)) 83 | 84 | 85 | # apply the distortion 86 | # import pdb; pdb.set_trace() 87 | exr = distortion + '(img,severity)' 88 | aa = torch.Tensor(eval(exr)) 89 | img_t = transform((aa/255).numpy()) # range [0,1] 90 | 91 | 92 | # import pdb; pdb.set_trace() 93 | # img_t = transform(img).float() 94 | 95 | 96 | if draw: 97 | plt.imshow(img) #img_t.permute((2,1,0)) ) 98 | plt.show() 99 | 100 | image_tensor = img_t.unsqueeze_(0) 101 | 102 | if torch.cuda.is_available(): 103 | image_tensor.cuda() 104 | 105 | # Predict the class of the image 106 | # import pdb; pdb.set_trace() 107 | output = model(image_tensor) 108 | 109 | confs, indices = torch.sort(output.data, descending=True) 110 | confs[0] = confs[0]/sum(confs[0]) 111 | return np.abs(confs[0][:5]), indices[0][:5] # index 112 | 113 | 114 | # In[ ]: 115 | 116 | 117 | 118 | 119 | 120 | # In[9]: 121 | 122 | 123 | # /////////////// Display Results /////////////// 124 | import collections 125 | 126 | print('\nUsing ObjectNet data') 127 | 128 | distortions = [ 129 | 'gaussian_noise', 130 | 'shot_noise', 131 | 'impulse_noise', 132 | 'defocus_blur', 'glass_blur', 133 | 'motion_blur', 134 | 'zoom_blur', 135 | 'snow', 136 | 'frost', 137 | 'fog', 138 | 'brightness', 139 | 'contrast', 140 | 'elastic_transform', 141 | # 'pixelate', 142 | 'jpeg_compression', 143 | # 'speckle_noise', 'gaussian_blur', 'spatter', 'saturate' 144 | ] 145 | 146 | 147 | 148 | 149 | # In[10]: 150 | 151 | 152 | # data.items() 153 | 154 | 155 | # In[3]: 156 | 157 | 158 | kind = 'Full' 159 | error_rates = dict() 160 | for distortion in distortions: 161 | print(distortion + '\n') 162 | error_rates[distortion] = [] 163 | error_rates_distortion = dict() 164 | 165 | for severity in range(4,5): #range(1,4): 166 | print(str(severity) + '\n') 167 | error_rates_distortion[severity] = [] 168 | 169 | # print(model_name + '\n') 170 | 171 | res_top1 = [] 172 | res_top5 = [] 173 | 174 | # save_file = os.path.join('./outputs_robust/', distortion + '_' + str(severity) + '_' + model_name + '.txt') 175 | 176 | # for n, category in enumerate(data): 177 | for inputs, labels in val_dataloader: 178 | 179 | # if n > 1: break 180 | count = [0, 0] 181 | lines = 0 182 | for im in boxes[0:10]: 183 | 184 | 185 | confs, idx = predict_image(os.path.join('../images/' + dirMap[category] + '/', fName), coords, False, kind, distortion, severity) 186 | 187 | # top 1 188 | count[0] += 1 if labels[idx[0]] in data[category] else 0 189 | 190 | # top 5 191 | flag = False 192 | for i in idx[0:]: 193 | flag = flag or (labels[i] in data[category]) 194 | count[1] += 1 if flag else 0 195 | 196 | # writing to the file 197 | with open(save_file, 'a') as ff: 198 | ff.write(f"{fName} {int(idx[0])} {int(idx[1])} {int(idx[2])} {int(idx[3])} {int(idx[4])} {confs[0]:.2f} {confs[1]:.2f} {confs[2]:.2f} {confs[3]:.2f} {confs[4]:.2f} \n") 199 | 200 | accs = np.array(count)*100/lines 201 | 202 | print(f"{n} -> {category}: top 1: {accs[0]:.2f} - top 5: {accs[1]:.2f} [num imgs: {lines}]") 203 | 204 | 205 | res_top1.append(accs[0]) 206 | res_top5.append(accs[1]) 207 | 208 | # avg. over categories 209 | sum1_model = sum(res_top1)/len(res_top1) 210 | sum5_model = sum(res_top5)/len(res_top5) 211 | print(sum1_model) 212 | print(sum5_model) 213 | 214 | # after severity 215 | error_rates_distortion[severity].append((model_name, sum1_model, sum5_model)) 216 | 217 | error_rates[distortion].append(error_rates_distortion) 218 | print(error_rates[distortion]) 219 | 220 | 221 | # In[4]: 222 | 223 | 224 | # import pickle 225 | 226 | # a = {'hello': 'world'} 227 | 228 | # with open('error_robust_box.pickle', 'wb') as handle: 229 | # pickle.dump(error_rates, handle, protocol=pickle.HIGHEST_PROTOCOL) 230 | 231 | # with open('filename.pickle', 'rb') as handle: 232 | # b = pickle.load(handle) 233 | 234 | -------------------------------------------------------------------------------- /code/analysis2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import torch\n", 10 | "import torchvision\n", 11 | "import torchvision.transforms as transforms\n", 12 | "import numpy as np\n", 13 | "import cv2\n", 14 | "from matplotlib import pyplot as plt\n", 15 | "import os\n", 16 | "\n", 17 | "from overlapped_classes import overlapped_classes\n", 18 | "import json\n", 19 | "from PIL import Image\n", 20 | "import scipy" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 2, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "with open(\"../mappings/objectnet_to_imagenet_1k.json\") as f:\n", 30 | " data = json.load(f)\n", 31 | "\n", 32 | "with open(\"../mappings/pytorch_to_imagenet_2012_id.json\") as f:\n", 33 | " idxMap = json.load(f)\n", 34 | " \n", 35 | "with open(\"../mappings/folder_to_objectnet_label.json\") as f:\n", 36 | " folderMap = json.load(f)\n", 37 | "\n", 38 | "with open('imagenet_classes.txt') as f:\n", 39 | "# with open('../mappings/imagenet_to_label_2012_v2.txt') as f: \n", 40 | " labels = [line.strip() for line in f.readlines()] " 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 3, 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "# inverting the folderMap\n", 50 | "dirMap = {}\n", 51 | "for u, v in folderMap.items():\n", 52 | " dirMap[v]= u" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 36, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "# dirMap" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "# help(json.load)\n", 71 | "# for i in overlapped_classes:\n", 72 | "# if not data[i] in labels:\n", 73 | "# print(f\"{i} : {data[i]}, {data[i] in labels}\")" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "# overlapped_classes\n", 83 | "# data['Alarm clock']\n", 84 | "# idxMap" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "metadata": {}, 91 | "outputs": [], 92 | "source": [ 93 | "# for i in overlapped_classes:\n", 94 | "# print(data[i] in labels)\n", 95 | "# print(f\"{i} : {data[i]}\")\n", 96 | "del model" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": 62, 102 | "metadata": {}, 103 | "outputs": [ 104 | { 105 | "data": { 106 | "text/plain": [ 107 | "AlexNet(\n", 108 | " (features): Sequential(\n", 109 | " (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))\n", 110 | " (1): ReLU(inplace=True)\n", 111 | " (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 112 | " (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n", 113 | " (4): ReLU(inplace=True)\n", 114 | " (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 115 | " (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 116 | " (7): ReLU(inplace=True)\n", 117 | " (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 118 | " (9): ReLU(inplace=True)\n", 119 | " (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 120 | " (11): ReLU(inplace=True)\n", 121 | " (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 122 | " )\n", 123 | " (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))\n", 124 | " (classifier): Sequential(\n", 125 | " (0): Dropout(p=0.5, inplace=False)\n", 126 | " (1): Linear(in_features=9216, out_features=4096, bias=True)\n", 127 | " (2): ReLU(inplace=True)\n", 128 | " (3): Dropout(p=0.5, inplace=False)\n", 129 | " (4): Linear(in_features=4096, out_features=4096, bias=True)\n", 130 | " (5): ReLU(inplace=True)\n", 131 | " (6): Linear(in_features=4096, out_features=1000, bias=True)\n", 132 | " )\n", 133 | ")" 134 | ] 135 | }, 136 | "execution_count": 62, 137 | "metadata": {}, 138 | "output_type": "execute_result" 139 | } 140 | ], 141 | "source": [ 142 | "from torchvision import models\n", 143 | "dir(models)\n", 144 | "\n", 145 | "\n", 146 | "\n", 147 | "model = models.alexnet(pretrained=True)\n", 148 | "# model = models.resnet152(pretrained=True)\n", 149 | "# model = models.inception_v3(pretrained=True)\n", 150 | "# model = models.googlenet(pretrained=True)\n", 151 | "\n", 152 | "model.eval()\n" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": 59, 158 | "metadata": {}, 159 | "outputs": [], 160 | "source": [ 161 | "def predict_image(image_path, box, draw=False):\n", 162 | " # Define transformations for the image, should (note that imagenet models are trained with image size 224)\n", 163 | " transform = transforms.Compose([\n", 164 | "# transforms.CenterCrop(224),\n", 165 | " transforms.ToTensor(),\n", 166 | "# transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n", 167 | " transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) \n", 168 | "\n", 169 | " ])\n", 170 | " \n", 171 | " \n", 172 | "# transform = transforms.Compose([ #[1]\n", 173 | "# # transforms.Resize(256), #[2]\n", 174 | "# # transforms.CenterCrop(224), #[3]\n", 175 | "# transforms.ToTensor(), #[4]\n", 176 | "# # transforms.Normalize( #[5]\n", 177 | "# # mean=[0.485, 0.456, 0.406], #[6]\n", 178 | "# # std=[0.229, 0.224, 0.225] #[7]\n", 179 | "# # )\n", 180 | "# transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)) \n", 181 | "# ])\n", 182 | " \n", 183 | "# print(\"Prediction in progress\")\n", 184 | " img = Image.open(image_path).convert(\"RGB\")\n", 185 | "\n", 186 | " \n", 187 | " xl, yl, xr, yr = box\n", 188 | " img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) \n", 189 | " img = img.resize((224, 224))\n", 190 | "\n", 191 | " img_t = transform(img).float()\n", 192 | "\n", 193 | "# img_t = img_t.permute((0,1,2))\n", 194 | "# img_t = torch.transpose(img_t, 0, 1) # rotate\n", 195 | " \n", 196 | " if draw:\n", 197 | " plt.imshow(img) #img_t.permute((2,1,0)) )\n", 198 | " # ## plt.imshow(torch.transpose(img_t.permute((1,2,0)), 0, 1))\n", 199 | " plt.show()\n", 200 | "\n", 201 | "# print(img_t.shape)\n", 202 | " # Preprocess the image\n", 203 | "# image_tensor = transformation(image).float()\n", 204 | "\n", 205 | " # Add an extra batch dimension since pytorch treats all images as batches\n", 206 | "# image_tensor = image_tensor.unsqueeze_(0)\n", 207 | " image_tensor = img_t.unsqueeze_(0)\n", 208 | "\n", 209 | " if torch.cuda.is_available():\n", 210 | " image_tensor.cuda()\n", 211 | "\n", 212 | " # Turn the input into a Variable\n", 213 | "# input = Variable(image_tensor)\n", 214 | "\n", 215 | " # Predict the class of the image\n", 216 | " output = model(image_tensor)\n", 217 | "\n", 218 | "# index = output.data.numpy().argmax() # top 1\n", 219 | " _, indices = torch.sort(output.data, descending=True)\n", 220 | "# [(labels[idx], percentage[idx].item()) for idx in indices[0][:5]]\n", 221 | " return indices[0][:5] # index" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": 41, 227 | "metadata": {}, 228 | "outputs": [ 229 | { 230 | "data": { 231 | "text/plain": [ 232 | "'alarm_clock'" 233 | ] 234 | }, 235 | "execution_count": 41, 236 | "metadata": {}, 237 | "output_type": "execute_result" 238 | } 239 | ], 240 | "source": [ 241 | "len(data)\n", 242 | "dirMap[category]\n" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": 63, 248 | "metadata": {}, 249 | "outputs": [ 250 | { 251 | "name": "stdout", 252 | "output_type": "stream", 253 | "text": [ 254 | "0 -> Alarm clock: top 1: 6.25 - top 5: 20.13888888888889 [num imgs: 144]\n", 255 | "1 -> Backpack: top 1: 27.659574468085108 - top 5: 53.61702127659574 [num imgs: 235]\n", 256 | "2 -> Banana: top 1: 44.255319148936174 - top 5: 62.5531914893617 [num imgs: 235]\n", 257 | "3 -> Band Aid: top 1: 20.081967213114755 - top 5: 37.704918032786885 [num imgs: 244]\n", 258 | "4 -> Basket: top 1: 5.524861878453039 - top 5: 23.756906077348066 [num imgs: 181]\n", 259 | "5 -> Bath towel: top 1: 10.06711409395973 - top 5: 26.845637583892618 [num imgs: 149]\n", 260 | "6 -> Beer bottle: top 1: 8.67579908675799 - top 5: 17.80821917808219 [num imgs: 219]\n", 261 | "7 -> Bench: top 1: 0.0 - top 5: 3.0303030303030303 [num imgs: 132]\n", 262 | "8 -> Bicycle: top 1: 20.408163265306122 - top 5: 40.816326530612244 [num imgs: 147]\n", 263 | "9 -> Binder (closed): top 1: 8.241758241758241 - top 5: 25.274725274725274 [num imgs: 182]\n", 264 | "10 -> Bottle cap: top 1: 8.870967741935484 - top 5: 19.35483870967742 [num imgs: 248]\n", 265 | "11 -> Bread loaf: top 1: 2.262443438914027 - top 5: 6.334841628959276 [num imgs: 221]\n", 266 | "12 -> Broom: top 1: 14.655172413793103 - top 5: 35.775862068965516 [num imgs: 232]\n", 267 | "13 -> Bucket: top 1: 10.18867924528302 - top 5: 25.660377358490567 [num imgs: 265]\n", 268 | "14 -> Butcher's knife: top 1: 8.717948717948717 - top 5: 24.615384615384617 [num imgs: 195]\n", 269 | "15 -> Can opener: top 1: 9.313725490196079 - top 5: 27.941176470588236 [num imgs: 204]\n", 270 | "16 -> Candle: top 1: 1.7167381974248928 - top 5: 6.866952789699571 [num imgs: 233]\n", 271 | "18 -> Chair: top 1: 18.636363636363637 - top 5: 41.81818181818182 [num imgs: 220]\n", 272 | "20 -> Coffee/French press: top 1: 2.803738317757009 - top 5: 14.018691588785046 [num imgs: 107]\n", 273 | "21 -> Combination lock: top 1: 24.305555555555557 - top 5: 37.5 [num imgs: 144]\n", 274 | "22 -> Computer mouse: top 1: 29.017857142857142 - top 5: 49.107142857142854 [num imgs: 224]\n", 275 | "23 -> Desk lamp: top 1: 3.5 - top 5: 15.0 [num imgs: 200]\n", 276 | "24 -> Dishrag or hand towel: top 1: 0.0 - top 5: 33.333333333333336 [num imgs: 3]\n", 277 | "68 -> Printer: top 1: 22.4 - top 5: 49.6 [num imgs: 125]\n", 278 | "69 -> Remote control: top 1: 33.333333333333336 - top 5: 49.18032786885246 [num imgs: 183]\n", 279 | "70 -> Ruler: top 1: 23.026315789473685 - top 5: 40.78947368421053 [num imgs: 152]\n" 280 | ] 281 | } 282 | ], 283 | "source": [ 284 | "# category = 'Banana'\n", 285 | "\n", 286 | "res_top1 = []\n", 287 | "res_top5 = []\n", 288 | "\n", 289 | "for n, category in enumerate(data):\n", 290 | "# if n > 1:\n", 291 | "# break\n", 292 | " txtfile = '../' + dirMap[category] + '.txt'\n", 293 | " if not os.path.exists(txtfile):\n", 294 | " continue\n", 295 | " \n", 296 | " with open(txtfile) as f:\n", 297 | " boxes = [line.strip() for line in f.readlines()] \n", 298 | "\n", 299 | "# preds = []\n", 300 | " count = [0, 0]\n", 301 | " lines = 0\n", 302 | " for im in boxes[0:]:\n", 303 | " ss = im.split(' ')\n", 304 | " if len(ss) > 1:\n", 305 | " lines += 1\n", 306 | " coords = (int(i) for i in ss[1:] if i)\n", 307 | " idx = predict_image(os.path.join('../images/' + dirMap[category] + '/', ss[0]), coords, False)\n", 308 | " \n", 309 | " # top 1\n", 310 | " count[0] += 1 if labels[idx[0]] in data[category] else 0 \n", 311 | " \n", 312 | " # top 5\n", 313 | " flag = False\n", 314 | " for i in idx[0:]:\n", 315 | " # print(labels[i], end = ' ')\n", 316 | "# predLabels = labels[idxMap[str(i.tolist())]]\n", 317 | "# print(i, labels[i])\n", 318 | " flag = flag or (labels[i] in data[category])\n", 319 | " count[1] += 1 if flag else 0\n", 320 | "# print('\\n')\n", 321 | "# preds.append(idx)\n", 322 | "\n", 323 | "\n", 324 | "\n", 325 | " accs = np.array(count)*100/lines\n", 326 | " \n", 327 | " print(f\"{n} -> {category}: top 1: {accs[0]} - top 5: {accs[1]} [num imgs: {lines}]\") \n", 328 | "\n", 329 | " res_top1.append(accs[0]) \n", 330 | " res_top5.append(accs[1])\n" 331 | ] 332 | }, 333 | { 334 | "cell_type": "code", 335 | "execution_count": 64, 336 | "metadata": {}, 337 | "outputs": [ 338 | { 339 | "name": "stdout", 340 | "output_type": "stream", 341 | "text": [ 342 | "13.996669092969494\n", 343 | "30.32472008287954\n" 344 | ] 345 | } 346 | ], 347 | "source": [ 348 | "print(sum(res_top1)/len(res_top1))\n", 349 | "print(sum(res_top5)/len(res_top5))\n" 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "# model = alexnet\n", 359 | "# for l in preds:\n", 360 | "# print(labels[l])\n", 361 | "# preds \n" 362 | ] 363 | }, 364 | { 365 | "cell_type": "code", 366 | "execution_count": null, 367 | "metadata": {}, 368 | "outputs": [], 369 | "source": [ 370 | "# \n", 371 | "if ss is ['']:\n", 372 | " print('ss')\n", 373 | "len(ss)" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": null, 379 | "metadata": {}, 380 | "outputs": [], 381 | "source": [ 382 | "# Import Pillow\n", 383 | "\n", 384 | "from PIL import Image\n", 385 | "img = Image.open(\"../images/air_freshener/01d44a4b77b44a0.png\").convert('RGB')\n", 386 | "\n", 387 | "img = img.crop((xl, yl, xr, yr)) #((left, top, right, bottom)) \n", 388 | "img = img.resize((224, 224))\n", 389 | "\n", 390 | "\n", 391 | "img_t = transform(img)#.float()\n", 392 | "\n", 393 | "img_t = img_t.permute((2,1,0))\n", 394 | "\n", 395 | "img_t = torch.transpose(img_t, 0, 1)\n", 396 | "\n", 397 | "# np.clip\n", 398 | "# img_t = torch.unsqueeze(img_t, 0)\n", 399 | "# help(img_t.transpose)\n", 400 | "\n", 401 | "plt.imshow(img_t)\n", 402 | "\n", 403 | "# img = cv2.imread(\"../images/air_freshener/01d44a4b77b44a0.png\", cv2.IMREAD_UNCHANGED)\n", 404 | "# img = cv2cvLoadImage(\"../images/air_freshener/01d44a4b77b44a0.png\", CV_LOAD_IMAGE_UNCHANGED)\n" 405 | ] 406 | }, 407 | { 408 | "cell_type": "code", 409 | "execution_count": null, 410 | "metadata": {}, 411 | "outputs": [], 412 | "source": [ 413 | "np.asarray(img_t).shape" 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": null, 419 | "metadata": {}, 420 | "outputs": [], 421 | "source": [ 422 | "img_t[:10,:10].min()" 423 | ] 424 | }, 425 | { 426 | "cell_type": "code", 427 | "execution_count": null, 428 | "metadata": {}, 429 | "outputs": [], 430 | "source": [ 431 | "help(torch.transpose)" 432 | ] 433 | }, 434 | { 435 | "cell_type": "code", 436 | "execution_count": null, 437 | "metadata": {}, 438 | "outputs": [], 439 | "source": [ 440 | "# plt.show(img)\n", 441 | "%matplotlib inline\n", 442 | "img = np.array(img)\n", 443 | "# img = img[:,:,::-1]\n", 444 | "\n", 445 | "\n", 446 | "\n", 447 | "\n" 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": null, 453 | "metadata": {}, 454 | "outputs": [], 455 | "source": [ 456 | "with open('../air_freshener.txt') as f:\n", 457 | " boxes = [line.strip() for line in f.readlines()] " 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": null, 463 | "metadata": {}, 464 | "outputs": [], 465 | "source": [ 466 | "xl, yl, xr, yr = [int(i) for i in boxes[11].split(' ')[1:] if i]\n", 467 | "img[yl:yr,xl:xr, :]\n", 468 | "\n", 469 | "# std = np.array([0.229, 0.224, 0.225])\n", 470 | "# inp = std * inp + mean\n", 471 | "# inp = np.clip(inp, 0, 1)\n", 472 | "\n", 473 | "img_t = transform().float()\n", 474 | "batch_t = torch.unsqueeze(img, 0)\n", 475 | "\n", 476 | "\n", 477 | "plt.imshow(batch_t)\n", 478 | "# cv2.imshow('t', img[xl:xr, yl:yr,:])\n", 479 | "# # help(cv2.imshow())\n", 480 | "# cv2.waitKey(0)\n", 481 | "\n", 482 | "# Image.crop()" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": {}, 489 | "outputs": [], 490 | "source": [ 491 | "# cv2.destroyAllWindows()\n" 492 | ] 493 | }, 494 | { 495 | "cell_type": "code", 496 | "execution_count": null, 497 | "metadata": {}, 498 | "outputs": [], 499 | "source": [ 500 | "xl, yl, xr, yr" 501 | ] 502 | }, 503 | { 504 | "cell_type": "code", 505 | "execution_count": null, 506 | "metadata": {}, 507 | "outputs": [], 508 | "source": [ 509 | "batch_t.shape" 510 | ] 511 | }, 512 | { 513 | "cell_type": "code", 514 | "execution_count": null, 515 | "metadata": {}, 516 | "outputs": [], 517 | "source": [ 518 | "# resnet = models.resnet101(pretrained=True)\n", 519 | "\n", 520 | "alexnet.eval()\n", 521 | "img_a = img_t.permute((2,1,0))\n", 522 | "batch_t = torch.unsqueeze(img_a, 0)\n", 523 | "out = alexnet(batch_t)\n", 524 | "\n", 525 | "print(out.shape)" 526 | ] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "execution_count": null, 531 | "metadata": {}, 532 | "outputs": [], 533 | "source": [ 534 | "# out\n", 535 | "batch_t.shape" 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": null, 541 | "metadata": {}, 542 | "outputs": [], 543 | "source": [ 544 | "with open('imagenet_classes.txt') as f:\n", 545 | " labels = [line.strip() for line in f.readlines()]" 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": null, 551 | "metadata": {}, 552 | "outputs": [], 553 | "source": [ 554 | "_, index = torch.max(out, 1)\n", 555 | "percentage = torch.nn.functional.softmax(out, dim=1)[0] * 100\n", 556 | "print(labels[index[0]], percentage[index[0]].item())" 557 | ] 558 | }, 559 | { 560 | "cell_type": "code", 561 | "execution_count": null, 562 | "metadata": {}, 563 | "outputs": [], 564 | "source": [ 565 | "_, indices = torch.sort(out, descending=True)\n", 566 | "2\n", 567 | "[(labels[idx], percentage[idx].item()) for idx in indices[0][:5]]" 568 | ] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "execution_count": null, 573 | "metadata": {}, 574 | "outputs": [], 575 | "source": [ 576 | "index" 577 | ] 578 | }, 579 | { 580 | "cell_type": "code", 581 | "execution_count": null, 582 | "metadata": {}, 583 | "outputs": [], 584 | "source": [] 585 | }, 586 | { 587 | "cell_type": "code", 588 | "execution_count": null, 589 | "metadata": {}, 590 | "outputs": [], 591 | "source": [] 592 | }, 593 | { 594 | "cell_type": "code", 595 | "execution_count": null, 596 | "metadata": {}, 597 | "outputs": [], 598 | "source": [] 599 | }, 600 | { 601 | "cell_type": "code", 602 | "execution_count": 4, 603 | "metadata": {}, 604 | "outputs": [ 605 | { 606 | "name": "stdout", 607 | "output_type": "stream", 608 | "text": [ 609 | "Cellphone\n", 610 | "Clothes hamper\n", 611 | "Mixing / Salad Bowl\n", 612 | "Monitor\n", 613 | "Mug\n", 614 | "Nail (fastener)\n", 615 | "Necklace\n", 616 | "Orange\n", 617 | "Padlock\n", 618 | "Paintbrush\n", 619 | "Paper towel\n", 620 | "Pen\n", 621 | "Pill bottle\n", 622 | "Pillow\n", 623 | "Plastic bag\n", 624 | "Plate\n", 625 | "Soap dispenser\n", 626 | "Sock\n", 627 | "Soup Bowl\n", 628 | "Spatula\n", 629 | "Speaker\n", 630 | "Still Camera\n", 631 | "Strainer\n", 632 | "Stuffed animal\n", 633 | "Suit jacket\n", 634 | "Sunglasses\n", 635 | "Sweater\n", 636 | "Swimming trunks\n", 637 | "T-shirt\n", 638 | "TV\n", 639 | "Teapot\n", 640 | "Wheel\n", 641 | "Whistle\n", 642 | "Wine bottle\n", 643 | "Winter glove\n", 644 | "Wok\n" 645 | ] 646 | } 647 | ], 648 | "source": [ 649 | "for n, category in enumerate(data):\n", 650 | "# if n > 1:\n", 651 | "# break\n", 652 | " txtfile = '../' + dirMap[category] + '.txt'\n", 653 | " if not os.path.exists(txtfile):\n", 654 | " print(category)" 655 | ] 656 | }, 657 | { 658 | "cell_type": "code", 659 | "execution_count": 9, 660 | "metadata": {}, 661 | "outputs": [], 662 | "source": [ 663 | "newFolderMap = {}\n", 664 | "for u, v in folderMap.items():\n", 665 | " newFolderMap[v] = u\n", 666 | " \n" 667 | ] 668 | }, 669 | { 670 | "cell_type": "code", 671 | "execution_count": 21, 672 | "metadata": {}, 673 | "outputs": [], 674 | "source": [ 675 | "for i in data: \n", 676 | " if newFolderMap.get(i,0) == 0:\n", 677 | " print(i)" 678 | ] 679 | }, 680 | { 681 | "cell_type": "code", 682 | "execution_count": 22, 683 | "metadata": {}, 684 | "outputs": [ 685 | { 686 | "data": { 687 | "text/plain": [ 688 | "'Wok'" 689 | ] 690 | }, 691 | "execution_count": 22, 692 | "metadata": {}, 693 | "output_type": "execute_result" 694 | } 695 | ], 696 | "source": [ 697 | "newFolderMap[i]\n", 698 | "i" 699 | ] 700 | }, 701 | { 702 | "cell_type": "code", 703 | "execution_count": 24, 704 | "metadata": {}, 705 | "outputs": [ 706 | { 707 | "data": { 708 | "text/plain": [ 709 | "'alarm_clock'" 710 | ] 711 | }, 712 | "execution_count": 24, 713 | "metadata": {}, 714 | "output_type": "execute_result" 715 | } 716 | ], 717 | "source": [ 718 | "newFolderMap.get('Alarm clock',0)" 719 | ] 720 | }, 721 | { 722 | "cell_type": "code", 723 | "execution_count": 35, 724 | "metadata": {}, 725 | "outputs": [ 726 | { 727 | "name": "stdout", 728 | "output_type": "stream", 729 | "text": [ 730 | "['Step stool', 'Pill organizer', 'Mouthwash', 'Honey container', 'Key chain', 'Tanktop', 'Document folder (closed)', 'Tarp', 'Webcam', 'Floss container', 'Figurine or statue', 'Tongs', 'Suitcase', 'Skateboard', 'Teabag', 'Earring', 'Wrench', 'Squeeze bottle', 'Shoelace', 'Flashlight', 'Table knife', 'Hair brush', 'Eraser (white board)', 'Tote bag', 'Trash bag', 'Peeler', 'Extension cable', 'Tweezers', 'Dress pants', 'Nut for a screw', 'Paper plates', 'Notebook', 'Ice', 'Phone (landline)', 'Hairtie', 'Laptop charger', 'Makeup', 'Multitool', 'DVD player', 'Sponge', 'Walking cane', 'Ring', 'Kettle', 'Makeup brush', 'Placemat', 'Notepad', 'Deodorant', 'Playing cards', 'Spray bottle', 'Loofah', 'Napkin', 'Nightstand', 'Standing lamp', 'Plastic cup', 'Walker', 'Throw pillow', 'Tape measure', 'Nail clippers', 'Paint can', 'Oven mitts', 'Rock', 'Usb flash drive', 'Poster', 'Toothpaste', 'Video Camera', 'Rolling pin', 'Slipper', 'Flour container', 'Pepper shaker', 'Soap bar', 'Drinking straw', 'Tablecloth', 'Jam', 'Dress shoe (women)', 'Thermometer', 'Trophy', 'Nail file', 'Egg carton', 'Ironing board', 'Tissue', 'Dress shirt', 'Sewing kit', 'Removable blade', 'Drawer (open)', 'Fork', 'Scarf', 'Whisk', 'Ziploc bag', 'Mouse pad', 'Paper bag', 'Photograph (printed)', 'Razor', 'Night light', 'Marker', 'Shampoo bottle', 'Dish soap', 'Receipt', 'Lettuce', 'Dust pan', 'Drying rack for clothes', 'Raincoat', 'Toothbrush', 'Usb cable', 'Shorts', 'Tape / duct tape', 'Stopper (sink/tub)', 'Ice cube tray', 'Egg', 'Leggings', 'Travel case', 'Scrub brush', 'Tablet / iPad', 'Milk', 'Stapler', 'Thermos', 'Handbag', 'Spoon', 'Light bulb', 'Rake', 'Dog bed', 'Nail polish', 'Squeegee', 'Plastic wrap', 'Earbuds', 'Power cable', 'First aid kit', 'Wine glass', 'Ribbon', 'Water filter', 'Hand mirror', 'Newspaper', 'Toy', 'Tomato', 'Power bar', 'Headphones (over ear)', 'Scissors', 'Glue container', 'Magazine', 'Pencil', 'Eyeglasses', 'Sugar container']\n" 731 | ] 732 | } 733 | ], 734 | "source": [ 735 | "# len(set(newFolderMap.keys()) - set(data))\n", 736 | "to_do = []\n", 737 | "for j in set(newFolderMap.keys()) - set(data):\n", 738 | " txtfile = '../' + dirMap[j] + '.txt'\n", 739 | " if not os.path.exists(txtfile):\n", 740 | " to_do.append(j)\n", 741 | " \n", 742 | "print(to_do)" 743 | ] 744 | }, 745 | { 746 | "cell_type": "code", 747 | "execution_count": 36, 748 | "metadata": {}, 749 | "outputs": [ 750 | { 751 | "data": { 752 | "text/plain": [ 753 | "141" 754 | ] 755 | }, 756 | "execution_count": 36, 757 | "metadata": {}, 758 | "output_type": "execute_result" 759 | } 760 | ], 761 | "source": [ 762 | "len(data)\n", 763 | "len(to_do)" 764 | ] 765 | }, 766 | { 767 | "cell_type": "code", 768 | "execution_count": 40, 769 | "metadata": {}, 770 | "outputs": [ 771 | { 772 | "data": { 773 | "text/plain": [ 774 | "0.8869204367171575" 775 | ] 776 | }, 777 | "execution_count": 40, 778 | "metadata": {}, 779 | "output_type": "execute_result" 780 | } 781 | ], 782 | "source": [ 783 | "np.exp(-.12)" 784 | ] 785 | }, 786 | { 787 | "cell_type": "code", 788 | "execution_count": 39, 789 | "metadata": {}, 790 | "outputs": [ 791 | { 792 | "data": { 793 | "text/plain": [ 794 | "0.8187307530779818" 795 | ] 796 | }, 797 | "execution_count": 39, 798 | "metadata": {}, 799 | "output_type": "execute_result" 800 | } 801 | ], 802 | "source": [ 803 | "np.exp(-.2)" 804 | ] 805 | }, 806 | { 807 | "cell_type": "code", 808 | "execution_count": null, 809 | "metadata": {}, 810 | "outputs": [], 811 | "source": [] 812 | } 813 | ], 814 | "metadata": { 815 | "kernelspec": { 816 | "display_name": "Python 3", 817 | "language": "python", 818 | "name": "python3" 819 | }, 820 | "language_info": { 821 | "codemirror_mode": { 822 | "name": "ipython", 823 | "version": 3 824 | }, 825 | "file_extension": ".py", 826 | "mimetype": "text/x-python", 827 | "name": "python", 828 | "nbconvert_exporter": "python", 829 | "pygments_lexer": "ipython3", 830 | "version": "3.7.6" 831 | } 832 | }, 833 | "nbformat": 4, 834 | "nbformat_minor": 4 835 | } 836 | -------------------------------------------------------------------------------- /code/file_check.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# checking folders\n", 8 | "## To see if there is a box for any single image" 9 | ] 10 | }, 11 | { 12 | "cell_type": "code", 13 | "execution_count": 5, 14 | "metadata": {}, 15 | "outputs": [], 16 | "source": [ 17 | "import os\n", 18 | "import json\n", 19 | "import glob\n" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 8, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "with open(\"../mappings/folder_to_objectnet_label.json\") as f:\n", 29 | " folders = json.load(f)" 30 | ] 31 | }, 32 | { 33 | "cell_type": "code", 34 | "execution_count": 91, 35 | "metadata": {}, 36 | "outputs": [ 37 | { 38 | "name": "stdout", 39 | "output_type": "stream", 40 | "text": [ 41 | "1, ../umbrella.txt, 160, lines = 160\n", 42 | "2, ../coaster.txt, 185, lines = 185\n", 43 | "3, ../winter_glove.txt, 144, lines = 144\n", 44 | "4, ../pet_food_container.txt, 151, lines = 151\n", 45 | "5, ../computer_mouse.txt, 224, lines = 224\n", 46 | "6, ../still_camera.txt, 108, lines = 108\n", 47 | "7, ../weight_scale.txt, 155, lines = 155\n", 48 | "8, ../cutting_board.txt, 179, lines = 179\n", 49 | "9, ../spatula.txt, 168, lines = 168\n", 50 | "10, ../plunger.txt, 134, lines = 134\n", 51 | "11, ../paper.txt, 225, lines = 225\n", 52 | "12, ../microwave.txt, 121, lines = 121\n", 53 | "13, ../keyboard.txt, 163, lines = 163\n", 54 | "14, ../chair.txt, 220, lines = 220\n", 55 | "15, ../mixing_salad_bowl.txt, 159, lines = 159\n", 56 | "16, ../blender.txt, 211, lines = 211\n", 57 | "17, ../sandal.txt, 267, lines = 267\n", 58 | "18, ../book_closed.txt, 267, lines = 267\n", 59 | "19, ../tv.txt, 155, lines = 155\n", 60 | "20, ../backpack.txt, 235, lines = 235\n", 61 | "21, ../jar.txt, 141, lines = 141\n", 62 | "22, ../frying_pan.txt, 189, lines = 189\n", 63 | "23, ../bench.txt, 132, lines = 132\n", 64 | "24, ../coffee_grinder.txt, 138, lines = 138\n", 65 | "25, ../orange.txt, 143, lines = 143\n", 66 | "26, ../speaker.txt, 159, lines = 159\n", 67 | "27, ../hand_towel_or_rag.txt, 178, lines = 178\n", 68 | "28, ../hat.txt, 193, lines = 193\n", 69 | "29, ../toaster.txt, 163, lines = 163\n", 70 | "30, ../match.txt, 111, lines = 111\n", 71 | "31, ../plate.txt, 187, lines = 187\n", 72 | "32, ../alarm_clock.txt, 144, lines = 144\n", 73 | "33, ../sunglasses.txt, 185, lines = 185\n", 74 | "34, ../ladle.txt, 154, lines = 154\n", 75 | "35, ../ruler.txt, 152, lines = 152\n", 76 | "36, ../blanket.txt, 243, lines = 243\n", 77 | "37, ../contact_lens_case.txt, 157, lines = 157\n", 78 | "38, ../watch.txt, 158, lines = 158\n", 79 | "39, ../trash_bin.txt, 242, lines = 242\n", 80 | "40, ../vase.txt, 156, lines = 156\n", 81 | "41, ../fan.txt, 185, lines = 185\n", 82 | "42, ../lighter.txt, 169, lines = 169\n", 83 | "43, ../bottle_cap.txt, 248, lines = 248\n", 84 | "44, ../wallet.txt, 189, lines = 189\n", 85 | "45, ../coffee_table.txt, 92, lines = 92\n", 86 | "46, ../cellphone_case.txt, 214, lines = 214\n", 87 | "47, ../canned_food.txt, 205, lines = 205\n", 88 | "48, ../letter_opener.txt, 68, lines = 68\n", 89 | "49, ../button.txt, 179, lines = 179\n", 90 | "50, ../hair_dryer.txt, 175, lines = 175\n", 91 | "51, ../bracelet.txt, 226, lines = 226\n", 92 | "52, ../portable_heater.txt, 103, lines = 103\n", 93 | "53, ../suit_jacket.txt, 95, lines = 95\n", 94 | "54, ../dress_shoe_men.txt, 168, lines = 168\n", 95 | "55, ../biscuits.txt, 199, lines = 199\n", 96 | "56, ../bicycle.txt, 147, lines = 147\n", 97 | "57, ../boots.txt, 183, lines = 183\n", 98 | "58, ../full_sized_towel.txt, 149, lines = 149\n", 99 | "59, ../pill_bottle.txt, 170, lines = 170\n", 100 | "60, ../candle.txt, 233, lines = 233\n", 101 | "61, ../calendar.txt, 177, lines = 177\n", 102 | "62, ../blouse.txt, 170, lines = 170\n", 103 | "63, ../leaf.txt, 142, lines = 142\n", 104 | "64, ../lipstick.txt, 158, lines = 158\n", 105 | "65, ../detergent.txt, 160, lines = 160\n", 106 | "66, ../mug.txt, 218, lines = 218\n", 107 | "67, ../toilet_paper_roll.txt, 204, lines = 204\n", 108 | "68, ../wok.txt, 123, lines = 123\n", 109 | "69, ../swimming_trunks.txt, 147, lines = 147\n", 110 | "70, ../clothes_hamper.txt, 163, lines = 163\n", 111 | "71, ../skirt.txt, 110, lines = 110\n", 112 | "72, ../broom.txt, 232, lines = 232\n", 113 | "73, ../brooch.txt, 144, lines = 144\n", 114 | "74, ../cork.txt, 144, lines = 144\n", 115 | "75, ../coffee_french_press.txt, 107, lines = 107\n", 116 | "76, ../hairclip.txt, 191, lines = 191\n", 117 | "77, ../pliers.txt, 171, lines = 171\n", 118 | "78, ../strainer.txt, 172, lines = 172\n", 119 | "79, ../comb.txt, 206, lines = 206\n", 120 | "80, ../water_bottle.txt, 236, lines = 236\n", 121 | "81, ../monitor.txt, 114, lines = 114\n", 122 | "82, ../box.txt, 222, lines = 222\n", 123 | "83, ../bills_money.txt, 217, lines = 217\n", 124 | "84, ../running_shoe.txt, 210, lines = 210\n", 125 | "85, ../coin_money.txt, 223, lines = 223\n", 126 | "86, ../paper_towel.txt, 164, lines = 164\n", 127 | "87, ../remote_control.txt, 183, lines = 183\n", 128 | "88, ../cheese.txt, 192, lines = 192\n", 129 | "89, ../can_opener.txt, 204, lines = 204\n", 130 | "90, ../shovel.txt, 81, lines = 81\n", 131 | "91, ../paintbrush.txt, 157, lines = 157\n", 132 | "92, ../tennis_racket.txt, 81, lines = 81\n", 133 | "93, ../battery.txt, 237, lines = 237\n", 134 | "94, ../stuffed_animal.txt, 191, lines = 191\n", 135 | "95, ../jeans.txt, 172, lines = 172\n", 136 | "96, ../air_freshener.txt, 185, lines = 185\n", 137 | "97, ../belt.txt, 273, lines = 273\n", 138 | "98, ../clothes_hanger.txt, 207, lines = 207\n", 139 | "99, ../bookend.txt, 86, lines = 86\n", 140 | "100, ../sweater.txt, 161, lines = 161\n", 141 | "101, ../sock.txt, 182, lines = 182\n", 142 | "102, ../cellphone_charger.txt, 240, lines = 240\n", 143 | "103, ../banana.txt, 235, lines = 235\n", 144 | "104, ../printer.txt, 125, lines = 125\n", 145 | "105, ../paperclip.txt, 139, lines = 139\n", 146 | "106, ../cooking_oil_bottle.txt, 179, lines = 179\n", 147 | "107, ../bread_loaf.txt, 221, lines = 221\n", 148 | "108, ../pillow.txt, 196, lines = 196\n", 149 | "109, ../drinking_cup.txt, 159, lines = 159\n", 150 | "110, ../envelope.txt, 284, lines = 284\n", 151 | "111, ../chopstick.txt, 171, lines = 171\n", 152 | "112, ../t-shirt.txt, 170, lines = 170\n", 153 | "113, ../padlock.txt, 164, lines = 164\n", 154 | "114, ../chess_piece.txt, 147, lines = 147\n", 155 | "115, ../cereal.txt, 205, lines = 205\n", 156 | "116, ../teapot.txt, 113, lines = 113\n", 157 | "117, ../board_game.txt, 201, lines = 201\n", 158 | "118, ../butchers_knife.txt, 195, lines = 195\n", 159 | "119, ../soup_bowl.txt, 161, lines = 161\n", 160 | "120, ../beer_bottle.txt, 219, lines = 219\n", 161 | "121, ../combination_lock.txt, 144, lines = 144\n", 162 | "122, ../nail_fastener.txt, 87, lines = 87\n", 163 | "123, ../bucket.txt, 265, lines = 265\n", 164 | "124, ../binder_closed.txt, 183, lines = 183\n", 165 | "125, ../baseball_glove.txt, 139, lines = 139\n", 166 | "126, ../coffee_beans.txt, 125, lines = 125\n", 167 | "127, ../drill.txt, 150, lines = 150\n", 168 | "128, ../sleeping_bag.txt, 55, lines = 55\n", 169 | "129, ../plastic_bag.txt, 166, lines = 166\n", 170 | "130, ../doormat.txt, 168, lines = 168\n", 171 | "131, ../bread_knife.txt, 135, lines = 135\n", 172 | "132, ../bathrobe.txt, 172, lines = 172\n", 173 | "133, ../soap_dispenser.txt, 163, lines = 163\n", 174 | "134, ../pen.txt, 189, lines = 189\n", 175 | "135, ../wheel.txt, 122, lines = 122\n", 176 | "136, ../dress.txt, 113, lines = 113\n", 177 | "137, ../helmet.txt, 142, lines = 142\n", 178 | "138, ../lemon.txt, 148, lines = 148\n", 179 | "139, ../hammer.txt, 144, lines = 144\n", 180 | "140, ../lampshade.txt, 123, lines = 123\n", 181 | "141, ../salt_shaker.txt, 153, lines = 153\n", 182 | "142, ../vacuum_cleaner.txt, 148, lines = 148\n", 183 | "143, ../iron_for_clothes.txt, 173, lines = 173\n", 184 | "144, ../laptop_open.txt, 153, lines = 153\n", 185 | "145, ../coffee_machine.txt, 161, lines = 161\n", 186 | "146, ../tie.txt, 131, lines = 131\n", 187 | "147, ../cd_case.txt, 212, lines = 212\n", 188 | "148, ../baseball_bat.txt, 120, lines = 120\n", 189 | "149, ../bottle_opener.txt, 238, lines = 238\n", 190 | "150, ../briefcase.txt, 151, lines = 151\n", 191 | "151, ../baking_sheet.txt, 166, lines = 166\n", 192 | "152, ../screw.txt, 203, lines = 203\n", 193 | "153, ../pitcher.txt, 153, lines = 153\n", 194 | "154, ../necklace.txt, 171, lines = 171\n", 195 | "155, ../band_aid.txt, 244, lines = 244\n", 196 | "156, ../measuring_cup.txt, 155, lines = 155\n", 197 | "157, ../weight_exercise.txt, 142, lines = 142\n", 198 | "158, ../bike_pump.txt, 157, lines = 157\n", 199 | "159, ../bottle_stopper.txt, 124, lines = 124\n", 200 | "160, ../chocolate.txt, 175, lines = 175\n", 201 | "161, ../safety_pin.txt, 176, lines = 176\n", 202 | "162, ../butter.txt, 222, lines = 222\n", 203 | "163, ../cellphone.txt, 231, lines = 231\n", 204 | "164, ../drying_rack_for_dishes.txt, 55, lines = 55\n", 205 | "165, ../tray.txt, 160, lines = 160\n", 206 | "166, ../wine_bottle.txt, 156, lines = 156\n", 207 | "167, ../whistle.txt, 109, lines = 109\n", 208 | "168, ../desk_lamp.txt, 200, lines = 200\n", 209 | "169, ../bed_sheet.txt, 194, lines = 194\n", 210 | "170, ../beer_can.txt, 191, lines = 191\n", 211 | "171, ../pop_can.txt, 141, lines = 141\n", 212 | "172, ../basket.txt, 181, lines = 181\n" 213 | ] 214 | } 215 | ], 216 | "source": [ 217 | "count = 1\n", 218 | "for f in folders:\n", 219 | " txtfile = '../' + f + '.txt'\n", 220 | " if not os.path.exists(txtfile):\n", 221 | " continue\n", 222 | " \n", 223 | " with open(txtfile) as ff:\n", 224 | " boxes = [line.strip() for line in ff.readlines() if len(line)>10] # excluding the last empty line \n", 225 | " files = set([x.split(' ')[0] for x in boxes]) \n", 226 | " \n", 227 | " imgs = glob.glob('../images/' + f + '/*.png') \n", 228 | " ims = [os.path.basename(x) for x in imgs]\n", 229 | "\n", 230 | " print(f\"{count}, {txtfile}, {len(ims)}, lines = {len(boxes)}\")\n", 231 | " \n", 232 | " assert len(imgs) == len(boxes), \"Oh no! does not match!\"\n", 233 | "\n", 234 | " if len(ims) > len(boxes):\n", 235 | " print(set(ims) - set(files))\n", 236 | " \n", 237 | " count += 1 \n", 238 | " \n", 239 | " " 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": 106, 245 | "metadata": {}, 246 | "outputs": [ 247 | { 248 | "data": { 249 | "text/plain": [ 250 | "'8bdb29a78e6242f.png'" 251 | ] 252 | }, 253 | "execution_count": 106, 254 | "metadata": {}, 255 | "output_type": "execute_result" 256 | } 257 | ], 258 | "source": [ 259 | "boxes[0].split(' ')[0]\n", 260 | "# list(boxes[0].strip(' '))\n", 261 | "set(['ss', 'ss'])\n", 262 | "# files\n", 263 | "'../' + f + '/*.png'\n", 264 | "os.path.basename(imgs[0])\n", 265 | "# set(boxes)" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": null, 271 | "metadata": {}, 272 | "outputs": [], 273 | "source": [] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": null, 278 | "metadata": {}, 279 | "outputs": [], 280 | "source": [] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 88, 292 | "metadata": {}, 293 | "outputs": [], 294 | "source": [ 295 | "dirMap = {}\n", 296 | "for u, v in folders.items():\n", 297 | " dirMap[v]= u" 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "execution_count": 46, 303 | "metadata": {}, 304 | "outputs": [], 305 | "source": [ 306 | "with open(\"../mappings/folder_to_objectnet_label.json\") as f:\n", 307 | " folders = json.load(f)\n", 308 | "\n", 309 | " \n", 310 | "from overlapped_classes import overlapped_classes" 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": 96, 316 | "metadata": {}, 317 | "outputs": [ 318 | { 319 | "name": "stdout", 320 | "output_type": "stream", 321 | "text": [ 322 | "1, ../umbrella.txt, 160, lines = 160\n", 323 | "2, ../winter_glove.txt, 144, lines = 144\n", 324 | "3, ../computer_mouse.txt, 224, lines = 224\n", 325 | "4, ../still_camera.txt, 108, lines = 108\n", 326 | "5, ../weight_scale.txt, 155, lines = 155\n", 327 | "6, ../spatula.txt, 168, lines = 168\n", 328 | "7, ../plunger.txt, 134, lines = 134\n", 329 | "8, ../microwave.txt, 121, lines = 121\n", 330 | "9, ../keyboard.txt, 163, lines = 163\n", 331 | "10, ../chair.txt, 220, lines = 220\n", 332 | "11, ../mixing_salad_bowl.txt, 159, lines = 159\n", 333 | "12, ../sandal.txt, 267, lines = 267\n", 334 | "13, ../tv.txt, 155, lines = 155\n", 335 | "14, ../backpack.txt, 235, lines = 235\n", 336 | "15, ../frying_pan.txt, 189, lines = 189\n", 337 | "16, ../bench.txt, 132, lines = 132\n", 338 | "17, ../orange.txt, 143, lines = 143\n", 339 | "18, ../speaker.txt, 159, lines = 159\n", 340 | "19, ../hand_towel_or_rag.txt, 178, lines = 178\n", 341 | "20, ../toaster.txt, 163, lines = 163\n", 342 | "21, ../match.txt, 111, lines = 111\n", 343 | "22, ../plate.txt, 187, lines = 187\n", 344 | "23, ../alarm_clock.txt, 144, lines = 144\n", 345 | "24, ../sunglasses.txt, 185, lines = 185\n", 346 | "25, ../ladle.txt, 154, lines = 154\n", 347 | "26, ../ruler.txt, 152, lines = 152\n", 348 | "27, ../watch.txt, 158, lines = 158\n", 349 | "28, ../trash_bin.txt, 242, lines = 242\n", 350 | "29, ../vase.txt, 156, lines = 156\n", 351 | "30, ../fan.txt, 185, lines = 185\n", 352 | "31, ../lighter.txt, 169, lines = 169\n", 353 | "32, ../bottle_cap.txt, 248, lines = 248\n", 354 | "33, ../wallet.txt, 189, lines = 189\n", 355 | "34, ../letter_opener.txt, 68, lines = 68\n", 356 | "35, ../hair_dryer.txt, 175, lines = 175\n", 357 | "36, ../portable_heater.txt, 103, lines = 103\n", 358 | "37, ../suit_jacket.txt, 95, lines = 95\n", 359 | "38, ../dress_shoe_men.txt, 168, lines = 168\n", 360 | "39, ../bicycle.txt, 147, lines = 147\n", 361 | "40, ../full_sized_towel.txt, 149, lines = 149\n", 362 | "41, ../pill_bottle.txt, 170, lines = 170\n", 363 | "42, ../candle.txt, 233, lines = 233\n", 364 | "43, ../lipstick.txt, 158, lines = 158\n", 365 | "44, ../mug.txt, 218, lines = 218\n", 366 | "45, ../toilet_paper_roll.txt, 204, lines = 204\n", 367 | "46, ../wok.txt, 123, lines = 123\n", 368 | "47, ../swimming_trunks.txt, 147, lines = 147\n", 369 | "48, ../clothes_hamper.txt, 163, lines = 163\n", 370 | "49, ../skirt.txt, 110, lines = 110\n", 371 | "50, ../broom.txt, 232, lines = 232\n", 372 | "51, ../coffee_french_press.txt, 107, lines = 107\n", 373 | "52, ../strainer.txt, 172, lines = 172\n", 374 | "53, ../water_bottle.txt, 236, lines = 236\n", 375 | "54, ../monitor.txt, 114, lines = 114\n", 376 | "55, ../running_shoe.txt, 210, lines = 210\n", 377 | "56, ../paper_towel.txt, 164, lines = 164\n", 378 | "57, ../remote_control.txt, 183, lines = 183\n", 379 | "58, ../can_opener.txt, 204, lines = 204\n", 380 | "59, ../shovel.txt, 81, lines = 81\n", 381 | "60, ../paintbrush.txt, 157, lines = 157\n", 382 | "61, ../tennis_racket.txt, 81, lines = 81\n", 383 | "62, ../stuffed_animal.txt, 191, lines = 191\n", 384 | "63, ../jeans.txt, 172, lines = 172\n", 385 | "64, ../sweater.txt, 161, lines = 161\n", 386 | "65, ../sock.txt, 182, lines = 182\n", 387 | "66, ../banana.txt, 235, lines = 235\n", 388 | "67, ../printer.txt, 125, lines = 125\n", 389 | "68, ../bread_loaf.txt, 221, lines = 221\n", 390 | "69, ../pillow.txt, 196, lines = 196\n", 391 | "70, ../drinking_cup.txt, 159, lines = 159\n", 392 | "71, ../envelope.txt, 284, lines = 284\n", 393 | "72, ../t-shirt.txt, 170, lines = 170\n", 394 | "73, ../padlock.txt, 164, lines = 164\n", 395 | "74, ../teapot.txt, 113, lines = 113\n", 396 | "75, ../butchers_knife.txt, 195, lines = 195\n", 397 | "76, ../soup_bowl.txt, 161, lines = 161\n", 398 | "77, ../beer_bottle.txt, 219, lines = 219\n", 399 | "78, ../combination_lock.txt, 144, lines = 144\n", 400 | "79, ../nail_fastener.txt, 87, lines = 87\n", 401 | "80, ../bucket.txt, 265, lines = 265\n", 402 | "81, ../binder_closed.txt, 183, lines = 183\n", 403 | "82, ../drill.txt, 150, lines = 150\n", 404 | "83, ../sleeping_bag.txt, 55, lines = 55\n", 405 | "84, ../plastic_bag.txt, 166, lines = 166\n", 406 | "85, ../doormat.txt, 168, lines = 168\n", 407 | "86, ../soap_dispenser.txt, 163, lines = 163\n", 408 | "87, ../pen.txt, 189, lines = 189\n", 409 | "88, ../wheel.txt, 122, lines = 122\n", 410 | "89, ../dress.txt, 113, lines = 113\n", 411 | "90, ../helmet.txt, 142, lines = 142\n", 412 | "91, ../lemon.txt, 148, lines = 148\n", 413 | "92, ../hammer.txt, 144, lines = 144\n", 414 | "93, ../lampshade.txt, 123, lines = 123\n", 415 | "94, ../salt_shaker.txt, 153, lines = 153\n", 416 | "95, ../vacuum_cleaner.txt, 148, lines = 148\n", 417 | "96, ../iron_for_clothes.txt, 173, lines = 173\n", 418 | "97, ../laptop_open.txt, 153, lines = 153\n", 419 | "98, ../tie.txt, 131, lines = 131\n", 420 | "99, ../screw.txt, 203, lines = 203\n", 421 | "100, ../pitcher.txt, 153, lines = 153\n", 422 | "101, ../necklace.txt, 171, lines = 171\n", 423 | "102, ../band_aid.txt, 244, lines = 244\n", 424 | "103, ../measuring_cup.txt, 155, lines = 155\n", 425 | "104, ../weight_exercise.txt, 142, lines = 142\n", 426 | "105, ../safety_pin.txt, 176, lines = 176\n", 427 | "106, ../cellphone.txt, 231, lines = 231\n", 428 | "107, ../drying_rack_for_dishes.txt, 55, lines = 55\n", 429 | "108, ../tray.txt, 160, lines = 160\n", 430 | "109, ../wine_bottle.txt, 156, lines = 156\n", 431 | "110, ../whistle.txt, 109, lines = 109\n", 432 | "111, ../desk_lamp.txt, 200, lines = 200\n", 433 | "112, ../pop_can.txt, 141, lines = 141\n", 434 | "113, ../basket.txt, 181, lines = 181\n" 435 | ] 436 | } 437 | ], 438 | "source": [ 439 | "total_files = []\n", 440 | "count = 1\n", 441 | "for f in folders:\n", 442 | " txtfile = '../' + f + '.txt'\n", 443 | " if not os.path.exists(txtfile):\n", 444 | " continue\n", 445 | "\n", 446 | " if not (folders[f] in overlapped_classes):\n", 447 | "# print(f)\n", 448 | " continue\n", 449 | "\n", 450 | " \n", 451 | " \n", 452 | " with open(txtfile) as ff:\n", 453 | " boxes = [line.strip() for line in ff.readlines() if len(line)>10] # excluding the last empty line \n", 454 | " files = set([x.split(' ')[0] for x in boxes]) \n", 455 | " \n", 456 | " imgs = glob.glob('../images/' + f + '/*.png') \n", 457 | " ims = [os.path.basename(x) for x in imgs]\n", 458 | " total_files.append(len(imgs))\n", 459 | "\n", 460 | " print(f\"{count}, {txtfile}, {len(ims)}, lines = {len(boxes)}\")\n", 461 | " \n", 462 | " assert len(imgs) == len(boxes), \"Oh no! does not match!\"\n", 463 | "\n", 464 | " if len(ims) > len(boxes):\n", 465 | " print(set(ims) - set(files))\n", 466 | " \n", 467 | " count += 1 \n", 468 | " \n", 469 | " " 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": 102, 475 | "metadata": {}, 476 | "outputs": [ 477 | { 478 | "data": { 479 | "text/plain": [ 480 | "164.3716814159292" 481 | ] 482 | }, 483 | "execution_count": 102, 484 | "metadata": {}, 485 | "output_type": "execute_result" 486 | } 487 | ], 488 | "source": [ 489 | "sum(total_files)/ len(total_files)" 490 | ] 491 | }, 492 | { 493 | "cell_type": "code", 494 | "execution_count": 86, 495 | "metadata": {}, 496 | "outputs": [ 497 | { 498 | "data": { 499 | "text/plain": [ 500 | "113" 501 | ] 502 | }, 503 | "execution_count": 86, 504 | "metadata": {}, 505 | "output_type": "execute_result" 506 | } 507 | ], 508 | "source": [ 509 | "overlapped_classes.sort()\n", 510 | "len(overlapped_classes)" 511 | ] 512 | }, 513 | { 514 | "cell_type": "code", 515 | "execution_count": 90, 516 | "metadata": {}, 517 | "outputs": [], 518 | "source": [ 519 | "count = 1\n", 520 | "for f in overlapped_classes:\n", 521 | " txtfile = '../' + dirMap[f] + '.txt'\n", 522 | " if not os.path.exists(txtfile):\n", 523 | " print(f)\n", 524 | " continue\n" 525 | ] 526 | }, 527 | { 528 | "cell_type": "code", 529 | "execution_count": null, 530 | "metadata": {}, 531 | "outputs": [], 532 | "source": [] 533 | }, 534 | { 535 | "cell_type": "code", 536 | "execution_count": 70, 537 | "metadata": {}, 538 | "outputs": [ 539 | { 540 | "data": { 541 | "text/plain": [ 542 | "['Alarm clock',\n", 543 | " 'Backpack',\n", 544 | " 'Banana',\n", 545 | " 'Band Aid',\n", 546 | " 'Basket',\n", 547 | " 'Bath towel',\n", 548 | " 'Beer bottle',\n", 549 | " 'Bench',\n", 550 | " 'Bicycle',\n", 551 | " 'Binder (closed)',\n", 552 | " 'Bottle cap',\n", 553 | " 'Bread loaf',\n", 554 | " 'Broom',\n", 555 | " 'Bucket',\n", 556 | " \"Butcher's knife\",\n", 557 | " 'Can opener',\n", 558 | " 'Candle',\n", 559 | " 'Cellphone',\n", 560 | " 'Chair',\n", 561 | " 'Clothes hamper',\n", 562 | " 'Combination lock',\n", 563 | " 'Computer mouse',\n", 564 | " 'Desk lamp',\n", 565 | " 'Dishrag or hand towel',\n", 566 | " 'Doormat',\n", 567 | " 'Dress',\n", 568 | " 'Dress shoe (men)',\n", 569 | " 'Drill',\n", 570 | " 'Drinking Cup',\n", 571 | " 'Drying rack for plates',\n", 572 | " 'Envelope',\n", 573 | " 'Fan',\n", 574 | " 'Coffee/French press',\n", 575 | " 'Frying pan',\n", 576 | " 'Hair dryer',\n", 577 | " 'Hammer',\n", 578 | " 'Helmet',\n", 579 | " 'Iron (for clothes)',\n", 580 | " 'Jeans',\n", 581 | " 'Keyboard',\n", 582 | " 'Ladle',\n", 583 | " 'Lampshade',\n", 584 | " 'Laptop (open)',\n", 585 | " 'Lemon',\n", 586 | " 'Letter opener',\n", 587 | " 'Lighter',\n", 588 | " 'Lipstick',\n", 589 | " 'Match',\n", 590 | " 'Measuring cup',\n", 591 | " 'Microwave',\n", 592 | " 'Mixing / Salad Bowl',\n", 593 | " 'Monitor',\n", 594 | " 'Mug',\n", 595 | " 'Nail (fastener)',\n", 596 | " 'Necklace',\n", 597 | " 'Orange',\n", 598 | " 'Padlock',\n", 599 | " 'Paintbrush',\n", 600 | " 'Paper towel',\n", 601 | " 'Pen',\n", 602 | " 'Pill bottle',\n", 603 | " 'Pillow',\n", 604 | " 'Pitcher',\n", 605 | " 'Plastic bag',\n", 606 | " 'Plate',\n", 607 | " 'Plunger',\n", 608 | " 'Pop can',\n", 609 | " 'Portable heater',\n", 610 | " 'Printer',\n", 611 | " 'Remote control',\n", 612 | " 'Ruler',\n", 613 | " 'Running shoe',\n", 614 | " 'Safety pin',\n", 615 | " 'Salt shaker',\n", 616 | " 'Sandal',\n", 617 | " 'Screw',\n", 618 | " 'Shovel',\n", 619 | " 'Skirt',\n", 620 | " 'Sleeping bag',\n", 621 | " 'Soap dispenser',\n", 622 | " 'Sock',\n", 623 | " 'Soup Bowl',\n", 624 | " 'Spatula',\n", 625 | " 'Speaker',\n", 626 | " 'Still Camera',\n", 627 | " 'Strainer',\n", 628 | " 'Stuffed animal',\n", 629 | " 'Suit jacket',\n", 630 | " 'Sunglasses',\n", 631 | " 'Sweater',\n", 632 | " 'Swimming trunks',\n", 633 | " 'T-shirt',\n", 634 | " 'TV',\n", 635 | " 'Teapot',\n", 636 | " 'Tennis racket',\n", 637 | " 'Tie',\n", 638 | " 'Toaster',\n", 639 | " 'Toilet paper roll',\n", 640 | " 'Trash bin',\n", 641 | " 'Tray',\n", 642 | " 'Umbrella',\n", 643 | " 'Vacuum cleaner',\n", 644 | " 'Vase',\n", 645 | " 'Wallet',\n", 646 | " 'Watch',\n", 647 | " 'Water bottle',\n", 648 | " 'Weight (exercise)',\n", 649 | " 'Weight scale',\n", 650 | " 'Wheel',\n", 651 | " 'Whistle',\n", 652 | " 'Wine bottle',\n", 653 | " 'Winter glove',\n", 654 | " 'Wok']" 655 | ] 656 | }, 657 | "execution_count": 70, 658 | "metadata": {}, 659 | "output_type": "execute_result" 660 | } 661 | ], 662 | "source": [ 663 | "overlapped_classes" 664 | ] 665 | }, 666 | { 667 | "cell_type": "code", 668 | "execution_count": null, 669 | "metadata": {}, 670 | "outputs": [], 671 | "source": [] 672 | } 673 | ], 674 | "metadata": { 675 | "kernelspec": { 676 | "display_name": "Python 3", 677 | "language": "python", 678 | "name": "python3" 679 | }, 680 | "language_info": { 681 | "codemirror_mode": { 682 | "name": "ipython", 683 | "version": 3 684 | }, 685 | "file_extension": ".py", 686 | "mimetype": "text/x-python", 687 | "name": "python", 688 | "nbconvert_exporter": "python", 689 | "pygments_lexer": "ipython3", 690 | "version": "3.7.6" 691 | } 692 | }, 693 | "nbformat": 4, 694 | "nbformat_minor": 4 695 | } 696 | -------------------------------------------------------------------------------- /code/imagenet_classes.txt: -------------------------------------------------------------------------------- 1 | tench, Tinca tinca 2 | goldfish, Carassius auratus 3 | great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias 4 | tiger shark, Galeocerdo cuvieri 5 | hammerhead, hammerhead shark 6 | electric ray, crampfish, numbfish, torpedo 7 | stingray 8 | cock 9 | hen 10 | ostrich, Struthio camelus 11 | brambling, Fringilla montifringilla 12 | goldfinch, Carduelis carduelis 13 | house finch, linnet, Carpodacus mexicanus 14 | junco, snowbird 15 | indigo bunting, indigo finch, indigo bird, Passerina cyanea 16 | robin, American robin, Turdus migratorius 17 | bulbul 18 | jay 19 | magpie 20 | chickadee 21 | water ouzel, dipper 22 | kite 23 | bald eagle, American eagle, Haliaeetus leucocephalus 24 | vulture 25 | great grey owl, great gray owl, Strix nebulosa 26 | European fire salamander, Salamandra salamandra 27 | common newt, Triturus vulgaris 28 | eft 29 | spotted salamander, Ambystoma maculatum 30 | axolotl, mud puppy, Ambystoma mexicanum 31 | bullfrog, Rana catesbeiana 32 | tree frog, tree-frog 33 | tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui 34 | loggerhead, loggerhead turtle, Caretta caretta 35 | leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea 36 | mud turtle 37 | terrapin 38 | box turtle, box tortoise 39 | banded gecko 40 | common iguana, iguana, Iguana iguana 41 | American chameleon, anole, Anolis carolinensis 42 | whiptail, whiptail lizard 43 | agama 44 | frilled lizard, Chlamydosaurus kingi 45 | alligator lizard 46 | Gila monster, Heloderma suspectum 47 | green lizard, Lacerta viridis 48 | African chameleon, Chamaeleo chamaeleon 49 | Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis 50 | African crocodile, Nile crocodile, Crocodylus niloticus 51 | American alligator, Alligator mississipiensis 52 | triceratops 53 | thunder snake, worm snake, Carphophis amoenus 54 | ringneck snake, ring-necked snake, ring snake 55 | hognose snake, puff adder, sand viper 56 | green snake, grass snake 57 | king snake, kingsnake 58 | garter snake, grass snake 59 | water snake 60 | vine snake 61 | night snake, Hypsiglena torquata 62 | boa constrictor, Constrictor constrictor 63 | rock python, rock snake, Python sebae 64 | Indian cobra, Naja naja 65 | green mamba 66 | sea snake 67 | horned viper, cerastes, sand viper, horned asp, Cerastes cornutus 68 | diamondback, diamondback rattlesnake, Crotalus adamanteus 69 | sidewinder, horned rattlesnake, Crotalus cerastes 70 | trilobite 71 | harvestman, daddy longlegs, Phalangium opilio 72 | scorpion 73 | black and gold garden spider, Argiope aurantia 74 | barn spider, Araneus cavaticus 75 | garden spider, Aranea diademata 76 | black widow, Latrodectus mactans 77 | tarantula 78 | wolf spider, hunting spider 79 | tick 80 | centipede 81 | black grouse 82 | ptarmigan 83 | ruffed grouse, partridge, Bonasa umbellus 84 | prairie chicken, prairie grouse, prairie fowl 85 | peacock 86 | quail 87 | partridge 88 | African grey, African gray, Psittacus erithacus 89 | macaw 90 | sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita 91 | lorikeet 92 | coucal 93 | bee eater 94 | hornbill 95 | hummingbird 96 | jacamar 97 | toucan 98 | drake 99 | red-breasted merganser, Mergus serrator 100 | goose 101 | black swan, Cygnus atratus 102 | tusker 103 | echidna, spiny anteater, anteater 104 | platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus 105 | wallaby, brush kangaroo 106 | koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus 107 | wombat 108 | jellyfish 109 | sea anemone, anemone 110 | brain coral 111 | flatworm, platyhelminth 112 | nematode, nematode worm, roundworm 113 | conch 114 | snail 115 | slug 116 | sea slug, nudibranch 117 | chiton, coat-of-mail shell, sea cradle, polyplacophore 118 | chambered nautilus, pearly nautilus, nautilus 119 | Dungeness crab, Cancer magister 120 | rock crab, Cancer irroratus 121 | fiddler crab 122 | king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica 123 | American lobster, Northern lobster, Maine lobster, Homarus americanus 124 | spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish 125 | crayfish, crawfish, crawdad, crawdaddy 126 | hermit crab 127 | isopod 128 | white stork, Ciconia ciconia 129 | black stork, Ciconia nigra 130 | spoonbill 131 | flamingo 132 | little blue heron, Egretta caerulea 133 | American egret, great white heron, Egretta albus 134 | bittern 135 | crane 136 | limpkin, Aramus pictus 137 | European gallinule, Porphyrio porphyrio 138 | American coot, marsh hen, mud hen, water hen, Fulica americana 139 | bustard 140 | ruddy turnstone, Arenaria interpres 141 | red-backed sandpiper, dunlin, Erolia alpina 142 | redshank, Tringa totanus 143 | dowitcher 144 | oystercatcher, oyster catcher 145 | pelican 146 | king penguin, Aptenodytes patagonica 147 | albatross, mollymawk 148 | grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus 149 | killer whale, killer, orca, grampus, sea wolf, Orcinus orca 150 | dugong, Dugong dugon 151 | sea lion 152 | Chihuahua 153 | Japanese spaniel 154 | Maltese dog, Maltese terrier, Maltese 155 | Pekinese, Pekingese, Peke 156 | Shih-Tzu 157 | Blenheim spaniel 158 | papillon 159 | toy terrier 160 | Rhodesian ridgeback 161 | Afghan hound, Afghan 162 | basset, basset hound 163 | beagle 164 | bloodhound, sleuthhound 165 | bluetick 166 | black-and-tan coonhound 167 | Walker hound, Walker foxhound 168 | English foxhound 169 | redbone 170 | borzoi, Russian wolfhound 171 | Irish wolfhound 172 | Italian greyhound 173 | whippet 174 | Ibizan hound, Ibizan Podenco 175 | Norwegian elkhound, elkhound 176 | otterhound, otter hound 177 | Saluki, gazelle hound 178 | Scottish deerhound, deerhound 179 | Weimaraner 180 | Staffordshire bullterrier, Staffordshire bull terrier 181 | American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier 182 | Bedlington terrier 183 | Border terrier 184 | Kerry blue terrier 185 | Irish terrier 186 | Norfolk terrier 187 | Norwich terrier 188 | Yorkshire terrier 189 | wire-haired fox terrier 190 | Lakeland terrier 191 | Sealyham terrier, Sealyham 192 | Airedale, Airedale terrier 193 | cairn, cairn terrier 194 | Australian terrier 195 | Dandie Dinmont, Dandie Dinmont terrier 196 | Boston bull, Boston terrier 197 | miniature schnauzer 198 | giant schnauzer 199 | standard schnauzer 200 | Scotch terrier, Scottish terrier, Scottie 201 | Tibetan terrier, chrysanthemum dog 202 | silky terrier, Sydney silky 203 | soft-coated wheaten terrier 204 | West Highland white terrier 205 | Lhasa, Lhasa apso 206 | flat-coated retriever 207 | curly-coated retriever 208 | golden retriever 209 | Labrador retriever 210 | Chesapeake Bay retriever 211 | German short-haired pointer 212 | vizsla, Hungarian pointer 213 | English setter 214 | Irish setter, red setter 215 | Gordon setter 216 | Brittany spaniel 217 | clumber, clumber spaniel 218 | English springer, English springer spaniel 219 | Welsh springer spaniel 220 | cocker spaniel, English cocker spaniel, cocker 221 | Sussex spaniel 222 | Irish water spaniel 223 | kuvasz 224 | schipperke 225 | groenendael 226 | malinois 227 | briard 228 | kelpie 229 | komondor 230 | Old English sheepdog, bobtail 231 | Shetland sheepdog, Shetland sheep dog, Shetland 232 | collie 233 | Border collie 234 | Bouvier des Flandres, Bouviers des Flandres 235 | Rottweiler 236 | German shepherd, German shepherd dog, German police dog, alsatian 237 | Doberman, Doberman pinscher 238 | miniature pinscher 239 | Greater Swiss Mountain dog 240 | Bernese mountain dog 241 | Appenzeller 242 | EntleBucher 243 | boxer 244 | bull mastiff 245 | Tibetan mastiff 246 | French bulldog 247 | Great Dane 248 | Saint Bernard, St Bernard 249 | Eskimo dog, husky 250 | malamute, malemute, Alaskan malamute 251 | Siberian husky 252 | dalmatian, coach dog, carriage dog 253 | affenpinscher, monkey pinscher, monkey dog 254 | basenji 255 | pug, pug-dog 256 | Leonberg 257 | Newfoundland, Newfoundland dog 258 | Great Pyrenees 259 | Samoyed, Samoyede 260 | Pomeranian 261 | chow, chow chow 262 | keeshond 263 | Brabancon griffon 264 | Pembroke, Pembroke Welsh corgi 265 | Cardigan, Cardigan Welsh corgi 266 | toy poodle 267 | miniature poodle 268 | standard poodle 269 | Mexican hairless 270 | timber wolf, grey wolf, gray wolf, Canis lupus 271 | white wolf, Arctic wolf, Canis lupus tundrarum 272 | red wolf, maned wolf, Canis rufus, Canis niger 273 | coyote, prairie wolf, brush wolf, Canis latrans 274 | dingo, warrigal, warragal, Canis dingo 275 | dhole, Cuon alpinus 276 | African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus 277 | hyena, hyaena 278 | red fox, Vulpes vulpes 279 | kit fox, Vulpes macrotis 280 | Arctic fox, white fox, Alopex lagopus 281 | grey fox, gray fox, Urocyon cinereoargenteus 282 | tabby, tabby cat 283 | tiger cat 284 | Persian cat 285 | Siamese cat, Siamese 286 | Egyptian cat 287 | cougar, puma, catamount, mountain lion, painter, panther, Felis concolor 288 | lynx, catamount 289 | leopard, Panthera pardus 290 | snow leopard, ounce, Panthera uncia 291 | jaguar, panther, Panthera onca, Felis onca 292 | lion, king of beasts, Panthera leo 293 | tiger, Panthera tigris 294 | cheetah, chetah, Acinonyx jubatus 295 | brown bear, bruin, Ursus arctos 296 | American black bear, black bear, Ursus americanus, Euarctos americanus 297 | ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus 298 | sloth bear, Melursus ursinus, Ursus ursinus 299 | mongoose 300 | meerkat, mierkat 301 | tiger beetle 302 | ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle 303 | ground beetle, carabid beetle 304 | long-horned beetle, longicorn, longicorn beetle 305 | leaf beetle, chrysomelid 306 | dung beetle 307 | rhinoceros beetle 308 | weevil 309 | fly 310 | bee 311 | ant, emmet, pismire 312 | grasshopper, hopper 313 | cricket 314 | walking stick, walkingstick, stick insect 315 | cockroach, roach 316 | mantis, mantid 317 | cicada, cicala 318 | leafhopper 319 | lacewing, lacewing fly 320 | dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk 321 | damselfly 322 | admiral 323 | ringlet, ringlet butterfly 324 | monarch, monarch butterfly, milkweed butterfly, Danaus plexippus 325 | cabbage butterfly 326 | sulphur butterfly, sulfur butterfly 327 | lycaenid, lycaenid butterfly 328 | starfish, sea star 329 | sea urchin 330 | sea cucumber, holothurian 331 | wood rabbit, cottontail, cottontail rabbit 332 | hare 333 | Angora, Angora rabbit 334 | hamster 335 | porcupine, hedgehog 336 | fox squirrel, eastern fox squirrel, Sciurus niger 337 | marmot 338 | beaver 339 | guinea pig, Cavia cobaya 340 | sorrel 341 | zebra 342 | hog, pig, grunter, squealer, Sus scrofa 343 | wild boar, boar, Sus scrofa 344 | warthog 345 | hippopotamus, hippo, river horse, Hippopotamus amphibius 346 | ox 347 | water buffalo, water ox, Asiatic buffalo, Bubalus bubalis 348 | bison 349 | ram, tup 350 | bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis 351 | ibex, Capra ibex 352 | hartebeest 353 | impala, Aepyceros melampus 354 | gazelle 355 | Arabian camel, dromedary, Camelus dromedarius 356 | llama 357 | weasel 358 | mink 359 | polecat, fitch, foulmart, foumart, Mustela putorius 360 | black-footed ferret, ferret, Mustela nigripes 361 | otter 362 | skunk, polecat, wood pussy 363 | badger 364 | armadillo 365 | three-toed sloth, ai, Bradypus tridactylus 366 | orangutan, orang, orangutang, Pongo pygmaeus 367 | gorilla, Gorilla gorilla 368 | chimpanzee, chimp, Pan troglodytes 369 | gibbon, Hylobates lar 370 | siamang, Hylobates syndactylus, Symphalangus syndactylus 371 | guenon, guenon monkey 372 | patas, hussar monkey, Erythrocebus patas 373 | baboon 374 | macaque 375 | langur 376 | colobus, colobus monkey 377 | proboscis monkey, Nasalis larvatus 378 | marmoset 379 | capuchin, ringtail, Cebus capucinus 380 | howler monkey, howler 381 | titi, titi monkey 382 | spider monkey, Ateles geoffroyi 383 | squirrel monkey, Saimiri sciureus 384 | Madagascar cat, ring-tailed lemur, Lemur catta 385 | indri, indris, Indri indri, Indri brevicaudatus 386 | Indian elephant, Elephas maximus 387 | African elephant, Loxodonta africana 388 | lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens 389 | giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca 390 | barracouta, snoek 391 | eel 392 | coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch 393 | rock beauty, Holocanthus tricolor 394 | anemone fish 395 | sturgeon 396 | gar, garfish, garpike, billfish, Lepisosteus osseus 397 | lionfish 398 | puffer, pufferfish, blowfish, globefish 399 | abacus 400 | abaya 401 | academic gown, academic robe, judge's robe 402 | accordion, piano accordion, squeeze box 403 | acoustic guitar 404 | aircraft carrier, carrier, flattop, attack aircraft carrier 405 | airliner 406 | airship, dirigible 407 | altar 408 | ambulance 409 | amphibian, amphibious vehicle 410 | analog clock 411 | apiary, bee house 412 | apron 413 | ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin 414 | assault rifle, assault gun 415 | backpack, back pack, knapsack, packsack, rucksack, haversack 416 | bakery, bakeshop, bakehouse 417 | balance beam, beam 418 | balloon 419 | ballpoint, ballpoint pen, ballpen, Biro 420 | Band Aid 421 | banjo 422 | bannister, banister, balustrade, balusters, handrail 423 | barbell 424 | barber chair 425 | barbershop 426 | barn 427 | barometer 428 | barrel, cask 429 | barrow, garden cart, lawn cart, wheelbarrow 430 | baseball 431 | basketball 432 | bassinet 433 | bassoon 434 | bathing cap, swimming cap 435 | bath towel 436 | bathtub, bathing tub, bath, tub 437 | beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon 438 | beacon, lighthouse, beacon light, pharos 439 | beaker 440 | bearskin, busby, shako 441 | beer bottle 442 | beer glass 443 | bell cote, bell cot 444 | bib 445 | bicycle-built-for-two, tandem bicycle, tandem 446 | bikini, two-piece 447 | binder, ring-binder 448 | binoculars, field glasses, opera glasses 449 | birdhouse 450 | boathouse 451 | bobsled, bobsleigh, bob 452 | bolo tie, bolo, bola tie, bola 453 | bonnet, poke bonnet 454 | bookcase 455 | bookshop, bookstore, bookstall 456 | bottlecap 457 | bow 458 | bow tie, bow-tie, bowtie 459 | brass, memorial tablet, plaque 460 | brassiere, bra, bandeau 461 | breakwater, groin, groyne, mole, bulwark, seawall, jetty 462 | breastplate, aegis, egis 463 | broom 464 | bucket, pail 465 | buckle 466 | bulletproof vest 467 | bullet train, bullet 468 | butcher shop, meat market 469 | cab, hack, taxi, taxicab 470 | caldron, cauldron 471 | candle, taper, wax light 472 | cannon 473 | canoe 474 | can opener, tin opener 475 | cardigan 476 | car mirror 477 | carousel, carrousel, merry-go-round, roundabout, whirligig 478 | carpenter's kit, tool kit 479 | carton 480 | car wheel 481 | cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM 482 | cassette 483 | cassette player 484 | castle 485 | catamaran 486 | CD player 487 | cello, violoncello 488 | cellular telephone, cellular phone, cellphone, cell, mobile phone 489 | chain 490 | chainlink fence 491 | chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour 492 | chain saw, chainsaw 493 | chest 494 | chiffonier, commode 495 | chime, bell, gong 496 | china cabinet, china closet 497 | Christmas stocking 498 | church, church building 499 | cinema, movie theater, movie theatre, movie house, picture palace 500 | cleaver, meat cleaver, chopper 501 | cliff dwelling 502 | cloak 503 | clog, geta, patten, sabot 504 | cocktail shaker 505 | coffee mug 506 | coffeepot 507 | coil, spiral, volute, whorl, helix 508 | combination lock 509 | computer keyboard, keypad 510 | confectionery, confectionary, candy store 511 | container ship, containership, container vessel 512 | convertible 513 | corkscrew, bottle screw 514 | cornet, horn, trumpet, trump 515 | cowboy boot 516 | cowboy hat, ten-gallon hat 517 | cradle 518 | crane 519 | crash helmet 520 | crate 521 | crib, cot 522 | Crock Pot 523 | croquet ball 524 | crutch 525 | cuirass 526 | dam, dike, dyke 527 | desk 528 | desktop computer 529 | dial telephone, dial phone 530 | diaper, nappy, napkin 531 | digital clock 532 | digital watch 533 | dining table, board 534 | dishrag, dishcloth 535 | dishwasher, dish washer, dishwashing machine 536 | disk brake, disc brake 537 | dock, dockage, docking facility 538 | dogsled, dog sled, dog sleigh 539 | dome 540 | doormat, welcome mat 541 | drilling platform, offshore rig 542 | drum, membranophone, tympan 543 | drumstick 544 | dumbbell 545 | Dutch oven 546 | electric fan, blower 547 | electric guitar 548 | electric locomotive 549 | entertainment center 550 | envelope 551 | espresso maker 552 | face powder 553 | feather boa, boa 554 | file, file cabinet, filing cabinet 555 | fireboat 556 | fire engine, fire truck 557 | fire screen, fireguard 558 | flagpole, flagstaff 559 | flute, transverse flute 560 | folding chair 561 | football helmet 562 | forklift 563 | fountain 564 | fountain pen 565 | four-poster 566 | freight car 567 | French horn, horn 568 | frying pan, frypan, skillet 569 | fur coat 570 | garbage truck, dustcart 571 | gasmask, respirator, gas helmet 572 | gas pump, gasoline pump, petrol pump, island dispenser 573 | goblet 574 | go-kart 575 | golf ball 576 | golfcart, golf cart 577 | gondola 578 | gong, tam-tam 579 | gown 580 | grand piano, grand 581 | greenhouse, nursery, glasshouse 582 | grille, radiator grille 583 | grocery store, grocery, food market, market 584 | guillotine 585 | hair slide 586 | hair spray 587 | half track 588 | hammer 589 | hamper 590 | hand blower, blow dryer, blow drier, hair dryer, hair drier 591 | hand-held computer, hand-held microcomputer 592 | handkerchief, hankie, hanky, hankey 593 | hard disc, hard disk, fixed disk 594 | harmonica, mouth organ, harp, mouth harp 595 | harp 596 | harvester, reaper 597 | hatchet 598 | holster 599 | home theater, home theatre 600 | honeycomb 601 | hook, claw 602 | hoopskirt, crinoline 603 | horizontal bar, high bar 604 | horse cart, horse-cart 605 | hourglass 606 | iPod 607 | iron, smoothing iron 608 | jack-o'-lantern 609 | jean, blue jean, denim 610 | jeep, landrover 611 | jersey, T-shirt, tee shirt 612 | jigsaw puzzle 613 | jinrikisha, ricksha, rickshaw 614 | joystick 615 | kimono 616 | knee pad 617 | knot 618 | lab coat, laboratory coat 619 | ladle 620 | lampshade, lamp shade 621 | laptop, laptop computer 622 | lawn mower, mower 623 | lens cap, lens cover 624 | letter opener, paper knife, paperknife 625 | library 626 | lifeboat 627 | lighter, light, igniter, ignitor 628 | limousine, limo 629 | liner, ocean liner 630 | lipstick, lip rouge 631 | Loafer 632 | lotion 633 | loudspeaker, speaker, speaker unit, loudspeaker system, speaker system 634 | loupe, jeweler's loupe 635 | lumbermill, sawmill 636 | magnetic compass 637 | mailbag, postbag 638 | mailbox, letter box 639 | maillot 640 | maillot, tank suit 641 | manhole cover 642 | maraca 643 | marimba, xylophone 644 | mask 645 | matchstick 646 | maypole 647 | maze, labyrinth 648 | measuring cup 649 | medicine chest, medicine cabinet 650 | megalith, megalithic structure 651 | microphone, mike 652 | microwave, microwave oven 653 | military uniform 654 | milk can 655 | minibus 656 | miniskirt, mini 657 | minivan 658 | missile 659 | mitten 660 | mixing bowl 661 | mobile home, manufactured home 662 | Model T 663 | modem 664 | monastery 665 | monitor 666 | moped 667 | mortar 668 | mortarboard 669 | mosque 670 | mosquito net 671 | motor scooter, scooter 672 | mountain bike, all-terrain bike, off-roader 673 | mountain tent 674 | mouse, computer mouse 675 | mousetrap 676 | moving van 677 | muzzle 678 | nail 679 | neck brace 680 | necklace 681 | nipple 682 | notebook, notebook computer 683 | obelisk 684 | oboe, hautboy, hautbois 685 | ocarina, sweet potato 686 | odometer, hodometer, mileometer, milometer 687 | oil filter 688 | organ, pipe organ 689 | oscilloscope, scope, cathode-ray oscilloscope, CRO 690 | overskirt 691 | oxcart 692 | oxygen mask 693 | packet 694 | paddle, boat paddle 695 | paddlewheel, paddle wheel 696 | padlock 697 | paintbrush 698 | pajama, pyjama, pj's, jammies 699 | palace 700 | panpipe, pandean pipe, syrinx 701 | paper towel 702 | parachute, chute 703 | parallel bars, bars 704 | park bench 705 | parking meter 706 | passenger car, coach, carriage 707 | patio, terrace 708 | pay-phone, pay-station 709 | pedestal, plinth, footstall 710 | pencil box, pencil case 711 | pencil sharpener 712 | perfume, essence 713 | Petri dish 714 | photocopier 715 | pick, plectrum, plectron 716 | pickelhaube 717 | picket fence, paling 718 | pickup, pickup truck 719 | pier 720 | piggy bank, penny bank 721 | pill bottle 722 | pillow 723 | ping-pong ball 724 | pinwheel 725 | pirate, pirate ship 726 | pitcher, ewer 727 | plane, carpenter's plane, woodworking plane 728 | planetarium 729 | plastic bag 730 | plate rack 731 | plow, plough 732 | plunger, plumber's helper 733 | Polaroid camera, Polaroid Land camera 734 | pole 735 | police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria 736 | poncho 737 | pool table, billiard table, snooker table 738 | pop bottle, soda bottle 739 | pot, flowerpot 740 | potter's wheel 741 | power drill 742 | prayer rug, prayer mat 743 | printer 744 | prison, prison house 745 | projectile, missile 746 | projector 747 | puck, hockey puck 748 | punching bag, punch bag, punching ball, punchball 749 | purse 750 | quill, quill pen 751 | quilt, comforter, comfort, puff 752 | racer, race car, racing car 753 | racket, racquet 754 | radiator 755 | radio, wireless 756 | radio telescope, radio reflector 757 | rain barrel 758 | recreational vehicle, RV, R.V. 759 | reel 760 | reflex camera 761 | refrigerator, icebox 762 | remote control, remote 763 | restaurant, eating house, eating place, eatery 764 | revolver, six-gun, six-shooter 765 | rifle 766 | rocking chair, rocker 767 | rotisserie 768 | rubber eraser, rubber, pencil eraser 769 | rugby ball 770 | rule, ruler 771 | running shoe 772 | safe 773 | safety pin 774 | saltshaker, salt shaker 775 | sandal 776 | sarong 777 | sax, saxophone 778 | scabbard 779 | scale, weighing machine 780 | school bus 781 | schooner 782 | scoreboard 783 | screen, CRT screen 784 | screw 785 | screwdriver 786 | seat belt, seatbelt 787 | sewing machine 788 | shield, buckler 789 | shoe shop, shoe-shop, shoe store 790 | shoji 791 | shopping basket 792 | shopping cart 793 | shovel 794 | shower cap 795 | shower curtain 796 | ski 797 | ski mask 798 | sleeping bag 799 | slide rule, slipstick 800 | sliding door 801 | slot, one-armed bandit 802 | snorkel 803 | snowmobile 804 | snowplow, snowplough 805 | soap dispenser 806 | soccer ball 807 | sock 808 | solar dish, solar collector, solar furnace 809 | sombrero 810 | soup bowl 811 | space bar 812 | space heater 813 | space shuttle 814 | spatula 815 | speedboat 816 | spider web, spider's web 817 | spindle 818 | sports car, sport car 819 | spotlight, spot 820 | stage 821 | steam locomotive 822 | steel arch bridge 823 | steel drum 824 | stethoscope 825 | stole 826 | stone wall 827 | stopwatch, stop watch 828 | stove 829 | strainer 830 | streetcar, tram, tramcar, trolley, trolley car 831 | stretcher 832 | studio couch, day bed 833 | stupa, tope 834 | submarine, pigboat, sub, U-boat 835 | suit, suit of clothes 836 | sundial 837 | sunglass 838 | sunglasses, dark glasses, shades 839 | sunscreen, sunblock, sun blocker 840 | suspension bridge 841 | swab, swob, mop 842 | sweatshirt 843 | swimming trunks, bathing trunks 844 | swing 845 | switch, electric switch, electrical switch 846 | syringe 847 | table lamp 848 | tank, army tank, armored combat vehicle, armoured combat vehicle 849 | tape player 850 | teapot 851 | teddy, teddy bear 852 | television, television system 853 | tennis ball 854 | thatch, thatched roof 855 | theater curtain, theatre curtain 856 | thimble 857 | thresher, thrasher, threshing machine 858 | throne 859 | tile roof 860 | toaster 861 | tobacco shop, tobacconist shop, tobacconist 862 | toilet seat 863 | torch 864 | totem pole 865 | tow truck, tow car, wrecker 866 | toyshop 867 | tractor 868 | trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi 869 | tray 870 | trench coat 871 | tricycle, trike, velocipede 872 | trimaran 873 | tripod 874 | triumphal arch 875 | trolleybus, trolley coach, trackless trolley 876 | trombone 877 | tub, vat 878 | turnstile 879 | typewriter keyboard 880 | umbrella 881 | unicycle, monocycle 882 | upright, upright piano 883 | vacuum, vacuum cleaner 884 | vase 885 | vault 886 | velvet 887 | vending machine 888 | vestment 889 | viaduct 890 | violin, fiddle 891 | volleyball 892 | waffle iron 893 | wall clock 894 | wallet, billfold, notecase, pocketbook 895 | wardrobe, closet, press 896 | warplane, military plane 897 | washbasin, handbasin, washbowl, lavabo, wash-hand basin 898 | washer, automatic washer, washing machine 899 | water bottle 900 | water jug 901 | water tower 902 | whiskey jug 903 | whistle 904 | wig 905 | window screen 906 | window shade 907 | Windsor tie 908 | wine bottle 909 | wing 910 | wok 911 | wooden spoon 912 | wool, woolen, woollen 913 | worm fence, snake fence, snake-rail fence, Virginia fence 914 | wreck 915 | yawl 916 | yurt 917 | web site, website, internet site, site 918 | comic book 919 | crossword puzzle, crossword 920 | street sign 921 | traffic light, traffic signal, stoplight 922 | book jacket, dust cover, dust jacket, dust wrapper 923 | menu 924 | plate 925 | guacamole 926 | consomme 927 | hot pot, hotpot 928 | trifle 929 | ice cream, icecream 930 | ice lolly, lolly, lollipop, popsicle 931 | French loaf 932 | bagel, beigel 933 | pretzel 934 | cheeseburger 935 | hotdog, hot dog, red hot 936 | mashed potato 937 | head cabbage 938 | broccoli 939 | cauliflower 940 | zucchini, courgette 941 | spaghetti squash 942 | acorn squash 943 | butternut squash 944 | cucumber, cuke 945 | artichoke, globe artichoke 946 | bell pepper 947 | cardoon 948 | mushroom 949 | Granny Smith 950 | strawberry 951 | orange 952 | lemon 953 | fig 954 | pineapple, ananas 955 | banana 956 | jackfruit, jak, jack 957 | custard apple 958 | pomegranate 959 | hay 960 | carbonara 961 | chocolate sauce, chocolate syrup 962 | dough 963 | meat loaf, meatloaf 964 | pizza, pizza pie 965 | potpie 966 | burrito 967 | red wine 968 | espresso 969 | cup 970 | eggnog 971 | alp 972 | bubble 973 | cliff, drop, drop-off 974 | coral reef 975 | geyser 976 | lakeside, lakeshore 977 | promontory, headland, head, foreland 978 | sandbar, sand bar 979 | seashore, coast, seacoast, sea-coast 980 | valley, vale 981 | volcano 982 | ballplayer, baseball player 983 | groom, bridegroom 984 | scuba diver 985 | rapeseed 986 | daisy 987 | yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum 988 | corn 989 | acorn 990 | hip, rose hip, rosehip 991 | buckeye, horse chestnut, conker 992 | coral fungus 993 | agaric 994 | gyromitra 995 | stinkhorn, carrion fungus 996 | earthstar 997 | hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa 998 | bolete 999 | ear, spike, capitulum 1000 | toilet tissue, toilet paper, bathroom tissue -------------------------------------------------------------------------------- /code/make_imagenet_64_c.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | import os 4 | from PIL import Image 5 | import os.path 6 | import time 7 | import torch 8 | import torchvision.datasets as dset 9 | import torchvision.transforms as trn 10 | import torch.utils.data as data 11 | import numpy as np 12 | 13 | from PIL import Image 14 | 15 | # /////////////// Data Loader /////////////// 16 | 17 | 18 | IMG_EXTENSIONS = ['.jpg', '.jpeg', '.png', '.ppm', '.bmp', '.pgm'] 19 | 20 | 21 | def is_image_file(filename): 22 | """Checks if a file is an image. 23 | Args: 24 | filename (string): path to a file 25 | Returns: 26 | bool: True if the filename ends with a known image extension 27 | """ 28 | filename_lower = filename.lower() 29 | return any(filename_lower.endswith(ext) for ext in IMG_EXTENSIONS) 30 | 31 | 32 | def find_classes(dir): 33 | classes = [d for d in os.listdir(dir) if os.path.isdir(os.path.join(dir, d))] 34 | classes.sort() 35 | class_to_idx = {classes[i]: i for i in range(len(classes))} 36 | return classes, class_to_idx 37 | 38 | 39 | def make_dataset(dir, class_to_idx): 40 | images = [] 41 | dir = os.path.expanduser(dir) 42 | for target in sorted(os.listdir(dir)): 43 | d = os.path.join(dir, target) 44 | if not os.path.isdir(d): 45 | continue 46 | 47 | for root, _, fnames in sorted(os.walk(d)): 48 | for fname in sorted(fnames): 49 | if is_image_file(fname): 50 | path = os.path.join(root, fname) 51 | item = (path, class_to_idx[target]) 52 | images.append(item) 53 | 54 | return images 55 | 56 | 57 | def pil_loader(path): 58 | # open path as file to avoid ResourceWarning (https://github.com/python-pillow/Pillow/issues/835) 59 | with open(path, 'rb') as f: 60 | img = Image.open(f) 61 | return img.convert('RGB') 62 | 63 | 64 | def accimage_loader(path): 65 | import accimage 66 | try: 67 | return accimage.Image(path) 68 | except IOError: 69 | # Potentially a decoding problem, fall back to PIL.Image 70 | return pil_loader(path) 71 | 72 | 73 | def default_loader(path): 74 | from torchvision import get_image_backend 75 | if get_image_backend() == 'accimage': 76 | return accimage_loader(path) 77 | else: 78 | return pil_loader(path) 79 | 80 | 81 | class DistortImageFolder(data.Dataset): 82 | def __init__(self, root, method, severity, transform=None, target_transform=None, 83 | loader=default_loader): 84 | classes, class_to_idx = find_classes(root) 85 | imgs = make_dataset(root, class_to_idx) 86 | if len(imgs) == 0: 87 | raise (RuntimeError("Found 0 images in subfolders of: " + root + "\n" + 88 | "Supported image extensions are: " + ",".join(IMG_EXTENSIONS))) 89 | 90 | self.root = root 91 | self.method = method 92 | self.severity = severity 93 | self.imgs = imgs 94 | self.classes = classes 95 | self.class_to_idx = class_to_idx 96 | self.idx_to_class = {v: k for k, v in class_to_idx.items()} 97 | self.transform = transform 98 | self.target_transform = target_transform 99 | self.loader = loader 100 | 101 | def __getitem__(self, index): 102 | path, target = self.imgs[index] 103 | img = self.loader(path) 104 | if self.transform is not None: 105 | img = self.transform(img) 106 | img = self.method(img, self.severity) 107 | if self.target_transform is not None: 108 | target = self.target_transform(target) 109 | 110 | save_path = '/share/data/lang/users/dan/ImageNet-64x64-C/' + self.method.__name__ + \ 111 | '/' + str(self.severity) + '/' + self.idx_to_class[target] 112 | 113 | if not os.path.exists(save_path): 114 | os.makedirs(save_path) 115 | 116 | save_path += path[path.rindex('/'):] 117 | 118 | Image.fromarray(np.uint8(img)).save(save_path, quality=85, optimize=True) 119 | 120 | return 0 # we do not care about returning the data 121 | 122 | def __len__(self): 123 | return len(self.imgs) 124 | 125 | 126 | # /////////////// Distortion Helpers /////////////// 127 | 128 | import skimage as sk 129 | from skimage.filters import gaussian 130 | from io import BytesIO 131 | from wand.image import Image as WandImage 132 | from wand.api import library as wandlibrary 133 | import wand.color as WandColor 134 | import ctypes 135 | from PIL import Image as PILImage 136 | import cv2 137 | from scipy.ndimage import zoom as scizoom 138 | from scipy.ndimage.interpolation import map_coordinates 139 | import warnings 140 | 141 | warnings.simplefilter("ignore", UserWarning) 142 | 143 | 144 | def auc(errs): # area under the alteration error curve 145 | area = 0 146 | for i in range(1, len(errs)): 147 | area += (errs[i] + errs[i - 1]) / 2 148 | area /= len(errs) - 1 149 | return area 150 | 151 | 152 | def disk(radius, alias_blur=0.1, dtype=np.float32): 153 | if radius <= 8: 154 | L = np.arange(-8, 8 + 1) 155 | ksize = (3, 3) 156 | else: 157 | L = np.arange(-radius, radius + 1) 158 | ksize = (5, 5) 159 | X, Y = np.meshgrid(L, L) 160 | aliased_disk = np.array((X ** 2 + Y ** 2) <= radius ** 2, dtype=dtype) 161 | aliased_disk /= np.sum(aliased_disk) 162 | 163 | # supersample disk to antialias 164 | return cv2.GaussianBlur(aliased_disk, ksize=ksize, sigmaX=alias_blur) 165 | 166 | 167 | # Tell Python about the C method 168 | wandlibrary.MagickMotionBlurImage.argtypes = (ctypes.c_void_p, # wand 169 | ctypes.c_double, # radius 170 | ctypes.c_double, # sigma 171 | ctypes.c_double) # angle 172 | 173 | 174 | # Extend wand.image.Image class to include method signature 175 | class MotionImage(WandImage): 176 | def motion_blur(self, radius=0.0, sigma=0.0, angle=0.0): 177 | wandlibrary.MagickMotionBlurImage(self.wand, radius, sigma, angle) 178 | 179 | 180 | # modification of https://github.com/FLHerne/mapgen/blob/master/diamondsquare.py 181 | def plasma_fractal(mapsize=64, wibbledecay=3): 182 | """ 183 | Generate a heightmap using diamond-square algorithm. 184 | Return square 2d array, side length 'mapsize', of floats in range 0-255. 185 | 'mapsize' must be a power of two. 186 | """ 187 | assert (mapsize & (mapsize - 1) == 0) 188 | maparray = np.empty((mapsize, mapsize), dtype=np.float_) 189 | maparray[0, 0] = 0 190 | stepsize = mapsize 191 | wibble = 100 192 | 193 | def wibbledmean(array): 194 | return array / 4 + wibble * np.random.uniform(-wibble, wibble, array.shape) 195 | 196 | def fillsquares(): 197 | """For each square of points stepsize apart, 198 | calculate middle value as mean of points + wibble""" 199 | cornerref = maparray[0:mapsize:stepsize, 0:mapsize:stepsize] 200 | squareaccum = cornerref + np.roll(cornerref, shift=-1, axis=0) 201 | squareaccum += np.roll(squareaccum, shift=-1, axis=1) 202 | maparray[stepsize // 2:mapsize:stepsize, 203 | stepsize // 2:mapsize:stepsize] = wibbledmean(squareaccum) 204 | 205 | def filldiamonds(): 206 | """For each diamond of points stepsize apart, 207 | calculate middle value as mean of points + wibble""" 208 | mapsize = maparray.shape[0] 209 | drgrid = maparray[stepsize // 2:mapsize:stepsize, stepsize // 2:mapsize:stepsize] 210 | ulgrid = maparray[0:mapsize:stepsize, 0:mapsize:stepsize] 211 | ldrsum = drgrid + np.roll(drgrid, 1, axis=0) 212 | lulsum = ulgrid + np.roll(ulgrid, -1, axis=1) 213 | ltsum = ldrsum + lulsum 214 | maparray[0:mapsize:stepsize, stepsize // 2:mapsize:stepsize] = wibbledmean(ltsum) 215 | tdrsum = drgrid + np.roll(drgrid, 1, axis=1) 216 | tulsum = ulgrid + np.roll(ulgrid, -1, axis=0) 217 | ttsum = tdrsum + tulsum 218 | maparray[stepsize // 2:mapsize:stepsize, 0:mapsize:stepsize] = wibbledmean(ttsum) 219 | 220 | while stepsize >= 2: 221 | fillsquares() 222 | filldiamonds() 223 | stepsize //= 2 224 | wibble /= wibbledecay 225 | 226 | maparray -= maparray.min() 227 | return maparray / maparray.max() 228 | 229 | 230 | def clipped_zoom(img, zoom_factor): 231 | h = img.shape[0] 232 | # ceil crop height(= crop width) 233 | ch = int(np.ceil(h / zoom_factor)) 234 | 235 | top = (h - ch) // 2 236 | img = scizoom(img[top:top + ch, top:top + ch], (zoom_factor, zoom_factor, 1), order=1) 237 | # trim off any extra pixels 238 | trim_top = (img.shape[0] - h) // 2 239 | 240 | return img[trim_top:trim_top + h, trim_top:trim_top + h] 241 | 242 | 243 | # /////////////// End Distortion Helpers /////////////// 244 | 245 | 246 | # /////////////// Distortions /////////////// 247 | 248 | def gaussian_noise(x, severity=1): 249 | c = [0.04, 0.08, .12, .15, .18][severity - 1] 250 | 251 | x = np.array(x) / 255. 252 | return np.clip(x + np.random.normal(size=x.shape, scale=c), 0, 1) * 255 253 | 254 | 255 | def shot_noise(x, severity=1): 256 | c = [250, 100, 50, 30, 15][severity - 1] 257 | 258 | x = np.array(x) / 255. 259 | return np.clip(np.random.poisson(x * c) / c, 0, 1) * 255 260 | 261 | 262 | def impulse_noise(x, severity=1): 263 | c = [.01, .02, .05, .08, .14][severity - 1] 264 | 265 | x = sk.util.random_noise(np.array(x) / 255., mode='s&p', amount=c) 266 | return np.clip(x, 0, 1) * 255 267 | 268 | 269 | def speckle_noise(x, severity=1): 270 | c = [.15, .2, 0.25, 0.3, 0.35][severity - 1] 271 | 272 | x = np.array(x) / 255. 273 | return np.clip(x + x * np.random.normal(size=x.shape, scale=c), 0, 1) * 255 274 | 275 | 276 | def gaussian_blur(x, severity=1): 277 | c = [.5, .75, 1, 1.25, 1.5][severity - 1] 278 | 279 | x = gaussian(np.array(x) / 255., sigma=c, multichannel=True) 280 | return np.clip(x, 0, 1) * 255 281 | 282 | 283 | def glass_blur(x, severity=1): 284 | # sigma, max_delta, iterations 285 | c = [(0.1,1,1), (0.5,1,1), (0.6,1,2), (0.7,2,1), (0.9,2,2)][severity - 1] 286 | 287 | x = np.uint8(gaussian(np.array(x) / 255., sigma=c[0], multichannel=True) * 255) 288 | 289 | # locally shuffle pixels 290 | for i in range(c[2]): 291 | for h in range(64 - c[1], c[1], -1): 292 | for w in range(64 - c[1], c[1], -1): 293 | dx, dy = np.random.randint(-c[1], c[1], size=(2,)) 294 | h_prime, w_prime = h + dy, w + dx 295 | # swap 296 | x[h, w], x[h_prime, w_prime] = x[h_prime, w_prime], x[h, w] 297 | 298 | return np.clip(gaussian(x / 255., sigma=c[0], multichannel=True), 0, 1) * 255 299 | 300 | 301 | def defocus_blur(x, severity=1): 302 | c = [(0.5, 0.6), (1, 0.1), (1.5, 0.1), (2.5, 0.01), (3, 0.1)][severity - 1] 303 | 304 | x = np.array(x) / 255. 305 | kernel = disk(radius=c[0], alias_blur=c[1]) 306 | 307 | channels = [] 308 | for d in range(3): 309 | channels.append(cv2.filter2D(x[:, :, d], -1, kernel)) 310 | channels = np.array(channels).transpose((1, 2, 0)) # 3x64x64 -> 64x64x3 311 | 312 | return np.clip(channels, 0, 1) * 255 313 | 314 | 315 | def motion_blur(x, severity=1): 316 | c = [(10,1), (10,1.5), (10,2), (10,2.5), (12,3)][severity - 1] 317 | 318 | output = BytesIO() 319 | 320 | x = Image.fromarray(np.uint8(x)).convert('RGB') # BORJI 321 | x.save(output, format='PNG') 322 | x = MotionImage(blob=output.getvalue()) 323 | 324 | x.motion_blur(radius=c[0], sigma=c[1], angle=np.random.uniform(-45, 45)) 325 | 326 | x = cv2.imdecode(np.fromstring(x.make_blob(), np.uint8), 327 | cv2.IMREAD_UNCHANGED) 328 | 329 | if x.shape != (64, 64): 330 | return np.clip(x[..., [2, 1, 0]], 0, 255) # BGR to RGB 331 | else: # greyscale to RGB 332 | return np.clip(np.array([x, x, x]).transpose((1, 2, 0)), 0, 255) 333 | 334 | 335 | def zoom_blur(x, severity=1): 336 | c = [np.arange(1, 1.06, 0.01), np.arange(1, 1.11, 0.01), np.arange(1, 1.16, 0.01), 337 | np.arange(1, 1.21, 0.01), np.arange(1, 1.26, 0.01)][severity - 1] 338 | 339 | x = (np.array(x) / 255.).astype(np.float32) 340 | out = np.zeros_like(x) 341 | for zoom_factor in c: 342 | out += clipped_zoom(x, zoom_factor) 343 | 344 | x = (x + out) / (len(c) + 1) 345 | return np.clip(x, 0, 1) * 255 346 | 347 | 348 | def fog(x, severity=1): 349 | c = [(.4,3), (.7,3), (1,2.5), (1.5,2), (2,1.75)][severity - 1] 350 | 351 | x = np.array(x) / 255. 352 | max_val = x.max() 353 | # import pdb; pdb.set_trace() 354 | # x += c[0] * plasma_fractal(wibbledecay=c[1])[:64, :64][..., np.newaxis] 355 | tmp = c[0] * plasma_fractal(wibbledecay=c[1])[:64, :64][..., np.newaxis] 356 | tmp = torch.tensor(cv2.resize(tmp, (224,224))) 357 | tmp = tmp[...,None] 358 | tmp = torch.cat([tmp, tmp, tmp],2) 359 | # import pdb; pdb.set_trace() 360 | x += tmp.numpy() 361 | return np.clip(x * max_val / (max_val + c[0]), 0, 1) * 255 362 | 363 | 364 | def frost(x, severity=1): 365 | c = [(1, 0.3), (0.9, 0.4), (0.8, 0.45), (0.75, 0.5), (0.7, 0.55)][severity - 1] 366 | idx = np.random.randint(5) 367 | filename = ['./frosts/frost1.png', './frosts/frost2.png', './frosts/frost3.png', './frosts/frost4.jpg', './frosts/frost5.jpg', './frosts/frost6.jpg'][idx] 368 | frost = cv2.imread(filename) 369 | # import pdb; pdb.set_trace() 370 | # frost = cv2.resize(frost, (0, 0), fx=0.3, fy=0.3) 371 | # randomly crop and convert to rgb 372 | x_start, y_start = np.random.randint(0, frost.shape[0] - 224), np.random.randint(0, frost.shape[1] - 224) 373 | frost = frost[x_start:x_start + 224, y_start:y_start + 224][..., [2, 1, 0]] 374 | 375 | return np.clip(c[0] * np.array(x) + c[1] * frost, 0, 255) 376 | 377 | # x_start, y_start = np.random.randint(0, frost.shape[0] - 64), np.random.randint(0, frost.shape[1] - 64) 378 | # frost = frost[x_start:x_start + 64, y_start:y_start + 64][..., [2, 1, 0]] 379 | 380 | # return np.clip(c[0] * np.array(x) + c[1] * frost, 0, 255) 381 | 382 | 383 | 384 | def snow(x, severity=1): 385 | c = [(0.1,0.2,1,0.6,8,3,0.8), 386 | (0.1,0.2,1,0.5,10,4,0.8), 387 | (0.15,0.3,1.75,0.55,10,4,0.7), 388 | (0.25,0.3,2.25,0.6,12,6,0.65), 389 | (0.3,0.3,1.25,0.65,14,12,0.6)][severity - 1] 390 | 391 | x = np.array(x, dtype=np.float32) / 255. 392 | snow_layer = np.random.normal(size=x.shape[:2], loc=c[0], scale=c[1]) # [:2] for monochrome 393 | 394 | snow_layer = clipped_zoom(snow_layer[..., np.newaxis], c[2]) 395 | snow_layer[snow_layer < c[3]] = 0 396 | 397 | snow_layer = PILImage.fromarray((np.clip(snow_layer.squeeze(), 0, 1) * 255).astype(np.uint8), mode='L') 398 | output = BytesIO() 399 | snow_layer.save(output, format='PNG') 400 | snow_layer = MotionImage(blob=output.getvalue()) 401 | 402 | snow_layer.motion_blur(radius=c[4], sigma=c[5], angle=np.random.uniform(-135, -45)) 403 | 404 | snow_layer = cv2.imdecode(np.fromstring(snow_layer.make_blob(), np.uint8), 405 | cv2.IMREAD_UNCHANGED) / 255. 406 | snow_layer = snow_layer[..., np.newaxis] 407 | 408 | # x = c[6] * x + (1 - c[6]) * np.maximum(x, cv2.cvtColor(x, cv2.COLOR_RGB2GRAY).reshape(64, 64, 1) * 1.5 + 0.5) 409 | x = c[6] * x + (1 - c[6]) * np.maximum(x, cv2.cvtColor(x, cv2.COLOR_RGB2GRAY).reshape(224, 224, 1) * 1.5 + 0.5) 410 | return np.clip(x + snow_layer + np.rot90(snow_layer, k=2), 0, 1) * 255 411 | 412 | 413 | def spatter(x, severity=1): 414 | c = [(0.62,0.1,0.7,0.7,0.6,0), 415 | (0.65,0.1,0.8,0.7,0.6,0), 416 | (0.65,0.3,1,0.69,0.6,0), 417 | (0.65,0.1,0.7,0.68,0.6,1), 418 | (0.65,0.1,0.5,0.67,0.6,1)][severity - 1] 419 | x = np.array(x, dtype=np.float32) / 255. 420 | 421 | liquid_layer = np.random.normal(size=x.shape[:2], loc=c[0], scale=c[1]) 422 | 423 | liquid_layer = gaussian(liquid_layer, sigma=c[2]) 424 | liquid_layer[liquid_layer < c[3]] = 0 425 | if c[5] == 0: 426 | liquid_layer = (liquid_layer * 255).astype(np.uint8) 427 | dist = 255 - cv2.Canny(liquid_layer, 50, 150) 428 | dist = cv2.distanceTransform(dist, cv2.DIST_L2, 5) 429 | _, dist = cv2.threshold(dist, 20, 20, cv2.THRESH_TRUNC) 430 | dist = cv2.blur(dist, (3, 3)).astype(np.uint8) 431 | dist = cv2.equalizeHist(dist) 432 | # ker = np.array([[-1,-2,-3],[-2,0,0],[-3,0,1]], dtype=np.float32) 433 | # ker -= np.mean(ker) 434 | ker = np.array([[-2, -1, 0], [-1, 1, 1], [0, 1, 2]]) 435 | dist = cv2.filter2D(dist, cv2.CV_8U, ker) 436 | dist = cv2.blur(dist, (3, 3)).astype(np.float32) 437 | 438 | m = cv2.cvtColor(liquid_layer * dist, cv2.COLOR_GRAY2BGRA) 439 | m /= np.max(m, axis=(0, 1)) 440 | m *= c[4] 441 | 442 | # water is pale turqouise 443 | color = np.concatenate((175 / 255. * np.ones_like(m[..., :1]), 444 | 238 / 255. * np.ones_like(m[..., :1]), 445 | 238 / 255. * np.ones_like(m[..., :1])), axis=2) 446 | 447 | color = cv2.cvtColor(color, cv2.COLOR_BGR2BGRA) 448 | x = cv2.cvtColor(x, cv2.COLOR_BGR2BGRA) 449 | 450 | return cv2.cvtColor(np.clip(x + m * color, 0, 1), cv2.COLOR_BGRA2BGR) * 255 451 | else: 452 | m = np.where(liquid_layer > c[3], 1, 0) 453 | m = gaussian(m.astype(np.float32), sigma=c[4]) 454 | m[m < 0.8] = 0 455 | # m = np.abs(m) ** (1/c[4]) 456 | 457 | # mud brown 458 | color = np.concatenate((63 / 255. * np.ones_like(x[..., :1]), 459 | 42 / 255. * np.ones_like(x[..., :1]), 460 | 20 / 255. * np.ones_like(x[..., :1])), axis=2) 461 | 462 | color *= m[..., np.newaxis] 463 | x *= (1 - m[..., np.newaxis]) 464 | 465 | return np.clip(x + color, 0, 1) * 255 466 | 467 | 468 | def contrast(x, severity=1): 469 | c = [.4, .3, .2, .1, 0.05][severity - 1] 470 | 471 | x = np.array(x) / 255. 472 | means = np.mean(x, axis=(0, 1), keepdims=True) 473 | return np.clip((x - means) * c + means, 0, 1) * 255 474 | 475 | 476 | def brightness(x, severity=1): 477 | c = [.1, .2, .3, .4, .5][severity - 1] 478 | 479 | x = np.array(x) / 255. 480 | x = sk.color.rgb2hsv(x) 481 | x[:, :, 2] = np.clip(x[:, :, 2] + c, 0, 1) 482 | x = sk.color.hsv2rgb(x) 483 | 484 | return np.clip(x, 0, 1) * 255 485 | 486 | 487 | def saturate(x, severity=1): 488 | c = [(0.3, 0), (0.1, 0), (2, 0), (5, 0.1), (30, 0.2)][severity - 1] 489 | 490 | x = np.array(x) / 255. 491 | x = sk.color.rgb2hsv(x) 492 | x[:, :, 1] = np.clip(x[:, :, 1] * c[0] + c[1], 0, 1) 493 | x = sk.color.hsv2rgb(x) 494 | 495 | return np.clip(x, 0, 1) * 255 496 | 497 | 498 | def jpeg_compression(x, severity=1): 499 | c = [65, 58, 50, 40, 25][severity - 1] 500 | 501 | x = Image.fromarray(np.uint8(x)).convert('RGB') # BORJI 502 | output = BytesIO() 503 | x.save(output, 'JPEG', quality=c) 504 | x = PILImage.open(output) 505 | 506 | return np.array(x) 507 | 508 | 509 | def pixelate(x, severity=1): 510 | c = [0.9, 0.8, 0.7, 0.6, 0.5][severity - 1] 511 | 512 | x = Image.fromarray(np.uint8(x)).convert('RGB') # BORJI 513 | x = x.resize((int(64 * c), int(64 * c)), PILImage.BOX) 514 | # x = x.resize((64, 64), PILImage.BOX) 515 | x = x.resize((224, 224), PILImage.BOX) 516 | 517 | return np.array(x) 518 | 519 | 520 | # mod of https://gist.github.com/erniejunior/601cdf56d2b424757de5 521 | def elastic_transform(image, severity=1): 522 | IMSIZE = 64 523 | c = [(IMSIZE*0, IMSIZE*0, IMSIZE*0.08), 524 | (IMSIZE*0.05, IMSIZE*0.3, IMSIZE*0.06), 525 | (IMSIZE*0.1, IMSIZE*0.08, IMSIZE*0.06), 526 | (IMSIZE*0.1, IMSIZE*0.03, IMSIZE*0.03), 527 | (IMSIZE*0.16, IMSIZE*0.03, IMSIZE*0.02)][severity - 1] 528 | 529 | image = np.array(image, dtype=np.float32) / 255. 530 | shape = image.shape 531 | shape_size = shape[:2] 532 | 533 | # random affine 534 | center_square = np.float32(shape_size) // 2 535 | square_size = min(shape_size) // 3 536 | pts1 = np.float32([center_square + square_size, 537 | [center_square[0] + square_size, center_square[1] - square_size], 538 | center_square - square_size]) 539 | pts2 = pts1 + np.random.uniform(-c[2], c[2], size=pts1.shape).astype(np.float32) 540 | M = cv2.getAffineTransform(pts1, pts2) 541 | image = cv2.warpAffine(image, M, shape_size[::-1], borderMode=cv2.BORDER_REFLECT_101) 542 | 543 | dx = (gaussian(np.random.uniform(-1, 1, size=shape[:2]), 544 | c[1], mode='reflect', truncate=3) * c[0]).astype(np.float32) 545 | dy = (gaussian(np.random.uniform(-1, 1, size=shape[:2]), 546 | c[1], mode='reflect', truncate=3) * c[0]).astype(np.float32) 547 | dx, dy = dx[..., np.newaxis], dy[..., np.newaxis] 548 | 549 | x, y, z = np.meshgrid(np.arange(shape[1]), np.arange(shape[0]), np.arange(shape[2])) 550 | indices = np.reshape(y + dy, (-1, 1)), np.reshape(x + dx, (-1, 1)), np.reshape(z, (-1, 1)) 551 | return np.clip(map_coordinates(image, indices, order=1, mode='reflect').reshape(shape), 0, 1) * 255 552 | 553 | 554 | # /////////////// End Distortions /////////////// 555 | 556 | 557 | # /////////////// Further Setup /////////////// 558 | 559 | 560 | def save_distorted(method=gaussian_noise): 561 | for severity in range(1, 6): 562 | print(method.__name__, severity) 563 | distorted_dataset = DistortImageFolder( 564 | root="/share/data/vision-greg/ImageNet/clsloc/images/val", 565 | method=method, severity=severity, 566 | transform=trn.Compose([trn.Resize((64, 64))])) 567 | distorted_dataset_loader = torch.utils.data.DataLoader( 568 | distorted_dataset, batch_size=100, shuffle=False, num_workers=6) 569 | 570 | for _ in distorted_dataset_loader: continue 571 | 572 | 573 | # /////////////// End Further Setup /////////////// 574 | 575 | 576 | if __name__ == '__main__': 577 | 578 | # /////////////// Display Results /////////////// 579 | import collections 580 | 581 | print('\nUsing ImageNet data') 582 | 583 | d = collections.OrderedDict() 584 | # d['Gaussian Noise'] = gaussian_noise 585 | # d['Shot Noise'] = shot_noise 586 | # d['Impulse Noise'] = impulse_noise 587 | # d['Defocus Blur'] = defocus_blur 588 | # d['Glass Blur'] = glass_blur 589 | # d['Motion Blur'] = motion_blur 590 | # d['Zoom Blur'] = zoom_blur 591 | # d['Snow'] = snow 592 | d['Frost'] = frost 593 | d['Fog'] = fog 594 | d['Brightness'] = brightness 595 | d['Contrast'] = contrast 596 | # d['Elastic'] = elastic_transform 597 | # d['Pixelate'] = pixelate 598 | # d['JPEG'] = jpeg_compression 599 | # 600 | # d['Speckle Noise'] = speckle_noise 601 | # d['Gaussian Blur'] = gaussian_blur 602 | # d['Spatter'] = spatter 603 | # d['Saturate'] = saturate 604 | 605 | for method_name in d.keys(): 606 | save_distorted(d[method_name]) 607 | -------------------------------------------------------------------------------- /code/overlapped_classes.py: -------------------------------------------------------------------------------- 1 | overlapped_classes = ["Alarm clock", 2 | "Backpack", 3 | "Banana", 4 | "Band Aid", 5 | "Basket", 6 | "Bath towel", 7 | "Beer bottle", 8 | "Bench", 9 | "Bicycle", 10 | "Binder (closed)", 11 | "Bottle cap", 12 | "Bread loaf", 13 | "Broom", 14 | "Bucket", 15 | "Butcher's knife", 16 | "Can opener", 17 | "Candle", 18 | "Cellphone", 19 | "Chair", 20 | "Clothes hamper", 21 | "Combination lock", 22 | "Computer mouse", 23 | "Desk lamp", 24 | "Dishrag or hand towel", 25 | "Doormat", 26 | "Dress", 27 | "Dress shoe (men)", 28 | "Drill", 29 | "Drinking Cup", 30 | "Drying rack for plates", 31 | "Envelope", 32 | "Fan", 33 | "Coffee/French press", 34 | "Frying pan", 35 | "Hair dryer", 36 | "Hammer", 37 | "Helmet", 38 | "Iron (for clothes)", 39 | "Jeans", 40 | "Keyboard", 41 | "Ladle", 42 | "Lampshade", 43 | "Laptop (open)", 44 | "Lemon", 45 | "Letter opener", 46 | "Lighter", 47 | "Lipstick", 48 | "Match", 49 | "Measuring cup", 50 | "Microwave", 51 | "Mixing / Salad Bowl", 52 | "Monitor", 53 | "Mug", 54 | "Nail (fastener)", 55 | "Necklace", 56 | "Orange", 57 | "Padlock", 58 | "Paintbrush", 59 | "Paper towel", 60 | "Pen", 61 | "Pill bottle", 62 | "Pillow", 63 | "Pitcher", 64 | "Plastic bag", 65 | "Plate", 66 | "Plunger", 67 | "Pop can", 68 | "Portable heater", 69 | "Printer", 70 | "Remote control", 71 | "Ruler", 72 | "Running shoe", 73 | "Safety pin", 74 | "Salt shaker", 75 | "Sandal", 76 | "Screw", 77 | "Shovel", 78 | "Skirt", 79 | "Sleeping bag", 80 | "Soap dispenser", 81 | "Sock", 82 | "Soup Bowl", 83 | "Spatula", 84 | "Speaker", 85 | "Still Camera", 86 | "Strainer", 87 | "Stuffed animal", 88 | "Suit jacket", 89 | "Sunglasses", 90 | "Sweater", 91 | "Swimming trunks", 92 | "T-shirt", 93 | "TV", 94 | "Teapot", 95 | "Tennis racket", 96 | "Tie", 97 | "Toaster", 98 | "Toilet paper roll", 99 | "Trash bin", 100 | "Tray", 101 | "Umbrella", 102 | "Vacuum cleaner", 103 | "Vase", 104 | "Wallet", 105 | "Watch", 106 | "Water bottle", 107 | "Weight (exercise)", 108 | "Weight scale", 109 | "Wheel", 110 | "Whistle", 111 | "Wine bottle", 112 | "Winter glove", 113 | "Wok"] -------------------------------------------------------------------------------- /code/test_noise_robustness.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | import argparse 4 | import os 5 | import time 6 | import torch 7 | from torch.autograd import Variable as V 8 | import torch.nn.functional as F 9 | import torch.backends.cudnn as cudnn 10 | import torchvision.datasets as dset 11 | import torchvision.transforms as trn 12 | import torchvision.models as models 13 | import torch.utils.model_zoo as model_zoo 14 | import numpy as np 15 | from model import * 16 | from utils import detect_edge_batch, save_img 17 | from dataset import folderDB 18 | from make_imagenet_64_c import * 19 | 20 | 21 | os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' 22 | 23 | 24 | parser = argparse.ArgumentParser(description='Evaluates robustness of various nets on ImageNet', 25 | formatter_class=argparse.ArgumentDefaultsHelpFormatter) 26 | 27 | parser.add_argument('--net_type', '-n', type=str, 28 | choices=['rgb', 'rgbedge', 'edge']) 29 | 30 | parser.add_argument('--data_dir', type=str) 31 | parser.add_argument('--classes', type=int, default=10, help='number of classes') 32 | parser.add_argument('--inp_size', type=int, default=28, help='size of the input image') 33 | 34 | # Architecture 35 | parser.add_argument('--model_name', '-m', type=str) 36 | #choices=['tinyImgnet.pth']) 37 | # Acceleration 38 | # parser.add_argument('--ngpu', type=int, default=1, help='0 = CPU.') 39 | args = parser.parse_args() 40 | print(args) 41 | 42 | # /////////////// Model Setup /////////////// 43 | 44 | # if args.model_name == 'tinyImgnet': 45 | # net = resnext_50_32x4d 46 | # net.load_state_dict(torch.load('')) 47 | # args.test_bs = 64 48 | 49 | 50 | data_dir = args.data_dir # 'tiny-imagenet-200' 51 | # inp_size = 64 52 | # n_classes = 50 #200 53 | 54 | 55 | net_type = args.net_type.lower() 56 | 57 | net, _, _, _ = build_model_resNet(net_type, args.data_dir, args.inp_size, args.classes) 58 | net.load_state_dict(torch.load(args.model_name,map_location=torch.device('cpu'))) 59 | 60 | 61 | device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") 62 | net.to(device) 63 | 64 | 65 | 66 | 67 | args.test_bs = 64 68 | args.prefetch = 4 69 | 70 | # for p in net.parameters(): 71 | # p.volatile = True 72 | 73 | # if args.ngpu > 1: 74 | # net = torch.nn.DataParallel(net, device_ids=list(range(args.ngpu))) 75 | 76 | # if args.ngpu > 0: 77 | # net.cuda() 78 | 79 | torch.manual_seed(1) 80 | np.random.seed(1) 81 | # if args.ngpu > 0: 82 | # torch.cuda.manual_seed(1) 83 | 84 | net.eval() 85 | cudnn.benchmark = True # fire on all cylinders 86 | 87 | print('Model Loaded') 88 | 89 | # /////////////// Data Loader /////////////// 90 | 91 | # mean = [0.485, 0.456, 0.406] 92 | # std = [0.229, 0.224, 0.225] 93 | 94 | mean = [0.3403, 0.3121, 0.3214] 95 | std = [0.2724, 0.2608, 0.2669] 96 | 97 | # clean_loader = torch.utils.data.DataLoader(dset.ImageFolder( 98 | # root="./tiny-imagenet-200/testset/images/", 99 | # # transform=trn.Compose([trn.Resize(256), trn.CenterCrop(224), trn.ToTensor(), trn.Normalize(mean, std)])), 100 | # transform=trn.Compose([trn.Resize(64), trn.ToTensor(), trn.Normalize(mean, std)])), 101 | # batch_size=args.test_bs, shuffle=False, num_workers=args.prefetch, pin_memory=True) 102 | 103 | 104 | # /////////////// Further Setup /////////////// 105 | 106 | def auc(errs): # area under the distortion-error curve 107 | area = 0 108 | for i in range(1, len(errs)): 109 | area += (errs[i] + errs[i - 1]) / 2 110 | area /= len(errs) - 1 111 | return area 112 | 113 | 114 | # correct = 0 115 | # for batch_idx, (data, target) in enumerate(clean_loader): 116 | # data = V(data.cuda(), volatile=True) 117 | # 118 | # output = net(data) 119 | # 120 | # pred = output.data.max(1)[1] 121 | # correct += pred.eq(target.cuda()).sum() 122 | # 123 | # clean_error = 1 - correct / len(clean_loader.dataset) 124 | # print('Clean dataset error (%): {:.2f}'.format(100 * clean_error)) 125 | 126 | 127 | def show_performance(distortion_name): 128 | errs = [] 129 | 130 | 131 | transform=trn.Compose([ trn.ToTensor(), trn.Normalize(mean, std)]) 132 | # transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor(), trn.Normalize(mean, std)]) 133 | 134 | for severity in range(1,6): 135 | # distorted_dataset = dset.ImageFolder( 136 | # # root='/share/data/vision-greg/DistortedImageNet/JPEG/' + distortion_name + '/' + str(severity), 137 | # root='./Tiny-ImageNet-C/' + distortion_name + '/' + str(severity), 138 | # # transform=trn.Compose([trn.CenterCrop(224), trn.ToTensor(), trn.Normalize(mean, std)])) 139 | # transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor(), trn.Normalize(mean, std)])) 140 | 141 | 142 | distorted_dataset = folderDB( 143 | root_dir='.', train=False, transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor()]), net_type=net_type, base_folder=data_dir) 144 | # root_dir='.', train=False, transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor(), trn.Normalize(mean, std)]), net_type=net_type, base_folder=data_dir) 145 | 146 | if net_type == 'edge': 147 | distorted_dataset = folderDB( 148 | root_dir='.', train=False, transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor()]), net_type='rgbedge', base_folder=data_dir) 149 | # root_dir='.', train=False, transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor(), trn.Normalize(mean, std)]), net_type=net_type, base_folder=data_dir) 150 | 151 | 152 | 153 | distorted_dataset_loader = torch.utils.data.DataLoader( 154 | distorted_dataset, batch_size=args.test_bs, shuffle=False, num_workers=2) 155 | 156 | 157 | 158 | # dataloader_dict = {'train': trainloader, 'val': testloader} 159 | 160 | 161 | # distorted_dataset = dset.ImageFolder( 162 | # # root='/share/data/vision-greg/DistortedImageNet/JPEG/' + distortion_name + '/' + str(severity), 163 | # root='./Tiny-ImageNet-C/' + distortion_name + '/' + str(severity), 164 | # # transform=trn.Compose([trn.CenterCrop(224), trn.ToTensor(), trn.Normalize(mean, std)])) 165 | # transform=trn.Compose([trn.CenterCrop(64), trn.ToTensor(), trn.Normalize(mean, std)])) 166 | # distorted_dataset_loader = torch.utils.data.DataLoader( 167 | # distorted_dataset, batch_size=args.test_bs, shuffle=False, num_workers=args.prefetch, pin_memory=True) 168 | 169 | 170 | 171 | correct = 0 172 | for batch_idx, (data, target) in enumerate(distorted_dataset_loader): 173 | data, target = data.to(device), target.to(device) 174 | # data = V(data.cuda(), volatile=True) 175 | # import pdb; pdb.set_trace() 176 | 177 | if batch_idx>0: break 178 | for idx,d in enumerate(data): 179 | if idx>0: break 180 | # import pdb; pdb.set_trace() 181 | # exr = distortion_name + '(d*255,severity)' 182 | exr = distortion_name + '(d[:3].permute(1,2,0)*255,severity)' 183 | aa = torch.Tensor(eval(exr)) 184 | data[idx][:3] = transform((aa/255).numpy()) 185 | # out_img = data[idx][:3].permute(1,2,0) 186 | 187 | # uncomment this to save some sample images 188 | # out_img = (out_img - out_img.min())/ (out_img.max() - out_img.min()) 189 | # out_img = out_img*255 190 | # save_img(out_img, "sampleRobustness/{}".format(distortion_name+str(severity)+'.jpg')) 191 | # idx = 0 192 | 193 | if net_type in ['rgbedge', 'edge']: 194 | # import pdb; pdb.set_trace() 195 | # edge_maps = torch.zeros((data.shape[0],1,data.shape[2],data.shape[2])) 196 | # data = torch.cat((data, edge_maps),dim=1)#[None] 197 | data = detect_edge_batch(data) 198 | 199 | if net_type == 'edge': 200 | data = data[:,3].unsqueeze(1) 201 | 202 | # saving some sample edge maps 203 | # import pdb; pdb.set_trace() 204 | # out_img = data[idx][:3].squeeze(0) 205 | # out_img = (out_img - out_img.min())/ (out_img.max() - out_img.min()) 206 | # out_img = out_img*255 207 | # save_img(out_img, "sampleRobustness/edge_{}".format(distortion_name+str(severity)+'.jpg')) 208 | 209 | 210 | # if net_type == 'rgbedge': 211 | # import pdb; pdb.set_trace() 212 | # edge_maps = torch.zeros((data.shape[0],1,data.shape[2],data.shape[2])) 213 | # data = torch.cat((data, edge_maps),dim=1)#[None] 214 | # data = detect_edge_batch(data) 215 | 216 | output = net(data) 217 | 218 | pred = output.data.max(1)[1] 219 | # correct += pred.eq(target.cuda()).sum() 220 | correct += pred.eq(target).sum() 221 | 222 | 223 | errs.append(1 - 1.*correct / len(distorted_dataset)) 224 | 225 | print('\n=Average', tuple(errs)) 226 | return np.mean(errs) 227 | 228 | 229 | # /////////////// End Further Setup /////////////// 230 | 231 | 232 | # /////////////// Display Results /////////////// 233 | import collections 234 | 235 | print('\nUsing ImageNet data') 236 | 237 | distortions = [ 238 | 'gaussian_noise', 'shot_noise', 'impulse_noise', 239 | 'defocus_blur', 'glass_blur', 240 | 'motion_blur', 241 | 'zoom_blur', 242 | 'snow', 243 | 'frost', 244 | 'fog', 245 | 'brightness', 246 | 'contrast', 247 | 'elastic_transform', 248 | 'pixelate', 249 | 'jpeg_compression', 250 | # 'speckle_noise', 'gaussian_blur', 'spatter', 'saturate' 251 | ] 252 | 253 | 254 | 255 | error_rates = [] 256 | for distortion_name in distortions: 257 | print(distortion_name) 258 | rate = show_performance(distortion_name) 259 | error_rates.append(rate) 260 | print('Distortion: {:15s} | CE (unnormalized) (%): {:.2f}'.format(distortion_name, 100 * rate)) 261 | 262 | 263 | print('mCE (unnormalized by AlexNet errors) (%): {:.2f}'.format(100 * np.mean(error_rates))) 264 | 265 | -------------------------------------------------------------------------------- /mappings/folder_to_objectnet_label.json: -------------------------------------------------------------------------------- 1 | { 2 | "squeegee": "Squeegee", 3 | "umbrella": "Umbrella", 4 | "eyeglasses": "Eyeglasses", 5 | "coaster": "Coaster", 6 | "winter_glove": "Winter glove", 7 | "pet_food_container": "Pet food container", 8 | "scissors": "Scissors", 9 | "computer_mouse": "Computer mouse", 10 | "still_camera": "Still Camera", 11 | "weight_scale": "Weight scale", 12 | "cutting_board": "Cutting board", 13 | "spatula": "Spatula", 14 | "plunger": "Plunger", 15 | "paper": "Paper", 16 | "paper_bag": "Paper bag", 17 | "microwave": "Microwave", 18 | "keyboard": "Keyboard", 19 | "standing_lamp": "Standing lamp", 20 | "chair": "Chair", 21 | "mixing_salad_bowl": "Mixing / Salad Bowl", 22 | "milk": "Milk", 23 | "blender": "Blender", 24 | "flashlight": "Flashlight", 25 | "floss_container": "Floss container", 26 | "rake": "Rake", 27 | "sandal": "Sandal", 28 | "book_closed": "Book (closed)", 29 | "extension_cable": "Extension cable", 30 | "drawer_open": "Drawer (open)", 31 | "tv": "TV", 32 | "backpack": "Backpack", 33 | "playing_cards": "Playing cards", 34 | "jar": "Jar", 35 | "frying_pan": "Frying pan", 36 | "bench": "Bench", 37 | "coffee_grinder": "Coffee grinder", 38 | "jam": "Jam", 39 | "tape": "Tape / duct tape", 40 | "dog_bed": "Dog bed", 41 | "throw_pillow": "Throw pillow", 42 | "orange": "Orange", 43 | "speaker": "Speaker", 44 | "paint_can": "Paint can", 45 | "hand_towel_or_rag": "Dishrag or hand towel", 46 | "hat": "Hat", 47 | "toaster": "Toaster", 48 | "match": "Match", 49 | "plate": "Plate", 50 | "ironing_board": "Ironing board", 51 | "alarm_clock": "Alarm clock", 52 | "sunglasses": "Sunglasses", 53 | "power_bar": "Power bar", 54 | "wine_glass": "Wine glass", 55 | "ladle": "Ladle", 56 | "ruler": "Ruler", 57 | "blanket": "Blanket", 58 | "contact_lens_case": "Contact lens case", 59 | "watch": "Watch", 60 | "trash_bin": "Trash bin", 61 | "vase": "Vase", 62 | "oven_mitts": "Oven mitts", 63 | "spoon": "Spoon", 64 | "thermos": "Thermos", 65 | "fan": "Fan", 66 | "egg_carton": "Egg carton", 67 | "lighter": "Lighter", 68 | "glue_container": "Glue container", 69 | "bottle_cap": "Bottle cap", 70 | "wallet": "Wallet", 71 | "coffee_table": "Coffee table", 72 | "nail_clippers": "Nail clippers", 73 | "cellphone_case": "Cellphone case", 74 | "canned_food": "Canned food", 75 | "letter_opener": "Letter opener", 76 | "stapler": "Stapler", 77 | "whisk": "Whisk", 78 | "tongs": "Tongs", 79 | "lettuce": "Lettuce", 80 | "shoelace": "Shoelace", 81 | "button": "Button", 82 | "hair_dryer": "Hair dryer", 83 | "bracelet": "Bracelet", 84 | "spray_bottle": "Spray bottle", 85 | "laptop_charger": "Laptop charger", 86 | "portable_heater": "Portable heater", 87 | "suit_jacket": "Suit jacket", 88 | "dress_shoe_men": "Dress shoe (men)", 89 | "rock": "Rock", 90 | "water_filter": "Water filter", 91 | "earbuds": "Earbuds", 92 | "biscuits": "Biscuits", 93 | "mouthwash": "Mouthwash", 94 | "slipper": "Slipper", 95 | "eraser_white_board": "Eraser (white board)", 96 | "bicycle": "Bicycle", 97 | "ziploc_bag": "Ziploc bag", 98 | "dish_soap": "Dish soap", 99 | "travel_case": "Travel case", 100 | "receipt": "Receipt", 101 | "boots": "Boots", 102 | "sponge": "Sponge", 103 | "full_sized_towel": "Bath towel", 104 | "flour_container": "Flour container", 105 | "pill_bottle": "Pill bottle", 106 | "candle": "Candle", 107 | "calendar": "Calendar", 108 | "tweezers": "Tweezers", 109 | "dvd_player": "DVD player", 110 | "plastic_wrap": "Plastic wrap", 111 | "ribbon": "Ribbon", 112 | "blouse": "Blouse", 113 | "walking_cane": "Walking cane", 114 | "leaf": "Leaf", 115 | "lipstick": "Lipstick", 116 | "hair_brush": "Hair brush", 117 | "night_light": "Night light", 118 | "kettle": "Kettle", 119 | "honey_container": "Honey container", 120 | "tarp": "Tarp", 121 | "ice": "Ice", 122 | "drinking_straw": "Drinking straw", 123 | "detergent": "Detergent", 124 | "mug": "Mug", 125 | "toilet_paper_roll": "Toilet paper roll", 126 | "wok": "Wok", 127 | "swimming_trunks": "Swimming trunks", 128 | "clothes_hamper": "Clothes hamper", 129 | "removable_blade": "Removable blade", 130 | "shampoo_bottle": "Shampoo bottle", 131 | "skirt": "Skirt", 132 | "loofah": "Loofah", 133 | "broom": "Broom", 134 | "photograph_printed": "Photograph (printed)", 135 | "multitool": "Multitool", 136 | "makeup": "Makeup", 137 | "nail_file": "Nail file", 138 | "brooch": "Brooch", 139 | "cork": "Cork", 140 | "coffee_french_press": "Coffee/French press", 141 | "toy": "Toy", 142 | "thermometer": "Thermometer", 143 | "hairclip": "Hairclip", 144 | "document_folder_closed": "Document folder (closed)", 145 | "pliers": "Pliers", 146 | "strainer": "Strainer", 147 | "comb": "Comb", 148 | "water_bottle": "Water bottle", 149 | "peeler": "Peeler", 150 | "monitor": "Monitor", 151 | "box": "Box", 152 | "pill_organizer": "Pill organizer", 153 | "stopper_sink_tub": "Stopper (sink/tub)", 154 | "walker": "Walker", 155 | "step_stool": "Step stool", 156 | "bills_money": "Bills (money)", 157 | "skateboard": "Skateboard", 158 | "running_shoe": "Running shoe", 159 | "coin_money": "Coin (money)", 160 | "magazine": "Magazine", 161 | "drying_rack_for_clothes": "Drying rack for clothes", 162 | "toothpaste": "Toothpaste", 163 | "paper_towel": "Paper towel", 164 | "remote_control": "Remote control", 165 | "sugar_container": "Sugar container", 166 | "dress_pants": "Dress pants", 167 | "scarf": "Scarf", 168 | "dress_shirt": "Dress shirt", 169 | "cheese": "Cheese", 170 | "can_opener": "Can opener", 171 | "shovel": "Shovel", 172 | "paintbrush": "Paintbrush", 173 | "tennis_racket": "Tennis racket", 174 | "battery": "Battery", 175 | "stuffed_animal": "Stuffed animal", 176 | "jeans": "Jeans", 177 | "tanktop": "Tanktop", 178 | "dust_pan": "Dust pan", 179 | "earring": "Earring", 180 | "tomato": "Tomato", 181 | "marker": "Marker", 182 | "makeup_brush": "Makeup brush", 183 | "ring": "Ring", 184 | "air_freshener": "Air freshener", 185 | "tablecloth": "Tablecloth", 186 | "teabag": "Teabag", 187 | "belt": "Belt", 188 | "razor": "Razor", 189 | "clothes_hanger": "Clothes hanger", 190 | "bookend": "Bookend", 191 | "sweater": "Sweater", 192 | "sock": "Sock", 193 | "usb_flash_drive": "Usb flash drive", 194 | "cellphone_charger": "Cellphone charger", 195 | "pepper_shaker": "Pepper shaker", 196 | "phone_landline": "Phone (landline)", 197 | "banana": "Banana", 198 | "printer": "Printer", 199 | "paperclip": "Paperclip", 200 | "fork": "Fork", 201 | "headphones_over_ear": "Headphones (over ear)", 202 | "cooking_oil_bottle": "Cooking oil bottle", 203 | "deodorant": "Deodorant", 204 | "usb_cable": "Usb cable", 205 | "shorts": "Shorts", 206 | "bread_loaf": "Bread loaf", 207 | "pillow": "Pillow", 208 | "drinking_cup": "Drinking Cup", 209 | "envelope": "Envelope", 210 | "mouse_pad": "Mouse pad", 211 | "chopstick": "Chopstick", 212 | "t-shirt": "T-shirt", 213 | "padlock": "Padlock", 214 | "ice_cube_tray": "Ice cube tray", 215 | "chess_piece": "Chess piece", 216 | "cereal": "Cereal", 217 | "hairtie": "Hairtie", 218 | "teapot": "Teapot", 219 | "board_game": "Board game", 220 | "butchers_knife": "Butcher's knife", 221 | "soup_bowl": "Soup Bowl", 222 | "beer_bottle": "Beer bottle", 223 | "nail_polish": "Nail polish", 224 | "hand_mirror": "Hand mirror", 225 | "combination_lock": "Combination lock", 226 | "nut_for_screw": "Nut for a screw", 227 | "nail_fastener": "Nail (fastener)", 228 | "figurine_or_statue": "Figurine or statue", 229 | "soap_bar": "Soap bar", 230 | "bucket": "Bucket", 231 | "binder_closed": "Binder (closed)", 232 | "video_camera": "Video Camera", 233 | "baseball_glove": "Baseball glove", 234 | "tape_measure": "Tape measure", 235 | "tissue": "Tissue", 236 | "coffee_beans": "Coffee beans", 237 | "scrub_brush": "Scrub brush", 238 | "drill": "Drill", 239 | "suitcase": "Suitcase", 240 | "newspaper": "Newspaper", 241 | "sleeping_bag": "Sleeping bag", 242 | "dress_shoe_women": "Dress shoe (women)", 243 | "trophy": "Trophy", 244 | "plastic_bag": "Plastic bag", 245 | "doormat": "Doormat", 246 | "webcam": "Webcam", 247 | "rolling_pin": "Rolling pin", 248 | "pencil": "Pencil", 249 | "table_knife": "Table knife", 250 | "bread_knife": "Bread knife", 251 | "toothbrush": "Toothbrush", 252 | "bathrobe": "Bathrobe", 253 | "paper_plates": "Paper plates", 254 | "placemat": "Placemat", 255 | "light_bulb": "Light bulb", 256 | "soap_dispenser": "Soap dispenser", 257 | "nightstand": "Nightstand", 258 | "pen": "Pen", 259 | "squeeze_bottle": "Squeeze bottle", 260 | "wheel": "Wheel", 261 | "dress": "Dress", 262 | "helmet": "Helmet", 263 | "lemon": "Lemon", 264 | "hammer": "Hammer", 265 | "lampshade": "Lampshade", 266 | "salt_shaker": "Salt shaker", 267 | "power_cable": "Power cable", 268 | "vacuum_cleaner": "Vacuum cleaner", 269 | "iron_for_clothes": "Iron (for clothes)", 270 | "laptop_open": "Laptop (open)", 271 | "poster": "Poster", 272 | "coffee_machine": "Coffee machine", 273 | "tie": "Tie", 274 | "cd_case": "CD case", 275 | "baseball_bat": "Baseball bat", 276 | "tablet_ipad": "Tablet / iPad", 277 | "bottle_opener": "Bottle opener", 278 | "briefcase": "Briefcase", 279 | "baking_sheet": "Baking sheet", 280 | "screw": "Screw", 281 | "pitcher": "Pitcher", 282 | "notepad": "Notepad", 283 | "tote_bag": "Tote bag", 284 | "raincoat": "Raincoat", 285 | "necklace": "Necklace", 286 | "band_aid": "Band Aid", 287 | "notebook": "Notebook", 288 | "measuring_cup": "Measuring cup", 289 | "weight_exercise": "Weight (exercise)", 290 | "handbag": "Handbag", 291 | "bike_pump": "Bike pump", 292 | "bottle_stopper": "Bottle stopper", 293 | "chocolate": "Chocolate", 294 | "safety_pin": "Safety pin", 295 | "plastic_cup": "Plastic cup", 296 | "butter": "Butter", 297 | "cellphone": "Cellphone", 298 | "drying_rack_for_dishes": "Drying rack for plates", 299 | "trash_bag": "Trash bag", 300 | "tray": "Tray", 301 | "wine_bottle": "Wine bottle", 302 | "whistle": "Whistle", 303 | "key_chain": "Key chain", 304 | "napkin": "Napkin", 305 | "desk_lamp": "Desk lamp", 306 | "first_aid_kit": "First aid kit", 307 | "bed_sheet": "Bed sheet", 308 | "beer_can": "Beer can", 309 | "wrench": "Wrench", 310 | "pop_can": "Pop can", 311 | "basket": "Basket", 312 | "leggings": "Leggings", 313 | "egg": "Egg", 314 | "sewing_kit": "Sewing kit" 315 | } 316 | -------------------------------------------------------------------------------- /mappings/imagenet_to_label_2012_v2.txt: -------------------------------------------------------------------------------- 1 | kit fox, Vulpes macrotis 2 | English setter 3 | Australian terrier 4 | grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus 5 | lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens 6 | Egyptian cat 7 | ibex, Capra ibex 8 | Persian cat 9 | cougar, puma, catamount, mountain lion, painter, panther, Felis concolor 10 | gazelle 11 | porcupine, hedgehog 12 | sea lion 13 | badger 14 | Great Dane 15 | Scottish deerhound, deerhound 16 | killer whale, killer, orca, grampus, sea wolf, Orcinus orca 17 | mink 18 | African elephant, Loxodonta africana 19 | red wolf, maned wolf, Canis rufus, Canis niger 20 | jaguar, panther, Panthera onca, Felis onca 21 | hyena, hyaena 22 | titi, titi monkey 23 | three-toed sloth, ai, Bradypus tridactylus 24 | sorrel 25 | black-footed ferret, ferret, Mustela nigripes 26 | dalmatian, coach dog, carriage dog 27 | Staffordshire bullterrier, Staffordshire bull terrier 28 | Bouvier des Flandres, Bouviers des Flandres 29 | weasel 30 | miniature poodle 31 | bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis 32 | fox squirrel, eastern fox squirrel, Sciurus niger 33 | colobus, colobus monkey 34 | tiger cat 35 | impala, Aepyceros melampus 36 | coyote, prairie wolf, brush wolf, Canis latrans 37 | Yorkshire terrier 38 | Newfoundland, Newfoundland dog 39 | red fox, Vulpes vulpes 40 | hartebeest 41 | grey fox, gray fox, Urocyon cinereoargenteus 42 | Pekinese, Pekingese, Peke 43 | guenon, guenon monkey 44 | mongoose 45 | indri, indris, Indri indri, Indri brevicaudatus 46 | tiger, Panthera tigris 47 | wild boar, boar, Sus scrofa 48 | zebra 49 | ram, tup 50 | orangutan, orang, orangutang, Pongo pygmaeus 51 | basenji 52 | leopard, Panthera pardus 53 | vizsla, Hungarian pointer 54 | squirrel monkey, Saimiri sciureus 55 | Siamese cat, Siamese 56 | chimpanzee, chimp, Pan troglodytes 57 | komondor 58 | proboscis monkey, Nasalis larvatus 59 | guinea pig, Cavia cobaya 60 | white wolf, Arctic wolf, Canis lupus tundrarum 61 | ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus 62 | gorilla, Gorilla gorilla 63 | ox 64 | Tibetan mastiff 65 | spider monkey, Ateles geoffroyi 66 | Doberman, Doberman pinscher 67 | warthog 68 | Arabian camel, dromedary, Camelus dromedarius 69 | siamang, Hylobates syndactylus, Symphalangus syndactylus 70 | golden retriever 71 | Border collie 72 | hare 73 | boxer 74 | patas, hussar monkey, Erythrocebus patas 75 | baboon 76 | macaque 77 | capuchin, ringtail, Cebus capucinus 78 | flat-coated retriever 79 | hog, pig, grunter, squealer, Sus scrofa 80 | Eskimo dog, husky 81 | Brittany spaniel 82 | dial telephone, dial phone 83 | maze, labyrinth 84 | Gordon setter 85 | dingo, warrigal, warragal, Canis dingo 86 | hamster 87 | Arctic fox, white fox, Alopex lagopus 88 | water buffalo, water ox, Asiatic buffalo, Bubalus bubalis 89 | American black bear, black bear, Ursus americanus, Euarctos americanus 90 | Angora, Angora rabbit 91 | bison 92 | howler monkey, howler 93 | hippopotamus, hippo, river horse, Hippopotamus amphibius 94 | giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca 95 | tabby, tabby cat 96 | marmoset 97 | Saint Bernard, St Bernard 98 | armadillo 99 | redbone 100 | polecat, fitch, foulmart, foumart, Mustela putorius 101 | marmot 102 | gibbon, Hylobates lar 103 | llama 104 | wood rabbit, cottontail, cottontail rabbit 105 | lion, king of beasts, Panthera leo 106 | Irish setter, red setter 107 | dugong, Dugong dugon 108 | Indian elephant, Elephas maximus 109 | beaver 110 | Madagascar cat, ring-tailed lemur, Lemur catta 111 | Rhodesian ridgeback 112 | lynx, catamount 113 | African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus 114 | langur 115 | timber wolf, grey wolf, gray wolf, Canis lupus 116 | cheetah, chetah, Acinonyx jubatus 117 | sloth bear, Melursus ursinus, Ursus ursinus 118 | German shepherd, German shepherd dog, German police dog, alsatian 119 | otter 120 | koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus 121 | tusker 122 | echidna, spiny anteater, anteater 123 | wallaby, brush kangaroo 124 | platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus 125 | wombat 126 | revolver, six-gun, six-shooter 127 | umbrella 128 | schooner 129 | soccer ball 130 | accordion, piano accordion, squeeze box 131 | ant, emmet, pismire 132 | starfish, sea star 133 | chambered nautilus, pearly nautilus, nautilus 134 | grand piano, grand 135 | laptop, laptop computer 136 | strawberry 137 | airliner 138 | warplane, military plane 139 | airship, dirigible 140 | balloon 141 | space shuttle 142 | fireboat 143 | gondola 144 | speedboat 145 | lifeboat 146 | canoe 147 | yawl 148 | catamaran 149 | trimaran 150 | container ship, containership, container vessel 151 | liner, ocean liner 152 | pirate, pirate ship 153 | aircraft carrier, carrier, flattop, attack aircraft carrier 154 | submarine, pigboat, sub, U-boat 155 | wreck 156 | half track 157 | tank, army tank, armored combat vehicle, armoured combat vehicle 158 | missile 159 | bobsled, bobsleigh, bob 160 | dogsled, dog sled, dog sleigh 161 | bicycle-built-for-two, tandem bicycle, tandem 162 | mountain bike, all-terrain bike, off-roader 163 | freight car 164 | passenger car, coach, carriage 165 | barrow, garden cart, lawn cart, wheelbarrow 166 | shopping cart 167 | motor scooter, scooter 168 | forklift 169 | electric locomotive 170 | steam locomotive 171 | amphibian, amphibious vehicle 172 | ambulance 173 | beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon 174 | cab, hack, taxi, taxicab 175 | convertible 176 | jeep, landrover 177 | limousine, limo 178 | minivan 179 | Model T 180 | racer, race car, racing car 181 | sports car, sport car 182 | go-kart 183 | golfcart, golf cart 184 | moped 185 | snowplow, snowplough 186 | fire engine, fire truck 187 | garbage truck, dustcart 188 | pickup, pickup truck 189 | tow truck, tow car, wrecker 190 | trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi 191 | moving van 192 | police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria 193 | recreational vehicle, RV, R.V. 194 | streetcar, tram, tramcar, trolley, trolley car 195 | snowmobile 196 | tractor 197 | mobile home, manufactured home 198 | tricycle, trike, velocipede 199 | unicycle, monocycle 200 | horse cart, horse-cart 201 | mosquito net 202 | oxcart 203 | bassinet 204 | cradle 205 | crib, cot 206 | four-poster 207 | bookcase 208 | china cabinet, china closet 209 | medicine chest, medicine cabinet 210 | chiffonier, commode 211 | table lamp 212 | file, file cabinet, filing cabinet 213 | pay-phone, pay-station 214 | park bench 215 | barber chair 216 | throne 217 | folding chair 218 | rocking chair, rocker 219 | studio couch, day bed 220 | toilet seat 221 | desk 222 | pool table, billiard table, snooker table 223 | dining table, board 224 | entertainment center 225 | wardrobe, closet, press 226 | Granny Smith 227 | orange 228 | lemon 229 | fig 230 | pineapple, ananas 231 | banana 232 | jackfruit, jak, jack 233 | custard apple 234 | pomegranate 235 | acorn 236 | hip, rose hip, rosehip 237 | ear, spike, capitulum 238 | rapeseed 239 | corn 240 | buckeye, horse chestnut, conker 241 | organ, pipe organ 242 | upright, upright piano 243 | chime, bell, gong 244 | drum, membranophone, tympan 245 | gong, tam-tam 246 | maraca 247 | marimba, xylophone 248 | steel drum 249 | banjo 250 | cello, violoncello 251 | lampshade, lamp shade 252 | harp 253 | acoustic guitar 254 | electric guitar 255 | cornet, horn, trumpet, trump 256 | French horn, horn 257 | trombone 258 | harmonica, mouth organ, harp, mouth harp 259 | ocarina, sweet potato 260 | panpipe, pandean pipe, syrinx 261 | bassoon 262 | sax, saxophone 263 | flute, transverse flute 264 | daisy 265 | yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum 266 | cliff, drop, drop-off 267 | valley, vale 268 | alp 269 | volcano 270 | promontory, headland, head, foreland 271 | sandbar, sand bar 272 | coral reef 273 | lakeside, lakeshore 274 | seashore, coast, seacoast, sea-coast 275 | geyser 276 | hatchet 277 | cleaver, meat cleaver, chopper 278 | letter opener, paper knife, paperknife 279 | plane, carpenter's plane, woodworking plane 280 | power drill 281 | lawn mower, mower 282 | hammer 283 | corkscrew, bottle screw 284 | can opener, tin opener 285 | plunger, plumber's helper 286 | screwdriver 287 | shovel 288 | plow, plough 289 | chain saw, chainsaw 290 | cock 291 | hen 292 | ostrich, Struthio camelus 293 | brambling, Fringilla montifringilla 294 | goldfinch, Carduelis carduelis 295 | house finch, linnet, Carpodacus mexicanus 296 | junco, snowbird 297 | indigo bunting, indigo finch, indigo bird, Passerina cyanea 298 | robin, American robin, Turdus migratorius 299 | bulbul 300 | jay 301 | magpie 302 | chickadee 303 | water ouzel, dipper 304 | kite 305 | bald eagle, American eagle, Haliaeetus leucocephalus 306 | vulture 307 | great grey owl, great gray owl, Strix nebulosa 308 | black grouse 309 | ptarmigan 310 | ruffed grouse, partridge, Bonasa umbellus 311 | prairie chicken, prairie grouse, prairie fowl 312 | peacock 313 | quail 314 | partridge 315 | African grey, African gray, Psittacus erithacus 316 | macaw 317 | sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita 318 | lorikeet 319 | coucal 320 | bee eater 321 | hornbill 322 | hummingbird 323 | jacamar 324 | toucan 325 | drake 326 | red-breasted merganser, Mergus serrator 327 | goose 328 | black swan, Cygnus atratus 329 | white stork, Ciconia ciconia 330 | black stork, Ciconia nigra 331 | spoonbill 332 | flamingo 333 | American egret, great white heron, Egretta albus 334 | little blue heron, Egretta caerulea 335 | bittern 336 | crane 337 | limpkin, Aramus pictus 338 | American coot, marsh hen, mud hen, water hen, Fulica americana 339 | bustard 340 | ruddy turnstone, Arenaria interpres 341 | red-backed sandpiper, dunlin, Erolia alpina 342 | redshank, Tringa totanus 343 | dowitcher 344 | oystercatcher, oyster catcher 345 | European gallinule, Porphyrio porphyrio 346 | pelican 347 | king penguin, Aptenodytes patagonica 348 | albatross, mollymawk 349 | great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias 350 | tiger shark, Galeocerdo cuvieri 351 | hammerhead, hammerhead shark 352 | electric ray, crampfish, numbfish, torpedo 353 | stingray 354 | barracouta, snoek 355 | coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch 356 | tench, Tinca tinca 357 | goldfish, Carassius auratus 358 | eel 359 | rock beauty, Holocanthus tricolor 360 | anemone fish 361 | lionfish 362 | puffer, pufferfish, blowfish, globefish 363 | sturgeon 364 | gar, garfish, garpike, billfish, Lepisosteus osseus 365 | loggerhead, loggerhead turtle, Caretta caretta 366 | mud turtle 367 | terrapin 368 | box turtle, box tortoise 369 | banded gecko 370 | common iguana, iguana, Iguana iguana 371 | American chameleon, anole, Anolis carolinensis 372 | whiptail, whiptail lizard 373 | agama 374 | frilled lizard, Chlamydosaurus kingi 375 | alligator lizard 376 | Gila monster, Heloderma suspectum 377 | green lizard, Lacerta viridis 378 | African chameleon, Chamaeleo chamaeleon 379 | Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis 380 | triceratops 381 | African crocodile, Nile crocodile, Crocodylus niloticus 382 | American alligator, Alligator mississipiensis 383 | thunder snake, worm snake, Carphophis amoenus 384 | ringneck snake, ring-necked snake, ring snake 385 | hognose snake, puff adder, sand viper 386 | green snake, grass snake 387 | king snake, kingsnake 388 | garter snake, grass snake 389 | water snake 390 | vine snake 391 | night snake, Hypsiglena torquata 392 | boa constrictor, Constrictor constrictor 393 | rock python, rock snake, Python sebae 394 | Indian cobra, Naja naja 395 | green mamba 396 | sea snake 397 | horned viper, cerastes, sand viper, horned asp, Cerastes cornutus 398 | stone wall 399 | sidewinder, horned rattlesnake, Crotalus cerastes 400 | European fire salamander, Salamandra salamandra 401 | common newt, Triturus vulgaris 402 | eft 403 | spotted salamander, Ambystoma maculatum 404 | axolotl, mud puppy, Ambystoma mexicanum 405 | bullfrog, Rana catesbeiana 406 | tree frog, tree-frog 407 | tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui 408 | whistle 409 | wing 410 | paintbrush 411 | oxygen mask 412 | snorkel 413 | loudspeaker, speaker, speaker unit, loudspeaker system, speaker system 414 | microphone, mike 415 | screen, CRT screen 416 | mouse, computer mouse 417 | electric fan, blower 418 | oil filter 419 | strainer 420 | space heater 421 | stove 422 | guillotine 423 | barometer 424 | rule, ruler 425 | odometer, hodometer, mileometer, milometer 426 | scale, weighing machine 427 | analog clock 428 | digital clock 429 | wall clock 430 | hourglass 431 | sundial 432 | parking meter 433 | stopwatch, stop watch 434 | digital watch 435 | stethoscope 436 | syringe 437 | magnetic compass 438 | binoculars, field glasses, opera glasses 439 | projector 440 | sunglasses, dark glasses, shades 441 | loupe, jeweler's loupe 442 | radio telescope, radio reflector 443 | bow 444 | cannon 445 | assault rifle, assault gun 446 | rifle 447 | projectile, missile 448 | computer keyboard, keypad 449 | typewriter keyboard 450 | crane 451 | lighter, light, igniter, ignitor 452 | abacus 453 | cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM 454 | slide rule, slipstick 455 | desktop computer 456 | hand-held computer, hand-held microcomputer 457 | notebook, notebook computer 458 | padlock 459 | harvester, reaper 460 | thresher, thrasher, threshing machine 461 | printer 462 | slot, one-armed bandit 463 | vending machine 464 | sewing machine 465 | joystick 466 | switch, electric switch, electrical switch 467 | hook, claw 468 | car wheel 469 | paddlewheel, paddle wheel 470 | pinwheel 471 | potter's wheel 472 | gas pump, gasoline pump, petrol pump, island dispenser 473 | carousel, carrousel, merry-go-round, roundabout, whirligig 474 | swing 475 | reel 476 | radiator 477 | puck, hockey puck 478 | hard disc, hard disk, fixed disk 479 | sunglass 480 | pick, plectrum, plectron 481 | car mirror 482 | solar dish, solar collector, solar furnace 483 | remote control, remote 484 | disk brake, disc brake 485 | buckle 486 | hair slide 487 | knot 488 | combination lock 489 | web site, website, internet site, site 490 | nail 491 | safety pin 492 | screw 493 | muzzle 494 | seat belt, seatbelt 495 | ski 496 | candle, taper, wax light 497 | jack-o'-lantern 498 | spotlight, spot 499 | torch 500 | neck brace 501 | pier 502 | tripod 503 | maypole 504 | hand blower, blow dryer, blow drier, hair dryer, hair drier 505 | mousetrap 506 | spider web, spider's web 507 | trilobite 508 | harvestman, daddy longlegs, Phalangium opilio 509 | scorpion 510 | black and gold garden spider, Argiope aurantia 511 | barn spider, Araneus cavaticus 512 | garden spider, Aranea diademata 513 | black widow, Latrodectus mactans 514 | tarantula 515 | wolf spider, hunting spider 516 | tick 517 | centipede 518 | isopod 519 | Dungeness crab, Cancer magister 520 | rock crab, Cancer irroratus 521 | fiddler crab 522 | king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica 523 | American lobster, Northern lobster, Maine lobster, Homarus americanus 524 | spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish 525 | crayfish, crawfish, crawdad, crawdaddy 526 | hermit crab 527 | tiger beetle 528 | ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle 529 | ground beetle, carabid beetle 530 | long-horned beetle, longicorn, longicorn beetle 531 | leaf beetle, chrysomelid 532 | dung beetle 533 | rhinoceros beetle 534 | weevil 535 | fly 536 | bee 537 | grasshopper, hopper 538 | cricket 539 | walking stick, walkingstick, stick insect 540 | cockroach, roach 541 | mantis, mantid 542 | cicada, cicala 543 | leafhopper 544 | lacewing, lacewing fly 545 | dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk 546 | damselfly 547 | admiral 548 | ringlet, ringlet butterfly 549 | monarch, monarch butterfly, milkweed butterfly, Danaus plexippus 550 | cabbage butterfly 551 | sulphur butterfly, sulfur butterfly 552 | lycaenid, lycaenid butterfly 553 | jellyfish 554 | sea anemone, anemone 555 | brain coral 556 | flatworm, platyhelminth 557 | nematode, nematode worm, roundworm 558 | conch 559 | snail 560 | slug 561 | sea slug, nudibranch 562 | chiton, coat-of-mail shell, sea cradle, polyplacophore 563 | sea urchin 564 | sea cucumber, holothurian 565 | iron, smoothing iron 566 | espresso maker 567 | microwave, microwave oven 568 | Dutch oven 569 | rotisserie 570 | toaster 571 | waffle iron 572 | vacuum, vacuum cleaner 573 | dishwasher, dish washer, dishwashing machine 574 | refrigerator, icebox 575 | washer, automatic washer, washing machine 576 | Crock Pot 577 | frying pan, frypan, skillet 578 | wok 579 | caldron, cauldron 580 | coffeepot 581 | teapot 582 | spatula 583 | altar 584 | triumphal arch 585 | patio, terrace 586 | steel arch bridge 587 | suspension bridge 588 | viaduct 589 | barn 590 | greenhouse, nursery, glasshouse 591 | palace 592 | monastery 593 | library 594 | apiary, bee house 595 | boathouse 596 | church, church building 597 | mosque 598 | stupa, tope 599 | planetarium 600 | restaurant, eating house, eating place, eatery 601 | cinema, movie theater, movie theatre, movie house, picture palace 602 | home theater, home theatre 603 | lumbermill, sawmill 604 | coil, spiral, volute, whorl, helix 605 | obelisk 606 | totem pole 607 | castle 608 | prison, prison house 609 | grocery store, grocery, food market, market 610 | bakery, bakeshop, bakehouse 611 | barbershop 612 | bookshop, bookstore, bookstall 613 | butcher shop, meat market 614 | confectionery, confectionary, candy store 615 | shoe shop, shoe-shop, shoe store 616 | tobacco shop, tobacconist shop, tobacconist 617 | toyshop 618 | fountain 619 | cliff dwelling 620 | yurt 621 | dock, dockage, docking facility 622 | brass, memorial tablet, plaque 623 | megalith, megalithic structure 624 | bannister, banister, balustrade, balusters, handrail 625 | breakwater, groin, groyne, mole, bulwark, seawall, jetty 626 | dam, dike, dyke 627 | chainlink fence 628 | picket fence, paling 629 | worm fence, snake fence, snake-rail fence, Virginia fence 630 | diamondback, diamondback rattlesnake, Crotalus adamanteus 631 | grille, radiator grille 632 | sliding door 633 | turnstile 634 | mountain tent 635 | scoreboard 636 | honeycomb 637 | plate rack 638 | pedestal, plinth, footstall 639 | beacon, lighthouse, beacon light, pharos 640 | leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea 641 | mashed potato 642 | bell pepper 643 | head cabbage 644 | broccoli 645 | cauliflower 646 | zucchini, courgette 647 | spaghetti squash 648 | acorn squash 649 | butternut squash 650 | cucumber, cuke 651 | artichoke, globe artichoke 652 | cardoon 653 | mushroom 654 | shower curtain 655 | jean, blue jean, denim 656 | carton 657 | handkerchief, hankie, hanky, hankey 658 | sandal 659 | ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin 660 | safe 661 | plate 662 | necklace 663 | croquet ball 664 | fur coat 665 | thimble 666 | pajama, pyjama, pj's, jammies 667 | running shoe 668 | oboe, hautboy, hautbois 669 | cocktail shaker 670 | chest 671 | manhole cover 672 | modem 673 | tub, vat 674 | tray 675 | balance beam, beam 676 | bagel, beigel 677 | violin, fiddle 678 | prayer rug, prayer mat 679 | kimono 680 | hot pot, hotpot 681 | whiskey jug 682 | knee pad 683 | book jacket, dust cover, dust jacket, dust wrapper 684 | spindle 685 | ski mask 686 | beer bottle 687 | crash helmet 688 | bottlecap 689 | tile roof 690 | mask 691 | maillot 692 | Petri dish 693 | football helmet 694 | bathing cap, swimming cap 695 | teddy, teddy bear 696 | holster 697 | pop bottle, soda bottle 698 | photocopier 699 | vestment 700 | crossword puzzle, crossword 701 | golf ball 702 | trifle 703 | suit, suit of clothes 704 | water tower 705 | feather boa, boa 706 | cloak 707 | red wine 708 | drumstick 709 | shield, buckler 710 | Christmas stocking 711 | hoopskirt, crinoline 712 | menu 713 | stage 714 | bonnet, poke bonnet 715 | meat loaf, meatloaf 716 | baseball 717 | face powder 718 | scabbard 719 | sunscreen, sunblock, sun blocker 720 | beer glass 721 | hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa 722 | guacamole 723 | wool, woolen, woollen 724 | hay 725 | bow tie, bow-tie, bowtie 726 | mailbag, postbag 727 | water jug 728 | bucket, pail 729 | dishrag, dishcloth 730 | soup bowl 731 | eggnog 732 | mortar 733 | trench coat 734 | paddle, boat paddle 735 | chain 736 | swab, swob, mop 737 | mixing bowl 738 | potpie 739 | wine bottle 740 | shoji 741 | bulletproof vest 742 | drilling platform, offshore rig 743 | binder, ring-binder 744 | cardigan 745 | sweatshirt 746 | pot, flowerpot 747 | birdhouse 748 | jinrikisha, ricksha, rickshaw 749 | hamper 750 | ping-pong ball 751 | pencil box, pencil case 752 | consomme 753 | apron 754 | punching bag, punch bag, punching ball, punchball 755 | backpack, back pack, knapsack, packsack, rucksack, haversack 756 | groom, bridegroom 757 | bearskin, busby, shako 758 | pencil sharpener 759 | broom 760 | abaya 761 | mortarboard 762 | poncho 763 | crutch 764 | Polaroid camera, Polaroid Land camera 765 | space bar 766 | cup 767 | racket, racquet 768 | traffic light, traffic signal, stoplight 769 | quill, quill pen 770 | radio, wireless 771 | snow leopard, ounce, Panthera uncia 772 | dough 773 | cuirass 774 | military uniform 775 | lipstick, lip rouge 776 | shower cap 777 | monitor 778 | oscilloscope, scope, cathode-ray oscilloscope, CRO 779 | mitten 780 | brassiere, bra, bandeau 781 | French loaf 782 | vase 783 | milk can 784 | rugby ball 785 | paper towel 786 | earthstar 787 | envelope 788 | miniskirt, mini 789 | cowboy hat, ten-gallon hat 790 | trolleybus, trolley coach, trackless trolley 791 | perfume, essence 792 | bathtub, bathing tub, bath, tub 793 | hotdog, hot dog, red hot 794 | coral fungus 795 | bullet train, bullet 796 | pillow 797 | toilet tissue, toilet paper, bathroom tissue 798 | cassette 799 | carpenter's kit, tool kit 800 | ladle 801 | stinkhorn, carrion fungus 802 | lotion 803 | hair spray 804 | academic gown, academic robe, judge's robe 805 | dome 806 | crate 807 | wig 808 | burrito 809 | pill bottle 810 | chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour 811 | theater curtain, theatre curtain 812 | window shade 813 | barrel, cask 814 | washbasin, handbasin, washbowl, lavabo, wash-hand basin 815 | ballpoint, ballpoint pen, ballpen, Biro 816 | basketball 817 | bath towel 818 | cowboy boot 819 | gown 820 | window screen 821 | agaric 822 | standard poodle 823 | cellular telephone, cellular phone, cellphone, cell, mobile phone 824 | nipple 825 | barbell 826 | mailbox, letter box 827 | lab coat, laboratory coat 828 | fire screen, fireguard 829 | minibus 830 | packet 831 | brown bear, bruin, Ursus arctos 832 | pole 833 | horizontal bar, high bar 834 | sombrero 835 | pickelhaube 836 | rain barrel 837 | wallet, billfold, notecase, pocketbook 838 | cassette player 839 | comic book 840 | piggy bank, penny bank 841 | street sign 842 | bell cote, bell cot 843 | fountain pen 844 | Windsor tie 845 | volleyball 846 | overskirt 847 | sarong 848 | purse 849 | bolo tie, bolo, bola tie, bola 850 | bib 851 | parachute, chute 852 | sleeping bag 853 | television, television system 854 | swimming trunks, bathing trunks 855 | measuring cup 856 | espresso 857 | pizza, pizza pie 858 | breastplate, aegis, egis 859 | shopping basket 860 | wooden spoon 861 | saltshaker, salt shaker 862 | chocolate sauce, chocolate syrup 863 | ballplayer, baseball player 864 | goblet 865 | gyromitra 866 | stretcher 867 | water bottle 868 | skunk, polecat, wood pussy 869 | soap dispenser 870 | jersey, T-shirt, tee shirt 871 | school bus 872 | jigsaw puzzle 873 | plastic bag 874 | reflex camera 875 | diaper, nappy, napkin 876 | Band Aid 877 | ice lolly, lolly, lollipop, popsicle 878 | velvet 879 | tennis ball 880 | gasmask, respirator, gas helmet 881 | doormat, welcome mat 882 | Loafer 883 | ice cream, icecream 884 | pretzel 885 | quilt, comforter, comfort, puff 886 | maillot, tank suit 887 | tape player 888 | clog, geta, patten, sabot 889 | iPod 890 | bolete 891 | meerkat, mierkat 892 | scuba diver 893 | pitcher, ewer 894 | matchstick 895 | bikini, two-piece 896 | sock 897 | CD player 898 | lens cap, lens cover 899 | thatch, thatched roof 900 | vault 901 | beaker 902 | bubble 903 | cheeseburger 904 | parallel bars, bars 905 | flagpole, flagstaff 906 | coffee mug 907 | rubber eraser, rubber, pencil eraser 908 | stole 909 | carbonara 910 | dumbbell 911 | Siberian husky 912 | English springer, English springer spaniel 913 | malamute, malemute, Alaskan malamute 914 | Walker hound, Walker foxhound 915 | Welsh springer spaniel 916 | whippet 917 | Weimaraner 918 | soft-coated wheaten terrier 919 | Dandie Dinmont, Dandie Dinmont terrier 920 | Old English sheepdog, bobtail 921 | otterhound, otter hound 922 | bloodhound, sleuthhound 923 | Airedale, Airedale terrier 924 | giant schnauzer 925 | black-and-tan coonhound 926 | papillon 927 | Mexican hairless 928 | Cardigan, Cardigan Welsh corgi 929 | malinois 930 | Lhasa, Lhasa apso 931 | Norwegian elkhound, elkhound 932 | Rottweiler 933 | Saluki, gazelle hound 934 | schipperke 935 | Brabancon griffon 936 | West Highland white terrier 937 | Sealyham terrier, Sealyham 938 | Irish wolfhound 939 | EntleBucher 940 | French bulldog 941 | Bernese mountain dog 942 | Maltese dog, Maltese terrier, Maltese 943 | Norfolk terrier 944 | toy terrier 945 | cairn, cairn terrier 946 | groenendael 947 | clumber, clumber spaniel 948 | Afghan hound, Afghan 949 | Japanese spaniel 950 | borzoi, Russian wolfhound 951 | toy poodle 952 | Kerry blue terrier 953 | Scotch terrier, Scottish terrier, Scottie 954 | Boston bull, Boston terrier 955 | Greater Swiss Mountain dog 956 | Appenzeller 957 | Shih-Tzu 958 | Irish water spaniel 959 | Pomeranian 960 | Bedlington terrier 961 | miniature schnauzer 962 | collie 963 | Irish terrier 964 | affenpinscher, monkey pinscher, monkey dog 965 | silky terrier, Sydney silky 966 | beagle 967 | Leonberg 968 | German short-haired pointer 969 | dhole, Cuon alpinus 970 | Chesapeake Bay retriever 971 | bull mastiff 972 | kuvasz 973 | pug, pug-dog 974 | curly-coated retriever 975 | Norwich terrier 976 | keeshond 977 | Lakeland terrier 978 | standard schnauzer 979 | Tibetan terrier, chrysanthemum dog 980 | wire-haired fox terrier 981 | basset, basset hound 982 | chow, chow chow 983 | American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier 984 | Shetland sheepdog, Shetland sheep dog, Shetland 985 | Great Pyrenees 986 | Chihuahua 987 | Labrador retriever 988 | Samoyed, Samoyede 989 | bluetick 990 | kelpie 991 | miniature pinscher 992 | Italian greyhound 993 | cocker spaniel, English cocker spaniel, cocker 994 | Sussex spaniel 995 | Pembroke, Pembroke Welsh corgi 996 | Blenheim spaniel 997 | Ibizan hound, Ibizan Podenco 998 | English foxhound 999 | briard 1000 | Border terrier 1001 | -------------------------------------------------------------------------------- /mappings/objectnet_to_imagenet_1k.json: -------------------------------------------------------------------------------- 1 | { 2 | "Alarm clock": "analog clock; digital clock", 3 | "Backpack": "backpack, back pack, knapsack, packsack, rucksack, haversack", 4 | "Banana": "banana", 5 | "Band Aid": "Band Aid", 6 | "Basket": "shopping basket", 7 | "Bath towel": "bath towel", 8 | "Beer bottle": "beer bottle", 9 | "Bench": "park bench", 10 | "Bicycle": "mountain bike, all-terrain bike, off-roader; bicycle-built-for-two, tandem bicycle, tandem", 11 | "Binder (closed)": "binder, ring-binder", 12 | "Bottle cap": "bottlecap", 13 | "Bread loaf": "French loaf", 14 | "Broom": "broom", 15 | "Bucket": "bucket, pail", 16 | "Butcher's knife": "cleaver, meat cleaver, chopper", 17 | "Can opener": "can opener, tin opener", 18 | "Candle": "candle, taper, wax light", 19 | "Cellphone": "cellular telephone, cellular phone, cellphone, cell, mobile phone", 20 | "Chair": "barber chair; folding chair; rocking chair, rocker", 21 | "Clothes hamper": "hamper", 22 | "Coffee/French press": "espresso maker", 23 | "Combination lock": "combination lock", 24 | "Computer mouse": "mouse, computer mouse", 25 | "Desk lamp": "table lamp", 26 | "Dishrag or hand towel": "dishrag, dishcloth", 27 | "Doormat": "doormat, welcome mat", 28 | "Dress shoe (men)": "Loafer", 29 | "Drill": "power drill", 30 | "Drinking Cup": "cup", 31 | "Drying rack for plates": "plate rack", 32 | "Envelope": "envelope", 33 | "Fan": "electric fan, blower", 34 | "Frying pan": "frying pan, frypan, skillet", 35 | "Dress": "gown", 36 | "Hair dryer": "hand blower, blow dryer, blow drier, hair dryer, hair drier", 37 | "Hammer": "hammer", 38 | "Helmet": "football helmet; crash helmet", 39 | "Iron (for clothes)": "iron, smoothing iron", 40 | "Jeans": "jean, blue jean, denim", 41 | "Keyboard": "computer keyboard, keypad", 42 | "Ladle": "ladle", 43 | "Lampshade": "lampshade, lamp shade", 44 | "Laptop (open)": "laptop, laptop computer", 45 | "Lemon": "lemon", 46 | "Letter opener": "letter opener, paper knife, paperknife", 47 | "Lighter": "lighter, light, igniter, ignitor", 48 | "Lipstick": "lipstick, lip rouge", 49 | "Match": "matchstick", 50 | "Measuring cup": "measuring cup", 51 | "Microwave": "microwave, microwave oven", 52 | "Mixing / Salad Bowl": "mixing bowl", 53 | "Monitor": "monitor", 54 | "Mug": "coffee mug", 55 | "Nail (fastener)": "nail", 56 | "Necklace": "necklace", 57 | "Orange": "orange", 58 | "Padlock": "padlock", 59 | "Paintbrush": "paintbrush", 60 | "Paper towel": "paper towel", 61 | "Pen": "ballpoint, ballpoint pen, ballpen, Biro; quill, quill pen; fountain pen", 62 | "Pill bottle": "pill bottle", 63 | "Pillow": "pillow", 64 | "Pitcher": "pitcher, ewer", 65 | "Plastic bag": "plastic bag", 66 | "Plate": "plate", 67 | "Plunger": "plunger, plumber's helper", 68 | "Pop can": "pop bottle, soda bottle", 69 | "Portable heater": "space heater", 70 | "Printer": "printer", 71 | "Remote control": "remote control, remote", 72 | "Ruler": "rule, ruler", 73 | "Running shoe": "running shoe", 74 | "Safety pin": "safety pin", 75 | "Salt shaker": "saltshaker, salt shaker", 76 | "Sandal": "sandal", 77 | "Screw": "screw", 78 | "Shovel": "shovel", 79 | "Skirt": "hoopskirt, crinoline; miniskirt, mini; overskirt", 80 | "Sleeping bag": "sleeping bag", 81 | "Soap dispenser": "soap dispenser", 82 | "Sock": "sock", 83 | "Soup Bowl": "soup bowl", 84 | "Spatula": "spatula", 85 | "Speaker": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system", 86 | "Still Camera": "Polaroid camera, Polaroid Land camera; reflex camera", 87 | "Strainer": "strainer", 88 | "Stuffed animal": "teddy, teddy bear", 89 | "Suit jacket": "suit, suit of clothes", 90 | "Sunglasses": "sunglasses, dark glasses, shades", 91 | "Sweater": "sweatshirt", 92 | "Swimming trunks": "swimming trunks, bathing trunks", 93 | "T-shirt": "jersey, T-shirt, tee shirt", 94 | "TV": "television, television system", 95 | "Teapot": "teapot", 96 | "Tennis racket": "racket, racquet", 97 | "Tie": "bow tie, bow-tie, bowtie; Windsor tie", 98 | "Toaster": "toaster", 99 | "Toilet paper roll": "toilet tissue, toilet paper, bathroom tissue", 100 | "Trash bin": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin", 101 | "Tray": "tray", 102 | "Umbrella": "umbrella", 103 | "Vacuum cleaner": "vacuum, vacuum cleaner", 104 | "Vase": "vase", 105 | "Wallet": "wallet, billfold, notecase, pocketbook", 106 | "Watch": "digital watch", 107 | "Water bottle": "water bottle", 108 | "Weight (exercise)": "dumbbell", 109 | "Weight scale": "scale, weighing machine", 110 | "Wheel": "car wheel; paddlewheel, paddle wheel", 111 | "Whistle": "whistle", 112 | "Wine bottle": "wine bottle", 113 | "Winter glove": "mitten", 114 | "Wok": "wok" 115 | } 116 | -------------------------------------------------------------------------------- /mappings/pytorch_to_imagenet_2012_id.json: -------------------------------------------------------------------------------- 1 | { 2 | "0": 355, 3 | "1": 356, 4 | "2": 348, 5 | "3": 349, 6 | "4": 350, 7 | "5": 351, 8 | "6": 352, 9 | "7": 289, 10 | "8": 290, 11 | "9": 291, 12 | "10": 292, 13 | "11": 293, 14 | "12": 294, 15 | "13": 295, 16 | "14": 296, 17 | "15": 297, 18 | "16": 298, 19 | "17": 299, 20 | "18": 300, 21 | "19": 301, 22 | "20": 302, 23 | "21": 303, 24 | "22": 304, 25 | "23": 305, 26 | "24": 306, 27 | "25": 399, 28 | "26": 400, 29 | "27": 401, 30 | "28": 402, 31 | "29": 403, 32 | "30": 404, 33 | "31": 405, 34 | "32": 406, 35 | "33": 364, 36 | "34": 639, 37 | "35": 365, 38 | "36": 366, 39 | "37": 367, 40 | "38": 368, 41 | "39": 369, 42 | "40": 370, 43 | "41": 371, 44 | "42": 372, 45 | "43": 373, 46 | "44": 374, 47 | "45": 375, 48 | "46": 376, 49 | "47": 377, 50 | "48": 378, 51 | "49": 380, 52 | "50": 381, 53 | "51": 379, 54 | "52": 382, 55 | "53": 383, 56 | "54": 384, 57 | "55": 385, 58 | "56": 386, 59 | "57": 387, 60 | "58": 388, 61 | "59": 389, 62 | "60": 390, 63 | "61": 391, 64 | "62": 392, 65 | "63": 393, 66 | "64": 394, 67 | "65": 395, 68 | "66": 396, 69 | "67": 629, 70 | "68": 398, 71 | "69": 506, 72 | "70": 507, 73 | "71": 508, 74 | "72": 509, 75 | "73": 510, 76 | "74": 511, 77 | "75": 512, 78 | "76": 513, 79 | "77": 514, 80 | "78": 515, 81 | "79": 516, 82 | "80": 307, 83 | "81": 308, 84 | "82": 309, 85 | "83": 310, 86 | "84": 311, 87 | "85": 312, 88 | "86": 313, 89 | "87": 314, 90 | "88": 315, 91 | "89": 316, 92 | "90": 317, 93 | "91": 318, 94 | "92": 319, 95 | "93": 320, 96 | "94": 321, 97 | "95": 322, 98 | "96": 323, 99 | "97": 324, 100 | "98": 325, 101 | "99": 326, 102 | "100": 327, 103 | "101": 120, 104 | "102": 121, 105 | "103": 123, 106 | "104": 122, 107 | "105": 119, 108 | "106": 124, 109 | "107": 552, 110 | "108": 553, 111 | "109": 554, 112 | "110": 555, 113 | "111": 556, 114 | "112": 557, 115 | "113": 558, 116 | "114": 559, 117 | "115": 560, 118 | "116": 561, 119 | "117": 132, 120 | "118": 518, 121 | "119": 519, 122 | "120": 520, 123 | "121": 521, 124 | "122": 522, 125 | "123": 523, 126 | "124": 524, 127 | "125": 525, 128 | "126": 517, 129 | "127": 328, 130 | "128": 329, 131 | "129": 330, 132 | "130": 331, 133 | "131": 333, 134 | "132": 332, 135 | "133": 334, 136 | "134": 449, 137 | "135": 336, 138 | "136": 344, 139 | "137": 337, 140 | "138": 338, 141 | "139": 339, 142 | "140": 340, 143 | "141": 341, 144 | "142": 342, 145 | "143": 343, 146 | "144": 345, 147 | "145": 346, 148 | "146": 347, 149 | "147": 3, 150 | "148": 15, 151 | "149": 106, 152 | "150": 11, 153 | "151": 985, 154 | "152": 948, 155 | "153": 941, 156 | "154": 41, 157 | "155": 956, 158 | "156": 995, 159 | "157": 925, 160 | "158": 943, 161 | "159": 110, 162 | "160": 947, 163 | "161": 980, 164 | "162": 965, 165 | "163": 921, 166 | "164": 988, 167 | "165": 924, 168 | "166": 913, 169 | "167": 997, 170 | "168": 98, 171 | "169": 949, 172 | "170": 937, 173 | "171": 991, 174 | "172": 915, 175 | "173": 996, 176 | "174": 930, 177 | "175": 920, 178 | "176": 932, 179 | "177": 14, 180 | "178": 916, 181 | "179": 26, 182 | "180": 982, 183 | "181": 959, 184 | "182": 999, 185 | "183": 951, 186 | "184": 962, 187 | "185": 942, 188 | "186": 974, 189 | "187": 36, 190 | "188": 979, 191 | "189": 976, 192 | "190": 936, 193 | "191": 922, 194 | "192": 944, 195 | "193": 2, 196 | "194": 918, 197 | "195": 953, 198 | "196": 960, 199 | "197": 923, 200 | "198": 977, 201 | "199": 952, 202 | "200": 978, 203 | "201": 964, 204 | "202": 917, 205 | "203": 935, 206 | "204": 929, 207 | "205": 77, 208 | "206": 973, 209 | "207": 69, 210 | "208": 986, 211 | "209": 969, 212 | "210": 967, 213 | "211": 52, 214 | "212": 1, 215 | "213": 105, 216 | "214": 83, 217 | "215": 80, 218 | "216": 946, 219 | "217": 911, 220 | "218": 914, 221 | "219": 992, 222 | "220": 993, 223 | "221": 957, 224 | "222": 971, 225 | "223": 933, 226 | "224": 945, 227 | "225": 928, 228 | "226": 998, 229 | "227": 989, 230 | "228": 56, 231 | "229": 919, 232 | "230": 983, 233 | "231": 961, 234 | "232": 70, 235 | "233": 27, 236 | "234": 931, 237 | "235": 117, 238 | "236": 65, 239 | "237": 990, 240 | "238": 954, 241 | "239": 940, 242 | "240": 955, 243 | "241": 938, 244 | "242": 72, 245 | "243": 970, 246 | "244": 63, 247 | "245": 939, 248 | "246": 13, 249 | "247": 96, 250 | "248": 79, 251 | "249": 912, 252 | "250": 910, 253 | "251": 25, 254 | "252": 963, 255 | "253": 50, 256 | "254": 972, 257 | "255": 966, 258 | "256": 37, 259 | "257": 984, 260 | "258": 987, 261 | "259": 958, 262 | "260": 981, 263 | "261": 975, 264 | "262": 934, 265 | "263": 994, 266 | "264": 927, 267 | "265": 950, 268 | "266": 29, 269 | "267": 821, 270 | "268": 926, 271 | "269": 114, 272 | "270": 59, 273 | "271": 18, 274 | "272": 35, 275 | "273": 84, 276 | "274": 968, 277 | "275": 112, 278 | "276": 20, 279 | "277": 38, 280 | "278": 0, 281 | "279": 86, 282 | "280": 40, 283 | "281": 94, 284 | "282": 33, 285 | "283": 7, 286 | "284": 54, 287 | "285": 5, 288 | "286": 8, 289 | "287": 111, 290 | "288": 51, 291 | "289": 770, 292 | "290": 19, 293 | "291": 104, 294 | "292": 45, 295 | "293": 115, 296 | "294": 830, 297 | "295": 88, 298 | "296": 60, 299 | "297": 116, 300 | "298": 43, 301 | "299": 890, 302 | "300": 526, 303 | "301": 527, 304 | "302": 528, 305 | "303": 529, 306 | "304": 530, 307 | "305": 531, 308 | "306": 532, 309 | "307": 533, 310 | "308": 534, 311 | "309": 535, 312 | "310": 130, 313 | "311": 536, 314 | "312": 537, 315 | "313": 538, 316 | "314": 539, 317 | "315": 540, 318 | "316": 541, 319 | "317": 542, 320 | "318": 543, 321 | "319": 544, 322 | "320": 545, 323 | "321": 546, 324 | "322": 547, 325 | "323": 548, 326 | "324": 549, 327 | "325": 550, 328 | "326": 551, 329 | "327": 131, 330 | "328": 562, 331 | "329": 563, 332 | "330": 103, 333 | "331": 71, 334 | "332": 89, 335 | "333": 85, 336 | "334": 10, 337 | "335": 31, 338 | "336": 100, 339 | "337": 108, 340 | "338": 58, 341 | "339": 23, 342 | "340": 47, 343 | "341": 78, 344 | "342": 46, 345 | "343": 66, 346 | "344": 92, 347 | "345": 62, 348 | "346": 87, 349 | "347": 90, 350 | "348": 48, 351 | "349": 30, 352 | "350": 6, 353 | "351": 39, 354 | "352": 34, 355 | "353": 9, 356 | "354": 67, 357 | "355": 102, 358 | "356": 28, 359 | "357": 16, 360 | "358": 99, 361 | "359": 24, 362 | "360": 118, 363 | "361": 867, 364 | "362": 12, 365 | "363": 97, 366 | "364": 22, 367 | "365": 49, 368 | "366": 61, 369 | "367": 55, 370 | "368": 101, 371 | "369": 68, 372 | "370": 42, 373 | "371": 73, 374 | "372": 74, 375 | "373": 75, 376 | "374": 113, 377 | "375": 32, 378 | "376": 57, 379 | "377": 95, 380 | "378": 76, 381 | "379": 91, 382 | "380": 21, 383 | "381": 64, 384 | "382": 53, 385 | "383": 109, 386 | "384": 44, 387 | "385": 107, 388 | "386": 17, 389 | "387": 4, 390 | "388": 93, 391 | "389": 353, 392 | "390": 357, 393 | "391": 354, 394 | "392": 358, 395 | "393": 359, 396 | "394": 362, 397 | "395": 363, 398 | "396": 360, 399 | "397": 361, 400 | "398": 451, 401 | "399": 759, 402 | "400": 803, 403 | "401": 129, 404 | "402": 252, 405 | "403": 152, 406 | "404": 136, 407 | "405": 138, 408 | "406": 582, 409 | "407": 171, 410 | "408": 170, 411 | "409": 426, 412 | "410": 593, 413 | "411": 752, 414 | "412": 658, 415 | "413": 444, 416 | "414": 754, 417 | "415": 609, 418 | "416": 674, 419 | "417": 139, 420 | "418": 814, 421 | "419": 875, 422 | "420": 248, 423 | "421": 623, 424 | "422": 824, 425 | "423": 214, 426 | "424": 610, 427 | "425": 588, 428 | "426": 422, 429 | "427": 812, 430 | "428": 164, 431 | "429": 715, 432 | "430": 815, 433 | "431": 202, 434 | "432": 260, 435 | "433": 693, 436 | "434": 816, 437 | "435": 791, 438 | "436": 172, 439 | "437": 638, 440 | "438": 900, 441 | "439": 756, 442 | "440": 685, 443 | "441": 719, 444 | "442": 841, 445 | "443": 849, 446 | "444": 160, 447 | "445": 894, 448 | "446": 742, 449 | "447": 437, 450 | "448": 746, 451 | "449": 594, 452 | "450": 158, 453 | "451": 848, 454 | "452": 713, 455 | "453": 206, 456 | "454": 611, 457 | "455": 687, 458 | "456": 442, 459 | "457": 724, 460 | "458": 621, 461 | "459": 779, 462 | "460": 624, 463 | "461": 857, 464 | "462": 758, 465 | "463": 727, 466 | "464": 484, 467 | "465": 740, 468 | "466": 794, 469 | "467": 612, 470 | "468": 173, 471 | "469": 578, 472 | "470": 495, 473 | "471": 443, 474 | "472": 145, 475 | "473": 283, 476 | "474": 743, 477 | "475": 480, 478 | "476": 472, 479 | "477": 798, 480 | "478": 655, 481 | "479": 467, 482 | "480": 452, 483 | "481": 797, 484 | "482": 837, 485 | "483": 606, 486 | "484": 147, 487 | "485": 896, 488 | "486": 249, 489 | "487": 822, 490 | "488": 734, 491 | "489": 626, 492 | "490": 809, 493 | "491": 288, 494 | "492": 669, 495 | "493": 209, 496 | "494": 242, 497 | "495": 207, 498 | "496": 709, 499 | "497": 595, 500 | "498": 600, 501 | "499": 276, 502 | "500": 618, 503 | "501": 705, 504 | "502": 887, 505 | "503": 668, 506 | "504": 905, 507 | "505": 579, 508 | "506": 603, 509 | "507": 487, 510 | "508": 447, 511 | "509": 613, 512 | "510": 149, 513 | "511": 174, 514 | "512": 282, 515 | "513": 254, 516 | "514": 817, 517 | "515": 788, 518 | "516": 203, 519 | "517": 335, 520 | "518": 686, 521 | "519": 805, 522 | "520": 204, 523 | "521": 575, 524 | "522": 662, 525 | "523": 762, 526 | "524": 772, 527 | "525": 625, 528 | "526": 220, 529 | "527": 454, 530 | "528": 81, 531 | "529": 874, 532 | "530": 427, 533 | "531": 433, 534 | "532": 222, 535 | "533": 728, 536 | "534": 572, 537 | "535": 483, 538 | "536": 620, 539 | "537": 159, 540 | "538": 804, 541 | "539": 880, 542 | "540": 741, 543 | "541": 243, 544 | "542": 707, 545 | "543": 909, 546 | "544": 567, 547 | "545": 416, 548 | "546": 253, 549 | "547": 168, 550 | "548": 223, 551 | "549": 786, 552 | "550": 565, 553 | "551": 716, 554 | "552": 704, 555 | "553": 211, 556 | "554": 141, 557 | "555": 185, 558 | "556": 827, 559 | "557": 904, 560 | "558": 262, 561 | "559": 216, 562 | "560": 692, 563 | "561": 167, 564 | "562": 617, 565 | "563": 842, 566 | "564": 205, 567 | "565": 162, 568 | "566": 255, 569 | "567": 576, 570 | "568": 663, 571 | "569": 186, 572 | "570": 879, 573 | "571": 471, 574 | "572": 863, 575 | "573": 181, 576 | "574": 700, 577 | "575": 182, 578 | "576": 142, 579 | "577": 244, 580 | "578": 818, 581 | "579": 133, 582 | "580": 589, 583 | "581": 630, 584 | "582": 608, 585 | "583": 421, 586 | "584": 485, 587 | "585": 802, 588 | "586": 155, 589 | "587": 281, 590 | "588": 748, 591 | "589": 503, 592 | "590": 455, 593 | "591": 656, 594 | "592": 477, 595 | "593": 257, 596 | "594": 251, 597 | "595": 458, 598 | "596": 275, 599 | "597": 695, 600 | "598": 601, 601 | "599": 635, 602 | "600": 466, 603 | "601": 710, 604 | "602": 832, 605 | "603": 199, 606 | "604": 429, 607 | "605": 888, 608 | "606": 564, 609 | "607": 496, 610 | "608": 654, 611 | "609": 175, 612 | "610": 869, 613 | "611": 871, 614 | "612": 747, 615 | "613": 464, 616 | "614": 678, 617 | "615": 681, 618 | "616": 486, 619 | "617": 826, 620 | "618": 799, 621 | "619": 250, 622 | "620": 134, 623 | "621": 280, 624 | "622": 897, 625 | "623": 277, 626 | "624": 592, 627 | "625": 144, 628 | "626": 450, 629 | "627": 176, 630 | "628": 150, 631 | "629": 774, 632 | "630": 881, 633 | "631": 801, 634 | "632": 412, 635 | "633": 440, 636 | "634": 602, 637 | "635": 436, 638 | "636": 725, 639 | "637": 825, 640 | "638": 690, 641 | "639": 885, 642 | "640": 670, 643 | "641": 245, 644 | "642": 246, 645 | "643": 689, 646 | "644": 893, 647 | "645": 502, 648 | "646": 82, 649 | "647": 854, 650 | "648": 208, 651 | "649": 622, 652 | "650": 413, 653 | "651": 566, 654 | "652": 773, 655 | "653": 782, 656 | "654": 828, 657 | "655": 787, 658 | "656": 177, 659 | "657": 157, 660 | "658": 778, 661 | "659": 736, 662 | "660": 196, 663 | "661": 178, 664 | "662": 671, 665 | "663": 591, 666 | "664": 776, 667 | "665": 183, 668 | "666": 731, 669 | "667": 760, 670 | "668": 596, 671 | "669": 200, 672 | "670": 166, 673 | "671": 161, 674 | "672": 633, 675 | "673": 415, 676 | "674": 504, 677 | "675": 190, 678 | "676": 492, 679 | "677": 489, 680 | "678": 499, 681 | "679": 661, 682 | "680": 823, 683 | "681": 456, 684 | "682": 604, 685 | "683": 667, 686 | "684": 258, 687 | "685": 424, 688 | "686": 417, 689 | "687": 240, 690 | "688": 777, 691 | "689": 845, 692 | "690": 201, 693 | "691": 410, 694 | "692": 829, 695 | "693": 733, 696 | "694": 468, 697 | "695": 457, 698 | "696": 409, 699 | "697": 665, 700 | "698": 590, 701 | "699": 259, 702 | "700": 784, 703 | "701": 850, 704 | "702": 903, 705 | "703": 213, 706 | "704": 431, 707 | "705": 163, 708 | "706": 584, 709 | "707": 212, 710 | "708": 637, 711 | "709": 750, 712 | "710": 757, 713 | "711": 790, 714 | "712": 691, 715 | "713": 697, 716 | "714": 479, 717 | "715": 834, 718 | "716": 627, 719 | "717": 187, 720 | "718": 500, 721 | "719": 839, 722 | "720": 808, 723 | "721": 795, 724 | "722": 749, 725 | "723": 469, 726 | "724": 151, 727 | "725": 892, 728 | "726": 278, 729 | "727": 598, 730 | "728": 872, 731 | "729": 636, 732 | "730": 287, 733 | "731": 284, 734 | "732": 763, 735 | "733": 831, 736 | "734": 191, 737 | "735": 761, 738 | "736": 221, 739 | "737": 696, 740 | "738": 745, 741 | "739": 470, 742 | "740": 279, 743 | "741": 677, 744 | "742": 460, 745 | "743": 607, 746 | "744": 446, 747 | "745": 438, 748 | "746": 476, 749 | "747": 753, 750 | "748": 847, 751 | "749": 768, 752 | "750": 884, 753 | "751": 179, 754 | "752": 766, 755 | "753": 475, 756 | "754": 769, 757 | "755": 441, 758 | "756": 835, 759 | "757": 192, 760 | "758": 474, 761 | "759": 873, 762 | "760": 573, 763 | "761": 482, 764 | "762": 599, 765 | "763": 125, 766 | "764": 445, 767 | "765": 217, 768 | "766": 568, 769 | "767": 906, 770 | "768": 783, 771 | "769": 423, 772 | "770": 666, 773 | "771": 659, 774 | "772": 490, 775 | "773": 860, 776 | "774": 657, 777 | "775": 846, 778 | "776": 261, 779 | "777": 717, 780 | "778": 425, 781 | "779": 870, 782 | "780": 127, 783 | "781": 634, 784 | "782": 414, 785 | "783": 491, 786 | "784": 285, 787 | "785": 493, 788 | "786": 463, 789 | "787": 708, 790 | "788": 614, 791 | "789": 739, 792 | "790": 858, 793 | "791": 165, 794 | "792": 286, 795 | "793": 775, 796 | "794": 653, 797 | "795": 494, 798 | "796": 684, 799 | "797": 851, 800 | "798": 453, 801 | "799": 631, 802 | "800": 461, 803 | "801": 411, 804 | "802": 194, 805 | "803": 184, 806 | "804": 868, 807 | "805": 128, 808 | "806": 895, 809 | "807": 481, 810 | "808": 833, 811 | "809": 729, 812 | "810": 764, 813 | "811": 419, 814 | "812": 140, 815 | "813": 581, 816 | "814": 143, 817 | "815": 505, 818 | "816": 683, 819 | "817": 180, 820 | "818": 497, 821 | "819": 712, 822 | "820": 169, 823 | "821": 585, 824 | "822": 247, 825 | "823": 434, 826 | "824": 907, 827 | "825": 397, 828 | "826": 432, 829 | "827": 420, 830 | "828": 418, 831 | "829": 193, 832 | "830": 865, 833 | "831": 218, 834 | "832": 597, 835 | "833": 153, 836 | "834": 702, 837 | "835": 430, 838 | "836": 478, 839 | "837": 439, 840 | "838": 718, 841 | "839": 586, 842 | "840": 735, 843 | "841": 744, 844 | "842": 853, 845 | "843": 473, 846 | "844": 465, 847 | "845": 435, 848 | "846": 210, 849 | "847": 156, 850 | "848": 886, 851 | "849": 580, 852 | "850": 694, 853 | "851": 852, 854 | "852": 878, 855 | "853": 898, 856 | "854": 810, 857 | "855": 664, 858 | "856": 459, 859 | "857": 215, 860 | "858": 688, 861 | "859": 569, 862 | "860": 615, 863 | "861": 219, 864 | "862": 498, 865 | "863": 605, 866 | "864": 188, 867 | "865": 616, 868 | "866": 195, 869 | "867": 189, 870 | "868": 673, 871 | "869": 732, 872 | "870": 197, 873 | "871": 148, 874 | "872": 501, 875 | "873": 583, 876 | "874": 789, 877 | "875": 256, 878 | "876": 672, 879 | "877": 632, 880 | "878": 448, 881 | "879": 126, 882 | "880": 198, 883 | "881": 241, 884 | "882": 571, 885 | "883": 781, 886 | "884": 899, 887 | "885": 877, 888 | "886": 462, 889 | "887": 698, 890 | "888": 587, 891 | "889": 676, 892 | "890": 844, 893 | "891": 570, 894 | "892": 428, 895 | "893": 836, 896 | "894": 224, 897 | "895": 137, 898 | "896": 813, 899 | "897": 574, 900 | "898": 866, 901 | "899": 726, 902 | "900": 703, 903 | "901": 680, 904 | "902": 407, 905 | "903": 806, 906 | "904": 819, 907 | "905": 811, 908 | "906": 843, 909 | "907": 738, 910 | "908": 408, 911 | "909": 577, 912 | "910": 859, 913 | "911": 722, 914 | "912": 628, 915 | "913": 154, 916 | "914": 146, 917 | "915": 619, 918 | "916": 488, 919 | "917": 838, 920 | "918": 699, 921 | "919": 840, 922 | "920": 767, 923 | "921": 682, 924 | "922": 711, 925 | "923": 660, 926 | "924": 721, 927 | "925": 751, 928 | "926": 679, 929 | "927": 701, 930 | "928": 882, 931 | "929": 876, 932 | "930": 780, 933 | "931": 675, 934 | "932": 883, 935 | "933": 902, 936 | "934": 792, 937 | "935": 640, 938 | "936": 642, 939 | "937": 643, 940 | "938": 644, 941 | "939": 645, 942 | "940": 646, 943 | "941": 647, 944 | "942": 648, 945 | "943": 649, 946 | "944": 650, 947 | "945": 641, 948 | "946": 651, 949 | "947": 652, 950 | "948": 225, 951 | "949": 135, 952 | "950": 226, 953 | "951": 227, 954 | "952": 228, 955 | "953": 229, 956 | "954": 230, 957 | "955": 231, 958 | "956": 232, 959 | "957": 233, 960 | "958": 723, 961 | "959": 908, 962 | "960": 861, 963 | "961": 771, 964 | "962": 714, 965 | "963": 856, 966 | "964": 737, 967 | "965": 807, 968 | "966": 706, 969 | "967": 855, 970 | "968": 765, 971 | "969": 730, 972 | "970": 267, 973 | "971": 901, 974 | "972": 265, 975 | "973": 271, 976 | "974": 274, 977 | "975": 272, 978 | "976": 269, 979 | "977": 270, 980 | "978": 273, 981 | "979": 266, 982 | "980": 268, 983 | "981": 862, 984 | "982": 755, 985 | "983": 891, 986 | "984": 237, 987 | "985": 263, 988 | "986": 264, 989 | "987": 238, 990 | "988": 234, 991 | "989": 235, 992 | "990": 239, 993 | "991": 793, 994 | "992": 820, 995 | "993": 864, 996 | "994": 800, 997 | "995": 785, 998 | "996": 720, 999 | "997": 889, 1000 | "998": 236, 1001 | "999": 796 1002 | } 1003 | --------------------------------------------------------------------------------