├── Adversarial_Examples_EfficientNet_AdvProp.ipynb
├── Adversarial_Examples_EfficientNet_NST.ipynb
├── Adversarial_Training_NSL.ipynb
├── Adversarial_Training_NSL_NST.ipynb
├── GANs_w_Adversaries.ipynb
├── Image_Adversaries_Basics.ipynb
├── Optimizer_Susceptibility.ipynb
├── Optimizer_Susceptibility_Targeted_Attacks.ipynb
└── README.md


/GANs_w_Adversaries.ipynb:
--------------------------------------------------------------------------------
   1 | {
   2 |   "nbformat": 4,
   3 |   "nbformat_minor": 0,
   4 |   "metadata": {
   5 |     "colab": {
   6 |       "name": "GANs_w_Adversaries.ipynb",
   7 |       "provenance": [],
   8 |       "collapsed_sections": [],
   9 |       "authorship_tag": "ABX9TyPzL9ozeev9OckSAqDJDlEu",
  10 |       "include_colab_link": true
  11 |     },
  12 |     "kernelspec": {
  13 |       "name": "python3",
  14 |       "display_name": "Python 3"
  15 |     },
  16 |     "accelerator": "GPU"
  17 |   },
  18 |   "cells": [
  19 |     {
  20 |       "cell_type": "markdown",
  21 |       "metadata": {
  22 |         "id": "view-in-github",
  23 |         "colab_type": "text"
  24 |       },
  25 |       "source": [
  26 |         "<a href=\"https://colab.research.google.com/github/sayakpaul/Image-Adversaries-101/blob/master/GANs_w_Adversaries.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
  27 |       ]
  28 |     },
  29 |     {
  30 |       "cell_type": "code",
  31 |       "metadata": {
  32 |         "id": "evdgWJ1M6k1Q",
  33 |         "colab_type": "code",
  34 |         "colab": {
  35 |           "base_uri": "https://localhost:8080/",
  36 |           "height": 34
  37 |         },
  38 |         "outputId": "6d7f9843-e2ee-4f37-d3fd-1dcfd7b11030"
  39 |       },
  40 |       "source": [
  41 |         "# Import TensorFlow as verify the version\n",
  42 |         "import tensorflow as tf\n",
  43 |         "print(tf.__version__)"
  44 |       ],
  45 |       "execution_count": 1,
  46 |       "outputs": [
  47 |         {
  48 |           "output_type": "stream",
  49 |           "text": [
  50 |             "2.2.0-rc3\n"
  51 |           ],
  52 |           "name": "stdout"
  53 |         }
  54 |       ]
  55 |     },
  56 |     {
  57 |       "cell_type": "code",
  58 |       "metadata": {
  59 |         "id": "pZgEZWIj6zfz",
  60 |         "colab_type": "code",
  61 |         "outputId": "75dc5438-9f81-4077-d613-073d9e5f06f8",
  62 |         "colab": {
  63 |           "base_uri": "https://localhost:8080/",
  64 |           "height": 306
  65 |         }
  66 |       },
  67 |       "source": [
  68 |         "# Which GPU?\n",
  69 |         "!nvidia-smi"
  70 |       ],
  71 |       "execution_count": 2,
  72 |       "outputs": [
  73 |         {
  74 |           "output_type": "stream",
  75 |           "text": [
  76 |             "Fri Apr 17 13:39:47 2020       \n",
  77 |             "+-----------------------------------------------------------------------------+\n",
  78 |             "| NVIDIA-SMI 440.64.00    Driver Version: 418.67       CUDA Version: 10.1     |\n",
  79 |             "|-------------------------------+----------------------+----------------------+\n",
  80 |             "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
  81 |             "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
  82 |             "|===============================+======================+======================|\n",
  83 |             "|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |\n",
  84 |             "| N/A   43C    P0    30W / 250W |      0MiB / 16280MiB |      0%      Default |\n",
  85 |             "+-------------------------------+----------------------+----------------------+\n",
  86 |             "                                                                               \n",
  87 |             "+-----------------------------------------------------------------------------+\n",
  88 |             "| Processes:                                                       GPU Memory |\n",
  89 |             "|  GPU       PID   Type   Process name                             Usage      |\n",
  90 |             "|=============================================================================|\n",
  91 |             "|  No running processes found                                                 |\n",
  92 |             "+-----------------------------------------------------------------------------+\n"
  93 |           ],
  94 |           "name": "stdout"
  95 |         }
  96 |       ]
  97 |     },
  98 |     {
  99 |       "cell_type": "code",
 100 |       "metadata": {
 101 |         "id": "IL7QdHbH7ptO",
 102 |         "colab_type": "code",
 103 |         "colab": {}
 104 |       },
 105 |       "source": [
 106 |         "# Other imports\n",
 107 |         "from tensorflow.keras.layers import *\n",
 108 |         "from tensorflow.keras.models import *\n",
 109 |         "from imutils import build_montages\n",
 110 |         "from sklearn.utils import shuffle \n",
 111 |         "from tqdm import tqdm\n",
 112 |         "import matplotlib.pyplot as plt\n",
 113 |         "import numpy as np\n",
 114 |         "import cv2"
 115 |       ],
 116 |       "execution_count": 0,
 117 |       "outputs": []
 118 |     },
 119 |     {
 120 |       "cell_type": "code",
 121 |       "metadata": {
 122 |         "id": "tOxVL1Qz61pZ",
 123 |         "colab_type": "code",
 124 |         "outputId": "8016147d-2396-478e-ff3c-a0bfdb7f3957",
 125 |         "colab": {
 126 |           "base_uri": "https://localhost:8080/",
 127 |           "height": 170
 128 |         }
 129 |       },
 130 |       "source": [
 131 |         "# Load data\n",
 132 |         "((X_train, _), (X_test, _)) = tf.keras.datasets.fashion_mnist.load_data()\n",
 133 |         "X_train.shape, X_test.shape"
 134 |       ],
 135 |       "execution_count": 4,
 136 |       "outputs": [
 137 |         {
 138 |           "output_type": "stream",
 139 |           "text": [
 140 |             "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
 141 |             "32768/29515 [=================================] - 0s 0us/step\n",
 142 |             "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
 143 |             "26427392/26421880 [==============================] - 0s 0us/step\n",
 144 |             "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
 145 |             "8192/5148 [===============================================] - 0s 0us/step\n",
 146 |             "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
 147 |             "4423680/4422102 [==============================] - 0s 0us/step\n"
 148 |           ],
 149 |           "name": "stdout"
 150 |         },
 151 |         {
 152 |           "output_type": "execute_result",
 153 |           "data": {
 154 |             "text/plain": [
 155 |               "((60000, 28, 28), (10000, 28, 28))"
 156 |             ]
 157 |           },
 158 |           "metadata": {
 159 |             "tags": []
 160 |           },
 161 |           "execution_count": 4
 162 |         }
 163 |       ]
 164 |     },
 165 |     {
 166 |       "cell_type": "code",
 167 |       "metadata": {
 168 |         "id": "SzylG4pp7CoC",
 169 |         "colab_type": "code",
 170 |         "colab": {}
 171 |       },
 172 |       "source": [
 173 |         "# Add a channel dimension so that it is compatible with Conv2D\n",
 174 |         "X_train = X_train.reshape(-1, 28, 28, 1)\n",
 175 |         "X_test = X_test.reshape(-1, 28, 28, 1)"
 176 |       ],
 177 |       "execution_count": 0,
 178 |       "outputs": []
 179 |     },
 180 |     {
 181 |       "cell_type": "code",
 182 |       "metadata": {
 183 |         "id": "LW9QNjXw7J9n",
 184 |         "colab_type": "code",
 185 |         "colab": {}
 186 |       },
 187 |       "source": [
 188 |         "# Combine the train and test images and scale the the image pixel values to [-1, 1]\n",
 189 |         "trainImages = np.concatenate([X_train, X_test])\n",
 190 |         "trainImages = (trainImages.astype(\"float\") - 127.5) / 127.5"
 191 |       ],
 192 |       "execution_count": 0,
 193 |       "outputs": []
 194 |     },
 195 |     {
 196 |       "cell_type": "code",
 197 |       "metadata": {
 198 |         "id": "tXPxzNQL7l8K",
 199 |         "colab_type": "code",
 200 |         "colab": {}
 201 |       },
 202 |       "source": [
 203 |         "# Define the Hyperparameters\n",
 204 |         "latent_dim = 100\n",
 205 |         "height = 28\n",
 206 |         "width = 28\n",
 207 |         "nb_channels = 1"
 208 |       ],
 209 |       "execution_count": 0,
 210 |       "outputs": []
 211 |     },
 212 |     {
 213 |       "cell_type": "code",
 214 |       "metadata": {
 215 |         "id": "G4I6oLeX7xVa",
 216 |         "colab_type": "code",
 217 |         "colab": {}
 218 |       },
 219 |       "source": [
 220 |         "# Generator\n",
 221 |         "def generator_network():\n",
 222 |         "    generator = Sequential()\n",
 223 |         "    inputShape = (7, 7, 64)\n",
 224 |         "    chanDim = -1\n",
 225 |         "\n",
 226 |         "    generator.add(Dense(input_dim=latent_dim, units=512))\n",
 227 |         "    generator.add(LeakyReLU())\n",
 228 |         "    generator.add(BatchNormalization())\n",
 229 |         "\n",
 230 |         "    generator.add(Dense(7 * 7 * 64))\n",
 231 |         "    generator.add(LeakyReLU())\n",
 232 |         "    generator.add(BatchNormalization())\n",
 233 |         "\n",
 234 |         "    generator.add(Reshape(inputShape))\n",
 235 |         "    generator.add(Conv2DTranspose(32, (5, 5), strides=(2, 2),\n",
 236 |         "        padding=\"same\"))\n",
 237 |         "    generator.add((LeakyReLU()))\n",
 238 |         "    generator.add(BatchNormalization(axis=chanDim))\n",
 239 |         "\n",
 240 |         "    generator.add(Conv2DTranspose(nb_channels, (5, 5), strides=(2, 2),\n",
 241 |         "        padding=\"same\", activation=\"tanh\"))\n",
 242 |         "\n",
 243 |         "    return generator"
 244 |       ],
 245 |       "execution_count": 0,
 246 |       "outputs": []
 247 |     },
 248 |     {
 249 |       "cell_type": "code",
 250 |       "metadata": {
 251 |         "id": "d5B5vEWY7y5M",
 252 |         "colab_type": "code",
 253 |         "outputId": "65836cdc-7c24-4ea5-9351-0fe4ea4c1e56",
 254 |         "colab": {
 255 |           "base_uri": "https://localhost:8080/",
 256 |           "height": 527
 257 |         }
 258 |       },
 259 |       "source": [
 260 |         "# Instantiate the Generator network\n",
 261 |         "generator = generator_network()\n",
 262 |         "generator.summary()"
 263 |       ],
 264 |       "execution_count": 9,
 265 |       "outputs": [
 266 |         {
 267 |           "output_type": "stream",
 268 |           "text": [
 269 |             "Model: \"sequential\"\n",
 270 |             "_________________________________________________________________\n",
 271 |             "Layer (type)                 Output Shape              Param #   \n",
 272 |             "=================================================================\n",
 273 |             "dense (Dense)                (None, 512)               51712     \n",
 274 |             "_________________________________________________________________\n",
 275 |             "leaky_re_lu (LeakyReLU)      (None, 512)               0         \n",
 276 |             "_________________________________________________________________\n",
 277 |             "batch_normalization (BatchNo (None, 512)               2048      \n",
 278 |             "_________________________________________________________________\n",
 279 |             "dense_1 (Dense)              (None, 3136)              1608768   \n",
 280 |             "_________________________________________________________________\n",
 281 |             "leaky_re_lu_1 (LeakyReLU)    (None, 3136)              0         \n",
 282 |             "_________________________________________________________________\n",
 283 |             "batch_normalization_1 (Batch (None, 3136)              12544     \n",
 284 |             "_________________________________________________________________\n",
 285 |             "reshape (Reshape)            (None, 7, 7, 64)          0         \n",
 286 |             "_________________________________________________________________\n",
 287 |             "conv2d_transpose (Conv2DTran (None, 14, 14, 32)        51232     \n",
 288 |             "_________________________________________________________________\n",
 289 |             "leaky_re_lu_2 (LeakyReLU)    (None, 14, 14, 32)        0         \n",
 290 |             "_________________________________________________________________\n",
 291 |             "batch_normalization_2 (Batch (None, 14, 14, 32)        128       \n",
 292 |             "_________________________________________________________________\n",
 293 |             "conv2d_transpose_1 (Conv2DTr (None, 28, 28, 1)         801       \n",
 294 |             "=================================================================\n",
 295 |             "Total params: 1,727,233\n",
 296 |             "Trainable params: 1,719,873\n",
 297 |             "Non-trainable params: 7,360\n",
 298 |             "_________________________________________________________________\n"
 299 |           ],
 300 |           "name": "stdout"
 301 |         }
 302 |       ]
 303 |     },
 304 |     {
 305 |       "cell_type": "code",
 306 |       "metadata": {
 307 |         "id": "pDZxoMb58IpK",
 308 |         "colab_type": "code",
 309 |         "colab": {}
 310 |       },
 311 |       "source": [
 312 |         "# Discriminator\n",
 313 |         "def disc_network():\n",
 314 |         "    disc = Sequential()\n",
 315 |         "    inputShape = (height, width, nb_channels)\n",
 316 |         "\n",
 317 |         "    disc.add(Conv2D(32, (5, 5), padding=\"same\", strides=(2, 2),\n",
 318 |         "        input_shape=inputShape))\n",
 319 |         "    disc.add(LeakyReLU(alpha=0.2))\n",
 320 |         "\n",
 321 |         "    disc.add(Conv2D(64, (5, 5), padding=\"same\", strides=(2, 2)))\n",
 322 |         "    disc.add(LeakyReLU(alpha=0.2))\n",
 323 |         "\n",
 324 |         "    disc.add(Flatten())\n",
 325 |         "    disc.add(Dense(512))\n",
 326 |         "    disc.add(LeakyReLU(alpha=0.2))\n",
 327 |         "    disc.add(Dropout(0.3))\n",
 328 |         "    disc.add(Dense(1))\n",
 329 |         "    disc.add(Activation(\"sigmoid\"))\n",
 330 |         "    \n",
 331 |         "    return disc"
 332 |       ],
 333 |       "execution_count": 0,
 334 |       "outputs": []
 335 |     },
 336 |     {
 337 |       "cell_type": "code",
 338 |       "metadata": {
 339 |         "id": "0sGntIQF8XjR",
 340 |         "colab_type": "code",
 341 |         "outputId": "c1878583-1da7-43ed-bdce-548c68f4093e",
 342 |         "colab": {
 343 |           "base_uri": "https://localhost:8080/",
 344 |           "height": 493
 345 |         }
 346 |       },
 347 |       "source": [
 348 |         "# Instantiate the Discriminator network\n",
 349 |         "disc = disc_network()\n",
 350 |         "disc.summary()"
 351 |       ],
 352 |       "execution_count": 11,
 353 |       "outputs": [
 354 |         {
 355 |           "output_type": "stream",
 356 |           "text": [
 357 |             "Model: \"sequential_1\"\n",
 358 |             "_________________________________________________________________\n",
 359 |             "Layer (type)                 Output Shape              Param #   \n",
 360 |             "=================================================================\n",
 361 |             "conv2d (Conv2D)              (None, 14, 14, 32)        832       \n",
 362 |             "_________________________________________________________________\n",
 363 |             "leaky_re_lu_3 (LeakyReLU)    (None, 14, 14, 32)        0         \n",
 364 |             "_________________________________________________________________\n",
 365 |             "conv2d_1 (Conv2D)            (None, 7, 7, 64)          51264     \n",
 366 |             "_________________________________________________________________\n",
 367 |             "leaky_re_lu_4 (LeakyReLU)    (None, 7, 7, 64)          0         \n",
 368 |             "_________________________________________________________________\n",
 369 |             "flatten (Flatten)            (None, 3136)              0         \n",
 370 |             "_________________________________________________________________\n",
 371 |             "dense_2 (Dense)              (None, 512)               1606144   \n",
 372 |             "_________________________________________________________________\n",
 373 |             "leaky_re_lu_5 (LeakyReLU)    (None, 512)               0         \n",
 374 |             "_________________________________________________________________\n",
 375 |             "dropout (Dropout)            (None, 512)               0         \n",
 376 |             "_________________________________________________________________\n",
 377 |             "dense_3 (Dense)              (None, 1)                 513       \n",
 378 |             "_________________________________________________________________\n",
 379 |             "activation (Activation)      (None, 1)                 0         \n",
 380 |             "=================================================================\n",
 381 |             "Total params: 1,658,753\n",
 382 |             "Trainable params: 1,658,753\n",
 383 |             "Non-trainable params: 0\n",
 384 |             "_________________________________________________________________\n"
 385 |           ],
 386 |           "name": "stdout"
 387 |         }
 388 |       ]
 389 |     },
 390 |     {
 391 |       "cell_type": "code",
 392 |       "metadata": {
 393 |         "id": "Hxuad7i28dUr",
 394 |         "colab_type": "code",
 395 |         "colab": {}
 396 |       },
 397 |       "source": [
 398 |         "# Optimizer for the Disc\n",
 399 |         "disc_opt = tf.keras.optimizers.RMSprop(lr=0.0008, clipvalue=1.0, decay=1e-8)\n",
 400 |         "disc.compile(optimizer=disc_opt, loss=\"binary_crossentropy\")"
 401 |       ],
 402 |       "execution_count": 0,
 403 |       "outputs": []
 404 |     },
 405 |     {
 406 |       "cell_type": "code",
 407 |       "metadata": {
 408 |         "id": "1Nrlh3Tw8hJq",
 409 |         "colab_type": "code",
 410 |         "colab": {}
 411 |       },
 412 |       "source": [
 413 |         "# GAN\n",
 414 |         "disc.trainable = False\n",
 415 |         "gan_input = Input(shape=(latent_dim, ))\n",
 416 |         "gan_output = disc(generator(gan_input))\n",
 417 |         "gan = Model(gan_input, gan_output)\n",
 418 |         "\n",
 419 |         "gan_opt = tf.keras.optimizers.RMSprop(lr=0.0008, clipvalue=1.0, decay=1e-8)\n",
 420 |         "gan.compile(loss=\"binary_crossentropy\", optimizer=gan_opt)"
 421 |       ],
 422 |       "execution_count": 0,
 423 |       "outputs": []
 424 |     },
 425 |     {
 426 |       "cell_type": "code",
 427 |       "metadata": {
 428 |         "id": "9m5MrmSt8m2I",
 429 |         "colab_type": "code",
 430 |         "outputId": "748200ae-3b8b-4a25-aab2-49f6bb7b0c89",
 431 |         "colab": {
 432 |           "base_uri": "https://localhost:8080/",
 433 |           "height": 255
 434 |         }
 435 |       },
 436 |       "source": [
 437 |         "# Summary of the GAN architecture\n",
 438 |         "gan.summary()"
 439 |       ],
 440 |       "execution_count": 14,
 441 |       "outputs": [
 442 |         {
 443 |           "output_type": "stream",
 444 |           "text": [
 445 |             "Model: \"model\"\n",
 446 |             "_________________________________________________________________\n",
 447 |             "Layer (type)                 Output Shape              Param #   \n",
 448 |             "=================================================================\n",
 449 |             "input_1 (InputLayer)         [(None, 100)]             0         \n",
 450 |             "_________________________________________________________________\n",
 451 |             "sequential (Sequential)      (None, 28, 28, 1)         1727233   \n",
 452 |             "_________________________________________________________________\n",
 453 |             "sequential_1 (Sequential)    (None, 1)                 1658753   \n",
 454 |             "=================================================================\n",
 455 |             "Total params: 3,385,986\n",
 456 |             "Trainable params: 1,719,873\n",
 457 |             "Non-trainable params: 1,666,113\n",
 458 |             "_________________________________________________________________\n"
 459 |           ],
 460 |           "name": "stdout"
 461 |         }
 462 |       ]
 463 |     },
 464 |     {
 465 |       "cell_type": "code",
 466 |       "metadata": {
 467 |         "id": "-v6L5Zv48yh2",
 468 |         "colab_type": "code",
 469 |         "colab": {}
 470 |       },
 471 |       "source": [
 472 |         "# More hyperparameters\n",
 473 |         "epochs = 10000\n",
 474 |         "batch_size = 20"
 475 |       ],
 476 |       "execution_count": 0,
 477 |       "outputs": []
 478 |     },
 479 |     {
 480 |       "cell_type": "code",
 481 |       "metadata": {
 482 |         "id": "M1OMjLS581nQ",
 483 |         "colab_type": "code",
 484 |         "outputId": "83c9e1d7-73ac-40c7-a657-e5b09ed9ac3d",
 485 |         "colab": {
 486 |           "base_uri": "https://localhost:8080/",
 487 |           "height": 374
 488 |         }
 489 |       },
 490 |       "source": [
 491 |         "# Start training\n",
 492 |         "################\n",
 493 |         "start = 0\n",
 494 |         "for epoch in tqdm(range(epochs)):\n",
 495 |         "    # Sample random noise from a normal distribution\n",
 496 |         "    benchmark_noise = np.random.normal(size=(batch_size, latent_dim))\n",
 497 |         "    # Map this noise to images via the generator network\n",
 498 |         "    generated_images = generator.predict(benchmark_noise)\n",
 499 |         "    \n",
 500 |         "    # Calculate where to stop the current epoch, segregate the current batch\n",
 501 |         "    # of real images and combine them with the fake ones\n",
 502 |         "    stop = start + batch_size\n",
 503 |         "    real_images = trainImages[start:stop]\n",
 504 |         "    combined_images = np.concatenate([generated_images, real_images])\n",
 505 |         "    \n",
 506 |         "    # 1 -> Fake images, 0 -> Real images\n",
 507 |         "    # Add a bit of random noise to the labels\n",
 508 |         "    labels = np.array(([1] * batch_size) + ([0] * batch_size)).astype(\"float\")\n",
 509 |         "    labels += 0.05 * np.random.random(labels.shape)\n",
 510 |         "    \n",
 511 |         "    # Shuffle the datapoints of the newly created dataset\n",
 512 |         "    (combined_images, labels) = shuffle(combined_images, labels)\n",
 513 |         "    \n",
 514 |         "    # Train the discriminator on these new dataset\n",
 515 |         "    d_loss = disc.train_on_batch(combined_images, labels)\n",
 516 |         "    \n",
 517 |         "    # Again sample random noise from a normal distribution &\n",
 518 |         "    # set the labels to such that they resemble real images to fool\n",
 519 |         "    # discriminator\n",
 520 |         "    random_latent_vectors = np.random.normal(size=(batch_size, latent_dim))\n",
 521 |         "    fake_labels = np.array([0] * batch_size)\n",
 522 |         "    \n",
 523 |         "    # Train the GAN with this random noise and fake labels, but only\n",
 524 |         "    # update the weights of generator\n",
 525 |         "    a_loss = gan.train_on_batch(random_latent_vectors, fake_labels)\n",
 526 |         "    \n",
 527 |         "    # Set the next batch\n",
 528 |         "    start += batch_size\n",
 529 |         "    if start > len(trainImages) - batch_size:\n",
 530 |         "        start = 0\n",
 531 |         "    \n",
 532 |         "    # Logging\n",
 533 |         "    if epoch % 500 == 0:\n",
 534 |         "        print(f\"Step: {epoch} Disc Loss: {d_loss} Adv Loss: {a_loss}\")\n",
 535 |         "        images = ((generated_images * 127.5) + 127.5).astype(\"int\")\n",
 536 |         "        images = np.repeat(images, 3, axis=-1)\n",
 537 |         "        vis = build_montages(images, (28, 28), (4, 5))[0]\n",
 538 |         "        cv2.imwrite(\"vis\" + str(epoch) + \".png\", vis)"
 539 |       ],
 540 |       "execution_count": 16,
 541 |       "outputs": [
 542 |         {
 543 |           "output_type": "stream",
 544 |           "text": [
 545 |             "  0%|          | 3/10000 [00:08<17:03:54,  6.15s/it]"
 546 |           ],
 547 |           "name": "stderr"
 548 |         },
 549 |         {
 550 |           "output_type": "stream",
 551 |           "text": [
 552 |             "Step: 0 Disc Loss: 0.687881350517273 Adv Loss: 0.14508673548698425\n"
 553 |           ],
 554 |           "name": "stdout"
 555 |         },
 556 |         {
 557 |           "output_type": "stream",
 558 |           "text": [
 559 |             "  5%|▌         | 503/10000 [00:33<07:38, 20.73it/s]"
 560 |           ],
 561 |           "name": "stderr"
 562 |         },
 563 |         {
 564 |           "output_type": "stream",
 565 |           "text": [
 566 |             "Step: 500 Disc Loss: 0.47640880942344666 Adv Loss: 1.350799798965454\n"
 567 |           ],
 568 |           "name": "stdout"
 569 |         },
 570 |         {
 571 |           "output_type": "stream",
 572 |           "text": [
 573 |             " 10%|█         | 1003/10000 [00:57<07:16, 20.63it/s]"
 574 |           ],
 575 |           "name": "stderr"
 576 |         },
 577 |         {
 578 |           "output_type": "stream",
 579 |           "text": [
 580 |             "Step: 1000 Disc Loss: 0.49821704626083374 Adv Loss: 1.7119624614715576\n"
 581 |           ],
 582 |           "name": "stdout"
 583 |         },
 584 |         {
 585 |           "output_type": "stream",
 586 |           "text": [
 587 |             " 15%|█▌        | 1504/10000 [01:21<07:10, 19.76it/s]"
 588 |           ],
 589 |           "name": "stderr"
 590 |         },
 591 |         {
 592 |           "output_type": "stream",
 593 |           "text": [
 594 |             "Step: 1500 Disc Loss: 0.49107104539871216 Adv Loss: 1.8809871673583984\n"
 595 |           ],
 596 |           "name": "stdout"
 597 |         },
 598 |         {
 599 |           "output_type": "stream",
 600 |           "text": [
 601 |             " 20%|██        | 2003/10000 [01:46<06:39, 20.03it/s]"
 602 |           ],
 603 |           "name": "stderr"
 604 |         },
 605 |         {
 606 |           "output_type": "stream",
 607 |           "text": [
 608 |             "Step: 2000 Disc Loss: 0.536552906036377 Adv Loss: 2.172856569290161\n"
 609 |           ],
 610 |           "name": "stdout"
 611 |         },
 612 |         {
 613 |           "output_type": "stream",
 614 |           "text": [
 615 |             " 25%|██▌       | 2503/10000 [02:10<06:07, 20.42it/s]"
 616 |           ],
 617 |           "name": "stderr"
 618 |         },
 619 |         {
 620 |           "output_type": "stream",
 621 |           "text": [
 622 |             "Step: 2500 Disc Loss: 0.2591400444507599 Adv Loss: 2.582822561264038\n"
 623 |           ],
 624 |           "name": "stdout"
 625 |         },
 626 |         {
 627 |           "output_type": "stream",
 628 |           "text": [
 629 |             " 30%|███       | 3004/10000 [02:34<05:43, 20.37it/s]"
 630 |           ],
 631 |           "name": "stderr"
 632 |         },
 633 |         {
 634 |           "output_type": "stream",
 635 |           "text": [
 636 |             "Step: 3000 Disc Loss: 0.4406290054321289 Adv Loss: 2.676551342010498\n"
 637 |           ],
 638 |           "name": "stdout"
 639 |         },
 640 |         {
 641 |           "output_type": "stream",
 642 |           "text": [
 643 |             " 35%|███▌      | 3503/10000 [02:59<05:14, 20.67it/s]"
 644 |           ],
 645 |           "name": "stderr"
 646 |         },
 647 |         {
 648 |           "output_type": "stream",
 649 |           "text": [
 650 |             "Step: 3500 Disc Loss: 0.28072601556777954 Adv Loss: 2.271653413772583\n"
 651 |           ],
 652 |           "name": "stdout"
 653 |         },
 654 |         {
 655 |           "output_type": "stream",
 656 |           "text": [
 657 |             " 40%|████      | 4004/10000 [03:24<04:51, 20.54it/s]"
 658 |           ],
 659 |           "name": "stderr"
 660 |         },
 661 |         {
 662 |           "output_type": "stream",
 663 |           "text": [
 664 |             "Step: 4000 Disc Loss: 0.4415341317653656 Adv Loss: 2.348633050918579\n"
 665 |           ],
 666 |           "name": "stdout"
 667 |         },
 668 |         {
 669 |           "output_type": "stream",
 670 |           "text": [
 671 |             " 45%|████▌     | 4505/10000 [03:48<04:24, 20.80it/s]"
 672 |           ],
 673 |           "name": "stderr"
 674 |         },
 675 |         {
 676 |           "output_type": "stream",
 677 |           "text": [
 678 |             "Step: 4500 Disc Loss: 0.3864983022212982 Adv Loss: 1.804436445236206\n"
 679 |           ],
 680 |           "name": "stdout"
 681 |         },
 682 |         {
 683 |           "output_type": "stream",
 684 |           "text": [
 685 |             " 50%|█████     | 5004/10000 [04:12<04:02, 20.63it/s]"
 686 |           ],
 687 |           "name": "stderr"
 688 |         },
 689 |         {
 690 |           "output_type": "stream",
 691 |           "text": [
 692 |             "Step: 5000 Disc Loss: 0.25370922684669495 Adv Loss: 2.260925054550171\n"
 693 |           ],
 694 |           "name": "stdout"
 695 |         },
 696 |         {
 697 |           "output_type": "stream",
 698 |           "text": [
 699 |             " 55%|█████▌    | 5504/10000 [04:37<03:38, 20.59it/s]"
 700 |           ],
 701 |           "name": "stderr"
 702 |         },
 703 |         {
 704 |           "output_type": "stream",
 705 |           "text": [
 706 |             "Step: 5500 Disc Loss: 0.4161396622657776 Adv Loss: 1.7695811986923218\n"
 707 |           ],
 708 |           "name": "stdout"
 709 |         },
 710 |         {
 711 |           "output_type": "stream",
 712 |           "text": [
 713 |             " 60%|██████    | 6002/10000 [05:01<03:32, 18.82it/s]"
 714 |           ],
 715 |           "name": "stderr"
 716 |         },
 717 |         {
 718 |           "output_type": "stream",
 719 |           "text": [
 720 |             "Step: 6000 Disc Loss: 0.40372294187545776 Adv Loss: 2.133988618850708\n"
 721 |           ],
 722 |           "name": "stdout"
 723 |         },
 724 |         {
 725 |           "output_type": "stream",
 726 |           "text": [
 727 |             " 65%|██████▌   | 6502/10000 [05:26<02:46, 21.03it/s]"
 728 |           ],
 729 |           "name": "stderr"
 730 |         },
 731 |         {
 732 |           "output_type": "stream",
 733 |           "text": [
 734 |             "Step: 6500 Disc Loss: 0.3453351855278015 Adv Loss: 1.365769863128662\n"
 735 |           ],
 736 |           "name": "stdout"
 737 |         },
 738 |         {
 739 |           "output_type": "stream",
 740 |           "text": [
 741 |             " 70%|███████   | 7003/10000 [05:51<02:22, 20.99it/s]"
 742 |           ],
 743 |           "name": "stderr"
 744 |         },
 745 |         {
 746 |           "output_type": "stream",
 747 |           "text": [
 748 |             "Step: 7000 Disc Loss: 0.3185539245605469 Adv Loss: 2.548447608947754\n"
 749 |           ],
 750 |           "name": "stdout"
 751 |         },
 752 |         {
 753 |           "output_type": "stream",
 754 |           "text": [
 755 |             " 75%|███████▌  | 7505/10000 [06:15<02:02, 20.36it/s]"
 756 |           ],
 757 |           "name": "stderr"
 758 |         },
 759 |         {
 760 |           "output_type": "stream",
 761 |           "text": [
 762 |             "Step: 7500 Disc Loss: 0.2985878586769104 Adv Loss: 2.064674139022827\n"
 763 |           ],
 764 |           "name": "stdout"
 765 |         },
 766 |         {
 767 |           "output_type": "stream",
 768 |           "text": [
 769 |             " 80%|████████  | 8004/10000 [06:40<01:38, 20.28it/s]"
 770 |           ],
 771 |           "name": "stderr"
 772 |         },
 773 |         {
 774 |           "output_type": "stream",
 775 |           "text": [
 776 |             "Step: 8000 Disc Loss: 0.42655688524246216 Adv Loss: 1.4810914993286133\n"
 777 |           ],
 778 |           "name": "stdout"
 779 |         },
 780 |         {
 781 |           "output_type": "stream",
 782 |           "text": [
 783 |             " 85%|████████▌ | 8503/10000 [07:04<01:13, 20.34it/s]"
 784 |           ],
 785 |           "name": "stderr"
 786 |         },
 787 |         {
 788 |           "output_type": "stream",
 789 |           "text": [
 790 |             "Step: 8500 Disc Loss: 0.52017742395401 Adv Loss: 1.9420932531356812\n"
 791 |           ],
 792 |           "name": "stdout"
 793 |         },
 794 |         {
 795 |           "output_type": "stream",
 796 |           "text": [
 797 |             " 90%|█████████ | 9004/10000 [07:28<00:49, 20.22it/s]"
 798 |           ],
 799 |           "name": "stderr"
 800 |         },
 801 |         {
 802 |           "output_type": "stream",
 803 |           "text": [
 804 |             "Step: 9000 Disc Loss: 0.5068351030349731 Adv Loss: 2.759629011154175\n"
 805 |           ],
 806 |           "name": "stdout"
 807 |         },
 808 |         {
 809 |           "output_type": "stream",
 810 |           "text": [
 811 |             " 95%|█████████▌| 9503/10000 [07:53<00:24, 20.39it/s]"
 812 |           ],
 813 |           "name": "stderr"
 814 |         },
 815 |         {
 816 |           "output_type": "stream",
 817 |           "text": [
 818 |             "Step: 9500 Disc Loss: 0.4068868160247803 Adv Loss: 2.576944351196289\n"
 819 |           ],
 820 |           "name": "stdout"
 821 |         },
 822 |         {
 823 |           "output_type": "stream",
 824 |           "text": [
 825 |             "100%|██████████| 10000/10000 [08:17<00:00, 20.08it/s]\n"
 826 |           ],
 827 |           "name": "stderr"
 828 |         }
 829 |       ]
 830 |     },
 831 |     {
 832 |       "cell_type": "markdown",
 833 |       "metadata": {
 834 |         "id": "PE6PpKI-9n-d",
 835 |         "colab_type": "text"
 836 |       },
 837 |       "source": [
 838 |         "## Generate images using the trained generator from random noise"
 839 |       ]
 840 |     },
 841 |     {
 842 |       "cell_type": "code",
 843 |       "metadata": {
 844 |         "id": "uHQaTTWv-0oQ",
 845 |         "colab_type": "code",
 846 |         "outputId": "67f2323a-8fa1-45bb-882d-c2c181cf9837",
 847 |         "colab": {
 848 |           "base_uri": "https://localhost:8080/",
 849 |           "height": 34
 850 |         }
 851 |       },
 852 |       "source": [
 853 |         "# Sample random noise from a normal distribution\n",
 854 |         "benchmark_noise = np.random.normal(size=(batch_size, latent_dim))\n",
 855 |         "# Map this noise to images via the generator network\n",
 856 |         "generated_images = generator.predict(benchmark_noise)\n",
 857 |         "# Scale back\n",
 858 |         "images = ((generated_images * 127.5) + 127.5).astype(\"int\")\n",
 859 |         "images.shape"
 860 |       ],
 861 |       "execution_count": 17,
 862 |       "outputs": [
 863 |         {
 864 |           "output_type": "execute_result",
 865 |           "data": {
 866 |             "text/plain": [
 867 |               "(20, 28, 28, 1)"
 868 |             ]
 869 |           },
 870 |           "metadata": {
 871 |             "tags": []
 872 |           },
 873 |           "execution_count": 17
 874 |         }
 875 |       ]
 876 |     },
 877 |     {
 878 |       "cell_type": "code",
 879 |       "metadata": {
 880 |         "id": "mofASNk8_EFA",
 881 |         "colab_type": "code",
 882 |         "outputId": "b01c837c-4c52-44da-fccb-ed539e9e661a",
 883 |         "colab": {
 884 |           "base_uri": "https://localhost:8080/",
 885 |           "height": 265
 886 |         }
 887 |       },
 888 |       "source": [
 889 |         "# How do the images look like?\n",
 890 |         "random_idx = np.random.choice(images.shape[0], 1)\n",
 891 |         "image_1 = images[random_idx].reshape(28, 28)\n",
 892 |         "plt.imshow(image_1, cmap=plt.cm.binary)\n",
 893 |         "plt.show()"
 894 |       ],
 895 |       "execution_count": 20,
 896 |       "outputs": [
 897 |         {
 898 |           "output_type": "display_data",
 899 |           "data": {
 900 |             "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAASz0lEQVR4nO3dbWyVZZoH8P9lKchLwZY2ldctO0FF1AVSySaYiQqMLx9AEmPgw4SNup0PmDBxPqxxP4x+k83OTDDZTFKUDGxmnQwZjWjM7rg4CZKY0WKqBRoBSXFaSymvhQJC4doPfZztYJ/rquc55zwHrv8vIW3P1eecu0/9e07P9dz3LaoKIrr53ZL3AIioPBh2oiAYdqIgGHaiIBh2oiDGlfPB6uvrtampqZwPeUM4e/asWe/v7zfrU6dOTa3dcov9//MrV66Y9YkTJ5r1oaEhs26N7dq1a+axt956q1mn7+rq6sKJEydktFqmsIvIowA2A6gC8JqqvmJ9f1NTE9ra2rI8ZMlkaUGKjHpux3zf7777rlnfsmWLWV++fHlqzQobAPT09Jj1hQsXmvUTJ06Y9ZUrV6bWLly4YB67YMECs+6d94iam5tTawW/jBeRKgD/AeAxAHcDWCcidxd6f0RUWln+Zl8K4LCqHlHVywB+B2B1cYZFRMWWJeyzAPxlxNfdyW1/Q0RaRKRNRNq8vz2JqHRK/m68qraqarOqNjc0NJT64YgoRZaw9wCYM+Lr2cltRFSBsoT9EwDzRWSeiIwHsBbAzuIMi4iKreDWm6oOichzAP4Hw623raq6v2gjK7MsbZxLly6Zda+F1NXVZdarq6vN+jvvvGPWs/D69Fl65d7Pdeedd5r1jRs3mvW1a9em1saNK+slJhUh00+squ8BeK9IYyGiEuLlskRBMOxEQTDsREEw7ERBMOxEQTDsREHEazYWyJq3vWzZMvPYkydPmvX6+nqz7vWErami3jUAEyZMMOu1tbVm3bv/SZMmpda8qb+Dg4NmfcOGDWb96tWrqbX169ebx96M+MxOFATDThQEw04UBMNOFATDThQEw04UBFtvY9Ta2ppaGxgYMI+dP3++We/t7TXr33zzjVmvq6tLrXlLPZ8/f96se0tJe0tRnz59uuD79qbAetNvP/roo9TamjVrzGO9VXlvRHxmJwqCYScKgmEnCoJhJwqCYScKgmEnCoJhJwqCffaEN93y8OHDqbV7773XPLa9vd2sT58+3ax7O62eOnUqteYt9ez1uvv6+sy6t62yNcW1pqbGPNbbytqawgoAe/fuTa299tpr5rHPP/+8Wb8R8ZmdKAiGnSgIhp0oCIadKAiGnSgIhp0oCIadKAj22RNez/azzz5LrVlLOQP+nPIzZ86Y9cbGRrNu9cKrqqrMY7054d5W1t79W/PdL1++bB7r9fC9ef79/f2ptY8//tg89maUKewi0gXgHICrAIZUtbkYgyKi4ivGM/tDqnqiCPdDRCXEv9mJgsgadgXwRxHZKyIto32DiLSISJuItFl/QxFRaWUN+wOqugTAYwA2iMgPr/8GVW1V1WZVbW5oaMj4cERUqExhV9We5ONxAG8BWFqMQRFR8RUcdhGZLCI1334O4EcA9hVrYERUXFnejW8E8FbShx0H4L9U9b+LMqoceGu/d3R0pNa8LZe9frK3rvzRo0fNujUf3rt+wFub3TsvkydPLvh4b232ixcvmnXveOsagQMHDpjHeusbeNcfVKKCw66qRwD8QxHHQkQlxNYbURAMO1EQDDtREAw7URAMO1EQnOKa8KY8WtMpvfaVt5yz1+aZNWuWWbeWkvamiS5cuNCsW1N7AWBwcNCsWz+bN73W207am1o8d+7c1Nq0adPMY72tqMePH2/WKxGf2YmCYNiJgmDYiYJg2ImCYNiJgmDYiYJg2ImCYJ89sWPHDrNu9dm95ba8aaZe3dtW2epXe1MxvemzTU1NZt3aFhkAxo1L/0/M2w7a63V7rOOtLbgBf5vsefPmFTSmPPGZnSgIhp0oCIadKAiGnSgIhp0oCIadKAiGnSgI9tkTJ07Ye1Na/WqvT24t9Qz4SyZ787qtOevefPMVK1aY9cWLF5t1b0nmS5cupda8Zai98+pdQ2BthW2NCwAOHjxo1tlnJ6KKxbATBcGwEwXBsBMFwbATBcGwEwXBsBMFwT574quvvjLr1vrnQ0ND5rHe+uheH76rq8usV1VVpda8Hv0999xj1r115b3118+dO1fwsd41AhMmTDDr3pr5lk2bNpn1Rx55pOD7zov7zC4iW0XkuIjsG3FbnYi8LyKHko+1pR0mEWU1lpfxvwHw6HW3vQBgl6rOB7Ar+ZqIKpgbdlXdDeD6/YVWA9iWfL4NwBNFHhcRFVmhb9A1qmpv8vkxAI1p3ygiLSLSJiJt3lptRFQ6md+N1+F3rlLfvVLVVlVtVtXmhoaGrA9HRAUqNOx9IjIDAJKPx4s3JCIqhULDvhPA+uTz9QDeLs5wiKhU3D67iLwB4EEA9SLSDeDnAF4B8HsReQbAUQBPlXKQxeDtkd7b22vWa2vTu4vevGpv7vSUKVPMutePtvr03d3d5rEzZ84067Nnz85U99YJyML7s9D6vXh73re1tRU0pkrmhl1V16WUlhd5LERUQrxcligIhp0oCIadKAiGnSgIhp0oiDBTXK2ploA/FdRq83hbD69atcqst7e3m3WvtVddXZ1a86Z5WlsqA3bLEbCn1wL2NFRrqWfA/515j21NPfbu29qiG7CnPAP+7ywPfGYnCoJhJwqCYScKgmEnCoJhJwqCYScKgmEnCiJMn93rm3rLPWc51ptGunv3brPuTce0euVeL9vrF3v1pqYms37o0KHU2sDAgHnslStXzPrcuXPNemdnZ2rN2y7amxLNPjsRVSyGnSgIhp0oCIadKAiGnSgIhp0oCIadKIgwffaDBw+ada9vavWEvX6wN6fcWxLZ25r41Knrt+L7fydPnjSPXbBggVn3eNtJW33+22+/3TzWW4baO69Wr9zbitrbwtu7fqGurs6s54HP7ERBMOxEQTDsREEw7ERBMOxEQTDsREEw7ERBhOmzv/nmm5mOv+OOO1JrXs/V6wfX1NSYdauPDthzyr151VevXs1Ut9as93hrDHjrwnvn1br+weuDe9c2eNdWVCL3mV1EtorIcRHZN+K2l0SkR0Tak3+Pl3aYRJTVWF7G/wbAo6Pc/itVXZT8e6+4wyKiYnPDrqq7AdivI4mo4mV5g+45Efk8eZmfuiGYiLSISJuItPX392d4OCLKotCw/xrADwAsAtAL4Bdp36iqrararKrN3oQPIiqdgsKuqn2qelVVrwHYAmBpcYdFRMVWUNhFZMaIL9cA2Jf2vURUGdw+u4i8AeBBAPUi0g3g5wAeFJFFABRAF4CflHCMRdHb22vWL1++bNat+e5ev9jbf9073uvpWsd76593dHSY9RUrVph1rxdu9fkHBwfNYydOnGjWx48fb9at39kXX3xhHpv1vDU2Npr1PLhhV9V1o9z8egnGQkQlxMtliYJg2ImCYNiJgmDYiYJg2ImCCDPF1VtS2dt22ZrqWVuberUwAODcuXNm3Zsi67FaUENDQ+axp0+fNuteW9D72ayWprXVNOC39bypv1br7uuvvzaP9ZYWt7aDBvyWZR74zE4UBMNOFATDThQEw04UBMNOFATDThQEw04URJg+u7f979SpU816T09Pau2+++4zj12yZIlZ97Y9njx5slnv6+tLrXlTNWfPnm3WvX6zt/qQ9fjTpk0zj/V6/N7vdOXKlam1PXv2mMd61yd4W4BXIj6zEwXBsBMFwbATBcGwEwXBsBMFwbATBcGwEwURps9+8eJFs+4tS2zNrb7rrrvMY71+sLdUtDc2a+tib8vmmTNnmnVviW1vS2drnQCvj+7dt3dtxKxZs1JrFy5cMI/11jfw5vFXIj6zEwXBsBMFwbATBcGwEwXBsBMFwbATBcGwEwURps8+adIks37s2DGzbvVda2pqzGP3799v1r1etteHP3/+fGrN67N7c8rPnj1r1r31+K358N7P5c3F7+7uNusHDhxIrWW9fsDb8rkSuc/sIjJHRP4kIgdEZL+IbExurxOR90XkUPLR3imBiHI1lpfxQwB+pqp3A/hHABtE5G4ALwDYparzAexKviaiCuWGXVV7VfXT5PNzADoBzAKwGsC25Nu2AXiiVIMkouy+1xt0ItIEYDGAPwNoVNXepHQMQGPKMS0i0iYibf39/RmGSkRZjDnsIjIFwB8A/FRVB0bWdPhdmFHfiVHVVlVtVtVmb3FCIiqdMYVdRKoxHPTfquqbyc19IjIjqc8AcLw0QySiYnBbbzLcu3kdQKeq/nJEaSeA9QBeST6+XZIRjpG35PHg4GCm+7emyHpTLb0prnV1dQWN6VvW1sTeefGmz3qttwkTJph1q73mTTP1zovVcgTsrbS9paI93mNXorH02ZcB+DGADhFpT257EcMh/72IPAPgKICnSjNEIioGN+yqugdA2pUZy4s7HCIqFV4uSxQEw04UBMNOFATDThQEw04UxE0zxdXrJ48bZ/+ot912m1m3euXefXtjq66uNuve/Vu8Ka5en7y3t9ese/dv9dm95Zq96bf333+/WX/55ZdTazt27DCP9X4n3tgr0Y03YiIqCMNOFATDThQEw04UBMNOFATDThQEw04UxE3TZ/d4c87nzp1r1vfs2ZNa+/LLL81jvX7xwMCAWc+ydXGWLZUB4MMPPzTr3nx462e3tpoG7Hn6ANDT02PWvWsnLN7vzNumuxLxmZ0oCIadKAiGnSgIhp0oCIadKAiGnSgIhp0oiJumz551HXBvXXlr+2Bra2AAqKqqMuveuvJeL/zUqVNmPYuOjg6zfuTIEbNuzeX31o33+vDeea2vry/4WG9shw4dMuuViM/sREEw7ERBMOxEQTDsREEw7ERBMOxEQTDsREGMZX/2OQC2A2gEoABaVXWziLwE4J8B9Cff+qKqvleqgXq89cu9edfefHerz7527Vrz2CeffNKsb9y40ax7a5h3dnam1rw53d55mz59uln3rk/w5upbHnroIbO+efNms25dn5B13fczZ85kOj4PY7moZgjAz1T1UxGpAbBXRN5Par9S1X8v3fCIqFjGsj97L4De5PNzItIJYFapB0ZExfW9XsuISBOAxQD+nNz0nIh8LiJbRaQ25ZgWEWkTkbb+/v7RvoWIymDMYReRKQD+AOCnqjoA4NcAfgBgEYaf+X8x2nGq2qqqzara3NDQUIQhE1EhxhR2EanGcNB/q6pvAoCq9qnqVVW9BmALgKWlGyYRZeWGXYbfrn0dQKeq/nLE7TNGfNsaAPuKPzwiKpaxvBu/DMCPAXSISHty24sA1onIIgy347oA/KQkIxwjb1vk2tpR31L4q7NnzxZ8/x988IF5bEtLi1nfvn27Wfem71rLRXtTOb223pYtW8y6x5qm6rX9SsnaShrw/3t5+umnizmcshjLu/F7AIz2W8mtp05E3x+voCMKgmEnCoJhJwqCYScKgmEnCoJhJwqi7EtJW/3qLH1Xr1+8adMms378+HGzbk2nfPXVV81jPd7P7f1sXj0Lb9vkG9WqVavM+rPPPmvWH3744WIOpyz4zE4UBMNOFATDThQEw04UBMNOFATDThQEw04UhHjzwIv6YCL9AI6OuKkegL1fcX4qdWyVOi6AYytUMcf2d6o66vpvZQ37dx5cpE1Vm3MbgKFSx1ap4wI4tkKVa2x8GU8UBMNOFETeYW/N+fEtlTq2Sh0XwLEVqixjy/VvdiIqn7yf2YmoTBh2oiByCbuIPCoiX4jIYRF5IY8xpBGRLhHpEJF2EWnLeSxbReS4iOwbcVudiLwvIoeSj/YC5+Ud20si0pOcu3YReTynsc0RkT+JyAER2S8iG5Pbcz13xrjKct7K/je7iFQBOAhgJYBuAJ8AWKeqB8o6kBQi0gWgWVVzvwBDRH4I4DyA7ap6T3LbvwE4paqvJP+jrFXVf6mQsb0E4Hze23gnuxXNGLnNOIAnAPwTcjx3xrieQhnOWx7P7EsBHFbVI6p6GcDvAKzOYRwVT1V3Azh13c2rAWxLPt+G4f9Yyi5lbBVBVXtV9dPk83MAvt1mPNdzZ4yrLPII+ywAfxnxdTcqa793BfBHEdkrIva+TfloVNXe5PNjABrzHMwo3G28y+m6bcYr5twVsv15VnyD7rseUNUlAB4DsCF5uVqRdPhvsErqnY5pG+9yGWWb8b/K89wVuv15VnmEvQfAnBFfz05uqwiq2pN8PA7gLVTeVtR93+6gm3y0V8oso0raxnu0bcZRAecuz+3P8wj7JwDmi8g8ERkPYC2AnTmM4ztEZHLyxglEZDKAH6HytqLeCWB98vl6AG/nOJa/USnbeKdtM46cz13u25+ratn/AXgcw+/IfwngX/MYQ8q4/h7AZ8m//XmPDcAbGH5ZdwXD7208A2A6gF0ADgH4XwB1FTS2/wTQAeBzDAdrRk5jewDDL9E/B9Ce/Hs873NnjKss542XyxIFwTfoiIJg2ImCYNiJgmDYiYJg2ImCYNiJgmDYiYL4PzSVfqmOAwpaAAAAAElFTkSuQmCC\n",
 901 |             "text/plain": [
 902 |               "<Figure size 432x288 with 1 Axes>"
 903 |             ]
 904 |           },
 905 |           "metadata": {
 906 |             "tags": [],
 907 |             "needs_background": "light"
 908 |           }
 909 |         }
 910 |       ]
 911 |     },
 912 |     {
 913 |       "cell_type": "markdown",
 914 |       "metadata": {
 915 |         "id": "wlHCVh3F9v4c",
 916 |         "colab_type": "text"
 917 |       },
 918 |       "source": [
 919 |         "## Forming adversarial examples\n",
 920 |         "\n",
 921 |         "- Train an image classifier\n",
 922 |         "- Form examples"
 923 |       ]
 924 |     },
 925 |     {
 926 |       "cell_type": "code",
 927 |       "metadata": {
 928 |         "id": "zj9UF_j1X2hq",
 929 |         "colab_type": "code",
 930 |         "colab": {}
 931 |       },
 932 |       "source": [
 933 |         "# Define a utility function for the image classifier\n",
 934 |         "def get_train_model():\n",
 935 |         "    model = Sequential([\n",
 936 |         "        Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n",
 937 |         "        MaxPooling2D((2,2)),\n",
 938 |         "        Conv2D(64, (3, 3), activation='relu'),\n",
 939 |         "        MaxPooling2D((2,2)),\n",
 940 |         "        Conv2D(64, (3, 3), activation='relu')\n",
 941 |         "    ])\n",
 942 |         "\n",
 943 |         "    model.add(Flatten())\n",
 944 |         "    model.add(Dense(64, activation='relu'))\n",
 945 |         "    model.add(Dense(10, activation='softmax'))\n",
 946 |         "\n",
 947 |         "    model.compile(optimizer='adam',\n",
 948 |         "                loss='sparse_categorical_crossentropy',\n",
 949 |         "                metrics=['accuracy'])\n",
 950 |         "    \n",
 951 |         "    return model"
 952 |       ],
 953 |       "execution_count": 0,
 954 |       "outputs": []
 955 |     },
 956 |     {
 957 |       "cell_type": "code",
 958 |       "metadata": {
 959 |         "id": "pp5HxWUMYh3d",
 960 |         "colab_type": "code",
 961 |         "colab": {
 962 |           "base_uri": "https://localhost:8080/",
 963 |           "height": 357
 964 |         },
 965 |         "outputId": "5175eec8-5405-4ef7-bfd6-4a61f0409b7e"
 966 |       },
 967 |       "source": [
 968 |         "# Load the dataset again\n",
 969 |         "((X_train, y_train), (X_test, y_test)) = tf.keras.datasets.fashion_mnist.load_data()\n",
 970 |         "X_train = X_train.reshape(-1, 28, 28, 1)\n",
 971 |         "X_test = X_test.reshape(-1, 28, 28, 1)\n",
 972 |         "\n",
 973 |         "# Normalize the dataset\n",
 974 |         "X_train = X_train / 255.\n",
 975 |         "X_test = X_test / 255.\n",
 976 |         "\n",
 977 |         "# Train the model\n",
 978 |         "apparel_model = get_train_model()\n",
 979 |         "history = apparel_model.fit(X_train, y_train,\n",
 980 |         "                  validation_data=(X_test, y_test),\n",
 981 |         "                  batch_size=128,\n",
 982 |         "                  epochs=10)"
 983 |       ],
 984 |       "execution_count": 23,
 985 |       "outputs": [
 986 |         {
 987 |           "output_type": "stream",
 988 |           "text": [
 989 |             "Epoch 1/10\n",
 990 |             "469/469 [==============================] - 3s 5ms/step - loss: 0.6266 - accuracy: 0.7707 - val_loss: 0.4383 - val_accuracy: 0.8426\n",
 991 |             "Epoch 2/10\n",
 992 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.3936 - accuracy: 0.8590 - val_loss: 0.3717 - val_accuracy: 0.8689\n",
 993 |             "Epoch 3/10\n",
 994 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.3366 - accuracy: 0.8773 - val_loss: 0.3378 - val_accuracy: 0.8768\n",
 995 |             "Epoch 4/10\n",
 996 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.3023 - accuracy: 0.8899 - val_loss: 0.3114 - val_accuracy: 0.8879\n",
 997 |             "Epoch 5/10\n",
 998 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.2772 - accuracy: 0.8987 - val_loss: 0.2831 - val_accuracy: 0.8976\n",
 999 |             "Epoch 6/10\n",
1000 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.2554 - accuracy: 0.9061 - val_loss: 0.2868 - val_accuracy: 0.8981\n",
1001 |             "Epoch 7/10\n",
1002 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.2396 - accuracy: 0.9118 - val_loss: 0.2729 - val_accuracy: 0.9005\n",
1003 |             "Epoch 8/10\n",
1004 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.2231 - accuracy: 0.9176 - val_loss: 0.2683 - val_accuracy: 0.9022\n",
1005 |             "Epoch 9/10\n",
1006 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.2101 - accuracy: 0.9229 - val_loss: 0.2726 - val_accuracy: 0.9050\n",
1007 |             "Epoch 10/10\n",
1008 |             "469/469 [==============================] - 2s 5ms/step - loss: 0.1986 - accuracy: 0.9264 - val_loss: 0.2580 - val_accuracy: 0.9076\n"
1009 |           ],
1010 |           "name": "stdout"
1011 |         }
1012 |       ]
1013 |     },
1014 |     {
1015 |       "cell_type": "code",
1016 |       "metadata": {
1017 |         "id": "rYlYWtXkaFT0",
1018 |         "colab_type": "code",
1019 |         "colab": {
1020 |           "base_uri": "https://localhost:8080/",
1021 |           "height": 279
1022 |         },
1023 |         "outputId": "65ebb743-aae2-4560-ea4e-8b6c432602a1"
1024 |       },
1025 |       "source": [
1026 |         "# Plot training progress\n",
1027 |         "plt.plot(history.history[\"loss\"], label=\"train_loss\")\n",
1028 |         "plt.plot(history.history[\"val_loss\"], label=\"val_loss\")\n",
1029 |         "plt.plot(history.history[\"accuracy\"], label=\"train_acc\")\n",
1030 |         "plt.plot(history.history[\"val_accuracy\"], label=\"val_acc\")\n",
1031 |         "plt.xlabel(\"Epoch #\")\n",
1032 |         "plt.ylabel(\"Loss/Accuracy\")\n",
1033 |         "plt.legend(loc=\"lower left\")\n",
1034 |         "plt.show()"
1035 |       ],
1036 |       "execution_count": 27,
1037 |       "outputs": [
1038 |         {
1039 |           "output_type": "display_data",
1040 |           "data": {
1041 |             "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxddZn48c9z9+x7mjZpm0I3WloolBYEBEGgLLKNUNkUR+WHgrIoQ1FGEdFRB50RZYpsIgyKDA7KCFoVgSIItGXpRjdKl7RNm6VJbpab3OX5/XFu0pv0tk3b3Nwk93m/Xud1z36ee9t8n3O+55zvV1QVY4wxmcuV7gCMMcaklyUCY4zJcJYIjDEmw1kiMMaYDGeJwBhjMpwn3QEcrNLSUq2urk53GMYYM6wsW7asXlXLki0bdomgurqapUuXpjsMY4wZVkRk876WWdWQMcZkOEsExhiT4SwRGGNMhrNEYIwxGc4SgTHGZDhLBMYYk+EsERhjTIYbdu8RGGPMcKKqRDRCV7Sr9xBzPjujnYRj4Z75nbFOwtHw3stiXZxWdRpHlx494DFaIjDGZAxVJRwL0xHpoCPSQXuknVAk1DPdEenoNd29vDPambQQ7zUe69pnYa8MTL8vZVlllgiMMSNfNBbtKYQTC+iOcAcd0d7ToWho/+v0KdhDkRBRjR5UPF6Xl4A7gNftxef24Xf78br2jPtcPnK8Ofhc8WXu3su8bm+/lvnczrTf5Yz3DC7n0+vyIiIp+c0tERhjDlk4GqY90k57uD35Z8J4R7hjv+t0RDpoD7cTioYOKgaPeMjyZBHwBMjyZPUM2Z5sSgIlveb1Hfpuk2y5xzXyi8mR/w2NMXudZfcUzgnjPYVxn/G9CvCE8Ugs0u8YAm6n0M32Zvd8dhfW3eOJn4kFerICO+AJkO3Jxuv2pvCXywwpTQQiMg/4CeAGHlbV7/dZPh54FCgDGoGrVbUmlTEZM5RFY1FC0VDSgrq/490Fd+L8zmhnv2NwiYssTxY5nhyyvE5BnOXJojBQyBjPmKSFdrYnu2fdpMs8Wbhd7hT+ckOTqkIkgnYP4TAajkAk7Ix3z4tE0HD3eNjZJsn87OOOwz9x4oDHmbJEICJu4H7gLKAGWCIiz6nq6oTV7gUeV9VfisgZwL8B16QqJmNSKaYx2sJttHa1EgwHnc+uYM94azg+3RXc5zrtkfZ+H0+QXmfW3eN5vjxG5Yzac0bdfQaecKa9v3G/25+yuujDodFoQmEaRrvCewrUcJ+CtWdImE5Yl8R1u/axba/1EwrmXoV0l7Msknw54fCA/gYV3/rm8EoEwBxgg6puBBCRp4CLgMREMA24NT7+EvC7FMZjzD6pKh2RDqeQTiyw4+N95+1VyMcL+gM9HeJ1ecnz5ZHnyyPXm0uuL5ey7LKe8VxvLjnenF6FeGIhn+3JJuAOkO3y41cPRKNOQdXrrDMC0d7T2hmGtj7zIg17tosXZm2RMG2RCBqJorEodH9Go2g05uw3GkOjEej12b1OFGLRvbePRNFYzDle7ADbRxPWjX8/DYchFkvZv794vT0DCePi9SIez55PjwdXwA+5OYjH22u++Lzg8TjzE7fxOuM9y5LO9yBeX6/54vE4yxLicOfnp+T7pzIRVAJbE6ZrgLl91nkPuBSn+ugSIE9ESlS1IXElEbkOuA5g3LhxKQvYDH+qSjAcpLmzmZbOFpo7m2nubKKlaRcd9TvpbKgjsns30abd0BTE3dKGN9iBr7ULV8wpxEVBABI+A0A2QqW48Ygbj3jwiBt3fNotfjyahVtczjQu3OLGLS5cuHCLCzcuBAHV+BBGaQRt3DMvFnMK5mjUqSII7ym8Y5EIwUiE4ACfZR6Q2424XE6h1OvTjbjciNvtrJP004W4nfXF60X8/n2v43ZBz6ezbzxuXD6fUzh3F5Z9C2hfn+nEQt2TMO1Lsk53YTsEr4AGU7pvFn8N+JmIXAssBrYBez3bpaoPAg8CzJ49e2AeyDVDWkxjBLuCTmHe1UxzZzNNnU00h5oIBhvobKijq7GeSGMj2tyCNLfiaWnHFwyR26HktUN+h5LbAVXt4NnHyWTMJXTm+YnmFjhney6ncHe53LjcbtwuZ9zt8uByuUDEKcyl7wAiriTzBPZan/3vx5Nwlun19D7L7Nc8954Crucs1NOzLn2mk83D43EKaZcr4wvJTJDKRLANGJswXRWf10NVt+NcESAiucA/qWpTCmMyaaKqtIZbqWuvo66jjl3tu6jrqKOhaQehhjq6dteju5vR5hbczc5Zek6Hkt8OeR2QFx+v6gDfPh5UUYFwboBofg5akIdrbAGeomL8xaVklZSTXVaBr7gUT1ER7vjgys21gs5kvFQmgiXAJBGZgJMAPgVcmbiCiJQCjaoaA+7AeYLIDDNt4bY9BXzbThoat9Gycysdu3bQVV+HNuzGtbuF7GCYgnYoaFMK2uCYdsjez8MskdwAsfxcpDAf15hCvMUlBErKyCotx1dc6hTmhYW4C4twFxXizs93zmKNMQclZYlAVSMiciOwCOfx0UdVdZWI3A0sVdXngNOBfxMRxakauiFV8ZiD1x5up76j3inc67bQtGMz7bu2E6qrJdLQgDQ2425qJac1QkGbUtgGlW1wxD7O2MO5AbQoH1dpCf6pZWSVjyarvAJ3UTHuosJeZ+ruAqeqxhiTeqI6vKrcZ8+erdZ5/eGJaYzalu3U7ljP7h0f0lq7jY66HXTV1aGNu3E1teBr7iC3NUJBGxS0Ja9jjwl05QeIFuYhxYV4SkrJKq8gt7yS7FFj8JSW4ikpwV1SgqeoCPH5Bv/LGmMAEJFlqjo72TI75RqmVJVYWxuxlhai8SEWDBJtCRILthBpbqZ9dx3Bxlo6dtcTbm5Cg6242jrwd0TI7oQsnCFR1C105PsJFxSgo/PxlBQTKS3HP6qSvIqx5I2qwlNSiqe0BHdhoVXFGDMCWCJIE1VFQyGn4G5pJhoM7inMm1uIBVuItgSJBluINbcQDQadQj9hvQM9V93uhzY/tAWgPSBoQTbuqjL8hcXkFJWRUzaGvFFVFFaMp2D0eLylpbjy8+3mqTEZxhJBiqkq4S1b6Fi+go4VywktX0HXli1Eg8EDvnUoWVnODdC8XMLZPkK5LlpKcmny+qlz57BDWqjztNMWcAr8joCL3OJySsrGM6p8AtVFRzA+fzxH541ndO7ojGg8yxhz8KxkGGCRhgY6li8ntGIFHctXEFqxgmhzM+AU7IHp08j7+MdxFxTgys/DnV+AOz8Pzc2mwdPJDmlmizawMbqTTe01bG7ZTG3bll5vrJZmlTI+fzzj88dzbPyzOr+aqrwq/G5/ur66MWaYskRwGGLt7YRWr+51th/eFn9VwuXCP2kSeWefRWDmTLJmzsR/5JF0aBcr6lewuWUzm1o2saXlPTa3bKamroaI7nncJs+XR3V+NcePOp5x+eOozq9mfP54xuWNI9eXm6ZvbIwZiSwR9JNGInRu2NDrbL9z/fqeenpvZSWBmTMouuoqsmbOIDBtGq7s7F77eHvn29zx6h1sb9sOOM3yjssfx6SiSZw1/qxeBX6hv9Dq6o0xg8ISQRKqSnjbdkLL34uf7a8gtGoVGnI6zHAXFBCYOZO8M88kMHMGWTNm4Ckp2ef+wrEwD7z3AA+veJgxOWO472P3cVTJUZRnl+MS12B9LWOMScoSARDZvZvQypXO2X684I82NgIgPh+BadMovPwysmbMJOuYmXjHju332fqWli0seHUBK+pXcPHEi1kwZwE53pxUfh1jjDkoGZcIYqEQodXvE1qxvOdsP7xli7NQBP/EI8k9/XSnemfGDAKTJzutGB4kVeV3G37Hv731b3hdXu497V7OqT5ngL+NMcYcvoxJBM3PP0/DI4/QuW49RJybsp6KCrJmzKDo8ssIzJhJYPp03LmHf7beFGri7jfu5i+b/8Kcijl895TvUpFTcdj7NcaYVMiYRCAuF57CQnI/9znnbP/oGXhHlQ/4cd7Y8QbfePUbNHY2cuvxt/KZ6Z+x+wDGmCEtYxJB/rnnkn/uuSnbf1e0i/vevo9frv4lEwom8LMzf8ZRJUel7HjGGDNQMiYRpNIHTR9w++LbWbt7LfOnzOers79KlqdvKz7GGDM0WSI4DKrKb9b+hnuX3kuON4efnvFTTh97errDMsaYg2KJ4BDVd9Tzrde/xeKaxZxceTL3nHwPpVml6Q7LGGMOmiWCQ7C4ZjH/+tq/0trVyh1z7uCKqVfYW8DGmGHLEsFBCEVC/Gjpj3hq7VNMLprMI2c/wsSiiekOyxhjDoslgn5a07iG2xffzsbmjXx62qe56bib8Lmtxy1jzPCX0gfcRWSeiKwVkQ0isiDJ8nEi8pKIvCMiy0XkvFTGcyhiGuOxlY9xxfNXEOwK8vOzfs5tJ9xmScAYM2Kk7IpARNzA/cBZQA2wRESeU9XVCavdCTytqgtFZBrwAlCdqpgO1s62nXzjtW/w5o43OXPcmXzrpG9RFChKd1jGGDOgUlk1NAfYoKobAUTkKeAiIDERKJAfHy8AtqcwnoPyl81/4a7X7yIcC3PXSXdx6aRL7YawMWZESmUiqAS2JkzXAHP7rHMX8GcR+TKQA3w8hfH0S3u4ne+/9X2e3fAs00um8/1Tv091QXW6wzLGmJRJ983iK4DHVPVHInIS8ISIHK2qvXplF5HrgOsAxo0bl7JgVtStYMGrC9ga3MoXZnyBLx77Rbyug2951BhjhpNUJoJtwNiE6ar4vESfA+YBqOo/RCQAlAK7EldS1QeBBwFmz56tDLBoLMrDKx5m4XsLKc8u59FzHmV2xeyBPowxxgxJqUwES4BJIjIBJwF8CriyzzpbgDOBx0TkKCAA1KUwpr1sa93GHa/ewTu73uHcCedy54l3ku/LP/CGxhgzQqQsEahqRERuBBYBbuBRVV0lIncDS1X1OeCrwEMicgvOjeNrVXXAz/j35Q8b/8B33/guAP926r9xwREXDNahjTFmyEjpPQJVfQHnkdDEed9MGF8NnJzKGJJp6Wrhu298lxc+fIFZ5bP43infoyqvarDDMMaYISHdN4sH3bKdy7jj1TvY1b6LG469gc/P+DweV8b9DMYY0yNjSsBwLMzCdxfyyMpHqMyt5PFzH2dm2cx0h2WMMWmXMYlg4bsLeWjFQ1wy8RJun3M7Od7D75vYGGNGgoxJBJ+Z/hmml07nzHFnpjsUY4wZUjKmV/UCf4ElAWOMSSJjEoExxpjkLBEYY0yGs0RgjDEZzhKBMcZkOEsExhiT4SwRGGNMhrNEYIwxGc4SgTHGZDhLBMYYk+EsERhjTIazRGCMMRnOEoExxmQ4SwTGGJPhLBEYY0yGS2kiEJF5IrJWRDaIyIIky/9DRN6ND+tEpCmV8RhjjNlbyjqmERE3cD9wFlADLBGR5+Id1gOgqrckrP9lYFaq4jHGGJNcKq8I5gAbVHWjqnYBTwEX7Wf9K4BfpzAeY4wxSaQyEVQCWxOma+Lz9iIi44EJwN/2sfw6EVkqIkvr6uoGPFBjjMlkQ+Vm8aeAZ1Q1mmyhqj6oqrNVdXZZWdkgh2aMMSNbKhPBNmBswnRVfF4yn8KqhYwxJi1SmQiWAJNEZIKI+HAK++f6riQiU4Ei4B8pjMUYY8w+9CsRiMiPRGT6wexYVSPAjcAi4H3gaVVdJSJ3i8iFCat+CnhKVfVg9m+MMWZg9Pfx0feBB0XEA/wC+LWqNh9oI1V9AXihz7xv9pm+q58xGGOMSYF+XRGo6sOqejLwaaAaWC4ivxKRj6UyOGOMManX73sE8RfEpsaHeuA94FYReSpFsRljjBkE/aoaEpH/AC7Aec7/e6r6VnzRD0RkbaqCM8YYk3r9vUewHLhTVduSLJszgPEYY4wZZP2tGmoiIWmISKGIXAzQn5vGxhhjhq7+JoJvJRb4qtoEfCs1IRljjBlM/U0EydZLWculxhhjBk9/E8FSEfmxiBwZH34MLEtlYMYYYwZHfxPBl4Eu4DfxoRO4IVVBGWOMGTz9qt6JPy20Vw9jxhhjhr/+vkdQBvwLMB0IdM9X1TNSFJcxxphB0t+qoSeBNTidx3wb2ITTuqgxxphhrr+JoERVHwHCqvqKqv4zMKyuBlSVzQ3J3oczxpjM1t9EEI5/7hCR80VkFlCcophS4r4XNzDvP19lVzCU7lCMMWZI6W8iuEdECoCvAl8DHgZuSVlUKXDRsWMIR2P89MUN6Q7FGGOGlAMmgniro5NUtVlVV6rqx1T1eFXdq7exoay6NIf5J4zl129tYVO9VREZY0y3AyaCeIfyVwxCLCl305mT8Lpd/Ogv69IdijHGDBn9rRp6TUR+JiKnishx3UNKI0uB8vwA/3xKNf/33nZWbrO28owxBvqfCI7FeYfgbuBH8eHeA20kIvNEZK2IbBCRpC+kicjlIrJaRFaJyK/6G/ih+n+nHUlhtpcfLrJuFIwxBvr/ZvFBd0kZv7dwP3AWUAMsEZHnVHV1wjqTgDuAk1V1t4iUH+xxDlZ+wMuXTj+S772whtc/qOcjR5am+pDGGDOk9ffN4m8mm6+qd+9nsznABlXdGN/HU8BFwOqEdb4A3K+qu+P729WfeA7Xp0+q5hevbeIHf1rL775UgogMxmGNMWZI6m/VUFvCEAXOxenEfn8qga0J0zXxeYkmA5NF5DUReUNE5iXbkYhcJyJLRWRpXV1dP0Pet4DXzS0fn8x7W5tYtKr2sPdnjDHDWb8Sgar+KGH4LnA6cMQAHN8DTIrv7wrgIREpTHL8B1V1tqrOLisrG4DDwqXHVTKxPJcfLlpLJBobkH0aY8xw1N8rgr6ygaoDrLMNGJswXRWfl6gGeE5Vw6r6IbAOJzGknMft4rZzprCxro1nltUMxiGNMWZI6lciEJEVIrI8PqwC1gL/eYDNlgCTRGSCiPiATwF9X0L7Hc7VACJSilNVtPEg4j8sZ08bxaxxhfznX9cTCkcH67DGGDOk9PeK4ALgE/HhbGCMqv5sfxuoagS4EVgEvA88raqrRORuEbkwvtoioEFEVgMvAbepasMhfI9DIiLcPm8qtS0hHnt902Ad1hhjhhRR1QOvJHIisEpVg/HpPGCaqr6Z4vj2Mnv2bF26dOmA7vPaX7zF25t38+q/nEFBtndA922MMUOBiCxT1dnJlvX3imAh0Jow3RafNyL8yzlTCXZGeGDxB+kOxRhjBl1/E4FowqWDqsbo5zsIw8G0MflcdMwYfvHah+xssWaqjTGZpb+JYKOIfEVEvPHhJgbxpu5guPWsKURjyn/+dX26QzHGmEHV30RwPfARnMc/a4C5wHWpCiodxpVkc+WccTy9dCsb61oPvIExxowQ/X2hbJeqfkpVy1V1lKpeOVjNQQymG8+YhN/j4kd/tmaqjTGZo7/vEfwy8Y1fESkSkUdTF1Z6lOX5+fypR/D8ih0sr2lKdzjGGDMo+ls1NFNVe0rGeCNxs1ITUnp94dQJFOf4+MGf1qQ7FGOMGRT9TQQuESnqnhCRYkbQU0OJ8gJebvjYRF7b0MCr6w+/gTtjjBnq+psIfgT8Q0S+IyL3AK8D/566sNLr6hPHUVmYxQ//tJZY7MAv3BljzHDW35vFjwOXAjuBWuDS+LwRye9xc+tZk1mxrZkXVu5IdzjGGJNS/W59VFVXx9sX+iPwT/HG50asi2dVMmVUHvcuWkvYmqk2xoxg/X1qaIyI3CIiS4BV8e0+ldLI0sztEm47ZwqbGtr5zZKtB97AGGOGqf0mgnjPYC8BLwMlwOeAHar6bVVdMQjxpdWZR5Uze3wRP3lxPR1d1ky1MWZkOtAVwc/i61ypqneq6nIgY+6eiggLzp1KXbCTR1/7MN3hGGNMShwoEYwGfg38SETWish3gIxqp3l2dTEfP6qcB175gKb2rnSHY4wxA26/iUBVG1T1AVU9DTgTaAJ2isj7IvK9QYlwCLjtnKm0dkb4r5etmWpjzMhzoHsEY7rHVbUm3nn9bOAiIGPaa55Skcclsyp57PVNbG/qSHc4xhgzoA5UNfSwiLwhIt8XkdNFxAOgqutU9e5BiG/IuPWsyaDwE2um2hgzwhyoaug8nM7lXwYuAd4Qkf+NP0007kA7F5F58XsLG0RkQZLl14pInYi8Gx8+f2hfI/WqirK5+sTx/M+yrWzYFUx3OMYYM2AO+B6BqoZU9U+qelO8WuirOO0M/UxE3trXdiLiBu4HzgWmAVeIyLQkq/5GVY+NDw8f2tcYHDeeMZFsn4d/X7Q23aEYY8yA6e8LZTki0r2uF6dzmn8CTtnPZnOADaq6UVW7gKdw7i0MW8U5Pq776BEsWrWTt7fsTnc4xhgzIPrbxMRiICAilcCfgWuAX8QL+H2pBBJfya2Jz+vrn0RkuYg8IyJjk+0oXhW1VESW1tWlt0XQz50ygdJcHz/44xoSunE2xphh62A6r2/HaXjuv1T1MmDGABz//4BqVZ0J/AX4ZbKVVPVBVZ2tqrPLysoG4LCHLsfv4ctnTOLNDxt5ZZ01U22MGf76nQhE5CTgKuD5fm67DUg8w6+Kz+sRf0+hMz75MHB8P+NJqyvmjGNccTY/sGaqjTEjQH8Twc3AHcCzqrpKRI4AXjrANkuASSIyQUR8OI3UPZe4goiMTpi8EHi/n/Gklc/j4qtnT+b9HS383/Lt6Q7HGGMOS3/7I3hFVS9U1R/EbxrXq+pXDrBNBLgRWIRTwD8dTyJ3i8iF8dW+IiKrROQ94CvAtYf8TQbZJ2aO4ajR+fzoz+voilgz1caY4au/Tw39SkTyRSQHWAmsFpHbDrSdqr6gqpNV9UhV/W583jdV9bn4+B2qOl1Vj1HVj6nqsOko2OUS/mXeFLY0tvPUki3pDscYYw5Zf6uGpqlqC3AxTsc0E3CeHMpop08uY+6EYu57cT1tnZF0h2OMMYekv4nAKyJenETwnKqGyaDmqPdFRLj93KnUt3bxyN+tmWpjzPDU30Twc2ATkAMsFpHxQEuqghpOjhtXxDnTR/Hg4o00tHYeeANjjBli+nuz+D5VrVTV89SxGfhYimMbNm47ZwrtXRHuf8maqTbGDD/9vVlcICI/7n67V0R+hHN1YICJ5Xl88vgq/vuNzdTsbk93OMYYc1D6WzX0KBAELo8PLcAvUhXUcHTzxyeDwH/8xZqpNsYML/1NBEeq6rfiDchtVNVvA0ekMrDhZkxhFtd+pJr/faeGtbXWTLUxZvjobyLoEJGelkZF5GTAuurq40unH0mu38O/Lxo2r0MYY0y/E8H1wP0isklENgE/A/5fyqIapgqzfVx/2pH89f1dLN3UmO5wjDGmX/r71NB7qnoMMBOYqaqzgDNSGtlAi0YgnPpulj97cjVleX5+8CdrptoYMzz094oAAFVtib9hDHBrCuJJnXefhJ8eB28/4SSFFMn2ebjpzEks2bSbv63ZlbLjGGPMQDmoRNCHDFgUg6FsCuRVwHM3wsKPwPt/gBSdsc8/YSzVJdn88E9riVoz1caYIe5wEsHwKuHGnQiffxEufwI0Br+5Ch45Gza9NuCH8rpdfPXsKazdGeR372w78AbGGJNG+00EIhIUkZYkQxAYM0gxDhwRmHYhfOkN+MR90FwDj50H//1JqF0xoIc6f8Zojq7M58d/WUdnJDqg+zbGmIEkw+2G5uzZs3Xp0qW95oXDYWpqagiFDvJmsCp0BSHU4lwl+HIgUAAuz4DEGgpHqW/tojDLS25gYPaZaoFAgKqqKrxeb7pDMcYMIBFZpqqzky0bHqXTAdTU1JCXl0d1dTUih3DrIhaB1l3QWgdEIacIckeB+/ALw411rYTCMSZX5OJ2HU5NXOqpKg0NDdTU1DBhwoR0h2OMGSRDu2Tqp1AoRElJyaElAXCuAPLHwKijILsY2upg12po2QGxw6vWqSgIEInFqGvtOqz9DAYRoaSk5OCvrIwxw9qISATAoSeBRG4fFI6DsqPAnw+ttU5CaN3lVB0dgmyfh4IsL/XBTsLRod+l5YD8jsaYYSWliUBE5onIWhHZICIL9rPeP4mIikjS+qtB5w1A8QQonQyeALRsg13vQ3vjIT1yWpEfQBXqgtZfgTFm6ElZIhARN3A/cC4wDbhCRKYlWS8PuAl4M1WxHDJfDpRMhOIjQdzQtBnq1kKo+aASgt/rpijHS0Nblz1BZIwZclJ5RTAH2BBvrbQLeAq4KMl63wF+AAzNimkRCOQ7L6QVjgeNQuNGaNgAXW0ANDU18V//9V/73c2o/AAC7GzZc1Vw3nnn0dTUdNAhXXvttTzzzDMHvZ0xxiSTykRQCWxNmK6Jz+shIscBY1X1+f3tSESu6+4Up66ubuAj7Q8R50Zy+VFQUAWRENSvg4aNNNXVJk0Ekciepiy8bhcluT6a2rvo6HKuCl544QUKCwsH7SsYY0wyaXt8VERcwI+Baw+0rqo+CDwIznsE+1v32/+3itXbB7Y75Wlj8vnWJ6Y7E+KCnDLIij9d1LqTBV/7Fz74YAPHHnMMXp+PQCBAUVERa9asYd26dVx88cVs3bqVUCjEZZ+5jqs+88+MKcxi+pSJLF26lNbWVs4991xOOeUUXn/9dSorK/n9739PVlbWAWN78cUX+drXvkYkEuGEE05g4cKF+P1+FixYwHPPPYfH4+Hss8/m3nvv5X/+53/49re/jdvtpqCggMWLFw/o72SMGZ5SmQi2AWMTpqvi87rlAUcDL8efVKkAnhORC1W19xtjQ5HL7bRdlF3K979zFyvXXsO7f/wlL7+9nvPnX8vKlSt7nsV/9NFHKS4upqOjg+OOn81Z511IR7iISEwJdoRBlfXr1/PrX/+ahx56iMsvv5zf/va3XH311fsNIRQKce211/Liiy8yefJkPv3pT7Nw4UKuueYann32WdasWYOI9FQ/3X333SxatIjKyspDqpIyxoxMqUwES4BJIjIBJwF8Criye6GqNgOl3dMi8jLwtcNNAj1n7oPF7YH80eDxQ1YRhJqYc+w0JpRmOe8guNzcd999PPvsswBs31YDLbWMqR6DKmzZ3U6ks4Px1dXMnHkMAMcffzybNm064KHXrl3LhAkTmDx5MgCf+cxnuP/++7nxxhC0kN0AAB5gSURBVBsJBAJ87nOf44ILLuCCCy4A4OSTT+baa6/l8ssv59JLL03N72GMGXZSdo9AVSPAjcAi4H3gaVVdJSJ3i8iFqTpu+ggUjYfCceTk5EFwB+xazct//B1//etf+cc//sF7773HrFmzCHd1Uprrx+sWqgqzcAHi9rKmNsiulhAirl73Fw6Wx+Phrbfe4pOf/CR/+MMfmDdvHgAPPPAA99xzD1u3buX444+noaFhgL67MWY4S+k9AlV9AXihz7xv7mPd01MZSyrl5eURDMb7Kfb4wZcNJZMguJ3m2g8pyvaQLZ2seX8zb7zxRq9tC7J9VJfm4PO4yPK5qW0JsSvYiYbDdEWi+DzufR53ypQpbNq0iQ0bNjBx4kSeeOIJTjvtNFpbW2lvb+e8887j5JNP5ogjnO6lP/jgA+bOncvcuXP54x//yNatWykpKUnZ72KMGR5GRFtD6VZSUsLJJ5/M0UcfTVZWFqNGjQJ/LvgmMe/Sq3jgv/+Xo2Yex5SJR3LinBP22l5EcIkwoTSHjq4oAa+LhrYoa2tbKcj2UpbrI8u39z9VIBDgF7/4BZdddlnPzeLrr7+exsZGLrroIkKhEKrKj3/8YwBuu+021q9fj6py5plncswxx6T8tzHGDH0jovXR999/n6OOOipNEfWDKnQ0QrAWol3gzXZaOfXnOeNJmnXoisSob+2ksa2LmCq5fg9leX5y/Z6UNwMx5H9PY8xBG/Gtjw55IpBd4txMbquPJ4UdziBu5+rBn+8kBo8fAJ/HxZjCLMrz/TS2ddHQ2sWH9W0EvG7K8vwUZHlxWbtAxpgBYIlgMIkLcsudIRpx+kLobIFQ0Gm2AsDtdxKCP48bvvp1Xnv99Z7NozHlin++ngs+eQW1bheluX6Kc3y4XZYQjDGHzhJBurg9zhVCVpFTdRTpdJJCZ9C5Ymiv5/47vwC+m/dcLXizUSAYilDX2smO5g52BUMU5/jiTyGNmMZkjTGDyBLBUCDitHjqDThXCxpz2jHqDDpDQjWS+HPJ9+eTX5RHeyxAXbCT+mAn9a1dFGV5Kc3zE/Du+0kjY4zpyxLBUCSunuohIKEaKd6tZrwaKdvtZ7w/j3BRLnVdHhrbwzS2d5Ef8FKW5yfb57b+BYwxB2SJYDg4QDWSt72eMUCFL4d2sqjr8rOxLkyWz0NZro/8LK8lBGPMPlkiGG72U43k6gySG64nF4i5XLRFsmjenUV9cw6FeTkUZftw2Y1lY0wflgjSIDc3l9bW1qTLNm3axAUXXMDKlSv7t7N9VCO5OoPkhlrIi7WB1tPZ7KGpJRtXIJ/c/CI8HvunN8Y4rDQYaRKqkSRejaSdLbg7WigMB3GFWtCOGjrdWbgCeXj8OYgvB9zedEdujEmTkZcI/rgAalcM7D4rZsC539/n4gULFjB27FhuuOEGAO666y48Hg8vvfQSu3fvJhwOc88993DRRck6aNu3UCjEF7/4RZYuXYrH4+HHP/4xH/vYx1i1ahWf/exn6erqIhaL8dvf/pYxY8Zw+eWXU1NTQzQa5V//9V+ZP38+eAOIN4AnXo3U1REk1NqMJ9yKr20X0u4cKypexJeNy5/jdLrT2eq86GaMGfFGXiJIg/nz53PzzTf3JIKnn36aRYsW8ZWvfIX8/Hzq6+s58cQTufDCCw/qpu3999+PiLBixQrWrFnD2Wefzbp163jggQe46aabuOqqq+jq6iIajfLCCy8wZswYnn/e6eytubl57x2KC192Ab7sAsLRGE0dXXSG2qCrjUAsRFaoDX9nM7TuQr9/Blo2FVfl8dA9lE9zrjiMMSPKyPur3s+Ze6rMmjWLXbt2sX37durq6igqKqKiooJbbrmFxYsX43K52LZtGzt37qSioqLf+/373//Ol7/8ZQCmTp3K+PHjWbduHSeddBLf/e53qamp4dJLL2XSpEnMmDGDr371q9x+++1ccMEFnHrqqfvdt9ftoig3ALkBVIvpCEdpDkVoD3XSQCc/CV/McTs3Mqv+9+S98wQA6slCxhwbTwzHOZ+F45O2lWSMGT5GXiJIk8suu4xnnnmG2tpa5s+fz5NPPkldXR3Lli3D6/VSXV1NKBQakGNdeeWVzJ07l+eff57zzjuPn//855xxxhm8/fbbvPDCC9x5552ceeaZfPObSVv83ouIkO3zkO3zQH6Atvp8pl/5ff6yro5vrttFbPcmjpUPONm9mbn1HzK25iHcsU5n4+zSPVcM3Qkiu3hAvqcxZnBYIhgg8+fP5wtf+AL19fW88sorPP3005SXl+P1ennppZfYvHnzQe/z1FNP5cknn+SMM85g3bp1bNmyhSlTprBx40aOOOIIvvKVr7BlyxaWL1/O1KlTKS4u5uqrr6awsJCHH374kL+LS4SzjhrFWdNGAbCloZ3F6+v42/o67tnQQEdniKNcWzm/ZDunZG3hyLq1BNb/GSHekm3xEb2TQ8VM53FXY8yQZIlggEyfPp1gMEhlZSWjR4/mqquu4hOf+AQzZsxg9uzZTJ069aD3+aUvfYkvfvGLzJgxA4/Hw2OPPYbf7+fpp5/miSeewOv1UlFRwde//nWWLFnCbbfdhsvlwuv1snDhwgH7buNKsrm6ZDxXnziecDTGe1ubWLyujj+tr+eHW5uIKVQEurh8TAMfy93KlOg6sje9Biv+x9mBywOjjt6TGKpmOx33uKxtJGOGAuuPwOzlYH7PpvYuXtvQwOJ1dSxeX8eOZqf6a0JpDuePV84q3MZRsfX4at+Gbe84TWUA+PKgclY8McyBsXMgp3Q/RzLGHI609UcgIvOAnwBu4GFV/X6f5dcDNwBRoBW4TlVXpzImM7AKs32cP3M0588cjaryQV0bi9fV8er6Oh5Z3sjPwkV43XM5btw8PnpiCWeVtTAxshbXtmWwbRm8/lOIxftnLj4Sxs6FcXOdz9IpdtVgzCBI2RWBiLiBdcBZQA2wBLgisaAXkXxVbYmPXwh8SVXn7W+/I+WKYMWKFVxzzTW95vn9ft588800RbTHQP2enZEoyzbt5pX1dby6rp7VO1oAKM7xccrEUk6dVMpHJ+QyqnUNbH0Ttr7lfLbXOzvwF8DYE5ykMHaOc/XQ/Qa1MeagpOuKYA6wQVU3xoN4CrgI6EkE3UkgLgcYXvVUh2HGjBm8++676Q4jpfweNx+ZWMpHJpZyx7lQF+zk7xucpLB4fT3PvbcdgLHFWcys/Cgzqj7BjNn5zMxuIK/u7T3J4aXvAeo0pzHq6HhiiCeHwnH2+KoxhymViaAS2JowXQPM7buSiNwA3Ar4gDOS7UhErgOuAxg3btyAB2oGR1men0tmVXHJrCpiMWVNbZC/b6jj3a1NLN/WxPMrdvSsW10ymhlV1zLj6K9wbJmLo3Ud2bVLneTw7q9gyUPOinmjnYTQnRwqZoLHl6ZvaMzwlPanhlT1fuB+EbkSuBP4TJJ1HgQeBKdqaHAjNKngcgnTxuQzbUx+z7zdbV2s2NbsDDXNvL15N/8Xv2oAOKLsZGZUnsfMU3KYm7uLSZ2r8O9YClvfgNW/d1byBGDMrHhyONFuQhvTD6lMBNuAsQnTVfF5+/IUMHDPPJphpyjHx0cnl/HRyWU98xpaO3sSw/Jtzbz1YSO/f9dJDiJVHFk2lZmVN3DCjE5OcG9gfPtKvNuXwD/+C177ibOT7pvQ3VcOZVPtJrQxCVKZCJYAk0RkAk4C+BRwZeIKIjJJVdfHJ88H1mNMgpJcP6dPKef0KeU983YFQ6zc1syKmhZWbGvi7xvq+d93OoFCXHIKk8rP5djJAU7Pq2GmrqWiZTnu9X+G937l7OBQbkLHok5jfJHO+BCCaFefefH5+1vWMz8EkYR1ovF1NAZ5FZBf6QwFlXvG8yrAZd2QmoGXskSgqhERuRFYhPP46KOqukpE7gaWqupzwI0i8nEgDOwmSbXQcNDU1MSvfvUrvvSlLx3Udueddx6/+tWvKCwsTFFkI1N5XoAzpgY4Y+qonnk7W0I9Vw0rapp4cUMzv2n1AzNxu45hUtn1fGxSK6cEPmBK1/sUN76Da0PCTeiyqU5T3D2FeZ9CuvsR18Ph8jhVVx4/uP3Opyfg3NPonq8KO1fD+r9AuL339uJ27onkj+mdIAoqIb/KmZ9bbsnCHLQR90LZD976AWsa1wzoMacWT+X2Obfvc/m+OpOJRCLDsgOY4fg4bl+qSm1LiOU1zazc1szyGufeQ2NbFwAel3BcuXBOYQ0neNZT3bmBHL8btzfQp3A+QMHt9u8ZT7osYfnBFNCqEGqC5m3QEh+SjUf6tF/l8kDemH0ki/iQU2ZVYxkobS+UZYoFCxbwwQcfcOyxx+L1egkEAhQVFbFmzRrWrVvHxRdfzNatWwmFQtx0001cd911AFRXV7N06VJaW1s599xzOeWUU3j99deprKzk97//PVlZWUmP99BDD/Hggw/S1dXFxIkTeeKJJ8jOzmbnzp1cf/31bNy4EYCFCxfykY98hMcff5x7770XEWHmzJk88cQTg/bbpIuIMLogi9EFWZwz3WnxVVXZ3hxiRU1TT2L46RY3Te1jgNNwCVSX5jC1Io/Jo/KYWpHHlIp8xhVn4x7sLj5F9vRTXXF08nVUob0xITnUQMv2PYli2zJ4//+c6qhEbl/8yqJyP8mi1B7LzSAj7oogHRKvCF5++WXOP/98Vq5cyYQJEwBobGykuLiYjo4OTjjhBF555RVKSkp6JYKJEyeydOlSjj32WC6//HIuvPBCrr766qTHa2hooKSkBIA777yTUaNG8eUvf5n58+dz0kkncfPNNxONRmltbaWmpoZLLrmE119/ndLS0p5Y9ifdv+dgUlVqdnewYlsza2qDrK1tYW1tkM2N7XT/aQS8LiaPSkwOzlCW6z+o/iXSQhXa6qElniSatyUZ3wGxcO/txAWerHj/2NnOVY03a8/gyeozHV+v1/r7mu6zvdt3aEknFoVwh3NV1OuzEyIdEA4dwmd86DsvkA8FVVAw1vnMr0yYrgRfzsD8e6WQXREMsjlz5vQkAYD77ruPZ599FoCtW7eyfv36noK824QJEzj22GMBOP7449m0adM+979y5UruvPNOmpqaaG1t5ZxzzgHgb3/7G48//jgAbrebgoICHn/8cS677DJKS51HKA+UBDKNiDC2OJuxxdmcN2N0z/z2rgjrd7aydmeQtbXO8PLaOp5ZVtOzTnGOjymj9iSGKRV5TBmVR45/CP1ZiUBumTOMmZV8nVgM2uqcpNC8zUkSbbt6F4bh9ngB2Q5d7dDWEF/WkVAItyff/4GDTJI04kkCSV5Yhzv2Tl4Hw+13jted7BI//XlO9Vl3HB4/hJqdK66NL0Nwh3NTP1FWce9E0TPEp3NHDenquCH0P3bkyMnZc3bw8ssv89e//pV//OMfZGdnc/rppyftl8Dv9/eMu91uOjo69rn/a6+9lt/97nccc8wxPPbYY7z88ssDGr+BbJ+HY8YWcszY3jfyG1o7ncQQTxBraoM8vXQr7V3RnnXGFmcxZVS+U8VU4VxFTCjNweseogWBywV5o5yh8vhD30+8j+w9SSMxUSRJGr0STN91uxOLQqBg34X2vj49/oQrlT6fnsDhFcrRsJMMmmviw9Y947s/hE2vQmdL721c3vh9m2SJIn51kcbmUywRDIC8vDyCwWDSZc3NzRQVFZGdnc2aNWt44403Dvt4wWCQ0aNHEw6HefLJJ6msrATgzDPPZOHChb2qhs444wwuueQSbr31VkpKSvpVNWT2rSTXz0cm+vnIxD0vqcViTvXSmni1UneSeGntLqIxp37J53ZxRFlOz32HKRW5TKnIZ0xBYOhXL/WXSPysfoT3PeH2Ok2bFO6nlYNQs3N11TdRNNfA5ted+zga7b1NoGDfVxQFVZBbkbKuYi0RDICSkhJOPvlkjj76aLKyshg1as9jjfPmzeOBBx7gqKOOYsqUKZx44omHfbzvfOc7zJ07l7KyMubOnduThH7yk59w3XXX8cgjj+B2u1m4cCEnnXQS3/jGNzjttNNwu93MmjWLxx577LBjMHu4XMK4kmzGlWRz9vQ9XZF2RqJ8sKuNtTtbWFMbZF1tkLc+bOR37+55Wzov4OlVvTR5VB7jirMZlR8Y/BvUZuAECpxh1LTky2NRCNYmTxTNNbDlDeepsUTihvPvhdn/PODh2s1isxf7PVOruSPMup3BXjen19QGCYb2vKvgdQtjCrOoKspibFG281mc3TNdmuvHZYliZOtsjT8BlpAoJp8LVYdWfWc3i40ZQgqyvJxQXcwJ1Xuq6Lrfe1i/s5Wtu9up2d3B1kbn86/v76S+tfcjoD6Pi6qiLKqKshnb/Vm8Z7o4xzdyqpwylT8XyqY4Q4pZIhjCbrjhBl577bVe82666SY++9nPpikikyqJ7z0k09EVpaY7QfRJFCtqmtjd3vsJmmyfez+JIpv8LI8lCtPDEsEQdv/996c7BDNEZPncTBqVx6RRyZ8sCYbCbGvqYGtjBzW72/d87u5gyYeNBDt7N5GR5/dQ2ae6KbH6KS/gHYyvZYYISwTGjAB5AS9TK7xMrchPury5PRy/kuh9NbG5oY2/r6+nI9z7CZbCbC/j4u9XjEsYxhZlM7owMHQfhTWHxBKBMRmgINtLQXYBR1cW7LVMVWls6+pV7bSlsZ2tje2s2tbMopW1RGJ7Hipxu4QxhYE9yaFPoijM9lq10zBjicCYDCcilOT6Kcn17/UCHUA05tzI3tLgJIctCcNfVu99IzvP79mTHEp6J4rKwix8HruaGGosERhj9svtEioLs6gszOKkI0v2Wt7WGWHr7na2NLT3XElsaWxnQ10rf1u7i67InuYYRGB0fqB3lVM8WTiPxdrTTulgiSANcnNzaW1tTXcYxgyIHL+HqRX5Se9PxGJKXWuncwXRnSh2O8li8fo6drZ09lo/y+vuVd1UWZTFqHw/FfkBRuUHKM/34/dYfwsDbcQlgtrvfY/O9we2PwL/UVOp+PrXB3SfxmQCl0sYFS/EE9+b6BYKO4/FdieKrQn3J17/oL5XG07dinN88X3uSRCj8gNUFPidz/wARdk+e+HuIIy4RJAOCxYsYOzYsdxwww0A3HXXXXg8Hl566SV2795NOBzmnnvu4aKLLjrgvlpbW7nooouSbpesX4F99UFgzHAQ8LqZWJ7HxPK9H4tVVZo7wtS2hNjZ0snO5lB83BlqW0Ks3NZCQ1snfRtI8LqF8rwAFQVOwuhOEHuShjM/22dFIFgTEwPinXfe4eabb+aVV14BYNq0aSxatIiCggLy8/Opr6/nxBNPZP369YjIfquGIpEI7e3te223evXqpP0KJOuDoKBg7ydDDka6f09jDkY4GqMu2EltS4hdLSFqm0PUtnQ64/FhV0snrZ17dzeaF/DsdVVRkR+gPJ44KgoClOb6R0S7T2lrYkJE5gE/wemz+GFV/X6f5bcCnwciQB3wz6q6OZUxpcKsWbPYtWsX27dvp66ujqKiIioqKrjllltYvHgxLpeLbdu2sXPnTioqKva7L1Xl61//+l7b/e1vf0var0CyPgiMySRet4sxhVmMKUz+Vna31s4Itc2hXgliZ7NztVHbEuKDD+rZFezsaTG2m0ucVmfLcv2U5fkp7fn0UZbnjHcvK8gano/OpiwRiIgbuB84C6gBlojIc6q6OmG1d4DZqtouIl8EfgjMT1VMqXTZZZfxzDPPUFtby/z583nyySepq6tj2bJleL1eqqurk/ZD0NehbmeM2b9cv4eJ5blMLM/d5zrRmNLQ1snO5s6e6qedLSHqgp3UBTupb+1k/c4gda2dhKN716Z43dKTKMpy9ySNZAkk1z90mvlI5RXBHGCDqm4EEJGngIuAnkSgqi8lrP8GkLxvxmFg/vz5fOELX6C+vp5XXnmFp59+mvLycrxeLy+99BKbN/fvQqe5uTnpdvvqVyBZHwR2VWDMoXG7nHsL5XkBZrDvvyNVpaUjQl1riLpgF3WtexJFd9KobQmxYlszDW1de11lgNMFaq+kkbf3VUf3dJYvtU9KpTIRVAJbE6ZrgLn7Wf9zwB9TGE9KTZ8+nWAwSGVlJaNHj+aqq67iE5/4BDNmzGD27NlMnTq1X/vZ13bTp09P2q/AvvogMMakjojE39b2MrF8/+vGYsrudidZ1Ae74smjk/rWrp6ksaWxnWWbd9PY3rXXjW9wrmZKc33cctZkLjq2cuC/T6puFovIJ4F5qvr5+PQ1wFxVvTHJulcDNwKnqWpnkuXXAdcBjBs37vi+Z9d2c3Ng2e9pTHpEojEa27rYlXh10ZNAOpk/eyynTCo98I6SSNfN4m3A2ITpqvi8XkTk48A32EcSAFDVB4EHwXlqaOBDNcaY9PO4XZTHn1oa1OOmcN9LgEkiMgEnAXwKuDJxBRGZBfwc58phVwpjGXJWrFjBNddc02ue3+/nzTffTFNExphMlbJEoKoREbkRWITz+OijqrpKRO4Glqrqc8C/A7nA/8Tvnm9R1QsP8XhD5g58f8yYMYN333033WHsZbi9V2KMOXwpfY9AVV8AXugz75sJ4x8fiOMEAgEaGhooKSkZVslgqFFVGhoaCAQG97LUGJNeI+L96qqqKmpqaqirq0t3KMNeIBCgqqoq3WEYYwbRiEgEXq+XCRMmpDsMY4wZlqyHCGOMyXCWCIwxJsNZIjDGmAw37JqhFpE64FBbKC0F6gcwnOHOfo/e7PfYw36L3kbC7zFeVcuSLRh2ieBwiMjSfb1inYns9+jNfo897LfobaT/HlY1ZIwxGc4SgTHGZLhMSwQPpjuAIcZ+j97s99jDfoveRvTvkVH3CIwxxuwt064IjDHG9GGJwBhjMlzGJAIRmScia0Vkg4gsSHc86SIiY0XkJRFZLSKrROSmdMc0FIiIW0TeEZE/pDuWdBORQhF5RkTWiMj7IpKxfZ+KyC3xv5OVIvJrERmRTfNmRCIQETdwP3AuMA24QkSmpTeqtIkAX1XVacCJwA0Z/Fskugl4P91BDBE/Af6kqlOBY8jQ30VEKoGvALNV9WicflU+ld6oUiMjEgEwB9igqhtVtQt4CrgozTGlharuUNW34+NBnD/yge8NexgRkSrgfODhdMeSbiJSAHwUeARAVbtUtSm9UaWVB8gSEQ+QDWxPczwpkSmJoBLYmjBdQ4YXfgAiUg3MAjK9f8z/BP4FiKU7kCFgAlAH/CJeVfawiOSkO6h0UNVtwL3AFmAH0Kyqf05vVKmRKYnA9CEiucBvgZtVtSXd8aSLiFwA7FLVZemOZYjwAMcBC1V1FtAGZOQ9NREpwqk5mACMAXJE5Or0RpUamZIItgFjE6ar4vMykoh4cZLAk6r6v+mOJ81OBi4UkU04VYZniMh/pzektKoBalS1+yrxGZzEkIk+DnyoqnWqGgb+F/hImmNKiUxJBEuASSIyQUR8ODd8nktzTGkhTqfOjwDvq+qP0x1PuqnqHapaparVOP8v/qaqI/Ksrz9UtRbYKiJT4rPOBFanMaR02gKcKCLZ8b+bMxmhN85HRFeVB6KqERG5EViEc+f/UVVdleaw0uVk4BpghYi8G5/3dVV9IY0xmaHly8CT8ZOmjcBn0xxPWqjqmyLyDPA2ztN27zBCm5qwJiaMMSbDZUrVkDHGmH2wRGCMMRnOEoExxmQ4SwTGGJPhLBEYY0yGs0RgMpqIREXk3YRhwN6iFZFqEVl5EOvniMhf4+N/j7dvY0zK2X80k+k6VPXYdAcRdxLwj3jTBm2qGkl3QCYz2BWBMUmIyCYR+aGIrBCRt0RkYnx+tYj8TUSWi8iLIjIuPn+UiDwrIu/Fh+6mCNwi8lC8Tfs/i0hWkmMdGX+577+BK4FlwDHxK5TyQfrKJoNZIjCZLqtP1dD8hGXNqjoD+BlOC6UAPwV+qaozgSeB++Lz7wNeUdVjcNrm6X5zfRJwv6pOB5qAf+obgKp+EL8qWYbTZPovgc+p6rGqumtAv60xSdibxSajiUirquYmmb8JOENVN8Yb6atV1RIRqQdGq2o4Pn+HqpaKSB1QpaqdCfuoBv6iqpPi07cDXlW9Zx+xLFHVE0Tkt8BNqlozwF/XmKTsisCYfdN9jB+MzoTxKEnuy4nIA/GbypPiVUTzgD+IyC2HeExjDoolAmP2bX7C5z/i46+zp7vCq4BX4+MvAl+Env6PC/p7EFW9Hvg28B3gYuD5eLXQfxxe+Mb0jz01ZDJdVkIrrOD01dv9CGmRiCzHOau/Ij7vyzi9d92G05NXd8ucNwEPisjncM78v4jTq1V/nQY8DpwKvHJI38SYQ2T3CIxJIn6PYLaq1qc7FmNSzaqGjDEmw9kVgTHGZDi7IjDGmAxnicAYYzKcJQJjjMlwlgiMMSbDWSIwxpgM9/8BdYsOSoGqPWoAAAAASUVORK5CYII=\n",
1042 |             "text/plain": [
1043 |               "<Figure size 432x288 with 1 Axes>"
1044 |             ]
1045 |           },
1046 |           "metadata": {
1047 |             "tags": [],
1048 |             "needs_background": "light"
1049 |           }
1050 |         }
1051 |       ]
1052 |     },
1053 |     {
1054 |       "cell_type": "code",
1055 |       "metadata": {
1056 |         "id": "DhA9iS9JcUx1",
1057 |         "colab_type": "code",
1058 |         "colab": {}
1059 |       },
1060 |       "source": [
1061 |         "# Define class labels\n",
1062 |         "LABELS = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
1063 |         "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
1064 |       ],
1065 |       "execution_count": 0,
1066 |       "outputs": []
1067 |     },
1068 |     {
1069 |       "cell_type": "code",
1070 |       "metadata": {
1071 |         "id": "7C7hRsy_a7K5",
1072 |         "colab_type": "code",
1073 |         "colab": {}
1074 |       },
1075 |       "source": [
1076 |         "# Create adversarial example\n",
1077 |         "############################\n",
1078 |         "\n",
1079 |         "# Clipping utility so that the pixel values stay within [0,1]\n",
1080 |         "EPS = 1./10000\n",
1081 |         "\n",
1082 |         "def clip_eps(delta_tensor):\n",
1083 |         "    return tf.clip_by_value(delta_tensor, clip_value_min=-EPS, clip_value_max=EPS)\n",
1084 |         "\n",
1085 |         "def generate_adversaries_targeted(image_tensor, delta, \n",
1086 |         "    true_index, target_index, model):\n",
1087 |         "    # Loss and optimizer\n",
1088 |         "    scc_loss = tf.keras.losses.SparseCategoricalCrossentropy()\n",
1089 |         "    optimizer = tf.keras.optimizers.Adam(learning_rate=2e-1)\n",
1090 |         "\n",
1091 |         "    for t in range(300):\n",
1092 |         "        plt.show()\n",
1093 |         "        with tf.GradientTape() as tape:\n",
1094 |         "            tape.watch(delta)\n",
1095 |         "            inp = (image_tensor + delta)/255\n",
1096 |         "            predictions = model(inp, training=False)\n",
1097 |         "            loss = (- scc_loss(tf.convert_to_tensor([true_index]), predictions) + \n",
1098 |         "\t                    scc_loss(tf.convert_to_tensor([target_index]), predictions))\n",
1099 |         "            if t % 20 == 0:\n",
1100 |         "                print(t, loss.numpy())\n",
1101 |         "                # plt.imshow(50*delta.numpy().squeeze()+0.5)\n",
1102 |         "            \n",
1103 |         "        # Get the gradients\n",
1104 |         "        gradients = tape.gradient(loss, delta)\n",
1105 |         "        \n",
1106 |         "        # Update the weights\n",
1107 |         "        optimizer.apply_gradients([(gradients, delta)])\n",
1108 |         "\n",
1109 |         "        # Clip so that the delta values are within [0,1]\n",
1110 |         "        delta.assign_add(clip_eps(delta))\n",
1111 |         "\n",
1112 |         "    return delta\n",
1113 |         "\n",
1114 |         "def perturb_image(image, true, target, model):\n",
1115 |         "    # Load and preprocess image but a but without any `preprocess_input`\n",
1116 |         "    plt.imshow((image.reshape(28, 28)), cmap=plt.cm.binary)\n",
1117 |         "\n",
1118 |         "    # Add batch dim\n",
1119 |         "    image = np.expand_dims(image, 0)\n",
1120 |         "\n",
1121 |         "    # Generate predictions before any adversaries\n",
1122 |         "    unsafe_preds = model.predict(image)\n",
1123 |         "    print('Predicted before adv.:', LABELS[unsafe_preds[0].argmax()])\n",
1124 |         "\n",
1125 |         "    # Initialize the perturbation quantity\n",
1126 |         "    image_tensor = tf.constant(image*255, dtype=tf.float32)\n",
1127 |         "    delta = tf.Variable(tf.zeros_like(image_tensor), trainable=True)\n",
1128 |         "\n",
1129 |         "    # Get the adversary vector\n",
1130 |         "    delta_tensor = generate_adversaries_targeted(image_tensor, delta, true, target, model)\n",
1131 |         "\n",
1132 |         "    # See if the image changes\n",
1133 |         "    plt.imshow((image_tensor + delta_tensor).numpy().squeeze()/255, cmap=plt.cm.binary)\n",
1134 |         "    plt.show()\n",
1135 |         "\n",
1136 |         "    # Generate prediction\n",
1137 |         "    perturbed_image = (image_tensor + delta_tensor)/255\n",
1138 |         "    preds = model.predict(perturbed_image)\n",
1139 |         "    print('Predicted:', LABELS[preds[0].argmax()])\n",
1140 |         "\n",
1141 |         "    return (image_tensor + delta_tensor)/255"
1142 |       ],
1143 |       "execution_count": 0,
1144 |       "outputs": []
1145 |     },
1146 |     {
1147 |       "cell_type": "code",
1148 |       "metadata": {
1149 |         "id": "G5t8ILAjc_8s",
1150 |         "colab_type": "code",
1151 |         "colab": {
1152 |           "base_uri": "https://localhost:8080/",
1153 |           "height": 819
1154 |         },
1155 |         "outputId": "581e465f-c513-4fe9-bfe2-feff5d4a25dc"
1156 |       },
1157 |       "source": [
1158 |         "# Randomly select an image\n",
1159 |         "idx = int(np.random.choice(X_train.shape, 1))\n",
1160 |         "sample_image = X_train[idx]\n",
1161 |         "\n",
1162 |         "# Let's go (target -> Sandal (idx: 5))\n",
1163 |         "print(\"Original label \",LABELS[y_train[idx]])\n",
1164 |         "perturbed_image = perturb_image(sample_image, y_train[idx], 5, apparel_model)"
1165 |       ],
1166 |       "execution_count": 82,
1167 |       "outputs": [
1168 |         {
1169 |           "output_type": "stream",
1170 |           "text": [
1171 |             "Original label  T-shirt/top\n",
1172 |             "Predicted before adv.: T-shirt/top\n"
1173 |           ],
1174 |           "name": "stdout"
1175 |         },
1176 |         {
1177 |           "output_type": "display_data",
1178 |           "data": {
1179 |             "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAARqklEQVR4nO3dXYyV5bUH8P8SGEAG+ZBxBMFDJZAomgLZGU8saTxpTlFugBtSLionNtILSVpDYo1e1JuTqJH21ORIpEpKj9XSSDliYk7wIAQbTeMGQUA4Dhq0AsLwLfIN61zMSzPFedca9rPf/W5n/X/JZGb22u/ez2zmz7tnr/08j6gqiKj/u67sARBRYzDsREEw7ERBMOxEQTDsREEMbOSdjRkzRidOnNjIu+wXzp49a9Y///zz3NqoUaPMY6+//nqzLiJJdWvsx44dM48dPHiwWb/55pvN+oABA8x6f7R3714cPny413+UpLCLyH0AfgNgAIAXVfUp6/oTJ05EtVpNucvCeC1I75e6SLt27TLrixcvzq3Nnz/fPHb69OlmvaWlxawPHGj/Cu3cuTO3tmbNGvPY2267zaw/+uijZn3kyJFmvT+qVCq5tZqfxovIAAD/CeB+AHcAWCAid9R6e0RUrJS/2TsA7FHVT1X1PIA/AphTn2ERUb2lhP0WAH/r8f0X2WX/QEQWiUhVRKpdXV0Jd0dEKQp/NV5Vl6tqRVUrbW1tRd8dEeVICfs+ABN6fD8+u4yImlBK2N8HMFlEviMiLQB+BGBtfYZFRPUmKbPeRGQ2gP9Ad+tthar+u3X9SqWiRbXeymydffDBB2Z91apVZn316tVm3esXnzp1Krd25swZ89ijR4+a9SJNmTLFrF93nX0u2r17t1m3+vCzZs0yj12yZIlZv+uuu8x6WSqVCqrVav377Kr6JoA3U26DiBqDb5clCoJhJwqCYScKgmEnCoJhJwqCYScKoqHz2YuU2kc/efKkWX/ggQdya9u2bTOP9d4D0NraataHDh1q1q05616P/uLFi2b9xIkTZt2bD2/df+q/WUdHh1m35tK/++675rEbN2406zNnzjTrL7/8slkvA8/sREEw7ERBMOxEQTDsREEw7ERBMOxEQfSb1luqefPmmXVrueb29nbzWK/FdOnSJbOesiSyd9teW/DGG29Muv2U+05ltSyHDBliHuv9m73zzjtm3VsR+PbbbzfrReCZnSgIhp0oCIadKAiGnSgIhp0oCIadKAiGnSiIMH32zZs3m3Wrjw4AY8aMya1500Q93nLP+/bZe29Yx1++fNk81tuF1euje8s9W86fP2/WBw0aZNaHDx9u1sePH59b835uj/dzv/jii2Z96dKlSfdfC57ZiYJg2ImCYNiJgmDYiYJg2ImCYNiJgmDYiYII02ffsGGDWT937pxZt5Yl9nquXq978ODBZv2ZZ54x62PHjs2tTZgwwTx2//79Nd824P9sVq/c67NbW1EDwJYtW8z6c889l1tra2szj71w4YJZ9/7NvW24y+izJ4VdRPYC+ArAJQAXVbVSj0ERUf3V48z+L6p6uA63Q0QF4t/sREGkhl0BrBORzSKyqLcriMgiEamKSLWrqyvx7oioVqlhn6mqMwDcD+BhEfn+1VdQ1eWqWlHViveiCBEVJynsqrov+3wIwBoA9k57RFSamsMuIsNEZPiVrwH8EMCOeg2MiOor5dX4dgBrsvW1BwJ4RVX/py6jKsBrr71m1r212a1+sjc3+vTp02Z9xIgRZv2hhx4y6+vWrcutefP4H3zwQbP+wgsvmPWpU6eadev9Cd5c+ZtuusmsP/LII2b9+eefz615fXRr3AAwbNgws757926z/vHHH+fWpkyZYh5bq5rDrqqfAvhuHcdCRAVi640oCIadKAiGnSgIhp0oCIadKIgwU1y3bdtm1r2poFabyJse6zlx4kTS8bNmzcqttba2msd6Wws/++yzZt3b6vqNN97IrXlLcE+fPt2se1NcrZao1w71prB6de/36b333sutFdV645mdKAiGnSgIhp0oCIadKAiGnSgIhp0oCIadKIh+02ffvn27WfdWyfGmuFp9dm+qprcl8+jRo826Z+fOnbk1b5nqAwcOmPUnnnjCrKuqWbeWkvaOtXrRfWEtg+0toe39PmRTu3MNHTrUrG/atCm3tnDhQvPYWvHMThQEw04UBMNOFATDThQEw04UBMNOFATDThREv+mzP/3002bd63V7SwOnzI0eMmSIWbd60QBQrVbN+pEjR3JrR48eNY/1llQ+ePCgWffGbv3s3pbNx48fN+urVq0y68eOHcuteX1w7769473H1Vviuwg8sxMFwbATBcGwEwXBsBMFwbATBcGwEwXBsBMF0W/67Pfcc49Z9/rFe/bsMevW2u5en33y5Mlm3VuD/O677zbr1tzr1PXPra2qAb+fbM1Z97a69tYJuOGGG8y6tf76119/bR7r/dzeXPxx48aZ9blz55r1IrhndhFZISKHRGRHj8tGi8hbItKZfR5V7DCJKFVfnsb/DsB9V132GID1qjoZwPrseyJqYm7YVXUTgKvfczkHwMrs65UAGv+chIiuSa0v0LWr6pXFy74E0J53RRFZJCJVEal2dXXVeHdElCr51XjtfqUi99UKVV2uqhVVrXiLPhJRcWoN+0ERGQsA2edD9RsSERWh1rCvBXBlvduFAF6vz3CIqCji9QtF5FUA9wIYA+AggF8C+G8AfwJwK4DPAMxXVXviNIBKpaLe3OyyWHOfAaCzszO3tmzZMvPYjRs3mvVbb73VrHv7t48cOTK35s0Z9/rJRfJ+97yxeesEWI/bnXfeaR77yiuvmPVmValUUK1We13U3n1TjaouyCn9IGlURNRQfLssURAMO1EQDDtREAw7URAMO1EQ/WaKa6pRo+yJex0dHbk1b1vkt99+26x72/+eO3fOrFvTNS9evGge601x9XjtM6vu3bf3c3vLWJ89eza35k2J7o94ZicKgmEnCoJhJwqCYScKgmEnCoJhJwqCYScKIkyf3esHe0sit7S05Na8Pvnw4cPNurdksrVUdF/u39KHKc4133bRUqbnWtOC+8L7N/PeQ1DG48ozO1EQDDtREAw7URAMO1EQDDtREAw7URAMO1EQYfrsXl/TmxttmTRpkln3thb25pxbPX6P93M3c5/d+7m9ZbItI0aMqPlYwO/xe++NKAPP7ERBMOxEQTDsREEw7ERBMOxEQTDsREEw7ERBhOmze1L6pkOHDjWP9daVt9Y3B/z3AFhz8VP76CnrwgNpc869LZlPnz5t1q2xNWMfvGjumV1EVojIIRHZ0eOyJ0Vkn4hszT5mFztMIkrVl6fxvwNwXy+X/1pVp2Ufb9Z3WERUb27YVXUTgKMNGAsRFSjlBbrFIvJh9jQ/d6M0EVkkIlURqXZ1dSXcHRGlqDXsywBMAjANwAEAS/OuqKrLVbWiqpW2trYa746IUtUUdlU9qKqXVPUygN8CyN/ilIiaQk1hF5GxPb6dB2BH3nWJqDm4fXYReRXAvQDGiMgXAH4J4F4RmQZAAewF8NMCx9gQKfO2vTXCU9cQT+2Fp9x2Sp8csMeWMm7Af1yttd1T96Vv5vX087hhV9UFvVz8UgFjIaIC8e2yREEw7ERBMOxEQTDsREEw7ERBcIprA+zfv9+se9sHe9sDW1KnqJbJG5s39dc63lu+uz/imZ0oCIadKAiGnSgIhp0oCIadKAiGnSgIhp0oCPbZM0VOWUxdttjbmtiarpnaZy9yKWrvWO/n9pbotm4/tc/+bZziyjM7URAMO1EQDDtREAw7URAMO1EQDDtREAw7URDsszeA1w9O2S7aOz51GWuvH+3NKbdu35un741t4MDaf32PHz9e87HfVjyzEwXBsBMFwbATBcGwEwXBsBMFwbATBcGwEwXBPnsDpM5n96TMGfd4vfCUXnfqVtTe8dZ7AM6cOWMe6+mX89lFZIKIbBCRj0Rkp4j8LLt8tIi8JSKd2edRxQ+XiGrVl6fxFwEsUdU7APwzgIdF5A4AjwFYr6qTAazPvieiJuWGXVUPqOqW7OuvAOwCcAuAOQBWZldbCWBuUYMkonTX9AKdiEwEMB3AXwG0q+qBrPQlgPacYxaJSFVEql1dXQlDJaIUfQ67iLQCWA3g56p6smdNu19J6fXVFFVdrqoVVa20tbUlDZaIatensIvIIHQH/Q+q+ufs4oMiMjarjwVwqJghElE9uH0T6e4xvARgl6r+qkdpLYCFAJ7KPr9eyAj7gZQtl/uiyDZQkVs6e+P2pv56x1stz9OnT5vH9kd9aZJ+D8CPAWwXka3ZZY+jO+R/EpGfAPgMwPxihkhE9eCGXVX/AiDvv9Af1Hc4RFQUvl2WKAiGnSgIhp0oCIadKAiGnSgITnHNlDll0esnFym1j57yHoLUKa7e42ZNvy36vQ/NiGd2oiAYdqIgGHaiIBh2oiAYdqIgGHaiIBh2oiDYZ8+kLltsaWlpMeupyxpbvC2bi9wuui/3b0ntw1tjT+2z98ulpImof2DYiYJg2ImCYNiJgmDYiYJg2ImCYNiJgmCfvQmk9rqtfrN326l1r4+eMl8+dV15C+ezE1G/xbATBcGwEwXBsBMFwbATBcGwEwXBsBMF0Zf92ScA+D2AdgAKYLmq/kZEngTwEICu7KqPq+qbRQ20aEXOTx43bpxZ7+zsNOvW+ueA3ev2+uDnz5+v+bYB/3Gz6t7PdeHCBbOeIuJ89r68qeYigCWqukVEhgPYLCJvZbVfq+qzxQ2PiOqlL/uzHwBwIPv6KxHZBeCWogdGRPV1TX+zi8hEANMB/DW7aLGIfCgiK0RkVM4xi0SkKiLVrq6u3q5CRA3Q57CLSCuA1QB+rqonASwDMAnANHSf+Zf2dpyqLlfViqpW2tra6jBkIqpFn8IuIoPQHfQ/qOqfAUBVD6rqJVW9DOC3ADqKGyYRpXLDLt0vO74EYJeq/qrH5WN7XG0egB31Hx4R1UtfXo3/HoAfA9guIluzyx4HsEBEpqG7HbcXwE8LGWE/cPz4cbN+6tQps+61oI4cOZJb81pM3jTRIttfXuvNG/v48ePNurVE9yeffGIe6ylyCe2i9OXV+L8A6K2p+K3tqRNF1Hz//RBRIRh2oiAYdqIgGHaiIBh2oiAYdqIguJR0psgtm2fMmGHWp06datZHjhxp1lN64V6/uLW11aynbKucMnUXAAYNGmTWrfc3dHSkveGzGfvonm/fiImoJgw7URAMO1EQDDtREAw7URAMO1EQDDtREJKype4135lIF4DPelw0BsDhhg3g2jTr2Jp1XADHVqt6ju2fVLXX9d8aGvZv3LlIVVUrpQ3A0Kxja9ZxARxbrRo1Nj6NJwqCYScKouywLy/5/i3NOrZmHRfAsdWqIWMr9W92Imqcss/sRNQgDDtREKWEXUTuE5H/E5E9IvJYGWPIIyJ7RWS7iGwVkWrJY1khIodEZEePy0aLyFsi0pl97nWPvZLG9qSI7Mseu60iMruksU0QkQ0i8pGI7BSRn2WXl/rYGeNqyOPW8L/ZRWQAgI8B/CuALwC8D2CBqn7U0IHkEJG9ACqqWvobMETk+wBOAfi9qt6ZXfYMgKOq+lT2H+UoVf1Fk4ztSQCnyt7GO9utaGzPbcYBzAXwbyjxsTPGNR8NeNzKOLN3ANijqp+q6nkAfwQwp4RxND1V3QTg6FUXzwGwMvt6Jbp/WRouZ2xNQVUPqOqW7OuvAFzZZrzUx84YV0OUEfZbAPytx/dfoLn2e1cA60Rks4gsKnswvWhX1QPZ118CaC9zML1wt/FupKu2GW+ax66W7c9T8QW6b5qpqjMA3A/g4ezpalPS7r/Bmql32qdtvBull23G/67Mx67W7c9TlRH2fQAm9Ph+fHZZU1DVfdnnQwDWoPm2oj54ZQfd7POhksfzd820jXdv24yjCR67Mrc/LyPs7wOYLCLfEZEWAD8CsLaEcXyDiAzLXjiBiAwD8EM031bUawEszL5eCOD1EsfyD5plG++8bcZR8mNX+vbnqtrwDwCz0f2K/CcAnihjDDnjug3AtuxjZ9ljA/Aqup/WXUD3axs/AXAjgPUAOgH8L4DRTTS2/wKwHcCH6A7W2JLGNhPdT9E/BLA1+5hd9mNnjKshjxvfLksUBF+gIwqCYScKgmEnCoJhJwqCYScKgmEnCoJhJwri/wFw4AcBUhlL0QAAAABJRU5ErkJggg==\n",
1180 |             "text/plain": [
1181 |               "<Figure size 432x288 with 1 Axes>"
1182 |             ]
1183 |           },
1184 |           "metadata": {
1185 |             "tags": [],
1186 |             "needs_background": "light"
1187 |           }
1188 |         },
1189 |         {
1190 |           "output_type": "stream",
1191 |           "text": [
1192 |             "0 16.117859\n",
1193 |             "20 16.117216\n",
1194 |             "40 16.099031\n",
1195 |             "60 12.984386\n",
1196 |             "80 9.905032\n",
1197 |             "100 6.4673576\n",
1198 |             "120 3.6043863\n",
1199 |             "140 1.7249596\n",
1200 |             "160 0.06058836\n",
1201 |             "180 -1.2979188\n",
1202 |             "200 -2.7338905\n",
1203 |             "220 -4.0762415\n",
1204 |             "240 -6.040269\n",
1205 |             "260 -8.053326\n",
1206 |             "280 -10.19889\n"
1207 |           ],
1208 |           "name": "stdout"
1209 |         },
1210 |         {
1211 |           "output_type": "display_data",
1212 |           "data": {
1213 |             "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAV2klEQVR4nO3da2yVZbYH8P+iCJTWFtpCW4pyE1AEuaReIjAyjmciGqPzBcfohKPmMB80mUnmg8bzYYyfzMmZGflwnKRzNMOczAFHZ4iSVBlPoxKjjpT7Re5SaOlFKC03obd1PvTFVOy7Vt3vvp3z/H8JadmrT/ez393Vd3evdz2PqCqI6P+/UbmeABFlB5OdKBBMdqJAMNmJAsFkJwrE6GzeWWlpqVZVVcXGk1QG+vv7zbiImPFRo+zfewMDA997TiO9b4/32C5evBgb8x7X6NH2j4D3nHjf35q7d0wLCgrM+JgxY8y4xXtOkj5n3nHp6emJjRUWFqZ8v62trejq6hp28omSXUTuB7AWQAGA/1TVl62vr6qqwquvvhobT5LsnZ2dZnzs2LFmfNy4cWb866+/jo158/YSyvvB6urqMuOff/55bKykpMQcW15ebsZ7e3vNuHfczp07Fxu7cOGCObasrMyMT5kyxYxbvyy8XyTec+bFvV9EJ0+ejI0tXLjQHGtZvXp1bCzll/EiUgDgPwCsBDAPwGMiMi/V70dEmZXkb/Y7ABxR1WOq2gNgA4CH0zMtIkq3JMleA2Doa5Hm6LZvEZE1ItIoIo3ey1EiypyMvxuvqnWqWquqtRMmTMj03RFRjCTJ3gLghiH/nxrdRkR5KEmybwUwW0RmiMgYAD8F8E56pkVE6ZZy6U1V+0TkWQCbMVh6e11V9yWZzPnz5824VaIqLS01x16+fNmMW6U1wC61HD582By7Y8cOM+6VDb0/f6xa9oEDB8yxHqseDPjHzSrNXX/99eZYL/7hhx+mPN4rGd52221mvKbmO29PfYt3DYH32DIhUZ1dVesB1KdpLkSUQbxcligQTHaiQDDZiQLBZCcKBJOdKBBMdqJAZLWf/cqVK2hqaoqNe62iVptqRUWFObaoqMiMnzhxwox//PHHsbFTp06ZY9vb2834jBkzzHhxcbEZt/rZvVZOayzgHzevdfj06dOxMa/W7V1/4M3NurbCOy6bN29O+XsDwFNPPWXGk/Tip4pndqJAMNmJAsFkJwoEk50oEEx2okAw2YkCkdXSW3l5OZ544onYuLUSKQB89tlnsTGvhOSt4LplyxYzfunSpdiYV0LyylPe4/aWJbZWkPVWcD179qwZnzp1qhn3Hru1eu3Ro0fNsZMmTTLjXhup1Zbszdtr7fXamr12be+xZQLP7ESBYLITBYLJThQIJjtRIJjsRIFgshMFgslOFIis1tn7+/vN+mNfX585ftq0abExr27a0NBgxr2tqTo6OmJjlZWV5lirzROA2fYLAHPnzjXjViuoV2f37turw3vtty0t8fuGVFdXm2OtnU4B/zmfP39+bMxrcfV+HrwdZDdt2mTGn3vuOTOeCTyzEwWCyU4UCCY7USCY7ESBYLITBYLJThQIJjtRILJaZ+/t7TXrrl7ft1Xz9XqbvZ5xr95sbZtcWFhojvW2NV64cKEZnzNnjhm3ePXgtrY2M+7Vwr3jai2Z7G2z7c3t4MGDZry+Pn6D4Yceesgce+bMGTM+ffp0M378+HEz3traGhvL1HbOiZJdRI4DOA+gH0CfqtamY1JElH7pOLP/UFXtS8SIKOf4NztRIJImuwL4u4hsE5E1w32BiKwRkUYRafSusyaizEma7MtUdQmAlQCeEZEfXPsFqlqnqrWqWjtx4sSEd0dEqUqU7KraEn3sALARwB3pmBQRpV/KyS4iRSJy/dXPAfwYwN50TYyI0ivJu/GVADZG67GPBvDfqvqeNaC3t9fc3tirs1s1Y28L3ebmZjM+b948M26tQd7b22uO9bbnLSsrM+OLFi0y4/v374+NebXo5cuXm/GtW7ea8VWrVplxa214r9d+8uTJZnzFihVm3Lp2Yu9e+7zk7TPgbcPt/bxZa/1nSsrJrqrHANhXgxBR3mDpjSgQTHaiQDDZiQLBZCcKBJOdKBBZX0q6u7s7Nl5RUWGOt9oGvS1yrZZCwG8rVNWU73vWrFlmfNu2bWb83nvvNePvvvtubKympsYce+TIETN+7NgxM15XV2fGrRKVt1S019o7c+ZMM261JXvLlnvLVK9cudKMb9682Yzv2rUrNlZVVWWOTRXP7ESBYLITBYLJThQIJjtRIJjsRIFgshMFgslOFIis1tnHjBljbrvs1aNHjYr/3WS1UgJAUVGRPTmH1cbqtWp62wN7ddUFCxaY8Q0bNsTGvK2qvVbOxx9/3Ix7c7/11ltjY15r8Jdffpnovq06/IkTJ8yx3hLcBw4cMON33323Gbeu28gUntmJAsFkJwoEk50oEEx2okAw2YkCwWQnCgSTnSgQWa2zjxo1CuPHj4+NX7p0yRxfXFwcG9uyZYs51tta2Ns+2KqLelsyX7lyxYx79eIdO3aY8alTp8bGTp+299zs6Ogw496Sx95S1dZjt5YVB2CufQD4dXrrObeWBgcG116weHV2j5UHmcIzO1EgmOxEgWCyEwWCyU4UCCY7USCY7ESBYLITBSKrdXZVNWujXo+x1e/urY/u1XS9bZWtnnWvJlteXm7GDx8+bMbXrl1rxq01ArzrCz744AMz7q2nb/WrA0B9fX1szLsGYOLEiWbc26bbun7h7Nmz5ljvuFlr0gNAV1eXGfd+ZjLBPbOLyOsi0iEie4fcViYi74vI4eij/awQUc6N5GX8HwHcf81tzwNoUNXZABqi/xNRHnOTXVW3AOi85uaHAayLPl8H4JE0z4uI0izVN+gqVfXq5mltACrjvlBE1ohIo4g0en8nEVHmJH43Xgc7RGK7RFS1TlVrVbXWe8OFiDIn1WRvF5FqAIg+2q1TRJRzqSb7OwBWR5+vBvB2eqZDRJni1tlFZD2AFQAqRKQZwK8BvAzgLyLyNIAmAKtGcmcFBQVm7dR7mW/tqe31lL/00ktm3KsXv/nmm7Gx9957zxzb3Nxsxr1rBKz18gF7jXNrDQAAmD17dqL79nr1rTXzFy9ebI71at1Jjttdd91ljr3vvvvM+JIlS1K+b8BeH2Hfvn3m2FS5ya6qj8WEfpTmuRBRBvFyWaJAMNmJAsFkJwoEk50oEEx2okBktcV1YGAAFy9ejI177ZQDAwOxMa9F1btUd//+/WZ87ty5sTFvq+lXXnnFjFstqoDfDmnFrWMGAGVlZWa8p6fHjHtbD1uluUOHDplj58yZk+i+rTbTBx980Bx75513Jrpv77hv3LgxNuY97lTxzE4UCCY7USCY7ESBYLITBYLJThQIJjtRIJjsRIHI+pbN48aNi417WzZ/8sknsTFvC9yxY8eacREx41Y92hvr8bZN9h6bdUw9FRUVZry9vd2Me9smW9tJe+23Xourd22E1fbstfZ6Wzp7LbBe668XzwSe2YkCwWQnCgSTnSgQTHaiQDDZiQLBZCcKBJOdKBBZrbP39fWZ2/RaS0UDdl32nnvuMcfu3r3bjHv9yVbc2s4ZABYsWGDGvWWwvZ5yq5/de1xeDd+rwyfZVtlaZhrwa/glJSVm3Pr+y5YtM8d6Nf633nrLjHt1dGtdh0zhmZ0oEEx2okAw2YkCwWQnCgSTnSgQTHaiQDDZiQKR1Tp7f3+/Wb+cOXOmOd6qs58/f94c69U1vXXnrXqytWUykLyO7vXie+MtXq3bqpMD/nr8Fu+6Cq/O7t23dVy859s7pt71BS0tLWb80UcfjY01NTWZY1PlntlF5HUR6RCRvUNue1FEWkRkZ/TvgYzMjojSZiQv4/8I4P5hbv+dqi6K/tWnd1pElG5usqvqFgCdWZgLEWVQkjfonhWR3dHL/Ng/YERkjYg0ikhjd3d3grsjoiRSTfbfA5gFYBGAVgC/iftCVa1T1VpVrS0tLU3x7ogoqZSSXVXbVbVfVQcA/AHAHemdFhGlW0rJLiLVQ/77EwB7476WiPKDW2cXkfUAVgCoEJFmAL8GsEJEFgFQAMcB/Hwkd9bX14czZ87Exr3aZFFRUWzM2j8d8Pu6vTXMrbl5+8rv27fPjN94441m3Jt7YWFhbMxbi9875l6/u1eHt/rhOzvt932ttQ8Afw90S5I97wH/OZ80aZIZt9ZXyNSfu26yq+pjw9z8WgbmQkQZxMtliQLBZCcKBJOdKBBMdqJAMNmJApHVFteCggKzNdDb+tgqIx08eNAc67UsenFrbpWVleZYr73Wa4H1ykC7du1K+b690ppX/qqtrTXj1113XWzMe769x+0t92y1wHpbMlvzBuxyJ+A/p1b7bqZKbzyzEwWCyU4UCCY7USCY7ESBYLITBYLJThQIJjtRILJaZx8/fjyWLFkSG/fqrps2bYqNLV261By7Z88eM+61JFo13ebmZnOsV7Nta2sz4147pTU3qy0YABYvXmzGP/roIzPutchWVVXFxiZPnmyO9R63d41AfX38Oqhvv/22OXb58uVmfNQo+zzptd96S1lnAs/sRIFgshMFgslOFAgmO1EgmOxEgWCyEwWCyU4UiKzW2UePHm32s3u1R6su29raao71llS2lrgG7P7jsrIyc6zXl+1tN+1tTWxt6XzhwgVzrHdcpk2bZsatOjoAdHV1xcaOHj1qjvXm5tW6b7/99tiYtzy3t0S2d22F1y9vXd/w1VdfmWNTxTM7USCY7ESBYLITBYLJThQIJjtRIJjsRIFgshMFIqt19suXL+PQoUOx8enTp5vjrTXQq6urY2OAX8O36uiAvaWzt0a4d99J1yC3tkX2rj/w7ts7LlaNHwAOHDgQG5syZYo51luT3lv/oKOjIzY2f/58c6z3uPr6+sy4d1ytawxKSkrMsalyz+wicoOIfCAi+0Vkn4j8Irq9TETeF5HD0cf4q2WIKOdG8jK+D8CvVHUegLsAPCMi8wA8D6BBVWcDaIj+T0R5yk12VW1V1e3R5+cBfAGgBsDDANZFX7YOwCOZmiQRJfe93qATkekAFgP4B4BKVb36B2EbgGE3PBORNSLSKCKN3jXeRJQ5I052ESkG8FcAv1TVb61wqINdBcN2FqhqnarWqmqt1QRDRJk1omQXkeswmOh/VtW/RTe3i0h1FK8GEP/WJxHlnFt6k8H6xmsAvlDV3w4JvQNgNYCXo4/22rwAxo0bh5tvvjk23tDQYI6/cuVKbMxrE/WWJba+N2C3PHqtmN6Sx14LrFVaA4AjR46kPNZb8tg7Ll75y2pLnjlzpjnWa2H1nlOvZGnxWlS9+/aW8C4vL4+NdXd3m2NTNZI6+1IAPwOwR0R2Rre9gMEk/4uIPA2gCcCqjMyQiNLCTXZV/RhA3K/vH6V3OkSUKbxcligQTHaiQDDZiQLBZCcKBJOdKBBZbXFVVfT09MTGFyxYYI636q7WksXAYI3fUlk57NW+37Dq+N5S0rfccosZ964v8Gq6Vp1/3rx55lhvO2lvWWOvfde6atJbztm7fsG7RsBqU/Vad706uTf3JMc1U9s588xOFAgmO1EgmOxEgWCyEwWCyU4UCCY7USCY7ESByGqdXUQS1RCtWvq5c+diY4Bfu2xrazPjVt10woQJ5livHuxtq7xy5Uozvn79+tiYd31B0hp/Z2enGbdq3d4aBN5z5vXaWz3p3hoC3nHzloouKCgw497PRCbwzE4UCCY7USCY7ESBYLITBYLJThQIJjtRIJjsRIHIap19YGDA7FH26qbWNrk33XSTOdbbujjJ+ucnT540x3r1Yq/m622bZc3d+95eX7e39vrevXvNuFWnLy0tNcd6c/eOq7X+gVdH97YP9/rZT506Zca9fvdM4JmdKBBMdqJAMNmJAsFkJwoEk50oEEx2okAw2YkCMZL92W8A8CcAlQAUQJ2qrhWRFwH8C4CrC2C/oKr11vc6e/Ys3njjjdj4k08+ac7FqrN7dcuqqiozbq1vDgC7du2KjS1cuNAc29TUZMa93uck9WRvn3Ev7vXqe7341mPzavzetQ9erdvqOfeu6bD68AHg9OnTZtz7/iUlJbEx7/qCVI3kopo+AL9S1e0icj2AbSLyfhT7nar+e0ZmRkRpNZL92VsBtEafnxeRLwDUZHpiRJRe3+tvdhGZDmAxgH9ENz0rIrtF5HURGfZ1sIisEZFGEWn0liEioswZcbKLSDGAvwL4paqeA/B7ALMALMLgmf83w41T1TpVrVXVWm89MyLKnBElu4hch8FE/7Oq/g0AVLVdVftVdQDAHwDckblpElFSbrLL4FuirwH4QlV/O+T26iFf9hMAdvsTEeXUSN6NXwrgZwD2iMjO6LYXADwmIoswWI47DuDn3jcqLCw0y1Tbt28fwXQyo7u724xb7ZjessB79uwx49OmTTPjBw8eNONW++2nn35qji0uLjbjVrkT8FtFrfdpLl68aI71jmt5ebkZP3PmTGzMK9WeOHHCjHtz87Z8zlR5zTKSd+M/BjBcwdOsqRNRfuEVdESBYLITBYLJThQIJjtRIJjsRIFgshMFIqtLSeczr13Sa7e0eHX0mhq7r2j8+PFm3KuFJ+G1eiZpQ/Vae73v7Y23li2fNWuWOdZjtRXnq/97MyailDDZiQLBZCcKBJOdKBBMdqJAMNmJAsFkJwqEePXltN6ZyFcAhq6rXAHAXpM3d/J1bvk6L4BzS1U65zZNVScNF8hqsn/nzkUaVbU2ZxMw5Ovc8nVeAOeWqmzNjS/jiQLBZCcKRK6TvS7H92/J17nl67wAzi1VWZlbTv9mJ6LsyfWZnYiyhMlOFIicJLuI3C8iB0XkiIg8n4s5xBGR4yKyR0R2ikhjjufyuoh0iMjeIbeVicj7InI4+mjvNZ3dub0oIi3RsdspIg/kaG43iMgHIrJfRPaJyC+i23N67Ix5ZeW4Zf1vdhEpAHAIwD8BaAawFcBjqro/qxOJISLHAdSqas4vwBCRHwC4AOBPqjo/uu3fAHSq6svRL8qJqvpcnsztRQAXcr2Nd7RbUfXQbcYBPALgn5HDY2fMaxWycNxycWa/A8ARVT2mqj0ANgB4OAfzyHuqugVA5zU3PwxgXfT5Ogz+sGRdzNzygqq2qur26PPzAK5uM57TY2fMKytykew1AE4O+X8z8mu/dwXwdxHZJiJrcj2ZYVSqamv0eRuAylxOZhjuNt7ZdM0243lz7FLZ/jwpvkH3XctUdQmAlQCeiV6u5iUd/Bssn2qnI9rGO1uG2Wb8G7k8dqluf55ULpK9BcANQ/4/NbotL6hqS/SxA8BG5N9W1O1Xd9CNPnbkeD7fyKdtvIfbZhx5cOxyuf15LpJ9K4DZIjJDRMYA+CmAd3Iwj+8QkaLojROISBGAHyP/tqJ+B8Dq6PPVAN7O4Vy+JV+28Y7bZhw5PnY53/5cVbP+D8ADGHxH/iiAf83FHGLmNRPArujfvlzPDcB6DL6s68XgextPAygH0ADgMID/AVCWR3P7LwB7AOzGYGJV52huyzD4En03gJ3RvwdyfeyMeWXluPFyWaJA8A06okAw2YkCwWQnCgSTnSgQTHaiQDDZiQLBZCcKxP8C9BDOhdjd9n4AAAAASUVORK5CYII=\n",
1214 |             "text/plain": [
1215 |               "<Figure size 432x288 with 1 Axes>"
1216 |             ]
1217 |           },
1218 |           "metadata": {
1219 |             "tags": [],
1220 |             "needs_background": "light"
1221 |           }
1222 |         },
1223 |         {
1224 |           "output_type": "stream",
1225 |           "text": [
1226 |             "Predicted: Sandal\n"
1227 |           ],
1228 |           "name": "stdout"
1229 |         }
1230 |       ]
1231 |     },
1232 |     {
1233 |       "cell_type": "markdown",
1234 |       "metadata": {
1235 |         "id": "j13P4BcF9zo1",
1236 |         "colab_type": "text"
1237 |       },
1238 |       "source": [
1239 |         "## Seeing if the adversarial examples are misclassified by the Discriminator of the GAN"
1240 |       ]
1241 |     },
1242 |     {
1243 |       "cell_type": "code",
1244 |       "metadata": {
1245 |         "id": "6UBKDYOnj_-p",
1246 |         "colab_type": "code",
1247 |         "colab": {
1248 |           "base_uri": "https://localhost:8080/",
1249 |           "height": 34
1250 |         },
1251 |         "outputId": "c8a16505-a3b2-4cde-d532-7babf638f397"
1252 |       },
1253 |       "source": [
1254 |         "# 0 -> Real image 1-> Fake image\n",
1255 |         "disc.predict_classes(perturbed_image)"
1256 |       ],
1257 |       "execution_count": 85,
1258 |       "outputs": [
1259 |         {
1260 |           "output_type": "execute_result",
1261 |           "data": {
1262 |             "text/plain": [
1263 |               "array([[1]], dtype=int32)"
1264 |             ]
1265 |           },
1266 |           "metadata": {
1267 |             "tags": []
1268 |           },
1269 |           "execution_count": 85
1270 |         }
1271 |       ]
1272 |     },
1273 |     {
1274 |       "cell_type": "markdown",
1275 |       "metadata": {
1276 |         "id": "-fdOzykGoOaW",
1277 |         "colab_type": "text"
1278 |       },
1279 |       "source": [
1280 |         "The generator is able to predict the perturbed image as a fake one. **But this does not draw any conclusion since we used a relatively simple and small dataset.**"
1281 |       ]
1282 |     }
1283 |   ]
1284 | }


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # Adversarial Examples in Deep Learning
 2 | 
 3 | Deep Learning has brought us tremendous achievements in the field of Computer Vision. In spite of the impeccable success, modern Deep Learning systems are still prone to adversaries. Let's talk in terms of Computer Vision. Consider an image of a polar bear and an instance of it (X1). A Deep Learning-based image classifier is able to successfully X1 as a polar bear. Now consider another instance of a polar bear X2 which is a slightly perturbed version of X1. To the human eyes, it would still be a polar bear but for that same image classifier, it would be an ant. These perturbations are referred to as image adversaries.
 4 | 
 5 | <div align="center"><img src="https://i.ibb.co/Wkg8CSX/download.png"></img></div>
 6 | 
 7 | This repository contains code for a short crash-course related adversarial examples in deep learning. The crash course would include introduction to adversarial examples, training models that are adversarial-aware, situations where adversarial-aware models could fail, and so on.   
 8 | 
 9 | The crash course would be presented in form of [Weights and Biases reports](https://docs.wandb.com/reports). The first report in this line is now up - 
10 | - [An Introduction to Adversarial Examples in Deep Learning](https://app.wandb.ai/authors/adv-dl/reports/An-Introduction-to-Adversarial-Examples-in-Deep-Learning--VmlldzoyMTQwODM)
11 | 
12 | ## Contents (to be updated):
13 | - `Image_Adversaries_Basics.ipynb`: Shows how to create adversaries that can fool a ResNet50 model pre-trained on ImageNet. Includes both vanilla and targeted attacks. 
14 | - `Adversarial_Training_NSL.ipynb`: Shows how to train adversarially robust image classifiers using [`Neural Structured Learning`](https://www.tensorflow.org/neural_structured_learning). 
15 | - `GANs_w_Adversaries.ipynb`: Shows how to incorporate GANs (plain old DCGAN) to tackle adversarial situations. 
16 | - `Optimizer_Susceptibility.ipynb`: Studies the susceptibility of different optimizers against simple attacks.
17 | - `Optimizer_Susceptibility_Targeted_Attacks.ipynb`: Studies the susceptibility of different optimizers against targeted attacks.
18 | 
19 | **Note**: The materials are strictly for learning purpose and should not be considered for production systems.
20 | 
21 | ## Coded in:
22 | - TensorFlow 2.x (at time of writing Google Colab had TensorFlow `2.3.0`)
23 | 
24 | ## References:
25 | - [J. Z. Kolter and A. Madry: Adversarial Robustness - Theory and Practice (NeurIPS 2018 Tutorial)](https://adversarial-ml-tutorial.org/)
26 | - Chapter 10 (Adversarial examples), [GANs in Action](https://www.manning.com/books/gans-in-action)
27 | - [Introduction to Adversarial Machine Learning](https://blog.floydhub.com/introduction-to-adversarial-machine-learning/)
28 | - [Adversarial example using FGSM](https://www.tensorflow.org/tutorials/generative/adversarial_fgsm)
29 | - [Adversarial regularization for image classification](https://www.tensorflow.org/neural_structured_learning/tutorials/adversarial_keras_cnn_mnist)
30 | 


--------------------------------------------------------------------------------