├── Readme.md ├── images ├── accuracy.png ├── keras-tqdm.png ├── loss.png └── model.png ├── models └── model4.h5 └── understanding-deep-convolutional-neural-networks-with-a-practical-use-case-in-tensorflow-and-keras.ipynb /Readme.md: -------------------------------------------------------------------------------- 1 | ## Understanding deep Convolutional Neural Networks with a practical use-case in Tensorflow and Keras 2 | 3 | 4 | 5 | 6 | This jupyter notebook reassembles the code of this 7 | article 8 | 9 | It also contains a trained CNN model, so that you can use it yourself and test it. 10 | 11 | 12 | ### Summary of this article 13 | 14 | This post is a 4-part tutorial where I: 15 | 16 | 1. Present an image dataset from the Cat vs. Dog Kaggle competition and explain the complexity of the image classification task 17 | 2. Go over the details about Convolutional Neural Nets, explaining their inner meachanisms and the reason why they perform better than fully connected networks. 18 | 3. Set up a deep learning dedicated environment on a powerful GPU-based EC2 instance from Amazon Web Services (AWS) 19 | 4. Train two deep learning models: one from scratch in an end-to-end pipeline using Keras and Tensorflow, and another one by using a pre-trained network on a large dataset. 20 | 21 | These 4 parts are independent. 22 | 23 | If you're looking to understand the theory behind convnets please refer to the article link posted above. 24 | 25 | ### Environment setup: 26 | 27 | 1. Use Python 3.6: No hassle, intall the Conda distribution that encapsulates the PyData stack (SciPy, Pandas, Matplotlib, etc.). Here's the installation link 28 | 2. Install the lastest version of Tensorflow: https://www.tensorflow.org/install/install_windows 29 | I used a windows machine, same applies for Linux or Mac OS X 30 | 31 | 3. Install the following python dependencies: 32 | 33 | ```bash 34 | pip install keras 35 | pip install tqdm 36 | pip install keras-tqdm 37 | conda install -c conda-forge opencv 38 | ``` 39 | 40 | 4. [Optional] Dependencies to obtain GraphViz plots of the CNN architectures: 41 | 42 | 1. Install Graphiz: http://www.graphviz.org/Download..php 43 | 2. Add the Graphiz binaries to you PATH 44 | 3. Install Graphiz Python bindings 45 | ```bash 46 | pip install graphviz 47 | pip install pydot 48 | ``` 49 | 50 |

51 | 53 | -------------------------------------------------------------------------------- /images/accuracy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahmedbesbes/Understanding-deep-Convolutional-Neural-Networks-with-a-practical-use-case-in-Tensorflow-and-Keras/1a36392a502e27c15fd5e8da928e26b96af2e725/images/accuracy.png -------------------------------------------------------------------------------- /images/keras-tqdm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahmedbesbes/Understanding-deep-Convolutional-Neural-Networks-with-a-practical-use-case-in-Tensorflow-and-Keras/1a36392a502e27c15fd5e8da928e26b96af2e725/images/keras-tqdm.png -------------------------------------------------------------------------------- /images/loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahmedbesbes/Understanding-deep-Convolutional-Neural-Networks-with-a-practical-use-case-in-Tensorflow-and-Keras/1a36392a502e27c15fd5e8da928e26b96af2e725/images/loss.png -------------------------------------------------------------------------------- /images/model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahmedbesbes/Understanding-deep-Convolutional-Neural-Networks-with-a-practical-use-case-in-Tensorflow-and-Keras/1a36392a502e27c15fd5e8da928e26b96af2e725/images/model.png -------------------------------------------------------------------------------- /models/model4.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahmedbesbes/Understanding-deep-Convolutional-Neural-Networks-with-a-practical-use-case-in-Tensorflow-and-Keras/1a36392a502e27c15fd5e8da928e26b96af2e725/models/model4.h5 -------------------------------------------------------------------------------- /understanding-deep-convolutional-neural-networks-with-a-practical-use-case-in-tensorflow-and-keras.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "This notebook contains the code of the article: Understanding deep Convolutional Neural Networks with a practical use-case in Tensorflow and Keras as well as portions of the post's text. This will help understanding the code.\n", 8 | "\n", 9 | "Link here " 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "This is what the convolution layer does: given a filter, it scans the input and generates a feature map.\n", 17 | "\n", 18 | "> ** But what does this convolution operation really represent? How to interpret the resulting feature map? **\n", 19 | "\n", 20 | "I started by stating that convolution layers capture visual patterns within an image. Let me illustrate this to convince you.\n", 21 | "\n", 22 | "I'm going to load a cat image from the dataset. I'll then apply different convolutions on it, changing the kernel each time, and visualizing the results." 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": { 29 | "collapsed": true 30 | }, 31 | "outputs": [], 32 | "source": [ 33 | "%matplotlib inline\n", 34 | "from scipy.signal import convolve2d\n", 35 | "import numpy as np \n", 36 | "import cv2\n", 37 | "from matplotlib import pyplot as plt\n", 38 | "\n", 39 | "image = cv2.imread('./data/train/cats/cat.46.jpg')\n", 40 | "# converting the image to grayscale\n", 41 | "image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "I'll define a function that takes a kenel as an input, performs a convolution on the image and then plots the original image and the convolved one next to it.\n", 49 | "\n", 50 | "The kernel is a small square matrix (gray square in the figures above)." 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": { 57 | "collapsed": true 58 | }, 59 | "outputs": [], 60 | "source": [ 61 | "def show_differences(kernel):\n", 62 | " convolved = convolve2d(image, kernel)\n", 63 | " fig = plt.figure(figsize=(15, 15))\n", 64 | " plt.subplot(121)\n", 65 | " plt.title('Original image')\n", 66 | " plt.axis('off')\n", 67 | " plt.imshow(image, cmap='gray')\n", 68 | " \n", 69 | " plt.subplot(122)\n", 70 | " plt.title('Convolved image')\n", 71 | " plt.axis('off')\n", 72 | " plt.imshow(convolved, cmap='gray')\n", 73 | " return convolved" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "Let's start by this filter:\n", 81 | "$$ \\frac{1}{9} \\left[ \n", 82 | "\\begin{array}{cccc}\n", 83 | "1 & 1 & 1 \\\\\n", 84 | "1 & 1 & 1 \\\\\n", 85 | "1 & 1 & 1 \\\\ \\end{array} \\right]$$\n", 86 | "\n", 87 | "It's called box blur. When this filter is applied to a pixel value in the input image, it basically takes this pixel and its 8 neighbors (that's why we have 1 everywhere) and compute their average pixel value (that's why we devide by 9).\n", 88 | "\n", 89 | "Mathematically, this is a simple average. Visually, it results in smoothing abrupt contrast transitions that appear in the image.\n", 90 | "\n", 91 | "Box blur is highly used in **noise removal**.\n", 92 | "\n", 93 | "Let's apply it on a cat image and see what it does." 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": { 100 | "collapsed": true 101 | }, 102 | "outputs": [], 103 | "source": [ 104 | "kernel = np.array([[1, 1, 1], [1, 1, 1], [1, 1, 1]])/9\n", 105 | "output = show_differences(kernel)" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "If you closely look at the convolved image, you'll notice that it's smoother, with less white pixels (noise) sprinkled on it." 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "Now let's look at a more agressive bluring filter:\n", 127 | "$$\\frac{1}{64}\\left[ \n", 128 | "\\begin{array}{cccc}\n", 129 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 130 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 131 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 132 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 133 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 134 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 135 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n", 136 | "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\ \\end{array} \\right]$$" 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": { 143 | "collapsed": true 144 | }, 145 | "outputs": [], 146 | "source": [ 147 | "kernel = np.ones((8,8), np.float32)/64\n", 148 | "dx = show_differences(kernel)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": {}, 154 | "source": [ 155 | "" 156 | ] 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "metadata": {}, 161 | "source": [ 162 | "Some filters are used to capture intrinsic image details like **edges**.\n", 163 | "\n", 164 | "Here is one example that computes an approximation of the vertical changes in the image A.\n", 165 | "\n", 166 | "$$G_x= A * \\left[ \n", 167 | "\\begin{array}{cccc}\n", 168 | "-1 & 0 & 1\\\\\n", 169 | "-2 & 0 & 2 \\\\\n", 170 | "-1 & 0 & 1 \\\\ \\end{array} \\right]$$" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": null, 176 | "metadata": { 177 | "collapsed": true 178 | }, 179 | "outputs": [], 180 | "source": [ 181 | "kernel = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], np.float32)\n", 182 | "dx = show_differences(kernel)" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "The white areas are the ones that better respond to the filter, indicating the presence of a vertical edge. Look closely at the cat's left ear for example and notice how its edge is captured. " 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "Cool, right? Here is a second filter that does the same operation but for the horizontal changes.\n", 204 | "\n", 205 | "\n", 206 | "$$G_y = A * \\left[ \n", 207 | "\\begin{array}{cccc}\n", 208 | "1 & 2 & 1\\\\\n", 209 | "0 & 0 & 0 \\\\\n", 210 | "-1 & -2 & -1 \\\\ \\end{array} \\right]$$" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": null, 216 | "metadata": { 217 | "collapsed": true 218 | }, 219 | "outputs": [], 220 | "source": [ 221 | "kernel = np.array([[1, 2, 1], [0, 0, 0], [-1, -2, -1]], np.float32)\n", 222 | "dy = show_differences(kernel)" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": {}, 228 | "source": [ 229 | "" 230 | ] 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "metadata": {}, 235 | "source": [ 236 | "Notice how the whiskers are detected." 237 | ] 238 | }, 239 | { 240 | "cell_type": "markdown", 241 | "metadata": {}, 242 | "source": [ 243 | "The two previous filters are gradient operators. They allow, to some extent, to reveal an inherent structure in the image with respect to a direction. \n", 244 | "\n", 245 | "However, when G_x and G_y are combined in the following formula:\n", 246 | "\n", 247 | "$$G = \\sqrt{G_x^2 + G_y^2}$$\n", 248 | "\n", 249 | "they allow an even better edge detection." 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": null, 255 | "metadata": { 256 | "collapsed": true 257 | }, 258 | "outputs": [], 259 | "source": [ 260 | "mag = np.hypot(dx, dy) # magnitude\n", 261 | "mag *= 255.0 / np.max(mag) # normalize (Q&D)\n", 262 | "\n", 263 | "fig = plt.figure(figsize=(15, 15))\n", 264 | "plt.subplot(121)\n", 265 | "plt.title('Original image')\n", 266 | "plt.axis('off')\n", 267 | "plt.imshow(image, cmap='gray')\n", 268 | "\n", 269 | "plt.subplot(122)\n", 270 | "plt.title('Convoluted image with highlighted edges')\n", 271 | "plt.axis('off')\n", 272 | "plt.imshow(mag, cmap='gray')" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "## 4 - 1 Builiding a convnet from scratch\n", 287 | "\n", 288 | "In this first section, we'll setup an end-to-end pipeline to train a CNN. We'll go through data preparation and augmentation, architecture design, training and validation. We'll plot the loss and accuracy metrics on both the train and validation set: this will allow us to assess the improvement of the model over the training." 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": {}, 294 | "source": [ 295 | "### Data preparation" 296 | ] 297 | }, 298 | { 299 | "cell_type": "markdown", 300 | "metadata": {}, 301 | "source": [ 302 | "The first thing to do before starting is to download and unzip the train dataset from Kaggle.\n" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 1, 308 | "metadata": { 309 | "collapsed": true, 310 | "scrolled": false 311 | }, 312 | "outputs": [], 313 | "source": [ 314 | "%matplotlib inline\n", 315 | "from matplotlib import pyplot as plt\n", 316 | "from PIL import Image\n", 317 | "import numpy as np\n", 318 | "import os \n", 319 | "import cv2\n", 320 | "from tqdm import tqdm_notebook\n", 321 | "from random import shuffle\n", 322 | "import shutil\n", 323 | "import pandas as pd" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "metadata": { 329 | "collapsed": true 330 | }, 331 | "source": [ 332 | "We must now organize the data so that it's easily processed by keras.\n", 333 | "\n", 334 | "We'll create a data folder in which we'll have in two subfolders:\n", 335 | "\n", 336 | "+ train\n", 337 | "+ validation\n", 338 | "\n", 339 | "Each of them will have 2 folders: \n", 340 | "\n", 341 | "+ cats \n", 342 | "+ dogs\n", 343 | "\n", 344 | "At the end we'll have the following structure:" 345 | ] 346 | }, 347 | { 348 | "cell_type": "markdown", 349 | "metadata": {}, 350 | "source": [ 351 | "

\n", 352 | "```javascript\n", 353 | "data/\n", 354 | " train/\n", 355 | " dogs/\n", 356 | " dog001.jpg\n", 357 | " dog002.jpg\n", 358 | " ...\n", 359 | " cats/\n", 360 | " cat001.jpg\n", 361 | " cat002.jpg\n", 362 | " ...\n", 363 | " validation/\n", 364 | " dogs/\n", 365 | " dog001.jpg\n", 366 | " dog002.jpg\n", 367 | " ...\n", 368 | " cats/\n", 369 | " cat001.jpg\n", 370 | " cat002.jpg\n", 371 | "``` \n", 372 | "
" 373 | ] 374 | }, 375 | { 376 | "cell_type": "markdown", 377 | "metadata": { 378 | "collapsed": true 379 | }, 380 | "source": [ 381 | "This structrue will allow our model to know from which folder to fetch the images as well as their labels for either training or validation. \n", 382 | "\n", 383 | "Here's a function that allows you to construct this file tree. It has two parameters: the total number of images **n** and the ratio of validation set **r**." 384 | ] 385 | }, 386 | { 387 | "cell_type": "code", 388 | "execution_count": 2, 389 | "metadata": { 390 | "collapsed": false, 391 | "scrolled": false 392 | }, 393 | "outputs": [], 394 | "source": [ 395 | "def organize_datasets(path_to_data, n=4000, ratio=0.2):\n", 396 | " files = os.listdir(path_to_data)\n", 397 | " files = [os.path.join(path_to_data, f) for f in files]\n", 398 | " shuffle(files)\n", 399 | " files = files[:n]\n", 400 | " \n", 401 | " n = int(len(files) * ratio)\n", 402 | " val, train = files[:n], files[n:]\n", 403 | " \n", 404 | "\n", 405 | " shutil.rmtree('./data/')\n", 406 | " print('/data/ removed')\n", 407 | "\n", 408 | " for c in ['dogs', 'cats']: \n", 409 | " os.makedirs('./data/train/{0}/'.format(c))\n", 410 | " os.makedirs('./data/validation/{0}/'.format(c))\n", 411 | "\n", 412 | " print('folders created !')\n", 413 | "\n", 414 | " for t in tqdm_notebook(train):\n", 415 | " if 'cat' in t:\n", 416 | " shutil.copy2(t, os.path.join('.', 'data', 'train', 'cats'))\n", 417 | " else:\n", 418 | " shutil.copy2(t, os.path.join('.', 'data', 'train', 'dogs'))\n", 419 | " \n", 420 | " for v in tqdm_notebook(val):\n", 421 | " if 'cat' in v:\n", 422 | " shutil.copy2(v, os.path.join('.', 'data', 'validation', 'cats'))\n", 423 | " else:\n", 424 | " shutil.copy2(v, os.path.join('.', 'data', 'validation', 'dogs'))\n", 425 | " \n", 426 | " print('Data copied!')" 427 | ] 428 | }, 429 | { 430 | "cell_type": "markdown", 431 | "metadata": {}, 432 | "source": [ 433 | "I used:\n", 434 | "\n", 435 | "+ n : 25000 (the entire dataset)\n", 436 | "+ r : 0.2" 437 | ] 438 | }, 439 | { 440 | "cell_type": "code", 441 | "execution_count": 3, 442 | "metadata": { 443 | "collapsed": true, 444 | "scrolled": false 445 | }, 446 | "outputs": [], 447 | "source": [ 448 | "ratio = 0.2\n", 449 | "n = 25000\n", 450 | "organize_datasets(path_to_data='./train/', n=n, ratio=ratio)" 451 | ] 452 | }, 453 | { 454 | "cell_type": "markdown", 455 | "metadata": {}, 456 | "source": [ 457 | "Let's load Keras and its dependencies." 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": 4, 463 | "metadata": { 464 | "collapsed": false, 465 | "scrolled": false 466 | }, 467 | "outputs": [ 468 | { 469 | "name": "stderr", 470 | "output_type": "stream", 471 | "text": [ 472 | "Using TensorFlow backend.\n" 473 | ] 474 | } 475 | ], 476 | "source": [ 477 | "import keras\n", 478 | "from keras.preprocessing.image import ImageDataGenerator\n", 479 | "from keras_tqdm import TQDMNotebookCallback\n", 480 | "from keras.models import Sequential\n", 481 | "from keras.layers import Dense\n", 482 | "from keras.layers import Dropout\n", 483 | "from keras.layers import Flatten\n", 484 | "from keras.constraints import maxnorm\n", 485 | "from keras.optimizers import SGD\n", 486 | "from keras.layers.convolutional import Conv2D\n", 487 | "from keras.layers.convolutional import MaxPooling2D\n", 488 | "from keras.utils import np_utils\n", 489 | "from keras.callbacks import Callback" 490 | ] 491 | }, 492 | { 493 | "cell_type": "markdown", 494 | "metadata": {}, 495 | "source": [ 496 | "### Image generators and data augmentation" 497 | ] 498 | }, 499 | { 500 | "cell_type": "markdown", 501 | "metadata": {}, 502 | "source": [ 503 | "When training a model, we're not going to load the entire dataset in memory. This won't be efficient, especially if you use your local machine.\n", 504 | "\n", 505 | "We're going to use the ImageDataGenerator class. It allows you to indefinitely stream the images by batches from the train and validation folders. Every batch flows through the network, makes a forward-prop, a back-prop then a parameter updates (along with the test on the validation data). Then comes the next batch and does the same thing, etc.\n", 506 | "\n", 507 | "Inside the ImageDataGenerator object, we're going to introduce random modifications on each batch. It's a process we call **data augmentation**. It allows to generate more data so that our model would never see twice the exact same picture. This helps prevent overfitting and makes the model generalize better. \n", 508 | "\n", 509 | "We'll create two ImageDataGenerator objects.\n", 510 | "\n", 511 | "**train_datagen** for the training set and **val_datagen** one for the validation set. The two of them will apply a rescaling on the image, but the train_datagen will introduce more modifications." 512 | ] 513 | }, 514 | { 515 | "cell_type": "code", 516 | "execution_count": 5, 517 | "metadata": { 518 | "collapsed": true, 519 | "scrolled": false 520 | }, 521 | "outputs": [], 522 | "source": [ 523 | "batch_size = 32" 524 | ] 525 | }, 526 | { 527 | "cell_type": "code", 528 | "execution_count": 6, 529 | "metadata": { 530 | "collapsed": true, 531 | "scrolled": false 532 | }, 533 | "outputs": [], 534 | "source": [ 535 | "train_datagen = ImageDataGenerator(rescale=1/255.,\n", 536 | " shear_range=0.2,\n", 537 | " zoom_range=0.2,\n", 538 | " horizontal_flip=True\n", 539 | " )\n", 540 | "val_datagen = ImageDataGenerator(rescale=1/255.)" 541 | ] 542 | }, 543 | { 544 | "cell_type": "markdown", 545 | "metadata": {}, 546 | "source": [ 547 | "From the two previous objects we're going to create two file generators:\n", 548 | "\n", 549 | "+ train_generator\n", 550 | "+ validation_generator\n", 551 | "\n", 552 | "Each one generates, from its directory, batches of tensor image data with real-time data augmentation. The data will be looped over (in batches) indefinitely." 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": 7, 558 | "metadata": { 559 | "collapsed": false, 560 | "scrolled": false 561 | }, 562 | "outputs": [ 563 | { 564 | "name": "stdout", 565 | "output_type": "stream", 566 | "text": [ 567 | "Found 20000 images belonging to 2 classes.\n", 568 | "Found 5000 images belonging to 2 classes.\n" 569 | ] 570 | } 571 | ], 572 | "source": [ 573 | "train_generator = train_datagen.flow_from_directory(\n", 574 | " './data/train/',\n", 575 | " target_size=(150, 150),\n", 576 | " batch_size=batch_size,\n", 577 | " class_mode='categorical')\n", 578 | "\n", 579 | "validation_generator = val_datagen.flow_from_directory(\n", 580 | " './data/validation/',\n", 581 | " target_size=(150, 150),\n", 582 | " batch_size=batch_size,\n", 583 | " class_mode='categorical')" 584 | ] 585 | }, 586 | { 587 | "cell_type": "markdown", 588 | "metadata": {}, 589 | "source": [ 590 | "### Model architecture" 591 | ] 592 | }, 593 | { 594 | "cell_type": "markdown", 595 | "metadata": {}, 596 | "source": [ 597 | "I'll use a CNN with three convolution/pooling layers and two fully connected layers.\n", 598 | "\n", 599 | "The three conv layers will use respectively 32, 32 and 64 3x3 filters.\n", 600 | "\n", 601 | "I used dropout on the two fully connected layers to prevent overfitting." 602 | ] 603 | }, 604 | { 605 | "cell_type": "code", 606 | "execution_count": 8, 607 | "metadata": { 608 | "collapsed": true, 609 | "scrolled": false 610 | }, 611 | "outputs": [], 612 | "source": [ 613 | "model = Sequential()\n", 614 | "\n", 615 | "model.add(Conv2D(32, (3, 3), input_shape=(150, 150, 3), padding='same', activation='relu'))\n", 616 | "model.add(MaxPooling2D(pool_size=(2, 2)))\n", 617 | "\n", 618 | "model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))\n", 619 | "model.add(MaxPooling2D(pool_size=(2, 2)))\n", 620 | "\n", 621 | "model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\n", 622 | "model.add(MaxPooling2D(pool_size=(2, 2)))\n", 623 | "\n", 624 | "model.add(Dropout(0.25))\n", 625 | "model.add(Flatten())\n", 626 | "model.add(Dense(64, activation='relu'))\n", 627 | "model.add(Dropout(0.5))\n", 628 | "model.add(Dense(2, activation='softmax'))" 629 | ] 630 | }, 631 | { 632 | "cell_type": "markdown", 633 | "metadata": {}, 634 | "source": [ 635 | "I used the stochastic gradient descent optimizer with learning rate of 0.01 and momentum of 0.9. \n", 636 | "\n", 637 | "I used the **binary crossentropy** loss function." 638 | ] 639 | }, 640 | { 641 | "cell_type": "code", 642 | "execution_count": 9, 643 | "metadata": { 644 | "collapsed": true, 645 | "scrolled": false 646 | }, 647 | "outputs": [], 648 | "source": [ 649 | "epochs = 50\n", 650 | "lrate = 0.01\n", 651 | "decay = lrate/epochs\n", 652 | "sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)\n", 653 | "model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])" 654 | ] 655 | }, 656 | { 657 | "cell_type": "markdown", 658 | "metadata": {}, 659 | "source": [ 660 | "Keras provides a convenient method to display a summary of the model. For each layer, this shows up the output shape and the number of trainable parameters.\n", 661 | "\n", 662 | "This a sanity check before starting fitting the model." 663 | ] 664 | }, 665 | { 666 | "cell_type": "code", 667 | "execution_count": 10, 668 | "metadata": { 669 | "collapsed": false, 670 | "scrolled": false 671 | }, 672 | "outputs": [ 673 | { 674 | "name": "stdout", 675 | "output_type": "stream", 676 | "text": [ 677 | "_________________________________________________________________\n", 678 | "Layer (type) Output Shape Param # \n", 679 | "=================================================================\n", 680 | "conv2d_1 (Conv2D) (None, 150, 150, 32) 896 \n", 681 | "_________________________________________________________________\n", 682 | "max_pooling2d_1 (MaxPooling2 (None, 75, 75, 32) 0 \n", 683 | "_________________________________________________________________\n", 684 | "conv2d_2 (Conv2D) (None, 75, 75, 32) 9248 \n", 685 | "_________________________________________________________________\n", 686 | "max_pooling2d_2 (MaxPooling2 (None, 37, 37, 32) 0 \n", 687 | "_________________________________________________________________\n", 688 | "conv2d_3 (Conv2D) (None, 37, 37, 64) 18496 \n", 689 | "_________________________________________________________________\n", 690 | "max_pooling2d_3 (MaxPooling2 (None, 18, 18, 64) 0 \n", 691 | "_________________________________________________________________\n", 692 | "dropout_1 (Dropout) (None, 18, 18, 64) 0 \n", 693 | "_________________________________________________________________\n", 694 | "flatten_1 (Flatten) (None, 20736) 0 \n", 695 | "_________________________________________________________________\n", 696 | "dense_1 (Dense) (None, 64) 1327168 \n", 697 | "_________________________________________________________________\n", 698 | "dropout_2 (Dropout) (None, 64) 0 \n", 699 | "_________________________________________________________________\n", 700 | "dense_2 (Dense) (None, 2) 130 \n", 701 | "=================================================================\n", 702 | "Total params: 1,355,938\n", 703 | "Trainable params: 1,355,938\n", 704 | "Non-trainable params: 0\n", 705 | "_________________________________________________________________\n" 706 | ] 707 | } 708 | ], 709 | "source": [ 710 | "model.summary()" 711 | ] 712 | }, 713 | { 714 | "cell_type": "markdown", 715 | "metadata": {}, 716 | "source": [ 717 | "Let's look at the network architecture:" 718 | ] 719 | }, 720 | { 721 | "cell_type": "markdown", 722 | "metadata": {}, 723 | "source": [ 724 | "### Visualizing the architecture" 725 | ] 726 | }, 727 | { 728 | "cell_type": "markdown", 729 | "metadata": { 730 | "collapsed": true 731 | }, 732 | "source": [ 733 | "" 734 | ] 735 | }, 736 | { 737 | "cell_type": "markdown", 738 | "metadata": { 739 | "collapsed": true 740 | }, 741 | "source": [ 742 | "### Training the model" 743 | ] 744 | }, 745 | { 746 | "cell_type": "markdown", 747 | "metadata": {}, 748 | "source": [ 749 | "Before training the model, I defined two callback functions that will be called while training.\n", 750 | "\n", 751 | "+ One for **early stopping** the training when the loss function stops improving on the validation data \n", 752 | "+ One for storing the validation loss and accuracy of each epoch: this allows to plot the training error\n" 753 | ] 754 | }, 755 | { 756 | "cell_type": "code", 757 | "execution_count": 11, 758 | "metadata": { 759 | "collapsed": true, 760 | "scrolled": false 761 | }, 762 | "outputs": [], 763 | "source": [ 764 | "## Callback for loss logging per epoch\n", 765 | "class LossHistory(Callback):\n", 766 | " def on_train_begin(self, logs={}):\n", 767 | " self.losses = []\n", 768 | " self.val_losses = []\n", 769 | " \n", 770 | " def on_epoch_end(self, batch, logs={}):\n", 771 | " self.losses.append(logs.get('loss'))\n", 772 | " self.val_losses.append(logs.get('val_loss'))\n", 773 | " \n", 774 | "history = LossHistory()\n", 775 | "\n", 776 | "## Callback for early stopping the training\n", 777 | "early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss',\n", 778 | " min_delta=0,\n", 779 | " patience=2,\n", 780 | " verbose=0, mode='auto')" 781 | ] 782 | }, 783 | { 784 | "cell_type": "markdown", 785 | "metadata": {}, 786 | "source": [ 787 | "I also used keras-tqdm which is an **awesome progress-bar ** that perfectly integrates with Keras. \n", 788 | "\n", 789 | "It allows you to monitor the training of you models very easily.\n", 790 | "\n", 791 | "What you need to do is simply load the TQDMNotebookCallback class from keras_tqdm then pass it as a third callback functions.\n", 792 | "\n", 793 | "```python\n", 794 | "from keras_tqdm import TQDMNotebookCallback\n", 795 | "```\n", 796 | "\n", 797 | "Here's what keras-tqdm looks like on simple example:" 798 | ] 799 | }, 800 | { 801 | "cell_type": "markdown", 802 | "metadata": {}, 803 | "source": [ 804 | "" 805 | ] 806 | }, 807 | { 808 | "cell_type": "markdown", 809 | "metadata": {}, 810 | "source": [ 811 | "A few words about the training:\n", 812 | "\n", 813 | "- We'll use the fit_generator method which is a variant (of the standard fit method) that takes a generator as input.\n", 814 | "\n", 815 | "- We'll train the model over 50 epochs: over one epoch 20000 unique and augmented images will flow by batch of 32 to the network, performing a forward and back propagation and adjusting the weights with SGD. The idea of using multiple epochs is to prevent overfitting.\n" 816 | ] 817 | }, 818 | { 819 | "cell_type": "markdown", 820 | "metadata": { 821 | "scrolled": true 822 | }, 823 | "source": [ 824 | "
\n", 825 | "```python\n", 826 | "fitted_model = model.fit_generator(\n", 827 | " train_generator,\n", 828 | " steps_per_epoch= int(n * (1-ratio)) // batch_size,\n", 829 | " epochs=50,\n", 830 | " validation_data=validation_generator,\n", 831 | " validation_steps= int(n * ratio) // batch_size,\n", 832 | " callbacks=[TQDMNotebookCallback(leave_inner=True, leave_outer=True), early_stopping, history],\n", 833 | " verbose=0)\n", 834 | "```\n", 835 | "
" 836 | ] 837 | }, 838 | { 839 | "cell_type": "markdown", 840 | "metadata": {}, 841 | "source": [ 842 | "This is a **heavy computation**: \n", 843 | "\n", 844 | "- If you're on your laptop this may take about 15 minutes per epoch\n", 845 | "- If you're using an the p2.xlarge EC2 instance like me, this takes about 2 minutes or so per epoch\n", 846 | "\n", 847 | "tqdm allows you to monitor the validation loss and accuracy on each epochs. This is useful to check the quality of your model." 848 | ] 849 | }, 850 | { 851 | "cell_type": "markdown", 852 | "metadata": {}, 853 | "source": [ 854 | "### Classification results" 855 | ] 856 | }, 857 | { 858 | "cell_type": "markdown", 859 | "metadata": {}, 860 | "source": [ 861 | "We reached **89.4% accuracy** (on the validation data) in 34 epochs (training/validation error and accuracy displayed below) \n", 862 | "\n", 863 | "This is a good result given that I didn't invest too much time designing the architecture of the network.\n", 864 | "\n", 865 | "Now let's save the model for later use." 866 | ] 867 | }, 868 | { 869 | "cell_type": "markdown", 870 | "metadata": { 871 | "collapsed": true 872 | }, 873 | "source": [ 874 | "
\n", 875 | "```python\n", 876 | "model.save('./models/model4.h5')\n", 877 | "```\n", 878 | "
" 879 | ] 880 | }, 881 | { 882 | "cell_type": "markdown", 883 | "metadata": {}, 884 | "source": [ 885 | "Let's plot the train and validation losses on the same graph:" 886 | ] 887 | }, 888 | { 889 | "cell_type": "markdown", 890 | "metadata": {}, 891 | "source": [ 892 | "
\n", 893 | "```python\n", 894 | "losses, val_losses = history.losses, history.val_losses\n", 895 | "fig = plt.figure(figsize=(15, 5))\n", 896 | "plt.plot(fitted_model.history['loss'], 'g', label=\"train losses\")\n", 897 | "plt.plot(fitted_model.history['val_loss'], 'r', label=\"val losses\")\n", 898 | "plt.grid(True)\n", 899 | "plt.title('Training loss vs. Validation loss')\n", 900 | "plt.xlabel('Epochs')\n", 901 | "plt.ylabel('Loss')\n", 902 | "plt.legend()\n", 903 | "```\n", 904 | "
" 905 | ] 906 | }, 907 | { 908 | "cell_type": "markdown", 909 | "metadata": {}, 910 | "source": [ 911 | "" 912 | ] 913 | }, 914 | { 915 | "cell_type": "markdown", 916 | "metadata": {}, 917 | "source": [ 918 | "We interrupt the training when the validation loss doesn't improve on two successive epochs." 919 | ] 920 | }, 921 | { 922 | "cell_type": "markdown", 923 | "metadata": {}, 924 | "source": [ 925 | "Let's now plot the accuracy on both the training set and validation set." 926 | ] 927 | }, 928 | { 929 | "cell_type": "markdown", 930 | "metadata": {}, 931 | "source": [ 932 | "
\n", 933 | "```python\n", 934 | "losses, val_losses = history.losses, history.val_losses\n", 935 | "fig = plt.figure(figsize=(15, 5))\n", 936 | "plt.plot(fitted_model.history['acc'], 'g', label=\"accuracy on train set\")\n", 937 | "plt.plot(fitted_model.history['val_acc'], 'r', label=\"accuracy on validation set\")\n", 938 | "plt.grid(True)\n", 939 | "plt.title('Training Accuracy vs. Validation Accuracy')\n", 940 | "plt.xlabel('Epochs')\n", 941 | "plt.ylabel('Loss')\n", 942 | "plt.legend()\n", 943 | "```\n", 944 | "
" 945 | ] 946 | }, 947 | { 948 | "cell_type": "markdown", 949 | "metadata": {}, 950 | "source": [ 951 | "" 952 | ] 953 | }, 954 | { 955 | "cell_type": "markdown", 956 | "metadata": {}, 957 | "source": [ 958 | "These two metrics keep increasing before reaching a plateau where the model eventually starts overfitting (from epoch 34)." 959 | ] 960 | }, 961 | { 962 | "cell_type": "markdown", 963 | "metadata": {}, 964 | "source": [ 965 | "## 4 - 2 Loading a pre-trained model " 966 | ] 967 | }, 968 | { 969 | "cell_type": "markdown", 970 | "metadata": {}, 971 | "source": [ 972 | "So far so good: we designed a custom convnet that performs reasonably well on the valiation data with ~ 89% accuracy.\n", 973 | " \n", 974 | "There is however a way to get a better score: loading the weights of a pre-trained convnet on a large dataset that includes images of cats and dogs among 1000 classes. Such a network would have learnt relevant features that are relevant to our classification.\n", 975 | "\n", 976 | "I'll load the weights of the VGG16 network: more specifically, I'm going to load the network weights up to the last conv layer. This network part acts as a feature detector to which we're going to add fully connected layers for our classification task.\n", 977 | " \n", 978 | "VGG16 is a very large network in comparison to LeNet5. It has 16 layers with trainable weights and around 140 millions parameters. To learn more about VGG16 please refer to this pdf link." 979 | ] 980 | }, 981 | { 982 | "cell_type": "markdown", 983 | "metadata": {}, 984 | "source": [ 985 | "We first start by loading the VGG16 weights (trained on ImageNet) by specifying that we're not interested in the last three FC layers." 986 | ] 987 | }, 988 | { 989 | "cell_type": "code", 990 | "execution_count": 12, 991 | "metadata": { 992 | "collapsed": true, 993 | "scrolled": false 994 | }, 995 | "outputs": [], 996 | "source": [ 997 | "from keras import applications\n", 998 | "# include_top: whether to include the 3 fully-connected layers at the top of the network.\n", 999 | "model = applications.VGG16(include_top=False, weights='imagenet')\n", 1000 | "datagen = ImageDataGenerator(rescale=1. / 255)" 1001 | ] 1002 | }, 1003 | { 1004 | "cell_type": "markdown", 1005 | "metadata": {}, 1006 | "source": [ 1007 | "Now we pass the images through the network to get a feature representation that we'll input to a neural network classifier.\n", 1008 | "\n", 1009 | "We do this for the training set and the validation set." 1010 | ] 1011 | }, 1012 | { 1013 | "cell_type": "code", 1014 | "execution_count": 13, 1015 | "metadata": { 1016 | "collapsed": false, 1017 | "scrolled": true 1018 | }, 1019 | "outputs": [ 1020 | { 1021 | "name": "stdout", 1022 | "output_type": "stream", 1023 | "text": [ 1024 | "Found 20000 images belonging to 2 classes.\n" 1025 | ] 1026 | } 1027 | ], 1028 | "source": [ 1029 | "generator = datagen.flow_from_directory('./data/train/',\n", 1030 | " target_size=(150, 150),\n", 1031 | " batch_size=batch_size,\n", 1032 | " class_mode=None,\n", 1033 | " shuffle=False)\n", 1034 | "\n", 1035 | "bottleneck_features_train = model.predict_generator(generator, int(n * (1 - ratio)) // batch_size)\n", 1036 | "np.save(open('./features/bottleneck_features_train.npy', 'wb'), bottleneck_features_train)" 1037 | ] 1038 | }, 1039 | { 1040 | "cell_type": "code", 1041 | "execution_count": 14, 1042 | "metadata": { 1043 | "collapsed": false, 1044 | "scrolled": false 1045 | }, 1046 | "outputs": [ 1047 | { 1048 | "name": "stdout", 1049 | "output_type": "stream", 1050 | "text": [ 1051 | "Found 5000 images belonging to 2 classes.\n" 1052 | ] 1053 | } 1054 | ], 1055 | "source": [ 1056 | "generator = datagen.flow_from_directory('./data/validation/',\n", 1057 | " target_size=(150, 150),\n", 1058 | " batch_size=batch_size,\n", 1059 | " class_mode=None,\n", 1060 | " shuffle=False)\n", 1061 | "\n", 1062 | "bottleneck_features_validation = model.predict_generator(generator, int(n * ratio) // batch_size,)\n", 1063 | "np.save('./features/bottleneck_features_validation.npy', bottleneck_features_validation)" 1064 | ] 1065 | }, 1066 | { 1067 | "cell_type": "markdown", 1068 | "metadata": {}, 1069 | "source": [ 1070 | "When the images are passed to the network, they are in the right orders. So we associate the labels to them easily." 1071 | ] 1072 | }, 1073 | { 1074 | "cell_type": "code", 1075 | "execution_count": 15, 1076 | "metadata": { 1077 | "collapsed": true, 1078 | "scrolled": false 1079 | }, 1080 | "outputs": [], 1081 | "source": [ 1082 | "train_data = np.load('./features/bottleneck_features_train.npy')\n", 1083 | "train_labels = np.array([0] * (int((1-ratio) * n) // 2) + [1] * (int((1 - ratio) * n) // 2))\n", 1084 | "\n", 1085 | "validation_data = np.load('./features/bottleneck_features_validation.npy')\n", 1086 | "validation_labels = np.array([0] * (int(ratio * n) // 2) + [1] * (int(ratio * n) // 2))" 1087 | ] 1088 | }, 1089 | { 1090 | "cell_type": "markdown", 1091 | "metadata": {}, 1092 | "source": [ 1093 | "Now we design a small fully connected network that plugs in to the features extracted from the VGG16 and acts as the classification part of a CNN." 1094 | ] 1095 | }, 1096 | { 1097 | "cell_type": "code", 1098 | "execution_count": 16, 1099 | "metadata": { 1100 | "collapsed": true, 1101 | "scrolled": false 1102 | }, 1103 | "outputs": [], 1104 | "source": [ 1105 | "model = Sequential()\n", 1106 | "model.add(Flatten(input_shape=train_data.shape[1:]))\n", 1107 | "model.add(Dense(512, activation='relu'))\n", 1108 | "model.add(Dropout(0.5))\n", 1109 | "model.add(Dense(256, activation='relu'))\n", 1110 | "model.add(Dropout(0.2))\n", 1111 | "model.add(Dense(1, activation='sigmoid'))\n", 1112 | "\n", 1113 | "model.compile(optimizer='rmsprop',\n", 1114 | " loss='binary_crossentropy', metrics=['accuracy'])" 1115 | ] 1116 | }, 1117 | { 1118 | "cell_type": "code", 1119 | "execution_count": 17, 1120 | "metadata": { 1121 | "collapsed": false, 1122 | "scrolled": false 1123 | }, 1124 | "outputs": [ 1125 | { 1126 | "name": "stdout", 1127 | "output_type": "stream", 1128 | "text": [ 1129 | "\r" 1130 | ] 1131 | } 1132 | ], 1133 | "source": [ 1134 | "fitted_model = model.fit(train_data, train_labels,\n", 1135 | " epochs=15,\n", 1136 | " batch_size=batch_size,\n", 1137 | " validation_data=(validation_data, validation_labels[:validation_data.shape[0]]),\n", 1138 | " verbose=0,\n", 1139 | " callbacks=[TQDMNotebookCallback(leave_inner=True, leave_outer=False), history])" 1140 | ] 1141 | }, 1142 | { 1143 | "cell_type": "markdown", 1144 | "metadata": {}, 1145 | "source": [ 1146 | "We reached **90.7% accuracy** on 15 epochs only. Quite not bad.\n", 1147 | "\n", 1148 | "Note that each epoch takes around 1 minute on my personal laptop." 1149 | ] 1150 | }, 1151 | { 1152 | "cell_type": "code", 1153 | "execution_count": 18, 1154 | "metadata": { 1155 | "collapsed": false, 1156 | "scrolled": false 1157 | }, 1158 | "outputs": [ 1159 | { 1160 | "data": { 1161 | "text/plain": [ 1162 | "" 1163 | ] 1164 | }, 1165 | "execution_count": 18, 1166 | "metadata": {}, 1167 | "output_type": "execute_result" 1168 | }, 1169 | { 1170 | "data": { 1171 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3sAAAFNCAYAAAC5cXZ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VOX9/vH3JwkJ+xYgIAEyKqLsmwgqkBxbi7vYWrVa\nl1ap1rZal2+t/bW1tYsWW5dWRWpbt7pWrUtRaAsBxI3FgFCQfQfZZAlbtuf3xzMhQwgkkExOJrlf\n13WuZM6cOfOZeRKYO89yzDmHiIiIiIiI1C9JYRcgIiIiIiIiNU9hT0REREREpB5S2BMREREREamH\nFPZERERERETqIYU9ERERERGRekhhT0REREREpB5S2BMRqcPMLNnM8s2sa00eewx1/MrMnqrp8yYK\nM7vezHKj3x/xfY499hifa5KZXXmsjz/CeZ8zs3tq+rwiIlJ3KeyJiNSgaAgo3UrMbG/M7aP+AO+c\nK3bONXfOra7JYxsSM2tqZjvNbEQF9/3RzF48mvPV5PtcUYh2zp3tnPt7dc9dVx1Ne5jZlWY208x2\nm9kmM/vQzG4s95ghZjbBzLZHtwVmdq+ZtY7e39nM3jKzDWbmzCyzguc928w+iT7PGjO7JB6vXUSk\ntinsiYjUoGgIaO6caw6sBi6I2XfIB3gzS6n9KhsW59we4BXg6tj9ZtYIuBx4Ooy6GqqqtoeZ/Qj4\nPfBbICO6fRcYaWbJ0WOGA5OBXOAk51xr4NzoKXtHv5YAE4CvVVSPmfUBngXuAloBA4C86r9SEZHw\nKeyJiNSiaE/OS2b2gpntAq4ys2HRHovt0d6HR6IffDGzlGhvRFb09nPR+98xs11m9oGZRY722Oj9\n55jZYjPbEe1RmWFm11bxdYyO9qBsN7PJZtYj5r67zWx9tPdmkZllR/cPNbM50f2fm9nYw5x7iZmN\nirmdambbzKxvtFfoeTPbGn3uj82sXRVKfhr4mpk1jtl3DlAETIo+z/8zs+XR92qBmV14mPrKv8/t\nzezt6Ov6EIiUO/5PZrY2ev9MMzs9uv984P+AK6M9v7Oj+98rbQczSzKzn5nZqmjP1lNm1jJ634nR\nOq6Onn+zmd1VhfeitK4bzWxp9L38p5l1innOR6LPt8PM5plZz9KazWxh9D1aa2Y/rOrzlXPE9jCz\nNsA9wHecc6855/KdN8c5d4Vzrjj6mLHAn51zv3PObQJwzq1yzv3UOfde9PYG59zjwOzD1PJT4DHn\n3ETnXJFzbotzbvkxvi4RkTpFYU9EpPaNBp7H9yK8hP+AewvQDjgDGAV85wiP/wb+A2pbfO/hvUd7\nrJl1AF4G7ow+7wpgSFWKN7NT8D0h3wfaA/8B3jSzRmbWK1r7QOdcS/wH+NLhjn8Exkb3nwj84zBP\n8QJwRcztc4D1zrl5wHVAUyATSMf39OyrQtnTgW3AxTH7vgn8PSY4LMa//62AXwPPm1lGFc79OLAL\n6AiMAb5V7v6PgL74NvgH8IqZpTnn3gZ+F62huXNuUAXnvh64CsgGTgDaAA+XO+Z0/Pv5FeAXZta9\nsoLN7Gzgl/jers7AeqC05/kcYCjQPfp8l+PfO4C/Ad92zrWIvqaplT3XYVTWHmfgP6O8dYTX0BL/\nM/vqMdZQaiiQZGbzo39seSYaNkVEEp7CnohI7XvPOfeWc67EObfXOTfTOfdRtFdhOTAeGHmEx//D\nOTfLOVeI/4De/xiOPR/Ic869Eb3vQWBLFeu/HHjTOTc5+tj78AHpNHxwbQz0MrMU59yKmF6SQqC7\nmaU753Y55z46zPmfBy6O6fX5RnRf6TnaASdG587Ncs7lV1awc84BzxAdOmh+PtcFxAzhdM69HO0F\nKnHOPQ+sBAYf6bzRHtiLgZ865/ZEA+mz5Z77WefcNudcET7clYbdqrgSeCD6Pu4C7ga+YWax/3/f\n45zb55ybAywA+lXxvE865/Kcc/vwQxhHmp/PVhit8eRo/f9zzm2MPq4Q6GlmLaKvaU4VX8dBqtAe\n7YDNzrmS0sdEe3G3m58Hezo+PBuwMeaYP0SP2V2VXk4zM3zYvQrfjidFX/tDx/K6RETqGoU9EZHa\ntyb2hpmdbGb/MrONZrYT3+NypKGJG2O+3wM0P4Zjj4utI/rhe20Vai997KqYx5ZEH9vZOfcZcDv+\nNWwyP1y1Y/TQ64CewGfRD+7nUgHn3CJgGXCemTXHB9PSsPcUvifxZTNbZ2b3WdXnPT4DfDnaW/d1\nYKFz7tPSO83sWjObGw0L2/Fhp7IhohlAMge36arYA8zs/6LDWXcAXwDNqnDeUge919HvU/E9qgDE\nBDGo/OehwvM653ZGa+vsnJsEjMP3WH5uZuPMrEX00NHAhcBqM8s1s9MqOrmZfWZlCxMNO0wNR2qP\nrUCH2FDrnBsSnZO3A//5ZRvggE4xx9wWPeYtoNKfi+jP/T7gr865pdFA/VvK5v2JiCQ0hT0Rkdrn\nyt1+ApiP761qCfwM32MRTxvwQyGBg3o4qmI90C3msUnRc60DcM4955w7Az93LRn/4Rnn3GfOucuB\nDviFN14tN2crVulQztH4HsiV0XMUOOfucc6dApwZvb9Kq5xGexg/iB7/TWJ69czseHy4uQlIjwaG\nRVTeDp/jFwDpErPvwCUZzCwHuA34KtAaPywyP+a85X8WyjvovY6euwDYXMnjKlO+DVtEayttw4ec\ncwPxi5z0jL4Goj3QF+Lb8G2gwpVMnXM9YhYm+uAwxxy2PYAZQDG+t69C0YA6C6juypnzOLgdKmsT\nEZGEobAnIhK+Fvjeit3R+XBHmq9XU94GBprZBdGesVuI6S2qxMvAhWaWHR3GeCd+ztpHZnaKmeWY\nWRqwN7qVAJjZN82sXbQncAf+Q3VJxU/BC/i5Y2Mo69XDzAIz6x0NmDvxwwoPd46KPB19rafFnhff\nG+bwIcrM7AaiwxiPJDqM9Z/4uXJNzKw3PriUaoEf2roFaIRfdKRZzP2fA1nRsF2RF4DbzCwrGsh+\nDbwQO7zxGL0AfNv8ojdp+EA+3Tm31vylDIZEfy5248NlSfT1fcPMWkZf9y6O7r2vSIXt4Zzbhp9f\nOs7MLjGz5tGFYwYATWIefycwJtp72h7AzLpwcEAm+keFtOjNtOhrLvW36HuRZWZNgR/hfz9ERBKe\nwp6ISPhuB67Bf3h+Ar9oS1w55z4HLgP+gB8ydwLwCbC/Co9dgK/3cXw4GgVcGA0Aafh5aVvwQ0jb\nAD+JPvRcYKH5VUgfAC5zzhUc5jnW4ntthuLDZanjgNfwQW8Bfkjn8wBm9qSZ/amS8l/BD6GcWLp6\nY/T55uEXkPkY3+vZA7+wSlXcFH2dnwN/wYeHUhOiNS7BzwHcGT1/qZfwwzK3mdnHFZz7z9FjpgPL\n8T8jt1SxrsNyzr2LH2r7erSerpT1kLaOvo7t0Zo34H9OwLf7quhw42/j57pVR4XtEa3xN/jgdTew\nCf/z9Dj+9+Wj6DFTgS8BOcDS6PDbd/Dv+WNw4PIme6OvB2ApPsSW+jM+/M7CD23dDRzrKqMiInWK\n+eHqIiLSkJm/btl64GvOuelh1yMiIiLVp549EZEGysxGmVnr6JC2n+KHRFbUuyQiIiIJSGFPRKTh\nOhM/NHAz/hpto51zlQ7jFBERkcSgYZwiIiIiIiL1kHr2RERERERE6iGFPRERERERkXooJewCjla7\ndu1cVlZW2GUcYvfu3TRr1qzyA6XOUJslHrVZYlF7JR61WeJRmyUWtVfiqattNnv27C3OuUqvj5tw\nYS8rK4tZs2aFXcYhcnNzyc7ODrsMOQpqs8SjNkssaq/EozZLPGqzxKL2Sjx1tc3MbFVVjtMwThER\nERERkXpIYU9ERERERKQeUtgTERERERGphxJuzp6IiIiIiISjsLCQtWvXsm/fvrBLqRWtWrVi4cKF\noT1/48aNyczMpFGjRsf0eIU9ERERERGpkrVr19KiRQuysrIws7DLibtdu3bRokWLUJ7bOcfWrVtZ\nu3YtkUjkmM6hYZwiIiIiIlIl+/btIz09vUEEvbCZGenp6dXqRVXYExERERGRKlPQqz3Vfa8V9kRE\nREREJCFs376dxx577Jgee+6557J9+/YqH3/PPffwyCOPHNNz1RUKeyIiIiIikhCOFPaKioqO+NgJ\nEybQunXreJRVZ8U17JnZKDP7zMyWmtldFdzfxsxeN7N5ZvaxmfWOZz3xMu/zeby5/s2wyxARERER\nqdfuuusuli1bRv/+/bnzzjvJzc1l+PDhXHjhhfTs2ROAiy++mEGDBtGrVy/Gjx9/4LFZWVls2bKF\nlStXcsopp3DDDTfQq1cvzj77bPbu3XvE583Ly2Po0KH07duX0aNH88UXXwDwyCOP0LNnT/r27cvl\nl18OwNSpU+nfvz/9+/dnwIAB7Nq1C4CxY8dy6qmn0rdvX37+858DsHv3bs477zz69etH7969eeml\nl2r0/Ypb2DOzZOBR4BygJ3CFmfUsd9jdQJ5zri9wNfBwvOqJp38u+icPLXmIL/Z+EXYpIiIiIiL1\n1n333ccJJ5xAXl4eY8eOBWDOnDk8/PDDLF68GIC//vWvzJ49m1mzZvHII4+wdevWQ86zZMkSbr75\nZhYsWEDr1q159dVXj/i8V199Nffffz/z5s2jT58+/OIXvzhQzyeffMK8efMYN24cAA888ACPPvoo\neXl5TJ8+nSZNmjBp0iSWLFnCxx9/TF5eHrNnz2batGm8++67HHfcccydO5f58+czatSomny74nrp\nhSHAUufccgAzexG4CPhfzDE9gfsAnHOLzCzLzDKcc5/Hsa4aF0QCfp77c6aumsrFJ18cdjkiIiIi\nInF367u3krcxr0bP2b9jfx4a9dBRPWbIkCEHXZrgkUce4fXXXwdgzZo1LFmyhPT09IMeE4lE6N+/\nPwCDBg1i5cqVhz3/jh072L59OyNHjgTgmmuu4dJLLwWgb9++XHnllVx88cVcfLHPAWeccQa33XYb\nV155JZdccgmZmZlMmjSJSZMmMWDAAADy8/NZsmQJw4cP5/bbb+dHP/oR559/PsOHDz+q116ZeIa9\nzsCamNtrgdPKHTMXuASYbmZDgG5AJnBQ2DOzMcAYgIyMDHJzc+NU8rEpLCkkLSmNZ957htYbG9Y4\n4ESWn59f536W5MjUZolF7ZV41GaJR22WWOpDe7Vq1erAsMSCggKKi4tr9PwFBQUHzl+R/Px8SkpK\nDhyzZ88e0tLSDtyePn06EydOZNKkSTRt2pRzzz2Xbdu2sWvXLpxz5Ofnk5+fT6NGjQ48pqioiN27\ndx/yvPv37yc5OfnAY0vvj63hxRdfZMaMGbzzzjvce++9fPjhh9x8881kZ2czadIkTj/9dF5//XX2\n79/PD3/4Q771rW8d8pqmTp3KpEmT+PGPf8zIkSO5666DZ7/t27fvmH9uwr6o+n3Aw2aWB3wKfAIc\n8hPjnBsPjAcYPHiwy87Ors0aq6Tv/L4sLlhMXaxNKpabm6v2SjBqs8Si9ko8arPEozZLLPWhvRYu\nXHjgIuOPXXhsq2JWR6dOndi9e/eBGpo2bUpKSsqB24WFhbRr146MjAwWLVrEzJkzadq0KS1atMDM\naN68OQBJSUkHHpOWlkZhYeEhF09PS0sjKSmJzMxM2rZtS15eHsOHD+f1118nJyeHZs2asXr1as47\n7zzOPvtsunXrhpmxadMmhg4dytChQ5k3bx5r1qzhggsu4Kc//Snf/va3ad68OevWraNRo0YUFRWR\nkZHBDTfcQKdOnXjyyScPqaNx48YHegSPVjzD3jqgS8ztzOi+A5xzO4HrAMxfRGIFsDyONcXNgNYD\nGL9iPJ/nf05G84ywyxERERERqXfS09M544wz6N27N+eccw7nnXfeQfePGjWKcePGccopp9CjRw+G\nDh1aI8/79NNPc+ONN7Jnzx6OP/54/va3v1FcXMxVV13Fjh07cM7xgx/8gNatW/PTn/6UKVOmkJSU\nRK9evTjnnHNIS0tj4cKFDBs2DIDmzZvz3HPPsXTpUu68806SkpJo1KgRjz/+eI3UWyqeYW8m0N3M\nIviQdznwjdgDzKw1sMc5VwBcD0yLBsCEM7DNQFgBU1ZO4fLel4ddjoiIiIhIvfT8888fdDu2tzQt\nLY133nmnwseVzstr164d8+fPP7D/jjvuqPD4e+6558DQzf79+/Phhx8ecsx77713yL4//vGPFZ7v\nlltu4ZZbbjlo3wknnMBXvvKVCo+vCXFbjdM5VwR8D5gILAReds4tMLMbzezG6GGnAPPN7DP8qp23\nVHy2uu/E5ifSKq0Vk1dMDrsUERERERGR+M7Zc85NACaU2zcu5vsPgJPiWUNtSbZkRmaNVNgTERER\nEZE6Ia4XVW9ogqyAZV8sY9X2VWGXIiIiIiIiDZzCXg0KIgHg5+2JiIiIiIiESWGvBvXq0Iv2Tdtr\nKKeIiIiIiIROYa8GJVkSOZEcJq+YjHMu7HJERERERKQBU9irYUFWwLpd61iybUnYpYiIiIiINHil\nF1Kv6v76RGGvhpXO29NQThERERERCZPCXg07se2JZLbMVNgTEREREalhd911F48++uiB2/fccw8P\nPPAA+fn5nHXWWQwcOJA+ffrwxhtvVPmczjnuvPNOevfuTZ8+fXjppZcA2LBhA6NGjaJ///707t2b\n6dOnU1xczLXXXnvg2AcffBCAZcuWMWrUKAYNGsTw4cNZtGgRAK+88gq9e/emX79+jBgxogbfiaqJ\n63X2GiIzI4gETFgygRJXQpIpT4uIiIiI1ITLLruMW2+9lZtvvhmAl19+mYkTJ9K4cWNef/11WrZs\nyZYtWxg6dCgXXnghZlbpOV977TXy8vKYO3cuW7Zs4dRTT2XEiBE8//zznHXWWfzyl7+kuLiYPXv2\nkJeXx7p165g/fz4A27dvB2DMmDGMGzeO7t2789FHH/Hd736XyZMn88tf/pKJEyfSuXPnA8fWJoW9\nOAiyAp6Z+wzzN82nb0bfsMsREREREal5t94KeXk1e87+/eGhhw5794ABA9i0aRPr169n8+bNtGnT\nhi5dulBYWMjdd9/NtGnTSEpKYt26dXz++ed07Nix0qd87733uOKKK0hOTiYjI4ORI0cyc+ZMTj31\nVK699lqSkpK4+OKL6d+/P8cffzzLly/n+9//Pueddx5nn302+fn5vP/++1x66aUHzrl//34Azjjj\nDK699lq+/vWvc8kll1T//TlK6naKg5xIDqB5eyIiIiIiNe3SSy/lH//4By+99BKXXXYZAH//+9/Z\nvHkzs2fPJi8vj4yMDPbt21et5xkxYgTvvvsunTt35tprr+WZZ56hTZs2zJ07l+zsbMaNG8f1119P\nSUkJrVu3Ji8v78C2cOFCAMaNG8evfvUr1qxZw6BBg9i6dWu1X//RUM9eHHRt1ZUT257I5BWTuXXo\nrWGXIyIiIiJS847QAxdPl112GTfccANbtmxh6tSpAOzYsYMOHTrQqFEjpkyZwqpVq6p8vuHDh/PE\nE09wzTXXsG3bNqZNm8bYsWNZtWoVHTp04IYbbmD//v3MmTOHc889l9TUVL761a/So0cPrrrqKlq2\nbEkkEuGVV17h0ksvxTnHvHnz6NevH8uWLeO0007jtNNO45133mHNmjWkp6fH6605hMJenARZAS8u\neJGikiJSkvQ2i4iIiIjUhF69erFr1y46d+5Mp06dALjyyiu54IIL6NOnD4MHD+bkk0+u8vlGjx7N\nBx98QL9+/TAzfve739GxY0eefvpp7r//ftLS0mjevDnPPPMM69at47rrrqOkpASA3/72t4DvWbzp\nppv41a9+RWFhIZdffjn9+vXjzjvvZMmSJTjnOOuss+jXr1/NvyFHoBQSJ0EkYPyc8czZMIchnYeE\nXY6IiIiISL3x6aefHnS7Xbt2fPDBBxUem5+ff8T9ZsbYsWMZO3bsQfdfc801XHLJJbRo0eKg/XPm\nzDnkXJFIhHffffeQ/a+99trhX0Qt0Jy9OMnOygY0b09ERERERMKhsBcnGc0z6N2ht8KeiIiIiIiE\nQmEvjoKsgPdWv8f+ov1hlyIiIiIiIg2Mwl4cBZGAvUV7+WjdR2GXIiIiIiJSI5xzYZfQYFT3vVbY\ni6ORWSNJsiQN5RQRERGReqFx48Zs3bpVga8WOOfYunUrjRs3PuZzaDXOOGrduDUDOw1k8orJ3JN9\nT9jliIiIiIhUS2ZmJmvXrmXz5s1hl1Ir9u3bV62wVV2NGzcmMzPzmB+vsBdnQVbAgx8+yO6C3TRL\nbRZ2OSIiIiIix6xRo0ZEIpGwy6g1ubm5DBgwIOwyjpmGccZZEAkoLClkxpoZYZciIiIiIiINiMJe\nnJ3Z9UxSklI0b09ERERERGqVwl6cNUttxtDMoQp7IiIiIiJSq+Ia9sxslJl9ZmZLzeyuCu5vZWZv\nmdlcM1tgZtfFs56wBFkBszfMZvu+7WGXIiIiIiIiDUTcwp6ZJQOPAucAPYErzKxnucNuBv7nnOsH\nZAO/N7PUeNUUliASUOJKmLZqWtiliIiIiIhIAxHPnr0hwFLn3HLnXAHwInBRuWMc0MLMDGgObAOK\n4lhTKIZmDqVxSmMN5RQRERERkVoTz7DXGVgTc3ttdF+sPwGnAOuBT4FbnHMlcawpFGkpaZzZ9UyF\nPRERERERqTXmnIvPic2+Boxyzl0fvf1N4DTn3PfKHXMGcBtwAvBvoJ9zbme5c40BxgBkZGQMevHF\nF+NSc3Xk5+fTvHnzw97/99V/58kVT/LasNdok9qmFiuTw6mszaTuUZslFrVX4lGbJR61WWJReyWe\nutpmOTk5s51zgys7Lp4XVV8HdIm5nRndF+s64D7nE+dSM1sBnAx8HHuQc248MB5g8ODBLjs7O141\nH7Pc3FyOVFeTtU148i9PUti5kOxehz9Oak9lbSZ1j9ossai9Eo/aLPGozRKL2ivxJHqbxXMY50yg\nu5lFoouuXA68We6Y1cBZAGaWAfQAlsexptAMOm4QLVJbaCiniIiIiIjUirj17Dnniszse8BEIBn4\nq3NugZndGL1/HHAv8JSZfQoY8CPn3JZ41RSmlKQURmaNVNgTEREREZFaEc9hnDjnJgATyu0bF/P9\neuDseNZQlwRZAW8vfps1O9bQpVWXyh8gIiIiIiJyjOJ6UXU5WBAJAJiyckrIlYiIiIiISH2nsFeL\n+mT0Ib1JuoZyioiIiIhI3Cns1aIkSyInksPkFZOJ1yUvREREREREQGGv1gVZAWt2rmHZF8vCLkVE\nREREROoxhb1aVjpvT0M5RUREREQknhT2atlJ6SdxXIvjFPZERERERCSuFPZqmZkRRALN2xMRERER\nkbhS2AtBkBWwec9mFmxeEHYpIiIiIiJSTynshUDz9kREREREJN4U9kLQrXU3jm9zvMKeiIiIiIjE\njcJeSIKsgNyVuRSXFIddioiIiIiI1EMKeyEJIgE79u/gk42fhF2KiIiIiIjUQwp7IcmJ5AAwZcWU\nkCsREREREZH6SGEvJB2bd6Rn+55MXql5eyIiIiIiUvMU9kIUZAVMXzWdguKCsEsREREREZF6RmEv\nREEkYHfhbmaumxl2KSIiIiIiUs8o7IVoZNZIDNMlGEREREREpMYp7IWobZO2DOg0QPP2RERERESk\nxinshSzICnh/zfvsLdwbdikiIiIiIlKPKOyFLIgEFBQX8P6a98MuRURERERE6hGFvZCd2fVMUpJS\nNG9PRERERERqlMJeyFqktWBI5yGatyciIiIiIjVKYa8OCLICZq6byc79O8MuRURERERE6gmFvTog\nJ5JDsStm+qrpYZciIiIiIiL1RFzDnpmNMrPPzGypmd1Vwf13mlledJtvZsVm1jaeNdVFwzKHkZac\npnl7IiIiIiJSY+IW9swsGXgUOAfoCVxhZj1jj3HOjXXO9XfO9Qd+DEx1zm2LV011VZNGTTi9y+ma\ntyciIiIiIjUmnj17Q4ClzrnlzrkC4EXgoiMcfwXwQhzrqdOCSEDexjy27tkadikiIiIiIlIPxDPs\ndQbWxNxeG913CDNrCowCXo1jPXVaEAkAyF2ZG24hIiIiIiJSL6SEXUDUBcCMww3hNLMxwBiAjIwM\ncnNza7G0qsnPz69WXUUlRTROasyzM54lfVN6zRUmh1XdNpPapzZLLGqvxKM2Szxqs8Si9ko8id5m\n8Qx764AuMbczo/sqcjlHGMLpnBsPjAcYPHiwy87OrqESa05ubi7VrStnQw6fbf+s2ueRqqmJNpPa\npTZLLGqvxKM2Szxqs8Si9ko8id5m8RzGORPobmYRM0vFB7o3yx9kZq2AkcAbcawlIQSRgEVbFrF+\n1/qwSxERERERkQQXt7DnnCsCvgdMBBYCLzvnFpjZjWZ2Y8yho4FJzrnd8aolUZTO25uyYkrIlYiI\niIiISKKL65w959wEYEK5fePK3X4KeCqedSSKfhn9aNO4DZNXTObKvleGXY6IiIiIiCSwuF5UXY5O\nclIy2VnZut6eiIiIiIhUm8JeHRNEAlZuX8mKL1aEXYqIiIiIiCQwhb06pnTe3uQV6t0TEREREZFj\np7BXx5zS7hQymmVoKKeIiIiIiFSLwl4dY2YEkYDJKybjnAu7HBERERERSVAKe3VQEAnYmL+RRVsW\nhV2KiIiIiIgkKIW9Okjz9kREREREpLoU9uqgSOsI3Vp107w9ERERERE5Zgp7dVDpvL0pK6ZQ4krC\nLkdERERERBKQwl4dFUQCvtj3BXM3zg27FBERERERSUAKe3VUTlYOoHl7IiIiIiJybBT26qjOLTvT\nI72H5u2JiIiIiMgxUdirw4JIwLRV0ygsLgy7FBERERERSTAKe3VYEAnIL8hn1vpZYZciIiIiIiIJ\nRmGvDsvOygY0b09ERERERI6ewl4d1q5pO/pl9NO8PREREREROWoKe3VcEAmYsXoG+4r2hV2KiIiI\niIgkEIW9Oi6IBOwv3s8Haz4IuxQREREREUkgCnt13IhuI0i2ZM3bExERERGRo6KwV8e1TGvJ4OMG\na96eiIiIiIgcFYW9BBBEAj5e9zG79u8KuxQREREREUkQCnsJIIgEFJUU8d7q98IuRUREREREEoTC\nXgI4vcsubZsNAAAgAElEQVTppCanat6eiIiIiIhUmcJeAmjaqCnDModp3p6IiIiIiFRZXMOemY0y\ns8/MbKmZ3XWYY7LNLM/MFpjZ1HjWk8iCSMAnGz5h295tYZciIiIiIiIJIG5hz8ySgUeBc4CewBVm\n1rPcMa2Bx4ALnXO9gEvjVU+iCyIBDsfUlcrDIiIiIiJSuXj27A0BljrnljvnCoAXgYvKHfMN4DXn\n3GoA59ymONaT0IZ0HkLTRk01b09ERERERKoknmGvM7Am5vba6L5YJwFtzCzXzGab2dVxrCehpSan\nMrzrcM3bExERERGRKkmpA88/CDgLaAJ8YGYfOucWxx5kZmOAMQAZGRnk5ubWdp2Vys/Pj3td3Uq6\nMXHzRF6b9BptU9vG9bkagtpoM6lZarPEovZKPGqzxKM2Syxqr8ST6G0Wz7C3DugSczszui/WWmCr\nc243sNvMpgH9gIPCnnNuPDAeYPDgwS47OzteNR+z3Nxc4l1X8/XNGf/n8ezvtJ/sPvF9roagNtpM\napbaLLGovRKP2izxqM0Si9or8SR6m8VzGOdMoLuZRcwsFbgceLPcMW8AZ5pZipk1BU4DFsaxpoQ2\noOMAWqW10rw9ERERERGpVNx69pxzRWb2PWAikAz81Tm3wMxujN4/zjm30MzeBeYBJcCTzrn58aop\n0SUnJZOdla15eyIiIiIiUqm4ztlzzk0AJpTbN67c7bHA2HjWUZ8EkYA3PnuDldtXktU6K+xyRERE\nRESkjorrRdWl5gWRAIApK6aEXImIiIiIiNRlCnsJplf7XrRv2l5DOUVERERE5IgU9hKMmRFEAiav\nmIxzLuxyRERERESkjlLYS0BBJGD9rvUs3rq48oNFRERERKRBUthLQKXz9nQJBhERERERORyFvQR0\nQpsT6NKyi+btiYiIiIjIYSnsJaDSeXu5K3MpcSVhlyMiIiIiInWQwl6CCiIBW/ZsYf4mXYNeRERE\nREQOpbCXoHKycgDN2xMRERERkYpVKeyZ2Qlmlhb9PtvMfmBmreNbmhxJl1Zd6N62u8KeiIiIiIhU\nqKo9e68CxWZ2IjAe6AI8H7eqpEqCSMDUVVMpKikKuxQREREREaljqhr2SpxzRcBo4I/OuTuBTvEr\nS6oiiATs3L+TORvmhF2KiIiIiIjUMVUNe4VmdgVwDfB2dF+j+JQkVZWdlQ1o3p6IiIiIiByqqmHv\nOmAY8Gvn3AoziwDPxq8sqYoOzTrQp0MfhT0RERERETlElcKec+5/zrkfOOdeMLM2QAvn3P1xrk2q\nICcrh/dWv8f+ov1hlyIiIiIiInVIVVfjzDWzlmbWFpgD/NnM/hDf0qQqgkjA3qK9fLTuo7BLERER\nERGROqSqwzhbOed2ApcAzzjnTgO+FL+ypKpGZo0kyZI0lFNERERERA5S1bCXYmadgK9TtkCL1AGt\nG7dmYKeBCnsiIiIiInKQqoa9XwITgWXOuZlmdjywJH5lydEIsgI+XPshuwt2h12KiIiIiIjUEVVd\noOUV51xf59xN0dvLnXNfjW9pUlVBJKCwpJAZa2aEXYqIiIiIiNQRVV2gJdPMXjezTdHtVTPLjHdx\nUjVndj2TlKQUDeUUEREREZEDqjqM82/Am8Bx0e2t6D6pA5qlNmNo5lCFPREREREROaCqYa+9c+5v\nzrmi6PYU0D6OdclRCrICZm+YzfZ928MuRURERERE6oCqhr2tZnaVmSVHt6uArfEsTI5OEAkocSVM\nWzUt7FJERERERKQOqGrY+xb+sgsbgQ3A14BrK3uQmY0ys8/MbKmZ3VXB/dlmtsPM8qLbz46idokx\nNHMojVMaayiniIiIiIgAkFKVg5xzq4ALY/eZ2a3AQ4d7jJklA48CXwbWAjPN7E3n3P/KHTrdOXf+\nUVUth0hLSePMrmcq7ImIiIiICFD1nr2K3FbJ/UOApdHLNBQALwIXVeP5pBJBVsCnmz5l0+5NYZci\nIiIiIiIhq07Ys0ru7wysibm9NrqvvNPNbJ6ZvWNmvapRT4MXRAIAclfmhluIiIiIiIiErkrDOA/D\n1cDzzwG6Oufyzexc4J9A9/IHmdkYYAxARkYGubm5NfDUNSs/Pz/0uopdMU2Tm/LcjOfosLlDqLUk\ngrrQZnJ01GaJRe2VeNRmiUdtlljUXokn0dvsiGHPzHZRcagzoEkl514HdIm5nRndd4BzbmfM9xPM\n7DEza+ec21LuuPHAeIDBgwe77OzsSp669uXm5lIX6go2BizasqhO1FLX1ZU2k6pTmyUWtVfiUZsl\nHrVZYlF7JZ5Eb7MjDuN0zrVwzrWsYGvhnKusV3Am0N3MImaWClyOvzD7AWbW0cws+v2QaD26pEM1\nBFkBS7YtYc2ONZUfLCIiIiIi9VZ15uwdkXOuCPgeMBFYCLzsnFtgZjea2Y3Rw74GzDezucAjwOXO\nuZoYHtpglc7bm7JySsiViIiIiIhImKozZ69SzrkJwIRy+8bFfP8n4E/xrKGh6ZPRh/Qm6UxeMZmr\n+10ddjkiIiIiIhKSuPXsSTiSLImcSA6TV0xGnaQiIiIiIg2Xwl49FGQFrNm5hmVfLAu7FBERERER\nCYnCXj1UOm9v8orJIVciIiIiIiJhUdirh05KP4njWhynsCciIiIi0oAp7NVDZkYQCTRvT0RERESk\nAVPYq6eCrIDNezazYPOCsEsREREREZEQKOzVU5q3JyIiIiLSsCns1VPdWnfj+DbHK+yJiIiIiDRQ\nCnv1WJAVkLsyl+KS4rBLERERERGRWqawV48FkYAd+3fwycZPwi5FRERERERqmcJePZYTyQE0b09E\nREREpCFS2KvHOjbvSM/2PRX2REREREQaIIW9ei7ICpi+ejoFxQVhlyIiIiIiIrVIYa+eCyIBewr3\n8PG6j8MuRUREREREapHCXj03Mmskhmkop4iIiIhIA6OwV8+1bdKWAZ0GKOyJiIiIiDQwCnsNQJAV\n8MHaD9hTuCfsUkREREREpJYo7DUAQSSgoLiA99e8H3YpIiIiIiJSSxT2GoAzu55JSlKKhnKKiIiI\niDQgCnsNQIu0FgzpPERhT0RERESkAVHYayCCrICZ62eyY9+OsEsREREREZFaoLDXQASRgBJXwvTV\n08MuRUREREREaoHCXgMxrMsw0pLTNJRTRERERKSBUNhrIBqnNOaMrmco7ImIiIiINBBxDXtmNsrM\nPjOzpWZ21xGOO9XMiszsa/Gsp6ELsgLmfj6XLXu2hF2KiIiIiIjEWdzCnpklA48C5wA9gSvMrOdh\njrsfmBSvWsQLIgEAuStzwy1ERERERETiLp49e0OApc655c65AuBF4KIKjvs+8CqwKY61CDD4uME0\nT22uoZwiIiIiIg1APMNeZ2BNzO210X0HmFlnYDTweBzrkKhGyY0Y0W2Ewp6IiIiISAOQEvLzPwT8\nyDlXYmaHPcjMxgBjADIyMsjNza2d6o5Cfn5+nayrvG7F3ZiwdQKvTHyF9mntwy4nVInSZlJGbZZY\n1F6JR22WeNRmiUXtlXgSvc3iGfbWAV1ibmdG98UaDLwYDXrtgHPNrMg598/Yg5xz44HxAIMHD3bZ\n2dnxqvmY5ebmUhfrKq/VhlY8Pv5x9nfaT3bf7LDLCVWitJmUUZslFrVX4lGbJR61WWJReyWeRG+z\neA7jnAl0N7OImaUClwNvxh7gnIs457Kcc1nAP4Dvlg96UrP6dexHm8ZtNJRTRERERKSei1vPnnOu\nyMy+B0wEkoG/OucWmNmN0fvHxeu55fCSLImcSA7/XfFfnHMcafisiIiIiIgkrrjO2XPOTQAmlNtX\nYchzzl0bz1qkTJAV8NrC11ixfQXHtzk+7HJERERERCQO4npRdambSq+3p6GcIiIiIiL1l8JeA3Ry\nu5Pp2LwjU1ZOCbsUERERERGJE4W9BsjMCCIBk1dMxjkXdjkiIiIiIhIHCnsNVJAVsDF/I4u2LAq7\nFBERERERiQOFvQZK8/ZEREREROo3hb0GKtImQrdW3Zi8UmFPRERERKQ+UthrwIJIwJQVUyhxJWGX\nIiIiIiIiNUxhrwELIgFf7PuCuRvnhl2KiIiIiIjUMIW9BiwnKwfQvD0RERERkfpIYa8B69yyMz3S\ne2jenoiIiIhIPaSw18AFkYBpq6ZRWFwYdikiIiIiIlKDFPYauCASkF+Qz6z1s8IuRURERKR+2rED\nHniAfrfdBj/6EUydCoX6Q7vEn8JeTVi+nBaLFkFJ4q1qmZ2VDWjenoiIiEiNW70abr8dunSBO+8k\ndds2+MMfIDsb2rWDSy+Fv/0NNm4Mu1KppxT2asLjjzPoppv8L/JNN8G778L+/WFXVSXtmrajX0Y/\nzdsTERERqSmzZ8M3vgHHHw8PPwznnw+zZjHzqadg61Z47TX4+tfh/ffhW9+CTp1g8GD42c/gww+h\nuDjsVyD1hMJeTbjrLhb++McwdCg8+yyccw60bw+XXQbPPw/bt4dd4REFkYAZq2ewr2hf2KWIiIiI\nJKaSEvjXvyAnxwe3t9+GW26B5cv958FBg/xxLVvC6NHw5z/D2rXwySfw619D48b+67BhkJEBV10F\nL7zgw6HIMVLYqwnp6Xx+9tnw6quwZYv/5b78cj8e+8orffD70pfgT3/y3fl1TBAJ2F+8nw/WfBB2\nKSIiIiKJZd8+ePJJ6NXL9+AtWQJjx8KaNfD730PXrod/rBn07w933w3vvQebN/uAd+65MHGi7x3s\n0AHOOMMHwU8+Aedq77VJwlPYq2mNG8N558H48bB+ve+ev/12WLcOvv996NYNBg6EX/wC5s6tE7+w\nI7qNINmSNW9PREREpKq2bIF77/Wf7W64wX8GfO45WLEC7rgDWrU6+nO2bes7DJ55xs/j+/BD+H//\nDwoK/NeBA6FzZ/j2t30nw86dNf+6pF5R2IunpCTfFX/ffbBwISxaBPffD02a+LDXvz9EIr6Lf/Lk\n0FZlapnWksHHDda8PREREZHKLFkC3/2u77H72c/88Mz//AfmzPEjuho1qpnnSU6G007znxlnzvTh\n76mnYPhwH/S+9jVIT4cggAcegP/9r050IkjdorBXm3r0gP/7P5gxAzZs8GO1+/SBJ56As87y47O/\n+U34xz9g165aLS2IBHy87mN27a/d5xURERGp85zzn99Gj/af5/7yF7jiCpg/HyZM8J/jzOJbQ0YG\nXHMNvPSSH+45daofPbZlC9x5px9GGon4IPqvf8GePfGtRxKCwl5YMjLg+uvhrbfKVmW68EL/D8al\nl/rleEuHg27YEPdygkhAUUkR761+L+7PJSIiIpIQiov9H+GHDYMzz/QB6+67YdUqH/h69QqnrkaN\nYMQIP3ps3jy/JsQTT/hRY8884+cOtm3rFw384x9h2bJw6pTQKezVBc2a+b8UPfUUfP455ObCzTf7\nYZ/f+Q4cd5xf6fO3v/XDQePQRX96l9NJTU7VvD0RERGR/Hwfkrp393+E37zZL7S3Zg386lfQsWPY\nFR6sSxcYMwb++U/fiTBpkr8c2PLl8IMfwIkn+h7J227zQ04T5BJhUn0Ke3VNSgqMHOkvuLl0KXz6\nqZ/8W1zs/5LUs6f/Zb3zTr9qUw1dh6Vpo6YMyxymeXsiIiLScG3Y4D9vde3qQ1LHjn5+3OLF/g/x\nzZqFXWHl0tLgy1+GBx+Ezz7zcwwfecRf8++xx/x96elw8cV+BNmaNWFXLHGksFeXmUHv3n71pZkz\n/S/jY4/58dgPP+wn6Hbq5FdkevNN2Lu3Wk8XRAI+2fAJ2/Zuq6EXICIiIpIA5s+H667zK2ved5+/\nVt6MGX5V9Usu8YulJKoTT/Qrwr/zju/1e+stuPpqfxmH73zHB9u+feGuu2D6dCgqCrtiqUEKe4kk\nM9N3yU+c6IcTvPiiv37fP/4BF13k/0pTOhx0y5ajPn0QCXA4pq6cWvO1i4iIiNQlzvkhjaNG+QXz\nXnrJD4VcvNj35p1+etgV1rxmzfx8vsceg5UrYcECf03Adu38NQFHjPDfX3YZPP20n14kCS2uYc/M\nRpnZZ2a21MzuquD+i8xsnpnlmdksMzsznvXUK61a+V/E55/3wW/SJPjWt2DWLP+XqYwM/wv7hz9U\neVLukM5DaNqoqebtiYiI1GXO+Wv5/vvfNFu2TD0xR6ugAJ59FgYM8EMa8/L8PLw1a/y8vBNPDLvC\n2mHmpwfdcYe/BNiWLb4D4atfhWnT4Npr/TDWU0+Fn/8cPv4YSkrCrlqOUkq8TmxmycCjwJeBtcBM\nM3vTOfe/mMP+C7zpnHNm1hd4GTg5XjXVW6mp/h+rL3/ZTyaeMwfeeMNvt9/ut169fO/fRRfB4MH+\nGoDlT5OcyvCuwzVvT0REpK4oLPTzrubO9aEkL89/v3kzAKcCfO97PrgMHly29eiR2EMP42H7dj9H\n7ZFHYN06OOUUePJJf228xo3Dri58rVr5oPfVr/pQN3euXyV+wgQfhn/5S2jf3veEnnsunH22X/Gz\nviouhqIikvbtC7uSaolb2AOGAEudc8sBzOxF4CLgQNhzzuXHHN8M0JUgq8vMX9xz0CD/S7liRVnw\nu/9++M1v/OqeF17og19Ojp/IGxVEAn70nx+xMX8jHZvXsZWmRERE6rMdO/wH7Nhgt2BB2cqJaWl+\nLv8FF/gl9nv35n+TJ9Nzzx4/sudvf/N/9AU/XG/gwIMD4IknVvjH3npv1Sq/1sGf/+xX2QwCH/pG\njWqY70dVJCX5PyAMGAA/+Ymf6zdxYln4e/ZZf8zpp/vgd8450Lmz72UuLDz0a1X31aXjo6vfn5ae\nfkzTo+qKeIa9zkDs8j5rgdPKH2Rmo4HfAh2A8+JYT8MUicCtt/pt61b/C/rGG/6XdNw4aNHC/2N3\n8cVw7rkEkQCAKSumcEWfK0IuXkREpB5yzgeQ0l660mC3cmXZMe3b+0D3gx9Av37++x49/KrdMTaZ\n0TM7298oLvbzzWbNKtvGjStbwK1lS//H4NgAGInE/2LgYZk1y89De+UVf/uyy/xop4EDw60rEaWn\nwze+4bfiYr9w4IQJ/uLtd9/tt3gz89cXTEk5+GtF+2Lva9QImjY9+sdFv1+1cSMnxf/VxY25OFyz\nDcDMvgaMcs5dH739TeA059z3DnP8COBnzrkvVXDfGGAMQEZGxqAXX3wxLjVXR35+Ps2bNw+7jCpL\nKiig9ezZtJsxg3bvv0/qF19QkpzM9n59+U2nBWw940yuGfbTsMuMq0RrM1GbJRq1V+JRm9W8pIIC\nmq5cSfOlS2m+bNmBrym7dwPgzNibmUn+iSeSf8IJ/uuJJ1LQtm2VQlhlbWbFxTRdtYoWn312YGu+\nbBlJhYUAFLZowa4ePdh10kn+a48e7O/QIXEDYEkJ6R9+SJeXX6b13LkUNW3KhvPPZ+0ll7A/IyPs\n6url71jq1q20mT2blN27KUlOxiUn41JS/Fb6fXKyv6/cvqO5L6xe2LraZjk5ObOdc4MrOy6eYW8Y\ncI9z7ivR2z8GcM799giPWQ4Mcc4dtq908ODBbtasWTVdbrXl5uaSXfqXtURTUgIffVQ23HPRIr9/\nwAA/1HPUKP+XwJR4dgTXvoRuswZKbZZY1F6JR21WTZs3l/XUlX5duLDsmrjNmvkl7vv3L+ut6927\nWtduO6Y2Kyjww0NnzizrAfz007KFXtq3P7j3b/BgPwWkLtu3z49a+v3v/RzHLl38qKbrr/c9mnWE\nfscST11tMzOrUtiL56f3mUB3M4sA64DLgW/EHmBmJwLLogu0DATSgK1xrEkqkpQEw4b57b77eObl\nnzD/z7/hlzuTaPyLX8A99/hJu0HgF4H50pf8uP9E/aufiIhIdRQXw9Klhwa79evLjsnM9IHuoovK\nwt0JJ9SNOWKpqWXzscaM8fv27YN58w4eAjpxYtnqi506HRoAO3QI7zWU2rLFX0bgT3/yYXvgQPj7\n3+HSS/1QPJEGLm5hzzlXZGbfAyYCycBfnXMLzOzG6P3jgK8CV5tZIbAXuMzFq6tRqmxg9hVcs/A3\nnHLhzVyXeb5fjvff//bb66/7g7p186Hvy1/2IbB9+3CLFhERiYf8fN/rFRvs5s2DPXv8/Skpfvn6\ns87yoa402KWnh1v30WrcGIYM8VupPXv8a44NgG+/fWDhCrp0OTj8DRpUe6978WJ48EF/beF9++C8\n8/x8vOxs/TFaJEZcx+U55yYAE8rtGxfz/f3A/fGsQY5er/a9aN+0PZNXTua6Adf5Cc2XXeb/cV+6\n1F+A9N//9tdi+ctf/IMGDCgLf2eeCU2ahPsiRETiwTnYudMPu6tnQ9sbvNJr18X21M2dC0uWlIWb\n1q19mLvhhrJgd8opB61qXa80bepXW4y9uPiuXfDJJwcHwNI/BINf8KV8AGzVqmbqcQ5mzIAHHoA3\n3/Q9d1dfDT/8oQ/cInII/U8lhzAzgkjA5BWTcc5hpX8hM4Pu3f12001+bP/s2WXh76GHYOxY/5/e\nmWeWDfkcMKBuDFsREalMYaG//taqVbB6ddkWe3v3bv9vWseOvmcjM7Pir506KRDWVYWFfn56+dUw\nt8bMJDn+eB/mrryyLNh16aJeoxYtYMQIv5Xavt1f4zc2AJaugAlw0kkHB8ABA+BoFrwoKvKB8oEH\n/IW927b1lwO4+Wb/eygih6X/haRCQSTgpQUvsXjrYnq061HxQSkpcNppfvvJT/wHoGnTysLfXXf5\n49LTD57vF4nU3gsRESnlnL+O2ZGC3Pr1Zb04pdq390PXTzkFvvIVfy2pXbtgzRpYu9YvdPHuu/7f\nwFhJST7wHS4MZmYqEMZTSYkPIVu2+AA/b15ZsFuwwC9SAn74Yu/eMHp02aIpffvWqUU96rzWrf3/\n80FQtm/rVv8H4dJFYKZNg+ef9/eZ+d+n2ADYr5/vSYyVnw9//asfrrlypV8v4NFH4ZprqrWojUhD\nov9hpEKl19ubvGLy4cNeec2a+YtqnnOOv71xI/z3v2Xhr/SvfCecUDbkMyfH/4VORKS6ior8h/rD\nBbnVq31Ii5Wa6oNXt27+36SuXf3WrZv/2qVL1YallwbJtWvLQmDs1/nz4Z13FAiPVUkJfPGFDxBb\ntvitsu+3bStbXKRUhw4+zN16a9ncupNO0vsbD+npcPbZfiu1caMPgLELwDzzjL8vORl69SoLf6tX\n+2sEbt/uh5H+4Q9w4YX+OBGpMv3rJhU6oc0JdGnZhckrJ3PTqTcd20k6dvTDX6680n8Q+uyzsoVe\nnn8ennjC/3Vv8OCy8Hf66fV37oOIVM+OHQcHt/Jhbt26Qz/cp6f70Na9u19Ao3yY69ChZoaZm/ne\njdatfS9RRUoDYWkIPNpAeKQhox07Jk5gKS4u63GrSmjburXi4FYqNRXatfNbejr06VP2fen+Dh38\nfg35C1fHjn4hlfPO87dL50nGDv98803fm5eU5Htbb7/drxYuIsckQf5nkNpWOm/v7cVvU+JKSLJq\nfhgyg5NP9tv3v+/nS3z8cVmv3+9+B7/9rf8L+ogRZUM++/TRfD9JXIWFsGGDDyGH29avh/37fc94\nPLYmTRJjjlFRkX+vjtQrt2PHwY9JSSnrlcvJKQtypWGuS5e6NdQrNhD26VPxMeUDYfmvn34KEyaU\nrQRZKinJXwftSD2E8QiExcXH1uN2uIW3Y4Nbu3Z+OGVskKvo+2bNEuNnXA5l5odFd+7sL1EB/mdj\n1Sr/s5qZGW59IvWAwp4cVhAJeHru0zz4wYMM7zacnu170jz1KCZUH0mjRnDGGX77+c/96nZTp5aF\nvzvu8Md16OD/Gl8a/rp0qZnnF6mO0g/kRwpx69bBpk2HfqhNTfUfyjt39teDuuACH8h27z5027XL\nD3sqv/9orlBj5ufBHCkQVnb/kbaqDqnatevIQW7t2rILT5dq08YHt0gERo48tFcuI6P+DemqaiDc\nvr3i3sG1a/3ctH/969BAmJxc6ZDRRjt2+IVLjqbH7XA/j2lpB4ezfv2OHNrS0xXcxLd/VlbYVYjU\nGwp7clhnn3A2bZu05Y5/33FgX6R1hN4deh+09UjvQVpKNYdetmzpP/RecIG/vW6dD36l2wsv+P09\nepQN+czOrrnlnEVKFRb6gFVZkCv/QRr8h9XOnX2YGzCg7C/WsVu7dtX7MOucv6ZUReHwaLdt2w7d\nV1R0dPWkpR02CPb54gvYu9cHu+3bD35ccrIPGd26wfDhFffKtWhx7O9TfWbmg3CbNlULhBX1Eh4m\nEJ5xuOdMS/ML1ZSGswEDyr4/XHhr2lTBTUQkZAp7clgdm3dk0x2bWP7FcuZvms/8TfNZsHkB8zfN\n552l71BU4j8UJlsy3dO7+/DXviwEntD2BFKSjvFHrHNnv9rWNdf4Dy0LFvgev//8x19A9dFH/YfF\nIUPKwt/Qob7HUKQipddHqyzEff75kXvjBgyA888/NMQdd5xf1S/ezHxPYJMm/gN1TSsoqJkguWkT\nqTt2+KHbZ5xxaK9cp071r1euLjmaQBgTApcsWED3YcMODXAKbiIiCUlhT44oOckHue7p3Rl9yugD\n+wuKC1i8dfFBITBvYx6v/u9VHP6DclpyGie3O/mQnsCurboe3RxAM7/gQe/e/sKpBQXw4Ydl4e/X\nv4Z77/XX7Bk5smzIZ8+e+nDSUBQVVa03rvzCF+BXgy0NbP37x6c3LpGkpvqtTZtqn2p2bi7Z2dnV\nr0niIzYQ9u0LwLrcXLqrzURE6g2FPTkmqcmpB8JbrD2Fe1i4eeFBIXDaqmn8/dO/HzimeWpzerXv\nRe8OvQ987d2hNx2bdyy7gPsRnzy17IKu997r/zKdm1sW/v71L39cp05lvX5nneV7XiQxFBX5OV47\nd/qv0e87TpkCM2YcfW9c//5+9beweuNEREREQqCwJzWqaaOmDDpuEIOOG3TQ/h37dhwYAlq6vbX4\nLf7yyV8OHNO2SduDhoL26uCDYNsmlVyHr3VruPhiv4Ff7KF0oZd33oFnn/X7e/UqC38jRtTkyxbn\nyrEb6lAAABOQSURBVBYUqcoWG+Iq2vbtq/BpTi79Rr1xIiIiIpVS2JNa0apxK07vcjqndzn9oP2b\ndm9iwaaYELh5Ps99+hw79+88cEyn5p0OGQp6xJVBu3aFb33LbyUlfiGC0vD3xBPw8MOQksJp7dv7\nBV5SU/3iA2F+DSOUFBRUP5SVbvn5h78GViwzP9y2RYuDt27d/CI95feX2z5cuZKhl1xStYtci4iI\niDRwCnsSqg7NOtAh0oGcSM6Bfc451u1ad1Av4PxN8xk3axx7i/YeOC6rddYhi8L0aNeDxikxw/KS\nknzPT//+/nIO+/bB++/Df/7DjlmzaNKmjb/GWUFB2ddduw6+/f/bu/cYuc7yjuO/Z647uzO7viSx\nY6+JIYkAk3Jx3YQAap1QKlMQrtSKSylKWxAClUsrVAitxF9VhdqqBUpalKZcJAJRRQNF5RJCiNtI\n3AKEOIlDiEkMu767Xu99Z+fy9I85Ozuzc9/d8dmZ/X6k0TnnPe+cfcavvd7fvudSb9nJre/bEY/X\nhsDVBsdIpL0Qt7jYXm3JZG3w2r69dGvsTKatkFbuMzi4pucmLuRyBD0AAIA2Efaw4ZiZRodHNTo8\nqkPXHSq3F4oFnbh0omoW8PFzj+u+4/cpV8xJWr4zaOW1gDdcdYOu23Zd6c6gAwPSrbdKt96qnx05\nop2ruRGBe+l5YM3CYLeWU1PN92ezpRm2eoHriitqw1ergJZOlwIkAAAAeg5hDz0jGonq2m3X6tpt\n1+rwCw6X23OFnJ6++HTVLODRs0d175P3lu8Mmogm9MIrXlh1U5iJuQnNLM50/qB4MykWK72Ghtbz\nI64Pd65VAwAAAGEPvS8ejWvflfu078p9euOL3lhun8/N68kLTy5fE3j+cT30q4eq7gx628O3acvA\nlvJM4mhmVHtG9pS39wyX1jPJHnq4M0EPAAAAIuyhj6XiKe2/er/2X72/qn1yYVLHzh/T1777NWV2\nZzQ+Na7x6XGNT43rkdOP6Ozs2ZpjDSeHq8LfyjA4OjyqkYGRy/XRAAAAgJYIe9h0RgZGdPOem5Xd\nkdXBVx2s2b9YWNSp6VManxrX2ORYKQxOjWtsqrR+9OxRnZk5Uz5FdEkmkWkYBEeHSzOGI8mR9p4l\nCAAAAKwRYQ9YIRFNaO+Wvdq7ZW/DPrlCrhwIV4bB8alx3feL+3R6+nRNIByKD1WFv9FMdRgcHR7V\n1oGtBEIAAACsGWEPWIV4NK5rtlyja7Zc07BPrpDTmZkzNUFwafvbz3xbp6ZPqejVz6cbjA9WzQiu\nvI5wdHhU21PbCYQAAABoirAHdEk8GteekT3aM7JHN+vmun3yxXw5EJaD4ORY+RrCIyeO6OTUSRW8\nUPW+gdhAw1NGr05frUwyo0wio3QirXQirWgkejk+MgAAADYQwh4QolgkVg5pjRSKBZ2dPVtzDeH4\ndGn7oV8+pJPTJ5Uv5hseIxVLKZ1IK5MsBcDKIJhJZpSOL++ru39F22B8kJlFAACADY6wB2xw0UhU\nuzK7tCuzSzfuvrFun6IXdW72nMYmx3Rm5oxmFmc0vTitmcWZ0np2uqZtYmFCY1NjVfuaBcZKJqsK\ngjUBMVivt79RWyLKw9sBAADWE2EP6AMRi2hneqd2pneu6TjZfHY5IAbBcCkMthMgT8+crtm/8iY1\njcQj8eYBMZ7W1Lkp/SD2A20Z2KKtqa2l5UBpufSKR+Nr+jMAAADoF10Ne2Z2SNLHJUUl3eXuH12x\n/62SPiTJJE1Lere7P9rNmgA0lowllYwltX1w+7ocz901l5trOyxOZ6c1k6vef37ufHl7Yn5CXxj7\nQtOvORQfqhsEl9aX9tVryyQynJ4KAAD6RtfCnplFJd0h6TWSxiU9bGZfdfdjFd2elfRb7j5hZq+V\ndKekm7pVE4DLy8w0lBjSUGJIO7Rjzcd78MEHddOrbtKlhUuamJ/QpYVLpfWFiaq2pe1LC5c0NjWm\nx849pon5CU1mJ5seP2KR+kEwuaXhbGJlWzKWXPNnBAAAWC/dnNm7UdJxd39GkszsHkmHJZXDnrt/\nt6L/9yU1vksFgE3PzDQYH9RgfFC7Mrs6fn+hWND04nTdULiybWl5+vzp8r75/HzT4w/EBhoGwWYz\njCPJEQ0lhpSMJplZBAAA66abYW+3pLGK7XE1n7V7u6RvdLEeAJtcNBItB6zVyOazDWcT6wXFszNn\n9dSFp8r7Vz5Co6Y+i5ZmQuNDSifSGkoEy8rteIP2JtupeEoRi6zqMwMAgN5l7u3dPKHjA5v9gaRD\n7v6OYPttkm5y9/fU6XuLpH+R9Cp3/786+98p6Z2StGPHjl+/5557ulLzWszMzCidToddBjrAmPWe\nXh4zd9d8YV7T+WnN5Gc0k58pr88V5rRQWNB8Yb68nC9WrFe0LxRLy8XiYkdffyAyoFQ0pVQ0pYFo\naX2pbSA6ULdtqX+99qVl1Bo/w7GXx2uzYsx6D2PWWxiv3rNRx+yWW275sbsfaNWvmzN7JyXtqdge\nDdqqmNmLJd0l6bX1gp4kufudKl3PpwMHDvjBgwfXvdi1OnLkiDZiXWiMMes9jNmyQrGg2dysZhdn\nNbM4o9lcsGy1vaJ9KjelU4unNDu7vL8TyWiy4ezi7MSsnjf6PA3Fh2pmLJfalt6zcn8qluKU1hDw\nb6z3MGa9hfHqPb0+Zt0Mew9Lut7MnqtSyHuzpD+s7GBmz5F0r6S3ufvPu1gLAPSVaCSq4eSwhpPD\n63rcohc1n5tvPzw2CJMnp07q/Mx5PXX8qXKfVqexVjKVrs9sKxx2GCYT0QRBEgCwKXQt7Ll73sze\nI+k+lR698Gl3f8LM3hXs/5Skj0jaLulfgv948+1MRwIAuiNikfIdVK8aumpNx6r8bai7a7GwWDMb\nObs4WxUcG+2v7HNu9lxNn3af5yjVvzaykwA5FB9SPBqXyRSxiMyC5Yrtem1L283e106f1bwPALD5\ndPU5e+7+dUlfX9H2qYr1d0h6RzdrAACEz8zKz3Hcltq2rsd2dy3kFzoOkCu3JxcmdXLqZFXbXG5u\nXWsNk8lahkUrmoYfGdZgfFCpWKp899tUvGI91mC9UZ8V7fFoPOw/CgDYNLoa9gAA6DYzUyqeUiqe\n0pW6cl2PXfSi5nJzNQGy4AUVvSh3Ly3lVdv12jrp0+x97fRZ7dd/duxZbbtqm+bz85rLzWkuN6f5\n3Lwuzl8sra9o72RGdUksEmsrKK4mSFa2D8QGuAstgE2PsAcAQAMRiyidSCud2Hh3YuuGTm5E4O7K\nFrLl4FcOgUEgrNfWtD1fCpX19i8WOrv77JJULFUOhEvXuJZfieHatopXJplZXk9kmJEE0JMIewAA\noGNmpoHYgAZiA1Kqu1+rUCy0DIzNwuVsblbTi9Oayk5pKjul8anx8vp0drqtGcpULNU0HFYFxUSm\n4b6B2ADXUAK4bAh7AABgQ4tGol2bYS16UbOLs+Xw1+hVGRaXXicunSivT2YnlS/mW369WCS2qpC4\n8pVOpDlNFUBLhD0AALBpRSyiTDKjTDKj3dq96uMsndbaMjRmg9C4uNx2bvacjl88Xt5u98ZA6URa\nkWJEqR+lFIvE1uUVtei6Havm2JH2j52IJsqn4RJqgdUj7AEAAKxR5Wmta31sSb6YXw6FTWYZJxcm\n9ezYs9qxc4cKXlC+mG/5WsgvtNWv8lV57DAko8nytZflG/jEU1XrS/vqtrX73ngpNAP9hL/RAAAA\nG0gsEtPW1FZtTW1t2beTm+qs1dJdWzsNi+2EyJWvbD5bdT1meX1F2/m58+XrNefz8+X1XDG3qs8Y\nj8TbD4pthMeV/S5kL+jszNnyI1AqX5WPRqlqDx6bAqwGYQ8AAAAtmZmiFlU0ElVSybDLaSpfzJcD\nYWUYrAmPdYJiZbCsbJtYmNCp6VM1780Wsp0V9/3VfaZ6IbCT0Lia93TSPx6JKxFNKBlLKhEpLZPR\n5HJbNKFkNFm1vrSvXr9m+2KRGAG4TYQ9AAAA9JVYJFa+FrPbil7UQn6hdXjMzevRY4/quuuvq3re\nZeWr8jmaVe0d9q95j1Z/rJX988V8zXsKxdIsbbaQVTaf1WJhUdlCsMxnOw/ELZisrRDZcF8H/X9x\n8Rc6qIPrWv/lRNgDAAAAViliEQ3GBzUYH2zZ98ilIzr4Gwe7X9QG4+7lMFgZAJfW64XDlfta9ivW\n9s/mSzdNavU1m9mW2KYP6oOX6U9q/RH2AAAAAHSNmSkejSsejYddSg13V66Yaxgmf/jwD8MucU0I\newAAAAA2JbPSKaGJaKLu/ovpi5e5ovXFg0sAAAAAoA8R9gAAAACgDxH2AAAAAKAPEfYAAAAAoA8R\n9gAAAACgDxH2AAAAAKAPEfYAAAAAoA8R9gAAAACgDxH2AAAAAKAPEfYAAAAAoA+Zu4ddQ0fM7Lyk\nX4ZdRx1XSLoQdhHoCGPWexiz3sJ49R7GrPcwZr2F8eo9G3XMrnH3K1t16rmwt1GZ2Y/c/UDYdaB9\njFnvYcx6C+PVexiz3sOY9RbGq/f0+phxGicAAAAA9CHCHgAAAAD0IcLe+rkz7ALQMcas9zBmvYXx\n6j2MWe9hzHoL49V7enrMuGYPAAAAAPoQM3sAAAAA0IcIe+vAzA6Z2VNmdtzMbg+7HjRnZnvM7EEz\nO2ZmT5jZ+8OuCa2ZWdTMHjGz/w67FrRmZlvM7Etm9jMze9LMbg67JjRmZn8RfD983My+aGYDYdeE\nWmb2aTM7Z2aPV7RtM7P7zezpYLk1zBqxrMF4/X3wffGomX3ZzLaEWSOq1Ruzin0fMDM3syvCqG21\nCHtrZGZRSXdIeq2kfZLeYmb7wq0KLeQlfcDd90l6uaQ/Y8x6wvslPRl2EWjbxyV9091fIOklYuw2\nLDPbLel9kg64+w2SopLeHG5VaOCzkg6taLtd0gPufr2kB4JtbAyfVe143S/pBnd/saSfS/rw5S4K\nTX1WtWMmM9sj6Xck/epyF7RWhL21u1HScXd/xt0XJd0j6XDINaEJdz/t7j8J1qdV+iF0d7hVoRkz\nG5X0Okl3hV0LWjOzEUm/KenfJcndF939UrhVoYWYpJSZxSQNSjoVcj2ow93/V9LFFc2HJX0uWP+c\npN+7rEWhoXrj5e7fcvd8sPl9SaOXvTA01ODfmCT9k6QPSuq5m50Q9tZut6Sxiu1xERx6hpntlfQy\nST8ItxK08DGVvskWwy4EbXmupPOSPhOcenuXmQ2FXRTqc/eTkv5Bpd9Yn5Y06e7fCrcqdGCHu58O\n1s9I2hFmMejIn0r6RthFoDkzOyzppLs/GnYtq0HYw6ZlZmlJ/ynpz919Kux6UJ+ZvV7SOXf/cdi1\noG0xSfsl/au7v0zSrDi1bMMKrvE6rFJI3yVpyMz+KNyqsBpeusV6z808bEZm9tcqXVZyd9i1oDEz\nG5T0V5I+EnYtq0XYW7uTkvZUbI8GbdjAzCyuUtC7293vDbseNPVKSW8wsxMqnSZ9q5l9PtyS0MK4\npHF3X5ox/5JK4Q8b029Letbdz7t7TtK9kl4Rck1o31kzu1qSguW5kOtBC2b2x5JeL+mtzjPQNrpr\nVfpF2KPBzyGjkn5iZjtDraoDhL21e1jS9Wb2XDNLqHRR+1dDrglNmJmpdC3Rk+7+j2HXg+bc/cPu\nPurue1X69/Udd2fWYQNz9zOSxszs+UHTqyUdC7EkNPcrSS83s8Hg++OrxQ11eslXJd0WrN8m6b9C\nrAUtmNkhlS5LeIO7z4VdD5pz98fc/Sp33xv8HDIuaX/w/1xPIOytUXCR7Xsk3afSf47/4e5PhFsV\nWnilpLepNEP00+D1u2EXBfSZ90q628yOSnqppL8NuR40EMzAfknSTyQ9ptLPBneGWhTqMrMvSvqe\npOeb2biZvV3SRyW9xsyeVmmW9qNh1ohlDcbrk5Iyku4Pfv74VKhFokqDMetpxuwxAAAAAPQfZvYA\nAAAAoA8R9gAAAACgDxH2AAAAAKAPEfYAAAAAoA8R9gAAAACgDxH2AACbipkVKh678lMzu30dj73X\nzB5fr+MBALAWsbALAADgMpt395eGXQQAAN3GzB4AAJLM7ISZ/Z2ZPWZmPzSz64L2vWb2HTM7amYP\nmNlzgvYdZvZlM3s0eL0iOFTUzP7NzJ4ws2+ZWSro/z4zOxYc556QPiYAYBMh7AEANpvUitM431Sx\nb9Ldf03SJyV9LGj7Z0mfc/cXS7pb0ieC9k9I+h93f4mk/ZKeCNqvl3SHu79I0iVJvx+03y7pZcFx\n3tWtDwcAwBJz97BrAADgsjGzGXdP12k/IelWd3/GzOKSzrj7djO7IOlqd88F7afd/QozOy9p1N2z\nFcfYK+l+d78+2P6QpLi7/42ZfVPSjKSvSPqKu890+aMCADY5ZvYAAFjmDdY7ka1YL2j5+vjXSbpD\npVnAh82M6+YBAF1F2AMAYNmbKpbfC9a/K+nNwfpbJT0UrD8g6d2SZGZRMxtpdFAzi0ja4+4PSvqQ\npBFJNbOLAACsJ36rCADYbFJm9tOK7W+6+9LjF7aa2VGVZufeErS9V9JnzOwvJZ2X9CdB+/sl3Wlm\nb1dpBu/dkk43+JpRSZ8PAqFJ+oS7X1q3TwQAQB1cswcAgMrX7B1w9wth1wIAwHrgNE4AAAAA6EPM\n7AEAAABAH2JmDwAAAAD6EGEPAAAAAPoQYQ8AAAAA+hBhDwAAAAD6EGEPAAAAAPoQYQ8AAAAA+tD/\nA+eq6Y9ASgToAAAAAElFTkSuQmCC\n", 1172 | "text/plain": [ 1173 | "" 1174 | ] 1175 | }, 1176 | "metadata": {}, 1177 | "output_type": "display_data" 1178 | } 1179 | ], 1180 | "source": [ 1181 | "fig = plt.figure(figsize=(15, 5))\n", 1182 | "plt.plot(fitted_model.history['loss'], 'g', label=\"train losses\")\n", 1183 | "plt.plot(fitted_model.history['val_loss'], 'r', label=\"val losses\")\n", 1184 | "plt.grid(True)\n", 1185 | "plt.title('Training loss vs. Validation loss - VGG16')\n", 1186 | "plt.xlabel('Epochs')\n", 1187 | "plt.ylabel('Loss')\n", 1188 | "plt.legend()" 1189 | ] 1190 | }, 1191 | { 1192 | "cell_type": "code", 1193 | "execution_count": 19, 1194 | "metadata": { 1195 | "collapsed": false, 1196 | "scrolled": false 1197 | }, 1198 | "outputs": [ 1199 | { 1200 | "data": { 1201 | "text/plain": [ 1202 | "" 1203 | ] 1204 | }, 1205 | "execution_count": 19, 1206 | "metadata": {}, 1207 | "output_type": "execute_result" 1208 | }, 1209 | { 1210 | "data": { 1211 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAFNCAYAAABVKNEpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8lNW9x/HPyb6SkIVAEkJYw05CQoKyCFoQFbCyuKAI\nKHoRsGirVrm16tW2aC0uiFJqC65VL8rVWmsrYAQUwYCRNSxCgBCEsGUl65z7x5lMZrIDCZNJfu/X\n63kx88yznGfOAPOdc55zlNYaIYQQQgghhBBth5uzCyCEEEIIIYQQ4vKSICiEEEIIIYQQbYwEQSGE\nEEIIIYRoYyQICiGEEEIIIUQbI0FQCCGEEEIIIdoYCYJCCCGEEEII0cZIEBRCiIuklHJXShUopWKa\nclvhWpRSG5VSM62PZyil/tWYbS/iPN2UUgUXV0ohhBDCkQRBIUSbYQ1ilYtFKXXe7vntF3o8rXWF\n1jpAa32kKbe9WEqp2UoprZSa3FznaG2UUncopX6sZb2XUuqUUmrchRxPa/2G1vq6JipbllJqlN2x\nD2qtA5ri2HWcz00pdVgptb25ztHSKaXWKKV+W8v6yUqpY0opd+vzZKXUZ0qpc9Zll1LqaaVUsN0+\nkUqpvyqlsq3/xhxUSv1NKRVnt81flVL7rP8e3VHLeXtYz5Nv/Tz+obmuXQjR9kgQFEK0GdYgFmD9\nMn0EmGC37p3q2yulPC5/KS/JDOAMcOflPnHlF2QX9BEQrpQaXm399UAp8MXlL5LTjAZCgN5KqYTL\neeIW9HftDWB6LeunA29rrSuUUiOAdUAq0EtrHYz5vAD0B1BKhQPfAl7AcCAQSAS+Bn5md9zvgTnA\nD9VPqJTyxnz+/g1EAJ2Bdy/t8oQQoooEQSGEsFJKPaOUel8p9XelVD5wh1LqCqXUt9Zf/Y8rpV5W\nSnlat/ewtsDFWp+/bX39X9Zf8Dcppbpe6LbW16+zthTkKqWWKKW+rq9LoVKqOzAMuBe4zvpF1P71\nSUqpdKVUnlLqgFJqrHV9qFJqpfXaziqlPrSun62USrXbv7byL1VKfa6UKgRGKKUm2p3jiFLq8Wpl\nGGl9L3OVUkeVUtOt72+2UsrNbrublVJba7nGYdZWGfttpyqltlkfD1VKbbOe/4RS6o91vV+VtNZF\nwCpqhuc7gXesX/xDra0yOdb36B9KqajajlfL+zZOKbXXes0vAcrutZ5KqS+VUmesrT1vKaWCrK/9\nHYgE/mVtTfqltXVI2+0frZT61Lr/fqXUXXavPWP9HL9t/XztVEoNbuDtmIEJxp9bH9tfV62fE+tr\ndX22HFo0rWVaaX3cw/p5mqWUOgL8R5kWyVVKqZ+U+fuWqpTqY7e/n1LqBetnK1cptV4p5a2U+rdS\n6r5q5d2tlJrQwPXW5iOgo1LqSvtrxwS9N62r/gj8RWv9nNb6JIDW+rDW+nGt9UbrNr8CcoA7rS25\nWmt9Vmv9V6310spja61f0VqvA0pqKcvdQKbW+iWtdZHW+rzWesdFXJMQQtRKgqAQQji6CfOrexDw\nPlAOLADCMEFrHPBf9ew/DXgc07JyBHj6QrdVSnUAPgAetp73EJDcQLnvBL7VWn8I/Gg9NtbjXQn8\nDfPlNBjT8nPY+vK7mFaLvkAH4KUGzlO9/E9hWjs2AQXA7dZzTAAWKKXGW8vQFfgMWAyEAgnADq31\nJiAfuMbuuNOp+tJt7xugDLiqWhkqW0mWAH/UWrcDemACXmO8AUxVSvlYyxoC3GBdD+b/yr8AMUAX\naxkafJ+s9bgKeBRTj1lAiv0mwDNAR8z73w3zeUBrfRuQDVxnbbFeXMsp3sd8NiKBW4DnlFL2783P\ngbcw9fEv4OV6yhoATALesS7TlGMrXa2fkwY+W40xEuiNeb8BPgV6Yt6TndbyV3oBGIh5D0OAhYAF\nU0+2bpVKqUTM+13nvZp10VoXUvOHgVuB7VrrXUqpdpi/ix/Wtr+dnwGrtda6ge3qMxQ4Yg26p5RS\n65RS/S7heEII4UCCoBBCONqotf6H1tpi/QX+O631Zq11udb6ILAcxyBS3SqtdZrWugzzhTr+IrYd\nD6RrrT+2vvYCcKqugyilFOaLa2UgehfHL7J3Y1ow1lqv66jWeq9SqjMmgN1nba0o01qvr6e81a3W\nWm+yHrNEa71Oa73L+vwH4D2q3qs7gH9prT+wvpentNbp1tfetL6OUirMWqa/Vz+Z9Uv1e8Bt1m2D\ngWut68AEtJ5KqVCtdb7WenMjr2M9cA6YaH1+C7BTa73Tet4crfVq6+chD/g99X8GKlXW42prPf4J\n00pUeT37rHVSam1ZeqGRx60M1snAo1rrYq31NmAFjt0av9Ja/1trXYEJVPV9Fqdggvxa4BPAF7jO\neq76Pie1frYacw1WT9i1dlm01iutdVcMPAkkKqX8lel6PBP4hdb6uPWe243W93U10E9VtahPB97T\nWpdfQDnsvQHcrEzXTDB/lyp/FAjBBPifKjdWSi22tmAWKqUeta4Oq7bNJOs2+UqpzxpZjmjMZ/1P\nmLD/BfCxsvZIEEKISyVBUAghHB21f6KU6q2U+qe1u1oe8D+YL3l1+cnucRFQ3+AedW0baV8OawDK\nquc4IzFfGt+3Pn8XGKyU6m993hnTSlhdZ+CU1jq3nmPXp/p7dYW1O1+OUioXmE3Ve1VXGcCElBuV\nUr6Y1pcvK7vc1eJdYLL1y/BkYLPWuvK9mYVpsdqrlNqilLq+jmM4sL6/b1IVnh1aJJVSAUqp161d\nEvMw94fV9xmoVL0eLdjVo1Kqo1LqA2W6u+YBKxt53Mpjn7K2YFU6DNh3Wa3++fKv53gzgPetAes8\nJlxVdg+t73NSX702hu39UWZk3eeUGVQlDzhgfSkMc4+cV23nspZ3FaYrtzvmM/RW9e2s53hdVQ0Q\n9UgdZfoKyAMmKKV6YVqvK3+YOANooJPd+X9pvU/wH0BlK+rpatt8ZN3mYet1NMZ5TJj/j9a6FHjW\nesxejdxfCCHqJUFQCCEcVe/K9WdMF7Ue1i6Hv8XuPq9mchwT7ABbi1+t96RZzcD8e75DKfUTZkAK\nTdUX+aNA91r2OwqEWbu7VVcI+Nk971jLNtXfq/cwXeY6a62DgNepeq/qKgPajKS6FdOVcTp1fIm3\nbrsdE3CuxbFbKFrrvVrrWzFdF/8EfFjZ3bMR3gTGWrs6JuE4KMfDQFcg2foZuLqRxzyOCUqAGZUT\nu3rFfLEvAQZYjzsTx89Wfd0KszF1Zx/uYoBjjSybjVKqC6Ylcqb1B4+fMHUxQSnVnvo/J3XWK434\nDFXrOnkn5l68qzFds3tUFhE4gRm8p65zvYHpljwWOKu1/q62jbTWs3XVAFHP1bGN/Q8D04HPtNan\nrK/lAWmYbrT1WQvcZP27e7G24/gZ0NT/mRBCiAsiQVAIIeoXCOQChdaBK+q7P7CpfIpp0ZtgvU9r\nARBe24ZKKT9Mt767MV3/KpcHgdutLSR/BWYrpUZbB+SIVkrFaa2PAmuApUqpYKWUp1JqpPXQPwAD\nlVIDrC11TzSi3IHAGa11sVJqKKZlptLbwDhlhuH3UEqFKaUG2b3+JvAY5n6xjxs4z7vW67sCu/sA\nlRl8Jsza8paL+dJsaUS50Vr/CGy2HvtfWuscu5cDMS1qZ60Dh9SYXqAOnwLxSqkbrS2YD+JYj4GY\nsJRr7X75ULX9T2DuG6ytvIcwgeT31gFT4jEtom83smz27gR2A3FUfX7iMIH71gY+J7V+tqyvpQO3\nWus7mYbDUyAmGJ/GBMjf2V1vBabF9EVrS6q7MoMHVXaT3IhpaXuWen5IuABvYu4HvouqbqGVHgbu\nVUo9oqyDMlnrr4vdNs9jfpB4U5n5H5U1SNt/5iunKfHBhF1PpZSPXXh8CxiulLra+vf4IcwPABfS\n9VYIIeokQVAIIer3K0zLWj6mdfD9+je/dFrrE5j71BZjvhR3xwwzX9vIgpOsZXtba/1T5YIZ3MQX\nGKO1/ga4BzNYSC7wJVUtVZWDbOzDBI/7rWXYjbkXLhXzxbMx9w7eB/xBmRFXF2IGvKm8pkOYAWR+\njeletw0YYLfvh5jQs8ra1a8+72Jajb7QWp+1W389sMd6/ueBW7TWpdbQUKCUuqKB476B+TJffaCa\nxZgWqtOYAWsaNQiJXT3+EXOPZwwmbFZ6AnOfXy7mvrzqA5D8HnjKem/ZA7Wc4hbMwCo/YQLxQq11\namPKVs2dwFL7z4/W+jjm817ZqlzX56S+z9Z/Y4L9OcwgOA1NfbACE3SygV2Y99reg8AeTOvxGcz7\no6zlqGzF64+53/aSaK0PAFsAb+Cf1V77CjMYzGjggFLqHOYzsQZ41brNScxgL+XW68jHfOZ9gHl2\nh1uH6QKajBl05zxmUKrKv4MzMC3rZzGf759fwr2PQgjhQOlLGtBKCCFEc7O2BmQDU7TWG5xdnuZg\nbQU5BMy8yDAj2jhlps+4U2s9ytllEUIIVyAtgkII0QIpM/9csDIjFz6OGRFzi5OL1ZxuxrR4fuXs\nggjXY71Xci5mVF8hhBCNIEFQCCFapuHAQcx0A9cCN2mta+sa6vKUUhsxXQvnaemmIi6QUuoG4CRm\nLs5m77othBCthXQNFUIIIYQQQog2RloEhRBCCCGEEKKNkSAohBBCCCGEEG2Mh7ML0JTCwsJ0bGys\ns4tRQ2FhIf7+/g1vKFoEqS/XI3XmeqTOXIvUl+uROnM9Umeup6XW2datW09prWudf9heqwqCsbGx\npKWlObsYNaSmpjJq1ChnF0M0ktSX65E6cz1SZ65F6sv1SJ25Hqkz19NS60wpdbgx20nXUCGEEEII\nIYRoYyQICiGEEEIIIUQbI0FQCCGEEEIIIdqYVnWPYG3KysrIysqiuLjYaWUICgpiz549Tju/uDDO\nrC8fHx+io6Px9PR0yvmFEEIIIUTb0OqDYFZWFoGBgcTGxqKUckoZ8vPzCQwMdMq5xYVzVn1prTl9\n+jRZWVl07dr1sp9fCCGEEEK0Ha2+a2hxcTGhoaFOC4FCNJZSitDQUKe2XgshhBBCiLah1QdBQEKg\ncBnyWRVCCCGEEJdDmwiCwnnS09P57LPPLni/7OxspkyZ0gwlMlJTU/nmm2+a7fhCCCGEEEK0ZBIE\nW5Hy8nJnF6GG+oJgfeWNjIxk1apVzVUsCYJCCCGEEKJNkyB4Gdx2220kJibSr18/li9fblv/+eef\nM3jwYAYNGsQ111wDQEFBAbNmzWLAgAEMHDiQDz/8EICAgADbfqtWrWLmzJkAzJw5kzlz5pCSksIj\njzzCli1buOKKK0hISODKK69k7969AFRUVPDQQw/Rv39/Bg4cyJIlS1i3bh0///nPbcf94osvuOmm\nm2qUf+3atSQkJDBgwADuuusuSkpKAIiNjeWJJ55g8ODBDBgwgIyMDIf9SktL+e1vf8v7779PfHw8\n77//Pk8++STTp09n2LBhTJ8+nczMTEaMGMHgwYMZPHiwLZxlZmbSv39/AFauXMmkSZMYN24cPXv2\n5JFHHqn1fX700Ufp27cvAwcO5KGHHgIgJyeHyZMnM2TIEIYMGcLXX39NZmYmy5Yt44UXXiA+Pp4N\nGzY0siaFEEIIIURrVmGpoKC0gJOFJ8k8l8nunN2kZaex/vB6Pj/wOR/t+Yi3t7/N8q3LySnJcXZx\nL0mrHzW0JVi6dCldunTh/PnzDBkyhMmTJ2OxWLjnnntYv349Xbt25cyZMwA8/fTTBAUFsWPHDgDO\nnj3b4PGzsrL45ptvcHd3Jy8vjw0bNuDh4cGaNWtYuHAhH374IcuXLyczM5P09HQ8PDw4c+YM7du3\nZ+7cueTk5BAeHs6KFSu46667HI5dXFzMzJkzWbt2Lb169eLOO+/ktdde44EHHgAgLCyMbdu28eqr\nr/L888/z+uuv2/b18vLif/7nf0hLS+OVV14B4Mknn2T37t1s3LgRX19fioqK+OKLL/Dx8WH//v3c\ndtttpKWl1bjG9PR0vv/+e7y9vYmLi+P++++nc+fOttdPnz7N6tWrycjIQCnFuXPnAFiwYAEPPvgg\nw4cP58iRI1x77bXs2bOHOXPmEBAQYAuMQgghhBCi5dJaU1pRSlFZ0cUv5TXXFZYWOjwvqShpdJkW\n9V/UjFfc/NpUEHzg8wdI/ym9SY8Z3zGeF8e9WO82y5Yts3WPPHr0KPv37ycnJ4eRI0fapgkICQkB\nYM2aNbz33nu2fdu3b99gGaZOnYq7uzsAubm5zJgxg/3796OUoqyszHbcOXPm4OHh4XC+6dOn8/bb\nbzNr1iw2bdrEm2++6XDsvXv30rVrV3r16gXAjBkzWLp0qS0ITpo0CYDExEQ++uijBssKMHHiRHx9\nfQEzz+P8+fNJT0/H3d2dffv21brPNddcQ1BQEAB9+/bl8OHDDkEwKCgIHx8f7r77bsaPH8/48eNt\n1717927bdnl5eRQUFDSqnEIIIYQQovG01pwvP8+54nO2Ja8kr9FBrbCssN7XLdpywWXy8fDBz9Ov\n1iXML8w89qj99YaWXWm7muFdvHzaVBB0htTUVFJTU9m0aRN+fn6MGjXqoqYHsB9Nsvr+/v7+tseP\nP/44o0ePZvXq1WRmZjJq1Kh6jztr1iwmTJiAj48PU6dOtQXFxvL29gbA3d290fco2pf3hRdeICIi\ngh9++AGLxYKPj0+956nrXB4eHmzZsoW1a9eyatUqXnnlFdatW4fFYuHbb7+t87hCCCGEEMLQWlNU\nVuQQ5OpcSmpfX25p3PdBN+WGv6d/rQEr3C/8ooJZ9cXXwxd3N/dme7/2u+1vtmNfDm0qCDbUctcc\ncnNzCQ4Oxs/Pj4yMDL799lsAhg4dyty5czl06JCta2hISAhjxoxh6dKlvPiiKevZs2dp3749ERER\n7Nmzh7i4OFavXl3nhOe5ublERUUB5t66SmPGjOHPf/4zo0ePtnUNDQkJITIyksjISJ555hnWrFlT\n43hxcXFkZmZy4MABevTowVtvvcVVV13V6OsPDAwkPz+/3vcnOjoaNzc33njjDSoqKhp9bHsFBQUU\nFRVx/fXXM2zYMLp16wbA2LFjWbJkCQ8//DBgupjGx8cTGBhIXl7eRZ1LCCGEEKIl0lpTWFbYuCBn\nt+SW5DY6yPl6+BLsE0ywTzBBPkGE+YXRI6QHwd7BtvX2S6B3YK2Bz8vdS6bNcrI2FQSdYdy4cbzy\nyiv06dOHuLg4hg4dCkB4eDjLly9n0qRJWCwWOnTowBdffMFvfvMb5s2bR//+/XF3d+eJJ55g0qRJ\nLFq0iPHjxxMeHk5SUlKd3RsfeeQRZsyYwTPPPMMNN9xgWz979mz27dvHwIED8fT05J577mH+/PkA\n3H777eTk5NCnT58ax/Px8WHFihVMnTqV8vJyhgwZwpw5cxp9/aNHj2bRokXEx8fz2GOP1Xh97ty5\nTJ48mTfffJNx48Y5tBZeiPz8fG688UaKi4vRWrN48WIAXn75ZebNm8fAgQMpLy9n5MiRLFu2jAkT\nJjBlyhQ+/vhjlixZwogRIy7qvEIIIYQQTUVrTUFpwQUFt+pLha7/R3U/Tz+CvINsQa2Dfwd6hfaq\nNcRVX4K8g/D28K73+MJ1KK21s8vQZJKSknT1gUb27NlTa8C5nPLz8+tswWsJ5s+fT0JCAnfffbez\ni9IiOLu+WsJn1tWkpqY22A1atCxSZ65F6sv1SJ21XKeLTpNxKoO9p/ey99Re9p7ey77T+zh27hgF\nFQUN3gfn5+lXe1Cro0XOIcj5BOHl7nWZrrT1a6l/z5RSW7XWSQ1t16wtgkqpccBLgDvwutZ6UbXX\n2wN/A7oDxcBdWuuddq+7A2nAMa31+OYsa1uVmJiIv78/f/rTn5xdFCGEEEKIVqGsoowfz/7I3lN7\nq0KfNfidPn/atp2Xuxc9QnrQO6w3cV5x9Over97WOAlyoik1WxC0hrilwBggC/hOKfWJ1nq33WYL\ngXSt9U1Kqd7W7a+xe30BsAdo11zlbOu2bt3q7CIIIYQQQrgcrTU5RTm2Vr29p/aScTqDvaf2cvDs\nQYcumhH+EfQO683kPpOJC4sjLjSOuLA4YoNj8XAzX8dbauuSaL2as0UwGTigtT4IoJR6D7gRsA+C\nfYFFAFrrDKVUrFIqQmt9QikVDdwA/A74ZTOWUwghhBBCiFqVlJfw49kfTcteZeg7bVr6zhWfs23n\n7e5Nz9CeDIwYyM39braFvbjQOIJ8gpx4BULUrjmDYBRw1O55FpBSbZsfgEnABqVUMtAFiAZOAC8C\njwAt9+Y6IYQQQgjh8rTWnCg8YQt69vfwHTp3yOG+vcjASOJC47i13622oNc7rDcxQTHNOlWBEE2t\n2QaLUUpNAcZprWdbn08HUrTW8+22aYe5hzAB2AH0Bu7BhMHrtdZzlVKjgIfqukdQKXUvcC9ARERE\nov1k7GAmGu/Ro0cTX92FqaiosE34Llo+Z9fXgQMHyM3Nddr5XVFBQQEBAQHOLoa4AFJnrkXqy/VI\nndWu1FJKVlEWR88f5WjRUY6cP8LRIvO4sKLQtp2XmxfRvtHE+MXQ2bcznf06E+MXQ7RvNP4eFzfC\neUOkzlxPS62z0aNHO32wmGNAZ7vn0dZ1NlrrPGAWgDITiRwCDgK3ABOVUtcDPkA7pdTbWus7qp9E\na70cWA5m1NDqfav37Nnj9BE7nT0Kpbgwzq4vHx8fEhISnHZ+VyT3VbgeqTPXIvXletpynWmtOV5w\nvNbWvcxzmWiqGkGi20UTFxbH1aFXmwFbrC18nYM646bcLmu523KduSpXr7PmDILfAT2VUl0xAfBW\nYJr9BkqpYKBIa10KzAbWW8PhY9YFuxbBGiFQtD6pqak8//zzfPrpp3zyySfs3r2bRx99tMZ2AQEB\ndc6lCHDu3Dneffdd5s6dC0B2dja/+MUvWLVqVbOVvTapqal4eXlx5ZVXXtbzCiGEaFvKKso4WXiS\nk4UnOVF4gs05mzmXcQ4PNw883DxwV+5Vj93c61zXmPVuyq1FTAReVFbE/tP7HaZhyDiVwb7T+8gv\nzbdt5+fpR1xoHCnRKdw56E7bvXu9QnsR4NXyWnOEuFyaLQhqrcuVUvOBf2Omj/ib1nqXUmqO9fVl\nQB/gDaWUBnYBMpHdJSgvL8fDo1lnBLmsJk6cyMSJEy9q33PnzvHqq6/agmBkZORlD4FggmBAQIAE\nQSGEEBfsfNl5ThSe4ETBCVvAO1FwwvxZaF1nfX7m/JmaB9hdc1VTqQyFFxIg61pf77bKcb9yS7lt\nWoYjuUccWvdigmKIC41jZvxMh4FaotpFXfbWPSFcQbOmBq31Z8Bn1dYts3u8CejVwDFSgdRmKN5l\nc9ttt3H8+HGKi4tZsGAB9957LwCff/45CxcupKKigrCwMNauXUtBQQH3338/aWlpKKV44oknmDx5\nskML2KpVq/j0009ZuXIlM2fOxMfHh++//55hw4Zx6623smDBAoqLi/H19WXFihXExcVRUVHBr3/9\naz7//HPc3Ny455576NevHy+//DL/93//B8AXX3zBq6++yurVqx3Kv3btWh566CHKy8sZMmQIr732\nGt7e3sTGxjJjxgz+8Y9/UFZWxv/+7//Su3dvh32HDh3KX//6V/r16wfAqFGjeP7557FYLLWW097K\nlStJS0vjlVde4dChQ0ybNo2CggJuvPFG2zaVz8+ePUtZWRnPPPMMN954I48++ig//vgj8fHxjBkz\nhnnz5jF+/Hh27txJcXEx9913H2lpaXh4eLB48WJGjx7NypUr+eSTT8jLy+Pw4cPcdNNNPPfcczXq\n89FHH+WTTz7Bw8ODsWPH8vzzz5OTk8OcOXM4cuQIAC+++CJRUVEsW7YMd3d33n77bZYsWULv3r1r\nbDds2LCL/mwJIYRwHVpr8kvzq8JcAwHPvlXLXjvvdkT4R9DBvwN9wvswKnYUEf4RRASYdRH+EWRs\nz2Bw4mAqdAXllnLKLeVUWMzj2tY1Zv2FbFvXNvbryi3lFJcXX1CZALq178awmGHcFXqX6c4ZGkfP\n0J74efpdzuoUwuW1nuajFmzp0qV06dKF8+fPM2TIECZPnozFYuGee+5h/fr1dO3alTNnzC95Tz/9\nNEFBQezYsQOAs2fPNnj8rKwsvvnmG9zd3cnLy2PDhg14eHiwZs0aFi5cyIcffsjy5cvJzMwkPT0d\nDw8Pzpw5Q/v27Zk7dy45OTmEh4ezYsUK7rrrLodjFxcXM3PmTNauXUuvXr248847ee2113jggQcA\nCAsLY9u2bbz66qs8//zzvP766w7733LLLXzwwQc89dRTHD9+nOPHj5OUlFRnOeuyYMEC7rvvPu68\n806WLl1qW+/j48Pq1atp164dp06dYujQoUycOJFFixaxc+dO0tPTAcjMzHSoD6UUO3bsICMjg7Fj\nx7Jv3z4A0tPTWb9+PWFhYcTFxXH//ffTuXPVra6nT59m9erVZGRkoJTi3LlztvI9+OCDDB8+nCNH\njnDttdeyZ88e5syZQ0BAAA899BAA06ZNq3U7IYQQrsmiLZw5f8ahdc4h4NmFvJOFJykuL671OKG+\noUQERBDhH0FSZBId/DrYntsHvA7+HfD19G2wXGUHy0joJPebCyHq1raC4AMPgDUYNJn4eHjxxXo3\nWbZsGZ99ZhpGjx49yv79+8nJyWHkyJF07doVgJCQEADWrFmD/cin7du3b7AIU6dOtY1ymZuby4wZ\nM9i/fz9KKcrKymzHnTNnjq3raOX5pk+fzttvv82sWbPYtGkTb775psOx9+7dS9euXenVyzTczpgx\ng6VLl9qC4KRJkwBITEzko48+qlG2m2++mbFjx/LUU0/xwQcfMGXKlHrLWZevv/7aFhSnT5/Or3/9\na8D8urpw4ULWr1+Pm5sbx44d48SJE/Uea+PGjdx///0A9O7dmy5dutiC4DXXXENQUBA+Pj707duX\nw4cPOwTBytfuvvtuxo8fz/jx423v7+7dVX1w8vLyar2Hsa7tWuKIU0II0VaVW8rJKcyp2VpnF+gq\nn+cU5VBuKa9xDHflTrh/uC3I9Q7rbQty9gEvwj+CML8wPN09nXClQoi2rG0FQSdITU0lNTWVTZs2\n4efnx6ht+ml8AAAgAElEQVRRoygurv3XwPrY35RdfX9//6phjB9//HFGjx7N6tWryczMbHAko1mz\nZjFhwgR8fHyYOnXqBd9j6O3tDYC7uzvl5TX/I4yKiiI0NJTt27fz/vvvs2zZsosqJ1DrjenvvPMO\nOTk5bN26FU9PT2JjYy/q/a1+PXVdk4eHB1u2bGHt2rWsWrWKV155hXXr1mGxWPj222/x8fGp9/iN\n3U4IIUTzKSorYmv2Vr7L/o4juUdq3G93uui0w71nlbzdvW2tc1GBUQzuONgW5qoHvBDfELkvTQjR\norWtINhAy11zyM3NJTg4GD8/PzIyMvj2228Bc+/c3LlzOXTokK1raEhICGPGjGHp0qW8aC3r2bNn\nad++PREREezZs4e4uDhWr15d5/QGubm5REVFAeYeu0pjxozhz3/+M6NHj7Z1DQ0JCSEyMpLIyEie\neeYZ1qxZU+N4cXFxZGZmcuDAAXr06MFbb73FVVdddUHvwS233MJzzz1Hbm4uAwcOrLecdRk2bBjv\nvfced9xxB++8847D9Xbo0AFPT0++/PJLDh8+DEBgYCD5+bXfWzFixAjeeecdrr76avbt28eRI0eI\ni4tj27ZtDZajoKCAoqIirr/+eoYNG0a3bt0AGDt2LEuWLOHhhx8GTBfT+Ph4AgMDycvLs+1f13ZC\nCCGaR4WlgoxTGWw+tpnNWZvZkr2FHSd2UKHN/WYBXgG28NYztCfDY4Y7tNbZB7x23u1axGiZQgjR\nFNpWEHSCcePG8corr9CnTx/i4uIYOnQoAOHh4SxfvpxJkyZhsVjo0KEDX3zxBb/5zW+YN28e/fv3\nx93dnSeeeIJJkyaxaNEixo8fT3h4OElJSXVOnfDII48wY8YMnnnmGW644Qbb+tmzZ7Nv3z4GDhyI\np6cn99xzD/Pnzwfg9ttvJycnhz59+tQ4no+PDytWrGDq1Km2wWLmzJlzQe/BlClTWLBgAY8//niD\n5azLSy+9xLRp03j22WcdBou5/fbbmTBhAgMGDCApKck2WE1oaCjDhg2jf//+XHfddcybN8+2z9y5\nc7nvvvsYMGAAHh4erFy50qElsD75+fnceOONFBcXo7Vm8eLFALz88svMmzePgQMHUl5ezsiRI1m2\nbBkTJkxgypQpfPzxxyxZsqTO7YQQQjSN7PxsthzbwuaszWw+tpm07DTboCtB3kEkRyXz2PDHSI5K\nJjkqmYiACCeXWAghnENpXbPrg6tKSkrSaWlpDuv27NlTa8C5nJw9QXlD5s+fT0JCAnffLbN3gPPr\nqyV8Zl2Nq0/o2hZJnbmWllpfBaUFbM3eyuZjm034O7aZrLwsADzcPIjvGE9yZDIp0SmkRKXQM7Rn\nm+mu2VLrTNRN6sz1tNQ6U0pt1VonNbSdtAi2cYmJifj7+/OnP/3J2UURQggh6lRhqWB3zm6HLp47\nT+7Eoi2AmVJgRMwIUqJSSI5KJqFTAj4ecj+2EELURYJgG7d161ZnF0EIIYSoISsvywQ+a0tfWnYa\nhWWFALT3aU9yVDI/j/s5KdEpDIkcQrh/uJNLLIQQrkWCoBBCCCGcKr8kn7TsNNPaZ+3mmZ2fDYCX\nuxfxHeO5K+EukqOSSYlKoUdIDxm0RQghLlGbCIJaa/kPQ7iE1nTPrhBC1KbcUs7OkzsdBnTZnbPb\nNl1Dz5CejI4dTUpUCinRKQyKGIS3R+MG9BJCCNF4rT4I+vj4cPr0aUJDQyUMihZNa83p06dljkEh\nRKuhteZo3lFb4NtybAtbj2+lqKwIgFDfUFKiU5jadyop0ebevhDfECeXWggh2oZWHwSjo6PJysoi\nJyfHaWUoLi6WL/cuxJn15ePjQ3R0tFPOLYS4dG29B0pucW6NLp4/FfwEmMnYEzolcM/ge2wDunRr\n361Nv19CCOFMrT4Ienp60rVrV6eWITU1lYSEBKeWQTSe1JcQoiGlFaVknMrgh59+4IcT1uWnHzhV\ndAo/Tz/8vfzx8/Sre/Go57VGLO5u7s5+CyirKGPHyR22wVw2Z20m41SGrYtnXGgcY7qNsXXxHBgx\nEC93LyeXWgghRKVWHwSFEEKIS5FTmGMLepWhb0/OHsosZQD4ePjQv0N/JsZNpFNAJ86Xn6eorMhh\nKSwr5Mz5M2TlZTmuLy2kQldccJm83L0aFxrtAmeD4bTa4u3ubWut01qTeS6zRhfP4vJiAML9wkmJ\nTmHagGkkRyUzJHII7X3bN10lCCGEaHISBIUQQgjMICb7Tu+r0cp3vOC4bZvIwEgGRQzi+h7XM6jj\nIAZFDKJnaE883C7+v9OyirIawfFClsKyQofnZ8+frbHN+fLzF1wuhbKFwpLSEvLW5wEm+CZ2SuS+\npPtsrX1dgrpIF08hhHAxEgSFEEK0OWfPn2X7ie0OLX27cnbZWrg83TzpG96XMd3HMCjCBL5BHQcR\n5hfW5GXxdPckyD2IIJ+gJj92JYu2UFxefHFBs7SQY9nHuC7hOlKiUxjQYQCe7p7NVlYhhBCXhwRB\nIYQQrZZFW/jxzI81unYeyT1i2ybcL5xBHQcxb8g8W+DrHda7Vd3P5qbcbK17FyM1NZVRQ0Y1baGE\nEEI4lQRBIYQQrUJ+SX6NVr4dJ3fYpipwV+7EhcUxrPMw5ibNtXXt7BjQUbo1CiGEaHMkCAohhHAp\nlQOXVG/lO3j2oG2bYJ9gBkUMYnbCbFvg69ehHz4eMpWPEEIIARIEhRBCtGBFZUXsPLnTIfBtP7Gd\nvBIzcIlC0TO0J4mdErkr/i5b6ItuFy2tfEIIIUQ9JAgKIYRwOq01x/KP1Rixc/+Z/Vi0BYBAr0AG\nRgzkjgF32AJf/w798ffyd3LphRBCCNcjQVAIIcRlVVxezL78fRz6/pBDK9+Z82ds23QN7sqgjoO4\ntf+ttgFcYoNjcVNuTiy5EEII0XpIEBRCCNFkLNpCTmEOR3KPcCT3CEfzjtZ4fKLgBBoN28DXw5cB\nEQOY3GeyLfANjBhIO+92zr4UIYQQolWTICiEEKLR8kvyHcNd7lGO5FU9Ppp3lNKKUod9/Dz9iAmK\noXO7ztzQ8wY6t+uM5aSFaVdPo0dID9zd3J10NUIIIUTbJUFQCCEEAGUVZRzLP1YV8Gpp0TtXfM5h\nHzflRlRgFJ2DOjMkagiT+0ymc1BnYoJibOEvxDekxsAtqampxIXFXc7LE0IIIYQdCYJCCNEGaK05\nVXSqRjdN+6B3PP+46bJpJ8Q3hJigGGKDYxnZZaQt3FUGvU6BnfBwk/9KhBBCCFcj/3sLIUQrUFha\n2GCXzeLyYod9fDx8bKFubPexxLSztuJZW/Q6t+ssI3IKIYQQrZQEQSGEaOHKLeVk52fbumvW1mXT\nfsRNMPPrRQZG0jmoMwmdEpgYN9Ghu2ZMUAxhfmEy154QQgjRRkkQFEKIFsKiLew7vY8tx7awOWsz\nP5z4gcO5h8nOz7bNpVcp2CfYFuiu7HylQ3fNzkGdiQqMwtPd00lXIoQQQoiWToKgEEI4yfH842w5\ntsUEv2Ob+S77O/JK8gAI8AogoWMCV3e9mph2MTUGYAn0DnRy6YUQQgjhypo1CCqlxgEvAe7A61rr\nRdVebw/8DegOFAN3aa13KqU6A28CEYAGlmutX2rOsgohRHPKL8ln6/GtDsEvKy8LAHflzsCIgUzr\nP43kqGSSo5LpHdZbplUQQgghRLNptiColHIHlgJjgCzgO6XUJ1rr3XabLQTStdY3KaV6W7e/BigH\nfqW13qaUCgS2KqW+qLavEEK0SGUVZew8udMh9O3O2W0bkbNb+24MjxlOSlQKyVHJJHRMwNfT18ml\nFkIIIURb0pwtgsnAAa31QQCl1HvAjYB9mOsLLALQWmcopWKVUhFa6+PAcev6fKXUHiCq2r5CCOF0\nWmsOnTtkC31bjm1h2/FtnC8/D0CobyjJUclM6TuFlKgUhkQNIcwvzMmlFkIIIURb15xBMAo4avc8\nC0ipts0PwCRgg1IqGegCRAMnKjdQSsUCCcDmZiyrEEI0yumi01WhL9v8earoFGCmYxjcaTD/lfhf\nJEclkxKdQtfgrjIypxBCCCFaHKW1bnirizmwUlOAcVrr2dbn04EUrfV8u23aYe4hTAB2AL2Be7TW\n6dbXA4CvgN9prT+q4zz3AvcCREREJL733nvNcj2XoqCggICAAGcXQzSS1Jfraa46K6ko4UDBAfbk\n72FP/h4y8jLILs4GzPQMXfy60Ltdb/oE9qFPYB+6+neVydUbSf6euRapL9cjdeZ6pM5cT0uts9Gj\nR2/VWic1tF1zfmM5BnS2ex5tXWejtc4DZgEo85P5IaCyK6kn8CHwTl0h0HqM5cBygKSkJD1q1Kim\nu4ImkpqaSkssl6id1JfraYo6s2gLGacybFM3bMnewvYT2ym3lAMQ3S6aoV2HkhxpBnNJikySkTsv\ngfw9cy1SX65H6sz1SJ25Hlevs+YMgt8BPZVSXTEB8FZgmv0GSqlgoEhrXQrMBtZrrfOsofCvwB6t\n9eJmLKMQoo06lnfMYTCXtOw08kvzAWjn3Y4hkUN4+MqHbaN4RgZGOrnEQgghhBBNp9mCoNa6XCk1\nH/g3ZvqIv2mtdyml5lhfXwb0Ad5QSmlgF3C3dfdhwHRgh1Iq3bpuodb6s+YqrxCi9corySMtO80W\n+rYc20J2vuni6eHmwaCIQUwfON0W+uLC4nBTbk4utRBCCCFE82nWm1mswe2zauuW2T3eBPSqZb+N\ngIyuIIS4YKUVpew4scM2mMvmrM1knMqwTd3QI6QHo2JH2aZuiO8Yj4+Hj5NLLYQQQghxecmoBkII\nl2XRFn488yNrTqzh/z7/P9vUDSUVJQCE+YWREpXCrf1vJSUqhaTIJEL9Qp1caiGEEEII55MgKIRw\nCRWWCvaf2c/W7K1sO76Nrce38v1P35NXkgeAr4cviZGJzBsyzzZ1Q5egLjJ1gxBCCCFELSQICiFa\nnHJLOXtP7WXr8arQl/5TOgWlBQB4u3szqOMgbh9wO4M7DUYf08y8fiae7p5OLrkQQgghhGuQICiE\ncKpySzm7c3Y7tPSl/5TO+fLzgGnpi+8Yz8xBMxncaTCJkYn0CevjEPpS81IlBAohhBBCXAAJgkKI\ny6a0opRdJ3c5tPRtP7Gd4vJiAPw9/UnolMC9ifea0NcpkbiwOJmkXQghhBCiicm3KyFEsygpL2HH\nyR0OLX07Tu6gtKIUgECvQAZ3GszcpLm2lr6eIT1xd3N3csmFEEIIIVo/CYJCiEt2vuw8209sd2jp\n23lyJ+WWcgCCfYIZ3GkwC1IWkNgpkcGdBtM9pLvM1SeEEEII4SQSBIUQF6SwtJAfTvxgWvp+2sbW\n7K3sztlNha4AIMQ3hMROiTx0xUMkRprQ1zW4q4zeKYQQQgjRgkgQFELUKb8kn/Sf0h1a+jJOZWDR\nFgDC/cJJjExkYtxEW0tfTFCMhD4hhBBCiBZOgqAQAoDc4ly+/+l7W+Dbmr2Vfaf3odEAdAroRGJk\nIlP6TLG19EUFRknoE0IIIYRwQRIEhWiDzp4/y7bj26pC3/GtHDhzwPZ6dLtoEjslMm3ANFtLX6fA\nTk4ssRBCCCGEaEoSBIVo5UrKS0jLTmPjkY2kHU9ja/ZWDp07ZHu9S1AXEiMTmTlopq2lr4N/ByeW\nWAghhBBCNDcJgkK0MrnFuXxz9Bs2HNnAxiMb2XJsCyUVJQB0a9+NpMgk7k28l8ROiSR0SiDML8zJ\nJRZCCCGEEJebBEEhXFx2fjYbDpvQt+HIBraf2I5G4+HmweBOg5mfPJ8RMSMYFjNMQp8QQgghhAAk\nCArhUrTW7Du9jw1HNtha/A6ePQiAv6c/V3S+gieueoIRXUaQEpWCv5e/k0sshBBCiDalpAS2bIHU\nVLMcOQLh4dChQ83Ffn1oKHhINLmc5N0WogUrt5Tz/fHvbaFv45GN5BTlAGbqhuExw5k/ZD7DY4YT\n3zEeT3dPJ5dYCCGEEG1K9eD3zTdQXAxKwaBBkJgIp0/DoUOweTPk5EBFRc3jKGXCYG2BsXpo7NAB\ngoLMPuKiSRAUogUpLC1k87HNbDhsWvy+zfqWwrJCwNzfd33P6xkeM5wRMSPoFdpLpm4QrqG8HI4e\nhR9/NEtmJt0OH4avvgJf36rFz8/xefWl8nVvb/nPv7lobb7UlZSYL3LWx94nTzq7ZEKIlqKkxAS6\nyuC3aVNV8IuPh/vug1GjYMQIaN++5v4WC5w9CydP1lxycqoe//CD+fPs2drL4enZ+NDYoYP5/0M4\nkCAohBOdKjpl7u07vIGNRzey7fg2yi3lKBQDIwYyK34WI7qMYHjMcCIDI51dXCHqVlBgQt7Bg1WB\nr/Lx4cMmDFby8CBaKSgru7hzKQU+PhcXIi/mdR+f5g2eFguUljoEr+pBrFGPL2af6vuXltZaxCvA\nfPH7wx/Ay6v53gshRMtzqcGvOjc30/IXGgp9+jS8fWkpnDpVe3C0D49798KJE3D+fO3HCQhoXGDs\n0AHCwtpEN9XWf4VCtBBaazLPZdoGddlwZAMZpzIA8Hb3JjkqmUeufIThMcO5svOVBPkEObnEQtjR\nGn76qe6wV73FqH176N4dkpLg5pvN4+7doVs3iIpi/YYNjBoxwnyZOH/ecSkqqrmuMa9Vvp6TU/tr\nJSUXf/2NDZFubhce0C42EFfn5mZCq7e3WWp77O9vvnxVrq9ru2qPj/3v/xK1eDFs3Ajvvw+xsU1T\nZiFKSyE/v2opKHB8Xn1daSn07An9+kH//hAdLT0EmlpxsWNXz0sNfpfKywsiI83SGIWFdYfGyuB4\n+DB89515Xls3VajZTbWW0OhRUNB01+kEEgSFaCYWbWHnyZ22bp4bj2zkWP4xAIK8gxgWM4wZg2Yw\nImYESZFJeHt4O7nEos0rLYXMzNrD3sGDJmRVUgo6dzbhbuJEE/Dsw15jvhy4u5tg4n+ZBjWqqKg9\neF5qAD11qup1i8UxTHl7m1+hK8NXXaGrkYGs3tea8dfr/dHRRE2fDnffDQkJsHIl3Hhjs51PtGDV\ng1tjwltd6woK6myFrsHdHQIDzef81Kmq9e3amUBov/TrZ76oi8YpLq7Z4ldS4rzgd6n8/aFrV7M0\nxGKBc+caDo47dpjHZ8447N7u2Wdh/PhmupDmJ0FQiCZSUl7Cd9nf2Vr8vj7yNbkluQBEBUaZLp6d\nhzOiywj6d+iPm3JzcolFm3TuXM3WvMrHR4+a/xQr+fpWBbwxYxzDXpcuJny4kssdPFubKVNMCLz5\nZvj5z+HBB2HRIukq2tKVlNQdwi4mvDW2BdvDwwS3gADzZ+USEeH4vLZtaltnf2/wmTOwaxfs3Fm1\nrFoFy5dXnT88vGY47NcPgoOb/j12NfUFv4QEmDfPBL/hw10j+F0KNzcICTFL794Nb19W5tBNNc/+\nB1IXJEFQiItkP3H7hiMb+O7Yd7aJ2/uE9eHmfjczImYEI7qMoEtQFxnYRVweFgscO1Z32Kv2ayYd\nOpiAN3y4Y4te9+7QsaN0uRKOunc3IwI+9BC88AJ8/bV0FW1pzpyBX/6SYatXm1bqCw1u1UNYp06N\nD2v2z5tzUKeQENM6NWJE1Tqtzf1h9uFw505YscKE2UrR0TUDYt++pot3ayXBr+l4epq/E506AVCe\nmurc8lwiCYJCNFJjJm4fHjOcYZ2HEe4ffnkLV1JifqGqbzlzxvxHX9kiYr/4+dW+vrZFfv13vvPn\nzTDctYW9Q4ccu1p5eJjWu8r79ezDXrdu5gubEBfC2xuWLIGrrqrqKrpihWklFM71j3/AvffCqVOc\n+tnP6BQfX39Ys1/n6qPxKmV+vOrYEX72s6r1Fovp7VA9IH75ZdV9w0qZfw/tw2H//hAX55r/5xUX\nw7ffVgW/b7+tPfiNGCEtpG2cBEEhauHUidsrKkxoy8lpONxVLvn5dR8vONh0kQkJMSM3FhY6Lhc6\ngEZdYfJig6X94uvr2l9E6mOxmPe/osL8af+4tnWVdVUZ+OzDXna247EDA02469/f3K9nH/ZiYtrE\nyGfCCSq7it5yC9x0EzzwADz7rGt+cXZ1Z8+a9//NN2HAAPjsM/bm5tJp1Chnl8z53NzMj2FdusAN\nN1StLy83/65WD4ifflo1eIiHB/TqVTMgdu9uupq3FBL8xEWSbwdCWGmt+ef+f/LcrufI+C6jaSZu\n1xpycx1DW0MB7+xZs19t/P3NkMZhYSbcxcVVPa9tCQkx3RjqU1HhGAyLimqGxcYsubkmoFQ/Vl3X\nUhul6g6R9YTLqMxMSE+vP1Q1FLou9nFjt72Q96E2kZHmy8fYsY736nXvbgYiaa0BWrRs3bub7qEP\nPwwvvljVVbQxgzSIpvHZZ3DPPaZb5OOPw29+Y8K4i3dZa3aVIa9XL5g0qWp9SQns2+cYDtPS4IMP\nqrbx8THTHlQPiDExl+ff4rqCn5ubCX7z51d19ZTgJ+ohQVC0eRZtYfWe1Tyz4RnSf0onzCuM6/tc\nb+7vs5+4XWsTbLKya4a3usLd6dOO86fZ8/JyDG3x8VUBr7ZQFxraPJOhurubUdfatWv6Y2tdNeLi\nxYRL++XMmZrrrO9tz/rK4OlprtHDwyyVj2tbV9djH5+L2+9Ctq1tPx+fqpHPZCJc0VJ5e8PLL5sv\nnnfdVdVV9KabnF2y1i031wzYs2KFCSKffAKJic4ulevz9jatqgMGOK4vLIQ9exwD4rp18NZbVdsE\nBlaFwso/+/c3A+RcSkA8f75m8CstleAnLpkEQdFmVVgq+GDXB/xuw+/YlbOLXqG9eP/KFxn8Xjo9\njgdCzlo49b5jsCsurv1glZOjVoa2Xr3gyisdg1z1gBcQ0PpbcSpb+Pz8zDU3tbIyKCzk69RUho0a\nVTNUucnIrEJcNpMmmR+0brnFPF6wAJ57TrqKNod//xtmzza9MBYuhN/+1vVG8XU1/v7mPuukJMf1\nZ8+aEUztRzFdvRpef71qm9DQmq2H/fqZXju1aSj43X+/BD/RJCQIijanrKKMd3a8w+83/J79Z/bT\nL7wf7056l5sDh+J+zc/MPQPBwVWBLTq6qrWu+lIZ7oKDJXQ4g6cnBAdTFhws/xkK0RJ062YmnX/k\nEXjpJTPCqHQVbTp5eWbE1r/8xXRN3LQJkpOdXaq2rX17E8iGD69ap7WZXmDnTseA+Oabjvf0R0Y6\njFwau2EDPPGEBD9x2UgQFG1GaUUpK9NXsmjjIg6dO0R8x3hWTV3FTX1uwu3wERg9Gs6eZeurr5J4\n333OLq4QQrgmb28TAq+6SrqKNqU1a8worVlZ8Otfw5NPmu7jouVRynQHjYiAa66pWq911Qim9gHx\ntdeguJgubm4weDD84hdVwS8oyGmXIVo/CYKi1SsuL+b1ba/z7NfPkpWXRXJUMi9f9zI39LzB3Pt3\n8KAJgfn5sHYt+fWNwCmEEKJxJk2qmoB+0iTz5fa556QL44XKzzeD8fz5z2aAsK+/hqFDnV0qcTGU\nMgPKxMTA9ddXra+ogMxMNu7Zw4jx451XPtHmNGtfNqXUOKXUXqXUAaXUo7W83l4ptVoptV0ptUUp\n1b+x+wrRkMLSQhZvWkzXl7py/7/uJzY4ln/f8W++vftbxvcab0LggQPmV+uCAli7Vm60F0KIptS1\nq+kqumCBGVBm+HDz45tonHXrzKAly5ebLqHffy8hsDVyd4fu3akICHB2SUQb02xBUCnlDiwFrgP6\nArcppfpW22whkK61HgjcCbx0AfsKUau8kjwWbVxE7Eux/Oo/v6JveF++nPEl62euZ2z3sSYAghke\n+qqrzAAw69aZX66FEEI0LW9vM7XERx/B/v2m69tHHzm7VC1bQYEZCfKaa8xgOxs2wB//KKMHCyGa\nVHO2CCYDB7TWB7XWpcB7wI3VtukLrAPQWmcAsUqpiEbuK4SDs+fP8lTqU8S+GMtjax8jKTKJr+/6\nmrV3rmVU7KiqAAiQkWFCYFkZfPklDBrkvIILIURbcNNNpkWrVy+YPNl0FS0pcXapWp6vvoKBA+HV\nV830EOnpMGyYs0slhGiFmjMIRgFH7Z5nWdfZ+wGYBKCUSga6ANGN3FcIAE4VneK/1/43sS/F8uRX\nTzKyy0i2zN7Cv27/F1d2vrLmDrt2mZuwtTbDMvfvX3MbIYQQTa+yq+gDD8CSJSbgSFdRo7DQdKEd\nNcqMFvnVV7B4sZl+RwghmoHSWjfPgZWaAozTWs+2Pp8OpGit59tt0w7THTQB2AH0Bu4BejS0r90x\n7gXuBYiIiEh87733muV6LkVBQQEB0u+7yZ0pPcMHRz/g4+yPKbGUMDJ8JHfE3EGPgB517uN/8CCD\nfvUrtLs7PyxeTFFMTI1tpL5cj9SZ65E6cy3NUV9hGzcS9+yzKK3JePhhTl11VZMe35UE7dhB70WL\n8M3OJmvSJA7Ono3lEruByt8x1yN15npaap2NHj16q9Y6qaHtmjMIXgE8qbW+1vr8MQCt9R/q2F4B\nh4CBQL8L2bdSUlKSTktLa7JraCqpqamMGjXK2cVoNY7lHeO5r59j+bbllFaUclv/21g4YiF9wxu4\njfSHH8z9Fj4+pjtoz561bib15XqkzlyP1Jlrabb6ysw0E9Bv2WLuiXv++bY1qmhREfzmN+YeythY\n+NvfTItgE5C/Y65H6sz1tNQ6U0o1Kgg25/QR3wE9lVJdgWPArcA0+w2UUsFAkfU+wNnAeq11nlKq\nwX1F23P43GEWbVzE39L/hkVbmD5wOo8Nf4yeobUHOgfbtsGYMeDvb0Jg9+7NX2AhhBD1i401A6E8\n+ii88IKZIP3999vGv9HffAMzZ5oBdObNg0WLoAW2LAghWq9mC4Ja63Kl1Hzg34A78Det9S6l1Bzr\n68uAPsAbSikN7ALurm/f5iqraNkOnDnAHzb8gTe3v4mbcmNW/CweHf4oscGxjTtAWpoJgUFBJgR2\n7dqs5RVCCHEBvLzMvXBXXWWC0eDB8Ne/wpQpzi5Z8zh/Hn77W/jTn8x8cmvXwtVXO7tUQog2qFkn\nlFubFcEAACAASURBVNdafwZ8Vm3dMrvHm4Bejd1XtC17cvbwuw2/4+87/46Xuxf3Jd3HI8MeIbpd\ndOMPsnkzXHsthISYKSJiY5utvEIIIS7BjTeaUUVvuQWmTm2dXUU3bzZhNyMD/uu/zJQQgYHOLpUQ\noo1q1iAoxMXYfmI7z6x/hlW7V+Hr6csvh/6SX135KzoGdLywA33zDYwbBx06mBBYy8AwQgghWpDK\nrqKPPWZaCb/5Bj74wPW7ihYXw5NPmuAXFQX/+Y/pqSKEEE4kQVC0GFuzt/L0+qf5eO/HBHoF8tjw\nx3jwigcJ8wu78INt3AjXXQedOpkQGH0BrYhCCCGcx8vLdJu07yr6+uumldAVffeduY7du2H2bHNt\n7do5u1RCCNGs8wgK0Sibjm7i+neuJ+kvSXx1+CuevOpJDj9wmN9d87uLC4FffWVaAqOizDyBEgKF\nEML1TJxouor26QM332y6ihYXO7tUjVdSAv/933DFFZCbC59/Dn/5i4RAIUSLIS2Cwim01nx1+Cue\nXv806w6tI8wvjN9f/XvmJc+jnfcl/Ce5bh2MH2+6F61bBx0vsDupEEKIlqNLF1i/HhYuNC1plV1F\ne9Q9X2yLsG0bzJgBO3fCrFmmm2twsLNLJYQQDqRFUFxWWmv+8+N/GLlyJKPfGM2uk7t4fszzZC7I\n5LERj11aCPziC7jhBnMvSWqqhEAhhGgNvLzMoDEff2zmHRw82ITBlqi01IwImpwMZ87AP/9p5gaU\nECiEaIGkRVBcFlpr/rn/nzy9/mm2HNtCdLtolly3hLsT7sbX0/fST/D55/Dzn0NcHKxZA+Hhl35M\nIYQQLUdlV9FbbzUji371lWkl9PFxdsmM9HTTCrh9O9x5p5kkvn17Z5dKCCHqJC2CollZtIWP9nxE\n4vJEJvx9AicLT/Ln8X/mwP0HmJ88v2lC4GefmWHH+/Qx3UElBAohROtU2VX0oYfg1VfhyivhwAHn\nlqmsDJ56CoYMgZMn4ZNP4I03JAQKIVo8CYKiWVRYKvj7jr8z8LWBTP5gMgWlBay4cQX75u/j3sR7\n8fZoonmh/vEP0xI4YICZlDc0tGmOK4QQomXy9DTTMHzySVVX0fffd05Ztm+HlBQzNcQtt8CuXTBh\ngnPKIoQQF0iCoGhSZRVlvJH+Bn1f7cu0j6ah0bwz6R32zNvDzPiZeLp7Nt3JVq+GyZMhIcF0Bw0J\nabpjCyGEaNkmTDDdMfv1M91F5869fKOKlpXx/+3de5xVdb3/8deHARSBNMNGAswbomiAyC9NOoZ6\nTPFeXklUmDpqR8usR16yk5WVWaaWUYq5Aa94y8LL8ZJG2ckKbQYREOWAIYiKmuGICgPf3x9r1xmR\nywCzZ82a/Xo+Hjzce+11+czjgzO857vW98t3vgPDhsGiRdnPoxtv9OeQpELxGUG1iuUrlzOpYRKX\n/OES5r8+n8G1g7n9uNv59G6fplNU4PcNd9wBo0ZlP4Tvvx+23LL1ryFJat+22y67VfTCC7NRwsce\nyyaS6d+/ctd86qlsXcAnnsgC6FVXQa+NWOpIknLmiKA2ydtNbzPuL+PY+Sc7c9o9p9Fri15MOXEK\n9afXc+zAYysTAm+9Nfvhu/fe8MADhkBJqmZdusAPfpA9KrBgAey1V2VuFW1qgksuyc6/YEH2C8lb\nbjEESiosRwS1UZatWMY1j1/DD//4QxY3LmZ4v+Fce8S1fHKnTxIRlbvwzTfDySfDxz+eTcvdo0fl\nriVJKo7DD/+/WUVPPDFbRuiKK1pnVtFZs7JRwGnT4LjjYNw4JyaTVHiOCGqjHHHLEXz5wS+za69d\neeSUR3h07KMcvPPBlQ2B11+fhcBPfCKbKdQQKElqbrvtsmUlzj0Xrr4a9tkHnn1248+3cmV2y+nQ\noTBvXjbSeNtthkBJHYJBUBts1pJZPDL/ES7e/2IeOfUR9t9h/8oGQIAJE7Lfxh5wANxzD3TvXtnr\nSZKKqUsXuPTS7GfF889nIW7y5A0/z5w52d0n554Lhx6azQh6/PGtX68k5cQgqA02oX4CnTt15j+G\n/kfbXPDaa6GuDg46KJsufIst2ua6kqTiOuywbFbRQYOyycXOOAPeemv9x61cmS1UP2QIPPNM9kjC\nnXdCbW3la5akNmQQ1AZZsXIF1z95PYfvcji1Pdrgh+LVV8Npp8HIkfDrX0O3VliAXpJUHfr1y54V\nPPdcuOYa+NjHsnC3Ns8+C/vtly1Yf/DB2SjgqFFQ6bteJCkHBkFtkPuevY+X33yZuiF1lb/YT38K\nn/98NgHAXXe1zgP/kqTq8s9bRe+9FxYuzGb9vOWWd++zahX8+McweDDMng033JD93Nl223xqlqQ2\nYBDUBik1lNi2x7aM7D+yshe68kr4whfgqKOyW3I226yy15MkdWyHHprNKjp4MHzmM3D66dmtonPn\nwogR8KUvwYEHZusEjh7tKKCkDs8gqBZ7sfFF7n3mXk4dfCqdO1Vw5ZEf/QjOOQeOOQZuvx26dq3c\ntSRJ1aNfP/jtb+H882H8eNhzzywYPvkkTJyYPYf+oQ/lXaUktQmDoFrs+unXszKtZOyQsZW7yKWX\nZs9mHH98dutOly6Vu5Ykqfp06ZItDH/vvfDaa9lo4MyZcOqpjgJKqiouKK8WSSlRqi8xvN9wBvQa\nUJmLfPe78PWvZ7fsTJoEnf3rKUmqkEMPhRdfhE7+TlxSdfK7n1rksYWPMefVOdTtWaFJYr71rSwE\nnnxytnC8IVCSVGmGQElVrEXfASNip4jYrPx6RER8MSK2qmxpak9K9SW6d+nOcQOPa90TpwTf+AZ8\n85swdmy2cHxNTeteQ5IkSdK7tPRXYXcCKyNiZ2A80A+4uWJVqV1pXN7IrTNv5fjdj6fnZj1b78Qp\nwYUXwsUXw+c+B7/4hSFQkiRJagMtDYKrUkpNwKeAq1JKXwV6V64stSd3zLqDxuWNrXtbaEpw3nnZ\nA/tnnJEt9OstOpIkSVKbaOmDWCsiYhRwKnBEeZvTOVaJUn2JXT6wC8P7DW+dE6YEX/kKXHEFnHUW\n/OQnztQmSZIktaGWDsGMBT4GfDelND8idgBuqFxZai+eefUZHl3wKHVD6ojWCGspwdlnZyHw7LMN\ngZIkSVIOWjQimFKaBXwRICLeD/RMKV1aycLUPkyon0BN1HDK4FM2/WSrVmUjgD//eTYi+MMfGgIl\nSZKkHLR01tCpEfG+iNga+CtwbURcXtnSlLemVU1Mmj6Jkf1H0rvnJj4SumpV9izgz3+ePRtoCJQk\nSZJy09JbQ7dMKS0FPg1cn1LaG/j39R0UEYdExJyImBsR56/h8y0j4u6ImB4RMyNibLPPzilveyoi\nbomIzVv6Ral1PDD3ARY3LqZuyCZOErNyZTYr6LXXZrOEXnKJIVCSJEnKUUuDYOeI6A0cD9zTkgMi\nogYYB4wEBgKjImLgarudCcxKKQ0GRgA/ioiuEdGH7FbUYSmlPYAa4MQW1qpWUmoosc0W23DYLodt\n/ElWroS6umx9wIsuypaKMARKkiRJuWppEPw28ADwvymlaRGxI/Dseo75KDA3pTQvpbQcmAwctdo+\nCegZ2SwkPYDXgKbyZ52BbhHRGdgCeKGFtaoVLHlzCVPmTOHkQSfTtabrxp2kqQlOOQWuvx6+/e1s\n0XhDoCRJkpS7lk4Wcztwe7P384Bj1nNYH+D5Zu8XAnuvts9PgSlkIa8ncEJKaRWwKCIuAxYAbwEP\nppQebEmtah03PnkjTauaNn7twKYmGD0abr0Vvvc9uOCC1i1QkiRJ0kaLlNL6d4roC1wF/HMhuUeB\ns1NKC9dxzLHAISmlz5XfnwzsnVI6a7V9hgNfBnYCHgIGk90KeidwAvA6WQi9I6V04xqucxpwGkBt\nbe1ekydPXu/X09YaGxvp0aNH3mW0WEqJusfr6FbTjZ8N/dkGHx9NTez2ne/wwd/9jv89/XSeP7FY\nd/UWrV+yZ0Vkz4rFfhWPPSsee1Y87bVn+++//xMppWHr26+lC8pPAG4Gjiu/H13edtA6jlkE9Gv2\nvm95W3Njge+nLI3OjYj5wK7Ah4H5KaUlABHxS2Bf4D1BMKU0HhgPMGzYsDRixIgWfkltZ+rUqbTH\nutZm2qJpPPf757jm8GsYsdeIDTt4+XI48UT43e/g8svZ6Zxz2KkiVVZO0fole1ZE9qxY7Ffx2LPi\nsWfFU/SetfQZwW1SShNSSk3lPxOBbdZzzDSgf0TsEBFdySZ7mbLaPguAAwEiohYYAMwrb98nIrYo\nPz94IDC7hbVqE11Xfx3dOnfjhN1P2LAD33kHjj0W7rorWyj+nHMqU6AkSZKkTdLSEcFXI2I0cEv5\n/Sjg1XUdkFJqioizyCaZqQFKKaWZEXFG+fOrgYuBiRExAwjgvJTSK8ArEXEH2ZqFTUA95VE/Vday\nFcu45albOHbgsWy5+ZYtP/Dtt+GYY+C++2DcOPjP/6xckZIkSZI2SUuDYB3ZM4JXkM30+UdgzPoO\nSindB9y32rarm71+AfjkWo69CLiohfWplfxy9i9Z+s7SDZsk5q234NOfhvvvh2uugdNOq1yBkiRJ\nkjZZS2cN/RtwZPNtEfEl4MpKFKX8lOpL7Pj+Hdnvw/u17IBly+Doo+E3v4HrrsvWDJQkSZLUrrX0\nGcE1+XKrVaF2Yd7f5/Hb537L2CFj6RQt+Kvx5ptwxBFZCJwwwRAoSZIkFURLbw1dE1cG72AmNkwk\nCMYMGbP+nRsb4fDD4dFHswXjR4+ueH2SJEmSWsemjAiufwFCFcbKVSuZ2DCRg3c+mL7v67v2HVOC\nX/0K9tkH/vAHuOkmQ6AkSZJUMOsMghHxRkQsXcOfN4APtVGNagMPz3+Y55c+T92QtdzemRLccw8M\nGwaf+lS2XuCvf52tGShJkiSpUNZ5a2hKqWdbFaJ8XVd/HVt325ojBxz57g9SymYDvegimDYNdtwR\nJk6Ek06CzptyZ7EkSZKkvGzKraHqIF5d9iq/evpXjP7IaDbrvFm2MSV48EHYd1849FBYsiSbFfTp\np+HUUw2BkiRJUoEZBMXNM25m+crl2dqBKcHDD8O//RscfDAsWpStDThnTjYraJcueZcrSZIkaRM5\nrCNKDSWG9h7K4Dmvwwkj4Pe/hz594Gc/y8LfZpvlXaIkSZKkVmQQrHL1i+vp/ucGJs/YBR4fAb17\nw1VXwec+B5tvnnd5kiRJkirAIFjNHnuMzc88gT/Uw6oPvg5XXAGnnw7duuVdmSRJkqQK8hnBavSX\nv8DIkbDvvvR6ZiE3njKETvPnw5e+ZAiUJEmSqoBBsJr89a9wxBGw994wbRrTz/kM25+d2PaiH8IW\nW+RdnSRJkqQ2YhCsBg0NcPTRsNde8D//A9/9Lsyfz1f3XEKvbbbjgB0OyLtCSZIkSW3IINiRzZgB\nxx4Le+4JU6fCt78Nzz0HX/saf1v5Gr+Z9xvGDhlLp/CvgSRJklRNnCymI5o1C771LbjtNnjf++Ci\ni7Ln/7ba6l+7TJo+iURizJAx+dUpSZIkKRcGwY7k6aezUb/Jk6F7d/j61+Gcc2Drrd+126q0igkN\nEzhwhwPZfqvt86lVkiRJUm68J7AjePZZOOUU2H13mDIFzjsP5s+Hiy9+TwgEmPrcVJ57/Tk+u+dn\ncyhWkiRJUt4cESyyefOysHfDDdC1K3zlK/DVr8I226zzsFJ9ia0234qjdz26jQqVJEmS1J4YBIvo\nuefgO9+BiROhSxf44hezUcDa2vUe+vrbr3Pn7DupG1JHty6uGShJkiRVI4NgkSxYkC39UCpBTQ2c\neSacfz707t3iU0x+ajJvN71N3Z51FSxUkiRJUntmECyChQvhkkvg2mshAk4/HS64APr02eBTXVd/\nHYNqBzG099AKFCpJkiSpCAyC7dnixVkAHD8eVq2Cujr42tdgu+026nRPvvQkj7/wOFcefCUR0crF\nSpIkSSoKg2B79NJLcOml8POfw4oVMHYsXHghbL/9Jp12Qv0EunTqwkmDTmqdOiVJkiQVkkGwPVmy\nBH7wAxg3DpYvz5aE+PrXYccdN/nUy1cu54Ynb+CoXY+i1xa9WqFYSZIkSUVlEGwPXn0VLrsMrroK\n3noLRo+G//ov2HnnVrvE3XPu5tW3XnXtQEmSJEkGwVy99hpcfjn8+Mfw5pswahR84xswYECrX6rU\nUKJPzz4ctONBrX5uSZIkScViEMzD66/DFVfAlVfCG2/A8cdnAXDgwIpcbtHSRdw/934u+PgF1HSq\nqcg1JEmSJBWHQbAt/eMf2ejf5Zdnr489Fi66CPbYo6KXvX769axKqxgzZExFryNJkiSpGAyCbaBm\n2TL43vey5wD//nc4+mj45jdh8OCKXzulRKmhxCc+/Al23rr1njmUJEmSVFydKnnyiDgkIuZExNyI\nOH8Nn28ZEXdHxPSImBkRY5t9tlVE3BERT0fE7Ij4WCVrrYjGRrj0UvYZNSpb/uHjH4cnnoC77mqT\nEAjw6IJHmfvaXOr2rGuT60mSJElq/yoWBCOiBhgHjAQGAqMiYvWH4M4EZqWUBgMjgB9FRNfyZz8G\n7k8p7QoMBmZXqtaKWbwYLryQpbvtBn/5C0yZAkOHtmkJpfoSPbv25JjdjmnT60qSJElqvyp5a+hH\ngbkppXkAETEZOAqY1WyfBPSMiAB6AK8BTRGxJbAfMAYgpbQcWF7BWiujf3945hlmLFjAiP/3/9r8\n8kvfWcrts25n9EdG071r9za/viRJkqT2qZK3hvYBnm/2fmF5W3M/BXYDXgBmAGenlFYBOwBLgAkR\nUR8Rv4iIYiaZVlgMfmPdNvM2lq1Y5m2hkiRJkt4lUkqVOXHEscAhKaXPld+fDOydUjprtX2GA18G\ndgIeIrsNdBfgT8DwlNKfI+LHwNKU0n+t4TqnAacB1NbW7jV58uSKfD2borGxkR49erT5dc+qP4vG\npkYmDJtANuiqlsirX9p49qx47Fmx2K/isWfFY8+Kp732bP/9938ipTRsfftV8tbQRUC/Zu/7lrc1\nNxb4fsrS6NyImA/sCiwAFqaU/lze7w7gPZPNAKSUxgPjAYYNG5ZGjBjRal9Aa5k6dSptXdfsJbOZ\n+buZXHbQZey/7/5teu2iy6Nf2jT2rHjsWbHYr+KxZ8Vjz4qn6D2r5K2h04D+EbFDeQKYE4Epq+2z\nADgQICJqgQHAvJTSi8DzETGgvN+BvPvZQq3HhIYJdO7UmdGDRuddiiRJkqR2pmIjgimlpog4C3gA\nqAFKKaWZEXFG+fOrgYuBiRExAwjgvJTSK+VTfAG4qRwi55GNHqoFVqxcwaTpkzh8l8Op7VGbdzmS\nJEmS2pmKLiifUroPuG+1bVc3e/0C8Mm1HNsArPfeVr3Xfc/ex8tvvkzdECeJkSRJkvReFV1QXvko\nNZTYtse2jOw/Mu9SJEmSJLVDBsEO5sXGF7n3mXs5dfCpdO5U0QFfSZIkSQVlEOxgbph+AyvTSsYO\n8ZFKSZIkSWtmEOxAUkqUGkoM7zecAb0GrP8ASZIkSVXJINiB/Gnhn3j6laep29NJYiRJkiStnUGw\nAynVl+jepTvHDTwu71IkSZIktWMGwQ6icXkjk2dO5vjdj6fnZj3zLkeSJElSO2YQ7CDumHUHjcsb\nvS1UkiRJ0noZBDuIUn2JXT6wC8P7Dc+7FEmSJEntnEGwA3jm1Wd4dMGj1A2pIyLyLkeSJElSO2cQ\n7AAmNkykJmo4ZfApeZciSZIkqQAMggXXtKqJSdMnMbL/SHr37J13OZIkSZIKwCBYcA/+74O88MYL\n1A1xkhhJkiRJLWMQLLhSfYltttiGw3Y5LO9SJEmSJBWEQbDAlry5hClzpnDyoJPpWtM173IkSZIk\nFYRBsMBufPJGVqxa4dqBkiRJkjaIQbCgUkpcV38de/fZm90/uHve5UiSJEkqEINgQT3+wuPMXDLT\n0UBJkiRJG8wgWFCl+hLdOnfjhN1PyLsUSZIkSQVjECygZSuWcfNTN3PswGPZcvMt8y5HkiRJUsEY\nBAvortl3sfSdpd4WKkmSJGmjGAQLqNRQYsf378h+H94v71IkSZIkFZBBsGDm/X0ej8x/hLFDxtIp\nbJ8kSZKkDWeSKJiJDRMJgjFDxuRdiiRJkqSCMggWyMpVK5nYMJGDdz6Yvu/rm3c5kiRJkgrKIFgg\nD89/mOeXPk/dECeJkSRJkrTxDIIFUqovsXW3rTlywJF5lyJJkiSpwAyCBfHaW69x19N3Mfojo9ms\n82Z5lyNJkiSpwAyCBXHzjJtZvnK5awdKkiRJ2mQGwYIo1ZcY2nsog7cdnHcpkiRJkgquokEwIg6J\niDkRMTcizl/D51tGxN0RMT0iZkbE2NU+r4mI+oi4p5J1tnf1i+upf7HeSWIkSZIktYqKBcGIqAHG\nASOBgcCoiBi42m5nArNSSoOBEcCPIqJrs8/PBmZXqsaiKNWX2KxmMz7zkc/kXYokSZKkDqCSI4If\nBeamlOallJYDk4GjVtsnAT0jIoAewGtAE0BE9AUOA35RwRrbvbeb3uamGTfx6d0+zfu7vT/vciRJ\nkiR1AJUMgn2A55u9X1je1txPgd2AF4AZwNkppVXlz64EzgVWUcV+/fSv+fvbf3eSGEmSJEmtpnPO\n1z8YaAAOAHYCHoqIR4H9gJdTSk9ExIh1nSAiTgNOA6itrWXq1KkVLXhjNDY2bnRdlz15GbWb1dLp\nb52YumDjzqENsyn9Uj7sWfHYs2KxX8Vjz4rHnhVP0XtWySC4COjX7H3f8rbmxgLfTyklYG5EzAd2\nBYYDR0bEocDmwPsi4saU0ujVL5JSGg+MBxg2bFgaMWJEq38hm2rq1KlsTF0L/rGAJ373BN/4xDc4\nYMQBrV+Y1mhj+6X82LPisWfFYr+Kx54Vjz0rnqL3rJK3hk4D+kfEDuUJYE4Epqy2zwLgQICIqAUG\nAPNSSheklPqmlLYvH/fImkJgRzepYRKJxJghY/IuRZIkSVIHUrERwZRSU0ScBTwA1ACllNLMiDij\n/PnVwMXAxIiYAQRwXkrplUrVVCSr0iomNEzgwB0OZPutts+7HEmSJEkdSEWfEUwp3Qfct9q2q5u9\nfgH45HrOMRWYWoHy2rWpz01l/uvz+e4B3827FEmSJEkdTEUXlNfGK9WX2GrzrTh616PzLkWSJElS\nB2MQbIdef/t17px9J5/Z4zN069It73IkSZIkdTAGwXZo8lOTebvpbdcOlCRJklQRBsF2qFRfYlDt\nIIb2Hpp3KZIkSZI6IINgOzPjpRlMe2EadUPqiIi8y5EkSZLUARkE25kJDRPo0qkLJw06Ke9SJEmS\nJHVQBsF2ZPnK5dzw5A0ctetR9NqiV97lSJIkSeqgDILtyN1z7uaVZa/w2T0/m3cpkiRJkjowg2A7\nUmoo0adnHw7a8aC8S5EkSZLUgRkE24lFSxdx/9z7GTNkDDWdavIuR5IkSVIHZhBsJ66ffj2r0irG\nDBmTdymSJEmSOjiDYDuQUqLUUOITH/4EO2+9c97lSJIkSergDILtwB8W/IG5r82lbs+6vEuRJEmS\nVAUMgu1AqaFEz649OWa3Y/IuRZIkSVIVMAjm7I133uC2mbcxao9RdO/aPe9yJEmSJFUBg2DObp15\nK8tWLPO2UEmSJEltxiCYs1J9iYHbDOSjfT6adymSJEmSqoRBMEezl8zmsYWPUTekjojIuxxJkiRJ\nVcIgmKMJDRPo3KkzoweNzrsUSZIkSVXEIJiTFStXcP306zl8l8Op7VGbdzmSJEmSqohBMCf/Pfe/\neenNl6gb4iQxkiRJktqWQTAnpfoS2/bYlpH9R+ZdiiRJkqQqYxDMwYuNL3LPM/dw6uBT6dypc97l\nSJIkSaoyBsEc3DD9BlamlYwdMjbvUiRJkiRVIYNgG0spUWooMbzfcAb0GpB3OZIkSZKqkEGwjf1p\n4Z94+pWnqdvTSWIkSZIk5cMg2MZK9SW6d+nOcQOPy7sUSZIkSVXKINiG3lz+JpNnTub43Y+n52Y9\n8y5HkiRJUpUyCLahO2bdQePyRm8LlSRJkpQrg2AbKjWU2OUDuzC83/C8S5EkSZJUxSoaBCPikIiY\nExFzI+L8NXy+ZUTcHRHTI2JmRIwtb+8XEb+NiFnl7WdXss628Oyrz/L7v/2euiF1RETe5UiSJEmq\nYhULghFRA4wDRgIDgVERMXC13c4EZqWUBgMjgB9FRFegCfhKSmkgsA9w5hqOLZQJDROoiRpOGXxK\n3qVIkiRJqnKVHBH8KDA3pTQvpbQcmAwctdo+CegZ2RBZD+A1oCmltDil9FeAlNIbwGygTwVrraiV\naSWTpk9iZP+R9O7ZO+9yJEmSJFW5SClV5sQRxwKHpJQ+V35/MrB3SumsZvv0BKYAuwI9gRNSSveu\ndp7tgd8De6SUlq7hOqcBpwHU1tbuNXny5Ip8PZti6qKpfGvut/j27t/m33r9W97laD0aGxvp0aNH\n3mVoA9iz4rFnxWK/iseeFY89K5722rP999//iZTSsPXt17ktilmHg4EG4ABgJ+ChiHj0n4EvInoA\ndwJfWlMIBEgpjQfGAwwbNiyNGDGiLereIBeNu4htttiG8z51Hl1ruuZdjtZj6tSptMe/R1o7e1Y8\n9qxY7Ffx2LPisWfFU/SeVfLW0EVAv2bv+5a3NTcW+GXKzAXmk40OEhFdyELgTSmlX1awzopa8uYS\n/vjqHzl50MmGQEmSJEntQiWD4DSgf0TsUJ4A5kSy20CbWwAcCBARtcAAYF75mcHrgNkppcsrWGPF\n3TTjJppSk2sHSpIkSWo3KhYEU0pNwFnAA2STvdyWUpoZEWdExBnl3S4G9o2IGcDDwHkppVeA4cDJ\nwAER0VD+c2ilaq2UlBKl+hK79dyN3T+4e97lSJIkSRJQ4WcEU0r3Afettu3qZq9fAD65huP+ABR+\nsb0nX3qSGS/P4Mv9v5x3KZIkSZL0L3lPFtOhDaodxLT/mMZLs17KuxRJkiRJ+pdKPiNY9SKCdMhq\nbAAABwpJREFUYR8aRvfO3fMuRZIkSZL+xSAoSZIkSVXGIChJkiRJVcYgKEmSJElVxiAoSZIkSVXG\nIChJkiRJVcYgKEmSJElVxiAoSZIkSVXGIChJkiRJVcYgKEmSJElVxiAoSZIkSVUmUkp519BqImIJ\n8Le861iDXsAreRehFrNfxWPPiseeFYv9Kh57Vjz2rHjaa88+nFLaZn07dagg2F5FxOMppWF516GW\nsV/FY8+Kx54Vi/0qHntWPPaseIreM28NlSRJkqQqYxCUJEmSpCpjEGwb4/MuQBvEfhWPPSsee1Ys\n9qt47Fnx2LPiKXTPfEZQkiRJkqqMI4KSJEmSVGUMghUUEYdExJyImBsR5+ddj9YtIvpFxG8jYlZE\nzIyIs/OuSesXETURUR8R9+Rdi9YvIraKiDsi4umImB0RH8u7Jq1bRJxT/p74VETcEhGb512T3i0i\nShHxckQ81Wzb1hHxUEQ8W/7v+/OsUe+2lp79sPy98cmIuCsitsqzRv2fNfWr2WdfiYgUEb3yqG1T\nGAQrJCJqgHHASGAgMCoiBuZbldajCfhKSmkgsA9wpj0rhLOB2XkXoRb7MXB/SmlXYDD2rl2LiD7A\nF4FhKaU9gBrgxHyr0hpMBA5Zbdv5wMMppf7Aw+X3aj8m8t6ePQTskVIaBDwDXNDWRWmtJvLefhER\n/YBPAgvauqDWYBCsnI8Cc1NK81JKy4HJwFE516R1SCktTin9tfz6DbJ/oPbJtyqtS0T0BQ4DfpF3\nLVq/iNgS2A+4DiCltDyl9Hq+VakFOgPdIqIzsAXwQs71aDUppd8Dr622+ShgUvn1JODoNi1K67Sm\nnqWUHkwpNZXf/gno2+aFaY3W8v8YwBXAuUAhJ10xCFZOH+D5Zu8XYqgojIjYHtgT+HO+lWg9riT7\nBrwq70LUIjsAS4AJ5dt5fxER3fMuSmuXUloEXEb22+7FwD9SSg/mW5VaqDaltLj8+kWgNs9itMHq\ngP/OuwitXUQcBSxKKU3Pu5aNZRCUVhMRPYA7gS+llJbmXY/WLCIOB15OKT2Rdy1qsc7AUODnKaU9\ngTfxdrV2rfxc2VFkIf5DQPeIGJ1vVdpQKZsivpAjFtUoIi4ke1zlprxr0ZpFxBbA14Bv5F3LpjAI\nVs4ioF+z933L29SORUQXshB4U0rpl3nXo3UaDhwZEc+R3Xp9QETcmG9JWo+FwMKU0j9H2u8gC4Zq\nv/4dmJ9SWpJSWgH8Etg355rUMi9FRG+A8n9fzrketUBEjAEOB05KrvHWnu1E9guy6eV/h/QF/hoR\n2+Za1QYyCFbONKB/ROwQEV3JHq6fknNNWoeICLJnl2anlC7Pux6tW0rpgpRS35TS9mT/fz2SUnKk\noh1LKb0IPB8RA8qbDgRm5ViS1m8BsE9EbFH+HnkgTvBTFFOAU8uvTwV+nWMtaoGIOITscYcjU0rL\n8q5Ha5dSmpFS+mBKafvyv0MWAkPLP+cKwyBYIeWHfc8CHiD7oXlbSmlmvlVpPYYDJ5ONLDWU/xya\nd1FSB/MF4KaIeBIYAnwv53q0DuXR2zuAvwIzyP7dMD7XovQeEXEL8BgwICIWRsRnge8DB0XEs2Qj\nu9/Ps0a921p69lOgJ/BQ+d8gV+dapP5lLf0qvHDUWZIkSZKqiyOCkiRJklRlDIKSJEmSVGUMgpIk\nSZJUZQyCkiRJklRlDIKSJEmSVGUMgpIkARGxstnSMQ0RcX4rnnv7iHiqtc4nSdKm6px3AZIktRNv\npZSG5F2EJEltwRFBSZLWISKei4gfRMSMiPhLROxc3r59RDwSEU9GxMMRsV15e21E3BUR08t/9i2f\nqiYiro2ImRHxYER0K+//xYiYVT7P5Jy+TElSlTEISpKU6bbaraEnNPvsHymljwA/Ba4sb7sKmJRS\nGgTcBPykvP0nwO9SSoOBocDM8vb+wLiU0u7A68Ax5e3nA3uWz3NGpb44SZKai5RS3jVIkpS7iGhM\nKfVYw/bngANSSvMiogvwYkrpAxHxCtA7pbSivH1xSqlXRCwB+qaU3ml2ju2Bh1JK/cvvzwO6pJS+\nExH3A43Ar4BfpZQaK/ylSpLkiKAkSS2Q1vJ6Q7zT7PVK/u85/cOAcWSjh9Miwuf3JUkVZxCUJGn9\nTmj238fKr/8InFh+fRLwaPn1w8DnASKiJiK2XNtJI6IT0C+l9FvgPGBL4D2jkpIktTZ/6yhJUqZb\nRDQ0e39/SumfS0i8PyKeJBvVG1Xe9gVgQkR8FVgCjC1vPxsYHxGfJRv5+zyweC3XrAFuLIfFAH6S\nUnq91b4iSZLWwmcEJUlah/IzgsNSSq/kXYskSa3FW0MlSZIkqco4IihJkiRJVcYRQUmSJEmqMgZB\nSZIkSaoyBkFJkiRJqjIGQUmSJEmqMgZBSZIkSaoyBkFJkiRJqjL/HxuFoa7pA2lgAAAAAElFTkSu\nQmCC\n", 1212 | "text/plain": [ 1213 | "" 1214 | ] 1215 | }, 1216 | "metadata": {}, 1217 | "output_type": "display_data" 1218 | } 1219 | ], 1220 | "source": [ 1221 | "fig = plt.figure(figsize=(15, 5))\n", 1222 | "plt.plot(fitted_model.history['acc'], 'g', label=\"accuracy on train set\")\n", 1223 | "plt.plot(fitted_model.history['val_acc'], 'r', label=\"accuracy on validation sete\")\n", 1224 | "plt.grid(True)\n", 1225 | "plt.title('Training Accuracy vs. Validation Accuracy - VGG16')\n", 1226 | "plt.xlabel('Epochs')\n", 1227 | "plt.ylabel('Loss')\n", 1228 | "plt.legend()" 1229 | ] 1230 | }, 1231 | { 1232 | "cell_type": "markdown", 1233 | "metadata": {}, 1234 | "source": [ 1235 | "Many deep learning pioneers encourage to use pre-trained networks for classification tasks. In fact, this usually leverages the training of a very large network on a very large dataset. So the motto is \n", 1236 | "\n", 1237 | "> Don't be a hero. Don't reinvent the wheel." 1238 | ] 1239 | }, 1240 | { 1241 | "cell_type": "markdown", 1242 | "metadata": {}, 1243 | "source": [ 1244 | "# Conclusion" 1245 | ] 1246 | }, 1247 | { 1248 | "cell_type": "markdown", 1249 | "metadata": {}, 1250 | "source": [ 1251 | "This article was an opportunity to go through the theory behind convolutional neural networks and explain each of their principal components in more details.\n", 1252 | "\n", 1253 | "This was also a hands-on guide to setup a deep learning dedicated environment on AWS and develop an end-to-end model from scratch as well as an enhanced model based on a pre-trained one. \n", 1254 | "\n", 1255 | "Using python for deep learning is extermely fun. Keras made it easier for preprocessing the data and building up the layers. \n", 1256 | "Keep in mind that if you want someday to build custom neural net components, you'll have to switch to another framework. \n", 1257 | "\n", 1258 | "I hope this gave you a practical intuition about the convnets mechanisms and a strong appetite to learn more.\n", 1259 | "\n", 1260 | "Convnets are amazingly powerful. Anyone working on computer vision believes in their robustness and efficiency.\n", 1261 | "\n", 1262 | "Their applications are getting a larger scope. NLP practictioners are now the ones who are switching to convnets. Here are some of their applications:\n", 1263 | "\n", 1264 | "- Text classification using CNNs : link \n", 1265 | "- Automated image captioning (Image + Text): link \n", 1266 | "- Text classification at **character level**: link " 1267 | ] 1268 | }, 1269 | { 1270 | "cell_type": "markdown", 1271 | "metadata": {}, 1272 | "source": [ 1273 | "# References\n", 1274 | "\n", 1275 | "Here's a list of some references I used to learn about neural nets and convnets:\n", 1276 | "\n", 1277 | "1. neuralnetworksanddeeplearning.com : By far the best notes notes neural networks and deep learning. Recommend this to anyone who want to start learning about neural nets.\n", 1278 | "\n", 1279 | "2. CS231n Convolutional Neural Networks for Visual Recognition : Andrej Karpathy's lectures at Stanford. Great mathematical focus.\n", 1280 | "\n", 1281 | "3. A Beginner Guide to Understanding Neural Networks: A three-part post explaining CNNs, from the basic high level inuition to the architecture details. Very interesting read. Learnt a lot from it.\n", 1282 | " + part 1 \n", 1283 | " + part 2 \n", 1284 | " + part 3 \n", 1285 | "\n", 1286 | "4. Running jupyter notebooks on GPU on AWS \n", 1287 | "\n", 1288 | "5. Building powerful image classification models using very little data \n", 1289 | "\n", 1290 | "6. CatdogNet - Keras Convnet Starter \n", 1291 | "\n", 1292 | "7. A quick introduction to Neural Networks \n", 1293 | "\n", 1294 | "8. An Intuitive Explanation of Convolutional Neural Networks\n", 1295 | "\n", 1296 | "9. Visualizing parts of Convolutional Neural Networks using Keras and Cats \n" 1297 | ] 1298 | } 1299 | ], 1300 | "metadata": { 1301 | "anaconda-cloud": {}, 1302 | "kernelspec": { 1303 | "display_name": "Python 3", 1304 | "language": "python", 1305 | "name": "python3" 1306 | }, 1307 | "language_info": { 1308 | "codemirror_mode": { 1309 | "name": "ipython", 1310 | "version": 3 1311 | }, 1312 | "file_extension": ".py", 1313 | "mimetype": "text/x-python", 1314 | "name": "python", 1315 | "nbconvert_exporter": "python", 1316 | "pygments_lexer": "ipython3", 1317 | "version": "3.5.2" 1318 | } 1319 | }, 1320 | "nbformat": 4, 1321 | "nbformat_minor": 2 1322 | } 1323 | --------------------------------------------------------------------------------