├── autograd.ipynb ├── classifier.ipynb ├── getting_started.ipynb └── neural_networks.ipynb /autograd.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "language_info": { 4 | "codemirror_mode": { 5 | "name": "ipython", 6 | "version": 3 7 | }, 8 | "file_extension": ".py", 9 | "mimetype": "text/x-python", 10 | "name": "python", 11 | "nbconvert_exporter": "python", 12 | "pygments_lexer": "ipython3", 13 | "version": "3.8.3-final" 14 | }, 15 | "orig_nbformat": 2, 16 | "kernelspec": { 17 | "name": "python3", 18 | "display_name": "Python 3" 19 | } 20 | }, 21 | "nbformat": 4, 22 | "nbformat_minor": 2, 23 | "cells": [ 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Autograd: Automatic Differentiation\n", 29 | "Central to all neural networks in PyTorch is the autograd package. Let’s first briefly visit this, and we will then go to training our first neural network.\n", 30 | "\n", 31 | "The autograd package provides automatic differentiation for all operations on Tensors. It is a define-by-run framework, which means that your backprop is defined by how your code is run, and that every single iteration can be different.\n", 32 | "\n", 33 | "Let us see this in more simple terms with some examples.\n", 34 | "\n" 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": {}, 40 | "source": [ 41 | "## Tensor\n", 42 | "torch.Tensor is the central class of the package. If you set its attribute .requires_grad as True, it starts to track all operations on it. When you finish your computation you can call .backward() and have all the gradients computed automatically. The gradient for this tensor will be accumulated into .grad attribute.\n", 43 | "\n", 44 | "To stop a tensor from tracking history, you can call .detach() to detach it from the computation history, and to prevent future computation from being tracked.\n", 45 | "\n", 46 | "To prevent tracking history (and using memory), you can also wrap the code block in with torch.no_grad():. This can be particularly helpful when evaluating a model because the model may have trainable parameters with requires_grad=True, but for which we don’t need the gradients.\n", 47 | "\n", 48 | "There’s one more class which is very important for autograd implementation - a Function.\n", 49 | "\n", 50 | "Tensor and Function are interconnected and build up an acyclic graph, that encodes a complete history of computation. Each tensor has a .grad_fn attribute that references a Function that has created the Tensor (except for Tensors created by the user - their grad_fn is None).\n", 51 | "\n", 52 | "If you want to compute the derivatives, you can call .backward() on a Tensor. If Tensor is a scalar (i.e. it holds a one element data), you don’t need to specify any arguments to backward(), however if it has more elements, you need to specify a gradient argument that is a tensor of matching shape." 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 1, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "import torch" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 2, 67 | "metadata": { 68 | "tags": [] 69 | }, 70 | "outputs": [ 71 | { 72 | "output_type": "stream", 73 | "name": "stdout", 74 | "text": "tensor([[1., 1.],\n [1., 1.]], requires_grad=True)\n" 75 | } 76 | ], 77 | "source": [ 78 | "# Create a tensor and set requires_grad=True to track computation with it\n", 79 | "x = torch.ones(2, 2, requires_grad=True)\n", 80 | "print(x)" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": 3, 86 | "metadata": { 87 | "tags": [] 88 | }, 89 | "outputs": [ 90 | { 91 | "output_type": "stream", 92 | "name": "stdout", 93 | "text": "tensor([[3., 3.],\n [3., 3.]], grad_fn=)\n" 94 | } 95 | ], 96 | "source": [ 97 | "y = x + 2\n", 98 | "print(y)" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": 4, 104 | "metadata": { 105 | "tags": [] 106 | }, 107 | "outputs": [ 108 | { 109 | "output_type": "stream", 110 | "name": "stdout", 111 | "text": "\n" 112 | } 113 | ], 114 | "source": [ 115 | "# y was created as a result of an operation, so it has a grad_fn.\n", 116 | "print(y.grad_fn)" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 5, 122 | "metadata": { 123 | "tags": [] 124 | }, 125 | "outputs": [ 126 | { 127 | "output_type": "stream", 128 | "name": "stdout", 129 | "text": "tensor([[27., 27.],\n [27., 27.]], grad_fn=) tensor(27., grad_fn=)\n" 130 | } 131 | ], 132 | "source": [ 133 | "z = y * y * 3\n", 134 | "out = z.mean()\n", 135 | "\n", 136 | "print(z, out)" 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": 6, 142 | "metadata": { 143 | "tags": [] 144 | }, 145 | "outputs": [ 146 | { 147 | "output_type": "stream", 148 | "name": "stdout", 149 | "text": "False\nTrue\n\n" 150 | } 151 | ], 152 | "source": [ 153 | "a = torch.randn(2, 2)\n", 154 | "a = ((a * 3) / (a - 1))\n", 155 | "print(a.requires_grad)\n", 156 | "a.requires_grad_(True)\n", 157 | "print(a.requires_grad)\n", 158 | "b = (a * a).sum()\n", 159 | "print(b.grad_fn)" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "## Gradients\n", 167 | "Let’s backprop now. Because out contains a single scalar, out.backward() is equivalent to out.backward(torch.tensor(1.))." 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": 7, 173 | "metadata": {}, 174 | "outputs": [], 175 | "source": [ 176 | "out.backward()" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": 8, 182 | "metadata": { 183 | "tags": [] 184 | }, 185 | "outputs": [ 186 | { 187 | "output_type": "stream", 188 | "name": "stdout", 189 | "text": "tensor([[4.5000, 4.5000],\n [4.5000, 4.5000]])\n" 190 | } 191 | ], 192 | "source": [ 193 | "# Print gradients d(out)/dx\n", 194 | "\n", 195 | "print(x.grad)" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": 9, 201 | "metadata": { 202 | "tags": [] 203 | }, 204 | "outputs": [ 205 | { 206 | "output_type": "stream", 207 | "name": "stdout", 208 | "text": "tensor([ -83.0253, -1423.6444, 1307.3494], grad_fn=)\n" 209 | } 210 | ], 211 | "source": [ 212 | "x = torch.randn(3, requires_grad=True)\n", 213 | "\n", 214 | "y = x * 2\n", 215 | "while y.data.norm() < 1000:\n", 216 | " y = y * 2\n", 217 | "\n", 218 | "print(y)" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": 10, 224 | "metadata": { 225 | "tags": [] 226 | }, 227 | "outputs": [ 228 | { 229 | "output_type": "stream", 230 | "name": "stdout", 231 | "text": "tensor([2.0480e+02, 2.0480e+03, 2.0480e-01])\n" 232 | } 233 | ], 234 | "source": [ 235 | "# Now in this case y is no longer a scalar. torch.autograd could not compute the full Jacobian directly, but if we just want the vector-Jacobian product, simply pass the vector to backward as argument:\n", 236 | "v = torch.tensor([0.1, 1.0, 0.0001], dtype=torch.float)\n", 237 | "y.backward(v)\n", 238 | "\n", 239 | "print(x.grad)" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": 11, 245 | "metadata": { 246 | "tags": [] 247 | }, 248 | "outputs": [ 249 | { 250 | "output_type": "stream", 251 | "name": "stdout", 252 | "text": "True\nTrue\nFalse\n" 253 | } 254 | ], 255 | "source": [ 256 | "# You can also stop autograd from tracking history on Tensors with .requires_grad=True either by wrapping the code block in with torch.no_grad():\n", 257 | "\n", 258 | "print(x.requires_grad)\n", 259 | "print((x ** 2).requires_grad)\n", 260 | "\n", 261 | "with torch.no_grad():\n", 262 | " print((x ** 2).requires_grad)" 263 | ] 264 | }, 265 | { 266 | "cell_type": "code", 267 | "execution_count": 12, 268 | "metadata": { 269 | "tags": [] 270 | }, 271 | "outputs": [ 272 | { 273 | "output_type": "stream", 274 | "name": "stdout", 275 | "text": "True\nFalse\ntensor(True)\n" 276 | } 277 | ], 278 | "source": [ 279 | "# Or by using .detach() to get a new Tensor with the same content but that does not require gradients:\n", 280 | "\n", 281 | "print(x.requires_grad)\n", 282 | "y = x.detach()\n", 283 | "print(y.requires_grad)\n", 284 | "print(x.eq(y).all())" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": null, 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [] 293 | } 294 | ] 295 | } -------------------------------------------------------------------------------- /classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "language_info": { 4 | "codemirror_mode": { 5 | "name": "ipython", 6 | "version": 3 7 | }, 8 | "file_extension": ".py", 9 | "mimetype": "text/x-python", 10 | "name": "python", 11 | "nbconvert_exporter": "python", 12 | "pygments_lexer": "ipython3", 13 | "version": "3.8.3-final" 14 | }, 15 | "orig_nbformat": 2, 16 | "kernelspec": { 17 | "name": "python3", 18 | "display_name": "Python 3" 19 | } 20 | }, 21 | "nbformat": 4, 22 | "nbformat_minor": 2, 23 | "cells": [ 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Training a classifier\n", 29 | "This is it. You have seen how to define neural networks, compute loss and make updates to the weights of the network.\n", 30 | "\n", 31 | "Now you might be thinking,\n", 32 | "\n", 33 | "## What about data?\n", 34 | "Generally, when you have to deal with image, text, audio or video data, you can use standard python packages that load data into a numpy array. Then you can convert this array into a torch.*Tensor.\n", 35 | "\n", 36 | "* For images, packages such as Pillow, OpenCV are useful\n", 37 | "* For audio, packages such as scipy and librosa\n", 38 | "* For text, either raw Python or Cython based loading, or NLTK and SpaCy are useful\n", 39 | "\n", 40 | "Specifically for vision, we have created a package called torchvision, that has data loaders for common datasets such as Imagenet, CIFAR10, MNIST, etc. and data transformers for images, viz., torchvision.datasets and torch.utils.data.DataLoader.\n", 41 | "\n", 42 | "This provides a huge convenience and avoids writing boilerplate code.\n", 43 | "\n", 44 | "For this tutorial, we will use the CIFAR10 dataset. It has the classes: ‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’, ‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’. The images in CIFAR-10 are of size 3x32x32, i.e. 3-channel color images of 32x32 pixels in size.\n", 45 | "\n" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "## Training an image classifier\n", 53 | "We will do the following steps in order:\n", 54 | "\n", 55 | "* Load and normalizing the CIFAR10 training and test datasets using torchvision\n", 56 | "* Define a Convolutional Neural Network\n", 57 | "* Define a loss function\n", 58 | "* Train the network on the training data\n", 59 | "* Test the network on the test data" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "### 1. Loading and normalizing CIFAR10\n", 67 | "Using torchvision, it’s extremely easy to load CIFAR10." 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": 1, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "import torch\n", 77 | "import torchvision\n", 78 | "import torchvision.transforms as transforms" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 2, 84 | "metadata": { 85 | "tags": [] 86 | }, 87 | "outputs": [], 88 | "source": [ 89 | "transform = transforms.Compose(\n", 90 | " [transforms.ToTensor(),\n", 91 | " transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n", 92 | "try:\n", 93 | " trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n", 94 | " download=False, transform=transform)\n", 95 | " trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n", 96 | " shuffle=True, num_workers=2)\n", 97 | "\n", 98 | " testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n", 99 | " download=False, transform=transform)\n", 100 | " testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n", 101 | " shuffle=False, num_workers=2)\n", 102 | "except NameError:\n", 103 | " trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n", 104 | " download=True, transform=transform)\n", 105 | " trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n", 106 | " shuffle=True, num_workers=2)\n", 107 | "\n", 108 | " testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n", 109 | " download=True, transform=transform)\n", 110 | " testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n", 111 | " shuffle=False, num_workers=2)\n", 112 | "classes = ('plane', 'car', 'bird', 'cat',\n", 113 | " 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": 3, 119 | "metadata": { 120 | "tags": [] 121 | }, 122 | "outputs": [ 123 | { 124 | "output_type": "display_data", 125 | "data": { 126 | "text/plain": "
", 127 | "image/svg+xml": "\n\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n", 128 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9u48sy5be91sRkZlV1d177/O6r7nzEMgZEtIAhABKMgQBAgQBMgTQI0Q5MgRcS77GE8bjHyBrDEKSo4cnGQRkCBDkyCAdSqSkoQYccu659zz2s7urKjPjtWSsyKzq3vuc02efe3V4Mb2A7uquysqMjIz4Yq1vPUJUlUd5lEd5lEf5zRP3fTfgUR7lUR7lUd5PHgH8UR7lUR7lN1QeAfxRHuVRHuU3VB4B/FEe5VEe5TdUHgH8UR7lUR7lN1QeAfxRHuVRHuU3VL4TgIvIfyAifyoifyYif/SratSjPMqjPMqjfLPI+8aBi4gH/inw7wOfAv8A+Duq+n/96pr3KI/yKI/yKF8l4Tt8998E/kxV/xmAiPx3wN8CvhLAd7udPnv27Dtc8lEe5VEe5S+ffPbZZy9U9ZP7738XAP8t4Odn/38K/Ftf94Vnz57xs5/97Dtc8lEe5VEe5S+f/PEf//G/eNf7v3Ynpoj8TET+oYj8w+Px+Ou+3KM8yqM8yl8a+S4A/gvgt8/+/2l7746o6p+o6t9U1b+52+2+w+Ue5VEe5VEe5Vy+C4XyD4DfF5F/BQPu/wj4j7/NCbabno+eXRKCR2tFAVRZ3KoiYn8onN5tom+980DR5YR3vr9cCuTtr7zrLLqc63T82l7k9PbdQ6i1Ukuh1kqKkapK3/eErkNEcGLfVW0XETl9vZ3/dj9ysx9Z/M8iwnDxjH572S557+bObk0eeH9/GaSWxHj7mpLm9b0YI9c3N+RScCLW5csr1tci2PhRxYnQ9x3eCc4J3p3G7HK8fV9AnPW+OHsW4nBiOpQs47J9WQS7vl3+NJzauKu1ogqqlVq1fUvvXNNO0nQ059aTWODC3QFyd+7p+cXODpLTdVo7tFS0Ks51OD+cjhWhXD6jXDy5d5EzkXfNa3nrf1m/uvyuKLr2jTX11DZVRRRcqbROQvTuqbUqWut6VtVKjNHmZy2UWhEB77w9cxyiQs6ZeTpSa0W0gFaoBUoErWhOUEtrnd0j4tvcc4g4UHt+51jUBgZd8Hz49Amb4awvv0beG8BVNYvIfwb8z4AH/p6q/pNvc45PPrzi3/k3/jpXFxtSztRSWn/bg3Jt0GnVddAtn6nqOpBOYLrIacTLGZJq6zCtmftDR1oHirgViPX+YDrdO7UNDEVskIngnGsTXNaRparWHGfvxWliHifmaeLVy5fknPnw4495+uwpznu6rgcRtBQbiMv5MIAA+Mf/9Bf8H//Pz9e+EOf58Ke/z4c//Wuna+q9lWPtFrlzv79O+baL4v/fMu3f8Iv/+3/n8ObL9b2b/Z5/9I//CbeHPV3whGDP1HtBHPjgcU7QUqm50neBTz54wmbo2QyeTe8BhWr9773HOY94jw+9gbjvEHGE0BFC3wC6ICiiBaHindAHhxPonOCljbuqDWwStRRSycSYUGxMqoILHucDeAe+N/DuAxo8aEFrsbGr64yw3wqqZQVBrQuQc762tDlXoVTSNFFTph8+pN98bAAFqDjm3/59pr/6N+yrDSxXQBaozk5aWwtEBSrQ5hQqSDt+vSZKlkSVjAOCgC69p0LVSikFVyr9ccblitTangdoW33znKgpt36r5Jx5+fol4zgy15k5jXjn2G52BB/w2uEJ7I/XfP7LG9I84eqI1IjGAzq+hjRTb19DHKkIBRDx0A3gPMF1BNdRSyHHGdVqP9SmvDmuLi/4t//1v8Fv/fAtf+U75bto4Kjq3wf+/vt+vwuep1c7nl5tSSmRc1nOCzQAZxm4dbmqDa5zID9D4vvaqoGyrdr23YrWsg7a0+ELsNmEVZW3FJCz+z5rg2uKssN507DELedgvYZ4A+HJK4GCJ7P3IFXZDp7LiwHvA/3QI4hpWLU2AHdNmbJrbYaOc70EEUK/Zbh4euq/dwD4OWifrIVfj9w9/b+cAF5rQfzdKVBKYX88cHu7p+89IXicowG4EIK3xbgotRRK7ogXPd4pnfcU703ba4AhdAgVRFH1iNiYMVFETFdzbXyKmGbnEIKAc0Jwapp9VSrV5oMUihQqGUemqiLLvEHbFTxIsWs7hzjXxqwB+AJqqzIjCto+a22+o/uuCo2u4FPKTM2Z2qW7nSugw5Zy9QwQA19tI0EFdUpt3bAAOBWktmu0D0Xd2gwDO6VIpEq277RzFHVUhFqyAXjK+NzuvxSkVFsTEFSV7IQqzgCcShJhqo6xClNWxlwJTqAYSHqEgGMswu1UiHPGlYjUGZ0n9HCAPFNvb9B5pFgvg/NItzUA9z3B9dScSfNkOHQO4O355JIfPIa/E4B/VzHTRZuGffZ++6c2ADNz8R32FwsQnUzHFYjtjRPNwLKCO1TaYD87/vzaWq11evbe+tn699KMdk6tlLxM2nrHXBUByWYVHA9Hbq+vOewPfPH558Q50g09w2Yw7RtbALSa+XcCcEHE+qGUcta683bXO+092WanNi/a/Jndtn5+3q9fLw84/s768X5k19ty/1pfwRN9oyz9Ud+iCQTBuYALAXFupSKcW+gUO0a80PlA3wV88Hgv1KqkVNbzCIIXh/iACx2h6xHnER8QcQbKatZgrQnR2q6hoO7s/pxd2NkkR5WgFSkOhVXz1sYpOO9wzoPIqrFLbdZqFWoVUCgpGYg4h/Meg/a2lChULQ1YWzuaDlVzJsdEzYV0HCkxEXyCMxeXNIux68RAs96bZ2IL4p0nVlnb6WqjPZoioioUNeumU2/z1ynq26LVNHaNhXoccePE8MUXuGnCzyMuzpSqTKlSqiJzQlIx48RDyJnd6xcwjfg84+OEc45hs8G7QOcHOj9Q97c8vfmcFGeoE6IZTUdIE1oS0UENnoQSVcEJ0gl4QagUIpVMybPN1zbPVQR1QklxnccPke8VwBuC3wHn+3/f1XZN7iuPC3CftGi5c6yeL9VUG+lyly5Zr611NS1P7Xl3OxrBxQL2td4Fg3OQXajReZo57A/sb295/eo18zzzwccfMY0TtSqh69okPb+3BuTWXW8/4Hv9+HZfnffpeZ/oW315D3nfIfdB893HL8bHr49F+apF4Zvaf/bdd5lYzdJxrmnasjyD9troOSeO4AIhBLxzRqugpLzw176BsQOx8/nQmXXmQjtX03S1Qs2mjclpoViA2zRHVooOVXy1qVuD4kvzH/kTleea8lJqs8Zqu5Qa2GlVyJlaMvgA4qkiK3dbtZ5RF6xjDKCW2gA8k+ZIjYmyzW1wnbpysVxUDMDP54O2+apntqQ4oJpy5YsildVOrmpatih01eFUqAJlofiXTqoV5ogbR/rXL/GHPWG8xU8HUq7MY0aKInOGXPFe6HvBlcJw/ZIyjUhJSJkR5+j7Huc8fbelCwNlHLk4vialBBpRLVBmyBGtGSeQvT3ZrAoOXACc0TuqSiVTa0ZLafe79Alozu8el18h3y+Ao81pUB+86pxA+m3N2K3UxX3t1K7VzrDydG+d22bvaore+X12rVqV2syfxZlkE8dzHzwWOFnoIO89XQh0oWO72eDF0XcdwXubdLU2hUfPzuvWe1+u/3D5VWm/XyUPROZ34Or76s8Plndi+df3h4jDhdDA1igOzsbaIk5c47fN7K2lgjcnpjjju51z+NAbyPtgz9E5DEkB6smMLsmAvI3hKkopDWi1ouIQqThvQFXVQFtEcN5bo7wtLt55vHM2ThqnTa1Qik0c12OOwNi4bEU1GdfvbBzmqtC091oa/dgsZY2JOkdKLpSUKSm3Y97qzGbF0BaG9jBksZBhgfVV03eK1Nbvze8q6hp705aAojgBXyvdlNpZbe5JylAqrhS2OeLTjJuP+PEWlwr9MZOzIlPCxUzwQteBq5XL8UiXI4VCFrNHfI4IQkiF4Ca6aSbHSM6ZUiK1GoXl1KESmIYtWQb2OUGOxoW3oAUz+ARyhloQPa1QK5Gg+q2m7PdLoSiUWtefb0rrP9ey7wJqvXPM+eu7qBcnfv1s5Srbub2TFSiVt8+7XG+a7KGUnMmlEHyg22zWKBJ7Hm0xEXNmgTB0HZt+oG4LT588IcXExXZHFzqcOEpe2bN1gp7fk6pSvmGxe9cittz7V8pDFNdvc77vcOivTN7jmuIcodvQ9cXoDc0GMosaajwFznlC6Bp1Usi5ElyHcwEJHf32wnwafaD3Huc9PgQEqMXMZNUMmmxM5WRj0ZvDE3UkFOeEnKVFxAjO36UJrb02jcWbZbkAuNZK0YpZ6gWNFRcG/LBDpVKxa2pJaB3NsdocblIruVajUbItLtqiOuqcSPuRUgppnAzE89u8rThBwqIigwGwrMDtF+u2oViV1s/O/jcgD8hihbi2aCXT0LsxsbkdEVWq70AcMkUkZ0KKXMxH/LxH9q/g5iVTKqTbRMqVvJ8pcyKI0nkFUQanVAHpHDJ4qJUyj2hVJCqSlFQKl1Mi10qMEzknQnD0g6P6wHR5Seo8L6cDcrwllsI+Gl2iC2NSFLcEKTS/gmnhglvNnYfJ9wzgiwa+aLN3G36fof0mekWX1autZnrn26dQsOVtC+kzx8hyjr7vGtje1+JPESELD51zJs6RlCJ939N3XYsYahTN2U00evJk4jqHd57qazN5F/OYprHcu/bSX/quReltebiTUu68PPj4b7rWA8bgiZ2//7WHLD7vaMc3X/IBR5kGLc5beNg7r3NGVbiTZrlYYd43cA/BtHTvTvTHQsfVgjkMy+q/MMdg46ihWXfS6BlzaLL4Q9oYW3tKFlrwjE5sdIw7syBsHLrmkO2bYxRrDw1IVHGqOKQBjFB1oVaM4lh+vu45O8AvoL3CtP0vap9x512lrEpTbe3V1SnraXMhJzRFmGbk9oAoSNfb4pcSUgsOZegD3dAjmx7ygIuFqTpyquQKxXu8VDox30MJoK4BeO9NQZulWRcFqHgHRZVcKqIBJ2oA3nkIHtlu6fvA7JQjlS4nYq2QM7mW1bcFJ6czujyZha96uHyvAF5rYZ5HprCENJ3A9vT73l9nqL4wI6vTs1TOAUnO/lgmHdCcI3AcR6ZpYponrq/fgAg/+tEPefL0SYv6aBaN/VqBd55nXr16ZaGAr95wfX3D06dP+J3f+W2GoWezGei6bm27TSKjbpxziLPQMryt8s4HQuhOKN9u1BaLZWLbolFqNd7yN0LuL8Gnt040VnPQNT5Y10Fsz3J1Qp9FFd0f4/J1KPJtWyxiMc1uOGt6gZrPPneErmfYbDFMzSiVrhvY7S7p+w1XTz4w6kQUJwsVkY0ySRM1R9CM1nxyTKtSfV4X9FwCrjlQF17be9cchHdDVgVWJeDUUy1ySSpazHITFO9AXM/m45/gfMe0v+Z48xItmZxGakp0ztFJB75D3YBqJcWJUiPeVQie7AQZBnLobPzeEWVTCz4lVISyRJAAVQy8OzXNV7HXqVYmrWtotSJ0KmzEqKSLXKg5M33+BenNG3h+i/vzFzafP3wK2w1cBHjSETrlkz/4XZ5IpYu/Q58OxKLsU6UWRY8JnQtOC65kECVLRUVJmolq/oE4jtRcKMdInRJzLtyMiVwKh3FkjhHnFOfAhcAnH32A32z4cc38XkkcxiN//umnHI4H3ry+5vb6htIWybv2x+J9+E0CcFVyzqQc3/FpS2I4A+9lJX9XMoqq3Fm9zukHc5bY9xbXkVYlzYlpnNjv9zx//gIErq6u2O52lpTRqMVa652FouTMeDxyOBx5+fIVL1++IufMJ598DEDXBwKe86m0toVz7ehck3Nru9fomKbCL4tIrYWSyx3K6NcrjTL4BqX1BLJfdYRw7sI6Oena81gjLpYTnVkZSx/cOd25iXLP5HqAGv71FozgJOAkoC5B9bSwpPXay0LsQ4eThWcWnA/0/cAwDGy3Fj8MFtddckJLbnx0bgBuNE1VC0lsagi12vkVs/q8d0gVvHOgi5/Hn0DcnZzx67hZNO61s6tFuTRL0nnP5vIpXb9DEPI0U9JEiRNSK15a/Hr7vmKabxGLzMne4YASAuJ0pQjPJdTKUIoBOGcAjgF434BbG3VStRLXBCVzcHoxoBc1LbamSLy5Ib94gXx2Tfpnn+EQGCNc7eDjLXV3iW47Lj/5gGdDx1ZnBo1kVZ5UCzZwY4ZYkFogRUDJFCqVKc+MaaLkTDwejSY9TORjZEoFHWdSLnA84mNs46PgQ+Dpxx8x7C7IoiSBm9tbbvYHgvPM+5GjDcDFp3xSvlnA/DeIQjGF0wLYTzTGWdZbg/DzyapV14F976g28dtxenICslynmZ1OHIjivaMLgVIKL1+8Mi479NzeHnjy5JKPP/kQ7xzCKXNOxGKxU8rMc+TmZs+L568Ax+vX18SY6bqOLvTM88w4WsakRSUI0zgxjkfmOa5AXJsGZi3HQGLh4mul1GZm653H/rXyXpT2W/LN4P0N327SQqVqppSEaiXnYpRZtRAxmsWDgA+uhe255qATrHqxYFHO57TPGRHzLcH73USN4NR+SuUsntueR9f1dL1p35tt08CLQ6j0w45+s6MftgybHSF0aI1Ny26WVC2U0qI/tKCaG2AZv6ttRlsIbVkjYHxzlp7FR7E8ZRvi2vq4smR6Lpbbcqiq2kIy7fFdQuszxMGwvcB98mNynDm+8eQ0QgN7Gi2kKE4uKF2Pi5PRC6WCNzqi67o7/S8Ku2ni6c2bBuDW9qTVojNqgZztK0NnvL93qPNkVY6zcfepzowqliSzf0ONM8ef/wXjly9Ir0bqdAs4yqFDdWbKbxj3wss+EL685MkQ2HawDcZvLxEiIYOvGFW0UBrBxmHWQkZwXc/wwYYeoXtqgN/FTNwfCbngc+Ki2AItWnDec/n0Gf0woN5RvedyHCl9z/6wp+v/lJQt8er29mhOZVoYKBa+rPJw8IbvOwpFMKByJ5C+C8xn2ncbiLWFRYmcxZHKaRLLOnAbOLYJaM5F17SPYBluwVOHgZwLv/zl54zjyO3+yMXlJb/7Oz/l6dMn0HcG4qtHXaiqzHNiHGdevnjDLz79ghQrH334MVdPZrbbHZvNjtvbI1988dzAubXfuH7TonOtdr5qmWBwAhXnDfCrLj6Clj79QAB/P9z91cD+W21oyQo1TcRpTymZ4/5AzhktSs3gvKPrLYSy3w50Q2+RG/3G6JYlAUbOtMr7zf22TX9HN4paSJpvAF6rrr4LcY5u2LDZbtleXLK7vDLbIJmWvtlesd09Ydhs2V0+JYRAjhOlRLQWJi3UYtp3yZaJhy4Lt4GHpSqcksBWDlsWOu+cljolAglYFIdglMniyF78RC0bsaSZWCK+23D57AeIKLsnT+k++glpHgneEcc9JR4o87E5STtAqP1AVSWOB6RkSqkElKICQ3/vUSiXxz0/fPEcnFCcaZhTTsRayDExHydwDn/1BDf0hM2GsNsyF2XaJ1IuzCmTUqYeb0hf/pw6Hrn9818wPX9FNylpr6jzzNeFPA28fr7nebql947Pt1u2wbN7smF3MSCdIFuH88LGd/TO450n+A7nhL4PuBbT771j0w/86KOPGPqeHkfAcZxm4usbUimWuOfMuvSUVlZhg/ce3/eEoSfnzA9+73eZxhFEuN3vORwOTClDSmsceK3VkrTkIbP7JN9zGOFJdOXvm1ax6hrSBuHC/beknzMz8VynUs5CE9eQO6NPFk++1opwCu8xftsDjmmcKUW5udlze3tg2PRshg0heIv+EGGaIsfjxPE4Ms8zKdogW35KMe0kJaNo7l7HOM1F+z6FBp7TIhbyL86tYK8sGpyetKpfXe+3Dn1f8G7PQc8XYFhi6nOaqSWR0kicjpSSKXG0DL6s1FypzkHtGi1gwFZ9sDhZcYjrELEkGNfKEsgSL/9Nzf6qY77qe8tgO/MpLaPI+0AIfUuDD43c6RGthL4ndAMh9KYkeG/oW06Oy1rz+kzXwOzWlpP1WFfab6GPzpq0NtEcime3I2+vZ+uTlZOmbnVAii0kaW7t9YSuoxu2bXwVNEeWuHNaLoJrVpFzRg/52rJM3wE8IUY2xwPqLLHS3AkZX4uVAjhOlkQUOkt3R6jOUYuSp0jOFtNNTNTjkXQ4UMYjOUZLR1dTAKtTxpJJGY7zyDjuySJcTzOjc0xpy3HeIJ3gdg5xsAkdXQNwvwD40BG8wweHD57dZsPlxSVVhCKBXuCYMm+OR1LOrWSBw1HxVJyIaebO05fC0EZO6HsGEcIwWJ5HCCdi4V6ww/0EvW+S7z2MsBalZKW0QX3H2fguCmWZULWxRVrJa/0Ge62lrKCZUjLgVbVML7EMOhEzxauaVvfhhx/R9xs+/+JLrm9uOY4jVSsXFxf81k9/yuXFBSklYpx5/eYN/++f/XOu31zz/MtXjOPMeJyYDhOdD0yHiWkzcfvmludfPCelTC4WLnZxseXiYte4TeMxl3aeT7/loVphnQwKSybmO2NuHyjvjk751Wnd69m0UHMi58Sr57/kcLih5omajqBWTkDUalKkObJkPIg4uu2O0A/m7PWWvdhtLgmhp9/s2OyuEOcJfW9ZqzxgPXsHiL/rrlUrmiI1zVQtFvVjo4pOHP2w4/LyKRcXW3YXl0ahVKNSrp59xOUHH9GFjm63M6svHshlJqWReTxQUjRnYJpxYvHMFk1iPpDcFnPnHAGzMr0uKo2NexHz6Lg2Vzzu5EsRq5+yAO/qFPaVKg6q0VeaI+PNC/J8hA9/wvbiGWEYuProx9ScObz5gqNAzZE671Et+M7KPdTQ0Q+XRgWlI64kCkq+25FcvHnJRzGBg9qoi4RRKFPM3O4jxXnm/YG82ZA3G/a7HVNRbvaJKVfScaKME3m8ZX7xSzTOdDER+p7o4baDrHBdj0xH5XD9kptXXyKl8iZmXAV/ucFfDEgANsYwBSNG7Z/mg+r6gFsAvPM8vbriD//6X+PZ1RO2oWcTer589Yp/9Kd/ynGa8J0BeBDoWp/7YPH+T58+48MPP+Bit+O3f/pb9BdbwsWO7vICnxO1KZrqaLSBrLTvt5HvXQNfwgBrVUpZiroYcBsQfPUdLbxwbaFYVTNLck3JmVorc5zNw9+qjzlxlAbg4iwDDWCz2Vqm1py4vr5ht9vx/MUrpjnx7IOPCKG3iJVp5PbmwPX1LW+ubxjHyRIask2MnAo5ZXLKxDkxHkdiMiCrWs002wzmBPN+BdRVA79zu/aQSy2ATfZTlMbD5F2A/dAQw3cd9+5L333TXGbGedcSGY+3HG5eoyWi+WgA461AU5lnyjQ3p5UHccaVpw2IN83bBWoVSl9wEqh9sdokqquW+fU38i3uXZU1YPfMpVSb48n7QNdb2YMQOss2xJ5jN2zphi0hBJzvGtBq07qN/88lGY1SMuLEJjCLo1BYsw+q1Qpxapq/LuNEzZo07ftuRI7NHdNKXYvcWPy94hrgV5BmDeR5BFVKno2KEU+/uUBrZR5vcKEHbT6YWqCrLTrL4XxnFkm2GO37KoWoEuaJbal3ALwTC8hzcyEdItl5cujMOV8KqVZigfmQmXMl7o/EcaSMe8b9AXLkQi2uvghMDlKtHKbMmDPjPDEdDpAyx/2MlIqMGzj0hnYb7F4X60Va3zo5A3CP7xz7p0d++IMfUirsuoFtN/DizTU///xzDuN4B8B7t8Tpe8Q5Po6RjPKsFH6M0oeA6zpcZ5aZ6jK6Tpm372NUf+9x4AvdkNsDvPN5o3vXhBs5TcA7iTxqALdq4C1ao5TCOM3GtbbMMhFHH0pzSGZwnqrK06dX9EPPj370A3znGYYNr1/fcDzOKJ7ddsscI3GeOR6PHI4TpcBms6PzHU+eXNJ3nuCFGCcO+xum0egCrcWoE2mV7O5ECGAlKucZrbr2gfdLtIEtSIi2mOP7dMv3K/fhcPm/pMh4+8a0zXhE6owjtzj5as9HKzHOzHFuuGl+iipKpwUXekLfLLIWclc1G72EUA3yf6Xi+4GLD3+EbCNTjcSayGlknt6YydzoBh86QmdlZL03wOy3W4bttvlhrLpmmo/Mxz3zeCDNEyVHszZhsaFN+2odVxuQVxFULAK6isPLkpZo31kWlVwrknPTvAWco/cdXRds3hSr00Oyok3eC76at7iWRIqQ54k4j/jQ03e7Fci77RXFOcp0Sy3G12qakYoV9RIlzUCpEN6Gn1AjfcktCtNoFi+OIpYK73pPxDG2hJhjnHgzHphUOM4wFyWPM3mMlJjJdOAcdbOFrifHyHiYSGQiFp9dnSJdK0bnK6oFccVoOeHkMNeFhW0wqkqqZel9QOjSzH6a6KYRUbN0xnni5nDL7fFwolAUgrayFN4btnhvdVRCYIwR3wWSFopAWTCqFDwevC2uSwLWA/Ur6+P3Hei/ClkAPOcT4K58sFpct7bXUkozE43/PJkbS3QGLGXLFqdgKYXjYW51C2wRcOJIIVjMrPPNYy88++Apl7kwzTOb3Zb9/sir19eA8PrNLd4HC3lMaa3ljao5LJ8Enjy5YOgD3kGcRm4pjOPBsu5UV9NqCRdcRMRqDE/TZBM+WlW3Jb1+Md9tbtpqfaol/PXy8GSebyuLn2K9kL1wAvCcZvbXL8nzRJn2SJkJTgnBHINxniklM8WZeY6NGmqLslQgE3RD11vlRdVM1UTR3BZsR2ghaPfb9V0kdBuuPv4JXVJCnJhzZB6vSTXjvOCCOVd9CHR9h/eupcp7ht2OYbezeOppT0mROB2YDzfEcU+cj6Z9t3KuhiByh6xWlnokjeKjAflaU1pXvlyBUqwMhXdC8GLEiuvous761EtLEirQQCi0lPiaE5ozKR6J85Fewe+e4sNAv72gn6/ITkjHN9Y51ZyxiPHly5pySk6515c1smnVDUuLsqk+UJ1nkJ5tv2VS4fk8kkvl6ITXzjGrsK+BpEKdMjpZ5E6WDgkd9fIperEj7Q8c50xSJQJZ65pJqQrqLcpHXLawUISKN6WQxbppNJNVxwVzJVIUwjxzM434sSeoo8NznCbe7J4+TH0AACAASURBVG+4OewRb/VypCpuqaTaAh18COy2O0LXcYwzYehItVDEkpVKsZh2t4SBIsa/O/+t5u33nMijxJgIDuOqzwGcRQNf6BXTupbIEzg5fRbnjJkiS81kO2ahKmhBDJYMEVq8rGPJSpNGZ+x2Vps8ptwcRZUYM87XNaTMOcd2u0EQNsHTB0ffm9YTurCGANriYpRHozpXR+rCWZ4elt3jNE1G9wylxdieKuGJnIUjPFDePRi+Hui+dgCde1ze8R2bzNXqmRfTmlGLty3VamcY7bXU8rAyp2YdLY7ddU2wseAq4pxlNgZLNXdtoiicFeo/d9u9vywOOfORCCoB5wfEWzjjEk7qnG90QitmpRbvXcuJ507zTIyRGNOp5n3LPLZsXJpD1l7VGWBLu/7qBxJjwZdNHlqdrdWXX6s2S1Pou/6uNSun2PWqFbdQHi18Tqs9q1qzaadOcKGj6weomTDsqM6jSdA8I+Jx4tHm/PcuGL9+T2rnyWEpCd0iY3wPLqASKHSUAikmUi3ErMy1MiMk7Ug4JDelDPN1CYJ6gc4jvcf1AScVF5yFcwZnneMEFav4shQN07YYLnRFbYvh2pmu9b+zMVDFFoVcCrmV+6i1Gp2m1WqVq2ux9e2mnVlBRZVUMlOKHI5HxAljjO08uj7TkxPTrIZvq3N9I4CLyN8D/kPgS1X9w/beh8B/D/we8M+Bv62qr7/dpe3BvXlzw9R55lYg5jw1fnForiVngSUKZblRy4rzdzTUVVPHMQxb+l5PwIkN5oWnRcF5IYRABX7Axzx99gRE+PyLl5ScSXkGhc0wsNlsGPqOD55e0oVAzREticvLHZdXF3R9t3Jpzi3aT8W5vlWta0kg3hHCUhAJQJmmkS8+/4JSCpe7CwtfGnq2uwHnT0kc3z+F0sB6AcqV2lKLMMmJnGY0T2geIc9oiYzjgf3hFucclxeXhC7gAoTB6nbUbNZH6DzeW3LPlCJehYthYHt5ZZrhdoOIB+fOwPtu295XFKVoIWsl5sIclaodYfiQLgguGDfvQ2ehjs6eozih5plx/5KSItPta3KcuX7zkv31G+I0cjiMFhJaLKRPWoalODFLwzl85/DBxq517GmRtwqIbXxbYIhBm1qW5WGeTWdXtQxQZ5qyk6bZeQXNpqU2y7WoUstMjkcbswI+BIbtDrSSNzucD9QUmW++JB3NKhXn8MXTb3YgHvUDSc5T+2G+2rL/wOrbL3kQrrtAwkBOleNcGGPmZjxyXSJvDiOvbo9EhKMfKM7TdwNdP5Bq4pgncHDVKXLZE1xhU3f4GEhEdFRyCmjn0ArZVaorRpXiWE2G5uh1gHiH66z0LmEBVVBRSnBMJXHMkV2JxNKTqlF4ldoWV10LmIk4/NDhvKc62E8j3Hr++Wefstlu+PL1Sw7TxJSTOU59sH6xIgHN+bxkJj9MHqKB/1fAfwn8N2fv/RHwv6jq3xWRP2r//+cPvmqTWpUUE1Jth5EFwGujRBZzYqFLzmtem1i8pmkNrCuYabbGji6UxeKht281k79da9HSHQbS3nv6vl/7Uau2DVbEJm7Xs91u6btAnqHEumrftgFA2zqpWQbLqD7PDL37Y8fVWpjmiZwywQWLPHHCUDvjVSstZv7bBhvdl4cOkHdd5R54n1MpLCGRxYCqWtU1rQVqoeTMPE1476m7C04ApW1+OSMQWp8syV2KIi3MzYeuxci3ga5vNe07SvOlNN/DWo/Ed1YJcBlDS9RHiwW2xbpQYiGnmTRP5DgT40yKkZRMA9dGeWhVxLmWhu0QX/HI6rRcrnE/w9ItSginvOTF0jN/S6MPc7biIaEVupK7429NXsGctrXmUz2UZqWGVp++G3YUH0jH4awULmbRuoD3FvJ6txeFHBzzYHSll1YDfdjgw4Y8Z+YyM+XCrJW5ZOYYmceRJI4UoC51ZehQKlmzJX45zBoKgus8Tj0u2P94Q2azZAyILfGvrXiC1RFverl4rAyvE7T5nWiWPM4SkMqidTd6ZQ3lbDbSYl6LMyemC95S8ktmTpH9eCTVwjjPRqPUVv73zNSUZtl+2yH8jQCuqv+biPzevbf/FvDvtr//a+B/5T0A3AadUsSci94bb0etrU/OsjJlcVjeqyustJBBYam/7zwsJspKYUhdB97ykJbO1/VwK4bv1RGCp+sCznm2uytC17PbbrjYbgle2A4OJ0qNSm57452SfZp5r61gVouAsew61m22aCBVarEBmrPVXSiF2Dh27007XaIQ/mWQE08LK2oKFrFQEjlOpHlkno/keeSwvyVOlrjTBysW5qWFcVXT2AXofLOOVCk5I11H328Jw4bN9oLN9goJPaeszF+9lBKZj88Zp0iZLAZ5sbK9eiSD5oLobuW+l3DvPB+Z5tEyGm9fk1Pk9vqG2+tbco5MU2w5CA1HnFrIGuCX8EAVEN8WrKFZaubcOoUcNpqxnjIzVZVc7P1xnBCteB/Y5mJp+Wv+QF2jL5ZelFotXLCkNdpGvLdQThFK2uCcJ28uG92SKHFGUXzoqSo4ujuLqaI8H2/R19d4CYQw4FxgKJ5u8Lx6fc1ffPoZx2nm0y9fcXMceb6fuL6dKM5TthdoCDgpOF/IKaFlsgleZigzmkfyvLfyrtOBPI1oiabMeSFsemoQZNvDpkO6gL/Y2CYLrf9c8PgumEbcdYh3Btqo1boJy9Z0ggSHC46u84TOt0VCmgJoG4C4ziHekakc8kSJ8Gp/Qxc7ro8HDtNMSnnpecOG5uNaqLlvA+Pvy4H/UFU/a39/DvzwfU5i4Ns8x2ITweiTu066ZS/IVRNftfCmGRcb/LVRK77RJOcRG23s29+LGeocDkvj54yXtjoXRs34zvHBRx+x3V6w2wzsthtEM+QjWhJJTu2VxjXaJGv+7Gp7FdaquDv7JJ6KOC01xUvJFqGQizlJqzL0HVrUHHt62orr23c2D8K8hzhQTmV37h4rgJZMSTM5TsTZgPx43DMfD5YsslToQwxIGnj4lvHnxHjakk157IaBbtgytDR1xKOtjt3pxt7jRr/q3mpkHl8wj0fKHCElq+/tA146yD1aBKHYe94RmgU+xcjx5g05zRxubwzAb2+53R8oJdszRRsYt1o7ralLlT9drBJnGrD3rm2YfNoUBLUyzHqm6Kja5g2lKOM0U3O0AlN6KmUsjUf393h0S9rJxoMvGwa3crkChG5AxNENu2ZljOQYjcr0VvXHVQ/51O+qysvpyHXZ411H313gfc+F2zG4DZ/e3PB//vwvOI4jb97cMk2R/TFze0ioD0gpSN/Zlp4DLWvXNlmgzOZQzTN5PhiAz0fKPKElIqJGiw4dNTjYdrAJ+L4jXG0Q7yDZhgoueANu5wjDxsITVSmqDMNg9IoXxDvEC64l+YRgYYwVtfe65tDsjDrNFMYUyR66wy1dDNyOI2OMzQ8Ea+w3S6E8OdE8D5Tv7MRUVRX56gR+EfkZ8DOAp0+f3v9sBczFhFi2UVvA7R3XW18XjXa5Y2lORtN47e27ZTSXCzdNvTkwfV126YFcrEZHyVZqtqoyHo+UogQnXGw37Rp17fzzti0ZlwirU1ZrXX8aF2T3prVtNVVbYH9pnHk7Z6ONSi3mJS/F9v37ig0d3t1jZx+e9fv7yfJc7O/levZui2KulsBjKePmtAs+oMNAWGgQ7+na6zAMjd6C0PYUnaOV6pW2sHF2X2/fX2uFnP9/Xx4O6iLgOyEUAdxK13gntnFN81eIk6Z9e5yz+Aa31soRyybM1SzM0irgLU5DVWqjS4x2kjWqxNqw+Gpa5qNYmo+0qAnW3Im2a46wjlXFwD0mWwxyWWhJO7d30rb3giXmXVpKv2nXRn8J2opktcJabUch3/W280wrubyOJb3bzyLCMPRsLy5xEvB+BxJIVUnHidvjxPU4chwn9tNMnBNzKq2olyA5mVKWZmoKtgFxnE05ihM6jdRppIxHSozUeaTGGW0RZ945houdLUBPt/jLDdJ5/MWACsz7A2ma1w0nnIOus3tMpVJzZqEEz8N+rcidW300S4b3OcAstE3FnPVFbWPlej5blnF99rrQxN9GP3tfAP9CRH6sqp+JyI+BL7/qQFX9E+BPAH7yk5/caZrtmdfR937Vrtfqgarv/GnntNKqzdO+0G+1mYl2jtj4cH+HiqE5b5RTRIilt09mxjQeJs7RknFyYZqfm1n7Wz/m2dMr0GVS5jUyZQmJFGc7hddamSeraLa0d0njt8aqObOgFasq5JjbeVvEi7cwsRgTvhRzYjrhrWqyy+BpmtU31wt/CKDdP4esP0tW4ALcjlacXws1TuTpQBqPzNNIzZHNZsPFbtOedb8u3CLC5eXlShOlOFNyYZpvrAiYH9iu1emWxfId9/aN2Hy+1Hy9OC8MF47aBVtctbOZVS0233dghdA8w2YwPw1GoTkfENcBiZgKc8zrj55rzNhCUFnLl5hLTK03BQPN4E3T82r1rXWh49QS1Uo5K4SlUNTOk3OBEulLwXtbKFMu5FIZgudisyQgNW64ZjTPaJ4paaSkHu97vDdaxCwjoW52Rj84Z5ULU4R5Mp/BfSgRePrsCT/8yYCqJ5eBXITPX8y8vnnNz1+84s+fv2KaJuajbQqhWdAktlvNqJAcvis4Z3NR96NFeLy5IgdHurlhfvklMUbSzZ4cbY9PSqHvOz75+COGzYaLH3/A7gfPLKU/mKX7xc8/5eblK6pa9qv3FoHWD1uO40TORjXVYjkLimn13juGLhC7jtycwIsfBIfVoWnzo2BzItZCra5tcuxQEdpGS6vusezCZQv9Nw7TVd4XwP8n4D8B/m57/R/f8zyrc+WbQOddIL5sdLAotkv2JXKKRvHeEmBMo6FBgPVcFVs9a5G2fRatzoM7oz4qqURUpTlZFzjQVWs6cVZLzQtdw430dAOnNreTrFZCC7F7SwO/c+9WnU2axvZ1YPTNGvb5518FbF//3kpHre+28MFamua9gFYzMb1bNdbz9jlncdS1FIpYAstyz/evqUvn37nH1v57zX17PMnDMFzAByHUszK3Fah13ftyKcC2RJDY3o2n9io2IZfNN841q3UJ0rOyEHr6aSPr7bauB9c1q/iUxbzstWhfs41KCs75dfemXCq52E7rVavt/u70jiZuz8vGoXdLvza/hLPklKXGi/OeWv2ZWftWN+KcxTZXtR+pQsqZcY6Mc2JKmSllUs6UXHDFwvKoQMkg3qy57NHcdgYSocaZEq30bZlnakr2Wds5iKqN+unoh57dxY7LJ1eoU4qr5JwYNj2hC0ZdVRq95+laEMLyQLSctnxc6NjgA10ILUy2RaPI/VmlqwW9hB2uzrY2Fg2/mjYuehor3zBEz+UhYYT/Leaw/FhEPgX+Cwy4/wcR+U+BfwH87W9xzVVsF2/bjXvdN05PvPV50ss5eAOEYBvKllKY50jJmVevXnM4HFi60uK1LbW56wJ93xmohyU06+SVXyoO+mDOpVyWaoaVaZrJWdcSsA6LkHFBye5eVII71XL2LVzQUpqN8xW1ma3adh9RZZonYpwZD6NpGlVXYAver5zpujjo26FGCxe69NXD5aF0irw1WZcyDqIVzcl8AuOB6XBDmo42wNW2G7Pqodmcs7Bq4LSFcAF/02QdV1dX9LudUS0tkqI2R995LsBXtvZdSsEDbtU7YdvbHpbr11WRWnEusN319J0VJVJaaeF5pORokSe55TOAOZ6XMECRNd18ySquxWpGO69Itr1QfcqIN8dpjhGCZ930pzSuWqv5S5oVmouFX9YWeVWKlXYQycQU8d5R1LWNFZSUC9VZ5p9FNRWjKsJIns1X4TYO73vrA+ebJt43q9W09hxn4hwpRZHsodxdbG9ePKceD+B6intCLo4vvjjy5ZvI9Zvrpm06K5PQFjBLca9W3U8rdapENUs2HWYE4fZlR01HxnEk7q8puSDJrNRSFM2W8RinI86D7zxXH1y16JRCLpl0+JBh6VhpVk8YEAnM+wnmREmFI7eUbuRGlaFkaoz86OOPmeIVL16/Zn/YW7RLrWadNDqlqiUDUQsxJ6uDrq2sNFawq5ZTfogWw5pcvnlryXN5SBTK3/mKj/69B1/lq89NzmYe5qY917OVrlt2aL/3HVgiOazEWS2VlDLX1ze8fv1mXcKc91xeXtL3tkvOdrvB+2Wn6XbeBuDmSDUm1zmbXNquF2NqlQYtC1Na6BctfXkNz1oB3BJApIUuqjsr+WknBV02RrasxGkamafZNNdGJVlNjSVxZNHiVlXtK+UhFs3dTuUbwO1t7nvVhLGFSYtpQSVOpOlIjhNWLtXA2/ZyPXE/iya+bE1nzjUbzKHb0m16umFYd31fnsUduvutNt6952/dD5jWOHTe6uScnRmtOPH0fWfb7gXL4q1aSClS0kTKcd01SRc/ThsXa7YwpzFcl42DEQMeDFxdSraxcKuXXb3gRdZFTqvtT2lF2xI5RbuWX85lySKmdSdz+koAPBWsjDGCim/hf2oRKK06YYkTtduerF1xrdZY19bwDVoy4gI+XON8xFW58whUlePtNfP1Z+A20E9kDbx+NfHmOrHfH63eurZkqbrQSUbJLVF/mpJF1+RCmWYAxhuPlkiMkTweDTyLBQnUUtFcqcmRUsQnjw+O3eXOUi+9LX7x2RW+mpXipLOFOJoTOCBozBQRZj2iwXP0nkMz4j54+pTcNnUZx2PL3LR5Ka24g6ItPr+QSqGKo6plggt2v2aB2+RbfBtWLvvh4/V7zcQ0DjzQdx7npZkqbXMDgS50p5KhzUgN1QrBhOAbt5fY7/eM48ibN9e8efOGRbP23lNKoes6ttsNMZo2sjjO3hVNvSiZL1+85Hh9TUyZMqdGX9SW7gpeTCuZzxeYc4cHZ0DTTKWlQtzC3dd62i7Nrm1a/N2MrMXoXjIy3YM10AfLnUPfRvPV8juzExdQM+5+Yrp5TY4Ttzev2d9eG6DFGdVC8MvekacLnvfB4ox2pzPbdRfTXg2sck54HC4sbdQ77X8/C+ReV4gQgrOUaMRWHqxwlLRaNlDJKXI83JqGddxT0sw0HpmmIynZphWI1Tlf9ljVYs9/je8+08pTMcvO5Yr3Be+sIBqq+Go1rNcd7BulduqXfOdBLdZsrWKbZKsiwS8h7FQVSgPO0tJO3RL3nlPLJj1dyx69tNITBjSh2wCC7zb4LrYAgrMcDbUaK3m6Rn2EzUChI0XbBFzV4t5LW+iWOueydDl14YMQFrRb6KDE3DaV8J0luFlikzBNVjgOVeI0gzNLRktuWGkbOAyhQzcbhn7DbnOJqrDfT8RYuA23SKNOjlNkFuFChJ23FPl+u8EFg92aM3jDGnEW8ine44px8U7EVqZinLrp+osT+eyxscLct5LvF8C9Y7Md2A7hBN61kHMxTSx0DdDOJmV7DcHqURwOB56/eM7N9S2/+MUvefXq9RqX6f2JQtlut+x2W3zwbDabdt5GZdRiSRatxkktmeN+z83r17aKug6cx2lh6BzBC1570MIcPJO0NbTWlZBfuMWTc5a1kNUSnWJOqLryaK6li6OsSUd2mopilIqlXz+sFsr7yX3gl7feX7G4JDTPzIcbvvzlXzAd91y/+JzD9ctmE5ojmaEntFg7i9JYEmSW56rNCrrrhF2Sglwt5BSReabH0fWbr7EYvlu/WFH+gF8dFI0HV8tVcB6Qyjjtefn8C9BCHq/REpmOR6bjyJKMJrJUL+xxCXPQYbvriDO+v2Dhf+Z4tOcMpp0P3UjnPdJ5NPi2JdqyAcBCTyXm2TRT8QXEtWQh87VEb87X3hl/rVj51apC1pYFqIKrFamFlCZwnm5oMd/NGhQECV0ryBas1G8c6LY35FLxJSJMZ0qRMu1viK8+g7CDS6FIzzR6YnTUkvHiLflxqV3Qutv0nDaPakG0WNy2GDU1p5Goia4LDJsBJ47OLfWNRlI0Ku6wv2WKI9PxQI0RF9qeAKpcDQMXl8LTJ8/44Q9+glb48stXHA4Tt8+vkVRJ08Tt9Rs0Z0Kc8Hnm8uqSp0+uQDqcKjnOVu97GHDB0fcDLnjmnJFo9dSlVFArbevEUTiP9V6UDtbXXykH/usU05JbgScBdQrFdG2h7QUoC9CenFYCLVXdjp1jZI4zKUVSTnjn2+T3eG+8tV88+sWvmu6i/VnYVaRqJc+zZQyOIyUlVLF6C1him2NJcl4047M6CIsGfv/n3iNRPX3XpFEvi2YGS1BJ66gzvFpIs18bgJ+1c/3rHoirRVOUnCgt1nuejszjgZwmSkkIFXdHt1hG5gLOX6UpLwlQimvOuNIyC31zGhmYLmd614LzHfpmXWiFNZ1dT3E3CzNSS2GeJ6iFGmPbsKJlEzcYWyKOlqxRWUzlcw5KF0XT+siKu1W8K5SmlWcnbdwpS3mzpf/WMFU1CxHRtYaQ9aM9q1oV1xSBZVjWBuRVl4iY881Q2q5Wi8NcFsvBSj+YM7NREC4gku50owJaCiUmICE5t80aTtcxZ+wyLk4Jeku00eK0dgLawvdWHb+NQd98RMG3jM+VdjRtOYTA0H7MGmp+l24D1bMbtuz6TVswB1Io5ncSZ1swnrVxCevsWhGr7TBYYl/f02+3tqvU0Fts+RL9ttStF4/2hdoVagirD+5c69ZTzz14uH6/GrgIoXN0g6fWVjckCKENGtcGvjnu6tl7tkLnXJimI9fXr7m+vmZOM9COrRaHOUczuSw2syJOOBwPIJBytrTjWplzMk9zjNScCTnTzQnvA1fbjmGz4ZlzDDkhRcllsqpyi5mklj2IKiknm4jNlF68zMuPtAlsjjJt1kCPqsO5sfHsLXnDybrbuGrj5r8iDvzXJdb8xueqIjVBzRxefc71i1+a5v3lz82ZlyZ6l3EOQnNUBseSLkWrdt02jF6sjDPaBCGVQtZILHBMiu8GcD1aK0EEf3GBndF9i6H+MHEiFg9MYPFcnH47qnoUx/E4Mh1uDUh0RrRYQklMKKcdoSx81K3hqXK2p6euwGkF06pWSrLtxnIf6L3QBU+KzkDKCRtvpQdUTxU701J4LdkcOYXiClEwi80lqjY6p7NEqLlAxmpZUz3Uxrt7W4hyztZ2Wttb+VQp1R6gOHw/4LsZ5+8COAolK2kyDdpvM6qenJUUhXnOHI9zc3DXVtu81fZ3Shbb6T0MgX7YoMDQFvxUbVOIzbDl6uqJlfd1Zp1WerJ2DH3HD3/wMRcXW/7w9/4qf/Cj37G2+2AL2jSjMdN3A9vukpwLY5iRzvFkuOJqd0nvAq7YNngffvgRP/jkB1xe7PjkyRMLI/1X/zX+4K/8FZCl9G8rFYuSayaXjPee7fYS5xzH57dMrw686L7g5uefkWK0oxvF+j7Kx/e+J6Zvq6KsIWRW8lIX4lWbElBt0C/Ox9LKcuacmKYj0zQaJWH6jDkRq5KL7brjUmrB9yfNJaZoIF4Kc7Y0YmKCUthUpW8OjQv3/zH3Lj+WZFm612/thz3OOe4eERmZWVXddbuvLmKCGDBmyB/A7M4QSEh3jMSAK/6CO0Ji2hIDkBhwJZBgipAYMGEAQkKiJ92XflRXVmVmPPx1jpntF4O1t5l5PCojqgoCC53wCHc/dsy27b32Wt/61rcso/OMxuBTUp2DGFbaXyvRzynp16heY65QyceOLUGriQ9rw0p/VALDRodsnmnzT37yaJ/7KVj4+7D37kebp7CW7uSEpMByvuf+zffMl0emhzfEMGFFK4+NUeMj0lrZVfGw1ZM1bKndPe4NreVXkYUcCtYHwnyh63pt81WFhNKHLvgPPUTlFFqjj/pNTPOBkwolhRB4uH+AknFGG9u2wiWlnFVhttQqgeXJs9iHyqUqCaaKkyruW1iqA5CiRmedM9jOVSGmzUvNuSp21krfrVejIUqmGFFMNkYQwZVSk5m6tkwWfBFSFlLOarRahyypjCGoEJIjS8aKOlbKfa+yzO8cORVSyBibMTlTRHnssTKSwhKU7dUckortk8vGSjEW62vtQBu+ukadU+NuK71RxNCN0C2ZwzDy9cuXXF+d+NmLr/j59XMVnLIOELJfKDEqIi2OYBK97Rlspncdg+/VfgyasD0cDpyOJ06HkWM/4r3jeHWkWENMhRC1aGeKS9VOUdljax2n8YQVy13qeSg94eG8ajhRSnVqqm36zOn6ZQ14aQU5UnWNyxNDIuwNbkvybQJXuTZoaAZGO2U33fCtwlNqxuBd3mwISrNKO1yaGuImIMBK5i8xk+dAuMxkyUxpUtW6FGsWOu+w7Y0SKU8W7Y4L/gHD2jxzNd4bJbExZUpWveMNTvr4uP7EL/DOQH/02PTlClISOQQeb38gzBfub18znx9JcaZzFm96pFYNWmvoagEI0ky/mt5clG2x8eWfDMD6vSJaveaKzo+m0/FTPPg/5FBDVTvarJfV/i+Q1LMNIXE+z1AyvddNq9QK4JzRfo45Vw9907hvGxQUEkqfizlpL9Wo7JJUC5sOvSd7uyr5BSukJWnVoDZxZwm5aqAUQtLPt65gndVEoUDKAiGRiRQxOF+whRrBWDAObAfWawck6yhFSCnW9db0fVRD39ToR2yuG90HFPQE1U4fThTnmUMkBhT/jqLOTlGIqFQnJaUKOxnFirXATmUMjCgTp1CwkkjUzS5p4Uy7BOc94/HI6XTim2+/5fnNNd+8eMlXp2ugsn0K5F4orpXWWuYlaMyVIiZFbAr0lKp8CS8OIzd9x6nvuOk9ve/wpwNu6ClFxyOVzMN8YUmRJUzMYaLrOl4+f4n3HW/7E3fHe5b5gus94oQS2cGsnz+nv6weeCm1DVnLnKvx3OQzWaEDUA+gJfLUO8hraApCTIkQwqou2I52vsZ8aAZ2CYFpmjX0afrU9fejgakutBgTeUnEy8zycCZJ5lzORGVyr5tH++yPGfB2NK+8eeD7wo313ncG3NSu7KpiVsi5Mo6IuwAAIABJREFUIfG/7/FpE6UG+vWiU+X+nnn1/T9wvr9lunvFdH+LkcLYdUrHqp6hs5ahiiG1j6uKMcSUCfFCzpEQNIGsnmM1VFG7NBWxZPG4jFKxyt5476RLnxx/IKhSPc4sK2KtVYvWULKwoOyNeQrc3T0iFPLoVAbAbIZomRWvt6K5gLxe/1bAk0ohVKdjmQMxJhZRbz2GyOAdXRVaEtEY4CyCNXDoLd4ZlR3Q/CfTomvIZ4sromXzWQvZskRsUqPdDbqTWLHa6MB0YAcVCnM9xvW1liYqBdZoDkiMqWqEmq9SAS67i6SeDqTpRtzpGUvOnOfAkgLzbImLocSytsRrcz5TKFXz2zQlQueUzCCCr4SGYBKRpMqNUROt2AyiSUTbH3n27Bm//Ef/iK+/es4vf/4LfvH8pVb5npW9kr2Hksmi2ZqLNfhSMCHgQqALCwKcTke8d3x7OvJy7DmNPV8PA33f8/zl1xxvrjG2w7mBmDOvz3dcwsLD5Z77x7eM44E/+0d/xjCMvH5+y9vXasC7sUPuZS3HLTt52s85vnhPzO3YUeeqMdtP9gYftC49K8SCFiQYY3lXxWvv8e471b/32p0f05oqFSKFpRSWrHKXUwxMy0I2mSCRWDH1JqX5brUo8EED/uSuZcP4947znmpWGmq6T5L+jqO0AZPPl6f88NmKtgJbLsyXM/P0yDw9klY4Y1u+7ROl4vYilR5KZVPVCEW7G+0MuIiW4yPECgcUhGI2ETDW8ZB3Ej9QhUKeXvlPRiIfOZrjgG4T0paW1KYNSTVz4qJ6Hck3hLzp0KOendlf4TuyyHU81GPfzcUajsRkNY8iqTq/Oicz6tg4o+cJUT3wlHPlf2shmmlevqjRi0kbY6T1s+qNVullMcp9bwyTVc4CdnNa3pnPO8mLdwZfQNkZxxOEiJznesOsFboa1dVf3mH3TRTOtEx1xZvWfFIplblSmV8ilJQRSWDcavx9p9LPtlZWphi5PD4o4ycnIOv9W09aFkhBGS9oCzbdXDJGMqVEUlqIwTAvFyAzTWesdzifoLekkvVeQ8S0rz4iMWFqFWzfOTpnKyVY3pnEn398cRZKoyU1Q5Fzk8rUxb6Vpqs3G2MVu6o3bcQyjkeWJfLW3QJPDWljBoQQ1uKR9rU1W9iPX6kTaM6Zc8m4EpH5zJgCcv8WeTuq7vCgxn7oenolhW7Kh/JU2bCUspZft+vbT9Zm0GKINNRZtabbQtr48blkcnZQdhhtYV1o7L7+oca7AFTu8ePtK958/yvm6cyb3/w98/mBzhR6S10cyjduspolqycqaCFUSoklJpYYCTHx8Hgm1H/HmNSA18QYTZLXqJRnMR6Mw/lBm+mKRiC5YfNPDPVHVsMnRqiNaVDqc4OyPccEYZ6YLpnz/R13t28rLe2oHZk6h/ceUGU7KJia8M05kRpk1CK7ivXG1hA7JhWlqnDL+bLgncO4grEVWop615cJnKXOnUAuyiVvJfUZxYyjUUixK+BTQowlRGWrGOOwrsN3A91wwHc9Xa9fjemqsW2R3xYZrj6EmFpdq8yt8nQguf76Z/S/uOH+4YHLr35NuszI+QJxQRK6ydRm1kXqxuQs1gqds1pzUQp5Dmv/T72mpDouMZOnueJXhRItZrC4fsR5RzcM9ONASonHhwfevnrFX/3l/8V0PmPmBRMD/XjgdHOjkOl5woaAZVajTCKWQEnC7WMAuadzjofH75Wa/P0Vvh85HK65ef41glDOEyVGpumRMN0zjwcerCeOB0Q8R2c5OKuQozPkZasF+H2OL+yBS8WgZOVIm6rk1ahSOW8GHDYsXCk6ulN77z5atdkSMe3f+76U73LL1xDACFlgphBL4ZwiCbgPM/fzBZtVE7hJ0W7qiSiEbjbjrKfd/f8DRmTjhtfiheo5P7nOmuBZdZ3fO8knQN/br37oSXz4d2v137JceLh/y3J5ZDrfs0xnXOewvcImKSpAUirdrC3+AtU4qcc9LYuyh2bVRY61J6qIYCq9SqwDY1XCU1QwSJOLtQvPOxBKC55+hyjm5+1mzYmW7T+NBJiTalOHoJWARoQQqqKiNVin92+srY6lQn2IbFS99Tk2Kl9e8wEpKytDoUWFE6xog4CUCiGoUmAu2nWnFH2PziEdb5sy1rZ8Sd2Uam4orlGo3qAxWniidROuil95TTJXaLbkUnuP1nneprHotTfH4skQitCNB46nA1EctnuNCRlh0rCj1I23VBvQrrNuoNaY1XMuSZ2DXEWlChXyLlnlBSgqMaATdtWpsfW+SimEZeFyfuTVD99zvr/HXSZMCBxPJyQEirW1lTEYkrZQLBrzpAxLFM5TIlhLyTPWGB4vE9Z2nK5minisGPxlxsSk8rbTo4rR3T9gUsENR3zncKaqWxrTzN976MGnHl8WA0+ZeV5o7M4nCb91FbFGzxo5qwGzpjYCdZbj8URKma5mq9cS6lJqVZdQSqv8a4lNqWXPFWLRGYR3TnnkFrw61swlEVNhKYlk9DMP/YD3jrEf6H2Pd55hGFUcq1awrZ5+aWG4wj2uGapd0q41YW633DaTkjMh1MawKVYP/EP49x+I/X7oyJnH+zvm6czbH7/n7Y+/IccFUwKDE6xkLcFOSWUAcsHYUMNwnaClFC7TpRpr9cJzLqRQi0TYEtOI6nU0iMQZyzAc6A9HxsORYTzgfU8DNv7fu/enmK5KuhqKZKwpOJuhBMIyAcLl7Cr/e6wd67UVmOrNOw3LjVXefIzMl7k2Zy4odVvbchVrVf0uJ1I2LClRRBg67QSVS9TEbsnkqElALYqqDXXrMNgENtWNdNURr8lkqfK1tEhVabDWd1in/SpVc72esDRO+B5SYW1GHuaJ5fJIqtHj/khYFrHMxTGFwhy0zVvfdcScMChrLGaVkDVWi/esNfTO4WztHBQSoKJT654qYAuYkjC5YHNSrz4FpL5KaqJqun6Weebtqx+5f/sWe/+AuczM1zeYJWE6T7o+Ip3H2Uw/qKCdkRbnLVXuw1DyrO3tZMbila/vepUHfn0P08w8n5kuD4ynI9Y7DldXXL34msONp3eWb16+RBAeHh6Zp4VlWjg/Tp9txr+oAU9ZDTg5rRoiazWisOKaK/SAeoT67wJScN5xdXVFKajGyc7wqZOhk7gVJzQBJZBV+1uFZ/RnvRi8dVAM2Sl+O6dIITGVTDIg1nIaR4bO03cD3nc45xnGQ03CKWRgrH1yv63aUsPsDeppEIp64RXqRV/KrKmStLlyhbP/yIj+cQxZm0Q5Jx7u3nJ/+4Y3P/yGNz98hymJm0NH541Swyr0M50vygCyHmoHdepm+3g+q15II2aIrDgrNOhEu5usUpuAN47hcGQ8nPR1PGF9v+mMPLniP64Rlyd/NwMO1hSsyZQcCMuFUuBsBLd4jDVatFG72Wgnndqf0lpSWIhhWUXRmocrpVZnAkE2NtVSvcpBDNZ5LX9fks7XkKuWzk6mtkZ+JhV91XmkTXpl7RZVqIVKVZBDjNHmEb5DjFfdlLZeQHXZq5ZHQaPiVAuslunMfLknJi1pb0cBolhm6aoBhzloo4ih61QWQYJi4VlTeM45+s5jrdB7p8qJS6CEWNeyWdcGaBcjW8XlTFI9HZMikoJWCbdX7ikls8wTr374nttXP2Jf3WIezizPX+BDwo0j1n+DuIMa8NEqbJvbOC3qSCEsUp2x4JBstQG69UgqhF/9QL7XPNF0uWe8vsIPHadnN9rV63ii95Zvv/ka7z23dw+czxP3t/dM5+Wdef3TxxenETbNBpWBrdhvxZL3Sczdm3RSVuK/VNzMe7cKrbdkpR4tMKpqd8aQsxZXqJJbWs8Lsuo2t0ovI1IFx2vzUlg9+I3OmFdDi9QKzxoBrEm9HcTSsPKm7bxWpRXY4BiFh+Bp4nWXcX16yBbBfCx59zETp+Zv4xK0BsMhLEyP95zv3xKmc03w5J3/u0/aVtgrp814S6ME1vFAN1Npyb7a2V2MrRkQfboihiIG33WM44FxPOC7TvH1xjeucEl9ar/j7n7foy2kp3kFU1ucKVZrasMGXdgtTwNKexRR3YtSW8VZ59bITOepJuWNFGwdlzW0rsntPWPJSIMaa/uM0vJ7O0wDadIbFWeoTUus27oh7RlOpvH0W9JZ55E2GMmbEmKbvnX+phiJtXl1DDOpeKB7MoIplaqLvmmjx1gqbPSURLDhYLt1WNAGExViE7ttEFJAbMEUMEUrVKUUTE6rEVcp2gCUmtTsOJyOxHmCRW2BPx0whwEz9tjOI85hMfhUKKlgU0ZyqTIGNdIRQYrgokY6/ZDoVIydYITsDaU4RHpk8CRviAbOMeCnM4HM9fNniHMY29F1Z1JIP0l4+NDxxWmES1TdgrCDECogph6syOqZG1H4Qjm5jT9cOBwGSk6MQ0/f+ZpYUYGf6oCTcySnALlqW4hhXhamOapX7Kxm6UMk5EbqUTxu6K3KjHqHqx5VtVHEkEihEKw2f2jMEUQ1LkxtkdS6teRatFEKxKAedahJvpxaJV0rxa0Mglg56vv4cX9UDO1dzvmHjj1uXNq6r6e2oszgJZw5v33NdHnkh7/7K17/8B2SZ3oC1ghOVIhIo4HqBYomxlIO5LJQqEloIBWloyGtO7nBug4jFus9znkV1q+YOcZRjOXm2Qt+9vM/ZTycuLp6RteN6qHvSoL++Eerltz0ZlTbRK+tGzqMtRyOPeOhIyyRZQ4sS+TqWvtQgjBNi8oZDx0d2uihH454p/0knRTF0meNuqyrqnzJ6cYvQk6RCNVJ0SpQH5QTHcOiwGOdXwX9xYJogc6S6bzlMHo6ZxlHbarR9z19VXn0nV37O1rn1gQ0LQqIC6CGaI1sC8QQOD8+MD3c8Xj3hsfbV+TuitL7FfqjwOUSiCHw5u2ZV68fuZzPK+13rr1ftSdkq+VopAUtwS9AnhbKsmCsw/e1bVl9NtaCr+92JaugVZxhNpipIz7csfQeubmiHwZunj/jH/9r/4SHb14yvX3L8vjIeDwxvPgK6z3+6oR4z+F7z/EhIjFwmiM2Z0pa1IZQSKLr9BA9XXaMJnD1ApIxxNNIHCxORsTc4MaB8GzkcXQ8Trf8w28uSLH86//mv0GOhR+++567N3f8tf9rfvV3/7AiDJ96fFEDvu7mImvp7r7QxXj3JCFhjKhspdEdULIojFJ1TlQp0KgoUp1HptnTHY6+D++b4D1YRWxy3i0Mq4vHVAEro8UEpp5ohXSyYuglRJ1gtpW+bx54NbHARhvb5HM3T2vDxutnlE0n43M26N+1m2+Gb6/N0X5SKCkS5jPh8sh8vufycEdvM96V2hV997tlTyFTPmszeFp0U7bPWBPPrRlCTTQ5p/0gU2uoq8UkXTes2LcKm1nyO3fw/8XRgp5CVRdEKn3MVlpr1K9AKyxrdNecCsVpxGiMBVs9cGdVK0QqTVSUNaI9NjforS3oluBreYVtMtdDtrxRrhdcSiuIMTsP3K0Mpz1s+X6SvSbN13nZrqcmW4PK2DYPHDvS4qF2xJi06/wUmOfIPEes12YQKbdGBw1HElpIUWg9ATSBWVKu8yojpRIHUOhF9cPBZI1kyEnbscVI2Xngpnrgx6srRNSjNt7iDwfMacRYj3iv41LAhoQJmW5O+ErRzCmRKQTRiusuGvoMfcj0qRAFxBmKcYizWA/SdySvycolLoQ5cBiOfHNzjcGwnGeIhXEY1rn2OfP7y2LgSTV1g5Wtf+TOkOXVq8wgUbPSnbYw6usCssYzDAM5ZYZhYBh6fHb0vd6a93bz4NeiH0Mu8MOrW1J+wBplspjqe5ETuSiFyFqDG470nWcceg6Hkc5X5kXWXoeaiLKUmlhtHXdS1TRpxzr5U3rvGW0O8xartt9vAknOmspT/0MMmLz3tQWmWgoeOD/c8+r775jOD0yPt5RwAbGrcFDOmQCq2xGrpEELhUV07ytSW4ShBSBFl10uugHEoouopFhTWZVRLpar04nxeM2z5y+5efaCrh9xvvuI173/zueHoB8+Ko2xFBoEp8lVEOswBobDyPXzG+ZLIDMRQ2YYD4zHE1CZJhTmEJiXpSrqBciJlAUxHuPAdeXJLRhf6Cu0MtfmFzlF5ukCGPq+wzXxtYpTNyPYWD8K+6kxlmqsu75jGEc639FVOEoThr5qYlcjTl0iq/a4WbvR5ArzhWXh8e4t08Mt4fJAWh4Rf3hSWpZL4ccffuS7H3/k8fGRu9szS1hU1taqFsrSmvtaQaTofS5J5RjQJm2SEqYoy+Ry/0gR0QjYFK1STR5jhGAUTvVDT38YkbHHpYCvTJUlRDJCf7rS8ndjKIcBnOfsvc6d84WSC29e33L72ze4JdJNcd2kyVpANKM67WPuoDjwI3z9AJ0HXyjOYjqLGxQafPPqLbkUpjmzLJkXz1/yzfEbjDOqZRNj5aa3JNGnz9RP6cjzS+C/QjvPF+AvSin/uYi8AP4b4M+BvwH+aSnlzad/tBq4yxRwDdbceY0iAlkq7JBX5okpqkrmjQNnsMbRdz0pJvq+o+s84Cj4VW/cGKnGfaifq6pv0xx5PE8VmlGRG6rEa06ZmBJSHNYIndPPGcdRq/JKbYBcsTJTCuJcNYeKA+eSq5Tads+lLs4n97n+cAsQ1gKkVe+5YETPX1bQ8MMGa2Xh7I8nvy5PvtdQ+pwSOSxM5wfevvqe6fzAcr6nxBnxHdZ0GCOquZE1fxBaI46y21aaN5ebhoZtATJNWbIU9Z5yimrIES3CsIZ+PHLz/Ctunr3gdP0M36mY1bs39OG7//i4fNrRQjWD9kqridX6M7EWg9CNI1fX13i/MM+GxSb6cWQYRwqFGBZSTkyXiWVe1i4zUr1IxCnrwleMu84DX/H/EALloTVFjizzhPM9w6jKec45bTBSQKrsa2vbVcymJdS8bd919MOAdw7nHc46nPVrZNNC1GbENw88r3mhNidjWLg83DE93LLMj6RwwaTwxICXUnj94xv+77/6G0IITBetgCw+gjXE2kqtIHijeukpJ+Ki5f9dzR/YrNBITInLPJNLwdkqhuYtLseaolIFTOJILxmZD7gUcbVRc4iRIkJ/PFK8Y3aGOPcsGS5Z11maZtIcePvmnvsfbvFL5DQlJBWo+Z9IIhSV7010gIfDCR7OMHYU34FzmM7jhp5lWbh9/ZZ5WricF6ZLxEZD+fOssNnKQNuUNv/YHngE/uNSyv8uIlfA/yYi/yPwHwD/UynlX4jIPwf+OfCffPInw5rETOVpYUuDEPYFMZSyJXJquJVTfiIcJUZw3q1wibXCOA445zgcRk7HAwU1QCkm7h7O3D+eawWdes9WOsWBY2IOEeccV6cDh3Gk75X9oVD0rnK0ui3NQOkqYhUZgp9+JKU89avXhbT/kNW6r1fx0eM9COVptF0/66nHlVJUWtiizRhimNEGvuqd5VLWZCtoxxcNg2lBrY5BK1BJiVKkVlVWznPdpUTryrHOKBbuHP14wvmO0/UNp+sbhsMJY5XWpnCSQmbbPZR3bm6P8O/v//OMeqlGvDRxatSgbxRH5az3h4EijuFYsC7TDRopKD6ryfJWmFNSJIeAdh4yGOehST4Im5SwUejO1k40qfa2jCkhVWJWoblaVNMoO6U+TSlrotVW9UCNPm2FIa1WLlvl2df6/49MlT1UWOHFlEkxsMxnlvmihu39UwCsYnMpxZWr3oRxV22V9tTqo2tQ0iqvnAHRQi9jlS9uHArFVUMpguqFF3DW0llLZwxOmgR0QUkJKitgDXhrGJzDNO2aIpyXyHyeeJhm7ueFLkTOIVNSwQv4OueWVcsmU0gYYOg8qR94dnVFN3owhSIZkwvuMpEuE/YcMVOCy0KcZgJGx6d25vqQpMxPHZ/SUu074Lv673sR+UvgT4B/F+2VCfBfAv8zn2nAc64JPMlPqiSbjomtE6+ac7RJqiYxc0yEnFmWmXmaWOYZZ40abGtwXpkpL148ZxgHnt1c8/z5DSArh5XW1KEWUQjCYejonGOaF87nC847fvbtNxwOB46nsbIntjJ3MdWzrAZOlRD1eyFHQoqKo6NVhg3fWzcmPoxXS6V9NfhnzyX/w/xLWb8W1B6amhNYpjOP9295vL/l8fEtYb5gTGYYFTcN2gOr0i9rBWvz9MVQpBCDjm2KEJa8ak6vRrGG9orjZjrb4W1HdzjxzS/+lGE88s3PfsnzF1/jXIfpBj13aUVNrJP8KZi0H5UNpZf1/59qxFtTL1ZWXKMt1kwEBfCHAzdfv2ScE8ZdscyZm+fPGQ+nqp1SVsaJsYZlunCezkgp2i6uslja5thKq63T4pMQAv14JIbIw8M9l8tFVTXtvM4/5xwlaEUnUMv4tSHF0Hf0nav0QI/3vb6cw/cKoRjnwXlVE6wDJmu6olYOS0vi6gYfY2C+PPDw9nvm8x05LzhnwL4ztkWTtNPlQsy11yOCQyuvjRGMqxs+ivXb6kg50cjZiUAJNamrdQEFVG/bGuzgcMdBPe8gkAND33HddVz5jkGEnoKtDBJDxtd8xFXyjCIsReiz4Vxm/v7+wqvXb/j161v+/vaeMUZ8hGOBZ7ajs46U4b6SCm6IJI31eX66xtxc8exPf0G8PvH48Jr72x85x8Tj67fI3T1xgjgL9Hc8vrljGXV8QtC6AGOpDUM+fYV/FgYuIn8O/FvA/wp8W407wG9QiOVD7/lnwD8DuLm5ee/nW1hd1tBf/72nqClGRtU/gJrYK6yNXZUaqO2wnHN0tay5r9DJOA4cxnFNnIaYdJL33VqibEQYh4Gh86tglneOYegZhp7OuzWZtNKuTOWrtGwpPIEqlOkhvGs8dNJW73ylCG74/24E1+TSWo7/U2qEHznWdzVsuo6rUJ7Qw1IK5BTJOapCW72exl1WZcAPZ8sbPJUy1VuEqjlX25TpS6qynXUe3w34fmQ4nNakZTeMChlV71vHTHZ/ryP8oTv88P8+wYavRrqxLtq9765BcX0V6i8kdI/JaiydGiftw5xx3uO9J8ewRZLWYiqlr0WMTTnT1kbWiNB1PSrhuiU1Gz2V9T1PYcd2LltbuUkVwmr6Io0UIJXGKbJRErcR2Bnx9fyNIVKbbISFFKv8sf2wC94iMnYsHp3ndT1Ic0eaE0ClSqrjZoxQTKaYhgvXB2iMVuoap7ClFMWiU67a4PW99VpzK+gpSeFQazDO4QqYYijZEp2qY4agbLI5qX7JkqEraM/c2jmoSd1WaTkMKp3svac7HCinExLOBGvIBvoUSTHgo2CjQAgs80Q2svPA87ppfs7q/mQDLiIn4L8F/qNSyt07lLUiH6ljLqX8BfAXAL/4xS+e/E7zIqxsEEqpO7f+4vIEF7bWMPRdhTtqy6hl4fH8QAgLxgrjODIMHcejdjS/vjrR1R6Y03QB1LCklDACQ+91UmU9/9cvXnB1OvL4+Mjbt7dYa3l2fWIcR7re136aCtUYUXnKnJsojqmhdy1pNnbFBTvjVonSJYQn8EgIYavCXLuY6/u0CbPSs/pes+SXyfGHimGvnnihyqDmmrXX/oHWFLLqdUKJxCS1CW3Zdc5uy69UY60RVYxR11puUJhGH8YaxGnByNWLl3T9yPWLr7h5/oJ+OPD85dd0fU8/HsF2FExdHtWQrtBRXo3YZ970Jx2pivPTIBOo+it6kiJgO2G8cviQKQRiKBzGI90wYIzQ+aY0qfmXsBw5HQZK0b6qzVC1aFOkbai6UTd96xSVampq3qVh0e097TwKGSrNbqjJdlcZPpsEASCmQitVs6Y2mFDueFXerAyRNQqs0EtOibjMFd/XxhVuGLFeSG4kPPE7hH7sOD07Mk8L0+0DMWbKDKZJwK6cfn055xh6j7eGcdBIOvlZKYUhEs4X1Su3uqkb5ynjUWGR3iIlYjpP6XuWAr/58QfuponFCEkyvbP87DRgZUQqv3vKwn2yvH0486/GkTfOY42rSpiQbCah0U0vQhCDt55Uk8cLCfFwGjzjaeD6T76m+/Ylr78P/OBec/Yd3QvDWeA3d5rvId7y69/+LaXreXh4ZJoWYpoZDp5x7DAtKfgJxycZcBHxqPH+r0sp/1399m9F5OellO9E5OfA95/8qdt51VPYPfhSykonXDW664R11iCkqpGgi3hZFub5QoxRF07nGceRq6srFZyprBFQgfwVd08ZkULXudUB8NZxc3PFs+trOucoSStEj+PIMA6qbla981YOn5o4kYiK6rB5HgJaUVGgaSWuLJTd/Tb5WdiJ1tczqTendMquhr4uyoYKfPbxFIRR9EQxQsVs1VMxUjCmqN5yUbbNEhr1co8+64alDS5UUjfFiCC4NUoxq6KesQ7f9VzfvGA8XvHy21/w8tuf47uew/U11rodA1sbPH/0Pj9kxFcv+/Nw7/XtUkuVZAMkm3jWHpoxztIZsD5TiifFwuAHnOs0ye7tmovpOk8KvToLOUPR2geptQFbZNT+KmRj1yhwCYtCcktgnib11mhSrDyBX4wxdL6j77rartBWJlSNHERWCmG7wAqhr13U1yjYNHininnlTCpBI7SivWdc36v3S0csOw11wHeO8ThoUdDbRMoRomCyRjDivf6iabbA0XmFl7phwFlLNAaxliAz6fFcBa303pNx0I+IFaX+odXPxVoC8Pr2lodppj+NDIcOezrw/PnXDN7jAVPgkg23yeKN5dD32prN2GrAC0kKqW6UHqUUWyylCLlEreC2haGzHEfPt1/dcPzZV/T5FXLxXJKjnAznRZiWzOMcmNMjP775nuQ8cdG1lXKk6y1d73jaAPx3H5/CQhHgvwD+spTyn+1+9D8A/z7wL+rX//6TP3U7N85oufHKrMjbsmtOvuYFM6VWbD5J8O0mYDOsijvqJE0ps0hUjm1LNFYHzlpH33WV7ZFrJxZZvS8xrRVWgzveXzS5Jdaq0WhhLAjFKj2xlE3HIZeyhuXqRZCWAAAgAElEQVSmhksmWuyOPtQWvjGAWDrT1Wa7WkRiJ8XaP3bsGSjv4+t5NY7KaCnKY0+FtCws01k54HEixplUO3qnbEhZGRLiajRRmnpkIpOqgdAx2+8vrUuNMx5nLN51XJ1uON0853i6ph+OmCqu35Qn2nTYienqOL8HmexNxvblUyGTDx5Vf71prouYBk6yRh3Vh0AKrjeqGFiNZZMY1paBnZaqW52XJSdSmKtGR15zCVvbvA1OK3VjddYofIcmBkvjJbcEecWIG2TSOPZSmyK06lU9bYPgthqDXD1vayGZrPdWaYSIhVJnfqlUQsA4hykeKyCSSNFrB5TdYa2h6x3z3BpZK5Oj5KR4r7Fa07F34Cp8lU1VhDRqkIsYQsxVetgiRVSOOGrJadPTmQs85EIPjF7xfj+MHA8HBu9hCeQYOU8TJSyk7oAbr+kl00uhl0InBV8ynkJnhA6wZKRELAUvKlPgS8GVjORAnM8s53vevvqOyS28+fHX3L35gfnuLSFcyDlgLfS9RQZPOQ4k15N6TaJOlzOu81hnP7BmP358igf+bwP/HvB/isj/Ub/3n6KG+1+KyH8I/C3wTz/5U+uhbaK0urFlqTEasqwhrLB2uxGhSluyNkMuRtkBiGC9p6Pg+x7vO8QIS4gQopbbO5WtbSXK3ndcnU61cnNZscGiF7d6vmoLGs+5eTG1MTK1y+Pq9ZUqkSsY3Cqqn2oHGsWb81rE0kqmDbJi0KDyocbpBmecev2+6xV7nRZgfjKWe0z9Y4cmW1tbhUZJFEqAHAvz+cz54Q2X81vm6Y64zMyXQAxJaXx2UNpmf8Q2GluIlVyVVpaItULT2chF8w05g3EDznqG4cA33/yc5y+/Zbi6ob+6oQCxpC0nut0VT432nj4o73x95/h9jDdSaQ5UD8JUo1i9RdUC0DxCqcbT1i7wWdQBMdpQtzXZRVghv5wTl4dbwjwRlokQzoBgfGX6pFZm3njYmc47rBkIQSlzORdiUIaLq2X5sGHWW8m8MjeMVZikdS9TzFtbw4Eab4XxcjX6ZjPgRhs2CELKSTdjCqbvcU7wbsBIIV0E4i5aEvCd5XDsWJaJUhIpBnUAKBinbBzd6EwTJKxr3pCtJXmnmL8IaQpcZl2nEgVcxnc985y0C1HvEQOPRO5T4FDgNAx0pwPjzTUvXrxgyAl5vBBj4O6H33C+v+Xw8hue/dk/IUjkZDJXpnAgM5TMQGE0htEIPmYkJ22mYSFLYUyZoSRMmljuX1PMwvlfFXh95O7Nr3j76u9Ij2eWyx0pKQ30eOoYrkYOL24o3UBBpaFjXBh+rbpK76qq/q7jU1go/wsfXwr/zid/0k8cbdfZV4StxSCStUhGZLeU36Ed1ky+qSyV5gGl0mCYFYmp9/WUpig1oaIO6VNjXcdh9Yq285Un194Wfesk1JJWpYpyFakyoEU3L1thmGwjWeJ239upatJpq2CU9sPf+yirY1r9N32VTCmJklW9rVTjqx6RVZnRbsQ6xzAesd5jl4BxmvA0xlJyJIWFEhfVv44LK6uyaJTVHsFKa6ssjVyoAh7vbkLvb0qFLTrbRgv+wIGpp6i5GAOyS7YqnVHnmTQAY70v3ZzXvMQKT+g5VgMueW1EQOVXq1cqFYZ4d7NinYNPKLXkOrc2uh3wRMb4Q7Rc9bqbB75FObm02gSpRWa6gcBGIoB2r7neojKDTG3tJub956QRcWXb1Hm8rpu2GNd1pd/P1VXKqBRD+5rWaK8oZbAVFuWqrVKfScJQWzJo0rzi/y26CWGBZWGaLlzOj9jpTAgTMS6UFKDChw3atc5iRWqtQt5BPprjcmjAFlIkh5n0+Eg2ifPjWRU6l0DMhVwMRSziPNZ58I7iPaVYSqlspEZS+Ixp/IVL6dUzFVO2kNNapO+qIW3FA14nzi4rrvCGrfKdrp7PIyL4rtskW7PCBI0PW3aTs+HRDZZBYAkLl4sWUjSMsk2ulFPF5xU/N0aqRyHVo95hvmJUlK+r3PG6cHQy6ufZWsDwIAKNTVNph7J/jqJTOuYIWVb50G0g21+62N7bWJ6MedPOqDBUyQgzIjPOLngXsDap8h0OP57opOPq5gXPv/4W3/Wcbp7jfaca3zHqYooKC7z+/jvu3rzicn7g7asfFPcUlSKgQFwWlnnmfH5geDwgQ0/HqW6YpcIm+zn8sajij2CsP3RWUWW+4gzGKAvGWKXjKSPEa/KvDnfOmWVZViZTiUn7qDZ5hJK0ZVaO5LCQY+Dx4ZYwnYlhJiyTzpVS1StbErtKMpArNl3zQTrH1Ihnq9uxRH2WtjovbjWYZp3DGxwopCKU3HA6IYZEYcKIIYZFy+8rxdFYXzcR5bWHGMhkrHcYB12nmiRuyWjJSHs62n3mNA7EY+D66oSznumcWBbFMBtBOMcEUgjOMs9OaZHdgs2WuATiEpjmoHrnSWoQVChRoT+NWhwJS0QIFDrx0PVI17OUxMN0RqYzb374LXmeuP3hO853bzikhVsvPC6R17ff83B+AyZy9exIXwq9tXgRSErXDCUzl6ARxrHnNDjyzRXfzxO5JB7+dmbpLDk+kkKGYCjxikJm8keSOSD9Na47UnzHEgIxBzJpM+CfcXxxA674qeLTe29Yf1a51bt72tgazbuohtlaTNYdcWtovL2x6ZfILsrb65Hszx+ils8+pVFR+eIK8Khz3ZJzunAaJNI8nqY1odRDS+Net1Cx6VosjxcWa2lUytWhFxDZEoYpb6p37w9mc2qe/uwJrLImFdqXUv2WiBAxErFW20mpt2xw/oBzR47XX/Him1/Q9SM3z1/gu77K4CY1UimQU6C0yrLaxSUXtHBFlErYuMTaEGFmqDIBH7LH8t4/PvqN7b7e+dFPwUrvnbkaaZzFGK+c5Zp4lSq+ZaxbC2hSThSxuvmaSDbaoLKESEu2aBI4qmpfCIT5wjydySmQwowYQ3K6mZvWzALWyHLzhEud74WCBSlrvkGk6vTs5lxrePK+F17nSo1wc9brFVHnQLF7B7g1ua5j2aSNc61GtjinnHdjFYffD7c1lq7zdJ1n6HtygjBNVZ9726hzVnmyFLW6txhhqSyxGHSOLak2ui66z1e3e60+z8WQsWQyCUsWbdYs1hFLYY4LeZ4ID/ek6cztwy3nh1vmzhLfHrkskfPlnmU5UyTTjx1d0cIgC3UjhZQjMRQwYK8O9KcRhp77EIk58SZOTKZgJCm7LgmStYo4mCPFHcEfMK6jWAexJoSlrHnyz4DAv3RHHlbvVsXtnybmUpUi1SRKebIYG1MlxqbkpzzwJpf59DO2xCHURWqofPFu9cRFZNWKiKLGqGWEV1YM7Xt+5ftaFKvPSVZvXNr1S2OBxzUE30I6hVhSjqSsC2OrnNslnpJGH3UJf9iAf9JgQ7saoWBKIseJ6fyaGM7My5mYEq4b+OrrX0IxjMef0XVXnJ494+bF1zjv6caTGrKcMUlVTExRru1wfE1/fGCeA9Z2lAzeqpqjre2yjGTCclHB++WGXEuq/2Cv+jPf/qFRNMbiRhX2t7bH1E41rhv0uTilclI0qsol0y1aNp+rwl5JiTLPlJRZzg+EeSLGwHS+J4WFsFwUXqqNBshFxyCr/oetNQVNfpbZtt63SteEmm+oY1bnRtYGmwjdmiNa5WJpUGNNyBpNDBaUwVEzsqsolzGaSEwpEaruSggLKS1AwnndZLreYC1YB+qB79eoClnFJaGa6jURXMpaNt544qDfiyGq+swSMbkQQ6yaO7uesVV6WmG6iBhDCIlcNctBSEvm8e4BSYnfSIblTL6cWV79QF4mpssjIQbc4wOvv/8tc8rcLRdmScjoGb66xqbMJUT1upPwkCGIZT50iDXcvTjRnQaiGJYUyBlmq4l4K0WjTjGqspshWAguIwTccqZEw/l8ZllmLtNZx+LT4W/g/wcGPO+Md+tH2Txw7ZiTVi95j+k1DYH2yrnya62s3Nq9OFbDr9V+qvHsuo5hGNaGCiJCP/QMw8BiZNMhVqxBObJBaYwl13AWlZzNBRUsElGsU7bNSd/dsvd2bR1mK44Zk2ohAKtW9BaJ5Lpx6KLNCDlXIO4njubRb98AKZVDWyKGQFzO3N99xzI/cLncEVOk749c//LPcG7g+tkvGQ7PcENHdxxBNNrZXAUVAHOiXubxzR3H88QyR5wfoEDvLd4aFYJyYCUzXx54dIbD5TlpCWpUWgTzx5laP318wIIb5xiPN9hxwPkB6yp1sx8UWnDKn9Y/GiGGVPVgdjBYuFzIMZBKZF4uLHHm4f6NFsDMZ3IMVRVAoGRiWCrUgeqtWIv16rmJPSu+W7a+l8YoJt+aYFAKOcf6WFT4TGprsn00qlCKr8lJLTTPBcza5q1h3KqeGENknmesjYTlQowTEOgHlQUeBod1gr976nwVtCjmcp5Z5qjRi9kUP0vOlCp/0dzxnDIhRBU+mxckWlI14CmkNepZFUBrm76CIK7DFKP0VxHSlHn76i3TwwPh8S2vXnlymFju31BSRPKC5Ei+DaSHO2KGuymxSIFTz6nrSCHycHdPCgGSQbJgnMWOPc5bXn9zZLkeuDxO3L55BODQeZwzeAPeNe12JQtEEaLJCAv2ckcWeLi7Z75MPJ4fKOTPznF9cQilNGrSDrdtWPUmt7rBHPvy82bI9w0cniQd2RlxY2oPvqfJnna+/fkV8jBPPouWNF0NV1mvUb338uS6VrRiB1eAeulFqm54S+DklrzbtM/372+FQWsxy2c44E8ohUBLA5ccyXkhxpnQJEEBa3u8PzCM1zg30I8Hun7AeO1Yrv1Ca1/KNh7UMS1FKwl9j/OdVhRat+v/VwtVyMS4qEcXw5rIk7YxfM4N/sQ9f+4hKFfd1N6Q1vpaNem3AhujxtuIzqksYLJ2gBdjlAKXM9kYjPeIs5tswyrRWmoeYgX0qKHmdh21+EbE6mvX3COX1thku9d1nqzOyi5ZuL/DRjNsVbHt8xt0BxtuXp0IhSkSpUSaPo4xZcXK5V3mRFG9+8t51kKVEEmx1jusicsMWbSParuXXJBcNOpEVCyurf92X+uaUKZTK/WvXpTCm5X5FS0sAaYlk8LCEjVStEWZX6loS7dcVCEzic5lnD7DIDWuKEo/dgUGa8mVox5EqYtTqmqIVb1SKre9oC0ISoFkVEvciKZZ1RxlhU+kfLbxhi9uwGvhB3n1ohsMAo0NsrUcgwoByIZzpxQr9Y6KW+pEalh5S/4IEEULh/pOe2d679kXDkGtnqwUP+9VKMc5VwsfCipwz3p9y6wav3v83lQK1xa2smYlS4qUqCcw1YaHuKC4uqGrmsRtQaScWOJSQ+p9u7J3j/3T/5gBK7WXO8zzPcv5NdN0z8PbV4RwYehvuLn+hmG44fr5n6r3OQyqmSGGLJqIzbV6r9kbg8IJZEPXH7i6fka8PHIcTwRj6V3GGYWxii0YE7mc7wgpcPXia8I8Y31R7ZOaoPtsI/4B/Pv3OcRYvB9x3QHXj9p4orZIawlqpDXfrbooyVGKNhVoXZa6YSSlyDxftHvNMq+whT7D6rGWslOI0JuQ2qnIef3MrusrxAIpTdXZCOqVx6QSvdU5KVRlvdqotzGhtvszGN8ptq+ADaUkyHHdNETA9722CDSGlIJKJOSJzISzMAyqTOm9xYrB2enJ+JdSePPqnr/561+rIb8s5JRZ5lpMV4XTEIPxtdgoZZCqrLgkMJCWoLDKHMhZi8xKqxXJKvcg2WgzFSJOClgtQlvCrOd3lmSswli10tSWJh69adcHtHi4GMAUgoHHnAkpscyZMBeGw8ALsRTrmCKYKfI4J+6XBKYgYumdEKzQagSjqZuTBJCMtanCSQbvFY71ncM4hbw+BwT/8g0dSgbSCoWYirttCZxN76QdglBMoZhNp3hL/D31wJ/AL2xzbF9htsfNm5e9JUIbZm52CQZNZrZCFuV3b967zXkz5nXBN3pQFmUoCJV1VljPJXVjav0RV28rKwZem9p82tjuDOD6liJU1W1yWgiLFu0sy0QMC+Pg6bsr+uGa8XCDcR04pdRtaoP7qEJfe3NrraPr+rVPKCngbMKZXAs0NMRPcaGIdpZJKWpl3kev/xNu+o+Eu+gzs9ULf/pa2SEN5qlfDc27NjW5XcjOYVLU3phdXzffDbOWdaP6yM1Im39m7Rgva2RaBdRKa/bRnoVyzduckV3EtgZusvPAqxEpaVdcJqxOlHLYtfuV9pjV3rDaQ7NqvjTYy5j3HsE0LdzfPZJiItQ6iJTqlVaIVEyujVmqZnz1okuqUr5JmT05Nc+9CvuWSt+sdFdyRnJeve+ca3PoVLQheVQRtrnKBJiiUVTz5Aso9VCgSKEYSAKxQMgwp8wcM5I0r5XFqPRAUmmJUNd0pGBNQYyo4UY3hlLASEEkVSegqiNawRal0q7R+2fM1y/ugWuj3qQd12vFXjatIMGvPNfmrdZ3VsPaQnI99lzyPQyjxl/DtlwsLihNKoSFEJd1EewRigK1FJKV2tPUAdWj1/BXS4+VSZOi6oCbymDZQlnBeVv1LDTjvhrmUhNYMVe+N7WoIddxSZVqaTDdVgH6gdFsV/0R31Un/DLPkCLnu9fcvf2BlCJdd8J3J45X33C8+gbXjWC98millVWpqdrDMO0wu1GzlbHRDQP9YcQ54dnJM/ZWsV1vSRkelkLIQlhm7m/f0A1HXD/iuj8cQvlDj8bNWUWXqqEUI3UEGhWTFZNtrA5qEk055LtGCU1ztRq7Uj1A06AzUUPYkqRaWWhpDYe7Xo2gtY5StAGAyiJHYgi1m0/LH2kCndrCT9YhbRO85i5qUjOnWGVrDbbrdRPuB/qx10u2+mxzHsnFaAFPJzWPszk878JfORXiUlRNouZr2yJu8gy0AuQkiM0YW8BkSrFgEjGoB15i7S6/xxDXYqdIU2zPUohGDSPWaFReNdDF2opHF6RY7e7TNgCq4a5PT4DQB0ooLNNC4YGwnJUjTsZR6AfPeOqwXU83XCFGGG86XK96M97p0w1VQ8jUDdaIykwAlLRQSqoFhO3Ppx9fHANPKSI5VhW8jSKofSqdagMLNVzdYeWU2nD1Xdrd5j03492426VkbLZ4J5hcDXgIq32SSnVrWOKqDKau8YYLoq6wslsKGY0Ctihhg29CCKpyOComrDiiYoExKGxk6yalWKIa8NZvMmXFDo3VHd6Z1tLtgyPKu+a14ZuCcpHDdEdcJh5uf+Dtq++x3nO4fo7zA8ebn3G6/rluXNYqNkdE/YpWHl8b9da/2ydKXZzOWbp+M+Clt3z19TOuT4MyWIaOeYn86reveLzMhPnC7ZvXHE6Rq2cvoPhtCv+kK/IubvJHwlHWbUravl8bJ5Qqt9Dw3lp08mTX3zkRorS/9lIGSH0VdQZsi/har1CruYbW7LlV4IKwhLQyU5ZF50UMibCEqrKnBj8X9TZXA75KVbDdFxvkVwRKShWm6/Fdp89v7JGq9yKmeseo+p9pHmQ1du/ylwsobXBpCorvRG+lkGuT4NQizGbAxVa+tyVWbZ2SKy1zFVsAiiaMMUahQSlkUxRrNlCaAfce5z22FKzp9RqyVkDqRqI8dDE1Cq64fgwRKYZlXljmxPnhrHsCWk4/jB2H6wPH4vnqucoeS685D2NVFrgAsVRmT47YCgORldkTkxYpmZ0B/xwH5ouzUOq2zNOLfvq9DUKWVUl180I3dgm8752+y4HV3GEz0mX1ppunpJe0/Ry2kuncVvIHd8m2SJqHscE3iFa4KSyi7dr2Gf936Y+6CVW1OqP9Pluh008O524km/EuOWkrr7gwXx4JFZfVakiP74/4bsD4jmKbGuLuGby3IzTgZIe8tyRulUr1Xcd4PFKSpxvUKLjO47ueLMoACkklEdSLqgJPFeD8WL+dp8dWtPR0BD7wrc+y65u1ac9U2+Stsd4n+UllB13Uro66EYhU49yKuajb4p63bdZ5K7Vq1VqrTkCF6NboqGyVjNt4tIXSCn80oth+DzWajaFSX9YZfXnV1DcGjBc1jlkNnqxGVHZD/v4AbwFzM7ltYyzvfL/67lkdIhFlqOgi0nmxAt/ts2qwk6MWikmMZAmqd1TFu5zr8N4zDgeuTuOTxSG5SUloklSkrM6Tyg9YwrLgxDBPM2mOSC6cjiPffvs1w9Dx/OU1x2cjkh2m9HodXht1GOvWzTbk2s8zRyRF3XjiTMqJkLTrkhFDSZm8Sud+2vGFDfhTHuhuHtW1ovi4CJWy17zkakLbgyVTiqzFNNC0q8uKcevkVcOYW6l4451LK8LRCrtWndlwxFSUuid5M7jGNn3BeicVEtmzX5psbRZY5kwymtGnaEGO1ERY5z1dp5OtsTW8c7iuU4+28yvGro1c4cmC2UX7upE0PFPDtpQW4vTIMp15/f0/MD3cY61lGE704xU3X/0Jvj9i+hPJ2epZ1YKlnSndjFZTx5bt8+uist7Tycjp2TN+9ss/paTA1WgZvEJifd/hQuLZXOjHhYQn5ZlSFigB7XuyaYD/UY7PONW68Za0bSx7LDm3ubmdVJpR2sF8LTRPORFzIuTEkhOJTGedVi86T++0UjfXJgFNJ9zYTSfEOY8VoR8WhsOIWawm/3NST7OyRFJS89j0QzAWcR3iPMVYMpV1ESNgGIy2WsspVsMldKOnHzoOp57j1aDets0gmRi0uK7kRG4JnPJhd2YdT2nIUnUnpNY/7K3pms9VKqSIqOi2tPWS17Z7KxSF6vcs5wnjIh6L9QGGDmHAdI7j4Tmn04E/+flLfv71C5WjrjLQrtJABdZI3rgqRlbHfponfvP9b7lcJu5/ecvj3R0311f841/+qfYHuO5wg0OyVnxTRDWDmriWtZRcWGJQGYC41L6zsergLNjvfo3Jb3gjt6QlalXmO/Uwv+v44h74GuKL7vJ7bJv1ZxvZqRXWtETOUw98m0h7/Fk9GbP+fKVx1d8xFT556r2X986TV/hmfzbeec8uXN0Z89bxp6xSoqIYp2y60GuiFFbj/iQ5tEYF28i9/+nNe95dT46kMBOXmWXSKsBhPNJZbQHmuwO+HynO12SWfo686809+aQN75T9v00Vy+86hsOBkgPeZaV5O00GuqIFUznDkg05FfW834m6/qhG/JOPZqm3+aEMj7J7njtn/Mn29hS+aiFxbsa8euFFmlJgDbNLoRg9/1psIxtlVcPr2g7N2tXAN6bT6gs/8cDrtTW3sv6e5gHr7+3mYIv6rG1euMU5U2Uu9Fw5Gwp2G4BdxPmTR4Mc2wRvnvD+7YpT1dZ1ifaLUvJKNVwTwWy1IgXBhqARjLPVWVcPvO8GDuOJq9M13lkOvW6GTkQ7AEnLBej8lFZ8Yw2XaWKaJrquwxlh6BzPrq/5+puXDEOPPThMp3IHqiIggEeT2QqBqRywGuUUnNJmUyJYS3Cezvc467WWJO2bV3za8UUNuDHarMAgtbFnE9DZG/HNqOt79L3bmirEpA0ELtNETElDzRq+bIU0uohkN+maJ66LY8v676GUPbzxJEnaVrC1SNHJlStlsfU3zNbgnd153YogUzcMTdJK9fw3g5FzZpnnld5YStMK189M0fFuIY+G6Q2drq5i1Sc5373h9offEIOKTLmu53h1w/WLl7huXHU+dOw1olk3sN35Pzav9gVD+w3JeUeJmSXMhDngnF8bBlhr6fseh6MTRz8etLDpSx9FGUb/T3vnFiPLet313/rq0t2zZ199Tnw5tshBsRKZCJIoihyBEDIg7BDFPOTBEIERlvyCRLhIKJYfAImXCEQAyTiyEohBUQyYhFiWggjGUsRDDDFBxviSnMQotvGJffZ1Zrq7Lt+3eFjrq6qey94z+5zM7JZ6bc3u7urqqu9W61vX/4qxQ7zYr6ImlVEgMRrImoBkkLKMTzyK37bRJi9DFq20Xtcb8NesmiOhHMMK0WH8h5qVYVyTSQtUhKqqmc0XhFDQNM1QcabvSmPPg0JkJsEiBIqyssIHnmyWdNJUB+oq65oFe5R1yWyvZjYvqWpxe7TFk4/P0elr4cy1kdvlu4wN2ab2ODRdsjDAoJVPxBJHHc0/ssSfvu0IISISiF1HFSDMaoLC9Wv73L55i+du3+HbnnveIGCDaY+F5udkNOqo9zOny3dtAylSqFIXAeqaQmC5PLICMmtBSgvl7JqWlKDr8UQ7a6dV/zJfV9c2dF1HXVVc39+3DVuFqqgopRitkxfg4FfMwIWqKK3YgUAIY2ge4BL5RFU94bC0h2cApGpsI7D47ZEJb/yhqIcyjSaPDBJUTFonA2PKDOnUMB8PyUKVMpiNMZ8LhdsNla5bu7PEpKGMXT7Ffsn2OAWDzZwcB78uQorhxByrWyZNRnIo165B+5b1wQPuf+tlUorUHt632L/OzTvfhhSVSd4h+DKeBgUek+YeQ5l3DTgwhfUvaqRZ9cR2TVVGx5gxvJG6qNCiRkNNNZufTAZ5DWlisX3Cea7Gp56YuiHRpEgW1JskETzszPiJDgw728vVOaRmJ/qEiacY0crs2oNt2uxjJgiEDF9bDAJFKECTpdbXsxwbXlkOROxpGzN75UiMLGBMwyGRMKTjj84yi0kv65qiTFR1yWxRMpuXlLUM0Scc0y2yJM/xv+Nj6cKySft5EmRDhZHJdxtPlmZdMzPvLHyP56Sk9G3n68bNmlVJ2XWIKvuLPW7s3+D2zdvcufM6RCP0a9BEGGzrNk9JLaEnpkTsI23XGaCdWtZDXVgBiiDCarWiaZvBYtC2LcvVkhgT68bS/lMa+UsWxNbrhqZtuHH9BsULL1AWpTHwUHmW6nEd7sl05XHgFlaXNhhVlrhziSjfKP2Yp6+H7CCEeXKpLkwA7ScSoUm4aTC1jN9ZNpmIDBV6coWU7GDMahqMyUNTZqDuoBjSg1WJuOlmkCYSMvQrtylnuLkHWs3R2TnjNk3A1N2YLMurcDVUydAAAB1KSURBVND6vi+A2YnxFFc30YjGnvXS7N6rpWGcBBHq+R51Paea7SGFp1RPTUea7W/5Acq2y/H94yj7DrquY7VaEbs1zWpJ36yoq8o22mD4GeLp3KqRkMfP8rrHB//CdDqrHuZr45qncR3IEUsD3KsEw9oJICmSXODAQ0qZQK9qxu5RHcJi8xqxdniB4ORrAjcj+BoZm5dNd/m6vv5DQSpLqqq2og4x0s1mxpQ0EtANx+Sw3t2MKHkMNG/WyR2VJVVdDOGuIhM7R5ba85uN0TPufZ6ZygLJkIVsB63/x6yX00gkzQwgb5C4+UbyRpQGOFzN6FbOeIN4P9Uk4KOH94mxM+doShu3jWpgtqumZble07YNd+/do20aT+nvqeqSa/t7Q9Wlogg0jTPwPnK0bOm6HP0WB/6RUSvbriP1ids3b1NX1ZhEqOcVMTbpPBV55sCvYRyjBD6uqn9PRF4EPga8Dvgs8JdVtb3IzQd0M3oPmbNY1JzJWNXm0LFEHvO8z2abgOcxJubzue2grp5Paw2C+EMwFguwBWpxs1mSaLsGUXHbVRiYsYVsGVMty9Jgaqd9cLU49XGssuK/s8KyjvOsFu1SOMi+KkN26TQmfJWMAZQepZDREQUGR2tTBrTcY5hwNSyLQqN5uvs1sWu4+81v8PD+XYNwbTtm8wU3n3sD+9dvUi320dquMfKMMfljg82JuAj0uCVmv+j7nq5rWR4ecffuXUPeO7xH366Y1TP2FgurhDS/bsktpUIJGkq62ENRul32IitpSuf/4WlMRzHGS7TohiG9ewItLEXciFoaJTl3evp1YoykvkNjDzENiSp930FSgz51abnyAhBlZkmak3Rckkv2gFdelSnt71PXNXVl0pumiPYtqBUqyRWpLJIpR1l5/LrHTqt2KC3lLLC4NqOsSmaLypJ0CtAcpJ2Fq8FClD+nYXzOu9lmJj5K4JyY6+m1NqT34c+rSgkuEBj+d0qJqmvRvoHYUkikDIYSnmLLwaMHvPRbX2K9Wg55J0VZWPEXseeSIBwcHPDg0QNWqxUvf+Nl1qu1JRRFZT6bcfPWDaq64vr+HvP5jKZpODw6out7Hj5Y0jQd66ZlvW5HrdpNvX2MPP/cc8zrBYvFwvuSa5BmLeT8a/g8EngDvENVD8VqY/43EfkV4G8DP6WqHxORnwbeB3z43Hd2mobzZcohdNmWbNKsSSNDySogd7QqyyHWtCcdk8DxM0dpZJTCXRJ242D+J5M2DQtu0tZB2ctqXv7PP2dJzJI17O7BzSsy2DXV83bHfhvuszGDnDCSQZKy+B7OlExd/dLoMKVW+Xq9Wtnm4ep0Wc+o5gvLsszxzKOws9GefNVxpMd30zZMzVx5M+o9Br7tWtq2pW9bcywVBUWRCEVnPitKBMsBUAfsL3LK6cTuehY9LZ/Xs649MCd/8GSUxvGIDwNewx0y6iiaOkjbdt28BkZzlI+cObQ0WnSG+zYs+mgEdxPXAkygzEBT6lK1bDg0y7JEkzhyZ9rI6tt4hoZ5ynWksgReGFKkp3K7L32j7YOkPbGdqI/X42doXB+jhp3nNksEeuolxnBDhqIo03kaOLsnyjBocY4ImRL4pppcuz08PGB5dDTA4palJS0FEUvpF2G5WnJ0dMRqueTg4ID1am1RMRH6rrPqXnVlaIMoTdPQrBu6vjczybpjtW5YLVeDMJcl8Zgiq+WK1XrtTuOxopI7xy60ps9TkUeBQ/9Y+Z8C7wD+kh//KPD3eQoGnvebwr3qZVFQltmGrYYdkv80oH1hGYJDA/FQN0XDoCQiKScFubWvDJTBiyv4As12SBvczHwZVMaYMsRs5Q+Mm2cmrS+lMszh2BNDQDylV1Upq4pZVbs5wIvF4glFYBgjTKqqOOYxQFHPqErTPsTLw81qa0dMc1Zxup7VJPDU0ayOeHT3ZZr1isNHD1mvVuzfuMmtO89RzxfM9m8h9TUoSuIQS3OeqOvHyN6TjS55eGbsLeuzXTc0TUPfNAa36vHny1VnCSv1HqFesGh7FjfumC24KA2DebJ5/oHQGcxHYzIkylyxHSFKZ9oZZnZLMvo+1B3UA9MQU/NTtrMybHuoJtpmafd2j2IRCrp6TlmU9DF5nLchAZp5r3cs+HFjyRjSUgTKukKTPReiasW3K8uIndXm8yirgrKw5KKkJhmGIlFWUFXCbFa4H0hHCdeForyhJc86Hja0LKhkM+IThls25nMqVp9x3gbDhs3oqrwhCIgS6ZEgtKsVUiqrecXRwwcczisOHuxxsKi598orfP2rX+Po6NASe4qS2WzGtX33c/Wm9TbrxoIIup6AUAZLiDLM8sTyaEXZdHa8dxiQzuCA66omhMoirBqrQh/7sYYACH0fuX/vPuvlipu3brC3txic+rPyNS6p5gNaYGaS7wA+BPwO8EBVcwmOrwEvnPHb9wPvB7h58+aZ9wguYRZFrl0JKVr4jUZLRMkMHYqJ2j+G9BXZ1KeMNkffIQqrjzSRoh27JNi1xDEsYszx4SYpZQlmKHk0qDn2GgqrvKOFpUWnlAj+kJVVTTmf+9nZHmYOLWDAPBnGgDHWvKhKiqqyz2UkBGE+n1MUJcumgtUmQw2YDTS2aw4e3KNZr1gdHdK2DUVZcvN1z1PNFlSLfaSaTx4dIUfW+8dTnqmNtI1TaUNL8Ye97yzNu+s6+s7Cp2JvNmUpHGJ01lPUVqi27zrKKlLWT8m0L2hGPFWP0VGS1mRZtialGmJdkOjiQrLYX5gwtZFhZ/VeGVHmsobTdY3he/h9MuBU6WXbsoRtyV9CHx2xcRL9lFz1FsdZVw3uCIWyKj0MsKKqKq+RmUvzKao9qoKE5Fje4qXPzNxoI2NFxK3JaRA+7OEakQEH5n0OSRymJhT/fJGpPmUzz/ZxxaKD+raBVaJZHrFeHrI6WrA6PGR5eI2DR4+4+8pdDg8PWFy7xmw2JyalrGaEoqByE2LX9vStZYdn1EkdMIQMwK7vemZ1LjSdEVGhKiuKQmirzvM2zPGtgwm1IMXE4cEhfdexd20BLCzjtqpMeL1AVZ5zMXA1EeN7ROQW8EvAd533Bqr6EeAjAG9605s2ZiCIV6V3D5+9ZPS0nCHpNmMpPXMts8+pTQ43OEwhGQeZh43TZUwGyOnNimvs4Njebr92U0w9qzfU1UEq8T4EhEIVKc2Cmb3PVVFSV5VPsjkrpAhINBtvxo/IUow44BNg0Kp2ko+L4T6fhdPXdh2r9ZrlasVyuaRrG2bzOVU949r+DerZniPqjZvfsEyOr5cT62f8xXGJOEtBg9NLGWx+w1REJfUJKSCRLGlEFAmm4krqIXVo6py5pEFlTpP7jSaA0awzNYSIq+NPEtqfJNXnCuohZdjfqQnFCjRbVvAk9DLbwAcGLgN2R05ez422JB/HEO97NATaJhCDxVgnxyUpC0PD7OMEqTPnQUS7V4yGlY1CyHVeEU/JL2ycSW6mMT/ObG6MvaoKylKccQ+dd9OR9zGbhtQQBFOMJolPnfc5jPKctCmJn++80YKy+TvxsbZ2+9qLidj1rJcrVkdLlkdLjg6XLA9XLJdrlkdruh7KsmWx6Ii9b26+ca1WRyyPDui6juXh0gURgy2we9rG2TZrZrN6NAOJgJhpcr1aWxFntfkvwpgFK5gg1/WB1WpFUQTW6/VTaZsXikJR1Qci8mngB4FbIlK6FP5m4OsXvXmOAy/DCPozVmYfkdFCKI3JM9rxcjgU2V4JvhFgUonXm4xxEj6o9l1ZmtMikbVE+6EipGrcEMDjwD3cryxLysIYoMHU4oD5xmyvFbYZ5FqeFjnskn3fDQkdySMOLJsTut7xHhSKNKrcOR4keFhixuE+Ps2qymq15uHDRzx68JB79+6hKfH6N7yR6zdusnfjDteu33azSWGSpPf4QhLrmQvMRV9lQIIjKUEtoCT1idj2VgiiNFtvLQb8Reysmnk/Q2OD9hVJ98j+iFExH/0JU51hZI7+hE+jJx7b5rP7mFKPxGBmEr+DBkvtTnRIyJg3U9tutnlbNESWwM0xDDlV287rSbEdgKhA6BurjbleLynL2WZWsV83h7TaPXPsOZBMGKmKmkIKkIKyrCmL0gscJMPwKGE+K9m/sbAU872KemZMfNj8UvSpHM0nye8VPYtQPb59MBv55vVa0hCxwlR8OHPWyPZP7UFb6NYNB/cfUkpgfz5nVlbcu3ufh/cecXBwQNIDEmJp9teX9hy22bnZ2fzESNM0xJjo2o6us5DQtmlR1KJ23HdQVcYj6vkeRVESY6J3EDozwebaurYWuq5BNfLgwX1WK7O1Zyn9InSeKJTngc6Z9wL4s8BPAp8GfhSLRHkv8MsXujOYaSMEq2qdvfqqoxTqUkARRsljeGjywslOJ/9KJFeFtwPii1Hy9QaHkYPxm1ju98xATVkbMBaRwxmLELwWoJLLoQ326xwPLiA6Lr6QH4IQvHQVvlmPD2MQKwKAWm1Q685E8hw7N9qEjpH7cOx6vqFU9YzZbE5V1y7RhxO/lVPePZaeYKbIzt8hNEpPfi+TTyOinIU+jlLsiW3qye282No/uwcpFwrY/AMrahDStC2jIzv7N+wylnVrZpU4CBo5bpxj45Rc+sux4SbMuFBzCgNnkHoD4pVoVAzJcwR8UnBIhIykGQozl5SVl7fLmekTh6X1Z3TkD/bunPg2wFdMolHOYT55WnritE6X2WABsqi0rutompb1ak2zbuk6Q3LsLYoQ0YayWCMozXpJ7K0yvRU8sXBYe7XqPxmgzniObWRlNM3JMFS6wRmfkpp5d/o8+PrKoclN0wwhhmnQZs4/NueRwN8IfFQyeDD8O1X9pIh8AfiYiPxD4DeBnz3/bY0KCczqiiqkITW+CAWV19vLqdzBmWcmC4pfW5iWq7sCvgsWE0hW42qSB9ez3Oq6JoTCYV2BQR0WxBHdplJd8oEPEiwKBHE7VWat5siaYpYDDPHc5ALGYg+2q+BZkjTYUR3sn5MAjI1rmbdaKLocJjB8S1GWVLM5+zdu86a3vIgI3LnzOvb2riH1YmCbr5rHDar25sWyCSp6Way+j3ReyRsCEkoPayvcjqsu1XWkvqOpKlZHhyhCMd+HhWspkxsPczLF0VYuaEh9PGlKxL4xnGrfXEKyWHULNSvJWZPjj0blfjTZmR+gWx/Srg7p2yWkzsruMcYoF1nTcCbZOewBktX60XQg2Y7uQgpiWOQFBSqFVbaXgr4XYrS1F2Pr0vmCxXzGYq9kPyND1kJZWoUYklVaT46kOdi9dTQJRjflqKZRgPLooVyo+DLorDWsmDBUhYqggXbZsipXvPL7d2lXLffu3WW1bGjWPW2T6HtlHToO769QVdr1EbFvbV7EhQgPgoiaiAMj9tWYl2KW/pM6vv0YSooGunasb2AmsY62b8iaGohFbLUthUeenZfOE4XyOeB7Tzn+u8APnPtOp5CIUBaBMgiROCC1hTAqEoLbmWXCwN20YaBDIwNXigwjAZKxPGwSMgPMduyckIPi2Av2w8LVnUlHBzjazETEzTQ6PKujujk0GsCTgnB7d8ZDkaEvYy+zBjLiKjOx+Rrl8Mhjh91UbmGCs/mCKtwhiLC3f4PZfEGSwsDqj911aPuxOTkXnXFaBgmLHg+fko2vaSfHQOtdqkuqXuy3oahqC5vM4y2QISjl+K423PTxbboYqcVIR0jDPI31WF0HPnZjyHa3URL0Gpl9S9+uSX07MO8sHQdGyZuMUEki0ZGl8Ol05Pc23+K4QAGREpFosrYUpFSTUouqh6GmQChmlCXUVWBWl+7ohCIkRCIZqdAk9rTBuMfX6JE1Y4jk1B7+dCQbL8NYnnn2aZOswyWy01EQ+q6nXXccHVhRiYNHh3St1djs2kjXetJVakgp0q6OiLGlDFC6EFgU4mUEh2U4tiFbIEdGYAlfEz6QxCFw8/oRt7R39ux1/abg11fzDb/Pk+hqMzGxDidf0KMlOg+QSwASIIyZkYMDMVhpWUpzClZlaUkNqNXSU69eolZiCbfR6gDnOmhbE3BURd02nZ9X9UVqbR7NBPYhuz4YSsAND7IEc9SRNeixzmfuP2Qv9lTC0gmTt3aJm5uM0Wc9caTC41lJFVp6+8s6y3qDWvwkaTWHBOa+XpQn1lVNWixI7TWu7V+nLQva9SFd25gkk8xRG5I5qHMxAQPaWhOKcqjYbox+ZKJMHhwflmP06jl4ipFudYS2gbJqLBU9FISisrH3bNisDo+kk/99I4uR5vAR7fKQvlmRoif1ZBPR8WtkU4ZLForkfWFYi17YyRgVQhWEKtj6Lr2U2KyKVFWkqqCuhaoKlA4TC0rfm/1V6endWd/5mlds8xyfs8zAzXyUsskkH+uzM3WzIv25Z2NDkxyPqTLuWHkQzly645eGOdKxkjWvfPMuB/MDD6usWK2WPLj/yDTE1nhERikdavOm4FHyhixEKS5oFg50lf1wQrZK5oCH3N4xi9s1KB/HkYGrx4czjC2Mnx+3gR2nq0UjVKtkntTSq0Ou+DIsnNyRaDCWOsVJgbIc61dm6dVSzc1JlJISMwMXNcQ3SWZrDcG/8wSafC9Ng9MweBmgwQs/wfAew+aMWxvwrfqG4A9AKCCUyMTGnqJJp6CTa+gg2IlPtFl0xnR8c2p5Qd0QyUUjjISyqpnN99zGuvCj7qjN3nyPqMiP1onn5pj0fV52OI0WmM1nVFVJoR3trds0q5rDgwfoamnj0keSWIQ1wUuqdR1tWbJeHoJA16yN0Umwyio2SLbFj7i5vsPIRCJ+9ZRix+rwiCQ6VKSXYNgtg6aUA1d1cyPOZBu8mV+OHt5ldfSIdrVEu9b6daI4gTLarDP/nvh2bJCt18FSz4tgaHp1EOaVpcPXtVKExGIWmdU9dSUs5lb+rJ6Z3RtJtO2KEAJddETDQcMdY9oHey2+PodXHSRwkziN8XUdnLZiHhtxMmg0bCxGzZfakCBkPP+0S5HRCRMxWaGWZt0w5JOIJ5gNqet5HgULNRhvnJKSa/+UYrkfZV1R1rYGrFpSXnq5CIRdIGrvvAKSr9WM5pjroGYGDlNwMZvvODD089GVMvCkQps8PVldvSY7HmWTsYkxS3VbeUwFUS10KmlhCHE4A09KlyzmO2pOCighFQTMYRiSkDQnn4gzVaEXczzmSkCWkDEBwk8T5ucTnoWmPOnJHZwpiRVIlaxfQExCTDL5vfcLk7qTL9SgJqVmG0xIpsaJWImqTVL6dk27fIS32Ble/lYGFnEWW7bbyAkm/jjKjuLx/Rje1jdHaOwg9RSCQZOqIInBnGVlwAKoJVBo6kl9S7desj586Dbzcpj7cbyPmZgG1WUyIqMt48wHol0eWI7Bxg9d2RMI0ccyYXkI2XYvWUo6qQkBXq/RHKGFChUFGipm1ZwUIj0FGiOxjKSyd8nLoR5Sbm9mlmM/RbI5ESqP666LwMwjIDKefFXOKcKMEEqQGqUgRrONK5h9X7AqT0Px7byoTmPgeRwnDs7s1HVNt+9lY5wF2FvMuXP75gYD182FP4z5cdKN/ybmUPTUFTxcd1gSYz3a7GTNETW2Z2TmnRk5ZIN2kGR/hVDPLIa+rCuKqnRTyMjA7V6WpGOMufdXRyUU2AAtO7EpZqXL2re/d81Clc9JV8rAm1jwrdV8wzkDxzbk43r8xiLSDaaTr5FNFXaas61OCL0n4qzDxmau0/NyNMyEJwxg9H7fE6QnH2WTF6flzzIDVVSL4f30xe4p4+vYKVejjYl0/SZT0hS5//9e4ujeN+CU6+pkbF9bmlgkB0HZVmSMPX2zJqXIouypr883xi6HiGqamYZUFNAe0cU197+64uDlr/jid4n7seozm2sETgztaRdIsac5erhxrNDAXqxt2UUZ1gLkIhwThq8n3pz4VMk10nxOqiNx8bwLJZNEGN/9M7OeXmUwpU76Nmpk1ueMZz1Nlx8yhoMQY0lKwmEqWB5lP0s/Gdt8D3/qpgz2jJ5tMmFb1323ycBDCHznW1/kDa9/ntPpeH9PfLvxSU4ePPtXWbCQCS8ZNsXp6dnHIOPvFTdRMvgaEAtaCFOujceNT2SIgS8N4zSuzU139LEWT9ZRUZTcun52wuNxulIGHlVYxUtqwgXUkpP0WjK/V3Ot0yU+VaU5eniCGT0rVAqmvp9Kk+OpI6WOdbe+lHadRoJQ6lSlznT62D+JCiooKutm9Ro08IKUJby0ATP3tA7Hs+h008mtmze4dfPGa3yvHU3pGUDQ39GOdrSjHT0N7Rj4jna0ox1tKe0Y+I52tKMdbSnJ0wCoPPXNRL4FHAGvXNpN/2DoOba7D9veftj+Pmx7+2H7+7BN7f9DqnrCI3ypDBxARH5DVb//Um/6GtO292Hb2w/b34dtbz9sfx+2vf2wM6HsaEc72tHW0o6B72hHO9rRltJVMPCPXME9X2va9j5se/th+/uw7e2H7e/Dtrf/8m3gO9rRjna0o9eGdiaUHe1oRzvaUrpUBi4i7xSRL4vISyLyE5d576chEXmLiHxaRL4gIv9HRH7cj98RkV8Vkd/219tX3dbHkYgUIvKbIvJJ//yiiHzG5+Hfikh91W18HInILRH5uIh8SUS+KCI/uIVz8Ld8DX1eRH5BRObP8jyIyL8UkW+KyOcnx04dczH6596Pz4nI911dy0c6ow//yNfR50Tkl8Tq/ObvPuB9+LKI/LmrafXF6NIYuFhFnw8B7wLeBvxFEXnbZd3/KakH/o6qvg14O/DXvc0/AXxKVd8KfMo/P8v048AXJ59/EvgpVf0O4D7wvitp1fnpnwH/SVW/C/hjWF+2Zg5E5AXgbwDfr6rfjSGjvIdnex5+DnjnsWNnjfm7gLf63/uBD19SG59EP8fJPvwq8N2q+keB3wI+AODP9XuAP+K/+RfOs55pukwJ/AeAl1T1d1W1xWppvvsS739hUtVvqOr/9PcHGON4AWv3R/20jwJ/4Wpa+GQSkTcDfx74Gf8swDuAj/spz3r7bwJ/Ei/Zp6qtqj5gi+bAqQQWIlICe8A3eIbnQVV/Dbh37PBZY/5u4F+r0a9jBc/feDktPZtO64Oq/me1QuwAv44VZAfrw8dUtVHVrwAv8Sorjl0GXSYDfwH46uTz1/zYVpCIfDtWWu4zwOtVNWO3vgy8/oqadR76p8DfZYSgex3wYLKIn/V5eBH4FvCv3Az0MyJyjS2aA1X9OvCPgd/DGPdD4LNs1zzA2WO+rc/2XwN+xd9vZR92TsxzkIjsA/8B+Juq+mj6neZ6Sc8gicgPA99U1c9edVteBZXA9wEfVtXvxaAYNswlz/IcALit+N3YZvQm4BonVfutomd9zJ9EIvJBzET681fdlldDl8nAvw68ZfL5zX7smSYRqTDm/fOq+ot++Peziuiv37yq9j2B/jjwIyLyfzGT1Tswe/ItV+Xh2Z+HrwFfU9XP+OePYwx9W+YA4M8AX1HVb6lqB/wiNjfbNA9w9phv1bMtIn8V+GHgx3SMo96qPmS6TAb+P4C3uue9xhwGn7jE+1+Y3F78s8AXVfWfTL76BPBef/9e4Jcvu23nIVX9gKq+WVW/HRvv/6qqPwZ8GvhRP+2ZbT+Aqr4MfFVEvtMP/WngC2zJHDj9HvB2EdnzNZX7sDXz4HTWmH8C+CsejfJ24OHE1PJMkYi8EzMp/oiqLidffQJ4j4jMRORFzCH736+ijRcinZR2+oP+A34I8/z+DvDBy7z3U7b3T2Bq4ueA/+V/P4TZkT8F/DbwX4A7V93Wc/TlTwGf9Pd/GFucLwH/Hphddfue0PbvAX7D5+E/Are3bQ6AfwB8Cfg88G+A2bM8D8AvYPb6DtOC3nfWmGMleT7kz/X/xqJtntU+vITZuvPz/NOT8z/offgy8K6rbv95/naZmDva0Y52tKW0c2LuaEc72tGW0o6B72hHO9rRltKOge9oRzva0ZbSjoHvaEc72tGW0o6B72hHO9rRltKOge9oRzva0ZbSjoHvaEc72tGW0o6B72hHO9rRltL/B4wvpq9htSpHAAAAAElFTkSuQmCC\n" 129 | }, 130 | "metadata": { 131 | "needs_background": "light" 132 | } 133 | }, 134 | { 135 | "output_type": "stream", 136 | "name": "stdout", 137 | "text": "deer bird dog truck\n" 138 | } 139 | ], 140 | "source": [ 141 | "# Let us show some of the training images, for fun.\n", 142 | "\n", 143 | "import matplotlib.pyplot as plt\n", 144 | "import numpy as np\n", 145 | "\n", 146 | "# functions to show an image\n", 147 | "def imshow(img):\n", 148 | " img = img / 2 + 0.5 # unnormalize\n", 149 | " npimg = img.numpy()\n", 150 | " plt.imshow(np.transpose(npimg, (1, 2, 0)))\n", 151 | " plt.show()\n", 152 | "\n", 153 | "\n", 154 | "# get some random training images\n", 155 | "dataiter = iter(trainloader)\n", 156 | "images, labels = dataiter.next()\n", 157 | "\n", 158 | "# show images\n", 159 | "imshow(torchvision.utils.make_grid(images))\n", 160 | "# print labels\n", 161 | "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "### 2. Define a Convolutional Neural Network\n", 169 | "Copy the neural network from the Neural Networks section before and modify it to take 3-channel images (instead of 1-channel images as it was defined)." 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": 4, 175 | "metadata": {}, 176 | "outputs": [], 177 | "source": [ 178 | "import torch.nn as nn\n", 179 | "import torch.nn.functional as F\n", 180 | "\n", 181 | "\n", 182 | "class Net(nn.Module):\n", 183 | " def __init__(self):\n", 184 | " super(Net, self).__init__()\n", 185 | " self.conv1 = nn.Conv2d(3, 6, 5)\n", 186 | " self.pool = nn.MaxPool2d(2, 2)\n", 187 | " self.conv2 = nn.Conv2d(6, 16, 5)\n", 188 | " self.fc1 = nn.Linear(16 * 5 * 5, 120)\n", 189 | " self.fc2 = nn.Linear(120, 84)\n", 190 | " self.fc3 = nn.Linear(84, 10)\n", 191 | "\n", 192 | " def forward(self, x):\n", 193 | " x = self.pool(F.relu(self.conv1(x)))\n", 194 | " x = self.pool(F.relu(self.conv2(x)))\n", 195 | " x = x.view(-1, 16 * 5 * 5)\n", 196 | " x = F.relu(self.fc1(x))\n", 197 | " x = F.relu(self.fc2(x))\n", 198 | " x = self.fc3(x)\n", 199 | " return x\n", 200 | "\n", 201 | "\n", 202 | "net = Net()" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 5, 208 | "metadata": { 209 | "tags": [] 210 | }, 211 | "outputs": [ 212 | { 213 | "output_type": "stream", 214 | "name": "stdout", 215 | "text": "cuda:0\n" 216 | } 217 | ], 218 | "source": [ 219 | "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n", 220 | "\n", 221 | "# Assuming that we are on a CUDA machine, this should print a CUDA device:\n", 222 | "\n", 223 | "print(device)" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 6, 229 | "metadata": {}, 230 | "outputs": [ 231 | { 232 | "output_type": "execute_result", 233 | "data": { 234 | "text/plain": "Net(\n (conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))\n (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n (fc1): Linear(in_features=400, out_features=120, bias=True)\n (fc2): Linear(in_features=120, out_features=84, bias=True)\n (fc3): Linear(in_features=84, out_features=10, bias=True)\n)" 235 | }, 236 | "metadata": {}, 237 | "execution_count": 6 238 | } 239 | ], 240 | "source": [ 241 | "net.to(device)" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "### 3. Define a Loss function and optimizer\n", 249 | "Let’s use a Classification Cross-Entropy loss and SGD with momentum.\n", 250 | "\n" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": 7, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [ 259 | "import torch.optim as optim\n", 260 | "\n", 261 | "criterion = nn.CrossEntropyLoss()\n", 262 | "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)\n" 263 | ] 264 | }, 265 | { 266 | "cell_type": "markdown", 267 | "metadata": {}, 268 | "source": [ 269 | "### 4. Train the network\n", 270 | "This is when things start to get interesting. We simply have to loop over our data iterator, and feed the inputs to the network and optimize." 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": 8, 276 | "metadata": { 277 | "tags": [] 278 | }, 279 | "outputs": [ 280 | { 281 | "output_type": "stream", 282 | "name": "stdout", 283 | "text": "[1, 2000] loss: 2.235\n[1, 4000] loss: 1.872\n[1, 6000] loss: 1.688\n[1, 8000] loss: 1.586\n[1, 10000] loss: 1.520\n[1, 12000] loss: 1.466\n[2, 2000] loss: 1.410\n[2, 4000] loss: 1.397\n[2, 6000] loss: 1.336\n[2, 8000] loss: 1.351\n[2, 10000] loss: 1.304\n[2, 12000] loss: 1.270\n[3, 2000] loss: 1.221\n[3, 4000] loss: 1.219\n[3, 6000] loss: 1.217\n[3, 8000] loss: 1.210\n[3, 10000] loss: 1.183\n[3, 12000] loss: 1.187\n[4, 2000] loss: 1.119\n[4, 4000] loss: 1.118\n[4, 6000] loss: 1.133\n[4, 8000] loss: 1.098\n[4, 10000] loss: 1.112\n[4, 12000] loss: 1.088\nFinished Training\n" 284 | } 285 | ], 286 | "source": [ 287 | "for epoch in range(4): # loop over the dataset multiple times\n", 288 | "\n", 289 | " running_loss = 0.0\n", 290 | " for i, data in enumerate(trainloader, 0):\n", 291 | " # get the inputs; data is a list of [inputs, labels]\n", 292 | " inputs, labels = data[0].to(device), data[1].to(device)\n", 293 | "\n", 294 | " # zero the parameter gradients\n", 295 | " optimizer.zero_grad()\n", 296 | "\n", 297 | " # forward + backward + optimize\n", 298 | " outputs = net(inputs)\n", 299 | " loss = criterion(outputs, labels)\n", 300 | " loss.backward()\n", 301 | " optimizer.step()\n", 302 | "\n", 303 | " # print statistics\n", 304 | " running_loss += loss.item()\n", 305 | " if i % 2000 == 1999: # print every 2000 mini-batches\n", 306 | " print('[%d, %5d] loss: %.3f' %\n", 307 | " (epoch + 1, i + 1, running_loss / 2000))\n", 308 | " running_loss = 0.0\n", 309 | "\n", 310 | "print('Finished Training')" 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": null, 316 | "metadata": {}, 317 | "outputs": [], 318 | "source": [] 319 | } 320 | ] 321 | } -------------------------------------------------------------------------------- /getting_started.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "language_info": { 4 | "codemirror_mode": { 5 | "name": "ipython", 6 | "version": 3 7 | }, 8 | "file_extension": ".py", 9 | "mimetype": "text/x-python", 10 | "name": "python", 11 | "nbconvert_exporter": "python", 12 | "pygments_lexer": "ipython3", 13 | "version": "3.8.3-final" 14 | }, 15 | "orig_nbformat": 2, 16 | "kernelspec": { 17 | "name": "python3", 18 | "display_name": "Python 3" 19 | } 20 | }, 21 | "nbformat": 4, 22 | "nbformat_minor": 2, 23 | "cells": [ 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Getting Started" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "## Tensors" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 1, 41 | "metadata": {}, 42 | "outputs": [ 43 | { 44 | "output_type": "execute_result", 45 | "data": { 46 | "text/plain": "tensor([[1.6114e-19, 1.6020e-19, 4.4721e+21],\n [7.3840e+22, 1.3513e-14, 1.4917e-07],\n [4.4859e+21, 7.2150e+22, 2.4501e-09],\n [7.5553e+28, 5.2839e-11, 1.8888e+31],\n [4.7414e+16, 3.1434e-12, 8.0775e+17]])" 47 | }, 48 | "metadata": {}, 49 | "execution_count": 1 50 | } 51 | ], 52 | "source": [ 53 | "import torch\n", 54 | "\n", 55 | "# An uninitialized matrix is declared, but does not contain definite known values before it is used. When an uninitialized matrix is created, whatever values were in the allocated memory at the time will appear as the initial values.\n", 56 | "x = torch.empty(5, 3)\n", 57 | "x" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": 2, 63 | "metadata": {}, 64 | "outputs": [ 65 | { 66 | "output_type": "execute_result", 67 | "data": { 68 | "text/plain": "tensor([[0.1768, 0.0252, 0.3825],\n [0.0727, 0.9963, 0.3651],\n [0.8616, 0.5529, 0.1209],\n [0.1791, 0.3994, 0.0585],\n [0.0548, 0.9263, 0.6761]])" 69 | }, 70 | "metadata": {}, 71 | "execution_count": 2 72 | } 73 | ], 74 | "source": [ 75 | "# Construct a randomly initialized matrix:\n", 76 | "x = torch.rand(5, 3)\n", 77 | "x" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 3, 83 | "metadata": {}, 84 | "outputs": [ 85 | { 86 | "output_type": "execute_result", 87 | "data": { 88 | "text/plain": "tensor([[0, 0, 0],\n [0, 0, 0],\n [0, 0, 0],\n [0, 0, 0],\n [0, 0, 0]])" 89 | }, 90 | "metadata": {}, 91 | "execution_count": 3 92 | } 93 | ], 94 | "source": [ 95 | "# Construct a matrix filled zeros and of dtype long:\n", 96 | "x = torch.zeros(5, 3, dtype=torch.long)\n", 97 | "x" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": 4, 103 | "metadata": { 104 | "tags": [] 105 | }, 106 | "outputs": [ 107 | { 108 | "output_type": "stream", 109 | "name": "stdout", 110 | "text": "tensor([5.5000, 3.0000])\n" 111 | }, 112 | { 113 | "output_type": "execute_result", 114 | "data": { 115 | "text/plain": "torch.Size([2])" 116 | }, 117 | "metadata": {}, 118 | "execution_count": 4 119 | } 120 | ], 121 | "source": [ 122 | "# Construct a tensor from data\n", 123 | "x = torch.tensor([5.5, 3])\n", 124 | "print(x)\n", 125 | "x.shape" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": 5, 131 | "metadata": { 132 | "tags": [] 133 | }, 134 | "outputs": [ 135 | { 136 | "output_type": "stream", 137 | "name": "stdout", 138 | "text": "tensor([[1., 1., 1.],\n [1., 1., 1.],\n [1., 1., 1.],\n [1., 1., 1.],\n [1., 1., 1.]], dtype=torch.float64)\ntensor([[-0.0870, 0.8627, -0.5135],\n [-0.7507, 0.4283, 0.5405],\n [ 0.6390, 0.2455, -0.1478],\n [-3.0340, 0.9263, 0.2301],\n [ 1.4263, -0.7427, -0.2583]])\n" 139 | } 140 | ], 141 | "source": [ 142 | "# or create a tensor based on an existing tensor. These methods will reuse properties of the input tensor, e.g. dtype, unless new values are provided by user\n", 143 | "x = x.new_ones(5, 3, dtype=torch.double) # new_* methods take in sizes\n", 144 | "print(x)\n", 145 | "\n", 146 | "x = torch.randn_like(x, dtype=torch.float) # override dtype!\n", 147 | "print(x) # result has the same size" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 6, 153 | "metadata": {}, 154 | "outputs": [ 155 | { 156 | "output_type": "execute_result", 157 | "data": { 158 | "text/plain": "torch.Size([5, 3])" 159 | }, 160 | "metadata": {}, 161 | "execution_count": 6 162 | } 163 | ], 164 | "source": [ 165 | "x.size()\n", 166 | "# torch.Size is in fact a tuple, so it supports all tuple operations." 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "## Operations" 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": 7, 179 | "metadata": { 180 | "tags": [] 181 | }, 182 | "outputs": [ 183 | { 184 | "output_type": "stream", 185 | "name": "stdout", 186 | "text": "tensor([[ 0.0859, 1.4640, 0.0112],\n [-0.0722, 1.0417, 1.2508],\n [ 1.3985, 0.8384, -0.0922],\n [-2.4628, 1.6542, 1.1757],\n [ 2.0619, -0.1439, -0.0910]])\ntensor([[ 0.0859, 1.4640, 0.0112],\n [-0.0722, 1.0417, 1.2508],\n [ 1.3985, 0.8384, -0.0922],\n [-2.4628, 1.6542, 1.1757],\n [ 2.0619, -0.1439, -0.0910]])\ntensor([[ 0.0859, 1.4640, 0.0112],\n [-0.0722, 1.0417, 1.2508],\n [ 1.3985, 0.8384, -0.0922],\n [-2.4628, 1.6542, 1.1757],\n [ 2.0619, -0.1439, -0.0910]])\ntensor([[ 0.0859, 1.4640, 0.0112],\n [-0.0722, 1.0417, 1.2508],\n [ 1.3985, 0.8384, -0.0922],\n [-2.4628, 1.6542, 1.1757],\n [ 2.0619, -0.1439, -0.0910]])\n" 187 | } 188 | ], 189 | "source": [ 190 | "# Addition\n", 191 | "y = torch.rand(5, 3)\n", 192 | "print(x + y)\n", 193 | "print(torch.add(x, y))\n", 194 | "\n", 195 | "result = torch.empty(5, 3)\n", 196 | "torch.add(x, y, out=result)\n", 197 | "print(result)\n", 198 | "\n", 199 | "# Addition: in-place\n", 200 | "# adds x to y\n", 201 | "y.add_(x)\n", 202 | "print(y)\n", 203 | "# Any operation that mutates a tensor in-place is post-fixed with an _. For example: x.copy_(y), x.t_(), will change x." 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": 8, 209 | "metadata": { 210 | "tags": [] 211 | }, 212 | "outputs": [ 213 | { 214 | "output_type": "stream", 215 | "name": "stdout", 216 | "text": "torch.Size([4, 4]) torch.Size([16]) torch.Size([2, 8])\n" 217 | } 218 | ], 219 | "source": [ 220 | "# Resizing: If you want to resize/reshape tensor, you can use torch.view:\n", 221 | "x = torch.randn(4, 4)\n", 222 | "y = x.view(16)\n", 223 | "z = x.view(-1, 8) # the size -1 is inferred from other dimensions\n", 224 | "print(x.size(), y.size(), z.size())\n" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": 9, 230 | "metadata": { 231 | "tags": [] 232 | }, 233 | "outputs": [ 234 | { 235 | "output_type": "stream", 236 | "name": "stdout", 237 | "text": "tensor([0.0110])\n0.010992790572345257\n" 238 | } 239 | ], 240 | "source": [ 241 | "# If you have a one element tensor, use .item() to get the value as a Python number\n", 242 | "\n", 243 | "x = torch.randn(1)\n", 244 | "print(x)\n", 245 | "print(x.item())" 246 | ] 247 | }, 248 | { 249 | "cell_type": "markdown", 250 | "metadata": {}, 251 | "source": [ 252 | "## Converting a Torch Tensor to a NumPy Array\n", 253 | "Converting a Torch Tensor to a NumPy array and vice versa is a breeze.\n", 254 | "The Torch Tensor and NumPy array will share their underlying memory locations (if the Torch Tensor is on CPU), and changing one will change the other." 255 | ] 256 | }, 257 | { 258 | "cell_type": "code", 259 | "execution_count": 10, 260 | "metadata": { 261 | "tags": [] 262 | }, 263 | "outputs": [ 264 | { 265 | "output_type": "stream", 266 | "name": "stdout", 267 | "text": "tensor([1., 1., 1., 1., 1.])\n[1. 1. 1. 1. 1.]\n" 268 | } 269 | ], 270 | "source": [ 271 | "a = torch.ones(5)\n", 272 | "print(a)\n", 273 | "b = a.numpy()\n", 274 | "print(b)" 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": 11, 280 | "metadata": { 281 | "tags": [] 282 | }, 283 | "outputs": [ 284 | { 285 | "output_type": "stream", 286 | "name": "stdout", 287 | "text": "tensor([2., 2., 2., 2., 2.])\n[2. 2. 2. 2. 2.]\n" 288 | } 289 | ], 290 | "source": [ 291 | "# See how the numpy array changed in value.\n", 292 | "a.add_(1)\n", 293 | "print(a)\n", 294 | "print(b)" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "## Converting NumPy Array to Torch Tensor\n", 302 | "See how changing the np array changed the Torch Tensor automatically\n", 303 | "\n" 304 | ] 305 | }, 306 | { 307 | "cell_type": "code", 308 | "execution_count": 12, 309 | "metadata": { 310 | "tags": [] 311 | }, 312 | "outputs": [ 313 | { 314 | "output_type": "stream", 315 | "name": "stdout", 316 | "text": "[2. 2. 2. 2. 2.]\ntensor([2., 2., 2., 2., 2.], dtype=torch.float64)\n" 317 | } 318 | ], 319 | "source": [ 320 | "import numpy as np\n", 321 | "a = np.ones(5)\n", 322 | "b = torch.from_numpy(a)\n", 323 | "np.add(a, 1, out=a)\n", 324 | "print(a)\n", 325 | "print(b)\n", 326 | "# All the Tensors on the CPU except a CharTensor support converting to NumPy and back." 327 | ] 328 | }, 329 | { 330 | "cell_type": "markdown", 331 | "metadata": {}, 332 | "source": [ 333 | "## CUDA Tensors\n", 334 | "Tensors can be moved onto any device using the .to method." 335 | ] 336 | }, 337 | { 338 | "cell_type": "code", 339 | "execution_count": 13, 340 | "metadata": { 341 | "tags": [] 342 | }, 343 | "outputs": [ 344 | { 345 | "output_type": "stream", 346 | "name": "stdout", 347 | "text": "tensor([1.0110], device='cuda:0')\ntensor([1.0110], dtype=torch.float64)\n" 348 | } 349 | ], 350 | "source": [ 351 | "# let us run this cell only if CUDA is available\n", 352 | "# We will use ``torch.device`` objects to move tensors in and out of GPU\n", 353 | "if torch.cuda.is_available():\n", 354 | " device = torch.device(\"cuda\") # a CUDA device object\n", 355 | " y = torch.ones_like(x, device=device) # directly create a tensor on GPU\n", 356 | " x = x.to(device) # or just use strings ``.to(\"cuda\")``\n", 357 | " z = x + y\n", 358 | " print(z)\n", 359 | " print(z.to(\"cpu\", torch.double)) # ``.to`` can also change dtype together!" 360 | ] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": null, 365 | "metadata": {}, 366 | "outputs": [], 367 | "source": [] 368 | } 369 | ] 370 | } -------------------------------------------------------------------------------- /neural_networks.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "language_info": { 4 | "codemirror_mode": { 5 | "name": "ipython", 6 | "version": 3 7 | }, 8 | "file_extension": ".py", 9 | "mimetype": "text/x-python", 10 | "name": "python", 11 | "nbconvert_exporter": "python", 12 | "pygments_lexer": "ipython3", 13 | "version": "3.8.3-final" 14 | }, 15 | "orig_nbformat": 2, 16 | "kernelspec": { 17 | "name": "python3", 18 | "display_name": "Python 3" 19 | } 20 | }, 21 | "nbformat": 4, 22 | "nbformat_minor": 2, 23 | "cells": [ 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Neural Networks\n", 29 | "Neural networks can be constructed using the torch.nn package.\n", 30 | "\n", 31 | "Now that you had a glimpse of autograd, nn depends on autograd to define models and differentiate them. An nn.Module contains layers, and a method forward(input)that returns the output.\n" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "A typical training procedure for a neural network is as follows:\n", 39 | "\n", 40 | "* Define the neural network that has some learnable parameters (or weights)\n", 41 | "* Iterate over a dataset of inputs\n", 42 | "* Process input through the network\n", 43 | "* Compute the loss (how far is the output from being correct)\n", 44 | "* Propagate gradients back into the network’s parameters\n", 45 | "* Update the weights of the network, typically using a simple update rule: weight = weight - learning_rate * gradient" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "## Define the network\n" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 1, 58 | "metadata": { 59 | "tags": [] 60 | }, 61 | "outputs": [ 62 | { 63 | "output_type": "stream", 64 | "name": "stdout", 65 | "text": "Net(\n (conv1): Conv2d(1, 6, kernel_size=(3, 3), stride=(1, 1))\n (conv2): Conv2d(6, 16, kernel_size=(3, 3), stride=(1, 1))\n (fc1): Linear(in_features=576, out_features=120, bias=True)\n (fc2): Linear(in_features=120, out_features=84, bias=True)\n (fc3): Linear(in_features=84, out_features=10, bias=True)\n)\n" 66 | } 67 | ], 68 | "source": [ 69 | "import torch\n", 70 | "import torch.nn as nn\n", 71 | "import torch.nn.functional as F\n", 72 | "\n", 73 | "\n", 74 | "class Net(nn.Module):\n", 75 | "\n", 76 | " def __init__(self):\n", 77 | " # This is calling the __init__() method of the parent class of MyComplexModel, in this case, nn.Module\n", 78 | " super(Net, self).__init__()\n", 79 | "\n", 80 | " # 1 input image channel, 6 output channels, 3x3 square convolution kernel\n", 81 | " self.conv1 = nn.Conv2d(1, 6, 3)\n", 82 | " self.conv2 = nn.Conv2d(6, 16, 3)\n", 83 | "\n", 84 | " # an affine operation: y = Wx + b\n", 85 | " self.fc1 = nn.Linear(16 * 6 * 6, 120) # 6*6 from image dimension\n", 86 | " self.fc2 = nn.Linear(120, 84)\n", 87 | " self.fc3 = nn.Linear(84, 10)\n", 88 | "\n", 89 | " def forward(self, x):\n", 90 | " # Max pooling over a (2, 2) window\n", 91 | " x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))\n", 92 | "\n", 93 | " # If the size is a square you can only specify a single number\n", 94 | " x = F.max_pool2d(F.relu(self.conv2(x)), 2)\n", 95 | " \n", 96 | " x = x.view(-1, self.num_flat_features(x))\n", 97 | " x = F.relu(self.fc1(x))\n", 98 | " x = F.relu(self.fc2(x))\n", 99 | " x = self.fc3(x)\n", 100 | " return x\n", 101 | "\n", 102 | " def num_flat_features(self, x):\n", 103 | " size = x.size()[1:] # all dimensions except the batch dimension\n", 104 | " num_features = 1\n", 105 | " for s in size:\n", 106 | " num_features *= s\n", 107 | " return num_features\n", 108 | "\n", 109 | "\n", 110 | "net = Net()\n", 111 | "print(net)" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": 2, 117 | "metadata": { 118 | "tags": [] 119 | }, 120 | "outputs": [ 121 | { 122 | "output_type": "stream", 123 | "name": "stdout", 124 | "text": "10\ntorch.Size([6, 1, 3, 3])\n" 125 | } 126 | ], 127 | "source": [ 128 | "# You just have to define the forward function, and the backward function (where gradients are computed) is automatically defined for you using autograd. You can use any of the Tensor operations in the forward function.\n", 129 | "\n", 130 | "# The learnable parameters of a model are returned by net.parameters()\n", 131 | "\n", 132 | "params = list(net.parameters())\n", 133 | "print(len(params))\n", 134 | "print(params[0].size()) # conv1's .weight" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 3, 140 | "metadata": { 141 | "tags": [] 142 | }, 143 | "outputs": [ 144 | { 145 | "output_type": "stream", 146 | "name": "stdout", 147 | "text": "tensor([[-0.0974, 0.0793, -0.1379, -0.0572, 0.0125, -0.0123, 0.0292, -0.1446,\n -0.0622, -0.1426]], grad_fn=)\n" 148 | } 149 | ], 150 | "source": [ 151 | "# Let’s try a random 32x32 input. \n", 152 | "input = torch.randn(1, 1, 32, 32)\n", 153 | "out = net(input)\n", 154 | "print(out)" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 5, 160 | "metadata": { 161 | "tags": [] 162 | }, 163 | "outputs": [], 164 | "source": [ 165 | "net.zero_grad()\n", 166 | "out.backward(torch.randn(1, 10))" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "### NOTE\n", 174 | "\n", 175 | "torch.nn only supports mini-batches. The entire torch.nn package only supports inputs that are a mini-batch of samples, and not a single sample.\n", 176 | "\n", 177 | "For example, nn.Conv2d will take in a 4D Tensor of nSamples x nChannels x Height x Width.\n", 178 | "\n", 179 | "If you have a single sample, just use input.unsqueeze(0) to add a fake batch dimension." 180 | ] 181 | }, 182 | { 183 | "cell_type": "markdown", 184 | "metadata": {}, 185 | "source": [ 186 | "## Loss Function\n", 187 | "A loss function takes the (output, target) pair of inputs, and computes a value that estimates how far away the output is from the target.\n", 188 | "\n", 189 | "There are several different loss functions under the nn package . A simple loss is: nn.MSELoss which computes the mean-squared error between the input and the target." 190 | ] 191 | }, 192 | { 193 | "cell_type": "code", 194 | "execution_count": 6, 195 | "metadata": { 196 | "tags": [] 197 | }, 198 | "outputs": [ 199 | { 200 | "output_type": "stream", 201 | "name": "stdout", 202 | "text": "tensor(0.7646, grad_fn=)\n" 203 | } 204 | ], 205 | "source": [ 206 | "output = net(input)\n", 207 | "target = torch.randn(10) # a dummy target, for example\n", 208 | "target = target.view(1, -1) # make it the same shape as output\n", 209 | "criterion = nn.MSELoss()\n", 210 | "\n", 211 | "loss = criterion(output, target)\n", 212 | "print(loss)" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 7, 218 | "metadata": { 219 | "tags": [] 220 | }, 221 | "outputs": [ 222 | { 223 | "output_type": "stream", 224 | "name": "stdout", 225 | "text": "\n\n\n" 226 | } 227 | ], 228 | "source": [ 229 | "# So, when we call loss.backward(), the whole graph is differentiated w.r.t. the loss, and all Tensors in the graph that has requires_grad=True will have their .grad Tensor accumulated with the gradient.\n", 230 | "\n", 231 | "# For illustration, let us follow a few steps backward:\n", 232 | "\n", 233 | "print(loss.grad_fn) # MSELoss\n", 234 | "print(loss.grad_fn.next_functions[0][0]) # Linear\n", 235 | "print(loss.grad_fn.next_functions[0][0].next_functions[0][0]) # ReLU" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "## Backprop\n", 243 | "To backpropagate the error all we have to do is to loss.backward(). You need to clear the existing gradients though, else gradients will be accumulated to existing gradients." 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": 8, 249 | "metadata": { 250 | "tags": [] 251 | }, 252 | "outputs": [ 253 | { 254 | "output_type": "stream", 255 | "name": "stdout", 256 | "text": "conv1.bias.grad before backward\ntensor([0., 0., 0., 0., 0., 0.])\nconv1.bias.grad after backward\ntensor([ 0.0125, 0.0034, -0.0051, -0.0055, -0.0009, -0.0055])\n" 257 | } 258 | ], 259 | "source": [ 260 | "# Now we shall call loss.backward(), and have a look at conv1’s bias gradients before and after the backward.\n", 261 | "net.zero_grad() # zeroes the gradient buffers of all parameters\n", 262 | "\n", 263 | "print('conv1.bias.grad before backward')\n", 264 | "print(net.conv1.bias.grad)\n", 265 | "\n", 266 | "loss.backward()\n", 267 | "\n", 268 | "print('conv1.bias.grad after backward')\n", 269 | "print(net.conv1.bias.grad)\n" 270 | ] 271 | }, 272 | { 273 | "cell_type": "markdown", 274 | "metadata": {}, 275 | "source": [ 276 | "## Update the weights\n" 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 10, 282 | "metadata": {}, 283 | "outputs": [], 284 | "source": [ 285 | "# The simplest update rule used in practice is the Stochastic Gradient Descent (SGD):\n", 286 | "# weight = weight - learning_rate * gradient\n", 287 | "learning_rate = 0.01\n", 288 | "for f in net.parameters():\n", 289 | " f.data.sub_(f.grad.data * learning_rate)" 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": 11, 295 | "metadata": {}, 296 | "outputs": [], 297 | "source": [ 298 | "# However, as you use neural networks, you want to use various different update rules such as SGD, Nesterov-SGD, Adam, RMSProp, etc. To enable this, we built a small package: torch.optim that implements all these methods. Using it is very simple:\n", 299 | "\n", 300 | "import torch.optim as optim\n", 301 | "\n", 302 | "# create your optimizer\n", 303 | "optimizer = optim.SGD(net.parameters(), lr=0.01)\n", 304 | "\n", 305 | "# in your training loop:\n", 306 | "optimizer.zero_grad() # zero the gradient buffers\n", 307 | "output = net(input)\n", 308 | "loss = criterion(output, target)\n", 309 | "loss.backward()\n", 310 | "optimizer.step() # Does the update" 311 | ] 312 | }, 313 | { 314 | "cell_type": "markdown", 315 | "metadata": {}, 316 | "source": [ 317 | "### NOTE\n", 318 | "\n", 319 | "Observe how gradient buffers had to be manually set to zero using optimizer.zero_grad(). This is because gradients are accumulated as explained in the Backprop section." 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": null, 325 | "metadata": {}, 326 | "outputs": [], 327 | "source": [] 328 | } 329 | ] 330 | } --------------------------------------------------------------------------------