├── Tools
    └── FMNIST_00.jpeg
├── LICENSE
├── README.md
├── TensorFlow2.0__02.01_Autoencoder_for_Dimensionality_Reduction.ipynb
├── TensorFlow2.0__00.03_Save_and_Restore_models.ipynb
└── TensorFlow2.0__03.01_Convolutional_Neural_Network.ipynb


/Tools/FMNIST_00.jpeg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/IvanBongiorni/TensorFlow2.0_Notebooks/HEAD/Tools/FMNIST_00.jpeg


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2019 Ivan Bongiorni
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | Author: Ivan Bongiorni, Data Scientist at GfK; [LinkedIn](https://www.linkedin.com/in/ivan-bongiorni-b8a583164/).
 2 | 
 3 | # TensorFlow 2.0 Notebooks
 4 | 
 5 | 
 6 | This is a collection of my Notebooks on TensorFlow 2.0
 7 | 
 8 | The training of models is based on TensorFlow's **eager execution** method. I'll try to minimize referencese to Keras.
 9 | ## Summary of Contents:
10 | - Basic feed forward stuff
11 | - Autoencoders
12 | - Convolutional Neural Networks
13 | - Recurrent Neural Networks
14 | - Applications to NLP
15 | 
16 | ---
17 | ---
18 | 
19 | ## Contents:
20 | 
21 | **Basic feed forward stuff**:
22 | 
23 | 1. [Basic classifier](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__00.01_basic_Classifier.ipynb):  implementation of a **feed forward Classifier** with simple, full-Batch Gradient Descent in **Eager execution**.
24 | 
25 | 2. [Mini batch gradient descent](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__00.02_MiniBatch_Gradient_Descent.ipynb):  training a model with **Mini Batch Gradient Descent**.
26 | 
27 | 3. [Save and restore models](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__00.03_Save_and_Restore_models.ipynb):  how to train a model, save it, then restore it and keep training.
28 | 
29 | 0. Train a Neural Network with frozen layers
30 | 
31 | ---
32 | 
33 | **Autoencoders**:
34 | 
35 | 1. [Autoencoder for dimensionality reduction](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__02.01_Autoencoder_for_Dimensionality_Reduction.ipynb):  implementation of a stacked **Autoencoder for dimensionality reduction** of datasets.
36 | 
37 | 2. Denoising Autoencoder (see CNN section below).
38 | 
39 | 0. Recurrent Autoencoder (see RNN section below).
40 | 
41 | ---
42 | 
43 | **Convolutional Neural Networks**:
44 | 
45 | 1. [Basic CNN classifier](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__03.01_Convolutional_Neural_Network.ipynb): a basic **Convolutional Neural Network** for multiclass classification.
46 | 
47 | 2. Advanced CNN classifier with custom data augmentation.
48 | 
49 | 3. Mixed-CNN classifier.
50 | 
51 | 4. Denoising Autoencoder.
52 | 
53 | ---
54 | 
55 | **Recurrent Neural Networks**:
56 | 
57 | 1. [LSTM many-to-one forecast model](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__04.01_RNN_many2one.ipynb)
58 | 
59 | 2. [LSTM many-to-many forecast model](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2.0__04.02_RNN_many2many.ipynb)
60 | 
61 | 3. [Multivariate LSTM regression](https://github.com/IvanBongiorni/TensorFlow2.0_Notebooks/blob/master/TensorFlow2__04.03_RNN_multivariate_regression.ipynb).
62 | 
63 | 0. Seq2seq models.
64 | 
65 | ---
66 | 
67 | **RNN + Natural Language Processing**
68 | 
69 | 1. LSTM [Text generator](https://github.com/IvanBongiorni/TensorFlow2-RNN_text_generator-Dante_DivineComedy/blob/master/RNN_text_generator_00.ipynb) from [this repository of mine](https://github.com/IvanBongiorni/TensorFlow2-RNN_text_generator-Dante_DivineComedy).
70 | 
71 | 


--------------------------------------------------------------------------------
/TensorFlow2.0__02.01_Autoencoder_for_Dimensionality_Reduction.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {
  6 |     "colab_type": "text",
  7 |     "id": "GTY-Msjal7zU"
  8 |    },
  9 |    "source": [
 10 |     "# Autoencoder for Dimensionality Reduction in TensorFlow 2.0\n",
 11 |     "\n",
 12 |     "### Author: Ivan Bongiorni, Data Scientist at GfK.\n",
 13 |     "\n",
 14 |     "[LinkedIn profile](https://www.linkedin.com/in/ivan-bongiorni-b8a583164/).\n",
 15 |     "\n",
 16 |     "\n"
 17 |    ]
 18 |   },
 19 |   {
 20 |    "cell_type": "markdown",
 21 |    "metadata": {
 22 |     "colab_type": "text",
 23 |     "id": "Z3M5ezRyrjA3"
 24 |    },
 25 |    "source": [
 26 |     "Summary:\n",
 27 |     "\n",
 28 |     "\n",
 29 |     "1.   What are Autoencoders?\n",
 30 |     "2.   Why would you need an Autoencoder?\n",
 31 |     "3.   Implementation and Training\n",
 32 |     "4.   Extraction of the encoded dataframe\n",
 33 |     "5.   Practical application"
 34 |    ]
 35 |   },
 36 |   {
 37 |    "cell_type": "markdown",
 38 |    "metadata": {
 39 |     "colab_type": "text",
 40 |     "id": "Kg83oWxMm92f"
 41 |    },
 42 |    "source": [
 43 |     "## 1. What are Autoencoders?\n",
 44 |     "\n",
 45 |     "This tutorial is about a very specific form of Neural Networks: **Autoencoders**.  \n",
 46 |     "\n",
 47 |     "There are several kinds of Autoencoders around, such as denoising Autoencoders or generative models (variational Autoencoders and GANs); they are all meant to accomplish very specific and different tasks. In this tutorial, I'll focus on their **dimensionality reduction** capabilities. Autoencoders for dimensionality reduction can be conceived as *the Neural Network-equivalent of Principal Component Analysis*.\n",
 48 |     "\n",
 49 |     "The purpose of this class fo models is neither to learn how to classify observations, nor to predict a continuous output, but to compress, or simplify a dataset. For this reason, they are **unsupervised models**, differently from mainstream Deep Learning models. All the data that pass through an Autoencoder are unlabeled.\n",
 50 |     "\n",
 51 |     "\n"
 52 |    ]
 53 |   },
 54 |   {
 55 |    "cell_type": "markdown",
 56 |    "metadata": {
 57 |     "colab_type": "text",
 58 |     "id": "IpcaQoQ9srG6"
 59 |    },
 60 |    "source": [
 61 |     "### Structure\n",
 62 |     "\n",
 63 |     "If I had to explain what an Autoencoder is to my grand mother, I'd do it like this: it can be seen as a couple of funnels joined by the tip: in order to make data must flow from one extreme to the other, you must force them through the bottleneck in the middle.\n",
 64 |     "\n"
 65 |    ]
 66 |   },
 67 |   {
 68 |    "cell_type": "markdown",
 69 |    "metadata": {
 70 |     "colab_type": "text",
 71 |     "id": "AvZTU-HzmZEo"
 72 |    },
 73 |    "source": [
 74 |     "More formally, it can be described as follows:\n",
 75 |     "\n",
 76 |     "![alt text](https://cdn-images-1.medium.com/max/800/1*44eDEuZBEsmG_TCAKRI3Kw@2x.png)\n",
 77 |     "\n",
 78 |     "(image from [towardsdatascience](https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798))"
 79 |    ]
 80 |   },
 81 |   {
 82 |    "cell_type": "markdown",
 83 |    "metadata": {
 84 |     "colab_type": "text",
 85 |     "id": "cAxMjUCBUa43"
 86 |    },
 87 |    "source": [
 88 |     "Each Autoencoder is composed of two parts, an **Encoder** and a **Decoder**. The goal of the Encoder is to \"compress\" the dataset, representing its features with a number of nodes/variables that is narrower than the output layer; the goal of the Decoder is to learn how to reproduce, from the central encoded layer, the initial input values. Autoencoder are thus meant to reproduce the input data on the other extreme as accurately as possible.\n",
 89 |     "\n",
 90 |     "At this point, you have the right to be puzzled. The first time I've read about this architecture, I was like \"Wait a minute, what's the point of all that?!\". And in fact, it's completely pointless. But reproducing the same dataset on the other side of the Network is a trick that allows for a dimensionality reduction.\n",
 91 |     "\n",
 92 |     "The fact that the central hidden layers is narrower than the extremes forces the network to find the most efficient way to represent, at that layer, the same data in a more compressed form. The data that flow through that central layer can be seen as a form of dimensionality reduction."
 93 |    ]
 94 |   },
 95 |   {
 96 |    "cell_type": "markdown",
 97 |    "metadata": {
 98 |     "colab_type": "text",
 99 |     "id": "KMWIJUSerRVD"
100 |    },
101 |    "source": [
102 |     "## 2. Why would you need an Autoencoder?\n",
103 |     "\n",
104 |     "Why would you need dimensionality reduction?\n",
105 |     "\n",
106 |     "As you have probably learned from PCA, dimensionality reduction techniques are particularly useful when dealing with very complex and/or multicollinear datasets. Thanks to the \"simplification\" they perform, it is also possible to train Machine Learnign models on datasets otherwise too big for your run-down old laptop (joking).\n",
107 |     "\n",
108 |     "### Ok, but why not good ol' PCA?\n",
109 |     "\n",
110 |     "The advantages of this kind of Neural Network architecture over the more classical PCA is that with an Autoencoder you can potentially capture any non-linear pattern of your data. From PCA instead, you can extract factors that are only linearly associated with your actual variables, making the process of dimensionality reduction much more \"rigid\"."
111 |    ]
112 |   },
113 |   {
114 |    "cell_type": "markdown",
115 |    "metadata": {
116 |     "colab_type": "text",
117 |     "id": "eqK0oiaTsAQn"
118 |    },
119 |    "source": [
120 |     "## 3. Implementation and Training\n",
121 |     "\n"
122 |    ]
123 |   },
124 |   {
125 |    "cell_type": "markdown",
126 |    "metadata": {
127 |     "colab_type": "text",
128 |     "id": "-3rPWJBemIEb"
129 |    },
130 |    "source": [
131 |     "### Import data from UCI ML repository\n",
132 |     "\n",
133 |     "The purpose of this dataset is to reduce the dimensionality of the University of Wisconsin's breast cancer dataset."
134 |    ]
135 |   },
136 |   {
137 |    "cell_type": "code",
138 |    "execution_count": 1,
139 |    "metadata": {
140 |     "colab": {},
141 |     "colab_type": "code",
142 |     "id": "kOKrNBGSlz2t"
143 |    },
144 |    "outputs": [
145 |     {
146 |      "name": "stdout",
147 |      "output_type": "stream",
148 |      "text": [
149 |       "2.0.0-alpha0\n"
150 |      ]
151 |     }
152 |    ],
153 |    "source": [
154 |     "import numpy as np\n",
155 |     "import pandas as pd\n",
156 |     "\n",
157 |     "import tensorflow as tf\n",
158 |     "print(tf.__version__)\n",
159 |     "\n",
160 |     "import matplotlib.pyplot as plt"
161 |    ]
162 |   },
163 |   {
164 |    "cell_type": "code",
165 |    "execution_count": 2,
166 |    "metadata": {},
167 |    "outputs": [
168 |     {
169 |      "data": {
170 |       "text/html": [
171 |        "<div>\n",
172 |        "<style scoped>\n",
173 |        "    .dataframe tbody tr th:only-of-type {\n",
174 |        "        vertical-align: middle;\n",
175 |        "    }\n",
176 |        "\n",
177 |        "    .dataframe tbody tr th {\n",
178 |        "        vertical-align: top;\n",
179 |        "    }\n",
180 |        "\n",
181 |        "    .dataframe thead th {\n",
182 |        "        text-align: right;\n",
183 |        "    }\n",
184 |        "</style>\n",
185 |        "<table border=\"1\" class=\"dataframe\">\n",
186 |        "  <thead>\n",
187 |        "    <tr style=\"text-align: right;\">\n",
188 |        "      <th></th>\n",
189 |        "      <th>0</th>\n",
190 |        "      <th>1</th>\n",
191 |        "      <th>2</th>\n",
192 |        "      <th>3</th>\n",
193 |        "      <th>4</th>\n",
194 |        "      <th>5</th>\n",
195 |        "      <th>6</th>\n",
196 |        "      <th>7</th>\n",
197 |        "      <th>8</th>\n",
198 |        "      <th>9</th>\n",
199 |        "      <th>...</th>\n",
200 |        "      <th>22</th>\n",
201 |        "      <th>23</th>\n",
202 |        "      <th>24</th>\n",
203 |        "      <th>25</th>\n",
204 |        "      <th>26</th>\n",
205 |        "      <th>27</th>\n",
206 |        "      <th>28</th>\n",
207 |        "      <th>29</th>\n",
208 |        "      <th>30</th>\n",
209 |        "      <th>31</th>\n",
210 |        "    </tr>\n",
211 |        "  </thead>\n",
212 |        "  <tbody>\n",
213 |        "    <tr>\n",
214 |        "      <th>0</th>\n",
215 |        "      <td>842302</td>\n",
216 |        "      <td>M</td>\n",
217 |        "      <td>17.99</td>\n",
218 |        "      <td>10.38</td>\n",
219 |        "      <td>122.80</td>\n",
220 |        "      <td>1001.0</td>\n",
221 |        "      <td>0.11840</td>\n",
222 |        "      <td>0.27760</td>\n",
223 |        "      <td>0.3001</td>\n",
224 |        "      <td>0.14710</td>\n",
225 |        "      <td>...</td>\n",
226 |        "      <td>25.38</td>\n",
227 |        "      <td>17.33</td>\n",
228 |        "      <td>184.60</td>\n",
229 |        "      <td>2019.0</td>\n",
230 |        "      <td>0.1622</td>\n",
231 |        "      <td>0.6656</td>\n",
232 |        "      <td>0.7119</td>\n",
233 |        "      <td>0.2654</td>\n",
234 |        "      <td>0.4601</td>\n",
235 |        "      <td>0.11890</td>\n",
236 |        "    </tr>\n",
237 |        "    <tr>\n",
238 |        "      <th>1</th>\n",
239 |        "      <td>842517</td>\n",
240 |        "      <td>M</td>\n",
241 |        "      <td>20.57</td>\n",
242 |        "      <td>17.77</td>\n",
243 |        "      <td>132.90</td>\n",
244 |        "      <td>1326.0</td>\n",
245 |        "      <td>0.08474</td>\n",
246 |        "      <td>0.07864</td>\n",
247 |        "      <td>0.0869</td>\n",
248 |        "      <td>0.07017</td>\n",
249 |        "      <td>...</td>\n",
250 |        "      <td>24.99</td>\n",
251 |        "      <td>23.41</td>\n",
252 |        "      <td>158.80</td>\n",
253 |        "      <td>1956.0</td>\n",
254 |        "      <td>0.1238</td>\n",
255 |        "      <td>0.1866</td>\n",
256 |        "      <td>0.2416</td>\n",
257 |        "      <td>0.1860</td>\n",
258 |        "      <td>0.2750</td>\n",
259 |        "      <td>0.08902</td>\n",
260 |        "    </tr>\n",
261 |        "    <tr>\n",
262 |        "      <th>2</th>\n",
263 |        "      <td>84300903</td>\n",
264 |        "      <td>M</td>\n",
265 |        "      <td>19.69</td>\n",
266 |        "      <td>21.25</td>\n",
267 |        "      <td>130.00</td>\n",
268 |        "      <td>1203.0</td>\n",
269 |        "      <td>0.10960</td>\n",
270 |        "      <td>0.15990</td>\n",
271 |        "      <td>0.1974</td>\n",
272 |        "      <td>0.12790</td>\n",
273 |        "      <td>...</td>\n",
274 |        "      <td>23.57</td>\n",
275 |        "      <td>25.53</td>\n",
276 |        "      <td>152.50</td>\n",
277 |        "      <td>1709.0</td>\n",
278 |        "      <td>0.1444</td>\n",
279 |        "      <td>0.4245</td>\n",
280 |        "      <td>0.4504</td>\n",
281 |        "      <td>0.2430</td>\n",
282 |        "      <td>0.3613</td>\n",
283 |        "      <td>0.08758</td>\n",
284 |        "    </tr>\n",
285 |        "    <tr>\n",
286 |        "      <th>3</th>\n",
287 |        "      <td>84348301</td>\n",
288 |        "      <td>M</td>\n",
289 |        "      <td>11.42</td>\n",
290 |        "      <td>20.38</td>\n",
291 |        "      <td>77.58</td>\n",
292 |        "      <td>386.1</td>\n",
293 |        "      <td>0.14250</td>\n",
294 |        "      <td>0.28390</td>\n",
295 |        "      <td>0.2414</td>\n",
296 |        "      <td>0.10520</td>\n",
297 |        "      <td>...</td>\n",
298 |        "      <td>14.91</td>\n",
299 |        "      <td>26.50</td>\n",
300 |        "      <td>98.87</td>\n",
301 |        "      <td>567.7</td>\n",
302 |        "      <td>0.2098</td>\n",
303 |        "      <td>0.8663</td>\n",
304 |        "      <td>0.6869</td>\n",
305 |        "      <td>0.2575</td>\n",
306 |        "      <td>0.6638</td>\n",
307 |        "      <td>0.17300</td>\n",
308 |        "    </tr>\n",
309 |        "    <tr>\n",
310 |        "      <th>4</th>\n",
311 |        "      <td>84358402</td>\n",
312 |        "      <td>M</td>\n",
313 |        "      <td>20.29</td>\n",
314 |        "      <td>14.34</td>\n",
315 |        "      <td>135.10</td>\n",
316 |        "      <td>1297.0</td>\n",
317 |        "      <td>0.10030</td>\n",
318 |        "      <td>0.13280</td>\n",
319 |        "      <td>0.1980</td>\n",
320 |        "      <td>0.10430</td>\n",
321 |        "      <td>...</td>\n",
322 |        "      <td>22.54</td>\n",
323 |        "      <td>16.67</td>\n",
324 |        "      <td>152.20</td>\n",
325 |        "      <td>1575.0</td>\n",
326 |        "      <td>0.1374</td>\n",
327 |        "      <td>0.2050</td>\n",
328 |        "      <td>0.4000</td>\n",
329 |        "      <td>0.1625</td>\n",
330 |        "      <td>0.2364</td>\n",
331 |        "      <td>0.07678</td>\n",
332 |        "    </tr>\n",
333 |        "  </tbody>\n",
334 |        "</table>\n",
335 |        "<p>5 rows × 32 columns</p>\n",
336 |        "</div>"
337 |       ],
338 |       "text/plain": [
339 |        "         0  1      2      3       4       5        6        7       8   \\\n",
340 |        "0    842302  M  17.99  10.38  122.80  1001.0  0.11840  0.27760  0.3001   \n",
341 |        "1    842517  M  20.57  17.77  132.90  1326.0  0.08474  0.07864  0.0869   \n",
342 |        "2  84300903  M  19.69  21.25  130.00  1203.0  0.10960  0.15990  0.1974   \n",
343 |        "3  84348301  M  11.42  20.38   77.58   386.1  0.14250  0.28390  0.2414   \n",
344 |        "4  84358402  M  20.29  14.34  135.10  1297.0  0.10030  0.13280  0.1980   \n",
345 |        "\n",
346 |        "        9   ...     22     23      24      25      26      27      28      29  \\\n",
347 |        "0  0.14710  ...  25.38  17.33  184.60  2019.0  0.1622  0.6656  0.7119  0.2654   \n",
348 |        "1  0.07017  ...  24.99  23.41  158.80  1956.0  0.1238  0.1866  0.2416  0.1860   \n",
349 |        "2  0.12790  ...  23.57  25.53  152.50  1709.0  0.1444  0.4245  0.4504  0.2430   \n",
350 |        "3  0.10520  ...  14.91  26.50   98.87   567.7  0.2098  0.8663  0.6869  0.2575   \n",
351 |        "4  0.10430  ...  22.54  16.67  152.20  1575.0  0.1374  0.2050  0.4000  0.1625   \n",
352 |        "\n",
353 |        "       30       31  \n",
354 |        "0  0.4601  0.11890  \n",
355 |        "1  0.2750  0.08902  \n",
356 |        "2  0.3613  0.08758  \n",
357 |        "3  0.6638  0.17300  \n",
358 |        "4  0.2364  0.07678  \n",
359 |        "\n",
360 |        "[5 rows x 32 columns]"
361 |       ]
362 |      },
363 |      "execution_count": 2,
364 |      "metadata": {},
365 |      "output_type": "execute_result"
366 |     }
367 |    ],
368 |    "source": [
369 |     "df = pd.read_csv(\"https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data\", sep=\",\", header=None)\n",
370 |     "\n",
371 |     "df.head()"
372 |    ]
373 |   },
374 |   {
375 |    "cell_type": "markdown",
376 |    "metadata": {
377 |     "colab_type": "text",
378 |     "id": "HTXgNVxD2VYp"
379 |    },
380 |    "source": [
381 |     "### Dataprep\n",
382 |     "\n",
383 |     "Data preprocessing for Autoencoder require less effort than canonical Neural Networks. First, there is no need to split the data in train and test set, since my goal is just to produce a compressed version of the same dataset. Second, data do not require any labeling; we are in the field of unsupervised Machine Learning."
384 |    ]
385 |   },
386 |   {
387 |    "cell_type": "code",
388 |    "execution_count": 3,
389 |    "metadata": {
390 |     "colab": {},
391 |     "colab_type": "code",
392 |     "id": "R1NPnVJRbUbe"
393 |    },
394 |    "outputs": [],
395 |    "source": [
396 |     "# get one-hot encoded target variable:\n",
397 |     "target = pd.get_dummies(df.iloc[:,1])\n",
398 |     "target = target.values.astype(np.float32) #convert to float32 for later\n",
399 |     "\n",
400 |     "# I don't need the first two columns:\n",
401 |     "# the first is an id, the other is the target variable ('M'/'B')\n",
402 |     "df = df.iloc[:,2:]\n",
403 |     "df = df.values   # Turn to numpy object"
404 |    ]
405 |   },
406 |   {
407 |    "cell_type": "code",
408 |    "execution_count": 4,
409 |    "metadata": {
410 |     "colab": {},
411 |     "colab_type": "code",
412 |     "id": "8MZswZsz2POl"
413 |    },
414 |    "outputs": [],
415 |    "source": [
416 |     "# Normalize the data\n",
417 |     "# no need to use sklearn StandardScaler() this time, but you can if you want\n",
418 |     "def normalize(x):\n",
419 |     "    normalized = (x - np.mean(x))/(np.std(x))\n",
420 |     "    return normalized\n",
421 |     "\n",
422 |     "for i in range(df.shape[1]):\n",
423 |     "    df[:,i] = normalize(df[:,i])"
424 |    ]
425 |   },
426 |   {
427 |    "cell_type": "markdown",
428 |    "metadata": {
429 |     "colab_type": "text",
430 |     "id": "Ibv0OMUj7sdH"
431 |    },
432 |    "source": [
433 |     "### Architecture\n",
434 |     "\n",
435 |     "Given the shape of my training dataframe:"
436 |    ]
437 |   },
438 |   {
439 |    "cell_type": "code",
440 |    "execution_count": 5,
441 |    "metadata": {
442 |     "colab": {
443 |      "base_uri": "https://localhost:8080/",
444 |      "height": 35
445 |     },
446 |     "colab_type": "code",
447 |     "executionInfo": {
448 |      "elapsed": 2590,
449 |      "status": "ok",
450 |      "timestamp": 1550692670707,
451 |      "user": {
452 |       "displayName": "Ivan Bongiorni",
453 |       "photoUrl": "https://lh6.googleusercontent.com/-saQtoUWf1x0/AAAAAAAAAAI/AAAAAAAADl0/slMwgVYQe0w/s64/photo.jpg",
454 |       "userId": "13961768404337863095"
455 |      },
456 |      "user_tz": -60
457 |     },
458 |     "id": "Ac14zUWt-tGG",
459 |     "outputId": "008f1378-74d7-40c0-b6a5-dc6342fcd757"
460 |    },
461 |    "outputs": [
462 |     {
463 |      "data": {
464 |       "text/plain": [
465 |        "(569, 30)"
466 |       ]
467 |      },
468 |      "execution_count": 5,
469 |      "metadata": {},
470 |      "output_type": "execute_result"
471 |     }
472 |    ],
473 |    "source": [
474 |     "df.shape"
475 |    ]
476 |   },
477 |   {
478 |    "cell_type": "markdown",
479 |    "metadata": {
480 |     "colab_type": "text",
481 |     "id": "J-cA8x_D-6bo"
482 |    },
483 |    "source": [
484 |     "I must now choose an Autoencoder architecture. I want to compress a `(569, 30)` shaped dataset to a `(569, 10)` encoded version. This is an arbitrary choice. In real datascience problems the chioce depends from the nature of your specific dataset (and your specific needs), there is no general rule to determine the right number of encoded variables to be obtained.\n"
485 |    ]
486 |   },
487 |   {
488 |    "cell_type": "code",
489 |    "execution_count": 6,
490 |    "metadata": {
491 |     "colab": {},
492 |     "colab_type": "code",
493 |     "id": "qus4X_4XH0V8"
494 |    },
495 |    "outputs": [],
496 |    "source": [
497 |     "# I choose the size of each layer\n",
498 |     "\n",
499 |     "n_input_layer = df.shape[1]\n",
500 |     "n_hidden1 = 30\n",
501 |     "n_hidden2 = 15\n",
502 |     "n_hidden3 = 15\n",
503 |     "\n",
504 |     "n_encoding_layer = 10\n",
505 |     "\n",
506 |     "n_hidden5 = 15\n",
507 |     "n_hidden6 = 15\n",
508 |     "n_hidden7 = 30\n",
509 |     "n_output_layer = n_input_layer  # size of output layer = size of input layer, of course"
510 |    ]
511 |   },
512 |   {
513 |    "cell_type": "markdown",
514 |    "metadata": {
515 |     "colab_type": "text",
516 |     "id": "ITiY0308HcuD"
517 |    },
518 |    "source": [
519 |     "As you can see, the architecture of the Network si perfectly symmetric. Of course, Encoder and Decoder don't need to be the exact mirror image of each other."
520 |    ]
521 |   },
522 |   {
523 |    "cell_type": "code",
524 |    "execution_count": 7,
525 |    "metadata": {
526 |     "colab": {},
527 |     "colab_type": "code",
528 |     "id": "kcOyYg3Q8OZY"
529 |    },
530 |    "outputs": [],
531 |    "source": [
532 |     "from tensorflow.keras.layers import Dense\n",
533 |     "from tensorflow.keras.activations import elu\n",
534 |     "\n",
535 |     "Autoencoder = tf.keras.models.Sequential([\n",
536 |     "    # ENCODER\n",
537 |     "    Dense(n_input_layer, input_shape = (n_input_layer,), activation = elu),   # Input layer    \n",
538 |     "    Dense(n_hidden1, activation = elu), # hidden layer 1    \n",
539 |     "    Dense(n_hidden2, activation = elu), # hidden layer 2    \n",
540 |     "    Dense(n_hidden3, activation = elu), # hidden layer 3\n",
541 |     "    \n",
542 |     "    # CENTRAL LAYER\n",
543 |     "    Dense(n_encoding_layer, activation = elu, name = 'central_layer'), \n",
544 |     "    \n",
545 |     "    # DECODER\n",
546 |     "    Dense(n_hidden5, activation = elu), # hidden layer 5\n",
547 |     "    Dense(n_hidden6, activation = elu), # hidden layer 6\n",
548 |     "    Dense(n_hidden7, activation = elu), # hidden layer 7\n",
549 |     "    Dense(n_output_layer, activation = elu)  # Output layer\n",
550 |     "])"
551 |    ]
552 |   },
553 |   {
554 |    "cell_type": "markdown",
555 |    "metadata": {
556 |     "colab_type": "text",
557 |     "id": "HrgXlzQMWzek"
558 |    },
559 |    "source": [
560 |     "Now that the Network's structure is ready, let me define other hyperparameters:"
561 |    ]
562 |   },
563 |   {
564 |    "cell_type": "code",
565 |    "execution_count": 8,
566 |    "metadata": {
567 |     "colab": {},
568 |     "colab_type": "code",
569 |     "id": "X6iFujQiXZwR"
570 |    },
571 |    "outputs": [],
572 |    "source": [
573 |     "# number of epochs\n",
574 |     "n_epochs = 5000\n",
575 |     "\n",
576 |     "# Loss: mean squared error\n",
577 |     "loss = tf.keras.losses.MeanSquaredError()\n",
578 |     "\n",
579 |     "# Adam Optimizer\n",
580 |     "optimizer = tf.optimizers.Adam(learning_rate = 0.0005)"
581 |    ]
582 |   },
583 |   {
584 |    "cell_type": "markdown",
585 |    "metadata": {
586 |     "colab_type": "text",
587 |     "id": "1KiGUWCol-uG"
588 |    },
589 |    "source": [
590 |     "### Training"
591 |    ]
592 |   },
593 |   {
594 |    "cell_type": "code",
595 |    "execution_count": 9,
596 |    "metadata": {
597 |     "colab": {
598 |      "base_uri": "https://localhost:8080/",
599 |      "height": 968
600 |     },
601 |     "colab_type": "code",
602 |     "executionInfo": {
603 |      "elapsed": 32395,
604 |      "status": "ok",
605 |      "timestamp": 1550692700669,
606 |      "user": {
607 |       "displayName": "Ivan Bongiorni",
608 |       "photoUrl": "https://lh6.googleusercontent.com/-saQtoUWf1x0/AAAAAAAAAAI/AAAAAAAADl0/slMwgVYQe0w/s64/photo.jpg",
609 |       "userId": "13961768404337863095"
610 |      },
611 |      "user_tz": -60
612 |     },
613 |     "id": "Oevz4mHWmCKs",
614 |     "outputId": "0e0cedfe-d45f-4a1d-cc35-78086e7efe47"
615 |    },
616 |    "outputs": [
617 |     {
618 |      "name": "stdout",
619 |      "output_type": "stream",
620 |      "text": [
621 |       "200.\tLoss: 0.2983798086643219\n",
622 |       "400.\tLoss: 0.16014118492603302\n",
623 |       "600.\tLoss: 0.11733168363571167\n",
624 |       "800.\tLoss: 0.09321168065071106\n",
625 |       "1000.\tLoss: 0.07829537242650986\n",
626 |       "1200.\tLoss: 0.07143492251634598\n",
627 |       "1400.\tLoss: 0.06630369275808334\n",
628 |       "1600.\tLoss: 0.06210746988654137\n",
629 |       "1800.\tLoss: 0.059432562440633774\n",
630 |       "2000.\tLoss: 0.057390931993722916\n",
631 |       "2200.\tLoss: 0.05561588332056999\n",
632 |       "2400.\tLoss: 0.05405626446008682\n",
633 |       "2600.\tLoss: 0.052680548280477524\n",
634 |       "2800.\tLoss: 0.05113258957862854\n",
635 |       "3000.\tLoss: 0.04885231703519821\n",
636 |       "3200.\tLoss: 0.04681725054979324\n",
637 |       "3400.\tLoss: 0.045288633555173874\n",
638 |       "3600.\tLoss: 0.04415624216198921\n",
639 |       "3800.\tLoss: 0.04331396520137787\n",
640 |       "4000.\tLoss: 0.042595453560352325\n",
641 |       "4200.\tLoss: 0.0420219786465168\n",
642 |       "4400.\tLoss: 0.04152001813054085\n",
643 |       "4600.\tLoss: 0.04105399176478386\n",
644 |       "4800.\tLoss: 0.04111744463443756\n",
645 |       "5000.\tLoss: 0.04028546065092087\n",
646 |       "\n",
647 |       "Encoding complete\n"
648 |      ]
649 |     }
650 |    ],
651 |    "source": [
652 |     "\n",
653 |     "loss_history = []  # save loss improvement\n",
654 |     "\n",
655 |     "for epoch in range(n_epochs):\n",
656 |     "    \n",
657 |     "    with tf.GradientTape() as tape:\n",
658 |     "        current_loss = loss(Autoencoder(df), df)\n",
659 |     "    \n",
660 |     "    gradients = tape.gradient(current_loss, Autoencoder.trainable_variables)    # get the gradient of the loss function\n",
661 |     "    optimizer.apply_gradients(zip(gradients, Autoencoder.trainable_variables))  # update the weights\n",
662 |     "    \n",
663 |     "    loss_history.append(current_loss.numpy())  # save current loss in its history\n",
664 |     "    \n",
665 |     "    # show loss improvement every 200 epochs\n",
666 |     "    if (epoch+1) % 200 == 0:\n",
667 |     "        print(str(epoch+1) + '.\\tLoss: ' + str(current_loss.numpy()))\n",
668 |     "#\n",
669 |     "print('\\nEncoding complete')"
670 |    ]
671 |   },
672 |   {
673 |    "cell_type": "markdown",
674 |    "metadata": {
675 |     "colab_type": "text",
676 |     "id": "8xMVYavgX3x6"
677 |    },
678 |    "source": [
679 |     "The computation of `current_loss` deserves attention.\n",
680 |     "\n",
681 |     "As explained above, an Autoencoder doesn't require a dependent variable, i.e. the loss function will not contain any \"test data\". We only need to train the Autoencoder to replicate the input data at the output layer (like comparing the initial dataset with a copy of itself), that's why we are minimizing the distance between the Network's output `Autoencoder(df)` and the actual `df`."
682 |    ]
683 |   },
684 |   {
685 |    "cell_type": "markdown",
686 |    "metadata": {},
687 |    "source": [
688 |     "### Check compression improvements visually"
689 |    ]
690 |   },
691 |   {
692 |    "cell_type": "code",
693 |    "execution_count": 10,
694 |    "metadata": {
695 |     "colab": {
696 |      "base_uri": "https://localhost:8080/",
697 |      "height": 362
698 |     },
699 |     "colab_type": "code",
700 |     "executionInfo": {
701 |      "elapsed": 32653,
702 |      "status": "ok",
703 |      "timestamp": 1550692700950,
704 |      "user": {
705 |       "displayName": "Ivan Bongiorni",
706 |       "photoUrl": "https://lh6.googleusercontent.com/-saQtoUWf1x0/AAAAAAAAAAI/AAAAAAAADl0/slMwgVYQe0w/s64/photo.jpg",
707 |       "userId": "13961768404337863095"
708 |      },
709 |      "user_tz": -60
710 |     },
711 |     "id": "p2f1s2U2mpqK",
712 |     "outputId": "6c996b63-ca02-47f7-a2f5-32d034773300"
713 |    },
714 |    "outputs": [
715 |     {
716 |      "data": {
717 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcZHV97//Xu6q7epuefYZtBmaAQUEiiwOK20WMgkbx3pgbQY1oUO81ouYmmmiMxBB/98YlUYl4DcEFRUE0auZnUEBZXFkG2fcBBmZYZmH26Zleqj/3j/OtnpqeXmqGOVXdXe/n41GPOud7lvqc6ur61Pf7Ped7FBGYmZkBFBodgJmZTRxOCmZmNsRJwczMhjgpmJnZECcFMzMb4qRgZmZDnBRswpAUko5sdBx7S9LfSLpkf6+7D3Fsk3R4Hvu25iFfpzCxSboBOA44MCJ692K7AJZExIq8YtvfnkvMkk4FPhkRp+7ldjcAl0VELl/UNnH5bz8y1xQmMEmLgFcAAZzZ0GAmMEktk3Hfk5Xfk6nNSWFiewdwE/AN4JzqBZJukPTuqvl3SvpVmv5FKr4zNSm8JZW/R9IKSRskLZN0cNX2z5d0bVr2oKQ/rlr2DUkXSfpPSVsl3SzpiKrlL6jado2kv0nlbZK+IOmp9PiCpLaq7T4i6em07E+HHV+bpM9JeiLt8yuSOtKyUyWtlvTXkp4Bvj5sW0n6vKS1krZIulvSscPfXEn/H1nS/VJ6n76UykPS+yU9DDycyr4oaVXa322SXlG1n09KuixNL0rbn5NiXy/p4/u4boekSyVtlHS/pL+StHr4cVStP9T8lv5mX5b0k3Rsv5Z0YPobbJT0gKQTqrZdKeljku5Ly78uqX2s93u0z5Ok/yvpc8Ni+w9Jf5GmD5b075LWSXpM0geHvT/fk3RZ+qzdLemoFNva9Dd4bdX6MyR9NX2OnpT0KUnFtOydkn6VPkcb02u9bqy/vQER4ccEfQArgD8DXgT0AwdULbsBeHfV/DuBX1XNB3Bk1fxpwHrgRKAN+BfgF2lZF7AKeBfQApyQ1j0mLf8G8Cxwclr+beCKtKwbeBr4S6A9zb84LbuALKnNB+YBvwH+IS07A1gDHJte/zvVMQOfB5YBs9M+/3/g/6RlpwIDwKfTsXQMe99OB24DZgICjgYOGuU93u19rHrvrk2v3ZHK3g7MScf/l8AzQHta9kmyZgiARWn7fwM6yJr+eoGj92HdfwRuBGYBC4C7gNVjfF6q379vpL/hi9Lf5TrgMbIfGkXgU8D1VduuBO4BFqbj/jXwqdHeb8b+PL2S7PNUaZ6eBewADib7IXobcD5QAg4HHgVOr3p/dqa/YQvwzRT3x4FW4D3AY1Vx/xD4V7LP0HzgFuB/VP1P9KdtisD7gKeq4trjb+9HOClM1Afw8vSBnpvmHwD+V9Xy3T7QjJ8Uvgp8pmp+Wtr/IuAtwC+Hvf6/An+Xpr8BXFK17PXAA2n6bOD2UY7hEeD1VfOnAyvT9NeAf6xadlQlZrIv8u3AEVXLT6l8GaQvqT7Sl/IIr3sa8BDwEqAwzvu8xxdDiuO0cbbbCByXpj/Jnl/0C6rWvQU4ax/WHfqyTPPvZu+Swr9VLfsAcH/V/O8Bm6rmVwL/c9jf+JHR3u9xPk8CngBemZa9B7guTb8YeGJY3B8Dvl71/lxbteyNwDagmOa703HOBA4gS6IdVeufTUp2ZP8TK6qWdaZtDxztb+9H4LbBiesc4JqIWJ/mv5PKPr+P+zsY+F1lJiK2SXoWOAQ4DHixpE1V67cA36qaf6ZquofsSwCyX5aPjPGaj1fNP57KKstuG7asYh7ZP/BtkiplIvu1V7EuInaO9KIRcV1qDrgIOEzSD4APR8SWUeIcyarqGUkfBs5NcQcwHZg7xvajvV97s+7Bw+LYLaYarKma3jHC/PCYqvdf/beCPd/vUT9PEbFS0hVkX9C/AN4KXJZWPQw4eNhnrQj8coy410dEuWqeFPvBZLWHp6s+J4VhxzH03kZET1pvrL9F03NSmIBS2/kfA8XUhgtZFX2mpOMi4k6yX9KdVZsdOM5unyL7h6y8RhdZc8iTZP9EN0bEa/Yh3FXAWeO85r1p/tBUBlmT08KqdQ+tml5P9s//goh4cpR9j3naXERcCFwoaT5wJfAR4BN7sZ+h8tR/8FfAq4F7I2JQ0kayRJWnp8maje5L8wvHWHd/GP73eKpqfvj7NNbnCeBy4BpJ/0hWO/hvqXwVWY1vyX6IdxVZTWFuRAzsw/Y+9XIE7miemP4rUAaOAY5Pj6PJfk29I61zB/CHkjpT5+K5w/axhqy9tuJy4F2SjlfW2fu/gZsjYiXwY+AoSX8iqTU9TpJ0dA2x/hg4SNKfK+sc7pb04qrX/FtJ8yTNJWtHrvxivBJ4p6RjJHUCf1fZYUQMkrWzfz59qSPpEEmn1xAPKfYXS2olS547gcFRVh/+Po2km6xNfR3QIul8sppC3q4EPiZplqRDgPNyfr33S1ogaTZZG/53x1h3rM8TEXE7WXK/BLg6Iio1g1uAranTukNSUdKxkk7a22Aj4mngGuCfJE2XVJB0hKT/UuMuavnbNx0nhYnpHLI21ici4pnKA/gS8DZlpwR+nqyddw1wKVnnb7VPApdK2iTpjyPiZ2S/lP+d7BfoEaRf+BGxFXhtmn+KrMpd6VQcU9r2NWRtv8+Qna3zqrT4U8Bysg7Su8maGz6VtvsJ8AWyDtAV6bnaX6fymyRtAX4GPG+8eJLpZEllI1kzyLPAZ0dZ94vAH6WzUy4cZZ2rgZ+S9VM8TpZk9rYpZ19cAKwm62j9GfB9sl/GefkO2Zfso2RNgp8abcWxPk/D9vf76bmyXRl4A9kPncfYlThm7GPM7yDrsL6P7O/9feCgGret5W/fdHzxmtkkIel9ZJ3Qtf4S3pt9ryTrdP3Z/t63TS6uKZhNUJIOkvSy1CzyPLJTYX/Y6LhsanNHs9nEVSI7NXgxsAm4AvhyQyOyKc/NR2ZmNsTNR2ZmNmTSNR/NnTs3Fi1a1OgwzMwmldtuu219RMwbb71JlxQWLVrE8uXLGx2GmdmkIunx8ddy85GZmVVxUjAzsyFOCmZmNiS3pCDpa+mmGPeMslySLkw36bhL0ol5xWJmZrXJs6bwDbIbqYzmdcCS9Hgv8H9zjMXMzGqQW1KIiF8AG8ZY5U3ANyNzE9mw0LUOZGVmZjloZJ/CIew+0uTqVLYHSe+VtFzS8nXr1tUlODOzZjQpOpoj4uKIWBoRS+fNG/faixHdunIDn736AcqDHtbDzGw0jUwKT7L7nZ4WsOuuTfvdHU9s4qLrH6Gnb19u0GRm1hwamRSWAe9IZyG9BNic7qSUi45SdnvfHX3lcdY0M2teuQ1zIely4FRgrqTVZLdbbAWIiK8AVwGvJ7u7Vg/wrrxiAehMSWG7k4KZ2ahySwoRcfY4ywN4f16vP1wlKbj5yMxsdJOio3l/6Chl+c/NR2Zmo2uapLCrpuCkYGY2mqZJCh2tTgpmZuNpmqRQqSns6HefgpnZaJooKWR9Cq4pmJmNrnmSQpuvUzAzG0/zJAX3KZiZjatpkkJLsUCpWHBSMDMbQ9MkBciGutjhi9fMzEbVVEmhs1R0TcHMbAxNlRQ6SkV6+p0UzMxG01RJobNU9NlHZmZjaK6k0NriAfHMzMbQVEmhwzUFM7MxNVVScEezmdnYmiopdDgpmJmNqamSQmepyA6ffWRmNqomSwruaDYzG0uTJYUiO/sHGRyMRodiZjYhNV1SANyEZGY2iqZKCh2+p4KZ2ZiaKinsGj7b/QpmZiNpqqTQ1eZ7KpiZjaWpkoKbj8zMxtZUSaGr5OYjM7OxNFVS6Ew1he29rimYmY2kyZKCawpmZmNprqTgjmYzszE1VVLoGupodk3BzGwkTZUUOtJ1Cu5TMDMbWVMlhUJBdLR6pFQzs9E0VVKA7AK27b1uPjIzG0nTJQXfaMfMbHS5JgVJZ0h6UNIKSR8dYfmhkq6XdLukuyS9Ps94IOtsdkezmdnIcksKkorARcDrgGOAsyUdM2y1vwWujIgTgLOAL+cVT4Xv02xmNro8awonAysi4tGI6AOuAN40bJ0ApqfpGcBTOcYDZFc1u0/BzGxkeSaFQ4BVVfOrU1m1TwJvl7QauAr4wEg7kvReScslLV+3bt1zCso1BTOz0TW6o/ls4BsRsQB4PfAtSXvEFBEXR8TSiFg6b9685/SCXW0tTgpmZqPIMyk8CSysml+QyqqdC1wJEBG/BdqBuTnGlM4+cvORmdlI8kwKtwJLJC2WVCLrSF42bJ0ngFcDSDqaLCk8t/ahcXS5+cjMbFS5JYWIGADOA64G7ic7y+heSRdIOjOt9pfAeyTdCVwOvDMiIq+YILvRTk9fmcHBXF/GzGxSaslz5xFxFVkHcnXZ+VXT9wEvyzOG4So32tnRX6arLdfDNzObdBrd0Vx3nW2+JaeZ2WiaLym0+kY7Zmajabqk0NXm4bPNzEbTdEmhcp/mHf2uKZiZDdeEScE1BTOz0YyZFCQVJV1fr2DqodO35DQzG9WYSSEiysCgpBl1iid37lMwMxtdLSfqbwPulnQtsL1SGBEfzC2qHHWk5qMe35LTzGwPtSSFH6THlNBVaT7y8NlmZnsYNylExKVp7KKjUtGDEdGfb1j56UjXKWz3xWtmZnsYNylIOhW4FFgJCFgo6ZyI+EW+oeWjUBAdrUV2uKPZzGwPtTQf/RPw2oh4EEDSUWSD170oz8Dy1NVWdE3BzGwEtVyn0FpJCAAR8RDQml9I+esstbhPwcxsBLXUFJZLugS4LM2/DVieX0j58y05zcxGVktSeB/wfqByCuovgS/nFlEdOCmYmY1szKQgqQh8LSLeBvxzfULKX1dbC9vcfGRmtodarmg+LJ2SOmVkZx+5pmBmNlwtzUePAr+WtIzdr2ietDWHrrYWtvuUVDOzPdSSFB5JjwLQnW849dFZKtLjsY/MzPZQS59Cd0R8uE7x1IU7ms3MRlZLn8LL6hRL3XSWWtjRX6Y8GI0OxcxsQqml+eiO1J/wPXbvU5i0g+RVhs/e0V9mWlstb4GZWXOo5RuxHXgWOK2qLJjEI6d2VN1ox0nBzGyXWkZJfVc9Aqmnrso9FXrLU6Tr3Mxs/xi1T0HSlVXTnx627Jo8g8pb5ZacPi3VzGx3Y3U0L6mafs2wZfNyiKVuOlNNwRewmZntbqykMNapOZP6tJ1KR7OHujAz291YfQqdkk4gSxwdaVrp0VGP4PLS3Z6N/L11p5OCmVm1sZLC0+waBO8Zdh8Q75ncIqqD7vbssF1TMDPb3ahJISJeVc9A6qlyGurWnZP2VtNmZrmo5c5rU05XqQXJzUdmZsM1ZVIoFMS0thYnBTOzYXJNCpLOkPSgpBWSPjrKOn8s6T5J90r6Tp7xVJve3uqkYGY2zKh9CpJOHGvDiPjdWMvTCKsXkV3jsBq4VdKyiLivap0lwMeAl0XERknz9yb45yKrKbhPwcys2lhnH/1Tem4HlgJ3kp2O+kJgOXDKOPs+GVgREY8CSLoCeBNwX9U67wEuioiNABGxdm8PYF91t7v5yMxsuFGbjyLiVekMpKeBEyNiaUS8CDgBeLKGfR8CrKqaX53Kqh0FHCXp15JuknTGSDuS9F5JyyUtX7duXQ0vPb7u9ha29rqmYGZWrZY+hedFxN2VmYi4Bzh6P71+C9lwGqcCZwP/Jmnm8JUi4uKUlJbOm7d/Rtjobm9lm2sKZma7qWXc6LskXQJclubfBtxVw3ZPAgur5hewZw1jNXBzRPQDj0l6iCxJ3FrD/p8TNx+Zme2plprCu4B7gQ+lx32pbDy3AkskLZZUAs4Clg1b50dktQQkzSVrTnq0psifo2lOCmZme6jlfgo7JX0FuCoiHqx1xxExIOk84GqgCHwtIu6VdAGwPCKWpWWvlXQfUAY+EhHP7tOR7KXp7a30lQfZ2V+mvbVYj5c0M5vwxk0Kks4EPguUgMWSjgcuiIgzx9s2Iq4CrhpWdn7VdAB/kR51VT3+kZOCmVmmluajvyM7vXQTQETcASzOM6h6qCQFNyGZme1SS1Loj4jNw8om9f0UAKa1VYbP9mmpZmYVtZx9dK+ktwLFdAXyB4Hf5BtW/lxTMDPbUy01hQ8ALwB6ge8Am4E/zzOoenBSMDPb05g1hTR+0QUR8WHg4/UJqT6mp7uvbXHzkZnZkDFrChFRBl5ep1jqamZnlhQ29fQ1OBIzs4mjlj6F2yUtA74HbK8URsQPcouqDqa1tdBSEBt7XFMwM6uoJSm0A88Cp1WVBTCpk4IkZnaWXFMwM6tSyxXNtQxpMSnN6mxl43bXFMzMKmq5orkdOJfsDKT2SnlE/GmOcdXFrM4SG11TMDMbUsspqd8CDgROB24kG+10a55B1cusrlY2uU/BzGxILUnhyIj4BLA9Ii4F/gB4cb5h1YdrCmZmu6tpmIv0vEnSscAMoG73Us5T1tHcTzYun5mZ1ZIULpY0C/gE2f0Q7gM+k2tUdTKrMxs+u6ev3OhQzMwmhFrOProkTd4IHJ5vOPU1q7MEwMaePrraajk718xsaqvl7KPzRyqPiAv2fzj1teuq5n4WzGpwMGZmE0AtP4+3V023A28A7s8nnPqa1bWrpmBmZrU1H/1T9bykz5HdRnPSqzQfbdjupGBmBrV1NA/XSXatwqQ3r7sNgHVbexsciZnZxFBLn8Ld7LrTWhGYB0z6/gSA6e0ttLUUWOukYGYG1Nan8Iaq6QFgTURMiTvTSGL+9DbWbtnZ6FDMzCaEWpLC8CEtpksamomIDfs1ojqb393umoKZWVJLUvgdsBDYCAiYCTyRlgWT/NqFedPaWLFuW6PDMDObEGrpaL4WeGNEzI2IOWTNSddExOKImNQJAXDzkZlZlVqSwksi4qrKTET8BHhpfiHV1/zuNrbsHGBnv4e6MDOrJSk8JelvJS1Kj48DT+UdWL3M785uEeHTUs3MaksKZ5OdhvrD9JifyqaEedOzaxXWuAnJzKymK5o3AB8CSKOlboopNNb0ITM7AHhy0w6WNjgWM7NGG7WmIOl8Sc9P022SrgNWAGsk/X69AsxbJSms3rijwZGYmTXeWM1HbwEeTNPnpHXnA/8F+N85x1U3XW0tzOkqsXpjT6NDMTNruLGSQl9VM9HpwOURUY6I+6nt+oZJY8HsTlZtcE3BzGyspNAr6VhJ84BXAddULevMN6z6Wjirg1WuKZiZjZkUPgR8H3gA+HxEPAYg6fXA7XWIrW4Wzu7kqU07KA9Omf5zM7N9MmpSiIibI+L5ETEnIv6hqvyqiKjplFRJZ0h6UNIKSR8dY703SwpJDTkB6NDZnfSXg6c2uQnJzJrbvtxPoSaSisBFwOuAY4CzJR0zwnrdZLWSm/OKZTxHzp8GwIq1HgPJzJpbbkkBOBlYERGPRkQfcAXwphHW+wfg00DDrh5bkpLCw2uHDwhrZtZc8kwKhwCrquZXp7Ihkk4EFkbEf461I0nvlbRc0vJ169bt90BndpaY193GQ2tcUzCz5lbTqaWSXgosql4/Ir75XF5YUgH4Z+Cd460bERcDFwMsXbo0l97gow6YxsNrXFMws+ZWy+04vwUcAdwBVIYSDWC8pPAk2X0YKhaksopu4FjghnTTngOBZZLOjIjlNUW/Hy2Z382Vy1cxOBgUChp/AzOzKaiWmsJS4Jh9GO/oVmCJpMVkyeAs4K2VhRGxGZhbmZd0A/DhRiQEgOcf2E1PX5lVG3s4bE5XI0IwM2u4WvoU7iH7Fb9X0n2czwOuBu4HroyIeyVdIOnMvd1f3n5vwQwA7ly9ucGRmJk1Ti01hbnAfZJuAYZuOhAR436xp5vzXDWs7PxR1j21hlhyc9QB3bS1FLhz1SbOPO7gRoZiZtYwtSSFT+YdxETQWizwgoOnc9fqTY0OxcysYWq5n8KN9QhkInjhgpl899ZVDJQHaSnmebaumdnENO43n6SXSLpV0jZJfZLKkrbUI7h6O37hTHb0l3nYVzabWZOq5efwl8huv/kw0AG8m2z4iinn+IUzAbj9CTchmVlzqqmNJCJWAMV0P4WvA2fkG1ZjHDank7nT2rh15YZGh2Jm1hC1dDT3SCoBd0j6DPA0+Q6P0TCSOGnRLCcFM2tatXy5/0la7zxgO9lVym/OM6hGOmnRbFZv3MHTmz2Mtpk1n1rOPnpcUgdwUET8fR1iaqiTFs0G4NaVGznzuI4GR2NmVl+1nH30RrJxj36a5o+XtCzvwBrl6IO66SoVWe4mJDNrQrU0H32S7N4ImwAi4g5gcY4xNVRLscCJh83ilsecFMys+dSSFPrT4HXVpvTNjJceNpsH12xl847+RodiZlZXtSSFeyW9FShKWiLpX4Df5BxXQ520eBYR8LvHNzY6FDOzuqolKXwAeAHZYHiXA1uAP88zqEY7YeEsWovipseebXQoZmZ1VcvZRz3Ax9OjKXSUihy/cCY3PeKkYGbNZdSkMN4ZRrUMnT2ZnXL4HL50/Qq27Oxnentro8MxM6uLsWoKpwCryJqMbgaa6h6VLzliDhdet4JbH9vAq48+oNHhmJnVxVh9CgcCf0N2H+UvAq8B1kfEjc0wnPaJh86i1FLgt25CMrMmMmpSSIPf/TQizgFeAqwAbpB0Xt2ia6D21iInHjqT3z7qpGBmzWPMs48ktUn6Q+Ay4P3AhcAP6xHYRHDK4XO57+ktbOrpa3QoZmZ1MWpSkPRN4LfAicDfR8RJEfEPEfFk3aJrsFOOmEME3Oyrm82sSYxVU3g7sAT4EPAbSVvSY+tUvfPacMctnEF7q/sVzKx5jHr2UURMyXsm7I22liJLD5vNTe5XMLMm0fRf/OM55Yg5PPDMVp7d1tvoUMzMcuekMI5TjpgDwK9WrG9wJGZm+XNSGMdxC2Yyp6vEz+5f2+hQzMxy56QwjmJBvPro+dzwwFr6BgYbHY6ZWa6cFGrwmmMOZGvvgG+8Y2ZTnpNCDV5+5FzaWwtcfe8zjQ7FzCxXTgo16CgVefXRB/Cfdz/tJiQzm9KcFGr05hMPYcP2Pm58aF2jQzEzy42TQo1esWQec6eV+MHvVjc6FDOz3Dgp1Ki1WODM4w7h5/ev9YVsZjZl5ZoUJJ0h6UFJKyR9dITlfyHpPkl3Sfq5pMPyjOe5OvvkhfSVB7n8licaHYqZWS5ySwqSisBFwOuAY4CzJR0zbLXbgaUR8ULg+8Bn8opnf1hyQDevPGoe3/zt4+5wNrMpKc+awsnAioh4NCL6gCuAN1WvEBHXR0RPmr0JWJBjPPvFuS9fzNqtvSy786lGh2Jmtt/lmRQOIbvHc8XqVDaac4Gf5BjPfvHKJXM5+qDp/Mt1D9Nfdm3BzKaWCdHRLOntwFLgs6Msf6+k5ZKWr1vX2FNCJfGR04/i8Wd7+O6tq8bfwMxsEskzKTwJLKyaX5DKdiPp94GPA2dGxIin9UTExRGxNCKWzps3L5dg98arnjefpYfN4os/f5gtO/sbHY6Z2X6TZ1K4FVgiabGkEnAWsKx6BUknAP9KlhAmzTCkkvjEG47h2W29fO7qBxsdjpnZfpNbUoiIAeA84GrgfuDKiLhX0gWSzkyrfRaYBnxP0h2Slo2yuwnnuIUzeccpi/jWTY/zuyc2NjocM7P9QhHR6Bj2ytKlS2P58uWNDgOAbb0DvPafb6TUUuDHH3wF09pGvbupmVlDSbotIpaOt96E6GierKa1tfCFs07giQ09fOJH9zQ6HDOz58xJ4Tk6efFsPvTqo/jh7U9yha90NrNJzklhPzjvtCN5xZK5/O2P7uG3jzzb6HDMzPaZk8J+UCyIL731RBbN7eJ9376Nx9Zvb3RIZmb7xElhP5nR0crXzjmJgsTbL7mZVRt6xt/IzGyCcVLYjw6d08k3//RktvUOcNbFN7HSNQYzm2ScFPazYw+Zwbff/WK29w3w3778a25+1H0MZjZ5OCnk4NhDZvCjP3sZs7tKvP2rN3PJLx9lcHByXQ9iZs3JSSEni+Z28YM/exmnPm8+n/rP+3n7V292c5KZTXhOCjma0dHKxX/yIj795t/jzlWbeM3nb+T/XHU/m3d4ED0zm5g8zEWdrN2yk89d8yDfu20100otvP2Uw3jXyxYxv7u90aGZWROodZgLJ4U6u++pLXz5hhVcdffTFAvi948+gP++dAGvXDKPlqIrbmaWDyeFCe6x9du57KbH+dHtT/Ls9j7mdbfxhyccwh+9aAFLDuhudHhmNsU4KUwSfQODXP/gWr5/22quf2AtA4PBcQtm8EcvWsAbjzuYmZ2lRodoZlOAk8IktH5bL/9xx1N8b/kqHnhmK+2tBc466VDe88rDOWRmR6PDM7NJzElhkrvnyc184zcr+dHt2R1Mzzz+YN7zisM5+qDpDY7MzCYjJ4Up4qlNO/jqrx7j8lueoKevzCuWzOXcly/mFUvmUSyo0eGZ2SThpDDFbO7p59u3PM7Xf72SdVt7OXB6O2cceyCvPGouSxfNZnp7a6NDNLMJzElhiuodKHPNvWtYdudT/OKhdfQODFIQvODgGbzosFmccOhMnndgN4vndtHWUmx0uGY2QTgpNIEdfWVuf2IjNz22gZsffZa7Vm9mR38ZgILgsDldHDFvGkfM7+KQmR0cOL2dA2e0c+D0duZMa3Pzk1kTqTUp+E7zk1hHqchLj5zLS4+cC8BAeZCH127j4bXbWLFmKyvWbePhNdu48aG19Jd3T/7Fgjigu40DUpI4YHo7B83IksYB07Oy+dPb6Cz5I2LWTPwfP4W0FAscfdD0Pc5QGhwM1m/v5ZnNO3lm807WbNnJM1t28szmXtZs2clDa7byy4fXs613YI99dpWKzJnWxtxpJWZ3tTGnq8SsrhKzu1qZ3dXG7K5WZnWWmNPVxqyuVqa1tSC5BmI2WTkpNIFCQczvbmd+dzsvXDD6ett6B4aSxtObd7Juay/rtvby7PZe1m/rZfXGHu5avYmNPX171DwqSsUCs1KimN1VGnqe0dHK9I4Wuttb6WproatUpLPUQlfb7s+dpSKtHu7DrGGcFGzItLYWjpw/jSPnTxtzvYhgW+8AG7b3sWF7Hxt7+tiwvZ9tNxC8AAAKcElEQVQN23vZsL2fjdv7eDaV3//MFjZs72PLjn5qvaVEqVigs61IV6mFjlKRzvRoaylSailQainQVizQWiwMzZdaCpSKBWZ2ttLd3kp7a2HX+sUCpRZRKmbzrUVV7adIa4soFQsee8oMJwXbB5Lobs++fA+b01XTNhHB9r4yW3b009NXpqdvgO296bmvTE/vsOe0fEf/AD19Zbb3DrCpp4/egUH6yoP0DQzSn577UtlotZdaFcRQomlryZLO1p0DLJzdSXtrJbkUaCmIlmKWXIqFAq0F0VLMyloKoqWQLWupWl5qKTCrs0Rb2k9ba4H2liJtKXm1taTn1sKu6ZYCBZ8MYHXmpGB1IYlpbS1Ma8vvIzc4GGzs6WN7b5negTI7+wfpK5fpHcgSxq4EUqZ/IOgdJblUnnf2ldm8ox+JtK9BtvUOMFAO+suDlAeDgcFd0/3lYGBwkIHdnp9boioVU5KoTh6tRbpKRdpbi1kySomoWBQHTm9nensrXW27aklZIiruVqMaSk4p+QUwu7NEeykrd79Q83JSsCmjUBBzprUxZ+zWr7qKCMqDWXPbtt4B+gYGhxJMb3+WsLJHlsR6B8r09u8q6x0YTPPZ9M7+Mjv7y/T0ldnRX2Zg5+BQMtrY08/Wnf3s7B98TjFL0FIQkihKFAuioOyMtVldJdpaihQLUFS2TqUy09WW9Qll61ceZM+FXdMSzOgoUUo1qWIh+9tVXmu3x9C2u29f2X82TYpj1zoMrbMX21CJdde2lXVE1X6GHcu0thY6WotDiXSgPMjOgcGhmuVYn42JmHydFMxyJGXNSDM7S3Ub8XagPEhPf5m+lHD6qh+p5lSZrySlHf3loSS1s79MeTAoRzA4GJQHYTCyxLNhex99A8FgVB67Et/mHf2s3dKbbRdBRLZdeXDX9GDVus+1uW8ikqBy6VelOXJ4shooB32pdjm9vYXWYoFiQQwGDAwOUi4Hba2FofLq5PihVy/hjccdnOsxOCmYTTEtxQLTJ0mn+WBqgqski4HBlIhSQhoYzMqBoSSUJZzq6d2fq9fJynYlpOHr7s02lQRYPV+OYNvOAXoHykPLW4sF2lsLWXl5MFt/cNdrVE50ANjeWx5qfiwURGsh+/Lf2T/IQHmQcnpfKol1Rkf+w9k4KZhZwxQKouTO9AllcvycMDOzunBSMDOzIU4KZmY2JNekIOkMSQ9KWiHpoyMsb5P03bT8ZkmL8ozHzMzGlltSkFQELgJeBxwDnC3pmGGrnQtsjIgjgc8Dn84rHjMzG1+eNYWTgRUR8WhE9AFXAG8ats6bgEvT9PeBV2siXs1hZtYk8kwKhwCrquZXp7IR14mIAWAzMGf4jiS9V9JyScvXrVuXU7hmZjYpOpoj4uKIWBoRS+fNm9focMzMpqw8L157ElhYNb8glY20zmpJLcAM4Nmxdnrbbbetl/T4PsY0F1i/j9tOVj7m5uBjbg7P5ZgPq2WlPJPCrcASSYvJvvzPAt46bJ1lwDnAb4E/Aq6LcW4aHRH7XFWQtLyWe5ROJT7m5uBjbg71OObckkJEDEg6D7gaKAJfi4h7JV0ALI+IZcBXgW9JWgFsIEscZmbWILmOfRQRVwFXDSs7v2p6J/Df84zBzMxqNyk6mvejixsdQAP4mJuDj7k55H7MGqcJ38zMmkiz1RTMzGwMTgpmZjakaZLCeIPzTSaSviZpraR7qspmS7pW0sPpeVYql6QL03HfJenEqm3OSes/LOmcRhxLLSQtlHS9pPsk3SvpQ6l8Kh9zu6RbJN2ZjvnvU/niNHjkijSYZCmVjzq4pKSPpfIHJZ3emCOqnaSipNsl/TjNT+ljlrRS0t2S7pC0PJU17rMdQ7egm7oPslNiHwEOB0rAncAxjY7rORzPK4ETgXuqyj4DfDRNfxT4dJp+PfATQMBLgJtT+Wzg0fQ8K03PavSxjXK8BwEnpulu4CGyQRan8jELmJamW4Gb07FcCZyVyr8CvC9N/xnwlTR9FvDdNH1M+ry3AYvT/0Gx0cc3zrH/BfAd4MdpfkofM7ASmDusrGGf7WapKdQyON+kERG/ILuuo1r14IKXAv+1qvybkbkJmCnpIOB04NqI2BARG4FrgTPyj37vRcTTEfG7NL0VuJ9s3KypfMwREdvSbGt6BHAa2eCRsOcxjzS45JuAKyKiNyIeA1aQ/T9MSJIWAH8AXJLmxRQ/5lE07LPdLEmhlsH5JrsDIuLpNP0McECaHu3YJ+V7kpoITiD75Tyljzk1o9wBrCX7J38E2BTZ4JGwe/yjDS45qY4Z+ALwV8Bgmp/D1D/mAK6RdJuk96ayhn22c714zRojIkLSlDvXWNI04N+BP4+ILaoaZX0qHnNElIHjJc0Efgg8v8Eh5UrSG4C1EXGbpFMbHU8dvTwinpQ0H7hW0gPVC+v92W6WmkItg/NNdmtSNZL0vDaVj3bsk+o9kdRKlhC+HRE/SMVT+pgrImITcD1wCllzQeXHXHX8Q8em3QeXnEzH/DLgTEkryZp4TwO+yNQ+ZiLiyfS8liz5n0wDP9vNkhSGBudLZy6cRTYY31RSGVyQ9PwfVeXvSGctvATYnKqlVwOvlTQrndnw2lQ24aR24q8C90fEP1ctmsrHPC/VEJDUAbyGrC/lerLBI2HPY668F9WDSy4Dzkpn6iwGlgC31Oco9k5EfCwiFkTEIrL/0esi4m1M4WOW1CWpuzJN9pm8h0Z+thvd816vB1mv/UNk7bIfb3Q8z/FYLgeeBvrJ2g7PJWtL/TnwMPAzYHZaV2S3RX0EuBtYWrWfPyXrhFsBvKvRxzXG8b6crN31LuCO9Hj9FD/mFwK3p2O+Bzg/lR9O9gW3Avge0JbK29P8irT88Kp9fTy9Fw8Cr2v0sdV4/Key6+yjKXvM6djuTI97K99Njfxse5gLMzMb0izNR2ZmVgMnBTMzG+KkYGZmQ5wUzMxsiJOCmZkNcVIwSySV00iVlcd+G01X0iJVjWprNlF5mAuzXXZExPGNDsKskVxTMBtHGu/+M2nM+1skHZnKF0m6Lo1r/3NJh6byAyT9UNm9EO6U9NK0q6Kkf1N2f4Rr0pXKSPqgsntF3CXpigYdphngpGBWrWNY89FbqpZtjojfA75ENpInwL8Al0bEC4FvAxem8guBGyPiOLL7XtybypcAF0XEC4BNwJtT+UeBE9J+/mdeB2dWC1/RbJZI2hYR00YoXwmcFhGPpoH5nomIOZLWAwdFRH8qfzoi5kpaByyIiN6qfSwiG+9+SZr/a6A1Ij4l6afANuBHwI9i130UzOrONQWz2sQo03ujt2q6zK4+vT8gG8/mRODWqhFBzerOScGsNm+pev5tmv4N2WieAG8Dfpmmfw68D4ZulDNjtJ1KKgALI+J64K/Jhn/eo7ZiVi/+RWK2S0e601nFTyOiclrqLEl3kf3aPzuVfQD4uqSPAOuAd6XyDwEXSzqXrEbwPrJRbUdSBC5LiUPAhZHdP8GsIdynYDaO1KewNCLWNzoWs7y5+cjMzIa4pmBmZkNcUzAzsyFOCmZmNsRJwczMhjgpmJnZECcFMzMb8v8AQ1IPUnQxGcsAAAAASUVORK5CYII=\n",
718 |       "text/plain": [
719 |        "<Figure size 432x288 with 1 Axes>"
720 |       ]
721 |      },
722 |      "metadata": {
723 |       "needs_background": "light"
724 |      },
725 |      "output_type": "display_data"
726 |     }
727 |    ],
728 |    "source": [
729 |     "plt.plot(loss_history)\n",
730 |     "plt.title(\"Autoencoder's training improvement\")\n",
731 |     "plt.xlabel('Epochs')\n",
732 |     "plt.ylabel('Mean Squared Error')\n",
733 |     "plt.show()"
734 |    ]
735 |   },
736 |   {
737 |    "cell_type": "markdown",
738 |    "metadata": {},
739 |    "source": [
740 |     "## 4. Extraction of the encoded dataframe\n",
741 |     "\n",
742 |     "Now that the training of the Autoencoder is done, we can extract our data at the central layer, where they received the strongest compression.\n",
743 |     "\n",
744 |     "In order to do that, we must create an apposite **extraction function** using the **Keras backend submodule**. We use it to implement a Keras function that takes the Autoencoder's input layer as input, and returns the outcome of the central layer as output."
745 |    ]
746 |   },
747 |   {
748 |    "cell_type": "code",
749 |    "execution_count": 11,
750 |    "metadata": {},
751 |    "outputs": [
752 |     {
753 |      "name": "stdout",
754 |      "output_type": "stream",
755 |      "text": [
756 |       "(569, 10)\n"
757 |      ]
758 |     }
759 |    ],
760 |    "source": [
761 |     "from tensorflow.keras import backend as K\n",
762 |     "\n",
763 |     "# create a Keras function\n",
764 |     "extract_encoded_data = K.function(inputs = Autoencoder.layers[0].input, \n",
765 |     "                                  outputs = Autoencoder.layers[4].output)\n",
766 |     "\n",
767 |     "# extract encoded dataframe\n",
768 |     "encoded_dataframe = extract_encoded_data(df)\n",
769 |     "\n",
770 |     "print(encoded_dataframe.shape)"
771 |    ]
772 |   },
773 |   {
774 |    "cell_type": "markdown",
775 |    "metadata": {},
776 |    "source": [
777 |     "I defined `Autoencoder.layers[4]` as the output of that function, since `.layers[4]` is the index of the central layer that compressed my initial dataframe."
778 |    ]
779 |   },
780 |   {
781 |    "cell_type": "markdown",
782 |    "metadata": {
783 |     "colab_type": "text",
784 |     "id": "mEDmoV4dfSD4"
785 |    },
786 |    "source": [
787 |     "## 5. Practical application\n",
788 |     "\n",
789 |     "Now that we have an encoded version of our initial dataset, we can use it for our Machine Learning task. Let's implement a Neural Network Classifier and train it on the encoded dataset! Because I want to save time, I'll make it in the most minimal way I know; however, if you need more details, please check [my other tutorials](https://github.com/IvanBongiorni/TensorFlow2.0_Tutorial) in which all of these passages are explained in more detail."
790 |    ]
791 |   },
792 |   {
793 |    "cell_type": "code",
794 |    "execution_count": 12,
795 |    "metadata": {},
796 |    "outputs": [],
797 |    "source": [
798 |     "\n",
799 |     "# Train-Test split\n",
800 |     "from sklearn.model_selection import train_test_split\n",
801 |     "X_train, X_test, y_train, y_test = train_test_split(encoded_dataframe, target, test_size=0.25, random_state=173)\n",
802 |     "\n",
803 |     "# Scaling of data\n",
804 |     "from sklearn.preprocessing import StandardScaler\n",
805 |     "scaler = StandardScaler()\n",
806 |     "X_train = scaler.fit_transform(X_train)\n",
807 |     "X_test = scaler.transform(X_test)\n",
808 |     "\n",
809 |     "# Model architecture - layers: [10, 5, 5, 2]\n",
810 |     "from tensorflow.keras.layers import Dense\n",
811 |     "from tensorflow.keras.activations import elu, softmax\n",
812 |     "\n",
813 |     "model = tf.keras.models.Sequential([\n",
814 |     "    Dense(10, input_shape = (X_train.shape[1], ), activation = elu), \n",
815 |     "    Dense(5, activation = elu), \n",
816 |     "    Dense(5, activation = elu),     \n",
817 |     "    Dense(2, activation = softmax)])\n",
818 |     "\n",
819 |     "bce_loss = tf.keras.losses.BinaryCrossentropy()\n",
820 |     "optimizer = tf.optimizers.Adam(learning_rate = 0.001)\n",
821 |     "\n",
822 |     "# Training\n",
823 |     "for epoch in range(500):\n",
824 |     "    with tf.GradientTape() as tape:\n",
825 |     "        current_loss = bce_loss(model(X_train), y_train)    \n",
826 |     "    gradients = tape.gradient(current_loss, model.trainable_variables)\n",
827 |     "    optimizer.apply_gradients(zip(gradients, model.trainable_variables))"
828 |    ]
829 |   },
830 |   {
831 |    "cell_type": "markdown",
832 |    "metadata": {},
833 |    "source": [
834 |     "Let's check its predictions:"
835 |    ]
836 |   },
837 |   {
838 |    "cell_type": "code",
839 |    "execution_count": 13,
840 |    "metadata": {},
841 |    "outputs": [
842 |     {
843 |      "name": "stdout",
844 |      "output_type": "stream",
845 |      "text": [
846 |       "[[86  1]\n",
847 |       " [ 3 53]]\n"
848 |      ]
849 |     }
850 |    ],
851 |    "source": [
852 |     "prediction = model.predict(X_test)  # prediction of test data\n",
853 |     "prediction = np.argmax(prediction, axis=1)  # reverse one-hot encoding\n",
854 |     "testdata = np.argmax(y_test, axis=1)  # reverse for target data too\n",
855 |     "\n",
856 |     "from sklearn.metrics import confusion_matrix\n",
857 |     "CM = confusion_matrix(prediction, testdata)\n",
858 |     "print(CM)"
859 |    ]
860 |   },
861 |   {
862 |    "cell_type": "code",
863 |    "execution_count": 14,
864 |    "metadata": {},
865 |    "outputs": [
866 |     {
867 |      "data": {
868 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAD8CAYAAAA2Y2wxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAD+pJREFUeJzt3X2QXXV9x/H3d3eTTYJCiElDIAqCQIpTDRWpPCjIwxSwEtBKBa2I1Oi02NJaBe1IxUFrrZRiRx0XJDxKQKwgyIM8iloFUomAICoIhUwkgAYlkA1777d/5Kpb8nDvmvvbc/fk/WJ+k73n3v3dL0zmM19+53fOicxEklROX9UFSFLdGbSSVJhBK0mFGbSSVJhBK0mFGbSSVJhBK0mFGbSSVJhBK0mFDZT+gueeeNBLz7SOqdu+tuoS1ING1iyLTZ1jLJkzaeaOm/x9nbCjlaTCine0kjSumo2qK1iHQSupXhojVVewDoNWUq1kNqsuYR0GraR6aRq0klSWHa0kFebJMEkqzI5WkspKdx1IUmGeDJOkwlw6kKTCPBkmSYXZ0UpSYZ4Mk6TCPBkmSWVlukYrSWW5RitJhbl0IEmF2dFKUmGN56quYB0GraR6celAkgpz6UCSCrOjlaTCDFpJKiu7dDIsInYFLhl1aEfgFGA68G7g8dbxD2fm1Ruby6CVVC9dWqPNzPuB+QAR0Q8sA74KHAeckZmf7nQug1ZSvZRZOjgQeCAzH46IMf9yX/frkaQKZbPjERELI2LJqLFwA7O+Fbh41OsTIuKuiDgnIrZuV5JBK6lems2OR2YOZeYeo8bQ86eLiMnA4cCXW4c+D+zE2mWF5cDp7Upy6UBSvXR/H+2hwPcz8zGA3/wJEBFnAVe1m8CglVQvI12/8ffRjFo2iIg5mbm89fJI4J52Exi0kuqlix1tRGwBHAy8Z9ThT0XEfCCBh5733noZtJLqpYu7DjJzFfCi5x37y7HOY9BKqhfvdSBJhXkJriQVZkcrSYV1f9fBJjNoJdVLZtUVrMOglVQvrtFKUmEGrSQV5skwSSqs0ai6gnUYtJLqxaUDSSrMoJWkwlyjlaSysuk+Wkkqy6UDSSrMXQeSVJgd7ebj/MVf5StXXktEsPNOO3Dah/+ByZMn8Zmh8/jGzd+mr6+PvzjyDbz9LQuqLlUVOWvodN5w2EGsePwJ5u9+YNXl1IdBu3l47PEnuOiyK7jioi8wZXCQ93/kE1xzwzdJkp+veIIrvzREX18fT/5yZdWlqkLnn38pn/vcIhYtOrPqUuplIt5UJiLmAQuA7VqHlgFfy8z7ShY20Y00GgwPr2Ggf4BnVw8za+YM/vOs8/nUR0+ir2/tU95ftPX0iqtUlb717dvYfvu5VZdRPz3Y0fZt7M2IOAlYDARwe2sEcHFEnFy+vIlp9qyZvPPoN3PQm97B6xccwwu3mMY+f/IqHlm2nGtu/CZHvetvee/7P8LDjyyrulSpfprZ+RgnGw1a4Hjg1Zn5ycy8sDU+CezZek/r8dSvfs3N3/oe1315ETddcRHPrh7myutuYs1zzzE4eTKXnvMZ3vzGQ/jIJ86oulSpfhqNzsc4aRe0TWDb9Ryf03pvvSJiYUQsiYglZ59/8YY+VlvfW7KU7badzYytpzNpYIAD99ubpXffyzazZnLQfvsAcNB+e/PjB35WcaVS/WSz2fEYL+3WaE8EboyInwCPtI69BHgZcMKGfikzh4AhgOeeeLD3VqYLmzN7Fnfd8yOeXb2aKYOD3LZkKS+ftzMv2GIat3//B8zddhvuuPNutn/xdu0nkzQ2E+3KsMy8NiJ2Ye1SweiTYXdkZu/tCu4Rr3j5PA5+/b4cddz76O/vZ94uO/GWBYeyengNJ536KS645HKmTZ3CqSefWHWpqtCFF3yW/V63FzNnzuChB5dw6sc+zaJzF1dd1sTXg/c6iCy8FWJz7GjV3tRtX1t1CepBI2uWxabOsepjb+s4c7Y45aJN/r5OuI9WUr2M9N7/bBu0kuqlB5cODFpJ9TLRToZJ0kQzntu2OtVuH60kTSxdvDIsIqZHxGUR8aOIuC8i9oqIGRFxfUT8pPXn1u3mMWgl1Ut3L8E9E7g2M+cBrwTuA04GbszMnYEbW683yqUDSfXSpUtrI2Ir4HXAOwEycw2wJiIWAPu3PnYecAtw0sbmsqOVVCvZzI5HGy8FHgcWRcSdEXF2RGwBzM7M5a3P/ByY3W4ig1ZSvYxh6WD0fVlaY+GomQaAPwY+n5m7A6t43jJBrr3iq21iu3QgqV7GsOtg9H1Z1uNR4NHMvK31+jLWBu1jETEnM5dHxBxgRbvvsaOVVC9dOhmWmT8HHomIXVuHDgTuBb4GHNs6dixwRbuS7Ggl1Ut3L1h4H3BRREwGHgSOY22DemlEHA88DBzVbhKDVlKtZKN7Fyxk5lJgj/W8NaanaRq0kurFS3AlqawOtm2NO4NWUr0YtJJUWO/dU8aglVQvOdJ7SWvQSqqX3stZg1ZSvXgyTJJKs6OVpLLsaCWpNDtaSSorR6quYF0GraRa6cGnjRu0kmrGoJWksuxoJakwg1aSCstGVF3COgxaSbViRytJhWXTjlaSirKjlaTCMu1oJakoO1pJKqzprgNJKsuTYZJUmEErSYVl792O1qCVVC92tJJUmNu7JKmwhrsOJKksO1pJKqwX12j7qi5Akrops/PRiYjoj4g7I+Kq1utzI+JnEbG0Nea3m8OOVlKtFOho/w64D9hy1LEPZOZlnU5gRyupVhrNvo5HOxExF3gDcPam1GTQSqqVsSwdRMTCiFgyaix83nT/AXyQdR/5+PGIuCsizoiIwXY1GbSSaqWZ0fHIzKHM3GPUGPrNPBHxZ8CKzPyf533Fh4B5wKuBGcBJ7WoyaCXVSmZ0PNrYBzg8Ih4CFgMHRMSFmbk81xoGFgF7tpvIoJVUK93adZCZH8rMuZm5A/BW4KbMfHtEzAGIiACOAO5pV1PxXQfTX3JA6a/QBLR8v5dVXYJqqln+goWLImIWEMBS4L3tfsHtXZJqpZPdBGOVmbcAt7R+HnP3aNBKqpUevEuiQSupXsZh6WDMDFpJteJNZSSpsB58CK5BK6leEjtaSSpqxKUDSSrLjlaSCnONVpIKs6OVpMLsaCWpsIYdrSSV1YPPZjRoJdVL045WksrypjKSVJgnwySpsGa4dCBJRTWqLmA9DFpJteKuA0kqzF0HklSYuw4kqTCXDiSpMLd3SVJhDTtaSSrLjlaSCjNoJamwHnxkmEErqV7saCWpMC/BlaTCenEfbV/VBUhSNzXHMDYmIqZExO0R8YOI+GFEnNo6/tKIuC0ifhoRl0TE5HY1GbSSaqVbQQsMAwdk5iuB+cAhEfEa4F+BMzLzZcAvgePbTWTQSqqVHMPY6DxrPd16Oak1EjgAuKx1/DzgiHY1GbSSaqUZnY92IqI/IpYCK4DrgQeAlZk50vrIo8B27eYxaCXVSmMMIyIWRsSSUWPh6Lkys5GZ84G5wJ7AvN+nJncdSKqV5hhulJiZQ8BQB59bGRE3A3sB0yNioNXVzgWWtft9O1pJtdLFXQezImJ66+epwMHAfcDNwJ+3PnYscEW7muxoJdVKF2/8PQc4LyL6WduUXpqZV0XEvcDiiDgNuBP4YruJDFpJtdKtS3Az8y5g9/Ucf5C167UdM2gl1cpI9N7DbAxaSbXSezFr0EqqGe/eJUmFjWV713gxaCXVSu/FrEErqWZcOpCkwho92NMatJJqxY5WkgpLO1pJKsuOdjM1ODjIN66/hMHJg/QP9HP55dfw8dPOqLosVWTGBYvJZ5+FZoNsNFj5N+9h2rHvYnDvfSGbNFeu5Nf/9i80n3yy6lInJLd3baaGh4c57NBjWLXqGQYGBrjhxsv4xnW3cMcdd1Zdmiqy8h9PJH/11G9fP/vlxTxz3jkATD3izUx7+7E8fea/V1XehNZ7MWvQjptVq54BYNKkASZNGujJdSRVJ5955ncvpkzpzbSYIEZ68D/e7x20EXFcZi7qZjF11tfXx3f++yp23HF7hr5wAUvuWFp1SapKwlaf/DRksvrrV7L66isBmHbcXzHloD8lVz3Nyg+cWHGRE1cvNjGbcuPvUzf0xujHQ4yM/HoTvqI+ms0me73mMHbZeS9etccr2W23XaouSRVZ+fcnsPKv381T//RBph5+BJP+6BUAPLPobH7xtrew+qYbmLrgTRVXOXF18Sm4XbPRoI2IuzYw7gZmb+j3MnMoM/fIzD0GBl7Y9aInsqee+hW33vpdDj54v6pLUUWaTz4BQK5cyfB3vsXArn/4/94fvvF6Bvd9XRWl1UKO4Z/x0q6jnQ28A3jjeoanRDs0c+YMttpqSwCmTBnkgAP25f4fP1BxVarElCnE1Km//Xnyq17NyEM/o3+73z1IdfLe+9J45H8rKnDi68WOtt0a7VXACzJznQXFiLilSEU1tM02f8DQWafT39dHX18fX/mvr3PtNTdVXZYq0Dd9a7b66GlrX/T3M3zzDTy35Ha2POVj9M99MWTSeOwxnj7z9GoLncAa2XtrtJGFi9pi2g6992+tyj20z/ZVl6AeNOv6b8amznHM9kd2nDlfevirm/x9nXB7l6Ra6cVdBwatpFrxElxJKsxLcCWpMJcOJKmwXtx1YNBKqhWXDiSpME+GSVJhrtFKUmEuHUhSYaWvdv19bMptEiWp5zTIjkc7EXFORKyIiHtGHftoRCyLiKWtcVi7eQxaSbXSJDseHTgXOGQ9x8/IzPmtcXW7SVw6kFQr3Vw6yMxbI2KHTZ3HjlZSrXS5o92QE1oPQTgnIrZu92GDVlKtjOUJC6Mfu9UaCzv4is8DOwHzgeVA25sHu3QgqVbGcgluZg4BQ2OZPzMf+83PEXEWax+QsFEGraRaKb2PNiLmZOby1ssjgXs29nkwaCXVTDeDNiIuBvYHZkbEo8A/A/tHxHwggYeA97Sbx6CVVCtd3nVw9HoOf3Gs8xi0kmrFS3AlqTBvKiNJhTWy926UaNBKqpVevKmMQSupVlyjlaTCXKOVpMKaLh1IUll2tJJUmLsOJKkwlw4kqTCXDiSpMDtaSSrMjlaSCmtko+oS1mHQSqoVL8GVpMK8BFeSCrOjlaTC3HUgSYW560CSCvMSXEkqzDVaSSrMNVpJKsyOVpIKcx+tJBVmRytJhbnrQJIK82SYJBXm0oEkFeaVYZJUmB2tJBXWi2u00YvpX1cRsTAzh6quQ73Fvxf111d1AZuZhVUXoJ7k34uaM2glqTCDVpIKM2jHl+twWh//XtScJ8MkqTA7WkkqzKAdJxFxSETcHxE/jYiTq65H1YuIcyJiRUTcU3UtKsugHQcR0Q98FjgU2A04OiJ2q7Yq9YBzgUOqLkLlGbTjY0/gp5n5YGauARYDCyquSRXLzFuBX1Rdh8ozaMfHdsAjo14/2jomaTNg0EpSYQbt+FgGvHjU67mtY5I2Awbt+LgD2DkiXhoRk4G3Al+ruCZJ48SgHQeZOQKcAFwH3Adcmpk/rLYqVS0iLga+C+waEY9GxPFV16QyvDJMkgqzo5WkwgxaSSrMoJWkwgxaSSrMoJWkwgxaSSrMoJWkwgxaSSrs/wDpovrEe/2wbwAAAABJRU5ErkJggg==\n",
869 |       "text/plain": [
870 |        "<Figure size 432x288 with 2 Axes>"
871 |       ]
872 |      },
873 |      "metadata": {
874 |       "needs_background": "light"
875 |      },
876 |      "output_type": "display_data"
877 |     }
878 |    ],
879 |    "source": [
880 |     "import seaborn\n",
881 |     "seaborn.heatmap(CM, annot=True)\n",
882 |     "plt.show()"
883 |    ]
884 |   },
885 |   {
886 |    "cell_type": "code",
887 |    "execution_count": 15,
888 |    "metadata": {},
889 |    "outputs": [
890 |     {
891 |      "name": "stdout",
892 |      "output_type": "stream",
893 |      "text": [
894 |       "Test Accuracy: 0.972027972027972\n"
895 |      ]
896 |     }
897 |    ],
898 |    "source": [
899 |     "print('Test Accuracy: ' + str(np.sum(np.diag(CM)) / np.sum(CM)))"
900 |    ]
901 |   },
902 |   {
903 |    "cell_type": "markdown",
904 |    "metadata": {},
905 |    "source": [
906 |     "Cool, I have achieved pretty much the same accuracy that I reached with the [full-Batch model in my first Notebook](https://github.com/IvanBongiorni/TensorFlow2.0_Tutorial/blob/master/TensorFlow2.0_01_basic_Classifier.ipynb)! But this time using a much lighter model trained on a compressed version of the original dataframe.\n",
907 |     "\n",
908 |     "This is the power of Autoencoders. You'll find that computational times will be much slower, and with very similar results. In case of very multicollinear datasets, you might even find a significant improvement in your model performance."
909 |    ]
910 |   },
911 |   {
912 |    "cell_type": "markdown",
913 |    "metadata": {},
914 |    "source": [
915 |     "..."
916 |    ]
917 |   },
918 |   {
919 |    "cell_type": "markdown",
920 |    "metadata": {},
921 |    "source": [
922 |     "PS: If you want to encode many dataframes, make sure you remember [how to save and restore it](https://github.com/IvanBongiorni/TensorFlow2.0_Tutorial/blob/master/TensorFlow_2.0_03_Save_and_Restore_models.ipynb)!"
923 |    ]
924 |   }
925 |  ],
926 |  "metadata": {
927 |   "colab": {
928 |    "collapsed_sections": [],
929 |    "name": "TensorFlow_3_Autoencoder_Dimensionality_Reduction.ipynb",
930 |    "provenance": [],
931 |    "toc_visible": true,
932 |    "version": "0.3.2"
933 |   },
934 |   "kernelspec": {
935 |    "display_name": "Python 3",
936 |    "language": "python",
937 |    "name": "python3"
938 |   },
939 |   "language_info": {
940 |    "codemirror_mode": {
941 |     "name": "ipython",
942 |     "version": 3
943 |    },
944 |    "file_extension": ".py",
945 |    "mimetype": "text/x-python",
946 |    "name": "python",
947 |    "nbconvert_exporter": "python",
948 |    "pygments_lexer": "ipython3",
949 |    "version": "3.6.7"
950 |   }
951 |  },
952 |  "nbformat": 4,
953 |  "nbformat_minor": 1
954 | }
955 | 


--------------------------------------------------------------------------------
/TensorFlow2.0__00.03_Save_and_Restore_models.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {},
  6 |    "source": [
  7 |     "# Save and Restore Models in TensorFlow 2.0\n",
  8 |     "\n",
  9 |     "#### Author: Ivan Bongiorni, Data Scientist at GfK.\n",
 10 |     "\n",
 11 |     "[LinkedIn profile](https://www.linkedin.com/in/ivan-bongiorni-b8a583164/)\n",
 12 |     "\n",
 13 |     "Summary:\n",
 14 |     "\n",
 15 |     "1. Load data + quick dataprep,\n",
 16 |     "2. Model architecture,\n",
 17 |     "3. Save the model afte training,\n",
 18 |     "4. Restore the model and keep training."
 19 |    ]
 20 |   },
 21 |   {
 22 |    "cell_type": "code",
 23 |    "execution_count": 1,
 24 |    "metadata": {},
 25 |    "outputs": [
 26 |     {
 27 |      "name": "stdout",
 28 |      "output_type": "stream",
 29 |      "text": [
 30 |       "2.0.0-alpha0\n"
 31 |      ]
 32 |     }
 33 |    ],
 34 |    "source": [
 35 |     "import numpy as np\n",
 36 |     "import pandas as pd\n",
 37 |     "\n",
 38 |     "import tensorflow as tf\n",
 39 |     "print(tf.__version__)\n",
 40 |     "\n",
 41 |     "from matplotlib import pyplot as plt"
 42 |    ]
 43 |   },
 44 |   {
 45 |    "cell_type": "markdown",
 46 |    "metadata": {},
 47 |    "source": [
 48 |     "## 1. Load data + dataprep\n",
 49 |     "\n",
 50 |     "The spam classification dataset, [available from the UCI ML repository](https://archive.ics.uci.edu/ml/datasets/spambase), contains an already preprocessed collection of email data. Each datapoint corresponds to an email, and is classified as either \"spam\" or \"not spam\"."
 51 |    ]
 52 |   },
 53 |   {
 54 |    "cell_type": "code",
 55 |    "execution_count": 2,
 56 |    "metadata": {},
 57 |    "outputs": [
 58 |     {
 59 |      "data": {
 60 |       "text/plain": [
 61 |        "(4601, 58)"
 62 |       ]
 63 |      },
 64 |      "execution_count": 2,
 65 |      "metadata": {},
 66 |      "output_type": "execute_result"
 67 |     }
 68 |    ],
 69 |    "source": [
 70 |     "# load dataframe\n",
 71 |     "df = pd.read_csv(\"https://archive.ics.uci.edu/ml/machine-learning-databases/spambase/spambase.data\", header=None)\n",
 72 |     "df.shape"
 73 |    ]
 74 |   },
 75 |   {
 76 |    "cell_type": "code",
 77 |    "execution_count": 3,
 78 |    "metadata": {},
 79 |    "outputs": [],
 80 |    "source": [
 81 |     "# one-hot encode target variable\n",
 82 |     "classification = pd.get_dummies(df.iloc[:,57])\n",
 83 |     "\n",
 84 |     "df = df.drop(df.columns[57], axis=1)   # drop target col from explanatory df\n",
 85 |     "\n",
 86 |     "df = df.values.astype(np.float64)\n",
 87 |     "classification = classification.values.astype(np.float64)"
 88 |    ]
 89 |   },
 90 |   {
 91 |    "cell_type": "code",
 92 |    "execution_count": 4,
 93 |    "metadata": {},
 94 |    "outputs": [
 95 |     {
 96 |      "name": "stdout",
 97 |      "output_type": "stream",
 98 |      "text": [
 99 |       "X_train shape: (3450, 57)\n",
100 |       "y_train shape: (3450, 2)\n",
101 |       "\n",
102 |       "X_test shape: (1151, 57)\n",
103 |       "y_test shape: (1151, 2)\n"
104 |      ]
105 |     }
106 |    ],
107 |    "source": [
108 |     "## TRAIN-TEST SPLIT\n",
109 |     "from sklearn.model_selection import train_test_split\n",
110 |     "\n",
111 |     "X_train, X_test, y_train, y_test = train_test_split(df, classification, test_size=0.25, random_state=173)\n",
112 |     "\n",
113 |     "print('X_train shape: ' + str(X_train.shape) + '\\ny_train shape: ' + str(y_train.shape))\n",
114 |     "print('\\nX_test shape: ' + str(X_test.shape) + '\\ny_test shape: ' + str(y_test.shape))\n",
115 |     "\n",
116 |     "\n",
117 |     "# SCALE DATA\n",
118 |     "from sklearn.preprocessing import StandardScaler\n",
119 |     "\n",
120 |     "scaler = StandardScaler()\n",
121 |     "X_train = scaler.fit_transform(X_train)\n",
122 |     "X_test = scaler.transform(X_test)"
123 |    ]
124 |   },
125 |   {
126 |    "cell_type": "markdown",
127 |    "metadata": {},
128 |    "source": [
129 |     "## 2. Model architecture"
130 |    ]
131 |   },
132 |   {
133 |    "cell_type": "code",
134 |    "execution_count": 5,
135 |    "metadata": {},
136 |    "outputs": [],
137 |    "source": [
138 |     "from tensorflow.keras.layers import Dense, Dropout\n",
139 |     "from tensorflow.keras.activations import elu, softmax\n",
140 |     "\n",
141 |     "# Architecture\n",
142 |     "n_input = X_train.shape[1]\n",
143 |     "n_hidden1 = 30\n",
144 |     "n_hidden2 = 30\n",
145 |     "n_hidden3 = 15\n",
146 |     "n_output = y_train.shape[1]\n",
147 |     "\n",
148 |     "# set dropout probability\n",
149 |     "dropout_prob = 0.1\n",
150 |     "\n",
151 |     "\n",
152 |     "\n",
153 |     "model = tf.keras.models.Sequential([\n",
154 |     "    Dense(n_input, input_shape = (n_input,), activation = elu),   # Input layer\n",
155 |     "    \n",
156 |     "    Dense(n_hidden1, activation = elu), # hidden layer 1\n",
157 |     "    Dropout(dropout_prob), \n",
158 |     "    \n",
159 |     "    Dense(n_hidden2, activation = elu), # hidden layer 2\n",
160 |     "    Dropout(dropout_prob), \n",
161 |     "    \n",
162 |     "    Dense(n_hidden3, activation = elu), # hidden layer 3\n",
163 |     "    Dropout(dropout_prob), \n",
164 |     "    \n",
165 |     "    Dense(n_output, activation = softmax)  # Output layer\n",
166 |     "])\n",
167 |     "\n",
168 |     "# set number of epochs\n",
169 |     "n_epochs = 500\n",
170 |     "\n",
171 |     "# Loss: Binary cross-entropy \n",
172 |     "bce_loss = tf.keras.losses.BinaryCrossentropy()\n",
173 |     "\n",
174 |     "# Binary Accuracy\n",
175 |     "accuracy = tf.keras.metrics.BinaryAccuracy()\n",
176 |     "\n",
177 |     "# Adam Optimizer\n",
178 |     "optimizer = tf.optimizers.Adam(learning_rate = 0.001)"
179 |    ]
180 |   },
181 |   {
182 |    "cell_type": "markdown",
183 |    "metadata": {},
184 |    "source": [
185 |     "To train the Network, I will use the wrapper function from [the first Notebook](https://github.com/IvanBongiorni/TensorFlow2.0_Tutorial/blob/master/TensorFlow2.0_01_basic_Classifier.ipynb):"
186 |    ]
187 |   },
188 |   {
189 |    "cell_type": "code",
190 |    "execution_count": 6,
191 |    "metadata": {},
192 |    "outputs": [],
193 |    "source": [
194 |     "\n",
195 |     "def train_model(model, train, target, loss, optimizer, metrics, n_epochs, verbose=True, plot=True):\n",
196 |     "    '''\n",
197 |     "    Trains a TensorFlow 2.0 model\n",
198 |     "    \n",
199 |     "    Returns:\n",
200 |     "        prediction: The model's prediction - numpy\n",
201 |     "        loss_history: list\n",
202 |     "        metrics_history: list\n",
203 |     "    \n",
204 |     "    Arguments:\n",
205 |     "        model: a Sequential() model\n",
206 |     "        train: Train dataset (numpy)\n",
207 |     "        target: Target dataset - True outcome (numpy)\n",
208 |     "        loss: loss object or function\n",
209 |     "        optimizer: optimizer object (learnin rate is specified within)\n",
210 |     "        metrics: an evaluation metrics of the model's goodness\n",
211 |     "        n_epochs: number of iterations of training\n",
212 |     "        verbose: print training progress yes/no\n",
213 |     "        plot = plot loss and accuracy histories at the end of training y/n\n",
214 |     "    '''\n",
215 |     "    import numpy as np\n",
216 |     "    import tensorflow as tf\n",
217 |     "    import matplotlib.pyplot as plt\n",
218 |     "    \n",
219 |     "    \n",
220 |     "    loss_history = []\n",
221 |     "    metrics_history = []\n",
222 |     "    \n",
223 |     "    for epoch in range(n_epochs):\n",
224 |     "    \n",
225 |     "        with tf.GradientTape() as tape:\n",
226 |     "            current_loss = loss(model(train), target)\n",
227 |     "        \n",
228 |     "        gradients = tape.gradient(current_loss, model.trainable_variables)\n",
229 |     "        optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
230 |     "        \n",
231 |     "        loss_history.append(current_loss.numpy())\n",
232 |     "        metrics.update_state(target, model(train))\n",
233 |     "        current_metrics = metrics.result().numpy()\n",
234 |     "        metrics_history.append(current_metrics)\n",
235 |     "        \n",
236 |     "        if verbose:            \n",
237 |     "            if (epoch+1) % 100 == 0:\n",
238 |     "                print(str(epoch+1) \n",
239 |     "                      + '.\\tTraining Loss: ' + str(current_loss.numpy()) \n",
240 |     "                      + ',\\tAccuracy: ' + str(current_metrics))\n",
241 |     "    \n",
242 |     "        accuracy.reset_states()\n",
243 |     "    \n",
244 |     "    print('\\nTraining complete.')\n",
245 |     "    \n",
246 |     "    prediction = model(X_train)\n",
247 |     "    \n",
248 |     "    if plot:\n",
249 |     "        plt.figure(figsize = (15, 4))\n",
250 |     "        plt.subplots_adjust(wspace=0.2)  # this distances the plots\n",
251 |     "\n",
252 |     "        plt.subplot(1, 2, 1)\n",
253 |     "        plt.plot(loss_history)\n",
254 |     "        plt.title('Loss History')\n",
255 |     "        plt.xlabel('epochs')\n",
256 |     "        plt.ylabel('Binary Cross-Entropy')\n",
257 |     "\n",
258 |     "        plt.subplot(1, 2, 2)\n",
259 |     "        plt.plot(metrics_history)\n",
260 |     "        plt.title('Metrics History')\n",
261 |     "        plt.xlabel('epochs')\n",
262 |     "        plt.ylabel('Metrics')\n",
263 |     "        \n",
264 |     "        plt.show()\n",
265 |     "    \n",
266 |     "    return loss_history, metrics_history"
267 |    ]
268 |   },
269 |   {
270 |    "cell_type": "markdown",
271 |    "metadata": {},
272 |    "source": [
273 |     "And call it:"
274 |    ]
275 |   },
276 |   {
277 |    "cell_type": "code",
278 |    "execution_count": 7,
279 |    "metadata": {},
280 |    "outputs": [
281 |     {
282 |      "name": "stdout",
283 |      "output_type": "stream",
284 |      "text": [
285 |       "100.\tTraining Loss: 1.08087158203125,\tAccuracy: 0.9356522\n",
286 |       "200.\tTraining Loss: 0.8725841641426086,\tAccuracy: 0.9472464\n",
287 |       "300.\tTraining Loss: 0.7593502402305603,\tAccuracy: 0.95246375\n",
288 |       "400.\tTraining Loss: 0.7209241986274719,\tAccuracy: 0.9542029\n",
289 |       "500.\tTraining Loss: 0.7009227275848389,\tAccuracy: 0.95507246\n",
290 |       "\n",
291 |       "Training complete.\n"
292 |      ]
293 |     },
294 |     {
295 |      "data": {
296 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3IAAAEWCAYAAAAjLaWEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XmcZHV57/HPU13dXb3vM0DPzr6DDqigBkEEiUIQo+AStyt643aNxqs3xi3uMclN3FGIoigxqIQocWPRiwGcYYcBZPaVme6e7um9a3vuH+f0TE2v1cvp6qr5vl+vevWpc37nnKdbqTNP/X6/52fujoiIiIiIiBSPWKEDEBERERERkZlRIiciIiIiIlJklMiJiIiIiIgUGSVyIiIiIiIiRUaJnIiIiIiISJFRIiciIiIiIlJklMiJLCJm9noz+1Wh4xAREZkLM/s/ZvbtBb6nnqFyRFEiJ0ccM9tqZi8twH3fbGb3TBWPu9/k7i/L41rfMbNPRxGniIiUpvB5kzSz1jH7HzYzN7NVeVzjAjPbOV07d/+su/+P2Uc77r56hoqMoURO5AhkZmWFjkFERApiC3DN6BszOx2oms8bmFl8Pq+32OgZKouFEjmRHGb2djPbaGb7zew2Mzsm3G9m9k9mts/MDpjZo2Z2WnjsMjPbYGZ9ZrbLzD44h/sf/MZxsnua2bXA64EPmVm/mf1n2P5kM7vbzHrM7Akzuzznut8xs6+b2e1mNgD8lZntzX3YmtlVZvbwbGMXEZGi8D3gL3Levwm4MbeBmVWa2ZfMbHv4rPiGmVWZWQ3wX8Ax4fOn38yOMbNPmNktZvZ9M+sF3hzu+37ONV9oZv8dPqN2mNmbw/16horMkhI5kZCZXQh8DngNcDSwDbg5PPwy4MXACUAj8FqgKzx2PfAOd68DTgPunKeQJrynu18H3AR80d1r3f2VZlYO/CfwK2AJ8B7gJjM7Med6rwM+A9QBXw7jvzjn+BsIHvAiIlK67gPqw8SljODZ8v0xbb5A8Ow5CzgOaAc+5u4DwMuB3eHzp9bdd4fnXAHcQvC8uin3Yma2giAB/DLQFl53NOnRM1RklpTIiRzyeuAGd3/Q3UeAjwAvCOcMpAg+vE8CzN2fdPc94Xkp4BQzq3f3bnd/cIp7PD/8tu/gC1gxSdup7jnuukAt8Hl3T7r7ncDPyBk+A/yHu//e3bPuPgx8l+DBg5k1A5cAP5gidhERKQ2jvXIXA08Bu0YPmJkBbwfe7+773b0P+Cxw9TTXvNfdbw2fMUNjjr0e+I27/9DdU+7e5e6jiZyeoSKzpERO5JBjCHrhAHD3foJv3NrDD/WvAF8F9prZdWZWHza9CrgM2GZmvzWzF0xxj/vcvTH3BWyfqOE095wo9h3uns3Zt43gW9RRO8ac833glWZWS9AL+f+meMiJiEjp+B5BD9ObGTOskqDHrBp4ICdZ+kW4fypjnzG5lgObJjmmZ6jILCmREzlkN7By9E04F6CF8JtKd/8Xd38ucCrBUI2/Dvevc/crCIZj3Ar8aL4CmuyegE8Q+3Izy/1vegU537KOPcfddwH3AlcCb0RDQkREjgjuvo2g6MllwE/GHO4EhoBTcxKmBnevHT19sstOccsdwLGTxKJnqMgsKZGTI1W5mSVyXnGCIRFvMbOzzKySYCjJ/e6+1czOMbPnhePoB4BhIGNmFRasW9Pg7imgF8jMR4CT3TM8vBdYk9P8/rDNh8ys3MwuAF7JoTl+k7kR+BBwOvDT+YhbRESKwtuAC8N5bweFvVLfAv7JzJYAmFm7mV0SNtkLtJhZwwzudRPwUjN7jZnFzawlfNbqGSoyB0rk5Eh1O8E3jqOvT7j7HcDfAj8G9hB8ezg6J6Ce4MHWTTDcogv4UnjsjcDWsFLXOwnHzM+Dqe55PcGcgh4zu9Xdk8DlBJPQO4GvAX/h7k9Nc4+fEvRC/nTsw1xEREqXu29y9/WTHP7fwEbgvvDZ9hvgxPC8p4AfApvDZ9AxedxrO0Hv3weA/QSFTs4MD+sZKjJL5j5VT7iIlDoz20RQMew3hY5FRESkmOgZKoWkHjmRI5iZXUUw7n++yj2LiIgcEfQMlUKLT99EREqRmd0NnAK8cUylLhEREZmCnqGyGGhopYiIiIiISJHR0EoREREREZEis6iGVra2tvqqVasKHYaIiETsgQce6HT36RYYlpCejyIiR458n5GLKpFbtWoV69dPVglXRERKhZltK3QMxUTPRxGRI0e+z0gNrRQRERERESkySuRERERERESKjBI5ERERERGRIqNETkREREREpMgokRMRERERESkySuRERERERESKjBI5ERERERGRIhPZOnJmdiLwbzm71gAfc/f/G9U9v/W7zSxvrubS046K6hYiIiIiIjLP9vUO88jOA2Sy2TldJ511NncMkM7M7Tqz9Z6Ljqe8bGH6yiJL5Nz9aeAsADMrA3YBP43qfgDfvXcr565qViInIiKRMrNLgX8GyoBvu/vnxxxfCdwAtAH7gTe4+87wWAZ4LGy63d0vX7DARWRWUpks7tFcO+vO1q4BhlMzSzwy2SybOgbIZCcPLJ11tnQMkJ5BcpTOOnc9tY/eodSM4pkLBwaTmXm9ptm8Xi5vf/mS4ygvW5h7RZbIjXERsMnd81qlfLZqK+P0jaSjvIWIiBzhwi8nvwpcDOwE1pnZbe6+IafZl4Ab3f27ZnYh8DngjeGxIXc/a0GDFpHDZLNOZ/8ImzsHGEplSGech3d0c++mLg6MSWCSmSw79g8VKNK5qyovI1E+sx6i09obOHFpXUQRTaytrpK1q5qprph7FrSypZrqioVKcwpnoX7Dq4EfTnTAzK4FrgVYsWLFnG5Snyinf1iJnIiIROpcYKO7bwYws5uBK4DcRO4U4P3h9l3ArQsaoYgctGP/ILc8sJMHt3cD8OyBYXb3DDEwpgeoLGactbyRk46uP2x/zIwrz15GZTy64XLtjVU0VJXP+LwVLdXUTJOwtNVVUhYrUPeURCryRM7MKoDLgY9MdNzdrwOuA1i7du2cOq1rE3H29Q3P5RIiIiLTaQd25LzfCTxvTJtHgKsIhl9eCdSZWYu7dwEJM1sPpIHPu/uESd58ftEpUmiDyTTpcAhgV3+Svb3R/Hst686WzgF6BlP86++30DucJpkOhhWe3t5AeZmxvLma849rZXVrDSuaq2msLsfMOLathrrEzJMpkUJZiB65lwMPuvveqG9UWxlnc4d65EREJFITfbU99ovIDwJfMbM3A78jmCc++oBa4e67zWwNcKeZPebum8ZdcB6/6BQZlc06uw8MsX3/ILt7htm+f/Cw490DSfYcOJRktdRU0DWQZGVLNWctb6S8bOKenb29I3QNJMGdrV2Dh8132tk9yFPP9kXzC03hpKPquOTUo2iqruDP1y5jZUvNgscgEqWFSOSuYZJhlfOtLhGnX3PkREQkWjuB5TnvlwG7cxu4+27gVQBmVgtc5e4Hco7h7pvN7G7gbGBcIicyH9KZLFmHkXSGezd18amfbWBn9+TzvaoryljZUoMR9G7dt7mLYxoT/O6ZDq6/Z0te9zy6IUFTdcXB9y21Ffyvlx5PbWXwz87ayjgrmqsn/kpkHixvqqYuEaehKuhpEylVkSZyZlZNMBn8HVHeZ1RtIk6v5siJiEi01gHHm9lqgp62q4HX5TYws1Zgv7tnCaYW3BDubwIG3X0kbHM+8MWFDF6K03Aqw2+e3MvWzgE2dwyQmqRSobuzu2eI3uE0WXe2dQ0eVtVweXMVf/dnp7GsqYrlTVWsbq3Na/7UcCrD1q6BSSs31leV095YNavfTURmJ9JEzt0HgZYo75GrrjJOMp1lJJ2hMr5AdT9FROSI4u5pM3s38EuC5QducPcnzOxTwHp3vw24APicmTnB0Mp3haefDHzTzLJAjGCO3IZxN5GS19k/wu6eIbZ0DtA9kOTWh3cznMrQVld5WLuRVDaorJhMHyzOsbS+csoCF801FQcrDl58ylLqE+WYwWnHNHDu6mYSs6iNnigv46Sj6qdvKCILpqTqco5OUB0YUSInIiLRcffbgdvH7PtYzvYtwC0TnPffwOmRByiLQjqT5ck9few+MMTunmA4Y3dYhKNvzAiiE5fWsaS+koExU0RiZlx4UhuJ8jIuOnkp565qpmoeyrOLSPErqURudOx133CK5pqKaVqLiIiI5KdnMEnv0KEkywmKerg7g8kMP/zDdrZ0Dow5JzXh3P3nrmziklOXsqK5mjVttSTiZSxrqiKmEvEiMgOllcglRhM5zZMTERGRmctknUzW+ePePjbs7qWtrpL/eHgXtz68e8rz2hurOHd1M7m1NWor45yzqpn2pipWt9RgBoZRXxVXEQ4RmbOSSuTqwkROlStFRERkKsl0lsd29dA/Esw7K48ZHf0jfPSnj9M35t8RZvD2F60eN0fsqIYEBmTced7qFioiXDBaRGSs0krkKoM5cuqRExERkbGGUxke2NbNPRs7+Y+HdrH7wPhFqU9cWscrzzya9qYqzljWyOO7DnDC0jpOPlqFPkRkcSmpRK72YI9cqsCRiIiISKE9sqOHWx/excM7etjZPURn/wjuEI8Z565u5qOvOIWl9QkABkbS9A2n+ZMT2w7OuQc4tq22UOGLiEyppBK5g0Mr1SMnIiJyxBpOZfjgvz/Czx7dQ2U8xlnLG3nRca0sa67mrOUNnLu65bBkTUSkGJXUp9joh7IWBRcRESkNI+kMmzsGGElnefrZXu7Z2MXecEhkOptlb+8Iz1/TQtfACCubq9nRPcTTzwYl/9970fFc++I1StpEpCSV1CdbZTxGeZmp2ImIiEiR6htO8ejOAzy8o4c7n9rHA9u6DzveVlfJsW01GEZZrIz2pirufGovTdUV/H5jJ43VFZzR3sDfvuJkLj3t6AL9FiIi0SupRM7MqEuUa2iliIhIEXnq2V4e3XGA2x7ZzT0bOw/uX9FczV9ecCwnLK2jLhFneXM1xy+pnbR0/3AqQzxmxMtUPVJESl9JJXIQDK/sG1axExERkcUsncnyL3du5MZ7t9IzGDy3W2sree9Fx3NsWw0XnLiEhqryGV0zUV4WQaQiIotTSSZyGlopIiKyeGSzzrfv2cyD23pY1VpDR98Ij+7s4Zl9/Zx/XAsnLq3ntecs57gltZTFtFC2iEg+Si6Rq0vEtY6ciIjIIjGUzPCeHz7Ib57cx1H1CX654VmaqytY2VLNV1/3HP70DM1jExGZjZJM5Hb3jF/gU0RERBbW9q5BPvDvD7N+WzefeOUpvOm8VZPObxMRkZkpuUROQytFREQK78Bgird+dx17e4f5x9ecyZVnLyt0SCIiJaXkErm6RLkSORERkQLZuK+fv731cTbs6WUwmea7bzmX845rLXRYIiIlp+QSudpEXMsPiIiIFEA26/z1LY/w0PYeXnV2O39x3irOWt5Y6LBEREpS6SVylXGSmSzDqYzKEIuIiCygH67bzkPbe/iHPz+Tq56roZQiIlEquRUz6xNBbqrhlSIiIgvD3bnnmU4+d/tTvGBNC696TnuhQxIRKXml1yM3msgNp2mtrSxwNCIiIqVtW9cAH7rlUe7fsp+6RJxPX3maKlOKiCyA0kvkKssBtJaciIhIxPpH0rz9xvU8e2CYT15+Klc9dxm1lSX3TwsRkUWp5D5t68Ieub6RVIEjERERKV3/vbGTv7n1cbbvH+Q7bzmHFx3fVuiQRESOKJHOkTOzRjO7xcyeMrMnzewFUd4POPhNoCpXioiIRGNL5wDv/P4DjKQyfO31z1ESJyJSAFH3yP0z8At3f7WZVQDVEd/vUI+cEjkREZF5l85keet31hEvi3HztS9gRUvkj3YREZlAZD1yZlYPvBi4HsDdk+7eE9X9RtUlgjlyqlopIiJRMrNLzexpM9toZh+e4PhKM7vDzB41s7vNbFnOsTeZ2TPh600LG/nc3PnUPrZ0DvCZPztNSZyISAFFObRyDdAB/KuZPWRm3zazmrGNzOxaM1tvZus7OjrmfNOaymDtOCVyIiISFTMrA74KvBw4BbjGzE4Z0+xLwI3ufgbwKeBz4bnNwMeB5wHnAh83s6aFin0uOvpG+OR/bqC9sYqLT1la6HBERI5oUSZyceA5wNfd/WxgABj3jaW7X+fua919bVvb3MfYV8bLqIjH6B1WsRMREYnMucBGd9/s7kngZuCKMW1OAe4It+/KOX4J8Gt33+/u3cCvgUsXIOY5+/5929h9YIhvvOG5xMtKbilaEZGiEuWn8E5gp7vfH76/hSCxi1xdZVzFTkREJErtwI6c9zvDfbkeAa4Kt68E6sysJc9z533Eylwl01lueWAn5x/byunLGgodjojIES+yRM7dnwV2mNmJ4a6LgA1R3S9XXSKuoZUiIhKliVa89jHvPwj8iZk9BPwJsAtI53nuvI9Ymasv3/kMu3qGeOsLVxU6FBERIfqqle8BbgorVm4G3hLx/QCoTcTpHdLQShERicxOYHnO+2XA7twG7r4beBWAmdUCV7n7ATPbCVww5ty7owx2rh7fdYCv3b2JVz2nnQtP0tw4EZHFINJEzt0fBtZGeY+JNFSV06uhlSIiEp11wPFmtpqgp+1q4HW5DcysFdjv7lngI8AN4aFfAp/NKXDysvD4ovXN322muqKMj7/y1EKHIiIioZKcqVyfKFePnIiIRMbd08C7CZKyJ4EfufsTZvYpM7s8bHYB8LSZ/RFYCnwmPHc/8HcEyeA64FPhvkVp/0CSXzy+hz9/7nIaqsoLHY6IiISiHlpZEA1V5RxQIiciIhFy99uB28fs+1jO9i0Ehb4mOvcGDvXQLWp3PLmXVMa58uxx9VhERKSASrJHTomciIjI3G3pHOCztz/J0Q0JTmuvL3Q4IiKSoyQTufqqckbSWYZTmUKHIiIiUrS+fMczdA+m+NgrTsFsomKbIiJSKCWbyAFaFFxERGSWUpksdzy1j1c9p52Xn350ocMREZExSjORSwRT/1TwREREZHZ++tAuDgyluOw0JXEiIotRSSZyo1W1DgxpCQIREZGZ6htO8cVfPM3ZKxq56OQlhQ5HREQmUNKJnHrkREREZu7Ge7fR2T/Cx195qubGiYgsUiWZyGmOnIiIyOzd80wnp7XXc9byxkKHIiIikyjJRO7Q0EolciIiIjORyTqP7OzhOSuaCh2KiIhMYdpEzszWm9m7zKxoPtHrE2EiN6hETkREZCYe2t7NYDKjRE5EZJHLp0fuauAYYJ2Z3Wxml9giHzBfEY9RVV6moZUiIiIz4O78zU8fp62ukgtObCt0OCIiMoVpEzl33+jufwOcAPwAuAHYbmafNLPmqAOcrYaqcg2tFBERmYEndvfy9N4+PnDxCTRWVxQ6HBERmUJec+TM7AzgH4C/B34MvBroBe6MLrS5qa+K06vlB0RERPL26w17iRlcfMrSQociIiLTiE/XwMweAHqA64EPu/tIeOh+Mzs/yuDmQj1yIiIiM3Pv5i5Ob2+gpbay0KGIiMg0pk3kgD93980THXD3V81zPPOmPlHOngPDhQ5DRESkKKQzWR7d2cM1564odCgiIpKHfIZWHjCzfzGzB83sATP7ZzNriTyyOWqoKlexExERkTw99Wwfw6ksZ6tapYhIUcgnkbsZ6ACuIpgb1wH8W5RBzYd6Da0UERHJ24bdvQCcuayhwJGIiEg+8hla2ezuf5fz/tNm9mdRBTRf6qvK6RtOk8k6ZbFFvVqCiIhIwW3q6KciHmNZU3WhQxERkTzk0yN3l5ldbWax8PUa4OdRBzZXDVXBouB9Gl4pIiIyrU0d/axprdGXnyIiRSKfRO4dBOvHJcPXzcBfmVmfmfVGGdxcNIaJXM+gEjkREZHpbOoY4Ni22kKHISIiecpnQfA6d4+5ezx8xcJ9de5evxBBzkZTTZDIdQ8mCxyJiIjI4jaSzrB9/yDHttUUOhQREclTPnPkMLPLgReHb+9295/led5WoA/IAGl3XzubIGejsboCUI+ciIjIdLZ3DZLJOscuUY+ciEixyGdB8M8D5wA3hbveZ2YvdPcP53mPl7h752wDnK3mMJFTj5yIiMjUNnX0A2hopYhIEcmnR+4y4Cx3zwKY2XeBh4B8E7mCaAoTuf0DSuRERESmsqljAIDVrRpaKSJSLPIpdgLQmLM9kwVmHPhVuJD4tRM1MLNrzWy9ma3v6OiYwaWnVpeIEzMNrRQRkWiY2aVm9rSZbTSzcV9umtkKM7vLzB4ys0fN7LJw/yozGzKzh8PXNxY++sNt2tfP0Q0JairzmnEhIiKLQD6f2J8DHjKzuwAjmCv3kTyvf7677zazJcCvzewpd/9dbgN3vw64DmDt2rWef+hTi8WMxuoKDa0UEZF5Z2ZlwFeBi4GdwDozu83dN+Q0+yjwI3f/upmdAtwOrAqPbXL3sxYy5qls6ujXsEoRkSIzZY+cmRlwD/B84Cfh6wXufnM+F3f33eHPfcBPgXPnFO0MNVaXq0dORESicC6w0d03u/vo0jxXjGnjwGh15wZg9wLGlzd3D5ce0LBKEZFiMmUi5+4O3Orue9z9Nnf/D3d/Np8Lm1mNmdWNbgMvAx6fc8Qz0KQeORERiUY7sCPn/c5wX65PAG8ws50EvXHvyTm2Ohxy+Vsze9FEN4hq6sFY+/pG6B9Jq2KliEiRyWeO3H1mds4srr0UuMfMHgH+APzc3X8xi+vMWlN1hYqdiIhIFGyCfWOnB1wDfMfdlxEUDvuemcWAPcAKdz8b+CvgB2Y2bl1Wd7/O3de6+9q2trZ5Dv+QTftUsVJEpBjlM0fuJcA7zGwbMEDw8HJ3P2Oqk9x9M3Dm3EOcvabqch7fpaGVIiIy73YCy3PeL2P80Mm3AZcCuPu9ZpYAWsPpBiPh/gfMbBNwArA+8qgnsLkzqFi5RkMrRUSKSj6J3MsjjyIiTTUaWikiIpFYBxxvZquBXcDVwOvGtNkOXAR8x8xOBhJAh5m1AfvdPWNma4Djgc0LF/rhdnQPUlEWY2ldolAhiIjILOQztPLT7r4t9wV8OurA5kNjdTkj6SxDyUyhQxERkRLi7mng3cAvgScJqlM+YWafMrPLw2YfAN4eTjH4IfDmcO75i4FHw/23AO909/0L/1sEdnYP0d5URSw20WhRERFZrPLpkTs1901Ycvm50YQzv0YXBe8eTFJVUVXgaEREpJS4++0ERUxy930sZ3sDcP4E5/0Y+HHkAeZpV/cQy5r0jBQRKTaT9siZ2UfMrA84w8x6w1cfsA/4jwWLcA6aqssBVPBERERkEjuVyImIFKVJEzl3/5y71wF/7+714avO3VvcPd8FwQtqtEdOa8mJiIiMN5zK0Nk/QnujEjkRkWIz7dBKd/+ImbUDK3Pbu/vvogxsPjTVHBpaKSIiIofbvn8QgOXN1QWOREREZmraRM7MPk9QjWsDMFo1xIFFn8g1hkMre5TIiYiIjLO5I1hDbk2r1pATESk2+RQ7uRI40d1Hog5mvo0OrezSHDkREZFxRteQW6015EREik4+yw9sBsqjDiQK5WUxGqrKVexERERkAls6BlhSV0ltZT7f64qIyGKSzyf3IPCwmd0BHOyVc/f3RhbVPGqtraCzv+g6E0VEZAGY2buBm9y9u9CxFMLmzgFWt6o3TkSkGOWTyN0WvopSS20lnf3qkRMRkQkdBawzsweBG4Bfhot2HxG2dA5wyalHFToMERGZhUkTOTOrd/ded//uBMdWRBvW/GmtreDpZ/sKHYaIiCxC7v5RM/tb4GXAW4CvmNmPgOvdfVNho4tWz2CS/QNJ1qhHTkSkKE01R+7u0Y1wWGWuWyOJJgKt6pETEZEphD1wz4avNNAE3GJmXyxoYBE7WOhEiZyISFGaamil5Ww3T3FsUWupqeTAUIpkOktFPJ/aLiIicqQws/cCbwI6gW8Df+3uKTOLAc8AHypkfFHa0hEkcmtUsVJEpChNlcj5JNsTvV+0WmoPLQq+tD5R4GhERGSRaQVe5e7bcne6e9bMXlGgmBbEju5gMfBlTVoMXESkGE2VyC0xs78i6H0b3SZ83xZ5ZPOktbYSgM7+ESVyIiIy1u3A/tE3ZlYHnOLu97v7k4ULK3o9gynqEnGNVhERKVJTfXp/C6gDanO2R99/O/rQ5kdr2COneXIiIjKBrwP9Oe8Hwn0l78BQioaqolwmVkREmKJHzt0/uZCBRKUl7JHr0lpyIiIynuUuNxAOqTwiVsc+MJSisVqJnIhIsZrReIpwnZ2iMtoj16UeORERGW+zmb3XzMrD1/uAzYUOaiH0DCbVIyciUsRmOjC+aKpVjqqtDMb/d6pHTkRExnsncB6wC9gJPA+4tqARLRANrRQRKW4zHT7y80iiiJCZ0VpToTlyIiIyjrvvA64udByFcGAoTUNVRaHDEBGRWZo2kTOzGmDI3bPAjWZ2OfBf7p6KPLp50lpXSdeAeuRERORwZpYA3gacChwsbezuby1YUAvA3TkwpKGVIiLFLJ+hlb8DEmbWDtwBvAX4Tr43MLMyM3vIzH42uxDnrqWmQkMrRURkIt8DjgIuAX4LLAP6ChrRAhhKZUhlXMVORESKWD6JnLn7IPAq4MvufiVwygzu8T6goGvxtNVV0tGnRE5ERMY5zt3/Fhhw9+8CfwqcXuCYItczGAyqUY+ciEjxyiuRM7MXAK/n0By5vObWmdkygodiQdedW1qfoKNvhEzWp28sIiJHktFpAj1mdhrQAKwqXDgLQ4mciEjxyyeR+1/AR4CfuvsTZrYGuCvP6/9f4ENAdrIGZnatma03s/UdHR15XnZmltQnyLrWkhMRkXGuM7Mm4KPAbcAG4AuFDSl6+weCAmAtNSp2IiJSrKZN5Nz9t+5+ubt/wcxiQKe7v3e688zsFcA+d39gmutf5+5r3X1tW1tb/pHPwNK6YFHwvb1K5EREJBA+03rdvdvdf+fua9x9ibt/M8/zLzWzp81so5l9eILjK8zsrnCe+KNmdlnOsY+E5z1tZpfM46+Vl9F5463h81FERIrPtImcmf3AzOrD6pUbgKfN7K/zuPb5wOVmthW4GbjQzL4/p2hn6aiGoBDZs73Dhbi9iIgsQmE15newfjB9AAAgAElEQVTP5lwzKwO+CrycYN74NWY2dv74R4EfufvZBEscfC0895Tw/anApcDXwustmIOJXI0SORGRYpXP0MpT3L0X+DPgdmAF8MbpTnL3j7j7MndfRfDAutPd3zCXYGdraX2QyO1VIiciIof7tZl90MyWm1nz6CuP884FNrr7ZndPEnxhecWYNg7Uh9sNwO5w+wrgZncfcfctwMbwegumayBJPGbUV810OVkREVks8vkELzezcoJE7ivunjKzoqoa0lJTQcxgnxI5ERE53Oh6ce/K2efAmmnOawd25LzfCTxvTJtPAL8ys/cANcBLc869b8y57WNvYGbXAtcCrFixYppwZqazb4SW2grMbF6vKyIiCyefHrlvAlsJHkK/M7OVQO9MbuLud7v7K2Ye3vyIl8Vora3U0EoRETmMu6+e4DVdEgcwUQY09kvOa4DvuPsy4DLge+G8vHzOjXQOeddAktZaDasUESlm0/bIufu/AP+Ss2ubmb0kupCicVRDQsVORETkMGb2FxPtd/cbpzl1J7A85/0yDg2dHPU2gjlwuPu9ZpYAWvM8N1Jd/SO0KJETESlq+RQ7aTCzfxxdIsDM/oGgd66oLKlLaI6ciIiMdU7O60UEwyEvz+O8dcDxZrbazCoI5oLfNqbNduAiADM7GUgAHWG7q82s0sxWA8cDf5j7r5K/roEkzdVaQ05EpJjlM0fuBuBx4DXh+zcC/wq8KqqgorC0vpIHt3cXOgwREVlE3P09ue/NrAH4Xh7npc3s3cAvgTLghnCt1U8B6939NuADwLfM7P0EQyff7O4OPGFmPyKoBJ0G3uXumXn9xaYxmMxQU6lCJyIixSyfT/Fj3f2qnPefNLOHowooKkvrE+wfSDKSzlAZX9AqzyIiUjwGCXrIpuXutxNUc87d97Gc7Q0ES/FMdO5ngM/MPsy5GUymqa7Qs1BEpJjlk8gNmdkL3f0eADM7HxiKNqz5t7Q+mAuwr3eE5c3VBY5GREQWAzP7Tw4VGokRrAn3o8JFFL1s1hlOZamqUI+ciEgxy+dT/J3AjeFwE4Bu4E3RhRSN3LXklMiJiEjoSznbaWCbu+8sVDALYTgdjOJUj5yISHGbMpELyySf6O5nmlk9QLg4eNE5prEKgF09Q6wtcCwiIrJobAf2uPswgJlVmdkqd99a2LCiM5gMErmqciVyIiLFbMqqle6eBd4dbvcWaxIH0J6TyImIiIT+HcjmvM+E+0rW0Ggipx45EZGils+C4L82sw+a2XIzax59RR7ZPKupjNNYXc6ubiVyIiJyUNzdk6Nvwu2KAsYTudEeOQ2tFBEpbvnMkXtr+PNdOfscWDP/4USrvbGK3eqRExGRQzrM7PJwuQDM7Aqgs8AxRWoopURORKQUTJvIufvqhQhkIbQ3VrG1a6DQYYiIyOLxTuAmM/tK+H4n8BcFjCdyg8k0AAnNkRMRKWqTJnJm9gbA3P17Y/a/HRhw9x9EHdx8a2+q4vcbO3F3zKzQ4YiISIG5+ybg+WZWS/DM6yt0TFEbOji0UssPiIgUs6nmyH0AuHWC/f8WHis67Y1VDCQzHBhKFToUERFZBMzss2bW6O797t5nZk1m9ulCxxUlzZETESkNUyVyZRN9MxlWriyPLqToLGsKKlfuVMETEREJvNzde0bfuHs3cFkB44nc6Bw5LT8gIlLcpkrkys2sZuxOM6ujSCt6tTcGC4FrCQIREQmVmVnl6BszqwIqp2hf9IbUIyciUhKmSuSuB24xs1WjO8Ltm8NjReeYxgSAKleKiMio7wN3mNnbzOxtwK+B7xY4pkgNah05EZGSMOlMZ3f/kpn1A78NJ4E7MAB83t2/vlABzqfmmgoS5TGtJSciIgC4+xfN7FHgpYABvwBWFjaqaA2NVq2MK5ETESlmU5ascvdvAN8olWpeZsbypmq27x8sdCgiIrJ4PAtkgdcAW4AfFzacaA2lMlSVlxGLqXqziEgxy6v2sLv3Rx3IQlnZUsO2LiVyIiJHMjM7AbgauAboIqjIbO7+koIGtgAGkxnNjxMRKQFTzZErSataqtm2f4Bs1gsdioiIFM5TwEXAK939he7+ZSBT4JgWxFAyo8XARURKwJGXyLXWMJzKsrdvuNChiIhI4VxFMKTyLjP7lpldRDBHruSpR05EpDRMm8iZ2Xoze5eZNS1EQFFb1RKsqLC1U8MrRUSOVO7+U3d/LXAScDfwfmCpmX3dzF5W0OAiNphSIiciUgry6ZG7GjgGWGdmN5vZJWY27beWZpYwsz+Y2SNm9oSZfXLO0c6DlS3BWnLbugYKHImIiBSauw+4+03u/gpgGfAw8OEChxWp4WRGSw+IiJSAaRM5d9/o7n8DnAD8ALgB2G5mnzSz5ilOHQEudPczgbOAS83s+fMR9Fwc01hFRVmMLUrkREQkh7vvd/dvuvuFhY4lSoOpNFWaIyciUvTymiNnZmcA/wD8PUFZ5lcDvcCdk53jgdFql+Xhq+AVRspixvLmKrZpaKWIiByBgjlyeRWtFhGRRWzaT3IzewDoAa4HPuzuI+Gh+83s/GnOLQMeAI4Dvuru988x3nmxqqWGreqRExGRI9CQhlaKiJSEKXvkzCwG/NjdL3L3H+QkcQC4+6umOt/dM+5+FsG8g3PN7LQJ7nFtWFBlfUdHxyx+hZlb1RokclqCQEREjjRDKnYiIlISpkzk3D0LXDrXm7h7D0FVsHHXcvfr3H2tu69ta2ub663ysqYtWIJg94GhBbmfiIiUFjO71MyeNrONZjauOIqZ/ZOZPRy+/mhmPTnHMjnHblvYyIOhleqRExEpfvkMkv+1mX0Q+Dfg4HhEd98/1Ulm1gak3L3HzKqAlwJfmEuw8+X4JXUAPLO3n2VN1QWORkREikk4beCrwMXAToKqzre5+4bRNu7+/pz27wHOzrnEUDhaZcFlsk4ynVWxExGREpBPIvfW8Oe7cvY5sGaa844Gvhs+8GLAj9z9ZzMPcf6dsLQWgD/u7eMlJy0pcDQiIlJkzgU2uvtmADO7GbgC2DBJ+2uAjy9QbFMaTKYBNLRSRKQETJvIufvq2VzY3R/l8G8gF43G6gra6ir5497+6RuLiIgcrh3YkfN+J/C8iRqa2UpgNYdXeU6Y2XogDXze3W+d5NxrgWsBVqxYMQ9hB/PjAKpUtVJEpOjl9UkeFik5BUiM7nP3G6MKaiGcsLSWZ/b1FToMEREpPjbBvsmqZ10N3OLumZx9K9x9t5mtAe40s8fcfdO4C7pfB1wHsHbt2nmpzjWUDMKo1tBKEZGiN+06cmb2ceDL4eslwBeByyOOK3LHL6njmb39qlwpIiIztRNYnvN+GbB7krZXAz/M3eHuu8OfmwkKgS3Y6JXB5GiPnBI5EZFil8+C4K8GLgKedfe3AGcClZFGtQBOWFrHUCrDrh5VrhQRkRlZBxxvZqvNrIIgWRtXfdLMTgSagHtz9jWZWWW43Qqcz+Rz6+adEjkRkdKRTyI3FC5DkDazemAf0xc6WfRGC55oeKWIiMyEu6eBdwO/BJ4kKOb1hJl9ysxyR6xcA9zs7rlDP04G1pvZI8BdBHPkFiyRG05paKWISKnIZ47cejNrBL4FPAD0A3+INKoFcMJRwRIET+7p48KTlhY4GhERKSbufjtw+5h9Hxvz/hMTnPffwOmRBjeF0R65ahU7EREpevlUrfzLcPMbZvYLoD6sSFnU6hPlrGyp5vFdBwodioiIyIIYXX6gqiKfATkiIrKY5Vu1sh1YOdrezF7s7r+LMrCFcFp7A4/s6Cl0GCIiIgtiKKnlB0RESsW0n+Rm9gXgtQSTsUfLJztQ/IncMQ38/NE9dA8kaaqpKHQ4IiIikRrSHDkRkZKRz1dyfwac6O4jUQez0E5vbwDgid29vPD41gJHIyIiEi1VrRQRKR35DJLfDJRHHUghnNZeD8BjmicnIiJHgKFkhphBZVxz5EREil0+PXKDwMNmdgdwsFfO3d8bWVQLpLG6guXNVSp4IiIiR4TBZIaq8jLMrNChiIjIHOWTyN3GBAudloozljXy8HYVPBERkdI3lEqr0ImISInIZ/mB7y5EIIVyzsomfv7oHnb1DNHeWFXocERERCIzlMxQrflxIiIlYdJEzsx+5O6vMbPHCKpUHsbdz4g0sgWydlUzAOu37qf9rPYCRyMiIhKdQSVyIiIlY6oeufeFP1+xEIEUyslH11NXGecPW/ZzhRI5EREpYUOpDAktPSAiUhImTeTcfU/4c9voPjNrBbrcfVwPXbEqixnPWdnE+q3dhQ5FREQkUuqRExEpHZPWHzaz55vZ3Wb2EzM728weBx4H9prZpQsXYvTOWdXE03v76BlMFjoUERGRyGiOnIhI6ZhqIZmvAJ8FfgjcCfwPdz8KeDHwuQWIbcG84NgWAH6/savAkYiIiERnKJVR1UoRkRIxVSIXd/dfufu/A8+6+30A7v7UwoS2cM5c1kh9Is7dT+8rdCgiIiKRGUymqSrXYuAiIqVgqk/zbM720JhjJTNHDiBeFuNFJ7Tx2z92UELT/0RERA4TzJFTj5yISCmYKpE708x6zawPOCPcHn1/+gLFt2AuOKGNfX0jbNjTW+hQREREIjGcylClOXIiIiVhqqqVR9Qn/Z+c0AbA3U93cOoxDQWORkREZH6lMllSGadayw+IiJSEyAbKm9lyM7vLzJ40syfM7H3Tn1U4S+oTnLm8kf96fE+hQxEREZl3g8kMgHrkRERKRJQzntPAB9z9ZOD5wLvM7JQI7zdnrzzjaB7f1cuWzoFChyIiIjKvhpTIiYiUlMgSOXff4+4Phtt9wJNAe1T3mw9/esbRAPzskd0FjkRERGR+DaWCRE7ryImIlIYFqUFsZquAs4H7F+J+s3V0QxXnrGritkd2q3qliIiUlMFkGoCqclWtFBEpBZEncmZWC/wY+F/uPq4kpJlda2brzWx9R0dH1OFM68qzl/HMvn4e3N5T6FBERETmzejQSvXIiYiUhkgTOTMrJ0jibnL3n0zUxt2vc/e17r62ra0tynDycsVZx1BbGeem+7cVOhQREZF5o2InIiKlJcqqlQZcDzzp7v8Y1X3mW01lnCvPbudnj+6heyBZ6HBERGSRMrNLzexpM9toZh+e4Pg/mdnD4euPZtaTc+xNZvZM+HrTQsQ7OkeuSssPiIiUhCh75M4H3ghcmPMguyzC+82bNzx/Jcl0Vr1yIiIyITMrA74KvBw4BbhmbGVmd3+/u5/l7mcBXwZ+Ep7bDHwceB5wLvBxM2uKOmYNrRQRKS1RVq28x93N3c8YfZC5++1R3W8+nXhUHS85sY3r79lycHK4iIhIjnOBje6+2d2TwM3AFVO0vwb4Ybh9CfBrd9/v7t3Ar4FLI42WQ0MrqytU7EREpBQsSNXKYvTuC4+jezDFD+7fXuhQRERk8WkHduS838kkS+yY2UpgNXDnTM6d72Jgh6pWqkdORKQUKJGbxHNXNnPesS18/e5N9A2nCh2OiIgsLjbBvsnWrbkauMXdMzM5d76LgWlBcBGR0qJEbgoffvlJdA0k+drdmwodioiILC47geU575cBuydpezWHhlXO9Nx5M5TKEI8ZFXE9+kVESoE+zadwxrJGrjy7nevv2cLWzoFChyMiIovHOuB4M1ttZhUEydptYxuZ2YlAE3Bvzu5fAi8zs6awyMnLwn2RGkxm1BsnIlJClMhN48MvP4nKeIz//eNHyWYnGzUjIiJHEndPA+8mSMCeBH7k7k+Y2afM7PKcptcAN7u755y7H/g7gmRwHfCpcF+khpIZVawUESkhKl01jaX1Cf7mspP58E8e43v3beNN560qdEgiIrIIhJWYbx+z72Nj3n9iknNvAG6ILLgJDKUyJFToRESkZKhHLg+vPWc5Lzmxjc/8/Eke2dEz/QkiIiKLTCqTpVLz40RESoY+0fNgZvzja86ira6Sv7zpQboHkoUOSUREZEZSmSzlZXrsi4iUCn2i56mppoKvv+E5dPSN8I7vPXCwjLOIiEgxSGZciZyISAnRJ/oMnLGskX987Zms27af/3nTAyTT2UKHJCIikpdUOkuFEjkRkZKhT/QZesUZx/DZK0/n7qc7eNcPHmQ4pZ45ERFZ/FKZLOXxidYiFxGRYqREbhauOXcFn7z8VH7z5F5e96372K85cyIisshpjpyISGnRJ/osvem8VXztdc/h8d29vPLL97Bua+RLAImIiMya5siJiJQWfaLPwctPP5ofveMFlMWM137zXr7wi6cYGEkXOiwREZFxUhnNkRMRKSX6RJ+js5Y3cvv7XsSrn7uMr9+9iQu+dDc/uH+75s6JiMiiEgyt1Bw5EZFSoURuHtRWxvniq8/kJ395Hsubqvg/P32M8z9/J//wq6fZ0jlQ6PBERERIpTVHTkSklMQLHUApec6KJn78P8/j3k1dXH/PFr5y10a+fOdGzlzWwCvOOIaLTl7CmrbaQocpIiJHoGTGKY8rkRMRKRVK5OaZmXHeca2cd1wrzx4Y5rZHdnHrQ7v5zO1P8pnbn2R1aw0XnrSEi05awnNXNVEZLyt0yCIicgTQHDkRkdKiRC5CRzUkuPbFx3Lti49lZ/cgdz21jzue2sf37tvG9fdsoTIe46zljZy7uplzVjVz9opG6hLlhQ5bRERKkObIiYiUFiVyC2RZUzVvfMEq3viCVQwm0/x+Yxf3be5i3db9fO3uTWSyGwFY3VrDae0NnN5ez2nHNHBqewMNVUruRERkbrSOnIhIaVEiVwDVFXEuPmUpF5+yFID+kTQPbe/mkR09PLbrAA9u6+Y/H9l9sP3KlmpOOqqOY9tqOW5JLce21bKmrUa9dyIikhd3J6V15ERESooSuUWgtjLOi45v40XHtx3c19U/whO7e3ls1wEe33WAP+7t444n95HO+sE2R9UnWNNWw6rWGla1VLOypYbVrTWsaK4mUa65dyIiEkhlgmdHhYqdiIiUjMgSOTO7AXgFsM/dT4vqPqWqpbaSF5/QxotPOJTcpTJZtnUNsqmjn437+tnU0c/mjgH+67E9dA+mDjv/mIYEK1tqWNVazaqWmsOSvKoKJXkiIkeSVCYLoDlyIiIlJMoeue8AXwFujPAeR5TyshjHLQmGV15y6uHHDgym2No1ELw6B9nWNcCWrgF++cRe9g8kD2t7VH1iTIIX9OataqlRkiciUoIOJXLqkRMRKRWRJXLu/jszWxXV9eVwDdXlnFndyJnLG8cdOzCUYlvXAFu7BtnaGSR727oG+fWGvXSNSfKW1lcGyV1LDSvDZG9FczVL6itpqamkLKZvc0VEik1SiZyISMnRHLkjQENVOWcsa+SMZeOTvN7hFNs6B8PkboAtYW/eHU/tpbP/8CSvLGa01lawtD7BkrpKltQnWFqXYEl9JUvrK1kSbivhExFZXA7OkVMiJyJSMgqeyJnZtcC1ACtWrChwNEee+kQ5py9r4PRlDeOO9Q2n2NY1yM7uITr6htnbO8Le3mH29Y2ws3uIB7f3jBu2CUHC11RdTnNNBU3VFbTUBj+baw5/NVVX0FRTQX0iTm1lHDMlfyIiUUilwx65uD5nRURKRcETOXe/DrgOYO3atT5Nc1lAdYlyTmtv4LT28UneqGQ6S0d/mOCFSd7e3mH2D6TYPzBC90CKp5/to3swRfdgEp/kf+GymFGfiNNQVU5DVTn14ash51WfOPx90C5OXaJcPYAiIlPQHDkRkdJT8EROiltFPEZ7YxXtjVXTts1knQNDKfYPJOkeTNLVn+TAUJIDQ6mDr96h9MHtXd1DB7dzl10YyyxYwiE32auviudslwdJYnX5mH3BdqI8pt5AESlpmiMnIlJ6olx+4IfABUCrme0EPu7u10d1P1n8ymJ2cFjlTLg7Q6nMoYRvMEXvcHpMAhj87BsOfm7tHKQ33B5MZqa8fnmZHUzq6qrKqakoo7qijKqKODUVZVSF76sr4lSVjx4ro6YifnC7OtyujMeoLC8jUR6jokwJokipMrNLgX8GyoBvu/vnJ2jzGuATgAOPuPvrwv0Z4LGw2XZ3vzzqeDVHTkSk9ERZtfKaqK4tRxYzCxOlOEc3TN/zN1Yqk6UvTPx6h1IHE7zeoXTO9qHkcHAkTc9giqFUhoGRNEPJDIOpDJkpegUnjhsq4zES5WUk4kFyVzn6s7zs0LHyMhLxGJXlsbBd7rHgnMqccxNjzq1LxGmuriCm4aUiC8LMyoCvAhcDO4F1Znabu2/IaXM88BHgfHfvNrMlOZcYcvezFjJmDa0UESk9GlopJa+8LDarnsBc7k4ykw2SumSGwWQ6/Jk5bN9wOstIKsNIOstwKsPwYdtZRtLBz+FUht7hNB19IxO2nWHOSEVZjCX1lRzdkGBpfSLnZxVHNSRob6xiSV2lkj2R+XEusNHdNwOY2c3AFcCGnDZvB77q7t0A7r5vwaPMkUxrQXARkVKjRE4kD2YW9IzFy2isjvZe7k4665MmgSPpDCNhMjiSztIzmOTZsKLongNDPL7rAL95ci/Dqexh1y2LGXWJODUV8eBnZZzKeIyKeIzysmAoaHmZUV4WozwevA+Ohfty2lTEy8KfsYPHRt9XlB3aVxEPzo2XxSgzIxaDMjPisViwHTNiZpTFLDyuf2RKUWgHduS83wk8b0ybEwDM7PcEwy8/4e6/CI8lzGw9kAY+7+63TnST+azqfHCOXFw9ciIipUKJnMgiY2YHk6e6WV7DPSgs82zvMHt6htnVM8SeA0P0D6fpG0nTP5ymfyRNMp1lYCRNMuOkMlmS6SypTDZnO9g/VbGZ+WTGwYQuSPjC7YMJH4eSwJx2ZbFDr1ju+3A7aAdlsRhlYxPIce0mul54brg9s/vmtmOG9835G0yTCMdjpjmZC2eiP/TY/0jiwPEEc8WXAf/PzE5z9x5ghbvvNrM1wJ1m9pi7bxp3wXms6jy6/IDmyImIlA4lciIlyMxorK6gsbqCk46qn/P1stlgaGmQ5I1N+pxkOptzfDQRDM5JprNkslkyWci4k806mdGXBz+zOduj+4N2BOd6sD15uzHXC3+ms1lG0k7GGddu3LVyzju8HQfbLXYTJcJmwf8fzILsIxZuw/h9BgeTwVgMjMP3j26/5fzVvOH5Kwv1ay4GO4HlOe+XAbsnaHOfu6eALWb2NEFit87ddwO4+2Yzuxs4GxiXyM2n0WInmiMnIlI6lMiJyLRiMSMRC4qrHKncnaxzeMLnTiYzXWLIpIlmfgkps06E01nHPYjdIdgm+D2CNR1Hj0N2TBvG7Qve49Ayh/mmJWIdcLyZrQZ2AVcDrxvT5lbgGuA7ZtZKMNRys5k1AYPuPhLuPx/4YtQBL62v5LLTj6KhqjzqW4mIyAJRIicikgez0eGZGr54pHP3tJm9G/glwfy3G9z9CTP7FLDe3W8Lj73MzDYAGeCv3b3LzM4DvmlmWSBGMEduwyS3mjdrVzWzdlVz1LcREZEFpERORERkhtz9duD2Mfs+lrPtwF+Fr9w2/w2cvhAxiohIadNgeRERERERkSKjRE5ERERERKTIKJETEREREREpMkrkREREREREiowSORERERERkSKjRE5ERERERKTIKJETEREREREpMhYsdbM4mFkHsG2Ol2kFOuchnFKiv8l4+ptMTH+X8fQ3GW8+/iYr3b1tPoI5Euj5GCn9XcbT32Q8/U0mpr/LeAv2jFxUidx8MLP17r620HEsJvqbjKe/ycT0dxlPf5Px9DcpTvrf7f+3d/+hkpV1HMffn9zUdEVd0xCVtk0JFfRqYdpamP3AJMJio8xMQvAf/9AISin6RX8UVEogZVRoJCVli7JEaqsJ/ZFr6qrrr1xDaFFaIl2xUPzx7Y957nZ3Z3bN7s6dOXPeLzjMnGeee+5zPjPnfu8z58y9o5nLMDMZZiajmcuwpczESyslSZIkqWOcyEmSJElSx8ziRO5Hkx7AFDKTYWYymrkMM5NhZtJNPm+jmcswMxlmJqOZy7Aly2TmPiMnSZIkSbNuFs/ISZIkSdJMcyInSZIkSR0zMxO5JGcleTTJ5iSXTXo8SynJT5NsTbJpQduKJLcmeazdHtzak+T7Laf7k5w8uZGPT5Kjktye5OEkDya5pLX3Npck+ybZkOS+lsnXW/tbktzZMrk+yd6tfZ+2vrk9vnKS4x+nJHsluTfJurZuJskTSR5IsjHJn1tbb4+frutrjbQ+DrM+jmaN3DVr5I6mqT7OxEQuyV7AVcCHgOOAc5McN9lRLalrgLN2arsMWF9VxwDr2zoMMjqmLRcBP1iiMS61l4DPV9WxwKnAxe010edcXgDOrKoTgTngrCSnAt8GrmiZPA1c2PpfCDxdVUcDV7R+s+oS4OEF62Yy8N6qmlvw/3D6fPx0Vs9r5DVYH3dmfRzNGrlr1shh01Efq6rzC3AacPOC9cuByyc9riXOYCWwacH6o8Dh7f7hwKPt/tXAuaP6zfIC3Ah8wFy2799+wD3AO4F/AMta+/ZjCbgZOK3dX9b6ZdJjH0MWRzL4oXsmsA5I3zNp+/cE8Mad2jx+Orj0vUZaH181H+vjcCbWyP9mYY0czmRq6uNMnJEDjgD+tmB9S2vrszdV1VMA7faw1t67rNqp/ZOAO+l5Lu3yiI3AVuBW4HHgmap6qXVZuN/bM2mPbwMOWdoRL4krgS8Ar7T1QzATgAJuSXJ3kotaW6+Pnw7z+dmRr+PG+rgja+RI1shhU1Mfl+2pDU1YRrT5fxVG61VWSZYDNwCXVtWzyajdH3Qd0TZzuVTVy8BckoOAtcCxo7q125nPJMmHga1VdXeSM+abR3TtTSYLrK6qJ5McBtya5JHd9O1TLl3k8/O/6VVO1sdh1sgdWSN3aWrq46yckdsCHLVg/UjgyQmNZVr8PcnhAO12a2vvTVZJXs+gSF1XVb9pzb3PBaCqngH+wODzEQclmX9TZ+F+b8+kPX4g8M+lHfAqQ4UAAAO6SURBVOnYrQY+kuQJ4JcMLh25kn5nAkBVPdlutzL4heYUPH66yudnR71/HVsfd88auZ01coRpqo+zMpG7Czim/RWdvYFPAjdNeEyTdhNwQbt/AYNr4OfbP9P+is6pwLb5U8GzJIO3Fn8CPFxV31vwUG9zSXJoe5eRJG8A3s/gw8u3A2tat50zmc9qDXBbtQu8Z0VVXV5VR1bVSgY/N26rqvPocSYASfZPcsD8feCDwCZ6fPx0nDVyR71+HVsfR7NGDrNGDpu6+jjpDwzuqQU4G/gLg+uZvzTp8Szxvv8CeAp4kcHM/0IG1ySvBx5rtyta3zD462WPAw8A75j0+MeUyekMTl3fD2xsy9l9zgU4Abi3ZbIJ+EprXwVsADYDvwL2ae37tvXN7fFVk96HMedzBrDOTLbv/31teXD+Z2qfj5+uL32tkdbHkZlYH0fnYo3cfT7WyJq++pj2TSRJkiRJHTErl1ZKkiRJUm84kZMkSZKkjnEiJ0mSJEkd40ROkiRJkjrGiZwkSZIkdYwTOWlKJTkjybpJj0OSpGlifZQGnMhJkiRJUsc4kZMWKcmnk2xIsjHJ1Un2SvJcku8muSfJ+iSHtr5zSf6U5P4ka5Mc3NqPTvL7JPe1r3lr2/zyJL9O8kiS65Kk9f9Wkofadr4zoV2XJGmXrI/SeDmRkxYhybHAJ4DVVTUHvAycB+wP3FNVJwN3AF9tX/Iz4ItVdQLwwIL264CrqupE4F3AU639JOBS4DhgFbA6yQrgo8DxbTvfHO9eSpL02lgfpfFzIictzvuAtwN3JdnY1lcBrwDXtz4/B05PciBwUFXd0dqvBd6T5ADgiKpaC1BVz1fVv1ufDVW1papeATYCK4FngeeBHyf5GDDfV5KkaWF9lMbMiZy0OAGuraq5trytqr42ol+9yjZ25YUF918GllXVS8ApwA3AOcDvXuOYJUkaN+ujNGZO5KTFWQ+sSXIYQJIVSd7M4Nha0/p8CvhjVW0Dnk7y7tZ+PnBHVT0LbElyTtvGPkn229U3TLIcOLCqfsvgspK5ceyYJEmLYH2UxmzZpAcgdVlVPZTky8AtSV4HvAhcDPwLOD7J3cA2Bp8TALgA+GErRH8FPtvazweuTvKNto2P7+bbHgDcmGRfBu9Wfm4P75YkSYtifZTGL1W7O6Mt6f+R5LmqWj7pcUiSNE2sj9Ke46WVkiRJktQxnpGTJEmSpI7xjJwkSZIkdYwTOUmSJEnqGCdykiRJktQxTuQkSZIkqWOcyEmSJElSx/wHA9xzYhtRKs4AAAAASUVORK5CYII=\n",
297 |       "text/plain": [
298 |        "<Figure size 1080x288 with 2 Axes>"
299 |       ]
300 |      },
301 |      "metadata": {
302 |       "needs_background": "light"
303 |      },
304 |      "output_type": "display_data"
305 |     }
306 |    ],
307 |    "source": [
308 |     "loss_history, accuracy_history = train_model(model, X_train, y_train, \n",
309 |     "                                             bce_loss, optimizer, accuracy, n_epochs)"
310 |    ]
311 |   },
312 |   {
313 |    "cell_type": "markdown",
314 |    "metadata": {},
315 |    "source": [
316 |     "## 3. Save the model after training\n",
317 |     "\n",
318 |     "In TensorFlow 2.0 it's as simple as that:"
319 |    ]
320 |   },
321 |   {
322 |    "cell_type": "code",
323 |    "execution_count": 13,
324 |    "metadata": {},
325 |    "outputs": [],
326 |    "source": [
327 |     "# just specify model's name and directory\n",
328 |     "tf.saved_model.save(model, 'home/ ... /path/to/folder')"
329 |    ]
330 |   },
331 |   {
332 |    "cell_type": "markdown",
333 |    "metadata": {},
334 |    "source": [
335 |     "At this point, you'll find a saved_model.pb object in the directory your speficied. That file contains your Neural Network, with its architecture and its trained weights."
336 |    ]
337 |   },
338 |   {
339 |    "cell_type": "markdown",
340 |    "metadata": {},
341 |    "source": [
342 |     "## 4. Restore the model and keep training\n",
343 |     "\n",
344 |     "Let's say you have time to train your model a little more. Let's restore it and continue to train it."
345 |    ]
346 |   },
347 |   {
348 |    "cell_type": "code",
349 |    "execution_count": 9,
350 |    "metadata": {},
351 |    "outputs": [],
352 |    "source": [
353 |     "# you just need 'load' from the same directory you wrote above\n",
354 |     "restored_model = tf.saved_model.load('home/ ... /path/to/folder')"
355 |    ]
356 |   },
357 |   {
358 |    "cell_type": "markdown",
359 |    "metadata": {},
360 |    "source": [
361 |     "At this point, we can keep training our model for other 500 epochs:"
362 |    ]
363 |   },
364 |   {
365 |    "cell_type": "code",
366 |    "execution_count": 10,
367 |    "metadata": {},
368 |    "outputs": [
369 |     {
370 |      "name": "stdout",
371 |      "output_type": "stream",
372 |      "text": [
373 |       "100.\tTraining Loss: 0.6489710807800293,\tAccuracy: 0.9582609\n",
374 |       "200.\tTraining Loss: 0.6375355124473572,\tAccuracy: 0.9588406\n",
375 |       "300.\tTraining Loss: 0.6364323496818542,\tAccuracy: 0.9588406\n",
376 |       "400.\tTraining Loss: 0.6316857933998108,\tAccuracy: 0.9591304\n",
377 |       "500.\tTraining Loss: 0.6307226419448853,\tAccuracy: 0.9591304\n",
378 |       "\n",
379 |       "Training complete.\n"
380 |      ]
381 |     },
382 |     {
383 |      "data": {
384 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAEWCAYAAAAzYSkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XmYXGWZ9/Hvr9eQfWfJDiRAgAASEI3IJqsjYXEwoGhcYFQYHXhB4VUZxYWIODM68DoiZgQFAYNgdMImi6zBhF0SwoSwpBOWmITsW3ff7x/nVKfS6aWSdKVOdf8+11VX13nOdhd61cldz/PcjyICMzMzMzMz6zoqSh2AmZmZmZmZ7VxOBM3MzMzMzLoYJ4JmZmZmZmZdjBNBMzMzMzOzLsaJoJmZmZmZWRfjRNDMzMzMzKyLcSJo1olI+qSk+0odh5mZ2Y6Q9H8l3bCT7+lnqHUpTgTNtpGk1yV9pAT3nSzpsbbiiYibI+KEAq71K0nfK0acZmbWOaXPm42SBjZrf05SSBpZwDWOllTX3nER8YOI+ML2R7vVff0MNWvGiaCZbTNJlaWOwczMSuI14OzchqQDgV068gaSqjryelnjZ6hlhRNBsw4k6TxJ8yUtkzRd0h5puyT9u6R3Ja2Q9IKkA9J9p0iaI2mVpEWSLtmB+zf94tnaPSWdD3wS+Jqk1ZL+mB6/n6SHJb0n6SVJp+Zd91eSfiZphqQ1wMWS3sl/WEs6U9Jz2xu7mZmVhV8Dn87b/gxwU/4BkmolXSPpzfRZ8V+SdpHUA7gb2CN9/qyWtIekb0uaJuk3klYCk9O23+Rd80OSnkifUQslTU7b/Qw1205OBM06iKRjgauAs4DdgTeAW9PdJwAfBsYAfYFPAEvTfb8E/ikiegEHAA92UEgt3jMirgduBq6OiJ4R8TFJ1cAfgfuAwcA/AzdL2ifveucA3wd6Af+Zxn983v5PkfwDwczMOq+ZQO808akkebb8ptkxPyR59hwM7A0MAa6IiDXAycDi9PnTMyIWp+dMBKaRPK9uzr+YpOEkCeR/AoPS6+aSJj9DzbaTE0GzjvNJYGpEPBMRG4DLgQ+kcyY2kXz57wsoIuZGxFvpeZuAsZJ6R8TyiHimjXsckf7a2PQChrdybFv33Oq6QE9gSkRsjIgHgT+RN/wH+ENEPB4RjRGxHriR5MGFpP7AicAtbcRuZmadQ65X8HjgZWBRbockAecBF0XEsohYBfwAmNTONZ+MiLvSZ8y6Zvs+Cfw5In4bEZsiYmlE5BJBP0PNtpMTQbOOswdJLyAAEbGa5Be/IelD4VrgOuAdSddL6p0eeiZwCvCGpL9I+kAb95gZEX3zX8CbLR3Yzj1bin1hRDTmtb1B8ituzsJm5/wG+JikniS9oI+28ZA0M7PO49ckPVyTaTYslKTHrjvwdF6ydU/a3pbmz5h8w4BXW9nnZ6jZdnIiaNZxFgMjchvpXIgBpL+URsRPI+JQYH+SoSaXpu2zImIiyXCSu4DbOyqg1u4JRAuxD5OU/50wnLxfeZufExGLgCeB04Fz8ZAWM7MuISLeICkacwrw+2a7/w6sA/bPS7j6RETP3OmtXbaNWy4E9molFj9DzbaTE0Gz7VMtqVveq4pkSMdnJR0sqZZkKMxTEfG6pMMkvT+dR7AGWA80SKpRsm5Rn4jYBKwEGjoiwNbume5+B9gz7/Cn0mO+Jqla0tHAx9g8x7E1NwFfAw4E7uyIuM3MrCx8Hjg2nffXJO0V+wXw75IGA0gaIunE9JB3gAGS+mzDvW4GPiLpLElVkgakz1o/Q812gBNBs+0zg+QXz9zr2xHxAPAt4A7gLZJfL3NzInqTPBiXkwwXWQpck+47F3g9rZT2RdI5Ax2grXv+kmROxXuS7oqIjcCpJJP4/w78P+DTEfFyO/e4k6QX9M7m/xgwM7POKyJejYjZrez+OjAfmJk+2/4M7JOe9zLwW2BB+gzao4B7vUnS+/h/gGUkhWIOSnf7GWq2nRTRVk+8mVnbJL1KUrHtz6WOxczMrJz4GWql5B5BM9tuks4kmffQUeW6zczMugQ/Q63Uqto/xMxsa5IeBsYC5zarlGZmZmZt8DPUssBDQ83MzMzMzLoYDw01MzMzMzPrYjrN0NCBAwfGyJEjSx2GmZntBE8//fTfI6K9Baot5WekmVnXsC3Px06TCI4cOZLZs1urYmxmZp2JpDdKHUM58TPSzKxr2Jbno4eGmpmZmZmZdTFOBM3MzMzMzLoYJ4JmZmZmZmZdjBNBMzMzMzOzLsaJoJmZmZmZWRfjRNDMzMzMzKyLcSJoZmZmZmbWxRR1HUFJJwE/ASqBGyJiSrP9/w4ck252BwZHRN9032eAb6b7vhcRNxYzVoAf3zePsbv3ZsLogfTuVl3s25mZmZmZlY17/vYWcxavLHUYnVb32iq+eNReO+1+RUsEJVUC1wHHA3XALEnTI2JO7piIuCjv+H8GDknf9wf+FRgPBPB0eu7yYsW7Yu0mfvXE66xaX09VhXjfiH4cvc8gjh4zmP1274WkYt3azMzMzCzzvjbtBVaur8f/LC6OgT1rO0ciCBwOzI+IBQCSbgUmAnNaOf5skuQP4ETg/ohYlp57P3AS8NtiBdunezXPfOt4nn3zPR6e9y4Pz1vC1ffM4+p75rFr71qOGjOIY/YZzIfHDKJHbVE7Us3MzMzMMiUiWL2hnn8+dm/+zwn7lDoc6wDFzGiGAAvztuuA97d0oKQRwCjgwTbOHVKEGLdQXVnB4aP6c/io/nztpH15d+V6Hn5lCX+Zt4R7/vY2t8+uo6aqgiP3HsiJ++/GiQfsRp9dPITUzMzMzDq3dZsaaAzoXuMOkc6imMViWuo0jlaOnQRMi4iGbTlX0vmSZkuavWTJku0Ms3WDe3fjrPHDuO6T7+OZbx3PbecfwafeP4KX317F1+54gQlTHuS2WW92+H3NzKy8SDpJ0jxJ8yVd1sL+EZIekPSCpIclDc3b1yDpufQ1Pa/9WEnPSPqbpBslVaXtkvTT9F4vSHrfzvmUZtaVrdmQ/DO9Z21liSOxjlLMRLAOGJa3PRRY3Mqxk9hy2GdB50bE9RExPiLGDxo0aAfDbVtVZQXv33MAV3xsLI99/Rj+cMEEDhrWh6/f8SIPvvxOUe9tZmbZlTcn/mRgLHC2pLHNDrsGuCkixgFXAlfl7VsXEQenr1PTa1YANwKTIuIA4A3gM+nxJwOj09f5wM+K88nMzDZbs6EecI9gZ1LMRHAWMFrSKEk1JMne9OYHSdoH6Ac8mdd8L3CCpH6S+gEnpG2ZIImDhvXll585jJEDuvOLR14rdUhmZlY6TXPiI2IjkJsTn28s8ED6/qEW9jc3ANgQEa+k2/cDZ6bvJ5IklRERM4G+knbf0Q9hZtaWNRuTRNC1MjqPoiWCEVEPXEiSwM0Fbo+IlyRdKenUvEPPBm6NiMg7dxnwXZJkchZwZa5wTJZ0q67kqDGDeL7uPeobGksdjpmZlUYh89qfZ3MidzrQS9KAdLtbOs1hpqTT0ra/A9WSxqfbH2fzSJmSzKM3s65t89BQJ4KdRVH/l4yIGcCMZm1XNNv+divnTgWmFi24DnLoyP7c+OQbzH1rFQcO7VPqcMzMbOcrZF77JcC1kiYDjwCLgPp03/CIWCxpT+BBSS9GxKuSJgH/LqkWuC/v+ILn0ZMMHWX48OHb+JHMzLaU6xHs7jmCnUYxh4Z2Cfvv0RuA+UtWlTgSMzMrkXbntUfE4og4IyIOAb6Rtq3I7Uv/LgAeJl1TNyKejIgjI+JwkuTxfwu9X3r+TptHb2adX26OoHsEOw8ngjuoe03yq8iGTR4aambWRbU7J17SwLQADMDlpCNe0rnwtbljgAmk6+1KGpz+rQW+DvxXev504NNp9dAjgBUR8VYxP6CZ2dp0aGju375W/pzS76CayuS5vqHeiaCZWVcUEfWScnPiK4GpuTnxwOyImA4cDVwlKUh69y5IT98P+LmkRpIfZ6dExJx036WS/iFt/1lE5NbanQGcAswH1gKfLfqHNLMub7V7BDsd/y+5g2qrk19FNjoRNDPrstqbEx8R04BpLZz3BHBgK9e8FLi0hfZgcyJpZrZTrN3o5SM6Gw8N3UGbewQbShyJmZmZmVlxrN7QQE1lBTVVTh86C6f0O6i6UkjuETQzM7OtrVy/iXv/9jYNjVsVdjUrKy/UvUcPVwztVJwI7iBJ1FZVeI6gmZmZbeWOp+v4zh/ntH+gWRkY56XSOhUngh2gptKJoJmZmW1t7cZk6sgjlx5DdVVLS0CalY9+3WtKHYJ1ICeCHaC2utKJoJmZmW0lN3VkWP9dkJwImll2eLZnB0h6BF0sxszMzLa0qaExrSfgJNDMssWJYAeora5wsRgzMzPbysb6xqYK42ZmWeJvpg7gOYJmZmbWkk0NjVS73L6ZZZC/mTpAbXWlewTNzMxsKxsbgmr3CJpZBvmbqQPUeo6gmZmZtcBDQ80sq/zN1AE8R9DMzMxasqmhkRoPDTWzDPI3UwfwgvJmZmbWklzVUDOzrHEi2AFqnAiamZlZCzbWN3qOoJllkr+ZOkBtlYvFmJmZ2dY2NjgRNLNs8jdTB/CC8mZmZtYSzxE0s6zyN1MHcLEYMzMza8mmhnDVUDPLpKJ+M0k6SdI8SfMlXdbKMWdJmiPpJUm35LX/UNLf0tcnihnnjvKC8mZmXVt7zztJIyQ9IOkFSQ9LGpq3r0HSc+lrel77cZKeSdsfk7R32j5Z0pK8c76wcz6lbY9kjqCLxZhZ9lQV68KSKoHrgOOBOmCWpOkRMSfvmNHA5cCEiFguaXDa/lHgfcDBQC3wF0l3R8TKYsW7I9wjaGbWdRXyvAOuAW6KiBslHQtcBZyb7lsXEQe3cOmfARMjYq6kLwPfBCan+26LiAuL8HGsg23yHEEzy6hifjMdDsyPiAURsRG4FZjY7JjzgOsiYjlARLybto8F/hIR9RGxBngeOKmIse6QmspK6huD+gYng2ZmXVAhz7uxwAPp+4da2N+SAHqn7/sAizsgVtvJNnqOoJllVDG/mYYAC/O269K2fGOAMZIelzRTUi7Zex44WVJ3SQOBY4BhzW8g6XxJsyXNXrJkSRE+QmFqq5P/jBudCJqZdUWFPO+eB85M358O9JI0IN3ulj7LZko6Le+cLwAzJNWR9B5Oydt3ZjrMdJqkrZ6PkJ1nZFe3sb7RcwTNLJOK+c3U0oD4aLZdBYwGjgbOBm6Q1Dci7gNmAE8AvwWeBOq3uljE9RExPiLGDxo0qCNj3yZVFclHbWhs/vHMzKwLKOR5dwlwlKRngaOARWx+rg2PiPHAOcB/SNorbb8IOCUihgL/Dfxb2v5HYGREjAP+DNzYUlBZeUZ2dR4aamZZVcxvpjq27MUbytbDWuqAP0TEpoh4DZhHkhgSEd+PiIMj4niSh+z/FjHWHSIl/wZwHmhm1iW1+7yLiMURcUZEHAJ8I21bkduX/l0APAwcImkQcFBEPJVe4jbgg+lxSyNiQ9r+C+DQYnwo6xibGoLqKheLMbPsKWYiOAsYLWmUpBpgEjC92TF3kQz7JB0COgZYIKkyN2RG0jhgHHBfEWPdIWmHII3OBM3MuqJ2n3eSBkrKPXMvB6am7f0k1eaOASYAc4DlQB9JY9JzjgfmpsftnnfpU3Ptlk2b6hupqawsdRhmZlspWtXQiKiXdCFwL1AJTI2IlyRdCcyOiOnpvhMkzQEagEsjYqmkbsCjaU/bSuBTEbHV0NCsqGjqEXQiaGbW1RT4vDsauEpSAI8AF6Sn7wf8XFIjyY+zU3LVRiWdB9yR7lsOfC495yuSTiUZWrqMzZVELYM2NDS6R9DMMqloiSBARMwgmeuX33ZF3vsALk5f+cesJ6mwVhYqKjw01MysKyvgeTcNmNbCeU8AB7ZyzTuBO1tov5ykV9EyLiLY1OBiMWaWTf5m6gC5oaHhHkEzMzNLNTQGEbhYjJllkr+ZOkCFi8WYmZlZM7llpbyOoJllkb+ZOkBTsRj3CJqZmVlqU33y7wL3CJpZFvmbqQPklo/wOoJmZmaW09QjWOliMWaWPUUtFtNVVKaJoDsEzczMOt5bK9axbmNDqcPYZu+uSpZ7dI+gmWWRE8EOUJF+v3toqJmZWcd6oe49Tr328VKHsUN6dvM/t8wse/zN1AG8jqCZmVlxLH5vHQBfO2kfhvTdpcTRbLuaygqO3W9wqcMwM9uKE8EOIFcNNTMzK4o1G5IhoR89cHdGDOhR4mjMzDoPD1rvAK4aamZmVhxrNtYD0L3Gv12bmXUkJ4IdoNJDQ83MzIoi1yPYs9aJoJlZR3Ii2AGahoY2ljgQMzOzTmbNhnoqBN2q/U8WM7OO5G/VDuChoWZmZsWxekM9PWqqmn50NTOzjtFuIihptqQLJPXbGQGVowqvI2hmZlYUazfW08PDQs3MOlwhPYKTgD2AWZJulXSi/LPcFnLrCDY4EzQzM+tQazY00L22stRhmJl1Ou0mghExPyK+AYwBbgGmAm9K+o6k/sUOsBx4HUEzM7PiWLOx3oVizMyKoKA5gpLGAT8GfgTcAXwcWAk8WLzQysfmoaFOBM3MuiJJJ0maJ2m+pMta2D9C0gOSXpD0sKShefsaJD2XvqbntR8n6Zm0/TFJe6fttZJuS+/1lKSRO+MzlsqadI6gmZl1rHa/WSU9DbwH/BK4LCI2pLuekjShmMGViwovKG9m1mVJqgSuA44H6kimUkyPiDl5h10D3BQRN0o6FrgKODfdty4iDm7h0j8DJkbEXElfBr4JTAY+DyyPiL0lTQJ+CHyiGJ8tC1ZvaGBI3+pSh2Fm1ukU0iP4jxFxXETckpcEAhARZxQprrLSVDXUmaCZWVd0ODA/IhZExEbgVmBis2PGAg+k7x9qYX9LAuidvu8DLE7fTwRuTN9PA47rzHP3XSzGzKw4CvlmXSHpp8CHSB5KjwFXRsTSokZWRnLPXxeLMTPrkoYAC/O264D3NzvmeeBM4CfA6UAvSQPSZ2k3SbOBemBKRNyVnvMFYIakdSTTMY5ofr+IqJe0AhgA/D3/hpLOB84HGD58eEd8zh2yob5hu6prr9lQT3cPDTUz63CFfLPeCjxC8gAD+CRwG/CR9k6UdBLJQ68SuCEiprRwzFnAt0mSzOcj4py0/WrgoyS9lvcDX42MTsLL9QhmMzozMyuylnrjmj8RLgGulTSZ5Jm6iCTxAxgeEYsl7Qk8KOnFiHgVuAg4JSKeknQp8G8kyWEh9yMirgeuBxg/fnxJn1C3/vVNLvv9i9t9fu9dnAiamXW0Qr5Z+0fEd/O2vyfptPZOKmTOhKTRwOXAhIhYLmlw2v5BYAIwLj30MeAo4OEC4t3pKitcNdTMrAurA4blbQ9l8zBOACJiMXAGgKSewJkRsSJvHxGxQNLDwCGSVgIHRcRT6SVuA+5pdr86SVUkw0aXFeFzdZh576yitqqCr35k9DafWyHxsYP2KEJUZmZdWyGJ4EPpZPTb0+2PA/9TwHlNcyYAJOXmTORPnj8PuC4ilgNExLtpewDdgBqSXz6rgXcKuGdJyMVizMy6slnAaEmjSHr6JgHn5B8gaSCwLCIaSX4AnZq29wPWRsSG9JgJwNXAcqCPpDER8QrJj6pz08tNBz4DPEnyTH4wqyNmctZsqKdv92q+fPTepQ7FzMxShSSC/wRcDPwm3a4A1ki6GIiI6N3KeYXMmRgDIOlxkuGj346IeyLiSUkPAW+RJILXRsTcZudmZv5DU7GYbD+HzcysCNJ5ehcC95I8y6ZGxEuSrgRmR8R04GjgKklBMjT0gvT0/YCfS2okeb5OyY2ckXQecEe6bznwufScXwK/ljSfpCdw0s74nDtizcYGF3wxM8uYdr+VI6LXdl67kDkMVcBokgfkUOBRSQcAA0kejrl1lu6X9OGIeKRZbJmY/+B1BM3MuraImAHMaNZ2Rd77aSQVPpuf9wRwYCvXvBO4s4X29cA/7mDIO5XXAjQzy56CvpUlnQp8ON18OCL+VMBp7c6ZSI+ZGRGbgNckzWNzYjgzIlan97+bpFraI2RQLhFsaCxxIGZmZhm0dkMDPWorSx2GmZnlaXcdQUlTgK+SzO2bA3w1bWtP05wJSTUkQ1emNzvmLuCY9D4DSYaKLgDeBI6SVCWpmqRQzFZDQ7OiIv2v6KGhZmZmW1u9oZ6eHhpqZpYphXwrnwIcnE5wR9KNwLPAZW2dVOCciXuBEyTNARqASyNiqaRpwLHAiyTDSe+JiD9u30csPg8NNTMza93ajV4L0Mwsawr9Vu7L5tLUfQq9eAFzJoKkEM3FzY5pIClSUxYqXDXUzMysVas3uFiMmVnWFPKtfBXwbFrFUyRzBS8valRlxlVDzczMWrd2Yz09ajxH0MwsS9pMBJUskPcYSaGWw0gSwa9HxNs7IbayoaZiMU4EzczM8jU2Bmu9fISZWea0+a0cESHprog4lK0LvViqsiI3R7DEgZiZmWXM2k0NAC4WY2aWMe1WDQVmSjqs6JGUMQ8NNTMza9maDfUAdPfyEWZmmVLIz3PHAP8k6Q1gDcnw0IiIcUWNrIy4WIyZmVnLcomgewTNzLKlkG/lk4seRZmTewTNzMxa9C+3PQdADy8fYWaWKYUMDf1eRLyR/wK+V+zAyklTj6C7BM3MzLbw5rK1ALx/z/4ljsTMzPIVkgjun78hqRI4tDjhlKdcsRjngWZmZluqkDj3iBH06lZd6lDMzCxPq4mgpMslrQLGSVqZvlYB7wJ/2GkRlgEPDTUzM2tZY0TTc9LMzLKj1UQwIq6KiF7AjyKid/rqFREDIsILyufJDQ0NJ4JmZmZbiEiqzJmZWba0O3M7Ii6XNAQYkX98RDxSzMDKiauGmpmZtSwikLsEzcwyp91EUNIUYBIwB2hImwNwIpjKrSPY4EzQzMxsCxF4aKiZWQYVUsv5dGCfiNhQ7GDKlZp6BJ0Impl1RZJOAn4CVAI3RMSUZvtHAFOBQcAy4FMRUZfuawBeTA99MyJOTdsfBXql7YOBv0bEaZKOJpmr/1q67/cRcWWxPtuOCkAeHGpmljmFJIILgGrAiWArclVDnQeamXU9aTXt64DjgTpglqTpETEn77BrgJsi4kZJxwJXAeem+9ZFxMHNrxsRR+bd4w62LNT2aET8Qwd/lKKIiKaRM2Zmlh2FJIJrgeckPUBeMhgRXylaVGWmwlVDzczKnqQLgZsjYvk2nno4MD8iFqTXuRWYSDKlImcscFH6/iHgrm2IqxdwLPDZbYwrExo9NNTMLJMKWUdwOvBd4Ang6byXpVwsxsysU9iNpDfvdkknqfAKJ0OAhXnbdWlbvueBM9P3pwO9JA1It7tJmi1ppqTTWrj+6cADEbEyr+0Dkp6XdLek/Vs4B0nnp9edvWTJkgI/SscLouk5aWZm2dFqj6Ck3hGxMiJubGHf8OKGVV68jqCZWfmLiG9K+hZwAknv27WSbgd+GRGvtnFqS1lO8wfCJen1JpMUW1sE1Kf7hkfEYkl7Ag9KerHZ/c4GbsjbfgYYERGrJZ1C0rs4uoXPcz1wPcD48eNL9oBqTCYJmplZxrTVI/hw7k06LDRfwUNauoKmHkF3CZqZlbVIFoR9O33VA/2AaZKubuO0OmBY3vZQYHGz6y6OiDMi4hDgG2nbity+9O8CkmfvIbnz0l7Dw4H/ybvWyohYnb6fAVRLGrg9n3enCBeLMTPLorYSwfxv7f5t7OvyKj001Mys7En6iqSngauBx4EDI+JLwKFsHtbZklnAaEmjJNWQLLk0vdm1B0rKPXMvJ6kgiqR+kmpzxwAT2HJu4T8Cf4qI9XnX2i03bFXS4STP8qXb+bGLLhkaWuoozMysubYSwWjlfUvbLUrnWMyTNF/SZa0cc5akOZJeknRL2naMpOfyXutbmTeRCR4aambWKQwEzoiIEyPidxGxCSAiGoFWK3RGRD1wIXAvMBe4PSJeknSlpFPTw44G5kl6BdgV+H7avh8wW9LzJEVkpjSrNjoJ+G2zW34c+Ft6zk+BSWlPZia5WIyZWTa1VTV0sKSLSXr/cu9Jtwe1d+FCymlLGk3yy+iEiFguaTBARDwEHJwe0x+YD9y3rR9uZ5GElJTINjOzsjWDZI0/oKla59iIeCoi5rZ1YjpEc0aztivy3k8DprVw3hPAgW1c9+gW2q4Frm0rnixJlo9wJmhmljVt9Qj+gmQh255573PbN7RxXk5TOe2I2AjkymnnOw+4LleqOyLebeE6Hwfujoi1BdyzZCokDw01MytvPwNW522vSdtsBzSG55OYmWVRqz2CEfGdHbx2S+W039/smDEAkh4HKoFvR8Q9zY6ZBPxbSzeQdD5wPsDw4aUtZFohaHCPoJlZOVP+EMuIaJRUyHq71h73CJqZZU4h6wg2kfTMthzeQlvzTKmKpOT10aTlsSX1zbvf7iRDZu5t6QYRcX1EjI+I8YMGtTtataiSHkEngmZmZWxBWjCmOn19FVhQ6qDKWS6vdrEYM7Ps2aZEkG0b3dFuOe30mD9ExKaIeA2Yx5ZrIZ0F3JmbsJ9lFRLOA83MytoXgQ+SrPGXG8VyfkkjKnO5KRNePsLMLHu2NRH8n/YPadJuOW2S9QiPgaay2WPY8tfXs9m6WlomVcjrCJqZlbOIeDciJkXE4IjYNSLOaWXuuhXIPYJmZtnV7twHST2AdWn57JvSUth3t9dLFxH1knLltCuBqbly2sDsiJie7jtB0hygAbg0Ipam9x1J0qP4l+3+dDuRi8WYmZU3Sd2AzwP7A91y7RHxuZIFVeaaegSdCJqZZU4hk+AfAY6U1A94AJgNfAL4ZHsnFlBOO4CL01fzc18nKThTFiSvI2hmVuZ+DbwMnAhcSfKca3PZCGtbpKUB5EzQzCxzChkaqnTphjOA/4yI04GxxQ2r/FRWuFiMmVmZ2zsivgWsiYgbgY/Sxhp/1r5wj6CZWWYVlAhK+gDJL6O5OYIup92Mq4aamZW93JSH9yQdAPQBRpYunPIXLhZjZpZZhSR0/wJcTlK98yVJewIPFTes8iPPETQzK3fXp9MgvklS3Kwn8K3ShlTeckNDXSzGzCx72k0EI+IvpAVbJFUAf4+IrxQ7sHJ7kjR4AAAgAElEQVRToc3V0czMrLykz7eVEbGcZG78niUOqVNwsRgzs+xqd2iopFsk9U6rh84B5km6tPihlZcKiQZ3CZqZlaW0MvaFpY6js8n9QOqhoWZm2VPIHMGxEbESOI2kAuhw4NyiRlWGKoSHhpqZlbf7JV0iaZik/rlXqYMqZ7nHonsEzcyyp5A5gtWSqkkSwWsjYpMkpzzNVLhqqJlZucutF3hBXlvgYaLbLRqTv14+wswsewpJBH8OvA48DzwiaQSwsphBlaMKCeeBZmblKyJGlTqGzsbFYszMsquQYjE/BX6a1/SGpGOKF1J5qvCC8mZmZU3Sp1tqj4ibCjj3JOAnQCVwQ0RMabZ/BDAVGAQsAz4VEXXpvgbgxfTQNyPi1LT9UaBX2j4Y+GtEnKake+0nwCnAWmByRDyzLZ91Z2kqFlPaMMzMrAXtJoKS+gD/Cnw4bfoLcCWwoohxlZ0KLx9hZlbuDst73w04DngGaDMRlFQJXAccD9QBsyRNj4g5eYddA9wUETdKOha4is3z7ddFxMHNrxsRR+bd4w7gD+nmycDo9PV+4Gfp38xpKhbjoaFmZplTyNDQqcDfgLPS7XOB/wbOKFZQ5UiCRmeCZmZlKyL+OX87/SH01wWcejgwPyIWpOfdCkwkqbSdMxa4KH3/EHBXoXFJ6gUcC3w2bZpIklQGMFNSX0m7R8RbhV5zZ8k9FT001MwsewqpGrpXRPxrRCxIX9/BE+e3UuliMWZmnc1akl639gwBFuZt16Vt+Z4Hzkzfnw70kjQg3e4mabakmZJOa+H6pwMPpBW8C70fks5Przt7yZIlBXyMjtf0XHSPoJlZ5hTSI7hO0oci4jEASROAdcUNq/wkQ0OdCJqZlStJfySvE4ukF+/2Qk5toa35A+ES4FpJk0kWrF8E1Kf7hkfEYkl7Ag9KejEiXs0792zghm28HxFxPXA9wPjx40vzgPIcQTOzzCokEfwicFM6RAZgOfCZ4oVUnuQ5gmZm5e6avPf1wBu5gi7tqAOG5W0PBRbnHxARi0mnVEjqCZwZESvy9hERCyQ9DBwCvJoeO4Bk6Onp23K/rNg8NNSpoJlZ1rQ5NFRSBbBPRBwEjAPGRcQhEfHCTomujFQIFi1fx0/+/L80OCM0MytHbwJPRcRfIuJxYKmkkQWcNwsYLWmUpBpgEjA9/wBJA9NnKsDlJPPvkdRPUm3uGGACW84t/EfgTxGxPq9tOvBpJY4AVmRxfiBsHhrqPNDMLHvaTAQjohG4MH2/Mm9+gjVTITHnrZX8+59f4fm690odjpmZbbvfAY152w1pW5siop7kWXkvMBe4PSJeknSlpFPTw44G5kl6BdgV+H7avh8wW9LzJEVkpjSrNjoJ+G2zW84AFgDzgV8AXy74E+5kuRkTLhZjZpY9hQwNvV/SJcBtwJpcY0QsK1pUZeitFZunTf5t0QreN7xfCaMxM7PtUBURG3MbEbEx7eFrV0TMIEnQ8tuuyHs/DZjWwnlPAAe2cd2jW2gL4IJC4iq1ph5BzxI0M8ucQhLBz6V/8x86gSuHbuHvq5v+7cALdV5i0cysDC2RdGpETAeQNBH4e4ljKmtNNdScB5qZZU67iWBEjNoZgXQWH9p7IM8tTIaGrly/iW5VldRUFbJKh5mZldgXgZslXZtu1wGfLmE8nYaLxZiZZU+rGYqkT0k6t4X28ySdU8jFJZ0kaZ6k+ZIua+WYsyTNkfSSpFvy2odLuk/S3HT/yELuWWpHjRnE/HdXc/vshYz79n38YMbcUodkZmYFiIhXI+IIkmUj9o+ID0bE/FLHVc42Dw01M7Osaaur6v8Ad7XQflu6r02SKoHrgJNJHqpnSxrb7JjRJNXTJkTE/sC/5O2+CfhRROxHUjr73fbuWUqPfu0YHvv6MRy1zyAAvjYtKax6y1/fdBVRM7MyIOkHkvpGxOqIWJVW9PxeqeMqZ03FYjwwxswsc9r6aq6MiFXNG9PKodUFXPtwYH5ELEgn398KTGx2zHnAdRGxPL32uwBpwlgVEfen7asjYm0B9yyZYf27M7Rfd0YP7slHD9ydmqoKPjthJBvrG5uGipqZWaadHBFNX9jps+mUEsZT9lwsxswsu9pKBKsl9WjeKKkXUEgVtSHAwrzturQt3xhgjKTHJc2UdFJe+3uSfi/pWUk/SnsYm8dyvqTZkmYvWbKkgJCKTxLXffJ9zPnOiZx3ZFJPZ+5bXnXDzKwMVObW9AOQtAtQ28bx1o6mWjHOA83MMqetRPCXwLT8uXnp+1vTfe1p6Wu/+RjJKmA0yfpKZwM3SOqbth8JXAIcRlKhdPJWF4u4PiLGR8T4QYMGFRDSzlNVWcHufbrRvaaS+e+uLnU4ZmbWvt8AD0j6vKTPA/cDN5Y4prKWGxoqZ4JmZpnTatXQiLhG0mrgL5J6kiRxa0gWu/1ZAdeuA4blbQ8FFrdwzMyI2AS8JmkeSWJYBzwbEQsAJN0FHEFhCWhmSGKvQT15dYkTQTOzrIuIqyW9AHyE5MfMe4ARpY2qvIWLxZiZZVab07cj4r8iYgTJg3BURIwoMAkEmAWMljQqXZB3EjC92TF3AccASBpIMiR0QXpuP0m5br5jgTkF3jdT9h7ck1fdI2hmVi7eBhqBM4HjAJd+3gG5YUBePsLMLHsKWVCeiNjmTCYi6iVdCNwLVAJTI+IlSVcCs9MFe+8FTpA0B2gALo2IpQCSLiEZoiPgaeAX2xpDFuw9uCd3PruINRvq6VFb0H9uMzPbiSSNIfmx8mxgKUl1bEXEMSUNrBNoKhbjPNDMLHOKmplExAxgRrO2K/LeB3Bx+mp+7v3AuGLGtzPsNSipt/PqktWMG9q3xNGYmVkLXgYeBT6WWzdQ0kWlDalzaJojWNowzMysBV7Zp8j2HtwTwPMEzcyy60ySIaEPSfqFpONw7tIhXCzGzCy72k0E0+UZLpDUb2cE1NmMGNCDqgq5cqiZWUZFxJ0R8QlgX+Bh4CJgV0k/k3RCSYMrcx4aamaWXYX0CE4C9gBmSbpV0onyT3sFq66sYOTAHvxtkdcSNDPLsohYExE3R8Q/kFS6fg64rMRhdQouFmNmlj3tJoIRMT8ivkFS0fMWYCrwpqTvSOpf7AA7g6PGDOLJV5eyav2mUodiZmYFiIhlEfHziDi21LGUs0YvH2FmllkFzRGUNA74MfAj4A7g48BK4MHihdZ5nHzAbmxsaOQPzzVfRtHMzDoDSSdJmidpvqStehEljZD0gKQXJD0saWjevgZJz6Wv6XntkvR9Sa9ImivpK2n70ZJW5J1zRfP7ZcXmOYKljcPMzLbWbtVQSU8D75Es5n5ZRGxIdz0laUIxg+ssDh3Rj8NG9uPH981jcK9aAHbt3Y0Dh/ShosJPRzOzciapErgOOB6oI5lKMT0i8te/vQa4KSJulHQscBVwbrpvXUQc3MKlJwPDgH0jolHS4Lx9j6ZDWDPN6wiamWVXm4mgpArgjoj4QUv7I+KMokTVyUjih2eO4+xfzOT8Xz/d1D6oVy3f/Oh+TDx4SAmjMzOzHXQ4MD8iFgBIuhWYCOQngmNJitAAPATcVcB1vwScExGNABHxbodFvJM0ev0IM7PManNoaPrwOWknxdKp7TmoJ/dddBQ3fe5wpl84gZ9MOphh/XbhX257jodeLrtnu5mZbTYEWJi3XZe25XueZJkKgNOBXpIGpNvd0grdMyWdlnfOXsAn0n13Sxqdt+8Dkp5P2/dvKShJ56fnzl6yZMl2f7gdkcsD3SNoZpY9hcwRvF/SJZKGSeqfexU9sk6ozy7VfHjMIMYN7cvEg4dw8xeOYL/devOVW5/ljaVrSh2emZltn5aynGi2fQlwlKRngaOARUB9um94RIwHzgH+Q9JeaXstsD7d9wuSYm0AzwAjIuIg4D9ppXcxIq6PiPERMX7QoEHb+dF2TLhYjJlZZhWSCH4OuAB4BHg6fc0uZlBdxS41lfz83EMRcOEtz7KhvqHUIZmZ2barI5nLlzMU2KI6WEQsjogzIuIQ4Btp24rcvvTvApJ1DA/Ju+4d6fs7gXHpcSsjYnX6fgZQLWlgx3+sHZfLht0haGaWPYUsHzGqhdeeOyO4rmBY/+786B8P4sVFK/jh3fNKHY6ZmW27WcBoSaMk1ZCsvzs9/wBJA9N59wCXk/buSeonqTZ3DDCBzXML7wJyy1ccBbySHrdbbj1fSYeTPMuXFumz7RAPDTUzy652q4YCSDqAZKJ7t1xbRNxUrKC6mhP3343JHxzJ1Mdf4wN7DeD4sbuWOiQzMytQRNRLuhC4F6gEpkbES5KuBGZHxHTgaOAqSUEywuaC9PT9gJ9LaiRJ6KbkVRudAtws6SJgNfCFtP3jwJck1QPrgEmRG4OZMV5H0MwsuwpZPuJfSR5gY4EZwMnAY4ATwQ50+Sn7MvuNZVzyu+eZ8dUjGdJ3l1KHZGZmBUqHaM5o1nZF3vtpwLQWznsCOLCVa74HfLSF9muBa3cw5J2iKT11JmhmljmFzBH8OHAc8HZEfBY4iGQCu3Wg2qpKrj37fTQ0BpOn/pX/fWdVqUMyMzPbIZHOEvTQUDOz7CkkEVyXLiNRL6k38C7gOYJFMHJgD64/91CWrdnIx659jF8+9hoNjZkc7WNmZtYuLyNoZpZdhSSCsyX1JSld/TRJ2eq/FjWqLuyDew/k7q8eyYS9BvLdP83hrJ8/ycJla0sdlpmZ2TZrKhZT4VTQzCxrCqka+uWIeC8i/gs4HvhMOkTUimRw727c8Jnx/McnDuaVd1bxsWsfY8aLb5HRWgBmZmYtcrEYM7PsKrRq6BBgRO54SR+OiEeKGVhXJ4nTDhnCQcP68qXfPM2Xb36GfXfrxZnvG8pHx+3OHi4mY2ZmGed1BM3MsquQqqE/BD5Bsq5RbsXzXPlrK7JRA3vwp3/+EL9/dhE3Pfk6358xlx/cPZcP7T2QSYcN5yNjB1NbVVnqMM3MzLaSG8kiZ4JmZplTSI/gacA+EbFhWy8u6STgJyTrKt0QEVNaOOYs4NskyeXzEXFO2t4AvJge9mZEnLqt9+8sqiorOGv8MM4aP4zX/76G3z+7iGmzF3LBLc/Qr3s1Ew8ekvQeDu3jh62ZmWWGi8WYmWVXIYngAqAa2KZEUFIlcB3JvMI6YJak6XkL5SJpNHA5MCEilksanHeJdRFx8LbcsysYObAHFx8/hq8eN5rH5v+d22ct5Ja/vsmvnnidUQN7cNy+g/nwmEEcPqo/3ardU2hmZqXj5SPMzLKrkERwLfCcpAfISwYj4ivtnHc4MD8iFgBIuhWYSDLENOc84LqIWJ5e891tiL1Lq6wQR40ZxFFjBrFy/SbuefFt/vjCYm568g1ueOw1aqsqOGxkf8aP7MfhI/tz8PC+dK8paEqomZlZh2hsTP46DzQzy55CMoPp6WtbDQEW5m3XAe9vdswYAEmPkwwf/XZE3JPu6yZpNlAPTImIu7Yjhi6hd7dqzjpsGGcdNox1GxuY+dpSHnllCTMXLOMnD/wvEUnieMAevTlkeD/GDe3DgUP6sOegnlS6pLeZmRVJU7EYDw41M8ucdhPBiLhxO6/d0rd+8/UPqoDRwNHAUOBRSQdExHvA8IhYLGlP4EFJL0bEq1vcQDofOB9g+PDh2xlm57JLTSXH7DOYY/ZJRtmuXL+Jp99YzuzXlzHr9eXcNmshv3ridQC611RywB59OGBInyQ5HNqHUQN6eL0nMzPrEJuLxZQ4EDMz20qriaCk2yPiLEkvsnUCR0SMa+fadcCwvO2hwOIWjpkZEZuA1yTNI0kMZ0XE4vQ+CyQ9DBwCbJEIRsT1wPUA48eP9yJ7LejdrXqLxLChMViwZDUv1K3gxUUreKHuPW756xtMfTwZv9OjppJ9duvFfrv3bnrtu1svetR6WKmZmW2bxlyxGCeCZmaZ09a/7r+a/v2H7bz2LGC0pFHAImAScE6zY+4CzgZ+JWkgyVDRBZL6AWsjYkPaPgG4ejvjsDyVFWL0rr0YvWsvzjx0KAD1DY3MT5PDlxatYO5bq5j+/GJufupNIHmAj+jffYvkcL/dezGk7y6uUmpmZm1wsRgzs6xqNRGMiLfSv2/k2tKkbGnkxnq0ISLqJV0I3Esy/29qRLwk6UpgdkRMT/edICm3RuGlEbFU0geBn0tqBCpI5gjOaeVWtoOqKivYd7fe7LtbbxifdOJGBHXL1zH3rZXMfWsVL7+9krlvreTuv73ddF7vblXst3tvJn9wJCcfuHupwjczs4xyj6CZWXa1NTT0CGAKsAz4LvBrYCBQIenTeUVdWhURM4AZzdquyHsfwMXpK/+YJ4ADC/8Y1tEkMax/d4b1784J++/W1L5mQz0vv70qTRBXMnPBUr508zP87osf4LCR/UsYsZmZZc3mdQSdCZqZZU1bQ0OvBf4v0Ad4EDg5ImZK2hf4LdBuImidT4/aKg4d0Y9DR/QDYN3GBo68+iH+30Pz+e/PHl7i6MzMLEs2ryNY4kDMzGwrFW3sq4qI+yLid8DbETETICJe3jmhWTnYpaaSI0cPZP6S1aUOxcysZCSdJGmepPmSLmth/whJD0h6QdLDkobm7WuQ9Fz6mp7XLknfl/SKpLmSvpLX/tP0Xi9Iet/O+ZTbzkNDzcyyq60ewca89+ua7XOFTmvSr3sNy9dsKnUYZmYlIakSuA44nqQa9ixJ05vNbb8GuCkibpR0LHAVcG66b11EHNzCpSeTVN/eNyIaJQ1O208mqbA9mmR93p+x9Tq9mbB5+QhngmZmWdNWIniQpJUk6wHukr4n3e5W9MisbPTvUc3qDfVsqG+gtqqy1OGYme1shwPzI2IBgKRbgYlAfiI4Frgoff8QSdXs9nwJOCciGgEi4t20fSJJUhnATEl9Je2eK/KWJZvnCJqZWda0OjQ0IiojondE9IqIqvR9brt6ZwZp2davRw0A7611r6CZdUlDgIV523VpW77ngTPT96cDvSQNSLe7SZotaaak0/LO2Qv4RLrvbkmjt+F+SDo/PXf2kiVLtu+T7aDcHEH3CJqZZU9bcwTNCtK/e5IILluzscSRmJmVREtZTvMpFJcAR0l6FjiKZH3d+nTf8IgYT7LW7n9I2ittrwXWp/t+AUzdhvsREddHxPiIGD9o0KBt+kAdJdcj6GIxZmbZ40TQdliuR3C5E0Ez65rqSOby5QwFFucfEBGLI+KMiDgE+EbatiK3L/27AHgYOCTvunek7+8ExhV6v6xo9PIRZmaZ5UTQdlj/NBFcttaJoJl1SbOA0ZJGSaoBJgHT8w+QNFBS7pl7OWnvnqR+kmpzxwAT2Dy38C7g2PT9UcAr6fvpwKfT6qFHACuyOD8Q8ovFlDgQMzPbSlvFYswK0q+7ewTNrOuKiHpJFwL3ApXA1Ih4SdKVwOyImA4cDVwlKYBHgAvS0/cDfi6pkeTH2Sl51UanADdLughYDXwhbZ8BnALMB9YCny32Z9xe4eUjzMwyy4mg7bB+3ZPaQUudCJpZFxURM0gStPy2K/LeTwOmtXDeE8CBrVzzPeCjLbQHmxPJTHOxGDOz7PLQUNthVZUVdK+pZPX6+vYPNjOzLsPFYszMssuJoHWI7jVVrNnYUOowzMwsQ1wsxswsu5wIWofoWVvJmg3uETQzs802Dw0tcSBmZrYVJ4LWIXrUVjkRNDOzLTS6WIyZWWY5EbQO0aO2itVOBM3MLF9u+QgPDTUzyxwngtYhetRUsmajE0EzM9ss7RB0sRgzswxyImgdokdtFWs3uFiMmZlt1tjo5SPMzLLKiaB1iJ4eGmpmZs3kegSdBpqZZY8TQesQLhZjZmbNbV5H0KmgmVnWOBG0DpHMEWxoGgZkZmbWGE0LCZqZWcYUNRGUdJKkeZLmS7qslWPOkjRH0kuSbmm2r7ekRZKuLWactuN61FYBsHZTMk+wsTF4d9V6NjU0ljIsMzPLABeLMTPLnqpiXVhSJXAdcDxQB8ySND0i5uQdMxq4HJgQEcslDW52me8CfylWjNZxcongmg313P3iW/z4vld4e+V6etRUctohQ/inD+/F8AHdSxylmZntTLkeQReLMTPLnqIlgsDhwPyIWAAg6VZgIjAn75jzgOsiYjlARLyb2yHpUGBX4B5gfBHjtA7QM00EP/3LvzLvnVW8b3hf/umoPXlx0Qp+93Qdv5tdx2c+OIILjxlNn+7VJY7WzMx2Bo8MNTPLrmImgkOAhXnbdcD7mx0zBkDS40Al8O2IuEdSBfBj4FzguNZuIOl84HyA4cOHd1zkts369agBYN47q7j85H35wpF7UpmOBfraifvyb/fP44bHXuN3T9fxlWNH88kjhlNbVVnKkM3MrMg2ryPoVNDMLGuKmQi29K3fvJJIFTAaOBoYCjwq6QDgU8CMiFjY1nCSiLgeuB5g/PjxrlJSQhP2GsB/Tz6MvQf3ZFj/LYeA7tanG1d//CAmf3AUP5gxlyv/NIcf3zePD+w1gAOG9GHs7r3Zf0gf9ujTzcOHzMw6kc1DQ0sciJmZbaWYiWAdMCxveyiwuIVjZkbEJuA1SfNIEsMPAEdK+jLQE6iRtDoiWiw4Y6VXVVnBMfs2n+K5pbF79ObXnz+cJ15dyv+8+BYzX13KAy+/2zR0qG/3asbs2ouRA7ozYkAPhvfvzh59d2G3Pt0Y3KuW6koXuTWzbJJ0EvATktEtN0TElGb7RwBTgUHAMuBTEVGX7msAXkwPfTMiTk3bfwUcBaxI902OiOckHQ38AXgtbf99RFxZpI+2Q5qGhjoRNDPLnGImgrOA0ZJGAYuAScA5zY65Czgb+JWkgSRDRRdExCdzB0iaDIx3Etg5SGLC3gOZsPdAANZurGfuW6uY89ZK5ixewfx3V/PQvCUsWVXX7DwY2LOW3ft0Y7fe3ditT/rqveXf7jXF/L+0mdnWCimOBlwD3BQRN0o6FriKZPoDwLqIOLiVy18aEdNaaH80Iv6hgz5C0USuR9CzBM3MMqdo/2qOiHpJFwL3kvxCOjUiXpJ0JTA7Iqan+06QNAdoIHngLS1WTJY93WuqOHREPw4d0W+L9jUb6lm4fC1vrVjP2/mvlet5Y+laZi5Yysr1Wy9g37tbFQN71TKgRw39e9TQv8fm9wN65tpqGNCjlv49aqipci+jme2wQoqjjQUuSt8/RPJDaKfnHkEzs+wqavdJRMwAZjRruyLvfQAXp6/WrvEr4FfFidCyqkdtFfvu1pt9d+vd6jFrN9Y3JYf5f5eu3sjSNRt47e9rePqN5Sxbs5HW1rnvVVtF712q6du9mj675L2abffdpWaL7V7dqqjwwlhmliikONrzwJkkw0dPB3pJGpD++NlN0mygHpgSEflJ4vclXQH/v717D5KrLPM4/v1N93RPJhPIhXAx3ASDBkpMFBGMrogrBdQWorK74g0td1m32CrdVRcot9S13FJ33UVXWcViAS/Uyi5eoCgUMSDuanE3XMJFLoLERAJJyGUy09dn/zjvTDrTnWRI0tM9079P1alzzttvn37PM9PzznPOe85hBXBRRJRS+cmS7iO75OLjEbFqYqO64YZqvlmMmVn38jg6m7YGC3mOWjjEUQuHdlmvXg82jVRYP1xmw3CZDcOlbHlrmfXDZTaPVNiUpsfWbR1fLlfrO92mBPsNZAnhUDHPfgP9DKXloYE8c4rbl4eKeeYM5Jkz0L/j6wN5ZvXnfIMcs+lvMjdH+zjwtXS5wy/ILpkYG9ZweESskXQUcIukByLiCbLn7P4BKJDdGO1C4LPAvcAREbFV0plkZxcXNzWgC26oNn6zmE58uJmZ7ZITQZvx+vrEvNmF8UdcTNZopcYL27YniZtGKrywrcymkcp48rhltMqWUpWto1We25KdhdwympWXdpFIjrdNpESxn8FCLk15Bgs5ZhVyzC7ks3lxe3ljnbH57GKOWYU8g/05Bos5Crk+J5hmU2e3N0eLiDXAOwAkDQHvjIhNDa8REU9K+jmwDHgiItamt5ckXUmWTBIRmxu2e6Ok/5B0QEQ8346d2xseGmpm1r2cCJrtxEB/joP3z3Hw/gN79P5ytc5wqcrWUpUto2Pzyg7rW9N882iFkXKNbeVaNuR1c2V8eVupxrZKjdrOxre2kOvTDkljMd/HrEKOgXyOgf4+BvpzDVNaT6+N1Ss21JvVol4xlffn5KTTet1ub46Wboi2ISLqZGf6rkjl84BtEVFKdZYD/5xeOyQi1ir7gp0NPJjKDwaejYiQdCLQB3Tl9fXjN4vx3wgzs67jRNCsTQr5Pgr5F38mspWIoFStM1KuMVyujieNY8vD5Roj5SrDpRojlRrDpSrbyrXx+qOVOqVqjdFKjee3VhmpZMujlTqlSo3Rao1Kbc9GjvWJ8YSxmO+jmO+jkO+jmM+lebZeyGXJYzbvG58Xd1JeyOW2v3fCdlt9Ts7XbFqHTPLmaKcAn5cUZENDL0hvXwJcJqlOltB9oeFuo1dLWkg2snIl8OFUfg7w15KqwAjwrhjLuLpM4LOBZmbdyomg2TQgaTzZ2heJZSvVWp3Raj0liFmS2LRczZZHKrUsgWx4baRSo1ytU6rWKVfrlGtZ8lmu1tlaqu7wWqlao9Swvi/k+7RDwrj9TGY2n1VoWC/0ZWX9OWYX8yyaN4sjF8xmySH7OaG0PTKJm6NdCzQ9BiIifgW8cifbPHUn5V8DvrY37Z0qEb5RjJlZt3IiaGYA5HN9DOX6GCpO7Z+FiKBcq09IFFMyWd2eTJbGp1rLuhPLR9MZ0JFKndFyjee2lBipZGdJxxLXkUqNxvMoi+bO4p/POX78OZdmtnfqEb5RjJlZl3IiaGYdJSkN98wxZ4o/OyLYWqqyeuMIj/5hC1+95TE+/J17+PFH38ih8wanuDVmM0/gM4JmZt3KT9M2s54liTkD/Sw5ZD/OXraIqz54IlvLVa69Z3Wnm2Y2I9Qj/OwIM7Mu5UTQzCw5bP4gr8MZLIcAAA4oSURBVDl8Hj9d9Wynm2I2MzgPNDPrWk4EzcwavGXJQTy0djPPby11uilm056HhpqZdS8ngmZmDV5zxDwAPnfDQ7zpX25l00ilwy0ym77q9fDjI8zMupQTQTOzBq9ctD8AP1q5hqfXb+Pepzd2uEVm01fgoaFmZt3KiaCZWYNZhRzHHDQ0vn7NXc/wV9+5mwdWb+pgq8ymp3qEh4aamXUpPz7CzGyC7/7F68j39fGBK+/kJ6v+AMAdv93ALy88lW3lGkPFPLMKuQ630qz7hU8Jmpl1LSeCZmYTHDhnAICLz1jCJT/7DcsOm8tlv3iSEz73M0YqNY57yX587/yTmDPQ3+GWmnU/nxE0M+tOTgTNzHbi5KMXcPLRJwNQqta56ldPceKR87n76Q285/I7OHrhEM9s2MZZS1/CuScejoB6QCHvUfdmkA0NdR5oZtadnAiamU3CZ846jo+ddgxzBvq5buXv+fT1q1i9cYQDhgp86rpVfPHHj1Cu1anWg9ceMZ9lh8+lWg9yfWLB7ALzZxdYMFRg/uzi+PpgIYf8X7LNYH6evJlZ93IiaGY2SWNDQd+2dBFnveol4+W3/eY5Vjy8jtnFPBLc+sg6rvzlU/TnRKUelKv1ltvL9YnBQo7ZhTyDhRyDxRyDhTyzC9l8oD/HrEIfs/pzzCrks3l/XyrPUcznGEjr2ZTVHejPMZDPUezvo5jvc7JpHRP4ZjFmZt3KiaCZ2R5oTK5OefmBnPLyA8fXLzz9FePLEcFwucaGrWXWD5fYMFxm/XCZDcNltoxWGC7V2Fausq1cS1OV9cNlntk4wki5xmilxkiaIvaknVDMNySI/TmK+b7mxDGVF/N9FPtzFHJjy30U87mm5UK+dXnja/05OQntcfXAQ0PNzLpUWxNBSacDXwFywOUR8YUWdf4M+AzZ44bui4h3SzoC+EF6Xz/w1Yj4RjvbambWDpIYKuYZKuY5fMHgHm8nIihV62xLyWE21RmtTlhvnFdrjJZrjFbrTXVGKjVKlTrrh8vj5eVqnVK1RinVr+9B4rnjvpOSw+0JYyHXx0VnLOGtxx60dxu3KXXNXb/j8v/97Yt+37ObRynkfYddM7Nu1LZEUFIOuBR4K7AauEvS9RHxUEOdxcDFwPKI2Chp7JD6WuD1EVGSNAQ8mN67pl3tNTPrZpLGz9xNlWqtTqmaTY1JYqnSsFzNEspybWJ5nVKlYbnh/fvPmnl3W93dgc90gPMKYCGwAXhvRKxOr9WAB1LV30XEWan8KuBNwNhDLD8QESuVnWb9CnAmsC2V39vG3WPeYIHFDc/XnKzFBw3x6sPntaFFZma2t9p5RvBE4PGIeBJA0veAtwEPNdT5S+DSiNgIEBHr0rzcUKeIH3xvZjbl8rk+8rk+Zhc73ZLuNpkDn8CXgG9HxLcknQp8Hnhfem0kIpbuZPOfiIhrJ5SdASxO0+uAr6d525x23MGcdtzB7fwIMzObYu1MsBYBzzSsr05ljY4BjpH0S0m3pyOqAEg6TNL9aRtfbHU2UNL5ku6WdPdzzz3Xhl0wMzPbrfEDn+lA5tiBz0bHAivS8q0tXn8x3kaWVEZE3A7MlXTIXmzPzMx6UDsTwVaXh0+84iRPdkTzFOBc4HJJcwEi4pmIOB54GXCepKYLSiLimxFxQkScsHDhwn3aeDMzs0mazIHP+4B3puW3A3MkLUjrA+mg5u2Szp7wvn+SdL+kSySNnZudzOf5YKmZme1SOxPB1cBhDeuHAhPP6q0GrouISkT8FniULDEcl84ErgLe2Ma2mpmZ7anJHPj8OPAmSb8mu+7v90A1vXZ4RJwAvBv4sqSjU/nFwCuA1wLzgQtfxOf5YKmZme1SOxPBu4DFkl4qqQC8C7h+Qp0fAW8GkHQA2VDRJyUdKmlWKp8HLCdLEs3MzLrNbg98RsSaiHhHRCwDPpnKNo29luZPAj8HlqX1tWn4Zwm4kmwI6qQ+z8zMbHfalghGRBX4G+Am4GHgvyNilaTPSjorVbsJWC/pIbJrJj4REeuBJcAdku4DbgO+FBEPNH+KmZlZx+32wKekAySN9bkXk91BFEnzxoZ8pgOiy0k3VRu77i/dJfRs4MH0/uuB9ytzErApIta2cwfNzGzmaetzBCPiRuDGCWWfalgO4O/S1FjnZuD4drbNzMxsX4iIqqSxA5854IqxA5/A3RFxPdm18J+XFMAvgAvS25cAl0mqkx2c/ULD3UavlrSQbCjoSuDDqfxGskdHPE72+IgPtnsfzcxs5mlrImhmZtYLJnHg81pg4mMgiIhfAa/cyTZP3Ul5sD2RNDMz2yN+Pp+ZmZmZmVmPUXZgcfqT9Bzw9D7Y1AHA8/tgOzOJY9LMMWnNcWnmmDTbFzE5IiJ8K8xJ2kd9pH+XW3NcmjkmzRyT1hyXZnsbk0n3jzMmEdxXJN2dbuNtiWPSzDFpzXFp5pg0c0ymJ//cWnNcmjkmzRyT1hyXZlMZEw8NNTMzMzMz6zFOBM3MzMzMzHqME8Fm3+x0A7qQY9LMMWnNcWnmmDRzTKYn/9xac1yaOSbNHJPWHJdmUxYTXyNoZmZmZmbWY3xG0MzMzMzMrMc4ETQzMzMzM+sxTgQTSadLelTS45Iu6nR7ppKkKyStk/RgQ9l8STdLeizN56VySfr3FKf7Jb26cy1vH0mHSbpV0sOSVkn6SCrv2bhIGpB0p6T7Ukz+MZW/VNIdKSbXSCqk8mJafzy9fmQn299OknKSfi3phrTumEhPSXpA0kpJd6eynv3+THe92ke6f2zm/rE195E75z5yR93UPzoRJPsFBS4FzgCOBc6VdGxnWzWlrgJOn1B2EbAiIhYDK9I6ZDFanKbzga9PURunWhX4WEQsAU4CLki/E70clxJwakS8ClgKnC7pJOCLwCUpJhuBD6X6HwI2RsTLgEtSvZnqI8DDDeuOSebNEbG04XlIvfz9mbZ6vI+8CvePE7l/bM195M65j2zWHf1jRPT8BJwM3NSwfjFwcafbNcUxOBJ4sGH9UeCQtHwI8Ghavgw4t1W9mTwB1wFvdVzG928QuBd4HfA8kE/l498l4Cbg5LScT/XU6ba3IRaHkv3RPhW4AVCvxyTt31PAARPK/P2ZhlOv95HuH3cbH/ePzTFxH7k9Fu4jm2PSNf2jzwhmFgHPNKyvTmW97KCIWAuQ5gem8p6LVRqasAy4gx6PSxresRJYB9wMPAG8EBHVVKVxv8djkl7fBCyY2hZPiS8Dfw/U0/oCHBOAAH4q6R5J56eynv7+TGP++ezIv8eJ+8cduY9syX1ks67pH/P7akPTnFqU+bkarfVUrCQNAd8HPhoRm6VWu59VbVE24+ISETVgqaS5wA+BJa2qpfmMj4mkPwHWRcQ9kk4ZK25RtWdi0mB5RKyRdCBws6RHdlG3l+IyHfnnMzk9FSf3j83cR+7IfeROdU3/6DOCmdXAYQ3rhwJrOtSWbvGspEMA0nxdKu+ZWEnqJ+vkro6IH6Tino8LQES8APyc7PqQuZLGDio17vd4TNLr+wMbpralbbccOEvSU8D3yIa+fJnejgkAEbEmzdeR/UN0Iv7+TFf++eyo53+P3T/umvvIce4jW+im/tGJYOYuYHG6i1EBeBdwfYfb1GnXA+el5fPIrgEYK39/uovRScCmsVPZM4myQ5v/CTwcEf/W8FLPxkXSwnSUE0mzgD8mu/j7VuCcVG1iTMZidQ5wS6QB7jNFRFwcEYdGxJFkfzduiYj30MMxAZA0W9KcsWXgNOBBevj7M825j9xRT/8eu39szX1kM/eRzbquf+z0BZPdMgFnAr8hG8/9yU63Z4r3/b+AtUCF7MjDh8jGZK8AHkvz+amuyO4e9wTwAHBCp9vfppi8gezU+/3AyjSd2ctxAY4Hfp1i8iDwqVR+FHAn8DjwP0AxlQ+k9cfT60d1eh/aHJ9TgBsck/H9vy9Nq8b+pvby92e6T73aR7p/bBkT94+t4+I+ctfxcR8Z3dc/Kn2ImZmZmZmZ9QgPDTUzMzMzM+sxTgTNzMzMzMx6jBNBMzMzMzOzHuNE0MzMzMzMrMc4ETQzMzMzM+sxTgTNZiBJp0i6odPtMDMz6zbuI80yTgTNzMzMzMx6jBNBsw6S9F5Jd0paKekySTlJWyX9q6R7Ja2QtDDVXSrpdkn3S/qhpHmp/GWSfibpvvSeo9PmhyRdK+kRSVdLUqr/BUkPpe18qUO7bmZmtkvuI83ay4mgWYdIWgL8ObA8IpYCNeA9wGzg3oh4NXAb8On0lm8DF0bE8cADDeVXA5dGxKuA1wNrU/ky4KPAscBRwHJJ84G3A8el7XyuvXtpZmb24rmPNGs/J4JmnfMW4DXAXZJWpvWjgDpwTarzXeANkvYH5kbEban8W8AfSZoDLIqIHwJExGhEbEt17oyI1RFRB1YCRwKbgVHgcknvAMbqmpmZdRP3kWZt5kTQrHMEfCsilqbp5RHxmRb1Yjfb2JlSw3INyEdEFTgR+D5wNvCTF9lmMzOzqeA+0qzNnAiadc4K4BxJBwJImi/pCLLv5TmpzruB/4uITcBGSW9M5e8DbouIzcBqSWenbRQlDe7sAyUNAftHxI1kQ2KWtmPHzMzM9pL7SLM2y3e6AWa9KiIekvQPwE8l9QEV4AJgGDhO0j3AJrJrJADOA76ROrEngQ+m8vcBl0n6bNrGn+7iY+cA10kaIDtS+rf7eLfMzMz2mvtIs/ZTxK7OqJvZVJO0NSKGOt0OMzOzbuM+0mzf8dBQMzMzMzOzHuMzgmZmZmZmZj3GZwTNzMzMzMx6jBNBMzMzMzOzHuNE0MzMzMzMrMc4ETQzMzMzM+sxTgTNzMzMzMx6zP8D5p4y513EXLIAAAAASUVORK5CYII=\n",
385 |       "text/plain": [
386 |        "<Figure size 1080x288 with 2 Axes>"
387 |       ]
388 |      },
389 |      "metadata": {
390 |       "needs_background": "light"
391 |      },
392 |      "output_type": "display_data"
393 |     }
394 |    ],
395 |    "source": [
396 |     "loss_history, accuracy_history = train_model(model, X_train, y_train, \n",
397 |     "                                             bce_loss, optimizer, accuracy, n_epochs)"
398 |    ]
399 |   },
400 |   {
401 |    "cell_type": "markdown",
402 |    "metadata": {},
403 |    "source": [
404 |     "As you can see, the loss and accuracy scores are a continuation from previous training. Training a model in multiple sessions by saving and restoring it when needed can be vital for very heavy models."
405 |    ]
406 |   },
407 |   {
408 |    "cell_type": "code",
409 |    "execution_count": null,
410 |    "metadata": {},
411 |    "outputs": [],
412 |    "source": []
413 |   },
414 |   {
415 |    "cell_type": "code",
416 |    "execution_count": null,
417 |    "metadata": {},
418 |    "outputs": [],
419 |    "source": []
420 |   },
421 |   {
422 |    "cell_type": "code",
423 |    "execution_count": null,
424 |    "metadata": {},
425 |    "outputs": [],
426 |    "source": []
427 |   }
428 |  ],
429 |  "metadata": {
430 |   "kernelspec": {
431 |    "display_name": "Python 3",
432 |    "language": "python",
433 |    "name": "python3"
434 |   },
435 |   "language_info": {
436 |    "codemirror_mode": {
437 |     "name": "ipython",
438 |     "version": 3
439 |    },
440 |    "file_extension": ".py",
441 |    "mimetype": "text/x-python",
442 |    "name": "python",
443 |    "nbconvert_exporter": "python",
444 |    "pygments_lexer": "ipython3",
445 |    "version": "3.6.6"
446 |   }
447 |  },
448 |  "nbformat": 4,
449 |  "nbformat_minor": 2
450 | }
451 | 


--------------------------------------------------------------------------------
/TensorFlow2.0__03.01_Convolutional_Neural_Network.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {},
  6 |    "source": [
  7 |     "# Convolutional Neural Network\n",
  8 |     "\n",
  9 |     "### Author: Ivan Bongiorni, Data Scientist at GfK.\n",
 10 |     "\n",
 11 |     "[LinkedIn profile](https://www.linkedin.com/in/ivan-bongiorni-b8a583164/)\n",
 12 |     "\n",
 13 |     "In this Notebook I will implement a **basic CNN in TensorFlow 2.0**. I will use the famous **Fashion MNIST** dataset, [published by Zalando](https://github.com/zalandoresearch/fashion-mnist) and made [available on Kaggle](https://www.kaggle.com/zalando-research/fashionmnist). Images come already preprocessed in 28 x 28 black and white format.\n",
 14 |     "\n",
 15 |     "It is a multiclass classification task on the following labels:\n",
 16 |     "0. T-shirt/top\n",
 17 |     "1. Trouser\n",
 18 |     "2. Pullover\n",
 19 |     "3. Dress\n",
 20 |     "4. Coat\n",
 21 |     "5. Sandal\n",
 22 |     "6. Shirt\n",
 23 |     "7. Sneaker\n",
 24 |     "8. Bag\n",
 25 |     "9. Ankle boot "
 26 |    ]
 27 |   },
 28 |   {
 29 |    "cell_type": "markdown",
 30 |    "metadata": {},
 31 |    "source": [
 32 |     "![](https://limetta.se/globalassets/nyhetbloggpress-bilder/fashion-mnist.jpg)"
 33 |    ]
 34 |   },
 35 |   {
 36 |    "cell_type": "markdown",
 37 |    "metadata": {},
 38 |    "source": [
 39 |     "Summary:\n",
 40 |     "\n",
 41 |     "0. Import data + Dataprep\n",
 42 |     "0. CNN architecture\n",
 43 |     "0. Training with Mini Batch Gradient Descent\n",
 44 |     "0. Test"
 45 |    ]
 46 |   },
 47 |   {
 48 |    "cell_type": "code",
 49 |    "execution_count": 2,
 50 |    "metadata": {},
 51 |    "outputs": [
 52 |     {
 53 |      "name": "stdout",
 54 |      "output_type": "stream",
 55 |      "text": [
 56 |       "2.0.0-beta0\n"
 57 |      ]
 58 |     }
 59 |    ],
 60 |    "source": [
 61 |     "# Import necessary modules\n",
 62 |     "\n",
 63 |     "import numpy as np\n",
 64 |     "import pandas as pd\n",
 65 |     "\n",
 66 |     "import tensorflow as tf\n",
 67 |     "print(tf.__version__)\n",
 68 |     "\n",
 69 |     "from sklearn.utils import shuffle\n",
 70 |     "\n",
 71 |     "from matplotlib import pyplot as plt\n",
 72 |     "import seaborn"
 73 |    ]
 74 |   },
 75 |   {
 76 |    "cell_type": "markdown",
 77 |    "metadata": {},
 78 |    "source": [
 79 |     "# 0. Import data + Dataprep\n",
 80 |     "\n",
 81 |     "The dataset comes already divided in 60k and 10k Train and Test images. I will now import Training data, and leave Test for later. In order to dataprep image data, I need to reshape the pixel into `(, 28, 28, 1)` arrays; the 1 at the end represents the channel: 1 for black and white images, 3 (red, green, blue) for colored images. Pixel data are also scaled to the `[0, 1]` interval."
 82 |    ]
 83 |   },
 84 |   {
 85 |    "cell_type": "code",
 86 |    "execution_count": 3,
 87 |    "metadata": {},
 88 |    "outputs": [],
 89 |    "source": [
 90 |     "df = pd.read_csv('fashion-mnist_train.csv')\n",
 91 |     "\n",
 92 |     "# extract labels, one-hot encode them\n",
 93 |     "label = df.label\n",
 94 |     "label = pd.get_dummies(label)\n",
 95 |     "label = label.values\n",
 96 |     "label = label.astype(np.float32)\n",
 97 |     "\n",
 98 |     "df.drop('label', axis = 1, inplace = True)\n",
 99 |     "df = df.values\n",
100 |     "df = df.astype(np.float32)\n",
101 |     "\n",
102 |     "# reshape and scale data\n",
103 |     "df = df.reshape((len(df), 28, 28, 1))\n",
104 |     "df = df / 255."
105 |    ]
106 |   },
107 |   {
108 |    "cell_type": "markdown",
109 |    "metadata": {},
110 |    "source": [
111 |     "# 1. CNN architecture\n",
112 |     "\n",
113 |     "I will feed images into a set of **convolutional** and **max-pooling layers**:\n",
114 |     "\n",
115 |     "- Conv layers are meant to extract relevant informations from pixel data. A number of *filters* scroll through the image, learning the most relevant informations to extract.\n",
116 |     "- Max-Pool layers instead are meant to drastically reduce the number of pixel data. For each (2, 2) window size, Max-Pool will save only the pixel with the highest activation value. Max-Pool is meant to make the model lighter by removing the least relevant observations, at the cost of course of loosing a lot of data!\n",
117 |     "\n",
118 |     "Since I'm focused on the implementation, rather than on the theory behind it, please refer to [this good article](https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d) on how Conv and Max-Pool work in practice. If you are a die-hard, check [this awesome page from a CNN Stanford Course](http://cs231n.github.io/convolutional-networks/?source=post_page---------------------------#overview).\n",
119 |     "\n",
120 |     "Con and Max-Pool will extract and reduce the size of the input, so that the following feed-forward part could process it. The first convolutional layer requires a specification of the input shape, corresponding to the shape of each image.\n",
121 |     "\n",
122 |     "Since it's a multiclass classification tasks, softmax activation must be placed at the output layer in order to transform the Network's output into a probability distribution over the ten target categories."
123 |    ]
124 |   },
125 |   {
126 |    "cell_type": "code",
127 |    "execution_count": 2,
128 |    "metadata": {},
129 |    "outputs": [],
130 |    "source": [
131 |     "from tensorflow.keras import Sequential\n",
132 |     "from tensorflow.keras.layers import Conv2D, MaxPool2D, Flatten, Dense, BatchNormalization, Dropout\n",
133 |     "from tensorflow.keras.activations import relu, elu, softmax\n",
134 |     "\n",
135 |     "\n",
136 |     "CNN = Sequential([\n",
137 |     "    \n",
138 |     "    Conv2D(32, kernel_size = (3, 3), activation = elu, \n",
139 |     "           kernel_initializer = 'he_normal', input_shape = (28, 28, 1)), \n",
140 |     "    MaxPool2D((2, 2)), \n",
141 |     "    \n",
142 |     "    Conv2D(64, kernel_size = (3, 3), kernel_initializer = 'he_normal', activation = elu), \n",
143 |     "    BatchNormalization(), \n",
144 |     "    \n",
145 |     "    Conv2D(128, kernel_size = (3, 3), kernel_initializer = 'he_normal', activation = elu), \n",
146 |     "    BatchNormalization(), \n",
147 |     "    Dropout(0.2), \n",
148 |     "    \n",
149 |     "    \n",
150 |     "    Flatten(), \n",
151 |     "    \n",
152 |     "    \n",
153 |     "    Dense(400, activation = elu), \n",
154 |     "    BatchNormalization(), \n",
155 |     "    Dropout(0.2), \n",
156 |     "    \n",
157 |     "    Dense(400, activation = elu), \n",
158 |     "    BatchNormalization(), \n",
159 |     "    Dropout(0.2), \n",
160 |     "    \n",
161 |     "    Dense(10, activation = softmax)\n",
162 |     "    \n",
163 |     "])\n"
164 |    ]
165 |   },
166 |   {
167 |    "cell_type": "code",
168 |    "execution_count": 1,
169 |    "metadata": {},
170 |    "outputs": [
171 |     {
172 |      "name": "stdout",
173 |      "output_type": "stream",
174 |      "text": [
175 |       "Model: \"sequential\"\n",
176 |       "_________________________________________________________________\n",
177 |       "Layer (type)                 Output Shape              Param #   \n",
178 |       "=================================================================\n",
179 |       "conv2d (Conv2D)              (None, 26, 26, 32)        320       \n",
180 |       "_________________________________________________________________\n",
181 |       "max_pooling2d (MaxPooling2D) (None, 13, 13, 32)        0         \n",
182 |       "_________________________________________________________________\n",
183 |       "conv2d_1 (Conv2D)            (None, 11, 11, 64)        18496     \n",
184 |       "_________________________________________________________________\n",
185 |       "batch_normalization (BatchNo (None, 11, 11, 64)        256       \n",
186 |       "_________________________________________________________________\n",
187 |       "conv2d_2 (Conv2D)            (None, 9, 9, 128)         73856     \n",
188 |       "_________________________________________________________________\n",
189 |       "batch_normalization_1 (Batch (None, 9, 9, 128)         512       \n",
190 |       "_________________________________________________________________\n",
191 |       "dropout (Dropout)            (None, 9, 9, 128)         0         \n",
192 |       "_________________________________________________________________\n",
193 |       "flatten (Flatten)            (None, 10368)             0         \n",
194 |       "_________________________________________________________________\n",
195 |       "dense (Dense)                (None, 400)               4147600   \n",
196 |       "_________________________________________________________________\n",
197 |       "batch_normalization_2 (Batch (None, 400)               1600      \n",
198 |       "_________________________________________________________________\n",
199 |       "dropout_1 (Dropout)          (None, 400)               0         \n",
200 |       "_________________________________________________________________\n",
201 |       "dense_1 (Dense)              (None, 400)               160400    \n",
202 |       "_________________________________________________________________\n",
203 |       "batch_normalization_3 (Batch (None, 400)               1600      \n",
204 |       "_________________________________________________________________\n",
205 |       "dropout_2 (Dropout)          (None, 400)               0         \n",
206 |       "_________________________________________________________________\n",
207 |       "dense_2 (Dense)              (None, 10)                4010      \n",
208 |       "=================================================================\n",
209 |       "Total params: 4,408,650\n",
210 |       "Trainable params: 4,406,666\n",
211 |       "Non-trainable params: 1,984\n",
212 |       "_________________________________________________________________\n"
213 |      ]
214 |     }
215 |    ],
216 |    "source": [
217 |     "CNN.summary()"
218 |    ]
219 |   },
220 |   {
221 |    "cell_type": "code",
222 |    "execution_count": null,
223 |    "metadata": {},
224 |    "outputs": [],
225 |    "source": []
226 |   },
227 |   {
228 |    "cell_type": "markdown",
229 |    "metadata": {},
230 |    "source": [
231 |     "# 2. Training with Mini Batch Gradient Descent\n",
232 |     "\n",
233 |     "The training part is no different from mini batch gradient descent training of feed-forward classifiers. I wrote [a Notebook on this technique](https://github.com/IvanBongiorni/TensorFlow2.0_Tutorial/blob/master/TensorFlow2.0_02_MiniBatch_Gradient_Descent.ipynb) in which I explain it in more detail.\n",
234 |     "\n",
235 |     "Assuming you already know how it works, I will define a function to fetch mini batches into the Network and process with training in eager execution."
236 |    ]
237 |   },
238 |   {
239 |    "cell_type": "code",
240 |    "execution_count": 6,
241 |    "metadata": {},
242 |    "outputs": [],
243 |    "source": [
244 |     "@tf.function\n",
245 |     "def fetch_batch(X, y, batch_size, epoch):\n",
246 |     "    start = epoch*batch_size\n",
247 |     "    \n",
248 |     "    X_batch = X[start:start+batch_size, :, :]\n",
249 |     "    y_batch = y[start:start+batch_size, :]\n",
250 |     "    \n",
251 |     "    return X_batch, y_batch"
252 |    ]
253 |   },
254 |   {
255 |    "cell_type": "code",
256 |    "execution_count": 7,
257 |    "metadata": {},
258 |    "outputs": [],
259 |    "source": [
260 |     "# To measure execution time\n",
261 |     "import time\n",
262 |     "start = time.time()"
263 |    ]
264 |   },
265 |   {
266 |    "cell_type": "markdown",
267 |    "metadata": {},
268 |    "source": [
269 |     "There is one big difference with respect to previous training exercises. Since this Network has a high number of parameters (approx 4.4 million) it will require comparatively longer training times. For this reason, I will measure training not just in *epochs*, but also in *cycles*.\n",
270 |     "\n",
271 |     "At each training cycle, I will shuffle the dataset and feed it into the Network in mini batches until it's completed. At the following cycle, I will reshuffle the data using a different random seed and repeat the process. (What I called cycles are nothing but Keras' \"epochs\".)\n",
272 |     "\n",
273 |     "Using 50 cycles and batches of size 120 on a 60.000 images dataset, I will be able to train my CNN for an overall number of 25.000 epochs."
274 |    ]
275 |   },
276 |   {
277 |    "cell_type": "code",
278 |    "execution_count": 8,
279 |    "metadata": {},
280 |    "outputs": [
281 |     {
282 |      "name": "stdout",
283 |      "output_type": "stream",
284 |      "text": [
285 |       "1.\tTraining Loss: 2.963158,\tAccuracy: 0.825\n",
286 |       "2.\tTraining Loss: 2.2184157,\tAccuracy: 0.875\n",
287 |       "3.\tTraining Loss: 1.6165245,\tAccuracy: 0.90833336\n",
288 |       "4.\tTraining Loss: 1.8334527,\tAccuracy: 0.89166665\n",
289 |       "5.\tTraining Loss: 1.2200023,\tAccuracy: 0.93333334\n",
290 |       "6.\tTraining Loss: 1.9942344,\tAccuracy: 0.89166665\n",
291 |       "7.\tTraining Loss: 2.1937194,\tAccuracy: 0.8833333\n",
292 |       "8.\tTraining Loss: 1.8208188,\tAccuracy: 0.89166665\n",
293 |       "9.\tTraining Loss: 1.6057307,\tAccuracy: 0.9\n",
294 |       "10.\tTraining Loss: 1.036152,\tAccuracy: 0.94166666\n",
295 |       "11.\tTraining Loss: 1.1800997,\tAccuracy: 0.925\n",
296 |       "12.\tTraining Loss: 0.9550361,\tAccuracy: 0.9583333\n",
297 |       "13.\tTraining Loss: 0.71287704,\tAccuracy: 0.96666664\n",
298 |       "14.\tTraining Loss: 1.1355927,\tAccuracy: 0.93333334\n",
299 |       "15.\tTraining Loss: 0.42137903,\tAccuracy: 0.975\n",
300 |       "16.\tTraining Loss: 1.2537751,\tAccuracy: 0.925\n",
301 |       "17.\tTraining Loss: 0.6678084,\tAccuracy: 0.96666664\n",
302 |       "18.\tTraining Loss: 0.67579913,\tAccuracy: 0.96666664\n",
303 |       "19.\tTraining Loss: 0.93005073,\tAccuracy: 0.95\n",
304 |       "20.\tTraining Loss: 0.9756758,\tAccuracy: 0.95\n",
305 |       "21.\tTraining Loss: 0.5439663,\tAccuracy: 0.96666664\n",
306 |       "22.\tTraining Loss: 0.48298022,\tAccuracy: 0.975\n",
307 |       "23.\tTraining Loss: 0.71461415,\tAccuracy: 0.96666664\n",
308 |       "24.\tTraining Loss: 0.59260106,\tAccuracy: 0.96666664\n",
309 |       "25.\tTraining Loss: 0.52771163,\tAccuracy: 0.975\n",
310 |       "26.\tTraining Loss: 0.70353717,\tAccuracy: 0.96666664\n",
311 |       "27.\tTraining Loss: 0.8188721,\tAccuracy: 0.95\n",
312 |       "28.\tTraining Loss: 0.5126863,\tAccuracy: 0.975\n",
313 |       "29.\tTraining Loss: 0.45719293,\tAccuracy: 0.975\n",
314 |       "30.\tTraining Loss: 0.54228246,\tAccuracy: 0.96666664\n",
315 |       "31.\tTraining Loss: 0.55225444,\tAccuracy: 0.96666664\n",
316 |       "32.\tTraining Loss: 0.9566125,\tAccuracy: 0.94166666\n",
317 |       "33.\tTraining Loss: 0.29893246,\tAccuracy: 0.98333335\n",
318 |       "34.\tTraining Loss: 0.70337826,\tAccuracy: 0.9583333\n",
319 |       "35.\tTraining Loss: 0.5612643,\tAccuracy: 0.96666664\n",
320 |       "36.\tTraining Loss: 0.59767526,\tAccuracy: 0.96666664\n",
321 |       "37.\tTraining Loss: 0.67508984,\tAccuracy: 0.96666664\n",
322 |       "38.\tTraining Loss: 0.67434996,\tAccuracy: 0.9583333\n",
323 |       "39.\tTraining Loss: 0.0228432,\tAccuracy: 1.0\n",
324 |       "40.\tTraining Loss: 0.2526902,\tAccuracy: 0.9916667\n",
325 |       "41.\tTraining Loss: 0.17682979,\tAccuracy: 0.9916667\n",
326 |       "42.\tTraining Loss: 0.4235972,\tAccuracy: 0.975\n",
327 |       "43.\tTraining Loss: 0.13609551,\tAccuracy: 0.9916667\n",
328 |       "44.\tTraining Loss: 0.5388846,\tAccuracy: 0.96666664\n",
329 |       "45.\tTraining Loss: 0.293708,\tAccuracy: 0.98333335\n",
330 |       "46.\tTraining Loss: 0.04402418,\tAccuracy: 1.0\n",
331 |       "47.\tTraining Loss: 0.5923631,\tAccuracy: 0.96666664\n",
332 |       "48.\tTraining Loss: 0.7310451,\tAccuracy: 0.9583333\n",
333 |       "49.\tTraining Loss: 0.41430146,\tAccuracy: 0.975\n",
334 |       "50.\tTraining Loss: 0.14125405,\tAccuracy: 0.9916667\n",
335 |       "\n",
336 |       "Training complete.\n",
337 |       "Final Loss: 0.14125405. Final accuracy: 0.9916667\n"
338 |      ]
339 |     }
340 |    ],
341 |    "source": [
342 |     "from tensorflow.keras.losses import CategoricalCrossentropy\n",
343 |     "from tensorflow.keras.metrics import CategoricalAccuracy\n",
344 |     "\n",
345 |     "loss = tf.keras.losses.CategoricalCrossentropy()\n",
346 |     "accuracy = tf.keras.metrics.CategoricalAccuracy()\n",
347 |     "\n",
348 |     "optimizer = tf.optimizers.Adam(learning_rate = 0.0001)\n",
349 |     "\n",
350 |     "\n",
351 |     "###  TRAINING\n",
352 |     "\n",
353 |     "cycles = 50\n",
354 |     "batch_size = 120\n",
355 |     "\n",
356 |     "loss_history = []\n",
357 |     "accuracy_history = []\n",
358 |     "\n",
359 |     "for cycle in range(cycles):\n",
360 |     "    \n",
361 |     "    df, label = shuffle(df, label, random_state = cycle**2)\n",
362 |     "    \n",
363 |     "    for epoch in range(len(df) // batch_size):\n",
364 |     "        \n",
365 |     "        X_batch, y_batch = fetch_batch(df, label, batch_size, epoch)\n",
366 |     "        \n",
367 |     "        with tf.GradientTape() as tape:\n",
368 |     "            current_loss = loss(CNN(X_batch), y_batch)\n",
369 |     "        \n",
370 |     "        gradients = tape.gradient(current_loss, CNN.trainable_variables)\n",
371 |     "        optimizer.apply_gradients(zip(gradients, CNN.trainable_variables))\n",
372 |     "        \n",
373 |     "        loss_history.append(current_loss.numpy())\n",
374 |     "        \n",
375 |     "        current_accuracy = accuracy(CNN(X_batch), y_batch).numpy()\n",
376 |     "        accuracy_history.append(current_accuracy)\n",
377 |     "        accuracy.reset_states()\n",
378 |     "        \n",
379 |     "    print(str(cycle + 1) + '.\\tTraining Loss: ' + str(current_loss.numpy()) \n",
380 |     "                  + ',\\tAccuracy: ' + str(current_accuracy))    \n",
381 |     "#\n",
382 |     "print('\\nTraining complete.')\n",
383 |     "print('Final Loss: ' + str(current_loss.numpy()) + '. Final accuracy: ' + str(current_accuracy))"
384 |    ]
385 |   },
386 |   {
387 |    "cell_type": "code",
388 |    "execution_count": 9,
389 |    "metadata": {},
390 |    "outputs": [
391 |     {
392 |      "name": "stdout",
393 |      "output_type": "stream",
394 |      "text": [
395 |       "12431.102645158768\n"
396 |      ]
397 |     }
398 |    ],
399 |    "source": [
400 |     "end = time.time()\n",
401 |     "print(end - start)  # around 3.5 hours :("
402 |    ]
403 |   },
404 |   {
405 |    "cell_type": "code",
406 |    "execution_count": 10,
407 |    "metadata": {},
408 |    "outputs": [
409 |     {
410 |      "data": {
411 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3gAAAEWCAYAAAA0DzVNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd8leX5x/HPlZxMyACSsEIMS2QooBFRECeKC+rGWke1Uv1VbdVaR91Wa4e22mGr1bpardraUrWOKmpdVbCCgqKgIHtvSMi4fn+ck8PJPknOySHJ9/16nVfOcz/389xXgibneu5l7o6IiIiIiIi0f0mJDkBERERERERiQwmeiIiIiIhIB6EET0REREREpINQgiciIiIiItJBKMETERERERHpIJTgiYiIiIiIdBBK8ERERERERDoIJXgibcTMFpnZkYmOQ0REJJ7M7DUz22BmaYmORaQzUoInIiIiIjFhZsXAwYADk9uw3UBbtSWyu1OCJ5JgZnaBmS0ws/VmNt3M+oTKzcx+YWarzWyTmc0xsxGhc8ea2Twz22Jmy8zs+4n9LkRERAA4G3gXeAg4p7rQzDLM7E4zWxz6m/ammWWEzo03s7fNbKOZLTGzc0Plr5nZtyLuca6ZvRlx7Gb2HTP7HPg8VHZ36B6bzWyWmR0cUT/ZzK41s4Whv5+zzKyfmf3GzO6M/CbM7J9m9r14/IBE4k0JnkgCmdnhwI+B04DewGLgidDpo4AJwJ5ALnA6sC507gHg2+6eBYwAXm3DsEVERBpyNvCn0OtoM+sZKv85sB9wENAd+AFQZWZFwL+AXwH5wCjgw2a09zXgAGBY6Pj90D26A38GnjKz9NC5y4EzgGOBbOA8YDvwMHCGmSUBmFkecATweHO+cZHdhRI8kcQ6E3jQ3T9w9zLgGuDA0BCXciAL2Aswd//E3VeErisHhplZtrtvcPcPEhC7iIhImJmNB/YAnnT3WcBC4OuhxOk84LvuvszdK9397dDfvTOBf7v74+5e7u7r3L05Cd6P3X29u+8AcPfHQveocPc7gTRgSKjut4Dr3H2+B80O1X0P2EQwqQOYCrzm7qta+SMRSQgleCKJ1Ydgrx0A7r6VYC9dX3d/Ffg18BtglZndZ2bZoaonE3wCudjMXjezA9s4bhERkdrOAV5y97Wh4z+HyvKAdIIJX239GiiP1pLIAzO7wsw+CQ0D3QjkhNpvqq2HgW+E3n8DeLQVMYkklBI8kcRaTvBpJwBm1gXoASwDcPd73H0/YDjBoZpXhsrfd/cpQAHwd+DJNo5bREQkLDSf7jTgEDNbaWYrgcuAkQSnIJQCA+u5dEkD5QDbgMyI41711PGIGA4GrgrF0c3dcwn2zFkUbT0GTDGzkcBQgn9bRdolJXgibSvFzNKrXwQTs2+a2ajQctK3A/9190Vmtr+ZHWBmKQT/yJUClWaWamZnmlmOu5cDm4HKhH1HIiIiwblwlQTnwo0KvYYC/yE4L+9B4C4z6xNa7OTA0N+9PwFHmtlpZhYwsx5mNip0zw+Bk8ws08wGAec3EUMWUAGsAQJmdgPBuXbV/gDcamaDQwuZ7WNmPQDcfSnB+XuPAn+tHvIp0h4pwRNpW88DOyJeBwPXA38FVhB8sjg1VDcbuB/YQHAY5zqCk9QBzgIWmdlm4EJ2DSsRERFJhHOAP7r7V+6+svpFcKrBmcDVwEcEk6j1wE+AJHf/iuCUgytC5R8S7PUD+AWwE1hFcAjln5qI4UWCC7Z8RvDvZik1h3DeRfDB6ksEH44+AGREnH8Y2BsNz5R2zty96VoiIiIiIh2YmU0gOFSz2N2rEh2PSEupB09EREREOrXQdIjvAn9QciftnRI8EREREem0zGwosJHgYjC/THA4Iq2mIZoiIiIiIiIdhHrwREREREREOohAogOIRl5enhcXFyc6DBERibNZs2atdff8RMfRXujvo4hI5xHt38h2keAVFxczc+bMRIchIiJxZmaLEx1De6K/jyIinUe0fyM1RFNERERERKSDUIInIiIiIiLSQSjBExERERER6SCU4ImIiIiIiHQQSvBEREREREQ6CCV4IiIibcDMHjSz1Wb2cQPnzczuMbMFZjbHzPZt6xhFRKT9U4InIiLSNh4CJjVy/hhgcOg1Dbi3DWISEZEOpl3sg9daby9Yy7tfrufyiXsmOhQREemk3P0NMytupMoU4BF3d+BdM8s1s97uvqJNApTd0uertrBhezlj+ncPl20uLee1+WuYPLIPAKXllZz94Hs8dv4BpAZ2PbsvLa/k2TkrOHnfvvz9w2UkmbF43XaO26c3KzeVMm5QXrjui3NXMrool7nLN7Nnzyz65mawanMpc5ZuYuKwnjw7ZznjBubx8ier6J2TTp/cDAbkdeHKp+dw3rj+DOuTDcCnKzcza/EGvlyzjf87bBDdu6SG2/hwyUYCScb2nZXkZqawpbSCZ+csZ9mGHVxz7FD653Xh3tcWUtgtgz17ZrGltJyS4u6s3lzKjdPn8ovTR5GekszPX5xPRmoyi9Zu4/C9Cthvj2689tkaTtm3kCUbtvObGQvYVlZJTmYKt5+4Nz954VOmf7iciqoqTi/pR8+cdAq7ZXL3vz/j6OG9+NrovhRkpfHXD5Yx49PVZKUH+Pcnq7h76mjGDcpjzZYyfvTcPK49dihvfr6WrukBvv3orPD3lRZIoqyiioe+uT9vfr6WbTsr2VZWwfTZywHo1z2DYb2zeXHuKvrmZrCjvJLrjx/KZX+ZTVZagHevPYId5ZWc+8f3+HjZZg4a2IO3F65j0vBe7NMvh1P2K6QgK50Zn67mmw+9z4Q983njszUAPDFtLFVVTkogia5pAd74bA0fLtnIso07uOaYobzyySr+8OaXHDw4j/98vjYcc7fMFDZsL2fCnvksWLWF5ZtK2bcolw++2shevbKYNmEAby5Yy98+WAbA5JF9mLp/P26cPpfPV28N3+fJbx/Iab9/h3GDerB331x+9/pCAN648jAm/GwGd502ksufnE1qIImdFVV1/vvumhZga1kF8380iZumz2Xh6m28t2h9+PxevbLYpzCHzTsqeGHuyhrXji7KZdOOcr5Ys42s9ABbSisAyEoPUF5ZRXpKMhu3l9dpc69eWXy6ckuNsqOH98QdXvl0NT2z0rh5yggueGQmYwd0590vgvEM75PN3OWbGTugOwf078Hdr3xe4x7987rw5dptfHvCAD5atoktpRV8tGxTONZn/m9cnVjixYJ/R+JwY7MHgeOB1e4+ota57wM/A/LdfW1910cqKSnx1mzketdL8/nVjAV8+ePjWnwPERGJPzOb5e4liY4jXkIJ3rO1/y6Gzj0L3OHub4aOXwGucveZtepNI9jDR1FR0X6LF2tv+I6s+OrnAFh0x67PMBc8MpOX563i5csmMLhnFgf/9FWWrN/BkJ5ZvHjZhHC9m6bP5aG3F3HJ4YP41asL6ty7+p6l5ZXsdf0LDOmZxfxVW8hODzDnpqOZ8NMZfLV+O//5wWEc/NMZda7/07cO4Mw//LfGvarjBSjZoxtPX3RQne+lIYvuOK5OnciyMw8o4oqjhrDvrS/Xe/1PT96HH/x1To2yv39nHF/7zVuNttuvewY3Tx7OeQ/V/ay56I7jGH7DC2zbWdnoPVqjd046FVXOmi1l9Z4fVNCVf19+SJM/v/Zs8sg+4YS4o3ru0vEM75PTqntE+zcynkM0H6KeoShm1g+YCHwVx7ZFRETaG6unrM5TWHe/z91L3L0kPz+/DcKS3c3yjTsAKC0P9ogsWR88XrBma416qzaX1vjakMqq4H9mX63fDsDmUE9I9XFZRf3JTXWPSUOWbtjR6PnmWrW5jPLKur1A1Tbu2FmnbFtZ4zFC8OfX2PcSz+QOYMWm0gaTO4AloX+HjmxxJ/geq/9/bQtx68GD+p9UmtnTwK3AP4CSturBu+fVBTWefomIyO6nk/fg/R54zd0fDx3PBw5tbIhma/8+thdL1m/n+n98zG/P3JfM1ODskorKKi594n9cdMgg9i5s3lPxu17+jH7dMjhlv0KOvOt1Fq7ZxtkH7sEtU2r+s/xz9nI+WraJa48dWqO8uiflv9cewZVPz2HS8F5c+8xHFHbL4MYThnPBIzP5ycl78+ycFZw4ui8ffLWBH31tbwDcnf7XPM/J+xZy5tgiTvrt25Ts0Y0rjx7CmwvW8qtXF/C1UX345dTRvPDxSi58bFaNtk8rKeTJmUsBGJDXhRsnD+ecB99r1vdf7cihBazbtpPM1GTeWrCuRfcQkegct3dvfnNm69bO2h168Oows8nAMnefHUXdaWY208xmrlmzprUNt+56ERGR+JsOnB1aTXMssEnz74J+/K9PeG3+GmZ8uuvzwOL123n+o5V894n/Nft+97zyOVc+HRzKt3DNNgAeeafuUNdLHv8f973xRYP3+e2MBbzx2RqufeYjINhjdcEjwYT7qr9+xH8+X8vlT87msXd3DVoKdZbx1w+WctJv3wZg5uINnP3ge+FhlH//MDhUrXZyB4STO4Av1m5rcXIH8O9PVvO/rzYquRNpA8991Ha/zttskRUzywR+CBwVTX13vw+4D4JPKOMYmoiISNyZ2ePAoUCemS0FbgRSANz9d8DzwLHAAmA78M3ERLr7iddgozgOYmq23SgUEWnn2nIVzYFAf2C2BXvUCoEPzGyMu69s9EoREZF2zt3PaOK8A99po3B2W9XDH39+6khO2a+wxrn6BuQ4cPidr7Fpezmzrp8Yvsek4b1Yv20n7y1az4PnlnD4Xj1ZsWkHB/741fC1A659vt4YTvvdOzVW8rvw0Vl1VvADeLieXr+GXPHkbP76wdIGz9deYfDoX7wR9b1FRCK12RBNd//I3Qvcvdjdi4GlwL5tkdxpgKaIiEj78vf/LQu/b6qn7Ys121i3reYCGy/MXRlO0l74OPhR4/NVW+tcW5/I5K76Xq3VWHJXn/mrtjRdSUSkHnHrwatvKIq7PxCv9kRERKTtrd5Sysbt5ezZM6tF13+6cjN5XdNYvbmM3jnp4XKvZ9DijE9Xs2JTKbkZKazdGlx18Mu122rEUpCVXue6FZtKmb9yC7+eUXergEjj7niV8RF7w4lIfBhVBKgimUoCVJJMVfhrMlUkW83yOnVt17mkiGt3HVeRZLXuGbpHjePwfarC90mqdZ9kq+e6WscBi2i3dhyh4zsrTgXaZsHHuCV4UQxFKY5X2yIiItI2xt3xKuWV3uKVqif98j/kZKSwaUc5fXMzwuWRvXbVyd5TsxrvBRt3x6t8ftuxdcr/8/lajv5l00Mel23cwV9mLokycpHdhZNKBWmUk0p58L3tJJWKiONyUqggQCUpobopVJJkVaRQSRoR9a2CFCpIjaifYpUEQmUpVJBCZSjhqiLVgu2mhOqmUkGKVZBcI9nxULJVGUq+dp9Zp+UejLSCJCpJCkWZtOvltY5D5ytICl0XvL7MU9gROt5Vz8L119K6PfCaoy3n4CWcu2NaUVNERCRmyitb/0Ft045yIJhgJToWkaZ5jeQpJZTQ7Ep+qt9XhsrLSaOcdHaGE63qhCzNdgbLa50PUEWACtLZSbqVkxaqk0pFKKHaldClWdN7/TVHpRs7SaGcZCpIppwA5QSo8OD7nQQoD/VhVZDMdk9jI13ZSYAKkoNfqwKhJGhXclQRToySqfTkUEIV+TUi0fKa5fVdX5081ddGzcSsZqIV2Y637YYCbaZTJHjK6URERBLnpbkrqXJnRN8cxv9kBj8+aW9unD6XeTcf3eA1by9cx+otpfzo2U94ce6qqNuqXqRFOhMngzK6UEa67SSNYNJUnTClRxxXJ0vplNeomxY63lW3OvkKJVcWTNIyKCODMpJj2AO1w1MpI4VSUinzlFBCFUygSkml1FPYRBfKSGEnAXZWBb+WkRo8JoUyD4TOp4SOU8Lnwl89mIBVhO5bHk60kkP3C15bSXLMvjdJjE6R4ImIiEjiTHs0uJ9bVnrwY8c1fwvuG/fqp6sbve7e1xYyffby+AYnbczpQilZbCfbttOFUrpYKV0oJTP0PpNSMq2MdHaSQRmZlJFhwfddbUfwPGXh67pQ2uIhf6UeTKyqk6vq96WkstUzWEcOpaSw01NCQ/DS2E4aOzw1nDxV93Dt9EC45yuyrJxA8P6hxKv6XGkoQdNygBJrnSrBc1dvnoiISKLs2FlZ47iqic/km7aXxzEaaYlUyslmO1m2PZykZYWOsyO+Ztc+H/E+2t6vUk8JJlOkscPT2EEq28hglXdjO+lsr0pjGxlsJZ1tns520kM9Xrt6xEo9NZxc7TpOoSxUpuRKOqJOkeCZ/ucVERGJ2urNpYy5/RWemDaWsQN6hMtfmruSaY/OYs5NR5GdnsIrnzQ8dLJ6qGQgadff4IpaGd2Fj81qNI6/RWyVILEVoIJubKWbbaGHbSaXreTaVrqxhe62JVjOFrqHznWxUrpSSpo1nnRXubGVDDaTyRbPZDOZLPcebKEfm6syI8q7sMUz2E46W0PJ2TbS2eYZoaQutcPOjxKJt06R4ImIiEj03l+0AYBH3llUI8Gr3mbgyzXbGNkvt8ltB6BuUifxkUQVPdhMgW0kP/TKZSvdbCt5bKKHbaaHbSaHrXS3LeTY9gbvtdXT2eBZrCeLdZ7NQvqwrSqdrWSEk7b6vm4hk62kKzETqcfNk4e3WVudKsHTnxgREZGmVW9LUF7pvLNwHQcO7FHj/KYd5WzaXs7GiCGU736xjuQkIzM1mczUTvXxIq7SKSPfNlLARvJtEwW2IZjEEfHeNtGDTfUOfdzpyawjh/UeTNa+ooB1VdnhBC7y6wbvyka6UkZqAr5TkY4tNzOlzdrqFL+BNe9ORESk+V6et4qX563ixyftzRljisLlZz/4Xp26U+97ty1Da+ecXLZSYBtDydrGXe8jyvJtI9lWd+uICg/uqbXac1np3ZlTNYDV5LLGq185rCaXjd6VrWSgeWYiiRdIarue7U6R4ImIiEjLvffles4YU1Rj83FpXBd20M/WUGSr6GerKbS19Lb19LPV7GGr6Gqlda7Z6umh5Kwbn3gRb1Ttw2rvxppQMrfGc1ntuawnS8MgRXYTeV1T+fqYIu55tfEh68lJbfegpVMleO6OnmKJiIg0jxl8vGwTHy3blOhQdhvJVNLb1lFsq9gj9Cq2lRTYBgptLXm2uUb9LZ7BSu/OUs/jvaq9WOIFrPJuwcSNHFaHVoYUkfZlwp757Ffcvcl6KclK8GJKKZ2IiEjLGcbxv3oz0WG0uWQq6WerGWAr6Gdrwr1vA205/WwNKbZr24dST+ErL2Cld+elqj1Y4j35ygvCr010QZ9IRGKne5dUzj2omLte/qxV9/n6AUX8+b9fhY+vPmYv7vjXpzXq7NmzK5+t2srpJf34y8wlNc4Fkox9+uY02c7oom6tirM5OkWCJyIiItGrPRSzDUcWtZCTz0YKbS29bD19bB0FtiG0L9sOstjOTgL8rOJ0PvN+Na5MpyzcA1dkqym2leGvfWwdAasK193uaSz2AuZ7P16oGsMi78niql4s8p6sJlfDJqXT6ZubwbKNdeeJAowszGH20l29/u9dewRjbn+lyXt+fPPRjLjxRQAW3n4syUkW3nal2qI7jgu/P2vsHoy+9eV67/Xlj4/FQotxVN8jNzMlvEDUpYcP4vKjhnD7iXuHz194yEDOGrsHw298kYyUZD65dVL4fuu2loUTvH7dM1iyfgfJSUa3Lqk1Ypr86zeZE/reX/v+oRTndWny+44lJXgiIiISdudL83nuoxU1yp6atTRB0dTHKbLVjEn6lH3sC4YkLWGILSHXttWoVeYpbArttbaFTIbYEiamfcCXVT3ZQToBKsixbfS0jTWu2+BdWewFfOiDmF51EIu8F19U9Wax92Qd2agXTmSXjNTkOmVd0wJsLasgPaXuuWh4xBOmaKZXNbaYonvd85EPsJIaeHqVFLqoS1rT30N932fkXZMSsNpjp0rwNDdcRESkcb9qYqGARMiklEOSZjMxeRYHJc2llwX36dvsGcz3fjxXOZbPvJCvvIAV3oMV3r3OkMgebOKk5P8wOmkBKVRSTjJbqzJYHBpKuch7sdgL2EzXBH2XIvW7e+oorvrrHErLq5qse+EhA/nd6wsbrfPdIwbzz9nL+foBRYzom8PU+97l4sMGRbWvZbX8rDQmDuvJNcfsxZjbXmFH+a7hyn//zjhem7+aE0b24fq/f8xL81YBNffEzMlIwd25/+wSTr/vXabu348n3g/2jEV+Xg8kB3vFb5kynBv+MbfeWHIz627rccp+hezVK6tGAnfk0AL+/clqAPK6prF2axnHjOgdPn/XaSPp1z0TCCauPzx2KEcMLahx3x5d08LvJ4/sQ5XDdw4bVDeoUFJ35NCe9OueUW/c8dQpEjxtkyAiItK+BKjgsKQPOS35dSYkzSHNylnvXXmragT/rRrKf6uGssD7RD0sch053F95PFQ2XVc6nmgSn6bUHnIYKT0lKaoELNJ+e3RjS2k5n63a2mCdBbcdQyA5idc/W8PfPlhW41x1T1m1cw8q5upj9uJfH69g8bpdG9kPzO/CwjW7ergvm7gnl03cM3y86I7jWLJ+e9QJXuRQRIBPbp1UYwjloIKuDCoIPii57+yS8LnSiCRw3KAe/PbM/Wrc77k5K9hSVlHvar1nH1jcYIIHsHffHD5atomxA7rz7hfrGVTQlW8dPKBGnZ+fOpJRt7yMu5OdEWDt1jKSI359nLRvYY36F0yoeX1thnHVpCENnAv6v8MGhoeItiUNFhcREZHdRh6buCLwJG+nXcr9qXcxMmkhf6o8gqk7r2P/snu5pPxSHqucyOdeqDlvErXiHplN1slOD4QTk/qMaGQhjfGD8uqUNTV3dVjv7HqTmfrygRF96rY9uiiXPjm7Vl6tvm5kYW6t+zWdYGTWM9Qy1iKHMg4uyKpzfr/i4CIkLVltckTfbGDXQiZF3ev+e6cGgr8v9t2jW/hn1CWt5X1d/RuZV1cdT25G221uHqlT9OBV0/49IiIiu6dCW8O05Gc5Lfk1Uqng1apRPFF5ODOqRlFJ/D98SuKcul8h+/fvzg+entPie7z3wyPISEnm9uc/4fH3ltQ5/7XRfRmQ35XF67ZxZT3tPHb+AQztnUUgKYkFa7awfWclW0sr6JWTTnmls0ePTNIDyfwpYrXFSL8/q4SB1z4fPr7vrP0YXdSN/W/7d7js3IOKeejtRQDcc8ZoJg3vxXH3/KfGfZ75v4Pon9eFUbcEFw2p/uh63vj+/O71hdxx8t7kdU1jZ0UVe/XOpryiinMfep/ZS3bNJf3pKftwakkhL81dxaPvLo5q1miPrmlMv3gc3/nzByxZH1w05d+XH8KRd70OwAvfO5iv1m2nqIFE+YKD+3P/f77k/PH965x786rDAOiTm8HfvzOO7WUVHDCgR516v/n6vny5dhuZqfWnJ//5wWEs27iD4h51E6ubJg/n9P2LGFmYw2FDCti/uO6KlZmpAZ69ZDz987qQnGScc1AxvXOaP3xy3i1H88R7Szh5v8IG69xw/HBO2reQAfmJGfLdKRK8RHSNioiISNOy2colgb9zTnJw1by/Vk7g95XHs8h7N3GldBRnjt2DUf1yW5XgFWQFe7J+fNI+9SZ4ZjCmf3dWb6m7wTzA+MG7euD226P+Pc0ih0MeNaxneG4Z1N3EemBBV/Kz0mqUnbJfYTjBmzyyT53752elhXugAklWY84awHs/PLJuUGlwwj69mb1kIxZK5dJTkjl4cD75WWnBBC/Kj8H7FOaS3zUtnOBF9mbu1SubvXplN3ht9Ty46l6ySIXddiWFo/rl1jlfrUtaoNFe0n7dM8Nz5GpLCySH7z2mf8N70kXev7FYGpOZGuC8ehLZSKmBJPZtw20Raovb2AYze9DMVpvZxxFlPzOzT81sjpk9Y2Yt+8mKiIhIu2ZUcVbyS7yedjnnJ/+Lv1UezISyX3JNxQVK7joZb2CIVfcuuxbPOGpYT745rrjG+QsPGRh1G9XJzz59W/7RMzLOE0f3Db8/dEg+QI1eo4KI5C41NNGrb24GuZkp9fYu1b7nrjabjuuggcHkdOKwnvVeaxF9eKc00utUn8EFXSns1nQv14TBwZ/B4XsVNFEzOkcP79l0JWlQPHvwHgJ+DTwSUfYycI27V5jZT4BrgKviGEMNrnU0RUREEm6ALeeOlPsZkzSftyqH86OKb/CJ75HosCRKi+44rs6+ZPsU5jBn6Sb26pXFpyu3hMvn/2gSQ657AYBfnD6Sy/4yO3xuZL9cZi/ZGP50Vvu+H1w/MXx839klANx4wvAa7Ta2cMqiO45j4LXPU1nl4flwRT0yaywSUvv7aEx1h1pWWoBj9u7NmOLuvLdoPReFEs2nLjyo3hgifXjDUQ3e/5gRvcLvmzP4bFif7DrtQESCZ7v+fb4xtvH/z6r/Lf560YEAvHz5IVHFsHdhTr0xtNTvzyqJ2b06o7gleO7+hpkV1yp7KeLwXeCUeLUvIiIi0bvrpfkcHfEBMz6c85Jf4KrAE+wglct3Xsjfqg5Ge8t1XNbIv231mXiukVAVunlMputUx9mG/7nGqnOi+SHr/8n2LJHLT50H/CuB7YuIiLQZM5tkZvPNbIGZXV3P+T3M7JXQNIbXzKx5Y6laoayiknteXcBx97wZtzay2cr9KXdxQ8qjvFG1DxPLfsbfqiagD5Ktk9e17h5gAN+utcT7zZOHc8yIXvzjO+NqlGc0czPq331jv3rLp00YwNHDe/LweWPom1tzSN9Fhw7kgXPq9sjcedpIpozqwz6Fu+ZFXXfcUAYXdOXnp44EgoufnDeu4flOj18wlnMObLhX6qlvH8iZBxQ1uKLleeP689j5BzR4faTsjABnjOnHo7XqtyYF+/XX92WfwhyO26d3jflh1Ylxa5LfPXt25cTRfbl76mjuPG0Uk0f2YXifhufRQXArgckja/6bSPuTkEVWzOyHQAXwp0bqTAOmARQVFcWkXa2iKSIiiWBmycBvgInAUuB9M5vu7vMiqv0ceMTdHzazw4EfA2e1fbSxN9y+5Pepv6CADdxcfhbayekxAAAgAElEQVR/rJyEErvYeGLaWI6864065dccO5Rrjh0aHoJ4zkHFnHNQcY15ZIvuOI6qKmdAxOqPANcfP4xbn51HZmoy23cG9y7rm5vBW1cfXqedAwf04J0v1tEtMzU8rO6tqw9n0LXPhxcJuWrSXgA887+lNa4dmN+Vu6eOrlH2rYMH1Ni/bPzgvBoLoNRpf2APDhxYd0XGaiXF3SkpbnjRjRtOGNbgudrMjB+ftE9EQdSXNmhIryymXzy+nsZaf+9AchK/OH1U+PieM0Y3UjtoYH7XqOrJ7q3Ne/DM7BzgeOBMb2hWLeDu97l7ibuX5Ofnt7LNVl0uIiLSWmOABe7+hbvvBJ4AptSqMwx4JfR+Rj3n26WJSTN5KvUWAE7deSN/rDwGJXexkxZo3RYS9X1G6hFa3CRyOfrae35VX9c3tABH7f3EqutH3j83o/7exlgrqLV6ZdzEseNgQD0/P5FotWkPnplNIrioyiHuvr0t2xYREUmgvkDk2u1LgdrjwmYDJwN3AycCWWbWw93XRVaKxwiX+HDOT/4XPwz8iTk+gAt2XsEatHh2pOuOG8qPnvsECA6hLOqeyS/+/Rlzlm6K6vpLjxhMv+6Z/O4b+/Gvj1fwjw+X16nz2vcPZfWWsgbvETk37Wuj+nDokAKmjOpDVnqAkf1yWbmplHMefI8Hz92/xnVvXHkYSzZsZ1S/XA7ZM7/OkvN/vmAsHy3bSEryrr6EQ4fkc/fUUQwuyGJHeWVU32NL/POS8TUWemmPHvvWAcxZurHVCbx0TnFL8MzsceBQIM/MlgI3Elw1Mw14OfQL5V13vzBeMYiIiOwm6nsOX/v5//eBX5vZucAbwDKC0xlqXuR+H3AfQElJSUz6EBpbCKNlnCsDf+E7gek8XzmGy8svopQ26lXZzT14bgnnPTQToMbQwnMOKgbg89Vbok7w9i0KJlWTRvTizQVr6q1TnNeF4ry6G0PXJyU5ia+Fluo/Ymhwmfq8rmnMun5inbqRe5Kd0MCebofvVXOpezNjyqi6WwHEWs/sdHpmp8e9nXh2ROd1rfvzE4lWPFfRPKOe4gfi1Z6IiMhubCnQL+K4EKjR3eLuy4GTAMysK3Cyu0f3SX+34twQeJTzAi/wp4ojuK7im3hC13RrO/26Z4Q3iW5IUcRGzbU3wobgZtPR2qNHdIlbfU6qZ881DQdsniP2KuC9L9fXWVRGJNGa/I1rZhebWeK2Yo+B2D+ZFBERaZb3gcFm1t/MUoGpwPTICmaWZ2bVf5evAR5s4xhjwLkt8CDnBV7ggYpj+GHFeZ0muTtkz3xevuwQxg7YtaDHa98/lNk31tz3rKh7F2ZddyT/u34iBVl1e5nGDujBbSeOAGB4n2zeueZwnrs0uAhH5MbfH1w/sc68uGiYGbNvPIqfnrJrsZBrjw0ugqLPS80zbcIA/nf9xHBPpsjuIprfur0Irvb1ZGiJZ/3fLyIi0gzuXgFcDLwIfAI86e5zzewWM5scqnYoMN/MPgN6ArclJNgWc64J/JkzA69wb8UJ3FrxDTrTYipd0pJJT0mma8RiI8V5XcjJSKlRr8qdHl3T6Nal4QVHqhc5KeyWQe+cDLLSgvfITN01H6t7reubs1J4TkYKgYi5cVnpKY3UloaYWaP/jiKJ0mSC5+7XAYMJDq88F/jczG43s4Fxji3mtE2CiIgkirs/7+57uvtAd78tVHaDu08PvX/a3QeH6nzL3RteGSPGYvHo9qLkf/LtwHM8XDGRn1RMpaMmd5ccPggIDrU8NzRvDuDCQ4Ifi846MFjW0M80NbnmR6/DhuRz+cQ9o2rbDH52yj4M7d3wXmaF3TIYN6jhbQOaur+ItH9RzcFzdzezlcBKghO+uwFPm9nL7v6DeAYYC/qFJSIiEj9Tk1/lqpQn+HvlQdxUcQ4dNbkDuOKoIVxx1BAAnp2znIfeXsSxe/cKz50bXNAVgF71LPKx6I7j6pT98ZtjGmyr+sG0R6zHc2pJP04t6dfAFfDtQwZy1tiGN/5urB0R6RiaTPDM7FLgHGAt8AfgSncvD80T+BzY7RM8ERERiY+Dkj7mR4EHmVE5ku+XX9hp5tzBrjlrkQlSclKwLLtVwx5rJsjV96w93DNS9T50aYHm//yrr6m9l52ItE/R/J+cB5zk7osjC929ysyOj09Y8eHx3JFSRESknWppf1uxreC3KXez0PtwSfklVLTt9roxddiQfGbMX8NBA3vwncMGceYf/gvAP74zjlN+9zYnjOzDkUNrL/tf9z49s9O56YRhHDW8V7js/rNLyM1sfsJX/amlsFsmNxw/jGP37t1g3e8dOZistEC9q2M2ZcqoPqzYtIPzxvdv9rUisvtp8jexu99gZvua2RSCv2vecvcPQuc+iXeAsdBxB4qIiIi0Xksef2azjQdSfk4lSZxf/n22svuuJPj0hQdyyu/eafD8RYcOZOyAHsyYv4bkJGPcoLzwuZH9cvn8tmMbvX/tIY7njquZKE0c1vr9zJpKvjJTA1xyxOAW3TuQnMTFh7fsWhHZ/USzTcL1wMNAD4K9eX80s+viHZiIiIjsrpw7U+6ln63mop3fY6kXJDqgRjU1F39wQVd6Zgf3pBvZjH3oqvc/G9G34UVPRETaWjQDtb8O7O/uN7r7jcBY4Mz4hhUfmkQsIiLSet9Kfp6JyR9wW8U3eM+Htlm7/732CLLTdw0+GtbAapLTJgzgb/93UJ0VKxty4ui+7NUrm+cvPZjLolzREoK9e89dOp7/O3RQ1NdEQ4vDiUhrRPObbxEQuRRUGrAwLtHEiX5RioiIxMa+9hlXBZ7gX5X783DlUU1fEEM9s9PJz0oLH19xVP3JWP+8Luxb1I3h4Z61xj8IVG/xO6xPdnhBk2gN75NDUjOviZYeTItIS0ST4JUBc83sITP7I/AxsNXM7jGze+IbnoiIiOwuctjKr1J/xQrvzlXl00jELHeLeGq7d2FOvXXGDgjuA1e9T92AvC4N3u+KZvTYiYi0B9Esd/VM6FXttfiEEn96ECYiItJyt6b8kXw2cnL5zWym4aQplhbdcRzFVz9Xp/ylyyZQkJVe7/n+oYRuyqi+TBnV+KqSLV2YRERkdxXNKpoPm1kqUP2Ia767l8c3rNgyraMpIiLSKsclvcvk5Hf4WflpfOQDEhZH9V/0jjx8cdenlg78TYpI3ESz0fmhBFfRXETwd04/MzvH3d+Ib2giIiKyO8hnI7emPMiHVQP4XeUJcWvnB5OG8NMX5jO6KJeTRvdl285KAG4/cW+6d0kFGp9X/9j5B/Dpys2NtnHr10bQOzud3MwU3lm4Lmaxx5Jp8QARaYVohmjeCRzl7vMBzGxP4HFgv3gGJiIiIrsD5/aUB8ikjCvKL6KS5Ji38N0jBodXr6xvRcqvH1BUT1R1e7fGD85j/OC8OuWRzhq7R/h9SXH35oYqIrLbi2aRlZTq5A7A3T8DUuIXUvx4Rx7PISIiEgcnJL3DxORZ/KziNBZ64/PZWqpXTnrTlUIG5ncFIDMlmmfUIiKdTzS/HWea2QPAo6HjM4FZ8Qsp9jTSQUREpPmy2cYNKY8yu2oAf6w8ptnX53VNY+3Wshplt504gh8+83GNstNL+kV9z5+fOpLT9+9HUY/MZsfTHM9dOp4uqYlNIvVcWkRaIpoevIuAucClwHeBecCF8QxKREREEu/7gSfpzmZ+WH4eVVF9ZKhp36LcOmXVWxhEas4+cl3SAhw6pKDZsTTX8D45FDeyvUI86bm0iLRGo7+tzSwZeMDd73L3k9z9RHf/hbuXNXbd7koPwkRERKIz0hbwjeR/80jlUXzcwlUzjxzWM8ZRNdLW0MaTvoObmJu3OxlUEByGOrENf34i0nE0OvbA3SvNLN/MUt19Z3NubGYPAscDq919RKisO/AXoJjgqpynufuGlgQuIiIisVF7KGASVdyW8iBryOHOilOjukdWWoD3rzuSva5/AYB5txxNZmqAHzw9J1znt2fuG37fNzeDZRt3tD544NNbJ5GS3HgP40PfHEN5ZVVM2ou34rwuzLvlaDJSYr+gjYh0fNGMt1gEvGVm15vZ5dWvKK57CJhUq+xq4BV3Hwy8EjoWERGR3cjpyTMYkbSIW8vPYivRz3VLj0hIMuuZv5YW2PWxIzXQ/CGfjbWb3MQwz+QkqxHf7i4zNaDtEkSkRaKZPbw89EoCskJlTY52dPc3zKy4VvEU4NDQ+4eB14CrooghJjRZWUREpHFd2c7lgad4r2oIz1aNjfq6e84YXW/5OQfuwdqtO1m4ZisHDOjBqs2lQHCe2cWHDWo3vWoiIu1FNAnePHd/KrLAzKIbr1FXT3dfAeDuK8yswQHzZjYNmAZQVFR3/5vm0BMwERGR6FwUmE6+beb8nVfSnOU+Dtur/j/pN08ZUeN4VcT77x89pAURiohIY6IZH3FNlGUx5e73uXuJu5fk5+fHuzkREZFO66NlmwDoyxq+lfwv/lY5njk+MC5tpYbmynXvkhqX+4uIdHYN9uCZ2THAsUBfM7sn4lQ2UNHC9laZWe9Q711vYHUL79MyGqIpIiJSx+3PfwLAVSlPUIXxs/LT66138+Th3Dh9bp3yf18+Ifz+6QsPpKKq4T+4/bpncsdJe7fpCpsiIp1JYz14y4GZQCnBjc2rX9OBo1vY3nTgnND7c4B/tPA+zaIBmiIiIg1zd4bbl0xOfof7K49lBXX3qgM456DiOmX3nrkvgwqywsclxd3r3esu0tQxReR1TWtVzCIiUr8Ge/DcfTYw28z+7O7lzb2xmT1OcEGVPDNbCtwI3AE8aWbnA18BLZ3LJyIi0q6Y2STgbiAZ+IO731HrfBHBBchyQ3Wudvfn2yI2B64IPMVG78L9Fcc369p+3aNfZVNEROIvmkVWxpjZTcAeofoGuHvju566+xkNnDqiWRGKiIi0c2aWDPwGmAgsBd43s+nuPi+i2nXAk+5+r5kNA54nuG9s3O25cx6HJ3/IHeVT2RLltgin7lfI/x02iP55XeIcnYiINEc0Cd4DwGUEh2dWxjec+HJNwhMRkcQYAyxw9y8AzOwJglsHRSZ4TnCeO0AOwakSbWLq1kdY49k8XHlU1NeYoeRORGQ3FE2Ct8nd/xX3SOJIuySIiEiC9QWWRBwvBQ6oVecm4CUzuwToAhxZ341iuY0QAF+8zuiKOdxccRY7SG+wWnZ6NB8ZREQk0aLZJmGGmf3MzA40s32rX3GPTEREpOOo71Fj7WElZwAPuXshwVWsHzWzOn+nY7qNkDvMuI0V3p0/VzY+g2LOTcH11X5y8t6ta1NEROIqmsdx1U8YSyLKHDg89uHEl2uEpoiIJMZSoF/EcSF1h2CeD0wCcPd3zCwdyCOeWwqVboKkAL+qOJEyotuXLik0LCY5KZpnxCIi0taaTPDc/bC2CCSeNEJTREQS7H1gsJn1B5YBU4Gv16rzFcGFyB4ys6FAOrAmrlFl5MK5z/H4Nc82Wu17Rw4Ov58yqi+frNjCd48Y3MgVIiKSKA0+fjOzX0a8/26tcw/FMaaYe+6jFQCs2FSa4EhERKQzcvcK4GLgReATgqtlzjWzW8xscqjaFcAFZjYbeBw4170Nxp6Y4U3M2PjekXuG36cGkrjhhGHkZKbEOzIREWmBxnrwJkS8P4fg3j3V9olPOPHx/qINAHy6cjPD+mQ3UVtERCT2QnvaPV+r7IaI9/OAcW0dl4iIdCyNJXjWwHsRERHpBGZ8/1C2llYkOgwREWmGxhK8JDPrRnAYZ/X76kQvOe6RxYEWWREREYme9rkTEWl/Gkvwcghubl6d1H0Qca5dpkrtMmgREREREZEoNZjguXtxG8YhIiIiIiIirdSsTWzM7KY4xSEiIiIiIiKt1NxdSic3XWX31RarTYuIiIiIiCRKcxO8dr2aptI7ERERERHpyJqb4O0XlyjaSLvOTkVEROJg5abSRIcgIiIx1GSCZ2Y/NbNsM0sBXjaztWb2jTaILebUgyciIq1lZheHtg7qEP775bp6y1/43sFtHImIiMRCND14R7n7ZuB4YCmwJ3BlXKOKF2V4IiLSer2A983sSTObZGYdcoDIXr2yEx2CiIi0QDQJXkro67HA4+6+Po7xiIiI7Nbc/TpgMPAAcC7wuZndbmYDExpYC5WWVyY6BBERiaFoErx/mtmnQAnwipnlA+1ywL6rC09ERGLAg8syrwy9KoBuwNNm9tOEBtYCZRVViQ5BRERiqMkEz92vBg4ESty9HNgGTGlNo2Z2mZnNNbOPzexxM0tvzf2ipV0SRESktczsUjObBfwUeAvY290vIrgQ2ckJDa4FOuT4UhGRTiyaRVZOBSrcvdLMrgMeA/q0tEEz6wtcSjBhHAEkA1Nbej8REZE2lgec5O5Hu/tToYefuHsVwfnq7d7A/C6JDkFERFoomiGa17v7FjMbDxwNPAzc28p2A0CGmQWATGB5K+8XlY45DV5ERNrY80B4PrqZZZnZAQDu/knCooqhf14yPtEhiIhIC0WT4FXPvj4OuNfd/wGktrRBd18G/Bz4ClgBbHL3l2rXM7NpZjbTzGauWbOmpc3VajsmtxERkc7tXmBrxPE2Wv/gM2G+XLs90SGIiEgMRZPgLTOz3wOnAc+bWVqU19UrtHfQFKA/waGeXerbV8/d73P3Encvyc/Pb2lzNe8Zk7uIiEgnZ6FFVoDw0MxAAuNplQff+jLRIYiISAxFk6idBrwITHL3jUB3WrcP3pHAl+6+JjRv4W/AQa24n4iISFv6IrTQSkro9V3gi0QHJSIiAtGtorkdWAgcbWYXAwX1Dalshq+AsWaWGdoc9gigTeYsaIimiIjEwIUEH0wuA5YCBwDTEhpRjCVp0rqISLvV5JCS0JPJCwj2tAE8Zmb3ufuvWtKgu//XzJ4GPiC4d9D/gPtacq9mt61BmiIi0kruvpoOvvpzekpyokMQEZEWimbOwPnAAe6+DcDMfgK8A7QowQNw9xuBG1t6vYiISKKE9m49HxgOhPdxdffzEhaUiIhISDRz8IxdK2kSet8ux25oyImIiMTAo0AvglsHvQ4UAlsSGlEMXXr4oESHICIirRBND94fgf+a2TOh468BD8QvpNj79iED+P3rXzC8T3aiQxERkfZvkLufamZT3P1hM/szwcXI2r1/XjyevQtzEh2GiIi0QjSLrNwFfJPgpq4bgG+6+y/jHVgsbdi2E4A5SzclOBIREekAykNfN5rZCCAHKE5cOLGjueoiIu1fowmemSWZ2cfu/oG73+Pud7v7/9oquFh554t1AHyweEOCIxERkQ7gvtCertcB04F5wE+ausjMJpnZfDNbYGZX13P+F2b2Yej1mZltjH3ojatSfici0u41OkTT3avMbLaZFbn7V20VVKxVz72r0j4JIiLSCmaWBGx29w3AG8CAKK9LBn4DTCS4tcL7Zjbd3edV13H3yyLqXwKMjmXs0dDfSRGR9i+aOXi9gblm9h6wrbrQ3SfHLaoYq15apUKPJkVEpBVCDz4vBp5s5qVjgAXu/gWAmT0BTCHY+1efM0jAatOuBE9EpN2LJsG7Oe5RxNnyjaUAPDtnBb/+eoKDERGR9u5lM/s+8BdqPvhc38g1fYElEcfVG6TXYWZ7AP2BVxs4P43QxupFRUXNCrwpeg4qItL+NZjgmdkgoKe7v16rfAKwLN6BxdLOyqpEhyAiIh1H9X5334kocxofrlnfPj0NpVNTgafdvbK+k+5+H3AfQElJSUxTskpleCIi7V5ji6z8kvr39dkeOiciItLpuHv/el5NzcVbCvSLOC4EljdQdyrweCxiba6+uRmJaFZERGKosSGaxe4+p3ahu880s+K4RSQiIrIbM7Oz6yt390cauex9YLCZ9Sc4CmYqUGfSgJkNAboB78Qg1Gbr1z0zEc2KiEgMNZbgpTdyTo/4RESks9o/4n06cATwAdBggufuFaHFWV4EkoEH3X2umd0CzHT36aGqZwBPuFY7ERGRFmoswXvfzC5w9/sjC83sfGBWfMOKreQkC88rcHfM6psKISIi0jR3vyTy2MxygEejuO554PlaZTfUOr4pBiGKiEgn1liC9z3gGTM7k10JXQmQCpwY78BiKdmMytBc9i1lFWSnpyQ4IhER6UC2A4MTHYSIiAg0kuC5+yrgIDM7DBgRKn7O3etdtnl3lpxkEFqLTH13IiLSGmb2T3atgJkEDKP5++KJiIjERZP74Ln7DGBGG8QSN4EkpXUiIhIzP494XwEsdveliQpGREQkUjQbnbd/Efmd5t+JiEgrfQWscPdSADPLMLNid1+U2LBEREQa3wevw1BKJyIiMfQUUBVxXBkqExERSbhOkeAlRwzR3LBtZwIjERGRDiDg7uE/JqH3qQmMR0REJKzBBM/MtpjZ5npeW8xsc1sG2VqRwzL/8J8vEhiJiIh0AGvMbHL1gZlNAdYmMB4REZGwxlbRzIpXo2aWC/yB4OqcDpzn7u/Erb143VhERDqjC4E/mdmvQ8dLgbMTGI+IiEhY1IusmFkBkF597O5ftaLdu4EX3P0UM0sFMltxryZFrquycM22eDYlIiIdnLsvBMaaWVfA3H1LomMSERGp1uQcPDObbGafA18CrwOLgH+1tEEzywYmAA9AcO6Cu29s6f2ibDX87s0FGkUjIiItZ2a3m1muu2919y1m1s3MfpTouERERCC6RVZuBcYCn7l7f+AI4K1WtDkAWAP80cz+Z2Z/MLMutSuZ2TQzm2lmM9esWdOK5kRERGLqmMgHk+6+ATg2gfGIiIiERZPglbv7OiDJzJJCG5+PakWbAWBf4F53Hw1sA66uXcnd73P3Encvyc/Pb0VzcNiQ1l0vIiISIdnM0qoPzCwDSGukvoiISJuJJsHbGJpn8AbBSeV3AxWtaHMpsNTd/xs6fppgwhc3x+zdK563FxGRzuUx4BUzO9/MzgdeBh5OcEwiIiJAdAneFGA7cBnwArAQOKGlDbr7SmCJmQ0JFR0BzGvp/aJhWkdTRERixN1/CvwIGAoMI/i3cY+EBiUiIhISzSqaBcAKdy8FHg4NRekJrGtFu5cQ7A1MBb4AvtmKezVN+Z2IiMTWSqAKOI3gImR/TWw4IiIiQdEkeE8BB0UcV4bK9m9po+7+IVDS0uubK8mU4YmISOuY2Z7AVOAMgg85/0Jwm4TDEhqYiIhIhGgSvIC776w+cPedoZ63dkPpnYiIxMCnwH+AE9x9AYCZXZbYkERERGqKZg7eGjObXH1gZlMAbSYnIiKdzckEh2bOMLP7zewI9AxRRER2M9H04F1IcL7crwn+IVsCnB3XqGJMIzRFRKS13P0Z4JnQ3q1fI7j4WE8zuxd4xt1fSmiAIiIiRJHguftCYGxoqwRz9y3xDyu2tIqmiIjEirtvA/5E8OFnd+BUgvu5KsETEZGEazDBM7NvuPtjZnZ5rXIA3P2uOMcWM+rBExGReHD39cDvQy8REZGEa6wHr0voa1ZbBBJPyu9ERERERKQzaDDBc/ffm1kysNndf9GGMcWeMjwREREREekEGl1F090rgcmN1WkPNAdPREQSzcwmmdl8M1tgZlc3UOc0M5tnZnPN7M9tHaOIiLR/0ayi+XZoBc2/ANuqC939g7hFFWP7FOYkOgQREenEQiNifgNMBJYC75vZdHefF1FnMHANMM7dN5hZQWKiFRGR9iyaBO+g0NdbIsocODz24cRHl7Sa3+bSDdsp7JaZoGhERKQTGgMscPcvAMzsCWAKMC+izgXAb9x9A4C7r27zKEVEpN2LZpuEw9oikLY0/iczeOWKQxiY3zXRoYiISOfQl+A+stWWAgfUqrMngJm9BSQDN7n7C7VvZGbTgGkARUVFcQlWRETar0bn4AGYWY6Z3WVmM0OvO82s3Y95PPV37yQ6BBER6TzqmwzutY4DwGDgUOAM4A9mllvnIvf73L3E3Uvy8/NjHqiIiLRvTSZ4wIPAFuC00Gsz8Md4BtUWtpVVJDoEERHpPJYC/SKOC4Hl9dT5h7uXu/uXwHyCCZ+IiEjUoknwBrr7je7+Reh1MzAg3oHFW+3HpiIiInH0PjDYzPqbWSowFZheq87fgcMAzCyP4JDNL9o0ShERafeiSfB2mNn46gMzGwfsiF9IbUQZnoiItBF3rwAuBl4EPgGedPe5ZnaLmVVvR/QisM7M5gEzgCvdfV1iIhYRkfYqmlU0LwIeDs27M2A9cG48g2oLVa4MT0RE2o67Pw88X6vshoj3DlweerW5vK5piWhWRERiLJpVND8ERppZduh4c9yjagNK70RERHZ58XsHJzoEERGJgSYTPDO7vNYxwCZgVij5a5dcPXgiIiJhPdSDJyLSIUQzB68EuJDgHj59Ce69cyhwv5n9IH6hxZfSOxERERER6WiiSfB6APu6+xXufgXBhC8fmEAr5uKZWbKZ/c/Mnm3pPVrDHeYu35SIpkVEREREROIimgSvCNgZcVwO7OHuO4CyVrT9XYIriSXMcfe8ydIN2xMZgoiIiIiISMxEk+D9GXjXzG40sxuBt4DHzawLMK8ljZpZIXAc8IeWXB9Lqza3JkcVERERERHZfUSziuatZvY8MJ7gNgkXuvvM0OkzW9juL4EfAFkNVTCzaQTn+1FUVNTCZkRERERERDqPaHrwADKAze7+S2CxmfVvaYNmdjyw2t1nNVbP3e9z9xJ3L8nPz29pc01yd278x8csWrstbm2IiIiIiIi0hSYTvNCwzKuAa0JFKcBjrWhzHDDZzBYBTwCHm1lr7heV4h6Z9ZbPX7WFh99ZzIWPNZpviojI/7d33+FRVOsDx7/vphJCAkhvhqIgKN2KiBVQ7OXaO5Zru96rXsGK7WfvvaLXi4qiXJGuCAgovXcChBpCgPSQZMv5/TGTzW62ECDJhuT9PM8+O3vmzMyZk92dvHvOnKOUUkqpGq8iLXiXARcDBQDGmJ2E6Vp5IId34XYAACAASURBVMaY4caYNsaYFOAa4HdjzA2Hur+Kmv7wmSHKYz2v3ZWnc+MppZRSSimljmgVCfBKjBX5GAB7cJUji8eD7NkQdJVvSPfT4h3VUx6llFJKKaWUqgIVCfC+F5GPgYYicgfwG5U0+qUxZoYx5sLK2FdYvz4Jn51LS/YGrFqyJcu7nJqZD4DT7WHupr3kFjmrvGhKKaWUUkopVVkOGOAZY14DxgA/Ap2Bp4wx71R1wSrVibeDx8XrMR8iePxW/bSkrNWutIfmK5PXcs0nc+k+Ymp1llIppZRSSimlDktFBll52RjzqzHmEWPMw8aYX0Xk5eooXKVp3AHOf5nTolZze9SksFldbg+fztpcTQVTSimllFJKqcpTkS6a5wVJO7+yC1Llet3AFHdfHokeTRfZGjRLdmEJ45enB6Qv2rKPeZsCu3cqpZRSSimlVE0SMsATkb+LyAqgs4gs93lsBpZXXxEriQjDnEPJIZG3Yt4njpKALN8t2MaO7P0B6Vd8+BdXfzL3oA43e8MeRs7RlkCllFJKKaVU9QnXgvcNcBEwzn4uffSpjmkNqkIWSTzsvIsujm08E/1l0DyvTlnn93pfQWAgWBE3fD6PZ35ZfUjbKqWUUkoppdShCBngGWNyjDFpxphrjTFbgP1Yswokiki7aithJfvD04N3XJdyTfQMroqaccD8vZ/7teoLpZRSSimllFKVoCKDrFwkIhuAzcBMIA0IP1JJDfeW60pmu7vxXPRIukpapIujlFJKKaWUUpWiIoOsPA+cAqw3xrQHzgHmVGmpqpgHB/9w3kcWDfgg5m2SyT+o7XMKnTw6Zjn7S9xVVEKllFJKKaWUOngVCfCcxpi9gENEHMaY6UDPKi5XldtLMveWPEBL2cuHMW8Rg6vC2/Z4diqjF27j2/nWaJxpewq4/csFFDkjG/AZY/h+4Tbyiyt+LkoppZRSSqnaoyIBXraIJAJ/AKNE5G04iGioBltsjmWY8w5Oi1rNc9FfYN1iGFpOodPv9bPjV3Pqi9N45pdVTFu7mz837mFjZj6PjV1RhaUObdGWLP49ZjlP/bwyIsdXSimllFJKRVZFArxLgELgn8BkYCPWaJq1wlhPf961B125I2pC2Lw9np3qbbUrlZ5T5A0Lb/tyIee8PpNv5pXl+WHhNk584Td25RQdVjm/nruFlGETcLk9IfMU2F1GM/OKD+tYSimllFJKqSNTuHnwOolIP2NMgTHGY4xxGWO+ApYCDauviFXvDdeVjHefzPDob7nY8WfYvMN/Cmydm7EuM2T+R8YsJzOvmFNenHZYZXxx4hoAilyhAzyllFJKKaVU3RauBe8tIC9IeqG9rtYwOHjI+Xfmmy68HvMhZzmWVMlxCkusnq3GGHKLnEEnVT+QIqebW0bOZ/OegsMuz4rtOQx8c2bE7tmbvWEPE1ekR+TYSilV3URksIisE5FUERkWZP0tIpIpIkvtx9BIlFMppdSRLVyAl2KMWV4+0RizEEipshJFSDGxDC15iDWmHR/GvMXJsqbSj/H8hDXsL3HTfvhEuo+YSr+Xfveuyyl08ukfmzDG6vD5Z+oeMnIDu3XOSd3DjHWZPD++YpOo5xe7vPss7+XJa1mfkc+SrVmHcDaH74bP53HPqMURObZSSlUnEYkC3gfOB7oC14pI1yBZRxtjetqPz6q1kEoppWqFcAFefJh19Sq7IDVBPgncXPIo20wzPot9jd6yvlL3/828rRz31GS/tGKXm2Xbsunx7FRemLiGvzbtBeC6z+Zx8v9N897PVxqjbcwM3XIn9vOsDXtIGTaB58av5vinp/DBjI2Veh5KKaUO2klAqjFmkzGmBPgO6x53pZRSqlKFC/AWiMgd5RNF5HZgUdUVKbKySOKGkuFkmmS+jn2RUx2rqvR4nZ+YzCXvl00rWFLuHrvHxq6gyOlmvz0FwzvTNgAHGu/T8vnszQAhu0GaCu1FKaVUJWgNbPN5vd1OK+8KEVkuImNEpG2wHYnInSKyUEQWZmaGvgdcKaVU3RQuwHsQuFVEZojI6/ZjJjAU+Ef1FC8yMmjM1SVPsd00ZWTMKwxwLKu2YwebS+/MV2cEpO3M3k9BBe+dW52ey9Jt2SHXi7ftL7jXpqwLOriMUkqpCgv2RVv+V7ZfsG6P6A78BnwVbEfGmE+MMX2NMX2bNm1aycVUSil1pAsZ4BljMowxpwHPAGn24xljzKnGmF3VU7zIyaQh15Q8Qappzacxr3GpY3a1HPfu/y5mT77/NAe7gtyLt3ZXHt2engJYUzGs3pkbcp/GwKU+rYQB6w/Qkvfe9NSA6SGUUkodlO2Ab4tcG2CnbwZjzF5jTOkF4FOgTzWVTSmlVC0SfaAMxpjpwPRqKEuNk0US15U8zscxb/JW7Ae0c+7mHfdlBP8htvL0ff63CudNGVY2d9/9Z3c6YP6/Nu4lZ38JfVMaMyd17yGVTyml1EFbABwjIu2BHcA1wHW+GUSkpTGmtE/9xUDlj/allFKq1qvIROeVSkTaish0EVkjIqtEpEZ398ylPjc5hzHGfQb/ihnD6zEfEUdJpIsV1Lu/p4Zdv3x7Ntd+Ope7/7uYVyev86a7PIYR41axckcOF707u1KmYDgYpaN85hY5Sc85+KkjlFKqpjPGuID7gClYgdv3xphVIvKsiFxsZ3vAvi4uAx4AbolMaZVSSh3Jqj3AA1zAQ8aY44BTgHtDDBVdYziJ5mHnXbzuvJIromYxJnYEbSUj0sU6aBe/V9ZNc/TCsnv9R83dypd/pnHhu7NZsSPHO5BLOIu3ZjF6QVm3zf0lbiaFGMwlPWc/70zbEHK6htKRQwe9+Qenvvh70Dylfly0ndkb9hywfPM27eWNqesOmE8ppaqLMWaiMeZYY0xHY8wLdtpTxphx9vJwY0w3Y0wPY8xZxpi1kS2xUkqpI9EBu2hWNrv7Sbq9nCcia7BGEqvYxG4RI7zrvpw15mhej/mQCbGP85Dzbn719I10wQ5b6QTspdwew89Ld9C6YT227isMus3lH/wJwI+LdzB/8z4GHNuUmesz+fnefvRo29Av7z2jFrNkazYDuzWnS4ukgH053Vbgl54TeK9heQ/9YA14k/bSkLD5rv5kLgD/GtjZmzZ6wVY8Bq49qd0Bj6OUUkoppdSRKBIteF4ikgL0AuYFWVfpw0A3Sog57H385unDkJL/Y4tpxqexb/BC9OfU58juVli+YW3csp3847ulXPnRX/zr+7IRRO/7JnBS8vmb9wEwc731N/pgRmpAS11hsTvgOGe/PsO7HOqOxq/+TPPuvzI8+uMKHQ1UKaWUUkrVahEL8EQkEfgReNAYEzAEZFUMAz35wTP4/q5TD3s/200zriwZwSeuIVwb9TuTY4dV+Xx5Vam0i+SBjF+eTsqwCWzPCt6qBzBlVQaTV+5i5Y4cducW4fEY7yidItYgL1v3FrIpzITtmzLzSRk2gafHreJvH/8VNM9//kojv4LTRADVfl9hbTJ2yXbWpIcepVUppZRSStUc1d5FE0BEYrCCu1HGmJ+q67jNk+JpnhRfKfsqJpb/c13PFHdfXo35mG9jX+BHd39ecl5DJo0q5Rg11XWfBjS4+vn7KP+WvqOPSgAgr8jFtZ/ODch/0xfzObZ5ovf1i5OC33bi8ZQ1AT718yqWb8/htat6VKjMZ702I+z6lGETOP/4Fnx4g/+o5HlFTlbtzOWUDkdV6Di10T9HV6xbrFJKKaWUirxIjKIpwOfAGmPMG9V9/Mq2yHTmgpIX+cB1MRc6/mJ63EPcGfULsTgjXbQqE+q+vFC27LXyz1wXuqvt+ox87/Kvq/0HsNmZbXWB7fDYRL/0fQWHPppp6T6hbBTPSSsDp3e8Z9RirvlkLtmF1rHyi108N3510AnpS7k9hmE/LufRMcvD5lNWfbo94edhVEoppZRSFReJLpr9gBuBs0Vkqf24oDoL8OjgLpW6vyLieMV1DYNKXmau5zgei/mWGXH/5IaoX2t1oHew3psefhqHUD6euZF7Ri0KSC+wu2juyS/m2McnMSc1cHTNf45eyrQ1/gHjL8t2ctpLv3P5B3NIGTaBm76Y7133zbytPD++bLyfWfaInSUuDwDvT0/l89mbGTVvK26PYeCbM5myahdOt4csO+AcOWcz3y3YxuiF23j39wOPSFpXlbg8HP/0FEaMq7zuzb+vzWDr3oP7AUIppZRSqjap9gDPGDPbGCPGmO7GmJ72Y+KBt6w8jiqapzzNtGSo8xGuLxnOTtOE52NGMj3uX9waNYkG6D+dh2rcsp1MXBHYujZv8z5Of/l3+j7/GyVuD9d/No/TX/7dG2gBjF2yg9u/Wui33f3fLgFg8dZsoCyIA3hs7Ao+m70ZwO++s4d+WMZPi7fz4YyNALjcHlbsyGF9Rj6P/ricf45eSq/nfgXg+QllcxMXFAe24Hk8hjXpuTz980q++jON2Rv28MXszSGnkQhmf4mblGETGLtkOwBFTrdfF1ZfuUVOVu7IqfC+q8ItI+fz89IdfmlOtxU0j1m0/bD2vW1fIcu2WX/L275cyFk+A/gopZRSStU1EbkHL9JuPPVo0vYW8u38rQfOfAjmeE5gTsnx9Hes4IHon3g65msejv6eH91nMMp9DuuMDtN/MLIKQ7eCbs/aH/C6NNA6XGk+A7PM2rDHLxD0vU9QsAagCUYEpq/bTccmibSz70X8cOZGXp0SfI6+205vH5BWUOyi29NTGHFRV27pZ63flWtNKfHWbxsY3K0lxz01mbsHdGTY+f6t01v2FjDg1RnW+VTwHrq9+cVk5BbTtVWSt5W0vJRhE7ijf3seH1KxKSxnrMtkxrpMYqIcXHBCS8CqG8A7CM+h6v/KdADWPT8YoEJdPof/tJwBxzZl8PEtD+vYSimllFI1TUSnSYiUhNhoXrz8hCo+ijDL052rSkZwYfHzTPKczNVR05kSN4xJsY/y96hxtJHKmf5BVb6UYRMCBosJxTcALd8KN3JOGreOXMCA16wgpKDYxf+W+LdklXp2vP9UkBsy8vhh4Tb25lstkp/P2exd59sIXdoy9s28LQH7LA3ugpUtlMFvz+KCd2Zx68j5PPG/lSHzfTprs9/rlGETeHTM8rD7vmfUYopdVqumw47wKnoL3vqMPFJ354VcvyY99Lryvp2/jbv/a/19t+wt4MxXp7M7r4hFW7KCdvUtVeR0k1tUOd2uS1weHhu7gt15B57/USmllFKqoupkgFfdVpoOPOy8m1OL3+NJ5y0UEs+jMd8xO+4fTI19hMeiR3GaYyVxHPqgIapmuPXLBUHTjbHuBzzvjZls2J0fNA/A2l25pAybwIhxqzjvzT94ZMxyVu20uld6PIH5t+wtZJg9t19uUVlr26wNmaQMm+CX94cgXSGXbM3i32OW4fEY9uQXA5CZZz1PX5cZEOw43R7eL3cv5bhlO7n8gzkAjF64jfSc/Tw+doW3C2Z5n/6xCYDJ9qA2voHniu05vDvNum/x5clrGfjmTO+6gW/+wblv/OG3L9+Bdg6mi2upOal7GDknjbS9hZz0wjSu+PBPrv8s9CixXZ6cTPcRUwPSC4pdvDF1nfecF6btY9aG8D/gTFuTwTfztlbqPYjVzeX2hOwarJRSSqnIqJNdNCNlH0l87R7I1+6BtJUMBjkWcqZjKbdETebO6Ak4TRSrzdEs8XRiqacTG0xrNppWFBEX6aKrCpoRZqTQsSFa7nwNfmsWAF/+meZNK21J3JG9H4/HsGx7NreFCCRL3fj5/IC0f49ZzhW92xDlENbtyqN9k/rc+Pl88otdNE+K593fU/njkbP8ttltB3ulvp2/1a976etT1/Hu7/4B3/CfVjBjXSbndm3OWZ2bBZSjsMTN+ow8Hhy9FLCC31IXvTcbgPvPOcZ7vyOUBYO+Copd9Pbpjls+zNidV8SouVt58NxjEAl+422oYO6vjXs5taP/1Bh5YVru3vx1PZ/N3kzrRvW4+sR2XPmRNX9juG6x7kMISA9WyrAJ3HJaCiMu7lYl++/0+CQuOKEFH1zf58CZlVJKKVUtNMCLkG2mOZ+5h/CZewgJFHGKYzV9HOvpLalcFTWTW6LLWgm2eZqy2bQgwzRiF43ZZRqz2zQkjwTyTD3ySCDf1COeEs6JWkwvRypHkYtByKcemSbZetCQTJPMLnMUu0wjsknEv7OfqunKTxVR3vjlO3nml9Uh13d8bCKPDu7Cy5PX0iIp3jtZfGmQtiN7f8hty7cI+m7nqzTIvXXkAgZ1a87HN/b1W//BjI2s21XWndLlMSzblk3L5LI5Kq/2meB+854C7v5v2SiqWQUlDPtpOfExUX773Zfv3wL+0PfLmLVhD/06NaGwxEVekYvjWjZgRQUGnPlw5kZ6H92QuGjrGH+m7uGX5TtD5t9vT4dR4vJ4W1zL25CRh8Mh/LxkB4OPb+kNbMXnM7g3v5hnfllNi+R4Hh3chahyI0INfHMmV5/YjkYJMTRKiKVt4wTaNKrnVxdzUvewaEsWD5xzDGD9WHAoAV7KsAncfnp7nrww/H2WwQZAUkoppVTkaIBXAxQSz++e3vzu6Q1AFG46SDqdZIf1cOzkaMngGMcOmpFFlIT/5T/dNCbDWJOtt2U3ZzhySJLAUTyLTIw3aNxpjiLdHMUek0yGacRu05AMrGdtQTxy3PfNkgPmeXmyNUBM6UAtvsqPdHm4pqzK4IMZgUHgtLW7/V5f8v4cHjz3GO/reZv3eZfLTzXx2tR1TFnlP/UFwND/+I+WWlhiBV1P/byStbsqfn8ewB/rM+n8xGQmPHA6O7L2c+fX/tN07Mkvpkmi9bnwDXyf/Nm/u+X6jDzWpOdySc/WnPdmWffSd35P5fWregAwYUU6z+YX8970VEbOSfPm6depCQOObeq3r/UZ+TxX7l7NC7u35L3rrO+O7MISb6tkaYBXavLKXczfvI+nLqrYwDgAn8/eTK92DXGIeAfHUUoppVTNpgEesPSp85i6OoMebRrStnE9uj41JaLlcRPFBtOGDaZNaYJXFG6akEMTsYK2BhSSyH4ayH48CIs9x7LKpATsM44SmkoOzciiheyjhVjPze3nvrKeFo59xEjgsP65JoEM04gM05DdNGJ36bL9nGGnFRNbRTWiqst3C7ZV+j5fmRx8xNDy3vot+JyBPy32DzpHzTvw6LcnvfCbt3vpwQZ3voa8Mzto+tjFO3hn2ga6t00Ou/1AO6j7x3dLA9bl+4xQ+sLENQHnWf6ewoFv+t9/WGr88nTeu85a9u26u7/E/7Nc2gq6aU8+n97Ul637CtlXUMKybdkM7d/BL+9XPl2ES380CNfddPn2bGZt2MPdAzoGtDoqpZRSqnppgAc0TIjlb33bRroYFeImigwak2EaB950FEYxsWw3TdlO0zDbGRqSTzPJprlk0Qz7WbK8aSfJWpqSTZwEDp+fYweCpa1/mT6BYDb1yTaJ5JBItqlPPvXQ7qGqqpS/d7CyvTDRmutwTureQ96Hb/fWYCOr3jJygXcqitenhg+SOz8xiY5NE1ntM3fjZfbANwDzNpWVc8a6TP5Yn+k3P2RpgDd++U6iHcLTIQZ+ycwrptjlpk2jBL/0i9+zjvXRzI2sGDEobFmVUkopVbU0wAsjJkpwuuvSCHFCNg3INg1Yb8IFvFYg2Nwn8GtG2XJzyeJkWUszsogN0iII4DIOcrxBn/WcTSI5pn7Za5NINvXJNfXJJtGb101U0H0qdSQpHbUUQk8V8emszQHTUQRT7PL4BXfg33J59Sdz/daVb1UNdn9lMCe+8BsAUQ4JOt9gXlHweROVUkopVX00wAvjwXOP5dUp67i1Xwoj56RxXtfmtG5Yjy//TMMhFZ+/q/YpCwTDT9puaEQeTSWHZApoKPk0lHzvsu9zE8mhEztIdhSQHOR+QV+5ph453sCwvh0YWsFgju/rcmlFxKKthkrBuoyD77bqGwRWZDJ5pZRSSkVGnQ7wvrz1RO9ADMHce1Yn/ta3LQvS9jFyThrRDuGJIcfxt75t6doqicISl/d+vYFdmzN1dQZPX9Q17CiGdYuQRRJZJsl6WcH/CR14SCoNCCkgWQpIJt/7uqHk+6QV0IVtJDusdcHuISxVbGLIoT65JoFcEsgx9cmlPnmmnv2cQB71yDP1KCSeAuIpMNZzof1cQDyuuv2xUUoppZRSNVid/k/1zCBzdJXXtEGc3zxd0VEOurayApaE2Gh6tm3I0m3ZREdpy1Bl8eDwthACB3GvoSGBYhrawaAVBJYGilYwmEQ+SbKfJApoLHmksIsGDut1qO6k5RWbaG8AWGjirGUTb6fFeYNBK91aX0wMxSbGeiaGYmLt17EUlaaZsnUlRGNwHFL9KaWUUkqpuqtOB3ihnNm5KUOCDAkebK7kT2/qy5zUPSzZmmXlAa49qR1J8dF8/Mcmv7z/u7cfk1akB6QD3HlGBz4Jkq4OhlBoB1Y7TZODGoQGDHE4SaKQRNlPAsXUZz8JUkQiRdSTYhKx06WIBIqoTxEJUkR9iqkv+2lEvpXusNfL4Q30UWyiAwLCEnyCRDtALC4XIBaVphmfdeUCyNABZ9lDu7MqpZRSSh15NMAL4stbT/J7bcJECk0bxHFpr9beAA/gxctPACA6Sji+VTJPjVtFZl4xrZLjGX7BcRzfOpn7vy2br+yv4WfTMrmeBngRJRQTSyaxZJqGZcmHcauR4CGBYupRQhwlxImTOHweUlLutdPK5/faSosPsq6B7KcJuSH27TzsGikf8DlNFC6icRGFkyjcROHCgdNE4ySaEu8jxpvmJAqnvU0JMd6g1VpnpbuNAxfRFNvblhCN05QtF/ukOX3zEI1HWzmVUkoppfxogFcBpV00JUyLRqfmVnfCto3Lhg9/ZFAXAJ4qN+T4RT1a+QV4LZPrBd3na1f14OEflln7b5ZI6u58/j24M/ec2YkuT06iyOnhtn7t+WKONcpefIyDIqfnIM9OVRWDgwLqUUC90oTyGaqM4CEWV4WCxzicxAcNNn22ESfRuInGRQxuonEThcdKEzcJFBCDyz5mCdEOtzdfDC5icAWdWuNwuY14g0unXSonUbhMFC7KHmUBaRROE1WWzw4yXVhBpss4cBPlDVY9OKwgFAcuUxbU+j+Xrbfyh17v9u6v9JhlaR7jwI0DD9Zz6bIHCUjX1lWllFJKhaIBXgX069SERgkx3D2gY8g8N5zcjuNbJdGrXaOAddec2JZ3f08lMT6wuuc/do53+Ykhx/H8hDU8Mqgzr05ZxzHNEr3rbj4thSf/t5KWyfGAdf9fkbOEe8/qSFK9aJxuD48M6lLh4c5V7WZw2F0xY0sTymeIAOMNOkuDvmjcRInHJzi014mLWJzEYT3H2mml+Uq3t/K57XQrjIuxg85oAh8x4iZenERTFLheSoNWF3H2/h12EBslNWvUSI/xD/o8SEBg6F32yevxW19+Hw7cdpBp7HXBAk7vvkxg4LnXJAGhJ0RXSimlVNXTAK8CGtePZclTA8PmEZGgwR3Av847lgfOOYaYqMDuZM2S4r3LQ/t3YGj/DhhjuKJ3G1okl627/qR2tGlUjzOPbQqAw/4B32Os6RxKfXvHKfy1cQ/v+EyiXN4TQ45jaP8O/Jm6h+s+mxf2vJSqPGJ3r4zxTw4WO9WseAow3hbLsmc30XisZ7Gf7fVRpem+z+Iut325/UjZtg7vs/FZtvcrPsv2+vJ5HPY6/7xWGBhln0uw/UaLm1irPdFvu2B5xd6Pb/oO0yTSf6gaTUQGA28DUcBnxpiXQuS7EvgBONEYszBYHqWUUioUDfCqgYgQU26UzUVPnEuRK3h3ShHxBnendGjM3E37cDiEs3xG/ezVrhG/rs4gNto/aDy141Gc2vEo5mzcy6ItWYy5+1Su/Ogv7/qT2zdmaP8OAJzWqQkf3dCHu/+7yDvNA8DIW0/k1pELABjUrTlTVmUc8BxbJMXz4uUnMGreVn5bc+D8pVY/O8g71YRSNZfYoVhU8NUVCUhrXNBaNdIiXYAaSkSigPeB84DtwAIRGWeMWV0uXwPgAUB/fVNKKXVIdISCCDkqMY7WDYPfe+dr1NBT2PDC+QHpb1/Tk3H39SO5XkyQrUIr/z9mN3vKh4t7tuLC7i157tLj/QLJy3q1YfOLF3BOl/BTSsx97BzO6tKMz27uy2//GsAJrZMBmDPsbL987XzuUVw+YiAJscF/Y1j1zKCKnpJSSh0JTgJSjTGbjDElwHfAJUHyPQe8AhRVZ+GUUkrVHhEJ8ERksIisE5FUERkWiTIcKaIcErRrZ0JsNN3bNAyyhaVvitVdtEliHFMePIO/9W0DQGy5fbVtnEDaS0O4sHsr3ruuNzeecjQAXVpYg8ac3L4xIsLwC47zbtOv01HeKSN+vrcfMx4+02+fnZol8sv9p5P20hBaN6xHj7YNObZ5ItMeGkCPtmVlToq3gtOUoxIor36cf+D3wNmdvMuX927tXS5/bF93ntGBBY+fyzvX9mLJk+eFzKeUUtWgNbDN5/V2O81LRHoBbY0x48PtSETuFJGFIrIwMzOz8kuqlFLqiFbtXTQr2k1FHZ5HBnbmqj5tSWlSH4CXLu/OUYlx3HJaSoW2/+q2k5i1YQ+N6luDdHRqlsjc4daAMA0TYthXUMKCtH1+AVsoP9/bz7v8zMXdqB8bxUMDO3vTbu/fgSf/t5LrT27H0P4dWLQly2/7l684gatPbOe9r/Cly7tzUY9WdGuVRLMG8YTymB2UXtyjVdD1xzZP5PqTj+bpcqOcDj+/C5NW7mLptmy/9It6tGLyynQ8Btwe/7bQW/ul8O9BXTjuqckhywNw14AOfDzTmg6jaYM4MvOsufI6NKnPpj0FAIy95zRmrs/krd82BGx/Re82/Lh4e9hjhPLZTX0Z+p/A23lOTGnEgrSsIFtUv2ObJ7I+Iz/SxVCqKgQb+tT7RSIirbX+jwAADZxJREFUDuBN4JYD7cgY8wnwCUDfvn3rSOdfpZRSFRWJe/C83VQARKS0m4oGeJUoOspBJ59ROB0O4dHBXSq8ffOkeK7s08YvzXfQl1YN63FJz9blNzugxvVjeemK7n5p7Y+ygtDjWibRvkl92ttBKViTzl99YjsA7h7QERGIjXb4dSMtdfvp7dmUmc/0dcF/0X7gnGP4dv5WLu/dmo9nbmLqPwdgjCEzr5hLe7Xm6o//Ym9BCXcN6MhdPiOm3jNqERNX7OKJIcfx7rW9KCh2MWLcKmauz2R3XjFpL5WNGrju+cFMXJHO+GXpTFu7m49v7MPqnbm8PW0DzZPiGH7+cdzZvwMPjl7KG3/rybfzt3JH/w6UuD30eGYqYN1f2atdI2KiHLw6ZR2A3zGu7NOGaz+dS1J8NLlFLm/drUnPDTjn849vQecWDfhs1mbO7dqcy3q1ZuySHX55vr/rVIpdHro8GT44PVxpLw0JO8pr2ktDKHK6D1iOL289keE/rSA9p/J7sPXrdBRzUvdW+n6VwvpBs63P6zbATp/XDYDjgRlidZFoAYwTkYt1oBWllFIHQ4yp3h//7NHBBhtjhtqvbwRONsbcVy7fncCdAO3ateuzZcuWai2nql6rd+ZyXMsGiJT9yJ1b5KReTFTQLqq+Vu7IoXH9WFrZ9zSuSc+lXkyUt/WyonIKnRSUuLz7KVVY4mLVzlxOTGnsl15Q7KKgxBW0FXF3XhHfztvG/Wd3wuEQ8otdRDuE+JgQg3QAk1akc2L7xjRJjPOmfT13Cxd3b0VyQtm9lsYYvpiTxpV92jBl1S56tGlI5xYNKHa5mbxyF20a1WPJ1mx6tm1I33JlBtieVcjoBdsYNW8rH93Qh5PaW3n+WJ/JCa2TWZ2eS/OkOL6Yk8aunCKaJ8Uxbc1u3r22Fx2aJpKzv4RtWfsZcExTrv7kLzLzihlwbFPO6tKMu75eRNvGCZzQOpkmibEM6taCBWlZ3NG/PdFRDlbuyOGGz+dxtn1P5/GtkkmqF8N5XZt77ye97IM5XNWnLY+NXQHAK1d2p15MFGd2bsqqnbmc0uEoAFJ357Eps4D3Z2zkkh6tuO309vznrzTmbtpLYlw0C9KyuHtAB8YvT+eszs14drz1G1KPNsms3JnLL/edTmZ+MTd/MZ/rTm7H4G4tOOPYpgz9agG/rdnN8a2TWLkjl64tk+jULJFxy3byw92n8tWfafyxPpPx9/cnIS6K16eu58o+rXnml9Uk14th1oY9fvX9j3OO4e1pG5g7/Bzmbd7LP75bGvA3aZUcz5MXduXvoxb7pb99TU+G/7SCISe0JDbawbOXHE/HxyZ611/euzW5+538tmY3SfHR7He6cbqNd8oVX32PbkTvoxuRu9/J6IXbmPnwWSzdns3a9Fw+mLERsLpmr92VB8ALlx3P42NXerdvlBDDiIu7ect/Re82TFqZTmGJ25vnmztO5rSOhzeSpogsMsb0Payd1EAiEg2sB84BdgALgOuMMatC5J8BPHyg4K5v375m4cLDi/++X7iN9k3qB3zHKaWUqlkqeo2MRIB3FTCoXIB3kjHm/lDbVMYFTCmlVM1XWwM8ABG5AHgLa5qEL4wxL4jIs8BCY8y4cnlnUE0BnlJKqSNDRa+RkeiieaBuKkoppVStY4yZCEwsl/ZUiLxnVkeZlFJK1T6RGEVzAXCMiLQXkVjgGmDcAbZRSimllFJKKXUA1d6CZ4xxich9wBTKuqkEvQdBKaWUUkoppVTFRaKLZtBuKkoppZRSSimlDk9EJjpXSimllFJKKVX5NMBTSimllFJKqVpCAzyllFJKKaWUqiU0wFNKKaWUUkqpWqLaJzo/FCKSCWw5zN00AfZUQnFqE62T4LReAmmdBNI6CVQZdXK0MaZpZRSmLqik6yPo+zkYrZNAWieBtE4CaZ0EV23XyCMiwKsMIrKwIjO/1yVaJ8FpvQTSOgmkdRJI6+TIpX+7QFongbROAmmdBNI6Ca4660W7aCqllFJKKaVULaEBnlJKKaWUUkrVEnUpwPsk0gWogbROgtN6CaR1EkjrJJDWyZFL/3aBtE4CaZ0E0joJpHUSXLXVS525B08ppZRSSimlaru61IKnlFJKKaWUUrWaBnhKKaWUUkopVUvUiQBPRAaLyDoRSRWRYZEuT1UTkTQRWSEiS0VkoZ3WWER+FZEN9nMjO11E5B27bpaLSG+f/dxs598gIjdH6nwOhYh8ISK7RWSlT1ql1YGI9LHrONXeVqr3DA9eiDoZISI77PfKUhG5wGfdcPv81onIIJ/0oJ8nEWkvIvPsuhotIrHVd3aHRkTaish0EVkjIqtE5B92ep19r4Spkzr9Xqmt9PpY966PoNfIYPQaGUivkYGOmGukMaZWP4AoYCPQAYgFlgFdI12uKj7nNKBJubRXgGH28jDgZXv5AmASIMApwDw7vTGwyX5uZC83ivS5HUQdnAH0BlZWRR0A84FT7W0mAedH+pwPsU5GAA8HydvV/qzEAe3tz1BUuM8T8D1wjb38EfD3SJ9zBeqkJdDbXm4ArLfPvc6+V8LUSZ1+r9TGR7i/UW19oNfH0nPWa2TF6qROf++FuR7U2fdKmDqpUe+VutCCdxKQaozZZIwpAb4DLolwmSLhEuAre/kr4FKf9P8Yy1ygoYi0BAYBvxpj9hljsoBfgcHVXehDZYz5A9hXLrlS6sBel2SM+ctYn77/+OyrxgpRJ6FcAnxnjCk2xmwGUrE+S0E/T/YvbmcDY+ztfeu3xjLGpBtjFtvLecAaoDV1+L0Spk5CqRPvlVpKr4+WOnV9BL1GBqPXyEB6jQx0pFwj60KA1xrY5vN6O+H/ELWBAaaKyCIRudNOa26MSQfrzQk0s9ND1U9trLfKqoPW9nL59CPVfXZXii9Ku1lw8HVyFJBtjHGVSz9iiEgK0AuYh75XgIA6AX2v1Da18Xv+QPT6GJp+7wWn33voNTKYmnyNrAsBXrC+vLV9boh+xpjewPnAvSJyRpi8oeqnLtXbwdZBbaqbD4GOQE8gHXjdTq9TdSIiicCPwIPGmNxwWYOk1cp6CVIn+l6pferi30KvjwevLn/G9XsPvUYGU9OvkXUhwNsOtPV53QbYGaGyVAtjzE77eTcwFqsZOMNuCsd+3m1nD1U/tbHeKqsOttvL5dOPOMaYDGOM2xjjAT7Feq/AwdfJHqyuGNHl0ms8EYnB+pIeZYz5yU6u0++VYHWi75VaqTZ+z4el18ew6vT3XjD6vafXyGCOhGtkXQjwFgDH2CPSxALXAOMiXKYqIyL1RaRB6TIwEFiJdc6loxbdDPxsL48DbrJHPjoFyLGb26cAA0Wkkd3MPNBOO5JVSh3Y6/JE5BS7r/RNPvs6opR+Qdsuw3qvgFUn14hInIi0B47BuhE66OfJ7js/HbjS3t63fmss++/3ObDGGPOGz6o6+14JVSd1/b1SS+n1Ua+Pvurs914odf17T6+RgY6Ya6SpASPSVPUDa1Sf9Vij1Twe6fJU8bl2wBqJZxmwqvR8sfr0TgM22M+N7XQB3rfrZgXQ12dft2HdDJoK3BrpczvIevgWq4ncifUrye2VWQdAX/vDuxF4D5BIn/Mh1snX9jkvt7+EWvrkf9w+v3X4jGoV6vNkv/fm23X1AxAX6XOuQJ2cjtX1YTmw1H5cUJffK2HqpE6/V2rrI9TfqDY+0Oujb13oNbJidVKnv/fCXA/q7HslTJ3UqPeK2DtSSimllFJKKXWEqwtdNJVSSimllFKqTtAATymllFJKKaVqCQ3wlFJKKaWUUqqW0ABPKaWUUkoppWoJDfCUUkoppZRSqpbQAE+pI4yInCki4yNdDqWUUqom0eujUhYN8JRSSimllFKqltAAT6kqIiI3iMh8EVkqIh+LSJSI5IvI6yKyWESmiUhTO29PEZkrIstFZKyINLLTO4nIbyKyzN6mo737RBEZIyJrRWSUiIid/yURWW3v57UInbpSSikVkl4flapaGuApVQVE5DjgaqCfMaYn4AauB+oDi40xvYGZwNP2Jv8BHjXGdAdW+KSPAt43xvQATgPS7fRewINAV6AD0E9EGgOXAd3s/TxftWeplFJKHRy9PipV9TTAU6pqnAP0ARaIyFL7dQfAA4y28/wXOF1EkoGGxpiZdvpXwBki0gBobYwZC2CMKTLGFNp55htjthtjPMBSIAXIBYqAz0TkcqA0r1JKKVVT6PVRqSqmAZ5SVUOAr4wxPe1HZ2PMiCD5zAH2EUqxz7IbiDbGuICTgB+BS4HJB1lmpZRSqqrp9VGpKqYBnlJVYxpwpYg0AxCRxiJyNNZn7ko7z3XAbGNMDpAlIv3t9BuBmcaYXGC7iFxq7yNORBJCHVBEEoFkY8xErO4pPavixJRSSqnDoNdHpapYdKQLoFRtZIxZLSJPAFNFxAE4gXuBAqCbiCwCcrDuQwC4GfjIvkBtAm61028EPhaRZ+19XBXmsA2An0UkHuvXzX9W8mkppZRSh0Wvj0pVPTEmXAu4UqoyiUi+MSYx0uVQSimlahK9PipVebSLplJKKaWUUkrVEtqCp5RSSimllFK1hLbgKaWUUkoppVQtoQGeUkoppZRSStUSGuAppZRSSimlVC2hAZ5SSimllFJK1RIa4CmllFJKKaVULfH/LDp/q2p4UvQAAAAASUVORK5CYII=\n",
412 |       "text/plain": [
413 |        "<Figure size 1080x288 with 2 Axes>"
414 |       ]
415 |      },
416 |      "metadata": {
417 |       "needs_background": "light"
418 |      },
419 |      "output_type": "display_data"
420 |     }
421 |    ],
422 |    "source": [
423 |     "plt.figure(figsize = (15, 4))  # adjust figures size\n",
424 |     "plt.subplots_adjust(wspace=0.2)  # adjust distance\n",
425 |     "\n",
426 |     "from scipy.signal import savgol_filter\n",
427 |     "\n",
428 |     "# loss plot\n",
429 |     "plt.subplot(1, 2, 1)\n",
430 |     "plt.plot(loss_history)\n",
431 |     "plt.plot(savgol_filter(loss_history, len(loss_history)//3, 3))\n",
432 |     "plt.title('Loss')\n",
433 |     "plt.xlabel('epochs')\n",
434 |     "plt.ylabel('Categorical Cross-Entropy')\n",
435 |     "\n",
436 |     "# accuracy plot\n",
437 |     "plt.subplot(1, 2, 2)\n",
438 |     "plt.plot(accuracy_history)\n",
439 |     "plt.plot(savgol_filter(accuracy_history, len(loss_history)//3, 3))\n",
440 |     "plt.title('Accuracy')\n",
441 |     "plt.xlabel('epochs')\n",
442 |     "plt.ylabel('Accuracy')\n",
443 |     "\n",
444 |     "plt.show()"
445 |    ]
446 |   },
447 |   {
448 |    "cell_type": "markdown",
449 |    "metadata": {},
450 |    "source": [
451 |     "# 3. Test\n",
452 |     "\n",
453 |     "Let's now test the model on the Test set. I will repeat the dataprep part on it:"
454 |    ]
455 |   },
456 |   {
457 |    "cell_type": "code",
458 |    "execution_count": 11,
459 |    "metadata": {},
460 |    "outputs": [],
461 |    "source": [
462 |     "# Dataprep Test data\n",
463 |     "\n",
464 |     "test = pd.read_csv('fashion-mnist_test.csv')\n",
465 |     "\n",
466 |     "test_label = pd.get_dummies(test.label)\n",
467 |     "test_label = test_label.values\n",
468 |     "test_label = test_label.astype(np.float32)\n",
469 |     "\n",
470 |     "test.drop('label', axis = 1, inplace = True)\n",
471 |     "test = test.values\n",
472 |     "test = test.astype(np.float32)\n",
473 |     "\n",
474 |     "test = test / 255.\n",
475 |     "\n",
476 |     "test = test.reshape((len(test), 28, 28, 1))\n",
477 |     "\n",
478 |     "prediction = CNN.predict(test)\n",
479 |     "\n",
480 |     "prediction = np.argmax(prediction, axis=1)\n",
481 |     "test_label = np.argmax(test_label, axis=1)  # reverse one-hot encoding"
482 |    ]
483 |   },
484 |   {
485 |    "cell_type": "code",
486 |    "execution_count": 12,
487 |    "metadata": {},
488 |    "outputs": [
489 |     {
490 |      "name": "stdout",
491 |      "output_type": "stream",
492 |      "text": [
493 |       "[[855   2  13  13   4   1  92   0   4   0]\n",
494 |       " [  0 980   2   2   0   2   4   0   1   0]\n",
495 |       " [ 19   1 848   6  23   0  41   0   2   0]\n",
496 |       " [ 18  13   9 926   9   0  23   0   1   0]\n",
497 |       " [  2   1  74  35 936   0  74   1   3   0]\n",
498 |       " [  0   0   0   0   0 963   1   9   0   6]\n",
499 |       " [ 92   2  50  16  26   0 757   0   4   0]\n",
500 |       " [  1   0   0   0   0  17   0 966   2  32]\n",
501 |       " [ 13   1   4   1   2   3   8   0 983   5]\n",
502 |       " [  0   0   0   1   0  14   0  24   0 957]]\n",
503 |       "\n",
504 |       "Test Accuracy: 0.9171\n"
505 |      ]
506 |     }
507 |    ],
508 |    "source": [
509 |     "from sklearn.metrics import confusion_matrix\n",
510 |     "\n",
511 |     "CM = confusion_matrix(prediction, test_label)\n",
512 |     "print(CM)\n",
513 |     "print('\\nTest Accuracy: ' + str(np.sum(np.diag(CM)) / np.sum(CM)))"
514 |    ]
515 |   },
516 |   {
517 |    "cell_type": "code",
518 |    "execution_count": 13,
519 |    "metadata": {},
520 |    "outputs": [
521 |     {
522 |      "data": {
523 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD8CAYAAABJsn7AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXd8VMXXh5/Z3SSbhBRCIJAEOwQLKtJ7R5qCiCBNUBAFFVABCyKC6A9p9oYiTaRJr1IEpIYAQZASKUJICBBaSEjfnfeP3cSElN2Y3Ztk33n43A93b5kzc+/N2dlzZ85XSClRKBQKhfboSroCCoVC8f8V5YAVCoWihFAOWKFQKEoI5YAVCoWihFAOWKFQKEoI5YAVCoWihFAOWKFQKEoI5YAVCoWihFAOWKFQKEoIg7MNpGz8RpOpdj6dP9LCDAB6nWt+b5nNZs1saTX/8k7fII0swbmblzSzJTSzpN29AshMjy120zKunLG7ym6B92h5KfPgmp5EoVAoygBO7wErFAqFpphNJV0Du1EOWKFQuBamzJKugd0oB6xQKFwKKbV7l1FclANWKBSuhYYvk4uLcsAKhcK1KEM9YIeNgggLC3s9LCzsaFhY2F9hYWELwsLCjLcf89vBv+n20Ty6fTSPt2evL7bNhFupvPTVMp6YMJsqlY1kjQ4r520gNMST0BBPQqp44u6et5mhocFs3riEI4e38eeh33nt1YHFrk9Ovv9+KuejIzl4YHP2tnHjRrI/YiP7wjewds18qlRxzBApLW3djk6nI2Lfb6xYPscp5QP8MGMaF2L+5FDkFqfZGDC4F+t3LGb9ziUMeKk3AG9/MIKNe5aydvsivp0zFR/fcg63+3i7Fhz96w9OHNvJ6FGvOLz8nGhxr0DbNuWL2WT/UsI4xAGHhYWFAMOAOlFRUQ8BeuDZnMecu3ydnzbtZ/brz7BsTD9GP93c7vIjTsYwdt7GPNt/2rSf+tWrsvr9AaSkmPD3cwcgI9PMhbgUYmJTuH4jnYoVPPKcm5mZyajR46n5cAsaN3mCIUMGcP/91YrS7EKZN28JTzzZL9e26dO/o07ddtSr35516zYz5t3hZc7W7Qx7bRDHT5x0StlZzJ27mE6d+zit/Oo17qVnv6d4qt1zdG7+LK3aNeWue6qyc9teOjTpQafmPfnndDRDRrzgULs6nY4vPv+Izk/0peYjLenZs6tDn8Hb0eJead2mfJFm+5cSxqYDFkLUEEK8JYT4QgjxuXX9/nwONQCeYWFhBsALuJBz57LdR+nZ9GF8vSwd4wAfr+x9szcfoPeUBTzzv5/5Zu0euyu/7chpnqj/AACJSZl4e1kiKmlp5uwwUGqaCYMh71jrixcvE3noLwCSkm5x4sRJQoIr223bFjt3hnP9+o1c2xITk7LXvby9cJQalJa2chISUoUOHVrz008LHF94DnbsDOfabe1zJPdWv5vIA0dITUnFZDKxb/cB2nVqxc5tezGZLL2kQ/uPUDm4kkPt1qtbi9Onz/LPP9FkZGSwePFKnnzicYfayEKre6VlmwpCmjLtXkqaQh2wEOItYCGWiTf7gAjr+gIhxNtZx0VFRcUCU4FoIA5IiIqKytVlPXf5Oucu36D/9MX0m7aIXcfOArD7+Dmi428wf+SzLHqrD8fPX+bAqVi7Kn81MZmKft4AmEwSvT6vo/Up50ZySuE/Ne68M5RHH3mI8H2RdtktDuPHj+bUqXB6PfsU4ydMLdO2pk0bzzvvTNR0Bp0z+Pv4aeo1fAz/8n4YPY00b9OEKsG5Qzbd+3Rh+5bdDrUbHFKZ8zH/9lNiYuMIdmAnICda3Sst21QgZrP9Swljqwc8EKgrpZwkpfzZukwC6ln3ARAWFlYe6ALcDQSnp6c/4O/vf0YIsX/mup0AmMxmouNv8OPwp5nUvz3jF2zhZnIae09Es+fEOXp+8gvPTv6Fs5euEx1/HYC+UxfSY9J8Jvyyme1/naHHpPn0mDSf3cfP2dU4o1GPr48bV6+lFXiMt7cXixf9wBsjx+XqNTqLceMmc9999VmwcDlDhgwos7Y6dmxD/OUrHIw84tByS4LTJ//h+y9mM2fpN8xa/BUnjv5NpunfL+2hrw/ElJnJyiXrHGpXiLwdBmeI5Gp5r7RqU6GUoRCErVEQZiAYuN3jVbHuy6IN8E9UVFQ8QFhY2KdBQUENbty4MTQrF0SQfzlq3lUFN72ekEA/7qrkT3T8daSUDGxbl+5NauYx/vNISxg54mQMq/Ye48N+7XLtr+DjRXzCLSr6eaPXC0ymf2+0u5uOSoEexF1MKfCLzmAwsGTRDyxYsJwVK4r/UrAoLFq0ghXL5/Dhh9PLpK1GjerQuXM72rdvhdHoga+vD3Nmf0H/AcMcZkNLlsxfyZL5KwF4c8yrXLxgyevQrWdnWrZrSr9uLzvcZmxMHFVDg7M/h4ZUIS7O8fkktLxXWrWpUErByzV7sdUDHgFsEUKsF0LMsC4bgC1Azrc60UCDsLAwr7CwMAG0Bo7nLKjlw/cScTIGgOtJKZy7fIPQQD8a3n8nK/YeJTktHYBLN5K4lphsV+Wb17yH1eHHAPApZ+BWsiWmY9ALKgcZuRSfSkZmwd++P8yYxvETp/js8xl22Ssu9917V/Z6505tiYo6VWZtvffeJO6+pw7VqjegT9+hbN26q8w6X4AKgeUBqBJSmcc7t2T1sg00a9WIwcMG8FLfEaSmpDrcZsT+Q9x3393cdVdV3Nzc6NGjC6vX5H3ZXFy0vFdatalQXKUHLKXcIISojiXkEIIl/hsDREgps79moqKiwsPCwn4FDgKZQCQwIywsbMJng5+gRc17aHT/new5EU23j+ahE4LXuzbB39uTRvffyT+XrvHctMUAeHm48dFzj+d6SVcQL7Stw+if1rF871E8PfVcumz5Iylf3h2dTmSPfpBA7IWUXOc2blSXfn27c/jIMfZHWB6QsWMnsX7D77avmh3MnfsVzZo2IDAwgNOn9vHhxGm0f7wV1avfi9lsJjo6hldfe7fM2SoJfp73Nc2bNSQwMICzZ/YzfsJUZs1e6FAbX8+ain+AH5kZmXww+hNuJiTywaS3cPdwY86v3wJw6MARxo782GE2TSYTw0e8x7q1v6DX6Zg9ZxHHjv3tsPJLglLRplLwcs1ehLPjMyodZdlBpaMsHiodZfFxRDrKtMO/2V1lj4cfL9F0lGomnEKhcCly/Dgv9SgHrFAoXItSENu1F+WAFQqFa1EKxvfai3LACoXCtVA9YIVCoSghTBklXQO7UQ5YoVC4FioE8S9aDQ9LubBDEzsAnsFNNbOlKB7RGg4N0xKNJ/eWLVQIQqFQKEoI1QNWKBSKEkI5YIVCoSgZpHoJp1AoFCWEigErFApFCVGGQhClLquMPYJ+fr5uVA3xpGqIJ36+bnn2Jybd4pXR4+jWfyhd+rzE8rXFT4eXcDORQcPfpWPPgaVOALSkbGklvqhlm7JwNQFLLYRNsyhxUc4ylI6yVDlgewT93N10+PoYiLmQwvnYFLy89Ljdpvm2YOlq7r3rDpbN+YZZX33ClC9/ICPDvrjQvoOHGTNxWp7tP85bTIM6j7Ju0cxSJwBaEra0FF/U8vpl4WoCls4WNs2iVIhyupAkkabYI+jn5iZITTNni0ympprw9s4dSRFCcCs5BSklySmp+Pn6oNfrAfhp/q/0HDiMp54bwlc/zrO7blt37KFLhzZA6RMALQlbWoovann9wDUFLJ0tbJpFaRDlVD3g/4g9gn7pGWaMRj06HQgBXp4GDLeJcfZ++gnOnD1Pyy59eOq5Ibw94mV0Oh27wg8QHRPLwh8/Z+nsrzkWdYr9h+zTybp6/QYVAwOAsiMA6kxbJSW+qMX1+38lYOlgSkWbMjPtX0qY//wSTgjxvJRyliMrY4+gX0aG5MaNdIIre2KWkJZuzjMraNe+A9Sodg8/fTmJ87FxvDjiXWo/8iC7Iw6ye99Bug94FYDklBTOnb9AnUdr0uvFEaSnZ5CckkLCzUSe7m+JXb0x9AUa169ts+5ZAqCxcQXLKWkpAOpsWyUhvqjF9cspYNmsWUOn2MiiVAhYOphS0aZS0LO1l+KMghgP5OuAhRCDgcEAQu+HTudtV4H2CvolJmWSmGT59goo705mZu4LvnztJgb17YEQgjtCgwmpUpl/zsWAhEH9etKja8c8ZS744TPAEgNeuW4TH733Zq79Fcr7E3/lGhUDA0q9AKgWtrQWX9Tq+v2/E7B0MKWiTaUgtmsvhYYghBCHC1iOAAVqvUgpZ0gp60gp69jrfMF+QT+9zvIta9ALvL0MJN3K/VOiSlBF9h44BMCVa9c5Gx1DaHBlGtV7jOVrN5KcbNGHuxR/hat2xsVaNGnAyvWbgdIvAKqFLa3FF7W6fv/vBCwdTKloUxmKAdvqAQcBjwPXb9sugN2Oroy9gn5BQUb0OoGUkitX0zCbYdHytQD0fKoTLw/ozZiPpvFUvyFIKXl96AuU9/ejcf3anDl3nj4vvQGAl6eR/70/igrl/W3WbVC/Hrw59mOWrfmt1AmAloQtLcUXtbx+WqLlNdRC2BRKiShnGeoBFyrKKYSYCcySUu7MZ98vUsretgwY3EM0CQCpbGiK/HBV8UpXxRGinCnLPrb7Vnh2e7dERTkLDUFIKQfm53yt+2w6X4VCodAcB46CEEK8LoQ4KoT4SwixQAhhFELcLYQIF0KcFEIsEkK4W4/1sH4+Zd1/l63yS9UwNIVCoSg2Utq/FIIQIgQYBtSRUj4E6IFngU+AT6WU1bCEZ7OmZg4Erksp7wM+tR5XKMoBKxQK18KxM+EMgKcQwgB4AXFAK+BX6/45QFfrehfrZ6z7W4v8xuXlQDlghULhWjjIAUspY4GpQDQWx5sAHABuSCmz4hcxQIh1PQQ4bz0303p8hcJsKAesUChciyIMQxNCDBZC7M+xDM4qRghRHkuv9m4gGPAGOuRnMeuUQvbli0pHqVAoXAtT4ekAciKlnAEUNLi8DfCPlDIeQAixDGgE+AshDNZebiiQNfc6BqgKxFhDFn7AtcLsO90BG3R6Z5sAwCe0BZlm+y98cUhc+ZYmdgB8u9iM4zsMg1677+MMkzbz8Mu5e2piByAxPcX2QQrn47hxwNFAAyGEF5ACtAb2A1uB7sBCoD+w0nr8KuvnPdb9v0sb87BdpgeslfNVKBSlHAc5YClluBDiV+AgkAlEYuktrwUWCiEmWrfNtJ4yE5gnhDiFpef7rC0bLuOAFQqFAnDoFGMp5Thg3G2bzwD18jk2FXimKOUrB6xQKFwKaS47cxKVA1YoFK5FGcoFoRywQqFwLYowCqKkUQ5YoVC4FmWoB1ziEzG+/34K0dEHOXBgU/a2mjXvZ9u25ezfv5GlS3/Cx6ecw+3aoxKbU325UkUPbp9UuDL8OC3HzKTH5IX0mLyQZXuOFbteCbdSeemblTwx8Wde+mZlLvXlkBBPQkI8CS5AfTm73n6+LFw4gyNHtnP48DYa2KHoYQ+hoVXYsGEhkZFbOHBgE6+88jwA77//Jvv2bWDv3nWsXj2PKlUqOcReTpyttKvT6di+axULl1iGhL74Uj8O/LmF60mnCKhQ3uH2stBCQVhrVekSV0VWopz2M2/eEp588rlc2779djJjx06iTp12rFq1gTfeeMnhdm2pxOr1Aj9ft2z1ZYHFCd5Ou1rVWDz6WRaPfpZuDR+w237EyVjGzs/r/H/acpD61UNZ/V5f6lcPzaW+HBeXQqxVfTkwH/XlLD6dPoGNv22lZs3m1K7d1mHqvpmZJt5+eyK1arWmefOuvPTSc9SoUY1PP/2eevXa06BBR9av38I77wx3iL0stFDafXnoAP6OOpX9ee+eA3R94jmiz8U41E5OtFIQ1lJVulSoIjsoGY8W2HTAQogaQojWQohyt21v74gK7Ny5j+u3qVJUr34PO3aEA7Blyw665iMhVFzsUYkVguxer9AJMk3237DZvx+k97QlPPPJQr5ZH273eduO/MMTdWsA8ETdGnjlo76cVoD6MoCPTzmaNKnPT7Msir4ZGRkkJNy0235hXLx4mUO51IlPERwclEufzcvLy+EaYM5W2g0Orky79i2YO2dx9rYjh49xPjrWYTbyQysFYS1VpUuFKrKr9ICFEMOwzPJ4DfhLCNElx+6PnVWpo0ej6Ny5LQDdunUiNLSKs0wViMkkuZGQwZ1VvbnrDm/MZklKPorHWw6f5plPFjJy1gYuXk8EYPeJaKLjE5j/RncWjerJ8fPxHDh9Ic+5+XE1MZmKfhYZp4p+3kVWX77nnju5cuUqM3/8lIh9v/H9d1Pw8nL8bLA77gjl0UcfJCLCIv30wQejOHlyD88+25UPP5zuUFvOVtr9ePJ7jHvvE8waD18qCQVhZ6tKlwpVZLO0fylhbPWAXwRqSym7Ai2AsUKIrN+XTssk/9JLo3j55f7s3r0WH59ypKdnOMtUgeh04O2l59z5W5yNvoVOiDwhiOYP3c26959jyVvPUr96KGN/sYQU9kadZ8+J8/Scsohnpy7i7OXrRMdbett9py+hx+SFTFj0O9uP/pMdP959PNquehmNenx83Lh2LS3f/Qa9nlq1avL993OpW+9xbt1KZvToV4txJfLi7e3FggXfMWrUhOze7wcfTKFatYYsXLiCl1/u71B7zlTafbx9S67EX+XPQ0cdUl5R0FpBWAtV6VKhimwy2b+UMLZGQeillEkAUsqzQogWwK9CiDspxAHnVEU2GMqj1xftJdrff5+mc+e+ANx33920b9+qSOc7Ak+jnoxMmf0rJSk5E6NRn0sA1N/bmL3ereEDfL56D2B54Aa2eYzujR/KU+7Pb1gmykScjGXVvhN82Kd1rv0VfLyIT7hFRT9v4hNu5VFfrhjowcVC1JdjYuOIiYljX4Slh7N02VpGj3KcAzYYDCxY8B2LFq1g5coNefYvXrySZctmMXHipw6z6Uyl3foNatO+Y2vatmuOh9EDH59yfP/jNF4a9Kbtk4uJlgrCWqlKlwZVZFkKQgv2YqsHfFEI8WjWB6sz7gwEAjULOimnKnJRnS9AxYqWFJpCCN55Zxg//vhzkcsoLpkmidFDlx0D9jLqSc/IfWPjE25lr2//6yx3B1neljescQcrwo+TnJYOwKUbSVxLTLbLbvOH7mJ1xAkAVkecINmqvqzXC4KCjFy2ob586VI8MTEXqF79XgBatWrC8eOOE0X87rvJREWd4osvfszedu+9d2Wvd+rUlr//Pu0we+Bcpd0JH0zlobAmPPJgCwYOGMGO7Xs0cb6grYKwVqrSpUIVuQyFIGz1gJ/DkoQiG2sKtueEEN87ogJz535J06YNCQwsz6lT4UycOB1vb29eftkyMmLFig3MyfFyxFHYUolNSzOTdMtEaLAXIElLN3PzZgbl/d3Z9tc/tHjobhb8cZhtR//BoNPh62VkQm9Lb7ZRjTv459J1nvtsKQBe7m581K8tAT626/VCm9qMnr2B5XuPU6V8OW4kWJx4lvpyztEPt6svZzHi9bHMnfMl7u5unPknmkGD3viPVyk3jRrVoU+fpzly5Dh7964DYNy4KQwY0JNq1e7BbDYTHR3LsGHvOsReFiWhtDt4yHMMGzGYoKBAdu5dw6bftjP81bLZLi1VpUuFKnIpkJu3l0JVkR2B0XiHJl8zWmZDU+koi49W6Sh9VDrKMoUjVJFvTehjt8/xfn9+iaoiq5lwCoXCtcgs+Zdr9qIcsEKhcC3KUAhCOWCFQuFalIKXa/aiHLBCoXApytIwNOWAFQqFa6F6wAqFQlFCKAesPXqddond/LpO1sxW0p/aTULxebSfZra0ItWk/TR2RQlTCqYY24vLOGCFQqEApQmnUCgUJYdywAqFQlFCqFEQCoVCUUKoHrBCoVCUEMoBKxQKRckgTWUnBFHiopz5qSI//PADbN++gvDw9ezatYY6dR5xkK2pnI+O5OCBzdnbxo0byf6IjewL38DaNfOpUiXIIbZu59VXBxJ5cDOHIrfw2mt5VWl9fQyEBHsSEuyJr2/e78Wt4Yd4eviHPDNiIs+++TEHj53Kc0xRSUi8xeBxn9F5yFgGj/ssW4HZ21ufXZcqlY24u+X/mNhqkyNxptKuK6s9Z2GPCrijKHlV5LKTD7jE01E2aVKPpKRkZs78lNq1LTpwa9b8zBdf/MjGjdt4/PGWvPnmy7Rr17NQOxLb7WjSpD5JSbf4aeZnPFa7DWARscySZ3ll6PPcf381Xn2t8LyvRb1mDz4Qxs8/f02jxp1JT89gzZqfee21dzl16h8A3NwElSoauRCXgpRQOcjIlatpZGZKEg/NAyA5JRVPowdCCP4+G8PIKT+w6uvxdtmPOBLFyt/3MHH4gFzbp89eip+PNwOfbs/MpRsY/9Virl/PwMNDR0aGRQTU01OPv78bcXGpRWqTLcxFuIY6nY7jR3fQvmMvYmLi2LtnHX37DeX4cdtqz252pNisXLkSlStX4tChvyhXzpvdu9fQo8dgYmPjsp+NoUMHUKNGNYYNG1NgOUVNsVmcdhWVptZnf9asz3m0VmvbJ/xHitsmR6SjTHi+jd0Pl9+szSWajtIeVeR6Qoi61vUHhBBvCCEcJlOcnyqylBJfX0v2cj8/H4dJmuzcGZ7HVi5FX28vpyhV16hxH+HhkaSkpGIymdjxx166dPlXVNrNTUdaminbdmqqCW+v3I7Dy9OYrbeVkpqeS3tr1vKN9Br5P54e/iFfL1htd7227jvMky0bAvBky4YFKzDnIwxqq02OxNlKu66q9pwTe1TAHUHpUEUuOz3gQrsHQohxQAfAIITYBNQHtgFvCyFqSSk/ckalRo4cz5o185g0aQxC6GjZ8ilnmMlm/PjR9OnzNDcTEmn3eA+Hl3/0WBQTJrxFQIA/KSmptG/figMHD2fvz8gwYyzvjk4HUlp6nWnpeeNYW/ZG8vm8FVxLSOTr9yw6b7sjjxF94TK/THkbKSXDPv6W/UdPUufBajbrde3GTSoG+AFQMcAPvS6voy1XzpCvGrStNjmS/JR269Wt5RRb+ak99+nTjYSERNq3f9ahtrRsl1aUijaVnRCwzZdw3YFHAQ/gIhAqpbwphJgChAP5OuDiinIOHtyPUaMmsGLFep5+ujPffTeFjh17F6mMojBu3GTGjZvMqFGvMGTIAIfLqp84cYopU79h/boFJCXd4vCRY2Rm/vtzNSNDciMhg8pBRqTEoj2Xz5dz6wa1aN2gFvuPnuSrX1bxw4QR7D50jD2HjtHjdcutSE5NI/rCZeo8WI3eoyaRkZFJcmoaCUm3eGbERABG9H+KxrUetFlvo1GHTzk34i7mVXqw1SZHopXSbkFqzx98MIWRI4fy8sv9HSo2WioUhB1MaWiTzCw7HtiWA86UUpqAZCHEaSnlTQApZYoQosBWSilnADPgv0kS9e37NG++OQ6ApUvX8O232sjyLFq0ghXL5zjcAQPMnr2Q2VbNuQ8nvEVMbFyu/UlJmSQlWRxYeX83Mk0FX7Y6D1bjvYvxXL+ZBBIGdm/PM483y3PcL1PeBgqOAQf4+xJ/LYGKAX7EX0vAlOMnmZubRX/u4qXUAse122qTo9BCadfV1J5LilLRprLjf23GgNOFEF7W9dpZG4UQfjixmXFxl2jWrAEALVs25tSps84yxX05FH07d2pLVFTxRxfkR5bSc9WqwXTt2oFFi1bm2p81AkGvF3h5G7h1K3dvMjrucnZP4tjpaDIzM/H38aZRrQdYvnk3ySmWl2SXrl7n6o2bdtWpRb2HWbV1DwCrtu7JrcBcyUj8FcuLwP/aJkehhdKuq6k9lxSloU3SLO1eShpbPeBmUso0AClz6Xy4Af0dUYH8VJGHDn2bqVM/wGDQk5qaxiuvvO0IU8yd+xXNmjYgMDCA06f28eHEabR/vBXVq99rVfSNsTkC4r+yaOEMKlQoT0ZGJsOGj+HGjYRc+4MqGdHpBBLJ1atpmM3g42Ng8YY/6NG+GZv3RLJ6614Mej0eHm5MHvkiQgga1XqAMzFx9H3LkqHNy9OD/414gQr+tus0sNvjjJzyA8s376JyYAAJCZbMYeX93dDpBBUquFsOlHDhtlEQ9rTJUThbaff/g9qzLRVwR1EqVJHLUA+4xIehOQp7hqE5zJaGMa2sYWhaoGU6yqIMQysO9gxDcxRaKT27Mo4YhnbtqeZ2P1wBy7eX7mFoCoVCUaYwF2GxgRDCXwjxqxDihBDiuBCioRAiQAixSQhx0vp/eeuxQgjxhRDilBDisBDiMVvlKwesUChcCplp/2IHnwMbpJQ1gEeA48DbwBYpZTVgi/UzWIbsVrMug4FvbRWuHLBCoXAppNn+pTCEEL5AM2AmgJQyXUp5A+gCzLEeNgfoal3vAsyVFvYC/kKIKoXZUA5YoVC4Fo4LQdwDxAOzhBCRQogfhRDeQJCUMg7A+n9WkpAQ4HyO82Os2wpEOWCFQuFSFKUHLIQYLITYn2MZnKMoA/AY8K2UshZwi3/DDfmR3wu9Ql8IqnSUCoXCpbAVWsh1bI5JY/kQA8RIKcOtn3/F4oAvCSGqSCnjrCGGyzmOr5rj/FDgAoXgdAecaS47CqX2ElwuQDNbleoO0sxW4hHHjwstCO+HCs9u5ygqevpqYgfgQtI1zWwpCkaaHDOyTEp5UQhxXggRJqWMAloDx6xLf2CS9f+sGUirgFeFEAux5M1JyApVFITqASsUCpeiKD1gO3gNmC+EcAfOAM9jCd0uFkIMBKKBZ6zHrgM6AqeAZOuxhaIcsEKhcCmk2XFzK6SUh4A6+ezKk1RZWmZoFSkDvXLACoXCpXBwD9ipKAesUChcCilLdHZxkVAOWKFQuBSqB6xQKBQlhNlBoyC0oFRNxAgNDWbzxiUcObyNPw/9zmuvOk9p19kqsffcdxfrty/JXo6e28PAl/tm7x/8an+irx2hfIAdeSMLwcPDnd+3LWPnnjXsjVjPO2OGA/DNd5M5/Nc2duxezY7dq6lZ8/7sc7y8BRUq6alQSY+Xd8EP619//8OjXQaxcdf+YtURICExicFjp9F58DsMHjstO/9xOW8DoSGehIZ4ElLFE3f3/B9JZyrtanWvbkcrpWKhY7sXAAAgAElEQVQPDw/27FrDgf2b+PPQ74x7/02n2itpVWRpFnYvJU2p6gFnZmYyavR4Iq3qtPvCN7B5yx9OUYmdO3cx33wzi1mzPnd42QBnTp2lQ3PL6BSdTse+o1vYsMbyh1YlJIimLRoSc77QMdp2kZaWzhOd+nLrVjIGg4HfNi1i08btAIx9bxIrV+RWdzAYwNNbx9V4E0goX0FHWqrEdNtwbZPJzKdzfqVRrYeKVJ+IIydYuXkXE1/P/eU589f11H/4fgY+05GZS9axftshrl1PJyPTzIW4FMxm8PLUU7GCB7FxuSWQdDodX3z+US6l3dVrNjrsudDqXt2Os5/BLNLS0mjTrkf2M/LHtuVs2LCV8H0HHW7L2ffKHkqDY7WXIveAhRBznVERsKjTRuZSpz1JSHBlp9jSSiUWoHHz+kSfPU9sjGVM9riPRvPxuOkOyyt861YyAG5uBtzcDIWWqzcIMtJl9gTJ9HSJ0TPvA/vLmi20bVSbAD+fXNtnLdtAr9c/5OnXxvH1/BV213FreCRPtm4EwJOtG2WrPudUYE5NM2Ew5K2Llkq7zr5XOdHyGcz5jBjc3JyW07o0qCJLaf9S0hTqgIUQq25bVgPdsj47s2J33hnKo488RPi+SGea0YQnu3Vg5dL1ALRt34KLcZc5ftRxKgE6nY4du1dz6p99bP19Fwf2/wnA2PffZNfetXw8aQzu7hZ1i8xMibuHQOgAAR5GHbrbZOcvXb3O73sO8kz7Frm27z74F9EXLvHL9PdY8vk4jp86x/6/ouyqo0WB2fITvmKAP/p8pO59yrmRnI8Cc35Ku8FO+mJ29r0qKXQ6HfsjNhIXe5gtW/5gX4Rz/q60vFcF4UohiFAs0+5+xNJnElgGJU8r7KScqshC74dO512kSnl7e7F40Q+8MXJctjptWcXNzUDb9i34ZMLnGD2NvPrmi/Tt9pJDbZjNZpo2egI/Px9+XvAd9z9QnfHjpnDpUjzu7u58/uVHjHhjMJMnfYUpE24lmgmooEdKyMzI2w2Y/MMCRgzojl6f+/t5d+RR9kQepcfw8UAOBeaHwuj95sR/FZgTk3hm2AcAjBjQncaP2Q5jGI16fH3ciI1LzrNPK6VdLe5VSWE2m6lTtx1+fr4sXTKTBx8M4+hR+748i0KpUEV2oWFodYDhwBhglJTykBAiRUq5vbCTcia4MLiHFOnqGwwGliz6gQULlrNixfqinFoqadGmKX8dPs6V+KuE3V+NqneEsGHHrwBUCQ5i3bbFPNmmF/GXrxbbVkJCIjt37KVNm2Z8aRWXTE9PZ/7Pv/LasH9zSqQkS1KSLT3Ncr46TLcpMB89eY63pnwPwPWbSew4cASD9a3ZwO4deaZDizy2f5n2HlBwDNiiwHyDigH+xF+7kcumu5uOSoEexF1MyVeBWSulXS3vVUmRkHCT7X/strwoc4IDLg2qyCZXGQUhpTRLKT/FMqd5jBDiK5z84u6HGdM4fuIUn31eUIKiskWXp//9SRt1/CSPhbWg8aPtafxoe+IuXKJjix7F+oOuEBiAnzVOazR60KJlY/7++zRBQRWzj+nUuS3HcwgjZo1A0OnBaBSkJud2wBtmfsKGmZPZMHMybRvVZsyQvrRq+BiNaj3I8s07/6MC86Os2rIbgFVbdnPLqsBs0AsqBxm5FJ9KRgEKzFop7Tr7XpUUgYEB+PlZkhIZjUZat2pKVJRjFZ6zKBWqyFLYvZQ0djlTKWUM8IwQohNg31/cf6Bxo7r069udw0eOsT/CctPGjp3E+g2/O9yWFiqxRk8jTVs05J3XJzi03JxUDqrIdzOmoNPr0el0LF+2lt82bGX12p+pEBiAEIIjh4/x+vCx2ef4B+jR6SwxpZsJZqQETy/B4vXb6JFP7zaLRo89ZFFgHvUxAF5GD/735otU8LedcWxg946M/ORblm/aQeWKAdxISAegfHl3dDpBxQoegKVOsRdyj4LQQmlXi3t1O1opFVepEsRPMz9Dr9eh0+n49dfVrF232eF2oHSoIpeG2K69OF0VuaghiLKAlukoE9LzxkSdxeWDc2wf5CC0Skep5b1S6SiLjyNUkY9X62i3z7n/5LoS9dalahywQqFQFJey1ANWDlihULgUJnOpmuBbKMoBKxQKl6I0TLCwF+WAFQqFS2EuBaMb7EU5YIVC4VKUhuFl9qIcsEKhcClUCMLFcdXhRloNDQNIPrPB9kEOwOue9prYAdDlMw3XWZg19DJlpz9pQYUgFAqFooRQoyAUCoWihChDEQjlgBUKhWuhQhAKhUJRQqhREAqFQlFClCFR5NIlygnaCvq5oq3S1iY/XzeqhnhSNcQTP1+3fI+JOPQX3QePpOsLIxjw+vvFrld6egYjP5xOx36vElLFM1vmyNOoJzTYIgAaGuyJp1Gf51ythDKzePXVgUQe3MyhyC289przRGi1fC78/HxZuHAGR45s5/DhbTSoX9up9m5HIuxeSppS5YCzBP06P9GXmo+0pGfPrtx/fzVlq5TZsdeWu5sOXx8DMRdSOB+bgpeXHrfbNN9uJt1i4uc/8uWHb7Hip8+YVgTF3tiLl3n+jbwOe9n6LfiW82bdvK9IuJlBhfIWOSaTWRJ3KZWY2BQux6dRqaJHnnPnzl1Mp8597K5DcXjwgTAGvtCLRo07U7tOOzp2bMN9993tcDtaPhcAn06fwMbftlKzZnNq127L8RPaCXICZEph91LSlCoHrKWgnyvaKm1tcnMTpKaZswfGp6aa8PbOHfVat2UHrZvWp4o1gXyF8n7Z+1Zv+oNeQ9+m++CRjJ/+PabbpZsLYOvuCJ5s1wKApFuZeHpabKanm7OVONIzzPnK52gplFmjxn2Eh0eSkpKKyWRixx976dLF8eOWtXwufHzK0aRJfX6atQCAjIwMEhKclkI8X1y2ByyEaCKEeEMI0c4ZldFS0M8VbZW2NqVnmDEaLcnfhQAvTwOG28Q4z8XEcTMxieffeJ8eL49m1cZtAJw5F8Nv23Yx94uJ/DpjKnq9jrVbdthVt8tXrlG5UmD2Z7NZZquAZOHtpSct3T6H7iyOHouiadP6BAT44+lppH37VoTmkPNxFFo+F/fccydXrlxl5o+fErHvN77/bgpeXp5OsVUQ5iIsJU2hL+GEEPuklPWs6y8CrwDLgXFCiMeklJMcWRktBf1c0VZpa1NGhuTGjXSCK3tilpCWbs4zRjPTZOL4yTP8MGUcaenp9H3tXR6+vzp7I49w7OQZeg19G4C0tHQCrMobw9+fTOzFy2RkZBJ3+QrdB48EoE+3jjzVvpXNNru56agQ4MGFiymFHudsTpw4xZSp37B+3QKSkm5x+MgxMjMzHW5Hy+fCoNdTq1ZNRowYy76ISKZPG8/o0a/ywQdTnGIvP0pDz9ZebI2CyPnWZDDQVkoZL4SYCuwF8nXA/1UVWUtBP1e0VRrblJiUSWKSxakElHcnMzN3vyOoYgXK+/ng5WnEy9NI7ZoPEHXmLFJKnmzXghGD8sZjP58w2lKHi5d5b/JXzJo+IU+ZFy9foXLFCgDodCJb7FNv1aC7HJ9KZgEadFoye/ZCZltliD6c8BYxsXEOt6HlcxETG0dMTFy27P3SZWsZPepVp9gqiNLQs7UXWyEInRCivBCiAhb5ongAKeUtoMCvainlDCllHSllnaJI0msp6OeKtkpjm/Q6S2/EoBd4exlIupX7sWnVqC4Hjxwn02QiJTWNIydOcs8doTSoVZNNf+zh6vUEABJuJnLhUrxddWvRsE52KKOct4GUFItNnQ6qBBm5di2N1LTS8Wda0folUbVqMF27dmDRopUOt6Hlc3HpUjwxMReoXv1eAFq1asLx49pqwpkQdi8lja0esB9wAEs+DimEqCylvCiEKIcTcnRoKejnirZKY5uCgozodQIpJVeupmE2g6+PgcWrf6PHE49zz52hNK5bi6cHvYlOJ+jWsTXV7r4DgNee78VLb32I2WzGYDAwZtgggnOoPRdEt46teed/X9Cx36v4+blx6bJFxdnX1w03Nx3l/d0p7285Nu5iKibzvz1hrYQys1i0cAYVKpQnIyOTYcPHcONGgsNtaC2UOeL1scyd8yXu7m6c+SeaQYPecJqt/ChDikT/TZRTCOEFBEkp/7F1rCuKciqKj8qGVjxcNRtahgNEOVdW7m33xely8ZeyJ8oppUwGbDpfhUKh0Jqy1ONTU5EVCoVLUTqi+/ZRqiZiKBQKRXExC2H3Yg9CCL0QIlIIscb6+W4hRLgQ4qQQYpEQwt263cP6+ZR1/122ylYOWKFQuBSmIix2Mhw4nuPzJ8CnUspqwHUgK4nHQOC6lPI+4FPrcYWiHLBCoXApzML+xRZCiFCgE/Cj9bMAWgG/Wg+ZA3S1rnexfsa6v7XIbxZMDpQDVigULoUZYfcihBgshNifYxl8W3GfAaP5N7RcAbghpcwa0B4DhFjXQ4DzANb9CdbjC8TpL+Hu9A1ytolszt10zuye2ynvWU4TOwCJadpNl9XfnjDBiWg1POzmlCc0sQPgO2q1Zra0HDtVlkYVQNHqK6WcAczIb58QojNwWUp5QAjRImtzISYL25cvLjMKQivnq1AoSjcOnIjRGHhSCNERMAK+WHrE/kIIg7WXGwpkZTqKAaoCMUIIA5aJbIVKqKsQhEKhcCkclQ1NSvmOlDJUSnkX8Czwu5SyD7AV6G49rD+QNX98lfUz1v2/Sxsz3VymB6xQKBQAJufHZ94CFgohJgKRwEzr9pnAPCHEKSw932dtFaQcsEKhcCmcMRFDSrkN2GZdPwPUy+eYVOCZopSrHLBCoXApytJMOOWAFQqFS1EKpN7splS8hBswuBfrdyxm/c4lDHipNwBvfzCCjXuWsnb7Ir6dMxUfX8cO/QoNDWbzxiUcObyNPw/9zmuvOl6R9sDhLWzfvYqtO1awadtSAPzL+7FkxU+EH/yNJSt+ws+q8lAcvv9+CtHRBzlwYFOu7UOGDODw4a0cPLiZjz56t9h2QkKqsG79Ag4c3EzE/o0MHfp89r6XX+5P5KEtROzfyMSJbxfb1u3YUvWtXv1e9kdszF6uXTnBsNcG5TlOF1IdY+8xGPu+j8fTDkiTqDfg3mEQxv4T2L1zNXfeGQpAm9ZNCd+7nsiDmwnfu56WLRr/p3Y5Ep1OR8S+31ixfI7tg4uBlm3Kj7IkSfSf0lEWhXsDHyvUQPUa9/L5D//jqXbPkZGewazFX/H+qI8JvSOEPTsiMJlMjH5/GACTJ3xRYDlFHYZWuXIlqlSuROShvyhXzpt94Rt4uvsLHD9uW8HV3nHABw5voW2L7ly7dj172/sTRnHj+g2++PQHhr3+In7+fnw4bmqBZdgzDrhJk3okJSUzc+an1K7dFoDmzRvy1luv0bXrANLT06lYsQLx8VcLLcfWOODKlStSuXIlDh06Srly3uzctZpnew6mUqWKjB79Ct26vWC3rbTMDJvtykKn03H86A7ad+xFTEwce/eso2+/oQXeK51OR/TZAzRq0pm/Xnvs3x3unhh7jCZt5RfIxOvg6QMpiXbVQfhUwL1df9KWTs+13fBwc0RgCBm//8KgCEHXLh3o3WcIjz76IJcuXSEu7hIPPhjGujXzufPuOsVqV6762FXr3IwYPpjHaj+Mr48PXZ/qb/sEK0XxEMVpE0CmA9JRflm1r91Vfu38zyXaXy70L04IUV8I4Wtd9xRCjBdCrBZCfCKE8CvsXHu5t/rdRB44QqpVGXbf7gO069SKndv2ZqvgHtp/hMrBlRxhLpuLFy8TeegvAJKSbnHixElCnCRUmJMOHVuz6JcVACz6ZQUdO7Updpk7d+7j+m1Kvi++2I+pU78hPT0dwKZDtIeLF+M5dOgoYLlmUVGnCQ6uzKAX+zBt2rcOtZWToqr6tm7VhDNnzhEdHZtru6FGPUynIy3OF3I5X31YPTx6vo2x9xjcWvW2qIjagf6ehzEd2wPA0qVradWyCQCHDh3Nlv05ejQKo9GIu7t7sdpVHEJCqtChQ2t++mmBU8rPQss2FYQjpyI7G1shiJ+AZOv651gGFn9i3TbLERX4+/hp6jV8DP/yfhg9jTRv04Qqwblnz3Xv04XtW3Y7wly+3HlnKI8+8hDh+yIdWq4ElqyYyebtS+k3oAdgkaC5ZJXWuXQpnsCKAQ61mUW1anfTuHE9/vhjJZs2LaZ27YcdWv4dd4TyyCMPEBFxiGrV7qFR43ps276CDb8t4jEH2yqqqm+PHl1YuGhFnu3CvxJ4eOHx9BsYn30HfY36lu3lK2OoXoe0JZNJ/eUjkBJ9WJ6X3PkivP2RSRaHbjKZSEi4SYUK5XMd061bJw4d+iv7C+q/tqs4TJs2nnfemYjZ7Nwf3lq2qSDKUgjC1ks4XY45z3WklFm/53YKIQ45ogKnT/7D91/MZs7Sb0i+lcKJo3+Tafo3T9HQ1wdiysxk5ZJ1jjCXB29vLxYv+oE3Ro4jMTHJoWV3ateLSxcvExgYwJIVszj19xmHll8YBoMBf38/mjXrQp06jzB//jfUqNHEIWV7e3vxy4JvGT16AomJSRj0evz9fWnRvCu16zzCvHlf8+ADTR1iC4qm6uvm5sYTndsx5r3/5d2p06OrdAdpyz4DgxvGnm9hvvgP+qo1EJXuwPjsO5bjDG6QnIgJcO/0Mjq/CqAzIHzKY+w9BoCMQ79ber751u3f9QceqM7/PnqXDp16F6tdxaFjxzbEX77CwcgjNGvW0OHl50RLBeaCKA2O1V5sOeC/hBDPSylnAX8KIepIKfcLIaoDBQbxcqoiB3pXxdcYWKiRJfNXsmS+ZTLJm2Ne5eIFy0+3bj0707JdU/p1e9nuBhUFg8HAkkU/sGDBclasWO/w8i9dvAzAlSvXWLdmE7VqP0x8/FWCgipy6VI8QUEVuRJf6EzF/0xsbBwrV1ratH//n5jNksDAAK5cKZ49g8HAL798x6KFK1i18jeLrQsXs9cP7P8Ts9nsEFtZFEXVt337lkRGHuHy5St59smk65hTkiAzHTLTMceeRBcYCgJMx/eSsTtvrzl97XdAwTFgmXQdUa48MukGer0ePz/f7Jh/SEgVfl0yk+dfGM6ZM+eK1a7i0KhRHTp3bkf79q0wGj3w9fVhzuwv6D9gmMNtaanAXBBlKXeFrRDEIKC5EOI08ACwRwhxBvjBui9fcqoi23K+ABUCLT/ZqoRU5vHOLVm9bAPNWjVi8LABvNR3BKkpqfa2p0j8MGMax0+c4rPP883FUSy8vDzxLuedvd6iVWNOHDvJhvW/07O3JXtdz95dWb9ui8NtA6xatZEWLRoBcN99d+Pu7uYQh/jtt58QFXWKL7+cmb1t9eqNNG/R0OG2siiKqu+zPbvmG34AMJ3+E13IfSB0YHBDF3QX5usXMZ2PQl/tMctLOQAPL4SPfaEh05nD6B+wtP3ppzuxddsuAPz8fFm1ci5j3vsfu/fsL3a7isN7703i7nvqUK16A/r0HcrWrbuc4nxBWwXmgihLMeBCe8BSygRggBDCB7jHenyMlNKhX2lfz5qKf4AfmRmZfDD6E24mJPLBpLdw93Bjzq/fAnDowBHGjvzYYTYbN6pLv77dOXzkGPsjLA/I2LGTWL/hd4eUX7FSBWb//DUABoOeZb+u4fctO4g8eIQf53xGn37diYmJY2D/4cW2NXfulzRt2pDAwPKcOhXOxInTmTNnETNmTOHAgU2kp6c7RJm2YcM69O7zNH8dOc6evZaQ0AfjJjN3zmK++24yERG/kZ6RweAX3yy2rZzYq+rr6WmkTetmDBn6VvY2Q01LKCTzyA7k9YuYzh7F2GcsSDOZR3chr1rilRm7V2J8apglpGAykb5tITLR9pdI5tFduD/+PMb+E3i9USy9+w4F4JWhz3PfvXcx5t0RjHl3BAAdOvbK9YJSa7ViLSgNbSpCovUSp8SHoTkKLbOhqXSUxacow9CKg0pHWXy0/EnviGFoH93Zx+4qjzk3v+ypIisUCkVpxZVewikUCkWZoiy9hFMOWKFQuBSqB6xQKBQlRKYoO31g5YAVCoVLUXbcr3LACoXCxVAhiBy4oljm9RTHTlkuLWSatRtBadDpNbGj5dCw5LPaTTjwuquddrbcPDSz5QjMZagPrHrACoXCpSg77lc5YIVC4WKoEIRCoVCUEKYy1AdWDlihULgUqgesUCgUJYRUPWCFQqEoGcpSD7hUqCLn5IcZ07gQ8yeHIp2TJzcnWqq3amXLVdqUn9LzvHlfEx6+nvDw9URF7SI83PFJ9MF2u/x83aga4knVEE/8fN3y7J+1cAXdB71B90Fv8NTzw3mkdXcSbton/lkQ6ekZjBw/lY59htJ7yFsYDJYkXp5GPaHBnoSGeBIa7ImnMe/wPmcrgHt4uLN1+3J27V1LeMQG3h1jSb/540+fciByM3sj1vP1t59gMGjT3zMj7V5KmlLngOfOXUynzn2cbken0/HF5x/R+Ym+1HykJT17duX++6uVaVuu1KZ585bw5JPP5drWr98r1K/fgfr1O7B8+XpWrtzgMHtZ2GqXu5sOXx8DMRdSOB+bgpeXHjdD7oyGzz/blV9/nM6vP05n+It9qfPIA/j5+thlP/biZZ4fMTbP9mXrNuPrU45187+h3zNPUKG8ReDTZJbEXUolJjaFy/FpVKqYd8xuZmYmo0aPp+bDLWjc5AmGDBng0HuVlpZO5459aNygE40bdqZN22bUrfsoixetpHatNjSo2wFPTyP9B/R0mM3CkEVYShpbqsjDhBBVtaoMwI6d4Vy7TeHXGWip3qqVLVdqU35Kzznp3r0zixatdJi9LGy1y81NkJpmztZ9S0014e1dcM9u3ZaddGj1rz7e6k3b6TVkNN0HvcH4ad9mK3/bYuuuCJ58vCUAbZs3xNPTYjM93YzJZKlMeoY5X002LRTAb92yaPe6uRkwuBmQUrLxt23Z+w/s/5PgEG3EOTORdi8lja0e8IdAuBBihxBiqBCiohaV0gIt1Vu1suWKbcqPJk3qcenSFU6fPuvwsm21Kz3DjNGoR6eziGd4eRow6PPP6Z2SmsauiEjaNmsAwJlzMfy2dRdzv/yYX3+cjl6nY+3mP+yq1+UrV6lcqQIABr0es1lye/58by89aemFO3RnKYDrdDp27lnD6bMRbP19F/v3/5m9z2Aw0LNXVzZvsq+txUUW4V9JYysocwaoDbQBegLjhRAHgAXAMillvoGtnKKcQu+HTuftuBo7CC3VW7Wy5Yptyo8ePbqweLHje79gu10ZGZIbN9IJruyJWUJaesGRxO27I6j1UI3s8MPeg4c59vdper08GoC09HQCyvsBMHzsJGLjLpORmUncpSt0t0pI9Xm6E091aI2tS+vmpqNCgAcXLhasoOJMBXCz2UyThp3x8/Nh/oLvuP+B6hy3ShFN/2wCu3dFsGd3hENtFlgXTaw4BlsOWEopzcBGYKMQwg3oAPQCpgL59oillDOAGQAG95CS/5rJBy3VW7Wy5Yptuh29Xk+XLu1p1KiTU8q3p12JSZkkJmUCEFDenczM/P/k12/dSYdWTbI/Syl58vGWjHixb55jP//wbYv9i5d5b9KXzPrsw1z7gypW4OLlq1SuGEimyYROJzBbzer1gspBRi7Hp5KZmf+fm7MVwLNISEhk545w2rRtxvFjf/P2O8MIDAygT68hTrN5O6WhZ2svtkIQuboDUsoMKeUqKWUv4A7nVcv5aKneqpUtV2zT7bRq1YS//z5NbOxFp5RvT7v0OsufhUEv8PYykHQrM085iUm32P/nMVo2rpe9rcFjD7Np+x6uWmPbCTcTuXDxsl31atGoLqt+2wrApu17SEmx2NTpoEqQkWvX0khNK7jv50wF8AqBAfj5WXr5RqMHLVo25mTUGZ7r34PWbZrywoDhmv06AksP2N6lpLHVAy7wtaWU0ilqkT/P+5rmzRoSGBjA2TP7GT9hKrNmL3S4HS3VW7Wy5Uptyk/pefbsRfTo8SSLFq1ymJ3bsaddQUFG9DqBlJIrV9Mwm8HXx8DiVb/R40nLC7stO8NpVOcRvDyN2efde1dVXnuhFy+NmoBZSgx6PWNGvEhw5Uo269WtU2ve+fhzOvYZip9vOa5eTwfA19cNNzcd5f3dKe9vOTbuYiom878Oz9kK4JUrV+K7GVPQ6/XodILlS9exYcPvXEv4m/PRsWzeuhSA1St/45NJXzrEZmGYNHT2xcXpqsilNQShKFm0SkepZYpNlY6y+Ny8dabYKsW973zKbp/zy7nlShVZoVAoHEVZigErB6xQKFyK0hDbtZdSNxNOoVAoioOjpiILIaoKIbYKIY4LIY4KIYZbtwcIITYJIU5a/y9v3S6EEF8IIU4JIQ4LIR6zVVflgBUKhUvhwIkYmcCbUsr7gQbAK0KIB4C3gS1SymrAFutnsAzRrWZdBgPf2jKgHLBCoXApTFLavRSGlDJOSnnQup4IHAdCgC7AHOthc4Cu1vUuwFxpYS/gL4SoUpgNFQNWKBQuhTOynAkh7gJqAeFAkJQyDixOWgiRNY4wBDif47QY67a4gsp1ugPW3z5h3UmYzNqF3rUct1J23ucWDa2Gh+nymVrsLLQcGpZyYYdmtryCm9o+qBRRFE+QM22ClRnWmbw5jykHLAVGSClv5jddPevQfLYV+iesesAKhcKlKMowtJxpE/LDmn5hKTBfSrnMuvmSEKKKtfdbBciazhgD5MweGQpcoBBUDFihULgUDhwFIYCZwHEp5fQcu1YB/a3r/YGVObY/Zx0N0QBIyApVFITqASsUCpfCgbN7GwP9gCNCiEPWbe8Ck4DFQoiBQDTwjHXfOqAjcApIBp63ZUA5YIVC4VI4SpZeSrmTgl/5tM7neAkUSZtLOWCFQuFSlAatN3tRDlihULgUWqa+LC4l/hLu+++ncj46koMHNmdvGzduJPsjNrIvfANr18ynSjs2/dEAABGFSURBVJUgh9vVUn0ZLJItEft+Y8XyObYPLgZatcvZSrs58fDwYM+uNRzYv4k/D/3OuPffdJotgGHDBnEocguRBzczb+5XeHg4LxtYcRWYE5Nu8crocXTrP5QufV5i+driZ2RLuJnIoOHv0rHnQAYNfzdb+qict4GQEE9CQjwJruKJu3v+7uPk33uJPLiZ/REb2btnXbHrU1SUKnIRmDdvCU882S/XtunTv6NO3XbUq9+edes2M+bd4Q63q5X6chbDXhvE8RMnnW5Hq3Y5W2k3J2lpabRp14PaddpSu047Hm/Xgvr1bE6z/08EB1fmlVdeoEHDTtR6rA16vZ4ePZ50ii1HKDAvWLqae++6g2VzvmHWV58w5csfyMjIsMv+voOHGTNxWp7tP85bTIM6j7Ju0Uwa1HkUfz+LAnNGppm4uBRiY1O4fiOdwAoFfzG1afsMdeq2o0HDjnbVxZGUJU04W6rI7kKI54QQbayfewshvhJCvGIdH1dsdu4Mz6N+m1Ovysvby6Ye1n9BK/VlgJCQKnTo0JqfflrgdFtatUsLpd2c5FbddXPqz0yD3oCnpxG9Xo+nl6fTpJYcocAshOBWcgpSSpJTUvHz9UGvt+Ra/mn+r/QcOIynnhvCVz/Os7teW3fsoUuHNgB06dAGLy+LzbQ0c7YMUlqaCYOhRFPpFoijpiJrga0e8CygEzBcCDEPy3CLcKAu8KMzKzZ+/GhOnQqn17NPMX7CVGeacjrTpo3nnXcmYtZwtp6WOEtpNyc6nY79ERuJiz3Mli1/sC/CObYuXLjIp599z+lT4USfO8jNhEQ226lcXFQcocDc++knOHP2PC279OGp54bw9oiX0el07Ao/QHRMLAt//Jyls7/mWNQp9h86Yle9rl6/QcXAAAAqBgagz0f12aecG8kp+c9mlFKyft0CwveuZ9BA7X5lZlGWQhC2XsLVlFI+LIQwALFAsJTSJIT4GfizoJNyTu/TG/zR68sVuWLjxk1m3LjJjBr1CkOGDODDD6fbPqkU0rFjG+IvX+Fg5BGaNWtY0tVxOM5U2s2J2WymTt12+Pn5snTJTB58MIyjR6Mcbsff348nOrejelhDbty4ycIF39G7Vzd+WbDM9slFxBEKzLv2HaBGtXv46ctJnI+N48UR71L7kQfZHXGQ3fsO0n3AqwAkp6Rw7vwF6jxak14vjiA9PYPklBQSbibydH9L7PmNoS/QuH5tm/U2GvX4+LhxIS453/3NW3QlLu4SFStWYMP6hZyIOsXOneF2XpXiUxocq73YcsA6IYQ74A14AX7ANcADKDAEkXN6n4exarGuxqJFK1ixfE6ZdcCNGtWhc+d2tG/fCqPRA19fH+bM/oL+A4aVdNWKjVZKuzlJSLjJ9j92W15eOcEBt27VhLNnz3PlyjUAVqxYT4OGtZ3igB2hwLx87SYG9e2BEII7QoMJqVKZf87FgIRB/XrSo2veGOyCHz4DLDHgles28dF7uV9qVijvT/yVa1QMDCD+yjVMpn//hN3ddFQM9ODixRQK+kGX1Yb4+KusWLmeunUf1dQBu9IoiJnACeAQMAZYIoT4AYgAHK+UaeW+e+/KXu/cqS1RUaecZcrpvPfeJO6+pw7VqjegT9+hbN26yyWcLzhXaTcngYEB+Pn5AmA0GmndqilRUaedYiv6/AXq16+Fp1VMs2XLJpw44ZznzxEKzFWCKrL3gGWS1pVr1zkbHUNocGUa1XuM5Ws3kpxs0c69FH8lW43ZFi2aNGDlesuopJXrN5OcbLGp1wuCgoxcjk8lIzN/J+fl5fl/7Z17lFV1Fcc/e2ZEhodAA+HEgCNqJ1BTgaUIiDQ8EhAKdZkvDEqlEEJJUUhCkzIxX2mZClasBDXENwq+0MwGR2FaMMEtUMEZGIesILQYZtj9cc7QIAP3wJzfuQ/3Z62z7rn3zv19f/ueufv+7j7n9/vSpk3rPftDh5zp5IvyQGRNCUJV7xSRR4P9zSIyHxgCPKiqb0XRgfnz72XgGX3p2PFzbFj/FjfPvp2zvlrCF794DLt372bTpkomTZ4RhdRexOW+HDdxxeXaabcxhYWdeWjeXeTm5pCTk8OiRc/w3JKXkr/wECgrW8XixUt4a8UL1NXVUV5ewdy5DzvRao4D86NPPMc3xozkO+Mu4gc/vp0xY7+LqnL1xG/RoX07+p/Wm3c3fsDFE6YC0Cq/Jbf88FoKGqyTD8BlY8/n+zN/wuJnl1LYuRP/2uY7MHfo0IKcHNnr6oeqzXubo3fu3IlFv58HQG5eLo888iTLli1vztt00KTD1Q1hce6K3NwSRFhsOUqjKeJcjnJ3jD99s3U5yl21Vc0+YL0KB4Q+ECu3vGGuyIZhGFGRSTVgS8CGYWQV6VDbDYslYMMwsopMqgFbAjYMI6uIsxbfXCwBG4aRVdgI2DAMI0XUa+ZM+XeegOO8PCwuMuf7NX3JRrfsw/MiWZ8qFPkxXhr2yYb4l5RsDlaCMAzDSBFWgjAMw0gRNgI2DMNIETYCNgzDSBH12vQ6xemIJWDDMLKKTJqKnHJPuE+TzKTQtA5MnGajLmNqyqy1gauvmsDO/35AQUGHSDUbcBlXly6FLHl+Ie+sfImyt5cxceL4vZ6fMuVyPv7k/chjCxNTMgNQgLLyCs6bcC1f//ZUxk2d1ex+1dbu4pqb72TEpZO5aNIMPM8rBvA8b6jnee94nrc6uC0J22YmLUeZVgk4mUmhaSUnLlNO1zE1ZdYKUFRUyODBZ7BxU2VkWo1xHVd9fR0zps+md68hfGXQGK6YMJYvfelYwE/OJSVnsCni2MLEFMYAdPuOj5n987nc86PreHLeHdw+c2roPlRV1zB+6o37PL74+Vc4om1rlsy/h7HnjgS4NXjq78CoRCJxIvBNILSpnaqG3lJNWiXgZCaFppWcuEw5XcfUlFkrwG1zZjF9xo+dfXhcx1VdvZXy8grANzNNJDbs8YG7dc5MbrjhlshNaMPEFMYAdMnLbzB4wGkUdu4IQEGHdnuee+al17nwyumcN+FabrrzAerrw11//eqbbzN62CAAhg7sCzDY8zxJJBKrEolEg2FeBdDS87z92zA3Yrdq6C3VJE3AInKMiFwjIneLyO0i8h0RaZfsdYdCMpNC00ofUhHT2SOHsnlzNatXr3WmEWdc3boVcdJJPSkrK2fEyCFs2fyhk9jCxBTGAHRj1Ra279jB+Kk3cv53r+PpZa8B8O7GSpYuf5P5d9/MovtvIzcnh+deDrdecc1H/+DITgUA5PluztuAgk/92bnAqkQisTNMm5lkS3/Ak3Ai8j1gFPAavhNyOdAV+JOITFTV5VF2JplJoWmlD3HHlJ/fkuuum+y8vBJXXK1bt2LBwvuYNu1H1NXVMW3aJEaP2rfkEgVhYgpjAFpXX8/av77Hg7fNZGdtLZd87wa+3PM4Slet4S9/e48Lr5wOwM6dtXyuvW8hNWXWbVRV17BrVx1bav7OeROuBeDiMSMYc9ZX9vfe7nnQ87zj8csSw8LGm01TkS8HTg6ckO8AlqjqIBG5H3gKOKWpFzV2RZbcduTktA7VmTAmhVGRrVpxEXdM3bsXU1zclbKypXv0SkufZ8CAUXz44dbIdOKIKy8vjwULfsWjjzzJ008t5fjjPYqPKqJ0hW9s2qXLkfzxzWc5c+DXI4ktbEzJDEA7dyygwxFtaZXfklb5Lel9Yg8SGzaiqoweeiZXXXbRPm3efZOfcKuqa7hhzi/59R037tNm9daPOLJTAXX19fB/4188zysCngAuTRyECWAmDW7C1IAbkvThQFsAVd1EEldkVe2jqn3CJl8IZ1IYFdmqFRdxx1RRsY6u3U7B8/rhef2orNpC377DI02+EE9c9913K4nEeu65x/dOq6hIUFzch549BtCzxwCqqqrp3+/syGILG1MyA9CSfn1YuWYddfX1/Oe/O1m9bj3du3Whb68TefEPpXz0z20AbNu+g80h+z6oX2+eDjzjXny9FOCVRCKhnue1B54DpicSiT8eTLyZVANONgKeC5SJSCkwkOAMpYh0IviWipIwJoWmdWDiMuV0HVNTZq2/+c2jkbW/P1zHdfrpfbjo4nNZs3otfyr1F7m5cdYcli5dHpnGpwkb0/4MQB97ZhnnjxpG96OK6N/nZM69/BpycnI4Z3gJxx3dDYDJ4y5gwvWz2b1bycvL5QeTv80XOndK2rdzhpcw/af3MuLSybRr2wbg+uCpScCxwEzP82YGjw1LJBI1ydrMpBFwUlNOETke6AGsUdV1ByuQ16JL5rwbRmzYamjNY2fdrti04lwNrUXXk5ptktmuzTGhc862HRvS25RTVSvwLwMxDMNIezJpBGxTkQ3DyCqy6SoIwzCMjCIdTq6FJa1mwhmGYTSXKKcii8hZIpIQkfUicn3SFxwkloANw8gqopoJJyK5wC+A4UBP4EIR6RllXy0BG4aRVUQ4Aj4VWK+q76pqLfAI8LUo+2o1YMMwsooIa8BdgA8a3a8ETouqcYghAdfVVh3SdXYicoWqPhB1f1KlY1qZpZWNMWWzVmMOJuc0XjYh4IFGfW6qnUjP8KVzCeKK5H+SUTqmlVla2RhTNmsdEo2XTQi2xl8YlfiLjzVQBGwmQtI5ARuGYaSSMuA4ETlaRFoAFwBPRylgNWDDMIwmUNU6EZkELAVygYeCmcGRkc4JOK7aUZw1KtPKHK1sjCmbtZygqksAZ4thJF2MxzAMw3CD1YANwzBSRNolYNdT/xrpPCQiNSKyxpVGI62uIvKqiKwVkQoRmeJQq6WIvCUifw60bnKlFejlisgqEXnWsc77IrJaRMpF5G3HWu1FZJGIrAuO2emOdLwgnoZtu4hc5Ujr6uD/YY2ILBSRli50Aq0pgU6Fq3iyhoOZNeJ6wy90bwC6Ay2APwM9HWkNBHrhr3PsOq5CoFew3xb4q8O4BGgT7B8GrAD6OoxtKrAAeNbxe/g+0NH1sQq0fgtcFuy3ANrHoJkLVANHOWi7C/AekB/cfwwY5yiOE4A1QCv8c0wvAcfFcdwycUu3EbDzqX8NqOrrOHD12I/WFlVdGez/G1iL/6FwoaWquiO4e1iwOSn0i0gRMBLfOSUrEJEj8L+c5wGoaq2q/isG6cHABlXd6Kj9PCBfRPLwk2Ok17M2ogdQqqqfqGodvqHvGEdaGU+6JeCmpv45SVSpQkSK8c1MVzjUyBWRcqAGeFFVXWndBUwD4liAVYFlIvJOMHvJFd2BrcCvg9LKXBEJb2x46FwALHTRsKpWAT8DNgFbgG2q6srAbw0wUEQKRKQVMIK9JzMYjUi3BOx86l8qEZE2wOPAVaq63ZWOqtar6sn4M3dOFZETotYQkbOBGlV9J+q290N/Ve2FvzLVlSIy0JFOHn5p6j5VPQX4mP/7lDkhuMh/NPB7R+13wP8leTTwBaC1iFziQktV1+J7R74IvIBfRqw74Is+w6RbAnY+9S9ViMhh+Mn3YVVdHIdm8NN5OXCWg+b7A6NF5H38UlGJiPzOgQ4Aqro5uK3Btyo/1ZFUJVDZ6FfDIvyE7JLhwEpV3dcrPhqGAO+p6lZV3QUsBvo50kJV56lqL1UdiF/m+5srrUwn3RKw86l/qUBEBL+muFZV73Cs1UlE2gf7+fgfvoM2U02Gqk5X1SJVLcY/Tq+oqpNRlYi0FpG2DfvAMPyfupGjqtXAByLiBQ8NBv7iQqsRF+Ko/BCwCegrIq2C/8XB+OchnCAinw9uuwHn4Da2jCatZsJpDFP/GhCRhcAgoKOIVAKzVHWeCy380eJYYHVQmwWYof4sm6gpBH4bLCadAzymqk4vEYuBzsATfu4gD1igqi841JsMPBwMAt4FxrsSCuqkQ4EJrjRUdYWILAJW4pcDVuF2ltrjIlIA7AKuVNV/OtTKaGwmnGEYRopItxKEYRjGZwZLwIZhGCnCErBhGEaKsARsGIaRIiwBG4ZhpAhLwIZhGCnCErBhGEaKsARsGIaRIv4HX13bCkwaMtUAAAAASUVORK5CYII=\n",
524 |       "text/plain": [
525 |        "<Figure size 432x288 with 2 Axes>"
526 |       ]
527 |      },
528 |      "metadata": {
529 |       "needs_background": "light"
530 |      },
531 |      "output_type": "display_data"
532 |     }
533 |    ],
534 |    "source": [
535 |     "seaborn.heatmap(CM, annot=True)\n",
536 |     "plt.show()"
537 |    ]
538 |   },
539 |   {
540 |    "cell_type": "markdown",
541 |    "metadata": {},
542 |    "source": [
543 |     "My Convolutional Neural Network classified 91.7% of Test data correctly. The confusion matrix showed that category no. 6 (Shirt) has been misclassified the most. The next goal is to correct it; one possible solution would be to increase regularization, another to build an ensemble of models."
544 |    ]
545 |   },
546 |   {
547 |    "cell_type": "code",
548 |    "execution_count": null,
549 |    "metadata": {},
550 |    "outputs": [],
551 |    "source": []
552 |   },
553 |   {
554 |    "cell_type": "markdown",
555 |    "metadata": {},
556 |    "source": [
557 |     "Thanks for coming thus far. In the next Notebooks I will implement more advanced Convolutional models, among other things."
558 |    ]
559 |   },
560 |   {
561 |    "cell_type": "code",
562 |    "execution_count": null,
563 |    "metadata": {},
564 |    "outputs": [],
565 |    "source": []
566 |   },
567 |   {
568 |    "cell_type": "code",
569 |    "execution_count": null,
570 |    "metadata": {},
571 |    "outputs": [],
572 |    "source": []
573 |   }
574 |  ],
575 |  "metadata": {
576 |   "kernelspec": {
577 |    "display_name": "Python 3",
578 |    "language": "python",
579 |    "name": "python3"
580 |   },
581 |   "language_info": {
582 |    "codemirror_mode": {
583 |     "name": "ipython",
584 |     "version": 3
585 |    },
586 |    "file_extension": ".py",
587 |    "mimetype": "text/x-python",
588 |    "name": "python",
589 |    "nbconvert_exporter": "python",
590 |    "pygments_lexer": "ipython3",
591 |    "version": "3.6.8"
592 |   }
593 |  },
594 |  "nbformat": 4,
595 |  "nbformat_minor": 2
596 | }
597 | 


--------------------------------------------------------------------------------