├── Conv_Basic_with_TensorFlow ├── MNIST_2DConv_Visual.ipynb ├── MNIST_fashion_1hiddenLayer.ipynb ├── Understanding_Conv_Pool.ipynb ├── Understanding_conv1.png ├── Understanding_conv10.png ├── Understanding_conv2.png ├── Understanding_conv5.png ├── convdemo.png └── house_price_easy.ipynb ├── Diffusion ├── L0_diffusion_Area.py └── StableDiffusion_Math.ipynb ├── Facial_Keypoint_Kaggle.ipynb ├── Multi_Class ├── Multi_class_classify.ipynb ├── Prediction_RSP.png ├── rps_visual_conv2d_1.png ├── rps_visual_conv2d_2.png ├── rps_visual_conv2d_3.png └── rps_visual_max_pooling2d_3.png ├── NLP_Embedding1 ├── Embedding_Course3_W2.ipynb ├── Week2_Course3_1.png ├── Week2_Course3_2.png ├── Week2_Course3_4.png ├── Week3_Course3_3.png ├── meta_imdb.tsv └── vecs_imdb.tsv ├── NLP_LSTM_Glove ├── C3_W3_Quiz │ ├── week3_Course3_1.png │ ├── week3_Course3_2.png │ └── week3_Course3_3.png ├── NLP_Week3LSTM_Conv.ipynb ├── Wd_cld_neg_sent.png ├── Wd_cld_pos_sent.png └── Week3_Final_Exercise_Glove.ipynb ├── NLP_Poetry ├── Quiz_Week4_C3 │ ├── week4_course3_345.png │ ├── week4_course3_67.png │ ├── week4_course3_78.png │ └── week4_course3_first2.png ├── Shakespeare_poetry_NLP_Week4.ipynb └── sonnets.txt ├── README.md ├── TF-Proba ├── Bayesian_uncertainty.ipynb ├── Norm-Flows │ ├── NormFlow_TransformDistribution.ipynb │ ├── NormFlows_AutoReg.ipynb │ ├── Normalizing Flows_GettingStarted.ipynb │ ├── norm_flow1.png │ └── norm_flow2.png └── PCA_ProbabilisticApproach.ipynb ├── TF_Data_Intro.ipynb ├── Time_SeriesW1 ├── General_W1_Lesson.ipynb ├── Q1W1.png ├── Q2W1.png ├── Q3W1.png ├── Q4W1.png ├── Q5W1.png └── Time_SeriesW1_Answer.ipynb ├── Time_SeriesW2 ├── Deep_NN_TimeSeries.ipynb ├── Features_Labels_TimeSeries.ipynb ├── Q1W2.png ├── Q2W2.png ├── Q3W2.png ├── Q4W2.png ├── Q5W2.png ├── Q6W2.png └── Single_Layer_Network_TimeSeries.ipynb ├── Time_SeriesW3 ├── LSTM_TimeSeriesW3.ipynb ├── Quiz │ ├── Q1Week3.png │ ├── Q2Week3.png │ ├── Q3Week3.png │ └── Q4Week3.png ├── RNN_TimeSeriesW3.ipynb └── Week_3_Final_Q.ipynb ├── Time_SeriesW4 ├── LSTM_Conv1D_Week4.ipynb ├── Min_Temp_Melbourne.png ├── Sunspot_W4.ipynb ├── Sunspots.csv ├── Week4_Quiz_RNN.ipynb ├── min_temp_Melbourne.csv └── quiz │ ├── W4Q1.png │ ├── W4Q2.png │ ├── W4Q3.png │ ├── W4Q4.png │ ├── W4Q5.png │ ├── W4Q6.png │ ├── W4Q7.png │ └── W4Q8.png ├── Transfer_Learning ├── Prediction_with_Vgg_CatsDogs.png ├── Transfer_Learning_Vgg16.ipynb ├── building_VGG16.ipynb └── building_VGG16_visualization.ipynb ├── ViT_TensorFlow ├── Self-Attention1.png ├── Self-Attention2.png ├── Self-Attention3.png ├── Self-Attention4.png ├── Self_MultiAttention.jpg ├── Understand&Implement_VIT_TensorFlow.ipynb ├── transformer_VIT_org.jpg ├── transformer_ViT.png └── transformer_arch.png ├── cassava-classification.ipynb ├── detectpneumonia-inceptionresnetv2-class-imbalance.ipynb ├── im_generator ├── Using_ImageGenerator.ipynb ├── detect_pneumonia.png └── lecture_slides_Optimizers.pdf ├── resnet ├── Implement_Resnet_TensorFlow.ipynb ├── heatmap.png ├── residual_block.png └── train_acc.png └── ri_challenge ├── AEC_Class_train.jpg ├── AEC_Class_train_acc.jpg ├── AEC_train.jpg ├── Ridge-i_Challenge.ipynb ├── Simple_conv_train.jpg ├── heatmap_aecconv.jpg ├── heatmap_aecdense.jpg ├── heatmap_aecsimpconv.jpg └── ridge-inet.png /Conv_Basic_with_TensorFlow/MNIST_2DConv_Visual.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Visualizing How Convolution Detect Features " 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "In this notebook we look back again on the [Fashion MNIST](https://github.com/zalandoresearch/fashion-mnist) data but, instead of the simplistic one hidden layer neural network, we implement a two convolutional and two max pooling layers. We expect better performance compared to the previous case, but also we will see how the features of various dresses are found by the convolutional layers. " 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 3, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [ 23 | "import tensorflow as tf\n", 24 | "import numpy as np\n", 25 | "from tensorflow import keras\n", 26 | "import matplotlib.pyplot as plt" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": {}, 32 | "source": [ 33 | "#### Load Data from Keras Dataset" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": 4, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "mnist_fashion = keras.datasets.fashion_mnist\n", 43 | "(X_train, y_train), (X_test, y_test) = mnist_fashion.load_data()" 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "**Label and Class for the Fashion Dataset**\n", 51 | "\n", 52 | "|Label | Class|\n", 53 | "|---|---|\n", 54 | "|0| Top|\n", 55 | "|1| Trouser|\n", 56 | "|2| Pullover|\n", 57 | "|3| Dress|\n", 58 | "|4| Coat|\n", 59 | "|5| Sandal|\n", 60 | "|6| Shirt|\n", 61 | "|7| Sneaker|\n", 62 | "|8| Bag|\n", 63 | "|9| Ankle Boot|" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "#### Normalize the Pixels\n", 71 | "Since the pixel values are number between 0 and 255, it is better if we normalize the values between 0 and 1 and it is considered as one of the most important image _pre-processing_ steps. " 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": 5, 77 | "metadata": {}, 78 | "outputs": [], 79 | "source": [ 80 | "X_train = X_train/255.0\n", 81 | "X_test = X_test/255." 82 | ] 83 | }, 84 | { 85 | "cell_type": "markdown", 86 | "metadata": {}, 87 | "source": [ 88 | "#### Difference with Previous Example (1 Hidden dense Layer) " 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": {}, 94 | "source": [ 95 | "* First, it's better to understand Convolution and What it Actually Does. Get started with Andrew Ng's Deep Learning Specialization and Check the amazing [CS231n Course Page](http://cs231n.github.io/convolutional-networks/)\n", 96 | "\n", 97 | "* Since we are going to use [Conv2d](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) and [MaxPool2D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D) layers, check the descriptions to know about the input arguments, output etc. \n", 98 | "\n", 99 | "* Here we start with simplest form of convolution, $3\\times 3$ without any stride and no padding. \n", 100 | "\n", 101 | "* Images are graysclaed, so only 1 channel. \n", 102 | "\n", 103 | "* Use relu activation function. \n", 104 | "\n", 105 | "Let's begin!" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "#### Reshape the Data" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "Treat the input data as tensors, with a format (batch, height, width, channels) " 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 6, 125 | "metadata": {}, 126 | "outputs": [ 127 | { 128 | "name": "stdout", 129 | "output_type": "stream", 130 | "text": [ 131 | "(10000, 28, 28)\n" 132 | ] 133 | } 134 | ], 135 | "source": [ 136 | "X_train_reshape = X_train.reshape(60000, 28, 28, 1)\n", 137 | "X_test_reshape = X_test.reshape(10000, 28, 28, 1)\n", 138 | "print (X_test.shape)" 139 | ] 140 | }, 141 | { 142 | "cell_type": "markdown", 143 | "metadata": {}, 144 | "source": [ 145 | "#### Create Layers Conv and Maxpool (Twice), Then Dense Layer " 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": 8, 151 | "metadata": {}, 152 | "outputs": [], 153 | "source": [ 154 | "model = keras.models.Sequential([\n", 155 | " keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)), \n", 156 | " keras.layers.MaxPooling2D(2, 2), \n", 157 | " keras.layers.Conv2D(32, (3, 3), activation='relu'), \n", 158 | " keras.layers.MaxPooling2D(2, 2), \n", 159 | " keras.layers.Flatten(), \n", 160 | " keras.layers.Dense(128, activation='relu'), \n", 161 | " keras.layers.Dense(10, activation='softmax')])" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": 9, 167 | "metadata": {}, 168 | "outputs": [ 169 | { 170 | "name": "stdout", 171 | "output_type": "stream", 172 | "text": [ 173 | "Model: \"sequential_1\"\n", 174 | "_________________________________________________________________\n", 175 | "Layer (type) Output Shape Param # \n", 176 | "=================================================================\n", 177 | "conv2d_2 (Conv2D) (None, 26, 26, 32) 320 \n", 178 | "_________________________________________________________________\n", 179 | "max_pooling2d_2 (MaxPooling2 (None, 13, 13, 32) 0 \n", 180 | "_________________________________________________________________\n", 181 | "conv2d_3 (Conv2D) (None, 11, 11, 32) 9248 \n", 182 | "_________________________________________________________________\n", 183 | "max_pooling2d_3 (MaxPooling2 (None, 5, 5, 32) 0 \n", 184 | "_________________________________________________________________\n", 185 | "flatten_1 (Flatten) (None, 800) 0 \n", 186 | "_________________________________________________________________\n", 187 | "dense_2 (Dense) (None, 128) 102528 \n", 188 | "_________________________________________________________________\n", 189 | "dense_3 (Dense) (None, 10) 1290 \n", 190 | "=================================================================\n", 191 | "Total params: 113,386\n", 192 | "Trainable params: 113,386\n", 193 | "Non-trainable params: 0\n", 194 | "_________________________________________________________________\n" 195 | ] 196 | } 197 | ], 198 | "source": [ 199 | "model.summary()" 200 | ] 201 | }, 202 | { 203 | "cell_type": "markdown", 204 | "metadata": {}, 205 | "source": [ 206 | "#### Understanding Model Summary" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "* 1st layer is 2D Conv Layer, takes input image of size (28, 28, 1). Filter dimension is (3, 3). Thus output shape $(w-f+1, h-f+1, n_c)$ where $n_c$ is the number of convolution filters. \n", 214 | "\n", 215 | "* Number of Parameters: Input image size (28, 28) with only one channel. Filter size (3, 3). Number of filters 32, thus total number of weights = $f\\times f \\times n_c \\times 1 = 288$, number of bias paramters = number of filters (32). Thus total number of parameters $ 288 + 32 = 320 $. Let's verify this for the next conv layer. \n", 216 | "\n", 217 | "* In the next conv layer, number of channels = 32 (should be equal to the number of filters used in the previous conv layer which is $n_c$).
So total number of weights $f\\times f\\times n_c^{[L]} \\times n_c^{[L-1]} = 3\\times 3\\times 32 \\times 32 = 9216$. Number of bias paramters = 32 (number of filters), thus total number of paramters = $9216 + 32 = 9248$. Notations are influenced by Andrew Ng's Deep Learning Course. \n", 218 | "\n", 219 | "* Dense layer is also known as fully connected layer and thus depends on the number of neurons in the dense layer. Number of weights = $128 \\times 800 = 102400$, 800 is obtained after flattening the image of shape (5, 5, 32). Number of bias = 128, thus total number of parameter = 102528. \n", 220 | "\n", 221 | "* In the final layer, the input of the dense layer is coming from last 128 layers and the output is connected to 10 neurons, thus total number of paramters $ 128 \\times 10 + 10 = 1290$. " 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": {}, 227 | "source": [ 228 | "### Compile Model and Fit Training Data" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": 10, 234 | "metadata": {}, 235 | "outputs": [ 236 | { 237 | "name": "stdout", 238 | "output_type": "stream", 239 | "text": [ 240 | "Epoch 1/4\n", 241 | "60000/60000 [==============================] - 19s 322us/sample - loss: 0.4708 - acc: 0.8303\n", 242 | "Epoch 2/4\n", 243 | "60000/60000 [==============================] - 19s 312us/sample - loss: 0.3189 - acc: 0.8832\n", 244 | "Epoch 3/4\n", 245 | "60000/60000 [==============================] - 19s 310us/sample - loss: 0.2755 - acc: 0.8988\n", 246 | "Epoch 4/4\n", 247 | "60000/60000 [==============================] - 19s 311us/sample - loss: 0.2456 - acc: 0.9098\n" 248 | ] 249 | }, 250 | { 251 | "data": { 252 | "text/plain": [ 253 | "" 254 | ] 255 | }, 256 | "execution_count": 10, 257 | "metadata": {}, 258 | "output_type": "execute_result" 259 | } 260 | ], 261 | "source": [ 262 | "model.compile(optimizer='Adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n", 263 | "\n", 264 | "model.fit(X_train_reshape, y_train, epochs=4)" 265 | ] 266 | }, 267 | { 268 | "cell_type": "markdown", 269 | "metadata": {}, 270 | "source": [ 271 | "#### Evaluate the Performance of the Model with Test Data" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": 11, 277 | "metadata": {}, 278 | "outputs": [ 279 | { 280 | "name": "stdout", 281 | "output_type": "stream", 282 | "text": [ 283 | "10000/10000 [==============================] - 1s 107us/sample - loss: 0.2939 - acc: 0.8922\n", 284 | "0.8922\n" 285 | ] 286 | } 287 | ], 288 | "source": [ 289 | "test_loss, test_acc = model.evaluate(X_test_reshape, y_test)\n", 290 | "print (test_acc)" 291 | ] 292 | }, 293 | { 294 | "cell_type": "code", 295 | "execution_count": 12, 296 | "metadata": {}, 297 | "outputs": [ 298 | { 299 | "name": "stdout", 300 | "output_type": "stream", 301 | "text": [ 302 | "[2 0 5 7 1 8 2 2 2 2 4 2 6 1 9 8 5 1 9 1 2 8 3 8 9 2 4 2 8 8 2 0 5 8 8 5 3\n", 303 | " 9 4 3 4 4 5 1 0 1 2 0 6 9]\n" 304 | ] 305 | } 306 | ], 307 | "source": [ 308 | "print (y_test[500:550])" 309 | ] 310 | }, 311 | { 312 | "cell_type": "code", 313 | "execution_count": 15, 314 | "metadata": {}, 315 | "outputs": [ 316 | { 317 | "data": { 318 | "text/plain": [ 319 | "" 320 | ] 321 | }, 322 | "execution_count": 15, 323 | "metadata": {}, 324 | "output_type": "execute_result" 325 | }, 326 | { 327 | "data": { 328 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASwElEQVR4nO3dfXBc1XkG8OfV6gt/SzYWxja2MQZqKLVBMR+hQAJNgU4CdFqIQwMlzCiZgU6Y0lIGQkyaNONJobSdtrRKw8S0CZRpIHgCUwIOLVBSY9k1xsaAsZFtCdkyCCPJtrTa3bd/6EIF6LxX7N7du+F9fjMarfbV2T2+9uO7u+eec0RVQUSffDVpd4CIKoNhJ3KCYSdygmEncoJhJ3KitpJPVi8N2ojJlXxKIleGcAhZHZbxaiWFXUQuBvA3ADIA/llVV1u/34jJOFMuLOUpiciwXtcFa0W/jBeRDIC/B3AJgKUAVorI0mIfj4jKq5T37CsAvK6qu1Q1C+BBAJcl0y0iSlopYZ8LYO+Yn7ui+z5ARNpEpENEOkYwXMLTEVEpyv5pvKq2q2qrqrbWoaHcT0dEAaWEvRvA/DE/z4vuI6IqVErYNwBYIiKLRKQewBcBrE2mW0SUtKKH3lQ1JyI3AngCo0Nv96nqtsR6RkSJKmmcXVUfB/B4Qn0hojLi5bJETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROMOxETjDsRE6UtIsrVT+ptf+KNZcr6fFr580167mu7pIePy0HrznbrNd+ab9ZXz7T/nNvvfO0YK3hsQ1m22KVFHYR6QQwACAPIKeqrUl0ioiSl8SZ/TOq+lYCj0NEZcT37EROlBp2BfBzEdkoIm3j/YKItIlIh4h0jGC4xKcjomKV+jL+XFXtFpHZAJ4UkVdU9Zmxv6Cq7QDaAWCaNGuJz0dERSrpzK6q3dH3XgCPAFiRRKeIKHlFh11EJovI1PduA/gcgK1JdYyIklXKy/gWAI+IyHuP82NV/Y9EekUfS01jY7BWGBoy22ZmNtsP/u/hxwaAi2fb/78f3xAej77xqWvMtr92xxtm/ZU7jzfrK5btCNbumPuY2bZRnjPrG4bmm/WLJnWZ9dtXhc+znXbXilZ02FV1F4DfSLAvRFRGHHojcoJhJ3KCYSdygmEncoJhJ3JCVCt3Uds0adYz5cKKPd8nhdTVm3UdyRb92H0/O9Gs/9EJvzDrmw4tNOtNtYeDtelGDQB6s9PM+oXTtpn1ozOHgrWBgn1Mp9bYx3R+pmDWX8xOMeszao4Ea7d/9kqzbe6N3cHael2Hfu2T8Wo8sxM5wbATOcGwEznBsBM5wbATOcGwEznBsBM5waWkfwXUHGVPM80b4+w77z7LbPvsaXeZ9X8bONWsL248YNY7h2YGa9sG5phtT5tmL8f87Z2fN+ufaXktWDuh0V4K+jtbLjXrU39mj6N/4eanzXpb08Zg7dXv2NOOF18dHme38MxO5ATDTuQEw07kBMNO5ATDTuQEw07kBMNO5ATH2auBjDv9+H2FI/Zy0JaT/tpe0vjN37XndZ8zKbwcMwAcLjSY9aNr+4O12hp7TviWfns76Evm2PPZX+yfF6yNaMZsO3TQvrZh8XWdZr2pNjyXHgA6c+HjvuX8fzLbXlHkXiw8sxM5wbATOcGwEznBsBM5wbATOcGwEznBsBM5wXH2CbLWbpdMaf9nxq3dr8PDRT92bq89zn7Lzt8z6+uWrjXrm4fD658DwNT68DUCNWKPs5/YuM+s/+/gArOeLYTH0lvqwuP/APDIRX9n1g8WjjLrzx2y1+O3xvnPW3WD2XYmfmnWQ2L/lYrIfSLSKyJbx9zXLCJPisiO6HtTUc9ORBUzkVPSDwFc/KH7bgWwTlWXAFgX/UxEVSw27Kr6DIC+D919GYA10e01AC5PuF9ElLBi37O3qGpPdHsfgJbQL4pIG4A2AGjEpCKfjohKVfKn8Tr66VLwEyZVbVfVVlVtrYM9aYKIyqfYsO8XkTkAEH3vTa5LRFQOxYZ9LYBro9vXAng0me4QUbnEvmcXkQcAXABgloh0AVgFYDWAh0TkegC7AdgbSk9UjT3HGIV82R67pr7Ofuqh8HixjhTVo8TUNIbnXlv9BoDai/aY9bf22vOyT6yzj9v+fHhN+4OZQbPthsHjzfoXmjaZ9b58eG333dlZZtv2t84368c2HDTrcfPZD+TDe8/PftZ+oVxsCmLDrqorA6ULi3xOIkoBL5clcoJhJ3KCYSdygmEncoJhJ3JC4qZXJmmaNOuZUsKH+DFLLptK/HNmZoa30d1+1yKzrRy2Bz1O/vM3zHp+f3rXLNUuPM6s/+uzD5r1N/Phv7N7D1xgth0u2Metud4e3jqu4e1gbdeRo822M+vsx45binpBw1tmfUbmcLCWV/sc3H5ieEhyva5Dv/aNe9B5ZidygmEncoJhJ3KCYSdygmEncoJhJ3KCYSdyovJLSVtj5TFj4ZIJj21qLldsjwAAB9aeZNa/vTQ8ZX9v9lWz7bF175j17efZWxP/tOs0sz79m+HlvnTDS2bbODuvC297DABNGXupsTv2nRmsjcSMJ8c5Y7J9fcLc2vBxP6Ox02zbOWJPgd01PNusH1P7rv34MVNsLbXzwv9eZF94yjHP7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROVHacXeyxcqkPb4sMAIXD4TnAcfpXnmXWN7X+o1nvzYfnN7+dCy9ZDABbjthzwocK9nLM31jymFn/nUfDy0X/+vovmW2vWmwvx/zErH8w6/f32+PFvcPhY3PatG67bXaqWV9YZ88Z35E9JlhrFHv97+1Dx5r1RQ0HzHr3iL2x8f6R6cHapEzMFt3WFuHGZSw8sxM5wbATOcGwEznBsBM5wbATOcGwEznBsBM5UdlxdrXnnZcyJ33PN88x69u/Zo8X39TTatZfebclWPuLRY+Yba01wgHgYN6eE762b7lZv7drRrD2BydsMNue1Nhj1m/sDs9HB4B9Q/ZY+OnT94bbZsPbFgNAU5193A4XGsy6NZa9bdAeR2/I2P8WG8Su7x4K7zMAAOdNfy1Ye2PYXtM+tzt8TNXYPzz2zC4i94lIr4hsHXPfnSLSLSKbo69L4x6HiNI1kZfxPwRw8Tj336Oqy6Kvx5PtFhElLTbsqvoMgL4K9IWIyqiUD+huFJEt0cv84IXAItImIh0i0jGCmGt+iahsig37vQAWA1gGoAfA3aFfVNV2VW1V1dY62B+oEFH5FBV2Vd2vqnlVLQD4PoAVyXaLiJJWVNhFZM6YH68AsDX0u0RUHWLH2UXkAQAXAJglIl0AVgG4QESWAVAAnQC+mkRnem62x8pHjCHdP7nqYbPtlmx4zjcAPLr+dLNu/bd4xsn2PPw87DXE48aLl0/ZU3R9asb+c78wGN7rG4hf231246BZt+ZmT4mZt31cfXh/dQD4r8GTzXpzbXgNglOmvGm2XVAft7+6vX/7VdPtv/O7ey8K1rZ/41SzbT06zHpIbNhVdeU4d/+gqGcjotTwclkiJxh2IicYdiInGHYiJxh2IidEY7ZJTlLj4rm64HvhUbp7lj9kts9qeBnqfbnwNE8AGMw3mvXfnrLNrJ9cFx4e25y1pzueWm+s7wtgRPNm/e637WFBa4ipJxue5jkR+ZjzQUHtP1tDTfjYvDZob3v8qRmddv0oe8vm9n3nB2tT6+whyetmPWfWH3zHnvr7/N9+yqw3rfmlWbdIbXgQ7X9yT6C/0DfuXwrP7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROVHQp6braPI5tCk/9G1J76+L7e84O1uZNOmi2nV0/YNb/8/CJZv1ZFIK1uKWiu3P2VM649gsa7OmWv3gnPNXzqIy9NfHVs+zx3gM5e7nnjYcWmvVJNdlg7ZjGfrPtOyOTzfoNP/6aWZ/33eeDNXvyLLAKZ8T8hn1tRROKH0fPTLOPeb7fOG7GZTM8sxM5wbATOcGwEznBsBM5wbATOcGwEznBsBM5UdH57NMbWvScY68O1k9+uNts/5WZ/x2sPXv4BLPtxoGFZn3PYHAHKwBAfzY8n/1Anz0uetQke5z9lNn7zHrcePQ5U3cEa13ZmWbbvpw9lj291r4GIG6dgD1HwlsXnz19p9l29Qvj7Sf6/5Zcu8msl1NNo/3nRia89gIAFA4bxzUukxJeQ2B94Sn0K+ezE7nGsBM5wbATOcGwEznBsBM5wbATOcGwEzlR0fnsmh1BbvfeYH1rzBTir1zzx8HaVbc8YbbNxWw9/N3j7S2fNxxZFKztbLHXP98xYNffGZpk1jvfDY9VA0BfNtw+V7DHe2fUHzHr+4djriHIhOerA0DXofB6/vUz7DnhS+6163GssfDCkL1ufJxS25ekyGtjYs/sIjJfRJ4WkZdFZJuIfD26v1lEnhSRHdF3+6oUIkrVRF7G5wDcrKpLAZwF4AYRWQrgVgDrVHUJgHXRz0RUpWLDrqo9qropuj0AYDuAuQAuA7Am+rU1AC4vVyeJqHQf6z27iCwEsBzAegAtqtoTlfYBaAm0aQPQBgCNsN+bElH5TPjTeBGZAuAnAG5S1Q/MzNDR2TTjfmqgqu2q2qqqrXUITyYhovKaUNhFpA6jQf+Rqr73sfV+EZkT1ecA6C1PF4koCbFTXEVEMPqevE9Vbxpz/18CeFtVV4vIrQCaVfUW67GmSbOeKRcm0O3k5T5rj/vVDYSHmLpvs7dc/unp7Wb9/oP29r/fOtreTnrVgVOCtT+dudFse82uz5v1xVPsZax/v+kFs/6t3wx/lJPrsqc008e3XtcFp7hO5D37pwF8GcBLIrI5uu82AKsBPCQi1wPYDeDKJDpLROURG3ZVfQ5AaLZ8dZ6miegjeLkskRMMO5ETDDuREww7kRMMO5ETFV1KOnacvcaejil14cEDHbaXa/5Vlllqbyetu/YEa/nTTzLbyvMvFtWnJGRmTDfr+XftJbSLner5SWaNs/PMTuQEw07kBMNO5ATDTuQEw07kBMNO5ATDTuRERZeSjlWw54XrsF3/pMq//FrRbdMcR4+TP/hu2l1whWd2IicYdiInGHYiJxh2IicYdiInGHYiJxh2IicYdiInGHYiJxh2IicYdiInGHYiJxh2IicYdiInGHYiJ2LDLiLzReRpEXlZRLaJyNej++8UkW4R2Rx9XVr+7hJRsSayeEUOwM2quklEpgLYKCJPRrV7VPWu8nWPiJIykf3ZewD0RLcHRGQ7gLnl7hgRJetjvWcXkYUAlgNYH911o4hsEZH7RKQp0KZNRDpEpGMEn9wtmoiq3YTDLiJTAPwEwE2q2g/gXgCLASzD6Jn/7vHaqWq7qraqamsdGhLoMhEVY0JhF5E6jAb9R6r6MACo6n5VzatqAcD3AawoXzeJqFQT+TReAPwAwHZV/asx988Z82tXANiafPeIKCkT+TT+0wC+DOAlEdkc3XcbgJUisgyAAugE8NWy9JCIEjGRT+OfAzDefs+PJ98dIioXXkFH5ATDTuQEw07kBMNO5ATDTuQEw07kBMNO5ATDTuQEw07kBMNO5ATDTuQEw07kBMNO5ATDTuSEqGrlnkzkAIDdY+6aBeCtinXg46nWvlVrvwD2rVhJ9m2Bqh49XqGiYf/Ik4t0qGprah0wVGvfqrVfAPtWrEr1jS/jiZxg2ImcSDvs7Sk/v6Va+1at/QLYt2JVpG+pvmcnospJ+8xORBXCsBM5kUrYReRiEXlVRF4XkVvT6EOIiHSKyEvRNtQdKfflPhHpFZGtY+5rFpEnRWRH9H3cPfZS6ltVbONtbDOe6rFLe/vzir9nF5EMgNcA/BaALgAbAKxU1Zcr2pEAEekE0KqqqV+AISLnARgEcL+qnhrd9z0Afaq6OvqPsklV/6xK+nYngMG0t/GOdiuaM3abcQCXA/hDpHjsjH5diQoctzTO7CsAvK6qu1Q1C+BBAJel0I+qp6rPAOj70N2XAVgT3V6D0X8sFRfoW1VQ1R5V3RTdHgDw3jbjqR47o18VkUbY5wLYO+bnLlTXfu8K4OcislFE2tLuzDhaVLUnur0PQEuanRlH7DbelfShbcar5tgVs/15qfgB3Uedq6qnA7gEwA3Ry9WqpKPvwapp7HRC23hXyjjbjL8vzWNX7PbnpUoj7N0A5o/5eV50X1VQ1e7oey+AR1B9W1Hvf28H3eh7b8r9eV81beM93jbjqIJjl+b252mEfQOAJSKySETqAXwRwNoU+vERIjI5+uAEIjIZwOdQfVtRrwVwbXT7WgCPptiXD6iWbbxD24wj5WOX+vbnqlrxLwCXYvQT+Z0Abk+jD4F+HQ/gxehrW9p9A/AARl/WjWD0s43rAcwEsA7ADgBPAWiuor79C4CXAGzBaLDmpNS3czH6En0LgM3R16VpHzujXxU5brxclsgJfkBH5ATDTuQEw07kBMNO5ATDTuQEw07kBMNO5MT/AXPpthPkmS7CAAAAAElFTkSuQmCC\n", 329 | "text/plain": [ 330 | "
" 331 | ] 332 | }, 333 | "metadata": { 334 | "needs_background": "light" 335 | }, 336 | "output_type": "display_data" 337 | } 338 | ], 339 | "source": [ 340 | "#Let's see some test images\n", 341 | " \n", 342 | "plt.imshow(X_test.reshape(10000, 28, 28)[502])" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": 16, 348 | "metadata": {}, 349 | "outputs": [ 350 | { 351 | "data": { 352 | "text/plain": [ 353 | "" 354 | ] 355 | }, 356 | "execution_count": 16, 357 | "metadata": {}, 358 | "output_type": "execute_result" 359 | }, 360 | { 361 | "data": { 362 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAQ40lEQVR4nO3df2xd5X3H8c/X19d2SGKISQhRfhDCojIYI1CTTIAqOlQG6aTQbmWlGgsbU9BWpFbqJhjTVKRJG5pKEdNWplAQaUWpGJRCVTSgGSqrqkUxLM0PfiXQBBJCHAghv7BjX3/3h08qAz7fY+7v9nm/JMvX53ufe55c+5Nz733Ocx5zdwH4zdfR6g4AaA7CDiSCsAOJIOxAIgg7kIjOZu6sy7q9R9ObuUsgKUM6quM+bJPVagq7mV0p6S5JJUnfdvfbo/v3aLpW2OW17BJAYIOvz61V/TLezEqS/l3SVZLOkXStmZ1T7eMBaKxa3rMvl7TD3V9z9+OSvi9pVX26BaDeagn7fElvTPh5d7btA8xsjZkNmNnAiIZr2B2AWjT803h3X+vu/e7eX1Z3o3cHIEctYd8jaeGEnxdk2wC0oVrCvlHSUjM708y6JH1R0uP16RaAeqt66M3dR83sJklPanzo7T5331a3ngGoq5rG2d39CUlP1KkvABqI02WBRBB2IBGEHUgEYQcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBE1LdlsZjslHZZUkTTq7v316BSA+qsp7JlPu/vbdXgcAA3Ey3ggEbWG3SU9ZWbPmdmaye5gZmvMbMDMBkY0XOPuAFSr1pfxl7r7HjM7TdLTZvaSuz878Q7uvlbSWknqtT6vcX8AqlTTkd3d92TfByU9Kml5PToFoP6qDruZTTezmSduS7pC0tZ6dQxAfdXyMn6upEfN7MTjfM/d/6suvULzjP/+queNe2dmF5wb1o+f2hPWyz95rvqdd5Ti+lil+sdukarD7u6vSTq/jn0B0EAMvQGJIOxAIgg7kAjCDiSCsAOJqMdEGPw6a+DQmSQNr7wot7ZrVdz2qgu3hPU/6tsY1m/67o25tUW3/TzeeYOH1kqzZuXWKu++GzeOhkuDXydHdiARhB1IBGEHEkHYgUQQdiARhB1IBGEHEsE4Oxpq/w3Hcmt3n/9w2PbZw58I68+/vzisv7jmW7m1C9/6q7Dt6Y/sCOu7/2xpWL/oC5vD+utHZ+TWOi4vGGev8twIjuxAIgg7kAjCDiSCsAOJIOxAIgg7kAjCDiTCvMHzmSfqtT5fYZc3bX8oZuWusO4jx8P6rofOC+tXLHkptzbq8eWaL+ndHtaLDI2Vc2vX974Zti1ZfBzcNBwvZXZeV/6+Jensn/5Fbm3JlzaFbSMbfL0O+YFJJ7xzZAcSQdiBRBB2IBGEHUgEYQcSQdiBRBB2IBHMZ09c0Tj66O9/MqzP79sf1lfMfC23VtJY2LavdCSsjygepy+V8s8h+eHRU8K2PR3x8/LO6OywfnLHzvjxN50U1huh8MhuZveZ2aCZbZ2wrc/Mnjaz7dn3/CveA2gLU3kZf7+kKz+07RZJ6919qaT12c8A2lhh2N39WUkHPrR5laR12e11kq6uc78A1Fm179nnuvve7PZbkubm3dHM1khaI0k9av77FADjav403sdn0uR+EuLua9293937y+qudXcAqlRt2PeZ2TxJyr4P1q9LABqh2rA/Lml1dnu1pMfq0x0AjVL4nt3MHpR0maTZZrZb0tcl3S7pITO7QdIuSdc0spNonM6FC8L6jj+Px8Jvnj8Q1l8ZOj23dmZ3PEZfsnjfIwXz4Y8H9dM7D4ZtKwXHwZFSHJ3pHcEa6pK6363hOhJVrs9eGHZ3vzanxFUogF8jnC4LJIKwA4kg7EAiCDuQCMIOJIIpridEwxlS1cvkTkXHSfFpxGPH8pc9LlKae1pYH70/bv+dM+8N63e9+Zmw/vk5z+fWZpbeD9sOeXw55hGP/3zLNppb21/pDdv22EhYr3h8nDw6Fv+9XPCX+Us6v35P2JQlmwHECDuQCMIOJIKwA4kg7EAiCDuQCMIOJIJx9hOauHT1h9Uyji5JfvH5ubXr7/9h2PZbOz8d1p86HC/J/CenbQzrczoP5daKLiVdNI4uiy/33FEwRTZysBKf+1A0/Xb7SHzB5f9Y8D+5tZW6MGxbLY7sQCIIO5AIwg4kgrADiSDsQCIIO5AIwg4kgnH2KbLO/KfKR/PnTU/psT95blj/5dXx3Ovv/eldubV/2Bkvw3frkh+H9ReG54f16R3DYX3n8fyljeeX3w3blq0S1osuJV0J6kVj+F0F+z40Fs+1HxydE9Zf6dyRW9v9dxeHbRf888/Deh6O7EAiCDuQCMIOJIKwA4kg7EAiCDuQCMIOJMK8ifO4e63PV1iw+GvRtdsbqYbnodQbj4Mf/dTZYf0f/3VtWP+nnZ8N6xfOeiO3tqj7nbDtzNJQWJ9Typ+PLknHvDusDwXj0UVzxns64mu3F13bPbou/cyO+N+9sHQkrPeV4jH+rcfj56Uv2P8DB1eEbTcuy9/3Bl+vQ35g0iAVHtnN7D4zGzSzrRO23WZme8xsU/a1suhxALTWVF7G3y/pykm23+nuy7KvJ+rbLQD1Vhh2d39W0oEm9AVAA9XyAd1NZrY5e5mfe8EtM1tjZgNmNjCi+DxqAI1TbdjvlnSWpGWS9kq6I++O7r7W3fvdvb+s+EMLAI1TVdjdfZ+7V9x9TNI9kpbXt1sA6q2qsJvZvAk/fk7S1rz7AmgPhfPZzexBSZdJmm1muyV9XdJlZrZMkkvaKenGuvSmhdduL1L6rTNza3u+0RO2/ewZ/xvWHz5wUVi/bn7c/uhY/tuj3o54DfS+gvHks7viOeczLT5edATnTsyw+G1dqeCxR7xovnt+fetIfE7Hk0c/EdZfHz41rL8zMj2sP/Pfy3JrnUfjvi0qD+QXg39XYdjd/dpJNt9b1A5Ae+F0WSARhB1IBGEHEkHYgUQQdiARbXUp6dK58XDH8NwZubXuffEQ0tjLr4b112+Ozwva8tf/llv7wqt/ELY9OBpP5TylM16y+azyYFiPlibeX5kZtt1fiafnvnksXnp409FFYf2l9+bm1gaP5P8+JengwXj4yt7uCuvTBvOPZeV45q6sYBh42tvxks2HF8VTYHsP5T/+4UXxvq0rf9qwjeYPvXFkBxJB2IFEEHYgEYQdSARhBxJB2IFEEHYgEU29lPTJ3XP94tO/lFvf/cfxmO2M3flTFsfK8bTAYChaktQxGj8PJ+3Jv/Svd8b/Z1amxWOu5YPx5boqJ8XLA4+eFDx+wdW5K91x38c6a7u8dyXoupfix35/Tlw/siSe4updwS+9o2AsuxTXu18ruIT24uNhvWdX/jkCQ6fHS4DP+kX+7/uVh+/UscE3qruUNIDfDIQdSARhBxJB2IFEEHYgEYQdSARhBxLR1PnslRldeveSBbn1oVPjsc3pe4O5uiNx29Fp8f9rQ6fEY7rvLc6fWz0WT6tWpWAhnLHueBy99H7BOQTBcPNIb8F5FAVlq/E0jEp3/gN40Vh3Ja53HIt/p51v549Hd8TD4CoV1CsFv/Poks6SNG0w/982tDA+KeTIGfn/ruhvkSM7kAjCDiSCsAOJIOxAIgg7kAjCDiSCsAOJaOo4e8dQRb0vH86t/+GtwVK0kgavyr8G+o9eOC9s6wfiwe7ykXhcNBqXrfTE48Glofixi+qj0woeP+hb+b3a5vkXzYf3gnr5UP4dPJ7mLy84FEXnFxS1L/qdBatgS5JGZsbty73xQP3Bc/I79zcXPxm2/c+/vTK3tu9Ifr8Kj+xmttDMnjGzF8xsm5l9JdveZ2ZPm9n27Hu8mgCAlprKy/hRSV9z93Mk/Z6kL5vZOZJukbTe3ZdKWp/9DKBNFYbd3fe6+/PZ7cOSXpQ0X9IqSeuyu62TdHWjOgmgdh/rAzozWyzpAkkbJM11971Z6S1Jky7qZWZrzGzAzAZGRuM1zQA0zpTDbmYzJD0i6avu/oFl8Xz8qpWTfjLg7mvdvd/d+8ud8QKHABpnSmE3s7LGg/6Au/8g27zPzOZl9XmS4qVGAbRU4dCbmZmkeyW96O7fnFB6XNJqSbdn3x8r3NuxIfn/bcst//R3p8Xtl/92bqlrZUHbc/OH/CTplBnxW4z5M97LrZ1czr/MtCTNLKgPDsXLKhe1LwXzUDsLxqd6O+PHbqRyQd96OkbCerRUdZFSwdze2Z3xms6/HD4trL98JH+pakmaviR/aO6OZ1aGbZf+eENuzTz/73gq4+yXSLpO0hYz25Rtu1XjIX/IzG6QtEvSNVN4LAAtUhh2d/+Z8k+tuLy+3QHQKJwuCySCsAOJIOxAIgg7kAjCDiSiqUs291qfrzA+wAcaZYOv1yE/wJLNQMoIO5AIwg4kgrADiSDsQCIIO5AIwg4kgrADiSDsQCIIO5AIwg4kgrADiSDsQCIIO5AIwg4kgrADiSDsQCIIO5AIwg4kgrADiSDsQCIIO5AIwg4kojDsZrbQzJ4xsxfMbJuZfSXbfpuZ7TGzTdlXvKg0gJaayvrso5K+5u7Pm9lMSc+Z2dNZ7U53/0bjugegXqayPvteSXuz24fN7EVJ8xvdMQD19bHes5vZYkkXSNqQbbrJzDab2X1mNiunzRozGzCzgREN19RZANWbctjNbIakRyR91d0PSbpb0lmSlmn8yH/HZO3cfa2797t7f1nddegygGpMKexmVtZ40B9w9x9Ikrvvc/eKu49JukfS8sZ1E0CtpvJpvEm6V9KL7v7NCdvnTbjb5yRtrX/3ANTLVD6Nv0TSdZK2mNmmbNutkq41s2WSXNJOSTc2pIcA6mIqn8b/TNJk6z0/Uf/uAGgUzqADEkHYgUQQdiARhB1IBGEHEkHYgUQQdiARhB1IBGEHEkHYgUQQdiARhB1IBGEHEkHYgUSYuzdvZ2b7Je2asGm2pLeb1oGPp1371q79kuhbterZtzPcfc5khaaG/SM7Nxtw9/6WdSDQrn1r135J9K1azeobL+OBRBB2IBGtDvvaFu8/0q59a9d+SfStWk3pW0vfswNonlYf2QE0CWEHEtGSsJvZlWb2spntMLNbWtGHPGa208y2ZMtQD7S4L/eZ2aCZbZ2wrc/Mnjaz7dn3SdfYa1Hf2mIZ72CZ8ZY+d61e/rzp79nNrCTpFUmfkbRb0kZJ17r7C03tSA4z2ymp391bfgKGmX1K0hFJ33H338m2/YukA+5+e/Yf5Sx3v7lN+nabpCOtXsY7W61o3sRlxiVdLel6tfC5C/p1jZrwvLXiyL5c0g53f83dj0v6vqRVLehH23P3ZyUd+NDmVZLWZbfXafyPpely+tYW3H2vuz+f3T4s6cQy4y197oJ+NUUrwj5f0hsTft6t9lrv3SU9ZWbPmdmaVndmEnPdfW92+y1Jc1vZmUkULuPdTB9aZrxtnrtqlj+vFR/QfdSl7n6hpKskfTl7udqWfPw9WDuNnU5pGe9mmWSZ8V9p5XNX7fLntWpF2PdIWjjh5wXZtrbg7nuy74OSHlX7LUW978QKutn3wRb351faaRnvyZYZVxs8d61c/rwVYd8oaamZnWlmXZK+KOnxFvTjI8xsevbBicxsuqQr1H5LUT8uaXV2e7Wkx1rYlw9ol2W885YZV4ufu5Yvf+7uTf+StFLjn8i/KunvW9GHnH4tkfSL7Gtbq/sm6UGNv6wb0fhnGzdIOlXSeknbJf1EUl8b9e27krZI2qzxYM1rUd8u1fhL9M2SNmVfK1v93AX9asrzxumyQCL4gA5IBGEHEkHYgUQQdiARhB1IBGEHEkHYgUT8PzGYFbZ6P3mHAAAAAElFTkSuQmCC\n", 363 | "text/plain": [ 364 | "
" 365 | ] 366 | }, 367 | "metadata": { 368 | "needs_background": "light" 369 | }, 370 | "output_type": "display_data" 371 | } 372 | ], 373 | "source": [ 374 | "plt.imshow(X_test.reshape(10000, 28, 28)[503])" 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": 23, 380 | "metadata": {}, 381 | "outputs": [ 382 | { 383 | "data": { 384 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0EAAAIJCAYAAABjgTZsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdfZRddZ3n+8+nHpLKI0lICCGJBBDRSPPUaUTx2ijSE5UlOm17YRqGttXYPT6ATS8b7TtD9zg9Td9RW++9LqfTgtAjYtM8jCwuKoh4GVoIBAhPCRKMAQIJCQFMCCFJVX3vH3WCRfbvJPtUnbP3OWe/X2tl5Zzf+e69v/tU/arqd/bv992OCAEAAABAVfSUnQAAAAAAFIlBEAAAAIBKYRAEAAAAoFIYBAEAAACoFAZBAAAAACqFQRAAAACAShnXIMj2Utu/sP2E7YublRQAAAAAtIrHep8g272SHpd0hqQNku6VdE5ErK63zQRPjAFNGdPxgO168fmImFN2Hu2OfobxoJ/lN6F/SgxMnFF2GkneM9TwNrF7dwsyQT30tXzauZ811Y6dhR5uzrG7Cj2eJM3safzn0njd99Cuuv2sbxz7PVnSExGxTpJsf1/SWZLqDoIGNEVv8+njOCSq7Cdx7ZNl59AJ6GcYD/pZfgMTZ+jkE/609Qcaw2eV/RtfbHibwfVPNX4gjBl9LZ/C+tloY7s+MC6+68FCj7fshnWFHk+Sfn/qtsKP2Tvvibr9bDzT4eZLenrU8w21ttexvcz2Stsr96j4USfQDZh6CgAA0DwtL4wQEcsjYklELOnXxFYfDug6tamn35T0PkmLJZ1je3G5WQHdhw8bAKA6xjMIekbSwlHPF9TaADTXa1NPI2K3pL1TTwE0CR82AEC1jGcQdK+ko20fYXuCpLMl3dictACMcsCpp0w7BcaNDxsAoELGPAiKiEFJn5H0Y0lrJF0TEY82KzEA+THtFBi3xte5Du4oLDkAQHONpzqcIuJmSTc3KRcAaUw9BdpERCyXtFySpk+dX0INKQBAM7S8MAKAcWPqKdB6fNgAABXCIAhoc0w9BQrBhw0AUCHjmg4HoBhMPa2GeMfxuWP7123KHTu46bmxpFMpETFoe++HDb2SLufDBgDoXlwJAgBAIx82RMSbIuKoiPibsvMBuhH340K74EoQAABdZnD9U2WnsF+L7pnU8Da/N6PxC3M/3/7Ghrf56rz7G97mvl27G97mJ4sa3qTjjbof1xkaqcB4r+0bI2J1uZmhirgSBAAAgCJwPy60DQZBAAAAKAL340LbYBAEAACAtvG6G4D3TSk7HXQp1gQhl54p2R9CQ8en51qnqlZRnQoAgMrjflxoG1wJAgAAQBG4HxfaBleCAAAA0HLcjwvthEEQAAAACsHNv9EumA4HAAAAoFK4ElSC3rccnWx/6fiDk+3Tvn93S/IYevdJmbbe29M3iRvekS1R6Z8/mIwdHF9aQF29Rx+ZO3bdeXMb2vfh/+muRtPJJdXP6qnX/1KGJ08eSzoAAEBcCQIAAABQMQyCAAAAAFQKgyCgzdleaPt226ttP2r7grJzAgAA6GSsCQLa36CkiyLiftvTJN1n+9aIWF12YkDVeTga26DReElufBONYZNCPfDNExreZuFFLza8za1PH9PwNmdccFzD2/TcuarhbaSnxrANgGbhShDQ5iJiY0TcX3u8XdIaSfPLzQoAAKBzjetKkO31krZLGpI0GBFLmpFUN0lVs1p3zuxk7OyHhludzutMeO7lTNtQi47VU6eS1fArr7ToiN3J9iJJJ0pasU/7MknLJGlAVA0DAADYn2ZMh3t3RDzfhP0A2A/bUyVdJ+nCiNg2+rWIWC5puSRN96x2nwkDAEBuDU87He/xhor/NVr0Ede8eljBR5SOW/2Bwo8pfbnuK0yHAzqA7X6NDICuiojry84H6DYUIAGAahnvICgk3WL7vtp0nAzby2yvtL1yj3aN83BA9di2pMskrYmIr5WdD9Cl9hYgWSzpFEmftr245JwAAC0y3kHQOyPiJEnv08gvjHftGxARyyNiSUQs6dfEcR4OqKRTJZ0n6T22V9X+vb/spIBuQgESAKiWca0Jiohnav9vtn2DpJMl3dGMxDpN74yDku1Pf/jQTNtAnRVUU65dkX4hdbw5c3LHDm3Zkm5f/Ximre/IRcnY4Wc3ZdtefTUZ27dwQabtl594QzL28Et+nmzHb0TEnZJcdh7tYO0n5uaOnXv8cw3tu5E+1Yi+J1/IHTs8ZUru2F994fjcsQt+lu6rST+9Nn9sl6pXgKT22m+KkExI/9wHALS/MV8Jsj2lds8S2Z4i6fckPdKsxAAAKNr+CpBI+8xu6M8/aAUAtJfxXAmaK+mGkeUK6pP0vYj4UVOyAgCgYBQgAYDqGPMgKCLWSco/HwMAgDZFARIAqBZKZAMAQAESAKiUZtwsFQCAjkYBEgCoFgZBzdKXfitfPTh7D+C6Vav+r/yHq1fxLaVn8uT0C27g931vb+7QmJ5dLNy3Mx079O6Tsoe6/f7cxwKA0uzYKd39UMsPU/y961tv5pV3NbzNv157cMPbzNuxpuFtAFQD0+EAAAAAVAqDIAAAABTC9uW2N9vmtiooFYMgAAAAFOUKSUvLTgJgEAQAAIBCRMQdkl4oOw+AwghNMvT81mT79F9m2+48L30PvtPf8/Fke99P7xtzXpI0/Mor+WPX7Ui29771mGzjo79Ixg4l2uevThdh6Jt/WKZtcD/5odpS/ameev2snn+z5YQGs8mnp4H+50ULcscu+i8rc8f2zp2TOxYAymZ7maRlkjQw4aCSs0G34koQAAAA2kZELI+IJRGxpL8/W3EWaAYGQQAAAAAqhUEQ0AFs99p+wPZNZecCAADQ6RgEAZ3hAknc9Q8A0NFsXy3pLknH2N5gO70gGmgxCiMAbc72AkkfkPQ3kv6s5HQAABiziDin7BwAiUFQy825/N5M28f++H9Lxn7o//5Jsv2mt85sak579UzJLjYc3pGuDucXfp1t60t/+8Rgtr6b+/qTsYMbntlfihjxdUlfkDStXsDrKulockFpAQAAdCamwwFtzPaZkjZHxH7rpL+uko4mFpQdAABAZ+JKENDeTpX0QdvvlzQgabrt70bEuSXnBaAga6/47Ya3OfqPxnd/uXZUb6ZC0zl9X7v9imh+HgBaiitBQBuLiC9GxIKIWCTpbEk/ZQAEAAAwPgyCAAAAAFRKV0+H6zt8YaZtz/xZyVj//MGW5JAqErDuv7wlGft/fPOHyfZv/89PZtoO/VD+ask9k9ML5X34/GzjmrXJ2MGNmzJtvQfXeS+nTc027sm+D5I0+MyzyXZkRcTPJP2smfvsO3JR7tgt75qXO3bG46/kjm2k781eflfu2F/+5cu5YyXpqHsHcseuX5q/+ETMOyR/Ehs35w7tnX9o/v328HkXAACjHfA3o+3LbW+2/ciotlm2b7W9tvZ/a8qXAQBQIG5MDADVkOfjwSskLd2n7WJJt0XE0ZJuqz0HAKDTcWNiAKiAA06Hi4g7bC/ap/ksSafVHl+pkSk6f9HEvAAAKBQ3JgYgSb/8fG/hxzzy3xV7vP91XP4p4M2yYM7zhR/z0f28NtaJ4nMjYmPt8SZJc+sF2l5me6XtlXu0a4yHAwCg5fbemHi4XgC/0wCgO4x7tWxEhKS6BfK5iSMAoN1xY2IAqJaxVod7zva8iNhoe56k/CWNxilVzape1apZV2V/l/U8+1wytsjbnA3cdE+y/czz/zTZ/u5F2Yptv2zgeJ6UvuQZiYpRfXPTlawGN2Xft6GtL6QP+OKvs/udn/4apb6eg+vWp/cLAK3DjYkBoELGeiXoRknn1x6fL+kHzUkHAIDicWNiAKiWPCWyr5Z0l6RjbG+w/XFJl0o6w/ZaSe+tPQcAAACAtpenOtw5dV46vcm5AABQulbcmBgA0F7GuiYIAAA06pTjGt5k0i8aL8Cw7nsnNLzNkf9uVcPbdKPe2bMb3mZoy5YWZAKgldp2ENQzkF7M/+z7D8u07Th1RzJ25hW7m5pTqx35Zy8m25/sTRUreCoZ2zc/+/7EjvT7o77sbMihF19Kx/YkauYPDyVDe6dOyebwcjqHoRfT54yxc2+veg+amSs21Z/qqdfPUubckS2OUVfie7aewWeezR37uXeenT8HSerNv0TSA4O5YyPRz+pK9bN6BtP9Lx1LKWcAAEYbd4lsAAAAAOgkDIIAAAAAVAqDIKAD2J5h+1rbj9leY/vtZecEAEAjbC+0fbvt1bYftX1B2Tmhutp2TRCA1/mGpB9FxEdsT5A0ueyEAABo0KCkiyLiftvTJN1n+9aIWF12YqgergQBbc72QZLeJekySYqI3RFRp4IFAADtKSI2RsT9tcfbJa2RNL/crFBVbXElqHdmtpLV5n/75mTswNbhTNthf5muRDU8bVqmrWfOwcnYwXXrM23un5CM7T0kWz6zkapV9Qw+vSF3bO+Mg5Ltsf3lbOOE/mSs92SrSw3vSleRSlXr87RZ6f1OzL5vsW17Mrbv0LmZtsFNzyVjK+wISVskfcf28ZLuk3RBROQv1wYAQBuxvUjSiZJWlJsJqoorQUD765N0kqRvRcSJknZIunh0gO1ltlfaXrk7Xi0jRwAAcrE9VdJ1ki6MiG2J11/7nbZnD5/3oTUYBAHtb4OkDRGx99OyazUyKHpNRCyPiCURsWSC0/fYAgCgbLb7NTIAuioirk/FjP6d1t+fvfcg0AwMgoA2FxGbJD1t+5ha0+mSWEQKAOgotq2R9a1rIuJrZeeDamuLNUEADuizkq6qVYZbJ+ljJecDAECjTpV0nqSHba+qtX0pIm4uMSdUFIMgoANExCpJS8rOAwCAsYqIOyW57DwAqehB0OQB+a3HZtufzFYD2zM13UcOvuyuTNtgncPtPOvkTNvWxelTXvC36zNtsWd3MjZVCa5v4YJkbLzySqZtaOsLydh6+hbkrx45uOGZTFvvnDnJWG/NVll2X/r9iaFsVb56P8Vi585M29C2zLrHEfXaMWYxsV/DR+X7nhnuzb/fQ78/MXfs+v/9sNyxC/7257lj6/WzlKHNW3LHSlLvnGzVx3piav7bNPVs2po/iWn5577Hr/P3nUZ/5gAA0O24EgQAQFHufqjhTRbc3fhhHv+H32l4m199/7iGtzni7MbPp0gbL3pHw9v0n/Z8w9sM/NORDW+ja65tfBsATUNhBAAAAACVwiAIAAAAQKUwCAIAQJLtGbavtf2Y7TW23152TgCA1jjgmiDbl0s6U9LmiDi21vZXkj4pae/K4/zlDSMyTUNbsguY5/339KJfD2RvBDn86qvJ2Cm3PpJpm/jSMYlIyf0TMm31CiOkDG3anGxP7SN1LElST7rUwNCW7Pzknql1FlD3JFa6zzooGTr8bLYgRQymy0z0LXpD4ljpMfTguvXp3ACgvX1D0o8i4iO1cvT5K2AAADpKnitBV0hammj/+4g4ofaP+u4AgI5l+yBJ79LIjRwVEbsjIltCEwDQFQ44CIqIOyRRXxUA0M2O0Mjshu/YfsD2t23nr1kOAOgo4ymR/Rnb/17SSkkXRcSLqSDbyyQtk6SBCelpWQAAlKxP0kmSPhsRK2x/Q9LFkv7j6KDX/U5jthzQejt2jqm0/HgMfqzxEvPjNZYS9ePR15e9/2OrTZqYf5lJ05xZ/6WxFkb4lqSjJJ0gaaOkr9YLjIjlEbEkIpb09/ELAwDQljZI2hARK2rPr9XIoOh1Xvc7TflvIAwAaC9jGgRFxHMRMRQRw5L+UdLJzU0LAIDiRMQmSU/b3ls953RJq0tMCQDQQmOaDmd7XkRsrD39sKRsGbbUdkOhnu3ZSm5DqdgJdSqoHTE/09T36x3J0MEnn8609T/wy/R+j8xWPxte91QyNFXxrV4lub4F2Xw1oT8Z20hVtehNVIGTpOHEu7kxXbluePv23Md74e2HZdoGB9LV7Ob0ZXMbejz9vvctXJBpi4np90dr95MgXrP7UGvdn+f7fOOo/5p/uV9PnX6WMvGF2bljB9/z27lj+x7flDu2d/683LGSNLxla+5Yb8vfdwa3pStdprzwsfwVmac+m/89nvTLBqYiV7uffVbSVbXKcOskfazkfAAALZKnRPbVkk6TNNv2BkmXSDrN9gmSQtJ6SZ9qYY4AALRcRKyStKTsPAAArXfAQVBEnJNovqwFuQAAAABAy42nOhyAgtj+vKRPaOTq68OSPhYR6bsEA6i8N33q3rJT2K/dSxuvvvXcJ3Y2vM2iT65peJuhryaL3QLoMmOtDgegILbnS/qcpCURcaykXklnl5sVAABA5yr0StCu+T164j9n7z131MezbXUX7T/0WKbJ06cnQ/sOX5g7t92HTMvmMP+3krETH84WTIg6+Q5teT7T1nvoIclY96eLQSSLLtQrjJDKoYGF2fXMfDC7gH7PrHTJ851HzMy0TRo+Ihk7+MSvxpdYdfRJmmR7j6TJkp4tOR8AAICOxZUgoM1FxDOSviLpKY3cl+vXEXHL6Bjby2yvtL1yaFv+Km4AAABVxCAIaHO2Z0o6S9IRkg6TNMX2uaNjRt/AsXd69soqAAAAfoNBEND+3ivpVxGxJSL2SLpe0jtKzgkAAKBjMQgC2t9Tkk6xPdm2NXIn+8ZLHgEAUCLbA7bvsf2g7Udt/3XZOaG6KJENtLmIWGH7Wkn3SxqU9ICk5eVmBQBAw3ZJek9EvGy7X9Kdtn8YEXeXnRiqp9BBUG/vsGZNfyXT/thX35ppm3N3uvrZzCvuyjZOnJiMHXzy6dy59b3wUqbN89JV3DR7RrbtsNnJ0Hgw+4F9vLorGdt7SHofQ5uzFeY0NJTObZzcl/6WGFr9eKat3mXEVI271mRbHRFxiaRL8sTW62cpj/9R+nsuZfKzs3LHOnKHasHNW3LHDh+crgSZzGFHg7dRaqBPDZ5wdO7YnnsezR076zuJn29NQP8D0A4iIiS9XHvaX/vXwG8MoHmYDgcAAIBC2O61vUrSZkm3RsSKRMxrFU/3KP3BMTBeDIIAAABQiIgYiogTJC2QdLLtYxMxr1U87Vd6tg8wXgyCAAAAUKiIeEnS7ZKWlp0LqolBEAAAAFrO9hzbM2qPJ0k6Q9Jj5WaFqiq0MELfBmvmF7KHfOmj2SIIW49Lr5PrPeeUTNvL89NjuTkPHZ5pe/63Usv2pYXXPJVpG3z8l+kcpmcXZw9v25aOnTMn0zb03OZ07BuPSLbHnt252iQp3nF8ps0/fzAZm9x+cDB3LACgPfUduajhbZ78g8Ma3sanZIsK5dH3k/6Gt1n4kXsb3oaiIG1nnqQrbfdq5IP4ayLippJzQkVRIhsAAAAtFxEPSTqx7DwAielwAAAAACqGQRAAAACASmEQBACAJNuft/2o7UdsX217oOycAACtwSAIAFB5tudL+pykJRFxrKReSWeXmxUAoFUOWBjB9kJJ/yRprqSQtDwivmF7lqR/lrRI0npJH42IF/e3r9j5qoYfyVZCPPyRbGzv0Ucm9zF08NRsjsOTk7G7p2Wrzr38huFk7IZ/+4ZM28wn5iVjXz40+7bNuTd96s+fNDO73396IRn7wtvmJtsPmj0t23j3Q8nY4YnZc54w++Bk7NDzW5Pt6Gy9a3fpoPc/kSv2oFPSfSdl++H5Yze9I3eoNr9jdu7YngZKPfXsSVeYrKf/lUNyx058aU/uWFNxsZP0SZpke4+kyZKeLTkfAECL5LkSNCjpoohYLOkUSZ+2vVjSxZJui4ijJd1Wew4AQMeJiGckfUXSU5I2Svp1RNyyb5ztZbZX2l65R7uKThMA0CQHHARFxMaIuL/2eLukNZLmSzpL0pW1sCslfahVSQIA0Eq2Z2rk99oRkg6TNMX2ufvGRcTyiFgSEUv6NbHoNAEATdLQmiDbizRS332FpLkRsbH20iaNTJdLbcOnZkAOti+3vdn2I6PaZtm+1fba2v/Z+ZUAmuG9kn4VEVsiYo+k6yU1MLETANBJct8s1fZUSddJujAittl+7bWICNvJCfgRsVzSckma7lmNTdIHquUKSf+PRtbg7bV32umlti+uPf+LEnIDut1Tkk6xPVnSTkmnS1pZbkoAyrD477YUfsynvpp/3W0z9P1kRqHHk6TZ3/x54cfcn1yDINv9GhkAXRUR19ean7M9LyI22p4naXMzExtauy79wtps07QVzjZKUmTHXG++dXoydPCtR2TanNhekob76xwvYeK2bCGGvkPSC8FnrUoXV9j+puw36vQF89MHvP3+TFMDa8lRooi4o3a1dbSzJJ1We3ylpJ+JQRDQdBGxwva1ku7XyFrYB1T7AA8A0H0OOB3OI5d8LpO0JiK+NuqlGyWdX3t8vqQfND89oPKYdgoUJCIuiYg3R8SxEXFeRNCZAKBL5bkSdKqk8yQ9bHtVre1Lki6VdI3tj0t6UtJHW5MiAIlppwDyG1y3vuFt5v9d49tghJcc2/hG917b/EQA5HbAQVBE3Cmp3vyv05ubDoB9tHTaKQAAQBU1VB0OQOGYdgoAANBkDIKANmH7akl3STrG9obaVNNLJZ1he61GSvheWmaOAAAA3SB3iey2VqeKW8rQtm3Jdt/1YO59TEq0ZWvAjZiyZkKmbXDP7nTwpueSzZMfzbYN1jkeOldEnFPnpdZNO737odyh0+7Ov9tp/zyGXHJwf7Y/1RP1+hkAAKg8rgQBAAAAqBQGQQAAAAAqhUEQAAAAgEphEAQAAACgUrqjMEIbY3E2AADAb9julbRS0jMRcWbZ+aCauBIEAACAIl0gaU3ZSaDaGAQBAACgELYXSPqApG+XnQuqjUEQAAAAivJ1SV9Q/VssyvYy2yttr9yjXcVlhkphTRAAAECNf+e3Gt5muJ/PlPOwfaakzRFxn+3T6sVFxHJJyyVpumdFQemhYui1AAAAKMKpkj5oe72k70t6j+3vlpsSqoorQQA6BtUWAaBzRcQXJX1RkmpXgv48Is4tNSlUFleCAAAAAFQKgyAAQGXYvtz2ZtuPjGqbZftW22tr/88sM0egCiLiZ9wjCGViEAS0iTp/nP0324/Zfsj2DbZnlJkj0AWukLR0n7aLJd0WEUdLuq32HADQxRgEAe3jCmX/OLtV0rERcZykx1WbSw1gbCLiDkkv7NN8lqQra4+vlPShQpMCABSOQRDQJlJ/nEXELRExWHt6t6QFhScGdL+5EbGx9niTpLn1Arl/CQB0hwMOgmwvtH277dW2H7V9Qa39r2w/Y3tV7d/7W58uUGl/LOmHqRf4wwxojogISXXvSxIRyyNiSUQs6dfEAjMDADRTnhLZg5Iuioj7bU+TdJ/tW2uv/X1EfKV16QGQJNt/qZG+eFXqdW4sB4zLc7bnRcRG2/MkbS47IQBAax3wSlBEbIyI+2uPt0taI2l+qxMDMML2H0k6U9If1j6lBtBcN0o6v/b4fEk/KDEXAEABGloTZHuRpBMlrag1faZWteryeiVFmaYDjJ3tpZK+IOmDEfFK2fkAnc721ZLuknSM7Q22Py7pUkln2F4r6b215wCALpZnOpwkyfZUSddJujAittn+lqQva2Tu9JclfVUjaxZeh2k6QD61P85OkzTb9gZJl2ikGtxESbfalqS7I+JPSksS6HARcU6dl04vNBEAQKmcZ3aN7X5JN0n6cUR8LfH6Ikk3RcSxB9jPFklP1p7OlvR8g/l2Cs6tNQ6PiDklHbtj7NPPRuP7snMVeX70s5wq2tcawfuw//eAvpbDfvrZgVTl+68K5zmec6zbzw54JcgjHz9fJmnN6AHQ3kWktacflvRIavvRRidhe2VELDnQNp2Ic0OZ6nX2bv7adfO5Sd1/fp2qin2tEbwPvAfNMNaBYlXe+yqcZ6vOMc90uFMlnSfpYduram1fknSO7RM0Mh1uvaRPNTs5AAAAAGi2Aw6CIuJOSU68dHPz0wEAAACA1mqoOlyTLS/x2K3GuaEddfPXrpvPTer+8+s2fL1G8D7wHpSpKu99Fc6zJeeYqzACAAAAAHSLMq8EAQAAAEDhGAQBAAAAqJTCB0G2l9r+he0nbF9c9PGbzfbltjfbfmRU2yzbt9peW/t/Zpk5jpXthbZvt73a9qO2L6i1d8X5VUW39bl92V5v+2Hbq2yvLDuf8ejmnydV0e39LY9u6pONoP+2jyr0w3p/o3Uj2722H7B9UzP3W+ggyHavpG9Kep+kxRops724yBxa4ApJS/dpu1jSbRFxtKTbas870aCkiyJisaRTJH269vXqlvPrel3a51LeHREndMG9Eq5Q9/486XoV6m95dEufbMQVov+WrkL9sN7faN3oAklrmr3Toq8EnSzpiYhYFxG7JX1f0lkF59BUEXGHpBf2aT5L0pW1x1dK+lChSTVJRGyMiPtrj7dr5Btwvrrk/Cqi6/pcN+vmnycVQX+rMPpv26hEP9zP32hdxfYCSR+Q9O1m77voQdB8SU+Per5BXfgFkzQ3IjbWHm+SNLfMZJrB9iJJJ0paoS48vy5WhT4Xkm6xfZ/tZWUn0wL0t85Rhf6WR7f3yUbQf4tXuX64z99o3ebrkr4gabjZOz7gzVIxPhERtju6DrntqZKuk3RhRGyzf3Pv3G44P3S8d0bEM7YPkXSr7cdqn8h2HfobOkRl+mQj6L9ohX3/Ris7n2ayfaakzRFxn+3Tmr3/oq8EPSNp4ajnC2pt3eY52/Mkqfb/5pLzGTPb/RrpXFdFxPW15q45vwro+j4XEc/U/t8s6QaNTIXoJvS3ztH1/S2PCvTJRtB/i1eZfljnb7RucqqkD9per5Fpje+x/d1m7bzoQdC9ko62fYTtCZLOlnRjwTkU4UZJ59ceny/pByXmMmYeueRzmaQ1EfG1US91xflVRFf3OdtTbE/b+1jS70l6ZP9bdRz6W+fo6v6WR0X6ZCPov8WrRD/cz99oXSMivhgRCyJikUa+jj+NiHObtf9Cp8NFxKDtz0j6saReSZdHxKNF5tBstq+WdJqk2bY3SLpE0qWSrrH9cUlPSvpoeRmOy6mSzpP0sO1VtbYvqXvOr+t1Y5/bx1xJN9SmaPZJ+l5E/KjclMauy3+edL0K9Lc8uqpPNoL+2x4q1BqXZY4AACAASURBVA+Tf6NFxM0l5tRRHMH0VAAAAADVUfjNUgEAAACgTAyCAAAAAFQKgyAAAAAAlcIgCAAAAEClMAgCAAAAUCkMggAAAABUCoMgAAAAAJXCIAgAAABApTAIAgAAAFApDIIAAAAAVAqDIAAAAACVwiAIAAAAQKX0jWdj20slfUNSr6RvR8Sl+4uf4IkxoCnjOSQqbLtefD4i5pSdR7ujn2E86Gf5TeifEgMDM8pOo1QeHB7TdvHqriZn0kQewyYTJza8zbZXN9HXcqCftcjLOws/pCcNFH7MbTs31u1nYx4E2e6V9E1JZ0jaIOle2zdGxOp62wxoit7m08d6SFTcT+LaJ8vOoRPQzzAe9LP8BgZm6HdO+nTZaaRFFHKY/q07xrTd0Jq1Tc6kedzX+J9GPUcd2fA2P370v9LXcmjrftZMBfXZvXruXFXo8SSp55i3FH7MW1Z9uW4/G890uJMlPRER6yJit6TvSzprHPsDAAAAgJYbzyBovqSnRz3fUGt7HdvLbK+0vXKP2vjyN9DGbC+1/QvbT9i+uOx8AAAAOlnLCyNExPKIWBIRS/rV+JxZoOpGTT19n6TFks6xvbjcrIDuw4cNAFAd4xkEPSNp4ajnC2ptAJqLqadAi/FhAwBUy3gGQfdKOtr2EbYnSDpb0o3NSQvAKAecesq0U2Dc+LABACpkzIOgiBiU9BlJP5a0RtI1EfFosxIDkB/TToFxa3id6+49Y6uMBgAo37juExQRN0u6uUm5AEhj6inQJiJiuaTlkjR92vxia9oCAJqm5YURAIwbU0+B1uPDBgCoEAZBQJtj6ilQCD5sAIAKGdd0OADFYOop0FoRMWh774cNvZIu58MGoPlsL5X0DY30s29HxKUlp4SKYhAEAID4sAFotVGl6M/QSPGRe23fGBGry80MVcQgCAAAqP+5bQ1vM7R2XQsyaZ6e49/S8DbDD65peJuh1Y83vE1FvVaKXpJs7y1FzyAIhWNNEAAAAIqQqxQ9UASuBAEAAKBt2F4maZkkTZx4UMnZoFtxJQgAAABFyFWKfvQNwCf0TyksOVQLgyAAAAAUgVL0aBtMhwMAAEDLUYoe7YRBEAAAAApBKXq0C6bDAQAAAKgUBkEAAAAAKoXpcN2mpzfT1Hf4gmTo4K+ebHU2QMfY8ZG35Y7tfTVyxw7cdM9Y0gEAAC3ElSAAAAAAlcIgCAAAAEClMAgC2pzthbZvt73a9qO2Lyg7JwAAgE7GmiCg/Q1Kuigi7rc9TdJ9tm+NiNVlJwagPTn/srXXDK1d1/xEmuiFP357w9ts/d1dDW+zdPFAw9vMmbC94W1+clzDmwBoIq4EAW0uIjZGxP21x9slrZE0v9ysAAAAOte4rgTZXi9pu6QhSYMRsaQZSeH1eqZNy7Rt/zeLk7GpqlVTH3y26TmhHLYXSTpR0op92pdJWiZJA5pceF4AAACdpBnT4d4dEc83YT8A9sP2VEnXSbowIraNfi0ilktaLknTPWsME2EAAEBZeu5cVejxxjK9dLx+508fKPyY2s/lGabDAR3Adr9GBkBXRcT1ZecDdBsKkABAtYx3EBSSbrF9X206TobtZbZX2l65R40vUASqzrYlXSZpTUR8rex8gC61twDJYkmnSPq07fS8YwBAxxvvIOidEXGSpPdp5BfGu/YNiIjlEbEkIpb0a+I4DwdU0qmSzpP0Hturav/eX3ZSQDehAAkAVMu41gRFxDO1/zfbvkHSyZLuaEZi+I1ff+Ctmbbv/59fScaec9FFmbbBpzc0Paf9ef5T2Xmmc695LBk79OKLrU6n40XEnZJcdh6dJt5+fEPx3/vaV3PHpvpZPa+eeXLu2IGb7skdm+pn9cz+h7tyx6J+AZLaa68VIZk48aBC8wIANM+YrwTZnlK7Z4lsT5H0e5IeaVZiAAAUbX8FSKTXz26Y0D+l+AQBAE0xnitBcyXdMLJcQX2SvhcRP2pKVgAAFIwCJABQHWMeBEXEOkmNzTcBAKANUYAEAKqFEtkAAFCABAAqpRk3SwUAoKNRgAQAqoVBUBupV81q1p88mWn73VsuTMa+5ZEXMm1DTv9ef/UDv5Npm3LnL5KxQy/9OtmekqpENZR7awDoIBENhbuxcElSz67BhreJld1Xp2jW5Y1XOZyw/W0Nb7N2wZyGt1n5P05seBvpB2PYBkCzMB0OAAAAhbB9ue3NtrtvpI6OwiAIAAAARblC0tKykwAYBAEAAKAQEXGHpOzcfaBgrAkCAABA27C9TNIySZo48aCSs0G3YhBUgt7p05Ptj507kGy/7Yh/ybT9yRf+NBnr7a9k2377rcnYgZvuybRt+0h6EenU9Tsybd248BbdY8Lfbm4ofqiBBevTfpXtD/W8vGhK7lgvOTZ37KE/fiZ3bOPL6gGgPBGxXNJySZo+bf4YyokAB8Z0OAAAAACVwiAIAAAAQKUwCAI6gO1e2w/YvqnsXAAAGCvbV0u6S9IxtjfY/njZOaGaWBMEdIYLJK2RlF5QBgBAB4iIc8rOAZC4EgS0PdsLJH1A0rfLzgUAAKAbcCWoBFvPSldrW/zW9cn2Dz3wyUzbvOF0sZSYNjnT1rNjVzqRGdmyk9N+vDoZ6kPnZBvfdFQ6dveeTNvg+qfSOSCPr0v6gqRp9QJGlxMdUPZ7AAAAAL/BlSCgjdk+U9LmiLhvf3ERsTwilkTEkn5NLCg7AACAzsSVIKC9nSrpg7bfL2lA0nTb342Ic0vOC4AkN3gHk77H89/faa+hLVsa3gYjpv7LisY3yt6a74Bm6unGNwJQKq4EAW0sIr4YEQsiYpGksyX9lAEQAADA+DAIAgAAAFApB5wOZ/tySXvXJRxba5sl6Z8lLZK0XtJHI+LF1qXZubadc0qm7bc/90Ay9t5vnZhsP/TyuzJtg+8+KRnbv2co0+ZX6hRG6J+QaeqZMSkdO5jdrxIFECRJQ9nY3oNnpUO3vpDeBzIi4meSflZyGqV68f89OnfskimNTU/55B99Lnds/56duWMPWrkxfxJ27tCt7zwsfw4UJgEA4HXyXAm6QtLSfdoulnRbRBwt6bbacwAAOho3JgaAajjgICgi7pC078f1Z0m6svb4SkkfanJeAACUYe+NiQEAXWys1eHmRsTeOR6bJM2tF8j9SwAAnWDUjYn/RtKflZwOgBrXuTdiy473r6sKPV4ZJm1NLHNosR+tWVz4Mfdn3IURIiIk1f3u5P4lAIAOsffGxMP1Amwvs73S9so9e3YUlxkAoKnGOgh6zvY8Sar9v7l5KQEAUKwx3Zi4f0pB2QEAmm2s0+FulHS+pEtr//+gaRl1qGf//B3J9knvzt7kbv0fzk/GzpqV/1PFepeG/dSmbGNPnYpTfYkv/6vpSnIxOJht7O1N59CTHVu/8L43JWMP+u7d6dwAoFjcmBgAKuSAV4JsXy3pLknH2N5g++MaGfycYXutpPfWngMA0JG4MTEAVMsBrwRFxDl1Xjq9ybkAAAAAQMuNdTocAABdiRsTA0D3YxAEAMBYvLyz4VK6xRel7R47zzq54W02/H5iPesBHPyzxivZzrr8roa3AVAuBkF7OVs8YPA9JyVDn/+t7A/IXbPShQoO+8DaTFvdX4KnHJdO7cS3Ztr6V29I7+OwQ7Lbb30pGRpDiSqwfXWWiUXi/HaliygM796TaZuyMduG7rXhi+lCISmvvnVn7tjeBwdyx6798dTcsZLUty39/ZziwboVlDNi4oSG8siroT518m/lj11xbePJAADQYcZ9nyAAAAAA6CQMggAAAABUCoMgoAPYnmH7WtuP2V5j++1l5wQAQCNsL7R9u+3Vth+1fUHZOaG6WBMEdIZvSPpRRHzE9gRJk8tOCACABg1Kuigi7rc9TdJ9tm+NiNVlJ4bq4UoQ0OZsHyTpXZIuk6SI2B0R6WoXAAC0qYjYGBH31x5vl7RG0vxys0JVVe5KUO/Mmcn2J//kLZm2elWr3njuzzNtPce+ORmbqiE1/LsnJmN7/r8Hku298w7NtEWdymypKm5x8Ix07J5s6dBIVMmrazhdIWt47bpMW99t96X3kapadc/D+XOohiMkbZH0HdvHS7pP0gURsWNvgO1lkpZJ0gAXiQAAbc72IkknSlqReO03v9MmHlRoXqgOrgQB7a9P0kmSvhURJ0raIeni0QERsTwilkTEkn41fo8LAACKYnuqpOskXRgR2/Z9/XW/0/qnFJ8gKoFBEND+NkjaEBF7Py27ViODIgAAOortfo0MgK6KiOvLzgfVxSAIaHMRsUnS07aPqTWdLolFpACAjmLbGlnfuiYivlZ2Pqi2yq0JAjrUZyVdVasMt07Sx0rOBwCARp0q6TxJD9teVWv7UkTcXGJOqCgGQUAHiIhVkpaUnQcAAGMVEXdKaqACE9A6XT0IevkP3pZp23haOvaN33sl0/bK4w0sMO9N9+m+BdnKj8MvvZqM9VuPSbYPr9+QbduxIxEp+fHseXhi+jxiMFsdTsPZ6nKSFHt2Z9rqVdrrO3Rupm1w03PJWCrBNZ8n9KvvsIW5YgeffDr3fh//h9/Jn8PgUO7Yw26YkDt22hOtqwzeszWzNreuOGhq7tjBWfkX9b46N//PnFfm9OaOnb0q238BAKiyrh4EAQDQ6X55Vfq2Cvuz8H80/ut9wo/ubXibsVr3d29veJu+I19ueJvY1ni1zGH+MgIqgcIIAAAAACqFQRAAAACASmEQBACAJNszbF9r+zHba2w3PmcLANARDjjz1fblks6UtDkijq21/ZWkT0raUgvLVd6w3oLtRhZnp9RbsD15fXbh8NGf/nnu/U477s3J9p3vyx5v4g/Tc6mjL/sW11vOPDxjWrLdvdktUgUXpDoLtiNd7EB92THw0JT0/Omhydnz2DktfSY7Z2Xbo/fIZOzs7yTetxPfkozVPdem2wGgOb4h6UcR8ZFaOfrJZScEAGiNPFeCrpC0NNH+9xFxQu0f9d0BAB3L9kGS3qWRGzkqInZHROvKEQIASnXAQVBE3CHphQJyAQCgLEdoZHbDd2w/YPvbtjP1zW0vs73S9so92lV8lgCAphhPIcjP2P73klZKuigiXkwF2V4maZkkDfSmp3sBAFCyPkknSfpsRKyw/Q1JF0v6j6ODImK5pOWSNN2z6sw1BtAsbzpyq275lysKPeYbb/9YoceTpKP+8IFCj7fl3J2FHk8aW8n6VhprYYRvSTpK0gmSNkr6ar3AiFgeEUsiYsmE3kljPBwAAC21QdKGiFhRe36tRgZFAIAuNKZBUEQ8FxFDETEs6R8lndzctAAAKE5EbJL0tO1jak2nS1pdYkoAgBYa03Q42/MiYmPt6YclPZJnu6GpE/Ti27NVzaaNszrc5PX9yfYpG/PPVNjwpXdk2nr2pGN3nvhKpq33nelKqhNedKZtOJ2uPJxuH9g6MxGcjk3d6XrP1HTwwPPZ92fOPenlX32/zsZOvHdTMnbStm2Zts2fzr6/khSDg5m23qeeS8Yin3r9LKWRvlevnyXV+f5M2fT72f5Uz5Zfzcgd2/dyA0lImvR8/n3vmpF/36l+Vs+cFVtzx26fPzt37NZjE1Uj67knf2gX+qykq2qV4dZJKn5ODACgEHlKZF8t6TRJs21vkHSJpNNsnyApJK2X9KkW5ggAQMtFxCpJS8rOAwDQegccBEXEOYnmy1qQC4A6bH9e0ic08sHDw5I+FhGvlpsVAABAZxpPdTgABbA9X9LnJC2OiJ22r5F0tkbu4QWgJG867hX9+MerGtrmuK/+h4aPc9Qf5r/Jd6c48i/uKjsFABU31upwAIrVJ2mS7T6N3MX+2ZLzAQAA6FiFXgnyYGjghexC+PFa8LfpT8ni7cdn2oZOS1c8fcNN2YIAXl/n78yF8zJNOxemF41Hb7ZtYFOdheCus9h6KP/C6p7t2X0PzUovio7e7Bh4aM0T6R0PD+XOIWXeT7ck21N73fL+o9I7uXxcKXSsiHjG9lckPSVpp6RbIuKW0TGj78c1YXKikAYAAABew5UgoM3ZninpLI3c0f4wSVNsnzs6ZvT9uPoHMje5BwAAwCgMgoD2915Jv4qILRGxR9L1ktI1xwEAAHBADIKA9veUpFNsT7ZtjdzEcU3JOQEAAHQsBkFAm4uIFZKulXS/Rspj90haXmpSAAA0yPaA7XtsP2j7Udt/XXZOqC5KZAMdICIu0ciNigEA6FS7JL0nIl623S/pTts/jIi7y04M1VPoIGjiYa/qiP/8WKZ9wy2J4CbwXQ9m2hLF2iRJw43s+NFtmaYJj+bfPH+tt8Y1UsMtVYuuZ0p6Ub37s98qnjYtncMhB2XaGnl/J704vkp0VffmBVv086/991yx737xE7n3W68KI/avXp9K8az8lf3m3v3r3LGvLKRYBoDyRURIern2tL/2r5V/FgF1MR0OAAAAhbDda3uVpM2Sbq1N+d43ZpntlbZXbtnKB6NoDQZBAAAAKEREDEXECZIWSDrZ9rGJmNdu+zDn4HpzeIDxYRAEAACAQkXES5Jul7S07FxQTQyCAAAA0HK259ieUXs8SdIZkrKLxYECFFoYYdGEl3XZG+7MtL/lP/2HTNuhK3Yn9zHprsczbUPbsoUKMDbDO3bkD36pzsLspzeMK4dJ//OecW0PAEVY+4sZev+7PtzQNvOeoMCIJPUMDDS+0RsXNbzJzsPTBXz2awzL9CfefG/jG1XTPElX2u7VyAfx10TETSXnhIqiRDYAAABaLiIeknRi2XkAEtPhAAAAAFQMgyAAACTZ/nztLvaP2L7a9hjmbAEAOgGDIABA5dmeL+lzkpZExLEaubf22eVmBQBoFQZBAACM6JM0yXafpMmSni05HwBAixywMILthZL+SdJcjdRMWR4R37A9S9I/S1okab2kj0bEi/vb12NPz9E7P/up7Atnb8807X5HukrZkx98S6ZtypPpG2n17Ek01qn60vtq9oUJ29PBrx6cHTt6ML1fR/4yM8l8Jcm5d6HhxFsRvekdeCibm+ukOzQhuw8Pp2Mnb87e3Xln4j2TpGnPZGOn/GJrese/SDfj9R55+WC95V/PyxU76/Mv5N7v88e/I3dsT7q4Y1K976NkbJ1+loxtoO9JUk8D+071h7p5JPpZPcP9+ffbs6eBny0NnFtVRcQztr8i6SlJOyXdEhG37Btne5mkZZI00De92CQBAE2T50rQoKSLImKxpFMkfdr2YkkXS7otIo6WdFvtOQAAHcf2TElnSTpC0mGSptg+d9+40Xeyn9A7qeg0AQBNcsBBUERsjIj7a4+3S1ojab5GfllcWQu7UtKHWpUkUAW2L7e92fYjo9pm2b7V9tra/zPLzBHoYu+V9KuI2BIReyRdLyn/5U8AQEdpaE2Q7UUaqe++QtLciNhYe2mTRqbLpbZZZnul7ZV7dr08jlSBrneFpKX7tHHFFSjGU5JOsT3ZtiWdrpEP/QAAXSj3zVJtT5V0naQLI2LbyO+IERERdno1SUQsl7RckqbOWjiG+zAD1RARd9Q+aBjtLEmn1R5fKelnkv6isKSAioiIFbavlXS/RqaBP6Da7y4A5Vn7ixl6/7s+XOgxj3rigUKPV4ZFf11vIXrr7Dy8+LsOPLWf13INgmz3a2QAdFVEXF9rfs72vIjYaHuepM0H2k/Pizs05boVmfYp12Vje2cfnNzHlGOyC1H3TOtPxg4NZBcZD9VZeJxa6Nxb5/tj8nOJldzjLF7Q6D4aEXWu90XiePUKI/S9kn1haGI6dtf07AEHXkzveMch2Tdjx9xD0juuZmGE3FdcVVus3Tf7oIJSA7pLRFwi6ZKy8wAAtN4Bp8PVpgVcJmlNRHxt1Es3Sjq/9vh8ST9ofnoA9oqIUJ36hqMXa/dOn1JwZgAAAJ0lz5WgUyWdJ+lh26tqbV+SdKmka2x/XNKTkj7amhSBSmv4iiuAYsSu3Rp64ldlp9GRhl99tfGNHnms4U0mPnLgGADVdMBBUETcqfoTtU5vbjoA9rH3iuul4oorAABAUzRUHQ5A69i+WtJdko6xvaF2lfVSSWfYXquREr6XlpkjAABAN8hdHQ5Aa0XEOXVe4oorAABAE7XtIGjo+a3JdifaJ7Q6GaCDTFi3U2/4g4ebvt+pWtf0fQIAAJSB6XAAAAAAKoVBEAAAAIBKYRAEAAAAoFIYBAEAAACoFAZBAAAAKIztXtsP2L6p7FxQXQyCAAAAUKQLJK0pOwlUG4MgAAAAFML2AkkfkPTtsnNBtTEIAgAAQFG+LukLkobrBdheZnul7ZW7h3YWlxkqhUEQAAAAWs72mZI2R8R9+4uLiOURsSQilkzonVRQdqgaBkEAAAAowqmSPmh7vaTvS3qP7e+WmxKqikEQAAAAWi4ivhgRCyJikaSzJf00Is4tOS1UFIMgAEBl2L7c9mbbj4xqm2X7Vttra//PLDNHAEDrMQgC2kSdP87+m+3HbD9k+wbbM8rMEegCV0hauk/bxZJui4ijJd1Wew6ghSLiZxFxZtl5oLoYBAHt4wpl/zi7VdKxEXGcpMclfbHopIBuEhF3SHphn+azJF1Ze3ylpA8VmhQAoHAMgoA2kfrjLCJuiYjB2tO7JS0oPDGg+82NiI21x5skzS0zGQBA6x1wEGR7oe3bba+2/ajtC2rtf2X7Gdurav/e3/p0gUr7Y0k/LDsJoJtFREiKeq+Pvn/JHu0qMDMAQDP15YgZlHRRRNxve5qk+2zfWnvt7yPiK61LD4Ak2f5LjfTFq+q8vkzSMkka0OQCMwO6wnO250XERtvzJG2uFxgRyyUtl6TpnlV3sAQAaG8HvBIUERsj4v7a4+2S1kia3+rEAIyw/UeSzpT0h7VPqTNG31iuXxMLzQ/oAjdKOr/2+HxJPygxFwBAARpaE2R7kaQTJa2oNX2mVrXq8nolRZk6AIyd7aWSviDpgxHxStn5AJ3O9tWS7pJ0jO0Ntj8u6VJJZ9heK+m9tecAgC6WZzqcJMn2VEnXSbowIrbZ/pakL2tk7vSXJX1VI2sWXoepA0A+tT/OTpM02/YGSZdopBrcREm32pakuyPiT0pLEuhwEXFOnZdOLzQRAECpcg2CbPdrZAB0VURcL0kR8dyo1/9R0k0tyRCoiDp/nF1WeCIAAABdznWWGPwmYOTj5yslvRARF45qn7e3pKjtz0t6W0ScfYB9bZH0ZO3pbEnPjyP3dsa5tcbhETGnpGN3jH362Wh8X3auIs+PfpZTRftaI3gf9v8e0Ndy2E8/O5CqfP9V4TzHc451+1meQdA7Jf0vSQ9LGq41f0nSOZJO0Mh0uPWSPjXqPgsHZHtlRCzJG99JODe0o27+2nXzuUndf37dhq/XCN4H3oMyVeW9r8J5tuocDzgdLiLulOTESzc3OxkAAAAAaLWGqsMBAAAAQKcrcxC0vMRjtxrnhnbUzV+7bj43qfvPr9vw9RrB+8B7UKaqvPdVOM+WnOMB1wQBAAAAQDdhOhwAAACASmEQBAAAAKBSCh8E2V5q+xe2n7B9cdHHbzbbl9vebPuRUW2zbN9qe23t/5ll5jhWthfavt32atuP2r6g1t4V51cV3dbn9mV7ve2Hba+yvbLsfMajm3+eVEW397c8uqlPNoL+2z6q0A/r/Y3WjWz32n7A9k3N3G+hgyDbvZK+Kel9khZLOsf24iJzaIErJC3dp+1iSbdFxNGSbqs970SDki6KiMWSTpH06drXq1vOr+t1aZ9LeXdEnNAF90q4Qt3786TrVai/5dEtfbIRV4j+W7oK9cN6f6N1owskrWn2Tou+EnSypCciYl1E7Jb0fUlnFZxDU0XEHZJe2Kf5LElX1h5fKelDhSbVJBGxMSLurz3erpFvwPnqkvOriK7rc92sm3+eVAT9rcLov22jEv1wP3+jdRXbCyR9QNK3m73vogdB8yU9Per5BnXhF0zS3IjYWHu8SdLcMpNpBtuLJJ0oaYW68Py6WBX6XEi6xfZ9tpeVnUwL0N86RxX6Wx7d3icbQf8tXuX64T5/o3Wbr0v6gqThZu+4r9k7xOtFRNju6DrktqdKuk7ShRGxzfZrr3XD+aHjvTMinrF9iKRbbT9W+0S269Df0CEq0ycbQf9FK+z7N1rZ+TST7TMlbY6I+2yf1uz9F30l6BlJC0c9X1Br6zbP2Z4nSbX/N5ecz5jZ7tdI57oqIq6vNXfN+VVA1/e5iHim9v9mSTdoZCpEN6G/dY6u7295VKBPNoL+W7zK9MM6f6N1k1MlfdD2eo1Ma3yP7e82a+dFD4LulXS07SNsT5B0tqQbC86hCDdKOr/2+HxJPygxlzHzyCWfyyStiYivjXqpK86vIrq6z9meYnva3seSfk/SI/vfquPQ3zpHV/e3PCrSJxtB/y1eJfrhfv5G6xoR8cWIWBARizTydfxpRJzbrP0XOh0uIgZtf0bSjyX1Sro8Ih4tModms321pNMkzba9QdIlki6VdI3tj0t6UtJHy8twXE6VdJ6kh22vqrV9Sd1zfl2vG/vcPuZKuqE2RbNP0vci4kflpjR2Xf7zpOtVoL/l0VV9shH03/ZQoX6Y/BstIm4uMaeO4gimpwIAAACojsJvlgoAAAAAZWIQBAAAAKBSGAQBAAAAqBQGQQAAAAAqhUEQAAAAgEphEAQAAACgUhgEAQAAAKgUBkEAAAAAKoVBEAAAAIBKYRAEAAAAoFIYBAEAAACoFAZBAAAAACqlbzwb214q6RuSeiV9OyIu3V/8BE+MAU0ZzyHTefSkx3JD0weyscN19rHtldzH27Uwew69u9KxfZt35N5vPXvmZo8XdYavEzZmj+eJE5KxsWv3uPIq2na9+HxEzCk7j3bXqn62a0H+fU7YFrljdx/k3LFzpm3LHbttdW/uWEnaPS//+U18YU/u2MFp/blj+7bm/zkUkf89bgT9LL9W9TWUK/U7/kD+//buPsquur73+OczZyaZZPJEDISQBJJAQHkSb9JWbQAAIABJREFUvAFRvJQHaSNwRXt7vXDVZVuvaVe1hdYuRe/qpb3t6qJLa6VdXV5TQdKKqBdxSakVKIJoQTAJKQSCJAYCCXkkhISQh3n43j/OiSazfyezz8ychzn7/VorK2e+53v2+Z4z85s5v73377vPOmZ7zY9Z8eQBxloOjLP2cerZ+f/GjZajjbNhT4JslyT9vaTLJW2U9FPbd0fEM9Ue060evd2XDfcpq+qYmB4cey8+IxMr7U/Pgsbduzz386391NszsSnr0h+4Zv7tI7m3W82WD74zE+udlM6d++fZ5yudOD+Z27/u+RHV1Wj/FnduaHYNY0G9xtnP/+iC3Llz7+vLnbvhyvyTld+55Ae5c39wVm1/NF/6WHacVTPvjs25c7dfdHzu3BlffyJ37sD+/blza8E4y69eYw3NlfobP5THf+PLNT+mNGsdYy0Hxln7uPfeVQ1/zqONs5GcDne+pHURsT4iDkr6hqSrR7A9AFXYXmz7Z7bX2b6h2fUAAACMZSOZBM2W9NJhX2+sxI5ge4nt5baX96rKOWMAqjrsqOt7JJ0u6Vrbpze3KqD9sLMBAIqj7o0RImJpRCyKiEVdGl/vpwPaEUddgTpjZwMAFMtIJkGbJM097Os5lRiA0TXkUVeOuAIjxs4GACiQkXSH+6mkhbbnqzz5uUbS/xiVqmo0cMaCZPzli7JzvFM++x/J3Fr6LL3577JdYH7lrqeSuf808fJMbPZNtTVLOP5vsvk7lrwj9+NraYCw87fT200t2K7XwmzULiKWSloqSVM8vT5tw4D2ltrZkFkhb3uJpCWS1K2JjakMADDqhj0Jiog+25+QdK/KLbJvjYinR60yAIdw1BVoEexwAID2MKLrBEXE9yR9b5RqAZDWMkddgTbGzgYAKJC6N0YAMDIR0Sfp0FHXNZK+xVFXYNT9YmeD7XEq72y4u8k1AQDqZERHggA0BkddgfriFG8AKJYxNwnqmDw5E3t1YfrK8N3bsge64sDIO2f1r12fif3zn12azPUHd2ViL/xFuvnAgr9MN20YeOONTOzYr65I5pYWzMvE+ta/kM6dNjW73R9vS+b2n70wE3PvQDI3nuBzw1gR73xr7lz3OnfuuHuX585deG/uVH3tO+flzj34F9mf76OZ/fDB/Nuec0zu3GpjKmXv5Wfnzu3+58dz5yIfdjYA9Wd7saSbVd7Z8JWIuKnJJaGgOB0OAAAAdcf1uNBKxtyRIAAAgHpZ+AeP1fyYK/72/cN4ps8N4zFj3i+uxyVJtg9dj+uZplaFQuJIEAAAABphyIt/S1wAHI3BJAgAAAAtIyKWRsSiiFjUpfHNLgdtikkQAAAAGoHrcaFljLk1Qf1nLcjEXjkr3bVq4V+uzj5+1Csq67kzfQ5xz53Z2P7/cn4yd98lZyTjHb3Zi5JPeGpjMvfg7GzXqjhxejK38/nt2dwt2ZgklQbelIntfcuxydzuJ5JhAABQbFz8Gy1jzE2CAAAAMPZwPS60EiZBAAAAaAiux4VWwZogAAAAAIXCJAgAAABAoYy50+EOThuXiXXM35vM7d+9e0TPtf+qdAOD7nsez72N0qknZ2JvzCglc6d/9dHc2+0/76xkfNxLr2RifS+8mMztS8Q652Ta9Zdz1z2fiU2YPDGZm23jgFa18/T09zCl2jgbqZfuPDN37rxP78udO2NZbaeZ/+Tk+blz599cw0+5041bUiZszP8ed5x2Su7c/p+ty50LAEARcCQIAAAAQKEwCQJanO25th+0/Yztp21f1+yaAAAAxrIxdzocUEB9kj4ZESttT5a0wvb9EfFMswsDAAAYi5gEAS0uIjZL2ly5vcf2GkmzJTEJAoBWsGtka5ABNN6IJkG2X5C0R1K/pL6IWDQaRQFIsz1P0rmSHmtuJQAAAGPXaBwJuiQidozCdo5QmjY1GX/9hGzJB18b7Wcv2/6bbyTjc+/Jv43+536eib1l2fZk7tav5t/u+j9KL+eaf/O0bLBKd7iU6JmQO9e9/cl4qmsV3alGzvYkSd+WdH1E7B503xJJSySpW/k7vgEAABQRp8MBY4DtLpUnQLdHxF2D74+IpZKWStIUT6dLOVAj23Ml/aOkmSp3+l8aETc3tyoAaB+/dsI5TXjW6jvhR9odLiTdZ3tFZU90hu0ltpfbXt6rAyN8OqB4bFvSLZLWRMQXml0P0KYONSA5XdIFkj5u+/Qm1wQAqJORToLeFRFvk/Qelf9gXDQ4ISKWRsSiiFjUpfEjfDqgkC6U9GFJl9peVfl3RbOLAtpJRGyOiJWV23skHWpAAgBoQyM6HS4iNlX+32b7O5LOl/TwaBQGoCwifizJza4DKAoakABA+xv2JMh2j6SOSsveHkm/Kun/jFpls49Phne9JbvcYd638y+BKM08Lhnv37otE9u3uzuZGxfmP6fR/74qE9v4JwuTuV1akYynau47WEo/4U9WZh+faFQgpZsVVGtg0NHTk4kNrH42mTvwruz70/GzZCqa7LVL9+XOnXPruNy51cZZSrVxlvL6W/I3fdj/J4kmIUfR9+H8v0d2vDX/Ue0ZX34yd25qnFXTe27690hyu4y/3I7WgKRyP01IAKANjORI0ExJ3ykvV1CnpK9HxPdHpSoAABpsqAYkEk1IAKBdDHsSFBHrJb11FGsBAKApaEACAMUy0sYIAAC0AxqQAECBcJ0gAEDh0YAEAIqFI0EAAAAACqVljwT1Tk933fGs/ZnYxKe3J3P7zzsrG/vpU/mLGEiHO199IxN7fWG6E9WERKzr39Jd4KpZ//GTM7HZ361SXMLWi49NxmdU6QSXMrB3b+5cs1QYAFAgA3teb3YJAGrEkSAAAAA0hO1bbW+zvbrZtaDYmAQBAACgUW6TtLjZRQBMggAAANAQEfGwpJ3NrgNo2TVBAAAAKB7bSyQtkaRupdeIAyPVspOgga70QaoFM7dmYn0bN6W3Mf+4TKyWQ1890/cl4/3PPJeJTXgmvY1Nn35nJjb7rx6poQrpyisfy8RW/+/8jRHe+3s/TMYf+fK43Nsozcy+lwNz0g0X9O+rcm8XdWDJnfmG9oKZO3JvdvMZJ+bOnXXvtty5H130Yu7cW/ZflDv31N+rrQHJJ764O3fuH12+PnfuFT/6QO7cPW8+Jnduz4b8C7HpVQJgLImIpZKWStIUT+dXGOqC0+EAAAAAFAqTIAAAAACFwiQIGANsl2w/YfueZtcCAMBw2b5D0qOSTrO90fZHm10Tiqll1wQBOMJ1ktZImtLsQgAAGK6IuLbZNQASR4KAlmd7jqQrJX2l2bUAAAC0g5Y4EpTqZPX67HTnsv17Jmdix1fZ7tbzJmRis36Uv65rTkl3l/qRunNvY+9JffmfsIrZ43dlYqtrOCBw47Hp1nW/pnMysdLppyZzU12rJj+8Lpnbn4j5P52RzI0VTyfjOMIXJX1KUvaHv4J2ogAAAPlxJAhoYbavkrQtIo7a7zkilkbEoohY1OXxDaoOAABgbGqJI0EAqrpQ0nttXyGpW9IU21+LiA81uS4AQEUcONDsEgDUiCNBQAuLiM9ExJyImCfpGkk/YAIEAAAwMkyCAAAAABTKkKfD2b5V0qF1CWdWYtMlfVPSPEkvSPpARLw63CI6pmYX+e871snc13dlF31Xa4ywf0ZkYp3Hz0zm9m3Zmol94+uXJnPH/W52u8f+30eTubMfSL+OWtz6T4uz29UjydzOk+ZmYu9dW+0d2pKJeF/6kP6uk0uZWM+GKtvd8Uom9PLFU5Ops4660gWHi4iHJD00VJ7HjVPHSSfm2uazL2UbXlTTmRhP1fz88xfkzt309fz7YmavTbXdSEs1XDma1Dir5qEr1uevo8qYStlxdnacVfPa/PSYSmGcAQBwpDyfPm6TNPjTwQ2SHoiIhZIeqHwNAMCYxoWJAaAYhpwERcTDknYOCl8taVnl9jJJ7xvlugAAaIZDFyYGALSx4a4JmhkRmyu3t0hKn2Om8vVLbC+3vbxXdE8BALQmLkwMAMUx4sYIERGSqi4WOOL6JeL6JQCAlnXowsQD1RLYsQcA7WG4k6CttmdJUuX/baNXEgAAjTWsCxOzYw8AxqzhXiz1bkkfkXRT5f/vjqiK6dMyoTdOSO+I616b/49O155sZ7af/fH8ZO7Jf5ztDjfl+XQNfd35O745f0Otqt2sZv9VuhNcyo5fmZONvbQ/mXtKojtc3/Mbkrl9PbMzsTfm9CRzJyQ+QkzcWnXHKgC0Ai5MDAAFMuSRINt3SHpU0mm2N9r+qMqTn8ttr5X07srXAACMSVyYGACKZcgjQRFxbZW7LhvlWgAAAACg7oZ7OhwAAG0p74WJAQBj14i7wwEAAADAWNISR4IGpmUX2E885bVk7tRlU3Jvd97XX8rEXvxvc3M/vvfDg68RW9bxzTfl3sbWD+zLxObflfvhkqT+i9+WiZUeWpnMPeab2fi2/3x27ufqnHV8Mt47KdvYoOffnk7mplogTFmffR9QH1EqJcdUypxZ6Z/xlIk3T8ydu+myqblzuy/ekTv3jF/PjulqNtTYruWNM9INRFKeei7/75GT5/XnL+KMPblTx907Kf92AQDAETgSBAAAAKBQmAQBAAAAKBQmQcAYYHua7TttP2t7je13NLsmAABqYXuu7QdtP2P7advXNbsmFFdLrAkCMKSbJX0/In7D9jhJ+RfoAADQGvokfTIiVtqeLGmF7fsj4plmF4bi4UgQ0OJsT5V0kaRbJCkiDkbEruZWBQBAbSJic0SsrNzeI2mNpNnNrQpF1bJHgqZNSHdq6t42PhPzuWckc/ueyHYvm/Pl9GfHVEezV3ZMThd3XjZ72j+lUy9ZsDYTeyGdqhiIZHz9b2djCx+qso0DBzKxSc91VXnGrANvPiEZ7zppbyY2sDcbq2bn6ekDF296JPcmimy+pO2Svmr7rZJWSLouIn7xDbC9RNISSeoel78zGwAAzWB7nqRzJT2WuO+Xf9M48QF1wpEgoPV1SnqbpC9FxLmS9kq64fCEiFgaEYsiYlFXZ7722AAANIPtSZK+Len6iNg9+P4j/qYpu/MbGA1MgoDWt1HSxog4tLfsTpUnRQAAjCm2u1SeAN0eETVeOREYPUyCgBYXEVskvWT7tEroMkksIgUAjCm2rfL61jUR8YVm14Nia9k1QQCO8PuSbq90hlsv6beaXA8AALW6UNKHJT1le1Ul9tmI+F4Ta0JBMQkCxoCIWCVpUbPrAABguCLix5Lc7DoAqUUmQX092e5l+/vSpfWsynZb27P4zHTuE9nYwJ49+Qt7PV3DjJX5zyK8/6dnZ2ILs41QJEkHFqeXeZReztZx4Mrzkrnj/+WnmdgJn8vfgq20ry8Z3//KhNzbSJn282zXOtRHdFn7Zub7fg1E/g5/ry+YlDv3hM/n/5l7bsH5uXPv33BM7txpH6vtbN/Sy/lzF9zwaP7kC7K/A6qpZZzNZkwBADBsrAkCAAAAUChMggAAAAAUCpMgAAAAAIXCJAgAAEm2p9m+0/azttfYfkezawIA1MeQjRFs3yrpKknbIuLMSuxPJX1M0vZK2ojaG/ZOypYxrtSfzB3Ym13I7YH0dg+8J9s8YPy/ZhsHVBMdkYy/ck42Xm259sKPp5sgpOw+Kf3tmPNgbybW11PKvd1adDy5Lhk/8Z5s84nXPnhBMnfq7T/JxMY/tyWZm27DAABNcbOk70fEb1Ta0U9sdkEAgPrIcyToNkmLE/G/iYhzKv/o7w4AGLNsT5V0kcoXclREHIyIXc2tCgBQL0NOgiLiYUk7G1ALAADNMl/lsxu+avsJ21+x3TM4yfYS28ttL+8VbcoBYKwayZqgT9h+0vattqtevIM/GACAMaBT0tskfSkizpW0V9INg5MiYmlELIqIRV0a3+gaAQCjZLiToC9JOlnSOZI2S/rraon8wQAAjAEbJW2MiEMLOe9UeVIEAGhDw5oERcTWiOiPiAFJ/yAp/yXfAQBoMRGxRdJLtk+rhC6T9EwTSwIA1NGQ3eFSbM+KiM2VL98vafVIiohhVfFLA1Uev29O9o5ajkUd+1i6A9sl1z+aia3qSOe6K1tDHEifFjhlQ7pX2sTVL2eD47qSuX2pOgbSnfZKb1mYifWvWZuu4YdrMrHO809N5nYumJcN9tIHrlGiZPVOyrd/o5a9IPun58/OLKQ4ig+84/Hcud96NP/+lmrjqZqJ2/O/vpf/+J25c+f+y/ahkyoW/L/0uE4ZvyH/Uk1GX26/L+n2Sme49ZJ+q8n1AADqJE+L7DskXSxphu2Nkm6UdLHtcySFpBck/U4dawQAoO4iYpWkRc2uAwBQf0NOgiLi2kT4ljrUAqAK238o6X+qvOPhKUm/FRH7m1sVAADA2DSS7nAAGsD2bEl/IGlR5YLFJUnXNLcqAACAsYtJEDA2dEqaYLtT5avYJxaKAQAAII8RtiQYHX3d2blYb1+6tNSC6wk7epO5Hsi/yDhlxt3PJuM/7LsgE+v4QHobmy8ayMTe/OlskwFJmvjcjmR8YMbUTGzPwinJ3Nf+6+xM7ITPP5LM9au7M7HSlPR2+3dnc7vXbUvmpsTBg7lzcaSI2GT785JelLRP0n0Rcd/hObaXSFoiSeMmVr1sFwAAAMSRIKDlVS5GfLXKV7Q/QVKP7Q8dnnPE9bi6a+nNBgAAUDxMgoDW925Jz0fE9ojolXSXpPw9mgEAAHAEJkFA63tR0gW2J9q2yhdxTJ9TCQAAgCExCQJaXEQ8JulOSStVbo/dIWlpU4sCAKBGtrttP277P2w/bfvPml0TiqslGiMAOLqIuFHlCxUDADBWHZB0aUS8brtL0o9t/2tE/KTZhaF4GjoJsq2O7u5M/OBkZ2Kv7Z6Y3Mb0RKyvu5TMnfLk9kysv0ptnXOyXdX6Nm5K5k69Pf9Ynbj1bZnYwfNOTdfwgxW5t9sz/qxkfP/USbm30bdlayZWmnlcOjnRHe7gvBnJ1I4fPpG7Boy+gY70mEp5bWe6G2DKCZurjZ6szgXzcueu/vXInXvyvL7cueF878Eh43fkv/ZsLeNMO1/LnTpwYrYTZNXNvv343LlT1r+QOxcA6iUiQtLrlS+7Kv/y/xEARhGnwwEAAKAhbJdsr5K0TdL9lVO+B+cssb3c9vJeHWh8kSgEJkEAAABoiIjoj4hzJM2RdL7tMxM5v7zsg8Y3vkgUApMgAAAANFRE7JL0oKTFza4FxcQkCAAAAHVn+1jb0yq3J0i6XNKzza0KRdXQxgghKfoHMvGO3mzuwEB6UXPnrOxi4P2ldG7/2vX5ixvXlTu1c/5JmVjf8xuSuaUHV2Zjp8xP5uZfdi7p8aeS4Tc9XstGsgZ2pRdxJxtaVGlIsf/aCzKxKXfQ+AUAgIKbJWmZ7ZLKO+K/FRH3NLkmFBQtsgEAAFB3EfGkpHObXQcgcTocAAAAgIJhEgQAgCTbf1i5iv1q23fYzp4HDABoC0yCAACFZ3u2pD+QtCgizpRUknRNc6sCANTLkJMg23NtP2j7mcoesusq8em277e9tvL/MfUvFwCAuumUNMF2p6SJkl5ucj0AgDrJ0xihT9InI2Kl7cmSVti+X9JvSnogIm6yfYOkGyR9+qhbilD0J3qgRTY0sDvdrS0Gst3lep57JZmb6rbWcc7p6dq27kzHUzXs2p2JpbrWSdLAcdm54UBXuqtaLdw1LhnvmNSTifW/+mp6Ix3ZOkrT03PZvs1bMrFx9y5P5s5+ZFomtv2OdAkYfVGSDk5Jd0wcrG93+ucopdo4S9m1KD0eUqau2p47t2vnvty5Ma62vi+l7enOiCkz76syplI68h9wrzamUt66Iv/rW8P4G1JEbLL9eUkvSton6b6IuG9wnu0lkpZIUrcmNrZIAMCoGfKvc0RsjoiVldt7JK2RNFvS1ZKWVdKWSXpfvYoEAKCeKmczXC1pvqQTJPXY/tDgPK5kDwDtoaY1Qbbnqdza8DFJMyNic+WuLZJmVnnMEtvLbS/v1YERlAq0N9u32t5me/VhMU47BRrj3ZKej4jtEdEr6S5J72xyTQCAOsk9CbI9SdK3JV0fEUecDxYRoeRJbew1A2pwm6TFg2I3qHza6UJJD1S+BjD6XpR0ge2Jti3pMpXPfAAAtKFckyDbXSpPgG6PiLsq4a22Z1XunyVpW31KBIohIh6WNHhxGqedAg0QEY9JulPSSklPqfz3cWlTiwIA1M2QK2sre8RukbQmIr5w2F13S/qIpJsq/393yG11llSaNjUT3z8jsYi7yvSsf2tirpWKVbH71CnJ+KRVz2RipVNPTtfw3M9zP19pyqRMrNYF28ntzq6y8Lw/2zhCVRojuJRtjBB9qXYStZkx/vVMLP/SdwyS+7RTVRZrd03mjDlgOCLiRkk3NrsOAED95TkSdKGkD0u61Paqyr8rVJ78XG57rcrnUt9UxzqBwst72mlpYrZDIAAAAH5pyEMSEfFjSdX67V42uuUAGGSr7VkRsZnTTgEAAEZHTd3hADTcodNOpZynnQIAAODomAQBLcL2HZIelXSa7Y22PypOOwUAABh1I1+hD2BURMS1Ve7itFMAAIBR1NhJUEjqz3Yfi2yTMqmUXP+t0inzM7H+dc/nLuHV09IHv7I93Kp3ges4+82ZWDyzLv2Er2Q7s7kr/banX3Fa3wsv1pCd5u7sdZsGqnWSG5/NjQPpi98e17UnE1t98YXJ3NJDK49WIobDVcZUSpVxlhKbtuTO3fqx43LnTnm2K3euXG15YtbAuLxvQlnH3n35t71799BJFR43LnduaWb+9+24rg25c9doQu5cAACKgNPhAAAAABQKkyAAAAAAhcIkCAAAAEChMAkCAABAw9gu2X7C9j3NrgXF1RLd4dyXjXXurFJaKf9i546enkzs4JRa2g+ked/B7HPNOj6ZO/CmKZlYf096oXT+Jd9S6dhjk/H+7duz2000NZCk2J9tbFBtEbcnZ1tH9G9NX7fzld7s+/7SZeka5j2UDAMAgPZ1naQ1krIfkoAG4UgQAAAAGsL2HElXSvpKs2tBsTEJAgAAQKN8UdKnJA1US7C9xPZy28t7lb4cBzBSTIIAAABQd7avkrQtIlYcLS8ilkbEoohY1KX06fTASLXEmiAAADD2uHN4HyN+/pfn1fyY7u21rJwtO+Fzj9T8GNTVhZLea/sKSd2Sptj+WkR8qMl1oYA4EgQAAIC6i4jPRMSciJgn6RpJP2AChGZpiSNBfROysf4JVU4VfeXV3Nt949IzMrGuvfn3JHVMnpyM969dn3sbA/POzdY1K31oN9t/rTpPmpiMd+zpztawf396Ix3ZTnvuTtc28MrO3LX983NnZWJz37Ep9+MxMmGpvytfrvfl3w/y6vuz39dqahlnB47LdhOspnSg6inkGbvnZcfC0bxpa/4R2NGXaGlZRZyY7hyZ4l2v584dqKmfJAAAOBxHggAAhWH7VtvbbK8+LDbd9v2211b+P6aZNQJFEBEPRcRVza4DxcUkCGgRVT6cfc72s7aftP0d29OaWSPQBm6TtHhQ7AZJD0TEQkkPVL4GALQxJkFA67hN2Q9n90s6MyLOlvScpM80uiignUTEw5IGn997taRlldvLJL2voUUBABqOSRDQIlIfziLivog4tADlJ5LmNLwwoP3NjIjNldtbJM2slsj1SwCgPQzZGMH2XEn/qPIfhZC0NCJutv2nkj4maXsl9bMR8b2jb0xSKbsYv2tvInVzNk+SBubNygZ3vJLMPTA1O8cb99pRKzzy8W8/NRmfsD77fH3rX0jmdm3elc2dl14oXTr22GS8f/v2TCxK6fnrwIHsH+XSMenT21MLtjteS3wzJCmyZ2H1bXgpmdq5OrvQ/ZL//lgy90eqbfF6wf22pG+m7rC9RNISSeqcwnIGYLgiImzHUe5fKmmpJE3x9Kp5AIDWlqc7XJ+kT0bEStuTJa2wfX/lvr+JiM/XrzwAkmT7f6k8Fm9P3X/4B7PuE+bywQyozVbbsyJis+1ZkrY1uyAAQH0NeTpcRGyOiJWV23skrZE0u96FASiz/ZuSrpL0wYhgggOMvrslfaRy+yOSvtvEWgAADVDTmiDb8ySdK+nQuU2fqHSturVaS9HDz58+OFDlejUAkmwvlvQpSe+NiDeaXQ8w1tm+Q9Kjkk6zvdH2RyXdJOly22slvbvyNQCgjeW+WKrtSZK+Len6iNht+0uS/lzldUJ/LumvVV6zcITDT9OZ2nUse7GBKiofzi6WNMP2Rkk3qtwNbryk+21L0k8i4nebViQwxkXEtVXuuqyhhQAAmirXJMh2l8oToNsj4i5Jioith93/D5LuqUuFQEFU+XB2S8MLAQAAaHMeaomBy7ufl0naGRHXHxafdailqO0/lPT2iLhmiG1tl7Sh8uUMSTtGUHsr47XVx0kRkW6hh18YNM4Ox8/l2NXI18c4y6mgY60WvA9Hfw8YazkcZZwNpSg/f0V4nSN5jVXHWZ5J0Lsk/UjSU5IGKuHPSrpW0jkqnw73gqTfOew6C0OyvTwiFuXNH0t4bWhF7fy9a+fXJrX/62s3fL/KeB94D5qpKO99EV5nvV7jkKfDRcSPVb7Cz2BHvyYQAAAAALSgmrrDAQAAAMBY18xJ0NImPne98drQitr5e9fOr01q/9fXbvh+lfE+8B40U1He+yK8zrq8xiHXBAEAAABAO+F0OAAAAACFwiQIAAAAQKE0fBJke7Htn9leZ/uGRj//aLN9q+1ttlcfFptu+37bayv/H9PMGofL9lzbD9p+xvbTtq+rxNvi9RVFu425wWy/YPsp26tsL292PSPRzr9PiqLdx1se7TQma8H4bR1FGIfVPqO1I9sl20/Yvmc0t9vQSZDtkqS/l/QeSadLutb26Y2soQ5uk7R4UOwGSQ9ExEJJD1S+Hov6JH0yIk6XdIGkj1e+X+3y+tpem465lEsi4pw2uFbCbWrf3ydtr0Aei8xvAAACsUlEQVTjLY92GZO1uE2M36Yr0Dis9hmtHV0nac1ob7TRR4LOl7QuItZHxEFJ35B0dYNrGFUR8bCknYPCV0taVrm9TNL7GlrUKImIzRGxsnJ7j8o/gLPVJq+vINpuzLWzdv59UhCMtwJj/LaMQozDo3xGayu250i6UtJXRnvbjZ4EzZb00mFfb1QbfsMkzYyIzZXbWyTNbGYxo8H2PEnnSnpMbfj62lgRxlxIus/2CttLml1MHTDexo4ijLc82n1M1oLx23iFG4eDPqO1my9K+pSkgdHecOdobxBHioiwPab7kNueJOnbkq6PiN22f3FfO7w+jHnviohNto+TdL/tZyt7ZNsO4w1jRGHGZC0Yv6iHwZ/Rml3PaLJ9laRtEbHC9sWjvf1GHwnaJGnuYV/PqcTazVbbsySp8v+2JtczbLa7VB5ct0fEXZVw27y+Amj7MRcRmyr/b5P0HZVPhWgnjLexo+3HWx4FGJO1YPw2XmHGYZXPaO3kQknvtf2Cyqc1Xmr7a6O18UZPgn4qaaHt+bbHSbpG0t0NrqER7pb0kcrtj0j6bhNrGTaXD/ncImlNRHzhsLva4vUVRFuPOds9ticfui3pVyWtPvqjxhzG29jR1uMtj4KMyVowfhuvEOPwKJ/R2kZEfCYi5kTEPJW/jz+IiA+N1vYbejpcRPTZ/oSkeyWVJN0aEU83sobRZvsOSRdLmmF7o6QbJd0k6Vu2Pyppg6QPNK/CEblQ0oclPWV7VSX2WbXP62t77TjmBpkp6TuVUzQ7JX09Ir7f3JKGr81/n7S9Aoy3PNpqTNaC8dsaCjQOk5/RIuJ7TaxpTHEEp6cCAAAAKI6GXywVAAAAAJqJSRAAAACAQmESBAAAAKBQmAQBAAAAKBQmQQAAAAAKhUkQAAAAgEJhEgQAAACgUP4/DYPsD9KDo8EAAAAASUVORK5CYII=\n", 385 | "text/plain": [ 386 | "
" 387 | ] 388 | }, 389 | "metadata": { 390 | "needs_background": "light" 391 | }, 392 | "output_type": "display_data" 393 | } 394 | ], 395 | "source": [ 396 | "fig, axs = plt.subplots(3, 4, figsize=(15, 9))\n", 397 | "first_im = 502 # sandal\n", 398 | "sec_im = 503 # sneaker\n", 399 | "third_im = 500 # pullover\n", 400 | "CONVOLUTION_NUMBER=10\n", 401 | "\n", 402 | "\n", 403 | "\n", 404 | "from tensorflow.keras import models\n", 405 | "layer_outputs = [layer.output for layer in model.layers]\n", 406 | "activation_model = keras.models.Model(inputs = model.input, outputs=layer_outputs)\n", 407 | "\n", 408 | "for x in range(0, 4):\n", 409 | " f1 = activation_model.predict(X_test_reshape[first_im].reshape(1, 28, 28, 1))[x]\n", 410 | " axs[0,x].imshow(f1[0, : , :, CONVOLUTION_NUMBER], cmap='viridis')\n", 411 | " axs[0,x].grid(False)\n", 412 | " f2 = activation_model.predict(X_test_reshape[sec_im].reshape(1, 28, 28, 1))[x]\n", 413 | " axs[1,x].imshow(f2[0, : , :, CONVOLUTION_NUMBER], cmap='viridis')\n", 414 | " axs[1,x].grid(False)\n", 415 | " f3 = activation_model.predict(X_test_reshape[third_im].reshape(1, 28, 28, 1))[x]\n", 416 | " axs[2,x].imshow(f3[0, : , :, CONVOLUTION_NUMBER], cmap='viridis')\n", 417 | " axs[2,x].grid(False)\n", 418 | "\n", 419 | "# plt.savefig('Understanding_conv10.png', dpi=300) " 420 | ] 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "metadata": {}, 425 | "source": [ 426 | "Let's see what's happening here. First we print out some of the test labels, each label represents different class. Keras api gives each conv, pool or dense etc as layers. So it's possible to get output of each layers with [layers.output](https://keras.io/layers/about-keras-layers/). Now we treat each item in the layer as separate model to see the outputs. So image (0, 0) shows the outout of first convolution. As you can see the dimension is $(26 \\times 26)$. Then pooling reduces the size to half (0, 1), then one more conv layer reduces the size to $(11 \\times 11)$. Finally the pooling layer reduces the size of the image to $(5 \\times 5)$. \n", 427 | "\n", 428 | "Change the Convolution number to see how different filters identify different features. " 429 | ] 430 | }, 431 | { 432 | "cell_type": "code", 433 | "execution_count": null, 434 | "metadata": {}, 435 | "outputs": [], 436 | "source": [] 437 | } 438 | ], 439 | "metadata": { 440 | "kernelspec": { 441 | "display_name": "Python 3", 442 | "language": "python", 443 | "name": "python3" 444 | }, 445 | "language_info": { 446 | "codemirror_mode": { 447 | "name": "ipython", 448 | "version": 3 449 | }, 450 | "file_extension": ".py", 451 | "mimetype": "text/x-python", 452 | "name": "python", 453 | "nbconvert_exporter": "python", 454 | "pygments_lexer": "ipython3", 455 | "version": "3.6.8" 456 | } 457 | }, 458 | "nbformat": 4, 459 | "nbformat_minor": 2 460 | } 461 | -------------------------------------------------------------------------------- /Conv_Basic_with_TensorFlow/MNIST_fashion_1hiddenLayer.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## MNIST Fashion Data and Multi-Class Classification " 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 9, 13 | "metadata": {}, 14 | "outputs": [ 15 | { 16 | "name": "stdout", 17 | "output_type": "stream", 18 | "text": [ 19 | "tensorflow version: 1.14.0\n" 20 | ] 21 | } 22 | ], 23 | "source": [ 24 | "import tensorflow as tf\n", 25 | "from tensorflow import keras\n", 26 | "print (\"tensorflow version: \", tf.__version__)\n", 27 | "import numpy as np\n", 28 | "import matplotlib.pyplot as plt\n", 29 | "import seaborn as sns" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "#### Load the MNIST Data from Keras API. \n", 37 | "Check [Keras Dataset](https://keras.io/datasets/) page for a short tutorial. \n", 38 | "Since we will use only tensorflow, check the [TensorFlow](https://www.tensorflow.org/tutorials/keras/basic_classification) page for loading and using the data. " 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 2, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "mnist_fashion = keras.datasets.fashion_mnist\n", 48 | "(X_train, y_train), (X_test, y_test) = mnist_fashion.load_data()" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "#### Study the Shape of the Images and Labels " 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": 7, 61 | "metadata": {}, 62 | "outputs": [ 63 | { 64 | "name": "stdout", 65 | "output_type": "stream", 66 | "text": [ 67 | "training data shape: (60000, 28, 28)\n", 68 | "training label shape: (60000,)\n", 69 | "test data shape: (10000, 28, 28)\n", 70 | "test label shape: (10000,)\n", 71 | "type of data: \n", 72 | "type of labels: \n" 73 | ] 74 | } 75 | ], 76 | "source": [ 77 | "print (\"training data shape: \", X_train.shape)\n", 78 | "print (\"training label shape: \", y_train.shape)\n", 79 | "print (\"test data shape: \", X_test.shape)\n", 80 | "print (\"test label shape: \", y_test.shape)\n", 81 | "print (\"type of data:\", type(X_train))sgd'\n", 82 | "print (\"type of labels:\", type(y_test))" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": 21, 88 | "metadata": {}, 89 | "outputs": [ 90 | { 91 | "data": { 92 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAt0AAAIZCAYAAAB6ToNQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfAElEQVR4nO3df/BldX3f8ddbFn8RFZANofwIpGFMSDoq2UFSUrWSAJJEaEcZnRoZS0ozQ6wmNqmaSVAM09ofMTE/nDJAshp/hGAMxDLqBo0maUUW8RegZUMkQEFWFzQGjUHf/eOedb/ifmW/m+/n3u9+9/GY+c733M85597PPcPsPjl77rnV3QEAAMZ5xKInAAAA653oBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGGzDoicw2mGHHdbHHnvsoqcBAMA6d8MNN3yuuzfubt26j+5jjz02W7duXfQ0AABY56rq9uXWubwEAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAabW3RX1cFVdWVVfaqqbqmqH66qQ6tqS1XdOv0+ZNq2quoNVbWtqj5eVScueZ5zp+1vrapz5zV/AADYW/M80/0bSd7d3d+X5MlJbknyiiTXdvfxSa6dHifJs5McP/2cn+SNSVJVhya5MMnTkpyU5MKdoQ4AAGvVXKK7qp6Q5OlJLkuS7v5qd9+f5Kwkm6fNNic5e1o+K8mbeuZDSQ6uqiOSnJ5kS3fv6O77kmxJcsY83gMAAOyteZ3pPi7J9iS/W1U3VtWlVXVQksO7++5pm3uSHD4tH5nkjiX73zmNLTcOAABr1oY5vs6JSV7S3ddV1W9k16UkSZLu7qrq1Xixqjo/s8tScswxxyy73Q/9wptW4+XWnBv+24tWvM/fXPTPBsxk8Y75lU+seJ9TfvOUATNZvL98yV+ueJ8PPP0ZA2ayWM/44Af2ar/fevmfrPJMFu9n/8dP7tV+F7/wuas8k8X7pd+/cq/2u+Xi963yTBbv+3/pWXu136tf/erVncgasLfv6Yo/PGl1J7IGnPO8D+/Vfk++8j2rPJPF+9hzT9+r/eZ1pvvOJHd293XT4yszi/DPTpeNZPp977T+riRHL9n/qGlsufFv0t2XdPem7t60cePGVX0jAACwUnOJ7u6+J8kdVfWkaejUJDcnuTrJzjuQnJvkqmn56iQvmu5icnKSL0yXobwnyWlVdcj0AcrTpjEAAFiz5nV5SZK8JMlbquqRSW5L8uLMov+Kqjovye1Jzpm2vSbJmUm2JXlg2jbdvaOqXpvk+mm7i7p7x/zeAgAArNzcoru7P5pk025WnbqbbTvJBcs8z+VJLl/d2QEAwDi+kRIAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYLC5RXdVfaaqPlFVH62qrdPYoVW1papunX4fMo1XVb2hqrZV1cer6sQlz3PutP2tVXXuvOYPAAB7a95nuv9ldz+luzdNj1+R5NruPj7JtdPjJHl2kuOnn/OTvDGZRXqSC5M8LclJSS7cGeoAALBWLfrykrOSbJ6WNyc5e8n4m3rmQ0kOrqojkpyeZEt37+ju+5JsSXLGvCcNAAArMc/o7iTvraobqur8aezw7r57Wr4nyeHT8pFJ7liy753T2HLjAACwZm2Y42v9SHffVVXfmWRLVX1q6cru7qrq1XihKerPT5JjjjlmNZ4SAAD22tzOdHf3XdPve5O8M7Nrsj87XTaS6fe90+Z3JTl6ye5HTWPLjT/0tS7p7k3dvWnjxo2r/VYAAGBF5hLdVXVQVT1u53KS05J8MsnVSXbegeTcJFdNy1cnedF0F5OTk3xhugzlPUlOq6pDpg9QnjaNAQDAmjWvy0sOT/LOqtr5mm/t7ndX1fVJrqiq85LcnuScaftrkpyZZFuSB5K8OEm6e0dVvTbJ9dN2F3X3jjm9BwAA2Ctzie7uvi3Jk3cz/vkkp+5mvJNcsMxzXZ7k8tWeIwAAjLLoWwYCAMC6J7oBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgc43uqjqgqm6sqndNj4+rquuqaltV/UFVPXIaf9T0eNu0/tglz/HKafzTVXX6POcPAAB7Y95nul+a5JYlj1+X5PXd/b1J7kty3jR+XpL7pvHXT9ulqk5I8vwkP5DkjCS/U1UHzGnuAACwV+YW3VV1VJIfT3Lp9LiSPCvJldMmm5OcPS2fNT3OtP7Uafuzkry9u/++u/86ybYkJ83nHQAAwN6Z55nuX0/yi0m+Pj1+YpL7u/vB6fGdSY6clo9MckeSTOu/MG3/jfHd7PMNVXV+VW2tqq3bt29f7fcBAAArMpforqqfSHJvd98wj9fr7ku6e1N3b9q4ceM8XhIAAJa1YU6vc0qS51TVmUkeneTxSX4jycFVtWE6m31Ukrum7e9KcnSSO6tqQ5InJPn8kvGdlu4DAABr0lzOdHf3K7v7qO4+NrMPQr6vu/9Nkvcnee602blJrpqWr54eZ1r/vu7uafz5091NjktyfJIPz+M9AADA3prXme7l/Kckb6+qX01yY5LLpvHLkry5qrYl2ZFZqKe7b6qqK5LcnOTBJBd099fmP20AANhzc4/u7v6zJH82Ld+W3dx9pLu/kuR5y+x/cZKLx80QAABWl2+kBACAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYbI+ju6qet8z4c1dvOgAAsP6s5Ez3ZcuMX7IaEwEAgPVqw8NtUFXfMy0+oqqOS1JLVn9Pkq+MmBgAAKwXDxvdSbYl6cxi+68esu6eJK9e5TkBAMC68rDR3d2PSJKq+kB3P2P8lAAAYH3Z42u6BTcAAOydPbm8JEkyXc99cZKnJPmOpeu6+5hVnhcAAKwbexzdSd6a2TXdL0/ywJjpAADA+rOS6P6BJKd099dHTQYAANajldyn+4NJnjpqIgAAsF6t5Ez3Z5K8u6remdmtAr+hu39lNScFAADryUqi+6Ak70pyYJKjx0wHAADWnz2O7u5+8ciJAADAerWSWwZ+z3Lruvu21ZkOAACsPyu5vGTp18Hv1NPvA1ZtRgAAsM6s5PKSb7rTSVV9V5ILk/z5ak8KAADWk5XcMvCbdPc9SV6W5D+v3nQAAGD92evonjwpyWNXYyIAALBereSDlH+eXddwJ7PY/oEkF632pAAAYD1ZyQcpL33I479L8rHuvnUV5wMAAOvOSj5IuXnkRAAAYL3a42u6q+rAqnpNVd1WVV+Zfr+mqh45coIAALCvW8nlJf81yUlJfibJ7Um+O8kvJ3l8kp9b/akBAMD6sJK7lzwvyXO6+73d/enufm+Sf5XknIfbsaoeXVUfrqqPVdVNVfWaafy4qrquqrZV1R/sPGteVY+aHm+b1h+75LleOY1/uqpOX8mbBQCARVhJdNcKx5f6+yTP6u4nJ3lKkjOq6uQkr0vy+u7+3iT3JTlv2v68JPdN46+ftktVnZDk+ZndNeWMJL9TVb4NEwCANW0l0f2HSf6kqk6vqu+vqjOS/PE0/m31zJemhwdOP53kWUmunMY3Jzl7Wj5repxp/alVVdP427v777v7rzP7avqTVvAeAABg7lYS3b+Y5E+T/HaSG5L8ZpL3JfmFPdm5qg6oqo8muTfJliR/leT+7n5w2uTOJEdOy0cmuSNJpvVfSPLEpeO72QcAANakh43uqjqlql7X3V/t7l/p7u/t7sd29/FJHpXkxD15oe7+Wnc/JclRmZ2d/r5/1My//ZzPr6qtVbV1+/bto14GAAD2yJ6c6X5Vkg8us+79SX5pJS/Y3fdP+/1wkoOraucdVI5Kcte0fFeSo5NkWv+EJJ9fOr6bfZa+xiXdvam7N23cuHEl0wMAgFW3J9H9lCTvXmbdnyb5oYd7gqraWFUHT8uPSfJjSW7JLL6fO212bpKrpuWrp8eZ1r+vu3saf/50d5Pjkhyf5MN78B4AAGBh9uQ+3Y9P8sgkX97NugOTPG4PnuOIJJunO408IskV3f2uqro5ydur6leT3Jjksmn7y5K8uaq2JdmR2R1L0t03VdUVSW5O8mCSC7r7a3vw+gAAsDB7Et2fSnJadp2FXuq0af231d0fT/LU3Yzflt3cfaS7v5LZfcF391wXJ7n44V4TAADWij2J7tcn+Z/TWeo/7u6vV9UjMru9328n+fmREwQAgH3dw0Z3d7+1qr4rs/tmP6qqPpfksMy+8ObC7n7b4DkCAMA+bU/OdKe7f62qLs3sjiNPzOxOIv+nu784cnIAALAe7FF0J8kU2O8ZOBcAAFiXVvKNlAAAwF4Q3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABptLdFfV0VX1/qq6uapuqqqXTuOHVtWWqrp1+n3INF5V9Yaq2lZVH6+qE5c817nT9rdW1bnzmD8AAPxjzOtM94NJXt7dJyQ5OckFVXVCklckuba7j09y7fQ4SZ6d5Pjp5/wkb0xmkZ7kwiRPS3JSkgt3hjoAAKxVc4nu7r67uz8yLf9tkluSHJnkrCSbp802Jzl7Wj4ryZt65kNJDq6qI5KcnmRLd+/o7vuSbElyxjzeAwAA7K25X9NdVccmeWqS65Ic3t13T6vuSXL4tHxkkjuW7HbnNLbc+ENf4/yq2lpVW7dv376q8wcAgJWaa3RX1XckeUeSl3X3F5eu6+5O0qvxOt19SXdv6u5NGzduXI2nBACAvTa36K6qAzML7rd09x9Nw5+dLhvJ9PveafyuJEcv2f2oaWy5cQAAWLPmdfeSSnJZklu6+9eWrLo6yc47kJyb5Kol4y+a7mJycpIvTJehvCfJaVV1yPQBytOmMQAAWLM2zOl1TknyU0k+UVUfncZeleS/JLmiqs5LcnuSc6Z11yQ5M8m2JA8keXGSdPeOqnptkuun7S7q7h3zeQsAALB35hLd3f0XSWqZ1afuZvtOcsEyz3V5kstXb3YAADCWb6QEAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhsLtFdVZdX1b1V9cklY4dW1ZaqunX6fcg0XlX1hqraVlUfr6oTl+xz7rT9rVV17jzmDgAA/1jzOtP9e0nOeMjYK5Jc293HJ7l2epwkz05y/PRzfpI3JrNIT3JhkqclOSnJhTtDHQAA1rK5RHd3fzDJjocMn5Vk87S8OcnZS8bf1DMfSnJwVR2R5PQkW7p7R3ffl2RLvjXkAQBgzVnkNd2Hd/fd0/I9SQ6flo9McseS7e6cxpYb/xZVdX5Vba2qrdu3b1/dWQMAwAqtiQ9Sdncn6VV8vku6e1N3b9q4ceNqPS0AAOyVRUb3Z6fLRjL9vncavyvJ0Uu2O2oaW24cAADWtEVG99VJdt6B5NwkVy0Zf9F0F5OTk3xhugzlPUlOq6pDpg9QnjaNAQDAmrZhHi9SVW9L8swkh1XVnZndheS/JLmiqs5LcnuSc6bNr0lyZpJtSR5I8uIk6e4dVfXaJNdP213U3Q/9cCYAAKw5c4nu7n7BMqtO3c22neSCZZ7n8iSXr+LUAABguDXxQUoAAFjPRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACDiW4AABhMdAMAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugEAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGAw0Q0AAIOJbgAAGEx0AwDAYKIbAAAGE90AADCY6AYAgMFENwAADCa6AQBgMNENAACD7ZPRXVVnVNWnq2pbVb1i0fMBAIBvZ5+L7qo6IMlvJ3l2khOSvKCqTljsrAAAYHn7XHQnOSnJtu6+rbu/muTtSc5a8JwAAGBZ+2J0H5nkjiWP75zGAABgTaruXvQcVqSqnpvkjO7+6enxTyV5Wnf/7JJtzk9y/vTwSUk+PfeJfqvDknxu0ZNYIxyLXRyLXRyLXRyLXRyLXRyLXRyLGcdhl7VyLL67uzfubsWGec9kFdyV5Oglj4+axr6huy9Jcsk8J/Vwqmprd29a9DzWAsdiF8diF8diF8diF8diF8diF8dixnHYZV84Fvvi5SXXJzm+qo6rqkcmeX6Sqxc8JwAAWNY+d6a7ux+sqp9N8p4kByS5vLtvWvC0AABgWftcdCdJd1+T5JpFz2OF1tTlLgvmWOziWOziWOziWOziWOziWOziWMw4Drus+WOxz32QEgAA9jX74jXdAACwTxHdg/nK+l2q6vKqureqPrnouSxSVR1dVe+vqpur6qaqeumi57QoVfXoqvpwVX1sOhavWfScFq2qDqiqG6vqXYueyyJV1Weq6hNV9dGq2rro+SxSVR1cVVdW1aeq6paq+uFFz2kRqupJ038PO3++WFUvW/S8FqWqfm76c/OTVfW2qnr0oue0KFX10uk43LSW/5tweclA01fW/98kP5bZl/hcn+QF3X3zQie2IFX19CRfSvKm7v7BRc9nUarqiCRHdPdHqupxSW5Icvb++N9FVVWSg7r7S1V1YJK/SPLS7v7Qgqe2MFX180k2JXl8d//EouezKFX1mSSbunst3Hd3oapqc5I/7+5Lp7t2Pba771/0vBZp+vv1rsy+p+P2Rc9n3qrqyMz+vDyhu79cVVckuaa7f2+xM5u/qvrBzL6d/KQkX03y7iQ/093bFjqx3XCmeyxfWb9Ed38wyY5Fz2PRuvvu7v7ItPy3SW7Jfvqtqj3zpenhgdPPfnsmoKqOSvLjSS5d9FxYG6rqCUmenuSyJOnur+7vwT05Nclf7Y/BvcSGJI+pqg1JHpvk/y14Povy/Umu6+4HuvvBJB9I8q8XPKfdEt1j+cp6vq2qOjbJU5Nct9iZLM50OcVHk9ybZEt377fHIsmvJ/nFJF9f9ETWgE7y3qq6YfqW4f3VcUm2J/nd6bKjS6vqoEVPag14fpK3LXoSi9LddyX570n+JsndSb7Q3e9d7KwW5pNJ/kVVPbGqHpvkzHzzlyiuGaIbFqSqviPJO5K8rLu/uOj5LEp3f627n5LZt8ueNP1T4X6nqn4iyb3dfcOi57JG/Eh3n5jk2UkumC5P2x9tSHJikjd291OT/F2S/f3zQY9M8pwkf7jouSxKVR2S2b+cH5fknyQ5qKpeuNhZLUZ335LkdUnem9mlJR9N8rWFTmoZonush/3KevZP0/XL70jylu7+o0XPZy2Y/sn8/UnOWPRcFuSUJM+ZrmV+e5JnVdXvL3ZKizOdyUt335vknZldrrc/ujPJnUv+BejKzCJ8f/bsJB/p7s8ueiIL9KNJ/rq7t3f3PyT5oyT/fMFzWpjuvqy7f6i7n57kvsw+T7fmiO6xfGU932L68OBlSW7p7l9b9HwWqao2VtXB0/JjMvvQ8acWO6vF6O5XdvdR3X1sZn9WvK+798szV1V10PQh40yXUpyW2T8h73e6+54kd1TVk6ahU5Psdx+6fogXZD++tGTyN0lOrqrHTn+nnJrZ54P2S1X1ndPvYzK7nvuti53R7u2T30i5r/CV9d+sqt6W5JlJDquqO5Nc2N2XLXZWC3FKkp9K8onpWuYkedX0Tav7myOSbJ7uRPCIJFd09359qzySJIcneeesJbIhyVu7+92LndJCvSTJW6aTN7clefGC57Mw0/+E/ViSf7/ouSxSd19XVVcm+UiSB5PcmH3gGxkHekdVPTHJPyS5YK1+2NgtAwEAYDCXlwAAwGCiGwAABhPdAAAwmOgGAIDBRDcAAAwmugH4hqr6s6r66XnvC7DeiW6AdaqqPlNVP7roeQAgugEAYDjRDbAfqapDqupdVbW9qu6blo96yGb/tKo+XFVfrKqrqurQJfufXFX/u6rur6qPVdUz5/sOAPZNohtg//KIJL+b5LuTHJPky0l+6yHbvCjJv01yRGZfMf2GJKmqI5P8ryS/muTQJP8xs69f3jiXmQPsw0Q3wH6kuz/f3e/o7ge6+2+TXJzkGQ/Z7M3d/cnu/rskv5zknKo6IMkLk1zT3dd099e7e0uSrUnOnOubANgHbVj0BACYn6p6bJLXJzkjySHT8OOq6oDu/tr0+I4lu9ye5MAkh2V2dvx5VfWTS9YfmOT9Y2cNsO8T3QD7l5cneVKSp3X3PVX1lCQ3Jqkl2xy9ZPmYJP+Q5HOZxfibu/vfzWuyAOuFy0sA1rcDq+rRO38yO7v95ST3Tx+QvHA3+7ywqk6YzopflOTK6Sz47yf5yao6vaoOmJ7zmbv5ICYADyG6Ada3azKL7J0/Byd5TGZnrj+U5N272efNSX4vyT1JHp3kPyRJd9+R5Kwkr0qyPbMz378Qf5cAPKzq7kXPAQAA1jVnJwAAYDDRDQAAg4luAAAYTHQDAMBgohsAAAYT3QAAMJjoBgCAwUQ3AAAMJroBAGCw/w/J3nwcPevPVQAAAABJRU5ErkJggg==\n", 93 | "text/plain": [ 94 | "
" 95 | ] 96 | }, 97 | "metadata": { 98 | "needs_background": "light" 99 | }, 100 | "output_type": "display_data" 101 | } 102 | ], 103 | "source": [ 104 | "#check the countplot\n", 105 | "fig = plt.figure(figsize=(12, 9))\n", 106 | "sns.countplot(y_train)\n", 107 | "plt.xlabel('Label', fontsize=12)\n", 108 | "plt.ylabel('Count', fontsize=12)\n", 109 | "plt.show()" 110 | ] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "metadata": {}, 115 | "source": [ 116 | "**Label and Class for the Fashion Dataset**\n", 117 | "\n", 118 | "|Label | Class|\n", 119 | "|---|---|\n", 120 | "|0| Top|\n", 121 | "|1| Trouser|\n", 122 | "|2| Pullover|\n", 123 | "|3| Dress|\n", 124 | "|4| Coat|\n", 125 | "|5| Sandal|\n", 126 | "|6| Shirt|\n", 127 | "|7| Sneaker|\n", 128 | "|8| Bag|\n", 129 | "|9| Ankle Boot|" 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": 24, 135 | "metadata": {}, 136 | "outputs": [], 137 | "source": [ 138 | "class_list=['Top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle Boot'] " 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": 20, 144 | "metadata": {}, 145 | "outputs": [ 146 | { 147 | "data": { 148 | "text/plain": [ 149 | "1000" 150 | ] 151 | }, 152 | "execution_count": 20, 153 | "metadata": {}, 154 | "output_type": "execute_result" 155 | } 156 | ], 157 | "source": [ 158 | "# labels are uniformly distributed\n", 159 | "list(y_test.flatten()).count(5) # change the number from 0 to 9 to see " 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "#### Let's Check one of the training images" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 26, 172 | "metadata": {}, 173 | "outputs": [ 174 | { 175 | "name": "stdout", 176 | "output_type": "stream", 177 | "text": [ 178 | "It's a: Bag\n" 179 | ] 180 | }, 181 | { 182 | "data": { 183 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAUO0lEQVR4nO3df2xc1ZUH8O+Z8fhH/CPEJBjHuOQHASWl2hTcQPmxywptlyJWoWwXEdRt0FakWjVqq+1KRUG7IK1aoRVtxR9VpXSJSNkWCgWWUNHSbESVkpQUJ01DAoWE/CAxie3gJHbiX+OZs3/4hTXB91xn3sy8Mff7kSzbc/xmjp/n+M288+69oqogoo+/VNIJEFF5sNiJAsFiJwoEi50oECx2okBUlfPBqqVGa1FfzocMnmTsP/HI7Br7DjyHA8nZ8eqeIWdM83l7YzpvwziDUR2RyWKxil1EbgHwCIA0gP9S1Yesn69FPa6Rm+M85MeTTPq3+X8x2qNVcy424wf+aYEZH2uwH7v6lJ17+w9fd8byAwPmtnT+tukmZ6zgl/EikgbwQwCfB7AEwAoRWVLo/RFRacV5z74MwD5V3a+qowCeBLC8OGkRUbHFKfY2AIcnfH8kuu1DRGSViHSKSGcWIzEejojiKPnZeFVdq6odqtqRgedkEBGVTJxi7wLQPuH7S6LbiKgCxSn21wAsEpH5IlIN4C4AG4qTFhEVW8GtN1UdE5HVAF7CeOttnaruKVpmIYk58vC9f73OGRvpOG1um3rLvu/WrXYj/ehn02Z84Ok5zljv9ivNbefd/3sz7iNV7qe3jo3Fuu/pKFafXVVfBPBikXIhohLi5bJEgWCxEwWCxU4UCBY7USBY7ESBYLETBaKs49mDFXMI67v/7u6jA8Bwq7tnfPldu+zHjmn+C4VvO/SC/fQ7/Au7D9/+xd1m3Oylp+zrA5D3DNSfhnhkJwoEi50oECx2okCw2IkCwWInCgSLnSgQbL2dFaM9lqqtNTfNDw+b8f4V15rxkYX29pev3GHGLVJjzx6kI56pxGK0sGb/3dvmpkMvzTfjB777WTM+f417iKxvim0dYeuNiKYpFjtRIFjsRIFgsRMFgsVOFAgWO1EgWOxEgWCffYqsfrSvj+6TuafbjF9+m732hjVAVjLV9ra+PrpPCYeC1v3tATN+5859ZvwPv7zKHdz6J3Nb737LjprxSsQjO1EgWOxEgWCxEwWCxU4UCBY7USBY7ESBYLETBYJ99rM80zmby/96etXvPGyPV9c3zDAuy9r95lRjozOWHxiw7zxJMadzfvbpG8346OpBZ2zhVvuhJW0fBzVrb1+JYhW7iBwEMAAgB2BMVTuKkRQRFV8xjux/rarHi3A/RFRCfM9OFIi4xa4AfiMi20Vk1WQ/ICKrRKRTRDqziHkdNhEVLO7L+BtUtUtELgKwUUT+rKqbJ/6Aqq4FsBYAmqTZPgtGRCUT68iuql3R5x4AzwFYVoykiKj4Ci52EakXkcazXwP4HAB7WU0iSkycl/EtAJ6T8fnWqwD8TFV/XZSsKlD+zJmCt73yM3affOQ2dz8YAPKe+9fhMM+FtH/HbpbP2XqBM9bruW/vHAXTcMnngotdVfcD+Isi5kJEJcTWG1EgWOxEgWCxEwWCxU4UCBY7USA+PkNcYyy5DCBWK6X/bnsI67s9dtvuEwOv24/tEWta47j7LQ5PeyrudM6dR9qdsaZ7FpvbznrMvdwzAEjK3m+qCe5XBx7ZiQLBYicKBIudKBAsdqJAsNiJAsFiJwoEi50oENOrz270wiVj/yo66ulFxxiSeOJ2u49e/ap7qucpmYbDKYvC08v2ye1vcMZ6/8p+Psx6zL5vHRsrIKNk8chOFAgWO1EgWOxEgWCxEwWCxU4UCBY7USBY7ESBmF59dqOfrCPJ9ZrXdjxuxv9l4z+XNgHfmHRLAuOqp8p7bYRHfZd7v/zbHU+b2z7aYs9RkOvuMeOpGTPMuGbdfXod86wHXeDfjEd2okCw2IkCwWInCgSLnSgQLHaiQLDYiQLBYicKxPTqsydIrv6kM/bw4YvtbeNeAhBnvLpvLLyvRa+eBaPFc7ywti9xj3+wzX3/tSm7hz941aVmvOZXdp89P2gvw50E75FdRNaJSI+I7J5wW7OIbBSRvdHnWaVNk4jimsrL+McA3HLObfcB2KSqiwBsir4nogrmLXZV3Qyg75yblwNYH329HsDtRc6LiIqs0PfsLap6NPr6GIAW1w+KyCoAqwCgFvb1wkRUOrHPxquqAnCeCVHVtaraoaodGdTEfTgiKlChxd4tIq0AEH22T00SUeIKLfYNAFZGX68E8Hxx0iGiUhH19DpF5AkANwGYDaAbwAMA/gfAUwA+AeAQgDtV9dyTeB/RJM16jdxccLJnfr3AGfvypa+a22475d4WAJY2HjbjG4+71/M+dMLuPFal7T559rezzfgl/73PjPvGVofq8P3XOWND8+wx47VHMnb8fc+Dey4hGDGeMnO3DJvbpl/e4Yxt003o175Jr57wnqBT1RWOUOFVS0Rlx8tliQLBYicKBIudKBAsdqJAsNiJAjGthrheNGPAGWtKDZnbXjfTbl/15erN+OKmY87Yl1p/b267bWChGW/68m4zPvilajOeMcbQPvvMjea28550/14AgOMnzLDU2Lmd/ox7qOi7t9oPfcUVXWb8nratZvyPxijT6xveNrfdPdRuxmdW2UNYL646ZcY/VXPUGfv7q+81t537shl24pGdKBAsdqJAsNiJAsFiJwoEi50oECx2okCw2IkCMa367GPqnhY55/m/9e7ohWb89Jg9i87JrHtKrZ/128v7zqiypy1+K+uc1Wv8sYfrzPjiWd3O2NdX2FMNpO62x2J2jdrDd60e/7hDzsjhYfu+e4cbzPimk0s8j+326unLzPhF1f1m/KVe99TiADCn5rQZP9TgHtY8PGRfu1AoHtmJAsFiJwoEi50oECx2okCw2IkCwWInCgSLnSgQ06rP3pRxT7G7MGNPp7xnsM2+7yp7+t4Fdb3OmK/XvH3AXv7X1+PPeKai3nJ4vjO2t2mOue3cenvcdfsMezz7sWyTGX9/xJ4nwDKat5+eJ0bt6w8urDnjjM2ssuc/uHGGPd69p8n+vX3XfcxIjThjY6fYZyeiGFjsRIFgsRMFgsVOFAgWO1EgWOxEgWCxEwViWvXZjw+7e7b9+Vpz26wxFh4Acnn7/97vTixyxgbH7L7o0Ji9/O8l9SfNeHVqzIzXVdnLD1u6hxrN+IL642Z8WeMBM/74SfdY/5p0vN/rfeP5AAD7T7jnMPhj+hJz219l7PHqzbXuHj4A7DpiX9dx52L3sst1R0pTlt4ju4isE5EeEdk94bYHRaRLRHZGH57p/okoaVN5Gf8YgFsmuf0Hqro0+nixuGkRUbF5i11VNwPoK0MuRFRCcU7QrRaRXdHLfOdkYiKySkQ6RaQzC/f1wERUWoUW+48ALASwFMBRAN9z/aCqrlXVDlXtyMAe8EFEpVNQsatqt6rmVDUP4McAlhU3LSIqtoKKXURaJ3z7BQD2msNElDhvQ09EngBwE4DZInIEwAMAbhKRpQAUwEEAXy1hjh/oPePuq16YtvueeRUz7hvffGXje86Ybzy7r8c/mLP79I1V9tuf4yPu+dX7s/b1Bymx543/84A9p/3+M+75zwF7LL5vLH192p5vv6XOPlaNNrqf3qdG7f2SV/u+PzXT/XwAgKpU3ox/t2WXM7Zll70OQaG8xa6qKya5+dES5EJEJcTLZYkCwWInCgSLnSgQLHaiQLDYiQIxrYa4nhpwL5t8RcZudVyYsVtzM6sGzfjpnLtV0zdmD7Uc8wyf9bVpZmXs3KxpsIdy9vDavqyde6Nnim3fFNyNxvTfF9fYyyKnYLcF3/fkXpd2D5FtrbHbfm019hTaJzx/81Mj9jTXp/LuVm9q1H4+FIpHdqJAsNiJAsFiJwoEi50oECx2okCw2IkCwWInCsS06rPjqLvX3ZCyhyzWpuxpidOwe5spccdnpOyhmCMxd3M2bw+RrbGmmrY3xUzYQ3tnea4/SBv7BQAa0u6pyHx99NM5e2ivb3iutV+svAD/8+HwsHMmNgD+6cMfPemeqrpuxyFzW3tAtRuP7ESBYLETBYLFThQIFjtRIFjsRIFgsRMFgsVOFIhp1Wev7iv8f5PVJweAwXzhq9U0pu0x3Rmxe66+qaZ9rF63b5pr3zUCLRl73Pewlu53M68fADArZV8D4Lu2wuL/m9rPp6Yae/vLao45Yy/1NpnbFopHdqJAsNiJAsFiJwoEi50oECx2okCw2IkCwWInCsS06rPP3uPuu24ZtvueviV4Gzx91cG8e1llXy/b1+P3ibMktO/39uV2QdruZZ8x9gsADKbdcV+P35d7DvYy3Nb2vv7/cN6+fmBuzUkzvq37UjP+VO8yI2rfd6G8R3YRaReRl0XkDRHZIyLfiG5vFpGNIrI3+myP5ieiRE3lZfwYgG+p6hIA1wL4mogsAXAfgE2qugjApuh7IqpQ3mJX1aOquiP6egDAmwDaACwHsD76sfUAbi9VkkQU33m9ZxeReQA+DWAbgBZVPRqFjgFocWyzCsAqAKiFe602IiqtKZ+NF5EGAM8A+KaqfmhFPlVVYPLZA1V1rap2qGpHBoUPNiGieKZU7CKSwXih/1RVn41u7haR1ijeCqCnNCkSUTF4X8aLiAB4FMCbqvr9CaENAFYCeCj6/HxJMpyg4ZV3nLHra+3/W3tHT5vxWrGHQ+bV3eaJM5QSAHKeFpOvTZQ3/mf7mn6+37sxZU817RviOjPt3v6CtL2M9kDOXvbYJ2fsl2qxh8+ezNlvOX1DYBfN6jXjW968zBm7HJ3mtoWaynv26wH8I4DXRWRndNsajBf5UyLyFQCHANxZkgyJqCi8xa6qrwDOqxduLm46RFQqvFyWKBAsdqJAsNiJAsFiJwoEi50oENNqiGvu+PvO2G+H7P9bF1fZUyK/M3qRGff1oy2jau9m3/LAviGuVj+5wXMNgK/XnfWs+ey7BsDqZ2c9+yXj6YVb1xcAcFzTOTWNKbuP7vu9b2p+y4zvOLL4vHOKi0d2okCw2IkCwWInCgSLnSgQLHaiQLDYiQLBYicKxLTqs1v+48BtZvw7C54z475etjWlsm9K44FsvHHZvn6z1fP1TQVdnxox474x5b79Zl1D4BsLP8OTm79P787Nd23DGbVnVfLl3pyy509o22z/bqXAIztRIFjsRIFgsRMFgsVOFAgWO1EgWOxEgWCxEwXiY9Nnr7nDXua2q9NeZNY3rtvqq3ZnZ5rb+nrRvj66b47yZnH3dH199Lh84/ytaxBmiJ2bb8lm35hya7+Perb1rQXgu37hkQP2xMt1m7ab8VLgkZ0oECx2okCw2IkCwWInCgSLnSgQLHaiQLDYiQIxlfXZ2wH8BEALxmfiXquqj4jIgwDuBXB2Ieo1qvpiqRL1yfX3m/Fvv3C3GX/piw+b8Z+futoZy6TsPnmN2GOnrXnfAeBM3h5bDbjjvjHfPr51yn29cN9Yf4uvj+6b2z0n7tzSnr/JYM4er/7JuvfMeN0DjWbcJJ59poVNiD+VZ8IYgG+p6g4RaQSwXUQ2RrEfqKpdJURUEaayPvtRAEejrwdE5E0AbaVOjIiK67zes4vIPACfBrAtumm1iOwSkXUiMun1qCKySkQ6RaQzi/JPxUNE46Zc7CLSAOAZAN9U1X4APwKwEMBSjB/5vzfZdqq6VlU7VLUjY7y3JKLSmlKxi0gG44X+U1V9FgBUtVtVc6qaB/BjAMtKlyYRxeUtdhERAI8CeFNVvz/h9tYJP/YFALuLnx4RFYuo5zS+iNwA4HcAXgc+mH93DYAVGH8JrwAOAvhqdDLPqUma9Rqxh/4lZdaWZjO+ps3dVezztKd8w0CvrbVbTFSYzUZnztfWm5seMOP/sONeM952xx4zXirbdBP6tW/S3t1Uzsa/AkzaLE2sp05E549X0BEFgsVOFAgWO1EgWOxEgWCxEwWCxU4UiI/NVNJxnbi+z4yvvv3rztipefZuzDbYj+2ZaRqeEbSwWsZa+AhTAIBnJGi8uGekpnjiqVE7XjXovgNPmx31x+w/Stsv/2DfQQXikZ0oECx2okCw2IkCwWInCgSLnSgQLHaiQLDYiQLhHc9e1AcT6QVwaMJNswEcL1sC56dSc6vUvADmVqhi5napqs6ZLFDWYv/Ig4t0qmpHYgkYKjW3Ss0LYG6FKldufBlPFAgWO1Egki72tQk/vqVSc6vUvADmVqiy5Jboe3YiKp+kj+xEVCYsdqJAJFLsInKLiLwlIvtE5L4kcnARkYMi8rqI7BSRzoRzWSciPSKye8JtzSKyUUT2Rp8nXWMvodweFJGuaN/tFJFbE8qtXUReFpE3RGSPiHwjuj3RfWfkVZb9Vvb37CKSBvA2gL8BcATAawBWqOobZU3EQUQOAuhQ1cQvwBCRvwRwGsBPVPXK6Lb/BNCnqg9F/yhnqeq3KyS3BwGcTnoZ72i1otaJy4wDuB3APUhw3xl53Yky7LckjuzLAOxT1f2qOgrgSQDLE8ij4qnqZgDnTqGzHMD66Ov1GH+ylJ0jt4qgqkdVdUf09QCAs8uMJ7rvjLzKIolibwNweML3R1BZ670rgN+IyHYRWZV0MpNombDM1jEALUkmMwnvMt7ldM4y4xWz7wpZ/jwunqD7qBtU9SoAnwfwtejlakXS8fdgldQ7ndIy3uUyyTLjH0hy3xW6/HlcSRR7F4D2Cd9fEt1WEVS1K/rcA+A5VN5S1N1nV9CNPvcknM8HKmkZ78mWGUcF7Lsklz9PothfA7BIROaLSDWAuwBsSCCPjxCR+ujECUSkHsDnUHlLUW8AsDL6eiWA5xPM5UMqZRlv1zLjSHjfJb78uaqW/QPArRg/I/8OgPuTyMGR1wIAf4o+9iSdG4AnMP6yLovxcxtfAXAhgE0A9gL4XwDNFZTb4xhf2nsXxgurNaHcbsD4S/RdAHZGH7cmve+MvMqy33i5LFEgeIKOKBAsdqJAsNiJAsFiJwoEi50oECx2okCw2IkC8X/vb0mvmNjdZAAAAABJRU5ErkJggg==\n", 184 | "text/plain": [ 185 | "
" 186 | ] 187 | }, 188 | "metadata": { 189 | "needs_background": "light" 190 | }, 191 | "output_type": "display_data" 192 | } 193 | ], 194 | "source": [ 195 | "plt.imshow(X_train[100])\n", 196 | "print (\"It's a: \", class_list[y_train[100]])" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "#### Normalize the Pixels\n", 204 | "Since the pixel values are number between 0 and 255, it is better if we normalize the values between 0 and 1 and it is considered as one of the most important image _pre-processing_ steps. " 205 | ] 206 | }, 207 | { 208 | "cell_type": "code", 209 | "execution_count": 27, 210 | "metadata": {}, 211 | "outputs": [], 212 | "source": [ 213 | "X_train = X_train/255.0\n", 214 | "X_test = X_test/255." 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "### Model Instance in Keras, Create the Neural Net" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": {}, 227 | "source": [ 228 | "In the ast simplest example possible, we saw a neural net with one layer, one node and one input, and that was good enough for simple linear regression tasks. But here the data set is huge and the problem is rather complex, so we create our first neural network with one hidden layer, the hidden layer is **Dense**, i.e., every output from previous layer is connected to all the nodes in the dense _(Fully Connected)_ layer. " 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": 29, 234 | "metadata": {}, 235 | "outputs": [], 236 | "source": [ 237 | "model = keras.models.Sequential([keras.layers.Flatten(), \n", 238 | " keras.layers.Dense(128, activation=tf.nn.relu), \n", 239 | " keras.layers.Dense(10, activation=tf.nn.softmax)])" 240 | ] 241 | }, 242 | { 243 | "cell_type": "markdown", 244 | "metadata": {}, 245 | "source": [ 246 | "Let's explain what's happening out there.
\n", 247 | "* First we flatten the input, i.e. the images. The images are of dimension $28 \\times 28$, thus instead of having 28 rows and columns, we will have 784 numbers stacked over each other.\n", 248 | "\n", 249 | "* Dense layer is the hidden layer in our neural net, thus each input is connected to 128 neurons linearly and then the relu activation function is used.\n", 250 | "\n", 251 | "* As this is a multi-class problem, we have used softmax layer in the output. It picks the highest number from the 10 outputs and assign the image to the corresponding label. Important point here is that, last layer contains 10 softmax units, as there are 10 classes in our dataset. These classes are independent of each other. " 252 | ] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "metadata": {}, 257 | "source": [ 258 | "### Compile the Model and Fit the Training Data" 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": 30, 264 | "metadata": {}, 265 | "outputs": [ 266 | { 267 | "name": "stdout", 268 | "output_type": "stream", 269 | "text": [ 270 | "Epoch 1/10\n", 271 | "60000/60000 [==============================] - 4s 63us/sample - loss: 0.4961 - acc: 0.8265\n", 272 | "Epoch 2/10\n", 273 | "60000/60000 [==============================] - 4s 61us/sample - loss: 0.3762 - acc: 0.8640\n", 274 | "Epoch 3/10\n", 275 | "60000/60000 [==============================] - 4s 60us/sample - loss: 0.3379 - acc: 0.8766\n", 276 | "Epoch 4/10\n", 277 | "60000/60000 [==============================] - 4s 60us/sample - loss: 0.3144 - acc: 0.8837\n", 278 | "Epoch 5/10\n", 279 | "60000/60000 [==============================] - 4s 60us/sample - loss: 0.2961 - acc: 0.8906\n", 280 | "Epoch 6/10\n", 281 | "60000/60000 [==============================] - 3s 58us/sample - loss: 0.2837 - acc: 0.8956\n", 282 | "Epoch 7/10\n", 283 | "60000/60000 [==============================] - 3s 57us/sample - loss: 0.2695 - acc: 0.9000\n", 284 | "Epoch 8/10\n", 285 | "60000/60000 [==============================] - 4s 60us/sample - loss: 0.2589 - acc: 0.9045\n", 286 | "Epoch 9/10\n", 287 | "60000/60000 [==============================] - 4s 60us/sample - loss: 0.2497 - acc: 0.9069\n", 288 | "Epoch 10/10\n", 289 | "60000/60000 [==============================] - 3s 57us/sample - loss: 0.2427 - acc: 0.9093\n" 290 | ] 291 | }, 292 | { 293 | "data": { 294 | "text/plain": [ 295 | "" 296 | ] 297 | }, 298 | "execution_count": 30, 299 | "metadata": {}, 300 | "output_type": "execute_result" 301 | } 302 | ], 303 | "source": [ 304 | "model.compile(optimizer=tf.train.AdamOptimizer(), \n", 305 | " loss='sparse_categorical_crossentropy', \n", 306 | " metrics=['accuracy'])\n", 307 | "\n", 308 | "\n", 309 | "# usually suggested that sparse_categorical_crossentropy to be used when targets are integers, \n", 310 | "# whereas if they are one hot encoded then use categorical_crossentropy \n", 311 | "\n", 312 | "model.fit(X_train, y_train, epochs=10)" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": {}, 318 | "source": [ 319 | "So the result suggests that our network is about 91% accurate, let's see how it performs on the test data" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": 32, 325 | "metadata": {}, 326 | "outputs": [ 327 | { 328 | "name": "stdout", 329 | "output_type": "stream", 330 | "text": [ 331 | "10000/10000 [==============================] - 0s 34us/sample - loss: 0.3507 - acc: 0.8759\n", 332 | "accuracy of the model on the test data: 0.8759\n" 333 | ] 334 | } 335 | ], 336 | "source": [ 337 | "test_res = model.evaluate(X_test, y_test)\n", 338 | "print (\"accuracy of the model on the test data: \", test_res[1])" 339 | ] 340 | }, 341 | { 342 | "cell_type": "markdown", 343 | "metadata": {}, 344 | "source": [ 345 | "______________________________________________________________________________________________________________" 346 | ] 347 | }, 348 | { 349 | "cell_type": "markdown", 350 | "metadata": {}, 351 | "source": [ 352 | "### Some Other Important Points" 353 | ] 354 | }, 355 | { 356 | "cell_type": "markdown", 357 | "metadata": {}, 358 | "source": [ 359 | "#### 1. Use Predict Method in Model Class\n", 360 | "\n", 361 | "\"Generates output predictions for the input samples.\"" 362 | ] 363 | }, 364 | { 365 | "cell_type": "code", 366 | "execution_count": 38, 367 | "metadata": {}, 368 | "outputs": [ 369 | { 370 | "name": "stdout", 371 | "output_type": "stream", 372 | "text": [ 373 | "1 , this is a: Trouser\n", 374 | "from the test label: Trouser\n", 375 | "Great Prediction\n" 376 | ] 377 | } 378 | ], 379 | "source": [ 380 | "predict=model.predict(X_test)\n", 381 | "# let's see one result \n", 382 | "print (np.argmax(predict[200]), \", this is a:\", class_list[np.argmax(predict[200])])\n", 383 | "print (\"from the test label: \", class_list[y_test[200]])\n", 384 | "if class_list[y_test[200]]==class_list[np.argmax(predict[200])]:\n", 385 | " print (\"!!! Great Prediction !!!\")\n", 386 | "else: \n", 387 | " print (\"you need to work on your model :(\")" 388 | ] 389 | }, 390 | { 391 | "cell_type": "markdown", 392 | "metadata": {}, 393 | "source": [ 394 | "#### 2. Use Callback to Stop Training After a Given Accuracy " 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "execution_count": 40, 400 | "metadata": {}, 401 | "outputs": [ 402 | { 403 | "name": "stdout", 404 | "output_type": "stream", 405 | "text": [ 406 | "Epoch 1/10\n", 407 | "60000/60000 [==============================] - 10s 160us/sample - loss: 0.4707\n", 408 | "Epoch 2/10\n", 409 | "59936/60000 [============================>.] - ETA: 0s - loss: 0.3579\n", 410 | " Loss has gone down below 40%, !!Stopping!!\n", 411 | "60000/60000 [==============================] - 10s 160us/sample - loss: 0.3581\n" 412 | ] 413 | }, 414 | { 415 | "data": { 416 | "text/plain": [ 417 | "" 418 | ] 419 | }, 420 | "execution_count": 40, 421 | "metadata": {}, 422 | "output_type": "execute_result" 423 | } 424 | ], 425 | "source": [ 426 | "class trycallback(keras.callbacks.Callback):\n", 427 | " def on_epoch_end(self, epoch, logs={}):\n", 428 | " if(logs.get('loss')<0.4):\n", 429 | " print (\"\\n Loss has gone down below 40%, !!Stopping!!\")\n", 430 | " self.model.stop_training = True\n", 431 | "\n", 432 | " \n", 433 | "callbacks = trycallback()\n", 434 | "model1 = keras.models.Sequential([keras.layers.Flatten(), \n", 435 | " keras.layers.Dense(1024, activation=tf.nn.relu), \n", 436 | " keras.layers.Dense(10, activation = tf.nn.softmax)])\n", 437 | "\n", 438 | "\n", 439 | "model1.compile(optimizer='Adam', loss='sparse_categorical_crossentropy')\n", 440 | "model1.fit(X_train, y_train, epochs=10, callbacks=[callbacks])" 441 | ] 442 | }, 443 | { 444 | "cell_type": "code", 445 | "execution_count": null, 446 | "metadata": {}, 447 | "outputs": [], 448 | "source": [] 449 | } 450 | ], 451 | "metadata": { 452 | "kernelspec": { 453 | "display_name": "Python 3", 454 | "language": "python", 455 | "name": "python3" 456 | }, 457 | "language_info": { 458 | "codemirror_mode": { 459 | "name": "ipython", 460 | "version": 3 461 | }, 462 | "file_extension": ".py", 463 | "mimetype": "text/x-python", 464 | "name": "python", 465 | "nbconvert_exporter": "python", 466 | "pygments_lexer": "ipython3", 467 | "version": "3.6.8" 468 | } 469 | }, 470 | "nbformat": 4, 471 | "nbformat_minor": 2 472 | } 473 | -------------------------------------------------------------------------------- /Conv_Basic_with_TensorFlow/Understanding_conv1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Conv_Basic_with_TensorFlow/Understanding_conv1.png -------------------------------------------------------------------------------- /Conv_Basic_with_TensorFlow/Understanding_conv10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Conv_Basic_with_TensorFlow/Understanding_conv10.png -------------------------------------------------------------------------------- /Conv_Basic_with_TensorFlow/Understanding_conv2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Conv_Basic_with_TensorFlow/Understanding_conv2.png -------------------------------------------------------------------------------- /Conv_Basic_with_TensorFlow/Understanding_conv5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Conv_Basic_with_TensorFlow/Understanding_conv5.png -------------------------------------------------------------------------------- /Conv_Basic_with_TensorFlow/convdemo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Conv_Basic_with_TensorFlow/convdemo.png -------------------------------------------------------------------------------- /Diffusion/L0_diffusion_Area.py: -------------------------------------------------------------------------------- 1 | ''' 2 | trying to figure out why Eq. 13 in 2006.11239 (stable diffusion) 3 | is only important when the network prediction is really good 4 | ''' 5 | 6 | 7 | import numpy as np 8 | import matplotlib.pyplot as plt 9 | from scipy.stats import norm 10 | 11 | # Given mean and variance 12 | mean_good = 88 / 255 13 | mean_bad = 93/255 14 | 15 | variance = 1e-4 16 | 17 | # Standard deviation (sigma) is the square root of the variance 18 | std_dev = np.sqrt(variance) 19 | 20 | # Define the range for x values 21 | x_g = np.linspace(mean_good - 3 * std_dev, mean_good + 3 * std_dev, 100) 22 | x_b = np.linspace(mean_bad - 3 * std_dev, mean_bad + 3 * std_dev, 100) 23 | 24 | # Calculate the corresponding probabilities (PDF) for each x 25 | def pdf(x, mean, std): 26 | pdf = (1 / (std * np.sqrt(2 * np.pi))) * np.exp(-(x - mean)**2 / (2 * std**2)) 27 | return pdf 28 | 29 | good_pdf = pdf(x_g, mean_good, std_dev) 30 | bad_pdf = pdf(x_b, mean_bad, std_dev) 31 | 32 | ### good preds 33 | x_l_t = 86/255 34 | x_h_t = 88/255 35 | 36 | 37 | 38 | # Calculate the cumulative distribution function (CDF) for x_l and x_h 39 | cdf_x_l_g = norm.cdf(x_l_t, mean_good, std_dev) 40 | cdf_x_h_g = norm.cdf(x_h_t, mean_good, std_dev) 41 | 42 | cdf_x_l_b = norm.cdf(x_l_t, mean_bad, std_dev) 43 | cdf_x_h_b = norm.cdf(x_h_t, mean_bad, std_dev) 44 | 45 | # Calculate the area under the curve between x_l and x_h 46 | area_under_curve_g = cdf_x_h_g - cdf_x_l_g 47 | area_under_curve_b = cdf_x_h_b - cdf_x_l_b 48 | 49 | print(f"Area under the curve between x_l and x_h for good prediction: {area_under_curve_g:.4f}") 50 | print(f"Area under the curve between x_l and x_h for bad prediction: {area_under_curve_b:.4f}") 51 | 52 | 53 | # Plot the normal distribution 54 | fig = plt.figure(figsize=(12, 5)) 55 | fig.add_subplot(121) 56 | plt.plot(x_g, good_pdf, label='Normal Distribution', color='blue') 57 | plt.fill_between(x_g, good_pdf, 0, where=((x_g >= x_l_t) & (x_g <= x_h_t)), color='black', alpha=0.5) 58 | 59 | # Add vertical lines at x_l and x_h 60 | 61 | plt.axvline(x_l_t, color='red', linestyle='--', label='x_l') 62 | plt.axvline(x_h_t, color='green', linestyle='--', label='x_h') 63 | 64 | plt.xlabel('x') 65 | plt.ylabel('Probability Density') 66 | plt.title('Mean = %3.5f and Variance = 1e-4: Area = %3.5f'%(mean_good, area_under_curve_g)) 67 | plt.legend() 68 | plt.grid(True) 69 | fig.add_subplot(122) 70 | plt.plot(x_b, bad_pdf, label='Normal Distribution', color='blue') 71 | plt.fill_between(x_b, bad_pdf, 0, where=((x_b >= x_l_t) & (x_b <= x_h_t)), color='black', alpha=0.5) 72 | 73 | # Add vertical lines at x_l and x_h 74 | 75 | plt.axvline(x_l_t, color='red', linestyle='--', label='x_l') 76 | plt.axvline(x_h_t, color='green', linestyle='--', label='x_h') 77 | 78 | plt.xlabel('x') 79 | plt.ylabel('Probability Density') 80 | plt.title('Mean = %3.5f and Variance = 1e-4: Area = %3.5f'%(mean_bad, area_under_curve_b)) 81 | plt.legend() 82 | plt.grid(True) 83 | plt.tight_layout() 84 | plt.show() 85 | 86 | -------------------------------------------------------------------------------- /Multi_Class/Prediction_RSP.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Multi_Class/Prediction_RSP.png -------------------------------------------------------------------------------- /Multi_Class/rps_visual_conv2d_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Multi_Class/rps_visual_conv2d_1.png -------------------------------------------------------------------------------- /Multi_Class/rps_visual_conv2d_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Multi_Class/rps_visual_conv2d_2.png -------------------------------------------------------------------------------- /Multi_Class/rps_visual_conv2d_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Multi_Class/rps_visual_conv2d_3.png -------------------------------------------------------------------------------- /Multi_Class/rps_visual_max_pooling2d_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Multi_Class/rps_visual_max_pooling2d_3.png -------------------------------------------------------------------------------- /NLP_Embedding1/Embedding_Course3_W2.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Embedding_Course3_W2.ipynb","provenance":[{"file_id":"https://github.com/lmoroney/dlaicourse/blob/master/TensorFlow%20In%20Practice/Course%203%20-%20NLP/Course%203%20-%20Week%202%20-%20Lesson%201.ipynb","timestamp":1572335418819}],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"code","metadata":{"id":"P-AhVYeBWgQ3","colab_type":"code","outputId":"e62dd9dd-6753-46a1-b3d2-9b62fafccb0a","executionInfo":{"status":"ok","timestamp":1572485624446,"user_tz":-540,"elapsed":38844,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":555}},"source":["# NOTE: PLEASE MAKE SURE YOU ARE RUNNING THIS IN A PYTHON3 ENVIRONMENT\n","\n","# import tensorflow as tf\n","# print(tf.__version__)\n","\n","# This is needed for the iterator over the data\n","# But not necessary if you have TF 2.0 installed\n","!pip install tensorflow==2.0.0-beta0\n","\n","\n","# tf.enable_eager_execution()\n","\n","!pip install -q tensorflow-datasets"],"execution_count":1,"outputs":[{"output_type":"stream","text":["Collecting tensorflow==2.0.0-beta0\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/6c/19/0d0c7f240db7bcd6b83783b9a89a67f38584d100e23ad5ae93114be92232/tensorflow-2.0.0b0-cp36-cp36m-manylinux1_x86_64.whl (87.9MB)\n","\u001b[K |████████████████████████████████| 87.9MB 114kB/s \n","\u001b[?25hRequirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.2.2)\n","Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.12.0)\n","Collecting tb-nightly<1.14.0a20190604,>=1.14.0a20190603\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/a4/96/571b875cd81dda9d5dfa1422a4f9d749e67c0a8d4f4f0b33a4e5f5f35e27/tb_nightly-1.14.0a20190603-py3-none-any.whl (3.1MB)\n","\u001b[K |████████████████████████████████| 3.1MB 38.6MB/s \n","\u001b[?25hRequirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.15.0)\n","Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (3.10.0)\n","Collecting tf-estimator-nightly<1.14.0.dev2019060502,>=1.14.0.dev2019060501\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/32/dd/99c47dd007dcf10d63fd895611b063732646f23059c618a373e85019eb0e/tf_estimator_nightly-1.14.0.dev2019060501-py2.py3-none-any.whl (496kB)\n","\u001b[K |████████████████████████████████| 501kB 30.7MB/s \n","\u001b[?25hRequirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.1.7)\n","Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.1.0)\n","Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.8.1)\n","Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.8.0)\n","Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.11.2)\n","Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.33.6)\n","Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.1.0)\n","Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.17.3)\n","Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.0.8)\n","Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow==2.0.0-beta0) (3.1.1)\n","Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow==2.0.0-beta0) (0.16.0)\n","Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow==2.0.0-beta0) (41.4.0)\n","Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.6->tensorflow==2.0.0-beta0) (2.8.0)\n","Installing collected packages: tb-nightly, tf-estimator-nightly, tensorflow\n"," Found existing installation: tensorflow 1.15.0\n"," Uninstalling tensorflow-1.15.0:\n"," Successfully uninstalled tensorflow-1.15.0\n","Successfully installed tb-nightly-1.14.0a20190603 tensorflow-2.0.0b0 tf-estimator-nightly-1.14.0.dev2019060501\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"_IoM4VFxWpMR","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"a3554714-db31-4445-923e-0105e3d71d7c","executionInfo":{"status":"ok","timestamp":1572485995704,"user_tz":-540,"elapsed":944,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["import tensorflow as tf\n","print(\"check tensorflow version: \", tf.__version__)\n","import tensorflow_datasets as tfds\n","imdb, info = tfds.load(\"imdb_reviews\", with_info=True, as_supervised=True)"],"execution_count":3,"outputs":[{"output_type":"stream","text":["check tensorflow version: 2.0.0-beta0\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"wHQ2Ko0zl7M4","colab_type":"code","outputId":"2a73c6d1-4a22-468b-f790-4c8fe2eb7dd8","executionInfo":{"status":"ok","timestamp":1572487063796,"user_tz":-540,"elapsed":18717,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":72}},"source":["import numpy as np\n","\n","train_data, test_data = imdb['train'], imdb['test']\n","\n","training_sentences = []\n","training_labels = []\n","\n","testing_sentences = []\n","testing_labels = []\n","\n","# str(s.tonumpy()) is needed in Python3 instead of just s.numpy()\n","for s,l in train_data:\n"," training_sentences.append(str(s.numpy()))\n"," training_labels.append(l.numpy())\n"," \n","for s,l in test_data:\n"," testing_sentences.append(str(s.numpy()))\n"," testing_labels.append(l.numpy())\n"," \n","print (\"example training sentences: \", training_sentences[403])\n","print (\"example corresponding training labels: \", training_labels[403]) # 1 is positive review\n","\n","training_labels_final = np.array(training_labels)\n","testing_labels_final = np.array(testing_labels)"],"execution_count":4,"outputs":[{"output_type":"stream","text":["example training sentences: b'Karl Jr and his dad are now running an army on a remote island. They capture a trio of guys who stumble upon the island. Whom after a while fight back. (well the survivors) This one has non-stop blood, gore and carnage, which would have been good if any of it looked remotely real, or if the production didn\\'t look like it was made with a weeks worth of saved up lunch money (I may be overexxagerating there. it was probably just a couple days worth). The horrendous dubbing didn\\'t bother me as much and I suspect if I had been really drunk, some of it MIGHT have been slightly humorous....maybe. But as it is, at merely 78 minutes the movie still felt way too long by.. Oh I don\\'t know... 78 minutes. Don\\'t waste your time.

My Grade: F

DVD Extras: Bonus movie: \"Zombie \\'90: Extreme Pestilence\"; and Trailers for other Shock-o-Rama released films'\n","example corresponding training labels: 0\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"7n15yyMdmoH1","colab_type":"code","outputId":"7729cbc6-ec9f-4f69-c099-e70f73dd6b00","executionInfo":{"status":"ok","timestamp":1572487199063,"user_tz":-540,"elapsed":12189,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":176}},"source":["vocab_size = 10000 # maximum number of words to keep, based on word frequency. Only the most common num_words-1 words will be kept.\n","embedding_dim = 16 # \n","max_length = 120 # maximum length of of review to be kept\n","trunc_type='post' # trucate review from the end beyond 120 words\n","oov_tok = \"\" # unknown \n","\n","\n","from tensorflow.keras.preprocessing.text import Tokenizer\n","from tensorflow.keras.preprocessing.sequence import pad_sequences\n","\n","tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\n","tokenizer.fit_on_texts(training_sentences) # fit on training sentences \n","\n","word_index = tokenizer.word_index\n","print (\"word index type and length: \", type(word_index), len(word_index))\n","\n","\n","for x in list(word_index)[450:454]:\n"," print (\"example word index key value pair: \", x, word_index[x])\n","\n","sequences = tokenizer.texts_to_sequences(training_sentences)\n","print (\"tokenizer.texts_to_sequences type: \", type(sequences))\n","print (\"length of sequences: \", len(sequences))\n","print (\"an example sequence: \", sequences[450])\n","\n","padded = pad_sequences(sequences,maxlen=max_length, truncating=trunc_type)\n","# \n","testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n","testing_padded = pad_sequences(testing_sequences,maxlen=max_length)\n","\n"],"execution_count":5,"outputs":[{"output_type":"stream","text":["word index type and length: 86539\n","example word index key value pair: mr 451\n","example word index key value pair: overall 452\n","example word index key value pair: called 453\n","example word index key value pair: children 454\n","tokenizer.texts_to_sequences type: \n","length of sequences: 25000\n","an example sequence: [1, 17, 2334, 1, 3, 680, 1, 703, 5270, 14, 35, 5659, 2520, 1022, 2, 54, 89, 2284, 394, 5, 2520, 1068, 108, 1, 7456, 600, 33, 2, 274, 7, 24, 226, 126, 363, 2716, 8, 8, 703, 5270, 300, 3704, 6715, 30, 323, 5, 4, 600, 742, 497, 1, 26, 1938, 1226, 7, 2, 1031, 30, 3168, 3704, 48, 61, 682, 85, 76, 2, 3512, 5, 2, 1, 3, 4, 2749, 16, 8704, 2, 1165, 15, 524, 15, 1226, 48, 879, 27, 61, 1225, 3484, 3704, 21, 4960, 1981, 3704, 1, 4, 115, 6, 522, 1226, 562, 10, 21, 24, 4, 159, 115, 10, 21, 164, 2, 1, 9, 148, 113, 5, 4, 600, 777, 1861, 1463, 16, 30, 5, 70, 1165, 8, 8, 10, 253, 41, 192, 2520, 195, 6, 157, 46, 13, 3704, 7, 2, 2778, 268, 482, 160, 600, 48, 6, 192, 273, 3704, 6965, 3, 95, 1113, 4, 1339, 2442, 161, 2520, 48, 193, 2341, 6, 8306, 4, 5459, 8, 8, 163, 1709, 362, 17, 600, 33, 2, 274, 19, 10, 21, 24, 180, 356, 2522, 11, 64, 43, 201, 1, 42, 1, 2, 1305, 122, 12, 197, 679, 802, 703, 5270, 3, 2, 179, 5, 1, 19, 1, 42, 1, 7, 4, 1, 53, 3122, 3, 5068, 19, 12, 7, 134, 4, 88, 394, 47]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"9axf0uIXVMhO","outputId":"641a4c61-7f4c-4ac3-9618-e69488c44f99","executionInfo":{"status":"ok","timestamp":1572487228309,"user_tz":-540,"elapsed":1792,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":86}},"source":["reverse_word_index = dict([(value, key) for (key, value) in word_index.items()]) # reverse the key value pair in word_index dict\n","\n","# compare with the previous cell \n","for y in list(reverse_word_index)[450:454]:\n"," print (\"example word index key value pair: \", y, reverse_word_index[y]) \n","\n","# def decode_review(text):\n","# return ' '.join([reverse_word_index.get(i, '?') for i in text])\n","\n","# print(decode_review(padded[1]))\n","# print(training_sentences[1])"],"execution_count":6,"outputs":[{"output_type":"stream","text":["example word index key value pair: 451 mr\n","example word index key value pair: 452 overall\n","example word index key value pair: 453 called\n","example word index key value pair: 454 children\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"5NEpdhb8AxID","colab_type":"code","outputId":"f66d6510-95fa-4254-bc2f-df6193a4065e","executionInfo":{"status":"ok","timestamp":1572487233167,"user_tz":-540,"elapsed":953,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":295}},"source":["model = tf.keras.Sequential([\n"," tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n"," tf.keras.layers.Flatten(),\n"," tf.keras.layers.Dense(6, activation='relu'),\n"," tf.keras.layers.Dense(1, activation='sigmoid')\n","])\n","model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n","model.summary()"],"execution_count":7,"outputs":[{"output_type":"stream","text":["Model: \"sequential\"\n","_________________________________________________________________\n","Layer (type) Output Shape Param # \n","=================================================================\n","embedding (Embedding) (None, 120, 16) 160000 \n","_________________________________________________________________\n","flatten (Flatten) (None, 1920) 0 \n","_________________________________________________________________\n","dense (Dense) (None, 6) 11526 \n","_________________________________________________________________\n","dense_1 (Dense) (None, 1) 7 \n","=================================================================\n","Total params: 171,533\n","Trainable params: 171,533\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"V5LLrXC-uNX6","colab_type":"code","outputId":"b22e6b24-f6cb-4bea-a962-46be05130eb4","executionInfo":{"status":"ok","timestamp":1572487488505,"user_tz":-540,"elapsed":48882,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":523}},"source":["num_epochs = 10\n","model.fit(padded, training_labels_final, epochs=num_epochs, validation_data=(testing_padded, testing_labels_final))"],"execution_count":8,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Use tf.where in 2.0, which has the same broadcast rule as np.where\n"],"name":"stdout"},{"output_type":"stream","text":["WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Use tf.where in 2.0, which has the same broadcast rule as np.where\n"],"name":"stderr"},{"output_type":"stream","text":["Train on 25000 samples, validate on 25000 samples\n","Epoch 1/10\n","25000/25000 [==============================] - 5s 200us/sample - loss: 0.4920 - accuracy: 0.7485 - val_loss: 0.3846 - val_accuracy: 0.8269\n","Epoch 2/10\n","25000/25000 [==============================] - 5s 189us/sample - loss: 0.2412 - accuracy: 0.9056 - val_loss: 0.3768 - val_accuracy: 0.8345\n","Epoch 3/10\n","25000/25000 [==============================] - 5s 187us/sample - loss: 0.0957 - accuracy: 0.9757 - val_loss: 0.4435 - val_accuracy: 0.8294\n","Epoch 4/10\n","25000/25000 [==============================] - 5s 188us/sample - loss: 0.0247 - accuracy: 0.9969 - val_loss: 0.5189 - val_accuracy: 0.8261\n","Epoch 5/10\n","25000/25000 [==============================] - 5s 188us/sample - loss: 0.0060 - accuracy: 0.9995 - val_loss: 0.5855 - val_accuracy: 0.8266\n","Epoch 6/10\n","25000/25000 [==============================] - 5s 189us/sample - loss: 0.0018 - accuracy: 1.0000 - val_loss: 0.6367 - val_accuracy: 0.8268\n","Epoch 7/10\n","25000/25000 [==============================] - 5s 187us/sample - loss: 8.9690e-04 - accuracy: 1.0000 - val_loss: 0.6747 - val_accuracy: 0.8286\n","Epoch 8/10\n","25000/25000 [==============================] - 5s 188us/sample - loss: 4.8627e-04 - accuracy: 1.0000 - val_loss: 0.7166 - val_accuracy: 0.8275\n","Epoch 9/10\n","25000/25000 [==============================] - 5s 188us/sample - loss: 2.7565e-04 - accuracy: 1.0000 - val_loss: 0.7501 - val_accuracy: 0.8278\n","Epoch 10/10\n","25000/25000 [==============================] - 5s 189us/sample - loss: 1.6326e-04 - accuracy: 1.0000 - val_loss: 0.7859 - val_accuracy: 0.8275\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{"tags":[]},"execution_count":8}]},{"cell_type":"code","metadata":{"id":"yAmjJqEyCOF_","colab_type":"code","outputId":"91a01823-74c1-48ef-ad7a-43f55ba0e474","executionInfo":{"status":"ok","timestamp":1572487497485,"user_tz":-540,"elapsed":1190,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["e = model.layers[0] # 0th layer or the 1st layer is indeed the embedding layer\n","weights = e.get_weights()[0] # get weights from the embeddings layer\n","print(weights.shape) # shape: (vocab_size, embedding_dim) # we are considering a 16 dimensional space "],"execution_count":9,"outputs":[{"output_type":"stream","text":["(10000, 16)\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"qp-uWtjXldfX","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":124},"outputId":"464588b7-3bc6-4406-ef99-1c44118ca53a","executionInfo":{"status":"ok","timestamp":1572488029085,"user_tz":-540,"elapsed":21591,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["from google.colab import drive\n","drive.mount('/content/gdrive')"],"execution_count":14,"outputs":[{"output_type":"stream","text":["Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n","\n","Enter your authorization code:\n","··········\n","Mounted at /content/gdrive\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"jmB0Uxk0ycP6","colab_type":"code","colab":{}},"source":["import io\n","\n","out_v = io.open('/content/gdrive/My Drive/Colab Notebooks/NLP_Week2/vecs.tsv', 'w', encoding='utf-8')\n","out_m = io.open('/content/gdrive/My Drive/Colab Notebooks/NLP_Week2/meta.tsv', 'w', encoding='utf-8')\n","for word_num in range(1, vocab_size):\n"," word = reverse_word_index[word_num]\n"," embeddings = weights[word_num]\n"," out_m.write(word + \"\\n\")\n"," out_v.write('\\t'.join([str(x) for x in embeddings]) + \"\\n\")\n","out_v.close()\n","out_m.close()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"dm4nFwImnl67","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"VDeqpOCVydtq","colab_type":"code","colab":{}},"source":["\n","## !!!!! Not needed !!!! \n","# try:\n","# from google.colab import files\n","# except ImportError:\n","# pass\n","# else:\n","# files.download('vecs.tsv')\n","# files.download('meta.tsv')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"YRxoxc2apscY","colab_type":"code","outputId":"3626e62a-fe6b-4fe5-fe77-d7af4c9e76e6","executionInfo":{"status":"ok","timestamp":1572487762387,"user_tz":-540,"elapsed":974,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}},"colab":{"base_uri":"https://localhost:8080/","height":52}},"source":["sentence = [\"The movie was awful. Don't waste your money on it.\"]\n","sequence_test = tokenizer.texts_to_sequences(sentence)\n","print(sequence_test)\n","\n","padded_test = pad_sequences(sequence_test,maxlen=max_length, truncating=trunc_type)\n","\n","ypr = model.predict(padded_test) \n","print (\"prediction label: \", ypr)"],"execution_count":13,"outputs":[{"output_type":"stream","text":["[[2, 18, 14, 380, 174, 446, 131, 277, 22, 10]]\n","prediction label: [[6.153002e-05]]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"wvl_Mp3OfblH","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /NLP_Embedding1/Week2_Course3_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Embedding1/Week2_Course3_1.png -------------------------------------------------------------------------------- /NLP_Embedding1/Week2_Course3_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Embedding1/Week2_Course3_2.png -------------------------------------------------------------------------------- /NLP_Embedding1/Week2_Course3_4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Embedding1/Week2_Course3_4.png -------------------------------------------------------------------------------- /NLP_Embedding1/Week3_Course3_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Embedding1/Week3_Course3_3.png -------------------------------------------------------------------------------- /NLP_LSTM_Glove/C3_W3_Quiz/week3_Course3_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_LSTM_Glove/C3_W3_Quiz/week3_Course3_1.png -------------------------------------------------------------------------------- /NLP_LSTM_Glove/C3_W3_Quiz/week3_Course3_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_LSTM_Glove/C3_W3_Quiz/week3_Course3_2.png -------------------------------------------------------------------------------- /NLP_LSTM_Glove/C3_W3_Quiz/week3_Course3_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_LSTM_Glove/C3_W3_Quiz/week3_Course3_3.png -------------------------------------------------------------------------------- /NLP_LSTM_Glove/Wd_cld_neg_sent.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_LSTM_Glove/Wd_cld_neg_sent.png -------------------------------------------------------------------------------- /NLP_LSTM_Glove/Wd_cld_pos_sent.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_LSTM_Glove/Wd_cld_pos_sent.png -------------------------------------------------------------------------------- /NLP_Poetry/Quiz_Week4_C3/week4_course3_345.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Poetry/Quiz_Week4_C3/week4_course3_345.png -------------------------------------------------------------------------------- /NLP_Poetry/Quiz_Week4_C3/week4_course3_67.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Poetry/Quiz_Week4_C3/week4_course3_67.png -------------------------------------------------------------------------------- /NLP_Poetry/Quiz_Week4_C3/week4_course3_78.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Poetry/Quiz_Week4_C3/week4_course3_78.png -------------------------------------------------------------------------------- /NLP_Poetry/Quiz_Week4_C3/week4_course3_first2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/NLP_Poetry/Quiz_Week4_C3/week4_course3_first2.png -------------------------------------------------------------------------------- /NLP_Poetry/Shakespeare_poetry_NLP_Week4.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Shakespeare_poetry_NLP_Week4.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"aDnQQXBpA2_H","colab_type":"text"},"source":["## NLP Shakespeare \n","Idea of the exercise is to find whether a deep network can create a poetry or not. We will train the model using Shakespeare Sonnet. "]},{"cell_type":"markdown","metadata":{"id":"ejzkrqQvCUDS","colab_type":"text"},"source":["#### First Mount the Drive "]},{"cell_type":"code","metadata":{"id":"v0KaFlWKDV1O","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"4f3e6ab9-d91b-41b1-f095-d92a153a3f34","executionInfo":{"status":"ok","timestamp":1573983976844,"user_tz":-540,"elapsed":949,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["from google.colab import drive\n","drive.mount('/content/gdrive')"],"execution_count":35,"outputs":[{"output_type":"stream","text":["Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"RTTLfYGwD0Dc","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":663},"outputId":"a2314d2d-1e5b-46f1-c2dc-2812c14b2fd3","executionInfo":{"status":"ok","timestamp":1573956460094,"user_tz":-540,"elapsed":35790,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["!pip3 install tensorflow==2.0.0-beta0"],"execution_count":2,"outputs":[{"output_type":"stream","text":["Collecting tensorflow==2.0.0-beta0\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/6c/19/0d0c7f240db7bcd6b83783b9a89a67f38584d100e23ad5ae93114be92232/tensorflow-2.0.0b0-cp36-cp36m-manylinux1_x86_64.whl (87.9MB)\n","\u001b[K |████████████████████████████████| 87.9MB 104kB/s \n","\u001b[?25hRequirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.8.1)\n","Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.17.4)\n","Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.2.2)\n","Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.11.2)\n","Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.15.0)\n","Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.1.0)\n","Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.0.8)\n","Collecting tf-estimator-nightly<1.14.0.dev2019060502,>=1.14.0.dev2019060501\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/32/dd/99c47dd007dcf10d63fd895611b063732646f23059c618a373e85019eb0e/tf_estimator_nightly-1.14.0.dev2019060501-py2.py3-none-any.whl (496kB)\n","\u001b[K |████████████████████████████████| 501kB 48.6MB/s \n","\u001b[?25hCollecting tb-nightly<1.14.0a20190604,>=1.14.0a20190603\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/a4/96/571b875cd81dda9d5dfa1422a4f9d749e67c0a8d4f4f0b33a4e5f5f35e27/tb_nightly-1.14.0a20190603-py3-none-any.whl (3.1MB)\n","\u001b[K |████████████████████████████████| 3.1MB 52.2MB/s \n","\u001b[?25hRequirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.1.0)\n","Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (1.12.0)\n","Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.8.0)\n","Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.1.8)\n","Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (0.33.6)\n","Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-beta0) (3.10.0)\n","Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.6->tensorflow==2.0.0-beta0) (2.8.0)\n","Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow==2.0.0-beta0) (0.16.0)\n","Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow==2.0.0-beta0) (41.4.0)\n","Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow==2.0.0-beta0) (3.1.1)\n","Installing collected packages: tf-estimator-nightly, tb-nightly, tensorflow\n"," Found existing installation: tensorflow 1.15.0\n"," Uninstalling tensorflow-1.15.0:\n"," Successfully uninstalled tensorflow-1.15.0\n","Successfully installed tb-nightly-1.14.0a20190603 tensorflow-2.0.0b0 tf-estimator-nightly-1.14.0.dev2019060501\n"],"name":"stdout"},{"output_type":"display_data","data":{"application/vnd.colab-display-data+json":{"pip_warning":{"packages":["tensorboard","tensorflow","tensorflow_estimator"]}}},"metadata":{"tags":[]}}]},{"cell_type":"markdown","metadata":{"id":"nUmYt1CnEyj0","colab_type":"text"},"source":["#### Import Libraries "]},{"cell_type":"code","metadata":{"id":"NmWga-PUFWda","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"eb7ac0a2-d6cd-412c-9432-ac16d442934c","executionInfo":{"status":"ok","timestamp":1573983984130,"user_tz":-540,"elapsed":963,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["import tensorflow as tf\n","print (\"check tensorflow version: \", tf.__version__)\n","\n","from tensorflow.keras.preprocessing.text import Tokenizer\n","from tensorflow.keras.models import Sequential\n","from tensorflow.keras.preprocessing.sequence import pad_sequences\n","from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout, Bidirectional\n","from tensorflow.keras.optimizers import Adam\n","from tensorflow.keras import regularizers\n","import tensorflow.keras.utils as ku\n","\n","import matplotlib.pyplot as plt\n","import numpy as np\n"],"execution_count":36,"outputs":[{"output_type":"stream","text":["check tensorflow version: 2.0.0-beta0\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"ow-qSvJvMu2b","colab_type":"text"},"source":["#### Read the sonnet file and create the corpus \n","use tokenizer in the next step"]},{"cell_type":"code","metadata":{"id":"zMVJvrQsHPDX","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":89},"outputId":"27fb82b5-429d-4c06-fd6f-31ed3085e11f","executionInfo":{"status":"ok","timestamp":1573983990944,"user_tz":-540,"elapsed":950,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["sonnets = open('/content/gdrive/My Drive/Colab Notebooks/NLP_Week4/sonnets.txt').read()\n","corpus = sonnets.lower().split(\"\\n\") # split function is called on the line-break, so that every line of the sonnet is now individual elements of the corpus list. \n","print (\"corpus type: \", type(corpus))\n","print (\"first few elements of corpus: \", corpus[5:10])\n","\n","tokenizer = Tokenizer()\n","tokenizer.fit_on_texts(corpus) # creates the dictionary of the overall corpus, with key as word and value is the correpsonding token. \n","\n","total_words = len(tokenizer.word_index) + 1\n","print (\"total number of words + unknown : \", total_words)"],"execution_count":37,"outputs":[{"output_type":"stream","text":["corpus type: \n","first few elements of corpus: [\"feed'st thy light'st flame with self-substantial fuel,\", 'making a famine where abundance lies,', 'thyself thy foe, to thy sweet self too cruel.', \"thou that art now the world's fresh ornament\", 'and only herald to the gaudy spring,']\n","total number of words + unknown : 3211\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"JFnDYI8lLhF9","colab_type":"text"},"source":["#### Create the training data from the corpus\n","Here's a brief outline of how we can create list of n-gram sequence. \n","1. First we iterate over each element of the corpus list, which is basically a single line of the sonnet. \n","2. Use _texts_to_sequences_ method to create list of tokens that represent the words on the line. Note: _texts_to_sequences_ on ([line]) creates a list of list (ex: [[2, 3, 4,5]]), so, to access the list we need to use _texts_to_sequences_([line])[0]. \n","3. Iterate over the list of tokens to create a number of n gram sequences. So the first two words will be one squence, first 3 another sequence and so on.. \n","4. We append all the sequences to create a new list\n","5. Then we figure out the longest sentence of the corpus so that later we can pad our sequences. \n","6. Now we pad all the sequences so that they are all of same lengths. Here we use pre padding. This is important because now our sequences will look like as [0 0 0 0 0 2 3], [0 0 0 0 2 3 4], [0 0 0 2 3 4 5] and so on.... With this it is now easy to create x(input) and y(output). Always the last element of the list will be the prediction and everything else will be features. \n","7. Create features and labels by slicing the array. \n","8. One hot encode the labels, here the number of classes will be the number of the words. So after the one hot encoding our y will be an array of length of number of words in the corpus, and the value that is set to 1 is the index of the label. that means [0 0 2 3 4 5 6], 6th element of y will be set to 1 and everything else will be zero. "]},{"cell_type":"code","metadata":{"id":"IM1WEiMSY1AO","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":52},"outputId":"1e8d0ecb-28d8-4abb-a3fd-daa297d30972","executionInfo":{"status":"ok","timestamp":1573983996636,"user_tz":-540,"elapsed":1123,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["input_seq = []\n","for line in corpus:\n"," token_list = tokenizer.texts_to_sequences([line])[0]\n"," for x in range(1, len(token_list)):\n"," n_gram_seq = token_list[:x+1]\n"," input_seq.append(n_gram_seq)\n","\n","max_seq_len = max([len(x) for x in input_seq])\n","input_seq = np.array(pad_sequences(input_seq, maxlen=max_seq_len, padding='pre'))\n","print (\"check the shape of the input_seq: \", input_seq.shape) # two dimensional array\n","pred, label = input_seq[:, :-1], input_seq[:, -1]\n","label_one_hot = ku.to_categorical(label, num_classes=total_words)\n","\n","print (\"check the one hot encoded label shape: \", label_one_hot.shape) # perfect!"],"execution_count":38,"outputs":[{"output_type":"stream","text":["check the shape of the input_seq: (15462, 11)\n","check the one hot encoded label shape: (15462, 3211)\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"GyTyGonglCtH","colab_type":"text"},"source":["#### Build the Model \n","The layers will be-- \n","1. Embedding layer. \n","2. Bi-directional LSTM\n","3. Dropout \n","4. LSTM\n","5. Dense layer with number of nodes = total_words/2\n","6. Dense layer with number of nodes = total_words"]},{"cell_type":"code","metadata":{"id":"NcFj4OPAqzKa","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":364},"outputId":"3c0ac42e-f967-49a8-a92d-ef15546a2fc1","executionInfo":{"status":"ok","timestamp":1573984016279,"user_tz":-540,"elapsed":2449,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["model = Sequential()\n","model.add(Embedding(total_words, 100, input_length=max_seq_len - 1))\n","model.add(Bidirectional(LSTM(120, return_sequences='True')))\n","model.add(Dropout(0.3))\n","model.add(LSTM(100))\n","model.add(Dense(total_words/2, activation='relu', kernel_regularizer=regularizers.l2(l=0.01)))\n","model.add(Dense(total_words, activation='softmax'))\n","model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n","\n","model.summary()"],"execution_count":39,"outputs":[{"output_type":"stream","text":["Model: \"sequential_5\"\n","_________________________________________________________________\n","Layer (type) Output Shape Param # \n","=================================================================\n","embedding_5 (Embedding) (None, 10, 100) 321100 \n","_________________________________________________________________\n","bidirectional_5 (Bidirection (None, 10, 240) 212160 \n","_________________________________________________________________\n","dropout_5 (Dropout) (None, 10, 240) 0 \n","_________________________________________________________________\n","lstm_11 (LSTM) (None, 100) 136400 \n","_________________________________________________________________\n","dense_10 (Dense) (None, 1605) 162105 \n","_________________________________________________________________\n","dense_11 (Dense) (None, 3211) 5156866 \n","=================================================================\n","Total params: 5,988,631\n","Trainable params: 5,988,631\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"CFgjNb6TuMy6","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":1000},"outputId":"482a242d-5fff-45cd-85a5-b619ab59c613","executionInfo":{"status":"ok","timestamp":1573990593539,"user_tz":-540,"elapsed":6567909,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["history = model.fit(pred, label_one_hot, epochs=120, verbose=2)"],"execution_count":40,"outputs":[{"output_type":"stream","text":["Train on 15462 samples\n","Epoch 1/120\n","15462/15462 - 56s - loss: 6.9149 - accuracy: 0.0206\n","Epoch 2/120\n","15462/15462 - 55s - loss: 6.4986 - accuracy: 0.0228\n","Epoch 3/120\n","15462/15462 - 55s - loss: 6.3769 - accuracy: 0.0266\n","Epoch 4/120\n","15462/15462 - 55s - loss: 6.2555 - accuracy: 0.0335\n","Epoch 5/120\n","15462/15462 - 55s - loss: 6.1689 - accuracy: 0.0364\n","Epoch 6/120\n","15462/15462 - 55s - loss: 6.0905 - accuracy: 0.0391\n","Epoch 7/120\n","15462/15462 - 55s - loss: 6.0127 - accuracy: 0.0424\n","Epoch 8/120\n","15462/15462 - 55s - loss: 5.9306 - accuracy: 0.0452\n","Epoch 9/120\n","15462/15462 - 54s - loss: 5.8302 - accuracy: 0.0500\n","Epoch 10/120\n","15462/15462 - 54s - loss: 5.7248 - accuracy: 0.0576\n","Epoch 11/120\n","15462/15462 - 55s - loss: 5.6139 - accuracy: 0.0627\n","Epoch 12/120\n","15462/15462 - 55s - loss: 5.5113 - accuracy: 0.0672\n","Epoch 13/120\n","15462/15462 - 54s - loss: 5.4018 - accuracy: 0.0717\n","Epoch 14/120\n","15462/15462 - 54s - loss: 5.2993 - accuracy: 0.0797\n","Epoch 15/120\n","15462/15462 - 54s - loss: 5.2023 - accuracy: 0.0853\n","Epoch 16/120\n","15462/15462 - 54s - loss: 5.0984 - accuracy: 0.0912\n","Epoch 17/120\n","15462/15462 - 54s - loss: 5.0034 - accuracy: 0.0979\n","Epoch 18/120\n","15462/15462 - 54s - loss: 4.9036 - accuracy: 0.1057\n","Epoch 19/120\n","15462/15462 - 54s - loss: 4.8057 - accuracy: 0.1142\n","Epoch 20/120\n","15462/15462 - 54s - loss: 4.7045 - accuracy: 0.1187\n","Epoch 21/120\n","15462/15462 - 54s - loss: 4.6104 - accuracy: 0.1298\n","Epoch 22/120\n","15462/15462 - 54s - loss: 4.5157 - accuracy: 0.1395\n","Epoch 23/120\n","15462/15462 - 54s - loss: 4.4182 - accuracy: 0.1472\n","Epoch 24/120\n","15462/15462 - 54s - loss: 4.3133 - accuracy: 0.1573\n","Epoch 25/120\n","15462/15462 - 54s - loss: 4.2334 - accuracy: 0.1712\n","Epoch 26/120\n","15462/15462 - 54s - loss: 4.1406 - accuracy: 0.1819\n","Epoch 27/120\n","15462/15462 - 54s - loss: 4.0567 - accuracy: 0.1923\n","Epoch 28/120\n","15462/15462 - 54s - loss: 3.9629 - accuracy: 0.2101\n","Epoch 29/120\n","15462/15462 - 54s - loss: 3.8797 - accuracy: 0.2187\n","Epoch 30/120\n","15462/15462 - 54s - loss: 3.7969 - accuracy: 0.2328\n","Epoch 31/120\n","15462/15462 - 54s - loss: 3.7185 - accuracy: 0.2538\n","Epoch 32/120\n","15462/15462 - 54s - loss: 3.6415 - accuracy: 0.2639\n","Epoch 33/120\n","15462/15462 - 54s - loss: 3.5663 - accuracy: 0.2810\n","Epoch 34/120\n","15462/15462 - 54s - loss: 3.4919 - accuracy: 0.2941\n","Epoch 35/120\n","15462/15462 - 55s - loss: 3.4155 - accuracy: 0.3165\n","Epoch 36/120\n","15462/15462 - 55s - loss: 3.3460 - accuracy: 0.3291\n","Epoch 37/120\n","15462/15462 - 55s - loss: 3.2908 - accuracy: 0.3377\n","Epoch 38/120\n","15462/15462 - 55s - loss: 3.2232 - accuracy: 0.3489\n","Epoch 39/120\n","15462/15462 - 55s - loss: 3.1645 - accuracy: 0.3634\n","Epoch 40/120\n","15462/15462 - 55s - loss: 3.0922 - accuracy: 0.3802\n","Epoch 41/120\n","15462/15462 - 54s - loss: 3.0488 - accuracy: 0.3902\n","Epoch 42/120\n","15462/15462 - 55s - loss: 2.9680 - accuracy: 0.4085\n","Epoch 43/120\n","15462/15462 - 54s - loss: 2.9241 - accuracy: 0.4188\n","Epoch 44/120\n","15462/15462 - 55s - loss: 2.8765 - accuracy: 0.4248\n","Epoch 45/120\n","15462/15462 - 55s - loss: 2.8117 - accuracy: 0.4413\n","Epoch 46/120\n","15462/15462 - 55s - loss: 2.7580 - accuracy: 0.4515\n","Epoch 47/120\n","15462/15462 - 55s - loss: 2.7130 - accuracy: 0.4680\n","Epoch 48/120\n","15462/15462 - 55s - loss: 2.6615 - accuracy: 0.4786\n","Epoch 49/120\n","15462/15462 - 55s - loss: 2.6237 - accuracy: 0.4816\n","Epoch 50/120\n","15462/15462 - 55s - loss: 2.5897 - accuracy: 0.4925\n","Epoch 51/120\n","15462/15462 - 54s - loss: 2.5338 - accuracy: 0.5018\n","Epoch 52/120\n","15462/15462 - 55s - loss: 2.4960 - accuracy: 0.5116\n","Epoch 53/120\n","15462/15462 - 55s - loss: 2.4474 - accuracy: 0.5233\n","Epoch 54/120\n","15462/15462 - 54s - loss: 2.4051 - accuracy: 0.5336\n","Epoch 55/120\n","15462/15462 - 55s - loss: 2.3665 - accuracy: 0.5412\n","Epoch 56/120\n","15462/15462 - 54s - loss: 2.3390 - accuracy: 0.5457\n","Epoch 57/120\n","15462/15462 - 55s - loss: 2.2986 - accuracy: 0.5520\n","Epoch 58/120\n","15462/15462 - 54s - loss: 2.2581 - accuracy: 0.5628\n","Epoch 59/120\n","15462/15462 - 54s - loss: 2.2273 - accuracy: 0.5713\n","Epoch 60/120\n","15462/15462 - 54s - loss: 2.1880 - accuracy: 0.5828\n","Epoch 61/120\n","15462/15462 - 55s - loss: 2.1659 - accuracy: 0.5913\n","Epoch 62/120\n","15462/15462 - 55s - loss: 2.1247 - accuracy: 0.5947\n","Epoch 63/120\n","15462/15462 - 55s - loss: 2.0886 - accuracy: 0.6029\n","Epoch 64/120\n","15462/15462 - 55s - loss: 2.0746 - accuracy: 0.6052\n","Epoch 65/120\n","15462/15462 - 55s - loss: 2.0415 - accuracy: 0.6145\n","Epoch 66/120\n","15462/15462 - 55s - loss: 2.0171 - accuracy: 0.6187\n","Epoch 67/120\n","15462/15462 - 54s - loss: 1.9788 - accuracy: 0.6261\n","Epoch 68/120\n","15462/15462 - 55s - loss: 1.9497 - accuracy: 0.6312\n","Epoch 69/120\n","15462/15462 - 55s - loss: 1.9257 - accuracy: 0.6358\n","Epoch 70/120\n","15462/15462 - 55s - loss: 1.9101 - accuracy: 0.6387\n","Epoch 71/120\n","15462/15462 - 55s - loss: 1.8725 - accuracy: 0.6486\n","Epoch 72/120\n","15462/15462 - 55s - loss: 1.8501 - accuracy: 0.6531\n","Epoch 73/120\n","15462/15462 - 55s - loss: 1.8259 - accuracy: 0.6604\n","Epoch 74/120\n","15462/15462 - 55s - loss: 1.8045 - accuracy: 0.6694\n","Epoch 75/120\n","15462/15462 - 55s - loss: 1.7802 - accuracy: 0.6696\n","Epoch 76/120\n","15462/15462 - 55s - loss: 1.7697 - accuracy: 0.6691\n","Epoch 77/120\n","15462/15462 - 55s - loss: 1.7375 - accuracy: 0.6773\n","Epoch 78/120\n","15462/15462 - 55s - loss: 1.7104 - accuracy: 0.6863\n","Epoch 79/120\n","15462/15462 - 55s - loss: 1.6905 - accuracy: 0.6903\n","Epoch 80/120\n","15462/15462 - 55s - loss: 1.6726 - accuracy: 0.6905\n","Epoch 81/120\n","15462/15462 - 55s - loss: 1.6628 - accuracy: 0.6921\n","Epoch 82/120\n","15462/15462 - 55s - loss: 1.6519 - accuracy: 0.6936\n","Epoch 83/120\n","15462/15462 - 55s - loss: 1.6232 - accuracy: 0.7020\n","Epoch 84/120\n","15462/15462 - 55s - loss: 1.6077 - accuracy: 0.7064\n","Epoch 85/120\n","15462/15462 - 55s - loss: 1.5850 - accuracy: 0.7131\n","Epoch 86/120\n","15462/15462 - 55s - loss: 1.5792 - accuracy: 0.7080\n","Epoch 87/120\n","15462/15462 - 55s - loss: 1.5516 - accuracy: 0.7155\n","Epoch 88/120\n","15462/15462 - 55s - loss: 1.5372 - accuracy: 0.7171\n","Epoch 89/120\n","15462/15462 - 55s - loss: 1.5136 - accuracy: 0.7258\n","Epoch 90/120\n","15462/15462 - 55s - loss: 1.5132 - accuracy: 0.7211\n","Epoch 91/120\n","15462/15462 - 56s - loss: 1.5062 - accuracy: 0.7212\n","Epoch 92/120\n","15462/15462 - 55s - loss: 1.4812 - accuracy: 0.7317\n","Epoch 93/120\n","15462/15462 - 55s - loss: 1.4647 - accuracy: 0.7352\n","Epoch 94/120\n","15462/15462 - 55s - loss: 1.4410 - accuracy: 0.7396\n","Epoch 95/120\n","15462/15462 - 55s - loss: 1.4438 - accuracy: 0.7357\n","Epoch 96/120\n","15462/15462 - 55s - loss: 1.4385 - accuracy: 0.7365\n","Epoch 97/120\n","15462/15462 - 55s - loss: 1.4158 - accuracy: 0.7422\n","Epoch 98/120\n","15462/15462 - 55s - loss: 1.4027 - accuracy: 0.7471\n","Epoch 99/120\n","15462/15462 - 54s - loss: 1.3976 - accuracy: 0.7447\n","Epoch 100/120\n","15462/15462 - 55s - loss: 1.3803 - accuracy: 0.7526\n","Epoch 101/120\n","15462/15462 - 55s - loss: 1.3671 - accuracy: 0.7526\n","Epoch 102/120\n","15462/15462 - 55s - loss: 1.3508 - accuracy: 0.7577\n","Epoch 103/120\n","15462/15462 - 55s - loss: 1.3487 - accuracy: 0.7572\n","Epoch 104/120\n","15462/15462 - 55s - loss: 1.3343 - accuracy: 0.7615\n","Epoch 105/120\n","15462/15462 - 54s - loss: 1.3227 - accuracy: 0.7626\n","Epoch 106/120\n","15462/15462 - 55s - loss: 1.3152 - accuracy: 0.7593\n","Epoch 107/120\n","15462/15462 - 55s - loss: 1.3126 - accuracy: 0.7595\n","Epoch 108/120\n","15462/15462 - 55s - loss: 1.3109 - accuracy: 0.7590\n","Epoch 109/120\n","15462/15462 - 54s - loss: 1.2728 - accuracy: 0.7717\n","Epoch 110/120\n","15462/15462 - 55s - loss: 1.2817 - accuracy: 0.7656\n","Epoch 111/120\n","15462/15462 - 55s - loss: 1.2623 - accuracy: 0.7709\n","Epoch 112/120\n","15462/15462 - 55s - loss: 1.2552 - accuracy: 0.7712\n","Epoch 113/120\n","15462/15462 - 55s - loss: 1.2373 - accuracy: 0.7786\n","Epoch 114/120\n","15462/15462 - 55s - loss: 1.2368 - accuracy: 0.7756\n","Epoch 115/120\n","15462/15462 - 55s - loss: 1.2375 - accuracy: 0.7739\n","Epoch 116/120\n","15462/15462 - 55s - loss: 1.2212 - accuracy: 0.7742\n","Epoch 117/120\n","15462/15462 - 55s - loss: 1.2098 - accuracy: 0.7808\n","Epoch 118/120\n","15462/15462 - 55s - loss: 1.2093 - accuracy: 0.7805\n","Epoch 119/120\n","15462/15462 - 54s - loss: 1.1970 - accuracy: 0.7822\n","Epoch 120/120\n","15462/15462 - 55s - loss: 1.1939 - accuracy: 0.7828\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"v-bKnE_PxXUF","colab_type":"text"},"source":["#### View Accuracy and Loss over Epochs"]},{"cell_type":"code","metadata":{"id":"BiQpatD-CY50","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":441},"outputId":"6350345e-8a95-4ecb-c2c0-c390c1c1a9aa","executionInfo":{"status":"ok","timestamp":1573990630883,"user_tz":-540,"elapsed":1496,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["train_acc = history.history['accuracy']\n","train_loss = history.history['loss']\n","\n","num_epochs = range(len(train_acc))\n","\n","plt.figure(figsize=(11, 6))\n","plt.subplot(1, 2, 1)\n","plt.plot(num_epochs, train_acc, linestyle='--', label='Train Accuracy', color='blue')\n","plt.legend(fontsize=12)\n","\n","plt.subplot(1, 2, 2)\n","plt.plot(num_epochs, train_loss, linestyle='--', label='Train Loss', color='red')\n","plt.legend(fontsize=12)\n","plt.tight_layout()\n","plt.show()"],"execution_count":41,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAxAAAAGoCAYAAADW/wPMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd5hV1fm38Xsx9CrKKEoRVBR7Q9BY\nY4vlJ2IXuzG2oGAXS1CxRcEoGhNFYsTeor6iiCZ2DVGxK2AEFASDFKkifb1/7AERKQPMzDrl/lzX\nueacffbMfIcyZz9nrfWsEGNEkiRJksqjWuoAkiRJkvKHBYQkSZKkcrOAkCRJklRuFhCSJEmSys0C\nQpIkSVK5VU/1jZs0aRJbtWqV6ttLklbi/fffnxRjLK2sr+/rgCTltuW9DiQrIFq1asWQIUNSfXtJ\n0kqEEEZX5tf3dUCSctvyXgecwiRJkiSp3CwgJEmSJJWbBYQkSZKkcrOAkCRJklRuyRZRL8/ChQsZ\nO3YsP/zwQ+ooygH16tWjefPmVKtmrStJUj7y2i531ahRg3XXXZeGDRuu0ueVq4AIIRwA9AFKgH4x\nxj8u9XxLoD+wVtk53WOMA1cpSZlJkyYRQmCzzTbzorHILVy4kHHjxjFp0iTWXXfd1HEkSdJq8Nou\nN8UY+fHHHxk3bhzAKhURK/1bDCGUAHcCBwJbAJ1DCFssddqVwOMxxu2BY4G/lDvBUqZOncp6663n\nPzBRrVo11ltvPaZNm5Y6iiRJWk1e2+WmEAJ169alWbNmTJgwYZU+tzx/k+2BETHGUTHGucCjwKFL\nnROBRWVLI+DbVUqxhAULFlCjRo3V/XQVmBo1ajB//vzUMSRJ0mry2i631alTh3nz5q3S55SngGgG\nfLPE47Flx5Z0NXBCCGEsMBA4d1lfKIRwRghhSAhhyMSJE5f7DUMI5YilYuC/BSk/hBA2CyF8tMRt\negjhvNS5JOUGX89z1+r83VTUWFJn4L4YY3PgIOCBEMIvvnaMsW+MsV2MsV1p6S92xZYk5akY4xcx\nxu1ijNsBOwKzgKcTx5IkVYLyFBDjgBZLPG5edmxJpwGPA8QYBwO1gSYVEbBQLViwgPr16zNmzJjU\nUSSpou0DjIwxjk4dRJKqSjFd25WngHgPaBNCaB1CqEm2SPrZpc4ZQ/aCQQhhc7ICYvlzlPJQ/fr1\nF9+qVatGnTp1Fj9+6KGHVvnrlZSUMHPmTFq2bLnamaZPn069evU45JBDVvtrSFIlOBZ4ZFlPlHcq\nqyRVtly6thsxYkReTfNaaRvXGOP8EMI5wItkLVrvjTF+HkLoCQyJMT4LXAjcE0I4n2xB9SkxxliZ\nwavazJkzF99v1aoV/fr1Y999913u+fPnz6d69crdZuOJJ56gdu3aDBo0iAkTJlRpq9Oq+Pkk5Z+y\nN5o6Apct6/kYY1+gL0C7du0K6nVCUn7JxWu7fFGuNRAxxoExxk1jjBvHGK8vO9ajrHggxjg0xrhr\njHHbsjmwL1Vm6Fx05ZVXcswxx9C5c2caNGjAgw8+yODBg9l5551Za621WH/99enateviVe7z588n\nhMDXX38NwAknnEDXrl058MADadCgAbvssgtfffXVCr9n//79Oeecc9h88815+OGHf/bc6NGj6dSp\nE6WlpTRp0oRu3botfu7uu++mbdu2NGjQgK222oqPP/74F3kWZbr66qsB+Ne//kWrVq244YYbaNq0\nKaeffjqTJ0/moIMOorS0lMaNG3PIIYcs7iUMMHnyZE455RTWX399GjduzBFHHAFA27ZteeGFFxaf\nN2fOHBo3bsynn366yn/uknLOgcAHMcbvUgeRpDWR4tpuWWbPnk3Xrl1Zf/31adasGRdccAFz584F\nYMKECRx00EGstdZarL322uyxxx6LP++GG25ggw02oGHDhrRt25bXXnttjf9MFsmLhrx77fXL21/K\ndpqYNWvZz993X/b8pEnLfv6xx7Lnv1myv9QaevrppznuuOOYNm0axxxzDNWrV6dPnz5MmjSJt99+\nm0GDBnH33Xcv9/Mffvhhrr32Wr7//ntatmzJH/7wh+WeO2rUKN566y2OP/54jj/+ePr377/4ufnz\n53PwwQezySab8PXXX/PNN99w9NFHA/DII49w3XXX8dBDDzF9+nSeeuop1l577XL9fGPHjmXmzJmM\nGTOGv/zlLyxcuJDTTz+dMWPGMHr0aGrUqPGzQuW4445j7ty5DB06lAkTJix+7qSTTuLBBx9cfN5z\nzz1Hq1at2HrrrcuVQ1JO68xypi9J0mJ5cnFXldd2y9OzZ0+GDBnCJ598wocffsjbb7/NjTfeCECv\nXr3YaKONmDhxIuPHj+e6664D4PPPP+fuu+/mgw8+YPr06bzwwgtrNG1+aXlRQOSL3XbbjUMOOWTx\nPLqddtqJDh06UL16dTbaaCPOOOMMXn/99eV+/pFHHkm7du2oUaMGxx9/PB999NFyz73//vvZYYcd\n2HTTTencuTMff/zx4nfwBw8ezKRJk7jpppuoV68ederUYddddwWgX79+dO/enR133JEQAptuuikt\nWrRY7vdZUvXq1bn66qupWbMmderUobS0lMMOO4w6derQsGFDLr/88sU/3zfffMPLL7/MX//6Vxo3\nbkyNGjUWV8UnnngiAwYMWLyl/QMPPMCJJ55YrgySlm3hQvjkE3juObjnHhidYPlyCKEesB/wVNV/\nd0mqeFV5bbc8Dz30EFdffTWlpaWsu+669OjRgwceeADI9sv69ttvGTNmDDVr1lx8rVW9enVmz57N\n559/zvz582ndujUbbbTR6v0hLENeTORa0YhL3borfr5JkxU/X85r53JZ+kJ8+PDhXHjhhbz//vvM\nmjWL+fPn06FDh+V+ftOmTRffr1u37s/m5i0pxsj999/Puedm2220bNmS3Xbbjf79+9O7d2+++eYb\nWrVqRUlJyS8+95tvvmHjjTdenR+P9dZbj5o1ay5+PHPmTM477zxeeuklpk6dCsCMGTMWf58mTZrQ\nqFGjX3ydFi1a0L59e5566ikOPvhgXnrpJe66667VyiQVo9dfh1degVNOgdat4YEH4KKLYMmNRB9/\nHDbcsGpzxRh/ANapkm/21ltw1VVw771V/4NKWnN5cnFXVdd2K/Ltt9+y4RK/5zbccMPFU8a7d+/O\nVVddxT777ENJSQlnnXUWF198MZttthm33HILPXr0YNiwYfzmN7/h1ltv/VmeNeEIRAVaevX8mWee\nyVZbbcWIESOYPn06PXv2pCLWlr/55pt89dVXXHvttTRt2pSmTZvy/vvv89BDD7FgwQJatGjB6NGj\nWbBgwS8+t0WLFowcOfIXx6tXr06tWrWYNWvW4mPjx49f4c/Xq1cvvvrqK959912mT5/OK6+88rPv\nM2nSJKZPn77Mn+Hkk0/mwQcf5LHHHmOPPfaosH/QUj6bPx9W9ivi1Vdhv/2gZ0/49tvsWLNmsO++\nWSHxzjswZgx06lT5eZOaPTuropZYtyVJFa2qru1WZIMNNmD0EsPKY8aMoVmzbE/nhg0bcuutt/L1\n11/zzDPPcNNNNy0eETnhhBN4++23+eqrr1iwYAGXXbbM3harxQKiEs2YMYNGjRpRr149hg0btsI5\ncquif//+HHDAAQwdOpSPPvqIjz76iE8//ZTp06fz0ksvscsuu7DOOutw+eWXM2vWLH788Ufefvtt\nAH73u99x88038+GHHxJj5Msvv+SbsrmC22677eIi5Pnnn+ett95a6c9Xt25dGjduzOTJk+nZs+fi\n51q0aMG+++5Lly5dmDp1KvPmzeONN95Y/Pzhhx/OO++8w5///GdOOumkCvlzkfLdHXfATjvBs89m\nU5KW9umnWWGw6aYweTLsskt2fO+94aGH4IQToH377M23GjWqNnuVK3vxZNzS2xJJUuWprGu7RWbP\nnv2z28KFC+ncuTM9e/Zk0qRJTJw4kWuvvZYTTjgBgAEDBjBy5EhijDRq1IiSkhKqVavGsGHDePXV\nV5kzZw516tShTp06VKtWcZf9FhCV6JZbbqF///40aNCAM888k2OOOWaNv+asWbN44okn6Nq16+LR\nh6ZNm7LRRhstXkxdvXp1nnvuOYYNG0aLFi1o2bIlTz75JACdO3fm0ksv5ZhjjqFhw4YcfvjhTJky\nBYDbb7+dp59+mrXWWosnnniCjh07rjDLBRdcwLRp01hnnXX41a9+xYEHHviz5xctlN50001Zb731\nuOOOOxY/V69ePTp16sSYMWPoVPBvlUrLN2ECXHxx9oZ6s2YwZQocemh2/3e/+2mUfvZs6NgR6teH\ngQNh7bWhAl8L8s+iAmLs2LQ5JBWVyri2W9Kii/1FtzfeeIOrrrqKbbfdlq222optttmGDh06LB5N\n+OKLL9h7772pX78+u+66K926dWP33Xdnzpw5XHLJJTRp0oSmTZsyZcoUrr/++grLGVJt19CuXbs4\nZMiQXxwfNmwYm2++eYJEqmo9evRgzJgx3Leoq8Jy+G9C+WriRKhTJ7voX5Z587LpSO+8k9222SY7\n9o9/wDPPwKBBsOuu8Pzz2fnbbJNNU9p226rJH0J4P8bYrrK+/vJeB8qtYUM49VTo06fiQkmqcL6O\n577l/R0t73UgLxZRq/BMnjyZv//97zy2qOWaVIDOOw9eeAE++ww22CA7NmcO/PrXMGwYzJyZrXu4\n//6sOIBs6tGxx2a3efOy9QyLvPce1KpV9T9HztptN2jcOHUKSSo6xTwArkT++te/0rJlSw499FB+\n9atfpY4jVZquXWHaNNhnH/iubFu1WrXgppvgpJOyqUuPPw7L62JcowYs2TTN4mEpAwdC2WaXkqSq\n4wiEqtzZZ5/N2WefnTqGVGkWLswKhg4dsjUMBxwA220Hp5+edU/afffsJklSPnIEQpIq2MCB2dYE\ngwdnhcKAATB9erbeN9Gys8LUrx9ssQUso2W1JKny5GQBkWpht3KP/xaUS0aMyKbc//rX8OGHyz/v\n5pth/fWhXdmys733hu+/z/Y8W6qluNbE3LnZYpIld9CTlJN8Pc9dC5fVN3wlcq6AqF27NpMnT/Yf\nmogxMnnyZGrXrp06iorUggVwww2waO+dli3hiCOyRdE77pjtBN2790/nDx4Mt90Gb74JF1zw870Y\nXL9QCdwLQsoLXtvlphgjc+fOZdy4cdSrV2+VPjfn1kA0b96csWPHMnHixNRRlANq165N8+bNU8dQ\nEXnnnWykYJ11soLgtdfgmGOydQ01a2azZnr3huuvz7qHzpuXdVuqXj177t57oUkTOO201D9JEViy\ngGhXad1mJa0hr+1yV/Xq1WnUqBFNmjRZtc+rpDyrrUaNGrRu3Tp1DElFasyYbA+GyZOzPRz+9rds\nq4Elpx6ttRb06gXXXJMVFiUl2fHeveH887Pnl7f3gyqQIxBSXvDarvDkXAEhSVVh2rRsXwWA//0P\nXnwR7rsPjjoqu8WYTWGqvoLfknXr/vxx48ZuS1Cl1l0X9t0XSktTJ5GkomIBIanozJgBO+0EX375\n07GttoIpU366Fg1hxcWDckBJCfzzn6lTSFLR8eVRUtH57rusOHjwwazdagiwww7ZlCVJkrRiFhCS\nCtrw4dnmbfvvny2GrlMHNtkEPvnEEYaCcO652Wr3IUNSJ5GkopFzbVwlqaKMHg077wyPP54thG7Y\nEDp2hB9/tHgoGNWqwX//mzqFJBUVCwhJBatlS7j0Uhg5El59FTp1gm++yXaFVoFo1ixb1OJfqiRV\nGd+Dk5R3YszarW644bKf+9Of4MADYYstftoEbsMNYa+9qjSmqsKifWLGjcuGmCRJlc4RCEl5Zdq0\nbC1Dq1bQuXNWMCwSI1xxBVx0UbZ/g4qAe0FIUpVzBEJS3nj/fTj66Gxtw3HHZSMMIcBnn2U7QH/y\nCbz8Mpx5ZrbRm4rAxhvDySdnW4dLkqqEBYSkvPC//8GvfpXtHfb667Drrj9/7q67YKON4KqrstuS\nO0ergDVvnu0AKEmqMhYQknLaF1/AZpvB+uvDPffAwQf/8s3mvfeGH36waChaMcKcOVC7duokklQU\nXAMhKWd99x20b5+1+Qc46aRlz1QpKbF4KGrt22cLYyRJVcICQlLO6tEDZs1yertWokkTF1FLUhWy\ngJCUzMyZcPrp8NJLv3zu00+hXz/o0gU23bTqsymPtGkDw4Zl89gkSZXOAkJSMjfckBUJBx4IvXv/\n1JI1RrjwQmjUKBuFkFboyCOzoaqnn06dRJKKggWEpGQuvjhrv3rEEdn9Qw/Njg8bBq+8khUPa6+d\nNqPywG67ZTsFPvBA6iSSVBTswiSpyv3wA9SoAY0bw6mnwimnwLbbwvXXZ89vsUU2hWnjjZPGVL6o\nVg369LHalKQq4giEpCrzww/wwgvZZnC/+hXMnZsdDyHbQXrJKeybbw41a6bJqTx06KGw++6pU0hS\nUbCAkFQlrr8+e4P4oIOy6UlHHfXLAsFWrFojn32WLayRJFUqCwhJFWr8eDjuOHjttZ+OffQRXHll\nVjy89BJ8/z1cemmyiCpUr76aDWV9+mnqJJJU0CwgJFWoSy+FRx6B/ff/aU3rdtvBM8/Ao4/CfvtB\nnTppM6pAHXssVK8ODz6YOokkFTQLCEkV6o474Pnns8Y4J53000jDoYdCrVpps6nAlZZmPYEfeggW\nLEidRpIKlgWEpDU2Z07WBGfWLGjYMJuqNGgQnHwy3H03TJqUOqGKxoknZrtSv/pq6iSSVLDKVUCE\nEA4IIXwRQhgRQui+jOdvDSF8VHb7bwhhasVHlZSLRo2CXXeF886Dxx776XjNmnDffTB2LDRpkiye\nis0hh0DLljB6dOokklSwVroPRAihBLgT2A8YC7wXQng2xjh00TkxxvOXOP9cYPtKyCopx7zwQjbt\nvFq1bBPgTp1+eU79+lWfS0Wsdm346qvsH6UkqVKU5zdse2BEjHFUjHEu8Chw6ArO7ww8UhHhJOWu\nMWOy4qF1a/jww2UXD1IS1apBjDBtWuokklSQylNANAO+WeLx2LJjvxBC2BBoDbyynOfPCCEMCSEM\nmThx4qpmlZRDZsyAtm2zkYdWrVKnkZZy8MHZjoWSpApX0WO8xwJPxhiX2f4ixtg3xtguxtiutLS0\ngr+1pKq05Zbwn/9kIxBSztlxR/jXv+Dbb1MnkaSCU54CYhzQYonHzcuOLcuxOH1JKkgxwrBhcMst\ncNZZMHOmO0crh514IixcCH37pk4iSQWnPAXEe0CbEELrEEJNsiLh2aVPCiG0BRoDgys2oqSqtGAB\nTJiQtWaFbJThxBNh/fVhiy3goovgrbfgxx/T5pRWaNNNs4U5t94KU6akTiNJBWWlBUSMcT5wDvAi\nMAx4PMb4eQihZwih4xKnHgs8GmOMlRNVUmV7913YaitYbz147bXs2LffZns67L039OsHI0fCp59m\ne3ZJOe2aa2D6dEchJKmCrbSNK0CMcSAwcKljPZZ6fHXFxZJUlebOhZ494cYboVmz7E3bzTfPnuvY\nMXsj166YyjvbbAP//CfsuWfqJJJUUMpVQEgqbN26wV13wamnZsVDo0Y/PVfd3xLKZ/vum31cuNAq\nWJIqiL9NJXHqqXDbbXDvvT8vHqSC8NJL2ZqI//0vdRJJKggWEFIRW7RiqX37bBRCKkgbbZTtfNij\nx8rPlSStlAWEVMQuuADOPvunQkIqSJtsAuecA3/7G3z8ceo0kpT3LCCkIvXGG9CnD5SUuJ+DisAf\n/gCNG8OFF1oxS9IasoCQitCUKXDCCbDxxlnnJangNW4MV18NL78Mg92uSJLWhP1VpCITI5x5Zrae\n9N//hgYNUieSqshZZ2X9iXfZJXUSScprFhBSgZo3LysQfvgh2zW6Zk3Yddds9GHgQLjuOthpp9Qp\npSpUo8ZPbV1nz4batdPmkaQ8ZQEhFaAY4Zhj4Omnf3783//O3nz9/HNo3jxNNim5+++Hyy7LtlRf\ne+3UaSQp77gGQipA338PQ4dC9+7wzjtZ45n33oOtt86e33DDbPG0VJFCCGuFEJ4MIQwPIQwLIeTm\nXKHtt4fx47Pt1yVJq8wRCKkArbMOfPRRtou0O0mrCvUBBsUYjwwh1ATqpg60TFtvDaefDnfemfUx\n3myz1IkkKa84AiEVmP79YebMbHq3xYOqSgihEbAH8DeAGOPcGOPUtKlWoGdPqFsXLr44dRJJyjsW\nEFIBefRROOUU6Ns3dRIVodbARODvIYQPQwj9Qgj1lj4phHBGCGFICGHIxIkTqz7lIuuum62DeP55\n+PLLdDkkKQ9ZQEgF4sMP4be/hd12yzbdlapYdWAH4K8xxu2BH4DuS58UY+wbY2wXY2xXWlpa1Rl/\nrmtX+OQTaNMmbQ5JyjMWEFIBmDABOnWCJk3gySezlq1SFRsLjI0xvlP2+EmygiJ31a0LW26Z3f/h\nh7RZJCmPWEBIBeCSS7Ii4plnYL31UqdRMYoxjge+CSEsWpG8DzA0YaTyu/RSaN8e5s9PnUSS8oIF\nhFQAuneHBx6AHXL7/V4VvnOBh0IInwDbATckzlM+O++c9T1+8MHUSSQpL1hASHnqkUeyTpQxQtu2\ncOSRqROp2MUYPypb37BNjLFTjHFK6kzl0qkTbLst9O6d/YeSJK2QBYSUh0aOhFNPheHD4ccfU6eR\n8lwIcN552RbtL7+cOo0k5TwLCCkPnX8+1KgBjz2WrQOVtIY6d84WEP35z6mTSFLOc5spKc88/zwM\nGAA33wwbbJA6jVQgatWCJ57I5gNKklbIAkLKIzHC5Zdn1zjduqVOIxWY3XdPnUCS8oJTmKQ8MX9+\nNlV7wAB46CH3epAqxbvvwj77wPffp04iSTnLAkLKcf/7H5x4Iuy3XzYC0bKl7VqlSlOnDrzyCvz9\n76mTSFLOsoCQctibb2bFwpNPwo47us+VVOm23hr22AP+8hdYsCB1GknKSRYQUg6KEW67DX79a2jQ\nAN57L2tRX6NG6mRSEejSBUaNgkGDUieRpJxkASHloBkz4M474ZBDsuJhq61SJ5KKyGGHwfrrZ/8J\nJUm/YBcmKYf88EM2ytCwIbz9NjRpAtUs86WqVaMGXH11NoUpxqx7gSRpMQsIKUfMnp0tlN58c/jb\n32DddVMnkorYGWekTiBJOcv3NqUcECOceSYMHgwHHpg6jSQgm0t4zz3w44+pk0hSTrGAkHJAnz5w\n//1wzTVw5JGp00gC4P33s5GIJ59MnUSScooFhJTYv/4FF12Urdu88srUaSQttuee0KYN9O2bOokk\n5RQLCCmxddaB7beH/v1dMC3llBCyEYi33oKhQ1OnkaSc4eWKlMB332X7PEBWPLzzTrbfg6Qcc/LJ\nWVeme+5JnUSScoYFhFTFYoTTT4fLLoNx47JjjjxIOaq0FA4/HL74InUSScoZtnGVqtiTT8KAAdCr\nFzRrljqNpJW67z6oXTt1CknKGb7vKVWhKVPg3HNhhx3gvPNSp5FULouKh6lT0+aQpBxhASFVoUsu\ngUmToF8/qO74n5Q/XnwRmjaFDz5InUSSkitXARFCOCCE8EUIYUQIoftyzjk6hDA0hPB5COHhio0p\n5a8ffoDx47P7xx4LN9+cLZyWlEc6dMiq/jvuSJ1EkpJbaQERQigB7gQOBLYAOocQtljqnDbAZcCu\nMcYtASdnSMDkybDVVtC9rOzeZx+44IK0mSSthrXWyjoyPfwwTJiQOo0kJVWeEYj2wIgY46gY41zg\nUeDQpc45HbgzxjgFIMbob1cVvRjhtNOyTkvHHZc6jaQ1du65MHeuG8tJKnrlKSCaAd8s8Xhs2bEl\nbQpsGkJ4O4TwnxDCAcv6QiGEM0IIQ0IIQyZOnLh6iaU8cddd8P/+H/zxj7D//qnTSFpjbdvCb36T\n/edesCB1GklKpqKWcVYH2gB7Ac2BN0IIW8cYf9ayIsbYF+gL0K5du1hB31vKOcOGZVOVDjjAbktS\nQenVK+vKVFKSOokkJVOeAmIc0GKJx83Lji1pLPBOjHEe8FUI4b9kBcV7FZJSyjPrrw9t2mTt490k\nTiogW2+dOoEkJVeeS5v3gDYhhNYhhJrAscCzS53zDNnoAyGEJmRTmkZVYE4p5336KRx1FMyfn623\nfPddWG+91KkkVbhRo+CII2Do0NRJJCmJlRYQMcb5wDnAi8Aw4PEY4+chhJ4hhI5lp70ITA4hDAVe\nBS6OMU6urNBSrpk1C44+Gt56C74pWzHkxrVSgWrQINtO3sXUkopUudZAxBgHAgOXOtZjifsRuKDs\nJhWdSy+F4cPhX/+C1q1Tp5FUqUpLsxGI/v3hxhuhTp3UiSSpSjk7W1pDL74If/5ztlh6n31Sp5FU\nJc48E6ZOhccfT51EkqqcBYS0BmKEyy6DzTeHG25InUZSldlzT9hss6ylqyQVmYpq4yoVpRBg0CD4\n/ntnMUhFJYRsi/mvv872hLCtq6QiYgEhrabZs6FmTVh33ewmqcicckrqBJKUhFOYpNX0hz9A+/Yw\nZ07qJJKSmT8fnnoKJk1KnUSSqowFhLQaxo+HO+/M1j7UqpU6jaRk/vvfrCPTPfekTiJJVcYCQloN\nN94Ic+fCVVelTiIpqS22yNqv/eUvMG9e6jSSVCUsIKRV9NlnWeOVk0+GTTZJnUZSct26wdix8Mwz\nqZNIUpWwgJBW0c03w1prZaMQksRBB8FGG0GfPqmTSFKVsICQVlG/fvDKK3ZeklSmpATOPRe++w6m\nTEmdRpIqnQWEVE5Dh2Ybz9asCVtumTqNpJzy+9/D8OHQuHHqJJJU6SwgpHKYPx+OOgo6dkydRFJO\nqlkzG4mYNSvbWVKSCpgFhFQO992XjUCcd17qJJJy1uzZ0KaN7dkkFTwLCGklfvghux7YZRc47LDU\naSTlrNq1Yf/94d57YfLk1GkkqdJYQEgrcdtt8O230KsXhJA6jaScdtFF2TSmv/41dRJJqjQWENJK\nfPwxdOoEu+6aOomknLfllnDggfDnP2e7TUpSAbKAkFbi8cfh4YdTp5CUN7p2zVq6DhqUOokkVQoL\nCGkZFi6Eiy+GkSOzx3XqpM0jKY/svz+88w4cckjqJJJUKSwgpKXECF26QO/e8PzzqdNIyjvVqkH7\n9i6aklSwLCCkJUyfnu33cBKIEXUAACAASURBVNddcOml2UwESVotF12UbTAnSQXGAkIq8/XX0K4d\nPPNM1nHpxhtTJ5KU12bPhr/9DSZMSJ1EkiqUBYRUpmlT2GwzePXV7I1DZx9IWiPnnJN1YurbN3US\nSapQFhAqejNmwI8/ZntADRgAu++eOpGkgtC2bdbS9fbbs18yklQgLCBU1GKEU06BvfaC+fNTp5FU\ncC69FCZOhPvuS51EkiqMBYSK2i23wFNPwTHHQPXqqdNIKjh77AHXXQf77JM6iSRVGC+ZVLRGj4Yr\nr8x2mT7//NRppPwXQvgamAEsAObHGNulTZQDQoArrkidQpIqlCMQKlrdu2ft2m+/3QXTUgX6dYxx\nO4uHpXz4YdadIcbUSSRpjVlAqCj98AMMG5btNt2iReo0kgrehx9mcyb/9a/USSRpjVlAqCjVqwfv\nvw+XX546iVRQIvBSCOH9EMIZyzohhHBGCGFICGHIxIkTqzheQscfn/WKvvXW1EkkaY1ZQKjoDBkC\nU6ZASQnUqpU6jVRQdosx7gAcCHQJIeyx9Akxxr4xxnYxxnalpaVVnzCVWrWgSxd44YVs+FOS8pgF\nhIrKvHlZx6Wjj06dRCo8McZxZR8nAE8D7dMmyjFnnZVtOHPbbamTSNIasYBQUbn/fhg1Crp1S51E\nKiwhhHohhAaL7gP7A5+lTZVjmjTJfvm48EpSnrONq4rG3LlZO/addoKDD06dRio46wFPh6ylWXXg\n4RjjoLSRctAf/5g6gSStMQsIFY3+/eHrr+Evf7Ftq1TRYoyjgG1T58gLCxfCwIGw334uxJKUl5zC\npKLx/vuw885wwAGpk0gqaq+/DoccAo88kjqJJK0WCwgVjbvugpdfdvRBUmJ77QXbbJPtC+HGcpLy\nkAWECt748TB8eHa/bt20WSSJEODCC+Gzz+Cll1KnkaRVZgGhgnfxxdnC6WnTUieRpDLHHgsbbAC9\ne6dOIkmrrFwFRAjhgBDCFyGEESGE7st4/pQQwsQQwkdlt99VfFRp1b3xBjz4IJx3HjRqlDqNJJWp\nWRO6doWRI313Q1LeWWkBEUIoAe4k21l0C6BzCGGLZZz6WIxxu7JbvwrOKa2yefOyjV833BAuuyx1\nGklaSrdu8N//+u6GpLxTnjau7YERZS36CCE8ChwKDK3MYNKauvXWbIrx00+79kFSDqpdO/s4e3b2\njkeDBmnzSFI5lWcKUzPgmyUejy07trQjQgifhBCeDCEsc5vNEMIZIYQhIYQhEydOXI24UvnNnAlH\nHgmdOqVOIknLMXMmtG4NN9+cOokklVtFLaIeALSKMW4D/BPov6yTYox9Y4ztYoztSktLK+hbS8vW\nsyc89ljqFJK0AvXrQ4cOWZ/pH39MnUaSyqU8BcQ4YMkRheZlxxaLMU6OMc4pe9gP2LFi4kmr7tln\n4ZVXsvvV7DMmKdd16waTJsHDD6dOIknlUp7Lq/eANiGE1iGEmsCxwLNLnhBCWH+Jhx2BYRUXUSq/\nGTPg9NPhiivcn0lSnli0sdxtt/mLS1JeWGkBEWOcD5wDvEhWGDweY/w8hNAzhNCx7LSuIYTPQwgf\nA12BUyorsLQit9wCEyZAnz7uOC0pT4SQ9Zr+7DP4979Tp5GklSpPFyZijAOBgUsd67HE/csAG2Uq\nqfHjsz2ZjjwS2rdPnUaSVkHnzrDRRvCrX6VOIkkrVa4CQsoHPXvCnDlwww2pk0jSKqpdG/bcM3UK\nSSoXl5iqYGyzDXTvDm3apE4iSavpyiuzhVySlMMsIJTX5s7NNooDOOssuPbatHkkaY3MmQP33guj\nRqVOIknLZQGhvPXZZ9C2LRx+OLzzTuo0klQBzj8fqlfPFnRJUo6ygFBeihG6dMnatr7wgoumJRWI\nDTaAk0/ORiHGj0+dRpKWyQJCeenZZ+GNN7KF0wccYMtWSQXk4ouz+Zl9+qROIknLZBcm5Z0Ys43i\n2raF3/0udRpJqmBt2sAdd8Cvf506iSQtkwWE8k4I8MQTMHUq1KiROo0kVYIuXVInkKTlcgqT8tLm\nm8Muu6ROIUmVaOhQOOOMrDOTJOUQCwjllUceybouTZ2aOokkVbJvv4V77oEHHkidRJJ+xgJCeeXv\nf4cPP4RGjVInkaRKts8+sOOOcPPNsGBB6jSStJgFhPLG+PHw8stw/PF2XZJUBEKASy+FL7/MWs9J\nUo6wgFDeePRRWLgwKyAkqSgcfji0bu3GcpJyil2YlDcefBB22CFbQC1JRaGkBP7wBxg+HObNs/Wc\npJxgAaG8sGABHHwwbLJJ6iSSVMVOPTV1Akn6GQsI5YWSErjmmtQpJCmRGGHQoGwHzdatU6eRVORc\nA6GcN2dO1r514cLUSSQpkQkToFMn10JIygkWEMp5ffrAccfB22+nTiJJiay3HpxwQtbLesqU1Gkk\nFTkLCOW08ePhuuvg//4Pdt89dRpJSujcc+HHH+G++1InkVTkLCCU0664AmbPhltuSZ1EkhLbbjvY\nbTe4807ndEpKygJCOeuDD7LR+m7dYNNNU6eRpBzQpQvMnQujR6dOIqmIWUAoZ333HfzmN3DllamT\nSFKOOPJIGDXKTkySkrKNq3LWgQdmN0lSmeplL9tz52brIRo1SptHUlFyBEI5adgwmDo1dQpJykFz\n5kCbNtCjR+okkoqUBYRy0mmnwX77pU4hSTmoVi3YZx+46y74+uvUaSQVIQsI5ZyRI2Hw4GyqryRp\nGXr2hJKSrFWdJFUxCwjlnIcfzj4ed1zaHJKUs5o3hwsuyH5hDhmSOo2kImMBoZwSIzz0EOy5J7Ro\nkTqNJOWwSy6B0tLsl6YkVSG7MCmnDB0KX3wBF12UOokk5biGDeG996Bly9RJJBUZCwjllC23hOHD\nYf31UyeRpDyw4YbZxylTYK21IIS0eSQVBacwKSeMG/fTKPxmm2VvrEmSymHw4GxNxCuvpE4iqUhY\nQCi5qVNh773h97+HCRNSp5GkPLPDDtmGcjfckDqJpCJhAaHkrrkGRoyAAQNg3XVTp5GkPFOrVrZw\n7JVX4D//SZ1GUhGwgFBSw4fDn/8Mv/sd7LFH6jSSlKfOOAPWXhtuvDF1EklFwAJCSV10EdStC9de\nmzqJJOWx+vWhWzd47jkYMyZ1GkkFzgJCycQIp50Gffo4dUmS1ljXrvDBB7Z1lVTpbOOqZEKAww5L\nnUKSCsRaa2U3gLlzoWbNtHkkFSxHIJTE0KHZtKVJk1InkaQCc/75sP/+2TCvJFWCchUQIYQDQghf\nhBBGhBC6r+C8I0IIMYTQruIiqhA98ghcfbWvb5JU4TbZBF5/HV58MXUSSQVqpQVECKEEuBM4ENgC\n6BxC2GIZ5zUAugHvVHRIFZ5//AP23BNKS1MnkaQCc/rp0Lo1XH6579JIqhTlGYFoD4yIMY6KMc4F\nHgUOXcZ51wI3AbMrMJ8K0LBh2e3ww1MnkVTRQgglIYQPQwjPpc5StGrWhB494MMPYdCg1GkkFaDy\nFBDNgG+WeDy27NhiIYQdgBYxxudX9IVCCGeEEIaEEIZMnDhxlcOqMDz1VPbRBdRSQeoGDEsdougd\ndxy0aAF/+lPqJJIK0Bovog4hVAP+BFy4snNjjH1jjO1ijO1KnbtStKZMgb32gmbNVnqqpDwSQmgO\nHAz0S52l6NWsCQ8/DPffnzqJpAJUnjau44AWSzxuXnZskQbAVsBrIQSApsCzIYSOMcYhFRVUhaN3\nb1i4MHUKSZXgNuASstcFpbbbbqkTSCpQ5RmBeA9oE0JoHUKoCRwLPLvoyRjjtBhjkxhjqxhjK+A/\ngMWDlmnOnOxjNRsISwUlhPB/wIQY4/srOc+prFXpv/+FPfaATz9NnURSAVnpZVyMcT5wDvAi2bzW\nx2OMn4cQeoYQOlZ2QBWOefNg++3ht79NnURSJdgV6BhC+Jqs2cbeIYQHlz7JqaxVrEmTbHfqG25I\nnURSASnX+8AxxoExxk1jjBvHGK8vO9YjxvjsMs7dy9EHLcudd2bdl1w8LRWeGONlMcbmZSPRxwKv\nxBhPSBxLa68N554Ljz0GH3+cOo2kAuFEElWJCRPgqqvgN7+B//u/1GkkqYhccgk0agRXXJE6iaQC\nYQGhKnHFFTBrFtx2G2Rr7SUVqhjjazFG3yrIFY0bQ/fu8Pzz8NZbqdNIKgDl6cIkrZFp0+DZZ6Fr\nV2jbNnUaSSpC554LdevCjjumTiKpAFhAqNI1agTDh0NJSeokklSk6tbNighJqgBOYVKlWrgQYsxG\n0Bs2TJ1GkorcgAFwyCGwYEHqJJLymAWEKtU//gGbbQajR6dOIknixx/huefgoYdSJ5GUxywgVKke\nfhhmzoTmzVMnkSRx5JHZOogePX7a2VOSVpEFhCrNlCkwcCAce6zrHyQpJ1SrBjfemA0L33VX6jSS\n8pQFhCrNU0/B3Llw3HGpk0iSFttvP9hnH7juumyIWJJWkV2YVGkefhjatLFroCTlnF69YORIqFcv\ndRJJecgCQpWmSxeYP9+N4yQp52y/fXaTpNVgAaFKc/jhqRNIklbollvgiy+gb9/USSTlEddAqMIt\nWAA33QTjxqVOIklaoalT4Z574D//SZ1EUh6xgFCFGzQIunf39UiSct6ll0LTpnDBBdmun5JUDhYQ\nqnB33ZW9HnXsmDqJJGmF6tfPujENHgyPPpo6jaQ8YQGhCjV6NDz/PPzud1CjRuo0kqSVOuWUbEF1\n9+4wb17qNJLygIuoVaHuuSfrunT66amTSJLKpaQkW0Qdo+/8SCoXCwhVqMmTs6lLLVumTiJJKrd2\n7X66v3BhtmO1JC2HBYQq1F//mr32SJLy0LnnZp2ZHnggdRJJOcy3GFQhZsyATz7J7vvGlSTlqbXW\nggcfhLfeSp1EUg7zUk8Vok8f2G47GDkydRJJ0mrr3h1atMhGIhYsSJ1GUo6ygNAa+/576NUrW/uw\n8cap00iSVlu9etC7N3z0UdYVQ5KWwQJCa+ymm7IpTNdemzqJJGmNHXUU7LUX3HADzJ+fOo2kHOQi\naq2R99+HP/0JTjoJtt46dRpJ0hoLIRt9qFULqnuZIOmX/M2gNTJ0aNay9dZbUyeRJFWYTTbJPsYI\n06Zli6slqYxTmLRGTjwRhg2Dxo1TJ5EkVbjjj4eDDrI/t6SfsYDQavn3v+Hxx7P7NWumzSJJqiT7\n7QeDB2etXSWpjAWEVsull8JFF8Hs2amTSJIqzcknQ4cOcMkl2VQmScICQqvh7bezPYYuughq106d\nRpJUaapVgzvugAkT4LrrUqeRlCMsILTKbroJ1lkHTjstdRJJUqXbaSc49VR44gmHnSUBdmHSKvrs\nMxgwAK6+OttvSJJUBHr1yha8OewsCUcgtIomTYLttoNzzkmdRJJUZdZeG+rXh7lzYdSo1GkkJeYI\nhFbJXnvBhx+mTiFJSuKII+DLL+GTT2zBJxUxRyBUbqNHw4wZqVNIkpI5+2z44otsMZykomUBoXI7\n99xsLZ0kqUgddBAcfXTWkWn48NRpJCViAaFymT8fXnsN9twzdRJJUlJ9+kDdunDGGe5QLRUpCwiV\ny3vvZdOX9t03dRJJUlJNm0Lv3jBzJkyenDqNpAQsIFQuL7+cffz1r9PmkCTlgN/+Ft59F0pLUyeR\nlEC5CogQwgEhhC9CCCNCCN2X8fxZIYRPQwgfhRDeCiFsUfFRldK//gXbbw9NmqROIklKLgSoXh2m\nT4cuXWDKlNSJJFWhlbZxDSGUAHcC+wFjgfdCCM/GGIcucdrDMca7ys7vCPwJOKAS8iqR22/39UGS\ntJRhw+Cee7K9IZ57DkpKUieSVAXKMwLRHhgRYxwVY5wLPAocuuQJMcbpSzysB8SKi6hcsM02LqCW\nJC2lQwe44w4YNAiuvTZ1GklVpDwFRDPgmyUejy079jMhhC4hhJHAzUDXZX2hEMIZIYQhIYQhEydO\nXJ28SuCpp+CZZ1KnkCTlpDPPhOOPh+uvh48/Tp1GUhWosEXUMcY7Y4wbA5cCVy7nnL4xxnYxxnal\nLrzKG9ddl3XtkyRpmfr0gbXXhgsvTJ1EUhUoTwExDmixxOPmZceW51Gg05qEUu4YPRo++gj22Sd1\nEklSzlpnnWy4+sEHUyeRVAXKU0C8B7QJIbQOIdQEjgWeXfKEEEKbJR4eDHxZcRGVUp8+UK0anHRS\n6iSSpJy2667ZHhELFsD48anTSKpEK+3CFGOcH0I4B3gRKAHujTF+HkLoCQyJMT4LnBNC2BeYB0wB\nTq7M0Koa06ZBv35w9NHQsmXqNJKkvHDCCfDpp9k+EXXrpk4jqRKstIAAiDEOBAYudazHEve7VXAu\n5YCRI7N9H5zSKkkqt1NPhQMOgHPOgXvvTZ1GUiVwJ2ot1w47wIgRsOOOqZNIkvLG/vvDFVfA3/8O\n/funTiOpElhAaJlGjYLZs7P1D5IkrZKrr4a99oLf/z57QZFUULw81DIdf3z2JpIkSauspAQeeADa\ntoXJk1OnkVTByrUGQsVl5Ej4z3/g5ptTJ5Ek5a3mzWHIEAghdRJJFcwRCP3Co49mH485Jm0OSVKe\nCwFmzoRrr80+SioIFhD6hUcfzdp527pVkrTGPvsMevSAq65KnURSBbGA0M8MHZr9ru/cOXUSSVJB\n2HlnOPNM+NOf4JZbUqeRVAFcA6Gf2XzzbP3DJpukTiJJKhh33AHffw8XXQTz5kH37qkTSVoDFhD6\nmRCgQ4fUKSRJBaVGDXj44exj797w29/CuuumTiVpNTmFSYt99BGcdRZ8+23qJJLyTQihdgjh3RDC\nxyGEz0MI16TOpBxTvTrcfz+8+67Fg5TnLCC02N13w733Qp06qZNIykNzgL1jjNsC2wEHhBB2TpxJ\nuaakBDbaCBYsyObLSspLFhAC4PPP4Z574LTToHHj1Gkk5ZuYWdSns0bZLSaMpFx2/fWw++7w3/+m\nTiJpNVhAiBjh/POhQYOsVbckrY4QQkkI4SNgAvDPGOM7yzjnjBDCkBDCkIkTJ1Z9SOWGM8+EWrXg\n8stTJ5G0GiwgxIAB8M9/wjXXQJMmqdNIylcxxgUxxu2A5kD7EMJWyzinb4yxXYyxXWlpadWHVG5Y\nbz245BL4xz+yFyBJecUCQnToAFdcAWefnTqJpEIQY5wKvAockDqLctgFF0DbtnDYYfDmm6nTSFoF\nFhBivfXguuuy7nqStDpCCKUhhLXK7tcB9gOGp02lnFa/Prz6Kuy6K2ywQeo0klaBBUSRu/NOePbZ\n1CkkFYD1gVdDCJ8A75GtgXgucSbluqZN4cUXYeONswV5o0alTiSpHCwgitiCBfCHP8Azz6ROIinf\nxRg/iTFuH2PcJsa4VYyxZ+pMyjM9e8IOO8DHH6dOImklLCCK2HvvwZQpcICzlCVJqZ1yStYO8De/\ngS+/TJ1G0gpYQBSxQYOgWjXYd9/USSRJRW/DDbOOTAsWZO9szZiROpGk5bCAKGIvvADt28Paa6dO\nIkkSWVemp5+Gr7+GCy9MnUbSclRPHUBpzJkD06ZB586pk0iStITddoM//hG22SZ1EknLYQFRpGrV\nguHDYf781EkkSVrKxRf/dD9GCCFdFkm/4BSmIhVj9rG6JaQkKVdde202VL7oRUtSTrCAKEILF8JW\nW8Htt6dOIknSCtStC489lhUSknKG7z8XoTffhKFDXTwtScpxF1yQ7Qtx1VWw5ZZwxBGpE0nCEYii\ndOed0LgxHH546iSSJK1ACNC3L+y8M5x0EnzwQepEkrCAKDrffpt1yPvtb7ORYUmSclrt2vDUU7De\neln3D0nJOYWpyNxzT9Z56ayzUieRJKmc1l8fPv8c6tTJHs+Zk7UTlJSEIxBF5qijssXTm2ySOokk\nSatgUfHw4ouw2WYwYkTaPFIRcwSiyGyxRXaTJCkvbbghzJgBBx8MgwfbEURKwBGIIhEjXH01vPde\n6iSSJK2Btm3hmWfg66/hsMOy6UySqpQFRJHo1w+uuQaGDEmdRJKkNbT77vD3v8Mbb2RdQRYuTJ1I\nKipOYSoCo0ZlrbT33hvOPDN1GkmSKsBxx2WjEOPGZe1eJVUZC4gCt2ABnHIKVKuWvVlTzTEnSVKh\nuPzybI5uCNkOqQ0aQIsWqVNJBc/LyQL3l79kO0/ffju0bJk6jSRJFSyEbArTMcdAu3bwyiupE0kF\nzwKiwB1/fLZx3EknpU4iSVIlqVYNHn8c1lkH9tsPbrjBdRFSJbKAKHBrrw2dOjk9VJJU4DbfHN59\nNxuJuOIK6NYtdSKpYJWrgAghHBBC+CKEMCKE0H0Zz18QQhgaQvgkhPByCGHDio+qVdW/fzaFSZKk\nolC/Pjz0EJx3HgwfbotXqZKstIAIIZQAdwIHAlsAnUMIS29F9iHQLsa4DfAkcHNFB9WqiRH++Ed4\n4onUSSRJqkIhQK9e8MILUKtW6jRSQSrPCER7YESMcVSMcS7wKHDokifEGF+NMc4qe/gfoHnFxtSq\n+vzz7M2Xo45KnUSSpCpWvXp2+9//oHNn+Pbb1ImkglKeAqIZ8M0Sj8eWHVue04AXlvVECOGMEMKQ\nEMKQiRMnlj+lVtkTT2Rvwhx+eOokkiQlMm4cDBgAHTrAJ5+kTiMVjApdRB1COAFoB/Ra1vMxxr4x\nxnYxxnalpaUV+a21lCeegD32gKZNUyeRJCmRdu3grbeyeb277ppNa5K0xspTQIwDltyVpXnZsZ8J\nIewLXAF0jDG6aimh6dOz7kvHHJM6iSRJiW23HbzzDrRpA4ccAi++mDqRlPfKsxP1e0CbEEJrssLh\nWOC4JU8IIWwP3A0cEGOcUOEptUoaNvzpDRdJkopes2bw2mtw9tmw5Zap00h5b6UjEDHG+cA5wIvA\nMODxGOPnIYSeIYSOZaf1AuoDT4QQPgohPFtpibVCI0fC2LHZffd+kCSpTMOGWYvX5s1hwYKfXiwl\nrbLyjEAQYxwIDFzqWI8l7u9bwbm0GmKEM8+EL76Ar77KGlBIkqSldO0KTz0FL78MWyzdmV7SyrgT\ndQEZODD7XXjJJRYPkiQtV5cu2cc994QPP0ybRcpDFhAFYt48uOgi2HRTOOus1GkkScphW2wBb7wB\nderA3nvD4MGpE0l5xQKiQNxzT7ZxXK9eUKNG6jSSJOW4Nm3gzTehSRM47DCYNWvlnyMJKOcaCOW+\nr7/O3kQ55JDUSSRJyhMbbgj//jd8+SXUrZs6jZQ3LCAKxM03Z9OY7LwkSdIqKC3NbgC33gqvvw53\n3AEtWqz486Qi5hSmPPfttzBkSHbfqUuSJK2BhQvhpZdg882hTx83VJKWwwIiz11xBey+O0yenDqJ\nJEl57sILYejQrDvTeefB6afD/PmpU0k5xwIij40ZA/ffn22suc46qdNIklQAWrWC556DK6+E/v3h\ngw9SJ5JyjgVEHrvrruxjt25pc0iSVFBCgGuvhc8+g/bts2MLFqTNJOUQC4g8NXt21rq1Y8esiYQk\nSapgm22WfXzySejQAb77Lm0eKUdYQOSpTz+FuXPhnHNSJ5EkqcA1aADDhsGuu8KIEanTSMlZQOSp\nnXaCceOyvR8kSVIl+s1v4OWXYcoU2GEHuO8+OzSpqFlA5KEZM7LfW/Xru++DJElVYued4f33Yfvt\n4dRTs12spSJlAZGHTjsNdtvNNz8kSapSrVrBK6/As8/CHntkx1xcrSJkAZFn3nwTnngC9t3X0QdJ\nkqpcSQkcckh2//33Ycst4ZNP0maSqpgFRB5ZsADOPRdatIBLL02dRpKkIlezJsycmS2uvuqqbI2E\nVAQsIPJIv37w8cfQuzfUrZs6jSRJRW7rrWHwYNh/f+jZM5vidNNNqVNJlc4CIk/ECA8+CHvuCUcd\nlTqNJEkCsmkB//hH9g7fnnu6QFFFoXrqACqfELIOcpMnu/ZBkqScs8028P/+308v0p9/Dm3aZNOc\npALjCEQeePFFmDYt+x20/vqp00iSpGVaVDxMnpy1SzzwwOy+VGAsIHLckCHQsaOLpiVJyhvrrAO3\n3w5vvQXt22ejEVIBsYDIYVOnZusdmjaF669PnUaSJJXbiSfC66/DrFnZJnS+kKuAWEDksCuvhDFj\n4PHHszczJElSHtl552wqwV57wahRPx0fNixZJKkiWEDkqA8+gL/+Fbp0gQ4dUqeRJEmrpVkzGDAg\n68UOMHBgtvncZZfB/Plps0mryQIiR629Nhx/PFx7beokkrRyIYQWIYRXQwhDQwifhxC6pc4k5ZRF\nC6x//Ws4/XT44x+zhdbvvps2l7QaLCByVKtWcP/90KhR6iSSVC7zgQtjjFsAOwNdQghbJM4k5Z46\ndeDuu+Hhh+Grr7JpBuedlzqVtEosIHLM1Klw3HHw9depk0hS+cUY/xdj/KDs/gxgGNAsbSoph3Xu\nDCNGZAse27VLnUZaJRYQOaZXL3jkkayQkKR8FEJoBWwPvJM2iZTjGjTI5iqfcEL2uG9feOyxtJmk\ncrCAyCHjx8Ntt8Gxx8J226VOI0mrLoRQH/gHcF6Mcfoynj8jhDAkhDBk4sSJVR9QylULF2bvIB57\nLFxyCfz4Y+pE0nJZQOSQ666DuXNdOC0pP4UQapAVDw/FGJ9a1jkxxr4xxnYxxnalpaVVG1DKZdWq\nwQsvZAuse/WC7beHf/87dSppmSwgcsSoUdnI5WmnwSabpE4jSasmhBCAvwHDYox/Sp1Hyku1a2cX\nAy+9lI1A7Lnn/2/vzqOrKs89jn8fCCBBy6yMClbABgUFRK1jcQIHUMAColKH0lYszrewrFqtdSmK\nw3UhVyu9oEXQi6AocBGHtlpFSJkuiGBAVBABRUWxKiHv/ePZWYmMh+QkO3vn91lrr+zhAM/Lezgv\nz3kn+PjjuKMS2YkSiCqiQQNfhOGWW+KORESkTE4ALgF6mNmi6Dg77qBEEumMM2DpUh/S1KKF3xs3\nDhYvhhDijU0EyIk7gOpu0SKoVcv3lBk1Ku5oRETKJoTwBmBxxyGSGgccAP37+/maNfCb38C2bZCX\nBzfeCJddFmt4Ur2ppe0fQAAAEZJJREFUByJGL7zgu9xfo+2WREREZHfatPGhTI88Arm5cPnl/p+H\n7dvjjkyqKSUQMVm3DoYM8Z6HSZPijkZERESqtCZNvBdi7ly47jqYOhU++yzuqKSaUgIRg6Ii73n8\n7jtPHrQQiYiIiGSkZk24/36fD3HggT7Z+v774euv445MqhElEDGYMAHmzIH77oP27eOORkRERBKn\nUSP/OXMm3HADtG0LDz3k68GLVLCMEggz62lmK8yswMxG7OL5yWa2wMwKzax/9sNMlwEDYMwY+PWv\n445EREREEq1fPx/W1KmTL+f4k5/4btZarUkq0F4TCDOrCYwBegF5wCAzy9vhZR8CvwCeynaAafLv\nf8NXX/n8p6uuAtN6JSIiIlJexx4LL7/sG9HVqwcPP1zyTImEVIBMeiC6AwUhhNUhhO+ByUCf0i8I\nIawJISwBiiogxtQYPhy6d9fu9CIiIpJlZtCzJyxcCFOm+PUnn0CXLjBtmhIJyapMEoiWwEelrtdG\n92QfPPUUPP449O0LdevGHY2IiIikUs2a0KyZn2/c6Cu29O0LJ58Mkyf7MpAi5VSpk6jNbKiZ5ZtZ\n/qZNmyrzj47VggUwdCicdBLcfnvc0YiIiEi10KkTLFkCjz4KBQUwaJBPttZQCCmnTBKIdUDrUtet\nonv7LITwWAihWwihW9NqsnbpmjVw9tnQuLEn/jna+1tEREQqS06Of4v50UeQnw/jx/tQiBBg7FjY\nujXuCCWBMkkg5gPtzKytmdUGBgLTKzas9Khb178AmDULWrSIOxoRERGplnJyoGtXuOgiv87Ph2HD\nfI7E3/8eb2ySOHtNIEIIhcDVwGxgOfBMCGGZmd1hZr0BzOwYM1sLXAg8ambLKjLoJPjqKx92eNBB\n8NJLkLfjulUiIiIicTnmGHjlFd834tRTYcgQnzMhkoGMBtSEEGYCM3e4d2up8/n40CYBtmyBXr18\ng8ipU7Vcq4iIiFRBP/sZLFsGd97pu9suXgyLFsUdlSSAdqLOsuLkYd48uPhiJQ8iIiJSheXmwl13\n+WTr++7ze998A4MHw+zZUKQV+mVnmtKbRZs3e/KwYIFPmO7XL+6IRERERDJw+OF+ACxfDnPm+Br0\nbdr4/hLHHw/nnQcNG8YaplQN6oHIov79vedvyhQlDyIiIpJQXbv6qk2TJ3tSMXGiz5FYv96fv/66\nP9PmdNWWEogsuvdemDED+vTZ+2tFREREqqw6dWDAAF9G8vPPfYhTcQ/FlCm+p8Tpp8OqVfHGKbFQ\nAlFO27fDc895Et61q/9bEhEREUmNmjXhyCOhRvTfxgce8M3p8vP9/siR8N578cYolUoJRDndfTdc\ncAG8+mrckYiIiIhUgho1fHO6Zct8t9xRo7ynAqCw0A9JNSUQ5fDmm3Dbbd6L16NH3NGIiIiIVKJW\nrXw407p1PkcCYMIE6NgRnngCVq70TbEkdbQKUxktX+6Jw8EH+07wWq5VREREqqVmzUrODz7Y508U\nJxRm0LkzvPEG1KsXT3ySdUogymDrVjjlFD+fNQvq1483HhEREZEq4YwzfEnKuXOhoABWr/b7xcnD\n11/D/vvHF59khRKIfbBtG9Sq5f8Gxo+Ho46CFi3ijkpERESkCqlRA376Uz9Kmz8fTjzRj3794KKL\noEGDeGKUctEciAwVFED37p44gM8ZUvIgIiIikqFGjeCmm3zOxLBhcOihvgb+t9/GHZnsIyUQGXj+\neV+i9YMPoGnTuKMRERERSaAf/xjuvNMnks6fD8ce68tZFk+0/uUv4eST4corYfRoHwolVZISiL14\n8EFfprVDB1i4EM45J+6IRERERBLMDLp184mkS5eWTCZt2NA31po+HW68EY4+Gq6/Pt5YZZc0B2IP\n5s2D666Dvn3hr3+FunXjjkhEREQkRZo3LzkfNarkfONGeOghaN/er7dsgRUrPPHQ0pexUw/EHnTv\nDrNnwzPPKHkQERERqTQHHgh/+lPJcrBPP+3/MTviCJg4EYqK4o2vmlMCsQsvvujLFQOceabv4C4i\nIiIiMbnwQnjsMV8O8+KLoUsX/5a32NSpMGOGD4nSTtgVTgnEDlau9FXFRozwYXgiIiIiErMGDXyS\n9YIF8NRTPqSp9JCn3/4Wzj0XjjzSJ64+8QRs3x5fvCmnBKKUTz+F/v2hdm1/b2qInYiIiEgVUqMG\nDBoE774LY8aU3J8/H956CyZM8EnZQ4bAXXf5s+3b9a1wlmkSdWTxYjj/fFi/Hl54wXdiFxEREZEq\nqHZtOPzwkusWLfw47ji45BKYNq1kI7tp02D4cF9GtnVr6NTJX9OyZTyxp4B6ICKTJvlO06+/7ruw\ni4iIiEgCmfkSms2a+XXTpnD66T6pde5cGDkSDjnEN7QD9U6UQbXtgdi+HaZMgdxcOO8839fk+ut9\n0r+IiIiIpMQpp/hRrKAAXnqppAdi4EAfxz5gAPz85z7fQvaoWvZAzJ4NnTv7++XPf/Z7OTlKHkRE\nRERS77DD4KqrSq47dvTeiF/9ynstevf2DcCKbdhQ+TFWcdUqgViyBHr29OPbb31J4eeeizsqERER\nEYnNrbfC8uU+EXvoUJ+g/d57/mzDBp9bccwxcMcd8M9/+pj3ai71Q5iKirye69Tx98Pbb8Po0TBs\nmN8TERERkWrOzHe57tbth/dr1YK774Znn4U//AFuuw3239/HwZ91Frz/Prz5pq/8dMABPjY+N9cn\nbO+3XyxFqQypTSCKirxu//hHH6p0880+n6ZXL69fEREREZE9atQIbrrJj88+g7/9DV57DQ491J+/\n/nrJbtmlTZ0KF1xQqaFWplQmEF9+6SspzZ/vK3zl5fn9nBwlDyIiIiJSBo0bQ79+fhTr18+Xjt2y\nxY9vvvFei7PO8ueffuq/bs0aWLXKl5bNzY0l/GxKXQJRWOgT6Bcu9L1EBg/2VbtERERERLKqXj1o\n337Xz958E84808fMb97s9xo18l6L4m+3Eyp1CcTKlTBvHjz6KFx6adzRiIiIiEi11LGjb1hXWOhz\nK5o3hxkzoEMHf37ttZCfDyecAFdeCe3axRvvPkhNAlFY6PuA5OX5xPkmTeKOSERERESqrfr1YezY\nH97r3bvkvHlz/zl6NIwa5ZvdXX45DBrk92+5xZcNzc31oVKdOlVO3BlIxTKuCxbAscfCPff4tZIH\nEREREanSfvc7eOMN+PBDX/Vn1Sp4+eWS5w8/DGPG+G7HnTvDUUfBrFnxxVtKYhOIoiL/Ox40CLp3\nh48/TvxwMhERERGpblq0gN//HlavhnHjSu5/8YVPyt6wwZOJ2rV9pSCAZcs8qTj6aP8WfdgwmD4d\ntm6tlJATm0AMH+4rLc2e7efvvOPLtIqIiIiIpEaTJnD11T7Jd8AAv/fJJ76UbOvWvi/FhAnQpw8s\nXerPn3wSVqyosJASOwfissvgpJP87yrF+3SIiIiIiDgz/3naaX4U++473yW7eCO8H/0IGjSosDAS\nm0B07eqHiIiIiEi1VqcO9OhRct2nT4X+cYkdwiQiIiIiIpVPCYSIiIiIiGRMCYSIiIiIiGRMCYSI\niIiIiGQsowTCzHqa2QozKzCzEbt4XsfMno6ev21mbbIdqIiIiIiIxG+vCYSZ1QTGAL2APGCQme24\nZdsVwOchhMOAB4B7sh2oiIiIiIjEL5MeiO5AQQhhdQjhe2AysOPaUH2ACdH5FOA0s+KFakVERERE\nJC0ySSBaAh+Vul4b3dvla0IIhcCXQONsBCgiIslgZn8xs41mtjTuWEREpOJU6iRqMxtqZvlmlr9p\n06bK/KNFRKTijQd6xh2EiIhUrEwSiHVA61LXraJ7u3yNmeUA9YHPdvyNQgiPhRC6hRC6NW3atGwR\ni4hIlRRC+AewOe44RESkYmWSQMwH2plZWzOrDQwEpu/wmunAkOi8P/BqCCFkL0wREUkD9USLiCTf\nXhOIaE7D1cBsYDnwTAhhmZndYWa9o5eNAxqbWQFwPbDTUq8iIiLqiRYRSb6cTF4UQpgJzNzh3q2l\nzr8FLsxuaCIiIiIiUtVoJ2oREREREcmYEggREckKM5sEvAV0MLO1ZnZF3DGJiEj2ZTSESUREZG9C\nCIPijkFERCqexbVYkpltAj4o52/TBPg0C+FURSpbMqW5bJDu8qlsOzskhFBhM52z0A6kuc4g3eVT\n2ZIpzWWDdJcvq+1AbAlENphZfgihW9xxVASVLZnSXDZId/lUtuRJa7mKpbl8KlsypblskO7yZbts\nmgMhIiIiIiIZUwIhIiIiIiIZS3oC8VjcAVQglS2Z0lw2SHf5VLbkSWu5iqW5fCpbMqW5bJDu8mW1\nbImeAyEiIiIiIpUr6T0QIiIiIiJSiZRAiIiIiIhIxhKZQJhZTzNbYWYFZjYi7njKw8xam9lrZvaO\nmS0zs2ui+43MbI6ZvRf9bBh3rGVlZjXNbKGZvRhdtzWzt6P6e9rMascdY1mZWQMzm2Jm75rZcjM7\nPi11Z2bXRe/JpWY2ycz2S3LdmdlfzGyjmS0tdW+XdWXuP6NyLjGzLvFFvne7Kdu90ftyiZlNM7MG\npZ6NjMq2wszOiifq8lE7kCxqB5JZd2lqB9LcBkDltwOJSyDMrCYwBugF5AGDzCwv3qjKpRC4IYSQ\nBxwHDIvKMwJ4JYTQDngluk6qa4Dlpa7vAR4IIRwGfA5cEUtU2fEQ8L8hhMOBzng5E193ZtYSGA50\nCyEcAdQEBpLsuhsP9Nzh3u7qqhfQLjqGAmMrKcayGs/OZZsDHBFC6ASsBEYCRJ8vA4GO0a95JPpc\nTQy1A4mkdiBhUtgOjCe9bQBUcjuQuAQC6A4UhBBWhxC+ByYDfWKOqcxCCOtDCAui86/wD56WeJkm\nRC+bAJwfT4TlY2atgHOAx6NrA3oAU6KXJLls9YGTgXEAIYTvQwhfkJK6A3KAumaWA+QC60lw3YUQ\n/gFs3uH27uqqD/BEcHOBBmbWvHIi3Xe7KlsI4aUQQmF0ORdoFZ33ASaHEL4LIbwPFOCfq0midiBB\n1A4kt3ykqB1IcxsAld8OJDGBaAl8VOp6bXQv8cysDXA08DZwUAhhffToE+CgmMIqrweB/wCKouvG\nwBel3tBJrr+2wCbgv6Ou+cfNrB4pqLsQwjrgPuBDvMH4EvgX6am7Yrurq7R9zlwOzIrO01C2NJRh\nl9QOJI7ageTWHVSfNgCy3A4kMYFIJTPbH3gWuDaEsKX0s+Br7SZuvV0zOxfYGEL4V9yxVJAcoAsw\nNoRwNLCVHbqpE1x3DfFvKNoCLYB67Nw1mipJrau9MbOb8SEyE+OORfZM7UAiqR1IiaTWUyYqoh1I\nYgKxDmhd6rpVdC+xzKwW3mhMDCFMjW5vKO4ui35ujCu+cjgB6G1ma/AhBj3wsaINou5QSHb9rQXW\nhhDejq6n4A1JGurudOD9EMKmEMI2YCpen2mpu2K7q6tUfM6Y2S+Ac4HBoWTTnzSULQ1l+AG1A4mt\nP7UDya07SHkbABXXDiQxgZgPtItWAaiNTwKZHnNMZRaNBR0HLA8h3F/q0XRgSHQ+BHi+smMrrxDC\nyBBCqxBCG7yeXg0hDAZeA/pHL0tk2QBCCJ8AH5lZh+jWacA7pKDu8C7r48wsN3qPFpctFXVXyu7q\najpwabQSx3HAl6W6uRPBzHriw0Z6hxC+KfVoOjDQzOqYWVt8kuC8OGIsB7UDCaF2AEhu+apDO5Da\nNgAquB0IISTuAM7GZ5OvAm6OO55yluVEvMtsCbAoOs7Gx4i+ArwHvAw0ijvWcpbzVODF6PzQ6I1a\nAPwPUCfu+MpRrqOA/Kj+ngMapqXugNuBd4GlwJNAnSTXHTAJH8e7Df/W8Ird1RVg+Co/q4D/w1ch\nib0M+1i2AnyMa/Hnyn+Vev3NUdlWAL3ijr+MZVY7kLBD7UD8sZahbKlpB9LcBuyhfBXWDlj0m4iI\niIiIiOxVEocwiYiIiIhITJRAiIiIiIhIxpRAiIiIiIhIxpRAiIiIiIhIxpRAiIiIiIhIxpRAiIiI\niIhIxpRAiIiIiIhIxv4fTtQgwQw7OucAAAAASUVORK5CYII=\n","text/plain":["
"]},"metadata":{"tags":[]}}]},{"cell_type":"markdown","metadata":{"id":"FL0H8CVxFeq6","colab_type":"text"},"source":["#### Create Our Own Poem "]},{"cell_type":"code","metadata":{"id":"2ITRbWJOe0s9","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":54},"outputId":"9ed32ebe-7484-4737-91d7-e1d7c7662036","executionInfo":{"status":"ok","timestamp":1573990702626,"user_tz":-540,"elapsed":1207,"user":{"displayName":"Swap vi","photoUrl":"https://lh3.google.com/a-/AAuE7mCC_wphRBrFVnISKMK6FoFPXiXssn3ZsnPhl1uH=s64","userId":"01936573407644251994"}}},"source":["seed_text = \"Help me Lord, You are my light\"\n","next_words = 50\n","\n","\n","for _ in range(next_words):\n","\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n","\ttoken_list = pad_sequences([token_list], maxlen=max_seq_len-1, padding='pre')\n","\tpredicted = model.predict_classes(token_list, verbose=0)\n","\toutput_word = \"\"\n","\tfor word, index in tokenizer.word_index.items():\n","\t\tif index == predicted:\n","\t\t\toutput_word = word\n","\t\t\tbreak\n","\tseed_text += \" \" + output_word\n","\n","\n","\n","print(\"finalized prediction: \", seed_text)\n"],"execution_count":44,"outputs":[{"output_type":"stream","text":["finalized prediction: Help me Lord, You are my light knew his speak had away alone doth dwell in pleasure thee aside aside long seen seen seen told days pain date light wind hearted decrease aside ' did lie anew to trust away long more hour their taste doth stand 'tis thee away new fangled sight and lie on your\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"Ogk_FTO0v-QU","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tutorials for TensorFlow 2 | 3 | * _Conv_Basic_with_TensorFlow:_ Start with building blocks of Deep Neural Networks and Slowly Build and Understand Convolutional Neural Network. 4 | Below is an example of how Conv and Max Pool layer understand features, training images are from MNIST Fashion Data-Set. 5 | ![Convunderstadning](https://github.com/suvoooo/Learn-TensorFlow/blob/master/Conv_Basic_with_TensorFlow/Understanding_conv2.png) 6 | 7 | * _im_generator:_ Learn to Preprocess Image Using Keras, 'flow_from_directory' and augemntation on the fly. 8 | 9 | * _Multi_Class:_ Moving from binary classification to multi-class classification. 10 | 11 | * _Transfer_Learning:_ Using VGG16 weights to train a model. 2800 images of cats and dogs are used for training and 600 images for validation. 12 | 13 | * _NLP_Embedding:_ Moving on from computer vision tasks, here we discuss classifying IMDB movie reviews and visualizng how embedding can help cluster words (positive & negative). 14 | * _NLP_LSTM_Glove:_ Using pretrained Glove to classify tweets of positve sentiments from the negative one. [Dataset - 1.6M Tweets](https://www.kaggle.com/kazanova/sentiment140) 15 | 16 | * _Time_SeriesW#:_ Deal with time series data, starting from simple linear regression to RNN, LSTM, 1D Convolution were used to build deep networks. Example of predicting minimum temperature of Melbourne is shown below-- 17 | 18 | * _TFDataIntro.ipynb:_* Introducing [TF Dataset API](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) and how to use them effectively to build efficient and fast data analysis pipeline. More details in [TDS](https://towardsdatascience.com/time-to-choose-tensorflow-data-over-imagedatagenerator-215e594f2435). 19 | 20 | * _cassava-classification.ipynb:_ Cassava leaf disease classification competetion in [Kaggle](https://www.kaggle.com/c/cassava-leaf-disease-classification/overview). The notebook here is the same one used for submission in the competetion. 21 | 22 | * _detectpneumonia...ipynb_ Pneumonia detection using TensorFlow dataset API. The dataset is available in [Kaggle](https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia). Many important comcepts such as standardization of images, class imbalance, roc curve as performance metric were influenced by DeepLearning.ai [course](https://www.deeplearning.ai/program/ai-for-medicine-specialization/). 23 | 24 | * _ViT_TensorFlow_ How to implement Vision Transformer using TensorFlow2.0 from scratch and detail understanding of how attention mechanism actually works. More details on [TDS](https://towardsdatascience.com/understand-and-implement-vision-transformer-with-tensorflow-2-0-f5435769093). 25 | ![ViT_Attention](https://github.com/suvoooo/Learn-TensorFlow/blob/master/ViT_TensorFlow/Self-Attention4.png). 26 | 27 | * _TF-Proba/Bayesian_uncertainty.ipynb_ Two types of uncertainties Aleatoric and Epistemic and how we can use Bayesian Neural Net for estimating uncertainties. Example using real weather data and TensorFlow Probability Library. More on [TDS](https://towardsdatascience.com/bayesian-deep-learning-estimating-uncertainty-9907f5208cc0). 28 | -------------------------------------------------------------------------------- /TF-Proba/Norm-Flows/norm_flow1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/TF-Proba/Norm-Flows/norm_flow1.png -------------------------------------------------------------------------------- /TF-Proba/Norm-Flows/norm_flow2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/TF-Proba/Norm-Flows/norm_flow2.png -------------------------------------------------------------------------------- /Time_SeriesW1/Q1W1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW1/Q1W1.png -------------------------------------------------------------------------------- /Time_SeriesW1/Q2W1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW1/Q2W1.png -------------------------------------------------------------------------------- /Time_SeriesW1/Q3W1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW1/Q3W1.png -------------------------------------------------------------------------------- /Time_SeriesW1/Q4W1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW1/Q4W1.png -------------------------------------------------------------------------------- /Time_SeriesW1/Q5W1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW1/Q5W1.png -------------------------------------------------------------------------------- /Time_SeriesW2/Features_Labels_TimeSeries.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Features_Labels_TimeSeries.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"code","metadata":{"id":"xwtLW_CkV73A","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":593},"outputId":"fe1ac551-7945-4d2f-a3a6-dfa22954d103","executionInfo":{"status":"ok","timestamp":1574498480001,"user_tz":-540,"elapsed":34034,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["!pip3 install tensorflow==2.0.0-rc1\n"],"execution_count":1,"outputs":[{"output_type":"stream","text":["Collecting tensorflow==2.0.0-rc1\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/40/d0/7359227ccfb6e2ae2a8fe345db1d513f010282022a7a6a899530cc63c41f/tensorflow-2.0.0rc1-cp36-cp36m-manylinux2010_x86_64.whl (86.3MB)\n","\u001b[K |████████████████████████████████| 86.3MB 122kB/s \n","\u001b[?25hRequirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (0.1.8)\n","Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.1.0)\n","Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.0.8)\n","Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (0.8.0)\n","Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (0.8.1)\n","Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.11.2)\n","Collecting tb-nightly<1.15.0a20190807,>=1.15.0a20190806\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/bc/88/24b5fb7280e74c7cf65bde47c171547fd02afb3840cff41bcbe9270650f5/tb_nightly-1.15.0a20190806-py3-none-any.whl (4.3MB)\n","\u001b[K |████████████████████████████████| 4.3MB 42.5MB/s \n","\u001b[?25hRequirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.17.4)\n","Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (0.33.6)\n","Collecting tf-estimator-nightly<1.14.0.dev2019080602,>=1.14.0.dev2019080601\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/21/28/f2a27a62943d5f041e4a6fd404b2d21cb7c59b2242a4e73b03d9ba166552/tf_estimator_nightly-1.14.0.dev2019080601-py2.py3-none-any.whl (501kB)\n","\u001b[K |████████████████████████████████| 501kB 51.0MB/s \n","\u001b[?25hRequirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.15.0)\n","Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (3.1.0)\n","Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.1.0)\n","Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (0.2.2)\n","Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (3.10.0)\n","Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.0.0-rc1) (1.12.0)\n","Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==2.0.0-rc1) (2.8.0)\n","Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.15.0a20190807,>=1.15.0a20190806->tensorflow==2.0.0-rc1) (0.16.0)\n","Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.15.0a20190807,>=1.15.0a20190806->tensorflow==2.0.0-rc1) (3.1.1)\n","Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.15.0a20190807,>=1.15.0a20190806->tensorflow==2.0.0-rc1) (41.6.0)\n","Installing collected packages: tb-nightly, tf-estimator-nightly, tensorflow\n"," Found existing installation: tensorflow 1.15.0\n"," Uninstalling tensorflow-1.15.0:\n"," Successfully uninstalled tensorflow-1.15.0\n","Successfully installed tb-nightly-1.15.0a20190806 tensorflow-2.0.0rc1 tf-estimator-nightly-1.14.0.dev2019080601\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"0pvmMWXaavcS","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"6ba9b491-2c7b-41e2-e80c-ee1a4fc4a217","executionInfo":{"status":"ok","timestamp":1574498482438,"user_tz":-540,"elapsed":2425,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["import tensorflow as tf\n","print (\"check tensorflow version: \", tf.__version__)"],"execution_count":2,"outputs":[{"output_type":"stream","text":["check tensorflow version: 2.0.0-rc1\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"TFhkNTYbbCbO","colab_type":"text"},"source":["This notebook is a basic tutorial for setting features and labels for time series analysis. Feature is effectively a number of values in the series,\n","with label being the next value. We'll call that number of values that will treat as our feature, the window size, where we're taking a window of the data and training an ML model to predict the next value. So for example, if we take our time series data, say, 30 days at a time, we'll use 30 values as the feature and the next value is the label. Then over time, we'll train a neural network to match the 30 features to the single label. "]},{"cell_type":"code","metadata":{"id":"bU7k6Me3dUwB","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":260},"outputId":"48e3becf-de80-41e8-fe31-3526ddc91ae1","executionInfo":{"status":"ok","timestamp":1574499124239,"user_tz":-540,"elapsed":901,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(14)\n","for val in dataset:\n"," print(val.numpy()) # because tensroflow 2 supports eager execution, so no need to start session"],"execution_count":3,"outputs":[{"output_type":"stream","text":["0\n","1\n","2\n","3\n","4\n","5\n","6\n","7\n","8\n","9\n","10\n","11\n","12\n","13\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"DUdFWVeHdf_j","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":139},"outputId":"154619bb-f37b-488e-f778-77ee78876261","executionInfo":{"status":"ok","timestamp":1574499220829,"user_tz":-540,"elapsed":613,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(14)\n","dataset = dataset.window(5, shift=2) # now select a window of 5, where our idea is that last value will be label, first 4 being the features.\n","for window_dataset in dataset:\n"," for val in window_dataset:\n"," print(val.numpy(), end=\" \")\n"," print()"],"execution_count":5,"outputs":[{"output_type":"stream","text":["0 1 2 3 4 \n","2 3 4 5 6 \n","4 5 6 7 8 \n","6 7 8 9 10 \n","8 9 10 11 12 \n","10 11 12 13 \n","12 13 \n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"j2lW6_Rdd06r","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":191},"outputId":"08093906-3e15-4517-e3c1-58b5888dac66","executionInfo":{"status":"ok","timestamp":1574499278219,"user_tz":-540,"elapsed":540,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(14)\n","dataset = dataset.window(5, shift=1, drop_remainder=True) # choose only the entries that contain 5 elements\n","for window_dataset in dataset:\n"," for val in window_dataset:\n"," print(val.numpy(), end=\" \")\n"," print()"],"execution_count":6,"outputs":[{"output_type":"stream","text":["0 1 2 3 4 \n","1 2 3 4 5 \n","2 3 4 5 6 \n","3 4 5 6 7 \n","4 5 6 7 8 \n","5 6 7 8 9 \n","6 7 8 9 10 \n","7 8 9 10 11 \n","8 9 10 11 12 \n","9 10 11 12 13 \n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"5h1s_77leFrR","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":191},"outputId":"b14a0a90-0dfb-4935-e8f8-4acd268335be","executionInfo":{"status":"ok","timestamp":1574500177597,"user_tz":-540,"elapsed":915,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(14)\n","dataset = dataset.window(5, shift=1, drop_remainder=True)\n","dataset = dataset.flat_map(lambda window: window.batch(5))\n","for window in dataset:\n"," print(window.numpy())"],"execution_count":7,"outputs":[{"output_type":"stream","text":["[0 1 2 3 4]\n","[1 2 3 4 5]\n","[2 3 4 5 6]\n","[3 4 5 6 7]\n","[4 5 6 7 8]\n","[5 6 7 8 9]\n","[ 6 7 8 9 10]\n","[ 7 8 9 10 11]\n","[ 8 9 10 11 12]\n","[ 9 10 11 12 13]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"H-FEX_6qhhJ9","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":121},"outputId":"06aa2c49-6cc9-410b-e8e9-f51916c95c3f","executionInfo":{"status":"ok","timestamp":1574500417046,"user_tz":-540,"elapsed":1029,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(10)\n","dataset = dataset.window(5, shift=1, drop_remainder=True)\n","dataset = dataset.flat_map(lambda window: window.batch(5))\n","dataset = dataset.map(lambda window: (window[:-1], window[-1:]))# last but all the elements are features \n","for x,y in dataset:\n"," print(x.numpy(), y.numpy())"],"execution_count":8,"outputs":[{"output_type":"stream","text":["[0 1 2 3] [4]\n","[1 2 3 4] [5]\n","[2 3 4 5] [6]\n","[3 4 5 6] [7]\n","[4 5 6 7] [8]\n","[5 6 7 8] [9]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"OVyjVEO-ibZL","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":191},"outputId":"e19300a4-cf40-4f74-e02b-0d0b023a1cab","executionInfo":{"status":"ok","timestamp":1574500711295,"user_tz":-540,"elapsed":967,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(14)\n","dataset = dataset.window(5, shift=1, drop_remainder=True)\n","dataset = dataset.flat_map(lambda window: window.batch(5))\n","dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n","dataset = dataset.shuffle(buffer_size=14) # shuffle the data-set, buffer size = dataset size\n","for x,y in dataset:\n"," print(x.numpy(), y.numpy())"],"execution_count":10,"outputs":[{"output_type":"stream","text":["[2 3 4 5] [6]\n","[ 8 9 10 11] [12]\n","[0 1 2 3] [4]\n","[6 7 8 9] [10]\n","[ 9 10 11 12] [13]\n","[1 2 3 4] [5]\n","[5 6 7 8] [9]\n","[ 7 8 9 10] [11]\n","[3 4 5 6] [7]\n","[4 5 6 7] [8]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"hxLBBALQi1GC","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":364},"outputId":"d046594f-870e-4013-9591-929e3800ffc4","executionInfo":{"status":"ok","timestamp":1574500796944,"user_tz":-540,"elapsed":960,"user":{"displayName":"Swap vi","photoUrl":"","userId":"01936573407644251994"}}},"source":["dataset = tf.data.Dataset.range(14)\n","dataset = dataset.window(5, shift=1, drop_remainder=True)\n","dataset = dataset.flat_map(lambda window: window.batch(5))\n","dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n","dataset = dataset.shuffle(buffer_size=14)\n","dataset = dataset.batch(3).prefetch(1) # a batch of 3 inputs \n","for x,y in dataset:\n"," print(\"x = \", x.numpy())\n"," print(\"y = \", y.numpy())"],"execution_count":12,"outputs":[{"output_type":"stream","text":["x = [[3 4 5 6]\n"," [6 7 8 9]\n"," [4 5 6 7]]\n","y = [[ 7]\n"," [10]\n"," [ 8]]\n","x = [[ 2 3 4 5]\n"," [ 9 10 11 12]\n"," [ 8 9 10 11]]\n","y = [[ 6]\n"," [13]\n"," [12]]\n","x = [[ 7 8 9 10]\n"," [ 1 2 3 4]\n"," [ 5 6 7 8]]\n","y = [[11]\n"," [ 5]\n"," [ 9]]\n","x = [[0 1 2 3]]\n","y = [[4]]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"kX6h9o8hjvO7","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /Time_SeriesW2/Q1W2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW2/Q1W2.png -------------------------------------------------------------------------------- /Time_SeriesW2/Q2W2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW2/Q2W2.png -------------------------------------------------------------------------------- /Time_SeriesW2/Q3W2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW2/Q3W2.png -------------------------------------------------------------------------------- /Time_SeriesW2/Q4W2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW2/Q4W2.png -------------------------------------------------------------------------------- /Time_SeriesW2/Q5W2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW2/Q5W2.png -------------------------------------------------------------------------------- /Time_SeriesW2/Q6W2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW2/Q6W2.png -------------------------------------------------------------------------------- /Time_SeriesW3/Quiz/Q1Week3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW3/Quiz/Q1Week3.png -------------------------------------------------------------------------------- /Time_SeriesW3/Quiz/Q2Week3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW3/Quiz/Q2Week3.png -------------------------------------------------------------------------------- /Time_SeriesW3/Quiz/Q3Week3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW3/Quiz/Q3Week3.png -------------------------------------------------------------------------------- /Time_SeriesW3/Quiz/Q4Week3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW3/Quiz/Q4Week3.png -------------------------------------------------------------------------------- /Time_SeriesW4/Min_Temp_Melbourne.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/Min_Temp_Melbourne.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q1.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q2.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q3.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q4.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q5.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q6.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q7.png -------------------------------------------------------------------------------- /Time_SeriesW4/quiz/W4Q8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Time_SeriesW4/quiz/W4Q8.png -------------------------------------------------------------------------------- /Transfer_Learning/Prediction_with_Vgg_CatsDogs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/Transfer_Learning/Prediction_with_Vgg_CatsDogs.png -------------------------------------------------------------------------------- /ViT_TensorFlow/Self-Attention1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/Self-Attention1.png -------------------------------------------------------------------------------- /ViT_TensorFlow/Self-Attention2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/Self-Attention2.png -------------------------------------------------------------------------------- /ViT_TensorFlow/Self-Attention3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/Self-Attention3.png -------------------------------------------------------------------------------- /ViT_TensorFlow/Self-Attention4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/Self-Attention4.png -------------------------------------------------------------------------------- /ViT_TensorFlow/Self_MultiAttention.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/Self_MultiAttention.jpg -------------------------------------------------------------------------------- /ViT_TensorFlow/transformer_VIT_org.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/transformer_VIT_org.jpg -------------------------------------------------------------------------------- /ViT_TensorFlow/transformer_ViT.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/transformer_ViT.png -------------------------------------------------------------------------------- /ViT_TensorFlow/transformer_arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ViT_TensorFlow/transformer_arch.png -------------------------------------------------------------------------------- /im_generator/detect_pneumonia.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/im_generator/detect_pneumonia.png -------------------------------------------------------------------------------- /im_generator/lecture_slides_Optimizers.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/im_generator/lecture_slides_Optimizers.pdf -------------------------------------------------------------------------------- /resnet/heatmap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/resnet/heatmap.png -------------------------------------------------------------------------------- /resnet/residual_block.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/resnet/residual_block.png -------------------------------------------------------------------------------- /resnet/train_acc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/resnet/train_acc.png -------------------------------------------------------------------------------- /ri_challenge/AEC_Class_train.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/AEC_Class_train.jpg -------------------------------------------------------------------------------- /ri_challenge/AEC_Class_train_acc.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/AEC_Class_train_acc.jpg -------------------------------------------------------------------------------- /ri_challenge/AEC_train.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/AEC_train.jpg -------------------------------------------------------------------------------- /ri_challenge/Simple_conv_train.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/Simple_conv_train.jpg -------------------------------------------------------------------------------- /ri_challenge/heatmap_aecconv.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/heatmap_aecconv.jpg -------------------------------------------------------------------------------- /ri_challenge/heatmap_aecdense.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/heatmap_aecdense.jpg -------------------------------------------------------------------------------- /ri_challenge/heatmap_aecsimpconv.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/heatmap_aecsimpconv.jpg -------------------------------------------------------------------------------- /ri_challenge/ridge-inet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suvoooo/Learn-TensorFlow/eeb04d3984f6f8096d6244e0b97f8b23e084c4bc/ri_challenge/ridge-inet.png --------------------------------------------------------------------------------