├── portadas-libros-1a-y-1a-parte.png ├── README.md ├── 4.RedesNeuronalesConvolucionales.ipynb ├── 8.3.programando-una-GAN.ipynb ├── 7.RedesNeuronalesRecurrentes.ipynb ├── 3.ComoSeEntrenaUnaRedNeuronal.ipynb └── 2.RedesNeuronalesDensamenteConectas.ipynb /portadas-libros-1a-y-1a-parte.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/HEAD/portadas-libros-1a-y-1a-parte.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # El contenido de este repositorio de código lleva tiempo mostrando síntomas de antigüedad ;-). Y por ello, después de 5 años, ha llegado el momento de descatalogar el contenido de los dos libros. Siento como autor no tener el tiempo para actualizarlo. Y sinceramente siento las molestias que esto pueda ocasionar. Porque quiero agradecer a todos y todas que lo habéis usado, es de agradecer dado el esfuerzo que me supuso en su momento. Estamos hablando del momento en que se popularizó el Deep Learning más allá de los laboratorios de investigación. Por suerte ahora hay infinidad de buenos libros sobre el tema. ¡Gracias! 2 | 3 | ## DeepLearning - Introduccion práctica con Keras 4 | Este repositorio contiene los notebooks de Jupyter que implementan los ejemplos de código que se encontraban en el libro **[Deep Learning - Introducción práctica con Keras]([https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/2.RedesNeuronalesDensamenteConectas.ipynb](https://www.amazon.es/stores/Jordi-Torres/author/B0089PKU76))** que se editó en dos partes 5 | 6 | 7 | [![solarized dualmode](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/portadas-libros-1a-y-1a-parte.png?raw=true)](#features) 8 | 9 | ### Código de la primera parte del libro 10 | 11 | * Capítulo 2: [Redes neuronales densamente conectadas](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/2.RedesNeuronalesDensamenteConectas.ipynb) 12 | * Capítulo 3: [Cómo se entrena una red neuronal](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/3.ComoSeEntrenaUnaRedNeuronal.ipynb) 13 | * Capítulo 4: [Redes neuronales convolucionales](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/4.RedesNeuronalesConvolucionales.ipynb) 14 | 15 | 16 | ### Código de la segunda parte del libro 17 | 18 | * Capítulo 5: [Datos y *Overfitting*](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras) 19 | * Capítulo 6: [*Data Augmentation* y *Transfer Learning*](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/6.DataAugmentation-TransferLearning.ipynb) 20 | * Capítulo 7: [Redes neuronales recurrentes](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/7.RedesNeuronalesRecurrentes.ipynb) 21 | * Capítulo 8: API funcional de Keras y GANs con TensorFlow 2.0 22 | * 8.1 [API funcional de Keras](https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/8.1.API-funcional-de-Keras.ipynb) 23 | * 8.3 [Programando una GAN]( 24 | https://github.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/blob/master/8.3.programando-una-GAN.ipynb) 25 | -------------------------------------------------------------------------------- /4.RedesNeuronalesConvolucionales.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "#### 4 Redes neuronales convolucionales" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "scrolled": false 15 | }, 16 | "outputs": [ 17 | { 18 | "name": "stderr", 19 | "output_type": "stream", 20 | "text": [ 21 | "/usr/local/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", 22 | " from ._conv import register_converters as _register_converters\n", 23 | "Using TensorFlow backend.\n" 24 | ] 25 | }, 26 | { 27 | "data": { 28 | "text/plain": [ 29 | "'2.1.3'" 30 | ] 31 | }, 32 | "execution_count": 1, 33 | "metadata": {}, 34 | "output_type": "execute_result" 35 | } 36 | ], 37 | "source": [ 38 | "import keras\n", 39 | "keras.__version__" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 2, 45 | "metadata": {}, 46 | "outputs": [ 47 | { 48 | "name": "stdout", 49 | "output_type": "stream", 50 | "text": [ 51 | "_________________________________________________________________\n", 52 | "Layer (type) Output Shape Param # \n", 53 | "=================================================================\n", 54 | "conv2d_1 (Conv2D) (None, 24, 24, 32) 832 \n", 55 | "_________________________________________________________________\n", 56 | "max_pooling2d_1 (MaxPooling2 (None, 12, 12, 32) 0 \n", 57 | "=================================================================\n", 58 | "Total params: 832\n", 59 | "Trainable params: 832\n", 60 | "Non-trainable params: 0\n", 61 | "_________________________________________________________________\n" 62 | ] 63 | } 64 | ], 65 | "source": [ 66 | "from keras import layers\n", 67 | "from keras import models\n", 68 | "\n", 69 | "model = models.Sequential()\n", 70 | "model.add(layers.Conv2D(32, (5, 5), activation='relu', input_shape=(28, 28, 1)))\n", 71 | "model.add(layers.MaxPooling2D((2, 2)))\n", 72 | "model.summary()" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": 3, 78 | "metadata": { 79 | "scrolled": true 80 | }, 81 | "outputs": [ 82 | { 83 | "name": "stdout", 84 | "output_type": "stream", 85 | "text": [ 86 | "_________________________________________________________________\n", 87 | "Layer (type) Output Shape Param # \n", 88 | "=================================================================\n", 89 | "conv2d_2 (Conv2D) (None, 24, 24, 32) 832 \n", 90 | "_________________________________________________________________\n", 91 | "max_pooling2d_2 (MaxPooling2 (None, 12, 12, 32) 0 \n", 92 | "_________________________________________________________________\n", 93 | "conv2d_3 (Conv2D) (None, 8, 8, 64) 51264 \n", 94 | "_________________________________________________________________\n", 95 | "max_pooling2d_3 (MaxPooling2 (None, 4, 4, 64) 0 \n", 96 | "=================================================================\n", 97 | "Total params: 52,096\n", 98 | "Trainable params: 52,096\n", 99 | "Non-trainable params: 0\n", 100 | "_________________________________________________________________\n" 101 | ] 102 | } 103 | ], 104 | "source": [ 105 | "\n", 106 | "model = models.Sequential()\n", 107 | "model.add(layers.Conv2D(32, (5, 5), activation='relu', input_shape=(28, 28, 1)))\n", 108 | "model.add(layers.MaxPooling2D((2, 2)))\n", 109 | "model.add(layers.Conv2D(64, (5, 5), activation='relu'))\n", 110 | "model.add(layers.MaxPooling2D((2, 2)))\n", 111 | "model.summary()" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": 4, 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "model.add(layers.Flatten())\n", 121 | "model.add(layers.Dense(10, activation='softmax'))" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 5, 127 | "metadata": { 128 | "scrolled": true 129 | }, 130 | "outputs": [ 131 | { 132 | "name": "stdout", 133 | "output_type": "stream", 134 | "text": [ 135 | "_________________________________________________________________\n", 136 | "Layer (type) Output Shape Param # \n", 137 | "=================================================================\n", 138 | "conv2d_2 (Conv2D) (None, 24, 24, 32) 832 \n", 139 | "_________________________________________________________________\n", 140 | "max_pooling2d_2 (MaxPooling2 (None, 12, 12, 32) 0 \n", 141 | "_________________________________________________________________\n", 142 | "conv2d_3 (Conv2D) (None, 8, 8, 64) 51264 \n", 143 | "_________________________________________________________________\n", 144 | "max_pooling2d_3 (MaxPooling2 (None, 4, 4, 64) 0 \n", 145 | "_________________________________________________________________\n", 146 | "flatten_1 (Flatten) (None, 1024) 0 \n", 147 | "_________________________________________________________________\n", 148 | "dense_1 (Dense) (None, 10) 10250 \n", 149 | "=================================================================\n", 150 | "Total params: 62,346\n", 151 | "Trainable params: 62,346\n", 152 | "Non-trainable params: 0\n", 153 | "_________________________________________________________________\n" 154 | ] 155 | } 156 | ], 157 | "source": [ 158 | "model.summary()" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 6, 164 | "metadata": {}, 165 | "outputs": [ 166 | { 167 | "name": "stdout", 168 | "output_type": "stream", 169 | "text": [ 170 | "(60000, 28, 28)\n" 171 | ] 172 | } 173 | ], 174 | "source": [ 175 | "from keras.datasets import mnist\n", 176 | "from keras.utils import to_categorical\n", 177 | "\n", 178 | "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n", 179 | "\n", 180 | "print (train_images.shape)\n", 181 | "train_images = train_images.reshape((60000, 28, 28, 1))\n", 182 | "train_images = train_images.astype('float32') / 255\n", 183 | "\n", 184 | "test_images = test_images.reshape((10000, 28, 28, 1))\n", 185 | "test_images = test_images.astype('float32') / 255\n", 186 | "\n", 187 | "train_labels = to_categorical(train_labels)\n", 188 | "test_labels = to_categorical(test_labels)" 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": 8, 194 | "metadata": {}, 195 | "outputs": [ 196 | { 197 | "name": "stdout", 198 | "output_type": "stream", 199 | "text": [ 200 | "Epoch 1/5\n", 201 | "60000/60000 [==============================] - 55s 924us/step - loss: 0.1107 - acc: 0.9676\n", 202 | "Epoch 2/5\n", 203 | "60000/60000 [==============================] - 60s 1ms/step - loss: 0.0995 - acc: 0.9706\n", 204 | "Epoch 3/5\n", 205 | "60000/60000 [==============================] - 58s 965us/step - loss: 0.0916 - acc: 0.9729\n", 206 | "Epoch 4/5\n", 207 | "60000/60000 [==============================] - 54s 904us/step - loss: 0.0848 - acc: 0.9747\n", 208 | "Epoch 5/5\n", 209 | "60000/60000 [==============================] - 55s 914us/step - loss: 0.0801 - acc: 0.9762\n" 210 | ] 211 | }, 212 | { 213 | "data": { 214 | "text/plain": [ 215 | "" 216 | ] 217 | }, 218 | "execution_count": 8, 219 | "metadata": {}, 220 | "output_type": "execute_result" 221 | } 222 | ], 223 | "source": [ 224 | "batch_size = 100\n", 225 | "epochs = 5\n", 226 | "\n", 227 | "model.compile(loss='categorical_crossentropy',\n", 228 | " optimizer='sgd',\n", 229 | " metrics=['accuracy'])\n", 230 | "\n", 231 | "model.fit(train_images, train_labels,\n", 232 | " batch_size=batch_size,\n", 233 | " epochs=epochs,\n", 234 | " verbose=1\n", 235 | " )" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "Evaluación modelo" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": 9, 248 | "metadata": { 249 | "scrolled": true 250 | }, 251 | "outputs": [ 252 | { 253 | "name": "stdout", 254 | "output_type": "stream", 255 | "text": [ 256 | "10000/10000 [==============================] - 3s 306us/step\n", 257 | "Test loss: 0.0668140901574865\n", 258 | "Test accuracy: 0.9797\n" 259 | ] 260 | } 261 | ], 262 | "source": [ 263 | "test_loss, test_acc = model.evaluate(test_images, test_labels)\n", 264 | "\n", 265 | "print('Test loss:', test_loss)\n", 266 | "print('Test accuracy:', test_acc)" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": {}, 272 | "source": [ 273 | "Redes con nombre propio" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": 10, 279 | "metadata": {}, 280 | "outputs": [ 281 | { 282 | "name": "stdout", 283 | "output_type": "stream", 284 | "text": [ 285 | "_________________________________________________________________\n", 286 | "Layer (type) Output Shape Param # \n", 287 | "=================================================================\n", 288 | "conv2d_4 (Conv2D) (None, 224, 224, 64) 1792 \n", 289 | "_________________________________________________________________\n", 290 | "conv2d_5 (Conv2D) (None, 224, 224, 64) 36928 \n", 291 | "_________________________________________________________________\n", 292 | "max_pooling2d_4 (MaxPooling2 (None, 112, 112, 64) 0 \n", 293 | "_________________________________________________________________\n", 294 | "conv2d_6 (Conv2D) (None, 112, 112, 128) 73856 \n", 295 | "_________________________________________________________________\n", 296 | "conv2d_7 (Conv2D) (None, 112, 112, 128) 147584 \n", 297 | "_________________________________________________________________\n", 298 | "max_pooling2d_5 (MaxPooling2 (None, 56, 56, 128) 0 \n", 299 | "_________________________________________________________________\n", 300 | "conv2d_8 (Conv2D) (None, 56, 56, 256) 295168 \n", 301 | "_________________________________________________________________\n", 302 | "conv2d_9 (Conv2D) (None, 56, 56, 256) 590080 \n", 303 | "_________________________________________________________________\n", 304 | "conv2d_10 (Conv2D) (None, 56, 56, 256) 590080 \n", 305 | "_________________________________________________________________\n", 306 | "max_pooling2d_6 (MaxPooling2 (None, 28, 28, 256) 0 \n", 307 | "_________________________________________________________________\n", 308 | "conv2d_11 (Conv2D) (None, 28, 28, 512) 1180160 \n", 309 | "_________________________________________________________________\n", 310 | "conv2d_12 (Conv2D) (None, 28, 28, 512) 2359808 \n", 311 | "_________________________________________________________________\n", 312 | "conv2d_13 (Conv2D) (None, 28, 28, 512) 2359808 \n", 313 | "_________________________________________________________________\n", 314 | "max_pooling2d_7 (MaxPooling2 (None, 14, 14, 512) 0 \n", 315 | "_________________________________________________________________\n", 316 | "conv2d_14 (Conv2D) (None, 14, 14, 512) 2359808 \n", 317 | "_________________________________________________________________\n", 318 | "conv2d_15 (Conv2D) (None, 14, 14, 512) 2359808 \n", 319 | "_________________________________________________________________\n", 320 | "conv2d_16 (Conv2D) (None, 14, 14, 512) 2359808 \n", 321 | "_________________________________________________________________\n", 322 | "max_pooling2d_8 (MaxPooling2 (None, 7, 7, 512) 0 \n", 323 | "_________________________________________________________________\n", 324 | "flatten_2 (Flatten) (None, 25088) 0 \n", 325 | "_________________________________________________________________\n", 326 | "dense_2 (Dense) (None, 4096) 102764544 \n", 327 | "_________________________________________________________________\n", 328 | "dense_3 (Dense) (None, 4096) 16781312 \n", 329 | "_________________________________________________________________\n", 330 | "dense_4 (Dense) (None, 1000) 4097000 \n", 331 | "=================================================================\n", 332 | "Total params: 138,357,544\n", 333 | "Trainable params: 138,357,544\n", 334 | "Non-trainable params: 0\n", 335 | "_________________________________________________________________\n" 336 | ] 337 | } 338 | ], 339 | "source": [ 340 | "from keras.models import Sequential\n", 341 | "from keras.layers import Dense, Activation, Dropout, Flatten\n", 342 | "from keras.layers import Conv2D\n", 343 | "from keras.layers import MaxPooling2D\n", 344 | "\n", 345 | "input_shape = (224, 224, 3)\n", 346 | "\n", 347 | "model = Sequential()\n", 348 | "model.add(Conv2D(64, (3, 3), input_shape=input_shape, padding='same',activation='relu'))\n", 349 | "model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\n", 350 | "model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n", 351 | "\n", 352 | "model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))\n", 353 | "model.add(Conv2D(128, (3, 3), activation='relu', padding='same',))\n", 354 | "model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n", 355 | "\n", 356 | "model.add(Conv2D(256, (3, 3), activation='relu', padding='same',))\n", 357 | "model.add(Conv2D(256, (3, 3), activation='relu', padding='same',))\n", 358 | "model.add(Conv2D(256, (3, 3), activation='relu', padding='same',))\n", 359 | "model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n", 360 | "\n", 361 | "model.add(Conv2D(512, (3, 3), activation='relu', padding='same',))\n", 362 | "model.add(Conv2D(512, (3, 3), activation='relu', padding='same',))\n", 363 | "model.add(Conv2D(512, (3, 3), activation='relu', padding='same',))\n", 364 | "model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n", 365 | "\n", 366 | "model.add(Conv2D(512, (3, 3), activation='relu', padding='same',))\n", 367 | "model.add(Conv2D(512, (3, 3), activation='relu', padding='same',))\n", 368 | "model.add(Conv2D(512, (3, 3), activation='relu', padding='same',))\n", 369 | "model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))\n", 370 | "\n", 371 | "model.add(Flatten())\n", 372 | "model.add(Dense(4096, activation='relu'))\n", 373 | "model.add(Dense(4096, activation='relu'))\n", 374 | "model.add(Dense(1000, activation='softmax'))\n", 375 | "\n", 376 | "\n", 377 | "model.summary()" 378 | ] 379 | }, 380 | { 381 | "cell_type": "code", 382 | "execution_count": 12, 383 | "metadata": {}, 384 | "outputs": [ 385 | { 386 | "name": "stdout", 387 | "output_type": "stream", 388 | "text": [ 389 | "_________________________________________________________________\n", 390 | "Layer (type) Output Shape Param # \n", 391 | "=================================================================\n", 392 | "input_1 (InputLayer) (None, 224, 224, 3) 0 \n", 393 | "_________________________________________________________________\n", 394 | "block1_conv1 (Conv2D) (None, 224, 224, 64) 1792 \n", 395 | "_________________________________________________________________\n", 396 | "block1_conv2 (Conv2D) (None, 224, 224, 64) 36928 \n", 397 | "_________________________________________________________________\n", 398 | "block1_pool (MaxPooling2D) (None, 112, 112, 64) 0 \n", 399 | "_________________________________________________________________\n", 400 | "block2_conv1 (Conv2D) (None, 112, 112, 128) 73856 \n", 401 | "_________________________________________________________________\n", 402 | "block2_conv2 (Conv2D) (None, 112, 112, 128) 147584 \n", 403 | "_________________________________________________________________\n", 404 | "block2_pool (MaxPooling2D) (None, 56, 56, 128) 0 \n", 405 | "_________________________________________________________________\n", 406 | "block3_conv1 (Conv2D) (None, 56, 56, 256) 295168 \n", 407 | "_________________________________________________________________\n", 408 | "block3_conv2 (Conv2D) (None, 56, 56, 256) 590080 \n", 409 | "_________________________________________________________________\n", 410 | "block3_conv3 (Conv2D) (None, 56, 56, 256) 590080 \n", 411 | "_________________________________________________________________\n", 412 | "block3_pool (MaxPooling2D) (None, 28, 28, 256) 0 \n", 413 | "_________________________________________________________________\n", 414 | "block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160 \n", 415 | "_________________________________________________________________\n", 416 | "block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808 \n", 417 | "_________________________________________________________________\n", 418 | "block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808 \n", 419 | "_________________________________________________________________\n", 420 | "block4_pool (MaxPooling2D) (None, 14, 14, 512) 0 \n", 421 | "_________________________________________________________________\n", 422 | "block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808 \n", 423 | "_________________________________________________________________\n", 424 | "block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808 \n", 425 | "_________________________________________________________________\n", 426 | "block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808 \n", 427 | "_________________________________________________________________\n", 428 | "block5_pool (MaxPooling2D) (None, 7, 7, 512) 0 \n", 429 | "_________________________________________________________________\n", 430 | "flatten (Flatten) (None, 25088) 0 \n", 431 | "_________________________________________________________________\n", 432 | "fc1 (Dense) (None, 4096) 102764544 \n", 433 | "_________________________________________________________________\n", 434 | "fc2 (Dense) (None, 4096) 16781312 \n", 435 | "_________________________________________________________________\n", 436 | "predictions (Dense) (None, 1000) 4097000 \n", 437 | "=================================================================\n", 438 | "Total params: 138,357,544\n", 439 | "Trainable params: 138,357,544\n", 440 | "Non-trainable params: 0\n", 441 | "_________________________________________________________________\n" 442 | ] 443 | } 444 | ], 445 | "source": [ 446 | "from keras.applications import VGG16\n", 447 | "\n", 448 | "conv_base = VGG16(weights='imagenet')\n", 449 | "conv_base.summary()" 450 | ] 451 | }, 452 | { 453 | "cell_type": "code", 454 | "execution_count": null, 455 | "metadata": {}, 456 | "outputs": [], 457 | "source": [] 458 | }, 459 | { 460 | "cell_type": "code", 461 | "execution_count": null, 462 | "metadata": {}, 463 | "outputs": [], 464 | "source": [] 465 | } 466 | ], 467 | "metadata": { 468 | "kernelspec": { 469 | "display_name": "Python 3", 470 | "language": "python", 471 | "name": "python3" 472 | }, 473 | "language_info": { 474 | "codemirror_mode": { 475 | "name": "ipython", 476 | "version": 3 477 | }, 478 | "file_extension": ".py", 479 | "mimetype": "text/x-python", 480 | "name": "python", 481 | "nbconvert_exporter": "python", 482 | "pygments_lexer": "ipython3", 483 | "version": "3.6.3" 484 | } 485 | }, 486 | "nbformat": 4, 487 | "nbformat_minor": 2 488 | } 489 | -------------------------------------------------------------------------------- /8.3.programando-una-GAN.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"accelerator":"GPU","colab":{"name":"8.3.programando-una-GAN.ipynb","version":"0.3.2","provenance":[{"file_id":"1lf1GEQUD-D-ilkh_kgdWEyfwJCVumuEt","timestamp":1566240158762},{"file_id":"10Pi7fY3yWf5cz4dCVW6SwZfp3e81cNwA","timestamp":1566239915206},{"file_id":"1-ARcRheRU7-bZSxw_uE2wC_UP2MfPBCy","timestamp":1566239315010},{"file_id":"https://github.com/tensorflow/docs/blob/master/site/en/r2/tutorials/generative/dcgan.ipynb","timestamp":1566022481822}],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"colab_type":"text","id":"rF2x3qooyBTI"},"source":["## 8.3 Deep Convolutional Generative Adversarial Network\n","### Deep Learning, Introducción práctica con Keras - Segunda parte"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"g5RstiiB8V-z","colab":{}},"source":["from __future__ import absolute_import, division, print_function, unicode_literals\n","\n","try:\n"," # %tensorflow_version only exists in Colab.\n"," %tensorflow_version 2.x\n","except Exception:\n"," pass\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"WZKbyU2-AiY-","outputId":"7c845b95-0e57-4ae7-9dc2-098fbe28e176","executionInfo":{"status":"ok","timestamp":1566247437675,"user_tz":-120,"elapsed":2019,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh4.googleusercontent.com/-SfQ_VFnu6kE/AAAAAAAAAAI/AAAAAAAABzE/lQO8NYx2THo/s64/photo.jpg","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["import tensorflow as tf\n","tf.__version__"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'2.0.0-beta1'"]},"metadata":{"tags":[]},"execution_count":21}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"YfIk2es3hJEd","colab":{}},"source":["import imageio\n","import matplotlib.pyplot as plt\n","import numpy as np\n","import os\n","from tensorflow.keras import layers\n","import time\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"a4fYMGxGhrna","colab":{}},"source":["\n","(train_images, _), (_, _) = tf.keras.datasets.mnist.load_data()\n","\n","train_images = train_images.reshape(train_images.shape[0], 28, 28, 1).astype('float32')\n","\n","train_images = (train_images - 127.5) / 127.5 \n","\n","BUFFER_SIZE = 60000\n","BATCH_SIZE = 256\n","train_dataset = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yZU3Ilr2gn5y","colab_type":"text"},"source":["## Modelos"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"-tEyxE-GMC48"},"source":["### Generador"]},{"cell_type":"code","metadata":{"id":"QsDYOFtzKIHi","colab_type":"code","colab":{}},"source":["def make_generator_model():\n"," model = tf.keras.Sequential()\n"," model.add(layers.Dense(7*7*256, use_bias=False, input_shape=(100,)))\n","\n"," model.add(layers.Reshape((7, 7, 256)))\n","\n"," model.add(layers.Conv2DTranspose(128, (5, 5), strides=(2, 2), padding='same'))\n"," model.add(layers.BatchNormalization())\n"," model.add(layers.LeakyReLU(alpha=0.01))\n","\n"," model.add(layers.Conv2DTranspose(64, (5, 5), strides=(1, 1), padding='same'))\n"," model.add(layers.BatchNormalization())\n"," model.add(layers.LeakyReLU(alpha=0.01))\n","\n"," model.add(layers.Conv2DTranspose(1, (5, 5), strides=(2, 2), padding='same', activation='tanh'))\n","\n"," return model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"-QlXbX2EwMCX","colab_type":"code","colab":{}},"source":["generator = make_generator_model()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"gl7jcC7TdPTG","outputId":"0414a6cc-2394-4aa0-bd48-479fb8c13a0a","executionInfo":{"status":"ok","timestamp":1566247439130,"user_tz":-120,"elapsed":3457,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh4.googleusercontent.com/-SfQ_VFnu6kE/AAAAAAAAAAI/AAAAAAAABzE/lQO8NYx2THo/s64/photo.jpg","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":286}},"source":["noise_dim = 100\n","noise = tf.random.normal([1, noise_dim])\n","generated_image = generator(noise, training=False)\n","\n","plt.imshow(generated_image[0, :, :, 0], cmap='gray')"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{"tags":[]},"execution_count":27},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGcVJREFUeJzt3Xlw1dXZB/Dvw74IsqchYEGlsgsY\nEfdqVRRwYWSsWiu1WpwqrU7tjB2kLdrBOu+orU5RwRcqRdGKKGpFEdKCFEEJyCKCQiGyBdmRRTZ5\n3j9yfSe1nO+hSbg3zvl+ZhjC/ebJPbnJw03u+Z1zzN0hIumpkesBiEhuqPlFEqXmF0mUml8kUWp+\nkUSp+UUSpeYXSZSaXyRRan6RRNXK5p3Vq1fPGzVqFMxr165N6w8ePBjMatasSWu//PJLmsfqjxw5\nEsxq1eIP4+HDhyt137F6NrbYx45d4VmnTp0K3zcAmFkwiz1u7Ot9LPfNvuY1avDnvco+buzzBio3\nNnbfu3fvxv79+/mdZ1Sq+c3scgCPAagJ4H/d/SH2/o0aNcLAgQOD+be+9S16f+vXrw9mJ5xwAq3d\ntWsXzU888USaHzhwIJg1b96c1m7bto3mjRs3pvnWrVtpvnfv3mAW+7xi/ykWFBTQPNagrIlatGhB\naz/99FOas68JAOzcuTOY1a1bl9bGvqb79++neb169WjOxtawYcMK3/eUKVNobXkV/rHfzGoCGAXg\nCgCdAdxgZp0r+vFEJLsq8zt/bwCr3H21ux8E8AKAq6tmWCJyvFWm+QsArCv37/WZ2/6NmQ0xs2Iz\nK479qCQi2XPcX+139zHuXujuhbHfg0QkeyrT/BsAtC337zaZ20TkG6AyzT8fQAcza29mdQBcD+C1\nqhmWiBxvFZ7qc/fDZjYUwDSUTfWNc/dlrKZmzZpo2rRpMP/888/pfZ5++unB7OOPP6a1PXv2pHls\nbnXjxo3BLDYXnp+fT/PYlFaPHj1oPmvWrGDWoUMHWsuuuwDin9uOHTtovnv37mC2cOFCWjtgwACa\nFxUV0fzkk08OZg0aNKC1MU2aNKH5smW0FXDjjTcGs/fff5/Wsu/V2BRmeZWa53f3qQCmVuZjiEhu\n6PJekUSp+UUSpeYXSZSaXyRRan6RRKn5RRKV1fX8hw4dQmlpaTA/9dRTaT2bz2bXAADA1Kl8RvKC\nCy6gOZvn79ixI61dt24dzbt3707zefPm0bxz5/BiygULFtDayy+/nOYTJ06kebdu3Wg+d+7cYNau\nXTtaO3nyZJrHlnG3bds2mLGvJwCsXLmS5rFrEGLr+RctWhTM1q5dS2vZUujYEu3y9Mwvkig1v0ii\n1PwiiVLziyRKzS+SKDW/SKIstgVxVWrdurXfdtttwZxNWQF8997YDrlnnnkmzffs2UNz9jh99tln\ntDa2E2xsCeeFF15I8w8++CCY9erVi9a++eabNI9NY7JlswCwevXqYBZbFhtbdhubQm3fvn0wiy2z\nrl+/Ps1nzJhB829/+9s0/9e//hXMWrVqRWvZ9/ro0aOxcePGY9q6W8/8IolS84skSs0vkig1v0ii\n1PwiiVLziyRKzS+SqKwu6XV3etx0bN6WLXWMzcu+9NJLNI+dELxv375gFptLf+CBB2j+0EP0cGOM\nHz+e5uz6iAkTJtBatsQaiH9N7rzzTpqzLdVjy09jR7a3adOG5iNHjgxmZ5xxBq2NHdHNtqAHgDlz\n5tD8nnvuCWax79U+ffoEs9hW6+XpmV8kUWp+kUSp+UUSpeYXSZSaXyRRan6RRKn5RRJVqfX8ZlYC\nYDeALwEcdvdC9v4FBQX+05/+NJivWLGC3l/fvn2DWWyL6tjW3LNnz6Y524p5yZIltDa2vXVsTvnm\nm2+m+YsvvhjMunTpQmsXL15M89gR32xdOgDcdNNNwezQoUO09qOPPqJ5bIvrwsLwt+Mbb7xBa087\n7TSax46Ej33Nn3jiiWD2s5/9jNaybcXHjh17zOv5q+Iin4vcfWsVfBwRySL92C+SqMo2vwN428wW\nmNmQqhiQiGRHZX/sP8/dN5hZKwDTzWyFu79T/h0y/ykMAYATTzyxkncnIlWlUs/87r4h8/dmAK8A\n6H2U9xnj7oXuXtiwYcPK3J2IVKEKN7+ZNTSzRl+9DeAyAB9W1cBE5PiqzI/9eQBeyUyB1QIw0d3f\nqpJRichxl9V9+1u1auWDBg0K5rH122vWrAlmF198Ma2dOXMmzWP7z9eoEf4hKba//NatfCY0tmY+\nLy+P5jt37gxme/fupbWxNfVs/wUA6NGjB83nz58fzGLHqq9atYrm9913H83Z8eKxI7TZ9xoA9O/f\nn+bsvAIA2L59ezCLHT3euHHjYPbnP/8ZpaWl2rdfRMLU/CKJUvOLJErNL5IoNb9IotT8IonK6tbd\ntWrVosdV79ixg9Z/73vfC2azZs2itT/4wQ9ozo65BvjS1Ngx1zfeeCPNY1NasW3Fb7nllmA2evRo\nWvvWW/zSjNjR5mw6DQAmTZoUzIYNG0ZrH3zwQZo//PDDNL/++uuDWWwZ9lVXXUXz5557juZXXnkl\nzdkR4SUlJbS2devWwWzy5Mm0tjw984skSs0vkig1v0ii1PwiiVLziyRKzS+SKDW/SKKyuqS3RYsW\nzpZCxub52THZ3bt3p7VHjhyh+cGDB2k+ffr0YBbbWvuZZ56heWw5cmxJMNsebdmyZbS2X79+NGfb\nRAPA2WefTfPhw4cHM3Z9AhD/msW2a7/mmmuC2d///nday5ZJA/Gxx64DYNe7xK6tYNekzJgxA9u3\nb9eSXhEJU/OLJErNL5IoNb9IotT8IolS84skSs0vkqisr+dv1apVMB8wYACtf/fdd4NZbE6YHdcM\nxI+yZtsps88JACZMmEDz4uJimjdq1IjmbG340qVLae2HH/JzVh599FGaxz63p59+Oph17dqV1sb2\nMZg6dSrN2THb3/nOd2htbGyxxy22hwP7mp9yyim0tn79+sFs7ty5tLY8PfOLJErNL5IoNb9IotT8\nIolS84skSs0vkig1v0iiouv5zWwcgAEANrt718xtzQD8FUA7ACUArnN3vhgfQH5+vv/oRz8K5lu2\nbKH1bI/52Hr+9957j+ZXXHEFzTdt2hTMioqKaO11111H83nz5tG8d+/eNO/Vq1cwe/zxx2lt7DGP\nHYP94osv0nzgwIHBrF69erSWHe8NAN26daM528P+rLPOorWLFy+mOTsTAADeeOMNml922WXBLLZ/\nAzuie+zYsVV6RPczAC7/2m2/AlDk7h0AFGX+LSLfINHmd/d3AGz/2s1XAxifeXs8gPCWKSJSLVX0\nd/48dy/NvL0JQF4VjUdEsqTSL/h52YsGwRcOzGyImRWbWTHbg09Esquizf+ZmeUDQObvzaF3dPcx\n7l7o7oUNGjSo4N2JSFWraPO/BmBw5u3BAF6tmuGISLZEm9/MngcwF8BpZrbezG4F8BCAS81sJYBL\nMv8WkW+QrO7b37JlS7/22muDeWzNfbNmzYLZs88+S2t79uxJ88OHD9P8/vvvD2axc+YHDx5Mc7bm\nHQBuvfVWms+YMSOYxc6ZX7FiBc23bdtG89heBm3atAlmU6ZMobWXX/71GeZ/9/rrr9O8b9++wYx9\nHwLApEmTaB7b979169Y0Z2dUxPYSGDlyZDBbsGABdu/erX37RSRMzS+SKDW/SKLU/CKJUvOLJErN\nL5KorB/Rzaae2LQQAKxbty6Y1a1bl9Zu2LCB5qeeeirNZ8+eHcwKCgpo7cyZM2kem3aqUYP/H92x\nY8dgNm3aNFpbs2ZNmseOwR4/fjzN77333mDGxg0AixYtojnbTh0A+vTpE8zYMdcAcMkll9CcLaMG\n+PcLALRr1y6YzZo1i9a2aNEimL3yyivYsmWLpvpEJEzNL5IoNb9IotT8IolS84skSs0vkig1v0ii\nsjrP36ZNG7/rrruCee3atWk9O7o4tt3xzp07af7JJ5/QnB25vGbNGlobO2L7hRdeoHn79u1pzpaP\nsrluAFiyZAnNDx06RPNatfgp702aNAlmse2xY583+9gA3449dpR17PqI2Lbhse3a2bUd/fv3p7Ul\nJSXBbPz48di0aZPm+UUkTM0vkig1v0ii1PwiiVLziyRKzS+SKDW/SKL4JG0V27dvHxYuXBjM8/L4\nkX9vvvkm/dhMbM44dprQbbfdFsxOOukkWhtbtx5bU8+2eQaAgwcPBrPnn3+e1nbq1InmbO04ADRt\n2pTmBw4cCGZ79uyhtU899RTNBwwYQHN2HcE11/CzZUeNGkXzSy+9lOZLly6lObumhR0HD/Dv9SNH\njtDa8vTML5IoNb9IotT8IolS84skSs0vkig1v0ii1PwiiYqu5zezcQAGANjs7l0zt40A8BMAWzLv\nNszdp8buLD8/39lx1bG90OvUqRPMYvObsfnsJ598kuZsL4HS0lJaG1vb/dJLL9G8c+fONJ8zZ04w\nu++++2gt2z8eiB9FHbsGgX1dYrXsSHYAWLt2Lc3Z5xbb/yF27UXsrIVnnnmG5r179w5msa/3Qw89\nFMzGjh2L0tLSKlvP/wyAox2U/gd375H5E218Ealeos3v7u8A2J6FsYhIFlXmd/6hZrbEzMaZGb/G\nU0SqnYo2/5MATgHQA0ApgEdC72hmQ8ys2MyKY9ffi0j2VKj53f0zd//S3Y8AeBpA8NULdx/j7oXu\nXhhbPCMi2VOh5jez/HL/HAjgw6oZjohkS3RJr5k9D+C7AFqY2XoAvwXwXTPrAcABlAC4/TiOUUSO\ng6zu29+yZUtn86OxNfcrVqwIZrt27aK1n3/+Oc0bN25M8y1btgQzdg0AACxfvpzmbdu2pTm7vgEA\n6tWrF8xiZ71feOGFNC8oKKA5OzMAAJYtWxbMiouLaa0Zn64+99xzaT558uRgNnToUFo7ZcoUmhcW\nFtI81lcbNmwIZrFrEJo3bx7MioqKsGPHDu3bLyJhan6RRKn5RRKl5hdJlJpfJFFqfpFEZXXr7rp1\n69Jpsc2bN9P6e+65J5g9+uijtPaqq66i+apVq2jOlsb+6U9/orWnnXYazdetW0fzRx4JXj0NAJg3\nb14wix0Pvnv3bprfcMMNNI9h27HHPu+BAwfSPPa43n333cEstlR5woQJNF+wYAHNv/jiC5r37Nkz\nmMWWWbMpzNi0cHl65hdJlJpfJFFqfpFEqflFEqXmF0mUml8kUWp+kURldUlvkyZN/IILLgjmt9xy\nC61/4IEHglnsyOTZs2fTPHYdwMSJE4NZbF42Pz+f5rGxxY7J3rlzZzA7++yzaW3siO1t27bR/IMP\nPqA5u34itv11bCn0GWecQfNXX301mP3xj3+ktQ8//DDN+/fvT/NJkybR/KKLLgpmsWsEzjnnnGA2\nYsQIrFmzRkt6RSRMzS+SKDW/SKLU/CKJUvOLJErNL5IoNb9IorK6nr9x48a47LLLgnlJSQmtf/zx\nx4PZ2LFjae1TTz1F89dff53m7BqE2BbUY8aMofm4ceNovnHjRpqztensKGgAePbZZ2l+55130jx2\nNPoTTzwRzKZNm0Zrf/vb39I8drT5ypUrg9n9999Pa2OPy/Dhw2k+bNgwmrM9GK688kpa+9prrwWz\nPXv20Nry9Mwvkig1v0ii1PwiiVLziyRKzS+SKDW/SKLU/CKJiq7nN7O2AP4CIA+AAxjj7o+ZWTMA\nfwXQDkAJgOvcfQf7WPn5+T548OBgfuDAAToWNjd68OBBWhvbv/4Xv/gFzdl1AieffDKtZfPNAHD+\n+efTPHadwB133BHMYvPVsesAOnToQHO2lwDA90GIHQ9+1lln0Ty2lwA7upwdkQ0Aq1evpnls7wk2\nFw/wscW+ly+55JJgNmrUKGzYsKHK1vMfBnCPu3cG0AfAnWbWGcCvABS5ewcARZl/i8g3RLT53b3U\n3Rdm3t4NYDmAAgBXAxifebfxAK45XoMUkar3X/3Ob2btAPQE8B6APHcvzUSbUPZrgYh8Qxxz85vZ\nCQAmA7jb3T8vn3nZCwdHffHAzIaYWbGZFe/bt69SgxWRqnNMzW9mtVHW+M+5+8uZmz8zs/xMng/g\nqKdsuvsYdy9098IGDRpUxZhFpApEm9/MDMBYAMvdvfxRuK8B+Oql+8EAwluliki1cyxTfecBmA1g\nKYCv1m8OQ9nv/S8COAnApyib6tvOPlbr1q19yJAhwXz//v10LKeffnowe/nll4MZALRv357mW7du\npfn3v//9YDZ9+nRa26dPH5q///77NI8t8Xz77beDWd26dWltrVp8VXfXrl1pHnvcY58706lTJ5rH\nHvdzzz03mM2dO5fW9urVi+ZseTkQn+pjU8uxZdSbNm2itZ988skxTfVF1/O7+z8BhD7Y947lTkSk\n+tEVfiKJUvOLJErNL5IoNb9IotT8IolS84skKqtHdOfl5TmbL2/VqhWtZ1tkFxYW0lq2vTUAdOzY\nkeb/+Mc/glnjxo1pbbdu3WgeW27MljIDwA9/+MNg9s4779DaQYMG0Ty2xXW/fv1ovnDhwmAWWy4c\nuwYh9v2yYsWKYBY73vvdd9+leexq1UWLFtG8S5cuwWz7dnq5DG666aZg9pvf/AarV6/WEd0iEqbm\nF0mUml8kUWp+kUSp+UUSpeYXSZSaXyRRWZ3nb9Omjf/85z8P5mvWrKH1BQUFwSy2bv3aa6+l+cyZ\nM2l+0kknBbPYnG6bNm1oXrZfSljDhg1pzvYD6N+/P62dPXs2zWNbey9evJjmeXnhrR1ja+pvvvlm\nmj/44IM0v+uuu4IZu24DAM477zyax7ZTf+SRR2j+4x//OJgNGDCA1tapUyeYjRw5Ep9++qnm+UUk\nTM0vkig1v0ii1PwiiVLziyRKzS+SKDW/SKKyOs/frFkzv/TSS4N5bK5+z549wWzbtm20NvaxO3fu\nTHN2DULbtm1pLduHAOBnAgDAW2+9RfNzzjknmMX2MWBz4QAwbdo0msc+tyNHjgSziy66iNayayuA\n+HUha9euDWb169entevWraP5mWeeSfPYtR+33357MJsxYwatZUfCjxo1CuvXr9c8v4iEqflFEqXm\nF0mUml8kUWp+kUSp+UUSpeYXSVT0iG4zawvgLwDyADiAMe7+mJmNAPATAFsy7zrM3aeyj1W7dm26\nJj+2T3vr1q2D2XPPPUdrR4wYQfPYXDrbGz+2x/svf/lLmk+cOJHmv/71r2n+t7/9LZjF5vFZLQAM\nGTKE5s2bN6d506ZNg9ny5ctpbcy+fftofu+99wazkSNH0lq2Nz4AbNmyheajR4+m+e9///tgNnDg\nQFo7f/78YMauq/i6aPMDOAzgHndfaGaNACwws+mZ7A/u/vAx35uIVBvR5nf3UgClmbd3m9lyAOGn\nbxH5Rvivfuc3s3YAegJ4L3PTUDNbYmbjzOyoP9+Z2RAzKzaz4i+++KJSgxWRqnPMzW9mJwCYDOBu\nd/8cwJMATgHQA2U/GRx10zJ3H+Puhe5eGLueWkSy55ia38xqo6zxn3P3lwHA3T9z9y/d/QiApwHw\nnR5FpFqJNr+VbS07FsByd3+03O355d5tIIAPq354InK8RJf0mtl5AGYDWArgq3mEYQBuQNmP/A6g\nBMDtmRcHg/Ly8vz6668P5rEtqtnUUOzI5QkTJtD84osvpjnbZrpv3760tkYN/n9s7PP+6KOPaL53\n795gdsopp9Da2FLoXbt20Tz2q1zLli2D2QknnEBrP/yQP5/EtkQvKSkJZjt27KC1PXr0oHlsu/UY\ntty4Zs2atJZtK/7YY48d85LeY3m1/58AjvbB6Jy+iFRvusJPJFFqfpFEqflFEqXmF0mUml8kUWp+\nkUQdy6q+KlOjRg00aNAgmLN5WQDo1KlTMFu2bBmtHT58OM2Liopozq5PiB3vzY4lB4Df/e53NL/j\njjtovmDBgmDWpUsXWjtnzhyax47JnjdvHs3Z/ceOB2fLqIH4FtdDhw4NZuPHj6e13bt3p/nHH39M\n8/PPP5/mkyZNCmaDBg2itWwZ9v79+2lteXrmF0mUml8kUWp+kUSp+UUSpeYXSZSaXyRRan6RRGX1\niG4z2wLg03I3tQCwNWsD+O9U17FV13EBGltFVeXYvu3u4U0Uyslq8//HnZsVu3thzgZAVNexVddx\nARpbReVqbPqxXyRRan6RROW6+cfk+P6Z6jq26jouQGOrqJyMLae/84tI7uT6mV9EciQnzW9ml5vZ\nx2a2ysx+lYsxhJhZiZktNbNFZlac47GMM7PNZvZhuduamdl0M1uZ+Tt8DG72xzbCzDZkHrtFZtYv\nR2Nra2b/MLOPzGyZmd2VuT2njx0ZV04et6z/2G9mNQF8AuBSAOsBzAdwg7vzzemzxMxKABS6e87n\nhM3sAgB7APzF3btmbvsfANvd/aHMf5xN3T18FnV2xzYCwJ5cn9ycOVAmv/zJ0gCuAfAj5PCxI+O6\nDjl43HLxzN8bwCp3X+3uBwG8AODqHIyj2nP3dwBs/9rNVwP4aieK8Sj75sm6wNiqBXcvdfeFmbd3\nA/jqZOmcPnZkXDmRi+YvALCu3L/Xo3od+e0A3jazBWY2JNeDOYq8cicjbQKQl8vBHEX05OZs+trJ\n0tXmsavIiddVTS/4/afz3L0XgCsA3Jn58bZa8rLf2arTdM0xndycLUc5Wfr/5fKxq+iJ11UtF82/\nAUDbcv9uk7mtWnD3DZm/NwN4BdXv9OHPvjokNfP35hyP5/9Vp5Obj3ayNKrBY1edTrzORfPPB9DB\nzNqbWR0A1wN4LQfj+A9m1jDzQgzMrCGAy1D9Th9+DcDgzNuDAbyaw7H8m+pycnPoZGnk+LGrdide\nu3vW/wDoh7JX/P8F4L5cjCEwrpMBLM78WZbrsQF4HmU/Bh5C2WsjtwJoDqAIwEoAMwA0q0Zjm4Cy\n05yXoKzR8nM0tvNQ9iP9EgCLMn/65fqxI+PKyeOmK/xEEqUX/EQSpeYXSZSaXyRRan6RRKn5RRKl\n5hdJlJpfJFFqfpFE/R8JdbGWJ2lgOQAAAABJRU5ErkJggg==\n","text/plain":["
"]},"metadata":{"tags":[]}}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"D0IKnaCtg6WE"},"source":["### Discriminador"]},{"cell_type":"code","metadata":{"id":"mypJk88So23v","colab_type":"code","colab":{}},"source":["def make_discriminator_model():\n"," model = tf.keras.Sequential()\n"," model.add(layers.Conv2D(32, (5, 5), strides=(2, 2), padding='same',\n"," input_shape=[28, 28, 1]))\n"," model.add(layers.LeakyReLU(alpha=0.01))\n"," \n"," model.add(layers.Conv2D(64, (5, 5), strides=(2, 2), padding='same'))\n"," model.add(layers.BatchNormalization())\n"," model.add(layers.LeakyReLU(alpha=0.01))\n"," \n"," model.add(layers.Conv2D(128, (5, 5), strides=(2, 2), padding='same'))\n"," model.add(layers.BatchNormalization())\n"," model.add(layers.LeakyReLU(alpha=0.01))\n","\n"," model.add(layers.Flatten())\n"," model.add(layers.Dense(1, activation='sigmoid'))\n","\n"," return model\n","\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"gDkA05NE6QMs","outputId":"76b91a81-4181-402d-e7a5-f18b5a3d1eca","executionInfo":{"status":"ok","timestamp":1566247439132,"user_tz":-120,"elapsed":3447,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh4.googleusercontent.com/-SfQ_VFnu6kE/AAAAAAAAAAI/AAAAAAAABzE/lQO8NYx2THo/s64/photo.jpg","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":510}},"source":["discriminator = make_discriminator_model()\n","decision = discriminator(generated_image)\n","print (decision)\n","\n","discriminator.summary ()"],"execution_count":0,"outputs":[{"output_type":"stream","text":["tf.Tensor([[0.4995159]], shape=(1, 1), dtype=float32)\n","Model: \"sequential_3\"\n","_________________________________________________________________\n","Layer (type) Output Shape Param # \n","=================================================================\n","conv2d_3 (Conv2D) (None, 14, 14, 32) 832 \n","_________________________________________________________________\n","leaky_re_lu_7 (LeakyReLU) (None, 14, 14, 32) 0 \n","_________________________________________________________________\n","conv2d_4 (Conv2D) (None, 7, 7, 64) 51264 \n","_________________________________________________________________\n","batch_normalization_6 (Batch (None, 7, 7, 64) 256 \n","_________________________________________________________________\n","leaky_re_lu_8 (LeakyReLU) (None, 7, 7, 64) 0 \n","_________________________________________________________________\n","conv2d_5 (Conv2D) (None, 4, 4, 128) 204928 \n","_________________________________________________________________\n","batch_normalization_7 (Batch (None, 4, 4, 128) 512 \n","_________________________________________________________________\n","leaky_re_lu_9 (LeakyReLU) (None, 4, 4, 128) 0 \n","_________________________________________________________________\n","flatten_1 (Flatten) (None, 2048) 0 \n","_________________________________________________________________\n","dense_3 (Dense) (None, 1) 2049 \n","=================================================================\n","Total params: 259,841\n","Trainable params: 259,457\n","Non-trainable params: 384\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"0FMYgY_mPfTi"},"source":["## Funciones de *Loss* y optimizadores\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"psQfmXxYKU3X","colab":{}},"source":["cross_entropy = tf.keras.losses.BinaryCrossentropy(from_logits=True)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"PKY_iPSPNWoj"},"source":["### Discriminator loss"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"wkMNfBWlT-PV","colab":{}},"source":["def discriminator_loss(real_output, fake_output):\n"," real_loss = cross_entropy(tf.ones_like(real_output), real_output)\n"," fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)\n"," total_loss = real_loss + fake_loss\n"," return total_loss"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"Jd-3GCUEiKtv"},"source":["### Generator loss"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"90BIcCKcDMxz","colab":{}},"source":["def generator_loss(fake_output):\n"," return cross_entropy(tf.ones_like(fake_output), fake_output)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"MgIc7i0th_Iu"},"source":["### Optimizadores"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"iWCn_PVdEJZ7","colab":{}},"source":["generator_optimizer = tf.keras.optimizers.Adam(1e-4)\n","\n","discriminator_optimizer = tf.keras.optimizers.Adam(1e-4)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"UuXySZBahFjt","colab_type":"text"},"source":["## Entrenamiento"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"Rw1fkAczTQYh"},"source":["###Training step\n","\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"3t5ibNo05jCB","colab":{}},"source":["@tf.function\n","def train_step(images):\n"," noise = tf.random.normal([BATCH_SIZE, noise_dim])\n","\n"," with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:\n"," generated_images = generator(noise, training=True)\n","\n"," real_output = discriminator(images, training=True)\n"," fake_output = discriminator(generated_images, training=True)\n","\n"," gen_loss = generator_loss(fake_output)\n"," disc_loss = discriminator_loss(real_output, fake_output)\n","\n"," gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)\n"," gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)\n","\n"," generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))\n"," discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Nts43kLUdPG-","colab_type":"text"},"source":["### Training loop"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"2M7LmLtGEMQJ","colab":{}},"source":["grid_size_x= 10\n","grid_size_y= 10\n","seed = tf.random.normal([grid_size_x*grid_size_y , noise_dim])\n","\n","def train(dataset, epochs):\n"," for epoch in range(epochs):\n"," start = time.time()\n","\n"," for image_batch in dataset:\n"," train_step(image_batch)\n","\n"," generate_images(generator,seed)\n"," print ('Time for epoch {} is {} sec'.format(epoch + 1, time.time()-start))\n","\n"," generate_images(generator, seed)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"2aFF7Hk3XdeW"},"source":["### Visualización de las imagenes (en realidad solo las que genera el Generador para ver que va aprendiendo)\n","\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"RmdVsmvhPxyy","colab":{}},"source":["\n","def generate_images(model, test_input):\n"," \n"," predictions = model(test_input, training=False)\n","\n"," fig = plt.figure(figsize=(grid_size_x,grid_size_y))\n"," for i in range(predictions.shape[0]):\n"," plt.subplot(grid_size_x, grid_size_y, i+1)\n"," plt.imshow(predictions[i, :, :, 0] * 127.5 + 127.5, cmap='gray')\n"," plt.axis('off') \n"," plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"dZrd4CdjR-Fp"},"source":["### Entrenar el model"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"Ly3UN0SLLY2l","outputId":"0ca58f5a-1c43-49ae-e967-63662a6c5132","executionInfo":{"status":"ok","timestamp":1566250960274,"user_tz":-120,"elapsed":3524569,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh4.googleusercontent.com/-SfQ_VFnu6kE/AAAAAAAAAAI/AAAAAAAABzE/lQO8NYx2THo/s64/photo.jpg","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":1000,"output_embedded_package_id":"1O8bS1NTiLOCyoE8M8orUou7msus69BR-"}},"source":["%%time\n","EPOCHS = 100\n","train(train_dataset, EPOCHS)"],"execution_count":0,"outputs":[{"output_type":"display_data","data":{"text/plain":"Output hidden; open in https://colab.research.google.com to view."},"metadata":{}}]},{"cell_type":"code","metadata":{"id":"C6LjGXEk1d4R","colab_type":"code","colab":{}},"source":["import os, signal\n","\n","os.kill( os.getpid() , \n"," signal.SIGKILL\n"," )"],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /7.RedesNeuronalesRecurrentes.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"accelerator":"GPU","colab":{"name":"7.RNN.ipynb","provenance":[{"file_id":"https://github.com/tensorflow/docs/blob/master/site/en/r2/tutorials/text/text_generation.ipynb","timestamp":1565900142296}],"collapsed_sections":[],"last_runtime":{"build_target":"//learning/brain/python/client:tpu_hw_notebook","kind":"private"}},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"eK8eUOQU7q2V","colab_type":"text"},"source":["## 7. Redes Neuronales Recurrentes\n","### Deep Learning, Introducción práctica con Keras - Segunda parte"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"WGyKZj3bzf9p"},"source":["### Importar TensorFlow 2.0 y otras librerias"]},{"cell_type":"code","metadata":{"id":"zWrJnG1jZd00","colab_type":"code","outputId":"909498c7-b016-4c76-dcb4-dd1db0211cc4","executionInfo":{"status":"ok","timestamp":1568818555492,"user_tz":-120,"elapsed":40472,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":507}},"source":["!pip install tensorflow-gpu==2.0.0-alpha0\n","\n","# from __future__ import absolute_import, division, print_function, unicode_literals\n","\n","import tensorflow as tf\n","\n","print(tf.__version__)\n","\n","import numpy as np\n","import os\n","import time"],"execution_count":1,"outputs":[{"output_type":"stream","text":["Collecting tensorflow-gpu==2.0.0-alpha0\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/1a/66/32cffad095253219d53f6b6c2a436637bbe45ac4e7be0244557210dc3918/tensorflow_gpu-2.0.0a0-cp36-cp36m-manylinux1_x86_64.whl (332.1MB)\n","\u001b[K |████████████████████████████████| 332.1MB 83kB/s \n","\u001b[?25hCollecting tf-estimator-nightly<1.14.0.dev2019030116,>=1.14.0.dev2019030115 (from tensorflow-gpu==2.0.0-alpha0)\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/13/82/f16063b4eed210dc2ab057930ac1da4fbe1e91b7b051a6c8370b401e6ae7/tf_estimator_nightly-1.14.0.dev2019030115-py2.py3-none-any.whl (411kB)\n","\u001b[K |████████████████████████████████| 419kB 51.7MB/s \n","\u001b[?25hRequirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (1.1.0)\n","Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (1.15.0)\n","Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (1.1.0)\n","Requirement already satisfied: google-pasta>=0.1.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (0.1.7)\n","Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (0.8.0)\n","Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (1.12.0)\n","Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (3.7.1)\n","Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (0.33.6)\n","Collecting tb-nightly<1.14.0a20190302,>=1.14.0a20190301 (from tensorflow-gpu==2.0.0-alpha0)\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/a9/51/aa1d756644bf4624c03844115e4ac4058eff77acd786b26315f051a4b195/tb_nightly-1.14.0a20190301-py3-none-any.whl (3.0MB)\n","\u001b[K |████████████████████████████████| 3.0MB 46.9MB/s \n","\u001b[?25hRequirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (0.8.0)\n","Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (1.0.8)\n","Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (1.16.5)\n","Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.0.0-alpha0) (0.2.2)\n","Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tensorflow-gpu==2.0.0-alpha0) (41.2.0)\n","Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow-gpu==2.0.0-alpha0) (0.15.6)\n","Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow-gpu==2.0.0-alpha0) (3.1.1)\n","Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.6->tensorflow-gpu==2.0.0-alpha0) (2.8.0)\n","Installing collected packages: tf-estimator-nightly, tb-nightly, tensorflow-gpu\n","Successfully installed tb-nightly-1.14.0a20190301 tensorflow-gpu-2.0.0a0 tf-estimator-nightly-1.14.0.dev2019030115\n","2.0.0-alpha0\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"EHDoRoc5PKWz"},"source":["### Descarga del conjunto de datos"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"pD_55cOxLkAb","outputId":"240a44d7-e047-4438-8b4d-668b7ba5cb04","executionInfo":{"status":"ok","timestamp":1568818555494,"user_tz":-120,"elapsed":40466,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":121}},"source":["path_to_fileDL = tf.keras.utils.get_file('DL-Introduccion-practica-con-Keras-1aParte.txt', 'https://raw.githubusercontent.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/master/DeepLearning-Introduccion-practica-con-Keras-PRIMERA-PARTE.txt') \n","\n","#path_to_fileDL = tf.keras.utils.get_file('Shakespear.txt', 'https://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt')\n","\n","text = open(path_to_fileDL, 'rb').read().decode(encoding='utf-8')\n","print('Longitud del texto: {} carácteres'.format(len(text)))\n","\n","vocab = sorted(set(text))\n","\n","print ('El texto está compuesto de estos {} carácteres:'.format(len(vocab)))\n","print (vocab)"],"execution_count":2,"outputs":[{"output_type":"stream","text":["Downloading data from https://raw.githubusercontent.com/jorditorresBCN/Deep-Learning-Introduccion-practica-con-Keras/master/DeepLearning-Introduccion-practica-con-Keras-PRIMERA-PARTE.txt\n","204800/203286 [==============================] - 0s 0us/step\n","Longitud del texto: 203251 carácteres\n","El texto está compuesto de estos 92 carácteres:\n","['\\n', '\\r', ' ', '!', '\"', '#', '%', \"'\", '(', ')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?', '@', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', '[', ']', '_', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '\\xad', 'ÿ', 'Š', '‡', '…']\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"IalZLbvOzf-F","outputId":"a2fe0ad2-6286-432e-b1fe-1dad86e5e94b","executionInfo":{"status":"ok","timestamp":1568818555495,"user_tz":-120,"elapsed":40462,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":1000}},"source":["char2idx = {u:i for i, u in enumerate(vocab)}\n","idx2char = np.array(vocab)\n","\n","for char,_ in zip(char2idx, range(len(vocab))):\n"," print(' {:4s}: {:3d},'.format(repr(char), char2idx[char]))"],"execution_count":3,"outputs":[{"output_type":"stream","text":[" '\\n': 0,\n"," '\\r': 1,\n"," ' ' : 2,\n"," '!' : 3,\n"," '\"' : 4,\n"," '#' : 5,\n"," '%' : 6,\n"," \"'\" : 7,\n"," '(' : 8,\n"," ')' : 9,\n"," '*' : 10,\n"," '+' : 11,\n"," ',' : 12,\n"," '-' : 13,\n"," '.' : 14,\n"," '/' : 15,\n"," '0' : 16,\n"," '1' : 17,\n"," '2' : 18,\n"," '3' : 19,\n"," '4' : 20,\n"," '5' : 21,\n"," '6' : 22,\n"," '7' : 23,\n"," '8' : 24,\n"," '9' : 25,\n"," ':' : 26,\n"," ';' : 27,\n"," '<' : 28,\n"," '=' : 29,\n"," '>' : 30,\n"," '?' : 31,\n"," '@' : 32,\n"," 'A' : 33,\n"," 'B' : 34,\n"," 'C' : 35,\n"," 'D' : 36,\n"," 'E' : 37,\n"," 'F' : 38,\n"," 'G' : 39,\n"," 'H' : 40,\n"," 'I' : 41,\n"," 'J' : 42,\n"," 'K' : 43,\n"," 'L' : 44,\n"," 'M' : 45,\n"," 'N' : 46,\n"," 'O' : 47,\n"," 'P' : 48,\n"," 'Q' : 49,\n"," 'R' : 50,\n"," 'S' : 51,\n"," 'T' : 52,\n"," 'U' : 53,\n"," 'V' : 54,\n"," 'W' : 55,\n"," 'X' : 56,\n"," 'Y' : 57,\n"," '[' : 58,\n"," ']' : 59,\n"," '_' : 60,\n"," 'a' : 61,\n"," 'b' : 62,\n"," 'c' : 63,\n"," 'd' : 64,\n"," 'e' : 65,\n"," 'f' : 66,\n"," 'g' : 67,\n"," 'h' : 68,\n"," 'i' : 69,\n"," 'j' : 70,\n"," 'k' : 71,\n"," 'l' : 72,\n"," 'm' : 73,\n"," 'n' : 74,\n"," 'o' : 75,\n"," 'p' : 76,\n"," 'q' : 77,\n"," 'r' : 78,\n"," 's' : 79,\n"," 't' : 80,\n"," 'u' : 81,\n"," 'v' : 82,\n"," 'w' : 83,\n"," 'x' : 84,\n"," 'y' : 85,\n"," 'z' : 86,\n"," '\\xad': 87,\n"," 'ÿ' : 88,\n"," 'Š' : 89,\n"," '‡' : 90,\n"," '…' : 91,\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"ML-DuUyi_aHy","colab_type":"code","colab":{}},"source":["text_as_int = np.array([char2idx[c] for c in text])"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"l1VKcQHcymwb","outputId":"ae9f8163-e3ba-4e8d-d5e8-9963ee511479","executionInfo":{"status":"ok","timestamp":1568818555902,"user_tz":-120,"elapsed":40858,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":84}},"source":["print ('texto: {}'.format(repr(text[:50])))\n","print ('{}'.format(repr(text_as_int[:50])))"],"execution_count":5,"outputs":[{"output_type":"stream","text":["texto: 'Prologo\\r\\nEn 1953, Isaac Asimov publico Segunda Fun'\n","array([48, 78, 75, 72, 75, 67, 75, 1, 0, 37, 74, 2, 17, 25, 21, 19, 12,\n"," 2, 41, 79, 61, 61, 63, 2, 33, 79, 69, 73, 75, 82, 2, 76, 81, 62,\n"," 72, 69, 63, 75, 2, 51, 65, 67, 81, 74, 64, 61, 2, 38, 81, 74])\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"hgsVvVxnymwf"},"source":["### Preparar los datos para entrenar la RNN\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"0UHJDA39zf-O","colab":{}},"source":["char_dataset = tf.data.Dataset.from_tensor_slices(text_as_int)\n","\n","seq_length = 100\n"," \n","sequences = char_dataset.batch(seq_length+1, drop_remainder=True)\n","\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"l4hkDU3i7ozi","outputId":"359c3657-54a9-4bee-c284-d2ce5e74b3a3","executionInfo":{"status":"ok","timestamp":1568818555904,"user_tz":-120,"elapsed":40849,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":185}},"source":["for item in sequences.take(10):\n"," print(repr(''.join(idx2char[item.numpy()])))"],"execution_count":7,"outputs":[{"output_type":"stream","text":["'Prologo\\r\\nEn 1953, Isaac Asimov publico Segunda Fundacion, el tercer libro de la saga de la Fundacion '\n","'(o el decimotercero segun otras fuentes, este es un tema de debate). En Segunda Fundacion aparece por'\n","' primera vez Arkady Darell, uno de los principales personajes de la parte final de la saga. En su pri'\n","'mera escena, Arkady, que tiene 14 anos, esta haciendo sus tareas escolares. En concreto, una redaccio'\n","'n que lleva por titulo ?El Futuro del Plan Sheldon?. Para hacer la redaccion, Arkady esta utilizando '\n","'un ?transcriptor?,un dispositivo que convierte su voz en palabras escritas. Este tipo de dispositivo,'\n","' que para Isaac Asimov era ciencia ficcion en 1953, lo tenemos al alcance de la mano en la mayoria de'\n","' nuestros smartphones, y el Deep Learning es uno de los responsables de que ya tengamos este tipo de '\n","'aplicaciones, siendo la tecnologia otro de ellos.En la actualidad disponemos de GPUs (Graphics Proces'\n","'sor Units), que solo cuestan alrededor de 100 euros, que estarian en la lista del Top500 hace unos po'\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"9NGu-FkO_kYU","colab":{}},"source":["def split_input_target(chunk):\n"," input_text = chunk[:-1]\n"," target_text = chunk[1:]\n"," return input_text, target_text\n","\n","dataset = sequences.map(split_input_target)\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"GNbw-iR0ymwj","outputId":"62cb263e-b815-4e6f-e5ee-4de68ed4e1b2","executionInfo":{"status":"ok","timestamp":1568818555905,"user_tz":-120,"elapsed":40839,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":70}},"source":["for input_example, target_example in dataset.take(1):\n"," print ('Input data: ', repr(''.join(idx2char[input_example.numpy()])))\n"," print ('Target data:', repr(''.join(idx2char[target_example.numpy()])))"],"execution_count":9,"outputs":[{"output_type":"stream","text":["Input data: 'Prologo\\r\\nEn 1953, Isaac Asimov publico Segunda Fundacion, el tercer libro de la saga de la Fundacion'\n","Target data: 'rologo\\r\\nEn 1953, Isaac Asimov publico Segunda Fundacion, el tercer libro de la saga de la Fundacion '\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"0eBu9WZG84i0","outputId":"e1ba7b19-33a8-4df0-c7a9-7914bce17b02","executionInfo":{"status":"ok","timestamp":1568818555906,"user_tz":-120,"elapsed":40833,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["print (dataset)"],"execution_count":10,"outputs":[{"output_type":"stream","text":["\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"p2pGotuNzf-S","outputId":"0a08cc7c-6684-4a88-a592-f3a527284642","executionInfo":{"status":"ok","timestamp":1568818555907,"user_tz":-120,"elapsed":40826,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["BATCH_SIZE = 64\n","\n","BUFFER_SIZE = 10000\n","\n","dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\n","\n","print (dataset)"],"execution_count":11,"outputs":[{"output_type":"stream","text":["\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"r6oUuElIMgVx"},"source":["### Construcción del modelo"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"zHT8cLh7EAsg","colab":{}},"source":["vocab_size = len(vocab)\n","embedding_dim = 256\n","rnn_units = 1024"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"MtCrdfzEI2N0","colab":{}},"source":["def build_model(vocab_size, embedding_dim, rnn_units, batch_size):\n"," model = tf.keras.Sequential([\n"," tf.keras.layers.Embedding(vocab_size, embedding_dim,\n"," batch_input_shape=[batch_size, None]),\n"," tf.keras.layers.LSTM(rnn_units,\n"," return_sequences=True,\n"," stateful=True,\n"," recurrent_initializer='glorot_uniform'),\n"," tf.keras.layers.Dense(vocab_size)\n"," ])\n"," return model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"wwsrpOik5zhv","outputId":"98e212ec-f9c6-4ba0-92d6-861fd9178d19","executionInfo":{"status":"ok","timestamp":1568818555909,"user_tz":-120,"elapsed":40814,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":54}},"source":["model = build_model(\n"," vocab_size = len(vocab),\n"," embedding_dim=embedding_dim,\n"," rnn_units=rnn_units,\n"," batch_size=BATCH_SIZE)"],"execution_count":14,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:: Note that this layer is not optimized for performance. Please use tf.keras.layers.CuDNNLSTM for better performance on GPU.\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"PpHGXDMcZ0Zt","colab_type":"code","outputId":"8d448fdd-ffad-417d-e492-ebe27d8ac17e","executionInfo":{"status":"ok","timestamp":1568818557122,"user_tz":-120,"elapsed":42021,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":252}},"source":["model.summary()"],"execution_count":15,"outputs":[{"output_type":"stream","text":["Model: \"sequential\"\n","_________________________________________________________________\n","Layer (type) Output Shape Param # \n","=================================================================\n","embedding (Embedding) (64, None, 256) 23552 \n","_________________________________________________________________\n","unified_lstm (UnifiedLSTM) (64, None, 1024) 5246976 \n","_________________________________________________________________\n","dense (Dense) (64, None, 92) 94300 \n","=================================================================\n","Total params: 5,364,828\n","Trainable params: 5,364,828\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"C-_70kKAPrPU","outputId":"dfab0fc7-3326-4491-8318-07d1fbdb473b","executionInfo":{"status":"ok","timestamp":1568818557123,"user_tz":-120,"elapsed":42017,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":50}},"source":["for input_example_batch, target_example_batch in dataset.take(1):\n"," print(\"Input:\", input_example_batch.shape, \"# (batch_size, sequence_length)\")\n"," print(\"Target:\", target_example_batch.shape, \"# (batch_size, sequence_length)\")\n"],"execution_count":16,"outputs":[{"output_type":"stream","text":["Input: (64, 100) # (batch_size, sequence_length)\n","Target: (64, 100) # (batch_size, sequence_length)\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"udE_6eTldMhB","colab_type":"code","outputId":"c59f76ef-b449-4143-e1d0-87c442727fd2","executionInfo":{"status":"ok","timestamp":1568818562660,"user_tz":-120,"elapsed":47547,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["for input_example_batch, target_example_batch in dataset.take(1): \n"," example_batch_predictions = model(input_example_batch)\n"," print(\"Prediction: \", example_batch_predictions.shape, \"# (batch_size, sequence_length, vocab_size)\")"],"execution_count":17,"outputs":[{"output_type":"stream","text":["Prediction: (64, 100, 92) # (batch_size, sequence_length, vocab_size)\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"4V4MfFg0RQJg","colab":{}},"source":["sampled_indices = tf.random.categorical(example_batch_predictions[0], num_samples=1)\n","sampled_indices_characters = tf.squeeze(sampled_indices,axis=-1).numpy()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"YqFMUQc_UFgM","outputId":"9ed0d2be-20c8-4089-de47-26c3f97f5915","executionInfo":{"status":"ok","timestamp":1568818562662,"user_tz":-120,"elapsed":47539,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":118}},"source":["sampled_indices_characters"],"execution_count":19,"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([ 1, 89, 85, 20, 6, 73, 54, 40, 7, 57, 3, 67, 81, 47, 19, 14, 81,\n"," 35, 78, 37, 90, 34, 50, 46, 35, 7, 73, 4, 59, 75, 6, 48, 28, 82,\n"," 14, 78, 2, 68, 72, 23, 87, 66, 70, 19, 78, 38, 48, 65, 6, 36, 87,\n"," 66, 69, 76, 57, 2, 89, 54, 26, 57, 37, 12, 1, 47, 0, 51, 52, 18,\n"," 48, 87, 9, 48, 55, 67, 79, 22, 54, 53, 49, 2, 17, 76, 40, 32, 60,\n"," 59, 22, 32, 82, 47, 35, 59, 66, 86, 8, 39, 60, 61, 68, 40])"]},"metadata":{"tags":[]},"execution_count":19}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"trpqTWyvk0nr"},"source":["Entrenar el modelo"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"4HrXTACTdzY-","colab":{}},"source":["def loss(labels, logits):\n"," return tf.keras.losses.sparse_categorical_crossentropy(labels, logits, from_logits=True)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"DDl1_Een6rL0","colab":{}},"source":["model.compile(optimizer='adam', loss=loss)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"ieSJdchZggUj"},"source":["Configurar el *checkpoints*"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"W6fWTriUZP-n","colab":{}},"source":[" # directorio\n","checkpoint_dir = './training_checkpoints'\n","# nombre fichero\n","checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt_{epoch}\")\n","\n","checkpoint_callback=tf.keras.callbacks.ModelCheckpoint(\n"," filepath=checkpoint_prefix,\n"," save_weights_only=True)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"3Ky3F_BhgkTW"},"source":["*Training*"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"UK-hmKjYVoll","outputId":"b6b61770-7df4-4fb4-9ef4-aa8162808ae7","executionInfo":{"status":"ok","timestamp":1568818694957,"user_tz":-120,"elapsed":179814,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":1000}},"source":["EPOCHS=50\n","history = model.fit(dataset, epochs=EPOCHS, callbacks=[checkpoint_callback])"],"execution_count":23,"outputs":[{"output_type":"stream","text":["Epoch 1/50\n","31/31 [==============================] - 3s 107ms/step - loss: 3.2626\n","Epoch 2/50\n","31/31 [==============================] - 3s 82ms/step - loss: 2.7938\n","Epoch 3/50\n","31/31 [==============================] - 3s 81ms/step - loss: 2.4345\n","Epoch 4/50\n","31/31 [==============================] - 3s 83ms/step - loss: 2.2087\n","Epoch 5/50\n","31/31 [==============================] - 3s 83ms/step - loss: 2.0815\n","Epoch 6/50\n","31/31 [==============================] - 3s 83ms/step - loss: 1.9603\n","Epoch 7/50\n","31/31 [==============================] - 3s 84ms/step - loss: 1.8362\n","Epoch 8/50\n","31/31 [==============================] - 3s 86ms/step - loss: 1.7192\n","Epoch 9/50\n","31/31 [==============================] - 3s 88ms/step - loss: 1.6094\n","Epoch 10/50\n","31/31 [==============================] - 3s 86ms/step - loss: 1.5072\n","Epoch 11/50\n","31/31 [==============================] - 3s 87ms/step - loss: 1.4165\n","Epoch 12/50\n","31/31 [==============================] - 3s 88ms/step - loss: 1.3356\n","Epoch 13/50\n","31/31 [==============================] - 3s 88ms/step - loss: 1.2693\n","Epoch 14/50\n","31/31 [==============================] - 3s 89ms/step - loss: 1.2046\n","Epoch 15/50\n","31/31 [==============================] - 3s 88ms/step - loss: 1.1445\n","Epoch 16/50\n","31/31 [==============================] - 3s 88ms/step - loss: 1.0885\n","Epoch 17/50\n","31/31 [==============================] - 3s 88ms/step - loss: 1.0371\n","Epoch 18/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.9919\n","Epoch 19/50\n","31/31 [==============================] - 3s 86ms/step - loss: 0.9482\n","Epoch 20/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.9019\n","Epoch 21/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.8546\n","Epoch 22/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.8130\n","Epoch 23/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.7694\n","Epoch 24/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.7322\n","Epoch 25/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.6897\n","Epoch 26/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.6468\n","Epoch 27/50\n","31/31 [==============================] - 3s 82ms/step - loss: 0.6114\n","Epoch 28/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.5719\n","Epoch 29/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.5349\n","Epoch 30/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.4936\n","Epoch 31/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.4589\n","Epoch 32/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.4301\n","Epoch 33/50\n","31/31 [==============================] - 3s 86ms/step - loss: 0.4010\n","Epoch 34/50\n","31/31 [==============================] - 3s 82ms/step - loss: 0.3739\n","Epoch 35/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.3510\n","Epoch 36/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.3335\n","Epoch 37/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.3160\n","Epoch 38/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.2952\n","Epoch 39/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.2707\n","Epoch 40/50\n","31/31 [==============================] - 3s 83ms/step - loss: 0.2473\n","Epoch 41/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.2284\n","Epoch 42/50\n","31/31 [==============================] - 3s 84ms/step - loss: 0.2078\n","Epoch 43/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.1907\n","Epoch 44/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.1738\n","Epoch 45/50\n","31/31 [==============================] - 3s 86ms/step - loss: 0.1572\n","Epoch 46/50\n","31/31 [==============================] - 3s 86ms/step - loss: 0.1446\n","Epoch 47/50\n","31/31 [==============================] - 3s 86ms/step - loss: 0.1352\n","Epoch 48/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.1224\n","Epoch 49/50\n","31/31 [==============================] - 3s 86ms/step - loss: 0.1096\n","Epoch 50/50\n","31/31 [==============================] - 3s 85ms/step - loss: 0.0976\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"kKkD5M6eoSiN"},"source":["### Generación de texto"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"zk2WJ2-XjkGz","outputId":"65cc78fc-d194-44d3-acb2-1d8cf2c8d2d2","executionInfo":{"status":"ok","timestamp":1568818694958,"user_tz":-120,"elapsed":179807,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":34}},"source":["tf.train.latest_checkpoint(checkpoint_dir)"],"execution_count":24,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'./training_checkpoints/ckpt_50'"]},"metadata":{"tags":[]},"execution_count":24}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"LycQ-ot_jjyu","outputId":"575cb534-a089-4eaf-f894-ca056b623177","executionInfo":{"status":"ok","timestamp":1568818694959,"user_tz":-120,"elapsed":179802,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":54}},"source":["model = build_model(vocab_size, embedding_dim, rnn_units, batch_size=1)\n","\n","model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))\n","\n","model.build(tf.TensorShape([1, None]))"],"execution_count":25,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:: Note that this layer is not optimized for performance. Please use tf.keras.layers.CuDNNLSTM for better performance on GPU.\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab_type":"code","id":"WvuwZBX5Ogfd","colab":{}},"source":["def generate_text(model, start_string):\n","\n"," num_generate = 500\n"," input_eval = [char2idx[s] for s in start_string]\n","\n"," input_eval = tf.expand_dims(input_eval, 0)\n"," text_generated = []\n","\n","\n"," temperature = 0.5\n","\n"," model.reset_states()\n"," for i in range(num_generate):\n"," predictions = model(input_eval)\n"," \n"," predictions = tf.squeeze(predictions, 0)\n","\n"," predictions = predictions / temperature\n"," predicted_id = tf.random.categorical(predictions, num_samples=1)[-1,0].numpy()\n","\n","\n"," input_eval = tf.expand_dims([predicted_id], 0)\n","\n"," text_generated.append(idx2char[predicted_id])\n","\n"," return (start_string + ''.join(text_generated))"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"ktovv0RFhrkn","outputId":"e462b01b-0462-459c-cd75-dd57e33509f8","executionInfo":{"status":"ok","timestamp":1568818697309,"user_tz":-120,"elapsed":182141,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":171}},"source":["print(generate_text(model, start_string=u\"Alcohol \"))"],"execution_count":27,"outputs":[{"output_type":"stream","text":["Alcohol \n","\r\n","\n","EBUNK9‡+[I**JJJxJK‡@'D]+k'J[‡!9‡DBX*X*WOIB­xRM‡]#WJ['IW\"@‡Wx@I%WGNYI%‡#xY[‡[%B[DIK‡]AQBJ/DB!W\"YYR‡?'IWQ%*YJII­H+B3Dx‡[A‡NQY*­5JQ+XIJwJWWJ\"N7D]YBCR‡YI‡%NHDK\n","U@[x3I­L…#JQJB#R<[L\n","G%IhBD6JK'#]UHIVIY[BTJ\"j%x[zHHD4‡YV9JKW\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"PsNst9-GeiZo","colab_type":"code","outputId":"27d0c31a-2c56-4e18-f07c-6beed97ed8dd","executionInfo":{"status":"ok","timestamp":1568818699459,"user_tz":-120,"elapsed":184282,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":54}},"source":["print(generate_text(model, start_string=u\"modelo\"))"],"execution_count":28,"outputs":[{"output_type":"stream","text":["modelo se usa para implementar modelos de redes muy grandes. Para nuestro ejemplo simple, vemos que indica que se requiere mucha expericencambiendo su funcionamos el proceso de aprendizaje de una red neuronal. Ademas veremos algunos de los parametros W y b de tal manera que se minimice la funcion de loss que usaremos para evaluar el grado de error es un espacio de dos cientificos y cientificas de datos y que puedan solurio en la comunidad de programadores ya que permite ample mescampo. Ahora, es el t\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"NFOd58ksem1a","colab_type":"code","outputId":"e6cdf1a2-e3b9-4f06-a022-96e2da3eea87","executionInfo":{"status":"ok","timestamp":1568818701619,"user_tz":-120,"elapsed":186436,"user":{"displayName":"Jordi Torres","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mDkD6udBQxc1Cx-zMrYPf0xjtQYxGjQEpB40wVHTg=s64","userId":"14559872433417615139"}},"colab":{"base_uri":"https://localhost:8080/","height":54}},"source":["print(generate_text(model, start_string=u\"activacion\"))"],"execution_count":29,"outputs":[{"output_type":"stream","text":["activacion si la primera visto y es usaramos una de las predicciones por computador ya se conocian en 1989; tambien los algoritmos fundamentales de Deep Learning puede hacerlo a traves de la como de la UPC model prodecir esta siguiente paralelas Deep Learning ello tenemos otra metrica llamada Sensitivity (o recall) que nos indica como de bien el modelo evita el BSC para referirse a las diferentes actualizaciones de su supercomputador Marenostrum que tanta la provecho de las nuevas se pasan en glabal de da\n"],"name":"stdout"}]}]} -------------------------------------------------------------------------------- /3.ComoSeEntrenaUnaRedNeuronal.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 3 Cómo se entrena una red neuronal" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "### Activation functions" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 2, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [ 23 | "import math\n", 24 | "import matplotlib.pyplot as plt\n", 25 | "import numpy as np\n", 26 | "\n", 27 | "x = np.arange(-6, 6, 0.1)" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": {}, 33 | "source": [ 34 | "Linear" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": 3, 40 | "metadata": {}, 41 | "outputs": [ 42 | { 43 | "data": { 44 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8lfX5//HXJyFhJBBkhRX2nkoCiFZLFBUVB45vtW6rVFs7/crWah3gaK21WqtWW6ttWhmCiDgwuFFAJYuwwt4zZJB5rt8fod+ftUjgnJPc59x5Px+PPB6c5L4/93V5zDt37tznOs7MEBER/4jxugAREQkvBbuIiM8o2EVEfEbBLiLiMwp2ERGfUbCLiPiMgl1ExGcU7CIiPqNgFxHxmUZeHLRNmzbWrVu3oPYtKSkhISEhvAV5RL1ELj/1o14iUzC9rFixYq+Zta1tO0+CvVu3bixfvjyofZcsWcLo0aPDW5BH1Evk8lM/6iUyBdOLc27T8WynSzEiIj6jYBcR8RkFu4iIz4Ql2J1zLZ1zs5xz+c65Vc65UeFYV0RETly4/nj6BLDIzK5wzsUDzcK0roiInKCQg905lwScCdwIYGYVQEWo64qISHDCcSmmO7AHeNE596Vz7nnnnD9uNBURiUIu1LfGc86lAUuB083sM+fcE8AhM7v7G9tNACYAJCcnp2ZkZAR1vOLiYhITE0OqOVKol8jlp37US2QorjDmra9gfK94msW5oHpJT09fYWZptW5oZiF9AO2BjV97fAbwxrH2SU1NtWBlZmYGvW+kUS+Ry0/9qBdvBQIBW7Byu6Xe/7b1nPKGvZO708yC6wVYbseRyyFfYzeznc65Lc65vma2GjgbyAt1XRGRaLfrUBl3v5bD23m7GNwpib/9YCT9O7So8+OG666YnwCvHLkjpgC4KUzriohEHTPjX8u38MAbq6ioCjDl/H784DvdaRRbPy8dCkuwm9lXQO3XfUREfG7zvlKmzM3i43X7GNG9FQ9fPoTuber3fhJPhoCJiPhNdcD4yycbeeyt1cTGOB64dBDfH9GFmBhX77Uo2EVEQrRmVxETZ2Xx1ZaDnNWvHQ9cOoiOLZt6Vo+CXUQkSBVVAZ55fz1PvreWxMaNeOKqk7l4aEecq/+z9K9TsIuIBCFr60Emzsoif2cRFw3tyK8uGkCbxMZelwUo2EVETsjhimp+9+4anvuwgLbNG/Pc9WmcMyDZ67L+g4JdROQ4fbp+H1PmZLFxXylXj0hh8vn9SWoa53VZ/0XBLiJSi0Nllcx8M5+/f7aZLq2a8fdbRnJarzZel/WtFOwiIseweNUups3NYXdRGbee0Z1fntOXpvGxXpd1TAp2EZGj2Fdczn2v5zF/5Xb6JjfnmetSOTmlpddlHRcFu4jI15gZ81du577X8ygqq+TnY3rzo9G9iG8UPe8kqmAXETliZ2EZ01/L4d1Vuxia0pJHLh9C3/bNvS7rhCnYRaTBMzMylm3hoTdWURkIMP3C/tx0endiPRgHEA4KdhFp0DbtK2Hy7Gw+LdjHqB6tmXn5YLq2ju43gVOwi0iDVB0wXvhoA795ZzVxMTHMuGwwVw1P8XwcQDgo2EWkwVm9s4iJs1aycmshY/q344FLB9M+qYnXZYWNgl1EGoyKqgBPZa7j6SXraNEkjievPoVxQzr44iz96xTsItIgfLn5AJNmZ7FmVzGXntyRey4aSKuEeK/LqhMKdhHxtcMV1fz2ndX8+aMNJLdowgs3pnFWv8ga2hVuCnYR8a1P1u1l8pxsNu8v5fsjuzDl/H40bxJ5Q7vCTcEuIr5TeLiSGQtXkbFsC91aNyNjwqmc2qO112XVGwW7iPjKO3m7mP5aNnuKyvnhmT34+Zg+ET+0K9wU7CLiC3uLy7l3fi4LsnbQr31znrs+jSGdo2NoV7gp2EUkqpkZ877azn2v51JSXs2d5/ThttE9iYuNnqFd4aZgF5Gotf3gYabNzSZz9R5O6VIztKt3cvQN7Qq3sAW7cy4WWA5sM7Nx4VpXROSbAgHjlc838/Cb+VQHjHvGDeCG07pF7dCucAvnGfvPgFVAizCuKSLyHzbsLWHS7Cw+37Cf03u1Zsb4IXRp3czrsiJKWILdOdcZuBB4EPhlONYUEfm6quoACwsqmPfuB8Q3iuGRy4dwZVpn340DCIdwnbH/DpgI6OKWiIRd3vZDTJqdRfa2Ss4dkMz9lw4iuYV/hnaFmzOz0BZwbhxwgZn9yDk3Gvjfo11jd85NACYAJCcnp2ZkZAR1vOLiYhITE0OoOHKol8jlp36iuZfKgDF/fSULCypJiIMrehhndE3wxVl6MM9Lenr6CjNLq3VDMwvpA5gBbAU2AjuBUuDlY+2TmppqwcrMzAx630ijXiKXn/qJ1l6Wb9xvZ/9miXWdtMB+kfGl7S8uj9pejiaYXoDldhy5HPKlGDObAkwB+NoZ+7WhrisiDVNpRRWPvrWav3yykY5JTfnLTcMZ3bed12VFFd3HLiIR46O1e5k8J4utBw5z/aiuTBzbj8TGiqkTFdb/Yma2BFgSzjVFxP8KSyt5cGEe/1q+lR5tEvjXD0cxonsrr8uKWvpRKCKeWpSzk7vn5bC/pILbvtuTn4/pTZO4hjW0K9wU7CLiiT1FNUO73sjewYAOLXjxxuEM6pTkdVm+oGAXkXplZsz5Yhu/XpDH4cpq7jqvLxPO7NGgh3aFm4JdROrN1gOlTJ2bwwdr9pDa9SQevnwIvdpF5z32kUzBLiJ1LhAw/rZ0Ew8vygfg3osGcP2obsRoaFedULCLSJ1av6eYSbOyWL7pAGf0bsND4weT0kpDu+qSgl1E6kRldYBnPyjgicVraRoXy2NXDuXyYZ18MQ4g0inYRSTscrcXMnFWFrnbD3H+oPbcd8lA2jXX0K76omAXkbApq6zmyffW8sz7BZzULJ5nrh3G2EEdvC6rwVGwi0hYLN+4n4mzsyjYU8KVqZ2ZfuEAkprFeV1Wg6RgF5GQFJdX8eiifF5auomOSU156eYRnNmnrddlNWgKdhEJ2vtr9jB1TjbbCw9zw6hu3HVeXxI0tMtzegZE5IQdLK3g/gWrmP3FVnq2TeDVH44irZuGdkUKBbuInJA3s3dw97xcDpRWcEd6L+44q5eGdkUYBbuIHJfdh8q4Z14ui3J3MqhTC/5683AGdtTQrkikYBeRYzIzXl2xlQcW5FFWFWDS2H7cekZ3GmloV8RSsIvIt9qyv5Spc7P5cO1eRnRrxczLB9OjrYZ2RToFu4j8l+qA8dKnG3lk0WpiHNx/6SCuGdFFQ7uihIJdRP7Dut1FTJyVxRebDzK6b1seHD+YTi2bel2WnAAFu4gANUO7/vT+en6/eB3NGsfy+PeGcunJGtoVjRTsIkL21kLumrWS/J1FXDikA/deNJC2zRt7XZYEScEu0oCVVVbzu3fX8tyHBbROiOdP16Vy3sD2XpclIVKwizRQnxXsY/KcbDbsLeF7aSlMvbA/SU01tMsPFOwiDUxRWSUPL8rn5aWbSWnVlFduGcnpvdp4XZaEUcjB7pxLAV4CkgEDnjWzJ0JdV0TCLzN/N9PmZrPjUBk/+E537jy3D83idX7nN+F4RquAO83sC+dcc2CFc+4dM8sLw9oiEgYHSir49YI85n65jd7tEpl9+2kM63KS12VJHQk52M1sB7DjyL+LnHOrgE6Agl3EY2bG5zuquPO371N4uJKfntWLH5/Vi8aNNLTLz8L6O5hzrhtwCvBZONcVkRO361AZ01/L4Z28coZ0TuLlW0bSv0MLr8uSeuDMLDwLOZcIvA88aGZzjvL1CcAEgOTk5NSMjIygjlNcXExioj9mVaiXyBXN/ZgZH2yrIiO/gqoAXNjFuKhPArE+GAcQzc/LNwXTS3p6+gozS6t1QzML+QOIA94Cfnk826emplqwMjMzg9430qiXyBWt/WzaW2Lff+5T6zppgf3PM5/Yhj3FUdvL0TT0XoDldhwZG467YhzwZ2CVmf021PVE5MRVB4wXP97AY2+vplFMDA+OH8TVw2uGdm30ujipd+G4xn46cB2Q7Zz76sjnpprZwjCsLSK1WLOrZmjXV1sOcla/djw4fhAdkjS0qyELx10xHwHRf/FOJMpUVAX445L1/CFzLc2bxPHEVSdz8dCOGtoleuWpSDRaueUgk2Znkb+ziIuGduTeiwbQOlFDu6SGgl0kihyuqObxd9fw/IcFtG3emOeuT+OcAclelyURRsEuEiU+Xb+PKXOy2LivlKtHpDDlgv60aKKhXfLfFOwiEe5QWSUzFubzj88307V1M/5+60hO66mhXfLtFOwiEWzxql1Mm5vD7qIyJpzZg1+M6UPTeI0DkGNTsItEoH3F5dz3eh7zV26nb3JznrkulZNTWnpdlkQJBbtIBDEz5q/czn2v51FUVsnPx/TmR6N7Ed8oxuvSJIoo2EUixM7CMqbNzWZx/m6GprTkkcuH0Ld9c6/LkiikYBfxWCBgZCzbwoyFq6gMBJh+YX9uOr27L4Z2iTcU7CIe2ri3hMlzslhasJ/TerZmxmWD6do6weuyJMop2EU8UFUd4IWPN/Cbt9cQHxvDzMsG873hKRoHIGGhYBepZ/k7DzFpVhYrtxYypn8yD1w6iPZJTbwuS3xEwS5ST8qrqnkqcz1PZ64jqWkcT159CuOGdNBZuoSdgl2kHny5+QCTZmexZlcxl57ckXsuGkirhHivyxKfUrCL1KHSiip++/YaXvh4A8ktmvDCjWmc1U9Du6RuKdhF6sgn6/YyeU42m/eXcs3ILkw+vx/NNbRL6oGCXSTMCg9XMmPhKjKWbaF7mwQyJpzKqT1ae12WNCAKdpEwejt3J9Nfy2FvcTk/PLMHvzinD03iNLRL6peCXSQM9haXc+/8XBZk7aBf++Y8f0MaQzpraJd4Q8EuEgIzY95X27nv9VxKyqu585w+3Da6J3GxGtol3lGwiwRp+8HDTJubTebqPZzSpWZoV+9kDe0S7ynYRU5QIGC88vlmZi5cRcDgnnEDuOG0bhraJRFDwS5yAjbsLWHS7Cw+37Cf7/Rqw4zLBpPSqpnXZYn8BwW7yHGoqg7w/EcbePydNTRuFMMjVwzhytTOGgcgEUnBLlKLvO2HmDQ7i+xthZw3MJn7LxlEuxYa2iWRKyzB7pwbCzwBxALPm9nMcKwr4qXyqmr+8N46/rhkPS2bxfH0NcO4YHAHr8sSqVXIwe6ciwWeAs4BtgLLnHPzzSwv1LVFvLLuQDUP/P4j1u0u5rJhnbhn3ABaNtPQLokO4ThjHwGsM7MCAOdcBnAJoGCXqFNSXsVjb6/mL5+V0bFlU/5y03BG923ndVkiJ8SZWWgLOHcFMNbMbjny+DpgpJnd8Y3tJgATAJKTk1MzMjKCOl5xcTGJiYkh1Rwp1EtkydlbzV9yy9l72Dijg/H9gQk0bRT9fxz1w3Pzbw29l/T09BVmllbbdvX2x1MzexZ4FiAtLc1Gjx4d1DpLliwh2H0jjXqJDIWllTzwRh6vrthKjzYJPH39EEo3ZUVtP98Uzc/NN6mX4xOOYN8GpHztcecjnxOJeItydnL3vBz2l1Rw++ie/Ozs3jSJi2XJJq8rEwleOIJ9GdDbOdedmkC/Cvh+GNYVqTO7i8r41bxc3szZyYAOLXjxxuEM6pTkdVkiYRFysJtZlXPuDuAtam53fMHMckOuTKQOmBlzvtjGrxfkcbiymrvO68uEM3toaJf4SliusZvZQmBhONYSqStbD5QydW4OH6zZQ1rXk5h5+RB6tfPHH+JEvk6vPBXfCwSMlz/bxMNv5mPAfRcP5LpTuxKjoV3iUwp28bX1e4qZNCuL5ZsOcEbvmqFdnU/S0C7xNwW7+FJldYBnPyjgicVraRoXy2NXDuXyYZ00tEsaBAW7+E7OtkImzc4id/shLhjcnnsvHki75hraJQ2Hgl18o6yymt8vXsufPiigVUI8z1w7jLGDNLRLGh4Fu/jCso37mTQ7i4I9JVyZ2pnpFw4gqVmc12WJeELBLlGtuLyKRxfl89LSTXRq2ZS//WAEZ/Ru63VZIp5SsEvUen/NHqbOyWZ74WFuGNWNu87rS0Jj/S8tou8CiToHSyu4f8EqZn+xlZ5tE5h12yhSu7byuiyRiKFgl6iyMHsH98zL4WBpJXek9+KOs3rRJC7W67JEIoqCXaLC7kNl3DMvl0W5OxnUqQV/vXkEAztqaJfI0SjYJaKZGa+u2MoDC/IorwowaWw/bj2jO400tEvkWynYJWJt2V/K1LnZfLh2LyO6tWLm5YPp0VZDu0Rqo2CXiFMdMF76dCOPLFpNjIP7Lx3ENSO6aGiXyHFSsEtEWbe7iImzsvhi80FG923Lg+MH06llU6/LEokqCnaJCJXVAf70/np+v3gdCY1jefx7Q7n0ZA3tEgmGgl08l721kLtmrSR/ZxHjhnTg3osH0iaxsddliUQtBbt4pqyymsffXcNzHxTQJrExz16XyrkD23tdlkjUU7CLJz4r2MfkOdls2FvCVcNTmHJBf5KaamiXSDgo2KVeFZVV8vCifF5eupmUVk155ZaRnN6rjddlifiKgl3qTWb+bqbOzWbnoTJ+8J3u3HluH5rF639BkXDTd5XUuf0lFfz69Vxe+2o7vdslMvv20xjW5SSvyxLxLQW71BkzY0HWDu6dn0vh4Up+elYvfnxWLxo30tAukbqkYJc6setQGdNfy+GdvF0M6ZzEK7eOpF/7Fl6XJdIghBTszrlHgYuACmA9cJOZHQxHYRKdzIx/LtvCgwtXUVEVYOoF/bj5dA3tEqlPoZ6xvwNMMbMq59zDwBRgUuhlSTTavK+UyXOy+GT9PkZ2b8XDlw+hW5sEr8sSaXBCCnYze/trD5cCV4RWjkSj6oDx1sZK5i5+n0YxMTw0fjBXDU/R0C4RjzgzC89Czr0O/NPMXv6Wr08AJgAkJyenZmRkBHWc4uJiEhP9MbrVD71sLQrwQk45BYUBhraN5YaB8bRqEv2XXfzw3PybeolMwfSSnp6+wszSatuu1mB3zr0LHO113tPMbN6RbaYBacBldhw/KdLS0mz58uW1bXZUS5YsYfTo0UHtG2miuZeKqgBPL1nHU5nraN4kjv/p6Zh09dm+GdoVzc/NN6mXyBRML8654wr2Wi/FmNmYWg50IzAOOPt4Ql2i38otB5k4K4vVu4q4eGhHfnXRALKXf+qbUBeJdqHeFTMWmAh818xKw1OSRKrDFTVDu57/sIB2zZvw/PVpjBmQ7HVZIvINod4V8wegMfDOkbO1pWZ2W8hVScT5dP0+Js/JYtO+Uq4e0YUpF/SjRRMN7RKJRKHeFdMrXIVIZDpUVsmMhfn84/PNdG3djL/fOpLTempol0gk0ytP5VstXrWLaXNz2F1UxoQze/CLMX1oGq9xACKRTsEu/2VfcTn3vZ7H/JXb6de+OX+6LpWhKS29LktEjpOCXf6PmTF/5XbunZ9LcXkVvxjTh9tH9yS+UfTfly7SkCjYBYAdhYeZPjeHxfm7OTmlJY9cMYQ+yc29LktEgqBgb+ACAeMfyzYzY2E+VYEA0y/sz02ndydW4wBEopaCvQHbuLeEyXOyWFqwn9N6tmbmZUPo0rqZ12WJSIgU7A1QVXWAFz7ewG/eXkN8bAwzLqsZ2qVXjor4g4K9gcnfeYhJs7JYubWQMf2TeeDSQbRPauJ1WSISRgr2BqK8qpqnMtfzdOY6kprG8eTVpzBuSAedpYv4kIK9Afhi8wEmzcpi7e5ixp/SiXvGDeCkhHivyxKROqJg97HSiip+8/YaXvh4A+1bNOHFG4eT3q+d12WJSB1TsPvUx+v2MnlOFlv2H+baU7swaWw/mmtol0iDoGD3mcLDlcxYuIqMZVvo3iaBf044lZE9WntdlojUIwW7j7ydu5Ppr+Wwr6SC277bk5+P6U2TOA3tEmloFOw+sKeonHtfz+WNrB3079CCP98wnMGdk7wuS0Q8omCPYmbGa19t477X8ygtr+Z/z+3DD7/bk7hYDe0SacgU7FFq28HDTJubzZLVexjWpWZoV692GtolIgr2qBMIGK98tomZb+YTMPjVRQO4flQ3De0Skf+jYI8iBXuKmTw7m8837ueM3m14aPxgUlppaJeI/CcFexSoqg7w/EcbePydNTRuFMOjVwzhitTOGgcgIkelYI9wedsPMXH2SnK2HeK8gcncf8kg2rXQ0C4R+XYK9ghVVlnNH95bxzPvr6dls3j+eM0wzh/cweuyRCQKKNgj0IpN+5k4K4v1e0q4bFjN0K6WzTS0S0SOj4I9gpSUV/HoW6v566cb6ZjUlL/ePILv9mnrdVkiEmXCEuzOuTuBx4C2ZrY3HGs2NB+u3cOUOdlsPXCY60d1ZeLYfiQ21s9dETlxISeHcy4FOBfYHHo5DU9haSUPvJHHqyu20qNtAq/eNorh3Vp5XZaIRLFwnBI+DkwE5oVhrQZlxa4q7nr8ffaXVPCj0T356dka2iUioQsp2J1zlwDbzGyl7qk+fruLyrh3fi4Ls8sZ2LEFL944nEGdNLRLRMLDmdmxN3DuXaD9Ub40DZgKnGtmhc65jUDat11jd85NACYAJCcnp2ZkZARVcHFxMYmJiUHt6zUz4+PtVfwjv4Lyajg/xbikbwKNfDAOIJqfl6PxUz/qJTIF00t6evoKM0urdUMzC+oDGAzsBjYe+aii5jp7+9r2TU1NtWBlZmYGva+XNu8rsWufX2pdJy2wy5/+2NbuKoraXo7GT72Y+asf9RKZgukFWG7Hkc9BX4oxs2zg/95As7Yz9oYqEDBe+nQjj7y1Ggf8+pKBXDuyKzExjq15XlcnIn6k++nq0LrdxUyencXyTQc4s09bHho/iM4naWiXiNStsAW7mXUL11rRrrI6wLMfFPDE4rU0jYvlN1cO5bJhnTS0S0Tqhc7YwyxnWyGTZmeRu/0QFwxuz70XD6Rdcw3tEpH6o2APk7LKan6/eC1/+qCAVgnxPHPtMMYO0tAuEal/CvYwWLZxP5NmZVGwt4QrUzsz/cIBJDWL87osEWmgFOwhKC6v4pFF+bz06SY6tWzK334wgjN6a2iXiHhLwR6kzNW7mTYnmx2HyrjxtG7cdV5fEjS0S0QigJLoBB0oqeD+BXnM+XIbPdsmMOu2UaR21dAuEYkcCvbjZGYszN7Jr+bncLC0kjvSe/GTs3vRuJGGdolIZFGwH4fdh8q4e14Ob+XuYnCnJF66eSQDOrbwuiwRkaNSsB+DmfHqiq08sCCP8qoAk8b249YzutMoNsbr0kREvpWC/Vts2V/KlDnZfLRuLyO6t2LmZYPp0dYfU+VExN8U7N9QHTD++slGHn1rNbExjvsvHcQ1I7oQ44PRuiLSMCjYv2btriImzs7iy80HSe/blgfHD6Zjy6ZelyUickIU7NQM7frjkvX84b11JDSO5XffO5lLTu6ooV0iEpUafLBnby3krlkryd9ZxLghHbj34oG0SWzsdVkiIkFrsMFeVlnN4++u4fkPN9AmMZ7nrk/jnAHJXpclIhKyBhnsnxXsY/KcbDbsLeHqESlMPr8/SU01tEtE/KFBBXtRWSUz38znlc8206VVM/5+y0hO69XG67JERMKqwQT7e/m7mDY3h12HyrjlO9355bl9aBbfYNoXkQbE98m2v6SCX7+ey2tfbad3u0Sevv00TulyktdliYjUGd8Gu5nxetYO7pufS+HhSn52dm9+lN5TQ7tExPd8Gew7C8uY/lo2767azdDOSbxy60j6tdfQLhFpGHwV7GZGxrItPPTGKiqqA0y9oB83n66hXSLSsPgm2DftK2Hy7Gw+LdjHqT1aMfOyIXRrk+B1WSIi9S7qg706YLz48QYee3s1cTExPDR+MFcNT9HQLhFpsEIOdufcT4AfA9XAG2Y2MeSqjtPqnTVDu1ZuOcjZ/drxwPhBdEjS0C4RadhCCnbnXDpwCTDUzMqdc+3CU9axVVQFeHrJOp7KXEfzJnE8cdXJXDxUQ7tERCD0M/bbgZlmVg5gZrtDL+nYCg5WM+PJj1i9q4hLTu7Iry4aSKuE+Lo+rIhI1Ag12PsAZzjnHgTKgP81s2Whl3V0Ty5ey2+XlpHcAv58Qxpn99fQLhGRb3JmduwNnHsXaH+UL00DHgQygZ8Cw4F/Aj3sKIs65yYAEwCSk5NTMzIyTrjYpduryN5dxjUDE2gWF/2XXYqLi0lM9Mfb7fmpF/BXP+olMgXTS3p6+gozS6t1QzML+gNYBKR/7fF6oG1t+6WmplqwMjMzg9430qiXyOWnftRLZAqmF2C5HUc2h/rKndeAdADnXB8gHtgb4poiIhKCUK+xvwC84JzLASqAG478VBEREY+EFOxmVgFcG6ZaREQkDDRERUTEZxTsIiI+o2AXEfEZBbuIiM8o2EVEfKbWV57WyUGd2wNsCnL3NvjnXnn1Ern81I96iUzB9NLVzNrWtpEnwR4K59xyO56X1EYB9RK5/NSPeolMddmLLsWIiPiMgl1ExGeiMdif9bqAMFIvkctP/aiXyFRnvUTdNXYRETm2aDxjFxGRY4jaYHfO/cQ5l++cy3XOPeJ1PaFyzt3pnDPnXBuvawmWc+7RI89JlnNurnOupdc1nSjn3Fjn3Grn3Drn3GSv6wmWcy7FOZfpnMs78j3yM69rCpVzLtY596VzboHXtYTKOdfSOTfryPfLKufcqHCuH5XB/o030R4IPOZxSSFxzqUA5wKbva4lRO8Ag8xsCLAGmOJxPSfEORcLPAWcDwwArnbODfC2qqBVAXea2QDgVODHUdzLv/0MWOV1EWHyBLDIzPoBQwlzX1EZ7HjwJtp17HFgIhDVf/Aws7fNrOrIw6VAZy/rCcIIYJ2ZFRwZSZ1BzQlE1DGzHWb2xZF/F1ETHJ28rSp4zrnOwIXA817XEirnXBJwJvBnqBl/bmYHw3mMaA32f7+J9mfOufedc8O9LihYzrlLgG1mttLrWsLsZuBNr4s4QZ2ALV97vJUoDsN/c851A04BPvO2kpD8jpqTn4DXhYRBd2AP8OKRS0vPO+cSwnmAUN9Bqc7U8ibajYBW1PyKORz4l3PuqG+iHQlq6WUqNZdhosKxejGzeUe2mUZMPxM0AAABn0lEQVTNpYBX6rM2+W/OuURgNvBzMzvkdT3BcM6NA3ab2Qrn3Giv6wmDRsAw4Cdm9plz7glgMnB3OA8QkcxszLd9zTl3OzDnSJB/7pwLUDN3YU991Xcivq0X59xgan56r3TOQc2liy+ccyPMbGc9lnjcjvW8ADjnbgTGAWdH6g/aY9gGpHztcecjn4tKzrk4akL9FTOb43U9ITgduNg5dwHQBGjhnHvZzKL13du2AlvN7N+/Qc2iJtjDJlovxfjiTbTNLNvM2plZNzPrRs0TPixSQ702zrmx1Py6fLGZlXpdTxCWAb2dc92dc/HAVcB8j2sKiqs5U/gzsMrMfut1PaEwsylm1vnI98hVwHtRHOoc+f7e4pzre+RTZwN54TxGxJ6x10Jvoh2Z/gA0Bt458hvIUjO7zduSjp+ZVTnn7gDeAmKBF8ws1+OygnU6cB2Q7Zz76sjnpprZQg9rkv/vJ8ArR04gCoCbwrm4XnkqIuIz0XopRkREvoWCXUTEZxTsIiI+o2AXEfEZBbuIiM8o2EVEfEbBLiLiMwp2ERGf+X/fa9sk+bHEEQAAAABJRU5ErkJggg==\n", 45 | "text/plain": [ 46 | "" 47 | ] 48 | }, 49 | "metadata": {}, 50 | "output_type": "display_data" 51 | } 52 | ], 53 | "source": [ 54 | "def linear(x):\n", 55 | " a = []\n", 56 | " for item in x:\n", 57 | " a.append(item)\n", 58 | " return a\n", 59 | "\n", 60 | "y = linear(x)\n", 61 | "\n", 62 | "plt.plot(x,y)\n", 63 | "plt.grid()\n", 64 | "plt.show()\n" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "Sigmoid" 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": 4, 77 | "metadata": { 78 | "scrolled": true 79 | }, 80 | "outputs": [ 81 | { 82 | "data": { 83 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VfWd//HXJztJCFsgbFF2EBDZxL2CC+JSnaptta3VqsPUqf7sqK3aVp2xM61tnbG2ta1r1akjWrVClYpawdYqqwgYNsMihC2sIfvNvfczfyT6yyCQkNxw7vJ+Ph73kdx7v/fm/SXknZPvPfccc3dERCS5pAUdQEREYk/lLiKShFTuIiJJSOUuIpKEVO4iIklI5S4ikoRU7iIiSUjlLiKShFTuIiJJKCOoL1xYWOgDBgxo02Orq6vJy8uLbaCAJNNcILnmo7nEp1Sfy5IlS3a5e8+WxgVW7gMGDGDx4sVteuy8efOYPHlybAMFJJnmAsk1H80lPqX6XMzs49aM07KMiEgSUrmLiCQhlbuISBJSuYuIJKEWy93MnjCzcjP78BD3m5n9wsxKzWy5mY2PfUwRETkSrdlyfxKYdpj7zweGNl2mA79pfywREWmPFsvd3f8K7DnMkEuAp73RfKCrmfWJVUARETlysdjPvR+wudn1sqbbtsXguUVE4oa7Ux+OUhuKUBeOUBuKUB+ONl4amn0ejhAKR2mIRAmFo4QiTkMkSkPTbWcfV8QJxV07NOtRfROTmU2ncemGoqIi5s2b16bnqaqqavNj400yzQWSaz6aS3xq61yi7lQ1QFXIqW5wqhqcmganpgFqwk5N2KkNQ12zj3WRxo/1EQhFGj/G4qzTFds3sfeYzA79vsSi3LcAxc2u92+67TPc/RHgEYCJEyd6W99llurvUItnyTQfzSU+HTiXmlCYrfvq2F5Rx/b9dezYX8fOyvrGS1U9u6vq2V0doqK2AT9MM+dmpZOfnUF+TuOlZ3YGedkZ5GWlk5udQW5mOrlZ6eRkpZOTkU6nrHRyMtPIyUgnJzOd7Iw0sjPTyEpPJysjjeyMNDIz0shKTyMrI43MdCMzPY2MNMPMDjqXWIpFuc8CbjSzGcBJQIW7a0lGRGIiGnW27a9j465qNuyq5p3VIZ4rW8KmPTVs3VfL3pqGzzymc3YGPTtnU5ifzbCizvTIz6J7XjbdczPplpdF19wsunbKpEunTAo6ZdI5J4PM9OTaM7zFcjezZ4HJQKGZlQH3AJkA7v5bYDZwAVAK1ADf6KiwIpK83J3yynpWbt3P6u2VrN1RyUfllawrr6a2IfLpuIw0OLZHJcXdcxlb3JW+XTvRt2sOfbp0ondBDkUFOXTKSg9wJvGhxXJ39ytbuN+Bb8UskYikhL3VIZZu3ssHmytYtnkfJVsr2FUV+vT+Pl1yGNIrnysmdWdwz3wGFeYxoDCP1Uvnc9aUycEFTxCBHRVSRFJL+f463lu/m/nrd7No415Ky6sASDMY2qszk4f3YlTfAkb2KWBE7wK65GYe9HnWNq1Xy+Gp3EWkQ9SHIyzcsIe31+zk7bU7+aipzDvnZDDx2G58YVw/JhzbjeP7dSEvW1UUa/oXFZGYqaoP85dVO3i9ZAdvr91JVX2YrPQ0Jg3szuUT+nPq4EJG9i0gPU1b3x1N5S4i7VIfjjB39U5mfrCFt1aXUx+OUpifzedP6MPZI4o4dUgPcrNUNUeb/sVFpE1Wb9/Psws2MXPZVvbVNFCYn80VJxZz4Zi+TDy2G2naOg+Uyl1EWq0hEmX2im089e5G3t+0j6z0NKaOKuKyCf05Y0ghGUm2r3giU7mLSIv21zXwzPxNPPnuBnbsr2dgYR4/uPA4Lhvfn255WUHHk4NQuYvIIe2tDvH4Oxt46r2NVNaFOX1IIfddOoYzh/XUskucU7mLyGdU1jXw+DsbeOxvG6gOhZk2qjffmjKE0f26BB1NWknlLiKfCkeiPLtwEw+8+RF7qkNMG9WbW6YOY1hR56CjyRFSuYsIAO98tIt/+1MJH5VXcfKg7tx5/nEdfsxx6Tgqd5EUV15Zx7+/sopZy7ZyTPdcHr5qAlNHFn16WFpJTCp3kRTl7rywpIx7X1lJfUOUm88eyg2TB5OTqSMqJgOVu0gK2rG/jjtfWsFbq8uZNKA7P77seAb3zA86lsSQyl0kxbxesp3vvricuoYId180kmtOHaDdGpOQyl0kRdQ1RPjx7FU89d7HjOpbwC+uHKet9SSmchdJAbtro3zp4fdYXlbB9acP5DvThpOdobX1ZKZyF0lyfy/dxT3v1mJpDTxy1QSmjuoddCQ5ClTuIkns9/M/5p5ZJRTlGr//p9MYpGWYlKFyF0lCkajzH6+u4om/b+CsEb34Yv8qFXuK0fE5RZJMXUOEbz3zPk/8fQPXnjaQR78+kU4Z2hsm1WjLXSSJ7K9rYPrTi5m/fg93XzSSa08fGHQkCYjKXSRJ7KkOcdXjC1izvZIHrxjLJWP7BR1JAqRyF0kCu6rq+eqjC9i4u5pHr57IlOG9go4kAVO5iyS48so6vvLoAsr21vDENSdy2pDCoCNJHFC5iySwPdUhvvroArbuq+XJb0zi5EE9go4kcUJ7y4gkqP11DXz9iQVs2lPD41efqGKX/0PlLpKAakMRrv3dItZsr+S3X5vAKYNV7PJ/aVlGJMGEI1FuenYpSzbt5aGvjGfKCL14Kp+lLXeRBOLu3DWzhDdX7eDfLh7FBcf3CTqSxCmVu0gCeWhuKc8u3MQ/Tx7M108ZEHQciWOtKnczm2Zma8ys1MzuOMj9x5jZXDNbambLzeyC2EcVSW2zV2zj/tfX8g9j+/Kd84YHHUfiXIvlbmbpwEPA+cBI4EozG3nAsB8Az7v7OOAK4NexDiqSylaUVXDL8x8w/piu3HfZGJ28WlrUmi33SUCpu6939xAwA7jkgDEOFDR93gXYGruIIqmtfH8d1z+9iB552Tx81USdwFpaxdz98APMLgemufv1TdevAk5y9xubjekDvA50A/KAc9x9yUGeazowHaCoqGjCjBkz2hS6qqqK/PzkOHxpMs0Fkms+8TCXcNT5ycI6Pq6MctfJnSju3LaXyeJhLrGS6nOZMmXKEnef2OJAdz/sBbgceKzZ9auAXx0w5hbg1qbPTwFWAmmHe94JEyZ4W82dO7fNj403yTQX9+SaTzzM5Z6ZH/qxt7/iMz/Y0q7niYe5xEqqzwVY7C30tru3allmC1Dc7Hr/ptuauw54vumXxXtADqADXIi0wx+XlvHkuxu5/vSBXHxC36DjSIJpTbkvAoaa2UAzy6LxBdNZB4zZBJwNYGbH0VjuO2MZVCSVlJZX8r2XPuSkgd254/wRQceRBNRiubt7GLgRmAOsonGvmBIzu9fMLm4adivwj2a2DHgWuKbpzwcROUKNZ1JaSm5WOr+4chwZ6Xo7ihy5Vh1+wN1nA7MPuO3uZp+vBE6LbTSR1PRvfyphzY5Knrp2EkUFOUHHkQSlTQKROPLq8m08u3AzN0wezJnDegYdRxKYyl0kTmyrqOV7f1zB2OKu3HLusKDjSIJTuYvEgWjUufX5ZTREovz8y2PJ1Dq7tJP+B4nEgcff2cC763Zzz+dHMqAwL+g4kgRU7iIB+2hHJT+bs4apI4v40sTilh8g0goqd5EAhSNRbv3DMvJzMvjRpcfrgGASMzoTk0iAHv7repaXVfDQV8ZTmJ8ddBxJItpyFwnImu2V/PzNtVw4pg8XjtEZlSS2VO4iAYhEne++uJzOOZnce/GooONIElK5iwTgqXc3smzzPu75/Eh6aDlGOoDKXeQoK9tbw/2vr2Hy8J462qN0GJW7yFHk7vzg5Q8B+Pd/GK29Y6TDqNxFjqJXV2xj3pqd3Dp1OP275QYdR5KYyl3kKKmsa+DeP61kVN8Crj7l2KDjSJLTfu4iR8kDb3zEzqp6Hvn6RB2jXTqc/oeJHAUlWyt48t0NfGXSMYwt7hp0HEkBKneRDubu3D2zhG65WXz3PJ0yT44OlbtIB/vj0i0s+Xgvt58/gi65mUHHkRShchfpQJV1Dfxo9mrGFnfl8vH9g44jKUQvqIp0oF/85SN2V9fz+NUTSUvTPu1y9GjLXaSDlJZX8bu/b+TLE4s5QS+iylGmchfpID+avYpOmencdt7woKNIClK5i3SAt9fu5K3V5dx09hAdp10CoXIXibFwJMoPX1nJsT1yufrUAUHHkRSlcheJsf9ZuInS8iq+d8FxZGekBx1HUpTKXSSG9tc18MAbazl5UHemjiwKOo6kMJW7SAz9eu469tY08IMLR+pwvhIolbtIjJTtreGJv2/g0nH9GN2vS9BxJMWp3EVi5P45azDQro8SF1TuIjGwoqyClz/YyvVnDKRv105BxxFRuYu0l7vz4z+vonteFt88c3DQcUSAVpa7mU0zszVmVmpmdxxizJfMbKWZlZjZ/8Q2pkj8+utHu3h33W5uOmsInXN01EeJDy0eOMzM0oGHgHOBMmCRmc1y95XNxgwF7gROc/e9ZtarowKLxJNo1Lnvz6sp7t6Jr56kU+dJ/GjNlvskoNTd17t7CJgBXHLAmH8EHnL3vQDuXh7bmCLxaeayLazatp/bpg4nK0OrnBI/zN0PP8DscmCau1/fdP0q4CR3v7HZmJeBtcBpQDrwr+7+2kGeazowHaCoqGjCjBkz2hS6qqqK/Pz8Nj023iTTXCC55tPSXBqizp1/qyUv07jnlBzS4ni/9lT6viSStsxlypQpS9x9YosD3f2wF+By4LFm168CfnXAmFeAPwKZwEBgM9D1cM87YcIEb6u5c+e2+bHxJpnm4p5c82lpLr97Z70fe/sr/te15UcnUDuk0vclkbRlLsBib6G33b1VyzJbgOJm1/s33dZcGTDL3RvcfQONW/FDW/HcIgmpqj7ML98q5dTBPTh9SGHQcUQ+ozXlvggYamYDzSwLuAKYdcCYl4HJAGZWCAwD1scwp0hceeKdDeyuDvGd84brMAMSl1osd3cPAzcCc4BVwPPuXmJm95rZxU3D5gC7zWwlMBf4jrvv7qjQIkHaUx3ikb+u57xRRYw7plvQcUQOqlXnUHX32cDsA267u9nnDtzSdBFJar+ZV0pNKMxtU3WYAYlf2ndL5Ahsq6jlqfc+5gvj+jO0qHPQcUQOSeUucgR++VYp7s63z9H+AhLfVO4irbRxVzXPL9rMlZOOobh7btBxRA5L5S7SSj9/cy0Z6caNU4YEHUWkRSp3kVZYs72Smcu2cvWpA+hVkBN0HJEWqdxFWuGBN9aSn5XBNz+nQ/pKYlC5i7RgRVkFr5Vs57ozBtItLyvoOCKtonIXacH9r6+ha24m150+MOgoIq2mchc5jEUb9/D22p1888zBOhGHJBSVu8ghuDv3z1lDYX42V58yIOg4IkdE5S5yCKv2RFmwYQ83ThlMp6z0oOOIHBGVu8hBuDsvrg3Rt0sOV550TNBxRI6Yyl3kIOauKWddRZQbzxpKdoa22iXxqNxFDhCNOv/5+lp6djK+OLF/0HFE2kTlLnKAOSXbKdm6n0uGZJKZrh8RSUytOp67SKqIRJ0H3lzL4J55nNr38CePF4ln2iwRaeaV5VtZu6OKfzl3GGk6fZ4kMJW7SJNwJMoDb6xlRO/OXDC6T9BxRNpF5S7S5KX3t7Bxdw23Th1OWpq22iWxqdxFgPpwhAf/8hEn9O/COcf1CjqOSLup3EWA5xZtZsu+Wm6dOhzTWrskAZW7pLzaUIRfvlXKpAHdOWNoYdBxRGJC5S4p7+n3NrKzsp7bztNWuyQPlbuktMq6Bn779jo+N6wnkwZ2DzqOSMyo3CWlPf7OBvbWNHDrucOCjiISUyp3SVl7q0M89rcNTBvVmxOKuwYdRySmVO6Ssn779jqqQ2Fumaqtdkk+KndJSTv21/Hkuxv5wth+DCvqHHQckZhTuUtK+sVfPiISdb59jrbaJTmp3CXlbNxVzXOLNnPFpGKO6ZEbdByRDtGqcjezaWa2xsxKzeyOw4y7zMzczCbGLqJIbP3XG2vJTE/j/501NOgoIh2mxXI3s3TgIeB8YCRwpZmNPMi4zsDNwIJYhxSJlZKtFcxatpVrTx9Ar4KcoOOIdJjWbLlPAkrdfb27h4AZwCUHGfdD4CdAXQzzicTUz+asoUunTKZ/bnDQUUQ6VGvKvR+wudn1sqbbPmVm44Fid381htlEYmr++t3MW7OTf548mC6dMoOOI9Kh2n2aPTNLA/4LuKYVY6cD0wGKioqYN29em75mVVVVmx8bb5JpLhC/83F3fji/jm7ZxsDwJubN29ziY+J1Lm2hucSnDp2Lux/2ApwCzGl2/U7gzmbXuwC7gI1NlzpgKzDxcM87YcIEb6u5c+e2+bHxJpnm4h6/83l1+VY/9vZX/LmFm1r9mHidS1toLvGpLXMBFnsLve3urVqWWQQMNbOBZpYFXAHMavbLocLdC919gLsPAOYDF7v74lj88hFpr4ZIlJ/NWcOwonwum9A/6DgiR0WL5e7uYeBGYA6wCnje3UvM7F4zu7ijA4q014xFm9mwq5rvnjeCdJ0+T1JEq9bc3X02MPuA2+4+xNjJ7Y8lEhtV9WEefHMtkwZ052ydPk9SiN6hKknt4bfXsasqxJ0XjNCJOCSlqNwlaW2rqOXRv63n8yf0Zdwx3YKOI3JUqdwlaf3n62uJRuG75w0POorIUadyl6RUsrWCF98v4xunDaC4uw4OJqlH5S5Jx925908r6dopk3+eMiToOCKBULlL0plTsoMFG/Zwy9ThOsyApCyVuySV+nCEH81exbCifK48sTjoOCKBUblLUnny7xvZtKeGuy4aSUa6/ntL6tL/fkka5ZV1/PKtUs4a0YszhvYMOo5IoFTukjTu+/NqQuEod130mXPJiKQclbskhSUf7+Gl97dw/RkDGViYF3QckcCp3CXhRaLO3TNL6F2Qw7e066MIoHKXJPDswk2UbN3P9y88jrzsdp9/RiQpqNwloe2srOenr63mlEE9uGhMn6DjiMQNlbsktB/PXkVtQ4Qf/sNoHfVRpBmVuySs99bt5qWlW5j+uUEM6ZUfdByRuKJyl4RUH45w18wP6d+tEzdOGRp0HJG4o1efJCH9Zt46Ssur+N01J9IpKz3oOCJxR1vuknA+2lHJQ3NLuWRsX6aM0KnzRA5G5S4JJRp1bn9xOXnZGXonqshhqNwlofz3/I95f9M+7rpwJIX52UHHEYlbKndJGJt213Dfn1dzxtBCLh3fL+g4InFN5S4JIRp1bnthGRlpxk8uG6N92kVaoHKXhPD0extZuGEPd100kr5dOwUdRyTuqdwl7q3fWcV9r61m8vCefHFi/6DjiCQElbvEtYZIlH957gNyMtO571Itx4i0lt7EJHHtF3/5iGVlFfz6q+Pp3SUn6DgiCUNb7hK3Fm/cw0NzS7l8Qn8uOF5HfBQ5Eip3iUsVNQ3cPOMD+nfL5V8vHhV0HJGEo2UZiTvuzndeWEZ5ZR0vfPNU8nUCDpEjpi13iTtPvbuR11fu4PZpIzihuGvQcUQSUqvK3cymmdkaMys1szsOcv8tZrbSzJab2V/M7NjYR5VUsGzzPn40ezVnj+jFdacPDDqOSMJqsdzNLB14CDgfGAlcaWYHHrFpKTDR3ccALwA/jXVQSX67q+q54fdL6Nk5m/u/eIJ2exRph9ZsuU8CSt19vbuHgBnAJc0HuPtcd69pujof0DtN5IiEI1FuenYpu6tDPHzVBLrlZQUdSSShtabc+wGbm10va7rtUK4D/tyeUJJ6fjpnDe+u281/fOF4RvfrEnQckYRn7n74AWaXA9Pc/fqm61cBJ7n7jQcZ+zXgRuBMd68/yP3TgekARUVFE2bMmNGm0FVVVeTnJ8c5M5NpLtC2+byzpYHHVoQ465gMvj4yfg7jm0zfG80lPrVlLlOmTFni7hNbHOjuh70ApwBzml2/E7jzIOPOAVYBvVp6TndnwoQJ3lZz585t82PjTTLNxf3I57Nww24f8r1X/SuPvuehcKRjQrVRMn1vNJf41Ja5AIu9FR3bmmWZRcBQMxtoZlnAFcCs5gPMbBzwMHCxu5e39jeQpLbNe2r4p/9eQnG3XH79lQlkpmvPXJFYafGnyd3DNC61zKFxy/x5dy8xs3vN7OKmYT8D8oE/mNkHZjbrEE8nAsCe6hBXP7GQSNR57OqJdMnNDDqSSFJp1Vv/3H02MPuA2+5u9vk5Mc4lSaw2FOG6pxaxZV8tz1x/EoN6Jsf6qUg80fu65ahqiES56dn3+WDzPn7z1QlMHNA96EgiSUmLnHLURKLOrc8v481V5dx7yWimje4ddCSRpKVyl6MiGnW+99IKZi3byu3TRnDVyTpChUhHUrlLh4tGnXtmlfDc4s3cdNYQbpg8OOhIIklPa+7SoaJR5/svr+DZhZuZ/rlB3HLusKAjiaQElbt0mEjUuePF5fxhSRnfmjKY26YO18HARI4Slbt0iLqGCDfPWMqckh3cfPZQvn3OUBW7yFGkcpeYq6xr4B+fXsz89Xu45/Mj+cZpOi67yNGmcpeY2lUb5fLfvMe6nVU8eMVYLhl7uAOIikhHUblLzCzdtJd736vD09J58huTOH1oYdCRRFKWyl1i4oUlZXz/jysoyIRnbziNIb10SAGRIKncpV3qwxHu/dNKnlmwiVMH9+DKY2tV7CJxQG9ikjb7eHc1X/rtezyzYBP/dOYgnr52Ep2ztEeMSDzQlru0yctLt/CDlz8kzeC3XxvPtNF9go4kIs2o3OWI7KkOcdfMD3l1+TZOHNCNn18xjn5dOwUdS0QOoHKXVnF3XvtwO3fN/JCK2gZumzqMb545mAydPUkkLqncpUWb99Rwz6wS3lpdzqi+Bfz++pMY0bsg6Fgichgqdzmk2lCER/66nt+8XUqaGT+48DiuOXWAttZFEoDKXT4jGnVmLtvCT19bw7aKOs4f3Zu7LhpJX62tiyQMlbt8yt15c1U5//n6GlZvr2R0vwJ+/uWxnDSoR9DRROQIqdyFaNSZU7Kdh+aV8uGW/QwszOOXV47jwuP7kJam/dZFEpHKPYXVhiK8tLSMx9/ZwPqd1QwszOOnl4/h0nH9tK4ukuBU7ilo465qnl24iecWb2ZfTQOj+xXwyyvHccHxfUjXlrpIUlC5p4iaUJg5Jdv5w+Iy3l23m/Q049zjirj29IGcOKCbTqQhkmRU7kmsPhzhnY928erybbxWsp2aUIT+3Tpx67nD+NKJxRQV5AQdUUQ6iMo9yeyva+DtNTt5Y+UO5q4pp7IuTJdOmXx+TF8um9Cficd204ukIilA5Z7gGiJRlpdV8G7pLt5eu5Olm/cRiTo98rI4f3Rvzj++D6cNLiQrQy+QiqQSlXuCqa4Ps7ysgiUf72HRxr0s3riH6lAEgDH9u3DDmYOZMqInY4u76cVRkRSmco9jVfVh1mzfT8nW/Xy4pYLlZRWs3VFJ1BvvH1aUz6Xj+3PK4B6cPKgH3fOygg0sInFD5R4wd2d3VT0bd1ezfmc1pTurWFdexertlZTtrf10XLfcTMb078rUUb0ZW9yF8cd0o2uuylxEDk7l3sGiUWd3dYgd++vYVlHH1n21bNlXS9neGjbtqWHdjhpq57z56fjMdGNgYR5ji7vy5YnFjOhTwKi+BfTpkqPdFUWk1VpV7mY2DXgQSAcec/f7Drg/G3gamADsBr7s7htjGzU+RKJOZV0DFbUN7KtpYG9NiH01DeyuDrGnup7dVSF2VdWzs7Ke8srGj+FP1lGaZGWk0b9bJ4q75dKrTwanjhnKwMI8BhbmcUz3XL07VETarcVyN7N04CHgXKAMWGRms9x9ZbNh1wF73X2ImV0B/AT4ckcEbom70xBxQpEooXDjpT4coT4cpb4hSl04Ql1DhNpQhLpwlNpQmJpQhJpQ423VoTA19RGqQmGq68NU1YWpqg9TWRdmf10DVfVh3A/+tdPTjO55WfTIy6Jn52yG9OpMr4JsehfkUFSQQ+8uOfTr2onC/KxPt8LnzZvH5DMGHcV/IRFJBa3Zcp8ElLr7egAzmwFcAjQv90uAf236/AXgV2Zm7oeqwbZ7btEmHvhbDVkL5xKORGmIOg2RKOGIN5Z5JNrm504zyM3KIC87nbysDPJzMsjPzqA4L5eCnEw652RQ0CmTLk2XbrmZdM3NpGtuY6EX5GRqH3IRiQutKfd+wOZm18uAkw41xt3DZlYB9AB2NR9kZtOB6QBFRUXMmzfviANvLQ/Tt1OU7Mx60tOMDIP0NEg3yEhLb7pAZpo1ffzkYmSmN37MSoesNMhKN7LT///HzDQOWNcON10OEAYqGy9NH/7PP9CRqKqqatO/Q7xKpvloLvFJc2mdo/qCqrs/AjwCMHHiRJ88efIRP8dkYNy8ebTlsfFoXhLNBZJrPppLfNJcWqc1r9xtAYqbXe/fdNtBx5hZBtCFxhdWRUQkAK0p90XAUDMbaGZZwBXArAPGzAKubvr8cuCtjlhvFxGR1mlxWaZpDf1GYA6Nu0I+4e4lZnYvsNjdZwGPA/9tZqXAHhp/AYiISEBatebu7rOB2Qfcdnezz+uAL8Y2moiItJXeLSMikoRU7iIiSUjlLiKShFTuIiJJyILaY9HMdgIft/HhhRzw7tcElkxzgeSaj+YSn1J9Lse6e8+WBgVW7u1hZovdfWLQOWIhmeYCyTUfzSU+aS6to2UZEZEkpHIXEUlCiVrujwQdIIaSaS6QXPPRXOKT5tIKCbnmLiIih5eoW+4iInIYCV3uZnaTma02sxIz+2nQedrLzG41MzezwqCztJWZ/azpe7LczP5oZl2DznSkzGyama0xs1IzuyPoPG1lZsVmNtfMVjb9jNwcdKb2MrN0M1tqZq8EnaW9zKyrmb3Q9POyysxOieXzJ2y5m9kUGk/vd4K7jwLuDzhSu5hZMTAV2BR0lnZ6Axjt7mOAtcCdAec5Is3OGXw+MBK40sxGBpuqzcLAre4+EjgZ+FYCz+UTNwOrgg4RIw8Cr7n7COAEYjyvhC134AbgPnevB3D38oDztNcDwHeBhH4RxN1fd/dPzk04n8aTuySST88Z7O4h4JNzBiccd9/m7u83fV4uZupKAAACN0lEQVRJY3n0CzZV25lZf+BC4LGgs7SXmXUBPkfj4dJx95C774vl10jkch8GnGFmC8zsbTM7MehAbWVmlwBb3H1Z0Fli7Frgz0GHOEIHO2dwwhbiJ8xsADAOWBBsknb5OY0bQNGgg8TAQGAn8LumZabHzCwvll/gqJ5D9UiZ2ZtA74Pc9X0as3en8c/NE4HnzWxQvJ4BqoW5fI/GJZmEcLi5uPvMpjHfp3FZ4JmjmU0+y8zygReBb7v7/qDztIWZXQSUu/sSM5scdJ4YyADGAze5+wIzexC4A7grll8gbrn7OYe6z8xuAF5qKvOFZhal8TgNO49WviNxqLmY2fE0/hZfZmbQuIzxvplNcvftRzFiqx3u+wJgZtcAFwFnx+sv28NozTmDE4aZZdJY7M+4+0tB52mH04CLzewCIAcoMLPfu/vXAs7VVmVAmbt/8pfUCzSWe8wk8rLMy8AUADMbBmSRgAcTcvcV7t7L3Qe4+wAav+nj47XYW2Jm02j80/lid68JOk8btOacwQnBGrcWHgdWuft/BZ2nPdz9Tnfv3/QzcgWN52lO1GKn6ed7s5kNb7rpbGBlLL9GXG+5t+AJ4Akz+xAIAVcn4FZiMvoVkA280fSXyHx3/2awkVrvUOcMDjhWW50GXAWsMLMPmm77XtNpMyV4NwHPNG1ErAe+Ecsn1ztURUSSUCIvy4iIyCGo3EVEkpDKXUQkCancRUSSkMpdRCQJqdxFRJKQyl1EJAmp3EVEktD/AkTul+B4zjIcAAAAAElFTkSuQmCC\n", 84 | "text/plain": [ 85 | "" 86 | ] 87 | }, 88 | "metadata": {}, 89 | "output_type": "display_data" 90 | } 91 | ], 92 | "source": [ 93 | "def sigmoid(x):\n", 94 | " a = []\n", 95 | " for item in x:\n", 96 | " a.append(1/(1+math.exp(-item)))\n", 97 | " return a\n", 98 | "\n", 99 | "y = sigmoid(x)\n", 100 | "\n", 101 | "plt.plot(x,y)\n", 102 | "plt.grid()\n", 103 | "plt.show()\n" 104 | ] 105 | }, 106 | { 107 | "cell_type": "markdown", 108 | "metadata": {}, 109 | "source": [ 110 | "Tahn" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": 5, 116 | "metadata": {}, 117 | "outputs": [ 118 | { 119 | "data": { 120 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmYlfV99/H3d1aWYVgcGBCQRVDEqOAgRkEriko28Wk00Vy1GONDm8Y0TfK00WataVpNetWmjW1NlWgTk4mxGqnBEJcZjRgMSJB9GVBg2GYYYGAYZj3f549zk5zB2TjnzNxn+byu63ju5fc75/vzMPOZezn3be6OiIjIKTlhFyAiIqlFwSAiIh0oGEREpAMFg4iIdKBgEBGRDhQMIiLSgYJBREQ6UDCIiEgHCgYREekgL+wC4lFSUuITJ06Mq++JEycYPHhwcgsKicaSujJpPBpLaopnLG+99dYhdx/ZU7u0DIaJEyeyevXquPpWVlZyzTXXJLegkGgsqSuTxqOxpKZ4xmJmu3rTTruSRESkAwWDiIh0oGAQEZEOFAwiItKBgkFERDpISjCY2RIzqzGzDV2sNzP7VzOrMrN1ZnZpzLpFZrY9eCxKRj0iIhK/ZG0xPA4s6Gb9B4CpwWMx8B8AZjYC+DpwOTAb+LqZDU9STSIiEoekfI/B3V8zs4ndNFkI/LdH7yO60syGmdkY4BrgRXc/DGBmLxINmJ8koy4RSX3uTnNbhObWCM1t7dHp4Lmt3Wltj9DSHp1ui0Sf2yNOuwfPESfiEAmWRTw67+5EIo7D7+e3v9PKtpwduIND8Oycfodjd/99mz8sC5557+2Qe7pDclw3UO7iRRddOZGzigrjecVe668vuI0F9sTMVwfLulr+Hma2mOjWBqWlpVRWVsZVSENDQ9x9U43GkroyaTy9HYu7c7wFDjdFqG9x6pudYy1OQwucaHUa25zGVqexDZranJNt0NLuNLfH+YszXlu39Oe7JcQ6WTa6ZS9nF+X06b+xtPnms7t/H/g+wKxZszzeby9m+zcfU1UmjQUyazynj+VoYwtbDhxn+8Hj7Kg9wbt1J9hd10j10SZa2iLv6V+Yl8PwQQUMG5RPcXE+YwbkMbgwj6LCXAYV5DGoIJcB+aceORTm5VKQl0NBrlGQl0NeTg75uTnk5xp5uTnk5Ri5sQ+LPpvx+3kMcs0wMwzICZa9seJ1rrrqKgyILor2I2b+lKALZjHLYtadzjpb2If68t9YfwXDXmB8zPy4YNleoruTYpdX9lNNItKDSMTZdaydx15/hzW7jrBu71H2HD75+/VFhXlMOGsQ08YMYf70UsYMHcCYoQMYVTyAkUWFnFVUwKCC1Pn7c2CeUVSYOvWkqv76P7QUuMfMyokeaK539/1mthz4h5gDzjcA9/VTTSLSieNNrVRureXlzQep3FbL0cZWYBNjhw1kxvhhfGL2BKafXcx5pUWMLh7Q738pS99LSjCY2U+I/uVfYmbVRM80ygdw9/8ElgEfBKqARuCTwbrDZvZNYFXwUvefOhAtIv0nEnFe3V7LM2v28quNB2huizB8UD7XThtFSdsh7vzgHM4eNjDsMqWfJOuspNt7WO/AZ7pYtwRYkow6ROTMnGhu42er9/D4G+/ybl0jwwblc+uscSycMZZLzxlObo5RWVmpUMgy2tkmkoWaWtv50cpdPFxRxZHGVmaeM4wv3nA+N144moI8XRAh2ykYRLLMLzcc4JvPb2Lv0ZPMnVLC568/j7IJ+l6p/IGCQSRLHDzWxNee28DyjQe5YEwx377lYuZMKQm7LElBCgaRLFCxpYbPP7WWky3tfGnBNO6+ahL5udplJJ1TMIhksLb2CA+9tI2HK3YwfUwx3/vETCaPLAq7LElxCgaRDNXU2s49P17DS5truO2y8XzjpgsZkJ8bdlmSBhQMIhmo/mQrdz+xitW7jvB3N13Ioisnhl2SpBEFg0iGOdrYwu3/9SZVNcf5t9tn8uGLzw67JEkzCgaRDNLY0sZdj69iR00Djy26jKvPGxl2SZKGdFqCSIZoaYvw6R+tYe2eo/zr7TMUChI3bTGIZAB35ys/X8+r22p58KMXseB9Y8IuSdKYthhEMkD5qj08tbqaz147hY9fdk7Y5UiaUzCIpLm39xzl689t5OrzRvJX888LuxzJAAoGkTRW39jKXzy5hpFDCvnux2eQm6N7I0jidIxBJI3d//wmDhxr4plPX8nwwQVhlyMZIilbDGa2wMy2mlmVmd3byfqHzGxt8NhmZkdj1rXHrFuajHpEssErWw7yP2uq+YtrzuWS8cPCLkcySMJbDGaWCzwMXA9UA6vMbKm7bzrVxt0/H9P+s8DMmJc46e4zEq1DJJvUN7Zy3zPrOb90CPdcOyXsciTDJGOLYTZQ5e473b0FKAcWdtP+duAnSXhfkaz1jy9s5lBDC/906yUU5un6R5JcyQiGscCemPnqYNl7mNkEYBLwSsziAWa22sxWmtnNSahHJKNt2FvPT1fv4ZNXTuSicUPDLkcykEVvx5zAC5jdAixw97uD+TuAy939nk7afgkY5+6fjVk21t33mtlkooFxnbvv6KTvYmAxQGlpaVl5eXlc9TY0NFBUlBmXHdZYUldfjcfdeeC3Tew7EeHBqwYxKL/vz0LKpM8m28cyb968t9x9Vo8N3T2hB3AFsDxm/j7gvi7a/g64spvXehy4paf3LCsr83hVVFTE3TfVaCypq6/G84t1+3zCl573H618t09evzOZ9Nlk+1iA1d6L3+vJ2JW0CphqZpPMrAC4DXjP2UVmNg0YDvwmZtlwMysMpkuAOcCm0/uKSPT+Cv+wbDPTRg/h47PGh12OZLCEz0py9zYzuwdYDuQCS9x9o5ndTzSdToXEbUB5kFqnXAA8YmYRosc7HvCYs5lE5A/Kf7ub6iMn+eGnZpOn23JKH0rKF9zcfRmw7LRlXztt/hud9HsDuCgZNYhksqbWdv7j1R3MnjSCq6bqqqnSt/Rnh0ga+NnqPRw81sznrpsadimSBRQMIimuua2df6/cwawJw7ny3LPCLkeygIJBJMX9bHU1++ub+Nz8qZjpInnS9xQMIimsPeI88toOZp4zjLlTSsIuR7KEgkEkhb20+SB7Dp9k8VWTtbUg/UbBIJLCfrDiHcYOG8j100vDLkWyiIJBJEVt3n+MlTsP86dXTND3FqRf6V+bSIr6wYp3GJify226h7P0MwWDSAqqa2jm52v38ceXjmXooPywy5Eso2AQSUFPv1VNS1uEO6+cGHYpkoUUDCIpxt15avUeZk0YztTSIWGXI1lIwSCSYtbsPsKO2hN8TFdQlZAoGERSzFOrqhlUkMuHLh4TdimSpRQMIinkRHMbz6/bx4cvHsPgwqRc/FjkjCkYRFLIL9bv50RLu3YjSagUDCIp5Ger9zB55GDKJgwPuxTJYkkJBjNbYGZbzazKzO7tZP2dZlZrZmuDx90x6xaZ2fbgsSgZ9Yikoz2HG1n17hE+euk4XRdJQpXwTkwzywUeBq4HqoFVZra0k1t0/tTd7zmt7wjg68AswIG3gr5HEq1LJN08v24/ADddcnbIlUi2S8YWw2ygyt13unsLUA4s7GXfG4EX3f1wEAYvAguSUJNI2ln69j5mnjOM8SMGhV2KZLlkBMNYYE/MfHWw7HQfNbN1Zva0mZ06stbbviIZrarmOJv3H+MjF2trQcLXX+fD/S/wE3dvNrM/A54Arj2TFzCzxcBigNLSUiorK+MqpKGhIe6+qUZjSV1nOp5nt7dgwPCGd6is3NVndcUjkz4bjaWX3D2hB3AFsDxm/j7gvm7a5wL1wfTtwCMx6x4Bbu/pPcvKyjxeFRUVcfdNNRpL6jqT8UQiEZ/3nQq/7ZHf9F1BCcikzybbxwKs9l78Xk/GrqRVwFQzm2RmBcBtwNLYBmYW+xXOm4DNwfRy4AYzG25mw4EbgmUiWWPjvmPsPHSCj+igs6SIhHcluXubmd1D9Bd6LrDE3Tea2f1E02kp8JdmdhPQBhwG7gz6HjazbxINF4D73f1wojWJpJNfrN9Pbo7xgfeNDrsUESBJxxjcfRmw7LRlX4uZvo/oLqbO+i4BliSjDpF0tHzjAd4/eQTDBxeEXYoIoG8+i4SqquY4O2tPcOOF2lqQ1KFgEAnR8o0HAbh+emnIlYj8gYJBJES/2niAS8YNZczQgWGXIvJ7CgaRkOyvP8nb1fXcoN1IkmIUDCIheXFTdDfSjRdqN5KkFgWDSEiWbzzA5JGDmTJK93WW1KJgEAlB/clW3tx5mBumazeSpB4Fg0gIfr29lraIc90Fo8IuReQ9FAwiIajYUsvQgfnMHD8s7FJE3kPBINLPIhHn1W01/NF5I8nL1Y+gpB79qxTpZ+v21nOooYVrp2k3kqQmBYNIP6vYUoMZ/NF5I8MuRaRTCgaRflaxtYaZ44fponmSshQMIv2o5ngT66rrtRtJUpqCQaQfvbq1FoB5CgZJYQoGkX5Uua2WUUMKmT6mOOxSRLqUlGAwswVmttXMqszs3k7Wf8HMNpnZOjN72cwmxKxrN7O1wWPp6X1FMkV7xHl9+yGuPm8kZhZ2OSJdSvgObmaWCzwMXA9UA6vMbKm7b4pp9jtglrs3mtmngW8DHw/WnXT3GYnWIZLq1u+tp/5kK1frbCRJccnYYpgNVLn7TndvAcqBhbEN3L3C3RuD2ZXAuCS8r0haeW1bLWYwd0pJ2KWIdMvcPbEXMLsFWODudwfzdwCXu/s9XbT/HnDA3f8+mG8D1gJtwAPu/vMu+i0GFgOUlpaWlZeXx1VvQ0MDRUVFcfVNNRpL6upsPN9aeZK2CHz9yvS6KU8mfTbZPpZ58+a95e6zemzo7gk9gFuAR2Pm7wC+10XbPyG6xVAYs2xs8DwZeBc4t6f3LCsr83hVVFTE3TfVaCyp6/Tx1J9s8cn3/cK//cvN4RSUgEz6bLJ9LMBq78Xv9WTsStoLjI+ZHxcs68DM5gNfBm5y9+aYYNobPO8EKoGZSahJJKW8UVVHe8S5eqqOL0jqS0YwrAKmmtkkMysAbgM6nF1kZjOBR4iGQk3M8uFmVhhMlwBzgNiD1iIZ4dfbaxlckMvMc4aHXYpIjxI+K8nd28zsHmA5kAsscfeNZnY/0c2WpcB3gCLgZ8Fpervd/SbgAuARM4sQDakHvOPZTCJpz915bXstV5xbQkGevjokqS/hYABw92XAstOWfS1men4X/d4ALkpGDSKpalddI3sOn+T/XjU57FJEekV/voj0sderDgE6TVXSh4JBpI+tqDrE2UMHMKlkcNiliPSKgkGkD7VHnN/srGPOlBJdBkPShoJBpA9t2neMo42tzJ2q3UiSPhQMIn3o1PGFK89VMEj6UDCI9KEVVYeYNnoII4cUhl2KSK8pGET6SFNrO6vePaytBUk7CgaRPrJm1xGa2yLMnXpW2KWInBEFg0gfeb3qEHk5xuxJCgZJLwoGkT6yYkcdM8YPo6gwKRcYEOk3CgaRPnCi1VlffZQr9W1nSUMKBpE+sO1IOxGHOedqN5KkHwWDSB/YVNfOgPwcZpwzLOxSRM6YgkGkD2yua+eyiSMozMsNuxSRM6ZgEEmy2uPNVDe4vr8gaSspwWBmC8xsq5lVmdm9nawvNLOfBuvfNLOJMevuC5ZvNbMbk1GPSJh+s7MOgCt1fEHSVMLBYGa5wMPAB4DpwO1mNv20Zp8Cjrj7FOAh4MGg73SitwK9EFgA/HvweiJp642qQwzMg/eNHRp2KSJxScYWw2ygyt13unsLUA4sPK3NQuCJYPpp4DqLXoN4IVDu7s3u/g5QFbyeSNp6Y0cd00bkkpujy2xLekpGMIwF9sTMVwfLOm3j7m1APXBWL/uKpI09hxvZfbiR6SO04SvpK22+kmlmi4HFAKWlpVRWVsb1Og0NDXH3TTUaS+p5rboVgAkDmzNiPJA5nw1oLL2VjGDYC4yPmR8XLOusTbWZ5QFDgbpe9gXA3b8PfB9g1qxZfs0118RVbGVlJfH2TTUaS+p5tvx3lBTVMXVUbkaMBzLnswGNpbeSsStpFTDVzCaZWQHRg8lLT2uzFFgUTN8CvOLuHiy/LThraRIwFfhtEmoS6Xfuzhs76rjy3LN0G09JawlvMbh7m5ndAywHcoEl7r7RzO4HVrv7UuAx4IdmVgUcJhoeBO2eAjYBbcBn3L090ZpEwlBV00Dt8eboaaqN9WGXIxK3pBxjcPdlwLLTln0tZroJuLWLvt8CvpWMOkTC9MaO6PcX5kwpYce6nSFXIxI/ffNZJElWVB1i3PCBjB8xKOxSRBKiYBBJgvaIs3JnHXN0GQzJAAoGkSTYuK+eY01tXDlFl8GQ9KdgEEmCU8cXrpisYJD0p2AQSYIVVYeYOqqIUcUDwi5FJGEKBpEENbe1s+rdw7qaqmQMBYNIgtbsOkpTa4S5U0eGXYpIUigYRBK0ouoQuTnG5ZNHhF2KSFIoGEQStGLHIS4eN5TiAflhlyKSFAoGkQQca2rl7T1HmTtF31+QzKFgEEnAyh11RDx6GQyRTKFgEEnAiqpDDMzPZeY5w8IuRSRpFAwiCVixo47Zk0ZQmKc7tknmUDCIxOlAfRNVNQ3M0WUwJMMoGETi9HrVIQCu1IXzJMMoGETi9OvttZQUFTB9THHYpYgkVULBYGYjzOxFM9sePA/vpM0MM/uNmW00s3Vm9vGYdY+b2TtmtjZ4zEikHpH+Eok4r28/xNwpJeTk6DaeklkS3WK4F3jZ3acCLwfzp2sE/tTdLwQWAP9iZrGncPy1u88IHmsTrEekX2zaf4y6Ey1cfZ4ugyGZJ9FgWAg8EUw/Adx8egN33+bu24PpfUANoJ8mSWuvba8FYO5UHV+QzJNoMJS6+/5g+gBQ2l1jM5sNFAA7YhZ/K9jF9JCZFSZYj0i/eG1bLReMKWbUEF1mWzKPuXv3DcxeAkZ3surLwBPuPiym7RF3f89xhmDdGKASWOTuK2OWHSAaFt8Hdrj7/V30XwwsBigtLS0rLy/vfmRdaGhooKioKK6+qUZjCUdTm/OZlxu5cWI+Hzu/oNM26TSenmgsqSmescybN+8td5/VY0N3j/sBbAXGBNNjgK1dtCsG1gC3dPNa1wDP9+Z9y8rKPF4VFRVx9001Gks4Xtp0wCd86Xl/fXttl23SaTw90VhSUzxjAVZ7L37HJroraSmwKJheBDx3egMzKwCeBf7b3Z8+bd2Y4NmIHp/YkGA9In3u19sPMSA/h1kTO904Fkl7iQbDA8D1ZrYdmB/MY2azzOzRoM3HgKuBOzs5LfVJM1sPrAdKgL9PsB6RPvfatlreP/ksXQZDMlZeIp3dvQ64rpPlq4G7g+kfAT/qov+1iby/SH/bVXeCnYdOcMcVE8IuRaTP6JvPImfglS01AFw7bVTIlYj0HQWDyBl4ZUsNk0cOZsJZg8MuRaTPKBhEeqmxpY03dx7m2vO1tSCZTcEg0ksrqupoaY8wT7uRJMMpGER66ZUtNRQV5nHZxBFhlyLSpxQMIr3g7lRurWHulBIK8vRjI5lN/8JFemHz/uPsr29i3jRd/1Eyn4JBpBcqtkZPU71GB54lCygYRHrhVxsPcMm4oZQW62qqkvkUDCI92F9/krer67nhws4uMiySeRQMIj14cdNBAG5UMEiWUDCI9GD5xgOcO3IwU0ZlxnX8RXqiYBDpxtHGFlbuPKzdSJJVFAwi3XhlSw3tEdduJMkqCgaRbizfeIDRxQO4eOzQsEsR6TcKBpEunGxp57Vth7h+eik5ORZ2OSL9JqFgMLMRZvaimW0Pnju916GZtcfcvW1pzPJJZvammVWZ2U+D24CKpISXtxzkZGs7H7p4TNiliPSrRLcY7gVedvepwMvBfGdOuvuM4HFTzPIHgYfcfQpwBPhUgvWIJM3StfsoLS7URfMk6yQaDAuBJ4LpJ4Cbe9vRzAy4Fng6nv4ifelYUyuVW2v50EVnk6vdSJJlzN3j72x21N2HBdMGHDk1f1q7NmAt0AY84O4/N7MSYGWwtYCZjQdecPf3dfFei4HFAKWlpWXl5eVx1dzQ0EBRUWacj66x9J1fV7fy2IYWvvr+AZw7LPeM+6faeBKhsaSmeMYyb968t9x9Vo8N3b3bB/ASsKGTx0Lg6Gltj3TxGmOD58nAu8C5QAlQFdNmPLChp3rcnbKyMo9XRUVF3H1TjcbSd+547E2f++DLHolE4uqfauNJhMaSmuIZC7Dae/E7Nq8XwTG/q3VmdtDMxrj7fjMbA9R08Rp7g+edZlYJzAT+BxhmZnnu3gaMA/b2VI9IX6traGZF1SH+7OrJRDeERbJLoscYlgKLgulFwHOnNzCz4WZWGEyXAHOATUF6VQC3dNdfpL8t23CA9ojzkUvODrsUkVAkGgwPANeb2XZgfjCPmc0ys0eDNhcAq83sbaJB8IC7bwrWfQn4gplVAWcBjyVYj0jCnllTzXmlRUwbPSTsUkRC0eOupO64ex1wXSfLVwN3B9NvABd10X8nMDuRGkSSafvB4/xu91G+8qELtBtJspa++SwS46nVe8jLMW6eOTbsUkRCo2AQCbS0RXhmzV7mX1BKSVFh2OWIhEbBIBJ4ZUsNdSda+Nhl48IuRSRUCgaRwM9W76G0uJCrp44MuxSRUCkYRIAD9U1UbK3ho5eOIy9XPxaS3fQTIAL8cOW7ANw++5xwCxFJAQoGyXpNre38+M3dzL+glPEjBoVdjkjoFAyS9Z5bu5cjja18cs6ksEsRSQkKBslq7s4PVrzLtNFDeP9k3XdBBBQMkuV+s7OOLQeOc9ecSfqms0hAwSBZbcnr7zB8UD43zdAF80ROUTBI1tqwt56XNtfwyTmTGJB/5jfjEclUCgbJWv/68naGDMjjzjkTwy5FJKUoGCQrbdp3jF9tOshdcyZRPCA/7HJEUoqCQbLSv72ynSGFedylU1RF3kPBIFln075jvLDhAJ+cM5Ghg7S1IHK6hILBzEaY2Ytmtj14Ht5Jm3lmtjbm0WRmNwfrHjezd2LWzUikHpGeuDvfWraJYYPy+dTcyWGXI5KSEt1iuBd42d2nAi8H8x24e4W7z3D3GcC1QCPwq5gmf31qvbuvTbAekW69uOkgK6rq+Pz887S1INKFRINhIfBEMP0EcHMP7W8BXnD3xgTfV+SMNbe1861lm5kyqohPXK6L5Yl0xdw9/s5mR919WDBtwJFT8120fwX4Z3d/Pph/HLgCaCbY4nD35i76LgYWA5SWlpaVl5fHVXNDQwNFRUVx9U01GsuZeeGdVn66tYUvlhVy0ciEbnfeI302qSnbxzJv3ry33H1Wjw3dvdsH8BKwoZPHQuDoaW2PdPM6Y4BaIP+0ZQYUEt3i+FpP9bg7ZWVlHq+Kioq4+6YajaX3dted8OlffcHvXPJmn77PKfpsUlO2jwVY7b34Hdvjn03uPr+rdWZ20MzGuPt+MxsD1HTzUh8DnnX31pjX3h9MNpvZD4D/11M9ImfK3bn3mXWYGd+8+X1hlyOS8hI9xrAUWBRMLwKe66bt7cBPYhcEYXJqN9TNRLdERJLqx7/dzYqqOv72gxcwbrjutyDSk0SD4QHgejPbDswP5jGzWWb26KlGZjYRGA+8elr/J81sPbAeKAH+PsF6RDrYc7iRf/jFZuZOKeH22ePDLkckLSR0BM7d64DrOlm+Grg7Zv5dYGwn7a5N5P1FutPU2s5nfryGHDMe+OhFuqy2SC/17akZIiH6u//dxLrqer5/R5l2IYmcAV0SQzLSU6v28JPf7uYz887lhgtHh12OSFpRMEjGeX37Ib7y8w3MnVLCF64/P+xyRNKOgkEyyto9R1n8w9VMHjmYhz9xKbk5Oq4gcqYUDJIxth88zid/8FvOKirgibtm61pIInFSMEhG+N3uI9z6yG/Iy83hh3ddTmnxgLBLEklbCgZJe69uq+UT//UmxQPyefrPr2BiyeCwSxJJazpdVdKWu/Por9/hwV9u4bzSITx+12WMGqItBZFEKRgkLdWfbOVvnn6b5RsPcuOFpXzn1kt072aRJFEwSNp5cdNBvvrzDRxqaOYrH7qAT82dpG81iySRgkHSxp7DjfzjC5tZtv4A00YP4T/vKGPG+C5v/yEicVIwSMqrOd7Ev1fs4Mk3d5Fjxl/feD6Lr55Mfq7OnRDpCwoGSVmb9x9jyevv8NzafbS787FZ4/ncdVMZPVQHmEX6koJBUsqhhmaef3sfz/5uL29X1zMwP5ePXzaeT82dpNNQRfqJgkFC1dYeYdP+Yzy/s4V/2/wGa3YfwR2mjynmKx+6gFvKxjFsUEHYZYpklYSCwcxuBb4BXADMDu7D0Fm7BcB3gVzgUXc/dUOfSUA5cBbwFnCHu7ckUpOkLndn79GTbN5/nPXVR3m7up41u45wvLkNgPeNbecvr53KBy4azbTRxSFXK5K9Et1i2AD8MfBIVw3MLBd4GLgeqAZWmdlSd98EPAg85O7lZvafwKeA/0iwJgmJu3OsqY3a403sO9rE/vqTVB85ya66RnbVnaCqpoETLe0A5BhMHTWEj8w4m8snjcAPbOXmBVeFPAIRgcTv4LYZ6Okc8tlAlbvvDNqWAwvNbDNwLfCJoN0TRLc+FAx9JBJx2iJOe8Rpi0Roa3daTz23R2htj9DcFqGlLfrc1NpOU2v0ubGlncaWNhqa22hsaed4UyvHTrZRf7KVI40tHDnRwqETLbS0RTq8Z47B2OEDmTBiMLfOGs/U0iLOLx3C9LOLGVTwh39+lZXb+/t/h4h0oT+OMYwF9sTMVwOXE919dNTd22KWv+f2n8n05WfXU7GxkcFrXsUTeB337nt7lzMdF8W+jgPu4DEd3KOPUyLuv2/jDs0tLeS//iLuwTqiv/zdod2jARA7nSwD8nMoHpBP8cB8hgzIY3TxAKaNLuasogJGDSlk5JBCRhcP4OxhAyktHkBBnk4rFUknPQaDmb0EdHYLrC+7+3PJL6nLOhYDiwFKS0uprKw849doPtxCaWGEXDsZvGYC9STYtrP3tuA/FtMjttmpWwuc6tvWGqGgIIIFyyxon2PRrbgcyyEnWJcT88g1yDEjzyA3JzqflwO5OUZ+DuTnQF6OUZCXLMM7AAAEvUlEQVQD+blQmGsU5kJBrjEgl5h7HDjQGjxORBe1A0fh5FHYsRt29PL/UUNDQ1yfaarKpPFoLKmpT8fi7gk/gEpgVhfrrgCWx8zfFzwMOATkddauu0dZWZnHq6KiIu6+qUZjSV2ZNB6NJTXFMxZgtffid2x/bOOvAqaa2SQzKwBuA5YGRVYAtwTtFgH9tgUiIiKdSygYzOz/mFk10b/2f2Fmy4PlZ5vZMgCPHkO4B1gObAaecveNwUt8CfiCmVURPebwWCL1iIhI4hI9K+lZ4NlOlu8DPhgzvwxY1km7nUTPWhIRkRSh00VERKQDBYOIiHSgYBARkQ4UDCIi0oGCQUREOjDv4fIOqcjMaoFdcXYvIfrFukygsaSuTBqPxpKa4hnLBHcf2VOjtAyGRJjZanefFXYdyaCxpK5MGo/Gkpr6cizalSQiIh0oGEREpINsDIbvh11AEmksqSuTxqOxpKY+G0vWHWMQEZHuZeMWg4iIdCNrg8HMPmtmW8xso5l9O+x6EmVmXzQzN7OSsGuJl5l9J/hM1pnZs2Y2LOyazpSZLTCzrWZWZWb3hl1PvMxsvJlVmNmm4Gfkc2HXlCgzyzWz35nZ82HXkigzG2ZmTwc/L5vN7Ipkvn5WBoOZzQMWApe4+4XAP4VcUkLMbDxwA7A77FoS9CLwPne/GNhG9IZOacPMcoGHgQ8A04HbzWx6uFXFrQ34ortPB94PfCaNx3LK54he+j8TfBf4pbtPAy4hyePKymAAPg084O7NAO5eE3I9iXoI+Bs6vcN0+nD3X/kf7gG+EhgXZj1xmA1UuftOd28Byon+AZJ23H2/u68Jpo8T/cXTp/dk70tmNg74EPBo2LUkysyGAlcT3L/G3Vvc/Wgy3yNbg+E84Coze9PMXjWzy8IuKF5mthDY6+5vh11Lkt0FvBB2EWdoLLAnZr6aNP5leoqZTQRmAm+GW0lC/oXoH0+RsAtJgklALfCDYNfYo2Y2OJlvkNCNelKZmb0EjO5k1ZeJjnsE0U3ky4CnzGyyp+gpWj2M5W+J7kZKC92Nxd2fC9p8meiujCf7szZ5LzMrAv4H+Ct3PxZ2PfEwsw8DNe7+lpldE3Y9SZAHXAp81t3fNLPvAvcCX03mG2Qkd5/f1Toz+zTwTBAEvzWzCNHrjtT2V31noquxmNlFRP96eNvMILrrZY2ZzXb3A/1YYq9197kAmNmdwIeB61I1qLuxFxgfMz8uWJaWzCyfaCg86e7PhF1PAuYAN5nZB4EBQLGZ/cjd/yTkuuJVDVS7+6ktuKeJBkPSZOuupJ8D8wDM7DyggDS8sJa7r3f3Ue4+0d0nEv0Hc2mqhkJPzGwB0c39m9y9Mex64rAKmGpmk8ysALgNWBpyTXGx6F8ajwGb3f2fw64nEe5+n7uPC35GbgNeSeNQIPj53mNm5weLrgM2JfM9MnaLoQdLgCVmtgFoARal4V+nmeh7QCHwYrAFtNLd/zzcknrP3dvM7B5gOZALLHH3jSGXFa85wB3AejNbGyz72+D+7RK+zwJPBn+A7AQ+mcwX1zefRUSkg2zdlSQiIl1QMIiISAcKBhER6UDBICIiHSgYRESkAwWDiIh0oGAQEZEOFAwiItLB/wfxnTAjadG1PwAAAABJRU5ErkJggg==\n", 121 | "text/plain": [ 122 | "" 123 | ] 124 | }, 125 | "metadata": {}, 126 | "output_type": "display_data" 127 | } 128 | ], 129 | "source": [ 130 | "def tanh(x, derivative=False):\n", 131 | " if (derivative == True):\n", 132 | " return (1 - (x ** 2))\n", 133 | " return np.tanh(x)\n", 134 | "\n", 135 | "\n", 136 | "y = tanh(x)\n", 137 | "\n", 138 | "plt.plot(x,y)\n", 139 | "plt.grid()\n", 140 | "plt.show()" 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": {}, 146 | "source": [ 147 | "ReLU" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 6, 153 | "metadata": {}, 154 | "outputs": [ 155 | { 156 | "data": { 157 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHK9JREFUeJzt3Xl4lOW9xvHvQwgBEiRgILJJQHaRJUEEXA7Rat1t6wZIW2tPaUEU64prazfbWqscUbupbQ9LWMRdcQ3uG0nY932HsCdAyPY7fxB6eVogk0kmzzsz9+e65pLAO8z9c8g9b555k8eZGSIiEj0a+A4gIiI1o+IWEYkyKm4RkSij4hYRiTIqbhGRKKPiFhGJMipuEZEoo+IWEYkyKm4RkSjTMBJ/aVpammVkZIR13wMHDpCcnFy3gTzRLMGkWYIrluap6Sx5eXk7zaxVKMdGpLgzMjKYO3duWPedM2cOQ4cOrdtAnmiWYNIswRVL89R0Fufc+lCP1VKJiEiUCam4nXOpzrmZzrllzrmlzrnBkQ4mIiLHFupSyQRgtpld45xrBDSNYCYRETmBaovbOdccOA+4EcDMSoHSyMYSEZHjCWWppBNQCDzvnCtwzv3NORcbb/uKiEQhV91GCs65AcDnwNlm9oVzbgKw38we/LfjRgGjANLT07NycnLCClRcXExKSkpY9w0azRJMmiW4Ymmems6SnZ2dZ2YDQjrYzE54A04B1n3t43OB1090n6ysLAtXbm5u2PcNGs0STJoluGJpnprOAsy1avr46K3apRIz2wZsdM51r/qtC4AlIb+MiIjEgS/W7OLZj9cePcGNqFCvKrkFmFx1Rcka4AeRiyQiEl0Kiw5zy9QCkpMaMnxgB5o2isj3Nv5LSH+7mc0DQlt7ERGJIxWVxq1TC9h3qIx/3DQw4qUNEfqWdxGRePH4Oyv4bM0uHr2mDz3bnFQvj6lveRcRCVPu8h1MzF3FdQPac+2ADvX2uCpuEZEwbN57iJ9Om0ePU5rxi6t61+tjq7hFRGqotLySmyfnU15hPDMyi8aJCfX6+FrjFhGpod+8sZR5G/fyzA2ZdEqr/28k1xm3iEgNvL5gK3//dB03nd2JS85o4yWDiltEJERrCou554UF9D81lfGX9PCWQ8UtIhKCQ6UVjJmcT2KC46kRmTRq6K8+tcYtIhKCB19exPLtRTx/45m0TW3iNYvOuEVEqjH9q43MzNvE2OwuDO3e2nccFbeIyIks2bKfB19exNldTua2b3TzHQdQcYuIHNf+kjLGTM4jtWkiE4b1J6GB8x0J0Bq3iMgxmRl3z1jAxj2HyBk1iLSUJN+R/kVn3CIix/DcJ+uYvXgb91zcnTMzWvqO8/+ouEVE/k3e+j088sZSLuqVzo/O7ew7zn9QcYuIfM3uA6WMnZJPm9TGPHptX5wLxrr212mNW0SkSkWlMS6ngF0HSpk1egjNmyT6jnRMOuMWEaky8f1VfLRyJz+/4nR6t2vuO85xqbhFRICPV+7kifdW8O3+7Rg+sP42RQiHiltE4t62fSWMyymgS6sUfv3t3oFc1/46FbeIxLWyikrGTsnnUFkFz4zMrJfNfmsr+AlFRCLo0beWM3f9Hv5neH+6tG7mO05IdMYtInHrrcXb+MuHa/juoI5c2bet7zghU3GLSFxav+sAd86YT5/2zXng8p6+49SIiltE4k5J2ZFNERq4I5siJDWs381+ayukNW7n3DqgCKgAys1sQCRDiYhE0sOvLmHxlv08+/0BdGjZ1HecGqvJm5PZZrYzYklEROrBiwWbmPrlBkYPPY0Leqb7jhMWLZWISNxYsb2I+2Yt4qxOLbnjwmBsihAOZ2bVH+TcWmAPYMCfzewvxzhmFDAKID09PSsnJyesQMXFxaSkpIR136DRLMGkWYIrkvOUlBsPf3aIA2XwiyGNSW0c2fPWms6SnZ2dF/IytJlVewPaVf23NTAfOO9Ex2dlZVm4cnNzw75v0GiWYNIswRWpeSorK23slHzrNP41+2RVYUQe49/VdBZgroXQx2YW2lKJmW2u+u8O4EVgYMgvIyIink36fD2vzt/CHRd1Z8hpab7j1Fq1xe2cS3bONTv6a+AiYFGkg4mI1IX5G/fyy9eWkt29FaP/6zTfcepEKFeVpAMvVv3QlYbAFDObHdFUIiJ1YO/BUsZMzqdVsyT+eF0/GgRks9/aqra4zWwN0LcesoiI1JnKSuOO6fPZUVTCjJ8MoUVyI9+R6owuBxSRmPSnD1fz3rIdPHBZL/p1SPUdp06puEUk5ny+Zhd/eGs5l/Vpw/cGd/Qdp86puEUkpuwoKuGWqQVkpCXzu6v7BH5ThHDo53GLSMwor6jk1qkFFJWUMemHZ5GSFJsVF5tTiUhcevzdFXy+ZjePXduX7qdEx6YI4dBSiYjEhPeXbeep3NUMO7MDV2e19x0nolTcIhL1Nu05yE+nzadXm5P4+ZWn+44TcSpuEYlqh8sruHlKAZWVxtM3ZNI4Mbo2RQiH1rhFJKr95vWlzN+4lz+NzCQjLdl3nHqhM24RiVqvzt/CPz5bz3+f04mLe7fxHafeqLhFJCqtLixm/AsLyOrYgnsu6eE7Tr1ScYtI1DlYWs7oSXkkJSYwcUR/EhPiq8q0xi0iUcXMeOClRazcUcw/bxpIm+ZNfEeqd/H1MiUiUW/aVxuZlb+ZcRd05dyurXzH8ULFLSJRY9HmfTz0ymLO7ZrGLed39R3HGxW3iESFfYfKGDM5n5ZNG/HE9f1IiJFNEcKhNW4RCTwz464Z89my9xDTfjyIk1OSfEfySmfcIhJ4z368lreXbGf8JT3I6tjSdxzvVNwiEmhz1+3mkTeXcVGvdH54TiffcQJBxS0igbWr+DBjpxTQvkUTHr22b0xuihAOrXGLSCBVVBq3TZvH7oOlvDhmCM2bJPqOFBg64xaRQHry/ZV8tHInD195Oqe3be47TqCouEUkcD5cUciE91byncx2DDuzg+84gaPiFpFA2brvELdNm0e31s341bd6a137GEIubudcgnOuwDn3WiQDiUj8Kq80bp6cz+GyCp4emUnTRnob7lhqcsY9DlgaqSAiIjOWl5K/YS+/vboPp7VK8R0nsEIqbudce+Ay4G+RjSMi8Wr2oq28tb6cG4dkcEXftr7jBFqoZ9xPAHcDlRHMIiJxat3OA9w1YwGdmzfgvkt7+o4TeM7MTnyAc5cDl5rZGOfcUOBOM7v8GMeNAkYBpKenZ+Xk5IQVqLi4mJSU2PgSSbMEk2YJltIK45efl7C7pJJ7+hqnpkX3PEfV9LnJzs7OM7MBIR1sZie8AY8Am4B1wDbgIDDpRPfJysqycOXm5oZ936DRLMGkWYLlnpnzreM9r9n7S7fHxDxH1XQWYK5V08dHb9UulZjZvWbW3swygGHA+2Y2MuSXERGR45iZt4mcrzZyc/ZpZPdo7TtO1NB13CLixbJt+3ngpYUM7nwyP/1GN99xokqNLpI0sznAnIgkEZG4UVRSxphJ+TRrnMiE4f1oGGeb/daW/m+JSL0yM8bPWsi6XQd4cnh/Wjdr7DtS1FFxi0i9+udn63l9wVbu/GZ3BnU+2XecqKTiFpF6U7BhD796fQkX9GjNT847zXecqKXiFpF6sedAKWOnFNC6WWMeu64vDeJ4s9/a0k9wEZGIq6w0bp8+j8Kiw8wcPZjUpo18R4pqOuMWkYh75oPV5C4v5MHLe9KnfarvOFFPxS0iEfXp6p089vZyruzblpGDOvqOExNU3CISMTv2l3Dr1Hl0Skvmke+coU0R6ojWuEUkIsorKhk7tYADh8uZ8qOzSE5S3dQV/Z8UkYj4w9sr+HLtbh6/vi/d0pv5jhNTtFQiInXu3SXb+dMHqxlx1ql8u39733FijopbROrUxt0HuX36PHq3O4mHLu/lO05MUnGLSJ05XF7BmMn5GPDUiEwaJyb4jhSTtMYtInXml68tYeHmffzlu1l0PDnZd5yYpTNuEakTL8/bzKTPNzDqvM5cdPopvuPENBW3iNTaqh1F3DtrIWdmtOCub3b3HSfmqbhFpFYOlpYzelI+TRITeHJ4JonaFCHitMYtImEzM+5/cRGrCov535vO4pTm2hShPuilUUTCNvXLjbxYsJnbLujGOV3TfMeJGypuEQnLos37+PkrizmvWytuOb+L7zhxRcUtIjW272AZoyfncXJKI564vp82RahnWuMWkRoxM+6cOZ+te0uY9uPBtEzWpgj1TWfcIlIjf/1oDe8s2c69l/Ykq2ML33HikopbREL25drd/G72ci7pfQo3nZ3hO07cUnGLSEgKiw4zdko+p7Zsyu+v6aNNETyqtridc42dc1865+Y75xY75x6uj2AiEhwVlca4nAL2HSrj6RsyadY40XekuBbKm5OHgfPNrNg5lwh87Jx708w+j3A2EQmIJ95dwaerd/H7a/rQs81JvuPEvWqL28wMKK76MLHqZpEMJSLBMWf5Dp58fxXXZrXnugEdfMcRwB3p5WoOci4ByAO6AE+Z2T3HOGYUMAogPT09KycnJ6xAxcXFpKSkhHXfoNEswaRZQrfrUCU/+/QQqUmOBwc3ISkhsuva8fzcZGdn55nZgJAONrOQb0AqkAv0PtFxWVlZFq7c3Nyw7xs0miWYNEtoDpdV2FUTP7bTH5ptq3cURexxvi6enxtgroXYxTW6qsTM9lYV98U1uZ+IRJ9H3lzKvI17+d3VfejcKjbOgmNFKFeVtHLOpVb9uglwIbAs0sFExJ83F27l+U/WceOQDC7r08Z3HPk3oVxV0gb4R9U6dwNgupm9FtlYIuLL2p0HuGvmAvp1SOW+S3v6jiPHEMpVJQuA/vWQRUQ8KymrYPSkPBomOJ66IZNGDfU9ekGkHzIlIv/y0MuLWL69iOdvPJN2qU18x5Hj0MupiAAwfe5Gps/dxNjsLgzt3tp3HDkBFbeIsGTLfh58aRFDTjuZ277RzXccqYaKWyTOFZWUcfOUfJo3SWTCsP4kaFOEwNMat0gcMzPueWEBG3YfZOqPBtGqWZLvSBICnXGLxLHnP1nHGwu3cfc3uzOwU0vfcSREKm6ROJW/YQ+/eWMpF/ZKZ9R5nX3HkRpQcYvEod0HShk7OZ82qY35w7V9tSlClNEat0icqaw0bps2j53FpbwwegjNm2hThGijM26ROPNU7io+XFHIQ1f04oz2zX3HkTCouEXiyKerdvL4uyv4Vr+23HDWqb7jSJhU3CJxYtu+Em7NKaBzqxR+/e0ztK4dxbTGLRIHyioquWVqPgdLK8gZlUlykj71o5mePZE48Ie3lvPVuj1MGNaPLq2b+Y4jtaSlEpEY9/bibfz5wzWMHHQqV/Vr5zuO1AEVt0gM27DrIHfMmM8Z7Zrz4OW9fMeROqLiFolRJWUVjJmShwOeviGTpIYJviNJHdEat0iM+sVrS1i0eT9//d4AOrRs6juO1CGdcYvEoBcLNjHliw38+L86c2GvdN9xpI6puEVizIrtRdw3axEDO7Xkrou6+44jEaDiFokhBw6XM2ZyPslJDZk4vD8NE/QpHov0rIrECDPj3lkLWVNYzP8M60frkxr7jiQRouIWiRGTvtjAK/O3cPuF3RjSJc13HIkgFbdIDFiwaS+/fHUJQ7u3YszQLr7jSIRVW9zOuQ7OuVzn3BLn3GLn3Lj6CCYiodl3sIwxk/NJS2nE49f1o4E2+415oVzHXQ7cYWb5zrlmQJ5z7h0zWxLhbCJSjUoz7pgxj+37S5j+48G0SG7kO5LUg2rPuM1sq5nlV/26CFgK6AceiATA7LVlvLt0B/df2pP+p7bwHUfqSY3WuJ1zGUB/4ItIhBGR0H2xZhczV5ZxWZ82fH9Ihu84Uo+cmYV2oHMpwAfAr81s1jH+fBQwCiA9PT0rJycnrEDFxcWkpKSEdd+g0SzBFAuz7D1cyc8+LSGpQSUPn51Mk4axsa4dC8/NUTWdJTs7O8/MBoR0sJlVewMSgbeA20M5Pisry8KVm5sb9n2DRrMEU7TPUl5RacP+/Jl1f+AN++cr7/mOU6ei/bn5uprOAsy1EPrVzEK6qsQBzwJLzeyPIb98iEhEPP7OCj5bs4tffesMOjTTFb3xKJRn/Wzgu8D5zrl5VbdLI5xLRI4hd9kOJuau4voBHbgmq73vOOJJtZcDmtnHQGwsoIlEsU17DvLT6fPo2eYkHr7qdN9xxCN9nSUSBUrLK7l5SgEVFcYzN2TSOFGbIsQzbaQgEgV+88ZS5m/cy59GZpKRluw7jnimM26RgHttwRb+/uk6fnhOJy7u3cZ3HAkAFbdIgK0uLOaemQvIPDWV8Zf08B1HAkLFLRJQh0orGDMpn6TEBCaOyCRRmyJIFa1xiwSQmfHAS4tYsaOIf/xgIG1Tm/iOJAGil3CRAJo+dyMv5G/i1vO7cl63Vr7jSMCouEUCZvGWfTz08mLO6ZLGrRd09R1HAkjFLRIg+0uObIqQ2jSRJ4b1I0GbIsgxaI1bJCDMjLtnLGDTnkNMGzWItJQk35EkoHTGLRIQz368ltmLtzH+4h4MyGjpO44EmIpbJADy1u/mt28u46Je6fz3uZ18x5GAU3GLeLar+DA3Ty6gbWoTHr22L0d+krLI8WmNW8Sjikrjtmnz2H2wlFmjh9C8SaLvSBIFdMYt4tGT76/ko5U7+cWVp9O7XXPfcSRKqLhFPPloZSET3lvJdzLbcf2ZHXzHkSii4hbxYOu+Q4zLmUfX1in86lu9ta4tNaLiFqlnZRWVjJ1SwOGyCp4ZmUXTRnqrSWpG/2JE6tnvZy8jb/0enhzen9NapfiOI1FIZ9wi9Wj2om389aO1fG9wR67o29Z3HIlSKm6RerJ+1wHumjGfvu2bc/9lPX3HkSim4hapByVlFYyelE+DBo6JIzJJaqjNfiV8WuMWqQcPv7qYJVv389yNA+jQsqnvOBLldMYtEmEv5G1i6pcbGTP0NM7vke47jsQAFbdIBC3fVsT9Ly3krE4tuf3Cbr7jSIyotridc88553Y45xbVRyCRWFF8uJzRk/NISUrkyRH9aajNfqWOhPIv6e/AxRHOIRJTzIx7XljAup0HmDiiP62bNfYdSWJItcVtZh8Cu+shi0jM+Odn63l9wVbu/GZ3BnU+2XcciTHOzKo/yLkM4DUz632CY0YBowDS09OzcnJywgpUXFxMSkpsfDeZZgmmSM+yZm8Fv/6ihN5pCYzLTKJBBH8OSSw9LxBb89R0luzs7DwzGxDSwWZW7Q3IABaFcqyZkZWVZeHKzc0N+75Bo1mCKZKz7C4+bEMeec+GPPKe7TlwOGKPc1QsPS9msTVPTWcB5lqIHavruEXqSGWlcfv0eRQWHWbm6MGkNm3kO5LEKL3NLVJHnvlgNbnLC3nw8p70aZ/qO47EsFAuB5wKfAZ0d85tcs79MPKxRKLLp6t38tjby7mib1tGDuroO47EuGqXSsxseH0EEYlWO/aXcOvUeXRKS+aR75yhTREk4rTGLVIL5RWVjJ1awIHD5Uz50VmkJOlTSiJP/8pEauGxd1bw5drdPHZtX7qlN/MdR+KE3pwUCdN7S7fzzJzVDB/Ygauz2vuOI3FExS0Sho27D3L79Pn0anMSP7vidN9xJM6ouEVq6HB5BTdPyafSjGdGZtI4UZsiSP3SGrdIDf369aUs2LSPP43MouPJyb7jSBzSGbdIDbwyfwv//Gw9Pzq3Exf3PsV3HIlTKm6REK3aUcT4FxYwoGML7r64h+84EsdU3CIhOFhazuhJ+TRJTGDiiEwStSmCeKQ1bpFqmBn3v7iIVYXF/O9NZ3FKc22KIH7ptEGkGlO/3MiLBZu57YJunNM1zXccERW3yIks2ryPn7+6mHO7pnHL+V18xxEBVNwix7XvUBmjJ+dxcnIjJgzrT4MG+uFREgxa4xY5BjPjzhnz2bq3hGk/HkzLZG2KIMGhM26RY/jbR2t5Z8l2xl/Sg6yOLXzHEfl/VNwi/+ardbv57exlXNL7FH54TiffcUT+g4pb5Gt2Fh9m7JR8OrRowu+u6aNNESSQVNwiVSoqjXE5Bew9WMbTN2RxUuNE35FEjklvTopUmfDeSj5ZtYvfXX0Gvdqe5DuOyHHpjFsE+GBFIU++v5Jrstpz3YAOvuOInJCKW+Lelr2HuC2ngO7pzfjlVb21ri2Bp+KWuFZaXsnNU/IpqzCeviGTJo20KYIEn9a4Ja799s1lFGzYy1MjMuncKsV3HJGQ6Ixb4tYbC7fy3CdruXFIBpf1aeM7jkjIQipu59zFzrnlzrlVzrnxkQ4lEmlrdx7g7pkL6Nchlfsu7ek7jkiNVFvczrkE4CngEqAXMNw51yvSwUQipbTCGD0pj4YJjokj+tOoob7wlOgSyr/YgcAqM1tjZqVADnBVZGOJRM6kpaUs21bE49f3o32Lpr7jiNRYKG9OtgM2fu3jTcBZkQhzxZMfs2vfQZLzP4jEX1/vDhzULEFTYcaawnLGZnchu3tr33FEwlJnV5U450YBowDS09OZM2dOjf+OlMoSGiZVkuAO1VUsr1I0S/A46NLeyGy0hTlztvpOU2vFxcVhfa4FVSzNE9FZzOyEN2Aw8NbXPr4XuPdE98nKyrJw5ebmhn3foNEswaRZgiuW5qnpLMBcq6aPj95CWeP+CujqnOvknGsEDANeiczLiIiIVKfapRIzK3fOjQXeAhKA58xsccSTiYjIMYW0xm1mbwBvRDiLiIiEQBewiohEGRW3iEiUUXGLiEQZFbeISJRRcYuIRBl35LrvOv5LnSsE1od59zRgZx3G8UmzBJNmCa5Ymqems3Q0s1ahHBiR4q4N59xcMxvgO0dd0CzBpFmCK5bmieQsWioREYkyKm4RkSgTxOL+i+8AdUizBJNmCa5YmidiswRujVtERE4siGfcIiJyAoEtbufcLc65Zc65xc653/vOU1vOuTucc+acS/OdJVzOuUernpMFzrkXnXOpvjPVVKxsfO2c6+Ccy3XOLan6HBnnO1NtOecSnHMFzrnXfGepDedcqnNuZtXnylLn3OC6foxAFrdzLpsj+1r2NbPTgT94jlQrzrkOwEXABt9ZaukdoLeZ9QFWcGRTjagRYxtflwN3mFkvYBBwcxTPctQ4YKnvEHVgAjDbzHoAfYnATIEsbmA08FszOwxgZjs856mtx4G7gah+Q8HM3jaz8qoPPwfa+8wThpjZ+NrMtppZftWvizhSDu38pgqfc649cBnwN99ZasM51xw4D3gWwMxKzWxvXT9OUIu7G3Cuc+4L59wHzrkzfQcKl3PuKmCzmc33naWO3QS86TtEDR1r4+uoLbujnHMZQH/gC79JauUJjpzcVPoOUkudgELg+apln78555Lr+kHqbLPgmnLOvQuccow/up8juVpy5EvAM4HpzrnOFtBLYKqZ5T6OLJNEhRPNYmYvVx1zP0e+VJ9cn9nkPznnUoAXgNvMbL/vPOFwzl0O7DCzPOfcUN95aqkhkAncYmZfOOcmAOOBB+v6Qbwws28c78+cc6OBWVVF/aVzrpIj3/dfWF/5auJ4szjnzuDIK/B85xwcWVrId84NNLNt9RgxZCd6XgCcczcClwMXBPWF9AQ2Ax2+9nH7qt+LSs65RI6U9mQzm+U7Ty2cDVzpnLsUaAyc5JybZGYjPecKxyZgk5kd/epnJkeKu04FdankJSAbwDnXDWhEFP7gGTNbaGatzSzDzDI48qRmBrW0q+Ocu5gjX85eaWYHfecJQ8xsfO2OnAk8Cyw1sz/6zlMbZnavmbWv+hwZBrwfpaVN1ef2Rudc96rfugBYUteP4+2MuxrPAc855xYBpcD3o/DsLhZNBJKAd6q+gvjczH7iN1LoYmzj67OB7wILnXPzqn7vvqr9YcWvW4DJVScHa4Af1PUD6DsnRUSiTFCXSkRE5DhU3CIiUUbFLSISZVTcIiJRRsUtIhJlVNwiIlFGxS0iEmVU3CIiUeb/AHQ91pRElghyAAAAAElFTkSuQmCC\n", 158 | "text/plain": [ 159 | "" 160 | ] 161 | }, 162 | "metadata": {}, 163 | "output_type": "display_data" 164 | } 165 | ], 166 | "source": [ 167 | "def relu(x):\n", 168 | " a = []\n", 169 | " for item in x:\n", 170 | " if item > 0:\n", 171 | " a.append(item)\n", 172 | " else:\n", 173 | " a.append(0)\n", 174 | " return a\n", 175 | "\n", 176 | "\n", 177 | "y = relu(x)\n", 178 | "\n", 179 | "plt.plot(x,y)\n", 180 | "plt.grid()\n", 181 | "plt.show()" 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "### 3.4 Parametrización de los modelos" 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": 7, 194 | "metadata": {}, 195 | "outputs": [ 196 | { 197 | "name": "stderr", 198 | "output_type": "stream", 199 | "text": [ 200 | "/usr/local/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", 201 | " from ._conv import register_converters as _register_converters\n", 202 | "Using TensorFlow backend.\n" 203 | ] 204 | }, 205 | { 206 | "data": { 207 | "text/plain": [ 208 | "'2.1.3'" 209 | ] 210 | }, 211 | "execution_count": 7, 212 | "metadata": {}, 213 | "output_type": "execute_result" 214 | } 215 | ], 216 | "source": [ 217 | "import keras\n", 218 | "keras.__version__" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": 11, 224 | "metadata": {}, 225 | "outputs": [], 226 | "source": [ 227 | "from keras.datasets import mnist\n", 228 | "\n", 229 | "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n", 230 | "# the data, shuffled and split between train and test sets\n", 231 | "(x_train, y_train), (x_test, y_test) = mnist.load_data()" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": 12, 237 | "metadata": {}, 238 | "outputs": [], 239 | "source": [ 240 | "x_train = train_images.reshape(60000, 784)\n", 241 | "x_test = x_test.reshape(10000, 784)\n", 242 | "\n", 243 | "x_train = x_train.astype('float32')\n", 244 | "x_test = x_test.astype('float32')\n", 245 | "\n", 246 | "x_train /= 255\n", 247 | "x_test /= 255\n", 248 | "\n", 249 | "from keras.utils import to_categorical\n", 250 | "\n", 251 | "y_train = keras.utils.to_categorical(y_train, num_classes=10)\n", 252 | "y_test = keras.utils.to_categorical(y_test, num_classes=10)" 253 | ] 254 | }, 255 | { 256 | "cell_type": "markdown", 257 | "metadata": {}, 258 | "source": [ 259 | "modelo base" 260 | ] 261 | }, 262 | { 263 | "cell_type": "code", 264 | "execution_count": 18, 265 | "metadata": {}, 266 | "outputs": [ 267 | { 268 | "name": "stdout", 269 | "output_type": "stream", 270 | "text": [ 271 | "_________________________________________________________________\n", 272 | "Layer (type) Output Shape Param # \n", 273 | "=================================================================\n", 274 | "dense_11 (Dense) (None, 10) 7850 \n", 275 | "_________________________________________________________________\n", 276 | "dense_12 (Dense) (None, 10) 110 \n", 277 | "=================================================================\n", 278 | "Total params: 7,960\n", 279 | "Trainable params: 7,960\n", 280 | "Non-trainable params: 0\n", 281 | "_________________________________________________________________\n", 282 | "10000/10000 [==============================] - 0s 22us/step\n", 283 | "Test loss: 1.2463368036270142\n", 284 | "Test accuracy: 0.7618\n" 285 | ] 286 | } 287 | ], 288 | "source": [ 289 | "from keras.models import Sequential\n", 290 | "from keras.layers import Dense\n", 291 | "from keras.optimizers import sgd\n", 292 | "\n", 293 | "model = Sequential()\n", 294 | "model.add(Dense(10, activation='sigmoid', input_shape=(784,)))\n", 295 | "model.add(Dense(10, activation='softmax'))\n", 296 | "\n", 297 | "model.summary()\n", 298 | "\n", 299 | "batch_size = 100\n", 300 | "num_classes = 10\n", 301 | "epochs=5\n", 302 | "\n", 303 | "model.compile(loss='categorical_crossentropy',\n", 304 | " optimizer='sgd',\n", 305 | " metrics=['accuracy'])\n", 306 | "\n", 307 | "model.fit(x_train, y_train,\n", 308 | " batch_size=batch_size,\n", 309 | " epochs=epochs,\n", 310 | " verbose=0\n", 311 | " )\n", 312 | "\n", 313 | "test_loss, test_acc = model.evaluate(x_test, y_test)\n", 314 | "\n", 315 | "print('Test loss:', test_loss)\n", 316 | "print('Test accuracy:', test_acc)" 317 | ] 318 | }, 319 | { 320 | "cell_type": "markdown", 321 | "metadata": {}, 322 | "source": [ 323 | "con función de activación ReLu" 324 | ] 325 | }, 326 | { 327 | "cell_type": "code", 328 | "execution_count": 21, 329 | "metadata": {}, 330 | "outputs": [ 331 | { 332 | "name": "stdout", 333 | "output_type": "stream", 334 | "text": [ 335 | "_________________________________________________________________\n", 336 | "Layer (type) Output Shape Param # \n", 337 | "=================================================================\n", 338 | "dense_17 (Dense) (None, 10) 7850 \n", 339 | "_________________________________________________________________\n", 340 | "dense_18 (Dense) (None, 10) 110 \n", 341 | "=================================================================\n", 342 | "Total params: 7,960\n", 343 | "Trainable params: 7,960\n", 344 | "Non-trainable params: 0\n", 345 | "_________________________________________________________________\n", 346 | "10000/10000 [==============================] - 0s 21us/step\n", 347 | "Model2 - Test loss: 0.36076850221157075\n", 348 | "Model2 - Test accuracy: 0.8998\n" 349 | ] 350 | } 351 | ], 352 | "source": [ 353 | "batch_size = 100\n", 354 | "num_classes = 10\n", 355 | "epochs=5\n", 356 | "\n", 357 | "model2 = Sequential()\n", 358 | "model2.add(Dense(10, activation='relu', input_shape=(784,)))\n", 359 | "model2.add(Dense(10, activation='softmax'))\n", 360 | "\n", 361 | "\n", 362 | "model2.summary()\n", 363 | "\n", 364 | "\n", 365 | "model2.compile(loss='categorical_crossentropy',\n", 366 | " optimizer='sgd',\n", 367 | " metrics=['accuracy'])\n", 368 | "\n", 369 | "model2.fit(x_train, y_train,\n", 370 | " batch_size=batch_size,\n", 371 | " epochs=epochs,\n", 372 | " verbose=0\n", 373 | " )\n", 374 | "\n", 375 | "test_loss, test_acc = model2.evaluate(x_test, y_test)\n", 376 | "\n", 377 | "print('Model2 - Test loss:', test_loss)\n", 378 | "print('Model2 - Test accuracy:', test_acc)" 379 | ] 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "metadata": {}, 384 | "source": [ 385 | "con 512 nodos en capa intermedia" 386 | ] 387 | }, 388 | { 389 | "cell_type": "code", 390 | "execution_count": 20, 391 | "metadata": {}, 392 | "outputs": [ 393 | { 394 | "name": "stdout", 395 | "output_type": "stream", 396 | "text": [ 397 | "_________________________________________________________________\n", 398 | "Layer (type) Output Shape Param # \n", 399 | "=================================================================\n", 400 | "dense_15 (Dense) (None, 512) 401920 \n", 401 | "_________________________________________________________________\n", 402 | "dense_16 (Dense) (None, 10) 5130 \n", 403 | "=================================================================\n", 404 | "Total params: 407,050\n", 405 | "Trainable params: 407,050\n", 406 | "Non-trainable params: 0\n", 407 | "_________________________________________________________________\n", 408 | "10000/10000 [==============================] - 0s 40us/step\n", 409 | "Model3 - Test loss: 0.24130829737782478\n", 410 | "Model3 - Test accuracy: 0.9317\n" 411 | ] 412 | } 413 | ], 414 | "source": [ 415 | "model3 = Sequential()\n", 416 | "model3.add(Dense(512, activation='relu', input_shape=(784,)))\n", 417 | "model3.add(Dense(10, activation='softmax'))\n", 418 | "\n", 419 | "model3.summary()\n", 420 | "\n", 421 | "model3.compile(loss='categorical_crossentropy',\n", 422 | " optimizer='sgd',\n", 423 | " metrics=['accuracy'])\n", 424 | " \n", 425 | "epochs = 10\n", 426 | "model3.fit(x_train, y_train,\n", 427 | " batch_size=batch_size,\n", 428 | " epochs=epochs,\n", 429 | " verbose=0\n", 430 | " )\n", 431 | "\n", 432 | "test_loss, test_acc = model3.evaluate(x_test, y_test)\n", 433 | "\n", 434 | "print('Model3 - Test loss:', test_loss)\n", 435 | "print('Model3 - Test accuracy:', test_acc)\n", 436 | "\n", 437 | "\n" 438 | ] 439 | } 440 | ], 441 | "metadata": { 442 | "kernelspec": { 443 | "display_name": "Python 3", 444 | "language": "python", 445 | "name": "python3" 446 | }, 447 | "language_info": { 448 | "codemirror_mode": { 449 | "name": "ipython", 450 | "version": 3 451 | }, 452 | "file_extension": ".py", 453 | "mimetype": "text/x-python", 454 | "name": "python", 455 | "nbconvert_exporter": "python", 456 | "pygments_lexer": "ipython3", 457 | "version": "3.6.3" 458 | } 459 | }, 460 | "nbformat": 4, 461 | "nbformat_minor": 2 462 | } 463 | -------------------------------------------------------------------------------- /2.RedesNeuronalesDensamenteConectas.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "# 2 Redes neuronales densamente conectadas\n" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 23, 20 | "metadata": { 21 | "scrolled": true 22 | }, 23 | "outputs": [ 24 | { 25 | "data": { 26 | "text/plain": [ 27 | "'2.1.3'" 28 | ] 29 | }, 30 | "execution_count": 23, 31 | "metadata": {}, 32 | "output_type": "execute_result" 33 | } 34 | ], 35 | "source": [ 36 | "import keras\n", 37 | "keras.__version__" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "Precarga de los datos en Keras" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": 24, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "from keras.datasets import mnist\n", 54 | "\n", 55 | "# obtenemos los datos para train y test \n", 56 | "(x_train, y_train), (x_test, y_test) = mnist.load_data()" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": 25, 62 | "metadata": {}, 63 | "outputs": [ 64 | { 65 | "name": "stdout", 66 | "output_type": "stream", 67 | "text": [ 68 | "3\n" 69 | ] 70 | } 71 | ], 72 | "source": [ 73 | "print(x_train.ndim) " 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": 26, 79 | "metadata": {}, 80 | "outputs": [ 81 | { 82 | "name": "stdout", 83 | "output_type": "stream", 84 | "text": [ 85 | "(60000, 28, 28)\n" 86 | ] 87 | } 88 | ], 89 | "source": [ 90 | "print(x_train.shape)" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": 27, 96 | "metadata": {}, 97 | "outputs": [ 98 | { 99 | "name": "stdout", 100 | "output_type": "stream", 101 | "text": [ 102 | "uint8\n" 103 | ] 104 | } 105 | ], 106 | "source": [ 107 | "print(x_train.dtype)" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 28, 113 | "metadata": {}, 114 | "outputs": [ 115 | { 116 | "data": { 117 | "text/plain": [ 118 | "60000" 119 | ] 120 | }, 121 | "execution_count": 28, 122 | "metadata": {}, 123 | "output_type": "execute_result" 124 | } 125 | ], 126 | "source": [ 127 | "len (y_train)" 128 | ] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "execution_count": 29, 133 | "metadata": {}, 134 | "outputs": [ 135 | { 136 | "name": "stdout", 137 | "output_type": "stream", 138 | "text": [ 139 | "1\n" 140 | ] 141 | }, 142 | { 143 | "data": { 144 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADAJJREFUeJzt3V+InfWZwPHvs6ZBsQV1MxuDTZxukZUgbLoMYaW6RrqtVgqxXkhzUbIgTS8qbKEXq+7FeinL1urFWpiuoXHpmiy0agKyWzcsSLEUJ5L1T2PrHyY0MX8mWlMLYtQ+ezGvZaozZybnvOe8J32+HxjmnPd3Zs7DSb55zznvZN7ITCTV8yddDyCpG8YvFWX8UlHGLxVl/FJRxi8VZfxSUcYvFWX8UlGrRnlna9asycnJyVHepVTK7Owsp06dipXcdqD4I+JG4H7gPODfMvOeXrefnJxkZmZmkLuU1MPU1NSKb9v30/6IOA/4V+CLwEZgW0Rs7Pf7SRqtQV7zbwZezsxXM/MMsBvY2s5YkoZtkPgvA3614PqRZtsfiIgdETETETNzc3MD3J2kNg393f7MnM7MqcycmpiYGPbdSVqhQeI/CqxfcP2TzTZJ54BB4n8auCIiPhURq4GvAHvbGUvSsPV9qC8z34uI24H/Zv5Q387MfKG1ySQN1UDH+TPzceDxlmaRNEL+eK9UlPFLRRm/VJTxS0UZv1SU8UtFGb9UlPFLRRm/VJTxS0UZv1SU8UtFGb9UlPFLRRm/VJTxS0UZv1SU8UtFGb9UlPFLRRm/VJTxS0UZv1SU8UtFGb9UlPFLRRm/VJTxS0UZv1TUQGfpjYhZ4C3gfeC9zJxqYyi159133+25/tRTT/Vcv/POOwf6eo2vgeJvXJ+Zp1r4PpJGyKf9UlGDxp/AjyPiQETsaGMgSaMx6NP+azLzaET8GfBERLyYmU8uvEHzj8IOgA0bNgx4d5LaMtCePzOPNp9PAo8Amxe5zXRmTmXm1MTExCB3J6lFfccfERdGxCc+uAx8AXi+rcEkDdcgT/vXAo9ExAff5z8y879amUrS0PUdf2a+Cvxli7NoCE6fPt1zfcuWLT3XL7300p7rx48fH+jr1R0P9UlFGb9UlPFLRRm/VJTxS0UZv1RUG/+rT3/EljuU56G+c5d7fqko45eKMn6pKOOXijJ+qSjjl4oyfqko45eKMn6pKOOXijJ+qSjjl4oyfqko45eKMn6pKP8/vwby9ttvdz2C+uSeXyrK+KWijF8qyvilooxfKsr4paKMXypq2eP8EbET+BJwMjOvarZdAuwBJoFZ4NbM/PXwxtS4OnDgQM/1q6++ekST6GytZM//feDGD227A9ifmVcA+5vrks4hy8afmU8Cb3xo81ZgV3N5F3Bzy3NJGrJ+X/OvzcxjzeXjwNqW5pE0IgO/4ZeZCeRS6xGxIyJmImJmbm5u0LuT1JJ+4z8REesAms8nl7phZk5n5lRmTk1MTPR5d5La1m/8e4HtzeXtwGPtjCNpVJaNPyIeBn4K/EVEHImI24B7gM9HxEvA3zbXJZ1Dlj3On5nbllj6XMuzaAhWrer9R3zRRRf1XH/zzTd7rr/yyitnPZPGgz/hJxVl/FJRxi8VZfxSUcYvFWX8UlH+6u4/cssdyrv22mt7ru/bt6/NcTRG3PNLRRm/VJTxS0UZv1SU8UtFGb9UlPFLRRm/VJTxS0UZv1SU8UtFGb9UlPFLRRm/VJTxS0X5//k1kNdff73rEdQn9/xSUcYvFWX8UlHGLxVl/FJRxi8VZfxSUcse54+IncCXgJOZeVWz7W7ga8Bcc7O7MvPxYQ2p8bV3796uR1CfVrLn/z5w4yLbv5OZm5oPw5fOMcvGn5lPAm+MYBZJIzTIa/7bI+LZiNgZERe3NpGkkeg3/u8CnwY2AceAby91w4jYEREzETEzNze31M0kjVhf8Wfmicx8PzN/B3wP2NzjttOZOZWZUxMTE/3OKallfcUfEesWXP0y8Hw740galZUc6nsY2AKsiYgjwD8BWyJiE5DALPD1Ic4oaQiWjT8zty2y+cEhzKIOXH/99T3X9+3bN6JJNGr+hJ9UlPFLRRm/VJTxS0UZv1SU8UtF+au7i9uwYcNAX3/mzJme64cPH15y7fLLLx/ovjUY9/xSUcYvFWX8UlHGLxVl/FJRxi8VZfxSUR7nL27VqsH+CmRmz/V33nlnoO+v4XHPLxVl/FJRxi8VZfxSUcYvFWX8UlHGLxXlcf7itm7d2nP9yiuv7Ln+4osv9ly/7777llx74IEHen6thss9v1SU8UtFGb9UlPFLRRm/VJTxS0UZv1TUssf5I2I98BCwFkhgOjPvj4hLgD3AJDAL3JqZvx7eqOrCDTfc0HP9tdde67l+7733tjmOWrSSPf97wLcycyPw18A3ImIjcAewPzOvAPY31yWdI5aNPzOPZeYzzeW3gEPAZcBWYFdzs13AzcMaUlL7zuo1f0RMAp8BfgaszcxjzdJx5l8WSDpHrDj+iPg48EPgm5n5m4VrOf+L3Bb9ZW4RsSMiZiJiZm5ubqBhJbVnRfFHxMeYD/8HmfmjZvOJiFjXrK8DTi72tZk5nZlTmTk1MTHRxsySWrBs/BERwIPAocxc+NbtXmB7c3k78Fj740kalpX8l97PAl8FnouIg822u4B7gP+MiNuAw8CtwxlR42x+37C01atXj2gSna1l48/MnwBL/Ql/rt1xJI2KP+EnFWX8UlHGLxVl/FJRxi8VZfxSUf7qbg3k9OnTPdcfffTRJdduueWWtsfRWXDPLxVl/FJRxi8VZfxSUcYvFWX8UlHGLxXlcX71tGfPnp7r559/fs/1jRs3tjmOWuSeXyrK+KWijF8qyvilooxfKsr4paKMXyrK4/zq6brrruu5fujQoZ7rF1xwQZvjqEXu+aWijF8qyvilooxfKsr4paKMXyrK+KWilj3OHxHrgYeAtUAC05l5f0TcDXwNmGtueldmPj6sQdWN3bt3dz2ChmQlP+TzHvCtzHwmIj4BHIiIJ5q172TmvwxvPEnDsmz8mXkMONZcfisiDgGXDXswScN1Vq/5I2IS+Azws2bT7RHxbETsjIiLl/iaHRExExEzc3Nzi91EUgdWHH9EfBz4IfDNzPwN8F3g08Am5p8ZfHuxr8vM6cycysypiYmJFkaW1IYVxR8RH2M+/B9k5o8AMvNEZr6fmb8DvgdsHt6Yktq2bPwREcCDwKHMvHfB9nULbvZl4Pn2x5M0LCt5t/+zwFeB5yLiYLPtLmBbRGxi/vDfLPD1oUwoaShW8m7/T4BYZMlj+tI5zJ/wk4oyfqko45eKMn6pKOOXijJ+qSjjl4oyfqko45eKMn6pKOOXijJ+qSjjl4oyfqmoyMzR3VnEHHB4waY1wKmRDXB2xnW2cZ0LnK1fbc52eWau6PfljTT+j9x5xExmTnU2QA/jOtu4zgXO1q+uZvNpv1SU8UtFdR3/dMf338u4zjauc4Gz9auT2Tp9zS+pO13v+SV1pJP4I+LGiPhFRLwcEXd0McNSImI2Ip6LiIMRMdPxLDsj4mREPL9g2yUR8UREvNR8XvQ0aR3NdndEHG0eu4MRcVNHs62PiP+NiJ9HxAsR8ffN9k4fux5zdfK4jfxpf0ScB/wS+DxwBHga2JaZPx/pIEuIiFlgKjM7PyYcEX8D/BZ4KDOvarb9M/BGZt7T/MN5cWb+w5jMdjfw267P3NycUGbdwjNLAzcDf0eHj12PuW6lg8etiz3/ZuDlzHw1M88Au4GtHcwx9jLzSeCND23eCuxqLu9i/i/PyC0x21jIzGOZ+Uxz+S3ggzNLd/rY9ZirE13EfxnwqwXXjzBep/xO4McRcSAidnQ9zCLWNqdNBzgOrO1ymEUse+bmUfrQmaXH5rHr54zXbfMNv4+6JjP/Cvgi8I3m6e1YyvnXbON0uGZFZ24elUXOLP17XT52/Z7xum1dxH8UWL/g+iebbWMhM482n08CjzB+Zx8+8cFJUpvPJzue5/fG6czNi51ZmjF47MbpjNddxP80cEVEfCoiVgNfAfZ2MMdHRMSFzRsxRMSFwBcYv7MP7wW2N5e3A491OMsfGJczNy91Zmk6fuzG7ozXmTnyD+Am5t/xfwX4xy5mWGKuPwf+r/l4oevZgIeZfxr4LvPvjdwG/CmwH3gJ+B/gkjGa7d+B54BnmQ9tXUezXcP8U/pngYPNx01dP3Y95urkcfMn/KSifMNPKsr4paKMXyrK+KWijF8qyvilooxfKsr4paL+HwHpwMdFxzmuAAAAAElFTkSuQmCC\n", 145 | "text/plain": [ 146 | "" 147 | ] 148 | }, 149 | "metadata": {}, 150 | "output_type": "display_data" 151 | } 152 | ], 153 | "source": [ 154 | "import matplotlib.pyplot as plt\n", 155 | "plt.imshow(x_train[8], cmap=plt.cm.binary)\n", 156 | "print(y_train[8])" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": 32, 162 | "metadata": {}, 163 | "outputs": [ 164 | { 165 | "name": "stdout", 166 | "output_type": "stream", 167 | "text": [ 168 | "[[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 169 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 170 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 171 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 172 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 173 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 5 63 197 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 174 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 20 254 230 24 0 0 0 0 0 0 0 0 0 0 0 0]\n", 175 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 20 254 254 48 0 0 0 0 0 0 0 0 0 0 0 0]\n", 176 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 20 254 255 48 0 0 0 0 0 0 0 0 0 0 0 0]\n", 177 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 20 254 254 57 0 0 0 0 0 0 0 0 0 0 0 0]\n", 178 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 20 254 254 108 0 0 0 0 0 0 0 0 0 0 0 0]\n", 179 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 16 239 254 143 0 0 0 0 0 0 0 0 0 0 0 0]\n", 180 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 178 254 143 0 0 0 0 0 0 0 0 0 0 0 0]\n", 181 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 178 254 143 0 0 0 0 0 0 0 0 0 0 0 0]\n", 182 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 178 254 162 0 0 0 0 0 0 0 0 0 0 0 0]\n", 183 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 178 254 240 0 0 0 0 0 0 0 0 0 0 0 0]\n", 184 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 113 254 240 0 0 0 0 0 0 0 0 0 0 0 0]\n", 185 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 83 254 245 31 0 0 0 0 0 0 0 0 0 0 0]\n", 186 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 79 254 246 38 0 0 0 0 0 0 0 0 0 0 0]\n", 187 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 214 254 150 0 0 0 0 0 0 0 0 0 0 0]\n", 188 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 144 241 8 0 0 0 0 0 0 0 0 0 0 0]\n", 189 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 144 240 2 0 0 0 0 0 0 0 0 0 0 0]\n", 190 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 144 254 82 0 0 0 0 0 0 0 0 0 0 0]\n", 191 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 230 247 40 0 0 0 0 0 0 0 0 0 0 0]\n", 192 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 168 209 31 0 0 0 0 0 0 0 0 0 0 0]\n", 193 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 194 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n", 195 | " [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]]\n" 196 | ] 197 | } 198 | ], 199 | "source": [ 200 | "import numpy\n", 201 | "from numpy import linalg\n", 202 | "numpy.set_printoptions(precision=2, suppress=True, linewidth=120)\n", 203 | "print(numpy.matrix(x_train[8]))\n", 204 | "\n" 205 | ] 206 | }, 207 | { 208 | "cell_type": "code", 209 | "execution_count": 33, 210 | "metadata": {}, 211 | "outputs": [ 212 | { 213 | "name": "stdout", 214 | "output_type": "stream", 215 | "text": [ 216 | "4\n" 217 | ] 218 | }, 219 | { 220 | "data": { 221 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADdRJREFUeJzt3W+MVPW9x/HPV4XE2CZCWTcEsNsCudGQSJsJMSm5Vrk01jRiidmUB7gq6fqgRis8uEQlVx9o9Ia2aaKioBtorfwxrZEH5t56oUoasWE01AVs1WsWYUVYQmOtieEi3/tgD82Ke34zzJyZM8v3/Uo2O3O+85vzZfSzZ+acM+dn7i4A8VxQdgMAykH4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EdVE7VzZt2jTv6elp5yqBUIaGhnT8+HGr57FNhd/Mrpf0S0kXSnra3R9JPb6np0fVarWZVQJIqFQqdT+24bf9ZnahpMclfV/SlZKWmdmVjT4fgPZq5jP/Aknvufv77n5S0hZJS4ppC0CrNRP+GZIOjbl/OFv2BWbWb2ZVM6uOjIw0sToARWr53n53X+/uFXevdHV1tXp1AOrUTPiHJc0ac39mtgzABNBM+PdImmtm3zCzyZJ+JGl7MW0BaLWGD/W5+ykzu1PSf2v0UN+Au+8vrDMALdXUcX53f0nSSwX1AqCNOL0XCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaDaOkU3MFFcd911TY3fuXNnQZ20Dlt+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiqqeP8ZjYk6RNJn0s65e6VIpoCWu2ee+5J1nfv3p2s33LLLUW2U4oiTvK51t2PF/A8ANqIt/1AUM2G3yX93szeMLP+IhoC0B7Nvu1f6O7DZnaZpJfN7C/uvmvsA7I/Cv2SdPnllze5OgBFaWrL7+7D2e9jkl6QtGCcx6x394q7V7q6uppZHYACNRx+M7vEzL565rak70naV1RjAFqrmbf93ZJeMLMzz/Ocu/9XIV0BaLmGw+/u70u6qsBegEKtXr06t/bkk08mx06aNClZX7RoUUM9dRIO9QFBEX4gKMIPBEX4gaAIPxAU4QeC4tLdOG+9/vrrubWTJ08mxy5cuDBZ7+3tbainTsKWHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeC4jj/eW7Xrl3J+kMPPZSsb968OVmfOnXqOfdUlFq9DQ4O5tbmzJmTHLt27dqGeppI2PIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFAc5z/P9fenp1B85513kvUDBw4k67W+995Ktc5ROHHiRG7t6aefTo696qrz/6r0bPmBoAg/EBThB4Ii/EBQhB8IivADQRF+IKiax/nNbEDSDyQdc/d52bKpkrZK6pE0JKnX3f/WujbRqIsvvjhZN7Nk/bPPPiuynXOyd+/eZP2DDz5I1lP/tjL/XZ2ini3/RknXn7VstaQd7j5X0o7sPoAJpGb43X2XpLNPlVoiaVN2e5OkmwruC0CLNfqZv9vdj2S3P5LUXVA/ANqk6R1+7u6SPK9uZv1mVjWz6sjISLOrA1CQRsN/1MymS1L2+1jeA919vbtX3L3S1dXV4OoAFK3R8G+X1Jfd7pP0YjHtAGiXmuE3s82Sdkv6FzM7bGYrJD0iabGZvSvp37L7ACaQmsf53X1ZTmlRwb2gQWvWrMmt7du3Lzn2iiuuSNZb+b32Tz/9NFl/9NFHmxp/9dVX59Zuvvnm5NgIOMMPCIrwA0ERfiAowg8ERfiBoAg/EBSX7p4ADh06lKxv2LAht3bRRen/xI8//niy3sqzMleuXJmsb9u2LVmfMWNGsv7aa6+dc0+RsOUHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaA4zt8BBgcHk/WlS5cm66nLo911113Jsddcc02y3qy1a9fm1jZu3NjUc993331NjY+OLT8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMVx/gKcOnUqWX/22WeT9dtvvz1ZH50RLV9qKurdu3cnxz788MPJ+qpVq5L1EyfOnsP1i55//vncWq1/V19fX7J+xx13JOtIY8sPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0HVPM5vZgOSfiDpmLvPy5Y9IOnHks58kfxed3+pVU12ui1btiTrK1asSNZTx+nrMXfu3Nzanj17kmNr1bdv356sDw8PJ+sffvhhbu2yyy5Ljh0YGEjW0Zx6tvwbJV0/zvJfuPv87Cds8IGJqmb43X2XpPRpXAAmnGY+899pZm+Z2YCZTSmsIwBt0Wj410maLWm+pCOSfpb3QDPrN7OqmVVT15oD0F4Nhd/dj7r75+5+WtIGSQsSj13v7hV3r7Ry0kcA56ah8JvZ9DF3fyhpXzHtAGiXeg71bZb0XUnTzOywpP+Q9F0zmy/JJQ1J4ruVwARTM/zuvmycxc+0oJeOtnXr1tzabbfdlhw7efLkZP3SSy9N1p977rlkfcqU/P2tK1euTI599dVXk/Va5wE0c62B48ePJ8fOmjUrWX/llVeS9dmzZyfr0XGGHxAU4QeCIvxAUIQfCIrwA0ERfiAoLt1dp6eeeiq3VuuQ1P3335+s17p0dzMee+yxZL2/vz9Zr3Xp72acPn06Wb/22muTdQ7lNYctPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ExXH+Oi1ZsiS3tnTp0uTYWucBtFKtr83u37+/qeevddnyefPmNfzcM2fObHgsamPLDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBcZy/TnfffXfZLeT6+OOPc2vbtm1reKwkzZkzJ1nv7e1N1tG52PIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFA1j/Ob2SxJv5LULcklrXf3X5rZVElbJfVIGpLU6+5/a12ryPPEE0/k1tatW5cc293dnazv3LmzoZ7Q+erZ8p+StMrdr5R0taSfmNmVklZL2uHucyXtyO4DmCBqht/dj7j7m9ntTyS9LWmGpCWSNmUP2yTpplY1CaB45/SZ38x6JH1L0p8kdbv7kaz0kUY/FgCYIOoOv5l9RdJvJf3U3f8+tuburtH9AeON6zezqplVR0ZGmmoWQHHqCr+ZTdJo8H/j7r/LFh81s+lZfbqkY+ONdff17l5x90pXV1cRPQMoQM3wm5lJekbS2+7+8zGl7ZL6stt9kl4svj0ArVLPV3q/I2m5pEEz25stu1fSI5K2mdkKSQcl8d3OFjl48GCyvmHDhtzaBRek/77XmqKby2efv2qG393/KMlyyouKbQdAu3CGHxAU4QeCIvxAUIQfCIrwA0ERfiAoLt09ASxevDhZT50HsHz58uTYBx98sKGeMPGx5QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoDjOPwHceuutyfqaNWtyazfeeGPB3eB8wZYfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ky0Zm22qNSqXi1Wm3b+oBoKpWKqtVq3qX2v4AtPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EVTP8ZjbLzP5gZgfMbL+Z3Z0tf8DMhs1sb/ZzQ+vbBVCUei7mcUrSKnd/08y+KukNM3s5q/3C3de2rj0ArVIz/O5+RNKR7PYnZva2pBmtbgxAa53TZ34z65H0LUl/yhbdaWZvmdmAmU3JGdNvZlUzq46MjDTVLIDi1B1+M/uKpN9K+qm7/13SOkmzJc3X6DuDn403zt3Xu3vF3StdXV0FtAygCHWF38wmaTT4v3H330mSux9198/d/bSkDZIWtK5NAEWrZ2+/SXpG0tvu/vMxy6ePedgPJe0rvj0ArVLP3v7vSFouadDM9mbL7pW0zMzmS3JJQ5LuaEmHAFqinr39f5Q03veDXyq+HQDtwhl+QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoNo6RbeZjUg6OGbRNEnH29bAuenU3jq1L4neGlVkb19397qul9fW8H9p5WZVd6+U1kBCp/bWqX1J9NaosnrjbT8QFOEHgio7/OtLXn9Kp/bWqX1J9NaoUnor9TM/gPKUveUHUJJSwm9m15vZX83sPTNbXUYPecxsyMwGs5mHqyX3MmBmx8xs35hlU83sZTN7N/s97jRpJfXWETM3J2aWLvW167QZr9v+tt/MLpT0jqTFkg5L2iNpmbsfaGsjOcxsSFLF3Us/Jmxm/yrpH5J+5e7zsmX/KemEuz+S/eGc4u7/3iG9PSDpH2XP3JxNKDN97MzSkm6SdKtKfO0SffWqhNetjC3/Aknvufv77n5S0hZJS0roo+O5+y5JJ85avETSpuz2Jo3+z9N2Ob11BHc/4u5vZrc/kXRmZulSX7tEX6UoI/wzJB0ac/+wOmvKb5f0ezN7w8z6y25mHN3ZtOmS9JGk7jKbGUfNmZvb6ayZpTvmtWtkxuuiscPvyxa6+7clfV/ST7K3tx3JRz+zddLhmrpmbm6XcWaW/qcyX7tGZ7wuWhnhH5Y0a8z9mdmyjuDuw9nvY5JeUOfNPnz0zCSp2e9jJffzT500c/N4M0urA167Tprxuozw75E018y+YWaTJf1I0vYS+vgSM7sk2xEjM7tE0vfUebMPb5fUl93uk/Riib18QafM3Jw3s7RKfu06bsZrd2/7j6QbNLrH/38l3VdGDzl9fVPSn7Of/WX3JmmzRt8G/p9G942skPQ1STskvSvpfyRN7aDefi1pUNJbGg3a9JJ6W6jRt/RvSdqb/dxQ9muX6KuU140z/ICg2OEHBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiCo/webFEk6bu28lwAAAABJRU5ErkJggg==\n", 222 | "text/plain": [ 223 | "" 224 | ] 225 | }, 226 | "metadata": {}, 227 | "output_type": "display_data" 228 | } 229 | ], 230 | "source": [ 231 | "plt.imshow(x_train[9], cmap=plt.cm.binary)\n", 232 | "print(y_train[9])" 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": 34, 238 | "metadata": {}, 239 | "outputs": [ 240 | { 241 | "name": "stdout", 242 | "output_type": "stream", 243 | "text": [ 244 | "3\n" 245 | ] 246 | }, 247 | { 248 | "data": { 249 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADctJREFUeJzt3XGMlPWdx/HP92iJia0Kxx5BULdXyYnxD7hMSE3xwtkrEdOIGGPApHKGHDV29Qg10ViSUxMTQ66Qxlyq25OUnj2gCTWiMV6VXGJIzsZBV1DW0z1YLLiyi5QAiUKx3/tjH5pFd34zzDzPPLN8369kszPP93me3zcTPjwz85udn7m7AMTzF2U3AKAchB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFBfaedg06ZN8+7u7nYOCYQyODioI0eOWCP7thR+M7tJ0k8lTZL07+7+RGr/7u5uVavVVoYEkFCpVBret+mn/WY2SdK/SVos6VpJy83s2mbPB6C9WnnNP1/SgLvvc/fTkrZIWpJPWwCK1kr4Z0r6/Zj7B7Nt5zCzVWZWNbPqyMhIC8MByFPh7/a7e6+7V9y90tXVVfRwABrUSvgPSbpizP1Z2TYAE0Ar4X9D0mwz+4aZTZa0TNL2fNoCULSmp/rc/YyZ9Uj6L41O9W1093dz6wxAoVqa53f3lyS9lFMvANqIj/cCQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFRbl+hGMfbu3Vuz9uKLLyaPffrpp5P1+fPnJ+vz5s1L1lNWr16drE+ePLnpc6M+rvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFRL8/xmNijphKTPJZ1x90oeTeFc9ebiH3jggZq1kydPtjT2vn37kvUtW7Y0fe5KJf3P5cYbb2z63Kgvjw/5/L27H8nhPADaiKf9QFCtht8l/dbMdpnZqjwaAtAerT7tX+Duh8zsryS9YmbvuftrY3fI/lNYJUlXXnlli8MByEtLV353P5T9Hpb0nKQv/RWIu/e6e8XdK11dXa0MByBHTYffzC42s6+fvS1pkaR38moMQLFaedo/XdJzZnb2PP/p7i/n0hWAwpm7t22wSqXi1Wq1beNdKI4ePZqsz5kzp2ZteHg473Zyc9lllyXrW7duTdYXLVqUZzsXhEqlomq1ao3sy1QfEBThB4Ii/EBQhB8IivADQRF+ICi+unsCmDp1arL+6KOP1qytWbMmeeynn36arNf7SPaHH36YrKccO3YsWX/55fTHRpjqaw1XfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8Iinn+C8A999xTs/bUU08lj3377beT9UsuuaSpnvLQ09NT2tgRcOUHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaCY57/ArV27Nll//PHHk/W+vr482zkvp06dKm3sCLjyA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQdef5zWyjpO9JGnb367JtUyVtldQtaVDSHe7+h+LaRLNuv/32ZH3BggXJer3vxt+zZ89599Soep9R2LZtW2FjR9DIlf8Xkm76wraHJO1w99mSdmT3AUwgdcPv7q9JOvqFzUskbcpub5J0a859AShYs6/5p7v7UHb7Y0nTc+oHQJu0/Iafu7skr1U3s1VmVjWz6sjISKvDAchJs+E/bGYzJCn7PVxrR3fvdfeKu1e6urqaHA5A3poN/3ZJK7LbKyQ9n087ANqlbvjNbLOk/5H0N2Z20MxWSnpC0nfN7ANJ/5DdBzCB1J3nd/flNUrfybkXFODZZ59N1nfv3p2sFzmPX88NN9xQ2tgR8Ak/ICjCDwRF+IGgCD8QFOEHgiL8QFB8dfcE8N577yXrS5curVkbGBhIHnvmzJmmemqHW265pewWLmhc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKOb5J4D+/v5kff/+/TVrnTyPX8+GDRuS9SeffLJNnVyYuPIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFDM808Aqb/Xl6R169bVrD344IPJYz/77LOmemqHjz76qOwWLmhc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqLrz/Ga2UdL3JA27+3XZtkck/ZOkkWy3h939paKaRNr9999fszZ79uzksceOHWtp7HrfF9DT01Ozdvz48ZbGRmsaufL/QtJN42zf4O5zsx+CD0wwdcPv7q9JOtqGXgC0USuv+XvMbLeZbTSzKbl1BKAtmg3/zyR9U9JcSUOSflJrRzNbZWZVM6uOjIzU2g1AmzUVfnc/7O6fu/ufJP1c0vzEvr3uXnH3SldXV7N9AshZU+E3sxlj7i6V9E4+7QBol0am+jZLWihpmpkdlPQvkhaa2VxJLmlQ0g8K7BFAAeqG392Xj7P5mQJ6QQEWL15c6PndPVkfGBioWXvssceSx/b19SXrBw4cSNavuuqqZD06PuEHBEX4gaAIPxAU4QeCIvxAUIQfCIqv7kZLTp8+nazXm85LmTx5crI+adKkps8NrvxAWIQfCIrwA0ERfiAowg8ERfiBoAg/EBTz/GjJ2rVrCzv3ypUrk/VZs2YVNnYEXPmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjm+Rv0ySef1KzdfffdyWOXLVuWrN95551N9dQOQ0NDyXpvb29hY992222FnRtc+YGwCD8QFOEHgiL8QFCEHwiK8ANBEX4gqLrz/GZ2haRfSpouySX1uvtPzWyqpK2SuiUNSrrD3f9QXKvluu+++2rWXnjhheSx77//frI+c+bMlupXX311zdquXbuSx9brbd26dcn68ePHk/WUNWvWJOuXX3550+dGfY1c+c9I+pG7XyvpW5J+aGbXSnpI0g53ny1pR3YfwARRN/zuPuTub2a3T0jqlzRT0hJJm7LdNkm6tagmAeTvvF7zm1m3pHmSfidpuruf/eznxxp9WQBggmg4/Gb2NUnbJK1293Ne6Lm7a/T9gPGOW2VmVTOrjoyMtNQsgPw0FH4z+6pGg/8rd/9Ntvmwmc3I6jMkDY93rLv3unvF3StdXV159AwgB3XDb2Ym6RlJ/e6+fkxpu6QV2e0Vkp7Pvz0ARWnkT3q/Len7kvaYWV+27WFJT0j6tZmtlHRA0h3FtNgZUlN9+/fvTx77+uuvJ+sLFy5M1ru7u5P1OXPm1Kzt3LkzeeyJEyeS9VZdc801NWv1lu++6KKL8m4HY9QNv7vvlGQ1yt/Jtx0A7cIn/ICgCD8QFOEHgiL8QFCEHwiK8ANB8dXdDbr++uubqknSXXfdlazfe++9yfrg4GBL9SJNmTIlWe/v729TJzhfXPmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjm+XOwfv36ZP3UqVPJ+smTJ1sa/6233qpZ27x5c0vnvvTSS5P1V199taXzozxc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKBtdaas9KpWKV6vVto0HRFOpVFStVmt91f45uPIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFB1w29mV5jZf5vZXjN718z+Odv+iJkdMrO+7Ofm4tsFkJdGvszjjKQfufubZvZ1SbvM7JWstsHd/7W49gAUpW743X1I0lB2+4SZ9UuaWXRjAIp1Xq/5zaxb0jxJv8s29ZjZbjPbaGbjrttkZqvMrGpm1ZGRkZaaBZCfhsNvZl+TtE3Sanc/Lulnkr4paa5Gnxn8ZLzj3L3X3SvuXunq6sqhZQB5aCj8ZvZVjQb/V+7+G0ly98Pu/rm7/0nSzyXNL65NAHlr5N1+k/SMpH53Xz9m+4wxuy2V9E7+7QEoSiPv9n9b0vcl7TGzvmzbw5KWm9lcSS5pUNIPCukQQCEaebd/p6Tx/j74pfzbAdAufMIPCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVFuX6DazEUkHxmyaJulI2xo4P53aW6f2JdFbs/Ls7Sp3b+j78toa/i8NblZ190ppDSR0am+d2pdEb80qqzee9gNBEX4gqLLD31vy+Cmd2lun9iXRW7NK6a3U1/wAylP2lR9ASUoJv5ndZGb/a2YDZvZQGT3UYmaDZrYnW3m4WnIvG81s2MzeGbNtqpm9YmYfZL/HXSatpN46YuXmxMrSpT52nbbidduf9pvZJEnvS/qupIOS3pC03N33trWRGsxsUFLF3UufEzazv5N0UtIv3f26bNs6SUfd/YnsP84p7v5gh/T2iKSTZa/cnC0oM2PsytKSbpX0jyrxsUv0dYdKeNzKuPLPlzTg7vvc/bSkLZKWlNBHx3P31yQd/cLmJZI2Zbc3afQfT9vV6K0juPuQu7+Z3T4h6ezK0qU+dom+SlFG+GdK+v2Y+wfVWUt+u6TfmtkuM1tVdjPjmJ4tmy5JH0uaXmYz46i7cnM7fWFl6Y557JpZ8TpvvOH3ZQvc/W8lLZb0w+zpbUfy0ddsnTRd09DKze0yzsrSf1bmY9fsitd5KyP8hyRdMeb+rGxbR3D3Q9nvYUnPqfNWHz58dpHU7Pdwyf38WSet3DzeytLqgMeuk1a8LiP8b0iabWbfMLPJkpZJ2l5CH19iZhdnb8TIzC6WtEidt/rwdkkrstsrJD1fYi/n6JSVm2utLK2SH7uOW/Ha3dv+I+lmjb7j/3+SflxGDzX6+mtJb2c/75bdm6TNGn0a+EeNvjeyUtJfStoh6QNJr0qa2kG9/YekPZJ2azRoM0rqbYFGn9LvltSX/dxc9mOX6KuUx41P+AFB8YYfEBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGg/h+py0p9Jsq34QAAAABJRU5ErkJggg==\n", 250 | "text/plain": [ 251 | "" 252 | ] 253 | }, 254 | "metadata": {}, 255 | "output_type": "display_data" 256 | } 257 | ], 258 | "source": [ 259 | "plt.imshow(x_train[10], cmap=plt.cm.binary)\n", 260 | "print(y_train[10])" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": 35, 266 | "metadata": {}, 267 | "outputs": [ 268 | { 269 | "name": "stdout", 270 | "output_type": "stream", 271 | "text": [ 272 | "6\n" 273 | ] 274 | }, 275 | { 276 | "data": { 277 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADnlJREFUeJzt3X+MVPW5x/HPo5Y/tJio7CUI6vZW8EdIpDAhBM1NDZZYbRbQxNQYgwneJVqTW+GPGiupUWKMuaVBc4MuP1J60ys0aVUi5lovNrH1B2Hc5fqj3ivULBZcYAkaJCb2os/9Y4/NKjvfM8ycmTPr834lm505z5w5DwMfzpz5zjlfc3cBiOe0shsAUA7CDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqDPaubFJkyZ5d3d3OzcJhDI4OKgjR45YPY9tKvxmdq2ktZJOl7TB3R9OPb67u1vVarWZTQJIqFQqdT+24bf9Zna6pH+T9H1Jl0u62cwub/T5ALRXM8f8cyXtdff33P1vkrZIWlRMWwBarZnwT5X011H392fLvsTMes2sambV4eHhJjYHoEgt/7Tf3fvcveLula6urlZvDkCdmgn/AUkXjLo/LVsGYBxoJvy7JE03s2+Z2QRJP5S0rZi2ALRaw0N97n7CzO6S9LxGhvo2ufvbhXUGoKWaGud39+ckPVdQLwDaiK/3AkERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVFsv3Y3GfPrpp8n6/Pnza9YGBgaS6/b09CTrTz/9dLKO8Ys9PxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ExTh/B8gbx7/77ruT9d27d9esmaVna54zZ06yjq8v9vxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFRT4/xmNijpY0mfSTrh7pUimorm0UcfTdafeOKJZH3BggU1aw888EBy3Xnz5iXr+Poq4ks+V7v7kQKeB0Ab8bYfCKrZ8Luk35vZ62bWW0RDANqj2bf9V7n7ATP7B0kvmNn/uPtLox+Q/afQK0kXXnhhk5sDUJSm9vzufiD7fVjSU5LmjvGYPnevuHulq6urmc0BKFDD4Tezs8xs4he3JS2U9FZRjQForWbe9k+W9FR2yugZkv7D3f+zkK4AtFzD4Xf39yRdUWAvYQ0NDTW1/jXXXFOzxjg+amGoDwiK8ANBEX4gKMIPBEX4gaAIPxAUl+7uAMePH0/WJ0yYkKynhvqAWtjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPO3wQcffJCsb9iwIVmfP39+sj579uxT7glgzw8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHO3warV68uu4Vx6dVXX03W9+/f3/BzX3FF+qrzM2bMaPi5xwv2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVO44v5ltkvQDSYfdfWa27FxJWyV1SxqUdJO7f9i6Nse37du3N7X+7bffXlAn7XfHHXfUrOW9Lh9+mP4n9cknnzTUkySdffbZyfqKFSuS9VWrVjW87U5Rz57/l5Ku/cqyeyTtcPfpknZk9wGMI7nhd/eXJB39yuJFkjZntzdLWlxwXwBarNFj/snuPpTdPihpckH9AGiTpj/wc3eX5LXqZtZrZlUzqw4PDze7OQAFaTT8h8xsiiRlvw/XeqC797l7xd0rXV1dDW4OQNEaDf82SUuz20slPVNMOwDaJTf8ZvakpFclXWJm+81smaSHJX3PzPZIuia7D2AcsZFD9vaoVCperVbbtr12yRtvvvjii5P1M85If93i/fffP+We6nXixIlkvb+/P1lfvDg90HPw4MGatbx/e3mHiVdeeWWynuo97zWdOnVqsv7yyy8n6xdddFGy3iqVSkXVatXqeSzf8AOCIvxAUIQfCIrwA0ERfiAowg8ExaW7C5A3xfahQ4eS9eXLlxfZzpfkTQ/e19eXrD/44INNbT81ZHbrrbcm173zzjuT9WnTpjXUkyT19PQk63mnGw8NDSXrZQ31nQr2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOP8BRgYGGhq/enTpxfUycnypgd//PHHk3Wz9NmhCxYsSNbXrFlTszZz5szkuq2Ud5p1BOz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvkLkHfOfKu9++67NWtbtmxp6rl7e3uT9bVr1ybrEyZMaGr7ZZkzZ06yPnv27DZ10jrs+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqNxxfjPbJOkHkg67+8xs2f2S/lnScPawe939uVY12emOHTuWrOdNRd3sNOmPPfZYzdpHH32UXPeWW25J1tetW9dQT53u+PHjyXretOnj9fsLo9Wz5/+lpGvHWP4Ld5+V/YQNPjBe5Ybf3V+SdLQNvQBoo2aO+e8yszfMbJOZnVNYRwDaotHwr5P0bUmzJA1J+nmtB5pZr5lVzaw6PDxc62EA2qyh8Lv7IXf/zN0/l7Re0tzEY/vcveLula6urkb7BFCwhsJvZlNG3V0i6a1i2gHQLvUM9T0p6buSJpnZfkk/k/RdM5slySUNSmrdHNMAWiI3/O5+8xiLN7agl3Er79r2zdbzpK4nkPfcZV+LoJVSf7YNGzYk173xxhuLbqfj8A0/ICjCDwRF+IGgCD8QFOEHgiL8QFBcuvtroK+vr2btlVdeSa6bV3/ooYeS9eXL01/xOO+885L1Vrrhhhtq1s4888zkuitXriy6nY7Dnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmKcv06p00OHhoba2MnJUmPp/f39yXV7enqS9VWrViXrzz//fLL+7LPP1qxNnDix4XUlafXq1cn6wMBAzdp9992XXHfevHnJ+tcBe34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIpx/jqdf/75NWszZsxIrrtv375k/cUXX0zW886ZT52bPmXKlJo1Sdq1a1eynjfWftlllyXrqSnC886Zz7u8dt45+amx/LzvL0TAnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgsod5zezCyT9StJkSS6pz93Xmtm5krZK6pY0KOkmd/+wda12ro0b0zOWX3/99cn69u3bk/WFCxcm6ytWrKhZyxvnz7Nz585kPe+6/qn13T257iWXXNLUtpcsWZKsR1fPnv+EpJXufrmkeZJ+ZGaXS7pH0g53ny5pR3YfwDiRG353H3L3/uz2x5LekTRV0iJJm7OHbZa0uFVNAijeKR3zm1m3pO9I2ilpsrt/cf2qgxo5LAAwTtQdfjP7pqTfSvqxux8bXfORg7cxD+DMrNfMqmZWHR4ebqpZAMWpK/xm9g2NBP/X7v67bPEhM5uS1adIOjzWuu7e5+4Vd690dXUV0TOAAuSG38xM0kZJ77j7mlGlbZKWZreXSnqm+PYAtIrlDbeY2VWS/ijpTUmfZ4vv1chx/28kXShpn0aG+o6mnqtSqXi1Wm2253En79LeV199dbK+Z8+eItv5kjr+/lu27dtuuy1Zf+SRR5L1Mqf/7lSVSkXVarWuv7TccX53/5OkWk+24FQaA9A5+IYfEBThB4Ii/EBQhB8IivADQRF+ICgu3d0GeafVvvbaa8n61q1bk/W9e/fWrK1fvz657rJly5L1005rbv+Qev5LL720qedGc9jzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQuefzFynq+fxAu5zK+fzs+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiCo3PCb2QVm9gcz+7OZvW1m/5Itv9/MDpjZ7uznuta3C6Ao9UzacULSSnfvN7OJkl43sxey2i/c/V9b1x6AVskNv7sPSRrKbn9sZu9ImtrqxgC01ikd85tZt6TvSNqZLbrLzN4ws01mdk6NdXrNrGpm1eHh4aaaBVCcusNvZt+U9FtJP3b3Y5LWSfq2pFkaeWfw87HWc/c+d6+4e6Wrq6uAlgEUoa7wm9k3NBL8X7v77yTJ3Q+5+2fu/rmk9ZLmtq5NAEWr59N+k7RR0jvuvmbU8tFTzy6R9Fbx7QFolXo+7b9S0q2S3jSz3dmyeyXdbGazJLmkQUnLW9IhgJao59P+P0ka6zrgzxXfDoB24Rt+QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoMzd27cxs2FJ+0YtmiTpSNsaODWd2lun9iXRW6OK7O0id6/renltDf9JGzerunultAYSOrW3Tu1LordGldUbb/uBoAg/EFTZ4e8refspndpbp/Yl0VujSumt1GN+AOUpe88PoCSlhN/MrjWz/zWzvWZ2Txk91GJmg2b2ZjbzcLXkXjaZ2WEze2vUsnPN7AUz25P9HnOatJJ664iZmxMzS5f62nXajNdtf9tvZqdLelfS9yTtl7RL0s3u/ue2NlKDmQ1Kqrh76WPCZvZPko5L+pW7z8yWPSLpqLs/nP3HeY67/6RDertf0vGyZ27OJpSZMnpmaUmLJd2mEl+7RF83qYTXrYw9/1xJe939PXf/m6QtkhaV0EfHc/eXJB39yuJFkjZntzdr5B9P29XorSO4+5C792e3P5b0xczSpb52ib5KUUb4p0r666j7+9VZU367pN+b2etm1lt2M2OYnE2bLkkHJU0us5kx5M7c3E5fmVm6Y167Rma8Lhof+J3sKnefLen7kn6Uvb3tSD5yzNZJwzV1zdzcLmPMLP13Zb52jc54XbQywn9A0gWj7k/LlnUEdz+Q/T4s6Sl13uzDh76YJDX7fbjkfv6uk2ZuHmtmaXXAa9dJM16XEf5dkqab2bfMbIKkH0raVkIfJzGzs7IPYmRmZ0laqM6bfXibpKXZ7aWSnimxly/plJmba80srZJfu46b8drd2/4j6TqNfOL/F0k/LaOHGn39o6T/zn7eLrs3SU9q5G3g/2nks5Flks6TtEPSHkn/JencDurt3yW9KekNjQRtSkm9XaWRt/RvSNqd/VxX9muX6KuU141v+AFB8YEfEBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGg/h92sok3vzEgmAAAAABJRU5ErkJggg==\n", 278 | "text/plain": [ 279 | "" 280 | ] 281 | }, 282 | "metadata": {}, 283 | "output_type": "display_data" 284 | } 285 | ], 286 | "source": [ 287 | "plt.imshow(x_test[11], cmap=plt.cm.binary)\n", 288 | "print(y_test[11])" 289 | ] 290 | }, 291 | { 292 | "cell_type": "code", 293 | "execution_count": 36, 294 | "metadata": {}, 295 | "outputs": [], 296 | "source": [ 297 | "x_train = x_train.astype('float32')\n", 298 | "x_test = x_test.astype('float32')\n", 299 | "\n", 300 | "x_train /= 255\n", 301 | "x_test /= 255\n" 302 | ] 303 | }, 304 | { 305 | "cell_type": "code", 306 | "execution_count": 37, 307 | "metadata": {}, 308 | "outputs": [ 309 | { 310 | "name": "stdout", 311 | "output_type": "stream", 312 | "text": [ 313 | "[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 314 | " 0. 0. 0. 0. 0. ]\n", 315 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 316 | " 0. 0. 0. 0. 0. ]\n", 317 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 318 | " 0. 0. 0. 0. 0. ]\n", 319 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 320 | " 0. 0. 0. 0. 0. ]\n", 321 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 322 | " 0. 0. 0. 0. 0. ]\n", 323 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.02 0.25 0.77 0. 0. 0. 0. 0. 0. 0. 0.\n", 324 | " 0. 0. 0. 0. 0. ]\n", 325 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.08 1. 0.9 0.09 0. 0. 0. 0. 0. 0. 0.\n", 326 | " 0. 0. 0. 0. 0. ]\n", 327 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.08 1. 1. 0.19 0. 0. 0. 0. 0. 0. 0.\n", 328 | " 0. 0. 0. 0. 0. ]\n", 329 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.08 1. 1. 0.19 0. 0. 0. 0. 0. 0. 0.\n", 330 | " 0. 0. 0. 0. 0. ]\n", 331 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.08 1. 1. 0.22 0. 0. 0. 0. 0. 0. 0.\n", 332 | " 0. 0. 0. 0. 0. ]\n", 333 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.08 1. 1. 0.42 0. 0. 0. 0. 0. 0. 0.\n", 334 | " 0. 0. 0. 0. 0. ]\n", 335 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.06 0.94 1. 0.56 0. 0. 0. 0. 0. 0. 0.\n", 336 | " 0. 0. 0. 0. 0. ]\n", 337 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.7 1. 0.56 0. 0. 0. 0. 0. 0. 0.\n", 338 | " 0. 0. 0. 0. 0. ]\n", 339 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.7 1. 0.56 0. 0. 0. 0. 0. 0. 0.\n", 340 | " 0. 0. 0. 0. 0. ]\n", 341 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.7 1. 0.64 0. 0. 0. 0. 0. 0. 0.\n", 342 | " 0. 0. 0. 0. 0. ]\n", 343 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.7 1. 0.94 0. 0. 0. 0. 0. 0. 0.\n", 344 | " 0. 0. 0. 0. 0. ]\n", 345 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.44 1. 0.94 0. 0. 0. 0. 0. 0. 0.\n", 346 | " 0. 0. 0. 0. 0. ]\n", 347 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.33 1. 0.96 0.12 0. 0. 0. 0. 0. 0.\n", 348 | " 0. 0. 0. 0. 0. ]\n", 349 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.31 1. 0.96 0.15 0. 0. 0. 0. 0. 0.\n", 350 | " 0. 0. 0. 0. 0. ]\n", 351 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.84 1. 0.59 0. 0. 0. 0. 0. 0.\n", 352 | " 0. 0. 0. 0. 0. ]\n", 353 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.56 0.95 0.03 0. 0. 0. 0. 0. 0.\n", 354 | " 0. 0. 0. 0. 0. ]\n", 355 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.56 0.94 0.01 0. 0. 0. 0. 0. 0.\n", 356 | " 0. 0. 0. 0. 0. ]\n", 357 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.56 1. 0.32 0. 0. 0. 0. 0. 0.\n", 358 | " 0. 0. 0. 0. 0. ]\n", 359 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.9 0.97 0.16 0. 0. 0. 0. 0. 0.\n", 360 | " 0. 0. 0. 0. 0. ]\n", 361 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.66 0.82 0.12 0. 0. 0. 0. 0. 0.\n", 362 | " 0. 0. 0. 0. 0. ]\n", 363 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 364 | " 0. 0. 0. 0. 0. ]\n", 365 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 366 | " 0. 0. 0. 0. 0. ]\n", 367 | " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", 368 | " 0. 0. 0. 0. 0. ]]\n" 369 | ] 370 | } 371 | ], 372 | "source": [ 373 | "print(numpy.matrix(x_train[8]))" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": 33, 379 | "metadata": {}, 380 | "outputs": [ 381 | { 382 | "name": "stdout", 383 | "output_type": "stream", 384 | "text": [ 385 | "(60000, 784)\n", 386 | "(10000, 784)\n" 387 | ] 388 | } 389 | ], 390 | "source": [ 391 | "\n", 392 | "x_train = x_train.reshape(60000, 784)\n", 393 | "x_test = x_test.reshape(10000, 784)\n", 394 | "\n", 395 | "\n", 396 | "print(x_train.shape)\n", 397 | "print(x_test.shape)" 398 | ] 399 | }, 400 | { 401 | "cell_type": "code", 402 | "execution_count": 34, 403 | "metadata": {}, 404 | "outputs": [], 405 | "source": [ 406 | "from keras.utils import to_categorical\n", 407 | "\n", 408 | "y_train = keras.utils.to_categorical(y_train, num_classes=10)\n", 409 | "y_test = keras.utils.to_categorical(y_test, num_classes=10)" 410 | ] 411 | }, 412 | { 413 | "cell_type": "markdown", 414 | "metadata": {}, 415 | "source": [ 416 | "Definición del modelo" 417 | ] 418 | }, 419 | { 420 | "cell_type": "code", 421 | "execution_count": 35, 422 | "metadata": {}, 423 | "outputs": [ 424 | { 425 | "name": "stdout", 426 | "output_type": "stream", 427 | "text": [ 428 | "_________________________________________________________________\n", 429 | "Layer (type) Output Shape Param # \n", 430 | "=================================================================\n", 431 | "dense_3 (Dense) (None, 10) 7850 \n", 432 | "_________________________________________________________________\n", 433 | "dense_4 (Dense) (None, 10) 110 \n", 434 | "=================================================================\n", 435 | "Total params: 7,960\n", 436 | "Trainable params: 7,960\n", 437 | "Non-trainable params: 0\n", 438 | "_________________________________________________________________\n" 439 | ] 440 | } 441 | ], 442 | "source": [ 443 | "from keras.models import Sequential\n", 444 | "from keras.layers import Dense\n", 445 | "from keras.optimizers import sgd\n", 446 | "\n", 447 | "model = Sequential()\n", 448 | "model.add(Dense(10, activation='sigmoid', input_shape=(784,)))\n", 449 | "model.add(Dense(10, activation='softmax'))\n", 450 | "\n", 451 | "model.summary()" 452 | ] 453 | }, 454 | { 455 | "cell_type": "markdown", 456 | "metadata": {}, 457 | "source": [ 458 | "Aprendizaje, entrenamiento y evaluación" 459 | ] 460 | }, 461 | { 462 | "cell_type": "code", 463 | "execution_count": 36, 464 | "metadata": {}, 465 | "outputs": [ 466 | { 467 | "name": "stdout", 468 | "output_type": "stream", 469 | "text": [ 470 | "Epoch 1/10\n", 471 | "60000/60000 [==============================] - 2s 28us/step - loss: 2.0162 - acc: 0.4937\n", 472 | "Epoch 2/10\n", 473 | "60000/60000 [==============================] - 1s 24us/step - loss: 1.5334 - acc: 0.6890\n", 474 | "Epoch 3/10\n", 475 | "60000/60000 [==============================] - 1s 21us/step - loss: 1.1935 - acc: 0.7699\n", 476 | "Epoch 4/10\n", 477 | "60000/60000 [==============================] - 1s 21us/step - loss: 0.9736 - acc: 0.8175\n", 478 | "Epoch 5/10\n", 479 | "60000/60000 [==============================] - 1s 21us/step - loss: 0.8290 - acc: 0.8421\n", 480 | "Epoch 6/10\n", 481 | "60000/60000 [==============================] - 1s 21us/step - loss: 0.7286 - acc: 0.8558\n", 482 | "Epoch 7/10\n", 483 | "60000/60000 [==============================] - 1s 20us/step - loss: 0.6553 - acc: 0.8662\n", 484 | "Epoch 8/10\n", 485 | "60000/60000 [==============================] - 1s 21us/step - loss: 0.5999 - acc: 0.8735\n", 486 | "Epoch 9/10\n", 487 | "60000/60000 [==============================] - 1s 20us/step - loss: 0.5569 - acc: 0.8782\n", 488 | "Epoch 10/10\n", 489 | "60000/60000 [==============================] - 1s 21us/step - loss: 0.5229 - acc: 0.8822\n", 490 | "10000/10000 [==============================] - 0s 18us/step\n", 491 | "Test loss: 0.49781588258743287\n", 492 | "Test accuracy: 0.8895\n" 493 | ] 494 | } 495 | ], 496 | "source": [ 497 | "batch_size = 50\n", 498 | "num_classes = 10\n", 499 | "epochs=10\n", 500 | "\n", 501 | "model.compile(loss='categorical_crossentropy',\n", 502 | " optimizer='sgd',\n", 503 | " metrics=['accuracy'])\n", 504 | "\n", 505 | "model.fit(x_train, y_train,\n", 506 | " batch_size=batch_size,\n", 507 | " epochs=epochs,\n", 508 | " verbose=1\n", 509 | " )\n", 510 | "\n", 511 | "test_loss, test_acc = model.evaluate(x_test, y_test)\n", 512 | "\n", 513 | "print('Test loss:', test_loss)\n", 514 | "print('Test accuracy:', test_acc)" 515 | ] 516 | }, 517 | { 518 | "cell_type": "markdown", 519 | "metadata": {}, 520 | "source": [ 521 | "Predicciones" 522 | ] 523 | }, 524 | { 525 | "cell_type": "code", 526 | "execution_count": 37, 527 | "metadata": {}, 528 | "outputs": [], 529 | "source": [ 530 | "predictions = model.predict(x_test)" 531 | ] 532 | }, 533 | { 534 | "cell_type": "code", 535 | "execution_count": 38, 536 | "metadata": {}, 537 | "outputs": [ 538 | { 539 | "data": { 540 | "text/plain": [ 541 | "1.0000001" 542 | ] 543 | }, 544 | "execution_count": 38, 545 | "metadata": {}, 546 | "output_type": "execute_result" 547 | } 548 | ], 549 | "source": [ 550 | "import numpy as np\n", 551 | "np.sum(predictions[11])" 552 | ] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "execution_count": 39, 557 | "metadata": {}, 558 | "outputs": [ 559 | { 560 | "data": { 561 | "text/plain": [ 562 | "6" 563 | ] 564 | }, 565 | "execution_count": 39, 566 | "metadata": {}, 567 | "output_type": "execute_result" 568 | } 569 | ], 570 | "source": [ 571 | "np.argmax(predictions[11])" 572 | ] 573 | }, 574 | { 575 | "cell_type": "code", 576 | "execution_count": 49, 577 | "metadata": {}, 578 | "outputs": [], 579 | "source": [ 580 | "# Look at confusion matrix \n", 581 | "#Note, this code is taken straight from the SKLEARN website, an nice way of viewing confusion matrix.\n", 582 | "def plot_confusion_matrix(cm, classes,\n", 583 | " normalize=False,\n", 584 | " title='Confusion matrix',\n", 585 | " cmap=plt.cm.Blues):\n", 586 | " \"\"\"\n", 587 | " This function prints and plots the confusion matrix.\n", 588 | " Normalization can be applied by setting `normalize=True`.\n", 589 | " \"\"\"\n", 590 | " plt.imshow(cm, interpolation='nearest', cmap=cmap)\n", 591 | " plt.title(title)\n", 592 | " plt.colorbar()\n", 593 | " tick_marks = np.arange(len(classes))\n", 594 | " plt.xticks(tick_marks, classes, rotation=45)\n", 595 | " plt.yticks(tick_marks, classes)\n", 596 | "\n", 597 | " if normalize:\n", 598 | " cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n", 599 | "\n", 600 | " thresh = cm.max() / 2.\n", 601 | " for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n", 602 | " plt.text(j, i, cm[i, j],\n", 603 | " horizontalalignment=\"center\",\n", 604 | " color=\"white\" if cm[i, j] > thresh else \"black\")\n", 605 | "\n", 606 | " plt.tight_layout()\n", 607 | " plt.ylabel('Observación')\n", 608 | " plt.xlabel('Predicción')\n", 609 | "\n" 610 | ] 611 | }, 612 | { 613 | "cell_type": "code", 614 | "execution_count": 50, 615 | "metadata": {}, 616 | "outputs": [ 617 | { 618 | "data": { 619 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVEAAAEmCAYAAADbUaM7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXd8FcXXh5+TBAIISg0loZck1PQgvfemdKkCYv29KhYURQRFkaKCiIiVpmCn916kE5EqICAJNfSewrx/7E24QMotu5CEefjsh7tlzszunZw7O+V8RSmFRqPRaFzD434XQKPRaDIz2olqNBqNG2gnqtFoNG6gnahGo9G4gXaiGo1G4wbaiWo0Go0baCf6ACEiOUVkjohcEJGf3bDTTUQWm1m2+4WI1BaRffe7HJrMi+h5ohkPEXkCGAAEAJeAKGC4Umqtm3Z7AP8DaiilEtwuaAZHRBRQXil14H6XRZN10S3RDIaIDAA+BT4ACgMlgAlAWxPMlwT+eRAcqCOIiNf9LoMmC6CU0lsG2YBHgMtAxzSu8cZwssds26eAt+1cPSAaeAU4BRwHnrSdGwrEAfG2PPoC7wLT7GyXAhTgZdvvDfyL0Ro+BHSzO77WLl0NYDNwwfZ/DbtzK4H3gHU2O4uBgqncW1L5X7crfzugBfAPcBYYZHd9BPAncN527Xggu+3catu9XLHdb2c7+wOBE8DUpGO2NGVteYTY9osBp4F697tu6C3jbrolmrF4FMgB/J7GNW8B1YEgoBqGI3nb7nwRDGfsi+EoPxeRfEqpIRit25lKqdxKqW/SKoiIPASMA5orpfJgOMqoFK7LD8yzXVsA+BiYJyIF7C57AngS8AGyA6+mkXURjGfgC7wDfAV0B0KB2sBgESltuzYReBkoiPHsGgLPASil6tiuqWa735l29vNjtMr722eslDqI4WCniUgu4DtgslJqZRrl1TzgaCeasSgAxKq0X7e7AcOUUqeUUqcxWpg97M7H287HK6XmY7TC/F0sz02gsojkVEodV0rtSuGalsB+pdRUpVSCUupHYC/Q2u6a75RS/yilrgE/YfwApEY8Rv9vPDADw0GOVUpdsuW/G+PHA6XUVqXUBlu+h4EvgboO3NMQpdQNW3luQyn1FXAA2AgUxfjR0mhSRTvRjMUZoGA6fXXFgCN2+0dsx5Jt3OGErwK5nS2IUuoKxivwM8BxEZknIgEOlCepTL52+yecKM8ZpVSi7XOSkztpd/5aUnoRqSAic0XkhIhcxGhpF0zDNsBppdT1dK75CqgMfKaUupHOtZoHHO1EMxZ/Ajcw+gFT4xjGq2gSJWzHXOEKkMtuv4j9SaXUIqVUY4wW2V4M55JeeZLKFONimZzhC4xylVdKPQwMAiSdNGlORxGR3Bj9zN8A79q6KzSaVNFONAOhlLqA0Q/4uYi0E5FcIpJNRJqLyEjbZT8Cb4tIIREpaLt+motZRgF1RKSEiDwCvJl0QkQKi0hbW9/oDYxugZsp2JgPVBCRJ0TES0Q6AxWBuS6WyRnyABeBy7ZW8rN3nD8JlHHS5lhgi1KqH0Zf70S3S6nJ0mgnmsFQSo3BmCP6NsbI8FHgBeAP2yXvA1uAHcDfwDbbMVfyWgLMtNnayu2Oz8NWjmMYI9Z1udtJoZQ6A7TCmBFwBmNkvZVSKtaVMjnJqxiDVpcwWskz7zj/LjBZRM6LSKf0jIlIW6AZt+5zABAiIt1MK7Emy6En22s0Go0b6JaoRqPRuIF2ohqNRuMG2olqNBqNG2gnqtFoNG6QoQIwiHce5fFQenOlXSOodIH0L3IRq4fm0pv4mFHJrEOWmfV5W8mRI4eJjY019dF4PlxSqYS7Fo2lirp2epFSqpmZZTCDDOVEPR4qSM5GQyyxve6H3pbYBSOIi5WIZM4/65s3rXsuVj5xT4/M+bytpGZkmOk2VcI1vP3TnXmWzPWoz61pYblJhnKiGo3mQUJAMn+PonaiGo3m/iBAJn3Lskc7UY1Gc//IAi3RDHsHzzUPZNPotmwe05bnWlS87dz/WlXi8k+9KZDH+7bjIWULcP7HnrSLvDMehuMsXrSQqpX8qRRQjlEjR7hsJyUCypcmPLgqkWHB1Kwebprd69evU+vRCCJCqhFSrRLvDTW3X9nsZ/JM/z6U9CtMWHCV5GOD3niN4CqBRIRWo0vHxzl//rxLtp/t34dSfoUJt7OdxLhPxpDb24PYWPdXpD7drw8livkQGlTZbVv32r6Vddw5BDw8Hd8yKBnSiVYsnpfeDStQd9Bcqr82m+YhfpQpnAcA3wK5aFi1GP+dvnxbGg8R3usWxrK/XA1oBImJibz0f88za84Ctu/Yzc8zfmTP7t1u3cudLFiynI1btrNuw2bTbHp7e7NwyXI2bfuLjVuiWLxoIRs3bDDFthXPpHuP3vwxZ8Ftxxo0bMzm7X+zaetflCtfntEjP3TJdrcUbANEHz3KsqVLKF6ihEt276RHr97MmrvQFFv30v69qONOIeL4lkHJkE7U3/cRNh84zbW4RBJvKtbuOUEbW+vyo14RvD19C3cOiD/TPJBZG49w+mJ6oSJTZ/OmTZQtW47SZcqQPXt2Onbuwtw5s9y5lXuCiJA7txGiMz4+noT4eNNG9K14JrVq1yF/vtsjzDVq3AQvL6N3KSKyOjExrkXSq1W7Dvny3R29buBrA3j/w49Mey61atchf37rouRZZT9D1XHBeJ13dMugZMiS7T56nhoBhcmf25uc2T1pEuyHX4GHaBlWnGNnr7LzyLnbri+aLxdtIkrw1eK9buV77FgMfn7Fk/d9ff1c/mNOCRGhdYum1IgM45uvJ5lmF4wWRmRoECWK+dCgUWMiIiNNsWv1M0mJKd9/R5Om5k0HnDt7FsWKFaNK1Wqm2cys3I/vM3WcaIVm4JaopQNLItIMIz6jJ/C1UsqhDph9MRf4ZNZOZr3dmKvXE/j78Fm8s3nw6mNVafv+3XLnI3tHMHj61rtapxmNpSvW4Ovry6lTp2jdvAn+/gHUql0n/YQO4OnpycatUZw/f57OHR5j186dVKpsTX+dlYwcMRwvLy+6dDUn+tzVq1cZPfJDZs1bZIo9jclk4Bamo1jmREXEE/gcaIyhsLhZRGYrpRzqgJmyYj9TVuwHYEjXEE6fv0ar8BL8OcpQDvYtkIu1H7Wm7pvzCC5bgO9fNKR1CjzsTdNgXxJuKuZu/s+pMhcr5kt09NHk/ZiYaHx9fdNI4RxJtnx8fGjdth1bNm8yzYkmkTdvXurWq8/ixQtNcaJWPxN7pk75ngXz5zFv4VLTXrv//fcghw8f4tFwQ9YpJjqaWtVDWbV2I4WLFEknddbjXn6fDpGBW5iOYmVLNAI4oJT6F0BEZmBopzvkRAs9nIPTF6/jV+Ah2kaUpP5b85iwYE/y+V3jO1DnzTmcuXSDyi/8mnx84nO1WLj1qNMOFCAsPJwDB/Zz+NAhivn68vPMGXw/9Qen7aTElStXuHnzJnny5OHKlSssW7qEN98abIrt06dPky1bNvLmzcu1a9dYtnQJr7w20BTbVj4TexYvWsinY0axcOlKcuXKlX4CB6lcuQqHo29JNFWsUJrV6zdTsGCGXPxiOffq+3QMPdk+PXwxorInEQ3c1VEnIv2xSddKrlvr26e/Up/8ebyJT7jJgG82cOFqnIVFNfDy8uKTseNp3bIpiYmJ9Ordh4qVKpli+9TJk3Tp+DgACQkJdOrS1bR+vxPHj/NUn14kJiZyU92kfYdOtGjZyhTbVjyTXj2eYM3qlZyJjaV8meK8PfhdRo8cwY24G7Ru0QSAiIhIxn3uvDJHbzvbFcoU563B79Lryb5ulTclenbvyppVK4mNjaVsKT8GvzOU3n3My8cq+1bWcafJIpPtLYtsLyIdgGY2rRpEpAcQqZR6IbU0nvlLK6vWzsfqtfP3HL12PutQMzKMrVu3mPpgPPIUU97B/R2+/vqaoVuVUuYv4ncTK1uiMUBxu30/7o0CpEajyRQIeGbcSfSOYmWHxGagvIiUFpHsQBdgtoX5aTSazEQWmSdqWUtUKZUgIi8AizCmOH2rlNplVX4ajSYTkkm7quyxdJ6oUmo+hi65RqPR3EHWGJ3P/Heg0WgyLyauWBKRb0XklIjstDuWX0SWiMh+2//5bMdFRMaJyAER2SEiIXZpetmu3y8ivdLLVztRjUZz/zC3T/R74M55g28Ay5RS5YFltn2A5kB529Yf+AIMpwsMwZiOGQEMSXK8qaGdqEajuT840wp1oCWqlFoNnL3jcFtgsu3zZKCd3fEpymADkFdEigJNgSVKqbNKqXPAEu52zLehgzJrNJr7h3N9ogVFZIvd/iSlVHqRfAorpY7bPp8ACts+p7QYyDeN46minahGo7l/ODc6H+vOZHullBIR09dpZCgnGlS6gGWqnPnCU10o5TbnNo+3zHZmxkOv/EmRhMSbltn28sxMPXT3ZHT+pIgUVUodt72un7IdT20xUAxQ747jK9PKIDM9cY1Gk5UQ7oU8yGwgaYS9FzDL7nhP2yh9deCC7bV/EdBERPLZBpSa2I6lSoZqiWo0mgcJc1uiIvIjRiuyoIhEY4yyjwB+EpG+wBEgSeh+PtACOABcBZ4EUEqdFZH3MFZcAgxTSt05WHUb2olqNJr7h4krlpRSXVM51TCFaxXwfCp2vgW+dTRf7UQ1Gs39Q69Yuj+4Ivk6cUg3jiz7kC0/D0o+9nijYLb+8hZXto4jpOLtKpCv9mnCzllD+Ov3wTR6NDD5eOMagfz1+2B2zhrCq082drrsiYmJVA8L5vG25sT7vBe2rZYHtkrC9+jRozRtVJ/gqhUJqVaJ8ePGZljb0UeP0qJJQ8KCKhMeXIUJ48cB8NabrxNStSLVw4Lo2sl1KWl7Mo5kMllCYynTOVFXJV+nztlA2+c/v+3YroPH6PLKV6zddvC24wFlitCxaQghHYbT5vkJjH2zEx4egoeH8OkbnWj7wgSC279Px2ahBJRxTmJi/Lix+AcGpn+hC1hl20p5YCslfL28vBgxcgzbd+xm1doNfDnx8wxr28vLiw8+GsWWqJ0sX72eSRMnsHfPbho0aMSmbTvYsCWKcuUrMGaUe04vQ0kmi2SJKE4Zt2Sp4Krk67ptBzl74eptx/YdOsn+I6fuurZVvar8vGgbcfEJHDl2hoNHYwmvXIrwyqU4eDSWwzFniE9I5OdF22hVr6rDZY+Ojmbhgnk82aefw2kygm0r5YGtlPAtWrQowSHGkug8efIQEBDIsWPmhLQ123aRokUJCr5lzz8ggGMxMTS0k5IOj4jkWHS0W+XOUJLJoFui94N7IfnqW+gRok/ckmWOOXWOYj6PUMznEaJP2h0/eQ7fQo84bPe1V15i+Icj8fAw/7FbadtK7pWE75HDh4mK2k54hDlS0lbaPnL4MDuiogi7w97Uyd/R2E1JmYwlmWyoNji6ZVQs+4tLKaLKg8z8eXPxKeRDSGhoprKdFbh8+TJdO7Vn1JhPefjhhzO07cuXL9O9a0dGjP74NnujRnyAl5cXnU2Sks4IGBJLmd+JWjk6/z0wHphiptF7Ifkac/oCfkVuBW7x9cnHsVMXAPArbHe8cD5iTl9wyOaf69cxd+5sFi6cz43r17l48SJP9uzOd1OmuV1eK21bjdXfZ3x8PF07tadz1260e+xx0+xaYTs+Pp7uXTrQqcsTtG13y960Kd+zYME85i5Y4rYzyVCSySJIFljVZllLNJWIKm5jL/kaFxfHzzNn0LJVG1PzmLdyBx2bhpA9mxclixWgXIlCbN55mC27jlCuRCFKFitANi9POjYNYd7KHQ7ZfG/4hxw8HM2+A4eZMn0G9eo3MM3JWWnbaqz8PpVSPPNUX/wDAnnx5QGm2LTKtlKK55/uh39AIP978eXk40sWL+TTj0cz85c/TJGSvhd/P86gW6ImYC+ZXLxEiXSudl3ydfKHvakdWp6CeXNzYOF7vDdxPucuXOHjgR0pmC83v417hh37Ymjz/Ofs+fcEvy7ezvZf3yIh8SYvjfjJplypePmjn5gz4Xk8PYTJszaw598T7j6CDI+V8sBWSviuX7eOH6ZPpXLlKkSGBgEw9P0PaNa8RYaz/ef6dfz4wzQqVa5CjQhjgGnIsPd5fcBL3Lhxg7YtmwLG4NLY8V+4XO4MJZlM5lWytccyyWQAESkFzFVKOTTBMDQ0TK3buCX9C11AByDRZBQyYwASKySTPfOXVrmbDnP4+oszej5wkskajUaTOmLbMjnaiWo0mvuCkLH7Oh3FyilOPwJ/Av4iEm2LoqLRaDTJ6IGlNEgjoopGo9EAWWNgSb/OazSa+4Z2ohqNRuMqemBJo9FoXEeQTBfrISW0E9VoNPcN/Tqv0Wg07pD5fWjGcqIK61ZznN4wzhK7AKWe+8Uy2wAHPjM3cMa9wkr5XitX/VhNwk3rVgl6uSyKeR8Q3RLVaDQat9BOVKPRaNxAO1GNRqNxkayy7FM7UY1Gc//I/D40c2gspSYne/bsWdq0aEJQJX/atGjCuXPn0rF0N8/270vp4kWICLklOLfjryjq16lBjYgQ6tSIYMvmTU7Z7N+oPKvebczKIY35ol8E3l4e/PFaPZYObsTSwY2IGtmS7557FICm1Yqy/B3j+KJBDYgoV8ChPKyW2LXiuaSGmRK+ma2u3EliYiJ1qofR+XEjUPKRw4doVOdRQir706dHV+Li4tyyf/36dWo9GkFESDVCqlXivaFD3LLnFpI11s5nCieampzsx6M/om79hkTt2kfd+g35ePRHTtvu1qMXv8+ef9uxwYMG8uZbg1m/aRtvvfMugwe94bC9Inlz0K9BOZoOX0a9oUvw9BDahRen3aiVNHpvKY3eW8qWf88wf5shDrZm7ykaDDOOvzR5K2N6OqaTZLXErtnPJTXMlvDNTHUlJSZ+Po4KAQHJ++++/SbP/u8ltu3cxyN58zH1+2/dsu/t7c3CJcvZtO0vNm6JYvGihWzcsMEtm+6gneg9IjU52XlzZtOte08AunXvydzZzku/1qpdh3z5bpcDFhEuXbwIwMULFyhatKhTNj09hBzZPPH0EHJm9+LEhevJ53Ln8KKWvw8Loo4BcPVGYvK5XN6eOBoj22qJXSueS0qYLeGb2eqKPTHR0SxeOJ+evfsAhmTI6lUraPtYewC6du/B/LnuyRuLCLlz5wYMTaeE+Pj76qDEQxzeMiqZrk/UXk729KmTFLFV2sJFinD61ElT8hgx+hMea9Wct954nZvqJktXrHU47Ynz1/li8T9sHdGS6/GJrNx9klW7b5WreVAx1u49xeXrCbcdG/R4ZQrmyUH3zxzPK4m0JHbbd+jktL3UcOe5pEZKEr6bNm102y5k/LpyJ4NeH8DQ90dw+fIlAM6eOcMjj+RN/lEs5uvHsWPH3C5zYmIiNSJCOXjwAE8/+zwRkebLSDtKRm5hOoqV8USLi8gKEdktIrtE5EV3baYmJ2vLz7Qv5JtJExkxagx7Dx5hxMgxPP/MUw6nfSRXNpoFFSNi0HyqvT6XXN6etI+8pR31WEQJft989LY0C6KOUfudxTw5YT0D2zqnd3MvJXbdeS73msxQV+xZOH8uBQv5EBRivey1p6cnG7dGceBwNFs2b2LXzvujau7Mq3xGdrZWvs4nAK8opSoC1YHnRaSiq8ZSkpMt5FOYE8ePA3Di+HEKFvIxodjww7QptLHl8Vj7jmzd4vhgQZ1AH/6LvcKZy3EkJCrmb4shvKwxWJQ/d3aCSuVj6Y7jKabdsD+WkgUfIn/u7A7llZ7E7jffTzO18rnzXFLDCgnfzFJX7Nm4YT0L582hakBZ+vbsxppVK3jjtZe5cOE8CQnGW8uxmGiKFStmSrkB8ubNS9169Vm8eKFpNp1FO9E0UEodV0pts32+BOwBXPrrSE1OtkWr1kyfZsjaT582hZatzZF+LVK0GGtXrwJg1YrllC1X3uG00WevEVomPzmzG+vvagf4sP+40WfWKsSPpTuOcyPh1pLFUoUeSv5cpUResnt5cvZy+iOw90pi1x53nktqmC3hm5nqij1Dhn3ArgNH2LH3IN9MmU7tuvX56rup1K5Tj1m//wrAj9Om0ryle+U+ffp08oyNa9eusWzpEvz9A9JJZR1ZwYnekz5Rm+pnMHBXZ9dtksnFU5ZMTk1OdsCrA+nVrQtTv/+W4iVKMnn6DKfL9mSPJ1izZhVnYmPxL1uCQW8P4bMJXzLw1ZdJSEggR44cjPt8osP2th86y9ytMSx+uyGJiYq/j55n6ppDALQLL85nC/fedn2rED86PlqC+ETF9bhEnv7KsZFSqyV2zX4uqWG2hG9mqiuO8O77H9K35xMMH/oOVasF0cM26OQqJ44f56k+vUhMTOSmukn7Dp1o0bKVSaV1gYzrGx3GUslkABHJDawChiulfkvr2pDQMLV6vTlzD+8l5f6X5m25jQ5Acjc6AEnK5MhmTQQSKySTvQuXV77dxjp8/aFPWmZIyWRLpziJSDbgV2B6eg5Uo9E8YJg82V5EXrYNYu8UkR9FJIeIlBaRjSJyQERmikh227Xetv0DtvOlXL0NK0fnBfgG2KOU+tiqfDQaTeZEABHHtzRtifgC/weEKaUqA55AF+Aj4BOlVDngHJCkOtwXOGc7/ontOpewsiVaE+gBNBCRKNvWwsL8NBpNpkLw8HB8cwAvIKeIeAG5gONAAyAp4O9koJ3tc1vbPrbzDcXF0SsrJZPXkiW6jTUajVWYNequlIoRkdHAf8A1YDGwFTivlEpa2RLNrRlCvsBRW9oEEbkAFABinc07Uyz71Gg0WRAnXuVtvragiGyx2/onmxLJh9G6LA0UAx4Cmt2L28h0yz41Gk3WQMDR1/QkYtMYnW8EHFJKnQYQkd8wuhTzioiXrTXqB8TYro8BigPRttf/R4Azzt+FbolqNJr7iFkDSxiv8dVFJJetb7MhsBtYAXSwXdMLSIrgMtu2j+38cuXifE/dEtVoNPcNE/tEN4rIL8A2jCXn24FJwDxghoi8bzv2jS3JN8BUETkAnMUYyXcJ7UQ1Gs39wbEWpsMopYYAd0aZ/heISOHa60BHM/LVTlSj0dwXjHmimX8CT4Zzoh4WPVQnO7CdYv84a5dllrFQ1/6/L82LN3oncQnWLc3M7pV5l5R6WVgXrVrGbY3VjB1YxFEynBPVaDQPDlnAh2onqtFo7hNi7RvivUI7UY1Gc1/IKn2imWKe6DP9+1DSrzBhwVWSj509e5ZWzZtQtWIFWjV3TQL3To4ePUrTRvUJrlqRkGqVGD/O8TBdKRF99CgtmzYkPLgyESG35Hv/3vEXDevWpHpYNTq1b8NFm9CZIzzduAKrhzVl1bCmTOxfHW8vD2oH+rD0ncYsH9KYOW/Up7SPIUTmmz8Xv71Wj2VDGrPy3SY0rFLE6Xsw+5mAIdtbv1Z1akYEExlShQ/ee/e2868PeJFiBR9OObETmF12KyWTU7L9946/aFC3JpGh1ej4uHP1JC3Onz/PE507ElQ5kOAqFdm44U9T7LqCifNE7xuZwol279GbP+YsuO3YmFEjqNegATt2/0O9Bg1clga2x8vLixEjx7B9x25Wrd3AlxM/d1u+d/iIUWzevpNlq9bz1ZeGfO8Lz/Zn6PsfsGHLX7Ru046xn4x2yF6RvDnp17AcTd5bSt13FhlyzJElGNk9lGe/2kCDoUv4beN/vNwqEICXWwUye/NRGg5dQv8vN/BRd+f1e8x+JmDI9s5ZuJR1m7azduM2li5exOaNRjDqbVu3cP68+z+IYH7ZrZRMTsn2C8/2Z9h7H7Bxq62efOxYPUmP1wa8ROOmTYnauYeNW6PwDwg0xa4rZIXI9pnCidaqXYf8d0jVGhK4xoKDbt17uSSBeydFixYlOOSW3G5AQCDHjsWkkyp1UpTvPRbDwQP/ULNWHQDqN2jM7D8cD7Xq5elBjuxJcsyenDx/DYUiT45sRj45s3HivE2iWRn7AA/nysbJ89ecvgeznwncLdsbn2DI9iYmJvLOoIEMG+5yVLLbMLvsVkomp2T7wP5/qFnbqCcNGjZmlhP1JDUuXLjA2rWr6f2kEREue/bs5M2b1227rpIVWqKZtk/01KmTyRW2SJEinDJJAjeJI4cPExW1nfAIc+RkjxyxyfeGRxIQWIl5c2bRqk07/vjtF2Kij6ZvADhx/hoTFu1j+8iWXItPZNWuk6zcdZKXv9/CDy/V5npcIpeux9N8+DIARs7exU8D6tC3QTlyeXvRYcwq9+7BxGeSmJhI3Rrh/HvwAP2efo6wiEi+GD+O5i1bJ0sbm4nZ32cSVshIJxFQsRJz58yidZt2/O5EPUmLw4cOUbBgIZ7u14cdO/4iOCSE0R+P5aGHHko/sdmI7hNNE1tU6U0i8pct2vRQC/My9cu4fPkyXTu1Z9SYT++S23XVXo+uHRkxypDvnfDl13w16Qvq1Ajn0uVLZMvumLpnkhxz2MD5VH1ljuEYq5fgmcYVeOLTNQS9NpcZaw8zrHMQAI9HlmDmusMEvTaXJ8au4fN+ES7/opv9TDw9PVm7cRu7D/zHti2bWbd2NX/89gtPP/eC27bvxOyy22OljPSEL7/m6y+/oPaj4Vy+5Hg9SYuExASitm+j39PPsGHzNh566CFGj3S/K8wVzAzKfD+x8nX+BtBAKVUNCAKaiUh1s4z7+BTmuE0C9/jx4xQySQI3Pj6erp3a07lrN9o95v4k+vj4eLp37UCnzk8kS+tW8A9g1txFrF6/mQ6dulC6dFmHbNWpWNgmx3yDhETFvK3RRJQrSKXiedl26CwAf2w+Sng5Q6L5iVqlmWXTuN9y8Aw5snlSILe3S/dg5jOxJ2/evNSuW481q1by778HCK5UgSr+Zbh69SpBlSq4bd/KsoM1MtJJ+PsHMGveItb8uZkOnbtQpoxj9SQtfH398PXzI8LWIn/s8Q5ERW13265raN35NFEGl2272WybaQsfDAlcIzD19GmTTZHAVUrxzFN98Q8I5MWXB5hi7/ln+uHvH8gLdvK9p0+dAuDmzZuMGjGcvk/1T83EbcScuUpomQK35JgDC7Pv2EXy5MxGmcJGH2PdioXZf/yScf3Zq9SuWBiA8kWEYjfnAAAgAElEQVTz4J3Nk9hLN5y+BzOfCUDsHbK9K5YtJSg4hP2Hj/H3vn/5e9+/5MqVi6hd/7iVjxVlvxMrZKSTuK2efDicPv0cqydpUaRIEfz8ivPPvn0ArFi+jMDA+zmwlPlbopb2iYqIJ0Z06XLA50qpuySTHaFXjydYs3olZ2JjKV+mOG8PfpdXXnuDHk90Zsp3hgTu1B9mul3e9evW8cP0qVSuXIXIUOOVeOj7H9CsuWuqJhvWr2OGTb63ZqQxwPHO0Pc5eOAAX305AYA2bR+je88nHbK37dBZ5m6NZuk7jUm4qdj53zmmrv6X4+eu8e1zNVAKzl+J46XvNgMwZOZffNwrjGcaV0Apxf9963wryexnAnDixHGeeepJbiYmcvPmTR5r35FmLcyX7TW77FZKJqdk+8qVK0yaaKsn7R6jRy/H6kl6jPlkHE/26k58XBylSpfhy6+/NcWu02SRyfaWSyYDiEhe4Hfgf0qpnXecu6U7X6JE6N79hy0pg5VfVryFa8QByj6v187fSWZeO28lnhbV85rVw9lmsmRynuIBKuilrx2+fu2rtR88yeQklFLnMYKj3hWuXyk1SSkVppQKK1iw0L0ojkajySDoPtE0EJFCthYoIpITaAzstSo/jUaT+dB9omlTFJhs6xf1AH5SSs21MD+NRpPJyMgtTEexUjJ5BxBslX2NRpPJyeAtTEfJtCuWNBpN5kYetKDMIlIDKGWfRik1xYIyaTSaB4Qs4EMdc6IiMhUoC0QBibbDCtBOVKPRuIxVckD3EkdbomFARVd1mTUajSYlsoAPddiJ7gSKAMctLItGo3mAELFuccC9xFEnWhDYLSKbMAKLAKCUcn/BukajeWB5kAaW3rWyEEkImXMtrZentWW2cmlm8f7uxxxIjaOTOltm28qlmV6e1i7ks7JX7FpcYvoXucBNi8qcBXyoY05UKbVKRAoD4bZDm5RSp6wrlkajyeoIxjSnzE6qP7kiUsLucydgE9AR6ARsFJEO1hdPo9FkZTzE8S2jklZLNFJEOiqlxgBvAeFJrU8RKQQsBawLL6TRaLI2GTywiKOk2hJVSv0MnEi67o7X9zNppdVoNBpHyAoBSNJ0hEqp6baPC0VkkYj0FpHewDxgfuoprePpfn0oUcyH0KDKmco2QED50oQHVyUyLJia1cPTT+AEixctpGolfyoFlGOUi5o5TzeuwJr3mrF6WDO+fNrQtK8V4MOyIU1YPawZ4/tGJE9JaRZUjJVDm7Li3SYseacxkeULupanyc88+uhRWjRpSFhQZcKDqzBh/DgAzp49S5sWTQiq5E+bFk04d849Webr169T69EIIkKqEVKtEu8NHWJG8ZMxWxs+qGI5akUEUffRUBrUNqRBhrw1kMjgytSODKZHlw5csKkN3CsEY7K9o1tGxaHWpFLqNWASUNW2TVJKDbSyYKnRo1dvZs1dmOlsJ7FgyXI2btnOug2bTbOZmJjIS//3PLPmLGD7jt38PONHp/XVi+TNyVONytN42BLqvLMQTw+hffWSjO8XyVMT11PnnYUcPXOVLjVLAbBmzynqDVlE/XcX8+K3m/ikt2s/CmY/cy8vLz74aBRbonayfPV6Jk2cwN49u/l49EfUrd+QqF37qFu/IR+Pdk+W2dvbm4VLlrNp219s3BLF4kUL2bhhg0l3YY02/Kz5S1n151aWrzEEJuo1aMS6zVGs2bidsuXL88kYc6SqnSHLt0TtUUr9qpQaYNt+t7JQaVGrdh3y58+f/oUZzLaVbN60ibJly1G6TBmyZ89Ox85dmDtnltN2bte09+LqjQTiEm7y70lDKmvVrhO0CvUD4MqNhOR0uby9cHUGjNnPvEjRogQF39Ka9w8I4FhMDPPmzKZb954AdOvek7mznX8+9ogIuXMbulbx8fEkxMeb1r93r7Th6zdsjJeXMSwSFh7J8Zho0/NIjywflFlE1tr+vyQiF+22SyJy8d4UMesgIrRu0ZQakWF88/Uk0+weOxaDn1/x5H1fXz9iYmKcsnHi/DUmLNxL1KhW7PykDRevxvPH5qN4eQjVSuUDoHVYcYrlz5WcpkWIL+uHN+eHF2vz4nfmqVyaxZHDh9kRFUVYRCSnT51M1rMvXKQIp0+ddNt+YmIikaFBlCjmQ4NGjYmINEfT3l4bvnp4CM8+3Y8rV664ZVNE6NC2OQ1qRTD526/uOv/D1O9p2OQu4QlLSVqx5OiWUUmvT7SW7f88SqmH7bY8SimHBLxFxFNEtovIAx+QeemKNfy5aSt/zJnPpC8msHbN6vtdpGQeyZWNZsG+hA6cR5UBs8nl7UmH6iXp/+WfvN8lmEVvN+Ly9Xhu3rzV5Jy/LYYaby2g1/h1vPGYNf3IrnL58mW6d+3IiNEf36U1b1bLxtPTk41bozhwOJotmzexa+fO9BM5gBXa8POWrGTFus3M/G0u30z6gvVr1ySfGzPyQzw9vejY+Ql3i+404sSWUXHodV5EqotIHrv9PCLi6M/ui8AeVwqX1fD19QXAx8eH1m3bsWWzOa23YsV8iY4+mrwfExOdnJej1E3StL9k07TfFkN4uYJsOXiG1iOW0/T9pfz5z2kOnrx0V9o//zlNyUK5yZ87u9v3Ygbx8fF079KBTl2eoK1NE76QT2FOHDdCP5w4fpyChXxMyy9v3rzUrVefxYvN6du1Qhu+WDGjPhTy8aFl63Zs22r0yf8wbTKLF87jy2+n3JdXZjNf50Ukr4j8IiJ7RWSPiDwqIvlFZImI7Lf9n892rYjIOBE5ICI7RCTE1XtwtE/0C+Cy3f4V27E0ERE/oCXguKRfFuXKlStcunQp+fOypUuoWMmc1ltYeDgHDuzn8KFDxMXF8fPMGbRs5VxYg+izt2va1wn0Yf/xixTM4w0Y6pr/ax7I9ysOAlDaJ3dy2qol8uHt5cHZy3Gm3I87KKV4/ul++AcE8r8XX04+3qJVa6ZPMyI3Tp82hZat3Qv7cPr0ac7bRrOvXbvGsqVL8PcPcMtmEmZrw99Z91YsX0JgxUosW7KIzz4Zw/SZv5MrV650rJiPMTpv6mT7scBCpVQAUA2j8fYGsEwpVR5YZtsHaA6Ut239ccCfpYaja+fFPgyeUuqmiDiS9lPgdSBPahfcIZmcrsGe3buyZtVKYmNjKVvKj8HvDKV3n74OFCV9rLR96uRJunQ0WkUJCQl06tKVJk3N6YPy8vLik7Hjad2yKYmJifTq3YeKlSo5ZWPbv2eZs+Uoy4Y0ISFR8fd/55iy6iBvPlaFJtWK4eEB3684yNq9xnThVqF+dKpRioTEm1yLS+Spia5NwTH7mf+5fh0//jCNSpWrUCPCaFwMGfY+A14dSK9uXZj6/bcUL1GSydNnuJwHGK3Zp/r0IjExkZvqJu07dKJFy1Zu2bTHTG3406dO0rOrscAwISGR9p260LBxU8KqBnDjxg3atzHqYVh4JGPGTTCl/A5h4oCRiDwC1AF6Ayil4oA4EWkL1LNdNhlYCQwE2gJTbH5tg60VW1Qp5XSkOod050XkN1vmSd76OaC+UqpdGmlaAS2UUs+JSD3gVaVUmrUsNDRMrdu4xcGiZxysDrNq5WuWDkByNzoAyd00qB1J1LatplbEAmUqqRbv/eDw9dO6Bx0BYu0OTVJKTQIQkSCMaZi7MVqhWzG6EmOUUkmqwwKcU0rltY3RjFBKJQ2eLwMGKqWcdkCO1pZngBpADBANRGJrPaZBTaCNiBwGZgANRGSaswXUaDRZFyf7RGOVUmF2m/0UFy8gBPhCKRWM0eX4hn1etlan6b9gjkZxOgV0ccawUupN4E0Au5Zod2cLqNFosiZJfaImEQ1EK6U22vZ/wXCiJ5Ne00WkKJC0fD0GKG6X3s92zGkc1VjKAfQFKgE5ko4rpfq4kqlGo9GAeV1VSqkTInJURPyVUvuAhhiv9ruBXsAI2/9JqyxmAy+IyAyMN+sLrvSHguMDS1OBvUBTYBjQDSemLSmlVmL0qWo0Gg1gm2xvbn///4DpIpId+Bd4EqPL8icR6QscwQjlCUbsjxbAAeCq7VqXcNSJllNKdRSRtkqpySLyA7Am3VQajUaTBmb6UKVUFIao5p00TOFaBTxvRr6OOtF42//nRaQyRog882YrazSaB5KMvCbeURx1opNsM/0HY/Ql5LZ91mg0GpfJAj7UYSf6nVIqEVgFlLGwPBqN5gFByNhxQh3F0Xmih0Rkkog0lKzQ/tZoNPcfJ2KJZmSv42hLNABohdER+62IzAFmJM32NwuFdStRrAylZfXvynWLVqEAHJlonRxzw0+si1K18H+1LLNtH6nKCuIsXG2VFPvAbKxqMWaFNpmjke2vKqV+Uko9DgQBD2O82ms0Go3LeDixZVQcLpuI1BWRCRhrUnNwa76VRqPROI2QNSLbO7pi6TCwHfgJeE0p5V6YbY1GoyFj68k7SrpOVEQ8gW+VUsPuQXk0Gs0DQpI8SGYn3dd529Qm8wIlukBqMri///oz4cFVeDinF9u2mhNCz2yp2iSskthNTEykzqNhdG5vBBlu3rgutauHUrt6KIFli9Ot8+Mu2X2mfx9K+hUmLLhK8rHffv2ZsKDK5M7h6dTzLpEvJ9/3CkneFv9fDTqF+vJ83dL80CeMyb1D+KBdRXJ7G4MiD+fw4rPOVVnyYk0GNCzrVLmjjx6lZdOGhAdXJiLkVl0BmDhhPKHVKhIRUoXBg9wXq/38s7GEBVchLKgy48d96rY9sH2f1cPo/LjxfU764nNCKvuTL5cXZ2Jj00mdPv/s20dkWHDyVrjAI6aV3RVMDsp8X3B0dH6diIwHZmKEmAJAKbXNklLdQZIMblBwCJcuXaL2o+E0aNiIwEqVmT7zF158/lnT8kqSqv1h5s/ExcVx9epVU+wmSezmzp2b+Ph4GtStRZOmzYmsXt0tuxM/H0cF/wAuXTJ0AxcsuTXe1/OJjrRo6VoE9+49evP0sy/wVJ9eyccqVqzMDzN/5f9eeMYpW/+du0bvyUZV8RD449nqrNofS8n8uZi4+hCJCp6tU5oekSX4YvUh4hJv8tXaw5Qp+BBlCjoXcd3Ly4vhI27VlTo1jLpy6tRJ5s+dzfpN2/H29ub0qVPpG0uDXbt28t23X7N63UayZ89O21bNad6iFWXLlXPL7sTPx1EhIIBLF43vs/qjNWjWoiWtmt61ctElKvj7s3GLITWSmJhI2VJ+tGn7mCm2XSEDd3U6jKMDS0EYEZyGAWNs22irCnUnqcngBgQEUqGCv2n5WClVKxZI7MbERLN44Xx69r47mNbFixdZvWoFLVq3dcl2rdp1yJ/vdinjgMBAKvi797zDSuYj5vw1Tl68wabD50i0zSbadfwiPjYpkuvxN9kRc5G4BOenAqVYV47F8M2kibz86ut4ext5FPJxb9Xyvr17CI+IIFeuXHh5eVG7Th1m/fGbWzZjou/+PqsGBVOiZCm37KbGiuXLKFOmLCVKlrTEfnoYofDE4S2j4ugUp/opbA2sLlxK2Mvgmo0VUrX2mC2xO+j1AQwdPgIPj7u/xvlzZlG3XoO7lC7vNw0DCrF0z+m7jresXIQ/D501Na8jR2x1JTySAwf2s37dWurXfpTmjeuzdctmt2xXrFiZ9WvXcubMGa5evcqihQuIsRMLdIVBrw9g6Pspf59W8PNPM+jY2akwwabzwExxEpHCIvKNiCyw7Ve0hZZKL91hEflbRKJExO1Oy7RkcM3ACqlae8yU2F24YC4FC/kQFBya4vlffp5B+4739w/kTrw8hFplC7B83+1OtGf14iQqxeLd7r1i23P58mV6dO3IiFFGXUlISODc2bMsX72e9z74iN7du7gl0xEQGMiAV1+nTcumtGvdnKpVq+Hh6fpE94Xzbd9nSMrfp9nExcUxf+4cHm/f8Z7klxpZYcWSow7+e2ARUMy2/w/wkoNp6yulgpRSKYWocpiUZHDNxgqp2pQwQ2J345/rWThvDlUDy9K3VzfWrFpB/z49ATgTG8u2rZtp0qyFWUU2hepl8vPPqcucuxqffKxFpcLULFuAoXP3mpZPfHw83bt2oFPnJ2hjqyvFfH1p0+4xRISw8AjEw8PtgZpeT/Zl3YYtLF62irz58lG+fAWXbW3cYPs+A8rSt+ft36cVLFq4gKDgEAoXLmxZHukhTrzKZ/rXeaCgUuon4CaAUioBsG4t4h2kJoNrNmZL1dpjtsTukGEfsGv/EXbsOcg3k6dTu259Jn1rSALP+uNXmjZrSY4cOdKxcm9pHFCIJXtutTYjS+XjiQg/Bv62ixsu9H+mhFKK55/ph79/IC/Y1ZVWrduyetVKAPbv/4f4uDgKFCzoVl6nbINTR//7j9l//E6nLk+4bGvIsA/YdeAIO/Ye5Jspt3+fVvDzzPv/Kg9ZoyXq6Oj8FREpgE3kSUSqAxccSKeAxSKigC/vEJbCZuuWZHLxlCWTU5PBvXHjBq8NeJHY06fp8Fhrqlatxh9zXW/dgblStfZYLbFrz2+/zOSlAa+7ZaNXjydYs3olZ2JjKV+mOG8Pfpd8+fPzysv/R+zp0zzerhVVqwYxe55jzztHNg/CS+Vj5OL9yccGNCpHNk8PPu1kTKPadewio5YcAOCX/hE8lN0TL08PapcvyMs//83hM+nPlNiwfh0zbHWlZqRRV94Z+j49evXhuaf7EhlalezZszPx6+/cHtjr1qUDZ8+cwStbNj4eO960QUh7vpzwGeM+Hs3JkyeoFRFM46bNGffFXX9GTnHlyhWWL1vCZxMmmlRK18nIU5ccxVHJ5BDgM6AysBMoBHRQSu1IJ52vUipGRHyAJcD/lFKpRqUICQ1Tq9dvcqb8DqMDkKRMdi/ruuwbj7VO/MDKACRWTwC3MgCJt0XfZ83q4WzbusXUB+NboYp6ZsLvDl//TuPyW93tFrQCR9U+t4lIXcAfY2bCPqVUfDrJUErF2P4/JSK/AxGAdaF9NBpN5iGDT6J3FEdH5zsCOZVSu4B2wExb6zStNA+JSJ6kz0ATjFasRqPRAEZgZkf/ZVQcbfsPVkpdEpFaGKJP3wBfpJOmMLBWRP4CNgHzlFLudVhqNJosQ5Lu/IOy7DOpU64l8JVSap6IvJ9WAqXUv0A1dwqn0WiyNhnZOTqKo040RkS+BBoDH4mINxl7EYFGo8kEZOQ4oY7iqCPshDHZvqlS6jyQH3jNslJpNJoszwP1Oq+UumoLzNxcRJoB65RSiy0tmUajydpk8En0juLo6Pw7wGSgAFAQ+E5E3rayYBqNJuuTFZZ9Oton2g2oppS6DiAiI4AoIM3BJY1Go0mNpNf5zI6jTvQYhjjdddu+NxBjSYkswsoObKsldnNYJIML1klUAyx+sbZltss894tlto9MtDaykbdYNyZrVT23yqpnBm5hOkqaTlREPsNY/34B2CUiS2ynGmHM/dRoNBqXMNQ+73cp3Ce9lmhSDNDdwDIMh5oArLCyUBqN5gEgg4+6O0p6TvQHYDjQBziC8eNRAvgOGGRt0TQaTVYnIw8YOUp6nTMjgXxAaaVUqFIqBCgDPAKMsrpwGo0m65L0Op/Z44mm50RbAf2VUpeSDiilLgLPYiwBvSekJpl89uxZ2rRoQlAlf9q0aMK5c+fcyufo0aM0bVSf4KoVCalWifHjxrplLyXZ4UFvvEZwlUAiQqvRpePjyYGa3eXpfn0oUcyH0KDKpth7tn9fShcvQkRI1eRjO/6Kon6dGtSICKFOjQi2bHatW/zZ/n0o5VeYcLvnMvy9dylf2o9Hw4N5NDyYRQvmO2Xz6cblWTW0CauGNmHiU5F4e3lQO8CHJYMbseydxsweWJ9SPg8B0LNuGVa+2yT5eIWieVy6j8WLFlK1kj+VAsoxykQZGatlja0qtytkhSlO6TlRpVIIOGrTord2SNqOJMnkLVE7Wb56PZMmTmDvnt18PPoj6tZvSNSufdSt35CPR3/kdj4jRo5h+47drFq7gS8nfs6e3btdtte9R2/+mLPgtmMNGjZm8/a/2bT1L8qVL8/okR+6VeYkevTqzSw3A1Lb061HL36ffbsjGzxoIG++NZj1m7bx1jvvMnjQGy7avvu5ALzwv5f4c/N2/ty8nabNHZc2KZI3B/0alKfp+0upO2QxHh5Cu4jifNQ9hOe+3kjDYUv4beN/vNyyIgC/bfyPeu8upuGwJXy+cC9DOwc5fQ+JiYm89H/PM2vOArbv2M3PM350q67YkyRrvHHLdtZv3ELOXLlMkzW2styuYHZLVEQ8RWS7iMy17ZcWkY0ickBEZopIdttxb9v+Adv5Uq7eQ3pOdLeI3CX0IiLdAfNEcdIhNcnkeXNm0627Ubxu3Xsyd/Yst/IpWrQowSG38gkICOTYMddncqUkO9yocRO8vIyu6IjI6sTEmDNTrFbtOuTPnz/9C52wl++OsotIsh76xQsXKFq0qGm23cXTU8iR3RNPDyFXdk9OnL+OAvLkyAbAwzmzcfL8NQAuX09ITpfL28slwbrNmzZRtmw5SpcpQ/bs2enYuQtz57hX/1LCbFnje1VuRxAsUft8Edhjt/8R8IlSqhxwDkgS2OwLnLMd/8R2nUukN7D0PPCbiPQBttqOhQE5AXN+Gp3EXjL59KmTFLH9IRcuUoTTp06amk9U1HbCLZBmTmLK99/RvmMny+ybzYjRn/BYq+a89cbr3FQ3Wbpiran2v5z4OT9Mn0pIaCgffDSGfPnyOZTuxPnrfLFoH9s+asW1+ERW7TrBqt0nGTB5C9NfrMX1uEQuXU+gxQfLktM8Wb8szzSuQDYvD9qPXuV0WY8di8HPr3jyvq+vH5s2bXTaTnqYLWt8r8rtEGLuvFYR8cPoZhwODBDDeAMgSfxqMvAuRhjPtrbPAL8A40VEUnrzTo80HbxSKkYpFQkMAw7btmFKqYikqPVpISJ5ReQXEdkrIntE5FFnC2hPWpLJImLaF3L58mW6dmrPqDGfWqbbPnLEcLy8vOjStZsl9q3gm0kTGTFqDHsPHmHEyDE8/8xTptnu1/9Z/t5zgD83b6dwkaIMGviKw2kfyZWNZkG+hL8xj2qvziGXtxftq5fg6cbl6TZ2LcGvz2PGukMM63wrMuN3Kw4SOWgB7/+yg5dbmSNGaDYZRdbYSsSJDSgoIlvstv53mPsUeB2boCbGMvXzNmFNgGjA1/bZFzgKycKbF2zXO41DrWSl1HKl1Ge2bVn6KZIZCyxUSgVgxBbdk871qZKSZHIhn8KcOH4cMITgChbycdX8bfl07dSezl270e4xa6SZp075ngXz5/Ht5GmZKhTYD9OmJEsQP9a+I1u3mLfeonDhwnh6euLh4cGTfZ5iy+bNDqetE1iY/2KvcOZyHAmJinnbYogoV5BKfnnZdugsALM2HyWs7N3qnr9vPkrzIN+7jqdHsWK+REcfTd6PiYnG19d5O2lhhazxvSi3owjgKeLwBsQqpcLstmTFPhFpBZxSSm1NLT+rsGz9mYg8AtTBiIKPUirOFkbPaVKTTG7RqjXTpxmystOnTaFl6zZulVkpxTNP9cU/IJAXXx7glq3UWLxoIZ+OGcVPv84iV65cluRhFUWKFmPtauPVd9WK5ZQtV94020k/hgBzZv1OxUqOzzKIOXuVkDL5yWlbHls70Id/jl0kT85slCmcG4C6FQuz/7jRn1vaJ3dy2sZVi/LvqUt3G02HsPBwDhzYz+FDh4iLi+PnmTNo2cq9+ncnVsga34tyO4OJA0s1gTa2aHMzMF7jxwJ5RSSp29KPW8vVY4DiRhnEC2Pa5hlX7sHRtfOuUBo4jRHxqRpGn+qLSqkr9he5I5k84NWB9OrWhanff0vxEiWZPH2GWwVev24dP0yfSuXKVYgMNUZsh77/Ac2cGCm2JyXZ4dEjR3Aj7gatWzQBICIiknGfuy9d27N7V9asWklsbCxlS/kx+J2h9O7TN/2EqfBkjydYs2YVZ2Jj8S9bgkFvD+GzCV8y8NWXSUhIIEeOHC6Xu7fdc6lQpjhvDX6XNatXseOvKESEkiVLOWV726GzzN0azZLBjUi8qfj7v/NMXf0vx85d5dtna3BTKc5fjePl74wFeH0blKN2oA8JiYoLV+P4v28db/Um4eXlxSdjx9O6ZVMSExPp1bsPFStVctpOalgla2x1uZ3DvC44pdSbwJsAIlIPeFUp1U1EfgY6YDjWXkDSKNps2/6ftvPLXekPBQclk10yLBIGbABqKqU2ishY4KJSanBqaayUTPbytC7og9UBSDwsXBtnZQASK7sqMnMAEqv+5sC6Z14zMoytJksml61YTX0w3fH5wF1C/BySTLZzoq1EpAyGA80PbAe6K6VuiEgOYCoQDJwFutgkjZzGypZoNBCtlEoa+vsFcG1ioUajyZJY4fSVUiuBlbbP/2JItd95zXXAlF9Ly5pnSqkTwFER8bcdaogRyESj0WgAp0fnMyRWtkQB/gdMt60S+Bd40uL8NBpNZsHkeaL3C0udqFIqCmNyvkaj0dxG0oqlzI7VLVGNRqNJFd0S1Wg0Gjd4EIIyazQajSUYr/OZ34tqJ6rRaO4bWeBtXjtRjUZzvxBEt0TNx9OiThIrV+ZYuAAFsHbFklXPG6wdNLByVVGh7pMtsw1welovy2zHJ1hTz62q4rolqtFoNC6i+0Q1Go3GHTK4AJ2jaCeq0WjuG9qJajQajRtkhYGlTLnqKqB8acKDqxIZFkzN6uFu2bJSjjn66FFaNm1IeHBlIkJu2e7dvQs1I0OoGRlCZf8y1IwMcesezJZ6vpPPxn5CaLXKhAVVoVf3J7h+/bppts2Wer4TM+SBn20eyMZRbdg0qi3PNTekRN7uFMSfH7Vm3YjW/DGoMUXy5QSgVsXCRH/blXUjjHMDH6+aluk0SUxMpHpYMI+3beWyDUi9HgJMnDCe0GoViQipwuBBA93Kx1kEY7K9o1tGJdO2RBcsWU7BgnfLPThLkhxzUMx/YqEAACAASURBVHAIly5dovaj4TRo2IhpUydTt35DXnltIGNGfcTHoz/iveHO/RF6eXkxfMQt23VqGLa/n3YrePSgga/y8COPuH0PI0aOITjEyKdGZCgNGzUmsGJFt+wCxMTEMOHzz9j21y5y5sxJ966d+fmnGfTo2dtt22BIPT/z3Av063OXqKzbJMkDz1uwBF8/P2pVD6dVqzZOPZdAv7z0blCeem/NIy7hJr+/2YiF26IZO2cX7/8UBcAzzQJ44/FqvPTNBgD+3HuSjiOXu13+8ePG4h8YmKyw6iqp1cNTp04yf+5s1m/ajre3N6dPnXK7zM6SkfXkHSVTtkTNxEo55hRt20kwK6X4/def6dDJPQkIs6We7yQhIYFr166RkJDA1WtXKVq0mGm2zZZ6tscMeWB/30fYciCWa3GJJN5UrN1zkjYRJbh0LT75moe8vUyfAhQdHc3CBfN4sk8/t22lVg+/mTSRl199HW9vbwAK+bivUeYs4sS/jEqmdKIiQusWTakRGcY3X09KP4GDWCnHfOSIzXb4LQnm9evW4FO4MOVM1CoyW+rZ19eXl15+Bf+yJSlTohiPPPwIjRo3McW21aQkDxwT49yPy56j56kR4EP+3N7kzO5J0yBffAs8BMA7nYPZ83kHOtUqw/CftieniShfiPUftebXNxoS4JfXpbK/9spLDP9wJB4e5v6J2tfDAwf2s37dWurXfpTmjeuzdYvzMinukFVe560UqvMXkSi77aKIvGSG7aUr1vDnpq38MWc+k76YwNo1q922aaUc8+XLl+nRtSMjRt1u+5efZtCho3lCZFZIPZ87d465c2az+59/OXgkhitXrvDj9Gmm2M4M7Dt2gU9m7+SPQY35/c3G7DhyjkSbHMywmdsJfP4Xflr7L/2bBgDw16GzVHzhV2oMnMOXC/fy4yv1nc5z/ry5+BTyISQ01NR7ubMeJiQkcO7sWZavXs97H3xE7+5dLJUuuRtn2qEZ14taGdl+n1IqSCkVBIQCV4HfzbCdJPHq4+ND67bt2LLZPV0mK+WY4+Pj6d61A506P5EsNwzGK/LsWb/zeIdObpXdPh8rpJ5XLFtKyVKlKFSoENmyZaNtu8fYsGG9afatxCx54CkrDlBn0FyaDV3I+Ss3OHD89j7KmWsP0TayJACXrsVz5YYhc744KoZsXh4UyOPtVH5/rl/H3Lmz8S9Xip7durByxXKe7Nnd6XLbk1I9LObrS5t2jyEihIVHIB4enImNdSsfp3BC6TMjd53eq9f5hsBBpdQRdw1duXKFS5cuJX9etnSJU/K6d2KlHLNSiuef6Ye/fyAv2NkGWLF8KRUqBODr5+dy2e3zsUrq2a9ECTZv3MjVq1dRSrFyxXICAgJNzcMqzJIHLvhwDgD8CjxEm/CS/LzuX8oWyZN8vmVYcf45dgEAn0dyJB8PLVsQD4Ezl244ld97wz/k4OFo9h04zJTpM6hXvwHfTXG99Z9aPWzVui2rV60EYP/+f4iPi6OACYO1zqDlQRynC/BjSiduk0wukbJksj2nTp6kS0fjlzQhIYFOXbrSpGkzlwtmpRzzhvXrmGGznTSN6Z2h79O0WQt+/XkmHTp1drnc9pgt9WxPREQk7R5vT42IULy8vKgWFEyffv3dtpuE2VLP9pglDzx9QD3y5/YmPvEmA77bwIWr8Xz+dE3KF3uYmzcVR2Ov8OLXxsh8u+ql6NfIn4SbN7kel8iT49zvanKX1Ophj159eO7pvkSGViV79uxM/Pq7exok2egTzcju0TEsk0xOzsDQVzoGVFJKpTk6ExIaptZtsKZzO9FCWWOru5GyeVn3wpAZ5XutRgcguZu6NSPYZrJk8v+3d97hVVTpH/+8JBiaSu+hk4RQQwsdCUVB2gLKAgYQFGXltyrsWlZYQKRYYJVVFAsKgo1mkCpNlhohhBaUIqD07tJbeH9/zCR7gUSSe2dIbjif57nPnXvuzPece2buO+ecOed9K1WN0E9nL0/z/vUr5ktTyOQ7zZ1oibYGNt7OgBoMhrsQ/7zP3sCdMKLdSKUrbzAY7m6yQnfe1QdLIpIbaAnMcjMfg8Hgn5gHS7dBVc8DBdzMw2Aw+DGZ2TqmEb9dO28wGPwbq4Xp/1bUGFGDwZAxZPJJ9GnFGFGDwZBhZAEbaoyowWDIQLKAFTVG1GAwZBCZ27FIWjFG1GAwZBhmTNQAuBu7HeBaojtL+cDdyc6Xrya6pu1mnR+Z4ryXfU8ihixyTXv9sJauaTtNZp//mVb80imzwWDIGiT56k3L6zY6wSKyXES2i0iCiDxrp+cXkcUisst+z2eni4iMF5HdIrJFRLwOdGaMqMFgyDAc9Cd6DRikquFAPeAZEQkHXgKWqmpFYKn9GSyfHhXtVz/gfW9/gzGiBoMhw3Bq2aeqHlbVjfb2WeAnoATQAUhyyzUZ6GhvdwCmqMU6IK+IFPPmN/idEd25YweRtSOSX0UK3M+749/2Wq9/v76UDS5K3Zr/C227dctmopo2JLJWdR7p1J4zXkZbfLpfH0qXLELtiKrJabNmTqd2jSrkyRHAxrgNXpc7tVDPSYx/exz35gjghJeeyt0sO1iROJvUr03XzpaTZFVlxLDB1K5eiciaVZg44d9e6aYWHnjrls00b9qQerWr82hn785p/359KFOyCHU86iSJ8f8aS56gbOmq7zIFczHr/+onv9YPbU7PhqUJLXovX/aPJObZBkzoGUHuoAAAGlQowIwB9Yh5tgEzBtQjslzaAvyldq3MnjmdOhFVuS9noM/n0yvSY0EtK1pQRDZ4vFJ0bCsiZYAIIBYooqqH7a+OAEXs7RLAfo/DDthp6cbvjGhIaCixG+KJ3RDPmtgN5MyVi/Yd/uS1Xo/oXsyeM/+GtAH9+/HqiFHExm2mXfuOvDPuLa+0H4vuzbffLbghLTy8Cl98PZNGjZt4XWb4X6jnDZu2sew/a/jwgwn8/NN2wPrTLFvyPcHBt3dynRFlB/jgvfGEhIYlf/7i88kcPHCAH+MTiN24jU5dvHNYnRQeeH38NpauWMNHE616GdC/H8NfG8W6DfY5/Vf6z2mPFOoErPpeumRxmpyKe7LvxAU6/Xstnf69li7vruXi1USWJBxlROfKjFu4kw7vrGFJwjH6NikLwOnzV+g/OZ4O76zh5enbeP3RW415SqR2rVSqXIVpX8+gYSPfz6e3pDPG0glVre3xuiVKpYjkAWYCz6nqDXdKtZznOu5A1++MqCfLly2lXLnylCpd2muNRo2bkC/fjXf03bt20tA2FFHNWxLzrXdOqBo1bkL+m7TDKlUiJDTUu8J6kFqoZ4CXXhjIiFGv++QU2c2yHzx4gO8Xzqdn7z7JaZM+/oAXXh6cHN3S2/C9qYUH/mX3zmRj0SyqJXO8OKcpXSsAL/59IK+N9q2+61UowP6TFzj0+yXKFMzF+r2nAViz+yQtK1uNp58On+W4HWpk19FzBGUPIHvA7fNM7VoJC6tESIjv59NbBGdjLIlIdiwDOk1Vk07w0aRuuv1+zE4/CAR7HF7STks3fm1Ep3/zFY90dS5aZhJh4ZWT45PPnjWDgwf23+aIjMUz1PPc72IoXrwEVatVz+hipco/XhjI8JFjbggHvHfvHmbN/IZmjSLp0vFhftm9y+d8PMMDh1WqzDz7nH7r4DmdOyeG4sWL+1zfbaoVZd6WIwDsPnqO5uHWTeTBqkUoljfHLfu3qlKEnw6d4Wpi+hpWntdKZsCpMVGx7mCfAD+p6jiPr+YASaEEegExHuk97af09YD/enT704Xb/kSft6cbbBORL0Xk1qvBS65cucL8ud/RqfMjTkkmM2Hix3w88X0a16/DubNnyX7PPY7n4RSeoZ4DAwMZ+8YYXvnn8IwuVqosXDCXgoUKUyPixnDAVy5fJkdQDpaviqXX408woP8TPuVzc3jgCRM/5qMP36dJgzqcPefMOb1w4QJvvTGawUNf9Ukne4AQVakwi7ZaRvSVmQl0qxfMjAH1yB0UyNWb5glXKJybQQ+FMHR2Qrry+aOw4BmGcw5FGwLRQJRHmPY2wBigpYjsAlrYnwHmA3uA3cBHwF+8/QmuTbYXkRLAX4FwVb0oIt9gBaz7zAn9RQsXUCOiJkWKFLn9zukkNDSMmHnWhOhdu3ayaOH82xyRMdwc6jlh21b27dtLgzoRgNVtblyvNj+sWkeRokUzuLQWsWvXsHDedyxetIDLly5x9uwZ+vXpSfESJWlnj223bd+RZ572PlhdSuGBQ0LDiJnrcU4X+H5O9+z5hX379lK/jhUc8OCBAzSqV4sVq2LTVd+NQwqy/dAZTp67AsDe4+d5YlIcYD18ahpaKHnfIvcF8e/oCF6avpX9py6mOY+UwoJnBpxa9qmqq0jd1DZPYX8FnnEib7e784FAThEJBHJhBaxzhOlfu9OVBzh+zBo2uX79Om+OHulodEunSCnUc+UqVdm7/wgJO/eQsHMPJUqUZOW6DZnGgAIMfXUUCbt+ZctPv/DJ5Gk0btqMDydNoU3b9qy0w/euXrmCChVCvNJPLTzwDed0zEj6Pun7Oa1SpSr7Dhxl+869bN+5lxIlS7JqXVy66/vh6sWYt/l/Pcn8ua1Wsgg83awcX8daQw/35gjkg961GLdwJ/G//p5m/dTCgmcGsknaX5kV14yoqh4E3gJ+Aw5jjTl8f/N+ItIvacrCiRPH06R9/vx5li1d7Mgd9fHo7jR/oCG7du4gtHwpJn/6CdO/+YoaVcKoWS2cosWLE93rca+0e0V3p1nTBuzauYOK5YKZ/OknzImZTcVywcSuW0unjm1p/7B34Z6TQj2v+GE5DerWpEHdmo62mN0se0o8P+hF5sTMokGdGrw69BXeeW+iVzpJ4YH/s2I5DSNr0jDSqpfp33xFRNUwalUPp1ix4jzWM/3ntHd0d6LsOgmx68RXcmYPoEHFAizediw57eHqRVkwqBHzBzbi2JnLzIqznnf0qF+KUgVy0j+qfPK0qCSD+0ekdq3MiZlNaPlS/Bi7li5/akfHts6dzzSTBeKDuBYy2V5eNRPoCvwOTAdmqOrU1I7x15DJbgfbuu5iWGM3y37FpfC94O7a+WwuN3tqD72lLeEYbq2db9LA+ZDJVavX1Fnfr07z/iFFc2XKkMlududbAHtV9biqXsUKVtfAxfwMBoM/kY7pTZnZ25ObRvQ3oJ6I5LKnHzTHWoplMBgMQJbozbv3dF5VY0VkBrARyzlAPHDLCgODwXAXk5mtYxpxO2TyUGCom3kYDAZ/xXi2NxgMBp/IzGOdacUYUYPBkCFk9rHOtGKMqMFgyDiygBU1RtRgMGQYbs+xvhMYI2owGDIM/zehxogaDIaMIpNPok8rmc6IurXCMTDAj12nurd6kksuhjUOyh7gmrabyz7dDPUMsGF4K9e0K/51tiu6J35Lu8OT9OH/VjTTGVGDwXB3kOTZ3t8xRtRgMGQYWcCGGiNqMBgyjqzQEvWLgcKUwveeOnWKtq1bUS08hLatW3H69Gmf89m/fz8PtmhGRLVwalavzLvj3/FZ0y3tlOrkHy/9nYiqlahbqzp/fqQTv//u/ThWjfAKNKpbg6b1axHV2IrHM+rVoTSOjKBp/Vp0bt+aw4e987HtdOjh1HjqiT6UKl6YWjWq+KwFcOnSJaIa16NhZE3q1arGqBHDAOv3VKtUgUaRtWgUWYstmzelW9uNOnmyeQWW/7MFy4Y0Z0LfOgQFWn/3FzuEs3J4S1YMbUHfZuWT968fUpDFr0Sx/J8tmDmwcbp/gzekM9pnpsQvjGhK4XvHvjmGB6Ki2LJ9Jw9ERTH2zTGpHJ12AgMDGfPGWOK3bGfFqnVM/OA9ftq+3WddN7RTqpOo5i1ZH7+VH+M2U6FiRd56Y7RPZY6Zv4QVa+NYtjIWgAHPDWJlbDwr1sbR6qE2vDX6Na90nQ49nBrRvXoTM3ehI1oAQUFBzFmwhNWxG1m5Lo6lixex/sd1AIwY9TqrYuNYFRtHteo10q3tdJ0UzZuDvs3K03r0MqJGLCVbNqFDnZJ0rV+a4vly0mTYYpoOX8K3Gw4AcF/O7IzuVoPeE9bS7NUl9Pvox3T/Bq/IAm6c/MKIphS+d953c+jxmBXEr8djvZg7JyalQ9NFsWLFiKj5v9CyYWGVOHTIqyiqrmunVCctWrYiMNAaoakbWY+DB50pexKewc0uXLjgdV/MzdDDN+eTP/+t+XiLiJAnTx7Aill09eo1x1pIbtRJYDYhR/YAArIJObMHcPT3S/RsWpZ/zfs5eRbMSTsE85/qBjM//hAHT1+8Id1tsoAN9Q8jmhLHjh2lWLFiABQtWpRjx446qv/rvn1s2hRPHRdCy7qpncSUzz6l1YPeh3sQEbp0aE1Uo7pMnvRRcvprw4ZQNbQsM77+kpcHD3OgpBZOhR52m8TERBpF1qJi6WI0a948OfTwiGFDaFA3gpdfGMjly84YIF/q5Mjvl3h/yS7Wj2rNptfbcPbSVVb8dIzSBXPTvnZJFrzcjKkDGlC2cG4AyhXOQ95c2ZkxsDELX25Gl0hnegN/hIi1Yimtr8yK2yGTn7XDJSeIyHMu5uNY6wWs0LLdHu3Mm2Pfdjy0rJvaSbwxZiSBgYH8uVsPrzXmLf6B5avX8/WsuXzy4fusWbUSgMHDRrB1x166dO3GxxMnOFJep0IP3wkCAgJYFRtHwq5fiduwnu0J2xg6fCTrNyWwfOU6Tp8+zdtj3/A5H1/r5P5c2XmwWjEiBy8k4sX55LonkE51gwkKDODy1URaj17OtFX7GBdtha4ODBCqlspH9Ltr6D5+Nc89HEa5wnl8/h23JQs0RV0zoiJSBXgSqAtUB9qKSAWn9AsXLsLhw1aExMOHD1OoUGFHdK9evUq3RzvTtVsPOv7J2dCybmon8fmUz1gwfx6TJk/16cZSvHgJAAoVLszD7TqyMe7G2FePdO3GdzHOTOz2DD0cHlI2OfTw0SNHHNF3g7x589K4yQMsXbyIosWKISIEBQXRI7oXGzf4HifM1zppHFaY/ScvcOrcFa5dV+bHH6J2+QIc/v0i8+OtB4ILNh2iUsn7ATh8+iIrth/l4pVETp2/QuyuE4Tb37lJFrChrrZEKwGxqnpBVa8BKwDHLEebtu2YNnUyANOmTubhdu191lRVnn6yL6FhlXj2+YE+690p7SS+X7SQt8e+yTczY8iVK5fXOufPn+fs2bPJ28uXLaZSeGV+2b0reZ/5c+dQMSTU5zKDc6GH3ebE8ePJMx4uXrzID8uWUDEklCP2zVxVmffdHCpVruxzXr7WycFTF6hZNj857VVjjcIKsfvwGRZuOkRDO459/ZCC7Dl6DoCFmw9Tp0KB5PHTiDL52HXkrM+/43ZkhRhLbs4T3QaMFJECwEWgDbDBG6Fe0d1Z+Z8fOHniBBXLBTN4yDAG/f0lort3ZcqnkwguVZrPv/ja5wKvWb2aL6Z9TpUqVYmsZT1hHf7aKB5q3SbTaadUJ2+9MYbLVy7Tro21rLBu3UjGv/dBurWPHztKz25dALh2LZHOj/6Z5i0fpFf3R9m9ayfZsgnBpUrz1jvveVX23h5lDykXzCtDhtHr8b5eaf0RPR/rxsoVP3DixAnKlynJkH8Op3cf7/M5cuQw/Z/sQ+L1RPT6dTp26sJDbdrSrnULTp44gapStVp1xo1P/zCH03USv+808zYeZNErUVxLvM62/f9l6qp95MgewLt96vBk8wqcv3yNv32+EYDdR87yQ8JRlg5pzvXryher97Hj0Bmv808bmXvqUlpxLWQygIj0Bf4CnAcSgMuq+txN+/QD+gEElypV6+dd+1wpi9thcN3kuovhns3a+Vtxe+28m34cXFs7P/MFrhzf7WilR9SsrctWxaZ5//y5A++6kMmo6ieqWktVmwCngZ0p7POhqtZW1doFCxZyszgGg8HgOK4u+xSRwqp6TERKYY2H1nMzP4PB4F9k5rHOtOL22vmZ9pjoVeAZVXXLn5bBYPBDssKYqNshk+/MAlyDweB3WJPtM7oUvmO8OBkMhozDGFGDwWDwHtOdNxgMBh/ICg+W/NYBicFg8H+cXPYpIg+JyA4R2S0iL7lU5FswRtRgMGQcDllREQkA3gNaA+FANxEJd6vYnhgjajAYMgwHPdvXBXar6h5VvQJ8BXRw/QeQycZE4zfGncgdlO3XNO5eEPA9hsSd13Zb32hnHW239dOjXdrpzOM3xi3KdY8UTMchOUTE0//Gh6r6ob1dAtjv8d0BwD2HvR5kKiOqqmle9ykiG9xaR+umttv6RjvraLut73bZb4eqeu81PBNhuvMGgyErcBAI9vhc0k5zHWNEDQZDVmA9UFFEyorIPcCfgTl3IuNM1Z1PJx/efpdMqe22vtHOOtpu67td9juGql4TkQHAIiAAmKSqCXcib1f9iRoMBkNWx3TnDQaDwQeMETUYDAYfMEbUkCbEyZjUdwgRye2idlF/rBOD8/iVERWRUBGpLyLZ7WVeTuu7EhRIRCqISG0RCXJBu7KINLWdXzut3UhEogFUVZ02GiLSTkSedVLTQ7sD8LqIOBNL+0btB4HZ3DilxinteiISbb/f47B2Rfs6zObWtX434jdGVEQ6ATHAa8AnwDMicp9D2iEAqpro9MUlIm2BWcCbwGdJeTmk3Rr4EngemCIijsQYtv9keYCJwMsi8jQkG1JHrhkRaQWMALY7oXeTdlPgdSBGVY85rN3K1i4GDHJYuz3WE/MWwN9wcJWQiHQEZgAvA+OAp9xsqd9N+IURFZHsQFegr6o2xzKmwcCLvhpS28htEpEvwFlDKiINsIxnL1VthhWszxHvMiLyAPAO8ISqdgSuAFWc0FbV66p6DpiMdcNqICLPJ33nq75dL58D/VR1sYjcLyKlRSSXr9o2tYCPbe3iItJSRCJF5H5fREWkBTAB6AFUBCqJSBMHyovdk3gG6K6qvYAzQA0RKSwiORzQfgropqqdgS3A48BAEbnXx6Lf9fiFEbW5D+vCBasrNRfIDnT3tptp34kHAM8BV0RkKjjeIn1dVePt7aFAfoe69UeBp1T1R7sFGgkMEJGJItLFoa73Nayb1WSgroiME5HRYuHLtXMSK+5WMfsP/i3wPlZL3YmyX/PYngH0wTrP74lIPh90A4Ce9vzD3MAOoDI4MmZ8DcgJhNkNgweAnsDbwGAfW43XgDxAUQBVnQTsw1o739YHXQOAqvrFC2iJtQKhsf05AOgOTMWe7+qlbnGsC6wg1h9uqoNlDgDu89guCcQDhey0Ag7l8wow2N7ujeXBppADuuWBl+ztQcAF4D2Hylwd2IPlKOJJrBt6H6zhifw+alfFMnBfAY/baeWAD4AHHSh7Nvv9IeAIUNWhOukCxAHrgCF2WhTwGVDdR+2n7f9KNDDS3n4K+MSJst/NL39qia4EvgeiRaSJqiaq6hdYRrC6t6KqekhVz6nqCayLKmdSi1REaopImA/aiap6xv4owO/AKVU9LiI9gNdEJKe3+h75jFTV1+ztz7Ba7U489LgIhIrIk1h/wjFAKRF5yldhVd2M1Qoao6ofqTWEMAnIB5TyUXsr1phiJFDWTtuDdSNLs5ObP9C/br8vxBrDbOtA6xxVnYE1HroS62aLqi4D7sX38dEvgQVAMyCnqj6mqhOBIk49W7hb8Ztln6p6SUSmAYr1sCMMuAwUAQ47lMdJ20C8KSI/Y/3pmjmkfQ04JyL7RWQ00AroraoXfdEVEVG7qWF/7oxVJ4d8KjDWDUZE9gNDsEJefycizYDdvmrb+tvxeLBkl70QzpzPBVjDJ8NEJMm9YgTWjcBJNmM92HtDVRN9FVPV0yKyDHhURK4AObBuBFt81P0vME1Evky6CYhITyA/4HO572oyuimc3hdwD5Zh+wqrmxPhQh7P42A3zdYUu+y/AL8BFR0ucxDQF0gAqjioGwzU8viczYX6Fqyu/HagssPaNYFRwFgnz+dNeXwDlHFQLy/wV2AF1lpwn7ryqeSRVN+u1Mnd9PLbtfP2gx9VB54W36SbD+tPMUhVfbr7p6LfG1ivDjtHsGcwtAR+UdUdTmrb+je0eJ3WBpoCR1T1ZzfycAM368TWvxdrvP/MbXdOv3ZpILuqOtKruJvxWyPqJiKSQ1UvuaTt6h/PYDDcWYwRNRgMBh/wp6fzBoPBkOkwRtRgMBh8wBhRg8Fg8AFjRA2ZFhEpIiK9MrocBsMfYYzoXYqIJIrIJhHZJiLTfXH+ISIPiMhce7u9iKTbyYqIPG1P/k76fC/WuvFl3pbLYLgTmKfzdykick5V89jb04A4VR3n8b1gXR+3nYdre5T6m6oaZxaGuw7TEjWAtVa7goiUEZEdIjIF2AYEi0grEVkrIhvtFmuS4X1IRH4WkY1ApyQhEektIu/a20VEZLaIbLZfDez0niKyxU773E4bJiJ/s7driMg6e5/ZSZ6XROQHEXldRH4UkZ0i0vhOVpLBkBLGiN7liEgg0BrYaidVBCaoamXgPDAYaKGqNYENWD4ocwAfAe2wfHem5gx6PLBCVatjLb9MEJHKtmaUnZ6SZ/spwIuqWs0u11CP7wJVtS6W+8KhKRxrMNxRjBG9e8kpIpuwDONvWM6XAX5V1XX2dj0gHFht79sLy5tQGLBXVXfZq6+mppJHFJafUNTyaPVfO226Wl6zUNVTngeI5Tg5r6qusJMmA56Oj2fZ73FAmXT/aoPBYfzGi5PBcS6qag3PBNuv8HnPJGCxqna7ab8bjrvDXLbfEzHXryETYFqihj9iHdBQRCqAFQlArBhRPwNlRKS8vV+3VI5fCvS3jw2wW5nLgEdsj/aISH7PA+zW6mmP8c5oLG9GBkOmxBhRQ6qo6nEsT/lfisgWYC0QZjtn6QfMsx8spRYM7lmgmYhsxep+h9veq0YCK0RkM1bQtJvpheXTdQtQA3jVwZ9lMDiKmeJkMBgMPmBaymfzagAAADdJREFUogaDweADxogaDAaDDxgjajAYDD5gjKjBYDD4gDGiBoPB4APGiBoMBoMPGCNqMBgMPvD/dJmW/Eqm4pEAAAAASUVORK5CYII=\n", 620 | "text/plain": [ 621 | "" 622 | ] 623 | }, 624 | "metadata": {}, 625 | "output_type": "display_data" 626 | } 627 | ], 628 | "source": [ 629 | "from collections import Counter\n", 630 | "from sklearn.metrics import confusion_matrix\n", 631 | "import itertools\n", 632 | "\n", 633 | "# Predict the values from the validation dataset\n", 634 | "Y_pred = model.predict(x_test)\n", 635 | "# Convert predictions classes to one hot vectors \n", 636 | "Y_pred_classes = np.argmax(Y_pred, axis = 1) \n", 637 | "# Convert validation observations to one hot vectors\n", 638 | "Y_true = np.argmax(y_test, axis = 1) \n", 639 | "# compute the confusion matrix\n", 640 | "confusion_mtx = confusion_matrix(Y_true, Y_pred_classes) \n", 641 | "# plot the confusion matrix\n", 642 | "plot_confusion_matrix(confusion_mtx, classes = range(10))" 643 | ] 644 | }, 645 | { 646 | "cell_type": "code", 647 | "execution_count": null, 648 | "metadata": {}, 649 | "outputs": [], 650 | "source": [] 651 | } 652 | ], 653 | "metadata": { 654 | "kernelspec": { 655 | "display_name": "Python 3", 656 | "language": "python", 657 | "name": "python3" 658 | }, 659 | "language_info": { 660 | "codemirror_mode": { 661 | "name": "ipython", 662 | "version": 3 663 | }, 664 | "file_extension": ".py", 665 | "mimetype": "text/x-python", 666 | "name": "python", 667 | "nbconvert_exporter": "python", 668 | "pygments_lexer": "ipython3", 669 | "version": "3.6.3" 670 | } 671 | }, 672 | "nbformat": 4, 673 | "nbformat_minor": 2 674 | } 675 | --------------------------------------------------------------------------------