├── C1W1_A_new_programming_paradigm_1.ipynb ├── C1W2_Introduction_to_Computer_Vision_1.ipynb ├── C1W3_Convolutional_Neural_Networks_1_Fashion_MNIST.ipynb ├── C1W3_Convolutional_Neural_Networks_2.ipynb ├── C1W4_Convolutional_Neural_Networks_1.ipynb ├── C2W1_Convolutional_Neural_Networks_1_Dogs_vs_Cats.ipynb ├── C2W2_Convolutional_Neural_Networks_1_Dogs_vs_Cats_Augmentation.ipynb ├── C2W2_Convolutional_Neural_Networks_3_Humans_vs_Horses_Augmentation.ipynb ├── C2W3_Inception_v3_Augmentation_Dropouts_Dogs_vs_Cats.ipynb ├── C2W4_Multiclass_Dropouts_RockPaperScissors.ipynb ├── C3W1_Tokenizer_stopwords_BBC_news_archive.ipynb ├── C3W2_Tokenizer_Embedding_1_imdb_reviews.ipynb ├── C3W2_Tokenizer_Embedding_2_BBC_news_archive.ipynb ├── C3W3_1_2_3_IMDB_Subwords_8K_with_Single_Layer_LSTM.ipynb ├── C3W3_4_5_Sarcasm_with_Bidirectional_LSTM.ipynb ├── C3W3_6_IMDB_Reviews_with_GRU_(and_LSTM_or_Conv1D).ipynb ├── C3W3_7_Overfitting_in_NLP_Transfer_Learning_GloVe_Twitter_API.ipynb ├── C3W4_1_Shakespeare_Tokenizer_Embedding_LSTM_Dropouts_Regularizers.ipynb ├── C4W1_1_Time_series_trend_seasonality_noise_autocor_impules_arima.ipynb ├── C4W1_2_TS_TSN_forecasting_naive_moving_avg_diff_mse_mae.ipynb ├── C4W1_3_TS_TSN_forecasting_naive_moving_avg_diff_mse_mae.ipynb ├── C4W2_1_Time_series_training_data_labels_preprocessing.ipynb ├── C4W2_2_Time_series_trend_seasonality_noise_Dense_mse.ipynb ├── C4W2_3_Time_series_TSN_Multi_Dense_learning_rate_mse.ipynb ├── C4W2_4_Time_series_TSN_Multi_Dense_learning_rate_mse.ipynb ├── C4W3_1_Time_series_TSN_Lambda_RNN_Dense_Huber_learning_rate_mae.ipynb ├── C4W3_2_Time_series_TSN_Lambda_LSTM_Dense_Huber_learning_rate_mae.ipynb ├── C4W3_3_Time_series_TSN_Lambda_LSTM_Dense_Huber_learning_rate_mae_mse.ipynb ├── C4W4_1_Time_series_TSN_Conv1D_LSTM_Dense_Lambda_Huber_learning_rate_mae.ipynb ├── C4W4_2_Sunspots_Conv1D_Multi_LSTM_Dense_Lambda_Huber_learning_rate_mae.ipynb ├── C4W4_3_Daily_min_temp_Conv1D_Multi_LSTM_Dense_Lambda_Huber_learning_rate_mae.ipynb ├── Presentations ├── Deep-Learning-Adventures-Chapter-1-Presentation-1.pdf ├── Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-2.pdf ├── Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-3.pdf ├── Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-4.pdf ├── Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-5.pdf ├── Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-6.pdf └── Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-7.pdf ├── README.md └── images ├── DLA-0.5.png ├── DLA-0.png ├── DLA-1.png ├── DLA-10.png ├── DLA-11.png ├── DLA-12.png ├── DLA-2.png ├── DLA-3.png ├── DLA-4.png ├── DLA-5.png ├── DLA-6.png ├── DLA-7.png ├── DLA-8.png ├── DLA-9.png ├── DLA.png ├── TensorFlow-in-Practice-1.png ├── TensorFlow-in-Practice-2.png └── TensorFlow-in-Practice-3.png /C1W2_Introduction_to_Computer_Vision_1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "C1W2-Introduction-to-Computer-Vision-1", 7 | "provenance": [], 8 | "private_outputs": true, 9 | "collapsed_sections": [], 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "language": "python", 15 | "name": "python3" 16 | }, 17 | "language_info": { 18 | "codemirror_mode": { 19 | "name": "ipython", 20 | "version": 3 21 | }, 22 | "file_extension": ".py", 23 | "mimetype": "text/x-python", 24 | "name": "python", 25 | "nbconvert_exporter": "python", 26 | "pygments_lexer": "ipython3", 27 | "version": "3.7.5rc1" 28 | } 29 | }, 30 | "cells": [ 31 | { 32 | "cell_type": "markdown", 33 | "metadata": { 34 | "id": "view-in-github", 35 | "colab_type": "text" 36 | }, 37 | "source": [ 38 | "\"Open" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": { 44 | "colab_type": "text", 45 | "id": "rX8mhOLljYeM" 46 | }, 47 | "source": [ 48 | "##### Copyright 2019 The TensorFlow Authors." 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "metadata": { 54 | "cellView": "form", 55 | "colab_type": "code", 56 | "id": "BZSlp3DAjdYf", 57 | "colab": {} 58 | }, 59 | "source": [ 60 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 61 | "# you may not use this file except in compliance with the License.\n", 62 | "# You may obtain a copy of the License at\n", 63 | "#\n", 64 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 65 | "#\n", 66 | "# Unless required by applicable law or agreed to in writing, software\n", 67 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 68 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 69 | "# See the License for the specific language governing permissions and\n", 70 | "# limitations under the License." 71 | ], 72 | "execution_count": 0, 73 | "outputs": [] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": { 78 | "colab_type": "text", 79 | "id": "qnyTxjK_GbOD" 80 | }, 81 | "source": [ 82 | "# Beyond Hello World, A Computer Vision Example\n", 83 | "In the previous exercise you saw how to create a neural network that figured out the problem you were trying to solve. This gave an explicit example of learned behavior. Of course, in that instance, it was a bit of overkill because it would have been easier to write the function Y=2x-1 directly, instead of bothering with using Machine Learning to learn the relationship between X and Y for a fixed set of values, and extending that for all values.\n", 84 | "\n", 85 | "But what about a scenario where writing rules like that is much more difficult -- for example a computer vision problem? Let's take a look at a scenario where we can recognize different items of clothing, trained from a dataset containing 10 different types." 86 | ] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "metadata": { 91 | "colab_type": "text", 92 | "id": "H41FYgtlHPjW" 93 | }, 94 | "source": [ 95 | "## Start Coding\n", 96 | "\n", 97 | "Let's start with our import of TensorFlow" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "metadata": { 103 | "colab_type": "code", 104 | "id": "q3KzJyjv3rnA", 105 | "colab": {} 106 | }, 107 | "source": [ 108 | "import tensorflow as tf\n", 109 | "print(tf.__version__)\n", 110 | "from tensorflow import keras\n", 111 | "\n", 112 | "import numpy as np\n", 113 | "np.set_printoptions(linewidth=200)\n", 114 | "\n", 115 | "import matplotlib.pyplot as plt\n", 116 | "from datetime import datetime\n", 117 | "\n", 118 | "# Load the TensorBoard notebook extension.\n", 119 | "%load_ext tensorboard" 120 | ], 121 | "execution_count": 0, 122 | "outputs": [] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": { 127 | "colab_type": "text", 128 | "id": "n_n1U5do3u_F" 129 | }, 130 | "source": [ 131 | "The Fashion MNIST data is available directly in the tf.keras datasets API. You load it like this:" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "metadata": { 137 | "colab_type": "code", 138 | "id": "PmxkHFpt31bM", 139 | "colab": {} 140 | }, 141 | "source": [ 142 | "mnist = tf.keras.datasets.fashion_mnist" 143 | ], 144 | "execution_count": 0, 145 | "outputs": [] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "metadata": { 150 | "id": "s3UUOBWmvity", 151 | "colab_type": "code", 152 | "colab": {} 153 | }, 154 | "source": [ 155 | "mnist" 156 | ], 157 | "execution_count": 0, 158 | "outputs": [] 159 | }, 160 | { 161 | "cell_type": "markdown", 162 | "metadata": { 163 | "colab_type": "text", 164 | "id": "GuoLQQBT4E-_" 165 | }, 166 | "source": [ 167 | "Calling load_data on this object will give you two sets of two lists, these will be the training and testing values for the graphics that contain the clothing items and their labels.\n" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "metadata": { 173 | "colab_type": "code", 174 | "id": "BTdRgExe4TRB", 175 | "colab": {} 176 | }, 177 | "source": [ 178 | "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()" 179 | ], 180 | "execution_count": 0, 181 | "outputs": [] 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "metadata": { 186 | "colab_type": "text", 187 | "id": "rw395ROx4f5Q" 188 | }, 189 | "source": [ 190 | "What does these values look like? Let's print a training image, and a training label to see...Experiment with different indices in the array. For example, also take a look at index 42...that's a a different boot than the one at index 0\n" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "metadata": { 196 | "colab_type": "code", 197 | "id": "FPc9d3gJ3jWF", 198 | "colab": {} 199 | }, 200 | "source": [ 201 | "random_index = np.random.randint(0,59999)\n", 202 | "plt.imshow(training_images[random_index])\n", 203 | "print(training_labels[random_index])\n", 204 | "print(training_images[random_index])" 205 | ], 206 | "execution_count": 0, 207 | "outputs": [] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": { 212 | "colab_type": "text", 213 | "id": "3cbrdH225_nH" 214 | }, 215 | "source": [ 216 | "You'll notice that all of the values in the number are between 0 and 255. If we are training a neural network, for various reasons it's easier if we treat all values as between 0 and 1, a process called '**normalizing**'...and fortunately in Python it's easy to normalize a list like this without looping. You do it like this:" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "metadata": { 222 | "colab_type": "code", 223 | "id": "kRH19pWs6ZDn", 224 | "colab": {} 225 | }, 226 | "source": [ 227 | "training_images = training_images / 255.0\n", 228 | "test_images = test_images / 255.0" 229 | ], 230 | "execution_count": 0, 231 | "outputs": [] 232 | }, 233 | { 234 | "cell_type": "markdown", 235 | "metadata": { 236 | "colab_type": "text", 237 | "id": "3DkO0As46lRn" 238 | }, 239 | "source": [ 240 | "Now you might be wondering why there are 2 sets...training and testing -- remember we spoke about this in the intro? The idea is to have 1 set of data for training, and then another set of data...that the model hasn't yet seen...to see how good it would be at classifying values. After all, when you're done, you're going to want to try it out with data that it hadn't previously seen!" 241 | ] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "metadata": { 246 | "colab_type": "text", 247 | "id": "dIn7S9gf62ie" 248 | }, 249 | "source": [ 250 | "Let's now design the model. There's quite a few new concepts here, but don't worry, you'll get the hang of them. " 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "metadata": { 256 | "colab_type": "code", 257 | "id": "7mAyndG3kVlK", 258 | "colab": {} 259 | }, 260 | "source": [ 261 | "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(input_shape=(28,28)), \n", 262 | " tf.keras.layers.Dense(128, activation=tf.nn.relu), \n", 263 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)])" 264 | ], 265 | "execution_count": 0, 266 | "outputs": [] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": { 271 | "colab_type": "text", 272 | "id": "-lUcWaiX7MFj" 273 | }, 274 | "source": [ 275 | "**Sequential**: That defines a SEQUENCE of layers in the neural network\n", 276 | "\n", 277 | "**Flatten**: Remember earlier where our images were a square, when you printed them out? Flatten just takes that square and turns it into a 1 dimensional set.\n", 278 | "\n", 279 | "**Dense**: Adds a layer of neurons\n", 280 | "\n", 281 | "Each layer of neurons need an **activation function** to tell them what to do. There's lots of options, but just use these for now. \n", 282 | "\n", 283 | "**Relu** effectively means \"If X>0 return X, else return 0\" -- so what it does it it only passes values 0 or greater to the next layer in the network.\n", 284 | "\n", 285 | "**Softmax** takes a set of values, and effectively picks the biggest one, so, for example, if the output of the last layer looks like [0.1, 0.1, 0.05, 0.1, 9.5, 0.1, 0.05, 0.05, 0.05], it saves you from fishing through it looking for the biggest value, and turns it into [0,0,0,0,1,0,0,0,0] -- The goal is to save a lot of coding!\n" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": { 291 | "colab_type": "text", 292 | "id": "c8vbMCqb9Mh6" 293 | }, 294 | "source": [ 295 | "The next thing to do, now the model is defined, is to actually build it. You do this by compiling it with an optimizer and loss function as before -- and then you train it by calling **model.fit ** asking it to fit your training data to your training labels -- i.e. have it figure out the relationship between the training data and its actual labels, so in future if you have data that looks like the training data, then it can make a prediction for what that data would look like. " 296 | ] 297 | }, 298 | { 299 | "cell_type": "code", 300 | "metadata": { 301 | "colab_type": "code", 302 | "id": "BLMdl9aP8nQ0", 303 | "colab": {} 304 | }, 305 | "source": [ 306 | "model.compile(optimizer = tf.optimizers.Adam(),\n", 307 | " loss = 'sparse_categorical_crossentropy',\n", 308 | " metrics=['accuracy'])" 309 | ], 310 | "execution_count": 0, 311 | "outputs": [] 312 | }, 313 | { 314 | "cell_type": "markdown", 315 | "metadata": { 316 | "id": "1h5zMOlrzQKm", 317 | "colab_type": "text" 318 | }, 319 | "source": [ 320 | "## Train the model and log data\n", 321 | "https://www.tensorflow.org/tensorboard/tensorboard_in_notebooks \n", 322 | "https://www.tensorflow.org/tensorboard/graphs\n", 323 | "https://colab.research.google.com/github/tensorflow/tensorboard/blob/master/docs/scalars_and_keras.ipynb\n", 324 | "\n", 325 | "Before training, define the Keras TensorBoard callback, specifying the log directory. By passing this callback to Model.fit(), you ensure that graph data is logged for visualization in TensorBoard." 326 | ] 327 | }, 328 | { 329 | "cell_type": "code", 330 | "metadata": { 331 | "id": "95ZOANO2zQTz", 332 | "colab_type": "code", 333 | "colab": {} 334 | }, 335 | "source": [ 336 | "# Define the Keras TensorBoard callback.\n", 337 | "logdir = \"logs/scalars/\" + datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n", 338 | "tensorboard_callback = keras.callbacks.TensorBoard(log_dir=logdir)" 339 | ], 340 | "execution_count": 0, 341 | "outputs": [] 342 | }, 343 | { 344 | "cell_type": "code", 345 | "metadata": { 346 | "id": "IDTGLa-KyM33", 347 | "colab_type": "code", 348 | "colab": {} 349 | }, 350 | "source": [ 351 | "training_history = model.fit(training_images, \n", 352 | " training_labels, \n", 353 | " #batch_size = 64,\n", 354 | " #verbose = 0,\n", 355 | " epochs = 5,\n", 356 | " #validation_data = (test_images, test_labels), \n", 357 | " callbacks=[tensorboard_callback])" 358 | ], 359 | "execution_count": 0, 360 | "outputs": [] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "metadata": { 365 | "id": "0pmQOtjFCjT5", 366 | "colab_type": "code", 367 | "colab": {} 368 | }, 369 | "source": [ 370 | "print(\"Average training loss: \", np.average(training_history.history['loss']))\n", 371 | "print(\"Average training accuracy: \", np.average(training_history.history['accuracy']))" 372 | ], 373 | "execution_count": 0, 374 | "outputs": [] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "metadata": { 379 | "id": "uKFwiKB8yTYw", 380 | "colab_type": "code", 381 | "colab": {} 382 | }, 383 | "source": [ 384 | "model.summary()" 385 | ], 386 | "execution_count": 0, 387 | "outputs": [] 388 | }, 389 | { 390 | "cell_type": "markdown", 391 | "metadata": { 392 | "id": "YbZs5wUT0rz9", 393 | "colab_type": "text" 394 | }, 395 | "source": [ 396 | "## Op-level graph\n", 397 | "Start TensorBoard and wait a few seconds for the UI to load. Select the Graphs dashboard by tapping “Graphs” at the top.\n", 398 | "\n" 399 | ] 400 | }, 401 | { 402 | "cell_type": "code", 403 | "metadata": { 404 | "id": "qb-yOewF0r8n", 405 | "colab_type": "code", 406 | "colab": {} 407 | }, 408 | "source": [ 409 | "%tensorboard --logdir logs/scalars" 410 | ], 411 | "execution_count": 0, 412 | "outputs": [] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": { 417 | "colab_type": "text", 418 | "id": "-JJMsvSB-1UY" 419 | }, 420 | "source": [ 421 | "Once it's done training -- you should see an accuracy value at the end of the final epoch. It might look something like 0.9098. This tells you that your neural network is about 91% accurate in classifying the training data. I.E., it figured out a pattern match between the image and the labels that worked 91% of the time. Not great, but not bad considering it was only trained for 5 epochs and done quite quickly.\n", 422 | "\n", 423 | "But how would it work with unseen data? That's why we have the test images. We can call model.evaluate, and pass in the two sets, and it will report back the loss for each. Let's give it a try:" 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "metadata": { 429 | "colab_type": "code", 430 | "id": "WzlqsEzX9s5P", 431 | "colab": {} 432 | }, 433 | "source": [ 434 | "model.evaluate(test_images, test_labels)" 435 | ], 436 | "execution_count": 0, 437 | "outputs": [] 438 | }, 439 | { 440 | "cell_type": "markdown", 441 | "metadata": { 442 | "colab_type": "text", 443 | "id": "6tki-Aro_Uax" 444 | }, 445 | "source": [ 446 | "For me, that returned a accuracy of about .8838, which means it was about 88% accurate. As expected it probably would not do as well with *unseen* data as it did with data it was trained on! As you go through this course, you'll look at ways to improve this. \n", 447 | "\n", 448 | "To explore further, try the below exercises:\n" 449 | ] 450 | }, 451 | { 452 | "cell_type": "markdown", 453 | "metadata": { 454 | "colab_type": "text", 455 | "id": "htldZNWcIPSN" 456 | }, 457 | "source": [ 458 | "# Exploration Exercises" 459 | ] 460 | }, 461 | { 462 | "cell_type": "markdown", 463 | "metadata": { 464 | "colab_type": "text", 465 | "id": "rquQqIx4AaGR" 466 | }, 467 | "source": [ 468 | "###Exercise 1:\n", 469 | "For this first exercise run the below code: It creates a set of classifications for each of the test images, and then prints the first entry in the classifications. The output, after you run it is a list of numbers. Why do you think this is, and what do those numbers represent? " 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "metadata": { 475 | "colab_type": "code", 476 | "id": "RyEIki0z_hAD", 477 | "colab": {} 478 | }, 479 | "source": [ 480 | "classifications = model.predict(test_images)\n", 481 | "\n", 482 | "print(classifications[0])" 483 | ], 484 | "execution_count": 0, 485 | "outputs": [] 486 | }, 487 | { 488 | "cell_type": "markdown", 489 | "metadata": { 490 | "colab_type": "text", 491 | "id": "MdzqbQhRArzm" 492 | }, 493 | "source": [ 494 | "Hint: try running print(test_labels[0]) -- and you'll get a 9. Does that help you understand why this list looks the way it does? " 495 | ] 496 | }, 497 | { 498 | "cell_type": "code", 499 | "metadata": { 500 | "colab_type": "code", 501 | "id": "WnBGOrMiA1n5", 502 | "colab": {} 503 | }, 504 | "source": [ 505 | "print(test_labels[0])" 506 | ], 507 | "execution_count": 0, 508 | "outputs": [] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": { 513 | "colab_type": "text", 514 | "id": "uUs7eqr7uSvs" 515 | }, 516 | "source": [ 517 | "### What does this list represent?\n", 518 | "\n", 519 | "\n", 520 | "1. It's 10 random meaningless values\n", 521 | "2. It's the first 10 classifications that the computer made\n", 522 | "3. It's the probability that this item is each of the 10 classes\n" 523 | ] 524 | }, 525 | { 526 | "cell_type": "markdown", 527 | "metadata": { 528 | "colab_type": "text", 529 | "id": "wAbr92RTA67u" 530 | }, 531 | "source": [ 532 | "####Answer: \n", 533 | "The correct answer is (3)\n", 534 | "\n", 535 | "The output of the model is a list of 10 numbers. These numbers are a probability that the value being classified is the corresponding value, i.e. the first value in the list is the probability that the handwriting is of a '0', the next is a '1' etc. Notice that they are all VERY LOW probabilities.\n", 536 | "\n", 537 | "For the 7, the probability was .999+, i.e. the neural network is telling us that it's almost certainly a 7." 538 | ] 539 | }, 540 | { 541 | "cell_type": "markdown", 542 | "metadata": { 543 | "colab_type": "text", 544 | "id": "CD4kC6TBu-69" 545 | }, 546 | "source": [ 547 | "### How do you know that this list tells you that the item is an ankle boot?\n", 548 | "\n", 549 | "\n", 550 | "1. There's not enough information to answer that question\n", 551 | "2. The 10th element on the list is the biggest, and the ankle boot is labelled 9\n", 552 | "2. The ankle boot is label 9, and there are 0->9 elements in the list\n" 553 | ] 554 | }, 555 | { 556 | "cell_type": "markdown", 557 | "metadata": { 558 | "colab_type": "text", 559 | "id": "I-haLncrva5L" 560 | }, 561 | "source": [ 562 | "####Answer\n", 563 | "The correct answer is (2). Both the list and the labels are 0 based, so the ankle boot having label 9 means that it is the 10th of the 10 classes. The list having the 10th element being the highest value means that the Neural Network has predicted that the item it is classifying is most likely an ankle boot" 564 | ] 565 | }, 566 | { 567 | "cell_type": "markdown", 568 | "metadata": { 569 | "colab_type": "text", 570 | "id": "OgQSIfDSOWv6" 571 | }, 572 | "source": [ 573 | "##Exercise 2: \n", 574 | "Let's now look at the layers in your model. Experiment with different values for the dense layer with 512 neurons. What different results do you get for loss, training time etc? Why do you think that's the case? \n" 575 | ] 576 | }, 577 | { 578 | "cell_type": "code", 579 | "metadata": { 580 | "colab_type": "code", 581 | "id": "GSZSwV5UObQP", 582 | "colab": {} 583 | }, 584 | "source": [ 585 | "import tensorflow as tf\n", 586 | "print(tf.__version__)\n", 587 | "\n", 588 | "mnist = tf.keras.datasets.mnist\n", 589 | "\n", 590 | "(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n", 591 | "\n", 592 | "training_images = training_images/255.0\n", 593 | "test_images = test_images/255.0\n", 594 | "\n", 595 | "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(input_shape=(28,28)),\n", 596 | " tf.keras.layers.Dense(1024, activation=tf.nn.relu),\n", 597 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n", 598 | "\n", 599 | "model.compile(optimizer = 'adam',\n", 600 | " loss = 'sparse_categorical_crossentropy')\n", 601 | "\n", 602 | "model.fit(training_images, training_labels, epochs=5)\n", 603 | "\n", 604 | "model.evaluate(test_images, test_labels)\n", 605 | "\n", 606 | "classifications = model.predict(test_images)\n", 607 | "\n", 608 | "print(classifications[0])\n", 609 | "\n", 610 | "print(test_labels[0])" 611 | ], 612 | "execution_count": 0, 613 | "outputs": [] 614 | }, 615 | { 616 | "cell_type": "code", 617 | "metadata": { 618 | "id": "J7rDAC3IFg7O", 619 | "colab_type": "code", 620 | "colab": {} 621 | }, 622 | "source": [ 623 | "model.summary()" 624 | ], 625 | "execution_count": 0, 626 | "outputs": [] 627 | }, 628 | { 629 | "cell_type": "markdown", 630 | "metadata": { 631 | "colab_type": "text", 632 | "id": "bOOEnHZFv5cS" 633 | }, 634 | "source": [ 635 | "###Question 1. Increase to 1024 Neurons -- What's the impact?\n", 636 | "\n", 637 | "1. Training takes longer, but is more accurate\n", 638 | "2. Training takes longer, but no impact on accuracy\n", 639 | "3. Training takes the same time, but is more accurate\n" 640 | ] 641 | }, 642 | { 643 | "cell_type": "markdown", 644 | "metadata": { 645 | "colab_type": "text", 646 | "id": "U73MUP2lwrI2" 647 | }, 648 | "source": [ 649 | "####Answer\n", 650 | "The correct answer is (1) by adding more Neurons we have to do more calculations, slowing down the process, but in this case they have a good impact -- we do get more accurate. That doesn't mean it's always a case of 'more is better', you can hit the law of diminishing returns very quickly!" 651 | ] 652 | }, 653 | { 654 | "cell_type": "markdown", 655 | "metadata": { 656 | "colab_type": "text", 657 | "id": "WtWxK16hQxLN" 658 | }, 659 | "source": [ 660 | "##Exercise 3: \n", 661 | "\n", 662 | "What would happen if you remove the Flatten() layer. Why do you think that's the case? \n", 663 | "\n", 664 | "You get an error about the shape of the data. It may seem vague right now, but it reinforces the rule of thumb that the first layer in your network should be the same shape as your data. Right now our data is 28x28 images, and 28 layers of 28 neurons would be infeasible, so it makes more sense to 'flatten' that 28,28 into a 784x1. Instead of wriitng all the code to handle that ourselves, we add the Flatten() layer at the begining, and when the arrays are loaded into the model later, they'll automatically be flattened for us." 665 | ] 666 | }, 667 | { 668 | "cell_type": "code", 669 | "metadata": { 670 | "colab_type": "code", 671 | "id": "ExNxCwhcQ18S", 672 | "colab": {} 673 | }, 674 | "source": [ 675 | "import tensorflow as tf\n", 676 | "print(tf.__version__)\n", 677 | "\n", 678 | "mnist = tf.keras.datasets.mnist\n", 679 | "\n", 680 | "(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n", 681 | "\n", 682 | "training_images = training_images/255.0\n", 683 | "test_images = test_images/255.0\n", 684 | "\n", 685 | "model = tf.keras.models.Sequential([#tf.keras.layers.Flatten(),\n", 686 | " tf.keras.layers.Dense(64, activation=tf.nn.relu),\n", 687 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n", 688 | "\n", 689 | "model.compile(optimizer = 'adam',\n", 690 | " loss = 'sparse_categorical_crossentropy')\n", 691 | "\n", 692 | "model.fit(training_images, training_labels, epochs=5)\n", 693 | "\n", 694 | "model.evaluate(test_images, test_labels)\n", 695 | "\n", 696 | "classifications = model.predict(test_images)\n", 697 | "\n", 698 | "print(classifications[0])\n", 699 | "print(test_labels[0])" 700 | ], 701 | "execution_count": 0, 702 | "outputs": [] 703 | }, 704 | { 705 | "cell_type": "markdown", 706 | "metadata": { 707 | "colab_type": "text", 708 | "id": "VqoCR-ieSGDg" 709 | }, 710 | "source": [ 711 | "##Exercise 4: \n", 712 | "\n", 713 | "Consider the final (output) layers. Why are there 10 of them? What would happen if you had a different amount than 10? For example, try training the network with 5\n", 714 | "\n", 715 | "You get an error as soon as it finds an unexpected value. Another rule of thumb -- the number of neurons in the last layer should match the number of classes you are classifying for. In this case it's the digits 0-9, so there are 10 of them, hence you should have 10 neurons in your final layer." 716 | ] 717 | }, 718 | { 719 | "cell_type": "code", 720 | "metadata": { 721 | "colab_type": "code", 722 | "id": "MMckVntcSPvo", 723 | "colab": {} 724 | }, 725 | "source": [ 726 | "import tensorflow as tf\n", 727 | "print(tf.__version__)\n", 728 | "\n", 729 | "mnist = tf.keras.datasets.mnist\n", 730 | "\n", 731 | "(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n", 732 | "\n", 733 | "training_images = training_images/255.0\n", 734 | "test_images = test_images/255.0\n", 735 | "\n", 736 | "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n", 737 | " tf.keras.layers.Dense(64, activation=tf.nn.relu),\n", 738 | " tf.keras.layers.Dense(5, activation=tf.nn.softmax)])\n", 739 | "\n", 740 | "model.compile(optimizer = 'adam',\n", 741 | " loss = 'sparse_categorical_crossentropy')\n", 742 | "\n", 743 | "model.fit(training_images, training_labels, epochs=5)\n", 744 | "\n", 745 | "model.evaluate(test_images, test_labels)\n", 746 | "\n", 747 | "classifications = model.predict(test_images)\n", 748 | "\n", 749 | "print(classifications[0])\n", 750 | "print(test_labels[0])" 751 | ], 752 | "execution_count": 0, 753 | "outputs": [] 754 | }, 755 | { 756 | "cell_type": "markdown", 757 | "metadata": { 758 | "colab_type": "text", 759 | "id": "-0lF5MuvSuZF" 760 | }, 761 | "source": [ 762 | "##Exercise 5: \n", 763 | "\n", 764 | "Consider the effects of additional layers in the network. What will happen if you add another layer between the one with 512 and the final layer with 10. \n", 765 | "\n", 766 | "Ans: There isn't a significant impact -- because this is relatively simple data. For far more complex data (including color images to be classified as flowers that you'll see in the next lesson), extra layers are often necessary. " 767 | ] 768 | }, 769 | { 770 | "cell_type": "code", 771 | "metadata": { 772 | "colab_type": "code", 773 | "id": "b1YPa6UhS8Es", 774 | "colab": {} 775 | }, 776 | "source": [ 777 | "import tensorflow as tf\n", 778 | "print(tf.__version__)\n", 779 | "\n", 780 | "mnist = tf.keras.datasets.mnist\n", 781 | "\n", 782 | "(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n", 783 | "\n", 784 | "training_images = training_images/255.0\n", 785 | "test_images = test_images/255.0\n", 786 | "\n", 787 | "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n", 788 | " tf.keras.layers.Dense(512, activation=tf.nn.relu),\n", 789 | " tf.keras.layers.Dense(256, activation=tf.nn.relu),\n", 790 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n", 791 | "\n", 792 | "model.compile(optimizer = 'adam',\n", 793 | " loss = 'sparse_categorical_crossentropy')\n", 794 | "\n", 795 | "model.fit(training_images, training_labels, epochs=5)\n", 796 | "\n", 797 | "model.evaluate(test_images, test_labels)\n", 798 | "\n", 799 | "classifications = model.predict(test_images)\n", 800 | "\n", 801 | "print(classifications[0])\n", 802 | "print(test_labels[0])" 803 | ], 804 | "execution_count": 0, 805 | "outputs": [] 806 | }, 807 | { 808 | "cell_type": "code", 809 | "metadata": { 810 | "id": "LwMKfeE7Fkwo", 811 | "colab_type": "code", 812 | "colab": {} 813 | }, 814 | "source": [ 815 | "model.summary()" 816 | ], 817 | "execution_count": 0, 818 | "outputs": [] 819 | }, 820 | { 821 | "cell_type": "markdown", 822 | "metadata": { 823 | "colab_type": "text", 824 | "id": "Bql9fyaNUSFy" 825 | }, 826 | "source": [ 827 | "#Exercise 6: \n", 828 | "\n", 829 | "Consider the impact of training for more or less epochs. Why do you think that would be the case? \n", 830 | "\n", 831 | "Try 15 epochs -- you'll probably get a model with a much better loss than the one with 5\n", 832 | "Try 30 epochs -- you might see the loss value stops decreasing, and sometimes increases. This is a side effect of something called 'overfitting' which you can learn about [somewhere] and it's something you need to keep an eye out for when training neural networks. There's no point in wasting your time training if you aren't improving your loss, right! :)" 833 | ] 834 | }, 835 | { 836 | "cell_type": "code", 837 | "metadata": { 838 | "colab_type": "code", 839 | "id": "uE3esj9BURQe", 840 | "colab": {} 841 | }, 842 | "source": [ 843 | "import tensorflow as tf\n", 844 | "print(tf.__version__)\n", 845 | "\n", 846 | "mnist = tf.keras.datasets.mnist\n", 847 | "\n", 848 | "(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n", 849 | "\n", 850 | "training_images = training_images/255.0\n", 851 | "test_images = test_images/255.0\n", 852 | "\n", 853 | "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(input_shape=(28,28)),\n", 854 | " tf.keras.layers.Dense(128, activation=tf.nn.relu),\n", 855 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n", 856 | "\n", 857 | "model.compile(optimizer = 'adam',\n", 858 | " loss = 'sparse_categorical_crossentropy')\n", 859 | "\n", 860 | "model.fit(training_images, training_labels, epochs=15)\n", 861 | "\n", 862 | "model.evaluate(test_images, test_labels)\n", 863 | "\n", 864 | "classifications = model.predict(test_images)\n", 865 | "\n", 866 | "print(classifications[34])\n", 867 | "print(test_labels[34])" 868 | ], 869 | "execution_count": 0, 870 | "outputs": [] 871 | }, 872 | { 873 | "cell_type": "code", 874 | "metadata": { 875 | "id": "mNGWCunaFrNM", 876 | "colab_type": "code", 877 | "colab": {} 878 | }, 879 | "source": [ 880 | "model.summary()" 881 | ], 882 | "execution_count": 0, 883 | "outputs": [] 884 | }, 885 | { 886 | "cell_type": "markdown", 887 | "metadata": { 888 | "colab_type": "text", 889 | "id": "HS3vVkOgCDGZ" 890 | }, 891 | "source": [ 892 | "#Exercise 7: \n", 893 | "\n", 894 | "Before you trained, you normalized the data, going from values that were 0-255 to values that were 0-1. What would be the impact of removing that? Here's the complete code to give it a try. Why do you think you get different results? " 895 | ] 896 | }, 897 | { 898 | "cell_type": "code", 899 | "metadata": { 900 | "colab_type": "code", 901 | "id": "JDqNAqrpCNg0", 902 | "colab": {} 903 | }, 904 | "source": [ 905 | "import tensorflow as tf\n", 906 | "print(tf.__version__)\n", 907 | "\n", 908 | "mnist = tf.keras.datasets.mnist\n", 909 | "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n", 910 | "#training_images=training_images/255.0\n", 911 | "#test_images=test_images/255.0\n", 912 | "\n", 913 | "model = tf.keras.models.Sequential([\n", 914 | " tf.keras.layers.Flatten(input_shape=(28,28)),\n", 915 | " tf.keras.layers.Dense(512, activation=tf.nn.relu),\n", 916 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n", 917 | "])\n", 918 | "\n", 919 | "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')\n", 920 | "model.fit(training_images, training_labels, epochs=5)\n", 921 | "model.evaluate(test_images, test_labels)\n", 922 | "\n", 923 | "classifications = model.predict(test_images)\n", 924 | "print(classifications[0])\n", 925 | "print(test_labels[0])" 926 | ], 927 | "execution_count": 0, 928 | "outputs": [] 929 | }, 930 | { 931 | "cell_type": "code", 932 | "metadata": { 933 | "id": "vxbpA8XcGMKW", 934 | "colab_type": "code", 935 | "colab": {} 936 | }, 937 | "source": [ 938 | "model.summary()" 939 | ], 940 | "execution_count": 0, 941 | "outputs": [] 942 | }, 943 | { 944 | "cell_type": "markdown", 945 | "metadata": { 946 | "colab_type": "text", 947 | "id": "E7W2PT66ZBHQ" 948 | }, 949 | "source": [ 950 | "#Exercise 8: \n", 951 | "\n", 952 | "Earlier when you trained for extra epochs you had an issue where your loss might change. It might have taken a bit of time for you to wait for the training to do that, and you might have thought 'wouldn't it be nice if I could stop the training when I reach a desired value?' -- i.e. 95% accuracy might be enough for you, and if you reach that after 3 epochs, why sit around waiting for it to finish a lot more epochs....So how would you fix that? Like any other program...you have callbacks! Let's see them in action..." 953 | ] 954 | }, 955 | { 956 | "cell_type": "code", 957 | "metadata": { 958 | "colab_type": "code", 959 | "id": "pkaEHHgqZbYv", 960 | "colab": {} 961 | }, 962 | "source": [ 963 | "import tensorflow as tf\n", 964 | "print(tf.__version__)\n", 965 | "\n", 966 | "class myCallback(tf.keras.callbacks.Callback):\n", 967 | " def on_epoch_end(self, epoch, logs={}):\n", 968 | " if(logs.get('loss')<0.4):\n", 969 | " print(\"\\nReached 60% accuracy so cancelling training!\")\n", 970 | " self.model.stop_training = True\n", 971 | "\n", 972 | "callbacks = myCallback()\n", 973 | "\n", 974 | "mnist = tf.keras.datasets.fashion_mnist\n", 975 | "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n", 976 | "training_images=training_images/255.0\n", 977 | "test_images=test_images/255.0\n", 978 | "\n", 979 | "model = tf.keras.models.Sequential([\n", 980 | " tf.keras.layers.Flatten(input_shape=(28,28)),\n", 981 | " tf.keras.layers.Dense(512, activation=tf.nn.relu),\n", 982 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n", 983 | "])\n", 984 | "\n", 985 | "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')\n", 986 | "model.fit(training_images, training_labels, epochs=5, callbacks=[callbacks])\n" 987 | ], 988 | "execution_count": 0, 989 | "outputs": [] 990 | }, 991 | { 992 | "cell_type": "code", 993 | "metadata": { 994 | "id": "mGAwUxxGGtWB", 995 | "colab_type": "code", 996 | "colab": {} 997 | }, 998 | "source": [ 999 | "model.summary()" 1000 | ], 1001 | "execution_count": 0, 1002 | "outputs": [] 1003 | }, 1004 | { 1005 | "cell_type": "markdown", 1006 | "metadata": { 1007 | "id": "7XNUtVwLJSAs", 1008 | "colab_type": "text" 1009 | }, 1010 | "source": [ 1011 | "#Extra resources" 1012 | ] 1013 | }, 1014 | { 1015 | "cell_type": "code", 1016 | "metadata": { 1017 | "id": "hgj2sHCXFypx", 1018 | "colab_type": "code", 1019 | "colab": {} 1020 | }, 1021 | "source": [ 1022 | "#https://ndres.me/post/matplotlib-animated-gifs-easily/\n", 1023 | "\n", 1024 | "import matplotlib\n", 1025 | "import matplotlib.pyplot as plt\n", 1026 | "import numpy as np\n", 1027 | "import imageio\n", 1028 | "\n", 1029 | "def plot_for_offset(power, y_max):\n", 1030 | " # Data for plotting\n", 1031 | " t = np.arange(0.0, 100, 1)\n", 1032 | " s = t**power\n", 1033 | "\n", 1034 | " fig, ax = plt.subplots(figsize=(10,5))\n", 1035 | " ax.plot(t, s)\n", 1036 | " ax.grid()\n", 1037 | " ax.set(xlabel='X', ylabel='x^{}'.format(power),\n", 1038 | " title='Powers of x')\n", 1039 | "\n", 1040 | " # IMPORTANT ANIMATION CODE HERE\n", 1041 | " # Used to keep the limits constant\n", 1042 | " ax.set_ylim(0, y_max)\n", 1043 | "\n", 1044 | " # Used to return the plot as an image rray\n", 1045 | " fig.canvas.draw() # draw the canvas, cache the renderer\n", 1046 | " image = np.frombuffer(fig.canvas.tostring_rgb(), dtype='uint8')\n", 1047 | " image = image.reshape(fig.canvas.get_width_height()[::-1] + (3,))\n", 1048 | "\n", 1049 | " return image\n", 1050 | "\n", 1051 | "kwargs_write = {'fps':1.0, 'quantizer':'nq'}\n", 1052 | "imageio.mimsave('./powers.gif', [plot_for_offset(i/4, 100) for i in range(10)], fps=1)" 1053 | ], 1054 | "execution_count": 0, 1055 | "outputs": [] 1056 | }, 1057 | { 1058 | "cell_type": "code", 1059 | "metadata": { 1060 | "id": "scR4pNl4F4_-", 1061 | "colab_type": "code", 1062 | "colab": {} 1063 | }, 1064 | "source": [ 1065 | "#http://louistiao.me/posts/notebooks/save-matplotlib-animations-as-gifs/\n", 1066 | "\n", 1067 | "import numpy as np\n", 1068 | "import matplotlib.pyplot as plt\n", 1069 | "\n", 1070 | "from matplotlib import animation, rc\n", 1071 | "from IPython.display import HTML, Image\n", 1072 | "\n", 1073 | "# equivalent to rcParams['animation.html'] = 'html5'\n", 1074 | "rc('animation', html='html5')\n", 1075 | "\n", 1076 | "# First set up the figure, the axis, and the plot element we want to animate\n", 1077 | "fig, ax = plt.subplots()\n", 1078 | "\n", 1079 | "ax.set_xlim(( 0, 2))\n", 1080 | "ax.set_ylim((-2, 2))\n", 1081 | "\n", 1082 | "line, = ax.plot([], [], lw=2)\n", 1083 | "\n", 1084 | "# initialization function: plot the background of each frame\n", 1085 | "def init():\n", 1086 | " line.set_data([], [])\n", 1087 | " return (line,)\n", 1088 | "\n", 1089 | "# animation function. This is called sequentially\n", 1090 | "def animate(i):\n", 1091 | " x = np.linspace(0, 2, 1000)\n", 1092 | " y = np.sin(2 * np.pi * (x - 0.01 * i))\n", 1093 | " line.set_data(x, y)\n", 1094 | " return (line,)\n", 1095 | "\n", 1096 | "# call the animator. blit=True means only re-draw the parts that \n", 1097 | "# have changed.\n", 1098 | "anim = animation.FuncAnimation(fig, animate, init_func=init,\n", 1099 | " frames=100, interval=20, blit=True)" 1100 | ], 1101 | "execution_count": 0, 1102 | "outputs": [] 1103 | }, 1104 | { 1105 | "cell_type": "code", 1106 | "metadata": { 1107 | "id": "NeeC9aIpGdEl", 1108 | "colab_type": "code", 1109 | "colab": {} 1110 | }, 1111 | "source": [ 1112 | "anim" 1113 | ], 1114 | "execution_count": 0, 1115 | "outputs": [] 1116 | }, 1117 | { 1118 | "cell_type": "code", 1119 | "metadata": { 1120 | "id": "c6xfQ5QIHbhB", 1121 | "colab_type": "code", 1122 | "colab": {} 1123 | }, 1124 | "source": [ 1125 | "!apt install imagemagick" 1126 | ], 1127 | "execution_count": 0, 1128 | "outputs": [] 1129 | }, 1130 | { 1131 | "cell_type": "code", 1132 | "metadata": { 1133 | "id": "UOJPX_g0Gd5n", 1134 | "colab_type": "code", 1135 | "colab": {} 1136 | }, 1137 | "source": [ 1138 | "anim.save('animation.gif', writer='imagemagick', fps=60)" 1139 | ], 1140 | "execution_count": 0, 1141 | "outputs": [] 1142 | }, 1143 | { 1144 | "cell_type": "code", 1145 | "metadata": { 1146 | "id": "1yLXYonCHV-t", 1147 | "colab_type": "code", 1148 | "colab": {} 1149 | }, 1150 | "source": [ 1151 | "Image(url='animation.gif')" 1152 | ], 1153 | "execution_count": 0, 1154 | "outputs": [] 1155 | }, 1156 | { 1157 | "cell_type": "code", 1158 | "metadata": { 1159 | "id": "mAaEcVCfHyDV", 1160 | "colab_type": "code", 1161 | "colab": {} 1162 | }, 1163 | "source": [ 1164 | "" 1165 | ], 1166 | "execution_count": 0, 1167 | "outputs": [] 1168 | } 1169 | ] 1170 | } -------------------------------------------------------------------------------- /C3W1_Tokenizer_stopwords_BBC_news_archive.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "C3W1-Tokenizer-stopwords-BBC news archive", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "include_colab_link": true 10 | }, 11 | "kernelspec": { 12 | "display_name": "Python 3", 13 | "name": "python3" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "metadata": { 20 | "id": "view-in-github", 21 | "colab_type": "text" 22 | }, 23 | "source": [ 24 | "\"Open" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": { 30 | "colab_type": "text", 31 | "id": "rX8mhOLljYeM" 32 | }, 33 | "source": [ 34 | "##### Copyright 2019 The TensorFlow Authors." 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "metadata": { 40 | "cellView": "form", 41 | "colab_type": "code", 42 | "id": "BZSlp3DAjdYf", 43 | "colab": {} 44 | }, 45 | "source": [ 46 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 47 | "# you may not use this file except in compliance with the License.\n", 48 | "# You may obtain a copy of the License at\n", 49 | "#\n", 50 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 51 | "#\n", 52 | "# Unless required by applicable law or agreed to in writing, software\n", 53 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 54 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 55 | "# See the License for the specific language governing permissions and\n", 56 | "# limitations under the License." 57 | ], 58 | "execution_count": 0, 59 | "outputs": [] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "metadata": { 64 | "colab_type": "code", 65 | "id": "zrZevCPJ92HG", 66 | "outputId": "56268043-e007-438e-f852-1d0efd3d37b6", 67 | "colab": { 68 | "base_uri": "https://localhost:8080/", 69 | "height": 204 70 | } 71 | }, 72 | "source": [ 73 | "!wget --no-check-certificate \\\n", 74 | " https://storage.googleapis.com/laurencemoroney-blog.appspot.com/bbc-text.csv \\\n", 75 | " -O /tmp/bbc-text.csv\n", 76 | "\n", 77 | " \n", 78 | "import csv\n", 79 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 80 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 81 | "\n", 82 | "#Dataset source: http://mlg.ucd.ie/datasets/bbc.html\n", 83 | "\n", 84 | "#Stopwords list from https://github.com/Yoast/YoastSEO.js/blob/develop/src/config/stopwords.js\n", 85 | "# Convert it to a Python list and paste it here\n", 86 | "stopwords = [ \"a\", \"about\", \"above\", \"after\", \"again\", \"against\", \"all\", \"am\", \"an\", \"and\", \"any\", \"are\", \"as\", \"at\", \"be\", \"because\", \"been\", \"before\", \"being\", \"below\", \"between\", \"both\", \"but\", \"by\", \"could\", \"did\", \"do\", \"does\", \"doing\", \"down\", \"during\", \"each\", \"few\", \"for\", \"from\", \"further\", \"had\", \"has\", \"have\", \"having\", \"he\", \"he'd\", \"he'll\", \"he's\", \"her\", \"here\", \"here's\", \"hers\", \"herself\", \"him\", \"himself\", \"his\", \"how\", \"how's\", \"i\", \"i'd\", \"i'll\", \"i'm\", \"i've\", \"if\", \"in\", \"into\", \"is\", \"it\", \"it's\", \"its\", \"itself\", \"let's\", \"me\", \"more\", \"most\", \"my\", \"myself\", \"nor\", \"of\", \"on\", \"once\", \"only\", \"or\", \"other\", \"ought\", \"our\", \"ours\", \"ourselves\", \"out\", \"over\", \"own\", \"same\", \"she\", \"she'd\", \"she'll\", \"she's\", \"should\", \"so\", \"some\", \"such\", \"than\", \"that\", \"that's\", \"the\", \"their\", \"theirs\", \"them\", \"themselves\", \"then\", \"there\", \"there's\", \"these\", \"they\", \"they'd\", \"they'll\", \"they're\", \"they've\", \"this\", \"those\", \"through\", \"to\", \"too\", \"under\", \"until\", \"up\", \"very\", \"was\", \"we\", \"we'd\", \"we'll\", \"we're\", \"we've\", \"were\", \"what\", \"what's\", \"when\", \"when's\", \"where\", \"where's\", \"which\", \"while\", \"who\", \"who's\", \"whom\", \"why\", \"why's\", \"with\", \"would\", \"you\", \"you'd\", \"you'll\", \"you're\", \"you've\", \"your\", \"yours\", \"yourself\", \"yourselves\" ]\n" 87 | ], 88 | "execution_count": 0, 89 | "outputs": [ 90 | { 91 | "output_type": "stream", 92 | "text": [ 93 | "--2020-04-18 21:37:28-- https://storage.googleapis.com/laurencemoroney-blog.appspot.com/bbc-text.csv\n", 94 | "Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.203.128, 2404:6800:4008:c04::80\n", 95 | "Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.203.128|:443... connected.\n", 96 | "HTTP request sent, awaiting response... 200 OK\n", 97 | "Length: 5057493 (4.8M) [application/octet-stream]\n", 98 | "Saving to: ‘/tmp/bbc-text.csv’\n", 99 | "\n", 100 | "\r/tmp/bbc-text.csv 0%[ ] 0 --.-KB/s \r/tmp/bbc-text.csv 100%[===================>] 4.82M --.-KB/s in 0.04s \n", 101 | "\n", 102 | "2020-04-18 21:37:28 (117 MB/s) - ‘/tmp/bbc-text.csv’ saved [5057493/5057493]\n", 103 | "\n" 104 | ], 105 | "name": "stdout" 106 | } 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "metadata": { 112 | "colab_type": "code", 113 | "id": "1rmYBjsyCv3K", 114 | "outputId": "f2731707-cd42-4b8e-a4d6-e5c3f5817ea5", 115 | "colab": { 116 | "base_uri": "https://localhost:8080/", 117 | "height": 71 118 | } 119 | }, 120 | "source": [ 121 | "sentences = []\n", 122 | "labels = []\n", 123 | "with open(\"/tmp/bbc-text.csv\", 'r') as csvfile:\n", 124 | " # Your Code here\n", 125 | " csvfile = csv.reader(csvfile, delimiter=',')\n", 126 | " next(csvfile)\n", 127 | "\n", 128 | " for row in csvfile:\n", 129 | " #print(row)\n", 130 | " #break\n", 131 | " labels.append(row[0])\n", 132 | " sentence = row[1]\n", 133 | "\n", 134 | " for word in stopwords:\n", 135 | " token = \" \" + word + \" \"\n", 136 | " sentence = sentence.replace(token, \" \")\n", 137 | " sentence = sentence.replace(\" \", \" \")\n", 138 | " sentences.append(sentence)\n", 139 | "\n", 140 | "print(len(sentences))\n", 141 | "print(sentences[0])\n", 142 | "\n", 143 | "#Expected output\n", 144 | "# 2225\n", 145 | "# tv future hands viewers home theatre systems plasma high-definition tvs digital video recorders moving living room way people watch tv will radically different five years time. according expert panel gathered annual consumer electronics show las vegas discuss new technologies will impact one favourite pastimes. us leading trend programmes content will delivered viewers via home networks cable satellite telecoms companies broadband service providers front rooms portable devices. one talked-about technologies ces digital personal video recorders (dvr pvr). set-top boxes like us s tivo uk s sky+ system allow people record store play pause forward wind tv programmes want. essentially technology allows much personalised tv. also built-in high-definition tv sets big business japan us slower take off europe lack high-definition programming. not can people forward wind adverts can also forget abiding network channel schedules putting together a-la-carte entertainment. us networks cable satellite companies worried means terms advertising revenues well brand identity viewer loyalty channels. although us leads technology moment also concern raised europe particularly growing uptake services like sky+. happens today will see nine months years time uk adam hume bbc broadcast s futurologist told bbc news website. likes bbc no issues lost advertising revenue yet. pressing issue moment commercial uk broadcasters brand loyalty important everyone. will talking content brands rather network brands said tim hanlon brand communications firm starcom mediavest. reality broadband connections anybody can producer content. added: challenge now hard promote programme much choice. means said stacey jolna senior vice president tv guide tv group way people find content want watch simplified tv viewers. means networks us terms channels take leaf google s book search engine future instead scheduler help people find want watch. kind channel model might work younger ipod generation used taking control gadgets play them. might not suit everyone panel recognised. older generations comfortable familiar schedules channel brands know getting. perhaps not want much choice put hands mr hanlon suggested. end kids just diapers pushing buttons already - everything possible available said mr hanlon. ultimately consumer will tell market want. 50 000 new gadgets technologies showcased ces many enhancing tv-watching experience. high-definition tv sets everywhere many new models lcd (liquid crystal display) tvs launched dvr capability built instead external boxes. one example launched show humax s 26-inch lcd tv 80-hour tivo dvr dvd recorder. one us s biggest satellite tv companies directtv even launched branded dvr show 100-hours recording capability instant replay search function. set can pause rewind tv 90 hours. microsoft chief bill gates announced pre-show keynote speech partnership tivo called tivotogo means people can play recorded programmes windows pcs mobile devices. reflect increasing trend freeing multimedia people can watch want want." 146 | ], 147 | "execution_count": 0, 148 | "outputs": [ 149 | { 150 | "output_type": "stream", 151 | "text": [ 152 | "2225\n", 153 | "tv future hands viewers home theatre systems plasma high-definition tvs digital video recorders moving living room way people watch tv will radically different five years time. according expert panel gathered annual consumer electronics show las vegas discuss new technologies will impact one favourite pastimes. us leading trend programmes content will delivered viewers via home networks cable satellite telecoms companies broadband service providers front rooms portable devices. one talked-about technologies ces digital personal video recorders (dvr pvr). set-top boxes like us s tivo uk s sky+ system allow people record store play pause forward wind tv programmes want. essentially technology allows much personalised tv. also built-in high-definition tv sets big business japan us slower take off europe lack high-definition programming. not can people forward wind adverts can also forget abiding network channel schedules putting together a-la-carte entertainment. us networks cable satellite companies worried means terms advertising revenues well brand identity viewer loyalty channels. although us leads technology moment also concern raised europe particularly growing uptake services like sky+. happens today will see nine months years time uk adam hume bbc broadcast s futurologist told bbc news website. likes bbc no issues lost advertising revenue yet. pressing issue moment commercial uk broadcasters brand loyalty important everyone. will talking content brands rather network brands said tim hanlon brand communications firm starcom mediavest. reality broadband connections anybody can producer content. added: challenge now hard promote programme much choice. means said stacey jolna senior vice president tv guide tv group way people find content want watch simplified tv viewers. means networks us terms channels take leaf google s book search engine future instead scheduler help people find want watch. kind channel model might work younger ipod generation used taking control gadgets play them. might not suit everyone panel recognised. older generations comfortable familiar schedules channel brands know getting. perhaps not want much choice put hands mr hanlon suggested. end kids just diapers pushing buttons already - everything possible available said mr hanlon. ultimately consumer will tell market want. 50 000 new gadgets technologies showcased ces many enhancing tv-watching experience. high-definition tv sets everywhere many new models lcd (liquid crystal display) tvs launched dvr capability built instead external boxes. one example launched show humax s 26-inch lcd tv 80-hour tivo dvr dvd recorder. one us s biggest satellite tv companies directtv even launched branded dvr show 100-hours recording capability instant replay search function. set can pause rewind tv 90 hours. microsoft chief bill gates announced pre-show keynote speech partnership tivo called tivotogo means people can play recorded programmes windows pcs mobile devices. reflect increasing trend freeing multimedia people can watch want want.\n" 154 | ], 155 | "name": "stdout" 156 | } 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "metadata": { 162 | "colab_type": "code", 163 | "id": "9LhzBBgSC3S5", 164 | "outputId": "af97c07e-e7d4-4780-db1c-b00a2936e59c", 165 | "colab": { 166 | "base_uri": "https://localhost:8080/", 167 | "height": 34 168 | } 169 | }, 170 | "source": [ 171 | "tokenizer = Tokenizer(oov_token=\"\")\n", 172 | "tokenizer.fit_on_texts(sentences)\n", 173 | "word_index = tokenizer.word_index\n", 174 | "print(len(word_index))\n", 175 | "# Expected output\n", 176 | "# 29714" 177 | ], 178 | "execution_count": 0, 179 | "outputs": [ 180 | { 181 | "output_type": "stream", 182 | "text": [ 183 | "29714\n" 184 | ], 185 | "name": "stdout" 186 | } 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "metadata": { 192 | "colab_type": "code", 193 | "id": "1Gr3dbQfC5VR", 194 | "outputId": "4c494e40-580c-47d5-b6e3-a2a5b210a93a", 195 | "colab": { 196 | "base_uri": "https://localhost:8080/", 197 | "height": 51 198 | } 199 | }, 200 | "source": [ 201 | "sequences = tokenizer.texts_to_sequences(sentences)\n", 202 | "padded = pad_sequences(sequences, padding='post')\n", 203 | "print(padded[0])\n", 204 | "print(padded.shape)\n", 205 | "\n", 206 | "# Expected output\n", 207 | "# [ 96 176 1158 ... 0 0 0]\n", 208 | "# (2225, 2442)" 209 | ], 210 | "execution_count": 0, 211 | "outputs": [ 212 | { 213 | "output_type": "stream", 214 | "text": [ 215 | "[ 96 176 1158 ... 0 0 0]\n", 216 | "(2225, 2442)\n" 217 | ], 218 | "name": "stdout" 219 | } 220 | ] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "metadata": { 225 | "colab_type": "code", 226 | "id": "fZufOahzC6yx", 227 | "outputId": "33dd3cf9-09e4-41cc-f930-2438f57ff230", 228 | "colab": { 229 | "base_uri": "https://localhost:8080/", 230 | "height": 71 231 | } 232 | }, 233 | "source": [ 234 | "tokenizer = Tokenizer()\n", 235 | "\n", 236 | "tokenizer.fit_on_texts(labels)\n", 237 | "label_word_index = tokenizer.word_index\n", 238 | "\n", 239 | "label_seq = tokenizer.texts_to_sequences(labels)\n", 240 | "#label_seq = pad_sequences(label_seq, padding='post')\n", 241 | "\n", 242 | "print(label_seq)\n", 243 | "print(label_word_index)\n", 244 | "\n", 245 | "# Expected Output\n", 246 | "# [[4], [2], [1], [1], [5], [3], [3], [1], [1], [5], [5], [2], [2], [3], [1], [2], [3], [1], [2], [4], [4], [4], [1], [1], [4], [1], [5], [4], [3], [5], [3], [4], [5], [5], [2], [3], [4], [5], [3], [2], [3], [1], [2], [1], [4], [5], [3], [3], [3], [2], [1], [3], [2], [2], [1], [3], [2], [1], [1], [2], [2], [1], [2], [1], [2], [4], [2], [5], [4], [2], [3], [2], [3], [1], [2], [4], [2], [1], [1], [2], [2], [1], [3], [2], [5], [3], [3], [2], [5], [2], [1], [1], [3], [1], [3], [1], [2], [1], [2], [5], [5], [1], [2], [3], [3], [4], [1], [5], [1], [4], [2], [5], [1], [5], [1], [5], [5], [3], [1], [1], [5], [3], [2], [4], [2], [2], [4], [1], [3], [1], [4], [5], [1], [2], [2], [4], [5], [4], [1], [2], [2], [2], [4], [1], [4], [2], [1], [5], [1], [4], [1], [4], [3], [2], [4], [5], [1], [2], [3], [2], [5], [3], [3], [5], [3], [2], [5], [3], [3], [5], [3], [1], [2], [3], [3], [2], [5], [1], [2], [2], [1], [4], [1], [4], [4], [1], [2], [1], [3], [5], [3], [2], [3], [2], [4], [3], [5], [3], [4], [2], [1], [2], [1], [4], [5], [2], [3], [3], [5], [1], [5], [3], [1], [5], [1], [1], [5], [1], [3], [3], [5], [4], [1], [3], [2], [5], [4], [1], [4], [1], [5], [3], [1], [5], [4], [2], [4], [2], [2], [4], [2], [1], [2], [1], [2], [1], [5], [2], [2], [5], [1], [1], [3], [4], [3], [3], [3], [4], [1], [4], [3], [2], [4], [5], [4], [1], [1], [2], [2], [3], [2], [4], [1], [5], [1], [3], [4], [5], [2], [1], [5], [1], [4], [3], [4], [2], [2], [3], [3], [1], [2], [4], [5], [3], [4], [2], [5], [1], [5], [1], [5], [3], [2], [1], [2], [1], [1], [5], [1], [3], [3], [2], [5], [4], [2], [1], [2], [5], [2], [2], [2], [3], [2], [3], [5], [5], [2], [1], [2], [3], [2], [4], [5], [2], [1], [1], [5], [2], [2], [3], [4], [5], [4], [3], [2], [1], [3], [2], [5], [4], [5], [4], [3], [1], [5], [2], [3], [2], [2], [3], [1], [4], [2], [2], [5], [5], [4], [1], [2], [5], [4], [4], [5], [5], [5], [3], [1], [3], [4], [2], [5], [3], [2], [5], [3], [3], [1], [1], [2], [3], [5], [2], [1], [2], [2], [1], [2], [3], [3], [3], [1], [4], [4], [2], [4], [1], [5], [2], [3], [2], [5], [2], [3], [5], [3], [2], [4], [2], [1], [1], [2], [1], [1], [5], [1], [1], [1], [4], [2], [2], [2], [3], [1], [1], [2], [4], [2], [3], [1], [3], [4], [2], [1], [5], [2], [3], [4], [2], [1], [2], [3], [2], [2], [1], [5], [4], [3], [4], [2], [1], [2], [5], [4], [4], [2], [1], [1], [5], [3], [3], [3], [1], [3], [4], [4], [5], [3], [4], [5], [2], [1], [1], [4], [2], [1], [1], [3], [1], [1], [2], [1], [5], [4], [3], [1], [3], [4], [2], [2], [2], [4], [2], [2], [1], [1], [1], [1], [2], [4], [5], [1], [1], [4], [2], [4], [5], [3], [1], [2], [3], [2], [4], [4], [3], [4], [2], [1], [2], [5], [1], [3], [5], [1], [1], [3], [4], [5], [4], [1], [3], [2], [5], [3], [2], [5], [1], [1], [4], [3], [5], [3], [5], [3], [4], [3], [5], [1], [2], [1], [5], [1], [5], [4], [2], [1], [3], [5], [3], [5], [5], [5], [3], [5], [4], [3], [4], [4], [1], [1], [4], [4], [1], [5], [5], [1], [4], [5], [1], [1], [4], [2], [3], [4], [2], [1], [5], [1], [5], [3], [4], [5], [5], [2], [5], [5], [1], [4], [4], [3], [1], [4], [1], [3], [3], [5], [4], [2], [4], [4], [4], [2], [3], [3], [1], [4], [2], [2], [5], [5], [1], [4], [2], [4], [5], [1], [4], [3], [4], [3], [2], [3], [3], [2], [1], [4], [1], [4], [3], [5], [4], [1], [5], [4], [1], [3], [5], [1], [4], [1], [1], [3], [5], [2], [3], [5], [2], [2], [4], [2], [5], [4], [1], [4], [3], [4], [3], [2], [3], [5], [1], [2], [2], [2], [5], [1], [2], [5], [5], [1], [5], [3], [3], [3], [1], [1], [1], [4], [3], [1], [3], [3], [4], [3], [1], [2], [5], [1], [2], [2], [4], [2], [5], [5], [5], [2], [5], [5], [3], [4], [2], [1], [4], [1], [1], [3], [2], [1], [4], [2], [1], [4], [1], [1], [5], [1], [2], [1], [2], [4], [3], [4], [2], [1], [1], [2], [2], [2], [2], [3], [1], [2], [4], [2], [1], [3], [2], [4], [2], [1], [2], [3], [5], [1], [2], [3], [2], [5], [2], [2], [2], [1], [3], [5], [1], [3], [1], [3], [3], [2], [2], [1], [4], [5], [1], [5], [2], [2], [2], [4], [1], [4], [3], [4], [4], [4], [1], [4], [4], [5], [5], [4], [1], [5], [4], [1], [1], [2], [5], [4], [2], [1], [2], [3], [2], [5], [4], [2], [3], [2], [4], [1], [2], [5], [2], [3], [1], [5], [3], [1], [2], [1], [3], [3], [1], [5], [5], [2], [2], [1], [4], [4], [1], [5], [4], [4], [2], [1], [5], [4], [1], [1], [2], [5], [2], [2], [2], [5], [1], [5], [4], [4], [4], [3], [4], [4], [5], [5], [1], [1], [3], [2], [5], [1], [3], [5], [4], [3], [4], [4], [2], [5], [3], [4], [3], [3], [1], [3], [3], [5], [4], [1], [3], [1], [5], [3], [2], [2], [3], [1], [1], [1], [5], [4], [4], [2], [5], [1], [3], [4], [3], [5], [4], [4], [2], [2], [1], [2], [2], [4], [3], [5], [2], [2], [2], [2], [2], [4], [1], [3], [4], [4], [2], [2], [5], [3], [5], [1], [4], [1], [5], [1], [4], [1], [2], [1], [3], [3], [5], [2], [1], [3], [3], [1], [5], [3], [2], [4], [1], [2], [2], [2], [5], [5], [4], [4], [2], [2], [5], [1], [2], [5], [4], [4], [2], [2], [1], [1], [1], [3], [3], [1], [3], [1], [2], [5], [1], [4], [5], [1], [1], [2], [2], [4], [4], [1], [5], [1], [5], [1], [5], [3], [5], [5], [4], [5], [2], [2], [3], [1], [3], [4], [2], [3], [1], [3], [1], [5], [1], [3], [1], [1], [4], [5], [1], [3], [1], [1], [2], [4], [5], [3], [4], [5], [3], [5], [3], [5], [5], [4], [5], [3], [5], [5], [4], [4], [1], [1], [5], [5], [4], [5], [3], [4], [5], [2], [4], [1], [2], [5], [5], [4], [5], [4], [2], [5], [1], [5], [2], [1], [2], [1], [3], [4], [5], [3], [2], [5], [5], [3], [2], [5], [1], [3], [1], [2], [2], [2], [2], [2], [5], [4], [1], [5], [5], [2], [1], [4], [4], [5], [1], [2], [3], [2], [3], [2], [2], [5], [3], [2], [2], [4], [3], [1], [4], [5], [3], [2], [2], [1], [5], [3], [4], [2], [2], [3], [2], [1], [5], [1], [5], [4], [3], [2], [2], [4], [2], [2], [1], [2], [4], [5], [3], [2], [3], [2], [1], [4], [2], [3], [5], [4], [2], [5], [1], [3], [3], [1], [3], [2], [4], [5], [1], [1], [4], [2], [1], [5], [4], [1], [3], [1], [2], [2], [2], [3], [5], [1], [3], [4], [2], [2], [4], [5], [5], [4], [4], [1], [1], [5], [4], [5], [1], [3], [4], [2], [1], [5], [2], [2], [5], [1], [2], [1], [4], [3], [3], [4], [5], [3], [5], [2], [2], [3], [1], [4], [1], [1], [1], [3], [2], [1], [2], [4], [1], [2], [2], [1], [3], [4], [1], [2], [4], [1], [1], [2], [2], [2], [2], [3], [5], [4], [2], [2], [1], [2], [5], [2], [5], [1], [3], [2], [2], [4], [5], [2], [2], [2], [3], [2], [3], [4], [5], [3], [5], [1], [4], [3], [2], [4], [1], [2], [2], [5], [4], [2], [2], [1], [1], [5], [1], [3], [1], [2], [1], [2], [3], [3], [2], [3], [4], [5], [1], [2], [5], [1], [3], [3], [4], [5], [2], [3], [3], [1], [4], [2], [1], [5], [1], [5], [1], [2], [1], [3], [5], [4], [2], [1], [3], [4], [1], [5], [2], [1], [5], [1], [4], [1], [4], [3], [1], [2], [5], [4], [4], [3], [4], [5], [4], [1], [2], [4], [2], [5], [1], [4], [3], [3], [3], [3], [5], [5], [5], [2], [3], [3], [1], [1], [4], [1], [3], [2], [2], [4], [1], [4], [2], [4], [3], [3], [1], [2], [3], [1], [2], [4], [2], [2], [5], [5], [1], [2], [4], [4], [3], [2], [3], [1], [5], [5], [3], [3], [2], [2], [4], [4], [1], [1], [3], [4], [1], [4], [2], [1], [2], [3], [1], [5], [2], [4], [3], [5], [4], [2], [1], [5], [4], [4], [5], [3], [4], [5], [1], [5], [1], [1], [1], [3], [4], [1], [2], [1], [1], [2], [4], [1], [2], [5], [3], [4], [1], [3], [4], [5], [3], [1], [3], [4], [2], [5], [1], [3], [2], [4], [4], [4], [3], [2], [1], [3], [5], [4], [5], [1], [4], [2], [3], [5], [4], [3], [1], [1], [2], [5], [2], [2], [3], [2], [2], [3], [4], [5], [3], [5], [5], [2], [3], [1], [3], [5], [1], [5], [3], [5], [5], [5], [2], [1], [3], [1], [5], [4], [4], [2], [3], [5], [2], [1], [2], [3], [3], [2], [1], [4], [4], [4], [2], [3], [3], [2], [1], [1], [5], [2], [1], [1], [3], [3], [3], [5], [3], [2], [4], [2], [3], [5], [5], [2], [1], [3], [5], [1], [5], [3], [3], [2], [3], [1], [5], [5], [4], [4], [4], [4], [3], [4], [2], [4], [1], [1], [5], [2], [4], [5], [2], [4], [1], [4], [5], [5], [3], [3], [1], [2], [2], [4], [5], [1], [3], [2], [4], [5], [3], [1], [5], [3], [3], [4], [1], [3], [2], [3], [5], [4], [1], [3], [5], [5], [2], [1], [4], [4], [1], [5], [4], [3], [4], [1], [3], [3], [1], [5], [1], [3], [1], [4], [5], [1], [5], [2], [2], [5], [5], [5], [4], [1], [2], [2], [3], [3], [2], [3], [5], [1], [1], [4], [3], [1], [2], [1], [2], [4], [1], [1], [2], [5], [1], [1], [4], [1], [2], [3], [2], [5], [4], [5], [3], [2], [5], [3], [5], [3], [3], [2], [1], [1], [1], [4], [4], [1], [3], [5], [4], [1], [5], [2], [5], [3], [2], [1], [4], [2], [1], [3], [2], [5], [5], [5], [3], [5], [3], [5], [1], [5], [1], [3], [3], [2], [3], [4], [1], [4], [1], [2], [3], [4], [5], [5], [3], [5], [3], [1], [1], [3], [2], [4], [1], [3], [3], [5], [1], [3], [3], [2], [4], [4], [2], [4], [1], [1], [2], [3], [2], [4], [1], [4], [3], [5], [1], [2], [1], [5], [4], [4], [1], [3], [1], [2], [1], [2], [1], [1], [5], [5], [2], [4], [4], [2], [4], [2], [2], [1], [1], [3], [1], [4], [1], [4], [1], [1], [2], [2], [4], [1], [2], [4], [4], [3], [1], [2], [5], [5], [4], [3], [1], [1], [4], [2], [4], [5], [5], [3], [3], [2], [5], [1], [5], [5], [2], [1], [3], [4], [2], [1], [5], [4], [3], [3], [1], [1], [2], [2], [2], [2], [2], [5], [2], [3], [3], [4], [4], [5], [3], [5], [2], [3], [1], [1], [2], [4], [2], [4], [1], [2], [2], [3], [1], [1], [3], [3], [5], [5], [3], [2], [3], [3], [2], [4], [3], [3], [3], [3], [3], [5], [5], [4], [3], [1], [3], [1], [4], [1], [1], [1], [5], [4], [5], [4], [1], [4], [1], [1], [5], [5], [2], [5], [5], [3], [2], [1], [4], [4], [3], [2], [1], [2], [5], [1], [3], [5], [1], [1], [2], [3], [4], [4], [2], [2], [1], [3], [5], [1], [1], [3], [5], [4], [1], [5], [2], [3], [1], [3], [4], [5], [1], [3], [2], [5], [3], [5], [3], [1], [3], [2], [2], [3], [2], [4], [1], [2], [5], [2], [1], [1], [5], [4], [3], [4], [3], [3], [1], [1], [1], [2], [4], [5], [2], [1], [2], [1], [2], [4], [2], [2], [2], [2], [1], [1], [1], [2], [2], [5], [2], [2], [2], [1], [1], [1], [4], [2], [1], [1], [1], [2], [5], [4], [4], [4], [3], [2], [2], [4], [2], [4], [1], [1], [3], [3], [3], [1], [1], [3], [3], [4], [2], [1], [1], [1], [1], [2], [1], [2], [2], [2], [2], [1], [3], [1], [4], [4], [1], [4], [2], [5], [2], [1], [2], [4], [4], [3], [5], [2], [5], [2], [4], [3], [5], [3], [5], [5], [4], [2], [4], [4], [2], [3], [1], [5], [2], [3], [5], [2], [4], [1], [4], [3], [1], [3], [2], [3], [3], [2], [2], [2], [4], [3], [2], [3], [2], [5], [3], [1], [3], [3], [1], [5], [4], [4], [2], [4], [1], [2], [2], [3], [1], [4], [4], [4], [1], [5], [1], [3], [2], [3], [3], [5], [4], [2], [4], [1], [5], [5], [1], [2], [5], [4], [4], [1], [5], [2], [3], [3], [3], [4], [4], [2], [3], [2], [3], [3], [5], [1], [4], [2], [4], [5], [4], [4], [1], [3], [1], [1], [3], [5], [5], [2], [3], [3], [1], [2], [2], [4], [2], [4], [4], [1], [2], [3], [1], [2], [2], [1], [4], [1], [4], [5], [1], [1], [5], [2], [4], [1], [1], [3], [4], [2], [3], [1], [1], [3], [5], [4], [4], [4], [2], [1], [5], [5], [4], [2], [3], [4], [1], [1], [4], [4], [3], [2], [1], [5], [5], [1], [5], [4], [4], [2], [2], [2], [1], [1], [4], [1], [2], [4], [2], [2], [1], [2], [3], [2], [2], [4], [2], [4], [3], [4], [5], [3], [4], [5], [1], [3], [5], [2], [4], [2], [4], [5], [4], [1], [2], [2], [3], [5], [3], [1]]\n", 247 | "# {'sport': 1, 'business': 2, 'politics': 3, 'tech': 4, 'entertainment': 5}" 248 | ], 249 | "execution_count": 0, 250 | "outputs": [ 251 | { 252 | "output_type": "stream", 253 | "text": [ 254 | "[[4], [2], [1], [1], [5], [3], [3], [1], [1], [5], [5], [2], [2], [3], [1], [2], [3], [1], [2], [4], [4], [4], [1], [1], [4], [1], [5], [4], [3], [5], [3], [4], [5], [5], [2], [3], [4], [5], [3], [2], [3], [1], [2], [1], [4], [5], [3], [3], [3], [2], [1], [3], [2], [2], [1], [3], [2], [1], [1], [2], [2], [1], [2], [1], [2], [4], [2], [5], [4], [2], [3], [2], [3], [1], [2], [4], [2], [1], [1], [2], [2], [1], [3], [2], [5], [3], [3], [2], [5], [2], [1], [1], [3], [1], [3], [1], [2], [1], [2], [5], [5], [1], [2], [3], [3], [4], [1], [5], [1], [4], [2], [5], [1], [5], [1], [5], [5], [3], [1], [1], [5], [3], [2], [4], [2], [2], [4], [1], [3], [1], [4], [5], [1], [2], [2], [4], [5], [4], [1], [2], [2], [2], [4], [1], [4], [2], [1], [5], [1], [4], [1], [4], [3], [2], [4], [5], [1], [2], [3], [2], [5], [3], [3], [5], [3], [2], [5], [3], [3], [5], [3], [1], [2], [3], [3], [2], [5], [1], [2], [2], [1], [4], [1], [4], [4], [1], [2], [1], [3], [5], [3], [2], [3], [2], [4], [3], [5], [3], [4], [2], [1], [2], [1], [4], [5], [2], [3], [3], [5], [1], [5], [3], [1], [5], [1], [1], [5], [1], [3], [3], [5], [4], [1], [3], [2], [5], [4], [1], [4], [1], [5], [3], [1], [5], [4], [2], [4], [2], [2], [4], [2], [1], [2], [1], [2], [1], [5], [2], [2], [5], [1], [1], [3], [4], [3], [3], [3], [4], [1], [4], [3], [2], [4], [5], [4], [1], [1], [2], [2], [3], [2], [4], [1], [5], [1], [3], [4], [5], [2], [1], [5], [1], [4], [3], [4], [2], [2], [3], [3], [1], [2], [4], [5], [3], [4], [2], [5], [1], [5], [1], [5], [3], [2], [1], [2], [1], [1], [5], [1], [3], [3], [2], [5], [4], [2], [1], [2], [5], [2], [2], [2], [3], [2], [3], [5], [5], [2], [1], [2], [3], [2], [4], [5], [2], [1], [1], [5], [2], [2], [3], [4], [5], [4], [3], [2], [1], [3], [2], [5], [4], [5], [4], [3], [1], [5], [2], [3], [2], [2], [3], [1], [4], [2], [2], [5], [5], [4], [1], [2], [5], [4], [4], [5], [5], [5], [3], [1], [3], [4], [2], [5], [3], [2], [5], [3], [3], [1], [1], [2], [3], [5], [2], [1], [2], [2], [1], [2], [3], [3], [3], [1], [4], [4], [2], [4], [1], [5], [2], [3], [2], [5], [2], [3], [5], [3], [2], [4], [2], [1], [1], [2], [1], [1], [5], [1], [1], [1], [4], [2], [2], [2], [3], [1], [1], [2], [4], [2], [3], [1], [3], [4], [2], [1], [5], [2], [3], [4], [2], [1], [2], [3], [2], [2], [1], [5], [4], [3], [4], [2], [1], [2], [5], [4], [4], [2], [1], [1], [5], [3], [3], [3], [1], [3], [4], [4], [5], [3], [4], [5], [2], [1], [1], [4], [2], [1], [1], [3], [1], [1], [2], [1], [5], [4], [3], [1], [3], [4], [2], [2], [2], [4], [2], [2], [1], [1], [1], [1], [2], [4], [5], [1], [1], [4], [2], [4], [5], [3], [1], [2], [3], [2], [4], [4], [3], [4], [2], [1], [2], [5], [1], [3], [5], [1], [1], [3], [4], [5], [4], [1], [3], [2], [5], [3], [2], [5], [1], [1], [4], [3], [5], [3], [5], [3], [4], [3], [5], [1], [2], [1], [5], [1], [5], [4], [2], [1], [3], [5], [3], [5], [5], [5], [3], [5], [4], [3], [4], [4], [1], [1], [4], [4], [1], [5], [5], [1], [4], [5], [1], [1], [4], [2], [3], [4], [2], [1], [5], [1], [5], [3], [4], [5], [5], [2], [5], [5], [1], [4], [4], [3], [1], [4], [1], [3], [3], [5], [4], [2], [4], [4], [4], [2], [3], [3], [1], [4], [2], [2], [5], [5], [1], [4], [2], [4], [5], [1], [4], [3], [4], [3], [2], [3], [3], [2], [1], [4], [1], [4], [3], [5], [4], [1], [5], [4], [1], [3], [5], [1], [4], [1], [1], [3], [5], [2], [3], [5], [2], [2], [4], [2], [5], [4], [1], [4], [3], [4], [3], [2], [3], [5], [1], [2], [2], [2], [5], [1], [2], [5], [5], [1], [5], [3], [3], [3], [1], [1], [1], [4], [3], [1], [3], [3], [4], [3], [1], [2], [5], [1], [2], [2], [4], [2], [5], [5], [5], [2], [5], [5], [3], [4], [2], [1], [4], [1], [1], [3], [2], [1], [4], [2], [1], [4], [1], [1], [5], [1], [2], [1], [2], [4], [3], [4], [2], [1], [1], [2], [2], [2], [2], [3], [1], [2], [4], [2], [1], [3], [2], [4], [2], [1], [2], [3], [5], [1], [2], [3], [2], [5], [2], [2], [2], [1], [3], [5], [1], [3], [1], [3], [3], [2], [2], [1], [4], [5], [1], [5], [2], [2], [2], [4], [1], [4], [3], [4], [4], [4], [1], [4], [4], [5], [5], [4], [1], [5], [4], [1], [1], [2], [5], [4], [2], [1], [2], [3], [2], [5], [4], [2], [3], [2], [4], [1], [2], [5], [2], [3], [1], [5], [3], [1], [2], [1], [3], [3], [1], [5], [5], [2], [2], [1], [4], [4], [1], [5], [4], [4], [2], [1], [5], [4], [1], [1], [2], [5], [2], [2], [2], [5], [1], [5], [4], [4], [4], [3], [4], [4], [5], [5], [1], [1], [3], [2], [5], [1], [3], [5], [4], [3], [4], [4], [2], [5], [3], [4], [3], [3], [1], [3], [3], [5], [4], [1], [3], [1], [5], [3], [2], [2], [3], [1], [1], [1], [5], [4], [4], [2], [5], [1], [3], [4], [3], [5], [4], [4], [2], [2], [1], [2], [2], [4], [3], [5], [2], [2], [2], [2], [2], [4], [1], [3], [4], [4], [2], [2], [5], [3], [5], [1], [4], [1], [5], [1], [4], [1], [2], [1], [3], [3], [5], [2], [1], [3], [3], [1], [5], [3], [2], [4], [1], [2], [2], [2], [5], [5], [4], [4], [2], [2], [5], [1], [2], [5], [4], [4], [2], [2], [1], [1], [1], [3], [3], [1], [3], [1], [2], [5], [1], [4], [5], [1], [1], [2], [2], [4], [4], [1], [5], [1], [5], [1], [5], [3], [5], [5], [4], [5], [2], [2], [3], [1], [3], [4], [2], [3], [1], [3], [1], [5], [1], [3], [1], [1], [4], [5], [1], [3], [1], [1], [2], [4], [5], [3], [4], [5], [3], [5], [3], [5], [5], [4], [5], [3], [5], [5], [4], [4], [1], [1], [5], [5], [4], [5], [3], [4], [5], [2], [4], [1], [2], [5], [5], [4], [5], [4], [2], [5], [1], [5], [2], [1], [2], [1], [3], [4], [5], [3], [2], [5], [5], [3], [2], [5], [1], [3], [1], [2], [2], [2], [2], [2], [5], [4], [1], [5], [5], [2], [1], [4], [4], [5], [1], [2], [3], [2], [3], [2], [2], [5], [3], [2], [2], [4], [3], [1], [4], [5], [3], [2], [2], [1], [5], [3], [4], [2], [2], [3], [2], [1], [5], [1], [5], [4], [3], [2], [2], [4], [2], [2], [1], [2], [4], [5], [3], [2], [3], [2], [1], [4], [2], [3], [5], [4], [2], [5], [1], [3], [3], [1], [3], [2], [4], [5], [1], [1], [4], [2], [1], [5], [4], [1], [3], [1], [2], [2], [2], [3], [5], [1], [3], [4], [2], [2], [4], [5], [5], [4], [4], [1], [1], [5], [4], [5], [1], [3], [4], [2], [1], [5], [2], [2], [5], [1], [2], [1], [4], [3], [3], [4], [5], [3], [5], [2], [2], [3], [1], [4], [1], [1], [1], [3], [2], [1], [2], [4], [1], [2], [2], [1], [3], [4], [1], [2], [4], [1], [1], [2], [2], [2], [2], [3], [5], [4], [2], [2], [1], [2], [5], [2], [5], [1], [3], [2], [2], [4], [5], [2], [2], [2], [3], [2], [3], [4], [5], [3], [5], [1], [4], [3], [2], [4], [1], [2], [2], [5], [4], [2], [2], [1], [1], [5], [1], [3], [1], [2], [1], [2], [3], [3], [2], [3], [4], [5], [1], [2], [5], [1], [3], [3], [4], [5], [2], [3], [3], [1], [4], [2], [1], [5], [1], [5], [1], [2], [1], [3], [5], [4], [2], [1], [3], [4], [1], [5], [2], [1], [5], [1], [4], [1], [4], [3], [1], [2], [5], [4], [4], [3], [4], [5], [4], [1], [2], [4], [2], [5], [1], [4], [3], [3], [3], [3], [5], [5], [5], [2], [3], [3], [1], [1], [4], [1], [3], [2], [2], [4], [1], [4], [2], [4], [3], [3], [1], [2], [3], [1], [2], [4], [2], [2], [5], [5], [1], [2], [4], [4], [3], [2], [3], [1], [5], [5], [3], [3], [2], [2], [4], [4], [1], [1], [3], [4], [1], [4], [2], [1], [2], [3], [1], [5], [2], [4], [3], [5], [4], [2], [1], [5], [4], [4], [5], [3], [4], [5], [1], [5], [1], [1], [1], [3], [4], [1], [2], [1], [1], [2], [4], [1], [2], [5], [3], [4], [1], [3], [4], [5], [3], [1], [3], [4], [2], [5], [1], [3], [2], [4], [4], [4], [3], [2], [1], [3], [5], [4], [5], [1], [4], [2], [3], [5], [4], [3], [1], [1], [2], [5], [2], [2], [3], [2], [2], [3], [4], [5], [3], [5], [5], [2], [3], [1], [3], [5], [1], [5], [3], [5], [5], [5], [2], [1], [3], [1], [5], [4], [4], [2], [3], [5], [2], [1], [2], [3], [3], [2], [1], [4], [4], [4], [2], [3], [3], [2], [1], [1], [5], [2], [1], [1], [3], [3], [3], [5], [3], [2], [4], [2], [3], [5], [5], [2], [1], [3], [5], [1], [5], [3], [3], [2], [3], [1], [5], [5], [4], [4], [4], [4], [3], [4], [2], [4], [1], [1], [5], [2], [4], [5], [2], [4], [1], [4], [5], [5], [3], [3], [1], [2], [2], [4], [5], [1], [3], [2], [4], [5], [3], [1], [5], [3], [3], [4], [1], [3], [2], [3], [5], [4], [1], [3], [5], [5], [2], [1], [4], [4], [1], [5], [4], [3], [4], [1], [3], [3], [1], [5], [1], [3], [1], [4], [5], [1], [5], [2], [2], [5], [5], [5], [4], [1], [2], [2], [3], [3], [2], [3], [5], [1], [1], [4], [3], [1], [2], [1], [2], [4], [1], [1], [2], [5], [1], [1], [4], [1], [2], [3], [2], [5], [4], [5], [3], [2], [5], [3], [5], [3], [3], [2], [1], [1], [1], [4], [4], [1], [3], [5], [4], [1], [5], [2], [5], [3], [2], [1], [4], [2], [1], [3], [2], [5], [5], [5], [3], [5], [3], [5], [1], [5], [1], [3], [3], [2], [3], [4], [1], [4], [1], [2], [3], [4], [5], [5], [3], [5], [3], [1], [1], [3], [2], [4], [1], [3], [3], [5], [1], [3], [3], [2], [4], [4], [2], [4], [1], [1], [2], [3], [2], [4], [1], [4], [3], [5], [1], [2], [1], [5], [4], [4], [1], [3], [1], [2], [1], [2], [1], [1], [5], [5], [2], [4], [4], [2], [4], [2], [2], [1], [1], [3], [1], [4], [1], [4], [1], [1], [2], [2], [4], [1], [2], [4], [4], [3], [1], [2], [5], [5], [4], [3], [1], [1], [4], [2], [4], [5], [5], [3], [3], [2], [5], [1], [5], [5], [2], [1], [3], [4], [2], [1], [5], [4], [3], [3], [1], [1], [2], [2], [2], [2], [2], [5], [2], [3], [3], [4], [4], [5], [3], [5], [2], [3], [1], [1], [2], [4], [2], [4], [1], [2], [2], [3], [1], [1], [3], [3], [5], [5], [3], [2], [3], [3], [2], [4], [3], [3], [3], [3], [3], [5], [5], [4], [3], [1], [3], [1], [4], [1], [1], [1], [5], [4], [5], [4], [1], [4], [1], [1], [5], [5], [2], [5], [5], [3], [2], [1], [4], [4], [3], [2], [1], [2], [5], [1], [3], [5], [1], [1], [2], [3], [4], [4], [2], [2], [1], [3], [5], [1], [1], [3], [5], [4], [1], [5], [2], [3], [1], [3], [4], [5], [1], [3], [2], [5], [3], [5], [3], [1], [3], [2], [2], [3], [2], [4], [1], [2], [5], [2], [1], [1], [5], [4], [3], [4], [3], [3], [1], [1], [1], [2], [4], [5], [2], [1], [2], [1], [2], [4], [2], [2], [2], [2], [1], [1], [1], [2], [2], [5], [2], [2], [2], [1], [1], [1], [4], [2], [1], [1], [1], [2], [5], [4], [4], [4], [3], [2], [2], [4], [2], [4], [1], [1], [3], [3], [3], [1], [1], [3], [3], [4], [2], [1], [1], [1], [1], [2], [1], [2], [2], [2], [2], [1], [3], [1], [4], [4], [1], [4], [2], [5], [2], [1], [2], [4], [4], [3], [5], [2], [5], [2], [4], [3], [5], [3], [5], [5], [4], [2], [4], [4], [2], [3], [1], [5], [2], [3], [5], [2], [4], [1], [4], [3], [1], [3], [2], [3], [3], [2], [2], [2], [4], [3], [2], [3], [2], [5], [3], [1], [3], [3], [1], [5], [4], [4], [2], [4], [1], [2], [2], [3], [1], [4], [4], [4], [1], [5], [1], [3], [2], [3], [3], [5], [4], [2], [4], [1], [5], [5], [1], [2], [5], [4], [4], [1], [5], [2], [3], [3], [3], [4], [4], [2], [3], [2], [3], [3], [5], [1], [4], [2], [4], [5], [4], [4], [1], [3], [1], [1], [3], [5], [5], [2], [3], [3], [1], [2], [2], [4], [2], [4], [4], [1], [2], [3], [1], [2], [2], [1], [4], [1], [4], [5], [1], [1], [5], [2], [4], [1], [1], [3], [4], [2], [3], [1], [1], [3], [5], [4], [4], [4], [2], [1], [5], [5], [4], [2], [3], [4], [1], [1], [4], [4], [3], [2], [1], [5], [5], [1], [5], [4], [4], [2], [2], [2], [1], [1], [4], [1], [2], [4], [2], [2], [1], [2], [3], [2], [2], [4], [2], [4], [3], [4], [5], [3], [4], [5], [1], [3], [5], [2], [4], [2], [4], [5], [4], [1], [2], [2], [3], [5], [3], [1]]\n", 255 | "{'sport': 1, 'business': 2, 'politics': 3, 'tech': 4, 'entertainment': 5}\n" 256 | ], 257 | "name": "stdout" 258 | } 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "metadata": { 264 | "id": "HgFSr5ZdPhlb", 265 | "colab_type": "code", 266 | "colab": {} 267 | }, 268 | "source": [ 269 | "" 270 | ], 271 | "execution_count": 0, 272 | "outputs": [] 273 | } 274 | ] 275 | } -------------------------------------------------------------------------------- /C3W2_Tokenizer_Embedding_2_BBC_news_archive.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "C3W2-Tokenizer-Embedding-2-BBC news archive", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "view-in-github", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "\"Open" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "metadata": { 31 | "id": "zX4Kg8DUTKWO", 32 | "colab_type": "code", 33 | "colab": {} 34 | }, 35 | "source": [ 36 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 37 | "# you may not use this file except in compliance with the License.\n", 38 | "# You may obtain a copy of the License at\n", 39 | "#\n", 40 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 41 | "#\n", 42 | "# Unless required by applicable law or agreed to in writing, software\n", 43 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 44 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 45 | "# See the License for the specific language governing permissions and\n", 46 | "# limitations under the License." 47 | ], 48 | "execution_count": 0, 49 | "outputs": [] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": { 54 | "colab_type": "text", 55 | "id": "view-in-github" 56 | }, 57 | "source": [ 58 | "\"Open" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "metadata": { 64 | "colab_type": "code", 65 | "id": "gnwiOnGyW5JK", 66 | "outputId": "4a819899-0b8a-4090-fa14-852848616b72", 67 | "colab": { 68 | "base_uri": "https://localhost:8080/", 69 | "height": 204 70 | } 71 | }, 72 | "source": [ 73 | "import csv\n", 74 | "import tensorflow as tf\n", 75 | "import numpy as np\n", 76 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 77 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 78 | "\n", 79 | "!wget --no-check-certificate \\\n", 80 | " https://storage.googleapis.com/laurencemoroney-blog.appspot.com/bbc-text.csv \\\n", 81 | " -O /tmp/bbc-text.csv" 82 | ], 83 | "execution_count": 0, 84 | "outputs": [ 85 | { 86 | "output_type": "stream", 87 | "text": [ 88 | "--2020-04-22 01:08:25-- https://storage.googleapis.com/laurencemoroney-blog.appspot.com/bbc-text.csv\n", 89 | "Resolving storage.googleapis.com (storage.googleapis.com)... 108.177.119.128, 2a00:1450:4013:c08::80\n", 90 | "Connecting to storage.googleapis.com (storage.googleapis.com)|108.177.119.128|:443... connected.\n", 91 | "HTTP request sent, awaiting response... 200 OK\n", 92 | "Length: 5057493 (4.8M) [application/octet-stream]\n", 93 | "Saving to: ‘/tmp/bbc-text.csv’\n", 94 | "\n", 95 | "\r/tmp/bbc-text.csv 0%[ ] 0 --.-KB/s \r/tmp/bbc-text.csv 100%[===================>] 4.82M --.-KB/s in 0.04s \n", 96 | "\n", 97 | "2020-04-22 01:08:25 (131 MB/s) - ‘/tmp/bbc-text.csv’ saved [5057493/5057493]\n", 98 | "\n" 99 | ], 100 | "name": "stdout" 101 | } 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "metadata": { 107 | "colab_type": "code", 108 | "id": "EYo6A4v5ZABQ", 109 | "colab": {} 110 | }, 111 | "source": [ 112 | "vocab_size = 1000\n", 113 | "embedding_dim = 16\n", 114 | "max_length = 120\n", 115 | "trunc_type = 'post'\n", 116 | "padding_type = 'post'\n", 117 | "oov_tok = ''\n", 118 | "training_portion = .8" 119 | ], 120 | "execution_count": 0, 121 | "outputs": [] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "metadata": { 126 | "colab_type": "code", 127 | "id": "iU1qq3_SZBx_", 128 | "outputId": "877a9493-b1cd-4ca0-a190-2165b186a5b7", 129 | "colab": { 130 | "base_uri": "https://localhost:8080/", 131 | "height": 34 132 | } 133 | }, 134 | "source": [ 135 | "sentences = []\n", 136 | "labels = []\n", 137 | "stopwords = [ \"a\", \"about\", \"above\", \"after\", \"again\", \"against\", \"all\", \"am\", \"an\", \"and\", \"any\", \"are\", \"as\", \"at\", \"be\", \"because\", \"been\", \"before\", \"being\", \"below\", \"between\", \"both\", \"but\", \"by\", \"could\", \"did\", \"do\", \"does\", \"doing\", \"down\", \"during\", \"each\", \"few\", \"for\", \"from\", \"further\", \"had\", \"has\", \"have\", \"having\", \"he\", \"he'd\", \"he'll\", \"he's\", \"her\", \"here\", \"here's\", \"hers\", \"herself\", \"him\", \"himself\", \"his\", \"how\", \"how's\", \"i\", \"i'd\", \"i'll\", \"i'm\", \"i've\", \"if\", \"in\", \"into\", \"is\", \"it\", \"it's\", \"its\", \"itself\", \"let's\", \"me\", \"more\", \"most\", \"my\", \"myself\", \"nor\", \"of\", \"on\", \"once\", \"only\", \"or\", \"other\", \"ought\", \"our\", \"ours\", \"ourselves\", \"out\", \"over\", \"own\", \"same\", \"she\", \"she'd\", \"she'll\", \"she's\", \"should\", \"so\", \"some\", \"such\", \"than\", \"that\", \"that's\", \"the\", \"their\", \"theirs\", \"them\", \"themselves\", \"then\", \"there\", \"there's\", \"these\", \"they\", \"they'd\", \"they'll\", \"they're\", \"they've\", \"this\", \"those\", \"through\", \"to\", \"too\", \"under\", \"until\", \"up\", \"very\", \"was\", \"we\", \"we'd\", \"we'll\", \"we're\", \"we've\", \"were\", \"what\", \"what's\", \"when\", \"when's\", \"where\", \"where's\", \"which\", \"while\", \"who\", \"who's\", \"whom\", \"why\", \"why's\", \"with\", \"would\", \"you\", \"you'd\", \"you'll\", \"you're\", \"you've\", \"your\", \"yours\", \"yourself\", \"yourselves\" ]\n", 138 | "print(len(stopwords))\n", 139 | "# Expected Output\n", 140 | "# 153" 141 | ], 142 | "execution_count": 0, 143 | "outputs": [ 144 | { 145 | "output_type": "stream", 146 | "text": [ 147 | "153\n" 148 | ], 149 | "name": "stdout" 150 | } 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "metadata": { 156 | "colab_type": "code", 157 | "id": "eutB2xMiZD0e", 158 | "outputId": "e30e3a65-2b31-4213-dff6-7e5df3a529ad", 159 | "colab": { 160 | "base_uri": "https://localhost:8080/", 161 | "height": 88 162 | } 163 | }, 164 | "source": [ 165 | "with open(\"/tmp/bbc-text.csv\", 'r') as csvfile:\n", 166 | " csvreader = csv.reader(csvfile, delimiter=',')\n", 167 | " next(csvreader)\n", 168 | "\n", 169 | " for row in csvreader:\n", 170 | " labels.append(row[0])\n", 171 | "\n", 172 | " sentence = row[1]\n", 173 | " for word in stopwords:\n", 174 | " token = \" \" + word + \" \"\n", 175 | " sentence = sentence.replace(token, \" \")\n", 176 | " \n", 177 | " sentences.append(sentence)\n", 178 | "\n", 179 | "print(len(labels))\n", 180 | "print(len(sentences))\n", 181 | "print(sentences[0])\n", 182 | "# Expected Output\n", 183 | "# 2225\n", 184 | "# 2225\n", 185 | "# tv future hands viewers home theatre systems plasma high-definition tvs digital video recorders moving living room way people watch tv will radically different five years time. according expert panel gathered annual consumer electronics show las vegas discuss new technologies will impact one favourite pastimes. us leading trend programmes content will delivered viewers via home networks cable satellite telecoms companies broadband service providers front rooms portable devices. one talked-about technologies ces digital personal video recorders (dvr pvr). set-top boxes like us s tivo uk s sky+ system allow people record store play pause forward wind tv programmes want. essentially technology allows much personalised tv. also built-in high-definition tv sets big business japan us slower take off europe lack high-definition programming. not can people forward wind adverts can also forget abiding network channel schedules putting together a-la-carte entertainment. us networks cable satellite companies worried means terms advertising revenues well brand identity viewer loyalty channels. although us leads technology moment also concern raised europe particularly growing uptake services like sky+. happens today will see nine months years time uk adam hume bbc broadcast s futurologist told bbc news website. likes bbc no issues lost advertising revenue yet. pressing issue moment commercial uk broadcasters brand loyalty important everyone. will talking content brands rather network brands said tim hanlon brand communications firm starcom mediavest. reality broadband connections anybody can producer content. added: challenge now hard promote programme much choice. means said stacey jolna senior vice president tv guide tv group way people find content want watch simplified tv viewers. means networks us terms channels take leaf google s book search engine future instead scheduler help people find want watch. kind channel model might work younger ipod generation used taking control gadgets play them. might not suit everyone panel recognised. older generations comfortable familiar schedules channel brands know getting. perhaps not want much choice put hands mr hanlon suggested. end kids just diapers pushing buttons already - everything possible available said mr hanlon. ultimately consumer will tell market want. 50 000 new gadgets technologies showcased ces many enhancing tv-watching experience. high-definition tv sets everywhere many new models lcd (liquid crystal display) tvs launched dvr capability built instead external boxes. one example launched show humax s 26-inch lcd tv 80-hour tivo dvr dvd recorder. one us s biggest satellite tv companies directtv even launched branded dvr show 100-hours recording capability instant replay search function. set can pause rewind tv 90 hours. microsoft chief bill gates announced pre-show keynote speech partnership tivo called tivotogo means people can play recorded programmes windows pcs mobile devices. reflect increasing trend freeing multimedia people can watch want want." 186 | ], 187 | "execution_count": 0, 188 | "outputs": [ 189 | { 190 | "output_type": "stream", 191 | "text": [ 192 | "2225\n", 193 | "2225\n", 194 | "tv future hands viewers home theatre systems plasma high-definition tvs digital video recorders moving living room way people watch tv will radically different five years time. according expert panel gathered annual consumer electronics show las vegas discuss new technologies will impact one favourite pastimes. us leading trend programmes content will delivered viewers via home networks cable satellite telecoms companies broadband service providers front rooms portable devices. one talked-about technologies ces digital personal video recorders (dvr pvr). set-top boxes like us s tivo uk s sky+ system allow people record store play pause forward wind tv programmes want. essentially technology allows much personalised tv. also built-in high-definition tv sets big business japan us slower take off europe lack high-definition programming. not can people forward wind adverts can also forget abiding network channel schedules putting together a-la-carte entertainment. us networks cable satellite companies worried means terms advertising revenues well brand identity viewer loyalty channels. although us leads technology moment also concern raised europe particularly growing uptake services like sky+. happens today will see nine months years time uk adam hume bbc broadcast s futurologist told bbc news website. likes bbc no issues lost advertising revenue yet. pressing issue moment commercial uk broadcasters brand loyalty important everyone. will talking content brands rather network brands said tim hanlon brand communications firm starcom mediavest. reality broadband connections anybody can producer content. added: challenge now hard promote programme much choice. means said stacey jolna senior vice president tv guide tv group way people find content want watch simplified tv viewers. means networks us terms channels take leaf google s book search engine future instead scheduler help people find want watch. kind channel model might work younger ipod generation used taking control gadgets play them. might not suit everyone panel recognised. older generations comfortable familiar schedules channel brands know getting. perhaps not want much choice put hands mr hanlon suggested. end kids just diapers pushing buttons already - everything possible available said mr hanlon. ultimately consumer will tell market want. 50 000 new gadgets technologies showcased ces many enhancing tv-watching experience. high-definition tv sets everywhere many new models lcd (liquid crystal display) tvs launched dvr capability built instead external boxes. one example launched show humax s 26-inch lcd tv 80-hour tivo dvr dvd recorder. one us s biggest satellite tv companies directtv even launched branded dvr show 100-hours recording capability instant replay search function. set can pause rewind tv 90 hours. microsoft chief bill gates announced pre-show keynote speech partnership tivo called tivotogo means people can play recorded programmes windows pcs mobile devices. reflect increasing trend freeing multimedia people can watch want want.\n" 195 | ], 196 | "name": "stdout" 197 | } 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "metadata": { 203 | "colab_type": "code", 204 | "id": "XfdaWh06ZGe3", 205 | "outputId": "f03091ec-dfbe-4be2-966e-8f2230cc7ad9", 206 | "colab": { 207 | "base_uri": "https://localhost:8080/", 208 | "height": 102 209 | } 210 | }, 211 | "source": [ 212 | "train_size = int(training_portion * len(sentences))\n", 213 | "\n", 214 | "train_sentences = sentences[:train_size]\n", 215 | "train_labels = labels[:train_size]\n", 216 | "\n", 217 | "validation_sentences = sentences[train_size:]\n", 218 | "validation_labels = labels[train_size:]\n", 219 | "\n", 220 | "print(train_size)\n", 221 | "print(len(train_sentences))\n", 222 | "print(len(train_labels))\n", 223 | "print(len(validation_sentences))\n", 224 | "print(len(validation_labels))\n", 225 | "\n", 226 | "# Expected output (if training_portion=.8)\n", 227 | "# 1780\n", 228 | "# 1780\n", 229 | "# 1780\n", 230 | "# 445\n", 231 | "# 445" 232 | ], 233 | "execution_count": 0, 234 | "outputs": [ 235 | { 236 | "output_type": "stream", 237 | "text": [ 238 | "1780\n", 239 | "1780\n", 240 | "1780\n", 241 | "445\n", 242 | "445\n" 243 | ], 244 | "name": "stdout" 245 | } 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "metadata": { 251 | "colab_type": "code", 252 | "id": "ULzA8xhwZI22", 253 | "outputId": "a34cd4b4-f80c-4a78-db16-f2e92c5ae26c", 254 | "colab": { 255 | "base_uri": "https://localhost:8080/", 256 | "height": 119 257 | } 258 | }, 259 | "source": [ 260 | "tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\n", 261 | "tokenizer.fit_on_texts(train_sentences)\n", 262 | "word_index = tokenizer.word_index\n", 263 | "\n", 264 | "train_sequences = tokenizer.texts_to_sequences(train_sentences)\n", 265 | "train_padded = pad_sequences(train_sequences, padding=padding_type, maxlen=max_length, truncating=trunc_type)\n", 266 | "\n", 267 | "print(len(train_sequences[0]))\n", 268 | "print(len(train_padded[0]))\n", 269 | "\n", 270 | "print(len(train_sequences[1]))\n", 271 | "print(len(train_padded[1]))\n", 272 | "\n", 273 | "print(len(train_sequences[10]))\n", 274 | "print(len(train_padded[10]))\n", 275 | "\n", 276 | "# Expected Ouput\n", 277 | "# 449\n", 278 | "# 120\n", 279 | "# 200\n", 280 | "# 120\n", 281 | "# 192\n", 282 | "# 120" 283 | ], 284 | "execution_count": 0, 285 | "outputs": [ 286 | { 287 | "output_type": "stream", 288 | "text": [ 289 | "449\n", 290 | "120\n", 291 | "200\n", 292 | "120\n", 293 | "192\n", 294 | "120\n" 295 | ], 296 | "name": "stdout" 297 | } 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "metadata": { 303 | "colab_type": "code", 304 | "id": "c8PeFWzPZLW_", 305 | "outputId": "aaae66c9-b6a3-4dc1-f6db-de4063fcfe93", 306 | "colab": { 307 | "base_uri": "https://localhost:8080/", 308 | "height": 51 309 | } 310 | }, 311 | "source": [ 312 | "validation_sequences = tokenizer.texts_to_sequences(validation_sentences)\n", 313 | "validation_padded = pad_sequences(validation_sequences, padding=padding_type, maxlen=max_length, truncating=trunc_type)\n", 314 | "\n", 315 | "print(len(validation_sequences))\n", 316 | "print(validation_padded.shape)\n", 317 | "\n", 318 | "# Expected output\n", 319 | "# 445\n", 320 | "# (445, 120)" 321 | ], 322 | "execution_count": 0, 323 | "outputs": [ 324 | { 325 | "output_type": "stream", 326 | "text": [ 327 | "445\n", 328 | "(445, 120)\n" 329 | ], 330 | "name": "stdout" 331 | } 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "metadata": { 337 | "colab_type": "code", 338 | "id": "XkWiQ_FKZNp2", 339 | "outputId": "e20a8737-1791-4f26-b739-9b9b09286050", 340 | "colab": { 341 | "base_uri": "https://localhost:8080/", 342 | "height": 153 343 | } 344 | }, 345 | "source": [ 346 | "label_tokenizer = Tokenizer()\n", 347 | "label_tokenizer.fit_on_texts(labels)\n", 348 | "\n", 349 | "training_label_seq = np.array(label_tokenizer.texts_to_sequences(train_labels))\n", 350 | "validation_label_seq = np.array(label_tokenizer.texts_to_sequences(validation_labels))\n", 351 | "\n", 352 | "print(training_label_seq[0])\n", 353 | "print(training_label_seq[1])\n", 354 | "print(training_label_seq[2])\n", 355 | "print(training_label_seq.shape)\n", 356 | "\n", 357 | "print(validation_label_seq[0])\n", 358 | "print(validation_label_seq[1])\n", 359 | "print(validation_label_seq[2])\n", 360 | "print(validation_label_seq.shape)\n", 361 | "\n", 362 | "# Expected output\n", 363 | "# [4]\n", 364 | "# [2]\n", 365 | "# [1]\n", 366 | "# (1780, 1)\n", 367 | "# [5]\n", 368 | "# [4]\n", 369 | "# [3]\n", 370 | "# (445, 1)" 371 | ], 372 | "execution_count": 0, 373 | "outputs": [ 374 | { 375 | "output_type": "stream", 376 | "text": [ 377 | "[4]\n", 378 | "[2]\n", 379 | "[1]\n", 380 | "(1780, 1)\n", 381 | "[5]\n", 382 | "[4]\n", 383 | "[3]\n", 384 | "(445, 1)\n" 385 | ], 386 | "name": "stdout" 387 | } 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "metadata": { 393 | "colab_type": "code", 394 | "id": "HZ5um4MWZP-W", 395 | "outputId": "7b9915b6-3cc4-495e-8d78-ed9d43e0abd5", 396 | "colab": { 397 | "base_uri": "https://localhost:8080/", 398 | "height": 289 399 | } 400 | }, 401 | "source": [ 402 | "model = tf.keras.Sequential([\n", 403 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 404 | " tf.keras.layers.GlobalAveragePooling1D(),\n", 405 | " tf.keras.layers.Dense(24, activation='relu'),\n", 406 | " tf.keras.layers.Dense(len(set(train_labels))+1, activation='softmax')\n", 407 | "])\n", 408 | "model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 409 | "model.summary()\n", 410 | "\n", 411 | "# Expected Output\n", 412 | "# Layer (type) Output Shape Param # \n", 413 | "# =================================================================\n", 414 | "# embedding (Embedding) (None, 120, 16) 16000 \n", 415 | "# _________________________________________________________________\n", 416 | "# global_average_pooling1d (Gl (None, 16) 0 \n", 417 | "# _________________________________________________________________\n", 418 | "# dense (Dense) (None, 24) 408 \n", 419 | "# _________________________________________________________________\n", 420 | "# dense_1 (Dense) (None, 6) 150 \n", 421 | "# =================================================================\n", 422 | "# Total params: 16,558\n", 423 | "# Trainable params: 16,558\n", 424 | "# Non-trainable params: 0" 425 | ], 426 | "execution_count": 0, 427 | "outputs": [ 428 | { 429 | "output_type": "stream", 430 | "text": [ 431 | "Model: \"sequential_3\"\n", 432 | "_________________________________________________________________\n", 433 | "Layer (type) Output Shape Param # \n", 434 | "=================================================================\n", 435 | "embedding_3 (Embedding) (None, 120, 16) 16000 \n", 436 | "_________________________________________________________________\n", 437 | "global_average_pooling1d_3 ( (None, 16) 0 \n", 438 | "_________________________________________________________________\n", 439 | "dense_6 (Dense) (None, 24) 408 \n", 440 | "_________________________________________________________________\n", 441 | "dense_7 (Dense) (None, 6) 150 \n", 442 | "=================================================================\n", 443 | "Total params: 16,558\n", 444 | "Trainable params: 16,558\n", 445 | "Non-trainable params: 0\n", 446 | "_________________________________________________________________\n" 447 | ], 448 | "name": "stdout" 449 | } 450 | ] 451 | }, 452 | { 453 | "cell_type": "code", 454 | "metadata": { 455 | "colab_type": "code", 456 | "id": "XsfdxySKZSXu", 457 | "outputId": "3f19813b-b4c7-4456-985f-23daa1db3540", 458 | "colab": { 459 | "base_uri": "https://localhost:8080/", 460 | "height": 357 461 | } 462 | }, 463 | "source": [ 464 | "num_epochs = 10\n", 465 | "history = model.fit(train_padded, training_label_seq, epochs=num_epochs, validation_data=(validation_padded, validation_label_seq)) #verbose=2" 466 | ], 467 | "execution_count": 0, 468 | "outputs": [ 469 | { 470 | "output_type": "stream", 471 | "text": [ 472 | "Epoch 1/10\n", 473 | "56/56 [==============================] - 0s 4ms/step - loss: 1.4198 - accuracy: 0.4522 - val_loss: 1.3583 - val_accuracy: 0.4539\n", 474 | "Epoch 2/10\n", 475 | "56/56 [==============================] - 0s 4ms/step - loss: 1.2588 - accuracy: 0.4567 - val_loss: 1.2008 - val_accuracy: 0.4719\n", 476 | "Epoch 3/10\n", 477 | "56/56 [==============================] - 0s 3ms/step - loss: 1.0968 - accuracy: 0.5219 - val_loss: 1.0517 - val_accuracy: 0.5326\n", 478 | "Epoch 4/10\n", 479 | "56/56 [==============================] - 0s 4ms/step - loss: 0.9391 - accuracy: 0.6713 - val_loss: 0.8956 - val_accuracy: 0.7596\n", 480 | "Epoch 5/10\n", 481 | "56/56 [==============================] - 0s 4ms/step - loss: 0.7693 - accuracy: 0.8421 - val_loss: 0.7351 - val_accuracy: 0.8989\n", 482 | "Epoch 6/10\n", 483 | "56/56 [==============================] - 0s 4ms/step - loss: 0.6077 - accuracy: 0.9236 - val_loss: 0.6037 - val_accuracy: 0.8944\n", 484 | "Epoch 7/10\n", 485 | "56/56 [==============================] - 0s 4ms/step - loss: 0.4773 - accuracy: 0.9478 - val_loss: 0.4925 - val_accuracy: 0.9124\n", 486 | "Epoch 8/10\n", 487 | "56/56 [==============================] - 0s 4ms/step - loss: 0.3799 - accuracy: 0.9528 - val_loss: 0.4184 - val_accuracy: 0.9124\n", 488 | "Epoch 9/10\n", 489 | "56/56 [==============================] - 0s 4ms/step - loss: 0.3102 - accuracy: 0.9618 - val_loss: 0.3637 - val_accuracy: 0.9169\n", 490 | "Epoch 10/10\n", 491 | "56/56 [==============================] - 0s 4ms/step - loss: 0.2599 - accuracy: 0.9652 - val_loss: 0.3254 - val_accuracy: 0.9169\n" 492 | ], 493 | "name": "stdout" 494 | } 495 | ] 496 | }, 497 | { 498 | "cell_type": "code", 499 | "metadata": { 500 | "colab_type": "code", 501 | "id": "dQ0BX2apXS9u", 502 | "outputId": "3ac930f3-7d0e-4825-ebae-1f8acb265b33", 503 | "colab": { 504 | "base_uri": "https://localhost:8080/", 505 | "height": 541 506 | } 507 | }, 508 | "source": [ 509 | "import matplotlib.pyplot as plt\n", 510 | "\n", 511 | "\n", 512 | "def plot_graphs(history, string):\n", 513 | " plt.plot(history.history[string])\n", 514 | " plt.plot(history.history['val_'+string])\n", 515 | " plt.xlabel(\"Epochs\")\n", 516 | " plt.ylabel(string)\n", 517 | " plt.legend([string, 'val_'+string])\n", 518 | " plt.show()\n", 519 | " \n", 520 | "plot_graphs(history, \"accuracy\")\n", 521 | "plot_graphs(history, \"loss\")" 522 | ], 523 | "execution_count": 0, 524 | "outputs": [ 525 | { 526 | "output_type": "display_data", 527 | "data": { 528 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhU1fnA8e+bPWQjO4QtLGGVJRABtSJLUVQEtaWASl1xqQhqF5f+qrRSta17VZSqKKDiVhQpgiAIIoosYZEAYSdhCQlkh8k25/fHHUKAAJOQySQz7+d55mHuvefeeTMk5733nHvPEWMMSimlvJePuwNQSinlXpoIlFLKy2kiUEopL6eJQCmlvJwmAqWU8nJ+7g6gpmJiYkxiYqK7w1BKqUZl7dq1OcaY2Oq2NbpEkJiYyJo1a9wdhlJKNSoisvds27RpSCmlvJwmAqWU8nKaCJRSyss1uj6C6pSVlZGZmYnNZnN3KAoICgqiZcuW+Pv7uzsUpZQTPCIRZGZmEhYWRmJiIiLi7nC8mjGGI0eOkJmZSdu2bd0djlLKCR7RNGSz2YiOjtYk0ACICNHR0Xp1plQj4hGJANAk0IDo/4VSjYtHNA0ppVRjYrcbbOUVHC+t4HhZBbayCmxldo6Xnb7uxLK1bUjnOHq2alrn8WgiUEqpahhjyD1WRlaBjUJbuVUpn1I5n1i2n7mutKJKRX/qdltZBSXl9lrFFBcWqIlAQXl5OX5++t+m1IUotJWRVVDC4QIbWYU2DuWXkFVg43ChjayCEg7l28guLKG0wrkKO8jfh2B/X4L9fQkKcPzr70uTAD+iQnwJDvAl2FEmyPEKdpQ7dZ+TZU7Z7u9LoJ8PPj6uaXbVGqUOXX/99WRkZGCz2Zg0aRJ33303CxYs4PHHH6eiooKYmBi++eYbioqKeOCBB1izZg0iwpNPPsmvfvUrQkNDKSoqAuDTTz9l3rx5vPvuu9x2220EBQWRmprKZZddxpgxY5g0aRI2m43g4GCmT59Op06dqKio4JFHHmHBggX4+Pgwfvx4unXrxiuvvMLnn38OwKJFi3j99deZM2eOO78qpVzCVlbB4YISsgptZBVYlbr1r/U67FguLq04Y9/QQD/iwgOJDwuib9uoyvdx4YFEBPu7pYKuLx6XCP765WbSDhTU6TG7JoTz5HXdzlvunXfeISoqiuPHj3PxxRczcuRIxo8fz/Lly2nbti1Hjx4F4KmnniIiIoJNmzYBkJube95jZ2ZmsnLlSnx9fSkoKOC7777Dz8+PxYsX8/jjj/PZZ58xbdo09uzZw/r16/Hz8+Po0aNERkbyu9/9juzsbGJjY5k+fTp33HHHhX0hStWz8go7OUWlHKqs0B1n7lUr+EIbecfKztg3wM+HeEel3iUhnIGd4qzlcKuSbxYeRFx4EKGBHlcdOs17f3IXeOWVVyrPtDMyMpg2bRoDBgyovJ8+KioKgMWLFzN79uzK/SIjI8977FGjRuHr6wtAfn4+t956K9u3b0dEKCsrqzzuvffeW9l0dOLzxo0bx6xZs7j99tv54YcfmDFjRh39xEo5r6S8gkJbOUW2cgpt5RTayihw/FtUcnJdoa2cQsfy0eISsgpKyCkq4fTp1X19hNjQQOLDA2kd3YS+baOIDw8kLjyI+PAg4h2VfESwv97Jdh4elwicOXN3hW+//ZbFixfzww8/0KRJEwYOHEivXr3YunWr08eo+st6+n34ISEhle//8pe/MGjQIObMmcOePXsYOHDgOY97++23c9111xEUFMSoUaO0j0HViDGGY6WOSrzkROV9stIuOqVSt8oUVilTVFJOga2cUic6SIP8fQgL8icsyI+wIH9iQgO5KCHCUblbZ/Xx4UHERwQSHRKIbyNvkmkotEaoI/n5+URGRtKkSRO2bt3Kjz/+iM1mY/ny5ezevbuyaSgqKoqhQ4fy2muv8dJLLwFW01BkZCTx8fFs2bKFTp06MWfOHMLCws76WS1atADg3XffrVw/dOhQ3nzzTQYNGlTZNBQVFUVCQgIJCQlMmTKFxYsXu/y7UI1TdmEJK3Zk8116DlsOFVJw3KrEi0rKqbCbc+4rYrWxhwX6VVbkMaEBJMaEOCp1P8KD/K0yQSfLhAZa68OC/AgN8sPf12MebWpUNBHUkWHDhvHGG2/QpUsXOnXqRP/+/YmNjWXatGnceOON2O124uLiWLRoEf/3f//H/fffz0UXXYSvry9PPvkkN954I88++yzDhw8nNjaWlJSUyo7j0/3pT3/i1ltvZcqUKVx77bWV6++66y7S09Pp0aMH/v7+jB8/ngkTJgBw8803k52dTZcuXerl+1ANX0l5BWv35LJ8ew7L07NJO2j1rUWFBNCrVVO6NAs7tdKu8j48yI/QQP/KSj4kwK/Rd5h6MzGnN7w1cCkpKeb0iWm2bNmiFdx5TJgwgeTkZO688856+Tz9P2l4jDHszC7mu+3ZLE/P5sddRzleVoGfj9CnTSQDOsYyICmWbgnhWql7IBFZa4xJqW6bXhF4gT59+hASEsLzzz/v7lBUPcs/Vsb3O60z/u+257A/7zgAbWNCGJXSkgFJsfRvH+3Vd8woTQReYe3ate4OQdWT8go76zPyWL49h++2Z7MhIw+7gbBAPy7tEM3vBrVnQFIsraKauDtU1YBoIlCqkcs4eozljuaelTuOUFhSjo9Aj5ZNmTA4iQFJMfRq1RQ/7YhVZ6GJQKlGpqiknB93HmH5dqu5Z3dOMQAJEUFc26M5lyfFclmHaJo2CXBzpKqx0ESgVANntxs2HyioPOtfty+XsgpDsL8v/dtFMa5/GwZ0jKV9bIg+OKVqRROBUg1QVoGtsoN3xY4cjhaXAtC1eTh3/KItVyTF0icxkkA/XzdHqjyBJgKlGoicohKmLd/Fsm3ZbMsqBCAmNIArOsYyoGMMl3WIIS4syM1RKk+kicANqo4yqhRYHb7j3l7F/rzjXJwYxaO9O3N5Ugxdmuk9/cr1NBF4MZ3boGHYnlXIuLd/4lhpObPv7k+fNlHuDkl5Gc+rBb56FA5tqttjNusOVz971s2PPvoorVq14v777wdg8uTJ+Pn5sXTpUnJzcykrK2PKlCmMHDnyvB9VVFTEyJEjq91vxowZPPfcc4gIPXr0YObMmWRlZXHvvfeya9cuAKZOnUpCQgLDhw/n559/BuC5556jqKiIyZMnVw6Gt2LFCsaOHUvHjh2ZMmUKpaWlREdH8/777xMfH1/tnAn5+fls3Lixcoyk//znP6SlpfHiiy9e0NfrzTZk5HHr9J/w8/Hho3suoUvzcHeH5Fnsdigtsl4lRVBaCCWFjvdF1vvKbVXLVNlWWgT2M+cvcIshT0DPMXV+WM9LBG4wevRoHnzwwcpE8PHHH7Nw4UImTpxIeHg4OTk59O/fnxEjRpz3ro6goCDmzJlzxn5paWlMmTKFlStXEhMTUzm3wcSJE7niiiuYM2cOFRUVFBUVnXd+g9LSUk4M05Gbm8uPP/6IiPDWW2/xz3/+k+eff77aORP8/f35+9//zr/+9S/8/f2ZPn06b7755oV+fV5r5Y4cxs9YQ1RoALPu7Eeb6JDz71QfSoshf7+7o7BUlFSpuAurr7BPVOjVlSkrdu5zxAcCwiAwFAJCT/4bGmf969tAqsrwFi45bAP56erQOc7cXSU5OZnDhw9z4MABsrOziYyMpFmzZjz00EMsX74cHx8f9u/fT1ZWFs2aNTvnsYwxPP7442fst2TJEkaNGkVMTAxwcq6BJUuWVM4v4OvrS0RExHkTwejRoyvfZ2ZmMnr0aA4ePEhpaWnl3AlnmzNh8ODBzJs3jy5dulBWVkb37t1r+G0pgIWbD/HAB6kkxjRh5p39iA93UydwmQ2yNsOBdXAg1XplbwVTuzl165Vf8GkVdxiExkNUe2s5MPzUSj0w7CzLYeAfbA2h6qU8LxG4yahRo/j00085dOgQo0eP5v333yc7O5u1a9fi7+9PYmLiGXMMVKe2+1Xl5+eH3X7yD/lccxs88MADPPzww4wYMYJvv/2WyZMnn/PYd911F08//TSdO3fm9ttvr1FcyvLxmgwe/WwjPVs1ZfptF9ffg18VZXB4i6PCd1T8WWlgd8zq1SQGWvSGLiMgukPDqBh9A05W8oFVKu+GdJbuAVz6TYrIMOBlwBd4yxjz7Gnb2wDvALHAUeAWY0ymK2NyldGjRzN+/HhycnJYtmwZH3/8MXFxcfj7+7N06VL27t3r1HHy8/Or3W/w4MHccMMNPPzww0RHR1fONTBkyBCmTp3Kgw8+WNk0FB8fz+HDhzly5AihoaHMmzePYcOGnfXzTsxt8N5771WuP9ucCf369SMjI4N169axcePGC/nKvNJb3+1iyv+2cHlSDG/c0ocQVw32Zq+AnO0nz/IPrLP6zsodJwVBEZCQDJdOgITe1vuIlg2j8lf1zmWJQER8gdeAoUAmsFpE5hpj0qoUew6YYYx5T0QGA88A41wVkyt169aNwsJCWrRoQfPmzbn55pu57rrr6N69OykpKXTu3Nmp45xtv27duvHnP/+ZK664Al9fX5KTk3n33Xd5+eWXufvuu3n77bfx9fVl6tSpXHLJJTzxxBP07duXFi1anPOzJ0+ezKhRo4iMjGTw4MHs3r0b4KxzJgD85je/Yf369U5Nsaksxhie/zqdV5fu4JruzXhxdK+6exjMGMjdDfurNO8c3GC1kwP4h0BCL7j4LqvCT0iGqHZa6atKLpuPQEQuASYbY65yLD8GYIx5pkqZzcAwY0yGWL2o+caYc942ofMRuN/w4cN56KGHGDJkyFnL6P/JSXa74Ym5PzPrx32MTmnF0zd2r/0Ui8ZAwX6rsq9a8dvyrO2+gdZdbi0cZ/kJvSEmCXz0CWRv5675CFoAGVWWM4F+p5XZANyI1Xx0AxAmItHGmCNVC4nI3cDdAK1bt3ZZwOrc8vLy6Nu3Lz179jxnEmh0Dm+FD0dbd5lUbY8+pWMx/MyOyTOWHev8m1SebZeW2/n9Jxv4csMB7rmiHY8O61yz8YCKDp9Z6Rcftrb5+EFcV+g68mTFH9cVfP1d8CUpT+bu3pY/AK+KyG3AcmA/cMYNu8aYacA0sK4I6jNAV9m0aRPjxp3aChYYGMiqVavcFNH5NW3alPT0dHeHUbfsdvhyItjyodsNp96OeCzHanI5se5EU8v5iA8EhGICQsg57sedpf480iyalnlxMOf0u1dOSygGOLTBUfGvh4ITXWYCsZ2hwy9PVvrxF4G/DjmhLpwrE8F+oFWV5ZaOdZWMMQewrggQkVDgV8aYvNp8mDGmUY282L17d9avX+/uMFyiUU1/uu5dyFgFI1+H5JvPXdZut+5LP+Xe9cJq72svKS5gxebdlJTkkRzvT/Pgcsjdc+p97hWlZ/+sqHbQur9V4bfoDc16WIlCKRdwZSJYDSSJSFusBDAGuKlqARGJAY4aY+zAY1h3ENVYUFAQR44cITo6ulElA09kjOHIkSMEBTWCM9XCQ7BoMiReDr1uOm9xfHwczT9h5yyWXVjCb9/5iR2Fhbw0OpnmPZpXX7C81JFECk4mB3sFxHeFYO2IV/XHZYnAGFMuIhOAhVi3j75jjNksIn8D1hhj5gIDgWdExGA1Dd1fm89q2bIlmZmZZGdn11H06kIEBQXRsmVLd4dxfgsetW6nHP5Snd1Bc2LwuKyCEt6+9WIGdIw9e2G/APCLgiY6tpByL5f2ERhj5gPzT1v3RJX3nwKfXujn+Pv7Vz4Rq5RT0hfC5jkw6P8gpkOdHHJ7ViG3vL2K46UVzLqrH33a6Fm9ahzc3VmsVP0rKYL//d7qfL1sUp0ccn1GHrdN/wl/Xx8+vvcSOjfTweNU46GJQHmfb5+B/Ay4fYHVPHOBvncMHhcTGsisO/vROrpJHQSpVP3RRKC8y4H18OPr0Od2aHPJBR9uwc+HmPhhKm1jQph5Z1/i3DV4nFIXQBOB8h4V5dYzAyGx8MvJF3y4j1dn8Oh/3TB4nFJ1TBOB8h4/vWmNwfPr6RDc9IIO9Z/lu/j7fGvwuDfH9aFJgP4pqcZLf3uVd8jLgCV/h6QrrSeIa8kYw3Nfb+O1pTu5tntzXhjds+4Gj1PKTTQRKM9nDMz/A2Dgmudq/cxAhd3wxBc/8/6qfYzt24op11/A4HFKNSCaCJTnS/sC0hfAlX+HyDa1OkRpuZ2HP17PvI0HuW9ge/50VSd9il15DE0EyrMdz4Ov/gTNe0K/e2t3iNIK7p21lmXp2Tx2dWfuuaJ9HQeplHtpIlCe7Zu/QnE23PRRraY2zD9Wxh3vrSZ1Xy7P3tidMX11GHTleTQRKM+1bxWseQf6/84axbOGDhfa+O3bP7Ezu4hXb+rNNd3PMnicUo2cJgLlmcpL4ctJENEKBv25xrtnHD3GLW+vIruwhHduu5jLk84xeJxSjZwmAuWZVr4M2Vtg7Ec1Hsc/PauQcW+vwlZmZ9Zd/ejdWgePU55NE4HyPEd2wrJ/WVM4dhpWo11T9+Vy+7urCfD14eN7LqFTs3PPPaCUJ9BEoDyLMTDvQfALhGH/qNGuK7bncPfMNcSGWYPHtYrSweOUd9BEoDzLhtmwezlc+zyEO9+5u+Dng0z8cD3tYkOYcYcOHqe8iyYC5TmKj8DCx6FVP+hzh9O7bcrM53fvr6NXq6ZMv60vEU38XRikUg2PJgLlOb7+szX/7/CXrPmFnfT8om1EBPvz7h19CQ/SJKC8j/N/LUo1ZLu+hQ0fWjOOxXd1ere1e4/y7bZs7rmivSYB5bU0EajGr+w4zHsIotrBgD/WaNfnv04nJjSQ315SuzGIlPIEmghU47f8OTi6C4a/CP7BTu+2ckcOK3ce4XcD2+t8AsqraSJQjVtWGnz/EvQcC+0GOr2bMYbnF6XTLDyIm/rp+EHKu2kiUI2X3W49MxAYbg0xXQPL0rNZuzeXCYM7EOSvE8so76bXw6rxWjsdMlbB9VMhJNrp3YwxvLAonZaRwfwmpZULA1SqcdArAtU4FR6CxX+FtgOsZqEaWJSWxcbMfCYOSSLAT/8ElNK/AtU4ffUIlNusZwZqMFOY3W5dDbSNCeHG5BYuDFCpxkMTgWp8ti2AtM/hij9CdM1mC5v/80G2HirkwV8m4eerv/5KgSYC1diUFFkT0cd2gUsn1WjXCrvhxUXpdIwPZXiPBBcFqFTjo4lANS5Ln4b8DLjuJfALqNGuX6zfz87sYh76ZUd8fXTieaVO0ESgGo8DqbBqKqTcAa3712jXsgo7Ly3eTreEcK7q1sxFASrVOLk0EYjIMBHZJiI7ROTRara3FpGlIpIqIhtF5BpXxqMasYpymDsRQmJhyJM13v2ztZnsO3qMh4d2xEevBpQ6hcsSgYj4Aq8BVwNdgbEicvpoYP8HfGyMSQbGAK+7Kh7VyK16Aw5thKv/AcFNa7RrSXkFr3yznV6tmjK4c5yLAlSq8XLlFUFfYIcxZpcxphSYDYw8rYwBwh3vI4ADLoxHNVZ5+2Dp3yHpKuh6fY13/2h1Bgfybfz+yo5IDW41VcpbuDIRtAAyqixnOtZVNRm4RUQygfnAA9UdSETuFpE1IrImOzvbFbGqhsoY+N8frPfXPlejZwYAbGUVvLpkB33bRvGLDjEuCFCpxs/dncVjgXeNMS2Ba4CZInJGTMaYacaYFGNMSmxsbL0Hqdxo8xzYvhAG/x80rfngcLN+3MvhwhJ+P1SvBpQ6G1cmgv1A1YFcWjrWVXUn8DGAMeYHIAjQ0zZlOZ4HCx6F5j2h7z013r24pJzXv93J5Ukx9Gvn/FhESnkbVyaC1UCSiLQVkQCszuC5p5XZBwwBEJEuWIlA236UZfFkKM6G614B35qPj/juyj0cLS7l4aEd6z42pTyIyxKBMaYcmAAsBLZg3R20WUT+JiIjHMV+D4wXkQ3Ah8BtxhjjqphUI7LvR2t00X73QUKvGu+ef7yMN5ftZEjnOJJbR7ogQKU8h0uHoTbGzMfqBK667okq79OAy1wZg2qEykvhy0kQ0QoGPV6rQ7y9YjcFtnIe0qsBpc5L5yNQDc/3L0P2VrjpYwgMrfHuucWlvLNiN1df1IyLWkS4IEClPIu77xpS6lQ5O2D5v6znBTpeVatDvLl8F8WlejWglLM0EaiGwxhr6km/IOsJ4lrILizhvZV7GNEzgY7xYXUcoFKeSZuGVMOx4UPY8x0MfxHCajcw3NRvd1JaYWfSkKQ6Dk4pz6VXBKphKM6BhY9Dq/7Q+7ZaHeJQvo1Zq/ZyY3IL2sXWvG9BKW+liUA1DAv/bE06c91L4FO7X8tXl27HGMNEvRpQqkY0ESj327kUNs6GyyZBXJdaHSLj6DE+Wp3Bb1Ja0SqqSR0HqJRn00Sg3KvsOMx7CKLawYA/1Pow/16yHRFhwuAOdRicUt5BO4uVey3/F+Tuht/OBf/gWh1id04xn63bz28vaUPziNodQylv5tQVgYj8V0SurW5kUKVqregwfP8K9BwL7a6o9WFeXpxOgK8P9w1sX4fBKeU9nK3YXwduAraLyLMi0smFMSlvsWE22MvgFw/V+hDbswr5YsMBfntpG+LCguowOKW8h1OJwBiz2BhzM9Ab2AMsFpGVInK7iPi7MkDloYyB1JnQsi/E1v684sXF6YQE+HHvAL0aUKq2nG7qEZFo4DbgLiAVeBkrMSxySWTKs2X8BDnp0HtcrQ+x+UA+8zcd4o7LEokMCajD4JTyLk51FovIHKATMBO4zhhz0LHpIxFZ46rglAdLnQH+IdDthlof4sVF6YQH+XHn5e3qMDClvI+zdw29YoxZWt0GY0xKHcajvEFJIfw8By66AQJrNx5Q6r5cFm85zB+u7EhEsLZOKnUhnG0a6ioiTU8siEikiPzORTEpT7d5DpQVQ/Jva32IFxalExUSwG2Xta3DwJTyTs4mgvHGmLwTC8aYXGC8a0JSHm/dTIjpCK361mr3n3Yf5bvtOdx7RTtCA/VRGKUulLOJwFdE5MSCiPgC2junai57G2T+BMnj4OSvlNOMMTz39TZiwwIZ1z+x7uNTygs5mwgWYHUMDxGRIVjzCy9wXVjKY62bAT5+1kNktfD9jiP8tPsoEwZ1IDjAt46DU8o7OXtd/QhwD3CfY3kR8JZLIlKeq7zUeois4zAIja3x7sYYnl+0jYSIIMb0beWCAJXyTk4lAmOMHZjqeClVO+kL4FgO9K5dJ/HSbYdJ3ZfHMzd2J9BPrwaUqivOPkeQBDwDdAUqn+M3xugN3Mp5qTMhrDm0H1LjXY0xPP91Oq2jmvDrPi1dEJxS3svZPoLpWFcD5cAgYAYwy1VBKQ9UcAB2LIZeN4Fvze/0Wbj5EJsPFDBpSBL+vjr2oVJ1ydm/qGBjzDeAGGP2GmMmA9e6Lizlcda/D8YOybfUeNcKu+GFRem0iw3h+uQWLghOKe/m7KlZiWMI6u0iMgHYD+iksMo5djukzoLEy60JaGpo3sYDpGcV8e+xyfj61PyWU6XUuTl7RTAJaAJMBPoAtwC3uioo5WH2roDcPdazAzVUXmHn5cXb6dwsjGu7N6/72JRS578icDw8NtoY8wegCLjd5VEpz7JuJgRGQNcRNd51Tup+duUU8+a4Pvjo1YBSLnHeKwJjTAXwi3qIRXmi43mwZS50/3WNp6IsLbfz8jfb6d4igiu7xrsoQKWUs30EqSIyF/gEKD6x0hjzX5dEpTzHpk+g3FareQc+WZtBZu5xnrr+IqQWw1EopZzjbCIIAo4Ag6usM8A5E4GIDMOawMYXeMsY8+xp21/Euh0VrD6IOGNMU5TnSJ0J8d2hea8a7WYrq+Df3+ygd+umDOxY86eQlVLOc/bJ4hr3Czj6Fl4DhgKZwGoRmWuMSaty3IeqlH8ASK7p56gG7OBGOLgBhv2jxgPMffjTPg4V2HjhNz31akApF3P2yeLpWFcApzDG3HGO3foCO4wxuxzHmA2MBNLOUn4s8KQz8ahGInUm+AZAj9/UaLfjpRW8tnQn/dtFcWmHGBcFp5Q6wdmmoXlV3gcBNwAHzrNPCyCjynIm0K+6giLSBmgLLDnL9ruBuwFat27tXMTKvcpssPFj6DwcmkTVaNcZP+whp6iEqbf0dk1sSqlTONs09FnVZRH5EFhRh3GMAT513KFU3edPA6YBpKSknHFlohqgrfPAllfjTuKiknLeWLaTAR1juTixZglEKVU7tR20JQmIO0+Z/UDVsYJbOtZVZwzWHAfKU6ybARGtoe3AGu02fcVuco+V8fuhHV0Tl1LqDM72ERRyah/BIaw5Cs5lNZAkIm2xEsAY4KZqjt0ZiAR+cCYW1Qjk7oHdy2DgY+Dj/LlG/rEypn23i192iadnK715TKn64mzTUFhND2yMKXeMS7QQ6/bRd4wxm0Xkb8AaY8xcR9ExwGxjjDb5eIrU9wGBXjfXaLe3Vuyi0FbOw3o1oFS9cvaK4AZgiTEm37HcFBhojPn8XPsZY+YD809b98Rpy5NrErBq4OwV1kij7QdBU+dnETtaXMo7K3ZzbffmdE0Id2GASqnTOXvd/uSJJABgjMlDb/VU1dm5FAr213iAuTeX7eR4WQUPDU1yUWBKqbNxNhFUV67ms4soz5c6E4KjoLPz01UcLrTx3g97GNmrBR3iatwKqZS6QM4mgjUi8oKItHe8XgDWujIw1QgVH4Gt/4Meo8Ev0OndXl+6k7IKw6QhejWglDs4mwgeAEqBj4DZgA2431VBqUZq40dgL6vRswMH8o7zwap9jOrTksSYEBcGp5Q6G2fvGioGHnVxLKoxM8ZqFkroDfHdnN7t30t2YDBMGNzBhcEppc7FqSsCEVnkuFPoxHKkiCx0XViq0dm/Dg6n1ehqYMfhQj5Zk8GYi1vTMrKJC4NTSp2Ls01DMY47hQAwxuRy/ieLlTdJnQF+wXDRr5wqbozhr1+mERzgy4O/1L4BpdzJ2URgF5HK0d5EJJFqRiNVXqq0GDZ9Bt2uh6AIp3b5Oi2L77bn8PDQjkSHOt+xrJSqe87eAvpnYIWILAMEuBzHaKBKkfYFlBY6/eyArayCp+al0TE+lFv6t3FxcEqp83G2s3iBiKRgVf6pwOfAcVcGphqRdTMhqj20uS89Tz0AABQDSURBVNSp4tOW7yIz9zgf3NUPf9/ajnuolKorzg4xcRcwCWsE0fVAf6xB4gafaz/lBXJ2wL6VMORJp2Yh2593nNe/3cE13ZvppDNKNRDOno5NAi4G9hpjBmFNKZl37l2UV0idCeILvc4YWLZaT8/fAsDj13RxZVRKqRpwNhHYjDE2ABEJNMZsBTq5LizVKFSUw4YPIelKCGt23uIrd+bwv40Hue+KDnq7qFINiLOdxZmO5wg+BxaJSC6w13VhqUZh+9dQlOXUswPlFXb+OjeNFk2DueeKdvUQnFLKWc52Ft/geDtZRJYCEcACl0WlGofUmRASZ10RnMf7q/axLauQN27pTZC/bz0Ep5RyVo1HEDXGLHNFIKqRKTwE6Qvh0gng63/OokeLS3n+621c1iGaq7qdvwlJKVW/9N49VTsbPgRT4dSzA/9auI3i0gomX9cNceLOIqVU/dJEoGrOGEidBa0vgZhzDw/x8/58Zq/ex62XJJIUr3MNKNUQaSJQNbfvBziy47xXA8YYnpy7magmAUzS8YSUarA0EaiaWzcTAsKssYXO4fP1+1m7N5dHhnUmIvjc/QhKKffRRKBqxlYAaZ/DRTdCwNknkikqKeeZ+Vvp0TKCX/dpWY8BKqVqSucdVjXz82dQdgx6//acxV5dsoPDhSW8Oa4PPj7aQaxUQ6ZXBKpmUmdCbBdo0eesRXZlF/H2il38uk9LkltH1mNwSqna0ESgnJeVBvvXWk8Sn+M20KfmpRHo58ufhukoJEo1BpoIlPNSZ4KPP/QYc9YiS7ZmsXRbNpOGJBEXFlSPwSmlaksTgXJOeQlsmA2dr4GQ6GqLlJRX8Lcv02gfG8KtlybWb3xKqVrTzmLlnG3z4fhRSD57J/HbK3az58gxZtzRlwA/PcdQqrHQv1blnHUzIbwltB9U7eZD+TZeXbKDoV3jGdAxtp6DU0pdCE0E6vzyMmDnEmvyGZ/qRw595qstlNsNf7m2az0Hp5S6UJoI1Pmt/wAwkHxztZtX7znKF+sPcM+AdrSO1glnlGpsXJoIRGSYiGwTkR0i8uhZyvxGRNJEZLOIfODKeFQt2O2wfha0vQIiE8/YXGE3PPnFZhIigrhvYPv6j08pdcFc1lksIr7Aa8BQIBNYLSJzjTFpVcokAY8BlxljckUkzlXxqFravQzy9lmT01fjw5/2kXawgFdvSqZJgN57oFRj5Morgr7ADmPMLmNMKTAbGHlamfHAa8aYXABjzGEXxqNqI3UmBDWFzsPP2JR3rJTnvt5Gv7ZRXNu9uRuCU0rVBVcmghZARpXlTMe6qjoCHUXkexH5UUSGVXcgEblbRNaIyJrs7GwXhavOcOwobJkHPX4D/mc+HPbConQKjpcxeYROOKNUY+buzmI/IAkYCIwF/iMiTU8vZIyZZoxJMcakxMbqrYn1ZtMnUFFS7bwDaQcKmPXjXsb1b0OX5uFuCE4pVVdcmQj2A62qLLd0rKsqE5hrjCkzxuwG0rESg3I3Y6xnB5r3hOY9TttkmPzlZiKC/XloaEc3BaiUqiuuTASrgSQRaSsiAcAYYO5pZT7HuhpARGKwmop2uTAm5ayD6yFrU7VXA/M2HuSn3Uf5w1WdaNokwA3BKaXqkssSgTGmHJgALAS2AB8bYzaLyN9EZISj2ELgiIikAUuBPxpjjrgqJlUD62aCXxB0H3XK6mOl5Tw9fwvdEsIZc3FrNwWnlKpLLr3fzxgzH5h/2ronqrw3wMOOl2ooyo7Dpk+hywgIPrXL5vWlOzmYb+PfY5Px1QlnlPII7u4sVg1R2lwoyYfkW05ZvfdIMdOW7+L6XgmkJEa5KTilVF3TRKDOlDoTmraBxMtPWf3UvC34+QqPXdPFTYEppVxBE4E61dFdsOc7q5PY5+Svx7fbDrN4SxYPDE4iPlwnnFHKk2giUKdKfR/Exxpp1KG03M7f5qXRNiaEO36R6L7YlFIuoYlAnWSvsEYabT8EIk4+BP7uyt3syi7mieFdCfSrfhhqpVTjpYlAnbTjGyg8YE1O73C4wMbLi7czuHMcgzrrmIBKeSJNBOqk1BnQJAY6Xl256h8LtlFaYecvw3XCGaU8lSYCZSnKhm1fQc8x4Gc9Lbx2by6frcvkzl+0o21MiJsDVEq5iiYCZdk4G+zllUNK2O2GyXM3Ex8eyAODO7g5OKWUK2kiUCcHmGt5McR1BuCTtRls2p/PY1d3ISRQJ5xRypNpIlCQuRpytlVeDeQfL+OfC7aR0iaSkb0S3BycUsrV9FRPwboZ4B8CF90IwEuL0zl6rJT3RvTVCWeU8gJ6ReDtSopg8xzodgMEhpGeVciMH/Yytm9rLmoR4e7olFL1QBOBt9s8B0qLoPc4a8KZuZsJDfTjD1d2cndkSql6oonA26XOhOgkaNWPBT8fYuXOI/z+yo5EheiEM0p5C00E3ix7G2Ssgt7jOF5mZ8r/ttC5WRg39dUJZ5TyJtpZ7M1SZ4KPH/QcyxvLdrI/7ziz7+6Pn6+eHyjlTfQv3ltVlMGG2dBxGBmlobyxbCfDezSnf7tod0emlKpnmgi8VfoCKM6G5HE8PX8LIvC4TjijlFfSROCNjh2FFS9BaDO+l1589fMh7h/YgYSmwe6OTCnlBtpH4G22L4IvJsCxI5QPf4XJ87bRKiqY8QPauTsypZSb6BWBtygpgnkPwfu/huBIGL+EGccuYfvhIv5ybVeC/HXCGaW8lV4ReIOMn+C/d0PuHrhkAgz+CzklwouLv+XypBiGdo13d4RKKTfSRODJykth2T9gxQsQ3hJumweJv6C03M6TX6zneGkFT17XTccTUsrLaSLwVIe3wn/Hw6GN0OtmGPYsBIWTcfQYEz5MZUNGHn+8qhMd4kLdHalSys00EXgaux1WTYXFf4XAUBg9C7pcB8BXmw7yp882AjD15t5c3b25OyNVSjUQmgg8Sd4++Px3sOc7a97hEa9AaBy2sgqenr+FGT/spWerprw6NplWUU3cHa1SqoHQROAJjLGeEv7qT2DsMOLf1iQzIuzKLmLCB6mkHSxg/OVt+eNVnQnw05vFlFInaSJo7IqPwLxJsOVLaH0JXD8VotoC8MX6/Tz+3034+/nw9q0pDOmidwcppc7k0lNDERkmIttEZIeIPFrN9ttEJFtE1jted7kyHo+TvhBe7w/bFsAv/wq3/Q+i2nK8tIJHPt3IpNnr6ZoQzvyJl2sSUEqdlcuuCETEF3gNGApkAqtFZK4xJu20oh8ZYya4Kg6PVFIECx+Hde9BXDcY919o1h2A9KxCJnywju2Hi5gwqAMP/jJJRxNVSp2TK5uG+gI7jDG7AERkNjASOD0RqJrY9yPMuQdy98Jlk2DQn8EvEGMMn6zN5IkvfiY00I8Zd/Tl8qRYd0erlGoEXJkIWgAZVZYzgX7VlPuViAwA0oGHjDEZ1ZRR5aXw7dPw/csQ0RJunw9tLgWgqKScv3z+M3NS93Np+2heGt2LuPAgNweslGos3N1Z/CXwoTGmRETuAd4DBp9eSETuBu4GaN3aC2fPykqzhojI2mTdDTTsGQgMA2DzgXwe+CCVPUeKeXhoR+4f1AFfH31SWCnlPFcmgv1AqyrLLR3rKhljjlRZfAv4Z3UHMsZMA6YBpKSkmLoNswGzV8APr8GSpyAwHMZ8CJ2vAcAYw6xV+3hqXhqRTfz5YHx/nVRGKVUrrkwEq4EkEWmLlQDGADdVLSAizY0xBx2LI4AtLoynccndC5/fB3u/h87DYfhLEGq1+RfYynj0s43M33SIgZ1ieX5UT6JDA90csFKqsXJZIjDGlIvIBGAh4Au8Y4zZLCJ/A9YYY+YCE0VkBFAOHAVuc1U8jYYxsP59+Mpxt+3I16HXTeAYGG5DRh4TPlzHgTwbj13dmfGXt8NHm4KUUhdAjGlcLS0pKSlmzZo17g7DNYqy4ctJsO1/0OYy6+GwyDaA1RT09ord/GPBVuLCgnhlbDJ92kS6OWClVGMhImuNMSnVbXN3Z7E6Yet8+HIi2PLhyinQ/37wse7/zztWyh8+2cDiLYe5sms8//x1D5o2CXBzwEopT6GJwN1sBbDwMUidBfHd4bdfQHy3ys1r9hxl4oepZBeV8OR1Xbnt0kSdP0ApVac0EbjT3pXWw2H5mfCLh2Hgo+Bndfra7YY3lu/k+a/TadE0mM/uu5QeLZu6OWCllCfSROAO5SWwZAqs/LfVB3D7V9C6f+XmnKISHvpoPd9tz+HaHs155sbuhAf5uzFgpZQn00RQnwoOwv41sPQZOLwZet8KVz1tTSDjsHJnDpNmr6fgeBlP39CdsX1baVOQUsqlNBG4SvEROJAKB9Y5/k2FQscjEyFxMPYj6DSssniF3fDvJdt55ZvtJMaEMOOOvnRpHu6m4JVS3kQTQV2w5cOB9adW/Hn7Tm6PToK2AyAh2Xo17wn+wZWbswpsTJqdyo+7jnJj7xY8NfIiQgL1v0YpVT+0tqmp0mI4uPHUSv/IjpPbm7aBhN5w8V0nK/2giLMebll6Ng9/tJ5jpRU8N6onv+7Tsh5+CKWUOkkTwbmUl0DWz7B/neOMfx1kb7WmgwQIS7Aq+55jHJV+MoQ4N95PWYWdFxalM/XbnXSKD+O1m5PpEBfmwh9GKaWqp4nghIpyyN5ineHvd5zpZ20Ge5m1vUm0dabfeTi06G1V/GHNavVR+/OOM/HDVNbuzWVs39Y8eV1Xgvx96/CHUUop53lnIrDb4cj2Uyv9Qxuh3GZtD4yAhF5wyf0nK/2IVpXj/VSnpLyC4pIKikvKKaryKna8ihzbCm1lfLwmkwq74ZWxyYzomVBPP7RSSlXPexJBxmrY8gXsT4WDG6C0EAC7fxNKYrpT1OlmcptexOGwrmQHJFBUaiiylVOcUU7RjkKKSzZSXHqyQi+yOSr6UquiL6twbsymQD8fureI4LlRPUmMCXHlT6yUUk7xmkSw7ofFXJQ2lXRpy0b7pawrT2SDvT07bQnYC31g94mSRxwvi49ASKAfoYF+hDheoYG+RIc0OWVdWJAfIQG+1ZT1IyTQt3Kdv84frJRqYLwmEeR2HssfywYTHBRMSKAfCYF+dKqssE9W1KGnrQv299UHupRSHs1rEsGQHokM6ZHo7jCUUqrB0XYKpZTycpoIlFLKy2kiUEopL6eJQCmlvJwmAqWU8nKaCJRSystpIlBKKS+niUAppbycGOPcGDkNhYhkA3truXsMkFOH4TR2+n2cSr+Pk/S7OJUnfB9tjDGx1W1odIngQojIGmNMirvjaCj0+ziVfh8n6XdxKk//PrRpSCmlvJwmAqWU8nLelgimuTuABka/j1Pp93GSfhen8ujvw6v6CJRSSp3J264IlFJKnUYTgVJKeTmvSQQiMkxEtonIDhF51N3xuIuItBKRpSKSJiKbRWSSu2NqCETEV0RSRWSeu2NxNxFpKiKfishWEdkiIpe4OyZ3EZGHHH8nP4vIhyIS5O6YXMErEoGI+AKvAVcDXYGxItLVvVG5TTnwe2NMV6A/cL8XfxdVTQK2uDuIBuJlYIExpjPQEy/9XkSkBTARSDHGXAT4AmPcG5VreEUiAPoCO4wxu4wxpcBsYKSbY3ILY8xBY8w6x/tCrD/yFu6Nyr1EpCVwLfCWu2NxNxGJAAYAbwMYY0qNMXnujcqt/IBgEfEDmgAH3ByPS3hLImgBZFRZzsTLKz8AEUkEkoFV7o3E7V4C/gTY3R1IA9AWyAamO5rK3hKREHcH5Q7GmP3Ac8A+4CCQb4z52r1RuYa3JAJ1GhEJBT4DHjTGFLg7HncRkeHAYWPMWnfH0kD4Ab2BqcaYZKAY8Mo+NRGJxGo5aAskACEicot7o3INb0kE+4FWVZZbOtZ5JRHxx0oC7xtj/uvueNzsMmCEiOzBajIcLCKz3BuSW2UCmcaYE1eJn2IlBm/0S2C3MSbbGFMG/Be41M0xuYS3JILVQJKItBWRAKwOn7lujsktRESw2n+3GGNecHc87maMecwY09IYk4j1e7HEGOORZ33OMMYcAjJEpJNj1RAgzY0hudM+oL+INHH83QzBQzvO/dwdQH0wxpSLyARgIVbP/zvGmM1uDstdLgPGAZtEZL1j3ePGmPlujEk1LA8A7ztOmnYBt7s5HrcwxqwSkU+BdVh326XioUNN6BATSinl5bylaUgppdRZaCJQSikvp4lAKaW8nCYCpZTycpoIlFLKy2kiUMpBRCpEZH2VV509USsiiSLyc10dT6m65BXPESjlpOPGmF7uDkKp+qZXBEqdh4jsEZF/isgmEflJRDo41ieKyBIR2Sgi34hIa8f6eBGZIyIbHK8TwxL4ish/HOPbfy0iwY7yEx3zQ2wUkdlu+jGVF9NEoNRJwac1DY2usi3fGNMdeBVrtFKAfwPvGWN6AO8DrzjWvwIsM8b0xBqn58RT7EnAa8aYbkAe8CvH+keBZMdx7nXVD6fU2eiTxUo5iEiRMSa0mvV7gMHGmF2OAfsOGWOiRSQHaG6MKXOsP2iMiRGRbKClMaakyjESgUXGmCTH8iOAvzFmiogsAIqAz4HPjTFFLv5RlTqFXhEo5Rxzlvc1UVLlfQUn++iuxZpBrzew2jEJilL1RhOBUs4ZXeXfHxzvV3Jy6sKbge8c778B7oPKuZAjznZQEfEBWhljlgKPABHAGVclSrmSnnkodVJwlRFZwZq398QtpJEishHrrH6sY90DWDN5/RFrVq8To3ROAqaJyJ1YZ/73Yc1wVR1fYJYjWQjwipdPDancQPsIlDoPRx9BijEmx92xKOUK2jSklFJeTq8IlFLKy+kVgVJKeTlNBEop5eU0ESillJfTRKCUUl5OE4FSSnm5/wd1A9jrKJqfBwAAAABJRU5ErkJggg==\n", 529 | "text/plain": [ 530 | "
" 531 | ] 532 | }, 533 | "metadata": { 534 | "tags": [], 535 | "needs_background": "light" 536 | } 537 | }, 538 | { 539 | "output_type": "display_data", 540 | "data": { 541 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd1yVdf/H8dfnMEUUFQciIu6JoKKpOLNhVmJu0xw5KkfdDe/qru72r+52qWXuLEfmyrS0oWXiRMNtprhwIi4cyPr+/rjIkYJYHC7gfJ6Ph4/kXN9zzofzSN5c3ynGGJRSSrkuh90FKKWUspcGgVJKuTgNAqWUcnEaBEop5eI0CJRSysW5213AzSpdurQJCQmxuwyllCpQ1q9ff9wYU+Z61wpcEISEhBATE2N3GUopVaCIyL6srmnXkFJKuTgNAqWUcnEaBEop5eIK3BiBUso1paamEh8fT3Jyst2l5Gve3t4EBQXh4eGR4+doECilCoT4+HiKFStGSEgIImJ3OfmSMYbExETi4+OpXLlyjp+nXUNKqQIhOTkZf39/DYFsiAj+/v43fdekQaCUKjA0BG7s73xGLhMEJ86l8PI3WzmfkmZ3KUopla84LQhEZJKIHBORLTdo11hE0kSkq7NqAYjedZwpK/fS9ZNVxJ8878y3UkoVUr6+vnaX4BTOvCOYArTProGIuAH/A753Yh0A3BsWyKR+jTlw8jwdR0ezOi7R2W+plFIFgtOCwBizHDhxg2YjgDnAMWfVcaW2tcoyf1gkJXw86DNhDZ+v2oue0KaUulnGGEaOHEm9evUIDQ3lyy+/BODw4cO0atWK8PBw6tWrx6+//kp6ejr9+/e/1Pb999+3ufpr2TZ9VEQqAPcBbYHGefW+Vcv4Mn9YJP+aGcsLX29l66EzvBxVFy93t7wqQSn1D738zVa2HTqTq69ZJ7A4L95bN0dt586dS2xsLBs3buT48eM0btyYVq1aMX36dO68806ee+450tPTOX/+PLGxsRw8eJAtW6xe8lOnTuVq3bnBzsHiD4CnjTEZN2ooIkNEJEZEYhISEv7xGxf39mB83wiGta3KzHUHuH/8Go4l6SIVpVTOrFixgl69euHm5ka5cuVo3bo169ato3HjxkyePJmXXnqJzZs3U6xYMapUqUJcXBwjRoxg8eLFFC9e3O7yr2HngrIIYGbmVKfSQAcRSTPGzP9rQ2PMOGAcQERERK705bg5hJF31qJWQHFGzt5Ix1HRjOvbiPpBJXLj5ZVSTpTT39zzWqtWrVi+fDmLFi2if//+PPHEE/Tt25eNGzeyZMkSxo4dy6xZs5g0aZLdpV7FtjsCY0xlY0yIMSYEmA0MvV4IONu9YYHMeaQ5bg6h69hVzPstPq9LUEoVMC1btuTLL78kPT2dhIQEli9fTpMmTdi3bx/lypVj8ODBDBo0iA0bNnD8+HEyMjLo0qULr732Ghs2bLC7/Gs47Y5ARGYAbYDSIhIPvAh4ABhjxjrrff+OuoF+LBgeydBpG3j8y41sO3SGp9vXwt3NZZZZKKVuwn333ceqVasICwtDRHjrrbcICAjgs88+4+2338bDwwNfX1+mTp3KwYMHGTBgABkZVi/4G2+8YXP115KCNmsmIiLCOOtgmtT0DF5duI2pq/bRsnppRvVqQAkfT6e8l1Lq5mzfvp3atWvbXUaBcL3PSkTWG2MirtfetX7lPXUg28sebg5eiarHm51DWR2XSNSYaHYeTcqj4pRSyh6uEwSbZ8OoRrDh8xs27dkkmJlDmnLuYjr3jYnm+61H8qBApZSyh+sEQdVboVIzWDAcFj4OaSnZNm9UqRTfjIikallfhny+ng9//IOMjILVjaaUUjnhOkHgUwr6zIXIf0HMJJhyN5w5nO1TyvsVYdZDzejcoALv/7iTodM2cO6iblqnlCpcXCcIABxucPvL0G0KHN0K41rD/tXZPsXbw413u4fx/N21+X7bETp/vJL9ibppnVKq8HCtIPhT3ftg0I/gWdS6M1g7HrKZPSUiDGpZhc8ebMKRM8l0HLOCFX8cz8OClVLKeVwzCADK1YHBy6BqO/j2Kfh6GKRmv81Ey+plWDA8krLFvOg7aQ0TV+zRTeuUUgWe6wYBQJES0GsmtH4GYqfBpDtvOMW0kn9R5g6N5Lba5Xh14Tae+moTyanpeVSwUqqgyO7sgr1791KvXr08rCZ7rh0EAA4HtH0Wes6AE3HWuMGe5dk+xdfLnbF9GvFYu+rM2RBPj3GrOXJaN61TShVMdm46l7/U6gCDl8LM3jC1E9z+CjQbBlmc/+lwCI/fXoPa5YvzxKxY7h29grF9GtGoUsk8LlwpF/TdM3Bkc+6+ZkAo3PVmlpefeeYZKlasyLBhwwB46aWXcHd3Z9myZZw8eZLU1FRee+01oqKibuptk5OTeeSRR4iJicHd3Z333nuPtm3bsnXrVgYMGEBKSgoZGRnMmTOHwMBAunfvTnx8POnp6bzwwgv06NHjH33boHcEVytdHQb/ZIXC98/BnIGQci7bp7SvF8C8oZEU8XCj17jVzFqXfdeSUqpg6tGjB7Nmzbr09axZs+jXrx/z5s1jw4YNLFu2jCeffPKmxw3HjBmDiLB582ZmzJhBv379SE5OZuzYsTz22GPExsYSExNDUFAQixcvJjAwkI0bN7Jlyxbat8/2EMgc0zuCv/IqBt0/hxXvw0+vQMLv0ONzKFUly6fUDCjGguGRDJ/+G/+es4lth8/w3N218dBN65Ryjmx+c3eWBg0acOzYMQ4dOkRCQgIlS5YkICCAxx9/nOXLl+NwODh48CBHjx4lICAgx6+7YsUKRowYAUCtWrWoVKkSO3fupFmzZrz++uvEx8fTuXNnqlevTmhoKE8++SRPP/0099xzDy1btsyV701/Ul2PCLR8AvrMhtPxMK4N/PFjtk8p4ePJlAGNGdiiMlNW7qXvxLWcOJf96mWlVMHSrVs3Zs+ezZdffkmPHj2YNm0aCQkJrF+/ntjYWMqVK0dycu6MF95///0sWLCAIkWK0KFDB5YuXUqNGjXYsGEDoaGhPP/887zyyiu58l4aBNmpdhsM+Rn8gmFaV1j+TrbrDdzdHLxwTx3e7RbG+v0n6Th6Ra4fp6eUsk+PHj2YOXMms2fPplu3bpw+fZqyZcvi4eHBsmXL2Ldv302/ZsuWLZk2bRoAO3fuZP/+/dSsWZO4uDiqVKnCo48+SlRUFJs2beLQoUP4+PjQp08fRo4cmWtnG2gQ3EipyjDwewjtCktfhS/7QHL2P9y7NApi1kPNSE3PoMsnK1m0KfutLJRSBUPdunVJSkqiQoUKlC9fnt69exMTE0NoaChTp06lVq1aN/2aQ4cOJSMjg9DQUHr06MGUKVPw8vJi1qxZ1KtXj/DwcLZs2ULfvn3ZvHkzTZo0ITw8nJdffpnnn38+V74vPY8gp4yB1Z/A98+Df1XoOd0aXM7GsTPJPPzFejbsP8XwttV44vYaOBzXn4WklMqenkeQc3oegbOIQLOh0PdrOH8CxrWFHYuyfUrZ4t7MGNKUHhEVGb1sF4OnxnAmOTWPClZKqZzRILhZlVvCQ79A6Wow835Y+jpkHkF3PV7ubrzZJZSXO9bl550J3DcmmriEs3lYsFLKLps3byY8PPyqP7fccovdZV1Dp4/+HX5BMGAxLHoSlr8Fh2Oh8zgocv3FZCJCv+Yh1ChXjKHT1hM1JppRvRrQpmbZPC5cqYLNGINkscgzPwoNDSU2NjZP3/PvdPfrHcHf5eENUaPh7ndh9zKrq+jotmyf0qyqPwuGt6BCiSI8OGUd45fH6aZ1SuWQt7c3iYmJ+m8mG8YYEhMT8fb2vqnn6WBxbti/BmY9ABeTIGoM1OucbfNzF9N46quNfLflCJ0bVOD/Oofi7eGWR8UqVTClpqYSHx+fa/P0Cytvb2+CgoLw8PC46vHsBos1CHJL0hGY1RcOrIHmI6DdS+CWdc9bRoZh9LJdvPfDTsIqlmDcA40oV/zmUlwppXJKZw3lhWIB0G8hNB4EK0fBF53hXGKWzR0O4dF21RnbpxF/HE3i3lEr+G3/yTwsWCmlLBoEucnd0xoziBpjHYE5rg0cyn6gqH29AOYObY6Xh4Me41YzZ3183tSqlFKZNAicoUEfeHAxmAzrsJvYGdk2rxVQnAXDWtAouCRPfrWR1xZuIy096ympSimVmzQInKVCQ2ufoqDGMP9h+HYkpGe9mKxkUU+mDmxCv2aVmLBiDwOmrOP0eV18ppRyPqcFgYhMEpFjIrIli+u9RWSTiGwWkZUiEuasWmzjWwYemA/NhsPacfBZR0g6mmVzDzcHL0fV483OoayOS6TTx9HsOpaUhwUrpVyRM+8IpgDZnZqwB2htjAkFXgXGObEW+7i5w52vQ+cJcOg36yjMA+uyfUrPJsFMH9yUpORUOo1ZydIdWYeHUkr9U04LAmPMcuBENtdXGmP+nCazGghyVi35Qv1uMOgHcPOEyXfB6rHZbmndOKQUXw9vQSV/HwZ+FsPHP+/ShTRKKafIL2MEA4HvsrooIkNEJEZEYhISEvKwrFwWEGqNG1S7DRY/bW1pfeFUls0rlCjC7Iebc3doed5a/DuPzYzlQkp6npWrlHINtgeBiLTFCoKns2pjjBlnjIkwxkSUKVMm74pzBp9S0GsG3PE67FwMn7aCg+uzbF7E041RvRow8s6afLPpEN0/XcXh0xfysGClVGFnaxCISH1gAhBljMl69VVhIwLNh8OA7yAjHSbeCWs+zbKrSEQY1rYa4x+IYM/xc9w7Kpr1+7LsdVNKqZtiWxCISDAwF3jAGLPTrjpsVbEJPPwrVGsH3/3b2qIi+XSWzW+rU455Q5vj6+VGz3GrmbXuQB4Wq5QqrJw5fXQGsAqoKSLxIjJQRB4WkYczm/wX8Ac+FpFYEcmHGwjlAZ9S0Gsm3P6qddDNp62s2UVZqF6uGPOHRdK0ij//nrOJlxZs1cVnSql/RDedy0/2r4HZD8K5Y9YYQpPBVjfSdaSlZ/DGdzuYuGIPkdX8Gd2rISWLeuZxwUqpgkI3nSsogm+xuoqqtIXvRsJX/bLsKnJ3c/DCPXV4u2t91u05SdSYaHYe1cVnSqmbp0GQ31zqKnoFti/M7CrKeuO6bhEVmflQUy6kpnPfmGi+33okD4tVShUGGgT5kcMBkY/BgG+t/Ykm3g5rx2c5q6hhcEm+Gd6CamV9GfL5ekb99IcuPlNK5ZgGQX4W3BQe+hWqtIFvn4Kv+mfZVRTg582XDzXjvgYVePeHnQyf/hvnU9LyslqlVAGlQZDfFfWHXl/CbS/D9m/g09ZZdhV5e7jxXvcw/tOhFt9tOUzXT1YRf/J8HheslCpoNAgKAocDWvwL+i+CtItWV9G6CdftKhIRhrSqyqT+jTlw8jxRo6NZu0cXnymlsqZBUJBUagYPr4DKrWDRkzB7ACSfuW7TNjXLMn9YJH4+Htw/fjXT1uzL42KVUgWFBkFBU9Qf7v8K2r0I2xZY21of3njdplXL+DJvaCQtqpfmuXlbeH7+ZlJ18ZlS6i80CAoihwNaPgH9F0JqMky4HdZNvG5XkV8RDyb2a8xDravwxer99JmwhsSzF20oWimVX2kQFGSVmlsL0EJawKInYM7A63YVuTmEZ++qzQc9wok9cIqOo6PZfvj6XUpKKdejQVDQFS0NvWdDu//C1nkwrg0c3nTdpp0aVGDWQ81Iy8ig88cr+W7z4bytVSmVL2kQFAYOB7R8EvothNTzMOE2iJl03a6isIol+GZ4C2qXL8Yj0zbw1uIdpGfo4jOlXJkGQWESEmktQAuJhIWPw5xBcPHa/YfKFvdmxpCm9GoSzMc/72bAlHWcOp9iQ8FKqfxAg6Cw8S0DvefArc/D1rlWV9GRLdc083J3443OobzROZTVuxN13EApF6ZBUBg5HNBqJPT7Bi6ehQntYP2U63YV9WoSzMyHmnIxLZ3OH69kwcZDeV+vUspWGgSFWUgLawFacDP45jGYO/i6XUUNg0vyzYgW1KtQnEdn/Mbri7bpYTdKuRANgsLOtwz0mQttn4ctc7LsKipbzJtpg5rSr1klxv+6h76T1up6A6VchAaBK3A4oPVI6LvAuiOY0A7Wf3ZNV5Gnu4OXo+rxTrcwYvadpOPoaDbHZ32GslKqcNAgcCWVW2Z2FTWFbx6FeQ9ByrW7k3ZtFMSch5sD0GXsSmavj8/rSpVSeUiDwNX4lrW6itr8BzbNsnYyPRF3TbPQID8WDI+kUXBJnvpqIy9+vUX3KVKqkNIgcEUON2jzNPT+Ck7HW+MGO7+/ppm/rxefD2zC4JaV+WzVPnqPX8OxpOS8r1cp5VQaBK6s+u0w5GfwC4bp3eHn/0HG1b/1u7s5eO7uOnzYM5xNB09x76gVbNh/0pZylVLOoUHg6kpVhoHfQ/0e8PP/wcxecOHUNc2iwisw95FIPN0d9Px0NTPW7rehWKWUM2gQKPD0gfvGwl1vw64fra6io1uvaVYnsDjfDG/BLVVK8ezczTw7dzMX09Lzvl6lVK7SIFAWEbhliHUcZuoFa+O6zbOvaVbCx5MpA5owtE1VZqzdT89xqzlyWscNlCrInBYEIjJJRI6JyLWrl6zrIiIficguEdkkIg2dVYu6CcFN4aFfoHyYdb7B4mchPfWqJm4O4d/ta/FJ74b8fiSJe0at0HORlSrAnHlHMAVon831u4DqmX+GAJ84sRZ1M4oFWPsU3fIwrP4YpkZB0tFrmt0VWp75wyIp5u3O/eNX89nKvZjr7GeklMrfnBYExpjlQHa/JkYBU41lNVBCRMo7qx51k9w84K7/QefxcHCDdTbygbXXNKtRrhjzh0XSukYZXlywlSe/2khyqo4bKFWQ2DlGUAE4cMXX8ZmPXUNEhohIjIjEJCQk5ElxKlP97jDoR3D3gskdYN2Ea7am8Cviwfi+ETzWrjpzNxyk69iVxJ+8dsWyUip/KhCDxcaYccaYCGNMRJkyZewux/UE1LPWG1RtC4uehPlDrQHlKzgcwuO312B83wj2HT9Px9HRrNx13JZylVI3x84gOAhUvOLroMzHVH5UpCT0+hJaPwMbp8PEO+Dk3mua3V6nHPOHR1KqqCd9Jq5hwq9xOm6gVD5nZxAsAPpmzh5qCpw2xuhp6vmZwwFtn7UC4eQ+a73Brp+uaVa1jC/zh0VyR50AXlu0nUdnxnI+JS3v61VK5Ygzp4/OAFYBNUUkXkQGisjDIvJwZpNvgThgFzAeGOqsWlQuq9kehiyDYoHwRRdY/vY1W1P4ernzSZ+GjLyzJgs3HaLzxyvZn6jjBkrlR1LQbtsjIiJMTEyM3WUogJRz1slnm7+CmnfDfZ+At981zX7+/RiPzvgNEeGjXg1oXUPHeZTKayKy3hgTcb1rBWKwWOVTnkWt6aXt34Q/lsD4W+HY9muatalZlm9GtKC8nzf9J69lzLJdOm6gVD6iQaD+GRFo+oi1AC35DIxvB1vmXtOskn9R5g5tzj31A3l7ye8MnbaBsxd13ECp/ECDQOWOSs3hoeVQri7MHgDfPw/pV/+g9/F056Oe4TzXoTZLth7hvjHRxCWctalgpdSfNAhU7ile3tq0rvFgWDkKPu8EZ69eACgiDG5Vhc8H3sLxsxeJGh3N0h3Xbl+hlMo7GgQqd7l7wt3vQKexEL/O2poi/trB/chqpflmRAuC/X0Y9FkMk6P32FCsUgo0CJSzhPeyDrxxuMHkuyBm8jVbUwSV9OGrh5txa61yvPzNNl78egtpei6yUnlOg0A5T/kwGPILhLSEhf+CBcMh9eqzC3w83fn0gUYMamGdizx4aowOIiuVxzQIlHP5lILeX0GrkfDbFzC5PZw6cFUTN4fw/D11eLVTPZb/cZxuY1dx+PSFLF5QKZXbNAiU8znc4NbnoecMSNwNn7aC3cuuafZA00pM7BfBgRPniRodzZaDp20oVinXo0Gg8k6tDjB4GfiWgy86w4r3rxk3aFOzLLMfaYaHm4NuY1fxwzadUaSUs2kQqLxVupp1vkGdTvDjSzDrAWsh2hVqBRRn3rDmVC/ny5DPY3QHU6WcTINA5T0vX+g6Ce54HXZ8CxNus7qMrlC2mDdfDmnGHXXK8dqi7bygM4qUchoNAmUPEWg+HPp+DeePw/i28MePVzUp4unGJ70bMaRVFb5YvZ+Bn8WQlJxqU8FKFV4aBMpelVta4wZ+wTC9G6z44KpxA4dD+E+H2vzffaGs2GXNKDp4SmcUKZWbNAiU/UpWgoFLoE4U/PgizBkEKVefXXD/LcFM7t+Ygycv0GlMNJviT9lUrFKFT46CQEQeE5HimaeJTRSRDSJyh7OLUy7Esyh0nQztXoQtc2DSnXBq/1VNWtUow5yhzfF0c9D901Us2XrEpmKVKlxyekfwoDHmDHAHUBJ4AHjTaVUp1yQCLZ+A+2ddPgpz74qrmtQoV4z5wyKpGVCch79Yz7jlu3VGkVL/UE6DQDL/2wH43Biz9YrHlMpdNe6AwT+Bjz9MjYK1468aNyhTzIuZg5vSvm4A//ftDp6bv4VUnVGk1N+W0yBYLyLfYwXBEhEpBui/POU8patb6w2q3QbfPgULRkDaxUuXi3i6Meb+hjzSpirT1+znwSnrOKMzipT6W3IaBAOBZ4DGxpjzgAcwwGlVKQXW+cc9Z0DLp+C3z2HKPZB0eVzA4RCebl+L/3UJZdXuRLp+spL4k+ezeUGl1PXkNAiaAb8bY06JSB/geUA3glHO53BAuxeg22dwdIs1bvCX8w16NA7mswebcPh0Mp3GrCT2gM4oUupm5DQIPgHOi0gY8CSwG5jqtKqU+qu6nWDgD+DmaZ1v8Nu0qy5HVivN3Eea4+3hoMenq/hu82GbClWq4MlpEKQZa2pGFDDaGDMGKOa8spS6joB6MORnCG4KXw+F756G9MvjAtUzZxTVCSzOI9M2MPYXnVGkVE7kNAiSRORZrGmji0TEgTVOoFTe8ikFfeZB06GwZix8fh+cS7x0ubSvFzMGN+We+uV587sdPDt3s84oUuoGchoEPYCLWOsJjgBBwNtOq0qp7Li5Q/s3rHORD6yF8W3gyOZLl7093PioZwOGta3KzHUHGDB5Hacv6IwipbKSoyDI/OE/DfATkXuAZGPMDccIRKS9iPwuIrtE5JnrXA8WkWUi8puIbBKRDjf9HSjXFd4LBnwH6Wkw8Q7YOu/SJYdDGHlnLd7qWp/VcdaMogMndEaRUteT0y0mugNrgW5Ad2CNiHS9wXPcgDHAXUAdoJeI1PlLs+eBWcaYBkBP4OObK1+5vKBG1rhBQCh81R9+egUy0i9d7h5RkakDm3D0TDKdxkSzYf9JuypVKt/KadfQc1hrCPoZY/oCTYAXbvCcJsAuY0ycMSYFmIk12HwlAxTP/LsfcCiH9Sh1WbFy0O8baNgPfn0XZvSC5Muzm5tXLc28YZEU9XKn17jVLNqkM4qUulJOg8BhjDl2xdeJOXhuBeDKU8rjMx+70ktAHxGJB74FRlzvhURkiIjEiEhMQkJCDktWLsXdC+79EO5+F3b/BOPbwfE/Ll2uWsaXeUObU6+CH8Omb2DMsl06o0ipTDkNgsUiskRE+otIf2AR1g/uf6oXMMUYE0TmPkaZM5KuYowZZ4yJMMZElClTJhfeVhVKItB4EPRdABdOwvhbYeeSS5f9fb2YNugWOoYF8vaS33l6ziZS0nRGkVI5HSweCYwD6mf+GWeMefoGTzsIVLzi66DMx640EJiV+R6rAG+gdE5qUipLIZHWuEGpyjC9Byx/59Kmdd4ebnzYM5xH21VnVkw8/Sev5fR5nVGkXFuOD6YxxswxxjyR+WfejZ/BOqC6iFQWEU+sweAFf2mzH2gHICK1sYJA+37UP1eiIgxYDPW6wNJXrYHklHMAiAhP3F6Dd7uFsW7vCTp/Es3+RJ1RpFxXtkEgIkkicuY6f5JE5Ex2zzXGpAHDgSXAdqzZQVtF5BUR6ZjZ7ElgsIhsBGYA/Y123Krc4ukDXSbA7a/A9gXWFNOTey9d7tIoiM8H3sLxsyl0+jiadXtP2FerUjaSgvZzNyIiwsTExNy4oVJX2vUjzH4QxA26TYEqrS9diks4y8DPYog/eZ5Xo+rRs0mwfXUq5SQist4YE3G9a3pmsXIN1W6DwcvAt6y1LcXqTy6NG1Qp48v8oZE0q1qaZ+Zu5r9f60E3yrVoECjX4V/VOuymRntY/AzMHwqpyQD4+XgwuX9jhrSqwtRV+3hg4hpOnEuxuWCl8oYGgXItXsWgxxfQ+hnYOB2mdIAz1jpGN4fwnw61ea97GBv2n6Lj6BVsP5ztUJhShYIGgXI9Dge0fdYKhGM7rMNuDqy9dLlzwyC+eqgZqekZdP54pZ5toAo9DQLlumrfa3UVeRSByR1g1ZhL+xSFVSzBN8NbUKt8MR6ZtoH3vv+djIyCNbFCqZzSIFCurVwdaxC52m2w5D/W6WeZW1OULe7NzCFN6dYoiI+W7uLhL9Zz9mKazQUrlfs0CJTyKQW9ZsB94yDhdxjbAqI/gox0vNzdeKtrfV68tw4/7ThG54+j2Zd4zu6KlcpVGgRKgbVPUVgPGLYGqraDH16wFqAl/I6IMCCyMlMfbMLRMxfpODqaFX8ct7tipXKNBoFSVyoWAD2nQZeJcCIOxraEFe9DehqR1UqzYHgk5Yp70W/yWiat2KM7mKpCQYNAqb8SgdCu1t1BjTvgx5dg4u1wdBuV/Isyd2gk7WqV5ZWF2/j37E1cTEu/4UsqlZ9pECiVFd+y0P1z6DoZTu2DT1vB8rfxdTeM7dOIR9tV56v18fQct5pjZ5Ltrlapv02DQKnsiEC9zjBsLdS+B5a+BuNvxXFsK0/cXoNPejdkx+Ek7h29go0HTtldrVJ/iwaBUjlRtLS1WV33qZB02FqE9vOb3FXbn7lDm+Ph5qDbp6uYuyHe7kqVumkaBErdjDpRMHQN1O0EP78B42+lNntZMLwFDYNL8MSsjby+aBtpummdKkA0CJS6WUX9rXMOek6Hc8dgfFtKrXmbz/s3oF+zSoz/dQ8DpqzTk89UgaFBoNTfVetuGLoa6nWF5W/hMeFWXm6cypudQ1kdl6OrTGMAABRtSURBVEjUmBXsOpZkd5VK3ZAGgVL/hE8p6Pwp9PoSLpyA8e3omTSFmQ824OzFNDqNWclP24/aXaVS2dIgUCo31GwPQ1dBWE/49V0aLe7Ed12LElLah0FTYxizbJcuPlP5lgaBUrmlSEno9DH0ng0Xkyjz5d3Mq/EDnUP9eXvJ7wyf8RvnU3TTOpX/aBAolduq327dHTTog8eqD3nnxAjea57Ct5sP0/WTVRw8dcHuCpW6igaBUs7g7QcdR0GfuUjqBTr/9iC/1P+JYydO0XHUCtbEJdpdoVKXaBAo5UzV2sEjK6FRf4J/n8jKkv+luecuek9Ywxer99ldnVKABoFSzuddHO55H/p+jSdpfHThWcb4f8Vr89fz3LzNpKTp4jNlLw0CpfJKlTbwyCqk8SDuTJrLKr8X+GPtEvpMXMPxsxftrk65MA0CpfKSly/c/Q70W0hJH3dmeb3KvQffp+eoH9l66LTd1SkX5dQgEJH2IvK7iOwSkWeyaNNdRLaJyFYRme7MepTKNyq3tMYObnmYBxxL+Ozi47z1yXgWbjpkd2XKBTktCETEDRgD3AXUAXqJSJ2/tKkOPAtEGmPqAv9yVj1K5TueReGu/8GA7yhXwofP3F7l1KzhvDQrmjPJuk+RyjvOvCNoAuwyxsQZY1KAmUDUX9oMBsYYY04CGGOOObEepfKnSs1xH7qStFuG0tt9KY9u7c64t5/h520H7a5MuQhnBkEF4MAVX8dnPnalGkANEYkWkdUi0v56LyQiQ0QkRkRiEhISnFSuUjby9MH9rjeQh5bjEVifp9InUHFmOyZOHMOpczqQrJzL7sFid6A60AboBYwXkRJ/bWSMGWeMiTDGRJQpUyaPS1QqD5WvT7Eh35LaYwYlinox8MB/+OOdW4n+dandlalCzJlBcBCoeMXXQZmPXSkeWGCMSTXG7AF2YgWDUq5LBI/aHfB/KobDka9Sg/00+7Ezq97rwYnDe+2uThVCzgyCdUB1EaksIp5AT2DBX9rMx7obQERKY3UVxTmxJqUKDjcPyt/+KD5PbSK24gM0Ov0jRT5twu8z/4O5eNbu6lQh4rQgMMakAcOBJcB2YJYxZquIvCIiHTObLQESRWQbsAwYaYzRTViUuoJH0ZI0HDSK+D7LWe/VhJo7xnD6rfqcWTkZMtLtLk8VAlLQ9kiPiIgwMTExdpehlC3SMwwLF82jUszrhMsuThavRYlO/0OqtLG5MpXfich6Y0zE9a7ZPVislLoJbg4h6t7O+A3/hQ/8nuHc6ePI1CguTO0GCTvtLk8VUBoEShVAlcv48uhjz7Dstm95N+N+0nb/SsbHTTGLnoJz2ruqbo4GgVIFlMMhPNCyJt3/9S7/DpzCtNS2ZKybSMaH4RD9EaTp+gOVMxoEShVwFUv58PGQO3Hv+D73mXf49WJV+OEFzOjGsHUeFLBxQJX3NAiUKgREhF5Nghn7+P1MDnmLPinPsv+swFf9YdKdcGCd3SWqfEyDQKlCJLBEESb3b0znrn3olPYm/0kfwrmju2HibTD7QTipp6Kpa2kQKFXIiAidGwax5Im2HK/eg8Zn3uLLIj3J2LEIRjeGH16EZD37QF2mQaBUIVW2uDefPtCIt+5vxv9SutL6wjts978Noj+AjxrCugmQnmZ3mSof0CBQqhATEe6pH8gPj7civF497trfmxHFPuCcXzVY9CR80hx2fq8Dyi5Og0ApF+Dv68WoXg349IFGrE6uSP19jzK/5ltkZKTB9G7w+X1wZIvdZSqbaBAo5ULurBvAj4+3plN4EP/aGESH1LeIv+W/cOg3+LQlLBgBSUftLlPlMQ0CpVyMn48H73YPY/KAxpxOEVotr8W7tWeS1vghiJ0BHzWAn9+EpCN2l6ryiG46p5QLS0pO5Y3vdjB9zX5C/H348A4/wna8B9u/AXFA1XYQfj/U7AAe3naXq/6B7Dad0yBQShG96zjPzN3EgRMX6NesEk83dsdn+1ewcSaciQdvP6jXBcLuh6AIELG7ZHWTNAiUUjd07mIaby/5nc9W7SXQrwjDb61GlwaBeB5YAbHTYdsCSLsA/tWtu4SwnlA80O6yVQ5pECilcmzd3hO8tnAbG+NPE+jnzSNtqtItoiLe6edg29dWKOxfCQhUbQvhvaHW3eBRxO7SVTY0CJRSN8UYwy87E/jopz/YsP8U5Yp78VCrqtx/SzDeHm5wIs7qNoqdAaf3g1dxqHufFQoVm2jXUT6kQaCU+luMMazcnciHP/3B2j0nKO3rxUOtqtC7aTA+nu6QkQH7ojO7jr6G1HNQqiqE94L6PaFERbu/BZVJg0Ap9Y+tjktk1NI/iN6VSKmingxqWZm+zULw9XK3Glw8C9sXWKGw91dAoHIr6y6h9j3gWdTW+l2dBoFSKtes33eCj37axS87Eyjh48GDkZXp1zwEvyIelxud3Gd1HW2cDif3gqcv1O1khUJwM+06soEGgVIq18UeOMXopX/w4/ZjFPN2Z0BkZR6MDKGEj+flRsbA/lUQOw22zoeUs1AyxJqGGtYTSlayrX5Xo0GglHKaLQdPM3rpLhZvPYKvlzt9m1ViUMsqlCrqeXXDlHOwfaEVCnuWAwZCWlpTUWt3BC9fW+p3FRoESimn23HkDKOW7uLbzYfxdnfjgWaVGNyyCmWKeV3b+NQB2DTTGk84EQceRaFOlBUKlSLBobvf5DYNAqVUntl1LInRS3exYOMhPNwc3H9LMA+3rkq54tfZosIYOLA2s+toHlw8AyWCIayX1XVUqkrefwOFlAaBUirP7Tl+jjHLdjHvt4O4OYQeERV5uE1VKpTIYuFZ6gXYscgKhd3LAANBTaDabVD1VqjQEBxuefo9FCa2BYGItAc+BNyACcaYN7No1wWYDTQ2xmT7U16DQKmCZX/ieT75ZRez18cD0LVRRYa2qUrFUj5ZP+n0QavraPtCa4tsDHiXgCptrFCo1g78gvKi/ELDliAQETdgJ3A7EA+sA3oZY7b9pV0xYBHgCQzXIFCqcDp46gJjf97Nl+sOkG4MnRtUYFjbaoSUvsH6gnOJsOdn2LUUdv8ESYetx0vXsHZHrXorhETqOoUbsCsImgEvGWPuzPz6WQBjzBt/afcB8AMwEnhKg0Cpwu3I6WTG/rKbGWv3k5qeQVS4FQjVyuZg1pAxkLADdv0Eu5daq5rTksHN01qf8OfdQrl6ulbhL+wKgq5Ae2PMoMyvHwBuMcYMv6JNQ+A5Y0wXEfmZLIJARIYAQwCCg4Mb7du3zyk1K6XyzrGkZMYvj+OL1ftJTkvn7tDyjLi1OjUDiuX8RVIvWOsU/gyGY5kdDkXLXg6FKm3Bt4xzvokCJF8GgYg4gKVAf2PM3uyC4Ep6R6BU4ZJ49iITVuxh6sq9nEtJ5656AQy/tRp1A/1u/sXOHLYCYfdSiFsG5xOtxwPqXw6Gik3B3TP71ymE8mXXkIj4AbuBs5lPCQBOAB2zCwMNAqUKp5PnUpgcvYfJ0XtJupjGbbXL8UibqjQMLoH8nW6ejAw4svHy3cKBNZCRZq1ZCGlhhULVW8G/mkt0I9kVBO5Yg8XtgINYg8X3G2O2ZtH+Z/SOQCmXd/pCKlOi9zIpeg+nL6RSyd+HqLBAOoZXyNk4QlYuJsGeXzPvGH6yFrIB+AVDtVutUKjcGoqUyJ1vJJ+xc/poB+ADrOmjk4wxr4vIK0CMMWbBX9r+jAaBUirT2YtpfLv5MAtiD7Fy93EyDNQNLE6n8ArcE1ae8n7/8CCcE3uu6Eb6BVKSrHOaK0RcvlsIbAhu7rnzDdlMF5QppQq0Y2eSWbjpMF9vPMTGA6cQgVsqlyIqvAJ31Qu4eqO7vyM9FeJjLt8tHNyAtXbBz1q7UCkSyodZs5EK6J5IGgRKqUJj7/FzLNh4iPmxB4lLOIeHm9C6RlmiwgO5rXY5injmwurj8ycg7mcrFHYvgzMHMy8I+Fe1QiGgPpSvDwFhUNT/n7+nk2kQKKUKHWMMWw+dYcHGQyyIPcSRM8n4eLpxZ90AOoYH0qJaaTzccmHzOmOsRWyHN8GRTXB4o/X30/svtykelBkK9a2QKF8filfIV4PQGgRKqUItI8Owdu8Jvo49xLebD3P6Qiqlinpyd2h5osIDaRhcEocjl38onz+RGQybLv/3+E4g82dqkVJWIFy6ewizjvG0aWdVDQKllMu4mJbO8p3H+Tr2ID9uP0pyagYVShShY3ggUeGB1Aoo7rw3TzkHR7dm3jVstALi2HZIT7Gue/pa4wxXBkSZWnmyrkGDQCnlks5eTOOHbUf4OvYQv/5xnPQMQ81yxegYHkjHsMDsN77LLWkp1rYYf941HN4IR7dYp7WBtT1GmVqZXUqZ4RBQL9f3TtIgUEq5vMSzF/l282G+jj1EzL6TADSqVJKo8EA6hJantO91DtBxlowMax3DkY2XxxyObLq8EhqB0tUvD0j/GRA+pf72W2oQKKXUFQ6cOM83m6xB5h1HknBzCC2qlSYqPJA76gbg62XD2gFjrNlJfx2UPhN/uU3kv+D2l//Wy2sQKKVUFnYcOcOC2EN8HXuIg6cu4O3hoF3tckSFBdK6Zhm83G0+DOdc4uVgCAy31jX8DRoESil1A8YYNuw/ydexh1i46TAnzqVQ3NudDqHlubNeAI1DStlzp5BLNAiUUuompKZnEL3rOAtiD7Fk6xHOpaTj5hDqB/nRtIo/zar4ExFSEh/PghMMGgRKKfU3JaemE7P3JKvjElkVl8jGA6dIyzB4uAlhQSVoVtUKhoaVSuLtkX/PVNYgUEqpXHLuYhox+06yanciq+MS2XzwNOkZBk83B+HBJWhWxZ9mVf1pEFzC/vGFK2gQKKWUkyQlpxKz9ySr4hJZtTuRrYdOk2HAy91Bw+CS1h1DVX/Cgkrg6W7PqmLQIFBKqTxz+kIq6/acuBQM24+cwRgo4uFGREhJmlbxp2kVf+oH+eXOXkg5pEGglFI2OXU+hdVxJ1gdZ3Ul7TiSBEBRTzciQkpdGmOoG1gcdycGgwaBUkrlE4lnL7JmzwlW7bYGn3cds7aaKOblTuPKpS6NMdQuXxy3XNwoL7sgKDhzn5RSqhDw9/WiQ2h5OoSWByAh6eKlGUmrdyeydMcxAIp7u3NLZjdSsyr+1Aoolvs7qGbSIFBKKRuVKebFvWGB3BsWCMCR08mXupFWxSXyw7ajAJT08WBY22oMalkl12vQIFBKqXwkwM+bTg0q0KlBBQAOnbpwqRupbHFvp7ynBoFSSuVjgSWK0KVREF0aBTntPeyb1KqUUipf0CBQSikXp0GglFIuToNAKaVcnAaBUkq5OA0CpZRycRoESinl4jQIlFLKxRW4TedEJAHY9zefXho4novlFHT6eVxNP4/L9LO4WmH4PCoZY8pc70KBC4J/QkRistp9zxXp53E1/Twu08/iaoX989CuIaWUcnEaBEop5eJcLQjG2V1APqOfx9X087hMP4urFerPw6XGCJRSSl3L1e4IlFJK/YUGgVJKuTiXCQIRaS8iv4vILhF5xu567CQiFUVkmYhsE5GtIvKY3TXZTUTcROQ3EVlody12E5ESIjJbRHaIyHYRaWZ3TXYRkccz/41sEZEZIuKcI8Js5hJBICJuwBjgLqAO0EtE6thbla3SgCeNMXWApsAwF/88AB4DtttdRD7xIbDYGFMLCMNFPxcRqQA8CkQYY+oBbkBPe6tyDpcIAqAJsMsYE2eMSQFmAlE212QbY8xhY8yGzL8nYf1Dr2BvVfYRkSDgbmCC3bXYTUT8gFbARABjTIox5pS9VdnKHSgiIu6AD3DI5nqcwlWCoAJw4Iqv43HhH3xXEpEQoAGwxt5KbPUB8G8gw+5C8oHKQAIwObOrbIKIFLW7KDsYYw4C7wD7gcPAaWPM9/ZW5RyuEgTqOkTEF5gD/MsYc8bueuwgIvcAx4wx6+2uJZ9wBxoCnxhjGgDnAJccUxORklg9B5WBQKCoiPSxtyrncJUgOAhUvOLroMzHXJaIeGCFwDRjzFy767FRJNBRRPZidRneKiJf2FuSreKBeGPMn3eIs7GCwRXdBuwxxiQYY1KBuUBzm2tyClcJgnVAdRGpLCKeWAM+C2yuyTYiIlh9wNuNMe/ZXY+djDHPGmOCjDEhWP9fLDXGFMrf+nLCGHMEOCAiNTMfagdss7EkO+0HmoqIT+a/mXYU0oFzd7sLyAvGmDQRGQ4swRr5n2SM2WpzWXaKBB4ANotIbOZj/zHGfGtjTSr/GAFMy/ylKQ4YYHM9tjDGrBGR2cAGrJl2v1FIt5rQLSaUUsrFuUrXkFJKqSxoECillIvTIFBKKRenQaCUUi5Og0AppVycBoFSmUQkXURir/iTaytqRSRERLbk1usplZtcYh2BUjl0wRgTbncRSuU1vSNQ6gZEZK+IvCUim0VkrYhUy3w8RESWisgmEflJRIIzHy8nIvNEZGPmnz+3JXATkfGZ+9t/LyJFMts/mnk2xCYRmWnTt6lcmAaBUpcV+UvXUI8rrp02xoQCo7F2KwUYBXxmjKkPTAM+ynz8I+AXY0wY1j49f65irw6MMcbUBU4BXTIffwZokPk6Dzvrm1MqK7qyWKlMInLWGON7ncf3ArcaY+IyN+s7YozxF5HjQHljTGrm44eNMaVFJAEIMsZcvOI1QoAfjDHVM79+GvAwxrwmIouBs8B8YL4x5qyTv1WlrqJ3BErljMni7zfj4hV/T+fyGN3dWCfoNQTWZR6ColSe0SBQKmd6XPHfVZl/X8nlowt7A79m/v0n4BG4dBayX1YvKiIOoKIxZhnwNOAHXHNXopQz6W8eSl1W5IrdWME6t/fPKaQlRWQT1m/1vTIfG4F1ktdIrFO9/tyl8zFgnIgMxPrN/xGsE66uxw34IjMsBPjIxY+GVDbQMQKlbiBzjCDCGHPc7lqUcgbtGlJKKRendwRKKeXi9I5AKaVcnAaBUkq5OA0CpZRycRoESinl4jQIlFLKxf0/ursi9FxJ2ZcAAAAASUVORK5CYII=\n", 542 | "text/plain": [ 543 | "
" 544 | ] 545 | }, 546 | "metadata": { 547 | "tags": [], 548 | "needs_background": "light" 549 | } 550 | } 551 | ] 552 | }, 553 | { 554 | "cell_type": "code", 555 | "metadata": { 556 | "colab_type": "code", 557 | "id": "w7Xc-uWxXhML", 558 | "colab": {} 559 | }, 560 | "source": [ 561 | "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n", 562 | "\n", 563 | "def decode_sentence(text):\n", 564 | " return ' '.join([reverse_word_index.get(i, '?') for i in text])\n" 565 | ], 566 | "execution_count": 0, 567 | "outputs": [] 568 | }, 569 | { 570 | "cell_type": "code", 571 | "metadata": { 572 | "colab_type": "code", 573 | "id": "OhnFA_TDXrih", 574 | "outputId": "468f4415-8372-4473-c482-e5794d841139", 575 | "colab": { 576 | "base_uri": "https://localhost:8080/", 577 | "height": 34 578 | } 579 | }, 580 | "source": [ 581 | "e = model.layers[0]\n", 582 | "weights = e.get_weights()[0]\n", 583 | "print(weights.shape) # shape: (vocab_size, embedding_dim)\n", 584 | "\n", 585 | "# Expected output\n", 586 | "# (1000, 16)" 587 | ], 588 | "execution_count": 0, 589 | "outputs": [ 590 | { 591 | "output_type": "stream", 592 | "text": [ 593 | "(1000, 16)\n" 594 | ], 595 | "name": "stdout" 596 | } 597 | ] 598 | }, 599 | { 600 | "cell_type": "code", 601 | "metadata": { 602 | "colab_type": "code", 603 | "id": "_POzcWWAXudL", 604 | "colab": {} 605 | }, 606 | "source": [ 607 | "import io\n", 608 | "\n", 609 | "out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n", 610 | "out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n", 611 | "for word_num in range(1, vocab_size):\n", 612 | " word = reverse_word_index[word_num]\n", 613 | " embeddings = weights[word_num]\n", 614 | " out_m.write(word + \"\\n\")\n", 615 | " out_v.write('\\t'.join([str(x) for x in embeddings]) + \"\\n\")\n", 616 | "out_v.close()\n", 617 | "out_m.close()" 618 | ], 619 | "execution_count": 0, 620 | "outputs": [] 621 | }, 622 | { 623 | "cell_type": "code", 624 | "metadata": { 625 | "colab_type": "code", 626 | "id": "VmqpQMZ_XyOa", 627 | "colab": {} 628 | }, 629 | "source": [ 630 | "try:\n", 631 | " from google.colab import files\n", 632 | "except ImportError:\n", 633 | " pass\n", 634 | "else:\n", 635 | " files.download('vecs.tsv')\n", 636 | " files.download('meta.tsv')" 637 | ], 638 | "execution_count": 0, 639 | "outputs": [] 640 | }, 641 | { 642 | "cell_type": "code", 643 | "metadata": { 644 | "id": "X0ULeQ_ZdITc", 645 | "colab_type": "code", 646 | "colab": {} 647 | }, 648 | "source": [ 649 | "" 650 | ], 651 | "execution_count": 0, 652 | "outputs": [] 653 | } 654 | ] 655 | } -------------------------------------------------------------------------------- /C3W4_1_Shakespeare_Tokenizer_Embedding_LSTM_Dropouts_Regularizers.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "C3W4-1-Shakespeare-Tokenizer-Embedding-LSTM-Dropouts-Regularizers", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "view-in-github", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "\"Open" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "metadata": { 31 | "id": "zX4Kg8DUTKWO", 32 | "colab_type": "code", 33 | "colab": {} 34 | }, 35 | "source": [ 36 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 37 | "# you may not use this file except in compliance with the License.\n", 38 | "# You may obtain a copy of the License at\n", 39 | "#\n", 40 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 41 | "#\n", 42 | "# Unless required by applicable law or agreed to in writing, software\n", 43 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 44 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 45 | "# See the License for the specific language governing permissions and\n", 46 | "# limitations under the License." 47 | ], 48 | "execution_count": 0, 49 | "outputs": [] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "metadata": { 54 | "colab_type": "code", 55 | "id": "BOwsuGQQY9OL", 56 | "colab": {} 57 | }, 58 | "source": [ 59 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 60 | "from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout, Bidirectional\n", 61 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 62 | "from tensorflow.keras.models import Sequential\n", 63 | "from tensorflow.keras.optimizers import Adam\n", 64 | "### YOUR CODE HERE\n", 65 | "from tensorflow.keras import regularizers\n", 66 | "###\n", 67 | "import tensorflow.keras.utils as ku \n", 68 | "import numpy as np " 69 | ], 70 | "execution_count": 0, 71 | "outputs": [] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "metadata": { 76 | "colab_type": "code", 77 | "id": "PRnDnCW-Z7qv", 78 | "outputId": "4e7d9f43-f277-4879-9e63-c41bc5b318b0", 79 | "colab": { 80 | "base_uri": "https://localhost:8080/", 81 | "height": 204 82 | } 83 | }, 84 | "source": [ 85 | "tokenizer = Tokenizer()\n", 86 | "!wget --no-check-certificate \\\n", 87 | " https://storage.googleapis.com/laurencemoroney-blog.appspot.com/sonnets.txt \\\n", 88 | " -O /tmp/sonnets.txt" 89 | ], 90 | "execution_count": 0, 91 | "outputs": [ 92 | { 93 | "output_type": "stream", 94 | "text": [ 95 | "--2020-04-24 16:54:47-- https://storage.googleapis.com/laurencemoroney-blog.appspot.com/sonnets.txt\n", 96 | "Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.14.112, 2607:f8b0:4007:802::2010\n", 97 | "Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.14.112|:443... connected.\n", 98 | "HTTP request sent, awaiting response... 200 OK\n", 99 | "Length: 93578 (91K) [text/plain]\n", 100 | "Saving to: ‘/tmp/sonnets.txt’\n", 101 | "\n", 102 | "\r/tmp/sonnets.txt 0%[ ] 0 --.-KB/s \r/tmp/sonnets.txt 100%[===================>] 91.38K --.-KB/s in 0.001s \n", 103 | "\n", 104 | "2020-04-24 16:54:47 (152 MB/s) - ‘/tmp/sonnets.txt’ saved [93578/93578]\n", 105 | "\n" 106 | ], 107 | "name": "stdout" 108 | } 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "metadata": { 114 | "id": "KBvxloSRD2qb", 115 | "colab_type": "code", 116 | "outputId": "ead295fe-7bb6-48e8-874f-35b93dd9edd5", 117 | "colab": { 118 | "base_uri": "https://localhost:8080/", 119 | "height": 34 120 | } 121 | }, 122 | "source": [ 123 | "data = open('/tmp/sonnets.txt').read()\n", 124 | "\n", 125 | "corpus = data.lower().split(\"\\n\")\n", 126 | "print(corpus[:3])\n", 127 | "\n", 128 | "tokenizer.fit_on_texts(corpus)\n", 129 | "total_words = len(tokenizer.word_index) + 1" 130 | ], 131 | "execution_count": 0, 132 | "outputs": [ 133 | { 134 | "output_type": "stream", 135 | "text": [ 136 | "['from fairest creatures we desire increase,', \"that thereby beauty's rose might never die,\", 'but as the riper should by time decease,']\n" 137 | ], 138 | "name": "stdout" 139 | } 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "metadata": { 145 | "id": "4I_YrBNlD67D", 146 | "colab_type": "code", 147 | "colab": {} 148 | }, 149 | "source": [ 150 | "# create input sequences using list of tokens\n", 151 | "input_sequences = []\n", 152 | "for line in corpus:\n", 153 | " #print([line])\n", 154 | " #print(tokenizer.texts_to_sequences([line]))\n", 155 | " #print(tokenizer.texts_to_sequences([line])[0])\n", 156 | " #break\n", 157 | " token_list = tokenizer.texts_to_sequences([line])[0]\n", 158 | " for i in range(1, len(token_list)):\n", 159 | " n_gram_sequence = token_list[:i+1]\n", 160 | " input_sequences.append(n_gram_sequence)\n", 161 | "\n", 162 | "\n", 163 | "# pad sequences \n", 164 | "max_sequence_len = max([len(x) for x in input_sequences])\n", 165 | "input_sequences = np.array(pad_sequences(input_sequences, maxlen=max_sequence_len, padding='pre'))\n", 166 | "\n", 167 | "# create predictors and label\n", 168 | "predictors, label = input_sequences[:,:-1],input_sequences[:,-1]" 169 | ], 170 | "execution_count": 0, 171 | "outputs": [] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "metadata": { 176 | "id": "1lk080yTD69L", 177 | "colab_type": "code", 178 | "colab": {} 179 | }, 180 | "source": [ 181 | "#https://keras.io/api/utils/python_utils/#to_categorical-function\n", 182 | "label = ku.to_categorical(label, num_classes=total_words)" 183 | ], 184 | "execution_count": 0, 185 | "outputs": [] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "metadata": { 190 | "id": "wlIcZj3MEtwy", 191 | "colab_type": "code", 192 | "outputId": "0dfc1474-c88b-42e8-e0f9-a350b9533d69", 193 | "colab": { 194 | "base_uri": "https://localhost:8080/", 195 | "height": 34 196 | } 197 | }, 198 | "source": [ 199 | "label[0]" 200 | ], 201 | "execution_count": 0, 202 | "outputs": [ 203 | { 204 | "output_type": "execute_result", 205 | "data": { 206 | "text/plain": [ 207 | "array([0., 0., 0., ..., 0., 0., 0.], dtype=float32)" 208 | ] 209 | }, 210 | "metadata": { 211 | "tags": [] 212 | }, 213 | "execution_count": 30 214 | } 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "metadata": { 220 | "id": "NWZu2wFPEoPd", 221 | "colab_type": "code", 222 | "outputId": "6d5bf389-79e1-4eff-c471-409da3c97922", 223 | "colab": { 224 | "base_uri": "https://localhost:8080/", 225 | "height": 34 226 | } 227 | }, 228 | "source": [ 229 | "len(label[0])" 230 | ], 231 | "execution_count": 0, 232 | "outputs": [ 233 | { 234 | "output_type": "execute_result", 235 | "data": { 236 | "text/plain": [ 237 | "3211" 238 | ] 239 | }, 240 | "metadata": { 241 | "tags": [] 242 | }, 243 | "execution_count": 31 244 | } 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "metadata": { 250 | "id": "m-FPM71JFnRZ", 251 | "colab_type": "code", 252 | "outputId": "72f6ef8a-637c-4c0c-dd71-0c88c0bace36", 253 | "colab": { 254 | "base_uri": "https://localhost:8080/", 255 | "height": 34 256 | } 257 | }, 258 | "source": [ 259 | "total_words" 260 | ], 261 | "execution_count": 0, 262 | "outputs": [ 263 | { 264 | "output_type": "execute_result", 265 | "data": { 266 | "text/plain": [ 267 | "3211" 268 | ] 269 | }, 270 | "metadata": { 271 | "tags": [] 272 | }, 273 | "execution_count": 32 274 | } 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "metadata": { 280 | "colab_type": "code", 281 | "id": "w9vH8Y59ajYL", 282 | "outputId": "ec69f1e6-f7d8-430d-a410-010a2d479ae6", 283 | "colab": { 284 | "base_uri": "https://localhost:8080/", 285 | "height": 374 286 | } 287 | }, 288 | "source": [ 289 | "model = Sequential()\n", 290 | "model.add(Embedding(input_dim=total_words, output_dim=100, input_length=max_sequence_len-1))\n", 291 | "model.add(Bidirectional(LSTM(150, return_sequences=True)))\n", 292 | "model.add(Dropout(0.2))\n", 293 | "model.add(LSTM(100))\n", 294 | "model.add(Dense(total_words/2, activation='relu', kernel_regularizer=regularizers.l2(0.01)))\n", 295 | "model.add(Dense(total_words, activation='softmax'))\n", 296 | "\n", 297 | "my_optimizer = Adam(learning_rate=0.001)\n", 298 | "model.compile(loss='categorical_crossentropy', optimizer=my_optimizer, metrics=['accuracy'])\n", 299 | "\n", 300 | "print(model.summary())" 301 | ], 302 | "execution_count": 0, 303 | "outputs": [ 304 | { 305 | "output_type": "stream", 306 | "text": [ 307 | "Model: \"sequential_3\"\n", 308 | "_________________________________________________________________\n", 309 | "Layer (type) Output Shape Param # \n", 310 | "=================================================================\n", 311 | "embedding_3 (Embedding) (None, 10, 100) 321100 \n", 312 | "_________________________________________________________________\n", 313 | "bidirectional_6 (Bidirection (None, 10, 300) 301200 \n", 314 | "_________________________________________________________________\n", 315 | "dropout_3 (Dropout) (None, 10, 300) 0 \n", 316 | "_________________________________________________________________\n", 317 | "lstm_7 (LSTM) (None, 100) 160400 \n", 318 | "_________________________________________________________________\n", 319 | "dense_4 (Dense) (None, 1605) 162105 \n", 320 | "_________________________________________________________________\n", 321 | "dense_5 (Dense) (None, 3211) 5156866 \n", 322 | "=================================================================\n", 323 | "Total params: 6,101,671\n", 324 | "Trainable params: 6,101,671\n", 325 | "Non-trainable params: 0\n", 326 | "_________________________________________________________________\n", 327 | "None\n" 328 | ], 329 | "name": "stdout" 330 | } 331 | ] 332 | }, 333 | { 334 | "cell_type": "code", 335 | "metadata": { 336 | "colab_type": "code", 337 | "id": "AIg2f1HBxqof", 338 | "outputId": "32dad8da-a475-444b-dc21-a5934258dcb5", 339 | "colab": { 340 | "base_uri": "https://localhost:8080/", 341 | "height": 1000 342 | } 343 | }, 344 | "source": [ 345 | "history = model.fit(predictors, label, epochs=100, verbose=1)" 346 | ], 347 | "execution_count": 0, 348 | "outputs": [ 349 | { 350 | "output_type": "stream", 351 | "text": [ 352 | "Epoch 1/100\n", 353 | "484/484 [==============================] - 6s 12ms/step - loss: 5.6682 - accuracy: 0.0564\n", 354 | "Epoch 2/100\n", 355 | "484/484 [==============================] - 6s 12ms/step - loss: 5.5644 - accuracy: 0.0653\n", 356 | "Epoch 3/100\n", 357 | "484/484 [==============================] - 6s 12ms/step - loss: 5.4746 - accuracy: 0.0687\n", 358 | "Epoch 4/100\n", 359 | "484/484 [==============================] - 6s 12ms/step - loss: 5.3714 - accuracy: 0.0761\n", 360 | "Epoch 5/100\n", 361 | "484/484 [==============================] - 6s 12ms/step - loss: 5.2699 - accuracy: 0.0823\n", 362 | "Epoch 6/100\n", 363 | "484/484 [==============================] - 6s 12ms/step - loss: 5.1707 - accuracy: 0.0894\n", 364 | "Epoch 7/100\n", 365 | "484/484 [==============================] - 6s 12ms/step - loss: 5.0740 - accuracy: 0.1002\n", 366 | "Epoch 8/100\n", 367 | "484/484 [==============================] - 6s 12ms/step - loss: 4.9721 - accuracy: 0.1068\n", 368 | "Epoch 9/100\n", 369 | "484/484 [==============================] - 6s 12ms/step - loss: 4.8807 - accuracy: 0.1138\n", 370 | "Epoch 10/100\n", 371 | "484/484 [==============================] - 6s 12ms/step - loss: 4.7824 - accuracy: 0.1260\n", 372 | "Epoch 11/100\n", 373 | "484/484 [==============================] - 6s 12ms/step - loss: 4.6837 - accuracy: 0.1360\n", 374 | "Epoch 12/100\n", 375 | "484/484 [==============================] - 6s 12ms/step - loss: 4.5882 - accuracy: 0.1440\n", 376 | "Epoch 13/100\n", 377 | "484/484 [==============================] - 6s 12ms/step - loss: 4.4890 - accuracy: 0.1555\n", 378 | "Epoch 14/100\n", 379 | "484/484 [==============================] - 6s 12ms/step - loss: 4.3745 - accuracy: 0.1693\n", 380 | "Epoch 15/100\n", 381 | "484/484 [==============================] - 6s 12ms/step - loss: 4.2824 - accuracy: 0.1785\n", 382 | "Epoch 16/100\n", 383 | "484/484 [==============================] - 6s 12ms/step - loss: 4.1904 - accuracy: 0.1931\n", 384 | "Epoch 17/100\n", 385 | "484/484 [==============================] - 6s 12ms/step - loss: 4.0830 - accuracy: 0.2051\n", 386 | "Epoch 18/100\n", 387 | "484/484 [==============================] - 6s 12ms/step - loss: 3.9772 - accuracy: 0.2203\n", 388 | "Epoch 19/100\n", 389 | "484/484 [==============================] - 6s 12ms/step - loss: 3.8847 - accuracy: 0.2354\n", 390 | "Epoch 20/100\n", 391 | "484/484 [==============================] - 6s 12ms/step - loss: 3.7842 - accuracy: 0.2521\n", 392 | "Epoch 21/100\n", 393 | "484/484 [==============================] - 6s 12ms/step - loss: 3.6994 - accuracy: 0.2668\n", 394 | "Epoch 22/100\n", 395 | "484/484 [==============================] - 6s 12ms/step - loss: 3.5931 - accuracy: 0.2868\n", 396 | "Epoch 23/100\n", 397 | "484/484 [==============================] - 6s 12ms/step - loss: 3.5113 - accuracy: 0.3035\n", 398 | "Epoch 24/100\n", 399 | "484/484 [==============================] - 6s 12ms/step - loss: 3.4228 - accuracy: 0.3202\n", 400 | "Epoch 25/100\n", 401 | "484/484 [==============================] - 6s 12ms/step - loss: 3.3420 - accuracy: 0.3378\n", 402 | "Epoch 26/100\n", 403 | "484/484 [==============================] - 6s 12ms/step - loss: 3.2673 - accuracy: 0.3569\n", 404 | "Epoch 27/100\n", 405 | "484/484 [==============================] - 6s 12ms/step - loss: 3.1884 - accuracy: 0.3732\n", 406 | "Epoch 28/100\n", 407 | "484/484 [==============================] - 6s 12ms/step - loss: 3.1033 - accuracy: 0.3904\n", 408 | "Epoch 29/100\n", 409 | "484/484 [==============================] - 6s 12ms/step - loss: 3.0384 - accuracy: 0.4045\n", 410 | "Epoch 30/100\n", 411 | "484/484 [==============================] - 6s 12ms/step - loss: 2.9588 - accuracy: 0.4251\n", 412 | "Epoch 31/100\n", 413 | "484/484 [==============================] - 6s 12ms/step - loss: 2.8917 - accuracy: 0.4399\n", 414 | "Epoch 32/100\n", 415 | "484/484 [==============================] - 6s 12ms/step - loss: 2.8310 - accuracy: 0.4532\n", 416 | "Epoch 33/100\n", 417 | "484/484 [==============================] - 6s 12ms/step - loss: 2.7758 - accuracy: 0.4635\n", 418 | "Epoch 34/100\n", 419 | "484/484 [==============================] - 6s 12ms/step - loss: 2.7130 - accuracy: 0.4807\n", 420 | "Epoch 35/100\n", 421 | "484/484 [==============================] - 6s 12ms/step - loss: 2.6554 - accuracy: 0.4924\n", 422 | "Epoch 36/100\n", 423 | "484/484 [==============================] - 6s 12ms/step - loss: 2.5997 - accuracy: 0.5082\n", 424 | "Epoch 37/100\n", 425 | "484/484 [==============================] - 6s 12ms/step - loss: 2.5421 - accuracy: 0.5151\n", 426 | "Epoch 38/100\n", 427 | "484/484 [==============================] - 6s 12ms/step - loss: 2.4914 - accuracy: 0.5285\n", 428 | "Epoch 39/100\n", 429 | "484/484 [==============================] - 6s 12ms/step - loss: 2.4422 - accuracy: 0.5413\n", 430 | "Epoch 40/100\n", 431 | "484/484 [==============================] - 6s 12ms/step - loss: 2.3856 - accuracy: 0.5485\n", 432 | "Epoch 41/100\n", 433 | "484/484 [==============================] - 6s 12ms/step - loss: 2.3452 - accuracy: 0.5623\n", 434 | "Epoch 42/100\n", 435 | "484/484 [==============================] - 6s 12ms/step - loss: 2.2964 - accuracy: 0.5711\n", 436 | "Epoch 43/100\n", 437 | "484/484 [==============================] - 6s 12ms/step - loss: 2.2496 - accuracy: 0.5814\n", 438 | "Epoch 44/100\n", 439 | "484/484 [==============================] - 6s 12ms/step - loss: 2.2215 - accuracy: 0.5891\n", 440 | "Epoch 45/100\n", 441 | "484/484 [==============================] - 6s 12ms/step - loss: 2.1712 - accuracy: 0.6034\n", 442 | "Epoch 46/100\n", 443 | "484/484 [==============================] - 6s 12ms/step - loss: 2.1210 - accuracy: 0.6101\n", 444 | "Epoch 47/100\n", 445 | "484/484 [==============================] - 6s 13ms/step - loss: 2.0850 - accuracy: 0.6191\n", 446 | "Epoch 48/100\n", 447 | "484/484 [==============================] - 6s 13ms/step - loss: 2.0475 - accuracy: 0.6285\n", 448 | "Epoch 49/100\n", 449 | "484/484 [==============================] - 6s 12ms/step - loss: 2.0123 - accuracy: 0.6348\n", 450 | "Epoch 50/100\n", 451 | "484/484 [==============================] - 6s 12ms/step - loss: 1.9768 - accuracy: 0.6444\n", 452 | "Epoch 51/100\n", 453 | "484/484 [==============================] - 6s 12ms/step - loss: 1.9433 - accuracy: 0.6499\n", 454 | "Epoch 52/100\n", 455 | "484/484 [==============================] - 6s 12ms/step - loss: 1.9172 - accuracy: 0.6550\n", 456 | "Epoch 53/100\n", 457 | "484/484 [==============================] - 6s 12ms/step - loss: 1.8831 - accuracy: 0.6620\n", 458 | "Epoch 54/100\n", 459 | "484/484 [==============================] - 6s 12ms/step - loss: 1.8411 - accuracy: 0.6734\n", 460 | "Epoch 55/100\n", 461 | "484/484 [==============================] - 6s 12ms/step - loss: 1.8184 - accuracy: 0.6770\n", 462 | "Epoch 56/100\n", 463 | "484/484 [==============================] - 6s 12ms/step - loss: 1.7982 - accuracy: 0.6793\n", 464 | "Epoch 57/100\n", 465 | "484/484 [==============================] - 6s 12ms/step - loss: 1.7684 - accuracy: 0.6883\n", 466 | "Epoch 58/100\n", 467 | "484/484 [==============================] - 6s 12ms/step - loss: 1.7396 - accuracy: 0.6907\n", 468 | "Epoch 59/100\n", 469 | "484/484 [==============================] - 6s 12ms/step - loss: 1.7080 - accuracy: 0.6980\n", 470 | "Epoch 60/100\n", 471 | "484/484 [==============================] - 6s 12ms/step - loss: 1.6925 - accuracy: 0.7027\n", 472 | "Epoch 61/100\n", 473 | "484/484 [==============================] - 6s 12ms/step - loss: 1.6774 - accuracy: 0.7037\n", 474 | "Epoch 62/100\n", 475 | "484/484 [==============================] - 6s 12ms/step - loss: 1.6312 - accuracy: 0.7144\n", 476 | "Epoch 63/100\n", 477 | "484/484 [==============================] - 6s 12ms/step - loss: 1.6200 - accuracy: 0.7167\n", 478 | "Epoch 64/100\n", 479 | "484/484 [==============================] - 6s 12ms/step - loss: 1.6055 - accuracy: 0.7174\n", 480 | "Epoch 65/100\n", 481 | "484/484 [==============================] - 6s 12ms/step - loss: 1.5676 - accuracy: 0.7253\n", 482 | "Epoch 66/100\n", 483 | "484/484 [==============================] - 6s 12ms/step - loss: 1.5443 - accuracy: 0.7340\n", 484 | "Epoch 67/100\n", 485 | "484/484 [==============================] - 6s 12ms/step - loss: 1.5164 - accuracy: 0.7393\n", 486 | "Epoch 68/100\n", 487 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4968 - accuracy: 0.7421\n", 488 | "Epoch 69/100\n", 489 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4979 - accuracy: 0.7391\n", 490 | "Epoch 70/100\n", 491 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4890 - accuracy: 0.7387\n", 492 | "Epoch 71/100\n", 493 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4622 - accuracy: 0.7444\n", 494 | "Epoch 72/100\n", 495 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4361 - accuracy: 0.7477\n", 496 | "Epoch 73/100\n", 497 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4230 - accuracy: 0.7529\n", 498 | "Epoch 74/100\n", 499 | "484/484 [==============================] - 6s 12ms/step - loss: 1.4118 - accuracy: 0.7566\n", 500 | "Epoch 75/100\n", 501 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3859 - accuracy: 0.7623\n", 502 | "Epoch 76/100\n", 503 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3740 - accuracy: 0.7625\n", 504 | "Epoch 77/100\n", 505 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3673 - accuracy: 0.7599\n", 506 | "Epoch 78/100\n", 507 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3408 - accuracy: 0.7698\n", 508 | "Epoch 79/100\n", 509 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3303 - accuracy: 0.7698\n", 510 | "Epoch 80/100\n", 511 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3200 - accuracy: 0.7733\n", 512 | "Epoch 81/100\n", 513 | "484/484 [==============================] - 6s 12ms/step - loss: 1.3032 - accuracy: 0.7754\n", 514 | "Epoch 82/100\n", 515 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2901 - accuracy: 0.7752\n", 516 | "Epoch 83/100\n", 517 | "484/484 [==============================] - 6s 13ms/step - loss: 1.2814 - accuracy: 0.7781\n", 518 | "Epoch 84/100\n", 519 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2731 - accuracy: 0.7780\n", 520 | "Epoch 85/100\n", 521 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2693 - accuracy: 0.7777\n", 522 | "Epoch 86/100\n", 523 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2523 - accuracy: 0.7818\n", 524 | "Epoch 87/100\n", 525 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2257 - accuracy: 0.7883\n", 526 | "Epoch 88/100\n", 527 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2077 - accuracy: 0.7912\n", 528 | "Epoch 89/100\n", 529 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2114 - accuracy: 0.7911\n", 530 | "Epoch 90/100\n", 531 | "484/484 [==============================] - 6s 12ms/step - loss: 1.2119 - accuracy: 0.7883\n", 532 | "Epoch 91/100\n", 533 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1963 - accuracy: 0.7884\n", 534 | "Epoch 92/100\n", 535 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1909 - accuracy: 0.7942\n", 536 | "Epoch 93/100\n", 537 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1948 - accuracy: 0.7881\n", 538 | "Epoch 94/100\n", 539 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1661 - accuracy: 0.7971\n", 540 | "Epoch 95/100\n", 541 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1476 - accuracy: 0.7974\n", 542 | "Epoch 96/100\n", 543 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1390 - accuracy: 0.7992\n", 544 | "Epoch 97/100\n", 545 | "484/484 [==============================] - 6s 12ms/step - loss: 1.1367 - accuracy: 0.7984\n", 546 | "Epoch 98/100\n", 547 | "484/484 [==============================] - 6s 13ms/step - loss: 1.1323 - accuracy: 0.8000\n", 548 | "Epoch 99/100\n", 549 | "484/484 [==============================] - 6s 13ms/step - loss: 1.1167 - accuracy: 0.7999\n", 550 | "Epoch 100/100\n", 551 | "484/484 [==============================] - 6s 13ms/step - loss: 1.1205 - accuracy: 0.8018\n" 552 | ], 553 | "name": "stdout" 554 | } 555 | ] 556 | }, 557 | { 558 | "cell_type": "code", 559 | "metadata": { 560 | "colab_type": "code", 561 | "id": "1fXTEO3GJ282", 562 | "outputId": "fd56354e-9a0b-4874-9774-d369a4b6cfce", 563 | "colab": { 564 | "base_uri": "https://localhost:8080/", 565 | "height": 545 566 | } 567 | }, 568 | "source": [ 569 | "import matplotlib.pyplot as plt\n", 570 | "acc = history.history['accuracy']\n", 571 | "loss = history.history['loss']\n", 572 | "\n", 573 | "epochs = range(len(acc))\n", 574 | "\n", 575 | "plt.plot(epochs, acc, 'b', label='Training accuracy')\n", 576 | "plt.title('Training accuracy')\n", 577 | "\n", 578 | "plt.figure()\n", 579 | "\n", 580 | "plt.plot(epochs, loss, 'b', label='Training Loss')\n", 581 | "plt.title('Training loss')\n", 582 | "plt.legend()\n", 583 | "\n", 584 | "plt.show()" 585 | ], 586 | "execution_count": 0, 587 | "outputs": [ 588 | { 589 | "output_type": "display_data", 590 | "data": { 591 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxV1dX/8c8igIKoKEJRBkHBKnVAG1HrjFpx+IHWoaD1kVZJtSLOFR8HFG0tLXWqOKAyqKU4lEdRUYoKohYxgYJl1AAqQZGoIKIghKzfH/ui15iQG7g3595zv+/XKy/uOWcnZx0OLnfW2Wdvc3dERCT3NYg6ABERSQ8ldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQpesYmYvmtn56W4rkg9M49Bla5nZmqTNpsA3wMbE9m/d/e/1H5VI/lFCl7Qys/eBC9395WqONXT3ivqPKrfo70m2lEoukjFmdoyZlZnZtWa2HBhpZjuZ2fNmVm5mKxOf2yZ9zxQzuzDxua+ZvWFmQxNtl5jZSVvYtqOZTTWzL83sZTMbZmaP1xB3bTHubGYjzeyjxPFnko71MrNZZrbazBaZWY/E/vfN7PikdjdvOr+ZdTAzN7MLzOxD4NXE/qfMbLmZfZGI/SdJ39/EzP5qZh8kjr+R2PeCmV1a5XreMbPT63r/JPcooUumtQZ2BnYHigj/5kYmttsDa4F7N/P9hwALgV2APwOPmJltQdsxwNtAC+Bm4LzNnLO2GB8jlJZ+ArQC7gQws27Ao8A1QHPgKOD9zZynqqOBfYATE9svAp0T55gJJJeuhgI/BX5G+Pv9PVAJjAZ+tamRmR0AtAFeqEMckqvcXV/6StsXIYEdn/h8DLAe2HYz7bsCK5O2pxBKNgB9gdKkY00BB1rXpS0hKVcATZOOPw48nuI1fRsjsCshce5UTbsHgTtr+3tJbN+86fxAh0Sse2wmhuaJNjsS/oezFjigmnbbAiuBzontocB9Uf+70Ff9fKmHLplW7u7rNm2YWVMzezBRKlgNTAWam1lBDd+/fNMHd/868bFZHdvuBnyetA9gaU0B1xJju8TPWlnNt7YDFtX0c1PwbUxmVmBmf0qUbVbzXU9/l8TXttWdK/F3/QTwKzNrAPQh/EYheUAJXTKt6lP3q4AfA4e4+w6EsgRATWWUdPgY2NnMmibta7eZ9puLcWniZzWv5vuWAnvW8DO/IvzWsEnratok/12dA/QCjif0yjskxfApsG4z5xoNnAscB3zt7tNqaCcxo4Qu9W17QrlglZntDAzK9And/QOgBLjZzBqb2WHA/9uSGN39Y0Jt+77Ew9NGZrYp4T8C/NrMjjOzBmbWxsz2ThybBfROtC8Ezqwl7O0Jwz8/I/yP4I9JMVQCI4A7zGy3RG/+MDPbJnF8GqEs9FfUO88rSuhS3+4CmhB6mW8BL9XTec8FDiMkyNsIZYlvamhbW4znARuABcAK4HIAd38b+DXhIekXwGuEB6sANxJ61CuBWwgPaTfnUeADYBkwLxFHsquB/wLFwOfAEL7/3/OjwH6EZwWSJzQOXfKSmT0BLHD3jP+GEAUz+x+gyN2PiDoWqT/qoUteMLODzWzPRCmkB6E+/Uxt35eLEs8KfgcMjzoWqV9K6JIvWhOGOa4B7gEudvf/RBpRBpjZiUA58Am1l3UkZlRyERGJCfXQRURiomFUJ95ll128Q4cOUZ1eRCQnzZgx41N3b1ndscgSeocOHSgpKYnq9CIiOcnMPqjpmEouIiIxoYQuIhITSugiIjGhhC4iEhNK6CIiMaGELiISEykldDPrYWYLzazUzAZWc7y9mU02s/8k1i88Of2hiojI5tQ6Dj2xSssw4ASgDCg2s/HuPi+p2Q3Ak+5+v5l1ASbw3YT8IiJ5Z906cIeCAqishEWLYN48mD8fTjkFfvrT9J8zlReLuhHWalwMYGZjCTPVJSd0B3ZIfN4R+CidQYqIROnLL2HaNFi7NiTqVavg3Xdh4UL46CPYYQfYaSfYdlv44IOQvFesqP5nmUGrVtEl9DZ8f/3FMsLq6sluBv5lZpcC2xGWzfoBMysirPxO+/bt6xqriEjGuMOECSFpn3566FkDzJoFZ5wBixd/v32TJtC5M7RtGxJ+aSl8/TXsvjv06hX+LCiAjRtD+44doUsX2GsvaNqUjEjXq/99gFHu/tfE8l6Pmdm+iaWyvuXuw0nM0VxYWKhpHkUk7dxDz/nVV2Hq1JBsCwqgYUPYc0848kg44gjYeefQvrISnnkGBg+G2bPDvn32gdtug9Wr4eKLQ9tnnoH27WGbbUKPfLfdoEGWDStJJaEv4/sL6rZN7Et2AdADwnqGZrYtYWXyGn7pEBFJr8pKeOwxuPlmeP/9sK9du1De2LgRNmyAF16AoUPDsWbNwv6KinCsc2cYPTr0vG+6KfTKAY49FsaODT8n26WS0IuBzmbWkZDIexNWJE/2IWGF8VFmtg+wLWGSfRGRjHKHV16Ba64J5ZGDD4b//V/o3h322CPUrDdZuxaKi+GNN+DTT0PPvaAADjgAzjor9OIhlFz+/ncoL4fLL/9uf7arNUx3rzCz/sBEoAAY4e5zzWwwUOLu44GrgIfM7ArCA9K+rpUzRGQrffklzJ373WiRhg2hUSNo3DjUq8eNgzFjQn27ffuQhHv3rrkU0qQJHHVU+Nqchg3h/PPTfz2ZFtmKRYWFha7pc0Xy29q1IWHPng0ffhgSacOGYRTJa69BScl3DxWr06ABHHcc9OkTEnmTJvUXe1TMbIa7F1Z3LEd+kRCRXOUe6tSVleHzggWhlv3886H8UV3CbtgQDjkEBg6Ebt1Cr7yy8rt69/r1oV337tC6df1eTzZTQheRjFmxAs48E15//YfHDj44JOwDDww17D32+C75N2gQkrjUjRK6iGyxigr46qvwZ0VFGN63KRHPnQunngqffALXXw/bbRceULZuDT161Nyz3jT+W+pOCV1E6uyLL+Cee+DOO2Hlyu/2N20aSiQHHQQPPxy2p06FwmorvpJuSugikhL3MBfJP/4Bw4aFB5c9e8Ixx4Sad4MG4XX4N9+Eu++GffeF8ePD6BOpH0roIrJZH34IDzwATz8N770X9p12Wnj55sADq/+etWvDvCbJY8Al85TQReQH3GH69NDTfuqpsK97d7jiijBPyW67bf7782H4YDZSQhcRIAwHfO65MKTwpZfCLII77ghXXgn9+6t0kguU0EVibNO0r1OnhgR9+OGhp92x43dtvvkmzGFy++1hDpQdd4QTToCTTgqvw2+/fWThSx0poYvEjHuYq+QvfwnTwW7cGIYC7rgjjBwZ2vzoR2HGwCZNwljx5cvDizx/+1sYUpgrc5fI9+m2icTIlCnhZZ3p06FFC7jqKjj+eDj00DC74IIFYVrZmTPDXCjr1oX5ufv1C71yPcTMbUroIjGwcWOYz/vWW8PCCsOGQd++P1xIYZ99wpfEkxK6SI776CM499zQO+/bF+69N7yVKfkny9bbEJG6ePJJ2G8/ePttGDUq1MiVzPOXErpIDnEPI1cWLQrTxf7yl9CpE8yYkZvzd0t6qeQiksWKi8OY8BkzwtdHH4VpZCFMgnXbbXDttRqVIoH+GYhkqUcegaKi0Cvfa6+wyk7HjrDTTmEI4uGH6wGnfF9KCd3MegB3E5age9jd/1Tl+J3AsYnNpkArd2+ezkBF8oU7/PnPYfjhiSeGybB22inqqCQX1JrQzawAGAacAJQBxWY23t3nbWrj7lcktb8UqGHKHhGpyh3mzAnrYn7ySXizc9SoUCMfPTqsnymSilR66N2AUndfDGBmY4FewLwa2vcBBqUnPJH4+uILePxxGD4c3nnnu/1mMGBAmGu8psWORaqTSkJvAyxN2i4DDqmuoZntDnQEXq3heBFQBNBeM/1Intq4MYwVv+EGWLMmTEF7//1hSbYf/QhatVKvXLZMuh+K9gaedvdq1+l29+HAcIDCwkJP87lFst7s2eE1++LiMPnV4MFazUfSJ5WEvgxol7TdNrGvOr2BS7Y2KJG4+Oqr8LLPW2+FJP7uu9CyJYwZE2rkmjtF0imVhF4MdDazjoRE3hs4p2ojM9sb2AmYltYIRXLUu+/CGWeEB5677RZKKn37hqGILVpEHZ3EUa0J3d0rzKw/MJEwbHGEu881s8FAibuPTzTtDYx1d5VSJO/93/+F5N2oEUycCD//edQRST5IqYbu7hOACVX23VRl++b0hSWSO9yhpARefjlMSztzZhiC2K1bWL5Nz/+lvuhNUZEttGQJDB0Kzz4LyxJPlfbcE376U7j0Urj4Ythmm2hjlPyihC6yBRYsCEu5rVoVVvg57TQ4+WTYZZeoI5N8poQuUkdz58Jxx4XPxcXwk59EG4/IJnoPTaQOSkrg2GPDG5xTpiiZS3ZRQhdJwapV4XX8Qw4JdfEpU2DvvaOOSuT7lNBFNuPrr8Nr+XvvHdbpvPjiMO/KXntFHZnID6mGLlKNjz4K8608+CB8/jkcdhhMmAAHHRR1ZCI1U0IXSVJaGuYiHz0aNmwIo1euuAKOOEKv6Uv2U0IXAdatCzXyRx4Jb3f+5jdw9dVhXLlIrlBCl7y3fHnoiU+fDpdfHtbobN066qhE6k4JXfLazJnQq1eok48bB6efHnVEIltOo1wkLy1fHkasdOsWauNvvqlkLrlPCV3yysaNcNtt0KkTPPwwXHQRzJgBXbtGHZnI1lPJRfLGypVwzjnw0ktw5plw++0hsYvEhRK65IX580Ot/P33w6LM/fpFHZFI+imhS+yNGxcWm2jSBF59NYwpF4kj1dAltioq4Pe/D8vA7bNPmFhLyVziTAldYmnhwjDF7V/+EkazTJ0K7drV/n0iuSylhG5mPcxsoZmVmtnAGtqcbWbzzGyumY1Jb5giqVm5Mryqv+++MGsWPPoo3HefVg6S/FBrDd3MCoBhwAlAGVBsZuPdfV5Sm87AdcDh7r7SzFplKmCRmsyZE1YR+vRTuPDCMDyxlf4lSh5J5aFoN6DU3RcDmNlYoBcwL6lNP2CYu68EcPcV6Q5UZHM+/zyMYikoCOPKDzww6ohE6l8qJZc2wNKk7bLEvmR7AXuZ2Ztm9paZ9ajuB5lZkZmVmFlJeXn5lkUsUkVFBfTuDWVlYUSLkrnkq3Q9FG0IdAaOAfoAD5lZ86qN3H24uxe6e2HLli3TdGrJd9ddB5MmhVr5YYdFHY1IdFJJ6MuA5PEBbRP7kpUB4919g7svAd4lJHiRjFm9Ory6P3QoXHIJXHBB1BGJRCuVhF4MdDazjmbWGOgNjK/S5hlC7xwz24VQglmcxjhFvmfixDCS5aGH4Kqr4M47o45IJHq1JnR3rwD6AxOB+cCT7j7XzAabWc9Es4nAZ2Y2D5gMXOPun2UqaMlf7nDjjdCjBzRrBv/+d+ihN2oUdWQi0TN3j+TEhYWFXlJSEsm5JTdt2BBKLCNGhBWFhg2DbbeNOiqR+mVmM9y9sLpjelNUcsKaNWFVoREjYNCgMPWtkrnI92lyLsl6c+bAWWfBu+/Cgw9CUVHUEYlkJ/XQJauNHBlWFVq5MgxNVDIXqZkSumStm28OtfJDDw3zsnTvHnVEItlNCV2y0pgxcMstYR7zSZOgdeuoIxLJfkroknWmTQs986OPDjXzgoKoIxLJDUroklWWLAmjWdq2hX/+Exo3jjoikdyhhC5ZYeNG+Nvf4IAD4Jtv4LnnoEWLqKMSyS1K6BK5OXPgkENgwIAwudaMGWHJOBGpG41Dl0i99x4ce2yok48dC2efDWZRRyWSm5TQJTLLl8OJJ4YE/vrr0Fnzc4psFSV0icTq1XDSSbBiBUyerGQukg5K6FLv1q+HM84ItfPnn4eDD446IpF4UEKXeuUeZkx8+WUYNSqUXEQkPTTKRerVH/4Q5mcZNAjOPz/qaETiRT10qRfuYcrbG2+E884LCV1E0ks9dMm4BQtCaaWoKEyw9dBDGpookgkpJXQz62FmC82s1MwGVnO8r5mVm9msxNeF6Q9VctFtt8F++8Hbb8M994S1QLfZJuqoROKp1pKLmRUAw4ATgDKg2MzGu/u8Kk2fcPf+GYhRctRTT4USy9lnh9f6W7WKOiKReEulh94NKHX3xe6+HhgL9MpsWJLrliyBfv3CK/2PP65kLlIfUknobYClSdtliX1VnWFm75jZ02bWrrofZGZFZlZiZiXl5eVbEK7kgg0boE+f8Pkf/4BGjaKNRyRfpOuh6HNAB3ffH5gEjK6ukbsPd/dCdy9s2bJlmk4t2ebGG2H69PDws2PHqKMRyR+pJPRlQHKPu21i37fc/TN3/yax+TDw0/SEJ7nmzjthyJAwouWss6KORiS/pJLQi4HOZtbRzBoDvYHxyQ3MbNekzZ7A/PSFKLninnvgyivDa/333ht1NCL5p9ZRLu5eYWb9gYlAATDC3eea2WCgxN3HAwPMrCdQAXwO9M1gzJKFhg2Dyy6D009X3VwkKubukZy4sLDQS0pKIjm3pNfrr4f1P3v2hCef1LJxIplkZjPcvbC6Y3pTVLbKV1/Br38NHTqE4YlK5iLR0VwuslUGDoRFi2DKFGjWLOpoRPKbeuiyxSZPDg8/BwwIJRcRiZYSumyRpUtDqaVTJ7j99qijERFQyUW2wOzZcPLJsGYNTJoETZtGHZGIgHroUkeTJsGRR4bpb994A7p1izoiEdlECV1SNmlS6Jl36ABvvRWmxRWR7KGSi6Tkv/8Nb4Dusw9MnQrNm0cdkYhUpR661GrZstAz3357mDBByVwkW6mHLpu1Zg2ceiqsWhXeCG3bNuqIRKQmSuhSI3e44AJ45x144QXo2jXqiERkc5TQpUZ33RXmZhkyBHr0iDoaEamNauhSrddeg2uugV/8IvwpItlPCV1+YPnysLBzp04wcmQYcy4i2U8lF/mB664LD0EnT4Yddog6GhFJlXro8j0zZ8Lo0WGxii5doo5GROpCCV2+5Q5XXQUtWsD110cdjYjUlUou8q1nnw3zmt93H+y4Y9TRiEhdpdRDN7MeZrbQzErNbOBm2p1hZm5m1S6PJNlr/fowmmWffaBfv6ijEZEtUWsP3cwKgGHACUAZUGxm4919XpV22wOXAdMzEahk1qBBUFoaXu1vqN/bRHJSKj30bkCpuy929/XAWKBXNe1uBYYA69IYn9SDMWPgT38KPfOTToo6GhHZUqkk9DbA0qTtssS+b5nZQUA7d39hcz/IzIrMrMTMSsrLy+scrKRfcXF4vf+oo8JyciKSu7Z6lIuZNQDuAK6qra27D3f3QncvbNmy5daeWrbSxx/DaadB69bw9NPQuHHUEYnI1kilWroMaJe03Taxb5PtgX2BKRZeKWwNjDeznu5ekq5AJb3cw5qgq1bBtGmg/7+K5L5UEnox0NnMOhISeW/gnE0H3f0LYJdN22Y2BbhayTy7PfIITJwIw4bB/vtHHY2IpEOtJRd3rwD6AxOB+cCT7j7XzAabWc9MByjp98EHcOWVcOyxcNFFUUcjIumS0gA1d58ATKiy76Ya2h6z9WFJpmya49wdRoyABnpXWCQ2NOI4zzzwALzySvizQ4eooxGRdFL/LI/Mnx/mavn5z6GoKOpoRCTdlNDzxDffwDnnwHbbwahRmuNcJI5UcskTN94Is2aFCbh23TXqaEQkE9RDzwOvvgpDh4YRLT01LkkktpTQY27t2jBHS6dO8Ne/Rh2NiGSSSi4xN2QILF4ML78MTZtGHY2IZJJ66DFWWhpmUezTB447LupoRCTTlNBjyh369w8TbqnUIpIfVHKJqXHjwlwtd92lUS0i+UI99BhauRIuvRQOOAAuuSTqaESkvqiHHkNXXgkrVsDzz2s5OZF8oh56zLz4YngT9Npr4aCDoo5GROqTEnqMrF4d5mjp0gVuqnYuTBGJM/1CHiNXXw0ffRSWk9tmm6ijEZH6ph56TLzwAjz0UEjqhxwSdTQiEgUl9BgoLw+LVuy/PwweHHU0IhKVlBK6mfUws4VmVmpmA6s5fpGZ/dfMZpnZG2bWJf2hSnXcQ9185Up47DGVWkTyWa0J3cwKgGHASUAXoE81CXuMu+/n7l2BPwN3pD1Sqdbo0fDMM/CHP2ixZ5F8l0oPvRtQ6u6L3X09MBboldzA3VcnbW4HePpClJosWQIDBsBRR8EVV0QdjYhELZVRLm2ApUnbZcAPHruZ2SXAlUBjoHt1P8jMioAigPbt29c1VkmycSOcf374/OijUFAQbTwiEr20PRR192HuvidwLXBDDW2Gu3uhuxe2bNkyXafOS3fcAa+/DvfeC7vvHnU0IpINUknoy4B2SdttE/tqMhY4bWuCks2bPRuuvx7OOAPOOy/qaEQkW6SS0IuBzmbW0cwaA72B8ckNzKxz0uYpwHvpC1GSVVSEUkuLFvDAA1rsWUS+U2sN3d0rzKw/MBEoAEa4+1wzGwyUuPt4oL+ZHQ9sAFYC52cy6Hw2fHjoof/zn7DLLlFHIyLZxNyjGZBSWFjoJSUlkZw7V33+OXTuDF27hiXl1DsXyT9mNsPdC6s7pjdFc8jNN8OqVWHRCiVzEalKCT1HzJ0L990HF10E++0XdTQiko2U0HNAZSVcdhnssIPmahGRmmn63Bzwpz/BK6/A/feH0S0iItVRDz3LTZwIN9wA55wDv/1t1NGISDZTQs9iS5ZAnz6hZj58uB6EisjmKaFnqXXr4Be/CNPjjhsH220XdUQiku1UQ89SN9wAs2bBc8/BnntGHY2I5AL10LPQa6+FybcuughOPTXqaEQkVyihZ5nVq8NcLXvuCUOHRh2NiOQSlVyyzGWXwdKl8OabqpuLSN2oh55FxoyBUaPguuvg0EOjjkZEco0SepaYOxf69YMjj4RBg6KORkRykRJ6Fli9OgxR3H57eOIJaNQo6ohEJBephh4xd7jgAli0KLzev+uuUUckIrlKCT1id98NTz8NQ4bA0UdHHY2I5DKVXCI0bRpccw2cdlr4U0RkayihR+TTT+Hss6F9exg5UvO0iMjWSymhm1kPM1toZqVmNrCa41ea2Twze8fMXjGz3dMfanxUVsJ550F5OTz1FDRvHnVEIhIHtSZ0MysAhgEnAV2APmbWpUqz/wCF7r4/8DTw53QHGhfu0L8/vPRSqJ8fdFDUEYlIXKTSQ+8GlLr7YndfD4wFeiU3cPfJ7v51YvMtoG16w4wHd7j66rBQxbXXQlFR1BGJSJykktDbAEuTtssS+2pyAfBidQfMrMjMSsyspLy8PPUoY2LQoDDp1qWXwu23q24uIumV1oeiZvYroBD4S3XH3X24uxe6e2HLli3Teeqsd//9cOutYcz5XXcpmYtI+qUyDn0Z0C5pu21i3/eY2fHA9cDR7v5NesKLhzfegAED4JRT4MEHoYHGFolIBqSSWoqBzmbW0cwaA72B8ckNzOxA4EGgp7uvSH+YuWvZMjjzTOjYER5/HAoKoo5IROKq1h66u1eYWX9gIlAAjHD3uWY2GChx9/GEEksz4CkLtYQP3b1nBuPOCd98E5L5V1/Bq69qeKKIZFZKr/67+wRgQpV9NyV9Pj7NceW8Tz6BX/4S3norvNrfpepATxGRNNNcLhnw9tth9sTPPoPHHoMzzog6IhHJB3o8l2bjxoU5zRs1gn//G371q6gjEpF8oR56GhUXw7nnhrc/n38eWrSIOiIRySfqoadJWRn06gWtW8OzzyqZi0j9Uw89Db76KiTzL7+Ef/0LWrWKOiIRyUdK6Ftp3To46yyYNQvGj4d99406IhHJV0roW2HdurA4xcSJ8NBD4U1QEZGoKKFvobVrQzKfNAkefjjM0SIiEiU9FN0CCxeG9T+VzEUkmyih10FlJdxzD3TtCosWhTdAf/ObqKMSEQlUcknRl19C794wYQKcfHLome+6a9RRiYh8Rwk9BWVlcOqpMGcO3Hsv/O53ms9cRLKPEnotZs8OPfIvv4QXXoATT4w6IhGR6imhb8bs2XDMMdCsWVikYv/9o45IRKRmSug1WLgQfv7zkMxffx06dIg6IhGRzdMol2q8/z4cn5jh/eWXlcxFJDcooSdxhyeegJ/9DNasCfOy/PjHUUclIpKalBK6mfUws4VmVmpmA6s5fpSZzTSzCjM7M/1hZt6CBXDCCWFo4q67wuTJcMABUUclIpK6WhO6mRUAw4CTgC5AHzOruqDah0BfYEy6A8y0zz6DAQNgv/1gxgy4776w4lDXrlFHJiJSN6k8FO0GlLr7YgAzGwv0AuZtauDu7yeOVWYgxoxwD2PKb7oJVq+GoiK45RZNfSsiuSuVkksbYGnSdlliX52ZWZGZlZhZSXl5+Zb8iLSorIT+/UPP/OCDw/DE++9XMheR3FavD0Xdfbi7F7p7YcuWLevz1N/asAHOPz+UVq65Jkx9qznMRSQOUknoy4B2SdttE/tyzqefwi9+AY8/Dn/8IwwZolf4RSQ+UknoxUBnM+toZo2B3sD4zIaVXpWV8MgjYQjiSy/BsGFw3XVK5iISL7UmdHevAPoDE4H5wJPuPtfMBptZTwAzO9jMyoCzgAfNbG4mg66LadPgyCPhwgvhJz+B//wnTK4lIhI3Kb367+4TgAlV9t2U9LmYUIrJGnPmwPXXh3U+W7WCkSND7Vy9chGJq9i9KbpmDVxxRXgpaMoUuO22sBhF375K5iISb7GanGviRPjtb+GDD+Dii+HWW6FFi6ijEhGpHzmd0Csr4bXX4Lnnwlzl774bHny+/joccUTU0YmI1K+cTeiffQbnnQcvvgiNG0P37nD55fDrX8O220YdnYhI/cvJhD59Opx9NixfDnffHRZqbtYs6qhERKKVcwl99Gjo1w/atIE334TCwqgjEhHJDjk3yqVTp7Bg88yZSuYiIslyrod++OHhS0REvi/neugiIlI9JXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZgwd4/mxGblwAdb+O27AJ+mMZxckY/XnY/XDPl53fl4zVD3697d3VtWdyCyhL41zKzE3fPuxf98vO58vGbIz+vOx2uG9F63Si4iIjGhhC4iEhO5mtCHRx1ARPLxuvPxmiE/rzsfrxnSeN05WVhXaugAAAOESURBVEMXEZEfytUeuoiIVKGELiISEzmX0M2sh5ktNLNSMxsYdTyZYGbtzGyymc0zs7lmdlli/85mNsnM3kv8uVPUsaabmRWY2X/M7PnEdkczm56430+YWeOoY0w3M2tuZk+b2QIzm29mh+XJvb4i8e97jpn9w8y2jdv9NrMRZrbCzOYk7av23lpwT+La3zGzg+p6vpxK6GZWAAwDTgK6AH3MrEu0UWVEBXCVu3cBDgUuSVznQOAVd+8MvJLYjpvLgPlJ20OAO929E7ASuCCSqDLrbuAld98bOIBw/bG+12bWBhgAFLr7vkAB0Jv43e9RQI8q+2q6tycBnRNfRcD9dT1ZTiV0oBtQ6u6L3X09MBboFXFMaefuH7v7zMTnLwn/gbchXOvoRLPRwGnRRJgZZtYWOAV4OLFtQHfg6USTOF7zjsBRwCMA7r7e3VcR83ud0BBoYmYNgabAx8Tsfrv7VODzKrtrure9gEc9eAtobma71uV8uZbQ2wBLk7bLEvtiy8w6AAcC04EfufvHiUPLgR9FFFam3AX8HqhMbLcAVrl7RWI7jve7I1AOjEyUmh42s+2I+b1292XAUOBDQiL/AphB/O831Hxvtzq/5VpCzytm1gz4J3C5u69OPuZhvGlsxpya2anACnefEXUs9awhcBBwv7sfCHxFlfJK3O41QKJu3IvwP7TdgO34YWki9tJ9b3MtoS8D2iVtt03six0za0RI5n9393GJ3Z9s+hUs8eeKqOLLgMOBnmb2PqGU1p1QW26e+JUc4nm/y4Ayd5+e2H6akODjfK8BjgeWuHu5u28AxhH+DcT9fkPN93ar81uuJfRioHPiSXhjwkOU8RHHlHaJ2vEjwHx3vyPp0Hjg/MTn84Fn6zu2THH369y9rbt3INzXV939XGAycGaiWayuGcDdlwNLzezHiV3HAfOI8b1O+BA41MyaJv69b7ruWN/vhJru7XjgfxKjXQ4FvkgqzaTG3XPqCzgZeBdYBFwfdTwZusYjCL+GvQPMSnydTKgpvwK8B7wM7Bx1rBm6/mOA5xOf9wDeBkqBp4Btoo4vA9fbFShJ3O9ngJ3y4V4DtwALgDnAY8A2cbvfwD8Izwg2EH4bu6CmewsYYRTfIuC/hBFAdTqfXv0XEYmJXCu5iIhIDZTQRURiQgldRCQmlNBFRGJCCV1EJCaU0EVEYkIJXUQkJv4/PSa7SmmliYoAAAAASUVORK5CYII=\n", 592 | "text/plain": [ 593 | "
" 594 | ] 595 | }, 596 | "metadata": { 597 | "tags": [], 598 | "needs_background": "light" 599 | } 600 | }, 601 | { 602 | "output_type": "display_data", 603 | "data": { 604 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhU5d3/8feXBAirQIgIRFkqghQ02rCqNaBSFNzBR0ARxQe0IkhtQdsfFRWtWDdwX9gesYAKKuJWFzaLimERQcGFRYIiS2UTkO3+/XFPNGpCEpjJOTPzeV3XXGZmzky+pyf9eHufezHnHCIiEl7lgi5AREQOTkEtIhJyCmoRkZBTUIuIhJyCWkQk5BTUIiIhp6CW0DOz18zsimgfW8oacswsL9rfK1ISqUEXIInJzHYUeFoZ+AHYH3ne3zn3TEm/yzl3diyOFYkXCmqJCedc1fyfzWw1cLVz7q1fHmdmqc65fWVZm0i8UdeHlKn8LgQzG2pm64FxZlbTzGaY2UYz+y7yc2aBz8wys6sjP/cxs3fN7J7IsavM7OxDPLaRmc0xs+1m9paZPWxmE0t4HsdHftcWM1tmZucVeO8cM/sk8r3rzOzPkddrR85ti5n918zmmpn+PyjF0h+JBOEooBbQAOiH/zscF3l+DLALeOggn28DrABqA3cDY8zMDuHYfwHzgXRgOHB5SYo3s/LAy8C/gSOB64FnzKxp5JAx+O6dakAL4J3I6zcCeUAGUAf4K6A1HKRYCmoJwgHgFufcD865Xc65zc65qc65nc657cAdwOkH+fwa59yTzrn9wASgLj74SnysmR0DtAL+7pzb45x7F5hewvrbAlWBuyKffQeYAfSIvL8XaG5m1Z1z3znnFhZ4vS7QwDm31zk312mxHSkBBbUEYaNzbnf+EzOrbGaPm9kaM9sGzAFqmFlKEZ9fn/+Dc25n5MeqpTy2HvDfAq8BrC1h/fWAtc65AwVeWwPUj/x8MXAOsMbMZptZu8jr/wS+AP5tZivN7KYS/j5JcgpqCcIvW5E3Ak2BNs656sDvI68X1Z0RDd8AtcyscoHXji7hZ78Gjv5F//IxwDoA59yHzrnz8d0iLwLPRl7f7py70TnXGDgP+JOZnXGY5yFJQEEtYVAN3y+9xcxqAbfE+hc659YAucBwM6sQafWeW8KPfwDsBIaYWXkzy4l8dnLku3qZ2RHOub3ANnxXD2bW1cyOjfSRb8UPVzxQ+K8Q+YmCWsLgAaASsAl4H3i9jH5vL6AdsBkYAUzBj/c+KOfcHnwwn42v+RGgt3NueeSQy4HVkW6cayK/B6AJ8BawA3gPeMQ5NzNqZyMJy3QvQ8QzsynAcudczFv0IqWhFrUkLTNrZWa/MbNyZtYZOB/fpywSKpqZKMnsKGAafhx1HnCtc25RsCWJ/Jq6PkREQk5dHyIiIReTro/atWu7hg0bxuKrRUQS0oIFCzY55zIKey8mQd2wYUNyc3Nj8dUiIgnJzNYU9Z66PkREQk5BLSIScgpqEZGQ0zhqkSSwd+9e8vLy2L17d/EHS0ylpaWRmZlJ+fLlS/wZBbVIEsjLy6NatWo0bNiQovdYkFhzzrF582by8vJo1KhRiT+nrg+RJLB7927S09MV0gEzM9LT00v9XzYKapEkoZAOh0O5DqEJ6v37YeRImD8/6EpERMIlNEG9Ywc89BBccQXs2hV0NSISTZs3byYrK4usrCyOOuoo6tev/+PzPXv2HPSzubm5DBw4sNjf0b59+6jUOmvWLLp27RqV74qW0NxMPOIIGDcOzjoL/vY3uO++oCsSkWhJT09n8eLFAAwfPpyqVavy5z//+cf39+3bR2pq4XGUnZ1NdnZ2sb9j3rx50Sk2hELTogY480y47jp44AGYPTvoakQklvr06cM111xDmzZtGDJkCPPnz6ddu3acdNJJtG/fnhUrVgA/b+EOHz6cq666ipycHBo3bszo0aN//L6qVav+eHxOTg7dunWjWbNm9OrVi/xVQl999VWaNWvG7373OwYOHFiqlvOkSZNo2bIlLVq0YOjQoQDs37+fPn360KJFC1q2bMn9998PwOjRo2nevDknnHACl1566WH/bxWaFnW+kSPhjTegTx9YsgSqVQu6IpHEcsMNEGncRk1Wlm9glVZeXh7z5s0jJSWFbdu2MXfuXFJTU3nrrbf461//ytSpU3/1meXLlzNz5ky2b99O06ZNufbaa381JnnRokUsW7aMevXqccopp/Cf//yH7Oxs+vfvz5w5c2jUqBE9evQocZ1ff/01Q4cOZcGCBdSsWZNOnTrx4osvcvTRR7Nu3TqWLl0KwJYtWwC46667WLVqFRUrVvzxtcMRqhY1QJUqMGECfPUVDBgAWi5bJHF1796dlJQUALZu3Ur37t1p0aIFgwcPZtmyZYV+pkuXLlSsWJHatWtz5JFH8u233/7qmNatW5OZmUm5cuXIyspi9erVLF++nMaNG/84frk0Qf3hhx+Sk5NDRkYGqamp9OrVizlz5tC4cWNWrlzJ9ddfz+uvv0716tUBOOGEE+jVqxcTJ04sskunNELXogZo3x6GDYNbb4W2beHaa4OuSCRxHErLN1aqVKny48/Dhg2jQ4cOvPDCC6xevZqcnJxCP1OxYsUff05JSWHfvn2HdEw01KxZk48++og33niDxx57jGeffZaxY8fyyiuvMGfOHF5++WXuuOMOPv7448MK7NC1qPP9/e9wzjkwaBAk8D0CEYnYunUr9evXB2D8+PFR//6mTZuycuVKVq9eDcCUKVNK/NnWrVsze/ZsNm3axP79+5k0aRKnn346mzZt4sCBA1x88cWMGDGChQsXcuDAAdauXUuHDh0YOXIkW7duZceOHYdVeyhb1ADlysHEidCqFXTrBgsXwlFHBV2ViMTKkCFDuOKKKxgxYgRdunSJ+vdXqlSJRx55hM6dO1OlShVatWpV5LFvv/02mZmZPz5/7rnnuOuuu+jQoQPOObp06cL555/PRx99xJVXXsmBAwcA+Mc//sH+/fu57LLL2Lp1K845Bg4cSI0aNQ6r9pjsmZidne2itXHAkiXQrh20aQNvveUDXERK59NPP+X4448PuozA7dixg6pVq+Kc47rrrqNJkyYMHjy4zOso7HqY2QLnXKHjEEMfeyecAKNHw8yZ8OCDQVcjIvHsySefJCsri9/+9rds3bqV/v37B11SiYS+RQ1+5Me558Lbb8OiRdCsWdS+WiQpqEUdLgnXogYwg6ee8kP3Lr8c9u4NuiKR+BOLRpmU3qFch7gIavA3Eh99FHJz4c47g65GJL6kpaWxefNmhXXA8tejTktLK9XnQjvqozDdu0PPnnD77X7o3kFu2opIAZmZmeTl5bFx48agS0l6+Tu8lEZc9FEX9N13/gZj5cp+yF6B8fIiInEr7vuoC6pZ008x/+wzGDIk6GpERGIv7oIaoGNHGDwYHnkEXnst6GpERGIrLoMa/A3FFi3gyitB3W4iksjiNqjT0uCZZ3yfdd++WmVPRBJX3AY1+JuKI0fCyy/DY48FXY2ISGzEdVADDBwIf/gD/OlP8MknQVcjIhJ9cR/U5crB+PF+J5iePWH37qArEhGJrrgPavCzFseNg48+0pA9EUk8CRHUAF26+L3gHnwQXnop6GpERKInYYIa4K674KST4KqrYO3aoKsREYmOhArqihVhyhTYswd69YL9+4OuSETk8JUoqM1stZl9bGaLzSw2i3hESZMm8PDDMHcu3HNP0NWIiBy+0rSoOzjnsopaNCRMLr/c77M4bBgsXhx0NSIihyehuj7ymfkJMLVrw2WXacieiMS3kga1A/5tZgvMrF9hB5hZPzPLNbPcMKx5m54OY8fCsmXw//5f0NWIiBy6kgb1qc65k4GzgevM7Pe/PMA594RzLts5l52RkRHVIg9V587wxz/CvffCq68GXY2IyKEpUVA759ZF/rkBeAFoHcuioumeeyAry3eBrFoVdDUiIqVXbFCbWRUzq5b/M9AJWBrrwqKlUiWYOtWvrtetm/qrRST+lKRFXQd418w+AuYDrzjnXo9tWdHVuDE8/bTfumvAgKCrEREpnWI3t3XOrQROLINaYqprV/jb3+COO6BDBz8hRkQkHiTk8LyiDB8Op54K114LK1cGXY2ISMkkVVCnpsLEiX5p1J49Ye/eoCsSESleUgU1QIMG8MQT8MEHcNttQVcjIlK8pAtqgEsu8Zvi3nEHzJ4ddDUiIgeXlEENMHo0HHusH1/93XdBVyMiUrSkDeqqVf0u5uvXQ79+2sVcRMIraYMaoFUrGDECnn/eb+UlIhJGSR3UAH/5ix9Xff31sGJF0NWIiPxa0gd1uXJ+1mKlSvA//6Mp5iISPkkf1AD168OECX4X8xtvDLoaEZGfU1BHdOniQ/qRR3yftYhIWCioC7jzTmjdGvr21ZKoIhIeCuoCKlSAyZP9z716wb59wdYjIgIK6l9p1Mjvt/jee37onohI0BTUhejRw+9kfvvtMG9e0NWISLJTUBfhoYf8Ak69esHWrUFXIyLJTEFdhOrV/RTztWthyJCgqxGRZKagPoh27WDQIL8s6nvvBV2NiCQrBXUxhg+HzEy45hqNAhGRYCioi1Gtml8SdckSGDUq6GpEJBkpqEvgggv85ri33AJffRV0NSKSbBTUJWAGDz7o16zu3VtdICJSthTUJdSwITz6qN+6a9iwoKsRkWSioC6F3r39bjB33QXTpwddjYgkCwV1KY0aBSef7EN75cqgqxGRZKCgLqW0NL8Mqhl066aNBkQk9hTUh6BRI78rzKJFfgsvEZFYUlAfoq5d4eab4amnYPz4oKsRkUSmoD4Mt93mN8a99lq/jZeISCwoqA9DaipMmgS1akHPnvDDD0FXJCKJSEF9mOrU8d0fn3zi168WEYk2BXUUnH02XHGFH1+9cGHQ1YhIolFQR8n998ORR8KVV8KePUFXIyKJREEdJTVr+r0WlyyBf/wj6GpEJJEoqKPovPP8TcURI2Dx4qCrEZFEUeKgNrMUM1tkZjNiWVC8Gz0a0tOhTx91gYhIdJSmRT0I+DRWhSSK9HR4/HE/rvrOO4OuRkQSQYmC2swygS7AU7EtJzGcf77fvfyOO9QFIiKHr6Qt6geAIcCBog4ws35mlmtmuRs3boxKcfFs9GioXRsuvxx27gy6GhGJZ8UGtZl1BTY45xYc7Djn3BPOuWznXHZGRkbUCoxXtWrBhAmwbJmfYu5c0BWJSLwqSYv6FOA8M1sNTAY6mtnEmFaVIDp1gr//Hf7v/2DMmKCrEZF4VWxQO+duds5lOucaApcC7zjnLot5ZQli2DA46ywYMMAviyoiUloaRx1jKSnwzDO+v7p7d9i+PeiKRCTelCqonXOznHNdY1VMosrI8KvsrVoFgwcHXY2IxBu1qMvIaafBTTf5vuoXXgi6GhGJJwrqMnTLLfC738HVV8PXXwddjYjECwV1GapQASZOhF27/BTz/fuDrkhE4oGCuow1a+Ynw7z5pu8KEREpTmrQBSSjq6/2a4Hccw8cfzxcdVXQFYlImKlFHZD77/fjq6+5BubMCboaEQkzBXVAUlNhyhRo3BguvhjWrw+6IhEJKwV1gGrW9EP1tm/XeiAiUjQFdcCOP97vXv7iizB5ctDViEgYKahD4E9/gjZt/Hog334bdDUiEjYK6hBISYFx4+D779UFIiK/pqAOifwukBdegH/+M+hqRCRMNI46RG68ERYuhKFDoWFDuOSSoCsSkTBQUIdIuXK+C2TtWujdG+rXh1NOCboqEQmauj5CJi3NjwA5+mi/Se7KlUFXJCJBU1CHUO3a8OqrftGmCy/0NxlFJHkpqEOqSRO/2cDHH0PfvhoJIpLMFNQh1rkz3Hmnn2p+771BVyMiQVFQh9zQoX6vxaFDfXeIiCQfBXXImcHYsZCV5YfrLVwYdEUiUtYU1HGgalWYMQPS06FLF1i9OuiKRKQsKajjRN268Nprfhuvc86B//436IpEpKwoqONI8+Z+jPWXX/qw3r496IpEpCwoqONMTo4fBZKb6yfE7NoVdEUiEmsK6jh0wQUwYQLMmuVHhOzZE3RFIhJLCuo41asXPPYYvPKK3xz3wIGgKxKRWNGiTHGsXz/YvBn++leoVw/uvjvoikQkFhTUce6mm2DdOr+Gdb16cMMNQVckItGmoI5zZjBqlN/FfPBgvzRq9+5BVyUi0aSgTgApKTBxog/r3r2hUSPIzg66KhGJFt1MTBBpaTBtGtSp44ftrVsXdEUiEi0K6gRy5JHw8suwbZsfwrdzZ9AViUg0KKgTTMuW8K9/wYIFcNFF2nRAJBEoqBPQuefCU0/Bm2/CmWdqXRCReFdsUJtZmpnNN7OPzGyZmd1aFoXJ4bnqKpg6FRYtgtNOg7y8oCsSkUNVkhb1D0BH59yJQBbQ2czaxrYsiYYLLoDXX/e7mp9xBmzaFHRFInIoig1q5+2IPC0feWgHvziRk+Onma9ZA+edpxuMIvGoRH3UZpZiZouBDcCbzrkPCjmmn5nlmlnuxo0bo12nHIbTToNnnoH334eePf3u5iISP0oU1M65/c65LCATaG1mLQo55gnnXLZzLjsjIyPadcphuvhiP4PxpZegf3+FtUg8KdXMROfcFjObCXQGlsamJImV66+HDRtgxAi/jvX48VC+fNBViUhxig1qM8sA9kZCuhJwFjAy5pVJTNx+O1Su7Ffc+/57mDzZz2oUkfAqSYu6LjDBzFLwXSXPOudmxLYsiaWbb4bq1WHAAL+l17RpUKNG0FWJSFGKDWrn3BLgpDKoRcrQddfBEUf48dbt2/uRIY0aBV2ViBRGMxOT2GWXwb//Dd98A23a+FEhIhI+Cuokl5PjA7paNTj9dHjySXAaJS8SKgpqoWlTmD8fOnTw23tdfbV2NxcJEwW1AJCe7vuphw2DsWP9JJn164OuSkRAQS0FpKTAbbfB9OmwfDm0awcrVgRdlYgoqOVXzj0XZs3y64K0bw/z5gVdkUhyU1BLobKzfUDXqgUdO/o9GUUkGApqKdJvfgPvvee7QC6/HP7yF60RIhIEBbUcVO3afqz1gAFwzz1+JqN2jBEpWwpqKVb58vDgg36M9cyZvltk8eKgqxJJHgpqKbGrr4a5c2HPHt8d8vTTQVckkhwU1FIqbdr4Hc7btoXevWHwYNi3L+iqRBKbglpKrU4dv8P5oEHwwAPQuTNs3hx0VSKJS0EthyQ11Yf02LG+O6R1a9/SFpHoU1DLYbnySpg9G374wfdb3303HDgQdFUiiUVBLYetbVtYssTvcj50KHTqBOvWBV2VSOJQUEtU1KoFzz0HTz3lJ8m0bAlTpgRdlUhiUFBL1JhB375+jPVxx8Gll0KvXrB1a9CVicQ3BbVEXZMm8O67fiW+Z5/1Nxo//TToqkTil4JaYiI11a9t/c47sGWLH3/94otBVyUSnxTUElOnneaH7TVrBhdeCP37w4YNQVclEl8U1BJzmZkwZ46fxTh2rO8a+ec//ZA+ESmeglrKRFoa3HcffPyxb2UPGQKtWsFHHwVdmUj4KailTDVrBjNm+O2+NmzwYT1ypNa5FjkYBbUE4txzYelSP0nmppt8K3v58qCrEgknBbUEpnZtP0lm4kS/iW5Wlp+CrtX4RH5OQS2BMvOTYpYtg7PP9lPQmzXz3SHffht0dSLhoKCWUDjqKJg2zT/q1/fdIZmZ8L//C9u3B12dSLAU1BIaZn6s9ezZvr/6j3/0w/mysuD994OuTiQ4CmoJpaZNYdQoP/76wAE49VQ/JV3915KMFNQSaqec4hd56tEDbrkFTj8dVq4MuiqRsqWgltA74gi/ke6//uVvOmZlwZgxGnstyUNBLXGjRw8/kzEry++I3qIFTJ6sHWUk8SmoJa40aACzZsHzz0NKig/vE07wMx2dC7o6kdgoNqjN7Ggzm2lmn5jZMjMbVBaFiRSlXDm4+GK//dekSbB3L5x/vu/Pnj076OpEoq8kLep9wI3OueZAW+A6M2se27JEileunN9FZtkyePJJ+OoryMmBjh0V2JJYig1q59w3zrmFkZ+3A58C9WNdmEhJpab6PuvPP4cHHvC7yeTk+MesWQEXJxIFpeqjNrOGwEnAB7EoRuRwVKoEgwb54XujRsFnn0GHDv6hFrbEsxIHtZlVBaYCNzjnthXyfj8zyzWz3I0bN0azRpFSqVQJBg6EL7/0gb18uW9dn3oqvPKKbjpK/ClRUJtZeXxIP+Ocm1bYMc65J5xz2c657IyMjGjWKHJI8gN75Up48EFYuxa6dvXD+x5+GDZvDrpCkZIpyagPA8YAnzrn7ot9SSLRVakSDBgAX3wBEyb4NUUGDIC6deGii/w0dZEwK0mL+hTgcqCjmS2OPM6JcV0iUVe+PPTu7aekL14M118P777rp6Wfdhq8/rq6RSScSjLq413nnDnnTnDOZUUer5ZFcSKxcuKJcO+9sGYNjB4Nq1f79bA7dvQ7z4iEiWYmSlKrVMm3rL/80vdb509Rv+EG+OaboKsT8RTUIkCFCn79688+g759fSu7fn0/2/G++3z/tkhQFNQiBdSuDY8/7ifN3Hor7NwJN94ITZrAscf61vfcuerLlrKloBYpRNOmMGwYLFrkh/c99JDfy3HMGPj9730f9xNPwPffB12pJAMFtUgxGjWC666DGTNg0ya/rki5ctC/P9Sr51vZy5YFXaUkMgW1SClUruzXFVm0yHeBnHuub1m3aOFnP06frvWxJfoU1CKHwMxPSZ84EfLyYORIWLXKL7farBncfTfMmwe7dwddqSQCczG4K5Kdne1yc3Oj/r0iYbZvH0yd6keJzJ/vXytf3g/3a90a2rTx/dsNGgRbp4STmS1wzmUX+p6CWiT61q+HDz6A99/3j9xc2LHD70ozdKi/UZmWFnSVEiYKapGA7d/vh/zddx+MG+e7Rx55xI/TrlAh6OokDA4W1OqjFikDKSn+huPYsX5NkZ07/XT1ypX9UMALL4QRI/x7WtVPfik16AJEks0f/uDXE3n5ZfjkE9/SXroUXnzRv1+unF886tZb4Zhjgq1VwkFBLRKAatWgZ8+fv7ZtGyxcCC+95LtFJk3y47f794fjjgumTgkHdX2IhET16n4s9v33+/0fe/TwPzdt6mdC3n67HwIoyUdBLRJCxxzjbzquWeM37K1eHW65BX7zG+jUCZ57DnbtCrpKKSsa9SESJ776yof3mDF+W7GKFf247E6d/G41+/b50SW1akHDhn7q+xFHBF21lJSG54kkkP374e234bXX4I03/M3IojRvDn36wGWX+TCX8FJQiySwb77xk2lSU/2IkU2b/I41X37pb0zOm+eHB3bu7EeTnHeeJtuEkYJaJImtWAHjx8PTT8O6dVCjBpxzDpx0kr9J2bIl1Knj1y+R4CioRYT9+2HmTL8T+6xZfjGpfDVq+NmSJ5/shw22b6/gLmsKahH5lc2bYckS+PhjWL7c93XPn+9nTTZuDJdc4lvdv/2tH8ddvnzQFSe2gwW1JryIJKn0dOjQwT/y7dgB06b5bpK77/5pbe1y5SAjA446yg8dvOgi6NYNqlYNpvZkoxa1iBRq1y7fv710qd/0d/16/1i2zG9PVqWKD+zsbN/iPu44v4RrSkrQlccntahFpNQqVfJraWdl/fx15+C99/wNyqlTfes7X/nyvtvkuOOgVSs47TS/DnelSmVaesJRi1pEDplzsHGjb3GvWOGnvn/+ue/vXr7cv1++vF858MQTfejnt7wbNPCtcvHUohaRmDCDI4/0j1NP/fl7330H//kPvPuu32Pytdd8K7yg9HTf533MMX6IYPXqfsGqhg399zVqpNEnoKAWkRipWRO6dvWPfN9+6yfirFnjF5hau9ZPjf/iC78TzvbtftRJvvr14ayz/ESd00/3NzWTkYJaRMpMnTr+0b590cfs2+e7TebOhTlz/CiU8eN9K7tHDzjzTGjXLrn6vdVHLSKhtmuX31Rh3Dh45x0/cadCBT9BZ9cuv47399/7oYQHDvj3Tj4Z2rb1mwrXrQu1a/tJPbt3++N37vQzMitWDPrsfqIJLyKSELZt833eM2f6VnfVqr5fu3Lln9Y62bEDPvzQ94vv21f0d2VkwFVXQb9+fqRK0BTUIpJ0du3yY8A3bvQLVW3Z4gO9enU/GmXyZJg+3bfCTz4ZzjjDP048MZi1TxTUIiKFyMvza5+88Ya/mbl3r3+9YkU/EqVWLb/SYKVKvvVeo4Z/NGjgu1ZOPPHnU+v37fMt+0OhoBYRKcb33/slYT/7zI9KWbMGtm71LfNdu3yXypYtftjh7t3+M2lpUK+eH62yfbsP9nXrDu33axy1iEgxqlTxQwHPOuvgxznnhxW+/76foblhg+9OqV7d93vHgoJaRKQUzH6apHPJJWXzO4sdPm5mY81sg5ktLYuCRETk50oyz2c80DnGdYiISBGKDWrn3Bzgv2VQi4iIFCJqM+fNrJ+Z5ZpZ7saNG6P1tSIiSS9qQe2ce8I5l+2cy86I1a1PEZEklKRrUYmIxA8FtYhIyJVkeN4k4D2gqZnlmVnf2JclIiL5YjKF3Mw2AmsO8eO1gU1RLCceJOM5Q3KedzKeMyTneZf2nBs45wq9wReToD4cZpZb1Hz3RJWM5wzJed7JeM6QnOcdzXNWH7WISMgpqEVEQi6MQf1E0AUEIBnPGZLzvJPxnCE5zztq5xy6PmoREfm5MLaoRUSkAAW1iEjIhSaozayzma0wsy/M7Kag64kVMzvazGaa2SdmtszMBkVer2Vmb5rZ55F/1gy61mgzsxQzW2RmMyLPG5nZB5FrPsXMKgRdY7SZWQ0ze97MlpvZp2bWLtGvtZkNjvxtLzWzSWaWlojXurC1+ou6tuaNjpz/EjM7uTS/KxRBbWYpwMPA2UBzoIeZNQ+2qpjZB9zonGsOtAWui5zrTcDbzrkmwNuR54lmEPBpgecjgfudc8cC3wGJOOt1FPC6c64ZcCL+/BP2WptZfWAgkO2cawGkAJeSmNd6PL9eq7+oa3s20CTy6Ac8Wqrf5JwL/AG0A94o8Pxm4Oag6yqjc38JOAtYAdSNvFYXWBF0bWOnDa8AAAJZSURBVFE+z8zIH25HYAZg+FlbqYX9DSTCAzgCWEXkpn2B1xP2WgP1gbVALfxWfzOAPyTqtQYaAkuLu7bA40CPwo4rySMULWp+urj58iKvJTQzawicBHwA1HHOfRN5az1QJ6CyYuUBYAhwIPI8HdjinNsXeZ6I17wRsBEYF+nyecrMqpDA19o5tw64B/gK+AbYCiwg8a91vqKu7WFlXFiCOumYWVVgKnCDc25bwfec/1duwoybNLOuwAbn3IKgayljqcDJwKPOuZOA7/lFN0cCXuuawPn4f0nVA6qQpFv5RfPahiWo1wFHF3ieGXktIZlZeXxIP+OcmxZ5+Vszqxt5vy6wIaj6YuAU4DwzWw1Mxnd/jAJqmFlq5JhEvOZ5QJ5z7oPI8+fxwZ3I1/pMYJVzbqNzbi8wDX/9E/1a5yvq2h5WxoUlqD8EmkTuDFfA33yYHnBNMWFmBowBPnXO3VfgrenAFZGfr8D3XScE59zNzrlM51xD/LV9xznXC5gJdIscllDnDOCcWw+sNbOmkZfOAD4hga81vsujrZlVjvyt559zQl/rAoq6ttOB3pHRH22BrQW6SIoXdGd8gc71c4DPgC+BvwVdTwzP81T8fw4tARZHHufg+2zfBj4H3gJqBV1rjM4/B5gR+bkxMB/4AngOqBh0fTE43ywgN3K9XwRqJvq1Bm4FlgNLgaeBiol4rYFJ+H74vfj/eupb1LXF3zx/OJJvH+NHxZT4d2kKuYhIyIWl60NERIqgoBYRCTkFtYhIyCmoRURCTkEtIhJyCmoRkZBTUIuIhNz/B2HxMekeAPkmAAAAAElFTkSuQmCC\n", 605 | "text/plain": [ 606 | "
" 607 | ] 608 | }, 609 | "metadata": { 610 | "tags": [], 611 | "needs_background": "light" 612 | } 613 | } 614 | ] 615 | }, 616 | { 617 | "cell_type": "code", 618 | "metadata": { 619 | "colab_type": "code", 620 | "id": "6Vc6PHgxa6Hm", 621 | "outputId": "1145f9f0-eb82-44b1-8349-6a1bbaa43757", 622 | "colab": { 623 | "base_uri": "https://localhost:8080/", 624 | "height": 54 625 | } 626 | }, 627 | "source": [ 628 | "seed_text = \"Help me Obi Wan Kenobi, you're my only hope\"\n", 629 | "next_words = 100\n", 630 | " \n", 631 | "for _ in range(next_words):\n", 632 | "\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n", 633 | "\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n", 634 | "\tpredicted = model.predict_classes(token_list, verbose=0)\n", 635 | "\toutput_word = \"\"\n", 636 | "\tfor word, index in tokenizer.word_index.items():\n", 637 | "\t\tif index == predicted:\n", 638 | "\t\t\toutput_word = word\n", 639 | "\t\t\tbreak\n", 640 | "\tseed_text += \" \" + output_word\n", 641 | "print(seed_text)" 642 | ], 643 | "execution_count": 0, 644 | "outputs": [ 645 | { 646 | "output_type": "stream", 647 | "text": [ 648 | "Help me Obi Wan Kenobi, you're my only hope thyself away desired be cross afford twain date of men ' doth groan ' be free free live away away your days here brought of one one more must grow of free ride hits green thee burn pleasure pleasure me of days much brought of pleasure pleasure of pleasure green die burn'd hits dwells beard ward fall by days another rolling ' doth ride ride ride ride ' her view free grew to ride cherish plight grow thus days did tell away away night exchanged mad so meant free his senses can alive pleasure ride ride light decease translate must\n" 649 | ], 650 | "name": "stdout" 651 | } 652 | ] 653 | }, 654 | { 655 | "cell_type": "code", 656 | "metadata": { 657 | "id": "oliB864cIlGA", 658 | "colab_type": "code", 659 | "outputId": "2102ede6-8362-4182-9f10-7d227c833c30", 660 | "colab": { 661 | "base_uri": "https://localhost:8080/", 662 | "height": 54 663 | } 664 | }, 665 | "source": [ 666 | "\"Help me Obi Wan Kenobi, you're my only hope of every pen shows know confounds thine thine eye aside bright than gently cheeks express'd place in mind keep something new hate of die much date cheeks day was me away the rhyme ' may be bevel disdain erred light light stand bright ' write me flowers so look me doth grow ill old old glory still say thee but now come away a gainer his part torn torn away away ill young new date lies be express'd rare rare ill ill near feast so 'will ' hast me well seen live up memory prove keep invention spent kind check\"" 667 | ], 668 | "execution_count": 0, 669 | "outputs": [ 670 | { 671 | "output_type": "execute_result", 672 | "data": { 673 | "text/plain": [ 674 | "\"Help me Obi Wan Kenobi, you're my only hope of every pen shows know confounds thine thine eye aside bright than gently cheeks express'd place in mind keep something new hate of die much date cheeks day was me away the rhyme ' may be bevel disdain erred light light stand bright ' write me flowers so look me doth grow ill old old glory still say thee but now come away a gainer his part torn torn away away ill young new date lies be express'd rare rare ill ill near feast so 'will ' hast me well seen live up memory prove keep invention spent kind check\"" 675 | ] 676 | }, 677 | "metadata": { 678 | "tags": [] 679 | }, 680 | "execution_count": 52 681 | } 682 | ] 683 | }, 684 | { 685 | "cell_type": "code", 686 | "metadata": { 687 | "id": "zYWmvGg9OHmp", 688 | "colab_type": "code", 689 | "colab": {} 690 | }, 691 | "source": [ 692 | "" 693 | ], 694 | "execution_count": 0, 695 | "outputs": [] 696 | } 697 | ] 698 | } -------------------------------------------------------------------------------- /C4W2_1_Time_series_training_data_labels_preprocessing.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "C4W2-1-Time series-training data-labels-preprocessing", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "view-in-github", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "\"Open" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "metadata": { 31 | "id": "zX4Kg8DUTKWO", 32 | "colab_type": "code", 33 | "colab": {} 34 | }, 35 | "source": [ 36 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 37 | "# you may not use this file except in compliance with the License.\n", 38 | "# You may obtain a copy of the License at\n", 39 | "#\n", 40 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 41 | "#\n", 42 | "# Unless required by applicable law or agreed to in writing, software\n", 43 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 44 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 45 | "# See the License for the specific language governing permissions and\n", 46 | "# limitations under the License." 47 | ], 48 | "execution_count": 0, 49 | "outputs": [] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "metadata": { 54 | "colab_type": "code", 55 | "id": "s6eq-RBcQ_Zr", 56 | "colab": {} 57 | }, 58 | "source": [ 59 | "try:\n", 60 | " # %tensorflow_version only exists in Colab.\n", 61 | " %tensorflow_version 2.x\n", 62 | "except Exception:\n", 63 | " pass" 64 | ], 65 | "execution_count": 0, 66 | "outputs": [] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "metadata": { 71 | "colab_type": "code", 72 | "id": "BOjujz601HcS", 73 | "outputId": "dbaa1c2b-21d9-4405-8d62-2811d6c9f99d", 74 | "colab": { 75 | "base_uri": "https://localhost:8080/", 76 | "height": 34 77 | } 78 | }, 79 | "source": [ 80 | "import tensorflow as tf\n", 81 | "import numpy as np\n", 82 | "import matplotlib.pyplot as plt\n", 83 | "print(tf.__version__)" 84 | ], 85 | "execution_count": 0, 86 | "outputs": [ 87 | { 88 | "output_type": "stream", 89 | "text": [ 90 | "2.2.0-rc3\n" 91 | ], 92 | "name": "stdout" 93 | } 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "metadata": { 99 | "colab_type": "code", 100 | "id": "asEdslR_05O_", 101 | "outputId": "e572945a-00a3-417e-db63-e522aaf99e58", 102 | "colab": { 103 | "base_uri": "https://localhost:8080/", 104 | "height": 187 105 | } 106 | }, 107 | "source": [ 108 | "dataset = tf.data.Dataset.range(10)\n", 109 | "for val in dataset:\n", 110 | " print(val.numpy())" 111 | ], 112 | "execution_count": 0, 113 | "outputs": [ 114 | { 115 | "output_type": "stream", 116 | "text": [ 117 | "0\n", 118 | "1\n", 119 | "2\n", 120 | "3\n", 121 | "4\n", 122 | "5\n", 123 | "6\n", 124 | "7\n", 125 | "8\n", 126 | "9\n" 127 | ], 128 | "name": "stdout" 129 | } 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "metadata": { 135 | "colab_type": "code", 136 | "id": "Lrv_ghSt1lgQ", 137 | "outputId": "a5ec3ba7-a9d4-4d5e-a55e-4a36162feabf", 138 | "colab": { 139 | "base_uri": "https://localhost:8080/", 140 | "height": 187 141 | } 142 | }, 143 | "source": [ 144 | "dataset = tf.data.Dataset.range(10)\n", 145 | "dataset = dataset.window(5, shift=1)\n", 146 | "for window_dataset in dataset:\n", 147 | " for val in window_dataset:\n", 148 | " print(val.numpy(), end=\" \")\n", 149 | " print()" 150 | ], 151 | "execution_count": 0, 152 | "outputs": [ 153 | { 154 | "output_type": "stream", 155 | "text": [ 156 | "0 1 2 3 4 \n", 157 | "1 2 3 4 5 \n", 158 | "2 3 4 5 6 \n", 159 | "3 4 5 6 7 \n", 160 | "4 5 6 7 8 \n", 161 | "5 6 7 8 9 \n", 162 | "6 7 8 9 \n", 163 | "7 8 9 \n", 164 | "8 9 \n", 165 | "9 \n" 166 | ], 167 | "name": "stdout" 168 | } 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "metadata": { 174 | "colab_type": "code", 175 | "id": "QLEq6MG-2DN2", 176 | "outputId": "d21f364c-d78d-4e66-e3fd-9d697ae5c854", 177 | "colab": { 178 | "base_uri": "https://localhost:8080/", 179 | "height": 119 180 | } 181 | }, 182 | "source": [ 183 | "dataset = tf.data.Dataset.range(10)\n", 184 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 185 | "for window_dataset in dataset:\n", 186 | " for val in window_dataset:\n", 187 | " print(val.numpy(), end=\" \")\n", 188 | " print()" 189 | ], 190 | "execution_count": 0, 191 | "outputs": [ 192 | { 193 | "output_type": "stream", 194 | "text": [ 195 | "0 1 2 3 4 \n", 196 | "1 2 3 4 5 \n", 197 | "2 3 4 5 6 \n", 198 | "3 4 5 6 7 \n", 199 | "4 5 6 7 8 \n", 200 | "5 6 7 8 9 \n" 201 | ], 202 | "name": "stdout" 203 | } 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "metadata": { 209 | "colab_type": "code", 210 | "id": "PJ9CAHlJ2ODe", 211 | "outputId": "cd9cc40d-0179-4797-b85f-d87841c06052", 212 | "colab": { 213 | "base_uri": "https://localhost:8080/", 214 | "height": 119 215 | } 216 | }, 217 | "source": [ 218 | "dataset = tf.data.Dataset.range(10)\n", 219 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 220 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 221 | "for window in dataset:\n", 222 | " print(window.numpy())\n" 223 | ], 224 | "execution_count": 0, 225 | "outputs": [ 226 | { 227 | "output_type": "stream", 228 | "text": [ 229 | "[0 1 2 3 4]\n", 230 | "[1 2 3 4 5]\n", 231 | "[2 3 4 5 6]\n", 232 | "[3 4 5 6 7]\n", 233 | "[4 5 6 7 8]\n", 234 | "[5 6 7 8 9]\n" 235 | ], 236 | "name": "stdout" 237 | } 238 | ] 239 | }, 240 | { 241 | "cell_type": "code", 242 | "metadata": { 243 | "colab_type": "code", 244 | "id": "DryEZ2Mz2nNV", 245 | "outputId": "e4c56ab1-739a-42e3-cf19-a0fd483d0ac6", 246 | "colab": { 247 | "base_uri": "https://localhost:8080/", 248 | "height": 119 249 | } 250 | }, 251 | "source": [ 252 | "dataset = tf.data.Dataset.range(10)\n", 253 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 254 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 255 | "dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n", 256 | "for x,y in dataset:\n", 257 | " print(x.numpy(), y.numpy())" 258 | ], 259 | "execution_count": 0, 260 | "outputs": [ 261 | { 262 | "output_type": "stream", 263 | "text": [ 264 | "[0 1 2 3] [4]\n", 265 | "[1 2 3 4] [5]\n", 266 | "[2 3 4 5] [6]\n", 267 | "[3 4 5 6] [7]\n", 268 | "[4 5 6 7] [8]\n", 269 | "[5 6 7 8] [9]\n" 270 | ], 271 | "name": "stdout" 272 | } 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "metadata": { 278 | "colab_type": "code", 279 | "id": "1tl-0BOKkEtk", 280 | "outputId": "50e7f857-bcc7-433f-9568-31d7b62060f8", 281 | "colab": { 282 | "base_uri": "https://localhost:8080/", 283 | "height": 119 284 | } 285 | }, 286 | "source": [ 287 | "dataset = tf.data.Dataset.range(10)\n", 288 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 289 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 290 | "dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n", 291 | "dataset = dataset.shuffle(buffer_size=10)\n", 292 | "for x,y in dataset:\n", 293 | " print(x.numpy(), y.numpy())\n" 294 | ], 295 | "execution_count": 0, 296 | "outputs": [ 297 | { 298 | "output_type": "stream", 299 | "text": [ 300 | "[0 1 2 3] [4]\n", 301 | "[2 3 4 5] [6]\n", 302 | "[5 6 7 8] [9]\n", 303 | "[1 2 3 4] [5]\n", 304 | "[4 5 6 7] [8]\n", 305 | "[3 4 5 6] [7]\n" 306 | ], 307 | "name": "stdout" 308 | } 309 | ] 310 | }, 311 | { 312 | "cell_type": "code", 313 | "metadata": { 314 | "colab_type": "code", 315 | "id": "Wa0PNwxMGapy", 316 | "outputId": "0d80dac9-3abe-4592-d5ed-b21013130a16", 317 | "colab": { 318 | "base_uri": "https://localhost:8080/", 319 | "height": 221 320 | } 321 | }, 322 | "source": [ 323 | "dataset = tf.data.Dataset.range(10)\n", 324 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 325 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 326 | "dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n", 327 | "dataset = dataset.shuffle(buffer_size=10)\n", 328 | "dataset = dataset.batch(2).prefetch(1)\n", 329 | "for x,y in dataset:\n", 330 | " print(\"x = \", x.numpy())\n", 331 | " print(\"y = \", y.numpy())\n" 332 | ], 333 | "execution_count": 0, 334 | "outputs": [ 335 | { 336 | "output_type": "stream", 337 | "text": [ 338 | "x = [[0 1 2 3]\n", 339 | " [5 6 7 8]]\n", 340 | "y = [[4]\n", 341 | " [9]]\n", 342 | "x = [[1 2 3 4]\n", 343 | " [2 3 4 5]]\n", 344 | "y = [[5]\n", 345 | " [6]]\n", 346 | "x = [[3 4 5 6]\n", 347 | " [4 5 6 7]]\n", 348 | "y = [[7]\n", 349 | " [8]]\n" 350 | ], 351 | "name": "stdout" 352 | } 353 | ] 354 | }, 355 | { 356 | "cell_type": "code", 357 | "metadata": { 358 | "id": "zKv6P-ChgIOm", 359 | "colab_type": "code", 360 | "colab": {} 361 | }, 362 | "source": [ 363 | "" 364 | ], 365 | "execution_count": 0, 366 | "outputs": [] 367 | } 368 | ] 369 | } -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Chapter-1-Presentation-1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Chapter-1-Presentation-1.pdf -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-2.pdf -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-3.pdf -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-4.pdf -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-5.pdf -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-6.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-6.pdf -------------------------------------------------------------------------------- /Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-7.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/Presentations/Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-7.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # TensorFlow-in-Practice 2 | 3 | Join our Deep Learning Adventures community and become an expert in Deep Learning, TensorFlow, Computer Vision, Convolutional Neural Networks, Kaggle Challenges, Data Augmentation and Dropouts Transfer Learning, Multiclass Classifications and Overfitting and Natural Language Processing NLP as well as Time Series Forecasting 😀 Join us and earn your TensorFlow Developer Certificate! All while having fun learning and participating in our Deep Learning Trivia games 🎉 4 | 5 | Meetup page 😀 6 | https://www.meetup.com/Deep-Learning-Adventures 7 | 8 | YouTube recordings of all Meetups 🎉 9 | http://bit.ly/deep-learning-tf 10 | 11 | Join us on Slack: 12 | https://join.slack.com/t/deeplearninga-nmk8930/shared_invite/zt-d52h9mm9-h~Q0ZXw5PXsTDzPIlNIvog 13 | 14 | TensorFlow in Practice Specialization: 15 | https://www.coursera.org/specializations/tensorflow-in-practice 16 | 17 | Visits Laurence's GitHub repository for this specialization: 18 | https://github.com/lmoroney/dlaicourse/ 19 | 20 | Differentiate yourself with the TensorFlow Developer Certificate: 21 | https://www.tensorflow.org/certificate 22 | 23 | View our interactive presentations here. If you prefer static pdf presentations, please look at our Presentations folder above. Special attribution goes to Coursera and deeplearning.ai for allowing us to borrow some of their great content. 24 | 25 | * Deep-Learning-Adventures-Chapter-1-Presentation-1 26 | https://docs.google.com/presentation/d/18nAqidaH_4CzZyh_1e23gR88aIgFxCuIT7YmMgmnjwg/edit?usp=sharing 27 | 28 | * Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-2 29 | https://docs.google.com/presentation/d/1GEvlIo8g_OFWWq7S3Ob82wRh5NroXpVSivRvVjFpm4Q/edit?usp=sharing 30 | 31 | * Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-3 32 | https://docs.google.com/presentation/d/1vKOwYrABaGCvHMM8wnK0vb6NNzfHLQREYQI2UXiFzJ0/edit?usp=sharing 33 | 34 | * Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-4 35 | https://docs.google.com/presentation/d/1DAXyuNew5YDEzSakYjvbdIvCf_eNbZ6s69iyDBvHT2w/edit?usp=sharing 36 | 37 | * Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-5 38 | https://docs.google.com/presentation/d/1gUJsRmuc_7beCoS3x2BiEWjJflcenyIQiqZMcyMLL-M/edit?usp=sharing 39 | 40 | * Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-6 41 | https://docs.google.com/presentation/d/1KdKcq97grYYcTmkoEvwSOElK1TI_Son69CRae3wPk14/edit?usp=sharing 42 | 43 | * Deep-Learning-Adventures-Tensorflow-In-Practice-Presentation-7 44 | https://docs.google.com/presentation/d/1UiIgy06YmYx8z1o23Z7Qb9ZyIFWismF52OSBpUq8niA/edit?usp=sharing 45 | 46 | Practice your skills while having fun with our Deep Learning Trivia games 🎉 47 | * Deep Learning Trivia Game 1, covering basic TensorFlow syntax, Keras, dense network, optimizer and more: 48 | https://quizizz.com/join/quiz/5e87bdbc07fa7f001b120404/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336 49 | 50 | * Deep Learning Trivia Game 2, covering computer vision, Fashion-MNIST, sparse categorical and binary crossentropy loss, convolutional neural networks, max pooling, ImageDataGenerator and more: 51 | https://quizizz.com/join/quiz/5e90e8e631fb32001f63510b/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336 52 | 53 | * Deep Learning Trivia Game 3, covering convolutional neural networks in TensorFlow, larger datasets, techniques to avoid overfitting, transfer learning, multiclass classification and more: 54 | https://quizizz.com/join/quiz/5eae10a807df73001b0c59a4/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336 55 | 56 | * Deep Learning Trivia Game 4, natural language processing in TensorFlow, sentiment in text, word embeddings, sequence models, RNNs, LSTMs, sequence models applications and more: 57 | https://quizizz.com/join/quiz/5eb71e0e09cdea001c916ac6/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336 58 | 59 | * Deep Learning Trivia Game 5, covering sequences, time series and prediction using TensorFlow, statistical methods, deep neural networks for time series, recurrent neural networks for time series, real-world time series data and more: 60 | https://quizizz.com/join/quiz/5eefd13d3db4b8001bb6d72b/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336 61 | 62 | 63 | ![alt text](images/DLA-0.png) 64 | ![alt text](images/DLA-1.png) 65 | ![alt text](images/DLA-0.5.png) 66 | ![alt text](images/DLA-2.png) 67 | ![alt text](images/DLA-3.png) 68 | ![alt text](images/DLA-4.png) 69 | ![alt text](images/DLA-5.png) 70 | ![alt text](images/DLA-6.png) 71 | ![alt text](images/DLA-7.png) 72 | ![alt text](images/DLA-8.png) 73 | ![alt text](images/DLA-9.png) 74 | ![alt text](images/DLA-10.png) 75 | ![alt text](images/DLA-11.png) 76 | ![alt text](images/DLA.png) 77 | ![alt text](images/DLA-12.png) 78 | 79 | ![alt text](images/TensorFlow-in-Practice-1.png) 80 | ![alt text](images/TensorFlow-in-Practice-2.png) 81 | ![alt text](images/TensorFlow-in-Practice-3.png) 82 | -------------------------------------------------------------------------------- /images/DLA-0.5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-0.5.png -------------------------------------------------------------------------------- /images/DLA-0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-0.png -------------------------------------------------------------------------------- /images/DLA-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-1.png -------------------------------------------------------------------------------- /images/DLA-10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-10.png -------------------------------------------------------------------------------- /images/DLA-11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-11.png -------------------------------------------------------------------------------- /images/DLA-12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-12.png -------------------------------------------------------------------------------- /images/DLA-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-2.png -------------------------------------------------------------------------------- /images/DLA-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-3.png -------------------------------------------------------------------------------- /images/DLA-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-4.png -------------------------------------------------------------------------------- /images/DLA-5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-5.png -------------------------------------------------------------------------------- /images/DLA-6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-6.png -------------------------------------------------------------------------------- /images/DLA-7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-7.png -------------------------------------------------------------------------------- /images/DLA-8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-8.png -------------------------------------------------------------------------------- /images/DLA-9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA-9.png -------------------------------------------------------------------------------- /images/DLA.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/DLA.png -------------------------------------------------------------------------------- /images/TensorFlow-in-Practice-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/TensorFlow-in-Practice-1.png -------------------------------------------------------------------------------- /images/TensorFlow-in-Practice-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/TensorFlow-in-Practice-2.png -------------------------------------------------------------------------------- /images/TensorFlow-in-Practice-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/georgezoto/TensorFlow-in-Practice/dcf84e7f8aed0fe572c53655f7efb0eaac05dc46/images/TensorFlow-in-Practice-3.png --------------------------------------------------------------------------------