├── redes.png ├── LICENSE ├── README.md ├── codes ├── feedforward_multclass.py ├── cnn.py ├── rbm.py ├── feedforward_binary.py ├── lstm.py ├── gan.py └── autoencoder.py ├── libraries.txt └── deepLearning_LSTM.ipynb /redes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hfarruda/deeplearningtutorial/HEAD/redes.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Deep learning tutorial (c) by Henrique Ferraz de Arruda, Alexandre Benatti, César Henrique Comin, and Luciano da Fontoura Costa 2 | 3 | Deep learning tutorial is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. 4 | 5 | You should have received a copy of the license along with this work. 6 | If not, see . 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning Tutorial 2 | 3 | This tutorial is part of the didactic text: [Learning Deep Learning](https://www.scielo.br/j/rbef/a/hMZfS8hRwMvVktkbCZtjJff/?format=html), authored by Henrique Ferraz de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. 4 | 5 | The purpose of this tutorial is to provide simple didactic examples of deep learning architectures and problem solution. The codes included here are based on toy datasets, and restricted to parameters allowing short processing time. So, these codes are not suitable for other data and/or applications, which will require modifications in the structure and parameters. These codes have absolutely no warranty. 6 | 7 | For all the codes presented here, we use [Keras](https://keras.io/) as the deep learning library. Keras is a useful and straightforward framework, which can be employed for simple and complex tasks. Keras is written in the Python language, providing self-explanatory codes, with the additional advantage of being executed under [TensorFlow](https://www.tensorflow.org/) backend. We also employ the [Scikit-learn](https://scikit-learn.org/), which is devoted to machine learning. 8 | 9 | ![](./redes.png) 10 | 11 | More details are available at [Learning Deep Learning](https://www.scielo.br/j/rbef/a/hMZfS8hRwMvVktkbCZtjJff/?format=html). 12 | 13 | 14 | ## Feedforward networks 15 | 16 | ### Binary Classification 17 | This is the first example of deep learning implementation, in which we address binary classification of wine data. In this example, we consider one feedforward network with 5 hidden layers and with 30 neurons in each layer. The provided networks were built only for a didactic purpose and are not appropriate for real applications. 18 | 19 | ### Multiclass Classification 20 | In this example, we illustrate a multiclass classification through a wine dataset, in which there are three classes, which were defined according to their regions. We employed the same dataset presented above, but here we considered the three classes. To do so, we use the *softmax* activation function. 21 | 22 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_feedforward.ipynb) 23 | 24 | 25 | ## Convolutional Neural Network (CNN) 26 | This tutorial is the second example of deep learning implementation, in which we exemplify a classification task. More specifically, we considered ten classes of colored pictures. 27 | 28 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_CNN.ipynb) 29 | 30 | 31 | ## Long Short-Term Memory (LSTM) 32 | 33 | This is the third example of deep learning implementation. Here we use a LSTM network to predict the Bitcoin prices along time by using the input as a temporal series. 34 | 35 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_LSTM.ipynb) 36 | 37 | 38 | ## Restricted Boltzmann Machine (RBM) 39 | 40 | This is the fourth example of deep learning implementation. Here we use a RMB network to provide a recommendation system of musical instruments. 41 | 42 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_RBM.ipynb) 43 | 44 | 45 | ## Autoencoders 46 | This example uses the Autoencoder model to illustrate a possible application. Here we show how to use the resulting codes to reduce the dimentionality. We also project our data by using a Principal Component Analysis(PCA). 47 | 48 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_autoencoder.ipynb) 49 | 50 | 51 | ## Generative Adversarial Networks (GAN) 52 | This example was elaborated to create a network that can generate handwritten characters automatically. 53 | 54 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_GAN.ipynb) 55 | 56 | 57 | ## Libraries 58 | All of these codes were developed and executed with the environment described in "libraries.txt". 59 | 60 | ## Citation Request 61 | If you publish a paper related to this material, please cite: 62 | 63 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning deep learning." Revista Brasileira de Ensino de Física 44, 2022. 64 | 65 | 66 | ## Acknowledgments 67 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). H. F. de Arruda also thanks Soremartec S.A. and Soremartec Italia, Ferrero Group, for partial financial support (from 1st July 2021). His funders had no role in study design, data collection, and analysis, decision to publish, or manuscript preparation. Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and FAPESP (proc. 15/22308-2) for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 2018/09125-4 and 2021/12354-8) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 15/22308-2. 68 | -------------------------------------------------------------------------------- /codes/feedforward_multclass.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_feedforward.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_feedforward.ipynb 8 | 9 | #Feedforward networks 10 | 11 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. These codes have absolutely no warranty. 12 | 13 | If you publish a paper related on this material, please cite: 14 | 15 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 16 | 17 | ##Multiclass Classification 18 | In this example, we illustrate a multiclass classification through a wine dataset, in which there are three classes, which were defined according to their regions. We employed the same dataset presented above, but here we considered the three classes. To do so, we use the *softmax* activation function. 19 | 20 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 21 | """ 22 | 23 | import numpy as np 24 | import keras 25 | from keras.utils import np_utils 26 | from keras.models import Sequential 27 | from keras.layers import Dense, Dropout 28 | from sklearn.datasets import load_wine 29 | from sklearn.model_selection import train_test_split 30 | from sklearn.metrics import accuracy_score 31 | from sklearn.preprocessing import LabelEncoder 32 | from sklearn.metrics import confusion_matrix 33 | 34 | """If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*).""" 35 | 36 | import tensorflow as tf 37 | from keras import backend as K 38 | 39 | print(K.tensorflow_backend._get_available_gpus()) 40 | 41 | number_of_cpu_cores = 8 42 | config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) 43 | session = tf.Session(config=config) 44 | keras.backend.set_session(session) 45 | 46 | """(*) In this example the dataset used is Wine. It is available at Sklearn library on [sklearn-datasets-wine](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_wine.html). For more information [wine-UCI](https://archive.ics.uci.edu/ml/datasets/Wine). 47 | 48 | These data show the results of a chemical analysis of wines grown in Italy, derived from three different cultivars in the same region, and can be loaded as follows. 49 | """ 50 | 51 | wine = load_wine() 52 | data = wine['data'] 53 | target = wine['target'] 54 | target_names = wine['target_names'] 55 | 56 | label_encoder = LabelEncoder() 57 | target = label_encoder.fit_transform(target) 58 | target_one_hot_encoding = np_utils.to_categorical(target) 59 | 60 | #Here, we divide our dataset into training and test sets. 61 | test_size = 0.25 #fraction 62 | training_data,test_data,training_target,test_target = train_test_split(data, 63 | target_one_hot_encoding, test_size=test_size) 64 | 65 | """In the following, we configure the neuronal network. It is not necessary to include bias because this parameter is set as true by default.""" 66 | 67 | #Set of parameters 68 | input_dim = data.shape[1] 69 | kernel_initializer = 'random_uniform' 70 | bias_initializer='zeros' 71 | activation_function_hidden = 'relu' 72 | activation_function_output = 'softmax' 73 | optimizer = 'adam' 74 | loss = 'categorical_crossentropy' 75 | metrics = ['categorical_accuracy'] 76 | number_of_layers = 5 77 | number_of_units_hidden = 30 78 | number_of_units_output = len(set(target_names)) 79 | dropout_percentage = 0.25 80 | 81 | 82 | #Creating model 83 | ff_model = Sequential() 84 | ff_model.add(Dense(units = number_of_units_hidden, 85 | activation = activation_function_hidden, 86 | kernel_initializer = kernel_initializer, 87 | input_dim = input_dim)) 88 | 89 | for i in range(number_of_layers-1): 90 | #Inserting a dense hidden layer 91 | ff_model.add(Dense(units = number_of_units_hidden, 92 | activation = activation_function_hidden, 93 | kernel_initializer = kernel_initializer, 94 | input_dim = number_of_units_hidden)) 95 | #Inserting dropout 96 | ff_model.add(Dropout(dropout_percentage)) 97 | 98 | ff_model.add(Dense(units = number_of_units_output, 99 | activation = activation_function_output)) 100 | ff_model.compile(optimizer = optimizer, loss = loss, metrics = metrics) 101 | ff_model.summary() 102 | 103 | """The training step is executed as follows.""" 104 | 105 | batch_size = 10 106 | epochs = 250 107 | ff_model.fit(training_data,training_target, batch_size = batch_size, 108 | epochs = epochs) 109 | 110 | """Because there are three classes, we show the classification results through a confusion matrix.""" 111 | 112 | predictions = ff_model.predict(test_data) 113 | 114 | found_target = predictions.argmax(axis=1) 115 | categorical_test_target = test_target.argmax(axis=1) 116 | 117 | accuracy = accuracy_score(categorical_test_target, found_target) 118 | print("Accuracy =", accuracy) 119 | 120 | print("Confusion matrix:") 121 | matrix = confusion_matrix(found_target,categorical_test_target) 122 | print(matrix) 123 | 124 | """ 125 | ## License 126 | 127 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 128 | 129 | ## Acknowledgments 130 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 131 | """ 132 | -------------------------------------------------------------------------------- /codes/cnn.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_CNN.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_CNN.ipynb 8 | 9 | # Convolutional Neural Network (CNN) 10 | 11 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. This code has absolutely no warranty. 12 | 13 | If you publish a paper related on this material, please cite: 14 | 15 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 16 | 17 | This tutorial is the second example of deep learning implementation, in which we exemplify a classification task. More specifically, we considered ten classes of color pictures. 18 | 19 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 20 | """ 21 | 22 | import keras 23 | from keras.utils import np_utils 24 | from keras.models import Sequential 25 | from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout 26 | from keras.layers.normalization import BatchNormalization 27 | from keras.preprocessing.image import ImageDataGenerator 28 | from keras.preprocessing import image 29 | from keras.datasets import cifar10 30 | from keras.utils.vis_utils import plot_model 31 | from sklearn.metrics import accuracy_score 32 | import numpy as np 33 | import matplotlib.pyplot as plt 34 | from sklearn.metrics import classification_report, confusion_matrix 35 | 36 | """If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*).""" 37 | 38 | import tensorflow as tf 39 | from keras import backend as K 40 | 41 | print(K.tensorflow_backend._get_available_gpus()) 42 | 43 | number_of_cpu_cores = 8 44 | config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) 45 | session = tf.Session(config=config) 46 | keras.backend.set_session(session) 47 | 48 | """(*) In this example, we used the CIFAR10, which is consists of a colored dataset of images. It is available in Keras library, available on [keras-datasets](https://keras.io/datasets/). 49 | This dataset is organized into two parts, where the first is called x_train/x_test and comprises RGB images with dimensions of 32x32x3 . The second represents the targets, and the variables are called y_train/y_test, which are represented by arrays of category tags from 0 to 9. 50 | 51 | The following command is used to load the data set. 52 | """ 53 | 54 | (train_data, train_target), (test_data, test_target) = cifar10.load_data() 55 | 56 | train_target_one_hot_encoding = np_utils.to_categorical(train_target) 57 | 58 | """In order to visualize a given figure, the following code can be executed.""" 59 | 60 | image_id = 700 61 | plt.imshow(test_data[image_id]) 62 | plt.title("Test image: " + str(image_id)) 63 | plt.show() 64 | 65 | """In the following, we define the network topology. In this case, because of the redundancy typically found in images, we do not employ dropout in the convolutional layers.""" 66 | 67 | input_shape = train_data.shape[1:] 68 | filters = 128 69 | kernel_size = (3,3) 70 | pool_size = (2,2) 71 | 72 | optimizer = 'adam' 73 | loss = 'categorical_crossentropy' 74 | metrics = ['categorical_accuracy'] 75 | activation = 'relu' 76 | activation_function_output = 'softmax' 77 | number_of_cnn_layers = 3 78 | number_of_ff_layers = 3 79 | number_of_units_output = train_target_one_hot_encoding.shape[1] 80 | 81 | cnn_model = Sequential() 82 | cnn_model.add(Conv2D(filters, kernel_size, input_shape = input_shape, 83 | activation = activation)) 84 | 85 | cnn_model.add(BatchNormalization()) 86 | cnn_model.add(MaxPooling2D(pool_size = pool_size)) 87 | 88 | for i in range(number_of_cnn_layers-1): 89 | cnn_model.add(Conv2D(filters, kernel_size, activation = activation)) 90 | cnn_model.add(BatchNormalization()) 91 | cnn_model.add(MaxPooling2D(pool_size = pool_size)) 92 | 93 | cnn_model.add(Flatten()) 94 | 95 | #Feedforward network 96 | for i in range(number_of_ff_layers): 97 | cnn_model.add(Dense(units = 128, activation = activation)) 98 | cnn_model.add(Dropout(0.3)) 99 | 100 | cnn_model.add(Dense(units = number_of_units_output, 101 | activation = activation_function_output)) 102 | 103 | cnn_model.compile(optimizer = optimizer, loss = loss, metrics = metrics) 104 | 105 | """We can use the following command to see the network topology.""" 106 | 107 | cnn_model.summary() 108 | #Saving the resultant figure as 'cnn_model.png'. 109 | plot_model(cnn_model, to_file='cnn_model.png', show_shapes=True, 110 | show_layer_names=True) 111 | 112 | """The training step is executed as follows. Because this network demands a high computational power, we can use a small number of epochs.""" 113 | 114 | batch_size = 30 115 | epochs = 50 116 | 117 | cnn_model.fit(train_data, train_target_one_hot_encoding, 118 | batch_size = batch_size, epochs = epochs) 119 | 120 | """Since there are more than two classes, we show the classification results through a confusion matrix.""" 121 | 122 | predictions = cnn_model.predict(test_data) 123 | found_target = predictions.argmax(axis=1) 124 | 125 | accuracy = accuracy_score(test_target, found_target) 126 | print("Accuracy =", accuracy) 127 | 128 | print("Confusion matrix:") 129 | matrix = confusion_matrix(found_target,test_target) 130 | 131 | plt.title("Confusion matrix:") 132 | plt.xticks(np.linspace(0,9,10)) 133 | plt.yticks(np.linspace(0,9,10)) 134 | plt.imshow(matrix) 135 | plt.show() 136 | 137 | """ 138 | ## License 139 | 140 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 141 | 142 | ## Acknowledgments 143 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 144 | """ 145 | -------------------------------------------------------------------------------- /codes/rbm.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_RBM.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/drive/1R7ZfTxrtIIG_22IlApzXgWxuilbScLLJ 8 | 9 | # Restricted Boltzmann Machine (RBM) 10 | 11 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. This code has absolutely no warranty. 12 | 13 | If you publish a paper related on this material, please cite: 14 | 15 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 16 | 17 | This is the fourth example of deep learning implementation. Here we use a RMB network to provide a recommendation system of CDs and vinyls. 18 | 19 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 20 | """ 21 | 22 | import numpy as np 23 | import pandas as pd 24 | from sklearn.neural_network import BernoulliRBM 25 | import matplotlib.pyplot as plt 26 | import urllib.request 27 | from keras.utils import np_utils 28 | from sklearn.preprocessing import LabelEncoder 29 | import matplotlib.pyplot as plt 30 | 31 | """The following code downlods a dataset regarding the ratings of CDs and vinyls from the Amazon website ([link](http://snap.stanford.edu/data/amazon/productGraph/)). 32 | These data is divided into four columns, as follows: user id, item id, rating, and timestamp. The latter was removed from our analysis. 33 | """ 34 | 35 | main_url = "http://snap.stanford.edu/data/amazon/productGraph/categoryFiles/" 36 | file_name = "ratings_CDs_and_Vinyl.csv" 37 | url = main_url + file_name 38 | col_names = ["user", "item", "rating", "timestamp"] 39 | urllib.request.urlretrieve(url, file_name) 40 | musical_instruments_reviews = pd.read_csv(file_name, names = col_names) 41 | 42 | """In the following, we preprocess the dataset.""" 43 | 44 | #Defining dataset variables 45 | rating = musical_instruments_reviews["rating"].get_values() 46 | rating /= np.max(rating) 47 | 48 | users = musical_instruments_reviews["user"].get_values() 49 | 50 | label_encoder = LabelEncoder() 51 | items = musical_instruments_reviews["item"].get_values() 52 | items = label_encoder.fit_transform(items) 53 | 54 | """In order to reduce the time for running this tutorial, we reduced the number of items.""" 55 | 56 | number_of_items = 6 57 | 58 | #Finding lines to erase 59 | unique_items, item_counts = np.unique(items, return_counts=True) 60 | item2count = dict(zip(unique_items, item_counts)) 61 | item_count = sorted(item2count.items(), key=lambda x: x[1])[::-1] 62 | item_count = item_count[0:number_of_items] 63 | selected_items = [item for item, count in item_count] 64 | 65 | #keeping only the most frequent items 66 | keep_lines = [i for i,item in enumerate(items) if item in selected_items] 67 | keep_lines = np.array(keep_lines) 68 | 69 | rating = rating[keep_lines] 70 | users = users[keep_lines] 71 | items = items[keep_lines] 72 | 73 | #Converting the categorical data into a matrix 74 | item2new_code = {item:i for i,item in enumerate(set(items))} 75 | items = [item2new_code[item] for item in items] 76 | items_one_hot_encoding = np_utils.to_categorical(items) 77 | 78 | items_one_hot_encoding.shape 79 | 80 | """In the following, we weight and merge the codings of the selected columns.""" 81 | 82 | items_weighted = [items_one_hot_encoding[i] * rating[i] 83 | for i in range(len(rating))] 84 | 85 | items_weighted = np.array(items_weighted) 86 | 87 | user2matrix_lines = {user: np.argwhere(user == users).T[0] 88 | for user in set(users)} 89 | 90 | user2purchases = {user:np.max(items_weighted[user2matrix_lines[user]],axis = 0) 91 | for user in set(users)} 92 | 93 | """In the next step, we eliminate the data from users that bought zero or one item. In our analysis, we do not consider the user names.""" 94 | 95 | data = list(user2purchases.values()) 96 | data = np.array(data) 97 | 98 | items_per_line = np.count_nonzero(data, axis=1) 99 | keep_lines = np.argwhere(items_per_line >= 2).T[0] 100 | 101 | data = data[keep_lines,:] 102 | 103 | """The code presented as follows define the neuronal network.""" 104 | 105 | batch_size = 10 106 | learning_rate = 0.01 107 | n_components = 10 #Number of binary hidden units. 108 | n_iter = 5000 109 | verbose = 1 110 | 111 | rbm_model = BernoulliRBM(batch_size = batch_size, learning_rate = learning_rate, 112 | n_components = n_components, n_iter = n_iter, 113 | verbose = verbose) 114 | 115 | """Next, we train the network.""" 116 | 117 | rbm_model = rbm_model.fit(data) 118 | 119 | """Finally, we test the network. 120 | For that, we first analyze some inputs to know if the output makes sense. We test the output as a person that bought only the first product (1). So, we show the matrix lines for others that bought the same product, as follows. 121 | """ 122 | 123 | product_test = 0 124 | selected_lines = np.argwhere(data[:,product_test] > 0).T[0] 125 | plt.imshow(data[selected_lines]) 126 | plt.colorbar() 127 | plt.show() 128 | 129 | """The following code tests the network in order to recommend the most relevant products for a given user. More specifically, for a vector of scores of the acquired products, the RBM returns the products that this user could like. Here, we selected the two first indications by excluding the already acquired products.""" 130 | 131 | test_set = np.zeros(number_of_items) 132 | test_set[product_test] = 1 133 | 134 | test_set = [0,1,0,0,0,0] 135 | 136 | #Here we test a single sample 137 | result_hidden_layer = rbm_model.transform([test_set])[0] 138 | weight_matrix = rbm_model.components_ 139 | 140 | result = np.matmul(weight_matrix.T,result_hidden_layer) 141 | recomended_products = np.argsort(result)[::-1] 142 | recomended_products = [product for product in recomended_products 143 | if product != product_test] 144 | print ("The two recommended product, in drecreasing order, are: " + 145 | str(recomended_products[0:2]) + '.') 146 | 147 | """ 148 | ## License 149 | 150 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 151 | 152 | ## Acknowledgments 153 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 154 | """ 155 | -------------------------------------------------------------------------------- /codes/feedforward_binary.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_feedforward.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_feedforward.ipynb 8 | 9 | #Feedforward networks 10 | 11 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. These codes have absolutely no warranty. 12 | 13 | If you publish a paper related on this material, please cite: 14 | 15 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 16 | 17 | ## Binary Classification 18 | This is the first example of deep learning implementation, in which we address binary classification of wine data. In this example, we consider one feedforward network with 5 hidden layers and with 30 neurons in each layer. The provided networks were built only for didactic purposes and are not appropriate for real applications. 19 | 20 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 21 | """ 22 | 23 | import numpy as np 24 | import keras 25 | from keras.models import Sequential 26 | from keras.layers import Dense, Dropout 27 | from keras.utils.vis_utils import plot_model 28 | from keras.models import model_from_json 29 | from sklearn.datasets import load_wine 30 | from sklearn.model_selection import train_test_split 31 | from sklearn.metrics import accuracy_score 32 | 33 | """If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*).""" 34 | 35 | import tensorflow as tf 36 | from keras import backend as K 37 | 38 | print(K.tensorflow_backend._get_available_gpus()) 39 | 40 | number_of_cpu_cores = 8 41 | config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) 42 | session = tf.Session(config=config) 43 | keras.backend.set_session(session) 44 | 45 | """Here, we use the Wine dataset. It is available at Sklearn library on [sklearn-datasets-wine](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_wine.html). For more information [wine-UCI](https://archive.ics.uci.edu/ml/datasets/Wine). 46 | Because this dataset comprises three classes and here we exemplify a binary classification, we considered only the two first classes. 47 | """ 48 | 49 | wine = load_wine() 50 | data = wine['data'] 51 | target = wine['target'] 52 | target_names = wine['target_names'] 53 | 54 | #The selected items are stored in the variable called "hold". 55 | hold = np.argwhere(target!=2).T[0] 56 | data = data[hold] 57 | target = target[hold] 58 | target_names = target_names[0:1] 59 | 60 | #Here, we divide our dataset into training and test sets. 61 | test_size = 0.25 #fraction 62 | training_data,test_data,training_target,test_target = train_test_split(data, 63 | target, test_size=test_size) 64 | 65 | """In the following, we configure the neuronal network. It is not necessary to include bias because this parameter is set as true by default.""" 66 | 67 | #Set of parameters 68 | input_dim = data.shape[1] 69 | kernel_initializer = 'random_uniform' 70 | bias_initializer='zeros' 71 | activation_function_hidden = 'relu' 72 | activation_function_output = 'sigmoid' 73 | optimizer = 'adam' 74 | loss = 'binary_crossentropy' 75 | metrics = ['binary_accuracy'] 76 | number_of_layers = 5 77 | number_of_units_hidden = 30 78 | number_of_units_output = 1 79 | dropout_percentage = 0.25 80 | 81 | 82 | #Creating model 83 | ff_model = Sequential() 84 | ff_model.add(Dense(units = number_of_units_hidden, 85 | activation = activation_function_hidden, 86 | kernel_initializer = kernel_initializer, 87 | input_dim = input_dim)) 88 | 89 | for i in range(number_of_layers-1): 90 | #Inserting a dense hidden layer 91 | ff_model.add(Dense(units = number_of_units_hidden, 92 | activation = activation_function_hidden, 93 | kernel_initializer = kernel_initializer, 94 | input_dim = number_of_units_hidden)) 95 | #Inserting dropout 96 | ff_model.add(Dropout(dropout_percentage)) 97 | 98 | ff_model.add(Dense(units = number_of_units_output, 99 | activation = activation_function_output)) 100 | ff_model.compile(optimizer = optimizer, loss = loss, metrics = metrics) 101 | 102 | """In order to check the network topology, you can use the subsequent command.""" 103 | 104 | ff_model.summary() 105 | 106 | """Another option is to visualize the topology as a figure.""" 107 | 108 | #Saving the resultant figure as 'ff_model.png'. 109 | plot_model(ff_model, to_file='ff_model.png', show_shapes=True, 110 | show_layer_names=True) 111 | 112 | """Next, we train the network""" 113 | 114 | batch_size = 10 115 | epochs = 200 116 | ff_model.fit(training_data,training_target, batch_size = batch_size, 117 | epochs = epochs) 118 | 119 | """In order to create an application, it is possible to save the network and the respective trained weights as follows.""" 120 | 121 | #Saving the network model 122 | ff_model_json = ff_model.to_json() 123 | with open('ff_model.json', 'w') as file: 124 | file.write(ff_model_json) 125 | 126 | #Saving weights 127 | ff_model.save_weights('ff_model.h5') 128 | 129 | """The following code can be employed to open a pre-trained model.""" 130 | 131 | with open('ff_model.json', 'r') as file: 132 | ff_model_json = file.read() 133 | 134 | ff_model = model_from_json(ff_model_json) 135 | ff_model.load_weights('ff_model.h5') 136 | 137 | """There are different analysis that can account for the quality of the results. Here, we consider only the measurement of accuracy.""" 138 | 139 | predictions = ff_model.predict(test_data) 140 | #Because it is a binary classification, we consider the values higher than 0.5 141 | #as being part of class 1 otherwise 0. 142 | predictions = (predictions > 0.5) 143 | accuracy = accuracy_score(test_target, predictions) 144 | print("Accuracy =", accuracy) 145 | 146 | 147 | """ 148 | ## License 149 | 150 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 151 | 152 | ## Acknowledgments 153 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 154 | """ 155 | -------------------------------------------------------------------------------- /codes/lstm.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_LSTM.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_LSTM.ipynb 8 | 9 | #Long Short-Term Memory (LSTM) 10 | 11 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. This code has absolutely no warranty. 12 | 13 | If you publish a paper related on this material, please cite: 14 | 15 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 16 | 17 | This is the third example of deep learning implementation. Here we use a LSTM network to predict the Bitcoin prices along time by using the input as a temporal series. 18 | 19 | 20 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 21 | """ 22 | 23 | import numpy as np 24 | import keras 25 | from keras.models import Sequential 26 | from keras.utils.vis_utils import plot_model 27 | from keras.layers import Dense, Dropout, LSTM 28 | from keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint 29 | from sklearn.model_selection import train_test_split 30 | from sklearn.metrics import accuracy_score 31 | from sklearn.preprocessing import MinMaxScaler 32 | import matplotlib.pyplot as plt 33 | import pandas as pd 34 | import pandas_datareader 35 | 36 | """If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*).""" 37 | 38 | import tensorflow as tf 39 | from keras import backend as K 40 | 41 | print(K.tensorflow_backend._get_available_gpus()) 42 | 43 | number_of_cpu_cores = 8 44 | config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) 45 | session = tf.Session(config=config) 46 | keras.backend.set_session(session) 47 | 48 | """(*) Here, we use the Bitcoin daily prices dataset, which is available at 49 | [yhaoo-stock-market](https://finance.yahoo.com/). The data contains seven columns, organized as follows: date, opening stock price, high daily price, low daily price, closing stock price, the currency volume traded on the day, and the adjusted closing price. 50 | """ 51 | 52 | train_size = 1200 53 | start_date = '2015-01-01'# Bitcoin started on '2010-07-16' 54 | 55 | dataset = pandas_datareader.data.get_data_yahoo("BTC-USD", start = start_date) 56 | data_oerder = ['Open','High', 'Low', 'Close', 'Volume', 'Adj Close'] 57 | dataset = dataset[data_oerder] 58 | 59 | 60 | train_dataset = dataset.iloc[0:train_size, 1::].values 61 | test_dataset = dataset.iloc[train_size::, 1::].values 62 | 63 | min_max_scaler = MinMaxScaler(feature_range=(0,1)) 64 | normalized_train_dataset = min_max_scaler.fit_transform(train_dataset) 65 | 66 | min_max_scaler_train = MinMaxScaler(feature_range=(0,1)) 67 | normalized_train_price = min_max_scaler_train.fit_transform(train_dataset[:,0:1]) 68 | 69 | """In the following, we define the network topology.""" 70 | 71 | window_size = 50 72 | number_of_lstm_layers = 3 73 | activation = 'sigmoid' 74 | return_sequences = True 75 | units_first_layer = 100 76 | units = 50 77 | 78 | data = [] 79 | train_price = [] 80 | for i in range(window_size, train_size): 81 | data.append(normalized_train_dataset[i-window_size:i, 0:6]) 82 | train_price.append(normalized_train_dataset[i, 0]) 83 | data, train_price = np.array(data), np.array(train_price) 84 | 85 | lstm_model = Sequential() 86 | lstm_model.add(LSTM(units = units_first_layer, 87 | return_sequences = return_sequences, 88 | input_shape = (data.shape[1], 5))) 89 | lstm_model.add(Dropout(0.2)) 90 | 91 | for i in range(number_of_lstm_layers-2): 92 | lstm_model.add(LSTM(units = units, return_sequences = return_sequences)) 93 | lstm_model.add(Dropout(0.2)) 94 | 95 | lstm_model.add(LSTM(units = units)) 96 | lstm_model.add(Dropout(0.2)) 97 | 98 | #Output layer 99 | lstm_model.add(Dense(units = 1, activation = activation)) 100 | 101 | """In order to check the network topology, the subsequent command can be used.""" 102 | 103 | lstm_model.summary() 104 | #Saving the resultant figure as 'ff_model.png'. 105 | plot_model(lstm_model, to_file='lstm_model.png', show_shapes=True, 106 | show_layer_names=True) 107 | 108 | """The training step is executed as follows.""" 109 | 110 | #Here we set verbose as true 111 | verbose = 1 112 | 113 | batch_size = 32 114 | epochs = 10 115 | filepath = 'weights.h5' #name of the file with the network weights 116 | monitor = 'loss' 117 | optimizer = 'adam' 118 | loss = 'mean_squared_error' 119 | metrics = ['mean_absolute_error'] 120 | 121 | lstm_model.compile(optimizer = optimizer, loss = loss, metrics = metrics) 122 | 123 | early_stopping = EarlyStopping(monitor = monitor, min_delta = 1e-15, 124 | patience = 10, verbose = verbose) 125 | reduce_learning_rate_on_plateau = ReduceLROnPlateau(monitor = monitor, 126 | factor = 0.2, patience = 5, 127 | verbose = verbose) 128 | model_checkpoint = ModelCheckpoint(filepath = filepath, monitor = monitor, 129 | save_best_only = True, verbose = verbose) 130 | lstm_model.fit(data, train_price, epochs = epochs, batch_size = batch_size, 131 | callbacks = [early_stopping, reduce_learning_rate_on_plateau, 132 | model_checkpoint]) 133 | 134 | """The following code verifies the data in the network.""" 135 | 136 | test_price = test_dataset[:, 0:1] 137 | complete_dataset = dataset.iloc[:,1::] 138 | 139 | train_data = complete_dataset[len(complete_dataset) - len(test_dataset) - 140 | window_size:].values 141 | train_data = min_max_scaler.transform(train_data) 142 | 143 | 144 | X_test = [] 145 | for i in range(window_size,len(train_data)): 146 | X_test.append(train_data[i-window_size:i, 0:6]) 147 | X_test = np.array(X_test) 148 | 149 | calculated_prices = lstm_model.predict(X_test) 150 | calculated_prices = min_max_scaler_train.inverse_transform(calculated_prices) 151 | 152 | train_price = min_max_scaler_train.inverse_transform([train_price]) 153 | train_price = train_price[0] 154 | 155 | """In the following, we plot the train set, as well as the prediction and the expected values.""" 156 | 157 | plt.plot(np.linspace(0,len(train_price)-1,len(train_price)), 158 | train_price, label = 'Real price train', color = 'k') 159 | plt.plot(np.linspace(len(train_price),len(train_price)+len(test_price)-1, 160 | len(test_price)), test_price, label = 'Real price') 161 | plt.plot(np.linspace(len(train_price),len(train_price)+len(test_price)-1, 162 | len(test_price)), calculated_prices, label = 'Prevision') 163 | plt.title('BTC prevision') 164 | plt.xlabel('Time') 165 | plt.ylabel('Value') 166 | plt.legend() 167 | plt.show() 168 | 169 | """ 170 | ## License 171 | 172 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 173 | 174 | ## Acknowledgments 175 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 176 | """ 177 | -------------------------------------------------------------------------------- /codes/gan.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_GAN.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/drive/1CbRNDN25uaN2WCeyklwPkbZZkXM_IDge 8 | 9 | # Generative Adversarial Networks 10 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. This code has absolutely no warranty. 11 | 12 | If you publish a paper related on this material, please cite: 13 | 14 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 15 | 16 | It was elaborated to create a network that can generate handwritten characters automatically. 17 | 18 | 19 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 20 | """ 21 | 22 | import numpy as np 23 | import pandas as pd 24 | import keras 25 | from keras.models import Sequential, model_from_json 26 | from keras.utils.vis_utils import plot_model 27 | from keras.datasets import mnist 28 | from keras.layers import InputLayer, Dense, Flatten, Reshape, Input, Dropout 29 | from keras.layers.advanced_activations import LeakyReLU 30 | from keras.layers import BatchNormalization 31 | from keras.models import Model,Sequential 32 | from keras.regularizers import L1L2 33 | from sklearn.model_selection import train_test_split 34 | from sklearn.metrics import accuracy_score 35 | from sklearn.preprocessing import MinMaxScaler 36 | import matplotlib.pyplot as plt 37 | import cv2 38 | 39 | """If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*).""" 40 | 41 | import tensorflow as tf 42 | from keras import backend as K 43 | 44 | print(K.tensorflow_backend._get_available_gpus()) 45 | 46 | number_of_cpu_cores = 8 47 | config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) 48 | session = tf.Session(config=config) 49 | keras.backend.set_session(session) 50 | 51 | """(*) In this example we used the MNIST database in which it is composed by grayscale images of the 10 handwritten digits. It is available at Keras library on [keras-datasets](https://keras.io/datasets/). 52 | 53 | The following command is used to load the data set. 54 | """ 55 | 56 | (train_data_raw, train_target_raw), (_, _) = mnist.load_data() 57 | 58 | """Because this code consumes too much of processing time, here we considered only the zeros and ones.""" 59 | 60 | train_data = [img for i, img in enumerate(train_data_raw) 61 | if train_target_raw[i] == 0 or train_target_raw[i] == 1] 62 | train_data = np.array(train_data) 63 | 64 | """In order to visualize a given figure, the following code can be executed.""" 65 | 66 | image_id = 1000 67 | plt.figure(figsize = (1,1)) 68 | plt.imshow(train_data[image_id], cmap='gray') 69 | plt.title("Test image: " + str(image_id)) 70 | #plt.axis('off') 71 | plt.show() 72 | 73 | """Definition of the used variables.""" 74 | 75 | input_shape = train_data.shape[1::] 76 | activation_output_generator = 'sigmoid' 77 | activation_output_discrimninator = 'sigmoid' 78 | input_dim = 50 79 | number_of_epochs = 1000 80 | batch_size = 100 81 | train_data = train_data.astype('float32') / 255 82 | 83 | """In the following, we present the generator model.""" 84 | 85 | generator_model = Sequential() 86 | 87 | generator_model.add(Dense(units=64,input_dim = input_dim, 88 | kernel_regularizer = L1L2(1e-5, 1e-5))) 89 | generator_model.add(BatchNormalization()) 90 | generator_model.add(LeakyReLU(alpha=0.3)) 91 | 92 | generator_model.add(Dense(units=128, kernel_regularizer = L1L2(1e-5, 1e-5))) 93 | generator_model.add(BatchNormalization()) 94 | generator_model.add(LeakyReLU(alpha=0.3)) 95 | 96 | generator_model.add(Dense(units=256, kernel_regularizer = L1L2(1e-5, 1e-5))) 97 | generator_model.add(BatchNormalization()) 98 | generator_model.add(LeakyReLU(alpha=0.3)) 99 | 100 | generator_model.add(Dense(units = input_shape[0] * input_shape[1], 101 | activation = activation_output_generator)) 102 | 103 | generator_model.add(Reshape(input_shape)) 104 | 105 | generator_model.compile(loss='binary_crossentropy', optimizer="adam") 106 | 107 | """The summary of the generator model is shown by employing the following code.""" 108 | 109 | generator_model.summary() 110 | 111 | """The following code represents the discriminator model.""" 112 | 113 | discriminator_model = Sequential() 114 | discriminator_model.add(InputLayer(input_shape = input_shape)) 115 | discriminator_model.add(Flatten()) 116 | 117 | discriminator_model.add(Dense(units=256,kernel_regularizer = L1L2(1e-5, 1e-5))) 118 | discriminator_model.add(LeakyReLU(alpha=0.3)) 119 | discriminator_model.add(Dropout(0.2)) 120 | 121 | 122 | discriminator_model.add(Dense(units=128,kernel_regularizer = L1L2(1e-5, 1e-5))) 123 | discriminator_model.add(LeakyReLU(alpha=0.3)) 124 | discriminator_model.add(Dropout(0.2)) 125 | 126 | discriminator_model.add(Dense(units=64,kernel_regularizer = L1L2(1e-5, 1e-5))) 127 | discriminator_model.add(LeakyReLU(alpha=0.3)) 128 | 129 | discriminator_model.add(Dense(units=1, 130 | activation = activation_output_discrimninator)) 131 | 132 | discriminator_model.compile(loss='binary_crossentropy', 133 | optimizer = "adam") 134 | 135 | """The summary of the discriminator model is shown by using the following code.""" 136 | 137 | discriminator_model.summary() 138 | 139 | """The following code incorporates the complete gan model.""" 140 | 141 | gan_input = Input(shape = (input_dim,)) 142 | gan_output= discriminator_model(generator_model(gan_input)) 143 | gan = Model(inputs = gan_input, outputs = gan_output) 144 | gan.compile(loss = 'binary_crossentropy', optimizer = 'adam') 145 | 146 | """The summary of the gan model is shown by using the following code.""" 147 | 148 | gan.summary() 149 | 150 | """Next, we train the GAN.""" 151 | 152 | y = np.ones(batch_size) 153 | 154 | #Parameters of the noise distribution 155 | mu = 0 156 | sigma = 1 157 | 158 | #We created this array to avoid number repetitions 159 | train_indices = np.arange(train_data.shape[0]) 160 | np.random.shuffle(train_indices) 161 | 162 | #Here we define the labels used to train the gan 163 | train_labels = np.zeros(2*batch_size,dtype = int) 164 | train_labels[0:batch_size] = 1#generated images 165 | 166 | for epoch in range(number_of_epochs): 167 | print("\rEpoch:", epoch + 1, "of", number_of_epochs, end = '') 168 | for _ in range(batch_size): 169 | input_noise = np.random.normal(loc = mu, scale = sigma, 170 | size = [batch_size, input_dim]) 171 | generated_images = generator_model.predict(input_noise) 172 | np.random.shuffle(train_indices) 173 | image_batch = train_data[train_indices[0:batch_size]] 174 | train_images = np.concatenate((image_batch, generated_images)) 175 | #Training the discriminator 176 | discriminator_model.trainable = True 177 | discriminator_model.train_on_batch(train_images, train_labels) 178 | #Training the gan 179 | discriminator_model.trainable = False 180 | train_noise = np.random.normal(loc = mu, scale = sigma, 181 | size = [batch_size, input_dim]) 182 | gan.train_on_batch(train_noise, y) 183 | 184 | #In order to visualize the training progress, we employ the following code. 185 | if epoch % 100 == 0: 186 | n_examples = 10 187 | scale_image = 1 * n_examples 188 | noise= np.random.normal(loc = mu, scale = sigma, 189 | size = (n_examples, input_dim)) 190 | generated_images = generator_model.predict(noise) 191 | n_pixels = generated_images.shape[1] 192 | n_pixels_col = np.int(np.sqrt(n_pixels)) 193 | fig, axes = plt.subplots(1,n_examples, 194 | figsize = (scale_image, 195 | scale_image * n_examples)) 196 | for i in range(generated_images.shape[0]): 197 | axes[i].imshow(generated_images[i], cmap = "gray") 198 | axes[i].axis('off') 199 | plt.show() 200 | print("") 201 | 202 | """In order to generate the figures the following code can be employed.""" 203 | 204 | n_examples = 5 205 | scale_image = 5 206 | noise= np.random.normal(loc = mu, scale = sigma, size = (n_examples, input_dim)) 207 | generated_images = generator_model.predict(noise) 208 | 209 | fig, axes = plt.subplots(1,n_examples, 210 | figsize = (scale_image, scale_image * n_examples)) 211 | for i in range(generated_images.shape[0]): 212 | axes[i].imshow(generated_images[i], cmap = "gray") 213 | axes[i].axis('off') 214 | 215 | plt.show() 216 | 217 | """ 218 | ## License 219 | 220 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 221 | 222 | ## Acknowledgments 223 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 224 | """ 225 | -------------------------------------------------------------------------------- /codes/autoencoder.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """deepLearning_autoencoder.ipynb 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/github/hfarruda/deeplearningtutorial/blob/master/deepLearning_autoencoder.ipynb 8 | 9 | # Autoencoders 10 | 11 | This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. This code has absolutely no warranty. 12 | 13 | If you publish a paper related on this material, please cite: 14 | 15 | H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, "Learning Deep Learning (CDT-15)," 2019. 16 | 17 | This example uses the Autoencoder model to illustrate a possible application concerning image clustering. Here we show how to use the resulting codes to reduce the dimensionality. We also project our data by using a Principal Component Analysis (PCA). 18 | 19 | First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend). 20 | """ 21 | 22 | import numpy as np 23 | import matplotlib.pyplot as plt 24 | import pandas as pd 25 | import keras 26 | from keras.models import Sequential, model_from_json, Model 27 | from keras.utils import np_utils 28 | from keras.utils.vis_utils import plot_model 29 | from keras.datasets import fashion_mnist 30 | from keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint 31 | from sklearn.model_selection import train_test_split 32 | from sklearn.metrics import accuracy_score 33 | from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Reshape 34 | from keras.layers import UpSampling2D 35 | from sklearn.preprocessing import MinMaxScaler 36 | import sklearn.decomposition 37 | from sklearn.preprocessing import StandardScaler 38 | 39 | """If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*).""" 40 | 41 | import tensorflow as tf 42 | from keras import backend as K 43 | 44 | print(K.tensorflow_backend._get_available_gpus()) 45 | 46 | number_of_cpu_cores = 8 47 | config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) 48 | session = tf.Session(config=config) 49 | keras.backend.set_session(session) 50 | 51 | """(*) In this example, we used the Fashion-MNIST database, composed by grayscale images of 10 categories of fashion items (trouser, pullover, dress, coat, sandal, shirt, sneaker, bag, and ankle boot). It is available at Keras library on [keras-datasets](https://keras.io/datasets/).""" 52 | 53 | (train_data, train_target), (test_data, test_target) = fashion_mnist.load_data() 54 | 55 | train_target_one_hot_encoding = np_utils.to_categorical(train_target) 56 | 57 | #Divide by the maximun value of a pixel (255) to have the values between 0 and 1 58 | train_data = train_data.astype('float32') / 255. 59 | test_data = test_data.astype('float32') / 255. 60 | 61 | """For the sake of simplicity, we add zeros to the images to have shape 32x32. Because 32 is a power of 2 it is easier to configure the decoder layer.""" 62 | 63 | train_data_auxiliar = [] 64 | for data in train_data: 65 | new_image = np.zeros((32,32)) 66 | new_image[2:data.shape[0]+2, 2:data.shape[1]+2] = data 67 | train_data_auxiliar.append(new_image) 68 | 69 | test_data_auxiliar = [] 70 | for data in test_data: 71 | new_image = np.zeros((32,32)) 72 | new_image[2:data.shape[0]+2, 2:data.shape[1]+2] = data 73 | test_data_auxiliar.append(new_image) 74 | 75 | train_data = np.array(train_data_auxiliar) 76 | test_data = np.array(test_data_auxiliar) 77 | 78 | train_data = train_data.reshape(train_data.shape[0], train_data.shape[1], 79 | train_data.shape[2], 1) 80 | test_data = test_data.reshape(test_data.shape[0], test_data.shape[1], 81 | test_data.shape[2], 1) 82 | 83 | """In order to visualize a given figure, the following code can be executed.""" 84 | 85 | image_id = 700 86 | image = test_data[image_id] 87 | image = image[:,:,0] 88 | plt.imshow(image, cmap = 'gray') 89 | plt.title("Test image: " + str(image_id)) 90 | plt.show() 91 | 92 | """In the following, we define the network topology. Similar to what was adopted for the CNN case, here we do not employ dropout after the convolutional layers. Because this network demands a high computational power, the variable epochs can receive a smaller number (e.g., 5). However, in this case, the resulting accuracy tends to be much lower. 93 | 94 | First, we define some necessary variables. 95 | """ 96 | 97 | input_shape = train_data.shape[1::] 98 | #if len(input_shape) == 2: 99 | # input_shape = (input_shape[0], input_shape[1], 1) 100 | filters_first_layer = 64 101 | filters = 32 102 | kernel_size = (3,3) 103 | pool_size = (2,2) 104 | 105 | activation = 'relu' 106 | activation_function_output = 'sigmoid' #the output should be between 0 and 1 107 | number_of_cnn_layers = 2 108 | number_of_units_output = train_target_one_hot_encoding.shape[1] 109 | padding = 'same' 110 | strides = (2,2) 111 | 112 | optimizer = 'adam' 113 | loss = 'binary_crossentropy' 114 | metrics = ['accuracy'] 115 | epochs = 50 116 | batch_size = 128 117 | 118 | #Network model 119 | autoencoder_model = Sequential() 120 | 121 | """We configure the encoder layers. Normally, for images the autoencoder is represented by a 2D matrix, but here we 122 | adopt flattening in order to be able to plot the respective PCA projection. 123 | """ 124 | 125 | autoencoder_model.add(Conv2D(filters = filters_first_layer, 126 | kernel_size = kernel_size, 127 | input_shape = input_shape, 128 | activation = activation, padding = padding )) 129 | 130 | autoencoder_model.add(MaxPooling2D(pool_size = pool_size, padding = padding)) 131 | 132 | 133 | for i in range(number_of_cnn_layers-1): 134 | autoencoder_model.add(Conv2D(filters = filters, kernel_size = kernel_size, 135 | activation = activation, padding = padding, 136 | strides = strides)) 137 | autoencoder_model.add(MaxPooling2D(pool_size = pool_size, 138 | padding = padding)) 139 | 140 | 141 | #This is the coding 142 | autoencoder_model.add(Flatten()) 143 | flatten_layer_name = autoencoder_model.output_names[0] 144 | 145 | """Here, we define the decoder.""" 146 | 147 | #First we define the input size 148 | output_len = autoencoder_model.output_shape[1] 149 | height = np.int(np.sqrt(output_len/filters)) 150 | 151 | #Find the shape of the decoder input 152 | autoencoder_model.add(Reshape((height, height, filters))) 153 | 154 | for i in range(number_of_cnn_layers): 155 | autoencoder_model.add(Conv2D(filters = filters, kernel_size = kernel_size, 156 | activation = activation, padding = padding)) 157 | autoencoder_model.add(UpSampling2D(size = pool_size)) 158 | 159 | autoencoder_model.add(Conv2D(filters = filters_first_layer, 160 | kernel_size = kernel_size, 161 | activation = activation, padding = padding)) 162 | autoencoder_model.add(UpSampling2D(size = pool_size)) 163 | autoencoder_model.add(Conv2D(filters = 1, kernel_size = kernel_size, 164 | activation = activation_function_output, 165 | padding = padding)) 166 | 167 | """We can use the following command to see the network topology.""" 168 | 169 | autoencoder_model.summary() 170 | #Saving the resultant figure as 'autoencoder_model.png'. 171 | plot_model(autoencoder_model, to_file='autoencoder_model.png', show_shapes=True, 172 | show_layer_names=True) 173 | 174 | """The entire configuration is then used to train the coding and decoding.""" 175 | 176 | autoencoder_model.compile(optimizer = optimizer, loss = loss, metrics = metrics) 177 | autoencoder_model.fit(train_data, train_data, epochs = epochs, 178 | batch_size = batch_size) 179 | 180 | """The following code shows how to use the already trained coding.""" 181 | 182 | output_model = autoencoder_model.get_layer(flatten_layer_name).output 183 | encoder = Model(inputs = autoencoder_model.input, 184 | outputs = output_model) 185 | encoder.summary() 186 | 187 | """The following code is used to compute the codings.""" 188 | 189 | codings = encoder.predict(test_data) 190 | 191 | """By employing the codings and the known classes, we plot a PCA (principal component analysis) of the test data.""" 192 | 193 | X = codings.copy() 194 | targets = test_target 195 | 196 | #Standardization 197 | X = StandardScaler().fit_transform(X) 198 | decomposition = sklearn.decomposition.PCA(n_components=2) 199 | pca = decomposition.fit(X) 200 | transform = pca.transform(X) 201 | 202 | plt.figure(figsize = (6,4)) 203 | classes = [] 204 | for target in set(targets): 205 | classes.append(target) 206 | pos = np.argwhere(targets == target).T[0] 207 | plt.scatter([transform[pos,0]],[transform[pos,1]], alpha = 0.3) 208 | 209 | 210 | label = "PC1 ({:1.2f}%)".format(pca.explained_variance_ratio_[0]*100) 211 | plt.xlabel(label) 212 | label = "PC2 ({:1.2f}%)".format(pca.explained_variance_ratio_[1]*100) 213 | plt.ylabel(label) 214 | 215 | plt.margins(0.05,0.05) 216 | plt.legend(classes, loc = 'best') 217 | plt.tight_layout() 218 | 219 | plt.show() 220 | 221 | """ 222 | ## License 223 | 224 | This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0)International License. 225 | 226 | ## Acknowledgments 227 | Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and NAP-PRP-USP for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 15/18942-8 and 18/09125-4) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 2015/22308-2. 228 | """ 229 | -------------------------------------------------------------------------------- /libraries.txt: -------------------------------------------------------------------------------- 1 | All of these codes were developed and executed in Python 3, with the following libraries: 2 | 3 | Package Version 4 | ------------------------ --------------------- 5 | absl-py 0.7.1 6 | alabaster 0.7.12 7 | albumentations 0.1.12 8 | altair 3.2.0 9 | astor 0.8.0 10 | astropy 3.0.5 11 | atari-py 0.1.15 12 | atomicwrites 1.3.0 13 | attrs 19.1.0 14 | audioread 2.1.8 15 | autograd 1.3 16 | Babel 2.7.0 17 | backcall 0.1.0 18 | backports.tempfile 1.0 19 | backports.weakref 1.0.post1 20 | beautifulsoup4 4.6.3 21 | bleach 3.1.0 22 | blis 0.2.4 23 | bokeh 1.0.4 24 | boto 2.49.0 25 | boto3 1.9.216 26 | botocore 1.12.216 27 | Bottleneck 1.2.1 28 | branca 0.3.1 29 | bs4 0.0.1 30 | bz2file 0.98 31 | cachetools 3.1.1 32 | certifi 2019.6.16 33 | cffi 1.12.3 34 | chainer 5.4.0 35 | chardet 3.0.4 36 | Click 7.0 37 | cloudpickle 0.6.1 38 | cmake 3.12.0 39 | colorlover 0.3.0 40 | community 1.0.0b1 41 | contextlib2 0.5.5 42 | convertdate 2.1.3 43 | coverage 3.7.1 44 | coveralls 0.5 45 | crcmod 1.7 46 | cufflinks 0.14.6 47 | cvxopt 1.2.3 48 | cvxpy 1.0.25 49 | cycler 0.10.0 50 | cymem 2.0.2 51 | Cython 0.29.13 52 | daft 0.0.4 53 | dask 1.1.5 54 | dataclasses 0.6 55 | datascience 0.10.6 56 | decorator 4.4.0 57 | defusedxml 0.6.0 58 | descartes 1.1.0 59 | dill 0.3.0 60 | distributed 1.25.3 61 | Django 2.2.4 62 | dlib 19.16.0 63 | dm-sonnet 1.34 64 | docopt 0.6.2 65 | docutils 0.15.2 66 | dopamine-rl 1.0.5 67 | easydict 1.9 68 | ecos 2.0.7.post1 69 | editdistance 0.5.3 70 | en-core-web-sm 2.1.0 71 | entrypoints 0.3 72 | ephem 3.7.7.0 73 | et-xmlfile 1.0.1 74 | fa2 0.3.5 75 | fancyimpute 0.4.3 76 | fastai 1.0.57 77 | fastcache 1.1.0 78 | fastdtw 0.3.2 79 | fastprogress 0.1.21 80 | fastrlock 0.4 81 | fbprophet 0.5 82 | feather-format 0.4.0 83 | featuretools 0.4.1 84 | filelock 3.0.12 85 | fix-yahoo-finance 0.0.22 86 | Flask 1.1.1 87 | folium 0.8.3 88 | fsspec 0.4.1 89 | future 0.16.0 90 | gast 0.2.2 91 | GDAL 2.2.2 92 | gdown 3.6.4 93 | gensim 3.6.0 94 | geographiclib 1.49 95 | geopy 1.17.0 96 | gevent 1.4.0 97 | gin-config 0.2.0 98 | glob2 0.7 99 | google 2.0.2 100 | google-api-core 1.14.2 101 | google-api-python-client 1.7.11 102 | google-auth 1.4.2 103 | google-auth-httplib2 0.0.3 104 | google-auth-oauthlib 0.4.0 105 | google-cloud-bigquery 1.14.0 106 | google-cloud-core 1.0.3 107 | google-cloud-datastore 1.8.0 108 | google-cloud-language 1.2.0 109 | google-cloud-storage 1.16.1 110 | google-cloud-translate 1.5.0 111 | google-colab 1.0.0 112 | google-pasta 0.1.7 113 | google-resumable-media 0.3.3 114 | googleapis-common-protos 1.6.0 115 | googledrivedownloader 0.4 116 | graph-nets 1.0.4 117 | graphviz 0.10.1 118 | greenlet 0.4.15 119 | grpcio 1.15.0 120 | gspread 3.0.1 121 | gspread-dataframe 3.0.3 122 | gunicorn 19.9.0 123 | gym 0.10.11 124 | h5py 2.8.0 125 | HeapDict 1.0.0 126 | holidays 0.9.11 127 | html5lib 1.0.1 128 | httpimport 0.5.16 129 | httplib2 0.11.3 130 | humanize 0.5.1 131 | hyperopt 0.1.2 132 | ideep4py 2.0.0.post3 133 | idna 2.8 134 | image 1.5.27 135 | imageio 2.4.1 136 | imagesize 1.1.0 137 | imbalanced-learn 0.4.3 138 | imblearn 0.0 139 | imgaug 0.2.9 140 | importlib-metadata 0.19 141 | imutils 0.5.3 142 | inflect 2.1.0 143 | intel-openmp 2019.0 144 | intervaltree 2.1.0 145 | ipykernel 4.6.1 146 | ipython 5.5.0 147 | ipython-genutils 0.2.0 148 | ipython-sql 0.3.9 149 | ipywidgets 7.5.1 150 | itsdangerous 1.1.0 151 | jax 0.1.43 152 | jaxlib 0.1.26 153 | jdcal 1.4.1 154 | jedi 0.15.1 155 | jieba 0.39 156 | Jinja2 2.10.1 157 | jmespath 0.9.4 158 | joblib 0.13.2 159 | jpeg4py 0.1.4 160 | jsonschema 2.6.0 161 | jupyter 1.0.0 162 | jupyter-client 5.3.1 163 | jupyter-console 5.2.0 164 | jupyter-core 4.5.0 165 | kaggle 1.5.5 166 | kapre 0.1.3.1 167 | Keras 2.2.5 168 | Keras-Applications 1.0.8 169 | Keras-Preprocessing 1.1.0 170 | keras-vis 0.4.1 171 | kiwisolver 1.1.0 172 | knnimpute 0.1.0 173 | librosa 0.6.3 174 | lightgbm 2.2.3 175 | llvmlite 0.29.0 176 | lmdb 0.97 177 | lucid 0.3.8 178 | lunardate 0.2.0 179 | lxml 4.2.6 180 | magenta 0.3.19 181 | Markdown 3.1.1 182 | MarkupSafe 1.1.1 183 | matplotlib 3.0.3 184 | matplotlib-venn 0.11.5 185 | mesh-tensorflow 0.0.5 186 | mido 1.2.6 187 | mir-eval 0.5 188 | missingno 0.4.2 189 | mistune 0.8.4 190 | mizani 0.5.4 191 | mkl 2019.0 192 | mlxtend 0.14.0 193 | more-itertools 7.2.0 194 | moviepy 0.2.3.5 195 | mpi4py 3.0.2 196 | mpmath 1.1.0 197 | msgpack 0.5.6 198 | multiprocess 0.70.8 199 | multitasking 0.0.9 200 | murmurhash 1.0.2 201 | music21 5.5.0 202 | natsort 5.5.0 203 | nbconvert 5.6.0 204 | nbformat 4.4.0 205 | networkx 2.3 206 | nibabel 2.3.3 207 | nltk 3.2.5 208 | nose 1.3.7 209 | notebook 5.2.2 210 | np-utils 0.5.11.1 211 | numba 0.40.1 212 | numexpr 2.7.0 213 | numpy 1.16.4 214 | nvidia-ml-py3 7.352.0 215 | oauth2client 4.1.3 216 | oauthlib 3.1.0 217 | okgrade 0.4.3 218 | olefile 0.46 219 | opencv-contrib-python 3.4.3.18 220 | opencv-python 3.4.5.20 221 | openpyxl 2.5.9 222 | opt-einsum 3.0.1 223 | osqp 0.5.0 224 | packaging 19.1 225 | palettable 3.2.0 226 | pandas 0.24.2 227 | pandas-datareader 0.7.4 228 | pandas-gbq 0.4.1 229 | pandas-profiling 1.4.1 230 | pandocfilters 1.4.2 231 | parso 0.5.1 232 | pathlib 1.0.1 233 | patsy 0.5.1 234 | pexpect 4.7.0 235 | pickleshare 0.7.5 236 | Pillow 4.3.0 237 | pip 19.2.3 238 | pip-tools 3.9.0 239 | plac 0.9.6 240 | plotly 3.6.1 241 | plotnine 0.5.1 242 | pluggy 0.7.1 243 | portpicker 1.2.0 244 | prefetch-generator 1.0.1 245 | preshed 2.0.1 246 | pretty-midi 0.2.8 247 | prettytable 0.7.2 248 | progressbar2 3.38.0 249 | prometheus-client 0.7.1 250 | promise 2.2.1 251 | prompt-toolkit 1.0.16 252 | protobuf 3.7.1 253 | psutil 5.4.8 254 | psycopg2 2.7.6.1 255 | ptyprocess 0.6.0 256 | py 1.8.0 257 | pyarrow 0.14.1 258 | pyasn1 0.4.6 259 | pyasn1-modules 0.2.6 260 | pycocotools 2.0.0 261 | pycparser 2.19 262 | pydot 1.3.0 263 | pydot-ng 2.0.0 264 | pydotplus 2.0.2 265 | pyemd 0.5.1 266 | pyglet 1.4.2 267 | Pygments 2.1.3 268 | pygobject 3.26.1 269 | pymc3 3.7 270 | pymongo 3.9.0 271 | pymystem3 0.2.0 272 | PyOpenGL 3.1.0 273 | pyparsing 2.4.2 274 | pyrsistent 0.15.4 275 | pysndfile 1.3.7 276 | PySocks 1.7.0 277 | pystan 2.19.0.0 278 | pytest 3.6.4 279 | python-apt 1.6.4 280 | python-chess 0.23.11 281 | python-dateutil 2.5.3 282 | python-louvain 0.13 283 | python-rtmidi 1.3.0 284 | python-slugify 3.0.3 285 | python-utils 2.3.0 286 | pytz 2018.9 287 | PyWavelets 1.0.3 288 | PyYAML 3.13 289 | pyzmq 17.0.0 290 | qtconsole 4.5.4 291 | requests 2.21.0 292 | requests-oauthlib 1.2.0 293 | resampy 0.2.2 294 | retrying 1.3.3 295 | rpy2 2.9.5 296 | rsa 4.0 297 | s3fs 0.3.3 298 | s3transfer 0.2.1 299 | scikit-image 0.15.0 300 | scikit-learn 0.21.3 301 | scipy 1.3.1 302 | screen-resolution-extra 0.0.0 303 | scs 2.1.1.post2 304 | seaborn 0.9.0 305 | semantic-version 2.6.0 306 | Send2Trash 1.5.0 307 | setuptools 41.2.0 308 | setuptools-git 1.2 309 | Shapely 1.6.4.post2 310 | simplegeneric 0.8.1 311 | six 1.12.0 312 | sklearn 0.0 313 | sklearn-pandas 1.8.0 314 | smart-open 1.8.4 315 | snowballstemmer 1.9.0 316 | sortedcontainers 2.1.0 317 | spacy 2.1.8 318 | Sphinx 1.8.5 319 | sphinxcontrib-websupport 1.1.2 320 | SQLAlchemy 1.3.7 321 | sqlparse 0.3.0 322 | srsly 0.1.0 323 | stable-baselines 2.2.1 324 | statsmodels 0.10.1 325 | sympy 1.1.1 326 | tables 3.4.4 327 | tabulate 0.8.3 328 | tblib 1.4.0 329 | tensor2tensor 1.11.0 330 | tensorboard 1.14.0 331 | tensorboardcolab 0.0.22 332 | tensorflow 1.14.0 333 | tensorflow-estimator 1.14.0 334 | tensorflow-hub 0.5.0 335 | tensorflow-metadata 0.14.0 336 | tensorflow-probability 0.7.0 337 | termcolor 1.1.0 338 | terminado 0.8.2 339 | testpath 0.4.2 340 | text-unidecode 1.2 341 | textblob 0.15.3 342 | textgenrnn 1.4.1 343 | tfds-nightly 1.2.0.dev201908260105 344 | tflearn 0.3.2 345 | Theano 1.0.4 346 | thinc 7.0.8 347 | toolz 0.10.0 348 | torch 1.1.0 349 | torchsummary 1.5.1 350 | torchtext 0.3.1 351 | torchvision 0.3.0 352 | tornado 4.5.3 353 | tqdm 4.28.1 354 | traitlets 4.3.2 355 | tweepy 3.6.0 356 | typing 3.7.4.1 357 | tzlocal 1.5.1 358 | umap-learn 0.3.10 359 | uritemplate 3.0.0 360 | urllib3 1.24.3 361 | vega-datasets 0.7.0 362 | wasabi 0.2.2 363 | wcwidth 0.1.7 364 | webencodings 0.5.1 365 | Werkzeug 0.15.5 366 | wheel 0.33.6 367 | widgetsnbextension 3.5.1 368 | wordcloud 1.5.0 369 | wrapt 1.11.2 370 | xarray 0.11.3 371 | xgboost 0.90 372 | xkit 0.0.0 373 | xlrd 1.1.0 374 | xlwt 1.3.0 375 | yellowbrick 0.9.1 376 | zict 1.0.0 377 | zipp 0.6.0 378 | zmq 0.0.0 379 | -------------------------------------------------------------------------------- /deepLearning_LSTM.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "deepLearning_LSTM.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "toc_visible": true, 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "name": "python3", 14 | "display_name": "Python 3" 15 | }, 16 | "accelerator": "GPU" 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "view-in-github", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "\"Open" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": { 32 | "id": "dTQnbI5ALRnR" 33 | }, 34 | "source": [ 35 | "#Long Short-Term Memory (LSTM)\n", 36 | "\n", 37 | "This example is part of the [*Deep Learning Tutorial*](https://github.com/hfarruda/deeplearningtutorial), authored by Henrique F. de Arruda, Alexandre Benatti, César Comin, and Luciano da Fontoura Costa. This code is not suitable for other data and/or applications, which will require modifications in the structure and parameters. This code has absolutely no warranty.\n", 38 | "\n", 39 | "If you publish a paper related on this material, please cite:\n", 40 | "\n", 41 | "H. F. de Arruda, A. Benatti, C. H. Comin, L. da F. Costa, \"Learning Deep Learning (CDT-15),\" 2019.\n", 42 | "\n", 43 | "This is the third example of deep learning implementation. Here we use a LSTM network to predict the Bitcoin prices along time by using the input as a temporal series.\n", 44 | "\n", 45 | "\n", 46 | "First of all, we import the necessary libraries. Here we opt for using Keras (using TensorFlow backend)." 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "source": [ 52 | "!pip install yfinance" 53 | ], 54 | "metadata": { 55 | "id": "rV_VpYzoxiji" 56 | }, 57 | "execution_count": null, 58 | "outputs": [] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "metadata": { 63 | "id": "Z_tGqT1mKu_4", 64 | "colab": { 65 | "base_uri": "https://localhost:8080/" 66 | }, 67 | "outputId": "f83cedc2-f626-4df5-d333-6a875972154e" 68 | }, 69 | "source": [ 70 | "%tensorflow_version 1.x\n", 71 | "import numpy as np\n", 72 | "import keras\n", 73 | "from keras.models import Sequential\n", 74 | "from keras.utils.vis_utils import plot_model\n", 75 | "from keras.layers import Dense, Dropout, LSTM\n", 76 | "from keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\n", 77 | "from sklearn.model_selection import train_test_split\n", 78 | "from sklearn.metrics import accuracy_score\n", 79 | "from sklearn.preprocessing import MinMaxScaler\n", 80 | "import matplotlib.pyplot as plt\n", 81 | "import pandas as pd\n", 82 | "#import pandas_datareader\n", 83 | "import yfinance as yf" 84 | ], 85 | "execution_count": null, 86 | "outputs": [ 87 | { 88 | "output_type": "stream", 89 | "name": "stdout", 90 | "text": [ 91 | "TensorFlow 1.x selected.\n" 92 | ] 93 | }, 94 | { 95 | "output_type": "stream", 96 | "name": "stderr", 97 | "text": [ 98 | "Using TensorFlow backend.\n" 99 | ] 100 | } 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": { 106 | "id": "DBk2wzXpMb7i" 107 | }, 108 | "source": [ 109 | "If you have a GPU, you can use the following code to allocate processing into it. Otherwise, proceed to (*)." 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "metadata": { 115 | "id": "rBk-5FD3Mf5j", 116 | "colab": { 117 | "base_uri": "https://localhost:8080/" 118 | }, 119 | "outputId": "f079924d-f08a-4123-c63f-914a472e81d4" 120 | }, 121 | "source": [ 122 | "import tensorflow as tf \n", 123 | "from keras import backend as K\n", 124 | "\n", 125 | "print(K.tensorflow_backend._get_available_gpus())\n", 126 | "\n", 127 | "number_of_cpu_cores = 8\n", 128 | "config = tf.ConfigProto(device_count = {'GPU': 1 , 'CPU': number_of_cpu_cores}) \n", 129 | "session = tf.Session(config=config) \n", 130 | "keras.backend.set_session(session)" 131 | ], 132 | "execution_count": null, 133 | "outputs": [ 134 | { 135 | "output_type": "stream", 136 | "name": "stdout", 137 | "text": [ 138 | "['/job:localhost/replica:0/task:0/device:GPU:0']\n" 139 | ] 140 | } 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": { 146 | "id": "SkpEIvMCMhc0" 147 | }, 148 | "source": [ 149 | "(*) Here, we use the Bitcoin daily prices dataset, which is available at\n", 150 | "[yhaoo-stock-market](https://finance.yahoo.com/). The data contains seven columns, organized as follows: date, opening stock price, high daily price, low daily price, closing stock price, the currency volume traded on the day, and the adjusted closing price." 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "metadata": { 156 | "id": "7XsGYIwMb_aI" 157 | }, 158 | "source": [ 159 | "train_size = 1500\n", 160 | "start_date = '2015-01-01'# Bitcoin started on '2010-07-16'\n", 161 | "end_date = '2020-04-01'\n", 162 | "\n", 163 | "\n", 164 | "tickerData = yf.Ticker(\"BTC-USD\")\n", 165 | "dataset = tickerData.history(period='max', interval='1d', start=start_date, end=end_date)\n", 166 | "data_oerder = ['Open','High', 'Low', 'Close', 'Volume']\n", 167 | "dataset = dataset[data_oerder]\n", 168 | "\n", 169 | "\n", 170 | "train_dataset = dataset.iloc[0:train_size, 1::].values\n", 171 | "test_dataset = dataset.iloc[train_size::, 1::].values\n", 172 | "\n", 173 | "min_max_scaler = MinMaxScaler(feature_range=(0,1))\n", 174 | "normalized_train_dataset = min_max_scaler.fit_transform(train_dataset)\n", 175 | "\n", 176 | "min_max_scaler_train = MinMaxScaler(feature_range=(0,1))\n", 177 | "normalized_train_price = min_max_scaler_train.fit_transform(train_dataset[:,0:1])\n" 178 | ], 179 | "execution_count": null, 180 | "outputs": [] 181 | }, 182 | { 183 | "cell_type": "markdown", 184 | "metadata": { 185 | "id": "xndlWJv8Oi0N" 186 | }, 187 | "source": [ 188 | "In the following, we define the network topology." 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "metadata": { 194 | "id": "imLL0-wBOf8o" 195 | }, 196 | "source": [ 197 | "window_size = 50\n", 198 | "number_of_lstm_layers = 3\n", 199 | "activation = 'sigmoid' \n", 200 | "return_sequences = True\n", 201 | "units_first_layer = 100\n", 202 | "units = 50\n", 203 | "\n", 204 | "data = []\n", 205 | "train_price = []\n", 206 | "for i in range(window_size, train_size):\n", 207 | " data.append(normalized_train_dataset[i-window_size:i, 0:5])\n", 208 | " train_price.append(normalized_train_dataset[i, 0])\n", 209 | "data, train_price = np.array(data), np.array(train_price)\n", 210 | "\n", 211 | "lstm_model = Sequential()\n", 212 | "lstm_model.add(LSTM(units = units_first_layer, \n", 213 | " return_sequences = return_sequences, \n", 214 | " input_shape = (data.shape[1], 4)))\n", 215 | "lstm_model.add(Dropout(0.2))\n", 216 | "\n", 217 | "for i in range(number_of_lstm_layers-2):\n", 218 | " lstm_model.add(LSTM(units = units, return_sequences = return_sequences))\n", 219 | " lstm_model.add(Dropout(0.2))\n", 220 | "\n", 221 | "lstm_model.add(LSTM(units = units))\n", 222 | "lstm_model.add(Dropout(0.2))\n", 223 | "\n", 224 | "#Output layer\n", 225 | "lstm_model.add(Dense(units = 1, activation = activation))" 226 | ], 227 | "execution_count": null, 228 | "outputs": [] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": { 233 | "id": "lgj_GKsJVms-" 234 | }, 235 | "source": [ 236 | "In order to check the network topology, the subsequent command can be used." 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "metadata": { 242 | "id": "ZDE5FFHQTD2c", 243 | "outputId": "f87b32e1-9de9-42d6-a30f-8ebb89d00c2c", 244 | "colab": { 245 | "base_uri": "https://localhost:8080/", 246 | "height": 1000 247 | } 248 | }, 249 | "source": [ 250 | "lstm_model.summary() \n", 251 | "#Saving the resultant figure as 'ff_model.png'.\n", 252 | "plot_model(lstm_model, to_file='lstm_model.png', show_shapes=True, \n", 253 | " show_layer_names=True)" 254 | ], 255 | "execution_count": null, 256 | "outputs": [ 257 | { 258 | "output_type": "stream", 259 | "name": "stdout", 260 | "text": [ 261 | "Model: \"sequential_7\"\n", 262 | "_________________________________________________________________\n", 263 | "Layer (type) Output Shape Param # \n", 264 | "=================================================================\n", 265 | "lstm_19 (LSTM) (None, 50, 100) 42000 \n", 266 | "_________________________________________________________________\n", 267 | "dropout_19 (Dropout) (None, 50, 100) 0 \n", 268 | "_________________________________________________________________\n", 269 | "lstm_20 (LSTM) (None, 50, 50) 30200 \n", 270 | "_________________________________________________________________\n", 271 | "dropout_20 (Dropout) (None, 50, 50) 0 \n", 272 | "_________________________________________________________________\n", 273 | "lstm_21 (LSTM) (None, 50) 20200 \n", 274 | "_________________________________________________________________\n", 275 | "dropout_21 (Dropout) (None, 50) 0 \n", 276 | "_________________________________________________________________\n", 277 | "dense_7 (Dense) (None, 1) 51 \n", 278 | "=================================================================\n", 279 | "Total params: 92,451\n", 280 | "Trainable params: 92,451\n", 281 | "Non-trainable params: 0\n", 282 | "_________________________________________________________________\n" 283 | ] 284 | }, 285 | { 286 | "output_type": "execute_result", 287 | "data": { 288 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbwAAANQCAIAAADcy01YAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVwT19o48DMhCUkggSgIlEVZFCriVjdQL+31SqsWFdFCUSv2leLSRhApRgQRccUCL71QP7aUvm/1ggt8xIq0/dBbtFTqtVdRS10QRRBFQMUACRDI/P44v3duyhJmCFmwz/evZmZy5snp8DjLOc8QJEkiAAAA9LAMHQAAAAwnkDQBAIABSJoAAMAAJE0AAGCArf6hrKwsJSXFUKEAAIAR2rJli7e3N/XxD2eatbW1p06d0ntIAAzSqVOnHj58aOgodO6XX3755ZdfDB3Fn9SpU6dqa2vVl7B7b3Ty5El9xQOAVgiCiIyMfOeddwwdiG6tWLECwR+mgRAE0WMJ3NMEAAAGIGkCAAADkDQBAIABSJoAAMAAJE0AAGAAkib40zl37pyFhcU333xj6EB0pbi4WCqV5uXlubi4EARBEMTq1avVN/Dz8xMKhSYmJp6enleuXDFIkElJScQfTZgwQX2D0tLS2bNnCwQCOzu7mJiYjo6OQeylvb3dw8Njx44d+OOZM2cOHDjQ3d2tTeSQNMGfzstd2Wvnzp3p6enbt28PDAy8d++eq6vryJEjjx49WlhYSG3z/fffnzx50t/fv6KiYurUqQaMtj8VFRV+fn7z5s1rbGzMz8//8ssvN2zYMIh2YmNjb9++TX1cvHgxj8ebN29ec3PzoGODpAn+dBYtWvTixQt/f39d70ihUPj4+Oh6L+r279+fm5t74sQJoVBILUxPT2exWOHh4S9evNBnMAP6+uuvSTW//fYbtWr37t22tra7du0yMzPz9vaOiYn56quvbt26xaj9ixcvqreJbd68edKkSQsXLuzq6hpc2JA0AdCVrKyshoYGve3u7t27cXFxu3bt4vF46st9fHwiIiLq6uq2bt2qt2C00dXVVVhY6OvrSw0sX7BgAUmSBQUF9BtRKBTR0dFpaWm9VyUkJJSXl/e5ig5ImuDPpbS01MnJiSCIv//97wihzMxMMzMzgUBQUFCwYMECkUjk4OCQk5ODN05PT+fxeKNGjVq/fr2dnR2Px/Px8bl06RJeK5FIuFyura0t/rhp0yYzMzOCIJqamhBCERERUVFRVVVVBEG4ubkhhL799luRSLRnzx4d/bT09HSSJBcvXtx7VVJS0rhx47744ovi4uI+v0uSZEpKyquvvmpqaioWi5cuXUqd1mnuIoRQd3d3fHy8k5MTn8+fOHHi8ePHtfwh9+7da21tdXJyopa4uroihK5fv06/kdjY2E2bNllbW/deJRaLfX1909LSBnejBpIm+HOZM2fOxYsXqY8bN26MjIxUKBRCofD48eNVVVUuLi5hYWFKpRIhJJFIQkND5XL55s2bq6urr1y50tXVNX/+fDwZOT09XX0GZ0ZGxq5du6iPaWlp/v7+rq6uJEnevXsXIYSfP6hUKh39tMLCQnd3d4FA0HsVn8//6quvWCxWWFhYW1tb7w0SEhKkUmlsbGxDQ8OFCxdqa2vnzp375MkTNFAXIYS2bdt28ODB1NTUx48f+/v7h4SE/Prrr3QClkqlYrGYy+U6OzsvXbr08uXLeHl9fT1CSP0OA4/H4/P5OB46fv7556qqqpCQkP42mDJlSl1d3bVr12g2qA6SJgAIIeTj4yMSiaytrYODg9va2mpqaqhVbDYbn4KNHz8+MzOzpaUlOzt7ELtYtGiRTCaLi4sbuqj/o62t7f79+/iMrE/e3t6RkZHV1dXbtm3rsUqhUKSkpCxbtmzVqlUWFhZeXl6HDx9uamo6cuSI+mZ9dlF7e3tmZmZAQEBgYKClpeWOHTs4HA6d/lmzZs2ZM2dqa2tbW1tzcnJqamp8fX0rKioQQvhBuYmJifr2HA5HoVDQ6QqFQhEREZGZmalhm7FjxyKEbty4QafBHiBpAvAHXC4XIUSdRvUwbdo0gUDA9ImEHjQ0NJAk2edpJiUpKcnd3T0jI6O0tFR9eUVFRWtr67Rp06gl06dP53K51I2IHtS76Pbt23K5nBotxOfzbW1t6fSPo6PjlClTzM3NuVzurFmzsrOzFQpFRkYGQgjfk+3xoKazs5PP5w/YLEJo+/btH3zwgb29vYZtcEfRP3VVB0kTAGZMTU0bGxsNHUVP7e3tCCFTU1MN2/B4vOzsbIIg3n//ffWzNjz+xtzcXH1jS0vLlpaWAfeLL/Z37NhBDbd88OCBXC5nGr+Xl5eJicmdO3cQQvg2sUwmo9bK5fL29nY7O7sB2yktLb1x48a6des0b4bzL+40piBpAsCAUqlsbm52cHAwdCA94Sww4LBtb2/vLVu2VFZW7t69m1poaWmJEOqRImn+TPykJTU1VX3wUFlZGdP4VSqVSqXCSd/Z2VkoFD548IBai28KT5w4ccB2srKyfvjhBxaLhTM4Dm/Pnj0EQajfae3s7ET/12lMQdIEgIGSkhKSJGfNmoU/stns/i7k9WzUqFEEQdAZibl7924PD4+rV69SSyZMmGBubq6eUy5dutTZ2fnaa68N2JqjoyOPxysvL2ca8Jtvvqn+8fLlyyRJ4gLpbDZ74cKFFy5coB6aFRUVEQTR58CAHrKzs9XTN74miI2NJUlS/f4D7igbGxumYSNImgAMSKVSPX/+vKur6/r16xEREU5OTqGhoXiVm5vbs2fPTp8+rVQqGxsb1U+OEEIjRox49OhRdXV1S0uLUqksKirS3ZAjgUDg4uJCp449vkhXf8zC4/GioqLy8/OPHj0qk8lu3LixYcMGOzu78PBwOq2tXbs2JycnMzNTJpN1d3c/fPjw8ePHCKHg4GAbG5v+pmnW1dXl5uY2NzcrlcqysrJ169Y5OTlR037i4uKePHmyc+fOtra2srKy5OTk0NBQd3d3vFZzy3TgjvLy8hrMl9WzMh5gRQIwTCCEjh8/zugrn376Kb5lJhAIFi9enJGRgZ8JjB07tqqq6siRIyKRCCE0evToO3fukCQZHh7O4XDs7e3ZbLZIJFq6dGlVVRXV2tOnT9944w0ej+fs7PzRRx9FR0cjhNzc3GpqakiSvHLlyujRo/l8/pw5c+rr68+dOycUCpOSkpj+zOXLly9fvnzAzSQSCYfDkcvl+GN+fj5+mG5lZfXhhx/22Dg6OnrJkiXUR5VKlZycPHbsWA6HIxaLAwICbt++jVcN2EUdHR0xMTFOTk5sNtva2jowMLCiooIkyYCAAIRQfHx8n9FGRUW5urqamZmx2WwHB4ewsLBHjx6pb3D+/PkZM2aYmpra2dlFR0e3t7dTqzS3rE79TFPdokWL7O3tVSrVgC30PsYgaYJhbBBJk6nw8PARI0bodBcDopk0Kysr2Wx2j7mJBtTd3T137tysrCxja7mpqYnH4x06dIjOxr2PMbg8B2AAWhbF0Rs3N7fExMTExMTW1lZDx4K6u7tPnz7d0tISHBxsbC0nJCRMnjxZIpEM7uuQNAF4eUil0hUrVgQHBxu8NkdJSUleXl5RUZHmoaP6bzklJaW8vPzcuXMcDmdwATBOmocOHcLP6Q4fPjy4XQ4JlUqVmprau4SMUqmMj493cXHhcrn29vZbt26lOYsAGWWZxV9++eXVV1/F4ydsbGySkpL0tmv1aoy2trarVq3S266Nx/bt27Ozs1+8eOHs7Dxc3m69Z88eiUSyb98+w4Yxb968Y8eOURPzjaTlgoKCjo6OkpISsVg8+AjUr9Vp3tOsrKxECH322WeM7iMMoTt37syePRshNGnSpB6rNm7cyOPxcnJyZDLZjz/+KBKJQkJCaDZ79uxZkUh05syZoY5XW3hwxvPnz/W/a1dXVwsLC/3vlyak+3uaxoDmPU2gC72PMV1dnuuukuC1a9e2bdu2YcOGyZMn91h17969w4cPv/fee8HBwUKh8PXXX5dIJP/4xz9u3rxJp+WXuMwiTUYbGADGQ1dJU3eVBCdNmpSXl7dy5creM8YuX76sUqlmzpxJLXnrrbcQQt99950uIhk0PZdZpM9oAwPAeAxB0sTDqQQCgUgk8vLykslkPSoJpqWlmZmZsVis1157zcbGhsPhmJmZTZ06de7cuXg6gaWl5ccffzwEP4bFQn+cGoVrmdA50xwuZRb1GRgdP/300/jx4y0sLHg8npeXF/73ad26dfhmqKurK555snbtWoFAYGFhcebMGdRPBcaDBw8KBAKhUNjQ0BAVFWVvb6/+ogIAjIX6tfog7mm2traKRKIDBw4oFIr6+vply5Y1NjaSJBkYGIgrCWI7d+5ECF26dKmtra2pqQmfABYWFjY2Nra1teFn/+Xl5YzuNcycObPHPU1cozQuLo5aggulBAQE0GkQF0n89NNP8cfY2FiE0A8//PDixYuGhoa5c+eamZl1dnbiteHh4WZmZr///nt7e3tFRcX06dOFQiEe0kyS5MqVK21sbKiWk5OTEUK4Z3p3ztmzZ4VCYWJiYn+B9binqbfASBr3NE+ePJmQkPDs2bOnT5/OmjVr5MiRVFMmJiZ1dXXUliEhIdT94q1bt5qamp46der58+fbt29nsVh4Fh3+aZs3b/7000+XLVt28+ZNDbsm4Z4m0L3ex5i2Z5rV1dUymczT05PH49nY2OTl5VlZWfW38fjx4wUCwciRI999912EkJOTk5WVlUAgwE9mtS+35eXl9dZbb2VkZPzzn/9sb2+vr6/Pz88nCEKb2cFGW2ZRD4HRsXz58p07d4rF4hEjRixevPjp06d4DsaGDRu6u7up/cpkssuXLy9cuBDRqMC4f//+Dz/8MC8vz8PDQ0dhAzBobC2/7+LiMmrUqFWrVm3evDk0NHTMmDF0voXr8VH18vCAqSEpfJCbmxsTE/Pee+89e/bMzs5u5syZJEmOHDlS+5aNtsyi8QSG/z/ioeB//etfx40b9+WXX27fvp0giNzc3ODgYDzfedAVGPsUFBQUFBQ0RL/AqFEvzAGGpW3S5PP5//znP7dt27Znz57ExMR33nknOzt7cAWXhoSFhYX6ANLHjx/n5OS88soreti1cZZZRDoOrLCwMDk5uaKiQiaTqSdugiDWr1+/ZcuWH3744W9/+9v//u//Hjt2DK+iKjBSb6NGCNEpldiniIgIXBrnJZaamooQioyMNHQgf0a9/0nWNmkihDw9Pb/55pvGxsaUlJT9+/d7enrqqKD/IOC3jrzxxhu63pHRllnURWAXLlz497//HRkZWVNTExAQsGzZsi+//PKVV1759NNP1R/ohYaGbt++/YsvvnB0dBSJRKNHj8bLqQqMERER2gfj7e2t/qKel9LJkycRQi/9zzROQ580Hz161NzcPH78eGtr63379n3//fe///67lm0Ooc8//9zZ2dnX11fXOzLaMou6COzf//63mZkZQujGjRtKpXLjxo0uLi6o1/WjWCwOCgrKzc0VCoVhYWHU8kFXYATAGGj7IOjRo0fr16+/detWZ2fn1atXHzx4gP8+e1QSHIpQaZkxY8aDBw+6urqqq6u3bt1aXFyclZWF7/oNOaMtszhUgfVuWalUPnnypKSkBCdN/JLV4uLi9vb2ysrK3q+U2bBhQ0dHx9mzZ9WnDGiowAjAMKD+KJ3OkKNPPvkElzs2MzNbtmxZdXW1j4+PWCw2MTF55ZVXYmNju7q6yD9WEpRKpXhq/ZgxY3766af9+/dbWFgghGxsbI4dO5abm4sbFIvFOTk5A44AKCsrmz17NnULzNbW1sfH5/z583jt/PnzLS0t2Wy2WCxetGgRHshCh3GWWfzll188PT3x+FNbW9s9e/boLbDPPvtMw6sN8/PzcYMxMTEjRoywtLRcsWIFHuLq6upKjXAiSXLKlClSqbTH7+qzAuOBAwfw3XBHR0ea9c0QDDkCOtb7GIN6moNkDGUW+2RsgS1cuPDevXs6ahySJtC13scYlIYbPKMts2jwwKhL++vXr+OzWsPGA8AQMq6keevWLaJ/2lQz1V3LoLeYmJjKyso7d+6sXbtW/a2HQD+Ki4ulUql6cb/Vq1erb+Dn5ycUCk1MTDw9PbV50442kpKSevwZUkN3sdLS0tmzZwsEAjs7u5iYmI6OjkHspb293cPDgxrcdubMmQMHDmh7VqF+2gmX5zRJpVL8cGnMmDEnT540dDj/YSSBxcbGslgsR0dHXdfZQ3B53kt8fLy/v79MJsMfXV1d8eSOs2fPqm9WVFSk/o4g/ev9r6mnpye19rfffuPz+XFxca2trRcvXrSyslq7du0g9rJlyxb0x3cEpaWl+fr60i+02PsYg6QJhjFdJ025XO7t7W3wpugnzX379o0bN06hUFBLXF1djx07xmKx7O3tm5ubqeXGkDQ1PO4LCgpydnamXnyWnJxMEMSAtQh6+Pnnn/38/FCvF6tJJBJvb2+lUkmnkd7HmHFdngNgVIawVp4eyu7dvXs3Li5u165dPB5PfbmPj09ERERdXd3WrVt1GsBQ6erqKiws9PX1pUb+LliwgCTJgoIC+o0oFIro6Oi0tLTeqxISEsrLy/tcRQckTfCSI0kyJSUFVzARi8VLly6l5rkzqpVnwHqANKWnp5MkuXjx4t6rkpKSxo0b98UXXxQXFzPtJc3VCFE/hf60ce/evdbWVjwKGMOj33AZM5piY2M3bdqEp5/1IBaLfX1909LS8IkkY+qnnXB5DoYXROPyPD4+nsvlfv31183NzdevX586daqVlVV9fT1ey6hWnj7rAaqjeXnu4uIyfvz4HgtdXV3v379PkuTFixdZLNaYMWNaW1vJXpfnmntJczXC/gr9abZ7924HBwdLS0sOhzNmzJglS5b861//wqvOnz+PEEpOTlbfns/nz5s3b8BmsdLS0sWLF5P9v/dcKpUihK5evTpgU72PMTjTBC8zhUKRkpKybNmyVatWWVhYeHl5HT58uKmp6ciRI4Nr0LD1ADVoa2u7f/++hvkI3t7ekZGR1dXV27Zt67GKZi/1WY1wwEJ//VmzZs2ZM2dqa2tbW1tzcnJqamp8fX0rKioQQvhBOa6JReFwODRfkqhQKCIiIjIzMzVsg8uT37hxg06DPUDSBC+zioqK1tbWadOmUUumT5/O5XJ7z/gcBAPWA+ytoaGBJEnNb7VNSkpyd3fPyMgoLS1VX860l9SrEQ660J+jo+OUKVPMzc25XO6sWbOys7MVCkVGRgZCCN+TpUpHYp2dnTTLp23fvv2DDz6wt7fXsA3uqCdPntBpsAdImuBl1tzcjBAyNzdXX2hpadnS0jIk7RtPPcD29naEUO8XZ6nj8XjZ2dkEQbz//vvqZ23a9BJV6I8abvngwQO5XM40fi8vLxMTkzt37iCE8K1hmUxGrZXL5e3t7XTqB5aWlt64cWPdunWaN8P5F3caU5A0wcvM0tISIdTjj3+oauUZVT1AnAUGHLbt7e29ZcuWyspK9WGS2vQSVehP/a5fWVkZ0/hVKpVKpcJJ39nZWSgUqleTuXv3LkJo4sSJA7aTlZX1ww8/sFgsnMFxeHv27CEI4tdff6U26+zsRH98nxh9kDTBy2zChAnm5ubqfy2XLl3q7Ox87bXX8EdtauUZVT3AUaNGEQTx4sWLAbfcvXu3h4cHfuEdNmAvaTDoQn/4zVcU/OwI15Nms9kLFy68cOGCSqXCa4uKigiC6HNgQA/Z2dnq6Vv9QZD6/QfcUbhUEFOQNMHLjMfjRUVF5efnHz16VCaT3bhxY8OGDXZ2duHh4XgDprXyjLYeoEAgcHFxefjw4YBb4ot09ccsA/aS5tb6K/QXHBxsY2PT3zTNurq63Nzc5uZmpVJZVla2bt06JyenDRs24LVxcXFPnjzZuXNnW1tbWVlZcnJyaGiou7s7Xqu5ZTpwR3l5eQ3my+pZGYYcgeEF0RhypFKpkpOTx44dy+FwxGJxQEDA7du3qbWMivjppx5gbzSHHEkkEg6HI5fL8cf8/Hz8MN3KyurDDz/ssXF0dLT6kCMNvTRgNcI+C/2RJBkQEIAQio+P7zPaqKgoV1dXMzMzNpvt4OAQFhb26NEj9Q3wu8FNTU3t7Oyio6Pb29upVZpbVtffkKNFixbZ29tTM4406H2MQdIEwxidpDmEDFV2j2bSrKysZLPZNEuR6kF3d/fcuXOzsrKMreWmpiYej3fo0CE6G/c+xuDyHAAGDF52TwM3N7fExMTExMTW1lZDx4K6u7tPnz7d0tIy5CXEtG85ISFh8uTJEolkcF+HpAnAy0Mqla5YsSI4OJjOEyGdKikpycvLKyoq0jx0VP8tp6SklJeXnzt3Dr9xehAgaQJAy/bt27Ozs1+8eOHs7Hzq1ClDh9OvPXv2SCSSffv2GTaMefPmHTt2jJqMbyQtFxQUdHR0lJSUiMXiQQcwBK/wBeDPYO/evXv37jV0FLT4+fnhkmighyVLlixZskTLRuBMEwAAGICkCQAADEDSBAAABiBpAgAAA308CDpx4oT+4wBgcAZRG2LYwXP+4A/TWKiPdNe+Tj0AALxkeswIIsjBvSUDAF165513EJxbAaME9zQBAIABSJoAAMAAJE0AAGAAkiYAADAASRMAABiApAkAAAxA0gQAAAYgaQIAAAOQNAEAgAFImgAAwAAkTQAAYACSJgAAMABJEwAAGICkCQAADEDSBAAABiBpAgAAA5A0AQCAAUiaAADAACRNAABgAJImAAAwAEkTAAAYgKQJAAAMQNIEAAAGIGkCAAADkDQBAIABSJoAAMAAJE0AAGAAkiYAADAASRMAABiApAkAAAxA0gQAAAYgaQIAAAOQNAEAgAGCJElDxwAAOnbsWFZWlkqlwh/v37+PEHJ2dsYfWSzWf/3Xf61cudJg8QHwfyBpAqNw/fr1SZMmadjg2rVrEydO1Fs8APQHkiYwFh4eHrdv3+5zlZubW2VlpZ7jAaBPcE8TGIvVq1dzOJzeyzkcztq1a/UfDwB9gjNNYCzu3bvn5ubW5wFZWVnp5uam/5AA6A3ONIGxcHFxmTp1KkEQ6gsJgpg2bRpkTGA8IGkCI/Lee++ZmJioLzExMXnvvfcMFQ8AvcHlOTAiDQ0NdnZ21MAjhBCLxXr06JGNjY0BowJAHZxpAiMyatQoX19f6mTTxMTk9ddfh4wJjAokTWBcVq9erX71s3r1agMGA0BvcHkOjItMJrO2tu7s7EQIcTichoYGS0tLQwcFwH/AmSYwLiKR6K233mKz2Ww2e+HChZAxgbGBpAmMzqpVq7q7u7u7u2GyOTBCcHkOjE57e7uVlRVJkk1NTXw+39DhAPAHek2aPcYtAwDAkNBnHmPrbU9YRESEt7e3nncKtFFWVpaWlnb8+HF97rS8vJwgCM11j4ZcUFAQHJ/DDj4+9blHfZ9pHj9+/J133tHbHoH2Tpw4ERQUpOfbOF1dXQghNluv/6jD8Tkc6f/41PeZJgB06DldAkAfPD0HAAAGIGkCAAADkDQBAIABSJoAAMAAJE2gE+fOnbOwsPjmm28MHYhhFBcXS6XSvLw8FxcXgiAIguhRecTPz08oFJqYmHh6el65csUgQSYlJRF/NGHCBPUNSktLZ8+eLRAI7OzsYmJiOjo6BrGX9vZ2Dw+PHTt24I9nzpw5cOBAd3f3EPwAA4GkCXTizzzTbOfOnenp6du3bw8MDLx3756rq+vIkSOPHj1aWFhIbfP999+fPHnS39+/oqJi6tSpBoy2PxUVFX5+fvPmzWtsbMzPz//yyy83bNgwiHZiY2PV35e3ePFiHo83b9685ubmoQtWryBpAp1YtGjRixcv/P39db0jhULh4+Oj673Qt3///tzc3BMnTgiFQmpheno6i8UKDw9/8eKFAWPr7euvvybV/Pbbb9Sq3bt329ra7tq1y8zMzNvbOyYm5quvvrp16xaj9i9evKjeJrZ58+ZJkyYtXLgQj8YddiBpguEtKyuroaHB0FH8f3fv3o2Li9u1axePx1Nf7uPjExERUVdXt3XrVkPFxkhXV1dhYaGvry819XnBggUkSRYUFNBvRKFQREdH9zldJyEhoby8XM8zeYYKJE0w9EpLS52cnAiC+Pvf/44QyszMNDMzEwgEBQUFCxYsEIlEDg4OOTk5eOP09HQejzdq1Kj169fb2dnxeDwfH59Lly7htRKJhMvl2tra4o+bNm0yMzMjCKKpqQkhFBERERUVVVVVRRAEfvnat99+KxKJ9uzZY4CfjVB6ejpJkosXL+69Kikpady4cV988UVxcXGf3yVJMiUl5dVXXzU1NRWLxUuXLqVO6zR3IEKou7s7Pj7eycmJz+dPnDhR+zmv9+7da21tdXJyopa4uroihK5fv06/kdjY2E2bNllbW/deJRaLfX1909LShuNtHEiaYOjNmTPn4sWL1MeNGzdGRkYqFAqhUHj8+PGqqioXF5ewsDClUokQkkgkoaGhcrl88+bN1dXVV65c6erqmj9/fm1tLUIoPT1dfV5jRkbGrl27qI9paWn+/v6urq4kSd69exchhJ8wqL9lSJ8KCwvd3d0FAkHvVXw+/6uvvmKxWGFhYW1tbb03SEhIkEqlsbGxDQ0NFy5cqK2tnTt37pMnT9BAHYgQ2rZt28GDB1NTUx8/fuzv7x8SEvLrr7/SCVgqlYrFYi6X6+zsvHTp0suXL+Pl9fX1CCH1Oww8Ho/P5+N46Pj555+rqqpCQkL622DKlCl1dXXXrl2j2aDxgKQJ9MfHx0ckEllbWwcHB7e1tdXU1FCr2Gw2PskaP358ZmZmS0tLdnb2IHaxaNEimUwWFxc3dFHT1dbWdv/+fXxG1idvb+/IyMjq6upt27b1WKVQKFJSUpYtW7Zq1SoLCwsvL6/Dhw83NTUdOXJEfbM+O7C9vT0zMzMgICAwMNDS0nLHjh0cDodO761Zs+bMmTO1tbWtra05OTk1NTW+vr4VFRUIIfygvMebQTkcjkKhoNMVCoUiIiIiMzNTwzZjx45FCN24cYNOg0YFkiYwAC6XixCiTpR6mDZtmkAgYPrMweAaGhpIkuzzNK3JO0YAACAASURBVJOSlJTk7u6ekZFRWlqqvryioqK1tXXatGnUkunTp3O5XOo2RQ/qHXj79m25XE6NFuLz+ba2tnR6z9HRccqUKebm5lwud9asWdnZ2QqFIiMjAyGE78n2eFDT2dlJs7zp9u3bP/jgA3t7ew3b4I6if+pqPCBpAmNkamra2Nho6CiYaW9vRwiZmppq2IbH42VnZxME8f7776ufteHxN+bm5uobW1patrS0DLhffLG/Y8cOarjlgwcP5HI50/i9vLxMTEzu3LmDEMI3kWUyGbVWLpe3t7fb2dkN2E5paemNGzfWrVuneTOcf3GnDS+QNIHRUSqVzc3NDg4Ohg6EGZwFBhy27e3tvWXLlsrKyt27d1ML8auQeqRImp2An7SkpqaqDx4qKytjGr9KpVKpVDjpOzs7C4XCBw8eUGvxLeOJEycO2E5WVtYPP/zAYrFwBsfh7dmzhyAI9Tut+N15w7EyPyRNYHRKSkpIkpw1axb+yGaz+7uQNyqjRo0iCILOSMzdu3d7eHhcvXqVWjJhwgRzc3P1nHLp0qXOzs7XXnttwNYcHR15PF55eTnTgN988031j5cvXyZJEtdgxm+1u3DhAvVIraioiCCIPgcG9JCdna2evvEVQ2xsLEmS6vcfcEcNx5faQ9IERkGlUj1//ryrq+v69esRERFOTk6hoaF4lZub27Nnz06fPq1UKhsbG9VPfxBCI0aMePToUXV1dUtLi1KpLCoqMtSQI4FA4OLi8vDhwwG3xBfp6o9ZeDxeVFRUfn7+0aNHZTLZjRs3NmzYYGdnFx4eTqe1tWvX5uTkZGZmymSy7u7uhw8fPn78GCEUHBxsY2PT3zTNurq63Nzc5uZmpVJZVla2bt06JycnatpPXFzckydPdu7c2dbWVlZWlpycHBoa6u7ujtdqbpkO3FFeXl6DbsFgSD1CCB0/flyfewTaw4P+GH3l008/xTfFBALB4sWLMzIy8F3/sWPHVlVVHTlyRCQSIYRGjx59584dkiTDw8M5HI69vT2bzRaJREuXLq2qqqJae/r06RtvvMHj8ZydnT/66KPo6GiEkJubW01NDUmSV65cGT16NJ/PnzNnTn19/blz54RCYVJS0iB+qfbHp0Qi4XA4crkcf8zPz8cP062srD788MMeG0dHRy9ZsoT6qFKpkpOTx44dy+FwxGJxQEDA7du38aoBO7CjoyMmJsbJyYnNZltbWwcGBlZUVJAkGRAQgBCKj4/vM9qoqChXV1czMzM2m+3g4BAWFvbo0SP1Dc6fPz9jxgxTU1M7O7vo6Oj29nZqleaW1amfaapbtGiRvb29SqUasAXNBnF8agmSJhiAHg7K8PDwESNG6HQXdGh/fFZWVrLZ7B5zEw2ou7t77ty5WVlZxtZyU1MTj8c7dOiQ9pHoP2nC5TkwCsO67A3Fzc0tMTExMTGxtbXV0LGg7u7u06dPt7S0BAcHG1vLCQkJkydPlkgkQxuYfhhX0jx06BC+m3748GEDhqFSqVJTU3uXgVAqlfHx8S4uLlwu197efuvWrTTH+qqXCLO1tV21alV/W167di04ONjZ2dnU1NTKymrSpElJSUl4VXBwMKHR2bNn1XfU3wDvlJQUgiBYLJaHh8eFCxfodQmgSyqVrlixIjg42OC1OUpKSvLy8oqKijQPHdV/yykpKeXl5efOneNwOEMbmJ7o87QW0bj8qaysRAh99tln+gmptzt37syePRshNGnSpB6rNm7cyOPxcnJyZDLZjz/+KBKJQkJC6Lfs6upqYWGhYYPr168LBILNmzffv39foVDcvn37448/njdvHl4bFBT0/fff49v2+Db/4sWLOzs729raGhoawsLCvvnmG2pHCCFbW9vOzs4eu+jq6ho9ejRCiGp2QLq+/JFKpXio9pgxY06ePKm7HQ2IzvFJ03fffRcTEzMkTb1kTp8+vXfv3q6urqFqEC7PadFdNbBr165t27Ztw4YNkydP7rHq3r17hw8ffu+994KDg4VC4euvvy6RSP7xj3/cvHlzqPZ+6NAhS0vLtLS0MWPG8Hi8cePG7d69mxrIRhDE7NmzLSwsqDc1EgTB4XAEAoG1tXWPsSmvvfZafX396dOne+wiLy9P8zwN/du7d29HRwdJkvfv31++fLmhwxkafn5++/fvN3QUxmjJkiVSqbTHBM3hZVgmTd1VA5s0aVJeXt7KlSt7z+u4fPmySqWaOXMmteStt95CCH333XdDtfenT5++ePHi2bNn1BIul0sVP8/JydFwNRQeHv72229THzdu3IgQ+uyzz3pslpKSEhUVNVQBA/AnZOxJEw96EAgEIpHIy8tLJpP1qAaWlpZmZmbGYrFee+01GxsbDodjZmY2derUuXPn4kG/lpaWH3/8sfaRsFgs9McJDLjiAHWmqX1RsunTp7e1tf31r3/9+eeftQsW/fWvf3311Vd//PFH9aLZP//8s1wu9/Pz07JxAP7MjDpptrW1LV68ePny5c+ePausrBw3blxnZ2ePamARERHR0dEkSX722Wf379+vr6//y1/+cvXqValUevXq1WfPnq1ZsyY5OVn7ClQeHh5ILUUihEaOHIkQoqZIa1+U7OOPP542bdq1a9fmzJnj6el58OBB9bNOptavX48QUn+k9sknn2zZsmXQDQIAkJEnzerqaplM5unpyePxbGxs8vLyrKys+tt4/PjxAoFg5MiR7777LkLIycnJyspKIBDgR9Xal8zx8vJ66623MjIy/vnPf7a3t9fX1+fn5xMEQc3w074oGZ/Pv3jx4n//9397eHj8/vvvMTExr7766vnz5wfX2po1a8zMzP7nf/4HP+K/d+/e5cuXNdQ3BADQwTZ0AJq4uLiMGjVq1apVmzdvDg0NHTNmDJ1v4UexVFUrPKxhSCYv5+bmxsTEvPfee8+ePbOzs5s5cyZJkvh8c6hwOByJRCKRSC5durR///7Tp0+vWLHi9u3bYrGYaVMWFhYhISGff/55bm7u2rVrU1NTN27cyOVycaEEpk6cODGIbw07g6hzAQzLAP/L9PmoHjEfcvTbb7+9/fbbbDabIIigoCA8QS0wMBBfnmM7d+5ECLW0tOCP+DUAV69exR9xWQSmkzRmzpzZe8hRD48ePUIISaVSmm0OOOSoNzwROC8vr8dyPORIfRJejx3dv3+f/L/fPmPGjOfPn9vY2Dx79owkSVxKh+mQIwCMGaM/Ky0Z9eU5QsjT0/Obb7559OhRTEzM8ePHDx06ZOiI/gO/G+CNN97QppELFy6kpqbi/w4MDOxR9hW/LHsQtRGxyZMnz5o161//+ld4ePiKFSsGcbpK0edBaSgIpvkOQ/r/R92ok+ajR49+//13hJC1tfW+ffumTp2KPxqJzz//3NnZ2dfXV5tG/v3vf5uZmeH/7ujo6PED8bNvOkUM+4PHHp06dSoyMlKLMAEA/5+xJ83169ffunWrs7Pz6tWrDx48wDUWe1QD01s8M2bMePDgQVdXV3V19datW4uLi7OysvAtVIQQ06JkSqXyyZMnJSUlVNJECAUEBJw4caK5ufnFixcFBQXbtm1bsmSJNknznXfesbKyCggIcHFxGXQjAID/0OeJNBro8ueTTz7BRUnNzMyWLVtWXV3t4+MjFotNTExeeeWV2NhYPPtKvRqYVCrFQ77HjBnz008/7d+/38LCAiFkY2Nz7Nix3Nxc3KBYLM7JyRkwwrKystmzZ1M1/W1tbX18fM6fP4/Xzp8/39LSks1mi8XiRYsW4aKtFA1FyagSYX3Kz8/Hm33//fdBQUGurq6mpqZcLtfd3T0hIUG9HhdJkjKZ7C9/+cuIESMQQiwWy83Nbc+ePb13pF6L7OOPP7548SL+7x07duCibSwWa/z48T/99NOAfaL/aWqGMuDxCYyQ/o9PgtTje4cJgjh+/Lj6G1mB8Ttx4kRQUJA+jxNDgeNzONL/8WnUl+cAAGBs/kRJ89atWxrqqg15zUEAwEvpT5Q0PTw8NNynyM3NNXSAYDgpLi6WSqXqBUzx+DCKn5+fUCg0MTHx9PTU5l062khKSupxckC9Hh0rLS2dPXu2QCCws7OLiYnp6Ohg1H5/lWcHbLm/tWfOnDlw4ICxV6TWz61TDMGN9mEIHgT1Fh8f7+/vL5PJ8EdXV1c8Mezs2bPqmxUVFfU3AUE/1N8SjHl6elJrf/vtNz6fHxcX19raevHiRSsrq7Vr19JvXEPlWc0ta16blpbm6+v7/PlzmmHAO4KA0dHDQSmXy729vQ3eFM3jc9++fePGjVMoFNQSV1fXY8eOsVgse3v75uZmarkxJE0Nc+GCgoKcnZ2pV5slJycTBHHz5k06LZeXly9btuzo0aOTJ0/unTQ1tzzgfiUSibe3t1KppBMJFCEGf0ZDWCBVd7VWsbt378bFxe3atYvH46kv9/HxiYiIqKur27p1q+72PoS6uroKCwt9fX0JgsBLFixYQJJkQUEBna9rqDyruWU6+01ISCgvL09LS9PyN+oIJE0wNEiSTElJefXVV01NTcVi8dKlS6nKUhKJhMvl4vGhCKFNmzaZmZkRBNHU1IQQ6lEgNT09ncfjjRo1av369XZ2djwez8fH59KlS4NoCg1FkdMe0tPTSZJcvHhx71VJSUnjxo374osviouLmXZRZmammZmZQCAoKChYsGCBSCRycHDAVRSw7u7u+Ph4JycnPp8/ceJE7ecO3rt3r7W11cnJiVqCR/hev35dpy3T2a9YLPb19U1LSyONcqAbJE0wNBISEqRSaWxsbENDw4ULF2pra+fOnfvkyROEUHp6uvrgx4yMjF27dlEfexRIlUgkoaGhcrl88+bN1dXVV65c6erqmj9/fm1tLdOm0FAUOe2hsLDQ3d29zxL6fD7/q6++YrFYYWFhbW1tvTfQ0EUbN26MjIxUKBRCofD48eNVVVUuLi5hYWHUhLdt27YdPHgwNTX18ePH/v7+ISEhv/76K52ApVKpWCzmcrnOzs5Lly7FBRMQQvX19QghoVBIbcnj8fh8Po5HG5pbprnfKVOm1NXVaV8GVxcgaYIhoFAoUlJSli1btmrVKgsLCy8vr8OHDzc1NR05cmRwDbLZbHxGNn78+MzMzJaWluzs7EG0o32RU3VtbW3379/XMLnL29s7MjKyurp627ZtPVbR7CIfHx+RSGRtbR0cHNzW1lZTU4MQam9vz8zMDAgICAwMtLS03LFjB4fDodMha9asOXPmTG1tbWtra05OTk1Nja+vb0VFBUIIP7Du8a4eDodD8wWrGmhumeZ+8WsRbty4oWUwugBJEwyBioqK1tbWadOmUUumT5/O5XKpy2ptTJs2TSAQaF9GWnsNDQ0kSWp+b21SUpK7u3tGRkZpaan6cqZdhGsa4DPN27dvy+VyarQQn8+3tbWl0yGOjo5TpkwxNzfncrmzZs3Kzs5WKBQZGRkIIXxPtkdVrc7OTvUXugyO5pZp7hd3svanvboASRMMgebmZoSQubm5+kJLS0tcu1N7pqam1GtFDKi9vR0Ho2EbHo+XnZ1NEMT777+vfvakTRfhi/0dO3ZQwy0fPHgwiIKBXl5eJiYmd+7cQQjh+8IymYxaK5fL29vbqcILg6a5ZZr7xTkUd7ixgaQJhoClpSVCqMfff3Nzs4ODg/aNK5XKoWpKS/gvecCh197e3lu2bKmsrFQfJqlNF1lbWyOEUlNT1Qe+DKJiuUqlUqlUOOk7OzsLhcIHDx5Qa/FdYG1KamGaW6a5X/x+Ae1Pe3UBkiYYAhMmTDA3N1d/NHHp0qXOzk7qbexsNnvQRfxKSkpIksRVAbVsSkujRo0iCOLFixcDbrl7924PDw9cOR8bsIs0wO9VLS8vZxrwm2++qf4R1+Xy9vZGCLHZ7IULF164cIF6SlZUVEQQRJ8DAxjR3DLN/eJOxiXKjA0kTTAEeDxeVFRUfn7+0aNHZTLZjRs3NmzYYGdnFx4ejjdwc3N79uzZ6dOnlUplY2Oj+okG6qtAqkqlev78eVdX1/Xr1yMiIpycnEJDQwfRFNMip5oJBAIXF5eHDx/S6ZDs7Gz1xx0DdpHm1tauXZuTk5OZmSmTybq7ux8+fIhfeRIcHGxjY9PfNM26urrc3Nzm5malUllWVrZu3TonJyf8DhWEUFxc3JMnT3bu3NnW1lZWVpacnBwaGuru7o7Xam5ZM80ta16L4U728vIaxN51Tp8j6RHMCBqGaM64UKlUycnJY8eO5XA4YrE4ICDg9u3b1NqnT5++8cYbPB7P2dn5o48+io6ORgi5ubnV1NSQfyyQWl9fHx4ezuFw7O3t2Wy2SCRaunRpVVXV4JrSUOS0NzrHp0Qi4XA4+F1VZD8FTCnR0dHqM4I0dFFGRgZ+9DF27NiqqqojR46IRCKE0OjRo+/cuUOSZEdHR0xMjJOTE5vNtra2DgwMrKioIEkyICAAIRQfH99ntFFRUa6urmZmZmw228HBISws7NGjR+obnD9/fsaMGaampnZ2dtHR0erFWzW3rLnyrOaWB1xLkuSiRYvs7e2pWUMawDRKYHT0f1CGh4ePGDFCn3vE6ByflZWVbDab6Xv6dKe7u3vu3LlZWVnDqOUBNTU18Xi8Q4cO0dkYplECgBCNhy2G4ubmlpiYmJiY2NraauhYUHd39+nTp1taWoa8sKHuWqYjISFh8uTJEolE/7umA5ImAMxIpdIVK1YEBwfTeSKkUyUlJXl5eUVFRZqHjhpVywNKSUkpLy8/d+4ch8PR865pgqQJjMv27duzs7NfvHjh7Ox86tQpQ4fTtz179kgkkn379hk2jHnz5h07doyaiT8sWtasoKCgo6OjpKREm9dN6xrb0AEA8Ad79+7du3evoaMYmJ+fn5+fn6GjeNksWbJkyZIlho5iAHCmCQAADEDSBAAABiBpAgAAA5A0AQCAAYLUY21kgiBmzZplDJUXAH0PHz785Zdfli9fbuhAdO7UqVNwfA47+PjUax7T585WrFiht32BYQ2XupgyZYqhAwHDw8mTJ/W2L70mTQBowu+0OHHihKEDAaAnuKcJAAAMQNIEAAAGIGkCAAADkDQBAIABSJoAAMAAJE0AAGAAkiYAADAASRMAABiApAkAAAxA0gQAAAYgaQIAAAOQNAEAgAFImgAAwAAkTQAAYACSJgAAMABJEwAAGICkCQAADEDSBAAABiBpAgAAA5A0AQCAAUiaAADAACRNAABgAJImAAAwAEkTAAAYgKQJAAAMQNIEAAAGIGkCAAADkDQBAIABSJoAAMAAJE0AAGAAkiYAADAASRMAABhgGzoAABBCSC6Xd3R0UB87OzsRQs+fP6eWmJqaCgQCA0QGwB8RJEkaOgYAUGZm5qZNmzRskJGRsXHjRr3FA0B/IGkCo9DY2GhnZ9fd3d3nWhMTk8ePH1tbW+s5KgB6g3uawChYW1vPmzfPxMSk9yoTE5O//e1vkDGBkYCkCYzFqlWr+rzuIUly1apV+o8HgD7B5TkwFi0tLdbW1uqPgzAul9vY2CgSiQwSFQA9wJkmMBZCodDf35/D4agvZLPZS5YsgYwJjAckTWBEVq5c2dXVpb6ku7t75cqVhooHgN7g8hwYkc7OTisrq5aWFmqJubl5U1OTqampAaMCQB2caQIjwuVyV6xYweVy8UcOhxMUFAQZExgVSJrAuISEhODpQAghpVIZEhJi2HgA6AEuz4FxUalUtra2jY2NCCErK6v6+vo+B28CYChwpgmMC4vFCgkJ4XK5HA5n5cqVkDGBsYGkCYzOu+++29nZCdfmwDgxq3JUVlZWW1uro1AAwEiSHDlyJELo/v371dXVhg4HvOQcHR29vb0ZfIFkYvny5TqLHAAADGD58uWM0iDjeprLly8/efKkLkIHL6UVK1YghJgeM7///jtCaPz48TqJSQdOnDgRFBQEj1WHHXx8MgJFiIExGkbpEvzZwIMgAABgAJImAAAwAEkTAAAYgKQJAAAMQNIEAAAGIGkCY3Tu3DkLC4tvvvnG0IHoSnFxsVQqzcvLc3FxIQiCIIjVq1erb+Dn5ycUCk1MTDw9Pa9cuWKQIJOSkog/mjBhgvoGpaWls2fPFggEdnZ2MTExvavua6ZSqVJTU318fHqv0txyf2vPnDlz4MCB/l7PN1QgaQJj9HIPeNy5c2d6evr27dsDAwPv3bvn6uo6cuTIo0ePFhYWUtt8//33J0+e9Pf3r6iomDp1qgGj7U9FRYWfn9+8efMaGxvz8/O//PLLDRs20P96ZWXlX/7yly1btsjlckYta1i7ePFiHo83b9685uZm7X9gv5jOCGI6eh78yRn5MSOXy729vbVv5/jx4zT/mvbt2zdu3DiFQkEtcXV1PXbsGIvFsre3b25uppYXFRUtWbJE+9gGbffu3V9//XV/a4OCgpydnVUqFf6YnJxMEMTNmzfptFxeXr5s2bKjR49Onjx50qRJjFoecL8SicTb21upVNKJZBDHJ5xpgj+1rKyshoYGve3u7t27cXFxu3bt4vF46st9fHwiIiLq6uq2bt2qt2C00dXVVVhY6OvrSxAEXrJgwQKSJAsKCuh8fdKkSXl5eStXruxdYVpzy3T2m5CQUF5enpaWpuVv7A8kTWB0SktLnZycCIL4+9//jhDKzMw0MzMTCAQFBQULFiwQiUQODg45OTl44/T0dB6PN2rUqPXr19vZ2fF4PB8fn0uXLuG1EomEy+Xa2trij5s2bTIzMyMIoqmpCSEUERERFRVVVVVFEISbmxtC6NtvvxWJRHv27NHRT0tPTydJcvHixb1XJSUljRs37osvviguLu7zuyRJpqSkvPrqq6ampmKxeOnSpbdu3cKrNHcRQqi7uzs+Pt7JyYnP50+cOBGfF2vj3r17ra2tTk5O1BJXV1eE0PXr13XaMp39isViX1/ftLQ0Ujc3eSBpAqMzZ86cixcvUh83btwYGRmpUCiEQuHx48erqqpcXFzCwsKUSiVCSCKRhIaGyuXyzZs3V1dXX7lypaura/78+bgcV3p6+jvvvEM1lZGRsWvXLupjWlqav7+/q6srSZJ3795FCOFnCCqVSkc/rbCw0N3dXSAQ9F7F5/O/+uorFosVFhbW1tbWe4OEhASpVBobG9vQ0HDhwoXa2tq5c+c+efIEDdRFCKFt27YdPHgwNTX18ePH/v7+ISEhv/76K52ApVKpWCzmcrnOzs5Lly69fPkyXl5fX48QEgqF1JY8Ho/P5+N4tKG5ZZr7nTJlSl1d3bVr17QMpk+QNMGw4ePjIxKJrK2tg4OD29raampqqFVsNhufgo0fPz4zM7OlpSU7O3sQu1i0aJFMJouLixu6qP+jra3t/v37+MyoT97e3pGRkdXV1du2beuxSqFQpKSkLFu2bNWqVRYWFl5eXocPH25qajpy5Ij6Zn12UXt7e2ZmZkBAQGBgoKWl5Y4dOzgcDp3+WbNmzZkzZ2pra1tbW3Nycmpqanx9fSsqKhBC+IF1jxLRHA5HoVDQ7o++aW6Z5n7Hjh2LELpx44aWwfQJkiYYfvCb16jTqB6mTZsmEAioS1fj0dDQQJJkn6eZlKSkJHd394yMjNLSUvXlFRUVra2t06ZNo5ZMnz6dy+VSNyJ6UO+i27dvy+VyarQQn8+3tbWl0z+Ojo5TpkwxNzfncrmzZs3Kzs5WKBQZGRkIIXxPtsf7ljs7O/l8/oDNaqa5ZZr7xZ2s/WlvnyBpgpeQqakpfsuQUWlvb0cIaX65Jo/Hy87OJgji/fffVz97wmNozM3N1Te2tLRUf91xf/DF/o4dO6jhlg8ePOg90GdAXl5eJiYmd+7cQQjh28QymYxaK5fL29vb7ezsmDbbg+aWae4X51Dc4UMOkiZ42SiVyubmZgcHB0MH0hP+Sx5w6LW3t/eWLVsqKyt3795NLbS0tEQI9UiRNH+mtbU1Qig1NVV93ExZWRnT+FUqlUqlwknf2dlZKBQ+ePCAWotvCk+cOJFpsz1obpnmfvELTbU/7e0TJE3wsikpKSFJctasWfgjm83u70Jez0aNGkUQxIsXLwbccvfu3R4eHlevXqWWTJgwwdzcXP3pzaVLlzo7O1977bUBW3N0dOTxeOXl5UwDfvPNN9U/Xr58mSRJ/GYINpu9cOHCCxcuUA/NioqKCILoc2AAI5pbprlf3Mk2NjZaBtMnSJrgZaBSqZ4/f97V1XX9+vWIiAgnJ6fQ0FC8ys3N7dmzZ6dPn1YqlY2NjeonKQihESNGPHr0qLq6uqWlRalUFhUV6W7IkUAgcHFxefjw4YBb4ot09ccdPB4vKioqPz//6NGjMpnsxo0bGzZssLOzCw8Pp9Pa2rVrc3JyMjMzZTJZd3f3w4cPHz9+jBAKDg62sbHpb5pmXV1dbm5uc3OzUqksKytbt26dk5MTNf0mLi7uyZMnO3fubGtrKysrS05ODg0NdXd3x2s1t6yZ5pY1r8VwJ3t5eQ1i7wNjNBTeyGd3ACM0iGPm008/xbeuBALB4sWLMzIy8H39sWPHVlVVHTlyRCQSIYRGjx59584dkiTDw8M5HI69vT2bzRaJREuXLq2qqqJae/r06RtvvMHj8ZydnT/66KPo6GiEkJubW01NDUmSV65cGT16NJ/PnzNnTn19/blz54RCYVJSEtOfSXNGkEQi4XA4crkcf8zPz8cP062srD788MMeG0dHR6vPCFKpVMnJyWPHjuVwOGKxOCAg4Pbt23jVgF3U0dERExPj5OTEZrOtra0DAwMrKipIkgwICEAIxcfH9xltVFSUq6urmZkZm812cHAICwt79OiR+gbnz5+fMWOGqampnZ1ddHR0e3s7tUpzy2VlZbNnz6ZuRNra2vr4+Jw/f55OywOuJUly0aJF9vb21KwhDQZxfELSBLqlh2MmPDx8xIgROt3FgGgmzcrKSjabrWFuop51d3fPnTs3KytrGLU8oKamJh6Pd+jQITobwzRK8CeloYPmAgAAIABJREFU68I2Q8XNzS0xMTExMbG1tdXQsaDu7u7Tp0+3tLQEBwcPl5bpSEhImDx5skQi0VH7Ok+a69atEwqFBEEM4j60EeqvmJVSqYyPj3dxceFyufb29lu3bqU5yle9OBjG5XJHjRr1+uuvJycnP3/+XAc/AhiSVCpdsWJFcHAwnSdCOlVSUpKXl1dUVKR56KhRtTyglJSU8vLyc+fOcTgcXe2D0Xnp4C618BzYq1evMv2isblz587s2bMRQr3rsmzcuJHH4+Xk5Mhksh9//FEkEoWEhNBv2dXV1cLCgiRJ/EDjxx9/DA0NJQjCzs4OP7IcvnR9eS6VSvFA7jFjxpw8eVJ3O9KMfpUj7LvvvouJidFdPH9Op0+f3rt3b1dXF/2vGOk9TaNNmozKgmkoZlVVVcVisT744ANqyY4dOxBCv//+O83GqaSp7uTJkywWa9SoUerlwowE/a77k9wHZ5o0gZEw0nuaVBEnY8OoLJiGYlaXL19WqVQzZ86klrz11lsIoe+++06b8JYvXx4aGtrQ0HD48GFt2tEFPVdUA8B46CRpkiSZnJzs7u5uampqYWGBB3lgBw8eFAgEQqGwoaEhKirK3t4eD5vor+aV5sJfSGO9LKZlwQaNxWKhP04/wPUCbt68iT8OuuAYHmxYVFSEXtKuA2D40cWpbGxsLEEQn3zyyfPnz+VyOZ7hT12ex8bGIoQ2b9786aefLlu27ObNm/Hx8Vwu9+uvv25ubr5+/frUqVOtrKzq6+vx9uHh4WZmZr///nt7e3tFRcX06dOFQiEeZEeSpObvrly50sbGhgosOTkZIdTY2Ig/BgYG4rJgjMycObPH5Tmu5RcXF0ctwQUFAgIC8MezZ88KhcLExMT+2uzz8pwkSTzH1tHREX8cjl0Hl+fAmBnFPU25XC4QCObPn08t6XFPE//lU+X+5XK5ubl5cHAwtf2//vUvhBCVYsLDw9UTCq7ot2vXLjrf1U/SJEnyrbfeGjFixA8//KBQKB4/fnzixAmCIN5++22abfaXNEmSJAjC0tIS//dw7DpImsCYDeL4ZA/5qevdu3flcvm8efNobs+05pV64S+m39Wd3NzcmJiY995779mzZ3Z2djNnziRJcuTIkVo229bWRpIknt3R23Dpul9++WXFihVD3qxRwfP2Xvqf+fL55ZdfqDIFNA190sRHDy6sQscgal5Rhb+0qZc1tCwsLNQf1zx+/DgnJ+eVV17RsllchsvDw6PPtS9H1wEwvAx90sRVQum/AZlpzSv1wl/a1MvSKXwh/MYbb2jZzrfffosQWrBgQZ9rh0vXzZo16+TJk0PerFE5ceJEUFDQS/8zXz6DuDgY+qfnEyZMYLFY58+fp789o5pX6oW/BvyuocqCff75587Ozr6+vto0Ul9fn5qa6uDg8P777/e5wUvZdQAYuaFPmriGyqlTp7KysmQy2fXr13u8xqQHOjWv+iv8NeB3GZUF0+ZXz5gx48GDB11dXdXV1Vu3bi0uLs7KysIzVRBCdAqOkSTZ2tqK67I0NjYeP3589uzZJiYmp0+f7u+e5svRdQAMM7p40tTS0rJu3bqRI0eam5vPmTMnPj4eIeTg4HDt2rUDBw7g8YyOjo5UrRcNNa/IgQp/af4uo7Jgmn+U5mJW8+fPt7S0ZLPZYrF40aJFPeY+aig4dubMmYkTJwoEAi6Xi8d74sflM2bMSExMfPr0KbXlMO06eHoOjNkgjk+CZPJqYHz9r+cbN+vXrz958uTTp0/1udOXgzF0nUGOGf3D9zQZ/TUBYzCI43N4lIYbLoW/jBB0HQBDa3gkTV27desW0T+D1AQEL7fi4mKpVKpeGHD16tXqG/j5+QmFQhMTE09Pz8G9NEJ7SUlJPf4WqPcAY6WlpbNnzxYIBHZ2djExMfTHzGD9FVocsOX+1p45c+bAgQM6P1HQ9fW/loyk8NdwZCRdB/c0e4uPj/f395fJZPijq6srngdx9uxZ9c2KiorUX3ehf+qvw8Q8PT2ptb/99hufz4+Li2ttbb148aKVldXatWvpN66h0KLmljWvTUtL8/X1ff78Oc0wjGIaJQDq9HDMMCrxp6Om6CfNffv2jRs3jpoLS5Kkq6vrsWPHWCyWvb29ehlAY0iaGt7MERQU5OzsTL2HJzk5mSCImzdv0mlZQ6HFAVsecL8SicTb21upVNKJxEhLwwGgU0NYp07XJe/u3r0bFxe3a9cuPAeE4uPjExERUVdXt3XrVt3tfQh1dXUVFhb6+vpShR8XLFhAkmRBQQGdr2sotKi5ZTr7TUhIKC8vT0tL0/I39geSJjAK5BDVqdNcEI9pybtB1/TrT3p6OkmSfb4cPCkpady4cV988UVxcTHTLsrMzDQzMxMIBAUFBQsWLBCJRA4ODrhQDtbd3R0fH+/k5MTn8ydOnIjPi7Vx79691tZWJycnagl+rSau+KW7lunsVywW+/r6pqWlkboZzABJExiFhIQEqVQaGxvb0NBw4cKF2trauXPnPnnyBCGUnp7+zjvvUFtmZGTs2rWL+piWlubv749LLt29e1cikYSGhsrl8s2bN1dXV1+5cqWrq2v+/Pm1tbVMm0L/N/ZApVIN1c8sLCx0d3fv8805fD7/q6++YrFYYWFhbW1tvTfQ0EUbN26MjIxUKBRCofD48eNVVVUuLi5hYWHUvINt27YdPHgwNTX18ePH/v7+ISEh6pPBNJBKpWKxmMvlOjs7L126FM8PRgjV19cjhIRCIbUlj8fj8/k4Hm1obpnmfqdMmVJXV3ft2jUtg+kTJE1geAqFIiUlZdmyZatWrbKwsPDy8jp8+HBTU5PmuWQasNlsfEY2fvz4zMzMlpaW7OzsQbSzaNEimUwWFxc3uDB6aGtru3//Pj4z6pO3t3dkZGR1dfW2bdt6rKLZRT4+PiKRyNraOjg4uK2traamBiHU3t6emZkZEBAQGBhoaWm5Y8cODodDp0PWrFlz5syZ2tra1tbWnJycmpoaX1/fiooK9H/FJUxMTNS353A4NN8nqIHmlmnuF1cBv3HjhpbB9AmSJjA8ndapUy+IZ1gNDQ0kSWp+QWNSUpK7u3tGRkZpaan6cqZdhAdO4DPN27dvy+VyarQQn8+3tbWl0yGOjo5TpkwxNzfncrmzZs3Kzs5WKBS4pji+J4uLbVM6OzvV318wOJpbprlf3Mnan/b2CZImMDxd16mjCuIZVnt7Ow5GwzY8Hi87O5sgiPfff1/97EmbLsIX+zt27KCGWz548EAulzON38vLy8TEBJcrxPeF8ZsFMLlc3t7eTs0zHjTNLdPcL86huMOHHCRNYHg6rVOnXhDPsPBf8oBDr729vbds2VJZWak+TFKbLsLFbVNTU9XHzZSVlTGNX6VSqVQqnPSdnZ2FQqF6GRd8F3jixIlMm+1Bc8s099vZ2Yn++NquIQRJExieTuvUqRfE07IpLY0aNYogiBcvXgy45e7duz08PK5evUotYVoGUJ2joyOPxysvL2ca8Jtvvqn+EZeh8fb2Rgix2eyFCxdeuHCBekpWVFREEESfAwMY0dwyzf3iTraxsdEymD5B0gSGN+R16voriMe0KTo1/egTCAQuLi741QYDdkh2drb64w46ZQA1tLZ27dqcnJzMzEyZTNbd3f3w4cPHjx8jhIKDg21sbPqbpllXV5ebm9vc3KxUKsvKytatW+fk5LRhwwa8Ni4u7smTJzt37mxraysrK0tOTg4NDXV3d8drNbesmeaWNa/FcCd7eXkNYu8DYzQUHmYEAaZoHjNDWKdOc0E8Rk1pqOnXA80ZQRKJhMPhyOVy/DE/Px8/TLeysvrwww97bBwdHa0+I0hDF2VkZOBHH2PHjq2qqjpy5AiuwTp69Og7d+6QJNnR0RETE+Pk5MRms3HF24qKCpIkAwICEELx8fF9RhsVFeXq6mpmZsZmsx0cHMLCwh49eqS+wfnz52fMmGFqampnZxcdHd3e3k6t0tyy5kKLmlsecC1JkosWLbK3t6dmDWkA0yiB0dH/MRMeHj5ixAh97pGknTQrKyvZbLaGuYl61t3dPXfu3KysrGHU8oCampp4PN6hQ4fobAzTKAFAyIgL4rm5uSUmJiYmJra2tho6FtTd3X369OmWlpYhr+Olu5bpSEhImDx5skQi0VH7kDQB0CupVLpixYrg4GA6T4R0qqSkJC8vr6ioSPPQUaNqeUApKSnl5eXnzp3jcDg62gUkTfBS2b59e3Z29osXL5ydnU+dOmXocPq2Z88eiUSyb98+w4Yxb968Y8eOUTPxh0XLmhUUFHR0dJSUlIjFYt3tZehf4QuAAe3du3fv3r2GjmJgfn5+fn5+ho7iZbNkyZIlS5boei9wpgkAAAxA0gQAAAYgaQIAAAOQNAEAgAFImgAAwADjp+enTp2i3s4BAE1/kmPmT/IzXzLLly9ntD1BMnmNRllZGX5tAAA6lZqaihCKjIw0dCDg5efo6IhLN9HELGkCoB/4TT4nTpwwdCAA9AT3NAEAgAFImgAAwAAkTQAAYACSJgAAMABJEwAAGICkCQAADEDSBAAABiBpAgAAA5A0AQCAAUiaAADAACRNAABgAJImAAAwAEkTAAAYgKQJAAAMQNIEAAAGIGkCAAADkDQBAIABSJoAAMAAJE0AAGAAkiYAADAASRMAABiApAkAAAxA0gQAAAYgaQIAAAOQNAEAgAFImgAAwAAkTQAAYACSJgAAMABJEwAAGICkCQAADEDSBAAABiBpAgAAA2xDBwAAQghdunTp2rVr1Md79+4hhI4cOUItmTRp0syZMw0QGQB/RJAkaegYAEBnz5719/c3MTFhsVgIIXxYEgSBEFKpVN3d3d98883bb79t4CgBgKQJjIRSqbSyspLJZH2uFYlEjY2NXC5Xz1EB0Bvc0wRGgcPhvPvuu32mRQ2rANA/SJrAWLz77rudnZ29lyuVypCQEP3HA0Cf4PIcGAuVSvXKK688efKkx3Jra+v6+np8rxMAg4MDERgLFou1evXqHpfhXC43NDQUMiYwHnAsAiPS+wq9s7Pz3XffNVQ8APQGl+fAuIwdO/bu3bvURxcXl6qqKgPGA0APcKYJjMuqVas4HA7+by6Xu2bNGsPGA0APcKYJjMvdu3fHjv1/7N17VBNn+jjwd0ISciEBlGuBKHdFEbW6K6ilLd/SqguIiFAvFfuTg9oWUUREBRHwUrHA0gU9tpbuqR7ACwveaD1ui9aK1lYRixURwQuIgAiBhEtI5vfHnJ3NRgiZ3NHn85eZmTzz5mV8zlzeeV538mNtba2Hh4cB2wOAAjjTBMbFzc3Nx8cHwzAMw3x8fCBjAmMDSRMYnY8++sjExMTExOSjjz4ydFsAUASX58DoNDc3Ozk54Tj++PFjBwcHQzcHgP+h16QZHh6ut32BUa2iogIh9Pbbbxu4HWCUOH78uN72pdfL8xMnTjx58kSfewSae/LkyYkTJ/S8U4FAMG7cOD3vFI7P0Uj/x6dezzQxDCsuLl6yZIne9gg0d+zYsYiICD3fxuno6EAIjRkzRp87heNzNNL/8QlFiIEx0nO6BEB18PQcAAAogKQJAAAUQNIEAAAKIGkCAAAFkDSBTpw7d87c3Pz06dOGboiuXLhwISkp6eTJky4uLsRLnytWrJDfIDAwkMfjmZiYTJo06caNGwZpZEZGBva/Jk+eLL/B5cuXZ8+ezeFw7O3tExMT+/v7KcWXyWTZ2dl+fn4vr1Ieebi1p06d+vzzz6VSKcUfqleQNIFOvNpvmu3YsSM3N3fr1q1hYWEPHjxwdXUdO3bskSNHzp49S25z/vz548ePBwUF1dTUTJ8+3YCtHU5NTU1gYGBAQEBbW1tJSck333yzdu1a1b9eV1f31ltvbdy4USwWU4qsZG1wcDCLxQoICOjs7NT8B+oKrkcIoeLiYn3uEWiuuLhYz8cJJWKx2NfXVyuhVDw+9+zZ4+Hh0dvbSy5xdXU9evQojUZzcHDo7Owkl5eXl4eEhGilbepJT0//7rvvhlsbERHh7Owsk8mIj5mZmRiG/fnnn6pErqqqWrRo0ZEjR6ZOnerj40Mp8oj7jY2N9fX1lUgkqrRE/8cnnGmC0e3w4cOtra162939+/eTk5N37tzJYrHkl/v5+cXFxTU1NW3atElvjdHE4ODg2bNn/f39icnlEULz5s3DcbysrEyVr/v4+Jw8eXLZsmWmpqaUIquy39TU1KqqqpycHA1/o45A0gTad/nyZYFAgGHYP/7xD4RQfn4+l8vlcDhlZWXz5s3j8/mOjo6FhYXExrm5uSwWy8bGZs2aNfb29iwWy8/P79q1a8Ta2NhYJpNpZ2dHfPzkk0+4XC6GYe3t7QihuLi4+Pj4+vp6DMPc3NwQQt9//z2fz9+1a5eOflpubi6O48HBwS+vysjI8PDw+Prrry9cuDDkd3Ecz8rKmjhxoqmpqaWl5cKFC+/evUusUt5FCCGpVJqSkiIQCNhs9pQpU4jTK008ePCgp6dHIBCQS1xdXRFC1dXVOo2syn4tLS39/f1zcnJwo7zJA0kTaN+cOXOuXLlCfly3bt2GDRt6e3t5PF5xcXF9fb2Li0t0dLREIkEIxcbGRkVFicXi9evXNzY23rhxY3Bw8L333nv8+DFCKDc3V/69xry8vJ07d5Ifc3JygoKCXF1dcRwnJskgniHIZDId/bSzZ896enpyOJyXV7HZ7G+//ZZGo0VHR4tEopc3SE1NTUpK2rZtW2tr66VLlx4/fjx37lxi9k3lXYQQ2rJly759+7Kzs58+fRoUFLR06dLffvtNlQYnJSVZWloymUxnZ+eFCxdev36dWN7S0oIQ4vF45JYsFovNZr88GyhVyiOruN9p06Y1NTXdunVLw8boAiRNoD9+fn58Pt/a2joyMlIkEj169IhcRafTiVMwLy+v/Pz87u7ugoICNXaxYMECoVCYnJysvVb/l0gkamhoIM6MhuTr67thw4bGxsYtW7YorOrt7c3Kylq0aNHy5cvNzc29vb0PHjzY3t5+6NAh+c2G7KK+vr78/PzQ0NCwsDALC4vt27czGAxV+mflypWnTp16/PhxT09PYWHho0eP/P39a2pqEELEA2sTExP57RkMRm9vr8r9MTTlkVXcL1G9//bt2xo2RhcgaQIDIObpJU+jFMyYMYPD4ZCXrsajtbUVx/EhTzNJGRkZnp6eeXl5ly9fll9eU1PT09MzY8YMcsnMmTOZTCZ5I0KBfBfV1taKxWJytBCbzbazs1Olf5ycnKZNm2ZmZsZkMmfNmlVQUNDb25uXl4cQIu7JDg4Oym8/MDDAZrNHDKuc8sgq7pfoZM1Pe3UBkiYwRqampm1tbYZuhaK+vj6E0MuPPuSxWKyCggIMwz7++GP5sydiDI2ZmZn8xhYWFt3d3SPul7jY3759Oznc8uHDhy8P9BmRt7e3iYnJvXv3EELEbWKhUEiuFYvFfX199vb2VMMqUB5Zxf0SOZTocGMDSRMYHYlE0tnZ6ejoaOiGKCL+J4849NrX13fjxo11dXXp6enkQgsLC4SQQopU8WdaW1sjhLKzs+UHvlRWVlJtv0wmk8lkRNJ3dnbm8XgPHz4k1xI3hadMmUI1rALlkVXc78DAAPpPhxsbSJrA6FRUVOA4PmvWLOIjnU4f7kJez2xsbDAM6+rqGnHL9PT0CRMm3Lx5k1wyefJkMzMz+ac3165dGxgYePPNN0eM5uTkxGKxqqqqqDb4/fffl/94/fp1HMd9fX0RQnQ6ff78+ZcuXSIfmpWXl2MYNuTAAEqUR1Zxv0Qn29raatgYXYCkCYyCTCZ78eLF4OBgdXV1XFycQCCIiooiVrm5uXV0dJSWlkokkra2NvmTFITQmDFjmpubGxsbu7u7JRJJeXm57oYccTgcFxcXVaq7Exfp8o87WCxWfHx8SUnJkSNHhELh7du3165da29vHxMTo0q0VatWFRYW5ufnC4VCqVT65MmTp0+fIoQiIyNtbW2He02zqampqKios7NTIpFUVlauXr1aIBCQr98kJyc/e/Zsx44dIpGosrIyMzMzKirK09OTWKs8snLKIytfSyA62dvbW42965w+R9IjeCNoFFLjjYsvv/ySuHXF4XCCg4Pz8vKI+/ru7u719fWHDh3i8/kIoXHjxt27dw/H8ZiYGAaD4eDgQKfT+Xz+woUL6+vryWjPnz9/5513WCyWs7PzZ599lpCQgBByc3N79OgRjuM3btwYN24cm82eM2dOS0vLuXPneDxeRkaGGr9UleMzNjaWwWCIxWLiY0lJCfEw3crK6tNPP1XYOCEhQf6NIJlMlpmZ6e7uzmAwLC0tQ0NDa2triVUjdlF/f39iYqJAIKDT6dbW1mFhYTU1NTiOh4aGIoRSUlKGbG18fLyrqyuXy6XT6Y6OjtHR0c3NzfIbXLx48S9/+Yupqam9vX1CQkJfXx+5SnnkysrK2bNnkzci7ezs/Pz8Ll68qErkEdfiOL5gwQIHBwfyrSEl9P9GECRNMAI9HJQxMTFjxozR6S5UocrxWVdXR6fTlbybqGdSqXTu3LmHDx8eRZFH1N7ezmKx9u/fr8rG8BoleE0ZeWEbkpubW1paWlpaWk9Pj6HbgqRSaWlpaXd3d2Rk5GiJrIrU1NSpU6fGxsbqf9eqgKQJADVJSUnh4eGRkZGqPBHSqYqKipMnT5aXlysfOmpUkUeUlZVVVVV17tw5BoOh512ryLiS5v79+4kHlAcPHjRIA9LS0ry8vPh8vqmpqZub2+bNmxVOKNSrPyhfddHOzm758uXDbXnr1q3IyEhnZ2dTU1MrKysfH5+MjAxiVWRkJKbUmTNn5Hc03FsxWVlZGIbRaLQJEyZcunRJ5b7Rla1btxYUFHR1dTk7O+t/rmD17Nq1KzY2ds+ePYZtRkBAwNGjR8kX80dFZOXKysr6+/srKiosLS31vGsK9HkvAKl2zwghdODAAf00SYG/v39eXt7z58+FQmFxcTGDwfjggw/ItX/88QebzU5OTu7p6bly5YqVldWqVatUD+7q6mpubq5kg+rqag6Hs379+oaGht7e3tra2s2bNwcEBBBrIyIizp8/TzwJJZ6cBgcHDwwMiESi1tbW6Ojo06dPkztCCNnZ2Q0MDCjsYnBwkJhPnAw7IiMvDadFqhyfwNjAPU2V9Pb2DlksWnNmZmbEQwkej7dkyZLQ0NDvv/+eqByBEEpPT7ezs9u5cyeXy/X19U1MTPz222+1+Lbf/v37LSwscnJyxo8fz2KxPDw80tPTyfG9GIbNnj3b3NycTqeTSxgMBofDsba2Vhju9+abb7a0tJSWlirs4uTJkw4ODtpqMACvoVGZNHVXQvHMmTPyY+usrKwQQsT7ahrWH1TF8+fPu7q6Ojo6yCVMJpOcMaKwsFDJDaaYmJi//e1v5Md169YhhA4cOKCwWVZWVnx8vLYaDMBryNiTJjGei8Ph8Pl8b29voVCoUEIxJyeHy+XSaLQ333zT1taWwWBwudzp06fPnTuXeI/CwsJi8+bN6u29qamJzWY7OzsjFeoAal7JcebMmSKR6N133/3ll1/UDkJ49913J06c+NNPP9XW1pILf/nlF7FYHBgYqGFwAF5nRp00RSJRcHDw4sWLOzo66urqPDw8BgYGFEooxsXFJSQk4Dh+4MCBhoaGlpaWt9566+bNm0lJSTdv3uzo6Fi5cmVmZqYahfnEYvGPP/4YHR1N1JsZsQ6g5pUcN2/ePGPGjFu3bs2ZM2fSpEn79u2TP+ukas2aNQgh+UdqX3zxxcaNG9UOCABARp40GxsbhULhpEmTWCyWra3tyZMnievlIXl5eXE4nLFjx3744YcIIYFAYGVlxeFwiEfVatx53L17t729PfnwesQ6gJpXcmSz2VeuXPn73/8+YcKEO3fuJCYmTpw48eLFi+pFW7lyJZfL/ec//0m08MGDB9evX1+6dKnazQMAICNPmi4uLjY2NsuXL09NTW1sbFTxW8SJIVmwjxjtRbXiQ0lJybFjx3744Qfy1FJ39QflMRiM2NjYP//88+rVqwsXLmxtbQ0PD3/x4oUaoczNzZcuXfrixYuioiKEUHZ29rp164jOUYPy0U6vBoRQRESEoVsBqImIiFDvkFYbXc/7o4TNZv/4449btmzZtWtXWlrakiVLCgoK9FAtqqioKCsrq6Ki4o033iAX6q7+4JD++te//utf/1q3bt2BAwd++umnRYsWqRFk3bp1X3311cGDB0NDQ48fP/7nn3+q3R7NJ6UxfhEREXFxcUQRIDBaVFZW6nkKNqNOmgihSZMmnT59uq2tLSsra+/evZMmTdLRTAakL7/88ocffvjxxx8V6sXqqP7gpUuXfv/99w0bNiCEwsLCiouLyRFFCKEVK1YcOHBAjXKzhKlTp86aNevq1asxMTHh4eGaDBiWn6jnVRUREeHr6/s6/NJXjJ6TplFfnjc3N9+5cwchZG1tvWfPnunTpxMfdQTH8cTExNu3b5eWlipkTKSz+oO///47l8sl/t3f36/wA4ln35rkZWLs0YkTJ4i8DADQkLEnzTVr1ty9e3dgYODmzZsPHz4kCtMqlFDU1u7u3Lmzb9++r776isFgyN802b9/P7GB8jqAVCs5SiSSZ8+eVVRUkEkTIRQaGnrs2LHOzs6urq6ysrItW7aEhIRokjSXLFliZWUVGhrq4uKidhAAwH/p8/UjNNJral988QVRq5nL5S5atKixsdHPz8/S0tLExOSNN97Ytm3b4OAg/r8lFJOSkogh3+PHj//555/37t1rbm6OELK1tT169GhRURER0NLSsrCwUHnzhpv6LjMzk9xGSR1AJZUcyaqLQyopKSE2O3/+fEREhKurq6mpKZPJ9PT0TE1NVSg1KBQK33rrrTFjxiCEaDSam5vbrl27Xt6RfHnHzZs3X7lyhfj39u3biZuzNBrNy8vr559/Vt5Krg1eAAAgAElEQVQnOLxGCYyb/o9PDNfjdOwYhhUXF8M9o9Hl2LFjERER+jxODAWOz9FI/8enUV+eAwCAsXmNkubdu3eVjPYySLFV8Cq5cOFCUlKSfHW+FStWyG8QGBjI4/FMTEwmTZqk3tw7msvIyFA48snp1AnqFT/UJPKpU6c+//zz0VKFGr1WSXPChAlK7lMQI8ABUM+OHTtyc3O3bt0aFhb24MEDV1fXsWPHHjly5OzZs+Q258+fP378eFBQUE1NzfTp0w3Y2uHU1NQEBgYGBAS0tbWVlJR888035CxsuoscHBzMYrECAgKIqeGN32uUNIHR0mKtP92VDVRi7969RUVFx44dky9NkJubS6PRYmJiDF7gXYHCBEd//PEHuUrD4odqR16/fr2Pj8/8+fMV3rgzTpA0geFpsdaf7soGDuf+/fvJyck7d+4kXrQl+fn5xcXFNTU1bdq0SZ/tUZvuih+qEjk1NbWqqkrPw9TVA0kTaAeO41lZWRMnTjQ1NbW0tFy4cCF5HhEbG8tkMsm5Ez755BMul4thWHt7O0JIodZfbm4ui8WysbFZs2aNvb09i8Xy8/O7du2aGqGQNur1jSg3NxfH8SHfccjIyPDw8Pj6668vXLgw5HeVdFp+fj6Xy+VwOGVlZfPmzePz+Y6OjoWFheR3pVJpSkqKQCBgs9lTpkzR/D3XEYsf6jSypaWlv79/Tk7OKBinoc3xSyNBMA5uFFJxHFxKSgqTyfzuu+86Ozurq6unT59uZWXV0tJCrF22bJmtrS25cWZmJkKora2N+BgWFkbU+iPExMRwudw7d+709fXV1NTMnDmTx+MRs5xTDXXmzBkej5eWlqbKL1Xv+HRxcfHy8lJY6Orq2tDQgOP4lStXaDTa+PHje3p6cBwvLy+Xnwldeadt27YNIfTvf/+7q6urtbV17ty5XC6XnMJk06ZNpqamJ06cePHixdatW2k02vXr10dsbXp6uqOjo4WFBYPBGD9+fEhIyK+//kqsIuppyY9KxnGczWarODOK5pGTkpIQQjdv3lRldySY7gKMSr29vVlZWYsWLVq+fLm5ubm3t/fBgwfb29sPHTqkXkA6nU6cf3l5eeXn53d3dxcUFKgRR/N6fcqJRKKGhgYlby74+vpu2LChsbFxy5YtCqtU7DQ/Pz8+n29tbR0ZGSkSiR49eoQQ6uvry8/PDw0NDQsLs7Cw2L59O4PBUKWLVq5ceerUqcePH/f09BQWFj569Mjf37+mpgapUPxQ15Hd3d0RQsO9Y2I8IGkCLaipqenp6ZkxYwa5ZObMmUwmk7ys1sSMGTM4HI4W52LSotbWVhzHlc9zm5GR4enpmZeXd/nyZfnlVDuNKOtHvDdcW1srFovJMT1sNtvOzk6VLnJycpo2bZqZmRmTyZw1a1ZBQUFvb29eXh7SuPih5pGJbiSrehstSJpAC4jBIgpVTiwsLLq7u7US39TUtK2tTSuhtKuvrw8hZGpqqmQbFotVUFCAYdjHH38sf26lSaeJRCKE0Pbt28lBkQ8fPlSjGpa3t7eJicm9e/eQtosfqhGZyKFElxozSJpACywsLBBCCv/bOzs7HR0dNQ8ukUi0FUrriP/nIw7M9vX13bhxY11dXXp6OrlQk06ztrZGCGVnZ8vfa6usrKTafplMJpPJiKSv3eKHakQeGBhA/+lSYwZJE2jB5MmTzczMfvvtN3LJtWvXBgYGyImF6XS62vWoKioqcBwnClxpGErrbGxsMAxTZSRmenr6hAkTbt68SS4ZsdOUICYNrKqqotrg999/X/4j8eyIqLusYfFDzSMT3UhU2DFmkDSBFrBYrPj4+JKSkiNHjgiFwtu3b69du9be3j4mJobYwM3NraOjo7S0VCKRtLW1yZ90oKFq/clkshcvXgwODlZXV8fFxQkEgqioKDVCUa3XRxWHw3FxcXny5MmIWxIX6fIPQ0bsNOXRVq1aVVhYmJ+fLxQKpVLpkydPnj59ihCKjIy0tbUd7jXNpqamoqKizs5OiURSWVm5evVqgUBAvpyjvPih7iITiG709vYe8ecbmD4f1SMYcjQKqTikQyaTZWZmuru7MxgMS0vL0NDQ2tpacu3z58/feecdFovl7Oz82WefJSQkIITc3NyIgUTytf5aWlpiYmIYDIaDgwOdTufz+QsXLqyvr1cvlJJ6fS9T7/iMjY1lMBhisZj4OGR1PlJCQoL8kCMlnZaXl0c8GHF3d6+vrz906BCfz0cIjRs37t69eziO9/f3JyYmCgQCOp1ubW0dFhZWU1OD43hoaChCKCUlZcjWxsfHu7q6crlcOp3u6OgYHR3d3Nwsv4GS4oe6i0xYsGCBg4ODTCYbuqOHof8hR5A0wQj0f1DGxMSMGTNGn3skqHd81tXV0el0hTcIDUgqlc6dO/fw4cOjKDKO4+3t7SwWa//+/VS/COM0AUBIhUcrxsPNzS0tLS0tLa2np8fQbUFSqbS0tLS7u1vrVbt0F5mQmpo6derU2NhYXQTXLkiaAGgqKSkpPDw8MjLS4LU5KioqTp48WV5ernzoqFFFRghlZWVVVVWdO3eOmHDbyEHSBMZl69atBQUFXV1dzs7OJ06cMHRzVLVr167Y2Ng9e/YYthkBAQFHjx4l380fFZHLysr6+/srKio0mS1Vn4x9Cl/wutm9e/fu3bsN3Qp1BAYGBgYGGroVo09ISEhISIihW0EBnGkCAAAFkDQBAIACSJoAAEABJE0AAKBA3w+C1KgpAAyL+JMdO3bM0A3RBzg+Rx39/8kwXI/F5ckZQgAAQIv0msf0uTMAVLRkyRL02pzegtEF7mkCAAAFkDQBAIACSJoAAEABJE0AAKAAkiYAAFAASRMAACiApAkAABRA0gQAAAogaQIAAAWQNAEAgAJImgAAQAEkTQAAoACSJgAAUABJEwAAKICkCQAAFEDSBAAACiBpAgAABZA0AQCAAkiaAABAASRNAACgAJImAABQAEkTAAAogKQJAAAUQNIEAAAKIGkCAAAFkDQBAIACSJoAAEABJE0AAKAAkiYAAFAASRMAACiApAkAABRA0gQAAAogaQIAAAUYjuOGbgMA6OjRo4cPH5bJZMTHhoYGhJCzszPxkUaj/b//9/+WLVtmsPYB8B+QNIFRqK6u9vHxUbLBrVu3pkyZorf2ADAcSJrAWEyYMKG2tnbIVW5ubnV1dXpuDwBDgnuawFisWLGCwWC8vJzBYKxatUr/7QFgSHCmCYzFgwcP3Nzchjwg6+rq3Nzc9N8kAF4GZ5rAWLi4uEyfPh3DMPmFGIbNmDEDMiYwHpA0gRH56KOPTExM5JeYmJh89NFHhmoPAC+Dy3NgRFpbW+3t7cmBRwghGo3W3Nxsa2trwFYBIA/ONIERsbGx8ff3J082TUxM3n77bciYwKhA0gTGZcWKFfJXPytWrDBgYwB4GVyeA+MiFAqtra0HBgYQQgwGo7W11cLCwtCNAuC/4EwTGBc+n//BBx/Q6XQ6nT5//nzImMDYQNIERmf58uVSqVQqlcLL5sAIweU5MDp9fX1WVlY4jre3t7PZbEM3B4D/hWuguLjY0M0HAABqiouLNcl7dK20QPMg4DVRWVmZk5Mz4jFTVVWFYZjyukdGLiIiIi4uztfX19ANAf8jIiJCwwhaSJpLlizRPAh4feTk5Ix4zCxatAghRKdr4fg0lIiICF9fX/jfYWyMImkCoHWjOl2CVxs8PQcAAAogaQIAAAWQNAEAgAJImgAAQAEkTTAKnDt3ztzc/PTp04ZuiK5cuHAhKSnp5MmTLi4uGIZhGKZQqSQwMJDH45mYmEyaNOnGjRsGaWRGRgb2vyZPniy/weXLl2fPns3hcOzt7RMTE/v7+3Ud+dSpU59//rlUKtXKD1QRJE0wCrza763t2LEjNzd369atYWFhDx48cHV1HTt27JEjR86ePUtuc/78+ePHjwcFBdXU1EyfPt2ArR1OTU1NYGBgQEBAW1tbSUnJN998s3btWl1HDg4OZrFYAQEBnZ2dWtmXSjR/I0iTCOB1Y+THjFgs9vX11UoopNqbJ3v27PHw8Ojt7SWXuLq6Hj16lEajOTg4dHZ2ksvLy8tDQkK00jb1pKenf/fdd8OtjYiIcHZ2lslkxMfMzEwMw/788089RI6NjfX19ZVIJKrsS8W/ixJwpgnAfx0+fLi1tVVvu7t//35ycvLOnTtZLJb8cj8/v7i4uKampk2bNumtMZoYHBw8e/asv78/OcXTvHnzcBwvKyvTQ+TU1NSqqqqcnBwN96UiSJrA2F2+fFkgEGAY9o9//AMhlJ+fz+VyORxOWVnZvHnz+Hy+o6NjYWEhsXFubi6LxbKxsVmzZo29vT2LxfLz87t27RqxNjY2lslk2tnZER8/+eQTLpeLYVh7eztCKC4uLj4+vr6+HsMwYiq377//ns/n79q1S0c/LTc3F8fx4ODgl1dlZGR4eHh8/fXXFy5cGPK7OI5nZWVNnDjR1NTU0tJy4cKFd+/eJVYp7yKEkFQqTUlJEQgEbDZ7ypQpmr8J/eDBg56eHoFAQC5xdXVFCFVXV+shsqWlpb+/f05ODq6X2ziQNIGxmzNnzpUrV8iP69at27BhQ29vL4/HKy4urq+vd3FxiY6OlkgkCKHY2NioqCixWLx+/frGxsYbN24MDg6+9957jx8/Rgjl5ubKv9eYl5e3c+dO8mNOTk5QUJCrqyuO4/fv30cIEU8Y5Ocs0q6zZ896enpyOJyXV7HZ7G+//ZZGo0VHR4tEopc3SE1NTUpK2rZtW2tr66VLlx4/fjx37txnz56hkboIIbRly5Z9+/ZlZ2c/ffo0KCho6dKlv/32myoNTkpKsrS0ZDKZzs7OCxcuvH79OrG8paUFIcTj8cgtWSwWm80m2qOHyNOmTWtqarp165aKu9MEJE0wWvn5+fH5fGtr68jISJFI9OjRI3IVnU4nTsG8vLzy8/O7u7sLCgrU2MWCBQuEQmFycrL2Wv1fIpGooaGBOG8akq+v74YNGxobG7ds2aKwqre3Nysra9GiRcuXLzc3N/f29j548GB7e/uhQ4fkNxuyi/r6+vLz80NDQ8PCwiwsLLZv385gMFTpn5UrV546derx48c9PT2FhYWPHj3y9/evqalBCBGPsxVmEmUwGL29vap0heaR3d3dEUK3b99WZXcagqQJRj0mk4kQIk+jFMyYMYPD4ZCXrsajtbUVx/EhTzNJGRkZnp6eeXl5ly9fll9eU1PT09MzY8YMcsnMmTOZTCZ5I0KBfBfV1taKxWJyTA+bzbazs1Olf5ycnKZNm2ZmZsZkMmfNmlVQUNDb25uXl4cQIu7JDg4Oym8/MDCgYjlUzSMT3aj6ia0mIGmCV5+pqWlbW5uhW6Gor68PIWRqaqpkGxaLVVBQgGHYxx9/LH9uRYywMTMzk9/YwsKiu7t7xP0SF/vbt28nB0U+fPhQLBZTbb+3t7eJicm9e/cQQsRtYqFQSK4Vi8V9fX329vZUw6oXmcihRJfqGiRN8IqTSCSdnZ2Ojo6Gbogi4v/5iAOzfX19N27cWFdXl56eTi4kpk5SSJEq/kxra2uEUHZ2tvwwmsrKSqrtl8lkMpmMSPrOzs48Hu/hw4fkWuKm8JQpU6iGVS8yMROffur8Q9IEr7iKigocx2fNmkV8pNPpw13I65mNjQ2GYV1dXSNumZ6ePmHChJs3b5JLJk+ebGZmJv/05tq1awMDA2+++eaI0ZycnFgsVlVVFdUGv//++/Ifr1+/juM4UWWZmAXv0qVL5EOz8vJyDMOGHBigi8hEN9ra2lL9UWqApAleQTKZ7MWLF4ODg9XV1XFxcQKBICoqiljl5ubW0dFRWloqkUja2trkT2EQQmPGjGlubm5sbOzu7pZIJOXl5bobcsThcFxcXJ48eTLilsRFuvzDEBaLFR8fX1JScuTIEaFQePv27bVr19rb28fExKgSbdWqVYWFhfn5+UKhUCqVPnny5OnTpwihyMhIW1vb4V7TbGpqKioq6uzslEgklZWVq1evFggE5Ms5ycnJz54927Fjh0gkqqyszMzMjIqK8vT0JNbqLjKB6EZvb+8Rf74WaDIy3sjf7gBGSI1j5ssvvyRubHE4nODg4Ly8POKuv7u7e319/aFDh/h8PkJo3Lhx9+7dw3E8JiaGwWA4ODjQ6XQ+n79w4cL6+noy2vPnz9955x0Wi+Xs7PzZZ58lJCQghNzc3B49eoTj+I0bN8aNG8dms+fMmdPS0nLu3Dkej5eRkaHGL0UqvHkSGxvLYDDEYjHxsaSkhHiYbmVl9emnnypsnJCQIP9GkEwmy8zMdHd3ZzAYlpaWoaGhtbW1xKoRu6i/vz8xMVEgENDpdGtr67CwsJqaGhzHQ0NDEUIpKSlDtjY+Pt7V1ZXL5dLpdEdHx+jo6ObmZvkNLl68+Je//MXU1NTe3j4hIaGvr49cpbvIhAULFjg4OJBvDSmhyt9lhAiafBmSJqBKD8dMTEzMmDFjdLoLVajyn7Ouro5Opyt5g1DPpFLp3LlzDx8+PIoi4zje3t7OYrH279+vysaaJ024PAevID2XvVGbm5tbWlpaWlpaT0+PoduCpFJpaWlpd3d3ZGTkaIlMSE1NnTp1amxsrC6Cv0zfSXP16tU8Hg/DMDXuQxuVtLQ0Ly8vPp9vamrq5ua2efNmheNevTJZ8sXBCEwm08bG5u23387MzHzx4oVufg0wmKSkpPDw8MjISFWeCOlURUXFyZMny8vLlQ8dNarICKGsrKyqqqpz584xGAytBx+aJqep6l1qEe/A3rx5U5NdG5y/v39eXt7z58+FQmFxcTGDwfjggw/ItX/88QebzU5OTu7p6bly5YqVldWqVatUD+7q6mpubo7jOPFA46effoqKisIwzN7enniwOHrp+vI8KSmJGMg9fvz448eP625HI0JULgN/+OGHxMREnbbnlVRaWrp79+7BwUHVv0Lp7zJ0BE2+/IolTUplwRYsWCD/pyLeaCYeJuCalcnC5ZKmvOPHj9NoNBsbG/lyYUZC9a57fe6Da/6fE+iC5n8XA9zTJEs8GRtKZcHOnDkjPwTEysoKIUS8VqGjMlmLFy+OiopqbW09ePCgJnF0Qc8V1QAwIH0kTRzHMzMzPT09TU1Nzc3NiUEehH379nE4HB6P19raGh8f7+DgQAybGK7mlfLCX0hpvSyqZcEoaWpqYrPZzs7OSIViVmoXHCMGG5aXl79KXQfAKKPJaaqKl1rbtm3DMOyLL7548eKFWCwm3sMnL8+3bduGEFq/fv2XX365aNGiP//8MyUlhclkfvfdd52dndXV1dOnT7eysmppaSG2j4mJ4XK5d+7c6evrq6mpmTlzJo/HI6+LlX932bJltra2ZMMyMzMRQm1tbcTHsLAwoiwYVSKRiMfjxcbGEh8vXryIEMrMzJTfhs1mBwQEEP8+c+YMj8dLS0sbLuCQl+c4jhNv4Do5OREfR2PXweU5MCzN/y46T5pisZjD4bz33nvkEoV7msT/fLLcv1gsNjMzi4yMJLf/9ddfEUJkiomJiZFPKETdvZ07d6ryXR0lzW3btnl4eAiFQuLj+fPnEUJZWVny2/D5fD8/PxUDDpc0cRzHMMzCwoLc76jrOkiawLA0/7vQdX0me//+fbFYHBAQoOL2VGteyRf+ovpdrSgpKTl27Nj58+fJOqkalslSQiQS4ThOvN3xslHUdceOHdNFWGOjRhUMYPx0njSJd0KJwiqqUKPmFVn4S5N6WeopKirKysqqqKh44403yIXaLZMljyiWNWHChCHXjqKui4iI0EVYY5OTk6O3iWuA3ug8aRKnXarPgEy15pV84S9N6mWp4csvv/zhhx9+/PFHhVyj3TJZ8r7//nuE0Lx584ZcO4q6Dn+lp+QlYBhWXFwsP7sGMAaaj97R+dPzyZMn02g04tmIittTqnklX/hrxO9qqywYjuOJiYm3b98uLS1VyJhI4zJZw2lpacnOznZ0dPz444+H3GBUdB0Ao54mN0RVvKkfHh5uYmLy9ddfd3V13bp165133kHDPwjCcXzHjh0MBuO7777r6uqqrq6eNm2avb19T08PsTYmJobH43V0dEgkklu3bnl5eQkEArLqifLvEmVc//Wvfw0MDLS2tn766adI7mlGdHQ0m81uaGgQCoUDAwNKftEff/wxZGeST8z/+OMPFou1fft24o2gsWPHyr8RNGLtHFdXVz6f393dLZVKZTJZa2trUVGRi4uLnZ3db7/9Rm42GrsOHgQBw9L876KPpNnd3b169eqxY8eamZnNmTMnJSUFIeTo6Hjr1q3PP/+ceDzi5ORE1npRUvMKH6nwl/LvUioLpuQXDTd/k/wwIyXFrJQkzVOnTk2ZMoXD4TCZTBqNhhAiHpf/5S9/SUtLe/78ObnlKO06SJrAsEZH0tQuIyn8NRoZQ9dB0gSGpfnfZVSWhhsthb+MEHQdABoalUlT1+7evYsNT0c1AQEAo8IoS5pbt24tKCjo6upydnY+ceKEjvYyYcIEJSfnRUVFOtqvTumn64B6Lly4kJSUJF9NdcWKFfIbBAYG8ng8ExOTSZMmDTfTjq5lZGQonECQk6cT1Kshq0nkU6dOff755/q+ftLk2v71uT8FtOX1OWaQyvfOUlJSgoKCyNdwXV1dx44dixA6c+aM/Gbl5eXycwTpn/wcwoRJkyaRazWpIatJ5JycHH9//xcvXqi4L9X/LsMZZWeaAIyot7fXz8/P2EINZ+/evUVFRceOHSNfw0UI5ebm0mi0mJgYg5dzV6AwnZH82Lv09HQ7O7udO3dyuVxfX9/ExMRvv/2WLJSlu8jr16/38fGZP3++wovLugNJE7xqtFjcU9d1Qu/fv5+cnLxz507ixTmSn59fXFxcU1PTpk2bdLd3LdJRDVkVI6emplZVVentjVVImsAY4Voq7qm8iijVOqFqF0IdTm5uLo7jQ74qlpGR4eHh8fXXX1+4cIFqF+Xn53O5XA6HU1ZWNm/ePD6f7+joSFQXI0il0pSUFIFAwGazp0yZQtwz0cSINWR1GtnS0tLf3z8nJwfXz+u5mlzbvz73p4C2qHjMaLG4p/IqopRCjVgIVR5S4d6Zi4uLl5eXwkJXV9eGhgYcx69cuUKj0caPH0+8l6VwT1N5FxFvi/373//u6upqbW2dO3cul8slX9batGmTqanpiRMnXrx4sXXrVhqNpsrcU+np6Y6OjhYWFgwGY/z48SEhIb/++iuxasQasrqOnJSUhFSbREeVv4tycKYJjE5vb29WVtaiRYuWL19ubm7u7e198ODB9vb2Q4cOqReQTqcTZ2ReXl75+fnd3d0FBQVqxFmwYIFQKExOTlavGQpEIlFDQwNx3jQkX1/fDRs2NDY2btmyRWGVil3k5+fH5/Otra0jIyNFItGjR48QQn19ffn5+aGhoWFhYRYWFtu3b2cwGKp0yMqVK0+dOvX48eOenp7CwsJHjx75+/vX1NSg/1TkkZ8ABiHEYDB6e3tV6QrNI7u7uyOEhntVT7sgaQKjo9PinvJVRA2rtbUVx3Hls9pmZGR4enrm5eVdvnxZfjnVLiJm6CRKrtTW1orFYnJMD5vNtrOzU6VDnJycpk2bZmZmxmQyZ82aVVBQ0NvbS0zEoGENWc0jE9347NkzVXanIUiawOjourgnWUXUsPr6+ojGKNmGxWIVFBRgGPbxxx/Ln1tp0kUikQghtH37dnJQ5MOHD4k5ASnx9vY2MTEharxqt4asGpGJHEp0qa5B0gRGR6fFPeWriBoW8f98xIHZvr6+GzdurKurkx/MqEkXERXBs7Oz5e/TqVFkXiaTyWQyIulrt4asGpEHBgbQf7pU1yBpAqOj0+Ke8lVENQylIRsbGwzDVBmJmZ6ePmHChJs3b5JLqNZOlefk5MRisaqqqqg2+P3335f/SDw78vX1RRrXkNU8MtGNtra2VH+UGiBpAqPDYrHi4+NLSkqOHDkiFApv3769du1ae3v7mJgYYgM3N7eOjo7S0lKJRNLW1iZ/GoIQGjNmTHNzc2NjY3d3N5EQZTLZixcvBgcHq6ur4+LiBAIBMRky1VDl5eVaHHLE4XBcXFyI+WBG7JCCggL5hyEjdpHyaKtWrSosLMzPzxcKhVKp9MmTJ0+fPkUIRUZG2traDveaZlNTU1FRUWdnp0QiqaysXL16tUAgWLt2LbE2OTn52bNnO3bsEIlElZWVmZmZUVFRnp6exFrdRSYQ3ejt7T3iz9cCTR69w5AjQJWKx4wWi3sqryJKKdSI1aPlIRWGtsTGxjIYDLFYTHwsKSkhHqZbWVl9+umnChsnJCTIDzlS0kV5eXnEgxF3d/f6+vpDhw4Rk/GNGzfu3r17OI739/cnJiYKBAI6nW5tbR0WFlZTU4PjeGhoKEIoJSVlyNbGx8e7urpyuVw6ne7o6BgdHd3c3Cy/gZIasrqLTFiwYIGDg4NMJhu6o+Wo8ncZIYImX4akCajS/zFjqCqiqvznrKuro9PpCm8QGpBUKp07d+7hw4dHUWQcx9vb21ks1v79+1XZWPOkCZfn4NVntFVE3dzc0tLS0tLSenp6DN0WJJVKS0tLu7u7tV78UHeRCampqVOnTo2NjdVF8JdB0gTAkJKSksLDwyMjIw1em6OiouLkyZPl5eXKh44aVWSEUFZWVlVV1blz5xgMhtaDDwmSJniVjYoqort27YqNjd2zZ49hmxEQEHD06FHyTfxREbmsrKy/v7+iosLS0lLrwYej83nPATCg3bt3796929CtGFlgYGBgYKChWzH6hISEhISE6HmncKYJAAAUQNIEAAAKIGkCAAAFkDQBAIACLTwICg8P1zwIeE0Qr7u9JsdMdnb28ePHDd0KoGUYrkGB+MrKyqysLC22BgACUZxi2rRphm4IeAVt3LiRqAaiHo2SJgA6smTJEoTQsWPHDN0QABTBPU0AAFuf/YUAACAASURBVKAAkiYAAFAASRMAACiApAkAABRA0gQAAAogaQIAAAWQNAEAgAJImgAAQAEkTQAAoACSJgAAUABJEwAAKICkCQAAFEDSBAAACiBpAgAABZA0AQCAAkiaAABAASRNAACgAJImAABQAEkTAAAogKQJAAAUQNIEAAAKIGkCAAAFkDQBAIACSJoAAEABJE0AAKAAkiYAAFAASRMAACiApAkAABRA0gQAAAogaQIAAAWQNAEAgAJImgAAQAHd0A0AACGExGJxf38/+XFgYAAh9OLFC3KJqakph8MxQMsA+F8YjuOGbgMAKD8//5NPPlGyQV5e3rp16/TWHgCGA0kTGIW2tjZ7e3upVDrkWhMTk6dPn1pbW+u5VQC8DO5pAqNgbW0dEBBgYmLy8ioTE5P/+7//g4wJjAQkTWAsli9fPuR1D47jy5cv1397ABgSXJ4DY9Hd3W1tbS3/OIjAZDLb2tr4fL5BWgWAAjjTBMaCx+MFBQUxGAz5hXQ6PSQkBDImMB6QNIERWbZs2eDgoPwSqVS6bNkyQ7UHgJfB5TkwIgMDA1ZWVt3d3eQSMzOz9vZ2U1NTA7YKAHlwpgmMCJPJDA8PZzKZxEcGgxEREQEZExgVSJrAuCxdupR4HQghJJFIli5datj2AKAALs+BcZHJZHZ2dm1tbQghKyurlpaWIQdvAmAocKYJjAuNRlu6dCmTyWQwGMuWLYOMCYwNJE1gdD788MOBgQG4NgfGyZBVjiorKx8/fmzABgDjhOP42LFjEUINDQ2NjY2Gbg4wOk5OTr6+vgbbPW44ixcvNtjPBgCMWosXLzZg4jJwPc3FixcfP37csG0AVGEYVlxcvGTJEt3t4s6dOwghLy8v3e1iROHh4QghOD6NDfF3MSAoQgyMkWHTJQBKwIMgAACgAJImAABQAEkTAAAogKQJAAAUQNIEAAAKIGkCPTl37py5ufnp06cN3RBduXDhQlJS0smTJ11cXDAMwzBsxYoV8hsEBgbyeDwTE5NJkybduHHDII3MyMjA/tfkyZPlN7h8+fLs2bM5HI69vX1iYuLLhfS1HvnUqVOff/75cHPqGSFImkBP8Fe6NMyOHTtyc3O3bt0aFhb24MEDV1fXsWPHHjly5OzZs+Q258+fP378eFBQUE1NzfTp0w3Y2uHU1NQEBgYGBAS0tbWVlJR88803a9eu1XXk4OBgFosVEBDQ2dmplX3pGiRNoCcLFizo6uoKCgrS9Y56e3v9/Px0vRd5e/fuLSoqOnbsGI/HIxfm5ubSaLSYmJiuri59NmZE3333nfz7LX/88Qe5Kj093c7ObufOnVwu19fXNzEx8dtvv717966uI69fv97Hx2f+/PkKdfuNEyRN8Ko5fPhwa2ur3nZ3//795OTknTt3slgs+eV+fn5xcXFNTU2bNm3SW2M0MTg4ePbsWX9/fwzDiCXz5s3DcbysrEwPkVNTU6uqqnJycjTclx5A0gT6cPnyZYFAgGHYP/7xD4RQfn4+l8vlcDhlZWXz5s3j8/mOjo6FhYXExrm5uSwWy8bGZs2aNfb29iwWy8/P79q1a8Ta2NhYJpNpZ2dHfPzkk0+4XC6GYe3t7QihuLi4+Pj4+vp6DMPc3NwQQt9//z2fz9+1a5eOflpubi6O48HBwS+vysjI8PDw+Prrry9cuDDkd3Ecz8rKmjhxoqmpqaWl5cKFC8mTL+VdhBCSSqUpKSkCgYDNZk+ZMqW4uFjDH/LgwYOenh6BQEAucXV1RQhVV1frIbKlpaW/v39OTo7x38aBpAn0Yc6cOVeuXCE/rlu3bsOGDb29vTwer7i4uL6+3sXFJTo6WiKRIIRiY2OjoqLEYvH69esbGxtv3LgxODj43nvvETWxcnNz5V97z8vL27lzJ/kxJycnKCjI1dUVx/H79+8jhIgnDDKZTEc/7ezZs56enhwO5+VVbDb722+/pdFo0dHRIpHo5Q1SU1OTkpK2bdvW2tp66dKlx48fz50799mzZ2ikLkIIbdmyZd++fdnZ2U+fPg0KClq6dOlvv/2mSoOTkpIsLS2ZTKazs/PChQuvX79OLG9paUEIyd9hYLFYbDabaI8eIk+bNq2pqenWrVsq7s5QIGkCQ/Lz8+Pz+dbW1pGRkSKR6NGjR+QqOp1OnIJ5eXnl5+d3d3cXFBSosYsFCxYIhcLk5GTttfq/RCJRQ0MDcd40JF9f3w0bNjQ2Nm7ZskVhVW9vb1ZW1qJFi5YvX25ubu7t7X3w4MH29vZDhw7JbzZkF/X19eXn54eGhoaFhVlYWGzfvp3BYKjSPytXrjx16tTjx497enoKCwsfPXrk7+9fU1ODECIeZytUfWYwGL29vap0heaR3d3dEUK3b99WZXcGBEkTGAViMjXyNErBjBkzOByO6k8k9Ka1tRXH8SFPM0kZGRmenp55eXmXL1+WX15TU9PT0zNjxgxyycyZM5lMJnkjQoF8F9XW1orFYnJMD5vNtrOzU6V/nJycpk2bZmZmxmQyZ82aVVBQ0Nvbm5eXhxAi7skqPIoZGBhgs9kjhtVKZKIbVT+xNRRImmB0MDU1JSYOMip9fX0IIeXzZbJYrIKCAgzDPv74Y/lzK2KEjZmZmfzGFhYW8jMYD4e42N++fTs5KPLhw4disZhq+729vU1MTO7du4cQIm4TC4VCcq1YLO7r67O3t6caVr3IRA4lutSYQdIEo4BEIuns7HR0dDR0QxQR/89HHJjt6+u7cePGurq69PR0cqGFhQVCSCFFqvgzra2tEULZ2dnyQ3wqKyuptl8mk8lkMiLpOzs783i8hw8fkmuJm8JTpkyhGla9yMQspCqe2BoQJE0wClRUVOA4PmvWLOIjnU4f7kJez2xsbDAMU2UkZnp6+oQJE27evEkumTx5spmZmfzTm2vXrg0MDLz55psjRnNycmKxWFVVVVQb/P7778t/vH79Oo7jxNQRdDp9/vz5ly5dIh+alZeXYxg25MAAXUQmutHW1pbqj9IzSJrASMlkshcvXgwODlZXV8fFxQkEgqioKGKVm5tbR0dHaWmpRCJpa2uTP4VBCI0ZM6a5ubmxsbG7u1sikZSXl+tuyBGHw3FxcXny5MmIWxIX6fIPQ1gsVnx8fElJyZEjR4RC4e3bt9euXWtvbx8TE6NKtFWrVhUWFubn5wuFQqlU+uTJk6dPnyKEIiMjbW1th3tNs6mpqaioqLOzUyKRVFZWrl69WiAQkC/nJCcnP3v2bMeOHSKRqLKyMjMzMyoqytPTk1iru8gEohu9vb1H/PkGprOJNEa2ePFiw871AdSDECouLqb0lS+//JK4scXhcIKDg/Py8oi7/u7u7vX19YcOHeLz+QihcePG3bt3D8fxmJgYBoPh4OBAp9P5fP7ChQvr6+vJaM+fP3/nnXdYLJazs/Nnn32WkJCAEHJzc3v06BGO4zdu3Bg3bhybzZ4zZ05LS8u5c+d4PF5GRgbVn6ni8RkbG8tgMMRiMfGxpKSEeJhuZWX16aefKmyckJAQEhJCfpTJZJmZme7u7gwGw9LSMjQ0tLa2llg1Yhf19/cnJiYKBAI6nW5tbR0WFlZTU4PjeGhoKEIoJSVlyNbGx8e7urpyuVw6ne7o6BgdHd3c3Cy/wcWLF//yl7+Ympra29snJCT09fWRq3QXmbBgwQIHBweZTDZ0R/+HwfMGJE1AmRpJk6qYmJgxY8bodBcjUvH4rKuro9PpCm8QGpBUKp07d+7hw4dHUWQcx9vb21ks1v79+0fc0uB5Ay7PgZEaLWVv3Nzc0tLS0tLSenp6DN0WJJVKS0tLu7u7IyMjR0tkQmpq6tSpU2NjY3URXLuMPWnu37+fuNd+8OBBgzQgLS3Ny8uLz+ebmpq6ublt3rz55f8bMpksOzubUpEI+QJidnZ2y5cvH27LW7duRUZGOjs7m5qaWllZ+fj4ZGRkEKsiIyMxpc6cOSO/o+EGeGdlZWEYRqPRJkyYcOnSJdV/BSAkJSWFh4dHRkYavDZHRUXFyZMny8vLlQ8dNarICKGsrKyqqqpz584xGAytB9c+A57lqn75gxA6cOCAHpr0Mn9//7y8vOfPnwuFwuLiYgaD8cEHH8hvcO/evdmzZyOEfHx8qAZ3dXU1NzdXskF1dTWHw1m/fn1DQ0Nvb29tbe3mzZsDAgKItREREefPnyduvRMPAYKDgwcGBkQiUWtra3R09OnTp8kdIYTs7OwGBgYUdjE4ODhu3DiEEBl2REjHl+dJSUnEQO7x48cfP35cdztSjupl4A8//JCYmKi79ryqSktLd+/ePTg4qOL2cHmuHbqrBmZmZkbcX+PxeEuWLAkNDf3++++Jl6ARQrdu3dqyZcvatWunTp2qi73v37/fwsIiJydn/PjxLBbLw8MjPT2dHMiGYdjs2bPNzc3pdDq5hMFgcDgca2trhZErb775ZktLS2lpqcIuTp486eDgoIvGq2337t39/f04jjc0NCxevNjQzVFVYGDg3r17Dd2K0SckJCQpKUnhJUtj9ookTd1VAztz5oz8n9PKygohRL564ePjc/LkyWXLlil/J0Rtz58/7+rq6ujoIJcwmUyy+HlhYaGSa6WYmJi//e1v5Md169YhhA4cOKCwWVZWVnx8vDYbDcArbfQlTWLgAofD4fP53t7eQqFQoRpYTk4Ol8ul0Whvvvmmra0tg8HgcrnTp0+fO3cuMSTYwsJi8+bN6u29qamJzWY7OzursrHmRclmzpwpEonefffdX375Re0ghHfffXfixIk//fRTbW0tufCXX34Ri8WBgYEaBgfg9THKkqZIJAoODl68eHFHR0ddXZ2Hh8fAwIBCNbC4uLiEhAQcxw8cONDQ0NDS0vLWW2/dvHkzKSnp5s2bHR0dK1euzMzMVKMClVgs/vHHH6Ojo4k7biPSvCjZ5s2bZ8yYcevWrTlz5kyaNGnfvn3yZ51UrVmzBiEk/0jtiy++2Lhxo9oBAXgNjbKk2djYKBQKJ02axGKxbG1tT548SVwvD8nLy4vD4YwdO/bDDz9ECAkEAisrKw6HQzyqVqNkzu7du+3t7cmH1yPSvCgZm82+cuXK3//+9wkTJty5cycxMXHixIkXL15UL9rKlSu5XO4///lPomzEgwcPrl+/vnTpUrWbB8BriG7oBlDj4uJiY2OzfPny9evXR0VFjR8/XpVvESeGZGUqYlgD1ZeXS0pKjh07dv78eflaqnrAYDBiY2NjY2OvXbu2d+/e0tLS8PDw2tpaS0tLqqHMzc2XLl361VdfFRUVrVq1Kjs7e926dUwmkyiUQEl2dvbx48epfmt0uXr1KkIoPDzc0A0B/+Pq1atkFQKDGGVnmmw2+8cff5wzZ86uXbtcXFwiIyNVrJCqoaKior1791ZUVKiYpnXhr3/967/+9a+1a9e2tbX99NNP6gUhHgcdPHiws7Pz+PHjxAU7AEB1o+xMEyE0adKk06dPt7W1ZWVl7d27d9KkSToqyk368ssvf/jhhx9//FGh9KGOXLp06ffff9+wYQNCKCwsrLi4mBxRhBBasWLFgQMH1KicSJg6deqsWbOuXr0aExMTHh6uxukqYcOGDfJzTrySiHPMV/6EetQx+Ln/KDvTbG5uvnPnDkLI2tp6z54906dPJz7qCI7jiYmJt2/fLi0t1U/GRAj9/vvvXC6X+Hd/f7/CDySefatX4pBAnGyeOHGCyMsAAEpGX9Jcs2bN3bt3BwYGbt68+fDhQ+LuhkI1MG3t7s6dO/v27fvqq68YDIb864n79+9X5etUi5JJJJJnz55VVFSQSRMhFBoaeuzYsc7Ozq6urrKysi1btoSEhGiSNJcsWWJlZRUaGuri4qJ2EABeXwZ8G0mV16G++OILoigpl8tdtGhRY2Ojn5+fpaWliYnJG2+8sW3bNuLtK/lqYElJScSQ7/Hjx//888979+41NzdHCNna2h49erSoqIgIaGlpWVhYqHzvw83xlJmZSWxQWVk5e/Zssmq/nZ2dn5/fxYsXibVKipKRBcSGVFJSQmx2/vz5iIgIV1dXU1NTJpPp6emZmpqqUFNLKBS+9dZbY8aMQQjRaDQ3N7ddu3a9vCP5SmWbN2++cuUK8e/t27cTRdtoNJqXl9fPP/+svE9wvVQ5MgYGf10PDMngfxcMN9wsw3DPaJTCMKy4uBjuaQKDMPjfZZRdngMAgGG91knz7t27Suqq6ahuIAAXLlxISkqSr9q3YsUK+Q0CAwN5PJ6JicmkSZOGm15C1zIyMhT+R5AzBhMuX748e/ZsDodjb2+fmJhITG6OEDp16tTnn38+WsqhquG1TpoTJkxQcueiqKjI0A0Er6AdO3bk5uZu3bo1LCzswYMHrq6uY8eOPXLkyNmzZ8ltzp8/f/z48aCgoJqamunTpxuwtcOpqakJDAwMCAhoa2srKSn55ptvyOmAgoODWSxWQEAAMUfxq+e1TprAOGmx0J/uagaqZ+/evUVFRceOHZN/ryw3N5dGo8XExBi8hrEChTk8/vjjD3JVenq6nZ3dzp07uVyur69vYmLit99+S76avH79eh8fn/nz55Ov4b1KIGkCo6PFQn+6qxmohvv37ycnJ+/cuZPFYskv9/Pzi4uLa2pq2rRpk6HaRsng4ODZs2f9/f0xDCOWzJs3D8fxsrIycpvU1NSqqqqcnBwDtVGHIGkCncBxPCsra+LEiaamppaWlgsXLiRPQ2JjY5lMJjHOCSH0ySefcLlcDMPa29sRQgqF/nJzc1kslo2NzZo1a+zt7Vkslp+f37Vr19QIhbRRrE8Tubm5OI4POY14RkaGh4fH119/feHChSG/q6Q/8/PzuVwuh8MpKyubN28en893dHQsLCwkvyuVSlNSUgQCAZvNnjJlSnFxsYY/5MGDBz09PQKBgFxCDGurrq4ml1haWvr7++fk5BhwfI6u6Gww08gMPt4KqAepME4zJSWFyWR+9913nZ2d1dXV06dPt7KyamlpIdYuW7bM1taW3DgzMxMh1NbWRnwMCwsjCv0RYmJiuFzunTt3+vr6ampqZs6cyePxiNl6qYY6c+YMj8dLS0tT5Wdq/fh0cXHx8vJSWOjq6trQ0IDj+JUrV2g02vjx43t6enAcLy8vl5/sV3l/btu2DSH073//u6urq7W1de7cuVwul5zaZNOmTaampidOnHjx4sXWrVtpNNr169dHbG16erqjo6OFhQWDwRg/fnxISMivv/5KrCLqbJGjlQlsNlthxpSkpCSE0M2bN6l00sgMnjfgTBNoX29vb1ZW1qJFi5YvX25ubu7t7X3w4MH29vZDhw6pF5BOpxMnWV5eXvn5+d3d3QUFBWrE0bxYn9pEIlFDQ4OSNxp8fX03bNjQ2Ni4ZcsWhVUq9qefnx+fz7e2to6MjBSJRI8ePUII9fX15efnh4aGhoWFWVhYbN++ncFgqNJ7K1euPHXq1OPHj3t6egoLCx89euTv719TU4MQIh6UK0xQwWAwFKrnuLu7I4SGe0Nk9IKkCbSvpqamp6dnxowZ5JKZM2cymUzysloTM2bM4HA4apRDNazW1lYcx5VP5ZiRkeHp6ZmXl3f58mX55VT7k6iFSLxPXFtbKxaLydFCbDbbzs5Old5zcnKaNm2amZkZk8mcNWtWQUFBb29vXl4eQoi4J6vwkGdgYICcvYpA/Nhnz56NuK/RBZIm0D5irIlCiRMLC4vu7m6txDc1NW1ra9NKKL3p6+tDCCmfS4rFYhUUFGAY9vHHH8uftWnSnyKRCCG0fft2crjlw4cP1aiS5e3tbWJicu/ePYQQcRNZKBSSa8VicV9fH/k+MYHIocQPf5VA0gTaZ2FhgRBS+C/d2dnp6OioeXCJRKKtUPpEZJARh3z7+vpu3Lixrq4uPT2dXKhJf1pbWyOEsrOz5e/KVVZWUm2/TCaTyWRE0nd2dubxeA8fPiTX3r9/H71UfIsobq1w+vkKgKQJtG/y5MlmZma//fYbueTatWsDAwPkrMJ0Ol3tYlQVFRU4jpO1uzUJpU82NjYYhqkyEjM9PX3ChAk3b94kl4zYn0oQkwlWVVVRbfD7778v/5F4duTr64sQotPp8+fPv3TpEjn/VXl5OYZhCgMDiB9L1Md5lUDSBNrHYrHi4+NLSkqOHDkiFApv3769du1ae3v7mJgYYgM3N7eOjo7S0lKJRNLW1iZ/zoKGKvQnk8levHgxODhYXV0dFxcnEAiioqLUCEW1WJ8WcTgcFxeXJ0+ejLglcZEu/5hlxP5UHm3VqlWFhYX5+flCoVAqlT558uTp06cIocjISFtb2+Fe02xqaioqKurs7JRIJJWVlatXrxYIBORrP8nJyc+ePduxY4dIJKqsrMzMzIyKivL09JSPQPxYb2/vERs5yhjkmT3B4EMHgHqQCkOOZDJZZmamu7s7g8GwtLQMDQ2tra0l1z5//vydd95hsVjOzs6fffZZQkICQsjNzY0YSCRf6K+lpSUmJobBYDg4ONDpdD6fv3Dhwvr6evVCKSnW9zKtH5+xsbEMBkMsFhMfh6zaR0pISJAfcqSkP/Py8ohHLu7u7vX19YcOHeLz+QihcePG3bt3D8fx/v7+xMREgUBAp9Otra3DwsJqampwHA8NDUUIpaSkDNna+Ph4V1dXLpdLp9MdHR2jo6Obm5vlNyAm0zY1NbW3t09ISFCoWIjj+IIFCxwcHGQymfpdNhSD5w1ImoAyVZKmFsXExIwZM0ZvuyNp/fisq6uj0+kK7yYakFQqnTt37uHDh3URvL29ncVi7d+/X+uRDZ434PIcjAKvRskcNze3tLS0tLS0np4eQ7cFSaXS0tLS7u5uHVXzSk1NnTp1amxsrC6CGxYkTQD0JykpKTw8PDIy0uC1OSoqKk6ePFleXq586Kh6srKyqqqqzp07R0yX/YqBpAmM2tatWwsKCrq6upydnU+cOGHo5mjBrl27YmNj9+zZY9hmBAQEHD16lHxtX4vKysr6+/srKirUnuvUyI2+KXzBa2X37t27d+82dCu0LDAwMDAw0NCt0JWQkJCQkBBDt0KH4EwTAAAogKQJAAAUQNIEAAAKIGkCAAAFkDQBAIAKAw6sX7x4saF/PQBg9DHsG0EYbrgZPCorKx8/fmyovQNjlp2djRDasGGDoRsCjJGTkxNRb8kgDJk0ARjOkiVLEELHjh0zdEMAUAT3NAEAgAJImgAAQAEkTQAAoACSJgAAUABJEwAAKICkCQAAFEDSBAAACiBpAgAABZA0AQCAAkiaAABAASRNAACgAJImAABQAEkTAAAogKQJAAAUQNIEAAAKIGkCAAAFkDQBAIACSJoAAEABJE0AAKAAkiYAAFAASRMAACiApAkAABRA0gQAAAogaQIAAAWQNAEAgAJImgAAQAEkTQAAoACSJgAAUABJEwAAKICkCQAAFEDSBAAACiBpAgAABXRDNwAAhBC6du3arVu3yI8PHjxACB06dIhc4uPj89e//tUALQPgf2E4jhu6DQCgM2fOBAUFmZiY0Gg0hBBxWGIYhhCSyWRSqfT06dN/+9vfDNxKACBpAiMhkUisrKyEQuGQa/l8fltbG5PJ1HOrAHgZ3NMERoHBYHz44YdDpkUlqwDQP0iawFh8+OGHAwMDLy+XSCRLly7Vf3sAGBJcngNjIZPJ3njjjWfPnikst7a2bmlpIe51AmBwcCACY0Gj0VasWKFwGc5kMqOioiBjAuMBxyIwIi9foQ8MDHz44YeGag8AL4PLc2Bc3N3d79+/T350cXGpr683YHsAUABnmsC4LF++nMFgEP9mMpkrV640bHsAUABnmsC43L9/393dnfxYW1vr4eFhwPYAoADONIFxcXNz8/HxwTAMwzAfHx/ImMDYQNIERuejjz4yMTExMTH56KOPDN0WABTB5TkwOs3NzU5OTjiOP3782MHBwdDNAeB/6C9pZmVlVVZW6mdfYLSrqKhACL399tsGbgcYJXx9fTdu3Kiffenv8ryysvLq1at62x0Y1QQCwbhx465evfo6HDNPnjw5ceKEoVsxil29elWfJ2R6rac5a9as48eP63OPYJTq6OhA6P+3d+9BUVx74sBPw7wYmOEhzyugvMSggCHqBdTFLBVylRVENFKr3hBXazQxiCKLoCACvoIXKLywlldCqtQVUCkgEVzLu4spNqxJShEdrwoIIhAEDDDMi8dM//7oX/r2HWGYHpgXfj//TZ8zp0+far70dJ/+HiQQCBBCc/6cqaio2Lp165w/TN3ZsmWLPncHSYiBMbKzszN0FwCYHDw9BwAAGiBoAgAADRA0AQCABgiaAABAAwRNMNfU1tZaW1t/++23hu6Irty5cyc1NfXGjRuenp7E+6Y7duygVoiIiODxeObm5kuWLLl//75BOpmTk4P9o6VLl1IrNDQ0rFq1isvluri4pKSkjI6OEttramrOnDmjUCgM0WuNQNAEc83cfsnt2LFjhYWFaWlpsbGxL1688PLymjdv3uXLl2/evEnWuX379rVr1zZs2CAUCoOCggzY26kIhcKIiIjw8PD+/v7Kysqvv/567969RFFUVBSHwwkPDx8aGjJsJ6cCQRPMNZGRkcPDwxs2bND1jmQyWWhoqK73QnX69OmysrKKigoej0duLCwsNDMzEwgEw8PD+uzMtC5duoRTPH78mCzKzs52dnY+fvy4paVlSEhISkrKN9988/TpU6J0//79gYGB69evn5iYMFDf1YGgCYCWSkpK+vr69La71tbW9PT048ePczgc6vbQ0NDExMTu7u5Dhw7prTMzMTExcfPmzbCwMGJde4TQunXrcByvrq4m62RmZjY1NRUUFBioj+pA0ARzSkNDg7u7O4Zhf/7znxFCxcXFlpaWXC63urp63bp1fD7f1dX16tWrROXCwkIOh+Po6Lhnzx4XFxcOhxMaGnrv3j2iNCEhgcViOTs7Ex+/+OILS0tLDMMGBgYQQomJiUlJSW1tbtj5AAAAIABJREFUbRiGeXt7I4Ru3brF5/NPnDiho0MrLCzEcTwqKurtopycnEWLFl28ePHOnTuTfhfH8by8vPfee4/NZtva2m7cuJG8rFM/RAghhUKRkZHh7u5uYWEREBBQXl4+wwN58eKFWCx2d3cnt3h5eSGEmpubyS22trZhYWEFBQVGeLMFgiaYU1avXv3DDz+QHz///PMDBw7IZDIej1deXt7W1ubp6bl79+7x8XGEUEJCQnx8vFQq3b9/f0dHx/379ycmJj766KNXr14hhAoLCz/55BOyqaKiouPHj5MfCwoKNmzY4OXlheM4sT4H8exCqVTq6NBu3rzp6+vL5XLfLrKwsPjmm2/MzMx2794tkUjerpCZmZmamnrkyJG+vr7vv//+1atXa9asIRb+VD9ECKHDhw9/9dVX+fn5v/zyy4YNG/71X//1559/1qTDqamptra2LBbLw8Nj48aNP/30E7G9t7cXIUS9w8DhcCwsLFQWIn3//fe7u7sfPnyo0ejoEQRN8E4IDQ3l8/kODg5xcXESiaSzs5MsYjAYxCWYn59fcXHxyMhIaWmpFruIjIwUiUTp6emz1+u/k0gk7e3txBXZpEJCQg4cONDR0XH48GGVIplMlpeXt2nTpu3bt1tbW/v7+58/f35gYODChQvUapMOkVwuLy4ujomJiY2NtbGxOXr0KJPJ1GR8Pv3005qamlevXonF4qtXr3Z2doaFhQmFQoQQ8aDc3NycWp/JZMpkMuoWIoH/o0ePpt2XnkHQBO8WYolg8jJKxfLly7lcLvnT1Xj09fXhOD7pZSYpJyfH19e3qKiooaGBul0oFIrF4uXLl5NbVqxYwWKxyBsRKqhD9OzZM6lUSs4WsrCwcHZ21mR83Nzc3n//fSsrKxaLFRwcXFpaKpPJioqKEELEPVmVhzxjY2MWFhbULcTBqlx+GgMImgD8Azab3d/fb+heqJLL5QghNputpg6HwyktLcUwbOfOndSrNmLujpWVFbWyjY3NyMjItPslfuwfPXqUnG758uVLqVRKt//+/v7m5ubPnz9HCBG3iUUiEVkqlUrlcrmLiwv1K0QMJQ7cqEDQBODvxsfHh4aGXF1dDd0RVUQEmXbKN5GLt6WlJTs7m9xoY2ODEFIJkRoepoODA0IoPz+fOnlIi+SVSqVSqVQSQd/Dw4PH4718+ZIsJW4KBwQEUL8yNjaGfjtwowJBE4C/q6+vx3E8ODiY+MhgMKb6Ia9njo6OGIZpMhMzOzt78eLFDx48ILcsXbrUysqK+vTm3r17Y2NjH3zwwbStubm5cTicpqYmuh3++OOPqR9/+uknHMdDQkIQQgwGY/369d9//z350Kyurg7DMJWJAcTBOjk50d21rkHQBO86pVI5ODg4MTHR3NycmJjo7u4eHx9PFHl7e//6669VVVXj4+P9/f3UiyOEkJ2dXU9PT0dHx8jIyPj4eF1dne6mHHG5XE9Pz66urmlrEj/SqY9ZOBxOUlJSZWXl5cuXRSLRo0eP9u7d6+LiQuR4nra1zz777OrVq8XFxSKRSKFQdHV1/fLLLwihuLg4JyenqV7T7O7uLisrGxoaGh8fb2xs3LVrl7u7O/naT3p6+uvXr48dOyaRSBobG3Nzc+Pj4319faktEAfr7+8/bSf1DdeXzZs3b968WW+7A3OAFufMuXPniFtmXC43KiqqqKiIeJ7g4+PT1tZ24cIFPp+PEFqwYMHz589xHBcIBEwmc/78+QwGg8/nb9y4sa2tjWztzZs3H374IYfD8fDw+PLLL5OTkxFC3t7enZ2dOI7fv39/wYIFFhYWq1ev7u3tra2t5fF4OTk5dA+TmPk4bbWEhAQmkymVSomPlZWVxMN0e3v7ffv2qVROTk6Ojo4mPyqVytzcXB8fHyaTaWtrGxMT8+zZM6Jo2iEaHR1NSUlxd3dnMBgODg6xsbFCoRDH8ZiYGIRQRkbGpL1NSkry8vKytLRkMBiurq67d+/u6emhVrh79+7KlSvZbLaLi0tycrJcLldpITIycv78+UqlctqR0XNsgaAJjJcezhmBQGBnZ6fTXUxLw6DZ0tLCYDBU3k00IIVCsWbNmpKSEl00PjAwwOFwzp49q0llPccW+HkO3nXGnFCHytvbOysrKysrSywWG7ovSKFQVFVVjYyMxMXF6aL9zMzMZcuWJSQk6KLxGYKgCYDJSE1N3bJlS1xcnMFzc9TX19+4caOurk791FHt5OXlNTU11dbWMpnMWW985ow6aO7atYvH42EYpsXDO6OSlZXl5+fH5/PZbLa3t/e///u/v32xoFQq8/PzaWXNoWZUJLBYLEdHx7Vr1+bm5g4ODs7qQcxBaWlppaWlw8PDHh4eprKI7okTJxISEk6dOmXYboSHh1+5coV8MX8WVVdXj46O1tfX29raznrjs0NvNwK0u+9AJA548OCBLrqkN2FhYUVFRW/evBGJROXl5Uwm8w9/+AO1wvPnz1etWoUQCgwMpNu4l5eXtbU1juPEU+D/+Z//iY+PxzDMxcWFmOdhut6R++Aa3tMEU4F7mqaBVi5FKysr4oEDj8f75JNPYmJibt26RWSFQAg9fPjw8OHDe/fuXbZs2Uy6hGGYjY3N2rVrS0tLKyoqXr9+TWSWnEmbuqD/NJQAzCJjD5pkxj1jQyuX4nfffUedN2dvb48QIt9FCwwMvHHjxrZt29S/JEfL5s2b4+Pj+/r6zp8/P1ttzhY9p6EEYHYZXdDEcTw3N9fX15fNZltbWxMz4whfffUVl8vl8Xh9fX1JSUnz588n5ppNlShQfbZEpDbJIN1cirR0d3dbWFh4eHhoUlnrLI3EDO26ujo0h4YOAMPT240ADe87HDlyBMOwP/3pT4ODg1KplEiLQt7TPHLkCEJo//79586d27Rp09/+9reMjAwWi3Xp0qWhoaHm5uagoCB7e/ve3l6ivkAgsLS0fPLkiVwuFwqFK1as4PF4xMxkHMfVf3fbtm1OTk5kx3JzcxFC/f39xMfY2FgilyJdEomEx+MlJCS8XfT73//+7Xua3333HY/Hy8rKmqpB8p6mCiIhgpubG/HRFIcO7mkCTbzTk9ulUimXy/3oo4/ILSoPgoi/fJlMRta3srKKi4sj6//4448IITLECAQCakAh0qAeP35ck+/qKGgeOXJk0aJFIpHo7aJJg+a0pgqaOI4TdznJ/Zrc0EHQBJrQ83nC0POFrXqtra1SqTQ8PFzD+nQTBVKzJdL97qyorKysqKi4ffs2NW21jkgkEhzHiVfi3mYqQ3f9+nWjva89u96Rw9SRzZs3621fxhU0iVf0iWxUmtAiUSCZLXEmSQa1U1ZWlpeXV19f/7vf/U5Hu6AichcuXrx40lJTGbrg4OADBw7MerNGpbGxsaCgYOZr77yz8vPz9bk74wqaREpnctn4adFNFEjNljiTJINaOHfu3H/913/993//t0qs0Z1bt24hhNatWzdpqakMnaurK3WhnrmqoKDgXThMHbl27Zo+d2dcT8+XLl1qZmZ29+5dzevTShRIzZY47XdnK5cijuMpKSmPHj2qqqrSW8Ts7e3Nz893dXXduXPnpBVMYugAMELGFTSJxFPXr18vKSkRiUTNzc0qaz+p0CRR4FTZEqf9Lq1cimo6+eTJk6+++uovf/kLk8mkvvJ49uxZTcZEkyyNOI6LxWIiiVZ/f395efmqVavMzc2rqqqmuqdpEkMHgDHS2yMnDZ9wjYyM7Nq1a968eVZWVqtXr87IyEAIubq6Pnz48MyZM0Tuezc3NzJBlppEgfh02RLVf5dWLkU1RzTVcnq5ublEhcbGxlWrVpELpDg7O4eGht69e5coVZOlsaamJiAggMvlslgsMzMz9NtLQStXrszKynrz5g1Z00SHDp6eA03o+TzBcH2txb5lyxak97sPe/bsuXbt2ps3b/S507nBGIbOIOeM/lVUVGzdulVvf4lzj57PE+P6ea4LppIt0QjB0AHwtrkfNHXt6dOn2NR0lKIVvMvu3LmTmppKTQy4Y8cOaoWIiAgej2dubr5kyZKp1vDRtZycHJW/BXLxdEJDQ8OqVau4XK6Li0tKSgo5Z6ampubMmTPG/A97LgdN/WRLXLx4sZrbH2VlZTrar06ZYqLJd8SxY8cKCwvT0tJiY2NfvHjh5eU1b968y5cv37x5k6xz+/bta9eubdiwQSgUBgUFGbC3UxEKhREREeHh4f39/ZWVlV9//TW55lpUVBSHwwkPDyemAxuhuRw0T548OTo6iuN4e3u7Pl8YmAPenaGbxTx1ekh5d/r06bKysoqKCuobZYWFhWZmZgKBwNjSAKosZ/T48WOyKDs729nZ+fjx45aWliEhISkpKd988w2Z82X//v2BgYHr16+fmJgwUN/VmctBE4BpzWKeOl2nvGttbU1PTz9+/DjxDggpNDQ0MTGxu7v70KFDutv7LJqYmLh582ZYWBj55ui6detwHK+uribrZGZmNjU1FRQUGKiP6kDQBCYPn6U8deoT4tFNead1Tr+pFBYW4jgeFRX1dlFOTs6iRYsuXrx4584dukNUXFxsaWnJ5XKrq6vXrVvH5/NdXV2JRDkEhUKRkZHh7u5uYWEREBAw89c9X7x4IRaL3d3dyS3EWsTNzc3kFltb27CwsIKCAmOcVKCbmUyTeEfm3IFZpOE5M4t56tQnxKPV1LQ5/UgaztP09PT08/NT2ejl5dXe3o7j+A8//GBmZrZw4UKxWIzjeF1dHXXdc/VDRGTA+utf/zo8PNzX17dmzRpLS8uxsTGi9NChQ2w2+/r164ODg2lpaWZmZposo5Kdne3q6mpjY8NkMhcuXBgdHf3jjz8SRcQrf+Q8ZYKFhUV4eDh1S2pqKtJsqRtY7gIAGmQyWV5e3qZNm7Zv325tbe3v73/+/PmBgQH175KpwWAwiCsyPz+/4uLikZGR0tJSLdqJjIwUiUTp6enadUOFRCJpb28nrsgmFRIScuDAgY6OjsOHD6sUaThEoaGhfD7fwcEhLi5OIpF0dnYihORyeXFxcUxMTGxsrI2NzdGjR5lMpiYD8umnn9bU1Lx69UosFl+9erWzszMsLEwoFKLfkktQ1zJACDGZTJlMRt3i4+ODEJrq3RADgqAJTJtO89RRE+IZVl9fH47j6tfLzcnJ8fX1LSoqamhooG6nO0QsFgshRLzh+uzZM6lUSs4WsrCwcHZ21mRA3Nzc3n//fSsrKxaLFRwcXFpaKpPJiJzixD1ZlYc8Y2NjxEtrJOJgX79+Pe2+9AyCJjBtus5TRybEMyy5XE50Rk0dDodTWlqKYdjOnTupV20zGSKJRIIQOnr0KDnd8uXLl+TyVprz9/c3Nzcn0hUS94WJlQUIUqlULpeTbxITiBhKHLhRgaAJTJtO89RRE+IZFhFBpp3yHRIScvDgwZaWluzsbHLjTIaISG6bn59PvanX2NhIt/9KpVKpVBJB38PDg8fjUdO4tLa2IoQCAgKoXxkbG0O/HbhRgaAJTJtO89RRE+LNsKkZcnR0xDBMk5mY2dnZixcvfvDgAbmFbhpAKjc3Nw6H09TURLfDH3/8MfUj8ewoJCQEIcRgMNavX//9998rlUqitK6uDsMwlYkBxME6OTnR3bWuQdAEpm3W89RNlRCPblOa5PTTHJfL9fT0JJY2mHZASktLqY9ZNEkDqKa1zz777OrVq8XFxSKRSKFQdHV1/fLLLwihuLg4JyenqV7T7O7uLisrGxoaGh8fb2xs3LVrl7u7O/naT3p6+uvXr48dOyaRSBobG3Nzc+Pj4319faktEAfr7+8/bSf1TW/P6WHKEaBLw3NmFvPUqU+IR6spNTn9VGg45SghIYHJZEqlUuJjZWUl8TDd3t5+3759KpWTk5OpU47UDFFRURHxyMXHx6etre3ChQtEDtYFCxY8f/4cx/HR0dGUlBR3d3cGg0FkvBUKhTiOx8TEIIQyMjIm7W1SUpKXl5elpSWDwXB1dd29e3dPTw+1wt27d1euXMlms11cXJKTk+VyuUoLkZGR8+fPJ7LEqvdOr0YJAJX+zxmBQGBnZ6fPPeIaB82WlhYGg6HybqIBKRSKNWvWlJSU6KLxgYEBDodz9uxZTSrDPE0ADMlo8+t4e3tnZWVlZWWJxWJD9wUpFIqqqqqRkREd5fHKzMxctmxZQkKCLhqfIQiaAJiM1NTULVu2xMXFGTw3R319/Y0bN+rq6tRPHdVOXl5eU1NTbW0tk8mc9cZnDoImAP+fSSTEO3HiREJCwqlTpwzbjfDw8CtXrpBv4s+i6urq0dHR+vp6W1vbWW98VhjXEr4AGNDJkydPnjxp6F5MLyIiIiIiwtC90JXo6Ojo6GhD90IduNIEAAAaIGgCAAANEDQBAIAGCJoAAECDXh8EdXV1VVRU6HOPwKQRL9LN+XOGyH8x5w9Td7q6uvSaVEVv0+jn9vpcAAAD0ucbQRhuhEtwgHfeJ598guDiCxgluKcJAAA0QNAEAAAaIGgCAAANEDQBAIAGCJoAAEADBE0AAKABgiYAANAAQRMAAGiAoAkAADRA0AQAABogaAIAAA0QNAEAgAYImgAAQAMETQAAoAGCJgAA0ABBEwAAaICgCQAANEDQBAAAGiBoAgAADRA0AQCABgiaAABAAwRNAACgAYImAADQAEETAABogKAJAAA0QNAEAAAaIGgCAAANEDQBAIAGCJoAAEADBE0AAKABgiYAANAAQRMAAGiAoAkAADRgOI4bug8AoCtXrpSUlCiVSuJje3s7QsjDw4P4aGZm9m//9m/btm0zWP8A+A0ETWAUmpubAwMD1VR4+PBhQECA3voDwFQgaAJjsXjx4mfPnk1a5O3t3dLSouf+ADApuKcJjMWOHTuYTObb25lM5meffab//gAwKbjSBMbixYsX3t7ek56QLS0t3t7e+u8SAG+DK01gLDw9PYOCgjAMo27EMGz58uUQMYHxgKAJjMgf//hHc3Nz6hZzc/M//vGPhuoPAG+Dn+fAiPT19bm4uJATjxBCZmZmPT09Tk5OBuwVAFRwpQmMiKOjY1hYGHmxaW5uvnbtWoiYwKhA0ATGZceOHdRfPzt27DBgZwB4G/w8B8ZFJBI5ODiMjY0hhJhMZl9fn42NjaE7BcDfwZUmMC58Pv8Pf/gDg8FgMBjr16+HiAmMDQRNYHS2b9+uUCgUCgW8bA6MEPw8B0ZHLpfb29vjOD4wMGBhYWHo7gDwD0wsaKrMfAYAzAGmFYUYhu4AbYmJiSEhIYbuBZhGY2NjQUFBeXm5dl9vamrCMEx93iMjsXXrVjgntUacJ4buBT2md6VZXl7+ySefGLojYBoVFRVbt27V+uyamJhACDEYJvBPHc7JmZjheWIQJnBSgneQSYRL8G6Cp+cAAEADBE0AAKABgiYAANAAQRMAAGiAoAmMSG1trbW19bfffmvojujKnTt3UlNTb9y44enpiWEYhmEqGUkiIiJ4PJ65ufmSJUvu379vkE7m5ORg/2jp0qXUCg0NDatWreJyuS4uLikpKaOjo8T2mpqaM2fOKBQKQ/RafyBoAiNiWlNP6Dp27FhhYWFaWlpsbOyLFy+8vLzmzZt3+fLlmzdvknVu37597dq1DRs2CIXCoKAgA/Z2KkKhMCIiIjw8vL+/v7Ky8uuvv967dy9RFBUVxeFwwsPDh4aGDNtJnYKgCYxIZGTk8PDwhg0bdL0jmUwWGhqq671QnT59uqysrKKigsfjkRsLCwvNzMwEAsHw8LA+OzOtS5cu4RSPHz8mi7Kzs52dnY8fP25paRkSEpKSkvLNN988ffqUKN2/f39gYOD69euJmbZzEgRN8C4qKSnp6+vT2+5aW1vT09OPHz/O4XCo20NDQxMTE7u7uw8dOqS3zszExMTEzZs3w8LCyBea161bh+N4dXU1WSczM7Opqcnk3vPRHARNYCwaGhrc3d0xDPvzn/+MECouLra0tORyudXV1evWrePz+a6urlevXiUqFxYWcjgcR0fHPXv2uLi4cDic0NDQe/fuEaUJCQksFsvZ2Zn4+MUXX1haWmIYNjAwgBBKTExMSkpqa2vDMIxYsu3WrVt8Pv/EiRM6OrTCwkIcx6Oiot4uysnJWbRo0cWLF+/cuTPpd3Ecz8vLe++999hstq2t7caNG8nLOvVDhBBSKBQZGRnu7u4WFhYBAQFav9VKevHihVgsdnd3J7d4eXkhhJqbm8kttra2YWFhBQUFc/ZmC25SEELl5eWG7gWYHvH3Sfdbr169QgidO3eO+HjkyBGE0F//+tfh4eG+vr41a9ZYWlqOjY0RpQKBwNLS8smTJ3K5XCgUrlixgsfjdXZ2EqXbtm1zcnIiW87NzUUI9ff3Ex9jY2O9vLzI0u+++47H42VlZWlxpJqck56enn5+fiobvby82tvbcRz/4YcfzMzMFi5cKBaLcRyvq6uLjo4mq2VkZLBYrEuXLg0NDTU3NwcFBdnb2/f29hKl6ofo0KFDbDb7+vXrg4ODaWlpZmZmP/3007RHlJ2d7erqamNjw2QyFy5cGB0d/eOPPxJFd+/eRQjl5uZS61tYWISHh1O3pKamIoQePHgw7b60O08MC640gbELDQ3l8/kODg5xcXESiaSzs5MsYjAYxCWYn59fcXHxyMhIaWmpFruIjIwUiUTp6emz1+u/k0gk7e3txBXZpEJCQg4cONDR0XH48GGVIplMlpeXt2nTpu3bt1tbW/v7+58/f35gYODChQvUapMOkVwuLy4ujomJiY2NtbGxOXr0KJPJ1GR8Pv3005qamlevXonF4qtXr3Z2doaFhQmFQoQQ8aBcZcVQJpMpk8moW3x8fBBCjx49mnZfpgiCJjAZLBYLITQ+Pj5p6fLly7lcLvnT1Xj09fXhOM7lctXUycnJ8fX1LSoqamhooG4XCoVisXj58uXklhUrVrBYLPJGhArqED179kwqlZKzhSwsLJydnTUZHzc3t/fff9/KyorFYgUHB5eWlspksqKiIoQQcU9W5SHP2NiYStpT4mBfv3497b5MEQRNMHew2ez+/n5D90KVXC5HCLHZbDV1OBxOaWkphmE7d+6kXrURc3esrKyolW1sbEZGRqbdr0QiQQgdPXqUnG758uVLqVRKt//+/v7m5ubPnz9HCBG3iUUiEVkqlUrlcrmLiwv1K0QMJQ587oGgCeaI8fHxoaEhV1dXQ3dEFRFBpp3yHRIScvDgwZaWluzsbHIjsUSSSojU8DAdHBwQQvn5+dT7cY2NjXT7r1QqlUolEfQ9PDx4PN7Lly/J0tbWVoRQQEAA9SvEunhzNes+BE0wR9TX1+M4HhwcTHxkMBhT/ZDXM0dHRwzDNJmJmZ2dvXjx4gcPHpBbli5damVl9fPPP5Nb7t27NzY29sEHH0zbmpubG4fDaWpqotvhjz/+mPqReHZEZFkmVrv7/vvvlUolUVpXV4dhmMrEAOJg5+qC9RA0gQlTKpWDg4MTExPNzc2JiYnu7u7x8fFEkbe396+//lpVVTU+Pt7f30+9OEII2dnZ9fT0dHR0jIyMjI+P19XV6W7KEZfL9fT07OrqmrYm8SOd+piFw+EkJSVVVlZevnxZJBI9evRo7969Li4uAoFAk9Y+++yzq1evFhcXi0QihULR1dX1yy+/IITi4uKcnJymek2zu7u7rKxsaGhofHy8sbFx165d7u7u5Gs/6enpr1+/PnbsmEQiaWxszM3NjY+P9/X1pbZAHKy/v/+0nTRJBnlmrzUEU45MhBZTSc6dO0fcMuNyuVFRUUVFRcTzBB8fn7a2tgsXLvD5fITQggULnj9/juO4QCBgMpnz589nMBh8Pn/jxo1tbW1ka2/evPnwww85HI6Hh8eXX36ZnJyMEPL29ibmJN2/f3/BggUWFharV6/u7e2tra3l8Xg5OTlaHKkm52RCQgKTyZRKpcTHyspK4mG6vb39vn37VConJydTpxwplcrc3FwfHx8mk2lraxsTE/Ps2TOiaNohGh0dTUlJcXd3ZzAYDg4OsbGxQqEQx/GYmBiEUEZGxqS9TUpK8vLysrS0ZDAYrq6uu3fv7unpoVa4e/fuypUr2Wy2i4tLcnKyXC5XaSEyMnL+/PlKpVL9sOCmOeXI1LoLQdNE6OGPQSAQ2NnZ6XQXmtDknGxpaWEwGCrvJhqQQqFYs2ZNSUmJLhofGBjgcDhnz57VpLIpBk34eQ5MmKkk1PH29s7KysrKyhKLxYbuC1IoFFVVVSMjI3FxcbpoPzMzc9myZQkJCbpo3BjM8aC5a9cuHo+HYZgWt8N1ZO3atdhbVOaUTIqaT4zAYrEcHR3Xrl2bm5s7ODioh84DraWmpm7ZsiUuLs7guTnq6+tv3LhRV1enfuqodvLy8pqammpra5lM5qw3biTmeNC8ePHiX/7yF0P3YnqrV6+etg6ZT8za2hrHcaVS2dfXV1FR4eHhkZKSsmTJEuoz1jkvLS2ttLR0eHjYw8Pj+vXrhu6ORk6cOJGQkHDq1CnDdiM8PPzKlSvki/mzqLq6enR0tL6+3tbWdtYbNx6w5p++cTgckUhEzQ+2Z88eLRaAxTDMxsZm7dq1a9eujYyM3Lp1a2Rk5PPnz62trWe1v0bq5MmTJ0+eNHQvaIuIiIiIiDB0L3QlOjo6Ojra0L3QuTl+pYkQInNYGYlbt25RI+arV68eP378z//8zzNpc/PmzfHx8X19fefPn59xBwEA6szBoInjeG5urq+vL5vNtra2JuaakCZNljVtii1ijgWXy+Xz+f7+/sRrZLOSd+v06dP79+8nP2qdo4yYn1hXV2echwnA3GHox/f0IA2mdxw5cgTDsD/96U+Dg4NSqZRINEBmqZoqWZaaFFtisZjP5585c0Ymk/X29m7atInIMKZd3i2qrq4uPz8/hUJBbpk2Rxl5T1MFEeDc3NyM5DBNcSqJdjQ5J8FUTPE8MbXuTneCSqVSLpf70UdARtS2AAAJuklEQVQfkVuIKykiaMpkMi6XGxcXR1Zms9mff/45/ls0kclkRBERaltbW/Hfcv1/99131B2paUpz+/bt+4//+A9aX5kqaOI4TtzlVN83vR2mKf4xaAeC5kyY4nky1x4Etba2SqXS8PDwSUs1T5ZFTbHl6enp6Oi4ffv2/fv3x8fHL1y4kFZTU+np6ampqSGS486cRCLBcZx4IcR4DrOiomKmB2YKtMiCAQgmOXSGjtr0oOn+q9fW1iKEqK86UK80//d///ftEQgODsbfugQjJir97W9/Iz4+fvz4X/7lXxgMBoZhW7dulUqlaprSUEJCQnZ2Nt0RmOpKk3iPOCIiwkgOE259As3R/SswrLn2IIhIkkouxKxC62RZS5Ys+fbbb3t6elJSUsrLy8+ePTvDvFu9vb3/+Z//+fnnn2t6YNO5desWQmjdunXImA5Tdyeu8UDw83wGTPGf61wLmkuXLjUzMyNWMnmbdsmyenp6njx5ghBycHA4depUUFDQkydPtM67RThz5sz27dvt7Oy0+7qK3t7e/Px8V1fXnTt3ImM6TADmnrkWNIlULtevXy8pKRGJRM3NzdTVVNQky1Kjp6dnz549T58+HRsbe/DgwcuXL4ODg7VrivD69euvv/76wIEDbxdpkqMMx3GxWEykkOnv7y8vL1+1apW5uXlVVRVxT9NIDhOAucnQl+f0IA1+Co2MjOzatWvevHlWVlarV6/OyMhACLm6uj58+BCfIlmW+hRbHR0doaGhtra25ubmv/vd744cOTIxMTFVU5ocxcGDB7dv3z5pkZocZTU1NQEBAVwul8VimZmZod9eClq5cmVWVtabN2+olQ1+mKb4VFQ7mpyTYCqmeJ5guEmtTYxhWHl5uRYvHQI9q6io2Lp1q2mdXdqBc3ImTPE8mWs/zwEAQKcgaM6mp0+fvp32jaSj9IUAAH2CoDmbFi9erOZWSFlZmaE7CIzOnTt3UlNTqclSd+zYQa0QERHB4/HMzc2XLFky1ao++qFUKvPz80NDQ6kba2pqzpw5YyrZoGcFBE0ADObYsWOFhYVpaWlkstR58+Zdvnz55s2bZJ3bt29fu3Ztw4YNQqEwKCjIUF1taWn5p3/6p4MHD6qsnB4VFcXhcMLDw4kl2t8FEDSBqZLJZCpXPcbQlOZOnz5dVlZWUVFBTRVYWFhoZmYmEAgMnuCd6uHDh4cPH967d++yZcveLt2/f39gYOD69esnJib03zf9g6AJTFVJSUlfX5+xNaWh1tbW9PT048ePE++wkUJDQxMTE7u7uw8dOqTP/qgXGBh448aNbdu2sdnsSStkZmY2NTUVFBTouWMGAUETGBKO43l5ee+99x6bzba1td24cSOZDSQhIYHFYpGrMnzxxReWlpYYhg0MDCCEEhMTk5KS2traMAzz9vYuLCzkcDiOjo579uxxcXHhcDihoaH37t3Toik0g6ymmissLMRxPCoq6u2inJycRYsWXbx48c6dO5N+V82gTZsyVUfZUW1tbcPCwgoKCkxr8pCWdD8VdDYhmEhsIjSctJyRkcFisS5dujQ0NNTc3BwUFGRvb9/b20uUbtu2zcnJiaxMZIQisnziOB4bG+vl5UWWCgQCS0vLJ0+eyOVyoVC4YsUKHo9HrHJOt6lps5pSaXdOenp6+vn5qWz08vJqb2/HcfyHH34wMzNbuHChWCzGcbyuro66Err6QVOTMhWfcRLY3//+94GBgZMWpaamIkriWg2Z4uR2uNIEBiOTyfLy8jZt2rR9+3Zra2t/f//z588PDAxQ33ylhcFgENdffn5+xcXFIyMjpaWlWrQTGRkpEonS09O168a0JBJJe3u7l5fXVBVCQkIOHDjQ0dFx+PBhlSINBy00NJTP5zs4OMTFxUkkks7OToSQXC4vLi6OiYmJjY21sbE5evQok8nUboje5uPjgxB69OjRrLRmzCBoAoMRCoVisXj58uXklhUrVrBYLPJn9UwsX76cy+XSynCqN319fTiOq19BNycnx9fXt6ioqKGhgbqd7qBRU6bOPAmsGsThvH79elZaM2YQNIHBEJNUVNZ8t7GxGRkZmZX22Wx2f3//rDQ1u+RyOUJoqocqBA6HU1paimHYzp07ZTIZuX0mgyaRSBBCR48eJV+4ePnypcoUIq1ZWFig3w5tboOgCQzGxsYGIaTy1z40NOTq6jrzxsfHx2erqVlHxJdpJ4SHhIQcPHiwpaUlOzub3DiTQZthdlT1xsbG0G+HNrdB0AQGs3TpUisrq59//pnccu/evbGxsQ8++ID4yGAwiN+VWqivr8dxPDg4eOZNzTpHR0cMwzSZiZmdnb148eIHDx6QW6YdNDV0mh2VOBwnJyddNG5UIGgCg+FwOElJSZWVlZcvXxaJRI8ePdq7d6+Li4tAICAqeHt7//rrr1VVVePj4/39/S9fvqR+3c7Orqenp6OjY2RkhAiISqVycHBwYmKiubk5MTHR3d2dWNmYblOaZDWdCS6X6+np2dXVNW1N4ke6ubk5dYv6QVPf2lTZUePi4pycnGbymiZxOP7+/lq3YDIM89BeWwimHJkIDaeSKJXK3NxcHx8fJpNpa2sbExPz7NkzsvTNmzcffvghh8Px8PD48ssviSXsvb29iYlE9+/fX7BggYWFxerVq3t7ewUCAZPJnD9/PoPB4PP5GzdubGtr064pNVlN36bdOZmQkMBkMqVSKfGxsrKSeJhub2+/b98+lcrJycnUKUdqBk19ylR86uyoMTExCKGMjIxJe9vY2Lhq1SoXFxciaDg7O4eGht69e5daJzIycv78+URubM2Z4pQjU+suBE0Tof8/BoFAYGdnp889ErQ7J1taWhgMxqVLl3TRJS0oFIo1a9ZQVySkZWBggMPhnD17lu4XTTFows9zMHeYUK4db2/vrKysrKwssVhs6L4ghUJRVVU1MjKidfbCzMzMZcuWJSQkzG7HjBMETQAMIzU1dcuWLXFxcQbPzVFfX3/jxo26ujr1U0enkpeX19TUVFtby2QyZ71vRgiCJpgL0tLSSktLh4eHPTw8rl+/bujuaOrEiRMJCQmnTp0ybDfCw8OvXLlCvptPS3V19ejoaH19va2t7ax3zDgxDN0BAGbByZMnT548aeheaCMiIiIiIsLQvdBedHR0dHS0oXuhV3ClCQAANEDQBAAAGiBoAgAADRA0AQCABgw3qUzLGIYFBwcbZxYGQNXV1fV///d/mzdvNnRHdO769etwTmqNOE9MLAqZVne3bNli6C4AAGbZtWvXDN0FGkwsaAIAgGHBPU0AAKABgiYAANAAQRMAAGiAoAkAADT8P+SyrQhw6ln1AAAAAElFTkSuQmCC\n", 289 | "text/plain": [ 290 | "" 291 | ] 292 | }, 293 | "metadata": {}, 294 | "execution_count": 62 295 | } 296 | ] 297 | }, 298 | { 299 | "cell_type": "markdown", 300 | "metadata": { 301 | "id": "wYn2dnj2TCm3" 302 | }, 303 | "source": [ 304 | "The training step is executed as follows.\n", 305 | "\n" 306 | ] 307 | }, 308 | { 309 | "cell_type": "code", 310 | "metadata": { 311 | "id": "i-npHi6vVdQB", 312 | "outputId": "e5286a2f-e0b2-4e2d-cc52-f2eb9c15f2af", 313 | "colab": { 314 | "base_uri": "https://localhost:8080/" 315 | } 316 | }, 317 | "source": [ 318 | "#Here we set verbose as true\n", 319 | "verbose = 1\n", 320 | "\n", 321 | "batch_size = 32\n", 322 | "epochs = 10\n", 323 | "filepath = 'weights.h5' #name of the file with the network weights\n", 324 | "monitor = 'loss'\n", 325 | "optimizer = 'adam'\n", 326 | "loss = 'mean_squared_error'\n", 327 | "metrics = ['mean_absolute_error'] \n", 328 | "\n", 329 | "lstm_model.compile(optimizer = optimizer, loss = loss, metrics = metrics)\n", 330 | "\n", 331 | "early_stopping = EarlyStopping(monitor = monitor, min_delta = 1e-15, \n", 332 | " patience = 10, verbose = verbose)\n", 333 | "reduce_learning_rate_on_plateau = ReduceLROnPlateau(monitor = monitor, \n", 334 | " factor = 0.2, patience = 5, \n", 335 | " verbose = verbose)\n", 336 | "model_checkpoint = ModelCheckpoint(filepath = filepath, monitor = monitor, \n", 337 | " save_best_only = True, verbose = verbose)\n", 338 | "lstm_model.fit(data, train_price, epochs = epochs, batch_size = batch_size,\n", 339 | " callbacks = [early_stopping, reduce_learning_rate_on_plateau, \n", 340 | " model_checkpoint])" 341 | ], 342 | "execution_count": null, 343 | "outputs": [ 344 | { 345 | "output_type": "stream", 346 | "name": "stdout", 347 | "text": [ 348 | "Epoch 1/10\n", 349 | "1450/1450 [==============================] - 6s 4ms/step - loss: 0.0567 - mean_absolute_error: 0.1899\n", 350 | "\n", 351 | "Epoch 00001: loss improved from inf to 0.05668, saving model to weights.h5\n", 352 | "Epoch 2/10\n", 353 | "1450/1450 [==============================] - 7s 4ms/step - loss: 0.0074 - mean_absolute_error: 0.0533\n", 354 | "\n", 355 | "Epoch 00002: loss improved from 0.05668 to 0.00736, saving model to weights.h5\n", 356 | "Epoch 3/10\n", 357 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0032 - mean_absolute_error: 0.0331\n", 358 | "\n", 359 | "Epoch 00003: loss improved from 0.00736 to 0.00316, saving model to weights.h5\n", 360 | "Epoch 4/10\n", 361 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0032 - mean_absolute_error: 0.0344\n", 362 | "\n", 363 | "Epoch 00004: loss did not improve from 0.00316\n", 364 | "Epoch 5/10\n", 365 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0023 - mean_absolute_error: 0.0291\n", 366 | "\n", 367 | "Epoch 00005: loss improved from 0.00316 to 0.00232, saving model to weights.h5\n", 368 | "Epoch 6/10\n", 369 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0018 - mean_absolute_error: 0.0252\n", 370 | "\n", 371 | "Epoch 00006: loss improved from 0.00232 to 0.00178, saving model to weights.h5\n", 372 | "Epoch 7/10\n", 373 | "1450/1450 [==============================] - 5s 4ms/step - loss: 0.0016 - mean_absolute_error: 0.0247\n", 374 | "\n", 375 | "Epoch 00007: loss improved from 0.00178 to 0.00164, saving model to weights.h5\n", 376 | "Epoch 8/10\n", 377 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0015 - mean_absolute_error: 0.0242\n", 378 | "\n", 379 | "Epoch 00008: loss improved from 0.00164 to 0.00149, saving model to weights.h5\n", 380 | "Epoch 9/10\n", 381 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0016 - mean_absolute_error: 0.0257\n", 382 | "\n", 383 | "Epoch 00009: loss did not improve from 0.00149\n", 384 | "Epoch 10/10\n", 385 | "1450/1450 [==============================] - 5s 3ms/step - loss: 0.0014 - mean_absolute_error: 0.0234\n", 386 | "\n", 387 | "Epoch 00010: loss improved from 0.00149 to 0.00143, saving model to weights.h5\n" 388 | ] 389 | }, 390 | { 391 | "output_type": "execute_result", 392 | "data": { 393 | "text/plain": [ 394 | "" 395 | ] 396 | }, 397 | "metadata": {}, 398 | "execution_count": 63 399 | } 400 | ] 401 | }, 402 | { 403 | "cell_type": "markdown", 404 | "metadata": { 405 | "id": "dvFufFaJMjXB" 406 | }, 407 | "source": [ 408 | "The following code verifies the data in the network." 409 | ] 410 | }, 411 | { 412 | "cell_type": "code", 413 | "metadata": { 414 | "id": "0yBjjCoCjCo3" 415 | }, 416 | "source": [ 417 | "test_price = test_dataset[:, 0:1]\n", 418 | "complete_dataset = dataset.iloc[:,1::]\n", 419 | "\n", 420 | "train_data = complete_dataset[len(complete_dataset) - len(test_dataset) - \n", 421 | " window_size:].values\n", 422 | "train_data = min_max_scaler.transform(train_data)\n", 423 | "\n", 424 | "\n", 425 | "X_test = []\n", 426 | "for i in range(window_size,len(train_data)):\n", 427 | " X_test.append(train_data[i-window_size:i, 0:5])\n", 428 | "X_test = np.array(X_test)\n", 429 | "\n", 430 | "calculated_prices = lstm_model.predict(X_test)\n", 431 | "calculated_prices = min_max_scaler_train.inverse_transform(calculated_prices)\n", 432 | "\n", 433 | "train_price = min_max_scaler_train.inverse_transform([train_price])\n", 434 | "train_price = train_price[0]" 435 | ], 436 | "execution_count": null, 437 | "outputs": [] 438 | }, 439 | { 440 | "cell_type": "markdown", 441 | "metadata": { 442 | "id": "IY8n4My_X4HO" 443 | }, 444 | "source": [ 445 | "In the following, we plot the train set, as well as the prediction and the expected values." 446 | ] 447 | }, 448 | { 449 | "cell_type": "code", 450 | "metadata": { 451 | "id": "HXWZWul0X3hH", 452 | "outputId": "00aaf3a5-fad8-4981-d96b-1d05eab6fddd", 453 | "colab": { 454 | "base_uri": "https://localhost:8080/", 455 | "height": 295 456 | } 457 | }, 458 | "source": [ 459 | "plt.plot(np.linspace(0,len(train_price)-1,len(train_price)), \n", 460 | " train_price, label = 'Real price train', color = 'k')\n", 461 | "plt.plot(np.linspace(len(train_price),len(train_price)+len(test_price)-1,\n", 462 | " len(test_price)), test_price, label = 'Real price')\n", 463 | "plt.plot(np.linspace(len(train_price),len(train_price)+len(test_price)-1,\n", 464 | " len(test_price)), calculated_prices, label = 'Prevision')\n", 465 | "plt.title('BTC prevision')\n", 466 | "plt.xlabel('Time')\n", 467 | "plt.ylabel('Value')\n", 468 | "plt.legend()\n", 469 | "plt.show()" 470 | ], 471 | "execution_count": null, 472 | "outputs": [ 473 | { 474 | "output_type": "display_data", 475 | "data": { 476 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd5xU9dX/32dnewN2QaQJqCsdQZEiohgRUayxoCaxRCU+aoqaRzE/faKPUWOiMbYHo8ZComhiAyNiEEVBREpARFGaKEtnF7awdWbO7497Z/ZuX2BnZst5v17zmnu/5d4zA3s/c77lHFFVDMMwDONgiIu1AYZhGEbrxUTEMAzDOGhMRAzDMIyDxkTEMAzDOGhMRAzDMIyDxkTEMAzDOGhMRAyjlSMiX4rI+EbaHCEixSLii5JZRjvBRMRoN4jIZhEpdR+me0XkHRHp5da965YXi0iliFR4zp8Sh1+IyBoR2S8iuSLyTxEZEuvPpaqDVHVBI22+V9V0VQ1EySyjnWAiYrQ3zlHVdKAbsBN4HEBVz3QfsunAS8AfQueqej3wKPBL4BdAFnAM8BYw+VANEpH4Q72GYcQKExGjXaKqZcBrwMDG2opIDnAjcJmqfqCq5apaoqovqerv6+mzQEQeEJGlIlIoIrNEJMut6yMiKiLXiMj3wAdu+U9FZK3rJb0nIr3d8uki8lCN688SkVvc480iMsE9Hikiy9177hSRP9W4Z7x73l1EZotIvohsEJHrPNe+W0T+ISIzRKTIHS4bcYBfsdFOMBEx2iUikgpMAZY0oflpQK6qLj3A21wB/BTH6/EDj9WoPwUYAJwhIucBvwF+CHQBFgIz3XYzgSkiIq7tnYCJwCt13PNR4FFVzQSOAv5Rj22vALlAd+Ai4H4R+YGn/ly3TUdgNvBE0z6y0d4wETHaG2+JyD6gADgd+GMT+mQD2w/iXn9T1TWquh+4C7ikxsT23aq6X1VLgeuBB1R1rar6gfuBYa43shBQYJzb7yLgU1XdVsc9K4GjRaSzqharai2RdOeBxgK3q2qZqq4CnsURvRCLVHWOO4fyN+DYg/j8RjvARMRob5yvqh2BZOAm4CMRObyRPnk43sSBssVz/B2QAHSup7438KiI7HNFLh8QoIc6UVJfAS5z216OM29TF9fgzNd8LSLLROTsOtp0B/JVtaiGfT085zs8xyVAss3dGHVhImK0S1Q1oKpvAAHgpEaazwd6HsS8QC/P8RE4XsIerxme4y3Az1S1o+eVoqqL3fqZwEWuZzIKeL2uG6rqelW9DDgMeBB4TUTSajTbBmSJSEYN+7Ye4OczDBMRo33iLtk9D+gErG2oraquB/4PmCki40UkUUSSReRSEZnWQNcfi8hAd/7lf4HXGlhi+xRwh4gMcu3rICIXe2xYiSNAzwLvqeq+ej7Xj0Wki6oGgVCbYI3PswVYDDzgfo6hOB7M3xv6HgyjLkxEjPbG2yJSDBQC9wFXquqXTej3C5zJ5SdxHs4bgQuAtxvo8zfgBZyhoWT3GnWiqm/ieA6viEghsAY4s0azl4EJ7nt9TAK+dD/jo8Cl7pxLTS4D+uB4JW8Cv1XV9xu4rmHUiVhSKsNofkRkAfB3VX021rYYRiQxT8QwDMM4aExEDMMwjIPGhrMMwzCMg8Y8EcMwDOOgaXebhzp37qx9+vSJtRmGYRitihUrVuxR1S41y9udiPTp04fly5fH2gzDMIxWhYh8V1e5DWcZhmEYB42JiGEYhnHQmIgYhmEYB027mxMxDCPyVFZWkpubS1lZWaxNMQ6Q5ORkevbsSUJCQpPam4gYhtHs5ObmkpGRQZ8+fXBzaRmtAFUlLy+P3Nxc+vbt26Q+NpxlGEazU1ZWRnZ2tglIK0NEyM7OPiAPMmIiIiK9RORDEfnKzdH8S7c8S0Tmich6972TWy4i8pib73m1iBznudaVbvv1InKlp/x4EfnC7fOY2P9Yw2gx2J9j6+RA/90i6Yn4gVtVdSAwGrhRRAYC04D5qpqDk+wnlI/hTCDHfU0FpoMjOsBvcRLxjAR+GxIet811nn6TIvh5DKPF8vXXX/Phhx/G2gyjHRIxEVHV7ar6H/e4CCfxTw/gPOBFt9mLwPnu8XnADHVYAnQUkW7AGcA8Vc1X1b3APGCSW5epqkvc9KEzPNcyjHbFgAED+MEPfhBrM1oUPp+PYcOGMXjwYM455xz27aszj1ejvPDCC9x0002HZMuJJ554SP0BFixYwOLFixtvWIPly5fzi1/Um8rmkInKnIiI9AGGA58BXVV1u1u1A+jqHveges7pXLesofLcOsrruv9UEVkuIst37959SJ/FMIzWQUpKCqtWrWLNmjVkZWXx5JNPRt0Gv98PcFAP/5o0JCKh+9TFiBEjeOyxxw75/vURcRERkXScfNC/UtVCb53rQUQ8jLCqPq2qI1R1RJcutUK/GIbRxhkzZgxbtzop5Ddu3MikSZM4/vjjGTduHF9//TUAb7/9NqNGjWL48OFMmDCBnTt3NnjNu+++m5/85CeMGTOGnJwcnnnmGcB52I8bN45zzz2XgQMHApCenh7u9+CDDzJkyBCOPfZYpk2b1qBNITZv3sxTTz3FI488wrBhw1i4cCFXXXUV119/PaNGjeK2225j6dKljBkzhuHDh3PiiSfyzTffhO05++yzwzb/9Kc/Zfz48Rx55JHNIi4RXeIrIgk4AvKSqr7hFu8UkW6qut0dktrllm8Fenm693TLtgLja5QvcMt71tHeMIwWxK9+9StWrVrVrNccNmwYf/7zn5vUNhAIMH/+fK655hoApk6dylNPPUVOTg6fffYZN9xwAx988AEnnXQSS5YsQUR49tln+cMf/sDDDz/c4LVXr17NkiVL2L9/P8OHD2fy5MkA/Oc//2HNmjW1lsm+++67zJo1i88++4zU1FTy8/MbtClEnz59uP7660lPT+fXv/41AH/961/Jzc1l8eLF+Hw+CgsLWbhwIfHx8bz//vv85je/4fXXX69lc2j+rKioiH79+vFf//VfTd4TUhcRExF3pdRfgbWq+idP1WzgSuD37vssT/lNIvIKziR6gSs07wH3eybTJwJ3qGq+iBSKyGicYbIrgMcj9XkMw2hdlJaWMmzYMLZu3cqAAQM4/fTTKS4uZvHixVx88cXhduXl5YCzt2XKlCls376dioqKJu2TOO+880hJSSElJYVTTz2VpUuX0rFjR0aOHFln//fff5+rr76a1NRUALKyshq0qTEuvvhifD4fAAUFBVx55ZWsX78eEaGysrLOPpMnTyYpKYmkpCQOO+wwdu7cSc+ePets2xQi6YmMBX4CfCEioZ8hv8ERj3+IyDXAd8Albt0c4CxgA1ACXA3gisW9wDK33f+qar57fAPwApACvOu+DMNoQTTVY2huQnMiJSUlnHHGGTz55JNcddVVdOzYsU7P6Oc//zm33HIL5557LgsWLODuu+9u9B41l8OGztPS0ppsZzAYrNemxvDe56677uLUU0/lzTffZPPmzYwfP77OPklJSeFjn8/X4HxKU4jk6qxFqiqqOlRVh7mvOaqap6qnqWqOqk4ICYK7KutGVT1KVYeo6nLPtZ5T1aPd1/Oe8uWqOtjtc5NamkajnRMIBGJtQosjNTWVxx57jIcffpjU1FT69u3LP//5T8DZof35558Dzi/5Hj2ctTkvvvhivdfzMmvWLMrKysjLy2PBggWccMIJDbY//fTTef755ykpKQEgPz+fzMzMem3ykpGRQVFRUb3X9tr/wgsvNMn+5sB2rBtGGyL0cDKqM3z4cIYOHcrMmTN56aWX+Otf/8qxxx7LoEGDmDXLGVG/++67ufjiizn++OPp3Llzk647dOhQTj31VEaPHs1dd91F9+7dG2w/adIkzj33XEaMGMGwYcN46KGHAOq1ycs555zDm2++GZ5Yr8ltt93GHXfcwfDhww/ZuzgQ2l2O9REjRqglpTLaGqFhlE2bNjU55lEkWbt2LQMGDIi1GRHl7rvvrjbR3Zao699PRFao6oiabc0TMYw2xN69e2NtgtHOsCi+htGGiOYwRnunKRPv7QHzRAyjDWET60a0MRExjDaEeSJGtDERMYw2hHkiRrQxETGMNoR5Ika0MRExjDaEeSJVtLVQ8C0VExHDaOWEotOCiYiXthYKvqViImIYrZitW7dWC55nw1l109pDwbdkbJ+IYbRiaj7oWqIncs/bX/LVtsLGGx4AA7tn8ttzBjWpbVsIBd+SMRExjFZMXFz1wYSWKCKxoj2Egm8JmIgYRiumZijyljic1VSPoblpD6HgWwI2J2IYrZiankhLFJFY01ZCwbdUTEQMoxVT85dwaWlpjCxp2bSFUPAtlYiFgheR54CzgV2qOtgtexXo5zbpCOxT1WEi0gdYC3zj1i1R1evdPsdTlb1wDvBLVVURyQJeBfoAm4FLVLXREKYWCt5oS3z55ZcMHjw4fP6nP/2J/v3788gjj/DWW2+Fx96jjYWCb920lFDwLwCTvAWqOiWU5RB4HXjDU73RkwHxek/5dOA6IMd9ha45DZivqjnAfPfcMNoVNYez9u/fzx133MG8efNYs2ZNjKwy2hMRm1hX1Y9dD6MW4vjglwA/aOgaItINyFTVJe75DOB8nFzq5wHj3aYvAguA2w/dcsNovZSUlJCZmQnArl27YmxN28ZCwTvEak5kHLBTVdd7yvqKyEoR+UhExrllPYBcT5tctwygq6pud493AF3ru5mITBWR5SKyfPfu3c30EQwj9gSDwWrngUCA+Hjnt2FlZWUsTDLaGbESkcuAmZ7z7cARqjocuAV4WUQym3oxdSZ26p3cUdWnVXWEqo7o0qXLwdpsGC2OmiISDAbx+XyAiYgRHaK+T0RE4oEfAseHylS1HCh3j1eIyEbgGGAr0NPTvadbBrBTRLqp6nZ32Mt8d6PdUZcnYiJiRJNYeCITgK9VNTxMJSJdRMTnHh+JM4G+yR2uKhSR0e48yhVAaO3bbOBK9/hKT7lhtBvq8kRCk+0mIkY0iJiIiMhM4FOgn4jkisg1btWlVB/KAjgZWC0iq4DXgOtVNd+tuwF4FtgAbMSZVAf4PXC6iKzHEabfR+qzGEZLpS5PJCQi7X3joTcU/MUXXxze4HcoLF++nF/84hf11m/bto2LLrrokO/Tmojk6qzL6im/qo6y13GW/NbVfjkwuI7yPOC0Q7PSMFo3XhHJzs7miSeeCJ+3d08kFPYE4Ec/+hFPPfUUt9xyS7je7/eHFyE0lREjRjBiRK2tEmG6d+/Oa6+9dnAGt1Jsx7phtGK8IlJTNNq7iHgZN24cGzZsqBWmPRAI8N///d+ccMIJDB06lL/85S8AXHrppbzzzjvh/ldddRWvvfYaCxYs4Oyzzwbgo48+YtiwYQwbNozhw4dTVFTE5s2bw5s/y8rKuPrqqxkyZAjDhw/nww8/BJwkVz/84Q+ZNGkSOTk53HbbbVH+NpoXC8BoGK0Yr4jUjPzaYkTk3Wmw44vmvebhQ+DMpo1g+/1+3n33XSZNcvYpe8O0P/3003To0IFly5ZRXl7O2LFjmThxIlOmTOEf//gHkydPpqKigvnz5zN9+nQ+++yz8HUfeughnnzyScaOHUtxcTHJycnV7vvkk08iInzxxRd8/fXXTJw4kXXr1gGwatUqVq5cSVJSEv369ePnP/85vXr1aqYvJ7qYJ2IYrRiviFRUVFSrazEiEiNCoeBHjBjBEUccEc4n4g3T/u9//5sZM2YwbNgwRo0aRV5eHuvXr+fMM8/kww8/pLy8nHfffZeTTz6ZlJSUatcfO3Yst9xyC4899hj79u2rNTS2aNEifvzjHwPQv39/evfuHRaR0047jQ4dOpCcnMzAgQP57rvvIv11RAzzRAyjFeMVkZpx8FqMiDTRY2huvHMiXrxh2lWVxx9/nDPOOKNWu/Hjx/Pee+/x6quvcumll9aqnzZtGpMnT2bOnDmMHTuW9957r5Y3Uh9JSUnhY5/P16oXQZgnYhitmJqrs7y05gdTtDjjjDOYPn16WHDXrVvH/v37AZgyZQrPP/88CxcuDA+Fedm4cSNDhgzh9ttv54QTTqiV0nbcuHG89NJL4et+//339OvXr9Z1WjsmIobRimlIRFqMJ9KCufbaaxk4cCDHHXccgwcP5mc/+1lYfCdOnMhHH33EhAkTSExMrNX3z3/+M4MHD2bo0KEkJCRw5plnVqu/4YYbCAaDDBkyhClTpvDCCy9U80DaChELBd9SsVDwRlti/vz5TJgwoc66X//61/zxj3+MskUO7SEUfFumpYSCNwwjwpgnYsQaExHDaMWYiBixxkTEMFoxLXlivb0NlbcVDvTfzUTEMFoxXhGpmeUwlp5IcnIyeXl5JiStDFUlLy+vyUuVwfaJGEarJiQin3zyCWeccQbFxcXhuliKSM+ePcnNzcWSwLU+kpOT6dmzZ+MNXUxEDKMVExKRlJQUUlNTW4yIJCQkhHeFG20bG84yjFZMSETi4uJISEioVmcT60Y0MBExjFaMV0Ra0pyI0X4wETGMVoxXREJpcUPEenWW0T6IZGbD50Rkl4is8ZTdLSJbRWSV+zrLU3eHiGwQkW9E5AxP+SS3bIOITPOU9xWRz9zyV0WkdlwCw2jjmCdixJpIeiIvALWjlsEjqjrMfc0BEJGBOGlzB7l9/k9EfG7e9SeBM4GBwGVuW4AH3WsdDewFrql5I8No65iIGLEmYiKiqh8D+Y02dDgPeEVVy1X1W5x86iPd1wZV3aSqFcArwHkiIsAPcPKxA7wInN+sH8AwWgENichHH30UC5OMdkYs5kRuEpHV7nBXJ7esB7DF0ybXLauvPBvYp6r+GuV1IiJTRWS5iCy3detGW6IhETGMaBDt/3XTgaOAYcB24OFo3FRVn1bVEao6okuXLtG4pWFEhfpEJDs7u9ZEu2FEgqiKiKruVNWAqgaBZ3CGqwC2At4Ewz3dsvrK84COIhJfo9ww2hX1rc4SEQKBgM2LGBEnqiIiIt08pxcAoZVbs4FLRSRJRPoCOcBSYBmQ467ESsSZfJ+tTkCeD4GL3P5XArOi8RkMoyXhFZGOHTuGy/fs2QPAvffeGxO7jPZDxMKeiMhMYDzQWURygd8C40VkGKDAZuBnAKr6pYj8A/gK8AM3qmrAvc5NwHuAD3hOVb90b3E78IqI/A5YCfw1Up/FMFoqXhF5+eWX6dWrV7X6xYsXx8Isox0RMRFR1cvqKK73Qa+q9wH31VE+B5hTR/kmqobDDKNd4hWRbt268Zvf/Ib7778/XJ+bmxsr04x2gi3nMIxWjFdEwAnE6MXmRIxIYyJiGK2YmiKSkZFRrT47OzvqNhntCxMRw2jF1BSR3r17A3DnnXcSHx9Pjx71bp8yjGbB8okYRiumpoicd955vPrqq5x//vnMnTuX8vLyWJpntANMRAyjFVNTRESESy65BICkpCQTESPi2HCWYbRiaoqIl8TERCoqKqJtUpvnvCcW8dZK29scwkTEMFoxDYmIz+cL1xvNx+e5Bfzq1VWxNqPFYCJiGK2YhkRERExEmhknWIbhxUTEMFoxDYlIXFycPfSamaB9nbUwETGMVkxjImKeSPMSNFGuhYmIYbRiTESii4lIbUxEDKMVYyISXbwaUlBayd79tvrNRMQwWjEhkXAyRlenpKSElStXMnPmzGib1WYJeCZFhv3vvxl+77wYWtMyMBExjFZMMBisNy1uKBX0HXfcEU2T2jSh4ayjJZdENS8ETEQMo1XTkIgkJycDsG/fvmia1KYJKiTg5/2k25iR+PtYm9MiMBExjFZMQyISGuqyXevNh6rSiSIARsV9HWNrWgYRExEReU5EdonIGk/ZH0XkaxFZLSJvikhHt7yPiJSKyCr39ZSnz/Ei8oWIbBCRx8Qd/BWRLBGZJyLr3fdOkfoshtFSaYqI2F6R5iOokCVFsTajRRFJT+QFYFKNsnnAYFUdCqwDvIO1G1V1mPu63lM+HbgOJ+96juea04D5qpoDzHfPDaNd0RQRsRVazUdQlSwpjLUZLYqIiYiqfgzk1yj7t6r63dMlQM+GriEi3YBMVV2izs+pGcD5bvV5wIvu8YuecsNoN5iIRJegKp0oDp8L9t3Gck7kp8C7nvO+IrJSRD4SkXFuWQ/AmyQ61y0D6Kqq293jHUDX+m4kIlNFZLmILA+tWDGMtoCJSHRRhWSq5phSsPmmmIiIiPw/wA+85BZtB45Q1eHALcDLIpLZ1Ou5Xkq9A7+q+rSqjlDVEV26dDkEyw2jZWFzItElqEqiVOWtT6MsfJxXXM7tr62mtGhvLEyLGVEXERG5Cjgb+JH78EdVy1U1zz1eAWwEjgG2Un3Iq6dbBrDTHe4KDXvtisoHMIwWhIlIdAkqJOIPn6dKlYg8On89Hy9fRcrDfeCTx2JgXWyIqoiIyCTgNuBcVS3xlHcREZ97fCTOBPomd7iqUERGu6uyrgBmud1mA1e6x1d6yg2j3dCQiHz9ddUS1A0bNkTLpDZNMKgkeYaw0igLi3SCL44j47Y5FatejoV5MSGSS3xnAp8C/UQkV0SuAZ4AMoB5NZbyngysFpFVwGvA9aoampS/AXgW2IDjoYTmUX4PnC4i64EJ7rlhtCv8fn+9IuIlJycnfDx9+nS+++67SJrVZgmqVvdEKKOs0vH4OqUmcDihoaz24/1FLMe6ql5WR/Ff62n7OvB6PXXLgcF1lOcBpx2KjYbR2nn66acPqP2+ffu44YYbyMnJYd26dRGyqu0SVEjyzolIOXuKy+mVlUpWWhIl4opIOxpCtB3rhtGOCAQCAOzcuTPGlrROHE+kSkRSKWPDbmfJb4U/QJqUOhWB8liYFxNMRAyjHVFZ6TwAg8EgIsK1114bY4taF/6AkoTXEymjpNwR5sqAVi35LS+uq3ubxETEMFop5eUN/9pdvHhxrbJQHK2QmPz1r3WOMBv1UBkIkkglFeoDnIn10kpHRCoCQZJx/03K209oFBMRw2ilnH766QDce++9ddaPGTOm2rmqhkWkMQEy6qYyECRJKtlLBgBplFJa4Uy0l/uDJIfmSwLlEKis7zJtChMRw2ilLFy4EKgK+d4Yfr8fi9hwaFQGlCT8FGsKFeojTao8kcpAkBQ84txOvBETEcNo5TQkIosWLeLII48EoKysjLvuuqtafXp6ekRta2ts3VdCEpWUk8h+UkinlJIKdzjLH6weBsVEpDoikhpJQwzDODg6dOhQb93YsWO5+eabAWcI68QTT6xWn5iYGFHb2ho3v/o5iVRSTgJlcSlkxpVT6hUR8XgiFftjZGV0aVREROREEfkK+No9P1ZE/i/ilhmG0STS0tIarA95KmVlZbUSVHXs2DFidrVVkqSSCuIpJoXMuDJe/HQzecXlznCWZw8JlSX1XqMt0RRP5BHgDCAU2+pznB3mhmG0AFJTGx4k8IpIWVlZg22Nhjm+dyfHE9EEijWFNC2lrDLIjS//h6IyP2lxFexRJ3bsnvz8Rq7WNmjScJaqbqlRFIiALYZhHASHIiKWOvfAyE5LJCM+QDkJJKRk0sHnbC7ckl/Kpj37yfBVkq/Oyq1/fvo1q7bso8+0d/gityCWZkeUpojIFhE5EVARSRCRXwNrI2yXYRhNpGvXelPpACYizUkgqCSJnzHHdKN//0F0DTg7//3BIEVllSRrBfk4nsi2Xfn8zywnO/iCb6IQZDwYgH01f+9HnqaIyPXAjTjJoLYCw9xzwzBiyOTJkwHo169fg+28IlJzf4iJyIHhDyoJWkl6WjrxnY8iS4rIZD9llUGKyvwkajl5rieSQimrXQ9EJArGLfg9/HkwbP1PFG5WRaMioqp7VPVHqtpVVQ9T1R+Hcn8YhhE7kpKSGDhwYKPtGvJEbNPhgREIurGzfInQxRHvfrKFI7JSKS73k6Bl5KmzWk49q7NeWByFqMkb3q/+HiUajeIrIs9TR1xjVf1pRCwyDKNJBAIBfD5fo+2SkpIAG85qDvzBIAlaCfHJ0G0YAIPjvuXf+4cjwUp8GgjvZk8KVn3Xe4rL2V1UTpeMpIjYVeEPsj9vF50A9m6OyD3qoynDWf8C3nFf84FMoP1EFzOMFkpTRaSh4axAIBCO7Gs0juOJVEB8EmR0Zbd2oL9sYVdRWXijYaGmUK7xpEr17/q8JxZFzK73v9pOatkO52RvdHPFNGU463XP6yXgEmBEUy4uIs+JyC4RWeMpyxKReSKy3n3v5JaLiDwmIhtEZLWIHOfpc6Xbfr2IXOkpP15EvnD7POZmPzSMdkEwGDxgEalriW8oGKPROJUBJV7d4Szgez2MHrKbyoCGgy/edf7xlJJEqpt/XQhydtynFBfk8dqK3IjY5S/YQZK4ybKKd0TkHvVxMGFPcoDDmtj2BWBSjbJpwHxVzcHxbKa55We6184BpgLTwREd4LfAKGAk8NuQ8LhtrvP0q3kvw2izNNUTSUhIAJzYWXWJiA1pHQCBSuIJQEIKANs0mx6yB4Bkcb/HhFT88amkuqIyMW4FTyQ+zn/Hv8r8tZHJ47Jn60YANge7EiiKwkowD03ZsV4kIoWhd+Bt4PamXFxVPwZq7rg5D3jRPX4RON9TPkMdlgAdRaQbzkbHeaqar6p7gXnAJLcuU1WXqJPkeIbnWobR5qmsrAwLREPExztTn36/v86JdJtcbzpxAVeEXRHZrtl0k3zAk0skIZnOnTpxlBsM4LI+TgytU1PW8+6aHeGc7CG+3FZAZSB4SHZt/X49AGu0D76KQvBH74dBU4azMlQ10/N+jJvO9mDpqqrb3eMdQGiRew/Au8g51y1rqDy3jnLDaBcUFRWRkZHRaLuQt1LTEwkNcy1durTB/r/61a+YO3fuIVjadpBKN3OhKyJ7NJNkqSSd0qoIvgmpkJhGsjrnnQLO7+hulbkkUUFReVWO9q37Spn82CLunv3lIdmVXOYsmP06eAQAgeLoRWuuV0RE5LiGXs1xc9eDiHgyYhGZKiLLRWS5hcI22gqFhYVkZmY22i7kiQQCAcrKyhg8eDAA55xzDgBnn312vX0rKip49NFHOfPMM5vB4taP+F0RjndE5OJTjgegsxSQEh7OSoGENJLUEfHODK0AACAASURBVJwMv/OA9xHgGMmlqKxKRELBGxd8c/DPpcpAkOTKvQRV2KjdAdi2LXqbDhta4vtwA3UK/OAg77lTRLqp6nZ3SCo0gLcV6OVp19Mt2wqMr1G+wC3vWUf72saqPg08DTBixIiIi5ZhRIOioqImhXKvOZx1yimnsGTJEmbPns0///nPevstXryYJ598stnsbQvEBap7IjlH9oXF0JkCkqmaEyExjcSA82hLr8yDTn1h77cMjPuOorJKwOmf6HN+x2/dV3rQNi1cv5tOWkBFYkd2VThjaBUF0ZsXqVdEVPXUCN1zNnAl8Hv3fZan/CYReQVnEr3AFZr3gPs9k+kTgTtUNd+dqxkNfAZcATweIZsNo8VRVlbWaNwsqD2clZSURFpaWqNh4MeOHdssdrYlqkTE/d7TnDVGnaWQONx5jYQUSEwl2V2dlV6ZB31OobJ4D0P9m/AHqn7HBvTQf9PuKiwnS4qQ9M7kFTueaWVhZCbw66LRzYYAIjIYGAiEs9+o6owm9JuJ40V0FpFcnFVWvwf+ISLXAN/hLBkGmAOcBWwASoCr3fvki8i9wDK33f+qamiy/gacFWApwLvuyzDaBaWlpU3Kauj1RMrKysJ9/H5/Q92MOojzlztPzQT3e08PiUgB5biLHBLTISGNTvGVTOjfhZQteZDRlaLsYzlu23rKgx4RCTrC82PfPEo/203KqKsOyJ4Nu4qZ9sYXvJpYiC+9S3i3fDTnRJqyY/23OEIwEOdBfyawCGc1VIOo6mX1VJ1WR1ulnphcqvoc8Fwd5cuBwY3ZYRhtDVWtJggNERKR1atXEwgEwn2Kig4s814wGCQurv0mQy0qqyQhtAs95ImkZgNwzfA0gkmdYAWQlAGJaSQESnj2kqPhDxWQfjhFnYdzzPZPWF1eDDjDTqFFWb9LeB7eheCIK4jzNf07fmTeOgCyKcSX0Y9/T5tMxSM+2L+nOT5yk2iKtRfhPPR3qOrVwLFA/anUDMOIOJWVlahqk0QkNJz18ssvA1Wrsi655JJwm6bsWn/zzTcPxtQ2w3//c3V4Q2FoTgRfAqR2pm9iIUd1cD2MxDRHSMqLochdiJrRldLsgfhEScj7JnzNQFDxeTJrfP7FygOyKbTSK0sKkbQuZKYkkkcHfKUtS0TKVDUI+EUkE2civFcjfQzDiCChpbopKSmNtq3pPYRiaWVmZvKnP/0JgOLixiMZ7dwZvXH2lsgXWz2T5/Ge771zDuxZ76TDjUtwQqKkdQYNwG5XMNK7UpblBMtMzq/KpBEIKhlUZUDM3fT1AdmUV1yOjwBZUgypnUlN9JGnmcSXRS9GbkNLfJ8UkZOApSLSEXgGx1n7D/BplOwzDKMOQiLSFE+kZjQgb5/QUFfv3r3ZunVrne0mTpwI2BzKoO6Z1ZfxhujSH3atdTyPJHe1XGpn532nG/Ep/XACHXpRrgkkFW4Odw2o0kGqov2W5X3fJFvyisv53bMzKd27g4sHhib5OyMiFMR1YH/+dq55YRn+Q9zE2BQa8kTWAX8EzgZ+g7MC6nTgSndYyzCMGHEgIlITb5/QjveCgoLwcJf3Htdccw2zZjkLKPfv3097IxhUcvc6nkJJRYBjstwwMzVFpGwf5G+CRHfzZ5orIts/d94zDsfn85FHBvFlVUE8AkGlA1Xf65bNG2rtaK+LJz7cwJ251zMr+HO6SGG1exb7OtJJC5n/9S52FkU+GkG9IqKqj6rqGJx86nk4E9tzgQtEJCfilhmGUS8HKiI33XRT+Ng7oR7yRABuu+228PHy5csBmDNnDklJSfh8viYNebU17py1hpMe/JBt+0rZX+EnLc4NVukVkcP6O+/ffgypWc5xSERylzvLgJPSiY8T9mptEcmUquGswyWv2o72+ti110l2lSGlZIv775nWBYD98Z3IdoWloCTywTWbEvbkO1V9UFWHA5fhxKc6sIE7wzCalQOZEwGq7QkZMmRI+Li+2FurV68G4J577kFECAQC3H///e0uWOPLnznDS7l7SykpD5AeEpF4j3i7eUUIVoYf5OH3sn3Q0QlF4osT8jSThPK6PZEK9dFV9rF5T8MeX2UgyH++2hA+74TribhDaBtLUkiTclIoczc2RpamBGCMF5FzROQlnH0Y3wA/jLhlhmHUS2mps+ktNEneGF4R6d+/f/jY64lkZWWFjwsLnQfTxRdfXO0633zzDe2RS/7yKcXlflKlwlne651nSukI2e7gTE0RATjpZgDi44R9pJNQvo/8/RV8vmUfQc+cSEWHI8mWQu56aw0NUVTmp7MUhM+z3Z3xoXuWJjjLh7OliMpA5AN01LtPREROx/E8zgKWAq8AU1W1/Q2MGkYLIyQiTdmxDtU9Dm+oFG95hw5VK/dDnk5NkTrQvSVtiZIKP6npldW9kBDdh0Pe+qrhrDgfnPs4+MthgBObLCneR6Gm4qss4rh75wHw4k9HhldnpXfvR/a+z/g8t4DCskoyk+v2EisDwWoicljZZpA4R8yAHxw/CJZBFoWHHB24KTTkidwBLAYGqOq5qvqyCYhhtAxCk9wHIyLeIbC9e/fWeRwKDx8SkZ/97GcATJgwISxgbZ1AsPqv+P0VASdSb0Id3/kQ12PrNbKq7LgrYOR14dMOqQkUkkZCZSGhuLMV/iDpUopKHHToRZY7l5GbX/93XOGvLiIdSzZDSpYjXMBJxzpLibMlxiKiqj9Q1WfdHB6GYbQgSkqcX69NFZEtW5yorr/73e+qLfldu7Zqz0JBQQFBNwxHeXk58fHx4T0mv/zlLwHHA1qzpuHhFnCCN27cuLFJtrVUSiv8hB72h2cmU+EPkkIZJNbxnR8zEX69AQaeV+/1MpLiKdJUfOonCWeu4pMNe0inlGB8GqR3IU3KSaKC/P31zz3tK6mkC1Ui0mH/pvDOeSA8qd9ZCqIynNV+YxgYRivm7rvvBpo+sR5KgdutW7dq5Zdffnn4WFUpLCyksLCQBx54oNq+kGOOOSZ8XHM/SV2MHTuWo48+ukm2tUS25JcQeH4yTyQ4MV0LSp3vLzVYDMn1BOxI71J3uUtcnFCII0CZ7mT6C4s3k0YZwcS08MR4NoXkl9QtImWVAc55YlE1TyTJX1y1GgzCx9ktYDjLMIwWSEFBAV999RUA3bt3b1Kf0N6DmhsPR44cWe183759fPbZZ7X6e9PwHshS3/fff7/JbVsKqsoP/jCPDjs/42zfEjKS4imtdEKTJAf21y8iTaBIXRHxLOtNk1KCCenhh3+WFLK3Hk+k0BWzzlJAZYInIZnXE0lMIxifQpYUmYgYhlGbvDwnpMUJJ5zQZE+kPhEBuOOOOzj99NMB2LFjB9OmTavzGs888wxwYCJy++1NyqTdoij3BzlCqkK8HJZUtUw2pSFPpAlMPN7x6DIpoZ98z2/iX6ID+1GPJ3JYXBE7C8vq7B/aQ9KZAso7HFVV4fVEAE3tTLYUVEuAFSlMRAyjlbF9uxPU76677mpyn4Z2Qd9///08+uijAGzcuDEsEhs2bKjWbsqUKUDjIuIN5uhd8dVaKK0IVNtF3l2rBCXRXwRJjWeTrI+zT3CWV180KJPpmTOYGv8OJ/m+RD2eSN/UMrYX1CMirih0lgLSuvSqsiW1uojEpR/Ghb5FZH/VaLD1Q8ZExDBaGSeddBJwcA/oujwRgL59+wKwefNmjj32WPr3789RRx1VrU1aWhrQ+DJf74bE0AKA1kRJZYA0qXqIj84OCYriqyg6JE8k1PdHwzrQ11eVfVATq0Tk8Piiej2IYre8d9J+JO2wqv0qNTwR6ezsWzlv259gT/UfA82NiYhhtFKakho3REPDWeCET/H5fJSWllJUVERGRkatNnFxcaSlpTXqiYSWB0PrFJHSCj+pVInIyQlfc4XvPR44+ygkUHGIIuJ6DmUFSHmVGGtSuuNVxCXQWYr4Lm8/zy7cVMuDLCqrJB4/SZX7nIRYfve7Tu9a/T6DLqg6zo/sKrmoi4iI9BORVZ5XoYj8SkTuFpGtnvKzPH3uEJENIvKNiJzhKZ/klm0QkboHcg2jDeGdAznuuOOa3O/BBx/k8ssv56KLLqq3TWJiIhUVFVRUVNSbOjc9PZ1NmzbRsWNHfvrTn9bZxisioZ3vrYmSigBpHhEZsuUl/jfhRS7LdJc2N4MnQkEuBDzBERPTHa8irQtZUsT6XcX87p213PfO2mrdi8r8ZOGJleV37ezUp/p9+k3iwf7/dI6LI5tvPeoioqrfqOowVR0GHI+TCjeU7eaRUJ2qzgEQkYHApcAgYBLwfyLiExEf8CROpsWBwGVuW8Nok+Tl5YV3ko8fP/6A+nbv3p2XXnqpwX0lIRGprKysN6ZWt27deOuttygoKOD555/nyy+/rNVmzx4nIdKAAQP47rvv2Ldv3wHZGmtKKgKkivOAV/E8IjctcN4PRUQSUkF8Tv4RYKM4cbXU3Z9Dhx50C+4IN3920bdsyS/hX6u38fj89RSWVdIltLw3/TDIcX9TZ/WtdatKnzP8SEVkA2fGejjrNGCjqn7XQJvzgFdUtVxVv8XJwT7SfW1Q1U2qWoETlqX+nT6G0co59dRTw8dNjZl1IMTHx/Poo49SUFBQr4iMHj262vn69etrtRk82MlYffLJJwOtL95WqccTKU/xDBNtXui815h/OCBEHBHKc4aYFvmcJdblhw116rsO4pjSVTyT8DDHijOXces/P+eml1fy8Lx1rNqyr2qPSFoXuOApuO7DOoXN73N/MLRxEbkUmOk5v0lEVovIcyLSyS3rAWzxtMl1y+orN4w2yRdffBE+joSIhJYOr169ulpgRi8hT+acc84BYPfu3dXqvWP4IcGpuXN937594Z3xLZHicj9pUkpAhTUn/hlOugV6jYZ97m/d1EMQEXBFxBHfo39wBRfxEGkjf+LUdXUE+HTfCv5fwksAJPiq5rH+tXo7RyS5E/1pXZxYXT3qHtZUXwIVxDsZFyNIzERERBKBcwF34I7pwFHAMGA78HAz3muqiCwXkeU1/9MbRmskEiLipT5PpGNHJ8hfKLpvaHgthHflVsgT8Zbt3LmTTp06HdDy5Gizp7icNMopj0th8KjTYcJvobNn9713Y9/BkJwJAWcF29ihA3nt7utITnbnugZdwObDJpCv6XQXR9Q/2VA91W3PRPf7TD+swduICCUkOxkXI0gsPZEzgf+oOouwVXWnqgbcfO7P4AxXAWylek73nm5ZfeW1UNWnVXWEqo7o0qXh0ASG0RqItIjU54ncfPPNzJo1i/PPPx+oPokOVauxpk+fTna287D1ZkR86623AJg7d26z29xcFJRWkkoZKekdSE5wd+pnHVnVIDWr7o5NJTT0JHG1r5XWmeDFL/Jc4Cx6yh4uGlr7XtkUODneExtenRcnQokmt11PBCfMfHgoS0S8QX0uAEJR3mYDl4pIkoj0BXJwQtMvA3JEpK/r1VzqtjWMNkfN/OaRFpFQUqqapKenc+6554bvX9MTCYlKYmJieOjr1ltvBSAYDPKHP/wBgEGDBkXE7ubAH1DSpQxJTKsq7OSZuI4/xO8+xRWG9K7hyLtejuySzq8umQTALcc7Yp5KGa8dPZejZCvHdiyHjK7Vc5rUQZzgeCIRnhOpN59IJBGRNJx87T/zFP9BRIbhhM3cHKpT1S9F5B/AV4AfuFFVA+51bgLeA3zAc6pae6mIYbQBav7iP5jc6o3Rp08fNm/eDNTerV6T0HBXTbtCGw2TkpJqDYktW7aMTZs2VWvXEqkMBkmTGiHfvZ7IodKpt/Oe0a3eJvGdnY2e3QPbAR93xf+NEbkfMu/o0Uh8EiR2rbdviLg4YT9t1BNR1f2qmq2qBZ6yn6jqEFUd6uYv2e6pu09Vj1LVfqr6rqd8jqoe49bdF+3PYRjRIvSwDgVCjIQn8q9//avJbUWEpKSksOiAM6keykkS2mdy4YUXhudRduyoWrpal4h88MEH9OjRI+a53EOeCEmeDZdd+kFmDxh9w6HfoOaejroIiVb+Rrqwjym+BQDEbfkM2f55o/Mh4Dgq+zUZLS+qN6BjcxDr1VmGYTSBkIgcdljjD4+DxTvEdOyxxzbavqKigrfffptXXnkFgKlTpzJq1CigSuT69OlDZWUlqsrjjzth1Tt37kx5eTnBYJB///vfYdG444472LZtW7VVaLEgEFRnn4h3OCshBX6xEiY9cOg36H0SIHDsZfW3Sc50Vl/lb+Ld45Y6I1cXvwiok7e95g71OogToViT+Pr7HQy/dx4V/sisiDMRMYxWQEhEQsEXn3vuuYjc55133uGCCy5g3rx5jbY94YQTKCgo4LLLLuOVV17h2WefDdeFPJG0tDRKSkqYNWsW8+fPB+DII4+kvLycuXPncsYZZ5CRkcEbb7wRFp6aQ2TRpjIQdPaJeEUEDn0uJESXY+A3W2HU1IbbZR0F21bRed0ryPAfw6DzIbOnU5fW+I+JOIH9JJOGkyXx2UWbDtXyuu8TkasahtGshJbJhjINhiarm5uzzjqLN954g6asYly6dGn4+Prrr69WFwrP0rlzZ1SVP/7xj+G6rKws9uzZw7Jly8JlF154IQsXOpv5WsJwVmpdItKcNOXaWUfCjtVOaJPjr3LKhrppeLsPa7R7aHVWaPf93z79LiLDWiYihtEKKChwpg/vvPNOAK655ppYmgNQLe9IyL4QoQCOw4Y5D7vFixeH67p06cLnn38ezs5Yk3POOYeRI0dy9NFHc/PNNzcYxj4S+IPqpsFteoDLiJDtzotk9oAexzvHp/4/uOodyJnYaHcRoZjk8O777QVl4QyNzYmJiGG0AkIP6cmTJ6OqtdLcxoIHHniAd999t866UIThmpkT33nnHU488cRqZXPmzOGpp56qVrZs2TI2btzIn//8Z15++eVmtLpx/IFA5D2RpjDoh9ChF5xye9VyXl8C9Dmp0eW94C7x1WRSpII4nPmQnp2alsTsQDARMYxWQEhEWlqSp1NOOaXO8pCd3lVkq1ev5qyzzqrV58wzz+RnP6ta7V8zPa+3LhpUVJQRTyD2IpJ9FNy8Bo6/8qC6x4m7xBfIoITjZB3xgdLmtBCI0T4RwzCazo4dO8Kb9FqaiHhF4tZbb+WTTz5h/fr11VaRnXLKKXz00Ufh1V8DBgzgqquuYtCgQeFsiQAnnngiHTt2ZOTIkTzwwAN89913dO3alXvuuQe/31/vLvrmpmy/G74+1sNZh0hoYh3giYTHGOdbAwWTnYn9ZsRExDBaON6hq6ysQwy50cyEJvoBjjjiCO677z6KioqqJb/617/+xZYtW6q1ff7552tda9GiReGd+aH5lgcffBBwVmxFS0TKS9zYVLH2RA4REWG/OiIyzreGwuyhZHbs1UivA8dExDBaMN585QMGDKg3WVQsqaioYMaMGVx++eUkJSXV2giZnp7OgAEDGr2OiNTa5e5d9htKzxtpigv2OjEwWrmIxIlQTNUcSObwC539Ls2MiYhhtGC8CZ3Wrl3bQMvYkZCQELHVYtHeO7JtXynpAVdE0lp3sNY4gZ3aqaog+6jI3CciVzUM45ApLCxs0i/4tkwoRli0ROS0hz+iM+6cSCsXEQV2qGf4sznjf3kwETGMFsrKlSurJX2K1AbDlkxIREIh5iNNaWWgeubAVsyX2wrJxxP/q1PtFLrNgYmIYbRQSkurlmNOmDCB++5rfzFG+/Z1Hny/+MUvInqfLfkl/PKVlaQnxXNqVj4kdYCUTo13bMEk+uJQ4pjhPx0Gng+JqY13OghMRAyjheLN1fHyyy9HPIdISyQ0nBeKuxUpnvhgA7NWbaO43M+AitXQ+8QmbehryVw7zhHg//FfDZe8GLH72MS6YbRQQp7IypUrmxTLqi3SoUMH+vTpE7HhrE27i/kidx/xbh7zbuTRuXwL9InuBsdI0P/wjMYbNQPmiRhGCyUkIp06te5hlUPl1FNPjcjS5i35JVzw8DsMe/NUZMUL9JTdvJn0P05l/8nNfr9oI1HypGImIiKyWUS+EJFVIrLcLcsSkXkist597+SWi4g8JiIbRGS1iBznuc6Vbvv1InJw8QEMowUSGs4KRcRtr6SkpFSbH2ouxv3hQyb6ltM7bhe3+17iV/Gvc7jspWLojyK2kina/O78wVx7UmQm1EPEejjrVFXd4zmfBsxX1d+LyDT3/HbgTJzc6jnAKGA6MEpEsoDfAiNwVrStEJHZqro3mh/CMCJB6MEZiVS4rYlIiQhAb9np3INyLsz4Eu17AYk//L+I3CsW/Hh074jfo6UNZ50HhGaAXgTO95TPUIclQEcR6QacAcxT1XxXOOYBk6JttGEcCn6/n2Cwdta50IPTPBFHRCIREr6nOEuo4yWIlOQhfcY1+z3aOrEUEQX+LSIrRCSU4qurJ7f6DiCUA7IHsMXTN9ctq6+8GiIyVUSWi8hy77p7w2gJJCQkcOWVtUdi586dG65vz6SkpKCqdeZlPxQOy0iil+wmeLgnwVPO6c16j/ZALEXkJFU9Dmeo6kYROdlbqc7Pjmb56aGqT6vqCFUd0V5XuRgtk/379wPw97//vVbdJ598Em1zWiQhT8y75Lk56JyexFEJ+cQdPhiumAWXzICORzTrPdoDMZsTUdWt7vsuEXkTGAnsFJFuqrrdHa7a5TbfCnjDT/Z0y7YC42uUL4iw6YbRbGzcuLHeug4dOpCdnR1Fa1omIREpLS1t3lD4/lI6BvKgU284cnzzXbedERNPRETSRCQjdAxMBNYAs4GQX38lMMs9ng1c4a7SGg0UuMNe7wETRaSTu5JroltmGK2C0JAVwNNPPx0+Dg3fXHDBBbEwq0URypLY3EPRnSrd36jmfRwSsRrO6gosEpHPgaXAO6o6F/g9cLqIrAcmuOcAc4BNwAbgGeAGAFXNB+4Flrmv/3XLDKNFU15ezvbt26t5It40sHv27KG0tJTu3bvHwrwWxeDBgwH4+uuvm/W62X53+rVj5FcwtWViMpylqpuAY+sozwNOq6NcgRvrudZzwHPNbaNhRJK6lu0WFjrRY0tLS8OZAU1EoHPnzgBcffXVjBs3jsMPP7xZrnt4YIdzYJ7IIdHSlvgaRrtl5cqV5OXlsXjx4nDZqFGjYmhRyyC0GGb//v1MnTo1XO73+w962W8gqBxWuZXKuCTI6NZ4B6NeTEQMI8rU3BOSl5cXPp49ezYTJkwA4IEHHghHsW3PeANPvv3226gq27dvJyEhgZtvvrnR/h+v282GXcVVBeXFfPz3+7nQt5DcuJ4QZ4/BQ8G+PcOIMt5sheDkTX/uOWdEduXKleHy22+/Pap2tRa2bdvG6NGjAXj00UdZsWIFb7/9dp1ty/0B7n7+LfZMPwv+8zf8q1+n/NHjOXXTH8jTTJ5P+lE0TW+TxDrsiWG0O7yrjP7rv/4LgHPOOQeAxx9/HIAbbrghagH0WgMZGRkUFRUBsG7dOr7//vtw3YgRIwBYsWIFxx13XLV+73+1i/+Of5XR+jnMvol44Jtgb35b+T8s1/7M/OHoqH2GtoqJiGFEmZCIvPfee0ycOBGoHanXNsVWp3fv3qxZswZwVmmlpaWFN2qG+Pbbb2uJyMy5C5gRt5y/+CezX1PYRUf+ERhPkDimjOjFmKNsH86hYiJiGFEmJCJeofD5fNXanHDCCVG1qaUzdOjQsIh888039OjRg3Xr1lVrU1xcXL2TKpeXzsRPHM/6z2I3nUjwCS9PHUWf7DQ6pbXvcDLNhYmIYUSZukSkJkOGDImWOa2Cp59+milTpjB16lSWL1/O3r17ueyyyxg6dCgfffQRc+fO5aqrruL1119n9uzZrPx0Af6PHuIsXciiblfw+1NOJy0pntFHmufR3NjEumFEmcZE5LXXXuOII2zvgpe0tDTOPfdcdu7cySeffMLu3bvp06cP06ZNY/bs2eF2b7/9NmPGjGHRO3/nhLKF/MU/mTX9f8lpA7qagEQIExHDiDJLliwhPj6+Vs70vXv38sEHH3DhhRfGyLLWRUiEExIS2LFjR3g125IlS7j1j3/jpNXnkn/iXVw20nakRxKJRIz+lsyIESN0+fLlsTbDaKeUl5eHd6u3t7+95uDyyy9n5syZgBP5+Ec/qr5E9/vvv6d37yrRsO+4+RCRFao6oma5zYkYRhRZsWIFYBPnB8vLL7/M0qVL2bhxY53DgUcccQRr167lrbfewu/3x8DC9oeJiGFEkQ8++ACoO3+I0TSmTJnC/fffz1FHHVVnff/+/Zk2bVqUrWq/2JyIYUSR0N6GY445JsaWtF7uvPNOvvzyy3pFxIguJiKG0QwEAgGmT5/OjBkz6syXHqKkpKR5Eyu1Q1JSUhg4cGCszTBcbDjLMJqBl156iRtuuAGAnJwcxowZQ2VlJffddx+//vWvmTt3LpmZmTz22GMxttQwmhcTEcM4RAKBQLVJ3E8//ZScnBzefvtt7rnnHr799ltmzJgRQwsNI3JEfThLRHqJyIci8pWIfCkiv3TL7xaRrSKyyn2d5elzh4hsEJFvROQMT/kkt2yDiNhMmhF1Vq5cSXx8PPPnzw+X3XrrrXTp0oW1a9cCtdO62nCW0ZaI+j4REekGdFPV/7h51lcA5wOXAMWq+lCN9gOBmcBIoDvwPhCalVwHnA7k4qTHvUxVv2ro/rZPxGhOHnnkEW655ZYD6rN79+5wtj7DaC20mH0iqrod2O4eF4nIWqBHA13OA15R1XLgWxHZgCMoABvcVLuIyCtu2wZFxDCaE+8kemZmZjjFbX18+OGHJiBGmyKmcyIi0gcYDnwGjAVuEpErgOXAraq6F0dglni65VIlOltqlNeZS1REpgJTAYtJZDQrgUAgfHz55Zczbtw4CgsLycrK4sc//jGVlZXh+rlz5zJ+/PgYWGkYkSNmS3xFJB14HfiVqhYC04GjgGE4nsrDzXUvVX1aVUeo6gjL02A0J3v27Akfp+5JMQAADjdJREFUn3zyyVx++eVcf/31XHLJJVRUVPCXv/wlXN+rV69YmGgYESUmIiIiCTgC8pKqvgGgqjtVNaCqQeAZqoastgLev76ebll95YYRNbzpbMeOHVur/rrrrgsf9+/fPyo2GUY0icXqLAH+CqxV1T95yrt5ml0ArHGPZwOXikiSiPQFcoClOBPpOSLSV0QSgUvdtoYRNbZv384RRxzB+vXr6xwq9aa4jYuzvb1G2yMWcyJjgZ8AX4jIKrfsN8BlIjIMUGAz8DMAVf1SRP6BM2HuB25U1QCAiNwEvAf4gOdU9ctofhCjfZOXl8dXX33FnXfeydFHHx1rcwwjJsRiddYiQOqomtNAn/uA++oon9NQP8OIFGVlZcycORNV5bTTTmuw7bp16xoMhWIYrRnbsW4YB0FKSkr4eMSIWkvnq5GTkxNpcwwjZtggrWEcAkOHDiUtLS3WZhhGzDARMYwD5F//+hcA99xzD8uWLYuxNYYRW0xEDOMAue222wBnSW9iYmKMrTGM2GIiYhgHQFlZGZs3b+a8885rdELdMNoDJiKG4bJx40ZmzJjBpk2bKC8vr7PNXXfdRWlpKddee22UrTOMlomtzjLaBbNmzeLEE0+kvrA3zz33HNdcc034fOjQoXz++edUVlby0EMP8cUXX/DBBx+wc+dORo8ezcSJE6NlumG0aExEjDbPmjVrOP/887n22mt55plnwuUFBQUsWrSIPn36VBMQgNWrV3PhhRfy6aefsn379nD59ddfz6OPPmpzIYbhYiJitGn8fn8438cXX3wRLl+yZAljxowJn/t8Pj744ANOPvlk1q1bR79+/XjjjTcAGDlyJB9//DE+n4/4ePuTMQwvNiditFn8fj+9evVi3rx5AHz22WcMGjQIn89XTUDGjh3LwoULOfnkkwE45phjmDlzJgCjR49m8eLFJCUlmYAYRh3YX4URcT7++GP69OkT0VwuBQUFgJPf4+OPPyY/P5+lS5eyY8cO+vXrx5AhQ3jttdf46isnZ9mYMWO49957611hdemll3LqqafStWvXiNlsGG0BExEjonjTx1511VV06NCBE044gcTERLp06UK3bt04+uij8fl8B3TdZcuWcemll7Jp0yaSkpLqXU11wQUX8PrrryMiPPLII3z77bfcd999ZGRkNHoPExDDaJyo51iPNZZjPToUFRXxyiuvMHXq1EbbHn744dx4441kZ2dTUVFBSkoKH3zwAZ9//jm9evWia9euBINB8vPzWbFiBbt37651jdNOO42RI50UNOPGjePoo4+mc+fOdOrUqdk/m2G0R+rLsW4i0saYN28emZmZjBpVPVNwSUkJy5YtIysri65du9KlSxfWr19P7969SUpKOuj77dy5k++++47DDjuMoqIi0tPTKS4uZujQoeE206dPJy8vj127dnHdddexefNmUlJSKC4uZs6cOTz99NN1Xnvw4MH4/X7Ky8sREbKyssjOziYnJ4cePXrw/9u7+9iq6juO4+9Pr7S0cluxWjCClaqoHdOBuOAUMwdTlE2HI4tmie6BmCUzmS5z0Zgs/LNsuodkz8s2yHQ41IWR6aIiTGULEQd2KFZFHgbBhseWALZN29t+98f5tZ5iy3oP3Htu4ftKbu65v557+7m/09vvvb9z7vktWrTI5yt3rki8iASnchExs4GJj3bt2sUFF1zAK6+8wrJly1iyZMmw98tkMixYsIArr7ySbDZLU1MTK1asoKqqisrKSi677DK6u7vp7u6mq6uL7u5uent7aWtrY+/evcM+7v33388DDzzAxIkTB03OdKydO3cyceJEDh06hCQ6OzsZN27csN/pcM4VnxeRIGkRyeVyHD58mLVr19Lc3MzRo0epr6/n6quvZvz48fT09DB16tSBI3h6e3vp7e2lrKwMSQPXAF1dXbS2tlJeXk5zczPr1q2jubmZa6+9duCfdG1tLRdffDGXX375oLH5w4cPk81mKSsrw8w4ePAg27dvZ/fu3SxfvpyVK1cOrFteXk53d/fA7RtuuIG5c+fS1NREXV0dY8eO5emnn6alZfCswmPHjmXWrFlMmjSJ1tZWWltbqaiooKKigvLycioqKpBETU0N06ZNGziLbU1NDZ2dnRw9epTp06cze/bsvPvZOVeaTtkiImke8DOi2Q3/YGY/PN76SYvInDlzeOmll467Tjabpbq6mvb2dtrb2+np6YnnpKamhmw2S2trKx0dHSP+3ZWVlWQyGTKZzMBRSNlsls7OTnK53MB65eXlLFy4kEWLFrFmzRr27dvHkSNHWLBgAfPnz6e6uvojj21m9PX1UVZWxv79+wdynsgQl3Pu1HNKFhFJGeA94LPA+0Tzrt9pZm8Pd5+kReTZZ59l/fr1NDY2MmfOHOrq6li3bh07duwgk8mQy+V49dVX6ejoIJvNUllZSW1tLX19ffT19dHT00NbWxsdHR1UV1czadIkxowZQ0NDA5MnT2batGkcOHCAqqoqxowZQ0tLC1u3bmXDhg20t7fT29tLLpdj1apVNDY2Ul9fT2VlJRMmTKChoYG6ujquuOIKqqqqknanc84N61QtItcAi83spnD7IQAz+8Fw9zmV94k451yhDFdERvs31s8Hdsduvx/aBpF0j6SNkjYOdXioc865ZEZ7ERkRM/udmc00s5l+xI9zzp08o72ItACTY7cnhTbnnHNFMNqLyAbgEklTJJUDdwDPpJzJOedOG6P63FlmlpN0L7CK6BDfpWbWnHIs55w7bYzqIgJgZs8Bz6WdwznnTkejfTjLOedciryIOOecS2xUf9kwCUkHgF0J734OcPAkxikEz3jiSj0feMaTpdQzllK+ejP7yHckTrsiciIkbRzqG5ulxDOeuFLPB57xZCn1jKWeD3w4yznn3AnwIuKccy4xLyL5GXoKvtLiGU9cqecDz3iylHrGUs/n+0Scc84l559EnHPOJeZFxDnnXGJeREZI0jxJWyRtk/RgShkmS3pZ0tuSmiV9K7QvltQiaVO43BK7z0Mh8xZJNxUp505Jm0OWjaHtbEmrJW0N1+NDuyT9PGR8U9KMIuS7NNZXmyQdkXRf2v0oaamk/ZLeirXl3W+S7g7rb5V0d4Hz/UjSuyHDSklnhfYLJXXG+vK3sftcFf4+toXnoAJnzHu7FvL1PkzGp2L5dkraFNpT6ce8mJlf/s+F6OSO24EGoBx4A2hMIcd5wIywnCWaGrgRWAx8Z4j1G0PWCmBKeA6ZIuTcCZxzTNujwINh+UHgkbB8C/A8IGAW8FoK23YvUJ92PwLXAzOAt5L2G3A2sCNcjw/L4wuY70bgjLD8SCzfhfH1jnmcf4fMCs/h5gL3YV7btdCv96EyHvPznwDfS7Mf87n4J5GR+SSwzcx2mFk38CRwW7FDmNkeM2sKy0eBdxhiJseY24AnzazLzP4LbCN6Lmm4DXgsLD8GfCHW/rhF1gNnSTqviLnmANvN7HhnMShKP5rZP4G2IX53Pv12E7DazNrM7BCwGphXqHxm9qKZ5cLN9URz+gwrZKw2s/UW/Sd8PPacCpLxOIbbrgV9vR8vY/g08SVg+fEeo9D9mA8vIiMzoml4i0nShcB04LXQdG8YUljaP+RBerkNeFHS65LuCW0TzGxPWN4LTEg5Y787GPyCLaV+hPz7Lc2sXyN6R9xviqT/SForaXZoOz9kKna+fLZrmn04G9hnZltjbaXUjx/hRWQUkjQOWAHcZ2ZHgN8AFwGfAPYQfRxO03VmNgO4GfimpOvjPwzvnFI/tlzRRGa3An8JTaXWj4OUSr8NRdLDQA54IjTtAS4ws+nAt4E/S6pOKV5Jb9dj3MngNzWl1I9D8iIyMiUzDa+kMUQF5Akz+yuAme0zs14z6wN+z4dDLankNrOWcL0fWBny7OsfpgrX+9PMGNwMNJnZvpC3pPoxyLffip5V0leAzwFfDoWOMETUGpZfJ9rHMDVkiQ95FTxfgu2ayvaWdAZwO/BUf1sp9eNwvIiMTElMwxvGS5cA75jZT2Pt8X0IC4D+oz6eAe6QVCFpCnAJ0c64QmY8U1K2f5lox+tbIUv/kUJ3A3+LZbwrHG00CzgcG74ptEHv+kqpH2Py7bdVwI2SxodhmxtDW0FImgd8F7jVzDpi7edKyoTlBqI+2xEyHpE0K/w93xV7ToXKmO92Tev1Phd418wGhqlKqR+Hlcbe/NF4IToa5j2idwIPp5ThOqLhjDeBTeFyC/AnYHNofwY4L3afh0PmLRTh6A2iI1reCJfm/r4CaoF/AFuBNcDZoV3Ar0LGzcDMIvXlmUArUBNrS7UfiQraHqCHaIz760n6jWjfxLZw+WqB820j2n/Q//f427DuF8P23wQ0AZ+PPc5Mon/k24FfEs6cUcCMeW/XQr7eh8oY2v8IfOOYdVPpx3wuftoT55xziflwlnPOucS8iDjnnEvMi4hzzrnEvIg455xLzIuIc865xLyIOFcgkmpjZ1/dGzuT7AeSfp12PudOBj/E17kikLQY+MDMfpx2FudOJv8k4lyRSfq0pL+H5cWSHpP0L0m7JN0u6dEwT8QL4TQ3/XNHrA0ntVxV5DMdOzcsLyLOpe8i4DNEJ4NcBrxsZh8HOoH5oZD8AlhoZlcBS4HvpxXWubgz0g7gnON5M+uRtJloQqQXQvtmokmJLgWmAavD5HUZotNmOJc6LyLOpa8LwMz6JPXYhzsq+4heowKazeyatAI6NxwfznKu9G0BzpV0DUTTAUj6WMqZnAO8iDhX8iyaonUh8IikN4jO6PqpdFM5F/FDfJ1zziXmn0Scc84l5kXEOedcYl5EnHPOJeZFxDnnXGJeRJxzziXmRcQ551xiXkScc84l9j8UQDVAA6Tl2QAAAABJRU5ErkJggg==\n", 477 | "text/plain": [ 478 | "
" 479 | ] 480 | }, 481 | "metadata": { 482 | "needs_background": "light" 483 | } 484 | } 485 | ] 486 | }, 487 | { 488 | "cell_type": "markdown", 489 | "metadata": { 490 | "id": "EmXldp6aSUwa" 491 | }, 492 | "source": [ 493 | "## License\n", 494 | "\n", 495 | "This Deep Learning Tutorial is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0) International License." 496 | ] 497 | }, 498 | { 499 | "cell_type": "markdown", 500 | "metadata": { 501 | "id": "IGfUENRWu4Pm" 502 | }, 503 | "source": [ 504 | "## Acknowledgments\n", 505 | "Henrique F. de Arruda acknowledges FAPESP for sponsorship (grant no. 2018/10489-0). H. F. de Arruda also thanks Soremartec S.A. and Soremartec Italia, Ferrero Group, for partial financial support (from 1st July 2021). His funders had no role in study design, data collection, and analysis, decision to publish, or manuscript preparation. Alexandre Benatti thanks Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Luciano da F. Costa thanks CNPq (grant no. 307085/2018-0) and FAPESP (proc. 15/22308-2) for sponsorship. César H. Comin thanks FAPESP (Grant Nos. 2018/09125-4 and 2021/12354-8) for financial support. This work has been supported also by FAPESP grants 11/50761-2 and 15/22308-2." 506 | ] 507 | } 508 | ] 509 | } --------------------------------------------------------------------------------