├── .gitignore ├── GAN-fashion-MNIST ├── README.md ├── __init__.py ├── gan.py ├── imgs │ ├── D_loss.png │ ├── G_loss.png │ ├── fashion-mnist │ │ ├── D_loss.png │ │ ├── G_loss.png │ │ ├── step-000.png │ │ ├── step-1000.png │ │ ├── step-17000.png │ │ ├── step-2000.png │ │ ├── step-29000.png │ │ ├── step-3000.png │ │ ├── step-4000.png │ │ ├── step-44000.png │ │ ├── step-62000.png │ │ └── train.gif │ ├── mnist │ │ ├── D_loss.png │ │ ├── G_loss.png │ │ ├── step-000.png │ │ ├── step-1000.png │ │ ├── step-13000.png │ │ ├── step-15000.png │ │ ├── step-16000.png │ │ ├── step-2000.png │ │ ├── step-25000.png │ │ ├── step-3000.png │ │ ├── step-37000.png │ │ ├── step-4000.png │ │ ├── step-50000.png │ │ └── train.gif │ ├── step-000.png │ ├── step-1000.png │ ├── step-2000.png │ ├── step-3000.png │ ├── step-4000.png │ ├── step-44000.png │ ├── step-62000.png │ ├── train.gif │ ├── vanilla_gan_arch.png │ └── vanilla_gan_detailed_arch.png └── utils.py ├── GoogleDevMLRecipes ├── 2 │ ├── iris.pdf │ └── src.py ├── 5 │ └── src.py ├── 6 │ ├── label_image.py │ └── test_examples │ │ ├── 201ce94e7998a27257cdc2426ae7060c.jpg │ │ └── brilliant-light-white-rose-1920x1200-photo-dsc09528.jpg └── README.md ├── README.md ├── exploring-MNIST ├── README.md └── __init__.py ├── exploring-census-dataset ├── .ipynb_checkpoints │ └── Exploring Census Dataset with Facets and LIME-checkpoint.ipynb └── Exploring Census Dataset with Facets and LIME.ipynb └── exploring-spiral-dataset ├── .ipynb_checkpoints └── tensorflow-spiral-checkpoint.ipynb └── tensorflow-spiral.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | *.gz 2 | exploring-census-dataset/graphs/* 3 | 4 | -------------------------------------------------------------------------------- /GAN-fashion-MNIST/README.md: -------------------------------------------------------------------------------- 1 | # Vanilla GAN for Fashion MNIST or classic MNIST 2 | 3 | GANs are a very hot topic in the Deep Learning world! The goal of this implementation is to 4 | have a simple and fun demo, of a simple GAN that you can train in a few minutes but still 5 | getting some cool results! 6 | 7 | ![](imgs/vanilla_gan_detailed_arch.png) 8 | 9 | [Want to learn more about GANs ?](https://github.com/mari-linhares/DeepLearning#gans) 10 | 11 | The code is basically the same available [here](https://github.com/wiseodd/generative-models/blob/master/GAN/vanilla_gan/gan_tensorflow.py) 12 | by @wiseodd, that has a great repo with a lot of GAN implementations 13 | and a great [blog post](http://wiseodd.github.io/techblog/2016/09/17/gan-tensorflow/) 14 | about this vanilla implementation. 15 | 16 | I've changed this code mainly in order to make it simpler for 17 | beginners to get started with GANs and TensorFlow. 18 | 19 | I've also implemented other features as: 20 | 21 | * TensorBoard visualization for the discriminator and 22 | generator losses; 23 | * Downloading the fashion mnist and classic mnist datasets 24 | automatically; 25 | * Added comments and refactored code to make it simpler. 26 | 27 | Looking for better generated samples? Here is a code for a 28 | [DCGAN](https://github.com/carpedm20/DCGAN-tensorflow) by @carpedm20. 29 | It will take a lot more time to train, but it will generate better results. 30 | 31 | ## How to run? 32 | 33 | ### Install 34 | 35 | * [TensorFlow](https://www.tensorflow.org/install/) 36 | * [Matplotlib](https://matplotlib.org/) 37 | 38 | ### Run 39 | 40 | Run fashion MNIST: 41 | 42 | ```bash 43 | python gan.py 44 | ``` 45 | 46 | run classic MNIST: 47 | 48 | ```bash 49 | python gan.py --mnist=mnist 50 | ``` 51 | 52 | There are some arguments you can play with, to check all of them 53 | run: 54 | 55 | ```bash 56 | python gan.py -h 57 | ``` 58 | 59 | You'll see something like: 60 | 61 | ``` 62 | usage: gan.py [-h] [--output_path OUTPUT_PATH] [--input_path INPUT_PATH] 63 | [--log_path LOG_PATH] [--mnist MNIST] [--z_dim Z_DIM] 64 | [--batch_size BATCH_SIZE] [--train_steps TRAIN_STEPS] 65 | 66 | optional arguments: 67 | -h, --help show this help message and exit 68 | --output_path OUTPUT_PATH 69 | Output path for the generated images. 70 | --input_path INPUT_PATH 71 | Input path for the fashion mnist.If not available data 72 | will be downloaded. 73 | --log_path LOG_PATH Log path for tensorboard. 74 | --mnist MNIST Choose to use "fashion" (fashion-mnist) or "mnist" 75 | (classic mnist) dataset. 76 | --z_dim Z_DIM Output path for the generated images. 77 | --batch_size BATCH_SIZE 78 | Batch size used for training. 79 | --train_steps TRAIN_STEPS 80 | Number of steps used for training. 81 | ``` 82 | 83 | ## Check tensorboard 84 | 85 | ```bash 86 | # tensorboard_log is the default path to tensorboard logs 87 | tensorboard --logdir=tensorboad_log 88 | 89 | # a more general command is 90 | tensorboard --logdir= 91 | ``` 92 | 93 | ## Training example 94 | 95 | ### Fashion MNIST 96 | 97 | ![](imgs/fashion-mnist/train.gif) 98 | 99 | ## Discriminator Loss 100 | 101 | ![](imgs/fashion-mnist/D_loss.png) 102 | 103 | ## Generator Loss 104 | 105 | ![](imgs/fashion-mnist/G_loss.png) 106 | 107 | 108 | ### Classic MNIST 109 | 110 | ![](imgs/mnist/train.gif) 111 | 112 | ## Discriminator Loss 113 | 114 | ![](imgs/mnist/D_loss.png) 115 | 116 | ## Generator Loss 117 | 118 | ![](imgs/mnist/G_loss.png) 119 | -------------------------------------------------------------------------------- /GAN-fashion-MNIST/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/__init__.py -------------------------------------------------------------------------------- /GAN-fashion-MNIST/gan.py: -------------------------------------------------------------------------------- 1 | ''' 2 | GAN Architecture and basic implementation based on https://github.com/wiseodd/generative-models/blob/master/GAN/vanilla_gan/gan_tensorflow.py 3 | Also check this great blog post about it: http://wiseodd.github.io/techblog/2016/09/17/gan-tensorflow/ 4 | 5 | Author: Marianne Linhares 6 | Date: September 2017. 7 | 8 | The main new features available on this implementation are: 9 | - Refactored and added comments to make implementation more understable for beginners; 10 | - Added maybe_download implementation to download dataset automatically; 11 | - Added TensorBoard to keep track of losses and generated samples during training; 12 | - Created a utils file to clean the gan implementation; 13 | ''' 14 | 15 | import argparse 16 | import numpy as np 17 | 18 | import utils # utils file containing auxiliar code for this script 19 | 20 | import tensorflow as tf 21 | from tensorflow.examples.tutorials.mnist import input_data # load mnist 22 | 23 | # ------------------------- Parser ----------------------- 24 | 25 | parser = argparse.ArgumentParser() 26 | 27 | # I/O related args 28 | parser.add_argument('--output_path', default='out/', type=str, 29 | help='Output path for the generated images.') 30 | 31 | parser.add_argument('--input_path', default='mnist/', type=str, 32 | help='Input path for the fashion mnist.' 33 | 'If not available data will be downloaded.') 34 | 35 | parser.add_argument('--log_path', default='tensorboard_log/', type=str, 36 | help='Log path for tensorboard.') 37 | 38 | parser.add_argument('--mnist', default='fashion', type=str, 39 | help='Choose to use "fashion" (fashion-mnist)' 40 | ' or "mnist" (classic mnist) dataset.') 41 | 42 | # hyper-parameters 43 | parser.add_argument('--z_dim', default=100, type=int, 44 | help='Output path for the generated images.') 45 | 46 | parser.add_argument('--batch_size', default=16, type=int, 47 | help='Batch size used for training.') 48 | 49 | parser.add_argument('--train_steps', default=100000, type=int, 50 | help='Number of steps used for training.') 51 | 52 | FLAGS = parser.parse_args() 53 | 54 | # ---------- Helper functions and variables ----------------------- 55 | 56 | # MNIST related constants 57 | # images have shape of 28x28 in gray scale 58 | MNIST_HEIGHT = 28 59 | MNIST_WIDTH = 28 60 | MNIST_DIM = 1 # gray scale 61 | 62 | # to keep things simple we'll deal with the images as a 63 | # flat tensor of MNIST_FLAT shape 64 | MNIST_FLAT_DIM = MNIST_HEIGHT * MNIST_WIDTH * MNIST_DIM 65 | 66 | # here is a nice blog post about xavier init in case you're not familiar with it 67 | # https://prateekvjoshi.com/2016/03/29/understanding-xavier-initialization-in-deep-neural-networks/ 68 | # in resume: we're using this to ensure that the weights are being initialized with 69 | # a reasonable range before we start training the network 70 | def xavier_init(shape): 71 | '''Implementing xavier init.''' 72 | first_dim = shape[0] 73 | xavier_stddev = 1. / tf.sqrt(first_dim / 2.) 74 | return tf.random_normal(shape=shape, stddev=xavier_stddev) 75 | 76 | 77 | # ---------- Variables and Placeholder definitions ----------------- 78 | 79 | # discriminator 80 | 81 | # flat MNIST image is expected to be feeded 82 | X = tf.placeholder(tf.float32, shape=[None, MNIST_FLAT_DIM]) # [batch_size, MNIST_FLAT_DIM] 83 | 84 | # hidden layer variables with 128 nodes 85 | D_W1 = tf.Variable(xavier_init([MNIST_FLAT_DIM, 128])) # [MNIST_FLAT_DIM, 128] 86 | D_b1 = tf.Variable(tf.zeros(shape=[128])) # [128] 87 | 88 | # final layer variables 89 | D_W2 = tf.Variable(xavier_init([128, 1])) # [128, 1] 90 | D_b2 = tf.Variable(tf.zeros(shape=[1])) # [1] 91 | 92 | # list with the discriminator variables 93 | theta_D = [D_W1, D_W2, D_b1, D_b2] 94 | 95 | # generator 96 | 97 | # Z is a random noise given as input to the generator 98 | # transform this into something close to what the discriminator 99 | # thinks is a real sample 100 | Z = tf.placeholder(tf.float32, shape=[None, FLAGS.z_dim]) # [batch_size, Z_DIM] 101 | 102 | # hidden layer variables with 128 nodes 103 | G_W1 = tf.Variable(xavier_init([FLAGS.z_dim, 128])) # [Z_DIM, 128] 104 | G_b1 = tf.Variable(tf.zeros(shape=[128])) # [128] 105 | 106 | # final layer variables 107 | G_W2 = tf.Variable(xavier_init([128, MNIST_FLAT_DIM])) # [128, MNIST_FLAT_DIM] 108 | G_b2 = tf.Variable(tf.zeros(shape=[MNIST_FLAT_DIM])) # [MNIST_FLAT_DIM] 109 | 110 | # list with the generator variables 111 | theta_G = [G_W1, G_W2, G_b1, G_b2] 112 | 113 | 114 | # ---------- Defining Generator and Discriminator models ----------------- 115 | def sample_Z(): 116 | '''Generate random noise for the generator.''' 117 | return np.random.uniform(-1., 1., size=[FLAGS.batch_size, FLAGS.z_dim]) 118 | 119 | 120 | def generator(z): 121 | '''Generator model, given a random noise it 122 | returns a fake sample. 123 | ''' 124 | G_h1 = tf.nn.relu(tf.matmul(z, G_W1) + G_b1) # first layer (relu) 125 | G_log_prob = tf.matmul(G_h1, G_W2) + G_b2 # second layer 126 | G_prob = tf.nn.sigmoid(G_log_prob) 127 | 128 | return G_prob 129 | 130 | 131 | def discriminator(x): 132 | '''Discriminator model, returns the probability 133 | of the samples at x be real. 134 | ''' 135 | D_h1 = tf.nn.relu(tf.matmul(x, D_W1) + D_b1) # first layer (relu) 136 | D_logit = tf.matmul(D_h1, D_W2) + D_b2 # second layer 137 | D_prob = tf.nn.sigmoid(D_logit) 138 | 139 | return D_prob, D_logit 140 | 141 | G_sample = generator(Z) # get a generator sample 142 | D_real, D_logit_real = discriminator(X) # give a real input to the discriminator 143 | D_fake, D_logit_fake = discriminator(G_sample) # give a fake input to the discriminator 144 | 145 | # D_loss = -tf.reduce_mean(tf.log(D_real) + tf.log(1. - D_fake)) 146 | # G_loss = -tf.reduce_mean(tf.log(D_fake)) 147 | # Alternative losses: 148 | # ------------------- 149 | D_loss_real = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=D_logit_real, labels=tf.ones_like(D_logit_real))) 150 | D_loss_fake = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=D_logit_fake, labels=tf.zeros_like(D_logit_fake))) 151 | 152 | D_loss = D_loss_real + D_loss_fake 153 | 154 | G_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=D_logit_fake, labels=tf.ones_like(D_logit_fake))) 155 | 156 | D_solver = tf.train.AdamOptimizer().minimize(D_loss, var_list=theta_D) # only updates the discriminator vars 157 | G_solver = tf.train.AdamOptimizer().minimize(G_loss, var_list=theta_G) # only updates the generator vars 158 | 159 | # -------------- TensorBoard summaries ----------------- 160 | 161 | summ_D_loss_real = tf.summary.scalar("D_loss_real", D_loss_real) 162 | summ_D_loss_fake = tf.summary.scalar("D_loss_fake", D_loss_fake) 163 | summ_D_loss = tf.summary.scalar("D_loss", D_loss) 164 | 165 | summ_D_losses = tf.summary.merge([summ_D_loss_real, summ_D_loss_fake, 166 | summ_D_loss]) 167 | 168 | summ_G_loss = tf.summary.scalar("G_loss", G_loss) 169 | 170 | # -------------- Load the dataset ------------------------ 171 | 172 | # download mnist if needed 173 | utils.maybe_download(FLAGS.input_path, FLAGS.mnist) 174 | 175 | # import mnist dataset 176 | data = input_data.read_data_sets(FLAGS.input_path, one_hot=True) 177 | 178 | # -------------- Train models ------------------------ 179 | 180 | # create session 181 | sess = tf.Session() 182 | sess.run(tf.global_variables_initializer()) 183 | 184 | # create summary writer 185 | summary_writer = tf.summary.FileWriter(FLAGS.log_path, graph=tf.get_default_graph()) 186 | 187 | for i in range(FLAGS.train_steps): 188 | 189 | # eventually plot images that are being generated 190 | if i % 1000 == 0: 191 | samples = sess.run(G_sample, feed_dict={Z: sample_Z()}) 192 | utils.save_plot(samples, FLAGS.output_path, i) 193 | 194 | # get real data 195 | X_batch, _ = data.train.next_batch(FLAGS.batch_size) 196 | 197 | # train discriminator 198 | _, D_loss_curr, summ = sess.run([D_solver, D_loss, summ_D_losses], 199 | feed_dict={X: X_batch, Z: sample_Z()}) 200 | summary_writer.add_summary(summ, i) 201 | 202 | # train generator 203 | _, G_loss_curr, summ = sess.run([G_solver, G_loss, summ_G_loss], feed_dict={Z: sample_Z()}) 204 | summary_writer.add_summary(summ, i) 205 | 206 | # eventually print train losses 207 | if i % 1000 == 0: 208 | print('Iter: {}'.format(i)) 209 | print('D loss: {:.4}'. format(D_loss_curr)) 210 | print('G_loss: {:.4}'.format(G_loss_curr)) 211 | print() 212 | -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/D_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/D_loss.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/G_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/G_loss.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/D_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/D_loss.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/G_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/G_loss.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-1000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-1000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-17000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-17000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-2000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-2000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-29000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-29000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-3000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-3000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-4000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-4000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-44000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-44000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/step-62000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/step-62000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/fashion-mnist/train.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/fashion-mnist/train.gif -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/D_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/D_loss.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/G_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/G_loss.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-1000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-1000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-13000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-13000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-15000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-15000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-16000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-16000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-2000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-2000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-25000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-25000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-3000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-3000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-37000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-37000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-4000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-4000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/step-50000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/step-50000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/mnist/train.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/mnist/train.gif -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-1000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-1000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-2000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-2000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-3000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-3000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-4000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-4000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-44000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-44000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/step-62000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/step-62000.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/train.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/train.gif -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/vanilla_gan_arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/vanilla_gan_arch.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/imgs/vanilla_gan_detailed_arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GAN-fashion-MNIST/imgs/vanilla_gan_detailed_arch.png -------------------------------------------------------------------------------- /GAN-fashion-MNIST/utils.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Author: Marianne Linhares 3 | Date: September 2017. 4 | 5 | This is a script containing util functions to used by the gan.py script, 6 | in order to plot images, save images, use xavier_init, etc. 7 | ''' 8 | 9 | import matplotlib.pyplot as plt 10 | import matplotlib.gridspec as gridspec 11 | 12 | import os 13 | 14 | from tensorflow.contrib.learn.python.learn.datasets import base 15 | 16 | 17 | def generate_4x4_figure(samples): 18 | '''Generate a 4x4 figure.''' 19 | fig = plt.figure(figsize=(4, 4)) 20 | gs = gridspec.GridSpec(4, 4) 21 | gs.update(wspace=0.05, hspace=0.05) 22 | 23 | for i, sample in enumerate(samples): 24 | ax = plt.subplot(gs[i]) 25 | plt.axis('off') 26 | ax.set_xticklabels([]) 27 | ax.set_yticklabels([]) 28 | ax.set_aspect('equal') 29 | plt.imshow(sample.reshape(28, 28), cmap='Greys_r') 30 | 31 | return fig 32 | 33 | def maybe_create_out_path(out_path): 34 | '''If output path does not exist it will be created.''' 35 | if not os.path.exists(out_path): 36 | os.makedirs(out_path) 37 | 38 | def save_plot(samples, out_path, train_step): 39 | '''Generates a plot and saves it.''' 40 | fig = generate_4x4_figure(samples) 41 | 42 | file_name = 'step-{}.png'.format(str(train_step).zfill(3)) 43 | full_path = os.path.join(out_path, file_name) 44 | 45 | print 'Saving image:', full_path 46 | 47 | maybe_create_out_path(out_path) 48 | 49 | plt.savefig(full_path, bbox_inches='tight') 50 | plt.close(fig) 51 | 52 | def maybe_download(input_path, mnist): 53 | '''If dataset not available in the input path download it.''' 54 | 55 | if mnist == 'fashion': 56 | 57 | base_url = 'http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/' 58 | file_names = ['train-images-idx3-ubyte.gz', 59 | 'train-labels-idx1-ubyte.gz', 60 | 't10k-images-idx3-ubyte.gz', 61 | 't10k-labels-idx1-ubyte.gz'] 62 | 63 | print 'Maybe will download the dataset, this can take a while' 64 | for name in file_names: 65 | base.maybe_download(name, input_path, base_url + name) 66 | 67 | elif mnist == 'mnist': 68 | pass 69 | else: 70 | raise ValueError('Invalid dataset use only mnist = ["fashion", "mnist"])') 71 | 72 | -------------------------------------------------------------------------------- /GoogleDevMLRecipes/2/iris.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GoogleDevMLRecipes/2/iris.pdf -------------------------------------------------------------------------------- /GoogleDevMLRecipes/2/src.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from sklearn.datasets import load_iris 3 | from sklearn import tree 4 | 5 | iris = load_iris() 6 | 7 | test_idx = [0, 50, 100] 8 | 9 | # training data 10 | train_target = np.delete(iris.target, test_idx) 11 | train_data = np.delete(iris.data, test_idx, axis = 0) 12 | 13 | # testing data 14 | test_target = iris.target[test_idx] 15 | test_data = iris.data[test_idx] 16 | 17 | # classifier 18 | clf = tree.DecisionTreeClassifier() 19 | clf.fit(train_data, train_target) 20 | 21 | print 'real target', 22 | print test_target 23 | print 'predicted value:', 24 | print clf.predict(test_data) 25 | 26 | # visualization 27 | from sklearn.externals.six import StringIO 28 | import pydot 29 | 30 | dot_data = StringIO() 31 | tree.export_graphviz(clf, 32 | out_file=dot_data, 33 | feature_names=iris.feature_names, 34 | class_names=iris.target_names, 35 | filled = True, 36 | rounded = True, 37 | impurity = False) 38 | 39 | graph = pydot.graph_from_dot_data(dot_data.getvalue()) 40 | graph.write_pdf("iris.pdf") 41 | -------------------------------------------------------------------------------- /GoogleDevMLRecipes/5/src.py: -------------------------------------------------------------------------------- 1 | from scipy.spatial import distance 2 | 3 | class myKNN(): 4 | 5 | def closest(self, row): 6 | best_dist = distance.euclidean(row, self.x[0]) 7 | best_index = 0 8 | for i in xrange(1, len(self.x)): 9 | dist = distance.euclidean(row, self.x[i]) 10 | if dist < best_dist: 11 | best_dist = dist 12 | best_index = i 13 | 14 | return self.y[best_index] 15 | 16 | def fit(self, x, y): 17 | self.x = x 18 | self.y = y 19 | 20 | def predict(self, test): 21 | pred = [] 22 | for row in test: 23 | label = self.closest(row) 24 | pred.append(label) 25 | return pred 26 | 27 | 28 | from sklearn.datasets import load_iris 29 | from sklearn.model_selection import train_test_split 30 | from sklearn.neighbors import KNeighborsClassifier 31 | from sklearn.metrics import accuracy_score 32 | 33 | iris = load_iris() 34 | 35 | x = iris.data 36 | y = iris.target 37 | 38 | x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.5) 39 | 40 | # classifier 41 | neigh_clf = KNeighborsClassifier() 42 | neigh_clf.fit(x_train, y_train) 43 | 44 | my_clf = myKNN() 45 | my_clf.fit(x_train, y_train) 46 | 47 | # pretidiction 48 | neigh_pred = neigh_clf.predict(x_test) 49 | my_pred = my_clf.predict(x_test) 50 | 51 | # accuracy 52 | print accuracy_score(y_test, neigh_pred) 53 | print accuracy_score(y_test, my_pred) 54 | -------------------------------------------------------------------------------- /GoogleDevMLRecipes/6/label_image.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf, sys 2 | 3 | image_path = sys.argv[1] 4 | 5 | # Read in the image_data 6 | image_data = tf.gfile.FastGFile(image_path, 'rb').read() 7 | 8 | # Loads label file, strips off carriage return 9 | label_lines = [line.rstrip() for line 10 | in tf.gfile.GFile("/tf_files/retrained_labels.txt")] 11 | 12 | # Unpersists graph from file 13 | with tf.gfile.FastGFile("/tf_files/retrained_graph.pb", 'rb') as f: 14 | graph_def = tf.GraphDef() 15 | graph_def.ParseFromString(f.read()) 16 | _ = tf.import_graph_def(graph_def, name='') 17 | 18 | with tf.Session() as sess: 19 | # Feed the image_data as input to the graph and get first prediction 20 | softmax_tensor = sess.graph.get_tensor_by_name('final_result:0') 21 | 22 | predictions = sess.run(softmax_tensor, \ 23 | {'DecodeJpeg/contents:0': image_data}) 24 | 25 | # Sort to show labels of first prediction in order of confidence 26 | top_k = predictions[0].argsort()[-len(predictions[0]):][::-1] 27 | 28 | for node_id in top_k: 29 | human_string = label_lines[node_id] 30 | score = predictions[0][node_id] 31 | print('%s (score = %.5f)' % (human_string, score)) 32 | -------------------------------------------------------------------------------- /GoogleDevMLRecipes/6/test_examples/201ce94e7998a27257cdc2426ae7060c.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GoogleDevMLRecipes/6/test_examples/201ce94e7998a27257cdc2426ae7060c.jpg -------------------------------------------------------------------------------- /GoogleDevMLRecipes/6/test_examples/brilliant-light-white-rose-1920x1200-photo-dsc09528.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/GoogleDevMLRecipes/6/test_examples/brilliant-light-white-rose-1920x1200-photo-dsc09528.jpg -------------------------------------------------------------------------------- /GoogleDevMLRecipes/README.md: -------------------------------------------------------------------------------- 1 | # Google Developers ML Recipes 2 | 3 | This folder contains the code for the exercises proposed at [Machine Learning Recipes from Google](https://www.youtube.com/playlist?list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal) 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DeepLearning: [bit.ly/awesome-dl](https://bit.ly/awesome-dl) 2 | 3 | This repository keeps track of my path towards understanding Deep Learning concepts (mainly with TensorFlow). 4 | 5 | ## What can you find here? 6 | 7 | * Links to my personal projects while learning Deep Learning and Tensorflow 8 | * Links for online resources to get started with Deep Learning and TensorFlow 9 | 10 | ## Personal Projects 11 | 12 | * [Getting Started with Deep Learning with MNIST](https://github.com/mari-linhares/mnist-tensorflow) 13 | * [MNIST running on Android](https://github.com/mari-linhares/mnist-android-tensorflow) 14 | * [Exploring Spiral Dataset](exploring-spiral-dataset/) 15 | * [Vanilla GANs for fashion MNIST](GAN-fashion-MNIST/) 16 | * [Easy 21 - Getting started with Reinforcement Learning](https://github.com/mari-linhares/easy21) 17 | * [TensorFlow Brasil - A repository for TensorFlow code samples and tutorials in Portuguese](https://github.com/mari-linhares/tensorflow-brasil) 18 | * [Teachable games - Using DeepLearnJS to play games just using a camera](https://github.com/mari-linhares/teachable-machine-games) 19 | * [Modified version of text generation tutorial including fun examples](https://github.com/mari-linhares/tf-eager-text-generation) 20 | 21 | ## Get Started with Deep Learning and TensorFlow 22 | 23 | Go to [bit.ly/awesome-dl](https://bit.ly/awesome-dl) 24 | 25 | ### DL and ML 101: blog posts and videos to get started 26 | 27 | * [A friendly introduction to Deep Learning and Neural Networks](https://www.youtube.com/watch?v=BR9h47Jtqyw) 28 | * [How Deep Neural Networks Work](https://www.youtube.com/watch?v=ILsA4nyG7I0) 29 | * [Tensorflow and deep learning, without a PhD, Martin Gorner, Google](https://www.youtube.com/watch?v=sEciSlAClL8&t=2163s) 30 | * [TensorFlow Tutorial (Sherry Moore, Google Brain)](https://www.youtube.com/watch?v=Ejec3ID_h0w). 31 | 32 | ### Math for Deep Learning and Machine Learning 33 | 34 | * [The Matrix Calculus You Need For Deep Learning](https://arxiv.org/abs/1802.01528) 35 | * [3Blue1Brown Youtube Channel](https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw) 36 | 37 | ### CNNs - Convolutional Neural Networks 38 | * [How Convolutional Neural Networks work](https://www.youtube.com/watch?v=FmpDIaiMIeA&t=1s) 39 | 40 | ### RNNs - Recurrent Neural Networks 41 | 42 | * [Understanding LSTM Networks](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) 43 | * [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) 44 | 45 | ### GANs - Generative Adversarial Networks 46 | 47 | * [Original Paper](https://arxiv.org/abs/1406.2661) 48 | * [Introduction to GANs (Keras)](https://www.analyticsvidhya.com/blog/2017/06/introductory-generative-adversarial-networks-gans/) 49 | * [GANs for beginners (TensorFlow)](https://www.oreilly.com/learning/generative-adversarial-networks-for-beginners) 50 | * [Blog post about the different kinds of GANs](http://guimperarnau.com/blog/2017/03/Fantastic-GANs-and-where-to-find-them) 51 | * [DCGAN implementation on TensorFlow](https://github.com/carpedm20/DCGAN-tensorflow) 52 | * [Generative models implementation (TensorFlow, Pytorch)](https://github.com/wiseodd/generative-models/) 53 | 54 | 55 | ### RL - Reinforcement Learning 56 | 57 | * [Full Course by David Silver](https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9) 58 | 59 | ### Blogs 60 | 61 | * [Colah's Blog](http://colah.github.io/) 62 | * [Open AI](https://blog.openai.com/) 63 | * [Fast AI](http://www.fast.ai/) 64 | 65 | ### University Courses 66 | 67 | * [CS 20SI: TensorFlow for Deep Learning Research](http://web.stanford.edu/class/cs20si/syllabus.html) 68 | * [CS231n: Convolutional Neural Networks for Visual Recognition](http://cs231n.github.io/) 69 | * [MIT 6.S094: Deep Learning for Self-Driving Cars](https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf) 70 | 71 | ### Youtube series 72 | 73 | * [Siraj Raval](https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A) 74 | * [sentdex](https://www.youtube.com/channel/UCfzlCWGWYyIQ0aLC5w48gBQ) 75 | * [Hvass Lab](https://www.youtube.com/user/hvasslabs/videos) 76 | * [Google Dev - Machine Learning Recipes](https://www.youtube.com/watch?v=cKxRvEZd3Mw&index=7&list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal) 77 | * [3Blue1Brown](https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw) 78 | 79 | ### GO DEEP - learn more about deep learning in 1 or 2 months 80 | 81 | * [How do I learn deep learning in 2 months](https://www.quora.com/How-do-I-learn-deep-learning-in-2-months) 82 | * [Learn deep learning, from novice to advanced](https://www.commonlounge.com/discussion/81f5bbcfea4e44b9b2bd081d1ea536ac/main) 83 | 84 | 85 | ### Advanced tutorials 86 | 87 | * [Jason Mayes - Rock Paper Scissors - Machine Learning Style using Tensor Flow](https://www.youtube.com/watch?v=mtRDNDqjUzM) 88 | -------------------------------------------------------------------------------- /exploring-MNIST/README.md: -------------------------------------------------------------------------------- 1 | ## Exploring MNIST with TensorFlow 2 | 3 | Go to [http://github.com/mari-linhares/mnist-tensorflow](http://github.com/mari-linhares/mnist-tensorflow) 4 | -------------------------------------------------------------------------------- /exploring-MNIST/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hereismari/DeepLearning/182d4d5cb68cbf9df9b9933ebb97739269af3e3a/exploring-MNIST/__init__.py -------------------------------------------------------------------------------- /exploring-spiral-dataset/tensorflow-spiral.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Hi,\n", 8 | "\n", 9 | "This is a demonstration of how powerfull deep neural networks are!\n", 10 | "\n", 11 | "We're going to generate the spiral database and then try to model this data with softmax and then with a neural network and see how it goes.\n", 12 | "\n", 13 | "We're going to follow this tutorial: http://cs231n.github.io/neural-networks-case-study/\n", 14 | "If you want more details check it! But here we're going to use TensorFlow!\n", 15 | "\n", 16 | "Have fun!" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Generating the data" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 1, 29 | "metadata": {}, 30 | "outputs": [ 31 | { 32 | "data": { 33 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VFX6+D/nTp8UQgq99yK9SREs9CLYUCyoq2Jfd921\n7/7c1bWt7lpWLHyxg4hgARWkWegl9F5CS2iB9GTqvff8/pgQMpmZECCkkPt5Hh4y95xz7zvJzH3v\ne94mpJQYGBgYGBicRqlsAQwMDAwMqhaGYjAwMDAwCMJQDAYGBgYGQRiKwcDAwMAgCEMxGBgYGBgE\nYSgGAwMDA4MgDMVgYGBgYBCEoRgMDAwMDIIwFIOBgYGBQRDmyhbgfEhMTJTNmjWrbDEMDAwMqhXr\n168/JaVMOtu8aqkYmjVrRnJycmWLYWBgYFCtEEIcKsu8ctlKEkJ8LIRIF0JsizAuhBDvCCH2CSG2\nCCG6FxsbLoTYXTj2dHnIY2BgYGBw/pSXj+FTYHgp4yOA1oX/JgHvAwghTMDkwvEOwAQhRIdyksnA\nwMDA4DwoF8UgpVwKZJYyZSzwuQywGogTQtQHegP7pJT7pZQ+4KvCuQYGBgYGlURFRSU1BFKLvU4r\nPBbpeAhCiElCiGQhRPLJkycvmqAGBgYGNZ1qE64qpZwipewppeyZlHRWp7qBgYGBwXlSUVFJR4DG\nxV43KjxmiXDcwMDAwKCSqCiLYS4wsTA66XIgR0p5DFgHtBZCNBdCWIFbCucaVEOkmoae/Rf0E73Q\n069Az/sfUnorWywDA4NzpFwsBiHEDOBKIFEIkQY8T8AaQEr5ATAPGAnsA1zA3YVjqhDiEWABYAI+\nllJuLw+ZDCoWqR1HZowDmQ/oIHOgYArStxripyGEqGwRDQwMyki5KAYp5YSzjEvg4Qhj8wgoDoNq\njMyfAtIF6MWOekHdDr41YLu8skQzMDA4R6qN89mg8pHaUaSagpR66KBvGaCGWeQG/7qLLpuBgUH5\nUS1LYhhULFJNQWb/CdSDIBQQURD7L4T96jOTlHjQwmXb20DEVZSoBgYG5YBhMRiUitRdyIwJoO4B\nvAELQD+FzP4T0n/GHSScdwKO8CdxjKoQWQ0MDMoHQzEYlI5nHkgfIEsMeJH5H555aR8BzpsAG+AI\nWBXCgYh7G6HEV5y8BgYGF4yxlWRQKlJNIRBIFjIC6r6iV0IIROzfkM47wbcyoBhsVyKU6AqT1cDA\noHwwFINBqQhLa6RwFkYcFUcBS7vQ+ebGYL65YoQzMDC4KBhbSQalYx8JwkHoR8WKiJpUGRIZGBhc\nZAzFYFAqQtgR8TPB0plAzqINlAaI2u8jwlgMBgYG1R9jK8ngrAhzE0TC10g9E6QXlHpGJrOBwSWM\noRgMyowRXWRgUDMwtpIMDAwMDIIwFIOBgYGBQRCGYjAwMDAwCMJQDAYGBgYGQRjO52qI1E4h8/8H\n3oWACRxjEVEPGlnGBgYG5YKhGKoZUs8NNMTRMwAtcLDgM6T3d0j4lkAjPAMDA4Pzx9hKqmbIgmmg\nn6RIKQDgA/UQeBZVllgGBgaXEOWiGIQQw4UQu4UQ+4QQT4cZf0IIsanw3zYhhCaEiC8cOyiE2Fo4\nllwe8lzSuGcRWukUwIv0/lrR0hiUI4eys3k/eQ1vrl7B5hPHK1scgxrMBW8lCSFMwGRgCJAGrBNC\nzJVS7jg9R0r5OvB64fwxwJ+llJnFTnOVlPLUhcpSI9AzShnLrTg5DMqVjzeu5/WVy9ClRNN1pm5I\nZmjL1vxn6AgUI8vcoIIpD4uhN7BPSrlfSukDvgLGljJ/AjCjHK5b45B6HlDKTcJyWYXJYlB+7M/K\n5I1Vy/FqGn5dRwfcqsqilH3M27u7ssUzqIGUh2JoCKQWe51WeCwEIYQTGA58U+ywBBYLIdYLISKW\n6xRCTBJCJAshkk+ePFkOYlcvdNccZHo/wB9hhgnhGFORIhmUE3N27UTVtJDjLtXPl9u2VIJEBjWd\nio5KGgOsKLGNNEBKeUQIUQdYJITYJaVcWnKhlHIKMAWgZ8+e4TbZL1mkmgK5fwe8EWbYwTEGYW5e\nkWIZlBMu1Y8qw3+kC3y+Cpbm3Nh47Cifb9lEekE+g5o255bLOhFrs1e2WAYXSHkohiNA42KvGxUe\nC8ctlNhGklIeKfw/XQjxHYGtqRDFUJORrq+JaCmIOETsc2C/tkJlMig/rmrWghnbtuDyB/+N7SYz\nI1u1qSSpzs7HG9fzn1XL8agqEth4/BifbFrP3FvuICkqqrLFM7gAymMraR3QWgjRXASC6G8B5pac\nJISoBQwC5hQ7FiWEiDn9MzAU2FYOMl1a6CcIDk8thqUjwjHWKINdjenbqDG9GzbCYT7znGYzmagb\nHc2tnbpUomSRyXC5+PfKZbgLlQKAR1U55XLxxqpllSqbwYVzwYpBSqkCjwALgJ3A11LK7UKIB4QQ\nDxSbeh2wUEpZUOxYXWC5EGIzsBb4SUr584XKVN2R6n70vP+h5/4b6UsGS1/AEWamHawDKlo8g3JG\nCMHEzl2xmcxFoQVJUVFMGT2OGJutUmWLxLLDh7AoobcPTUoWpOwLs8KgOlEuPgYp5TxgXoljH5R4\n/SnwaYlj+4Gq+UhUSej5H0H+WwQsBA3png6W/qDEF1oOauFMMygxCOdNlSesQbmwPf0ED837AY+q\nFh07lpfHXXO+4dc778FqMlWidOExKZEtVJNhvVZ7jMznKoRUUwqVgpeAApAg3eBfAVGTwHE9iBgQ\n0WAfjUj4DqHEVrLUBhfK5HVr8BZTChB48s71eli8v2o+fQ9q2gxVD3WYWxSF0W2Mlq/VHaNWUhVC\nun8krC9BusHzA0rCl1DrXxUul8HFZeepk2Fz2Qv8fvZkZDCyNUgpWXskjcUHUrCZzIxp2462CYkV\nLutpYm12Xrp6MH/7dTGqpqFKidNsISkqiscv719pchmUD4ZiqEpIDxGdzNJdoaIYlA+/HzzAayuX\nkZKZQaLTyf09enNH565BwQItatfmUE52yNooi4VmcXFous4j839g2aFDuFU/ihB8vGk9D/fqw8O9\nLq/ItxPE9e070rVefb7atoX0ggIGNGnKmDbtsJmN20p1x/gLVgBSaqBngRJbavVTYb8K6Z4B0lVi\nxAb2kRdXSINyZ2HKXh77+Se8hclrx/LzeWnZbxzOyeZvA68qmvdQrz6sTkvFXWw7SQA2s5nhrVrz\nw55dLC1UChDYZtJUlcnr1jCkRSvaXKDlcCQ3l/eS17D88CFq2+3c1bUHPes3YMnBFAAGt2hFw5jw\nW5Ytasfz7BVXXtD1DaoeQkZIrKnK9OzZUyYnV496e3rBNMh/u9AaEOC4HhH7bFgFIaVEZj8M3hXA\naQvBBqYGiIRvjH4L1Yx+H3/I8fz8kOOKEKy790FMiiAtN5f60TGsSD3E339djKrraFLSpFYt3ht5\nLS1qx3PL7JmsPZoW9hota8fzvxGjaZeYdF4yHs7J5toZ03D5fUVJdmYh0CEo6uiPvfvyYK8+53UN\ng6qDEGK9lLLn2eYZFsNFRHfNhLzXOXOTB9zfImU2Iu6tkPlCCIj7H3jmIl0zA8rEPhLhvNVQCtUM\nj+oPqxQAdCl5aN5cNh0/hsVkwqdpjGjVhuV3T+Jwbg5RFgtNasUVzfdqatjzAKRkZXLD11/y6bgb\n6NWg0TnL+Z9VK8j3edGLHTutILzFynS8u241fRs3oWu9+ud8DYPqhxGVdJGQUgYsBUr6BjzgWYLU\nwpdVFsKEcFyHkvAVSuL3KNGTDKVQDbGaSn/mSj56BK+mke/z4dM0ft63l//32xLaJSTi8vtZeySN\n/MJyGKNat8Veyr69W1V5/tclZZIr1+vhaF4ueuHNf9nhg0FKIRJeTeMro25TjcGwGC4aPtAzww8J\nK6gpYKpXsSIZnDOHc7JxqyqtasdjCpPQFQlFCKIsFgr84UuZaCW2cL2ayo97drH+6BFOulyYFYFf\n13msd1/u6NKNWTu2cSgnG1+YYnsAezMzcPv9OCyWsOOnXC7+umg+q1IPY1IUYqxW/jHoahxmC9l4\nzvp+dCnJ8px9nsGlgWExXDSsgXyDcEg/mBqHHzOoEuzLzGD49E8ZPv0zbvz6S3pPfZ/551gC+6Fe\nl5dWJD0EVdc5nJuDW/WT5/PhUVXeWbuK3w8d4Lubb2Nip64Rz6cIgTmC4tKl5JbZX7Ey9TB+Xcej\nqpx0ufjLop/p37hJqdbIaZxmC4NbtDyHd2NQnTEUw0VCCAFR9xBaysIC1q4Ic5PKEMugDLj8fm6e\nPZO9GRl4VJUCv58sj4e/LPqZTcePlfk893XvWaab7mnChYG4VZX31q3BabHwzBWDwkYHWRSFwS1a\nYomQIb0i9RAnCvJR9eBNI4+qciA7i+71GuA0W1AQ2E0mBAQpGavJRMPYWK41EtdqDMZWUhmReh4y\n/3/g+QGkDvahiOjHEKbIoYIi6n6kngWuGSAsAUvB2hsR92YFSm5QHJffz5ojqSgI+jRqhN0cuvUy\nb+9uvJoacqP2qirvJ6/hw9HjynQtj6qG3IwjYVEUtMImPSVJy83hrjnfsCYtDauiYDWZMCsKbr8f\np8VCnahoXrxqcMRzp2Rm4o8gx4GsLNbd9yDrjh5h3dEj1LbbGdi0GV9v38r3u3cCMK5tByb16GXk\nJ9QgjL90GZDSh8y4CbRUispfu78J9FhOnBexLIUQCiL2WWT0o6AdAKUOophfQUoNvEuR/k0IJQkc\noxBK7Qp4RzWT73ft4LlfFhX5CnQp+c+QEQxr1Tpo3sHs7JAS2BB4ok/JDPYbrU5L5aVlv7E74xQx\nVht3denGg736YFYU3li5DC3CDdkkRFANpKubt+CXA/uDchlOk+fzsezQQSTg1cCqKCQ5nVzfviMd\nEutwZbPmpfo/msbFYVGUsP6JpnFxCCHo3bARvRueiWp6vO8AHu976RZoTMvNYfrWzezPyqRbvQbc\n3LETtR3hClXWTAzFUBY8P4N+nOCeCCrouUjXTET0faUuF0oMKJ2Djkk9H5k5IaBspAuJHfJfh7gp\nCJsRL17e7DyZzrO/LAoqVAfw54XzmJc4kWZxZxRy64SEsI5jRQjaJ53JF1idlsof5n5bdM4sj5v3\n168lJSuTt4aP4rtdOyNG/Axt2Zr/Dh1BhttFgsOJ1WRi7Mzp7Mk4FXQDNwkRyG8pttan65x0uejV\noBGXNzq7r2pgk2bE2R14VDXI6e0wm3m0d9+zrr/UWJF6iEk/fI+q6/h1nWWHD/Hh+nV8M34CLWrH\nV7Z4VQLDx1AGpHdZmGxkAA94fz+/c+a9Aer+Yuf1gHQjsx8i0DrboDz5fMsm/GGemDVdZ8bW4DDM\n4S1bE221hlQJtZlMPNTzjNJ+ZfnvIYrGo6osSNnL4VIiiE434LGZzTSIicVmNiOEYNp1NzGyVRus\nJhMmIWgeV5s4uyOscvGoKhuOHS3TezcpCjNvvJnOdethM5mIsliIsdp4ftDVXNmsZnX903Sdx37+\nCbeqFm2veVSVXK+HZ5YsrGTpqg6GxVAWlEQCv6qSZr6AUnwMpeKZQ/iubBJ8q8E28PzOawBAvs/H\n7oyTJDicNIurzZHc3JAQUQC/rnMkLzfomM1sZvb4W/nLgvlsPH4URQjqREXz0tVDaJ9Up2jerlPh\ne4+bFRNbT5ygf+Mm/Hpwf4ivQiLDPunH2mz8d9hI/j1kOD5Nw2mxcN3M6WS4Qx9K7CYziU5nGX4T\nARrExPLN+Fs5lpdHrs9Li7jaEZ3V5U2eN9CStir0lth+Mj2kki1Q1IGutJDfmoShGMqAcN6EdE0n\nVDHYEM7bz+lcUkqkawYE9SsqOckomHe+SCl5a80q/m/DOiyKgl/XaZOQSN+GjVl9JDXEGewwW8Le\npBvGxPLVjTeT4/Hg1VSSnFEhXfJq2e2ccoXetD2qn+SjR3isT1/WHU3DXcwJ7TBbeLBnLxJKuamb\nFaUoKui+7j15YtGCojpJpxECRpxH28/6MTHUJ+ac150P+zIzeGrxAramnwCgY1IdXh08rFKrwp6t\nAFD1KxB0cTC2ksqAMLeA2OcBG+AE4QCsEP0ownrWsiNByIKPIe+1UiYEIpcMzo+Z27cydcM6PKpa\nlAuwPf0EX27bHDZCyGpSGNeuQ8Tz1bLbqRMVHbZ16t1duge14zyNJiUzt2/h7rnfMnXMdUy4rDOt\n4uPp37gJk0eO4ZFz2Ncf0aoNd3bpitVkItpiJdpqJdZq46Nrr68ST+CRyHS7uHHWDDYdP4aq66i6\nzpYTxxk/awYnXaU8FF1kLkuqEzYrXQBd6tbDaVgLQDlZDEKI4cDbgAmYKqV8tcT4lQR6PR8oPPSt\nlPKFsqytKijOG5D2awI+BamB7QqE6dwKl0npg4LJhJbJOI0Doh80IpOAo3m5vLp8Kb8c3I8iBKNa\nt+XJflecNXLk/eS1IZE9mpTk+cL7bRxmC1HneTOY1KMX+7IyC8Nbg/0JHk3D5/Hw1pqVTL9+/Hmd\nHwL5ME/2H8hdXbuz9kgaUVYr/Rs3rZJd3Yrz1bateFUt6AlcAj5NY/qWzfzp8n7ndL4st5tPNm1g\n8YEUYq027ujclZGt25xzr3OTovDmsJE8+NMc/IUKy2YyYTOZeeWaoed0rkuZC1YMQggTMBkYAqQB\n64QQc6WUO0pMXSalHH2ea6sEQokDx9jzP4F2PJADERYFYp9GcU44//NfImS53Yz9ahpZHk9RTZ9v\ndm5nVdphfr7tzrC5B6c5URC+cF0kTroK8Kjqee0rmxSF/wwdwYhWrXl0/o8hykEvbK7jVdULzgGo\nExVdrTqjbT5xLGzxP6+msflE2ZMEIVDOY8yMz8nyeIoc+tvST7Ai9RAvl7iZn/68KKUojIFNm/HT\nrRP5YsumonDVWzt1OSefzaVOeVgMvYF9hf2bEUJ8BYwFynJzv5C11Q8lnlA/xWnMCPvwipSmyjJt\n6ybyfb6iLzkEykWccrn4ae8ebmjfMey6I3m5NIiJ4WB2aNObSFhN5qKbdr7Px+L9KeT5vPRr1JiW\n8QllOkeCw4lFMYUohtPo1bC0/YXSKj6B3w4exK8H/04sikLLcwwJfT95DZlud1CSnkv18/3unUzs\n3A2frpHlcfPF5k38fugAEujXuAn/vPIamseFt76bxdXm78V6YhgEUx6KoSGQWux1GhAuEL+fEGIL\ncAT4q5Ry+zmsRQgxCZgE0KRJ9SwnIZRopH0oeBYR6Ot8GivYrjS2kApZcfhQ2Jusy+9nReqhEMVw\nPD+Ph+b9wM6T6RCmmpCtMFO4ZBy/3Wzm5o6dUIRg2aGDPPjTXIQATQ/kDYxu3YbXhgwv9ekT4LI6\ndVGU8HM6161XI6Ncbu3UhU83bcBfwkA2KwrjO3YiLTeHJGdUmSyphSn7wmZu+1SV62d9iYLAVcI5\nv+LwIa6fOZ1Fd/zBsATOg4pyPm8AmkgpOwP/A74/1xNIKadIKXtKKXsmJZ1fU5KqgIj9F1j7ADYQ\nMYH/rT0Rtaqka6VSqBcdE7ZYnEVRqB8dHFGzPyuTYdM+ZdPxY3g1LWj7wiQENpOJa9u248cJd9Aq\nPgGnxUK01YrNZGJgk2Y81f8Kcr0eHvhpDi7VT4Hfj0dT8Woq8/btYdaObWeV12Iy8caQ4djN5qLc\nB6tiItpq5aWrh1zQ76K60jAmlqljriPJGUWUJeDHSXA4uaJJU8bNnM7waZ/Rfcp7vLZiacTs8G3p\nJ3h0/g+kR9ge1AnkIJRUChDwZ3hUlS+3bg46fjA7izm7d7Ii9VDE6xqUj8VwBCge79eo8FgRUsrc\nYj/PE0K8J4RILMvaSw2hOBHxU5Hq4UCZDFMzhLlpZYtVpbizSzcW7t8XkjxmUhTGd+hU9Hr54UPc\nO/dbfBG+4K1qxzNnwh1Fjtp5t05ka/oJjuXn0S4hiaZxgWY4p2sClcStqny2aQM3d+wUdrw4g1u0\nYs7Nt/PJpvWBwnT1GzCxczfqRtfcXhp9Gzdh1T33s+vUSaSUfLVtC9/s2hH0d/1880YAnuofnLez\nZH8Kj/78Iz5NO++tOK+msa6w852q6zy+YB6L9u8rDAUWRFstfHHdTbQq45ZhTaI8FMM6oLUQojmB\nm/otwK3FJwgh6gEnpJRSCNGbgKWSAWSfbe2lijA3AaPCali61W/A0/0H8sry3zErpsLtHZ3Xhwwv\nupnrUvLnBfMiKgWAYwX5QdE7Qgg6161H57rBfTByvZ6Ixe6yvWXvQdA6ISHEGVrTUYSgQ1Id8rxe\nZu/cHrJF6FZVPt+8iT/16Ve0raRLydNLFoY8GJzGoiioun7WnAOTEEWd8D5IXsPiAymFVmVABpff\nxx3fzWbFHyaddbuwpnHBikFKqQohHgEWEAg5/VhKuV0I8UDh+AfAjcCDQgiVQKzmLTLQbDrs2guV\nqSKR0g/+TYFoI2u3sL2cDSLz455dvLVmJUfz8mhSqxZ/uXwAQ1q2YmKXboxt254VqYcwKQoDGjcl\nynrmd7v71Ek8YbYQilNWJ2fvho0xFybDFcckBFc0May58uBofh4WU2QH/UlXAY1iawFwICszJKHv\nNAqCe7r14ERBAT/s2VVq9VqLycTELt0A+HzzphBFIwkEHKxJS6VvY+MhrTjlkscgpZwHzCtx7INi\nP78LvFvWtVUZ6d8O/q2g1EWiQ85TUFTNRiJjX0FxnIkukloGMv8t8CwABNhHI2L+iFBqVYb4VYpP\nNm3gjZXLivIO9mRk8NiCn3jhymu4scNl1LLbGdm6bcg6l9/P0sMH8Wul7xFruuRIXm7YHgbF6VK3\nHn0bNWFl2uGim4eCIMpirZFF5i4G9aOjw9aqgkCJkOIOYrvZgqaHtwfMisI93XpS4PexIGVviGIQ\ngNNiBSSvFcuyzvV5Q09WePVwJUdqOkZJjDIipQeZdT/4NhYeUYAwH6icJ5HmFghLm0AF1YzrQD9F\nUZiq+yukbykk/oAQ9gqSvurhVVX+u2p5SDKaR1V5ZflSrmvXgb2ZGXyyaUNhM5n63NmlO4dysrnv\nh++QUuLTw99oTrP95AnGfTWdXyb+4axZwu+PupZPNm1g+tbNFPh9DGjclMf79i96ijW4MGJtdsa0\nacePe3cHPbnbzWYmXNY5KDelYWwszeLi2JNxKmi7SBGCDnXqkOB0koCT6deP5+nFC9iXmYEQgp71\nG3Jrp87EO5z0qN8gKOKpY1IdNoZpsqTqOl3r1b8o77k6YyiGMiLz/gu+DQSHmYbDj3RNQ9R6Aeme\nDXo2wbkLftBPgvtHcN548QSu4hzMySZcaCmAW/Uzc/sW/rXsd/yahiYlW04cZ/rWLehSD9uzIByB\njGcv3+3aUbSlEAmLycSkHr2Y1KPXub4VgzLy4lWDEUIwd/dOzIoJVde4sX1HnhkwKGTu/0aMZvzs\nr/CqGm410JDIYbbw1rCRQODB4rPNgYcGh9mMT9NIcDoY3KJl2ATIZwYMYuL3s0OU0ohWbQzlHwYh\nq2HyTc+ePWVycvJFvYbUM8G/LZCUZu6ITO9eeuG74lj7ocR/ip55N/hWhJ9jH4kS91b5CVzNSC/I\nZ9CnU8PuOVtNJmyKiTx/aBkLRYiwUSoKgkjuyG716vPN+BoR01AtyPV6OJafT4PomFItuQKfjx/3\n7mZ/ViZtEhIZ1bpN0U3/2SUL+X7XTjzFwpNtJhPDW7XhzULlUZL1x47w6vKlbEtPp5bdxt1dunNv\n956lNjm61BBCrJdSnrXAm2ExlEBKicx7FVxfBtpxooOSVHalgA0sPQI/KnUIbDmV3As3FZbyrrnU\niYqme/0GrDt6JGif2KIodK1bjx0RSlpHCl20mJSIjs1dp06iS2lEnlQRYm12Ym1n30aNslrDhgoX\n+Hx8t2tHyN/bq2nM37eHfwy6GlXXWZl2GJvJxBVNmuGwWOhRvyGzbjJKzpSFmqMqy4h0zQD3V4AX\nZH6gkY52mLLpUAHChnAGnk4D/4eLUrIgHDeXn9DVlLeHj6Zl7XicFkvRVkH7pDo82rsv52LJKkJw\nWZ06Ecc1XedkQeVV9DQoXzLcrohP+RbFxP/WrmLAJ1N4dslC/rLwZ3pNfZ8lB1IqWMrqjWExlMQ1\nNUw/BEngVyUIbq5jD1gT+tHAHOvliNh/IEyBhBlh7YKMeRzy3uDMr1qD2L8jLMF9hmsiiU4n826d\nyMbjxziYnUXL2vG0SUjArJhwmENba5qEIM5uD2Qnn44eEgKnxcJf+17BHd/NQg2rUATRViOM+FKh\nblQ0kZ4bfJrKl9u2BOUrADw6/0d+mfgH6kVXTC+K6o6hGEqiZYQ/LhRw3gP+DYW+h0SIug/huIHT\nW0WBYrHBKFF3IR3XFrYAVcA2KFCl1QAIJJ11r9+AI7m5PDTvB04U5GM3m7m6WQuWHNiPJnV8mobD\nbMZpsTDrpgksOZDCF5s3kefz0rdxEx6/vD8tasfTuV59Nh07GrRxZ1EUrm7eIigHwqB6YzObubd7\nD6ZuSA4KRHCYzSQ6o0jNzQlZo0vJtzt38FCvQCk2j+pnW3o6URYL7RKTzrl896WOoRhKYmkbSFgr\nidQR9uGImMfCLCq9Nr5Q4sFxXfnIdwny455dPLVkQZEV4PL7WXwghYFNmtIxqS4Hc7LpVq8+49p1\nINpq5Z5uPbmn2xn/mVdVWZF6iLu6dONNt5v0gnx0KRFC0LJ2vFFn/xLksT79sCgKUzYk49M0LIrC\nH7r1YMmB/WHn+zStqEHQjG1beGnZb0WBDPEOBx+OHkf7xOpbg628MRRDCUTMX5GZ9wLFSyHYwNoL\nYQlNtjK4cP69cllIVqpHVfn90EGeH3QN9WPOmP9LDqTw6vKlHMjOIs5u54rGTVm0fx9K4Z6z1CUP\n9OxNUlQUbRIS6VK3nvE0eAmiCMEjvfvyQM8+5Hg8xNpsgcxqVWVvxqmQLPYoi4XeDRuxMvUw/1r6\na5Cl4fL7uX7mdJwWK3k+L+0Tk3h2wCD6FLZ8LfD5mLt7J1vST9A8rjY3tO9YamvWSwHD+VwCYe2N\nqP0+mFoRcCY7wHkLovZ7lS3aJYmm66Tl5oYds5pM7M44VfR6Uco+Hp3/IylZmehSkul2M2fPLlyq\nSr7PR777x6qhAAAgAElEQVTPR4HqZ3LyGno2aEjXevUNpXCJY1YUEpxOLIU1sf7QrQdOizUoAs1q\nMtEgJpbBzVvyfvKasHkwXi3Q00HVdbamn+Duud+y9kgaR3JzufKzj3hp+e/M3L6Vt1av5MrPprLh\n2NEKe4+VgaEYwiBs/VGS5iHq7kDU2YQS+xxCVN3+utWVfJ+PO7//JuK4qutB1sJLy3+PWFit5Lrp\nJcotG9QM6kRFM/eW2xnesjVOi4VaNhu3dOzErJsmYDGZSM0J9T+EI5CB/ztPLV5AlseNqzAQwqOp\nFPj9PDzvh3OKnKtuGFtJpRDOmWxQfjy9eAHJx8JXWTcLQav4hKJaN6quk5pTts5sgblluwEYXHo0\nrlWLd0eOCTvWqW5d0vJyy1TKe8fJdHQpw87N83nZeeokHZIih0lXZwyLoZyQeg56/gfoGbeiZ/0R\n6Vt3ZkxNRc9+Ej29P/rJYegFXyBl6XV+LnVyvR4WH0gp6uFbknaJSUy99ozD3iREmfsm281mejds\nVC5yGlxaPNzrcmymsj3wxVptQR3/iqPpesSigJcChmIoB6R2EnlqJORPBn8yeBcgM+9Fz5+CVFOR\nGePAMzdQI0k7AHlvILMfr2yxKwVV11mUso//ropQKoRA2OGbw0aS5IwqOvbxpg1l+iKahCDKYmF8\nx8vKRV6DS4t2iUl8Ou4G2sQnYBKisMufOaRql91sZlSbthGqeQWinC5VawGMraRyQea/BXomcPrG\nJQE35L+D9G8rLKdRPErCDd5fkf5dCEu7Cpe3ssh0u7hx1gxOFhSEJK8VRxEiqLBZtsfNGyuXhX16\nc1osdKlbj7VHAp26BjVtzj+vvKZMJRcMaia9GjTi59vvosDnw2IykZabw8TvZ5PjCRTIVHWNUa3a\n0KN+A2bt2BbWWS2gyOF9KWIohvLAs5AzSqEYwlxYRC9c3wAdfGugBimGv/+6hLTc3FKbqzjM5oC5\nbzbj9vv5ce9uft63J+J8TdeZfv34IkegEYVkUFZOJz22qB3P0rvuI/noEU65XHSpW4+GsbEhZb+L\n0zSudsUJWgkYiqE8EGbCfoKkAGEH8sIMmkEpvYHMpYSq6yzev69UpVA3KopHe/dlwmWdOZSdzY2z\nvsStqkURIeE4/dRmKASDC0ERIsQvFciDqc/GY0eDen84zGb+0ncAPk1jQcpeVqelUi8qmhs6dKTB\nWZpCVRfKRTEIIYYDbxNIAZ4qpXy1xPhtwFMELLA84EEp5ebCsYOFxzRALUtJ2CqHfQy4phNcRwlA\nhai7IP9dAh1NiyEk2IZUjHxVAE3XIzryoq1Wpoway+XF2iv+eeE8sjyeUqNHLIrCmDY1x+IyqHim\njhnHc78s4ueUvQgC/qsn+19B/8ZNGDH9M9IL8inw+7EqJt5fv5Z3ho9icItWIefZm5HBrB1byfZ4\nuLJZC4a0aFmlt6IuWDGIQEznZGAIkAasE0LMlVLuKDbtADBISpklhBgBTAH6FBu/Skp5ikpCSh8y\n/31wzQhUVLV0QcQ+hbB0LtN6Ef0o0rsM9GOBaqyYA/9iX0Q4Rgf8DN5fCWwpmUFIRNxkhBJ9Ed9V\n1cJmNtMuITFsOW1Nl3Qp1kUrw+ViR3p6qUrBabFQNyqaJ/tdcVHkNTCAwHbTW8NH4fL7yfN6SXQ6\nMSkKz/2ykLTcnKIMa5+ugQ5/WjCPdfc+iMNyplnQ55s38uqKpUVNp+bt20OL2vHMvOHmoHlVifKw\nGHoD+6SU+wGEEF8BY4EixSClXFls/mqgSsUSyuxHwbuKojIY/nXIjDsg4UuEpeNZ1wslBhLngGcB\n0rcclCSE40aEuVlgvPbbSP+ugE9BiQHb0BqhFFx+P2uOpKIg6NOoES9cNZg7vpuFV9OKbvoOs5ln\nBgzEYbHg9vuZv28P29KPR2y6Y1YUhrVsxdCWrRnWsjXWKvzUZXDpcLo0/Gl+2LM7pOwGBLakvtmx\nnaWpB1l++BAWRcHl9wdZyy6/n70Zp/hoYzKPVNGe4uWhGBoCqcVepxFsDZTkHmB+sdcSWCyE0IAP\npZRTwi0SQkwCJgE0adIk3JTzQvp3ByuFItzIvP8i4j8qfb2UoO4JWAr2oSiO8Ik1wtKuRjmav9u1\ng7/9sqiobr4uJf8dOoLvb76dyetWs/nEcRrHxnJ/z970b9yUlMwMxs+eiVcL+BQieQzi7HbeHj66\nxjXd0TQNr8uHI9pu+FOqAFoEX5mmS15e8TteVUUSelc5jVfTmL1z+yWtGMqMEOIqAophQLHDA6SU\nR4QQdYBFQohdUsqlJdcWKowpEGjtWW5ChaukWpYxAkpFZj8E+ikC7hWJjHkWxXlTuYlXHdl5Mp3n\nflkUUr7iTwvmMf/WO3lr+KiQNQ/P/4Fsj7vITij5B1aEwGoy8co1Q2uUUtA0jWkvzObbt37C6/ER\nXcvJxH+MZ8yDwxBC4HV7ycsqoHadWpjMhvVUUQxq2pyF+/eFbHd6VT+KEBGjmYpTWiBGZVMeiuEI\n0LjY60aFx4IQQnQGpgIjpJRFTQ+klEcK/08XQnxHYGsqRDFcNJQEEKbwUUVK5JA0qbuQmbeDLFF6\nIfdFpKkJwlaa0XRp8/mWTWGT0VRdZ8a2LTw9YGDQ8cM52RzOyQn7JxBAq/gE2iUmcn+P3pd0UlE4\n3v/zp/z88a94XYEY+5xTefzfk9PweXyk7jrK4mlLEUJgsVmY+M/xjHtkhGFRVADPXjGI1UdScfn9\n+DQNQSApzmmxkuF2nXW91WRidOuqW625PBTDOqC1EKI5AYVwCxDUeV0I0QT4FrhDSrmn2PEoQJFS\n5hX+PBR4oRxkKju2gQTab5Zs/egA5x8ir/P+DDJcGKUHWfBBjVYMabk5YSOQVF0nLS+0hpHL78cU\n4WZmNZn44robqRNVfX0y7nw3O1btweaw0r5vG0xl9IvkZeUzb+oS/J7gz5nH5WXqM19iMin4Cse8\nbh8fPfMlFpuF0ZMiR7ul7TnKFy/MYvNvO6iVGMONj49h8B0DDWVyjjSKrcWC2+/i800bWZ56iHrR\n0dzdtQdvrVlJRlrpisFuNpPodPJAz94VJO25c8GKQUqpCiEeARYQ2E/5WEq5XQjxQOH4B8D/AxKA\n9wo/gKfDUusC3xUeMwNfSil/vlCZzgUhrBD/CTLrHpCeQsvBD46xCOctkRdqR4AIHwAtNfzxGsLl\njRqTfPQoXi14K8lhttC3YeOQ+a3iEwKhe2HyFepERQeVxqhu/PDBAj786xeYzApIsNgt/OPbJ7is\n/9n9TWl7jmG1WUIUA4Dm19D8wVaZ1+Xli398HVExHNqRyqN9n8Vb4EXXJRlHM3nn4f9j19q9PPru\nvef3BmswSc4o/tJvAH8ptjN+T7cebD5+LCRb2qoo9G/SlHyfjyEtWnFzx07E2KpuxWZRHUvH9uzZ\nUyYnJ5frOaVUA1FDejZYuyFMDUqf71mIzHmqsNxFcRSwDUep/Va5yledyHK7GfLFJ2QV8xlAIF9h\n+V33EWsPLVfx057dPLH45yKn3WmfwpTR4xjQpGmFyX4+uPPdLPzsNzYu2UZSowRGPzCEph0as/n3\n7Tw36pWibaDTOKLtTDv4HrHxpfcfPnUkgztbP1pkFZQFoQh+ck3HYg0Ng/z7ta+y5qf1If2SLXYL\nn+x8m7pNjQ5m5cHbq1fywfq1mBVTIIhCwHsjr60Sn2MhxPqy5IoZiuE8kVJFnhoG2jGg2NOBcCDi\nv67x3d6+37WDvy76Ocg5ZzeZGN2mHS9eNZipG5OZtWMbfk1neKvWPNSzD4dysnk/eQ0pWZm0T0zi\n4V6XV3mfQtaJbB7q9RR5mQV4XV4Uk4LFZuZPH97Pb1+tYM1PG0LWCEUQmxBDv7G9mPDMddRvXjfi\n+Z8d9TKbftmK33v2PhQAMfHRfHPy47BbQ9fG3oE7PzROxhFt59F372XIxEFluobB2TnpKmB1WipO\ns4UBTZqWuTLwxaasiqHGVVeVUkdqp5AyUiBZ2RDCjIifCbZBFCW0mVojak+t8UoB4P82JIdEbHg0\njbl7dnHjrBlMXruGwzk5HMvPY9qWTYz56gta1o7ntcHDuL1TV5KcUezJOIW3WB/olamH2XDsaJlq\n6VcUU578gqzjOUVWga7peF0+3rr/Q47tPxF2jdQlOSdzWfDprzzQ7QkO7UwLGj+ZlsGx/SeQUvLc\nl4/ReWAHzNaz31jsThvjnxgb0V9gjw5fWFAogqhal3aryoomyRnFmDbtuKZFyyqjFM6FGmUx6K5v\nIO+NQHYzEhyjEDHPI5TQL4WUPqRrNri/AfQin4MQoV8uKb0g/TUiaa2stPrff8PewO1mM7qUIX0Y\nbCYT17XvwA+7d6FJiUdVCztw2ZnYuSvvrF2FSVGQUuK0WPhw9Di6FsuWriwiPYU7Yx20v7w1m37Z\njqaWXi68fZ/WvLPqZQ5uT+XlW9/iyN5jRVbFXz9+mO7XdOJv177Cmh9DrQ8Ai82CogjGPjqCe16+\ntaj/dUk+/+fXzPz3HHxuX9DxqFpOvj4+FautambhGpQfZbUYqp8qO0909zzI/SdBKSfueUjtJCL+\n46JDUnch894A95cEVUXNS0G6v4eErwMO62IIYYNirT+lZz4y703Q0kCpA9EPIRw31ajIj1irjWxv\n6A1T1fWw8dteTWP29m2oJTJEParK66uWBymZAr+fO76bxYo/TKr08tqRHqz8Xj8WqwWLzXxWxbBz\nzV62LNvB8+P+TUF2QZEP4KQrg/839jUmr3uVA1sOR1x/+Zge/PWjh3DGOEq9zi1PX8e25bvYuXoP\nql/FbLUgBLw492lDKRgEUWMUA/lvEpqH6AVfMlJNQZhbIqVEZt0N/q2Elsr2BJrsuH8E5/URL6O7\nZkPuC2eupR+F3JeQ+ilE9EPl936qOBO7dGPKhnVBSW6CQGmBAp8vbDhr2OLkkTpoScnc3bu4vXPX\ncpL4/Og3rje/z1yBpgZL7/eqrJ23AcViIqlRAqeOZIQ4fYsz+Y8f4/eqIXNUn59/3vA66YfDlxJT\nTApWmwWfx3dWxWC1WXht4d/ZuWYv21fsJi4plgHX98YRXfo6g3MjLTeHl5b9xm8HD6AIhRGtWvPM\ngEEkOKvPdl2N2UrSj7cnfM+EaEStVxD2YUjfOmTWfYWF8CJgbodImB1iNQBIqSHT+4HMCrPQgai7\nGiFqxpfQr2k8vnAei/enYFFMSCS1HQ5euWYo9/3wfUhWtNVkQhEi5Hhp3N+9F0+VSJaraE4dzeTh\nXk9TkF2At8QWzWksdgt/nnI/b/zhPXQ1fLarI9oedkuqLFjsFkyKwvPfPkHPoV1Cxo8fTGf6v2az\nftEWouKcXNavHSmbD5JxNIuoWk6ad2rCsLuuots1nUKsWiklq+YmM/+jJXhcXq4c348hEwdhtYd+\n/g0CzaiGfPEJuV5v0cOPWVGoGxXNojvuwm4OtczyfT7m7N7JpmNHaV47nps6XEZS1MUJ0Taikkqg\np18BehhnoHAg4mcgLB2QBR8j8/5DaPns4phAiUfEzwCUgBIxt0AIM1I7gTw5GPCGLhPRiPhpCEuH\nc5K7unMoO5tt6SdIioqiZ4OGKEIwc9tWnv99CX5NKwpnFYRPPo9ElMXCa4OHM7J1m4sg9bmRl5XP\nN2/+yLypS8g6nn1e52jUtgGnUjPwuMJ8dsqII8bOrONTsTnObGse23+CB3s+iTvPg65FLsEghMDm\ntDL6waFMeOo6YhNikFLy+t2TWfbNajwFAbnsUTYatq7P2yv+hc1hQ1M1Nv26jZyTubTv26bUCKua\nwDtrVvF+8hq8JXxoTouF5wddzU0dglvOHsnNZdzM6bj8ftyqH5vJhElR+Pja6y9K33LDx1CSqAch\n7zWC+yJYwNTizM1aqQPCGiGj+TQa6KeQpwrr/QgTYEHG/gNhu4qItzfpBSX+gt9GdaNpXBxN4+KC\njt18WSe+2LKRXRmnivboIykFh9kcKEZWzJIwKwoJTidDWrS8SFKHJy8rnznvzmf5d2txxjq49sFh\nCEUw+Y8f48pz43OXPd+gJLf//UbeffQjhNtb6pZTaQghWL9wC/3G9gIgP7uAF2/+LwXZZy/RIKXE\nU+Dlu7fm8fvMldz5ws1M++csjh8MLpPuKfCStuco8z/6hc4DO/DU0BfxuX1IJJpfY9DN/fjL1AfL\nnN19qbEi9VCIUoCAv+ybHdsZ17Z9UB+GZ35ZSJbHXbRl6tU00DQemf8Dq+95oNLqgtUYxSCcE5D6\nSSj4qLDjmj/Qd6H2O2cm2a+B3H+U4WySIqvg9Jc45xmI/xjsw8DzI6G3OhX0k2Cqd6FvpdqTkpnB\ngeyss4admoTgv0NH0jA2lud/W8LmE8cxCcHQlq34x6BrKrTRSW5mHg92f5Ls9JyihLNda/eh+lSk\nfoFWt4DLR/egVddmvHLbOxzefQRFCGrVqUVeZj7uPPfZz8Hpm3tgOyptz1Ee6/8cuRn55ySKpmpk\nHsvirUkfovrDO829Lh+LPv+d6S/OJvtkbtDY0lmraNmlGTf8afQ5XfdSoUFMTETrd+Pxo9wwa0ZR\nHwavqrIqNTXs98Dt97Mt/QSd61bO/aLmKAYhEDGPIaPuBW0/KIkIU/0ScxwQ/2mhn8EDUiek81pE\nPMj8DyD6AfD8ROhHQyIzbkU6JyCiH0DUQOvhNEfz8zBHCKksjtVkItvrYVid1nwz/lb8moZJUSrl\nKWrWG3PJOpGD33vGKghXquJ8qN+8DlGxTqI6OPlg4+ucOpKB36dSr1kd7mr7xzIrBtWn0fXqwFbF\n63dPJi+zZFZ+2SjpSA+Hp8ATNiPb6/Lx7Vs/1VjFcFeX7ixM2RdSEgPAr+tBfRgCCiH8Q4UQolKr\nr9a4BDehRCEsnUKUQtG45TJE0nJE3AdQ6w0Q55B5qx1AqPsIFOULhxdc05GnrkXq4RzUNYO2CYkh\neQyRqFvMCWcpdFBXBsu+WROkFErlHEV8/P8eDHqd2DCB+s3rIoRgzANDsTrCh5KaLGcsJnuUjfFP\nXkt8vdrkZxewJzklYijthWJ32mh/eVv0CDeuvMxzs1IuJbrUq89zV1yJWYS/tZ7uwwDgsFi4rE54\nn4xARByrCGqcYigLQpgQtj4ojiGIWi8Cds7+bRdgbgdKUqHfIRJ+0LORBV+Un8DVjDpR0Yxu0w77\nWTJCParKQz/N5c3VKy7aTa6sWO1lj/Mva18Ek0Xh1mevp+tVl0WcM+7REXQZdBn2KBuKIrA5rFgd\nVv704SRGTRpC43YN6XpVR5798k/c9c9A0Udd1+EiKVB7lI2OA9px459HoUfYQmvbO7TncU3i1k5d\n+HPf/hGtYq3Y7+3la4YSZbFiKZxrEgK72cxr1wyt1O6ENWYr6XwR9qsg/gtk/mRQd4O5Gfh3F/Zh\nKP7Ua0NEPwDmtoCN0DLexfGBdwnE/PFiil5lWHIghTdXreBgTjYNY2J5rE8/XrlmKPWio/ls00Zc\nqp9oixW334+OLArzkwTKaLy3bg17MzPomFSHWjY7o1q3pbaj4sJ+Vb+KK7+MW4oicPN05bmRWvgb\nZ+eBHWjQsi4j7htMh8tLj6oyW8y89NMzbF+5m82/bicqzsmg8f2oXadWxDWx8TE0bteQA1sOlU3m\nomuZ0HUZMXoprk4t/vrxQ/Qa3hVFUegzsjtr520ICtO1Oa3c+8pt53TdS5GRrdrwzpqVlNxQsppM\njG5zpmRO+8QkFtx+J59s2sCm48doHlebu7v1oH1i5RY0rDHhquWJ1I4jc54E33pAASUBUetFhC3Q\nmF76dyOz7gU9k4ihr5beKAnTKkzmyuLbndv526+Lg6KKHGYzT/a/gju7dEdKyaL9+/hw/ToOZGWF\nzZYujt1sRgCTR17Llc2aX2TpAw7dJ4e8wKZftpVpflLjBJ6Z/hhPDX0xrA+ix5DOvLrg7+UtZgi7\nk1N44up/4PepqD4VxaRgsphQFIHXFZpvoZgD1sv8qUvIOBp+m/O2v93AXS+cKUWv+lVm/nsOcyf/\nTH52Ae36tOa+126nXe/WF+19VSf+vWIpn23eWORvsJvNJDmjmHPLbcTZKyefyQhXvYgIUz1E/OdI\nPSfgpFbqBCUGCUtbSPod6ZkPOU9AyHODAxF16T9VabrOy8t+D0lac6sqb6xcwS0dO/PNzu28tOy3\nsM66cJw+18Pz5rLm3geJtl7cRKu18zey+dftZZ7/3Iw/07FfW+599TamPj0dza+hazoms0LH/u14\nad6zF1HaAKpfZenXK9F1iepTsdjMXDagPZNev4PHB/6/sGvi68Ux8fnxrJqTHFYxWB1WaiXGBkpp\nWAK3DbPFzG3P3cBtz91wUd9PdeXJ/gPp17gp07duIsvjYXDzltxyWeeL/pktDwzFcAEIpRYQ3qQX\nQkE4RqFLT2EIrCCw9WQCxxiwDa84QSuJ4wX5uNTwFpPL72NRyj5eWb60zEqhJEsOpDC2bfuQ4+kF\n+RzOyaFpXNwFN/n58YOFZfZvWGxmmndqAsD1fxxFn5Hd+fWrFXhdXvqM6kHHfm0rpF7Wa3e+y6o5\n64q2ePxelR2rdnMs5QT3vnobU56cFtQjwuaw8sg79yCEYOwjw5n82CchPST8Hh8f/OUzpjzxOf3G\n9eaPk++lVmLsRX8v1Z0BTZpWiT4M54qhGC4yivMGpG0geBcGrAvbIIS5ZjjnYqy2IEdbcSTw3K+L\nON9sLreqsmR/sGLI9Xh4eN4PrDuahs1sxqtpDGvZin8PHn7epY8zi2UyW2Kg3fUaFotO+godfx6Y\nLHD8sBWLzcaIe64JqlfUsFV9bv/bjed13fMl/fBJVny/NmQby+vyMfXpaXy2912SGicy7cXZnDiY\nTpP2jbjzhZvpMqgjAMPuvootS3ewdPZqpC4RCvjcfqQEqenoGqz8fi37Nx9k6vY3a2wi26VOuSgG\nIcRw4G0CrT2nSilfLTEuCsdHEuiHeZeUckNZ1l4KCFMSOC/9raOSxNpsDGzajCUHUsKOa1Li18sW\nthqOhSl7OZaXR/2YGL7duZ2nFy8oqs7q8/kK56TgMC/h1cHDgtZKKdmTmUGOx0PHpDpEFZr3u06d\n5N21q9l0/BhWswnHdY3Iii6g7/h8Xh61GpOQ2ExnnLOaCn6fwo7N/ek+ZuJ5v5fyYv+Ww1is4duB\nHk0J9HjoO6YnfceE32ZWFIWnPnuUm58cx8YlW1ny5TL2JqcERSCpfo2MY1ms+WkD/a7tddHeS00g\nx+Nh8ro1/Lh3F4oQXNe2A/f37B2y3ZTtcSMlFRZ0ccGKQQhhAiYDQ4A0YJ0QYq6UckexaSOA1oX/\n+gDvA33KuNagGvP6kGH0nPJe2MqpUkoSnU5OFBQEZX+ahIKUetg1xdGkZMa2zezJyGDh/n1h53g1\nle937+TBnn04lp+HWShked28uvx3jufnY1YU/LrOY3360rNBQyZ+NxtPYXtRAKKg+e21eHXoEhzm\nUCVmtoDZotOj3zLI6IMePwVh6V7mLSOpZYB/CyhxYOl6wVtNSY0T0CPkiMQmRJf5/M06NqZZx8Z8\n9868sGGp7jwPKZsOGorhAijw+Rg7cxrH8/OL8nr+b2Myiw6kMOfm27CZzezJOMUTi35m16lAaZJW\n8Qn8e/AwOl7kHIfysBh6A/uklPsBhBBfAWOB4jf3scDnMrBZu1oIESeEqA80K8Nag2pMnN1B74aN\nWH0kLWRMAi9fPZSnlywg3+fDp+lYTQp1o6NJy8nBd5bMT01KJq9bc9biez5N46rPPyq1UN/rK5eD\nlGGV0YSWOzErZfGD5EHmBKSIR8a+iOIYEnGmlBKZ9yq4pgfqcyFB1IL4qRe01diySzMatq7PwW2H\ngzKYbU4bN/x5zDmfr26zpLCd6OzRduo0SSzzeTKOZbFh8Rasdiu9R3St0aW+j+Tm8sH6tfy8bw9Z\nbnfQZ86naRzOyeaLLRsZ164j42d/RZ7XW/S53XnqJLd88zWL7riLetGl9wy/EMpDMTQEUou9TiNg\nFZxtTsMyrgVACDEJmATQpEmTC5PYoEJ5qv9AJnz7dVB0kt1spl/jJgxq1pyld93Hrwf3k5qbS5v4\nBFrUjmfItE+gDCUBzsVDUdrc0uo2NYzKw3IuqaAyE3L+hK7eE2jW5EsGPSOQ/Gi7BoQCahr4VgA+\nkIXho9KFzJwISUsR4vy/mi/Pe5a/X/sah3akYraY8Xn9DLljIDc/NfaczzXh6evYuXpviDPabDYx\n8Ka+ZTrHFy/MYsYr32G2mEAIdE3n2S8fq5HWxv6sTMbNnI7H7w9qSlUcj6ry2oplvL1mFT5ND/nc\n+jWNL7Zs4ol+V1w0OauN81lKOQWYAoE8hkoWx+Ac6FKvPtOuu4mXl/3GlvQTxFit3NapK4/0vhwI\nlLoY2vJM7LtP0zBVoW53q9MbcEW9VJxhtpIi44eCD4IP6UfBXVrGuwTpBt9ysF15HpIGiK9Xm8lr\nX+XQzjQyjmbRvFOTUhPiSqP74M7c/8YdTHniC0xmBV2TxCRE88/vnsTn9vHB45+ydNZqpJT0v643\n97x8K/H1ahetX/fzRmb+ew5+rz+opMjLE97i073/I7FBzaoZ9tKy3yjw+c76QKNJSYE/fESfT9fY\neiJ8P/HyojwUwxGgcbHXjQqPlWWOpQxrDS4ButdvwOzxt5ZprtVkYlKPXnyQvPa8Q1nLk+8OtuWB\n9puwKS5MF7uIjNRBK58vfdP2jWja/sJr+o95YBhDJl7JnuQUHNF2WnVrjsfl5b7LHifjaGZRFdYl\n05axfuFmPtr+JlG1AmHC3749L8TaANB1yZJpS7n5yXEXLF91YmVq6jlZueGwKAqtExLKRZ5IlMfH\nfB3QWgjRXATamt0CzC0xZy4wUQS4HMiRUh4r49oqi9QzkQVfoOe9ifT+hpTnH2FjEMwjvS7nwZ69\nibZYsZvNOMwWYiopMShftXLdouuZn9YcVQ9E2OryvCNtz4IES6eLceILwu600XlgB1p3b4EQgl+m\nL1HoHhMAACAASURBVCPnVG5QaW5N1cjPLmD+x78UHYvUuMjv9QeFAtcUyqP+kVlRmNi5WzlIU8o1\nLvQEUkpVCPEIsIBAyOnHUsrtQogHCsc/AOYRCFXdRyBc9e7S1l6oTBWB9K5AZj9UWJrbi3Q5wdQM\n4qcjlIvTlq8mIYTgkd59ub9Hb7I8buLsDnacTOf272ah6nqp1VljrVbcqoq/nMoWCyDLF8OT60aw\nOKMlL101gGjTSaT3V8j/T7lc48yVFKR3GZiaVtnPUdqeo0z71+yirm7F8bp8JC/YzI2Fju7uQzpz\neGcafl+w5eeIttPlyo4VIm9V4rp27flq21Z8JcK0zUIghAj5zArAZjYXVRWOtlj577CRIc2vyhuj\nVtJ5IKUXmd4XZMnywlZwTkCJfS7MGj94FiA9PwA2hPN6sA6qkEzYS4nj+Xl8uXULc3bvJDU3J+yc\nWTfcwo/7dvP55o3nbbabhMD8/9s77/goqu2Bf8/M1lQChA5SRVAQaVZUFAtgf4pdfHZ9+qwoigU7\nFtSn/tSH6LMrFlREBUVRrCggTboUBekkpG6/vz9mE5LsbLJhU+F+P5/9ZDNz78yZsnPmnnPuOYbB\ns8NOZkCbtuQU+2iTnh5THCjinws550OVwbXVwQ1mO6T55AZXI/yv5Rv418Db49aIMAzh+JFHc/NL\n1wCwY1MOl/W6icKcwtKwV6fbSfv92vDcnEf2uglyBYEAZ7//Dn/uzKUwGMRjOhARnhk6nNtmTCfH\n5ysXCOF1OHj3zHNwmQ4iKLo1bZZU6nld87kWUb6vUTtvsVEMgGRgtCwvm1IB1I6REFyKNWACJAU8\nJyAZ47Ry2A2OfOVF1uflxSx3GgajDhvEZX37s2TrFs79YBKhSITiUAivw4nX4SAUiZAXiH3bdRoG\nR3fsTOesLHo0z+aYTl0SymsT2XEZBH4GYpPTVY4X3EdY0UmqYvlND6TfipF6QTW3WXsopRg15F4W\nfrMkbpoQd4qLJ769j3377Sq7umntFl4e8xazP52H0+1kyIVHMnLsiL02ZDUcifDturXM3biBlqlp\nnLzvfmR5vazNzeGGaZ+yfPs2RIQmbg8PH3s8R9VgskidRK82UcXEr+0cQIX+BLM1ItEc/sVTILiE\nctXgVBEUTwPvmeDa+8L2ksVp2L9pmmLgjHqIe2a34Lt/XsGU5UtZnbODHtktOKlbd37ZsJ5rPptC\nIBwmrBROw8Bpmrx1xojdKqUoWc+g8u6zrnNpNl0HsckTy3ZqCs79QYmNUgDwge8zaCCK4e8/NjH6\nxAfY+Ed8x7jpMLlq/Ej27deFovxiZrw+i0XfLaFVp5Zc8uB53PHmDXUoccPFNAyO6dSZYzp1Lre8\nY5MsPjrnArYUFuAPhWmXkVFvL416xLAbqPA21NajsX9DFMAD4oS0GzFSzyeybQSE5tu39Z6LZIzZ\npUQS2b8KA2Esf/3eyQtzfuHpX36KydzqNk1mjry0ysk/q3Zs53/z57Fqx3Z6tWjJP/v0o21Gcknh\nlPKjwjutuQtFE8H/DfbKwYmlQAys+yWC7YuG6yiMpi8mJVNNoJTioq7Xsnnd1rj1rZ0eJ9c9cylD\nLz2Wreu3c+3Bt1O0swhfkR+Hy8Q0Te569yYOHt6vjqXXlEWPGGoRMZuj0q6EwonR0UNZFFBsLc9/\nhEjx+xCKN5FbQfFbqOJJKNdAJOM+xBE/E6MKb0fljbWK/BBBOQ9AMsYizvgVwPZU/tmnLzNWr2LZ\n9m0UBYM4RHCYJqMPPzKhGaFdmzbjwWPiz0zeHUTciKMFOFoQyVtN/BFDyaiiEr+EpCApI2pUvt3l\n9x+WsXNrXlylABAKhPjk+ekE/SHmfjGf3C07Swv+hAJhQoR5+PyneW/LRJyuxF+CNPWDVgy7iZF2\nHcpxAKrwZesNMbKJ8hXdAHwQSiTIKgyB2ajtZ0L2F4iRFdPC8lOMgPBGSh84wYWoHRdAs48QR8fk\nDqiR4XY4mHTmOcxcu5qZa9eQ6fbwjx496dK0duO7E8ZoCuHVCTYu+RmWFId3gWcouIfUjmzVZPvG\n3CpNGiqiWDlvDWuXvGpNZLPRIQrF4u+XcdAxDS8cV1MerRiSQDyDEc9gVPFUVN5doCor51kVEVB+\nVNEkq0RoGZRvOirvwajyqYDyowr/i2Q+nMS+GyemYTCkc1eGdG54acwldSQqN1FzpwNSL8eK2A4i\nnmOQBjSXYd/+nQkF44x+KiSgssvqWpZ4ZUM1DQutGGoCRydqJlzRB8Hfyi2JFE2CvIco57guRxgC\ndv4LTb3iPp4qHdClBMDVD8N9WC0LtXu07tSSw08byI9liv8AiCGVmpcqEglHOOCI/WpDRE0NU9sT\n/PcKxLk/mF2xnIqJEM9pbIKkogpfQhW9SyS8HfIfJ75SKOnWvvL1mjpHRMAzPMHWCnKuRgUX1qpM\nyXDba9cx4tZTyWiWhmEadD5wH/Y/rHulfUynFTlmmAZur4sbJ1yF2+uuC3EbNTt9Pt5evJD/+/Vn\nZq//i435efyyYT1bCm3C42sJHZVUQ6hILir3FiueXRyAAWYnCK0Eyha494JjXwgtiLMlNxC2oppU\nGGskUsVbp+dcxLUvuI9DzBY1cDSa3UWF/4ZIITg6oYLLYMcZiXd2HoTRbJL9dpWCwC+owGyrpKxn\nmFUAqh75ddpvjDnp4bijhn37dcGV4iRQHMDhctKl9z6c8q8T6bi/fpGJxw9/reOKTz4GFL5QCBFB\nKUWay0UgHGZI5y48ftzQ3a5IqCe41RMqsgMiOaVv8Sr/MSiahJVv3wWpV0LBs5RXFsnixLJPK8i4\nEyPl7BrctiYRVOhPVO6/IfQHiAk4wXsuFL9hPxHSFkFaLotx9CoVQOVcDsEF0Si46Igz83EM7wmx\nm6kjlFJc2vMG/lr+d8w6d4qbs289hY+fnUZxoZ9AcQDDNHC6Hdzy0jUcffbh9SBxw6Y4GGTgxBco\nDMafKOk2TU7p3oNHhuzedU9UMWhTUg0jRlPE0QURFyIujIwxSMs5SPbXSIvZiLf6OfGrJoilaPyQ\n9wAqtKYW9qGJhxUxdi6ElgF+a8Ka2glFL4OKnWEdH9M2+kcVvgKB36IT4ZS1D/ywcxQqYp8WpC4Q\nEca8c6NVZ6HccnC6HCz5aSX5OwoIRP0SkXAEf1GAJy5/AX9xdc7L3sG369ZS1XQ2fzjMlOVLKQxU\nd5Z99dCKoQ4QcSFmtlV8xWgafaOsLUKo/CdQ4a21uA9NOXwzog/tigEIASwHdKITEeM8FoonYT/C\nNKx91xPrlq5n9AkPlPoSwFIW7Xu0Y/y39/LbVwtty4KKISyctbQuRW0UFAUDxJblicUQgxxfFX7H\nJNGKoY4RcULqpUDFPDEJKAvJxDIbpRH3IUIY/F+htg4mkjcubk4bTQ0SXhcnrQVAMTh7Y103D5X/\n5IJE7BS6imd2DFey39pFKcWY4Q+xc+tO/EW73l4dLpNeg3rQ6YAOlT7iDEPnB6vIIe3aE0ogI7Ah\nkJ1Su5l3tWKoByT1Gki7DCQV62HhAc9pQDrxH/heSPsXkv0N0vQlSLkSy1FtRwgIQNHb4Puw5g9A\nUx5HFyodFagAGCXXtoqf3I4LYut6uAcTN7LcfUTictYgK+b8Qd62/JiaFEF/iM9enMEDZz+B22N/\nTsLBML2O7FkHUjYu2qRncEGvA/E64kc3eh0Oruo/cLedz4mi5zHUAyIGknYdKvXKaC3gLEQ8qPB1\nqPzHwfcp5aeOGiBuxHsGYmSAmQ2OLijfxxDZxq4UCxUpRhW+hHirERmjqT7uwVZgQTx/QmgRCVen\njmyBwHflSntK2rUo35dRJ3bJtfaC9xTEUXOZN6tD3o4CxLBXciqimPX+z3H7ZrVqgsut02LYMWbQ\n0RzUqg0v/TaXbUWFNPF4WJObSyAcIsXp5Or+B3N53yp9x0mjFUM9IuICs/Wu/802SJMnUMHLULm3\nQniVtcLRC2nymKUUStoaGdD8Q1TBBCj+AFQetg+f8LZaPgqNiBOVfivk3UPSEx2VD4IryisGsxU0\n/wRVOBH834LRBEm5sBrzJGqe/QZ2JRSofJZzPHK3xqZL11iICMP37c7wfXfNEYkoRUEgQJrLlVQt\nhuqgTUkNEOWfadmtcQIuCC1D+b+PaSdGU4yM0UjWRCyTVEwLcB1Yy9JqAGtUZrTC3ldUDT+PeMBs\nE7vYbIGRcQdG9nSMZpMQ70n1WscjPSuNs0adgiel+hPWsts1kHxWjQRDhAy3u86UAiSpGESkqYh8\nKSIro39jsr+JSHsRmSkiS0TkdxG5vsy6sSKyQUTmRz/DkpFnT0AFfoWCCZSGJJaEoeY/igout+/k\n7A3OA4n1OXiQNJ0Dvy4QcSLN3onW1nAC7miwQHUG5WL189Rs1tfaYuTYs7lxwpV06tUhYSXlTnEz\ncOhBXN7rJk5Ku4DLDriR7z6Ib3bS1A/JjhhGA18ppboBX0X/r0gIuFkp1RM4BPiXiJT1PD2plOoT\n/XyWpDyNHlX0JvahiUFU8fu2fUQEaToBUs4HSQdMcPZFmr6GOLWTr64QsxVG09eQFj8i2V+CoweJ\n5UpyAF4wOyLN3kSkcaSNEBGOOW8QExaMZ8Stp+C08RuICC6Pk5R0L540DwOHHsTU/37J2t//wl/k\nZ92S9Twy8lmmPD+9Ho5AE49kfQynAkdHv78KfAPcVraBUmojsDH6PV9ElgJtgXhFCvZuIjnYmx7C\nENkRt5uIB8kYDRl2ullTWyilrMlsklJaOEmMTCATlfB8FQOavYs49m20ZV7Pvf0Mfp46j83rtuIr\n8GE6DEyng3/955/0P/5Adm7Lp223Vpzf8Rr8ReWd9P4iPy/f8RbDLjsWh1O7PRsCyV6FltEHP8Am\noGVljUWkI3AQMLvM4utE5CJgDtbIIidO3yuAKwA6dOiQnNQNGffgaLbUihNYUhD3UfUhkSYOkaIp\nUPBoVJkbKO8pSMadiFhzVMT7D1Rwnk0xp5gtIWbbRqsUAFIzUnh+7iN89/7P/DptPk1aZnLiPwez\nT08rNUyLDtn8/ccmQgH7EVQkHGHj6s207962LsXWxKFKxSAiMwC7Qrhjyv6jlFIiEtfLJiJpwAfA\nDUqpkrCE54H7sV6R7wfGA5fY9VdKTQAmgJUrqSq5GyviPRNV9BqEN7MrNNEFjnbgObHW96+UQhVN\nskpTRraBoweSfhOi61KXQ/m+gLw7KWf2K56CCm9Cmr6EiuxEhVaxq3xnJbeskR2d09K4cbqcHHPe\nII45b5DtejGlXNrusoSCYdKbptWmeJpqUKViUErFLSMlIptFpLVSaqOItAa2xGnnxFIKbyqlJpfZ\n9uYybV4EplZH+D0RMdKg2WRUwfNWMXgM8J6OpF5WKzWeVSQXwlvAbIcYKaj8R6HoLUpHLMG5qB2X\nQtYLSAOtF1AfqPzxxPqC/BD4hUhgAeT+O2r6qyonkED6bY16tJAIAV+AO4Y+ZLvOdBj0PrIHTbIz\n61gqTTySNSVNAUYC46J/P67YQKw7/iVgqVLqiQrrWpcxRZ0OLE5Snj0CMZogGbdDxu21tg8VKULt\nHA3+r0tTfCvvCCh+ByvHT1l8qPwHEfentSZPoyP8p/1ycULhi9bExZjzaIPjAAzvnh+MN/OdH9i2\nfrttiu4mLTK5/c3rbXpp6otko5LGAceJyEpgSPR/RKSNiJREGB0OXAgcYxOW+qiILBKRhcBg4MYk\n5dEkiNp5E/hnAoFoSVJfVCnEMXmEVhKJJBJhs+eiVBhVPJXIjouJ+9NRoaiPKLHsl5J2ZU2J12DI\n25HPsl9Wsn3jLnfh7E/n4Su0Hz117r0Pmc0zbNdp6oekRgxKqe3AsTbL/waGRb9/T5wEQEqpC5PZ\nv2b3UOGN4P+BWDNHFQ8z/3Tw1t9s2/pEKYXKvQ4CP1TiTBYsRWtrUbXfruuYKlMtNxbCoTDPXvcS\n01/5BpfHScAXpP8JB3L7G/8mq2UmhmnE1HwWEZq01Cakhoae+bw3Ev7Lyu1TXQon1rwsjYXAj9bH\nVim42TXjuRopMczOGMaeE575vzvf5svXZxH0ByncWUTQH2TO9AU8fP7TDLt8CE5X7LG6vC5OuvL4\nepBWUxlaMeyNmPtUUkCmkslVkb2zxoMK/205m+OluDY7Y6VRr06wnMfyIzUiFv+wjGsPuZ0TXWdz\nRvN/8vKdbxGM5ksKBUN8/H/TYuYoBP1B5n65gMzsDC5/9AJcHieeVDeeVDdOt5ML7zmLnofsWx+H\no6mEPed1RZMwYrZEuY+J+hjK/pC94D0Dij8EKj4EDXAeVHdCJoBSPusYIjvB1R9xdK35ffi+QuXe\nSKVmNiMLwsvirzf3B/cg8H0AkVxwdEXSRyH1lDJ7d1g6eyWjT7i/tPZC/o4C3hn3Ee8//gn7DujC\nGdcPJxyyHy053U62/LmNU/81lCPPPJSfp85FRRQDh/eleZumdXkYmgTRimEvRZo8hsq7F4o/ATEA\nA1KvsOo8hH6H4BLKPwzdSNp19SRtLCowB5VzBaAgWr9AeY5FMh9HaqhCXiS4CHKvo/K0Fl5IOQt2\nzgbC9k1EYWTcBBk31Yhc9cHE0W+UK8gDVnrtYCDE7z8sZ+W81RimvQEi6A/Stqs1FSqrZROGXhrj\nltzjyfUVs3jLFpqlpLBfs+YNPjxZK4a9FBE3kvkQKn2MNXPXbFE6T0JlvYIqGA/Fk6000M4+SMYd\niLNhDPmV8llKQRWUX+H7AqVuhPQbk65TENl5LxS/RaXmIfGC60jEMxRV+BKE4kRbhzfaL29ErJy7\nutL1geIgLq8Td4q7nDnJdJgcclK/vTbqSCnFoz9+xyvz5+EyTUIRRZv0dF465XQ6ZDapb/Hion0M\nezlipCKOduUmz4mRgpFxF0bL3zBaLcVo9jbi7FWPUlbAPxP7B3YQ/NNQ204hsvPO3S5rGtl5JxS/\nGWcfJZhIk/8iTZ5GxADvWfGbqtzdkqMhkciD3TAMTvnXCTjcu943I5EIP0+dy6v3TKpN8Rosk35f\nxGsLfsMfDpMfCFAcCrImN4fzJ79HpAGX3dWKQVMnqNCfRPKfJZL3MMr/A0olUdAmkltqPrLHb5nI\nfDHzLauWM7jIKnxUKQa4hyDuQ0pNAuI6gLiOe2m4b4aJcubNJ+GuovZCOBShx8HdMM1dpjwVUQR8\nQd4b/wlLfl5R22I2OF6Y+wvFofKmyIhS5Pp8/Lz+r3qSqmq0YthLUJEcIjvvJ7LlMOuT9zAqUjeV\ntCKF76C2DYfC56Dof6jcf6FyLkEpK6JF+b8hsv08IluOIpJzHSpYiSMXwJlIacNiVOGr1ZZVFX1E\nXF9BCZKBpN9WfpnjADDbEfuT8kDqxdWWo6Fx8tUncNyFR+J0O219CYZp0K1fZ3748BcCxbERb4Hi\nAJ+/9FVdiNqg2FpoH8mmUGwsyK9jaRJHK4a9ABUpRG3/hzWzObLN+hS9idp+phXZU5v7Dm+C/Aex\nop+ib06qCALzUEVvEyl4AZVzDQTnQGQj+L9AbR+BCsyLu01xdrOy0NpWrStDZAsqZG8bV5E8IoGF\nRHJvsRTStjNQxVPjh6SW4kGaf4442pWXSQRp+rIVuipekDTADd6TkNQrqthmw8cwDK5//gpeW/UM\n179wBVmtmmA6TUTAdJpktczkzndupHBnEXYWEqUUhTurOrd7Ht2b2VeriyhFz+bZdSxN4mjn816A\nKv4QwtvZla0VIGBlcC2eCiln1uz+wptQRe9AaDWoAPa2ep+VrC+8psJ6BfhQO8ci2VPi7kOaPGFl\noS2cGH9+RSQXte00lKMTkvUCYrZGhVahcm+D0FLKRRtFNqLyxoBrEFYFtjj1jCUFMe1/7GK2huaf\nWtsObwZnT8SsNBN9vVNc6OPXz3/DXxTgoCG9qgwfbd62Gfv27Yyv0HqhUApUOEJBbiFrf/+Lw08/\nmPkzF8ekv/CkujnitIG1dhwNlVGHDeLSTz7EV8ac5DZN+rZqQ4/sFvUoWeXI7jro6pP+/furOXPm\n1LcYjYbIjssgMMt+pft4jKxnrTdr/49gpFj2c2P3okhU4BdUzuVRH0AA690jXrhnGlAQZ50gLRdV\nmVFWKYXaPgJCC4nvLDbAbA9N34Ftx0ejmeK1dYHZFcJ2daQM8JyE0eTxSmVqLMz+bB4PnP0EYhgo\npYiEwpx588n88/5zK+13db9bWfXbmpjlTbIzeG31s9xw+F2sX7GRgM8Kb3V7XXTo0Zb//PggTlds\nlbc9nW/XruG+WTNZtzMXp2FyVs/9uWPQUXgcdX8uRGSuUqpKW6weMewNmM2wrIYVHb4mGNlEdo6B\n4im7ljEWmjyFeI6p1m4ioY2Qc22FtBHxlIKLqm+/BOYjhBZDaDmVRxBFrFFFwROVjGCiiBPSLoXC\nVyG0qExbAyQVSft31TI1AnI253L/iPExcxMmP/Up+w3sxqEn2z878nMKWLvYPrOsvzjA36s289QP\nD/DxM58z7ZWZFOcV0/GADpxx/bC9tjrbUR078VXHTvhDIZymidHA5zCA9jHsFYj3PKwHcUWcYGZb\n5iT80U8R4EPl3oCqpJRoWVQkl8iOf8K2IQmGZrrBaAaOzvGbmF0Tmqim8h/Fvka2DcElVFkfQYUR\nIwNpNgnS7wKzExjNwXMS0uxDxLFnVA+c+fYPtimwfYV+Jv8nfnp1w4j/UFNKYZgG3lQP+w7oyvYN\nOyjMK2beVwt58NynuP6IO/HbOKb3FtwOR6NQCqAVw16BuA6E9BsAV9QxmmJ9T78NfDOILSMaxfd5\nQttXOVdA4Bfi2uVLMcDcD9KuRpp/gqRegH2Ip0CmfVEXpQKosqGqgQUJyYiKgLMHlv+gMnwoZ39E\nTIzUCzCyp2O0+BGjyeN7jFIAa8QQ8Nlfr5xN8ZV7amYq+/bvYjtzNy0rlY77t8df7GfsGY/hK/Rb\nk90UFBf4+OO3Nbx+33s1dgya2kMrhr0EI/USJHsmkn4nkn4Xkv0tRur5ViF7WwKQQDirCq6A4HKq\nVgpYjtuM2zHSrrF8GJ5h4DmeXdFFBuCEtFsxXAeW309gPpFtp6M290Zt7k0k92ar+pyRXvV+cYKr\nD5J2LUhV5gwvErLzL+xZHDCoB9602Kguh8uk3/EH2vTYxS0vX0NaViruFGsU6vI48aZ5GPP2jYgI\nc6YvwO7FOOALMv3lmTUiv6Z22TuNfnspYmZbeX3K4joSiicR4wsQN7gSiCIJ/wliJphYVJV7MIsY\nSJPxqOBClO9rEA/iGRbzZq6CK1A7RrJrZBMB3zRUcCl4z7EqplVmTnL2RZq8gBgpkDURlXMDqDiR\nTDWUZ6mhM+DEPrTfry1rF/9ZOnIwDMGT6uGsm0+utG/77m15deUzTP/fTFbM+YP2+7Vl6GXHlkY0\nFRf4bM1UAL44NZ81DQutGPZyJO1KlG9qNFKnxETjAWc/cPategOOLqASGC1YjW0ztIqzN+LsHbeX\nKniOWN9AECJ/g7MnuI8E/5fYayc3kvmApRQAcQ2AFt+hdpwDwQU2fQSclb8x7wkYhsHjM8fyxn3v\n88Wr3xDwBRk4tA+XPHQezdvah+OWJT0rjTNvslcgfQbvTzgUO0lQDKHvkAaUWkUTl6RMSSLSVES+\nFJGV0b9ZcdqtjZbwnC8ic6rbX1N7iNkKaf4xeE+zHMJme0i7Dsn6b0IZIMXRKTqysPMVlNjzLd+G\nNPkPUqUpx4bgAmwL4CgfElqBkfUsZI4n1n/gAfexiGOf8jKLgWQ+DpLJLjOWA/AimY9VGSK7p+BN\n9XD5Ixfw3qaJfJz7KmPevpHWneLPu9jy1zbWLP6TULDyEq/N2zbjtOuG4knddU+YDoOUdC+Xj7ug\nxuTX1B5JzWMQkUeBHUqpcSIyGshSSt1m024t0F8ptW13+ldEz2NoWKhIESpvLPg+A8RycKddi4gX\nFZwLZnvEe+ZuT/aKbL8Agr/YrElBMu9BvKdbcgR+Q+U/AsHFYGRAyoVI6uVxlZGK5KCK3oXgPDA7\nICnnI46OuyXjnszmdVu5/+wnWLNwHabDxHSYXPXESE64eHDcPkopvp88m/efnEru5lz6HNOLc28/\nnVYdG+6krr2BROcxJKsYlgNHK6U2ikhr4BulVHebdmuxVwwJ9a+IVgwNE6WKLYe10bzGaiIAKP8s\nVM51xERPSQaSPavUTKSpecKhMBd2+RfbN+wgUsZv4E5xc++Ho+h33J5vdtuTSFQxJBuV1FIpVZJs\nfhMQ75VQATNEZK6IlE0ck2h/ROQKEZkjInO2bq2bEpM5W3bywi2vclHXa7niwJv55PnptrZTjYWI\nFzFb1qhSABD3kZB+E+CxchBJChitkKavaaVQy8z+bB4FuYXllAKAv8jPG/e/X09SaWqbKg2+IjID\naGWzakzZf5RSSkTiDT+OUEptEJEWwJciskwpNasa/VFKTQAmgDViqEruZMndupOrDhpF3vZ8QgHL\npvrfUa/zy+e/cd/HtzX4Ckx7GkbqSJT3TAguBCMVHL30NUiAgtxClv2yitTMFPYb2LXa5+zvVZsI\n+u2DCzas2lQTImoaIFUqBqXUkHjrRGSziLQuYwraEmcbG6J/t4jIh8BAYBaQUP/64P0nPiG/jFIA\n6y1p/szFLPlpBfsfVqXFS1PDiJEK7kPrW4xGw5sPfsBbD36Aw+VARRRpWak88MntdO69T9Wdo3To\n0Q6n20koEDtS3qdnO5semj2BZE1JU4CR0e8jgZjKKCKSKiLpJd+B44HFifavL378eA7BQGz0hb/I\nz5NXvMDpTS/mnLZX8Mo9k0qThWk0DYXvJs/mnXEfEvAFKcorprjAx9a/tjPq2LHVul/7Hd+brJZN\nMB3lzYNur4uL7hlR02JrGgjJKoZxwHEishIYEv0fEWkjIp9F27QEvheRBcAvwKdKqWmV9W8IpGR4\nbZcrBX8uXU9BbiHbN+bw3uMfM/qEB3a7jKRGUxu8M+7DmNTXAMFAiB8//jXh7ZimyZOz7qPPNVs+\nrAAAG/dJREFU4P1xuBy4vC6atcli9Bv/ptegHjUpsqYBkdQEN6XUduBYm+V/A8Oi31cDtqEL8fo3\nBE695kSe/v1F2x9XWR0QKA6y8rc1LPjmd/oMPqAOJdRo4rNtg30CxKA/GHddPJq2ymLc9LsoyC2k\nuMBHszZZGIbOprMno69uHI69YBCHnToAd4ob02Hi8rogjt/OV+hj0XdL61ZAjaYSug/oYpuvyOFy\n0q1vJVltKyGtSSrZ7ZpppbAXoFNixMEwDG5/43pW/baGOdPn40n18OVr37BibmypSLfXRWbz3Sts\no9HUBheNHcG8GQvL1Vtwuh20796G3kf1rEfJNI0BrRiqoOtBneh6UCcAslpm8vilz9mYl4Sjzz6s\n7oXTNHr+Wr6B98Z/wh/z19K5VwfOvPlk9unZPqltbtuwnSnPTcfpdhIOhgkFwzhcDo49/0iuemKk\nDvPVVIlWDNXgyLMOZclPy/nkv19imgZiCCqiuPu9m8lolkj6Z41mF99M+oFHLnqWcCiMUopVv61h\n5qQfk5pRnLM5l6v63kpBTmHpZEyX10W3vp24ccKV2gykSQhd83k32LxuK799tQhvmoeBw/viTY3N\na6/RVMbb4z7k5Tvesl2X3a4Zb657PuE3+8K8Ima8MYuVc/5g45otLPlxOaFg+XkH3jQPd713MwNO\n6JO07JrGi675XIu03CebEy+pXj1kjaaEpbNX8kYllczydxSwYdUm2nVrXeW2NqzayL8PG0OgOICv\n0I+I2IZOFxf4WDBzsVYMmoTQiqERUZhXxKJZS3G4HPQ+qicud1VlKjUNkc9e/NJ28mQJEaVwuRP7\naT528f+Rv6OgtDBOPAuA0+0kXZs7NQmiFUMjYeqEL3n+xldwOK0ZqIIw5p0bGHBibOEbTcMmb3tB\n3ApnAG06t6RFh2wAigt9fDvpR9b+/icd9mvH0eccTkq6NfmyILeQ5b+uqnRbJYgIx553RM0cgGaP\nRyuGRsCSn5bzwk2vEigOECiTefreM8fzyvL/JFRxS9NwOPTk/sybsdB28qTb6+KOt28ALDPRDYff\nia84gK/AhyfVzcTb3+TJ7+5nnx7tiIQj2E5WiOJ0O3A4HYTDEW595Vp9n2gSRocoNCBytuzkjQfe\nZ+wZj/HKPZPYtmE7AJOf+pRAcexDJBKOMP2Vb+pYSk2yDD73cFp0aI6zjCnQMA2yWmby6qpn6XSA\nVfP64QueZuf2fHwFVj1rX6GfgpwCHjr3KQAymqXTbl97P0R60zSuHD+S6569jEkbJnDUWTr5oCZx\n9IihgbB64TpuPPIuQoEQAV+QXz7/jclPTmXc9DvZvG4rdqbjoD/I5nV1U5tCUzPs3JbH+098QiQc\nIaNpGsFACE+qm2PPH8TZt51GaoZVXyJny05WL1gbYyZSCtav+Jstf26lebtmHH/R0Uy8/U1UJIJS\nVl1ll8fJba9dx8HDEqjZrdHYoBVDA+GRkc9QlLfLThT0Bwn6gzxw7lMMPucw/pi/NsZh6U3z0PvI\n3Z/FuuXPrcyf+TspGV4GnNgHt9eubrOmuiilbENNy9b4CPqta+lJddP3uN5c8uB55doGfQEkzpwD\nwzQoLvBx96mPsODbJZZJCUsptO7ckrGTR5WOOjSa3UErhgZAzpad/LVsg+26rX9uo/uAbrhT3ISC\n4dKoE9Npktk8gyPPPKTa+1NK8fyNr/DphC8xTBPDEBAYO3kUBx3TK6lj2VvxF/t5+Y63+fylr/AV\n+uncex+ufvJiDjx6f8LhMIu/W2bVP96aR7jMHANfoZ+fpsxhxdw/2Ldfl9Ll2e2b0yQ7gy1/bovZ\nlzfdy5rFf7Lgm9/L+SlURLFjYw65W3bW7sFq9ni0j6EhoJStqaiEaS99xTOzH2bA0D5WQj+Pk6PP\nPpxnZj+Ey+Oq9u5mvv09n7/0FQFfEF+hj6L8Yoryirn71Eco3FmYxIHsvdx96qNM/e8XFBf4UErx\nx4K1jBn+EDPemMX5Ha/hrlPHMXvq3HJKoYSgP8ic6QvKLRMRbp54Ne4UN4Zp/UwNQ3CnuLj5xauY\n8fosW+e1v8jPzLe/r52D1Ow16BFDAyCrZROatMhk2/rttusXfb+Mdt1a8+DUOxLaXmFeEct/WYU3\n3Uv3AV1i0iB88ORU24cKwKz3f2bopQ0yE3qDZdX8Nfz+4zICvvIlMP3FAcZf+lzMLOSKOJwmntRY\nM17fIb155qcHmfTYFFbPX0vHA9oz4tZT6dqnE1Oem267LaWIqc+s0VQXrRgaCKdfN5QXb3vDdl3J\n3IVEePfxKbx6zyScTgeRSITUzBTu/2Q0Xft0Km2Tuy3Ptm/AF2TnVvt1dYVSikXfLWX1wnW06tiC\nASf2iakeVt8EA0G+mfQj30z6EZfHSbM2TeOO+KpSCgABf5C3H/6Qxd8v46KxI+i4/64kep167cPo\n166L6XPsBUeycNaSGAXvSfUw+JzDq3dAGk0FtGJoIJx8zQm8NvZd/MXlyy46XQ6OOW9QQtuY/elc\nXhv7bnS+g7Wd4gIftwweyzsbJuBJsd5K+ww+gBmvzyp1Wpbuy+2g4wHtCQVDOJzJ3xqrflvDirmr\nyW7XlL5Delf5gC/ILWTUkHtZv2IjkVAY02mSku5l/Df30rZr1ekhqks4FEYMqVZiuYA/yC2Dx7Jm\n0brSh7LL4yQciti2j5eiAihNwoiC3C07+f7D2fw6bT5PzrqvNKNvPI4661CmvzKTJT8uL5XDk+rm\n4OF96Tukd8LHo9HYkVQSPRFpCkwCOgJrgRFKqZwKbbpH25TQGbhbKfWUiIwFLgdKYi7vUEp9RhXU\ndxK92uKHj37hwXOfJBQIWaGHImS2yODlJU+RnpVWZf8bBt3F7z8ss13XZ/D+jPviLkzTZOPqzVzV\ndxTF+b7Sh5bhMBDAME1Mh8HJV5/ApQ+dt1tv6/5iP3ed8ghLfloOCIYppKR7efzrsbTbt03cfg+d\n/x++/+DnctFXYggd9mvLxMVPVluOeKyY+wfPXPsSy39ZhekwGXTmIVz79CUJZcj9dMKXPH/Tq/iL\nKpjiBKjwU3J6nKhwxHbUkNk8nfycAiLh2N9fn2MO4LEZ91QpSzgc5vvJvzDz7e8wHSbHXXQ0A4cd\npDOoauKSaBK9ZO+g0cBXSqluwFfR/8uhlFqulOqjlOoD9AOKgA/LNHmyZH0iSmFPpnnbpohhlJol\nlFIU5/v4313vJNR/61+xESwlLPp+GW89+AEArTu35NnZD3PYaQNIyfDiTfcgIoRDEYL+IL5CP1Oe\nm8Yz107creOYOPpNfv9hGf6iAP4iP8X5PnZszOWOYQ/GfXsOBoIxSgGsSJtNa7eybslfuyVLRdav\n+Jubjx7LstkrUUoRCob47v2fuP7wMaVpqitjxuuzYpUCVoqSlHQv3jQPqRleXB4np1xzAgOH9cXt\njQ0QKMovtlUKAIu/t1fuFTFNk6POOpSxk2/lrndv5pCT+mmloKkRkr2LTgVejX5/FTitivbHAn8o\npdYlud89kv/e8lqpCagEf5GfaS99XToLujI6HbhP3HXhYJgPnvy09MHcvntbxn4wisnb/4dpmjHR\nMv6iAF+8+i152/OrdQxKKaa9/HWMI1YpRc6WPFbM+cO2X8AXjOs0NR0G+Tk1Ey319sMfEvCVP8eh\nYJjtf+fw89S5VfY3HHF+MgJDLz+WR768m9vfvJ431z3PVY+P5K53b+KsW04htUlqueYl8xjssHNE\nVyQSifDz1Lk8fMHTPHrxs8ybsTCu0tVoqkuyiqGlUmpj9PsmoGUV7c8B3q6w7DoRWSgiL4tIVryO\nInKFiMwRkTlbt+6Zs32Xzl5pu9zpcvD7D8vj9lu9cB1X9R3F3GnzK91+UV5xzFtx/o6CGL9GCS6P\nkw2rNlUhdXkikYjtGzWAaQo7t9krmpR0Ly07ZtuuC4cidOnTsVpyxGPJT8tjfCtg+WLsyrZW5ISL\nB9s+uF0eJ4PPPpweB3fj4OH9aJKdCYDD6WDkvWdz/EVHlYadVobL46wyKiwcDnP3aY/y4LlP8fVb\n3/Hla99yz+mPMv7S57Ry0NQIVd6pIjJDRBbbfE4t205Zd2Tcu1JEXMApQNlE9M9j+Rz6ABuB8fH6\nK6UmKKX6K6X6Z2fbP0AaOynpcQr+CHFTJuds2clNR93NH/PXVhkB07xd0xincmpmStwHVsAXJLt9\n9RKvmaZJxzizboP+EN0HdLFdJyJc+8ylMWYXT6qbi8aOIBwMk7ejeqMXO1p2bGG73OVxsWbROp6/\n8X/8PHUu4bD9uTz2/EH0PLQ7njRPVG7wpLgZeumxdB/QNe5+/161yVYhleBwOfCkuuk+sCsj7x1R\n6TF8/8FsFsxcjK/QV7rMV+jn2/d+YsE3v1faV6NJhCpDT5RSQ+KtE5HNItJaKbVRRFoDWyrZ1FBg\nnlJqc5ltl34XkReBqYmJvWcy/Mrj+ODJT2PMSS6PiwOPtk998fnEGQT9Qdt1ZXGnuLjkofNiljtd\nTk6++ng+eW56uZGD0+2g33G9ad6maTWPAq5+8mLuOmVcuUL0nlQ3p1xzApnNM+L2G3BCH8Z9cRev\n3j2J1QvX0aJDc44feTQz3/mB/415C0To0KMtt7x0Dd36dq62XAAjRp3K4u+XxYxqAr4Av06bTygQ\n4vOXvqZDj7Y8PvPe0kiuEkyHyUOf38HsT+cx672fcHqcnDDyaA44okel++3WvzOzP5tnu84wDS4a\nO4LeR/ak56H7Vlm57YvXvo07ue3rt7+nz+ADKu2v0VRFsqakKcDI6PeRwMeVtD2XCmakqDIp4XRg\ncZLyNGouuOssDjyqJ+4UN26vi5R0LxnN0nj48zGYpn100Io5q2Ps+SUYpoHpMGnWJovrn7+CIecf\nadvusofP55jzB+F0O0nNTMHlcdLvuAO5/c3rd+s4DjqmF+Om30WvQT1IyfDStltrrn7yn1w27oIq\n+x5w+H489tU9fLD1ZR6dcTev3/cey2avJBQMEwqEWL1gHTcPvoetcSYDVkXfY3tx+SPn405xk5Lh\nxZO268Efijq+iwt8rFn0J5Me/ch2G6ZpctgpAxj9+r+5+cWrq1QKAP0qCSF1uh0MPudw9j+se0Ll\nPFXEfuShVPx1Gk11SDZctRnwLtABWIcVrrpDRNoAE5VSw6LtUoE/gc5KqZ1l+r+OZUZSWOGuV5bx\nWcRlTw1XLWHVb2tYOnslTVs1YeCwg3C64ldqe+WeSbz76McxowaHy8Gp157IyLEj8KR6Enrg5G3P\nZ/3KjbTo0JzC3EI+fnYaf6/eTO8jezD8iuMqfduvDd5/4hNeufudciMPsHwup18/nMsfqVrRxKO4\noJilP69k4awlTH7qU4oLfDFtsts34611L+z2PspSlF/MmS0utR3dedI8fLDlpYTTm3z91nc8eeV/\nbSa3uRk7eRT9jjuwRmTW7HnUSc1npdR2rEijisv/BoaV+b8QiDFWK6UuTGb/eypdD+pU5QSnEk66\nYgiTn5waqxicJqddOxRvmjfh/WY0S6dns3S+fe8nHrv4WYKBEJFwhEXfLeX9J6fy7M8P06ZLq0q3\n8eOUX3nz/vfZvG4r++zfnovvO4deg6p+o7ZjxZw/YpQCQDAQYvmvq3ZrmyV407z0HdKbjas3x42G\nCsYZiYFletq4ejOZ2RmljubKSEn3ctKVx/HZxBnljsmd4uaMG4ZXK+fVUSMOY9rLX7N09ko9uU1T\nK+iZz42c5m2b8fC0O3n4/P+QuzUPEUjPSmP0G/+mVRxHa2UEfAHGX/ZcOX9DoDhA0B/k2ete5qHP\n4udr+ujZz5g4+q1S+/3Cb5dw+9AHuPOdmzjkpH7VkiMYiF9rwnQY7NOzXbW2F4/+J/RBRf5nsw+T\nQ08dELNcKcW7j33MGw98gAiEAmH6DunF6Nf/TVqFkNSKXDn+IpxuB1Oem46KKAzT4B83ncyFd59Z\nLZlNh8nD0+7k+8mz+eqt7zAdBsePHMzBw/smNDLUaKoiKVNSfbGnm5J2B6UU61f8jVLQvnub3X5A\nzPtqEff+47FytSFKMEyDz/1v206i8hf7ObPlZaXVxsrSYp/mvLH6uWrJdP/ZT/DzJ3Ns/SfuFDcv\nzHu00lnUSim+eO0b3rjvfbau306rfbIZed85tnmEXhz9BlP+b1rp23eJr+X5eY/GON8/mziD5254\npZzz2uFy0L1/F576/oGEji3gD5K3LY/M7IxKzYQaTU1TJ6YkTcNBRGjfvW3S2zGM+A/vyp7raxf/\nFbfvjo255G3PT9hHsWntlrhKwTANxk4eValSAHhv/BReG/te6QN8w6pNjL/seQpyCzj5qhPKtb18\n3AX0HtSDj575nNyteQwcehCnXz/M1kT0xv3vx0Q0hQIhVs1fw5pF6+jUK/4kwxJcbqeuv6xp0GjF\noClHz8O62y43TIOBQ/vGTbmQlpVqW2ugBHdK4tXhVi9ch8PlsFUMkXCEfsdVbkcP+AK8fl/sA9xf\nZBXTGXbZkJgcUAcP78fBw6s2d23bsMN2uekwWb9yU0KKQaNp6OjEKppyuNxObn/jetwprtJ03+4U\nNxnN0rn2mUvi9mvbtTVt920dM2pwOE0OGd4vZj5AZbTo0DzuZLDM5ulVmqT+/mNz3NFNKBDa7VBX\ngBbtm9suDwfDdNiv8lGMRtNY0IpBE8MhJ/VjwoLxnH79cI74x8Fc+tC5vLL8P7ToUPmM87GTR9Gs\nbVO86VYSOW+ah/b7teXGF6+s1v67HNiRtl1bY1bIS+ROcXPmzSdX2T+zeTqhgP3oJRwOk55VuZO4\nMi66d0TM6MfpdrDfId3Yp2f7OL00msaFdj5rapRwKMyc6fPZuGYLnQ7oQO+jeu6WI3zHphzuOuUR\n1i35C4fTQdAf5IRLjuHapy9JKIPo7Sc+wPyZi8ulCXG6HRxyUj/ufu+WastTlo+e/YxX7p5EOBgm\nHIpw6Cn9uXni1aSkJx4arNHUB4k6n7Vi0DRo/ly2ge1/76BTrw4JzRcoIW9HPncMfZB1S9ZjGAbh\ncJhufTvzwNTbSc1ISVquUNAySWU0TSM1c/dHIBpNXaIVg0aDNYt8w8qNdOjRVjuGNXs9OlxVU6f4\nivysmrcab7qXzr33aTATraozi1yj0VhoxaBJmk9emM6EUa9jmCaRSJgm2Znc99Gt+g1do2mk6Kgk\nTVLM/XIB/73lNXyFforyivAV+Nm0Zgs3Dx6LL07BHo1G07DRikGTFO+M+8g20V0oEOL7ybPrQSKN\nRpMsWjFokmLjms22y31F/rhJ8DQaTcNGKwZNUuzbvwtikyPJk+Kmy4Ed614gjUaTNFoxaJLi/DH/\nwF2hloDDaVWNGzC0Tz1JpdFokkErBk1SdDmwIw99PoZOvTpgOgwcTpNDTxnAU98/ELccqUajadgk\nFa4qImcBY4EewECllO2sMxE5EfgPYGKV/BwXXd4UmAR0xCrtOUIplZOMTJq6p9egHkxYMJ7iQh8O\np6lrDGg0jZxkRwyLgTOAWfEaiIgJ/B8wFOgJnCsiPaOrRwNfKaW6AV9F/9c0UrypHq0UNJo9gKQU\ng1JqqVJqeRXNBgKrlFKrlVIB4B3g1Oi6U4FXo99fBU5LRh6NRqPRJE9d+BjaAn+V+X99dBlAS6XU\nxuj3TUDLeBsRkStEZI6IzNm6VYdBajQaTW1RpWIQkRkistjmc2pVfauDsrL5xc3op5SaoJTqr5Tq\nn51deV0AjUaj0ew+VTqflVJDktzHBqBsBZN20WUAm0WktVJqo4i0BrYkuS+NRqPRJEldmJJ+BbqJ\nSCcRcQHnAFOi66YAI6PfRwIf14E8Go1Go6mEpBSDiJwuIuuBQ4FPRWR6dHkbEfkMQCkVAq4FpgNL\ngXeVUr9HNzEOOE5EVgJDov9rNBqNph5plIV6RCQfqCoaqj5pDmyrbyEqQcuXHFq+5NDyJUcy8u2j\nlKrSSdtY6zEsT6QKUX0hInO0fLuPli85tHzJoeXTKTE0Go1GUwGtGDQajUZTjsaqGCbUtwBVoOVL\nDi1fcmj5kmOvl69ROp81Go1GU3s01hGDRqPRaGqJBqsYROQsEfldRCIiEtcDLyInishyEVklIqPL\nLG8qIl+KyMro36walq/K7YtIdxGZX+aTJyI3RNeNFZENZdYNq2v5ou3WisiiqAxzqtu/NuUTkfYi\nMlNElkTvhevLrKvx8xfvXiqzXkTk6ej6hSLSN9G+NUEC8p0flWuRiPwoIgeWWWd7netYvqNFZGeZ\na3Z3on3rSL5RZWRbLCJhsUoD1NX5e1lEtojI4jjr6+7+U0o1yA9WjYfuwDdA/zhtTOAPoDPgAhYA\nPaPrHgVGR7+PBh6pYfmqtf2orJuw4ojBqmNxSy2ev4Tkw6qD0TzZ46sN+YDWQN/o93RgRZnrW6Pn\nr7J7qUybYcDngACHALMT7VtH8h0GZEW/Dy2Rr7LrXMfyHQ1M3Z2+dSFfhfYnA1/X1fmL7uNIoC+w\nOM76Orv/GuyIQTX8lN7V3f6xwB9KqXU1LEc8kj3+ej9/SqmNSql50e/5WDPn21ZsV0NUdi+Vlfk1\nZfEz0ESsHF+J9K11+ZRSP6pdha5+xspLVlckcw4axPmrwLnA2zUsQ6UopWYBOyppUmf3X4NVDAlS\nIym9d5Pqbv8cYm+066JDwpdr2lRTDfkUMENE5orIFbvRv7blA0BEOgIHAbPLLK7J81fZvVRVm0T6\nJkt193Ep1ttlCfGuc13Ld1j0mn0uIvtXs29dyIeIpAAnAh+UWVzb5y8R6uz+q9eZzyIyA2hls2qM\nUqrGEuoppZSIVDv8qjL5qrN9sZIHngLcXmbx88D9WDfc/cB44JJ6kO8IpdQGEWkBfCkiy6JvLon2\nr235EJE0rB/pDUqpvOjipM/fnoqIDMZSDEeUWVzlda4D5gEdlFIFUZ/QR0C3OpYhEU4GflBKlX17\nbwjnr86oV8WgGnhK78rkE5HqbH8oME8ptbnMtku/i8iLwNT6kE8ptSH6d4uIfIg1LJ1FAzl/IuLE\nUgpvKqUml9l20uevApXdS1W1cSbQN1kSkQ8R6Q1MBIYqpbaXLK/kOteZfGWUOkqpz0TkORFpnkjf\nupCvDDGj+zo4f4lQZ/dfYzcl1WdK7+psP8ZeGX0YlnA6Vv3smqRK+UQkVUTSS74Dx5eRo97Pn4gI\n8BKwVCn1RIV1NX3+KruXysp8UTQ65BBgZ9QclkjfZKlyHyLSAZgMXKiUWlFmeWXXuS7laxW9pojI\nQKznz/ZE+taFfFG5MoGjKHM/1tH5S4S6u/9qw7teEx+sH/t6wA9sBqZHl7cBPivTbhhWtMofWCao\nkuXNgK+AlcAMoGkNy2e7fRv5UrFu/swK/V8HFgELoxexdV3LhxXFsCD6+b2hnT8sU4iKnqP50c+w\n2jp/dvcScBVwVfS7AP8XXb+IMtFy8e7DGj5nVck3Ecgpc67mVHWd61i+a6P7X4DlHD+sIZ2/6P8X\nA+9U6FdX5+9tYCMQxHr2XVpf95+e+azRaDSacjR2U5JGo9FoahitGDQajUZTDq0YNBqNRlMOrRg0\nGo1GUw6tGDQajUZTDq0YNBqNRlMOrRg0Go1GUw6tGDQajUZTjv8HaX5Jv9GM80YAAAAASUVORK5C\nYII=\n", 34 | "text/plain": [ 35 | "" 36 | ] 37 | }, 38 | "metadata": {}, 39 | "output_type": "display_data" 40 | } 41 | ], 42 | "source": [ 43 | "import numpy as np\n", 44 | "import tensorflow as tf\n", 45 | "import matplotlib.pyplot as plt\n", 46 | "\n", 47 | "# In this section: Exact same code from the tutorial\n", 48 | "\n", 49 | "N = 100 # number of points per class\n", 50 | "D = 2 # dimensionality\n", 51 | "K = 3 # number of classes\n", 52 | "\n", 53 | "X = np.zeros((N * K, D)) # data matrix (each row = single example)\n", 54 | "y = np.zeros(N * K, dtype='uint8') # class labels\n", 55 | "\n", 56 | "for j in xrange(K):\n", 57 | " ix = range(N * j, N * (j + 1))\n", 58 | " r = np.linspace(0.0, 1, N) # radius\n", 59 | " t = np.linspace(j * 4,(j + 1) * 4, N) + np.random.randn(N) * 0.2 # theta\n", 60 | " X[ix] = np.c_[r * np.sin(t), r * np.cos(t)]\n", 61 | " y[ix] = j\n", 62 | "\n", 63 | "\n", 64 | "# lets visualize the data:\n", 65 | "plt.scatter(X[:, 0], X[:, 1], c=y, s=40)\n", 66 | "\n", 67 | "# Generating a Y with the shape: [None, [0.0, 0.0, 1.0]]\n", 68 | "y_ = np.zeros((N * K, K), dtype='uint8')\n", 69 | "for i in xrange(N * K):\n", 70 | " for j in xrange(K):\n", 71 | " if j == y[i]:\n", 72 | " y_[i][j] = 1.0\n", 73 | "\n", 74 | "plt.show()" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": {}, 80 | "source": [ 81 | "\"The toy spiral data consists of three classes (blue, red, yellow) that are not linearly separable.\"" 82 | ] 83 | }, 84 | { 85 | "cell_type": "markdown", 86 | "metadata": {}, 87 | "source": [ 88 | "## Training a Softmax Linear Classifier" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": 2, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [ 97 | "# placeholders\n", 98 | "input_pl = tf.placeholder(tf.float32, [N * K, D])\n", 99 | "labels_pl = tf.placeholder(tf.int32, [N * K, K])\n", 100 | "\n", 101 | "# initialize parameters randomly\n", 102 | "W = tf.Variable(tf.truncated_normal([D, K], stddev=0.01), tf.float32)\n", 103 | "b = tf.Variable(tf.zeros([1, K]), tf.float32)\n", 104 | "\n", 105 | "# compute class scores for a linear classifier\n", 106 | "scores = tf.matmul(input_pl, W) + b\n", 107 | "pred = tf.nn.softmax(scores)\n", 108 | "\n", 109 | "# loss\n", 110 | "loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=labels_pl,\n", 111 | " logits=scores))\n", 112 | "\n", 113 | "# gradient descent\n", 114 | "gd = tf.train.GradientDescentOptimizer(0.5).minimize(loss)\n", 115 | "\n", 116 | "# accuracy\n", 117 | "correct_prediction = tf.equal(tf.argmax(labels_pl, 1), tf.argmax(pred,1))\n", 118 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n", 119 | "\n", 120 | "init = tf.global_variables_initializer()\n" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 3, 126 | "metadata": {}, 127 | "outputs": [ 128 | { 129 | "name": "stdout", 130 | "output_type": "stream", 131 | "text": [ 132 | "step: 0 , loss: 1.09933 , accuracy: 0.31\n", 133 | "step: 100 , loss: 0.781919 , accuracy: 0.506667\n", 134 | "step: 200 , loss: 0.765471 , accuracy: 0.503333\n", 135 | "step: 300 , loss: 0.762648 , accuracy: 0.506667\n", 136 | "step: 400 , loss: 0.76199 , accuracy: 0.503333\n", 137 | "step: 500 , loss: 0.761817 , accuracy: 0.51\n", 138 | "step: 600 , loss: 0.761769 , accuracy: 0.51\n", 139 | "step: 700 , loss: 0.761755 , accuracy: 0.51\n", 140 | "step: 800 , loss: 0.761751 , accuracy: 0.51\n", 141 | "step: 900 , loss: 0.761749 , accuracy: 0.51\n", 142 | "step: 1000 , loss: 0.761749 , accuracy: 0.51\n", 143 | "step: 1100 , loss: 0.761749 , accuracy: 0.51\n", 144 | "step: 1200 , loss: 0.761749 , accuracy: 0.51\n", 145 | "step: 1300 , loss: 0.761749 , accuracy: 0.51\n", 146 | "step: 1400 , loss: 0.761749 , accuracy: 0.51\n", 147 | "step: 1500 , loss: 0.761749 , accuracy: 0.51\n", 148 | "step: 1600 , loss: 0.761749 , accuracy: 0.51\n", 149 | "step: 1700 , loss: 0.761749 , accuracy: 0.51\n", 150 | "step: 1800 , loss: 0.761749 , accuracy: 0.51\n", 151 | "step: 1900 , loss: 0.761749 , accuracy: 0.51\n", 152 | "step: 2000 , loss: 0.761749 , accuracy: 0.51\n", 153 | "step: 2100 , loss: 0.761749 , accuracy: 0.51\n", 154 | "step: 2200 , loss: 0.761749 , accuracy: 0.51\n", 155 | "step: 2300 , loss: 0.761749 , accuracy: 0.51\n", 156 | "step: 2400 , loss: 0.761749 , accuracy: 0.51\n", 157 | "step: 2500 , loss: 0.761749 , accuracy: 0.51\n", 158 | "step: 2600 , loss: 0.761749 , accuracy: 0.51\n", 159 | "step: 2700 , loss: 0.761749 , accuracy: 0.51\n", 160 | "step: 2800 , loss: 0.761749 , accuracy: 0.51\n", 161 | "step: 2900 , loss: 0.761749 , accuracy: 0.51\n", 162 | "step: 3000 , loss: 0.761749 , accuracy: 0.51\n", 163 | "step: 3100 , loss: 0.761749 , accuracy: 0.51\n", 164 | "step: 3200 , loss: 0.761749 , accuracy: 0.51\n", 165 | "step: 3300 , loss: 0.761749 , accuracy: 0.51\n", 166 | "step: 3400 , loss: 0.761749 , accuracy: 0.51\n", 167 | "step: 3500 , loss: 0.761749 , accuracy: 0.51\n", 168 | "step: 3600 , loss: 0.761749 , accuracy: 0.51\n", 169 | "step: 3700 , loss: 0.761749 , accuracy: 0.51\n", 170 | "step: 3800 , loss: 0.761749 , accuracy: 0.51\n", 171 | "step: 3900 , loss: 0.761749 , accuracy: 0.51\n", 172 | "step: 4000 , loss: 0.761749 , accuracy: 0.51\n", 173 | "step: 4100 , loss: 0.761749 , accuracy: 0.51\n", 174 | "step: 4200 , loss: 0.761749 , accuracy: 0.51\n", 175 | "step: 4300 , loss: 0.761749 , accuracy: 0.51\n", 176 | "step: 4400 , loss: 0.761749 , accuracy: 0.51\n", 177 | "step: 4500 , loss: 0.761749 , accuracy: 0.51\n", 178 | "step: 4600 , loss: 0.761749 , accuracy: 0.51\n", 179 | "step: 4700 , loss: 0.761749 , accuracy: 0.51\n", 180 | "step: 4800 , loss: 0.761749 , accuracy: 0.51\n", 181 | "step: 4900 , loss: 0.761749 , accuracy: 0.51\n", 182 | "Done!\n", 183 | "accuracy: 0.51\n" 184 | ] 185 | } 186 | ], 187 | "source": [ 188 | "## Train\n", 189 | "w_res = 0\n", 190 | "b_res = 0\n", 191 | "\n", 192 | "with tf.Session() as sess:\n", 193 | " \n", 194 | " sess.run(init)\n", 195 | " \n", 196 | " for i in xrange(5000):\n", 197 | " \n", 198 | " _, l, acc = sess.run([gd, loss, accuracy], feed_dict={input_pl: X, labels_pl: y_})\n", 199 | " if i % 100 == 0:\n", 200 | " print 'step: ', i, ', loss: ', l, ', accuracy: ', acc\n", 201 | " \n", 202 | " print 'Done!'\n", 203 | " print 'accuracy:', accuracy.eval({input_pl: X, labels_pl: y_})\n", 204 | " \n", 205 | " \n", 206 | " # saving vars\n", 207 | " w_res = W.eval();\n", 208 | " b_res = b.eval();\n", 209 | " " 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 4, 215 | "metadata": {}, 216 | "outputs": [ 217 | { 218 | "data": { 219 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl0ZFd96Pvv75yaNY+toefBbg90e2i3wQw2xuQax4kB\nQy5DJkhoICHDS+BCwr0hL7n3XYaV5EEggF9gAblJgNix28EGgwlgAzaesLHd3XZP7tas1jzUfM5+\nf1SVVJKqpJKqVCpJv89aWl2qOlVn67S0f2dPvy3GGJRSSm0+1loXQCml1NrQAKCUUpuUBgCllNqk\nNAAopdQmpQFAKaU2KQ0ASim1SWkAUEqpTUoDgFJKbVIaAJRSapPyrHUBFuOvqTGhppa1LoZSSq0b\nY+fODhljCqo4KzoAhJpaeO1//+u1LoZSSq0bd7/n188VemxFdwFFSax1EZRSasOq6ACwo+YCAx3P\nrXUxlFJqQ6roAADwif13sveqezhhD6x1UZRSakOp6ADgs/xsDe3htvqnueP6z2trQCmlSqiiA0BG\nZ2gPAdunrQGllCqhdREAAJr927Q1oJRSJbRuAkCGtgaUUqo01l0AgNnWwBsbntHWgFJKrdC6DAAZ\nHcHdM62BP7nxL7U1oJRSy7CuAwDMtgYsEW0NKKXUMqz7AJCRaQ18UlsDSilVkA0TACDVGujMag3s\nveqetS6SUkpVrA0VADIyrYHb6p/mT278S+0WUkqpHDZkAIC5rYHMlFGllFKzNmwAyNDWgFJK5bbh\nAwDkbg3oILFSarPbFAEgI7s1oFNGlVKb3aYKAKCtAaWUyth0ASCjI7hbk8sppTa1kgQAEfmyiAyK\nSM5aVERuEJFxEXk6/fUXpThvKWhrQCm1WZWqBfAV4OYljnnYGHNF+uuvSnTektDWgFJqMypJADDG\nPASMlOKz1pKmmlZKbSblHAO4TkR+ISLfFpHL8h0kIkdE5AkReWJ4OF7G4qVoqmml1GZRrgDwFLDd\nGHMA+Hsg77JcY8wdxphDxphDTU2+MhVvoexU09oaUEptRGUJAMaYCWPMVPrx/YBXRJrLce5iaGtA\nKbWRecpxEhFpAwaMMUZEDpMKPMPlOHcpdAR3MxTr4pP778Tsv5N7x64A4FvPvIL9zpY1Lp1SSq1M\nSQKAiPwrcAPQLCLdwMcAL4Ax5gvAW4D3i0gSiABvM8aYUpy7XJr92wAYinXxa03HiDlxfvX6p/nw\nibewpffyNS6dUkotX0kCgDHm7Uu8/lngs6U411rLBAKA3sgZPrH/Tu5tO6WtAaXUurNpVwKXgq4f\nUEqtZxoASkDXDyil1iMNACWSmTGkrQGl1HqhAaDEtDWglFovNACsAl0/oJRaDzQArKLMauJP7r+T\nP7nxL7U1oJSqKBoAVln2BjR3XP953ZxeKVUxNACUyfzN6bU1oJRaaxoAykhbA0qpSqIBYA3Mbw3o\nILFSai1oAFgjuTanV0qpctIAsMa0NaCUWisaACpArtaADhIrpVabBoAKkt0a0AVkSqnVpgGgwmhr\nQClVLhoAKpSmmlZKrTYNABVOk8sppVaLBoB1QFNNK6VWgwaAdURbA0qpUipJABCRL4vIoIjkvDWV\nlM+IyCkR+YWIXFWK825GmmpaKVUqpWoBfAW4eZHX3wDsS38dAT5fovNuWpkpo5/QVNNKqRUqSQAw\nxjwEjCxyyG3A10zKo0C9iLSX4tybWaY1kEkup60BpdRyeMp0nk6gK+v77vRzfWU6/4bWEdzNUKyL\nT+6/E7P/To786P3sd7asdbFKqs43xeu3P8mu+j66Jlr57vlDjERr17pYSq1r5QoABRORI6S6iejo\nDKxxadaPZv82AHojZ7jj+s9z79gVnHrqjWtcqtLYUdPPX1/3FTxWEp/tcGXLKX5598/4q0d/g5Nj\nW9e6eEqtW+WaBdQDbMv6fmv6uQWMMXcYYw4ZYw41NfnKUriNZCMkl+usvsBvXfoAf3zlnbx268/5\nwyv/nYAnhs92APDZDkFPnD++8i7ArG1hlVrHytUCuBf4gIh8HbgWGDfGaPfPKsluDXxi/53c23Zq\n3bQGru98miMH7sMWB49luHrLCwTsJCILj63zT9NWNUL/dFP5C6rUBlCSACAi/wrcADSLSDfwMcAL\nYIz5AnA/cAtwCggD7yrFedXiMmMDt9U/jbnxaT584i1s6b18rYuVV5U3wpED9+G3kzPPBT1JzCI3\n+TniglKqQCUJAMaYty/xugF+vxTnUsuTqzXwrWdeUZGDxFe0nMJ1LbDnPp/r7h9gIh6ib7px9Qum\n1AalK4E3ifWQatoSk/eW3hiIO6nIkHBsokkvn/n5m9E2gFIrV3GzgNTqqfTWwNODe7HFWfB80rV4\npO8SBsMN7K7r49zEFr5z7hqGIvVrUEqlNg4NAJtQR3A3ALfxNL96feWMDUwmQvzT8dfzzv3fx2sl\nsS1DNOlhOhHkq8duZixWvdZFVGpD0QCwiXWG9qxaa6DWN82hLS9gieGpwX0FL9r69kvXcnJsKzfv\neIyGwCRPD+7l+11XEU7qmhClSk0DwCa3Gq2B/7LjMd512QNIeo6+Qfj6Czdwz+lXF/T+U2OdfHbs\nTUWVQSm1NB0EVkDpUk3vrOnjdy7/Dh7LxbYMtmXwWC5vv/gH7G84V+JSV6a469AXn+RcbIyhxDSO\ncde6SErlpAFAzShFqulfv+TBmTv/bJYYfu2iH5WimBVtwolxPHqBweQ0o06U3sQkx6NDxN2Fg9tK\nrTUNAGqB7FTTy2kNNAfGeVnzmZzz9kWgo3qoxCWtLMYYzsXGMMwmqHCBpHHpiU+U9DyTToze+CSD\niWkSRoOLWhkdA1A5ZaaMvlGemRkbGB1oAcg7UPzBQ9/EWmQe/4Xwxp62Oe0m8mYmGndjGGOQfKva\nCuQaw+nYCBE3iYtBgL7EJNt9dTR4gkV9ttp8NACoRWWnmmZ/6s723rErsmYMGV7V8Ry37fkxO2sH\n867aNcCdJ19TxpKXnylDYrrBxDThrECT+fd8fJwa249HtFGvCqcBQC0p0xrIyJ4x9JH6Ll63/ecE\nPIm87zcGfta/n2eG9qx2UddUSLy4eYJAQDxETBIvFl6xcx5TiBEnnPMMAow7UZo8oRV/ttp8NACo\nZcusH/h/L7mTAwELryx+5zsRD/I3T76VjZa2IeIm6I1PMu0msEWoE3/eY6MmyanoCAZDjeVnh78O\ne5G7ddcYwm4CIRU8ELAQ3DyX2sCiSfOUykUDgCqInwh1jOIlTgIvgWAbNg6WGWSxij2a9PKFX/wK\nG22+wbQT52RsdhdU1xiGTGTR92RaBxNujFPRES4KNOUcExhNROhKTGDmdSr5xCZoeZh04zk/v8bW\n/TPU8mgAUEsKMUkjQ1jp6sgmho8BpqhGxCLXpiyuEY6PbOcbL9zAsZGd5S1wGZyPjxf1/ohJciY6\nyu5Aw0wQiLlJJp0YPYnJnN08ceMQNw4WMic4WAgNngB+S/+c1fLob4xagqGB4ZnKP8PCUMU0uSr/\nqGNz9NSr+LeTN5SniKtgwolxITFN0rjU2H5avKE5ffexEky9nDJxptw4QcvL2djonMHdxQTEQ7Xt\nY8KJYotFi6eKOjt/95NS+WgAUDNskoSYwsIlSogYfmySORd2AQiGUZppYHjme9dAjCS/qBnhhD1Q\nMZlGFxNzkzgYAuLBEqE3PslQMjzTZRNNJhlOhrk40IzPsjEl6mw3wFgySr+ZYtrNP4i+oLwmyUW+\nJjqoKUk51OalAUABs908pOeW1zBOlCAjNC8aAGoYZ5pqkngQICYBYnaAX2l4ll++/hcVk2k0l7jr\ncDY2StQkkfQ4RqunigvJ6Tk/sQEcDGdjo8SNg1PC6Z6G1GDvcviKmEVUqJibpD8xxZQbx4NFq7eK\nejtQ9DoGVVk21sicWhGL5Ewfv0VqSNfCECBCiGliBBZUeZnvfSSoZpI6xghTRYwgIDOriT+5/07+\n5Ma/XHFuodVijOFkbJiISWJIDdC6GPqTU3nfEzHJklb+AoQs30zwKfQ9W7yrmxY76iZ5ITrMqBMl\nYVwiJklXfJzexOSqnleVnwYARYjpnM9bGKqZYIhWknhwEVxSlb8wO/cn9djMdAVlNPu30RnagyXC\nHdd/nr1X3bOKP0Vhwm6Ck9FhnokMkMiTpK1csymbPSHqPP6CFpBZ6TDR5q2m3jM3NXbCOHTFxnk2\nPMBz4UG64xMki0hA1xufXLCewQWGkmHNabTBlGpT+JuBT5PazfUfjTEfn/f6DcBR4Gz6qX83xvxV\nKc6tiieYRbt5XDz0sY0AYRoZwmZhJSBAgNzTIOdvTn/kR+9ftbEBx7gMJcKMpQdImz0h6mw/IkLU\nTc3Fz7dYq5yqxEunL7VHQoMdZNSJzCmVAJ2eGpq8IWLGIWlcgpZnwdoBx7i8GB3OCmaG4WSYSSfG\nxYFmrHldNlNOnOFkGNcY6j2BnN06U3mmmQrClBun0dKUExtF0QFARGzgc8DrgW7gcRG51xhzbN6h\nDxtjbi32fKr0ogQxyIIgYIAIVenvBIONhZO3w8Is0pWRvR3lHdd/nnvHruDUU28svvBZkjOVYbqj\nxkA4HqfeDrLdX0dfYuGdbS7ZE1szrZ1CQ4YAF/ub6UtMMu7G8n5+s3d2xe42Xy2ehDUz8OzBot1b\nTVP6mIDk/zMdToYX3O0bIGFcxpwojVn5gXriEwwlZ1cST8Rj9DFJ0PKSxCUgHlq8VakFZ3l+YnuD\nLebb7ErRAjgMnDLGnAEQka8DtwHzA4CqUAn8hKkixPTMdE8XcLGZYDaBm5/IIpU/hFm6b3p+a6CU\ng8QX0pkxs6suFxh1InhiwriTu0Kez2W20g+Jhy3eagYSU4RNctH3WQi7/PUEbA+77Ib0OMMIkazp\nnQIELC/19mw3jojQ4auh3Vs9271W4GDrhBPPWVW7GAYTU9Sm8wNNO/E5lT/pny+OSzwdqKZJMOJE\nqLF8TLq5P7dGp5tuKKUYA+gEurK+704/N991IvILEfm2iFxWgvOqEhqhhRGaieEngZdJ6uhnKy6z\nM05crLx3+QYYpbGgc2WPDWRSTpfCmBPNWWkZYNAJL+uzMp8TNQ4iwu5A46L3vrWWj8uCLXMqSBFh\nr7+RDm8NQfEQFA8d3hr2+htzVvAigiWyrJk2iyV/ixqH45ELjCYjnIyNFNSKMcBEjsrfQtjtb1jQ\npaTWt3JNA30K2G6MmRKRW4B7gH25DhSRI8ARgI5O3Qe2fIQwNYSpwsbBS4w6RjAI09SQwE8cX86x\nAhcYoRnD8qYnlqI1YIzhQjKcuvun9Dtvpe6kp9kbSFXkPTlmwgjQ5qvJmdvHEqHFW0WLt2rBa6XQ\n4gkxnifwQWr66rkiVy0DdHhrqNZUExtOKVoAPUB2usit6edmGGMmjDFT6cf3A14Rac71YcaYO4wx\nh4wxh5qa9BeufAx1DLOVc3TQRQuD1DBJDRNsoZc2ztNK/8xmJ6mpk+DOBI6VLUrK1RpYzpTR7vgE\nfYnJVan8MzJ97E2eECHLO+ePxsp6fi1U2T7avNWr3jN/IZl7ppha30oRAB4H9onILhHxAW8D7s0+\nQETaJN2uFZHD6fMOL/gkVXaCS4gpWuijhvGZTp7sKZ4WBi/JOesEMq/208EILXOeXYnMuoHb6p8u\neDvKhHEYmTd7ZiW2eKposoN5f4JMkjUr3aWzzVdHreWn3g6wy99Ap3dtV+Ru8Vazw1e/qkEg35RZ\ntb4V3QVkjEmKyAeAB0hNA/2yMeZ5EXlf+vUvAG8B3i8iSSACvM2Uaj29WjEfUVron5kGulgFMv+1\nzCCpjcPiQ6OFy54p9In9d3Jv26msjWcWSqVLlqI3Yokbh+2+OsLRBNH0wrAMG6HVM9t9Y4nQ4AlW\n3O5bdbYfCynpQrVsQU00tyGV5H813a1z/7znvpD1+LPAZ0txLlUqhhb6sYvsOin2/bl0BHcDczee\nyTU24CnROsZJJ5YasA000p+YYiSZalXU2n7avdV4rdVPvVAsEWGHv56zsdE5M44shAY7wIgTWfH/\nlADta9zKUatDw/omFSCSd/HXcsRYvWmBnaE9DMW68rYGQpYXj1jEi87MmWrf2GLR6audWaC13tTa\nfvYHmhlORoiZJFWWjyZPEFssqpN+BpPTJI1DfJndOQboio1T7wnQ6AliITOb4FRbPs0PtI5pANik\nrGXcD7pZi8Qk67kw1Tis7uBnplsoV2tARNjjb+B4dKioc9RYG2eygd/y0OFbeLde7wnMpJA4GR1e\nVvZRgBgOA8lpBtKDwam2l2Ah7Ak0EFyjQXBVHM0FtEnFCOTs88+e5ZPEIoKfcRoYYAtRArgIDhbj\n1DNCzolcq6IztIeA7VswU2jKyZ22oFCpKZyrm1yt0nT6arGKHDJOzQAzJHFT6TV0SG9d0gCwSTl4\nmKIaN6siSFX8whAt9LIVBy9+4tQxyhYG8BNNH2GoY4waxspa5mb/NraG9vDGhme44/rP0918jK7E\nRFGfuS/QtOl20gpZXi4KNFFvB9JJvIvjYJgocJW1qiyb6zdfzTFKM3H81DCOjUOMAOM0ksBLGz14\nic+pHuaGCqhjjAR+ooQop47gbu457WG0q47ZbD3LZyNrNn9/rQUsDzv9qTQfE06Ml2KpYD5/H+JC\nRd0EoAs31xsNAJuYYIgSYpoasitRLzE8JJasVi0MNYyVPQBMxT184enX4JiVz84RoMGjFRakBo8v\nC7Yw6cRx03fzi60uzqU/Oc20m6DTV4tfbMJuIrWZjFjU24Gcq6TV2tMAsCkZGhiiitnNTyaoTyd+\nEzzLmNnvyZEaerV960xnUTOYhNSuWm06tXGGLdbMIHGDHaAvYXFhXvK4pUy6cU5Eh/Ck1yNk2mY9\nTLLLX6+J5CqQBoBNqIlBgoTnbPReyxgGYZJ6EngLqmANEC1js797MsT/+MlBeiaDJE2+O8rFu4Q8\nCJ2+WursQFkSmxnX4Dw/SPL4IFLlw3vtVqQ+tYhMPJV5V5zKTlpLu7eGuHEYSUaYcGIkjVtQyo3s\npXSpcSXD2dgYlwdbNZlchdEAsMlYJAkxvaCKtNIDu5PU0kDuaZXZVWtmwHgyK130akq6wh//4BCj\nUW86IUU++SsYAXb7GwnZpe33N7Ek2NaCCt0kHMJ/+xPc7nGIOWBB/N7j6cII9sXNBN55BVZraqWx\nSbo4Jy5gwgnsfU1YDWu72lhE8IuHdl8N7dRgjKErPrFg85pCTTixBbuZqbWlAWCTSW39YeVcwSu4\nVDOJn9iCajSVO96Ll2R67CDAGM0kV3kdQMajfc1Ek/Yilf/id/42wk5/fdGVvzEGMxQGS3BHI8T+\n+RncvkkQ8Bxsx//Og1g1qa6O+HdP4Z4fg0T6WrtzPgjnxAWm//cPqf7r1+NemCb8mZ+Ck65aky7e\nV+/A/7YDFbPQSkTY7q+j3VTTG5/Mm347H2cVE/apldEAsMkk8eTt3jEIIabmdA1lixNkoIxz/7P1\nTwdJuIttR5PvNZd6fxh/R4yageK2oUyeHCL65acwEzEwLiTnXqfkU70kjw0S/KPr8OxpJPGTc7OV\nf75ixx1iPzhD4sFTEJk79pL4yXnsHfV4r9tRVLlLzSs27d4axpxowe8xGKo30IK7jaIyOyFVyQkO\ntYzQQh8mvbl7NgMk8Cy6Qnjhu8pnd90krsm3dC1/YLAxtIQm+d22h5edajqbOzhN5NOPYIbDkHAW\nVP4zokkin3yI6b/9MWYk9x7JcyRcks/0gZvj8+IO8e+eWlF5V5vPstnlb5jZrH4xAjTawU233mI9\n0ACwCQgO7fRQyzg+Etjp5V/Zq34BfCTwksjbrA8xxRa68bC8NAKl8LLmsTwBYPEcpg42J0fb+dTP\nbmVHbKTgVNPzxf/zNDgFBkAD7omh3JX6fLYgAc9s18/8j5qs3AVWtbafy4Ot7PY30GKHcv5P+MVm\nm6+Ores0v9JGpyF5E0jl+U/OifaZAJDvntpldnN0yfryEWcLvfSwjXLeP8QcO534eWX94XHXw/95\n/rVcseVrBaWans/tHs9bSRfFEqztdbgv5tkeI+TF6ZskfvQ4yecHwLbwHt6K75cvxoxGkJBvZhB5\nLVgi1Nh+BGFo3rabFrDL3wDAS7GxmXUBzZ4QzZ5QxYxtbGYaADaB1Gbvy5MJELmeT20iEy5oE/hS\nqfImaQzGGYqsfBbJcNRPtX0xdf7EkqmmM9yBKWL3HMM5M7Li8y7OkPz+mfyvDkwT/tj3s55xSPzo\nLIkfnQW/BxwXa0s1wd+7Fqt5bQKBMYZz8bEFvy8ucDo6goPBTb/qGIfexCQRN8l2f13Zy6rm0i6g\nTSDfXXO++y+Z97XwdUMV4+QOEatDBN538EX8dnELzwLp9xeyDaU7OMX0//ohyad68/f5Fythln8Z\nM/120SQkXNyeCcKfehhTaBdViUVNEidPMrgE7kzln2GAUSdCzC3VVkJqpTQAbAJT1M5J+pax0ipN\ngAAxGrlQVLmW68btA/z3lz/L9popbHFZzk/gsRyu67iA3zNbSXYEd7M1tCfvNpSxe0+kKtlKT3Rp\nwESSOM8PrnVJlmXKLS6TqyqeBoBNYJoaogRx0xNA53+thJDqWvJQ3j/ia9uH2N84gS2FjAbM/pSN\ngTh/euhYzqPypZpOPt1bwpKvsqSLO7Q2G7cHZPk5RQXB1upnzen/wKYgDLGFQdqZoJ5xGuhlGwN0\nksBb1A1ugAKmOpbQl57dy4+6txB37QIGhGc7skYjPp7ob8I1cGKklp8PNhBJzCaTm59q+kLfzyBe\nRJdKQwDP63aX7y/MFqyOtZlpIyIzmUULfg+pWURqbZVkEFhEbgY+TWpT+H80xnx83uuSfv0WIAz8\ntjHmqVKcWxVKiBMgnpW7xwH62UaAMM0MpI+aW61m7y+bi1vGewjHhaOnthJzlp8FNGFsPvX4pXz8\nsUtJuhYeMYjAew6c5PaLuhiLerkQCdBedRHHRidwTsDAO3Zz9eXD/MHNz3FR0ziWcRkf9vDdf2nk\n659pIZnIXQ5pCRH66A1YIR/x3Y3EvvpUemqVSX15LAjYMFmi6bS2YDWFsC9em0V6ADW2nzrLz7ib\ne9pq9qwzQdjlqydpXGxEM4WuITFF7uQjIjbwIvB6oBt4HHi7MeZY1jG3AH9AKgBcC3zaGHPtUp99\n4GCdufeBVxRVPlWY1MyeaQKECRBOdxYJDp70FNKF99suQg87lsjNUzrTCZs33nMDTt5EcEvJFc4M\nHnFJGsFnuSRcO3WUgd/a9ywfPvgIvnn1vDGpr3//YjNf+n86cJ3U5zV3JnjTZ31ctHuM7qlW7j97\nLf3hRkzcwTk5DAL2vibEazP9V/+J213cZjapXd8t7ANbCP7GlVDlxQxMgQjSWlX2aZaOcTkZHSFu\nHNz070um0s+uZTKb1WcGh2ssP9v9dXg0EJTE3e/59SeNMYcKObYULYDDwCljzBkAEfk6cBuQ3eF6\nG/A1k4o2j4pIvYi0G2P6SnB+VQIGi2lq0nsDGLwk0ts/egGXVvrwpXMEZULBEFvKVvkDhDwOftsl\nnFzpOXPPaUqm9xWIu7OfW+2L8aEDP1tQ+UNqRpII3P6+Id783iGMSbVOPB4wRrAtw6VN57lx28/5\nxBNv49mh3Xgua53zGe7oEl1n+WrPeccE3nMIz8E2nNMjRL/0JGY6NSYjNX6C7zmEvbtx8fOUkC0W\nFweamHTjTDtxRpMR4gvmAKV+HCfr2Qk3xqnoCBcHmnRtQJmVIgB0Al1Z33eTustf6phOQANARRIS\neNNpIQxgMUgnXmIEiOBiE6aqrJU/pCrdm7b3cu+Zbax0F7BCXd20dMqITCAAsDOBQlIVm8dy8Vgu\nv3/gLm7+53cvaD/VtoTwTY8v+EzXAqetmmRnDbErthD6/lm8Z8ZzL9hzIfzlJzEeCyvuIMnZMQsz\nHGbqb3/M2B9cg1tb/r72SCBJ3OcW/N8UdZOcujCMx9FWwK728gXtilsIJiJHgCMAHZ2aOrb8DHWM\nUMNs90Rms5gEfhL4Fs0XtNref8VJvn22k8SC3cCyb5eLDw4xxy5JjKn2RNk/0c+Z/rl/1Ml9nXi7\nJpGsufvGEkx9iOgNl6bSRYchds0+vOefgmSu7K0gcQcTz702QpKGqge6iR/cWvwPsgwGGL+K2ehY\nCNdgDSXwl3dmccWZ2G7xwkuD+PzlqZpLcZYeYFvW91vTzy33GACMMXcAd0BqDKAE5VPLUM8w1Uwu\n2CwGDAl8NDCMnd4FLEwVIzRjWPnWjMvl97h88oan+POHrsSQ2ifAtmBX3SRNgRiP9LbkXPOwXI8P\ntRNNegjYxU1z9Xtd3nfFs9z5tVfNeyXE1IE99L9wnkQkhiDUbmmg7eLt2BfmVvYTl+6i97kzmDy5\nhfIu6HMNTQPTbBss765trsD91vJ+J2wXLu13aRne5H/ygw6PtdpQpp32ShEAHgf2icguUpX624B3\nzDvmXuAD6fGBa4Fx7f+vPJn9AOang85sFpPqDJoVYhovCfrpZH41ZOEABncVGpkHW8a487Yf8dOe\nVsbjXg40j7K3IbW95Z0vbOcfn9tLwsk1bF04x1i87yc389Xr/wOfZZZ1M5tNBK599Sn+8/6DjAzN\n3YKyurmOvc0vw006iCWIlbv7o3ZLA8nENgaOn1/euS0hWFve/ZoBLAM1kzBZ4KxUcQ3+ODRv9so/\n7XCRAftnyzi26L9OY0xSRD4APEBqGuiXjTHPi8j70q9/Abif1AygU6Smgb6r2POq0lsqy+f86kkA\nL3Ha6GKYNhL48BCniUF86QViSbwM0zJn+mkpBD0ur9vRv+D5t1x8nqvbhvnO2Q4e7W2ma6qKlfbl\nPDHUxnX/8RscuegZ3rTzBC3B2IoCgQEuv/I8D33vspyvW57F75YTsTgDJ7oWPSYXEaF+a8uy31cK\nlx1zefyQheNZ5IIZg7hQP2646qlStNvUcpXk9swYcz+pSj77uS9kPTbA75fiXGr1OItsFrNY3iAv\nSVrpZZRGmtLbSWaO95KglT762JqeUbT6dtVN8/4rTvL+K05y+9FXMxrLFXwyq4QzJV24+qGzepiB\n7maeuv+mIcBUAAAgAElEQVQQx2OHuOlXfs4tb36KPDfq+Rlw825ms7Tx3uHUvNNlENtixzX78fjK\nc83nax42XPuYw4mLLSZqBU8CHE9qkFvS6Y/2nnLZ2msIFr6vjCqxihsEVmsnM7tn/obxS0nN63Zp\nYihv8rgaxhlbg93E3nPgFJ95aj9RZ/ZX3Wc7XNkywo07+hmJ+OmZCvKf59tIuhaWGG7b283Nex7A\ntgxHvvR+gum1TQ/+x5WcO93K2979MI3N0wW3BkTg+C+20tA0ydREkERieX92yfjyF4xZlkWgpvzd\nP9kaR+G6R2fHMxI2nNwrjDQKYmC6ShhuhPZ+g627Ra4JDQBqjhFaaGSQEOFlNckX35YFqpkgQjWx\nEncFLeXmXX3EHJsvP7cnNbPHwOu29/OHV56Ykxjuj656gcm4h/Hki9iWy9Gxgxz74NXc5J1b+UbD\nPsZGqmlsXjrvjjGQiAtnT7by3/7n3QCIGH7yg/38xzcO47qFNSVC9TWM9QwtK9unwWCMqZh59TEf\nPPwqm7gXXI+AMYw2Cb2dhuOXwCt/6hAqb1YRhQYANY/BYpg2hD6CRErWL2sBrfTSTweJMgeB2/Z2\nc+vubsZiPqq9yTkVf4ZtGZJyBttyOfKl93PTEwn2zOs96dw+xAf+/D78/gIH6QycP9vKzr0X8Adm\nUx9f99oT2LbLv/+f6wr6mJqWerwBH4lwjEJX7rtJl7GeIRrWaAxgvhMXW8T8qamuwMwUUdcWYmJ4\n+qA1p7WgykNXXaicRmjBmdk8snQZkdvoJchUiT6tcLYFTcF4zso/I+rEOTp2kF2P5M5n88tveRKf\nbxkzNAR27RucU/kD+P0OL7/+RfyBwqaYiiXsvGY/9VubwSowJBvD0NnKmWjX3yazlf98ljDaIMT1\ndrTsNAConFw89LGNUZqZopoYPpaXgX+hTG7OJi5gUzmbgfRGztAdPg3Aw3/3cvZ4c/9Z7No3sOxZ\nQJaV+4q5jkVDU+Hpm22vh7b9O6htbSj4Pcno+sm3Lwbc8i0nUWkac1Vec/MDpZ6pZox6Rou8czCE\nmGSSwiuz1dITPo0BfvN77+HWfzccXmQBTiJhE1xiqmw2kfyTd2zbZWxk+Vs4RsaW0XoSIRGN4w34\nln2eUmvrM3RtI28LJhAFf+6Gl1pF2gJQyyBMUc8kdbjInA1msi212YwFa5pOYr5M5b+UZx7ftdzZ\nmDhJIRGfe2sbj9k8+chuopHlV8we/zKmdRrD6Z88y8i5pfMarbb9L7oEoyBO+gJmLqRrsJOGA8/q\nOoC1oAFALZMwThO9bOcCbThYC/5wMxl53DyrClyEGMEFz1skCTCNjyirvQ/jUKyL7vTdf+uPl/4z\naG6doLZ++TtuGSM8+ehuEnGbSNhLIm7z1KO7+bevvXIFpYamne15VwznPL9rGDzVTWxqbafY+ONw\n/UMOlx1zaRx2CYYhGDZ09Ble+VNHVwGvEe0CUiviYhMjuMh6AWGcBuoYnbOXgJvONBolCBh8xPCQ\nIMg0ISK4CILBwWaQ9lVbPBZ3E3zo+O1c85mGRbt9AOobp/iTvzxKIBCfGQMwZulcZ8mE8LOHL+LO\nr72Se/7l5TQ0TTE2UkUkvPLsnDWt9TTtamPodOHbVRrXMNY7xJaLti198CryOLDzvGHnea3sK4UG\nAFWUBD785O68jRGin2rqGCFIGIMwTTUTNGCl9xhIpZ8wMwPE6e1YkPTq4j62U+rUz5l+/8IY3vzO\nRwgE4mTnN1uq8jcGjj2zjbv+KbWhUTTio6+7NGl+W3Z3UNVYw/knX8ybIG4+N1nehHBqfdAAoIoy\nRiMt9M9pCbhAjAAJUn3cI7QueF8zvXiJL5piwoNDkGkiVJekrEOxLqJOamZMZq7/YkRc3v2H3+ey\nK84vO/2D68K/fuk1mBXvXpbvc13cpEOwrpr2y3bR9/zZgoJAdXNdScuhNgYNAKooMYJcYAsNDOMl\ngUGYooYx8t/tpvr6owXd12dWEBfLQwKvifNnL/4Kvk+2LFjhm8ttb39sRZW/MfDSydaiunrmcx2H\n/hNdTPQPpxaDZaYYFdAAEEuoblnepu1qc9AAoIqW6uoJUehmLHZ6x9h8iecyMonmimGRpIUBPCZG\nqx/uvfx+Hv+dfdz5tVcuUlbDrx/5IVdfd2bFaaDvu6ugLVkL1v3MaaZHJmYr/GVMR6rd0ljWlBAx\nH5zfJkzUCrUThu1dqXTPqvJoAFAlVFglk1jGr10y72YzZmZjGifv5xla6cdjYliSGmXweFxeeeML\nvOyq8/zDJ9/AQO/8tQiGP/2/72HrjpEVVf6uCxcGajnzYtvy35xHbDpKeHRyRROjxLZo3tVesrIs\nZawWHn25jWul0jwMtBpO74HrHnGonSxbMVSBdBqoKjNDFZO4BeQbdRGmWNh37SdCO12000UHXbTR\nRT0XaOM8rfSkU00YxqJnsWcq/1kiUFsf4Y8++i0Cwdlb06bWCf7rux9eUeVvDMSiNoN99fzDJ26h\nlAPXsanIiu7gg3VV7Dh0Mb6q8uReMsDPr7RJegXXTuf68QhJD/z8Cl3mW4m0BaDKyNBCP36iM9V/\nJgg4WOnFYbMV3RQ1hJldLWvhUMMYtczdJN1LAi+J9HNJ/AwyTRURgYjjpdqzsBtJBIKhOL/9+w8y\nNRnk4DUvYdvunI3ec/4Ei0z//Nwnb+H86RaKrfyT8QRTF8YwrqG6uQ5f0L+sBWhiCVsu3l72RHCR\nYOprYYGE6SpD1A8BXe1bUTQAqLIJEJlT+cPsojELN722IECUAFFCc9YABJimmYGZ6aLZcn0fMlO4\nBjySv+YUCy6+PJUwrZAb7HyVsDFw7nQL508vnO20XKPdFxh44TyZKzPwYhdNO9vwVQWITYYL+gzj\nGhKR8te0RlI5fRZ7XVUW7QJSZRMknHPgN7WhTGbaZ5ggkazK31DHMC0MYFH4vbUgPP6TKzjzXBvu\nIlknlrrjn+/fvnodibiNSX+m4wixqJdvfmVlK3uzxaYiDLxwHuMajOum/zUMvzRA8652LE/hf662\nr/z3dqEw+PIM9gYjqXw/qrJoC0CVjVvA/YaFIUAELzES+Klikpp5XT6FetOhFwhVr2wf3/mMgScf\n2c1Pf3AJPeeauOENz9LcMsnZU6388DsvW7Dh+0qM9Q7lnNNvXJeJ/hFsrxc3Wdid/XjvME07SjcQ\nXQgBrnja4bHDqUFgYwniGCwDVzzjaK6fClRUABCRRuAbwE7gJeDXjDGjOY57CZgEHCBpjCntHDm1\nLkxTna7Ml+rQNgSIksBPHWMrbqZW10SRItq4mf5+1xGmJgPc+43DAJw708pXP/e6lX9wHk48/5RX\nJ5HAXx0ouGsnHo4SD0fxhcq7+U7TaCrnz9mdFhO1hrpx2HnO1d2+KlSxLYCPAN83xnxcRD6S/v7D\neY59rTFmqMjzqXUsiW8mP5DMpHzIRXDS1b69RJ4eWDgwa9KpSIup/AGSCYszL7Zx4rlOHv3RxSVd\n2JVLdXMdE4OjC7d+FLC8nmWlghYRnMTa7LkQisBlxysn26vKr9gAcBtwQ/rxV4Efkj8AqArVd9Li\nwf/PS++LFjsPOrzudxM0b1udhF2T1BOhihDT+IgQIJLzDj+Snv2TwIuP/KuIXAMjjqFKPPiMCwLh\nKT81dYV1OGcGdud3EzmO8MiPLi5428ZSqGmtx3fWTzwcne0KEkEsYXpoPHfKhzybDhjAX51rSo5S\ns4oNAFuMMZl95/qBLXmOM8CDIuIAXzTG3FHkeVWJPPcDm8//bpBkPNXVcfbnNg/9s48j/xDhwOud\nkvSfz5fEywT1QB2NDBGat0XkBdox6bAwRiPNDORcNeACEdfwd4Mv49iHr+Jgc4zpqQBvfuejHH7V\ni9ieue+Z31JIJi1OHU/1k++9pB9PervIZNIiGvbx4LcOlu6HziM8OsmFUz3EpqN4gz6ad7UTnQwz\n3jcMBqpb6xnvG85Z+Ytt0X7pDvqef2nO62JZtOzpwLJ17r1a3JIBQEQeBHKNJn00+xtjjBHJOwns\nVcaYHhFpBb4nIieMMQ/lOd8R4AhAR2d5+y83G9eBL/9RgHhktlZ0EoKTgM++K0hDu+E3PxXl8teu\nViZJYYQWxqknQBQXiwhBsienRQkxTAsNjMxsI+lg4WLTE48zkISjd72Km6wEYyOpmUM/+u5lXP2K\nU9ieueVOxG0uDNbS1jFGNOLlpz/Yz3fuuQrXsbjmVSd59U3HCATjHHt6G9+/7yAT46FV+rlTJgZG\n6X3uLCY9TclJJOl9/iVa93Wy7zWp4BOfjqaCQR6Bmiq2XblvNogEfDTvbqd2S2kyj6qNbckAYIy5\nKd9rIjIgIu3GmD4RaQcG83xGT/rfQRG5GzgM5AwA6dbBHQAHDtZp4vBV1PuCNafyn8MIo73C5383\nyH+7O8yOA6vXp+vgZXqRvP8RqolQhWCyEkdDX/I0H/zYe7lpcG5it/6eBr76Dzfyjvc8hG2ndjKO\nRb185XM3cvZk7pkxjz18EY89fFGpfqQlGWPSUz7nXlfjulw41UN9ZwuWbaWmcy6yAMHj8+CvClB1\nuLYMpVYbTbFdQPcCvwV8PP3v0fkHiEgVYBljJtOPfwn4qyLPq0rA9i6dUywRhfs+7eP3vrTWk7hl\nZluZ3sgZXGM4OnaQpp4E5NjE/fmnt/M//uAddO4YxnWE3q4mTBlWIrlJh+nRSUSEUEMNlp17JNqJ\nJ0nG82ckjU1HCNZWYXs9VDXVpcYAsv6zRISqpjpsr87kVitX7G/Px4FvisjvAOeAXwMQkQ7gH40x\nt5AaF7g7ncvEA/yLMeY7RZ5XlUDbXpeaRsNwOH/FaIzQc6Jy1gt2h08Ds/v47llkwzDXteg6W750\nCIMnuxl+qR9IVdBGoPPy3dRumZ9wLjVNM99sWGPMnIq947KddD19iujENGIJxjUEaqvouGznavwY\nahMpKgAYY4aBBROijTG9wC3px2eA1R9NU8smAu/9YoS//a8hErFU///Cgwzt+ypjSl9v5AxHxw5y\n7INXc6u3snoH+55/ibHe2VnOJp2rv+fZMwRqLscXSk0hTURidP/iDNGJ/PsL+6uC+IKzU05tr4ed\n1+wnNhWZmduvM3xUKWj7cZPbdaXL//rpNA/9k4f7Pu0nGYfs2fm+ALzhDwpL5m4M/PhfPXz77/2M\nDwrt+1ze9JEYl91Q/CByptvnx317uSZHl89aik1H51T+cxhD//GXCNRVE6wL0Xfs3KILvgDaLtme\n83l/dVArflVSGgAUtc2GW/+vBNfcluSL7wvSf9LCssEbMPz6J2LsuTp/CyAegae+7WG426L7uMUz\nD3hIRFMB5NwvbD737iDv+XyEK//LyoPA/G6fSjM5uGDx+xzTI5NMj0zOdN8sRmxLp2+qstEAoGZs\n2W34i++GGe4W4hFhy253zkbo8/WcsPjU7UHiUSERJd2nPbcbKREVvvkXAa74pekFawqMgfPPWkSn\nhZ0HHfw5Zl12h09zdOwgD//dy7l1sHI2NncdBxFBLGvJO/qMQvbuFZGZ7iKlVpsGALVA09Z0LoVF\nGAOf/e0g06O5EjTPNdInxKYhkLW17/nnLD73riDTY4JlgZOE2/97jBvfNTszZijWBcB9d123YKrn\nWpkemaD/xPmZQdxAbYjoRGFpmpcilkXrvq3IcjchVmqFNACoFek5bjHat3TlD2BZ8Ivve3jhJzY1\nzYZrfjXB37w1RHicOe+/63/6advtcun1zky3z4eO385NTxRe+RvXJRlPYns9eadgroTruoz3DtN/\n4vycubOlqvy9IR9b9m2nplU3b1flowFArUh0SnAL6JGxfQZfyPDVPw0QDwtiGe7/e1+6O2hu8IhH\nhDs/ZXj3NanK/4Mfey+H53X7RCfCTAwMYwzUtjYQqKtKTbk0huGz/Qy/1Jeunw11Hc20XbxtyTtq\n4xqmRyZwEklCDTV4A745r4/3DdN//Byu665oX95C7H3lgdX5YKUWoQFArci2y51FFpEZbC/YHqhq\nNEwMyswUU+Om/81Tk3Y9FeKjf/Qaqqb2cNg3t/IfeLGL0a4LM6tnR7suUNvWSPulOxg+18/Q2b45\nK2vHe4dwHZfOy3fl/TkiE9N0PXUytQELgEkHjv3bEREi49P0HTu3YMVuKeliLrVWtLNRrYg/BDte\n5pDvlvhV74jz4aNhYpN51hfkJXgfnMQdGJnzbHhsck7lD+mNUgZGmBoaZ/hsf460CobJgZG8K25d\nx+X8ky/iJJK4jotxUrtwjfcOM9ZzAYDhcws/dzlEBG/Ijzfoy731mEDDtvLu3atUhgYAtWK7800P\nFXCTwiP/5iE8ka/yz+wGnOOVmMHTMzet1HjvcM6K2DjugsAw53VjGO2+MCeNQsbU0HjuVMquS/+J\n84x2X0gN9hbBGEPtlkY6Lt+Vs/63bJumneXduUupDA0AasUmh/MMAhuh9wWLH33Nl/v1tFSFmDsI\nJGNz79rdxe7CjUHy5a02MHy2j+5nTi0IAk48kTMwZN438EIXghS+EXEOYluE6qu5cKo393aPxpRs\nIFmp5dIAoFZs37UOvuDCSs3rN1TVmyV35PL4wRPIXbE78VRq5EwFHaytynmc2Ba17Y00bG/NO9ib\nGuSdTN3xZwnWV7NY7W5cl9h07r0Mba+HrVfuzfveDF8oQFVTLbGpPJW8MUQndb9EtTY0AKgVu+6t\nCQLVBrGys1QavAG4/MbkogHAFzT80p91s+3KMG6eMdCJ/pF0N0yMC6d6cn9OyE9tWyMtezqp72zO\nez7jpDZWB4hNReh6+hTnnniBJaf1CHj9vgVPN+1qo6a5ftEZRqHGGnYeuhgRwRNY+BmQHiMI5n5N\nqdWmAUCtWKAa/vy+MAduSmLZqUCw/9UOf37fNNe+OYnJeXNvaNrm8HtfinDoncM8+6GD1DXmnvtu\nXJfR8wMMne3Dnb9PLqnKs2VPJ5ZlISLUb11iMFWE2HSUlx47ztSFMdykk+qWWaSLxziGRGzhIPKF\nUz0k40maduXuvxdL2HpgD5YntZS6eVdHzmBheWyqm+oWL7dSq0Tnn6miNG01fOArUTJd9Nl13O98\nNso//n4ADCRi4K+CbZc6vO2rx7B9qTeYu1qoakwylWfPWyeRJDw6mfPcxhiik2FqWlIBZDxfQra0\n+vYmLpzuWRhMDLNBYH4R8uy5iwhTF8Zo3tlGZGyK8OgkxjWIlRoX6Tywe870ztotDcTDUYbO9CJi\nYYzBG/Sx7Yq96fcoVX4aAFRJ5OoJueoNSf73o9M8do+HyWFh/3UONVe/iFizid0O4xCd6YtfWNEG\n66tx4kkSkdiC18Sy8PhmNwRwk/lXptleD6HGGnqeO5PzdRHB4/elBp/T9bHH58VNOjiJHLl+jMG4\nLmJZbLtyH9HxaaZHJ7G9HmpaG/D4Fv5pNe9qp3FbK9HJMLbXg68qkH/wWqky0ACgVlVdq+H1R2a7\nULrDcORL7+fWrPQOgZoQVY01TI9MzN3c3LZo2buV+HSUnmfPLLwTF+ZstlLdXM94/whm/h2+CE27\n2hARbI8nZ/K2VHdSB/7qINHJMN6An1BDNf3Hz6VSPedoBFQ11828N1hfnR5UXpzlsQk11Cx5nFLl\noGMAqiyGYl0z+X1qzy/sz996cA/Nu9qxfV7EtqhqrGXnNfvxBX1MDIwsqPzFFrZdsXdON0t1Sx2B\nmtCcLhWxBG/AR0NnanygcZHZQjWt9QRqQtR3NFPVWDMzxmB7vXPu1MW2aNjWOmfTFqXWI20BqFWX\nndjtms80cJiFXTViWTTv7qB5d8fc9z5zmqmhsXkHCw1bW6lqrJ33tLD96osY7RpkrGcIjKGmrZGm\n7VtmBmPrt7YQmZhOzwiSmcVZ267clzMPv8fvZfcrLmO0a4CpoXFsr4eGba1UN+vArVr/NACoVdUb\nSfW550rstpRkLMHU0NjCweH06t7WvZ0L7uYty6JpRxtNO/LMzhGh47JdNO9qJzw6ieXxUN1ct2jm\nUI/PQ8ueTlr2dC6r/EpVOg0AatUMxbpwjeFDx29fduUPkIjGEMvC5Eo7agxO0sHjW1kvpi8UwBcK\nrOi9Sm0URY0BiMhbReR5EXFF5NAix90sIi+IyCkR+Ugx51Trw1Csi6gT50PHb6fxb1bWXeIN+vPm\n+MkM6CqlVq7YQeDngDcDD+U7QERs4HPAG4BLgbeLyKVFnldVuLib4OjYQczXm9mzwk3cPT4vtVsa\nF8yTF8uiaWe7zp9XqkhFBQBjzHFjzAtLHHYYOGWMOWOMiQNfB24r5ryqsnWHT+Maw12PH15R10+2\n9kt3UN/RjFip/Xct26Jp55a8K3CVUoUrRxu6E+jK+r4buDbfwSJyBDgC0NGpfbTrSW/kDK4xHB07\nyLEPXs2t3uK3zxLLou2SHbRetBUnnsTj9+qeuUqVyJIBQEQeBHLdbn3UGHO01AUyxtwB3AFw4GDd\nKm3Ap1aDMWZmhe8e79LHL4dl21jBhdM0lVIrt2QAMMbcVOQ5eoBtWd9vTT+nNpDMXH+l1PpRji6g\nx4F9IrKLVMX/NuAdZTivKoNMtw+k5vrfWmSfv1KqfIqdBvomEekGXgHcJyIPpJ/vEJH7AYwxSeAD\nwAPAceCbxpjniyu2qhRuutvnm+//3aIHfJVS5VVUC8AYczdwd47ne4Fbsr6/H7i/mHOpytOT7vZp\n/bEFOdI7KKUqm66kUcuWWeQFqcyeNw0u3DBFKVX5NACoZcn0+WcSu92EVv5KrVc6oVoti8mq/JVS\n65u2AFRBsrt9xn/aCnrnr9S6pwFAFSyzwvcmr1b+Sm0EGgDUkrLn+iulNg4NAGpRmRW+q5XiQSm1\ndjQAqLx6wqdLmthNKVVZdBaQyqk3cgYD3PX44RXn81dKVTZtAagF5nf7KKU2Jg0Aakb2VE9N7KbU\nxqcBQM1xdOwgD//dyzWxm1KbgAYABaQGfA3w4769WvkrtUloANjkFiR2e0IXeSm1WWgA2ORiTlwT\nuym1Sen8vk0s0+0z0N241kVRSq0BbQFsUkOxLgypbp9btdtHqU1JWwCbVNSJc3TsILseia11UZRS\na0RbAJtMJrHbzHRPr874UWqzKnZT+LeKyPMi4orIoUWOe0lEnhWRp0XkiWLOqVauJ3x6ZhP32J9d\no9M9ldrkim0BPAe8GfhiAce+1hgzVOT5VBG0z18pla2oAGCMOQ4gIqUpjVoV2fn8a8+7a1wapVSl\nKNcYgAEeFBEH+KIx5o58B4rIEeAIQEdnoEzF27jmJ3Y7jHb7KKVSlgwAIvIg0JbjpY8aY44WeJ5X\nGWN6RKQV+J6InDDGPJTrwHRwuAPgwME6TUVZhKFYF6CJ3ZRSuS0ZAIwxNxV7EmNMT/rfQRG5GzgM\n5AwAqjQyKR6Ojh2kqScBmtNfKTXPqtcKIlIlIjWZx8AvkRo8VqukJ3yaaDrFQ+zPrtENXZRSORU7\nDfRNItINvAK4T0QeSD/fISL3pw/bAvxYRJ4BHgPuM8Z8p5jzqvwy6R0++LH3cs1nGta6OEqpClbs\nLKC7gbtzPN8L3JJ+fAY4WMx5VOEM6QFf7fNXSi1BVwJvENkrfC/5hqN9/kqpJWkA2EA+dPx2Gv+m\nTvv8lVIF0ZpiA+hOp3gY6G7Uyl8pVTBtAaxj8xO7ab+/Umo5NACsY5nEbrrCVym1EtpfsE5lUjwE\nzmsMV0qtjNYe60x2YrcPfuy93DSomT2VUiujAWAdGYp1abePUqpktAtoHYmlc/tc8g2t+JVSxdMW\nwDqQSewGcN9d13GTV7t9lFLF0wBQ4TK5fT50/Hau+UwDN6GVv1KqNDQArAOZPn+llColHQOoYN3p\nu3+d6qmUWg1as1So3sgZjo4d5NgHr9Y+f6XUqtAWQAXKzPX/cd9eze2jlFo12gKoMPM3cVdKqdWi\nAaCCdIdPz3T73OrVyl8ptbq0f6FCDMW6gNQ8f+32UUqVg7YAKkCmz/9Dx2/npid0wFcpVR5iTOV2\nNYjIBeDcWpcDaAaG1roQBdKyrg4t6+pZT+VdD2XdYYxpKeTAig4AlUJEnjDGHFrrchRCy7o6tKyr\nZz2Vdz2VtRDa2ayUUpuUBgCllNqkNAAU5o61LsAyaFlXh5Z19ayn8q6nsi5JxwCUUmqT0haAUkpt\nUhoAchCRt4rI8yLiikjeEX8ReUlEnhWRp0XkiXKWMasMhZb1ZhF5QUROichHylnGrDI0isj3RORk\n+t+GPMet2XVd6jpJymfSr/9CRK4qZ/nmlWWpst4gIuPp6/i0iPzFWpQzXZYvi8igiDyX5/VKuq5L\nlbVirmvRjDH6Ne8LuAS4GPghcGiR414Cmiu9rIANnAZ2Az7gGeDSNSjrJ4GPpB9/BPhEJV3XQq4T\ncAvwbUCAlwM/W6P/90LKegPwrbUoX47yvga4Cnguz+sVcV0LLGvFXNdiv7QFkIMx5rgx5oW1Lkch\nCizrYeCUMeaMMSYOfB24bfVLt8BtwFfTj78KvHENyrCYQq7TbcDXTMqjQL2ItJe7oFTO/2lBjDEP\nASOLHFIp17WQsm4YGgCKY4AHReRJETmy1oVZRCfQlfV9d/q5cttijOlLP+4HtuQ5bq2uayHXqVKu\nZaHluC7dpfJtEbmsPEVbkUq5roVaL9d1UZs2F5CIPAi05Xjpo8aYowV+zKuMMT0i0gp8T0ROpO8e\nSqpEZS2Lxcqa/Y0xxohIviloZbmum8BTwHZjzJSI3ALcA+xb4zJtBBvmum7aAGCMuakEn9GT/ndQ\nRO4m1SwveUVVgrL2ANuyvt+afq7kFiuriAyISLsxpi/dvB/M8xllua45FHKdynYtl7BkOYwxE1mP\n7xeRfxCRZmNMJeayqZTruqR1dl0XpV1AKyQiVSJSk3kM/BKQc9ZABXgc2Cciu0TEB7wNuHcNynEv\n8Fvpx78FLGi9rPF1LeQ63Qv8ZnrWysuB8axurXJasqwi0iYikn58mNTf+3DZS1qYSrmuS1pn13Vx\na3Yq7/8AAACxSURBVD0KXYlfwJtI9UHGgAHggfTzHcD96ce7Sc28eAZ4nlR3TEWWNf39LcCLpGaO\nrFVZm4DvAyeBB4HGSruuua4T8D7gfenHAnwu/fqzLDJLrALK+oH0NXwGeBS4bg3L+q9AH5BI/77+\nTgVf16XKWjHXtdgvXQmslFKblHYBKaXUJqUBQCmlNikNAEoptUlpAFBKqU1KA4BSSm1SGgCUUmqT\n0gCglFKblAYApZTapP5/oZU6q3VM4owAAAAASUVORK5CYII=\n", 220 | "text/plain": [ 221 | "" 222 | ] 223 | }, 224 | "metadata": {}, 225 | "output_type": "display_data" 226 | } 227 | ], 228 | "source": [ 229 | "# plot the resulting classifier\n", 230 | "h = 0.02\n", 231 | "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", 232 | "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", 233 | "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n", 234 | " np.arange(y_min, y_max, h))\n", 235 | "Z = np.dot(np.c_[xx.ravel(), yy.ravel()], w_res) + b_res\n", 236 | "Z = np.argmax(Z, axis=1)\n", 237 | "Z = Z.reshape(xx.shape)\n", 238 | "fig = plt.figure()\n", 239 | "plt.contourf(xx, yy, Z, alpha=0.8)\n", 240 | "plt.scatter(X[:, 0], X[:, 1], c=y, s=40)\n", 241 | "plt.show()" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "Okay... that was disappointing, a random prediction can get a very close accuracy :/.\n", 249 | "Let's see.." 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "## Random approach" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 5, 262 | "metadata": {}, 263 | "outputs": [ 264 | { 265 | "name": "stdout", 266 | "output_type": "stream", 267 | "text": [ 268 | "Done!\n", 269 | "accuracy: 0.323333\n" 270 | ] 271 | } 272 | ], 273 | "source": [ 274 | "# generating a random prediction\n", 275 | "r = []\n", 276 | "for i in xrange(N * K):\n", 277 | " a = [0.0, 0.0, 0.0]\n", 278 | " a[np.random.randint(3, size=1)[0]] = 1.0\n", 279 | " r.append(a) \n", 280 | "random_pred = tf.constant(r)\n", 281 | "\n", 282 | "# random accuracy\n", 283 | "random_correct_prediction = tf.equal(tf.argmax(labels_pl, 1), tf.argmax(random_pred,1))\n", 284 | "random_accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n", 285 | "\n", 286 | "with tf.Session() as sess:\n", 287 | " \n", 288 | " # print sess.run(random_pred)\n", 289 | " sess.run(init)\n", 290 | " print 'Done!'\n", 291 | " print 'accuracy:', random_accuracy.eval({input_pl: X, labels_pl: y_})" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "\"Clearly, a linear classifier is inadequate for this dataset and we would like to use a Neural Network. One additional hidden layer will suffice for this toy data. We will now need two sets of weights and biases (for the first and second layers):\"" 299 | ] 300 | }, 301 | { 302 | "cell_type": "markdown", 303 | "metadata": {}, 304 | "source": [ 305 | "## Neural networks show us your power!" 306 | ] 307 | }, 308 | { 309 | "cell_type": "code", 310 | "execution_count": 8, 311 | "metadata": { 312 | "collapsed": true 313 | }, 314 | "outputs": [], 315 | "source": [ 316 | "h = 100 # size of hidden layer" 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": 9, 322 | "metadata": {}, 323 | "outputs": [], 324 | "source": [ 325 | "# placeholders\n", 326 | "input_pl2 = tf.placeholder(tf.float32, [N * K, D])\n", 327 | "labels_pl2 = tf.placeholder(tf.int32, [N * K, K])\n", 328 | "\n", 329 | "# initialize parameters randomly\n", 330 | "# first layer\n", 331 | "W1 = tf.Variable(tf.truncated_normal([D, h], stddev=0.01), tf.float32)\n", 332 | "b1 = tf.Variable(tf.zeros([1, h]), tf.float32)\n", 333 | "\n", 334 | "# evaluate class scores with a 2-layer Neural Network\n", 335 | "hidden_layer = tf.nn.relu(tf.matmul(input_pl2, W1) + b1) # note, ReLU activation\n", 336 | "\n", 337 | "# second layer (and last, wow that is a small network)\n", 338 | "W2 = tf.Variable(tf.truncated_normal([h, K], stddev=0.01), tf.float32)\n", 339 | "b2 = tf.Variable(tf.zeros([1, K]), tf.float32)\n", 340 | "scores = tf.matmul(hidden_layer, W2) + b2\n", 341 | "pred = tf.nn.softmax(scores)\n", 342 | "\n", 343 | "# loss\n", 344 | "loss2 = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=labels_pl2,\n", 345 | " logits=scores))\n", 346 | "# gradient descent\n", 347 | "gd2 = tf.train.GradientDescentOptimizer(0.5).minimize(loss2)\n", 348 | "\n", 349 | "# accuracy\n", 350 | "correct_prediction = tf.equal(tf.argmax(labels_pl2, 1), tf.argmax(pred,1))\n", 351 | "accuracy2 = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n", 352 | "\n", 353 | "init2 = tf.global_variables_initializer()" 354 | ] 355 | }, 356 | { 357 | "cell_type": "code", 358 | "execution_count": 10, 359 | "metadata": {}, 360 | "outputs": [ 361 | { 362 | "name": "stdout", 363 | "output_type": "stream", 364 | "text": [ 365 | "step: 0 , loss: 1.09856 , accuracy: 0.443333\n", 366 | "step: 100 , loss: 0.750885 , accuracy: 0.533333\n", 367 | "step: 200 , loss: 0.699071 , accuracy: 0.543333\n", 368 | "step: 300 , loss: 0.65848 , accuracy: 0.606667\n", 369 | "step: 400 , loss: 0.595756 , accuracy: 0.683333\n", 370 | "step: 500 , loss: 0.590207 , accuracy: 0.733333\n", 371 | "step: 600 , loss: 0.487347 , accuracy: 0.78\n", 372 | "step: 700 , loss: 0.390343 , accuracy: 0.833333\n", 373 | "step: 800 , loss: 0.317024 , accuracy: 0.86\n", 374 | "step: 900 , loss: 0.267774 , accuracy: 0.893333\n", 375 | "step: 1000 , loss: 0.232986 , accuracy: 0.906667\n", 376 | "step: 1100 , loss: 0.207324 , accuracy: 0.913333\n", 377 | "step: 1200 , loss: 0.185526 , accuracy: 0.926667\n", 378 | "step: 1300 , loss: 0.168282 , accuracy: 0.933333\n", 379 | "step: 1400 , loss: 0.15233 , accuracy: 0.94\n", 380 | "step: 1500 , loss: 0.14039 , accuracy: 0.94\n", 381 | "step: 1600 , loss: 0.128637 , accuracy: 0.95\n", 382 | "step: 1700 , loss: 0.119176 , accuracy: 0.956667\n", 383 | "step: 1800 , loss: 0.111407 , accuracy: 0.966667\n", 384 | "step: 1900 , loss: 0.1046 , accuracy: 0.966667\n", 385 | "step: 2000 , loss: 0.0985731 , accuracy: 0.966667\n", 386 | "step: 2100 , loss: 0.0931721 , accuracy: 0.97\n", 387 | "step: 2200 , loss: 0.088322 , accuracy: 0.973333\n", 388 | "step: 2300 , loss: 0.0839386 , accuracy: 0.976667\n", 389 | "step: 2400 , loss: 0.0799903 , accuracy: 0.976667\n", 390 | "step: 2500 , loss: 0.0764005 , accuracy: 0.98\n", 391 | "step: 2600 , loss: 0.0730892 , accuracy: 0.983333\n", 392 | "step: 2700 , loss: 0.0700426 , accuracy: 0.983333\n", 393 | "step: 2800 , loss: 0.0670639 , accuracy: 0.986667\n", 394 | "step: 2900 , loss: 0.0644435 , accuracy: 0.986667\n", 395 | "step: 3000 , loss: 0.0620758 , accuracy: 0.99\n", 396 | "step: 3100 , loss: 0.0599774 , accuracy: 0.99\n", 397 | "step: 3200 , loss: 0.0580689 , accuracy: 0.99\n", 398 | "step: 3300 , loss: 0.0563364 , accuracy: 0.99\n", 399 | "step: 3400 , loss: 0.0547494 , accuracy: 0.993333\n", 400 | "step: 3500 , loss: 0.0532854 , accuracy: 0.993333\n", 401 | "step: 3600 , loss: 0.0519228 , accuracy: 0.993333\n", 402 | "step: 3700 , loss: 0.0506572 , accuracy: 0.993333\n", 403 | "step: 3800 , loss: 0.0494752 , accuracy: 0.993333\n", 404 | "step: 3900 , loss: 0.0483752 , accuracy: 0.993333\n", 405 | "step: 4000 , loss: 0.0473592 , accuracy: 0.993333\n", 406 | "step: 4100 , loss: 0.04641 , accuracy: 0.993333\n", 407 | "step: 4200 , loss: 0.0455212 , accuracy: 0.993333\n", 408 | "step: 4300 , loss: 0.0446756 , accuracy: 0.993333\n", 409 | "step: 4400 , loss: 0.0438834 , accuracy: 0.993333\n", 410 | "step: 4500 , loss: 0.0431389 , accuracy: 0.993333\n", 411 | "step: 4600 , loss: 0.0424301 , accuracy: 0.993333\n", 412 | "step: 4700 , loss: 0.0417615 , accuracy: 0.993333\n", 413 | "step: 4800 , loss: 0.0411302 , accuracy: 0.993333\n", 414 | "step: 4900 , loss: 0.0405317 , accuracy: 0.993333\n", 415 | "Done!\n", 416 | "accuracy: 0.993333\n" 417 | ] 418 | } 419 | ], 420 | "source": [ 421 | "## Train\n", 422 | "w1_res = 0\n", 423 | "b1_res = 0\n", 424 | "w2_res = 0\n", 425 | "b2_res = 0\n", 426 | "\n", 427 | "with tf.Session() as sess:\n", 428 | " \n", 429 | " sess.run(init2)\n", 430 | " \n", 431 | " for i in xrange(5000):\n", 432 | " \n", 433 | " _, l, acc = sess.run([gd2, loss2, accuracy2], feed_dict={input_pl2: X, labels_pl2: y_})\n", 434 | " if i % 100 == 0:\n", 435 | " print 'step: ', i, ', loss: ', l, ', accuracy: ', acc\n", 436 | " \n", 437 | " print 'Done!'\n", 438 | " print 'accuracy:', accuracy2.eval({input_pl2: X, labels_pl2: y_})\n", 439 | " \n", 440 | " w1_res, b1_res, w2_res, b2_res = W1.eval(), b1.eval(), W2.eval(), b2.eval()" 441 | ] 442 | }, 443 | { 444 | "cell_type": "code", 445 | "execution_count": 12, 446 | "metadata": {}, 447 | "outputs": [ 448 | { 449 | "data": { 450 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQZNdd4Pnv7958V2W931X97pZaD6tlITWWYLGZkAmv\nQ4HABsLMMGZjwcKeYZnYWRx4g4gZmD9mMeOJWVjAWDbE4NgYDGEsSwNiHNawjA0YpLaQrFdL6u7q\n7up6vyur8n3v2T9uZlVW5bsy65m/T0RHV2XezHsqu/r87j3n/H5HjDEopZRqPdZBN0AppdTB0ACg\nlFItSgOAUkq1KA0ASinVojQAKKVUi9IAoJRSLUoDgFJKtSgNAEop1aI0ACilVIvyHXQDKunpCZix\nE+GDboY6QGk3xZ1kN/6M/h4oVYuVW+MLxpj+Wo491AFg7ESY577x6EE3Qx2gqcQNPv3WRxmcuv+g\nm6LUkfDMJ37mVq3H6hCQUkq1KA0A6tD7gaFrXLVnD7oZSh07GgDUoRaw/DzZ9QpPv//zGgSUajIN\nAOpQ6wueYDRyDkuE7sH5g26OUseKBgCllGpRGgCUUqpFaQBQSqkWpQFAKaValAYApZRqURoAlFKq\nRWkAUEqpFqUBQCmlWpQGAKWUalEaAJRSqkVpAFBKqRalAUAdGZ+9+FVmR14/6GYodWxoAFBHwkj4\nLJaIloZWqok0AKgjI2D5ETnoVih1fGgAUEqpFtWUACAifygicyJScoBWRD4gIqsi8kruz79pxnmV\nUkrtXrM2hf/PwO8AX65wzLeNMU806XxKKaUa1JQ7AGPMt4ClZryXUkqp/bGfcwCPicj3ROQvReS+\ncgeJyFMickVEriwupvexeUop1Vr2KwC8DJw0xjwA/D/A18sdaIx52hjzsDHm4d7ewD41TymlWs++\nBABjzJoxZj339fOAX0T69uPcSimlStuXACAiQyLeCm4RuZw77+J+nFsppVRpTVkFJCJ/DHwA6BOR\nO8C/BfwAxpjfB34C+JSIZIEE8DFjjGnGuZVSSu1OUwKAMeanqzz/O3jLRJVqyI92vsLfDJ6HqcGD\nbopSR55mAqsjoy94AhHhsxe/yvmHyq4jUErVSAOAOlJGwmcJ2bo6TKlm0ACglFItSgOAUkq1KA0A\nSinVojQAKKVUi9IAoJRSLUoDgFJKtSgNAEop1aI0ACilVIvSAKCUUi1KA4BSSrUoDQBKKdWiNACo\nI+lHu17h/ENf56o9e9BNUerI0gCgjpy+4AlCdoAnu16he3D+oJuj1JGlAUAdSfnS0Eqp3dMAoJRS\nLUoDgFJKtSgNAEop1aKaEgBE5A9FZE5EXi/zvIjIb4vINRH5nog81IzzKqWU2r1m3QH8Z+BDFZ7/\nn4ELuT9PAZ9v0nmVUkrtUlMCgDHmW8BShUOeBL5sPH8PdInIcDPOrZRSand8+3SeUWCi4Ps7ucem\n9+n86oizyNJOjAAp0gTYoOOgm6TUkbdfAaBmIvIU3jARI6OhA26NOgz8pBhkCjBYQIg4HayyrGkA\nSjVkvwLAJHCi4Pux3GNFjDFPA08DPHCp0+x909Rh4yNNO2vYOCSJEGUFwZDv771xS8P5gPe3Ump3\n9msZ6HPAx3Orgd4HrBpjdPhHFYmwxhCTRFmjjQ26mcdPhlIX+36BHx19U+sBKbVLzVoG+sfAd4C7\nReSOiPyciHxSRD6ZO+R54AZwDfgi8C+acV51vAgOPSxiFV3tlzlehMc73uLp939eg4BSu9CUISBj\nzE9Xed4A/7IZ51LHV5hEycfLDfW72PSGTzMZv7F3jVLqGNNMYHWIlB/PN4C77WthkQHKhwelVDWH\nbhWQal1JIiW7cwPEaSOLjwBpMgSI0YGDf7+bqNSxogFAHRouNsv00MXS5qofF8HFYpleXP11Vaqp\n9H+U2hMWDmE2AO/K3qnxV22dTtIEC5aBhlmnI5cBoJRqJg0AqunaWKWHxW2PrdBFjJ6aXp8mxBKa\nBKjUXtMAoJrKT4oeFovG8rtYIU2YFOEDadd+SrsOi9k4aePQZvnp9oWxRe9g1OGjAUA1VeeOK/9C\nHSwzf8wDwJqTYjy1DHiT16tOkpnsBncFewlY9sE2Tqkd9LJENY1NljDJkit5BPCT2e8m7StjDLdS\nKxi2FrS6QNa4TKbXmnqemJNiKh1jLrNBxjhNe2/VWvQOQDVNHzNlnzNAluN9BbzhZspmMqy6KYwx\nDW9k7xrD9dQSCTeLm1spNZ2JcTLQSbfveN9dqebTAKAaZIiwQZRlAmVq9uSt0b1vrToIZh8K081l\nNogXBJr837fTq0TtID6da1B10N8W1ZAuFulhnmCFzt9L5IqQJLKfTdt3EfHjlgkCIfGRMNmGh2uW\nnHjJMwjefINS9dA7ALVrNhnaiWFVufJ1sY5l2YaEm2EqHWPDzWCL0CnBsscmTZZrySUMhqgV5FSw\ns+LKINcY4q4XVEPiAwELwS3zURvAaGVsVScNAKomQRJ0soyfNBn8rNGNTfWrWRdhiT6O283mhpPm\n3dTWLqiuMSyY0sXsNo/JBco1N8W15BJ3hXpLzgksZxJMZNYwOwaVAmITtnzE3HTJ94/agfp/ENXS\nNACoqiLE6GFh80rfJkWAWdZpL/saA6QIsUr3nq79n0rcOJAtYW6nVxt6fcJkuZFc5myoezMIpNws\nMSfFZCZW8mdKG4e0cbCQbcHBQuj2hQha+t9Z1Ud/Y1QVhu5cjf5CFoY2NihVwdNFWKOTtRozf3dr\nKnGDry9f4s9ffZSLzmBT33vNSTGf2SBrXKJ2kH5/BL9srWJKNWHp5bpJs+6mCVt+xlPL2yZ3KwmJ\nj3Y7wJqTxBaLfl8bnXb54SelytEAoDbZZImwjoVLkggpgthkkTLdkmBYpo/uXPKXd10qpAnu24qf\nv50533Dnn3KzOBhC4sMSYSodYyEb3xyySWazLGbj3B3qI2DZmCYNthtgJZtkxqyz4daeI5EyWe4K\n9DJCtCntUK1LA4ACtoZ5yK0tj7JKkjBL9FUMAFFW2aCdLD4Eb9gnRYj9mPBttCNOuw7jqWWSJovk\n2jvga2M+u7HtJzaAg2E8tUzaODhNHHQyeJO99QjI3udTpNwsM5l11t00PiwG/G102aGG8xjU4aIB\nQGGR3TbGD17nHiJBhA1ShAjuyPDNHxkgg58MBmGGUbLs/UTkQmqCpJPmuZUHWZ7tZzfX/8YY3k0t\nkjHeNjP5EfWZ7HrZ0JUw2d01uAwBIlaAFSdVcw6BAIP+8nMvzZB0s7yTXNy8A8rgMpFeJe7LMBro\n2NNzq/2lAUARyZVt3snC0M4acwwzyFRu1Y93h1DYSXpfe3MF8wzveXsBnlt5kGsv/xgX63xd3M0w\nmV6rOOSyX5PKfb4Inb4gk5nqZSLyE79D/na6fNsrpWaMw0x6nRUniSB0+UIM+dt3nRQ2lY4V5TO4\nwEI2Tr+vTWsaHSNNCQAi8iHgtwAb+JIx5jd2PP8B4FlgPPfQ14wx/64Z51aN8zZfKT/M4+JjmhOE\niNPDQsnlnwKEyuzpu58c47KQibOSmyDt80XotIOICEnXW4tfLllrP7WJf/NqutsOs+wktrVKgFFf\nlF5/hJRxyBqXsOUryh1wjMs7ya07GTAsZuPEnBR3h/qwdgzZrDtpFrNxXGPo8oVKDuusl1lmKgjr\nbpoeS0tOHBcNBwARsYHfBT4I3AFeEpHnjDFv7jj028aYJxo9n2q+JGEMUhQEDJCgLfedYLCxcCpk\n/O79+PBU4gZumbH/7GZnmBulNxBPp+myw5wMdjKdKb6yLcVi6y7A4HXGtYYMAe4O9jGdibHqpsq+\nf59/Kyv6RKADX8banHj2YTHsb6c3d0xIyv83XczGyW52/lttzhiXFSdJT0F9oMn0GgvZrUzitXSK\naWKELT9ZXELio9/f5iWclfmJ7WOWzNfqmnEHcBm4Zoy5ASAiXwGeBHYGAHVIZQgSp40IG5vzAC7e\nFo1rdG0eFyRRpdzD3o5NL6QmMMbw1P/4VMmVP/O5ypiFXZcLLDsJfClh1SndIe/kstXpR8THoL+d\n2cw68SpzABbCmWAXIdvHGbs7N8+wRKJgeacAIctPl701jCMijASiDPvbNwNOrZOta066ZFftYpjL\nrNORqw+04aS3df7kfr40LulcoNogw5KTIGoFiLml3zeqy02PlWakZ44CEwXf38k9ttNjIvI9EflL\nEbmvCedVTbREP0v0kSJIBj8xOplhDLeggqeLVfYq3wDLe7zuH+C51QfLPrfiJEt2WgaYc+J1nSf/\nPknjICKcDfVUvPbtsALcF+7f1kGKCOeDPYz4o4TFR1h8jPijnA/2lOzgRQRLpK6VNpXG+ZPG4a3E\nPMvZBO+mlmq6izHAWonO30I4G+wuGlJSR9t+TQK/DJw0xqyLyIeBrwMXSh0oIk8BTwGMjOq2gPtH\niBMlThs2Dn5SdLKEQdggSoYgaQIl5wpcYIk+zB6Xe067maJ6N8YY5rNx7+oft/QLG+BdSW9wPuR1\n5JOZWNExAgwFoiVr+1gi9Pvb6Pe3FT3XDP2+CKtlAh94y1dvNZi1DDDij9KupSaOnWYEgEngRMH3\nY7nHNhlj1gq+fl5Efk9E+owxCzvfzBjzNPA0wAOXOg9+tq5lGDpZIsraZiefHwZpJ0YWG9+OFfD5\no+K0E9/jpKRyWb930mss7ZhAbbb8GHuvL8KykyTpboUaC+jxRYhY/j1sQXltdoAhfzszmfU9/Qzm\nsxvb5i3U8dCMAPAScEFEzuB1/B8D/mnhASIyBMwaY4yIXMb7f1N+70C1bwSXMHHaWCNIsmhM0Fvy\nafCTLZEHIMwwQpb9GRfemfWbMU5TOv9BXxtZ45Z9r3yRNSs3pLPqJFnOJrFE6PWFabcO9sp40N9O\nUHzcSq/sWRDImObfXamD13AAMMZkReQXgW/gLQP9Q2PMGyLyydzzvw/8BPApEckCCeBjpln59GrX\nAiTpZ2ZzGWil0d2dz+XvDmwcmpseVVqpXxevXLI0vBFL2jicDHQST2ZImuy2d7MRBnxbwzeWCN2+\n8J7tvjU+vUQq6eUoiAh3nx6o6XWddhALaWqWcqGwFpo7lpryr2qMeR54fsdjv1/w9e8Av9OMc6lm\nMfQzg93guHmjr68mv+yzVNavr0klpmNOypuwDfUwk1lnKevdCXTYQYb97fibmPj09s25qiUsznzH\nW5Uz/miQq+OzQPVgICKcCnYxnlretuLIQui2Qyw5iV3/Swkw7Ne6Q8eRhvUWFSJRNvmrHql9GP75\nlas/weDU/UVZvxHLj08s0g1X5vTub2yxGA10NK3cQbnO/vErVWr/+L3Adq7guBcHbK6aWYIhP2eG\nS6+26rCDXAz1sZhNkDJZ2qwAvb4wtli0Z4PMZTfIGod0ncM5BphIrdLlC9HjC2Mhm5vgtFsBrQ90\nhGkAaFFWHdeDbkGSmBQ8Fqcdh4OZ/ATvqvdcsJu3kkVrCeoSbcIY/q47+xpdnnO4Pplh/FHvXOXu\nBoKWj5FA8dV6ly+0WULi3eRiXdVHAVI4zGY3mM16ZUO8ECVYCOdC3YQPaBJcNUYDQItKESo55l/Y\nhTlYZPCTpI0UfjpZJUgKk6v3HytIEtsLlbJ+89ad0mULauUt4awvge3tm3Pbvs93/M3q7Ms557c4\ndyXDiwM248GlsncC1YwGOhouiZEvoediuJZc4r7wgOYIHEEaAFqUg4912mljfTP71+CVc1ikjzRB\n+pgnSJog6W1TrQJ0sgJAbA/q/i+kJkjlMlzzwz+lJJ0MEzUUUqvkQqi34k5a5Tr7/Dh93jn//m55\nmUpmGJ/eXRCIWH7uCvV65Z6dFNkGhwIdDGtOqqhInTr8NAC0sOVcRx9lFRsnt4VjDxn8DDGJn3SJ\nqp+Qv0/oZIUMQZI0b3144aTvtZd/rGyp56VMgtuZxhKcbKTs+v3C1TglO/t97vALXZ5zeBFYO7n7\nIBCyfJwOendwa06KmykvoO/ch7hWSTcDaAA4ajQAtDDBkCTCBlEKu3c/KXxkqpb9sjBEWWlKACi8\n6i9X6ycva1wmGuz8Beguc8Wa7/w7brtcnnMOtLMvJx8EUnc1/l4ddpD7wv3EnDRu7mq+UnZxKTPZ\nDTZcb7+AoNjE3Yy3mYxYdNmhklnS6uBpAGhJhm4WaGN985E1unKF3wRfHSv7fSVKQ9crf9VfbrXP\nTovZ+ur67CR4u2oNVVjauNn5twhbrM0hnG47xHTGYn5H8bhqYm6aq8kFfLl8hHxhu0linAl2aSG5\nQ0gDQAvqZY4w8W07gHWwgkGI0UUGf01LRA2QbOC2v56rfvC2KbyRWm5oQ3Yfwmigg047tC+TlsYY\nNhbXWF9cxfb76BzuxR/0hp3EauyquHcyw/hJq+KqoN3wqpN2MOyPkjYOS9kEa06KrHFrqrdUOKfg\nzSsZxlMr3K8TxYeOBoAWY5ElwkbR8I6FoZMVYnTQTelllfkruvzX+YCxG/Ve9RtjeDe5RLaBxDMB\nzgZ7iNjNXbLoOl7F0J0duuu43P7u2yTXExjHa/fC9anN59t6ogzdc4pAxAuixnXZWIrhZB0iXe34\nQ5WXp57zW/CdFDcf25uxdxEhKD6GA1GGiWKMYSK9VrR5Ta10ovjw0QDQYrytP6ySGbyCSzsxgqSK\nAoRXO96fqwlkSBJihT6ydeYBFO7nu7OwWyWrTqqhZYs2wulgV8OdvzGGTCKNCGRSaWbeuk1qPQEi\nRPs7GbrnFL6Ad46lWzMkY3GMW7rdG0sxbr74Fmcfew+ZRJKJf3wX4+YmYY2ha7SPwbtPVky0Oue3\nGDeGq+OVk8SaQUQ4Gexk2LQzlY6VLb9djrPHWeOqfhoAWkwWX9nhHYMQKVgWulOaMLP07frck/Hr\n25Z21rOfb9rkR5Xr50O4GO7f9R65efHlGFOvj5NNZ7zloNtKoxpicyusL65x8qELRLqirEwulO38\n81zHZXlilqXbc7jZ7UNbK5OLhDra6Bqp/Jk/nssNiJ3aj6pM4BebYX+UFSdZ82sM5sCL5qliGgBa\nhOAQZZUIG5jcln+F3aEBMvgqZgjLLq/gdnvVXyhs+XZ9/R8Qm6SbbaiefTqe5PbL72Lcyp+BcVxu\nvfQ2ke4omWT1JDXjGmLzKyWziI3rsnRzpmoAAG8+YG0P5gPKCVg2Z4Ld3EytVF06KkCPHa6Yb6EO\nhv6LtADBYZhJLJwdSV/bBchsPldq0CHCOn7SLDJY89BPI1f9hdpk90M3cZPlemqJ08EuOu3djUEv\n3Z7D1FFDJ75cvHFMSQK2z6Zop5ucbLq2q/r8fEC+gNxeDweBt3z0/vAAG26atWyKBad41VBQbAb9\n7XTv8nNXe0sDQAuIsopFdtsVf76cc7lyEC5bm6NLwZ8AaQaZYpITVNtRdCpxg2cbuOrf1qYGF48Y\nvM1jOkLBXRUvS63Ha98Zvg4iQjAaIb68XvJ5y2+T3EiwcG2S9cU1RITO4R56Tw+TTWWw/fbmJHJh\nqYi1k5l9uRuwRIjaQQRhYce2mxZwJuhlit9MrWzmBfT5IvT5IlpE7hDQANACvM3e65MPEKUeF1wi\nxGvaBH7nJi67ZSH4sRra9jFrXBwMvqopbltSG0nmr00SXyndQTfKGFi+PVf2+Uw8xfjfvbF1PLA8\nMc/yxDyWbWGMIRAJMfbgeQJhb539VuG44L4EAWNMyc1oXOB6cgknVzMIwDEOU5kYCTfLyWDnnrZL\nVafpeS2g3FYv5bpB2fGn+HlDG6vsySVxuTaJMOKP1tF1l2bV8Q7peJKb//AmsbnlvftRG9gXyXVc\njGtIrSe49dLVbRPO5/wWZ76Tqrr3QDMkTRanzHkyuEWrtwyw7CRIufszaa3K0wDQAtbpwC3R8e22\naxAgRIoe5htqV726/WFOB7oIyu42aOmwgnUlIs1fn8J1jsbSRTfrsL64vTzGOb9Fx22Xq+OzRUXt\nDoN1t7FKrqpxGgBawAZRkoRxcwtAd/7ZDcEbWvJR+T/xZy9+lfMPfZ2r9uwuz7Rdhx0kYvnrvhPw\nY3GiziGH2NxynWc5OF5+Qqro8ctzzuadwNXxWcanl5p+7pDUM6jmEQRbu58Dp/8CLUFYYJA5hlmj\ni1W6meIEs4ySwd/Q6EaIRNnnRsJnCdkBmjnXN5WpPwEJvKGIWNbrCONOhpiTwqmwqieVWcOtsoa/\nEjvgo/vE3i/H3CRCsL30PsXn/BaPX/GK2+3NqWWzsmjNr8EL5upgNSUAiMiHRORtEbkmIp8p8byI\nyG/nnv+eiDzUjPOqegjpXLnnNbpx8JMhyAwnmGcIF9m8QyhU7S7BreFX6Ec7X+Hp93++4bsAYwyL\n2d2VIQC4nVnl1cQs76QWuZ5a5rXEHHMZb4errHGJuxkc4zI4GEVMkrl/epaxf9/JZ1+e4Jlbr/Ps\nze/x5e++yc/8HzP4/OXrEfnDQc4+dj9DF08y8p4zYAliyeaEiliC5W/ePsOI4A8FiHRX37c3v49A\ns0XtIJ1W+Q5dCv62EM4EurxJ+Tq3p1TNJY1OEomIDbwDfBC4A7wE/LQx5s2CYz4M/G/Ah4HvB37L\nGPP91d77gUud5rlvPNpQ+1RtvJU9G4SIEyKeCwWCgy+3hLR4KtlFmOTUjpSy0qYSN/j68iWuvfxj\nu26jY1xeS+zTWLYx/OyF1/iVS98hYBc9hTHwtS/08Qf/fgTX8T6ZvuE0/+wzLufuXWN2sptvffM+\nFuY6cB2X+IqXFxDpjmJZFje+84ZXQqJBIkJbXyfD957G9tuk494wUCBSernr9YzLzcdCe7IyyDEu\n7yaXSBsHN/f7kl9GXNjL5INAfnI4agU5GexsOFNbeZ75xM981xjzcC3HNmMZ6GXgmjHmBoCIfAV4\nEniz4JgngS8bL9r8vYh0iciwMWa6CedXTWCw2CCa2xvA4CeDi+T2/HUZYJpArkZQPhQsMFhT5w8Q\nsPz8aNcr/MrI+bI7fFVj5fagbaQmUK3aA2k+/cA/FHX+ACLen49+coGP/MICxoDjgM8PrgGfBWfv\nmeHy+9/hD//vx3nnzVHae7fPP2RTzZkAHXngLNG+LhKr60y+Po6TyYIBX8DH6HvOEu7avlQ3Xzto\nL5aH2mJxd6iXmJtmw0mznE2QLvGvZfB2Ectbc1NcSy5xd6hXcwP2WTNC7igwUfD9ndxj9R6jDg3J\nzQ3kU8Es5hhlllFW6GGZPiY5VddGMH3BE4TsAJ+9+FVmR17fXatE9i2j9Pt6qw9XiYBlgW1DIACW\neJ0/gN8yhIJZfuJf/Q9EigNWsL3MZycQikboGuvn1OWLRHoqD+tMvT7Ou996ldsvv0M2mcY4LsZ1\nySTT3Hr5nZLlKB6/4tUy2ouVQSJChx0ki0u6jpyNtHHq3qheNe7Q3XOJyFMickVEriwu6jKx/Wfo\nZJExbjLCbca4SQfLgCFDkBidJHa5A1hf8ETD9eBHAx0Nvb5WKccunyhRh95gnJO/+W1eHNh+K9F/\nfqSofLSIEGqPcPr772H4nlNEOts58eB5xC7/39Q4Lk4mW7ronDEsT5ZeqptfGbQX8wHGGJay9Q1v\nGQxJzQvYd80IAJPAiYLvx3KP1XsMAMaYp40xDxtjHu7t1eqB+62LRaKsYWE2/3SwQgfLhFlnhNuM\ncosxbtLLLLKLHcF+YOjarttniXA+2LMtSU2AiPjokOb9vry0MEwy2/gIqSXCx7rf4XO//gX+/CNb\nESXSFWXs0jn8kSCI1/lHh7o5+X13bRsGsWybkfvOeJPIdTKuIRUr3RHncwT2alK43kE6QQhaTZwY\nVzVpRgB4CbggImdEJAB8DHhuxzHPAR/PrQZ6H7Cq4/+HT34/gJ3loPObxfQyhw9ns/ONsMEg05T6\n7+4Vniu+ogtYfp7seoV//U9+bdergtrtAPeHBzgZ6GTUH+VCqJe7wn2cDfc0JVsYwDEWn/zbD5Fy\npJFkXQQY9FuELeHLH/wiP/X5L23eDbT3dXL+B97D3R94L3f/k/cyev9ZbH9x0OkY7Gbg7hNFj1c9\ntyWEO8rfrV2ec/ZkaaiIEJL6gqdPLC0XfQAaDgDGmCzwi8A3gLeAPzXGvCEinxSRT+YOex64AVwD\nvgj8i0bPq5rPR+Ux2J2/LAL4STPEBP5cQpiPNIPcYZRbjHKbYSYIsFU3vi94gtHIuYY7aVssun1h\n+v1tRKytSqED/jbuCvXSb0cI0NgV5ZWFIR77r/+cp9+6xFw82EAgMJwK9TIWOQfA5379C7z0S1tJ\nZpbPrrg9ZCaVZvbqRNnnyxERusb6Kx7TO5khlcxwdbw5iXp5o4HaArEAEcvv3dXpBPC+a0oxOGPM\n83idfOFjv1/wtQH+ZTPOpfaOU2GzmEp1g/xkGWCKZXrozW0nmT/eT4YBpplmLLeiKPc6EZ5+/+c3\ny0Q3U9jyMxr0Mwq8Hp/dtkftdi7bB5LyvONPd8T45/eOc39fmmRklBWW6WJ5F8Fr6xVjkXNMJW4A\n3pLMc/7q12CrU4t11wwS2+LUIxc3dycrp7CC6HhoqWklpKN2kHPBHqZzhd/s3Oqt/E9hMAz62ujx\nRQjo0M+B0WqgapOLTZy2og3jq/HWdbv0slC2eFyUVVYKdhMbCZ9lITXBDwxd489n+5tSMbSUYX+U\nO5m1bT+N38pyb+8kj45dxzhjTK6H+avbQ2RdC0sMT56/w8+/5xq2tf0ziNFNmiA9zG8OhdUqQQSb\nDG7uruQ/3PNnPPu5S7z5y99XNQhk0/WvjrEsi1C0vsn6/HxAs4JAux3ggt27+X3WdZjNbrDupLEQ\nUsZh3U3TJSHdLP6AaABQ2yzRTw9zRIjX1cFVOlaAdtZI0E6K7cs4n+x6BS7RlD0DSsmeusGD7hpv\n3D6Bcb1WPn5qll967zWCPhe4BcC/euhtYmkf7YEsPqt88DPYOPjw1TD5nc+iThFkmDubj3eGO1mh\nhyd5lSe/9Cof/+YneOJr5c+5ub1kHYXpDAZjTM3DKlslpNmTHIGMcXgntUimIPN3w8mw4iSZIsZd\noV69EzgAGgDUNgaLRYYQpgmTaMqEKnjzBwNMMcMImVwQ6At6E5tP8gp/O3geppoTAM4/9PXNr/91\n1yuE7ABHV4cXAAAfu0lEQVTu/cJqKsTJtoFcx7+dbRm6QpWvtP2kGGCyromzFEGCpLbdUbWzBhjI\nDQd9+YNf5NlHLvHt//Q+Ls8VB5Zofxf+UIBMvPbyzm7WZWVyge4qcwCF8sNBLzzsb3oQmE6vb+v8\n8wyQxeVWeoULod7iF6o9pQFAlbREP0NMYuFi5cZumxEMhphigQESBZvJiAifvfhVfgXqng/YmVT2\n2YtfRYBgbv/fvuC5zecGQkADG8p0sVT3ZxDKZU8XsjC0E2OVHkbCZwG8u4FfL303IJZw+pGLzF+f\nZHlywUs3rsYYFsan6woAeWdyW0s2czhotcoG8nE3Q9a4Wg5in2kAUCW5+JjmBBE2CJLAR5oA6bKb\nxNQi/7pe5pkmhJP79RsJn2UqcaNqECjX2W8f5pDNTrXZAiSbdkcE4CNLBi9QjRXcDXx67KM88tvd\n2461/T6GLp7CyTiszdS2bj9bw6b0pZzzWyzedlk72dw5gWoqby2v9oIGAFXW9vpA3iPtrNDFcoPr\nhw0RYsTY6uTyQQAoyg/oHpzf986+lK3yZrWpdLQA2R3//UbCZ5mMX6/4nol6tqYUIZNM4w/Vv77+\n8pzDi0DsVHOyczvtEEtO+exgv9j4Dl9hgmNPA4Cqg7BOFzYuUdZyj5jc31t2Vn7cycJbNbRTwPLz\nmxe/ChdLnFn2t7MvJUEb7cTqvgtw2Z5D4SLEad9WSG8hNUHS8a7YV/9uAMrkZPiC/pL1fUoyhut/\n+xoD58foOVX//ErvZIa1kxZXx2e5eKax+ZmRQJRYMlVyHkCAE4EOzQM4ABoAVJ2EVXqJ0YWfNL3M\n4tvRmQtep2eQsmWkUxRvXjIQHCZAChebNEGaM+vQHD4y2LsoewEQp50I3r4Dkvt+qWBJbN6zK96y\n0Mf95Seje08PM/naDYxb21yGcQ1z1+7Q1ttRdsOYcgonhRvlE4uLoT6WsglWsknSxkEEIlaAQX8b\nYavxc6j6aQBQu+JikyJcIV9AWKWbTpa3BQE3V2k0SRgwBEjhI0OYDSIkcBEEg4PNHMPbkscOik2W\nwdyEeP7nqGVS3MXbjnOZfpZx8ZEli41pIEM5OtBF75khFq5P1fwa4xpWphYYvKv+chKAt68wswRD\n/obmA2yx6Pe30e9v2/V7qObSQTfVkPwkZikpIsxwgg3ac5WBbGJ0MMcIFi5DTDLAND3M5/IODHZu\n1ZEvl128+12Lm8XQxcK2zh+qd/4GSBJhOXelb7DIEGio88/rPzvCqUfurqtAnJvd3d0LbO0rnE5p\ntc7jRgOAasgKPTu6Ru/KN0WIDAEcfCwxwCSnmeIUq/RisOhlFj/pXMXR4g5VAB8O4dzQycEw9DFb\nd1Jc3iL9NHsYy3VdsukM4c52huuoEtre11n9oCr2ag8BdXB0CEg1JEWYeQbpZhE/GQzCOlFWKD9U\nYJElVOOSynwGcaN8ZLBwchvd1HYV3sUi4V10/vns33qu9vMTwOW4jsPM1QnWZha9ZDCR3N6U1d9b\nLKG9v75N23cqnA/Yz6Wham9pAFAN84Z6ItQ2Mg52bsfYcoXn8vKF5hphkaU/d7eR7y29ANVXoa2G\nbuZoZ2PX1++rFQLgTnfi1zcngMvVBbrz6nU2lta2Ovw6isN1DDav0mZHLj+gkoxxWMwkSJosYfHR\n649ogtchpQFANVFtnUymjl+7bNmraLO5Kscp+36GAWbw5xLY8qLEiBBnjmGyRXMYhkHuECCzq87f\nK23gL6p5VE4+9+Ev/uyxsqt/UhtJ4suxXU2HiG3Rd2a4/heWkc8PuMosIlJULiLuZriWXPJqEQGr\nwGx2gwuhHl3pcwhpAFD7zNBGDBcLqVJR00VYp3jsOkiCHuaxc6/P4CNJiBAJXOzctpVt+EnjK9GR\nC2DjMMgUU5zcXI/vI0MHy7vq/POF37L4mWeYWoLhnVzS18e/+QmeuFL+qjq1nkBE6s6UDXe2MXjx\nJIG25u6jvFU4LsjV8a3VQcYYbqVWtm0D730uhlupVS6Gi5e+qoOlAUDtI0M/MwRJbi4fzXcVDlYu\nOWyr41wnSpytJYMWDlFW6GB1W/fqJ4N/s9POEmSODdoqzh3kS1j3MY2DTRvxbc+V/wnKPz/LCJka\n8hemEjdwc0M4v/xvf4EndhSAy6YzrM+vYFxDe18ngXB9m9GIJQzefXJXdYBqVbiPwNrJDG/fnOP0\nyR7SpvRqo5TJknEd/Frx81DRAKD2TYjEts4ftsolWLi53IIQSUIkiWzLAQixQR+zJWsRlfq+jQ3i\nVSaPBa9YW6n3KKVcH2yANMHNKqeVbLvq/5rh8o7ksuU788y+fZv8JzP7zgS9p4cItIVIxeLFb1iq\nPa4hk0jVdGyjCu8Grt9ZwPRS9sM86AW9qpgGANU0heUMQnZgs9xzXji31n+nfKfu7SPsHbOxOfRj\n6GSp6Kq/FkGSJAkSrrDiqN73XKKXbpaQgspABimZ2VvKsyte2eedV/3gDfXMvn0b4+YHlDyLN2cZ\nuf8M02+O42ZrywC2A/v3Xzt/N2CAv/phm0Sk+FMNiI1fJ4IPHQ0AqiGFwxkAn37ro4C349WdHYXN\nAgGL9iq/cRaGEAn8pMgQpI0Y0V10/gBtxLAbKP9cyAAbtLFBJxmCRFnFR4Y0IdborJqxvBUcL5U9\nZmVqIdf57zi367I2s4Tt9+Nma7uyX51apPfUUE3HNosAD77i8OJlG0cAW7YmR1ZcZOzwlPZQnoYC\ngIj0AH8CnAZuAj9ljFkucdxNIAY4QNYY83Aj51UHazJ+fdt1/Me/+QkG/sa7ustvaPLLA7+w7TXy\nsQV+594/Y8BnVV3+CYYQSTIE6WRl19mKdlGKWn3y4/0GcLFYxduwJE2IxRpX+cDWsM+n3/LKPO8c\n9slz0uWXvDqZDMH2UM1DO+l4knQ8SSDS3AnganqX4f3fchg/bbHWYTDLhp4bDnce9CaM88tRm73j\nmNqdRu8APgP8d2PMb4jIZ3Lf/0qZY3/YGLPQ4PnUPtvZ2ec99QefArzNQ57wG9jRqe3c2eqFvxsg\ncQ+b9YFKVRHdIji5br+WAmw7J2bz7W30etMr5+DNR6wTrbuMw87J3lK7fRVq7+tkbW65eOtHAcvv\nq6sUtIjgZA6mdEMkAfe9VfgzWFy8kuF6xntscdTPVbMVDAJBnyaWHZBGA8CTwAdyX/8R8NeUDwDq\nkJp+1+KFL/qZesfi9CWH+/7ZdbrH0psdab6zz3v8SobH8+WKq2xovtPbiSVOhr2NZgIkCJEoeYWf\nyK3+yeAnQPksWRdIEM6N85vcY1bNQz/lgoU35NOxWcunXgupCVxjyk72lhId6CIwHiQdT24NBYkg\nlrCxsFpyeGgzI7hE++ut/rnX8klu53ITx+AFg7WTZk/2IVbVNRoABo0x07mvZ4ByRcMN8IKIOMAX\njDFPN3he1SSv/382v/fzIbJpwTjCjZct/ur/vYc7/8sFEu/p4oln2OrsG/D4lQwfP/kJvvzBL3Iz\nfpuxyDmgkx4WiLD9ynae4c21+Sv00MdsyaqjLpAlwCKDgGDj4GLRzSJtJer2l7pTSOaGcgpLU+SH\nfNaov3xCPrHLGMOzK5e450+cikEyvhxj/tokqY0k/nCAvjPDJGNxVqcXwUD7QBer04slO3+xLYbv\nPcX0Gze3PS+WRf+5ESz78C65LAwGzDm88LB/M6cA0DuCfVI1AIjIC0Cp2aRfLfzGGGNEpNzg7g8a\nYyZFZAD4pohcNcZ8q8z5ngKeAhgZ3d/xy1aykJrAdeBLv3SBTGKrg3KzFmRh5PPX8AX9rN97uimF\nxACe+Jrhl//mF/if/ve/50lexRKB8FlW6SJEEheLBGEKaxQmibBIP90sYefKQjhYuNhsEGWdjs3j\n8xnBMTqJsF4012AQ0vgIkMHFYp3oZsmGNtZpZxULlwQR1ujCrfH6aCE14bU1twIqPxG++ncDFWv7\nr80uM/X6+GZtfyeTZeqNmwxcGOXCD3mTxemNpBcMyghF2zjx3gtbQSQUoO/sMB2DR6sDfTyXU0Bu\nWe7VZOlMY9VcYurJMNn5YpG3gQ8YY6ZFZBj4a2PM3VVe82vAujHmc9Xe/4FLnea5bzy66/ap0vKT\nkl956RLv/hy48fKj5WJZnHrkbsIdza3h/ucfEb78wS8C5O4GqjF4ubC17UocZoMe5rcNCy0yWHOJ\nhloULnt9dsXrsL/9n95XdawfvDuEa9/+HtlUcYCwbIsL738Qy7ZwMlne/darpe8ALOHCD13C9h+/\nxXzXMy7jjwY3v9dgULtnPvEz3611oU2jvznPAT8L/Ebu72d3HiAibYBljInlvv4R4N81eF5Vo51L\nMWFr56mxVAqTfAsqjJcb12VxfJqxS+eb2q4nvmb4v/7kf+Xez313826g8paP+ZX3tUnQxiQRAqQw\nSG7fgsaXIZZa9trzHzs3hzQezqaJLccQESLdUSy79PCPk86STVco/7CRINzRhu330dbb6c0BFJxX\nRGjr7TyWnT9s5Rbk5YeI8hrdolJ5Gv3t+Q3gT0Xk54BbwE8BiMgI8CVjzIfx5gWeyc34+4D/Yoz5\nbw2eV5VReFUKlK0yec4PxhfCDvjIVtljNhkrv5l3I875LVL/5yN8/COX+fIHv7gtWNV2V1CNkG7C\nFX+pZa9PfM175BEgnwIw9+4dFm/OeGcWwQiM3n+WjsHuovdMx5NlU2ONMds69pH7TjPxyjWSaxuI\nJRjXEOpoY+S+0w3/bEfF4wXB4MUBu2wxOlWfhoaA9poOAdWm3FUpULa8cF5idZ3b330H1zVlSwy3\n93Vy4r0XmtfgEgqXCH7u178AlM4m3i8775ye+oNPceY73vh0qc90+o2brEyVWOUswrnH7icQ8YYz\nMokUd753g+Ra+Y1uQtEIZ953b9HjqfXE5tr+w7bCZ78VDhFpbsF29QwBaQA4onZ2UKWSsWqVTWdY\nnphnYXy6KAiIZXHq++4i3FV9UxZjDCuTCyzenCGbyhBsC9F/YZT23vonkV/6pWX+wz1/BjTrbqA2\n+WCaL9kA1T/P1EaSG3/3etnn23qihDrbCXdGmH7zVsWEL4DTly8S7mx8E5xWkL9w2BkMWjm3QAPA\nMTO5o7PfuT7/zHdSVa/0a5HeSHLnteukN5IggmVZDN1zsuKKEtdxic0tk0mmScbixOZWtgURsSxG\n3nOGjoHiYZBqXhywN+8Gqs8R7F7+8823unCIpxYL49PMX5uselx++KbiMbbFmcv3tPwV/m4U3kWu\nnbQQEQJBbyitlYLBfk4Cqz1SqlPKC932NZSMVU6gLcTZ991HJpHCdVwCbaGKO0kl1xPcvnIVx3Gh\nTMdmXJe5tyeI9ncVvZcxhmQsjpt1CXdGitatX55z+NNP/fzmiqE78eslp3FHd3GHUOrzDd32FWQ2\nV+Y6DiKCWFbVK/q8ap0/eFew+eEiVZ/yiWaWJpqVoQHgkFhITZB2Mxjjbfvx7Mol/uyly4C3YuaJ\nbTOGjSdmVeIPV++AjDFM/OM7OJnqw02ZVBrXcbF9Wx18MhZn4pVrOJmsV2/HwMCFUXpOFq/uyOcP\nzP1g8Wqljz7yIk/yqrc4tMZtD/NDPMWfb6ZqMN1YWmPm6u3NSdxQR4TkWm1lmqsRy2LgwhhiadXM\nRu1MNHtxwOaq0USznTQAHKCKCURXMjs6/cMltZ4ouYa9nPX5VeLLa/gCfqJD3dy68jZu1gse+Z9y\n7t1JAm2hknMGl+cc+FqJdnztEZ56+DE6H5vzHsgYZMXBdFgQLN2Rzt7pKRFUK3Ndl9WpRWau3t42\nxNWszt8fCTB44STRgcY2b1el5fctWBz1LiI00cyjAeCA5CdxSycQ7e0VfjO4jlPbDh8Clm0z9eb4\n5jDRwvh0yUON6zJ/baripHFyLc7a7CLGQMdAN6HONh6/ksG81MXi+AyLN6dz/bOhc6SPobtPlLii\n3pEh7Bo2ltZwMlki3VH8oe37BK9OLzLz1i1c192zXU3O/8ADe/PGatM5v+XdEcC2LS3zWjG3QAPA\nPqiUjJW/Va2lWNhhEopGKh8ggohg++267hSSaxvMvHWLwYsni4Z0Zt+ZYHlifrN0wvLEPB1DPQzf\ne4rFWzMsjE9vPgewOrWA67iM3n+m7PkSaxtMvPwuxs3tZGtygSN3/sTqBtNv3tr2vs12XJO5DrOd\niWb53IK8/D7Hx53+5u2BepKxjirLtglGw6TKJIl1jfbSNdrP7e++U/d7r0wtEu5qp3O4d/Ox+Eps\nW+cPuY1SZpeIDnhX/zs7aeMaYrNLZO8awxco/rBdx/VyILLbg+/q1CKhaJjusQEWbxW/bz1EBF84\nAMaQSWaKcy0Euk/s3d69qjaXc3MFwLZ9jo/7EJEGgCbZmS26MxnrKHf25YS72ssGAIw3dLKzc62F\ncV0Wb81sCwCrU4slO2LjuEWBYdvzxrB8Z56+M8NFdxTrC6ulSym7rjfWj3iTvQ0wxtAx2EN7Xwe3\nv/tO0eks26b39P7u3KUqu7yjQulxTjTTANCAUtmiHbe9jujynEOVXQKPvErLH5PriZo3MS9l57CR\nW+kq3Biv9EKpAXoDi+PTJNc2GLt0flsQcNIZyubBGJh9e4JgW2hrW7BdENsi0tXO/LWp0ts9GkNy\nLU6kO7q7E6g983gLbGKjAaBGtSRjVSr9exxFutpZn18p6ti8sf/qv1piiXdFXKITdtJeaeThe08h\nIoQ72libXip+D9uiY7iHUGeEpVtzpe8SXMPGUoz1hVWi/VurbLzs5vK9u3FdUhul73Bsv4/h+09z\n5x+vVfwZA5EQbb0dTL1+o/QBxpCMJTQAHFLVNrE56olmGgAqyG/uka+zU5iMBd768WYnYx0lnSN9\nLIxPF90JiG3R3tdJfDlGuc41v+Y9NrdMfCVW8rC1mSVCHRHaezvLZtoGIkE6hnoQEdysy/LEXMnj\njONtrB7t7yK1nmDu2mTF9m01FPyBAJkdBfN6zwwR7etCLKvs8FOkJ8qJ3F2HLxTAyRQHExHBHw6U\neLU6bErlFoDD2kmLq+OzR3IVkQaAAqWSsf5m+vyu1o23Attnc+byPcxcvc364ioYr+7N0MVT2AEf\nc+/cKfk6XyjA8L2naO/tpGusj6nXbnglJHYwrsvy7VmSa3Hcnfvk4nWe/edGsXLLPLvG+ssGgNwL\nSG0kufniW9vfr8IQj3EMGbf4zm7+2iSdw330nhli4fpU8aksYeyBc1i55Le+MyPbNn/Js3z2rmol\nqYO3uWw7n2iWq1B6lIaHWj4AVEvGegTYs8Xfx4A/HOTEey9sjqUXjrGP3H/GG/ow3li32BahaIST\nD921WSffsizaejpYL7PnrZPJ5q7Ui+VLSeSHdVZLVeMs0DXcy/z1yeJgUrhX5M4mlNlzFxHW51fo\nOz1EYmWd+HIM4xrE8jasGX3g7LZhsI7BbtLxJAs3phCxMMbgDwc48eD53GvUUbaVaOYHnCOTaNbS\nAeCoJ2MdJqXKMHQMdhPpeoDVmUWcdJZIT5S2no6iYyuNxYe72nHSWTKJVPE5LWvb8s5KK45sv49I\nT5TJMmPxIoIvGPAmn3PN8wX8uFkHJ1NistsYjOsilsWJ914gubrBxnIM2+8jOtCNL1D8X6vvzDA9\nJwZIxuLYfl/VWkvqaClMNCtcRZR3GHMLWioAVNvN6aglYx0FvqCf3lOVlzmGohHaeqJsLK1t39zc\ntug/P0Z6I8nkazdKrqEv3Gylva+L1ZklzM4rfBF6zwx5k9M+X8nVS95w0gjB9jDJWBx/KEiku52Z\nt255df5L3AS05fZKFhHCXe01lcy2fLZO+LaInZvYrJ3MbAaEwxIMjm0A2NnZ55XbzUkdrLFL51i8\nOcPSxDyu4xDpbGfgrjEC4QALN6aK9ymwhRMPnt82zNLe30koGiG5trEZSMTyruy7R71kq56TA8y+\nc6fkxG10oAvLtrdlOfefGyU2v4qbyW4Nc9kW3WP9BGoomqcUbE808zazYTMYHORQ0bEKAKW27rvn\nT7Zf1ddS6lftP7Es+s6O0Hd2ZNvjd169zvrCjgliEbrHBmjr6djxsHDy++5ieWKOlckFMIboUA+9\nJwc3J2O7xvpJrG2wNrMECPkRmBPvvVBUjhq8O5izj97H8sQs6wur2H4f3ScGaO/TiVu1O4VlKPI7\nm10dnz2Qu4IjHwAqJWM9Mee05PLM4yKbyrC+UJxnQC67d+D8aFGhN8uy6D01VHbYSUQYue8MfWeG\niS/HsHw+2vs6y27eDuAL+Og/N0r/udGGfyalCuWDQWH5Cdi/RLMjFwAWUhOkcit2NBnreMskU7l1\n9iXmZozByTr4ArsL8IFIiECk8Q3jlWqGywWJZuBtcbkfiWYNBQAR+Ung14B7gMvGmCtljvsQ8FuA\nDXzJGPMb9Zwnn5CVX5//6bc+yuwd7wNp9WSs48wfDpZNsspP6Cp1XBQWi8zfFRQmmu3FEFGj/4Ne\nBz4CfKHcASJiA78LfBC4A7wkIs8ZY96s9uZZk94c4tm5Pl/X5h9/voCfjsEe1maXtq8Osix6Tw/r\n+nl1rO1MNNuLCqUNBQBjzFtQdSu+y8A1Y8yN3LFfAZ4EqgaApWyIT7/1UcxX+nR9fosavvcUlm15\nSzFzk7Y9pwbpPaMVNFXrKEw0u2qat4nNftxDjwITBd/fAb6/3MEi8hTwFEA01MUjv90Nuj6/ZYll\nMXTPKQbuGsNJZ/EF/bpnrmpJm4lmBfsWNLqJTdUAICIvAKUut37VGPNsXWergTHmaeBpgMHOMR3n\nUYBXN98KFy/TVKpVldrEpjDzuBZVA4Ax5vHdNW/TJHCi4Pux3GNKKaWaoDAYvFXH6/bjXvol4IKI\nnBGRAPAx4Ll9OK9SSqkKGgoAIvLjInIHeBT4CxH5Ru7xERF5HsAYkwV+EfgGXnD6U2PMG401Wyml\nVKMaXQX0DPBMicengA8XfP888Hwj51JKKdVcupxCKaValAYApZRqURoAlFKqRWkAUEqpFqUBQCml\nWpQGAKWUalEaAJRSqkVpAFBKqRalAUAppVqUBgCllGpRGgCUUqpFaQBQSqkWpQFAKaValAYApZRq\nURoAlFKqRWkAUEqpFqUBQCmlWpQGAKWUalEaAJRSqkU1uin8T4rIGyLiisjDFY67KSKvicgrInKl\nkXMqpZRqjoY2hQdeBz4CfKGGY3/YGLPQ4PmUUko1SUMBwBjzFoCINKc1Siml9s1+zQEY4AUR+a6I\nPFXpQBF5SkSuiMiVRHpjn5qnlFKtp+odgIi8AAyVeOpXjTHP1nieHzTGTIrIAPBNEblqjPlWqQON\nMU8DTwMMdo6ZGt9fKaVUnaoGAGPM442exBgzmft7TkSeAS4DJQOAUkqp/bHnQ0Ai0iYi0fzXwI/g\nTR4rpZQ6QI0uA/1xEbkDPAr8hYh8I/f4iIg8nztsEPgbEXkVeBH4C2PMf2vkvEoppRrX6CqgZ4Bn\nSjw+BXw49/UN4FIj51FKKdV8mgmslFItSgOAUkq1KA0ASinVojQAKKVUi9IAoJRSLUoDgFJKtSgN\nAEop1aI0ACilVIvSAKCUUi1KA4BSSrUoDQBKKdWiNAAopVSL0gCglFItSgOAUkq1KA0ASinVojQA\nKKVUi9IAoJRSLUoDgFJKtSgNAEop1aLEGHPQbShLROaBWwfdDqAPWDjoRtRI27o3tK175yi19yi0\n9ZQxpr+WAw91ADgsROSKMebhg25HLbSte0PbuneOUnuPUltroUNASinVojQAKKVUi9IAUJunD7oB\nddC27g1t6945Su09Sm2tSucAlFKqRekdgFJKtSgNACWIyE+KyBsi4opI2Rl/EbkpIq+JyCsicmU/\n21jQhlrb+iEReVtEronIZ/azjQVt6BGRb4rIu7m/u8scd2Cfa7XPSTy/nXv+eyLy0H62b0dbqrX1\nAyKymvscXxGRf3MQ7cy15Q9FZE5EXi/z/GH6XKu19dB8rg0zxuifHX+Ae4C7gb8GHq5w3E2g77C3\nFbCB68BZIAC8Ctx7AG39TeAzua8/A3z2MH2utXxOwIeBvwQEeB/wDwf0715LWz8A/PlBtK9Ee38I\neAh4vczzh+JzrbGth+ZzbfSP3gGUYIx5yxjz9kG3oxY1tvUycM0Yc8MYkwa+Ajy5960r8iTwR7mv\n/wj4sQNoQyW1fE5PAl82nr8HukRkeL8byuH5N62JMeZbwFKFQw7L51pLW48NDQCNMcALIvJdEXnq\noBtTwSgwUfD9ndxj+23QGDOd+3oGGCxz3EF9rrV8Tofls6y1HY/lhlT+UkTu25+m7cph+VxrdVQ+\n14p8B92AgyIiLwBDJZ76VWPMszW+zQ8aYyZFZAD4pohczV09NFWT2rovKrW18BtjjBGRckvQ9uVz\nbQEvAyeNMesi8mHg68CFA27TcXBsPteWDQDGmMeb8B6Tub/nROQZvNvypndUTWjrJHCi4Pux3GNN\nV6mtIjIrIsPGmOnc7f1cmffYl8+1hFo+p337LKuo2g5jzFrB18+LyO+JSJ8x5jDWsjksn2tVR+xz\nrUiHgHZJRNpEJJr/GvgRoOSqgUPgJeCCiJwRkQDwMeC5A2jHc8DP5r7+WaDo7uWAP9daPqfngI/n\nVq28D1gtGNbaT1XbKiJDIiK5ry/j/X9f3PeW1uawfK5VHbHPtbKDnoU+jH+AH8cbg0wBs8A3co+P\nAM/nvj6Lt/LiVeANvOGYQ9nW3PcfBt7BWzlyUG3tBf478C7wAtBz2D7XUp8T8Engk7mvBfjd3POv\nUWGV2CFo6y/mPsNXgb8HHjvAtv4xMA1kcr+vP3eIP9dqbT00n2ujfzQTWCmlWpQOASmlVIvSAKCU\nUi1KA4BSSrUoDQBKKdWiNAAopVSL0gCglFItSgOAUkq1KA0ASinVov5/AXOnoKPeZ6MAAAAASUVO\nRK5CYII=\n", 451 | "text/plain": [ 452 | "" 453 | ] 454 | }, 455 | "metadata": {}, 456 | "output_type": "display_data" 457 | } 458 | ], 459 | "source": [ 460 | "# plot the resulting classifier\n", 461 | "h = 0.02\n", 462 | "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", 463 | "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", 464 | "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n", 465 | " np.arange(y_min, y_max, h))\n", 466 | "Z = np.dot(np.maximum(0, np.dot(np.c_[xx.ravel(), yy.ravel()], w1_res) + b1_res), w2_res) + b2_res\n", 467 | "Z = np.argmax(Z, axis=1)\n", 468 | "Z = Z.reshape(xx.shape)\n", 469 | "fig = plt.figure()\n", 470 | "plt.contourf(xx, yy, Z, alpha=0.8)\n", 471 | "plt.scatter(X[:, 0], X[:, 1], c=y, s=40)\n", 472 | "plt.show()" 473 | ] 474 | }, 475 | { 476 | "cell_type": "markdown", 477 | "metadata": {}, 478 | "source": [ 479 | "Now we're talking!" 480 | ] 481 | } 482 | ], 483 | "metadata": { 484 | "kernelspec": { 485 | "display_name": "Python 2", 486 | "language": "python", 487 | "name": "python2" 488 | }, 489 | "language_info": { 490 | "codemirror_mode": { 491 | "name": "ipython", 492 | "version": 2 493 | }, 494 | "file_extension": ".py", 495 | "mimetype": "text/x-python", 496 | "name": "python", 497 | "nbconvert_exporter": "python", 498 | "pygments_lexer": "ipython2", 499 | "version": "2.7.12" 500 | } 501 | }, 502 | "nbformat": 4, 503 | "nbformat_minor": 2 504 | } 505 | --------------------------------------------------------------------------------