├── DL_models for EEG classification.ipynb ├── EEG Eye State.txt ├── EEGdatasets.md ├── EEGtutorialusingdeeplearning.m ├── Eeglab_data.set ├── ML_Algorithms for EEG Classification.ipynb ├── README.md ├── brainvision_raw.ipynb ├── eeglab-data-sample-2.ipynb ├── eeglab-data-sample.ipynb ├── event-related-potentials.ipynb ├── motor-imagery-decoding-csp.ipynb └── readme.md /DL_models for EEG classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Deep Learning Models for classification of EEG dataset\n" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# Prepared By Abdul Qayyum\n", 17 | "# Date=14-7-2019" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": 1, 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "\n", 27 | "# Loading the dataset\n", 28 | "# Dataset is provided in this tutorial. \n", 29 | "You can process your won dataset and run the model \n", 30 | "fname = \"EEG Eye State.txt\"\n", 31 | "with open(fname) as f:\n", 32 | " content = f.readlines()\n", 33 | "\n", 34 | "content = [x.strip() for x in content] \n", 35 | "content = [x.split(\",\") for x in content]" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 2, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "# Dataset should be converted into numpy array for deep learning models\n", 45 | "# Converting list to numpy array\n", 46 | "\n", 47 | "import numpy as np\n", 48 | "\n", 49 | "content = np.array(content, dtype = 'float32')" 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": 3, 55 | "metadata": {}, 56 | "outputs": [], 57 | "source": [ 58 | "# Shuffling the dataset\n", 59 | "\n", 60 | "import random\n", 61 | "random.shuffle(content)" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 4, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "# Creating X and Y\n", 71 | "x = content[:, :-1]\n", 72 | "y = np.array(content[:, -1], dtype = 'int32')" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": 5, 78 | "metadata": {}, 79 | "outputs": [ 80 | { 81 | "data": { 82 | "text/plain": [ 83 | "array([4329.23, 4009.23, 4289.23, 4148.21, 4350.26, 4586.15, 4096.92,\n", 84 | " 4641.03, 4222.05, 4238.46, 4211.28, 4280.51, 4635.9 , 4393.85],\n", 85 | " dtype=float32)" 86 | ] 87 | }, 88 | "execution_count": 5, 89 | "metadata": {}, 90 | "output_type": "execute_result" 91 | } 92 | ], 93 | "source": [ 94 | "x[0]" 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": 6, 100 | "metadata": {}, 101 | "outputs": [ 102 | { 103 | "data": { 104 | "text/plain": [ 105 | "0" 106 | ] 107 | }, 108 | "execution_count": 6, 109 | "metadata": {}, 110 | "output_type": "execute_result" 111 | } 112 | ], 113 | "source": [ 114 | "y[0]" 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "# Normalization of data\n", 122 | "Normalization of data is required before using it on deep neural Nueral Networks" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": 7, 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "from sklearn.preprocessing import MinMaxScaler\n", 132 | "scaler = MinMaxScaler()" 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": 8, 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "scaler.fit(x)\n", 142 | "x_new = scaler.transform(x)" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": 9, 148 | "metadata": {}, 149 | "outputs": [], 150 | "source": [ 151 | "data_mean = x.mean()\n", 152 | "data_std = x.std()\n", 153 | "x = (x - data_mean)/data_std" 154 | ] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "execution_count": 10, 159 | "metadata": {}, 160 | "outputs": [], 161 | "source": [ 162 | "(x_train, x_test) = x[:11000], x[11000:]\n", 163 | "(y_train, y_test) = y[:11000], y[11000:]" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "# Training on long short term memory (LSTM) model" 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": {}, 176 | "source": [ 177 | "# Creating 3D array dataset for LSTM model\n", 178 | "# dataset for lstm and orther 1D signal based on 3D tensor.\n", 179 | "# you shoudl provide tempral information, number of channel and number of classes\n", 180 | "# dataset=samplesxfeaturesxchannels\n", 181 | "# dataset=11000x14x1\n", 182 | "# number of classes" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": 13, 188 | "metadata": {}, 189 | "outputs": [], 190 | "source": [ 191 | "X_train = np.asarray(np.reshape(x_train, (x_train.shape[0], 14, 1)))\n", 192 | "X_test = np.asarray(np.reshape(x_test, (x_test.shape[0], 14, 1)))" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 14, 198 | "metadata": {}, 199 | "outputs": [ 200 | { 201 | "name": "stderr", 202 | "output_type": "stream", 203 | "text": [ 204 | "Using TensorFlow backend.\n" 205 | ] 206 | } 207 | ], 208 | "source": [ 209 | "from keras.layers import Dense, Dropout, CuDNNLSTM, Embedding, LSTM\n", 210 | "from keras.models import Sequential" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": 16, 216 | "metadata": {}, 217 | "outputs": [ 218 | { 219 | "name": "stderr", 220 | "output_type": "stream", 221 | "text": [ 222 | "W0714 10:06:20.872536 10284 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n", 223 | "\n", 224 | "W0714 10:06:20.885502 10284 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:3376: The name tf.log is deprecated. Please use tf.math.log instead.\n", 225 | "\n", 226 | "W0714 10:06:20.889492 10284 deprecation.py:323] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\ops\\nn_impl.py:180: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 227 | "Instructions for updating:\n", 228 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n" 229 | ] 230 | }, 231 | { 232 | "name": "stdout", 233 | "output_type": "stream", 234 | "text": [ 235 | "_________________________________________________________________\n", 236 | "Layer (type) Output Shape Param # \n", 237 | "=================================================================\n", 238 | "lstm_3 (LSTM) (None, 14, 256) 264192 \n", 239 | "_________________________________________________________________\n", 240 | "lstm_4 (LSTM) (None, 256) 525312 \n", 241 | "_________________________________________________________________\n", 242 | "dense_2 (Dense) (None, 1) 257 \n", 243 | "=================================================================\n", 244 | "Total params: 789,761\n", 245 | "Trainable params: 789,761\n", 246 | "Non-trainable params: 0\n", 247 | "_________________________________________________________________\n" 248 | ] 249 | } 250 | ], 251 | "source": [ 252 | "model1 = Sequential()\n", 253 | "model1.add(LSTM(256,input_shape=(14, 1), return_sequences=True))\n", 254 | "model1.add(LSTM(256))\n", 255 | "model1.add(Dense(1, activation='sigmoid'))\n", 256 | "model1.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n", 257 | "model1.summary()" 258 | ] 259 | }, 260 | { 261 | "cell_type": "code", 262 | "execution_count": 17, 263 | "metadata": {}, 264 | "outputs": [ 265 | { 266 | "name": "stderr", 267 | "output_type": "stream", 268 | "text": [ 269 | "W0714 10:06:44.772954 10284 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:986: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n", 270 | "\n" 271 | ] 272 | }, 273 | { 274 | "name": "stdout", 275 | "output_type": "stream", 276 | "text": [ 277 | "Train on 9900 samples, validate on 1100 samples\n", 278 | "Epoch 1/100\n", 279 | " - 8s - loss: 0.6915 - acc: 0.5228 - val_loss: 0.7011 - val_acc: 0.4909\n", 280 | "\n", 281 | "Epoch 00001: val_loss improved from inf to 0.70114, saving model to LSTM.weights.best.hdf5\n", 282 | "Epoch 2/100\n", 283 | " - 7s - loss: 0.6896 - acc: 0.5333 - val_loss: 0.6876 - val_acc: 0.5282\n", 284 | "\n", 285 | "Epoch 00002: val_loss improved from 0.70114 to 0.68760, saving model to LSTM.weights.best.hdf5\n", 286 | "Epoch 3/100\n", 287 | " - 7s - loss: 0.6680 - acc: 0.5997 - val_loss: 0.6747 - val_acc: 0.5291\n", 288 | "\n", 289 | "Epoch 00003: val_loss improved from 0.68760 to 0.67472, saving model to LSTM.weights.best.hdf5\n", 290 | "Epoch 4/100\n", 291 | " - 7s - loss: 0.6453 - acc: 0.6363 - val_loss: 0.6649 - val_acc: 0.5482\n", 292 | "\n", 293 | "Epoch 00004: val_loss improved from 0.67472 to 0.66485, saving model to LSTM.weights.best.hdf5\n", 294 | "Epoch 5/100\n", 295 | " - 7s - loss: 0.6455 - acc: 0.6152 - val_loss: 0.6768 - val_acc: 0.5518\n", 296 | "\n", 297 | "Epoch 00005: val_loss did not improve from 0.66485\n", 298 | "Epoch 6/100\n", 299 | " - 7s - loss: 0.6160 - acc: 0.6815 - val_loss: 0.7218 - val_acc: 0.5773\n", 300 | "\n", 301 | "Epoch 00006: val_loss did not improve from 0.66485\n", 302 | "Epoch 7/100\n", 303 | " - 7s - loss: 0.6001 - acc: 0.6713 - val_loss: 0.6417 - val_acc: 0.5800\n", 304 | "\n", 305 | "Epoch 00007: val_loss improved from 0.66485 to 0.64171, saving model to LSTM.weights.best.hdf5\n", 306 | "Epoch 8/100\n", 307 | " - 7s - loss: 0.6468 - acc: 0.6321 - val_loss: 0.6569 - val_acc: 0.5455\n", 308 | "\n", 309 | "Epoch 00008: val_loss did not improve from 0.64171\n", 310 | "Epoch 9/100\n", 311 | " - 7s - loss: 0.6258 - acc: 0.6418 - val_loss: 0.6744 - val_acc: 0.5927\n", 312 | "\n", 313 | "Epoch 00009: val_loss did not improve from 0.64171\n", 314 | "Epoch 10/100\n", 315 | " - 7s - loss: 0.6124 - acc: 0.6551 - val_loss: 0.6347 - val_acc: 0.5891\n", 316 | "\n", 317 | "Epoch 00010: val_loss improved from 0.64171 to 0.63466, saving model to LSTM.weights.best.hdf5\n", 318 | "Epoch 11/100\n", 319 | " - 8s - loss: 0.6053 - acc: 0.6565 - val_loss: 0.6317 - val_acc: 0.6082\n", 320 | "\n", 321 | "Epoch 00011: val_loss improved from 0.63466 to 0.63169, saving model to LSTM.weights.best.hdf5\n", 322 | "Epoch 12/100\n", 323 | " - 8s - loss: 0.5873 - acc: 0.6766 - val_loss: 0.6237 - val_acc: 0.6291\n", 324 | "\n", 325 | "Epoch 00012: val_loss improved from 0.63169 to 0.62368, saving model to LSTM.weights.best.hdf5\n", 326 | "Epoch 13/100\n", 327 | " - 8s - loss: 0.5617 - acc: 0.7134 - val_loss: 0.6145 - val_acc: 0.6673\n", 328 | "\n", 329 | "Epoch 00013: val_loss improved from 0.62368 to 0.61454, saving model to LSTM.weights.best.hdf5\n", 330 | "Epoch 14/100\n", 331 | " - 8s - loss: 0.5750 - acc: 0.7099 - val_loss: 0.6183 - val_acc: 0.5909\n", 332 | "\n", 333 | "Epoch 00014: val_loss did not improve from 0.61454\n", 334 | "Epoch 15/100\n", 335 | " - 8s - loss: 0.5773 - acc: 0.6894 - val_loss: 0.6263 - val_acc: 0.6118\n", 336 | "\n", 337 | "Epoch 00015: val_loss did not improve from 0.61454\n", 338 | "Epoch 16/100\n", 339 | " - 8s - loss: 0.5891 - acc: 0.6766 - val_loss: 0.6207 - val_acc: 0.6445\n", 340 | "\n", 341 | "Epoch 00016: val_loss did not improve from 0.61454\n", 342 | "Epoch 17/100\n", 343 | " - 8s - loss: 0.5802 - acc: 0.6849 - val_loss: 0.6345 - val_acc: 0.6282\n", 344 | "\n", 345 | "Epoch 00017: val_loss did not improve from 0.61454\n", 346 | "Epoch 18/100\n", 347 | " - 8s - loss: 0.5364 - acc: 0.7412 - val_loss: 0.6621 - val_acc: 0.6655\n", 348 | "\n", 349 | "Epoch 00018: val_loss did not improve from 0.61454\n", 350 | "Epoch 19/100\n", 351 | " - 7s - loss: 0.6000 - acc: 0.6692 - val_loss: 0.6215 - val_acc: 0.6355\n", 352 | "\n", 353 | "Epoch 00019: val_loss did not improve from 0.61454\n", 354 | "Epoch 20/100\n", 355 | " - 8s - loss: 0.5363 - acc: 0.7315 - val_loss: 0.6498 - val_acc: 0.6264\n", 356 | "\n", 357 | "Epoch 00020: val_loss did not improve from 0.61454\n", 358 | "Epoch 21/100\n", 359 | " - 8s - loss: 0.5565 - acc: 0.6957 - val_loss: 0.6058 - val_acc: 0.6527\n", 360 | "\n", 361 | "Epoch 00021: val_loss improved from 0.61454 to 0.60575, saving model to LSTM.weights.best.hdf5\n", 362 | "Epoch 22/100\n", 363 | " - 8s - loss: 0.5183 - acc: 0.7372 - val_loss: 0.6067 - val_acc: 0.6745\n", 364 | "\n", 365 | "Epoch 00022: val_loss did not improve from 0.60575\n", 366 | "Epoch 23/100\n", 367 | " - 8s - loss: 0.5405 - acc: 0.7308 - val_loss: 0.7116 - val_acc: 0.4891\n", 368 | "\n", 369 | "Epoch 00023: val_loss did not improve from 0.60575\n", 370 | "Epoch 24/100\n", 371 | " - 8s - loss: 0.7065 - acc: 0.5056 - val_loss: 0.6911 - val_acc: 0.5891\n", 372 | "\n", 373 | "Epoch 00024: val_loss did not improve from 0.60575\n", 374 | "Epoch 25/100\n", 375 | " - 8s - loss: 0.6842 - acc: 0.5648 - val_loss: 0.6857 - val_acc: 0.5055\n", 376 | "\n", 377 | "Epoch 00025: val_loss did not improve from 0.60575\n", 378 | "Epoch 26/100\n", 379 | " - 8s - loss: 0.6745 - acc: 0.5803 - val_loss: 0.6859 - val_acc: 0.5627\n", 380 | "\n", 381 | "Epoch 00026: val_loss did not improve from 0.60575\n", 382 | "Epoch 27/100\n", 383 | " - 8s - loss: 0.6576 - acc: 0.6289 - val_loss: 0.6807 - val_acc: 0.5227\n", 384 | "\n", 385 | "Epoch 00027: val_loss did not improve from 0.60575\n", 386 | "Epoch 28/100\n", 387 | " - 8s - loss: 0.6589 - acc: 0.6111 - val_loss: 0.6727 - val_acc: 0.5418\n", 388 | "\n", 389 | "Epoch 00028: val_loss did not improve from 0.60575\n", 390 | "Epoch 29/100\n", 391 | " - 8s - loss: 0.6522 - acc: 0.6334 - val_loss: 0.7112 - val_acc: 0.5273\n", 392 | "\n", 393 | "Epoch 00029: val_loss did not improve from 0.60575\n", 394 | "Epoch 30/100\n", 395 | " - 8s - loss: 0.6570 - acc: 0.6135 - val_loss: 0.6655 - val_acc: 0.5445\n", 396 | "\n", 397 | "Epoch 00030: val_loss did not improve from 0.60575\n", 398 | "Epoch 31/100\n", 399 | " - 8s - loss: 0.6222 - acc: 0.6767 - val_loss: 0.6413 - val_acc: 0.6445\n", 400 | "\n", 401 | "Epoch 00031: val_loss did not improve from 0.60575\n", 402 | "Epoch 32/100\n", 403 | " - 8s - loss: 0.6263 - acc: 0.6347 - val_loss: 0.6606 - val_acc: 0.6418\n", 404 | "\n", 405 | "Epoch 00032: val_loss did not improve from 0.60575\n", 406 | "Epoch 33/100\n", 407 | " - 8s - loss: 0.5884 - acc: 0.7072 - val_loss: 0.6277 - val_acc: 0.6464\n", 408 | "\n", 409 | "Epoch 00033: val_loss did not improve from 0.60575\n", 410 | "Epoch 34/100\n", 411 | " - 8s - loss: 0.6234 - acc: 0.6391 - val_loss: 0.6479 - val_acc: 0.6745\n", 412 | "\n", 413 | "Epoch 00034: val_loss did not improve from 0.60575\n", 414 | "Epoch 35/100\n", 415 | " - 8s - loss: 0.5874 - acc: 0.7069 - val_loss: 0.6211 - val_acc: 0.6255\n", 416 | "\n", 417 | "Epoch 00035: val_loss did not improve from 0.60575\n", 418 | "Epoch 36/100\n", 419 | " - 8s - loss: 0.5601 - acc: 0.7264 - val_loss: 0.6229 - val_acc: 0.6755\n", 420 | "\n", 421 | "Epoch 00036: val_loss did not improve from 0.60575\n", 422 | "Epoch 37/100\n", 423 | " - 8s - loss: 0.5782 - acc: 0.7007 - val_loss: 0.6165 - val_acc: 0.6427\n", 424 | "\n", 425 | "Epoch 00037: val_loss did not improve from 0.60575\n", 426 | "Epoch 38/100\n", 427 | " - 7s - loss: 0.5503 - acc: 0.7327 - val_loss: 0.6065 - val_acc: 0.6809\n", 428 | "\n", 429 | "Epoch 00038: val_loss did not improve from 0.60575\n", 430 | "Epoch 39/100\n", 431 | " - 8s - loss: 0.5347 - acc: 0.7420 - val_loss: 0.6063 - val_acc: 0.6755\n", 432 | "\n", 433 | "Epoch 00039: val_loss did not improve from 0.60575\n", 434 | "Epoch 40/100\n", 435 | " - 8s - loss: 0.5143 - acc: 0.7553 - val_loss: 0.5911 - val_acc: 0.6791\n", 436 | "\n", 437 | "Epoch 00040: val_loss improved from 0.60575 to 0.59110, saving model to LSTM.weights.best.hdf5\n", 438 | "Epoch 41/100\n", 439 | " - 7s - loss: 0.5188 - acc: 0.7387 - val_loss: 0.6507 - val_acc: 0.6091\n", 440 | "\n", 441 | "Epoch 00041: val_loss did not improve from 0.59110\n", 442 | "Epoch 42/100\n", 443 | " - 8s - loss: 0.5027 - acc: 0.7420 - val_loss: 0.6265 - val_acc: 0.6273\n", 444 | "\n", 445 | "Epoch 00042: val_loss did not improve from 0.59110\n", 446 | "Epoch 43/100\n", 447 | " - 8s - loss: 0.4795 - acc: 0.7589 - val_loss: 0.5549 - val_acc: 0.6936\n", 448 | "\n", 449 | "Epoch 00043: val_loss improved from 0.59110 to 0.55486, saving model to LSTM.weights.best.hdf5\n", 450 | "Epoch 44/100\n", 451 | " - 8s - loss: 0.4622 - acc: 0.7707 - val_loss: 0.5461 - val_acc: 0.7055\n", 452 | "\n", 453 | "Epoch 00044: val_loss improved from 0.55486 to 0.54611, saving model to LSTM.weights.best.hdf5\n", 454 | "Epoch 45/100\n", 455 | " - 8s - loss: 0.4652 - acc: 0.7737 - val_loss: 0.5557 - val_acc: 0.7082\n", 456 | "\n", 457 | "Epoch 00045: val_loss did not improve from 0.54611\n", 458 | "Epoch 46/100\n", 459 | " - 8s - loss: 0.4858 - acc: 0.7557 - val_loss: 0.5986 - val_acc: 0.6836\n", 460 | "\n", 461 | "Epoch 00046: val_loss did not improve from 0.54611\n", 462 | "Epoch 47/100\n", 463 | " - 8s - loss: 0.4644 - acc: 0.7716 - val_loss: 0.5276 - val_acc: 0.7282\n", 464 | "\n", 465 | "Epoch 00047: val_loss improved from 0.54611 to 0.52762, saving model to LSTM.weights.best.hdf5\n", 466 | "Epoch 48/100\n", 467 | " - 8s - loss: 0.4404 - acc: 0.7923 - val_loss: 0.6217 - val_acc: 0.6755\n", 468 | "\n", 469 | "Epoch 00048: val_loss did not improve from 0.52762\n", 470 | "Epoch 49/100\n", 471 | " - 8s - loss: 0.4657 - acc: 0.7700 - val_loss: 0.5274 - val_acc: 0.7291\n", 472 | "\n", 473 | "Epoch 00049: val_loss improved from 0.52762 to 0.52740, saving model to LSTM.weights.best.hdf5\n", 474 | "Epoch 50/100\n", 475 | " - 8s - loss: 0.4229 - acc: 0.7989 - val_loss: 0.5498 - val_acc: 0.7045\n", 476 | "\n", 477 | "Epoch 00050: val_loss did not improve from 0.52740\n", 478 | "Epoch 51/100\n", 479 | " - 8s - loss: 0.4229 - acc: 0.8009 - val_loss: 0.5274 - val_acc: 0.7164\n", 480 | "\n", 481 | "Epoch 00051: val_loss improved from 0.52740 to 0.52738, saving model to LSTM.weights.best.hdf5\n", 482 | "Epoch 52/100\n", 483 | " - 8s - loss: 0.4431 - acc: 0.7817 - val_loss: 0.5756 - val_acc: 0.7118\n", 484 | "\n", 485 | "Epoch 00052: val_loss did not improve from 0.52738\n", 486 | "Epoch 53/100\n", 487 | " - 8s - loss: 0.4415 - acc: 0.7859 - val_loss: 0.5284 - val_acc: 0.7236\n", 488 | "\n", 489 | "Epoch 00053: val_loss did not improve from 0.52738\n", 490 | "Epoch 54/100\n", 491 | " - 8s - loss: 0.4099 - acc: 0.8072 - val_loss: 0.5251 - val_acc: 0.7318\n", 492 | "\n", 493 | "Epoch 00054: val_loss improved from 0.52738 to 0.52507, saving model to LSTM.weights.best.hdf5\n", 494 | "Epoch 55/100\n" 495 | ] 496 | }, 497 | { 498 | "name": "stdout", 499 | "output_type": "stream", 500 | "text": [ 501 | " - 8s - loss: 0.4203 - acc: 0.8017 - val_loss: 0.5623 - val_acc: 0.7082\n", 502 | "\n", 503 | "Epoch 00055: val_loss did not improve from 0.52507\n", 504 | "Epoch 56/100\n", 505 | " - 8s - loss: 0.4306 - acc: 0.7930 - val_loss: 0.5636 - val_acc: 0.7318\n", 506 | "\n", 507 | "Epoch 00056: val_loss did not improve from 0.52507\n", 508 | "Epoch 57/100\n", 509 | " - 8s - loss: 0.4129 - acc: 0.8077 - val_loss: 0.5946 - val_acc: 0.6727\n", 510 | "\n", 511 | "Epoch 00057: val_loss did not improve from 0.52507\n", 512 | "Epoch 58/100\n", 513 | " - 8s - loss: 0.4267 - acc: 0.7929 - val_loss: 0.5476 - val_acc: 0.7209\n", 514 | "\n", 515 | "Epoch 00058: val_loss did not improve from 0.52507\n", 516 | "Epoch 59/100\n", 517 | " - 8s - loss: 0.4217 - acc: 0.7956 - val_loss: 0.5062 - val_acc: 0.7391\n", 518 | "\n", 519 | "Epoch 00059: val_loss improved from 0.52507 to 0.50621, saving model to LSTM.weights.best.hdf5\n", 520 | "Epoch 60/100\n", 521 | " - 8s - loss: 0.3986 - acc: 0.8135 - val_loss: 0.5230 - val_acc: 0.7482\n", 522 | "\n", 523 | "Epoch 00060: val_loss did not improve from 0.50621\n", 524 | "Epoch 61/100\n", 525 | " - 8s - loss: 0.4053 - acc: 0.8086 - val_loss: 0.5262 - val_acc: 0.7436\n", 526 | "\n", 527 | "Epoch 00061: val_loss did not improve from 0.50621\n", 528 | "Epoch 62/100\n", 529 | " - 8s - loss: 0.3948 - acc: 0.8175 - val_loss: 0.5491 - val_acc: 0.7100\n", 530 | "\n", 531 | "Epoch 00062: val_loss did not improve from 0.50621\n", 532 | "Epoch 63/100\n", 533 | " - 8s - loss: 0.4009 - acc: 0.8128 - val_loss: 0.6121 - val_acc: 0.7082\n", 534 | "\n", 535 | "Epoch 00063: val_loss did not improve from 0.50621\n", 536 | "Epoch 64/100\n", 537 | " - 8s - loss: 0.4268 - acc: 0.7992 - val_loss: 0.5777 - val_acc: 0.7073\n", 538 | "\n", 539 | "Epoch 00064: val_loss did not improve from 0.50621\n", 540 | "Epoch 65/100\n", 541 | " - 8s - loss: 0.4033 - acc: 0.8107 - val_loss: 0.5169 - val_acc: 0.7427\n", 542 | "\n", 543 | "Epoch 00065: val_loss did not improve from 0.50621\n", 544 | "Epoch 66/100\n", 545 | " - 8s - loss: 0.3916 - acc: 0.8183 - val_loss: 0.5664 - val_acc: 0.7227\n", 546 | "\n", 547 | "Epoch 00066: val_loss did not improve from 0.50621\n", 548 | "Epoch 67/100\n", 549 | " - 8s - loss: 0.3957 - acc: 0.8130 - val_loss: 0.5201 - val_acc: 0.7364\n", 550 | "\n", 551 | "Epoch 00067: val_loss did not improve from 0.50621\n", 552 | "Epoch 68/100\n", 553 | " - 8s - loss: 0.3825 - acc: 0.8198 - val_loss: 0.5324 - val_acc: 0.7436\n", 554 | "\n", 555 | "Epoch 00068: val_loss did not improve from 0.50621\n", 556 | "Epoch 69/100\n", 557 | " - 8s - loss: 0.3833 - acc: 0.8194 - val_loss: 0.5104 - val_acc: 0.7364\n", 558 | "\n", 559 | "Epoch 00069: val_loss did not improve from 0.50621\n", 560 | "Epoch 70/100\n", 561 | " - 8s - loss: 0.3851 - acc: 0.8165 - val_loss: 0.5221 - val_acc: 0.7409\n", 562 | "\n", 563 | "Epoch 00070: val_loss did not improve from 0.50621\n", 564 | "Epoch 71/100\n", 565 | " - 8s - loss: 0.3789 - acc: 0.8217 - val_loss: 0.4980 - val_acc: 0.7509\n", 566 | "\n", 567 | "Epoch 00071: val_loss improved from 0.50621 to 0.49805, saving model to LSTM.weights.best.hdf5\n", 568 | "Epoch 72/100\n", 569 | " - 8s - loss: 0.3839 - acc: 0.8212 - val_loss: 0.5342 - val_acc: 0.7300\n", 570 | "\n", 571 | "Epoch 00072: val_loss did not improve from 0.49805\n", 572 | "Epoch 73/100\n", 573 | " - 7s - loss: 0.3839 - acc: 0.8217 - val_loss: 0.5006 - val_acc: 0.7400\n", 574 | "\n", 575 | "Epoch 00073: val_loss did not improve from 0.49805\n", 576 | "Epoch 74/100\n", 577 | " - 8s - loss: 0.3706 - acc: 0.8264 - val_loss: 0.5011 - val_acc: 0.7427\n", 578 | "\n", 579 | "Epoch 00074: val_loss did not improve from 0.49805\n", 580 | "Epoch 75/100\n", 581 | " - 8s - loss: 0.3918 - acc: 0.8133 - val_loss: 0.5142 - val_acc: 0.7473\n", 582 | "\n", 583 | "Epoch 00075: val_loss did not improve from 0.49805\n", 584 | "Epoch 76/100\n", 585 | " - 8s - loss: 0.3665 - acc: 0.8326 - val_loss: 0.5208 - val_acc: 0.7600\n", 586 | "\n", 587 | "Epoch 00076: val_loss did not improve from 0.49805\n", 588 | "Epoch 77/100\n", 589 | " - 8s - loss: 0.3506 - acc: 0.8444 - val_loss: 0.5182 - val_acc: 0.7509\n", 590 | "\n", 591 | "Epoch 00077: val_loss did not improve from 0.49805\n", 592 | "Epoch 78/100\n", 593 | " - 8s - loss: 0.3543 - acc: 0.8403 - val_loss: 0.5159 - val_acc: 0.7464\n", 594 | "\n", 595 | "Epoch 00078: val_loss did not improve from 0.49805\n", 596 | "Epoch 79/100\n", 597 | " - 8s - loss: 0.4200 - acc: 0.7980 - val_loss: 0.5585 - val_acc: 0.7036\n", 598 | "\n", 599 | "Epoch 00079: val_loss did not improve from 0.49805\n", 600 | "Epoch 80/100\n", 601 | " - 8s - loss: 0.4045 - acc: 0.8131 - val_loss: 0.5354 - val_acc: 0.7364\n", 602 | "\n", 603 | "Epoch 00080: val_loss did not improve from 0.49805\n", 604 | "Epoch 81/100\n", 605 | " - 8s - loss: 0.3953 - acc: 0.8143 - val_loss: 0.5422 - val_acc: 0.7218\n", 606 | "\n", 607 | "Epoch 00081: val_loss did not improve from 0.49805\n", 608 | "Epoch 82/100\n", 609 | " - 8s - loss: 0.3752 - acc: 0.8273 - val_loss: 0.5231 - val_acc: 0.7418\n", 610 | "\n", 611 | "Epoch 00082: val_loss did not improve from 0.49805\n", 612 | "Epoch 83/100\n", 613 | " - 12s - loss: 0.3858 - acc: 0.8222 - val_loss: 0.6047 - val_acc: 0.6655\n", 614 | "\n", 615 | "Epoch 00083: val_loss did not improve from 0.49805\n", 616 | "Epoch 84/100\n", 617 | " - 11s - loss: 0.4023 - acc: 0.8132 - val_loss: 0.5409 - val_acc: 0.7291\n", 618 | "\n", 619 | "Epoch 00084: val_loss did not improve from 0.49805\n", 620 | "Epoch 85/100\n", 621 | " - 11s - loss: 0.3707 - acc: 0.8323 - val_loss: 0.5435 - val_acc: 0.7464\n", 622 | "\n", 623 | "Epoch 00085: val_loss did not improve from 0.49805\n", 624 | "Epoch 86/100\n", 625 | " - 12s - loss: 0.3728 - acc: 0.8299 - val_loss: 0.5486 - val_acc: 0.7209\n", 626 | "\n", 627 | "Epoch 00086: val_loss did not improve from 0.49805\n", 628 | "Epoch 87/100\n", 629 | " - 12s - loss: 0.3635 - acc: 0.8339 - val_loss: 0.5314 - val_acc: 0.7527\n", 630 | "\n", 631 | "Epoch 00087: val_loss did not improve from 0.49805\n", 632 | "Epoch 88/100\n", 633 | " - 11s - loss: 0.3572 - acc: 0.8396 - val_loss: 0.5337 - val_acc: 0.7518\n", 634 | "\n", 635 | "Epoch 00088: val_loss did not improve from 0.49805\n", 636 | "Epoch 89/100\n", 637 | " - 11s - loss: 0.3355 - acc: 0.8528 - val_loss: 0.5386 - val_acc: 0.7436\n", 638 | "\n", 639 | "Epoch 00089: val_loss did not improve from 0.49805\n", 640 | "Epoch 90/100\n", 641 | " - 11s - loss: 0.3550 - acc: 0.8375 - val_loss: 0.5825 - val_acc: 0.7327\n", 642 | "\n", 643 | "Epoch 00090: val_loss did not improve from 0.49805\n", 644 | "Epoch 91/100\n", 645 | " - 11s - loss: 0.3509 - acc: 0.8432 - val_loss: 0.5687 - val_acc: 0.7318\n", 646 | "\n", 647 | "Epoch 00091: val_loss did not improve from 0.49805\n", 648 | "Epoch 92/100\n", 649 | " - 11s - loss: 0.3716 - acc: 0.8318 - val_loss: 0.5146 - val_acc: 0.7600\n", 650 | "\n", 651 | "Epoch 00092: val_loss did not improve from 0.49805\n", 652 | "Epoch 93/100\n", 653 | " - 11s - loss: 0.3340 - acc: 0.8567 - val_loss: 0.5085 - val_acc: 0.7482\n", 654 | "\n", 655 | "Epoch 00093: val_loss did not improve from 0.49805\n", 656 | "Epoch 94/100\n", 657 | " - 11s - loss: 0.3240 - acc: 0.8563 - val_loss: 0.5389 - val_acc: 0.7418\n", 658 | "\n", 659 | "Epoch 00094: val_loss did not improve from 0.49805\n", 660 | "Epoch 95/100\n", 661 | " - 11s - loss: 0.3484 - acc: 0.8460 - val_loss: 0.5948 - val_acc: 0.7173\n", 662 | "\n", 663 | "Epoch 00095: val_loss did not improve from 0.49805\n", 664 | "Epoch 96/100\n", 665 | " - 12s - loss: 0.3399 - acc: 0.8467 - val_loss: 0.5641 - val_acc: 0.7382\n", 666 | "\n", 667 | "Epoch 00096: val_loss did not improve from 0.49805\n", 668 | "Epoch 97/100\n", 669 | " - 12s - loss: 0.3399 - acc: 0.8497 - val_loss: 0.5064 - val_acc: 0.7464\n", 670 | "\n", 671 | "Epoch 00097: val_loss did not improve from 0.49805\n", 672 | "Epoch 98/100\n", 673 | " - 12s - loss: 0.3282 - acc: 0.8522 - val_loss: 0.5117 - val_acc: 0.7600\n", 674 | "\n", 675 | "Epoch 00098: val_loss did not improve from 0.49805\n", 676 | "Epoch 99/100\n", 677 | " - 11s - loss: 0.3463 - acc: 0.8425 - val_loss: 0.5101 - val_acc: 0.7718\n", 678 | "\n", 679 | "Epoch 00099: val_loss did not improve from 0.49805\n", 680 | "Epoch 100/100\n", 681 | " - 11s - loss: 0.3103 - acc: 0.8614 - val_loss: 0.5328 - val_acc: 0.7445\n", 682 | "\n", 683 | "Epoch 00100: val_loss did not improve from 0.49805\n" 684 | ] 685 | } 686 | ], 687 | "source": [ 688 | "from keras.callbacks import ModelCheckpoint\n", 689 | "\n", 690 | "checkpointer = ModelCheckpoint(filepath = 'LSTM.weights.best.hdf5', verbose = 1, save_best_only = True)\n", 691 | "hist = model1.fit(X_train, y_train, epochs = 100, batch_size=256, validation_split = 0.1, callbacks = [checkpointer], verbose = 2, shuffle = True)" 692 | ] 693 | }, 694 | { 695 | "cell_type": "code", 696 | "execution_count": 21, 697 | "metadata": {}, 698 | "outputs": [ 699 | { 700 | "data": { 701 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydd3hUxdeA30kvkEYPoffQOwgIiDRBRAGVIiAC/gQs6GdHXXpviiJIlab0XqWFLiWUEGogIYGEGhICqbvn++NuloRsCiUE8L7Pc5/snXru3c2cmTMzZ5SIoKOjo6Pz38UmpwXQ0dHR0clZdEWgo6Oj8x9HVwQ6Ojo6/3F0RaCjo6PzH0dXBDo6Ojr/cXRFoKOjo/MfR1cEOjmCUqq4UkqUUnZZSNtTKbX7acj1X0IpFayUejWn5dDJeXRFoJMp5gYjQSmV94Hwo+bGvHjOSJZKFlelVIxSan1Oy/IioJSao5QaltNy6DwddEWgk1UuAp2Tb5RSlQHnnBMnDR2BeKCFUqrQ06w4K6MaHZ1nGV0R6GSVeUD3FPc9gD9TJlBKuSul/lRKXVdKhSilBimlbMxxtkqpcUqpG0qpC0AbK3lnKqXClVKXlVLDlFK2DyFfD+B34DjQ9YGyiyillpvluqmUmpIiro9S6pRS6o5SKlApVcMcLkqp0inSWXrISqkmSqkwpdTXSqkIYLZSylMptdZcR6T5s0+K/F5KqdlKqSvm+JXm8ACl1Osp0tmb31G1Bx8wC3XsUEoNVUrtMT/P5pSjOKXUe+bv5aZS6vuHeLcPyvGSUuqgUirK/PelFHE9lVIXzPVfVEp1NYeXVkrtNOe5oZT6+1Hr13ny6IpAJ6vsB9yUUhXMDfQ7wPwH0vwCuAMlgcZoiuN9c1wfoC1QHaiF1oNPyVwgCShtTtMC6J0VwZRSRYEmwALz1T1FnC2wFggBigOFgb/McZ0Agzm9G9AOuJmVOoGCgBdQDOiL9r8023xfFIgFpqRIPw9wASoC+YGJ5vA/gW4p0r0GhIvIUSt1ZlYHQBe0d54fcAD+z/ysvsBU4D3AG8gD+PCQKKW8gHXAz+YyJgDrlFJ5lFKu5vDWIpIbeAlIfo6hwGbA01zvLw9bt042IiL6pV8ZXkAw8CowCBgJtAK2AHaAoDWwtmimGd8U+T4Edpg/bwP+lyKuhTmvHVDAnNc5RXxnYLv5c09gdwbyDQKOmj97A0aguvm+PnAdsLOSbxPwaTplClA6xf0cYJj5cxMgAXDKQKZqQKT5cyHABHhaSecN3AHczPdLga+y+L1Y6jDf7wAGpbjvB2w0f/4R+CtFnKv5GV5Np2zL8z4Q/h7w7wNh+8zfkStwG+iQ8rs0p/kTmA745PTvWb/SXvqIQOdhmIfW4+zJA2YhIC9aDzQkRVgIWg8ctAYv9IG4ZIoB9kC4Uuq2Uuo2MA2tV5sVuqONBBCRK8BONFMRQBEgRESSrOQrAgRlsY4HuS4icck3SikXpdQ0s+klGvADPMwjkiLALRGJfLAQs7x7gA5KKQ+gdfKzPEgmdSQTkeLzPSCX+XOq9y8id8n66Ccl3qT+7jDfFzaX+Q7wP7Tvcp1Sqrw5zVeAAv5VSp1USvV6hLp1sgldEehkGREJQZs0fg1Y/kD0DSARrVFPpihw2fw5HK1BTBmXTCjaiCCviHiYLzcRqZiZTGb7dBngW6VUhNlmXxfobJ7EDQWKpjOhGwqUSqfoe2imnGQKPhD/oNveL4ByQF0RcQNeThbRXI+XuaG3xlw081AnYJ+IXE4nXUZ1ZEaq96+UckEz7TwsV0j9HUOK71lENolIc7RR0GngD3N4hIj0ERFvtJHibynnYHRyFl0R6DwsHwCvmHt/FkTECCwGhiulciuligGfc38eYTHwiVLKRynlCXyTIm84mv14vFLKTSllo5QqpZRqnAV5eqCZqXzRTCXVgEpojXhr4F+0RnCU0paYOimlGpjzzgD+TylVU2mUNssNmm27i9ImuVuhzXlkRG40m/1tsx39pweebwNa4+dpnhB+OUXelUAN4FPSjrSyVEcWWAq0VUo1VEo5AEPI/P/f1vy+ki8HYD1QVinVRSllp5R6B+3dr1VKFVBKtTPPFcQDMWhmOpRSnVJMbEeiKVLjQ8ivk43oikDnoRCRIBE5lE70x8Bd4AKwG1gIzDLH/YFmkz8GHCHtiKI7mmkpEK2hWIrWq0wXpZQT8Dbwi7nHmXxdRDNj9TArqNfRJqEvAWFo5gtEZAkw3CznHbQG2ctc/KfmfLfRViGtzEgWYBLactobaBPrGx+Ifw9txHQauAZ8lhwhIrHAMqCElffyMHWki4icBPqjPWs42jsOyyTbN2iKJ/naJiI30Sb9v0AzLX0FtBWRG2jtyRdoo4ZbaMqzn7ms2sABpVQMsBptbuZiVuXXyV6UiH4wjY5OTqOU+hEoKyLdMk2so/OE0TfC6OjkMGYzzwdoowYdnaeObhrS0clBlFJ90CaTN4iIX07Lo/PfRDcN6ejo6PzH0UcEOjo6Ov9xnrs5grx580rx4sVzWgwdHR2d54rDhw/fEJF81uKeO0VQvHhxDh1Kb/Wijo6Ojo41lFIP7gi3oJuGdHR0dP7j6IpAR0dH5z+Orgh0dHR0/uPoikBHR0fnP46uCHR0dHSeceKT4rl291q2lf/crRrS0dHReZYJuBbA6jOrcbR1xNnemQ+qf4CjneNjlTnUbyjTDk/jZL+T5HfN6jEdWUdXBDo6OjpPCJOY6LysMwHXAixhPav1zFLe2MRYjl89Tmh0KG3LtsXJzgmAw1cOM2r3KLpV6ZYtSgB005COjo5OuogIJjExavcoLkReyDT9ssBlBFwL4M/2fxL1TRRX/+8qznbO3Eu8x2cbPyMyVjukziQmrsZcteQbsH4AuUfmpt7MenRa0omXZr7EhcgLJBgT6LmqJwVyFWBiy4npVfvY6CMCHR2dZ4btF7cTERPBa2Vew93JPUdlmbR/EvvC9jH61dGM2TOGBScWsLfXXnI75k43j52NHa1Lt6ZL5S7Y2tji5ugGwP6w/fx28Ddm+s9EobiTcAeAm1/dxMvZi0ZFG+Hh5EHNQjWJN8bz0bqPGLd3HK1LtybgWgBrO6/F09kz2571uXM6V6tWLdF3FuvovHj8dvA3+q/vD4CDrQMtSrXgu4bfUb9I/acqh4jw+6Hf6be+Hx0qdOCvjn+xI3gHrea34vVyrzOp5SSKeRSzyNy4WGMq5s/0VFX2hu5lztE5uNi74OboRj6XfHSr0s1qA38x8iIFchXAxd6FwOuB+ObzfeznUkodFpFaVuN0RaCjo/MssOLUChYGLOTTup+y4tQKZh2dxbsV32Vq26lPpf4kUxILji9g/L7xnLh2glalW7HynZWWid7J+yfz2SbtYLnDfQ9TyrMUFX6tQFxSHKveXcW5W+foUrmLxbb/rKErAh0dnWeWE1dPULlAZUDrjSul0nx+UsQmxvLzgZ8plLsQ1QtWp7RXaS5EXqBi/oqYxET5KeVxtHPk83qf07VKVxxsHSx5RYQRu0aQYExgQJ0B5HPNx8XIi7SY34ILkRcwiYnlby/nzQpvPlGZnxS6ItDR0clx4pPi0yyj/OfCP7Re0Jrd7++mrk/dLOezRmxiLEcjjlKncB1sbWytpknZq0/G3dGda19ew8HWgcvRl/HO7f1QCuja3Wu0W9QOG2XD7l67sVHP5hqcjBSBPlmso6OTbSQYE9hwbgNzj81l7dm19K7Rm19f+xWlFDEJMfRZ04eSniWpUqCK1fzvrXiP4NvB7Hp/V4b13Eu8R5uFbdgRvIMPqn/AjHYzrKYbUGcAvvl8KexWGP9wf07fOE1N75ootIa/sFvhh37G/K752ffBPoxifGaVQGboikBHRwcRwShG7GxSNwkRMRHMOzYPe1t7nO2c8XDyoK5PXYq5F0Mpxflb5xm/dzzLTy9HoXB1cMXJzomNXTdSxL0IE/dN5Jut35DfNT/NSjZj6qGpFM5dmO9f/p5B2wZZGnlne2ercpXwKMHCEwu5ce8GeV3yWk0TlxRH+7/aszN4J4MaDaJt2bYAhN8JZ0ngEgbUGUCCMYHo+Gjyu+aneanmAE9kAjYZpRR26vltTp9fyXV0dKxy8tpJPt34KU52TnSv2p125dplOIEZcC2Ad5a+Q8jtELpV6Ub/2v2pkK8CdjZ2uDm6MXH/RMJjwlPl+abBN4x8dSQXIy8y++hs2pdvj7ujO3cT7xKbFGsxrTQo2oC1ndfSsnRLbJUtY/eOpUvlLuwN3cvPB36mf+3+NCzaMF3Z2pZty1C/oWw8v5FuVbqliU8wJtBxcUe2XNjC7Ddmp9q8Nct/FoO2D2Lt2bWU9irNksAlBPYLJJ+r1bNZ/tPocwQ6Os8p4XfCaTG/BWXzlKVPjT40L9kcG2XDS7NeIuhWEI52joRFh+Hu6M7wV4bTv05/q+Vcv3udVgta4ZvPl6WBS4lLisPHzYegT4JwsHUgNjGWuKQ44pLiiIiJYF/YPqoXrE79IvUxiYnI2EjyuOR5KNmnHZrG+H3jOdz3cIbr8k1iwnu8N02KN+Gvjn8B2pLNSfsncWbAGYxipOfKnjQq2ogPa32YKq+IMP3wdAZuGkhsUizvV3ufWW/Meig5XyQymiNARLLtAloBZ4DzwDdW4osC2wF/4DjwWmZl1qxZU3R0nmeu370u68+ul4SkhHTT3I69LdMPTZd7CfesxptMJmk1v5U4DXOSvGPyCgZk0NZBIiJy7uY5ibgTIUnGJPkn6B95e8nbciDsQKr8AVcDpPuK7hKXGGcpT0Tkxt0bMnbPWPl84+dy696tJ/G46ZJcd2b0WtlLPEZ5SEJSgqw8tVKUQUmDmQ3SyJ4egdcC5estX0tkbORjy/w8AxySdNrVbBsRKKVsgbNAcyAMOAh0FpHAFGmmA/4iMlUp5QusF5HiGZWrjwh0nmdEhNYLWrMpaBM+bj70q9WPvjX7pupRJxoTaTm/JTdjb3K47+E0dnvQ/M/Um1mPSS0n0admH1adXkW1gtUok6dMhvUH3Qri2t1rtFnYBic7J3a9v4tSXqWe+HM+SfaF7iPgWgBVC1bllbmvUCFfBfx6+qU7r6BjnZxaNVQHOC8iF8xC/AW8AQSmSCOAm/mzO3AlG+XR0ck29lzawwz/GZy+cZr6PvWZ0HKC1XRJpiTKeJWhpGdJzt48y3fbvmPE7hEc7HOQ8nnLIyJ8tO4jtgdvZ277udjZ2HE34S6uDq6pyqnpXZMTH52gXJ5yKKXoVLFTpjLOOTqHvmv6Ym9rj3dubzZ320wJzxJP5Pmzk/pF6lPKqxS1/6iNu5M7q95dpSuBJ0x2KoLCQGiK+zDgwYXCBmCzUupjwBV41VpBSqm+QF+AokWLPnFBdXQel8t3LrPx/EbyuuRl4v6JNCnehHbl2qVJZ29rzy+v/WK5D7gWwOKTiymXpxwAw3cNZ6b/TH54+Qe6V+1O8O1gXp79MkObDqVpiaYcCDuAo50j7cq1o3ze8g8lY/vy7Zny7xRslA1ru6zNNk+W2YFJTJT0LMnY5mPxzu2d0+K8cGSnaagT0FJEepvv3wPqiMjHKdJ8bpZhvFKqPjATqCQipvTK1U1DOjmBSUx8uflLXivzGs1KNrOEn791nsK5C+Nk54RSigRjArX/qM31u9cJ7B+Ih5OHJe1s/9mU9ipNo2KNrNZx/tZ5yvxShs6VOrPgrQUopUg0JtJqQSu2XdxmSedi78KB3geolL/SIz2HQj3xHbtPA8mGncb/JXLKNBQGFElx70Na088HaBPKiMg+pZQTkBfIvqN4dHQege0XtzNh/wQm7J9AzLcxuDq4IiK8s/QdXO1d8XvfD9Ccpc1qN4tRe0aRaEy05L8UdYn+6/vTpmybdBWBiDCy2Ug+q/eZpcGzt7Vn2dvLmLx/Mvlc81GncB0q56/8yAedPK8bngBdCWQn6c0iP+6FpmQuACUAB+AYUPGBNBuAnubPFdAUhcqoXH3VkE5OMWHvBMGADN4xWEREdoXsEgzI7wd/t5reaDLKkpNLpNPiTuIy3EWchzlLcGTw0xRZR8cCGawayrbugYgkAQOATcApYLGInFRKDVFKJRtPvwD6KKWOAYvMSuH52tig88ITfkfbTDWw/kA6+XZi9J7RXI6+zOQDk/F08uS9qu9ZzRd0K4hhfsPwC/GjR9Ue7O612+K+WEfnWSJbdxaLyHpg/QNhP6b4HAg0yE4ZdHQyI9GYSPDtYEp7lU5jftgbupcmc5qw/J3ltC3bllGvjmLXpV1sOL+B5aeW8+VLX+Ji72K1XGd7Z3597Vfq+dRL1wmajs6zwPNrMNTReQzuJd7DZF6TsPbsWspOKcvE/amPAkwwJtBvXT/yu+anSfEmAJT0LEnIZyFEx0ejUPSvbX23LoCPmw8NijbQlYDOI3Ep6hK9VvXiYuTFbK9LVwQ6/zlux93m1T9fZeDGgQA0Kd6EUp6l+Hbrt6kOHf9s42ccu3qMX1/7lVwOuSzhDrYODKw3kIB+ARRxL5KmfB2dx+XszbM0nNWQ2UdnY9hpyPb6dEWg85/iasxVmsxpwqErh3i52MsAeDp7sveDvXg4edBteTfik+KZ7T+bqYem8uVLX/JG+TfSlKOUeuh1/Do6WeFYxDEazW5EvDGeduXasfDEQkKjQjPP+BjoikDnP0P4nXBenvMyZ2+eZU3nNXTw7WCJy++an5ntZnLs6jF+3P4jUfFRtCjVghHNRuSgxC8+9xLv8e7Sd/m/zf9HXFJcTovzyEzaP4k+q/s89jMcCT9Ck7lNcLR1ZNf7u/i51c+ICJP2T3pCklpH9z6q80Kw8fxG9oft58fGP1pdKy8iNPuzGQcuH2BTt03puj7+aftPvFLiFRoXb4xJTM/1uvtnnXuJ93h90etsv7gdQahSoAp/dfiLCvkq5Ig8CcYEbJSNVd9OGXEr9hY+E3yITYqlafGmrHp3VYYeVdMj2RzkYu/Czp47LSvMui3vxqozq7j02SWrB91nlYw2lOm/cp3nng3nNtBuUTsG7xzMuL3jrKZRSvFdo++Y/cbsDP3fD246mMbFGwPP9+arp0nI7RAGbRtEZGxklvPEJsbyxl9vsP3idv5880/Wdl7LlTtXqDm9JkN3DuVS1KVslDgtN+/dpMC4AtgPtcdjlAelfi7Fj9t/JD4pPtO80w5NIzYplh9e/gG/ED+az2vOrdhbD1V/WHQYzedpB+Zsfm9zqmXGX770JTEJMfx+6PeHe6iHIb0NBs/qpW8o00nJjos7xGmYk1T/vbp8uuFTORp+NE2a6LjoHJAs+1l/dr34/uorree3loXHF8rdhLtPXYatF7ZKntF5BAPSd3XfTNPHJcbJilMrpMHMBqIMSub4z7HEXYm+Im0WtBEMCAakyZwmsuPijicqb2xirMzxn5PmXSVvFvxy85fyyfpP5LUFrwkGpMKUCrIvdF+65cUnxUuhcYWkxbwWIiKy8tRKcRjqICUnl5T5x+ZLkjEpU5lCo0KlwpQKkntEbjl85bDVNC3ntZQCYwtIbGLsQzxtashgQ1mON+wPe+mK4D7nbp7L1Bf7i4zJZJIGMxtIhSkV5FrMtVRxcYlxEpcYJ1svbBXv8d4y9eDUHJLyyROXGCefbfhMMCDlfiknRSYUEQxIrhG5ZO2ZtVkuZ/3Z9TLv2Lx042PiY+SHbT9I6/mtZduFbanijCajjNszTmwG20iFKRWky7IuogwqzbkHRpNRAq8Fymz/2fLBqg/Ec5SnYEAKjC0gfx7902q9QbeCZMiOIVJ4fGEpOrFoln/jUXFRsiVoS4Zp+q/rLxiQH7f9aAkzmUxSYUoFqTejXqq068+ulyITiogyKPlh2w9W5Zh7dK5gQDae22gJ2xm8U6pMrSIYkPJTysuq06usymI0GWXaoWniNtJNXIa7yPaL29OVe+uFrYIBmXZoWobPlxG6IngB2Xtpr2BA9lzak+U8YVFhcvPezWyU6ulgMpkkJj5GRERu3rspl6Mvp4r/eP3HUmRCEXEe5iwYELeRbnLi6omcEPWJExkbKTWm1RAMyIB1A+Rewj0xmoyy/eJ2KTW5VJrGLD1MJpMUn1RcbAfbin+4f5q4BccXSOHxhQUDloNv3vr7Ldl2YZt8veVr8ZngIxiQDn93kOi4aImKi5KC4wpKrem1LL3gLUFbpNC4QpYevttIN+myrItsOLdBEo2Jmco4/dB0wUCa785kMonRZEyTvtPiToIBuRJ9xWp5K0+ttMjhPtLdclBNsquQmUdmpskTHRctPVb0EAxIl2VdUh2mYzKZpOrUquL7q28aJZHsXsT3V1+xHWwrZ2+cTRUfcSdCmsxpIhiQpnOayvmb5zN8FyaTSYb7DZegW0EZpssIXRG8gEzcN1EwkKYRTI9EY6IUHFdQco3IJd/+8+1zqxBu3L0h7Ra1k5bzWlptDEREFhxfIDWm1ZBP1n8iq0+vlqi4qKcsZfbRdVlXsR1sKytOrUgTN3bPWMGAnL5+OtNydofsFgyIzWCbVI230WSUbsu7CQak1vRasufSHrmXcE+G7RwmLsNdBANiO9hWXlvwmvwd8HeqBnDB8QUW30tj94wVm8E2UvHXijLryCwJvBaY7veVHmFRYYIBGbVrVKrwwTsGS+mfS0toVKglbO2ZtRaFY21UEBoVKl6jvaTGtBpyIOxAKp9R3Vd0l9wjcls6Fw9iMplkhN8IwYA0nt1YTl8/LZGxkbIlaItgQGYcnpHuM4TfCRfnYc7SdVnXVOFv/f2WOA1zkumHpj+1Ub2uCF5AOi3uJEUnFs3yP5dfsJ/ln9tmsI0cuXIkmyXMHlrOaykOQx1k8v7JOWoWuxh5UW7H3n7ofDfv3ZT3V74vdf+om+YYys3nN0uFKRXSPSJyccBiwYAYthusxl+JviK2g23l23++zVSOj9Z+JM7DnC297uT3+fH6jy11PPjbCo0KlYXHF8rVmKtWyzSZTNJ4dmOxGWwjGJCOizvKnfg7mcqSEdV+ryaNZjWy3MclxonXaC/BgFSdWlWi46LlTvwdKTqxqBSfVNzyLClJMibJy7NfFtfhrnLmxhkREXlj0RviMcpDQm6HiPMwZ/lwzYeZyrLw+EJxGOpgUTgYkHxj8mVqt/9y85eiDEpOXjspIiIbzm0QDMhwv+EP+zoeC10RvIAkD82rTK2SJi46LjpNr+jLzV+K/RB7iYqLkpDbIU9LzEfmz6N/yq6QXanCgm4FCQZkyI4hT0WGmPgYKTSukMw9OjdVuMlkEu/x3tJyXsuHKm9Z4DIpMLaA2A62FQzImN1jLHFJxiTx/dVXMCBrzqxJk/dK9BXxGu0ltafXzvCs4zYL2kjh8YUznKRMSEqQPKPzyLtL3xWTySQt57WUXCNyySfrPxEMyBebvnhkJRtwNUCKTSwmI3eNfCKKetDWQWIz2MYygk1Whl9u/lJsB9tKq/mt5NMNnwoGZFfILvEa7ZWmUV9zZk0a08+hy4cEA5Z3fujyoSzJE3gtUOb4z5EJeyfI91u/z3ROQkQ7ozrXiFzSaXEniUuMk9I/l5YyP5fJ8pnNTwpdETwuJpPIokUit7L3MO+sEhoVarF12g62TdUjSR6eY0Ai7kRYwo9cOSJ/HP7Dcr/05FKZdWTWU5U7q+wM3ml5hpT/LN/9853YDLZJZRLITpIbkDf/ejNV+Onrpy3y+QX7ZVpOkjFJ+q7uKxiQ6r9XF/9wf2k9v7V4jvK02Knn+M+xlJl8CH0yyQfVOw9zztTsk9xQbjq/Kd00yWaU1adXi4jIhVsXLPMpPVb0eKYWIOwL3ScYkIXHF4qINiIsMqGIJBmTLKOZlCuWGs5qKA1nNUxVxo/bfhSbwTZpVgolr1Cq/nv1bH+OQVsHCQak67KuaSaXnxYZKQJ9oXRWOH4cOneGGTNyWhIAouOjaVGqBf1q9cMoRk5dP2WJ8wvxs3xeErjE8rl6oer0rtHbcj//xHwmHcje3YqPQlxSHH3W9AHgq5e+IsmUZIk7ce0ErUu3xsfN56nIsvH8RgB2X9qt9ZrM7L60G4BcDrn4YfsPqeJiEmKIiouy3CcYE+iyvAvTj0zn6wZfc6D3AaoVrMaIZiOIjItk7J6xxCfF8+OOH6nlXYsqBarw75V/U8nhF+LHxvMbGdlsJOXylstQ5tfLvY6nkydzj81NN82CEwvwcvaiZemWAJTwLMGMdjP4tO6nzGg345k6AKa2d23yuuRl3bl1hEaFsjloMz2r9cTWxpY+NftgaGygcv7KjHp1FAC+eX05ee1kqu/kcPhhKuStkMZLbPLmw361+2X7c3xe/3PcHd1ZcGIBHSp0sLz7Z4b0NMSzeuXIiGDUKG3w9P77T7/uDAi8FigYSGW6qDW9lrwy9xWpMrWK1J9RX0REtl3YJpvPb07V0+u+orsUn1T8qcucGd/+8226E34iku6EXnZQanIpi0341PVTlvCeK3tKntF5ZPL+yYIB+SfoHxHRTFclJpUQ+yH20v6v9rLk5BJpPb+1YEDG7RmXpvzOSzuLy3AX+XrL15Zy+q7uKx6jPFLZ57/f+r3YDrbN8qR3v7X9xGmYk9U5jDvxd8RluIv8b83/HvZ15BjdV3QXr9Fe8tP2nwQDGa6cmbRvUprRcKFxheS95e9ZTR9yO+SpjYAm7J0g+cbkyzHTLDk1IlBKtVJKnVFKnVdKfWMlfqJS6qj5OquUup2d8jwymzZpf0+ffqLF/vrvr+wK2QXAx+s/Zrjf8FQ9mfSITYwFoLRXaRxsHSweM+OT4jkWcYyahWrSuVJn9oXt42LkRQbvHMz/bfm/VD29Tybuw7Aw/PEfYu9eqFULbj+Zr66oe1E+rvMxr5Z8FRFhWeAy1pxZY3lmVwfXJ1JPSowmI2HRYanCzt08R1BkEP+r+T8Ay/cE2oigYdGG9K3ZFx83H37Y/gMB1wJoOKshUfFR9K3Zl/1h++m0pBMbz29ketvpfPHSF2nqHdJ0CAnGBEbvGc2rJV+lWclm1Clch9txtzl/67wl3daLW6lduDZujm5Zep4e1XoQlxTH/9b9j5lHZuIX4seFyAvcjrvNytMruZd4jy6VuzzKq8oR2pRpw63YW4zZM4amxZtS0pZzoMIAACAASURBVLNkuml98/kCEHg9END8S4XHhFOzUE2r6Yu6F31qI6CB9Qdy+fPLFHUv+lTqeyjS0xCPewG2QBBQkvtHVfpmkP5jYFZm5T71EcGdOyL29iJKiXh4aPMFT4CouChxHOoon6z/REwmk2UnY7ItND1iE2PFaZiTjN0zVkS0HvSSk0tE5P4E2N8Bf8vFyIuWlQm2g23l+63fpyonxt1ZBMS0yWxLvnjx0R7kvfe00dLff2cp+b2Ee1Lt92pZmmQzmUxSY1oN8ZngI7lH5Ja/A7JWR1a4HXtbftj2gzSd01RyjcglGJD5x+Zb4n/e/7NgQM7fPC/5x+a39CjD74QLBizvf+rBqYIBcRnuIoXGFZKAqwEioi3X3XR+U4abhES03jsG5N+wf0VE5MTVE4IBy2arqLgoq99fRphMJnl7ydtiN8Qu1QqX5OthVps9C0TGRlqeJeV3ZI3kJadTDkwRkfvzIVmZy3nRIYdGBHWA8yJyQUQSgL+AtP5879MZ7bjK7CE0FEaNgiz0uFOxfTskJsLrr2u93mvXnog4q06vIt4YzzuV3kEpxep3V1PLuxafb/48lY35QY6EHyEuKY5SnqUAGNFsBB19OwJgFCPNSzantndtinsU59zH5yjmXgyjGGlXrt39Qu7exTVK62FL/37wxx9QujQEBKSpL0MSEmD1au3z5s1ZynLs6jGORhzlkw2fpAqPSYhh/vH5RMREWMKUUvzw8g+ERYdxJ+FOur26h+X63eu88ucrDN81nDsJd+hZtSe++Xz5ccePlgPnNwZtpIxXGUp5laJh0YbsuqSNCPZc2gNg8VfUq3ovSnuVpoBrAXb32k3F/BUBsLOxo0WpFpYDbdJjUqtJnOx3ktqFawNQIW8Fcjnk4t/L2jyBX4gfRjHySolXsvx8Sin+7vg3sd/HEvRJEBu7bmTOG3MY32I83zX8jlntZj1XfpQ8nDxoVLQR7o7uvFXhrQzTeuf2xs3RzTIiOBx+GIWiWsFqT0PU55f0NMTjXkBHYEaK+/eAKemkLQaEA7aZlfvII4Lhw7We608/PVy+fv1EXFxEVq/W8u/Y8Wj1P0CbBW2kyIQiqXpmhy4fEpvBNjJg3YB0843bM04wIOF3wkVE6/2FRYWluxTt7SVvS8FxBVP3AAMDRUDiunXWnumrr7TRTsuHWw4p69dr+QsUEClSJEujpZlHZgoG0uy0TN7nsO7sulThRpNRak6rKa3nt3442dIhLCpMyk8pL07DnFK5Y0heIfTH4T8kNjFWnIc5y8frPxaR+5v3QqNC5bMNn4nTMCeJT4q35L1179YT9fPTZE4TqfNHHRERS32P42PmReDsjbMZ+vxJSf0Z9aXJnCYiou0XKPdLuewU7bmBHBoRWDO8pdcdfxdYKiJGqwUp1VcpdUgpdej69euPJs2330KPHjB4MPz8c9bzbdoEr7wC1cw9iicwTxAZG8nmoM28U/GdVD2zmt416VerH3+f/DvdUcG+sH2U8ChBwVwFNfGCNuEz0Yd/L/+bxlOiiLDy9EqqFKiSugd4SfPs6PhhP3j3XRg3DgoV0p51w4asP8jSpZA7NwwapI24svBuAq4F4GznnMbOezTiKECanpuNssHvfT+Wv7M863KlQ0RMBA1nN+Ry9GU2dt1Im7JtLHFtyrShbuG6DPUbyj8X/iE2KZZWpVsB0KhoI0CbJ9gdupu6heviYOtgyevp7JnuucWPQh3vOhyNOEp8UjxbL26lQZEGONk5PbHyn0fK5ClDPZ96WUrrm09bOQTaiKCm95MZSb7IZKciCANSnuPnA1xJJ+27ZGAWEpHpIlJLRGrly5fv0aRRCiIjwcEBPv0UWrfWTBsZcf48BAVBy5ZQuDC4uj4RRXDm5hnyuOThnUrvpIkb3mw4gf0DcXdyTxMnIuwL20f9IvUtYRXzaaaIQ1cO4THagwn7JljilFLs6LGDBW8tSF1QSAgAI0MWEPr9ADCZIDwc8uWDgQMhKYlMSUyElSs1k1nbtlpYFsxDAdcCiEuKo8vyLlyOvmwJP3b1GHld8lIoV6E0eVzsXZ5IQ7gscBnBt4PZ0HWDxdV0MkophjQdwqWoS/Rb1w9HW0caF9PSVC1YlVwOudgYtBH/cP8M3Vg/Cer61CXBmMCWC1s4ce0EzUo0y9b6XjR88/ly/d51Aq8HEhYd9sRMii8y2akIDgJllFIllFIOaI396gcTKaXKAZ7AvmyUReOtt6BbN3B3h40boXfvjNMnrxZq1QpsbKBcOTh1KuM8WaCeTz3CBlr/gbo5upHXJS9Gk5HfDv5msVmDNgfwbcNv6VG1hyXMx80Hd0d3FgUsIi4pjmLuxVKVV79IffK65E1dSUgIYmvLD2d+J/iuWTe7ucH163DmDLRoAe3baw18796wb1/auZWdO+HWLejQAYoXh7Jls6QIVryzgvlvzWfxycUcCT9iCT8acZRqBatl6wqOoxFHyeOch5eKvGQ1vnnJ5jQs2pDQ6FBeLvayZYWSnY0dLxV5iUUnFmEUY7YrgjqF6wAware2Nr5ZSV0RPAzJK4fmH58PQI1CNXJSnOeCbFMEIpIEDAA2AaeAxSJyUik1RCmVYuaSzsBfZhtW9tKjB8ycqZkxPD1h3jxYuDD99Js2QcmSmEqVZMSuEdwomu+xRwQJxgRMYsLWxjbDRm/rxa30X9+frsu7kmRKYvvF7dgoGwbUGUCLUi0s6ZRSVMpfiYNXDgJkbRh86RKJhfJjtIX4q2ZFMGYM7NmjKcurVyE4GCIiYPFieOklqFED5szRRg8Ay5aBi4umJEEbNe3YAfEZH+Th6uDK62VfB7RRAECSKYmAawFUK5D5hN6hK4dYdGJRmiMB5f58U7r4R/hTvVD1dN+7UoqhTYcCmqkoJY2KNiLRlIhCUd+nvrXsTwwfNx+8c3uzJ3QP7o7uekP2kCQrggUntJFw9YLVc1Kc54JsXTogIutFpKyIlBKR4eawH0VkdYo0BhFJs8cgW8mdG7Zu1Xqx779v3S4eFwfbtkGrVpy8dpLvt33PUY9Yzaxy794jVz3zyEyKTizK1ZirGaZrUaoFE1pMYEngEir8WoFX/nyFRSesW88q568MgJezV5oRgVVCQjAV0XbnJl0zr9LJm1dr8Jctg5Mn4bXXoHx5OHsWfv9dUwDvvw+NGmmri5Yv19K4mG3jLVpo72XPnnSrPX71OF9t+YqYhBhKepa0KAJbZcuFTy8wsP5AS9qYhBirp0P1XNmTLsu7UGxSMX7c/iPLTy3nwzUfUnRSUSpNrZSuMkg0JmZJ2TQp3oT9H+zno9ofpQpPnieoUqCKVbPdkyZ5VNC4eOOHPjrxv04RtyLkcsjFpahLlPYq/VS+r+ed52cN2ZOmenXYvx8qVoQ334T58+/H3bunhd29Cx07WiYyyzVqD8ChHQtTuT4ANLt6hw4wKX23DbGJsYzfNx4vZy/yu+bPVMSB9Qcy5tUxXI6+zJAmQ3i74ttW0/WoppmKqhdMv7ebZEqi1fxWTD04FUJCUMWKA3A3IUZTiAUKpM6QK5c2WmrbFrp3h6NHtRHB6dNQtaq2jLZjx/vpmzQBe3uLeUhE8A/3t7w70JZCjt07FkGoVrAa/uH+1J9ZnyWBS/DO7Y13bm9L2gazGtB7TWrT3anrpzh5/SR9a/SlTuE6DPUbSofFHVgUsMiyZDA8xvomudM3ThNvjKd6ocx7h3V9Uk8Gg9Ywu9i7ZLoc9ElRt3BdAH1+4BFQSllGBfr8QBZJbznRs3o98Q1lBoOIu7u2DHLAAJEbN0QaNdI2kM3Q/IwP3DhQnIc5S9KxoyIgnTsqGeE3InU5P/6olVG0aLrLKL/75zvBgOW0p0RjYpaOUczKcXdTDkyRxQGL043/7d/ftMM1/n5bxNZWkr79RjAgw3YOS7/QFSu0Z/owhTfHq1dFunQRKV1aJPoB2Rs3FqlWTUTue+jsuLijJbrv6r7iNdpLTCaTjNo1SopNLCYYkBrTasikfZMs6S7dviQYEPsh9nL97nVL+JAdQ0QZlOUMhvM3z4tfsJ/EJ8XLjos7BAOy4dwGq4/y59E/BQMWV8CPwomrJx7J9fSjcCzimBQeX1gu3LrwVOp70ei5sqdgQEbvHp3TojwzoDudy4BixSAqSjNzTJkCRYpok6OLFsEHHwCabblKgSrYli0HNjY0T/Dh75N/3y9j1y4YNkwr69Il8PdPU83JaycZs3cMPar2oGmJpgB8t/U7Kk+tnKlt29bGNk1YdHw0H6z6wOIaoX+d/nSq2Mlq/sjYSH7Y/gMANlciwGjEtngJ4r6P4/uXv0+/4vbt4euvYdo0ba4AIH9+WLAAzp3TTGwpadlSGzmEh9NtRTeu3LnCpvObSDBqq7MCrgdQKX8llFJ81eAry2Ts8avHUzlJ2xG8A4BEUyILjt9f8bT01FIaFG1gGTmU8ipFo2KNcLB1oHIBzTx24uoJq4/iH+GPs50z5fJk7LQtIyrlr/TUzAxVClQh7PMwSniWeCr1vWj45tVHBA+Drgi6d9f2CJw8CX/+qa2AWb4c3rm/tNPR1lGbIHRyghIleOluHo5dPUbQrSBtSWrXrlCihDZZamMDK1akqWbO0Tm4O7ozrsU4AExiYt7xeYREhXD1bsbzBdZY+89vfPTRLOYvyHx6ZfDOwUTGRVLCowQuV8z7MIoVw9HOESZM0JRgegwdqk0I26WwU9+9qynPB3nrLe35J0xgV8gu3BzduJNwB7+z/yATJ3I5+IRlPmNz0GYCrwfSpkwbkkxJFM5d2FLMjuAdeDp5UrNQTWb6z0REOHvzLMevHqdjhY5p60WbH/Fx8+H4teNW449GHKVygcpWlarOi8ebFd7k3UrvplpqrZM+uiKwsdE2VIWEaKtlAgO1tfEp2NhtIxNbTdRuypenRIS2YmXFqeXQt6+2Bn/RIk2JvPwyrFjBnfg7+IX4seD4Au4m3GVM8zEc7HPQspRzz6U9FncKydvhHwbnn3+jVji4LFpCTEJMuulOXT/FlH+n0KdGH+r51CNXxC0tolgxhvkN4/T2JRm7lrC3h/XrtUb+4kXw8NDmD/Lmhf79tZVFyRw7BuXKIVN+ISkslC9f+hJnO2fiRwxBff45A/YkUil/JQAm7J+Ao62j1f0BO0J20Lh4Y3rX6M2Jayc4HH6YZYHLADJ0MVClQBWOX02rCEREWzGkrx75z1DaqzSLOix6ohv9XmR0RQDQrJm2wWzUKIiNzThthQo4BAVTu0AN3CdN1XbXDh8OtTVfMbz5Jpw8SVtDWRrPaUy3Fd24EHkBpVSqYf7SwKXYKq13+rCK4EboWZrvCAWg9ckE5h+bl27azzd/Ti6HXAxtOhRPJ088r0VrEUWKsPL0SqIuB0GePBlXmDwBnT+/tg9jxAjo1QumT4dSpe6bwn7/XdtnkZDI97s0fzzvO9ShxaIDoBRfnM1L76rvE3AtgM1Bm/HO7c2yU1oDnzwquhR1iQuRF2hSrAnvVnoXJzsnZh6ZyZLAJdTzqUcR9yLWJASgSv4qnLp+ioSk+FTf46WoS9yOu637m9HRSQddESQzebK2ScrZOVXw6N2jaTirIUaT2ftF+fIQF8e6m63os/SitnLm888t6a++qg1Fmx69zfK3l3Oq/6k0h4mYxMSyU8toW7YtHk4elu3wWSVo1JfkSoCb3TtR5hZsWDPh/jzDvXvQtCls2sTe0L1sPL+RQS8PIp9rPjydPcl/IxbJmxdcXXF3csclOi5zRZCMq6s2j/Ltt9q8walT8L//QZUqWvyiRVCuHDGervQ9DDXvujNhcTR2nnlhyhRUWBh223Ywcd9EXOxdaF6yOS8Hg3scnLx+kgRjgmV+oEnxJng4edDRtyNzj83FP8KfTr7W50CSqVKgCommRK6PHaLtBL+qKRf/CE1R6SMCHR3r6IogmTJltKWkQUFQs6bWuFWtSviuDdyKvXXftly+PAD5Bo3Q/PDv3auZh8x8eeYXjnjb8MXVUrxZ4U3K5y2fZinigbADXL5zmU6+nfDN50vgjfRHBDfv3WT07tHcSzTvXYiPp8z8jewu74rXcM2dhO/e8xbvmMybp81VzJjByN0jyeOch49qaWvivZy9KBIFxqLaHgI3Rzdy34nXzDyPQunSMH482JrfTYEC0K0buW/eAVtbjE0b43jIn+DBA/muWBDRuezZPbg3c4/NpWfVnrwR7s7OOfDvwercS7zHv5f/ZUfwDrycvSyTv72q9SI2Sevdd6jQIUNxqhTQFJJpyyZt7mbkSECbH7BRNpYydXR0UqMrggdxcABvb62Ru3iRVxb/m9qkYFYEeHuzsWs9uHIFZs9GdmkN8ZTXppCvax9y+5+Ey5etVKAdIelg60Dbsm3xzeuboWlozJ4xfLP1Gz5a9xEiQtTM3/CKTiDogzdRPj4Ya9agw1lbpvw7Rdv0NVGbyzBu3siGU2v5rN5nltU5nk6eFLsN8d7angE3RzdOF7DV9gU8Kbp2BeBO1Qp4XL/DmrJQ8tr3jDw0gTm+idT+N4yvKvRmeMMfeWW0tvKq9PajlL4J2y5uY0fwDhoXa2xxkte4eGNKeZaibuG6FPPIeLNc2TxlcbB1wO34Gc2cNXUqhIbiH+FPuTzldHuxjk466IrgQYoUgTVrYPlyYnt2o/XxWBrapfCUmSeP5r100yZsT2hLFY9++AbtzwwmPikeN0c3ivQ0+9pfntZjpoiwNHApLUq1wN3JnYr5K3Lj3g2u303rVTXRmMjcY3Nxd3Tnz2N/8sehaRjHjuFoAaj13tcA2LZ/k1qhRvYdXM6N5fM1X0EdOmAbHUOzCGf61+5vKc/TyYNiURBTKI/53pP/9cqvmXqeEHFFChG9ZS1rh3Vn7EtQYP5K3qzwJgCBr9fD0QgjrlbCY/I0nC5coucbYLKzZeTRPMw/Pp+Lty+m2rRlo2zY8t4WlnRakk6N97G3taexXWncb8bcN9cNHWrxY6Sjo2MdXRGkYF/oPvaH7bfY24+8VZ8pdaDigyaFjz+GSpWoFWGLf0GoXmgVYaZIImPNK3IqVABfX/jkE81R3QcfWNxYHLxykNDoUIu9+8Gj9VKy4fwGrt69yuw3ZtOiVAvO/TQAr+AI/mpVmIoFtNU3vKGd9fP6GTj93YfEF8xH0MivSLSBb+5UxdPZ01Je/ng7XBMhqoC2Fn5Sq0kEfxb8WO9sV8guXpn7ClfuaD6LdgbvxH1PWzbfPcb3reypWbMtX9TXjmn0bd5Z29E9aZI24fzuu4yYdxnVsyft99/mTsg5AJoWb5qqjhKeJTKcJE5J29vmHdsdO8KHHyKzZmF/4ZI+P6CjkwG6IkjBD9t/oPuK7qw+s5pjEcdQpUrh91l7fCtbPx3KY80Wxhqa0792f/ZWmkTBZm9oyymV0rybjhmjKYIVK7S1+h07snbnH9jb2Fscr1lVBCIQE8Ms/1kUcC1A27JtWZrwJmM3GFleHly7p3C9UKkSlCjB+JOFaXguDkOlG7T/5wP2FVU0DEy9rDTfdW2e4UZe87m/589r7iWSvaw+JEmmJP637n9sD97Opxs/BTT/7wBt/jrC0EO5sbWxpUHRBhzvfYQBdQZoq42CgrQ9GRMn4p3bG9uvvsHWKHy2H/I457Gc8vUo1L1sQ6IN3CjrA999h8nejp92pj3nQEdHJwXpbTlOvoBSgKP5cxPgE8Ajs3zZdWXXmcXhd8LFZrCNDNw4UPKPzS/1Z9TXTvUymUS2bBHZvj3jAs6c0c42fu01kYSE1HEJCSIjRojR0UEinZA/+tWzuKEwmUySe0Tu+6eSnTmjuWoAWVMWmTruXZENG0Ts7CSqXnVpPr2RhNwOSV3+wIEiICYXF+k9p4NgQFb0rK+5h7h82ZLs+vzpIiDL5n0nIiJHlvwiAnJ7ZeZnAU87NE02n9+cKiw2MVZ+2v6TtJrfSjAgq0+vlrf+fktKTS4l26t7yk13B5EhQ0SqVBH5TqtTbt4U8fERmT1bu9+8WeTaNUno1EGiHJHuM9qIXLigvfPg4EzlepAb9avJwUKaGw+TySSrXi8rRoXcvHjqocvS0XmRIAMXE1lRBEcBO6A02mH0E4H1meXLriu7FMEvB36x+KKZ7T/7/gHYRqNIuXIideqkzjBihHaMZUqmTtVeabduWr4UmEwm6T32ZfErYaul6dhR5NYtERGp+0ddaflHY61MR0cRDw850r6+XHNBS6uUSNWqIrfT8XOzY4eWrn9/MZlMsvfSXok9dEALmznTkix+/BgRkMlrfhARkZ2TvxABubhlSYbvJsmYJB6jPOT1ha9Li3kt0hwEHp8ULxV/rShN5zSVYhOLyTtL3pGe3d3vy96woch8K4eO372rpWnQQMTfX1NmSmlhIGJnJ9K3r0hISNq81jAaxeiWW36thUzaN0nmH5svTbuby9q8OfP8GWEyab+B6dMfrxwdnRzicRXBEfPfL4GPzZ/9M8uXXVd2KYIGMxtI5d8qi4h2Tm6136sJBmTQ1kEiv2g9Z9m7936GSpVEXn01bUHJZyP375/K+dyKUysEAzJxz3iR0aO1Rq5oUZFx4+RInSJy197cYHXqJKYrV6T8lPLSZGpdkWnTRDp3FgkPT194o1FkyhSR6/cdtInJJOLtrSmc5KDPPpO79sjXm78SEZGjIz4VAfHftyLDd3Pw8kHBgEw7NE1KTi4pxSYWkyE7hsiq06ssaYJuBUloVKhgQIbvHC78hCwa/d79EYnJJHLokMidO6kL79pVOxM6Lk5k0iSRr7/WnP1t2aIpWnt7EQcHbUSRmJihnHLqlAjIx2/nlpbzWor7SHd5bVJt7b2OH59x3sy4eVMrp0uXxytHRyeHeFxFcADt8JgAoIQ5LCCzfNl1ZYciuB17WzxGechwv+GWsH2h+wQDsvXCVs3LZsGCImXKiERGily5or260VY8G5pMIv/3fyKtWmmNm4jcTbgrRScWlcq/VZZEo7kxO3BApFQpEZDbBT3ll9rI7fVag7zn0h7BgMw4POPxHqxXL82zanID+tZbcia/rfRZ3UdERC58+5EIyLZjqzIoRGT07tGCAQm/Ey57L+0V28G2ggHpsaJHqnRRcVEy7dA0WXtmrWBA5h9LMQrYu1d7ZwsX3n9P8fEi69Zp4avSkSEkRKR7dy3Nq69q3mHTY+5cEZBeI+sJBsR1uKucv3lepEABkZ49M3zGTDl4UJOhXr3HKycn+OMPkfXrc1oKnRzmcRWBL/Az0Nl8XwL4JrN85rStgDPA+fTyAG8DgcBJYGFmZWbXiCA2MTaNS2hLoy0ismuX1otv29bS4MiRI9YLM5nuzxMEBMiqj1uI7Q+kManIvXsi58/L+jPrBAOyK2SXmEwmeXn2y+I12itLLqozZMkSTc7580V27hQpX178KrhaXEMH/z5K1pZBlp1cmmExLea1EN9ffS33o3aNkkLjCklYVJjV9BvObRAMyO6Q3fcDjUaRQoVE3nxTuz9yRMTLS2TrVpE8eUTefTfjZ5kxQxsZFC8usnixyMaNWt4rV+6n6d9fJHdu+WLdZ5YRjIhoCuRxfzd//aW9y/z5H6+cp43RKJI7t+Y2PB336Dr/DR5LEUjqRtsTqJLFtLbmOYWSgANwDPB9IE0ZwB/wNN/nz6zc7FIEWeL330XmzBF57z2RfPnSzANYI+LjXiIgJysVFEmyfq5AcGSwpeGaf2y+YECmH3oCtujbtzXTSrLNHWRR80LSbG4zEREJuR0i3uO9ZVngsnSLMJlMUubnMvLx+o9ThWd0RkLy2QdpFMWAASJOTpp56PPPNdlu3tTOOyhcOHPTz4EDWroUzyPu7ppJSESkdm2RJk0k6FaQTDkwRUzJDd/AgVq96bz/LDFixP06HzRvPcucOXNf7n37clqap0dAgEiLFmnPzPgP87gjgh2AG+AFXAIOAxOykK8+sCnF/bfAtw+kGQP0zqyslNeTVgTnbp6T8lPKy95LezNPnMz334t88UWmyRKSEqTa1Kry1Vtu2qsePtxqOqPJKK7DXaXnyp5ScFxBqfNHHW3FUlZJSBD55BNttc2D+PlpPeh//hHx95d2c1tL9d+rZ71s0ZTBvYR7WU7/1eavxGGoQ9pn2Lnz/gilYEGR9u218Bs3LGa0TImO1hTCnj3aaqp8+TSTXXi4pli++iptnlmztHrPnMlaHX5+IrGxqcN6977foB47lrVyngUWLrwv90cf5bQ0T4/Ro7Vn3rgxpyV5ZshIEWRlH4G7iEQDbwGzRaQm8GoW8hUGQlPch5nDUlIWKKuU2qOU2q+UamWtIKVUX6XUIaXUoevX0+7AfRy2XdzG6RunyeeaL+uZhg3TXFdnwug9ozl69Rj1h83Rzjf46Sc4dChNOhtlQ4V8FZhzdA5XY67y62u/WlwsZImjR7Xdzv36pY1r1Ag6ddI8rFarhnvuvETGRWpxrVunOnchPZRSONs7Z5oumeCoYIq5F0v7DA0aQMGC2oa8iAjNkylou7UdHbNWeO7cUKeOdr5yq1ba7u3gYGjcGBITtbgHqWzeEHjC+qE1qQgI0FyJz56dOvzCBc3pXvLn7CYpSXP/LRkfWpQphw9r77ZjR/j7b4hPew70C8np09rfI0dyVo6U3Lxp/Xz0Z4CstDZ2SqlCaLb8tQ9RtrXDcx/8VduhmYeaoE1Iz1BKeaTJJDJdRGqJSK18+R6iwc4CRyOO4u7oTinPUo9dVpdlXag7oy6DdwxmWeAyhuwcwjsV36F9hTc1vzcFCmgNthWSN5b1rdmXWt61Ukdm1hjUrg0lS6Y+PCYdPJ08uZW8A/rSJfaH7NHOMU6Hzss6M3Tn0EzLTUnw7WCKexRPG2Frq220a9YM3N2hTZv7cZs3a5vjbt16qLpo2FBzf332rHZvTRH4+mqb/DI6dyGZNWu0vw+eMhcUpCkbeDqKYPp07f3888/jlXP4sOZAsVcv7d2uX/9kpuWKUgAAIABJREFU5HvWOXNG+/ssKYLx47WNpcetH56Uk2RFEQwBNgFBInJQKVUSOJeFfGFASr8APsAVK2lWiUiiiFxEm1guk4Wynxj+Ef5UK1gt3UPfs8qNezdYFLCI0KhQBu8cTMclHXF3cueX1r9oCTw9Na+gs2ZZzf9qiVcp7VWa4a8MTx0xZox2GMzkyZpTufSoUydLPzBPZ0+i46M1t9o3bnBO3WLygcn0WtUrTdq7CXdZFrgsw4NvrJGuIgDNwd0XX2jurJ1SHEqTN692StzSpQ9VF6A1coMGaSMOH5+08S4umhPBrIwI1q3T/qZMm5AAoaGaV1oPj+xXBCKacgPYsuXRyzGZtIawZk1o3lzriMxL/+yKF4pkRXD4cM7KkZI9e7S/06blrBzWSM9m9LgXWm//Atoqo+TJ4ooPpGkFzDV/zotmSsqTUblPco4gyZgkLsNd5NMNnz52WX8H/C0YkH2h++T63euy8PhCOXT5kPXEkZEiERGZF7p6tWbnLFZM+9uiRepVMsl06yZSooSWJuVeAitM3j9ZMCA3Yq6L2NrKby28JNeIXFJ0YtE0aTed3yQYkI3nsm5njYmP0fYS+FmfD0kXk0mkYkVtF3J2rG556y2RsmUzTnPjhoiNjTbXkCvX/cUA585p73b2bJEaNURat37y8qUkeamtg4NItWqPXs7Zs1o5f/yh3Q8cqD1bRktwXwSuX9eeu1Ah7a954+YT4dgxkaCgh8+XkCDi7Kz9vnLnzpEFBzzOHIFSykcptUIpdU0pdVUptUwpZaXblUbBJAED0EYTp4DFInJSKTVEKdXOnGwTcFMpFQhsB74UkZsPoccei5iEGNqXb88rJaz7EkrJ5ejLLDyxkE82fELjOY3ZHLQ5Vfw/F/7BzdGNWt61yOuSl86VO1PT28rB2fHxmivrjDx+xsVpvbk2/9/eeYdHVXR//DOE3qtIE1BBWiCGKiJFuiiIItWCiAgKgv5EsUNsvKCivGJBEZRXKYIgShGlCIggLVTpNdJbKKGE5Pz+OLubTbK72ZQlbT7Ps8/uvXfu3Jm9u/fMnJn5ng4aR3nPHnUtLV8OX34ZP+2lS66AMAwYoK1XHxTLqyJ0Z48dgJgYLhfOz8WrFzkYeTAu5oGDmdtmkjNHTprc1MRnnu4ciDwA4L1H4A1j4IUXtFezcGHS6ZNLcLBqK/mKQPfrr/q9P/YYXLig4Ushrgdwyy3qggt0j+Dzz3Us5PnndfwnpeNiztZwPYer8ZFHdBxl+vS0KWdGxdkb6N5d39PKPRQbC23aqCv2n3+Sd+7Gjfrbe/ZZOH8evvsubcqUVnizEBLXav8NeBxt4ecEegO/JXVeoF7pMX1016ldkv+d/MJwJP87+aXAOwWk+aTm8dJU/qiydJrSyb8MBw3SNQn79sXtW79e59JXry4SFBQ3JdKdnTsT6xitWqWtnh9/9OvSP+/4WRiOrN28UKRPHxn0Qk1hOMJwZOPRuNkwP23/SRiOPP3L0z5yS8zcnbomIlmzsJxcuaKroe++O25fWvUOZszQ72mtl16aiK4aLlUqrkU+e7bu//RT3Y6I0FlJuXP7NXVYpk9P/ormU6dUZmTAAJ0dBbqGISW88ILm5fzNxMaK1KgR//vNinz1lX5vf/+t76NGpU2+zvyCglQv6+BB/8/9+GM999Ah7fXWqXPd13WQyllDpURkoohcc7wmAWk7YptOXLx60WnsfPLu8neJlVhWPbGKyGGRvHjniyzdv5SDkQcB2HtmL/vO7qP1za39u/CLL0KOHBojGdQf3bKl+oOrVtXegnOGijtVqmgw+VNunSZnqy80FGJivAbDcVI8X3EATuaNhQkTiKhb1XVsx8kdrs/NKzXntbteY0y7Mf7VycH+s/uBFPQIQIMCffQRPPecbotAx44wYgQMHqwDzSmllkO229s4QUyM5t++feJZRnv36sybMmW0R3D1qgYkSoq33oJXX9Uenr98+632Gp96Sn37RYqkfJzAOVCcK5duG6MD3mvW+B5vyuzs2KH3KzQUKlZMu3GC+fPjlIXPnYO2beP/F33x118a66R8ee25b9wIq1enTbnSAH8MwUljzMPGmCDH62HgurlvAskD0x+g1WTfM2H3ndnHtxu/5am6T9GwfENy5sjJw7V12uN3m7R79/tendnR6mZ/ZtWiP4Y+fXSK4qFD8OijOqC5Zg3Mnq0PkApe9Pf/8x+Nx3vxom6vX6/TL2+6SfO56y6fl3a6hs5cOAkiPFgjLvzjzlM7ORV1iqjoKArnKcxbd7+VKMxmUuw/u588QXkoXbB0ss5z8dBDcO+9+vnTT+GXX6B4cZXydhrOlHDrrTo47c0QrFqls2o6dICCBfWB724IKldW432LY3bZnj2+r3f8uJ5/+bKGM/UH5yDxHXfooHpQENx9txqC5E4jjY3VB2DdBO7J+vXVNbFjR/z9O3fCkiXJu0ZGZft2bTQFBakxSCvX0Pz5+v21agVz5ujvYsAA/85duVLvK2gUv4IF1d2bQfDHEPRBp44eBY4AXVBXUaZGRNhwZAMVi1T0me7d5e+SM0dOXrzzRde+m4vdzJ0V7mTypsmICL/v/Z3yhctTtURVHzklYNgw/XP/8ov6bH/7TR82SVG3rrYYnX/a4sX1wWmMtv727fM5BdPZIyj500LIk4ez2/RPUjJ/SXac2sH/Lfw/2kxug4iw7vA6Jm9M3iyT/Wf3U7GohzUEyeHUKXjwQRg4UFvoAwdqj+CPP1LeugsK0mmk3gzB3Lmapk0b3Q4OjpuFtXevGgaIe09qnMD9oeprCujRo7q+5PHH9aG/Ywf07x93vHVrOHgwacOTkD17tNXqyRCANjrcef557X1FRyfvOhmRHTt0zAy0/rt26XeRGk6e1BZ8+/a63ayZNrwWLky6d/Xvv3oPGzfW7UKFdA3NtGkQGZm6cqURSf5bReSgiHQUkVIicoOI3I8uLsvUHLlwhBNRJ3xGrjpw9gCTNk7iydAnKVuobLxjj9R+hH9O/sPaw2tZtG8RrW5ulbwpqBUr6uDlgAHaeqle3b/z7rpL3UbO+eCjRsGkSfo5NFTfE86Bd8MZsUxOnIDoaOad/IucOXJS+4ba7Dy1k6X7l1K2UFmMMYQtC6PPnD6cuXTG72rtO7svZW4hd65ciQvzOWGCGrm+ffUP9MEHKc83ONi3IWjSRKeHghrVXbu0Rb9nT1xPoEIFNRhJGYJFi6BwYWjUyLdr5733ICxM01y6pL+Hrl3jjrdy9DKTu57AaTATGoLq1fX3424Irl2DZct0gHzVquRd53qxahXMnJl0uuhovTdOQ+DHf8IvFi7Uhts998Tta9JEH+Rbt/o+96+/9N3ZIwBt6Fy5kmHcQylttj2fpqVIBzYc0R+Gr8hVI1eMJIfJwUtNXkp0rGvNruQOys0Lv73A6UunaVXZT7eQOzfdlPxz8uTR8YT58xO7C5w/eh+t5txBuTWI+6nTkCMHu2KOcy32GjcXu5ntJ7dzIPIADcs1RERYFbGKa7HXmLtrrt/F2392P5WKVEp+vdwpW1a73mvWqF8e1Ff+5JPaezp4MGX5BgdrC9y56tTJggXa+ndf4BYcrOMGy5erK8XZE8iVS+9bUoZg8WJo3lxXP69b57mXFhsLP/wA998PERH6sPv00/jrK269Va+X3HGCdet0zKVmgmhvTneJ+wr3deu0jpC6dQuBZMgQnQW0e7fvdHv2qGGrVk23nf+J1LqH5s3TtS713BZ73nmnvjvXB3jjr7/0noa4PWvq19cGTgYxvCk1BKlbfZUBCD8aDkCdG+vE238q6hTfb/6eh398mAkbJvDE7U9QvnDi2bLF8hXjvqr3sezAMgBa3twy8IV2cs89Kqvw/PPas3BOLyxRIv7g2Icf6mraP/6IZzSK5ytOzrNnoXhxzl7VLnP5wuWJvKLd1AblGrD/7H6OXzwOwOzts/0q1oWrFzgZdZLKxfxwcSXFfffF/9OBuoecshQpoUMHbaXXr689jZgYHYi+5x4dTH7ssbi0zgHjn37Sd6chcH52NwQbNsR/QB04oA+kli21RS/i2f++YgUcOeJb5sMYdQ8tXqzl9RfnQHFuD2M89evrtFSnG8hZtqpVAzN1N7UcPw5//60P+Fdf9Z3WOfbh7BGULq1jat4aR/6sZI+N1anF7drpOJGTm29WyZQVK3yfv3Kl/pbd70WRIuqqzOQ9glQKoKQ/d1W8i7davEXhPIVd+5YdWEa5D8vR68de/LrnV3oG9ySsRZjXPB6p/QgAtW6oxY0Fbwx4mV107KgPssOHtSVXsmTcsVGjVHPojz90BW+zZtoydeu+FstbjDxnziMlS7rkJpzlz0EOQsuE8leEdmcbV2jM/N3zuRTtY/69gwNnU7iGwF9uukndYCnpSYG2Ejdv1j9l3746JjN8uM6vX70abrghLq1zcDkpQ3D8uH7HrVrFzQ5atEjfW7bUFd+FCnl27UyfDvnyxQ2Oe6NVKzh71qNOlUeOHdP6eJLbADUEly/HSW4sWaI9h+7dtRd2xn9X4HXB2fvt2FG/s4TjG+44e3tOQwDqHnPvEVy9ClOn6n0rUQJGj/Z9/bVrdYzAOT7gxBh1D/nqEVy+rEbIOT7gTqNG2iNIrZ5UGuDVEBhjzhtjznl4nQfKejsvs9C0YlNea/qaa/vi1Ys8/tPjlCtcjtV9V3PshWNMun8SJfOX9JpH+yrtqVC4Avffdv/1KHIcZcrEBYEPDdUfpJOuXfUH1r+/PujWrNHjbtINxfIVY1nNgkT2epAY0Vams54P1niQArkLsCpiFQVyFeC1u14jKjrKNTMK4PjF40zfOj3R1FtnmrTQbfLJ/v06sJrcAVRQI7JokY41xMbqAr1Jk3TWljs5c2qLLSJCtxMaguPH1af+5ps6g+vAAZUBAc2/dGk9P1cuNcQJXS4xMXpPnLOUfNG6tbZE5/rpohsxQv3Pgwd7Pu4+YHz1qrZoW7TQgfLYWO19ZCR++UXdhZMnQ6lSOv3a+dv77TcYNy5ue8cObaUXKRJ3fmioGojhw9WYlC0LPXrovb3jDnjlFd8t83nz9D/Utm3iY3feqb9Hb9O216/Xnpf7+ICThg21R5KUu+t64G2BQUZ9pcWCsqirUbL237VyOTpO+njIfA1msmTfkmTn5UubP2AcOiQepYUvXRJp3lyPOaNSNW2q8g0O7p96vwR/GizL9i8ThiM1Pqkhf0f8LbnCcslLv70kIiL1x9eX5pOay5VrV6TIe0Xk8dmPi4hKZjef1FwYjoQtDXPlufnYZsn7dl5pO7lt8iS0U8Lu3SLFimm40EAu1X/sMf0eb7wx/v5p03T/lCkqGTBokAYsKlxY5NgxTd+jR1x652Iid5nwxYt13/Tp/pXlrrt0EVJSbN+uC54SxtN2JzZWgwL17SuyYoWWY+ZMXXhWqJDGh8goXLmi3+uTGlXPFTb2gw804JBTYvu77/R448b6+3fnt980jTG6YPPRR1WeOiZG5V4qVtSARwljgsfG6vdTtar3yHTOyHXTpnk+PkrjhHuUlNm0SY99+63fX0dqIK0C02SEV1oYgj/2/yEMR+bunCsiIisOrBAz3CR7FW264tQh+vTT+PuPH9f9NeIiirn+PNu2iYjImNdaSaNXSsu34d8Kw5EdJ3fIyYsnpdJHlaTjlI4SdTVKcobllGG/DRMRkZ4ze0qJ/5SQ6Jho+WzNZ8JwJPjTYFc4yqirUVJzXE0pPbq0HD3vh4ZSWrBwoT6Eu3YN3DU++EC/t8aN4+9fu1b3Fy+ugXFOnNCV4EFBcQ+nr9zCjG7dKvE0f0RE+vfXWM0XLvhXltGjNY8DB3ynu/9+1Uk6dsx3ujZt1LCEhekD0qk/1KmT6lY5iY0ViYz0r4wiGlfijTf0d5gWLFok8VZ5X7niCvEqJUqIjBmjD+nixfVhW7x4YkMWGyuyebP3IDUrV+q969ZNtbzmz9fv5bbb9DoFCmi0P09cvar3cdCgxMdiY3UV8e1e4n9cu6b36pln/PsuZs7UqIYpxBoCN3ae3ClVxlaR3G/llp93/Cxzts+R2/57m1QcUzH1oSGvNytWeJY6WLEifkv533/1Vo8cKXLihEQVyCM/VwuSsKVhwnDkUvQlV2S0Wz6+RVYcWCEMR2b/o3++H7b+IAxHvg3/Vgq+W1BafdtKLkdflhaTWkjut3JL28ltheHIr7t/vU4Vd+CMGuYt3nFqWbhQ83/44fj7T5+Oa4m+/37c/meeidvvLh8SG6vSGV266OfoaJWy6NbN/7I4I4198on3NMuXa5q33ko6v1df1Ydfo0bxexqffKJ57N6tD3WnYXGvjy+cgXBee82/9Enx3HMqk+H+e161Sn/Lzhb81q0q++E0wmPGJP867hHonK/GjUUmTEi613n33SpGmBBnb2u8j2iDLVr4F0bVKXviKU66n1hD4GDR3kVSbGQxKfhuQZe+DsORnGE5ZeHuhSnON1Owdq22QJ55RmKCckj1p5FeM3vJje/fKDXG1ZD87+SXXGG5JPdbueMFqxcROX/lvOR5K4/kDMspBd4pIPvO7BMRkdNRp6XaJ9WE4cjQhUOvf52uXlX3UJs2gcn/6FH9iwwfnvhY0aIiN98cP7La8ePqxnBvUTtxRji76SZ1IzndMcnhtts81/XQIY2jXbOmGpyLF5POa/bsuAfekCFx+52KpR9+KHLPPeJSQX3oIf/KeN99cfX0R48pKapWFWnXLul07g9yp0s0OcTE6EP24481kt6ZM/6f+/rr2jtN2OPo2VN/D756fS+/rLpjvlr6sbFqlG68MVWuUGsIROTi1Yty4/s3Ss1xNaXJhCZS+aPKsmDXAlnz75rr585Ib7ZuFQkKkk1dmurYwLga0uirRi6DWPW/VYXhSOgXoVLpo0rxTr33+3uF4ci4v8fF23/g7AF5/8/35cq1K9ezJnHs2+d/mMuUMHOmZzfLjz+KrFuXeP9vv+krIefPqxuvc2d1J91wQ/K7+UOHqoy001Vz/Li2Jp0PwBIl4lwoSeHsJSbsUcXGqs88KEiPffGFyIgR+vmPP+LSbd8uMmxY/IfcqVNavqpVNf2iRf6VZf16jaecMDyo0yj9979J53H1qrbKIWUy0anh11/1uu73/dgx/S6efdb3uT/9pOeuWOE9jVMw0d21mAJSZAiA88A5by9v5wX6lZoewbrD62TP6T0SNCJIXvn9lRTnkymJiXH9uWf88bkwHMkxIof0mNFDcoXlEoYj3X/o7jIKPWb0iHf6yoMrZdhvwwI/EJxSzp1To/D77yK9e4u0bZt48C+jEB3tX6s9IU7Xz/Tpej/btlW3yejRIuHhyW+Bly2rLdmErd8BA3Tc4OuvdfviRZEKFdTXfe2aDpCWLKllefvtuPPGj497qBUpIvLII76vv369xnZwGqScOTX+Qp8+Ih99JDJwoCQaZPfFrl0i77133VU9JTJSv8c334zb9957WnZPKsLuOHud7i5Gd5xjIrVq6XefClLVI0AjlD0NFEKD2A8AXkzqvEC9UjtG8PkafQiGHwlPVT6ZkqFDRb7/XhbsWuB64L/y+yuuz1+t+8r1+eNVH6d3af0nNlb93Dly6E+6cGF9LV6c3iVLW65d01b/ww/HuUI+/zzl+T32mGdJ6nPnRLZsib/P6fsfMEDHDCpVEmnWTB/4zsAvLVqIVKmi9+Opp3QQ1dMA7eXLOnMHdPbXyJHaw3rlFZHWrXX8xGkcatVKef2uJyEh6rpbvVrvU8WK+n34Q6VKOn7k5MSJOOnwjz7S72H+/FQXMbWGYLU/+7yc2w4NP7kbGObheG/gBBDuePVNKs/UGoK7v7lbbvvvbRJ7vVsNGYjVEatdD/zxa8dLxTEVpfTo0nLy4kkpOrKoMBxZHbE6vYuZPGbP1hlE06apyyU5Pt7MxKOP6iyWHDk0fkVqfsfR0YnjW3jD6acGHYuIiNDpj8aon/vff/XzG29oeufgprNX4eTqVXWPgbqWvPXajh7Vwfrdu1Nev+vJzJk6Ywm05wTeZxolpHt3jW8wfbpIkyZ6bo4cakwKFFDjmAbPq9QagpVALyAIXYDWC1jpx3lBwB7gZuJCVdZIkKY38ElSebm/UmMIjp4/KjlG5JDXF7+e4jyyAjtP7nQZgoW7F8rAuQPlrT90pkmDLxtInrfypJ/PP62ZMMF/X3VmwOkvvvXW5E3rTAu2bVN3jXuoy549teU/bJjEc4XExupYQdOmcWmvXdP1FSAyduz1Lfv14Nw57d2UKqUPcX+NrLPVDzrRICxMZ1316qVGwDHtO7Wk1hBUAn4CTjpa77OBSn6cdwfwq9v2y8DLCdJcV0Mw7u9xwnBk87HNKc4jK3Dy4kmXIdh5cme8Y6P/HC39f+6fTiVLYy5f1vUUpUplnTi9Fy+qS2fTpvQuibJrV9zAcsL58k731Xvv6edOneK2szJRUckbnzpyRGeVzZmT6nEAX6TLrCE0bsFXbtuPJHzoOwzBEWATMAOokFS+qTEEzSY2k+qfVM/WbiERkWsx14ThiBlu4q2uzpJs2qTd7OefT++SZF369dNHScKQkIcOacB298HgsDDPeVgCji9DkDMpCQpjTCngSUfPwJVeRPokdaqHfZJg+2dgiohcMcb0B74BEkWSN8b0A/oB3JRCwbEj54+w7MAy3mz2ZvLiBmRBgnIEUThPYQrlLkSenHnSuziBJThYVUU/+QQGDYJKldK7RFkPp7ZR797x95cvrwJ40dGq5ZQnT3xdLEuGwR/10Z+AIsDvwFy3V1JEAO7xFssD8QK9isgpEbni2PwSSBBFw5VuvIjUE5F6pUqlLFzyjG0zEISHaj6UovOzGsXzFQ+cSmhGIyxMRdtefz29S5I1ufFGFe7z9N8sVEij6OXNa41ABibJHgGQX0QSR2ZJmjVAFWNMZeBfoDvQ0z2BMaaMiBxxbHYE/knBdfzizpvuZHiz4dQoVSNQl8hUdKzakQpFvMRFzmqUL69Sw7Y3YLF4xKjryEcCY95GZwnNS3bmxtwDfITOIPpaRN4xxoShvqo5xpj3UANwDTgNDBCR7d5zhHr16slaf3XZLZbkMnasyng3aKCebduKtWQRjDHrRKSex2N+GILzQAHgChCN+v5FRAr7PDFAWENgSTHR0fDGG3D77fHjAjsJD9cgJo88ovGDW7SIH0jeYsnE+DIE/gSvLyQiOUQkn4gUdmynixGwWFLNkiXQr58GE3EnNlYju5UoAWPGaPjPV1+FU6fSpZgWy/XEr1CVxphixpgGxpimzlegC2axpDm5csGUKery6dlTY+A6mThRg4y//z4UK6bRxiIj7QCzJVuQpCEwxvQFlgG/AiMc78MDWyyLJUBUrgyff64P/Tfe0H0iGv7wrrvULQQ67fTpp+GLL9Rl5I6IhgqdOPH6lt1iCRD+9AgGA/WBAyLSArgdXWFssWROevSAxx/XAOaxsTog3K2bxi92HxweMUJ7ByNGxD9/4UI1AmPGXN9yWywBwp/po5dF5LIxBmNMHhHZboy5LeAls1gCyX//C99+q+sLAD79NHGaYsXghx+gZs34+3ft0nnxmzfDwYOQwkWOFktGwZ8eQYQxpiiqMfSbMeYnEiwMs1gyHQUKwIABSadr0QJuuCEugCHAwIGwZo1+npfsWdUWS4YjyR6BiHR2fBxujFmCrjJeENBSWSwZiUOHoEsXGD4cSpaEevW0lzBvno4rWCyZHL9mDTkRkT9EZI6IXA1UgSyWDEfp0jqd9PHHdaHZjBk6ltC+PRQs6F8eV+1fxpJxSZYhsFiyJblz64DxsWNQrhzcd5/uP3cO3n4b/vwz8TlXr+oYREyMDkgXLQr33hvnXrJYMhDWEFgs/tCzp65G/vBDHSgGNRDvvQfffZc4/QcfqOrp8uW6orlHD5g7F1asuL7ltlj8wJ91BAONMcWuR2EslgxLUBBMmxZfmiJvXmjdWh/w7i39vXtV8fTBB6F5c5VfHjsWCheG8eO9XyM6GrZtC1gVLBZv+NMjuBFYY4yZboxpZ7K7mL/F4k6HDjqFdOtW3RaBZ56BnDl1dbKTAgXg4Yd1Ourp04nzOX0aWrbUQegZM+IfE9GexMWLgauHJVvjj9bQa0AVYAIaUWyXMeZdY8wtAS6bxZLxuecefZ8wQd9//x0WLNCxg3Ll4qft108NxLp1ifMpWFCDt1SpoukiIuKOvfSSzk4qUya+cUkpW7bYwWtLPPwaI3CEOTvqeF0DigEzjDGjAlg2iyXjU64c3HmnxjxwbjdurL2ChNSpowPOrVvH7ZszB06e1PGG+fPVzXT1qo4vxMZqHIXRo1X64oEHEhuX5LBnD2zYALVr6yrqzMQvv8CoUXawPVB4i2HpfAHPAutQjaGHgFyO/TmAPUmdn9av1MQstlgCwuXLIufO6ecrV0Sio32nj43VIPRhYbpM7bnn4h//8kuR6tVF/vxT4/x26yYSExN3/Px5kZEjNUi6v0RHi9x8s0iXLiLNmonccIPmk1koUUK/qzlz0rskmRZSE7MYKAk8ICIHEhiQWGPMvWlplCyWTEmePPoCbdn7QkQHkLds0XGBRx6Bd9+Nn+aJJ3Q8IW9elc2uXz9OCgNg/XoYNkw/v+Rn8MCpU3UQ+8MPdV3EHXeoVlJmUFeNjdWYyAUL6piMJc3xxzU0D40eBoAxppAxpiGAiPgMLekYXN5hjNltjBnmI10XY4wYYzwGTbBYsgzGqLLpmTMwciR8803cdFT3NM59TZrEGRknTZvqmoT33osfL+HKFc+uk9hYeOcdve5992kEts6d1eV0IhPoR+7dCxcuqOHKkUNXeh84kPR5Fr/xxxB8Blxw277o2OcTY0wQMA5oD9QAehhjEgUMNsYUQt1Pq/0psMWS6Xn/fZ0m+tJLKQ+FOXIknD8f15s4dEiDxHfurA9Nd378EbZvh9dei+tZvPNRu1UQAAAgAElEQVSOGo2//055Pa4XGzbo++236wK91q11VffOnf6df+2aakPZ8QWv+GMIjMO/BKhLCP9USxsAu0Vkr6gkxVSgk4d0bwGjgMt+5GmxZH7y5oVq1VKXR82a0Ls3fPKJDkBXqAD33w8//6yD1+4t5vBwuO02XdfgpHp1OHw4c7haOnVSd1hwsK7n+OwzOH4cQkNh8uSkz3/xRZUGmTMn8GXNpPhjCPYaY541xuRyvAYDe/04rxxwyG07wrHPhTHmdqCCiPziKyNjTD9jzFpjzNoTmaEra7FcD0aM0NCaMTG6/d13OvPowAG45Rb46SfdP2SIHgsKin9+oUL6/uWX8NRTuqBt5Up44YXE4xbpSe7c2htwjr+0aBEXX/rRR+HJJ7239n/8MS5uxMqV16e8mRB/DEF/oDHwL/owbwj08+M8T31e190yxuQAxgD/l1RGIjJeROqJSL1SpUr5cWmLJRtQvryuNyhbNm5fmzawahXUqAF//KH7SpbUh6Y3Dh3SFc/Fi2tvYuxYbYUHipgYlfJe4KeI8WuvJdZzKl8eFi1So3X4cGJ3GMDu3SoUWL++9pr+85/Ulz2L4o8M9XGgewryjgAquG2XJ34cg0JALWCpY7HyjcAcY0xHEVmbgutZLNmPHB7actWqwaZN/ucRFqaupT/+0AVyHTpAkSL6wP7mG13TkLA3AXD2rIrpJZevv4Zx43S1dLt2vtMeParjGSVLqpFyJ2dOXVsAnsdacuXS2VGff64xJUDXaCQ1sysbYiSJARRjTF7gCaAm4JreICJ9kjgvJ7ATaIn2JtYAPUVkq5f0S4EXkjIC9erVk7VrrZ2wWALOnDnaM+jQQVvgZ87o/mnT9N2pwvrGG9rqPnJE3U0FC2o4z0GD9KH79ttxPYzTp6FqVc1vw4akB8vnz1fjtHQpNGvmPd2+fTo+8uyzOhOqQAFdqe3Of/6jMah37lQjks0wxqwTEY8zM/1xDU1GW+ttgT/Qlv35pE4SkWvAQHQh2j/AdBHZaowJM8Z09LfwFoslnejYUYPxLFmivvbwcH3Yx8bq8S5d1GXToIH68CtUUOlt0J5CaKimfegh+PVX3f/662pQvv1WjcCRI77lLpwzhkJCfJf1q69g8GBt/TdsqDIdCalaVQ3G/PnJ+hqyBd5WmjlfwAbH+ybHey5gcVLnBeplVxZbLBmIyEiR994TCQ0VGTpUZPfu+MdPnxapU0ckXz6R338XqVlTZOBAPfbPPyJ58+pKam906aIropPi4kWRypV19fGNN4qsXp04zdWreuy++/yvXxYCHyuL/ekRRDvezxpjaqGhKiuluUWyWCyZj8KFdZXzunXqr78lgRZlsWLqJqpbVz+vX68L4UCntAYH6xhAdHTivAH+/Vd7G0mRP79OJe3aVddGNGiQOE2uXDp4PHcuPPdc3P4vvoDISP/q643//Mf/Vd4ZEH/GCPoCM4FgYBJQEHhdRL4IeOk8YMcILJZMiIjn8YC5c3WV9PjxOg3UE5cvJ159nVIOHtSxjVq1dErttm0qwlexoo591EuBuMGGDeoGA13pXbx42pQ1jUnxGIFjiuc5ETkjIstE5GYRuSG9jIDFYsmkeBsUvucenQ30f/+nq589kVZGAOCmm2DjxriocjVqwLJluvq4cWMYOlQXq/lLbCw8/bR+fvhhlfnIhPg0BKKriAdep7JYLJbshjEwZYq6dpYujX/su++ge3ftEQSSxo21Vd+jh4ryNWnivxzF9Om6buObb9Q1VaZMYMsaIPxxDb0OXAKmoTpDAIiIhzBLgce6hiyWLIj7moTYWH0wP/887NihawmuFzt2qPvIPWaEO8OG6eK0Bx7QNEFBagx69lTjsXKlup1Ssr4iwPhyDfljCPZ52C0icnNaFC65WENgsWRhfv1V1y3ExGhvYcgQbaVfb86fh1de0Qd+ixa6b+ZMnTKbJ4+6gBo3jr/i2TlW8NVXKiWeEBE4d04X66UDqVpHICKVPbzSxQhYLJYszuLF+vD95htteaeHEQBdcPbLL2qIrl3TtQ6DB+sMptOndS3Cyy/HPyckRGdNTZ/uOc8//tAxirAw77Ok3Dl3Tq9/+HDSaVNJksvrjDGPetovIt+mfXEsFku2JqPoAeXLp3LhXbpoC79/f5g9W1dK58/vWRrDGF08N3q0zh4qUSL+8bvuUoXYN9/UvL74QmcrOeUvErJli8aoDgqCDz5I+zq64c86gvpur7uA4YBdGWyxWLI2Dzyg0eQGD9bV0PXq6VRTX3Ttqm6tWbPi79+xQ3WhvvlGV2lHROhahzp14tIkHAtp3FhlNWbNCngsBX9cQ4PcXk8CtwNWtclisWRtjIGPPlK30JAh/p0TEgK33qrrI5wcPqz7335btzt31vULX36pAYYANm+GKlVUMvv4cXWJxcToArh9+2B1YON2JTlYnOgEY3KhchPVA1Mk39jBYovFcl359Vd9uCdcNe2NXbvU5ZM7t7bkBw1SN9COHXCzl+HVK1d0HcKMGVC5shqPzZs1vvQNN2i8iI8/TlU1fA0W+zNG8DNxcQRyoGEnvYyGWCwWSxajbdvkpa9SRd9PndIxgdWroW9f70YAdCbS1Kkqlvf119ojcObz/PP+G6EU4s/0UXft12vAARGJCGipfGB7BBaLJVOwfbvGVj56VHsJ5csnfY6Iuo1q1Eh5PGsvpKpHABwEjojIZUdm+YwxlURkfxqW0WKxWLIW1aqpnMXx4/4ZAdCHf82aifefP6+ziO64I23L6MCfWUM/ALFu2zGOfRaLxWLxReHCOr6QWoYM0d5FgLSM/DEEOUXEFTnC8dmvWUPGmHbGmB3GmN3GmGEejvc3xmw2xoQbY1YYY2r4X3SLxWLJJnTtqusJvAnzpRJ/DMEJ94hixphOwMmkTjLGBAHjgPboAHMPDw/670UkWERCgFFAOi0jtFgslgxMq1Yazc193UEa4s8YQX/gO2PMJ47tCMDjauMENAB2i8heAGPMVKATsM2ZQETOuaUvQNzsJIvFYrE4CQrSV4BI0hCIyB6gkTGmIDrLKMl4xQ7KAYfctiOAhgkTGWOeAZ5H3U13e8rIGNMP6Adw0003+Xl5i8VisfhDkq4hY8y7xpiiInJBRM4bY4oZY972I29Pc58StfhFZJyI3AK8BLzmKSMRGS8i9USkXqlSpfy4tMVisVj8xZ8xgvYicta5ISJngHv8OC8CqOC2XR7wJaM3Fbjfj3wtFovFkob4YwiCjDF5nBvGmHxAHh/pnawBqhhjKhtjcgPdgTnuCYwxVdw2OwC7/MjXYrFYLGmIP4PF/wMWGWMmoq6dPkCSEtQics0YMxD4FQgCvhaRrcaYMGCtiMwBBhpjWgHRwBngsRTWw2KxWCwpxC/ROWNMO6AV6vdfKCK/Brpg3rASExaLxZJ8UisxgYgsABY4MrvTGDNORJ5JwzKmiujoaCIiIrgc6CDXFksakDdvXsqXL0+uXLnSuygWC+CnITDGhAA9gG7APuDHQBYquURERFCoUCEqVaqESWOhJoslLRERTp06RUREBJUrV07v4lgsgA9DYIypig7w9gBOAdNQV1KL61Q2v7l8+bI1ApZMgTGGEiVKcOLEifQuisXiwlePYDuwHLhPRHYDGGOeuy6lSgHWCFgyC/a3aslo+Jo++iBwFFhijPnSGNMSz4vELBaLxZKJ8WoIRGSWiHQDqgFLgeeA0saYz4wxba5T+TIVs2bNwhjD9gApBKYXY8eOpXr16vTq1Sve/vDwcObNm5fs/A4fPkyXLl2STHfPPfdw9uzZJNMll969ezNjxgyfaSZNmsThw77WP1osWQd/gtdfFJHvRORedHVwOJBIUtoCU6ZMoUmTJkydOjWg14mJiQlo/gn59NNPmTdvHt999128/b4MwbVr17zmV7Zs2SQfxADz5s2jaNGiyStsGmENgSU74desIScichr4wvHKsDSf1DzRvq41u/J0/aeJio7inu8SK2T0DulN75DenIw6SZfp8VurS3svTfKaFy5c4M8//2TJkiV07NiR4cOHu46NGjWKyZMnkyNHDtq3b8/IkSPZvXs3/fv358SJEwQFBfHDDz9w6NAh3n//fX755RcABg4cSL169ejduzeVKlWiT58+LFy4kIEDB3L+/HnGjx/P1atXufXWW5k8eTL58+fn2LFj9O/fn7179wLw2WefMX/+fEqWLMngwYMBePXVVyldujTPPvtsvDp8+OGHfP311wD07duXIUOGuPLq2LEjffr04bnndJjo6tWrvPHGG1y6dIkVK1bw8ssv888//3D48GH2799PyZIleffdd3nkkUe4ePEiAJ988gmNGzdm//793HvvvWzZsoVJkyYxZ84coqKi2LNnD507d2bUqFEAVKpUibVr13LhwgXat29PkyZNWLlyJeXKleOnn34iX758rFmzhieeeIICBQrQpEkT5s+fz5YtW+LVS0QYNGgQixcvpnLlyrivnQkLC+Pnn3/m0qVLNG7cmC+++IKZM2eydu1aevXqRb58+fjrr78YPXp0onTW12/JKvgjMWHxg9mzZ9OuXTuqVq1K8eLFWb9+PQDz589n9uzZrF69mo0bN/Liiy8C0KtXL5555hk2btzIypUrKVOmTJLXyJs3LytWrKB79+488MADrFmzho0bN1K9enUmTJgAwLPPPkuzZs3YuHEj69evp2bNmjzxxBN88803AMTGxjJ16tREbp5169YxceJEVq9ezapVq/jyyy/ZsGEDn3/+OWXLlmXJkiUuIwCQO3duwsLC6NatG+Hh4XTr1s2Vz08//cT333/PDTfcwG+//cb69euZNm1aIsPjJDw8nGnTprF582amTZvGoUOHEqXZtWsXzzzzDFu3bqVo0aLMnDkTgMcff5zPP/+cv/76iyAvMr2zZs1ix44dbN68mS+//JKVK1e6jg0cOJA1a9awZcsWLl26xC+//EKXLl2oV68e3333HeHh4eTLl89jOoslq5CsHkFmwVcLPn+u/D6Pl8xf0q8eQEKmTJnCkCFDAOjevTtTpkwhNDSU33//nccff5z8+fMDULx4cc6fP8+///5L586dAX3A+4PzYQuwZcsWXnvtNc6ePcuFCxdo27YtAIsXL+bbb1UBJCgoiCJFilCkSBFKlCjBhg0bOHbsGLfffjslSpSIl/eKFSvo3LkzBQoUAOCBBx5g+fLl3H777cn6Hjp27Ei+fPkAXeg3cOBAwsPDCQoKYufOnR7PadmyJUWKFAGgRo0aHDhwgAoVKsRLU7lyZUJCQgCoW7cu+/fv5+zZs5w/f57GjRsD0LNnT48P6GXLltGjRw+CgoIoW7Ysd98dp3a+ZMkSRo0aRVRUFKdPn6ZmzZrcd999ifLwN53FkhnJkobgenPq1CkWL17Mli1bMMYQExODMYZRo0YhIolcCN5kPXLmzElsbFx46IQrpZ0PadABz9mzZ1OnTh0mTZrE0qVLfZaxb9++TJo0iaNHj9KnT59Ex/2RGvEH9zKOGTOG0qVLs3HjRmJjY70avDx54jQMg4KCPI4vJExz6dKlZJXZkxvn8uXLPP3006xdu5YKFSowfPhwj6vT/U1nsWRWrGsoDZgxYwaPPvooBw4cYP/+/Rw6dIjKlSuzYsUK2rRpw9dff01UVBQAp0+fpnDhwpQvX57Zs2cDcOXKFaKioqhYsSLbtm3jypUrREZGsmjRIq/XPH/+PGXKlCE6OjreIG7Lli357LPPAB1UPndOg8B17tyZBQsWsGbNGlfvwZ2mTZsye/ZsoqKiuHjxIrNmzeKuu+7yWe9ChQpx/rz3OEWRkZGUKVOGHDlyMHny5DQf5C5WrBiFChVi1apVAF4H6Zs2bcrUqVOJiYnhyJEjLFmyBIgztCVLluTChQvxBrDd6+YrncWSFbCGIA2YMmWKy83j5MEHH+T777+nXbt2dOzYkXr16hESEsL7778PwOTJkxk7diy1a9emcePGHD16lAoVKtC1a1dq165Nr169fLpl3nrrLRo2bEjr1q2pVq2aa//HH3/MkiVLCA4Opm7dumzduhVQn36LFi3o2rWrR196aGgovXv3pkGDBjRs2JC+ffsm6RZq0aIF27ZtIyQkhGnTpiU6/vTTT/PNN9/QqFEjdu7cGa+3kFZMmDCBfv36cccddyAiLheTO507d6ZKlSoEBwczYMAAmjVrBkDRokV58sknCQ4O5v7776d+/fquc3r37k3//v0JCQkhT548XtNZLFkBv9RHMxKe1Ef/+ecfqlevnk4lyhzExsYSGhrKDz/8QJUqVZI+IZNw4cIFChYsCMDIkSM5cuQIH3/8cTqXKmnsb9ZyvfGlPmp7BNmAbdu2ceutt9KyZcssZQQA5s6dS0hICLVq1WL58uW89prHaKcWi8UHdrA4G1CjRg3XuoKsRrdu3eLNprJYLMknoD0CY0w7Y8wOY8xuY0yi1cjGmOeNMduMMZuMMYuMMRUDWR6LxWKxJCZghsAYEwSMA9oDNYAexpgaCZJtAOqJSG1gBjAqUOWxWCwWi2cC2SNoAOwWkb0ichWYCnRyTyAiS0QkyrG5CtUyslgsFst1JJCGoBzgrhUQ4djnjSeA+Z4OGGP6GWPWGmPW2oAeFovFkrYE0hB4UuTyOFfVGPMwUA8Y7em4iIwXkXoiUq9UqVJpWMS0JbvJUCeXpUuXcu+99wIwZ84cRo4c6TGdczqoN86ePcunn37q2vZX1jq5uJfXGymV4rZYMhKBNAQRgLtgTHkgka6vMaYV8CrQUUSuBLA8ASe7yVCnho4dOzJsWMrUzBMaAn9lrQOBNQSWrEAgp4+uAaoYYyoD/6Lxj3u6JzDG3I5KWrcTkeNpcdEhC4YQfjQ8LbJyEXJjCB+1+8hnmuwmQw3QsGFDvv76a2rWrAlA8+bN+eCDD4iJiWHIkCFcunSJfPnyMXHiRG677bZ415o0aRJr167lk08+Yd++ffTs2ZNr167Rrl27eN9pp06dOHPmDNHR0bz99tt06tSJYcOGsWfPHkJCQmjdujXPPPOMS9b68uXLDBgwgLVr15IzZ04+/PBDWrRo4VPu2p0FCxYwZMgQSpYsSWhoqGv/33//nahOlStXTiTFXbly5STrbrFkNAJmCETkmjFmIPArEAR8LSJbjTFhwFoRmYO6ggoCPzhEwQ6KSMdAlSmQeJKhDg0NjSdDnT9/fk6fPg2oDPWwYcPo3Lkzly9fJjY21qP8sjtOGWpQobsnn3wSgNdee40JEyYwaNAglwz1rFmziImJ4cKFC5QtW5YHHniAwYMHu2So//7773h5u8tQiwgNGzakWbNmfP755yxYsIAlS5ZQsmTJeOd0796d6dOnM2LECI4cOcLhw4epW7cu586dY9myZeTMmZPff/+dV155xSUb7YnBgwczYMAAHn30UcaNGxevvrNmzaJw4cKcPHmSRo0a0bFjR0aOHMmWLVsID1eDv3//ftc5zvM3b97M9u3badOmjUv1NDw8nA0bNpAnTx5uu+02Bg0aFE/l9PLlyzz55JMsXryYW2+9Nd76hGrVqnmsU1hYmMugAcmuu8WSEQjogjIRmQfMS7DvDbfPrdL6mkm13ANFdpSh7tq1K61bt2bEiBFMnz6dhx56CFCxuccee4xdu3ZhjCE6Otpnvf7880/Xw/KRRx7hpZdeAlQR9ZVXXmHZsmXkyJGDf//9l2PHjvnMa8WKFQwaNAjQh3fFihVdhiApuevt27dTuXJl1+rrhx9+mPHjxyerTsmtu8WSEbAri9OA7CpDXa5cOUqUKMGmTZuYNm0aX3yhgetef/11WrRowaxZs9i/fz/NmzdPMi9PMtHfffcdJ06cYN26deTKlYtKlSolKf/sqx7+yF17izrmb51SUneLJb2xWkNpQHaVoQbt/YwaNYrIyEiCg4MBbRWXK6czhSdNmpRkHnfeeadrgN29LpGRkdxwww3kypWLJUuWcODAAcC3/HXTpk1deezcuZODBw/67aOvVq0a+/btY8+ePYD28tzL4qlOCcuS3LpbLBkBawjSgOwqQw3QpUsXpk6dSteuXV37XnzxRV5++WXuvPNOv2Y4ffzxx4wbN4769esTGRnp2t+rVy/Wrl3rChvprGeJEiW48847qVWrFkOHDo2X19NPP01MTAzBwcF069aNSZMmxesJ+CJv3ryMHz+eDh060KRJEypWjFM88VanhFLcya27xZIRsDLU2YSsKkOdWbG/Wcv1xspQZ3Oysgy1xWJJPXawOBuQlWWoLRZL6rE9AovFYsnmWENgsVgs2RxrCCwWiyWbYw2BxWKxZHOsIUgjkpJODiQ//PAD1atXp0WLFvH279+/n++//z5FeTZu3DjJNH379mXbtm0pyt8Xw4cPd6238Mbs2bMDcm2LJTtiDUEWYMKECXz66acsWbIk3n5fhsCTvII7K1euTPK6X331FTVqJIw+en2whsBiSTuypiFo3jzxy6lfHxXl+bhTDuDkycTHUsiBAwdo2bIltWvXpmXLlhw8eBDQFnytWrWoU6cOTZs2BWDr1q00aNCAkJAQateuza5duxLlN2XKFIKDg6lVq5ZLmC0sLIwVK1bQv3//RKtshw0bxvLlywkJCWHMmDFMmjSJhx56iPvuu482bdpw4cIFWrZsSWhoKMHBwfz000+uc509nKVLl9K8eXO6dOlCtWrV6NWrl0vPp3nz5jgX9xUsWJBXX32VOnXq0KhRI5c43J49e2jUqBH169fnjTfe8Npzeuedd7jtttto1aoVO3bscO3/8ssvqV+/PnXq1OHBBx8kKiqKlStXMmfOHIYOHUpISAh79uzxmM5isfiJiGSqV926dSUh27Zti7+jWbPEr3Hj9NjFi56PT5yox0+cSHzMDwoUKJBo37333iuTJk0SEZEJEyZIp06dRESkVq1aEhERISIiZ86cERGRgQMHyv/+9z8REbly5YpERUXFy+vff/+VChUqyPHjxyU6OlpatGghs2bNclS3maxZsybR9ZcsWSIdOnRwbU+cOFHKlSsnp06dEhGR6OhoiYyMdFT7hNxyyy0SGxsbrz5LliyRwoULy6FDhyQmJkYaNWoky5cvT3RdQObMmSMiIkOHDpW33npLREQ6dOgg33//vYiIfPbZZx6/p7Vr10qtWrXk4sWLEhkZKbfccouMHj1aREROnjzpSvfqq6/K2LFjRUTksccekx9++MF1zFu6jEqi36zFEmBQ+X+Pz9WsuaDMlxJn/vy+j5cs6ft4Mvjrr7/48ccfAZVXfvHFFwEVWevduzddu3blgQceAOCOO+7gnXfeISIiggceeCDRCuA1a9bQvHlznKE6e/XqxbJly7j//vuTVabWrVtTvHhxwLvM84033hjvnAYNGlC+fHkAQkJC2L9/P02aNImXJnfu3K6wjnXr1uW3335zfQdOcb2ePXvywgsvJCrT8uXL6dy5s0uqu2PHuJAU3uS2E+JvOovFkpiAuoaMMe2MMTuMMbuNMYniEhpjmhpj1htjrhlj0j7obAbDKXH8+eef8/bbb3Po0CFCQkI4deoUPXv2ZM6cOeTLl4+2bduyePHieOdKGmlCuUtZu8s8h4eHU7p0aY8yz/7IN+fKlctVP29pfOFN/rl379588sknbN68mTfffNOrDLW/6SwWS2ICZgiMMUHAOKA9UAPoYYxJOLJ4EOgNpGxqSwancePG8eSVna3oPXv20LBhQ8LCwihZsiSHDh1i79693HzzzTz77LN07NiRTZs2xcurYcOG/PHHH5w8eZKYmBimTJlCs2bNfF7fl1wzeJd5TksaNWrkCjrjLZZz06ZNmTVrFpcuXeL8+fP8/PPPrmPe5LYT1s1bOovFkjSBdA01AHaLyF4AY8xUoBPgmuohIvsdx2I9ZZCZiIqKcrlPAJ5//nnGjh1Lnz59GD16NKVKlWLixIkADB06lF27diEitGzZkjp16jBy5Ej+97//kStXLm688UbeeOONePmXKVOG9957jxYtWiAi3HPPPXTq1MlnmWrXrk3OnDmpU6cOvXv3plixYvGO9+rVi/vuu88lke0uZ51WfPTRRzz88MN88MEHdOjQwRUhzJ3Q0FC6detGSEgIFStWjBcHwSm3XbFiRYKDg10P/+7du/Pkk08yduxYZsyY4TWdxWJJmoDJUDtcPe1EpK9j+xGgoYgM9JB2EvCLiMzwklc/oB/ATTfdVDdhy9VK+mZcoqKiyJcvH8YYpk6dypQpU+LNTsqu2N+s5XrjS4Y6kD0CT07fFFkdERkPjAeNR5CaQlmuL+vWrWPgwIGICEWLFuXrr79O7yJZLJYEBNIQRAAV3LbLA4cDeD1LBuSuu+5i48aN6V0Mi8Xig0DOGloDVDHGVDbG5Aa6A3MCeD2LxWKxpICAGQIRuQYMBH4F/gGmi8hWY0yYMaYjgDGmvjEmAngI+MIYszVQ5bFYLBaLZwK6oExE5gHzEux7w+3zGtRlZLFYLJZ0ImtqDVksFovFb6whSCMyogx1ctm/fz+1atUCYO3atTz77LMe01WqVImTJ0/6zOvdd9+Nt+2PrHVycS+vrzQpleK2WLIL1hBkAbzJUKeGevXqMXbs2BSfn9AQ+CNrHQisIbBYkibrGYIhQzzLTKfmNWRIioqS3jLU3bp1Y968uCGa3r17M3PmTPbv389dd91FaGgooaGhHh/SS5cudYnInTp1ijZt2nD77bfz1FNPxdM9uv/++6lbty41a9Zk/PjxgMpfX7p0iZCQEHr16gXE9ZhEhKFDh1KrVi2Cg4OZNm2a63re5K7dWbduHXXq1OGOO+5g3Lhxrv3e6pRQitufulss2Q5vsqQZ9ZWkDPXgwZ5lplPzGjw40TUTkhFlqH/88Ud59NFHXXmWL19eoqKi5OLFi3Lp0iUREdm5c6c4v9N9+/ZJzZo1RSS+hPWgQYNkxIgRIiLyyy+/CCAnTpwQEXFJWkXqJpMAAArnSURBVEdFRUnNmjVdctAJvw/n9owZM6RVq1Zy7do1OXr0qFSoUEEOHz7sU+7aneDgYFm6dKmIiLzwwguu8nqrU0Ipbm/prjdWhtpyvSFbyVB/9FF6l8BFestQt2/fnmeffZYrV66wYMECmjZtSr58+YiMjGTgwIGEh4cTFBTEzp07fdZj2bJlrnp06NAhnmbR2LFjmTVrFgCHDh1i165dlChRwmteK1asoEePHgQFBVG6dGmaNWvGmjVrKFy4cJJy15GRkZw9e9YltvfII48wf/58AKKjo/2qk7/pLJbsRNYzBBkYdxnq1atXM3fuXEJCQggPD6dnz540bNiQuXPn0rZtW7766ivuvvtu17mSAk2ovHnz0rx5c3799VemTZtGjx49ABgzZgylS5dm48aNxMbGkjdvXr/L7s7SpUv5/fff+euvv8ifPz/NmzdPUv7ZVz2SkrsWEa9y1f7WKSV1t1iyOllvjCADkd4y1KAqnRMnTmT58uWuYC2RkZGUKVOGHDlyMHnyZGJiYnzm0bRpU5e08/z58zlz5owrn2LFipE/f362b9/OqlWrXOfkypWL6Ohoj3lNmzaNmJgYTpw4wbJly2jQoEGS9QAoWrQoRYoUYcWKFQDx5Ka91SmhXHVy626xZAesIUgjnDLUzteHH37I2LFjmThxIrVr12by5Ml8/PHHgMpQOwd9mzZtSp06dZg2bRq1atUiJCSE7du38+ijj8bL312Guk6dOoSGhiYpQw3Qpk0bli1bRqtWrcidOzcATz/9NN988w2NGjVi586d8YLVeOLNN99k2bJlhIaGsnDhQm666SYA2rVrx7Vr16hduzavv/46jRo1cp3Tr18/ateu7RosdtK5c2dq165NnTp1uPvuuxk1alSiiGi+mDhxIs888wx33HEH+fLlc+33Vid3Ke4xY8Yku+4WS3YgYDLUgaJevXriDJjuxEr6WjIb9jdrud74kqG2PQKLxWLJ5lhDYLFYLNmcLGMIMpuLy5J9sb9VS0YjSxiCvHnzcurUKfsHs2R4RIRTp07ZaauWDEWWWEdQvnx5IiIiOHHiRHoXxWJJkrx587oWzlksGYEsYQhy5cpF5cqV07sYFovFkikJqGvIGNPOGLPDGLPbGDPMw/E8xphpjuOrjTGVAlkei8VisSQmYIbAGBMEjAPaAzWAHsaYGgmSPQGcEZFbgTHAfwJVHovFYrF4JpA9ggbAbhHZKyJXgalAwqWwnYBvHJ9nAC2NNzEZi8VisQSEQI4RlAMOuW1HAA29pRGRa8aYSKAEEC/8lTGmH9DPsXnBGLMjhWUqmTDvbEJ2rHd2rDNkz3pnxzpD8utd0duBQBoCTy37hPM7/UmDiIwHxqe6QMas9bbEOiuTHeudHesM2bPe2bHOkLb1DqRrKAKo4LZdHjjsLY0xJidQBDgdwDJZLBaLJQGBNARrgCrGmMrGmNxAd2BOgjRzgMccn7sAi8WuCrNYLJbrSsBcQw6f/0DgVyAI+FpEthpjwtCQaXOACcBkY8xutCfQPVDlcZBq91ImJTvWOzvWGbJnvbNjnSEN653pZKgtFovFkrZkCa0hi8VisaQcawgsFoslm5NtDEFSchdZAWNMBWPMEmPMP8aYrcaYwY79xY0xvxljdjnei6V3WdMaY0yQMWaDMeYXx3Zlh2zJLoeMSe70LmNaY4wpaoyZYYzZ7rjnd2STe/2c4/e9xRgzxRiTN6vdb2PM18aY48aYLW77PN5bo4x1PNs2GWNCk3u9bGEI/JS7yApcA/5PRKoDjYBnHPUcBiwSkSrAIsd2VmMw8I/b9n+AMY46n0HlTLIaHwMLRKQaUAetf5a+18aYcsCzQD0RqYVOROlO1rvfk4B2CfZ5u7ftgSqOVz/gs+ReLFsYAvyTu8j0iMgREVnv+HwefTCUI76UxzfA/elTwsBgjCkPdAC+cmwb4G5UtgSyZp0LA03RmXeIyFUROUsWv9cOcgL5HGuP8gNHyGL3W0SWkXhNlbd72wn4VpRVQFFjTJnkXC+7GAJPchfl0qks1wWHkuvtwGqgtIgcATUWwA3pV7KA8BHwIhDr2C4BnBWRa47trHi/bwZOABMdLrGvjDEFyOL3WkT+Bd4HDqIGIBJYR9a/3+D93qb6+ZZdDIFfUhZZBWNMQWAmMEREzqV3eQKJMeZe4LiIrHPf7SFpVrvfOYFQ4DMRuR24SBZzA3nC4RfvBFQGygIFUNdIQrLa/fZFqn/v2cUQ+CN3kSUwxuRCjcB3IvKjY/cxZ1fR8X48vcoXAO4EOhpj9qMuv7vRHkJRh+sAsub9jgAiRGS1Y3sGahiy8r0GaAXsE5ETIhIN/Ag0Juvfb/B+b1P9fMsuhsAfuYtMj8M3PgH4R0Q+dDvkLuXxGPDT9S5boBCRl0WkvIhUQu/rYhHpBSxBZUsgi9UZQESOAoeMMbc5drUEtpGF77WDg0AjY0x+x+/dWe8sfb8deLu3c4BHHbOHGgGRTheS34hItngB9wA7gT3Aq+ldngDVsQnaJdwEhDte96A+80XALsd78fQua4Dq3xz4xfH5ZuBvYDfwA5AnvcsXgPqGAGsd93s2UCw73GtgBLAd2AJMBvJktfsNTEHHQKLRFv8T3u4t6hoa53i2bUZnVCXrelZiwmKxWLI52cU1ZLFYLBYvWENgsVgs2RxrCCwWiyWbYw2BxWKxZHOsIbBYLJZsjjUElkyJMaaEMSbc8TpqjPnXbdsv5UljzES3efje0jxjjOmVRmVe4VDAdZZzWlrk65Z/hDGmaFrmacke2OmjlkyPMWY4cEFE3k+w36C/8ViPJ15njDErgIEiEh6g/COAWqLicxaL39gegSVLYYy51aFT/zmwHihjjBlvjFnr0LB/wy3tCmNMiDEmpzHmrDFmpDFmozHmL2PMDY40bxtjhrilH2mM+dvRsm/s2F/AGDPTce4Ux7VCklHm/xljPjPGLDfG7DTGtHfsz2eM+cYYs9kYs94Y09SxP6cxZoyjnpuMMU+7ZTfEIUK3yRhTNdVfqCVbYA2BJStSA5ggIreLqlUOE5F6qGZ/ay+xKIoAf4hIHeAvoI+XvI2INACGAk6jMgg46jh3JKr66o1pbq6hkW77KwDNgPuA8caYPKju/lURCQYeASY73F4DUMG1OiJSG9VYcnJMVITuK+B5H+WwWFzkTDqJxZLp2CMia9y2exhjnkB/72VRQ7EtwTmXRGS+4/M64C4vef/olqaS43MTNDAKIrLRGLPVR9m6eXENTXe4sHYYYw6hQUaaAKMd+W41xhwGbkWF1z4SkRjHMXfdevfy3eOjHBaLC2sILFmRi84PxpgqaPSyBiJy1hjzPyCvh3Ouun2Owft/44qHNJ5kgJNLwsE68ZGv8ZDeiafyWSw+sa4hS1anMHAeOOeQ7m0bgGusALoCGGOC0R5HcnnIoR5ZFXUT7QKWAb0c+VYHyqCiaguBAUZDsGKMKZ7qGliyNbbFYMnqrEfdQFuAvcCfAbjGf4FvjTGbHNfbgkbO8sQ0Y8wlx+djIuI0TLvRB/8NQD8RuWqM+S/whTFmM6pC+ahj/xeo62iTMeYaGqP28wDUy5JNsNNHLZZU4giIklNELjtcUQuBKhIXOjGp8/8HzBCR2YEsp8XiDdsjsFhST0FgkcMgGOApf42AxZIRsD0Ci8ViyebYwWKLxWLJ5lhDYLFYLNkcawgsFoslm2MNgcVisWRzrCGwWCyWbM7/Aw3INYa5jXWWAAAAAElFTkSuQmCC\n", 702 | "text/plain": [ 703 | "
" 704 | ] 705 | }, 706 | "metadata": { 707 | "needs_background": "light" 708 | }, 709 | "output_type": "display_data" 710 | } 711 | ], 712 | "source": [ 713 | "import matplotlib.pyplot as plt\n", 714 | "# summarize history for accuracy and loss\n", 715 | "plt.figure(figsize=(6, 4))\n", 716 | "plt.plot(hist.history['acc'], \"g--\", label=\"Accuracy of training data\")\n", 717 | "plt.plot(hist.history['val_acc'], \"g\", label=\"Accuracy of validation data\")\n", 718 | "plt.plot(hist.history['loss'], \"r--\", label=\"Loss of training data\")\n", 719 | "plt.plot(hist.history['val_loss'], \"r\", label=\"Loss of validation data\")\n", 720 | "plt.title('Model Accuracy and Loss')\n", 721 | "plt.ylabel('Accuracy and Loss')\n", 722 | "plt.xlabel('Training Epoch')\n", 723 | "plt.ylim(0)\n", 724 | "plt.legend()\n", 725 | "plt.show()" 726 | ] 727 | }, 728 | { 729 | "cell_type": "code", 730 | "execution_count": 22, 731 | "metadata": {}, 732 | "outputs": [ 733 | { 734 | "name": "stdout", 735 | "output_type": "stream", 736 | "text": [ 737 | "3980/3980 [==============================] - 2s 448us/step\n", 738 | "Accuracy: 0.714824120603015\n" 739 | ] 740 | } 741 | ], 742 | "source": [ 743 | "score = model1.evaluate(X_test, y_test, verbose=1)\n", 744 | "print(\"Accuracy: \", score[1])" 745 | ] 746 | }, 747 | { 748 | "cell_type": "code", 749 | "execution_count": 25, 750 | "metadata": {}, 751 | "outputs": [], 752 | "source": [ 753 | "from sklearn import metrics" 754 | ] 755 | }, 756 | { 757 | "cell_type": "code", 758 | "execution_count": 26, 759 | "metadata": {}, 760 | "outputs": [], 761 | "source": [ 762 | "predict1 = [1 if a>0.5 else 0 for a in model1.predict(X_test)]" 763 | ] 764 | }, 765 | { 766 | "cell_type": "code", 767 | "execution_count": 27, 768 | "metadata": {}, 769 | "outputs": [ 770 | { 771 | "name": "stdout", 772 | "output_type": "stream", 773 | "text": [ 774 | "Accuracy = 0.714824120603015\n", 775 | "Precision = 0.767266395821242\n", 776 | "Recall = 0.6429961089494164\n", 777 | "F1 Score = 0.6996559936491135\n" 778 | ] 779 | } 780 | ], 781 | "source": [ 782 | "print(\"Accuracy = {}\\nPrecision = {}\\nRecall = {}\\nF1 Score = {}\".format(metrics.accuracy_score(y_test, predict1), metrics.precision_score(y_test, predict1),metrics.recall_score(y_test, predict1),metrics.f1_score(y_test, predict1)))\n" 783 | ] 784 | }, 785 | { 786 | "cell_type": "markdown", 787 | "metadata": {}, 788 | "source": [ 789 | "# Another Model based on 1DCNN architecture" 790 | ] 791 | }, 792 | { 793 | "cell_type": "code", 794 | "execution_count": 11, 795 | "metadata": {}, 796 | "outputs": [ 797 | { 798 | "name": "stderr", 799 | "output_type": "stream", 800 | "text": [ 801 | "Using TensorFlow backend.\n" 802 | ] 803 | } 804 | ], 805 | "source": [ 806 | "from sklearn import metrics\n", 807 | "from sklearn.metrics import classification_report\n", 808 | "from sklearn import preprocessing\n", 809 | "\n", 810 | "import keras\n", 811 | "from keras.models import Sequential\n", 812 | "from keras.layers import Dense, Dropout, Flatten, Reshape, GlobalAveragePooling1D\n", 813 | "from keras.layers import Conv2D, MaxPooling2D, Conv1D, MaxPooling1D\n", 814 | "from keras.utils import np_utils\n", 815 | "from keras.models import Sequential\n", 816 | "from keras.layers import Reshape, Conv1D, MaxPooling1D, GlobalAveragePooling1D, Dropout, Dense\n", 817 | "import matplotlib.pyplot as plt" 818 | ] 819 | }, 820 | { 821 | "cell_type": "code", 822 | "execution_count": 12, 823 | "metadata": {}, 824 | "outputs": [], 825 | "source": [ 826 | "# same dataset used in this model\n", 827 | "X_train = np.asarray(np.reshape(x_train, (x_train.shape[0], 14, 1)))\n", 828 | "X_test = np.asarray(np.reshape(x_test, (x_test.shape[0], 14, 1)))" 829 | ] 830 | }, 831 | { 832 | "cell_type": "code", 833 | "execution_count": 13, 834 | "metadata": {}, 835 | "outputs": [ 836 | { 837 | "data": { 838 | "text/plain": [ 839 | "1" 840 | ] 841 | }, 842 | "execution_count": 13, 843 | "metadata": {}, 844 | "output_type": "execute_result" 845 | } 846 | ], 847 | "source": [ 848 | "temporal_dimension = X_train.shape[1]\n", 849 | "num_channels =X_train.shape[2]\n", 850 | "num_classes = 2\n", 851 | "X_train.shape[2]" 852 | ] 853 | }, 854 | { 855 | "cell_type": "code", 856 | "execution_count": 14, 857 | "metadata": {}, 858 | "outputs": [ 859 | { 860 | "name": "stdout", 861 | "output_type": "stream", 862 | "text": [ 863 | "(11000, 14, 1)\n" 864 | ] 865 | } 866 | ], 867 | "source": [ 868 | "print(X_train.shape)" 869 | ] 870 | }, 871 | { 872 | "cell_type": "code", 873 | "execution_count": 15, 874 | "metadata": {}, 875 | "outputs": [ 876 | { 877 | "data": { 878 | "text/plain": [ 879 | "14" 880 | ] 881 | }, 882 | "execution_count": 15, 883 | "metadata": {}, 884 | "output_type": "execute_result" 885 | } 886 | ], 887 | "source": [ 888 | "temporal_dimension" 889 | ] 890 | }, 891 | { 892 | "cell_type": "code", 893 | "execution_count": 16, 894 | "metadata": {}, 895 | "outputs": [ 896 | { 897 | "name": "stdout", 898 | "output_type": "stream", 899 | "text": [ 900 | "1\n" 901 | ] 902 | } 903 | ], 904 | "source": [ 905 | "num_channels =X_train.shape[2]\n", 906 | "print(num_channels)" 907 | ] 908 | }, 909 | { 910 | "cell_type": "code", 911 | "execution_count": 17, 912 | "metadata": {}, 913 | "outputs": [], 914 | "source": [ 915 | "input_shape = temporal_dimension" 916 | ] 917 | }, 918 | { 919 | "cell_type": "code", 920 | "execution_count": 18, 921 | "metadata": {}, 922 | "outputs": [ 923 | { 924 | "name": "stderr", 925 | "output_type": "stream", 926 | "text": [ 927 | "WARNING: Logging before flag parsing goes to stderr.\n", 928 | "W0714 11:20:04.142014 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n", 929 | "\n", 930 | "W0714 11:20:04.161777 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n", 931 | "\n", 932 | "W0714 11:20:04.163298 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n", 933 | "\n", 934 | "W0714 11:20:04.197792 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.\n", 935 | "\n", 936 | "W0714 11:20:04.243375 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.\n", 937 | "\n", 938 | "W0714 11:20:04.253115 23568 deprecation.py:506] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n", 939 | "Instructions for updating:\n", 940 | "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n" 941 | ] 942 | }, 943 | { 944 | "name": "stdout", 945 | "output_type": "stream", 946 | "text": [ 947 | "_________________________________________________________________\n", 948 | "Layer (type) Output Shape Param # \n", 949 | "=================================================================\n", 950 | "conv1d_1 (Conv1D) (None, 13, 100) 300 \n", 951 | "_________________________________________________________________\n", 952 | "conv1d_2 (Conv1D) (None, 12, 100) 20100 \n", 953 | "_________________________________________________________________\n", 954 | "max_pooling1d_1 (MaxPooling1 (None, 4, 100) 0 \n", 955 | "_________________________________________________________________\n", 956 | "conv1d_3 (Conv1D) (None, 3, 160) 32160 \n", 957 | "_________________________________________________________________\n", 958 | "conv1d_4 (Conv1D) (None, 2, 160) 51360 \n", 959 | "_________________________________________________________________\n", 960 | "global_average_pooling1d_1 ( (None, 160) 0 \n", 961 | "_________________________________________________________________\n", 962 | "dropout_1 (Dropout) (None, 160) 0 \n", 963 | "_________________________________________________________________\n", 964 | "dense_1 (Dense) (None, 2) 322 \n", 965 | "=================================================================\n", 966 | "Total params: 104,242\n", 967 | "Trainable params: 104,242\n", 968 | "Non-trainable params: 0\n", 969 | "_________________________________________________________________\n", 970 | "None\n" 971 | ] 972 | } 973 | ], 974 | "source": [ 975 | "model = Sequential()\n", 976 | "model.add(Conv1D(100,2, activation='relu', input_shape=(temporal_dimension, num_channels)))\n", 977 | "model.add(Conv1D(100, 2, activation='relu'))\n", 978 | "model.add(MaxPooling1D(3))\n", 979 | "model.add(Conv1D(160, 2, activation='relu'))\n", 980 | "model.add(Conv1D(160, 2, activation='relu'))\n", 981 | "model.add(GlobalAveragePooling1D())\n", 982 | "#model.add(Dense(100, activation='relu'))\n", 983 | "model.add(Dropout(0.1))\n", 984 | "model.add(Dense(num_classes, activation='sigmoid'))\n", 985 | "print(model.summary())" 986 | ] 987 | }, 988 | { 989 | "cell_type": "code", 990 | "execution_count": 19, 991 | "metadata": {}, 992 | "outputs": [], 993 | "source": [ 994 | "# labels converted into categorail for two classes \n", 995 | "y_train = np_utils.to_categorical(y_train, num_classes)" 996 | ] 997 | }, 998 | { 999 | "cell_type": "code", 1000 | "execution_count": 20, 1001 | "metadata": {}, 1002 | "outputs": [ 1003 | { 1004 | "name": "stderr", 1005 | "output_type": "stream", 1006 | "text": [ 1007 | "W0714 11:20:15.601167 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n", 1008 | "\n", 1009 | "W0714 11:20:15.619463 23568 deprecation_wrapper.py:119] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:3376: The name tf.log is deprecated. Please use tf.math.log instead.\n", 1010 | "\n", 1011 | "W0714 11:20:15.622763 23568 deprecation.py:323] From C:\\Users\\moona\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\ops\\nn_impl.py:180: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 1012 | "Instructions for updating:\n", 1013 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n" 1014 | ] 1015 | }, 1016 | { 1017 | "name": "stdout", 1018 | "output_type": "stream", 1019 | "text": [ 1020 | "Train on 9900 samples, validate on 1100 samples\n", 1021 | "Epoch 1/5\n", 1022 | "9900/9900 [==============================] - 2s 191us/step - loss: 0.6935 - acc: 0.5304 - val_loss: 0.6923 - val_acc: 0.5091\n", 1023 | "Epoch 2/5\n", 1024 | "9900/9900 [==============================] - 1s 116us/step - loss: 0.6884 - acc: 0.5307 - val_loss: 0.7126 - val_acc: 0.5082\n", 1025 | "Epoch 3/5\n", 1026 | "9900/9900 [==============================] - 1s 112us/step - loss: 0.6687 - acc: 0.5676 - val_loss: 0.6776 - val_acc: 0.5495\n", 1027 | "Epoch 4/5\n", 1028 | "9900/9900 [==============================] - 1s 110us/step - loss: 0.6491 - acc: 0.6034 - val_loss: 0.6853 - val_acc: 0.5759\n", 1029 | "Epoch 5/5\n", 1030 | "9900/9900 [==============================] - 1s 106us/step - loss: 0.6255 - acc: 0.6472 - val_loss: 0.6632 - val_acc: 0.6100\n", 1031 | "\n", 1032 | "--- Learning curve of model training ---\n", 1033 | "\n" 1034 | ] 1035 | }, 1036 | { 1037 | "data": { 1038 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXxU1fn48c9DCCTs2djCkkDYQgIBIiBaFkFAUBBBQKlKXahVRNufu9Za1H4tti5UXHABS1VwA1ERlwoCLpWgoIBAEtawGRIIhCSQ5fz+ODOTmWSSTJBJQvK8X695Zeaee++cm8B57j3nnueKMQallFJ1V73qroBSSqnqpYFAKaXqOA0ESilVx2kgUEqpOk4DgVJK1XEaCJRSqo7TQKCqhYhEiYgRkfo+rDtdRNZVRb3qEhHZLSIjqrseqvppIFAVcjQYp0UkvMTyjY7GPKp6auZRl8Yiki0iK6q7LrWBiCwUkUerux6qamggUL7aBVzl/CAi8UBw9VWnlEnAKWCkiLSpyi/25apGqZpMA4Hy1SLgWrfP1wH/dl9BRJqLyL9FJF1E9ojIgyJSz1EWICL/EJEjIrITGOtl21dE5KCI7BeRR0UkoBL1uw54AfgRmFZi3+1F5D1HvTJE5Fm3sptE5GcROSEiW0Wkr2O5EZEYt/VcZ8giMlRE0kTkHhE5BCwQkRAR+dDxHUcd79u5bR8qIgtE5ICjfJlj+WYRucxtvUDH7yih5AH68B2rReQREfnKcTyful/Ficg1jr9Lhog8UInfbcl6DBKR9SKS5fg5yK1suojsdHz/LhGZ5lgeIyJfOrY5IiJLzvT71dmngUD56lugmYj0cDTQU4D/lFjnX0BzoBMwBBs4fucouwm4FOgDJGLP4N29BhQAMY51RgI3+lIxEekADAVed7yudSsLAD4E9gBRQCSw2FF2JfCwY/1mwDggw5fvBFoDoUBHYAb2/9ICx+cOQC7wrNv6i4BGQE+gJfCUY/m/gd+6rTcGOGiM2ejlOyv6DoCrsb/zlkAD4E7HscYCzwPXAG2BMKAdlSQiocBHwFzHPp4EPhKRMBFp7Fh+iTGmKTAIcB7HI8CnQIjje/9V2e9WfmSM0Ze+yn0Bu4ERwIPA/wGjgc+A+oDBNrAB2K6ZWLftfg+sdrz/ArjZrWykY9v6QCvHtsFu5VcBqxzvpwPryqnfg8BGx/u2QCHQx/H5fCAdqO9lu0+A28vYpwFi3D4vBB51vB8KnAaCyqlTAnDU8b4NUASEeFmvLXACaOb4/A5wt49/F9d3OD6vBh50+3wLsNLx/iFgsVtZY8cxjChj367jLbH8GuC7Esu+cfyNGgPHgInuf0vHOv8G5gPtqvvfs75Kv/SKQFXGIuwZ53RKdAsB4dgz0D1uy/Zgz8DBNnj7SpQ5dQQCgYMickxEjgEvYs9qfXEt9koAY8wB4EtsVxFAe2CPMabAy3btgVQfv6OkdGNMnvODiDQSkRcdXS/HgTVAC8cVSXsg0xhztOROHPX9CpgoIi2AS5zHUlIF3+F0yO19DtDE8d7j92+MOYnvVz/u2uL5t8PxOdKxzynAzdi/5Uci0t2xzt2AAN+JyBYRuf4Mvlv5iQYC5TNjzB7soPEY4L0SxUeAfGyj7tQB2O94fxDbILqXOe3DXhGEG2NaOF7NjDE9K6qTo3+6C3CfiBxy9NkPAK5yDOLuAzqUMaC7D+hcxq5zsF05Tq1LlJdM2/v/gG7AAGNMM2Cws4qO7wl1NPTevIbtHroS+MYYs7+M9cr7jop4/P5FpBG2a6eyDuD5Nwa3v7Mx5hNjzMXYq6BtwEuO5YeMMTcZY9pirxSfcx+DUdVLA4GqrBuAixxnfy7GmELgLeAxEWkqIh2BP1E8jvAWMEtE2olICHCv27YHsf3H/xSRZiJST0Q6i8gQH+pzHbabKhbbVZIAxGEb8UuA77CN4ONibzENEpELHNu+DNwpIv3EinHUG2zf9tViB7lHY8c8ytMU22d/zNGP/pcSx/cxtvELcQwID3bbdhnQF7id0ldaPn2HD94BLhWRC0WkATCbiv//Bzh+X85XA2AF0FVErhaR+iIyBfu7/1BEWonIOMdYwSkgG9tNh4hc6TawfRQbSAsrUX/lRxoIVKUYY1KNMUllFN8GnAR2AuuAN4BXHWUvYfvkNwHfU/qK4lps19JWbEPxDvasskwiEgRMBv7lOON0vnZhu7GucwSoy7CD0HuBNGz3BcaYt4HHHPU8gW2QQx27v92x3THsXUjLyqsL8DT2dtoj2IH1lSXKr8FeMW0DfgHucBYYY3KBd4FoL7+XynxHmYwxW4Bbscd6EPs7Tqtgs3uxgcf5+sIYk4Ed9P9/2K6lu4FLjTFHsO3J/8NeNWRig+ctjn2dB/xPRLKB5dixmV2+1l/5lxijD6ZRqrqJyENAV2PMbytcWamzTCfCKFXNHN08N2CvGpSqcto1pFQ1EpGbsIPJHxtj1lR3fVTdpF1DSilVx+kVgVJK1XHn3BhBeHi4iYqKqu5qKKXUOWXDhg1HjDER3srOuUAQFRVFUlJZdy8qpZTyRkRKzgh3OecCgfKjggLIyIAjRyA9HXJyoEMH6NQJGjWqeHul1DlJA0FtdvJkcaOenl78vqxlmZll76ttW4iJgc6dS/9sUVbmBKXUucCvgcAxNf8ZbGbKl40xj5cofwoY5vjYCGhpjNFWxZuiIttQV9SYuy/LzfW+r/r1ISICwsPtz4QE+9N9WXg4BAfDnj2QkgKpqfbnypVw8KDn/sLCvAeImBho2RLEl1Q4Sqnq4rdA4MiIOA+4GDuVfb2ILDfGbHWuY4z5o9v6t2Hz0NcNeXmVO1vPyLDBwJumTYsb8NatIT7es1Ev2cA3b+5743z++aWXnTwJO3d6BojUVPj6a1i82LOeTZrYoOAtULRrBwGVefaMUsof/HlF0B9IMcbsBBCRxcB4bC4Zb66ickm0ag5j4Nixyp2tZ2d731e9evYM29mAx8aW36iHh0NQUNUeb+PGNtjEx5cuO30adu/2DBApKbB1K3z4oS13atAAoqO9X0lERdlypZTf+TMQROKZfz4Nmx64FEfGx2jsw0u8lc/APgWKDh06eFvl7Dp92p6B+3q2fuSIHWj1JjjYswHv1q3sRj0iAkJCbDA4VzVoAF272ldJhYWwf3/pK4mUFPjyS8/gWK+eHaj2diXRubMNRkqps8KfgcBb30NZ05inAu84MkWW3siY+dinG5GYmHhmU6GPHIGnn4bt221Df+qU7Z4pLLSNb3q67Q/PyrJ3y5SlWTMIDbXbtG4NvXpBx462EW/cuLh7pnVraNVK77ZxFxBgG/cOHeCiizzLjIFffikdIFJS4J13bGB217p12YPXoaEopXznz0CQhueDSNph09N6MxWbItd/Xn4ZHnus9PL69aFnT9uA5+eXDgLdusG779ryiy+GH3+E48dt9wfAsGHwuuOBUjExtgFzqlcPJkywDRnYoHH0KAQGFr/GjYO//c2Wjx5t+9fdy0ePhhtusMtnzvQsCwyEwYNhxAgb2ObPL13eu7ftXsrLg7VrS5e3b2+vRvLz4fBhz7KgIPuzKojYwNmqFQwaVLr82LHSQSI1FT77DF57zXPdkJCyB69bt9bBa6VK8GcgWA90EZFo7NOLpmIfc+hBRLphH2j9jR/rAhMn2j5tZ796ixa2oSsqsoOtAIcO2YHQ/Hz7On3artPT8aCs556zDbmzPD/fBgin+++3Z67u5d27F5ePGGEbNPfyVq2Ky52ByL08Ls6WFRTA2297luXnw7332v1mZ8OsWaWP+7HHbCA4fBhGjixd/swzdrvt2733+b/yClx/PWzbBn/6U/HvLyzM/hw92l4R5ebaABkWZoPr2daiBfTrZ18l5eTYweuSgeK77+zvrNDtQrNRo7IHr9u390/dVbGiInvScuqU/f8VElJ1JxuqTH5NOiciY7AP0wgAXjXGPCYis4EkY8xyxzoPYx8Cfm/ZeyqWmJhodGaxgzH2Va+e/Q/mHqROn7Y/nQPPeXmQlFQ6kMTH20YwM9Ne+Ti3y8+324wfb69kvv8efv/74jERZ3/+hx/C2LH252WX2WUtWhQHjOeft7en/vgjfPxx6UDSqZN/B4Xz80vfAuv8uXOnbZCcAgPtILW3K4noaGjY0H/1rArGFP9d8/KKu0dLviq7vDLbuN8s4BQSYm8zbtXK86e3902b6hXdGRKRDcaYRK9l51r2UQ0ENURenr36CQmxZ9m7dsGKFcW3ujoDxty59qpo/nwbSErassVesbz8MvzjH8WBwvm67z57u+vOnXYMwbm8MrfAlqWoyA5ee+tySkmBEyeK1xWxVwxlDV47ryrLU1j46xvdX9tI/9r/7yL2BoigIBsYg4K8v3wpq1/fnrwcPmz/tr/8Uvy+rMmNDRuWHyjcf4aH6xWeGw0EqmbIySkOEM7X+PF2kP2DD2DRIs9A4rxDq3lz2wX2978X7ysgwF5V7N1rG4eFC+GbbzyDSESE7boCeyYaGOh78DDG1sFbgEhNtfVy17Kl7SIrKiq7MS7rzrLKcDaiZ9II/9qG29l4V8UZeX6+/R27Bwf39yWX5eeX3oeI/TdSVqAo+b5JE/8fVzXSQKDOTc5/myL2imD7ds8gkpUFzz5r17nnHhsMMjKKxwRCQorPLCdPhvff9+yW6toVXnjBln/4oR3jKHlFUtZdX8ePlw4Q+/bZhvJsNcIllzdocG7fWuwvxth/C+UFCvf3WVne99OoUcVdU86foaHn3GRIDQSq7igqso30kSO2a6ePY7L6W2/ZcQ73QBISYq9EAC68EL76ynNfffvChg32/TXX2EbEPZDEx8MVV9jy7dvtGWVYWNVP8FOVc+pUcWDwJXgUermrvV49+2+gvK4p9/fBwVV/nCWUFwi0A03VLvXq2cHqkonwJk+2r7IsX27/07uPb7h3FQQF2cCya5ctO3bMDo47A8GwYcU5mJo0sY3E5MnF3Vn33We7wNyvNjp1snMqVNVq2NCO97RvX/G6zpswKuqi+u47+76sjAFNmvgeNKphUqleESh1Jpx33zgHid9/395+7B5I+ve3cz+KiuyVwrFjnvuYNcvevpuXZxuBkt1SkybZYJOXZ++4cl6JhIfbrgkdCK15cnK8X214Cx5HjnjPH+ZMCuktUIwebe/iOwN6RaDU2eacdOc0fnzZ69arZ88qnalLnIHCOYekoACmTy8uO3zY3k3l7NY6cKD4ysPds8/CrbfayY233VY6kFx0kb3t9dQpe6baosU51699zmnUyN6C7MtTFAsL7d+8oi6qHTvsz9xcewJwhoGgPBoIlKoqDRpAmzb25a5JE3tlUJbISPjhh9J3XJ13ni3Pzra3wW7aZO+0ycuzy996ywaCdevspEMR25A4A8WTT9qrlm3b7GB5yUDSoYMm/vOngIDis/2KGGMnu/qpy0gDgVI1XcOGdlJeWeLi7EC4k/M23ZAQ+zkmxgYa9yCSkVHcyK9fD3fdVXq/69dDYiK88QY8+mjpQOGcab5nT/FAeni4TvryBxG/3t6qYwRK1XXG2KuKklccl11mu5NWrrQT/kqW79tnr25mz4a/uGWQDwy04xnbttk5IIsX2+yyJQPJxRfbM1xjNHBUAR0jUEqVTcSexTdtaruSSho9unhinpP7CeQ119gcUCUDhXMgfevW4gyyzu0aNCjuwrrhBnvXljNrb+vWto/90Udt+ebNdrvWrW2AqSNzKYwx/HLyFzb/spnNv2ymbdO2XNnzSr98l14RKKWqRmFh8QOcsrLs+ATYrqevvrJ3XTlfjRvb/FRgkyV+9pl9HxBgB9kHDrS5sQBeeskOpLoHktatbcr4c8TxU8fZ8ssWOjTvQGSzSNbuWcsVb13BkZwjrnUm9pjIO5PfOePv0AllSqlz1/ff25nl7oEiLKx4jkZcnL3Lyt1FF8F//2vfT55sb/d1DxK9ehU/hvXUqSpPKJiZm8mcr+bw0y8/sfmXzezN2gvAvy75FzP7z2Rv1l5mfzmbuJZxxLWMI75lPC0bt0R+RReaBgKlVO1VVGRTibgHihYt4NJLbfnll9s0IIcO2asRgGuvtc+xMMZefTRoYANEmzb25/jxMHWqLf/kk+IAEhHh0y24xhiSM5Nd3TrOBn9ij4k8etGjZJ/OJnxOOF3Duno09v0j+9OqSasK938mdIxAKVV7OdM9hIcXP7/D3bJlxe/z8+39+U6FhfDAAzZIHD5sf27YUPwMkqNH4ZJLPL8rIgL+/Ge49VbM0aNk//UB0hoXkhJ4Atq04bLBN0L79gx4eQDH8o4hCJ1DOxPXMo4e4T0AaNKgCdn3Z1O/Xs1ogvWKQCmlynL6tH2Ox6FDZO9LJXtvCq2zgcsv57pTi/l5zXusnZtNw5LpiObPZ8XQSDruO0H3e54goE1bz66pMWPsgLgzK20VPINbrwiUUqqSPk39lJUpK13dOoeyD9GyTUsO33kYgM5frqfx8Gt4dWocCUFR9CgKpcWxPHtVcd55jOncGXJ/hLBwmy59/Xp7NVJUZJ/dERVlu53Gj7dzBNwDxSOP2Od47N5t75pq3drmRmrln24jDQRKqTrpdOFpdmTssH34h39ic/pmtqZv5ac//ERQ/SA+Tf2UF5JeIDYillGdRxHfMp64lnEYYxARHhryUMVf0quXnYfhVFhoxymcdzR17w7/93+eXVObNxfnIPr4Y7jlFvt+0iT76FU/0K4hpVStVmSK2H1st6vBv6HvDbRu0ponv3mS//fp/wMgQAJcA7fPjnmWlo1bcvL0SYLqBxFQrxrzM2VmQnKyDRDh4XDBBWe8K+0aUkrVesYYDp88TFD9IFoEtWD9/vXcuuJWtqRvISc/x7Ve/8j+tG7Smku7Xkqrxq2IbxVPt7BuNKzveQtp4wb+77evUGgoDBjg96/RQKCUOicdP3WcN39609WHv/mXzWTkZvDipS8yo98Mmgc1p1nDZtzU9yZXt05sRCxNG9oZz13DutI1rGs1H0XNoIFAKVVj5ebn8vORnz368Ud1HsUdA++gsKiQmz+6maYNmhLXMo6JPSYS1zKOIR2HALah//zaz6v5CH6dvII8dh/bTWpmKjuP7mRo1FDiW8Wf9e/xayAQkdHAM0AA8LIx5nEv60wGHgYMsMkYc7U/66SUqnkKigpIzrATsBoENGB89/EYY2j9z9YcP3UcgAYBDYiNiEWws2tDgkPYe8de2jVr96tm3FYnYwwZuRmuhj71qOfP/cf3Yygex3161NPnViAQkQBgHnAxkAasF5Hlxpitbut0Ae4DLjDGHBURHxJzK6XOVSdPn+TwycN0CukEwD2f3cMnqZ/w85GfOV14GoABkQMY3308IsKcEXMIaxRGXMs4YkJjSk3Aat/ch8dNVrP8wnz2Zu312tCnZqZy4vQJj/XbNm1Lp5BODI8eTqeQTnQO6Wx/hnYmolGEX+rozyuC/kCKMWYngIgsBsYDW93WuQmYZ4w5CmCM+aXUXpRS55Sc/BwaBTYC4K0tb/FJyickZyaTkpnCweyDRDaNJO1PaYDt52/TtA0jO490pVnoHt7dta/fJ/6+Wo6hsrLysko18DuP2Z97s/ZSaIpnnDUMaEh0SDSdQzrzmw6/8Wjoo1pEuX53VcmfgSAS2Of2OQ0oOfzdFUBEvsJ2Hz1sjFlZYh1EZAYwA6CDPuxbqRpj/f71fJr6KSlHU0jJtK8jOUfIuT+HwIBA1u5Zy4qUFcSExjAqZhRdQrvQJbSLa/vnL32+GmvvuyJTxP7j+7029DuP7iQjN8Nj/fBG4XQO6czAdgOZFj/N1dB3CulE26ZtqSc1K5W2PwOBt067kpMW6gNdgKFAO2CtiMQZYzye8m2MmQ/MBzuP4OxXVSnlzf7j+1m7dy3JGcmuxj45I5mkGUl0aN6BL3Z9wYOrHqRt07bEhMYwJmYMXcK6cLrwNIEBgTw9+mn+NeZf1X0YPsnJz2Hn0Z3FDb3bGf6uY7tcXVdg5x10bNGRziGdmRQ7yeOsvlNIJ5o1PHdSYIN/A0Ea4N6B1w444GWdb40x+cAuEdmODQzr/VgvpZRDTn4Om3/ZbBv6zBRXYz939FzOizyPL/d8ybT3pgHQrlk7YkJjuLz75a4B21vOu4WZ/WeWec99tU7GKsE5z8BbQ596NJVD2Yc81m/WsBmdQ2yyuPHdxns09B2ad6gxCePOBn8eyXqgi4hEA/uBqUDJO4KWAVcBC0UkHNtVtNOPdVKqTnHeleLstknJTCE5M5nrE65neKfhbDiwgcELBwMgCO2btycmNMbVpz06ZjQ//eEnOod0JjgwuNT+nffk1xSnCk6xJ2tPmXfhuE8sE4R2zdrRKaQTY2LGeDT0nUM6Exoces7ejVRZfgsExpgCEZkJfILt/3/VGLNFRGYDScaY5Y6ykSKyFSgE7jLGZJS9V6VUScYY0nPSixv6jGQGthvI2K5j2ZO1h+hnih8/KQgdW3RkTMwYAHq37s37U9+nS2gXokOiCaof5LHv0OBQQoNDq/R4ymOMITM30+vA7M6jO9mXtc/jdsvg+sGuBn5E9AiPhr5ji46ljreu0lxDSp0DnN0azsY+olEEY7uOpaCogJZPtORo3lHXuvWkHvdecC+PDX+MwqJC5v5vLl3CuhATGkN0i+hSqRRqmoKiAvZl7StzYDbrVJbH+q0at/Jo4N1/tm7Sus6c1VdEcw0pdQ4wxnAw+yApmSnkF+YzvNNwAEb9ZxRf7/ua7NPZrnUv7XopY7uOpX69+szsP5PwRuHEhMYQExpDVIsoGgQ0AGwf/R/P/2O1HE9ZjDEcyTlC2vE0dh3bVaobZ0/WHgqKClzrB9YLJDokmk4hnRjUfpBHQ98ppFPNyAl0jtNAoFQVKjJFHDhxgMPZh+nXth8A931+HytSVpCSmeLqw+7Vqhebbt4EQO9WvekR3sPV0MeExtCxeUfXPmcPm131B1KGnPwc9h/fz4ETB9h/Yr/n+xP2/YETBzzuwAEICQqhc2hn+rXtx+Sek12NfefQzkQ2jaxRg861kQYCpc6yIlPEwRMHiWwWCcCiTYt4b9t7pGSmkJqZSm5BLiFBIWTekwlAoSmkQ/MOXBR1kasLx/1e+zkXz6mW43BXWFTI4ZOHbaN+vLhRdzb2zs/H8o6V2rZxYGMim0XStmlbLmh/AZFN7fvIZpFEt7Bn+iHBIdVwVMpJA4FSZ6DIFCEIIsJXe79i6balrtmzqZmpnCo8RfZ92TRu0JgdGTvYkbGDmNAYRnYa6WrsnQ84qc6G3hjD8VPHixt2L438gRMHOJR9yGN2LNh76Vs3aU1ks0i6hnVlWNQwj0be+b5Zw2baT1/DaSBQqgL7svbx+c7PSc5MZkfGDleD//2M7+kW3o3vD37Ps989S0xoDF3DujImZgwxoTGu7R+56BEeueiRKq/36cLTHDxxsMxuGufnk/knS23bIqgFkU0jiWwWSc+WPe37Eo18y8YttcumltBAoOq83PxcfvrlJ5Izihv65Mxk/jnynwzuOJgNBzdw/fLrqV+vPp1COtEltAvDo4e7bj2c0W8Gt/a/tcrSBjjnBrifwXvri//lZOnUXQ0CGtjGvGkkfdr0YWyXsR5n784unOrId6OqjwYCVSdkn84mOcM28M6fk3tOZkyXMWxN38qAl20aLOd99l3Durpmzw6PHk7ybclEtYjyOpv0bN6OmZufW2E3zYETBzhVeKrUti0bt3Q18v3b9i/VRRPZLJKw4DDtplGlaCBQtUZeQR6pmamuLpyeET0Z23Us6SfTafkPzwznkU0juaC9ff5rj4geLJuyjK5hXekU0qlUw960YdNfPYO2sKiQX07+4nWA1b2Rd58P4OQ+2Dqo/SCv/fBtmrZx3TKqVGVpIFDnlPzCfHYf201yZjINAxoyvNNwjDH0mNeDHRk7PGaV3tzvZsZ2HUt4o3D+PuLvrm6dmNAYj3vPGwU2Ynz38WdcJ2MMB04cYHvGdtKOp3ntpjl44mC5g61dwrowNGqoDraqaqGBQNU4hUWF7Du+j6O5R+nTpg8AMz6Ywerdq9l1bJdrstHQqKEM7zQcEWFct3E0DmxMlzCb5rhLWBdaBLUAQES4+4K7f3W9ikwRe7P2sjV9q+v185Gf2Zq+1fUULSf3wdbYiFgdbFU1mgYCVS2cs0sjGtsnLr2Y9CIfp3xMcmay6/bLziGdSZmVAthBzoTWCVwZeyVdwrqUevD42bwFs7CokJ1Hd5Zq7H8+8rNH0rLWTVoTGxHLtb2upUdED7qHd6dD8w462KrOORoIlN8481iJCGv3rGVlykrXHTnJGckUmkJO3n+SelLP3rWTmUyX0C6MiRlD17CudAvv5trXs2OePev1O114mpTMFH5Otw391iO24d9+ZLvHYGz7Zu2JjYhlRt8ZxEbEEhsRS4+IHjUqGZtSv4YGAnVW7M3ay7q96+wtmJk7im/FvC2ZiMYRfLHrC/7+1d+JDomma1hXhnQcQpfQLhQUFdAgoIFfGnqnvII8th/Z7jqzd76SM5Nd3UyCENUiitiIWEZ1HuVq8LuHdz/nHjKiVGVpIFA+cTam7pOqdmTs4IWxLxDfKp7PUj/jxg9uRBA6NO9A17CuTIuf5hogvXPQndz/m/sJDAj0Wx1Pnj7JtiPbiht7xxn+zqM7KTJFgM3MGRMaQ2xELBO6T3A1+N3Cu2l3jqqz6lQgeP3H10nOTPZYFt4onJn9ZwLwyvevsCdrj0d526ZtuTnxZgDmfTePg9kHPco7hXTi+j7XA/DPr/9Z6tmlPcJ7cE3vawB4bM1jnDh9AvfU333a9GFq3FQAHvziQU4XnvYoH9huIBNjJ1Jkirj7Mzvg6V4+NGool3W7jNz8XB744oFS5aNjRjMqZhTH8o7x8OqHS5Vf3v1yhkUP43D2YR5b+5hH3XPzc/ldn98xqP0g1uxZw6j/jPL4vXQJ7UJeQR4A47uPZ2C7gXQO7ew1x/vZzBCZlZdV6ux+a/pWj79dYL1AuoZ1pU/rPkyLn+Zq8LuEdqnxaZiVqmp1KhC8sfkNViSv8FjWLaybKxD8+8d/s3bPWo/yxLaJrkCwYOMCNh7a6FE+NGqoKxC8/CJ9bGMAACAASURBVMPLpGamepRf2vVSVyB4ccOLrtmeztsBp8ZNdQWCF5JecA1GOstPFZ5iYuxEV7mTszw4MJjLul1GflE+L3//cqny1k1aMypmFDn5OSzcuLBUebfwbgyLHkbWqSz+8+N/POreIKABQ6OGMqj9IBLbJrJk0hK6hnUlJjSGJg2aeKwb3iic8EbhnE0ZORmejb3jDP/AieInngbVD6J7eHcu6HABN4XfRI+IHsRGxNI5pLNfrz6Uqk3q1INpnEm+VM3hfOBKybP7relbSc9Jd63XOLCxa5A2NjzWdYYf1SJKb8FUygf6YBoHDQLVxxhD2vG0Umf4P6f/7DGbtnnD5sRGxDKu2zhXYx8bEUu7Zu2qLJePUnVNnQoEyv+KTBG7j+0udQ/+1vStHk/YCm8UTmxELFN6TvFo8PXRgkpVPQ0E6owUFBWQmplaqsHfdmQbuQW5rvXaNGlDbEQsv0v4XfE9+OE9XBPJlFLVr84EgpOnT3Iy/6TrYSLOn0CpZeX9BMpdp7Y5VXCK5Mxk29in/+wasN2RscPjcYMdmncgNiKWYVHDXAO2PcJ76JOnlDoHVBgIRKQzkGaMOSUiQ4FewL+NMaWfSVd629HAM0AA8LIx5vES5dOBJ4D9jkXPGmNexg/mrZ/HPZ/f449dl3ImQaYy6/oSkM7GunkFeew8utM1F0AQOoV0IjYiljExYzwmXf3a7JxKqerjyxXBu0CiiMQArwDLgTeAMeVtJCIBwDzgYiANWC8iy40xW0ususQYM7PSNa+kiztdTONLGmMwGGNcWSqd7yv6WZl1nXdiVcu6Z7HOgfUCmdxzcvGkq7BuBAcG+/tPpZSqYr4EgiJjTIGITACeNsb8S0R+8GG7/kCKMWYngIgsBsYDJQNBlejTpo8rk6VSSqlivtyPly8iVwHXAR86lvkyUycS2Of2Oc2xrKSJIvKjiLwjIu297UhEZohIkogkpaene1tFKaXUGfIlEPwOOB94zBizS0Sigf9UsA2At5HTkrPXPgCijDG9gM+B17ztyBgz3xiTaIxJjIjQu02UUupsqrBryNGnPwtAREKApiUHfcuQBrif4bcDDrivYIxxT8zzEvB3H/arlFLqLKrwikBEVotIMxEJBTYBC0TkSR/2vR7oIiLRItIAmIodaHbfdxu3j+OAn32vulJKqbPBl66h5saY48AVwAJjTD9gREUbGWMKgJnAJ9gG/i1jzBYRmS0i4xyrzRKRLSKyCXvVMf1MDkIppdSZ8+WuofqOM/fJwAOV2bkxZgWwosSyh9ze3wfcV5l9KqWUOrt8uSKYjT2rTzXGrBeRTkByBdsopZQ6R/gyWPw28Lbb553ARH9WSimlVNXxZbC4nYgsFZFfROSwiLwrIu2qonJKKaX8z5euoQXYu33aYieEfeBYppRSqhbwJRBEGGMWGGMKHK+FgM7qUkqpWsKXQHBERH4rIgGO12+BjAq3UkopdU7wJRBcj7119BBwEJiETTuhlFKqFqgwEBhj9hpjxhljIowxLY0xl2MnlymllKoFzvRp4H86q7VQSilVbc40ENS+ZzIqpVQddaaBoGQ6aaWUUueoMmcWi8gJvDf4AujzCpVSqpYoMxAYY/Rp5EopVQecadeQUkqpWkIDgVJK1XEaCJRSqo7TQKCUUnXcmdw1BIAxpplfaqSUUqpKVXjXkIjMxuYZWoS9dXQaoHcUKaVULeFL19AoY8xzxpgTxpjjxpjn0SeUKaVUreFLICgUkWmOFNT1RGQaUOjviimllKoavgSCq7FpqA87Xlc6llVIREaLyHYRSRGRe8tZb5KIGBFJ9GW/Simlzh5fHl6/Gxhf2R2LSAAwD7gYSAPWi8hyY8zWEus1BWYB/6vsdyillPr1KgwEIhIB3AREua9vjLm+gk37AynGmJ2O/SzGBpStJdZ7BJgD3OlzrZVSSp01FQYC4H1gLfA5lRsbiAT2uX1OAwa4ryAifYD2xpgPRaTMQCAiM4AZAB06dKhEFZRSSlXEl0DQyBhzzxns29szC1zzEkSkHvAUML2iHRlj5gPzARITEzUFtlJKnUW+DBZ/KCJjzmDfaUB7t8/tgANun5sCccBqEdkNDASW64CxUkpVLV8Cwe3YYJArIsdF5ISIHPdhu/VAFxGJFpEGwFRgubPQGJNljAk3xkQZY6KAb4FxxpikMzgOpZRSZ8iXu4bOaBaxMaZARGYCnwABwKvGmC2OmcpJxpjl5e9BKaVUVfBljAARCQG6AEHOZcaYNRVtZ4xZAawoseyhMtYd6ktdlFJKnV2+3D56I7Z7qB2wEduX/w1wkX+rppRSqir4OkZwHrDHGDMM6AOk+7VWSimlqowvgSDPGJMHICINjTHbgG7+rZZSSqmq4ssYQZqItACWAZ+JyFE8bwNVSil1DvPlrqEJjrcPi8gqoDmw0q+1UkopVWV8umvIyRjzpb8qopRSqnroM4uVUqqO00CglFJ1XIWBQERmOiaUKaWUqoV8uSJojX2ozFuOJ455yyqqlFLqHFVhIDDGPIhNL/EKNmV0soj8TUQ6+7luSimlqoBPYwTGGAMccrwKgBDgHRGZ48e6KaWUqgK+5BqaBVwHHAFeBu4yxuQ7HiyTDNzt3yoqpZTyJ1/mEYQDVxhj9rgvNMYUicil/qmWUkqpquJL19AKINP5QUSaisgAAGPMz/6qmFJKqarhSyB4Hsh2+3zSsUwppVQt4EsgEMdgMWC7hKhkagqllFI1ly+BYKeIzBKRQMfrdmCnvyumlFKqavgSCG4GBgH7gTRgADDDn5VSSilVdXxJQ/0LMLUK6qKUUqoa+DKPIAi4AeiJ58Prr/djvZRSSlURX7qGFmHzDY0CvsQ+xP6ELzt35CbaLiIpInKvl/KbReQnEdkoIutEJLYylVdKKfXr+RIIYowxfwZOGmNeA8YC8RVtJCIBwDzgEiAWuMpLQ/+GMSbeGJMAzAGerFTtlVJK/Wq+BIJ8x89jIhKHfVRllA/b9QdSjDE7jTGngcXAePcVjDHH3T42BgxKKaWqlC/zAeY7nkfwILAcaAL82YftIoF9bp+ddxx5EJFbgT8BDYCLfNivUkqps6jcKwJHYrnjxpijxpg1xphOxpiWxpgXfdi3t+cWlDrjN8bMM8Z0Bu7BBhtv9ZghIkkikpSenu7DVyullPJVuYHAMYt45hnuOw1o7/a5HXCgnPUXA5eXUY/5xphEY0xiRETEGVZHKaWUN76MEXwmIneKSHsRCXW+fNhuPdBFRKJFpAF2LsJy9xVEpIvbx7HYtNZKKaWqkC9jBM75Are6LTNAp/I2MsYUiMhM4BMgAHjVGLNFRGYDScaY5cBMERmBHZA+in3ugVJKqSokbvnkzgmJiYkmKSmpuquhlFLnFBHZYIxJ9Fbmy8zia70tN8b8+9dWTCmlVPXzpWvoPLf3QcBw4HtAA4FSStUCviSdu839s4g0x6adUEopVQv4ctdQSTlAlwrXUkopdU7wZYzgA4ongtXD5g16y5+VUkopVXV8GSP4h9v7AmCPMSbNT/VRSilVxXwJBHuBg8aYPAARCRaRKGPMbr/WTCmlVJXwZYzgbaDI7XOhY5lSSqlawJdAUN+RRhoAx/sG/quSUkqpquRLIEgXkXHODyIyHjjivyoppZSqSr6MEdwMvC4izzo+pwFeZxsrpZQ69/gyoSwVGCgiTbC5iXx6XrFSSqlzQ4VdQyLyNxFpYYzJNsacEJEQEXm0KiqnlFLK/3wZI7jEGHPM+cEYcxQY478qKaWUqkq+BIIAEWno/CAiwUDDctZXSil1DvFlsPg/wH9FZAE21cT1aOZRpZSqNXwZLJ4jIj8CI7APpH/EGPOJ32umlFKqSvhyRYAxZiWwEkBELhCRecaYWyvYrMrk5+eTlpZGXl5edVdFqQoFBQXRrl07AgMDq7sqSgE+BgIRSQCuAqYAu4D3/FmpykpLS6Np06ZERUUhItVdHaXKZIwhIyODtLQ0oqOjq7s6SgHlBAIR6QpMxQaADGAJdh7BsCqqm8/y8vI0CKhzgogQFhZGenp6dVdFKZfyrgi2AWuBy4wxKQAi8scqqdUZ0CCgzhX6b1XVNOXdPjoROASsEpGXRGQ4drDYZyIyWkS2i0iKiNzrpfxPIrJVRH4Ukf+KSMfKVV8ppdSvVWYgMMYsNcZMAboDq4E/Aq1E5HkRGVnRjkUkAJgHXIJ9qtlVIhJbYrUfgERjTC/gHWDOGR1FDbF06VJEhG3btlV3Vc6quXPn0qNHD6ZNm+axfOPGjaxYsaLS+ztw4ACTJk2qcL0xY8Zw7NixCterrOnTp/POO++Uu87ChQs5cODAWf9upWqiCieUGWNOGmNeN8ZcCrQDNgKlzu696A+kGGN2OlJXLwbGl9j3KmNMjuPjt479n7PefPNNLrzwQhYvXuzX7yksLPTr/kt67rnnWLFiBa+//rrH8vICQUFBQZn7a9u2bYUNMcCKFSto0aJF5Sp7lmggUHWJT3cNORljMoEXHa+KRAL73D6nAQPKWf8G4OPK1KcsQxcOLbVscs/J3HLeLeTk5zDm9dIZMqYnTGd6wnSO5Bxh0lueZ6urp6+u8Duzs7P56quvWLVqFePGjePhhx92lc2ZM4dFixZRr149LrnkEh5//HFSUlK4+eabSU9PJyAggLfffpt9+/bxj3/8gw8//BCAmTNnkpiYyPTp04mKiuL666/n008/ZebMmZw4cYL58+dz+vRpYmJiWLRoEY0aNeLw4cPcfPPN7Ny5E4Dnn3+ejz/+mPDwcG6//XYAHnjgAVq1asWsWbM8juHJJ5/k1VdfBeDGG2/kjjvucO1r3LhxXH/99fzxj3aY6PTp0zz00EPk5uaybt067rvvPn7++WcOHDjA7t27CQ8P529/+xvXXHMNJ0+eBODZZ59l0KBB7N69m0svvZTNmzezcOFCli9fTk5ODqmpqUyYMIE5c+yFYVRUFElJSWRnZ3PJJZdw4YUX8vXXXxMZGcn7779PcHAw69ev54YbbqBx48ZceOGFfPzxx2zevNnjuIwx3HbbbXzxxRdER0djjHGVzZ49mw8++IDc3FwGDRrEiy++yLvvvktSUhLTpk0jODiYb775hieeeKLUetrXr2oLX1JMnClv/0uMl2WIyG+BROCJMspniEiSiCTV1Lstli1bxujRo+natSuhoaF8//33AHz88ccsW7aM//3vf2zatIm7774bgGnTpnHrrbeyadMmvv76a9q0aVPhdwQFBbFu3TqmTp3KFVdcwfr169m0aRM9evTglVdeAWDWrFkMGTKETZs28f3339OzZ09uuOEGXnvtNQCKiopYvHhxqW6eDRs2sGDBAv73v//x7bff8tJLL/HDDz/wwgsv0LZtW1atWuUKAgANGjRg9uzZTJkyhY0bNzJlyhTXft5//33eeOMNWrZsyWeffcb333/PkiVLSgUep40bN7JkyRJ++uknlixZwr59+0qtk5yczK233sqWLVto0aIF7777LgC/+93veOGFF/jmm28ICAjwuv+lS5eyfft2fvrpJ1566SW+/vprV9nMmTNZv349mzdvJjc3lw8//JBJkyaRmJjI66+/zsaNGwkODva6nlK1RaWuCCopDWjv9rkdUOpaW0RGAA8AQ4wxp7ztyBgzH5gPkJiY6DWYuCvvDL5RYKNyy8Mbhft0BVDSm2++yR133AHA1KlTefPNN+nbty+ff/45v/vd72jUqBEAoaGhnDhxgv379zNhwgTANvC+cDa2AJs3b+bBBx/k2LFjZGdnM2rUKAC++OIL/v1vmwEkICCA5s2b07x5c8LCwvjhhx84fPgwffr0ISwszGPf69atY8KECTRu3BiAK664grVr19KnT59K/R7GjRtHcHAwYCf6zZw5k40bNxIQEMCOHTu8bjN8+HCaN28OQGxsLHv27KF9+/Ye60RHR5OQkABAv3792L17N8eOHePEiRMMGjQIgKuvvtprA71mzRquuuoqAgICaNu2LRdddJGrbNWqVcyZM4ecnBwyMzPp2bMnl112Wal9+LqeUucifwaC9UAXEYkG9mPnJFztvoKI9MF2M402xvzix7r4VUZGBl988QWbN29GRCgsLEREmDNnDsaYUl0I7l0T7urXr09RUfHjoUvOlHY20mAHPJctW0bv3r1ZuHAhq1evLreON954IwsXLuTQoUNcf/31pcrLqlNludfxqaeeolWrVmzatImioqIyA17DhsU5DAMCAryOL5RcJzc3t1J19taNk5eXxy233EJSUhLt27fn4Ycf9jo73df1lDpX+a1ryBhTAMwEPgF+Bt4yxmwRkdluj758AmgCvC0iG0Vkub/q40/vvPMO1157LXv27GH37t3s27eP6Oho1q1bx8iRI3n11VfJybFj4pmZmTRr1ox27dqxbNkyAE6dOkVOTg4dO3Zk69atnDp1iqysLP773/+W+Z0nTpygTZs25OfnewziDh8+nOeffx6wg8rHjx8HYMKECaxcuZL169e7rh7cDR48mGXLlpGTk8PJkydZunQpv/nNb8o97qZNm3LiRNnPKcrKyqJNmzbUq1ePRYsWnfVB7pCQEJo2bcq3334LUOYg/eDBg1m8eDGFhYUcPHiQVatWAcWBNjw8nOzsbI8BbPdjK289pWoDf44RYIxZYYzpaozpbIx5zLHsIWPMcsf7EcaYVsaYBMdrXPl7rJnefPNNVzeP08SJE3njjTcYPXo048aNIzExkYSEBP7xj38AsGjRIubOnUuvXr0YNGgQhw4don379kyePJlevXoxbdq0crtlHnnkEQYMGMDFF19M9+7dXcufeeYZVq1aRXx8PP369WPLli2A7dMfNmwYkydP9tqX3rdvX6ZPn07//v0ZMGAAN954Y4XdQsOGDWPr1q0kJCSwZMmSUuW33HILr732GgMHDmTHjh0eVwtnyyuvvMKMGTM4//zzMca4upjcTZgwgS5duhAfH88f/vAHhgwZAkCLFi246aabiI+P5/LLL+e8885zbTN9+nRuvvlmEhISaNiwYZnrKVUbyNnqEqgqiYmJJikpyWPZzz//TI8ePaqpRueGoqIi+vbty9tvv02XLl2quzpnTXZ2Nk2aNAHg8ccf5+DBgzzzzDPVXKuK6b9ZVdVEZIMxJtFbmV+vCFTNsHXrVmJiYhg+fHitCgIAH330EQkJCcTFxbF27VoefPDB6q6SUuccfw4WqxoiNjbWNa+gtpkyZYrH3VRKqcrTKwKllKrjNBAopVQdp4FAKaXqOA0ESilVx2kgOIvqWhrqylq9ejWXXnopAMuXL+fxxx/3up7zdtCyHDt2jOeee8712de01pXlXt+ynGkqbqVqEg0EZ1FdS0P9a4wbN4577/Ulm3lpJQOBr2mt/UEDgaoNat3to3esvIONhzae1X0mtE7g6dFPl7tOXUtDDTBgwABeffVVevbsCcDQoUP55z//SWFhIXfccQe5ubkEBwezYMECunXr5vFdCxcuJCkpiWeffZZdu3Zx9dVXU1BQwOjRoz1+p+PHj+fo0aPk5+fz6KOPMn78eO69915SU1NJSEjg4osv5tZbb3Wltc7Ly+MPf/gDSUlJ1K9fnyeffJJhw4aVm+7a3cqVK7njjjsIDw+nb9++ruXfffddqWOKjo4ulYo7Ojq6wmNXqqapdYGgunhLQ923b1+PNNSNGjUiMzMTsGmo7733XiZMmEBeXh5FRUVe0y+7c6ahBpvo7qabbgLgwQcf5JVXXuG2225zpaFeunQphYWFZGdn07ZtW6644gpuv/12Vxrq7777zmPf7mmojTEMGDCAIUOG8MILL7By5UpWrVpFeHi4xzZTp07lrbfe4q9//SsHDx7kwIED9OvXj+PHj7NmzRrq16/P559/zv333+9KG+3N7bffzh/+8AeuvfZa5s2b53G8S5cupVmzZhw5coSBAwcybtw4Hn/8cTZv3szGjTbg796927WNc/uffvqJbdu2MXLkSFfW040bN/LDDz/QsGFDunXrxm233eaR5TQvL4+bbrqJL774gpiYGI/5Cd27d/d6TLNnz3YFNKDSx65UTVDrAkFFZ+7+UhfTUE+ePJmLL76Yv/71r7z11ltceeWVgE02d91115GcnIyIkJ+fX+5xffXVV67G8pprruGee+4BbEbU+++/nzVr1lCvXj3279/P4cOHy93XunXruO222wDbeHfs2NEVCCpKd71t2zaio6Nds69/+9vfMn/+/EodU2WPXamaoNYFgupQV9NQR0ZGEhYWxo8//siSJUt48UX74Lo///nPDBs2jKVLl7J7926GDh1a4b68pYl+/fXXSU9PZ8OGDQQGBhIVFVVh+ufyjsOXdNdlPXXM12M6k2NXqrrpYPFZUFfTUIO9+pkzZw5ZWVnEx8cD9qw4MjISsGMBFbngggtcA+zux5KVlUXLli0JDAxk1apV7NmzByg//fXgwYNd+9ixYwd79+71uY++e/fu7Nq1i9TUVMBe5bnXxdsxlaxLZY9dqZpAA8FZUFfTUANMmjSJxYsXM3nyZNeyu+++m/vuu48LLrjApzucnnnmGebNm8d5551HVlaWa/m0adNISkpyPTbSeZxhYWFccMEFxMXFcdddd3ns65ZbbqGwsJD4+HimTJnCwoULPa4EyhMUFMT8+fMZO3YsF154IR07dqzwmEqm4q7ssStVE2ga6jqitqahPlfpv1lV1TQNdR1Xm9NQK6V+PR0srgNqcxpqpdSvp1cESilVx2kgUEqpOk4DgVJK1XEaCJRSqo7zayAQkdEisl1EUkSkVKpJERksIt+LSIGInP08wlWootTJ/vT222/To0cPhg0b5rF89+7dvPHGG2e0z0GDBlW4zo033sjWrVvPaP/lefjhh13zLcqybNkyv3y3UnWR3wKBiAQA84BLgFjgKhGJLbHaXmA6cGatlQLglVde4bnnnmPVqlUey8sLBN7SK7j7+uuvK/zel19+mdjYkn/SqqGBQKmzx59XBP2BFGPMTmPMaWAxMN59BWPMbmPMj0CRtx2csaFDS7+c+etzcryXO9MBHDlSuuwM7dmzh+HDh9OrVy+GDx/O3r17AXsGHxcXR+/evRk8eDAAW7ZsoX///iQkJNCrVy+Sk5NL7e/NN98kPj6euLg4V2K22bNns27dOm6++eZSs2zvvfde1q5dS0JCAk899RQLFy7kyiuv5LLLLmPkyJFkZ2czfPhw+vbtS3x8PO+//75rW+cVzurVqxk6dCiTJk2ie/fuTJs2zZXPZ+jQoTgn9zVp0oQHHniA3r17M3DgQFdyuNTUVAYOHMh5553HQw89VOaV02OPPUa3bt0YMWIE27dvdy1/6aWXOO+88+jduzcTJ04kJyeHr7/+muXLl3PXXXeRkJBAamqq1/WUUj4yxvjlBUwCXnb7fA3wbBnrLgQmlbOvGUASkNShQwdT0tatWz0XDBlS+jVvni07edJ7+YIFtjw9vXSZDxo3blxq2aWXXmoWLlxojDHmlVdeMePHjzfGGBMXF2fS0tKMMcYcPXrUGGPMzJkzzX/+8x9jjDGnTp0yOTk5Hvvav3+/ad++vfnll19Mfn6+GTZsmFm6dKnjcIeY9evXl/r+VatWmbFjx7o+L1iwwERGRpqMjAxjjDH5+fkmKyvLcdjppnPnzqaoqMjjeFatWmWaNWtm9u3bZwoLC83AgQPN2rVrS30vYJYvX26MMeauu+4yjzzyiDHGmLFjx5o33njDGGPM888/7/X3lJSUZOLi4szJkydNVlaW6dy5s3niiSeMMcYcOXLEtd4DDzxg5s6da4wx5rrrrjNvv/22q6ys9WqqUv9mlfIzIMmU0cb6c0KZtzSOZ5TPwhgzH5gPNsVEhRuUl4mzUaPyy8PDyy+vhG+++Yb33nsPsOmV7777bsAmWZs+fTqTJ0/miiuuAOD888/nscceIy0tjSuuuKLUDOD169czdOhQIiIiAJuHZ82aNVx++eWVqtPFF19MaGgoUHaa59atW3ts079/f9q1awdAQkICu3fv5sILL/RYp0GDBq7HOvbr14/PPvvM9TtwJte7+uqrufPOO0vVae3atUyYMMGVqnvcuHGusrLSbZfk63pKqdL82TWUBrR3+9wOOODH76vxnCmOX3jhBR599FH27dtHQkICGRkZXH311Sxfvpzg4GBGjRrFF1984bGtOUs5odxTWbuned64cSOtWrXymubZl/TNgYGBruMra53ylJX+efr06Tz77LP89NNP/OUvfykzDbWv6ymlSvNnIFgPdBGRaBFpAEwFlvvx+2qcQYMGeaRXdp5Fp6amMmDAAGbPnk14eDj79u1j586ddOrUiVmzZjFu3Dh+/PFHj30NGDCAL7/8kiNHjlBYWMibb77JkCFDyv3+8tI1Q9lpns+mgQMHuh46U9aznAcPHszSpUvJzc3lxIkTfPDBB66ystJtlzy2stZTSlXMb4HAGFMAzAQ+AX4G3jLGbBGR2SIyDkBEzhORNOBK4EUR2eKv+vhbTk4O7dq1c72efPJJ5s6dy4IFC+jVqxeLFi3imWeeAeCuu+5yDfoOHjyY3r17s2TJEuLi4khISGDbtm1ce+21Hvtv06YN//d//8ewYcPo3bs3ffv2Zfz48d6q4tKrVy/q169P7969eeqpp0qVl5Xm+Wx6+umnefLJJ+nfvz8HDx50PSHMXd++fZkyZQoJCQlMnDjR4zkIZaXbnjp1Kk888QR9+vQhNTW1zPWUUhXTNNTKr3JycggODkZEWLx4MW+++abH3Ul1lf6bVVWtvDTUmn1U+dWGDRuYOXMmxhhatGjBq6++Wt1VUkqVoIFA+dVvfvMbNm3aVN3VUEqVQ3MNKaVUHaeBQCml6jgNBEopVcdpIFBKqTpOA8FZUhPTUFfW7t27iYuLAyApKYlZs2Z5XS8qKoojR46Uu6+//e1vHp99SWtdWe71TdqwtQAAC/NJREFULW+dM03FrVRdoYGgFigrDfWvkZiYyNy5c894+5KBwJe01v6ggUCpitW+QHDHHd7TTP+a1x13nFFVqjsN9ZQpU1ixYoXr8/Tp03n33XfZvXs3v/nNb+jbty99+/b12kivXr3alUQuIyODkSNH0qdPH37/+9975D26/PLL6devHz179mT+/PmATX+dm5tLQkIC06ZNA4qvmIwx3HXXXcTFxREfH8+SJUtc31dWumt3GzZsoHfv3px//vnMmzfPtbysYyqZituXY1eqzikrLWlNffXr169UelWPlL633+49zfSved1+e9m5XR1qYhrq9957z1x77bWufbZr187k5OSYkydPmtzcXGOMMTt27DDO3+muXbtMz549jTGeKaxvu+0289e//tUYY8yHH35oAJOenm6MMa6U1jk5OaZnz56udNAlfx/Oz++8844ZMWKEKSgoMIcOHTLt27c3Bw4cKDfdtbv4+HizevVqY4wxd955p6u+ZR1TyVTcZa1X1TQNtapqVFMa6urx9NPVXQOX6k5DfckllzBr1ixOnTrFypUrGTx4MMHBwWRlZTFz5kw2btxIQEAAO3bsKPc41qxZ4zqOsWPHEhIS4iqbO3cuS5cuBWDfvn0kJycTFhZW5r7WrVvHVVddRUBAAK1atWLIkCGsX7+eZs2aVZjuOisri2PHjrmS7V1zzTV8/PHHAOTn5/t0TL6up1RdUvsCQQ3mnob6f//7Hx999BEJCQls3LiRq6++mgEDBvDRRx8xatQoXn75ZS666CLXtuYMckIFBQUxdOhQPvnkE5YsWcJVV10FwFNPPUWrVq3YtGkTRUVFBAUF+Vx3d6tXr+bzzz/nm2++oVGjRgwdOrTC9M/lHUdF6a6NMWWmq/b1mM7k2JWq7WrfGEENUt1pqMFm6VywYAFr1651PawlKyuLNm3aUK9ePRYtWkRhYWG5+xg8eLArtfPHH3/M0aNHXfsJCQmhUaNGbNu2jW+//da1TWBgIPn5+V73tWTJEgoLC0lPT2fNmjX079+/wuMAaNGiBc2bN2fdunUAHummyzqmkumqK3vsStUFGgjOkpqYhhpg5MiRrFmzhhEjRtCgQQMAbrnlFl577TUGDhzIjh07PB5W481f/vIX1qxZQ9++ffn000/p0KEDAKNHj6agoIBevXrx5z//mYEDB7q2mTFjBr169XINFjtNmDCBXr160bt3by666CLmzJlT6olo5VmwYAG33nor559/PsHBwa7lZR1TyVTclT12peoCTUOtVDXQf7OqqpWXhlqvCJRSqo7TQKCUUnVcrQkE51oXl6q79N+qqmlqRSAICgoiIyND/4OpGs8YQ0ZGht62qmqUWjGPoF27dqSlpZGenl7dVVGqQkFBQa6Jc0rVBLUiEAQGBhIdHV3d1VBKqXOSX7uGRGS0iGwXkRQRuddLeUP5/+2da6wdVRXHf3/7AGIVAoVYaKWaFsIV6ENyraBIfEQBLR9ALEGggKkBeTQkGPQDqPFDiUaI1VCqYAoVuKQQUhoaAVGxBsujtLe31uKFmHB51SBtKVRq6/LD3geHcebeOb1nZk7PWb/kJHtmr9n7f9a9+6wz68yskfpi/1pJU8vU4ziO4/w/pQUCSWOAnwOnAz3AeZJ6UmaXAm+Y2TTgJuDGsvQ4juM42ZR5RtALDJrZC2a2G7gHSN8KexawLLZXAJ9TXjEZx3EcpxTK/I3gKODFxPYQ8Ik8GzPbI2k7cBjwnsdfSVoALIibOyVt2UdNE9NjtwmuqzlcV/O0qzbX1Ryj0XV0XkeZgSDrm336+s4iNpjZUmDpqAVJT+fdYl0nrqs5XFfztKs219UcZekqMzU0BExJbE8GXs6zkTQWOBj4Z4maHMdxnBRlBoKngOmSPiJpPDAPWJmyWQlcFNvnAI+Z3xXmOI5TKaWlhmLO/wrgN8AY4HYz2yTpB4RHpq0EbgPulDRIOBOYV5aeyKjTSyXhuprDdTVPu2pzXc1Riq79rgy14ziO01o6otaQ4ziOs+94IHAcx+lyOjIQtGtpiwK65kv6h6T18fWNinTdLmmrpIGcfkn6adTdL2l2m+g6TdL2hL+ur0DTFEm/k7RZ0iZJV2fYVO6vgrrq8NeBkp6UtCHq+n6GTeXrsaCuWtZjnHuMpGclrcroa72/zKyjXoQfpp8HPgqMBzYAPSmby4ElsT0P6GsTXfOBn9Xgs1OB2cBATv8ZwGrCfR9zgLVtous0YFXFvpoEzI7tDwDPZfwdK/dXQV11+EvAhNgeB6wF5qRs6liPRXTVsh7j3NcAd2X9vcrwVyeeEbRraYsiumrBzB5n+Ps3zgLusMCfgUMkTWoDXZVjZq+Y2brYfhPYTLhDPknl/iqoq3KiD3bGzXHxlb5CpfL1WFBXLUiaDJwJ/DLHpOX+6sRAkFXaIr0g3lPaAmiUtqhbF8DZMZ2wQtKUjP46KKq9Dj4ZT+9XS/pYlRPHU/JZhG+TSWr11zC6oAZ/xTTHemAr8IiZ5fqrwvVYRBfUsx5vBr4N/Cenv+X+6sRA0LLSFi2myJwPAlPN7ETgUf4X9eumDn8VYR1wtJnNABYDD1Q1saQJwH3AQjPbke7OOKQSf42gqxZ/mdleM5tJqC7QK+n4lEkt/iqgq/L1KOnLwFYze2Y4s4x9o/JXJwaCdi1tMaIuM3vdzN6Jm78APl6ypqIU8WnlmNmOxum9mT0EjJM0sex5JY0jfNj+2szuzzCpxV8j6arLX4n5twG/B76U6qq11EyerprW4ynAXEl/J6SPPytpecqm5f7qxEDQrqUtRtSVyiPPJeR524GVwIXxapg5wHYze6VuUZI+1MiNSuol/D+/XvKcItwRv9nMfpJjVrm/iuiqyV+HSzoktg8CPg/8NWVW+XosoquO9Whm3zGzyWY2lfAZ8ZiZfT1l1nJ/dcSjKpNYe5a2KKrrKklzgT1R1/yydQFIuptwRclESUPADYQfzzCzJcBDhCthBoG3gYvbRNc5wGWS9gC7gHkVBPRTgAuAjTG/DPBd4MMJXXX4q4iuOvw1CVim8KCq9wH3mtmqutdjQV21rMcsyvaXl5hwHMfpcjoxNeQ4juM0gQcCx3GcLscDgeM4TpfjgcBxHKfL8UDgOI7T5XggcPZLJB2WqAr5qqSXEtvjC47xK0nHjmDzLUnnt0jzGoXqsw2dfa0YNzH+UOPaeMdpBr981NnvkfQ9YKeZ/Ti1X4T/8byaLZUiaQ1whZmtH9F438YfAo6Pd8o6TmH8jMDpKCRNkzQgaQmhts4kSUslPa1Qd/76hO0aSTMljZW0TdKiWJDtCUlHRJsfSlqYsF+kUMd+i6ST4/73S7ovHnt3nGtmE5qXS7pF0h8lPSfp9Lj/IEnLJG2UtE7SqXH/WEk3xffZL+nyxHALFerY90s6ZtQOdboCDwROJ9ID3GZms8zsJeA6MzsJmAF8QVJPxjEHA3+IBdmeAC7JGVtm1gtcCzSCypXAq/HYRYTKn3n0JVJDixL7pwCfAb4CLJV0AHAVsNvMTiDcNXxnTHtdBhwJzIgF0e5JjPOamc0ilDC+ZhgdjvMuHVdiwnGA583sqcT2eZIuJfy/H0kIFH9JHbPLzFbH9jPAp3PGvj9hMzW2PwXcCGBmGyRtGkbb13JSQ/fGFNYWSS8C0+O4P4rjbpL0MjCNUBfnZjPbG/uSBceS+s4YRofjvIsHAqcTeavRkDQduBroNbNtCpUcD8w4ZneivZf8tfFOhk0rHqKS/rHOhhlXGfYNsvQ5zrB4asjpdD4IvAnsiNUkv1jCHGuAcwEknUA442iWr8ZqpccQ0kR/Ax4Hzo/jHkcolDYIPEwoHjcm9h066nfgdDX+jcHpdNYR0kADwAvAn0qYYzFwh6T+ON8A4alRWfRJ2hXbr5lZIzANEj74jwAWmNluSYuBWyVtBP4NXBj330pIHfXHSqK3AEtKeF9Ol+CXjzrOKFF4OMhYM/tXTEU9DEyPjxEscvxyYIWZVfaENcdJ4mcEjjN6JgC/jQFBwDeLBgHHaQf8jMBxHKfL8R+LHcdxuhwPBI7jOF2OBwLHcZwuxwOB4zhOl+OBwHEcp8v5L6kF4n7eRjIDAAAAAElFTkSuQmCC\n", 1039 | "text/plain": [ 1040 | "
" 1041 | ] 1042 | }, 1043 | "metadata": { 1044 | "needs_background": "light" 1045 | }, 1046 | "output_type": "display_data" 1047 | } 1048 | ], 1049 | "source": [ 1050 | "model.compile(loss='binary_crossentropy',\n", 1051 | " optimizer='adam', metrics=['accuracy'])\n", 1052 | "\n", 1053 | "BATCH_SIZE = 128\n", 1054 | "EPOCHS = 5 # You can increase number of epoches and batch size\n", 1055 | "\n", 1056 | "history = model.fit(X_train,\n", 1057 | " y_train,\n", 1058 | " batch_size=BATCH_SIZE,\n", 1059 | " epochs=EPOCHS,\n", 1060 | " validation_split=0.1,\n", 1061 | " verbose=1)\n", 1062 | "print(\"\\n--- Learning curve of model training ---\\n\")\n", 1063 | "\n", 1064 | "# summarize history for accuracy and loss\n", 1065 | "plt.figure(figsize=(6, 4))\n", 1066 | "plt.plot(history.history['acc'], \"g--\", label=\"Accuracy of training data\")\n", 1067 | "plt.plot(history.history['val_acc'], \"g\", label=\"Accuracy of validation data\")\n", 1068 | "plt.plot(history.history['loss'], \"r--\", label=\"Loss of training data\")\n", 1069 | "plt.plot(history.history['val_loss'], \"r\", label=\"Loss of validation data\")\n", 1070 | "plt.title('Model Accuracy and Loss')\n", 1071 | "plt.ylabel('Accuracy and Loss')\n", 1072 | "plt.xlabel('Training Epoch')\n", 1073 | "plt.ylim(0)\n", 1074 | "plt.legend()\n", 1075 | "plt.show()" 1076 | ] 1077 | }, 1078 | { 1079 | "cell_type": "code", 1080 | "execution_count": 21, 1081 | "metadata": {}, 1082 | "outputs": [], 1083 | "source": [ 1084 | "def show_confusion_matrix(validations, predictions):\n", 1085 | "\n", 1086 | " matrix = metrics.confusion_matrix(validations, predictions)\n", 1087 | " plt.figure(figsize=(6, 4))\n", 1088 | " sns.heatmap(matrix,\n", 1089 | " cmap=\"coolwarm\",\n", 1090 | " linecolor='white',\n", 1091 | " linewidths=1,\n", 1092 | " xticklabels=LABELS,\n", 1093 | " yticklabels=LABELS,\n", 1094 | " annot=True,\n", 1095 | " fmt=\"d\")\n", 1096 | " plt.title(\"Confusion Matrix\")\n", 1097 | " plt.ylabel(\"True Label\")\n", 1098 | " plt.xlabel(\"Predicted Label\")\n", 1099 | " plt.show()" 1100 | ] 1101 | }, 1102 | { 1103 | "cell_type": "code", 1104 | "execution_count": 22, 1105 | "metadata": {}, 1106 | "outputs": [ 1107 | { 1108 | "name": "stdout", 1109 | "output_type": "stream", 1110 | "text": [ 1111 | "(3980, 2)\n" 1112 | ] 1113 | } 1114 | ], 1115 | "source": [ 1116 | "#print(y_test.shape)\n", 1117 | "y_test = np_utils.to_categorical(y_test, num_classes)\n", 1118 | "print(y_test.shape)" 1119 | ] 1120 | }, 1121 | { 1122 | "cell_type": "code", 1123 | "execution_count": 23, 1124 | "metadata": {}, 1125 | "outputs": [ 1126 | { 1127 | "name": "stdout", 1128 | "output_type": "stream", 1129 | "text": [ 1130 | "3980/3980 [==============================] - 0s 86us/step\n", 1131 | "\n", 1132 | "Accuracy on test data: 0.59\n", 1133 | "\n", 1134 | "Loss on test data: 0.68\n", 1135 | "\n", 1136 | "--- Confusion matrix for test data ---\n", 1137 | "\n" 1138 | ] 1139 | }, 1140 | { 1141 | "data": { 1142 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEfCAYAAAC+klb0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5hdVb3/8fdnEkqogRBIoUUIg/QmvQRRpErvXiOCuShFBWkXfoB4QVBRQMo1CtIDRIqhXYhcuoQeIAEOBKIQCAQCAVIIycz398deEw7DZObM5OyZkz2f1/PsZ/ZZu6y1Z858zzprr72WIgIzMyuGuq4ugJmZVY+DuplZgTiom5kViIO6mVmBOKibmRWIg7qZWYE4qNsCk9RL0h2SPpY0cgHOc5ik+6pZtq4g6R5JQ7u6HNY9Oah3I5IOlfS0pOmSJqfgs20VTr0/sBLQJyIO6OhJIuL6iNi5CuX5EklDJIWkW5ulb5jSH6zwPGdJuq6t/SJi14i4uoPFNVsgDurdhKTjgQuBc8kC8KrAZcBeVTj9asCrETG3CufKy/vA1pL6lKUNBV6tVgbK+H/KupTfgN2ApGWBs4GjI+LWiJgREXMi4o6IODHts5ikCyW9k5YLJS2Wtg2RNEnSCZKmpFr+4WnbL4EzgIPSN4AjmtdoJa2easQ90+sfSHpD0qeSJko6rCz90bLjtpb0VGrWeUrS1mXbHpT0K0mPpfPcJ2mFVn4NnwO3Awen43sABwLXN/tdXSTpLUmfSHpG0nYpfRfgv8qu8/mycpwj6TFgJvC1lHZk2n65pL+Vnf98SfdLUsV/QLN2cFDvHrYCFgdua2Wf04AtgY2ADYHNgdPLtvcDlgUGAkcAl0paLiLOJKv93xQRS0XEFa0VRNKSwMXArhGxNLA1MLaF/ZYH7kr79gF+D9zVrKZ9KHA4sCKwKPCL1vIGrgG+n9a/A4wH3mm2z1Nkv4PlgRuAkZIWj4j/bXadG5Yd8x/AMGBp4N/NzncCsEH6wNqO7Hc3NDw+h+XEQb176AN80EbzyGHA2RExJSLeB35JFqyazEnb50TE3cB0oL6D5WkE1pPUKyImR8T4FvbZHXgtIq6NiLkRMQJ4BdizbJ+/RsSrETELuJksGM9XRPwTWF5SPVlwv6aFfa6LiKkpzwuAxWj7Oq+KiPHpmDnNzjcT+B7Zh9J1wLERMamN85l1mIN69zAVWKGp+WM+BvDlWua/U9q8czT7UJgJLNXegkTEDOAg4ChgsqS7JK1dQXmayjSw7PW7HSjPtcAxwI608M0lNTG9nJp8ppF9O2mtWQfgrdY2RsSTwBuAyD58zHLjoN49PA58Buzdyj7vkN3wbLIqX22aqNQMYImy1/3KN0bEvRHxbaA/We37zxWUp6lMb3ewTE2uBX4C3J1q0fOk5pGTydral4uI3sDHZMEYYH5NJq02pUg6mqzG/w5wUseLbtY2B/VuICI+JruZeamkvSUtIWkRSbtK+k3abQRwuqS+6YbjGWTNBR0xFthe0qrpJu2pTRskrSTpu6ltfTZZM05DC+e4G1grdcPsKekgYB3gzg6WCYCImAjsQHYPobmlgblkPWV6SjoDWKZs+3vA6u3p4SJpLeC/yZpg/gM4SVKrzURmC8JBvZuIiN8Dx5Pd/HyfrMngGLIeIZAFnqeBF4AXgWdTWkfyGg3clM71DF8OxHVkNw/fAT4kC7A/aeEcU4E90r5TyWq4e0TEBx0pU7NzPxoRLX0LuRe4h6yb47/Jvt2UN600PVg1VdKzbeWTmruuA86PiOcj4jWyHjTXNvUsMqs2+Sa8mVlxtHbjzHJUX19/JVlNdEqpVFovpS1PVsNdHfgXcGCpVPoobRtC9vDQIsAHpVJph5T+c+BIsnbdF4HDS6XSZ515LZaP+vr6nwI/ImvT/3OpVLqwI+8R617c/NJ1rgJ2aZZ2CnB/qVQaDNyfXlNfX9+b7OnP75ZKpXWBA1L6QOA4YLP0wdCD9HCNLdzq6+vXIwvom5M9N7BHfX39YNr5HrHux0G9i5RKpYfJ2pTL7QU0jRlyNV/0VjkUuLVUKr2Zjp1SdkxPoFd9fX1Psh4nHe2xYrXl68CYUqk0s1QqzQUeAvahY+8R60ZyaX6RtG9r2yPi1ta2d2MrlUqlyQClUmlyfX39iil9LWCR+vr6B8l6aFxUKpWuKZVKb9fX1/8OeBOYBdxXKpUW+lEODYBxwDn19fV9yP62u5HdyG7Xe6Tzi21dLa829T1b2RaAg3r79AQ2BXYCegGP19fXjyHrxbIXMAiYBoysr6//XqlU6mhXRKsRpVLp5fr6+vOB0WTdPp8n6245Py2+R0qlUtUGLLOFQy5BPSIO78hxkoaRjaHBn/70p00PW3/FNo5YuN3525P56YVXMePx2wNgtX4r8K97rom+vZfh/WmfsFq/FZjx+O1x7P67MHvOXI7a59vTAX55xUi2Xr++BPDPF0ucecQBUwDufOwZXnz9zb1mPH77tV13VflZcquspWHEY92jx9ZZV74yb/0ft/x+u2WW68cTo69h+F3vxdK9V+TTaVPos9LqjHgsYqf9jmfunNnsuPex0wH+fuVprLn+dqXu8Ls6ZBvBFw+Iddhdi9RX/MvafU6pZgdky71NXdLukk6SdEbTMr99I2J4RGwWEZsNGzYs76LVnO03Woc7H30GgDsffYYdNl4XgB02WYfnXp3I3IYGZs3+nHFvvMWgASvSr09vXnz9TWbN/pyI4MmXJjCof7E/CLuT6Z9MBWDa1Hd4+ZnRrL/F7tRv/E3GPpY9WjD2sdup33gnANbeeCfefO0ZGhrm8vnsWUya+AIr9P9al5V9YaRFVPFSy3Lt0ijpf8hu3u0I/IVsMoUn88xzYXHq5TfwzCtvMG36DHb5+Tkctfe3OXyPIZx86fXc/shT9Fu+N785+nsAfG3ASmy9fj0H/b8LqZPYe/tvsObK2ZP3O31jfQ4782J69KijftUB7Dtki668LKuimy89jpnTp9GjR092/94Z9FpyWbbd7UeMvPznPPfILSzbpz8H/PhCAPoOWIM119uOy8/YC9XVscl2+7PSymt18RUsXOp61nawrlSuDx9JeiEiNij7uRRwa4Wz28SMx29vey/rNrpb84tVplrNL/f2WbfiN9Z3po6v2U+AvB8+mpV+zpQ0gOxx70E552lm1m5FqannHdTvlNQb+C3ZWCJB1gxjZlZTar2tvFK5BvWI+FVavUXSncDiacRAM7Oa4pp6BdI8kLuTjVPRND9l04iBZmY1o0evYjxgn3fzyx1kw5e+SDaFmZlZTVIP19QrsXJEbJBzHmZmC6yuIEE97+8b90iqpPuimVmXUp0qXmpZ3jX1McBtafqvOWR9SSMilmn9MDOzzqUeblOvxAXAVsCL4SmWzKyG9VjEQb0SrwHjHNDNrNYVpU0976A+GXhQ0j1kM8cDuEujmdUc936pzMS0LJoWM7OapDo3v7QqPXi0VEScmFceZmbVUuu9WiqVW1CPiAZJm+R1fjOzanKbemXGShoFjARmNCV6jlIzqzV1PXt0dRGqIu+gvjzZcLvfLEvzHKVmVnPc/FKBjs5VambW2YrS/JLr7V5JK0u6TdIUSe9JukXSynnmaWbWEUUZJiDvPjx/BUYBA4CBZKM2/jXnPM3M2k11dRUvtSzv0vWNiL9GxNy0XAX0zTlPM7N2K0pNPe8bpR9I+h4wIr0+hOzGqZlZTanrWds18ErlfRU/BA4E3iUbMmD/lGZmVlOK0vySd++XN4Hv5pmHmVk1FKX3Sy5BXdIZrWyOsgmpzcxqQjXbyiVdCewBTImI9VLab4E9gc+B14HDI2Ja2nYqcATQABwXEfem9F2Ai4AewF8i4ry28s7re8SMFhbICn1yTnmamXVYlZtfrgJ2aZY2GlgvTfH5KnAqgKR1gIOBddMxl0nqkcbPuhTYFVgHOCTt26pcauoRcUHTuqSlgZ8ChwM3kk2cYWZWU6p5ozQiHpa0erO0+8pejiG7xwiwF3BjRMwGJkqaAGyetk2IiDcAJN2Y9n2ptbzzHKVxeeB44DDgamCTiPgor/zMzBZEJ98A/SFwU1ofSBbkm0xKaQBvNUvfoq0T59Wm/ltgX2A4sH5ETM8jHzOzamlPm7qkYcCwsqThETG8wmNPA+YC1zcltbBb0HLzeJuzyOVVUz+BbKaj04HTpHll9sTTZlaT2lNTTwG8oiD+pTykoWQ3UHcqm+ZzErBK2W4rA++k9fmlz1debeq13ZHTzKw55dulMfVkORnYISJmlm0aBdwg6fdkQ6oMBp4kqwQPljQIeJvsZuqhbeWT9xOlZmYLhSp3aRwBDAFWkDQJOJOst8tiwOjUejEmIo6KiPGSbia7AToXODoiGtJ5jgHuJevSeGVEjG8rbwd1MzOqO0lGRBzSQvIVrex/DnBOC+l3A3e3J28HdTMzPPG0mVmh1Proi5VyUDczw0HdzKxY3PxiZlYcyrlLY2dxUDczo7q9X7qSg7qZGbj5xcysSHyj1MysQCTX1M3MisM1dTOz4vCNUjOzInHzi5lZcfhGqZlZkbhLo5lZcfiJUjOzInFN3cysONTDvV/MzIrDN0rNzIrDT5SamRWJa+pmZsXhOUrNzIrEXRrNzArEvV/MzIrDzS9mZkXi3i9mZgXi3i9mZsXhfupmZkXiG6VmZgXiLo1mZgVSkN4vxbgKM7MFpbrKl7ZOJV0paYqkcWVpB0gaL6lR0mbN9j9V0gRJJUnfKUvfJaVNkHRKJZfhoG5mBlnvl0qXtl0F7NIsbRywL/BweaKkdYCDgXXTMZdJ6iGpB3ApsCuwDnBI2rdVbn4xM4Oq9lOPiIclrd4s7WVocYalvYAbI2I2MFHSBGDztG1CRLyRjrsx7ftSa3k7qJuZQVf2fhkIjCl7PSmlAbzVLH2Ltk7m5hczM8h6v1S4SBom6emyZdiC5NxCWrSS3irX1M3MoF3NLxExHBhepZwnAauUvV4ZeCetzy99vlxTNzODrEtjpUt1jQIOlrSYpEHAYOBJ4ClgsKRBkhYlu5k6qq2TuaZuZgZVffhI0ghgCLCCpEnAmcCHwB+BvsBdksZGxHciYrykm8lugM4Fjo6IhnSeY4B7gR7AlRExvq28HdTNzKDavV8Omc+m2+az/znAOS2k3w3c3Z68HdTNzMBjv5iZFUl47BczswLx0LtmZgXioG5mVhxufjEzK5I63yg1MyuOgoyn7qBuZoabX8zMiqXoN0ol3UYrI4JFxL65lMjMrAtE0YM6cEmnlcLMrKsVvfklIu5vWk8jhK0aERM6pVRmZp0sCtL7pc3vG5J2B14ERqfXG6WmGTOz4qjixNNdqZLSnU02hdI0gIgYC6yZZ6HMzDpbSBUvtayS3i9zImJas8lS25xSycxsoVLjNfBKVRLUX5Z0IFCXZuX4KV+eJNXMbKEXLU4JuvCp5KPpGGBToJFsgPfZwM/yLJSZWWeLup4VL7WszdJFxAzgZEm/zF7GrPyLZWbWuWq9rbxSlfR+2UTSc8CrwGuSnpG0Sf5FMzPrPKG6ipdaVsn3iL8CP4uIBwAkDUlpG+ZYLjOzzlWQmnolQX1GU0AHiIgHJU3PsUxmZp2u1mvglWpt7JcN0uoTki4FRpB1ZTwIeGB+x5mZLYyK0vultZr6pc1eb1C27n7qZlYoRRkmoLWxX7brzIKYmXWlqKiHd+2rqMOlpO8A6wKLN6VFxLl5FcrMrLMVpUtjm0Fd0mVAb2B7sl4v++EnSs2sYIpyo7SSq9g2Ig4FpkbE/yMb3GvlfItlZta5AlW81LJKml+aniD9TFI/YCqwem4lMjPrAkWpqVcS1O+R1Bv4HTAWaACuzrVUZmadrLG7BPWIOCutjpR0J9ALGJRnoczMOlt3qqnPkwbzmiVpLLBqPkUyM+t8td5WXqmOfjQV4+rNzJJqDugl6UpJUySNK0tbXtJoSa+ln8uldEm6WNIESS+UD5goaWja/zVJQyu5jo4GdT9RamaFUuXeL1cBuzRLOwW4PyIGA/en1wC7AoPTMgy4HLIPAeBMsh6HmwNnNn0QtKa1sV9uo+XgLaBPWyeuhiW32rszsrGFzCHb+IuiVV+jqjdMQEQ8LGn1Zsl7AUPS+tXAg8DJKf2aiAhgjKTekvqnfUdHxIcAkkaTfVCMaC3v1trUL+ngNjOzhU4nPFG6UkRMBoiIyZJWTOkDgbfK9puU0uaX3qrWxn65v70lrrZt93yoq4tgNeTRO3YA4K5F6ru4JFZLdp9Tqsp5IioP6pKGkTWVNBkeEcM7mHVLGUcr6a2q7cn2zMw6SXsG9EoBvL1B/D1J/VMtvT8wJaVPAlYp229l4J2UPqRZ+oNtZVKMjplmZguoE4YJGAU09WAZCvy9LP37qRfMlsDHqZnmXmBnSculG6Q7p7RWVVxTl7RYRMxuzxWYmS0sqtlPXdIIslr2CpImkfViOQ+4WdIRwJvAAWn3u4HdgAnATOBwgIj4UNKvgKfSfmc33TRtTSWjNG4OXAEsC6wqaUPgyIg4tuIrNDOrcY1VbLiIiEPms2mnFvYN4Oj5nOdK4Mr25F3JVVwM7EE2kBcR8TywY3syMTOrdRGqeKlllTS/1EXEv/Xl7j4NOZXHzKxLFGWYgEqC+lupCSYk9QCOBV7Nt1hmZp2rOwX1H5M1wawKvAf8I6WZmRVGtwnqETEFOLgTymJm1mVqva28UpX0fvkzLTzFFBHDWtjdzGyh1FCQx3YqaX75R9n64sA+fHk8AjOzhV63qalHxE3lryVdC4zOrURmZl2g27Spt2AQsFq1C2Jm1pW6TU1d0kd80aZeB3zIF4O7m5kVQreoqSt74mhD4O2U1JgeaTUzK5SGgtTUW73dmwL4bRHRkBYHdDMrpKIME1BJH54nyydCNTMrok4YerdTtDZHac+ImAtsC/xI0uvADLLZOCIiHOjNrDBqvQZeqdba1J8ENgE8+7OZFV6t18Ar1VpQF0BEvN5JZTEz6zKNBblj2FpQ7yvp+PltjIjf51AeM7Mu0RjFHyagB7AULc9obWZWKEXp29daUJ8cEWd3WknMzLpQY0Hqr222qZuZdQfdoffLVyZINTMrqsI3v0TEh51ZEDOzrtQdujSamXUbDY0O6mZmhVH45hczs+6kO/R+MTPrNlxTNzMrkO7QpdHMrNtoaOzqElSHg7qZGe7SaGZWKN1hlEYzs26jKDdKizHWpJnZAoqofGmLpJ9KGidpvKSfpbTlJY2W9Fr6uVxKl6SLJU2Q9MKCTh/qoG5mBjSGKl5aI2k94EfA5sCGwB6SBgOnAPdHxGDg/vQaYFdgcFqGAZcvyHU4qJuZAY2NlS9t+DowJiJmpnmeHwL2AfYCrk77XM0XU4XuBVwTmTFAb0n9O3odDupmZmQ3Sitd2jAO2F5SH0lLALsBqwArRcRkgPRzxbT/QOCtsuMnpbQO8Y1SMzPa9/CRpGFkTSVNhkfE8Ow88bKk84HRwHTgeWBua6drqTgVF6YZB3UzM9rX+yUF8OGtbL8CuAJA0rlkte/3JPWPiMmpeWVK2n0SWU2+ycrAO+0qfBk3v5iZUdXmFyStmH6uCuwLjABGAUPTLkOBv6f1UcD3Uy+YLYGPm5ppOsI1dTMzKroB2h63SOoDzAGOjoiPJJ0H3CzpCOBN4IC0791k7e4TgJnA4QuSsYO6mRnVfaI0IrZrIW0qLUwTGhEBHF2tvB3UzcwozhOlDupmZlS9+aXLOKibmeGauplZoTiom5kVSEO77pTW7tjrDupmZrimbmZWKL5RamZWIK6pm5kViKezMzMrENfUzcwKpKHBvV/MzArDNXUzswJpLEijuoO6mRmuqZuZFYqDuplZgbTvRmntclA3MwOiIFV1B3UzMzxMgJlZobimbmZWIAXp0eigbmYGEAWJ6g7qZma494uZWaG4Td3MrEDCvV/MzIqj0TV1M7PicPOLmVmBeJRGM7MCaXTvFzOz4nCbuplZgRTl4aO6ri6AmVktiMaoeGmLpJ9LGi9pnKQRkhaXNEjSE5Jek3STpEXTvoul1xPS9tUX5Doc1M3MyMZ+qXRpjaSBwHHAZhGxHtADOBg4H/hDRAwGPgKOSIccAXwUEWsCf0j7dZiDupkZ0NjQWPFSgZ5AL0k9gSWAycA3gb+l7VcDe6f1vdJr0vadJKmj1+GgbmZG1qWx0qU1EfE28DvgTbJg/jHwDDAtIuam3SYBA9P6QOCtdOzctH+fjl6Hg7qZGdnDR5UukoZJerpsGdZ0HknLkdW+BwEDgCWBXVvKsumQVra1m3u/mJnRvt4vETEcGD6fzd8CJkbE+wCSbgW2BnpL6plq4ysD76T9JwGrAJNSc82ywIcdughcUzczA6ra++VNYEtJS6S28Z2Al4AHgP3TPkOBv6f1Uek1afv/xQKMWeCaupkZ1Xv4KCKekPQ34FlgLvAcWa3+LuBGSf+d0q5Ih1wBXCtpAlkN/eAFyd9B3cwMKu3VUpGIOBM4s1nyG8DmLez7GXBAtfJ2UDczwwN6mZkVSlGGCXBQNzPD46mbmRVKNBZjPjsHdTMz3KZuZlYo1ez90pUc1M3M8I1SM7NCaQzX1M3MCsM1dTOzAnFQNzMrkIaGhq4uQlU4qJuZ4Zq6mVmhhG+UmpkVh2vqZmYF4qBuZlYg7qduZlYgjXPd+8XMrDDc/GJmViDu/WJVscrAXpx90jrzXg/otzh/uf5f9O2zGNts3oc5cxp5593POPeiV5g+o4GePcWJR6/F2msuRQRcNHwCz437uAuvwKplgz+fy4q7DeHzKVN5eOM9AVj7vJNYafcdaZwzh5mvv8nzR57K3I8/ZcAhe/K1E46Yd+wy69fz6Ob7MOP1t9jqwevnpfca2I+3bxjFSyec2+nXs7ApytC7quHZPmLbPR/q6jJ0qro6uO2qrRh2wrOsuvISPPv8RzQ0wo+HDgLg8qsnsu9uA6gfvDS/vqhE72UX4YKz1ufI45+ldv+M1fPoHTsAcNci9V1cknwsv+1mzJ0xk42uPH9eUF/hW9sw9YExREMDa5/7CwBe+a/ffem4pddbi81uuYwH6r/1lXNu+8QtvHTCr/nw0afzv4AusvucEoAW9DxD9n+84v+iB/+21QLnl5e6ri6AfWHTDZfj7cmzeO/92Tz1XBbQAcaXPqHvCosBsPqqS/DM8x8BMO3jOXw6Yy5rr7l0VxXZqujDR59mzodf/tb1wT8eI9Lj6x89MZbFV+73leMGHLQ779x051fSl1hzNRbt26fQAb2aGhsaKl5qWa7NL5L2bSH5Y+DFiJiSZ94Lo29t15d/PPzVX8vu3+7P/Y9k6RMmzmC7LVbg/oensGLfxalfY2lW7LsYL7/2aWcX1zrZKj/Yj8kj7/lKev8DduPp/X7ylfQBB+3B5JF3d0bRCsE3SitzBLAV8EB6PQQYA6wl6eyIuDbn/BcaPXuKbbZYgf+5ZuKX0r9/4Ko0NAT3PZgF9btGT2a1VZbgL3/YlHenfMa4Vz6moaEYb0abvzVPOYqY28DbN4z6UnrvzTegYdYspo9/7SvHDDhwN8YeflJnFXGhV5Q5SnNtU5d0B3BkRLyXXq8EXA4cCTwcEes1238YMCy9HB4Rw3MrXO3ZCzga2LksbShwVL9+/a5/9913L5nPcf8k+32+lHP5rHOsDtwJlP9vDAWOAnYCZjYlShoWEV8H3gea3wndEBgJrJVnYa325F1TX70poCdTgLUi4kNJc5rvnIJ4dwrk5Q4BRpS93gU4GdjhvffeuwdoCupLkN0UmgF8G5iLA3qRzXsfUBbQAerq6oYB/YDtWziu+fvJuom8g/ojku4kqzEA7Ac8LGlJYFrOeS9MliAL0P9ZlnYJsBgw+uWXX14L+B+y2tqKwL1AI/A28B+dW1TL0QiyJsoVgEnAmcCppPdB2mcM2fuAXXfddem03xstnOtAYLd8i2u1KO/mF5EF8m3IapePArdEDfejrEWSno6Izbq6HFZb/L6wluRaU0/B+29psY7rrk1S1jq/L+wr8q6p7wucT9ZkoLRERCyTW6ZmZt1Y3kF9ArBnRLycWyZmZjZP3k+UvtfdA7qkkHRB2etfSDqrk8twlaT9OzNP+ypJp0kaL+kFSWMlbVHFcw9JnRKQ9F1Jp1Tr3LZwybv3y9OSbgJuB2Y3JUbErTnnW0tmA/tK+nVEfNDegyX1jIi5OZTLOpGkrYA9gE0iYrakFYBF88grIkYBo9rc0Qop76C+DFnf2vIHagLoTkF9LtkNrZ8Dp5VvkLQacCXQl+wBksMj4k1JVwEfAhsDz0r6FBgE9Cd7mOR4YEtgV7JujXtGxBxJZwB7Ar3IHkr6T/c0qhn9gQ8iYjZA0we8pH8BNwE7pv0OjYgJkvqSdWNdNaX/LCIeS92B/wisT/b/e1ZE/L08I0k/ADaLiGPSe+kTYDOyPu0nRcTfJNWRdZvdAZhI9q39yohwp4aFXK7NLxFxeAvLD/PMs0ZdChwmadlm6ZcA10TEBsD1wMVl29YCvhURJ6TXawC7kz15eh3wQESsD8xK6QCXRMQ30pO6vchqhlYb7gNWkfSqpMsk7VC27ZOI2Jzs/XBhSrsI+ENEfIOsW/BfUvppwP+l9B2B36ZA35r+wLZk74fzUtq+ZE+vrk/2RPJWC3JxVjtyqalLOikifiPpj2Q18y+JiOPyyLdWRcQnkq4BjiMLwk22IvvnArgW+E3ZtpERUT4c3D2pNv4i0AP435T+Itk/J8COkk4ie5hpeWA8cEc1r8U6JiKmS9oU2I4sGN9U1u49ouznH9L6t4B1skc9AFhG0tJk33q/K+kXKX1xvqjNz8/tkc0A8VIaqgOyID8ypb8r6YH5H24Lk7yaX5pujnrMzy9cCDwL/LWVfco/AGc029b0tb1R0pyyZpVGoKekxYHLyL52v5Vuxi5elZJbVaQP6QeBB9OH89CmTeW7pZ91wFYRUV4JmPdAX0SUmqWvxPzNLltXs59WMLk0v0TEHenn1S0teeRZ6yLiQ+BmspErm/wTODitH0b2xG1HNQXwDyQtBbi3Sw2RVC9pcFnSRsC/0/pBZT8fT+v3AceUHb9RWr0XODYFdyRt3MEiPQrsJ6kufSAM6eB5rKUg7a4AAAV6SURBVMbk1fxyBy00uzSJiO/mke9C4ALK/lHJmmOulHQi6UZpR08cEdMk/ZmsOeZfwFMLUE6rvqWAP0rqTXbzfALZiKR7AItJeoKsknVI2v844FJJL5D9nz5MNubLr8i+9b2QAvu/6Ni9k1vIRn0cB7wKPEE214Et5HJ5+KjZTaCviIjuNU+d2Xyk3i+bdaS7axXyXiq19fcBngS2iYh3O7scVl251NTLg7akXsCqzdsAzazL3Zm+OSwK/MoBvRjyHiZgT+B3wKIRMSi1C57djZtfzMxylfcwAWcBm5PGTo+IsXzR/c7MzKos76A+NyJ888XMrJPkPUzAOEmHAj1Sd67jyLrxmZlZDvKuqR8LrEv28MMNZF2mfpZznlYFkhrSSILjJI2UtMQCnKviEQQl9Zb0kw7kcVbZU5ZtprdynunVyNesq+Q99svMiDgNGJLGJDk9Ij7LM0+rmlkRsVEaR+Zz0ryYTZRp9/snIkZFxHmt7NIbaHdQN7NMrkFd0taSXiINGyBpQ0mX5Zmn5eIRYE1Jq0t6Of0NnyUboGpnSY9LejbV6JcCkLSLpFckPcoX49sg6QeSLknrK0m6TdLzadmabMCpNdK3hN+m/U6U9FQah/yXZec6TVJJ0j+A+vZckKTbJT2jbHzzYc22XZCu5/40WiKS1pD0v+mYRySt3YHfo1nu8m5++QPwHWAqQEQ8D2yfc55WRZJ6kg3x+2JKqicbWXJjsvFpTicbTXITsrF+jk/j0PyZbBjg7ciGfG3JxcBDEbEhsAnZAGSnAK+nbwknStoZGEzWi2ojYFNJ26fBsQ4mG554X+Ab7by0H0bEpmRD0h6XHsABWBJ4Nl3PQ8CZKX04cGw65hdk4+yY1Zy8b5SSBpcqT2qY375WU3pJGpvWHwGuAAYA/46IMSl9S2Ad4LH0N16UbOyStYGJEfEagKTryB6Jb+6bwPdh3mBXH0tartk+O6flufR6KbIgvzRwW0TMTHm0d1KI4yTtk9ZXSeecSjZA2k0p/Trg1vTtY2tgZNl7ebF25mfWKfIO6m+lr9QhaVGy3i/denq7hcisiNioPCEFtPLRIwWMjohDmu23Ea2M/dNOAn4dEX9qlsfPOpqHpCFkQ9tuFREzJT3I/Ee0DLJvtNOa/z7MalHezS9HAUcDA8lm6NkovbZiGANsI2lNAElLSFoLeAUYJGmNtN8h8zn+fuDH6dgekpYBPiWrhTe5F/hhWVv9QEkrkg1wtY+kXsrGGd+zHeVeFvgoBfS1yb5xNKnjixEuDwUejYhPgImSDkhlkKQN25GfWafJtaaeBik6LM88rOtExPvKpk4bIampOeL0iHg13Xy8S9IHZMO8rtfCKX4KDJd0BFmz3I8j4nFJj0kaRzYxyImSvg48nr4pTAe+FxHPKpv/dizZELaPtFLU01PNvskawFHKRkAskX04NZkBrCvpGbIuuE3D4h4GXC7pdGAR4Ebg+bZ/S2adK++xX75GNi3XlmRfYx8Hfh4Rb+SWqZlZN5Z388sNZBND9Ce7yTaSL6buMjOzKss7qCsiro2IuWm5jurdQDMzs2bybn45j2yExhvJgvlBZF3BLoV5U7yZmVmV5B3UJ6bVpkzKO6xHRHwtt8zNzLqhvKaz+wbwVtNMKpKGAvuRzad4lmvoZmb5yKtN/U9kg0AhaXvg18DVZF3EhueUp5lZt5dXP/UeZbXxg4DhEXELcEvZo+dmZlZledXUe6SBoAB2Av6vbFvu482YmXVXeQXYEcBD6WnCWaSn/dLj5J7ezswsJ7n1fpG0JdlDR/dFxIyUthawVEQ8m0umZmbdXK5dGs3MrHPl/USpmZl1Igd1M7MCcVA3MysQB3UzswJxUDczK5D/D56nqwjNiEvKAAAAAElFTkSuQmCC\n", 1143 | "text/plain": [ 1144 | "
" 1145 | ] 1146 | }, 1147 | "metadata": { 1148 | "needs_background": "light" 1149 | }, 1150 | "output_type": "display_data" 1151 | }, 1152 | { 1153 | "name": "stdout", 1154 | "output_type": "stream", 1155 | "text": [ 1156 | "\n", 1157 | "--- Classification report for test data ---\n", 1158 | "\n", 1159 | " precision recall f1-score support\n", 1160 | "\n", 1161 | " 0 0.59 0.54 0.57 1974\n", 1162 | " 1 0.58 0.64 0.61 2006\n", 1163 | "\n", 1164 | " accuracy 0.59 3980\n", 1165 | " macro avg 0.59 0.59 0.59 3980\n", 1166 | "weighted avg 0.59 0.59 0.59 3980\n", 1167 | "\n" 1168 | ] 1169 | } 1170 | ], 1171 | "source": [ 1172 | "import seaborn as sns\n", 1173 | "x_test = X_test.astype(\"float32\")\n", 1174 | "y_test = y_test.astype(\"float32\")\n", 1175 | "\n", 1176 | "#y_test = np_utils.to_categorical(y_test, num_classes)\n", 1177 | "\n", 1178 | "score = model.evaluate(x_test, y_test, verbose=1)\n", 1179 | "\n", 1180 | "print(\"\\nAccuracy on test data: %0.2f\" % score[1])\n", 1181 | "print(\"\\nLoss on test data: %0.2f\" % score[0])\n", 1182 | "\n", 1183 | "# %%\n", 1184 | "\n", 1185 | "print(\"\\n--- Confusion matrix for test data ---\\n\")\n", 1186 | "\n", 1187 | "y_pred_test = model.predict(x_test)\n", 1188 | "# Take the class with the highest probability from the test predictions\n", 1189 | "max_y_pred_test = np.argmax(y_pred_test, axis=1)\n", 1190 | "max_y_test = np.argmax(y_test, axis=1)\n", 1191 | "LABELS=[\"Normal\",\n", 1192 | " \"Speeling\"]\n", 1193 | "show_confusion_matrix(max_y_test, max_y_pred_test)\n", 1194 | "\n", 1195 | "# %%\n", 1196 | "\n", 1197 | "print(\"\\n--- Classification report for test data ---\\n\")\n", 1198 | "\n", 1199 | "print(classification_report(max_y_test, max_y_pred_test))" 1200 | ] 1201 | }, 1202 | { 1203 | "cell_type": "markdown", 1204 | "metadata": {}, 1205 | "source": [ 1206 | "# GRU model for EEG classification" 1207 | ] 1208 | }, 1209 | { 1210 | "cell_type": "code", 1211 | "execution_count": 25, 1212 | "metadata": {}, 1213 | "outputs": [], 1214 | "source": [ 1215 | "from numpy import concatenate\n", 1216 | "from matplotlib import pyplot\n", 1217 | "from pandas import read_csv\n", 1218 | "from pandas import DataFrame\n", 1219 | "from pandas import concat\n", 1220 | "from sklearn.preprocessing import MinMaxScaler\n", 1221 | "from sklearn.preprocessing import LabelEncoder\n", 1222 | "from sklearn.metrics import mean_squared_error\n", 1223 | "from keras.models import Sequential\n", 1224 | "from keras.layers import Dense\n", 1225 | "from keras.layers import LSTM, Activation, Dropout, GRU\n", 1226 | "import pandas as pd\n", 1227 | "from sklearn import model_selection\n", 1228 | "from sklearn.ensemble import BaggingClassifier\n", 1229 | "from sklearn.tree import DecisionTreeClassifier\n", 1230 | "from keras.optimizers import SGD\n", 1231 | "import math" 1232 | ] 1233 | }, 1234 | { 1235 | "cell_type": "code", 1236 | "execution_count": 27, 1237 | "metadata": {}, 1238 | "outputs": [], 1239 | "source": [ 1240 | "# same dataset used in this model\n", 1241 | "X_train = np.asarray(np.reshape(x_train, (x_train.shape[0], 14, 1)))\n", 1242 | "X_test = np.asarray(np.reshape(x_test, (x_test.shape[0], 14, 1)))" 1243 | ] 1244 | }, 1245 | { 1246 | "cell_type": "code", 1247 | "execution_count": 31, 1248 | "metadata": {}, 1249 | "outputs": [ 1250 | { 1251 | "name": "stdout", 1252 | "output_type": "stream", 1253 | "text": [ 1254 | "\n", 1255 | "_________________________________________________________________\n", 1256 | "Layer (type) Output Shape Param # \n", 1257 | "=================================================================\n", 1258 | "gru_10 (GRU) (None, 14, 50) 7800 \n", 1259 | "_________________________________________________________________\n", 1260 | "dropout_11 (Dropout) (None, 14, 50) 0 \n", 1261 | "_________________________________________________________________\n", 1262 | "gru_11 (GRU) (None, 14, 50) 15150 \n", 1263 | "_________________________________________________________________\n", 1264 | "dropout_12 (Dropout) (None, 14, 50) 0 \n", 1265 | "_________________________________________________________________\n", 1266 | "gru_12 (GRU) (None, 14, 50) 15150 \n", 1267 | "_________________________________________________________________\n", 1268 | "dropout_13 (Dropout) (None, 14, 50) 0 \n", 1269 | "_________________________________________________________________\n", 1270 | "gru_13 (GRU) (None, 50) 15150 \n", 1271 | "_________________________________________________________________\n", 1272 | "dropout_14 (Dropout) (None, 50) 0 \n", 1273 | "_________________________________________________________________\n", 1274 | "dense_4 (Dense) (None, 2) 102 \n", 1275 | "_________________________________________________________________\n", 1276 | "activation_3 (Activation) (None, 2) 0 \n", 1277 | "=================================================================\n", 1278 | "Total params: 53,352\n", 1279 | "Trainable params: 53,352\n", 1280 | "Non-trainable params: 0\n", 1281 | "_________________________________________________________________\n", 1282 | "Train on 11000 samples, validate on 3980 samples\n", 1283 | "Epoch 1/50\n", 1284 | " - 9s - loss: 0.6931 - acc: 0.5133 - val_loss: 0.6975 - val_acc: 0.5040\n", 1285 | "Epoch 2/50\n", 1286 | " - 5s - loss: 0.6933 - acc: 0.5286 - val_loss: 0.6972 - val_acc: 0.5040\n", 1287 | "Epoch 3/50\n", 1288 | " - 5s - loss: 0.6932 - acc: 0.5286 - val_loss: 0.6972 - val_acc: 0.5040\n", 1289 | "Epoch 4/50\n", 1290 | " - 5s - loss: 0.6933 - acc: 0.5286 - val_loss: 0.6968 - val_acc: 0.5040\n", 1291 | "Epoch 5/50\n", 1292 | " - 5s - loss: 0.6931 - acc: 0.5286 - val_loss: 0.6968 - val_acc: 0.5040\n", 1293 | "Epoch 6/50\n", 1294 | " - 5s - loss: 0.6931 - acc: 0.5286 - val_loss: 0.6965 - val_acc: 0.5040\n", 1295 | "Epoch 7/50\n", 1296 | " - 5s - loss: 0.6931 - acc: 0.5286 - val_loss: 0.6965 - val_acc: 0.5040\n", 1297 | "Epoch 8/50\n", 1298 | " - 5s - loss: 0.6929 - acc: 0.5286 - val_loss: 0.6966 - val_acc: 0.5040\n", 1299 | "Epoch 9/50\n", 1300 | " - 5s - loss: 0.6930 - acc: 0.5286 - val_loss: 0.6964 - val_acc: 0.5040\n", 1301 | "Epoch 10/50\n", 1302 | " - 5s - loss: 0.6928 - acc: 0.5286 - val_loss: 0.6963 - val_acc: 0.5040\n", 1303 | "Epoch 11/50\n", 1304 | " - 5s - loss: 0.6928 - acc: 0.5286 - val_loss: 0.6962 - val_acc: 0.5040\n", 1305 | "Epoch 12/50\n", 1306 | " - 5s - loss: 0.6928 - acc: 0.5286 - val_loss: 0.6962 - val_acc: 0.5040\n", 1307 | "Epoch 13/50\n", 1308 | " - 5s - loss: 0.6928 - acc: 0.5286 - val_loss: 0.6962 - val_acc: 0.5040\n", 1309 | "Epoch 14/50\n", 1310 | " - 5s - loss: 0.6928 - acc: 0.5286 - val_loss: 0.6961 - val_acc: 0.5040\n", 1311 | "Epoch 15/50\n", 1312 | " - 5s - loss: 0.6927 - acc: 0.5286 - val_loss: 0.6961 - val_acc: 0.5040\n", 1313 | "Epoch 16/50\n", 1314 | " - 5s - loss: 0.6928 - acc: 0.5286 - val_loss: 0.6960 - val_acc: 0.5040\n", 1315 | "Epoch 17/50\n", 1316 | " - 5s - loss: 0.6927 - acc: 0.5286 - val_loss: 0.6959 - val_acc: 0.5040\n", 1317 | "Epoch 18/50\n", 1318 | " - 5s - loss: 0.6927 - acc: 0.5286 - val_loss: 0.6961 - val_acc: 0.5040\n", 1319 | "Epoch 19/50\n", 1320 | " - 4s - loss: 0.6927 - acc: 0.5286 - val_loss: 0.6959 - val_acc: 0.5040\n", 1321 | "Epoch 20/50\n", 1322 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6960 - val_acc: 0.5040\n", 1323 | "Epoch 21/50\n", 1324 | " - 5s - loss: 0.6927 - acc: 0.5286 - val_loss: 0.6958 - val_acc: 0.5040\n", 1325 | "Epoch 22/50\n", 1326 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6957 - val_acc: 0.5040\n", 1327 | "Epoch 23/50\n", 1328 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6958 - val_acc: 0.5040\n", 1329 | "Epoch 24/50\n", 1330 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6957 - val_acc: 0.5040\n", 1331 | "Epoch 25/50\n", 1332 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6957 - val_acc: 0.5040\n", 1333 | "Epoch 26/50\n", 1334 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1335 | "Epoch 27/50\n", 1336 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6958 - val_acc: 0.5040\n", 1337 | "Epoch 28/50\n", 1338 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6957 - val_acc: 0.5040\n", 1339 | "Epoch 29/50\n", 1340 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6957 - val_acc: 0.5040\n", 1341 | "Epoch 30/50\n", 1342 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1343 | "Epoch 31/50\n", 1344 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1345 | "Epoch 32/50\n", 1346 | " - 5s - loss: 0.6926 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1347 | "Epoch 33/50\n", 1348 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1349 | "Epoch 34/50\n", 1350 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6955 - val_acc: 0.5040\n", 1351 | "Epoch 35/50\n", 1352 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1353 | "Epoch 36/50\n", 1354 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1355 | "Epoch 37/50\n", 1356 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6956 - val_acc: 0.5040\n", 1357 | "Epoch 38/50\n", 1358 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6955 - val_acc: 0.5040\n", 1359 | "Epoch 39/50\n", 1360 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6955 - val_acc: 0.5040\n", 1361 | "Epoch 40/50\n", 1362 | " - 5s - loss: 0.6925 - acc: 0.5286 - val_loss: 0.6953 - val_acc: 0.5040\n", 1363 | "Epoch 41/50\n", 1364 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6954 - val_acc: 0.5040\n", 1365 | "Epoch 42/50\n", 1366 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6955 - val_acc: 0.5040\n", 1367 | "Epoch 43/50\n", 1368 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6955 - val_acc: 0.5040\n", 1369 | "Epoch 44/50\n", 1370 | " - 5s - loss: 0.6923 - acc: 0.5286 - val_loss: 0.6954 - val_acc: 0.5040\n", 1371 | "Epoch 45/50\n", 1372 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6954 - val_acc: 0.5040\n", 1373 | "Epoch 46/50\n", 1374 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6955 - val_acc: 0.5040\n", 1375 | "Epoch 47/50\n", 1376 | " - 5s - loss: 0.6923 - acc: 0.5286 - val_loss: 0.6954 - val_acc: 0.5040\n", 1377 | "Epoch 48/50\n", 1378 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6954 - val_acc: 0.5040\n", 1379 | "Epoch 49/50\n", 1380 | " - 5s - loss: 0.6924 - acc: 0.5286 - val_loss: 0.6954 - val_acc: 0.5040\n", 1381 | "Epoch 50/50\n", 1382 | " - 5s - loss: 0.6923 - acc: 0.5286 - val_loss: 0.6953 - val_acc: 0.5040\n" 1383 | ] 1384 | }, 1385 | { 1386 | "data": { 1387 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXiU1dn48e+dyUZWIAtb2A37ElZRXAAFEVRwQ6W01rYufetbrUsrbbXVlre2/WmtLVZR0dZ9q4oCCiogKiCg7DsBJWxZIPs6yf374wwSQgIBEkLy3J/rmmtmzpxn5hyMz/08ZxVVxRhjjPcENXQBjDHGNAwLAMYY41EWAIwxxqMsABhjjEdZADDGGI8KbugCnIj4+Hjt1KlTQxfDGGMalZUrV2aqakLV9EYVADp16sSKFSsauhjGGNOoiMg31aVbE5AxxniUBQBjjPEoCwDGGONRjaoPwBhjTlRZWRlpaWkUFxc3dFHqXXh4OElJSYSEhNQqvwUAY0yTlpaWRnR0NJ06dUJEGro49UZVycrKIi0tjc6dO9fqGGsCMsY0acXFxcTFxTXpkz+AiBAXF3dCdzoWAIwxTV5TP/kfcqL19EYA2DQbVj7f0KUwxpgzijcCwKqX4YOpkP1tQ5fEGOMx2dnZPPHEEyd83Lhx48jOzq6HEh3mjQAw9k/u+YOpDVsOY4zn1BQAysvLj3ncnDlzaN68eX0VC/BKAGjeAS78JWx6H7Z82NClMcZ4yH333cf27dtJSUlhyJAhjBw5ksmTJ9O3b18AJk6cyKBBg+jduzczZsz47rhOnTqRmZnJzp076dmzJzfffDO9e/dmzJgxFBUV1UnZvDMMdNjPYNUrMOde6HwBhDRr6BIZY06zB99bz4Y9uXX6nb3axvC7y3vX+PnDDz/MunXrWLVqFQsXLmT8+PGsW7fuu6GaM2fOpGXLlhQVFTFkyBCuvvpq4uLijviOrVu38sorr/D0008zadIk3nrrLaZMmXLKZffGHQBAcCiMfwSyv4HFjzZ0aYwxHjV06NAjxuk//vjj9O/fn2HDhrFr1y62bt161DGdO3cmJSUFgEGDBrFz5846KYt37gAAOp8PfSfB549B/+shrmtDl8gYcxod60r9dImMjPzu9cKFC/noo49YsmQJERERjBgxotpx/GFhYd+99vl8ddYE5J07gEPG/BGCw2HOPaDa0KUxxjRx0dHR5OXlVftZTk4OLVq0ICIigk2bNrF06dLTWjbvBYDoVjDqftj+CWx4p6FLY4xp4uLi4hg+fDh9+vTh3nvvPeKzsWPH4vf76devH/fffz/Dhg07rWUTbURXwYMHD9Y62RCmohxmjICCDLh9OYRFn/p3GmPOSBs3bqRnz54NXYzTprr6ishKVR1cNa+3+gAOCfLBZX+DZy6G2XdDx3OhMAsKDwQeWaAVcOVTEBl3/O8zxphGyJsBACBpMAz6Iax8Dta85tJCIiAiDpo1h31rYe0bMOy2Bi2mMcbUF+8GAIBxf4WhN0N4c4hoeeTcgH8Nh/VvWwAwxjRZ3usErswXAq16Q2y7oyeG9Z4Iu5ZCzu6GKZsxxtQzbweAY+l1pXveOKthy2GMMfXEAkBN4s+CVn1dM5AxxjRBFgCOpfdE2LUMctIauiTGmEbqZJeDBnjssccoLCys4xIdVqsAICJjRWSziGwTkftqyDNJRDaIyHoReblS+p9FZF3gcV2l9MUisirw2CMiZ96srN6BZqAN7zZsOYwxjdaZHACOOwpIRHzAdGA0kAYsF5FZqrqhUp5kYCowXFUPikhiIH08MBBIAcKARSIyV1VzVfX8Sse/BZx5Z9m4rtC6n2sGOudnDV0aY0wjVHk56NGjR5OYmMjrr79OSUkJV155JQ8++CAFBQVMmjSJtLQ0ysvLuf/++9m/fz979uxh5MiRxMfHs2DBgjovW22GgQ4FtqlqKoCIvApMADZUynMzMF1VDwKoanogvRewSFX9gF9EVgNjgdcPHSgi0cAo4KZTrEv96D0RPn4IsndB8/YNXRpjzKmYe5+b41OXWveFSx+u8ePKy0HPmzePN998ky+//BJV5YorruDTTz8lIyODtm3bMnv2bMCtERQbG8ujjz7KggULiI+Pr9syB9SmCagdsKvS+7RAWmXdgG4i8rmILBWRsYH01cClIhIhIvHASKDqWfRK4GNVrXaRbhG5RURWiMiKjIyMWhS3jvWa6J6tGcgYc4rmzZvHvHnzGDBgAAMHDmTTpk1s3bqVvn378tFHH/GrX/2KxYsXExsbe1rKU5s7gOq2ma+6gFAwkAyMAJKAxSLSR1XnicgQ4AsgA1gC+KscewPwTE0/rqozgBng1gKqRXnrVlxXaNPfNQOde/tp/3ljTB06xpX66aCqTJ06lVtvvfWoz1auXMmcOXOYOnUqY8aM4YEHHqj38tTmDiCNI6/ak4A91eR5V1XLVHUHsBkXEFDVaaqaoqqjccHku90ORCQO18Q0++SrcBr0vhJ2r4CD3zR0SYwxjUzl5aAvueQSZs6cSX5+PgC7d+8mPT2dPXv2EBERwZQpU7jnnnv46quvjjq2PtQmACwHkkWks4iEAtcDVWdHvYNr3iHQ1NMNSBURX+Akj4j0A/oB8yoddy3wvqoevQPCmcSagYwxJ6nyctDz589n8uTJnHPOOfTt25drrrmGvLw81q5dy9ChQ0lJSWHatGn89re/BeCWW27h0ksvZeTIkfVStlotBy0i44DHAB8wU1WnichDwApVnSUiAjyC6+AtB6ap6qsiEg58FfiaXOA2VV1V6XsXAg+r6ge1KWydLQd9Mp660K0ievMnDfP7xpiTYstBn+Jy0Ko6B5hTJe2BSq8VuCvwqJynGDcSqKbvHVGb3z8j9L4SPvqdawZq0bGhS2OMMafMZgLXVu9DzUBV5qsVHYTPHoPH+rohZsYY00hYAKitFp2g7cDDawMdSIU5v4RHe7s7g4oKWPYk7F3ToMU0xhytMe18eCpOtJ4WAE5E7ythz9fw0rXw+EBYMRN6TYDbPoOffg7NWsCHv7bN5o05g4SHh5OVldXkg4CqkpWVRXh4eK2P8faGMCeq90T4+EFIWw7n3+02k4luffjzEVNh7r2weS70GNdw5TTGfCcpKYm0tDQaZCLpaRYeHk5SUlKt83tzU/hTkbXdnfRDI4/+rLwM/nWu23T+f5ZCcOjpL58xxlRR0yggawI6UXFdqz/5g9thbMw0OLAdVjx7estljDEnyAJAXUseDV1GwsKHofBAQ5fGGGNqZAGgronAJdOgJBcW/aWhS2OMMTWyAFAfWvWGgT+A5U9D5tbj56+qIKvuy2SMMVVYAKgvI38Dwc1g/gms6LfrS/j3FfDXLrDy3/VXNmOMwQJA/YlKhPPvgs1zIHXRsfPu/gpevAaeHQ3717vN6Of+EvatOz1lNcZ4kg0DrU9lxTB9CBRkQmJPSOgBCd3dc3w3KMlzncWbZ7tJZMPvgCE3Q1khPHkehMXALQshLKqha2KMacRqGgZqAaC+7V/vmnMyNkHGZsjfd+TnYbFuo5mzb4PwmMPpqYvgPxOg33Vw5ZOuc9kYY07CKa0Gak5Bq94wrtJooKKDkLHFBYSyIuh/nbv6r6rLhXDhr2DRw9D5fBgw5fSV2RjjCRYATrdmLaDD2e5xPBf+Er75HGbfA+0GuWYkY4ypI9YJfCYL8sHVz7o+gDd+CKUFDV0iY0wTYgHgTBfdCq562vUfzLm3oUtjjGlCLAA0Bl1HwgX3wKqXYPbdkLO7oUtkjGkCrA+gsbjwPijIgBXPuVFFKZPdsNG4rg1dMmNMI2V3AI2FLxgu/zv8/GsYdCOsfhX+ORje/LEbamqMMSfI5gE0Vnn7Yel0WP4slOZDx+HQ7RJIvsRNNrN5A8aYAJsI1lQVHYTlz8D6d2B/YOmI5h0geYx7JA1x+xf4Qi0oGONRFgC8IGc3bJ0HW+dD6kIoqzJsNDgcgsPAFwaR8W72ccpkt5GNMabJsgDgNf4S2PkZpG+E8hL33l98+HnPKti7Clp0dnsZ973GzTswxjQ5FgDMkVRhywfwyTTYvxbiu8PIX0PPKyDIxgYY05TYnsDmSCLQ/VK49VO49nlA4Y0bYcaF8O2yhi6dMeY0sADgdUFB0PtK+J+lcOUMKMqGmZfA3PuOv/REeRmsfRM2vufuKIwxjYpNBDNOkM+tTNpjPHz8ICz7F2yZC1f8AzpfcGTe0kL46j/wxT8gN82ldTwPxv8/W7DOmEbE7gDMkcKiYNxf4YdzQILg35fD+79wm9cUHYRFf4XH+sAHv4Lm7WHyG3DZY24I6pPnwbzfQkn+if9uRbm7kyjOqfs6GWOqVatOYBEZC/wd8AHPqOrD1eSZBPweUGC1qk4OpP8ZGB/I9gdVfS2QLsAfgWuBcuBfqvr4scphncCnWWkhLJgGS6ZDdGt3Yi/Nc/MLzrsLOp5zOG9BFnz0O/j6BYhpB5f8H/SaULu5B/kZ8NaPYccidyfx/bchOLT+6mWMx5z0KCAR8QFbgNFAGrAcuEFVN1TKkwy8DoxS1YMikqiq6SIyHrgTuBQIAxYF8uSKyE3ASOCHqlpx6JhjlcUCQAPZtRw++r0LAufdCa37HiPvlzD7Lti3FrqMgFEPQNKgY+d//UYoOgD9r4eVz0PKFJjwz+MHj4oK0HKbx2DMcZzKjmBDgW2qmhr4oleBCcCGSnluBqar6kGASifyXsAiVfUDfhFZDYzFBYufApNVtaLKMeZM034I3DS7lnmHws0L3ezkRX+GZ0a55SlGToW2Aw7nU4VlT8G830BsEvx4HrTpD1Gt3S5ocV3h/Ltq/p0DO+C174O/CH442wUnY8wJqU0fQDtgV6X3aYG0yroB3UTkcxFZGmgyAlgNXCoiESISj7vibx/4rCtwnYisEJG5gbuIo4jILYE8KzIyMmpbL9OQfMEw7Da4cw1c9ADsWgYzRsArN8De1a4p6c0fuX6Es0a7je/b9HfHjrgP+lzjOqLXv1P992//xH1fzreQuxf+M9E1QRljTkht7gCquw+v2m4UDCQDI4AkYLGI9FHVeSIyBPgCyACWAP7AMWFAsaoOFpGrgJnA+Uf9kOoMYAa4JqBalNecKcKi4fy7YcjN7mp/yT/gqQsgMgEKs+Ci38HwO4+ceCYCE6ZDzi54+1aIbX+4CUkVvnjcNUcl9IDrX4LsXfDStfDilXDjexAe2yBVNaYxqs0dQBqHr9rBneD3VJPnXVUtU9UdwGZcQEBVp6lqiqqOxgWTrZWOeSvw+m2g38lVwZzxwmPgwnvhzrVu2YmYdvD9d1wTT3WzjkPC4fqXIaoVvHK9O8mXFriO4vkPuNnKP54PLbtAlwvhuhfcktgvTbJtM405AbUJAMuBZBHpLCKhwPXArCp53sE17xBo6ukGpIqIT0TiAun9cCf5eZWOGRV4fSGuo9k0ZeGxronn1kXuxH0skfEw+XW3dtHLk+DZMbDuv3Dx793M5bCow3m7XQJXPwNpX8Krk6GsuB4rYUzTcdwmIFX1i8jtwIe4YaAzVXW9iDwErFDVWYHPxojIBtyQzntVNUtEwnHNQQC5wJRAhzDAw8BLIvILIB/4SV1XzjRyiT1g0r/hxavdCf97b0LyxdXn7X0llBXBOz+FN37o7gpsdJAxx2SLwZkz3+6VrjkoNun4eb98GubcA72vgqtmWBAwhlMbBmpMw2p3jHkEVQ29GcoKXV9BWSFc8xyERtRf2YxpxGwpCNP0DL8Dxj8CWz6EF69yC9wdi6q7yyg8UL/lKsmr3+835gRZADBN05CfwDUzIW0FPDcO8vZVny9tBcwcC0+Pgsf6wvzfuaUp6pIqfP53+FN7mDHSzXa2YGDOABYATNPV5yr43htwcKcbRZS1/fBnB3bAGzfBMxfBwR1wyZ+g21g3z+Cxvm457Jzdp16Gcr9bTG/+A9B1pNuN7b074P91h1n/6+48GlE/nGlarBPYNH27V8KL10BQMFzzrGsa+nKGe3/uz+Hc/z08rDRzG3z2N1jzqlsNNWUydB/vJrWFRbnn0MDr4LBj/25JnhuRtO0jOO8Xbl0kEXfXsfJ5WP9f10/Rqi+07uP2bA6JgJBmbi5ESIRbivtYay8ZUwu2JaTxtowt8MKVgf0LBAZ8D0b+BmLaVp//4Deu2ebrF6C8tPo80W2h79XQ73p3Aq8sZze8fB2kb4DLHoVBPzz6+OIct6HO6lddE1VZobtDKCsEt0QW+EJh4r/cns3GnCQLAMbk7IalT0D/G44+YdekINM1IZXkujWMSvKgNN+9T1sJ2+ZDhR9a9YF+17kTdUGmm7xWkg+Tnoezapi7UBNVt9taQQa89RP49gsYdb9bVqM2y2sbU4UFAGPqQ0EmrH/bXcXvXgGIu2o/NJO5toGmJv4SePdnsPYNGPB9uOxvNc9tqKhwayj5QgLNSBGuLBY0PM/mARhTHyLj3dyDoTe7TuY1r0FOmrtij2lz6t8fHAZXPQ0tOsOnf3En+En/ObzonarrU1j/Nmx4B3KrdFxLkAsEoZHQuh90PBc6DndLc9umO55ndwDGNBZfv+hGEMUlw5g/QuoC2PBu4Ko/1DU1nXWxO+mXFbm+hLIi9yjOdoEic7P7ruBmkDTYBYOel9W+ozl9I+z+CiLi3KqukYHn0Ej3ebkf8ve7QJST5p7z090+Ecljjt9xbuqFNQEZ0xSkLoTXfgAlORAUAmdd5NZB6n5p7ZbCzs+Ab5fAN1/AN5+7ndtQaDcYBt/kltCoOnO6rMgFmhXPwa6l1X9vSIQbIVWQ6XZpq0x8Li28uStrv+ugwzBrmjqNLAAY01Qc2OE21ukyApo1P7XvKjzg+i9WPgeZWyAsFvpfB4NucsNkVz4Pq15ydxAtu7og0W2s6wQvyHQd1YeeS3IhMhFi20FMUuC5rRs2m7rQDa3d+L7bxa15R+g3yXWeV6dtCrTodGp1q66u3y51gW/XMncnNfpBiEqs2985A1kAMMbUTNXdFax8zl3tHxr6GhTimogG3eTmJJzqVXtJngsCa16DHYsOD3etKiwWfvoZNO9w8r9VWuhGae1Y7OqWvt6l+8LcDnR7V7k7lzF/cB3sTfiOxAKAMaZ2CrLcCVrLXXNNfV0h52e4O4eqig66ORSterv9nn0nMFbFXwLbPoZ1b8HmuVBWACGR0OHsSh3gA91Eu4zN8N6dbphtx/Pg8scgvtqdaWtWUeH6WirvT3EGsgBgjGk81rwB//2J20FuxH3HzqvqOsTXvunuLkpyoFlL6DXBLQfS4dyag0hFhZvsN/9+19dx/j1uMcGyQtd5XZABBemumSs/3XVwV34uSHfzQBJ7Q4/x7tGm/xl3N2EBwBjTuPz3Vlj7Otw013UaV6fcD3Pudn0VYTHQ83J30u984YntBZG3Hz6c6u4caiJBro8jKtHtTxHVyr0OaQapi9ydhFa4/o8e413HfGjk4dFQuXsCr/dAdGvoe63rTwkJP6F/lpNhAcAY07gU58JT50NFOdz22dEd3qWF8OaPYMtct9bShfed+sl028euvyAyPjDMNfCISoRmLSDIV/OxBVmw5QPYNBu2f+yW9agsJNJ1jEe3cc1P+ftcX0evK1xTW8fh1e+RXQcsABhjGp+0lTBzjLuyv+a5w00rBVnwynVubsO4v7qJeGeS0gLY+bkrb0xbiGnnhukeKn9FOez4FNa8DhtnueVFYpLcyKiBP4CWneu0OBYAjDGN0+JH4OOHYMJ0GDDFrc30wlWuWeXqZ1xwaMxKC1yH9epX3Z2DKnQdBYN/5JqITqQTvAYWAIwxjVNFOfxngpuBPOGfMPdXbpjq5Ndq7htorHLS4KsX4Kv/QN4e11w08AfuUZs9sWtgAcAY03jl7IYnh7shorHtYcpbkNC9oUtVf8r9sPVDN/t620eu6egnH0O7gSf1dbYYnDGm8YptB1c/60b7XPqXullo70zmCz48rPTgN7DuTTe8tI7ZHYAxxjRxNd0B2J7AxhjjURYAjDHGoywAGGOMR1kAMMYYj7IAYIwxHmUBwBhjPKpWAUBExorIZhHZJiLVrs0qIpNEZIOIrBeRlyul/1lE1gUe11VKf15EdojIqsAj5dSrY4wxpraOOxFMRHzAdGA0kAYsF5FZqrqhUp5kYCowXFUPikhiIH08MBBIAcKARSIyV1VzA4feq6pv1mmNjDHG1Ept7gCGAttUNVVVS4FXgQlV8twMTFfVgwCqmh5I7wUsUlW/qhYAq4GxdVN0Y4wxp6I2AaAdsKvS+7RAWmXdgG4i8rmILBWRQyf51cClIhIhIvHASKB9peOmicgaEfmbiIRV9+MicouIrBCRFRkZ1WwfZ4wx5qTUJgBUt7dZ1fUjgoFkYARwA/CMiDRX1XnAHOAL4BVgCeAPHDMV6AEMAVoCv6rux1V1hqoOVtXBCQkJtSiuMcaY2qhNAEjjyKv2JGBPNXneVdUyVd0BbMYFBFR1mqqmqOpoXDDZGkjfq04J8ByuqckYY8xpUpsAsBxIFpHOIhIKXA/MqpLnHVzzDoGmnm5Aqoj4RCQukN4P6AfMC7xvE3gWYCKw7tSrY4wxpraOOwpIVf0icjvwIeADZqrqehF5CFihqrMCn40RkQ1AOW50T5aIhAOL3TmeXGCKqh5qAnpJRBJwdwWrgNvqunLGGGNqZstBG2NME2fLQRtjjDmCBQBjjPEoCwDGGONRFgCMMcajLAAYY4xHWQAwxhiPsgBgjDEeZQHAGGM8ygKAMcZ4lAUAY4zxKAsAxhjjURYAjDHGoywAGGOMR1kAMMYYj7IAYIwxHmUBwBhjPMoCgDHGeJQFAGOM8SgLAMYY41EWAIwxxqMsABhjjEdZADDGGI+yAGCMMR5lAcAYYzzKAoAxxniUBQBjjPEoCwDGGONRFgCMMcajLAAYY4xH1SoAiMhYEdksIttE5L4a8kwSkQ0isl5EXq6U/mcRWRd4XFfNcf8QkfyTr4IxxpiTEXy8DCLiA6YDo4E0YLmIzFLVDZXyJANTgeGqelBEEgPp44GBQAoQBiwSkbmqmhv4fDDQvI7rZIwxphZqcwcwFNimqqmqWgq8CkyokudmYLqqHgRQ1fRAei9gkar6VbUAWA2Mhe8Cy1+BX556NYwxxpyo2gSAdsCuSu/TAmmVdQO6icjnIrJURMYG0lcDl4pIhIjEAyOB9oHPbgdmqereky++McaYk3XcJiBAqknTar4nGRgBJAGLRaSPqs4TkSHAF0AGsATwi0hb4NpA/mP/uMgtwC0AHTp0qEVxjTHG1EZt7gDSOHzVDu4Ev6eaPO+qapmq7gA24wICqjpNVVNUdTQumGwFBgBnAdtEZCcQISLbqvtxVZ2hqoNVdXBCQsIJVM0YY8yx1CYALAeSRaSziIQC1wOzquR5B9e8Q6CppxuQKiI+EYkLpPcD+gHzVHW2qrZW1U6q2gkoVNWz6qZKxhhjauO4TUCq6heR24EPAR8wU1XXi8hDwApVnRX4bIyIbADKgXtVNUtEwnHNQQC5wBRV9ddXZYwxxtSeqFZtzj9zDR48WFesWNHQxTDGmEZFRFaq6uCq6TYT2BhjPMoCgDHGeJQFAGOM8SgLAMYY41EWAIwxxqMsABhjjEdZADDGGI+yAGCMMR5lAcAYYzzKAoAxxniUBQBjjPEoCwDGGONRFgCMMcajLAAYY4xHWQAwxhiPsgBgjDEeZQHAGGM8ygKAMcZ4lAUAY4zxKAsAxhjjURYATpCqNnQRjDGmTng6AOzOLuK91Xso8ZcfN2+pv4KnFm1nwB/m8+Si7aehdMYYU7+CG7oADeXTLRn8/NWvyS4so3VMOLdd2IXrh3YgPMR3VN4Fm9J56P0N7MgsoF3zZjw8dxOtY8KZOKBdA5TcGGPqhufuACoqlOkLtnHjc1/SOiacv1+fQoeWEfz+vQ2c/5cFPP1pKoWlfgBSM/K56bkvuen55YjA8zcNYcE9IzinSxz3vrmaL7ZlNnBtjDHm5EljatMePHiwrlix4qSPzysu4+7XVzNvw36u6N+Wh6/uS0SouwlamprF4x9v5YvtWbSMDOWC5Hhmr91LWLCPOy5K5sZzOxEa7OJlTlEZ1z75BXtzinnztnPp3jq6TupnjDH1QURWqurgo9K9EgC27s/j1hdW8s2BQn49ric/Gt4JETkq38pvDvD4x9v4dGsG1w5K4t5LepAQHXZUvt3ZRVw5/XOCg4S3fzacVjHhJ1UuY4ypb54OALPX7OXeN1cTERrM9MkDOLtL3HGPKa9QfEFHB4jK1u/JYdKTS+gQF8nrtw4jOjzkhMtmjDH1raYA0OT7AFSV91bvoXvraN7/3/NqdfIHjnvyB+jdNpYnpgxiy/48/uelrygrrzjV4hpjzGnjiTuAghI/Ib6g79rw69rry3fxy7fW0KFlBP2SYunVNobebWPp1SbmiOaj4rJy0nNL2J9XzP7cYiJCfYzsnlhtU5QxxtSVmu4APDEMNDKsfqs5aUh7gn3CB+v2sWpXNu+v2fvdZwnRYbSMCGV/XjHZhWVHHXvHRcn8YnS3ei2fMcZUp1ZnRhEZC/wd8AHPqOrD1eSZBPweUGC1qk4OpP8ZGB/I9gdVfS2Q/iwwGBBgC/BDVc0/pdo0oKsGJnHVwCQAcgrL2LA3lw17c1m/J4e8Yj9DOregdUw4iTHhtIoJp3VMOM8sTuXvH28lKiyYmy/o0sA1MMZ4zXEDgIj4gOnAaCANWC4is1R1Q6U8ycBUYLiqHhSRxED6eGAgkAKEAYtEZK6q5gK/CDwjIo8CtwNHBZbGKDYihHO6xnFO12P3Nzx8dT8KS8uZNmcjkWHBTD67w2kqoTHG1K4TeCiwTVVTVbUUeBWYUCXPzcB0VT0IoKrpgfRewCJV9atqAbAaGBvIc+jkL0Az3J2Dp/iChL9dl8KoHon85p21vLtqd415t6Xn86c5G1n5zYHTWEJjTFNWmwDQDthV6X1aIK2ybkA3EflcRJYGmozAnfAvFZEIEYkHRgLtDx0kIs8B+4AewD+q+3ERuUVEVojIioyMjFpVqjEJDQ7iie8N5OzOLbnr9dXMW7/vu89UlS+2ZfKj55dz8aOLeOrTVG54ehlz1+49xjcaY5Bh3loAABB0SURBVEzt1CYAVDdEperVejCQDIwAbgCeEZHmqjoPmAN8AbwCLAH8332J6k1AW2AjcF11P66qM1R1sKoOTkhIqEVxG5/wEB/P3DiEvu1iuf3lr1mwKZ23VqYx7vHPmPzMMtakZfOLi7vx0V0X0qdtDP/z8lc8//mOhi62MaaRq00ASKPSVTuQBOypJs+7qlqmqjuAzbiAgKpOU9UUVR2NCyZbKx+oquXAa8DVJ1eFpiEqLJjnbxpCl4RIbnp+OXe/sZryigr+cnU/PvvVKO64OJmzEqN46SfDuLhnK37/3gb+NHcjFRWeazkzxtSR2owCWg4ki0hnYDdwPTC5Sp53cFf+zweaeroBqYEO5OaqmiUi/YB+wLxAu39XVd0WeH05sKluqtR4NY8I5YUfn80TC7cxonsiFyTHHzVHoFmojyenDOKBd9fx1KJU9ucU85dr+tfbHAdjTNN13ACgqn4RuR34EDcMdKaqrheRh4AVqjor8NkYEdkAlAP3Bk764cDiwEksF5gS+L4g4N8iEoO7K1gN/LQ+KtjYJESH8bvLex8zjy9I+OPEPrRt3oy/friZzPxSnpgykJg6WIpiX04xLSNDLaAY4wGemAnclL2xYhdT/7sWX5AwoENzzu4cx9ldWjKwQ4tq9zaojqqycHMGTy7azrIdBwj1BdGzbQwpSbH0S2pO//bN6RIfSVAtlscwxpx5PL0YXFP39bcHeW/1XpbtyGLD3lxUIdQXRP/2sQzp1DJwEo+ldUz4EU1KZeUVvLd6D08tSmXz/jzaxIYzeWgH8kv8rNqVzbrdORSUut3SosOCuWl4J+64uFut1kkyxpw5PL0URFM3oEMLBnRoAbi9ClbsPMCyHQdYlprFU5+mUh7oKI6PCqN/Uix9k2IJC/bxwpKd7MkpplurKB65tj+X9297RNNPeYWyPSOfVbuyWbApncc/2cbXu7L5+/UDaBkZeswyLdqSQWpGPtcObk9UPS/FYYw5OXYH0MQVl5WzYW8ua9NyWJ2WzZq0HLZn5KMKQzu35LYLu9R6QbpXv/yWB2atJyEqjH9NGUi/pOZH5dmWns8fZ29g4WY3ZyMhOoxfXtKdqwcmWROSMQ3EmoDMd/JL/BzIL6VDXMQJH7smLZufvvgVGfkl/GFCb64b4pavyCks47GPt/DCkm9oFuLj5xclM6BDc/44eyOrdmXTPymWBy7vzaCOLeq6OtUqK68gxGcd2caABQBThw4UlHLHq1+zeGsm1w1uT592MTw6fws5RWVcN6QDd4/pRnyUWwa7okJ5d/VuHp67if25JUxMacs9l3QnSIS9OUXszSlmb3Yxe3KKSM8tQVGCRAgOEnxBQfiCINgXxOherRjZPfG4ZSso8fOrt9awYFM6P78omR+d19kCgfE8CwCmTpVXKH+bv4V/LtgGwLAuLXngst70ahtTbf6CEj//WridGYtTKfUfvXFORKiP1jHh+IKEclXKKxR/uXsuKPWTV+znB+d05NfjetY4umlnZgG3vrCSrel5pLRvzlffZpOcGMVDE/ocd2E+Y5oyCwCmXny2NZMSfzmjetSuH2HXgULeW7OH5s1CadM8nLaxzWgdG05MeHCNx5f4y/nLB5t59rMddG8VzeM3DKB76+gj8izYnM4dr3xNUJDwjxsGcH5yAh9t2M/v31tP2sEiJqS05TfjepJoezcbD7IAYBq9hZvTueeN1eQW+/nt+J58f1hHAKYv2MYj87fQo3UMM74/iPYtD/dtFJeV88SCbTy5KJXQ4CDuvDiZawYl0Tzi2KOYTlR5hbJq10EWbMpgb04xPx3RlbMSo+r0N4w5WRYATJOQkVfCvW+uZuHmDC7qkUiIL4gP1u/jiv5t+fPV/WgWWnPz0O9mrWfRlgyCBAZ1bMHIHomM6pFI91bRR9x9qCoZeSVsy8hnR2YBFRVKfFQYcVFhxEeFEhcVRkx4MDlFZSzaksEnm9JZtCWD7MIyfEFCWHAQ/nLlZyPP4rYRXQgLrt2EPGPqiwUA02SoKs99vpOH527CX1HBr8f15MfndT5uE5Sqsjoth0827ueTzems250LQNvYcC7olkCJv4LUjHxSMwrIK/Ef87tCg4Pwl1dQoRAXGcqF3RMY2T2RC5ITKCkv5w/vb+S91XvomhDJn67qx9DOLeus/idDVW3vaQ+zAGCanG3p+RSXldOnXexJHb8/t5iFm9P5ZFM6X2zLIio8mK4JUXRNiKRLQhRdE6LokhBJsE/IzCslq6CEzPwSsvJLycwvJTwkiBHdE+nXLrbaOQ4LNqfz27fXsTu7iBuGtue+sT2JjXDrNVUEOrdzi/3kFZcRFRZMYnR4na/BVFxWzoxPU5n5+Q6uH9KBu0Z3s3WePMgCgDENoLDUz9/mb+HZz3YQHR5CTLNgcovcSb+6lbxbRoaSGB1G69hwWkWH4/MJuUVl5Bb7A89l5Bb5iQkP5upBSUwa3J6E6LCjvkdVeX/NXh6eu4nd2UX0bRfL2t059EuK5e/XD6BzfORpqL05U1gAMKYBrdudw9OLUwkSISY8mJhmIcSEhxDbLISo8GDyisvYn1vC/tziwKOEfbnFVFQosc1CiG4WcsRxOzLzWZp6gBCfMKZ3a753dgfO6RKHiLBudw4Pvree5TsP0qtNDA9c3othXeL4YN0+7vvvGkr9Ffz+it5cOyip2mahnZkFfLB+Hz4RrhrYjrioowOMaVwsABjTxGxLz+eVL7/lzZVp5BSV0SU+kp5tYpizbi8tI0K555LuTBrc/ojF+/bmFHHXa6tZkprF+H5t+L+JfYmNCCE1I585a/cyZ+0+NuzN/S5/qC+Iy/q1Yco5HRnQvnmD9CPszi5izpq9zF67l51ZBYzu2YqJA9oxrEucLUxYSxYAjGmiisvKmbN2Ly8t+5a1aTnceG5H/vei5Br3hyivUGZ8msoj8zaTGB1GTLMQNu3LA2Bgh+aM69uGS/u2obDEz4tLv+Gtr3aTX+Knd9sYvj+sI1ektCUitH4X+NudXcTcte6k//W32QD0bhtD14QoPtmUTn6Jn1YxYUxIaceElLb0ahODiKCqHCwsIz2vmPTcErIKSmgZGUaX+EjaNm9W5wHDX17BtwcK6RR3Zi+XbgHAGA84kdE+q3dlc/+76wjxBbmTfp/WtG3e7Kh8+SV+3vl6Ny8s+YbN+/OICPVxfnI8F/VoxcgeidX2QdSkqLSc7Rn5bE3PY+v+fDLzS8gvcTO98wId4nnFftLzSgB30h/Xtw3j+7ahU6DforisnI827uedr/ewcHM6/gqlfctmlJcrGfkllJVXf04LDQ6ic1wkXRIi6RwfyZBOLTkvOf6ElwopK69gyfYs5q7bx7z1+8gqKOXcrnE8Mqk/bWKP/vc7E1gAMMacElVl+c6DvLtqN59sSmdvTjEA/ds35+IeiZx7VhzlFZBX7Dqr8wId19mFZezILGBrej67DhZy6JQTHCTERYUSHR5CVFgw0eHBxARed4yPYFyfwyf9mhwsKGX22r0s3ppBVFgIiTFhJEaHkRgdTkJ0GHFRoWTmlZCaWcCOzILvhvl+e6AQf4XSMjKUy/q1YUJKOwZ2qLmJ60BBKat2HWTu2n3M27CfnKIyIkN9jOrZirMSonjq0+0EBwl/vLIvV/RvW2N5M/JKeHnZt2zal0txWTkl/gqKy8opLqug2F9OfGQYPz6/M6N7tqrTOwoLAMaYOqOqbNibyycb0/l4Uzqr07Kp6VTSLMRHx7gIzkqMIjkxmuRWUSQnRtExLrLBhqSW+Mv5dEsm76zazUcb9lPir6B9y2ZM6N+OPu1i2ZnlgsX2DPd8sLAMcBsjXdyrFZf2ac0F3RK+W5dqZ2YBd762ilW7spmY0pYHJ/QhttnhJrjN+/J49rNU3vl6D2UVFXSJj6RZqI/wYB/hIT7CgoMID/Gxbk8O32QV0qN1NLePOotL+7Spk2YrCwDGmHqTkVfCql3ZNAvxEdMs2A15DXfPZ/q8g7ziMj5cv593V+3m822Z3w3PjY8KqzQnJJLuraMZ2rlljTO7/eUVTF+wncc/2Uqr6DAemZRCaXkFzyxOZfHWTMJDgrh2UHtuGt6JLgnVLxPiL6/gvTV7+Ocn29ieUUDXhEh+NvIsrujfluBTWNXWAoAxxhxHep5bnrxTfOQRV/AnYtWubH7x2ip2ZBYAkBgdxo3ndmLy0A60OM5OeoeUVygfrNvHPz7ZyqZ9eXRoGcGMHwyiR+vqV9s9HgsAxhhzmhSW+vnPkm9IiAo7aqvVE1FRoXy0cT8vLvuWJ6cMPOnRVxYAjDHGo2oKAGd245wxxph6YwHAGGM8ygKAMcZ4lAUAY4zxKAsAxhjjURYAjDHGoywAGGOMR1kAMMYYj2pUE8FEJAP45iQPjwcy67A4jYXV21u8Wm/wbt1rU++OqppQNbFRBYBTISIrqpsJ19RZvb3Fq/UG79b9VOptTUDGGONRFgCMMcajvBQAZjR0ARqI1dtbvFpv8G7dT7renukDMMYYcyQv3QEYY4ypxAKAMcZ4lCcCgIiMFZHNIrJNRO5r6PLUFxGZKSLpIrKuUlpLEZkvIlsDzy0asoz1QUTai8gCEdkoIutF5I5AepOuu4iEi8iXIrI6UO8HA+mdRWRZoN6viUjt9iFsZETEJyJfi8j7gfdNvt4islNE1orIKhFZEUg76b/zJh8ARMQHTAcuBXoBN4hIr4YtVb15HhhbJe0+4GNVTQY+DrxvavzA3araExgG/Czw37ip170EGKWq/YEUYKyIDAP+DPwtUO+DwI8bsIz16Q5gY6X3Xqn3SFVNqTT2/6T/zpt8AACGAttUNVVVS4FXgQkNXKZ6oaqfAgeqJE8A/h14/W9g4mkt1GmgqntV9avA6zzcSaEdTbzu6uQH3oYEHgqMAt4MpDe5egOISBIwHngm8F7wQL1rcNJ/514IAO2AXZXepwXSvKKVqu4Fd6IEEhu4PPVKRDoBA4BleKDugWaQVUA6MB/YDmSrqj+Qpan+vT8G/BKoCLyPwxv1VmCeiKwUkVsCaSf9d35yW8w3LlJNmo19bYJEJAp4C7hTVXPdRWHTpqrlQIqINAfeBnpWl+30lqp+ichlQLqqrhSREYeSq8napOodMFxV94hIIjBfRDadypd54Q4gDWhf6X0SsKeBytIQ9otIG4DAc3oDl6deiEgI7uT/kqr+N5DsiboDqGo2sBDXB9JcRA5d3DXFv/fhwBUishPXpDsKd0fQ1OuNqu4JPKfjAv5QTuHv3AsBYDmQHBghEApcD8xq4DKdTrOAGwOvbwTebcCy1ItA+++zwEZVfbTSR0267iKSELjyR0SaARfj+j8WANcEsjW5eqvqVFVNUtVOuP+fP1HV79HE6y0ikSISfeg1MAZYxyn8nXtiJrCIjMNdIfiAmao6rYGLVC9E5BVgBG552P3A74B3gNeBDsC3wLWqWrWjuFETkfOAxcBaDrcJ/xrXD9Bk6y4i/XCdfj7cxdzrqvqQiHTBXRm3BL4GpqhqScOVtP4EmoDuUdXLmnq9A/V7O/A2GHhZVaeJSBwn+XfuiQBgjDHmaF5oAjLGGFMNCwDGGONRFgCMMcajLAAYY4xHWQAwxhiPsgBgjDEeZQHAGGM86v8DennqJoDhQVQAAAAASUVORK5CYII=\n", 1388 | "text/plain": [ 1389 | "
" 1390 | ] 1391 | }, 1392 | "metadata": { 1393 | "needs_background": "light" 1394 | }, 1395 | "output_type": "display_data" 1396 | } 1397 | ], 1398 | "source": [ 1399 | "# The GRU architecture\n", 1400 | "modelGRU = Sequential()\n", 1401 | "# First GRU layer with Dropout regularisation\n", 1402 | "modelGRU.add(GRU(units=50, return_sequences=True, input_shape=(X_train.shape[1],X_train.shape[2]), activation='tanh'))\n", 1403 | "#print(modelGRU.get_input_at())\n", 1404 | "modelGRU.add(Dropout(0.2))\n", 1405 | "print(modelGRU)\n", 1406 | "# Second GRU layer\n", 1407 | "modelGRU.add(GRU(units=50, return_sequences=True, input_shape=(X_train.shape[1],X_train.shape[2]), activation='tanh'))\n", 1408 | "modelGRU.add(Dropout(0.2))\n", 1409 | "# Third GRU layer\n", 1410 | "modelGRU.add(GRU(units=50, return_sequences=True, input_shape=(X_train.shape[1],X_train.shape[2]), activation='tanh'))\n", 1411 | "modelGRU.add(Dropout(0.2))\n", 1412 | "# Fourth GRU layer\n", 1413 | "modelGRU.add(GRU(units=50, activation='tanh'))\n", 1414 | "modelGRU.add(Dropout(0.2))\n", 1415 | "# The output layer\n", 1416 | "modelGRU.add(Dense(units=2))\n", 1417 | "modelGRU.add(Activation('sigmoid'))\n", 1418 | "modelGRU.summary()\n", 1419 | "# Compiling the RNN\n", 1420 | "modelGRU.compile(optimizer=SGD(lr=0.01, decay=1e-7, momentum=0.9, nesterov=False),loss='binary_crossentropy',metrics=['accuracy'])\n", 1421 | "# Fitting to the training set\n", 1422 | "history=modelGRU.fit(X_train,y_train,epochs=50,batch_size=150,validation_data=(X_test, y_test), verbose=2, shuffle=False)\n", 1423 | "#history=model2.fit(train_X,train_y,epochs=50,batch_size=32, validation_data=(test_X, test_y), verbose=2, shuffle=False)\n", 1424 | "pyplot.plot(history.history['loss'], label='train')\n", 1425 | "pyplot.plot(history.history['val_loss'], label='test')\n", 1426 | "pyplot.legend()\n", 1427 | "pyplot.show()\n" 1428 | ] 1429 | }, 1430 | { 1431 | "cell_type": "code", 1432 | "execution_count": 32, 1433 | "metadata": {}, 1434 | "outputs": [ 1435 | { 1436 | "name": "stdout", 1437 | "output_type": "stream", 1438 | "text": [ 1439 | "Accuracy: 50.40%\n" 1440 | ] 1441 | } 1442 | ], 1443 | "source": [ 1444 | "# evluation on the test set\n", 1445 | "scores = modelGRU.evaluate(X_test, y_test, verbose=0)\n", 1446 | "print(\"Accuracy: %.2f%%\" % (scores[1]*100))" 1447 | ] 1448 | }, 1449 | { 1450 | "cell_type": "code", 1451 | "execution_count": null, 1452 | "metadata": {}, 1453 | "outputs": [], 1454 | "source": [] 1455 | } 1456 | ], 1457 | "metadata": { 1458 | "kernelspec": { 1459 | "display_name": "Python 3", 1460 | "language": "python", 1461 | "name": "python3" 1462 | }, 1463 | "language_info": { 1464 | "codemirror_mode": { 1465 | "name": "ipython", 1466 | "version": 3 1467 | }, 1468 | "file_extension": ".py", 1469 | "mimetype": "text/x-python", 1470 | "name": "python", 1471 | "nbconvert_exporter": "python", 1472 | "pygments_lexer": "ipython3", 1473 | "version": "3.7.3" 1474 | } 1475 | }, 1476 | "nbformat": 4, 1477 | "nbformat_minor": 2 1478 | } 1479 | -------------------------------------------------------------------------------- /EEGdatasets.md: -------------------------------------------------------------------------------- 1 | # EEG-Datasets 2 | A list of all public EEG-datasets. This list of EEG-resources is not exhaustive. If you find something new, or have explored any unfiltered link in depth, please update the repository. 3 | 4 | * **Motor-Imagery** 5 | 1. [Left/Right Hand MI](http://gigadb.org/dataset/100295): Includes 52 subjects (38 validated subjects with discriminative features), results of physiological and psychological questionnares, EMG Datasets, location of 3D EEG electrodes, and EEGs for non-task related states 6 | 2. [Motor Movement/Imagery Dataset](https://www.physionet.org/physiobank/database/eegmmidb/): Includes 109 volunteers, 64 electrodes, 2 baseline tasks (eye-open and eye-closed), motor movement, and motor imagery (both fists or both feet) 7 | 3. [Grasp and Lift EEG Challenge](https://www.kaggle.com/c/grasp-and-lift-eeg-detection/data): 12 subjects, 32channels@500Hz, for 6 grasp and lift events, namely a). HandStart b). FirstDigitTouch c). BothStartLoadPhase d). LiftOff e). Replace f). BothReleased 8 | 4. [The largest SCP data of Motor-Imagery](https://doi.org/10.6084/m9.figshare.c.3917698): The dataset contains 60 hours of EEG BCI recordings across 75 recording sessions of 13 participants, 60,000 mental imageries, and 4 BCI interaction paradigms, with multiple recording sessions and paradigms of the same individuals. BCI interactions involving up to 6 mental imagery states are considered. [[Article]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6190745/pdf/sdata2018211.pdf) 9 | 5. [BCI Competition IV-1](http://www.bbci.de/competition/iv/#dataset1): 64 EEG channels at 1000Hz sampling rate for 2 classes of left hand, right hand, foot (+ idle state) for 7 subjects. Evaluation data is continuous EEG which contains also periods of idle state. 10 | 6. [BCI Competition IV-2a](http://www.bbci.de/competition/iv/#dataset2a): 22-electrode EEG motor-imagery dataset, with 9 subjects and 2 sessions, each with 288 four-second trials of imagined movements per subject. Includes movements of the left hand,the right hand, the feet and the tongue. [[Dataset Description]](http://www.bbci.de/competition/iv/desc_2a.pdf) 11 | 7. [BCI Competition IV-2b](http://www.bbci.de/competition/iv/#dataset2b): 3-electrode EEG motor-imagery dataset with 9 subjects and 5 sessions of imagined movements of the left or the right hand, the latest 3 sessions include online feedback. [[Dataset Description]](http://www.bbci.de/competition/iv/desc_2b.pdf) 12 | 8. [High-Gamma Dataset](https://github.com/robintibor/high-gamma-dataset): 128-electrode dataset obtained from 14 healthy subjects with roughly 1000 four-second trials of executed movements divided into 13 runs per subject. The four classes of movements were movements of either the left hand, the right hand, both feet, and rest. 13 | 9. [Left/Right Hand 1D/2D movements](https://sites.google.com/site/projectbci/): 19-electrode data of one subject with various combinations of 1D and 2D hand movements (actual execution). 14 | 10. [Imagination of Right-hand Thumb Movement](https://archive.ics.uci.edu/ml/datasets/Planning+Relax): In every trial, subjects were asked to rest and rest data was recorded for 5 mins. Further, 5 second epoch data was also recorded when subjects were asked to imagine right hand thumb movement. 5 of such imagined motor movement, and rest state was recorded for each trial. Single subject, 8 electrodes at 256Hz. 15 | 16 | 17 | * **Emotion-Recognition** 18 | 1. [DEAP](http://www.eecs.qmul.ac.uk/mmv/datasets/deap/): Includes 32 subjects, each watchine 1-min long excerpts of music-videos, rated by users in terms of arousal/valence/like-dislike/dominanace/famaliarity, and frontal face recording of 22/32 subejcts. 19 | 2. [Enterface'06](http://www.enterface.net/results/): Enterface'06 Project 07: EEG(64 Channels) + fNIRS + face video, Includes 16 subjects, where emotions were elicited through selected subset of IAPS dataset. 20 | 3. [Imagined Emotion](http://headit.ucsd.edu/studies/3316f70e-35ff-11e3-a2a9-0050563f2612): 31 subjects, subjects listen to voice recordings that suggest an emotional feeling and ask subjects to imagine an emotional scenario or to recall an experience in which they have felt that emotion before. 21 | 4. [NeuroMarketing](https://drive.google.com/open?id=0B2T1rQUvyyWcSGVVaHZBZzRtTms): 25 subjects, 14 electrodes, Like/Dislike on commercial e-commerce products over 14 categories with 3 images each. Article for the dataset: Analysis of EEG signals and its application to neuromarketing. [[Article]](https://link.springer.com/article/10.1007/s11042-017-4580-6) 22 | 5. [SEED](http://bcmi.sjtu.edu.cn/~seed/seed.html): 15 subjects were shown video clips eliciting positive/negative/neutral emotion and EEG was recorded over 62 channels. 23 | 6. [SEED-IV](http://bcmi.sjtu.edu.cn/~seed/seed-iv.html): 15 subjects were shown video clips ellicity happy/sad/neutral/fear emotions and EEG was recorded over 62 channels (with eye-tracking) for 3 sessions per subject (24 trials per session). 24 | 7. [SEED-VIG](http://bcmi.sjtu.edu.cn/~seed/seed-vig.html): Vigilance labels with EEG data in a simulated driving task. 18 electrodes and eye-tracking included. 25 | 8. [HCI-Tagging](https://mahnob-db.eu/hci-tagging/): Subjetcs were shown video clips (fragments of movies) and they were asked to annotate the emotional state on the scale of valence and arousal. During the whole experiment, audio, video, gaze data and physiological data were recorded simultaneously with accurate synchronisation between sensors. 26 | 27 | * **Error-Related Potentials (ErrP)** 28 | 1. [BCI-NER Challenge](https://www.kaggle.com/c/inria-bci-challenge): 26 subjects, 56 EEG Channels for a P300 Speller task, and labeled dataset for the response elicited when P300 decodes a correct or incorrect letter. 29 | 30 | 2. [Monitoring ErrP in a target selection task](http://bnci-horizon-2020.eu/database/data-sets): 6 subjects with 64 EEG electrodes, watching a cursor move towards a target square, and elicited responses are labeled based on whether the cursor moves in right or wrong direction. [[Dataset Description]](https://lampx.tugraz.at/~bci/database/013-2015/description.pdf) 31 | 32 | 3. [ErrPs during continuous feedback](https://www-ti.informatik.uni-tuebingen.de/~spueler/eeg_data/contErrP_description.pdf): 10 subjects with 28 EEG electrodes, playing a video game to study execution and outcome error. [[Dataset Part-1]](https://www-ti.informatik.uni-tuebingen.de/~spueler/eeg_data/Continous_ErrP_dataset_Part1.rar) [[Dataset Part-2]](https://www-ti.informatik.uni-tuebingen.de/~spueler/eeg_data/Continous_ErrP_dataset_Part2.rar) 33 | 34 | 4. [HCI-Tagging](https://mahnob-db.eu/hci-tagging/): Subjetcs were shown images or movie fragments with a tag at the bottom of the screen. In some cases, the tag correctly described something about the situation. However, in other cases the tag did not actually apply to the media item. After each item, a participant was asked to press a green button if they agreed with the tag being applicable to the media item, or press a red button if not. During the whole experiment, audio, video, gaze data and physiological data were recorded simultaneously with accurate synchronisation between sensors. 35 | 36 | * **Visually Evoked Potentials (VEPs)** 37 | 1. [c-VEP BCI](https://www-ti.informatik.uni-tuebingen.de/~spueler/eeg_data/cVEP_dataset.rar): 9 subjects, 32 EEG Channels for a VEP BCI speller (32 characters) task, and labeled dataset for the response elicited for the label associated with the speller. [[Dataset description]](https://www-ti.informatik.uni-tuebingen.de/~spueler/eeg_data/cVEP_description.pdf) [[Published article]](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0051077) 38 | 39 | 2. [c-VEP BCI with dry electrodes](https://www-ti.informatik.uni-tuebingen.de/~spueler/eeg_data/dry_cVEP_dataset.rar): 9 subjects, 15 dry-EEG Channels for a VEP BCI speller (32 characters) task, and labeled dataset for the response elicited for the label associated with the speller. [[Article]](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0172400) 40 | 41 | 3. [SSVEP - Visual Search/Discrimination and Handshake](https://archive.ics.uci.edu/ml/datasets/EEG+Steady-State+Visual+Evoked+Potential+Signals#): Includes 3 different tests, (i) Five Box visual test: attnded and unattended disc and square based stimuli, (ii) visual search within natural images: search of a yellow dot stimuli in B&W natural images, (iii) hand shake test: showing left/right hand closed/open images. 30 subjects, 14 electrodes. [[Article 1]](http://www.journalijar.com/uploads/154_IJAR-13703.pdf) [[Article 2]](https://www.hindawi.com/journals/ddns/2018/2143873/) [[More Dataset: Dataset 2]](http://www2.hu-berlin.de/eyetracking-eeg/testdata.html) 42 | 43 | * **Event Related Potentials [ERPs]** 44 | 1. [Pattern Visual Evoked Potentials](https://www2.le.ac.uk/departments/engineering/research/bioengineering/neuroengineering-lab/software): Dataset#5, 2 subjects for checkboard light pattern (oddball paradigm) recorded at O1 position. 45 | 46 | * **Resting State** 47 | 1. [Resting State EEG Data](https://dataverse.tdl.org/dataverse/txstatecogelectro): 22 subjects, 72 EEG Channels for a resting task of 8 mins with 4 mins of eyes closed and 4 mins of eyes open. [[Article]](https://www.frontiersin.org/articles/10.3389/fnins.2017.00425) 48 | 2. [EID-M, EID-S](https://drive.google.com/drive/folders/1t6tL434ZOESb06ZvA4Bw1p9chzxzbRbj): 8 subjects in rest state (with eyes closed) recorded from 14 electrodes using EPOC+ for 54s at 128 Hz (7000 samples each). EID-M has three trials and EID-S is a signle trial dataset. The dataset was used to develop a person identification system through brainwaves. [[Article]](https://arxiv.org/pdf/1711.06149.pdf) 49 | 50 | * **Music and EEG** 51 | 1. [Music Imagery Information Retrieval](https://github.com/sstober/openmiir): 10 subjects, 64 EEG Channels for a music imagery task of 12 different pieces w/ different meter, length and tempo. [[Article]](https://pdfs.semanticscholar.org/cde4/b1ec89f2c05a41f1143792a890a00e89541a.pdf) 52 | 53 | * **Eye-blinks/movements** 54 | 1. [Involuntary Eye Movements during Face Perception](http://www2.hu-berlin.de/eyetracking-eeg/testdata.html): Dataset 1, 26 electrodes, 500Hz sampling rate, and 120 trials. Eye movements and pupil diameter record, EEG and EOG data is present when subject is presented a happy/sad/angry face on the screen. [[Article]](http://www.jneurosci.org/content/suppl/2009/09/30/29.39.12321.DC1/Supplemental_Material.pdf) [P.S: Dataset available on request only] 55 | 2. [Voluntary-Involuntary Eye-Blinks](https://drive.google.com/file/d/0By5iwWd39NblS2tRWmVTdmRzZUU/view?usp=sharing): Voluntary eye-blinks (subject were asked to blink voluntarily within 1s of audio stimulus) and involuntary eye-blinks (natural) was recorded for 20 subjects on 14 electrodes using g.tec. For each subject, 3 sessions with 20 trials each are present in .mat format. [[Article]](https://www.sciencedirect.com/science/article/pii/S0925231216001569) 56 | 3. [EEG-eye state](https://archive.ics.uci.edu/ml/datasets/EEG+Eye+State): Eye-state labeled data for one continuous recording of EEG of 117 seconds with eye-closed and eye-open labels. The dataset was recorded from Emotiv headset. 57 | 58 | * **Miscellaneous** 59 | 1. [MNIST Brain Digits](http://mindbigdata.com/opendb/index.html): EEG data when a digit(0-9) is shown to the subject, recorded 2s for a single subject using Minwave, EPOC, Muse, Insight. Includes over 1.2M samples. 60 | 2. [Imagenet Brain](http://www.mindbigdata.com/opendb/imagenet.html): A random image is shown (out of 14k images from the Imagenet ILSVRC2013 train dataset) and EEG signals are recorded for 3s for one subject. Includes over 70k samples. 61 | 3. [Working Memory](https://github.com/pbashivan/EEGLearn/tree/master/Sample%20data): Participants briefly observe an array containing multiple English characters SET (500ms) and maintain the information for three seconds. A TEST character is then presented and participants respond by press of a button if TEST charter matches one of the characters in the SET. 15 students, 64 electrodes and 500Hz sampling rate. Only a small subset of data is available publicly. [[Original Paper]](https://www.memphis.edu/acnl/publications/pdfs/ejn2014b.pdf) [[Further Analysis in ICLR]](https://arxiv.org/pdf/1511.06448.pdf) 62 | 4. [Deep Sleep Slow Osciallation](https://challengedata.ens.fr/challenges/10): 10 seconds of recording starting 10 seconds before the end of a slow oscillation. Data is recorded with a goal to predict whether or not a slow oscillation will be followed by another one in sham condition, i.e. without any stimulation. 63 | 5. [Genetic Predisposition to Alcoholism](https://archive.ics.uci.edu/ml/datasets/EEG+Database): 120 trials for 120 subjects recorded from 64 electrides at 256Hz. Two groups of subjects were considered, alcoholic and control. Stimuli details are given in the paper. 64 | 65 | * **Clinical EEG** 66 | 1. [TUH EEG Resources](https://www.isip.piconepress.com/projects/tuh_eeg/html/downloads.shtml): Massive amount of data for (i) Abnormal EEG and (ii) EEG Seizures 67 | 68 | 69 | 70 | 71 | ### Others [Unfiltered] 72 | 1. https://sccn.ucsd.edu/~arno/fam2data/publicly_available_EEG_data.html - http://headit.ucsd.edu/studies 73 | 2. https://www2.le.ac.uk/departments/engineering/research/bioengineering/neuroengineering-lab/software 74 | 3. https://github.com/pbashivan/EEGLearn/tree/master/Sample%20data 75 | 4. Section 2: https://arxiv.org/pdf/1611.08024.pdf 76 | 6. EEG Databases for Emotion Recognition, NTU 77 | 7. https://engineuring.wordpress.com/2009/07/08/downloadable-eeg-data/ 78 | 8. http://www.brainsignals.de/ 79 | 9. http://www.fil.ion.ucl.ac.uk/spm/data/ 80 | 10. http://www.brainliner.jp/search/showall/1 81 | 11. http://bnci-horizon-2020.eu/database/data-sets 82 | 13. http://archive.ics.uci.edu/ml/datasets/EEG+Database 83 | 14. https://www.physionet.org/physiobank/database/#neuro 84 | 15. http://www.physionet.org/pn6/chbmit/ 85 | 16. https://sites.google.com/site/iitrcsepradeep7/resume 86 | 18. http://memory.psych.upenn.edu/RAM 87 | 19. http://fcon_1000.projects.nitrc.org/indi/cmi_eeg/ 88 | 20. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8616018 89 | 21. https://arxiv.org/pdf/1805.06427.pdf 90 | 22. http://www.gtec.at/Research/Biosignal-Data-Sets/content/Biosignal-Data-Sets 91 | 23. http://studycatalog.org/ 92 | 24. http://predict.cs.unm.edu/ 93 | 25. https://datadryad.org/resource/doi:10.5061/dryad.070jc 94 | 26. https://ieee-dataport.org/data-competitions 95 | 96 | 97 | 98 | -------------------------------------------------------------------------------- /EEGtutorialusingdeeplearning.m: -------------------------------------------------------------------------------- 1 | %% %%%%%%%%%%%% The Classification of EEG using CNN network %%%%%%%%%%%%%%%%%%%%%%%% 2 | 3 | 4 | %% Second approach Design Network with random generation dataset 5 | % You should use your own dataset according to design ceriteria 6 | Training_ReductWindows_G=rand(662,800); 7 | % The data specification 8 | height = 1; 9 | width = 800; 10 | channels = 1; 11 | sampleSize = 662; 12 | CNN_TrainingData = reshape(Training_ReductWindows_G,[height, width, channels, sampleSize]); 13 | % Training_Labels_Bin_G=[ones(1,662);2*ones(1,662);3*ones(1,662)] 14 | % CNN_TrainingLabels =categorical(Training_Labels_Bin_G)'; 15 | %% The Training Labels for each class 16 | label(1:220,:) = {'W'} % 1st Label % 17 | label(220:440,:) = {'X'}; % 2nd Label % 18 | label(440:662,:)={'Z'}; 19 | CNN_TrainingLabels = categorical(label); % Label vector is ready % 20 | %% The Network Design in the paper 21 | InputLayer = imageInputLayer([height,width,channels]); %'DataAugmentation', 'none'); %'Normalization', 'none'); 22 | %inputLayer=imageInputLayer([1 6000]); 23 | c1=convolution2dLayer([1 200],20,'stride',1); 24 | p1=maxPooling2dLayer([1 20],'stride',10); 25 | c2=convolution2dLayer([1 30],400,'numChannels',20); 26 | p2=maxPooling2dLayer([1 10],'stride',[1 2]); 27 | f1=fullyConnectedLayer(500); 28 | f2=fullyConnectedLayer(3); 29 | s1=softmaxLayer; 30 | outputLayer=classificationLayer; 31 | convnet=[InputLayer; c1; p1; c2; p2; f1; f2; s1;outputLayer] 32 | %% How to build hyperparameters of the network 33 | %opts = trainingOptions('sgdm'); 34 | % Define the Training options 35 | opts=trainingOptions('sgdm',... 36 | 'InitialLearnRate',0.001,... 37 | 'LearnRateSchedule','none',... 38 | 'LearnRateDropPeriod',8,... 39 | 'L2Regularization',0.005,... 40 | 'MaxEpochs',100,... 41 | 'MiniBatchSize',32,... 42 | 'Verbose',true) 43 | 44 | %% Train the Model using dataset and number of training labels, model and hyper-parameters 45 | convnetModel = trainNetwork(CNN_TrainingData, CNN_TrainingLabels, convnet, opts); 46 | 47 | % Design Feature matricis based on CCN features at particular layer 48 | trainingFeatures_conf1 = activations(convnetModel, CNN_TrainingLabels,'c2','MiniBatchSize', 32, 'OutputAs', 'columns'); 49 | %% Compute the accurcay of that network 50 | [labels,err_test] = classify(convnetModel, CNN_TrainingLabels, 'MiniBatchSize', 64); 51 | confMat = confusionmat(CNN_TrainingLabels.Labels, labels); 52 | confMat = bsxfun(@rdivide,confMat,sum(confMat,2)); 53 | mean(diag(confMat)); -------------------------------------------------------------------------------- /Eeglab_data.set: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RespectKnowledge/EEG-using-deep-Learning/9bef3bfe4b7b147f22cf3e40bf543b9422a2759d/Eeglab_data.set -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # EEG-using-deep-Learning 2 | In this Basic Tutorial, Machine learning and deep learning algorithms have been used for EEG signal Classification. 3 | 4 | The algorithms prepared in matlab and pythons. 5 | 6 | Install anaconda and jupyter notebook and simply run the models in python. 7 | 8 | The test dataset is provided for reference. 9 | 10 | You can choose and Prepare your own dataset. 11 | 12 | 13 | If you have any question, write email. 14 | engr.qayyum@gmail.com 15 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # MNE tutorial 2 | 3 | My MNE-Python Tutorial/Training Notebooks/Scripts. 4 | --- 5 | 6 | ### Used Datasets 7 | 8 | - Free EEG Datasets: https://sccn.ucsd.edu/~arno/fam2data/publicly_available_EEG_data.html 9 | 10 | - Datasets for BCI applications: http://bnci-horizon-2020.eu/database/data-sets 11 | 12 | ![](http://bnci-horizon-2020.eu/images/logo_BNCI.png) 13 | 14 | 15 | ### MNE-Python 16 | 17 | ![](https://martinos.org/mne/stable/_static/mne_logo.png) 18 | 19 | - Tutorial: https://martinos.org/mne/stable/auto_tutorials/index.html?highlight=tutorial 20 | 21 | - Documentation: https://martinos.org/mne/stable/documentation.html 22 | 23 | 24 | ### Works 25 | 26 | - motor imagery by EEG, CSP features 27 | 28 | - ERP 29 | --------------------------------------------------------------------------------