├── CIC IDS 2018 ├── FFNN_BC │ ├── keras_metadata.pb │ ├── saved_model.pb │ └── variables │ │ ├── variables.data-00000-of-00001 │ │ └── variables.index ├── FFNN_MC │ ├── keras_metadata.pb │ ├── saved_model.pb │ └── variables │ │ ├── variables.data-00000-of-00001 │ │ └── variables.index ├── Group6_cicids_FFNN.ipynb ├── Group6_cicids_LSTM.ipynb ├── LSTM_BC │ ├── keras_metadata.pb │ └── saved_model.pb ├── LSTM_MC_L2 │ ├── keras_metadata.pb │ └── saved_model.pb └── README.md ├── LICENSE ├── NSL-KDD ├── Best_CNN │ └── checkpoint.hdf5 ├── Best_NN │ └── checkpoint.hdf5 ├── Data │ ├── KDDTest+.txt │ ├── KDDTest-21.txt │ ├── KDDTrain+.txt │ └── KDDTrain+_20Percent.txt ├── Group_6_NSL_KDD.ipynb ├── README.md ├── img │ ├── FNN-arch.drawio │ ├── FNN-arch.png │ ├── acc.png │ ├── acc_CNN.png │ ├── acc_nn.png │ ├── arch_CNN.png │ ├── attacks_table.png │ ├── cm_CNN.png │ ├── cm_NN.png │ ├── cm_RF.png │ ├── flags.png │ ├── flags_hist.png │ ├── fpr.png │ ├── macro_category.png │ ├── services.png │ ├── services_hist.png │ └── tpr.png └── resources │ ├── column_info.pdf │ ├── flag_info.jpg │ └── lib_info.png └── README.md /CIC IDS 2018/FFNN_BC/keras_metadata.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_BC/keras_metadata.pb -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_BC/saved_model.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_BC/saved_model.pb -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_BC/variables/variables.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_BC/variables/variables.data-00000-of-00001 -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_BC/variables/variables.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_BC/variables/variables.index -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_MC/keras_metadata.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_MC/keras_metadata.pb -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_MC/saved_model.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_MC/saved_model.pb -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_MC/variables/variables.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_MC/variables/variables.data-00000-of-00001 -------------------------------------------------------------------------------- /CIC IDS 2018/FFNN_MC/variables/variables.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/FFNN_MC/variables/variables.index -------------------------------------------------------------------------------- /CIC IDS 2018/Group6_cicids_FFNN.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "include_colab_link": true 8 | }, 9 | "kernelspec": { 10 | "name": "python3", 11 | "display_name": "Python 3" 12 | }, 13 | "language_info": { 14 | "name": "python" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "view-in-github", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "\"Open" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "source": [ 31 | "\n", 32 | "## Intrusion Detection System (CIC IDS 2018)\n", 33 | "\n", 34 | "---\n", 35 | "\n", 36 | "\n", 37 | "### Group 6 - Sleety, Thejus, Tejas and Rahul \n", 38 | "#### Rajagiri School of Engineering and Technology (KTU 2019 Scheme)" 39 | ], 40 | "metadata": { 41 | "id": "CP6dTDOCPhCg" 42 | } 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": { 47 | "id": "vHveTnF7vBef" 48 | }, 49 | "source": [ 50 | "##**IDS**" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "metadata": { 56 | "id": "NLezzTSoMcI8" 57 | }, 58 | "source": [ 59 | "import tensorflow as tf\n", 60 | "import pandas as pd\n", 61 | "import numpy as np\n", 62 | "import sklearn\n", 63 | "from keras.models import Sequential, load_model\n", 64 | "from keras.utils import np_utils\n", 65 | "from sklearn.model_selection import train_test_split\n", 66 | "from sklearn.linear_model import LogisticRegression\n", 67 | "from sklearn import preprocessing\n", 68 | "from sklearn.preprocessing import LabelEncoder, StandardScaler\n", 69 | "from sklearn.metrics import confusion_matrix, precision_score, recall_score\n", 70 | "import seaborn as sn" 71 | ], 72 | "execution_count": 25, 73 | "outputs": [] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "metadata": { 78 | "colab": { 79 | "base_uri": "https://localhost:8080/" 80 | }, 81 | "id": "9TWNodDe1kHu", 82 | "outputId": "a4e73f3a-1f62-405f-af01-63d0f9281ac6" 83 | }, 84 | "source": [ 85 | "from google.colab import drive\n", 86 | "drive.mount('/content/drive')" 87 | ], 88 | "execution_count": 2, 89 | "outputs": [ 90 | { 91 | "output_type": "stream", 92 | "name": "stdout", 93 | "text": [ 94 | "Mounted at /content/drive\n" 95 | ] 96 | } 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "metadata": { 102 | "id": "KzFzH3CAMmxQ", 103 | "colab": { 104 | "base_uri": "https://localhost:8080/" 105 | }, 106 | "outputId": "f49092d6-6d2a-4e1c-b6e5-d72a3e85f1aa" 107 | }, 108 | "source": [ 109 | "df = pd.read_csv('/content/drive/My Drive/cicids/cic/02-16-2018.csv')\n", 110 | "df.drop(df.loc[df['Label'] == 'Label'].index, inplace=True)" 111 | ], 112 | "execution_count": 3, 113 | "outputs": [ 114 | { 115 | "output_type": "stream", 116 | "name": "stderr", 117 | "text": [ 118 | "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py:3326: DtypeWarning: Columns (0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78) have mixed types.Specify dtype option on import or set low_memory=False.\n", 119 | " exec(code_obj, self.user_global_ns, self.user_ns)\n" 120 | ] 121 | } 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "metadata": { 127 | "colab": { 128 | "base_uri": "https://localhost:8080/", 129 | "height": 338 130 | }, 131 | "id": "oQdjN1DTBLWl", 132 | "outputId": "3161be4a-3155-4104-98a3-17f9ae436f0a" 133 | }, 134 | "source": [ 135 | "df.describe()" 136 | ], 137 | "execution_count": 4, 138 | "outputs": [ 139 | { 140 | "output_type": "execute_result", 141 | "data": { 142 | "text/plain": [ 143 | " Dst Port Protocol Timestamp Flow Duration Tot Fwd Pkts \\\n", 144 | "count 1048574 1048574 1048574 1048574 1048574 \n", 145 | "unique 14463 6 3177 453875 42 \n", 146 | "top 80 6 16/02/2018 01:45:28 2 5 \n", 147 | "freq 461655 1040250 8403 58706 426407 \n", 148 | "\n", 149 | " Tot Bwd Pkts TotLen Fwd Pkts TotLen Bwd Pkts Fwd Pkt Len Max \\\n", 150 | "count 1048574 1048574 1048574 1048574 \n", 151 | "unique 36 480 861 181 \n", 152 | "top 0 0 0 0 \n", 153 | "freq 438014 572790 572823 572790 \n", 154 | "\n", 155 | " Fwd Pkt Len Min ... Fwd Seg Size Min Active Mean Active Std \\\n", 156 | "count 1048574 ... 1048574 1048574 1048574.0 \n", 157 | "unique 6 ... 10 5695 18.0 \n", 158 | "top 0 ... 32 0 0.0 \n", 159 | "freq 1040366 ... 905621 1031324 1040366.0 \n", 160 | "\n", 161 | " Active Max Active Min Idle Mean Idle Std Idle Max Idle Min \\\n", 162 | "count 1048574 1048574 1048574.0 1048574.0 1048574 1048574 \n", 163 | "unique 5696 5695 46001.0 137.0 46001 46001 \n", 164 | "top 0 0 0.0 0.0 0 0 \n", 165 | "freq 1031324 1031324 982182.0 1040244.0 982182 982182 \n", 166 | "\n", 167 | " Label \n", 168 | "count 1048574 \n", 169 | "unique 3 \n", 170 | "top DoS attacks-Hulk \n", 171 | "freq 461912 \n", 172 | "\n", 173 | "[4 rows x 80 columns]" 174 | ], 175 | "text/html": [ 176 | "\n", 177 | "
\n", 178 | "
\n", 179 | "
\n", 180 | "\n", 193 | "\n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | " \n", 213 | " \n", 214 | " \n", 215 | " \n", 216 | " \n", 217 | " \n", 218 | " \n", 219 | " \n", 220 | " \n", 221 | " \n", 222 | " \n", 223 | " \n", 224 | " \n", 225 | " \n", 226 | " \n", 227 | " \n", 228 | " \n", 229 | " \n", 230 | " \n", 231 | " \n", 232 | " \n", 233 | " \n", 234 | " \n", 235 | " \n", 236 | " \n", 237 | " \n", 238 | " \n", 239 | " \n", 240 | " \n", 241 | " \n", 242 | " \n", 243 | " \n", 244 | " \n", 245 | " \n", 246 | " \n", 247 | " \n", 248 | " \n", 249 | " \n", 250 | " \n", 251 | " \n", 252 | " \n", 253 | " \n", 254 | " \n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | " \n", 275 | " \n", 276 | " \n", 277 | " \n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " \n", 282 | " \n", 283 | " \n", 284 | " \n", 285 | " \n", 286 | " \n", 287 | " \n", 288 | " \n", 289 | " \n", 290 | " \n", 291 | " \n", 292 | " \n", 293 | " \n", 294 | " \n", 295 | " \n", 296 | " \n", 297 | " \n", 298 | " \n", 299 | " \n", 300 | " \n", 301 | " \n", 302 | " \n", 303 | " \n", 304 | " \n", 305 | " \n", 306 | " \n", 307 | " \n", 308 | " \n", 309 | " \n", 310 | " \n", 311 | " \n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | "
Dst PortProtocolTimestampFlow DurationTot Fwd PktsTot Bwd PktsTotLen Fwd PktsTotLen Bwd PktsFwd Pkt Len MaxFwd Pkt Len Min...Fwd Seg Size MinActive MeanActive StdActive MaxActive MinIdle MeanIdle StdIdle MaxIdle MinLabel
count1048574104857410485741048574104857410485741048574104857410485741048574...104857410485741048574.0104857410485741048574.01048574.0104857410485741048574
unique144636317745387542364808611816...10569518.05696569546001.0137.046001460013
top80616/02/2018 01:45:282500000...3200.0000.00.000DoS attacks-Hulk
freq46165510402508403587064264074380145727905728235727901040366...90562110313241040366.010313241031324982182.01040244.0982182982182461912
\n", 319 | "

4 rows × 80 columns

\n", 320 | "
\n", 321 | " \n", 331 | " \n", 332 | " \n", 369 | "\n", 370 | " \n", 394 | "
\n", 395 | "
\n", 396 | " " 397 | ] 398 | }, 399 | "metadata": {}, 400 | "execution_count": 4 401 | } 402 | ] 403 | }, 404 | { 405 | "cell_type": "code", 406 | "metadata": { 407 | "id": "YZFMTRPvM9ts", 408 | "colab": { 409 | "base_uri": "https://localhost:8080/", 410 | "height": 438 411 | }, 412 | "outputId": "319a5821-3304-4da8-cc75-1e7b464c579b" 413 | }, 414 | "source": [ 415 | "df.head()" 416 | ], 417 | "execution_count": 5, 418 | "outputs": [ 419 | { 420 | "output_type": "execute_result", 421 | "data": { 422 | "text/plain": [ 423 | " Dst Port Protocol Timestamp Flow Duration Tot Fwd Pkts \\\n", 424 | "0 0 0 16/02/2018 08:27:23 112640768 3 \n", 425 | "1 0 0 16/02/2018 08:30:12 112641773 3 \n", 426 | "2 35605 6 16/02/2018 08:26:55 20784143 23 \n", 427 | "3 0 0 16/02/2018 08:33:01 112640836 3 \n", 428 | "4 23 6 16/02/2018 08:27:59 20 1 \n", 429 | "\n", 430 | " Tot Bwd Pkts TotLen Fwd Pkts TotLen Bwd Pkts Fwd Pkt Len Max \\\n", 431 | "0 0 0 0 0 \n", 432 | "1 0 0 0 0 \n", 433 | "2 44 2416 1344 240 \n", 434 | "3 0 0 0 0 \n", 435 | "4 1 0 0 0 \n", 436 | "\n", 437 | " Fwd Pkt Len Min ... Fwd Seg Size Min Active Mean Active Std Active Max \\\n", 438 | "0 0 ... 0 0 0.0 0 \n", 439 | "1 0 ... 0 0 0.0 0 \n", 440 | "2 64 ... 20 2624734 0.0 2624734 \n", 441 | "3 0 ... 0 0 0.0 0 \n", 442 | "4 0 ... 20 0 0.0 0 \n", 443 | "\n", 444 | " Active Min Idle Mean Idle Std Idle Max Idle Min Label \n", 445 | "0 0 56300000.0 138.592929 56300000 56300000 Benign \n", 446 | "1 0 56300000.0 263.750829 56300000 56300000 Benign \n", 447 | "2 2624734 9058214.0 0.0 9058214 9058214 Benign \n", 448 | "3 0 56300000.0 82.024387 56300000 56300000 Benign \n", 449 | "4 0 0.0 0.0 0 0 Benign \n", 450 | "\n", 451 | "[5 rows x 80 columns]" 452 | ], 453 | "text/html": [ 454 | "\n", 455 | "
\n", 456 | "
\n", 457 | "
\n", 458 | "\n", 471 | "\n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | " \n", 497 | " \n", 498 | " \n", 499 | " \n", 500 | " \n", 501 | " \n", 502 | " \n", 503 | " \n", 504 | " \n", 505 | " \n", 506 | " \n", 507 | " \n", 508 | " \n", 509 | " \n", 510 | " \n", 511 | " \n", 512 | " \n", 513 | " \n", 514 | " \n", 515 | " \n", 516 | " \n", 517 | " \n", 518 | " \n", 519 | " \n", 520 | " \n", 521 | " \n", 522 | " \n", 523 | " \n", 524 | " \n", 525 | " \n", 526 | " \n", 527 | " \n", 528 | " \n", 529 | " \n", 530 | " \n", 531 | " \n", 532 | " \n", 533 | " \n", 534 | " \n", 535 | " \n", 536 | " \n", 537 | " \n", 538 | " \n", 539 | " \n", 540 | " \n", 541 | " \n", 542 | " \n", 543 | " \n", 544 | " \n", 545 | " \n", 546 | " \n", 547 | " \n", 548 | " \n", 549 | " \n", 550 | " \n", 551 | " \n", 552 | " \n", 553 | " \n", 554 | " \n", 555 | " \n", 556 | " \n", 557 | " \n", 558 | " \n", 559 | " \n", 560 | " \n", 561 | " \n", 562 | " \n", 563 | " \n", 564 | " \n", 565 | " \n", 566 | " \n", 567 | " \n", 568 | " \n", 569 | " \n", 570 | " \n", 571 | " \n", 572 | " \n", 573 | " \n", 574 | " \n", 575 | " \n", 576 | " \n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | " \n", 609 | " \n", 610 | " \n", 611 | " \n", 612 | " \n", 613 | " \n", 614 | " \n", 615 | " \n", 616 | " \n", 617 | " \n", 618 | " \n", 619 | " \n", 620 | "
Dst PortProtocolTimestampFlow DurationTot Fwd PktsTot Bwd PktsTotLen Fwd PktsTotLen Bwd PktsFwd Pkt Len MaxFwd Pkt Len Min...Fwd Seg Size MinActive MeanActive StdActive MaxActive MinIdle MeanIdle StdIdle MaxIdle MinLabel
00016/02/2018 08:27:23112640768300000...000.00056300000.0138.5929295630000056300000Benign
10016/02/2018 08:30:12112641773300000...000.00056300000.0263.7508295630000056300000Benign
235605616/02/2018 08:26:552078414323442416134424064...2026247340.0262473426247349058214.00.090582149058214Benign
30016/02/2018 08:33:01112640836300000...000.00056300000.082.0243875630000056300000Benign
423616/02/2018 08:27:5920110000...2000.0000.00.000Benign
\n", 621 | "

5 rows × 80 columns

\n", 622 | "
\n", 623 | " \n", 633 | " \n", 634 | " \n", 671 | "\n", 672 | " \n", 696 | "
\n", 697 | "
\n", 698 | " " 699 | ] 700 | }, 701 | "metadata": {}, 702 | "execution_count": 5 703 | } 704 | ] 705 | }, 706 | { 707 | "cell_type": "code", 708 | "metadata": { 709 | "id": "HixtSvMpM_Fq" 710 | }, 711 | "source": [ 712 | "metadata = ['fl_dur' #Flow duration\n", 713 | ",'tot_fw_pk' #Total packets in the forward direction\n", 714 | ",'tot_bw_pk' #Total packets in the backward direction\n", 715 | ",'tot_l_fw_pkt' #Total size of packet in forward direction\n", 716 | ",'fw_pkt_l_max' #Maximum size of packet in forward direction\n", 717 | ",'fw_pkt_l_min' #Minimum size of packet in forward direction\n", 718 | ",'fw_pkt_l_avg' #Average size of packet in forward direction\n", 719 | ",'fw_pkt_l_std' #Standard deviation size of packet in forward direction\n", 720 | ",'Bw_pkt_l_max' #Maximum size of packet in backward direction\n", 721 | ",'Bw_pkt_l_min' #Minimum size of packet in backward direction\n", 722 | ",'Bw_pkt_l_avg' #Mean size of packet in backward direction\n", 723 | ",'Bw_pkt_l_std' #Standard deviation size of packet in backward direction\n", 724 | ",'fl_byt_s' #flow byte rate that is number of packets transferred per second\n", 725 | ",'fl_pkt_s' #flow packets rate that is number of packets transferred per second\n", 726 | ",'fl_iat_avg' #Average time between two flows\n", 727 | ",'fl_iat_std' #Standard deviation time two flows\n", 728 | ",'fl_iat_max' #Maximum time between two flows\n", 729 | ",'fl_iat_min' #Minimum time between two flows\n", 730 | ",'fw_iat_tot' #Total time between two packets sent in the forward direction\n", 731 | ",'fw_iat_avg' #Mean time between two packets sent in the forward direction\n", 732 | ",'fw_iat_std' #Standard deviation time between two packets sent in the forward direction\n", 733 | ",'fw_iat_max' #Maximum time between two packets sent in the forward direction\n", 734 | ",'fw_iat_min' #Minimum time between two packets sent in the forward direction\n", 735 | ",'bw_iat_tot' #Total time between two packets sent in the backward direction\n", 736 | ",'bw_iat_avg' #Mean time between two packets sent in the backward direction\n", 737 | ",'bw_iat_std' #Standard deviation time between two packets sent in the backward direction\n", 738 | ",'bw_iat_max' #Maximum time between two packets sent in the backward direction\n", 739 | ",'bw_iat_min' #Minimum time between two packets sent in the backward direction\n", 740 | ",'fw_psh_flag' #Number of times the PSH flag was set in packets travelling in the forward direction (0 for UDP)\n", 741 | ",'bw_psh_flag' #Number of times the PSH flag was set in packets travelling in the backward direction (0 for UDP)\n", 742 | ",'fw_urg_flag' #Number of times the URG flag was set in packets travelling in the forward direction (0 for UDP)\n", 743 | ",'bw_urg_flag' #Number of times the URG flag was set in packets travelling in the backward direction (0 for UDP)\n", 744 | ",'fw_hdr_len' #Total bytes used for headers in the forward direction\n", 745 | ",'bw_hdr_len' #Total bytes used for headers in the forward direction\n", 746 | ",'fw_pkt_s' #Number of forward packets per second\n", 747 | ",'bw_pkt_s' #Number of backward packets per second\n", 748 | ",'pkt_len_min' #Minimum length of a flow\n", 749 | ",'pkt_len_max' #Maximum length of a flow\n", 750 | ",'pkt_len_avg' #Mean length of a flow\n", 751 | ",'pkt_len_std' #Standard deviation length of a flow\n", 752 | ",'pkt_len_va' #Minimum inter-arrival time of packet\n", 753 | ",'fin_cnt' #Number of packets with FIN\n", 754 | ",'syn_cnt' #Number of packets with SYN\n", 755 | ",'rst_cnt' #Number of packets with RST\n", 756 | ",'pst_cnt' #Number of packets with PUSH\n", 757 | ",'ack_cnt' #Number of packets with ACK\n", 758 | ",'urg_cnt' #Number of packets with URG\n", 759 | ",'cwe_cnt' #Number of packets with CWE\n", 760 | ",'ece_cnt' #Number of packets with ECE\n", 761 | ",'down_up_ratio' #Download and upload ratio\n", 762 | ",'pkt_size_avg' #Average size of packet\n", 763 | ",'fw_seg_avg' #Average size observed in the forward direction\n", 764 | ",'bw_seg_avg' #Average size observed in the backward direction\n", 765 | ",'fw_byt_blk_avg' #Average number of bytes bulk rate in the forward direction\n", 766 | ",'fw_pkt_blk_avg' #Average number of packets bulk rate in the forward direction\n", 767 | ",'fw_blk_rate_avg' #Average number of bulk rate in the forward direction\n", 768 | ",'bw_byt_blk_avg' #Average number of bytes bulk rate in the backward direction\n", 769 | ",'bw_pkt_blk_avg' #Average number of packets bulk rate in the backward direction\n", 770 | ",'bw_blk_rate_avg' #Average number of bulk rate in the backward direction\n", 771 | ",'subfl_fw_pk' #The average number of packets in a sub flow in the forward direction\n", 772 | ",'subfl_fw_byt' #The average number of bytes in a sub flow in the forward direction\n", 773 | ",'subfl_bw_pkt' #The average number of packets in a sub flow in the backward direction\n", 774 | ",'subfl_bw_byt' #The average number of bytes in a sub flow in the backward direction\n", 775 | ",'fw_win_byt' #Number of bytes sent in initial window in the forward direction\n", 776 | ",'bw_win_byt' ## of bytes sent in initial window in the backward direction\n", 777 | ",'Fw_act_pkt' ## of packets with at least 1 byte of TCP data payload in the forward direction\n", 778 | ",'fw_seg_min' #Minimum segment size observed in the forward direction\n", 779 | ",'atv_avg' #Mean time a flow was active before becoming idle\n", 780 | ",'atv_std' #Standard deviation time a flow was active before becoming idle\n", 781 | ",'atv_max' #Maximum time a flow was active before becoming idle\n", 782 | ",'atv_min' #Minimum time a flow was active before becoming idle\n", 783 | ",'idl_avg' #Mean time a flow was idle before becoming active\n", 784 | ",'idl_std' #Standard deviation time a flow was idle before becoming active\n", 785 | ",'idl_max' #Maximum time a flow was idle before becoming active\n", 786 | ",'idl_min' #Minimum time a flow was idle before becoming active\n", 787 | "]" 788 | ], 789 | "execution_count": 6, 790 | "outputs": [] 791 | }, 792 | { 793 | "cell_type": "code", 794 | "metadata": { 795 | "id": "s9ZjigalNCIZ", 796 | "colab": { 797 | "base_uri": "https://localhost:8080/" 798 | }, 799 | "outputId": "1b2a9372-efc0-42f9-ee37-cd496b7eaffe" 800 | }, 801 | "source": [ 802 | "df.columns" 803 | ], 804 | "execution_count": 7, 805 | "outputs": [ 806 | { 807 | "output_type": "execute_result", 808 | "data": { 809 | "text/plain": [ 810 | "Index(['Dst Port', 'Protocol', 'Timestamp', 'Flow Duration', 'Tot Fwd Pkts',\n", 811 | " 'Tot Bwd Pkts', 'TotLen Fwd Pkts', 'TotLen Bwd Pkts', 'Fwd Pkt Len Max',\n", 812 | " 'Fwd Pkt Len Min', 'Fwd Pkt Len Mean', 'Fwd Pkt Len Std',\n", 813 | " 'Bwd Pkt Len Max', 'Bwd Pkt Len Min', 'Bwd Pkt Len Mean',\n", 814 | " 'Bwd Pkt Len Std', 'Flow Byts/s', 'Flow Pkts/s', 'Flow IAT Mean',\n", 815 | " 'Flow IAT Std', 'Flow IAT Max', 'Flow IAT Min', 'Fwd IAT Tot',\n", 816 | " 'Fwd IAT Mean', 'Fwd IAT Std', 'Fwd IAT Max', 'Fwd IAT Min',\n", 817 | " 'Bwd IAT Tot', 'Bwd IAT Mean', 'Bwd IAT Std', 'Bwd IAT Max',\n", 818 | " 'Bwd IAT Min', 'Fwd PSH Flags', 'Bwd PSH Flags', 'Fwd URG Flags',\n", 819 | " 'Bwd URG Flags', 'Fwd Header Len', 'Bwd Header Len', 'Fwd Pkts/s',\n", 820 | " 'Bwd Pkts/s', 'Pkt Len Min', 'Pkt Len Max', 'Pkt Len Mean',\n", 821 | " 'Pkt Len Std', 'Pkt Len Var', 'FIN Flag Cnt', 'SYN Flag Cnt',\n", 822 | " 'RST Flag Cnt', 'PSH Flag Cnt', 'ACK Flag Cnt', 'URG Flag Cnt',\n", 823 | " 'CWE Flag Count', 'ECE Flag Cnt', 'Down/Up Ratio', 'Pkt Size Avg',\n", 824 | " 'Fwd Seg Size Avg', 'Bwd Seg Size Avg', 'Fwd Byts/b Avg',\n", 825 | " 'Fwd Pkts/b Avg', 'Fwd Blk Rate Avg', 'Bwd Byts/b Avg',\n", 826 | " 'Bwd Pkts/b Avg', 'Bwd Blk Rate Avg', 'Subflow Fwd Pkts',\n", 827 | " 'Subflow Fwd Byts', 'Subflow Bwd Pkts', 'Subflow Bwd Byts',\n", 828 | " 'Init Fwd Win Byts', 'Init Bwd Win Byts', 'Fwd Act Data Pkts',\n", 829 | " 'Fwd Seg Size Min', 'Active Mean', 'Active Std', 'Active Max',\n", 830 | " 'Active Min', 'Idle Mean', 'Idle Std', 'Idle Max', 'Idle Min', 'Label'],\n", 831 | " dtype='object')" 832 | ] 833 | }, 834 | "metadata": {}, 835 | "execution_count": 7 836 | } 837 | ] 838 | }, 839 | { 840 | "cell_type": "markdown", 841 | "metadata": { 842 | "id": "yGdYt-UEvj2J" 843 | }, 844 | "source": [ 845 | "## **FFNN**" 846 | ] 847 | }, 848 | { 849 | "cell_type": "markdown", 850 | "metadata": { 851 | "id": "_lhN6l71KkhQ" 852 | }, 853 | "source": [ 854 | "**Binary Class**" 855 | ] 856 | }, 857 | { 858 | "cell_type": "code", 859 | "metadata": { 860 | "id": "TZXxDViHNFt7" 861 | }, 862 | "source": [ 863 | "features = ['Flow Duration', 'Tot Fwd Pkts',\n", 864 | " 'Tot Bwd Pkts', 'TotLen Fwd Pkts', 'TotLen Bwd Pkts', 'Fwd Pkt Len Max',\n", 865 | " 'Fwd Pkt Len Min', 'Fwd Pkt Len Mean', 'Fwd Pkt Len Std',\n", 866 | " 'Bwd Pkt Len Max', 'Bwd Pkt Len Min', 'Bwd Pkt Len Mean',\n", 867 | " 'Bwd Pkt Len Std', 'Flow Byts/s', 'Flow Pkts/s', 'Flow IAT Mean',\n", 868 | " 'Flow IAT Std', 'Flow IAT Max', 'Flow IAT Min', 'Fwd IAT Tot',\n", 869 | " 'Fwd IAT Mean', 'Fwd IAT Std', 'Fwd IAT Max', 'Fwd IAT Min',\n", 870 | " 'Bwd IAT Tot', 'Bwd IAT Mean', 'Bwd IAT Std', 'Bwd IAT Max',\n", 871 | " 'Bwd IAT Min', 'Fwd PSH Flags', 'Bwd PSH Flags', 'Fwd URG Flags',\n", 872 | " 'Bwd URG Flags', 'Fwd Header Len', 'Bwd Header Len', 'Fwd Pkts/s',\n", 873 | " 'Bwd Pkts/s', 'Pkt Len Min', 'Pkt Len Max', 'Pkt Len Mean',\n", 874 | " 'Pkt Len Std', 'Pkt Len Var', 'FIN Flag Cnt', 'SYN Flag Cnt',\n", 875 | " 'RST Flag Cnt', 'PSH Flag Cnt', 'ACK Flag Cnt', 'URG Flag Cnt',\n", 876 | " 'CWE Flag Count', 'ECE Flag Cnt', 'Down/Up Ratio', 'Pkt Size Avg',\n", 877 | " 'Fwd Seg Size Avg', 'Bwd Seg Size Avg', 'Fwd Byts/b Avg',\n", 878 | " 'Fwd Pkts/b Avg', 'Fwd Blk Rate Avg', 'Bwd Byts/b Avg',\n", 879 | " 'Bwd Pkts/b Avg', 'Bwd Blk Rate Avg', 'Subflow Fwd Pkts',\n", 880 | " 'Subflow Fwd Byts', 'Subflow Bwd Pkts', 'Subflow Bwd Byts',\n", 881 | " 'Init Fwd Win Byts', 'Init Bwd Win Byts', 'Fwd Act Data Pkts',\n", 882 | " 'Fwd Seg Size Min', 'Active Mean', 'Active Std', 'Active Max',\n", 883 | " 'Active Min', 'Idle Mean', 'Idle Std', 'Idle Max', 'Idle Min']" 884 | ], 885 | "execution_count": 8, 886 | "outputs": [] 887 | }, 888 | { 889 | "cell_type": "code", 890 | "metadata": { 891 | "id": "17a7f5rWNNXZ" 892 | }, 893 | "source": [ 894 | "def targetify(s):\n", 895 | " if s == 'Benign':\n", 896 | " return 0\n", 897 | " else:\n", 898 | " return 1" 899 | ], 900 | "execution_count": 9, 901 | "outputs": [] 902 | }, 903 | { 904 | "cell_type": "code", 905 | "metadata": { 906 | "id": "ALq0oAMFNE3x", 907 | "colab": { 908 | "base_uri": "https://localhost:8080/" 909 | }, 910 | "outputId": "eff0f19a-b710-4db8-bc2b-c4b937d4fd26" 911 | }, 912 | "source": [ 913 | "X = df[features]\n", 914 | "X[features] = X[features].apply(pd.to_numeric, errors='coerce', axis=1)\n", 915 | "X = X.fillna(0)\n", 916 | "labels = df['Label'] #For multiclass classification\n", 917 | "df['Target']=df['Label'].apply(targetify)\n", 918 | "y = df['Target']" 919 | ], 920 | "execution_count": 10, 921 | "outputs": [ 922 | { 923 | "output_type": "stream", 924 | "name": "stderr", 925 | "text": [ 926 | "/usr/local/lib/python3.8/dist-packages/pandas/core/frame.py:3641: SettingWithCopyWarning: \n", 927 | "A value is trying to be set on a copy of a slice from a DataFrame.\n", 928 | "Try using .loc[row_indexer,col_indexer] = value instead\n", 929 | "\n", 930 | "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", 931 | " self[k1] = value[k2]\n" 932 | ] 933 | } 934 | ] 935 | }, 936 | { 937 | "cell_type": "code", 938 | "metadata": { 939 | "id": "8nWwSwSoNExM" 940 | }, 941 | "source": [ 942 | "min_max_scaler = preprocessing.MinMaxScaler()\n", 943 | "x_scaled = min_max_scaler.fit_transform(X.values)\n", 944 | "X = pd.DataFrame(x_scaled,columns=features)" 945 | ], 946 | "execution_count": 11, 947 | "outputs": [] 948 | }, 949 | { 950 | "cell_type": "code", 951 | "metadata": { 952 | "id": "GG4_M7ObNbKH", 953 | "colab": { 954 | "base_uri": "https://localhost:8080/" 955 | }, 956 | "outputId": "4b2fe5d7-bc99-422c-84a2-dd8b6c803806" 957 | }, 958 | "source": [ 959 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", 960 | "print (X_train.shape, y_train.shape)\n", 961 | "print( X_test.shape, y_test.shape)" 962 | ], 963 | "execution_count": 12, 964 | "outputs": [ 965 | { 966 | "output_type": "stream", 967 | "name": "stdout", 968 | "text": [ 969 | "(838859, 76) (838859,)\n", 970 | "(209715, 76) (209715,)\n" 971 | ] 972 | } 973 | ] 974 | }, 975 | { 976 | "cell_type": "code", 977 | "metadata": { 978 | "id": "maEm0IWENq_f" 979 | }, 980 | "source": [ 981 | "model = tf.keras.models.Sequential([\n", 982 | " tf.keras.layers.Dense(128, activation='relu'),\n", 983 | " tf.keras.layers.Dropout(0.2),\n", 984 | " tf.keras.layers.Dense(64, activation='relu'),\n", 985 | " tf.keras.layers.Dropout(0.4),\n", 986 | " tf.keras.layers.Dense(2, activation='softmax')\n", 987 | "])" 988 | ], 989 | "execution_count": 13, 990 | "outputs": [] 991 | }, 992 | { 993 | "cell_type": "code", 994 | "metadata": { 995 | "id": "3Tdxjo5VNq1U", 996 | "colab": { 997 | "base_uri": "https://localhost:8080/" 998 | }, 999 | "outputId": "128485cc-a2ee-488e-bf36-77c54c5a6178" 1000 | }, 1001 | "source": [ 1002 | "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n", 1003 | "model.fit(X_train.values, y_train.values, epochs=5)\n", 1004 | "model.save('drive/MyDrive/cicids/FFNN_BC')" 1005 | ], 1006 | "execution_count": 14, 1007 | "outputs": [ 1008 | { 1009 | "output_type": "stream", 1010 | "name": "stdout", 1011 | "text": [ 1012 | "Epoch 1/5\n", 1013 | "26215/26215 [==============================] - 55s 2ms/step - loss: 0.0138 - accuracy: 0.9979\n", 1014 | "Epoch 2/5\n", 1015 | "26215/26215 [==============================] - 60s 2ms/step - loss: 0.0117 - accuracy: 0.9983\n", 1016 | "Epoch 3/5\n", 1017 | "26215/26215 [==============================] - 54s 2ms/step - loss: 0.0115 - accuracy: 0.9983\n", 1018 | "Epoch 4/5\n", 1019 | "26215/26215 [==============================] - 55s 2ms/step - loss: 0.0115 - accuracy: 0.9983\n", 1020 | "Epoch 5/5\n", 1021 | "26215/26215 [==============================] - 55s 2ms/step - loss: 0.0113 - accuracy: 0.9983\n" 1022 | ] 1023 | } 1024 | ] 1025 | }, 1026 | { 1027 | "cell_type": "code", 1028 | "metadata": { 1029 | "id": "i8qkqLZfNqou", 1030 | "colab": { 1031 | "base_uri": "https://localhost:8080/" 1032 | }, 1033 | "outputId": "cb15357b-1db1-4641-d468-6fdaf38dad22" 1034 | }, 1035 | "source": [ 1036 | "predictions = model.predict(X_test.values)[:,1]\n", 1037 | "predictions = [int(round(x)) for x in predictions]\n", 1038 | "np.sum(predictions == y_test.values) / len(y_test.values)" 1039 | ], 1040 | "execution_count": 15, 1041 | "outputs": [ 1042 | { 1043 | "output_type": "stream", 1044 | "name": "stdout", 1045 | "text": [ 1046 | "6554/6554 [==============================] - 9s 1ms/step\n" 1047 | ] 1048 | }, 1049 | { 1050 | "output_type": "execute_result", 1051 | "data": { 1052 | "text/plain": [ 1053 | "0.9983119948501538" 1054 | ] 1055 | }, 1056 | "metadata": {}, 1057 | "execution_count": 15 1058 | } 1059 | ] 1060 | }, 1061 | { 1062 | "cell_type": "code", 1063 | "metadata": { 1064 | "colab": { 1065 | "base_uri": "https://localhost:8080/" 1066 | }, 1067 | "id": "Eg8wI42LHPPV", 1068 | "outputId": "49338ac2-6674-46f4-8969-ecca8771d866" 1069 | }, 1070 | "source": [ 1071 | "confMat = confusion_matrix(y_test.values, predictions)\n", 1072 | "confMat" 1073 | ], 1074 | "execution_count": 16, 1075 | "outputs": [ 1076 | { 1077 | "output_type": "execute_result", 1078 | "data": { 1079 | "text/plain": [ 1080 | "array([[ 88959, 354],\n", 1081 | " [ 0, 120402]])" 1082 | ] 1083 | }, 1084 | "metadata": {}, 1085 | "execution_count": 16 1086 | } 1087 | ] 1088 | }, 1089 | { 1090 | "cell_type": "code", 1091 | "metadata": { 1092 | "id": "HxGIS0PHUv04", 1093 | "colab": { 1094 | "base_uri": "https://localhost:8080/" 1095 | }, 1096 | "outputId": "217744a9-cf8c-415f-fb5a-b4fe8dea650a" 1097 | }, 1098 | "source": [ 1099 | "precision_score(y_test, predictions)" 1100 | ], 1101 | "execution_count": 17, 1102 | "outputs": [ 1103 | { 1104 | "output_type": "execute_result", 1105 | "data": { 1106 | "text/plain": [ 1107 | "0.9970684686475206" 1108 | ] 1109 | }, 1110 | "metadata": {}, 1111 | "execution_count": 17 1112 | } 1113 | ] 1114 | }, 1115 | { 1116 | "cell_type": "code", 1117 | "metadata": { 1118 | "id": "FUqP8hetU3ym" 1119 | }, 1120 | "source": [ 1121 | "#recall_score(y_test.values, predictions)" 1122 | ], 1123 | "execution_count": 61, 1124 | "outputs": [] 1125 | }, 1126 | { 1127 | "cell_type": "code", 1128 | "metadata": { 1129 | "id": "mU5wAR5r6c-J", 1130 | "outputId": "d5cbbf57-e34f-4a2e-8eb3-7eb8e48c025b", 1131 | "colab": { 1132 | "base_uri": "https://localhost:8080/", 1133 | "height": 282 1134 | } 1135 | }, 1136 | "source": [ 1137 | "cf_matrix = confusion_matrix(y_test.values, predictions)\n", 1138 | "sn.heatmap(cf_matrix / np.sum(cf_matrix), annot=True, fmt='.2%', cmap='Blues')" 1139 | ], 1140 | "execution_count": 19, 1141 | "outputs": [ 1142 | { 1143 | "output_type": "execute_result", 1144 | "data": { 1145 | "text/plain": [ 1146 | "" 1147 | ] 1148 | }, 1149 | "metadata": {}, 1150 | "execution_count": 19 1151 | }, 1152 | { 1153 | "output_type": "display_data", 1154 | "data": { 1155 | "text/plain": [ 1156 | "
" 1157 | ], 1158 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD4CAYAAABPLjVeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAYhUlEQVR4nO3deXgV1f3H8ff33hASKyAuBCSpUkPFgFZlqwKiLCUsgogLUNeiqUvcreJSWmlrqWupgBoUa7WCaGuNJoBUQAWLQrVlqaAREYKSoCDoTxBCzu+PxHjJdm/IzWQyfF4+8zx3Zs6cOfOIH45n5syYcw4REfFGqLEbICJyIFHoioh4SKErIuIhha6IiIcUuiIiHkpo6BOk3zxHj0dIFasmDW7sJogPJSVg9a0j+aTsmDNn57tT6n2+ulJPV0TEQw3e0xUR8ZT5uy+p0BWRYAmFG7sFtVLoikiwmOfDtHWi0BWRYNHwgoiIh9TTFRHxkHq6IiIeUk9XRMRDenpBRMRDGl4QEfGQhhdERDyknq6IiIcUuiIiHgrrRpqIiHc0pisi4iENL4iIeEg9XRERD6mnKyLiIfV0RUQ8pGnAIiIe0vCCiIiHNLwgIuIh9XRFRDzk89D1d+tEROoqFI59icLMMs1srZkVmNn4avZfYmZbzOw/5ctl0epUT1dEgiVOY7pmFgamAgOBQmCZmeU65/5XqeizzrnsWOtVT1dEgsVCsS+16wEUOOfWOed2A7OAEfVtnkJXRILFLObFzLLMbHnEkhVRU3tgY8R6Yfm2ykaZ2Qoze97M0qI1T8MLIhIoVofhBedcDpBTj9O9BMx0zn1jZj8HngT61XaAeroiEihW1oONaYliExDZc00t31bBOfe5c+6b8tXHgK7RKlXoikigWMhiXqJYBnQ0sw5mlgiMBnL3OZdZu4jV4cB70SrV8IKIBEpdhhdq45wrMbNsYB4QBmY451ab2URguXMuF7jWzIYDJcBW4JJo9Sp0RSRQ4hW6AM65fCC/0rYJEb9vA26rS50KXREJlHiGbkNQ6IpIsPg7cxW6IhIs6umKiHgoFPL3Q1kKXREJFPV0RUS85O/MVeiKSLCopysi4iGFroiIh2KY3tuoFLoiEijq6YqIeEihKyLiIYWuiIiHFLoiIl7yd+YqdEUkWDQNWETEQxpeEBHxkr8zV99Iq4uQQe4Nvcj5Wdm35+4f+yNeuaUP+Tf35vfnHU9CLQ9lH9w8gcV3nsGvRmYAkNQsxPRxXZl3Sx/m3NybXwz5YUXZC3sdRf7NvXlsXFeahcvq7Hp0a+4Y3qkBr07qa8kbrzN86CCGZQ7k8elVPzD77+XLOP+ckZx8Qgbz582t2P72W0s57+wRFUv3k45nwav/BOC2W27inJFn8qc/PlBRPueRaRX7pao4fpiyQSh06+CSPkdTUPRVxXruO5/wk3veYMh9i0lqFuK8njV/8v76zI68vW7rPtseX/QRg+55g+EPLuHko1tzWqfDARhx8pEMvX8x76z/gj7HHgFA9sBjmDL/wwa4KomHvXv3cvfvJjLtkcd4ITePufkv82FBwT5l2rZrx29+93sGDx22z/YePX/M7L+/yOy/v8j0GU+SlJTMKaf24v21a2ielMTzL7zE6lUr+fLLL9mypZiVK1bQr/8ALy+vSVHoBkTbVkmcftwRzH57Y8W219Zsqfi9YsN22rZqXu2xndu35PCDE1n8/mcV23btKWXph2UhvGevY/WmHbRrlVS206BZKERyYpiSvaWcdfKRvLbmM7bv3NMAVybxsGrlCtLSjiI1LY1miYlkDhnKooWv7lOmfftUfnhsJ0JW839281+ZR+8+fUhOTiYhoRnf7NpFaWkpJSUlhEMhpj30J67KvqahL6dJa/Kha2adzOxWM/tT+XKrmR3nReP85M4Rx/GHl9fiXNV9CSHjrK5H8vraz6rsM4Pbh3di0stra6y7RVIC/TLa8OYHnwPw9OKPef7aUzjykCT+vX4bo3qk8vSSj+N2LRJ/xUVFtG3XtmK9TUoKRUVFda5n7pw8MoeU9YR/cMwxtG59KKPPGclpp5/Bhg0bKHWlHJfROW7tDqI4foK9QdR6I83MbgXGALOAt8s3pwIzzWyWc25SDcdlAVkARwy8hpYnDI5fixvBGccdwedffcPqTTvoecyhVfbfdXZn3l63jeUfbauy74JTv8+i97awefuuausOh4w/XnAif1n8MRu37gTgH+98wj/e+QSA7IHpPPnGevp2OoKR3drz6Rc7ufulNdWGvzRtW7YUU/DB+5zaq3fFtltuu6Pi9zVXXcEvf30X0x99mPfXruHHp/Ri1LnnNUZTfa2pP70wDujsnNvn/2vN7AFgNVBt6DrncoAcgPSb5zT5eOh6dGv6Z6TQt9MRNE8Ic3BSAvePOYGbZq7gmoHpHHpwInc++U61x554VGu6d2jNT0/9Pgc1TyAxHOLrb0q4N/99AH57ThfWb/k//vzG+irHtmnZnBPSWjFlfgHPXNmTCx55i6sHpHNq+mEsKe8Viz+0SUlh86ebK9aLi4pISUmpUx2vzJ1Dv/4DadasWZV9Cxf8k4zOnfn666/ZuHED9z4wmSsuH8eQYWeSnJxc7/YHSVMP3VLgSKDy/9u2K993QLhvzvvcN6csJHsecyjj+nbgppkrOK9HKn2OPZwLH3m7xp7nTc/8t+L32d3ac3xaq4rAvSGzIy2SErj9uZXVHnvDoI5MnvcBAM2bhXBAqXMkJ4bjd3ESF527HM+GDespLNxISpsU5ubn8ft7769THXPy87j2+hurbN+zZw9P/+VJpjycw4aPP64IldLSvezZs0ehW4nPMzdq6F4PvGpmHwDf3kH6PpAOZDdkw5qCiaM688m2XTx3zSkAvLKqiCnzC+iS2pKxp3yf259bVeOxbVslcfWAdAqKvuLF63sB8PSSj5n9diEAGUe2BGD1ph0AvPTup+Tf1JtPv9jF9IUfNeRlyX5ISEjgtjsmcGXWZZSW7uWskaNIT+/I1Icm07lzF07v159VK1dww3XZ7Nixg9cWLWTa1Id4ITcPgE2bCtm8+VO6de9Rpe5nZ/6V4SNGkpyczA+PPZZdO3cx6qwz6d3nNFq2bOn1pfqe33u65qIMDppZCOgBtC/ftAlY5pzbG8sJgjC8IPG3alLTHueXhpGUUP+pDcfeOi/mzFn7h0GeJ3TUGWnOuVJgqQdtERGpN593dDUNWESCJaTP9YiIeMfvPV3NSBORQInnjDQzyzSztWZWYGbjayk3ysycmXWLVqdCV0QCxSz2pfZ6LAxMBQYDGcAYM8uoplwL4DrgrVjap9AVkUAJhUIxL1H0AAqcc+ucc7spm5k7oppyvwH+AFQ/7bRy++pyMSIifleXnq6ZZZnZ8oglK6Kq9nw3PwGgkO8enS0/l50MpDnn8mJtn26kiUig1GVyROQrC/bjPCHgAeCSuhyn0BWRQInj0wubgMiXZKeWb/tWC6ALsKg86NsCuWY23Dm3vKZKFboiEihxnAa8DOhoZh0oC9vRwNhvdzrntgOHR5x3EXBzbYELGtMVkYCJ19MLzrkSyt4xMw94D5jtnFttZhPNbPj+tk89XREJlHjOSHPO5QP5lbZNqKHs6bHUqdAVkUDx+1vGFLoiEig+z1yFrogEi3q6IiIe8nnmKnRFJFj0akcREQ9peEFExEMKXRERD/k8cxW6IhIs6umKiHjI55mr0BWRYNHTCyIiHgr5vKur0BWRQPF55ip0RSRYdCNNRMRDPh/SVeiKSLDoRpqIiIcMha6IiGd83tFV6IpIsOhGmoiIh3yeuQpdEQkWTY4QEfGQnl4QEfGQzzu6Cl0RCRYNL4iIeMjfkavQFZGA0SNjIiIe8vl9NIWuiASLnl4QEfGQ34cXQo3dABGReApZ7Es0ZpZpZmvNrMDMxlez/wozW2lm/zGzxWaWEbV9+3dZIiL+ZGYxL1HqCQNTgcFABjCmmlB9xjl3vHPuROAe4IFo7VPoikigWB2WKHoABc65dc653cAsYERkAefcjojV7wEuWqUa0xWRQAnX4UaamWUBWRGbcpxzOeW/2wMbI/YVAj2rqeNq4EYgEegX7ZwKXREJlLrcSCsP2JyoBWuvYyow1czGAncCF9dWXsMLIhIoZrEvUWwC0iLWU8u31WQWcFa0ShW6IhIoIbOYlyiWAR3NrIOZJQKjgdzIAmbWMWJ1KPBBtEo1vCAigRKvx3SdcyVmlg3MA8LADOfcajObCCx3zuUC2WY2ANgDbCPK0AKAORf1Zlu97CqJfjdPDjytu2c3dhPEh3a+O6XekXn1C+/FnDlTRx7n+UwK9XRFJFDCPp+RptAVkUDx+asXFLoiEiwKXRERD/n9hTcKXREJFPV0RUQ85POOrkJXRIIlweepq9AVkUDxeeYqdEUkWPQJdhERD/k8cxW6IhIsenpBRMRDdXmJeWNQ6IpIoPg8cxW6IhIsFsvXzxqRQldEAkU9XRERDyl0RUQ8pBfeiIh4KOzzLz8qdEUkUDQjTUTEQxrTFRHxkM87ugpdEQmWkJ7TFRHxjnq6IiIeSvD5oK5CV0QCRT1dEREP6ZExEREP+TxzFboiEiw+n5Cm0BWRYPH78ILf/1IQEamTkFnMSzRmlmlma82swMzGV7P/RjP7n5mtMLNXzeyoqO3bz+sSEfElq8NSaz1mYWAqMBjIAMaYWUalYu8C3ZxzJwDPA/dEa59CV0QCxSz2JYoeQIFzbp1zbjcwCxgRWcA5t9A593X56lIgNVqlCl0RCRQzq8uSZWbLI5asiKraAxsj1gvLt9VkHDAnWvt0I01EAqUuPUnnXA6QU99zmtkFQDegb7SyCl0RCZQ4Pr2wCUiLWE8t37YPMxsA3AH0dc59E61Sha6IBEocP9ezDOhoZh0oC9vRwNhK5zoJeBTIdM4Vx1KpQldEAiVeN6qccyVmlg3MA8LADOfcajObCCx3zuUC9wIHA8+Vh/0G59zw2upV6IpIoMTzw5TOuXwgv9K2CRG/B9S1ToWuiASKv+ejKXRFJGDCPp8GrNAVkUDxeeYqdEUkWMznAwwKXREJFPV0RUQ8pK8Bi4h4SD1dEREP+f0l5gpdEQkUn3+BXaErIsGipxdERDzk89EFvcR8fyx543WGDx3EsMyBPD696qs4d+/ezS9uup5hmQP56ehz2bSpsGLf49MfZVjmQIYPHcSSxW8AsHXrVi6+YAxnjxjGglf/WVH2uuwrKS4uavgLkv22Ju8uls2+naWzxrP4r7cA8NSkS1k6azxLZ41nTd5dLJ1V5dNaFUIh418zb+Vvk6+o2HbF+aex6sVfsfPdKRx2yPcqtp/V/0T+/fwd/PPx6zm0Vdn2DqmH89SkSxvo6pomq8M/jUE93Trau3cvd/9uIo9Of4KUlBTGnn8Op5/Rj2PS0yvKvPC352jZsiUvz53PnPw8/vjAfdx7/x/5sKCAufl5/D03j+LiIn5+2aXk5s1jTv7LnHv+aPoP+AnZV2bRr/8AFi1cQKfjMmjTJqURr1ZikZk1mc+/+L+K9QvHP1Hxe9KNI9n+1c4aj80eewZrPyqixfeSKrb96z/ryH99Fa88dt0+Za8c3ZfeF9zDiH4ncv7gbjw86zV+ffUwfj3t5TheTdPn9zFd9XTraNXKFaSlHUVqWhrNEhPJHDKURQtf3afMwgULGD5iJAADfzKIt5f+C+ccixa+SuaQoSQmJpKamkZa2lGsWrmCZgkJ7Nq5iz27dxMKhSgpKeGvTz3JJT+7rDEuUeJo1MCTmT3339Xua9/mEDJ7d+aJF97cZ/t/1xay4dOtVcqXlpbSvFkCByUlsqdkL71OOoaiz3bw4YYtDdL2piqeXwNukPY1ylmbsOKiItq2a1ux3iYlhaKifYcAiouLaNu2HQAJCQkc3KIFX3yxjaKiIlLafndsStsUiouKGDz0TBYtfJWfX34pl2VdwbOznmHYmSNITk725qJkvznneGlaNkv+egs/O7vXPvt6nXwMRVu/rDEU7/3FKO6Y/A9KS11M57p3xnzyHrmGIad1Yfbc5Yy/PJPfT59b72sImnh9Dbih7Pfwgpld6px7ooZ9WUAWwJRpjzLu8qzqikm5Fi1aMOXhsrHhHdu3M+OxHB6cPIW7JtzJjh07uOiSS/nRiSc1ciulOv0vfZBPtmzniNYH8/Ij2axdv5kl73wIwHmZ3Xhu7vJqjxvcpwvFW7/k3fc20qdrx5jOteCtNSz46RoAxg7rwbzFq+l4VBuuv6g/23Z8zc33Ps/OXXvic2FNmN+f061PT/eumnY453Kcc92cc92CFrhtUlLY/OnmivXioiJSUvYdd23TJoXNmz8FoKSkhK++/JJDDmlNSkoKRZu/O7ZocxFtKh376CPTuCzrCubk53HSyV35zd2TeHjqlAa8IqmPT7ZsB2DLtq/IXbCC7p2PBiAcDjGi3494ft471R53yok/YFjf41mTdxd/mXQpp3f/ITN+e1FM50xOasaFZ/bkkdmvc+cVQ7nsl0/x5n/WMXpw97hcU1Pn955uraFrZitqWFYCB+Qdns5djmfDhvUUFm5kz+7dzM3Po+8Z/fYpc/oZ/ch98QUA5r8yjx49f4yZ0feMfszNz2P37t0UFm5kw4b1dDn+hIrjPv54PcVFm+neoye7du3EQmWfif7mm12eXqPE5qCkRA4+qHnF7wGndGL1h58A0K/nsby/vohNxV9Ue+yEh3JJz/wlnYb+iovGP8GiZe/zszv/EtN5b7hoANNmvkZJSSnJSc1wOEpLSzkoKTE+F9bU+Tx1ow0vpACDgG2VthvwZtXiwZeQkMBtd0zgyqzLKC3dy1kjR5Ge3pGpD02mc+cunN6vPyNHncMd43/BsMyBtGzVinvuexCA9PSO/CRzMCOHDyEcDnP7nRMIh8MVdU+Z/CDZ190AQOaQYdxw7dXMeGw6V2df2yjXKrVrc1gLnn3gcgASwmGenbOc+W++B8C5g7pWuYHW7ohWTJswlpHXPFxrvVeN6cuNFw8g5bCWLJt9O3MXr+aqic9U1NGty1HcnTMHgIdnvsbip29h+5dfc96N0+N9iU2S34cXzLmaB/HN7HHgCefc4mr2PeOcG1vNYfvYVUJsdwnkgNK6e3ZjN0F8aOe7U+qdmMvWbY85c7r/oJXnCV1rT9c5N66WfVEDV0TEc/7u6GpyhIgEi969ICLiIZ8P6Sp0RSRYfJ65Cl0RCRbzeVdXoSsigeLzzFXoikiw+DxzFboiEjA+T12FrogEit8fGdOrHUUkUMxiX6LXZZlmttbMCsysyidAzOw0M3vHzErM7JxY2qfQFZFAiVfomlkYmAoMBjKAMWaWUanYBuAS4JlY26fhBREJlDgOL/QACpxz6wDMbBYwAvjftwWcc+vL95XGWql6uiISKHXp6ZpZlpktj1giXwDeHtgYsV5Yvq1e1NMVkUCpSz/XOZcDVP2kdwNS6IpIsMTv4YVNQFrEemr5tnpR6IpIoMTxJebLgI5m1oGysB0N1PuVthrTFZFAidfXepxzJUA2MA94D5jtnFttZhPNbDiAmXU3s0LgXOBRM1sdrX3q6YpIsMRxboRzLh/Ir7RtQsTvZZQNO8RMoSsigeL3GWkKXREJFL1lTETEQz7PXIWuiASLXmIuIuIhn2euQldEgsXnmavQFZGA8XnqKnRFJFD0yJiIiIc0pisi4qGQQldExEv+Tl2FrogEioYXREQ85PPMVeiKSLCopysi4iFNAxYR8ZC/I1ehKyIB4/OOrkJXRIJFM9JERLzk78xV6IpIsPg8cxW6IhIscfwEe4NQ6IpIoPg8cwk1dgNERA4k6umKSKD4vaer0BWRQNEjYyIiHlJPV0TEQwpdEREPaXhBRMRDfu/p6pExEQkUq8MStS6zTDNba2YFZja+mv3NzezZ8v1vmdnR0epU6IpIsMQpdc0sDEwFBgMZwBgzy6hUbBywzTmXDjwI/CFa8xS6IhIoIbOYlyh6AAXOuXXOud3ALGBEpTIjgCfLfz8P9Lcob1Fv8DHdpASfj2p7yMyynHM5jd0OP9j57pTGboJv6M9FfNUlc8wsC8iK2JQT8e+iPbAxYl8h0LNSFRVlnHMlZrYdOAz4rKZzqqfrrazoReQApD8XjcQ5l+Oc6xaxNPhffgpdEZHqbQLSItZTy7dVW8bMEoBWwOe1VarQFRGp3jKgo5l1MLNEYDSQW6lMLnBx+e9zgAXOOVdbpXpO11sat5Pq6M+FD5WP0WYD84AwMMM5t9rMJgLLnXO5wOPAU2ZWAGylLJhrZVFCWURE4kjDCyIiHlLoioh4SKHrkWjTCeXAY2YzzKzYzFY1dlvEOwpdD8Q4nVAOPH8GMhu7EeItha43YplOKAcY59zrlN3xlgOIQtcb1U0nbN9IbRGRRqTQFRHxkELXG7FMJxSRA4BC1xuxTCcUkQOAQtcDzrkS4NvphO8Bs51zqxu3VdLYzGwm8C/gWDMrNLNxjd0maXiaBiwi4iH1dEVEPKTQFRHxkEJXRMRDCl0REQ8pdEVEPKTQFRHxkEJXRMRD/w9qfT6Irn+F6QAAAABJRU5ErkJggg==\n" 1159 | }, 1160 | "metadata": { 1161 | "needs_background": "light" 1162 | } 1163 | } 1164 | ] 1165 | }, 1166 | { 1167 | "cell_type": "markdown", 1168 | "metadata": { 1169 | "id": "SnS1tL8eQMGp" 1170 | }, 1171 | "source": [ 1172 | "**Multicalss**" 1173 | ] 1174 | }, 1175 | { 1176 | "cell_type": "code", 1177 | "metadata": { 1178 | "id": "V_Pe0g0YUVbY" 1179 | }, 1180 | "source": [ 1181 | "categories = ['Benign', 'FTP-BruteForce', 'SSH-Bruteforce',\n", 1182 | " 'DoS attacks-GoldenEye', 'DoS attacks-Slowloris', 'DoS attacks-SlowHTTPTest',\n", 1183 | " 'DoS attacks-Hulk', 'Brute Force -Web', 'Brute Force -XSS',\n", 1184 | " 'SQL Injection', 'Infiltration', 'Bot']" 1185 | ], 1186 | "execution_count": 20, 1187 | "outputs": [] 1188 | }, 1189 | { 1190 | "cell_type": "code", 1191 | "metadata": { 1192 | "id": "K1uUFkSq0ZEv" 1193 | }, 1194 | "source": [ 1195 | "labels = df['Label']" 1196 | ], 1197 | "execution_count": 21, 1198 | "outputs": [] 1199 | }, 1200 | { 1201 | "cell_type": "code", 1202 | "metadata": { 1203 | "id": "VcKnKgcmT7x2" 1204 | }, 1205 | "source": [ 1206 | "encoder = LabelEncoder()\n", 1207 | "encoder.fit(categories)\n", 1208 | "y = encoder.transform(labels)\n", 1209 | "y = np_utils.to_categorical(y, num_classes=12)" 1210 | ], 1211 | "execution_count": 22, 1212 | "outputs": [] 1213 | }, 1214 | { 1215 | "cell_type": "code", 1216 | "metadata": { 1217 | "id": "ZXWj5nY5HRkC", 1218 | "outputId": "b75c075a-bae4-44a7-e6b0-736515bbc87f", 1219 | "colab": { 1220 | "base_uri": "https://localhost:8080/" 1221 | } 1222 | }, 1223 | "source": [ 1224 | "y.shape" 1225 | ], 1226 | "execution_count": 23, 1227 | "outputs": [ 1228 | { 1229 | "output_type": "execute_result", 1230 | "data": { 1231 | "text/plain": [ 1232 | "(1048574, 12)" 1233 | ] 1234 | }, 1235 | "metadata": {}, 1236 | "execution_count": 23 1237 | } 1238 | ] 1239 | }, 1240 | { 1241 | "cell_type": "code", 1242 | "metadata": { 1243 | "id": "rkGTxRFQIRRI", 1244 | "outputId": "f92afb8c-0917-46d0-ae25-3ecf9333f042", 1245 | "colab": { 1246 | "base_uri": "https://localhost:8080/" 1247 | } 1248 | }, 1249 | "source": [ 1250 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", 1251 | "print (X_train.shape, y_train.shape)\n", 1252 | "print( X_test.shape, y_test.shape)" 1253 | ], 1254 | "execution_count": 24, 1255 | "outputs": [ 1256 | { 1257 | "output_type": "stream", 1258 | "name": "stdout", 1259 | "text": [ 1260 | "(838859, 76) (838859, 12)\n", 1261 | "(209715, 76) (209715, 12)\n" 1262 | ] 1263 | } 1264 | ] 1265 | }, 1266 | { 1267 | "cell_type": "code", 1268 | "metadata": { 1269 | "id": "0bvqaOJyxFLP" 1270 | }, 1271 | "source": [ 1272 | "model = tf.keras.models.Sequential([\n", 1273 | " tf.keras.layers.Dense(128, activation='relu'),\n", 1274 | " tf.keras.layers.Dropout(0.2),\n", 1275 | " tf.keras.layers.Dense(64, activation='relu'),\n", 1276 | " tf.keras.layers.Dropout(0.4),\n", 1277 | " tf.keras.layers.Dense(12, activation='softmax')\n", 1278 | "])" 1279 | ], 1280 | "execution_count": 28, 1281 | "outputs": [] 1282 | }, 1283 | { 1284 | "cell_type": "code", 1285 | "metadata": { 1286 | "id": "P3HeMf-SxH2s", 1287 | "colab": { 1288 | "base_uri": "https://localhost:8080/" 1289 | }, 1290 | "outputId": "0f23f570-b5ae-4161-828c-783b2bce718c" 1291 | }, 1292 | "source": [ 1293 | "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n", 1294 | "model.fit(X_train, y_train, epochs=5)\n", 1295 | "model.save('drive/MyDrive/cicids/FFNN_MC')" 1296 | ], 1297 | "execution_count": 29, 1298 | "outputs": [ 1299 | { 1300 | "output_type": "stream", 1301 | "name": "stdout", 1302 | "text": [ 1303 | "Epoch 1/5\n", 1304 | "26215/26215 [==============================] - 59s 2ms/step - loss: 0.0174 - accuracy: 0.9971\n", 1305 | "Epoch 2/5\n", 1306 | "26215/26215 [==============================] - 58s 2ms/step - loss: 0.0117 - accuracy: 0.9983\n", 1307 | "Epoch 3/5\n", 1308 | "26215/26215 [==============================] - 61s 2ms/step - loss: 0.0115 - accuracy: 0.9983\n", 1309 | "Epoch 4/5\n", 1310 | "26215/26215 [==============================] - 57s 2ms/step - loss: 0.0114 - accuracy: 0.9983\n", 1311 | "Epoch 5/5\n", 1312 | "26215/26215 [==============================] - 59s 2ms/step - loss: 0.0114 - accuracy: 0.9983\n" 1313 | ] 1314 | } 1315 | ] 1316 | }, 1317 | { 1318 | "cell_type": "code", 1319 | "metadata": { 1320 | "id": "njwo7PmSxNcW", 1321 | "colab": { 1322 | "base_uri": "https://localhost:8080/" 1323 | }, 1324 | "outputId": "76f5e0cc-6f73-4954-d644-5ca04ac5abf2" 1325 | }, 1326 | "source": [ 1327 | "predictions = model.predict(X_test.values)[:,:1]\n", 1328 | "predictions = predictions.argmax(axis=1)\n", 1329 | "y_test = y_test.argmax(axis=1)" 1330 | ], 1331 | "execution_count": 30, 1332 | "outputs": [ 1333 | { 1334 | "output_type": "stream", 1335 | "name": "stdout", 1336 | "text": [ 1337 | "6554/6554 [==============================] - 8s 1ms/step\n" 1338 | ] 1339 | } 1340 | ] 1341 | }, 1342 | { 1343 | "cell_type": "code", 1344 | "metadata": { 1345 | "id": "IjJTIqmmPbMR", 1346 | "outputId": "3379df0f-7229-4e97-bb0a-0f1e2c00d36b", 1347 | "colab": { 1348 | "base_uri": "https://localhost:8080/" 1349 | } 1350 | }, 1351 | "source": [ 1352 | "predictions.shape\n", 1353 | "y_test.shape" 1354 | ], 1355 | "execution_count": 31, 1356 | "outputs": [ 1357 | { 1358 | "output_type": "execute_result", 1359 | "data": { 1360 | "text/plain": [ 1361 | "(209715,)" 1362 | ] 1363 | }, 1364 | "metadata": {}, 1365 | "execution_count": 31 1366 | } 1367 | ] 1368 | }, 1369 | { 1370 | "cell_type": "code", 1371 | "metadata": { 1372 | "id": "jaxlBVw8MBhn", 1373 | "outputId": "ea8acb77-9010-4172-80f9-10070e89cf58", 1374 | "colab": { 1375 | "base_uri": "https://localhost:8080/" 1376 | } 1377 | }, 1378 | "source": [ 1379 | "np.sum(predictions == y_test) / len(y_test)" 1380 | ], 1381 | "execution_count": 32, 1382 | "outputs": [ 1383 | { 1384 | "output_type": "execute_result", 1385 | "data": { 1386 | "text/plain": [ 1387 | "0.42555849605416873" 1388 | ] 1389 | }, 1390 | "metadata": {}, 1391 | "execution_count": 32 1392 | } 1393 | ] 1394 | }, 1395 | { 1396 | "cell_type": "code", 1397 | "metadata": { 1398 | "id": "9oQAaNYgxP5A", 1399 | "colab": { 1400 | "base_uri": "https://localhost:8080/" 1401 | }, 1402 | "outputId": "c484151e-9403-4ecf-9744-97bcbed8a49f" 1403 | }, 1404 | "source": [ 1405 | "confMat = confusion_matrix(y_test, predictions)\n", 1406 | "confMat" 1407 | ], 1408 | "execution_count": 33, 1409 | "outputs": [ 1410 | { 1411 | "output_type": "execute_result", 1412 | "data": { 1413 | "text/plain": [ 1414 | "array([[89246, 0, 0],\n", 1415 | " [92371, 0, 0],\n", 1416 | " [28098, 0, 0]])" 1417 | ] 1418 | }, 1419 | "metadata": {}, 1420 | "execution_count": 33 1421 | } 1422 | ] 1423 | }, 1424 | { 1425 | "cell_type": "code", 1426 | "metadata": { 1427 | "id": "KKlZORPDWv5y", 1428 | "outputId": "357a35a7-078b-4c1b-95b5-347bda7022ad", 1429 | "colab": { 1430 | "base_uri": "https://localhost:8080/" 1431 | } 1432 | }, 1433 | "source": [ 1434 | "precision_score(y_test, predictions, average='weighted')" 1435 | ], 1436 | "execution_count": 34, 1437 | "outputs": [ 1438 | { 1439 | "output_type": "stream", 1440 | "name": "stderr", 1441 | "text": [ 1442 | "/usr/local/lib/python3.8/dist-packages/sklearn/metrics/_classification.py:1318: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n", 1443 | " _warn_prf(average, modifier, msg_start, len(result))\n" 1444 | ] 1445 | }, 1446 | { 1447 | "output_type": "execute_result", 1448 | "data": { 1449 | "text/plain": [ 1450 | "0.18110003356388596" 1451 | ] 1452 | }, 1453 | "metadata": {}, 1454 | "execution_count": 34 1455 | } 1456 | ] 1457 | }, 1458 | { 1459 | "cell_type": "code", 1460 | "metadata": { 1461 | "id": "J_4fMPJTXAto", 1462 | "outputId": "63230c51-c283-40ac-da9e-71c3d41fc318", 1463 | "colab": { 1464 | "base_uri": "https://localhost:8080/" 1465 | } 1466 | }, 1467 | "source": [ 1468 | "recall_score(y_test, predictions, average='weighted')" 1469 | ], 1470 | "execution_count": 35, 1471 | "outputs": [ 1472 | { 1473 | "output_type": "execute_result", 1474 | "data": { 1475 | "text/plain": [ 1476 | "0.42555849605416873" 1477 | ] 1478 | }, 1479 | "metadata": {}, 1480 | "execution_count": 35 1481 | } 1482 | ] 1483 | }, 1484 | { 1485 | "cell_type": "code", 1486 | "metadata": { 1487 | "id": "21H96b-dQ1m2", 1488 | "outputId": "1c1dd5d2-2ae8-4acb-9604-0fd8d30615e5", 1489 | "colab": { 1490 | "base_uri": "https://localhost:8080/", 1491 | "height": 282 1492 | } 1493 | }, 1494 | "source": [ 1495 | "cf_matrix = confusion_matrix(y_test, predictions)\n", 1496 | "sn.heatmap(cf_matrix / np.sum(cf_matrix), annot=True, fmt='.2%', cmap='Blues')" 1497 | ], 1498 | "execution_count": 36, 1499 | "outputs": [ 1500 | { 1501 | "output_type": "execute_result", 1502 | "data": { 1503 | "text/plain": [ 1504 | "" 1505 | ] 1506 | }, 1507 | "metadata": {}, 1508 | "execution_count": 36 1509 | }, 1510 | { 1511 | "output_type": "display_data", 1512 | "data": { 1513 | "text/plain": [ 1514 | "
" 1515 | ], 1516 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAAD4CAYAAADbyJysAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwV1f3/8dcnCRGsgBsEhYgoWARcQED8uoAoEBZBVL7iUkDFVEt+bq0Wl6JitdZaLd8CCiJ+0a8Vd40SocrijoJL2RQbKEJQEisIUoGQ5PP7417jTQjJjdyQueP76WMe3jlzzsyZ0ceHw5lzzpi7IyIi9S+lvisgIiIRCsgiIgGhgCwiEhAKyCIiAaGALCISEGl1fYFGJ43VMI46tunNu+u7CiIJ0TAN29NzNOqcE3fM2fbRxD2+XiKphSwiEhB13kIWEdmrLHnbmQrIIhIuKan1XYMfTQFZRMLFAtUtXCsKyCISLuqyEBEJCLWQRUQCQi1kEZGAUAtZRCQgNMpCRCQg1GUhIhIQ6rIQEQkItZBFRAJCAVlEJCBS9VJPRCQY1IcsIhIQSdxlkbw1FxGpiln8W42nsiwzW2lm+WY2tpp855qZm1nXmLQbo+VWmlm/eKquFrKIhEuCWshmlgpMAvoABcAiM8t19xWV8jUGrgbei0nrAAwHOgKHAq+Z2VHuXlrdNdVCFpFwSVwLuTuQ7+6r3b0YmAkMqSLfHcAfge0xaUOAme6+w93/BeRHz1ctBWQRCZeU1Lg3M8s2s8UxW3bMmVoC62L2C6Jp5cysC5Dp7rMq1aLGslVRl4WIhEstuizcfSow9UddxiwFuA8Y9WPKV0UBWUTCJXHD3tYDmTH7raJp32sMdAIWWOSaLYBcMxscR9kqqctCRMLFUuLfqrcIaGdmbcwsnchLutzvD7r7Znc/2N0Pd/fDgYXAYHdfHM033Mz2MbM2QDvg/ZouqBayiIRLgkZZuHuJmeUAc4BUYLq7Lzez8cBid8+tpuxyM3sKWAGUAGNqGmEBCsgiEjYJXA/Z3fOAvEpp43aTt1el/TuBO2tzPQVkEQkXTZ0WEQmIJJ46rYAsIuGiFrKISDCYArKISDAoIIuIBISlJG9ATt7e791ISTHenXEVz947EoBHbjuff8z8NYv/7xoevPk80lKrvuWtb93FwhlXsXDGVTx9z4gKx277ZV+WPPlrPnriOn417L8AOLtXJz54/Fpee+CXHNhkXwDatDyQx+64oA7vLnjefvMNBg/sx6CsPjz80K4zUIuLi7n+19cwKKsPFw0fxvr1BeXHHn5oCoOy+jB4YD/efutNADZu3MjIiy/gnCGDmDf3tfK8V+dcSVFRYd3fUADpGdeOmcW9BU3oAnLOf5/MyjVF5fsz53zMccP/TNeL/0Kj9DQuGdytynLbduykx8j/ocfI/2HYDY+Wp/9i4Am0ytif44bfR+cL7uPp1/4BwJXD/otTLp3ItBfe4/y+xwNwW3Zfbpvy9zq8u2ApLS3lrjvHM/nBaTyfO4vZeS+zKj+/Qp7nn32aJk2a8PLsV7l4xCj+ct+9AKzKz2d23iyey53F5CnTuOv3t1NaWsoreS8z7PzhPD7zaR5/bAYAC+bPo/3RHWjePGOv32N90zOuPQXkgGjZrAlZJ7fnkdxF5Wlz3l1Z/nvxJwW0bN60VufMPqcHd02fi7sD8NWm/wBQVubsk57Gvg3T2VlaysnHHU7hxq2sKvg6AXeSHJYtXUJmZmtaZWbSID2drAEDWTB/boU88+fNY/CQoQD06duP9xe+i7uzYP5csgYMJD09nVatMsnMbM2ypUtokJbG9m3b2VlcTEpKCiUlJTz+2AxGXTq6Pm6x3ukZ154CckD86ZqzuHniK5SV+S7H0lJTuCCrM68u/KzKsg3T03hreg6vP/QrzjqtQ3l6m5YHct4Zx/LW9BxeuO8Sjmx1UORaj85n1v9cxoBTjuapv3/M2Et684dH5lZ57rAqKiykxSEtyvebZ2RQWFjxr7xFRYW0aHEIAGlpaezXuDHffLOJwsJCMlr8UDajRQZFhYX0H3gWC+bP5ZeXX8Lo7Ct4cubfGHTWEBo1arR3bipg9Ix/BKvFFjA1vtQzs/ZEFlv+fi3P9UCuu39SlxWrrf4nt6do01Y+WrmeUzsfscvxCdefzdsf/4u3/7GmyvI/P+ePfPHVFg4/9EBmT7ycZas28K/1G9mnQRo7iks45dKJDOnZkSk3n8eZV05h3qJ85l0yEYAL+3dhzrsraZd5MNdceBqbvt3Gb+5/iW07dtblLYdS48aNmfhApJ90y+bNTJ82lfsnTOT2cbewZcsWRoy6hOOO71zPtUxuYX/GQWz5xqvaFrKZ/ZbIKvlGZKWi96O/n6jh+1Lliz6XFH6cyPru1knHtmbQqR349Lnf8ugdF9DrhCOZfuv5ANx06Rk02/9n3DCh8hrSP/jiqy0ArPliI298uJrjjzoUgPVfbeaFBcsAePH15XRqe0iFco32acAvBpzAg8+8yy2j+zD6jqd4Z8kahvc7vi5uM1CaZ2Sw4csN5ftFhYVkZFTsg2zePIMNG74EoKSkhK3ffsv++x9ARkYGhRt+KFu4oZDmlcpOeXAyo7Ov4JW8WXTucgJ33HU3D0yaWId3FDx6xrWXkpIS9xY0NdXoMqCbu9/t7v8X3e4m8imSy3ZXyN2nuntXd++alrF3AtO4B+bQdsgfaH/OHxnxuydY8MEqLr39SUad1Y0+PY5ixK1PlPcDV7Z/40akN4gsSHJQ03056djWfPKvyIvBl15fQc8TjgTg1M5HkL/2qwplr73oNCY//TYlpWU02qcB7pH+5X0bptfh3QZDx07HsHbtGgoK1rGzuJjZebPoeXrvCnl6nd6b3BefB+DVv8+h+4k9MDN6nt6b2XmzKC4upqBgHWvXrqHTMceWl/v88zUUFW6gW/cT2b59G5YS6fPbsWM7PyV6xrWXzH3INXVZlBH5QN/nldIPiR4LvL/ecDZrN3zDgqm/AiKt3D9Mn0uX9i0ZPbQHv/rDs7Q/vBl//e05lJU5KSnGvY8t4NPoSI17H1vAI7cN5/8NP4X/fLeDK//wXPm5Dzm4MV07ZHLX9Ejf8QPPvMNb03PYvHUb//3bx/b+ze5laWlp3HjzOK7MHk1ZWSlnDz2Xtm3bMemvE+jYsRO9ep/B0HPP4+ax1zMoqw9NmjblnnvvB6Bt23b0zerP0MEDSE1N5aZbxpGa+sMqXRMn3E/O1dcCkDVgENdeNYbp0x5iTM5V9XKv9UXP+EcIXpyNm+2u1QiRT2ADE4F/8sP3oQ4D2gI57j67pgs0Omns7i8gCbHpzbvruwoiCdEwbc/D6cGjZsYdc/79v8MDFb6rbSG7+2wzO4pIF0XsS71F8Sy2LCKytwWxKyJeNY6ycPcyIp8mEREJvEROnY72Ekwg8sWQadF3aLHHrwDGAKXAViDb3VeY2eHAJ8D3EyEWuvsVNV1Pa1mISKgkqoVsZqnAJKAPUAAsMrNcd18Rk+1v7v5gNP9gIl+hzooeW+XutRrVELxxHyIieyCBoyy6A/nuvtrdi4kMAR4Sm8Hdt8Ts/gzYo3dmCsgiEiq1CcixcyaiW3bMqVryw2AGiLSSW1KJmY0xs1XAPUDsEJU2ZvaRmb1uZqfGU3d1WYhIqNSmy8LdpwK7LqFXC+4+CZhkZhcCtwAjgS+Bw9z9azM7AXjBzDpWalHvQi1kEQmXxK1lsR7IjNlvFU3bnZnA2QDuvsPdv47+/gBYBRxV0wUVkEUkVBI4dXoR0M7M2phZOjAcyI3NYGbtYnYHEpmzgZk1i74UxMyOANoBq2u6oLosRCRUEjXKwt1LzCwHmENk2Nt0d19uZuOBxe6eC+SY2ZnATmATke4KgNOA8Wa2k8is5ivcfWNN11RAFpFwSeC8EHfPA/IqpY2L+X31bso9Czxb2+spIItIqIR6pp6ISDJRQBYRCQgFZBGRgEjkWhZ7mwKyiISKWsgiIgGhgCwiEhBJHI8VkEUkXNRCFhEJiBS91BMRCYYkbiArIItIuKiFLCISEGohi4gEhF7qiYgERBLHYwVkEQmXOBaeDywFZBEJlWRuISfvHyUiIlWozVen4zhXlpmtNLN8MxtbxfErzGypmX1sZm+ZWYeYYzdGy600s37x1F0BWURCxSz+rfrzWCowCegPdAAuiA24UX9z92Pc/XjgHuC+aNkORL7B1xHIAiZ//4296iggi0ioJLCF3B3Id/fV7l5M5KvSQ2IzuPuWmN2fAR79PQSYGf369L+A/Oj5qqU+ZBEJldr0IZtZNpAdkzTV3adGf7cE1sUcKwBOrOIcY4DrgHSgd0zZhZXKtqypPgrIIhIqtZmpFw2+U2vMWP05JgGTzOxC4BZ++PJ0rdV9QN6+tc4vISLyvQRODFkPZMbst4qm7c5M4IEfWRZQH7KIhEyiXuoBi4B2ZtbGzNKJvKTLrXgtaxezOxD4Z/R3LjDczPYxszZAO+D9mi6oLgsRCZVEtZDdvcTMcoA5QCow3d2Xm9l4YLG75wI5ZnYmsBPYRLS7IprvKWAFUAKMcffSmq6pgCwioZLIiSHungfkVUobF/P76mrK3gncWZvrKSCLSKho+U0RkYDQam8iIgGhgCwiEhBJHI8VkEUkXNRCFhEJiCSOxwrIIhIuGmUhIhIQKUncRFZAFpFQSeJ4rIAsIuGil3oiIgGRxF3ICsgiEi56qSciEhCGArKISCAkcQNZAVlEwiWZX+rpiyEiEioJ/GIIZpZlZivNLN/MxlZx/DozW2FmS8xsrpm1jjlWamYfR7fcymWrohayiIRKoiaGmFkqMAnoQ+Sr0YvMLNfdV8Rk+wjo6u7fmdmVwD3A+dFj29z9+NpcUy1kEQmVlBSLe6tBdyDf3Ve7ezGRj5gOic3g7vPd/bvo7kIiHzP98XXfk8IiIkFTmy4LM8s2s8UxW3bMqVoC62L2C6Jpu3MZ8ErMfsPoORea2dnx1F1dFiISKrXpsnD3qcDUPb2mmV0MdAV6xiS3dvf1ZnYEMM/Mlrr7qurOoxayiISK1WKrwXogM2a/VTSt4vUiX52+GRjs7ju+T3f39dF/rwYWAJ1ruqACsoiEipnFvdVgEdDOzNqYWTowHKgwWsLMOgNTiATjopj0A8xsn+jvg4GTgdiXgVVSl4WIhEqiJoa4e4mZ5QBzgFRgursvN7PxwGJ3zwX+BOwHPB0N8GvdfTBwNDDFzMqINHzvrjQ6o0oKyCISKolcy8Ld84C8SmnjYn6fuZty7wDH1PZ6CsgiEirJPFNPAVlEQkVrWYiIBIRayCIiAZG84VgBWURCJjWJ+yxCNw45JcV494nf8uyEKyqk//mG8/jq7T/vttxvLu3Lshdv5R/P/44zTzq6PP3TWbez6KmbWDhzLG89fkN5+u+vGsL7T97ItDt+UZ42fEA3ci7slbibSQJvv/kGgwf2Y1BWHx5+aNcJT8XFxVz/62sYlNWHi4YPY/36gvJjDz80hUFZfRg8sB9vv/UmABs3bmTkxRdwzpBBzJv7Wnneq3OupKiosO5vKID0jGsngeOQ97rQBeScC09n5b8q/k/VpcNh7N94392WaX9EC4b160KX8+5k8JjJTLjxvysMncnKnkCP4XdzykX3ANBkv4Ycf3Qm3c//A8U7S+nY9lAa7tOAEYN78OBTb9TNjQVQaWkpd905nskPTuP53FnMznuZVfn5FfI8/+zTNGnShJdnv8rFI0bxl/vuBWBVfj6z82bxXO4sJk+Zxl2/v53S0lJeyXuZYecP5/GZT/P4YzMAWDB/Hu2P7kDz5hl7/R7rm55x7SVy+c29LVQBuWXz/ck6pSOPPP9OeVpKinHXNWdz84QXdltuUK9jeXrOhxTvLOHzL75m1bp/063T4bvNX1bmNEhLBWDfhunsLCnlmhFn8MDM1ykpKUvY/QTdsqVLyMxsTavMTBqkp5M1YCAL5s+tkGf+vHkMHjIUgD59+/H+wndxdxbMn0vWgIGkp6fTqlUmmZmtWbZ0CQ3S0ti+bTs7i4tJSUmhpKSExx+bwahLR9fHLdY7PePaSzGLewuaUAXkP11/LjdPeIGyMi9Pu/L8nsx6fSkb/r1lt+VaNmtKwYZN5fvrizZxaPOmALg7L03O4e3Hb+DSc04GYOt3O5jz1nIWzhzLhn9vZsvWbXTrdDgvLVhSR3cWTEWFhbQ4pEX5fvOMDAoLK/7tpKiokBYtDgEgLS2N/Ro35ptvNlFYWEhGix/KZrTIoKiwkP4Dz2LB/Ln88vJLGJ19BU/O/BuDzhpCo0aN9s5NBYyece0lcwv5R7/UM7NL3P2R3RzLBrIB0lr1Iu3gjj/2MnHrf2onijZ+y0efrOPUE9oBcEizppzTpzN9L5/wo897xiX388VXm2l2wH68/GAOK9ds4O0PV3HfjNe4b0ak/23yuAu544GXGTX0JM7scTRL/7meP06bk5D7+qlp3LgxEx+I9JNu2byZ6dOmcv+Eidw+7ha2bNnCiFGXcNzxNa7RItUI+zMOYt9wvPakhXz77g64+1R37+ruXfdGMAY46fgjGNTzGD6ddTuP3n0JvbodxQfP3MwRmc1Ynnsrn866nX0bNmDZi7fuUnb9V5tp1eKA8v2WzQ/gi6LNAHzxVeTfX23aSu68JXTreHiFssf9vBVm8NmaIs45swsX/3Y6R7RqxpGHNau7mw2I5hkZbPhyQ/l+UWEhGRkV+yCbN89gw4YvASgpKWHrt9+y//4HkJGRQeGGH8oWbiikeaWyUx6czOjsK3glbxadu5zAHXfdzQOTJtbhHQWPnnHtpZrFvQVNtQE5+p2oqralQKB6/8f9NZe2Wb+j/cBbGTH2ERYs+oxDe95Amz430X7grbQfeCvfbd9JpyG7/jkya8EShvXrQnqDNFofehBtD2vGomVr2LdhOvvtuw8Q6Ss+86T2LF/1RcXr/moQ4yfPokFaKqmpkf/AZV7Gvg3T6/6m61nHTsewdu0aCgrWsbO4mNl5s+h5eu8KeXqd3pvcF58H4NW/z6H7iT0wM3qe3pvZebMoLi6moGAda9euodMxx5aX+/zzNRQVbqBb9xPZvn0blhJ5K75jx/a9eo/1Tc+49lIs/i1oauqyyAD6AZsqpRvwzq7Zk8fAnsfQpcNh3PHALD5ZvYFn//4RHz17MyWlZVxz91OUlTnND2rMk/ddDkBaaipPvrKYV9/5pPwcZ/U6lg9XrOXLaCt6ycr1LHrqJpb9cz1LP9tl2dTQSUtL48abx3Fl9mjKyko5e+i5tG3bjkl/nUDHjp3o1fsMhp57HjePvZ5BWX1o0rQp99x7PwBt27ajb1Z/hg4eQGpqKjfdMo7U1NTyc0+ccD85V18LQNaAQVx71RimT3uIMTlX1cu91hc949oLYqCNl7n77g+aPQw84u5vVXHsb+5+YU0XaNQ5Z/cXkITYtCi5/4op8r2GaXs+0e7XL62MO+b8+ayfByp8V9tCdvfLqjlWYzAWEdnbkrmFrKnTIhIqAXxXF7dQjUMWEUkzi3uriZllmdlKM8s3s7FVHL/OzFZEBzvMNbPWMcdGmtk/o9vIeOqugCwioZKoiSFmlgpMAvoDHYALzKxDpWwfAV3d/VjgGeCeaNkDgVuBE4HuwK1mdgA1UEAWkVBJ4NTp7kC+u69292JgJjAkNoO7z3f376K7C4l8mRoio9NedfeN7r4JeBXIqrHutbhPEZHAq00L2cyyzWxxzJYdc6qWwLqY/YJo2u5cBrzyI8sCeqknIiFTm1EW7j4V2HVN01oys4uBrkDPPTmPArKIhEoCF6hfD2TG7LeKplVgZmcCNwM93X1HTNlelcouqOmC6rIQkVBJ4NTpRUA7M2tjZunAcCA3NoOZdQamAIPdvSjm0Bygr5kdEH2Z1zeaVi21kEUkVCxBX9Vz9xIzyyESSFOB6e6+3MzGA4vdPRf4E7Af8HR0lbm17j7Y3Tea2R1EgjrAeHffWNM1FZBFJFQSOVPP3fOAvEpp42J+n1lN2enA9NpcTwFZREJFU6dFRAIimReoV0AWkVBJTeKhCgrIIhIqQfx4abwUkEUkVNSHLCISEEncQFZAFpFwSUnQOOT6oIAsIqGiFrKISECkJXEnsgKyiISKWsgiIgGhYW8iIgGRxPFYAVlEwiWJJ+opIItIuKjLQkQkIBSQRUQCInnDcXJ3t4iI7KI2X52u+VyWZWYrzSzfzMZWcfw0M/vQzErM7LxKx0rN7OPollu5bFXUQhaRUEnUeshmlgpMAvoABcAiM8t19xUx2dYCo4DfVHGKbe5+fG2uqYAsIqGSwL/2dwfy3X01gJnNBIYA5QHZ3ddEj5Ul4oLqshCRUEkxi3szs2wzWxyzZcecqiWwLma/IJoWr4bRcy40s7PjKVDnLeRn/29czZlERBKkNl0W7j4VmFpHVWnt7uvN7AhgnpktdfdV1RVQC1lEQiWlFlsN1gOZMfutomlxcff10X+vBhYAneOpu4hIaFikKyKurQaLgHZm1sbM0oHhQFyjJczsADPbJ/r7YOBkYvqed0cBWURCxWqxVcfdS4AcYA7wCfCUuy83s/FmNhjAzLqZWQEwDJhiZsujxY8GFpvZP4D5wN2VRmdUSaMsRCRUUhM4U8/d84C8SmnjYn4vItKVUbncO8Axtb2eArKIhEoSz5xWQBaRcLEknjytgCwioaIWsohIQOir0yIiAaEWsohIQGg9ZBGRgEhJ3nisgCwi4aJRFiIiAZHEPRYKyCISLmohi4gEhPqQRUQCQqMsREQCInnDsQKyiISMWsgiIgGRvOFYC9SLSNgkaoV6wMyyzGylmeWb2dgqjp9mZh+aWYmZnVfp2Egz+2d0GxlP1dVCFpFQSVSXhZmlApOAPkS+OL3IzHIrffljLTAK+E2lsgcCtwJdAQc+iJbdVG3dE1JzEZGASGADuTuQ7+6r3b0YmAkMic3g7mvcfQlQVqlsP+BVd98YDcKvAlk1XVABWUTCpRYR2cyyzWxxzJYdc6aWwLqY/YJoWjx+VFl1WYhIqNRmpp67TwWm1l1takctZBEJFbP4txqsBzJj9ltF0+Lxo8oqIItIqCSwD3kR0M7M2phZOjAcyI2zGnOAvmZ2gJkdAPSNplVLAVlEQsXM4t6q4+4lQA6RQPoJ8JS7Lzez8WY2OHqtbmZWAAwDppjZ8mjZjcAdRIL6ImB8NK36urv7Htx6zfKWF9XtBYTeP29e31UQSYiGaXs+r+Pjtd/GHXOOP6xxoOaR6KWeiIRKoCJsLSkgi0i4JHFEVkAWkVBJ5gXqQ/VS74mJf+B3o87ij1ePKE/L+9s07rl2JH+67hIeuP06Nm/8927Lb//uP9w2+hyefej+8rR1q1ZyzzUjufNXw3lu2l/4vs/9pUcf4J5rR/L4hN+X5138+hxef+mpOriz4Hr7zTcYPLAfg7L68PBDuw7nLC4u5vpfX8OgrD5cNHwY69cXlB97+KEpDMrqw+CB/Xj7rTcB2LhxIyMvvoBzhgxi3tzXyvNenXMlRUWFdX9DAaRnXDsJHPa214UqIHc/vT/Zv7u3Qlrvsy/ghvtncP19j9Cx638x56n/3W35vCemcWTH4yqkPTPlz/z3lTdw06Qn+OrLAj796D22/WcrBas/44b7Z5CalsYXn6+ieMcO3p+Xxyn9z6mLWwuk0tJS7rpzPJMfnMbzubOYnfcyq/LzK+R5/tmnadKkCS/PfpWLR4ziL/dF/vusys9ndt4snsudxeQp07jr97dTWlrKK3kvM+z84Tw+82kef2wGAAvmz6P90R1o3jxjr99jfdMzrj0F5IA4suPx/KxxkwppDff9Wfnv4u3bdvsfYd2qlWz9ZiM/P65bedrmjf9m+7b/cPjPO2JmdOuVxdL33sRSUigtLcHd2Vm8g9TUNBa8+ASnDjiX1LSfTi/QsqVLyMxsTavMTBqkp5M1YCAL5s+tkGf+vHkMHjIUgD59+/H+wndxdxbMn0vWgIGkp6fTqlUmmZmtWbZ0CQ3S0ti+bTs7i4tJSUmhpKSExx+bwahLR9fHLdY7PePas1r8EzShCsi7M+vxqdx++bl88Mar9B9+2S7Hy8rKePF/JzJ45JgK6Zs3/pumBzUr3296UDM2b/yKho325eguPbj315fSZP+DaLjvz/j8nys45sTT6vxegqSosJAWh7Qo32+ekUFhYcW/8hYVFdKixSEApKWlsV/jxnzzzSYKCwvJaPFD2YwWGRQVFtJ/4FksmD+XX15+CaOzr+DJmX9j0FlDaNSo0d65qYDRM669ZG4h19icM7P2RBbFeM/dt8akZ7n77LqsXKIMvCibgRdl89qzj/HmK8/tEpTfnv08R3fpwf4Hxz+e94yhF3HG0IsAmDnpbrKGX8bCV1/i038s4tDWR9J3WFzLn0oljRs3ZuIDkX7SLZs3M33aVO6fMJHbx93Cli1bGDHqEo47vnM91zK5hf0ZBzDOxq3aFrKZXQW8CPw/YJmZxS49d1c15cpXUHrl6UcTU9MEOOG0vix59/Vd0tesXM5brzzH+F8OI3fGZBYtmM1Ljz1I0wMPZvPXX5Xn2/z1VzQ9sFmFsgWrPwOgecvD+Pjd+Yz6zXi+3rCer75YR9g1z8hgw5cbyveLCgvJyKjYB9m8eQYbNnwJQElJCVu//Zb99z+AjIwMCjf8ULZwQyHNK5Wd8uBkRmdfwSt5s+jc5QTuuOtuHpg0sQ7vKHj0jH+EBM6d3ttq6rK4HDjB3c8GegG/M7Oro8d2ezvuPtXdu7p71/7DRuwu214RGxiXvv8mzVsetkueX1w7jlunPsu4KU8zeOSv6NYri7N+cQVNDzyYho1+xpqVy3F3Fi2YTafup1Qo+8oT0+h/wWjKSksoK4ssiWopKRTv2FG3NxYAHTsdw9q1aygoWMfO4mJm582i5+m9K+TpdXpvcl98HoBX/z6H7if2wMzoeXpvZufNori4mIKCdaxdu4ZOxxxbXu7zz9dQVLiBbt1PZPv2bVhKZKrrjh3b9+o91jc949pLMYt7C5qauixSvu+mcPc1ZtYLeMbMWhPAP18eve828pd9xH++3cxto88ha/ilfPLhQorWr8VSjAOatWDYLyML+6/N/1sGRO0AAAXpSURBVJR35rzA8DG7fJWlgnOzr+OJv97FzuIdHN2lB0d36VF+bOl7b5B5ZHuaHngwAC0Pb8c914zkkNZH0rJN27q70YBIS0vjxpvHcWX2aMrKSjl76Lm0bduOSX+dQMeOnejV+wyGnnseN4+9nkFZfWjStCn33BsZUti2bTv6ZvVn6OABpKamctMt40hNTS0/98QJ95Nz9bUAZA0YxLVXjWH6tIcYk3NVvdxrfdEzrr3ABaZaqHYtCzObB1zn7h/HpKUB04GL3D11t4WjtJZF3dNaFhIWiVjL4rPC7+KOOUdl7Buo+F1Tl8UIYENsgruXuPsI4Kc1pEBEkkIyD3urtsvC3QuqOfZ24qsjIrJnAtg1HLefziwGEflJSOJ4rIAsIuFS08LzQfaTmKknIj8diZypZ2ZZZrbSzPLNbJchWWa2j5k9GT3+npkdHk0/3My2mdnH0e3BeOquFrKIhEqi2sdmlgpMAvoABcAiM8t19xUx2S4DNrl7WzMbDvwROD96bJW7H1+ba6qFLCLhkriZet2BfHdf7e7FwExgSKU8Q4AZ0d/PAGfYHvSZKCCLSKjUZthb7DIP0S075lQtgdg1EAqiaVSVJ/pR1M3AQdFjbczsIzN73cxOjafu6rIQkVCpTfvU3acCu676v+e+BA5z96/N7ATgBTPr6O5bqiukFrKIhEqKxb/VYD2QGbPfKppWZZ7oLOamwNfuvsPdvwZw9w+AVcBRNdY9nhsUEUkeCetEXgS0M7M2ZpYODAdyK+XJBb5fa/c8YJ67u5k1i74UxMyOANoBq2u6oLosRCRUEjUM2d1LzCwHmAOkAtPdfbmZjQcWu3su8DDwmJnlAxuJBG2ILC0x3sx2AmXAFe6+sca6V7e4UCJocaG6p8WFJCwSsbjQF98Uxx1zDt0/PVCzSNRCFpFQSeKJegrIIhIuyTx1WgFZREIlecOxArKIhEwSN5AVkEUkXIK48Hy8FJBFJFySNx4rIItIuCRxPFZAFpFwSUniTmQFZBEJlSSOx1rLQkQkKNRCFpFQSeYWsgKyiISKhr2JiASEWsgiIgGhgCwiEhDqshARCYhkbiFr2JuIhErCPuAEmFmWma00s3wzG1vF8X3M7Mno8ffM7PCYYzdG01eaWb946q6ALCLhkqCIHP0m3iSgP9ABuMDMOlTKdhmwyd3bAvcDf4yW7UDkc04dgSxg8vff2KuOArKIhEqKWdxbDboD+e6+2t2LgZnAkEp5hgAzor+fAc6wyAr5Q4CZ0a9P/wvIj56vWnXehzygY/Ok69Exs2x3n1rf9QgzPeO691N9xrX5Lp+ZZQPZMUlTY55ZS2BdzLEC4MRKpyjPE/0o6mbgoGj6wkplW9ZUH7WQq5ZdcxbZQ3rGdU/PuAbuPtXdu8Zs9foHmAKyiEjV1gOZMfutomlV5jGzNKAp8HWcZXehgCwiUrVFQDsza2Nm6URe0uVWypMLjIz+Pg+Y5+4eTR8eHYXRBmgHvF/TBTUOuWo/uX63eqBnXPf0jPdAtE84B5gDpALT3X25mY0HFrt7LvAw8JiZ5QMbiQRtovmeAlYAJcAYdy+t6ZoWCeYiIlLf1GUhIhIQCsgiIgGhgByjpmmSsufMbLqZFZnZsvquS1iZWaaZzTezFWa23Myuru86SXzUhxwVndb4GdCHyCDuRcAF7r6iXisWMmZ2GrAVeNTdO9V3fcLIzA4BDnH3D82sMfABcLb+Xw4+tZB/EM80SdlD7v4GkbfRUkfc/Ut3/zD6+1vgE+KYJSb1TwH5B1VNk9T/xJLUoquPdQbeq9+aSDwUkEVCysz2A54FrnH3LfVdH6mZAvIPftRUR5EgMrMGRILx4+7+XH3XR+KjgPyDeKZJigRedPnHh4FP3P2++q6PxE8BOcrdS4Dvp0l+Ajzl7svrt1bhY2ZPAO8CPzezAjO7rL7rFEInA78AepvZx9FtQH1XSmqmYW8iIgGhFrKISEAoIIuIBIQCsohIQCggi4gEhAKyiEhAKCCLiASEArKISED8f7nygDesqrUgAAAAAElFTkSuQmCC\n" 1517 | }, 1518 | "metadata": { 1519 | "needs_background": "light" 1520 | } 1521 | } 1522 | ] 1523 | }, 1524 | { 1525 | "cell_type": "code", 1526 | "metadata": { 1527 | "id": "5WoxcdTi-AdC", 1528 | "colab": { 1529 | "base_uri": "https://localhost:8080/" 1530 | }, 1531 | "outputId": "947c938d-4015-4d81-bb74-351bdf885299" 1532 | }, 1533 | "source": [ 1534 | "! pip3 install keras\n", 1535 | "! pip3 install ann_visualizer\n", 1536 | "! pip install graphviz\n", 1537 | "! pip install h5py" 1538 | ], 1539 | "execution_count": 37, 1540 | "outputs": [ 1541 | { 1542 | "output_type": "stream", 1543 | "name": "stdout", 1544 | "text": [ 1545 | "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", 1546 | "Requirement already satisfied: keras in /usr/local/lib/python3.8/dist-packages (2.9.0)\n", 1547 | "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", 1548 | "Collecting ann_visualizer\n", 1549 | " Downloading ann_visualizer-2.5.tar.gz (4.7 kB)\n", 1550 | "Building wheels for collected packages: ann-visualizer\n", 1551 | " Building wheel for ann-visualizer (setup.py) ... \u001b[?25l\u001b[?25hdone\n", 1552 | " Created wheel for ann-visualizer: filename=ann_visualizer-2.5-py3-none-any.whl size=4168 sha256=10a9e6a4d80111e731016f7bf03dbcfe2ca23f9ef4c632b4b9bc8625cff74b52\n", 1553 | " Stored in directory: /root/.cache/pip/wheels/4b/ef/77/9b8c4ae2f9a11de19957b80bc5c684accd99114bb8dc6b374c\n", 1554 | "Successfully built ann-visualizer\n", 1555 | "Installing collected packages: ann-visualizer\n", 1556 | "Successfully installed ann-visualizer-2.5\n", 1557 | "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", 1558 | "Requirement already satisfied: graphviz in /usr/local/lib/python3.8/dist-packages (0.10.1)\n", 1559 | "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", 1560 | "Requirement already satisfied: h5py in /usr/local/lib/python3.8/dist-packages (3.1.0)\n", 1561 | "Requirement already satisfied: numpy>=1.17.5 in /usr/local/lib/python3.8/dist-packages (from h5py) (1.21.6)\n" 1562 | ] 1563 | } 1564 | ] 1565 | }, 1566 | { 1567 | "cell_type": "code", 1568 | "metadata": { 1569 | "id": "GTWJZBBm2rUP" 1570 | }, 1571 | "source": [ 1572 | "from ann_visualizer.visualize import ann_viz;\n", 1573 | "ann_viz(model, title=\"Artificial Neural network - Model Visualization\")" 1574 | ], 1575 | "execution_count": 38, 1576 | "outputs": [] 1577 | }, 1578 | { 1579 | "cell_type": "code", 1580 | "source": [ 1581 | "! pip3 install keras-visualizer" 1582 | ], 1583 | "metadata": { 1584 | "colab": { 1585 | "base_uri": "https://localhost:8080/" 1586 | }, 1587 | "id": "FUP8Xmm-JPWr", 1588 | "outputId": "ec2c90f3-43c8-4b7d-8f5e-b2f4714eac09" 1589 | }, 1590 | "execution_count": 46, 1591 | "outputs": [ 1592 | { 1593 | "output_type": "stream", 1594 | "name": "stdout", 1595 | "text": [ 1596 | "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", 1597 | "Collecting keras-visualizer\n", 1598 | " Downloading keras_visualizer-2.4-py3-none-any.whl (5.4 kB)\n", 1599 | "Installing collected packages: keras-visualizer\n", 1600 | "Successfully installed keras-visualizer-2.4\n" 1601 | ] 1602 | } 1603 | ] 1604 | }, 1605 | { 1606 | "cell_type": "code", 1607 | "source": [ 1608 | "from keras_visualizer import visualizer" 1609 | ], 1610 | "metadata": { 1611 | "id": "d1rxPbzXJLVc" 1612 | }, 1613 | "execution_count": 47, 1614 | "outputs": [] 1615 | }, 1616 | { 1617 | "cell_type": "code", 1618 | "metadata": { 1619 | "id": "RaPSKJxr2-rO" 1620 | }, 1621 | "source": [ 1622 | "visualizer(model, format='png', view=True)" 1623 | ], 1624 | "execution_count": 49, 1625 | "outputs": [] 1626 | } 1627 | ] 1628 | } -------------------------------------------------------------------------------- /CIC IDS 2018/Group6_cicids_LSTM.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "collapsed_sections": [ 8 | "oEBYbD0jw2PI", 9 | "DER9DuHSa5hH" 10 | ], 11 | "include_colab_link": true 12 | }, 13 | "kernelspec": { 14 | "name": "python3", 15 | "display_name": "Python 3" 16 | }, 17 | "language_info": { 18 | "name": "python" 19 | } 20 | }, 21 | "cells": [ 22 | { 23 | "cell_type": "markdown", 24 | "metadata": { 25 | "id": "view-in-github", 26 | "colab_type": "text" 27 | }, 28 | "source": [ 29 | "\"Open" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "source": [ 35 | "\n", 36 | "## Intrusion Detection System (CIC IDS 2018)\n", 37 | "\n", 38 | "---\n", 39 | "\n", 40 | "\n", 41 | "### Group 6 - Sleety, Thejus, Tejas and Rahul \n", 42 | "#### Rajagiri School of Engineering and Technology (KTU 2019 Scheme)" 43 | ], 44 | "metadata": { 45 | "id": "siJMdbALhwDm" 46 | } 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": { 51 | "id": "oEBYbD0jw2PI" 52 | }, 53 | "source": [ 54 | "## **IDS**" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "metadata": { 60 | "id": "NLezzTSoMcI8" 61 | }, 62 | "source": [ 63 | "import tensorflow as tf\n", 64 | "import pandas as pd\n", 65 | "import numpy as np\n", 66 | "import sklearn\n", 67 | "from keras.models import Sequential, load_model\n", 68 | "from sklearn.model_selection import train_test_split\n", 69 | "from sklearn.linear_model import LogisticRegression\n", 70 | "from sklearn import preprocessing\n", 71 | "from sklearn.preprocessing import LabelEncoder, StandardScaler\n", 72 | "from sklearn.metrics import confusion_matrix\n", 73 | "from keras.utils import np_utils" 74 | ], 75 | "execution_count": null, 76 | "outputs": [] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "metadata": { 81 | "colab": { 82 | "base_uri": "https://localhost:8080/" 83 | }, 84 | "id": "9TWNodDe1kHu", 85 | "outputId": "887c20b7-c6a9-46d7-f419-ba689e8f6ae4" 86 | }, 87 | "source": [ 88 | "from google.colab import drive\n", 89 | "drive.mount('/content/drive')" 90 | ], 91 | "execution_count": null, 92 | "outputs": [ 93 | { 94 | "output_type": "stream", 95 | "name": "stdout", 96 | "text": [ 97 | "Mounted at /content/drive\n" 98 | ] 99 | } 100 | ] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "metadata": { 105 | "id": "KzFzH3CAMmxQ", 106 | "colab": { 107 | "base_uri": "https://localhost:8080/" 108 | }, 109 | "outputId": "6e9226bc-f84e-4540-f08b-5a01e0454f63" 110 | }, 111 | "source": [ 112 | "df = pd.read_csv('/content/drive/My Drive/cicids/cic/02-16-2018.csv')\n", 113 | "df.drop(df.loc[df['Label'] == 'Label'].index, inplace=True)" 114 | ], 115 | "execution_count": null, 116 | "outputs": [ 117 | { 118 | "output_type": "stream", 119 | "name": "stderr", 120 | "text": [ 121 | "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py:3326: DtypeWarning: Columns (0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78) have mixed types.Specify dtype option on import or set low_memory=False.\n", 122 | " exec(code_obj, self.user_global_ns, self.user_ns)\n" 123 | ] 124 | } 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "metadata": { 130 | "id": "HixtSvMpM_Fq" 131 | }, 132 | "source": [ 133 | "metadata = ['fl_dur' #Flow duration\n", 134 | ",'tot_fw_pk' #Total packets in the forward direction\n", 135 | ",'tot_bw_pk' #Total packets in the backward direction\n", 136 | ",'tot_l_fw_pkt' #Total size of packet in forward direction\n", 137 | ",'fw_pkt_l_max' #Maximum size of packet in forward direction\n", 138 | ",'fw_pkt_l_min' #Minimum size of packet in forward direction\n", 139 | ",'fw_pkt_l_avg' #Average size of packet in forward direction\n", 140 | ",'fw_pkt_l_std' #Standard deviation size of packet in forward direction\n", 141 | ",'Bw_pkt_l_max' #Maximum size of packet in backward direction\n", 142 | ",'Bw_pkt_l_min' #Minimum size of packet in backward direction\n", 143 | ",'Bw_pkt_l_avg' #Mean size of packet in backward direction\n", 144 | ",'Bw_pkt_l_std' #Standard deviation size of packet in backward direction\n", 145 | ",'fl_byt_s' #flow byte rate that is number of packets transferred per second\n", 146 | ",'fl_pkt_s' #flow packets rate that is number of packets transferred per second\n", 147 | ",'fl_iat_avg' #Average time between two flows\n", 148 | ",'fl_iat_std' #Standard deviation time two flows\n", 149 | ",'fl_iat_max' #Maximum time between two flows\n", 150 | ",'fl_iat_min' #Minimum time between two flows\n", 151 | ",'fw_iat_tot' #Total time between two packets sent in the forward direction\n", 152 | ",'fw_iat_avg' #Mean time between two packets sent in the forward direction\n", 153 | ",'fw_iat_std' #Standard deviation time between two packets sent in the forward direction\n", 154 | ",'fw_iat_max' #Maximum time between two packets sent in the forward direction\n", 155 | ",'fw_iat_min' #Minimum time between two packets sent in the forward direction\n", 156 | ",'bw_iat_tot' #Total time between two packets sent in the backward direction\n", 157 | ",'bw_iat_avg' #Mean time between two packets sent in the backward direction\n", 158 | ",'bw_iat_std' #Standard deviation time between two packets sent in the backward direction\n", 159 | ",'bw_iat_max' #Maximum time between two packets sent in the backward direction\n", 160 | ",'bw_iat_min' #Minimum time between two packets sent in the backward direction\n", 161 | ",'fw_psh_flag' #Number of times the PSH flag was set in packets travelling in the forward direction (0 for UDP)\n", 162 | ",'bw_psh_flag' #Number of times the PSH flag was set in packets travelling in the backward direction (0 for UDP)\n", 163 | ",'fw_urg_flag' #Number of times the URG flag was set in packets travelling in the forward direction (0 for UDP)\n", 164 | ",'bw_urg_flag' #Number of times the URG flag was set in packets travelling in the backward direction (0 for UDP)\n", 165 | ",'fw_hdr_len' #Total bytes used for headers in the forward direction\n", 166 | ",'bw_hdr_len' #Total bytes used for headers in the forward direction\n", 167 | ",'fw_pkt_s' #Number of forward packets per second\n", 168 | ",'bw_pkt_s' #Number of backward packets per second\n", 169 | ",'pkt_len_min' #Minimum length of a flow\n", 170 | ",'pkt_len_max' #Maximum length of a flow\n", 171 | ",'pkt_len_avg' #Mean length of a flow\n", 172 | ",'pkt_len_std' #Standard deviation length of a flow\n", 173 | ",'pkt_len_va' #Minimum inter-arrival time of packet\n", 174 | ",'fin_cnt' #Number of packets with FIN\n", 175 | ",'syn_cnt' #Number of packets with SYN\n", 176 | ",'rst_cnt' #Number of packets with RST\n", 177 | ",'pst_cnt' #Number of packets with PUSH\n", 178 | ",'ack_cnt' #Number of packets with ACK\n", 179 | ",'urg_cnt' #Number of packets with URG\n", 180 | ",'cwe_cnt' #Number of packets with CWE\n", 181 | ",'ece_cnt' #Number of packets with ECE\n", 182 | ",'down_up_ratio' #Download and upload ratio\n", 183 | ",'pkt_size_avg' #Average size of packet\n", 184 | ",'fw_seg_avg' #Average size observed in the forward direction\n", 185 | ",'bw_seg_avg' #Average size observed in the backward direction\n", 186 | ",'fw_byt_blk_avg' #Average number of bytes bulk rate in the forward direction\n", 187 | ",'fw_pkt_blk_avg' #Average number of packets bulk rate in the forward direction\n", 188 | ",'fw_blk_rate_avg' #Average number of bulk rate in the forward direction\n", 189 | ",'bw_byt_blk_avg' #Average number of bytes bulk rate in the backward direction\n", 190 | ",'bw_pkt_blk_avg' #Average number of packets bulk rate in the backward direction\n", 191 | ",'bw_blk_rate_avg' #Average number of bulk rate in the backward direction\n", 192 | ",'subfl_fw_pk' #The average number of packets in a sub flow in the forward direction\n", 193 | ",'subfl_fw_byt' #The average number of bytes in a sub flow in the forward direction\n", 194 | ",'subfl_bw_pkt' #The average number of packets in a sub flow in the backward direction\n", 195 | ",'subfl_bw_byt' #The average number of bytes in a sub flow in the backward direction\n", 196 | ",'fw_win_byt' #Number of bytes sent in initial window in the forward direction\n", 197 | ",'bw_win_byt' ## of bytes sent in initial window in the backward direction\n", 198 | ",'Fw_act_pkt' ## of packets with at least 1 byte of TCP data payload in the forward direction\n", 199 | ",'fw_seg_min' #Minimum segment size observed in the forward direction\n", 200 | ",'atv_avg' #Mean time a flow was active before becoming idle\n", 201 | ",'atv_std' #Standard deviation time a flow was active before becoming idle\n", 202 | ",'atv_max' #Maximum time a flow was active before becoming idle\n", 203 | ",'atv_min' #Minimum time a flow was active before becoming idle\n", 204 | ",'idl_avg' #Mean time a flow was idle before becoming active\n", 205 | ",'idl_std' #Standard deviation time a flow was idle before becoming active\n", 206 | ",'idl_max' #Maximum time a flow was idle before becoming active\n", 207 | ",'idl_min' #Minimum time a flow was idle before becoming active\n", 208 | "]" 209 | ], 210 | "execution_count": null, 211 | "outputs": [] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "metadata": { 216 | "id": "s9ZjigalNCIZ", 217 | "colab": { 218 | "base_uri": "https://localhost:8080/" 219 | }, 220 | "outputId": "07e772aa-03b6-40dc-a892-cfd7323292f3" 221 | }, 222 | "source": [ 223 | "df.columns" 224 | ], 225 | "execution_count": null, 226 | "outputs": [ 227 | { 228 | "output_type": "execute_result", 229 | "data": { 230 | "text/plain": [ 231 | "Index(['Dst Port', 'Protocol', 'Timestamp', 'Flow Duration', 'Tot Fwd Pkts',\n", 232 | " 'Tot Bwd Pkts', 'TotLen Fwd Pkts', 'TotLen Bwd Pkts', 'Fwd Pkt Len Max',\n", 233 | " 'Fwd Pkt Len Min', 'Fwd Pkt Len Mean', 'Fwd Pkt Len Std',\n", 234 | " 'Bwd Pkt Len Max', 'Bwd Pkt Len Min', 'Bwd Pkt Len Mean',\n", 235 | " 'Bwd Pkt Len Std', 'Flow Byts/s', 'Flow Pkts/s', 'Flow IAT Mean',\n", 236 | " 'Flow IAT Std', 'Flow IAT Max', 'Flow IAT Min', 'Fwd IAT Tot',\n", 237 | " 'Fwd IAT Mean', 'Fwd IAT Std', 'Fwd IAT Max', 'Fwd IAT Min',\n", 238 | " 'Bwd IAT Tot', 'Bwd IAT Mean', 'Bwd IAT Std', 'Bwd IAT Max',\n", 239 | " 'Bwd IAT Min', 'Fwd PSH Flags', 'Bwd PSH Flags', 'Fwd URG Flags',\n", 240 | " 'Bwd URG Flags', 'Fwd Header Len', 'Bwd Header Len', 'Fwd Pkts/s',\n", 241 | " 'Bwd Pkts/s', 'Pkt Len Min', 'Pkt Len Max', 'Pkt Len Mean',\n", 242 | " 'Pkt Len Std', 'Pkt Len Var', 'FIN Flag Cnt', 'SYN Flag Cnt',\n", 243 | " 'RST Flag Cnt', 'PSH Flag Cnt', 'ACK Flag Cnt', 'URG Flag Cnt',\n", 244 | " 'CWE Flag Count', 'ECE Flag Cnt', 'Down/Up Ratio', 'Pkt Size Avg',\n", 245 | " 'Fwd Seg Size Avg', 'Bwd Seg Size Avg', 'Fwd Byts/b Avg',\n", 246 | " 'Fwd Pkts/b Avg', 'Fwd Blk Rate Avg', 'Bwd Byts/b Avg',\n", 247 | " 'Bwd Pkts/b Avg', 'Bwd Blk Rate Avg', 'Subflow Fwd Pkts',\n", 248 | " 'Subflow Fwd Byts', 'Subflow Bwd Pkts', 'Subflow Bwd Byts',\n", 249 | " 'Init Fwd Win Byts', 'Init Bwd Win Byts', 'Fwd Act Data Pkts',\n", 250 | " 'Fwd Seg Size Min', 'Active Mean', 'Active Std', 'Active Max',\n", 251 | " 'Active Min', 'Idle Mean', 'Idle Std', 'Idle Max', 'Idle Min', 'Label'],\n", 252 | " dtype='object')" 253 | ] 254 | }, 255 | "metadata": {}, 256 | "execution_count": 5 257 | } 258 | ] 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "metadata": { 263 | "id": "Hi48qgaewUOb" 264 | }, 265 | "source": [ 266 | "## **LSTM**" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": { 272 | "id": "H84qHurZa0Kt" 273 | }, 274 | "source": [ 275 | "### **Binary Class**" 276 | ] 277 | }, 278 | { 279 | "cell_type": "code", 280 | "metadata": { 281 | "id": "xO_0J1yQOAVY" 282 | }, 283 | "source": [ 284 | "features = ['Timestamp', 'Fwd Pkt Len Std', 'Fwd Pkt Len Mean',\n", 285 | " 'Fwd Pkt Len Max', 'Fwd Seg Size Avg', 'Pkt Len Std', 'Flow IAT Std',\n", 286 | " 'Bwd Pkt Len Std', 'Bwd Seg Size Avg', 'Pkt Size Avg',\n", 287 | " 'Subflow Fwd Byts']" 288 | ], 289 | "execution_count": null, 290 | "outputs": [] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "metadata": { 295 | "id": "17a7f5rWNNXZ" 296 | }, 297 | "source": [ 298 | "def targetify(s):\n", 299 | " if s == 'Benign':\n", 300 | " return 0\n", 301 | " else:\n", 302 | " return 1" 303 | ], 304 | "execution_count": null, 305 | "outputs": [] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "metadata": { 310 | "id": "Jyv3ec_KwtFz", 311 | "colab": { 312 | "base_uri": "https://localhost:8080/" 313 | }, 314 | "outputId": "f7df6c86-db4e-4006-9dcf-be82835a8364" 315 | }, 316 | "source": [ 317 | "X = df[features]\n", 318 | "X[features] = X[features].apply(pd.to_numeric, errors='coerce', axis=1)\n", 319 | "X = X.fillna(0)\n", 320 | "labels = df['Label'] #For multiclass classification\n" 321 | ], 322 | "execution_count": null, 323 | "outputs": [ 324 | { 325 | "output_type": "stream", 326 | "name": "stderr", 327 | "text": [ 328 | "/usr/local/lib/python3.8/dist-packages/pandas/core/frame.py:3641: SettingWithCopyWarning: \n", 329 | "A value is trying to be set on a copy of a slice from a DataFrame.\n", 330 | "Try using .loc[row_indexer,col_indexer] = value instead\n", 331 | "\n", 332 | "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", 333 | " self[k1] = value[k2]\n" 334 | ] 335 | } 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "metadata": { 341 | "id": "gCbbLe-fsFll" 342 | }, 343 | "source": [ 344 | "df['Target']=df['Label'].apply(targetify)\n", 345 | "y = df['Target']" 346 | ], 347 | "execution_count": null, 348 | "outputs": [] 349 | }, 350 | { 351 | "cell_type": "markdown", 352 | "metadata": { 353 | "id": "ExRZ3yLNyswE" 354 | }, 355 | "source": [ 356 | "Normal Execution" 357 | ] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "metadata": { 362 | "id": "5ZupR410ws7J", 363 | "colab": { 364 | "base_uri": "https://localhost:8080/" 365 | }, 366 | "outputId": "2178ab47-c2bb-47be-b9b4-85330337a948" 367 | }, 368 | "source": [ 369 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", 370 | "print (X_train.shape, y_train.shape)\n", 371 | "print( X_test.shape, y_test.shape)" 372 | ], 373 | "execution_count": null, 374 | "outputs": [ 375 | { 376 | "output_type": "stream", 377 | "name": "stdout", 378 | "text": [ 379 | "(838859, 11) (838859,)\n", 380 | "(209715, 11) (209715,)\n" 381 | ] 382 | } 383 | ] 384 | }, 385 | { 386 | "cell_type": "markdown", 387 | "metadata": { 388 | "id": "vpr26iooyw4v" 389 | }, 390 | "source": [ 391 | "Faster Execution (1% rows)" 392 | ] 393 | }, 394 | { 395 | "cell_type": "code", 396 | "metadata": { 397 | "id": "MP40NJBNyKL6", 398 | "colab": { 399 | "base_uri": "https://localhost:8080/" 400 | }, 401 | "outputId": "043d8d13-017f-43f7-8b34-5b43e109ec79" 402 | }, 403 | "source": [ 404 | "# selecting 1% of random rows for better running time\n", 405 | "\n", 406 | "X_train = X_train.sample(frac=0.1, replace=True, random_state=1)\n", 407 | "y_train = y_train.sample(frac=0.1, replace=True, random_state=1)\n", 408 | "X_test = X_test.sample(frac=0.1, replace=True, random_state=1)\n", 409 | "y_test = y_test.sample(frac=0.1, replace=True, random_state=1)\n", 410 | "print (X_train.shape, y_train.shape)\n", 411 | "print( X_test.shape, y_test.shape)" 412 | ], 413 | "execution_count": null, 414 | "outputs": [ 415 | { 416 | "output_type": "stream", 417 | "name": "stdout", 418 | "text": [ 419 | "(83886, 11) (83886,)\n", 420 | "(20972, 11) (20972,)\n" 421 | ] 422 | } 423 | ] 424 | }, 425 | { 426 | "cell_type": "code", 427 | "metadata": { 428 | "id": "h4TWHdVGyQTz" 429 | }, 430 | "source": [ 431 | "min_max_scaler = preprocessing.MinMaxScaler()\n", 432 | "x_scaled = min_max_scaler.fit_transform(X_train.values)\n", 433 | "X_train = pd.DataFrame(x_scaled,columns=features)\n", 434 | "x_scaled_test = min_max_scaler.fit_transform(X_test.values)\n", 435 | "X_test = pd.DataFrame(x_scaled_test,columns=features)" 436 | ], 437 | "execution_count": null, 438 | "outputs": [] 439 | }, 440 | { 441 | "cell_type": "code", 442 | "metadata": { 443 | "id": "Rf2vrS4pwsuh" 444 | }, 445 | "source": [ 446 | "model = tf.keras.Sequential([\n", 447 | " tf.keras.layers.Embedding(100000, 64), # since it doesn't consider \"words,\" the embedding doesn't really matter\n", 448 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),\n", 449 | " tf.keras.layers.Dense(64, activation='relu'),\n", 450 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 451 | " # tf.keras.layers.Dense(1, activation='softmax') # loss too big\n", 452 | "])" 453 | ], 454 | "execution_count": null, 455 | "outputs": [] 456 | }, 457 | { 458 | "cell_type": "code", 459 | "metadata": { 460 | "id": "iS-VFQNxwslT", 461 | "colab": { 462 | "base_uri": "https://localhost:8080/" 463 | }, 464 | "outputId": "4b7cdcd2-1bcc-4a15-f653-db6be8bcdc04" 465 | }, 466 | "source": [ 467 | "model.compile(loss='binary_crossentropy',\n", 468 | " # optimizer='sgd', # almost same\n", 469 | " optimizer=tf.keras.optimizers.Adam(1e-4),\n", 470 | " metrics=['accuracy'])\n", 471 | "history = model.fit(X_train.values, y_train.values, epochs=3)\n", 472 | "model.save('drive/MyDrive/cicids/LSTM_BC')" 473 | ], 474 | "execution_count": null, 475 | "outputs": [ 476 | { 477 | "output_type": "stream", 478 | "name": "stdout", 479 | "text": [ 480 | "Epoch 1/3\n", 481 | "2622/2622 [==============================] - 244s 91ms/step - loss: 0.1005 - accuracy: 0.9488\n", 482 | "Epoch 2/3\n", 483 | "2622/2622 [==============================] - 246s 94ms/step - loss: 0.0249 - accuracy: 0.9959\n", 484 | "Epoch 3/3\n", 485 | "2622/2622 [==============================] - 252s 96ms/step - loss: 0.0248 - accuracy: 0.9959\n" 486 | ] 487 | }, 488 | { 489 | "output_type": "stream", 490 | "name": "stderr", 491 | "text": [ 492 | "WARNING:absl:Found untraced functions such as lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 4 of 4). These functions will not be directly callable after loading.\n" 493 | ] 494 | } 495 | ] 496 | }, 497 | { 498 | "cell_type": "code", 499 | "metadata": { 500 | "id": "ax68X_PzxEWC", 501 | "colab": { 502 | "base_uri": "https://localhost:8080/" 503 | }, 504 | "outputId": "91c8802c-0efa-424f-8e0e-da4b51443f92" 505 | }, 506 | "source": [ 507 | "pred_class = model.predict(X_test.values[:])\n", 508 | "predictions = [int(round(x[0])) for x in pred_class]\n", 509 | "true_class = list(y_test)\n", 510 | "np.sum(predictions == y_test.values) / len(y_test.values)" 511 | ], 512 | "execution_count": null, 513 | "outputs": [ 514 | { 515 | "output_type": "stream", 516 | "name": "stdout", 517 | "text": [ 518 | "656/656 [==============================] - 5s 6ms/step\n" 519 | ] 520 | }, 521 | { 522 | "output_type": "execute_result", 523 | "data": { 524 | "text/plain": [ 525 | "0.9957562464238031" 526 | ] 527 | }, 528 | "metadata": {}, 529 | "execution_count": 15 530 | } 531 | ] 532 | }, 533 | { 534 | "cell_type": "code", 535 | "metadata": { 536 | "id": "KufqWIFZxEQ_" 537 | }, 538 | "source": [ 539 | "def myRound(x, r):\n", 540 | " if x>r/float(1000):\n", 541 | " return 1\n", 542 | " else:\n", 543 | " return 0" 544 | ], 545 | "execution_count": null, 546 | "outputs": [] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "metadata": { 551 | "id": "y-rksHvxxEML" 552 | }, 553 | "source": [ 554 | "compdf = pd.DataFrame({'pred_class':predictions, 'true_class':true_class})\n", 555 | "compdf = compdf.sort_values('pred_class', ascending=True)\n", 556 | "predictions = list(compdf['pred_class'].apply(myRound, r=225))\n", 557 | "true_class = list(compdf['true_class'])" 558 | ], 559 | "execution_count": null, 560 | "outputs": [] 561 | }, 562 | { 563 | "cell_type": "code", 564 | "metadata": { 565 | "id": "SRrWsEQUxD20", 566 | "colab": { 567 | "base_uri": "https://localhost:8080/" 568 | }, 569 | "outputId": "c2e064e1-f6be-403c-99c9-4d727c427003" 570 | }, 571 | "source": [ 572 | "confm = confusion_matrix(true_class, predictions)\n", 573 | "confm" 574 | ], 575 | "execution_count": null, 576 | "outputs": [ 577 | { 578 | "output_type": "execute_result", 579 | "data": { 580 | "text/plain": [ 581 | "array([[ 8660, 89],\n", 582 | " [ 0, 12223]])" 583 | ] 584 | }, 585 | "metadata": {}, 586 | "execution_count": 18 587 | } 588 | ] 589 | }, 590 | { 591 | "cell_type": "markdown", 592 | "metadata": { 593 | "id": "DER9DuHSa5hH" 594 | }, 595 | "source": [ 596 | "### **Multi Class**" 597 | ] 598 | }, 599 | { 600 | "cell_type": "code", 601 | "metadata": { 602 | "id": "T0bKOdGia9Oa" 603 | }, 604 | "source": [ 605 | "categories = ['Benign', 'FTP-BruteForce', 'SSH-Bruteforce',\n", 606 | " 'DoS attacks-GoldenEye', 'DoS attacks-Slowloris', 'DoS attacks-SlowHTTPTest',\n", 607 | " 'DoS attacks-Hulk', 'Brute Force -Web', 'Brute Force -XSS',\n", 608 | " 'SQL Injection', 'Infiltration', 'Bot']" 609 | ], 610 | "execution_count": null, 611 | "outputs": [] 612 | }, 613 | { 614 | "cell_type": "code", 615 | "metadata": { 616 | "id": "9G0WRcoMa-uw" 617 | }, 618 | "source": [ 619 | "encoder = LabelEncoder()\n", 620 | "encoder.fit(categories)\n", 621 | "y = encoder.transform(labels)\n", 622 | "y = np_utils.to_categorical(y, num_classes=12)" 623 | ], 624 | "execution_count": null, 625 | "outputs": [] 626 | }, 627 | { 628 | "cell_type": "code", 629 | "metadata": { 630 | "id": "r3kN1kPwbHg7" 631 | }, 632 | "source": [ 633 | "model = tf.keras.Sequential([\n", 634 | " tf.keras.layers.Embedding(100000, 64, embeddings_regularizer='l2'), # since it doesn't consider \"words,\" the embedding doesn't really matter\n", 635 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),\n", 636 | " tf.keras.layers.Dropout(0.4),\n", 637 | " tf.keras.layers.Dense(64, activation='relu', use_bias=True, bias_regularizer='l2'),\n", 638 | " tf.keras.layers.Dropout(0.2),\n", 639 | " tf.keras.layers.Dense(12, activation='sigmoid')\n", 640 | " # tf.keras.layers.Dense(1, activation='softmax') # loss too big\n", 641 | "])" 642 | ], 643 | "execution_count": null, 644 | "outputs": [] 645 | }, 646 | { 647 | "cell_type": "code", 648 | "metadata": { 649 | "id": "dAWdPRbXolXO", 650 | "colab": { 651 | "base_uri": "https://localhost:8080/" 652 | }, 653 | "outputId": "6c7085bc-68fe-49c4-dd83-ed91c010b16b" 654 | }, 655 | "source": [ 656 | "X_train.shape, y_train.shape" 657 | ], 658 | "execution_count": null, 659 | "outputs": [ 660 | { 661 | "output_type": "execute_result", 662 | "data": { 663 | "text/plain": [ 664 | "((83886, 11), (83886,))" 665 | ] 666 | }, 667 | "metadata": {}, 668 | "execution_count": 22 669 | } 670 | ] 671 | }, 672 | { 673 | "cell_type": "code", 674 | "metadata": { 675 | "id": "qw5yKr6-bQe6", 676 | "colab": { 677 | "base_uri": "https://localhost:8080/" 678 | }, 679 | "outputId": "0949958f-3f39-4778-c507-d457224eb3e7" 680 | }, 681 | "source": [ 682 | "model.compile(loss='sparse_categorical_crossentropy',\n", 683 | " # optimizer='sgd', # almost same\n", 684 | " optimizer=tf.keras.optimizers.Adam(1e-4),\n", 685 | " metrics=['accuracy'])\n", 686 | "history = model.fit(X_train.values, y_train.values, epochs=3)\n", 687 | "model.save('drive/MyDrive/cicids/LSTM_MC_L2')" 688 | ], 689 | "execution_count": null, 690 | "outputs": [ 691 | { 692 | "output_type": "stream", 693 | "name": "stdout", 694 | "text": [ 695 | "Epoch 1/3\n", 696 | "2622/2622 [==============================] - 220s 82ms/step - loss: 3.3247 - accuracy: 0.9006\n", 697 | "Epoch 2/3\n", 698 | "2622/2622 [==============================] - 203s 77ms/step - loss: 0.0320 - accuracy: 0.9955\n", 699 | "Epoch 3/3\n", 700 | "2622/2622 [==============================] - 203s 77ms/step - loss: 0.0291 - accuracy: 0.9959\n" 701 | ] 702 | }, 703 | { 704 | "output_type": "stream", 705 | "name": "stderr", 706 | "text": [ 707 | "WARNING:absl:Found untraced functions such as lstm_cell_4_layer_call_fn, lstm_cell_4_layer_call_and_return_conditional_losses, lstm_cell_5_layer_call_fn, lstm_cell_5_layer_call_and_return_conditional_losses while saving (showing 4 of 4). These functions will not be directly callable after loading.\n" 708 | ] 709 | } 710 | ] 711 | }, 712 | { 713 | "cell_type": "code", 714 | "metadata": { 715 | "id": "Y2HCh4UhbheN", 716 | "colab": { 717 | "base_uri": "https://localhost:8080/" 718 | }, 719 | "outputId": "57eb16c6-834f-4e6e-bcfd-b4ad11c442f3" 720 | }, 721 | "source": [ 722 | "pred_class = model.predict(X_test.values[:])\n", 723 | "predictions = [int(round(x[0])) for x in pred_class]\n", 724 | "true_class = list(y_test)\n", 725 | "np.sum(predictions == y_test.values) / len(y_test.values)" 726 | ], 727 | "execution_count": null, 728 | "outputs": [ 729 | { 730 | "output_type": "stream", 731 | "name": "stdout", 732 | "text": [ 733 | "656/656 [==============================] - 5s 6ms/step\n" 734 | ] 735 | }, 736 | { 737 | "output_type": "execute_result", 738 | "data": { 739 | "text/plain": [ 740 | "0.5828247186725157" 741 | ] 742 | }, 743 | "metadata": {}, 744 | "execution_count": 24 745 | } 746 | ] 747 | }, 748 | { 749 | "cell_type": "code", 750 | "metadata": { 751 | "id": "4S6potN6buR9" 752 | }, 753 | "source": [ 754 | "compdf = pd.DataFrame({'pred_class':predictions, 'true_class':true_class})\n", 755 | "compdf = compdf.sort_values('pred_class', ascending=True)\n", 756 | "predictions = list(compdf['pred_class'].apply(myRound, r=225))\n", 757 | "true_class = list(compdf['true_class'])" 758 | ], 759 | "execution_count": null, 760 | "outputs": [] 761 | }, 762 | { 763 | "cell_type": "code", 764 | "metadata": { 765 | "id": "WTq1Vkoybyl2", 766 | "colab": { 767 | "base_uri": "https://localhost:8080/" 768 | }, 769 | "outputId": "f70d4cbf-6c55-4ac6-ba8e-464e1bca19d4" 770 | }, 771 | "source": [ 772 | "confm = confusion_matrix(true_class, predictions)\n", 773 | "confm" 774 | ], 775 | "execution_count": null, 776 | "outputs": [ 777 | { 778 | "output_type": "execute_result", 779 | "data": { 780 | "text/plain": [ 781 | "array([[ 0, 8749],\n", 782 | " [ 0, 12223]])" 783 | ] 784 | }, 785 | "metadata": {}, 786 | "execution_count": 26 787 | } 788 | ] 789 | }, 790 | { 791 | "cell_type": "code", 792 | "metadata": { 793 | "id": "7ilu8oBWEoBp", 794 | "colab": { 795 | "base_uri": "https://localhost:8080/" 796 | }, 797 | "outputId": "8ece0740-5977-47a3-827f-705c79657cbf" 798 | }, 799 | "source": [ 800 | "tf.keras.utils.plot_model(model, to_file='model.png', show_shapes=True, show_layer_names=False)" 801 | ], 802 | "execution_count": null, 803 | "outputs": [ 804 | { 805 | "output_type": "execute_result", 806 | "data": { 807 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa8AAALhCAYAAADo0SuAAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzde1RTZ9Y/8G9CEkIw3ESBoogEvKBWx8tUGR3r2GFarSiiglU7tW9n0NYi3gWVKoLV2kEGlfp2dFgz6qigvmittC47dazjZbVVi8UZihTvRUC5XyTA/v3hL6kxARIIJCfsz1r8wTnPOWc/OSGbnPOcZ4uIiMAYY4wJR4bY0hEwxhhjpuLkxRhjTHA4eTHGGBMcTl6MMcYER2LpAMztwoULSEpKsnQYjDFmNTIyMiwdgtnZ3DevO3fu4PDhw5YOgxnh4sWLuHjxoqXDEJS7d+/y+5sZzZbfLzb3zUvDFv/TsDUzZ84EwOfKFOnp6QgPD+fXjBlF836xRTb3zYsxxpjt4+TFGGNMcDh5McYYExxOXowxxgSHkxdjjDHB4eTFBO/kyZNwdnbGJ598YulQrNKCBQsgEom0P3PnztVrc/r0acTExODIkSPw8/PTtp03b55e2+DgYCiVStjZ2WHQoEG4fPlyZ3SjTYTSn+PHj2PLli1obGzUWZ6Zmalz7tzd3Ts9NmvFyYsJHhdGaJ2bmxuysrKQm5uLPXv26Kx77733kJKSgtjYWISFheHHH3+ESqVC9+7dsW/fPnz66ac67U+dOoWMjAxMmTIFOTk5GD58eGd2xSRC6U9ISAjkcjkmTpyIsrIy7fKpU6fi7t27OHv2LCZNmtTpcVkzTl5M8CZPnozy8nJMmTLF0qGgtrYWQUFBlg5Dj4ODA15++WX069cP9vb22uWbN2/GwYMHkZ6eDqVSqbNNSkoKxGIxIiMjUV5e3tkhm52192fx4sUYOnQoJk2ahIaGBgCASCSCt7c3xo0bh4CAAAtHaF04eTFmRnv27EFRUZGlwzDKjRs3sG7dOmzYsAFyuVxvfVBQEKKjo3Hv3j0sX77cAhGalxD6s379ely9ehXJycmWDsXqcfJignbu3Dn4+PhAJBJhx44dAIDU1FQ4OjpCoVDg2LFjeOWVV+Dk5IRevXrhwIED2m1TUlIgl8vRs2dPLFiwAF5eXpDL5QgKCsKlS5e07aKioiCTyeDp6ald9s4778DR0REikQglJSUAgOjoaCxbtgz5+fkQiUTw9/cHAHz22WdwcnJCYmJiZ7wkRktJSQERISQkpNk2CQkJ6NevH3bv3o3Tp0+3uD8iQlJSEgYOHAh7e3u4urpi2rRp+O9//6ttY+y5AYDGxkbExcXBx8cHDg4OeP7553Ho0KF29dna++Pq6orx48cjOTmZL4e3hmzMoUOHyAa7ZZNmzJhBM2bMaPd+7ty5QwBo+/bt2mVr1qwhAPTFF19QeXk5FRUV0bhx48jR0ZHq6+u17SIjI8nR0ZGuX79OdXV1lJOTQ6NGjSKlUkm3b9/WtpszZw55eHjoHHfr1q0EgIqLi7XLwsLCSKVS6bQ7ceIEKZVKio+Pb3df2/L+joyMJG9vb73lfn5+FBgYaHAblUpFBQUFRER0/vx5EovF5OvrS1VVVURElJWVRVOnTtXZJi4ujmQyGe3du5fKysooOzubhg8fTu7u7lRYWKhtZ+y5Wb58Odnb29Phw4eptLSUYmNjSSwW09dff21S/4XWn5iYGAJAV65c0Vm+ePFi6t69u0n9tuHPw3T+5sVsWlBQEJycnNCjRw9ERESguroat2/f1mkjkUi0/10HBgYiNTUVlZWVSEtLM0sMkydPRkVFBdatW2eW/ZlDdXU1CgoKoFKpWm07ZswYLFmyBDdv3sTq1asNtqmtrUVSUhKmT5+OuXPnwtnZGUOGDMGuXbtQUlKCjz/+WG+bls5NXV0dUlNTERoairCwMLi4uGDt2rWQSqXtPi/W3h/Nva1r1661q5+2jpMX6zJkMhkAQK1Wt9hu5MiRUCgUOpeHbE1RURGICAqFwqj2CQkJ6N+/P3bu3Ilz587prc/JyUFVVRVGjhyps3zUqFGQyWQ6l2ENefbc5ObmoqamBoMHD9a2cXBwgKenp1nOizX3R3NOHjx4YHrHuhBOXowZYG9vj+LiYkuH0WHq6uoAQGfkYUvkcjnS0tIgEonw5ptvora2Vme9Znh3t27d9LZ1cXFBZWWlSfFVV1cDANauXavznNOtW7dQU1Nj0r4Mseb+ODg4APj5HDHDOHkx9gy1Wo2ysjL06tXL0qF0GM0H5LMPxbZkzJgxWLp0KfLy8rBx40addS4uLgBg8EO9La9ljx49AADbtm0DEen8XLhwwaR9Ncda+1NfXw/g53PEDOPkxdgzzpw5AyLC6NGjtcskEkmrlxuFpGfPnhCJRCY/77Rx40YMGDAAV65c0Vk+ePBgdOvWDd98843O8kuXLqG+vh4jRoww6Ti9e/eGXC7H1atXTdrOVNbYH8058fDwMOkYXQ0nL9blNTU1obS0FA0NDcjOzkZ0dDR8fHzwxhtvaNv4+/vj0aNHyMzMhFqtRnFxMW7duqW3Lzc3N9y/fx83b95EZWUl1Go1srKyrG6ovEKhgJ+fH+7evWvSdprLbXZ2dnrLly1bhqNHj2Lfvn2oqKjAtWvXsHDhQnh5eSEyMtLk48yfPx8HDhxAamoqKioq0NjYiLt37+Knn34CAERERMDDw6Nd0zlZU380NOdkyJAhbe5Xl2CpcY4dxYaHhtoccwyV3759O3l6ehIAUigUFBISQjt37iSFQkEAKCAggPLz8+njjz8mJycnAkB9+vShH374gYieDCOXSqXk7e1NEomEnJycaNq0aZSfn69znIcPH9KECRNILpdT37596d1336UVK1YQAPL399cOq798+TL16dOHHBwcaOzYsVRYWEgnT54kpVJJCQkJ7eorkXmHykdFRZFUKqWamhrtsqNHj5JKpSIA5O7uTosWLTK4zxUrVugNLW9qaqKtW7dSQEAASaVScnV1pdDQUMrNzdW2MeXcPH78mFatWkU+Pj4kkUioR48eFBYWRjk5OUREFBoaSgAoLi6u2b4LqT8akydPJm9vb2pqatJZzkPldaTbXK9s+GTZHHM959UekZGR5ObmZtEYTGHO5JWXl0cSiYT27t1rrvA6VWNjI40bN4727Nlj6VDMpqSkhORyOX344Yd66zh56eDnvBgzZdCCUNXW1uLzzz9HXl6edkCAv78/4uPjER8fj6qqKgtHaJrGxkZkZmaisrISERERlg7HbNavX49hw4YhKioKwJNZPu7fv49z587hxo0bFo7OunDyYqwLePTokXZi3jfffFO7PCYmBjNnzkRERIRVTlbbnDNnzuDIkSPIysoy+lk1a5eUlISrV6/i5MmTkEqlAIBjx45pJ+Z9djb8rq7LJ6+LFy9i4MCBEIvFEIlE8PDwQEJCgqXD0vFsTSJPT0+DNZmYaWJjY5GWloby8nL07dsXhw8ftnRIHWLXrl06Q7P37dunsz4xMRFRUVF4//33LRSh6SZOnIj9+/frzDcpZMeOHcPjx49x5swZuLq6apdPmzZN59xp5tFkgIjItmZ/TE9PR3h4uMmTWr788sv4/PPPUVpaqn3Gw9r4+/ujpKREp96PkM2cORMAkJGRYeFIhKOt72/WNdnw+yWjy3/zskbWWhOKMcasBScvKySkmlCMMWYJnLyaYW01oUz11VdfITAwEM7OzpDL5RgyZAg+//xzAMBbb72lvX+mUqm0swvMnz8fCoUCzs7OOH78OICWaxB98MEHUCgUUCqVKCoqwrJly+Dt7Y3c3Nw2xcwYY0azzBD9jtPW5xp+97vfEQAqLS3VLrOmmlBET2oSOTs7G9WfjIwMWr9+PT169IgePnxIo0eP1nlGJCwsjOzs7OjevXs627322mt0/Phx7e+t1SDSvEaLFy+m7du30/Tp0+k///mPUTFaw3NeQmPDz+2wDmDD7xd+zssY1lATylQzZszAe++9B1dXV7i5uSEkJAQPHz7UzpS+cOFCNDY26sRXUVGBr7/+GpMmTQJgWg2izZs3Y9GiRThy5AgGDBjQeR1ljHVJEksHIDRCrQmleW5E80Dub37zG/Tr1w9//etfERsbC5FIhIMHDyIiIkI7z1tH11QCgMOHD0MkEpllX10Jv2asq+Pk1YEsWRPq008/xdatW5GTk4OKigq9ZCsSibBgwQIsXboUX3zxBV566SX8/e9/x/79+7Vtnq5BtHbtWp3tvby8zBLn6NGjsWTJErPsqyu4cOECkpOTtfcdGWuJ5v1iizh5dZDOrgl19uxZfPvtt1iyZAlu376N0NBQTJ8+HX/961/x3HPPYfv27Vi5cqXONm+88QZiY2Oxe/du9O7dG05OTujTp492/dM1iKKjozsk7l69emHWrFkdsm9blZyczK8ZMxonL2aSzq4J9e2338LR0REAcO3aNajVarz99tvw8/MDYPgyk6urK8LDw3Hw4EEolUr84Q9/0FnfWTWVGGPMVDxgw0w6uiZUc9RqNR48eIAzZ85ok5ePjw8A4PTp06irq0NeXp7OsP2nLVy4EI8fP8aJEycwZcoUnXWm1CBijLFOZenxjuZm6tDQixcv0qBBg0gsFhMA8vT0pMTERKuqCfXRRx9paxK19HP06FHtsVatWkVubm7k4uJCM2fOpB07dhAAUqlUOsP3iYh+8YtfUExMjMHXp6UaRFu2bCEHBwcCQL179za5tAYPlTedDQ99Zh3Aht8v6Ty3oRksWLAAGRkZePjwYacd05wmT56MHTt2oG/fvp16XJ7b0HQ2PFcd6wA2/H7huQ3NRUg1oZ6+DJmdnQ25XN7piYsxxtqDk1cXtGrVKuTl5eGHH37A/PnzsXHjRkuHxDrQggULtNOBiUQig+V0Tp8+jZiYGL3yO/PmzdNrGxwcDKVSCTs7OwwaNAiXL1/ujG60iVD6c/z4cWzZskXvn+DMzEydc+fu7t7psVkti1617ACdfY03JiaGZDIZASBfX1/KyMjotGO31Zo1a0gsFlPv3r11poLqbHzPy3RteX9HRkaSm5sbZWVlUW5uLtXV1emsj4uLoylTplBFRYV2mUqlou7duxMAOnHihN4+s7KyaOrUqW3rhAUIoT/Jyck0fvx4nSnqmpqa6O7du3T27FmaNGmSzhRvxrDle178zaudNm3ahMePH4OIUFBQgBkzZlg6pFYlJCSgsbERt2/f1hth2NV0RvkZayhx4+DgoK2kbG9vr12+efNmHDx4EOnp6VAqlTrbpKSkQCwWIzIyUlBVlptj7f1ZvHgxhg4dikmTJqGhoQHAk0dcNJWUAwICLByhdeHkxbq0zig/Y60lbm7cuIF169Zhw4YNkMvleuuDgoIQHR2Ne/fuYfny5RaI0LyE0J/169fj6tWrNvtgsTlx8mKCQkRISkrSToLs6uqKadOm6cy12J7yM51V4uazzz6Dk5MTEhMTO/T1aklKSgqICCEhIc22SUhIQL9+/bB7926cPn26xf0Zc26MLTUEtFyOp62svT+urq4YP348kpOTbXGEoHlZ8qJlR7Dha7w2py33vOLi4kgmk9HevXuprKyMsrOzafjw4eTu7k6FhYXadu0pP9MZJW5OnDhBSqWS4uPjTep/W+95eXt76y338/OjwMBAg9uoVCoqKCggIqLz58+TWCwmX19fqqqqIiLD94iMPTfGlhpqrRyPKYTUn5iYGAJAV65c0Vm+ePFivuf1M77nxYSjtrYWSUlJmD59OubOnQtnZ2cMGTIEu3btQklJCT7++GOzHaujS9xMnjwZFRUVWLdunVn2Z6rq6moUFBRApVK12nbMmDFYsmQJbt68idWrVxts05Zz01KpIVPK8ZjK2vujubd17dq1dvXT1nHyYoKRk5ODqqoqjBw5Umf5qFGjIJPJmp0CyxysrcRNexUVFYGIoFAojGqfkJCA/v37Y+fOnTh37pze+vaem2dLDXV0OR5r7o/mnDx48MD0jnUhnLyYYJSVlQEAunXrprfOxcUFlZWVHXp8S5a4Mbe6ujoA0Bl52BK5XI60tDSIRCK8+eabqK2t1Vlv7nPzdDmep59zunXrFmpqakzalyHW3B8HBwcAP58jZhgnLyYYLi4uAGDwg6Ojy890dombjqb5gDRlZpgxY8Zg6dKlyMvL03uw3dzn5ulyPESk83PhwgWT9tUca+1PfX09gJ/PETOMkxcTjMGDB6Nbt2745ptvdJZfunQJ9fX1GDFihHaZucvPdHaJm47Ws2dPiEQik5932rhxIwYMGIArV67oLDfl3Bijs8rxWGN/NOfEw8PDpGN0NZy8mGDI5XIsW7YMR48exb59+1BRUYFr165h4cKF8PLyQmRkpLZte8vPdHSJm6ysLIsOlVcoFPDz88Pdu3dN2k5zuc3Ozk5vubHnxtjjtFaOJyIiAh4eHu2azsma+qOhOSdDhgxpc7+6BEuNc+woNjw01Oa0Zah8U1MTbd26lQICAkgqlZKrqyuFhoZSbm6uTru2lp8pLCzs8BI3hYWFdPLkSVIqlZSQkGBS/805VD4qKoqkUinV1NRolx09elRbfsfd3Z0WLVpkcJ8rVqzQG1puzLkxpdRQS+V4iIhCQ0MJAMXFxTXbdyH1R2Py5Mnk7e1NTU1NOst5qLyOdJvrlQ2fLJtjrXMbauYCtEbmTF55eXkkkUhMrsNmLRobG2ncuHG0Z88eS4diNiUlJSSXy+nDDz/UW8fJSwc/58WYIUIqcWOM2tpafP7558jLy9MOCPD390d8fDzi4+NRVVVl4QhN09jYiMzMTFRWViIiIsLS4ZjN+vXrMWzYMERFRQF4MsvH/fv3ce7cOdy4ccPC0VkXTl6MdQGPHj3STsz75ptvapfHxMRg5syZiIiIsMrJaptz5swZHDlyBFlZWUY/q2btkpKScPXqVZw8eRJSqRQAcOzYMe3EvJ9++qmFI7QunLwYe0psbCzS0tJQXl6Ovn374vDhw5YOqd127dqlMzR73759OusTExMRFRWF999/30IRmm7ixInYv3+/ztySQnbs2DE8fvwYZ86cgaurq3b5tGnTdM6dZs5MBkgsHQBj1mTTpk3YtGmTpcPodMHBwQgODrZ0GF3W1KlTMXXqVEuHISj8zYsxxpjgcPJijDEmOJy8GGOMCQ4nL8YYY4JjswM20tPTLR0Ca4VmGhw+V8bTTOLKrxkzhrkmMbZGIiLbqjWdnp6O8PBwS4fBGGNWw8Y+5gEgw+aSF2PWQPNPFP95MdYhMvieF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcHh5MUYY0xwOHkxxhgTHE5ejDHGBIeTF2OMMcGRWDoAxoTu7t27+P3vf4/GxkbtstLSUiiVSrz44os6bfv374///d//7eQIGbM9nLwYa6devXrh1q1byM/P11v3r3/9S+f3X//6150VFmM2jS8bMmYGr7/+OqRSaavtIiIiOiEaxmwfJy/GzGDOnDloaGhosc2gQYMQGBjYSRExZts4eTFmBiqVCs8//zxEIpHB9VKpFL///e87OSrGbBcnL8bM5PXXX4ednZ3BdQ0NDZg5c2YnR8SY7eLkxZiZzJ49G01NTXrLxWIxRo8eDV9f384PijEbxcmLMTPx8vLCr371K4jFun9WYrEYr7/+uoWiYsw2cfJizIzmzZunt4yIMH36dAtEw5jt4uTFmBnNmDFD576XnZ0dXnrpJfTs2dOCUTFmezh5MWZGrq6u+O1vf6tNYESEuXPnWjgqxmwPJy/GzGzu3LnagRtSqRTTpk2zcESM2R5OXoyZWUhICOzt7QEAU6ZMQbdu3SwcEWO2h5MXY2bm6Oio/bbFlwwZ6xgiIiJLB9GS9PR0hIeHWzoMxhjrMqw8LQBAhmBmlT906JClQ2BtcOHCBSQnJ3e589fY2IhDhw7htddea9P24eHhiI6OxpgxY8wcGWPN0/y9CoFgktesWbMsHQJro+Tk5C55/kJDQyGXy9u0bXh4OMaMGdMlXzdmWUJJXnzPi7EO0tbExRhrHScvxhhjgsPJizHGmOBw8mKMMSY4nLwYY4wJDicvJggnT56Es7MzPvnkE0uHYvNOnz6NmJgYHDlyBH5+fhCJRBCJRAZnzA8ODoZSqYSdnR0GDRqEy5cvWyBi4wilP8ePH8eWLVvQ2NjY6ccWEk5eTBAE8NCkTXjvvfeQkpKC2NhYhIWF4ccff4RKpUL37t2xb98+fPrppzrtT506hYyMDEyZMgU5OTkYPny4hSJvnVD6ExISArlcjokTJ6KsrKzTjy8UnLyYIEyePBnl5eWYMmWKpUNBbW0tgoKCLB2G2W3evBkHDx5Eeno6lEqlzrqUlBSIxWJERkaivLzcQhGaj7X3Z/HixRg6dCgmTZqEhoYGS4djlTh5MWaiPXv2oKioyNJhmNWNGzewbt06bNiwweDzaUFBQYiOjsa9e/ewfPlyC0RoXkLoz/r163H16lXBPDTc2Th5Mat37tw5+Pj4QCQSYceOHQCA1NRUODo6QqFQ4NixY3jllVfg5OSEXr164cCBA9ptU1JSIJfL0bNnTyxYsABeXl6Qy+UICgrCpUuXtO2ioqIgk8ng6empXfbOO+/A0dERIpEIJSUlAIDo6GgsW7YM+fn5EIlE8Pf3BwB89tlncHJyQmJiYme8JGaXkpICIkJISEizbRISEtCvXz/s3r0bp0+fbnF/RISkpCQMHDgQ9vb2cHV1xbRp0/Df//5X28bYcwg8mW4rLi4OPj4+cHBwwPPPP9/uKcesvT+urq4YP348kpOT+bK5IWTlDh06RAIIkzXDXOfvzp07BIC2b9+uXbZmzRoCQF988QWVl5dTUVERjRs3jhwdHam+vl7bLjIykhwdHen69etUV1dHOTk5NGrUKFIqlXT79m1tuzlz5pCHh4fOcbdu3UoAqLi4WLssLCyMVCqVTrsTJ06QUqmk+Pj4dveViAgAHTp0yCz7Moafnx8FBgYaXKdSqaigoICIiM6fP09isZh8fX2pqqqKiIiysrJo6tSpOtvExcWRTCajvXv3UllZGWVnZ9Pw4cPJ3d2dCgsLte2MPYfLly8ne3t7Onz4MJWWllJsbCyJxWL6+uuvTe6rkPoTExNDAOjKlSsm97MtBPR5m87fvJjgBQUFwcnJCT169EBERASqq6tx+/ZtnTYSiUT7X3NgYCBSU1NRWVmJtLQ0s8QwefJkVFRUYN26dWbZX2eqrq5GQUEBVCpVq23HjBmDJUuW4ObNm1i9erXBNrW1tUhKSsL06dMxd+5cODs7Y8iQIdi1axdKSkrw8ccf623T0jmsq6tDamoqQkNDERYWBhcXF6xduxZSqbTd58/a+xMQEAAAuHbtWrv6aYs4eTGbIpPJAABqtbrFdiNHjoRCodC57NNVFRUVgYigUCiMap+QkID+/ftj586dOHfunN76nJwcVFVVYeTIkTrLR40aBZlMpnO51pBnz2Fubi5qamowePBgbRsHBwd4enqa5fxZc3805+TBgwemd8zGcfJiXZa9vT2Ki4stHYbF1dXVAYC2+nNr5HI50tLSIBKJ8Oabb6K2tlZnvWZ4t6EK0i4uLqisrDQpvurqagDA2rVrtc9oiUQi3Lp1CzU1NSbtyxBr7o+DgwOAn88R+xknL9YlqdVqlJWVoVevXpYOxeI0H5CmPBQ7ZswYLF26FHl5edi4caPOOhcXFwAw+KHelte8R48eAIBt27aBiHR+Lly4YNK+mmOt/amvrwfw8zliP+PkxbqkM2fOgIgwevRo7TKJRNLq5UZb1LNnT4hEIpOfd9q4cSMGDBiAK1eu6CwfPHgwunXrhm+++UZn+aVLl1BfX48RI0aYdJzevXtDLpfj6tWrJm1nKmvsj+aceHh4mHSMroCTF+sSmpqaUFpaioaGBmRnZyM6Oho+Pj544403tG38/f3x6NEjZGZmQq1Wo7i4GLdu3dLbl5ubG+7fv4+bN2+isrISarUaWVlZgh0qr1Ao4Ofnh7t375q0neZym52dnd7yZcuW4ejRo9i3bx8qKipw7do1LFy4EF5eXoiMjDT5OPPnz8eBAweQmpqKiooKNDY24u7du/jpp58AABEREfDw8GjXdE7W1B8NzTkZMmRIm/tlsyw1ztFYAhq6yQwwx/nbvn07eXp6EgBSKBQUEhJCO3fuJIVCQQAoICCA8vPz6eOPPyYnJycCQH369KEffviBiJ4MlZdKpeTt7U0SiYScnJxo2rRplJ+fr3Ochw8f0oQJE0gul1Pfvn3p3XffpRUrVhAA8vf31w6rv3z5MvXp04ccHBxo7NixVFhYSCdPniSlUkkJCQnt6qsGOnmofFRUFEmlUqqpqdEuO3r0KKlUKgJA7u7utGjRIoPbrlixQm9oeVNTE23dupUCAgJIKpWSq6srhYaGUm5urraNKefw8ePHtGrVKvLx8SGJREI9evSgsLAwysnJISKi0NBQAkBxcXHN9lFI/dGYPHkyeXt7U1NTU7P9MicBfd6mW32UAnoxmQHWcP4iIyPJzc3NojGYqrOTV15eHkkkEtq7d2+nHdOcGhsbady4cbRnzx5Lh2I2JSUlJJfL6cMPP+y0Y1rD36uR+Dkv1jXwDN0t8/f3R3x8POLj41FVVWXpcEzS2NiIzMxMVFZWIiIiwtLhmM369esxbNgwREVFWToUq2RzyevZsgeGfnx9fc1yrFGjRsHOzg7Dhg0zy/6e9tZbb0GpVEIkErV4Y7e5dlxChJkqJiYGM2fOREREhFVOVtucM2fO4MiRI8jKyjL6WTVrl5SUhKtXr+LkyZOQSqWWDscq2VzyerrsgbOzs3YIakNDA2pqavDgwQOzvcG//iX3rnoAACAASURBVPprTJgwwSz7etbu3bvxl7/8pc3tiOdCAwDExsYiLS0N5eXl6Nu3Lw4fPmzpkKxaYmIioqKi8P7771s6FKNNnDgR+/fv15mXUsiOHTuGx48f48yZM3B1dbV0OFZLYukAOoudnR0cHBzg4OCAfv36mXXfIpHIrPszB00Jka5u06ZN2LRpk6XDEJTg4GAEBwdbOowua+rUqZg6daqlw7B6NvfNyxiZmZlm3V9Hfa03Nil2RvIkImRkZBicx40xxjpbl0xeT0tOToajoyPEYjFGjBgBDw8PSKVSODo6Yvjw4Rg3bpz2oUIXFxesXLlSbx83btzAgAED4OjoCAcHB4wbN05vjrTWSiAQEbZu3Yr+/fvD3t4ezs7OWLFihd6xjGnXnhIimlg3bdqE/v37w8HBAe7u7ujbty82bdqEWbNmtfm1Zowxs7HkWEdjtHXopkqlImdnZ51lixcvpmvXrum1fe+99wgAXbp0iaqrq6mkpIRefvllAkCffvopFRcXU3V1NUVFRREAunr1qnbbiRMnkp+fHxUUFJBarabvv/+eXnjhBZLL5dpnOohaL4GwZs0aEolE9Kc//YlKS0uppqaGdu7cqVcOwdh27SkhkpiYSHZ2dnTs2DGqqamhb7/9ljw8POjFF180+TwIaOitVUEnD5VnjEhQf6+2PVS+vLxcZ5Thn//85xbbBwYGQqFQoHv37pg9ezYAwMfHB+7u7lAoFJg7dy4A6M38rFQq4evrC4lEgkGDBuEvf/kL6urqtJfYWiuBUFtbi23btuGll17C0qVL4eLiAgcHB7i5uekcx9h2rWmthEhmZiZGjBiBkJAQODg4YPjw4Zg6dSrOnj2rnWuNMcYsyaYHbDg7O2tnhAaeVME1lqaMQUNDg3aZ5t5Wa/PfDRkyBM7OzsjOzgbQegmEGzduoKamBhMnTmxxv8a2M4WhEiJ1dXV6peAbGxshlUr1ps4xVnp6etuD7KLMNeksY8YS0nvOppPXs5KTkzvtWFKpVJsQni6BsHbtWp12Xl5e2vnLNLNNN8fYdu01adIkbN26FceOHUNwcDBycnKQmZmJV199tc3JKzw83MxR2r7k5OROfc8yJiRdKnl1loaGBjx69Ag+Pj4AdEsgGPr29+WXXwIAHj9+3OJ+Nd+GWmvXXuvXr8e3336LN954A1VVVfDy8sKsWbPaNeks8XNnJhGJRDh06BAPkGGdKj09XTD/aNr0Pa/m/PTTT5g/f36H7f/LL79EU1MThg8fDqD1EgiDBw+GWCzGv/71rxb3a2y79srJyUF+fj6Ki4uhVqtx+/ZtpKam8gOTjDGr0aWSFxGhtrYWR44cgZOTk9n2W19fj/LycjQ0NODy5cuIiopCnz59tOU2WiuB0KNHD4SFheHw4cPYs2cPKioqkJ2drfdMlbHt2mvRokXw8fER3Bx3jLEuxNLjHVtj6tDNp8setPSzdu1aIiJKTk7WljHw9fWlr776ijZv3kzOzs4EgDw8PGj//v108OBB8vDwIADk6upKBw4cICKitLQ0mjBhAvXs2ZMkEgl1796dZs+eTbdu3dKJq7USCJWVlfTWW29R9+7dqVu3bjR27FiKi4sjANSrVy/67rvvjG7X3hIi//znP6l79+46r5dUKqWBAwfSkSNHOvT8sSfAQ+WZBQjo7zVdRGTdNyM012CtPEybkpqairy8PGzbtk27rL6+HqtXr0ZqaipKS0uNLkvO569t+J4XswQB/b1m8IANpqOwsBBRUVF69+dkMhl8fHygVquhVquNTl6MMdYRutQ9L9Y6BwcHSKVS7NmzBw8ePIBarcb9+/exe/duxMXFISIiwqz3CxljrC04eTEdzs7OOHXqFL7//nv069cPDg4OCAwMRFpaGjZv3oy//e1vlg6RteL06dOIiYnRq203b948vbbBwcFQKpWws7PDoEGDcPnyZQtEbBxb64+GWq3Gpk2b4O/vD5lMBhcXFwwePBg3b95sdpu6ujoMGDBA57nR48ePY8uWLV2n8KpFb7kZQUA3EJkBfP7aBm0csBEXF0dTpkyhiooK7TKVSqUdgHPixAm9bbKysmjq1Kntircz2Vp/QkNDqX///nTx4kVSq9V0//59CgkJMTgPq8bSpUsJAK1Zs0ZneXJyMo0fP55KS0vbFIuA/l5te25DxoAnc0IGBQUJ/hit2bx5Mw4ePIj09HQolUqddSkpKRCLxYiMjLSJOm+20p+DBw8iMzMTGRkZeOGFFyCRSODl5YVjx47pTCf3tPPnz+P77783uG7x4sUYOnQoJk2apDO1nS3i5MVs3p49e1BUVCT4Y7Tkxo0bWLduHTZs2KA3LyXwZDLm6Oho3Lt3D8uXL7dAhOZlK/356KOPMHz4cAwZMsSo9rW1tVixYkWL04atX78eV69etfmpxTh5MatDREhKSsLAgQNhb28PV1dXTJs2TWc2/6ioKMhkMp3S7++88w4cHR0hEolQUlIC4MlkzMuWLUN+fj5EIhH8/f2RkpICuVyOnj17YsGCBfDy8oJcLkdQUBAuXbpklmMAwGeffQYnJ6d2TatlrJSUFBARQkJCmm2TkJCAfv36Yffu3Th9+nSL+zPmHJhaI66lenZtIfT+1NfX4+LFixg2bJjR26xZswbvvPNOi/Oburq6Yvz48UhOThbCkPe2s+hVSyMI6BosM6At5y8uLo5kMhnt3buXysrKKDs7m4YPH07u7u5UWFiobTdnzhzy8PDQ2Xbr1q0EgIqLi7XLwsLCSKVS6bSLjIwkR0dHun79OtXV1VFOTg6NGjWKlEol3b592yzHOHHiBCmVSoqPjzep/0Sm3/Py8/OjwMBAg+tUKhUVFBQQEdH58+dJLBaTr68vVVVVEZHhe0TGngNja8S1Vs/OFLbSn4KCAgJAw4YNoxdffJE8PT3J3t6eBgwYQDt27KCmpiad9ufOnaOQkBAiIiouLjZ4z0sjJiZGr8afMQT0ecv3vJh1qa2tRVJSEqZPn465c+fC2dkZQ4YMwa5du1BSUmLWqbAkEon2P/HAwECkpqaisrISaWlpZtn/5MmTUVFRgXXr1pllf82prq5GQUEBVCpVq23HjBmDJUuW4ObNm1i9erXBNm05By3ViGutnl17CLk/munXevTogcTEROTk5ODBgweYNm0aFi1ahH/84x86fYiOjkZqaqpR+w4ICAAAXLt2zeh4hIaTF7MqOTk5qKqqwsiRI3WWjxo1CjKZTOeynrmNHDkSCoVCr9iotSsqKgIRQaFQGNU+ISEB/fv3x86dO3Hu3Dm99e09B8/WiGutnl17CbU/9vb2AIBBgwYhKCgIbm5ucHZ2xoYNG+Ds7KyTVGNjY/HHP/4R3t7eRu1b81548OCB0fEIDScvZlU0xUO7deumt87FxQWVlZUdenx7e3sUFxd36DHMra6uDsDPH4atkcvlSEtLg0gkwptvvona2lqd9eY+B0/Xs3u6svmtW7dQU1Nj0r4MEWp/vLy8AEB771RDJpOhT58+yM/PBwCcO3cO165dw1tvvWX0vjUz4GjeG7aIkxezKi4uLgBg8AOlrKwMvXr16rBjq9XqDj9GR9B8UJnycOqYMWOwdOlS5OXlYePGjTrrzH0Onq5nR0Q6P+aq3CvE/nTr1g0BAQG4fv263rqGhgY4OzsDeDKS9YsvvoBYLNYmSk0MiYmJEIlE+Oabb3S2r6+vBwCbnsaNkxezKoMHD0a3bt30/hgvXbqE+vp6jBgxQrtMIpFoL+WYw5kzZ0BEGD16dIcdoyP07NkTIpHI5OedNm7ciAEDBuDKlSs6y005B8ZorZ6duQixP+Hh4bhy5Qp+/PFH7bKamhrcunVLO3w+LS1NL0lqrg6sWbMGRKR3SVTzXvDw8Gh3jNaKkxezKnK5HMuWLcPRo0exb98+VFRU4Nq1a1i4cCG8vLwQGRmpbevv749Hjx4hMzMTarUaxcXFuHXrlt4+3dzccP/+fdy8eROVlZXaZNTU1ITS0lI0NDQgOzsb0dHR8PHx0dZha+8xsrKyOmWovEKhgJ+fH+7evWvSdprLbXZ2dnrLjT0Hxh6npXp2ABAREQEPD492TeckxP4sXbpUW/vv9u3bePjwIVatWoXa2tpmB6AYQ/NeMPb5MUGywBBHkwho6CYzoC3nr6mpibZu3UoBAQEklUrJ1dWVQkNDKTc3V6fdw4cPacKECSSXy6lv37707rvv0ooVKwgA+fv7a4e8X758mfr06UMODg40duxYKiwspMjISJJKpeTt7U0SiYScnJxo2rRplJ+fb7ZjnDx5kpRKJSUkJJj8usHEofJRUVEklUqppqZGu+zp2nbu7u60aNEig9uuWLFCb2i5MefAlBpxrdWzCw0NJQAUFxfXbB9trT8ad+7codmzZ5OrqyvZ29vTL3/5S8rKympxm9aGyk+ePJm8vb31htu3RkCft+lWH6WAXkxmgLWev8jISHJzc7N0GM0yNXnl5eWRRCKhvXv3dmBUHaexsZHGjRtHe/bssXQoZmHJ/pSUlJBcLqcPP/zQ5G2t9e/VAH7Oi3VdtjT7tr+/P+Lj4xEfH699fkgoGhsbkZmZicrKSkRERFg6nHazdH/Wr1+PYcOGISoqqtOP3Zk4eTFmI2JiYjBz5kxEREQIarLaM2fO4MiRI8jKyjL6WTVrZsn+JCUl4erVqzh58iSkUmmnHruzcfJiXU5sbCzS0tJQXl6Ovn374vDhw5YOyWwSExMRFRWF999/39KhGG3ixInYv3+/zhySQmap/hw7dgyPHz/GmTNn4Orq2qnHtgSJpQNgrLNt2rQJmzZtsnQYHSY4OBjBwcGWDoN1sqlTp2Lq1KmWDqPT8DcvxhhjgsPJizHGmOBw8mKMMSY4nLwYY4wJjmAGbMycOdPSIbA20ExTw+fPdNu2bUNGRoalw2BdiKlTjFmSiMi660RfuHABSUlJlg6DMZMUFhbiypUreOWVVywdCmMmE8A/TRlWn7wYE6L09HSEh4eD/7wY6xAZfM+LMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCw8mLMcaY4HDyYowxJjicvBhjjAkOJy/GGGOCI7F0AIwJnVqtRlVVlc6y6upqAEBpaanOcpFIBBcXl06LjTFbxcmLsXZ69OgRvL290djYqLfOzc1N5/cJEybgn//8Z2eFxpjN4suGjLWTh4cHfv3rX0MsbvnPSSQSYfbs2Z0UFWO2jZMXY2Ywb968VtvY2dlh+vTpnRANY7aPkxdjZhAWFgaJpPmr8HZ2dnj55ZfRvXv3ToyKMdvFyYsxM3BycsIrr7zSbAIjIsydO7eTo2LMdnHyYsxM5s6da3DQBgDIZDK8+uqrnRwRY7aLkxdjZvLqq69CoVDoLZdKpQgNDYWjo6MFomLMNnHyYsxM5HI5pk+fDqlUqrNcrVZjzpw5FoqKMdvEyYsxM3rttdegVqt1ljk5OeG3v/2thSJizDZx8mLMjF566SWdB5OlUilmz54NmUxmwagYsz2cvBgzI4lEgtmzZ2svHarVarz22msWjoox28PJizEzmz17tvbSoYeHB8aOHWvhiBizPZy8GDOzoKAgeHt7AwBef/31VqeNYoyZTu+Jyrt37+L8+fOWiIUxmzFq1Cjcu3cP3bt3R3p6uqXDYUzQZs2apbdMRET09IL09HSEh4d3WlCMMcZYS55JUwCQ0exkbAYaMyZ4mn/OOuP9ffjwYcyYMaPDj9MZRCIRDh06ZPA/YMY6SktfpvhiPGMdxFYSF2PWiJMXY4wxweHkxRhjTHA4eTHGGBMcTl6MMcYEh5MXY4wxweHkxVgbnDx5Es7Ozvjkk08sHYrVO336NGJiYnDkyBH4+flBJBJBJBJh3rx5em2Dg4OhVCphZ2eHQYMG4fLlyxaI2Di21h8NtVqNTZs2wd/fHzKZDC4uLhg8eDBu3rzZ7DZ1dXUYMGAA1q5dq112/PhxbNmypdkCre3FyYuxNuDnII3z3nvvISUlBbGxsQgLC8OPP/4IlUqF7t27Y9++ffj000912p86dQoZGRmYMmUKcnJyMHz4cAtF3jpb649GeHg4/v73v2P//v2oqanBf/7zH6hUKlRVVTW7zZo1a5Cbm6uzLCQkBHK5HBMnTkRZWZnZ4+TkxVgbTJ48GeXl5ZgyZYqlQ0FtbS2CgoIsHYaezZs34+DBg0hPT4dSqdRZl5KSArFYjMjISJSXl1soQvOxlf4cPHgQmZmZyMjIwAsvvACJRAIvLy8cO3YMgwcPNrjN+fPn8f333xtct3jxYgwdOhSTJk1CQ0ODWWPl5MWYwO3ZswdFRUWWDkPHjRs3sG7dOmzYsAFyuVxvfVBQEKKjo3Hv3j0sX77cAhGal63056OPPsLw4cMxZMgQo9rX1tZixYoVSE5ObrbN+vXrcfXq1RbbtAUnL8ZMdO7cOfj4+EAkEmHHjh0AgNTUVDg6OkKhUODYsWN45ZVX4OTkhF69euHAgQPabVNSUiCXy9GzZ08sWLAAXl5ekMvlCAoKwqVLl7TtoqKiIJPJ4OnpqV32zjvvwNHRESKRCCUlJQCA6OhoLFu2DPn5+RCJRPD39wcAfPbZZ3ByckJiYmJnvCR6UlJSQEQICQlptk1CQgL69euH3bt34/Tp0y3uj4iQlJSEgQMHwt7eHq6urpg2bRr++9//atsYew4AoLGxEXFxcfDx8YGDgwOef/55HDp0qF19Fnp/6uvrcfHiRQwbNszobdasWYN33nkHPXr0aLaNq6srxo8fj+TkZPNebqdnHDp0iAwsZswmmOv9fefOHQJA27dv1y5bs2YNAaAvvviCysvLqaioiMaNG0eOjo5UX1+vbRcZGUmOjo50/fp1qquro5ycHBo1ahQplUq6ffu2tt2cOXPIw8ND57hbt24lAFRcXKxdFhYWRiqVSqfdiRMnSKlUUnx8fLv7SkQEgA4dOmR0ez8/PwoMDDS4TqVSUUFBARERnT9/nsRiMfn6+lJVVRUREWVlZdHUqVN1tomLiyOZTEZ79+6lsrIyys7OpuHDh5O7uzsVFhZq2xl7DpYvX0729vZ0+PBhKi0tpdjYWBKLxfT1118b3Udb609BQQEBoGHDhtGLL75Inp6eZG9vTwMGDKAdO3ZQU1OTTvtz585RSEgIEREVFxcTAFqzZo3BfcfExBAAunLlitHxELX495rO37wYM7OgoCA4OTmhR48eiIiIQHV1NW7fvq3TRiKRaP/rDgwMRGpqKiorK5GWlmaWGCZPnoyKigqsW7fOLPszRXV1NQoKCqBSqVptO2bMGCxZsgQ3b97E6tWrDbapra1FUlISpk+fjrlz58LZ2RlDhgzBrl27UFJSgo8//lhvm5bOQV1dHVJTUxEaGoqwsDC4uLhg7dq1kEql7X79hdwfzYCMHj16IDExETk5OXjw4AGmTZuGRYsW4R//+IdOH6Kjo5GammrUvgMCAgAA165dMzqe1nDyYqwDyWQyANBWVm7OyJEjoVAodC4bCVVRURGICAqFwqj2CQkJ6N+/P3bu3Ilz587prc/JyUFVVRVGjhyps3zUqFGQyWQ6l1sNefYc5ObmoqamRmcAgoODAzw9Pc3y+gu1P/b29gCAQYMGISgoCG5ubnB2dsaGDRvg7Oysk1RjY2Pxxz/+UVt0tTWa98KDBw+Mjqc1nLwYsxL29vYoLi62dBjtVldXB+DnD8PWyOVypKWlQSQS4c0330Rtba3Oes0w627duult6+LigsrKSpPiq66uBgCsXbtW+4yWSCTCrVu3UFNTY9K+DBFqf7y8vABAez9VQyaToU+fPsjPzwfw5J7vtWvX8NZbbxm9bwcHBwA/vzfMgZMXY1ZArVajrKwMvXr1snQo7ab5oDLl4dQxY8Zg6dKlyMvLw8aNG3XWubi4AIDBD/W2vGaawQXbtm0DEen8XLhwwaR9NUeI/enWrRsCAgJw/fp1vXUNDQ1wdnYG8GR06xdffAGxWKxNlJoYEhMTIRKJ8M033+hsX19fD+Dn94Y5cPJizAqcOXMGRITRo0drl0kkklYvN1qjnj17QiQSmfy808aNGzFgwABcuXJFZ/ngwYPRrVs3vQ/ES5cuob6+HiNGjDDpOL1794ZcLsfVq1dN2s5UQuxPeHg4rly5gh9//FG7rKamBrdu3dIOn09LS9NLkporBmvWrAER6V0S1bwXPDw82h2jBicvxiygqakJpaWlaGhoQHZ2NqKjo+Hj44M33nhD28bf3x+PHj1CZmYm1Go1iouLcevWLb19ubm54f79+7h58yYqKyuhVquRlZVlsaHyCoUCfn5+uHv3rknbaS632dnZ6S1ftmwZjh49in379qGiogLXrl3DwoUL4eXlhcjISJOPM3/+fBw4cACpqamoqKhAY2Mj7t69i59++gkAEBERAQ8Pj3ZN5yTE/ixduhR9+vTBG2+8gdu3b+Phw4dYtWoVamtrmx2AYgzNe8HY58eMYsLQRMYEzxzv7+3bt5OnpycBIIVCQSEhIbRz505SKBQEgAICAig/P58+/vhjcnJyIgDUp08f+uGHH4joyVB5qVRK3t7eJJFIyMnJiaZNm0b5+fk6x3n48CFNmDCB5HI59e3bl959911asWIFASB/f3/tsPrLly9Tnz59yMHBgcaOHUuFhYV08uRJUiqVlJCQ0K6+asDEofJRUVEklUqppqZGu+zo0aOkUqkIALm7u9OiRYsMbrtixQq9oeVNTU20detWCggIIKlUSq6urhQaGkq5ubnaNqacg8ePH9OqVavIx8eHJBIJ9ejRg8LCwignJ4eIiEJDQwkAxcXFNdtHW+uPxp07d2j27Nnk6upK9vb29Mtf/pKysrJa3Ka1ofKTJ08mb29vveH2rWlpqDwnL9alWMP7OzIyktzc3Cwag6lMTV55eXkkkUho7969HRhVx2lsbKRx48bRnj17LB2KWViyPyUlJSSXy+nDDz80eVt+zosxK9NRM21bC39/f8THxyM+Pr7FCV2tUWNjIzIzM1FZWYmIiAhLh9Nulu7P+vXrMWzYMERFRZl1v+1OXs+WBdD8SCQSuLu746WXXsLRo0f1tjOmpMRbb70FpVIJkUikczPS0uUoLH38Dz/8UHtTfNeuXQbbNFeGwtPTE3Pnzm31GN999x0iIiLQt29f2Nvbw93dHUOHDkVCQoK2TUREhN55b+7nxIkTerG09gBtUlISRCIRxGIxBgwYgLNnz3Z4mQVmPjExMZg5cyYiIiIENVntmTNncOTIEWRlZRn9rJo1s2R/kpKScPXqVZw8eRJSqdS8Ozfha1qLVCoVOTs7a39/9OgRnT59mgYMGEAA6ODBgzrtT5w4QU5OTnT8+PEW93vgwAG9aUWM3bajWPr4RE8uywCgjz76SG9dXFwcTZkyhSoqKrTLnj0/LcnOziaFQkGLFy+mgoICqq2tpdzcXFq5ciVNnDhR2y48PJxOnTpFZWVlpFar6aeffiIAFBISQvX19VRdXU1FRUX0hz/8gT755BOdWACQp6enzhQ3T2toaKA+ffoQAJ1jEhElJyfT+PHjqbS01Kj+PM3Slw1jYmJIJpMRAPL19aWMjAyLxWIKmHjZ8Gmff/45rVq1yswRMWuXmZlJmzZtooaGhjbvo1PueTX34fj5558TAJo+fbrJ+yQynLw6U01NDY0ZM8Yix25Jc8nr/fffp379+lFtba3OclOS1+uvv07PPfec3vLHjx/Tq6++qv09IiKCqqurtb9rktezN6d37dqll7xGjBhBACg9Pd1gDIcOHaKgoCCDyYvoyYCAMWPGkFqtNqpPT+/XkslLqNqTvBhrK4ve8/L19QWANhcjE4lEZozGdNZYbqI5rZWhMNbDhw9RXl6OR48e6SyXyWQ6l0oPHDhg1GWIyMhIvPrqqzrL3n77bQBPSjAYkpSUhGXLljW7z44qs8AYE4YOT17Z2dkAgPHjx2uXGSopATwpE7B161b0798f9vb2cHZ2xooVK3T2Z2jbDz74AAqFAkqlEkVFRVi2bBm8vb2Rm5trVKmAvXv3YuTIkZDL5XB0dISvry82btxosNxES7Gbs8TBV199hcDAQDg7O0Mul2PIkCH4/PPPW3ytjSlDYYxRo0ahuroav/nNb/Dvf/+7Xftqzm9+8xsMHDgQX375pV4F1n//+9+oqalBcHBws9t3WJkFxpggdFjyqq2txWeffYbly5cjODhY57/osWPH4vz583rbrFu3DqtWrUJkZCQePHiAwsJCvQfjDG27cuVKLF26FFVVVdi0aRP69u2L0aNHg4iwevVqfPDBB9i2bRt++uknTJkyBa+99pr26fbk5GS8/vrrmDFjBu7fv4+7d+8iNjYWubm5SE5OxpQpU6BSqUBEuHHjRrOxr1+/HjExMVizZg2Kiopw9uxZ3LlzB+PGjdNORvn2229jyZIlqK2thVKpxKFDh5Cfnw8/Pz/84Q9/0JlN4cGDBwgPD8fNmzdx//59dOvWDXPmzGnxNf/000/Rv3//dt+UXblyJUaOHInvvvsOY8eOxaBBg/DBBx/ofRNrrwULFgCA3qCTP/3pT1i6dGmr2//iF7/AvXv38N1335k1LsaY9TNr8iovL9eOJFMoFNpvFnPmzGl1pEltbS22bduGl156CUuXLoWLiwscHBzg5uZmUgybN2/GokWLcOTIEfj6+rZYKkCtVmPDhg2YMGECVq9eDTc3N7i6uuJ//ud/MGrUKKOPae4SBwAwY8YMvPfee3B1dYWb9w5cOgAAIABJREFUmxtCQkLw8OHDZiduNaUMRWscHBxw/vx5/PnPf8aAAQNw/fp1rFq1CgMHDsS//vWvdu9f4/e//z0cHR3xt7/9TTt56Y8//oivv/4ar732Wqvbd0SZBcaYMEjMuTNnZ2ftva2GhgY8ePAAp06dQlRUFDZt2oRz587B3d3d4LY3btxATU0NJk6caLZ4WisVkJ2djbKyMvzud7/T2c7Ozg6LFy82+jjmLnFgiCb5NzdE3NQyFK2RSqWIiopCVFQULl26hM2bNyMzMxMzZ85Ebm4uXF1d230MZ2dnvPbaa/jLX/6CgwcPYv78+di2bRvefvttyGQy7WSezWlPmYWZM2e2KeaubNu2bcjIyLB0GKwLaWmKsQ67bCiRSODt7Y358+fjww8/RG5uLt5///1m22uCbKmctKlaKxVQUVEB4OdZntvK3CUOgCeXAF988UX06NED9vb2WLlyZYvtTS1DYYoXXngB//d//4eFCxeiuLgYX375pdn2rRm4sWvXLpSVlSEjI0N7ObE1HVFmgTEmDGb95tUczWSMhqba19CMjnv8+LHZjvt0qYDo6Gi99ZqBAs/WrzGVuUsc3L59G6GhoZg+fTr++te/4rnnnsP27dtbTGBtKUOhcfbsWXz77bdYsmQJACAsLAyHDh2CRKL79pg3bx4++ugjs9Q80hg2bBhGjx6NixcvIjIyEjNnzjT6W117yizwNwjTiEQiLFmyBLNmzbJ0KKwLSU9PR3h4uMF1nTI91LfffgsA6N+/f7NtBg8eDLFYbNZ7Kq2VCvD19YWbmxtOnTrVruOYu8TBtWvXoFar8fbbb8PPzw9yubzVRwbaWoYCeHJ+HB0dtb8/fvzY4D8ammT//PPPm3yMlmi+fR0+fFibQI3REWUWGGPCYPbkVVtbi6amJhAR7t+/j7S0NKxduxbu7u4tfjD16NEDYWFhOHz4MPbs2YOKigpkZ2cbHOxgrNZKBdjb2yM2NhZnz55FVFQU7t27h6amJlRWVmo/vA2VmzB0HHOWOPDx8QHwZIqnuro65OXltXrfrC1lKNRqNR48eIAzZ87oJC8ACA0NRXp6OsrKylBeXo5jx45h9erVmDp1qtmT16xZs+Du7o7Q0FD4+fkZvV2HlFlgjAmDCU80G/R0WYBnf+zt7SkgIIDefvttbfkGIsMlJYiIKisr6a233qLu3btTt27daOzYsRQXF0cAqFevXvTdd98Z3HbLli3k4OBAAKh37946M1m3ViqAiGjHjh00ZMgQksvlJJfL6Re/+AXt3LmTiPTLTaxdu9Zg7OYucbBq1Spyc3MjFxcXmjlzJu3YsYMAkEqloujoaPLw8CAA5OjoqJ29pLUyFC39HD16VLvNqVOnKDw8nFQqFdnb25NMJqP+/fvT+vXrqa6uTu89UFFRQb/+9a/Jzc2NAJBYLCZ/f39KTExs9r3ybAmJlStX0vnz57W/P/06i8ViCgwMpK+++kpnf20ps8AzbLQNeIYNZgFcEqWLEHoZClO0tcwCv7/bhpMXswQuidJFCLkMhak6qswCY0wYOHnZGKGWoTBFh5ZZYILRXNmfefPm6bUNDg6GUqmEnZ0dBg0ahMuXL1sgYtM1NTVh27ZtCAoKMrg+Pj4egYGBcHJygr29Pfz9/bFy5UqD/7z+4x//wKhRo6BUKtGnTx/Mnz8fhYWF2vWCKzdkwtc0JiC2WoaivWUW+P3dNrCyy4bNlf3p3r07AaATJ07obZOVlaVX8cCa/fDDD/SrX/2KANDQoUMNthk/fjzt3LmTHj58SBUVFXTo0CGSSqX08ssv67Q7ePAgAaAtW7ZQWVkZXblyhfz8/GjYsGE6lRnaU26oI/A9L8b+P2t4f3dGmR1zH8OakldLZX/2799PYrGYvL29qaysTGe9kJLX1atXafr06bRv3z4aNmxYs8lr8uTJev/IzZo1iwDoDJKbMGECPffcczqDmzSDwM6dO6ezfVvLDXUEvufFmBXpjDI7QirlY4rWyv4EBQUhOjoa9+7dw/Llyy0QoXkMHToUR44cwZw5c1qcNefEiROws7PTWaaZgu/pyQTu3LkDLy8vnedFe/fuDQC4deuWzvZCKTfEyYuxVpAR5W6ioqIgk8ng6empXfbOO+/A0dERIpFIO4uLoTI7KSkpkMvl6NmzJxYsWAAvLy/I5XIEBQXpPN/XnmMAwGeffQYnJyckJiZ26OvVkYwp+5OQkIB+/fph9+7dOH36dIv7M+bcmlLKyJgSTB3t3r17cHBwQN++fbXL/Pz89P6Z0dzvevbZSsGUGzLhaxpjgteW93dcXBzJZDLau3cvlZWVUXZ2Ng0fPpzc3d2psLBQ227OnDnk4eGhs+3WrVsJABUXF2uXhYWFkUql0mkXGRlJjo6OdP36daqrq6OcnBwaNWoUKZVKncs/7TnGiRMnSKlUUnx8vEn9J7Key4Z+fn4UGBhocJ1KpaKCggIiIjp//jyJxWLy9fWlqqoqIjJ82dDYc7tmzRoCQF988QWVl5dTUVERjRs3jhwdHam+vl7bbvny5WRvb0+HDx+m0tJSio2NJbFYTF9//XWb+/zCCy80e9nwWdXV1fT/2LvzqCjOrH/g34ZuaJZuFgVEEWRzQY2+jk6kjVHjxIxhXAgoGE2iZiboxCAuBAE1ikBUPMjByCQuhzmjiQJqMDFictSXeHw1nuQVIsE3iiiKCwEU2ZHt/v7Ij45tA3ZLQ1Ht/ZzDHz71VNWtKuxLVT/1XIVCQaGhoRrt2dnZJJPJKDk5maqqquiXX36hYcOG0WuvvdbudiIjIwWtYN+GHxsy9oyepdzNs5JKpeo7AB8fH6SkpKC6uhqpqakG2b6fnx+qqqqwbt06g2yvp+lT9sfX1xcrVqxAUVGRVk3ANoYuZdTQ0NBpCaaeEB8fD2dnZ8TGxmq0T5o0CREREQgNDYVSqcSIESNQXV2NPXv2tLsdMZQb4uTFWCe6Wu6mK8aOHQtLS0uNR1jPM33L/sTGxmLIkCHYuXMnzp49q7Xc0KWMnlaCqbsdOXIE6enp+Pbbb6FQKDSWRUdHY9euXTh16hRqampw/fp1qFQq+Pr6ori4WGtbXSk31FM4eTHWie4od6MPc3PzDguQPm/0Lfsjl8uRmpoKiUSCxYsXqwuetjH0tX1aCabudPDgQWzevBnZ2dkYNGiQxrJ79+5hy5YteO+99/DKK6/AysoK7u7u2L17N+7evYuEhASt7Ymh3BAnL8Y6YehyN/poamrq9n2IybOU/fH19cXKlStRUFCATZs2aSwz9LV9vAQTEWn8nD9/Xq9t6WPHjh3Yv38/Tp8+jf79+2stLygoQEtLi9YypVIJe3t75Ofna63TlXJDPYWTF2Od0KfcjVQq7bQatr6ys7NBRBg/fny37UNMnrXsz6ZNmzB06FDk5ORotBu6lNHTSjAZGhEhIiICeXl5yMzMbPcOEoA6Cd+7d0+jvbq6Gg8ePFAPmX+cGMoNcfJirBP6lLvx8vLCgwcPkJmZiaamJpSVlWm9QwN0XGantbUVFRUVaG5uxqVLlxAWFgZXV1csXLjQIPvIysoS9VD5Zyn7A/zx+PDJ96EMXcroaSWYACA4OBhOTk4GmZ7q8uXL2Lp1K3bv3g2ZTKbxqFIikWDbtm0AAHd3d0yZMgW7d+/GmTNnUF9fj+LiYvXxvfvuu1rbFkW5IT2GJjImes/y+61LuRsiovv379OUKVNILpeTu7s7ffDBBxQeHk4AyMvLSz3k/ckyOyUlJRQSEkIymYwGDBhAUqmUlEolzZ49mwoLCw22j+PHj5NCoaDY2Fi9zxt6yVD5p5X9ebLUzuPCw8O1hsobupTR00ow+fv7EwBav359p8d5/vx5mjBhAjk7O6vLFvXr149UKhV9//33RESUl5fXaZmjhIQE9fbKy8spLCyMvLy8yNzcnKytrWnChAn05Zdftrv/Zyk31B14eijG/r/e+vsdEhJC9vb2QofRod6SvMRe9qelpYUmTpxIe/fuFTqUDj1ruaHuwO95MSYCopnNW0BiLvvT0tKCzMxMVFdXIzg4WOhwOiSWckOcvBhjoiLWsj/Z2dk4fPgwsrKydH5XraeJqdwQJy/GBBYVFYXU1FRUVlbC3d0dhw4dEjqkXi8uLg6hoaH4+OOPhQ5FZ1OnTsXnn3+uMTdlb3L06FE8evQI2dnZsLOzEzqcp5IKHQBjz7v4+HjEx8cLHYboTJs2DdOmTRM6DKMxa9YszJo1S+gwdMZ3XowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSnw9GGEomkJ+NgrEfx77f+goKCEBQUJHQYjAFoJ3mpVCqkpaUJEQtjRuP8+fNISkri/0uMdRMJEZHQQTBmbNLT0xEUFAT+78VYt8jg77wYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6nLwYY4yJDicvxhhjosPJizHGmOhw8mKMMSY6UqEDYEzsysrK8OWXX2q0/fTTTwCAXbt2abQrFArMmzevx2JjzFhJiIiEDoIxMXv06BEcHR1RU1MDU1NTAEDbfyuJRKLu19TUhHfeeQf//ve/hQiTMWOSwY8NGesic3NzBAYGQiqVoqmpCU1NTWhubkZzc7P6301NTQCAN998U+BoGTMOnLwYM4A333wTjY2NnfaxtbXFK6+80kMRMWbcOHkxZgBTpkyBg4NDh8tlMhkWLFgAqZS/ZmbMEDh5MWYAJiYmmD9/PmQyWbvLm5qaeKAGYwbEyYsxA5k3b576u60n9e/fH76+vj0cEWPGi5MXYwby5z//GW5ublrtZmZmeOeddzRGHjLGuoaTF2MG9NZbb2k9OmxsbORHhowZGCcvxgxo/vz5Wo8Ovby8MHLkSIEiYsw4cfJizICGDh0KHx8f9SNCmUyGRYsWCRwVY8aHkxdjBvb222+rZ9pobm7mR4aMdQNOXowZ2Lx589DS0gIAGDNmDNzd3QWOiDHjw8mLMQNzdXXFiy++CAB45513BI6GMePEr/vr6Pz580hMTBQ6DCYSjx49gkQiwXfffYczZ84IHQ4TiYyMDKFDEA2+89JRcXExDh06JHQYRumHH37ADz/8IHQYBuXi4gInJyfI5fJu2f7t27f599GI8PXUH9956Yn/MjK8OXPmADC+c3vt2jV4eXl1y7bT09MRFBRkdOfsedV2PZnu+M6LsW7SXYmLMcbJizHGmAhx8mKMMSY6nLwYY4yJDicvxhhjosPJixmN48ePw8bGBl9//bXQofR6J0+eRGRkJA4fPgwPDw9IJBJIJBK89dZbWn2nTZsGhUIBU1NTDB8+HBcvXhQgYv21trZi+/btUKlU7S6PiYmBj48PlEolzM3N4eXlhQ8//BA1NTVafb/44guMGzcOCoUCbm5uWLRoEUpKStTLv/rqK2zZskU9swrrfpy8mNEgIqFDEIWPPvoIycnJiIqKQkBAAK5fvw5PT0/06dMH+/fvxzfffKPR/7vvvkNGRgZmzJiB/Px8jBkzRqDIdVdQUICXX34ZK1euRF1dXbt9Tp8+jWXLlqGoqAjl5eWIj49HUlKS+tWNNmlpaZg/fz7mzJmD27dv4+jRozhz5gymT5+O5uZmAMDMmTMhl8sxdepUPHz4sNuPj3HyYkbEz88PlZWVmDFjhtChoL6+vsO/+IW0efNmHDx4EOnp6VAoFBrLkpOTYWJigpCQEFRWVgoUYdf9/PPPWLNmDZYuXYrRo0d32M/a2hohISGwt7eHQqHA3Llz4e/vjxMnTqC4uFjd77PPPkP//v0RHh4OGxsbjB49GitXrkRubi4uXLig7rd8+XKMGjUKr7/+ujqpse7DyYuxbrB3716UlpYKHYaGa9euYd26ddi4cWO7M3+oVCqEhYXhzp07WL16tQARGsaoUaNw+PBhzJ8/H+bm5h32O3bsmHr2/zZ9+/YFAI27teLiYjg7O2tUwh44cCAA4ObNmxrrb9iwAbm5uUhKSurycbDOcfJiRuHs2bNwdXWFRCLBJ598AgBISUmBlZUVLC0tcfToUUyfPh1KpRIuLi44cOCAet3k5GTI5XI4OjpiyZIlcHZ2hlwuh0ql0vjLOjQ0FGZmZujXr5+67f3334eVlRUkEgnKy8sBAGFhYVi1ahUKCwshkUjULyufOHECSqUScXFxPXFKtCQnJ4OIMHPmzA77xMbGYvDgwdizZw9OnjzZ6faICImJiRg2bBjMzc1hZ2eH2bNn49dff1X30fUaAEBLSwvWr18PV1dXWFhY4IUXXkBaWlrXDlpPd+7cgYWFhUYlAA8PD60/RNq+7/Lw8NBot7Ozw6RJk5CUlMSPsbsbMZ2kpaURn67uERgYSIGBgV3eTnFxMQGgHTt2qNuio6MJAJ06dYoqKyuptLSUJk6cSFZWVtTY2KjuFxISQlZWVnT58mVqaGig/Px8GjduHCkUCrp165a63/z588nJyUljvwkJCQSAysrK1G0BAQHk6emp0e/YsWOkUCgoJiamy8f6LL+PHh4e5OPj0+4yT09PunHjBhERnTt3jkxMTGjQoEFUU1NDRERZWVk0a9YsjXXWr19PZmZmtG/fPnr48CFdunSJxowZQ3379qWSkhJ1P12vwerVq8nc3JwOHTpEFRUVFBUVRSYmJvTjjz/qdZyPe/HFF2nUqFE69a2trSWFQkGhoaEa7dnZ2SSTySg5OZmqqqrol19+oWHDhtFrr73W7nYiIyMJAOXk5OgcJ3++6C2d77zYc0GlUkGpVMLBwQHBwcGora3FrVu3NPpIpVL1XYSPjw9SUlJQXV2N1NRUg8Tg5+eHqqoqrFu3ziDb00dtbS1u3LgBT0/Pp/b19fXFihUrUFRUhDVr1rTbp76+HomJiXjjjTewYMEC2NjYYOTIkfj0009RXl6OXbt2aa3T2TVoaGhASkoK/P39ERAQAFtbW6xduxYymcxg5/9p4uPj4ezsjNjYWI32SZMmISIiAqGhoVAqlRgxYgSqq6uxZ8+edrfj7e0NAMjLy+v2mJ9nnLzYc8fMzAwA0NTU1Gm/sWPHwtLSUuMxmFiVlpaCiGBpaalT/9jYWAwZMgQ7d+7E2bNntZbn5+ejpqYGY8eO1WgfN24czMzMNB63tufJa3DlyhXU1dVhxIgR6j4WFhbo169fj5z/I0eOID09Hd9++63WQJbo6Gjs2rULp06dQk1NDa5fvw6VSgVfX1+NgR1t2s7xb7/91u1xP884eTHWCXNzc5SVlQkdRpc1NDQAQKcDGB4nl8uRmpoKiUSCxYsXo76+XmN523Bwa2trrXVtbW1RXV2tV3y1tbUAgLVr16rfOZNIJLh582aHQ90N5eDBg9i8eTOys7MxaNAgjWX37t3Dli1b8N577+GVV16BlZUV3N3dsXv3bty9excJCQla27OwsADwxzln3YOTF2MdaGpqwsOHD+Hi4iJ0KF3W9oGqz0u0vr6+WLlyJQoKCrBp0yaNZba2tgDQbpJ6lnPm4OAAANi+fTuISOPn/Pnzem1LHzt27MD+/ftx+vRp9O/fX2t5QUEBWlpatJYplUrY29sjPz9fa53GxkYAf5xz1j04eTHWgezsbBARxo8fr26TSqVPfdzYGzk6OkIikej9/tamTZswdOhQ5OTkaLSPGDEC1tbW+OmnnzTaL1y4gMbGRvzpT3/Saz8DBw6EXC5Hbm6uXus9KyJCREQE8vLykJmZ2e4dJAB1Er53755Ge3V1NR48eKAeMv+4tnPs5ORk4KjZ4zh5Mfb/tba2oqKiAs3Nzbh06RLCwsLg6uqKhQsXqvt4eXnhwYMHyMzMRFNTE8rKyrTe9QEAe3t73L17F0VFRaiurkZTUxOysrIEGypvaWkJDw8P3L59W6/12h4fPvk+lFwux6pVq3DkyBHs378fVVVVyMvLw9KlS+Hs7IyQkBC997No0SIcOHAAKSkpqKqqQktLC27fvq1OHMHBwXBycjLI9FSXL1/G1q1bsXv3bshkMo1HlRKJBNu2bQMAuLu7Y8qUKdi9ezfOnDmD+vp6FBcXq4/v3Xff1dp22zkeOXJkl+NknRBwqKOo8FDW7mOIofI7duygfv36EQCytLSkmTNn0s6dO8nS0pIAkLe3NxUWFtKuXbtIqVQSAHJzc6OrV68S0e9D5WUyGQ0YMICkUikplUqaPXs2FRYWauzn/v37NGXKFJLL5eTu7k4ffPABhYeHEwDy8vJSD6u/ePEiubm5kYWFBb300ktUUlJCx48fJ4VCQbGxsV06VqJn+30MDQ0lmUxGdXV16rYjR46Qp6cnAaC+ffvSsmXL2l03PDxca6h8a2srJSQkkLe3N8lkMrKzsyN/f3+6cuWKuo8+1+DRo0cUERFBrq6uJJVKycHBgQICAig/P5+IiPz9/QkArV+/vtPjPH/+PE2YMIGcnZ0JAAGgfv36kUqlou+//56IiPLy8tTL2vtJSEhQb6+8vJzCwsLIy8uLzM3NydramiZMmEBffvllu/v38/OjAQMGUGtra6dxPo4/X/SWzmdLR/zL1X0M9Z5XV4SEhJC9vb2gMejjWX4fCwoKSCqV0r59+7opqu7V0tJCEydOpL179wodSofKy8tJLpfTtm3b9FqPP1/0xu95MdbG2GcE9/LyQkxMDGJiYtqdOb03a2lpQWZmJqqrqxEcHCx0OB3asGEDRo8ejdDQUKFDMXqcvBh7jkRGRmLOnDkIDg4W1eS72dnZOHz4MLKysnR+V62nJSYmIjc3F8ePH4dMJhM6HKPHyaubPFknqe3HzMwMjo6OmDx5MhISElBRUSF0qM+9qKgopKamorKyEu7u7jh06JDQIXWruLg4hIaG4uOPPxY6FJ1NnToVn3/+uca8kr3J0aNH8ejRI2RnZ8POzk7ocJ4LnLy6yeN1kmxsbEBEaG1tRWlpKdLT0+Hu7o6IiAgMHz5ca7gx61nx8fF49OgRiAg3btxAYGCg0CF1u2nTpmHz5s1Ch2E0Zs2ahcjISK1Rmaz7cPLqQRKJBLa2tpg8eTJSU1ORnp6O3377TV2HSux6aw0rxpjx4eQloMDAQCxcuBClpaX49NNPhQ6ny3pjDSvGmHHi5CWwthdgs7KyAABbt26FpaUlFAoFSktLsWrVKgwYMABXrlzRqX6SrrWpAN3qMXW1hhVjjHUHTl4CaytTfv36dQDAhx9+iJUrV6Kmpgbx8fFwd3fH+PHjQUTYsGEDIiMjER0djdLSUpw5cwbFxcWYOHGiegbr0NBQLFy4EHV1dVi+fDmKiopw8eJFNDc349VXX9WYBVuX7SUnJ2Pu3LkaMe/cuRMbN27UaEtKSsKMGTPg6ekJIsK1a9e67ZwxxhgnL4EpFApIJJJ2JzjdvHkzli1bhsOHD8PNzU2v+klPq031LPWYGGOst5AKHcDzrra2FkQEpVLZab+u1k96sjZVV7dnaIcOHYJEIunRfRoDPmfsecXJS2BXr14FAAwdOrTTfoaon/R4bSpD12PqqvHjx2PFihU9uk8xO3/+PJKSkpCWliZ0KMwA2q4n0x0nL4GdOHECADB9+vRO+3W1ftKTtakMXY+pq1xcXLS+W2OdS0pK4nNmRDh56Ye/8xJQSUkJtm/fDhcXFyxevLjTvl2tn/RkbSp9tifWGlaMMePFyasHEBFqamrQ2toKIkJZWRnS0tIwYcIEmJqaIjMz86nfeelbP+lptan02V5Xalgxxlh34OTVTb7++muMGjUK9+7dQ0NDA2xsbGBqagpTU1MMHjwYiYmJWLhwIfLz8zXucrZu3YrExEQAwODBg7F//371so8++gjx8fGIiYlB3759MWnSJAwaNAjZ2dmwsrLS2H9DQwNGjhwJCwsLTJw4EYMHD8Z///d/w9zcXO/t/fOf/8SUKVMwb948DBkyBJs2bVKXOPf19VUPv1+6dCkcHR3h4+OD119/HQ8ePDD8iWWMMQASIiKhgxCD9PR0BAUFQQyna8mSJcjIyMD9+/eFDkUnc+bMAQBkZGQIHIl4iOn3kT0dX0+9ZfCdl5Ey9tpUjLHnGycvxp5DJ0+eRGRkpFbpnrfeekur77Rp06BQKGBqaorhw4fj4sWLAkSsv9bWVmzfvr3DyaJjYmLg4+MDpVIJc3NzeHl54cMPP2y3UOcXX3yBcePGQaFQwM3NDYsWLUJJSYl6+VdffYUtW7bwH409SYDyzaIkljLdkZGRZGZmRgBo0KBBlJGRIXRITxUYGEiBgYFChyEqXfl9XL9+Pc2YMYOqqqrUbZ6entSnTx8CQMeOHdNaJysri2bNmvXM8fa0q1ev0oQJEwgAjRo1qt0+kyZNop07d9L9+/epqqqK0tLSSCaT0V//+leNfgcPHiQAtGXLFnr48CHl5OSQh4cHjR49mpqamtT9kpKSaNKkSVRRUaF3vGL5fOlF0vnOy8g8j7WpDKEnyrn0hpIxmzdvxsGDB5Geng6FQqGxLDk5GSYmJggJCRF1iZ6ff/4Za9aswdKlS9Vzh7bH2toaISEhsLe3h0KhwNy5c+Hv748TJ05ozAH62WefoX///ggPD4eNjQ1Gjx6NlStXIjc3V2MmmuXLl2PUqFF4/fXX0dzc3K3HyPixIWMAeqaci9AlY65du4Z169Zh48aNkMvlWstVKhXCwsJw584drF69WoAIDWPUqFE4fPgw5s+frzG69knHjh3TKh7Zt29fAEBdXZ26rbi4GM7OzhpTcQ0cOBAAtF4Z2bBhA3Jzc/mF4x7AyYuJEnVzORddS8t0tWTMiROw0MH8AAAgAElEQVQnoFQqERcX163nC/j9zoqIMHPmzA77xMbGYvDgwdizZw9OnjzZ6fZ0uQYpKSmwsrKCpaUljh49iunTp0OpVMLFxQUHDhzQ2F5LSwvWr18PV1dXWFhY4IUXXujx6a/u3LkDCwsLuLu7q9s8PDy0/uho+77Lw8NDo93Ozg6TJk1CUlISjxzsboI+tRQRfibdfZ7lO6/169eTmZkZ7du3jx4+fEiXLl2iMWPGUN++famkpETdb/78+eTk5KSxbkJCAgGgsrIydVtAQAB5enpq9AsJCSErKyu6fPkyNTQ0UH5+Po0bN44UCgXdunXLIPs4duwYKRQKiomJ0ev4n+X30cPDg3x8fNpd5unpSTdu3CAionPnzpGJiQkNGjSIampqiKj977x0vQbR0dEEgE6dOkWVlZVUWlpKEydOJCsrK2psbFT3W716NZmbm9OhQ4eooqKCoqKiyMTEhH788Ue9jvNxL774YoffeT2ptraWFAoFhYaGarRnZ2eTTCaj5ORkqqqqol9++YWGDRtGr732WrvbiYyMJACUk5Ojc5z8+aI3/s6LiU9PlnN5WmmZrvLz80NVVRXWrVtnkO11pLa2Fjdu3ICnp+dT+/r6+mLFihUoKirCmjVr2u3zLNdApVJBqVTCwcEBwcHBqK2txa1btwD8/lJ9SkoK/P39ERAQAFtbW6xduxYymcxg5/pp4uPj4ezsjNjYWI32SZMmISIiAqGhoVAqlRgxYgSqq6uxZ8+edrfj7e0NAMjLy+v2mJ9nnLyY6AhZzuXJ0jJiUVpaCiKCpaWlTv1jY2MxZMgQ7Ny5E2fPntVa3tVrYGZmBgDqKcSuXLmCuro6jBgxQt3HwsIC/fr165FzfeTIEaSnp+Pbb7/VGsgSHR2NXbt24dSpU6ipqcH169ehUqk0Zpd5XNs5bivoyroHJy8mOkKXc3m8tIxYNDQ0AECnAxgeJ5fLkZqaColEgsWLF6O+vl5juaGvQW1tLQBg7dq16nfOJBIJbt68qTF4ojscPHgQmzdvRnZ2NgYNGqSx7N69e9iyZQvee+89vPLKK7CysoK7uzt2796Nu3fvIiEhQWt7bVOntZ1z1j04eTHREbKcy5OlZcSi7QNVn5dofX19sXLlShQUFGDTpk0aywx9DRwcHAAA27dvBxFp/Jw/f16vbeljx44d2L9/P06fPo3+/ftrLS8oKEBLS4vWMqVSCXt7e+Tn52ut09jYCOCPc866BycvJjpClnN5srRMd+yjOzg6OkIikej9/tamTZswdOhQ5OTkaLR3tUTPkwYOHAi5XI7c3Fy91ntWRISIiAjk5eUhMzOz3TtIAOokfO/ePY326upqPHjwQD1k/nFt59jJycnAUbPHcfJiotOT5VyeVlqmq/vIysrqkaHylpaW8PDwwO3bt/Var+3x4ZPvQ+lbokeX/SxatAgHDhxASkoKqqqq0NLSgtu3b6sTR3BwMJycnAwyPdXly5exdetW7N69GzKZTONRpUQiwbZt2wAA7u7umDJlCnbv3o0zZ86gvr4excXF6uN79913tbbddo5HjhzZ5ThZJwQc6igqPJS1+zzLUPnW1lZKSEggb29vkslkZGdnR/7+/nTlyhWNfvfv36cpU6aQXC4nd3d3+uCDDyg8PJwAkJeXl3rI+8WLF8nNzY0sLCzopZdeopKSEgoJCSGZTEYDBgwgqVRKSqWSZs+eTYWFhQbbx/Hjx0mhUFBsbKxex/8sv4+hoaEkk8morq5O3XbkyBHy9PQkANS3b19atmxZu+uGh4drDZXX5Rrs3LmTLC0tCQB5e3tTYWEh7dq1i5RKJQEgNzc3unr1KhERPXr0iCIiIsjV1ZWkUik5ODhQQEAA5efnExGRv78/AaD169d3epznz5+nCRMmkLOzMwEgANSvXz9SqVT0/fffExFRXl6eell7PwkJCertlZeXU1hYGHl5eZG5uTlZW1vThAkT6Msvv2x3/35+fjRgwABqbW3tNM7H8eeL3tL5bOmIf7m6T2+d2zAkJITs7e2FDqNdz/L7WFBQQFKplPbt29dNUXWvlpYWmjhxIu3du1foUDpUXl5Ocrmctm3bptd6/PmiN37Pi7HOGNMs4V5eXoiJiUFMTEy7M6f3Zi0tLcjMzER1dTWCg4OFDqdDGzZswOjRoxEaGip0KEaPkxdjz5HIyEjMmTMHwcHBopp8Nzs7G4cPH0ZWVpbO76r1tMTEROTm5uL48eOQyWRCh2P0OHkx1o6oqCikpqaisrIS7u7uOHTokNAhGUxcXBxCQ0Px8ccfCx2KzqZOnYrPP/9cYw7J3uTo0aN49OgRsrOzYWdnJ3Q4zwWp0AEw1hvFx8cjPj5e6DC6zbRp0zBt2jShwzAas2bNwqxZs4QO47nCd16MMcZEh5MXY4wx0eHkxRhjTHQ4eTHGGBMdHrChp/T0dKFDMDpt0+nwudVd22S1fM6MQ3dOPmysJERcq1oX6enpCAoKEjoMxpgR449jnWVw8mKsG7T9scP/vRjrFhn8nRdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHR4eTFGGNMdDh5McYYEx1OXowxxkSHkxdjjDHRkQodAGNid/v2bbzzzjtoaWlRt1VUVEChUGDy5MkafYcMGYLPPvushyNkzPhw8mKsi1xcXHDz5k0UFhZqLfv+++81/v3yyy/3VFiMGTV+bMiYAbz99tuQyWRP7RccHNwD0TBm/Dh5MWYA8+fPR3Nzc6d9hg8fDh8fnx6KiDHjxsmLMQPw9PTECy+8AIlE0u5ymUyGd955p4ejYsx4cfJizEDefvttmJqatrusubkZc+bM6eGIGDNenLwYM5B58+ahtbVVq93ExATjx4/HoEGDej4oxowUJy/GDMTZ2RkTJkyAiYnmfysTExO8/fbbAkXFmHHi5MWYAb311ltabUSEN954Q4BoGDNenLwYM6DAwECN771MTU3xl7/8BY6OjgJGxZjx4eTFmAHZ2dnh1VdfVScwIsKCBQsEjoox48PJizEDW7BggXrghkwmw+zZswWOiDHjw8mLMQObOXMmzM3NAQAzZsyAtbW1wBExZnw4eTFmYFZWVuq7LX5kyFj3kBARCR2EsZgzZw4OHTokdBiMsV4oLS0Nc+fOFToMY5HBs8ob2Pjx47FixQqhw+g1zp8/j6SkJKSlpQkdSo9qaWlBWloa3nzzzWdaPygoCGFhYfD19TVwZEwIQUFBQodgdDh5GZiLiwv/dfWEpKSk5/Kc+Pv7Qy6XP9O6QUFB8PX1fS7PmzHi5GV4/J0XY93kWRMXY+zpOHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxUTh+/DhsbGzw9ddfCx1Kr3fy5ElERkbi8OHD8PDwgEQigUQiaXfG+2nTpkGhUMDU1BTDhw/HxYsXBYhYf62trdi+fTtUKlW7y2NiYuDj4wOlUglzc3N4eXnhww8/RE1NjVbfL774AuPGjYNCoYCbmxsWLVqEkpIS9fKvvvoKW7ZsQUtLS7cdD9MfJy8mCvwuvW4++ugjJCcnIyoqCgEBAbh+/To8PT3Rp08f7N+/H998841G/++++w4ZGRmYMWMG8vPzMWbMGIEi111BQQFefvllrFy5EnV1de32OX36NJYtW4aioiKUl5cjPj4eSUlJWtWs09LSMH/+fMyZMwe3b9/G0aNHcebMGUyfPh3Nzc0Afp/uSy6XY+rUqXj48GG3Hx/TDScvJgp+fn6orKzEjBkzhA4F9fX1Hf7FL6TNmzfj4MGDSE9Ph0Kh0FiWnJwMExMThISEoLKyUqAIu+7nn3/GmjVrsHTpUowePbrDftbW1ggJCYG9vT0UCgXmzp0Lf39/nDhxAsXFxep+n332Gfr374/w8HDY2Nhg9OjRWLlyJXJzc3HhwgV1v+XLl2PUqFF4/fXX1UmNCYuTF2N62rt3L0pLS4UOQ8O1a9ewbt06bNy4sd33y1QqFcLCwnDnzh2sXr1agAgNY9SoUTh8+DDmz5+vnvy4PceOHdOoqwYAffv2BQCNu7Xi4mI4OztDIpGo2wYOHAgAuHnzpsb6GzZsQG5uLpKSkrp8HKzrOHmxXu/s2bNwdXWFRCLBJ598AgBISUmBlZUVLC0tcfToUUyfPh1KpRIuLi44cOCAet3k5GTI5XI4OjpiyZIlcHZ2hlwuh0ql0vjLOjQ0FGZmZujXr5+67f3334eVlRUkEgnKy8sBAGFhYVi1ahUKCwshkUjg5eUFADhx4gSUSiXi4uJ64pRoSU5OBhFh5syZHfaJjY3F4MGDsWfPHpw8ebLT7REREhMTMWzYMJibm8POzg6zZ8/Gr7/+qu6j6zUAfp8ua/369XB1dYWFhQVeeOGFHp8y7M6dO7CwsIC7u7u6zcPDQ+sPkbbvuzw8PDTa7ezsMGnSJCQlJfFj7N6AmMEEBgZSYGCg0GH0KmlpaWSIX7Pi4mICQDt27FC3RUdHEwA6deoUVVZWUmlpKU2cOJGsrKyosbFR3S8kJISsrKzo8uXL1NDQQPn5+TRu3DhSKBR069Ytdb/58+eTk5OTxn4TEhIIAJWVlanbAgICyNPTU6PfsWPHSKFQUExMTJePlYgIAKWlpenc38PDg3x8fNpd5unpSTdu3CAionPnzpGJiQkNGjSIampqiIgoKyuLZs2apbHO+vXryczMjPbt20cPHz6kS5cu0ZgxY6hv375UUlKi7qfrNVi9ejWZm5vToUOHqKKigqKiosjExIR+/PFHnY/xSS+++CKNGjVKp761tbWkUCgoNDRUoz07O5tkMhklJydTVVUV/fLLLzRs2DB67bXX2t1OZGQkAaCcnBy9YtX3erKnSuc7LyZ6KpUKSqUSDg4OCA4ORm1tLW7duqXRRyqVqu8ifHx8kJKSgurqaqSmphokBj8/P1RVVWHdunUG2Z4+amtrcePGDXh6ej61r6+vL1asWIGioiKsWbOm3T719fVITEzEG2+8gQULFsDGxgYjR47Ep59+ivLycuzatUtrnc6uQUNDA1JSUuDv74+AgADY2tpi7dq1kMlkBjv/TxMfHw9nZ2fExsZqtE+aNAkREREIDQ2FUqnEiBEjUF1djT179rS7HW9vbwBAXl5et8fMOsfJixkVMzMzAEBTU1On/caOHQtLS0uNx2BiVVpaCiKCpaWlTv1jY2MxZMgQ7Ny5E2fPntVanp+fj5qaGowdO1ajfdy4cTAzM9N43NqeJ6/BlStXUFdXhxEjRqj7WFhYoF+/fj1y/o8cOYL09HR8++23WgNZoqOjsWvXLpw6dQo1NTW4fv06VCoVfH19NQZ2tGk7x7/99lu3x806x8mLPbfMzc1RVlYmdBhd1tDQAACdDmB4nFwuR2pqKiQSCRYvXoz6+nqN5W3DwdurAG1ra4vq6mq94qutrQUArF27Vv3OmUQiwc2bNzsc6m4oBw8exObNm5GdnY1BgwZpLLt37x62bNmC9957D6+88gqsrKzg7u6O3bt34+7du0hISNDanoWFBYA/zjkTDicv9lxqamrCw4cP4eLiInQoXdb2garPS7S+vr5YuXIlCgoKsGnTJo1ltra2ANBuknqWc+bg4AAA2L59O4hI4+f8+fN6bUsfO3bswP79+3H69Gn0799fa3lBQQFaWlq0limVStjb2yM/P19rncbGRgB/nHMmHE5e7LmUnZ0NIsL48ePVbVKp9KmPG3sjR0dHSCQSvd/f2rRpE4YOHYqcnByN9hEjRsDa2ho//fSTRvuFCxfQ2NiIP/3pT3rtZ+DAgZDL5cjNzdVrvWdFRIiIiEBeXh4yMzPbvYMEoE7C9+7d02ivrq7GgwcP1EPmH9d2jp2cnAwcNdMXJy/2XGhtbUVFRQWam5tx6dIlhIWFwdXVFQsXLlT38fLywoMHD5CZmYmmpiaUlZVpvesDAPb29rh79y6KiopQXV2NpqYmZGVlCTZU3tLSEh4eHrh9+7Ze67U9PnzyfSi5XI5Vq1bhyJEj2L9/P6qqqpCXl4elS5fC2dkZISEheu9n0aJFOHDgAFJSUlBVVYWWlhbcvn1bnTiCg4Ph5ORkkOmpLl++jK1bt2L37t2QyWQajyolEgm2bdsGAHB3d8eUKVOwe/dunDlzBvX19SguLlYf37vvvqu17bZzPHLkyC7HybpIwKGORoeHymszxFD5HTt2UL9+/QgAWVpa0syZM2nnzp1kaWlJAMjb25sKCwtp165dpFQqCQC5ubnR1atXiej3ofIymYwGDBhAUqmUlEolzZ49mwoLCzX2c//+fZoyZQrJ5XJyd3enDz74gMLDwwkAeXl5qYfVX7x4kdzc3MjCwoJeeuklKikpoePHj5NCoaDY2NguHWsb6Dm0OjQ0lGQyGdXV1anbjhw5Qp6engSA+vbtS8uWLWt33fDwcK2h8q2trZSQkEDe3t4kk8nIzs6O/P396cqVK+o++lyDR48eUUREBLm6upJUKiUHBwcKCAig/Px8IiLy9/cnALR+/fpOj/P8+fM0YcIEcnZ2JgAEgPr160cqlYq+//57IiLKy8tTL2vvJyEhQb298vJyCgsLIy8vLzI3Nydra2uaMGECffnll+3u38/PjwYMGECtra2dxvkkfa8ne6p0Tl4GxMlLm6He8+qKkJAQsre3FzQGfen7YVdQUEBSqZT27dvXjVF1n5aWFpo4cSLt3btX6FA6VF5eTnK5nLZt26b3upy8DI7f82LPB2OfEdzLywsxMTGIiYlpd+b03qylpQWZmZmorq5GcHCw0OF0aMOGDRg9ejRCQ0OFDoWBv/MS1JMlK9p+zMzM4OjoiMmTJyMhIQEVFRVCh8pEIDIyEnPmzEFwcLCoJt/Nzs7G4cOHkZWVpfO7aj0tMTERubm5OH78OGQymdDhMHDyEtTjJStsbGxARGhtbUVpaSnS09Ph7u6OiIgIDB8+XGvkF9NNVFQUUlNTUVlZCXd3dxw6dEjokLpVXFwcQkND8fHHHwsdis6mTp2Kzz//XGNeyd7k6NGjePToEbKzs2FnZyd0OOz/kwodANMkkUhga2uLyZMnY/LkyfDz80NQUBD8/Pxw9epV2NjYCB2iqMTHxyM+Pl7oMHrUtGnTMG3aNKHDMBqzZs3CrFmzhA6DPYHvvHq5wMBALFy4EKWlpfj000+FDocxxnoFTl4i0PYuUlZWlrqtsxIT+pSq+P777/HnP/8ZlpaWUCqVGDlyJKqqqp66D8YYExInLxFoqxh7/fp1dduaNWuwdetWbN++Hffu3cOMGTPw5ptv4qeffsI///lPrFixAvX19VAoFEhLS0NhYSE8PDzwj3/8Qz2LRG1tLWbOnInAwEA8ePAABQUFGDx4sHoKnM72wRhjQuLkJQIKhQISiUQ915w+JSY6K1VRVFSEqqoqDB8+HHK5HE5OTjh8+DD69u3bK8pYMMZYR3jAhgjU1taCiKBUKgE8e4mJJ0tVeHh4wNHREQsWLMDy5cuxcOFC9czbhi5jkZ6ervc6z7vunLSWMbHj5CUCV69eBQAMHToUgGaJibVr12r0dXZ21nm7FhYWOH36NNasWYO4uDjExMRg7ty5SE1NNdg+2gQFBem9zvMuKSkJSUlJQofBWK/Ejw1F4MSJEwCA6dOnAzBsiYnhw4fj66+/xt27dxEREYG0tDRs27bN4GUsntwG/3T+AwBpaWmCx8E/hruezLA4efVyJSUl2L59O1xcXLB48WIAhisxcffuXVy+fBnA7wnx448/xpgxY3D58uUeL2PBGGP64OTVSxARampq0NraCiJCWVkZ0tLSMGHCBJiamiIzM1P9nZcuJSZ0cffuXSxZsgS//vorGhsbkZOTg5s3b2L8+PEG2wdjjHUHTl4C+vrrrzFq1Cjcu3cPDQ0NsLGxgampKUxNTTF48GAkJiZi4cKFyM/P1yoAmJSUhBUrVmDLli3o06cPnJ2dERYWhoqKCqSkpGD79u0AgBdeeAHXr1/H7t27sWrVKgDAX//6VxQUFMDBwQEtLS1QqVSwtLTE3/72NyxZsgTLli176j4YY0xIEuIHsgYzZ84cAEBGRobAkfQe6enpCAoK4uf+epJIJEhLS8PcuXOFDoUZAF9Pg8vgOy/GGGOiw8mLMcaY6HDyYuw5cPLkSURGRmrVkHvrrbe0+k6bNg0KhQKmpqYYPnw4Ll68KEDE+mlqakJ8fDy8vLxgZmYGW1tbjBgxAkVFRR2u09DQgKFDh2q8x/jVV19hy5YtRl+81Bhw8mLMyH300UdITk5GVFSURg25Pn36YP/+/fjmm280+n/33XfIyMjAjBkzkJ+fjzFjxggUue6CgoLwn//8B59//jnq6urwf//3f/D09Oy0qnR0dDSuXLmi0TZz5kzI5XJMnToVDx8+7O6wWRdw8mJGr76+HiqVSvT7eBabN2/GwYMHkZ6eDoVCobEsOTkZJiYmCAkJEVXl5ScdPHgQmZmZyMjIwIsvvgipVApnZ2ccPXpUY3qzx507dw6//PJLu8uWL1+OUaNG4fXXX0dzc3N3hs66gJMXM3p79+5FaWmp6Pehr2vXrmHdunXYuHEj5HK51nKVSoWwsDDcuXMHq1evFiBCw/jXv/6FMWPGYOTIkTr1r6+vR3h4eKdTb23YsAG5ubk8PVcvxsmL9TpEhMTERAwbNgzm5uaws7PD7NmzNSYEDg0NhZmZmUbp+Pfffx9WVlaQSCQoLy8HAISFhWHVqlUoLCyERCKBl5cXkpOTIZfL4ejoiCVLlsDZ2RlyuRwqlQoXLlwwyD6A36f1UiqViIuL69bz1ZHk5GQQEWbOnNlhn9jYWAwePBh79uzByZMnO92eLtdFn1pyhqgX19jYiB9++EFdNkgX0dHReP/999VToLXHzs4OkyZNQlJSEr/m0VsRM5jAwEAKDAwUOoxeJS0tjfT9NVu/fj2ZmZnRvn376OHDh3Tp0iUaM2YM9e3bl0pKStT95s+fT05OThrrJiQkEAAqKytTtwUEBJCnp6dGv5CQELKysqLLly9TQ0MD5efn07hx40ihUNCtW7cMso9jx46RQqGgmJgYvY6fiAgApaWl6b3e4zw8PMjHx6fdZZ6ennTjxg0iIjp37hyZmJjQoEGDqKamhoiIsrKyaNasWRrr6HpdoqOjCQCdOnWKKisrqbS0lCZOnEhWVlbU2Nio7rd69WoyNzenQ4cOUUVFBUVFRZGJiQn9+OOPOh/jjRs3CACNHj2aJk+eTP369SNzc3MaOnQoffLJJ9Ta2qrR/+zZszRz5kwiIiorKyMAFB0d3e62IyMjCQDl5OToHE9HDHE9mYZ0vvNivUp9fT0SExPxxhtvYMGCBbCxscHIkSPx6aefory8HLt27TLYvqRSqfouwsfHBykpKaiurjZYvTI/Pz9UVVVh3bp1BtmePmpra3Hjxg14eno+ta+vry9WrFiBoqIirFmzpt0+z3JdOqslZ6h6cW0DMhwcHBAXF4f8/Hz89ttvmD17NpYtW4YvvvhC4xjCwsKQkpKi07a9vb0BAHl5eTrHw3oOJy/Wq+Tn56OmpgZjx47VaB83bhzMzMw0HusZ2tixY2FpaflM9cp6m9LSUhARLC0tdeofGxuLIUOGYOfOnTh79qzW8q5elydryRmqXpy5uTmA36sjqFQq2Nvbw8bGBhs3boSNjY1GUo2KisJ7772HAQMG6LTttnP322+/6RwP6zmcvFiv0jY82draWmuZra2tupp0dzE3N0dZWVm37qMnNDQ0APjjw/1p5HI5UlNTIZFIsHjxYtTX12ssN/R1ebxeXNs7ZxKJBDdv3kRdXZ3O22mrLdf2/WMbMzMzuLm5obCwEABw9uxZ5OXl4e9//7vO27awsADwx7lkvQsnL9ar2NraAkC7H4YPHz6Ei4tLt+27qamp2/fRU9o+ePV52dbX1xcrV65EQUEBNm3apLHM0NfFUPXirK2t4e3trS7t87jm5mbY2NgA+H006KlTp2BiYqJOlG0xxMXFQSKR4KefftJYv7GxEcAf55L1Lpy8WK8yYsQIWFtba32QXLhwAY2NjRqz60ulUvVjKEPIzs4GEWH8+PHdto+e4ujoCIlEovf7W5s2bcLQoUORk5Oj0a7PddGFIevFBQUFIScnB9evX1e31dXV4ebNm+rh86mpqVpJsu0OOzo6GkSk9Ui07dw5OTl1OUZmeJy8WK8il8uxatUqHDlyBPv370dVVRXy8vKwdOlSODs7IyQkRN3Xy8sLDx48QGZmJpqamlBWVoabN29qbdPe3h53795FUVERqqur1cmotbUVFRUVaG5uxqVLlxAWFgZXV1csXLjQIPvIysoSbKi8paUlPDw8cPv2bb3Wa3t8aGpqqtWu63XRdT9PqxcXHBwMJyenp05PtXLlSri5uWHhwoW4desW7t+/j4iICNTX13c4AEUXbedO1/fHWA8TaJijUeKh8tqeZah8a2srJSQkkLe3N8lkMrKzsyN/f3+6cuWKRr/79+/TlClTSC6Xk7u7O33wwQcUHh5OAMjLy0s95P3ixYvk5uZGFhYW9NJLL1FJSQmFhISQTCajAQMGkFQqJaVSSbNnz6bCwkKD7eP48eOkUCgoNjZW7/MGAwytDg0NJZlMRnV1deq2I0eOkKenJwGgvn370rJly9pdNzw8XGuovC7XZefOnWRpaUkAyNvbmwoLC2nXrl2kVCoJALm5udHVq1eJiOjRo0cUERFBrq6uJJVKycHBgQICAig/P5+IiPz9/WcnRHMAACAASURBVAkArV+//qnHWlxcTPPmzSM7OzsyNzenP//5z5SVldXpOk8bKu/n50cDBgzQGm7/LAxxPZmGdE5eBsTJS9uzJK+eEBISQvb29kKH0SFDfNgVFBSQVCqlffv2GSiqntXS0kITJ06kvXv39vi+y8vLSS6X07Zt2wyyPU5eBsfvebHnl7HPHO7l5YWYmBjExMR0OkFtb9TS0oLMzExUV1cjODi4x/e/YcMGjB49GqGhoT2+b6YbTl6MGbHIyEjMmTMHwcHBopp8Nzs7G4cPH0ZWVpbO76oZSmJiInJzc3H8+HHIZLIe3TfTHScv9tyJiopCamoqKisr4e7ujkOHDgkdUreKi4tDaGgoPv74Y6FD0dnUqVPx+eefa8wr2ROOHj2KR48eITs7G3Z2dj26b6YfqdABMNbT4uPjER8fL3QYPWratGmYNm2a0GH0erNmzcKsWbOEDoPpgO+8GGOMiQ4nL8YYY6LDyYsxxpjocPJijDEmOjxgw8B++OEHzJkzR+gweo22KXb4nOhv+/btyMjIEDoMxnolTl4G5OvrK3QIvY6LiwsCAwOFDqPHlZSUICcnB9OnT3+m9Z/Hc2bMAgMDMXDgQKHDMCoSIiKhg2DM2KSnpyMoKAj834uxbpHB33kxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0OHkxxhgTHU5ejDHGRIeTF2OMMdHh5MUYY0x0pEIHwJjYNTU1oaamRqOttrYWAFBRUaHRLpFIYGtr22OxMWasOHkx1kUPHjzAgAED0NLSorXM3t5e499TpkzB6dOneyo0xowWPzZkrIucnJzw8ssvw8Sk8/9OEokE8+bN66GoGDNunLwYM4C33nrrqX1MTU3xxhtv9EA0jBk/Tl6MGUBAQACk0o6fwpuamuKvf/0r+vTp04NRMWa8OHkxZgBKpRLTp0/vMIERERYsWNDDUTFmvDh5MWYgCxYsaHfQBgCYmZnhb3/7Ww9HxJjx4uTFmIH87W9/g6WlpVa7TCaDv78/rKysBIiKMePEyYsxA5HL5XjjjTcgk8k02puamjB//nyBomLMOHHyYsyA3nzzTTQ1NWm0KZVKvPrqqwJFxJhx4uTFmAH95S9/0XgxWSaTYd68eTAzMxMwKsaMDycvxgxIKpVi3rx56keHTU1NePPNNwWOijHjw8mLMQObN2+e+tGhk5MTXnrpJYEjYsz4cPJizMBUKhUGDBgAAHj77befOm0UY0x/PDFvF5w/fx7FxcVCh8F6oXHjxuHOnTvo06cP0tPThQ6H9UIqlQouLi5ChyFaEiIioYMQqzlz5uDQoUNCh8EYE6G0tDTMnTtX6DDEKoPvvLooMDAQGRkZQodhdCQSiej/cx86dAiBgYE9tr85c+YAAP8+ioBEIhE6BNHjh/GMdZOeTFyMPW84eTHGGBMdTl6MMcZEh5MXY4wx0eHkxRhjTHQ4eTHGGBMdTl7MqB0/fhw2Njb4+uuvhQ6l1zt58iQiIyNx+PBheHh4QCKRQCKR4K233tLqO23aNCgUCpiammL48OG4ePGiABHrp6mpCfHx8fDy8oKZmRlsbW0xYsQIFBUVdbhOQ0MDhg4dirVr16rbvvrqK2zZsqXDwqOsZ3DyYkaN38HXzUcffYTk5GRERUUhICAA169fh6enJ/r06YP9+/fjm2++0ej/3XffISMjAzNmzEB+fj7GjBkjUOS6CwoKwn/+8x98/vnnqKurw//93//B09MTNTU1Ha4THR2NK1euaLTNnDkTcrkcU6dOxcOHD7s7bNYBTl7MqPn5+aGyshIzZswQOhTU19dDpVIJHYaWzZs34+DBg0hPT4dCodBYlpycDBMTE4SEhKCyslKgCLvu4MGDyMzMREZGBl588UVIpVI4Ozvj6NGjGDFiRLvrnDt3Dr/88ku7y5YvX45Ro0bh9ddfR3Nzc3eGzjrAyYuxHrJ3716UlpYKHYaGa9euYd26ddi4cSPkcrnWcpVKhbCwMNy5cwerV68WIELD+Ne//oUxY8Zg5MiROvWvr69HeHg4kpKSOuyzYcMG5ObmdtqHdR9OXsxonT17Fq6urpBIJPjkk08AACkpKbCysoKlpSWOHj2K6dOnQ6lUwsXFBQcOHFCvm5ycDLlcDkdHRyxZsgTOzs6Qy+VQqVS4cOGCul9oaCjMzMzQr18/ddv7778PKysrSCQSlJeXAwDCwsKwatUqFBYWQiKRwMvLCwBw4sQJKJVKxMXF9cQp0ZKcnAwiwsyZMzvsExsbi8GDB2PPnj04efJkp9sjIiQmJmLYsGEwNzeHnZ0dZs+ejV9//VXdR9drAAAtLS1Yv349XF1dYWFhgRdeeAFpaWl6HWNjYyN++OEHjB49Wud1oqOj8f7778PBwaHDPnZ2dpg0aRKSkpL48bQAOHkxo/XSSy/h3LlzGm3//Oc/sWLFCtTX10OhUCAtLQ2FhYXw8PDAP/7xD3UdrtDQUCxcuBB1dXVYvnw5ioqKcPHiRTQ3N+PVV19VVxNITk7Wmn9x586d2Lhxo0ZbUlISZsyYAU9PTxARrl27BgDqL/1bW1u75Rw8zTfffIMhQ4bA0tKywz4WFhb497//DRMTE/zjH/9AbW1th303bNiAyMhIREdHo7S0FGfOnEFxcTEmTpyI3377DYDu1wAA1qxZg61bt2L79u24d+8eZsyYgTfffBM//fSTzsd49+5dNDY24n//938xZcoU9R8iw4YNw86dO7USz//8z/+gsLBQpyKi//Vf/4U7d+7g559/1jkeZhicvNhzS6VSQalUwsHBAcHBwaitrcWtW7c0+kilUvVdhI+PD1JSUlBdXY3U1FSDxODn54eqqiqsW7fOINvTR21tLW7cuAFPT8+n9vX19cWKFStQVFSENWvWtNunvr4eiYmJeOONN7BgwQLY2Nhg5MiR+PTTT1FeXo5du3ZprdPZNWhoaEBKSgr8/f0REBAAW1tbrF27FjKZTK/z3zYgw8HBAXFxccjPz8dvv/2G2bNnY9myZfjiiy80jiEsLAwpKSk6bdvb2xsAkJeXp3M8zDA4eTEGwMzMDAA0/upvz9ixY2FpaanxGEysSktLQUSd3nU9LjY2FkOGDMHOnTtx9uxZreX5+fmoqanB2LFjNdrHjRsHMzMzjcet7XnyGly5cgV1dXUaAyosLCzQr18/vc6/ubk5AGD48OFQqVSwt7eHjY0NNm7cCBsbG42kGhUVhffee09dTPRp2s5d210l6zmcvBjTk7m5OcrKyoQOo8saGhoA/PHh/jRyuRypqamQSCRYvHgx6uvrNZa3DRu3trbWWtfW1hbV1dV6xdf2eHLt2rXqd84kEglu3ryJuro6nbfj7OwMAOrvH9uYmZnBzc0NhYWFAH7/jjQvLw9///vfdd62hYUFgD/OJes5nLwY00NTUxMePnxoFBVw2z549XnZ1tfXFytXrkRBQQE2bdqksczW1hYA2k1Sz3LO2gZLbN++HUSk8XP+/Hmdt2NtbQ1vb29cvnxZa1lzczNsbGwA/D4a9NSpUzAxMVEnyrYY4uLiIJFItL5ra2xsBPDHuWQ9h5MXY3rIzs4GEWH8+PHqNqlU+tTHjb2Ro6MjJBKJ3u9vbdq0CUOHDkVOTo5G+4gRI2Btba31AX/hwgU0NjbiT3/6k177GThwIORyOXJzc/Varz1BQUHIycnB9evX1W11dXW4efOmevh8amqqVpJsu8OOjo4GEWk9Em07d05OTl2OkemHkxdjnWhtbUVFRQWam5tx6dIlhIWFwdXVFQsXLlT38fLywoMHD5CZmYmmpiaUlZXh5s2bWtuyt7fH3bt3UVRUhOrqajQ1NSErK0uwofKWlpbw8PDA7du39Vqv7fGhqampVvuqVatw5MgR7N+/H1VVVcjLy8PSpUvh7OyMkJAQvfezaNEiHDhwACkpKaiqqkJLSwtu376Ne/fuAQCCg4Ph5OT01OmpVq5cCTc3NyxcuBC3bt3C/fv3ERERgfr6+g4HoOii7dzp+v4YMxxOXsxoffLJJxg3bhwAIOL/tXfvUVGV6x/AvxsGGMDhpoAEichFEm+HkyUURz2uwyo9iiYKZp3IVaFlCKgh4BUBrwtYnKSW5aKVmnLRoFI8LSsqj9rlKEVYhijeFfDG/f78/ug3k+MgzsjM7Nn4fNbiD/d+936f2Xvgce/97vdJSEBYWBhycnKQmZkJABg9ejTOnDmD9957D0uWLAEAPPPMM6isrFTto7W1FaNGjYK1tTVCQkLg5+eHr776Su050euvv45JkyZh7ty5GD58ONatW6e6jRQUFKQaVr9w4UK4uLhgxIgRmDJlCm7cuGGU49CbqVOnoqKiQu351ccffwwfHx9UVVVh3LhxePPNNzW2Gz9+POLj4zWWr169Gunp6UhJScGgQYMwYcIEDB06FKWlpbC1tQUAnc5BVlYW4uLisHHjRgwcOBBubm6IjY3FzZs3Afxx266mpgbFxcW9fk5HR0d8++238PDwwNixY+Hu7o7vv/8e+/fv1+n9r7v98MMPcHd3x+jRox94H+wBEXtg4eHhFB4eLnYY/RIAysvLEzWG6OhocnJyEjUGXTzI97GyspJkMhnt2LHDQFEZVldXF4WEhND27duN3nddXR3J5XLasmWLztuawvdb4vL5youxXvT3mcN9fHyQkpKClJSUXieoNUVdXV0oKipCQ0MDIiMjjd7/mjVrMHbsWMTExBi9b8a3DY3q7lITyh9LS0u4uLhg4sSJ2Lx5s+qWCGPGkJiYiNmzZyMyMlJSk++WlpZi7969KCkp0fpdNX3JyMhAWVkZDhw4AAsLC6P2zf7AycuI7iw1YW9vDyJCd3c3ampqkJ+fDy8vLyQkJCAgIECn6W+Y/iUlJSE3Nxe3b9+Gl5cXCgsLxQ7JoNLS0hATE4P169eLHYrWJk+ejF27dqnNK2kMxcXFaGtrQ2lpKRwdHY3aN/sTJy+RCYIABwcHTJw4Ebm5ucjPz8e1a9dUpTykzlTLgNxPeno62traQEQ4e/YswsPDxQ7J4EJDQ7FhwwaxwzB5YWFhSExM1BhtyYyLk5eJCQ8PR1RUFGpqavDuu++KHU6fmWIZEMaY9HHyMkHKd4hKSkoAAJs2bYKNjQ0UCgVqamqwZMkSuLu749SpU1qVoNC2vAegXUmLvpYBYYyxvuLkZYKU750oZwN46623EB8fj8bGRqSnp8PLywvjx48HEWlVgkLb8h6AdiUt+loGhDHG+oqTlwlSKBQQBKHHOeI2bNiARYsWYe/evfD09NSpBMX9yns8SEkLxhgTg0zsAJimpqYmEBHs7Ox6bdfXEhR3l/fo6/70LTMzEwUFBUbtU8qOHTsGAJg9e7bIkTBmeHzlZYJ+//13AIC/v3+v7fRRguLO8h76LmnBGGOGwldeJujgwYMAgGeffbbXdn0tQXF3eQ99l7Toq7i4OI1na+zelFdcfLVq+gRBEDsEyeMrLxNz9epVZGZmwsPDA/Pnz++1bV9LUNxd3kOX/Um1DAhjrH/g5CUSIkJjYyO6u7tVdYPy8vLw1FNPwdzcHEVFRfd95qVrCYr7lffQZX99KQPCGGN9xcnLiD799FOMGTMGV65cQWtrK+zt7WFubg5zc3P4+fkhIyMDUVFRqKioULvK2bRpEzIyMgAAfn5+2Llzp2qdNiUolLQp76Ht/qRcBoQxJn0CEZHYQUiVlJ4xLFiwAAUFBbh+/brYoWhFEATk5eXxMy8dSOn7+LDj73efFfCV10Okv5f3YIw9PDh5McYAAIcOHUJiYqJG6Z4XX3xRo21oaCgUCgXMzc0REBCA48ePixCxbjo6OpCeng4fHx9YWlrCwcEBI0eORHV19T23aW1thb+/P1asWKFa9sknn2Djxo38n0GRcfJ6CDxs5T2Y7lavXo3s7GwkJSWple4ZOHAgdu7cif3796u1//zzz1FQUIBp06ahoqICgYGBIkWuvYiICHz44YfYtWsXmpub8euvv8Lb27vXIpzJyck4deqU2rLp06dDLpdj8uTJqncjmfFx8noIPIzlPfrKGKVcTKVczIYNG7Bnzx7k5+dDoVCorcvOzoaZmRmio6MlXaJnz549KCoqQkFBAZ588knIZDK4ubmhuLgYI0eO7HGbI0eO4Jdffulx3eLFizFmzBhMmTIFnZ2dhgyd3QMnL8Z6YIxSLqZQLub06dNYuXIl1q5dC7lcrrE+ODgYsbGxuHTpEpYuXSpChPrxzjvvIDAwEKNGjdKqfUtLC5YtW4asrKx7tlmzZg3Kysp6bcMMh5MX6xcMXcpF27IyfS0Xc/DgQdjZ2SEtLc2gx0spOzsbRITp06ffs01qair8/Pzw/vvv49ChQ73uT5vzkJOTA1tbW9jY2KC4uBjPPvss7Ozs4OHhgd27d6vtr6urC6tWrcKQIUNgbW2N0aNHIy8vT6fP2N7ejmPHjqmqNWgjOTkZb7zxBpydne/ZxtHRERMmTEBWVhZ40LYIiD2w8PBwCg8PFzuMfgkA5eXlad1+1apVZGlpSTt27KBbt27Rzz//TIGBgTRo0CC6evWqqt28efPI1dVVbdvNmzcTAKqtrVUtmzVrFnl7e6u1i46OJltbWzp58iS1trZSRUUFjRs3jhQKBZ0/f14vfXz22WekUCgoJSVF68+u9CDfx2HDhtGIESN6XOft7U1nz54lIqIjR46QmZkZDR06lBobG4mIqKSkhMLCwtS20fY8JCcnEwD64osv6Pbt21RTU0MhISFka2tL7e3tqnZLly4lKysrKiwspJs3b1JSUhKZmZnRDz/8oPVnPHv2LAGgsWPH0sSJE2nw4MFkZWVF/v7+9Pbbb1N3d7da+8OHD9P06dOJiKi2tpYAUHJyco/7TkxMJAB04sQJreMh0v37zTTk85UXkzxjlnK5X1mZvpo6dSrq6+uxcuVKveyvN01NTTh79iy8vb3v2zYoKAhxcXGorq7G8uXLe2zzIOchODgYdnZ2cHZ2RmRkJJqamnD+/HkAf4z0y8nJwcyZMzFr1iw4ODhgxYoVsLCw0Ol4KwdkODs7Iy0tDRUVFbh27RpmzJiBRYsW4aOPPlL7DLGxscjJydFq376+vgCA8vJyreNh+sHJi0memKVc7i4rIyU1NTUgItjY2GjVPjU1FcOHD8fWrVtx+PBhjfV9PQ+WlpYAoJpC7NSpU2hublYbUGFtbY3BgwfrdLyVM8gEBAQgODgYTk5OsLe3x9q1a2Fvb6+WVJOSkvDaa6/B3d1dq30rj52yUCszHk5eTPLELuVyZ1kZKWltbQUAtenBeiOXy5GbmwtBEDB//ny0tLSordf3eWhqagIArFixQvXOmSAIOHfuHJqbm7Xej5ubGwConjcqWVpawtPTE1VVVQCAw4cPo7y8HK+88orW+1ZOiaY8lsx4OHkxyROzlMvdZWWkRPmHV5eXbYOCghAfH4/KykqsW7dObZ2+z4NysERmZiaISO3n6NGjWu9nwIAB8PX1xcmTJzXWdXZ2wt7eHsAfoz+/+OILmJmZqRKlMoa0tDQIgqBRcaG9vR3An8eSGQ8nLyZ5YpZyubusjCH6MBQXFxcIgqDz+1vr1q2Dv78/Tpw4oba8ryV67vboo49CLpejrKxMp+16EhERgRMnTuDMmTOqZc3NzTh37pxq+Hxubq5GklReUScnJ4OING6JKo+dq6trn2NkuuHkxSTPmKVc7ldWpq99lJSUGG2ovI2NDYYNG4aLFy/qtJ3y9qG5ubnGcl1K9GjTz8svv4zdu3cjJycH9fX16OrqwsWLF3HlyhUAQGRkJFxdXe87PVV8fDw8PT0RFRWF8+fP4/r160hISEBLS8s9B6BoQ3nstH1/jOmRSMMc+wUeKm840HEocXd3N23evJl8fX3JwsKCHB0daebMmXTq1Cm1dtevX6dJkyaRXC4nLy8vevPNN2nZsmUEgHx8fFRD3o8fP06enp5kbW1NTz/9NF29epWio6PJwsKC3N3dSSaTkZ2dHc2YMYOqqqr01seBAwdIoVBQamqqzsfsQb6PMTExZGFhQc3Nzapl+/btI29vbwJAgwYNokWLFvW47bJlyzSGymtzHrZu3Uo2NjYEgHx9famqqoq2bdtGdnZ2BIA8PT3p999/JyKitrY2SkhIoCFDhpBMJiNnZ2eaNWsWVVRUEBHRzJkzCQCtWrXqvp/1woULNHfuXHJ0dCQrKyt64oknqKSkpNdt7jdUfurUqeTu7q4x3P5+dP1+Mw35nLz6gJOX4ZjiL3d0dDQ5OTmJHcY9Pcj3sbKykmQyGe3YscNAURlWV1cXhYSE0Pbt243ed11dHcnlctqyZYvO25ri91ti+D0vxnTR32YS9/HxQUpKClJSUnqdoNYUdXV1oaioCA0NDYiMjDR6/2vWrMHYsWMRExNj9L4ZP/Ni7KGXmJiI2bNnIzIyUlKT75aWlmLv3r0oKSnR+l01fcnIyEBZWRkOHDgACwsLo/bN/sDJizEt9PeyMmlpaYiJicH69evFDkVrkydPxq5du9TmkTSG4uJitLW1obS0FI6Ojkbtm/1JJnYAjElBeno60tPTxQ7DoEJDQxEaGip2GCYvLCwMYWFhYofx0OMrL8YYY5LDyYsxxpjkcPJijDEmOZy8GGOMSQ4nL8YYY5LDow37qLCwEIIgiB1GvxQREYGIiAixw5Ac/j6yh4FARCR2EFJ19OhRXLhwQewwmAk6evQosrKykJeXJ3YozEQFBwdLspSOiSjg5MWYAeTn5yMiIgL868WYQRTwMy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjmcvBhjjEmOTOwAGJO62tpafPzxx2rLfvzxRwDAtm3b1JYrFArMnTvXaLEx1l8JRERiB8GYlLW1tcHFxQWNjY0wNzcHACh/rQRBULXr6OjASy+9hA8++ECMMBnrTwr4tiFjfWRlZYXw8HDIZDJ0dHSgo6MDnZ2d6OzsVP27o6MDAPD888+LHC1j/QMnL8b04Pnnn0d7e3uvbRwcHPD3v//dSBEx1r9x8mJMDyZNmgRnZ+d7rrewsMALL7wAmYwfMzOmD5y8GNMDMzMzzJs3DxYWFj2u7+jo4IEajOkRJy/G9GTu3LmqZ1t3e+SRRxAUFGTkiBjrvzh5MaYnTzzxBDw9PTWWW1pa4qWXXlIbecgY6xtOXozp0Ysvvqhx67C9vZ1vGTKmZ5y8GNOjefPmadw69PHxwahRo0SKiLH+iZMXY3rk7++PESNGqG4RWlhY4OWXXxY5Ksb6H05ejOnZv/71L9VMG52dnXzLkDED4OTFmJ7NnTsXXV1dAIDAwEB4eXmJHBFj/Q8nL8b0bMiQIXjyyScBAC+99JLI0TDWP/Hr/gY2e/ZssUNgImhra4MgCPj888/xzTffiB0OM7KgoCDEx8eLHUa/xldeBlZYWIiLFy+KHYZJOXbsGI4dOyZ2GAbl4eEBV1dXyOVyvezv4sWLKCws1Mu+mGEdO3YMR48eFTuMfo+vvIwgLi4Oc+bMETsMk6G8Gi0oKBA5EsM6ffo0fHx89LKv/Px8RERE9Ptj1h/w3Rbj4CsvxgxEX4mLMaaJkxdjjDHJ4eTFGGNMcjh5McYYkxxOXowxxiSHkxeTrAMHDsDe3h6ffvqp2KGYvEOHDiExMRF79+7FsGHDIAgCBEHAiy++qNE2NDQUCoUC5ubmCAgIwPHjx0WIWDcdHR1IT0+Hj48PLC0t4eDggJEjR6K6uvqe27S2tsLf3x8rVqxQLfvkk0+wceNG1QwpzHRx8mKSRURihyAJq1evRnZ2NpKSkjBr1iycOXMG3t7eGDhwIHbu3In9+/ertf/8889RUFCAadOmoaKiAoGBgSJFrr2IiAh8+OGH2LVrF5qbm/Hrr7/C29sbjY2N99wmOTkZp06dUls2ffp0yOVyTJ48Gbdu3TJ02KwPOHkxyZo6dSpu376NadOmiR0KWlpaEBwcLHYYGjZs2IA9e/YgPz8fCoVCbV12djbMzMwQHR2N27dvixRh3+3ZswdFRUUoKCjAk08+CZlMBjc3NxQXF2PkyJE9bnPkyBH88ssvPa5bvHgxxowZgylTpqCzs9OQobM+4OTFmB5s374dNTU1Yoeh5vTp01i5ciXWrl3b40wfwcHBiI2NxaVLl7B06VIRItSPd955B4GBgVrXTGtpacGyZcuQlZV1zzZr1qxBWVlZr22YuDh5MUk6fPgwhgwZAkEQ8PbbbwMAcnJyYGtrCxsbGxQXF+PZZ5+FnZ0dPDw8sHv3btW22dnZkMvlcHFxwYIFC+Dm5ga5XI7g4GB89913qnYxMTGwtLTE4MGDVcveeOMN2NraQhAE1NXVAQBiY2OxZMkSVFVVQRAE1cvJBw8ehJ2dHdLS0oxxSDRkZ2eDiDB9+vR7tklNTYWfnx/ef/99HDp0qNf9EREyMjLw2GOPwcrKCo6OjpgxYwZ+++03VRttzwEAdHV1YdWqVRgyZAisra0xevRo5OXl6fQZ29vbcezYMYwdCE/WXAAAEwpJREFUO1brbZKTk/HGG2/A2dn5nm0cHR0xYcIEZGVl8e1pE8XJi0nS008/jSNHjqgte/311xEXF4eWlhYoFArk5eWhqqoKw4YNw6uvvqqqcBwTE4OoqCg0Nzdj8eLFqK6uxvHjx9HZ2Yl//OMfuHDhAoA//vjfPa3X1q1bsXbtWrVlWVlZmDZtGry9vUFEOH36NACoHvp3d3cb5Bjcz/79+zF8+HDY2Njcs421tTU++OADmJmZ4dVXX0VTU9M9265ZswaJiYlITk5GTU0NvvnmG1y4cAEhISG4du0aAO3PAQAsX74cmzZtQmZmJq5cuYJp06bh+eefx48//qj1Z7x8+TLa29vxv//9D5MmTVL9R+Sxxx7D1q1bNRLPf//7X1RVVeH555+/777/8pe/4NKlS/jpp5+0jocZDycv1i8FBwfDzs4Ozs7OiIyMRFNTE86fP6/WRiaTqa4iRowYgZycHDQ0NCA3N1cvMUydOhX19fVYuXKlXvani6amJpw9exbe3t73bRsUFIS4uDhUV1dj+fLlPbZpaWlBRkYGnnvuObzwwguwt7fHqFGj8O6776Kurg7btm3T2Ka3c9Da2oqcnBzMnDkTs2bNgoODA1asWAELCwudjr9yQIazszPS0tJQUVGBa9euYcaMGVi0aBE++ugjtc8QGxuLnJwcrfbt6+sLACgvL9c6HmY8nLxYv2dpaQkAav/r78njjz8OGxsbtdtgUlVTUwMi6vWq606pqakYPnw4tm7disOHD2usr6ioQGNjIx5//HG15ePGjYOlpaXa7dae3H0OTp06hebmZrUBFdbW1hg8eLBOx9/KygoAEBAQgODgYDg5OcHe3h5r166Fvb29WlJNSkrCa6+9Bnd3d632rTx2yqtKZlo4eTF2BysrK9TW1oodRp+1trYC+POP+/3I5XLk5uZCEATMnz8fLS0tauuVw8YHDBigsa2DgwMaGhp0ik95e3LFihWqd84EQcC5c+fQ3Nys9X7c3NwAQPX8UcnS0hKenp6oqqoC8Mcz0vLycrzyyita79va2hrAn8eSmRZOXoz9v46ODty6dQseHh5ih9Jnyj+8urxsqyygWFlZiXXr1qmtc3BwAIAek9SDHDPlYInMzEwQkdqPLrWwBgwYAF9fX5w8eVJjXWdnJ+zt7QH8MRr0iy++gJmZmSpRKmNIS0uDIAgaz9ra29sB/HksmWnh5MXY/ystLQURYfz48aplMpnsvrcbTZGLiwsEQdD5/a1169bB398fJ06cUFs+cuRIDBgwQOMP/HfffYf29nb89a9/1amfRx99FHK5HGVlZTpt15OIiAicOHECZ86cUS1rbm7GuXPnVMPnc3NzNZKk8go7OTkZRKRxS1R57FxdXfscI9M/Tl7sodXd3Y2bN2+is7MTP//8M2JjYzFkyBBERUWp2vj4+ODGjRsoKipCR0cHamtrce7cOY19OTk54fLly6iurkZDQwM6OjpQUlIi2lB5GxsbDBs2TOcq3srbh+bm5hrLlyxZgn379mHnzp2or69HeXk5Fi5cCDc3N0RHR+vcz8svv4zdu3cjJycH9fX16OrqwsWLF3HlyhUAQGRkJFxdXe87PVV8fDw8PT0RFRWF8+fP4/r160hISEBLS8s9B6BoQ3nstH1/jBkXJy8mSW+//TbGjRsHAEhISEBYWBhycnKQmZkJABg9ejTOnDmD9957D0uWLAEAPPPMM6isrFTto7W1FaNGjYK1tTVCQkLg5+eHr776Su050euvv45JkyZh7ty5GD58ONatW6e6jRQUFKQaVr9w4UK4uLhgxIgRmDJlCm7cuGGU49CbqVOnoqKiQu351ccffwwfHx9UVVVh3LhxePPNNzW2Gz9+POLj4zWWr169Gunp6UhJScGgQYMwYcIEDB06FKWlpbC1tQUAnc5BVlYW4uLisHHjRgwcOBBubm6IjY3FzZs3Afxx266mpgbFxcW9fk5HR0d8++238PDwwNixY+Hu7o7vv/8e+/fv1+n9r7v98MMPcHd3x+jRox94H8yAiBkUAMrLyxM7DJMSHh5O4eHhosYQHR1NTk5Oosagi7y8PNL117WyspJkMhnt2LHDQFEZVldXF4WEhND27duN3nddXR3J5XLasmWLztuawvf7IZDPV17sodXfZw738fFBSkoKUlJSep2g1hR1dXWhqKgIDQ0NiIyMNHr/a9aswdixYxETE2P0vpl2OHkx1o8lJiZi9uzZiIyMlNTku6Wlpdi7dy9KSkq0fldNXzIyMlBWVoYDBw7AwsLCqH0z7XHyMiF311pS/lhaWsLFxQUTJ07E5s2bVc8E2INJSkpCbm4ubt++DS8vLxQWFoodkkGlpaUhJiYG69evFzsUrU2ePBm7du1Sm1fSGIqLi9HW1obS0lI4OjoatW+mG05eJuTOWkv29vYgInR3d6Ompgb5+fnw8vJCQkICAgICdJr/jalLT09HW1sbiAhnz55FeHi42CEZXGhoKDZs2CB2GCYvLCwMiYmJGqMtmenh5GXiBEGAg4MDJk6ciNzcXOTn5+PatWuqWlaMMfYw4uQlMeHh4YiKikJNTQ3effddscNhjDFRcPKSIOVLtCUlJaplvdVG0qXG0tdff40nnngCNjY2sLOzw6hRo1BfX3/fPhhjzJg4eUmQ8sXLO6fD6a02krY1lpqamjB9+nSEh4fjxo0bqKyshJ+fn2qON33UX2KMMX3g5CVBCoUCgiCoJknVpTZSbzWWqqurUV9fj4CAAMjlcri6umLv3r0YNGiQ3uovMcaYPsjEDoDprqmpCUQEOzs7AA9eG+nuGkvDhg2Di4sLXnjhBSxevBhRUVEYOnRon/q4l8LCQgiCoPN2Dzs+ZtLwMIxgFRsnLwn6/fffAQD+/v4A1GsjrVixQq2tst6RNqytrfHll19i+fLlSEtLQ0pKCubMmYPc3Fy99aE0fvx4xMXF6bzdw+ro0aPIysriZ4wSoJzbkRkWJy8JOnjwIADg2WefBaBeGyk2NrZP+w4ICMCnn36K2tpaZGRkYMOGDQgICFBN0aOPPgDAw8MDc+bM6fN+HiZZWVl8zCSgoKBA7BAeCvzMS2KuXr2KzMxMeHh4YP78+QD0Vxvp8uXLqqJ+zs7OWL9+PQIDA3Hy5Em91l9ijLG+4uRloogIjY2N6O7uVhXOy8vLw1NPPQVzc3MUFRWpnnlpUxtJG5cvX8aCBQvw22+/ob29HSdOnMC5c+cwfvx4vfXBGGP6wMnLhHz66acYM2YMrly5gtbWVtjb28Pc3Bzm5ubw8/NDRkYGoqKiUFFRoVG5trfaSNrWWHJ2dkZXVxeCg4NhY2ODf/7zn1iwYAEWLVp03z4YY8yYBCIisYPozwRBQF5eHj+ruMPs2bMB8LMBXeTn5yMiIgL862r6+PttFAV85cUYY0xyOHkx9hA4dOgQEhMTNcruvPjiixptQ0NDoVAoYG5ujoCAABw/flyEiHXX3d2NzMxMBAcH97g+JSUFI0aMgJ2dHaysrODj44O33npLrVDnJ598go0bN/b7QqX9AScvxvq51atXIzs7G0lJSWpldwYOHIidO3di//79au0///xzFBQUYNq0aaioqEBgYKBIkWuvsrISf/vb3xAfH4/m5uYe23z55ZdYtGgRqqurUVdXh/T0dGRlZalu8wHA9OnTIZfLMXnyZNy6dctY4bMHwMmLPZRaWlru+T90KfVxPxs2bMCePXuQn58PhUKhti47OxtmZmaIjo6WdHmdn376CcuXL8fChQtV8372ZMCAAYiOjoaTkxMUCgXmzJmDmTNn4uDBg7hw4YKq3eLFizFmzBhMmTIFnZ2dxvgI7AFw8mIPpe3bt6OmpkbyffTm9OnTWLlyJdauXQu5XK6xPjg4GLGxsbh06RKWLl0qQoT6MWbMGOzduxfz5s2DlZXVPdt99tlnGkUmBw0aBAAaV2tr1qxBWVkZsrKy9B8w0wtOXkwSiAgZGRl47LHHYGVlBUdHR8yYMUNtXsWYmBhYWlqqlY5/4403YGtrC0EQUFdXBwCIjY3FkiVLUFVVBUEQ4OPjg+zsbMjlcri4uGDBggVwc3ODXC5HcHAwvvvuO730AfwxO4qdnR3S0tIMeryAP66siAjTp0+/Z5vU1FT4+fnh/fffx6FDh3rdnzbnQJfyO6ZQYufSpUuwtraGl5eX2nJHR0dMmDABWVlZPMLTVBEzKACUl5cndhgmJTw8nMLDw3XaZtWqVWRpaUk7duygW7du0c8//0yBgYE0aNAgunr1qqrdvHnzyNXVVW3bzZs3EwCqra1VLZs1axZ5e3urtYuOjiZbW1s6efIktba2UkVFBY0bN44UCgWdP39eL3189tlnpFAoKCUlRafPn5eXR7r+ug4bNoxGjBjR4zpvb286e/YsEREdOXKEzMzMaOjQodTY2EhERCUlJRQWFqa2jbbnIDk5mQDQF198Qbdv36aamhoKCQkhW1tbam9vV7VbunQpWVlZUWFhId28eZOSkpLIzMyMfvjhB50+552efPJJGjNmjFZtm5qaSKFQUExMTI/rExMTCQCdOHFCpxge5PvNdJbPV17M5LW0tCAjIwPPPfccXnjhBdjb22PUqFF49913UVdXh23btumtL5lMprqyGDFiBHJyctDQ0KC3si9Tp05FfX09Vq5cqZf93UtTUxPOnj0Lb2/v+7YNCgpCXFwcqqursXz58h7bPMg56K38jimU2ElPT4ebmxtSU1N7XO/r6wsAKC8vN0o8TDecvJjJq6ioQGNjIx5//HG15ePGjYOlpaXabT19e/zxx2FjY/NAZV/EVFNTAyKCjY2NVu1TU1MxfPhwbN26FYcPH9ZY39dzcHf5HX2X2NHVvn37kJ+fj//85z8aA1mUlMfu2rVrBo+H6Y6TFzN5yiHLAwYM0Fjn4OCgKsppKFZWVqitrTVoH/rW2toKAL0OYLiTXC5Hbm4uBEHA/Pnz0dLSorZe3+fgzhI7ynfOBEHAuXPn7jnUXV/27NmDDRs2oLS0VFWvrifW1tYA/jyWzLRw8mImz8HBAQB6/AN569YteHh4GKzvjo4Og/dhCMo/vLq8bBsUFIT4+HhUVlZi3bp1auv0fQ7uLONDRGo/R48e1Wlfuvj3v/+NnTt34ssvv8QjjzzSa9v29nYAfx5LZlo4eTGTN3LkSAwYMAA//vij2vLvvvsO7e3tapMUy2Qy1a0pfSgtLQURYfz48QbrwxBcXFwgCILO72+tW7cO/v7+OHHihNpyXc6BNoxdYoeIkJCQgPLychQVFfV4BXk35bFzdXU1dHjsAXDyYiZPLpdjyZIl2LdvH3bu3In6+nqUl5dj4cKFcHNzQ3R0tKqtj48Pbty4gaKiInR0dKC2thbnzp3T2KeTkxMuX76M6upqNDQ0qJJRd3c3bt68ic7OTvz888+IjY3FkCFDEBUVpZc+SkpKjDJU3sbGBsOGDcPFixd12k55+/Du96F0OQfa9nO/EjuRkZFwdXXVy/RUJ0+exKZNm/Dee+/BwsJC7ValIAjYsmWLxjbKYzdq1Kg+988MQMShjg8F8FB5DQ8ylLi7u5s2b95Mvr6+ZGFhQY6OjjRz5kw6deqUWrvr16/TpEmTSC6Xk5eXF7355pu0bNkyAkA+Pj6qIe/Hjx8nT09Psra2pqeffpquXr1K0dHRZGFhQe7u7iSTycjOzo5mzJhBVVVVeuvjwIEDpFAoKDU1VafP/yBD5WNiYsjCwoKam5tVy/bt20fe3t4EgAYNGkSLFi3qcdtly5ZpDJXX5hxs3bqVbGxsCAD5+vpSVVUVbdu2jezs7AgAeXp60u+//05ERG1tbZSQkEBDhgwhmUxGzs7ONGvWLKqoqCAiopkzZxIAWrVqVa+f8+jRo/TUU0+Rm5sbASAANHjwYAoODqavv/6aiIjKy8tV63r62bx5s8Z+p06dSu7u7tTd3a3F0f4TD5U3inxOXgbGyUuTqf5yR0dHk5OTk9hh9OhBkldlZSXJZDLasWOHgaIyrK6uLgoJCaHt27cbve+6ujqSy+W0ZcsWnbc11e93P8PveTF2p/40m7iPjw9SUlKQkpKiNnO6FHR1daGoqAgNDQ2IjIw0ev9r1qzB2LFjERMTY/S+mXY4eTHWjyUmJmL27NmIjIyU1OS7paWl2Lt3L0pKSrR+V01fMjIyUFZWhgMHDsDCwsKofTPtcfJiDEBSUhJyc3Nx+/ZteHl5obCwUOyQ9CYtLQ0xMTFYv3692KFobfLkydi1a5faHJLGUFxcjLa2NpSWlsLR0dGofTPdyMQOgDFTkJ6ejvT0dLHDMJjQ0FCEhoaKHYbJCwsLQ1hYmNhhMC3wlRdjjDHJ4eTFGGNMcjh5McYYkxxOXowxxiSHB2wYgSEnGpUi5bQ7+fn5IkciHcrvEB8z03fx4kXJTeQsRQIR17g2JEEQxA6BMWZk4eHhKCgoEDuM/qyAr7wMjP9vwBhj+sfPvBhjjEkOJy/GGGOSw8mLMcaY5HDyYowxJjn/B2aO+HHEypZ3AAAAAElFTkSuQmCC\n", 808 | "text/plain": [ 809 | "" 810 | ] 811 | }, 812 | "metadata": {}, 813 | "execution_count": 27 814 | } 815 | ] 816 | }, 817 | { 818 | "cell_type": "code", 819 | "metadata": { 820 | "id": "j4PODlF4E0_d" 821 | }, 822 | "source": [], 823 | "execution_count": null, 824 | "outputs": [] 825 | } 826 | ] 827 | } -------------------------------------------------------------------------------- /CIC IDS 2018/LSTM_BC/keras_metadata.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/LSTM_BC/keras_metadata.pb -------------------------------------------------------------------------------- /CIC IDS 2018/LSTM_BC/saved_model.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/LSTM_BC/saved_model.pb -------------------------------------------------------------------------------- /CIC IDS 2018/LSTM_MC_L2/keras_metadata.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/LSTM_MC_L2/keras_metadata.pb -------------------------------------------------------------------------------- /CIC IDS 2018/LSTM_MC_L2/saved_model.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/CIC IDS 2018/LSTM_MC_L2/saved_model.pb -------------------------------------------------------------------------------- /CIC IDS 2018/README.md: -------------------------------------------------------------------------------- 1 | ## 2. CIC IDS 2018: The successor of the CIC IDS 2017 [Refer](https://www.unb.ca/cic/datasets/ids-2018.html) 2 |
[CICIDS2017 dataset](https://www.unb.ca/cic/datasets/ids-2017.html) contains benign and the most up-to-date common attacks, which resembles the true real-world data (PCAPs). It also includes the results of the network traffic analysis using CICFlowMeter with labeled flows based on the time stamp, source, and destination IPs, source and destination ports, protocols and attack (CSV files).
3 |
In CSE-CIC-IDS2018 dataset, we use the notion of profiles to generate datasets in a systematic manner, which will contain detailed descriptions of intrusions and abstract distribution models for applications, protocols, or lower level network entities. These profiles can be used by agents or human operators to generate events on the network. Due to the abstract nature of the generated profiles, we can apply them to a diverse range of network protocols with different topologies. Profiles can be used together to generate a dataset for specific needs.
4 | * B-profiles: Encapsulate the entity behaviours of users using various machine learning and statistical analysis techniques (such as K-Means, Random Forest, SVM, and J48). The encapsulated features are distributions of packet sizes of a protocol, number of packets per flow, certain patterns in the payload, size of payload, and request time distribution of a protocol. The following protocols will be simulated in our testbed environment: HTTPS, HTTP, SMTP, POP3, IMAP, SSH, and FTP. Based on our initial observations majority of traffic is HTTP and HTTPS. 5 | 6 | * M-Profiles: Attempt to describe an attack scenario in an unambiguous manner. In the simplest case, humans can interpret these profiles and subsequently carry them out. Idealistically, autonomous agents along with compilers would be employed to interpret and execute these scenarios. 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Sleety Matt George 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /NSL-KDD/Best_CNN/checkpoint.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/Best_CNN/checkpoint.hdf5 -------------------------------------------------------------------------------- /NSL-KDD/Best_NN/checkpoint.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/Best_NN/checkpoint.hdf5 -------------------------------------------------------------------------------- /NSL-KDD/README.md: -------------------------------------------------------------------------------- 1 | ## 1. NSL KDD: The enhanced KDD Cup 99 [Refer](https://www.unb.ca/cic/datasets/nsl.html) 2 |
[The KDD Cup 99 dataset](https://www.tensorflow.org/datasets/catalog/kddcup99) contains a standard set of data to be audited, which includes a wide variety of intrusions simulated in a military network environment. 3 | The competition task was to build a network intrusion detector, a predictive model capable of distinguishing between ''bad’’ connections, called intrusions or attacks, and “good” normal connections. 4 | ### Improvements to the KDD'99 dataset 5 | The NSL-KDD data set has the following advantages over the original KDD data set: 6 | 7 | * It does not include redundant records in the train set, so the classifiers will not be biased towards more frequent records. 8 | * There is no duplicate records in the proposed test sets; therefore, the performance of the learners are not biased by the methods which have better detection rates on the frequent records. 9 | * The number of selected records from each difficultylevel group is inversely proportional to the percentage of records in the original KDD data set. As a result, the classification rates of distinct machine learning methods vary in a wider range, which makes it more efficient to have an accurate evaluation of different learning techniques. 10 | * The number of records in the train and test sets are reasonable, which makes it affordable to run the experiments on the complete set without the need to randomly select a small portion. Consequently, evaluation results of different research works will be consistent and comparable. 11 | -------------------------------------------------------------------------------- /NSL-KDD/img/FNN-arch.drawio: -------------------------------------------------------------------------------- 1 | 7Vldb5swFP01PK4KNibJY5ukXadUmxp1S/oyueEWmAAjxySwXz8TTIC4+aimFjb1JfE9vra5x/dc82HgUZjecBp7d8yBwEA9JzXw2EDIROZQ/uVIViBkiAvA5b6jnCpg5v8GBfYUmvgOrBqOgrFA+HETXLIogqVoYJRztmm6PbOguWpMXdCA2ZIGOvrDd4RXoAPUr/DP4LteubJpq4BDWjqrSFYeddimBuGJgUecMVG0wnQEQU5eyUsx7vpA7+7COETinAGrKFuTye3TT/9xbvYGIfTxw6eBujaRlQGDI+NXJuPCYy6LaDCp0CvOksiBfNaetCqfKWOxBE0J/gIhMrWZNBFMQp4IA9UrL5hn83z8BSnNhZpua4zThpUpS49YkbBiCV/CkTDLzKHcBXHEDxV+OQe1BRSfN8BCkNcjHTgEVPjrZo5QlWruzq/aDdlQG/KKzVHzrmmQqJVuozgREprSDLiB7EAGc/WUt9y8RSzZZ+BL+Zs393a2uW8bzxcwi+mWt40Ub3OPDnK9Bi4gPcqO6rVU4melopW9qXRkluLwahqye2/Fp87Je2R76ot5rb2oEl9aVarnRlbP+3ndqI16E4UgXSGJdXk/R0+P6EsfvqeLSXZ/d13mZEcUgjSFjCFawVkKQcTW++9h+tA14ezOwvaE09M4+RCO4gafebRYnRIO1oXDWczyw2VfElud1DXTsjrIsHPyMDVSPuTRTPuT8iCdkof12nNFE0knD5Y+6Zx0UJvSqeSyqPWcks57PbSQM6Vjd0o65N89WYZm1+Rha2R+TcTRR8A6p0TvnrFnEdK0c1Rj1DbVJvk4xA9yc+5NLu5UKTJfuMv9H45xC7VYp469FWip6GvMvcDvQTIR7hqZrd4SvV8h2isSJyvTscQ7WZiKUvAXhUgN/cb8SFTJg+295MHDC9KcpAhBjUO11/N7U1n7eahPVUSpTbVNsl1ML+WdNKvvDIV79bUGT/4A -------------------------------------------------------------------------------- /NSL-KDD/img/FNN-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/FNN-arch.png -------------------------------------------------------------------------------- /NSL-KDD/img/acc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/acc.png -------------------------------------------------------------------------------- /NSL-KDD/img/acc_CNN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/acc_CNN.png -------------------------------------------------------------------------------- /NSL-KDD/img/acc_nn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/acc_nn.png -------------------------------------------------------------------------------- /NSL-KDD/img/arch_CNN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/arch_CNN.png -------------------------------------------------------------------------------- /NSL-KDD/img/attacks_table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/attacks_table.png -------------------------------------------------------------------------------- /NSL-KDD/img/cm_CNN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/cm_CNN.png -------------------------------------------------------------------------------- /NSL-KDD/img/cm_NN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/cm_NN.png -------------------------------------------------------------------------------- /NSL-KDD/img/cm_RF.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/cm_RF.png -------------------------------------------------------------------------------- /NSL-KDD/img/flags.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/flags.png -------------------------------------------------------------------------------- /NSL-KDD/img/flags_hist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/flags_hist.png -------------------------------------------------------------------------------- /NSL-KDD/img/fpr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/fpr.png -------------------------------------------------------------------------------- /NSL-KDD/img/macro_category.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/macro_category.png -------------------------------------------------------------------------------- /NSL-KDD/img/services.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/services.png -------------------------------------------------------------------------------- /NSL-KDD/img/services_hist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/services_hist.png -------------------------------------------------------------------------------- /NSL-KDD/img/tpr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/img/tpr.png -------------------------------------------------------------------------------- /NSL-KDD/resources/column_info.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/resources/column_info.pdf -------------------------------------------------------------------------------- /NSL-KDD/resources/flag_info.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/resources/flag_info.jpg -------------------------------------------------------------------------------- /NSL-KDD/resources/lib_info.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sleetymattgeorge/Deep-Learning-Evaluation-of-IDS-Datasets/3a3c3083c17865874c41d127d1dc1984dd57c759/NSL-KDD/resources/lib_info.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | > **Status:** :heavy_check_mark: 2 | # Deep-Learning-Evaluation-of-IDS-Datasets 3 | Deep Model Intrusion Detection Evaluation of NSL KDD and CIC IDS 2018 datasets. 4 | 5 | # Usage 6 | - install jupyter-notebook. 7 | ```bash 8 | pip install jupyter-notebook 9 | ``` 10 | - Download the .ipynb files via `git clone` or download `zip`. 11 | ```bash 12 | git clone https://github.com/WhiteHatCyberus/Deep-Learning-Evaluation-of-IDS-Datasets.git 13 | ``` 14 | - Now run the `jupyter-notebook` command in the directory where the extracted zip file is found. 15 | - All componenets of the jupyter-notebooks are all found in the folder. 16 | 17 | ## Credits 18 | Every developer deserves credit for the work and time they put in. 19 | 20 | Thank you [Joule Effect](https://github.com/jouleffect) for your contribution in the evaluation of Deep Learning models on NSL-KDD IDS dataset. 21 | --------------------------------------------------------------------------------