├── README.md ├── LSTM_inDeep_binary_class.ipynb ├── MLP_inDeep_binary_class.ipynb ├── InDeep_binary_class.ipynb ├── inDeep_Feature_study.ipynb └── RFE_feature_selection.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # CNN_UNSW-NB15 2 | DISEÑO Y EVALUACIÓN DE REDES NEURONALES CONVOLUCIONALES PARA UN SISTEMA DE DETECCIÓN DE INTRUSIONES. 3 | ## Descripción 🚀 4 | Año a año, las comunicaciones entre equipos, personas, corporaciones, gobiernos o todo aquello que use cualquier mecanismo moderno para comunicarse con otra entidad ha crecido de manera exponencial. Vivir comunicados en un mundo globalizado es indispensable. Debido a ello, cada vez es más notorio la importancia de salvaguardar la seguridad de las comunicaciones frente al continuo crecimiento de los ataques cibernéticos. Estos ataques, día a día mejoran, se fortalecen y se hacen más difícilmente detectables frente a los equipos de ciberseguridad. Entre esos equipos, el IDS es materia de nuestro interés, siendo una herramienta fundamental para la detección de intrusiones en la red. Dicho equipo, a pesar de sus diversos beneficios, presenta ciertas carencias en algunos aspectos, entre la que destaca el alto ratio de falsos positivos que genera. Por ello, se vio necesario la utilización de otros mecanismos para paliar dicha carencia. Como resultado, este estudio pretende utilizar como IDS una red neuronal convolucional que mejore el resultado de los falsos positivos que vienen arrastrando los IDS tradicionales. Además, se evalúa nuestra red con otras dos redes neuronales: una red MLP y otra LSTM. Adicionalmente, se usan diferentes técnicas para procesar el set de datos que utilizamos (UNSW-NB15) para entrenar la red neuronal convolucional. Los resultados de este estudio reflejan una mejora notable en la tasa de falsos positivos con respecto a otras redes y estudios realizados, una mejora en la clasificación multiclase y una buena eficiencia de entrenamiento por parte de la red. 5 | 6 | ### Pre-requisitos 📋 7 | 8 | Tener descargado en el Google Drive los ficheros: 9 | 10 | * [UNSW_NB15_training-set.csv](https://www.unsw.adfa.edu.au/unsw-canberra-cyber/cybersecurity/ADFA-NB15-Datasets/) 11 | * [UNSW_NB15_testing-set.csv](https://www.unsw.adfa.edu.au/unsw-canberra-cyber/cybersecurity/ADFA-NB15-Datasets/) 12 | 13 | ### Instalación 🔧 14 | 15 | _Se requiere vincular la cuenta de Google Drive a Google Colab_ 16 | 17 | ``` 18 | from google.colab import drive 19 | drive.mount('/content/drive') 20 | ``` 21 | 22 | ## Construido con 🛠️ 23 | 24 | * [Google Colab](https://colab.research.google.com/) - Entorno de ejecución y programación en Python en el navegador. 25 | 26 | 27 | 28 | -------------------------------------------------------------------------------- /LSTM_inDeep_binary_class.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"LSTM_inDeep_binary_class.ipynb","provenance":[],"collapsed_sections":[],"authorship_tag":"ABX9TyMzr38kJvukSYgEd4hqWT2I"},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"code","metadata":{"id":"oHUbEfIrroJU","colab_type":"code","colab":{}},"source":["#Importamos la segunda versión de tensorflow\n","%tensorflow_version 2.x"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"OniWnG5-rui7","colab_type":"code","colab":{}},"source":["#Instalamos las dependencias necesarias que no posee Google Colab.\n","!pip install bayesian-optimization\n","!pip install mlxtend --upgrade --no-deps"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"WXLBhGDEruk9","colab_type":"code","colab":{}},"source":["#Importamos los paquetes necesarios para el proyecto\n","import pandas as pd\n","import numpy as np\n","import matplotlib.pyplot as plt\n","from tensorflow.keras.utils import get_file\n","from sklearn import preprocessing\n","from sklearn.preprocessing import LabelEncoder\n","from PIL import Image\n","from sklearn.model_selection import StratifiedShuffleSplit\n","from sklearn import metrics\n","from keras import backend as K\n","from keras.utils.generic_utils import get_custom_objects\n","from bayes_opt import BayesianOptimization\n","from sklearn.model_selection import StratifiedKFold\n","from mlxtend.plotting import plot_confusion_matrix\n","from sklearn.model_selection import train_test_split\n","from imblearn.over_sampling import SMOTE\n","\n","\n","from tensorflow.keras.models import Sequential,Model\n","from tensorflow.keras.layers import Dense, Conv2D,LSTM, MaxPooling2D,UpSampling2D,Conv2DTranspose, Dropout, Flatten, Activation, LeakyReLU, ReLU, Input,concatenate,BatchNormalization\n","from tensorflow.keras.optimizers import Adam\n","from keras.regularizers import l1\n","from tensorflow.keras.callbacks import EarlyStopping"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"h2k63oqrrum_","colab_type":"code","colab":{}},"source":["#Descargamos el set de datos subido previamente al Google Drive (Tarda menos que descargarlo directamente de la página).\n","path_train = \"/content/drive/My Drive/TFG/UNSW_NB15_training-set.csv\"\n","path_test = \"/content/drive/My Drive/TFG/UNSW_NB15_testing-set.csv\"\n","\n","#Leemos los datos\n","df_train=pd.read_csv(path_train,dtype='unicode')\n","df_test=pd.read_csv(path_test,dtype='unicode')\n","\n","#Quitamos las columnas no necesarias\n","df_train.drop('id', axis=1, inplace=True)\n","df_train.drop('attack_cat', axis=1, inplace=True)\n","df_test.drop('id', axis=1, inplace=True)\n","df_test.drop('attack_cat', axis=1, inplace=True)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"WBJfZogHruo9","colab_type":"code","colab":{}},"source":["#Concatenamos el set de entrenamiento y el de test para manipular directamente el conjunto.\n","df = pd.concat([df_train,df_test],axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"oqwvw6OEruqw","colab_type":"code","colab":{}},"source":["#División del dataset por clases.\n","df_class_Normal = df[df['label'] == '0']\n","df_class_Attack = df[df['label'] == '1']\n","\n","#Under-sampling de la categoría attack hasta alcanzar el mismo valor que la categoria Normal\n","df_class_Attack = df_class_Attack.sample(df_class_Normal.shape[0])\n","\n","#Concatenado\n","df = pd.concat([df_class_Normal,df_class_Attack], axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"HbArosMZrutK","colab_type":"code","colab":{}},"source":["#Visualización en una gráfica circular del peso de cada clase\n","print(\"Shape of dataFrame: {} \\n\".format(df.shape))\n","print(\"Number of attack samples\")\n","display(df['label'].value_counts())\n","print(\"\")\n","print(\"Plotting balance of dataFrame\")\n","df_plot = (df['label'].value_counts(normalize=True) *100)\n","df_plot.plot(kind='pie',figsize=(10,10),title='Balance of dataset (%)')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"9VTk_5g7ruwh","colab_type":"code","colab":{}},"source":["#Conversión de los datos tipo string a enteros en un rango entre 0 y 255 (1 Byte de información)\n","def encode_string_byte (df,name):\n"," df[name] = LabelEncoder().fit_transform(df[name])\n","\n","encode_string_byte (df,'proto')\n","encode_string_byte (df,'state') \n","encode_string_byte (df,'service') "],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"SetOj5kjru0J","colab_type":"code","colab":{}},"source":["#Normalización de los números enteros en valores decimales en rango entre 0 y 1\n","def numerical_minmax_normalization (df, name):\n"," x = df[name].values.reshape(-1,1)\n"," min_max_scaler = preprocessing.MinMaxScaler()\n"," x_scaled = min_max_scaler.fit_transform(x)\n"," df[name] = x_scaled\n","\n","numerical_minmax_normalization(df,'dur')\n","numerical_minmax_normalization(df,'spkts')\n","numerical_minmax_normalization(df,'dpkts')\n","numerical_minmax_normalization(df,'sbytes')\n","numerical_minmax_normalization(df,'dbytes')\n","numerical_minmax_normalization(df,'rate')\n","numerical_minmax_normalization(df,'sttl')\n","numerical_minmax_normalization(df,'dttl')\n","numerical_minmax_normalization(df,'sload')\n","numerical_minmax_normalization(df,'dload')\n","numerical_minmax_normalization(df,'sloss')\n","numerical_minmax_normalization(df,'dloss')\n","numerical_minmax_normalization(df,'sinpkt')\n","numerical_minmax_normalization(df,'dinpkt')\n","numerical_minmax_normalization(df,'sjit')\n","numerical_minmax_normalization(df,'djit')\n","numerical_minmax_normalization(df,'swin')\n","numerical_minmax_normalization(df,'stcpb')\n","numerical_minmax_normalization(df,'dtcpb')\n","numerical_minmax_normalization(df,'dwin')\n","numerical_minmax_normalization(df,'tcprtt')\n","numerical_minmax_normalization(df,'synack')\n","numerical_minmax_normalization(df,'ackdat')\n","numerical_minmax_normalization(df,'smean')\n","numerical_minmax_normalization(df,'dmean')\n","numerical_minmax_normalization(df,'trans_depth')\n","numerical_minmax_normalization(df,'response_body_len')\n","numerical_minmax_normalization(df,'ct_srv_src')\n","numerical_minmax_normalization(df,'ct_state_ttl')\n","numerical_minmax_normalization(df,'ct_dst_ltm')\n","numerical_minmax_normalization(df,'ct_src_dport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_sport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_src_ltm')\n","numerical_minmax_normalization(df,'is_ftp_login')\n","numerical_minmax_normalization(df,'ct_ftp_cmd')\n","numerical_minmax_normalization(df,'ct_flw_http_mthd')\n","numerical_minmax_normalization(df,'ct_src_ltm')\n","numerical_minmax_normalization(df,'ct_srv_dst')\n","numerical_minmax_normalization(df,'is_sm_ips_ports')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"6ScZjFeAru2E","colab_type":"code","colab":{}},"source":["#Mapeo de los valores normalizados del paso anterior a valores enteros entre 0 y 255 (1 Byte de información)\n","def numerical_split_ohe (df,name):\n"," pd_to_np = df[name].tolist()\n"," np_split = []\n"," \n"," categories = np.linspace(0, 1, num=256,endpoint=False)\n"," quantization = range(0,256)\n","\n"," for value in pd_to_np:\n"," for i in range(len(categories)-1):\n"," if (categories[i] <= float(value) <= categories[i+1]):\n"," np_split.append(quantization[i])\n"," break\n"," if (float(value) > categories[-1]):\n"," np_split.append(quantization[-1])\n"," break\n"," \n"," df[name] = np_split\n","\n","\n","numerical_split_ohe(df,'dur')\n","numerical_split_ohe(df,'spkts')\n","numerical_split_ohe(df,'dpkts')\n","numerical_split_ohe(df,'sbytes')\n","numerical_split_ohe(df,'dbytes')\n","numerical_split_ohe(df,'rate')\n","numerical_split_ohe(df,'sttl')\n","numerical_split_ohe(df,'dttl')\n","numerical_split_ohe(df,'sload')\n","numerical_split_ohe(df,'dload')\n","numerical_split_ohe(df,'sloss')\n","numerical_split_ohe(df,'dloss')\n","numerical_split_ohe(df,'sinpkt')\n","numerical_split_ohe(df,'dinpkt')\n","numerical_split_ohe(df,'sjit')\n","numerical_split_ohe(df,'djit')\n","numerical_split_ohe(df,'swin')\n","numerical_split_ohe(df,'stcpb')\n","numerical_split_ohe(df,'dtcpb')\n","numerical_split_ohe(df,'dwin')\n","numerical_split_ohe(df,'tcprtt')\n","numerical_split_ohe(df,'synack')\n","numerical_split_ohe(df,'ackdat')\n","numerical_split_ohe(df,'smean')\n","numerical_split_ohe(df,'dmean')\n","numerical_split_ohe(df,'trans_depth')\n","numerical_split_ohe(df,'response_body_len')\n","numerical_split_ohe(df,'ct_srv_src')\n","numerical_split_ohe(df,'ct_state_ttl')\n","numerical_split_ohe(df,'ct_dst_ltm')\n","numerical_split_ohe(df,'ct_src_dport_ltm')\n","numerical_split_ohe(df,'ct_dst_sport_ltm')\n","numerical_split_ohe(df,'ct_dst_src_ltm')\n","numerical_split_ohe(df,'is_ftp_login')\n","numerical_split_ohe(df,'ct_ftp_cmd')\n","numerical_split_ohe(df,'ct_flw_http_mthd')\n","numerical_split_ohe(df,'ct_src_ltm')\n","numerical_split_ohe(df,'ct_srv_dst')\n","numerical_split_ohe(df,'is_sm_ips_ports')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"SC83D1aGru4M","colab_type":"code","colab":{}},"source":["#Quitando la columna attack_cat y guardandola en la variable y.\n","y_column = df['label']\n","df.drop('label',axis=1,inplace=True)\n","dummies = pd.get_dummies(y_column) \n","y = dummies.values"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"MQMi0pQmru6l","colab_type":"code","colab":{}},"source":["#Normalización de los valores entre -0,5 y 0,5\n","x = []\n","for image in np.array(df.to_numpy()):\n"," x.append((image/255 - 0.5))\n","x = np.array(x)\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"XxIke5Xpru83","colab_type":"code","colab":{}},"source":["#Separación del dataset en un set de entrenamiento y otro de validación.\n","sss = StratifiedShuffleSplit(n_splits=1, test_size=0.25, random_state=42)\n","\n","for train_index, test_index in sss.split(x,y):\n"," x_train, x_test = x[train_index], x[test_index]\n"," y_train, y_test = y[train_index], y[test_index]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"k4J7O6pmru_J","colab_type":"code","colab":{}},"source":["#Definición del modelo final.\n","def LSTM_model():\n","\n"," input_img = Input(shape = (None, 1))\n"," output = LSTM(128,activation='relu',activity_regularizer=l1(1e-5))(input_img)\n"," output = Dense(128, activation='relu',activity_regularizer=l1(1e-5))(output)\n"," output = Dense(64, activation='relu',activity_regularizer=l1(1e-5))(output)\n"," out = Dense(10, activation='sigmoid')(output)\n","\n"," model = Model(inputs = input_img, outputs = out)\n"," model.compile(optimizer='adam', loss=\"binary_crossentropy\", metrics=[\"accuracy\"])\n","\n"," return model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"AZJ4hQ4BrvBn","colab_type":"code","colab":{}},"source":["#Proceso de entrenamiento\n","model = LSTM_model()\n","es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, min_delta=0.01, patience=10, restore_best_weights=True)\n","history = model.fit(x_train,y_train,validation_data=(x_test,y_test), verbose=1, batch_size=256, epochs=200, callbacks=[es])"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Jy9SpA_prvEW","colab_type":"code","colab":{}},"source":["#Predicciones de la red entrenada, medidad con Accuracy, precision, reacall y F1.\n","y_pred = model.predict(x)\n","y_pred = np.argmax(y_pred,axis=1) \n","y_true = np.argmax(y,axis=1)\n","\n","print(\"Accuracy: {}\" .format(metrics.accuracy_score(y_true, y_pred)))\n","print(\"Precision: {}\" .format(metrics.precision_score(y_true, y_pred, average='macro')))\n","print(\"Recall: {}\" .format(metrics.recall_score(y_true, y_pred, average='macro')))\n","print(\"F1: {}\" .format(metrics.f1_score(y_true, y_pred, average='macro')))"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Wx05OXJ3sYGT","colab_type":"code","colab":{}},"source":["#Función para representar la matriz de confusión.\n","def plot_confusing_matrix (y_compare,pred,n_categories,outcome_labels):\n","\n"," cm = metrics.confusion_matrix(y_compare, pred, labels = list(range(n_categories)))\n"," plot_confusion_matrix(conf_mat=cm,figsize=(13,13),class_names = outcome_labels,show_normed=True)\n"," plt.title('Confusing Matrix')\n"," plt.ylabel('Target')\n"," plt.xlabel('Predicted')\n"," plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Vlg-OAdbsYLp","colab_type":"code","colab":{}},"source":["#Matriz de confusión.\n","outcome_labels = [\"Normal\",\"Attack\"]\n","plot_confusing_matrix(y_true,y_pred,2,outcome_labels)"],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /MLP_inDeep_binary_class.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"MLP_inDeep_binary_class.ipynb","provenance":[],"collapsed_sections":[],"mount_file_id":"1L7TurSo8Ah0Kun02XoC_cZRfbjgr7WQ9","authorship_tag":"ABX9TyPWhoA7VnaDeOCG5Uc6oIIK"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"code","metadata":{"id":"O1I89gm7yEwC","colab_type":"code","colab":{}},"source":["#Importamos la segunda versión de tensorflow\n","\n","%tensorflow_version 2.x"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"F6saNUBLyND5","colab_type":"code","colab":{}},"source":["#Instalamos las dependencias necesarias que no posee Google Colab.\n","!pip install bayesian-optimization\n","!pip install mlxtend --upgrade --no-deps"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"_P9Ro5MnyNF-","colab_type":"code","colab":{}},"source":["#Importamos los paquetes necesarios para el proyecto\n","import pandas as pd\n","import numpy as np\n","import matplotlib.pyplot as plt\n","from tensorflow.keras.utils import get_file\n","from sklearn import preprocessing\n","from sklearn.preprocessing import LabelEncoder\n","from PIL import Image\n","from sklearn.model_selection import StratifiedShuffleSplit\n","from sklearn import metrics\n","from keras import backend as K\n","from keras.utils.generic_utils import get_custom_objects\n","from bayes_opt import BayesianOptimization\n","from sklearn.model_selection import StratifiedKFold\n","from mlxtend.plotting import plot_confusion_matrix\n","from sklearn.model_selection import train_test_split\n","from imblearn.over_sampling import SMOTE\n","\n","\n","from tensorflow.keras.models import Sequential,Model\n","from tensorflow.keras.layers import Dense, Conv2D,LSTM, MaxPooling2D,UpSampling2D,Conv2DTranspose, Dropout, Flatten, Activation, LeakyReLU, ReLU, Input,concatenate,BatchNormalization\n","from tensorflow.keras.optimizers import Adam\n","from keras.regularizers import l1\n","from tensorflow.keras.callbacks import EarlyStopping"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"NHaznnfOyNH5","colab_type":"code","colab":{}},"source":["#Descargamos el set de datos subido previamente al Google Drive (Tarda menos que descargarlo directamente de la página).\n","path_train = \"/content/drive/My Drive/TFG/UNSW_NB15_training-set.csv\"\n","path_test = \"/content/drive/My Drive/TFG/UNSW_NB15_testing-set.csv\"\n","\n","#Leemos los datos\n","df_train=pd.read_csv(path_train,dtype='unicode')\n","df_test=pd.read_csv(path_test,dtype='unicode')\n","\n","#Quitamos las columnas no necesarias\n","df_train.drop('id', axis=1, inplace=True)\n","df_train.drop('attack_cat', axis=1, inplace=True)\n","df_test.drop('id', axis=1, inplace=True)\n","df_test.drop('attack_cat', axis=1, inplace=True)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"T7FuJYI9yNJw","colab_type":"code","colab":{}},"source":["#Concatenamos el set de entrenamiento y el de test para manipular directamente el conjunto.\n","df = pd.concat([df_train,df_test],axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"OM78CLysy1kN","colab_type":"code","colab":{}},"source":["#División del dataset por clases.\n","df_class_Normal = df[df['label'] == '0']\n","df_class_Attack = df[df['label'] == '1']\n","\n","#Under-sampling de la categoría attack hasta alcanzar el mismo valor que la categoria Normal\n","df_class_Attack = df_class_Attack.sample(df_class_Normal.shape[0])\n","\n","#Concatenado\n","df = pd.concat([df_class_Normal,df_class_Attack], axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"FhOiR9yOyNL2","colab_type":"code","colab":{}},"source":["#Visualización en una gráfica circular del peso de cada clase\n","print(\"Shape of dataFrame: {} \\n\".format(df.shape))\n","print(\"Number of attack samples\")\n","display(df['label'].value_counts())\n","print(\"\")\n","print(\"Plotting balance of dataFrame\")\n","df_plot = (df['label'].value_counts(normalize=True) *100)\n","df_plot.plot(kind='pie',figsize=(10,10),title='Balance of dataset (%)')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"WDKVmdvByNNx","colab_type":"code","colab":{}},"source":["#Conversión de los datos tipo string a enteros en un rango entre 0 y 255 (1 Byte de información)\n","def encode_string_byte (df,name):\n"," df[name] = LabelEncoder().fit_transform(df[name])\n"," #df[name] = [(x).to_bytes(1,byteorder='big') for x in df[name]]\n","\n","encode_string_byte (df,'proto')\n","encode_string_byte (df,'state') \n","encode_string_byte (df,'service') "],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"3uLeix38yNQG","colab_type":"code","colab":{}},"source":["#Normalización de los números enteros en valores decimales en rango entre 0 y 1\n","def numerical_minmax_normalization (df, name):\n"," x = df[name].values.reshape(-1,1)\n"," min_max_scaler = preprocessing.MinMaxScaler()\n"," x_scaled = min_max_scaler.fit_transform(x)\n"," df[name] = x_scaled\n","\n","numerical_minmax_normalization(df,'dur')\n","numerical_minmax_normalization(df,'spkts')\n","numerical_minmax_normalization(df,'dpkts')\n","numerical_minmax_normalization(df,'sbytes')\n","numerical_minmax_normalization(df,'dbytes')\n","numerical_minmax_normalization(df,'rate')\n","numerical_minmax_normalization(df,'sttl')\n","numerical_minmax_normalization(df,'dttl')\n","numerical_minmax_normalization(df,'sload')\n","numerical_minmax_normalization(df,'dload')\n","numerical_minmax_normalization(df,'sloss')\n","numerical_minmax_normalization(df,'dloss')\n","numerical_minmax_normalization(df,'sinpkt')\n","numerical_minmax_normalization(df,'dinpkt')\n","numerical_minmax_normalization(df,'sjit')\n","numerical_minmax_normalization(df,'djit')\n","numerical_minmax_normalization(df,'swin')\n","numerical_minmax_normalization(df,'stcpb')\n","numerical_minmax_normalization(df,'dtcpb')\n","numerical_minmax_normalization(df,'dwin')\n","numerical_minmax_normalization(df,'tcprtt')\n","numerical_minmax_normalization(df,'synack')\n","numerical_minmax_normalization(df,'ackdat')\n","numerical_minmax_normalization(df,'smean')\n","numerical_minmax_normalization(df,'dmean')\n","numerical_minmax_normalization(df,'trans_depth')\n","numerical_minmax_normalization(df,'response_body_len')\n","numerical_minmax_normalization(df,'ct_srv_src')\n","numerical_minmax_normalization(df,'ct_state_ttl')\n","numerical_minmax_normalization(df,'ct_dst_ltm')\n","numerical_minmax_normalization(df,'ct_src_dport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_sport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_src_ltm')\n","numerical_minmax_normalization(df,'is_ftp_login')\n","numerical_minmax_normalization(df,'ct_ftp_cmd')\n","numerical_minmax_normalization(df,'ct_flw_http_mthd')\n","numerical_minmax_normalization(df,'ct_src_ltm')\n","numerical_minmax_normalization(df,'ct_srv_dst')\n","numerical_minmax_normalization(df,'is_sm_ips_ports')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"0oPb1jXuyNT3","colab_type":"code","colab":{}},"source":["#Mapeo de los valores normalizados del paso anterior a valores enteros entre 0 y 255 (1 Byte de información)\n","def numerical_split_ohe (df,name):\n"," pd_to_np = df[name].tolist()\n"," np_split = []\n"," \n"," categories = np.linspace(0, 1, num=256,endpoint=False)\n"," quantization = range(0,256)\n","\n"," for value in pd_to_np:\n"," for i in range(len(categories)-1):\n"," if (categories[i] <= float(value) <= categories[i+1]):\n"," np_split.append(quantization[i])\n"," break\n"," if (float(value) > categories[-1]):\n"," np_split.append(quantization[-1])\n"," break\n"," \n"," df[name] = np_split\n","\n","\n","numerical_split_ohe(df,'dur')\n","numerical_split_ohe(df,'spkts')\n","numerical_split_ohe(df,'dpkts')\n","numerical_split_ohe(df,'sbytes')\n","numerical_split_ohe(df,'dbytes')\n","numerical_split_ohe(df,'rate')\n","numerical_split_ohe(df,'sttl')\n","numerical_split_ohe(df,'dttl')\n","numerical_split_ohe(df,'sload')\n","numerical_split_ohe(df,'dload')\n","numerical_split_ohe(df,'sloss')\n","numerical_split_ohe(df,'dloss')\n","numerical_split_ohe(df,'sinpkt')\n","numerical_split_ohe(df,'dinpkt')\n","numerical_split_ohe(df,'sjit')\n","numerical_split_ohe(df,'djit')\n","numerical_split_ohe(df,'swin')\n","numerical_split_ohe(df,'stcpb')\n","numerical_split_ohe(df,'dtcpb')\n","numerical_split_ohe(df,'dwin')\n","numerical_split_ohe(df,'tcprtt')\n","numerical_split_ohe(df,'synack')\n","numerical_split_ohe(df,'ackdat')\n","numerical_split_ohe(df,'smean')\n","numerical_split_ohe(df,'dmean')\n","numerical_split_ohe(df,'trans_depth')\n","numerical_split_ohe(df,'response_body_len')\n","numerical_split_ohe(df,'ct_srv_src')\n","numerical_split_ohe(df,'ct_state_ttl')\n","numerical_split_ohe(df,'ct_dst_ltm')\n","numerical_split_ohe(df,'ct_src_dport_ltm')\n","numerical_split_ohe(df,'ct_dst_sport_ltm')\n","numerical_split_ohe(df,'ct_dst_src_ltm')\n","numerical_split_ohe(df,'is_ftp_login')\n","numerical_split_ohe(df,'ct_ftp_cmd')\n","numerical_split_ohe(df,'ct_flw_http_mthd')\n","numerical_split_ohe(df,'ct_src_ltm')\n","numerical_split_ohe(df,'ct_srv_dst')\n","numerical_split_ohe(df,'is_sm_ips_ports')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"dSkLL7xkyNV_","colab_type":"code","colab":{}},"source":["#Quitando la columna attack_cat y guardandola en la variable y.\n","y_column = df['label']\n","df.drop('label',axis=1,inplace=True)\n","dummies = pd.get_dummies(y_column) \n","y = dummies.values"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"WqSHxrDMzcBR","colab_type":"code","colab":{}},"source":["#Normalización de los valores entre -0,5 y 0,5\n","x = []\n","for image in np.array(df.to_numpy()):\n"," x.append((image/255 - 0.5))\n","x = np.array(x)\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"At6RHtetyNYJ","colab_type":"code","colab":{}},"source":["#Separación del dataset en un set de entrenamiento y otro de validación.\n","sss = StratifiedShuffleSplit(n_splits=1, test_size=0.25, random_state=42)\n","\n","for train_index, test_index in sss.split(x,y):\n"," x_train, x_test = x[train_index], x[test_index]\n"," y_train, y_test = y[train_index], y[test_index]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"9wRKt9kcyNad","colab_type":"code","colab":{}},"source":["#Definición del modelo final.\n","def MLP_model():\n","\n"," input_img = Input(shape = 42)\n"," output = Dense(42, activation= 'relu')(input_img)\n"," output = Dense(512, activation= 'relu')(output)\n"," output = Dense(256, activation= 'relu')(output)\n"," output = Dense(128, activation= 'relu')(output)\n"," out = Dense(2, activation='sigmoid')(output)\n","\n"," model = Model(inputs = input_img, outputs = out)\n"," model.compile(optimizer=Adam(lr=0.001), loss=\"binary_crossentropy\", metrics=[\"accuracy\"])\n","\n"," return model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Bz9Lkh--zFSU","colab_type":"code","colab":{}},"source":["#Proceso de entrenamiento\n","model = MLP_model()\n","es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, min_delta=0.01, patience=10, restore_best_weights=True)\n","history = model.fit(x_train,y_train,validation_data=(x_test,y_test), verbose=1, batch_size=512, epochs=200, callbacks=[es])"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"LX6DV17_zRtT","colab_type":"code","colab":{}},"source":["#Predicciones de la red entrenada, medidad con Accuracy, precision, reacall y F1.\n","y_pred = model.predict(x)\n","y_pred = np.argmax(y_pred,axis=1) \n","y_true = np.argmax(y,axis=1)\n","\n","print(\"Accuracy: {}\" .format(metrics.accuracy_score(y_true, y_pred)))\n","print(\"Precision: {}\" .format(metrics.precision_score(y_true, y_pred, average='macro')))\n","print(\"Recall: {}\" .format(metrics.recall_score(y_true, y_pred, average='macro')))\n","print(\"F1: {}\" .format(metrics.f1_score(y_true, y_pred, average='macro')))"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"-JM6IiJgzRrG","colab_type":"code","colab":{}},"source":["#Función para representar la matriz de confusión.\n","def plot_confusing_matrix (y_compare,pred,n_categories,outcome_labels):\n","\n"," cm = metrics.confusion_matrix(y_compare, pred, labels = list(range(n_categories)))\n"," plot_confusion_matrix(conf_mat=cm,figsize=(13,13),class_names = outcome_labels,show_normed=True)\n"," plt.title('Confusing Matrix')\n"," plt.ylabel('Target')\n"," plt.xlabel('Predicted')\n"," plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"yUpngcauzRvm","colab_type":"code","colab":{}},"source":["#Matriz de confusión.\n","outcome_labels = [\"Normal\",\"Attack\"]\n","plot_confusing_matrix(y_true,y_pred,2,outcome_labels)"],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /InDeep_binary_class.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"InDeep_binary_class.ipynb","provenance":[],"collapsed_sections":[],"mount_file_id":"1_122ttph4oGcyizT8U-l_9vDEhN0jLbd","authorship_tag":"ABX9TyMzAWZzg3WmGWx5Qk41MxGU"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"code","metadata":{"id":"cEuuR-z_BmeD","colab_type":"code","colab":{}},"source":["#Importamos la segunda versión de tensorflow\n","%tensorflow_version 2.x"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"-zzLBI0iBs33","colab_type":"code","colab":{}},"source":["#Instalamos las dependencias necesarias que no posee Google Colab.\n","!pip install bayesian-optimization\n","!pip install mlxtend --upgrade --no-deps"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"ZdterqtlBs6B","colab_type":"code","colab":{}},"source":["#Importamos los paquetes necesarios para el proyecto\n","import pandas as pd\n","import numpy as np\n","import matplotlib.pyplot as plt\n","from tensorflow.keras.utils import get_file\n","from sklearn import preprocessing\n","from sklearn.preprocessing import LabelEncoder\n","from PIL import Image\n","from sklearn.model_selection import StratifiedShuffleSplit\n","from sklearn import metrics\n","from keras import backend as K\n","from keras.utils.generic_utils import get_custom_objects\n","from bayes_opt import BayesianOptimization\n","from sklearn.model_selection import StratifiedKFold\n","from mlxtend.plotting import plot_confusion_matrix\n","from sklearn.model_selection import train_test_split\n","from imblearn.over_sampling import SMOTE\n","\n","\n","from tensorflow.keras.models import Sequential,Model\n","from tensorflow.keras.layers import Dense, Conv2D,LSTM, MaxPooling2D,UpSampling2D,Conv2DTranspose, Dropout, Flatten, Activation, LeakyReLU, ReLU, Input,concatenate,BatchNormalization\n","from tensorflow.keras.optimizers import Adam\n","from keras.regularizers import l1\n","from tensorflow.keras.callbacks import EarlyStopping"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"MrSpVkOxBs8N","colab_type":"code","colab":{}},"source":["#Descargamos el set de datos subido previamente al Google Drive (Tarda menos que descargarlo directamente de la página).\n","path_train = \"/content/drive/My Drive/TFG/UNSW_NB15_training-set.csv\"\n","path_test = \"/content/drive/My Drive/TFG/UNSW_NB15_testing-set.csv\"\n","\n","#Leemos los datos\n","df_train=pd.read_csv(path_train,dtype='unicode')\n","df_test=pd.read_csv(path_test,dtype='unicode')\n","\n","#Quitamos las columnas no necesarias\n","df_train.drop('id', axis=1, inplace=True)\n","df_train.drop('attack_cat', axis=1, inplace=True)\n","df_test.drop('id', axis=1, inplace=True)\n","df_test.drop('attack_cat', axis=1, inplace=True)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"xhDayPhNBs-c","colab_type":"code","colab":{}},"source":["#Concatenamos el set de entrenamiento y el de test para manipular directamente el conjunto.\n","df = pd.concat([df_train,df_test],axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"QtizAFEQ7lYM","colab_type":"code","colab":{}},"source":["#División del dataset por clases.\n","df_class_Normal = df[df['label'] == '0']\n","df_class_Attack = df[df['label'] == '1']\n","\n","#Under-sampling de la categoría attack hasta alcanzar el mismo valor que la categoria Normal\n","df_class_Attack = df_class_Attack.sample(df_class_Normal.shape[0])\n","\n","#Concatenado\n","df = pd.concat([df_class_Normal,df_class_Attack], axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"eYw3fPlNBtDv","colab_type":"code","colab":{}},"source":["#Visualización en una gráfica circular del peso de cada clase\n","print(\"Shape of dataFrame: {} \\n\".format(df.shape))\n","print(\"Number of attack samples\")\n","display(df['label'].value_counts())\n","print(\"\")\n","print(\"Plotting balance of dataFrame\")\n","df_plot = (df['label'].value_counts(normalize=True) *100)\n","df_plot.plot(kind='pie',figsize=(10,10),title='Balance of dataset (%)')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"F8GyxFhhkTyR","colab_type":"code","colab":{}},"source":["#Conversión de los datos tipo string a enteros en un rango entre 0 y 255 (1 Byte de información)\n","def encode_string_byte (df,name):\n"," df[name] = LabelEncoder().fit_transform(df[name])\n","\n","encode_string_byte (df,'proto')\n","encode_string_byte (df,'state') \n","encode_string_byte (df,'service') "],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"ZWFV0AcMBtF8","colab_type":"code","colab":{}},"source":["#Normalización de los números enteros en valores decimales en rango entre 0 y 1\n","def numerical_minmax_normalization (df, name):\n"," x = df[name].values.reshape(-1,1)\n"," min_max_scaler = preprocessing.MinMaxScaler()\n"," x_scaled = min_max_scaler.fit_transform(x)\n"," df[name] = x_scaled\n","\n","numerical_minmax_normalization(df,'dur')\n","numerical_minmax_normalization(df,'spkts')\n","numerical_minmax_normalization(df,'dpkts')\n","numerical_minmax_normalization(df,'sbytes')\n","numerical_minmax_normalization(df,'dbytes')\n","numerical_minmax_normalization(df,'rate')\n","numerical_minmax_normalization(df,'sttl')\n","numerical_minmax_normalization(df,'dttl')\n","numerical_minmax_normalization(df,'sload')\n","numerical_minmax_normalization(df,'dload')\n","numerical_minmax_normalization(df,'sloss')\n","numerical_minmax_normalization(df,'dloss')\n","numerical_minmax_normalization(df,'sinpkt')\n","numerical_minmax_normalization(df,'dinpkt')\n","numerical_minmax_normalization(df,'sjit')\n","numerical_minmax_normalization(df,'djit')\n","numerical_minmax_normalization(df,'swin')\n","numerical_minmax_normalization(df,'stcpb')\n","numerical_minmax_normalization(df,'dtcpb')\n","numerical_minmax_normalization(df,'dwin')\n","numerical_minmax_normalization(df,'tcprtt')\n","numerical_minmax_normalization(df,'synack')\n","numerical_minmax_normalization(df,'ackdat')\n","numerical_minmax_normalization(df,'smean')\n","numerical_minmax_normalization(df,'dmean')\n","numerical_minmax_normalization(df,'trans_depth')\n","numerical_minmax_normalization(df,'response_body_len')\n","numerical_minmax_normalization(df,'ct_srv_src')\n","numerical_minmax_normalization(df,'ct_state_ttl')\n","numerical_minmax_normalization(df,'ct_dst_ltm')\n","numerical_minmax_normalization(df,'ct_src_dport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_sport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_src_ltm')\n","numerical_minmax_normalization(df,'is_ftp_login')\n","numerical_minmax_normalization(df,'ct_ftp_cmd')\n","numerical_minmax_normalization(df,'ct_flw_http_mthd')\n","numerical_minmax_normalization(df,'ct_src_ltm')\n","numerical_minmax_normalization(df,'ct_srv_dst')\n","numerical_minmax_normalization(df,'is_sm_ips_ports')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"xElC18PzBtKE","colab_type":"code","colab":{}},"source":["#Mapeo de los valores normalizados del paso anterior a valores enteros entre 0 y 255 (1 Byte de información)\n","def numerical_split_ohe (df,name):\n"," pd_to_np = df[name].tolist()\n"," np_split = []\n"," \n"," categories = np.linspace(0, 1, num=256,endpoint=False)\n"," quantization = range(0,256)\n","\n"," for value in pd_to_np:\n"," for i in range(len(categories)-1):\n"," if (categories[i] <= float(value) <= categories[i+1]):\n"," np_split.append(quantization[i])\n"," break\n"," if (float(value) > categories[-1]):\n"," np_split.append(quantization[-1])\n"," break\n"," \n"," df[name] = np_split\n","\n","\n","numerical_split_ohe(df,'dur')\n","numerical_split_ohe(df,'spkts')\n","numerical_split_ohe(df,'dpkts')\n","numerical_split_ohe(df,'sbytes')\n","numerical_split_ohe(df,'dbytes')\n","numerical_split_ohe(df,'rate')\n","numerical_split_ohe(df,'sttl')\n","numerical_split_ohe(df,'dttl')\n","numerical_split_ohe(df,'sload')\n","numerical_split_ohe(df,'dload')\n","numerical_split_ohe(df,'sloss')\n","numerical_split_ohe(df,'dloss')\n","numerical_split_ohe(df,'sinpkt')\n","numerical_split_ohe(df,'dinpkt')\n","numerical_split_ohe(df,'sjit')\n","numerical_split_ohe(df,'djit')\n","numerical_split_ohe(df,'swin')\n","numerical_split_ohe(df,'stcpb')\n","numerical_split_ohe(df,'dtcpb')\n","numerical_split_ohe(df,'dwin')\n","numerical_split_ohe(df,'tcprtt')\n","numerical_split_ohe(df,'synack')\n","numerical_split_ohe(df,'ackdat')\n","numerical_split_ohe(df,'smean')\n","numerical_split_ohe(df,'dmean')\n","numerical_split_ohe(df,'trans_depth')\n","numerical_split_ohe(df,'response_body_len')\n","numerical_split_ohe(df,'ct_srv_src')\n","numerical_split_ohe(df,'ct_state_ttl')\n","numerical_split_ohe(df,'ct_dst_ltm')\n","numerical_split_ohe(df,'ct_src_dport_ltm')\n","numerical_split_ohe(df,'ct_dst_sport_ltm')\n","numerical_split_ohe(df,'ct_dst_src_ltm')\n","numerical_split_ohe(df,'is_ftp_login')\n","numerical_split_ohe(df,'ct_ftp_cmd')\n","numerical_split_ohe(df,'ct_flw_http_mthd')\n","numerical_split_ohe(df,'ct_src_ltm')\n","numerical_split_ohe(df,'ct_srv_dst')\n","numerical_split_ohe(df,'is_sm_ips_ports')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"DlYHxzb6BtMF","colab_type":"code","colab":{}},"source":["#Quitando la columna attack_cat y guardandola en la variable y.\n","y_column = df['label']\n","df.drop('label',axis=1,inplace=True)\n","dummies = pd.get_dummies(y_column) \n","y = dummies.values"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"vdAXl3F2BtSm","colab_type":"code","colab":{}},"source":["#Padding para que cada fila de datos tenga 64 valores.\n","byte_images = np.pad(df.to_numpy(), ((0,0),(0,22)), 'constant')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"dsjSkRzUBtU6","colab_type":"code","colab":{}},"source":["#Normalización de los valores entre -0,5 y 0,5\n","x = []\n","for image in np.array(byte_images):\n"," x.append((image/255 - 0.5).reshape(8,8))\n","x = np.array(x)\n","x = x.reshape(x.shape[0],8,8,1)\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"ArRvwJJxBtWo","colab_type":"code","colab":{}},"source":["#Función de activación Swish\n","def swish(x):\n"," return (K.sigmoid(x) * x)\n","\n","get_custom_objects().update({'swish': Activation(swish)})"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"HQJZUO8KBtY3","colab_type":"code","colab":{}},"source":["#Separación del dataset en un set de entrenamiento y otro de validación.\n","sss = StratifiedShuffleSplit(n_splits=1, test_size=0.25, random_state=42)\n","\n","for train_index, test_index in sss.split(x,y):\n"," x_train, x_test = x[train_index], x[test_index]\n"," y_train, y_test = y[train_index], y[test_index]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"SW-tGkVYBtbI","colab_type":"code","colab":{}},"source":["#Definición del modelo final.\n","def InDeep_model():\n","\n"," input_img = Input(shape = (8, 8, 1))\n","\n"," block_1 = BatchNormalization()(input_img)\n"," block_1 = Activation(swish)(block_1)\n"," block_1 = Conv2D(16, (3,3), padding='same')(block_1)\n"," block_1 = BatchNormalization()(block_1)\n"," block_1 = Activation(swish)(block_1)\n"," block_1 = Conv2D(16, (3,3), padding='same')(block_1)\n","\n"," concat_1 = concatenate([input_img, block_1], axis = 3)\n","\n"," block_2 = BatchNormalization()(concat_1)\n"," block_2 = Activation(swish)(block_2)\n"," block_2 = Conv2D(32, (3,3), padding='same')(block_2)\n"," block_2 = BatchNormalization()(block_2)\n"," block_2 = Activation(swish)(block_2)\n"," block_2 = Conv2D(32, (3,3), padding='same')(block_2)\n","\n"," concat_2 = concatenate([input_img, block_1,block_2], axis = 3)\n","\n"," block_3 = BatchNormalization()(concat_2)\n"," block_3 = Activation(swish)(block_3)\n"," block_3 = Conv2D(64, (3,3), padding='same')(block_3)\n"," block_3 = BatchNormalization()(block_3)\n"," block_3 = Activation(swish)(block_3)\n"," block_3 = Conv2D(64, (3,3), padding='same',strides=(2,2))(block_3)\n","\n"," output = Flatten()(block_3)\n"," output = Dense(128, activation=Activation(swish))(output)\n"," output = Dropout(rate=0.2)(output)\n"," output = Dense(64, activation=Activation(swish))(output)\n"," output = Dropout(rate=0.2)(output)\n"," out = Dense(2, activation='sigmoid')(output)\n","\n"," model = Model(inputs = input_img, outputs = out)\n"," model.compile(optimizer='adam', loss=\"binary_crossentropy\", metrics=[\"accuracy\"])\n","\n"," return model"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"RqiyIccHlev6","colab_type":"code","colab":{}},"source":["#Proceso de entrenamiento\n","model = InDeep_model()\n","es = EarlyStopping(monitor='accuracy', mode='max', verbose=1, min_delta=0.005, patience=10, restore_best_weights=True)\n","history = model.fit(x_train,y_train,validation_data=(x_test,y_test), verbose=1, batch_size=256, epochs=200, callbacks=[es])"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"ERmJEAf9rR-0","colab_type":"code","colab":{}},"source":["#Predicciones de la red entrenada, medidad con Accuracy, precision, reacall y F1.\n","y_pred = model.predict(x)\n","y_pred = np.argmax(y_pred,axis=1) \n","y_true = np.argmax(y,axis=1)\n","\n","print(\"Accuracy: {}\" .format(metrics.accuracy_score(y_true, y_pred)))\n","print(\"Precision: {}\" .format(metrics.precision_score(y_true, y_pred, average='macro')))\n","print(\"Recall: {}\" .format(metrics.recall_score(y_true, y_pred, average='macro')))\n","print(\"F1: {}\" .format(metrics.f1_score(y_true, y_pred, average='macro')))"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"MOn0UlgClex9","colab_type":"code","colab":{}},"source":["#Función para representar la matriz de confusión.\n","def plot_confusing_matrix (y_compare,pred,n_categories,outcome_labels):\n","\n"," cm = metrics.confusion_matrix(y_compare, pred, labels = list(range(n_categories)))\n"," plot_confusion_matrix(conf_mat=cm,figsize=(13,13),class_names = outcome_labels,show_normed=True)\n"," plt.title('Confusing Matrix')\n"," plt.ylabel('Target')\n"," plt.xlabel('Predicted')\n"," plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"SZEGsohLle0E","colab_type":"code","colab":{}},"source":["#Matriz de confusión.\n","outcome_labels = [\"Normal\",\"Attack\"]\n","plot_confusing_matrix(y_true,y_pred,2,outcome_labels)"],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /inDeep_Feature_study.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"inDeep_Feature_study.ipynb","provenance":[],"collapsed_sections":[],"mount_file_id":"1jV-vAKbDunozKHHtpUOvkgD30cRexNxt","authorship_tag":"ABX9TyPMW3k9fKm3oDaVT4kDX+ra"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"code","metadata":{"id":"MMjylng_Y75w","colab_type":"code","colab":{}},"source":["#Importamos la segunda versión de tensorflow\n","%tensorflow_version 2.x"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"HUCdrpC0ZBq6","colab_type":"code","colab":{}},"source":["#Instalamos las dependencias necesarias que no posee Google Colab.\n","!pip install bayesian-optimization\n","!pip install mlxtend --upgrade --no-deps"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"jMTqRdAWZBtC","colab_type":"code","colab":{}},"source":["#Importamos los paquetes necesarios para el proyecto\n","import pandas as pd\n","import numpy as np\n","import matplotlib.pyplot as plt\n","from tensorflow.keras.utils import get_file\n","from sklearn import preprocessing\n","from sklearn.preprocessing import LabelEncoder\n","from PIL import Image\n","from sklearn.model_selection import StratifiedShuffleSplit\n","from sklearn import metrics\n","from keras import backend as K\n","from keras.utils.generic_utils import get_custom_objects\n","from bayes_opt import BayesianOptimization\n","from sklearn.model_selection import StratifiedKFold\n","from mlxtend.plotting import plot_confusion_matrix\n","from sklearn.model_selection import train_test_split\n","from imblearn.over_sampling import SMOTE\n","\n","\n","from tensorflow.keras.models import Sequential,Model\n","from tensorflow.keras.layers import Dense, Conv2D,LSTM, MaxPooling2D,UpSampling2D,Conv2DTranspose, Dropout, Flatten, Activation, LeakyReLU, ReLU, Input,concatenate,BatchNormalization\n","from tensorflow.keras.optimizers import Adam\n","from keras.regularizers import l1\n","from tensorflow.keras.callbacks import EarlyStopping"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"aBXIuEb6ZBu9","colab_type":"code","colab":{}},"source":["#Descargamos el set de datos subido previamente al Google Drive (Tarda menos que descargarlo directamente de la página).\n","path_train = \"/content/drive/My Drive/TFG/UNSW_NB15_training-set.csv\"\n","path_test = \"/content/drive/My Drive/TFG/UNSW_NB15_testing-set.csv\"\n","\n","#Leemos los datos\n","df_train=pd.read_csv(path_train,dtype='unicode')\n","df_test=pd.read_csv(path_test,dtype='unicode')\n","\n","#Quitamos las columnas no necesarias\n","df_train.drop('id', axis=1, inplace=True)\n","df_train.drop('label', axis=1, inplace=True)\n","df_test.drop('id', axis=1, inplace=True)\n","df_test.drop('label', axis=1, inplace=True)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"LzEjcA3bZBw4","colab_type":"code","colab":{}},"source":["#Concatenamos el set de entrenamiento y el de test para manipular directamente el conjunto.\n","df = pd.concat([df_train,df_test],axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"wSIPRImnZBzA","colab_type":"code","colab":{}},"source":["#Conversión de los datos tipo string a enteros en un rango entre 0 y 255 (1 Byte de información)\n","def encode_string_byte (df,name):\n"," df[name] = LabelEncoder().fit_transform(df[name])\n","\n","encode_string_byte (df,'proto')\n","encode_string_byte (df,'state') \n","encode_string_byte (df,'service') \n","\n","display(df.head())\n","print(\"\")\n","print(\"Proto column --> Max value: {} Min value: {} \".format(max(df['proto']),min(df['proto'])))\n","print(\"State column --> Max value: {} Min value: {} \".format(max(df['state']),min(df['state'])))\n","print(\"Service column --> Max value: {} Min value: {} \".format(max(df['service']),min(df['service'])))"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"9F-r932LZB06","colab_type":"code","colab":{}},"source":["#Normalización de los números enteros en valores decimales en rango entre 0 y 1\n","def numerical_minmax_normalization (df, name):\n"," x = df[name].values.reshape(-1,1)\n"," min_max_scaler = preprocessing.MinMaxScaler()\n"," x_scaled = min_max_scaler.fit_transform(x)\n"," df[name] = x_scaled\n","\n","numerical_minmax_normalization(df,'dur')\n","numerical_minmax_normalization(df,'spkts')\n","numerical_minmax_normalization(df,'dpkts')\n","numerical_minmax_normalization(df,'sbytes')\n","numerical_minmax_normalization(df,'dbytes')\n","numerical_minmax_normalization(df,'rate')\n","numerical_minmax_normalization(df,'sttl')\n","numerical_minmax_normalization(df,'dttl')\n","numerical_minmax_normalization(df,'sload')\n","numerical_minmax_normalization(df,'dload')\n","numerical_minmax_normalization(df,'sloss')\n","numerical_minmax_normalization(df,'dloss')\n","numerical_minmax_normalization(df,'sinpkt')\n","numerical_minmax_normalization(df,'dinpkt')\n","numerical_minmax_normalization(df,'sjit')\n","numerical_minmax_normalization(df,'djit')\n","numerical_minmax_normalization(df,'swin')\n","numerical_minmax_normalization(df,'stcpb')\n","numerical_minmax_normalization(df,'dtcpb')\n","numerical_minmax_normalization(df,'dwin')\n","numerical_minmax_normalization(df,'tcprtt')\n","numerical_minmax_normalization(df,'synack')\n","numerical_minmax_normalization(df,'ackdat')\n","numerical_minmax_normalization(df,'smean')\n","numerical_minmax_normalization(df,'dmean')\n","numerical_minmax_normalization(df,'trans_depth')\n","numerical_minmax_normalization(df,'response_body_len')\n","numerical_minmax_normalization(df,'ct_srv_src')\n","numerical_minmax_normalization(df,'ct_state_ttl')\n","numerical_minmax_normalization(df,'ct_dst_ltm')\n","numerical_minmax_normalization(df,'ct_src_dport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_sport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_src_ltm')\n","numerical_minmax_normalization(df,'is_ftp_login')\n","numerical_minmax_normalization(df,'ct_ftp_cmd')\n","numerical_minmax_normalization(df,'ct_flw_http_mthd')\n","numerical_minmax_normalization(df,'ct_src_ltm')\n","numerical_minmax_normalization(df,'ct_srv_dst')\n","numerical_minmax_normalization(df,'is_sm_ips_ports')\n","\n","df.head()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"IRd_C7OAZB2-","colab_type":"code","colab":{}},"source":["#Mapeo de los valores normalizados del paso anterior a valores enteros entre 0 y 255 (1 Byte de información)\n","def numerical_split_ohe (df,name):\n"," pd_to_np = df[name].tolist()\n"," np_split = []\n"," \n"," categories = np.linspace(0, 1, num=256,endpoint=False)\n"," quantization = range(0,256)\n","\n"," for value in pd_to_np:\n"," for i in range(len(categories)-1):\n"," if (categories[i] <= float(value) <= categories[i+1]):\n"," np_split.append(quantization[i])\n"," break\n"," if (float(value) > categories[-1]):\n"," np_split.append(quantization[-1])\n"," break\n"," \n"," df[name] = np_split\n","\n","\n","numerical_split_ohe(df,'dur')\n","numerical_split_ohe(df,'spkts')\n","numerical_split_ohe(df,'dpkts')\n","numerical_split_ohe(df,'sbytes')\n","numerical_split_ohe(df,'dbytes')\n","numerical_split_ohe(df,'rate')\n","numerical_split_ohe(df,'sttl')\n","numerical_split_ohe(df,'dttl')\n","numerical_split_ohe(df,'sload')\n","numerical_split_ohe(df,'dload')\n","numerical_split_ohe(df,'sloss')\n","numerical_split_ohe(df,'dloss')\n","numerical_split_ohe(df,'sinpkt')\n","numerical_split_ohe(df,'dinpkt')\n","numerical_split_ohe(df,'sjit')\n","numerical_split_ohe(df,'djit')\n","numerical_split_ohe(df,'swin')\n","numerical_split_ohe(df,'stcpb')\n","numerical_split_ohe(df,'dtcpb')\n","numerical_split_ohe(df,'dwin')\n","numerical_split_ohe(df,'tcprtt')\n","numerical_split_ohe(df,'synack')\n","numerical_split_ohe(df,'ackdat')\n","numerical_split_ohe(df,'smean')\n","numerical_split_ohe(df,'dmean')\n","numerical_split_ohe(df,'trans_depth')\n","numerical_split_ohe(df,'response_body_len')\n","numerical_split_ohe(df,'ct_srv_src')\n","numerical_split_ohe(df,'ct_state_ttl')\n","numerical_split_ohe(df,'ct_dst_ltm')\n","numerical_split_ohe(df,'ct_src_dport_ltm')\n","numerical_split_ohe(df,'ct_dst_sport_ltm')\n","numerical_split_ohe(df,'ct_dst_src_ltm')\n","numerical_split_ohe(df,'is_ftp_login')\n","numerical_split_ohe(df,'ct_ftp_cmd')\n","numerical_split_ohe(df,'ct_flw_http_mthd')\n","numerical_split_ohe(df,'ct_src_ltm')\n","numerical_split_ohe(df,'ct_srv_dst')\n","numerical_split_ohe(df,'is_sm_ips_ports')\n","\n","display(df.head())"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"xd_QqL0aZB9X","colab_type":"code","colab":{}},"source":["#Quitando la columna attack_cat y guardandola en la variable y.\n","y_column = df['attack_cat']\n","df.drop('attack_cat',axis=1,inplace=True)\n","dummies = pd.get_dummies(y_column) \n","y = dummies.values\n","\n","print(y[:5])\n","display(dummies.head())"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"t8I57rAXZB7I","colab_type":"code","colab":{}},"source":["#Función de activación Swish\n","def swish(x):\n"," return (K.sigmoid(x) * x)\n","\n","get_custom_objects().update({'swish': Activation(swish)})"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"AGAivxe7bC7t","colab_type":"code","colab":{}},"source":["#Lista de columnas seleccionadas por el RFE.\n","list_features = [['smean'],\n","['smean', 'dmean'],\n","['smean', 'dmean', 'ct_srv_dst'],\n","['synack', 'smean', 'dmean', 'ct_srv_dst'],\n","['proto', 'synack', 'smean', 'dmean', 'ct_srv_dst'],\n","['proto', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['proto', 'sttl', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['proto', 'service', 'sttl', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'sttl', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'sttl', 'sload', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'sjit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'dloss', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'dloss', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'sbytes', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'sbytes', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst', 'is_sm_ips_ports'],\n","['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'sbytes', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_ftp_cmd', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst', 'is_sm_ips_ports']]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"H0jJ1TrRZfUG","colab_type":"code","colab":{}},"source":["#Definición del modelo final.\n","def InDeep_model():\n","\n"," input_img = Input(shape = (8, 8, 1))\n","\n"," block_1 = BatchNormalization()(input_img)\n"," block_1 = Activation(swish)(block_1)\n"," block_1 = Conv2D(16, (3,3), padding='same')(block_1)\n"," block_1 = BatchNormalization()(block_1)\n"," block_1 = Activation(swish)(block_1)\n"," block_1 = Conv2D(16, (3,3), padding='same')(block_1)\n","\n"," concat_1 = concatenate([input_img, block_1], axis = 3)\n","\n"," block_2 = BatchNormalization()(concat_1)\n"," block_2 = Activation(swish)(block_2)\n"," block_2 = Conv2D(32, (3,3), padding='same')(block_2)\n"," block_2 = BatchNormalization()(block_2)\n"," block_2 = Activation(swish)(block_2)\n"," block_2 = Conv2D(32, (3,3), padding='same')(block_2)\n","\n"," concat_2 = concatenate([input_img, block_1,block_2], axis = 3)\n","\n"," block_3 = BatchNormalization()(concat_2)\n"," block_3 = Activation(swish)(block_3)\n"," block_3 = Conv2D(64, (3,3), padding='same')(block_3)\n"," block_3 = BatchNormalization()(block_3)\n"," block_3 = Activation(swish)(block_3)\n"," block_3 = Conv2D(64, (3,3), padding='same',strides=(2,2))(block_3)\n","\n"," output = Flatten()(block_3)\n"," output = Dense(128, activation=Activation(swish))(output)\n"," output = Dropout(rate=0.2)(output)\n"," output = Dense(64, activation=Activation(swish))(output)\n"," output = Dropout(rate=0.2)(output)\n"," out = Dense(10, activation='softmax')(output)\n","\n"," model = Model(inputs = input_img, outputs = out)\n"," model.compile(optimizer=Adam(lr=0.001), loss=\"categorical_crossentropy\", metrics=[\"accuracy\"])\n","\n"," return model\n"," \n","model = InDeep_model()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Hca39UpAizS8","colab_type":"code","colab":{}},"source":["#Balanceo del dataset mediante SMOTE (930000 por cada clase) y normalización entre -0,5 y 0,5\n","x=[]\n","for image in df.to_numpy():\n"," x.append((image/255 - 0.5))\n","sm = SMOTE(random_state=0)\n","x, y = sm.fit_sample(x, y)\n","x = np.array(x)\n","df_x = pd.DataFrame(data=x[:],columns=df.columns) "],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"N02ziEqlar7u","colab_type":"code","colab":{}},"source":["#Proceso iterativo de entrenamiento y evaluación de la red CNN en función de la lista de características seleccionadas por el RFE\n","import warnings\n","warnings.filterwarnings('ignore')\n","\n","for feature in list_features:\n"," df_features = df_x.loc[:,feature]\n"," x = np.pad(df_features.to_numpy(), ((0,0),(0,64-len(feature))), 'constant')\n"," x = x.reshape(x.shape[0],8,8,1)\n"," print(\"Number of features: {}\".format(len(feature)))\n"," display(df_features.head())\n","\n"," #Splitting to train and test input/output\n"," sss = StratifiedShuffleSplit(n_splits=1, test_size=0.25, random_state=42)\n","\n"," for train_index, test_index in sss.split(x,y):\n"," x_train, x_test = x[train_index], x[test_index]\n"," y_train, y_test = y[train_index], y[test_index]\n","\n"," \n"," history = model.fit(x_train,y_train,validation_data=(x_test,y_test),verbose=1,batch_size=256,epochs=25)\n"," \n"," y_pred = model.predict(x)\n"," y_pred = np.argmax(y_pred,axis=1) \n"," y_true = np.argmax(y,axis=1)\n","\n"," print(\"Accuracy: {}\" .format(metrics.accuracy_score(y_true, y_pred)))\n"," print(\"Precision: {}\" .format(metrics.precision_score(y_true, y_pred, average='macro')))\n"," print(\"Recall: {}\" .format(metrics.recall_score(y_true, y_pred, average='macro')))\n"," print(\"F1: {}\" .format(metrics.f1_score(y_true, y_pred, average='macro')))"],"execution_count":0,"outputs":[]}]} -------------------------------------------------------------------------------- /RFE_feature_selection.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"RFE_feature_selection.ipynb","provenance":[],"collapsed_sections":[],"mount_file_id":"1svMAvL8w8AAW3sHMWC0KlBRc9ZYDHbhu","authorship_tag":"ABX9TyMlWYoFQdvU42qMOBnnkJtd"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"code","metadata":{"id":"8nCj1d2NAhPA","colab_type":"code","colab":{}},"source":["#Importamos la segunda versión de tensorflow\n","%tensorflow_version 2.x"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Y7Nd06bVAnYr","colab_type":"code","colab":{}},"source":["#Instalamos las dependencias necesarias que no posee Google Colab.\n","!pip install mlxtend --upgrade --no-deps\n","!pip install bayesian-optimization"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"YaKGQHxYAo4U","colab_type":"code","colab":{}},"source":["#Importamos los paquetes necesarios para el proyecto\n","import pandas as pd\n","import numpy as np\n","import matplotlib.pyplot as plt\n","from tensorflow.keras.utils import get_file\n","from sklearn import preprocessing\n","from scipy import stats\n","from array import array\n","from sklearn.model_selection import StratifiedKFold\n","from sklearn import metrics\n","from imblearn.over_sampling import SMOTE\n","from mlxtend.plotting import plot_confusion_matrix\n","from bayes_opt import BayesianOptimization\n","from sklearn.utils import shuffle\n","from sklearn.ensemble import RandomForestClassifier\n","from sklearn.feature_selection import RFE\n","from sklearn.preprocessing import LabelEncoder\n","\n","from sklearn.model_selection import StratifiedShuffleSplit\n","from sklearn import metrics\n","from keras.regularizers import l1\n","from keras import backend as K\n","from keras.utils.generic_utils import get_custom_objects\n","from tensorflow.keras.optimizers import Adam\n","from tensorflow.keras.models import Sequential\n","from tensorflow.keras.layers import Dense, Conv2D,LSTM, MaxPool2D, Dropout, Flatten, Activation, LeakyReLU, ReLU\n","from tensorflow.keras.callbacks import EarlyStopping"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"mYkYWXvKAqrW","colab_type":"code","colab":{}},"source":["#Descargamos el set de datos subido previamente al Google Drive (Tarda menos que descargarlo directamente de la página).\n","path_train = \"/content/drive/My Drive/TFG/UNSW_NB15_training-set.csv\"\n","path_test = \"/content/drive/My Drive/TFG/UNSW_NB15_testing-set.csv\"\n","\n","#Leemos los datos\n","df_train=pd.read_csv(path_train,dtype='unicode')\n","df_test=pd.read_csv(path_test,dtype='unicode')\n","\n","#Quitamos las columnas no necesarias\n","df_train.drop('id', axis=1, inplace=True)\n","df_train.drop('label', axis=1, inplace=True)\n","df_test.drop('id', axis=1, inplace=True)\n","df_test.drop('label', axis=1, inplace=True)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"aJFeunEGCOIZ","colab_type":"code","colab":{}},"source":["#Concatenamos el set de entrenamiento y el de test para manipular directamente el conjunto.\n","df = pd.concat([df_train,df_test],axis=0)\n","print(df.shape)\n","df[:5]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"pDg5SANHElSj","colab_type":"code","colab":{}},"source":["#Número de valores por clase\n","attack_cat = df['attack_cat'].value_counts()\n","\n","count_class_Normal = attack_cat[0]\n","count_class_Generic = attack_cat[1]\n","count_class_Exploits = attack_cat[2]\n","count_class_Fuzzers = attack_cat[3]\n","count_class_Reconnaissance = attack_cat[4]\n","count_class_DoS = attack_cat[5]\n","count_class_Backdoors = attack_cat[6]\n","count_class_Analysis = attack_cat[7]\n","count_class_Shellcode = attack_cat[8]\n","count_class_Worms = attack_cat[9]\n","\n","#División del dataset por clases.\n","df_class_Normal = df[df['attack_cat'] == 'Normal']\n","df_class_Generic = df[df['attack_cat'] == 'Generic']\n","df_class_Exploits = df[df['attack_cat'] == 'Exploits']\n","df_class_Fuzzers = df[df['attack_cat'] == 'Fuzzers']\n","df_class_DoS = df[df['attack_cat'] == 'DoS']\n","df_class_Reconnaissance = df[df['attack_cat'] == 'Reconnaissance']\n","df_class_Backdoor = df[df['attack_cat'] == 'Backdoor']\n","df_class_Analysis = df[df['attack_cat'] == 'Analysis']\n","df_class_Shellcode = df[df['attack_cat'] == 'Shellcode']\n","df_class_Worms = df[df['attack_cat'] == 'Worms']\n","\n","#Under-sampling las categorías Normal, Generic, Exploits, Fuzzers, DoS, Reconnaissance, Backdoor, Analysis y Shellcode aleatoriamente a 1000 muestras\n","df_class_Normal = df_class_Normal.sample(1000)\n","df_class_Generic = df_class_Generic.sample(1000)\n","df_class_Exploits = df_class_Exploits.sample(1000)\n","df_class_Fuzzers = df_class_Fuzzers.sample(1000)\n","df_class_DoS = df_class_DoS.sample(1000)\n","df_class_Reconnaissance = df_class_Reconnaissance.sample(1000)\n","df_class_Backdoor = df_class_Backdoor.sample(1000)\n","df_class_Analysis = df_class_Analysis.sample(1000)\n","df_class_Shellcode = df_class_Shellcode.sample(1000)\n","\n","#Concatenado\n","df = pd.concat([df_class_Normal,df_class_Generic,df_class_Exploits,df_class_Fuzzers,df_class_DoS,df_class_Reconnaissance,df_class_Backdoor,df_class_Analysis,df_class_Shellcode,df_class_Worms], axis=0)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"lfETf-ZYFauX","colab_type":"code","outputId":"c91b9df1-c3bc-4270-c8ca-f74172aec419","executionInfo":{"status":"ok","timestamp":1591807480692,"user_tz":-120,"elapsed":1145,"user":{"displayName":"Farid Bagheri-Gisour Marandyn","photoUrl":"","userId":"05978557582753685866"}},"colab":{"base_uri":"https://localhost:8080/","height":880}},"source":["#Visualización en una gráfica circular del peso de cada clase\n","print(\"Shape of dataFrame: {} \\n\".format(df.shape))\n","print(\"Number of samples per attack\")\n","display(df['attack_cat'].value_counts())\n","print(\"\")\n","print(\"Plotting balance of dataFrame\")\n","df_plot = (df['attack_cat'].value_counts(normalize=True) *100)\n","df_plot.plot(kind='pie',figsize=(10,10),title='Balance of dataset (%)')"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Shape of dataFrame: (9174, 43) \n","\n","Number of samples per attack\n"],"name":"stdout"},{"output_type":"display_data","data":{"text/plain":["Shellcode 1000\n","Normal 1000\n","Analysis 1000\n","DoS 1000\n","Exploits 1000\n","Generic 1000\n","Fuzzers 1000\n","Reconnaissance 1000\n","Backdoor 1000\n","Worms 174\n","Name: attack_cat, dtype: int64"]},"metadata":{"tags":[]}},{"output_type":"stream","text":["\n","Plotting balance of dataFrame\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{"tags":[]},"execution_count":10},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAj4AAAI+CAYAAAC8MgCsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeXhcZd0+8Ps72Zqk7XTfW9J9pm26703SsgqUHRRE2RUqm6waEWVE1MqOWkTfV6Ev+FOUTTCAgjXpSmnpSjuU0n3fmy6ZzHa+vz/OVELokmVmnjMz9+e65moyyzl30ja55znPeY6oKoiIiIgygct0ACIiIqJkYfEhIiKijMHiQ0RERBmDxYeIiIgyBosPERERZQwWHyIiIsoYLD5EKU5EikRERSTbdJaGEJF8EXlLRKpF5G8NeP4UEdmajGzxJiK3iMjTTXztqyJyXrwzEWU6Fh8iBxCRjSISEJEjInJARCpEpKfpXAlyBYDOANqr6lfjuWERuV5E5sZzm03dj4jkAngQwGOxz90i8k8ROSgifxKRrDrP/b2IXFZvE78E8Ei8sxNlOhYfIue4UFVbAugKYBeAXxvOkyinAfhUVSOmgyTYxQA+UdVtsc9vAbAUdukrAnApAIjIBADdVPW1ui9W1Q8BtBaR0UlLTJQBWHyIHEZVawG8AmDQsftEZKqILBWRQyKyRUR8J3q9iNwgIn4ROSwi60XkljqPTRGRrSJyr4jsFpEdInJDncfzReQJEdkUOxQ1V0TyY4+NF5H5sRGL5SIy5SQZvCJSGXvuKhG5KHb/TwD8GMCVsdGtm47z2nwReSE28rUawJh6j5eLyLrY17daRI4VCC+A5wBMiG374Km+dyLSQkReEpF9sayLRKRz7DG3iPwh9j3aJiKPiEjWifZzHOcBqKrzeW8A/1HVIIA5APrERn2eAnDnCbZRCWDqib7PRNR4KTEngCiTiEgBgCsBfFDn7qMArgWwCsAQAO+JyDJVfeM4m9gN4AIA6wGUAXhHRBap6pLY410AuAF0B3A2gFdE5A1VPQDgcQCDAUwEsBPAOACWiHQHUAHgGgDvAjgTwKsi4lHVPfXy5wB4C8AfAZwDoATA30VktKo+JCIKoJ+qfvME34KHAPSN3QoBvFPv8XUASmP5vgrgJRHpp6p+EZkG4FuqWtLA7911se9FTwBBAMMBBGKveyH2vewXy/EPAFtU9Xcn2E99xfWyfwzgLBGZHcv/c9iF5x1VXX+Cbfhhf/+IKE444kPkHG/ERg+qYReSx449oKqVqrpSVS1VXQHgzwAmH28jqlqhquvUVgXgX7B/0R4TBvCwqoZV9W0ARwAMFBEXgBsBfFdVt6lqVFXnx0YovgngbVV9O5bhPQCLAZx/nAjjAbQEMF1VQ6o6C3Zp+HoDvw9fA/AzVd2vqlsA/Kre1/c3Vd0ey/EygLUAxp5oY6f43oUBtIddxKKq+pGqHoqN+pwP4C5VPaqqu2GPzFzVwK8BANoAOFzn8z/ALlkLYY/4LIddJJ8WkedEZLaI1J/Tczi2HSKKE474EDnHJar6fuzwx8UAqkRkkKruFJFxAKbDHrHIBZAH4LhnRMXOBHoIwADYb24KAKys85R99ebX1MAuKh0AtIA9olLfaQC+KiIX1rkvB8B/jvPcbrBHRqw6922CPcLUEN0AbKn32v8SkWsB3AN7ngzqZD+uU3zvXoQ92vMXEWkD4CUAP4T99eYA2CEixzblqpfrVA4AaHXsk9ghzJvr5PobgAcAfCO27ckA/iUi56rqu7GntQJwokNpRNQEHPEhcpjYyMNrAKL4/DDH/wPwJoCequqGPcdE6r9WRPIAvAr7kFVnVW0D4O3jPfc49gKohX2Iqb4tAF5U1TZ1boWqOv04z90OoGdsBOmYXgC2Hee5x7MDdhmp+1oAgIicBuB/ANwO+6ywNrAPIR37+vQ42zvh9y426vUTVR0E+/DeBbAPi22BfeirQ52vt7WqDj7JfupbAbt8fomInAtAYgWnGMBiVVXYo2hD6zzVC3tkiIjihMWHyGHEdjGAtrDneAD2O//9qlorImMBXH2Clx8b0dgDIBIb/TmnIfuNjdD8EcCTItItNpF3QqxMvQTgQhH5Suz+FrGJ0j2Os6mFsEeRviciObFJ0BcC+EtDcgD4K4AfiEjb2PbvqPNYIezSsQewJ3LDHsk5ZheAHmKfSn7MCb93InK6iBTHRtkOwT70ZanqDtiHCJ8QkdYi4hKRviIy+ST7qe9tHOdwpIi0gD0CdVfsrg0ApsS2NQn23KxjJuPLc5yIqBlYfIic4y0ROQL7F/DPAFynqqtij90K4GEROQz7rKi/Hm8DqnoY9oTZv8I+1HI17NGOhroP9mGxRQD2w15LxhWba3Mx7EMze2CPiNyP4/wMUdUQ7KJzHuxRpGcBXKuqnzQww09gH97aALt8vFhn26sBPAFgAezyUQxgXp3XzoI9iXmniOyN3Xey710X2GfQHYJdMqvq7O9a2EVyNezv5Suwlxo40X7qewuAR0S61bv/AQB/UtVjizL+Dvahuj0AtgJ4HQBEZAyAI7HT2okoTsQeXSUiongTkZsBDFLVu0755C+/9lUAf4hNQCeiOGHxISIioozBQ11ERESUMVh8iIiIKGOw+BAREVHGYPEhIiKijMHiQ0RERBmDxYeIiIgyBosPERERZQwWHyIiIsoYLD5ERESUMVh8iIiIKGOw+BAREVHGYPEhIiKijMHiQ0RERBmDxYeIiIgyBosPERERZQwWHyIiIsoYLD5ERESUMVh8iIiIKGOw+BAREVHGYPEhIiKijMHiQ0RERBmDxYeIiIgyBosPERERZQwWHyIiIsoYLD5ERESUMVh8iIiIKGOw+BAREVHGYPEhIiKijMHiQ0RERBmDxYeIiIgyBosPERERZQwWHyIiIsoYLD5ERESUMVh8iIiIKGOw+BAlmYhcIiIqIp5mbOMFEbmiCa97WETOaup+iYhSHYsPUfJ9HcDc2J9Jpao/VtX3k71fIiKnYPEhSiIRaQmgBMBNAK6K3TdFRCpF5BUR+URE/iQiEnvsxyKySEQ+FpHfH7u/zvbOEJE36nx+toi8LiJZsVGhj0VkpYjcHXv8vyNFIjJdRFaLyAoReTxJ3wIiIqNYfIiS62IA76rqpwD2icio2P0jANwFYBCAPgAmxe7/jaqOUdUhAPIBXFBve/8B4BGRjrHPbwDwRwDDAXRX1SGqWgzg+bovEpH2AC4FMFhVhwJ4JJ5fJBGRU7H4ECXX1wH8JfbxX/D54a4PVXWrqloAlgEoit1/uogsFJGVAM4AMLjuxlRVAbwI4Jsi0gbABADvAFgPoI+I/FpEzgVwqF6OagC1AP4gIpcBqInj10hE5FjZpgMQZQoRaQe7vBSLiALIAqAAKgAE6zw1CiBbRFoAeBbAaFXdIiI+AC2Os+nnAbwFu8j8TVUjAA6IyDAAXwEwDcDXANx47AWqGhGRsQDOBHAFgNtj2YiI0hqLD1HyXAHgRVW95dgdIlIFoPQEzz9WcvbG5gZdAeCV+k9S1e0ish3AgwDOim23A4CQqr4qImsAvFT3NbHtFajq2yIyD/YIERFR2mPxIUqerwP4Zb37XgXwHQDr6j9ZVQ+KyP8A+BjATgCLTrLtPwHoqKr+2OfdATwvIscOZ/+g3vNbAfh7bFRJANzTmC+EiChViT1FgIhSmYj8BsBSVf2D6SxERE7G4kOU4kTkIwBHAZytqsFTPZ+IKJOx+BAREVHG4OnsRERElDFYfIiIiChjsPgQERFRxmDxISIioozB4kNEREQZg8WHiIiIMgaLDxEREWUMFh8iIiLKGCw+RERElDFYfIiIiChjsPgQERFRxmDxISIioozB4kNEREQZI9t0ACJKX8UziwVAKwCtYzd37M+WAHIAZMVu2Sf4OAwgCKC2zp/1Pz4MYD+AgyuvW6lJ+tKIKEWJKn9OEFHDFc8sbg2gK4BusT+71vu8I75YcJI1shwFcAB2CdoXux37eDeArQA2A9gCYOvK61aGk5SLiByExYeIvqB4ZnEOgL4A+sf+7AugT+zPngAKzKWLGwvATtglaHPstgnAGgD+ldet3GIwGxElEIsPUYaKFRwvgOEAhsY+HgigCPZhpkx2BLESFLt9EvvzM44UEaU2Fh+iDFA8s7gN7IIzLPbncACDAOSazJWCwgBWA1hy7FZcG1zy/275tNZsLCJqKBYfojRTPLPYBaAYwKTYbQKA3kZDpbHnd+z6eHRtMArgw9htIYBV8FVbZpMR0fGw+BCluOKZxYUAxuGLRae10VCZQjX00cYtyP3yyNlBAHMAVAGoBLCURYjIGVh8iFJMbG7ORABfAXA27MNWXJrCgALL8i/ctNXbgKdW48tFKJrIbER0fCw+RCmgeGZxP9hF5xwAp8NeG4cMK64Nzvl/O3aVNuGl1QBmAfgHgLfhq94Z32SpR0QUwJOqem/s8/sAtFRVXxIzVAK4T1UXJ2uflHx8l0jkQMUzi/Nhj+acC7vw9DGbiI5nYqDJc5rdAC6N3RQ+90cAKmK3xfBVZ+I70iCAy0TkF6q6t7EvFpFsVY0kIBelGRYfIoconlncEsBUAJcDOB9AodlEdCplgUDnOGxGAIyO3R4CsBM+9zuwR4Peha+6Jg77SAURAL8HcDeAH9Z9QESKAPwRQAcAewDcoKqbReQF2Kt3jwAwT0TaAQjEPu8E4EYA18Ke97ZQVa+Pbe+3AMYAyAfwiqo+lNgvjZyExYfIoOKZxW0BXAS77JwNoIXZRNRgqoFBwVAiRuK6ALghdjsKn/stAC8DeAe+6mAC9uckMwCsEJFH693/awAzVXWmiNwI4FcALok91gPARFWNxopQW9hF5yIAb8Ke8P8tAItEZLiqLgPwQ1XdLyJZAP4tIkNVdUXCvzpyBBYfoiQrnlnsBvBVAF8DMAX2NasoxbRUXZcNDEnwbgoBXBW7VcPnfgN2CXoPvuq0O6yjqodE5P8A3Al75OaYCQAui338IoC6xehvqlp3ovhbqqoishLALlVdCQAisgr24pzLAHxNRG6G/TuwK+w1rVh8MgSLD1ESFM8szoI9onMd7HeqHNlJcQNCof1J3qUb9r+f6wDsg8/9GoA/A6hMszlBT8NeHPL5Bj7/aL3Pj42KWXU+PvZ5toj0BnAfgDGqeiA2SsT/jxkkWRcPJMpIxTOLhxTPLH4M9jWh3oH9zp0/ZNPAxJpak5f1aA/g27DPDFsHn/tB+NzdDeaJG1XdD+CvAG6qc/d82P93AOAbsJcGaKrWsMtStYh0BnBeM7ZFKYgjPkRxVjyzuB2Ab8J+Zz7ScBxKkLJAoIvpDDG9AfwUgA8+97sA/gDgrRQ/FPYEgNvrfH4HgOdF5H7EJjc3dcOqulxElsK+/toWAPOaE5RSD9fxIYqT4pnFI2H/sL4K9tkilK5UjyzfuKXA5dxR810AZgL4X/iq15oOQ+QkLD5EzVA8szgX9kTl2wGMNxyHksQdja6Yu3nbUNM5GmgW7Hkz/0izuUBETcJDXURNUDyzuAeAabDnWXQyHIeSzBMKHzCdoRHOiN3Wwuf+FYDn4auuPyGYKGNwxIeoEYpnFo8B8D3YZ2bxjUOGumf/gfk3VB+eaDpHEx0E8D8Afg1f9RbTYYiSjcWHqAGKZxafBeAHsN85U4Z7feuOjf3C4SLTOZopAuA1AE/CV73QdBiiZGHxITqB4pnFAvtaSj+AfTkBIkC1esXGLa3FvtREuvgPgIfhq640HYQo0Vh8iOopnlmcA3utkO8D8BiOQw7TNhpdOnvzthGmcyTIbNgF6N+mgxAlCucoEMXEztC6CfYIT0/DccihBgdDh0xnSKAyAO/D554H4KfwVf/TdCCieGPxoYwXu5zEdQB+BPtaPkQnVBII5JnOkASTALwLn3sh7AJUYToQUbw4dfEtooTze7zyrxLvpQBWw17ttshsIkoFJTW1PUxnSKJxAP4Bn3sBfO4S02GI4oHFhzKS3+M9B8Dinnvx2rhPLK5pQg0iqvtPi0QyqfgcMx7AHPjcb8DnHmg6DFFzsPhQRvF7vKP8Hu+/AfwTsetofafCyjWbilJF+6i13nQGwy4G8DF87ufgc3c2HYaoKVh8KCP4Pd7Ofo/3jwAWod5aPAUhDD5zqfWhmWSUSoqDQY4O2nNDbwHwGXxuH3zulqYDETUGiw+lNb/Hm+P3eO8D8CnsKzofd+2VG9632omqldRwlHLKAgFefPZzLQE8BLsAfRs+N3+fUErgP1RKW36P93wAHwN4DEDrkz03N4J+Fy/QBUkJRilrYk1tL9MZHKgzgN8DWAife5zpMESnwgUMKe34Pd4BsK9GfV5jXhdxYfM192V1jWZJTmKSUSpzqe5evnELL0h7cgrgBQDfh696j+EsRMfFER9KG36PN9/v8f4S9ihPo0oPAGRb6PX1SoujPnRcHaPRTaYzpACBfUh5DXzuW+Bzp9NlPShNsPhQWvB7vGcCWAn7yulNHrG5YJEOzA1rTdyCUdoYFgzx30XDtQXwHIAF8LmHmw5DVBeLD6U0v8fbzu/xPg/gfQB9m7s9l6LzTf+yFjU/GaWbsppAgekMKWgcgMXwuR+Hz82J4eQILD6Usvwe75WwV12+Pp7bnbJChxbUanU8t0mpb2IgUGQ6Q4rKAnAvgGXwuSeZDkPE4kMpx+/x9vR7vG8B+AvsM0riSoC2t/3DWhrv7VLqylLd0TFqdTSdI8UNADAbPvdTHP0hk1h8KKX4Pd5bAKwCcEEi9zN6rY5xH1GelUIAgM6R6GbTGdKEC8BdAFbA5y41HYYyE4sPpQS/x9spNsrzHIBWid6fAIV3vRFdnej9UGoYEQzWms6QZvoBqILP/Qx8bs6doqRi8SHH83u8F8A+Yyuhozz1DdqCCZ0O6rZk7pOcqawmwMsyxJ8AuBP26M9E02Eoc7D4kGP5Pd4Cv8f7HIC3ACR94TgBcu99Lboh2fsl55kQqO1tOkMa6wt79OdBXvaCkoH/yMiR/B7vGABLYV8M0ZiiXZjQa7dm+hW5M1q26pa2ltXOdI40lw3gpwD+DZ+7u+kwlN5YfMhR/B6vy+/xPghgPuyzQIwSIOu+V6O7TOcgc7pFIltNZ8ggUwAsh899oekglL5YfMgx/B5vRwD/gv3OL9twnP/qfBDjPVvUbzoHmTGqNhgynSHDtAfwJnzuX8PnzjMdhtIPiw85gt/jnQj70NaZprPUJ4Dc/Xr0qOkcZEZZTcBtOkOGuh32Fd89poNQemHxIeP8Hu9dACoBOPbYftujGD1qrbXMdA5KMlUdW8uJzQYNA7AIPvcVpoNQ+hBVNZ2BMpTf420F4I8AUuKH2tE8rLzhnuxi0zkoeXJUNyzZuIXFxxl+CeAB+Kot00EotXHEh4zwe7xDACxGipQeACgMonjKCutD0zkoeXqGIztMZ6D/+j6Ad+Bz8ww7ahYWH0o6v8d7FYCFcMBZW4110z+tNuAwacYYU1sbNp2BvuAc2Fd7H2Y6CKUuFh9KGr/HK36P92EAfwaQksvU50Uw4IIPdYHpHJQcZTWBtqYz0Jf0BjAfPvfVpoNQauIcH0oKv8ebD2AmgK+aztJcERc2ffP+rO6WSxxzyj0lgKq1cNPWQIFqoekodEJPAbgfvuqo6SCUOjji4yAiEhWRZSKySkSWi8i9InLSvyMRKRCRP4nIShH5WETmioijrivk93i7AqhCGpQeAMi2cNqVsy2O+qS5XMUGlh7HuxvAG/C5+fdEDcbi4ywBVR2uqoMBnA3gPAAPneI13wWwS1WLVXUIgJsAOGZegt/jHQFgEYAxprPE00UfaL+ciPKK3WmsKBzmxObUcAGA2fC5u5oOQqmBxcehVHU3gJsB3C62FiLyfGxkZ6mInB57alcA2+q8bo2qBk1krs/v8V4GYC4cvD5PU2Upul7/nrXQdA5KnHG1tTxtOnWMBPABfO4hpoOQ87H4OJiqrgeQBfvK5LfZd2kxgK8DmCkiLWCvg/N9EVkgIo+ISH9ziT/n93jvB/AKUnQSc0OcuUyL84N62HQOSoyymkB70xmoUXoBmAuf23Grv5OzsPikjhIALwGAqn4CYBOAAaq6DEAfAI8BaAdgkYh4TYWMnbn1OIBHAYipHMngAtpNe9v6yHQOSgDVyIhgsK/pGNRobthr/dxgOgg5F4uPg4lIHwBRALtP9jxVPaKqr6nqrbDL0fnJyFef3+PNBvACgHtN7N+E8Z/oqNZHdZ/pHBRfLVTX5ylamM5BTZID4I/wuR82HYScicXHoUSkI4DnAPxG7TUH5gD4RuyxAbCHddeIyCQRaRu7PxfAINijQUkVO139DQDXJnvfJgnQ6s43rY9N56D46hMOn/TNBqWEH8Hn/i18bv6eoy/gPwhnyT92OjuA9wH8C8BPYo89C8AlIisBvAzg+tgk5r4AqmL3L4V9GYhXkxna7/G2BfAegKnJ3K9TFG/UcR2qlWcApZHxAU5sThPTAPwffG6uuUX/xQUMqVn8Hm83AP8EkNFnU3zWFXMeuD671HQOio8Xtu/6ZFQw6DGdg+LmDQBXwVftiDNeySyO+FCT+T3efgDmI8NLDwD03YEJPfboRtM5KA5Ug8XBYB/TMSiuLgHwFnzutD3LlBqOxYeaxO/xDoC9GvNpprM4gQDZ970W5eGuNFCgui4XyDWdg+LubAD/gs/tNh2EzGLxoUaLlZ7/AOhmOouTdN2P8f236RrTOah5+oXCPEsvfU0C8B/43B1MByFzWHyoUfwe70AAlWDp+RIB5J7Xo4dM56DmmRioTev1pwgjAFTB5+5oOgiZweJDDRYrPf+BfZkMOo72hzFm2Dprhekc1HRlNYFOpjNQwg0C8D587namg1DysfhQg/g9Xg/skR6WnlO4802Lp0qmKtXAoFCIKzZnhqHgnJ+MxOJDp+T3eL2wR3q6mM6SClrVYljJKmux6RzUeK0sXZdlXx+PMsMoAO/C525lOgglD4sPnVTslPVZYOlplJvfsQrBRbJSzoBQaL/pDJR04wFUwOcuNB2EkoPFh07I7/H2gL2CNEtPI7UIw3vuR/qB6RzUOJMCtRztyUylAN6Ez83rs2UAFh86Lr/H2wH2ZSi4Tk8TXTPL6uKyNGo6BzVcSSDAkp+5zgDwOnxuruGU5lh86Ev8Hm8r2Jeh4JL9zZATRe/L51nzTeegBlI9MjAU7m06Bhl1LoAX4XNzSYM0xuJDX+D3ePMA/B3ASNNZ0sFl87RvdkR5faAU4LasdS7+TCTgawCeNB2CEof/yem//B6vC8BLAE43nSVdZCm6Xftva6HpHHRq3lDooOkM5Bh3wee+13QISgwWH6rrNwCuMB0i3ZyzRAe1COkR0zno5EpqanNMZyBHeQw+91WmQ1D8sfgQAMDv8X4fwHdM50hHLqDDze9YH5nOQSdXEgh0N52BHEUAzITPzRHwNMPiQ/B7vJcB+IXpHOls0mod3rJGD5jOQSegWt0nHOllOgY5Ti7sM72Gmg5C8cPik+H8Hu8oAC/CfndDCSKA+463eA0vp2pnWeuE/wfo+NwA3oHPzWKcJlh8Mpjf4+0O4E0ABaazZILh63Vs+0O603QO+rLBwdBh0xnI0boBeAs+d0vTQaj5WHwylN/jLQTwFuz/0JQEAuTf/Xp0rekc9GWlNYE80xnI8YYCeIlr/KQ+Fp8MVOe09RGms2Sa/tsxvus+3Ww6B33RpEBtT9MZKCVcDOBnpkNQ87D4ZKbpAC4xHSITCZBz32vRraZz0OdEdV+vSIRndFGDqOK2Gx/46eWmc1DTsfhkGL/H+00A95vOkcl67MX4PjuUh7wcon3U2mg6A6WGkGZtmhr6+Z5Z1sgXi8orOGKeolh8Mojf4y0G8DvTOTKdAK57X4vy1HaHKA4GubgkndIedX80Jvhb92ot6gsgH8AbReUVHU3nosZj8ckQfo/XDeBV8AwuR+h4CGOLN1gfm85BQFlNoIXpDORs86ODqsYFZwyvRss2de7uBeCvReUV2aZyUdOw+GSOFwD0Nx2CPvfdv1th0xkImBSoPc10BnImVQSfDF8x9+rwg5MtuLKO85Qp4OKvKYfFJwP4Pd7vgZOZHad1ACMm+K0lpnNkMpfqrq7RaBfTOch5oiq7vxl+YO2vopeVnOKp9xaVV1yQlFAUFyw+ac7v8U4G8HPTOej4pr1t8TCLQZ2iUS4tQF9yRFusLg0+E51nDRnSgKcLgJlF5RVcEiFFsPikMb/H2w3AywCON0RLDpAfwqCzl1gLTefIVMNqgzWmM5CzfGp1nzcq+Fyf7ejQtREvawfgL5zvkxpYfNKU3+PNgl16OpvOQid33ftWB1G1TOfIRGWBWk72JwCAKqy/RKZUnhN6bFIQuU0ZiZ0ILm6YElh80teDAE51bJocIDeKvpfO1wWmc2SiiYFAkekMZJ4qqu8O37qkPHLzlGZu6v6i8orz45GJEkdU1XQGijO/xzsewFzwEFfKiLiw9dr7sjpFsiTXdJZMkaW6fdnGLbxWXYYLafaGC0OPYI326h2nTe4FMGLj9Klcod2hOOKTZvwebysAfwJLT0rJttDj6v9YH5jOkUm6RKJbTGcgs3Zpm8Wjg8+2i2PpAYAOAP5cVF7B368Oxb+Y9PMrAH1Mh6DGO3+xenLDysm2STKyNlhrOgOZMztaXDU++JuRh9DSnYDNlwC4LwHbpThg8Ukjfo/3CgDXm85BTeNSdPr2u9Yi0zkyRVkg0Mp0Bko+VdQ+Gr5y3rXhH0xWuBL5O/DhovKKhpwOT0nGOT5pwu/x9gCwAkBb01mo6RSovvGuLBzNl0S8C6U65mzaeqCNZfH/SwaJquy8OvTD/Qt10KAk7XIZgLEbp0/lKu0OwhGfNOD3eAXATLD0pDwB3Lf9w1pmOke6y1bdzNKTWQ5r/qpJwV9LEksPAAwH8OMk7o8agMUnPdwB4AzTISg+Rn2mo9sc0T2mc6Sz7pHINtMZKHn8Vq+5o4LP9duJdibWNftBUXnFWAP7pRNg8Ulxfo+3CLwkRVoRoPDu16OrTedIZ6NqgyHTGSjxVBF9KXJm1Xmh6SUh5OQZipEF4P+KyivyDe2f6mHxSX2/A1BoOgTFl7tCaL8AACAASURBVGcrJnTer1wHJEHKagKcQ5XmVFF9R/iOZQ9GbppsOguAgeBV3B2DxSeF+T3eawGcYzoHxZ8Aufe9Ht1oOkdaUtWxgVou+ZDGgpq9/pzQowf+YU0YZTpLHXcWlVdwNX0HYPFJUX6PtxOAp0znoMTptRsTi3bqOtM50k0OsLGVamvTOSgxdmi7RaOCz3VYqz2KTGepRwD8rqi8Isd0kEzH4pO6fg37isCUpgRw3ft6lJOc46xXOLLddAZKjH9HR1RODP5q1BEUOLXYDgIXNjSOxScF+T3eiwB8zXQOSrzOBzHeu1k50TmOxtTWRk1noPhSReBn4W/Mvyl8/5QEL0oYDz8qKq/goVaDnP4PhOrxe7ytATxrOgclz91vRAOmM6STspoA1+9JIxF17fha6Mcb/yc6daLpLA2UD2CG6RCZjMUn9fwUQHfTISh52hzFqDFrrKWmc6QF1eio2iDfbaeJai1YOTH46+xF6vGaztJI5xaVV3DU3hAWnxTi93gHA7jVdA5KvlsrLE6IjIM81Q0Fqlz+IQ2stIrmjA4+N3A32nY0naWJni4qr+CyCgaw+KSWZwBkmw5ByVcYxJDTl1sfms6R6k4LR3aZzkDNo4rI85GvzL4w9PPSMLJzTedphq7g4rNGsPikCL/HezmAM03nIHNu/JfVVlQt0zlS2XhObE5pluLAd8LfXfGTyHVlprPEybSi8ooRpkNkGhafFOD3ePMBPGE6B5mVF0H/CxfqAtM5UllZTaC96QzUNLWa89mZoccPv2uNG2k6Sxy5wJ/tScfikxq+D+A00yHIvKuqrB5ZUQ2bzpGSVMPDg8G+pmNQ423T9h+OCj7XeYN262U6SwKcXlRecbHpEJmExcfh/B7vaQC+ZzoHOUO2hdOumm1x1KcJ8lXX5ylamM5BjfPP6OjKScFfjTmK/FamsyTQY1zROXlYfJzvCdjrPhABAC5YqANyw8q1fRqpTzi823QGajhV1DwUvnbBLeF7pgAipvMkWH8At5kOkSlYfBzM7/GWALjcdA5ylixFlxvesxaazpFqxgdqTUegBoqoa9tloZ9smRk9d4LpLEn046LyCl6GKAlYfJztF6YDkDOdsVyH5dfqIdM5UklZTaCD6Qx0age1cPm44Iy8pdp/oOksSdYWwEOmQ2QCFh+H8nu8UwGUmM5BziRA21srrCWmc6QM1WBxMNTPdAw6uaVW39mjg78dtA/uTC2p3ykqrxhgOkS6Y/FxIL/HK+DCVnQKYz/V0e6jutd0jlRQqLouB+DkUYdSReT3kamzLw39tCyC7Ez+e8oBR/oTjsXHma4GMNR0CHI2AVp+9+/WKtM5UkG/UJgF0aEslX03h+/5+OeRb6TLooTNdSkXNUwsFh+H8Xu8OQAeNp2DUsPgTTq+40HdbjqH000M1PJnnQMFNPfT00NPBN6zRg83ncVBBPwdkFD8YeA83wbAq0dTgwiQd+/r0XWmczhdaU2gs+kM9EWbrU4fjAo+132TdulhOosDXVBUXjHGdIh0xeLjIH6PtwDAj0znoNTSeycm9tijG0zncCzVmkGhEN9MOIQq9B/RcZVloafG1aBFoek8DsZRnwRh8XGW2wB0MR2CUosAWfe9Gt1pOodTtbJ0XRaQZToHAao4+qPIDQtvD393SgYsSthc5xaVV2TSOkZJw+LjEH6PtwWAe0znoNTU9QDGD9iqn5jO4UQDQ6EDpjMQENasLReHfrr9pejZ401nSSE/NR0gHbH4OMeN4GgPNZEAcs/r0SOmczjRpEAtR3sM26+tlo0NzihcoX37m86SYs4sKq/g2W5xxuLjAH6PNxvA/aZzUGprdwSjR3xmLTedw2lKAoGupjNkssXWgNljgs8OOYDWvBxD0/zEdIB0w+LjDFcDKDIdglLfHW9ZnDdRl+rhAaFwkekYmUgV4RmRi+ZcEfKVRZGVbTpPCpvCM7zii8XHsNgqzeWmc1B6aFmLoWUrrUWmcziF27LWu/hzLukslb03hu9f/VjkqlLTWdIEjwjEEX8gmHcpAK/pEJQ+vvVPqzVU1XQOJxgUCh00nSHT1Gjemsmhp4L/sUYMM50ljVxWVF7BJRnihMXHvB+YDkDppUUYA89fpAtM53CCkpraXNMZMsl6q8uCUcHnem7RTt1NZ0kzWeBZv3HD4mOQ3+M9G8Bo0zko/XzjP1Y3l6UR0zlMKwkE+As4CVShr0cnVZ4RenJCAHkFpvOkqRuKyivamw6RDlh8zLrbdABKTzkWir46x8rsUR/V6j7hSC/TMdKdKg5/P/LtRXeHb5tiOkuaKwBwq+kQ6YDFxxC/xzsQwLmmc1D6umSB9suOaNB0DlPaWdZ60xnSXVizNl0Q+tmuv0ZPH2s6S4a4vai8ooXpEKmOxcecO2FfhZcoIbIUXa973/rAdA5TBgdDh0xnSGd7tfWSMcFnW6/S3v1MZ8kgnQBcZzpEqmPxMWDJ8NLW4D9eSoKzl+qQ/KAeNp3DhLKaQJ7pDOnqA8s7e2zw2WEH0aqt6SwZ6A7TAVIdi48BC8b/5Maqksc3bup59jxLskKm81D6cgHtb3nbWmI6hwmTArU9TWdIN6oIPR25bM5VoR+VWXDxUiBmDC4qr0ja+kgi8kMRWSUiK0RkmYiME5GNItKhEduYIiL/iH18vYj8Jg65fCJyX1Ney9U0k2zGtFkC4NZodn7/dX0vwbo+F+3psHfF6gGf/W1gi+BBXquL4m7CJzriDzW6/3CBZMwlA0R1b89IhGd0xVFUZc/14e/vnGMN5aKE5k0DMCfROxGRCQAuADBSVYOxspPyS0RwxCf5zgbw+YX6xNVxb8fhk+ePf6TDgrEPLdjXbtAKc9EoHQnQ+s43rZWmcyRTh2h0o+kM6eSotvCXBZ+OzLGGFpvOQgCAy4vKKxo84tIMXQHsVbVPklDVvaq6PfbYHSKyRERWiogHAESkUET+KCIfishSEbn4ZBsXkc4i8rqILI/dJsbuv0dEPo7d7qrz/B+KyKciMhfAwDr39xWRd0XkIxGZcyzPibD4JN9tx71XJDtQ0GnC8qG3Da0sfXLN+qLz50Rd2bVJzkZpaugGHde+WneYzpEsQ4Oho6YzpIvPrG7zRwV/W7QNHXmxV+fIA3BDEvbzLwA9Y2XjWRGZXOexvao6EsBvARw75PRDALNUdSyA0wE8JiKFJ9n+rwBUqeowACMBrBKRUbC/tnEAxgP4toiMiN1/FYDhAM4HUPf6Zb8HcIeqjoplefZkXxSLTxLNmDarO4Cpp3qelZU3cGPR1NKq0qdrlhXfWhVo0WFrEuJRGhOgxT2vRz8znSNZSmsC+aYzpDpVWH+NlFWeFXp8Yi3y+P10npuLyisSemawqh4BMArAzQD2AHhZRK6PPfxa7M+P8PlFts8BUC4iywBUAmgB4GRraZ0BuzhBVaOqWg2gBMDrqno0tv/XAJTGbq+rao2qHgLwJgCISEsAEwH8Lbbf38EeqTohzvFJrmthLz3eMCLt9rcfPHlBO1+0Re3+hf3WvZrbae/yEYmLR+ms3w5M6L5XN23rIKeZzpJokwK1XLiwGVRx6L7wtDWvWmVTTGehE+oH4CwA7yVyJ6oahV1iKkVkJT4/I/nYGmFRfN4lBMDlqrqm7jZEpHMCI7oAHFTV4Y15ASVP005hF8mqzW8/7uMhN4+oLH3qs8/6XDI76srlUD41igDZ974W3WY6R6K5VHd1iUZ5okAThTR743mh6XtftcrGnPrZZNi0RG5cRAaKSP86dw0HsOkkL/kn7Lk/Env9qd6o/xvAd2LPzRIRN+xJ25eISEHsMNmlsftmx+7PF5FWAC4EgNjozwYR+WpsOyIiJ71ALotPksyYNmsC6kzGaiorK7ff5l5nl1WVPhlZOuzOqqMFnU/2j5DoC7rvw4S+2/VT0zkSqVM0yv8TTbRb2yweHXy27Sfai1cCTw0XFZVXJLLktwQwU0RWi8gKAIMA+E7y/J8CyAGwQkRWxT4/me8COD02kvQRgEGqugTACwA+BLAQwP+q6tLY/S8DWA7gHQCL6mznGwBuEpHlAFYBOPmkalU9RS6KhxnTZv0O9nHS+FLVvFD14r7rXpfOuxePEq4GTaewtxUW3Xp7dtq+mz/3yNGqx/bsm3zqZ1Jdc6JDqq4Ll5dwfZ6Uc9/G6VOfMB0ilXDEJwlmTJuVD+DKhGxcRIJ5bcasHnTD6MqyZzZ+2u+rsyNZLbhUP51Qh8MYM3R9+p7eXhYInOwsEqpHFcHHwl+be034gcksPSnpGtMBUg2LT3JcCsCd6J2oK7v31h5TymaXPO76aMQ9sw8Xdl+X6H1Savrum1bUdIZEmRCoTfvJ2/ESVdn1jfADn82IXlJiOgs12bCi8oqhpkOkEhaf5Lg+qXsTaVnt7lu2aMwDfedM/MWS7V0mfKgQK6kZyNFaBTB84mrrI9M54i1LdVuHqNXRdI5UcFjzV5UEf6XzrSGDTWehZuOoTyNwjk+CzZg2qwfsWfBGS6ZY0S1ddn6wvt/6N4blRGramMxCzhDIgf+6e7M8iJ2BkQ56hMMfvLN1x3jTOZxujdVj3kWhR0YFkdvCdBaKi20Aem2cPpVvcBuAIz6JdyUc8H1WV1bPHd0mTZ4z6dG8RaO+N+dQq9PS+sweOrX8MLxfWaILTeeIp5G1weCpn5W5VBH9f5Ezqr4SenQSS09a6Q57gT9qAC5gmHhXmA7wBSL5h1udVrp41PeQHT66vM/Gf9R02z53jEst/lvIQNf+2+r03giJWi5Ji0mtZYHaVqYzOJUqqu8M3772LWsiz3hLT18HUGU6RCrgoa4Eil2iYgucfoq5Rnd03vXRp/3XvTY4N3w4GRe+Iwf5a6lr7islrrSY3Dp309aDbsviodx6gpq9/sLQz+RT7dnbdBZKmH0Aum6cPjVsOojTGT8Ek+Yuh9NLDwBIVtddXcZOnjvxF60Wjn5g7gF3v9WmI1HyXD7PKsqOash0jubKVt3M0vNlO7XtotHB37Zn6Ul77QGcaTpEKmDxSSxnHeY6FZG8oy27lywdcfegqpLHP97U86x5lrj47iHNZVno8c1Z1gemczRX90iEF/Ot5z/RYVUTgr8edRiFCV9OgxzhEtMBUgEPdSXIjGmzusCeaZ/a5VKt3R33LFvd/7NXvS1CBxN5oTkyyBLsue6erIJgrqTs4n+XHzpS5du3n/NXAKiidnrk6x/9LnrhJNNZKKl2AOi+cfpU/mI/CU5oTZzLkOqlBwDE1WlPp5Gd9nQcES4I7Jrff+0rrdof8BebjkXx5VJ0/Pa7VtVvLspK2eIwORDgYS4AUXXt+Hrohwc+VC9LT+bpCmAM7Otc0Qmk/i9m57rcdIC4EsmpKegycfmw24urSp/0bzjtvLlRV3at6VgUP6WrdHjLgB40naNJVHVMoDbjL6x5SAtWTgz+KutD9Q4ynYWMOekFOonFJyFmTJvVAUDKvnM+lWhWnndD7wtKqkqfOrq8eFploEW77aYzUfMJ4L79LWu56RxNkQNsbKma0aeyr7JOmzs6+NsBu9Cuk+ksZBSLzymw+CTGeQDSYl2UkxJX+33ti6csGPdw5/njfvLBnvZDl5mORM0zYp2OaXtYd5vO0Vi9wuEdpjOYoorozMg5VVNDvygJISfPdB4ybnBReUVf0yGcjMUnMb5iOkBSiWTV5ncYv7L4luGVpU+tXdf7ojlRV26N6VjUeAIU3P169BPTORprbG0wI88+tBQHbgt/d/lDkevTdoSZmoRnd50Ei0+czZg2SwCcYzqHKVZWbv9Np32ltKr0yfDSoXdU1eR32mw6EzXOwG2Y0GW/bjGdozHKagLtTGdItlrNWXdW6PFDb1vjRprOQo7Dw10nwdPZ42zGtFmjASwyncMxVK284MHF/da9ntVpz0cjJRUWdCRs7oh5930rOzXOClKNfrhpazBftcB0lGTZpu0/PCf4qPco8jN6XhOdUBRAx43Tpx4wHcSJOOITf+eaDuAoIq5gi7ZjVw2+cVRl2dMbP+13+exIVt5h07Ho5HruwYTeO/Uz0zkaIk91fSaVnveiIytLgs+MZumhk8gCcLrpEE7F4hN/LD4noK6c3lt7nFE2u+QJfDT87qojhd02mM5ExyeA697XovtM52iIonBkl+kMyaCKmofD18z/dvi+KQoXf3bTqfDyFSfABQzjaMa0WW0AjDedw/FEWlW36Tf5w9EPaG7o0Ed9N7wZ7bJz4WiB8oe5g3SqxrjBm6xVq05zDTad5WTG1dZapjMkWkRd278W+vGhJTpgoukslDJYfE6Av2ji60xkwmns8SIioTz3KL/nmrGVZU9v/aT/VVXh7Pxq07Hoc3e9YQVNZziVyTWB9qYzJNJBLVwxPvibnCU6wGM6C6WUgUXlFd1Nh3AiFp/4yqzT2ONIXdm9tncvnTxn0mM5i0beP+dQy55rTWciwF2DkeM+sZaaznFCquHhtcF+pmMkygqr95wxwd969qJNR9NZKCVx1Oc4WHzia4rpAClPpOBw66LSxaPL+8+Z+MtlW7uVfqCQqOlYmew7FVau6Qwnkq+6LhdIu0X7VBH538h5sy8K/aw0jGzHfv/J8c4yHcCJWHziZMa0WZ0A9DedI52Ec1sO/3TAVeMry57ZudpzTWUop3C/6UyZqCCEwWcutRx50cO+4fAe0xnizVLZf0v47pWPRK4pM52FUh5HfI6DxSd+SkwHSFfqyuq+s8v4KXMn/rLgw9E/mHuwdR+/6UyZ5ob3rXai6rhJxOMD6XWd3FrNWXtG6PGj/7LGjDCdhdJCt6LyCs4Nq4fFJ35KTQdIeyItjrTsUbJk5L3e2ZMeW7m5xxnzLXFl5KUKki03gn4XL9AFpnPUV1YTSJu5L5utjh+MCj7XdaN27Wk6C6WVKaYDOA2LT/xwxCeJIjkFxZ/1u3xiZdkz+z8edGNlMNedchfWTDVfm2P1zIqqc4qmanBIMJTyF2NUhb4dHVtVFnp63FHktzSdh9LOBNMBnIbr+MTBjGmzCgEMN50jI4mr8+5Oozrv7jgyVFCzc96AtX9ztzu4ZojpWOko20Kvqyut2S+emeWIuSeFqp/lAI5eY+hUVHH0x5HrV74YPYcXGaVEGWc6gNNwxCc+xoMl0iyR3JrCrpOWDb9zSFXJE6s39vrKXEuyHb8GTaqZukg9uWGtMZ0DAPqHwimxsvSJRNS19dLQw1tfjJ7DRU8pkQYUlVe0NR3CSVh84oOHuRwkmt1i0Po+F5VUlj11eMWQWyoDee12mM6ULlyKTjf90xlneE0MBFL259cBbblsbHBG/jLtN9B0Fkp7Ao76fEHK/uBwmNS4inWmEVeHvR2GTlkw/uGOC8b5FuxtP2S56UjpYMpKHV5Qq8ZX2C6tqe1sOkNTLLH6zRkTfHbwfrjTesVpchSOKtbB4hMfo00HoJMQyQ7kd5ywovg7wypLn/p0fdEFc6KunIDpWKlKgDa3/cPwas6qNd5QqI/RDI2kivBvIxfOuSz0cGkE2Tmm81BG4YhPHaKqpjOktBnTZp0GYKPpHNRIqgfbHfAvH7D25b4Fgb09TMdJNQocvfmOrJrqlmLkdPJWUWvl/M1bi03suykslb3fCt+7bZY1cpjpLJSR9gPosHH6VP7CB0d84oELjaUikTb72w2a/MFYX7d543/64e6OI5aYjpRKBCi8643oalP794RCKbOKd43mrpkSerKWpYcMagdggOkQTsEzkZqPxSeVibiCLdqN/Xjwt+Cywuu6b6va1nvj2yOzo0Gup3IKg7ZgQqeDum13G0n6FaAnBWpT4lDRRqvzgvNC04cFkFdgOgtlvDEA1pgO4QQc8Wk+Fp80Ybly+m7peVbZ7JInrCXD76o6WtBlo+lMTiZA7r2vRTeY2HdpTaCrif02lCr0zeiEyimhJ8ez9JBDpMyh4UTjiE/zceHCdCPS+mCb/pMXjnlQc0PVi/uu/7t22fXhaLFPC6U6inZhQq/dun5zJ0neRGPVw/3D4aKk7a+RVHHkgchNq/4cPXOK6SxEdXBh1xiO+DTDjGmz2gPgdXXSlYiE8tqM9nuvG1NZ9szmNf2vrApn5Rs/jdtJBMi679XormTus41lrXdqCQ1r1uaLQo/s+HP0TJ5FQ07DEZ8YFp/m4WGuDKGu7NO2dS+bPKfksezFI+6bfbhlz89MZ3KKzgcx3rNF/cnanzcYOpisfTXGPm21dGxwRsuV2qe/6SxEx9GzqLyitekQTsDi0zw8zJVpRAoPuXuXLRpd3m/OxOlLt3WdtFAhUdOxTBJA7n49ejRZ+yt14MTmhZanamzw2eIDaN3OdBaik+DhLrD4NFdKXyCRmiec22rEmoFXj6sse2aHf+A3K0M5hSlzinW8tT2K0aPWWsuSsa+SQMAx6y6pIvTryCVzrgz9eHIUWZwzSU7H4gNObm4uDmkT1JXVY0fXCT12dBkfaHlk65yBa1/u5D60IeOuwXT7W1bWDfck9r2UqB7sHY70SuhOGiiqsueG8Pd2zLaGlZrOQtRALD7giE9zcUEo+pxI/pFWPUs/GnnfwNmTHl2xpfuUBZa4IqZjJUthEMVTViT2AqbtLGt9IrffUEc1zz859HR4tjVsqOksRI3ACc5g8WmyGdNmuQEYWa6fnC+SUzh0bf+vTqgse2bvx94bKoM5rfaYzpQMN/3TaoMEXgdncDB0OFHbbqh1Vtf5o4LPFW3Vjt1MZyFqJE7PAItPc3C0h05NXF12dx49Zd7EX7gXjvnhvANt+q8yHSmR8iIYcMGHuiBR2y+tCeQlatunogrr1Whp5ZmhJybWIi/fVA6iZuhYVF6R8avSc45P03F+DzWcSO7Rwm6Tlg6/C1mRwKqiTf882HPrrDEujeaajhZvV1da3d8eIxHLJXH/+VISCBhZN0sVh+6P3PLJK9HJU0zsnyiOegNYaTqESRzxaTqO+FCTRLPzB6/re8mkyrKnq1cM/nZVbV6bnaYzxVO2hdOunG3FfdRHVPf0iESTfl2wkGZtOj/0iz2vRCePTfa+iRKgt+kApnHEp+k44kPNI66OezsOn7y3w7BIfmDPggGf/a2w/f7VaTFZ9qIPtN8rJVobzpYW8dpmx2h0E5I8r26Puj86K/hY32q0bJPM/RIlUMYXH474NB1HfCg+RLIDBZ0mLB9629DK0ifXrC86f07UlV1rOlZzZCm6Xv+etTCe2xwaDCVtkUQAmBcdXDUuOGM4Sw+lGRYf0wFSWJHpAJR+rKy8gRuLppZWlT5ds6z41qpAiw5bTWdqqjOXaXF+UON2FlZpTSApE4pVEXwyfMXcb4R/ONmCKysZ+yRKoowvPpLAM0/T1oxps/IApPQ7ckoRqtEWtfsX91v3am6nvctT7tpwCzxS+dSlWVPisa33Nm/b1SUa7RyPbZ1IVGX3teEf7J5nDeFCb5SuVm6cPjUtDqk3Fef4NA3X76DkEMmqzW8/7uMhN8MVDa/rsa1yW++Nb4/KskKFpqM1xPhPdFTro7rvUKG0b852XKo7u0SjXeKV63iOaIvV5wQfbbsdHVh6KJ1l/IhPgw91ich3G3JfhmDxoaSzsnL6bu51dllV6ZORJcPurDpa0HmT6UynIkCr7/7d+ri52+kcjW6OR54T+dTqPm9U8Lk+29GhayL3Q+QALYvKKzqYDmFSY+b4XHec+66PU45Uw+JD5oi4D7YdOHnhmB/1mjfhZ4t2dhq9WAHHHrMesknHdajWHc3ZxvDaYCBeeepShfWXyJTKc0KPTQoiN25noBE5XEb/DjvloS4R+TqAqwH0FpE36zzUCkCmXo066WuJEH2JiATz2oxZPegG+D3XbOi+fe6WPhveGp4drW1tOlpdArS497XoZz+4IbvJoyllNYG4H9pTRfXd4VvXvmGVTIn3tokcrpPpACY1ZI7PfAA7AHQA8ESd+w8DWJGIUCkgo9syOY+6sntv7TGl99buk4+4D62fPeDTl7u3Orqtr+lcx/TZiQk99ujGrR2lqCmvnxCobdLrTiSk2RsuDD2CNdprdDy3S5QiWHxORlU3AdgEYELi46QMFh9yJpGW1e6+ZYvGPICc0KElfde/Gem684PRAjW6dIUA2fe9Ft1x1y3ZRY19bZbqtvaWFbdR1l3aZvFZwcf6H0ahO17bJEoxGV18GjO5ebyILBKRIyISEpGoiByKPRYVkWV1buVNCSMiL4jIFad4zv+KyKDYxw80ZT9xwENd5Hjh3NYjP/F8c2xl2TPb/AOurgpnFxw0mafrfozvv03XNPp1kciWeGWoig6tGh/8zUiWHspwGV18GnM6+28AXAXgbwBGA7gWn69eHFDV4XHOdlyq+q06nz4A4OfJ2G89CT2tliie1JXVc0e3ST13dJ0YaHVk85yBn77cufXhTUlfeVwAuef16KHv3N64VTRG1QaDzd23Kmp/Gbnqo+eiF01u7raI0kBGF59GDX+r6mcAslQ1qqrPAzj3RM8VEbeIrBGRgbHP/ywi3459fEREnhKRVSLybxH50vV3RORMEVkqIitF5I8ikhe7v1JERovIdAD5sRGmP4lIoYhUiMhyEflYRK5szNfWSO0SuG2ixBDJP9zqtNLFo743YPakR5dv7V62wBJXJJkR2h/GmGHrrEbNDSyrCTRrsnZUZedVoQfXPxe9aFJztkOURlh8GqhGRHIBLBORR0Xk7jqvP1ZAjt2uVNVqALcDeEFErgLQVlX/J/b8QgCLVXUwgCoAD9XdkYi0APACgCtVtRj2yNR36j5HVcsRG2lS1W/ALmHbVXWYqg4B8G4jvrbGapvAbRMlXCSncNin/a+cUFn29J5VnuuqQjmt9iZr33e+aTXq1PtxtbV9mrqvw5q/alLw17JQBw1q6jaI0hCLTwNdE3v+7QCOAugJ4PLYY8cKyLHbywCgqu8BWAlgBoC6h6gsAC/HPn4JQEm9fQ0EsEFVP419PhNAQJ5hMgAAIABJREFU2SnyrQRwtoj8UkRKY8Ur7mZMm1UIICcR2yZKOsnquqvL2MlzJ/6i1cLRD8w94O63OtG7bFWLYSWrrMUNeW626ma3pU2aj+O3es0dFXyu3060S+hlLohSEItPA+0FEFLVQ6r6EwD3A9h+sheIiAuAF0ANTj5K0uzF12IlaSTsAvSIiPy4uds8AR7movQjkne0ZfeSpSPuHlRV8vjHm3qeNc8SVzhRu7v5HasQDbhQYI9wZFtjt62K6IuRs6rOC00vCSEnr2kJidJaG9MBTGpM8fk3gII6n+cDeP8Ur7kbgB/2AojPi8ixkRIXgGNnb10NYG69160BUCQi/WKfXwP7kFh94WPbFJFuAGpU9SUAj8EuQYnAs0EorUWz84es63vppMqyZw6sHPStqtrcNrvivY8WYXjP/Ug/ONXzRtfWhhqzXUtx8I7wHct+FLmRk5iJTiwlrvWXKI05vaKFqh459omqHhGRY0UoX0SW1XnuuwCeh314a6yqHhaR2QAehD2f5yiAsSLyIIDdAL4wEVlVa0XkBgB/E5FsAIsAPHecTL8HsEJElgD4PwCPiYgFIIx6c4LiqFWCtkvkLOLqtKfTiE57Og4PFwR2ze+/9pVW7Q/4i+O1+WtmWV3+NVKilkuyTvScyTW1DX5nGtScdReEfpa1VnuMik9CorSVW1Rekb1x+tSkntzgFNKA0Wb7iSLzANyhqktin48C8BtVbfTChiJyRFVbNvZ1TjBj2qyvILETp4kcKysa9Pfa/N6+XlveG51lRZp9batXJsncv5Zl1Z/jZ1O1FmzaerSl6infbGzXdh+eE3zUcwQFjrpcB5GDtdk4fWpC5sI6XWNGfO6CPQKzHYDAXssmkaeMOxVHfChjRbPyvBt6X4ANRefva79/1QcD1v51QH7t/iavZH7pfO3z2kQNRrLlS3NxcoGNLVVPeUbX+9ERld8O31umcBldnZooxRQCYPE5GVVdJCIe2GdcAcAaVf3v5EcROTt2FldDtpWSoz0xLD5E4mq/r33xlAXthkRb1O77oP9nr7bouG9FoxcxzVJ0u/bf1uw/fiXrS2dt9gqHdwA4YfFRReCRyDeX/iF6/pTG7peIMneeT6OWUI0VnY9P8PAvATSo+KQ4niVCdIxIVm1+h/Eri2+BKxpa+//Zu+/wOK7rfPzv2Q5gUYhCFIJgF3vvpNgWVK+ULcu2RLnk58SKYicOnIRxEnsTN9mJbMftp8Ryk0sMy5ZlCbYs25JIQhRJgCAJkmAHG5YAWACiA9vmfP+YpQSRINF2987unM/z4JGwOzvzQiIXZ++9c8943xvNE8/+YbFVC6QO/mLd7Xt51s83cFefg971gWhprz98o9eE2NL4/sC/te/h6atGE18IEzNt4RPNoWGK4rmMbHj77QthEprVMe3shDvWbFvzteC+eZ/Y1pMy9txQXmcBcv/yFa3m2sfX9fYOuAVGO6ceWOX/ln0PT5852sxCmJgUPlEw6r14EoQUPkLcDFHmlewZ63Yt+2zxjhVfqLqQt7iGB3l/WH2YF7h7+MrbDzCHF/f5p1573EFtYuUS/zMzLmLMdW1uhBDDIoWPGDLZtVmIoSCy+F1jltXN/ujirWu/ceb41PdsD1mdnQMeCmR+4uV3eng5mU+5mFOufs+M0A9Cd267L/ClNUHYHPGIL0SSSxn8kOQ05NELIiK+5t53InIy89XOyWeiGczAZMRHiGFii32Sr9gzyTduQ2dme/226SfKS9zdjZP6H7PgFC/L6eDmlgwqmBQMXQAwDQA0xpUngn935lVtmWxKKET03HD/rGQ3nBGf7/f/hojcAH5/9XtmfihaoQxOCh8hRooovT1r6rqqJZ+Z+ObKL9U0FayoYpAGAASkfOo34RMAsLyvTwOAPrafKA38V+er2rKFKmMLkYRMO+MznB/cR0TfBQAiGgPgj9AbjJqNTHUJMVpEFHBmLj4yY/OyrWu/4Ts67f3bgraU9mmNWFHYwufW9vTm+jh392L/M4WnuahEdVwhkpBpC5/h7OPzWSL6KhE9A2AxgKeY+dexi2ZYMuIjRBSxxVbSOG5NSWPRrT3pnWd3bd7+m46GeUWu9wU+6QHILHeLChFvUvjcCBH1n8LaDeDfAFQBYCJ6iJlfiFU4g5LCR4goYw72hAPHDl3GAcwat5GaFv6et2R+57gP49sbUNJ7HuNxCWOdHchM74NrLINyQVIUCTEKZrkT+zpD+SV+3zXf74M+3XMf9P9wZit8TPuHRYho0sKtZ8P+/WfCgeNucM8cAMuyHGPrZ7sXjtu1v1VbtOTlfdPSjq8e6LUB2Psucn7zeYxvbUBJjw/jQxdQaLuCbHcPUrPDsBaCSKalhbixG24QmuwGLXyY+SPxCJJA/IMfIoS4FnM4oAVPHgr7D3RqofMlgDYJwISrzxMo5Cl8NEBELgI17a25d8WSpS/tTEnpvK4RsgNBVzF8E4vhm7gcO6+7lgbSWjmnuQlFlxowobMBJcEmFFELclM6kZ4VhCMfRNLQVJiZpjqAKsO5nf3HAP6Wmdsi348B8DQzfzRW4QwqoDqAEImCtY6mkP/AyXDgqBNaxywAi2507MqxD7xptzjWA4AVlvYQUFiz5/4lS5f+psrp6lk2nOtawJZcXC7IxeWCuTgw4DHdnNbRjMLmBpS0NaCkrxHF2kXkO9uRldEHVy6Dxsp0mkhiMuIzBPOuFj0AwMxXiMiMt5jKiI8QN8CshbXQ2cNhf22LFjxXBIRuAVA42OvGukrqilNvufXq93bYukMIgNlir65+cP6y5S/UOBx9i6OZNQ3dGVNwMmMKTg74fBA2/0XOv9CI4pZzKOk+j5JwMwqtrchO60FaThjWAhDJZooiUUnhMwQWIhrDrG8rT0TZw3x9spDCR4h+WOtpCQcOHQ3764i1KzMBzB3O661k611b8HAqEb39fuJke18v6YOrzFZnddWmmcuW/7rWbg/Mj276G7Mj5ByH8yXjcL5kKXZf9zwDfIWzLzRi3GUfSjoaUOJvxDhLC3JdncjICujTaZnxyivEMMlU1xA8DWAnET0PvSHpewF8MSapjE2muoSpMTNz6PyxkL+2WQuezgMCMwEMuAh5KNbmP1xtJdva/o+lsCPUhu63v9c0W2p11UOTly3/9SGbLThn5OmjhwDKRmt+Nlrz5+DggMf0cGpHMwov+DC+LXJ3Gl9Evr0dWel9cOVpsIwFkWlvKxZKhVQHUGU4+/g8R0Q1ADZEHnqImQ/HJpahyYiPMB1mf0fYf+RwOHAwxOHL0wCeAWDGaM9bnDp9b55r/JprH0+D87q7J8Nhe3p11abxy5a/cNRqDY362vGQip6MyajPmIz6AZ8PwRa4xGObz6O4tQElXT6MDzWjyNaK7NQepGWHYCsEkTPOsYU5dKkOoMqwpqqYuY6ILgFwAQARlTDzuZgkMy4pfIQpaKGL9WH/fl84WJ8J7p0NYEU0z2+3ONtXjr2/kAZYQOxm14B9hEIhZ2Z11abQ0mUvnLRaw9d1b080NoQchWgsKURjyRJUXfc8A9zGWZeaMO5iA0o6ffp0Gi5hbEoX0jP9cOaDKEtBdJH4pPAZDBHdD326qwjARei3oR4BMDs20QxLprpEUmIO9mqB44dC/oO9HG6eDGhTAEyJ1fVKCx89ZCHLgFNkbnbdcNFwMOjKqa5+UFu27DenLRZt0o2OSwYE0Bi05Y1BW94s1A14TC+ndDaj4IIP49t8+p5G2kUUONqQ5e5DSp4GS75Mp4kBdMbipET0dQBnmfkbke9fBdDAzP9f5PunAZxn5q/F4vpDMZwRn89D/8T3Z2ZeSEQbADwWm1iG1qM6gBDRooWvNIT9tafDgWOp4O7ZAJbG47pT0xfuynTk3XBdUBq7Um72+mAgNW9P9QOhJUtfbLBYeHz0EyaOFPSmT8Lp9Ek4PeDzYVhCl3hs43kUX/ahpNuH8cEmFFlbkZPWDfeYEGwFILrpf2+RlGJS+ADYAeB9AL5BesGdC6D/nlmrAHxqsJMQkZWZY3Ln2XAKnyAztxCRhYgszPwGEX0jFqEMrm3wQ4QwJuZwUAueqgv7a9u10PnxQHgygLgWDi6r++KinNum3eyYNHamD3Yev99dWFNzv2/JkpeaiHjQW+bNygrNVoDm4gI0Fy/GngGPaeOsy00outjv7jRcRp6rAxkZATjzmSzZcY4tYi9WU11vAfh65N9nAzgEoDCy918PgJkAMoloH/QapBrAE8zsJ6IzAMoB3Abgq0T0FID/A3AX9MXYfwngywCmAvhPZn6GiAojr8mInO8JZq68WcDhFD5tROQGsB3Az4joItDvtgvzaFUdQIjhYK3zQsh/8IQWOGJjrX02gAUq89xWtPkMEd10Q8JUdo4Zyrn6ejOK99bce2bR4pcvEmFsdBKaTxbacrPQljsTA9+v0sfO7gsobPZh/JXI3WnhCyiwtyErvRepORosBSAacF2WMKRQ84YFfbE4MTM3ElGIiEqgj+7sBDAOwEoA7QBOAHgWQCkzHyei5wA8AeDqQEoLMy8CgEjhc46ZF0Sm0H4E/Q5SF/SC6hkAHwTwKjN/kfQ/g6mDZRxO4fMAgF7oQ1SPAsgE8O/DeH2ykMJHGBoza1roXF3Yv79VC57LB4LTAeSrzgUAc7LWvJlqy7h1sOOcsGWCEQIN/h7V05M1cd++u08uXPh7KxFyopNU9OeCP20CzkyZgDMDPh+GJXSZ8843YtzlBkzo8mF8qAlFllbkpnQhLTsEewGIBv2FJOKmPcbnfwt60bMKwNegFz6rItf1Aehl5uORY38M4Em8U/iUX3OulyL/PAjAzcydADqJyE/6wv5qAD8gvTffi8y8f7Bwwyl8PsvM/wR906MfAwARfQXAPw3jHMngiuoAQlyLtd4r4UDdkbD/ELPWOuxNBOPBbRvTMCtr5ZA2ICS9V0QrY2ijON1dOVNra+84On/+qzYiyKaBcWaFZsvHhXH5uDBuIfYOeEwHZ7Q0Ytyl8xjfdg4l/kYU4xLynJ3IzPTDmcdkyY1zbDOL9Qf4HdALnbnQR2YaAJQB6ACwFcB7bvLaa2eSrt5JreHdd1VrAGzMvJ2I1gK4B8CPiOhrzPzczcINp/C5DdcXOXcN8FhSe/IZT/A7H3+9C4BbdRZhblqoMbKJ4KlssH8W9Dcao9I2Fm1uJaIhryeywNIehjbk6avOjrEzDh7cWDd37p8tRBh0jZCIrwx05GSgI2cGjgz4vJ+dPRdQcCGyp1GPD+O1CyiwtSHb3YOUXA3WfPTb3VuMSqwLn7cAfBrAqcgC5dbI6MxsAJ8AUEZEU5n5JIDNALaN9EJENAGAj5m/R/qeV4sAjK7wIaInAPw1gClE1L/bXzr0H86MWiGFj4gz5kBXOHCkLuw/FODwxWkATwcwXXWuoViSe2el05qybjivscPaFR7mrvrtbYWz6+o21M6e/cY0osHn+oVxOOFPLcHZSSU4O2kldlz3vAZLuIVzGs+juCWyCDtyd1puShfcY4L6dFqaguiJKNaFz0Hod3P9/JrH3MzsI6KPAHg+0qamGvpanZFaD+AfiCgIfcH244O9gJiv2yD13QfovWbGQF9JvaXfU53MbMr1Lt/5+Ov7oHiBqDAHLXTpdNhfey4cPJEO7p0DIOGaYo5xFJy4rejxEhrmDsTPO3a+1W7pGdEoVm7u2b0zZm6fRaRvtirMoRPpV5pQdMGH8R0NKOnVp9PGOtqRmemHa6xMp73tZ80bFphxOxoAQxjxYeZ2AO1EFGLms/2fI6KfMPPmmKUzLlnnI2KCOdSnBU4cCgUO9HCoaQKgTQKQsJv0WWAJeAo/qA236AEAF+zBka7AvHx5wqLjx1ZV3zL9rQVEsI/wNCLBpKNzTDqOjbkFxwZ8PsD2vgsoaDqP8Vd8GN/tQ0m4GQX2K8hO60VqThjWAuiLZJOdqX+HDWe+9F07NEeGqBZHN07CaFEdQCQPLdx+PuyvPRUOHHWCu+YAWKI6U7Ssyn9wp81iH9YU11VpfH2/ruG4eHHKUqs1tGvK1KqlRJBbrQUcCLrGo2HSeDQM+GFCA2mtnNOk351W0ulDSaAJ4yyX9em0rCAcBSBKhvVjl1UHUGkoa3z+GcBnAKQQUUe/p4IA/jdWwQzuvOoAInExayEtePqQvolgwzggPBX67Z5JJd818WBRytRBb12/kTR2jbrNQlPT9BVWa3DHxEn7VhJB2jaIm7KALbm4XJiLy4XzUDvgMV3sbm/W9zRqj+xphIsY6+hAVkYfXHkMysMA/ecMxtS/w4Yy1fVlAF8moi8D+CqAW4C3581H9YksgflUBxCJhbWuS2H/wWPhwBEba20zkeRrxKxk715T8N4MGsWmdm52RWXKweebs9pqDVWOLzl4KxGM/gtJGJwbXZlTcSJzKk4M+HwQNv9Fzm8+j/EtkbvTQs0otF1BtrsHaWPCsBaCSPVaPVP/DhvOVNcp6Ls2FwPYD71v104AnhjkMroG1QGEsembCDYcCftrL2vBM3lAcCaAPNW54mV9wSN7rWRdM5pzuAfp1zUcZ88uWGOxhrYXFx9ZG61zCjEQO0LOcTg/YRzOT1iGXdc9zwC3cs6FJhRdasCEq3en0WXkubqQPiYAx1joNxXFkhQ+Q/RJ6A0MdzHzBiKaAeBLsYlleFL4iOuw1tceDtTVhf11zNrl6bhmXZxZlKTN2pPrGjeqogcA0tgV1bUUp08tWWu1BrcVFp4c0ZojIaKBAMpBS34OWvLn4OCAx3Rzanszii5GWoT0NaJYu4B8R7s+nZbLsOSPcjpNCp8h6mPmPiICETmZ+SgRJcQeIjFg6j804h1aqOlE2F/bGA6eygL3zYGxNxGMOYfFdWV53j3F0ThXGjuj/qn35ImV62zW4Na8sWfXR/vcQkRLGnoyp+Bk5hScHPD5EGyBizy2uRHFLecwoduH8eELKLS0IietG6k5YdgKcOM7KbuaNywwdbPt4RQ+vsjOiy8C+BMRXQFwdpDXJKtG6Ntly2JJk2EOdIcDx+rC/oN+Dl+YAvA0ADftNG4mpYWPHbWQZWU0zuWCPRsMDVFelHz06Nr1FuvrW3Nyzq+P5nmFiBcbQo4iNJYUobFkCaque54BbuMxFxsx7lJks8dAI8bRZeS5+pDSMcApTWXQDQwHfBHROuhNSv/AzIGop0oA3/n46z4k4Z044npauOVs2F97Jhw4ng7umQ1g2HvSmMEtGUt2LswpjUrRc9WzztdaEKPGo3Pn/XFbVtYFmfYSZvNqqaf+TtUhVBpR3xNmHnFfjSQihU+SYg75tWB9Xdhf26mFGicC2gQAE1TnMrIUa3rzgmzPjGif1wpLWxhaTAqfgwduX7dgwe8r0zNaRr0eSYgEclp1ANWk4dvINQBYrjqEiA4OdzSFAgdOhgNHnNA6Z0NvdCeG6LaizT4iivrGizZYht2vazj277/r1kWLKt5Mc7eNeL8hIRLMGdUBVJPCZ+QGXnUmEgKzFtZCZ+rC/tpWLdgwDghNA1CoOlcimj9mfWWKLT0moyYOtvf6KRSLU0cQ7d17z8olS17amZLaGdVpOiEMSkZ8VAdIYEdVBxDDw1r35bD/0LFw4LCFtSszAcxTnSnRpdtzzk7PXBaz0bEU2AOd6I3V6SMs1pqa+5csWfpilcvVvSzGFxNCtVOqA6gmhc/ISeFjcMzMHDp/NOTff0HfRDAwE8Bq1bmSBYHCGwsf6yCimK1/SmVn7Oa5+mG22PdUPzB/2fIXahyOPrP2IBTmYPrfXVL4jNzA7X+FUvomgkeOhAOHQhy+PB3gmQBmqs6VjJbl3v2mw+qK6V1RaeyKW4sJZquzumrTrGXLX9hvt/uTuqWIMK2GUk99l+oQqsk+NCP05DOeNgAXVOcQgBa6UB/s/uPWvrbv1vrbv5sW6n1jBYcv3QqwaVpExFuOs+jYBPfsmG/WGK1+XUOlabaU6qpNU0Mh+8Bb6gqR2A6rDmAEMuIzOkcB5KsOYTbMwZ5w4Hhd2H+gl8PNkwGeAmCK6lxmYYHVv77g/RYiinlR4maXa/Cjoisctrurdj9UsnzFr49YrSEZLRTJRAofSOEzWkcByAZocaCFW89FNhFMBXfPgd43Tihwa/5DO20W+/p4XCuNne54XOda4bAjs2r3Jm3Z8hdOWK1h2ZlbJIsjqgMYgRQ+o2P6RWKxwhwOasH6g2H/gU4tdL4ECE8CUKI6l9kVpkyuLUiZFLcO57Ho1zVUoZBrTHXVptCy5S+ctli0SapyCBFFMuIDKXxGSwqfKGKtsznkP3BSCxyxsdYhmwgajI3sXbfmP5RNRHFbG5gCR3a8rjWQYDAlb0/1g6ElS39zzmJhKbxFopPCB1L4jJYsgBwFZta00NnIJoJnC4DQdAAFqnOJgW0o/MA+C1nj2t7BAosdjHYQlI38+P1phTV7HvAtWfrbRiIuUpVDiFFqLvXUX1Edwgjkrq5RePIZz3nInV3DwlpPa6iv+i1/+492+Nu+3hbsemGuFqxfFyl6hEFNdM+pynYWKulpZQEpf7Pu60svrtlzX5CZ5O+7SFQy2hMhIz6jVwPgbtUhjEwLNUY2ETyVDQ7MBhDz26BF9DgtqS3Lcu9StsbFBmtnALFsWzE0vb2ZE/btvbt+4aLfWYmQqzqPEMNUozqAUUjhM3p7IIXPuzD7O8OBo3Vh/8EQhy9NBXgGgKh37hbxsbHosRNElhWqru+ArccIhQ8AdHdnT6ndf+ex+Qv+YCNCluo8QgxDteoARiGFz+hJFQ1AC108FfbXNoSDJzPAvXMAKPtFKaJnRubyHW77GKVtPlxsD3RRn8oI79LZmTf94IHbDs+d9ycLETJU5xFiiKTwiZDCZ/RMWfgwh/q0wIlDoUBtD4eaJwHaZACTVecS0ZNqy2iaN2bdHOU52BkGOlXHeJf29oJZdYc8B2bPeX0KEdJU5xFiEJdKPfVnVIcwClncPEpmWuCshdt8wZ5t2/va/rfa3/ZNDva8soRDjWsBbbzqbCLq+Laix5uISNndVFelsTNu/bqG48qVcfOOHF53nBnGGY4SYmAy2tOPjPhER1IucNY3ETxdF/bXtmshXzEQngKgWHUuEXsLs0u3u6xphtiV3M0uw75PtbSULDx2bPWe6dN3zCOCQ3UeIW5ACp9+DPuGkmCSZoEza10Xw/6Dx8OBwzbW2mcBkC7VJpNpzz09LWPxMtU5rnKzy6k6w81cujh5idUa2jV16u4lRPKeKgxJCp9+5C9pdOxWHWCk9E0Ezx0J+2sva8GzY4HgDABjVecSahAoVFr0WC8RpajOclUauwy/hqa56ZYVVkvorUmTa1YQyRICYThVqgMYiRQ+0fEmAA0JsmaKtd62cODw4bD/ELPWMh3AbNWZhDGsyLtvh93iNMQU11VpUNevazjOn5+1ymINvTlhQu1qIhhyXZIwpbOlnvpLqkMYiRQ+UfDkM56O73z89f0wcG8pLdR0POSvbdSC9Tlg/yzIJoLiGrnO4iPj02YovXV9ICmstl/XcDScm3erzRrcXjz+cNwauQoxiErVAYxGCp/o2Q4DFT7Mga5w4OjhsP+gn8MXpwJ8C4BbVOcSxmQha9/6gkccRGS49wQbrC4wukBwq84yFKdPL15rtQa3FRadMNTImTCtN1QHMBrDvcklsG0A/k5lAC18+UzYX3s2HDiRDu6ZA8AwC1SFsa3Nf3i31WIz7C9qAl1hcEIUPgBw8uSKdVZbcNvYsWcM+99UmIYUPteQwid6KgEwEL+5feaQXwuePBTyH+jiUONEQJsIYGK8ri+Sw7jUafvGukoMPTVjg6UjiLDqGMNy7OiadVZraGtOjm+96izCtM6WeupPqw5hNAmxGDcRPPmMpwVAXayvo4XbG4M9ldv72r9X5W/7ZijY/fvFHPKtA7QJsb62SD52cnSsGvvgWCIy9GJcB2w9qjOMxOG6DevbrhRsU51DmNZW1QGMSEZ8oms7gKhu8c+shbTgmcNhf22rFmoYB4SmASiK5jWEeXkKHz1oIYvhFjRfy8l2fzf5VccYkYMHb1s3f8Er2zMyLht6VE0kJZnmGoAUPtG1DcBfj/YkrHVfimwiaGGtbRaAeaOPJsS7TU6fvzvLOdbwRQ8ApLIj1Ko6xCjU7r9zzcJFv3vT7b5yq+oswlS2qg5gRFL4RNdWjGCdDzMzh3xHQv7aS1rwTB4QmAkgLxYBhQAAlzXt0pKcO6aozjFUaexSHWGUiPbtvWfV4iUvvZWa2iFbSYh4OF3qqT+rOoQRyRqfKHryGc9FAHuHcixrfe2hvr07/R3Pvelv+/rlQNfzs7Tg8XVAYBbiuEBamNPGos2niShXdY6hSjNwv66hI0vNnvuW9fWlJexO7yKhyDTXDSTBm4nh/AHA4oGe0EIXToT9+xvDwfoscN9sACvjG00IYHbWqjfTbJkJNeVi9H5dQ2ex7al+YOHSZb/Z43T2LlGdRiS1V1QHMCopfKLvFQD/AgDMwZ5w4Fhd2H+wl8PNUwCeBmCa2njCzNJsWb7ZWbcm3JoxN5ypqjNEC7PVUV394Jxly17Y53D4F6rOI5JSEMCrqkMYlRQ+0bcr2Fv5h7D/cC64ey6ApaoDCRHBtxVtvkxExaqDDFcauzJUZ4gm1myu6qpNtyxb/sIBuz2QcIWoMLxtpZ76TtUhjErW+ETZk894wuG+6jZw9xIASTI8L5LB4pzbtzutqQtU5xiJVHaMUZ0h2jTNnlZdtWliKGQ7rDqLSDoVqgMYmRQ+sSF/6IShZDnG1k9JX7BcdY6RssPmBqNPdY5oC4cdGdVVmwrDYetx1VlEUnlZdQAjk8InNl4BEmx/fZG0CJagp/DRABEl9D3hBCTyVj43FAq5xlRXbcrWNEu96iwiKRwu9dSfUh3CyKTwiYGy8opWADtU5xACAFaNfeAtu8UxU3WO0bLC2q46Q6wEgymMBoZsAAAgAElEQVS51VWb3JpmkX1XxGjJjMMgpPCJnRdVBxBirKukblzqtIS6df1G7LB2q84QS4FAan7NnvttmkY+1VlEQpNprkFI4RM7z0PfxVkIJaxk61lb8HAqEVlVZ4kGF9sTs1nXMPT1pY/bW3NfmJmaVWcRCakFwE7VIYxOCp8YKSuv8EH+AAqF1hW8b4+VbJNU54iWFHaEVGeIh97ezAn79t7Tw4xLqrOIhPNCqade1pcOQgqf2CpXHUCY0/i0GTW5zuI1qnNEUyqcphlB7e4eM3n//ruuMOOK6iwiofyf6gCJQAqf2PoVAE11CGEudouzfUXefUVElFQ939zsSoopu6Hq6sy95UDt7c3MSNpF3SKqmgBsUx0iEUjhE0Nl5RWNAN5UnUOYS2nhY4csZClUnSPa3OxyqM4Qbx0d+TMPHSo9x4wu1VmE4f2y1FMvH7SHQAqf2Pul6gDCPKZlLNqZ6chdrTpHLLjZlTT9uoaj7UrR3MOH159kRq/qLMLQfqE6QKKQwif2ZLpLxIXL6r64MHvjdNU5YiWNnemqM6jS2jJ+wbGjt9YxI6A6izCk06We+l2qQyQKKXxirKy84gJk3lXEwW1Fj58lomzVOWIllZ1J169rOC5dmrTk5IkV+5hhirvbxLDIjTTDIIVPfPxEdQCR3OaOWVuZaktfqjpHLDlhzwQjqDqHSs3N05afql9SzSwtccS7yN1cwyCFT3z8EkCn6hAiOaXbxpybmblioeoc8ZCs/bqGo7Fx5sqzZxbsZJYNUgUAoK7UU39AdYhEIoVPHJSVV3RDhiJFbGgbiza3EZFbdZB4sMAit3YDaGiYe6uvYXal6hzCEJ5VHSDRSOETPz9QHUAkn6W5d213WFPmqc4RL3ZY5bbuiDNnFq1tPD9d1g+amx+ylGLYpPCJk7Lyip0AjqjOIZLHGEfBiUnuuatU54gnJ9v7VGcwkvr6ZesuXJi8VXUOocwLpZ76FtUhEo0UPvH1fdUBRHKwwBLwFH5QIyJTbeqXAoepFzcP5Pix1esvXx6/VXUOocT3VAdIRFL4xNdzgLnvShHRsTp/01s2iz1p9+y5kVQ2T7+u4ThyeP36K62FMu1lLicBbFUdIhFJ4RNHZeUVlwC8rDqHSGwFKZMOFKZMWas6hwpudsl71g0cOrRxXXt73nbVOUTcPFvqqZcPAiMgbyLxJyvwxYhZyd69Jv89mURkyr+7ZuzXNRwHau9Y09WZLXd7Jb8ggB+qDpGoTPnmqdgfoA9RCjFsGwrev9dC1gmqc6jiZleK6gzGRrRv392re7ozd6hOImLqpVJP/UXVIRKVFD5xVlZewQC+qTqHSDwT0mbtyXEVrVGdQ6U0dppiv6LRIUtNzb3Le3vd0rspef3/qgMkMil81PghANmITQyZw+K6sjzvnvGqc6iWys4s1RkSg8VWs+eBxX5/arXqJCLqaks99a+pDpHIpPBRoKy8oguyoaEYho1Fm48SWfJV51DNBfsYMDTVORIBs8VeXf3A3EDAtVd1FhFVX1cdINFJ4aPOtwB5AxeDm56x9K10e/ZK1TmMgEAWSL+uIWPN5qqu2jQjGHTUqs4ioqIZ0pB01KTwUaSsvOI0gJdU5xDGlmpNb5qfvWGW6hxGYpV+XcOiabbU6qqHJodC9jrVWcSofafUUx9QHSLRSeGj1jdUBxDGtrHo8UYiknUt/dhg7VSdIdGEw/b06qpN48Jh6zHVWcSI9UIWNUeFFD4KlZVXbAOwT3UOYUzzszdsT7G5F6vOYTROtvWqzpCIQiFnVnXVQ7maZqlXnUWMyHPSlys6pPBR779UBxDGk2HPOTM9Y+kS1TmMyCX9ukYsGHTlVFdtStc0yxnVWcSwMGRRc9RI4aNeOWRDQ9EPgcKlRY91EVGq6ixGlMoOuSlgFAKB1LF7qh9waBr5VGcRQ/a7Uk+9TFNGiRQ+ipWVV4QBfEl1DmEcy/LuedNhcc1RncOo3Owi1RkSnd/vLtpbc5/GTE2qs4gh+aLqAMlECh9j+AmAM6pDCPVynEXHJqTNWqU6h5G52WVXnSEZ9PZmluytuaePGZdUZxE39edST73swh1FUvgYQFl5RQjAU6pzCLUssPo3FLzfSkTyi/0mpF9X9PT0jJm0f9/dbcyyN5KBfV51gGQjhY9x/BCAzLmb2Jr89+yyWuxTVecwujR2Sb+uKOrqypl2oPaOC8zSRseAtpd66rerDpFspPAxiLLyigCAr6rOIdQoTJlSm58y0dQNSIcqlZ2ZqjMkm46OsTMPHdzYwIwu1VnEu3hVB0hGUvgYy/egb0kuTMRGjs5b8zflEJH8fRyCFNizwWDVOZJNW1vhnMN1G+qZIfskGcPWUk/9G6pDJCN5ozWQsvKKPsioj+lsKPxArYWsxapzJAoLLDZApmViobW1eP7RI2sOM8OvOovA51QHSFZS+BjPdwGcUx1CxMck99yqbGfBrapzJBoL6IrqDMnq8uWJi48fX1nLDNkoUp3XZG1P7EjhYzBl5RV+AP+mOoeIPacltWVp7p2TVOdIRNKvK7YuXpi6rL5+6R5mhFVnMSEG8BnVIZKZFD7G9FMAB1SHELG1sWjzCSJLnuocicgh/bpirqlxxsozZxbuYobslB1f5aWe+qpYnZyIwkS0n4hqiWgvEY1o3zAi+hERvXeQY9YTUcXIksaOFD4GVFZeoQHYojqHiJ2ZmSt2uO1ZK1TnSFQu2AOqM5iBr2HO6oaGOTtYFpPHSwCxH+3pZeYFzDwfwD8D+HKMrzdqRGSL5vmk8DGosvKKVwDIiv4klGrLbJw7Zq20pBiFVHbKFEycnD2zcE1j4wxZbxIf3y711J+O4/UyAFwBACJyE9FrkVGgg0T0wNWDiOhxIjoQGSX6ybUnIaLPR0aArER0JxEdJaK9AB7qd0w2Eb0YOc8uIpo3yONeIvoJEe2A3t0gaqJaRYmo+0cAVQCkN1Hy4NuKHr9ARAtVB0lkadKvK65O1S9dZ7UGtxYU1K9XnSWJXQHwhThcJ4WI9gNwASgE4Ik83gdgEzN3EFEugF1E9BKAWQD+FcAqZr5MRNn9T0ZE/wkgHcBHADihb8vigd58u7zfof8OYB8zP0hEHgDPAVhwk8cRufatzBzVqW0Z8TGwsvKKPQCeV51DRM+i7I2VLmuqFD2j5GaXfGiLsxPHV62/dKlkq+ocSewLpZ76eNyteHWqawaAOwE8R0QE/QP2l4joAIA/AxgHIB96EfM8M18GAGbu397k3wBkMvPHmZkBzABwmplPRL7/ab9jb0Vk5IaZXweQQ0QZN3kcAF6KdtEDSOGTCD4Dfd5XJLhMe96pqRmLlqrOkQzc7HKpzmBGR4+sW9/aWrRVdY4kdArAt+N9UWbeCSAXQB6ARyP/XMzMCwBcgD4qdDPVABZfOwoURd2xOKkUPgZXVl5RD+DrqnOI0SFQqLTo0T4ikgabUeBmZ5rqDGZVd6h0fXvb2G2qcySZz5R66uP+AZeIZgCwAmgBkAngIjMHiWgDgAmRw14H8DAR5URe07/I+QP0Btu/I6J0AEcBTCSiKZHnP9Dv2EroxRWIaD2Ay8zccZPHY0YKn8TweQANqkOIkVuRd9+bdotzluocyUL6dal14MDtazs7cypV50gSOwD8Mo7XS4nczr4f+hqcDzFzGMDPACwhooMAHodexICZ6wB8EcA2IqoF8LX+J2Pm56Gv63kJ+nTZX0IvhPYCuNjvUC/00aED0IulDw3yeMyQPg0njO7pR+59GPH9yyGiJM9VfHhDwQdvifYtmWYWhub/oesNp+oc5sbaosUv70xLa1+tOkkCCwFYVOqpP6g6iJnIiE+CKCuveB76gjORQCxk7VtX8IhLip7ossLiBEN2b1aKLHtr7l3R25u+U3WSBPZNKXriTwqfxPIJQPrnJJJ1+Q/vtpJtsuocyUj6dRmBxVqz5/4lfX2pMdtpOImdhzQiVUIKnwRSVl5xFLLQOWGMS71lX56rZK3qHMnKCouM+BgAs8W+p/rB+YGAq0Z1lgTzqVJPfZfqEGYkhU/i+TwAn+oQ4ubsFmf7qrEP5Ef2xxAx4IAtJre6iuFjtjqrqzbNDAYdtaqzJIhXSz31skebIlL4JJiy8oouAH+vOoe4OU/hBw9ZyFKkOkcyc7H06zISTbOlVlc9NCUUsh9SncXg/AD+RnUIM5PCJwFFFjr/VnUOMbAp6Qt2ZTnGyp0uMZbCzpDqDOLdwmG7u2r3Q+PDYdsR1VkM7CulnvqTqkOYmRQ+iesJRJrLCeNwWdMuLc65fZrqHGaQxk6ZRjSgcNiRWbV7U344bD2hOosBHUUCdENPdlL4JKiy8oomAJ9SnUO828aizaev7nAqYsvNLqvqDGJgoZAru7r6wSxNs8Sz07jRhQF8uNRT36c6iNlJ4ZPAysorfgzgFdU5hG521urKNFvmMtU5zMLNLtnA0MCCgdS8PdUPuDSNZNd53X+Veup3qw4hpPBJBn8JIKZ9TcTg3LYs3+ys1QtU5zATN7ukX5fB+f3uwpo99xMzNarOothhyJ49hiGFT4IrK6/wAfi06hwmp20serwl0qRPxEkqnBmqM4jB9fVlFO+tuTfA/K6+TWZydYrLrzqI0EnhkwTKyiu+B2lnoczinDsqndaU+apzmE0qO8aoziCGpqcna+K+fXd3MKNFdRYFvlrqqa9WHUK8Qwqf5PExyJRX3I1x5J+ckj5/heocZmSHLQ2MXtU5xNB0d+VMrd1/52VmtKvOEkeHoHcfFwYihU+SKCuvOAO9l5eIE4Il6Cn8YIiIZJGtIgRqVZ1BDF1nZ970gwc3+tgcDWZD0Ke4ZKNNg5HCJ4mUlVc8B+AXqnOYxaqxD7xlszhmqM5hZlZYZJQzwbS3Fc6uq9twihk9qrPE2GdLPfXSv8yApPBJPk8AOKc6RLLLd004NC512q2qc5idA1bp15WArrQWzz9yZO1RZiTrnjZ/AvCU6hBiYFL4JJmy8oo2AI9Bv5NAxICVbD1rCh52E5FsoKeYk+1yp0yCark8YdHxY6sOMiOoOkuUXQTweKmnnlUHEQOTwicJlZVXVELv4i5iYF3BI3usZJ2oOocAUtgh/boS2MWLU5bWn1xWw4xk+f/I0IueZtVBxI1J4ZO8Pg9gm+oQyWZ82oyaXOe4NapzCF0aXPKpOsE1NU1fcfr0ot3M0FRniYL/KvXUv6o6hLg5KXySVFl5hQbgUcCU+2bEhMPialuRd18REUlzTINws1OmG5PAed/s1efOzdvBjEQuZKsA/IvqEGJwUvgksbLyivPQi59k+CSlXGnho4ctZClUnUO8w80pspVAkjh3dv6a8+dnVqrOMULtAN5f6qlPtvVKSUkKnyRXVl7xKoDPqs6R6KZlLN6Z4chdpTqHeLc0dqaoziCi5/SpJWubmqYm4hT9X5R66qUTfYKQwsccvgTgRdUhElWK1X1hYXap7NdjQGnskn5dSebkiZXrLl6cuFV1jmH4cqmn/teqQ4ihk8LHBMrKKxjAhwAcU50lEW0seryBiKQvlAGlsiNLdQYRfceOrlnf0jJuq+ocQ/AKgH9VHUIMj6kLHyLKJ6KfE9EpIqohop1EtCnG1/w4ET0ey2sMpKy8ogPAJgBd8b52Ips3Zl1lqi19ieocYmBO2DPBkJYASehwnWd9W1u+kae9TgL4YKmnXtZQJhjTFj6RO3NeBLCdmScz82IA7wdQHMNr2pj5GWZ+LlbXuJmy8oojAD6s4tqJKN2efXZG5vJFqnOImyNA+nUlqYMHbl/X0ZG7XXWOAXQBeLDUU9+mOogYPtMWPgA8AALM/MzVB5j5LDN/i4isRPSfRFRNRAeI6K8AgIjWE9FWIvoVER0lop9dvbWZiBYT0bbIyNGrRFQYeXwrEX2DiPYA+Fsi8hLRpyPPTSWiPxNRLRHtJaIpsf6hy8orfg3gq7G+ThLQNhZubieiNNVBxM1ZYTFTt2/Tqd1/55rurqw3VefohwF8qNRTX6c6iBgZMxc+swHsvcFzfwGgnZmXAlgK4GNENCny3EIAfwdgFoDJAFYTkR3AtwC8NzJy9AMAX+x3PgczL2Hmp6+5zs8AfIeZ5wNYBaApCj/XUHwGwB/idK2EtCz37kqH1TVPdQ4xOJv060pyRHv33rOytyd9p+okEV8q9dS/oDqEGDkzFz7vQkTfiYy8VAO4HcDjRLQfwG4AOQCmRQ6tYmYfM2sA9gOYCGA6gDkA/hR5zb/i3VNm5QNcLx3AOGb+DQAwcx8zx6VbcVl5RRjA+wAciMf1Ek22o/D4RPeclapziKFxsj1ZG12Kt1mse/bcv7SvL2234iAVkO1BEp6ZC586AG+v32DmJwGUAsgDQAA+wcwLIl+TmPmPkUP7N0UMA7BFjq/rd/xcZr6933GG+0RaVl7RCeBexG+UKSFYYAlsKPwAiMihOosYmhQ4ZNM4U7DY9lQ/sCDgT9mjKEAN9E0KZTFzgjNz4fM6ABcRPdHvsdTIP18F8ERkCgtEdMsgaz2OAcgjopWR4+1ENPtmF2fmTgA+Inow8honEaXe7DXRVlZe0QC9+DFcYabK6vyHdtos9ltU5xBDl8bORG5zIIaB2eqsrn5wdjDg3BfnS58FcG+pp17eK5OAaQsfZmYADwJYR0SniagKwI8B/BOAZwEcBrCXiA4B+B/oIzs3OlcAwHsBfIWIaqFPgQ1ll9/NAD5JRAcAvAWgYBQ/0oiUlVfsBfABSFsLFKRMOlCYMlkakCaYNHaZ9n3MjDTNllJdvWlaKGQ/GKdLtgG4SzquJw/Sf/8Ls3v6kXs/CeC/VedQxUb2rk0T/rbVQtYS1VnE8By2+na9ZT+2QnUOEV9Wa6B9+YpfN1qtoZkxvEwAwO2lnnoj7yckhkk+KQkAQFl5xTcBfFN1DlXWF75/nxQ9icnNLunXZULhsCOzavemgnDYeiJGl2AAH5aiJ/lI4SP6+xSA36gOEW8T3LOrc5xFMsWVoNLYma46g1AjFHKNqa7alKVpllMxOP2/lHrq/y8G5xWKSeEj3lZWXqFBX+/zZ9VZ4sVhSWldnnv3BNU5xMilslP6dZlYMJiSV131YKqmWc5G8bTPlHrqvxzF8wkDkcJHvEtZeYUf+qJvo2wWFlMbix47RmQZqzqHGDkX7GPACKvOIdQJBNIKavbcb2Wm81E43c8APBmF8wiDksJHXKesvKIbwN1I8g0OZ2Queyvdni0bFSY4AhGkX5fp9fWlF9fsuS/ETKO5++pF6Ot6TH+XazKTwkcMqKy8og36DtaxWjioVKo1vWnemPU33WtJJA7p1yUAoLc3c8K+vXd3M+PyCF7+KoBHSj31oWjnEsYihY+4obLyigsANgJoUJ0lynhj0YcaiShTdRARHTZYu1RnEMbQ3Z09pXb/nS3MGE7n9O0ANpV66gOxyiWMQwofcVNl5RXnANwG4KLqLNGyIHtDZYotbbHqHCJ6nGzrVZ1BGEdnZ970Awdub2RGxxAOr4a+K7P8GTIJKXzEoMrKK45BL34uqc4yWhn23NO3ZCxdojqHiC6X9OsS1+hoz59Vd8hzhvmmLXkOALij1FPfGa9cQj0pfMSQlJVXHACwAcAF1VlGikDhjUWP9cS7J5qIvTR2ymJUcZ0rV8bNO3J43XFm9A3w9GHouzJfiXcuoZYUPmLIysor6gCsA9CoOstILM+7t9JuccqC5iSUxk55LxMDamkpWXjs2OpDzOi/fucAgPWlnvqE/SAnRk7eLMSwRKa91gHwqc4yHLnOcUdK0mauVp1DxIabXXbVGYRxXbo4ecnJE8v3MSMEYB8AT6mnPuGn7sXISOEjhq2svOIk9OInmjulxoyFrH3rC97vICL55Zik3Oxyqc4gjK25+Zblp08tLgdQWuqpb1GdR6gjhY8YkbLyilPQi59Y9MiJqjX5791ttdimqM4hYieNXW7VGYThbT9/ftYTsqZHSOEjRqysvOIs9OLnuOosN1KUOnV/vmuCNCBNcmnSr0vc3KsA7vR6vXL3lpDCR4xOWXmFD8BqAFWqs1zLRo7O1WMfzCUi+XOe5CL9ulh1DmFILwC43+v1yj49AoAUPiIKysorLgPwAPi96iz9eQo/WGsha7HqHCL2LLDYgGHt1CvM4bsA3uf1emVHZvE2YpYPSSI6nn7kXhuA7wH4sOIomOSeV7Us765lqnOI+PmB8/XTGvEk1TmEITCAz3i93qdUBxHGIyM+ImrKyitCZeUVHwHwZZU5nJbUy0tz75BfgCZjg1XWbwgACADYLEWPuBEpfETUlZVXfAbAJwAo2U33tqLN9USWPBXXFuo42NajOoNQrh3AXV6v92eqgwjjksJHxERZecW3ATwCwB/P687KXLkjzZ61PJ7XFMaQIv26zM4HYI3X631ddRBhbFL4iJgpK6/4FYD1AJrjcb00W+b5OWPWzI3HtYTxpLIjrDqDUOYQgJVer/eg6iDC+KTwETFVVl6xC8BSADUxvhRvLHr8IhFlxPg6wqDS2EWqMwglXgaw2uv1JlQbHaGOFD4i5iJ7/awB8ItYXWNRzm2VLmvqwlidXxifm1021RlEXDGALwB4wOv1dqgOIxKHvFGIuCgrr+gF8IGnH7n3EIDPA4jap/Msx9j6qekL5dZ1k5N+XabSDeDDXq/3V6qDiMQjIz4irsrKK74IYBOArmicj0AhT+GjASKSX3omJ/26TOM0gFVS9IiRksJHxF1ZecVvAayE/gY2KivH3r/DbnHMHH0qkejS2Cnru5Lf6wCWer3eA6qDiMQlhY9Qoqy84hCARQB+O9Jz5LnGHy5Onb46eqlEIkuBI0d1BhFT3wRwh9frbVEdRCQ2aVkhlHv6kXv/HsBTAOxDfY2VbL2bJvxts5VsskOzeNuzztc6QJCRn+TSDuBjXq/3edVBRHKQER+hXFl5xdcArAVwbqivWZv/cLUUPeJaFtAV1RlEVFUDWChFj4gmKXyEIUT2+1kI4HeDHVucesvePNf4NbFPJRKNFRbp15U8vg59f55RrwUUoj+Z6hKG8vQj9xKAfwDwRQyw3YLd4mx/sOST3RayFMU9nDC8/3O+WdVNftnaILG1Qr9V/WXVQURykhEfYShl5RVcVl7xVeitLs5c+7yn8NFDUvSIG3GxI6694UTU7QCwQIoeEUtS+AhDKiuv2AFgPoAfX31savrCXVmOPLmLS9yQ9OtKWCHoG5uu93q9DarDiOQmU13C8J5+5N73pFjdX7xv/F/nEpHcsixu6E3bkW1HbY3rVOcQw3IYwIe8Xu8e1UGEOciIjzC8svKKX98x7qPriWin6izC2NzssqrOIIZMA/BfABZJ0SPiSUZ8RELxban8KPS7PWSvFnGdk5amPVsdh5eoziEGdRL6AuYdqoMI85ERH5FQip9a8wMAcwH8SXUWYTxp7EpVnUHcFAP4NoD5UvQIVWTERyQs35bKzQCeBpCnOoswhg7q8f3SubNYdQ4xoFPQd2B+XXUQYW4y4iMSVvFTa34CYCaAH6rOIowhhZ3ZqjOI6wSg78s1R4oeYQQy4iOSgm9L5ToA/wNguuosQq1nna/1gCBTXsawFcATXq/3qOogQlwlhY9IGr4tlU4A/wxgCwCn4jhCke87X/cxsUx3qXUJwKe9Xu9zqoMIcS0pfETS8W2pnA7guwA8qrOI+Puxc+vhIIVnqc5hUgzgWQBbvF5vq+owQgxECh+RtHxbKh8E8J8ApqrOIuLn587KPT0UkFva468GwCe9Xu9bqoMIcTOyuFkkreKn1rwIYDaATwNoVxxHxImT7X2qM5hMA4DNAJZK0SMSgYz4CFPwbanMA/AfAD4GQHb3TWKv2PdtO29tlbYVsdcF4CkAX/N6vb2qwwgxVFL4CFPxbamcA+BrAG5TnUXExjbb4a0nbE3rVedIYmEA3wfwWa/Xe0F1GCGGSwofYUq+LZV3AfgCgEWqs4joqrGdqtxnO71GdY4k9SqAMq/XW6c6iBAjJYWPMC3flkoCsAnAvwOYoziOiJJj1saqSvuRZapzJJntAD7n9Xq3qg4ixGhJ4SNMz7el0gLgEQBeALeoTSNG67yl9dArjn1SyEbHm9ALHtlxWSQNKXyEiPBtqbRCvzvlswAmKY4jRqiNus/+yrlrguocCe4t6AXPn1UHESLapPAR4hq+LZV2AB8B8I8ApiiOI4YpgFDHc65tGapzJKjd0AueV1UHESJWpPAR4gYiI0DvAfAPAGRDvATyrPO1AAgO1TkSyBsAviIFjzADKXyEGALflkoP9BGgO1RnEYP7vvO1JiYUqs5hcGEAvwbwVa/XW6M6jBDxIoWPEMPg21I5D3oB9AgAm+I44gZ+5HzjWIi06apzGFQXgB8A+G+v13tKdRgh4k0KHyFGwLelsgTA3wD4KIAcxXHENX7mrKzppcBi1TkM5hyAbwH4ntfrlRYuwrSk8BFiFHxbKp0AHgbwBIBViuOIiF85du5os/SsVp3DADTomw7+L4AKr9cbUpxHCOWk8BEiSiLTYE8AeAyAW3EcU/udvWZbk7XNzP26mqC3lXjW6/WeVR1GCCORwkeIKPNtqUyHXvw8AWCu4jim9Ib90NZ664X1qnPEmQbgj9BHd16W0R0hBiaFjxAx5NtSuRD6pogfAFCgOI5pVNtOVtbazpqlX9dpAD8D8H2v13tGcRYhDE8KHyHiILIn0EboI0GbAKSpTZTcjlh9u3bYj61QnSOGLgL4JYCfe73enarDCJFIpPARIs58WyrToBc/mwGUArCqTZR8GiyXD7zqqJ2nOkeUdQJ4Efrozp+9Xm9YcR4hEpIUPkIo5NtSmQ/gfgAPQi+CnGoTJYdW6jr9gnN3MvRb64Z+V1Y59HU7vYrzCJHwpPARwiAii6Lvgl4E3Q0gU22ixNWHwJWfuirHqM4xQucBVAB4CcBrXq/XrziPEElFCh8hDCjSKHUD9CLoPgDFahMlFqmWsPEAAAhTSURBVAbz952vh0EJs7t2LfRC5yUANV6vV96YhYgRKXyESAC+LZWzAdwOvVfYWgApahMZ37PO1y6BkKc6xw1cArAVwOsAfu/1es+pjSOEeUjhI0SC8W2pdEHfJdoT+VoK6Rt2nR863zgRJm2a6hwR7QC2Qy90XgdwUEZ1hFBDCh8hElxkbdCt0IuhFQCWA0hXGsoAfurcvq+PggsVXb4FQDX0Yuc16NNXcheWEAYghY8QSca3pdICYDb0Imhl5Gs6AFKZK95+6XhrZ4eld2UcLuUHsB/AbgBVAHZ7vd6TcbiuEGIEpPARwgR8WyrHQJ8Smw+9jcZcADORxLfPv+zYs/2CpX1tlE/bA+AwgDoAe6AXO7VerzcQ5esYBhGFARyEPp16GsBmZm5Tm+p6RFQE4JvM/F7VWYSxSeEjhEn5tlTaAEwDMA/vFENzAZQgCTZVfM1+cOtp68X1I3x5H4Cj0AucOgCHIv88bba1OUTUxczuyL//GMBxZv6i4lhCjJgsiBTCpIqfWhMCcCTyVX718cit9BMATAEweYCvjLiHHQE3uyw3eZoBNAM4M8DXaQCnZE3OgHZCL5RBRFMAfAdAHvSRsI8x81EiygfwDPQ/KwDwBDO/RUR/D+CjkceeZeZvENFEAK8AeBP6GrXzAB5g5l4i2gp9RG0DgCwAf8HMlZHX/ATvtH35m8j5JwKoYOY5RDQbwA8BOABYALwHQCP0Nh/F0Av7zzNzORF9FvqWESkA3gLwV8zMN7m+FcBXANwJvTHs95j5W0S0GMDXALgBXAbwYWZuGs1/bBEbUvgIId6l+Kk1QQAnI1/X8W2pzAFQBCB/kK8cxH8qrRP6wuKWTE49BuAs9L5WFyJfTZHHzsrGgMMT+YVfCuD7kYf+F8DHmfkEES0H8F3odxl+E8A2Zt4UeY07UhR8BPrCewKwm4i2AbgCfdTxA8z8MSL6JfQi5aeRa9iYeRkR3Q3gc9D73V0EcBsz9xHRNAD/B2DJNXE/DuC/mflnROSAXujcDaCRme+J/DxXNwj9NjP/R+SxnwC4F8DLN7n+XwKYCGABM4eIKJuI7AC+Bb1ou0REjwD4It4p9ISBSOEjhBiW4qfWtEAvLg4OdmxkOs19k68U6J/IaZAvP/RRhZt9dRQ/tebttTbF0H9LiVFLIaL9AMZBHx38ExG5oY/QPE/09pr5q0WuB8DjAMDMYQDtRHQrgN8wczcAENELANZA37DxNDPvj7y2BnpRcdULAzxuB/BtIloAIAzglgEy7wTwL0RUDOCFSHF2EMDTRPQV6CNDlZFjNxDRPwJIBZANfUrzauEz0PU3AniGmUORn7GViOYAmBP5bwPohZaM9hiUFD5CiJiJTKe1Rb5EYupl5gVElAq9b9iTAH4EoI2ZF0Th/P1H3sJ49+ac/n6PX/199Snoo3fzoRfNfdeekJl/TkS7AdwD4PdE9FfM/DoRLYI+8vMFInoNwFehj1QtYeYGIvICcA1y/YEQgDpmjsddhGKUbjYHLoQQQgAAmLkHwCcBlEEfYTtNRA8DAOnmRw59DcATkcetkSmlSgAPElEqEaUB2BR5bCQyATQxswZgMwZYiE9EkwGcYuZvAvgtgHmRu756mPmnAP4TwCK8U+RcjoxiDeWOsD8B+CsiskWulQ3gGIA8IloZecweWWckDEgKHyGEEEPCzPsAHADwAQCPAvgLIqqFPj30QOSwv4U+fXQQ+hTRLGbeC32UqAr6guFnI+caie8C+FDkujOgd7C/1vsAHIpM0c0B8Bz0OxarIo99DsAXIrflfw/6XXuvQt90cjDPAjgH4EAkwweZOQC9aPpK5LH90KcChQHJ7exCCCGEMA0Z8RFCCCGEaUjhI4QQQgjTkMJHCCGEEKYhhY8QQgghTEP28RGin34NGa96kJnPKIojhBAiyuSuLiH66d+QMc7XJeh/H7V4X1sIIcxEprqEGAQRnSGi3Mi/L4k0LwQR/Z6I9ke+2onoQ0T0bL/HLhHR5yLH/gMRVRPRASL698hjE4noGBE9B30fkfFE9CMiOkREB4noU4p+ZCGESFoy1SXEu13tSwToPYQ23ehAZr4bACINGH8I4EVm/nHksQkA/gDgR0R0O/RGjMugb23/EhGthb4J2jQAH2LmXZHzjGPmOZFzZMXkJxRCCBOTwkeId+sdTv+hyEjQTwC8j5nbI4+5ADwP4BPMfJaIPgHgdgBXd6p1Qy94zgE4y8y7Io+fAjCZiL4F4HcA/hiNH0gIIcQ7pPARYnAhvDMt/HYDQyKyAvgFgP9g5kP9jn8GekfoP189FMCXmfl/+p+UiCai33b7zHwl0u/oDgAfh77t/kej+pMIIYTJyRofIQZ3BsDiyL+/p9/jTwE4wMy/uPoAET0JIJ2Zn+p33KsAPhppgggiGkdEY6+9SGT0yMLMvwbwr9CbKAohhIgiGfERYnD/DuD7RPR5AFv7Pf5pAHX91gR9NvJYsN9jzzDzM0Q0E8BO/eYtdAF4DED4muuMA/BDIrr6geSfo/6TCCGEycnt7EIIIYQwDZnqEkIIIYRpSOEjhBBCCNOQwkcIIYQQpiGFjxBCCCFMQwofIYQQQpiGFD5CCCGEMA0pfP5fu3UgAAAAACDI33qQiyIAYEN8AIAN8QEANsQHANgQHwBgQ3wAgA3xAQA2xAcA2BAfAGBDfACADfEBADbEBwDYEB8AYEN8AIAN8QEANsQHANgQHwBgQ3wAgA3xAQA2xAcA2BAfAGBDfACADfEBADbEBwDYEB8AYEN8AIAN8QEANsQHANgQHwBgQ3wAgA3xAQA2xAcA2BAfAGBDfACADfEBADbEBwDYEB8AYEN8AICNAMV3Y+8WqFHmAAAAAElFTkSuQmCC\n","text/plain":["
"]},"metadata":{"tags":[]}}]},{"cell_type":"code","metadata":{"id":"Cjyy9-p9I1tP","colab_type":"code","colab":{}},"source":["#Conversión de los datos tipo string a enteros en un rango entre 0 y 255 (1 Byte de información)\n","def encode_string_byte (df,name):\n"," df[name] = LabelEncoder().fit_transform(df[name])\n","\n","encode_string_byte (df,'proto')\n","encode_string_byte (df,'state') \n","encode_string_byte (df,'service') \n","\n","display(df.head())\n","print(\"\")\n","print(\"Proto column --> Max value: {} Min value: {} \".format(max(df['proto']),min(df['proto'])))\n","print(\"State column --> Max value: {} Min value: {} \".format(max(df['state']),min(df['state'])))\n","print(\"Service column --> Max value: {} Min value: {} \".format(max(df['service']),min(df['service'])))"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"K5PK4TkmNZMN","colab_type":"code","colab":{}},"source":["#Normalización de los números enteros en valores decimales en rango entre 0 y 1\n","def numerical_minmax_normalization (df, name):\n"," x = df[name].values.reshape(-1,1)\n"," min_max_scaler = preprocessing.MinMaxScaler()\n"," x_scaled = min_max_scaler.fit_transform(x)\n"," df[name] = x_scaled\n","\n","numerical_minmax_normalization(df,'dur')\n","numerical_minmax_normalization(df,'spkts')\n","numerical_minmax_normalization(df,'dpkts')\n","numerical_minmax_normalization(df,'sbytes')\n","numerical_minmax_normalization(df,'dbytes')\n","numerical_minmax_normalization(df,'rate')\n","numerical_minmax_normalization(df,'sttl')\n","numerical_minmax_normalization(df,'dttl')\n","numerical_minmax_normalization(df,'sload')\n","numerical_minmax_normalization(df,'dload')\n","numerical_minmax_normalization(df,'sloss')\n","numerical_minmax_normalization(df,'dloss')\n","numerical_minmax_normalization(df,'sinpkt')\n","numerical_minmax_normalization(df,'dinpkt')\n","numerical_minmax_normalization(df,'sjit')\n","numerical_minmax_normalization(df,'djit')\n","numerical_minmax_normalization(df,'swin')\n","numerical_minmax_normalization(df,'stcpb')\n","numerical_minmax_normalization(df,'dtcpb')\n","numerical_minmax_normalization(df,'dwin')\n","numerical_minmax_normalization(df,'tcprtt')\n","numerical_minmax_normalization(df,'synack')\n","numerical_minmax_normalization(df,'ackdat')\n","numerical_minmax_normalization(df,'smean')\n","numerical_minmax_normalization(df,'dmean')\n","numerical_minmax_normalization(df,'trans_depth')\n","numerical_minmax_normalization(df,'response_body_len')\n","numerical_minmax_normalization(df,'ct_srv_src')\n","numerical_minmax_normalization(df,'ct_state_ttl')\n","numerical_minmax_normalization(df,'ct_dst_ltm')\n","numerical_minmax_normalization(df,'ct_src_dport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_sport_ltm')\n","numerical_minmax_normalization(df,'ct_dst_src_ltm')\n","numerical_minmax_normalization(df,'is_ftp_login')\n","numerical_minmax_normalization(df,'ct_ftp_cmd')\n","numerical_minmax_normalization(df,'ct_flw_http_mthd')\n","numerical_minmax_normalization(df,'ct_src_ltm')\n","numerical_minmax_normalization(df,'ct_srv_dst')\n","numerical_minmax_normalization(df,'is_sm_ips_ports')\n","\n","df.head()"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"dZf6lDHLOWvW","colab_type":"code","colab":{}},"source":["#Mapeo de los valores normalizados del paso anterior a valores enteros entre 0 y 255 (1 Byte de información)\n","def numerical_split_ohe (df,name):\n"," pd_to_np = df[name].tolist()\n"," np_split = []\n"," \n"," categories = np.linspace(0, 1, num=256,endpoint=False)\n"," quantization = range(0,256)\n","\n"," for value in pd_to_np:\n"," for i in range(len(categories)-1):\n"," if (categories[i] <= float(value) <= categories[i+1]):\n"," np_split.append(quantization[i])\n"," break\n"," if (float(value) > categories[-1]):\n"," np_split.append(quantization[-1])\n"," break\n"," \n"," df[name] = np_split\n","\n","\n","numerical_split_ohe(df,'dur')\n","numerical_split_ohe(df,'spkts')\n","numerical_split_ohe(df,'dpkts')\n","numerical_split_ohe(df,'sbytes')\n","numerical_split_ohe(df,'dbytes')\n","numerical_split_ohe(df,'rate')\n","numerical_split_ohe(df,'sttl')\n","numerical_split_ohe(df,'dttl')\n","numerical_split_ohe(df,'sload')\n","numerical_split_ohe(df,'dload')\n","numerical_split_ohe(df,'sloss')\n","numerical_split_ohe(df,'dloss')\n","numerical_split_ohe(df,'sinpkt')\n","numerical_split_ohe(df,'dinpkt')\n","numerical_split_ohe(df,'sjit')\n","numerical_split_ohe(df,'djit')\n","numerical_split_ohe(df,'swin')\n","numerical_split_ohe(df,'stcpb')\n","numerical_split_ohe(df,'dtcpb')\n","numerical_split_ohe(df,'dwin')\n","numerical_split_ohe(df,'tcprtt')\n","numerical_split_ohe(df,'synack')\n","numerical_split_ohe(df,'ackdat')\n","numerical_split_ohe(df,'smean')\n","numerical_split_ohe(df,'dmean')\n","numerical_split_ohe(df,'trans_depth')\n","numerical_split_ohe(df,'response_body_len')\n","numerical_split_ohe(df,'ct_srv_src')\n","numerical_split_ohe(df,'ct_state_ttl')\n","numerical_split_ohe(df,'ct_dst_ltm')\n","numerical_split_ohe(df,'ct_src_dport_ltm')\n","numerical_split_ohe(df,'ct_dst_sport_ltm')\n","numerical_split_ohe(df,'ct_dst_src_ltm')\n","numerical_split_ohe(df,'is_ftp_login')\n","numerical_split_ohe(df,'ct_ftp_cmd')\n","numerical_split_ohe(df,'ct_flw_http_mthd')\n","numerical_split_ohe(df,'ct_src_ltm')\n","numerical_split_ohe(df,'ct_srv_dst')\n","numerical_split_ohe(df,'is_sm_ips_ports')\n","\n","display(df.head())"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"HESpjmI5l7tj","colab_type":"code","colab":{}},"source":["#Quitando la columna attack_cat y guardandola en la variable y.\n","y_column = df['attack_cat']\n","df.drop('attack_cat',axis=1,inplace=True)\n","dummies = pd.get_dummies(y_column) \n","y = dummies.values\n","\n","print(y[:5])\n","display(dummies.head())"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"f7JSiCRO_9Nk","colab_type":"code","colab":{}},"source":["#Balanceo del dataset mediante SMOTE (1000 por cada clase) y normalización entre -0,5 y 0,5\n","byte_images = df.to_numpy()\n","x = []\n","for image in np.array(byte_images):\n"," x.append((image/255 - 0.5))\n","sm = SMOTE(random_state=0)\n","x, y = sm.fit_sample(x, y)\n","x = np.array(x)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"8XFDF3PiIbb5","colab_type":"code","colab":{}},"source":["#Separación del dataset en un set de entrenamiento y otro de validación.\n","sss = StratifiedShuffleSplit(n_splits=1, test_size=0.25, random_state=42)\n","\n","for train_index, test_index in sss.split(x,y):\n"," x_train, x_test = x[train_index], x[test_index]\n"," y_train, y_test = y[train_index], y[test_index]"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"wsWqobWQNWAx","colab_type":"code","colab":{}},"source":["#Transformación del array 'y' en un valor úncio\n","def y_transform(y_train):\n"," y_train_rfe = []\n"," for value in y_train:\n"," y_train_rfe.append(list(value).index(1))\n"," return y_train_rfe\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"4Qb_5PRardhC","colab_type":"code","outputId":"4340b090-e4f2-41cf-8665-fec31afd2220","executionInfo":{"status":"ok","timestamp":1591810329799,"user_tz":-120,"elapsed":1064441,"user":{"displayName":"Farid Bagheri-Gisour Marandyn","photoUrl":"","userId":"05978557582753685866"}},"colab":{"base_uri":"https://localhost:8080/","height":1000}},"source":["#Proceso de selección de caraceterísitcas mediante RFE. Estimador RandomForestClassifier.\n","y_rfe = y_transform(y)\n","for index in range(1,42):\n"," sel = RFE(RandomForestClassifier(n_estimators=100,random_state=0,n_jobs=-1),n_features_to_select=index)\n"," sel.fit(x,y_rfe)\n"," print(\"Number of selected features: {}\".format(index))\n"," features =[]\n"," for i,value in enumerate(sel.support_):\n"," if value:\n"," features.append(i)\n"," print(\"Features: {}\".format([list(list(df.columns)[index_value] for index_value in features)]))"],"execution_count":0,"outputs":[{"output_type":"stream","text":["Number of selected features: 1\n","Features: [['smean']]\n","Number of selected features: 2\n","Features: [['smean', 'dmean']]\n","Number of selected features: 3\n","Features: [['smean', 'dmean', 'ct_srv_dst']]\n","Number of selected features: 4\n","Features: [['synack', 'smean', 'dmean', 'ct_srv_dst']]\n","Number of selected features: 5\n","Features: [['proto', 'synack', 'smean', 'dmean', 'ct_srv_dst']]\n","Number of selected features: 6\n","Features: [['proto', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 7\n","Features: [['proto', 'sttl', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 8\n","Features: [['proto', 'service', 'sttl', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 9\n","Features: [['dur', 'proto', 'service', 'sttl', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 10\n","Features: [['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 11\n","Features: [['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 12\n","Features: [['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 13\n","Features: [['dur', 'proto', 'service', 'sttl', 'sload', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 14\n","Features: [['dur', 'proto', 'service', 'sttl', 'sload', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 15\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 16\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 17\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 18\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 19\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 20\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 21\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 22\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 23\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'sload', 'sjit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 24\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 25\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 26\n","Features: [['dur', 'proto', 'service', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 27\n","Features: [['dur', 'proto', 'service', 'state', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 28\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'rate', 'sttl', 'dttl', 'sload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 29\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 30\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 31\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 32\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 33\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 34\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'dloss', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 35\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'dloss', 'sinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 36\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 37\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 38\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 39\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'sbytes', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst']]\n","Number of selected features: 40\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'sbytes', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst', 'is_sm_ips_ports']]\n","Number of selected features: 41\n","Features: [['dur', 'proto', 'service', 'state', 'spkts', 'dpkts', 'sbytes', 'dbytes', 'rate', 'sttl', 'dttl', 'sload', 'dload', 'sloss', 'dloss', 'sinpkt', 'dinpkt', 'sjit', 'djit', 'swin', 'stcpb', 'dtcpb', 'dwin', 'tcprtt', 'synack', 'ackdat', 'smean', 'dmean', 'trans_depth', 'response_body_len', 'ct_srv_src', 'ct_state_ttl', 'ct_dst_ltm', 'ct_src_dport_ltm', 'ct_dst_sport_ltm', 'ct_dst_src_ltm', 'ct_ftp_cmd', 'ct_flw_http_mthd', 'ct_src_ltm', 'ct_srv_dst', 'is_sm_ips_ports']]\n"],"name":"stdout"}]}]} --------------------------------------------------------------------------------