├── 01_LR_Introduction.pdf ├── 02_Odd_LogOdd_OddRatio.pdf ├── 03_Logit_Model.pdf ├── 04_Likelihood_Probability.pdf ├── 05_Maxium_Likelihood_Estimation.pdf ├── 06_LR_Assumptions.pdf ├── 07_LR_Assumptions.ipynb ├── 08_AIC_BIC.pdf ├── 09_Logistic_Regression.ipynb ├── 10_Multiclass_Classification.pdf ├── 11_Multi_Class_Classification.ipynb ├── 12_Regularization.pdf ├── 13_LR_Regularization.ipynb ├── 14_WOE_IV.pdf ├── 15_LR_WOE_IV.ipynb ├── 16_LR_Revision.pdf ├── 17_LR_1_Interview_Questions.pdf ├── 18_LR_2_Interview_Questions.pdf └── README.md /01_LR_Introduction.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/01_LR_Introduction.pdf -------------------------------------------------------------------------------- /02_Odd_LogOdd_OddRatio.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/02_Odd_LogOdd_OddRatio.pdf -------------------------------------------------------------------------------- /03_Logit_Model.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/03_Logit_Model.pdf -------------------------------------------------------------------------------- /04_Likelihood_Probability.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/04_Likelihood_Probability.pdf -------------------------------------------------------------------------------- /05_Maxium_Likelihood_Estimation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/05_Maxium_Likelihood_Estimation.pdf -------------------------------------------------------------------------------- /06_LR_Assumptions.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/06_LR_Assumptions.pdf -------------------------------------------------------------------------------- /08_AIC_BIC.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/08_AIC_BIC.pdf -------------------------------------------------------------------------------- /10_Multiclass_Classification.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/10_Multiclass_Classification.pdf -------------------------------------------------------------------------------- /11_Multi_Class_Classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "11 Multi Class Classification.ipynb", 7 | "provenance": [], 8 | "authorship_tag": "ABX9TyPWNOFvn1SCWX1AdS9Jn8me", 9 | "include_colab_link": true 10 | }, 11 | "kernelspec": { 12 | "name": "python3", 13 | "display_name": "Python 3" 14 | }, 15 | "language_info": { 16 | "name": "python" 17 | } 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "view-in-github", 24 | "colab_type": "text" 25 | }, 26 | "source": [ 27 | "\"Open" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "source": [ 33 | "**Binary Classifiers for Multi-Class Classification**\n", 34 | "\n", 35 | "- Classification is a predictive modeling problem that involves assigning a class label to an example.\n", 36 | "- **Binary classification** are those tasks where examples are assigned exactly one of two classes. \n", 37 | "- **Binary Classification:** Classification tasks with two classes.\n", 38 | "- **Multi-class classification** is those tasks where examples are assigned exactly one of more than two classes.\n", 39 | "- **Multi-class Classification:** Classification tasks with more than two classes.\n", 40 | "\n", 41 | "*Some algorithms are designed for binary classification problems. Examples include:*\n", 42 | "\n", 43 | "1. Logistic Regression\n", 44 | "2. Support Vector Machines\n", 45 | "\n", 46 | "\n", 47 | "Instead, heuristic methods can be used to split a multi-class classification problem into multiple binary classification datasets and train a binary classification model each.\n", 48 | "\n", 49 | "Two examples of these heuristic methods include:\n", 50 | "\n", 51 | "1. One-vs-Rest (OvR)\n", 52 | "2. One-vs-One (OvO)" 53 | ], 54 | "metadata": { 55 | "id": "VyVM3wn3FDmZ" 56 | } 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "source": [ 61 | "**One-Vs-Rest (OvR) for Multi-Class Classification**\n", 62 | "\n", 63 | "- **One-vs-rest (OvR)** for short, also referred to as **One-vs-All (OvA)** is a heuristic method for using binary classification algorithms for multi-class classification.\n", 64 | "\n", 65 | "- It involves splitting the multi-class dataset into multiple binary classification problems. \n", 66 | "- A binary classifier is then trained on each binary classification problem and predictions are made using the model that is the most confident.\n", 67 | "\n", 68 | "- For example, \n", 69 | " - Given a multi-class classification problem with examples for each class ‘red,’ ‘blue,’ and ‘green‘. \n", 70 | " - This could be divided into three binary classification datasets as follows:\n", 71 | " 1. Binary Classification Problem 1: red vs [blue, green]\n", 72 | " 2. Binary Classification Problem 2: blue vs [red, green]\n", 73 | " 3. Binary Classification Problem 3: green vs [red, blue]\n", 74 | "\n", 75 | "**Possible Issue**\n", 76 | "- A possible downside of this approach is that it requires one model to be created for each class. \n", 77 | "- For example, three classes requires three models. \n", 78 | " - This could be an issue for large datasets (e.g. millions of rows) \n", 79 | " - Very large numbers of classes (e.g. hundreds of classes).\n", 80 | "\n", 81 | "**Approach of OvA or OvR**\n", 82 | "- The obvious approach is to use a one-versus-the-rest approach (also called one-vs-all), in which we train C binary classifiers, fc(x), where the data from class c is treated as positive, and the data from all the other classes is treated as negative.\n", 83 | "- This approach requires that each model predicts a class membership probability or a probability-like score. The argmax of these scores (class index with the largest score) is then used to predict a class.\n", 84 | "- This approach is commonly used for algorithms that naturally predict numerical class membership probability or score, such as: Logistic Regression. \n", 85 | "As such, the implementation of these algorithms in the scikit-learn library implements the OvR strategy by default when using these algorithms for multi-class classification.\n", 86 | "\n", 87 | "**Python Example**\n", 88 | "- We can demonstrate this with an example on a 3-class classification problem using the LogisticRegression algorithm. \n", 89 | "- The strategy for handling multi-class classification can be set via the “multi_class” argument and can be set to “ovr” for the one-vs-rest strategy." 90 | ], 91 | "metadata": { 92 | "id": "LNYaXKSJFjUq" 93 | } 94 | }, 95 | { 96 | "cell_type": "code", 97 | "source": [ 98 | "# logistic regression for multi-class classification using built-in one-vs-rest\n", 99 | "from sklearn.datasets import make_classification\n", 100 | "from sklearn.linear_model import LogisticRegression\n", 101 | "\n", 102 | "# define dataset\n", 103 | "X, y = make_classification(n_samples=1000, n_features=10, n_informative=5, n_redundant=5, n_classes=3, random_state=1)\n", 104 | "\n", 105 | "# define model\n", 106 | "model = LogisticRegression(multi_class='ovr')\n", 107 | "\n", 108 | "# fit model\n", 109 | "model.fit(X, y)\n", 110 | "\n", 111 | "# make predictions\n", 112 | "yhat = model.predict(X)" 113 | ], 114 | "metadata": { 115 | "id": "LoAiNMAzFjEX" 116 | }, 117 | "execution_count": 1, 118 | "outputs": [] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "source": [ 123 | "- The scikit-learn library also provides a separate OneVsRestClassifier class that allows the one-vs-rest strategy to be used with any classifier.\n", 124 | "\n", 125 | "- This class can be used to use a binary classifier like Logistic Regression or Perceptron for multi-class classification, or even other classifiers that natively support multi-class classification." 126 | ], 127 | "metadata": { 128 | "id": "VoM4ch_1FqKC" 129 | } 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": 2, 134 | "metadata": { 135 | "id": "AcL956kJBPes" 136 | }, 137 | "outputs": [], 138 | "source": [ 139 | "# logistic regression for multi-class classification using a one-vs-rest\n", 140 | "from sklearn.datasets import make_classification\n", 141 | "from sklearn.linear_model import LogisticRegression\n", 142 | "from sklearn.multiclass import OneVsRestClassifier\n", 143 | "\n", 144 | "# define dataset\n", 145 | "X, y = make_classification(n_samples=1000, n_features=10, n_informative=5, n_redundant=5, n_classes=3, random_state=1)\n", 146 | "\n", 147 | "# define model\n", 148 | "model = LogisticRegression()\n", 149 | "\n", 150 | "# define the ovr strategy\n", 151 | "ovr = OneVsRestClassifier(model)\n", 152 | "\n", 153 | "# fit model\n", 154 | "ovr.fit(X, y)\n", 155 | "\n", 156 | "# make predictions\n", 157 | "yhat = ovr.predict(X)" 158 | ] 159 | }, 160 | { 161 | "cell_type": "markdown", 162 | "source": [ 163 | "**One-Vs-One for Multi-Class Classification**\n", 164 | "\n", 165 | "- One-vs-One (OvO for short) is another heuristic method for using binary classification algorithms for multi-class classification.\n", 166 | "\n", 167 | "- Like one-vs-rest, one-vs-one splits a multi-class classification dataset into binary classification problems. \n", 168 | "\n", 169 | "- Unlike one-vs-rest that splits it into one binary dataset for each class, the one-vs-one approach splits the dataset into one dataset for each class versus every other class.\n", 170 | "- For example\n", 171 | " - Consider a multi-class classification problem with four classes: ‘red,’ ‘blue,’ and ‘green,’ ‘yellow.’ \n", 172 | " - This could be divided into six binary classification datasets as follows:\n", 173 | " \n", 174 | " Binary Classification Problem 1: red vs. blue\n", 175 | "\n", 176 | " Binary Classification Problem 2: red vs. green\n", 177 | "\n", 178 | " Binary Classification Problem 3: red vs. yellow\n", 179 | "\n", 180 | " Binary Classification Problem 4: blue vs. green\n", 181 | "\n", 182 | " Binary Classification Problem 5: blue vs. yellow\n", 183 | "\n", 184 | " Binary Classification Problem 6: green vs. yellow\n", 185 | "\n", 186 | "- The formula for calculating the number of binary datasets, and in turn, models, is as follows:\n", 187 | "\n", 188 | " (NumClasses * (NumClasses – 1)) / 2\n", 189 | "\n", 190 | "- We can see that for four classes, this gives us the expected value of six binary classification problems:\n", 191 | "\n", 192 | " (NumClasses * (NumClasses – 1)) / 2\n", 193 | " \n", 194 | " (4 * (4 – 1)) / 2\n", 195 | " \n", 196 | " (4 * 3) / 2\n", 197 | " \n", 198 | " 12 / 2\n", 199 | " \n", 200 | " 6\n", 201 | "\n", 202 | "Each binary classification model may predict one class label and the model with the most predictions or votes is predicted by the one-vs-one strategy.\n", 203 | "\n", 204 | "- An alternative is to introduce K(K − 1)/2 binary discriminant functions, one for every possible pair of classes. \n", 205 | "- This is known as a **one-versus-one classifier**. \n", 206 | "- Each point is then classified according to a *majority vote amongst* the discriminant functions.\n", 207 | "- Similarly, if the binary classification models predict a numerical class membership, such as a *probability, then the argmax of the sum of the scores (class with the largest sum score) is predicted as the class label.*\n", 208 | "\n", 209 | "The support vector machine implementation in the scikit-learn is provided by the SVC class and supports the one-vs-one method for multi-class classification problems. This can be achieved by setting the “decision_function_shape” argument to ‘ovo‘." 210 | ], 211 | "metadata": { 212 | "id": "KrhI1hBuFqaP" 213 | } 214 | }, 215 | { 216 | "cell_type": "code", 217 | "source": [ 218 | "# SVM for multi-class classification using built-in one-vs-one\n", 219 | "from sklearn.datasets import make_classification\n", 220 | "from sklearn.svm import SVC\n", 221 | "\n", 222 | "# define dataset\n", 223 | "X, y = make_classification(n_samples=1000, n_features=10, n_informative=5, n_redundant=5, n_classes=3, random_state=1)\n", 224 | "\n", 225 | "# define model\n", 226 | "model = SVC(decision_function_shape='ovo')\n", 227 | "\n", 228 | "# fit model\n", 229 | "model.fit(X, y)\n", 230 | "\n", 231 | "# make predictions\n", 232 | "yhat = model.predict(X)" 233 | ], 234 | "metadata": { 235 | "id": "Avulnt8zL071" 236 | }, 237 | "execution_count": 3, 238 | "outputs": [] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "source": [ 243 | "- The scikit-learn library also provides a separate OneVsOneClassifier class that allows the one-vs-one strategy to be used with any classifier.\n", 244 | "\n", 245 | "- This class can be used with a binary classifier like SVM, Logistic Regression or Perceptron for multi-class classification, or even other classifiers that natively support multi-class classification.\n", 246 | "\n", 247 | "- It is very easy to use and requires that a classifier that is to be used for binary classification be provided to the OneVsOneClassifier as an argument." 248 | ], 249 | "metadata": { 250 | "id": "SraJXdoKL0h3" 251 | } 252 | }, 253 | { 254 | "cell_type": "code", 255 | "source": [ 256 | "# SVM for multi-class classification using one-vs-one\n", 257 | "from sklearn.datasets import make_classification\n", 258 | "from sklearn.svm import SVC\n", 259 | "from sklearn.multiclass import OneVsOneClassifier\n", 260 | "\n", 261 | "# define dataset\n", 262 | "X, y = make_classification(n_samples=1000, n_features=10, n_informative=5, n_redundant=5, n_classes=3, random_state=1)\n", 263 | "\n", 264 | "# define model\n", 265 | "model = SVC()\n", 266 | "\n", 267 | "# define ovo strategy\n", 268 | "ovo = OneVsOneClassifier(model)\n", 269 | "\n", 270 | "# fit model\n", 271 | "ovo.fit(X, y)\n", 272 | "\n", 273 | "# make predictions\n", 274 | "yhat = ovo.predict(X)" 275 | ], 276 | "metadata": { 277 | "id": "DRGfJE3nL15l" 278 | }, 279 | "execution_count": 4, 280 | "outputs": [] 281 | } 282 | ] 283 | } -------------------------------------------------------------------------------- /12_Regularization.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/12_Regularization.pdf -------------------------------------------------------------------------------- /13_LR_Regularization.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "13_LR_Regularization.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "authorship_tag": "ABX9TyOBOJ2j7rR2RWq4fqoSqdFZ", 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "name": "python3", 14 | "display_name": "Python 3" 15 | }, 16 | "language_info": { 17 | "name": "python" 18 | } 19 | }, 20 | "cells": [ 21 | { 22 | "cell_type": "markdown", 23 | "metadata": { 24 | "id": "view-in-github", 25 | "colab_type": "text" 26 | }, 27 | "source": [ 28 | "\"Open" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "source": [ 34 | "**Regularization**\n", 35 | "\n", 36 | "They work by penalizing (regularizing) the magnitude of coefficients of features along with minimizing the error between predicted and actual observations.\n", 37 | "\n", 38 | "**Ridge (L2) and Lasso (L1) regression** \n", 39 | "- They are powerful techniques generally used for creating parsimonious models in presence of a ‘large’ number of features. \n", 40 | "- Here ‘large’ can typically mean either of two things:\n", 41 | " 1. *Large enough to enhance the tendency of a model to overfit* (as low as 10 variables might cause overfitting)\n", 42 | " 2. *Large enough to cause computational challenges.* With modern systems, this situation might arise in case of millions or billions of features\n", 43 | "\n", 44 | "Though Ridge and Lasso might appear to work towards a common goal, the inherent properties and practical use cases differ substantially. \n", 45 | "- They work by penalizing the magnitude of coefficients of features along with minimizing the error between predicted and actual observations. \n", 46 | "- These are called **‘regularization’ techniques**. The key difference is in how they assign penalty to the coefficients:\n", 47 | "\n", 48 | "**Ridge Regression:**\n", 49 | "\n", 50 | "- Performs L2 regularization\n", 51 | "- i.e. adds penalty equivalent to square of the magnitude of coefficients\n", 52 | " \n", 53 | " Minimization objective = LS Obj + α * (sum of square of coefficients)\n", 54 | "\n", 55 | "**Lasso Regression:**\n", 56 | "- Performs L1 regularization\n", 57 | "- i.e. adds penalty equivalent to absolute value of the magnitude of coefficients\n", 58 | "\n", 59 | " Minimization objective = LS Obj + α * (sum of absolute value of coefficients)\n", 60 | "\n", 61 | "**Note** that here ‘LS Obj’ refers to ‘least squares objective’, i.e. the linear regression objective without regularization.\n", 62 | " \n", 63 | "**Why Penalize the Magnitude of Coefficients?**\n", 64 | "\n", 65 | "Lets try to understand the impact of model complexity on the magnitude of coefficients. \n", 66 | "\n", 67 | "As an example, I have simulated a sine curve (between 60° and 300°) and added some random noise:" 68 | ], 69 | "metadata": { 70 | "id": "hvDEPAfrK1r-" 71 | } 72 | }, 73 | { 74 | "cell_type": "code", 75 | "source": [ 76 | "# Importing libraries\n", 77 | "\n", 78 | "import numpy as np\n", 79 | "import pandas as pd\n", 80 | "import random\n", 81 | "import matplotlib.pyplot as plt\n", 82 | "%matplotlib inline\n", 83 | "from matplotlib.pylab import rcParams\n", 84 | "rcParams['figure.figsize'] = 12, 10\n", 85 | "\n", 86 | "#Define input array with angles from 60deg to 300deg converted to radians\n", 87 | "x = np.array([i*np.pi/180 for i in range(60,300,4)])\n", 88 | "np.random.seed(10) #Setting seed for reproducibility\n", 89 | "y = np.sin(x) + np.random.normal(0,0.15,len(x))\n", 90 | "data = pd.DataFrame(np.column_stack([x,y]),columns=['x','y'])\n", 91 | "plt.plot(data['x'],data['y'],'.')" 92 | ], 93 | "metadata": { 94 | "colab": { 95 | "base_uri": "https://localhost:8080/", 96 | "height": 610 97 | }, 98 | "id": "1mWldIWiMHQZ", 99 | "outputId": "171e8046-5d72-4279-c370-01909e7956b1" 100 | }, 101 | "execution_count": 1, 102 | "outputs": [ 103 | { 104 | "output_type": "execute_result", 105 | "data": { 106 | "text/plain": [ 107 | "[]" 108 | ] 109 | }, 110 | "metadata": {}, 111 | "execution_count": 1 112 | }, 113 | { 114 | "output_type": "display_data", 115 | "data": { 116 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAssAAAI/CAYAAACBC5TUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfCklEQVR4nO3da2zs+X3X8c/PZ45bVC41zqpdsllvrEaIqqAWW2FQEQLaorRESbkUpRxQi2KtkBJRxAMUVAlEH5UnXB4cQVenESlYTauWqAuNCL0cVFUwbTwlvSSh4mB1lF0C2bpDoeLiM50fD463Ot34l2Mfz8x/Lq+XdLS2Z+T/b1Z+8PbP3/n9S601AADA59vqegEAALCsxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQEOv6wW0vOlNb6ovvPBC18sAAGDNDYfDX621PnPZY0sbyy+88EJOTk66XgYAAGuulDJqPWYMAwAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEss3aGo3Hu3n+Q4Wjc9VIAgBXX63oBMEvD0Th37g1yPplmu7eV46N+DvZ2ul4WALCi7CyzVganZzmfTDOtycPJNIPTs66XBACsMLHMWunv72a7t5VbJbnd20p/f7frJQEAK8wYBmvlYG8nx0f9DE7P0t/fNYIBANyIWGbtHOztiGQAYCaMYQAAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1iGKxiOxrl7/0GGo3HXSwEAFsjtruEJhqNx7twb5HwyzXZvK8dHfbfTBoANYWcZnmBwepbzyTTTmjycTDM4Pet6SQDAgohleIL+/m62e1u5VZLbva3093e7XhIAsCDGMN5gOBpncHqW/v7u0v6pfRXWuE4O9nZyfNT3/xwANpBYfswqzKauwhrX0cHejv/PALCBjGE8ZhVmU1dhjQAA60IsP2YVZlNXYY3z4vg2AGDRSq216zVc6vDwsJ6cnCz8uqswD7wKa5w14ycAwLyUUoa11sPLHjOz/AarMJu6CmuctcvGTzbt/wEAsHjGMFgJmzx+AgB0x84ySZZ/tMPxbQBAF8QyKzMPvInjJwBAt4xh4Dg6AIAGsYx5YACABmMYXGseeNlnmwEAZmkmsVxK+WCSdyb5XK31qy55vCT5x0m+Kcn/TvLttdafm8W1mY2rzAOvymwzAMCszGoM458leccXePwbk7zt4t+LSf7JjK7bmU28m5zZZgBg08xkZ7nW+lOllBe+wFPeneT76qPbBQ5KKV9aSnm21vrZWVx/0TZ1h/X12eaHk+kTZ5uNawAA62BRM8tvTvKZxz5/5eJrKxnLm3o3uavONm/qLxMAwPpZqjf4lVJezKMxjTz//PMdr6btOjus6+Yqs82b+ssEALB+FhXLryZ5y2OfP3fxtd+m1vpSkpeS5PDwsC5madfnbnJf2Cb/MgEArJdFxfLLSd5fSvlwkj+S5NdXdV75de4m1+aXCQBgXczq6LjvT/InkryplPJKkr+b5HaS1Fr/aZKP5tGxcQ/y6Oi4vzqL67K8/DIBAKyDWZ2G8a1PeLwmed8srgUAAIvidtd0bhPPrAYAVsNSnYbB5nHMXDecgw0AVyOW6ZRj5hbPLygAcHXGMOjU68fM3SpxzNyCuG05AFydneUlsol/GnfM3OI5BxsArq48Oqhi+RweHtaTk5Oul7Ew/jTOIm3iL2YA0FJKGdZaDy97zM7ykjC7yyI5BxsArsbM8pIwuwsAsHzsLC8Js7vr4TrjDUYhAGD5ieUl4k/jq+06c+dm1AFgNRjDgBm5zpFsjm8DgNUglmFGrjN3bkYdAFaDo+NghswsA8DqcXQcLMh15s7NqAPA8jOGAQAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYXmPD0Th37z/IcDTueikAACvJTUnW1HA0zp17g5xPptnubeX4qO8GGAAA12RneU0NTs9yPplmWpOHk2kGp2ddLwkAYOWI5TXV39/Ndm8rt0pyu7eV/v5u10sCAFg5xjDW1MHeTo6P+hmcnqW/v2sEAwDgKYjlNXawtyOSAQBuwBgGAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAZWmtu6AzBPjo4DVpbbugMwb3aWgZXltu4AzJtYBlaW27oDMG/GMNhow9F4bW4Jvk6v5arc1h2AeRPLbKx1mnddp9dyXW7rDsA8GcNgY63TvOs6vRYAWCZimY21TvOu6/RaAGCZlFpr12u41OHhYT05Oel6Gay5dZrzXafXAgCLVEoZ1loPL3vMzDIbbZ3mXdfptQDAsjCGAQAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iOUFGI7GuXv/QYajcddLAQDgGnpdL2DdDUfj3Lk3yPlkmu3eVo6P+jnY2+l6WQAAXIGd5TkbnJ7lfDLNtCYPJ9MMTs+6XhIAAFckluesv7+b7d5WbpXkdm8r/f3drpcEAMAVGcOYs4O9nRwf9TM4PUt/f9cIBgDAChHLC3CwtyOSAQBWkDEMAABoEMsAANAglmHJOacbALpjZhmWmHO6AaBbdpZhiTmnGwC6JZZhiTmnGwC6ZQwDlphzugGgW2IZlpxzugGgO8YwAACgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYB3mA4Gufu/QcZjsZdLwWAjrmDH8BjhqNx7twb5HwyzXZvK8dHfXdQBNhgdpYBHjM4Pcv5ZJppTR5OphmcnnW9JAA6JJYBHtPf3812byu3SnK7t5X+/m7XSwKgQ8YwgI0xHI0zOD1Lf3+3OVpxsLeT46P+E58HwGYQy8BGuM4s8sHejkgGIIkxDGBJzfpECrPIADwNO8vA0pnHiRSvzyI/nEzNIgNwZWIZWDqX7QLfNJbNIgPwNMQysHTmtQtsFhmA6xLLwNKxCwzAshDLwFKyCwzAMnAaBgAANIhlAABoEMtA06zPOgaAVWNmGbjUPM46BoBVY2cZuJQ73gGAWAYaXj/r+FaJO94BsLGMYQCXctYxAMxoZ7mU8o5Syi+XUh6UUj5wyePfXkp5rZTyiYt/R7O4LjBfB3s7ed+f/AqhDMDGuvHOcinlVpK7Sb4hyStJPl5KebnW+qk3PPUHaq3vv+n1AABgUWaxs/z2JA9qrae11vMkH07y7hl8XwAA6NQsYvnNST7z2OevXHztjf58KeUXSik/VEp5ywyuCwAAc7Wo0zD+VZIXaq1/KMmPJfnQZU8qpbxYSjkppZy89tprC1oaAABcbhax/GqSx3eKn7v42m+ptZ7VWv/fxaf3khxc9o1qrS/VWg9rrYfPPPPMDJYGAABPbxax/PEkbyulvLWUsp3kPUlefvwJpZRnH/v0XUk+PYPrAgDAXN34NIxa66SU8v4kH0tyK8kHa62fLKV8V5KTWuvLSf56KeVdSSZJfi3Jt9/0ugAAMG+l1tr1Gi51eHhYT05Oul4GAABrrpQyrLUeXvaY210DAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhlgzoajce7ef5DhaNz1UgC4pl7XCwBYZ8PROHfuDXI+mWa7t5Xjo34O9na6XhYAV2RnGWCOBqdnOZ9MM63Jw8k0g9OzrpcEwDWIZYA56u/vZru3lVslud3bSn9/t+slAXANxjAA5uhgbyfHR/0MTs/S3981ggGwYsQywJwd7O2IZIAVZQwDAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhl4MaGo3Hu3n+Q4Wjc9VIAYKZ6XS8AWG3D0Th37g1yPplmu7eV46N+DvZ2ul4WAMyEnWXgRganZzmfTDOtycPJNIPTs66XBAAzI5aBG+nv72a7t5VbJbnd20p/f7frJQHAzBjDAG7kYG8nx0f9DE7P0t/fNYIBwFoRy8CNHeztXDmSh6OxsAZgZYhlYGG8GRCAVWNmGVgYbwYEYNWIZWBhvBkQgFVjDANYGG8GBGDViGVgoa7zZkAA6JoxDAAAaBDLAE9pOBrn7v0HGY7GXS8FgDkxhgHwFByDB7AZ7CwDPAXH4AFsBrEM8BQcgwewGYxhADwFx+ABbAaxDPCUHIMHsP6MYQAAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBVtBwNM7d+w8yHI27XgrAWut1vQAArmc4GufOvUHOJ9Ns97ZyfNTPwd5O18sCWEt2lgFWzOD0LOeTaaY1eTiZZnB61vWSANaWWAZYMf393Wz3tnKrJLd7W+nv73a9JIC1ZQwDYMUc7O3k+KifwelZ+vu7RjAA5kgsA6ygg70dkQywAMYwAACgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBlshwNM7d+w8yHI27XgoAcQc/gKUxHI1z594g55NptntbOT7qu0sfQMfsLAMsicHpWc4n00xr8nAyzeD0rOslAWw8sQywJPr7u9nubeVWSW73ttLf3+16SQAbzxgGwJI42NvJ8VE/g9Oz9Pd3jWAALAGxDLBEDvZ2RDLAEjGGAQAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoGEmsVxKeUcp5ZdLKQ9KKR+45PEvKqX8wMXjP1NKeWEW1wUAgHm6cSyXUm4luZvkG5N8ZZJvLaV85Rue9t4k41rrVyT5h0n+/k2vCwAA8zaLneW3J3lQaz2ttZ4n+XCSd7/hOe9O8qGLj38oydeVUsoMrg0AAHMzi1h+c5LPPPb5Kxdfu/Q5tdZJkl9PsjuDawMAwNws1Rv8SikvllJOSiknr732WtfLAQBgw80ill9N8pbHPn/u4muXPqeU0kvye5KcvfEb1VpfqrUe1loPn3nmmRksDQAAnt4sYvnjSd5WSnlrKWU7yXuSvPyG57yc5NsuPv4LSX6y1lpncG0AAJib3k2/Qa11Ukp5f5KPJbmV5IO11k+WUr4ryUmt9eUk35vkn5dSHiT5tTwKagAAWGo3juUkqbV+NMlH3/C1v/PYx/83ybfM4loAALAoS/UGPwAAWCZiGQAAGsQywBobjsa5e/9BhqNx10sBWEkzmVkGYPkMR+PcuTfI+WSa7d5Wjo/6Odjb6XpZACvFzjLAmhqcnuV8Ms20Jg8n0wxOP+94ewCeQCwDrKn+/m62e1u5VZLbva3093e7XhLAyjGGAbCmDvZ2cnzUz+D0LP39XSMYAE9BLAOssYO9HZEMcAPGMAAAoEEsAwBAg1gGoHPOgwaWlZllADrlPGhgmdlZBqBTzoMGlplYBqBTzoMGlpkxDAA65TxoYJmJZQA65zxoYFkZwwAAgAaxDAAADWIZAAAaxDIA1+IGIsAm8QY/AK7MDUSATWNnGYArcwMRYNOIZQCuzA1EgE1jDAOAK3MDEWDTiGUArsUNRIBNYgwDAAAaxDIASRwJB3AZYxgAOBIOoMHOMgCOhANoEMsAOBIOoMEYBgCOhANoEMsAJHEkHMBljGEAAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAZiL4Wicu/cfZDgad70UgKfW63oBAKyf4WicO/cGOZ9Ms93byvFRPwd7O10vC+Da7CwDMHOD07OcT6aZ1uThZJrB6VnXSwJ4KmIZgJnr7+9mu7eVWyW53dtKf3+36yUBPBVjGADM3MHeTo6P+hmcnqW/v2sEA1hZYhmAuTjY2xHJwMozhgEAAA1iGYCV4Tg6YNGMYQCwEhxHB3TBzjIAK8FxdEAXxDIAK8FxdEAXjGEAsBIcRwd0QSwDsDIcRwcsmjEMALgCJ3HAZrKzDABP4CQO2Fx2lgFYO7PeBXYSB2wuO8sArJV57AK/fhLHw8nUSRywYcQyAGvlsl3gm8aykzhgc4llANbKvHaBncQBm0ksA7BW7AIDsySWAVg7doGBWXEaBgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABouFEsl1J+bynlx0op//nivzuN5/1mKeUTF/9evsk1AQBgUW66s/yBJD9Ra31bkp+4+Pwy/6fW+tUX/951w2sCAMBC3DSW353kQxcffyjJN9/w+wEAwNK4aSx/Wa31sxcf/7ckX9Z43heXUk5KKYNSiqAGAGAl9J70hFLKjyf58kse+s7HP6m11lJKbXybvVrrq6WU/SQ/WUr5xVrrf7nkWi8meTFJnn/++ScuHgAA5umJsVxr/frWY6WU/15KebbW+tlSyrNJPtf4Hq9e/Pe0lPLvknxNks+L5VrrS0leSpLDw8NWeAMAwELcdAzj5STfdvHxtyX5kTc+oZSyU0r5oouP35Tka5N86obXBYCZGI7GuXv/QYajcddLAZbQE3eWn+C7k/xgKeW9SUZJ/mKSlFIOk/y1WutRkj+Q5HtKKdM8ivPvrrWKZQA6NxyNc+feIOeTabZ7Wzk+6udg79JTUIENdaNYrrWeJfm6S75+kuTo4uN/n+QP3uQ6ADAPg9OznE+mmdbk4WSawemZWAZ+G3fwA2Bj9fd3s93byq2S3O5tpb+/2/WSgCVz0zEMAFhZB3s7OT7qZ3B6lv7+rl1l4POIZQA22sHeTmeRPByNhTosObEMAB3w5kJYDWaWAaADl725EFg+YhkAOuDNhbAajGEAQAe8uRBWg1gGgI50+eZC4GqMYQAAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGACBJMhyNc/f+gwxH466XsjTc7hoAgAxH49y5N8j5ZJrt3laOj/puxx47ywAAJBmcnuV8Ms20Jg8n0wxOz7pe0lIQywAApL+/m+3eVm6V5HZvK/393a6XtBSMYQAAkIO9nRwf9TM4PUt/f9cIxgWxDABAkkfBLJJ/O2MYAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQBmaDga5+79BxmOxl0vBZiBXtcLAIB1MRyNc+feIOeTabZ7Wzk+6udgb6frZQE3YGcZAGZkcHqW88k005o8nEwzOD3reknADYllAJiR/v5utntbuVWS272t9Pd3u14ScEPGMABgRg72dnJ81M/g9Cz9/V0jGLAGxDIAzNDB3o5IhjViDAMAABrEMgAANIhlAABoEMsAANAglgEAoEEsA8CScwtt6I6j4wBgibmFNnTLzjIALDG30IZuiWUAWGJuoQ3dMoYBAEvMLbShW2IZAJacW2hDd4xhAABAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYA4FqGo3Hu3n+Q4Wjc9VLmzu2uAQC4suFonDv3BjmfTLPd28rxUX+tb8duZxkAgCsbnJ7lfDLNtCYPJ9MMTs+6XtJciWUAAK6sv7+b7d5WbpXkdm8r/f3drpc0V8YwAAC4soO9nRwf9TM4PUt/f3etRzASsQwAwDUd7O2sfSS/zhgGAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAaADbRJtyuGm3B0HABsmHW7XfFwNN6YM38ft6mve9HEMgBsmMtuV7yqsbVu4X9Vm/q6u2AMAwA2zDrdrviy8N8Em/q6u2BnGQA2zKrcrvgqYwavh//DyXTlw/86NvV1d6HUWrtew6UODw/ryclJ18sAgJWxTjOs1xkzWKfXfR2b+rrnoZQyrLUeXvaYnWUAWAPrNsN6nbnqg72dlX6tT2tTX/eimVkGgDWwbjOs6zRXfV2O9VsudpYBYA10PcM665GAVZmrnrV1+wvBOhDLALAGuozLeQXeJo4ZrNOxfutCLAPAmugqLgXe7HT9FwI+n1gGAG5E4M3Opo6fLDNHxwEAN+YYM1aZo+MAgLnaxPliNoOj4wAAoEEsAwBAg1gGAIAGsQwAAA1iGQCAuViHW3c7DQMAaJr1kXDrdsTcur2eWVqXW3eLZQDgUrOOnXWJp9etyuvpKuive2fHZf3FQywDAJea9W2s1+222KvweroM+uvc2XGZf/EQywDApWZ9G+t1uy32KryeLoP+OrfuXuZfPMQyAHCp68ROF9+va6vweroO+qve2bHrdX4hpdba9RoudXh4WE9OTrpeBgDASlvWWeA36nKdpZRhrfXwssfsLAMArLGr7u52bVnX6ZxlAABoEMsAANBwo1gupXxLKeWTpZRpKeXSOY+L572jlPLLpZQHpZQP3OSaAADztg53nmM2bjqz/EtJ/lyS72k9oZRyK8ndJN+Q5JUkHy+lvFxr/dQNrw0AcGVXfQPZMp/5y+LdKJZrrZ9OklLKF3ra25M8qLWeXjz3w0nenUQsAwALcZ0AXuYzf1m8RcwsvznJZx77/JWLrwEALMRlAdzy+pm/t0qW7sxfFu+JO8ullB9P8uWXPPSdtdYfmeViSikvJnkxSZ5//vlZfmsAYINd56YXq3CzERbnibFca/36G17j1SRveezz5y6+dtm1XkryUvLopiQ3vC4AQJLrB/CynvnL4i3ipiQfT/K2Uspb8yiS35PkLy3gugAAv0UA8zRuenTcny2lvJLkjyb50VLKxy6+/vtKKR9NklrrJMn7k3wsyaeT/GCt9ZM3WzYAAMzfTU/D+EiSj1zy9f+a5Jse+/yjST56k2sBAMCiuYMfAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAOZsOBrn7v0HGY7GXS+Fa+p1vQAAgHU2HI1z594g55NptntbOT7q52Bvp+tlcUV2lgEA5mhwepbzyTTTmjycTDM4Pet6SVyDWAYAmKP+/m62e1u5VZLbva3093e7XhLXYAwDAGCODvZ2cnzUz+D0LP39XSMYK0YsAwDM2cHejkheUcYwAACgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGkqttes1XKqU8lqS0Rwv8aYkvzrH789m8fPErPhZYpb8PDFL6/zztFdrfeayB5Y2luetlHJSaz3seh2sBz9PzIqfJWbJzxOztKk/T8YwAACgQSwDAEDDJsfyS10vgLXi54lZ8bPELPl5YpY28udpY2eWAQDgSTZ5ZxkAAL6gjYvlUsoHSymfK6X8UtdrYbWVUt5SSrlfSvlUKeWTpZTv6HpNrK5SyheXUn62lPLzFz9Pf6/rNbH6Sim3Sin/sZTyr7teC6utlPIrpZRfLKV8opRy0vV6FmnjxjBKKX88yW8k+b5a61d1vR5WVynl2STP1lp/rpTyu5IMk3xzrfVTHS+NFVRKKUm+pNb6G6WU20l+Osl31FoHHS+NFVZK+ZtJDpP87lrrO7teD6urlPIrSQ5rret6znLTxu0s11p/Ksmvdb0OVl+t9bO11p+7+Ph/Jfl0kjd3uypWVX3kNy4+vX3xb7N2M5ipUspzSf5MkntdrwVW2cbFMsxDKeWFJF+T5Ge6XQmr7OJP5p9I8rkkP1Zr9fPETfyjJH8rybTrhbAWapJ/W0oZllJe7HoxiySW4YZKKb8zyQ8n+Ru11v/Z9XpYXbXW36y1fnWS55K8vZRiVIynUkp5Z5LP1VqHXa+FtfHHaq1/OMk3JnnfxVjrRhDLcAMXs6U/nOS41vovu14P66HW+j+S3E/yjq7Xwsr62iTvupgz/XCSP1VK+RfdLolVVmt99eK/n0vykSRv73ZFiyOW4SldvCHre5N8utb6D7peD6utlPJMKeVLLz7+HUm+Icl/6nZVrKpa69+utT5Xa30hyXuS/GSt9S93vCxWVCnlSy7eyJ5Sypck+dNJNuZUsY2L5VLK9yf5D0l+fynllVLKe7teEyvra5P8lTzasfnExb9v6npRrKxnk9wvpfxCko/n0cyy476AZfBlSX66lPLzSX42yY/WWv9Nx2tamI07Og4AAK5q43aWAQDgqsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADf8fDguDvKEe9NQAAAAASUVORK5CYII=\n", 117 | "text/plain": [ 118 | "
" 119 | ] 120 | }, 121 | "metadata": { 122 | "needs_background": "light" 123 | } 124 | } 125 | ] 126 | }, 127 | { 128 | "cell_type": "markdown", 129 | "source": [ 130 | "This resembles a sine curve but not exactly because of the noise. \n", 131 | "\n", 132 | "Let’s try to estimate the sine function using polynomial regression with powers of x from 1 to 15. \n", 133 | "\n", 134 | "Let’s add a column for each power upto 15 in our dataframe. " 135 | ], 136 | "metadata": { 137 | "id": "q4N-tDcCK1xQ" 138 | } 139 | }, 140 | { 141 | "cell_type": "code", 142 | "source": [ 143 | "for i in range(2,16): #power of 1 is already there\n", 144 | " colname = 'x_%d'%i #new var will be x_power\n", 145 | " data[colname] = data['x']**i\n", 146 | "print(data.head())" 147 | ], 148 | "metadata": { 149 | "colab": { 150 | "base_uri": "https://localhost:8080/" 151 | }, 152 | "id": "dI3Do9x-MdEU", 153 | "outputId": "46e921a0-21cb-4320-d7c9-dcd82f84b2bc" 154 | }, 155 | "execution_count": 2, 156 | "outputs": [ 157 | { 158 | "output_type": "stream", 159 | "name": "stdout", 160 | "text": [ 161 | " x y x_2 ... x_13 x_14 x_15\n", 162 | "0 1.047198 1.065763 1.096623 ... 1.821260 1.907219 1.997235\n", 163 | "1 1.117011 1.006086 1.247713 ... 4.214494 4.707635 5.258479\n", 164 | "2 1.186824 0.695374 1.408551 ... 9.268760 11.000386 13.055521\n", 165 | "3 1.256637 0.949799 1.579137 ... 19.486248 24.487142 30.771450\n", 166 | "4 1.326450 1.063496 1.759470 ... 39.353420 52.200353 69.241170\n", 167 | "\n", 168 | "[5 rows x 16 columns]\n" 169 | ] 170 | } 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "source": [ 176 | "Now that we have all the 15 powers, let’s make 15 different linear regression models with each model containing variables with powers of x from 1 to the particular model number. For example, the feature set of model 8 will be – {x, x_2, x_3, … ,x_8}.\n", 177 | "\n", 178 | "First, we’ll define a generic function which takes in the required maximum power of x as an input and returns a list containing – [ model RSS, intercept, coef_x, coef_x2, … upto entered power ]. \n", 179 | "\n", 180 | "Here RSS refers to ‘Residual Sum of Squares’ which is nothing but the sum of square of errors between the predicted and actual values in the training data set.\n", 181 | "\n", 182 | "Note that this function will not plot the model fit for all the powers but will return the RSS and coefficients for all the models. \n", 183 | "\n", 184 | "Now, we can make all 15 models and compare the results. For ease of analysis, we’ll store all the results in a Pandas dataframe and plot 6 models to get an idea of the trend. " 185 | ], 186 | "metadata": { 187 | "id": "2S5LF7KeMeHG" 188 | } 189 | }, 190 | { 191 | "cell_type": "code", 192 | "source": [ 193 | "#Import Linear Regression model from scikit-learn.\n", 194 | "from sklearn.linear_model import LinearRegression\n", 195 | "import warnings\n", 196 | "warnings.filterwarnings(\"ignore\")\n", 197 | "def linear_regression(data, power, models_to_plot):\n", 198 | " #initialize predictors:\n", 199 | " predictors=['x']\n", 200 | " if power>=2:\n", 201 | " predictors.extend(['x_%d'%i for i in range(2,power+1)])\n", 202 | " \n", 203 | " #Fit the model\n", 204 | " linreg = LinearRegression(normalize=True)\n", 205 | " linreg.fit(data[predictors],data['y'])\n", 206 | " y_pred = linreg.predict(data[predictors])\n", 207 | " \n", 208 | " #Check if a plot is to be made for the entered power\n", 209 | " if power in models_to_plot:\n", 210 | " plt.subplot(models_to_plot[power])\n", 211 | " plt.tight_layout()\n", 212 | " plt.plot(data['x'],y_pred)\n", 213 | " plt.plot(data['x'],data['y'],'.')\n", 214 | " plt.title('Plot for power: %d'%power)\n", 215 | " \n", 216 | " #Return the result in pre-defined format\n", 217 | " rss = sum((y_pred-data['y'])**2)\n", 218 | " ret = [rss]\n", 219 | " ret.extend([linreg.intercept_])\n", 220 | " ret.extend(linreg.coef_)\n", 221 | " return ret\n", 222 | "\n", 223 | "#Initialize a dataframe to store the results:\n", 224 | "col = ['rss','intercept'] + ['coef_x_%d'%i for i in range(1,16)]\n", 225 | "ind = ['model_pow_%d'%i for i in range(1,16)]\n", 226 | "coef_matrix_simple = pd.DataFrame(index=ind, columns=col)\n", 227 | "\n", 228 | "#Define the powers for which a plot is required:\n", 229 | "models_to_plot = {1:231,3:232,6:233,9:234,12:235,15:236}\n", 230 | "\n", 231 | "#Iterate through all powers and assimilate results\n", 232 | "for i in range(1,16):\n", 233 | " coef_matrix_simple.iloc[i-1,0:i+2] = linear_regression(data, power=i, models_to_plot=models_to_plot)" 234 | ], 235 | "metadata": { 236 | "colab": { 237 | "base_uri": "https://localhost:8080/", 238 | "height": 297 239 | }, 240 | "id": "mVVO9UeSMej-", 241 | "outputId": "d1af115a-0f8c-4966-8581-ac529bc4bb50" 242 | }, 243 | "execution_count": 3, 244 | "outputs": [ 245 | { 246 | "output_type": "display_data", 247 | "data": { 248 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEYCAYAAAD8hukFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3xUVfqHn/femUnoLQgCgQAiTQwlBFAQxAKCim0VsICi/HTdXdfe11173bWsriIqKESwizQrSJSOgPQeDKB0BASSzNzz++PeIZMhPZNMO8/nM2Rm7r3nvsn9ct5z3vOec0QphUaj0Wg0kYARbgM0Go1Go/GjnZJGo9FoIgbtlDQajUYTMWinpNFoNJqIQTsljUaj0UQM2ilpNBqNJmKIOKckIrNF5MYQlSUi8o6I7BeRhaEoUxOZaN1oyorWTGQSFqckIlkiclREDovIThEZJyI1y1hGiogoEXEVc1pv4DygmVIqvUJGRykicrKITBGRHc7fKyXcNpUXrZuqQ0TOFpEVInJARPaKyKci0jTcdpUVrZmqRUQaikiGiPzuOOiJZS0jnD2li5RSNYGuQBrwUCXcowWQpZT6o6wXliDAiKCUNlrATODySjanqtC6qSCltHE1MEApVRdoAmwA/lephlUeWjMVpAw2fgL8BjQHTgKeL+u9wh6+U0ptB2YApwUfExFDRB4Ska0isktE3hWROs7hOc7PA04rqFfQtaOAsUAv5/i/nO9vEpGNIrLP6UE0CbhGicitIrIB+z9hsD3+FtNop+fxq4jcFXA8QURedI7tcN4nOMe+F5HLnfdnOuUMdj6fIyLLAsq5QUTWOC2NL0WkRWltLOTvu1Mp9RqwqKRzowmtmyrRzY6Ar3zAKSVdF8lozVSuZkTkfCAZuFsp9btSKk8ptbSk605AKVXlLyALONd5nwysAh5zPs8GbnTe3wBsBFoBNbG98HvOsRRAAa5i7jMS+CHgc39gD3aLKQF4BZgTcFwBXwP1gWqFlOe/5/tADaATsDvgd3kUmI/dQmgIzA34vR4FXnHePwBsAp4JOPaS836I8zu3B1zYrbq5xdkIHAB6l/A3dznXpoTjmWvdRJ9usFu7B7B723nAyHBrQGsmcjUD/AP4EpgA7MVuCPct8zMLo1AOO7/gVuC1gF86UCjfAn8OuK6t85/DVU6hvAU8G/C5plNeSsBD6F9Mef57tgv47lngLef9JmBQwLEB2F16gHOAn533M4EbgfnO5++By5z3M4BRAWUYwBGgRWlsLMb2WHFKWjdVqBvn2vrAvUDPcGtAayZyNQOMca4ZBbiBoc7fPakszyyc4btLlFJ1lVItlFJ/VkodLeScJthC8rMVWySNynnPAuUppQ5je/TAAdzsUpQTeM5Wp9wTyg86Ng84VUQaAZ2Bd4FkEUkC0skPEbQAXhJ7gPkAsA+QctgYq2jdVLFulFL7gPHA5xIF4x+FoDVTNZo5iu0Y31J26G6Sc/2ZZSgj/GNKJbAD+w/npzngBXZie+QKlSciNYAGwPaAc0pTbnKQTf7Ye2H27gBQSh0BlgC3ASuVUrnYXe47gE1KqT3ONdnA/zn/ifyvakqpuWW0MZ7Rugm9blzYoaLaFSgjktGaqbhmfi7k/DL/7SLdKb0P3C4iLcVO43wSmKyU8mLHVy3sGHBZyrteRDo7g4JPAguUUllltOthEakuIh2B64HJAeU/JHZaZBJ2jHVCwHXfA39xfoIdPgj8DPA6cL9TNiJSR0T+VEb7CiAiidhxbYAE53Mso3VTQd2IyGUi0tZJAGgI/BtY6vSaYhGtmYrXNZ8C9URkhIiYInIF0Az4sUylhDqGW8rYYxbOgF0hx2aTH+c1sP/Y2djCmADUCzj3Uef7AxQS7yYozut8dzN2PHYfMBV7XoH/mAJOKcbuFOec0ditkt+AewKOJwIvA786r5eBxIDjA5zr+zqfT3M+XxV0n2uBFcBB53d/uzgbsWPmfYqxWwW/wvHctW6iRzfAX4EtwB+OvZNwxhqi6aU1U+V1TR+nvMPA4uLOLeolTkGaUiD2xNMtgFvZLSiNpkS0bjRlJZ41E+nhO41Go9HEEdopaTQajSZi0OE7jUaj0UQMuqek0Wg0moghLBPhkpKSVEpKSjhurXFYsmTJHqVUw3DbURa0bsJPtOlGayYyKItuwuKUUlJSWLx4cThurXEQka0lnxVZaN2En2jTjdZMZFAW3ejwnUaj0WgihohwStn7joTbBE08kr0QMl+wf2o0pUFrptIJ++KK7y/8hX9OWcWY69Loe2rUhKo10U72Qhh/MfhywfTAiCmQHLcbhmpKg9ZMlRD2ntKAjo1p3bAmN41fzLdrdobbHE28kJVpVy7KZ/9cnqFbwJri0ZqpEsLulOrX8JBxUw/anVyLmycsYebK38JtkiYaKCqMUtrwSkofu7UrJhgmLM2A756wW8K6koldKqIbrZkqIezhO4C61T1MuLEHI99eyK0ZP/HiVZ25KLVJyRdqohYReRu4ENillDphe+piKSqMEvz9wKfh6F67MgkMs2QvtFu9/uO/b4Ml4/NbwFmZOiwToYRVNwCdh2JvOYTWTCUREU4JoHaim3dH9eCGdxZx26Sl5PksLuvarHQX+yuZwkSkiVTGAf/F3oCsbASGUbw5MPsp6Hd/ge+VNwc17U6UsvBh8k3CucyteR5N61bjpi23Y6o8xF8xASx7P79SSukTyt9TE1rGERLdHLPDb8npJ+pp+p1gWWAYMOgFSBtZuOMyPVozlUDEOCWAmgkuxt3QnRvHL+bOD5fj9Smu7J5c/EV68DEqUUrNcVZCLjv+MIo3B7Bg82zYOg8GPo0y3VhesBQYyocpCgOLgTkzOD/nK77f2xVULiIWPm8Oh9fMos7599q60Q2biKfCujFM8PkAZYffUoejWvRGmW7EBwpBLB+CQlkWMu0O+G25fX3geNLRvVozlUREOSWA6h4Xb4/szv+9t4R7Pv6ZHJ/FtT1bFH1B8OCj7kbHPsnpdoUw+ynbISkLfLn8unYBP/rO4pjXh7tZZ/60+1WULwdBIYCBj/4sAUOwlEGucjF6TiK9jPXcenYa7qJ0o3visUFyOnS5Bha/AyiU5WXRZ/9lyYGabD16NfXlMPtUTR5zj8OFDxFQyoda/DYKE8Mw7MCdv2eUnF68HrRuykX4nVIhDy7RbfJmfx9fHPqGiZ+vJ897CTf0bln49f5Ws7+nVK2BPWCphRATiMho7I3OaN68ef6B5HQ7ZLd1Hvhy8WJQf8MHXCoWkuDBGHwfMNAO0Sx51260AIIFCsRwcazfE5y0vTsvfrOB79buYkw/i8b7F9ka8o8pgO6JRxlFagYgdRhqWQaWNxevJaTumUY3sVCJblac8x5G8x7sWNua5Ln/QCm7x2RgOyefT/FD7UHU6XkdnZt1948s2XXY8gxAIHVY4eNUWjelJiROqdyDj8UMPHomXMplvlwuSnTxj+nZLNhcjR5nDyl+wLpaA5h5nxZCDKGUGgOMAUhLSyu4pL3TY8r85lO2blrHMNcsTCyw8mxd9LnTPqdxZ2ecwAnb2AVTTw7xyrAuDDqtMRkff0Sdjx5FkYfYXgvMBOg8TPfEo4ziNPOTasOr6h908P3MOU1ySd31OaIsUF66WCsheQAk3wrtuh9v0Chl95oMYOXh2jz/hZf2i3/g9nPbcF6trcj4C21tACydCCOn6ghOBQhVT2kc5Rl8LGHAWpQPN4onPONQmxTeLW/gajsAap4EjVNPdEBaCHHHO7805F/renNX+9MwsucW3mNOGwmNOtiVzNIMsLwFBqcv6HQyPXf8gWee1x5LALui8uUCSg9oxwhZe/5g6BvzaVynA3ffdC3t8tbC+BlFR1mcBo1MvxOUhWEm8H/XjKDRnqb8b/YmRr+3hCeSvmK4Ly+/1+Svd4IjOFo3pSYkTqncg4/FDFj7H6iIYCgLwUJZFmrtVABEDFAKUKUXgo7xRgwi8j7QD0gSkW3AI0qpt8pSxndrd/Lo1NUM6NiIW64ehGxvYz/fonrMyemQOrxQDdTr0B+16D94vTmYSmGJgWF67POLuEZT9VRENylJNXj4og5cdPrJ1K3uAdLzG7NFacbfoHGevzs5nT+lwKVdmvLB4m189eUWrlAmHrF3LJfA8SadCFEuqmxMqdA4bxED1vy2LH8+QONUZOZ9KO8xEHW8RaKUld86MVwlC0HHeCMKpdSwily/bf8Rbp+8nPaNa/PS0C6YhuQ7nswXiu4xFzU4nZyOjPgCtWkOY5ce5MDe3+jVdwi9A68rCt3YqTIqqpsTkqbKqRmXaTC8R3MGdRrNmx/Vo+76j6lb3UPXi26hSUlaA62ZYqgyp1RknDdowPr4TGl/iGXEMBgxBVmegfppAsrKs4cFjnskgS7DCwoI7Ae+c3X+gLUO7cUMSinumLwcy1K8dnVXEt1mwRPKEjoJqhzcyelcfYaX695ayJhvDvBxta84PW9F0ZWHbuzEBmUNtzm6qZvSh79cdzXfrD6Xuz5ajvejY7y9fybpskprppyEP/sOCvZwgmfXL8+AOsmQOhxJHY5alsHaTZtotX8uLrEwXAlI6vD8svwP3B8S9A9Y68luMYOI8ODg9uz9I4eUpBonnlDa0EkRlUN1j4u3RnTn4f++TZuZD6HElz/RNrgs3diJDcoSbitEN+d2SGfa3/rw4jsT6PTtA1jixXAlaM2Ug8hwSpDf1c1emD+7/oRe0xTkohdppxTvffgRvy3/muot+3JL0+4cbyv7HziW/dkfEtST3WKK1OS6xZ9Q0hwSKLZyqFPdzWOdD+Ce60WUhfLlIoVVHnpAO3YojWagSN00rVuNJ7v8jjHbi4GFz5uLsSUT0ZopEyFZkNUZfJwHtBWRbSIyqtyF+Vss/R+0J7pZ3oIP374f1/7pClz97uL5NXW584NleH2OE/I/cP+vJkbByW7+NOHC0HulxBeBC2wWUjnUaX82hunBqwxyceFr3vvEMgL1qsMw8UExunG3PgvDlYAPg1xlMnZbU5QqOJNBa6Z4QpV9V6HBxxMorNcU9PBFhDvOO5UEl8FzX66j2eGV3N5mJ2arswpm1AROgixuUq2O80Yv5R00Lilkk5yOMfILVmRO5V8r6tFnfT1uL2xxkdK2sDWRRWXoJjkdGTEFY0smk3Y044llNdlZew0PDm6PiBQsQ2umUCInfFcYpYjz3nr2KTQ7vILzF98B27yozOeREVPsHpGf0jgcHeeNTiramCipckhO5/Rh3UlxLefl7zaQ3rI+Z56SVHG7NeGlMnWTnI4kpzNSKbKqrWLsD1tIqpXAzX1bh8b2GCfs+ymVSEkhN2BI3c0kiBcTC8ubQ96mOQVPKMzhBFNCKEcToZTm2ZaHgFCuiPDEJZ1o3bAmt09ext7DOaG5hyZ8VJFuHrmoIxeefjJPz1jL58u2h+YeMU7kO6XSkNIHw5WAhUmucvHk6gYczfUVOF6iw9Fx3uikMhoT/lZ0wAZu1TwmLw/twoEjedz78c8njhNooosq0o1hCC9cmUp6y/rc/dHPLMs+UPH7xDiR5ZTKm2jgOBTjnAeZ2/ttxmWfxPXjFvJHjrfA8RIdTil6ZZoIozIaE0W0ojs0qc09A9vyzZpdfLh4W+HX6mSZ6KAKdZPgMnn9mm6cVCuB0e8uZvehoJ621kwBImdMKUQx3nOAFxtu544PljPi7YWMPx9q7Jhnt4QCx5k0sUOoB42LSdm94cyWfLNmJ//6YhV9q2+h0b5F+ceD19bTPe7Ipgp1U7+GhzevS+PS137klfETeaTTPjspa+dqe7FgZdnzKbVmIsgphTDRYEjnprgMg3GTJ+N674niJz9qNMEUk2BjGMJzV6Ry74tjqfvR4yi8iGECkr+AK+hkmXikhMSs9ifX5rWzfPT64V5klhfmPGOvXO9sq4IvR2uGSHJKIZ5QNvj0kzl1/e+YK7z2Yq5FTX4sCr02VXxTWCva0URySh/ubrsbc30eIhb458j5HRKik2XilWDdBNUj/RPX4RN7cq3yOduk+BFDa4ZIckqVsKpumx4X4FvzGl5vLl5Mchr1oE5hJwYKB3QYRnMiQeHl1AFPkbfBDSoP0+VCEFsvhmlP+vZv9qaJXwobknCSsrzeHHwYeEwT8eWBYcCgF7RmiCSnBKGP8SanY478gi2LZ3L/T3XYO9XHxJuOcdKBnws6Ib9wdBhGUxRB4WXj2D72Xf4hGR9k4Gt8JvcMbKd71pqCFDYk0edOZMQUdi77ir/Nq8F5HRpzc4sdWjcBRJZTqgyS02mZnM5tp+9l1PhFPPraOF7Je8RunZieoJ1FdRhGUwSFhJdPTk4ncU9TnvtyHV0OteC8PrpS0QRQ1JBEcjpNk9PpZq7h6TmbOa3HefRO1hOy/URWSngl0qt1A969IZ1TjizD5w1ovfh3FhUTTHfBuQtp1+vQncamiBTi0We1om2jWvxzyiqO5HoLXqNTfeObEtLO7zjvVFol1eCBT1fkz6vUmomDnlIAaSn1qX3Rn8ib+jHgxTDdGME7i4IOw2gKp5Dwsts0ePzS0/jT6/N46dsN3H9Be/uAXktRA8UOSSS6TR6/9DSGv7mAl7/bwL0dD2rNEGdOCeDUtHPYZExm2hcf8jOdeCCxA60a1jxhUcUC6Ew8TTF0T6nPlWnNeCtzC3/q1oxTTqql11LUlIozWidxRbdmvDlnM6PUcpK0ZuInfBdI6679OW/0MyxVbbhqzHw27DxU9MmFLB2iiWOKCK/cO7Ad1T0m//h8lb0EkV5LURNIMWG5+y+wtfPypkYorZn46yn5ae9dy5dpi7lncW2GjlFMvKkH7RrXPvFE3eLV+AkMyQWlfjeomcDdA9vx8Gcrmfrzr1yUGvopDpooJXA3bH/qd9rI44fztePl/PPford7bVxrJi57Sn6RJC18jrH8kwfUmzz5xnhWbv/9xHN1i1fjJ7iBsvidAr3n4enN6dikNlOmfkbu7Ofsa/RaipqsTNshYdlz2abfeUKPaXh6cy5ruJ2Vc6eT0+yMuNZMfDqlgMrF8OVymfUVY9SjPP3muwVX8fWPJQ18Wq8ergnY1di/WZsqsPCmaQjP98zh5dxHcM1+Uod7NTYpfewekh9lnbBVhrl9Ec8deZgb8zIw3hsS17qJT6cUVLkIigTx0ce9lmvGLmBx1r6CY0kz74vr7rTGwZ/imzbSXjyzkN5z+5zleI4vIxPCfXo00Utyuh2yM1z2UkJmwokRl6xMTCsPl1iIL4/D62aFx9YIID7HlPyVS8ByQmJ6uOKyoUye5uO6txcyo+siWuixJE0w/hTfwGkEgbpI6YPh8uDNy8USFx4d7tWA3ZBp1KHoMUanoax8ueQpk/d+bc4tYTE0/MSnU4JCK5cGyelManqMq8cu4N4ltZnocWNa6LEkzYkUNf8kOR0Z8QWLvvuM59Y25M6cVpxZ9dZpIpEStlBnxBQkK5OMbU157udanLvzEG0a1apaGyOA+AzfBRK0sd9JtRN5f3RPDjTowrCcB9jU6e96LElTNpLT6TL8MXbXS+Wxqavx+iw9U19TMk5ddOnFl1HD4+Kjzz+JS81op1QISTUTeP+mnhxp1JWBS9L46mDz/IO6ctGUgkS3yQMXtGftb4f45qupeq6bptTUr+HhkS6H+fv2u1DfPR53mtFOqQjq1fAw8caedGhShz9P/IlpP/+qJ9JqysTA0xpzTdPfqL3geZQ354RtsjWaohgic0iQXETFX8KMdkrFUKeamwmj0umcXJe3J01i55RH7N0hdeWiKQWybRGPHniAdPUzYDmZV3p8UlMC2Qtx/5yBAEqBEjOuNKOdUgnUSnTz3vlChucpGuyah1K6ctGUkqxMDCsPlyh8SjjarI8en9SUTFYmWD4EsESY6T4H1ax7uK2qMkLilERkoIisE5GNInJfKMqMJKrtmItHvMcrlx0NeujKJQTEum6Op/mKSR5uXvJdrjUTAuJFN4iJMhIYe7AHX63eGW6rqowKOyURMYFXgQuADsAwEelQ0XIjipQ+iFO5+AwPf9k+gHG/NAy3VVFNXOjGn+bb/0GmdXmD1zcnMXfTnnBbFdXEk27sVWQ+Z3+DLrz4zQYsS5V8bQwQip5SOrBRKbVZKZULTAKGhKDcyCGgcjFGTCG9ZX12TX+Kz7/4NNyWRTOxrxs4nuZ74eAhNK1bjcenrsEXJ5VLJRFXunG16MnfzjmFNb8e5MtVv4XbqiohFE6pKZAd8Hmb810BRGS0iCwWkcW7d+8OwW2rGEckbtPg3l33cqf7Q85fPJqPPvsk3JZFK/GhG4dEt8kz6Ufpt+s9Zn3zRbjNiWZK1E2saMbPxfW382DtGcycOSUuektVluiglBqjlEpTSqU1bBjFoa+sTMSXi4mFR7xsXjSTf3+93t5DR89hCjkxo5vshZw5dxR3uj+k99xR/LFpboFjWjehI2Y0A5C9EPO9IYzKncjThx9i7vfTCxyLRd2EYpmh7UBywOdmznexiX8Q0peLYXpo1SSZzbOfZ+aOVgzMfhGJ862My0B86SagMaOUlwWzp9C39Rl62/SyE3e6wZeLgYVbvKybP4Mz+g7C2L4oZnUTCqe0CGgjIi2xxTEUGB6CciMT/yBkViZy7CCXz3sR5fbh22ygxEICtzOIEZFUEvGlm4DGjCUuvtqSR+qXT1M3b6feRLJsxK1uxHCz4ZCHdR/9i/bVf49Z3VTYKSmlvCLyF+BLwATeVkqtqrBlkYz/4b9zAWJ57Q0wxMKnDJQoDNOD6DlMxRJ3uglozOT8vpd/Lnodc54Fptve0kAv/Fsq4lY3yzOQQ7t4dN27mKt9KNOFxKhuQrJKuFJqOjC9xBNjiaxMsKzjH8Uw+Tr5LlZu3ELNVv24uWl3PTO5BOJON05jptY7F6DEnhypfF4kbQTUSdZ7dpWSuNMNwLJJGN5juFH2pFqfD0m7LiZ1E79bV1SUlD7gSrC3OTYMZNALXNBtBGu+Xs+z321k40fLee6KVExDSi5LEz84jRkBFOBDkE7DMFv0CLdlmkjFv1M2duadhZCLiafTMIwY1I12SuUlIBzjb6kIcOf5bXGbBv/+ej1en+LfV6biMnWfSeMQ0JhRYvBQzgg6/daYq1uE2zBNxFJgXMlka/Kl3LGuPdcfSOaiGNSNdkoVIXDTruyFxx3U385Jx+MyeHrGWvJ8Fi8N7YLHpR2ThoKJMi16s3mGxZdfruPCTk2oU90dbus0kUhQAzi5aXcOvTiHl7/dwKBOJ8dcNEbXlKEgeEuLxeO42fiMV3rnMWPlb/x54hJyvL5wW6mJFJyJ2NK8B8+kH2N47kd88NnH4bZKE8kEbEZqbl/Ey8mzqLX7J6at+DXcloUc3VMKBf6Yr/LZY0zT7wTL4iLDIKnrAzy3dD1TXsng4kuuJKFlr3Bbq4kUshfSctow7nDlkLvuU379dj8ne47E3MC1JoQ4DeB2vlwyEkxenb4L6/ckjJaxoxntlEJBQMwXEbB8gALLoteaJ/iwmok64MX37kSOXfs5ia20Y9JwvDFjYuGWPBpmPogSEP9kSP852klp/DiaEeXDg+Jvx96A75Q9TumfQBswlBCNutFOKRQExnyrNXB6Sl77mLIwlQWiwMrjg08mcfFfu1MzQf/p457AAWzE2UNHgfcY/PgibPwuJmfsaypAoGZEMCwfBgrlzUFmPwXth8DM+6JaN7pmDBWBSQ9gOyZl2RMjEbC8iOnmk/0t+eitBYy7IZ3aiXpgO64JTHpIrI932l0YeAGFrJsJKFtDMTZjX1MBAjVTrQHMuBevNwdTLNg8G7bMsTUTxbrRTqkySBsJjTrkd6EBsjIxU/pw4+/J/PX9pVw7dgHv3tBDZ1zFO05jxgAObVpMvTUTMPz7YBsGIDE3Y19TQQIawMZJ7Vn63n2k5i21IzKKqNeNdkqVRXDPyXk/MBlev8bglgk/MezN+Uy4sQf1a3jCZKQmkmhw5gjy1n2IZeVhuDwYFzwDR/dG7diApvKR5j2QfveR+9U1JIgPw+WBgU9HtW60U6psggcdsxdyzp5MJg86jaEzDjN0zDwm3NiDk2olhttSTbhJTif3ms8YnzGBDdU782yX63DrideaEuh8xvk8/NPTNN6/mFHDrqNa6+hOpNKKr0wKmb/k/9xl1gg+HGySve8oQ8fM57ffj4XbWk0EUKP1GbS69GE+3d2UsZlbwm2OJgoQES4bchnPH7mQMVuSwm1OhdFOqTIJnL/ky4U1nxf4fLp3BeNvSGfn78e4asw8th84Gm6LNRHAgI6NuaX1Ho58+yzbV8wOtzmaKKBr83rc0nov1pzn2b/uh3CbUyG0U6pM/OmbYto/2w8p+DmlD+kt6/PejT3YdziXq96YR/a+IxW7Z4zuRhlPyLZF3LPzHm4zPyDp4yuxti7Qz1VTPNkLuXvn3fxVPqDGpMvydRKFutFjSpVJIYu2FsjKcwYhuzavx8SbenDtWwu56o15TLypJy2TapT9fnoX09ggKxPx5eHCApXHuq/G0H7nNP1cNUWTlYnhy8MQC6+Vx56V35IEUVkf6J5SZROwZlWhnx1Ob1aXjJt6cMxrcdUb89i463DZ7xUcLszKDMEvoKlynB62EhOf4WZZ9gGUfq6a4gjQTJ64eHVLY9SW6KwPtFOqSkroSndsUodJo3tiKRg6Zh7rfjtUtvKDw4VROEdBw/EetvR/kD+u+pgZZj9ylYmqzOcahWEeTQABmvm2+5u888tJLFAdKr8+qATd6PBdVREcWitiLsGpjWox+f96MvzN+QwdM4/3RvXgtKZ1SnePwsKFmujEmedWHxh++SkMm+jltlN20vf8y0L/XHXYNzZwNDPQZ9F2/Q/cOc/Lt8M/JXH73MqpDypJN7qnVFUUtpK4P1U8qJXRumFNPvi/XlT3uBj+5nyWZx8o/X2KCA9qopeBp53MKd36M3JjH3441ir0N9Bh35jCZRo8OqQj2w8c5T9r61ZefVBJutFOqaoIDK0ZhrM+VdEPs0WDGkwa3ZM61d1cM3YBS7buD4PRmoggeyGPJ33FRfW28ffJy9h1MMRz2nTYN+bo4d7E6y1ms+THL1m14/fKuUkl6UY7parCH2effAgAACAASURBVFrr/yAMegHMhBIfZnL96kwe3YukWglc99YCFmzeW/r76TGC2MAJkXi+f4oXc/9Bm5zV3JrxE7leK3T3CNSmDt1FP45mBux6iwnuJxibMblyNhmtJN3oMaWqJHA9vOAFWzNfKDTu2+TQCr7ovID7f6rDyHdg7Ig0zjwlaNZ2IUsZ6TGCGCEgRGL44MnWK/lww0rGTf6F0VcPDd19gtdq1EQvgXsuCXQ7MJN54zbTb0AljEdWgm60UwoX/ge5PAOWZtj7LwUnQOxcDdPvpKayeNn0cHvtx7hhnOKNa7vRr+1J9vWLx+Vvk2Em5Cc6BMd6dYUTnQRuIGmYtMz+jLvceeSs/5Rp0xIZPPiScFuoiTQCNGMYJle55yDZs/CNewdz5BcRXxdopxQu/L0Z7zHs9ebJT4BQCgzT3sFW2d1u8eXyZNcDrF/ZntHvLuG1q7tybq2tBTcU9OXk95j8FZkeI4huAjMqf98GS8ZjYOERL6vnTceV0pMBHRtXyq2VUohIpZStqUSCNONaMh4RC68vl4NrZ1E7wp2SHlMKF/7ejN8hAQj5jsjf0zl+zKD6qf14/6aeXJK0jWUZD7P1u7FgBY4tiF1xgS3KbtdB52F2j0uPL0Uv/ozK1GHHB5YNVwJ7ktL5a8ZS5m7cE7p7OWORB9b/wCWv/qgTbKKVAM2If1KtcvHoinr8keMN7b1CPH5doZ6SiPwJ+CfQHkhXSi0OhVFxQWBvRsTuHSmLAk4KALF7TYNeAKDOd3fzzOEMLFce3s0GPtOFaXntMsSAJeNh2ft2GHDZJLv3hWUf84f3IrylpCmCwF1HU/rw0LE82n70Pq+M3wAjhnNG8FhjWXF678qXSzXloprvQRJcnUJiuiZM+CfVZmWySjqyZfoaZvzvboZcchXulJ4VL78Sxq8rGr5bCVwGvFHBcuKPQsIytkMS8h2TAa37Qb/77d7OOxeA5UNQmNinTsrtS+ppnTit5kG7jOAVyXF6UlG8PbImgICxyFpLM7je8jLcNBkxTvHH8KGc16GRXVEszwDE7l0V9byDE2SyMlHOALmpFP/otJ8OpZ24rYlcnGSEtMXj+DDhcdR+H77xE8kbMSXfMfm1UK1ByRsEBuomQDPKl4uEoH6pkFNSSq0BdNy5vPgzV7IX2r0bZzAbJD/xod/99rmBY0cACIYrgfUNL+Sh5Y1482wf55rv57dY2g+BrfOCekp6fCnqCRqLFCBBYESNeSzNWEXiaW3oveFZxJdrn790IoycemJFUUiCjNW8N15cGEqhTDcdeg2q4l9OU2lkL4Tpd2IoLwgoK5fvPniJM7vNoUadk2DmfSVHVfyNnaUZKMuLV1y87B7Fny0TN4o8ZfKPxbVJS/yFIZ2bkug2y2VqlSU6iMhoYDRA8+bNq+q20UHw8kBQsAWb+ULBsSMxodsIJHUY9zfuRtZ7S7jxu938r+8bXFBz44krkpem9aOJDk4YixTEMLnA+x0DXHlYawQlPo43EwvrHTsVVGCCTO7G7/nrtrPZfex+/tZ6J33Pv1RrJZbIyixQhxhi0PePrzAzZ2KJgYFFsVGV7IWo8RejvMcQpzEkliK51jG+aPc67Y4tZ321VJZtTuLLaWsY0rlpuU0t0SmJyDdAYek9DyqlPi/tjZRSY4AxAGlpacEDJ5rgfP/A9yl9wJVgt2QMwx5fchxOIjDmum7cOvEnbvl+Fw9feBmjklsWXqYm+glKEafN+XDoV2THUkwsRMBSgojzX8z0IP6Gjj/s8vu2AhWUpYQZPy5mzxEXFw6+hL7NdyFbf4Bda3RjJlYIqkOM5HQ8W+chKLyWBSKIM3QgAVGVY5vnsWP51+zK3kSaNwcXCoXdJDJNgyv7pELa5cDlnJ69kMvNDA7n+Ej8rV65NVOiU1JKnVuukjWho7CeVMDgYsKIKbx2dRq3TVrKY1NXk+u1uKVf6/DaXAw6QaYCBGqhWoOCYRecdFrTZIGrOxuPVGeb2YaB0/5Hy4Tnqb1tNmJ5wTBRpgvl89rtYwWDvV9xYbXZmIn14N1ShHLCgNZNBQjWzYy7EcfBmAIWCp8yecs3iMYJubgNg33jXmWo9wtaYNEUE2WYWAgigvgTs2beZzeQAcYNRny51AJYM7nwsHEp0POUooXAXk/mCydMjvUkp/PKsC7c8cFynpm5llyvxd/OOSVSx/t0gkxF8GvBrwMsAhNkDKXo0fs8POZpXPn11bh/zcu/VsDrU0zyns0OlUTrhANcan1th2+svEhPkNG6qQiBurGc+Y/OvyYKMaBHIxcdd3+F6cvFQIHY5xhiId1GQJ3k/MSsQG0A+AJ0VgHNVDQl/FLgFaAhME1ElimlBlSkTE0pKGJyrMs0+M9VnXGbBv/5Zj25Ph93nd824hyTTpAJEcGhvIAEGWnZh65ZmYDXX/OgAAvBEjc10q/h3M5nk6rWY7w3JyoSZLRuQkQRujFMD12a14PdXoKnpogYkDr8xMSsQG2YbqdBQ4U0U9Hsu0+BTytShqYcFLNvkmkIz11xOh6X8OqsTeT5FPdf0C5q/yPrBJliKClBBgpUFGK4ka7X4kkdxqXHNdPjRC1FeYKM1kwJFKcbsB3O8ZBwwDzJwN2zC6t/Rk4r3VSEEhClqj7nIC0tTS1erMPBlYllKf75xSrenbeVkWek8MhFHQo4JhFZopRKq6z7lyZBRkRmA3eVdmxA66YcBM9ZggptAhltutGaKQeFzVmCKtONHlOKUQxD+NfFHfGYBmN/2ILXsnj8kqqbna8TZCKEwLHIKFg9XusmAgjO2q1i3WinFMOICA8Obk+C26B5/erhNkcTbvTq8ZryUMW60Quyxjgiwt0D2nFV98iJrYvIpSKyDeiFnSDzZbhtiguifIdZrZswUcW60T0lTZWjE2TCRDEJMtGA1k2YqGLdaKek0cQTepUPTXmoQt2EJftORHYDWyv5NklACDeaqVTCYWsLpVTDKr5nhdC6OQGtmxLQmjmBcNlaat2ExSlVBSKyuDJTV0NJNNka60TTs4gmW2OZaHoO0WCrTnTQaDQaTcSgnZJGo9FoIoZYdkpjwm1AGYgmW2OdaHoW0WRrLBNNzyHibY3ZMSWNRqPRRB+x3FPSaDQaTZShnZJGo9FoIoaYckoikiwis0RktYisEpHbwm1TSYiIKSJLRWRquG2JV7RuNOUh2nQTLZqJtRUdvMCdSqmfRKQWsEREvlZKrQ63YcVwG7AGqB1uQ+IYrRtNeYg23USFZmKqp6SU+lUp9ZPz/hD2A2gaXquKRkSaAYOBseG2JZ7RutGUh2jSTTRpJqacUiAikgJ0ARaE15JieRG4B3uLR00EoHWjKQ9RoJuo0UxMOiURqQl8DPxdKXUw3PYUhohcCOxSSi0Jty0aG60bTXmIdN1Em2ZizimJiBtbIBOVUp+E255iOBO4WESygElAfxGZEF6T4hetG015iBLdRJVmYmryrIgIMB7Yp5T6e7jtKS0i0g+4Syl1YbhtiUe0bjTlIRp1Ew2aibWe0pnAtdgtgWXOa1C4jdJEPFo3mvKgdVMJxFRPSaPRaDTRTaz1lDQajUYTxWinpNFoNJqIQTsljUaj0UQM2ilpNBqNJmLQTkmj0Wg0EYN2ShqNRqOJGLRT0mg0Gk3EoJ2SRqPRaCIG7ZQ0Go1GEzFop6TRaDSaiEE7JY1Go9FEDNopaTQajSZiiDinJCKzReTGEJUlIvKOiOwXkYWhKFMTGWidaMqD1k3kExanJCJZInJURA6LyE4RGefs3liWMlJERImIq5jTegPnAc2UUukVMjpKcf7jPCgiv4jIQRGZJCK1w21XadA6qTpE5GQRmSIiO5y/V0rQ8edFZIOIHBKRtSJyXXgsLRmtm6qjFLoZJyK5zrPwv8ziygxnT+kipVRNoCuQBjxUCfdoAWQppf4o64UliDEiKKWN12Hv+XIm0ASoBrxSmXaFGK2TClJKGy1gJnB5Ecf/AC4C6gAjgJdE5IzQWFgpaN1UkBDpBuBZpVTNgJevuALDHr5TSm0HZgCnBR8TEUNEHhKRrSKyS0TeFZE6zuE5zs8DjvftFXTtKGAs0Ms5/i/n+5tEZKOI7HM8fJOAa5SI3CoiG4ANhdjjbz2NdloGv4rIXQHHE0TkRefYDud9gnPsexG53Hl/plPOYOfzOSKyLKCcG0RkjRMW+FJEWpTWxkK4CHhLKZWtlDoMPANcJSLVS3FtxKB1Urk6UUrtVEq9Biwq4vgjSqm1SilLKbUAyAR6FXZuJKF1E17dlIewOyURSQYGAUsLOTzSeZ0NtAJqAv91jp3l/KzreN95gRcqpd4CbgbmOccfEZH+wFPAlcDJwFbsPesDuQToAXQoxuyzgTbA+cC9InKu8/2DQE+gM5AKpJPfQvse6Oe87wtsDvgd+jrHEZEhwAPAZUBD7P/87xdno4gcEJHexdgrQe8THPujBq2T458rUyelQkSqAd2BVRUtq7LRujn+OZy6+bPjpJf4HWexKKWq/AVkAYeBA9gP7jWgmnNsNnCj8/5b4M8B17UF8gAXkAIowFXMfUYCPwR8fgu7K+n/XNMpL8X5rID+xZTnv2e7gO+exe6JAGwCBgUcG4DdvQc4B/jZeT8TuBGY73z+HrjMeT8DGBVQhgEcAVqUxsZCbL4RWO/YXgeY4pTRKxzPXuskMnUSUI7LuTalmHPGO7ZJuDWidRPZusEOnzZwjg8CDgFnFldWOHtKlyil6iqlWiil/qyUOlrIOU2wReVnK/Yv16ic9yxQnrLDWXuBpgHnZJeinMBztjrlnlB+0LF5wKki0gi7pfMukCwiSdgtHn+4oAV2vP6AiBwA9mH3bspqo5+3sVtCs7FbtrOc77eVoYxwonVSNTopFSLyHHYo7Erl1DoRitZNBOhGKfWTUmqvUsqrlJoOTMTupRVJ2MN3JbAD+4/opzngBXZie+UKlSciNbC9+PaAc0pTbnKQTTuKsXcHgFLqCLAEuA1YqZTKBeYCdwCblFJ7nGuygf9z/kP5X9WUUnPLaCPOfS1ljwekKKWaYTum7RT8naMdrZMK6qQ0OOMmFwDnK6UOhrLsMKF1UwW6CUJRcDjhBCLdKb0P3C4iLcVO6XwSmKyU8gK7sTM/WpWxvOtFpLMzQPgksEAplVVGux4Wkeoi0hG4HpgcUP5DItLQaaH8A5gQcN33wF+cn2D3XgI/A7wO3O+UjYjUEZE/ldG+44hIfRFpLTYdgH8DjyqlrPKWGYFonVRQJ04ZidjjjQAJzmf/sfuB4cC5Sqm9FblPBKF1U/m6uUJEaoqdVHI+cA32EELRlDV2GIoXdsz33CKOzSY/5mtg/+GzsUUyAagXcO6jzvcHgJ6FlDWSgJiv893N2LHZfcBU7DkG/mMKOKUYu1Occ0Zjt1B+A+4JOJ4IvAz86rxeBhIDjg9wru/rfD7N+XxV0H2uBVYAB53f/e3ibMSOn/cpwuZTgXXYceOtwB3heOZaJ5Gtk4BrCryCjuU4ZfhfD4RbI1o3Ea+bTOB3517LgaElPT9xLtSUArEnhm0B3MpuTWk0J6B1oikPWjc2kR6+02g0Gk0coZ2SRqPRaCIGHb7TaDQaTcSge0oajUajiRjCsihgUlKSSklJCcetNQ5LlizZo5RqGG47yoLWTfiJNt1ozUQGZdFNWJxSSkoKixcvDsetNQ4isrXksyILrZvwE2260ZqJDMqim9gI32UvhMwX7J8aTWnRutGUFa2ZSifi9/QokeyFMP5i8OWC6YERUyA5Lvfb0pQFrRtNWdGaqRKiv6eUlWmLRPnsn1mZ4bZIEw1o3WjKitZMlRD9Timlj91qEdP+Wa2B7l5rSkbrJm6xrHJOg9GaqRLCHr6bP2cGK36YxvCrhlOjdcDuytkL7ZZISp+iu8j+cwY+DUf32iKZeZ/uXscB3q3zObrhe2q1PbvgM9a60RTBzoPHePiVt/lr69/odOaFZdcNQOehgEDjVK2ZSiIkTklE3gYuBHYppU7YdrhIshfS/fuRpHlzkYwPYeQX9oMNjt06lcfhk3sxbX8zFmXtp23eGq7feBumykP8oiise62FEpGUWzMA2QvxvXMx1cmD+f/JrxCK0M2hxj15aV09fty0l+TDK/iv95+4VR7iStC6iTIqopuG+5fzUt4jeNZ6YeOYEnVTwEEFn9MZrZlKIlQ9pXHY2wi/W6arsjIxrDxELHy+XJj9FPS7v2Al4c2B6XdiWRZuDOp4O9PTXR+fUqByEbGwvLkY/laO6ckXTkqfEP16mkpgHOXRDEBWJm7yMLBQ3hykGN0oy8KDQZq3M73qNsZT3cD4PQ/BwufNwdo0B3frs7RuoodxlFM3xi8/4MGLiQXeY7A8w3YkhegGywLDgF5/gcTa5O79BZc3B8PRzfKt++lsujF8aM2EmJA4JaXUHGeF27KR0gcxPXi9x2yhbJ4NW+fZLRV/JSGCZVkYWHiwGOBa7GwTZaJE8CkhV5mM39qEG89Iw+Vv+RbWDS9tF11T6ZRbM3C88eH15mBK0bpRloUE6EYOc1w3ljLIVS6eXd2Ae8/sRqLWTVRQUd0oMVGWBaKQpRmQOvy4npQvFwvA8mKCfd6PL2IBhjLwIhgYeMXF0792AbpwT9vdpPW7uHBdaN2UiypLdBCR0SKyWEQW79692/4yOR1GTGFr7e5YSkBZdoXy2zI7dtttBEtPf4gc5cJyNisU/0v5MJQPwzCZ1fIOnl5Zm5sn/MSxxt2gz52FVyzjL4bvnrB/6sHJqKAo3ciIKfzkSrV1UYhutp3xGMeUC1+AbgBQPls7hsHK0+9nXPZJ3PTuYnJPTtO6iREK1QxAcjq7Wl+Rv/Wp5bV7S1mZHOj7KGNcw3godyQKs8D2qCZgioVLQLqNIHHUNF6992ZqtzmDK1b14tM9TYJN0LqpAFXmlJRSY5RSaUqptIYNA1abSE5nT9od5OJGiQmGCUszYMm7WEszeHKx8ETS01hdRiKG+4RyRSkGtfbw6JCOfLNmJ7dMWEJu1jw7K2bxuPzsGJ3OGZUUpRtp3oP17f5CripEN8syeGS+4raEf5Fz+rWF6gal6H6S4pnLTydzwx7u/2QF6pcFWjcxQJF1DVC753Xk4MEiXzPquydI/PZB5uS246qb/4Hrwn8jhguQ445JAEMUZr1kSE6nYa0E/ndNN3q0rM/kTz5m3+RbYert+c5H66bchD37DuDUbudw9YwHuLvdbno1OApLxoPyYfkU51bbwFWj/oOruge6DrdbNod3w4av7ZaOE8+9LjkFl2Hw0Wcfwy9PoZQ9boAYYCYUDO3oGHBM0K33AK7+6QFubfkb5zTJzdeNV9FJrWDgLc9SvXFt6H5tkbq5MjmZHQeOMufbaXjXPoXbygOtm5il5iln8Lcaj9PHs5Y/nQIsGY8oHy6leLLLAVok14XkkdCog+1Ijh2Eef+1e+NmQoHn7zYNXu/no/r7j+JZ47V7V0snwsipeny7AkSEU6pXw4NKTufWX47wdXoi9Zdl4PMq8pSLi3t1ou6SV/LjsoHZMEHx2uE9mnPqhr0YG+zkCQWIP7RzdG9+ppWO8cYE7U+uTaee5zFq3lY+bG/SzczAcnRzYc+OnLLhTcgrWTe3ndOG1mu3I7vzQCz7PK2bmKXGKb3457IW9G9dg3pLbc0ow02LZsl2Dzm4rmk3uMjnX2/XAqzjQWLye0V97tS6KSch2U9JRN4H+gFJwE7gEaXUW0Wdn5aWpoIXSdy46zAX//cHGtVOpOXRVZyW9zOXnnE6LRc/Vra5ANkLyXvnQsSXiykK8bd49TyCAojIEqVUWhjvXybNQOG6OZLrZdBLmWTtPUJXWc/Zies5t1t72i97sky6ydk8D969GFPlad0UQ7TppjDNbN59mMEv/0C7k2tRa/dPnGGuZeS5XUn85sGyzzvKXogaN9i+DsBMQEZO1ZoJoiy6CVX23bCKlnHKSTV55vLT+dcXq3Cn9OCsPsNomf1O2ecCJKfjGvkFX07/iNm/+PhTh+p061tEdkwwOlumygiFZgCqe1x8dMsZzF63m50H23JFt2Y0Wv5qmXWT0KoXu6/8hMkfZpDjrsutPeuT2Kav1k2EEQrdtGpYkwcHt+ehz1bStG5HHr9pFImr/1e+eUfJ6cjIaWyb/Taz1+2iUa+RnKc1UyEiInzn56LUJlyUGpDJYpYhLhvwkKV5D869qTufTPyJy1fu5KXTmjKEEkSgF1uMWpJqJnBFt2b5X5Qlnh+gm4Yd+pB+XUeuHjufZVsa8HbfNNwlVR5aN1HJ1T2aUyPBpEfLBjSpW63sY0BBumh6TXc+ePVH9i3L5ex283H98qPWTDmJKKd0Ak7KeKmWjQl6yK7kdF4e1oXr3l7IhA8/5MLEpzCtvKJFoGf1xw4V0E16y3SeuLQT93z0M2MzJnPz1tuR4ioPrZuoRES4tEtAQ6a0moFCdSPJ6dx2ThtefTcDxj8NStc15SXyF2RNTi98/kggRaRfJrpNxo5IY3DtTShvCemZwYst6myZ6KYCurkyLZk/92vNobWzUN4crZt4oTSagSJ107/dSVxSbwtYuq6pCJHvlEpDMQ+5dqKbSy69Cq+48CoDy3AXLgJ/S6n/g7o7HS8Uo5u7B7Qloc1ZzsTtYioPrZv4owjdiAgdzhhEntZMhYis8F15B/9K6HrXPbU3e676hI8/mcQPee24z2xLx6LK0QKJPipBNyLCn68dzvNv+TC3/kjnnhdyflFla91EJ5Wgm25nDuDeRU/S6o9lXH/1NSRozZSZyHFKFR38K+ohO8JLSunDoFueY/wb87hm7AImje5F28a1Qme/JjxUom7cWZncef4ZjJ7Vntdm7+axOllc2yslZKZrwkgl6Ua2LeLWlr9x+4JT8G5J4i8tQ2hznBA54bvKWJYjaP2p5D9WknFTTzwug6vHzmfjrsMVv4cmvFSybjwTLuWNfj7ObX8SD3++ijfnbK54+ZrwU4m6abH8P0xKfIofZ81gx4GjFS83zogcp1QZg3+FCC8lqQYZN/UEhOFvzidrzx8Vv48mfFSBbhK2zeW1q7sxuNPJPDF9Df/+ej2hmHSuCSOVrBs3XrrLKv4+aRl5PqviZccRkeOUKmPwrwjhtW5Yk4ybepDns7h67ILCWzPZC/VWx9FAFenG4zJ4eVgXrkxrxsvfbuCJaWsKd0xaN9FBJetGTA9p/S5mYdY+Hpmyqvgt2LVmChCSZYbKSmFLf1QaxQxmrtj2O8PfnE+/Glt4tttBqp3azz6wPMNecdq/cGcMZsiEe7mY8hAJurEsxaNTVzNubhYPnX6IUc22Iy2dVrbWTcRRpZqBE3Tz1Iw1vPH9Zvqe2pBHh3SkRYMaBc/ZudreVNC/4GsMagbCsMxQRFPYgKQjik4pfZg0yKDV9IfxZHpRc5+zF6v35WLvJIie3BavFKEbIyuTRzr3pvmRwwxb81fUei+YLq0bjU2Qbu7reJAL9s/nydVJ9H1uN1ectJ0nDz2ES+WB6cKwfHaYGMCXozVDPDilYIKybjp2HoYlXgwsLF8eAOKvWBA9uU1jE6AbMT1cnzoUFaAbe5VorRtNANkLkXeH0NmXy6REN592+h95m1ZjqDxnW/U8LFH5YyhiaM0Qj04pOPkBheFKwPLmkqcE0xBcWPYGYF2ugdRhcd9y0XCCbkQAVwI+bw5eZeAyDUzl07rR5BOgGcMHl9ffAl2Ho8ZPQvlyURh4lcIUH6ZhIoNe0JohHp1S8MKLqcMhdTiyJZM3sxrz3ZpdPJt2kFPSL9AC0eRTiG4kdThq8xwe/bkeG3ce5qWeh2mcep7WjcamsEVek9MRZ+KtK6UPK37Zz3czP2FvvXQe6zoCM9w2RwDx55SKmI0tyenc2MvHzNfncumKI0w/+zSSw2yqJoIoQjeu5HRu75bDkP/+yOUrYXr/LtQJs6maCKGolR8Cxp26JENW9Y68Mnk5bedvZcQZKeGzN0KInJTwqqSIhRcT3Sb/u7obKPj75GV4/fMLdMqmBorUTVLNBF67uis7Dx7jvk9+zk8V17rRlGKR10s6N6VPmySe/3Id+9Zlxr1m4tMpFUNy/eo8cVknlmzdz39nbTxhVYh4FoumaFKT63LXgLbMWPkbHy3ZpnWjKTUiwr8u7kg73xpqTbo87jUTv06pmFbsxalNuLRLU/773UZ2/vx16Jcj0UQvxehmdJ9WpKfU57Gpqzm8bpbWjSafEnrNrRrWZGST7YiVF/eaiU+nFNiKHTcYpt5+glgeuagDdat7eGZtQ5Te+0QDJerGMISnL+9Ejtfi1S2N9Z45Ghu/br59HN65ABaPK/S0jr0GkUcJ217EAfHplILTwhe/c0J3uW51D49f0pFPdjfl89TX9N4nmlLpplXDmvztnDb8b1MSP509XutGY+vGmwNY9mof0+8stMeU0uVsnm30LP8zhuK99rO41Ux8OiV/qqYz5RFUod3lAbV/4YXG3zB50TZ2dPpz3IpE41BK3dzUcjcP1p7BWz9sIafX37Vu4p2UPmAEVLXKKjw0l72Q4Sf9wrdH2zD7j/jd8yI+nZI/VTNtpL3eVGEhFmc29mW/j+dteYyMjz4Km7maCKGUuvFMuJQbcyfy/JGHmTljStjM1UQIyekw6AUwXPaqDWbCiaE5J8R3yqqXyEh4kiU/zgyPrRFA/M1T8uOfK5A6vPAFW51QjSgfCQLWlkwWbB5Ej1YNwmezJvyUVjdYeMRL1pIvOXDOYOpW94TPZk34SRsJjToUvdNtQH3jRmFu/ZG9h4fSoGZCWMwNJ/HrlPwUtfNowGxsMT1scHfm+6mr+eIvvTEMOfF8TXxRGt0Ybr4/2pbDszby4OAOVW+jJrIobgv0IN3MzWlPvWU7GNU7/sJ48Rm+Kw0B+63IiCkMHjSEVTsO8vny7eG2TBPJBOjGGPkFrbr0Z/y8rfz6u96BVFMMQbrJOTmNKcviX4m3kQAAFFdJREFUs67RPaXiCGjZXNxU8WbmZmbMmMJFB4/ganWWHsDWFE6Abm6rcYTPl23nsy8+45YWOwoP3Wg0UEA3QzpvYuaMKeyb+QP1O/aPK81op1RKDEN4Iu0Ibb98BGOWFzKf16m+mhJJrl+duzsc5Nr1f0Nt8iExuvmfJrRc1nAH13qeJGG+Fxa/GFea0eG70pK9kM6bXscj9l4oKo5nXGvKQPZCRuS+j4c8JM5n6mtKT9KmT0iQ3Lisa3RPqTT4Z2R7czBQeJWgDBfuOJ1xrSkljm4SvDkoUfiUIC43htaNpjiyF8LSCQigFCjDROJIM7qnVBr8M/mxEAzWVOvKDdbDHGzYJdyWaSKZAN2AwVzViTda/CduwjCacpKVCZYPwa5xltQfFFeaCYlTEpGBIrJORDaKyH2hKDOi8KdrigmuBKqf/xCZx1ox/sescFsW1cSTbsSVwJq2t/LvtXXZcUBn4lWEeNKNz/Dw2v7u+dvoxAEVdkoiYgKvAhcAHYBhIhJbkzIC0jUZMYXWXftzbvuTGPvDFg4dywu3dVFJPOpm0KCLUQrGzNkcbsuilnjTzU/9xjPrj5b8sHFPuK2qMkLRU0oHNiqlNiulcoFJwJAQlBtZBG7Wlb2Qxxt8Retjqxg/NyvclkUrcaebZodX8kryLNYs/IZdh46F27JoJa500+XM8zmr2mZ+/+qZuNlfKRSJDk2B7IDP24AewSeJyGhgNEDz5s1DcNsw4QxeN/bl8n6ii5syXRw+syU1E3TOSBmJS90M9OXSzzT5cMbJXHflleG2KhopUTcxoxkg4dclvMVjyJ481PgJyIgvYn58qcoSHZRSY5RSaUqptIYNG1bVbUNPwPYFHryclruCCfO32sf09tchJ9Z0I8qHR7zsXfktvx9xQr9aNyElZjQDkJWJCy8usVDevIKp4TGqm1A077cDyQGfmznfxSZBa+I1OKkxR2c/T25iGp6vH7Qdlp4gWRriVzeGm525NVg5+RHO7HQqzLxP66b0xJ1uxPTg8+aQh4knsT5G5gtQrUHM6iYUTmkR0EZEWmKLYygwPATlRib+QcisTKjWgJEz7kVZucjMyYCy90rxT3aLEZFUEnGrG6NaAx6ddg/G1jxUtoEoS+um9MSlbtbPm874ZQd5YuZ9YOWBiK2ZGNRNhZ2SUsorIn8BvgRM4G2l1KoKWxbJ+Neomno7pi8HxD+h1kSQuN7KuLTEu27cKhdBYVkghgFaN6UiXnVzqqW4bO094MvBbvwazsaBsaebkIzOK6WmA9NDUVbU4My6BoUCfMpkUbsH6HUyetHNUhLPuhFHN3mYyIBn8OQe0LopJXGnm+yFmO8NIc06hiiFEgNxJcDAp+Ho3pjTjU4ZKy/OrGsbYVa183hscxdmXdYPj0svlKEpgiDdfOg9izxvf67vE3/75mhKiZMkY6DwirDUPJ3O1zyDO6VnuC2rFHTtWV4KzNZPpGGf69l+4CifLt0Wbss0kUyQbtY0Gszr32/iWJ6v5Gs18UmgZswEnjpyCWO2JIXbqkpD95TKS2DCQ0ofujbrzulLf+TVWZu4vGszXKb295pCCNLNoJxWTBy7gA+XbOPani3CbZ0mEgnQjJnSh0bfm7z07QbOPCWJzsl1w21dyNE1Z0UImK0v2xbx75O/JWn/Mj5btiPclmkimQDdnOHZxONJXzHn22nkeHVvSVMEAZp5Jv0YdyRO5ZXxE9l1MPZWBtE9pVDgzNZv7cv9//buPz6K+s7j+Oszs9mAKPJTqBBF8VC0EjkTRJEWuVoQBK2ed4ICenp6bb2TM9bS6ql3Uk69ov3hw+OsVUJJKKggiihagxhaS5oAQUTkTgQXEJAfgkLI7sx874/ZJEsMAZLd7M7yeT4ePOCRbHa+y7wz35nvT0pzbX69ZAfuV92wdXda1ZxIBTLrGm5yarnehKiaf5DLviFZ13GtkihSQcd513OnG2WSZzPj6a3cUdiJDv2GZU1mtFJKhsTZ+hgm1z6DLPVgmQ2jpkOP8+uba7IlOCoJ6nKDR1hiFK6bhlln/CHimhvVlIRrTa4Y7qqZgVVuMH/6L2TgzZA/ruF1Ac2NVkrJkDhbXwTLc7AA4znIa/eAFQLPybqZ16qVEnODnxsRwPNAc6OakpAZSwTxPCS+O62pfB5ZNRuQQOdG+5SSIWGpeRk1HZEQxoBAw4xr3QpbNZaQG2v0dIzYmLrvaW5UUxK3Qxk1HQnl4iFg8Oe+ubHA50aflJKlbrY+fjjcRUUIHpad46/yUHfnkkUzr1USJORm+96D9Fj+ALYYLDsEmhvVlITM0ON8rOpS3JUlGNfBiE2ObQU6N1oppYAU3Mq6WC/eWPQi5xVcxdj8XoFu41Vto/eVP+Dnm0/B/nQ5k268mS4dcjU3qnnxCsrOH0/ZGy/x1MaePDTmAvLd9wObG22+S5EBl36X98++jQdXdmB/94ENGwQq1Yy/vfY6nnav4fEPTj18Y0mlmpM3iMtumcaeLhcx+U9hDg2eHNjcaKWUQj8eeR5fHIwx452P010UFRB9unVgwuA+zKuMsH77/nQXRwVIuxybqddeyCe7DvDb5Z+kuzgtppVSCn2z16ncfe5e7D8+yefrgtfhqNLjX/7mHC5vt5HVJQ9iPl2R7uKooIhUcPn2Yv6xzy6eLd/IgVon3SVqEe1TSqVIBXdvLcKzongvLIB/WOR/XfsJVDM67V7NczIV9kfxiudg3/Kq/w3NjTqS+AR+3Cg/sXKoqpnC7D/35c6zdwcuN1oppdKmciw3hiUejhdjZ/nznLZxQVbuFqmSaFM5tokh4uG4UWJVJeSsnau5UUcWn1SLcbE8uPG0zbz+7uvcUT4VCVhutPkuleIT3YzYOBKiYtNeTKrnEEQqoHy6/7cKpvgW2EZsYibEmq37Uj/3RHMTbAkriWOHOfeSUfQ/tAbj1AYuN/qklErxiW6yqZyVbn+eW/IRI9uX+f/pqZhDkPAIH6Q7I9VIQm5mbvoGb6/fybx2r2N5aG5U0xqtPj+gdyG/Wb6R2MEF5IobqNxopZRq8XkElxpDh4+7MnHzA8y4vIaO/a9I/i9+wiN8/Z2RXlyCKZ6bcRdH+e0Ty7jvpEd47OL9qVnkV3OTHQ6bwA+Fl49k3Ks1PDXkIKfnXxmY3GjzXRsREaZ970JWmX7867bhmN6FyT9Io0f4IM7mVofrdFKYR675Ji/u7MUM79rUVBaam6x03V/34qOc/jxZc3WgcqOVUhvKO7CWWf3K2fvRcuav3JqCAySsi6VNMFnjqk4RftmrjHfeXsS6bSmYu6S5yT6RCk75y6+466/28uqabeyriSX/GCnKjTbftZV4+2uBG2VObojbXgkxuO/t9OrUPrnHSVwXSwVfPDdj3SgjQjZTSk7i0cm30S7HTu5xNDfZI6Gv504rh7ecKby8qj+TLuuT1MO4nuHfKtsz8oKJfCuve9LeV5+U2krinkvicLV5h2XPTsHZ/Od0l0xlssNy41Kw7w2WP/9THSWnjuyw4eExbu+4AnfZ9KRPxF66fielKz7lh6Uriew5mLT31UqprSS0v4plc0PoXf7uy1mY4rF6gVFHlpAby7K5Meddhm19BnfmGM2NalpiX49lMyL2NhMPzcZL8rWmZMVmup0cBgP3zFudtPfVSqmtJLa/DrwZ27iExEPcGP9X8Xryj6fzTrJD49zgERJ/U7c9H5Ql/3iam+BrlBkrfq1J5lylyJ6DvLPhc8ZfciYPDTxAYWQm+zYsT8p7a59SW6prt49UwOo5GDeKi82D1Z15+PIv6dfjlFYf4g/rdvDa4oVM+/J+wuJghXIR7bwOtoTcSDw3jrF5eE1npl3hcHJu636No47HE29t4LO1y3jswAPk4iChXB30EGSNMuM6UaLGZn+XQnok4e1fqIwgwIReO+jy0vcxoSgydyHc8mqrM6NPSulQNzly+P189ffzaZdjU/abKez+sIm7mGO5c42/5qWF87l9ViUXxt4nLA42HsYJ5u6TqgkJudkwooRt+2p4/ekfNd0veYy5OfD24zzw1HPMWPYxQ0LrCeHUb6+tuckC8cwcuOzHTHTuZ+Hqbc3n4hhz03nlU4w/fTvdd1dgebH6Vp9kZEaflNIlfjfRtbqUZ00JnuPgzJ3HgSv/kw7ePr9deMc6WFzkb41tH+HOtXImLC7C81xGmRw293uSm64Yj1XyAq5TS9TY7D71Ynq3/SdUqRA//wOqS5mbW4L5wsGbWYIZ/RhSs+e4cmMWF9HOc/l3k8N1VxUz+OxxeMWlOE4U19jk6nyl7JA3iI55g7jp058xesMPMP9r/CfhkY9Cze6G+UXVpbCqtGHX2sa5iVRAdSlmVQkTHAeic6H942CHcZxaPAkRTkJmtFJKl7phm84hLAyWACZG6K37MAJi2eC5/mxpALf26zOmIxWwuAjjOVhAWBwmn7MD66yb/buj9Uu5pSzMd/fm8U/p+Iwq+RJyY2NAwPGieIuKsI4jN2ZxEXgONtBOHAZbH0JeEdakV6koe5nH13fnv08dkJSmHpUBIhVcs+0JwEUA4xxCFheBMWDZgIAbxWAQ+PoKDQm5A0NIwBgHanYjk17hhXklrLYv5DFdZijA6oZtYuq/ZIlgjIdgMK4fnnpiNdzRRCr8n9+3Bc9zseLvYlkWclb8NXmD6AiMqfodn7y/DL7dt00+lkqxRrkxgCUcV24O7dpMjudix39erITX5A2iZ/4BBm+Yw5r33uTKEWPb7KOpFNpUjngeiF8PGQHxXMCA6zVURgY8hJiE+KJLIT0SrjWJufMQrLpVHPIGsf6ck4hUleG9Ox3rrNZtk9GqSklEbgAeBvoDg4wxla15vxNK3bBNNwriJ8UyHkaMf9rjwQHx735HTW8YJFE8Nj5IwsI1NiFxsSwbufSuw9t0i8cyyaklVlNK9OVqwhfflBEd15qbVmiUGzEGjIeIwRj8iw4giH8H3ERuxAiOsRHxsCwLGuXmzEXjuCenFvPefIhNgPxxmpug6zMUQrkYpxY3XjGJePFvmvrrjSs2q7qN4Ze7CrBerGamNRXLi/lZskIYD6KesLnLEPr1bbjRHZK7kSnyCJTFcMXCGj0dKbi1RUVt7ZPSWuA64H9a+T4nnsRVffdtgapiwCD+/S4i4BhhTXggtUN+xEWdO9N+0WTcbdVYTi2CB8awqusYCvMHIB26wRtTGlbsvWgcuFEsPMJ4sLoY1s7NlBFVmpuWaiI3Eq+GjPh3uw7Cts6X0GX0Q/7IvEWTcbZWY8Wbim0s9px7I6flnQPtu34tN+JGCeFhjIepfB5ZPUdzE3QJK8+z91OslbMQwIvfyFjiX31CBRMpzB/Prz4oY+1fysA95P+8B1w8kTe3hHl3i8sjX86GquUQz8YFsTXkEMPCYIwLi++FHhe0KDOtqpSMMR+Cv9ioaoFGQ8Rxo2DZfsXkOWDl8Ix1AzuXfMSc8FSMOFjx1j5XBAmFueTaHyJnXOKPmElcsRfj7+UUbwMWTMasAK25aaWj5MYlxOTtI5HnVlAankoYB7s+NxZWKMxpQ2/13+MIufGcQ4jmJrvEcxOKVMCauf4IS9vPDZ6D2GHoeREUj6WLU8tQ/CcpAxjLZk/f6/nn977i13lLsXbEDlsdvNP5wzGVT2DifVYYr8WZabM+JRG5A7gD4IwzzmirwwZDo71QAGRTOaE+Q3nq9AIir0wlpzp+suPNM5w1DHv4TxtOemKzjh2G/PGQPx6pLsWpmo1lvIY24ADR3DTjCLnJ7TOU/7DOZe+bj5Kz+fDcWGcPQ674yVFz460sxayeTQjPv1gFKDeamaNIeGqSuvNal6H6Pkuvvm/SRXjR+TZTf38QzxguGno1LJjVkJk+Q+mQN4jPvvUzev7xgYZRny3MjBhjmn+ByB+Ank18635jzML4a94B7j3WNt6CggJTWanNwccsUgEzR8fDgn/Cr3q8YThn4giZunA1HqXX6OsiUmWMKUhVkTU3GUBzo5k5XvWj7GoBD8TCs8PM7vR9utsHOKtgBOcVfue4MgPHl5ujPikZY75z/J9MJVXeILjlNX8eAQI98w/vB6hr7z/SSs9pWAFac5MBNDfqeCU+fbfvCjW7sdp3ZWJdbpb8Dnq+ktLM6JDwoEg82Y37ATKgvV9lKM2NOl6NK5Y2zk2rlhkSke+JyBbgUuA1EVmSnGKpZgV8p1DNTZpoblRLtHFuWjv6bgGwIEllUceqcQd3wO52NTdporlRLdHGudHmu6DSnUJVS2huVEu0YW50lXCllFIZ46hDwlNyUJHPgc0pPkw3YFeKj5Es6SjrmcaY7m18zFbR3HyN5uYoNDNfk66yHnNu0lIptQURqUzlfIpkClJZs12QzkWQyprNgnQeglBWbb5TSimVMbRSUkoplTGyuVJ6Jt0FOA5BKmu2C9K5CFJZs1mQzkPGlzVr+5SUUkoFTzY/KSmllAoYrZSUUkpljKyqlEQkT0SWisg6EflARO5Od5mORkRsEVklIovSXZYTleZGtUTQchOUzGTbMkMOUGSMWSkipwBVIvKWMWZdugvWjLuBD4GO6S7ICUxzo1oiaLkJRGay6knJGPOZMWZl/N9f4p+AXukt1ZGJSG9gNPBsustyItPcqJYIUm6ClJmsqpQSiUgfYCCwIr0ladYvgPsAL90FUT7NjWqJAOQmMJnJykpJRE4GXgImG2P2p7s8TRGRq4GdxpiqdJdF+TQ3qiUyPTdBy0zWVUoikoMfkBJjzPx0l6cZQ4CxIrIJ+D0wXERmp7dIJy7NjWqJgOQmUJnJqsmzIiJAMbDHGDM53eU5ViIyDLjXGHN1ustyItLcqJYIYm6CkJlse1IaAkzAvxNYHf8zKt2FUhlPc6NaQnOTAln1pKSUUirYsu1JSSmlVIBppaSUUipjaKWklFIqY2ilpJRSKmNopaSUUipjaKWklFIqY2ilpJRSKmP8P24yYBiRVo1RAAAAAElFTkSuQmCC\n", 249 | "text/plain": [ 250 | "
" 251 | ] 252 | }, 253 | "metadata": { 254 | "needs_background": "light" 255 | } 256 | } 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "source": [ 262 | "We would expect the models with increasing complexity to better fit the data and result in lower RSS values. This can be verified by looking at the plots generated for 6 models.\n", 263 | "\n", 264 | "This clearly aligns with our initial understanding. As the model complexity increases, the models tends to fit even smaller deviations in the training data set. Though this **leads to overfitting**, lets keep this issue aside for some time and come to our main objective, i.e. the impact on the magnitude of coefficients.\n", 265 | "\n" 266 | ], 267 | "metadata": { 268 | "id": "vzt4JtBRMeuA" 269 | } 270 | }, 271 | { 272 | "cell_type": "code", 273 | "source": [ 274 | "#Set the display format to be scientific for ease of analysis\n", 275 | "pd.options.display.float_format = '{:,.2g}'.format\n", 276 | "coef_matrix_simple" 277 | ], 278 | "metadata": { 279 | "colab": { 280 | "base_uri": "https://localhost:8080/", 281 | "height": 578 282 | }, 283 | "id": "m5lNtk8BNxtD", 284 | "outputId": "d1043c10-39e2-4d05-8c6d-58de217efbc5" 285 | }, 286 | "execution_count": 4, 287 | "outputs": [ 288 | { 289 | "output_type": "execute_result", 290 | "data": { 291 | "text/html": [ 292 | "\n", 293 | "
\n", 294 | "
\n", 295 | "
\n", 296 | "\n", 309 | "\n", 310 | " \n", 311 | " \n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | " \n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | " \n", 411 | " \n", 412 | " \n", 413 | " \n", 414 | " \n", 415 | " \n", 416 | " \n", 417 | " \n", 418 | " \n", 419 | " \n", 420 | " \n", 421 | " \n", 422 | " \n", 423 | " \n", 424 | " \n", 425 | " \n", 426 | " \n", 427 | " \n", 428 | " \n", 429 | " \n", 430 | " \n", 431 | " \n", 432 | " \n", 433 | " \n", 434 | " \n", 435 | " \n", 436 | " \n", 437 | " \n", 438 | " \n", 439 | " \n", 440 | " \n", 441 | " \n", 442 | " \n", 443 | " \n", 444 | " \n", 445 | " \n", 446 | " \n", 447 | " \n", 448 | " \n", 449 | " \n", 450 | " \n", 451 | " \n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | " \n", 497 | " \n", 498 | " \n", 499 | " \n", 500 | " \n", 501 | " \n", 502 | " \n", 503 | " \n", 504 | " \n", 505 | " \n", 506 | " \n", 507 | " \n", 508 | " \n", 509 | " \n", 510 | " \n", 511 | " \n", 512 | " \n", 513 | " \n", 514 | " \n", 515 | " \n", 516 | " \n", 517 | " \n", 518 | " \n", 519 | " \n", 520 | " \n", 521 | " \n", 522 | " \n", 523 | " \n", 524 | " \n", 525 | " \n", 526 | " \n", 527 | " \n", 528 | " \n", 529 | " \n", 530 | " \n", 531 | " \n", 532 | " \n", 533 | " \n", 534 | " \n", 535 | " \n", 536 | " \n", 537 | " \n", 538 | " \n", 539 | " \n", 540 | " \n", 541 | " \n", 542 | " \n", 543 | " \n", 544 | " \n", 545 | " \n", 546 | " \n", 547 | " \n", 548 | " \n", 549 | " \n", 550 | " \n", 551 | " \n", 552 | " \n", 553 | " \n", 554 | " \n", 555 | " \n", 556 | " \n", 557 | " \n", 558 | " \n", 559 | " \n", 560 | " \n", 561 | " \n", 562 | " \n", 563 | " \n", 564 | " \n", 565 | " \n", 566 | " \n", 567 | " \n", 568 | " \n", 569 | " \n", 570 | " \n", 571 | " \n", 572 | " \n", 573 | " \n", 574 | " \n", 575 | " \n", 576 | " \n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | " \n", 609 | " \n", 610 | " \n", 611 | " \n", 612 | " \n", 613 | " \n", 614 | " \n", 615 | " \n", 616 | " \n", 617 | " \n", 618 | " \n", 619 | " \n", 620 | " \n", 621 | " \n", 622 | " \n", 623 | " \n", 624 | " \n", 625 | " \n", 626 | " \n", 627 | " \n", 628 | " \n", 629 | " \n", 630 | " \n", 631 | " \n", 632 | " \n", 633 | " \n", 634 | "
rssinterceptcoef_x_1coef_x_2coef_x_3coef_x_4coef_x_5coef_x_6coef_x_7coef_x_8coef_x_9coef_x_10coef_x_11coef_x_12coef_x_13coef_x_14coef_x_15
model_pow_13.32-0.62NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
model_pow_23.31.9-0.58-0.006NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
model_pow_31.1-1.13-1.30.14NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
model_pow_41.1-0.271.7-0.53-0.0360.014NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
model_pow_513-5.14.7-1.90.33-0.021NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
model_pow_60.99-2.89.5-9.75.2-1.60.23-0.014NaNNaNNaNNaNNaNNaNNaNNaNNaN
model_pow_70.9319-5669-4517-3.50.4-0.019NaNNaNNaNNaNNaNNaNNaNNaN
model_pow_80.9243-1.4e+021.8e+02-1.3e+0258-152.4-0.210.0077NaNNaNNaNNaNNaNNaNNaN
model_pow_90.871.7e+02-6.1e+029.6e+02-8.5e+024.6e+02-1.6e+0237-5.20.42-0.015NaNNaNNaNNaNNaNNaN
model_pow_100.871.4e+02-4.9e+027.3e+02-6e+022.9e+02-8715-0.81-0.140.026-0.0013NaNNaNNaNNaNNaN
model_pow_110.87-755.1e+02-1.3e+031.9e+03-1.6e+039.1e+02-3.5e+0291-161.8-0.120.0034NaNNaNNaNNaN
model_pow_120.87-3.4e+021.9e+03-4.4e+036e+03-5.2e+033.1e+03-1.3e+033.8e+02-8012-1.10.062-0.0016NaNNaNNaN
model_pow_130.863.2e+03-1.8e+044.5e+04-6.7e+046.6e+04-4.6e+042.3e+04-8.5e+032.3e+03-4.5e+0262-5.70.31-0.0078NaNNaN
model_pow_140.792.4e+04-1.4e+053.8e+05-6.1e+056.6e+05-5e+052.8e+05-1.2e+053.7e+04-8.5e+031.5e+03-1.8e+0215-0.730.017NaN
model_pow_150.7-3.6e+042.4e+05-7.5e+051.4e+06-1.7e+061.5e+06-1e+065e+05-1.9e+055.4e+04-1.2e+041.9e+03-2.2e+0217-0.810.018
\n", 635 | "
\n", 636 | " \n", 646 | " \n", 647 | " \n", 684 | "\n", 685 | " \n", 709 | "
\n", 710 | "
\n", 711 | " " 712 | ], 713 | "text/plain": [ 714 | " rss intercept coef_x_1 ... coef_x_13 coef_x_14 coef_x_15\n", 715 | "model_pow_1 3.3 2 -0.62 ... NaN NaN NaN\n", 716 | "model_pow_2 3.3 1.9 -0.58 ... NaN NaN NaN\n", 717 | "model_pow_3 1.1 -1.1 3 ... NaN NaN NaN\n", 718 | "model_pow_4 1.1 -0.27 1.7 ... NaN NaN NaN\n", 719 | "model_pow_5 1 3 -5.1 ... NaN NaN NaN\n", 720 | "model_pow_6 0.99 -2.8 9.5 ... NaN NaN NaN\n", 721 | "model_pow_7 0.93 19 -56 ... NaN NaN NaN\n", 722 | "model_pow_8 0.92 43 -1.4e+02 ... NaN NaN NaN\n", 723 | "model_pow_9 0.87 1.7e+02 -6.1e+02 ... NaN NaN NaN\n", 724 | "model_pow_10 0.87 1.4e+02 -4.9e+02 ... NaN NaN NaN\n", 725 | "model_pow_11 0.87 -75 5.1e+02 ... NaN NaN NaN\n", 726 | "model_pow_12 0.87 -3.4e+02 1.9e+03 ... NaN NaN NaN\n", 727 | "model_pow_13 0.86 3.2e+03 -1.8e+04 ... -0.0078 NaN NaN\n", 728 | "model_pow_14 0.79 2.4e+04 -1.4e+05 ... -0.73 0.017 NaN\n", 729 | "model_pow_15 0.7 -3.6e+04 2.4e+05 ... 17 -0.81 0.018\n", 730 | "\n", 731 | "[15 rows x 17 columns]" 732 | ] 733 | }, 734 | "metadata": {}, 735 | "execution_count": 4 736 | } 737 | ] 738 | }, 739 | { 740 | "cell_type": "markdown", 741 | "source": [ 742 | "It is clearly evident that the size of coefficients increase exponentially with increase in model complexity. Compare Power 2 with Power 15 Coefficients.\n", 743 | "\n", 744 | "**What does a large coefficient signify?(Power 2 or 3 or 4)**\n", 745 | "\n", 746 | "- It means that we’re putting a lot of emphasis on that feature, i.e. the particular feature is a good predictor for the outcome. \n", 747 | "- When it becomes too large, the algorithm starts modelling intricate relations to estimate the output and ends up overfitting to the particular training data." 748 | ], 749 | "metadata": { 750 | "id": "b0fyONkdMfGq" 751 | } 752 | }, 753 | { 754 | "cell_type": "markdown", 755 | "source": [ 756 | "**Ridge Regression**\n", 757 | "\n", 758 | "- Ridge regression performs ‘L2 regularization‘, i.e. it adds a factor of sum of squares of coefficients in the optimization objective. \n", 759 | "\n", 760 | "- Thus, ridge regression optimizes the following:\n", 761 | " Objective = RSS + α * (sum of square of coefficients)\n", 762 | "\n", 763 | " Here, α (alpha) is the parameter which balances the amount of emphasis given to minimizing RSS vs minimizing sum of square of coefficients. \n", 764 | " \n", 765 | " α can take various values:\n", 766 | " \n", 767 | " α = 0: The objective becomes same as simple linear regression. We’ll get the same coefficients as simple linear regression.\n", 768 | "\n", 769 | " α = ∞: The coefficients will be zero. Why? Because of infinite weightage on square of coefficients, anything less than zero will make the objective infinite.\n", 770 | "\n", 771 | " 0 < α < ∞: The magnitude of α will decide the weightage given to different parts of objective.\n", 772 | "The coefficients will be somewhere between 0 and ones for simple linear regression.\n", 773 | "\n", 774 | "This gives some sense on how α would impact the magnitude of coefficients. One thing is for sure that any non-zero value would give values less than that of simple linear regression. \n", 775 | "\n", 776 | "Note the ‘Ridge’ function used here. \n", 777 | " - It takes ‘alpha’ as a parameter on initialization. \n", 778 | " - Also, keep in mind that normalizing the inputs is generally a good idea in every type of regression and should be used in case of ridge regression as well.\n", 779 | "\n", 780 | "Now, lets analyze the result of Ridge regression for 10 different values of α ranging from 1e-15 to 20. These values have been chosen so that we can easily analyze the trend with change in values of α. These would however differ from case to case.\n", 781 | "\n", 782 | "**Note** \n", 783 | " - Each of these 10 models will contain all the 15 variables and only the value of alpha would differ\n", 784 | " - This is different from the simple linear regression case where each model had a subset of features." 785 | ], 786 | "metadata": { 787 | "id": "17KNhEWQK13O" 788 | } 789 | }, 790 | { 791 | "cell_type": "code", 792 | "execution_count": 5, 793 | "metadata": { 794 | "id": "dUb6YeuhKxF1", 795 | "colab": { 796 | "base_uri": "https://localhost:8080/", 797 | "height": 297 798 | }, 799 | "outputId": "85633f1e-78f7-4333-9baa-aac7bc097656" 800 | }, 801 | "outputs": [ 802 | { 803 | "output_type": "display_data", 804 | "data": { 805 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEYCAYAAAD8hukFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeXxU5fX/3+fOJAHZdwQCYRdUFokRlyhYV9TiVguKglqX2lWp/dmqrVpxaUu/2lattsqiAooLsuOKREBD2JRVQIJh30RAIMnMfX5/3DtkMkyWSSaZJef9Yl7JzL3PvWdyP5zzPOfZxBiDoiiKosQDVqwNUBRFUZQAGpQURVGUuEGDkqIoihI3aFBSFEVR4gYNSoqiKErcoEFJURRFiRuqFZREZL6I/CwahojDOBH5TkRyo3HNCu43XkQej/a5SvmoZpSKUI3UbSoMSiKSLyJHReSwiOxy/5ANI7mJiGSIiBERbzmnnQdcDHQwxmRFcv1ERkQGi8gnIvK9iORX81qpIvKW+8yMiAwKOf6IiBS7zzLw6lKde5Zhh2qmBqllzYiIPC0i+9zX0yIi1bmne13VSA0S6XMTkRtFZIuI/CAi00SkedCx5iLyrntsi4jcGEHZX4pInogUisj4ythe2ZbSVcaYhsAZQCbwUCXLRUInIN8Y80OkBSsQZbzzA/AKcH+UrvcZMALYWcbxN4wxDYNe30TpvqGoZmqO2tTMncDVQF+gD3AVcFeU7qsaqTkq/dxE5FTgReBmoA1wBHg+6JTngCL32E3AC26ZypTdDjyOo9dKEVH6zhizDZgDnBZ6TEQsEXnIjZi7RWSiiDRxDy9wfx5wa0Znh5S9HfgfcLZ7/FH38ztEZKOI7BeR6SLSLqiMEZFfiMgGYEM4e0VkqojsdGuUCwJ/yDDnDRKRrSLyRxHZ69bibgo5rZmIzBKRQyLyhYh0DSr/rIgUiMhBEVkqItnl/R2DMcbkGmNeBcIGBxE5RUQ+cP8G60XkhnKuVWSMecYY8xngr6wNNYlqJuE1MxIYa4zZ6j7LscCoytpaGVQj0dcIkT23m4AZxpgFxpjDwMPAtSLSSEQaANcBDxtjDrs6mY4ThMotC2CMeccYMw3YV1nDIwpKIpIODAGWhzk8yn0NBroADYF/u8fOd382dWvni4MLGmNeBu4GFrvH/ywiFwJPAjcAJwNbgCkh97waOAvoXYbJc4DuQGtgGfB6OV+vLdASaI/zQF8SkZ5Bx4cBjwLNgI3AmKBjS4B+QHNgEjBVROoBiMh5InKgnPuWiSuID9xrtnZteF5Eyvq+leEq9z/jahH5eTWuUylUMwmvmVOBlUHvV7qfRQ3VSI1oJJLnVupcY8wmnJZRD/flM8Z8Xca1yitbNYwx5b6AfOAwcADnAT4P1HePzQd+5v7+EXBPULmeQDHgBTIAA3jLuc8o4LOg9y8Dfw1639C9Xob73gAXVmR/UPmmbpkm7vvxwOPu74MAH9Ag6Pw3cWoHgXP/F3RsCLCunHt9B/StrG1umYtw0gzBn/0UyAn57EXgz5W43lZgUMhnvYF2gAc4B9gBDI/Ezkp+F9VM8mjGD5wS9L67+zcR1Uj8aiSS5+b+je8O+Wyba382sDPk2B3A/IrKhnz2ODC+MrZXtqV0tTGmqTGmkzHmHmPM0TDntMMRV4AtOMJpU8l7lHs94zQN9+HUOAIUlFVYRDwi8pSIbBKRgzj/CcCptYTjO1M677zFtSFAcL79CI6YA/f6nYisdZvzB4Am5dwnEjoBZ4nIgcALp7ncVkQ6StCAhcpczBizxhiz3RjjN8YsAp4Fro+CneFQzSSBZnACR+Og942Bw8b1NNVENVJzGonkuYWeGzj/UAXHKipbJaLZkbcd5z9EgI44tYRdlH7gVbqem5ZogROFA5T3H+NGYChubRLngX4HlDUCpZmINAgSUEdgVUVGunne3wM/AlYbY2wRKe8+kVAAfGqMubiM4xGNVgqDITp2VhXVTPxrZjVOZ3lgOHVf97PaQjVSNY1E8twC5wbu3QVIA74GbMArIt2NMYE+tuBrlVe2SkRz8uxk4F4R6SzO0M4ncEZ6+YA9OF8ukuHHk4FbRaSfiKS51/vCGJNfyfKNgEKcWtBJbvmKeFScIbLZwJXA1EreJ/AdvSLyJ06sOZSJOB259YAU563UE5FU9/BMoIeI3CwiKe7rTBHpVc710gJ5ZyDVvZ64x4aKSDNxyAJ+DbxXWVtrANVMnGsGmAjcJyLtxRkQMBon7VRbqEaqoBEie26v4/Q1Z7tB+jHgHWPMITeYvgM8JiINRORcnKD8akVlwRnB6GrLA3hcbZXbGIpmUHrFNXQBsBk4BvwKwBhzBKcDb6GbUhhY0cWMMR/ijOR4G6fvoytOp2BlmYjTVN4GrAE+r+D8nTg1nu04f+i7jTHrKnGfecBcnJrBFpzvfbz57z6s8lIl5wNHgdk4taijwPsA7oO9BOd7b3dtfBqnJlIW691rtHdtO0pJzXAYTmfqIZy/z9PGmAmV+I41hWom/jXzIjAD+Aqnhj/L/ay2UI1UTSPlPjc3hZsNYIxZjTMg5HVgN05AvCfoWvcA9d1jk4Gfu2UqU/YhHD09gDPt4CgVDP2X6KSGExtxJgy+ZozpEGtblMRANaNUhGqkaujad4qiKErcoEFJURRFiRs0facoiqLEDdpSUhRFUeKGmCw42LJlS5ORkRGLWyc8S5cu3WuMaRVrO2oT1Uv1UM0okRBrvcQkKGVkZJCXlxeLWyc8IrKl4rOSC9VL9VDNKJEQa71o+k5RFEWJGxIzKBXkQs5Y56eiVITqRYkU1UzMSLxNrApyYcKPwV8EnlQYOR3S68yGkkqkqF6USFHNxJTEaynl5zhiMX7wFcL8J7U2o5RNsF78RbByktaAlfJRHxNTYhqUjDHsXLWAovl/K3noFTWbM7Kd2gsWYMM3851ajYqmbhBOH+VpJqAX8YDlgeWT4OMxqpm6QqR6AfUxMSYq6TsReQVn9dvdxpgTtjQui7VLPqLzrOF4xQefjYXLnoK5D5Q0my97Co7uw9fxXNZ5e2Ebw6m2wdNvGOz4ErYvB2M75+fnaBM7QaiqXsKmVaDkM8sD/UdA3+GltdBvGMdX/F86oaTVpJpJGKqkmYr0EuRjyMg+UTPqY2JCtPqUxuNsUTwxkkLdjizHwoeFDb5jsHxi6Wbz7NEY24/B4rWiUXxt0pmU9gSCD8vjBSsFbJ8jroxItq9XYsx4qqCX0mmVY04qrkl6UHrOD3njYPlrTnBq2/fESo4nteS9aiaRGE+kmgmXhmuWcYKPwbbBsmDIWGjTu3QlR31MrROVoGSMWSAiGZGWS+16AYXzn8KDjWCcmonldXZIEcHYfgSDFz9PpI5jZ9vBpOx0gpjt92Nl3uI4pdBajhLXVFUvZGQ7jsLvB4yTirv8r47D8B1zPsNg/EWQ9woGC8E42vIXOTXikdMdZ6WaSSiqpJlAGs5XyPE0nOUt5WOwXS3ZNsy6F3oOKQlaNjBAfUxtE9uBDulZLG9+Rck2j7Yf2p4GA0ay49zH8Rnr+NaoFjbtdn6CJYKNRaHxMEsGQfbo8GLRIZ3JR3oWu7v9BBNIxdk+2LkC+g2jsOtl+CQVPwLG0YwYG4zBj1AsXg62GehoJZxmVC/JR3qWUwnpOgiDBcbG7y9mLZ1401zIg0WjKHZ9DIAxNva6WRixnD5ITyr0vVH1UsvU2pBwEbkTuBOgY8eOxz8/fMr1FC6cQz3xIdiwfTlm1xrGpDxCO+8d/MF+ycnpAmAjBrC8TG3+Cx75LIUmPfZyXr1vnFQO4vQngA7pTHDC6cVvGx77tg9/Yypp+EE8mGWvg+3DNl7+XHwz5zfezqXH5mFhI1Kyr/WfCkcwe/IPPDZ0G0NbbFO9JCHhNLPgaGdm7ruCR81npFCMRww9/RvoKls41uc55v3QgiFb/gqBqo4x+G0/mzpcR8aPbictPcsJPPk5UL+F09qu36J0Wlj1ElVqLSgZY14CXgLIzMw83jhK7zOYmz75I893eJ+2ez8HY2P7Cul4bBnn3/Yksr+Xm/d1m9mAGMOw0xrwalFD/jd5CufKo4i/yLng8teh/02lhwFrB2XCEU4vHkv4/c9u4dHXvDTfnUs72cswzyd4xSZVfNyf3YoWl/8D8sY7qRhjI4BHhPvObcX6/AZMeONNrkgbg4dixwmpXpKGcJrx24Yv6cG8zBe5dPd4vFtzEGOTio9bTv4Wsv8Iee1g1r0Y41RkBMN7my3ee+Mofx84m7M/u60kBSiW8zK2DoCoIWI+T6lb64ZsOek0/mVfj/GkYeOhyHg5rVsXBm4b73Q83joHMkeBJ+14szq16wW8MGIAfXxfYfzFJRf0FwGmZBiwdlAmFR1bnMQTv7qdH//q7/S45E4sbypGPHg8Xlr4dzu12sxRcMX/OX0HYoEnjVan/4g37jqbezrvRIwvkABUvSQ5g3q2Yu5vz+fqq66l/iUPlfgQywPfby2lF3H1YnnrcdmV19Okfgo5H0zDHwhI4AQiOxCcVC81QVT2UxKRycAgoCWwC/izMeblss7PzMw0wYslTs79lj+88xW/7vEd9uYFNGzWhruO/Ndp/QQ3jwPN6KBOx/fnTeeCRbeSKq6j8aTBqJnOhZOwQ1tElhpjMmNtR3Worl5KUZDrpOKWTyoZJVWOXsy3X+AfdwUeu9jpePKkIUmsF1DNlCICvfj8Nu/NnMaQZXeRIsV4MIhbySlzKHkSEGu9RGv03fDqlP9pZjrvLNvKcxuFrIzbGNctB8kJk04JvIK4+JKreHrLP+hYMJ3LT2tLs3NGlpxT1gCIJHU+iUJ19VKK9Cznedq+SulFOp6F97bZLJ/xPKu3H6R+/xFcp3qJe6KmmQj04vVYXDf0Wta1a8xHc95hj92A2/o1puMZl5atBdVLtYmLte8sS5h8x0D8xpDm9UCBBYvGlj+fxH34kpHNHTcO49Jn2jNheyrTfX7ScsaGF4WuaZWcBIb+VjT/KMhh9Pv5OMZNWcH0xdvp2HwuZ7L6RM2oXpKTCPVySkY2jX49lpGv5DIl7whvtDpA3/wwPkb1EhXiIiiBUys5bkxgKGdZNY6Qh99i5HT+9pM+/Gv861gTnwR84UURug6adlAmBxXpBU7QjIyczl+vH0DK9iWc9tGDGPEjoZpRvSQnVdBL+5HTmXLnQMb8ZwI95j2IsXyIJ031UgPETVA6gTCpl+OEefiDs7Pwd96JbCsGKWNUTGVrSEriUZ5eIKxm6qVn8fBp+0lZ7EOMjfEXIcGaUb0kL1XQS8v0LP7S7wCpn6leapKYj76rEsGLbAY9/OyLr8YnXnxYGE/KiaII1JAufFCb1nWNMjTTtPeFiCcVn7Hwibe0ZlQvdZcy9NKw5yDEm4rPncB/uO3ZJWVUL1EhflpKkXQQltH8Tut8NluvncrUtyazt3EWj7TLJCVcWRVL4hNph3JZKZv0LDyjZjBrxlTGbWvPw/Sgb2g51UtyEAUfQ3oW1sgZbF/+PvfmNqTxwhT+29VgWVJSTvVSLeIjKFWlg7CMh9+hzyC62N15dsoK6s1Zx8NX9q4ho5WYUdUO5bIcRnoW2bf15/H/W8DoqSuZ+avzqJfiib7dSuyIoo8hPYsO6Vlc1SqfP723mhcXfMPPB3WtGbvrIPGRvgvXQVgV3PWohrbYxqhzMnj5s828t2JbdG1VYk+09ALHNdN4z3Keuq4PG3cf5rlPNkbPViU+iLKPoSCXmwd2YsjpbRn7/npWFhyIrr11mPhoKUWjgzCkJvTQzdNYs705/+/tL+nWuiGntmsSfbuV2BCtDuUQzVwwcjrX9m/PC/M3cWWfdvRs2yi6diuxowZ8jIyczpPX9GfFtwu4780VzPp1trawo0B8tJSi0UEYUhPyfruQ5246g2YnpXLnxKXs/6Go9Pm6ym/iEq0O5TC154eu7E2jel4efPcrbDtotRPVS2JTAz6G/ByanJTC09f3YdOeH/i/D74ufb5qpkrER0sJqt9BGKYm1KpRGv8ZMYCfvLiYf45/nT+dvh+rczbsWuMs8mpsZ8kQHSmTeESjQzmMZpo3SOUPQ3rx+7e+5OMPZ3BR/Q1w7CAs/rfqJdGpAR8DkN29FcOz0lm6cB47zTu07XOx+phqED9BqbqUMVqmb3pTXjjfxzkLfw8f+8DjdVYcN36nnL9QJ7nVVcrQzE8GdGDFwvc5b9EDGNwtVQKoXuou5Uy6ffD0w3i/HEPKEh9m2TOI+pgqkzxBCU6sCblDQH9UtBW/BHasLaZkWy+c1X51klvdJVgzQUtX/abbLry5PkTs0uerXuo2ZfiYht9vxabEx4j6mCqTXEEpmOBOScuD5UnB5y/Gj0Wqx0L8PrAsGDJWazDKCZ3YbS57iiIrBZ9dhEcMgjjbHahelAAhPkY8Kfj9xfiwSPV4EH+x+pgqkLxBKbhT0gYZcAv7rdb8clEDzuzQjPt77tGVfJUSQjuxj+7j2PB3GTfpNRo0bcXtZzRBOqtelCDC+Ji90pqfL6zPVX3acWv7repjqkDyBqXQTsm+N9I6PYsL6m/kb/PWc+pZFzMk/eRYW6nEC2E6sRunZ9HoonY8NnMNXVufyeD01rG2UoknwviYNulZdDu6kieWb+PCi++mU4sGsbYy4YiPIeE1QRlDQO86vwuntW/MI9NXc7jQ55yrQzeVMvQyYmAnMlqcxJjZa/H53f4l1YsCZWrmd5f0xGtZjH3fHSKueomI5G0pQdghoF6PxeNXn841zy/kmQ++5qG+h3UPFMUhjF5SvRYPXN6Lu19bytvLtvLTtjtVL0oJYTTTunE9bjsvg+c+2cRvT/mOLrNuVL1EQPK2lMqhX3pTfpqZzvhF+Xy3+uPoLVmjJCWXntqG/h2b8o8PvqZ40wLVi1Ihd57flSb1U1j26QzVS4Qkf1Aqo+l83yU9SPVavLytfdgl6pU6Shi9iAh/uLwXuw4WMutgF9WLUpowmmlSP4U7z+/CpF0dsa0U1UsEJHf6LjBk01dYMjQzcxQArRvV42fZXfjnR36uuWESXX9YriNl6johQ3zpPwL6Dof0LLI6N2dQz1Y8suIAF9/0Dg22L1a9KOX6mJHnZPC/nFMZ0+ppHj51n+qlkiR3Syk/xxELNtg+Z9mPoNrMXV32cm+9GbyZVwDZo1UwdZ3QYeF54xyH42rmd5f0pMvR1SxbMEMdjOJQjo9pmOblob6HSStYxMaT+qleKklyB6WMbKf2EsDYJTndglwaTL6WX/EGv932OzYt+zg2NirxQ2CIL+6GbZhS/QCn2euZUu9Jzs7/D2bCVTqaSqnQx1z71c+5L2UqHWcOV71UkuQOSulZTnPa8jpLfXjSSnK6bq3YwiYFH6sXzY6trUrsCQzxzRzlaCW0HyA/hxR8eMXG9hVrp7VSoY8RfzFebCy7mD2rPoqtrQlCcvcpgeNg2vQ+cRHFoIlvxvIyYXt7+u07QscWJ8XUXCXGBIb49r0xrGbEk4rfV0iR8eBvezYNY2utEg9U4GOMv4hi42HSrnR+E1NDE4PkD0oQfsn6oBV/D7bM4svXDvHKws088uNTY2OjEl+Uo5l9X33I3Tn1OX9LS37bPTbmKXFGOXqR/BwmbW3Pv75qzPUHjtK+af3Y2Jgg1I2gVBaukFoAV/ZZwdd5H1HYaDZp3S7QTkklPOlZtE7PosXePPI+m0eh5z3Vi1I2ro+57MBRnvjqE96fO13XxKuAuh2UgvhFt320X/MXUj71wcKxOvNaKZf7e39P+qbHVC9KpWjftD6/6LafYWvvxaz3I7q6Q5kk90CHCOi2YyZp4gx8MDrzWqmAHjtnkibFqhel0ow8aTFpFCG6ukO5aFACZ6jm8tcQwBgw4tGZ10rZHNeLwRiwVS9KRRTk0mLDm4iAAYzlVc2UgQYlcGosth8BbBFyGlyqzWqlbEL0MtO6ELv9mbG2SolngjVjhG87Xq0+pgyiEpRE5DIRWS8iG0XkgWhcs1YJDA8XD7aVyr/2Z7L74LFYW5XUJLRmgvSCJ40JPwzkg7W7Ym1VUpPQeoGS4eHioVhSePHAWbG2KG6pdlASEQ/wHHA50BsYLiK9q3vdWiVoX5Td17xJnr87U5dujbVVSUvCayZIL+aW99jVpC8v52yOtVVJS8LrBUqGh1/4IB9k/pdJO9qyatv3sbYqLolGSykL2GiM+cYYUwRMAYZG4bq1S3oWZI+m/emDGNilOVOWfIttm1hblawkvmZcvXg7DeTWczPIzd/PyoIDsbYqWUl8vcBxzVxw0RWclOph/KL8WFsUl0QjKLUHCoLeb3U/K4WI3CkieSKSt2fPnijctoYoyOVPTefS6ruVLNq0L9bWJCsVaiZh9AIMb7eT36bN4KMPZsbalGQlqXxM4z3Lebb9x3y7cj77DhfG2py4o9bmKRljXgJeAsjMzIzPJoi7DH0vfxGvp3l4PqcV53Uf4XweuoSIUqMkhF7g+MK+v5ZCCre8zb61nWjRME31EgMSQjOuj7nIX8R5Hg+zPkzn+quvVR8TRDSC0jYgPeh9B/ezxMNdpFWMn1QMbM7h4NedafzmdbqdcXRJOs0EFvb9dv7LtNg3V/USXZJOL2L8pIph76qP8PVvh/fVq1UzLtFI3y0BuotIZxFJBYYB06Nw3donaFSVeFLZ42/Awbl/AX+hbmccXZJSM0Y8eHd9hfGpXqJMUuoFy0vDYzvY/uk43TI9iGoHJWOMD/glMA9YC7xpjFld3evGhMCoqgG3YHW/iMdSJ3Ly/s8xxnaXpdftjKNBUmqm52V4xKa32QioXqJJsurFMn6Gez+m7Tdvu1tf6JbpEKU+JWPMbCB5NiRaMQV8x0jBOKs8YEGXQTDoD3W6WR1Nkk4zX89DjB9LwA9YnQchg1Uv0SJZ9eIBjPFx8JQRNG7bWfuU0BUdTiSwJTZOP6lthGJJ0YCklE1+Dti2W4EB23hYd8ovVC9KeFy9gLvkEBZTfedB9mjVDBqUTiSkX2lxs6u41X6YI23OiLVlSrySkQ3eNMACy8sYuZ3nNjaPtVVKvBKkF7G8TG37W55Z34wjRb5YWxYX6NYVoQRt/kdGNqn+biz8z2JmfrmDGzLTKy6v1D2CNCMZ2XhWNmTuonx2HTxGm8b1Ym2dEm+E+Jhuvm4cenExM1Zu56dndoy1dTFHW0rhcGdek55FprWBhxrPYdnCebG2SolnApoBfpkynT6sZ9IX38bYKCVuCfIxZ3o28Oemc1mSMxdj4nN6VW2iLaXyKMhFJg7lNl8hxYWT+e6N5TQ7Z6TmfZXwuBMjm/mLmJzm5e+Ld+JLbYW3y/mqGSU8ro8Z6Suk+KjF3jdW0OrcUXVaLxqUyiNoYmQqNilrX4cNbztN78BxHS2jBAgMkjF+UjD83v4v1icGcv4Olz0FR/epXpTShPiYFusmwcZ36rSP0aBUHoFBD75jIAYLg/EVInMfgJ2rwPbpDGylhIBe/EWICJbtx8KA7yjMus85R/WiBBPWxxxDFj4DGz+uk6s8aJ9SeQQ6JDNHYaxUfEYAG7Yt01UelBMJ2tJChozFiNfZyRgcrahelFBK+RhHL2Bg/dw662O0pVQR6VmQnoX0Gc7KV/8f/YpX4CHQGSk6A1spjasXAH/BMqwVE4JqfqoXJQyuZiwEO2+c01oyBrEs6qJmtKVUSaTjWRw4czRFJgU7sBxI5q11qlmtREZa5gh8Vho+Y2FUL0pF9B2O8Th6sT2pMGQsXPhgndOMtpQi4NzBQ7j78z9zVZNvuO7aYXVKKEoVSM9i+9A3eHPqZLoPuIxrrrw21hYp8Ux6FuaW9/jvxInsbJbJo5mjYm1RTNCWUgTUS/HQZ+DF/G7nj9hc/9RYm6MkABn9BrO80238fU1TfH471uYocY6300BM9n1M2NqG9TsPxdqcmKBBKUJGnN2JMz0b+fqtR515KYpSAb895TuGHppCbs7cWJuiJADDzuzIWd6NbJ72WJ30MZq+i5DWB77ktZQxWDuKMRMmIJc9rfNPlLIpyCVrwa0MSCnEP38adHW3TK+D80+UytF8/wpePe5jJtY5H6NBKVLyc0jBh4iN31eEZ/ZoMKbOzSVQKkl+DuIvwosNppg9n42j1aZ36+T8E6WSBPkY21eE1DEfo+m7SMnIRjyp+LGwjWBsu07OJVAqiTs50oiHYvGydsch3WVUKZ8gH+On7vkYDUqR4k52++6s3/OwbxQ+KyW6O0YW5ELO2DqZS05KXL3IhQ8ytffzPLt3ALYnippRvSQfrma29ruPh4tG4bdS65SP0fRdVUjPomV6FkcOLOeWNRn894KjNOw5uPrNandBT03tJBnu5MhLDx7j8ZWf8N/Oz3BXx+3V7yNQvSQv6Vl07HAmqwo+4zdHevCvs3/A6hyFPqUE0Iy2lKrBvRf3YIm/G08dHhKdBxu0oGddaarXJdo0rsd1Azowdm1TdveNws60qpekRkS4Z1A3Zn2Xzqwmw+uMj9GgVA06H13NC53ms27JR2zac7j6Fwza9bauLS1SV/h1j/3cwbvMmv1e9S+mekl6Lmv8LQ81nsPHH8zEtqOw11ICaEbTd1XFbQZf5C/iPK+Hv7/dlIfvHlW9a4bsSBlvzWqlmhTkcvK0n3Kft5Cite+yf107mp9SDaegekluCnKxXnX2cyssfIPPF3TgnEFDqnfNBNCMBqWq4jaDxfhJE+i8dTob395Ct6zLq/eggxb0VJIMVzMebFIoZtusx2jeYIzqRQlP0F5LKVJMSs7T+Du3wNPprOpdN841o+m7qhLUDBaPhxu8C8j46hnMhB/H7agWJcYENIOFRwydDy7BnnCV6kUJT7BeMJzhW1kn/IsGpaoSvHdO/xGk4MeLje0rjMvOQyUOCGim6yACgcn44rOzWYkDgvUijl6wiyjc+GmsLatRNChVh/QsyB4NfYcj3jT8WBQZL+vq9a3yJXM27OGtaW+z+d26ue5V0pOeBYP+gHjTsPFQZLx86T29WpfcfegYe9fmxPXcE6WKBPTiSXMmYBsvU3Z3qv5143iukvYpRQO3RuPb+Cn/XLiXZu+/S3rzk2jQ9ZyILvO/nG+YPfs9Xk99ghR82Kuewxo1I/qQLB4AACAASURBVK7zv0oVcPXi37SAfy/aS9NPptOzbSPSOp8d0WWOFft5aNoqvln2Ma+nPoEtPixvWlzOPVGqQWACdn4O4zafzMdfbueyhmNo0+fiKj3n/etzaDr1esRfjMThXCUNStEiPYu0XWu43/8Uxvbjf20q5tYZSMeznNrIykmAQN/w8w0WfjKb/R9O47etjlDvkB8xNj5/EWzOwYojwShRIj2LlF1rGF38JMb2Y098E26b6WijINdJ6dVvUeZCnIWbFzPt3Tf4Zk8nftllJ6nbfFjYGH8Rkp8TV05GiQLu4IRRnpe545vHsJbYmOX/QgIBpSIfU5DL92s/5oX8k7G+Xch9niK8Ep960aAULQpyYfZoLOMDAWMX8fX7/6Vn28mw7DWwi53zlr8Oo2aWiKAgl+8/n0jmqskMTLGxfvAilhfbD8XGwzdpfdCdm5KQMHrZ/ukrtGs6CZZPAn8xYINY4Cnd+vEvGYdn1mh+Ymyur5+Gt//TmF2p+HxF2HhJjcO5J0oUKMjlpA9/j8GPALbvGDL/Seg1FObc70yGhRN9TN547FmjaWj8/MaksKDHaMhPwecvxocHT/q5pMTsS52IBqVokZ8DtrOJmwEQIaPgXcxWP0LQpLfALGq3dmMm/JhGvmOIGATA9sOAWyg6qT03fejlgkMZGpSSkTB6abHxLQy+0noxdinNmG+/wMwajcf4EQFMERzdh4ycwdzpb/LWvgxeaX+mdhYnI65mBEczYgz2pvlYmxc4fiNAiF7sWfdh2X4sgXri49LOKTB4Jhty5/D/8hozbG97bsiI0XcKg2o3WmRkgzcNsBDxIC264RUnIJWahx2YRV2Qi/3JkxjfMazgM0SgbT/qdb+AHzf5hpO+ejVuOySVahCsF8uLr10WKRSfqBexHM3Ub4G9YCxL3nsejO0EpAD1W0B6FvV7XMBFRR/z/dRfql6SkSDNgICAhY1t+8P6mMMbFrF2yh/BdiowBmfpIr7fCkC3My/j9kZfUG/e7/Bv+aL2v08ZiDFVX7pCRH4CPAL0ArKMMXmVKZeZmWny8ip1amIRyOu66ReDu/21ASMWVqezoVVPSGuMWfxvjO1DAscsy9kzBQOWFxBsfxFiDIiFuB3Y0vGspcaYzFh+zepQFc0ktV4CfUdz7scE0i+AEQ+c/UusooOw52vMt59jjE2x8eCxBI8JtKgEvPXgsqew59yP+Iocf+VJO57CEZGE1Yz6mBACPmbZa5hAl4ABW0DwYHfIoii1KbsPHuPkvZ/hMT5nKDmCiOVUcowNlgeMca5hwHhSsW6dFRd6qW5LaRVwLbAgCrYkPulZ0CQdbB9gI05CDhGwjcG35XPsvHGYhc+A7cMCjAjSZRCccYtzDWM7/Qn+ouMtKMGO28UTq4BqJkBgSsHRfU5t1v1YANs2TP9iHcV5r2G2LATbj4UhRWw8Z4xAug52HAzG0cba97D8xYhTgVa9JCsBH2Ps43pBwAPYxoaCL6i/aQ6d9nxCKsV4xSBYjl4GjHT8i/E7PsYuRnD8k2UXx41eqtWnZIxZC26TUHEIzML2F4HlQRCnRSSAsbEwGMPx5rRleWDwH5yyKyYfLweC8ReD2NhYWHG6eGKkqGbCENCMrxDBxogFViqN66dgHfY5zue4Xizod6NTbsviki0Ieg2F/M+Ot7ZE9ZK8BPkYCfgK24clAsZH6b+UOCm/QWF8jDElA7DiSC+1NtBBRO4E7gTo2LFjbd229gld8BCQ/Bw89Vtg5j6A8RWC2BjEEdSQsSWjZMKUM/WbYx3dH7eLJ9YUdUYvUFoz9VsgR/fhzcjmQoDx7x8fVSVSjl7Ss6BNb6SCqQfJTJ3RTBk+Rtw08PFReFYKnHFzaS2ElKtoqkosqLBPSUQ+BNqGOfSgMeY995z5wO/qfL63IkLnn5QzD6UsYp3vrQzR1kyd1QuUnn/Stm/EeoH414z6mCgSOl8JIl4RPNZ6qbClZIy5qDYMqRMEr86bADtAVhXVTBQJaEb1olSGJPAxOiQ8ViTADpBKHKF6USIlQTVTraAkIteIyFbgbGCWiMyLjll1gATYAbImUM1UEdWL6iVSElQz1R199y7wbpRsqVskwA6QNYFqpoqoXpRISVDN6DJDsSTOd4BU4gzVixIpCaiZaq3oUOWbiuwBttTybVsCe2v5ntWhLHs7GWNa1bYxsUT1UinKs1c1Uzski2ZiqpeYBKVYICJ58TwsNpREszfZSLS/f6LZm4wk2jOIV3t19J2iKIoSN2hQUhRFUeKGuhSUXoq1ARGSaPYmG4n29080e5ORRHsGcWlvnelTUhRFUeKfutRSUhRFUeIcDUqKoihK3JBUQUlE0kXkExFZIyKrReQ3Yc4ZJCLfi8gK9/WnWNgaZE++iHzl2nLCssbi8E8R2SgiX4rIGbGwMxlRvSiRkmiaSUS9JNuKDj5gtDFmmYg0ApaKyAfGmDUh5+UYY66MgX1lMdgYU9aku8uB7u7rLOAF96dSfVQvSqQkomYSSi9J1VIyxuwwxixzfz8ErAXax9aqajMUmGgcPgeaisjJsTYqGVC9KJGShJqJO70kVVAKRkQygP7AF2EOny0iK0VkjoicWquGnYgB3heRpe7OmaG0BwqC3m8lsf8TxCWqFyVSEkQzCaeXZEvfASAiDYG3gd8aYw6GHF6Gs7bTYREZAkzDabrGivOMMdtEpDXwgYisM8YsiKE9dQ7VixIpCaSZhNNL0rWURCQFRyyvG2PeCT1ujDlojDns/j4bSBGRlrVsZrA929yfu3GW6A9d0ncbkB70voP7mRIFVC9KpCSSZhJRL0kVlEREgJeBtcaYf5RxTlv3PEQkC+dvsK/2rCxlSwO3sxQRaQBcAqwKOW06cIs7SmYg8L0xZkctm5qUqF6USEkkzSSqXpItfXcucDPwlYiscD/7I9ARwBjzH+B64Oci4gOOAsNM7Ja1aAO86+rXC0wyxswVkbuD7J0NDAE2AkeAW2NkazKielEiJZE0k5B60WWGFEVRlLghqdJ3iqIoSmKjQUlRFEWJGzQoKYqiKHGDBiVFURQlbtCgpCiKosQNGpQURVGUuEGDkqIoihI3aFBSFEVR4gYNSoqiKErcoEFJURRFiRs0KCmKoihxgwYlRVEUJW6odlASkfki8rNoGOMunz5ORL4TkdxoXLOC+40Xkcejfa5SeVQ/Sk2gukpcKhWURCRfRI6KyGER2eX+IRpGciMRyRARIyLlbZdxHnAx0MEYE7oZVdLiiv5pEdnnvp4O7MdSxvk3isgWEflBRKaJSPOgY81F5F332BYRuTHo2MkiMl1EtrvPIqNmv9nx+6p+apAo6+eXIpInIoUiMr5WvkAVUV3VLCLyiIgUu3/fwKtLTd83kpbSVcaYhsAZQCbwUA3Y0wnIN8b8EGnBCkQV79wJXA30BfoAVwF3hTtRRE4FXsTZ06UNzh4ozwed8hxQ5B67CXjBLQNgA3OB66L/FSpE9VNzRFM/24HHgVdq0N5oorqqWd4wxjQMen1T0zeMOH3nbq87Bzgt9JiIWCLykFsL2y0iE0WkiXs4sC/8ATfinh1S9nbgf8DZ7vFH3c/vEJGNIrLfreW3CypjROQXIrIB2BDOXhGZKiI7ReR7EVkQ5KBDzxskIltF5I8istethd0UclozEZklIodE5AsR6RpU/lkRKRCRgyKyVESyy/s7hjASGGuM2er+fccCo8o49yZghjFmgbvl8sPAtSLSSJzdJa8DHjbGHDbGfIazs+TNAMaYXcaY54ElEdgWVVQ/8asfAGPMO8aYacRod92qorqqEV3FhIiDkoik4+xUuDzM4VHuazDQBWgI/Ns9dr77s6kbcRcHFzTGvAzcDSx2j/9ZRC4EngRuAE4GtgBTQu55NXAW0LsMk+cA3YHWwDLg9XK+XlugJdAe5z/6SyLSM+j4MOBRoBnOTo1jgo4tAfoBzYFJwFQRqQcgIueJyIFy7nsqsDLo/Ur3swrPNcZswmkZ9XBfPmPM15W8Vq2j+olr/SQsqqsa0RXAVW7gXS0iP6/g3OhgjKnwBeQDh4EDOA/geaC+e2w+8DP394+Ae4LK9QSKcbbizQAM4C3nPqOAz4Levwz8Neh9Q/d6Ge57A1xYme/gnt/ULdPEfT8eeNz9fRDgAxoEnf8mTqsjcO7/go4NAdaVc6/vgL6VtMsPnBL0vrtrp4Q59yPg7pDPtrn2ZwM7Q47dAcwP+czrXj+jsn+76rxUP4mhn5DPHgfG14Y+VFdxq6veQDvAA5wD7ACG1/RzjaSldLUxpqkxppMx5h5jzNEw57RzxRFgi/vg20RwnzKvZ5x0wz6cGkOAgrIKi4hHRJ4SkU0ichBHxODUOsLxnSmdN97i2hBgZ9DvR3DEGLjX70RkrdscPwA0Kec+oRwGGge9bwwcNq4yKjg3cP6hCo7FGtVP/OsnEVFd1ZCujDFrjDHbjTF+Y8wi4Fng+sqUrQ7Rnqe0HadTMEBHnCi/C6cmUK3ruX0mLXBqdgHKu+6NwFDgIpyHkRG4VBnnN3PvEaCja0O5uHna3+M055sZY5oC35dzn1BW43RSB+jrflbhueKMhkkDvnZfXhHpXslrxRuqn9jqJ1lRXVVNV6GYapStNNEOSpOBe0WkszhDM5/AGb3hA/bgjP6KZEjhZOBWEeknImnu9b4wxuRXsnwjoBCnFnOSW74iHhWRVPeBXglMreR9At/RKyJ/4sTaaHlMBO4TkfZuh+lonGZ5OF7HyfNmu0J9DHjHGHPIrU29AzwmIg1E5Fwc8b8aKOzmk9Pct2mB/HKcoPqJoX7AGS3masIDeESkniT+CDLVVRV0JSJDRaSZOGQBvwbeq2z5qhLtoPQKjgNcAGwGjgG/AjDGHMHpgFsoIgdEZGBFFzPGfIgzOuhtnHxmV5xOvcoyEaepuw1YA3xewfk7cXKu23H+895tjFlXifvMwxlq/bV7v2MENd9dB3C4nPIvAjOAr4BVwCz3s0D5w4FRM8aY1Tgdr68Du3GEd0/Qte4B6rvHJgM/d8sEOIqTwgFY576PF1Q/sdfPQziaeAAY4f5eE8OsaxPVVdV0NQxn4MQh1+anjTETKnHfaiHh0851DxEZBLxmjOkQa1uUxEP1o9QEdVFXuvadoiiKEjdoUFIURVHiBk3fKYqiKHGDtpQURVGUuCEmQz1btmxpMjIyYnHrhGfp0qV7jTGtYm1HbaJ6qR6qGSUSYq2XmASljIwM8vLyYnHrhEdEtlR8VnKheqkeqhklEmKtl8RM3xXkQs5Y56eiVITqRYkU1UzMSLyZ2gW5MOHH4C8CTyqMnA7pdWbfrTrPsWI/9VI8lS+gelEiRTUTUxKvpZSf44jF+J2f+TmxtkipRe55fRm3j1/C+p2VXD9U9aJEimompiReUMrIdmov4gHLA99v1SZ2HcG2DZkZzcjN38/lzy7gwXe/Yv8PReUXCtaLJxXqt9C0TF2iKmk49TExJfbpu4JcpyaSke00kUPfh6PfMDi8BzZ8AEsnwIrJ2sSuA1iWcE/X/dzqX8nruzry5JICZn21g7+ddYwf1d+A1bkMzfQbBgi07QtzH9C0TF2hrDSc+pi4JipBSURewVm5drcx5oTtiMskVDSXPVXaaVz2FBzdVzpgBc4XAWM7r0ATWwWTEFRXL/X9RfzMk8olP53Ciws2cd6iBzDiw/Z4sfqPgL7DT9SLJ9XZfzM0LaOaSQiqpJmy0nChPmfnCkAc3QQfVx8TE6LVUhqPs73wxEgKFW9agNdfhBg/tu8Yhxe9TCP3vfEVIrNHg2074uh5OTRsXSIyY4FlAeKIKyPut55XShhPFfRSysn4jtGxYBp/6ZsOH/uwsDG+IkzeOFj+GtL9Eji0A/yFJU4F42gl4JBUM4nEeCLVTCAN5y8qScOtnBykoUKYdZ/zOzgtop6Xq4+JMVEJSsaYBSKSEWm5LY36k24LqYBgqLdvNUUIHiyMCB7bhwUYA7JuppPjBRALPGkntqSCqUwTXYkJVdULGdmOc/H7AQPLJ2Fd/lfwpmF8x0AMgsH4izDrZgbtRmY5TqXvjc4rnC5UL3FNlTSTnuWk3FZOguWTnKBjecDyOjsoiYDtD7qJH9bNco5D+GxNANVLjVFrfUoicidwJ0DHjh0BaHPq+exefT0dvnkDwZCC4WCzU9nZoCdrTGeu3DYWMX7E9S7GrdGI5XXEkjkq/M10SGfCE04vpGdB/xGQNw4wYPuc1Eu/YcjhPZgNH2D7ixBjjmvGuRiOXgIaCFeBUb0kPGVqJj/H0Yrxg99A+/5wcl+nj3HO/W4rOoBxWtYDRpZOA+fnwK41ToCq30L7JmuQWgtKxpiXgJcAMjMzDUCjeik0Gnw7fPse+AoRbJocWEWTQxvoOXI67GqPmXXf8WCEcSs3tkGO7CupCRfkOrWhQF44XC5ZRZNQhNML4DzfFZNLUjLLJzkOx5OKXP5XZOcK7KWvgvEd1wvgOJMAqpekpEzNBNJ4vkLAhu3LnQAzcjiMmgULn4F1synZwdxAkw6l+yUDZcVyXtrXVGPEfkh4oInddVDJww486MxRyG1zkcxboecV2J5UfFgUGg+PfNmMgv1HHNGMv8KpPee9AuOvdGoywcOANRecPAT0cuGDTqvpeA24yAk8Vz6DddscjnW5HFssfEbwSQqm03lOedVL3SOcj/EVwvwnnePDJsGVzzhpu0DXQEADgQoLtvPe2G4/t6V6qSFiPyQcHNEM+gNsWVzSJA7MJ8nIhiufQQBPQS725hzm/9Cdtz9P461nFjD1tMX09heXXCvgnEZO15xvspKeVVKLDW41BeaTpGdR/5YpFG5ezNyZbzFhewf6rGjInzoYrPwcUL3UPYJ9TKDV88185/3I6U5XQJveJ2ogtJVVmf5spVpEZT8lEZkMDAJaAruAPxtjXi7r/MzMTBN2scRA7rYSOdttB47ym8nLsb/9gjfTxuCh2EnnedJg1MykFYqILDXGZMbajuoQNb1ASSouKI0XrBfbNoyZvZaXP9vM1f3a8fezC/FOvKqkHyHJ9QKqmVIU5DotpG/mO60e8Tit7uzRZd882C9VFIiSYABErPUSrdF3w6NxneM14JyxFeb42zetz+Q7BzJmVhNuWAy/bpnHed1b4u1/U/liSALRJDpR0wuE6cgurRfLEh66ohfNG6Tyt3nrOVrchn/fPIOUVVM43qekeol7oupjQrMyZaXfgp99cNAKrBIRbkSeDpipNvGRvgsleH5BWaIpyCUlP4dH+mczrsXVjJrRg3Mat2BcsZ+0cIJxy6hokpAK9CIi/GJwNzodWcWaRS/xtyPn8rvb/0Gq1+1SLSvwqF6Sk0AfU3mVjfJWgyhLEzpgJirEZ1CqSDQhwrh15HQa/6Qvr789FV59EoMPCedEVDTJSSWdzJXL72ZISiGF29/l2fHCfbfdjGfbEnUydZFAVqYsynr25WmiMpVppULiMyhB+aIJI4zrskfTY8MBPGuLEbEx/iIk1ImoaJKXSjoZC5s08WM2f8Yf3+nHU61zEHUySihlPfvyNFGZypFSIfEblMqjDGGcfu4V+L5+AZ+/GCNeUkKdiIqm7hKkGcuTSvrpF/P8kgL6ZHbkJnUydYNI+gfLevYVaaKiypFSIfETlKIkGO+tM/lwzts8v7ktP93Zlp+mhymrokl8Ih2AEKKZYR3O5Ev5igdzC2g56CUubbBBnUwyU5X+wXDPXge+1DjxEZSiLJjBl17LhA/h4WmrOaVtY/qmN60525Xap6oDEII0I8Bfhp5Gg93L+PKzRbS+/Dr6q5NJXqLRP6gDX2qF2K/oANHZ6TEgmI/H4Hl1KM9l+2jVKI1fTFrG90eLKy6vJA5R2hnUuz2PB/c+wH3eqZzywQi2fTU/unYq8UPoZo9V6R/UHWlrhfgISjUgmMa7Puefw/uz4/tj/PHdr4jGJGElToiGXgDynUEOHmxS8DFnxlscLvRF11YlPghenqqqLZxIdVeVXW+VOEnfRaNDOczghwHpzbjv4h78bd56Lu7Vhqv7ty85X3PDiUu0BiAEaUasFOb+0I28N1fywogzkFLLjKN6SQaq2z9YmakqgWO71sDs0c6qEZ40TfVFQHwEJagxwdx9QVc+Xrebt957hx/tO0KjnoNVMMlANAYgBGnGk5HNpZtbMmb2Wl5ZmM/t53UucTLHDsLif6telBN1F25pNMvj7NMU2N3AX6hz3CIgfoJSNAgjGE9+Dv/p1ZCGOx8j9TMfZvHfERWMEiBIMz8zX9Di5I+YPGcD2fVOp8fcm0oW4gygelECBA98KLV1uk3JNhg4i7jqHLdKk1xBKZggwbQSwRY/FgbbX4yoYJRQCnKRiUO5xl/EkBQPc+cMprtdhAQHJFC9KCUE92MHb51ueZyf/mLnsyFjtRITAckblEIEI5YHn23jxyLV40FUMEowrl7E+EkT+KHQR3GKl1QM7t7ZjrNRvSQUa7Yf5JP1uzmlbSNOObkx7ZrUO7G/sKqE9mMHb2cB2gdZRZI3KIUIRi57il07tvKrRQ0Y2q89I9sVqGCUEoIHPXhS8Z5+I8OWZDOm//f06pKhe+ckKMu+/Y6/zVt//H2T+in0OrkRp7Zrwuntm3B6hyZ0btEAy6pCoKrM6g6gg2QiJHmDUhjBtAe6HlvJ4yu2kX3hXXRp1TDWVirxQohermuXyaQdixm+7gfev/x8WjeqF2sLlSowYmAnhvZrx9e7DrFmxyHW7jjImu0Hee3zLRT6nNRsozQvp3doQv+OTTmjYzPO6NiMZg1SK3eDigbc6ITbiEneoARhBfP7y05h7qqdjJm1lpdHnel8qDUZBUrpxQuM/UlfrvhnDn98ZxX/vWVASdpH9ZJQNKqXwoBOzRnQqfnxz3x+mw27D/PV1u9ZufUAK7ce4D+ffoPfdvqbu7ZqQFbn5pzVuQUDu7SgbZMqVkp0pfmISe6gFIZWjdL4xYXdeGrOOhZt3Ms5ad9oTUYJS7fWDbn/0p48Pmst01duZ2i/9lrzTRK8HoteJzem18mNueFMZ4HMo0V+Vm49wNIt35GXv5+ZX+5gcm4BAF1aNeC8bi05r1tLzunWkoZpQa6zvEqKrjQfMckflMIIZtQ5Gby6eAtjZq9lRr9cLK3JKAFC9HLruZ2Z+eUOHpm+mvO6taSF1nyTlvqpHgZ2cVpGAH7bsHbHQRZv2sfCTXuZmreViYu3kOIRzsxozo96teGKpgW0nXaDM3UgMHAqc1TJRXWl+YhJ7qAUqNWGCKZeioffXdqDe99YyaLTTuE8rckoULoVZHmg/wg8fYfz1+v7cMU/cxgzay3/OEdrvnUFjyWc1r4Jp7Vvwh3nd6HQ52fplu/4dP0ePlm/m7/MXMM+z3uMTjmGB4OxbWT2aGjTWwc5VIPkDkr5OSWTH23bWcXBFcyP+7bnxU+/4aGlfj68eRrebxeqcOo6pVpBfsgbBysm02PkdO6+oCv/+ngj155xFudpzbdOkub1cE7XlpzTtSV/GNKLb/cdYcXiIkzeWxjjRwT8tp+ln7xHxtV9aP39l5rqrQLxsSBrTZGR7U5oczH28ZV9PZbwlzOOMOTAZOav3w3Zo1UwdZ1A/p/A8GBzPEX3i8Hd6NTiJKa88xa+bxZoQFLo2OIkfnzlNXiv/Ad4vNhY+EjhqbUtGfjkR7z19mRsX6GuKh4hyd1SSs9yUnbB69wF0i0FuWQuGMUZKYUUL3qX4h4zSMkYGFt7ldgSyP+vnATLJ4HtO56iq5fi4Zlzijjl/YexPvFBzt+15qs4ZI5C2vRG8nNIy8jm7/V689bSrczJy+cK4yVFfBjxcrTNQBrH2tYEILmDEjidjm16n5huCdq2wBgfqxbNop8GJSUwLLzvjSdopr+9Cr/4sLAx/iJEBzkoAYKmE3TBmXriu7gHeQu78O3SeUzZ3Yk1rx7kJwNWcef5XUhvflJs7Y1jkj8oQfgJbm6qxviL8OPh+fyTec5vk+JJ7oymUknK0Ix4U/EVF2GLl1Qd5KCUg9djMfD8yxl4/uX033WI/+VsZl3eh0zJW0O97hcw9Mpr6NhCg1ModSMohcNN1Uh+DqvkNN6f6ePdZduOz1lQlBNIz8IaOYPPP5rG39e34vfFXdG2tVIZurdpxNNnHcOsexLjK6Lwm3e55R97OOXMi/jNRd1p2TAt1ibGDXW7WZCeBdmjOePcSzitfWM+/XgW9gLdKVIph/QszrjpL+xq0pc33n1b9aJUnpWTEV8hFjb1LD93dNzO5NxvGfS3+bz46SaKfHbF16gD1N2WUhAiwh9PP0T/Tx6Gj33g1Y3clLKpl+Lh6ayjDPj0QdWLUjkKcmH5awT2WRLLyyVDrmdevd48MWstT85Zx9SlW3nimtPJ6ty8/GslOXW7pRTEwEPvkyZFxzuxdfimUh7n/vCB6kWpPPk5zm60AAj0vxHSs+jaqiEvjzqTV0ZlcrTIzw0vLuZP763iSJEvpubGEg1KAAW5WCteRwBjwIhHZ+orZVOQiwTpxY9u/FfTiMhlIrJeRDaKyAOxtidiAnPgxAPees7oziAuPKUNH9x3Preem8Grn2/hin9+xpdbD8TI2NiiQQmO12IEsEX4uN7FmopRyiZIL0aEt/wXsLtpn1hblbSIiAd4Drgc6A0MF5HesbUqQgJz4C58sMxU70mpXv581alMvmMghcV+rnthEa98thljTJgLJi9RCUrJVIuxrVSe/+5MVm//PtZWJTUJrZlStd403vZn86+PNsbaqmQmC9hojPnGGFMETAGGxtimyHEHVgGQU/YAmYFdWjD7N9lc0KM1j81cw2+mrOBYsT/suclItYNSstViCm+cxvqUXvwvZ3OsrUpaEl4zQXqxRs6g55k/YnLut+Tv/SHWliUr7YGCoPdb3c9KISJ3ikieiOTt2bOn1oyLiMCivx+PcX6WEZianpTKSzcP4P5LezLjy+3cZrqElQAAE/JJREFU8OJidh86VsvGxoZotJSSqhbTMM3LPzt8wtYv57Pj+6OxtipZSXzNBGq96Vnc1+sA93jf4+3p78baqjqNMeYlY0ymMSazVatWsTYnPOG2PgEnOIW0nixL+MXgbrx0cyYbdh3m2ucXsbkOVHyiEZSSrhZz4Y7/MtE7hg/fnxlri5KVCjWTEHoBKMil+dSf8FvrTe7Zci+bl38Sa4uSkW1A8Kz2Du5niUdw6tfywPdbIW98ua2ni3u3YcqdAzlS5Ocn/1nMup0HY2N7LVFrAx0SqRYjxk+q+Ni/+mNnaGaYWoxSsySEXuC4ZixsUsTHsk+nq16izxKgu4h0FpFUYBgwPcY2VY1A6rfnZc4Q8aXjnQWj/eWvJt43vSlv3nU2XksY/tLnrN2RvIEpGkEpOWsxnlR2FNWnYOLdMP7KCnPASkQkp2bEQ4O9X2KPu0L1EkWMMT7gl8A8YC3wpjFmdWytqiZfz3OCkLGd4CTWcb9T1vSCbq0b8sZdA6mX4mHE/75g057DtWx07RCNoJR8tZgLH8S6/CkeTX2V7lvfwlRQi1EiJvk0M+AWPJbFxd6liB2mz0CpFsaY2caYHsaYrsaYMbG2p1rk5zibjgawPM4WO+UMFw/QqUUDXvvZWQDc/L8v2HUw+QY/VDsoJV0tJj3LWQ167XRS8WERmCMguv11lEhKzSDuVigG55/qRSmDjGxnaSossLxOQMocVemNRru2asiE27I4cLSY28Yv4YfC5Fr9ISpr3xljZgOzo3GtmBMYsuluo+5H8OMlNfNm6DtcJ9VGiaTTjLuumQGKxcOnJ13KxcPvVb0oJxJoXYfu8RYBp7VvwnM3ncHt45fwu6kree7GM7AsqbhgAqArOoQSGLKJjWCxtelZDCt8kI1Zj6mDUcITtK6ZIGxqfw137B/BoqIuMTZMiVuCphRUdWDM4J6t+cPlvZizaicvfLqphgytfTQohRIyW7/JkIdZZZ3C+EX5sbZMiVdC1jXrctHPaNM4jWc+2FDnlohRIiR4Mu34K2DmvREFp59ld+bKPicz9v31LN60rwYNrT00KIUSskZV0x7n8eN+7Xh76Ta+P1oca+uUeCREM2mdz+aeQd3Izd/PoiRxFEoNETqZNm9cyajNSrSgRISnrutDRosG3PvGCg4cKapF42sGDUrhCGla33/STHr51jI1r6DiskrdJGRds2En76Bt43o8+6G2lpRyCLSyCfQHGfAdg4XPVGo5IoCGaV6eHdafvYcLeWjaqloxuybRoFQebtO6Td5YpqSNocX8B/Bv+SLWVinxSlAqJu31a/hH1+VkFozjq88/iLVlSrwSaGVnjgIrxf3QwPq5FU6oDeb0Dk347UXdmfnlDuau2lHjZtckuvNseQQ1rVPwM9Q3DzPhEzhjBLTtC0f3VXn0jJKEBKdifIWcvf5JslL8MO8tzOFfIvWaqF6UE0nPOj6tgLxxOHMKDFgWkUxFueuCrsxZtZOHpq3m7K4taVI/pcIy8YgGpfIINK19xwCDJWDsIsh7xTkuFnh0K2zFJaAXfxGIILYfLwZj/LDwWdWL8v/bO/vgKOo0j3+e7pmELAJKeDUmoIgisosChsAdigouKivrvqmgLtbd6npn1elxeytaWlfl3h5rrZ6u3rnr3e0tqEDVniAUb4IgCwpu5CWsCGJEIgGDAgoISjLT/bs/uodMhgDOSzLdk+dTlUpmuqfzTOabfvp5+nme3+kZehvUzPH0YxfBhBlpXfhGbYtfff9b3PTsG/z61R089t0h7WB07lGndDoSofWW2cjm2bhOE2JMUvrXbQ6t9SSjJPeflJTCkmkYN44IXgu26kU5HTnqX7pzVH9mrq/jlivKGVLWLfd2tjHqlM5EIrQeOpnYxpdg84tEcLzOfbG0a19pyYlUjIcsmYbrOogxGLEQ1YtyOlL0kwkPjL+IhVs+5rFF25h7dxUi4WqqVaf0dSmvpLi8kuePjeKL917nJxNG0NU9ovcIlFMzYir0Hoyzcw2Pr9nPBZ2Pc9uPpqhelDNTX51xxNStJMoD4wbyyIJ3Wb7tE759aZ82MrJtUKeUJtdeN5Fr3+mC9eVAHhh/Ub7NUYJOeSXR8kr6RnYxfdE2zo8NoCrfNinBJlHFmeG9JYDbKiv433V1PLF8B+Mu6Y0dohFEWhKeJgN6nsW1g3rx4lsfcTzm5NscJSRMHllBry7F/PuK9/NtihJ0Uqo4WTIt7aVQIrbFP46/iPc/OcrCLeFaFUadUgbcP+gQtxz/I2tWLc63KUoYqK+m01tP8ehlR/nzrs9Yt/NAvi1SgkyL1Wktr0Amg6VQbhjSl0v6duWZVR/guEkN3AFfhFLTd+lSX82Q1+5gcLSR2Pr5uN0fxzr+md5bUlonKRVzo13E/LMeYfHifYy67BByvmpGaYXUKs5lDzan8tIokrEs4b6rL+TvZ29i6dYGJn7r3JNTgwFsT1CnlC51a/11c1yMicGSfwJMYD9gJc8kpWLEaeKfyzZTUb8AVsW9NXVUM0prJFfh9R6ccdHDhCF9uKBnZ/7j9Z3c+M2+SOqsvQC2J2j6Ll380NqIjRHLa4zM5SqjAQ+tlTRJTsXYRQzo1ZmoxBFcTC40o3opfJJncaaJbQn3XHkB2xuOeFPEU/QYxPYEjZTSxQ+tpW4tGxoMl737K4otBysXH3AIQmslTVIaIiNAvGYucacJsaPY2WhG9VL4ZFEanmDSZWU8vmwH//3GLkZPzb5Bt61Rp5QJfmh9eczh73aUcF3nWm794eTsP+AQhNZKBqQ2RP54Ab9/8QW2Fw/l12VXYGd6XNVLYZN80WHZcPntGa1+3Slqc3tVP55eWUvdgWP0z0GDblui6bss6BS1GT32eh78dDxvOxdmf8AQhNZK9kT6VdFn4sPMP1DGgposynVVL4XN6dZaSpPJIyuwLWF29e42MDS3qFPKhvpqfuzM4+rOdTy5PAf9JymLxQX5akbJEP8e0MSz67n03K48sfx9GuMZ9rupXgqb1tZayvA+ZO+unbhucG/+uKE+8P2Vmr7LFD+0jjpN/JdE+NGu6az74EJGX9gju+MGPLRWsiApHWPZRfzymllMWhhj5ro67r5yQGbHVL0ULkkDodk8G9x4VhHxlJH9WLp1H6++u49JpXsDe19JnVKmJIXWNnB7yTpq5+2i6pYpWP1G5ts6JYikdOoP3fkcf9N/Es+sivCD4eV071yU9a+YU72bmONyR1W/0A3iVFohaSB0tk5k9IBSys4uYcv6FUw6+GBgC2Q0fZcpSfl8sWwmsZopx17AzTDnq3QATqRjLMCFD1fz8IGfc3FsO08s35H14eOOy29W1vLa9k/VIRUaibJwgEX3w6IHWp5n6qtbfz5pu/Xmk9w38CAle9d57Qi5bGXJIRopZUpyqe/hPVgbZyLiEnebaPzgTxRnc+WRgzJQJYAkNLP63+DD1WBcLCfGvf0a+Nvq3dxWWZH5+jf11dRVL6Xvka5M/s6tOTVbCQj11fCHGz1HArDpBRh2h7cK9tKfNT+/+SWYuqj53LHhD978PONyi13EQ+4U4hIhCoEskFGnlA2J0Lq+GqmZg3GaiLk2cz+p4K5Mj6m9J4VNeSWMnQ4frT/xGY+8ZhKlc4/x8CtbmXfv6PQnOvuaOT/eyOziCJEuo4C+bWK+kkfq1oITa37sxryKPMsGN6l4Ibk9oL7ac0huHADLaWLIOXF+bv2CJyu/COSFr6bvckGiofaah1l63v0c2LqS995+reU+Z+q8T2zfMudEaJ2Tjn8leCRXzU2YwVkN63lydBNb6g8xa31d836n00zytrq1GH/0VRSHyO432+udKO1J/zFgR1OeNC0dEjRHP/XVXlTeYrsw/Oxj1B04xp5uw70iilOl/PKERkq5orwSPtnGzQ3TcCMOscXz+cqZQUnsEBw/Auuf9ab92inzzuqrT1TXGDdO3Fi4RrCxiGOzr8sw+uf1jSltgq+XRFpljF3Mo33v4fNXF7I/NoyeR7e3rLhK1kxSOga7GCbMIEYEyxjsaDRw6ZhsEJEfAv8CXAJUGmM25NeiPFJeCVMXw5tPwXtLAdffYADx+tUqqqDkHHjzaahd4UdWie0WiMWgj+czp+gV7HkG8B1Wasovj6hTyhV+mCxuHBswpgmW/QyDQU6IB3AaW4bWM2+C+HF/PxBj2Fz6HaLd+/GfdX14Z7HDyxVfUt79G/l6Z0pbkJJWkfhx7jr8LI7lYK+ei0EQ/OUGTpOOwWnk84P7uLvpIe7p18C4678fiBNLDtkKfA/4Xb4NCQTllVA2HHYsgxOrUQhgwBjY/ZZXwNACCwaMhXP6w8aZiHGIiv+aBAGaCKJOKVfUrQW32flYIhjjImIS1ykeYrUIrY3T6J18DLgiWJEiqm6+D4AZ3Vcy7e3tbPhokDqlQiNFLwDiOkQwGAMiiROGeJFSSamXrju8p+XrxOK5ur5ss3sz7MrhzeneAJxccoExZjug1YTJJKo4E+OHEO8iReTkVB7iTaMfO917WDMHnCZcLFzXJSqOd24KUMFDVk5JQ+sk+o/xPvz4ccAgGES8ixfED6DFhhue8PafeRMm3gi4xI1gJAIDxxPt1sdL6yx7kJ5OEzMtG9n7IfRIf+ZVEFHN+JzQS6N3MoHmK1xfN64VwR5+p1ddlVhTx7K9+wpODETY12csG3Z9xr+O6kL3/5vqHc+yPJ2NmJqvd5cXRORu4G6AioqKPFvThqQM+QWa115KrsKzol51XvK8PP91n5VewU9f2Mgvzn+Hwed2y2imXluRbaSkoXWClHJfMa6XkBMvLecg7C4dQ1HDHjrVLKJ7/DiCIW6Ehu4jKRt9K9ar0z1BifirTbqI43gVNjVzvOOHH9UMnNRSwMaZJzYJXtQ8N3YV55WM46rNT5y42MEFht8JgLv5JXp8vIo5xWuIMMVzSLheJLVkmrcOT0BONKdDRF4D+rSy6WFjzIKvexxjzPPA8wAjRowwZ9g93KRO8khee2nLbEA8RwMto2f/qxfQ2Fd4xBnJyxNHt6flZyQrp6ShdQop5b7ih9bGjeMY4dz9b2Dv/xO2n8s1AlYkSvn3HmvZ7W8sfxlkgxdjZT7zKmioZpJIaimgZk6zUxELsYs51u1SqtbciRH/vhOAFYGhk9m/dSXnOHEiuNjiIBaeZhKpPeMG5h7BmTDGjMu3DQVDsrM6Q3vJ+MG9eXplLQePNlJ6VnGeDD6Zdrun1JFDa6lbS/RQPWyahZjke0yCXD6lWSiJPLFdBBNmwL6anMy8CiMdRi9w8vLXXx1E+o/hrp1rsFc7yeM4+fC8Sfx+Qwm1G7/BrEjEc0h2kTeGps9lLavyOpBelFY4w9ImV1/ci6deq2Vt7QG+e3lZHg1tyRmdkobWGdBKaC311bBlLsQbvWo8sbyrXsS7okl1ZonX52DmVXuTC810KL1Aq4NVowBro16/GhAzNut3HqSWVfQbOpZjQ4bR6UB1szbKK7NaOjuIiMjNwDNAT2CxiNQYY76dZ7PCQXJBRCsXtd8s60Zp5yJW7/g0XE5JQ+sckXo1nIiCNs5svl/U2sTnEE6BVs3kCL8vRbbMhqP7idauYIq1min2OqRqFJSPAcac/JqQ6eV0GGPmA/PzbUcoOdWFro9lCVdd1JPXd3yK45r0J4m0ETrRoT1JDFUcMRW6lXtpuYAORVQCQnklTHwKyoYhbhwxDqJ6Ub4uiXPOKS5Urrq4J59/GeMvew61s2GnJiunJCI3i8geYBReaP1qbszqAHTQVUNVMxnSQfWitC1XD+rFnJ9Ucem5GQ4CbgOyrb7T0DpTzhBaFyqqmQzpoHpR2paunaKMGlCabzNaoBMd8kmB5f+VNkb1onQA9J6SoiiKEhjEmPavthWR/cBH7fxrewAH2vl3ZsOp7O1njOnZ3sbkE9XL1+J09qpm2odC0Uxe9ZIXp5QPRGSDMWZEvu34uoTN3kIjbH//sNlbiITtMwiqvZq+UxRFUQKDOiVFURQlMHQkp/R8vg1Ik7DZW2iE7e8fNnsLkbB9BoG0t8PcU1IURVGCT0eKlBRFUZSAo05JURRFCQwF5ZREpFxEXheRbSLyroj8Qyv7jBWRwyJS4389mg9bk+ypE5F3fFtOWhpcPH4jIh+IyF9EZFg+7CxEVC9KuoRNM2HUS6GNGYoD04wxm0SkC7BRRFYYY7al7LfWGDMxD/adiquNMadqurseGOh/jQSe878r2aN6UdIljJoJlV4KKlIyxjQYYzb5P38BbAeCs3pVZkwCZhmPt4CzRaRvvo0qBFQvSroUoGYCp5eCckrJiEh/4HLgz61sHiUiW0RkqYhc2q6GnYwBlovIRn8J8FTKgPqkx3sI9z9BIFG9KOkSEs2ETi+Flr4DQETOAl4G7jfGHEnZvAlvttNREbkBeAUvdM0Xf22M2SsivYAVIvKeMWZNHu3pcKhelHQJkWZCp5eCi5REJIonlpeMMfNStxtjjhhjjvo/LwGiItKjnc1Mtmev//1TvHWGUtcm2AuUJz0+z39OyQGqFyVdwqSZMOqloJySiAjwP8B2Y8yTp9inj78fIlKJ9zc42H5WtrCls3+zFBHpDFwHbE3ZbSFwp18lUwUcNsY0tLOpBYnqRUmXMGkmrHoptPTdXwF3AO+ISI3/3ENABYAx5rfAD4B7RSQOfAXcavI31qI3MN/XbwSYbYxZJiI/TbJ3CXAD8AHwJXBXnmwtRFQvSrqESTOh1IuOGVIURVECQ0Gl7xRFUZRwo05JURRFCQzqlBRFUZTAoE5JURRFCQzqlBRFUZTAoE5JURRFCQzqlBRFUZTA8P/kfGrx230//gAAAABJRU5ErkJggg==\n", 806 | "text/plain": [ 807 | "
" 808 | ] 809 | }, 810 | "metadata": { 811 | "needs_background": "light" 812 | } 813 | } 814 | ], 815 | "source": [ 816 | "from sklearn.linear_model import Ridge\n", 817 | "def ridge_regression(data, predictors, alpha, models_to_plot={}):\n", 818 | " #Fit the model\n", 819 | " ridgereg = Ridge(alpha=alpha,normalize=True)\n", 820 | " ridgereg.fit(data[predictors],data['y'])\n", 821 | " y_pred = ridgereg.predict(data[predictors])\n", 822 | " \n", 823 | " #Check if a plot is to be made for the entered alpha\n", 824 | " if alpha in models_to_plot:\n", 825 | " plt.subplot(models_to_plot[alpha])\n", 826 | " plt.tight_layout()\n", 827 | " plt.plot(data['x'],y_pred)\n", 828 | " plt.plot(data['x'],data['y'],'.')\n", 829 | " plt.title('Plot for alpha: %.3g'%alpha)\n", 830 | " \n", 831 | " #Return the result in pre-defined format\n", 832 | " rss = sum((y_pred-data['y'])**2)\n", 833 | " ret = [rss]\n", 834 | " ret.extend([ridgereg.intercept_])\n", 835 | " ret.extend(ridgereg.coef_)\n", 836 | " return ret\n", 837 | "\n", 838 | "#Initialize predictors to be set of 15 powers of x\n", 839 | "predictors=['x']\n", 840 | "predictors.extend(['x_%d'%i for i in range(2,16)])\n", 841 | "\n", 842 | "#Set the different values of alpha to be tested\n", 843 | "alpha_ridge = [1e-15, 1e-10, 1e-8, 1e-4, 1e-3,1e-2, 1, 5, 10, 20]\n", 844 | "\n", 845 | "#Initialize the dataframe for storing coefficients.\n", 846 | "col = ['rss','intercept'] + ['coef_x_%d'%i for i in range(1,16)]\n", 847 | "ind = ['alpha_%.2g'%alpha_ridge[i] for i in range(0,10)]\n", 848 | "coef_matrix_ridge = pd.DataFrame(index=ind, columns=col)\n", 849 | "\n", 850 | "models_to_plot = {1e-15:231, 1e-10:232, 1e-4:233, 1e-3:234, 1e-2:235, 5:236}\n", 851 | "for i in range(10):\n", 852 | " coef_matrix_ridge.iloc[i,] = ridge_regression(data, predictors, alpha_ridge[i], models_to_plot)" 853 | ] 854 | }, 855 | { 856 | "cell_type": "markdown", 857 | "source": [ 858 | "- Here we can clearly observe that as the value of alpha increases, the model complexity reduces. \n", 859 | "- Though higher values of alpha reduce overfitting, significantly high values can cause underfitting as well (eg. alpha = 5). \n", 860 | "- Thus alpha should be chosen wisely. A widely accept technique is cross-validation, i.e. the value of alpha is iterated over a range of values and the one giving higher cross-validation score is chosen.\n", 861 | "\n" 862 | ], 863 | "metadata": { 864 | "id": "HZVaWbqdK18h" 865 | } 866 | }, 867 | { 868 | "cell_type": "code", 869 | "source": [ 870 | "#Set the display format to be scientific for ease of analysis\n", 871 | "pd.options.display.float_format = '{:,.2g}'.format\n", 872 | "coef_matrix_ridge" 873 | ], 874 | "metadata": { 875 | "id": "e_XKJQx8K1Zo", 876 | "colab": { 877 | "base_uri": "https://localhost:8080/", 878 | "height": 423 879 | }, 880 | "outputId": "53978e33-b3b8-44a0-e4b1-24bf6132da79" 881 | }, 882 | "execution_count": 6, 883 | "outputs": [ 884 | { 885 | "output_type": "execute_result", 886 | "data": { 887 | "text/html": [ 888 | "\n", 889 | "
\n", 890 | "
\n", 891 | "
\n", 892 | "\n", 905 | "\n", 906 | " \n", 907 | " \n", 908 | " \n", 909 | " \n", 910 | " \n", 911 | " \n", 912 | " \n", 913 | " \n", 914 | " \n", 915 | " \n", 916 | " \n", 917 | " \n", 918 | " \n", 919 | " \n", 920 | " \n", 921 | " \n", 922 | " \n", 923 | " \n", 924 | " \n", 925 | " \n", 926 | " \n", 927 | " \n", 928 | " \n", 929 | " \n", 930 | " \n", 931 | " \n", 932 | " \n", 933 | " \n", 934 | " \n", 935 | " \n", 936 | " \n", 937 | " \n", 938 | " \n", 939 | " \n", 940 | " \n", 941 | " \n", 942 | " \n", 943 | " \n", 944 | " \n", 945 | " \n", 946 | " \n", 947 | " \n", 948 | " \n", 949 | " \n", 950 | " \n", 951 | " \n", 952 | " \n", 953 | " \n", 954 | " \n", 955 | " \n", 956 | " \n", 957 | " \n", 958 | " \n", 959 | " \n", 960 | " \n", 961 | " \n", 962 | " \n", 963 | " \n", 964 | " \n", 965 | " \n", 966 | " \n", 967 | " \n", 968 | " \n", 969 | " \n", 970 | " \n", 971 | " \n", 972 | " \n", 973 | " \n", 974 | " \n", 975 | " \n", 976 | " \n", 977 | " \n", 978 | " \n", 979 | " \n", 980 | " \n", 981 | " \n", 982 | " \n", 983 | " \n", 984 | " \n", 985 | " \n", 986 | " \n", 987 | " \n", 988 | " \n", 989 | " \n", 990 | " \n", 991 | " \n", 992 | " \n", 993 | " \n", 994 | " \n", 995 | " \n", 996 | " \n", 997 | " \n", 998 | " \n", 999 | " \n", 1000 | " \n", 1001 | " \n", 1002 | " \n", 1003 | " \n", 1004 | " \n", 1005 | " \n", 1006 | " \n", 1007 | " \n", 1008 | " \n", 1009 | " \n", 1010 | " \n", 1011 | " \n", 1012 | " \n", 1013 | " \n", 1014 | " \n", 1015 | " \n", 1016 | " \n", 1017 | " \n", 1018 | " \n", 1019 | " \n", 1020 | " \n", 1021 | " \n", 1022 | " \n", 1023 | " \n", 1024 | " \n", 1025 | " \n", 1026 | " \n", 1027 | " \n", 1028 | " \n", 1029 | " \n", 1030 | " \n", 1031 | " \n", 1032 | " \n", 1033 | " \n", 1034 | " \n", 1035 | " \n", 1036 | " \n", 1037 | " \n", 1038 | " \n", 1039 | " \n", 1040 | " \n", 1041 | " \n", 1042 | " \n", 1043 | " \n", 1044 | " \n", 1045 | " \n", 1046 | " \n", 1047 | " \n", 1048 | " \n", 1049 | " \n", 1050 | " \n", 1051 | " \n", 1052 | " \n", 1053 | " \n", 1054 | " \n", 1055 | " \n", 1056 | " \n", 1057 | " \n", 1058 | " \n", 1059 | " \n", 1060 | " \n", 1061 | " \n", 1062 | " \n", 1063 | " \n", 1064 | " \n", 1065 | " \n", 1066 | " \n", 1067 | " \n", 1068 | " \n", 1069 | " \n", 1070 | " \n", 1071 | " \n", 1072 | " \n", 1073 | " \n", 1074 | " \n", 1075 | " \n", 1076 | " \n", 1077 | " \n", 1078 | " \n", 1079 | " \n", 1080 | " \n", 1081 | " \n", 1082 | " \n", 1083 | " \n", 1084 | " \n", 1085 | " \n", 1086 | " \n", 1087 | " \n", 1088 | " \n", 1089 | " \n", 1090 | " \n", 1091 | " \n", 1092 | " \n", 1093 | " \n", 1094 | " \n", 1095 | " \n", 1096 | " \n", 1097 | " \n", 1098 | " \n", 1099 | " \n", 1100 | " \n", 1101 | " \n", 1102 | " \n", 1103 | " \n", 1104 | " \n", 1105 | " \n", 1106 | " \n", 1107 | " \n", 1108 | " \n", 1109 | " \n", 1110 | " \n", 1111 | " \n", 1112 | " \n", 1113 | " \n", 1114 | " \n", 1115 | " \n", 1116 | " \n", 1117 | " \n", 1118 | " \n", 1119 | " \n", 1120 | " \n", 1121 | " \n", 1122 | " \n", 1123 | " \n", 1124 | " \n", 1125 | " \n", 1126 | " \n", 1127 | " \n", 1128 | " \n", 1129 | " \n", 1130 | "
rssinterceptcoef_x_1coef_x_2coef_x_3coef_x_4coef_x_5coef_x_6coef_x_7coef_x_8coef_x_9coef_x_10coef_x_11coef_x_12coef_x_13coef_x_14coef_x_15
alpha_1e-150.8795-3e+023.8e+02-2.3e+02632.1-50.60.17-0.031-0.00510.000850.00024-6.1e-054.5e-06-9.3e-08
alpha_1e-100.9211-2931-152.90.17-0.091-0.0110.0020.000642.4e-05-2e-05-4.2e-062.2e-072.3e-07-2.3e-08
alpha_1e-080.951.3-1.51.7-0.680.0390.0160.00016-0.00036-5.4e-05-2.9e-071.1e-061.9e-072e-083.9e-098.2e-10-4.6e-10
alpha_0.00010.960.560.55-0.13-0.026-0.0028-0.000114.1e-051.5e-053.7e-067.4e-071.3e-071.9e-081.9e-09-1.3e-10-1.5e-10-6.2e-11
alpha_0.00110.820.31-0.087-0.02-0.0028-0.000221.8e-051.2e-053.4e-067.3e-071.3e-071.9e-081.7e-09-1.5e-10-1.4e-10-5.2e-11
alpha_0.011.41.3-0.088-0.052-0.01-0.0014-0.000137.2e-074.1e-061.3e-063e-075.6e-089e-091.1e-094.3e-11-3.1e-11-1.5e-11
alpha_15.60.97-0.14-0.019-0.003-0.00047-7e-05-9.9e-06-1.3e-06-1.4e-07-9.3e-091.3e-097.8e-102.4e-106.2e-111.4e-113.2e-12
alpha_5140.55-0.059-0.0085-0.0014-0.00024-4.1e-05-6.9e-06-1.1e-06-1.9e-07-3.1e-08-5.1e-09-8.2e-10-1.3e-10-2e-11-3e-12-4.2e-13
alpha_10180.4-0.037-0.0055-0.00095-0.00017-3e-05-5.2e-06-9.2e-07-1.6e-07-2.9e-08-5.1e-09-9.1e-10-1.6e-10-2.9e-11-5.1e-12-9.1e-13
alpha_20230.28-0.022-0.0034-0.0006-0.00011-2e-05-3.6e-06-6.6e-07-1.2e-07-2.2e-08-4e-09-7.5e-10-1.4e-10-2.5e-11-4.7e-12-8.7e-13
\n", 1131 | "
\n", 1132 | " \n", 1142 | " \n", 1143 | " \n", 1180 | "\n", 1181 | " \n", 1205 | "
\n", 1206 | "
\n", 1207 | " " 1208 | ], 1209 | "text/plain": [ 1210 | " rss intercept coef_x_1 ... coef_x_13 coef_x_14 coef_x_15\n", 1211 | "alpha_1e-15 0.87 95 -3e+02 ... -6.1e-05 4.5e-06 -9.3e-08\n", 1212 | "alpha_1e-10 0.92 11 -29 ... 2.2e-07 2.3e-07 -2.3e-08\n", 1213 | "alpha_1e-08 0.95 1.3 -1.5 ... 3.9e-09 8.2e-10 -4.6e-10\n", 1214 | "alpha_0.0001 0.96 0.56 0.55 ... -1.3e-10 -1.5e-10 -6.2e-11\n", 1215 | "alpha_0.001 1 0.82 0.31 ... -1.5e-10 -1.4e-10 -5.2e-11\n", 1216 | "alpha_0.01 1.4 1.3 -0.088 ... 4.3e-11 -3.1e-11 -1.5e-11\n", 1217 | "alpha_1 5.6 0.97 -0.14 ... 6.2e-11 1.4e-11 3.2e-12\n", 1218 | "alpha_5 14 0.55 -0.059 ... -2e-11 -3e-12 -4.2e-13\n", 1219 | "alpha_10 18 0.4 -0.037 ... -2.9e-11 -5.1e-12 -9.1e-13\n", 1220 | "alpha_20 23 0.28 -0.022 ... -2.5e-11 -4.7e-12 -8.7e-13\n", 1221 | "\n", 1222 | "[10 rows x 17 columns]" 1223 | ] 1224 | }, 1225 | "metadata": {}, 1226 | "execution_count": 6 1227 | } 1228 | ] 1229 | }, 1230 | { 1231 | "cell_type": "markdown", 1232 | "source": [ 1233 | "This straight away gives us the following inferences:\n", 1234 | "\n", 1235 | "- The RSS increases with increase in alpha, this model complexity reduces\n", 1236 | "- An alpha as small as 1e-15 gives us significant reduction in magnitude of coefficients. How? Compare the coefficients in the first row of this table to the last row of simple linear regression table.\n", 1237 | "- High alpha values can lead to significant underfitting. Note the rapid increase in RSS for values of alpha greater than 1\n", 1238 | "- Though the coefficients are very very small, they are NOT zero.\n", 1239 | "\n", 1240 | "The first 3 are very intuitive. But #4 is also a crucial observation. Let’s reconfirm the same by determining the number of zeros in each row of the coefficients data set:\n", 1241 | "\n" 1242 | ], 1243 | "metadata": { 1244 | "id": "3oz0KcDCPhqd" 1245 | } 1246 | }, 1247 | { 1248 | "cell_type": "code", 1249 | "source": [ 1250 | "coef_matrix_ridge.apply(lambda x: sum(x.values==0),axis=1)\n", 1251 | "\n", 1252 | "# Inference\n", 1253 | "# This confirms that all the 15 coefficients are greater than zero in magnitude \n", 1254 | "# (can be +ve or -ve). " 1255 | ], 1256 | "metadata": { 1257 | "colab": { 1258 | "base_uri": "https://localhost:8080/" 1259 | }, 1260 | "id": "mIGf1hmRPaVE", 1261 | "outputId": "b94a2ec1-9e88-4ffa-e22c-8d181d92f9c1" 1262 | }, 1263 | "execution_count": 7, 1264 | "outputs": [ 1265 | { 1266 | "output_type": "execute_result", 1267 | "data": { 1268 | "text/plain": [ 1269 | "alpha_1e-15 0\n", 1270 | "alpha_1e-10 0\n", 1271 | "alpha_1e-08 0\n", 1272 | "alpha_0.0001 0\n", 1273 | "alpha_0.001 0\n", 1274 | "alpha_0.01 0\n", 1275 | "alpha_1 0\n", 1276 | "alpha_5 0\n", 1277 | "alpha_10 0\n", 1278 | "alpha_20 0\n", 1279 | "dtype: int64" 1280 | ] 1281 | }, 1282 | "metadata": {}, 1283 | "execution_count": 7 1284 | } 1285 | ] 1286 | }, 1287 | { 1288 | "cell_type": "markdown", 1289 | "source": [ 1290 | "**Lasso Regression**\n", 1291 | "\n", 1292 | "- LASSO stands for Least Absolute Shrinkage and Selection Operator. \n", 1293 | "- There are 2 key words here – ‘absolute‘ and ‘selection‘.\n", 1294 | "- Lasso regression performs L1 regularization, i.e. it adds a factor of sum of absolute value of coefficients in the optimization objective. \n", 1295 | "- Thus, lasso regression optimizes the following:\n", 1296 | "\n", 1297 | " Objective = RSS + α * (sum of absolute value of coefficients)\n", 1298 | "\n", 1299 | " Here, α (alpha) works similar to that of ridge and provides a trade-off between balancing RSS and magnitude of coefficients. \n", 1300 | " Like that of ridge, α can take various values.\n", 1301 | " Lets iterate it here briefly:\n", 1302 | "\n", 1303 | " α = 0: Same coefficients as simple linear regression\n", 1304 | "\n", 1305 | " α = ∞: All coefficients zero (same logic as before)\n", 1306 | " \n", 1307 | " 0 < α < ∞: coefficients between 0 and that of simple linear regression\n", 1308 | "\n", 1309 | "Notice the additional parameters defined in Lasso function – ‘max_iter‘. \n", 1310 | "\n", 1311 | "This is the maximum number of iterations for which we want the model to run if it doesn’t converge before. \n", 1312 | "\n", 1313 | "This exists for Ridge as as well but setting this to a higher than default value was required in this case. " 1314 | ], 1315 | "metadata": { 1316 | "id": "Uqx_IIyaPh6X" 1317 | } 1318 | }, 1319 | { 1320 | "cell_type": "code", 1321 | "source": [ 1322 | "from sklearn.linear_model import Lasso\n", 1323 | "def lasso_regression(data, predictors, alpha, models_to_plot={}):\n", 1324 | " #Fit the model\n", 1325 | " lassoreg = Lasso(alpha=alpha,normalize=True, max_iter=1e5)\n", 1326 | " lassoreg.fit(data[predictors],data['y'])\n", 1327 | " y_pred = lassoreg.predict(data[predictors])\n", 1328 | " \n", 1329 | " #Check if a plot is to be made for the entered alpha\n", 1330 | " if alpha in models_to_plot:\n", 1331 | " plt.subplot(models_to_plot[alpha])\n", 1332 | " plt.tight_layout()\n", 1333 | " plt.plot(data['x'],y_pred)\n", 1334 | " plt.plot(data['x'],data['y'],'.')\n", 1335 | " plt.title('Plot for alpha: %.3g'%alpha)\n", 1336 | " \n", 1337 | " #Return the result in pre-defined format\n", 1338 | " rss = sum((y_pred-data['y'])**2)\n", 1339 | " ret = [rss]\n", 1340 | " ret.extend([lassoreg.intercept_])\n", 1341 | " ret.extend(lassoreg.coef_)\n", 1342 | " return ret\n", 1343 | "\n", 1344 | "#Initialize predictors to all 15 powers of x\n", 1345 | "predictors=['x']\n", 1346 | "predictors.extend(['x_%d'%i for i in range(2,16)])\n", 1347 | "\n", 1348 | "#Define the alpha values to test\n", 1349 | "alpha_lasso = [1e-15, 1e-10, 1e-8, 1e-5,1e-4, 1e-3,1e-2, 1, 5, 10]\n", 1350 | "\n", 1351 | "#Initialize the dataframe to store coefficients\n", 1352 | "col = ['rss','intercept'] + ['coef_x_%d'%i for i in range(1,16)]\n", 1353 | "ind = ['alpha_%.2g'%alpha_lasso[i] for i in range(0,10)]\n", 1354 | "coef_matrix_lasso = pd.DataFrame(index=ind, columns=col)\n", 1355 | "\n", 1356 | "#Define the models to plot\n", 1357 | "models_to_plot = {1e-10:231, 1e-5:232,1e-4:233, 1e-3:234, 1e-2:235, 1:236}\n", 1358 | "\n", 1359 | "#Iterate over the 10 alpha values:\n", 1360 | "for i in range(10):\n", 1361 | " coef_matrix_lasso.iloc[i,] = lasso_regression(data, predictors, alpha_lasso[i], models_to_plot)" 1362 | ], 1363 | "metadata": { 1364 | "colab": { 1365 | "base_uri": "https://localhost:8080/", 1366 | "height": 297 1367 | }, 1368 | "id": "kj2u_6LIPiGN", 1369 | "outputId": "cf9386f3-2b51-4148-f07a-a6dec8cf0df0" 1370 | }, 1371 | "execution_count": 8, 1372 | "outputs": [ 1373 | { 1374 | "output_type": "display_data", 1375 | "data": { 1376 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEYCAYAAAD8hukFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeXxU5fWHn3NnkoDs+zoQQGSrghCCCyjaRcS12iqLCq5Va1sttXW3Vq22v1KXulTrAiig4oosKopIcGNHZBOQYABBRAERSDJz398f9w6ZDJNJJplktvN8HMPMvfe9Z3K/Oeddzvu+YoxBURRFUZIBK9EGKIqiKEoQDUqKoihK0qBBSVEURUkaNCgpiqIoSYMGJUVRFCVp0KCkKIqiJA01CkoiMk9EroiHIeLwrIh8LyIL41FmJfebICL3xPtcJTqqGaUyVCOZTaVBSUQKReSAiOwTkR3uL7JhLDcRkVwRMSLijXLaYODnQEdjTH4s5acyInKKiLwvIntEpLCGZWWLyMvuMzMiMjTsuIjIP0Rkl/v6h4hITe5ZgR2qmVoknppxy/upiKwVkf1uuZ1Djk0QkRL3WQZfnjjcUzVSi8T6ty4io0Rks4j8KCKvi0jzkGPNReQ199hmERkVw7XXichiESkWkQlVsb2qLaWzjDENgf5AHnBbFa+Lhc5AoTHmx1gvrESUyc6PwDPAjXEqbwFwEbA9wrGrgHOBvsAxwFnAb+J033BUM7VH3DQjIi2BV4HbgebAYuDFsNP+aYxpGPIK1PS+LqqR2qPKf+si0gd4ArgYaAPsBx4LOeVRoMQ9Nhp43L2mKtduA+7B0WuViKn7zhizFZgN/CT8mIhYInKbGzG/EZFJItLEPTzf/bnbrRkdH3bt5cBTwPHu8bvcz68UkQ0i8p2ITBeR9iHXGBH5rYisB9ZHsldEponIdrdGOT/4i4xw3lAR2SIit4jIt24tbnTYac1EZKaI/CAin4pIt5DrHxKRIhHZKyJLRGRItN9jKMaYhcaY54AvK7Ctp4jMcX8H60TkgihllRhjHjTGLAAiOY4xwHhjzBb3WY4HxlbV1uqgmkluzQDnAauMMdOMMQeBvwJ9RaRnVe2pKaqR+GuE2P7WRwNvGmPmG2P24VRQzhORRiLSADgfuN0Ys8/1LdNxglDUawGMMa8aY14HdlXV8JiCkoj4gOHAsgiHx7qvU4CuQEPgEffYSe7Ppm5N6+PQC40xTwNXAx+7x+8UkVOB+4ALgHbAZuCFsHueCwwCeldg8mygO9AaWApMjvL12gItgQ44D/RJEekRcnwEcBfQDNgA3BtybBHQD6emOQWYJiL1AERksIjsjnLfCnEFMccts7Vrw2MiUtH3rYw+wIqQ9yvcz2oN1UzSa6acJtwWxUbK6+Ja14EvEZHzq2NXJTarRuKvkVj+1sM1sBGnZXSU+/IbY76ooKxo11YPY0zUF1AI7AN24zzAx4D67rF5wBXuv98Drg25rgdQCniBXMAA3ij3GQssCHn/NE63QfB9Q7e8XPe9AU6tzP6Q65u61zRx308A7nH/PRTwAw1Czn8Jp3YQPPepkGPDgbVR7vU90LeqtrnX/AynmyH0swuBgrDPngDurEJ5W4ChYZ8FgJ4h77u7vxOJxVbVTPpoxv2d3R/22YfAWPff/YEW7jMZDvwAnKgaSW6NEMPfuvs7vjrss62u/UOA7WHHrgTmVXZt2Gf3ABOqYntVW0rnGmOaGmM6G2OuNcYciHBOexxxBdmMI5w2VbxH1PKM0zTchVPjCFJU0cUi4hGR+0Vko4jsxfkjAKfWEonvTfl+582uDUFCx2j244g5eK8/icgatzm/G2gS5T6x0BkYJCK7gy+c5nJbEekkIYPPVSxvH9A45H1jYJ9xVRNnVDOpoZlwTeC+/wHAGLPUGLPLGOM3xszCaRWcFwc7QTUCtaeRWP7Wo2kgqj6qcDxm4jlPaRvOH0SQTji1hB04EbpG5bndEi1wonCQaOWOAs7BqU02walVAVSUgdLMvUeQTq4NUXH7ef+M0x3QzBjTFNgT5T6xUAR84P7hBl8NjTHXGGO+MiGDz1UsbxXOwGeQvu5niUI1k3jNlNOE+326UbEuTJzsrCqqkeppJJa/9XANdAVygC/cl1dEuldQVrRrq0U8g9JU4AYR6SJOauffgReNMX5gJ2Dj9AnHUt6lItJPRHLc8j41xhRW8fpGQDFOLegI9/rKuEuctOohwJnAtCreJ/gdvSJyB4fXHCpEnIHcekCW81bqiUi2e3gGcJSIXCwiWe5roIj0ilJeTrDfGch2ywsKeRLwRxHpIM7g7jicLoREoZpJvGZeA34iIue7Zd4BfGaMWeve61ci0tC95y9wMjunV9XWOKAaqYZGiO1vfTJwlogMcQPo34BXjTE/uK28V4G/iUgDETkRJyg/V9m14GQwurryAB5Xq1GzGuMZlJ5xDZ0PbAIOAr8DMMbsxxnA+9DtUjiussKMMe/iZHK8AnyNU3sbEYM9k3CayluB1cAnlZy/HafPdhvOL/rq4B9mJbwNvIVTM9iM870PNf/dhxWte+0k4AAwC6cWdQB4B8B9sL/A+d7bXBv/gVMTqYh1bhkdXNsOUFYzfAJ4E1gJfA7MdD9LFKqZBGvGGLMTJ7vqXve7DKL87+wPOL+P3cD/AVcaY+ZV4TvGC9VI9TQS9W/d7cIdAmCMWYWTEDIZ+AYnIF4bUta1QH332FTgGveaqlx7G44+b8Kp0BygktR/qZ3hhNRCnEmmzxtjOibaFiU1UM0olaEaqR669p2iKIqSNGhQUhRFUZIG7b5TFEVRkgZtKSmKoihJQ0IWHGzZsqXJzc1NxK1TniVLlnxrjGmVaDvqEtVLzVDNKLGQaL0kJCjl5uayePHiRNw65RGRzZWflV6oXmqGakaJhUTrRbvvFEVRlKQhNYNS0UIoGO/8VJTKUL0osaKaSRipt4lV0UKYeDYESsCTDWOmgy9jNpRUYkX1osSKaiahpF5LqbDAEYsJgL8Y5t2ntRmlYkL1EiiBFVO0BqxER31MQkl8UApvJlfWbM4d4tResAAbvpzn1GpUNJlBJH1E00xQL+IBywPLpsDce1UzmUKsegH1MQkmLt13IvIMzuq33xhjDtvSuELCm8nD7oe3bir//sAuRyShzed+I+Drz2DbMjC2c35hgTaxU4S46WWMu1h18DPLA8deBH1HHq6X4Ir/SyaWtZpUMylBrelFfUxSEq8xpQk4WxRPiumqcs3kg7BsUvlm86xxYNtgWTB8PLTpXd4BWVlg+x1x5cayfb2SYCYQD72smAJNfCHdcwFY/AwsnXS4XoIOyJNd9l41kypMoJp6MYESxASw/cXsnfU3aJZLE/cz9THJSVyCkjFmvojkxnxh7hDnwQcCgHFqJpYXbDAiYAcQDMa2kZk3QI/hZQ7IBgZc4jil8FqOktTETS/LpsDp/3Qchv8gBuO0h2w/zPwjtO8HgeKymu6BXU5tubBANZNC1EQvAfGCbePBptG2Bfi3fUwJgkcsQLDsABbGCUzqY5KCxGbf+fKd7pbFzwIG2w6wOedIlpT4WHzQx91ZE/ASQASMsbHXzkTEcl6ebOg7qmKhFC1U55NuhOsl4Oejgvf4Xk6mnv0tp8hSPNiuXgKwdYl7oauXoBYi6UH1kn748tl7wSt45v+Dxts+dLSBzTeNerHB040Pf2zPH0v+h4T4GNbORCzXLUbzMaqXWqPOgpKIXAVcBdCpU6dDnweOHoFZOgXsYjzGplPxWjrIl2Qd/wSf+330XfE3wEYAC4OxA/jFw/JeN9K/w0A84AhkxRRAnPEE0JTOFCeSXowxLG82jN4yGY9dSgBh4J7ZeAlge7JY2m40A76eihinhS2AMWCLYVmvv3Cs6iWtiaSZ5j2HQIMc5/n6i7GwabtvDW09XzJ4zHR2buxKi3k3EfQxBoNt2zDgEqx+bkAKBqD6LZzWdv0W5ce+VS9xpc6CkjHmSeBJgLy8PAOw4ZsfuH56CdkHbuL2htPpV7ocDzYe/JzT9EsYMg58zdx+X6fLxhKwMby/dB13f/0hjw7x45v+a0cgAMsmw7Gjy6cB6wBlyhFJLwA3LaxPe+sOxnbcwjGNfqDpmqmIcTST36sLnD7bCThLJmGMUwMGmLt0DbcVFfDgiSX0fGuk6iUNqUgz+PKdwDHvPieTLiRxodXQcdAwBzPzBoxxAlPA2Ly4zjBoUC+ODCZL+IsBG8RyXsbWBIhaIqEp4Y3qZXGgJMBlIy+k38X3I94cJ3XXk+3URgrcgcdLZ0PeWMTjHPd4czjup+ew9fsDvPTyVOxAaVmhgRLAlKUB6wBl2iAiPHHxAP578zWcfPn9NDt+jNMtF0z33rPFOfHMB+GMfzvdMGJheesx+Ofn8sNBP9PfeEn1kon48mHozeD6kEN6KVro+JYzHkAsL0Ys8OTwzo9HMvzhBSz5YDomUIIzwIQTiOxgcFK91AZx2U9JRKYCQ4GWwA7gTmPM0xWdn5eXZ4KLJdq2wbLc6mxoMzlS8zisH3fnD8U8Mmkyt3xzI9nid2zx5MDYGU55adjnKyJLjDF5ibajJtREL4cR7IpbNqUsS6oCvewr9jPhxRe5cuPvM0YvkPqaiVUvEEUzVdTLzqZ9+csrn7F73QJeqHcfWaYUCbaUPDkVp5KnAYnWS7yy70ZW99pDAQnKBqELxkfuTgkbpG7VKIfbrx7Lsy8eQf3VL9G1VQMGnXsdnuA5OkCZlNREL4fhy3eep+2vVC8Nc7xcd8lo3nunMdvnP0uTI7I58fzraKZ6SWoSoZdWwFOX5PHf+c0Y8Q6c23QTvzzxaBrZe6NrQfVSY5Jz7bvgjOpo80nch+/NHcIVIy/kgXf7c9t767nxg41cm7sA6RJBFLqmVXpSFb3AIc38tNcQCro8wVWTltB+hp+XWUCzbz493JGoXtKTKurF2rqIaz0FDBjWmzFzejLhk/pMvmIQ7faudCrOqpdaITmDUnBgsqIaR9jDlzHT+ePP82mzZwXnrfwD5ku/Mz4VLorwddB0gDI9qEwvcJhmhoyZzoRLB/Lgs5OpP/VujASc8alQzahe0pMY9TLIk82rZz3PhTOLufvxCTzq/ysSKD088Khe4kLi176rCF++m30X4aFGevjAqNZfkS1+LGxsf9nnhwhdB00HKNOLaHqBiJoZ1LUF9/bfjdf4ERNwBrRDNaN6SV9i1Evv4hVMuCyf7vuXO74lzPcAqpc4kZwtpcqooPktXYZgeXMI+IspMR6+qt+XHqHXVaWGpKQnFWima94w/CsfxR8owYiXrFBHonrJXCLoZYCvGVnDz6dk9qtO5deTjahe4k7yBKVYBggrevi+fGTMdErWz+OGTxvy2RxhZq8SmjXILn+tiiX1iXVAOYpmvJe+yXuzX+HRTW0Z+11HzvaFXad6SQ/i4GOOOe4XLNg/gY/ee4Oc3JP4fceBSPh1qpcakRxBqToDhBU9fF8+9X35/PaoPZz/+Efc+PJn/O+SAYjI4ecqqUl1B5SjaOaky/N4/MlPuOmVzzimQxNyWzaIv91K4oijjxl86hkssY/igXe/oMlHhYw9sUstGZ2ZJMeYUgVjRDETsk/K0R2b8OdhPXh3zQ5eWFQUX3uVxFILesnyWDw88li8lvCHF5dTGrDja7OSWOKsmd91/46f9WrDPTPXsGTz9/G1NcNJjqAUjwHCYE0oZAO3y07swgndWnDPjNVs3X0g/nYriaGW9NK+aX3+ft7RrCjazZPzv4y/3UriiLNmrOfO4aHBJbRrWo/fT13GngOllV+vVInkCErB/ttTb61+bn+EmpBlCf84/xgMcPOrKym3ekVlu08qyUst6QXgzGPac8bR7Xjo3fWs3/FD2fmql9SmFjTTYNvH/Gdkf3bsPchtr39++PmqmWqRHGNKUPMBwgqyq3zNj+Cm03vy+vTXWDNtJr2PPwN2rHYWeTW2s2SITnJLPWpJLwB3ndOHDzd+y4QXX+LufruxivfCx4+oXlKdWtBMP19T/vDT7oyf8wUXttnG4Ky1znnqY6pN8gSlmhIlHXN0+x1cmPN3PKv9mHWPIXbAqe2AswmcTnLLPKLopWXDHMYfV8wJH94Ec0uBkBa26iVzqUAz1wztRtHKD8ibf7MzCdvyOLsaqI+pFukTlODwmpCbAurZswVLAoixsQOlSKiTEUsnuWUqFeiF3CGcWu8LbPE7e3hBWdqv6iWziaAZb2EBt7TfhPd7v7Noa8AG9THVJr2CUiihKaCWB7G8BAIGv7HI8lpYAT9YFgwfrzUY5fClq4bdj3iy8fuL8YgbliyP6kUpI0QzTS0PfsuD3wbxePGIQKBUfUw1SN+gFDooaQMDLuFg/fb8pqA+vVo15tY+3+qsa6WM8MSHA7uwxr7JOzOmUbDV5k+DW9Gsz6mqF6WMcB9z7CU8+7mfZVYfHriwHzlbPlIfUw3SNyiFD0r2HUUDXz6DvRu5f/ZaTvnZGZzga5loK5VkIVLigy+f/qOP4Y//msf3u1rxmG9Aoq1UkokwzXiPHUXv3l2596lP6b2pJdedOi7RFqYkyZESXhtUkAI69oRcOjStz32z15aliGvqplKBXlo3rsdVJ3Vl1srtLC/a7ZyrelEgomZOPLIlp/+kLY++v5FtwbmRqpeYSN+WEkRMAa2X5eH6n3Xnxpc/Y/bn2xnetEj3QFEcKkgZvmJIVyZ9vJl/vb2O508T1YtSRgTN3DK8F++t/YZ/vb2Of59QonqJkfRtKUXhvP4dObJ1Q8a/sw57U5yWH1HSloY5Xq4d2o0FG75l85K3VS9KVHzNj+DywV14ddlWtq+Yo3qJkfQPShGazh5L+OPPj2Ljzh8pKO2pe6AoZVTQ1XLRcZ1p3SiHJ4vaY1QvSigRNHPt0G60aJDNY4XtVC8xkt7dd8GUTX9xWWpm3lgAhvVpS8+2jbhrmc07F7+O96sPNVMm0wmbRsCxF0HfkeDLp16Wh2uHduOvbxZz4TmTOMa/UvWiVOhjGtXL4nenHslf3yzhl2dN5Fj7c9VLFUnvllJhgSMWbLD9zrIfbm3GsoS/HruPYd9P4aONu6LvQqlkBuFp4YufPbRYK8CI/E78tEEhqz+apQ5GcYjiY0YO6sRpjTezYsEMTOfBqpcqkt5BKXeIU3sJYuyyPt2ihQwquIxxWdPInz8We/OnibFRSR6CKb6H1m8w5cYB6m1fwhPmLn61dyL2hLM0m0qJ6mNyvl7Co/67uGj/89ghlRslOukdlHz5TnPa8jpLfXhyyvp0CwuQQAkebLzGz4ZFbyXWViXxBFN888Y6WgkfBygswGP8eMXG6KC1ApX6GI8pPaQXe5PqpSqk95gSOA6mTe/DF950a8UmUIIfD09t6cA/jNEdajOdYIpv31ERNSOebGx/CSXGwzeNBtA5sdYqyUAUHxPUS6nxsCTQk8EJNTQ1SP+gBJHnn7i1YiksYN6+7rz0gYdzv9zFCd10lQeFqJop/mIel82rR7t1Tfl3v8SYpyQZUfTCpgL+8klD1i5vyFsnGyxLK77RyIygVBGukIaWBmi55H3emzODE7bt0kFspWJ8+dT35dNz/yo+/2QOe5u/TeOep6helMj48rF8+fysyTamT13GJ/Nnc4JnjfqYKGR2UHKpl+Xhpp/s4YxlN2J2BBCdea1UwtVdd9F48b3kfOiHTx9QvShROePodrzz9nT6z7vd2XNJfUyFpHeiQwycaT4ghxJEZ14rVaDNptfIkVIsNOlBqRyPJVzfagnZRn1MZWhQAihaSL3PpyICxoCxPDrzWqmYooWw7HkEgzFgo3pRKqFoId22vO74GHCy9VQzEdGgBE6NxQ4ggI2wouWZ2qxWKiZULyK8aZ2C3WFgoq1SkpnCAiSoGSN80+189TEVEJegJCLDRGSdiGwQkZviUWadEpw0KR4CVjYP7hzAwdJAoq1Ka1JaMyF6wZPDpP3H897abxJtVVqT0nqBsiko4qFUsvjf3uMSbVHSUuOgJCIe4FHgdKA3MFJEete03DolZF+Utb94nnn7uzB9xbZEW5W2pLxmQvRiLnmDrxsfw8SPChNtVdqS8nqBsikop97KG8f8l/8VtmT9jh8SbVVSEo+WUj6wwRjzpTGmBHgBOCcO5dYtvnwYMo6jj/s5Pdo0YsKHhWWbACrxJvU14+rF2/k4LjquMws2fKtOpvZIfb3AIc384rSzqJ/l4cn5XybaoqQkHkGpA1AU8n6L+1k5ROQqEVksIot37twZh9vWDrJlEf9s8y71ti9m4abvEm1OulKpZlJFLwAXd9jB77Km896cGYk2JV1JKx/T7Lvl/KfjXApXvM+OvQcTbU7SUWfzlIwxTwJPAuTl5SVnE8Rdhv6YQAmTczw8Orclg7pe7HwevoSIUqukhF4AihbS+KXzud5TTMn6V9m3oTMNc7yqlwSQEppxfcxPAyWc6PHwytsduOjXv1YfE0I8gtJWwBfyvqP7Werhbl0gJkA2YAoL+HZNLi1f+bVuZxxf0k4zHmyy8FP4/tP02jFT9RJf0k4vYgJki2HXqvc40L8j9af+UjXjEo/uu0VAdxHpIiLZwAhgehzKrXtCsqrEm8Uu05Af3robAsW6nXF8SUvN2GJhvl6BUb3Em7TUi1heWvi/YfPcp3TL9BBqHJSMMX7gOuBtYA3wkjFmVU3LTQjBrKoBl2Ad+TPuzpqEb89CjLHdZel1O+N4kJaa6TEMrxh6BNY7e+qoXuJGuurFMgFGeufSZevrGMurW6a7xGVMyRgzC5gVj7KSguUvgP8gXow7odZCug6FoTdndLM6nqSdZr54GzEBLIEAgqfrUNVLHElHvWACeABjAhR1/hWduhylY0roig6HE9wS21kMBBuhFK86GKViCgvAthEc1djGYkf/G1QvSmRcvYCjFyMWT+x20sVVMxqUDie0z9eTzboO5zOy+BY+t3ok2jIlWckdAt4cwMJYXu7wj2VCUetEW6UkKyF6EcvLJz1vZvK2tny2ZXeiLUsKNCiFEzJbn7EzaT/6v6z29uS5jzcn2jIlWQlq5qe3YV06m297jOLFRUW6VJUSmRC9cOls+p17PQ1zvDxVsCnRliUFGpQi4StrSjfZtYwH289l84r32bO/NNGWKclKUDPALY1mkbv/c2at/DrBRilJS4iPabRzGQ93nMvXn3/A13sOJNqyhKOb/EXDneh2WqCEUyxh4+TPaDLsN9rvq0TG1UtuoIQpOR6eeX8X/NhMB6+VinE1c0qghBO9wqrJK2h31jUZrRcNStEoN9ENem59GTNxJjJmetlxdThKkHKTrw1X7n0MMxdnl9Fh98OBXaoXpTxhPqbfN69iJs7OaB+jQSkawaQH/0HAYAG2vxh56ybY/jnYfp2BrZQR1EugBBHBCgQQY8B/AGb+0TlH9aKEEtHHHEQ+fBA2zM3IVR50TCkawQHJvLHgycGPINiwdamu8qAcTkiSjAwfjy1eZydjcLSielHCCfExYmVhwNnReN1bGetjtKVUGb588OUjfUdR9PJtdNq9EI8E13oUnYGtlMfVC8C+LxfTZNVzITU/1YsSgUOaEVj8LILBNgaxLDJRM9pSqiq+fBoPu50SsgjgLiGTd2lGNauV2Gh2/CWUSg5+LIzqRamMviPBWw8/FqVkwfDxztSUDNOMtpRioEWvIYzv9G+8RR/ym9FjqNf1+ESbpCQzvnyWnTqR+W+/xkmnnMtxJw9PtEVKMuPLR8ZMZ/kH0/n7qhbc1voc+ndqlmir6hxtKcXIyT89gwcOnsXL37RPtClKCjBw8DDeaDSCf6/NPOeiVANfPr1+/Vc21uvDkx9k5s60GpRiZEDnZvyq9TYOzv0/7M2fJtocJcnxWMKNfXaT99WzbFw6N9HmKClAgxwvf+6zh67rnmDrynmJNqfO0e67GJEti7h/320QKMFMmgan/0PnnygVU7SQc1ZcQ8Bbgv3m69DK3TI9A+efKFWkaCGj1l5HwFOC/errUPzPjPIxGpRipbAAjylFxCYQKIZZ48CYjJtLoFSRwgIkUIpXbPx2KT8ufI4Ga6Zl5PwTpYqU00wJZtY4Z75bhuhFu+9iJXcI4snGxoNtLIxtZ+RcAqWKuJMjjXgoxcva7T/oLqNKdEI0Y7AwdmbNcdOgFCvuZLeSk27mbnMZpZIV3x0jixZCwXjnp5L6uHqRU2/l8c4P8O9vBmA8cdSM6iX9CNHMq+2up9hkYTLIx2j3XXXw5VPPl4/3x9WM/qQjT518kCa9Tq15s9pdnFG7dtIMd3Lkad328PDDC3jl2Mf5VYtNNR8jUL2kL65m8nru46IHGvG7rjsYetp5GeFjtKVUA648qQvLOYp/7T8zPg82uOttBjXVM4k+7Ztwas/W3PtZQ37M/0PNNaN6SXu6tWpIx2OGcs3moexs2rfmBaaAZjQo1YB2e1fyn47vs27xe3yz92DNCwzZ9TbTlhbJFP7cZw8ji19mztvTa16Y6iX9KVrInU3foo+9lsfmbah5eSmgGe2+qy4hey2d7PEweUYbrhg1omZlBhdn1HTh9KRoIT3fvojuWcWULn2NH/u0pUG3E6pfnuolvXF9TPNACVOyvVz0qbBlcBc6Njui+mWmgGY0KFWXcvugGI5YM42972ymca9TavagQxb0VNIMVzMebIwpZfubf6XB+feoXpTIhHS1ZWH4nfUy015vwQ2XXlSzcpNcM9p9V11CmsHi8XK+NY8GH93vDCImaVaLkmCCmsHCIwbf9wsxE89SvSiRCdGLYHOitZKrC29g3eJ3E21ZraJBqbqE7J1jHXsRWWI7NeAkHTxUkoCgZroNJRiYbL/qRamAUL2IhYUhS/x8Mnc6AdtUenmqokGpJvjyYcg46DsS8Wbjx6LEeDGdB9es3CSfR6DUAF8+DL0Z8eYQwEOJ8bImJw5ZVaqZ9MTVC54ct1cmmze+78LkTzfXrNwk1ouOKcUDXz4y5k1WFcxg6uf7GP3RTI4Wibnf1hjDyk/m0HvOxc5SRkk6j0CpIW4NOLDhAx768Ftavfc63Vo3IDu3mluhpMDcE6UGhCQnWJ0H03fGanbNvo+vG42i3U9Ojr28JNeLBqV44cvn6O6r6In8F4kAACAASURBVPPFOGStjb3hCawxbzoPu2ghrJgCiLORVwQBbF05j3dnv4pn7xZ6e0oQsbEDJViFBUklGCVO+PLJ3rGaPwfux5QGsCe+BJfNKNNLYQHUb1HxQpxFC9n1+XtM+NqHVVjA76xivG73sahm0o/gDtiLJ3DHt3/ClgD+l1/D32g63s7HVe5jgprKHQKFBRh/MYJNwF/MR3Ne48jzj6Zdk/oJ+WrhaFCKF0ULsWb/CSGAAAF/MYFlU/CsmAJLnwe71Dlv2WQYO6NMNEUL+fqDZ2m5fhqjJQBZXizLiz/gp9T2sKfZQNom7EsptUbRQpg1Dsv4QcDYJWye+xSdW0yBZVMgUArYIJbTdRNSmzWLn8XM/BNN7QDXksVs3x8IbMsCuxRjeclKwrknShxwNSO2H4+AMcUUvnw7R548Gmbf6LR84HAfs3iCu3C0jfFkM6XZtZxnvGSJHz9eHtrYhn+V2gn7WuFoUIoXhQVg2whgAGMEs+x5LONHCBmUDCZCuDVi/7Nn0TpQjCUGcU6AYy9hX047Lp+XQ7/NLbn1Jwn5Rkpt4uoFHL0gQtsvX8FsCpTXi7HLaebglx/jnTEOjwlgCdQTP+f1rI857U1mTJ/Gc9t9PNT4aNol5EsptUqIZgA8Al32LiIwcwkeExJUwnwMs8aB7QfA9hfz9fatzOj/X37ZfBP1up7ExNb9OSLbU9ffpkI00SFe5A4Bbw5gIeLh+/qdsWwnIJXLk3FnUfs3f8Kml29DAsV4JOQMEWjbj8Y9T2F0myICi56ldN6/knJAUqkBoXqxvNgd8smSCHoRy9FM/RZsn3kvc6Y+CMZGxAlmAlC/BdJpEPmnnM25Mp/CCb9RvaQjIZpxnzweMRg7gA1lugmu1FC0EObd56wyjrPDjkEY08fLr/N8eLueBCum0GDOjciWRQn4QpERY6qfWigivwb+CvQC8o0xi6tyXV5enlm8uEqnphbBft1lUzDB7hfXcxgspPNx2C17sHV/Fu3XPI24tV3EQsRyVIMBywuIk15uDEYsLK/ThSOdBi0xxuQl9HvWgOpoJq31Ehw7mn2j87wBDNhisaXXFTSzDhDYsZbG3y4BY/CLhyzLwrJLOSQubz0Ydn/5Mjw5iNuFIyIpqxn1MWEEfYw7JBD03rYBIxal7QZSv0krAMz6OW6XniFgALHwWB7E2GB5HH8THFbw5Bzq8ku0XmraUvocOA+YHwdbUh9fPjTxge1HsBEE9z9sY/AXfopZPAHf6ifxEMAjIAjSdSj0v8Qpw9jOeEKgxOnGEbCwk3bxxGqgmgkSnFJwYBfYAbfu6zSWjYGClRvJ+XwqTXcuwjI2HjFki43VfzR0O8VpRWEcbax5AwKlCI7eRPWSngR9jNtdJ+7/PQIYQ9a2RZi1MzBrZ0CgGMEQQNjZ6gQ8A8Y4AckEHB8TDEiQVP6lRmNKxpg1ACJS2amZQ3AWdqAELA+CYGw/IoJlAlhu98yh35jlceYhACyfeug6ENfJ2GVdOGkwgK2aiUBQM25GFGLh8eZwavfWZG0oC1aA06LuO8p5s/njsrTeXudA4YKywW7VS/oSwcdg+/FYrq9xT3P6XQSPtx7tzrnL+XDFC2U+plxLKXn0UmeJDiJyFXAVQKdOnerqtnVP+IKHgBQW4KnfAt66qczxII4who8vy5IJu67StOA0JmP0AuU14z5vyR1Ce4BNr5QFGomiF18+tOld6dSDdCZjNBPBx1BYgNRvgYRk4YmVBf0vLq+F8OuSUC+VjimJyLsQMSv5VmPMG+4584A/ZXx/b2WEzz+pRsBJdH9vVYi3ZjJWL1B+/knbvtWqoCS7ZtTHxJHw+UoQ84rgidZLpS0lY8zP6sKQjCB0dd4kn1VdE1QzcSSoGdWLUhXSwMdoSniiSIEdIJUkQvWixEqKaqZGQUlEfikiW4DjgZki8nZ8zMoAUmAHyNpANVNNVC+ql1hJUc3UNPvuNeC1ONmSWaTADpC1gWqmmqhelFhJUc3oMkOJJMl3gFSSDNWLEispqJkarehQ7ZuK7ARquCFIzLQEvq3je9aEiuztbIxpVdfGJBLVS5WIZq9qpm5IF80kVC8JCUqJQEQWJ3NabDipZm+6kWq//1SzNx1JtWeQrPZq9p2iKIqSNGhQUhRFUZKGTApKTybagBhJNXvTjVT7/aeavelIqj2DpLQ3Y8aUFEVRlOQnk1pKiqIoSpKjQUlRFEVJGtIqKImIT0TeF5HVIrJKRP4Q4ZyhIrJHRJa7rzsSYWuIPYUistK15bBljcXhYRHZICKfiUj/RNiZjqhelFhJNc2kol7SbUUHPzDOGLNURBoBS0RkjjFmddh5BcaYMxNgX0WcYoypaNLd6UB39zUIeNz9qdQc1YsSK6momZTSS1q1lIwxXxtjlrr//gFYA3RIrFU15hxgknH4BGgqIu0SbVQ6oHpRYiUNNZN0ekmroBSKiOQCxwKfRjh8vIisEJHZItKnTg07HAO8IyJL3J0zw+kAFIW830Jq/xEkJaoXJVZSRDMpp5d0674DQEQaAq8A1xtj9oYdXoqzttM+ERkOvI7TdE0Ug40xW0WkNTBHRNYaY+Yn0J6MQ/WixEoKaSbl9JJ2LSURycIRy2RjzKvhx40xe40x+9x/zwKyRKRlHZsZas9W9+c3OEv0hy/puxXwhbzv6H6mxAHVixIrqaSZVNRLWgUlERHgaWCNMebfFZzT1j0PEcnH+R3sqjsry9nSwB0sRUQaAL8APg87bTpwiZslcxywxxjzdR2bmpaoXpRYSSXNpKpe0q377kTgYmCliCx3P7sF6ARgjPkv8CvgGhHxAweAESZxy1q0AV5z9esFphhj3hKRq0PsnQUMBzYA+4FLE2RrOqJ6UWIllTSTknrRZYYURVGUpCGtuu8URVGU1EaDkqIoipI0aFBSFEVRkgYNSoqiKErSoEFJURRFSRo0KCmKoihJgwYlRVEUJWnQoKQoiqIkDRqUFEVRlKRBg5KiKIqSNGhQUhRFUZIGDUqKoihK0lDjoCQi80TkingY4y6f/qyIfC8iC+NRZiX3myAi98T7XKXqqH6UeKOaSm2qFJREpFBEDojIPhHZ4f4yGsZyIxHJFREjItG2yxgM/BzoaIwJ34wqbXGF/w8R2eW+/hHcj6WC80eJyGYR+VFEXheR5iHHmovIa+6xzSIyKuRYOxGZLiLb3GeRW7vf7NB9VT+1SJz1c52ILBaRYhGZUCdfoBqopmoXETlFRN4XkT0iUliX946lpXSWMaYh0B/IA26rBXs6A4XGmB9jvbASYSU7VwHnAn2BY4CzgN9EOlFE+gBP4Ozp0gZnD5THQk55FChxj40GHnevAbCBt4Dz4/8VKkX1U3vEUz/bgHuAZ2rR3nihmqo9fsTRwI11feOYu+/c7XVnAz8JPyYilojc5tbCvhGRSSLSxD0c3Bd+t1u7OT7s2suBp4Dj3eN3uZ9fKSIbROQ7t5bfPuQaIyK/FZH1wPpI9orINBHZ7kb8+SEOOvy8oSKyRURuEZFv3ZrY6LDTmonITBH5QUQ+FZFuIdc/JCJFIrJXRJaIyJBov8cwxgDjjTFb3N/veGBsBeeOBt40xsx3t1y+HThPRBqJs7vk+cDtxph9xpgFODtLXgxgjNlhjHkMWBSDbXFF9ZO8+gEwxrxqjHmdBO2uWx1UU/HXlDFmoTHmOeDLql4TL2IOSiLiw9mpcFmEw2Pd1ylAV6Ah8Ih77CT3Z1NjTENjzMehFxpjngauBj52j98pIqcC9wEXAO2AzcALYfc8FxgE9K7A5NlAd6A1sBSYHOXrtQVaAh1w/tCfFJEeIcdHAHcBzXB2arw35NgioB/QHJgCTBORegAiMlhEdke5bx9gRcj7Fe5nlZ5rjNmI0zI6yn35jTFfVLGsOkf1k9T6SUlUU7WiqcRhjKn0BRQC+4DdOA/hMaC+e2wecIX77/eAa0Ou6wGU4mzFmwsYwBvlPmOBBSHvnwb+GfK+oVtervveAKdW5Tu45zd1r2nivp8A3OP+eyjgBxqEnP8STqsjeO5TIceGA2uj3Ot7oG8V7QoAPUPed3ftlAjnvgdcHfbZVtf+IcD2sGNXAvPCPvO65edW9XdXk5fqJzX0E/bZPcCEutCHair5NBVyzc9wui/r7NnG0lI61xjT1BjT2RhzrTHmQIRz2rsCCbLZffhtYrhPheUZp7thF06tIUhRRReLiEdE7heRjSKyF0fI4NQ8IvG9Kd93vNm1Icj2kH/vxxFk8F5/EpE1bpN8N9Akyn3C2Qc0DnnfGNhnXFVUcm7w/B8qOZZoVD/Jr59UQzVVe5pKGPGep7QNZ2AwSCecSL8DpzZQo/LcMZMWODW7INHKHQWcgxPtm+DUjAAqykxq5t4jSCfXhqi4fbV/xmnSNzPGNAX2RLlPOKtwBqmD9HU/q/RcEekK5ABfuC+viHSvYlnJhuonsfpJR1RT1dNUwoh3UJoK3CAiXcRJz/w78KIxxg/sxMn+6hpjeZeKSD8RyXHL+9QYU1jF6xsBxTg1mSPc6yvjLhHJdh/qmcC0Kt4n+B29InIHh9dGozEJ+KOIdHAHTcfhNM0jMRk4S0SGuGL9G/CqMeYHt0b1KvA3EWkgIifi/AE8F7zY7VPOcd/mBPuYkwTVTwL1A07GmKsJD+ARkXqS2llkqqlqaEqcBJF6QJbzVuqJSHZVr68J8Q5Kz+A4wPnAJuAg8DsAY8x+nEG4D0Vkt4gcV1lhxph3cbKDXgG+BrrhDOxVlUk4zd2twGrgk0rO347T77oN54/3amPM2irc522cVOsv3PsdJKQJ7zqAfVGufwJ4E1gJfA7MdD8LXr8vmDljjFmFM/g6GfgGR3zXhpR1LVDfPTYVuMa9JsgBnC4cgLXu+2RB9ZN4/dyGo4mbgIvcf9dGqnVdoZqqnqZOwnn2s3BaZweAd6pw3xojkbudMw8RGQo8b4zpmGhblNRD9aPEm0zVlK59pyiKoiQNGpQURVGUpEG77xRFUZSkQVtKiqIoStKQkFTPli1bmtzc3ETcOuVZsmTJt8aYVom2oy5RvdQM1YwSC4nWS0KCUm5uLosXL07ErVMeEdlc+VnpheqlZqhmlFhItF60+05RFCWcooVQMN75qdQpCQ1K/oDNA3O+4LsfS2K7UAWTsbz1+XY+2vhtbBepXpRYKFoIE8+Gufc6P1U3dUpClw9Z+tVuHnl/AxM/LuTm03tyQZ4PqXjDTIegYAIl4MmGMdPBlzEbQmY0wUrM9r0Hee3aE+jaqgobjapelFgpLHD0YgLOz8IC1UwdktCWUn6X5sz6/RCOatOIv7yyktFPfcqW7/dHvyhUMP5imHef1mQyBK/H4qkxeXgs4YqJi9mzv7TyiyI5GEWJRu4QpwIjHrA8sGeL+pg6JOFjSj1K1/BCr494cmiAFUW7+cuDT7PqxTsrFkH9FiCCs9itDV/O0yZ2BuFrfgRPXDyALd8f4JrJSygN2NEvCHUwnmxHP9qVlzlE6rqtSnduvxHQ43RAYMlE9TF1SFy670TkGZzVa78xxhy2JXGFuF0rVqCEX3iyKfjZ32gw9248q0spWfs4DLuP7JLdjmPx5Tvnv3UT2LYTmAxgbG1iZxJFCxlYVMATQ3tw6Xu7uOONVfw9bz+yeUGZTsLpNwIQaNvX0Y925aUUNfUv5Z43lP9s2P1wYFd5HxM8LuL4F/UxdUq8xpQm4GwxPCmmq8p1xR2k+boXMOJHsPHbJTD7RgzGGWfqcTo0bO2cjw3GcprWxjjiyq3y9vNKgomHkznFk83dA/7Da4veJfD5fXiN39HDsRdB35GHOxhPtrMxtI4VpCITqLF/cbv6m+WW/2zWuLJKbqiPMQHXx1iAqI+pQ+ISlIwx80UkN+YLc4c4jiQQAAx8/RliecEGSwRj/AhO3JG1M8DKAvd4xFqOkipMIA6VmNH1PqRhOwt2lYLYjo4WPwNLJ8Hx18H2zyBQXFbTxa3ABIOUOpmUoEb+xZPtBJ9gV7/lLfMhImC7vscAa2cAbmUXovuYooWOHtX3xJ06y74TkauAqwA6derkfOjLd2q2i58FjCOQDsdAu75YbftizxqHsf1lWyXapdBhALTrW1YbrggVTdISr0qMtXwKw39xH/7ZXsSUYolxtGL74cMHQy60HAfTd5TziqQL1UvKc5iP8eU7XXbz7nMCkrEh4IcOxzo+pG1ft6XkDynFBiMwYEzFPkYzOmuVOkt0MMY8aYzJM8bktWoVsoJF35HgreeaYsO2ZbB8KrTpjTV8PIjHGTpyKzOHjodStBBmXA8zbnD+rfMMUh4RuUpEFovI4p07dzofBisxwWqK7Sdn50oCR49gvgwkgBV5H2rBqfH68p3XkHGHByTVS8oT0cf48mHozeDJIZKPwfUxYSVBk45lGgkmRiye4PxcMVUzOmuRxG9zHLE24z7oIeOQNr3xL5tC0aqP6XRwHR5sTKAECY4HFC2ECWe43TPAsslw7GgdO0hxjDFPAk8C5OXllcWaviMdhxIocVpNy6bQwPZzsieL/xWfweXeWXiwkfDwdGBX2b+LFsKKKYA45em8lPSmEh9Dm95Oy3rdWzhdvDllXbvBCkuwC1Cs8l2A2g0cdxIflKCsNrP547ImcTB1N3cI3rMfpFPfTwlMOAtjl2LEi7fzYKe+XFgAgZD5Kjp2kN4EHUxhgTN/ZMlEMAEsG07ueyQXLL2dyztuZfhRRyAfP+o4oHAnE16JOf2fqpd0J9zHhM4/8uXDiCmRu3CDFRbcqQfGdoYZBlwCTXza3VsLxCslfCowFGgpIluAO40xT8dUSKizqd/isNRdT+dBWJe+ydy3XuXRTW05ZnlD7vQZJHcIeLLKnExlYwdK6hPshitaWK7V1LP+HkbkH81vPz2KG3v34LeXnlmBkwmrxBzYVaY91UvSERf/AmU+ZsUUWDbFqdAsn1o2JhR8hRKeLCEh45Oqk1ohXtl3I+NRziFRFIyP2J0inQZx6pX5fDxzDU8t2ESWR7hl+EBk7Mzy3TFBsVQ0SKnOJ6HUppP5tWcqW3v8i/97ex1dW/bn9CHjyl8TqRIT1ILqISmJm38B5xkXFjjJDVXprg2vLFeW7av+pcYkR/ddOMHaSYTuFBHh1jN60WbvZ3z/0YO8VHwaF553/uGDkpFEoVkzSUFtOhkJlHBdl+3MP9CFG15aTsdmR3B0xybOuUGHcfr/wfblHKrEQHTNqJNJL6L4l3KEPvvQyk1FPkb9S1xIzqAUWjuJ4AxkyyKu+PJ67KxiSla8xqz6Xoaffk7lotAB7fQkzMlkdTuJJwf05dxHP+SKSYuYft1g2uz5LLI2omlGnUx6Uol/ASp+9tE0of4lLiR87bsKiZS6G6SwAAmU4MEmW/x8/uFMZq38uvLFN8PXQdMB7fQg6GROvfWQk2jVKIenxuTxw0E/V0xcTOnG+ZG1EU0zuphr+hLNv0DFzz6aJtS/xIXkbClVRkjN2PJks7vZIK5/cTldzu5Lr2jN8qrUkJTUJMKYUK92jXl4xLFc+dxiHqzfmj95spFwbUTryqlqN4+SflT07KNpQv1LXBBjIk43rFXy8vLMYVsVx9p3H3L+9837cd7jH7HnQCmzfplD2+8Xpa0oRGSJMSYv0XbUJRH1EgNPzt/I32et5b6B+xnZ+qvYVnNIgzEl1YxLDXxMJq3+kWi9JEdLqTp99yE142bAM2MHcscjzzDzzXWMuGAUDdJQLEoIMTiGK4d0Zf2Ofdy8aAtHjBjBOb4O5U+IlnmnWXnpQQ19TLly0jggJQPJEZTiMEDY5cAqJnjuwRwsIfD8SwTGvomn86BaMlhJKDE6GBHh3l8ezebv9nPjy5/ha34E/Ts1UweTScQjCUETX+qE5Eh0iMcAYWEBHrsUr9h47FI+eu/1+NupJAfVSEDI9lr896IBtG1cj6smLWHn6gJd7y6TiJOP0cSX2ic5glKE7KmYcUVnxINtZfHA+ta8s2p7xedXZfdJJTmppoNp3iCbZ8bmUewPMGP6S5hYHIzqJbWJo4+psu5UM9UiObrvoOZ9967opLAA0/EE/DMDjJu2gjfbNCK3ZYPyXTU7VjtL1gfXRdNmeGpRgyynI1s34tFR/Xl44hpGZ3vJEpCKHExQMwf3wsePqF5SnTj5mColxaiPqTbJE5TigSu6HODRUfu55eGnmff0DEYN7Uf2nFvLFmK0A04NGZxN4HSSW+pRAwdz0lGtKDzzXEZMN/y+2w6GnnZexat/BNc8C6J6yWzCdRcMRKHrdaqPqRHpFZRC8P34uZP4sL8EZk8GMe6S9TaEbmsgls4/yUAuOT6X9Tt+zthPNvPPfu24wEf5mm746tBBVC9KkNDEBxHHv6iPqTFpG5SCiQ+Ijd8ItniwRNytjsVZKdqynE2+tAaTkdx5Vm8Kd/3Ira+tpE9gLX3mXFyWWTXs/vKrQ+NqR/WiBAlNfDCW409QH1NT0jcoBRMfAiUE8HCvGcsNJ7akcc9TnOOaCpzxeD0Wj4zqzy8f+5C5b71Ib0qQYOJD6HYWVVkdWsk8wld3GHZ/mU5AfUw1Sd+gFJL48F3TPF6cdoB1hU15/qcDsSxRoSgANKmfxTNjBnLHoyspNh5yQhMfdOKsEo3KEh9UO9UifYMSHHIq7YA7D37FX15ZyTMfbuKKIV3Ln6eTKDOa3JYNuOaiUVz8TIBft9zMeeddiDeaDlQvSpCqVFxULzGR3kEphAvyfMxZ/Q3/fHsdQ7q3okfbRs4BnaWtAMd3a8Gvzj2PP7+yktUrGvHXzhWcqHpRYkH1EjPJMXm2DhAR7j//aBrleBk3bTmlATerSmdpKy4XDuzE5YO7MOGjQp77ZHPkk1QvSjjRJsmqXmIm/VtKRQsPbZXesu9I7jn3J1wzeSmPz9vI73/aXbcnUMpxy9E/0GvDuyydMZfCr5uSm3da+Zqt6kUJJXQ+WzDTLm9s2XHVS8ykd1AqWggTznDnmwDLJnP62Bmc1bc9/5m7ntP6tKWH7oGiBClaiOe5czjfX8z5Xht7hWCvegRrzJtlulC9KKEUFpRNG7BtZxWHNr1VLzUgvbvvCgucuQJB3ObzX8/qTeN6WTw99QXs+eOdY9F2oVQyA7erRdwJsx4Mtr+EA+vnlZ2jg9ZKKLlD3PlJLsYu30WneomZ9G4p5Q4BT1ZZS8ltPrdomMNDJ5Yw4INbYK4fvLo2lUJZV4u/GMHGYFFqvNy3uiW3n2yTtW2xDlor5fHlO112oevcBbvoNMmhWqR3UPLlw9iZh8aU6DvykChO9K7FFj8WNiZQgujaVEpoV0v9FsiBXXxysAeT3hNK31jF31sWONup12RPHiX9yBvrdNmFt4jisYdTBpLeQQkqnEcgXYYg3mz8pSUExEt258FIAsxTkowwvZwCXBNYy+PzNnLikG6cqYPWSiQi+RlNcqgW6R+UKsKXjzXmTZZ+MJ2/r2rB2O99nN0p0UYpyciNv+jBlzv38fsFO2h35kQG2J/rGIFSOZrkUC3SO9GhMnz5HDvqbkrbD+TuGav5ceNHuimXchiWJTxwYT96tm3MmHfgi6OuchyMbuKmVIYvv3wSlWqmUjI7KAEeS7j3lz/B9+NKsp8/V7fHViJyRLaXp8bkUT/bw+UTF7HniwW6nboSG8HEB9VMVDI+KAEc07EpN7Vbjscu1pnXSoW0b1qf/12Sxzd7i/nktUcx/oOqF6XqrJgKqplK0aAEULSQgbtnOft0Acby6qCkEpF+vqY8farN0P3vAMbZyk31olRG0UJY9jyHNv9TzVSIBiWAwgLEDiCAbYQvO56jg5JKhQzOWkuW2AhgEDh2lOpFiU5hgbNFOoBqJipxyb4TkWHAQ4AHeMoYc388yq0zQjYE9OPhni39ePhgKY3qZSXasrQlpTWTOwTx5hDwF1NivKxsPAx1L7VLdfVy15urWL1tb63aVhW6lzTlduPFi8FPFndvOpr1T3ycaLMA6N2+MXee1SfRZhyixkFJRDzAo8DPgS3AIhGZboxZXdOy64yQDQG/qt+P918+yCNzN3Dz8F6JtiwtSXnN+PKRMdOxN87njmVNmPkuvNJ9L73aNU60ZWlJyusFWJ/dm7tb3Eefks9YlX0M67N7J9qkpCUeLaV8YIMx5ksAEXkBOAdIGcEAhya/dS9ayGOd3uLZj9bz5UAfXVs1TLRl6Ujqa8aXT5Yvnz/1P8j3Dz/DB0+/Sdtfj6RZj8GJtiwdqbZekqkFAMc7P3Q9vKjEIyh1AIpC3m8BBoWfJCJXAVcBdOqUpLNU3ZTN0wMlnOLx8MCrzbjlN2MSbVU6UqlmUkIvQJs9n/GkuQvbX0Jg6ksUj5lOTpfjE21WupF2PkbXw6uYOkt0MMY8aYzJM8bktWrVqq5uGxvBVaJNgBwJ4P3qQxas/1YnvCWAlNALQGEBll2KV2w8xs+cWa9gvvpU9ZIAUkIzFW36pz7mEPFoKW0FfCHvO7qfpR4ha1WJJxuT1Yzd067DBOYitl9rNvEjLTWDWHi/Xkbg2Ql4sVUv8SM99WJ5YM8WWDwB3rpJW08u8WgpLQK6i0gXEckGRgDT41Bu3RNcq+rUW5Fh9/MnewKnF78FAZ1UG2fSTzMDLsFjWfzCuwSPXap6iS9pqRcQWDLR2fZCfcwhahyUjDF+4DrgbWAN8JIxZlVNy00YvnynNrPmDSy7BI84k90Moiv9xom01AyCBEqwMCDOfDfbylK9xIF01cuhbjzbBrFAPOpjiNM8JWPMLGBWPMpKOMGByJCN3kqMxZq2Z9PvzGsyulkdT9JOM2Gz9V/np8ySodzb5BjaJNS49CCt9eLJgtP/CQd2aUYeuqLD4QQHIrEBC+k2lEc6P8SIbRfwdeOjE22dkoyEzdaX/hfT84qn+KikK1dNWszB0kDUy5UMI9LqDnljy68mnsFoUAonOBApHmeb9KE3c8G552HbMP6dLxJtnZKM0BxflAAAD7NJREFUlNNMPeg7it7tG/Pghf34bOse/jRtBcaYRFupJAsR9KKUkbmb/FVEhI25fMCYEzrz1IJNXHZiF3q315n7Sghh26gHB6p/0SefP5/Wk3+8tZYjWzfk+p8dlWBDlaRAN/+LigalSIRubezOvv7DUcfz0uIs/vHWWiZepiJSwgjqJWxi5NUnD2T9Nz/w4Lvr6d66EWcc0y6xdirJQQQfowHKQYNSNEJmXze0PExtdza3rf8JC9Z3ZXD3lom2Tkk2widGrpiCFBZwf/4JfLWrGeOmLcfXvD7HdGyaaEuVZCF0hQfLA8deBH1HZnRw0qAUjXJOJkCvba8wNed13p32KfbPT8M6+J3WbpQywidGLpsCgVKyLYuJA67muV27eWjCl9z7u8tp26Reoq1VkoEwH8PiZ2DpJBg+Htr0zsgWlAalaASdjP8gzkwlQzZ+Ti+ejcyc7cwt8ORk/AxsxSV0rCA4Ux8bbJsGix7hN1iMMV5ufzqbu6+7jPrZnkRbrCSaMB8DgO2HmX90NgLMwJVkNPsuGkEnkzfWCT4IAJa4x42tM7CV8vjyndTeviPBKv/nJdjkWAFaf7eIcdOWY9uakZfxhPoYCamkBH1LBq7yoEGpMnz5cOaDMHYG5I1FPDnYWGBwfuoMbCUSvnynC8byEqzMIBaWJ5seg05n1srtPPiuTjFQKPMxZ/zb0YtYzoTaYNp4hvkY7b6rKsFsmb6jkE0FPL10Dwf27OSykRfTIEOa1UqM5I0tGxeo3+LQjP2zOw7kw+LPeHjuBrq1bsg5/Tok2lIlGQjVSzAI6ZiSUim+fMSXz6CuezjrkQUcWN+cP3dLtFFK0hKa+usiwD3nHk3ht/u58eXP6NT8CI7t1Cwx9inJRbheMigYBdHuu2pydMcmnN23PU8v2MT2PQcTbY6SzETYKyfba/HfiwfQpnEOV05awrbdBxJooKIkDxqUqoPrZG495gdsY3hgjo4NKBUQnIcy917nZ9HCQ/pp/t1ynh4zkIOlAa6YuJj9Jf5EW6skCxm86Z9238VKyGS3Np5s/n3k71m7/HU2HzmKzv2GJto6JdmIMKGW5S8cWvXhqDHT+c+oY7l8wiJueHE5j48egHUovVPJSMK3TB92f0atIK4tpVgJdTL+Ys4sGs8N3mm0feOCjKzVKJUQuvimJ5ty++i4qb6n9GjNrWf05u1VOxg/Z12iLVYSTZiPYda48i3tNEeDUqyEOhnLQoyNFxuPXUrhkrdrXn4GN9vTkpDdjBkz3Zm/FCHV97ITcxmZ7+PR9zfy+rIYdvpWvaQfYT4GY8d3vlKSa0a772IlfEXot27CBErw4+GhjW34l23wVLf7JbzZnkGzuNOa8IyqCCtEiwh3nf0TNn37I39+5TN8zY9gQOdKMvJUL+lJBB9z6BnXdL5SCmhGg1J1CHUybXojhQUsKu3Ja+/A8Uu3cEGer3rlho8/FBYknWCUOBAhTRycjLzHRw/g3Mc+5DfPLeb1355Ix2ZHVFyO6iV9CfMxcZuvlAKa0e67muIuKzP4lOEc26kp//f2On4srmYWVfj4QwbN4s4YKuk6adYgm6fH5FFcanPFxMXRtaR6yQyCS1fFI3ikgGY0KNWEEAcjItx+Zm92/lDMY/M2VK+88PGHJKvBKDUkUnp4BI5s3YhHRvfnix0/cP2LUdbIU72kP/Ee/0kBzWj3XXWJ0Dfbv1M+5/Zrz/8KNnFhXic6tYjS9VIRFXTtKGlAeFbVvPtg6M0Rn/fJR7XijjN789c3V/PPt9dx0+k9I5epeklfamv8J8k1oy2l6hJp/knBeG7vuw+vJfxtxuq43KbEb8elHCUJCHadYAE2fDkvaotpzAm5jBrUif9+sJFXlmypS0uVZCBSJSZJM+biiQal6lIubdPd0G3uvbR45dfcm7efd9fs4L01O2p0ix+L/Qy8912mfPpVnIxWEkqw66TbUGcl6Eq2PnEy8vpwQrcW3PzqSpZs/q7ispM8zVepBjFWYtIFDUrV5f/bO/sYqcorDj/nzvItkYooirtSKlar7SJdlxKL0WIpUKy10aRSi5A2apMmbWJtq01sk9qWpqEa1KobG7/1Hz9JBAWMLVtDRVi2gFAspVggapVSFF13mbmnf7z37tyZncUddnfux54nWXZ27r27Z2YOv3Pe9z3vudG52fOudjfjCkZNlx6/m6knHcctz75GR1eh+t8dCMzml1/gUMcRPjVhzMDbb8RDfbObssuN6NNi87Ccxx++NZ1Tx43kuoc3se/ghz1P6uNalZEyqkxiqiLBSYwFpf4QvaFbpKKlbsqF3Pr1c9n/vw5uq/aeORGBaW5dwsVj9tA0+YTBsd+Ih2hCM3epE5lK4hAIx7gD7dx3zfl05l1F3uHOfKmoVCrzNbJBlUlMn0h4EmOFDgNB+Wa3Pa3MmAxXNTdwX+tu5n/2FKbx+tH3GoTicmhft8B4qiyc+Maxb8Y1kkt9M7y93bWQUd+JTriQvXeDW6Pc/Fj37bDPuGYFdy2czpIHXuXJlltZdPAOJLxu7lInVgO1wdJIFlF9CT/b1mU9tSTUkEoaEz2W8L1KFpQGigoic8slv6Jh9Ca2P/QEjYUVRRGJVtGUC5CXA68OvwBHyHFy4yXxvi5jcNi7wfmKH+xDKnQ6Pwh9odAFBKXggXBcOKuZ5Rd0MWfDcpBC8bqOAxW7RBgZIqyY2/hAUWO8Ord00HhV7wkOlF6TgiTGgtJAUS4y+Y8YtfrHXO8XIK/dd8Sm0FnMTMJhdP4jugXIB52+iEd2+GzgMyxv/nIcr8YYbPa0gh+trJSewSh8PiIc88f+E198JDhLxCsNROHUnQWm7NEjkemCjffD5kfAL7iRD/TUmPLkJ+FJjK0pDRQ9RAbwCwguIIUiQigi4TUlIuQEaOfEBTxz8JN8Z9J+vP2v1uwl1AIRuVJEXhMRX0Sa4rYnNibPgroRgOcy3oYZFX2BpiVOQABalyGjx+PVjaSAUFCPg6d9yR0LE5wXb4X757ns2MgWlTQGhcKRYkCCosbs3eDKyP0Kx+qbi1N5WVpTEpErgV8AZwPNqrpxIIxKJaHIdI96SnfhK1BQjzdn/pL6MIM5tNcJko+btps6Bx0zgda/rOPREXczclcBdt9THKInLKM5RrYB3wDujduQWClfh1x1IyU+Ix7M+53re/by7bDzeXc8NwKZu5T8vjak/THG/nst/oPr8KYtdHtZ8J1wrbzBXZtynzGNidBDY0KifpOD+cvc4we/FviEAuI0Zub3nc+9vd01es13uk7k85dB0+KavZSj0d/pOxOYkFBk/vQbt59AfcLxUfjvy7nz2bp+Kws/uIkTtrQU54U/fw1MbHQdx/OdLFYhJ+rWoAoFN0Rvf7yYMacYVd0Bbg/OkCdcJ2hdVprNhrzV7oJVoav4XDD9MmL86aj4iPrk813k8z7DPa+YSaufuAXsY8Q0JqSixkBxHkbg03Od32x+yPmKy3hdWfnZlxU7josEPqeJS2L6FZRMYMoIyzffWO8+eC8HCPh5xMsxi81cUNhArl1DF3JzvcefBh0H0EIXHj6+COLlwA9HXDrkSn1F5FrgWoCGhoaYrRlkwk2S4UhHvMgNAY+UnhuZ/pXccPxCF0f8HLe9PZ2fzJtGbtWPigvaCVvAPhZMY8o4isbg5eD11eBHfEY85wsX3VRadaeeC0wajLISlMTUrNBhyIhMpfLNoNTb2/QgHsWAFK4xfXjqTDbu+S8zNEcdilc3HJn72yDjKZYFp0VkRGQtMLHCoZ+p6rN9+R2q2gK0ADQ1NfXSkTQjlE/lhbe+BreI3T1S8uDMuSXXeHtaeen9M2hZV4c0TOGmJasSu4A92JjG7HOzKlGmXFTaXzFadTfjOlh/Z+KSmI8NSgMhMDAERSYqCOEaUvvj3dmwj+CrsLZrGi33vUKbnsmVJ/+an571LuPPnV28vnFh6kRGVa2OvVp6a5K5+DlXJn74HfjHGti5Cna9WCz5rW9mnipXd23j3j/vZupJjVwx64ba298PTGOOgd40JprE5Ea4KbtoRWZ51d1ZX02cvnxsUDKBGSAiDiGjxiNvtSObH2WOtDF7+Fa2zX6Yz838bs+Nsgnv6GsMMtF1p52rKm54FBF+fuk5/OvdD7j5qa1MHj86VV1ATGMGiPrmYhKDdK9T9+gyXh7MEqYvVhJeS8K2RE2L4fh6xC/g4TNM85znbxsSnRtE5HIR2QfMBJ4TkRfitikVfMzN2YblPO5aOJ1JnxjFjU9sIV+w7vJDkvpmWHA7LLjNTQOnsP1Uf0vCLwfuACbgBKZdVb8yIJZlnVBkErqrerBQ1aeBp+O2I3VUmnopY9xod9daX6Eul4180zSmH6RUY/pbfWcCc6z0QWQMo4Q+TLVMmXBcjYypDaYx/SClGmNthuIkgfO5hmFkiBRqTDbG+IZhGEYmENXaV06KyDvAGzX+sycC79b4b/aH3uw9XVUn1NqYODF/6RNHs9d8pjZkxWdi9ZdYglIciMhGVU1NA9C02Zs10vb+p83eLJK2zyCp9tr0nWEYhpEYLCgZhmEYiWEoBaWWuA2okrTZmzXS9v6nzd4skrbPIJH2Dpk1JcMwDCP5DKWRkmEYhpFwLCgZhmEYiSFTQUlE6kXkJRHZLiKvicgPKpxzkYgcEpH24OuWOGyN2LNHRLYGtvS41bM4lovILhHZIiLT47Azi5i/GNWSNp9Jo79krc1QHrhBVdtEZCywSUTWqOr2svNaVXVBDPb1xsWq2tumu3nA1OBrBnB38N3oP+YvRrWk0WdS5S+ZGimp6puq2hY8fh/YAUyK16p+cxnwkDr+CowTkVPiNioLmL8Y1ZJBn0mcv2QqKEURkcnAecArFQ7PFJG/icgqETmnpob1RIHVIrIpuJ1zOZOAvZGf95Hu/wSJxPzFqJaU+Ezq/CVr03cAiMhxwJPAD1X1vbLDbbjeTodFZD7wDG7oGhdfVNX9InISsEZE/q6q62K0Z8hh/mJUS4p8JnX+krmRkogMwznLo6r6VPlxVX1PVQ8Hj1cCw0TkxBqbGbVnf/D9P7j7xpT3md8P1Ed+Pi14zhgAzF+MakmTz6TRXzIVlEREgD8CO1T1972cMzE4DxFpxr0HB2pnZYktY4LFUkRkDDAH2FZ22gpgUVAl8wXgkKq+WWNTM4n5i1EtafKZtPpL1qbvLgC+DWwVkfbguZuBBgBVvQe4AvieiOSBDuCbGl9bi5OBpwP/rQMeU9XnReT6iL0rgfnALuBDYElMtmYR8xejWtLkM6n0F2szZBiGYSSGTE3fGYZhGOnGgpJhGIaRGCwoGYZhGInBgpJhGIaRGCwoGYZhGInBgpJhGIaRGCwoGYZhGInh/85cXCMeaOZOAAAAAElFTkSuQmCC\n", 1377 | "text/plain": [ 1378 | "
" 1379 | ] 1380 | }, 1381 | "metadata": { 1382 | "needs_background": "light" 1383 | } 1384 | } 1385 | ] 1386 | }, 1387 | { 1388 | "cell_type": "markdown", 1389 | "source": [ 1390 | "This again tells us that the model complexity decreases with increase in the values of alpha. \n", 1391 | "\n", 1392 | "But notice the straight line at alpha=1. Appears a bit strange to me. Let’s explore this further by looking at the coefficients:" 1393 | ], 1394 | "metadata": { 1395 | "id": "8mtV-ckZPh9T" 1396 | } 1397 | }, 1398 | { 1399 | "cell_type": "code", 1400 | "source": [ 1401 | "#Set the display format to be scientific for ease of analysis\n", 1402 | "pd.options.display.float_format = '{:,.2g}'.format\n", 1403 | "coef_matrix_lasso" 1404 | ], 1405 | "metadata": { 1406 | "colab": { 1407 | "base_uri": "https://localhost:8080/", 1408 | "height": 423 1409 | }, 1410 | "id": "nmXOcN9PQrtf", 1411 | "outputId": "90f91d7d-aba2-49c7-e0c3-898d476374db" 1412 | }, 1413 | "execution_count": 9, 1414 | "outputs": [ 1415 | { 1416 | "output_type": "execute_result", 1417 | "data": { 1418 | "text/html": [ 1419 | "\n", 1420 | "
\n", 1421 | "
\n", 1422 | "
\n", 1423 | "\n", 1436 | "\n", 1437 | " \n", 1438 | " \n", 1439 | " \n", 1440 | " \n", 1441 | " \n", 1442 | " \n", 1443 | " \n", 1444 | " \n", 1445 | " \n", 1446 | " \n", 1447 | " \n", 1448 | " \n", 1449 | " \n", 1450 | " \n", 1451 | " \n", 1452 | " \n", 1453 | " \n", 1454 | " \n", 1455 | " \n", 1456 | " \n", 1457 | " \n", 1458 | " \n", 1459 | " \n", 1460 | " \n", 1461 | " \n", 1462 | " \n", 1463 | " \n", 1464 | " \n", 1465 | " \n", 1466 | " \n", 1467 | " \n", 1468 | " \n", 1469 | " \n", 1470 | " \n", 1471 | " \n", 1472 | " \n", 1473 | " \n", 1474 | " \n", 1475 | " \n", 1476 | " \n", 1477 | " \n", 1478 | " \n", 1479 | " \n", 1480 | " \n", 1481 | " \n", 1482 | " \n", 1483 | " \n", 1484 | " \n", 1485 | " \n", 1486 | " \n", 1487 | " \n", 1488 | " \n", 1489 | " \n", 1490 | " \n", 1491 | " \n", 1492 | " \n", 1493 | " \n", 1494 | " \n", 1495 | " \n", 1496 | " \n", 1497 | " \n", 1498 | " \n", 1499 | " \n", 1500 | " \n", 1501 | " \n", 1502 | " \n", 1503 | " \n", 1504 | " \n", 1505 | " \n", 1506 | " \n", 1507 | " \n", 1508 | " \n", 1509 | " \n", 1510 | " \n", 1511 | " \n", 1512 | " \n", 1513 | " \n", 1514 | " \n", 1515 | " \n", 1516 | " \n", 1517 | " \n", 1518 | " \n", 1519 | " \n", 1520 | " \n", 1521 | " \n", 1522 | " \n", 1523 | " \n", 1524 | " \n", 1525 | " \n", 1526 | " \n", 1527 | " \n", 1528 | " \n", 1529 | " \n", 1530 | " \n", 1531 | " \n", 1532 | " \n", 1533 | " \n", 1534 | " \n", 1535 | " \n", 1536 | " \n", 1537 | " \n", 1538 | " \n", 1539 | " \n", 1540 | " \n", 1541 | " \n", 1542 | " \n", 1543 | " \n", 1544 | " \n", 1545 | " \n", 1546 | " \n", 1547 | " \n", 1548 | " \n", 1549 | " \n", 1550 | " \n", 1551 | " \n", 1552 | " \n", 1553 | " \n", 1554 | " \n", 1555 | " \n", 1556 | " \n", 1557 | " \n", 1558 | " \n", 1559 | " \n", 1560 | " \n", 1561 | " \n", 1562 | " \n", 1563 | " \n", 1564 | " \n", 1565 | " \n", 1566 | " \n", 1567 | " \n", 1568 | " \n", 1569 | " \n", 1570 | " \n", 1571 | " \n", 1572 | " \n", 1573 | " \n", 1574 | " \n", 1575 | " \n", 1576 | " \n", 1577 | " \n", 1578 | " \n", 1579 | " \n", 1580 | " \n", 1581 | " \n", 1582 | " \n", 1583 | " \n", 1584 | " \n", 1585 | " \n", 1586 | " \n", 1587 | " \n", 1588 | " \n", 1589 | " \n", 1590 | " \n", 1591 | " \n", 1592 | " \n", 1593 | " \n", 1594 | " \n", 1595 | " \n", 1596 | " \n", 1597 | " \n", 1598 | " \n", 1599 | " \n", 1600 | " \n", 1601 | " \n", 1602 | " \n", 1603 | " \n", 1604 | " \n", 1605 | " \n", 1606 | " \n", 1607 | " \n", 1608 | " \n", 1609 | " \n", 1610 | " \n", 1611 | " \n", 1612 | " \n", 1613 | " \n", 1614 | " \n", 1615 | " \n", 1616 | " \n", 1617 | " \n", 1618 | " \n", 1619 | " \n", 1620 | " \n", 1621 | " \n", 1622 | " \n", 1623 | " \n", 1624 | " \n", 1625 | " \n", 1626 | " \n", 1627 | " \n", 1628 | " \n", 1629 | " \n", 1630 | " \n", 1631 | " \n", 1632 | " \n", 1633 | " \n", 1634 | " \n", 1635 | " \n", 1636 | " \n", 1637 | " \n", 1638 | " \n", 1639 | " \n", 1640 | " \n", 1641 | " \n", 1642 | " \n", 1643 | " \n", 1644 | " \n", 1645 | " \n", 1646 | " \n", 1647 | " \n", 1648 | " \n", 1649 | " \n", 1650 | " \n", 1651 | " \n", 1652 | " \n", 1653 | " \n", 1654 | " \n", 1655 | " \n", 1656 | " \n", 1657 | " \n", 1658 | " \n", 1659 | " \n", 1660 | " \n", 1661 | "
rssinterceptcoef_x_1coef_x_2coef_x_3coef_x_4coef_x_5coef_x_6coef_x_7coef_x_8coef_x_9coef_x_10coef_x_11coef_x_12coef_x_13coef_x_14coef_x_15
alpha_1e-150.960.221.1-0.370.000890.0016-0.00012-6.4e-05-6.3e-061.4e-067.8e-072.1e-074e-085.4e-091.8e-10-2e-10-9.2e-11
alpha_1e-100.960.221.1-0.370.000880.0016-0.00012-6.4e-05-6.3e-061.4e-067.8e-072.1e-074e-085.4e-091.8e-10-2e-10-9.2e-11
alpha_1e-080.960.221.1-0.370.000770.0016-0.00011-6.4e-05-6.3e-061.4e-067.8e-072.1e-074e-085.3e-092e-10-1.9e-10-9.3e-11
alpha_1e-050.960.50.6-0.13-0.038-00007.7e-061e-067.7e-08000-0-7e-11
alpha_0.000110.90.17-0-0.048-0-0009.5e-065.1e-07000-0-0-4.4e-11
alpha_0.0011.71.3-0-0.13-0-0-0000001.5e-087.5e-10000
alpha_0.013.61.8-0.55-0.00056-0-0-0-0-0-0-0000000
alpha_1370.038-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0
alpha_5370.038-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0
alpha_10370.038-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0
\n", 1662 | "
\n", 1663 | " \n", 1673 | " \n", 1674 | " \n", 1711 | "\n", 1712 | " \n", 1736 | "
\n", 1737 | "
\n", 1738 | " " 1739 | ], 1740 | "text/plain": [ 1741 | " rss intercept coef_x_1 ... coef_x_13 coef_x_14 coef_x_15\n", 1742 | "alpha_1e-15 0.96 0.22 1.1 ... 1.8e-10 -2e-10 -9.2e-11\n", 1743 | "alpha_1e-10 0.96 0.22 1.1 ... 1.8e-10 -2e-10 -9.2e-11\n", 1744 | "alpha_1e-08 0.96 0.22 1.1 ... 2e-10 -1.9e-10 -9.3e-11\n", 1745 | "alpha_1e-05 0.96 0.5 0.6 ... 0 -0 -7e-11\n", 1746 | "alpha_0.0001 1 0.9 0.17 ... -0 -0 -4.4e-11\n", 1747 | "alpha_0.001 1.7 1.3 -0 ... 0 0 0\n", 1748 | "alpha_0.01 3.6 1.8 -0.55 ... 0 0 0\n", 1749 | "alpha_1 37 0.038 -0 ... -0 -0 -0\n", 1750 | "alpha_5 37 0.038 -0 ... -0 -0 -0\n", 1751 | "alpha_10 37 0.038 -0 ... -0 -0 -0\n", 1752 | "\n", 1753 | "[10 rows x 17 columns]" 1754 | ] 1755 | }, 1756 | "metadata": {}, 1757 | "execution_count": 9 1758 | } 1759 | ] 1760 | }, 1761 | { 1762 | "cell_type": "markdown", 1763 | "source": [ 1764 | "Apart from the expected inference of higher RSS for higher alphas, we can see the following:\n", 1765 | "\n", 1766 | "1. For the same values of alpha, the coefficients of lasso regression are much smaller as compared to that of ridge regression (compare row 1 of the 2 tables).\n", 1767 | "2. For the same alpha, lasso has higher RSS (poorer fit) as compared to ridge regression\n", 1768 | "3. Many of the coefficients are zero even for very small values of alpha\n", 1769 | "\n", 1770 | "Inferences #1,2 might not generalize always but will hold for many cases. The real difference from ridge is coming out in the last inference. " 1771 | ], 1772 | "metadata": { 1773 | "id": "hmVoR602Q_9e" 1774 | } 1775 | }, 1776 | { 1777 | "cell_type": "code", 1778 | "source": [ 1779 | "coef_matrix_lasso.apply(lambda x: sum(x.values==0),axis=1)" 1780 | ], 1781 | "metadata": { 1782 | "colab": { 1783 | "base_uri": "https://localhost:8080/" 1784 | }, 1785 | "id": "43M_fKAvQ3iK", 1786 | "outputId": "242be1eb-672b-450a-f011-d90b30d76b9c" 1787 | }, 1788 | "execution_count": 10, 1789 | "outputs": [ 1790 | { 1791 | "output_type": "execute_result", 1792 | "data": { 1793 | "text/plain": [ 1794 | "alpha_1e-15 0\n", 1795 | "alpha_1e-10 0\n", 1796 | "alpha_1e-08 0\n", 1797 | "alpha_1e-05 8\n", 1798 | "alpha_0.0001 10\n", 1799 | "alpha_0.001 12\n", 1800 | "alpha_0.01 13\n", 1801 | "alpha_1 15\n", 1802 | "alpha_5 15\n", 1803 | "alpha_10 15\n", 1804 | "dtype: int64" 1805 | ] 1806 | }, 1807 | "metadata": {}, 1808 | "execution_count": 10 1809 | } 1810 | ] 1811 | }, 1812 | { 1813 | "cell_type": "markdown", 1814 | "source": [ 1815 | "We can observe that even for a small value of alpha, a significant number of coefficients are zero. \n", 1816 | "\n", 1817 | "This also explains the horizontal line fit for alpha=1 in the lasso plots, its just a baseline model! \n", 1818 | "\n", 1819 | "This phenomenon of most of the coefficients being zero is called ‘sparsity‘. \n", 1820 | "\n", 1821 | "Although lasso performs feature selection, this level of sparsity is achieved in special cases only which we’ll discuss towards the end.\n", 1822 | "\n", 1823 | "This has some really interesting implications on the use cases of lasso regression as compared to that of ridge regression." 1824 | ], 1825 | "metadata": { 1826 | "id": "WgiW_7hNRNBV" 1827 | } 1828 | } 1829 | ] 1830 | } -------------------------------------------------------------------------------- /14_WOE_IV.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/14_WOE_IV.pdf -------------------------------------------------------------------------------- /16_LR_Revision.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/16_LR_Revision.pdf -------------------------------------------------------------------------------- /17_LR_1_Interview_Questions.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/17_LR_1_Interview_Questions.pdf -------------------------------------------------------------------------------- /18_LR_2_Interview_Questions.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sandipanpaul21/Logistic-regression-in-python/83f46ca3cbef084a01c368b8347f86a0c528d8be/18_LR_2_Interview_Questions.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Logistic-regression-in-python 2 | 3 | ### 01_LR_Introduction (Theory) 4 | - Predict categories based on MLE 5 | ### 02_Odd_LogOdd_OddRatio (Theory) 6 | - Probability : Something Happening / Everything that could Happen 7 | - Odds : Something Happening / Something Not Happening 8 | - Log(Odds) : To make Odds output symmetry 9 | ### 03_Logit_Model (Theory) 10 | - Indepth Logistic Output explained 11 | ### 04_Likelihood_Probability (Theory) 12 | ### 05_MLE (Theory) 13 | ### 06_LR_Assumptions (Theory) 14 | - Assumption 1 - Appropriate outcome type (Must be categorical) 15 | - Assumption 2 - Linearity of independent variables and log odds 16 | - Assumption 3 - No strongly influential outliers 17 | - Assumption 4 - Absence of multicollinearity 18 | - Assumption 5 - Independence of observations 19 | - Assumption 6 - Sufficiently large sample size 20 | ### 07_LR_Assumptions (Python Code) 21 | - Python Code for Logistic Regression Assumptions 22 | ### 08_AIC_BIC (Theory) 23 | - Akaike Information Criterion 24 | - Bayesian Information Criterion 25 | - Choose the lowest score 26 | ### 09_Logistic_Regression (Python Code) 27 | - Python Code for Logistic Regression 28 | ### 10_Multiclass_Classification (Theory) 29 | - One vs All (OvA) also known as One vs Rest (OvR) 30 | - One vs One (OnO) 31 | ### 11_Multi_Class_Classification (Python Code) 32 | - Python Code for Multi Class Classification 33 | ### 12_Regularization (Theory) 34 | - L1 Lasso 35 | - SSR + lamda * (slope)^2 36 | - Useless variable become 0 37 | - L2 Ridge 38 | - SSR + lamda * |slope| 39 | - Useless variable tends to become 0 but never = 0 40 | - Elastic Net : Combination of L1 & L2 41 | ### 13_LR_Regularization (Python Code) 42 | - Python Code of Regularization (L1 Lasso,L2 Ridge & Elastic Net) 43 | ### 14_WOE_IV (Theory) 44 | - Weight of Evidence : Predictive power of Independent Variables 45 | - Information Value : Technique to select important Variables 46 | ### 15_LR_WOE_IV (Python Code) 47 | - Python Code for WOE and IV 48 | ### 16_LR_Revision (Theory) 49 | - Logistic Regression Revision 50 | ### 17_LR_1_Interview_Questions (Theory) 51 | - Logistic Regression Interview quesion bank 52 | ### 18_LR_2_Interview_Questions (Theory) 53 | - Indepth Logistic Output explained 54 | --------------------------------------------------------------------------------