├── Air Quality - Machine Learning ├── .ipynb_checkpoints │ └── air_quality_regression-checkpoint.ipynb ├── AirQualityUCI.xlsx ├── README.md └── air_quality_regression.ipynb ├── Boston and Diabetes datasets Regression ├── .ipynb_checkpoints │ └── boston_diabetes_regression-checkpoint.ipynb ├── README.md └── boston_diabetes_regression.ipynb ├── README.md ├── Salaries based on Positions - dataset ├── .ipynb_checkpoints │ └── salary_position_regression-checkpoint.ipynb ├── Position_Salaries.csv ├── README.md └── salary_position_regression.ipynb ├── Salary Regression ├── .ipynb_checkpoints │ └── salary_regression-checkpoint.ipynb ├── README.md ├── Salary_Data.csv ├── __pycache__ │ └── linear_regression_numpy.cpython-35.pyc ├── linear_regression_numpy.py └── salary_regression.ipynb └── algorithms_numpy ├── linear_regression_numpy.py └── multi_linear_regression_numpy.py /Air Quality - Machine Learning/AirQualityUCI.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucko515/regression-python/f82b63317619af4b5d18958450af912d75b46342/Air Quality - Machine Learning/AirQualityUCI.xlsx -------------------------------------------------------------------------------- /Air Quality - Machine Learning/README.md: -------------------------------------------------------------------------------- 1 | # Air Qulity - Regression in Python 2 | 3 | The dataset contains data of hourly average reposnses from an array of 5 metal oxide chemical sensors embedded in an Air Quality Chemical Multisensor Device. I've used this data to predict the amount of CO based on other parameters. 4 | 5 | ## Dataset 6 | 7 | Dataset used in this project can be found [here]( https://archive.ics.uci.edu/ml/datasets/Air+Quality). 8 | 9 | ## Install 10 | 11 | ###     Supported Python version 12 |          - Python version used in this project: 3.5+ 13 | 14 | ###     Libraries used 15 | 16 | > * [Pandas](http://pandas.pydata.org) 0.18.0 17 | > * [Numpy](http://www.numpy.org) 1.10.4 18 | > * [Matplotlib](https://matplotlib.org) 1.5.1 19 | > * [Scikit-learn](http://scikit-learn.org/stable/) 0.17.1 20 | > * [Seaborn](https://seaborn.pydata.org) 0.8.0 21 | 22 | ## Code 23 | 24 | The code used in this project is inside **air_quality_regression.ipynb**. 25 | 26 | ## Run 27 | 28 | To run this project you will need some software, like Anaconda, which provides support for running .ipynb files (Jupyter Notebook). 29 | 30 | After making sure you have that, you can run from a terminal or cmd next lines: 31 | 32 | `ipython notebook air_quality_regression.ipynb` 33 | 34 | or 35 | 36 | `jupyter notebook air_quality_regression.ipynb` 37 | 38 | 39 | 40 | 41 | ## License 42 | 43 | MIT License 44 | 45 | Copyright (c) 2017 Luka Anicin 46 | 47 | Permission is hereby granted, free of charge, to any person obtaining a copy 48 | of this software and associated documentation files (the "Software"), to deal 49 | in the Software without restriction, including without limitation the rights 50 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 51 | copies of the Software, and to permit persons to whom the Software is 52 | furnished to do so, subject to the following conditions: 53 | 54 | The above copyright notice and this permission notice shall be included in all 55 | copies or substantial portions of the Software. 56 | 57 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 58 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 59 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 60 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 61 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 62 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 63 | SOFTWARE. 64 | -------------------------------------------------------------------------------- /Boston and Diabetes datasets Regression/README.md: -------------------------------------------------------------------------------- 1 | # Boston and Diabetes - Regression 2 | 3 | This project was pure testing of regression techniques. Two easy datasets used are Diabetes dataset and Boston dataset. 4 | 5 | ## Dataset 6 | 7 | Dataset used in this project is the part of Scikit-learn library. You can add them to your code with next lines: 8 | 9 | ``` 10 | from sklearn import datasets 11 | boston_data = datasets.load_boston() #For boston dataset 12 | diabetes = datasets.load_diabetes() #for diabetes dataset 13 | ``` 14 | ## Install 15 | 16 | ###     Supported Python version 17 |          - Python version used in this project: 3.5+ 18 | 19 | ###     Libraries used 20 | 21 | > * [Pandas](http://pandas.pydata.org) 0.18.0 22 | > * [Numpy](http://www.numpy.org) 1.10.4 23 | > * [Matplotlib](https://matplotlib.org) 1.5.1 24 | > * [Scikit-learn](http://scikit-learn.org/stable/) 0.17.1 25 | > * [Seaborn](https://seaborn.pydata.org) 0.8.0 26 | 27 | ## Code 28 | 29 | The code used in this project is inside **boston_diabetes_regression.ipynb**. 30 | 31 | ## Run 32 | 33 | To run this project you will need some software, like Anaconda, which provides support for running .ipynb files (Jupyter Notebook). 34 | 35 | After making sure you have that, you can run from a terminal or cmd next lines: 36 | 37 | `ipython notebook boston_diabetes_regression.ipynb` 38 | 39 | or 40 | 41 | `jupyter notebook boston_diabetes_regression.ipynb` 42 | 43 | 44 | 45 | 46 | ## License 47 | 48 | MIT License 49 | 50 | Copyright (c) 2017 Luka Anicin 51 | 52 | Permission is hereby granted, free of charge, to any person obtaining a copy 53 | of this software and associated documentation files (the "Software"), to deal 54 | in the Software without restriction, including without limitation the rights 55 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 56 | copies of the Software, and to permit persons to whom the Software is 57 | furnished to do so, subject to the following conditions: 58 | 59 | The above copyright notice and this permission notice shall be included in all 60 | copies or substantial portions of the Software. 61 | 62 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 63 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 64 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 65 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 66 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 67 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 68 | SOFTWARE. 69 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Regression using Python 2 | 3 | All projects are implementation of some regression techniques using many different datasets. 4 | 5 | Every project has its one README which will provide more information for project itself. 6 | 7 | ### Note 8 | 9 | Folder **algorithms_numpy** has algorithms wrote from scratch using only numpy. Some projects are implementing those algorithms as well and you are free to use them or improve them. 10 | 11 | ### License 12 | 13 | Each project has MIT license. 14 | -------------------------------------------------------------------------------- /Salaries based on Positions - dataset/Position_Salaries.csv: -------------------------------------------------------------------------------- 1 | Position,Level,Salary 2 | Business Analyst,1,45000 3 | Junior Consultant,2,50000 4 | Senior Consultant,3,60000 5 | Manager,4,80000 6 | Country Manager,5,110000 7 | Region Manager,6,150000 8 | Partner,7,200000 9 | Senior Partner,8,300000 10 | C-level,9,500000 11 | CEO,10,1000000 -------------------------------------------------------------------------------- /Salaries based on Positions - dataset/README.md: -------------------------------------------------------------------------------- 1 | # Salary based on Position in firms - Regression 2 | 3 | This project is typical example when Simple Linear Regression is to 'weak' for a dataset used, instead we have to use Polynomial Regression. 4 | 5 | Dataset is pretty small - only 10 samples, so, some Deep Learning technique would be useless in this case. 6 | 7 | ## Dataset 8 | 9 | Dataset used in this project can be found in CSV format included inside of this folder. 10 | 11 | ## Install 12 | 13 | ###     Supported Python version 14 |          - Python version used in this project: 3.5+ 15 | 16 | ###     Libraries used 17 | 18 | > * [Pandas](http://pandas.pydata.org) 0.18.0 19 | > * [Numpy](http://www.numpy.org) 1.10.4 20 | > * [Matplotlib](https://matplotlib.org) 1.5.1 21 | > * [Scikit-learn](http://scikit-learn.org/stable/) 0.17.1 22 | 23 | 24 | ## Code 25 | 26 | The code used in this project is inside **salary_position_regression.ipynb**. 27 | 28 | ## Run 29 | 30 | To run this project you will need some software, like Anaconda, which provides support for running .ipynb files (Jupyter Notebook). 31 | 32 | After making sure you have that, you can run from a terminal or cmd next lines: 33 | 34 | `ipython notebook salary_position_regression.ipynb` 35 | 36 | or 37 | 38 | `jupyter notebook salary_position_regression.ipynb` 39 | 40 | 41 | 42 | 43 | ## License 44 | 45 | MIT License 46 | 47 | Copyright (c) 2017 Luka Anicin 48 | 49 | Permission is hereby granted, free of charge, to any person obtaining a copy 50 | of this software and associated documentation files (the "Software"), to deal 51 | in the Software without restriction, including without limitation the rights 52 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 53 | copies of the Software, and to permit persons to whom the Software is 54 | furnished to do so, subject to the following conditions: 55 | 56 | The above copyright notice and this permission notice shall be included in all 57 | copies or substantial portions of the Software. 58 | 59 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 60 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 61 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 62 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 63 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 64 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 65 | SOFTWARE. 66 | -------------------------------------------------------------------------------- /Salary Regression/README.md: -------------------------------------------------------------------------------- 1 | # Salary and years of experience - Regression 2 | 3 | The goal in this project is to map years of experience for some employee to their salaries. We can create those mappings by applying Regression techniques. 4 | 5 | ## Dataset 6 | 7 | Dataset used in this project can be found in CSV format included inside of this folder. 8 | 9 | ## Install 10 | 11 | ###     Supported Python version 12 |          - Python version used in this project: 3.5+ 13 | 14 | ###     Libraries used 15 | 16 | > * [Pandas](http://pandas.pydata.org) 0.18.0 17 | > * [Numpy](http://www.numpy.org) 1.10.4 18 | > * [Matplotlib](https://matplotlib.org) 1.5.1 19 | > * [Scikit-learn](http://scikit-learn.org/stable/) 0.17.1 20 | 21 | 22 | ## Code 23 | 24 | The code used in this project is inside **salary_position_regression.ipynb**. The file **linear_regression_numpy.py** contains Linear Regression wrote from scratch using just numpy. The second mentioned file is for optional task. Main code is in **salary_position_regression.ipynb**. 25 | 26 | ## Run 27 | 28 | To run this project you will need some software, like Anaconda, which provides support for running .ipynb files (Jupyter Notebook). 29 | 30 | After making sure you have that, you can run from a terminal or cmd next lines: 31 | 32 | `ipython notebook salary_position_regression.ipynb` 33 | 34 | or 35 | 36 | `jupyter notebook salary_position_regression.ipynb` 37 | 38 | 39 | 40 | 41 | ## License 42 | 43 | MIT License 44 | 45 | Copyright (c) 2017 Luka Anicin 46 | 47 | Permission is hereby granted, free of charge, to any person obtaining a copy 48 | of this software and associated documentation files (the "Software"), to deal 49 | in the Software without restriction, including without limitation the rights 50 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 51 | copies of the Software, and to permit persons to whom the Software is 52 | furnished to do so, subject to the following conditions: 53 | 54 | The above copyright notice and this permission notice shall be included in all 55 | copies or substantial portions of the Software. 56 | 57 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 58 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 59 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 60 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 61 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 62 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 63 | SOFTWARE. 64 | -------------------------------------------------------------------------------- /Salary Regression/Salary_Data.csv: -------------------------------------------------------------------------------- 1 | YearsExperience,Salary 2 | 1.1,39343.00 3 | 1.3,46205.00 4 | 1.5,37731.00 5 | 2.0,43525.00 6 | 2.2,39891.00 7 | 2.9,56642.00 8 | 3.0,60150.00 9 | 3.2,54445.00 10 | 3.2,64445.00 11 | 3.7,57189.00 12 | 3.9,63218.00 13 | 4.0,55794.00 14 | 4.0,56957.00 15 | 4.1,57081.00 16 | 4.5,61111.00 17 | 4.9,67938.00 18 | 5.1,66029.00 19 | 5.3,83088.00 20 | 5.9,81363.00 21 | 6.0,93940.00 22 | 6.8,91738.00 23 | 7.1,98273.00 24 | 7.9,101302.00 25 | 8.2,113812.00 26 | 8.7,109431.00 27 | 9.0,105582.00 28 | 9.5,116969.00 29 | 9.6,112635.00 30 | 10.3,122391.00 31 | 10.5,121872.00 32 | -------------------------------------------------------------------------------- /Salary Regression/__pycache__/linear_regression_numpy.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucko515/regression-python/f82b63317619af4b5d18958450af912d75b46342/Salary Regression/__pycache__/linear_regression_numpy.cpython-35.pyc -------------------------------------------------------------------------------- /Salary Regression/linear_regression_numpy.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | class LinearRegressioN(object): 4 | 5 | #in constructor - defining all hyperparameter 6 | def __init__(self): 7 | self.learning_rate = 0.0001 8 | #for accurecy play with how many iterations our gradient descent will run 9 | #100k = same output as in sklearn 10 | self.no_of_iter = 100000 11 | self.starting_b = 0 12 | self.starting_m = 0 13 | 14 | def train(self, X, y): 15 | #y = m*x + b 16 | self.X_train = X 17 | self.y_train = y 18 | 19 | #calling gradient descent function, and output of it is going to be our the best possible (according to our dataset) M and B 20 | self.new_b, self.new_m = self.gradient_descent(self.starting_b, self.starting_m, self.X_train, self.y_train, self.learning_rate, self.no_of_iter) 21 | 22 | return self.new_b, self.new_m 23 | 24 | #main function for gradient descent 25 | #INPUTS: STARTING b and m (alwyas start with b=0 and m=0) 26 | # X and y - Our training features and labels 27 | # learn_rate - is another hyperparam which will define "how fast" are it is going to find good B and M 28 | # no_of_iter - how many times our algorith is going to run 29 | def gradient_descent(self, b, m, X, y, learn_rate, no_of_iter): 30 | bG = b 31 | mG = m 32 | 33 | for i in range(no_of_iter): 34 | bG, mG = self.gradient_descent_step(bG, mG, X, y, learn_rate) 35 | 36 | return bG, mG 37 | #helper function for Gradient descent 38 | # current m and b - for each iteration its going to use that B and M 39 | # X, y - training features and labels 40 | # learn_rate - how fast it is going to move and to find the best possible M and B 41 | def gradient_descent_step(self, b, m, X, y, learn_rate): 42 | gradient_b = 0 43 | gradient_m = 0 44 | 45 | for i in range(len(X)): 46 | gradient_b += -(2/float(len(X)) * (y[i] - ((m*X[i]) + b))) 47 | gradient_m += (2/float(len(X))) * (-X[i]) * (y[i] - ((m*X[i]) + b)) 48 | 49 | new_b = b - (learn_rate * gradient_b) 50 | new_m = m - (learn_rate * gradient_m) 51 | return new_b, new_m 52 | 53 | #predict function to predict labels (y - values) for TEST set 54 | def predict(self, X): 55 | y_pred = [] 56 | for i in range(len(X)): 57 | #y = m*x + b 58 | y_pred.append(self.new_m*X[i] + self.new_b) 59 | 60 | return y_pred -------------------------------------------------------------------------------- /Salary Regression/salary_regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "The dataset used in this notebook can be found in the folder of the project." 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "collapsed": true 15 | }, 16 | "outputs": [], 17 | "source": [ 18 | "import matplotlib.pyplot as plt\n", 19 | "import numpy as np\n", 20 | "import pandas as pd\n", 21 | "%matplotlib inline" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 2, 27 | "metadata": { 28 | "collapsed": true 29 | }, 30 | "outputs": [], 31 | "source": [ 32 | "salary_data = pd.read_csv(\"Salary_Data.csv\")" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 3, 38 | "metadata": { 39 | "collapsed": false 40 | }, 41 | "outputs": [ 42 | { 43 | "data": { 44 | "text/html": [ 45 | "
\n", 46 | "\n", 47 | " \n", 48 | " \n", 49 | " \n", 50 | " \n", 51 | " \n", 52 | " \n", 53 | " \n", 54 | " \n", 55 | " \n", 56 | " \n", 57 | " \n", 58 | " \n", 59 | " \n", 60 | " \n", 61 | " \n", 62 | " \n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | "
YearsExperienceSalary
01.139343.0
11.346205.0
21.537731.0
32.043525.0
42.239891.0
52.956642.0
63.060150.0
73.254445.0
83.264445.0
93.757189.0
103.963218.0
114.055794.0
124.056957.0
134.157081.0
144.561111.0
154.967938.0
165.166029.0
175.383088.0
185.981363.0
196.093940.0
206.891738.0
217.198273.0
227.9101302.0
238.2113812.0
248.7109431.0
259.0105582.0
269.5116969.0
279.6112635.0
2810.3122391.0
2910.5121872.0
\n", 207 | "
" 208 | ], 209 | "text/plain": [ 210 | " YearsExperience Salary\n", 211 | "0 1.1 39343.0\n", 212 | "1 1.3 46205.0\n", 213 | "2 1.5 37731.0\n", 214 | "3 2.0 43525.0\n", 215 | "4 2.2 39891.0\n", 216 | "5 2.9 56642.0\n", 217 | "6 3.0 60150.0\n", 218 | "7 3.2 54445.0\n", 219 | "8 3.2 64445.0\n", 220 | "9 3.7 57189.0\n", 221 | "10 3.9 63218.0\n", 222 | "11 4.0 55794.0\n", 223 | "12 4.0 56957.0\n", 224 | "13 4.1 57081.0\n", 225 | "14 4.5 61111.0\n", 226 | "15 4.9 67938.0\n", 227 | "16 5.1 66029.0\n", 228 | "17 5.3 83088.0\n", 229 | "18 5.9 81363.0\n", 230 | "19 6.0 93940.0\n", 231 | "20 6.8 91738.0\n", 232 | "21 7.1 98273.0\n", 233 | "22 7.9 101302.0\n", 234 | "23 8.2 113812.0\n", 235 | "24 8.7 109431.0\n", 236 | "25 9.0 105582.0\n", 237 | "26 9.5 116969.0\n", 238 | "27 9.6 112635.0\n", 239 | "28 10.3 122391.0\n", 240 | "29 10.5 121872.0" 241 | ] 242 | }, 243 | "execution_count": 3, 244 | "metadata": {}, 245 | "output_type": "execute_result" 246 | } 247 | ], 248 | "source": [ 249 | "salary_data" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "As you can see we have 30 employees in the dataset with different number of years of experience. We are trying to make regressor so we can easily predict/decide on salary for totally new employee." 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "metadata": {}, 262 | "source": [ 263 | "## Preprocessing\n", 264 | "\n", 265 | " 1. We will need to separate data into Features and labels\n", 266 | " 2. Create training and testing parts of the dataset\n", 267 | " \n", 268 | " In this situation we wont need to scale data because we have just one feature." 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": 4, 274 | "metadata": { 275 | "collapsed": true 276 | }, 277 | "outputs": [], 278 | "source": [ 279 | "X = salary_data.iloc[:, :-1].values\n", 280 | "y = salary_data.iloc[:, -1].values" 281 | ] 282 | }, 283 | { 284 | "cell_type": "markdown", 285 | "metadata": {}, 286 | "source": [ 287 | "### 2. Train and test set" 288 | ] 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": 6, 293 | "metadata": { 294 | "collapsed": false 295 | }, 296 | "outputs": [], 297 | "source": [ 298 | "from sklearn.cross_validation import train_test_split" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": 7, 304 | "metadata": { 305 | "collapsed": true 306 | }, 307 | "outputs": [], 308 | "source": [ 309 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "## Regression part" 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": 9, 322 | "metadata": { 323 | "collapsed": false 324 | }, 325 | "outputs": [], 326 | "source": [ 327 | "from sklearn.linear_model import LinearRegression, Ridge" 328 | ] 329 | }, 330 | { 331 | "cell_type": "code", 332 | "execution_count": 10, 333 | "metadata": { 334 | "collapsed": false 335 | }, 336 | "outputs": [ 337 | { 338 | "data": { 339 | "text/plain": [ 340 | "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)" 341 | ] 342 | }, 343 | "execution_count": 10, 344 | "metadata": {}, 345 | "output_type": "execute_result" 346 | } 347 | ], 348 | "source": [ 349 | "lRegressor = LinearRegression()\n", 350 | "lRegressor.fit(X_train, y_train)" 351 | ] 352 | }, 353 | { 354 | "cell_type": "code", 355 | "execution_count": 11, 356 | "metadata": { 357 | "collapsed": false 358 | }, 359 | "outputs": [ 360 | { 361 | "data": { 362 | "text/plain": [ 363 | "Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,\n", 364 | " normalize=False, random_state=None, solver='auto', tol=0.001)" 365 | ] 366 | }, 367 | "execution_count": 11, 368 | "metadata": {}, 369 | "output_type": "execute_result" 370 | } 371 | ], 372 | "source": [ 373 | "rig = Ridge(alpha=1.0)\n", 374 | "rig.fit(X_train, y_train)" 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": 13, 380 | "metadata": { 381 | "collapsed": true 382 | }, 383 | "outputs": [], 384 | "source": [ 385 | "pred_1 = lRegressor.predict(X_test)\n", 386 | "pred_2 = rig.predict(X_test)" 387 | ] 388 | }, 389 | { 390 | "cell_type": "code", 391 | "execution_count": 14, 392 | "metadata": { 393 | "collapsed": false 394 | }, 395 | "outputs": [ 396 | { 397 | "name": "stdout", 398 | "output_type": "stream", 399 | "text": [ 400 | "R^2 score for Linear Regression: 0.925437927773\n" 401 | ] 402 | } 403 | ], 404 | "source": [ 405 | "print(\"R^2 score for Linear Regression: \", lRegressor.score(X_test, y_test))" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": 15, 411 | "metadata": { 412 | "collapsed": false 413 | }, 414 | "outputs": [ 415 | { 416 | "name": "stdout", 417 | "output_type": "stream", 418 | "text": [ 419 | "R^2 score for Ridge regression: 0.925002520196\n" 420 | ] 421 | } 422 | ], 423 | "source": [ 424 | "print(\"R^2 score for Ridge regression: \", rig.score(X_test, y_test))" 425 | ] 426 | }, 427 | { 428 | "cell_type": "code", 429 | "execution_count": 30, 430 | "metadata": { 431 | "collapsed": false 432 | }, 433 | "outputs": [ 434 | { 435 | "data": { 436 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuclWW9///XW0CZ8TB4joMMlmzaJhlttplWWphoSZLb\nnQd2opn8LMtD5elH7TQlMMpD27QmNbXwnOGhdmRo2dZthuEWTygpgwweEBlQGWWAz/eP+xpmzTDn\nmbXWrJn38/FYj7mv6z591hpYn7nu+7quWxGBmZlZPm1V7ADMzKzvc7IxM7O8c7IxM7O8c7IxM7O8\nc7IxM7O8c7IxM7O8c7KxvJD0cUmLix1HXyBppKS3JA0odixmXeVkY90iaamkQ5rXR8RfImJMMWJq\nTtIFkurTF3atpIclfbTYcXVURCyLiO0iYmOhzy3pT5LeSZ/d65LulDS00HFY6XOysT5F0sBWVt0a\nEdsBuwAPALcX+Pyl7Gvps9sL2A74YT5OUuzPrtjn7+ucbCwvJB0saXlOeamkb0l6QtIaSbdKGpyz\n/ghJj+e0PD6Ys+48Sf+Q9KakpyV9PmfdiZIeknSZpFXABW3FFREbgDnAcEm7dvD8H5a0MJ3/9hT7\nxbnvU9K5kl4BftGB450rqSYdb7GkCal+P0kLJK2V9KqkS1P9KEnR8GUoaZikuyW9IWmJpFNyjn2B\npNsk3ZiO/5Sk8R3+xbX92dUCc4EP5Zxvq5zfz6p07p1y1p8gqTqt+05uSzjFeoekX0laC5zY1vEk\nDU7brkqf698k7Z7WnSjphfSeX5Q0JSe+b6cYXkufS0Wzz/VkScuA+3vic7JWRIRffnX5BSwFDmmh\n/mBgebPtHgWGATsBzwCnpnXjgNeAjwADgKlp+23S+n9P+20FHAO8DQxN604ENgBfBwYCZS3EcgHw\nq7S8NTALeB0Y2N750/bVwBnAIOAoYD1wcc773ABckrYva+d4Y4CXgGFp/1HA+9Ly/wJfTMvbAfvn\nbBM58T4IXAUMJvviXwl8Kue9vgN8Jp17JvBIN36/fwK+nJZ3Bv4I3JWz/gzgEWBEen8/A25O6/YG\n3gI+lj7HHwL1pH8vKdZ6YHL63Za1c7z/D7gHKE/v7V+AHYBtgbXAmLTdUOADaflLwBLgvekzvRP4\nZbPP9cZ0jC3+7fjVg98VxQ7Ar9J+0blk8x855R8AP03LVwMXNdt/MXBQK+d8HDgyLZ8ILGsnxgvI\nEkQtsBFYBRycs77V8wOfAGoA5az7H5omm/XA4A4eby+yRHQIMKjZNg8CFwK7NKtv+FIcCOyR3sP2\nOetnAtfnvNc/5qzbG6jrxu/3T8A6YE2K4XFgZM76Z4AJOeWhZAlkIPCfpESR1pWnzyo32TzY7Hxt\nHe9LwMPAB5vts2363f5b84QBzAe+mlMek3O8hs/1vcX+f9QfXr6MZoX0Ss7yOrK/NAEqgW+mSyO1\nkmrJvlSHweZLMY/nrNuH7N5Lg5c6cO7bImIIsDvwJNlfxQ3aOv8woCbSN1Ur51sZEe905HgRsQQ4\nk+yL9jVJt0galvY7Gfgn4Nl0ieiIFt7HMOCNiHgzp64aGJ5Tbv45D1YL9yMk/f/Kbvy/JemnLZyr\nwekRUQF8ENiRrNWR+15/k/M+nyFLhrunWDd/VhGxjizR52r+WbZ1vF8C84BbJK2Q9ANJgyLibbIW\n76nAy5J+K+n96XjD0ufToJos0ezeRgyWB0421hu8BMyIiCE5r/KIuFlSJfBz4GvAzilhPAkoZ/8O\nT10eEa8D04AL1NirqtXzAy+T3d/JPd8ezQ/b0feTYrgpIj5G9sUaZJfgiIjnI+I4YLdUd4ekbZsd\newWwk6Ttc+pGkrW+OiUivh9ZL7ftIuLUDmy/CLgY+EnO5/EScHiz9zo4ImrIPrvNiUlSGdmluCaH\nbVZu9XgRUR8RF0bE3sABwBHACSm2eRHxabKW0LNk/2Yg+7wqc44/kuyy56ttxGB54GRjPWFQunnb\n8Opsr56fA6dK+ogy20r6bPpC3Zbsy2AlgKSTyFo2XRYRi8n+Qj6nA+f/X7K/rL8maaCkI4H9uvp+\nJI2R9ClJ25DdW6kDNqX39h+Sdo2ITWSXhWhYlxP7S2SXkmamz/qDZC2iX3XnM+mEG8haBZ9L5Z8C\nM9IfBUjaNX1GAHcAkyQdIGlrstacaFurx5P0SUljlY03Wkt2OWyTpN0lHZkS87tk94kaPrebgbMk\n7SlpO+D7ZD0TN3Tzc7BOcrKxnvA7si/NhtcFndk5IhYApwBXAqvJbuiemNY9DfyI7Ev/VWAs8FAP\nxDwbmCZpt3bOv56sU8DJZAngP4B7yb7UOv1+yG56N3RQeIWsFXN+WncY8JSkt4ArgGMjoq6FUxxH\ndr9hBfAb4LsR8cdOvv8uSZ/HFcB3UtUVwN3AHyS9SXZz/yNp26fIOm7cQtbKeYvsflWrn11bxwPe\nQ5bA1pJdXvsz2aW1rYBvkH0eb5DdG/tK2ue6tM2DwItkCf7rXX3/1nVqeinazNoj6a9knRt+UexY\nSklqWdQCoyPixWLHY4Xllo1ZOyQdJOk96TLaVLIb5b8vdlylQNIkSeXpEtcPgUVkPROtn3GyMWvf\nGOD/yP4q/yZwdES8XNyQSsaRZJe3VgCjyS4N+nJKP+TLaGZmlndu2ZiZWd554rlkl112iVGjRhU7\nDDOzkvLYY4+9HhG7tredk00yatQoFixYUOwwzMxKiqTq9rfyZTQzMysAJxszM8s7JxszM8s7Jxsz\nM8s7JxszM8s7JxszM8s7JxszM8s7j7MxM+uj5i6sYfa8xayorWPYkDLOnjiGyeOGt79jHjjZmJn1\nQXMX1nD+nYuoq98IQE1tHeffuWjz+kInIScbM7M+aPa8xZsTTYO6+o1ceM9TvFO/qcUklM+E43s2\nZmZ90Iralh7yCqvX1beYhGbPW5zXeJxszMz6oGFDyjq1fWvJqafkLdlIuk7Sa5KezKmbLelZSU9I\n+o2kITnrzpe0RNJiSRNz6g9LdUsknZdTv6ekv6b6WyVtneq3SeUlaf2ofL1HM7Pe6uyJYygbNKBJ\nXdmgAQwpG9Ti9p1NTp2Vz5bN9cBhzeruA/aJiA8CzwHnA0jaGzgW+EDa5ypJAyQNAH4CHA7sDRyX\ntgW4BLgsIvYCVgMnp/qTgdWp/rK0nZlZvzJ53HBmHjWW4UPKEDB8SBkzjxrLBZ/7QItJ6OyJY/Ia\nT946CETEg81bFRHxh5ziI8DRaflI4JaIeBd4UdISYL+0bklEvAAg6RbgSEnPAJ8Cjk/b3ABcAFyd\njnVBqr8DuFKS/ChaM+tvJo8b3upN//7UG+1LwK1peThZ8mmwPNUBvNSs/iPAzkBtRGxoYfvhDftE\nxAZJa9L2rzcPQNI0YBrAyJEju/l2zMxKQ1tJKF+K0kFA0nRgAzCnGOdvEBFVETE+Isbvumu7D5oz\nM7MuKnjLRtKJwBHAhJxLWzXAHjmbjUh1tFK/ChgiaWBq3eRu33Cs5ZIGAhVpezMzK5KCtmwkHQac\nA3wuItblrLobODb1JNsTGA08CvwNGJ16nm1N1ong7pSkHqDxns9U4K6cY01Ny0cD9/t+jZlZceWt\nZSPpZuBgYBdJy4HvkvU+2wa4TxLAIxFxakQ8Jek24Gmyy2unRcTGdJyvAfOAAcB1EfFUOsW5wC2S\nLgYWAtem+muBX6ZOBm+QJSgzs16jN81ZVijyH/2Z8ePHx4IFC4odhpn1cc3nLIOs6/HMo8aWZMKR\n9FhEjG9vO88gYGZWQK3NWZbv6WKKzcnGzKyAWpsWJt/TxRSbk42ZWQG1Ni1MvqeLKTYnGzOzAmpt\nzrJ8TxdTbH6ejZlZATV0AuhvvdGcbMzMCqwY08UUm5ONmfUZ/XH8SqlwsjGzPqH5+JVCPe7YOsbJ\nxsz6hLbGr/TmZNNfWmNONmbWJ5Ti+JX+1Bpz12cz6xNKcfxKf5pNwMnGzPqEUhy/Uoqtsa5ysjGz\nPmHyuOHMPGosw4eUIWD4kLJeP7llKbbGusr3bMyszyjo+JX77oNtt4UDDujyIc6eOKbFGaB7c2us\nq9yyMTPrjDffhFNPhUMPhe9/v1uHKsXWWFe5ZWNm1lHz58PJJ8OyZfCtb8H3vtftQ/aX2QTcsjEz\nS+YsmsOoy0ex1YVbMeryUcxZNCdb8dZbcNppcMghsPXW8D//A7NnQ1nfu7eSL27ZmJmRJZpp90xj\nXf06AKrXVDPtnmns9ren+fTFN8PSpXDWWXDxxVBeXtxgS5CTjZkZMH3+9M2JBqB8Pcz83To+/ej3\n4X3vgwcfhI99rIgRljYnGzMzYNmaZZuXP1YNv5gLe62GH38ETp//f1nPM+syJxszM2BkxUg2Lqvm\npcuy8gtD4KAToXrfSk53ouk2JxszM+Avd+/CHg9Uby5/8CsQ25VTNWFGEaPqO5xszKx/q6mBESPY\nIxUf3XMb9p+6npEVI5kxYQZTxk4panh9hZONmfVfxxwDt93WWH7uOfYbPZpNxYuoz3KyMbP+55VX\nYOjQxvL48fC3vxUvnn7AgzrNrH+ZOrVponnmmU4nmlYHf1qr3LIxs/5h5UrYbbfG8tix8MQTnT5M\na4M/Ad/faYNbNmbW951yStNEs2hRlxINbDn4E2Bd/Tqmz5/enQj7PLdszKzvWrUKdtmlsbzXXvD8\n8906ZO7gz47UW8YtGzPrm047rWmiWbiw24kGssGfnam3jJONmfUtq1eDBFddlZVHjoQI+NCHeuTw\nMybMoHxQ04k4yweVM8ODP9vkZGNmfcc3vgE77dRYXrAAqqtb374LpoydQtWkKiorKhGisqKSqklV\n7hzQDkVEsWPoFcaPHx8LFiwodhhm1hVr10JFRWN5113htdeKF08/IumxiBjf3nZu2ZhZaTv//KaJ\n5pFHnGh6IfdGM7OSMHdhDbPnLWZFbR3DhpRx3sdHMOljYxo32G47ePPN4gVobXLLxsx6vbkLazj/\nzkXU1NYRwBfu+XnTRPOXvzjR9HJu2ZhZrzd73mLq6jdStv4dnrns6M31G7YawMCNG4oYmXVU3lo2\nkq6T9JqkJ3PqdpJ0n6Tn088dU70k/VjSEklPSPpwzj5T0/bPS5qaU/8vkhalfX4sSW2dw8xK14ra\nOk57+NYmiebY477P6LPvKmJU1hn5vIx2PXBYs7rzgPkRMRqYn8oAhwOj02sacDVkiQP4LvARYD/g\nuznJ42rglJz9DmvnHGZWiurqePGSIzj7L7/cXDXq3Ht5ZOQHGTakrIiBWWfkLdlExIPAG82qjwRu\nSMs3AJNz6m+MzCPAEElDgYnAfRHxRkSsBu4DDkvrdoiIRyLru31js2O1dA4zKzWzZ0N54wDKKcdc\nzKhz7wWgbNAAzp44prU9rZcp9D2b3SPi5bT8CrB7Wh4OvJSz3fJU11b98hbq2zrHFiRNI2tJMXKk\np5ow6zXeeQfKmrZa5j72Ekv/8BxKvdHOnjiGyeOGt3IA622K1kEgIkJSXkeUtneOiKgCqiAb1JnP\nWMz6quZdkrudBC6/HM46q7H8u9/B4YczGZj84RHdjteKo9DJ5lVJQyPi5XQprGHkVQ1sfgQ4wIhU\nVwMc3Kz+T6l+RAvbt3UOM+thDV2S6+o3AlBTW8f5dy4C6HzCWb8ettmmad2mTdk8Z1byCj3O5m6g\noUfZVOCunPoTUq+0/YE16VLYPOBQSTumjgGHAvPSurWS9k+90E5odqyWzmFmPayhS3KuuvqNzJ63\nuHMHuuqqponm7ruzyTOdaPqMvLVsJN1M1irZRdJysl5ls4DbJJ0MVANfSJv/DvgMsARYB5wEEBFv\nSLoIaHhm6/cioqHTwVfJeryVAf+dXrRxDjPrYStq6zpVv4X6eth666Z1bs30SXlLNhFxXCurJrSw\nbQCntXKc64DrWqhfAOzTQv2qls5hZj1v2JAyalpILB3qknzNNdkTNBvceSd8/vM9GJ31Jp5BwMy6\n7OyJY5rcs4EOdEnesAEGDWpat3EjbOXZs/oy/3bNrMsmjxvOzKPGMnxIGQKGDylj5lFjW+8ccMMN\nTRPNLbdk92acaPo8t2zMrFsmjxvefs+zjRthYLOvmw0bYMCA/AVmvYr/nDCz/LrppqaJ5pe/zFoz\nTjT9ils2ZpYfmzZtmVDcmum33LIxK0FzF9Zw4Kz72fO833LgrPuZu7Cm/Z0K6fbbmyaV665za6af\nc8vGrMT06Kj9ntZSa6a+fsv7NdbvuGVjVmJ6bNR+T5s7t2mi+elPs9aME43hlo1Zyen2qP2e1lLX\n5Xff3XJmAOvX3LIxKzGtjc4vyoPE7r23aaL5r//Kko8TjTXjlo1ZienSqP0uaPPRAS21Zt55Z8tZ\nm80St2zMSkynR+13QUMnhJraOoLGTghzF9bAvHlNE82ll2bJx4nG2uCWjVkJ6tCo/W5osRPC+g1b\nPrysrg4GD85bHNZ3uGVjZlto3tngo9X/x9IfTGqsmDUra8040VgHtdmykbQb8HZEvC2pDPgGsD1w\nRXqAmZn1QbmPDlh6yRFNV779NpSXFyEqK2XttWxuAXZOyxcCewGrgZvyGZSZFdfZE8fw8RVPN0k0\nPz7oi8z9+3InGuuSVls2kqYC7wMOTo9ePgb4AfAWUCnpBODxiHiiIJGaWcFM/uj7mPzuu5vLEy68\nl69P+lDxZyiwktXWZbQ/AXXAM0AF8CpwDyCyp2r+CViT3/DMrKAefhgOPLCx/O1vw0UXMb94EVkf\n0WqyiYhqSVcA9wKDgBMiYpmkkcCqiFhWqCDNrAB23BFqaxvLa9fC9tsXLx7rU9q8ZxMRV5NdShsR\nEfem6lXAcfkOzMwK5NFHQWpMNOeck/U0c6KxHtTuOJuIeKtZ+e38hWNmBfWe98CrrzaWa2uhoqJ4\n8Vif5XE2Zv3R3/+etWYaEs0ZZ2StGScayxPPIGDW3+y5Jyxd2lhetQp22qlo4Vj/4JaNWX/xxBNZ\na6Yh0Zx6ataacaKxAmi3ZSPpKOASYDeybs8CIiJ2yHNsZtZT/vmf4dlnG8uvvw4779z69mY9rCMt\nmx8An4uIiojYISK2d6IxKxFPPZW1ZhoSzUknZa0ZJxorsI7cs3k1Ip7JeyRm1rPGjYPHH28sv/oq\n7LZb8eKxfq0jyWaBpFuBucDm+Ssi4s68RWVmXbd4Mbz//Y3l44+HOXOKF48ZHUs2OwDrgENz6gJw\nsjHrbT76UXjkkcbyihUwdGjx4jFLOjKo86RCBGJm3bBkCYwe3Vj+t3+DO+4oXjxmzbQ16/M5EfED\nSf9F1pJpIiJOz2tkZiVuzqI5TJ8/nWVrljGyYiQzJsxgytgpPX+igw6CBx9sLC9fDsM9O7P1Lm21\nbBo6BSwoRCBmfcmcRXOYds801tWvA6B6TTXT7pkG0HMJ58UX4b3vbSxPmgR3390zxzbrYYrYotHS\nL40fPz4WLHBetZ4x6vJRVK+p3qK+sqKSpWcu7f4JDjsM5s1rLFdXw8iR3T+uWSdJeiwixre3naer\nMcuDZWtafgJHa/UdP/AyqKxsLH/60/CHP3TvmGYF4OlqzPJgZEXLrYzW6jvi5YMnNkk0f7jnYSca\nKxlONmZ5MGPCDMoHlTepKx9UzowJMzp/sJoakBj65yyx/HWPfRh17r2c8dc1zF1Y0xPhmuWdk41Z\nHkwZO4WqSVVUVlQiRGVFJVWTqjrfOeALX4ARIzYXD5pWxTHHzwKgrn4js+ct7smwzfKmKPdsJJ0F\nfJmsS/Ui4CRgKHALsDPwGPDFiFgvaRvgRuBfyJ4SekxELE3HOR84GdgInB4R81L9YcAVwADgmoiY\nVbh3Z5aZMnZK13uevfJKk8GYC4eO4fMn/GiLzVbU1nU1PLOCardlI2lAT55Q0nDgdGB8ROxDlhCO\nJZtZ+rKI2AtYTZZESD9Xp/rL0nZI2jvt9wHgMOAqSQNSvD8BDgf2Bo5L25qVhi9+semo/2ee4Wun\nX9XipsOGlBUoKLPu6chltOclze7hL+yBQJmkgUA58DLwKaBhyPMNwOS0fGQqk9ZPkKRUf0tEvBsR\nLwJLgP3Sa0lEvBAR68laS0f2YOxm+bFyZTZD869+lZX33Tebofn97+fsiWMoG9T0776yQQM4e+KY\nIgRq1nkdSTb7As8B10h6RNI0SV1+xEBE1AA/BJaRJZk1ZJfNaiNiQ9psOdAwBHo48FLad0Pafufc\n+mb7tFa/hfReFkhasHLlyq6+JbPu+/KXm87IvGhRkxmbJ48bzsyjxjJ8SBkChg8pY+ZRY5k8zjMF\nWGnoyNxobwI/B34u6SDgJuAySXcAF0XEks6cUNKOZC2NPYFa4Hayy2AFFxFVQBVkgzqLEYP1c6tW\nwS67NJb/6Z+yWZtbMHnccCcXK1kdumcj6XOSfgNcDvwIeC9wD/C7LpzzEODFiFgZEfVks0cfCAxJ\nl9UARgANfTprgD1SLAOBCrKOApvrm+3TWr1Z7/LVrzZNNI8/3mqiMSt1HemN9jzwADA7Ih7Oqb9D\n0ie6cM5lwP6SyoE6YALZ/GsPAEeT3WOZCtyVtr87lf83rb8/IkLS3cBNki4FhgGjgUfJHls9WtKe\nZEnmWOD4LsRplh+rV8NOOzWWR47Mppsx68PabNmknl3XR8TJzRIN0LWZnyPir2Q3+v9O1u15K7JL\nWecC35C0hOyezLVpl2uBnVP9N4Dz0nGeAm4DngZ+D5wWERvTfZ2vAfPIJhO9LW1rVnxnndU00Tz2\nmBON9QvtTsQp6YGI+GSB4ikaT8RpebVmDQwZ0ljeffdsLI1ZievoRJwd6Y32sKQrJX1c0ocbXj0Q\no1n/cN55TRPNI4840Vi/05F7Ngekn9/LqQuycTFm1pqVK5t2Z95+e1i7tnjxmBVRR7o+9/lLaGY9\nbswYeO65xvJDD8EBB7S+vVkf16G50SR9lmxamMENdRHxvdb3MOun3ngDdt65aZ0fUGjWoXE2PwWO\nAb5O1q3434HKNncy64/23bdporn6aicas6RD92wi4oOSnoiICyX9iGwgppnBlj3NwEnGrJmO9EZr\nmMN8naRhQD3ZVDNmdvLJTRPN5Zc70Zi1oCMtm3slDQFmkw3EDOCavEZl1tutWwfbbtu0btOmbNZm\nM9tCuy2biLgoImoj4tdk92reHxHfyX9oZr3UV7/aNNFcf33WmnGiMWtVqy0bSUe1sY6I8H0b61/q\n6qC8vGmdWzNmHdLWZbRJbawL3EnA+pOzzsruxzSoqoJTTilePGYlptVkExEnFTIQs17p3Xdh8OCm\ndW7NmHVaR3qjIemzks6R9J8Nr3wHZlZ055/fNNFceaXvzZh1Ubu90dKgznLgk2S90I4me26MWd9U\nXw9bb920buNG2KpDf5uZWQs68r/ngIg4AVgdERcCH6XpkzDN+o4LLmiaaH70o6w140Rj1i0dGWfT\nfFDnKjyo0/qaDRtg0KAt6wYMKE48Zn1MR/5caz6ocylwcz6DMiuomTObJpqZM7PWjBONWY/pyCMG\nLkqLv5Z0LzA4ItbkNyyzAti4EQY2+y9QX79lnZl1W6stG0n/Kuk9OeUTgNuAiyTt1Np+ZiXh0kub\nJpULLshaM040ZnnR1v+snwGHAEj6BDCL7DEDHwKqyHqlmZWWTZu2vDy2fv2W92taMHdhDbPnLWZF\nbR3DhpRx9sQxTB43PE+BmvUtbd2zGRARb6TlY4CqiPh1mhdtr/yHZtbDrryyaaI577ysNdPBRHP+\nnYuoqa0jgJraOs6/cxFzF9bkL16zPqStls0ASQMjYgMwAZjWwf3MepeWui6/8w5ss02HDzF73mLq\n6jc2qaur38jseYvdujHrgLZaNjcDf5Z0F1n3578ASNoLcAcBKw1VVU0TzVlnZcmnE4kGYEVtXafq\nzayptuZGmyFpPjAU+EPE5idCbUV278as92qpNVNXt+U8Zx00bEgZNS0klmFDypizaA7T509n2Zpl\njKwYyYwJM5gydkqXzmPWV7U5ziYiHomI30TE2zl1z0XE3/MfmlkX3XBD00Tzla9kyaeLiQbg7Ilj\nKBvUtGNB2aABHLD360y7ZxrVa6oJguo11Uy7ZxpzFs3p8rnM+iLfe7G+o6XWzNtvb/kMmi5ouC/T\nvDfamX8+kHX165psu65+HdPnT3frxiyHk431DTffDMcf31j+0pfg2mt79BSTxw3fojPAUXcva3Hb\nZWtarjfrr5xsrEcVZSxK8yn/33wTttsuv+dMRlaMpHpNdYv1ZtbIU9lajyn4WJRf/7ppojnuuOxS\nWoESDcCMCTMoH9T0Ml35oHJmTJhRsBjMSoGTjfWYtsai9LhjjoGjcyaxWLMGbrqp58/Tjiljp1A1\nqYrKikqEqKyopGpSle/XmDXjy2jWY/I5FqXh8tzgfzzP/GtObVxx9NFw++3dPn53TBk7xcnFrB1O\nNtZj2hqL0h0Nl+cumvtDjn5yPgB1A7fhvgef5HMf9cxJZqXAl9Gsx7Q2FuXsiWO6ddyb5tzPMxcf\nvjnRnHHEN/nnb/6aS/7sHl9mpcItG+sxrY1F6VZvtFNO4bZrrgFgg7Zin7Nu451B2eBMTxVjVjqc\nbKxHtTQWpUteeAHe977NxW995kzuGHtIk026e3nOzArHycZ6n9NOg6uu2ly856Hn+O1//wNyerp1\n9/Kcn01jVlhONtZ7VFfDqFGN5Z//HL78ZSYBG8vKeyw5NHQ4aOim3TAeCHDCMcuToiQbSUOAa4B9\ngAC+BCwGbgVGAUuBL0TEakkCrgA+A6wDTmyYCFTSVODb6bAXR8QNqf5fgOuBMuB3wBk5s1ZbD+jx\nlsGZZ8IVVzSW166F7bffXOyxy3P42TRmxVCs3mhXAL+PiPcD+wLPAOcB8yNiNDA/lQEOB0an1zTg\nagBJOwHfBT4C7Ad8V9KOaZ+rgVNy9jusAO+p3+jRmQJeeimbBaAh0Vx9dTYLQE6i6Wl+No1Z4RU8\n2UiqAD4BXAsQEesjohY4ErghbXYDMDktHwncGJlHgCGShgITgfsi4o2IWA3cBxyW1u2QHo8QwI05\nx7Ie0GMzBZxzDozMmUOsthZOPbX17XtIax0L3OHALH+K0bLZE1gJ/ELSQknXSNoW2D0iXk7bvALs\nnpaHAy/Q+KyPAAAOqElEQVTl7L881bVVv7yF+i1ImiZpgaQFK1eu7Obb6j+63TJYsSJrzcyenZV/\n/OOsNVNR0UMRti1f44HMrHXFSDYDgQ8DV0fEOOBtGi+ZAZBaJHm/xxIRVRExPiLG77rrrvk+XZ/R\nrZbB9OkwPCf3r14NXy/sg18njxvOzKPGMnxIGQKGDylj5lFjfb/GLI+K0UFgObA8Iv6ayneQJZtX\nJQ2NiJfTpbDX0voaYI+c/Uekuhrg4Gb1f0r1I1rY3nrI2RPHNOnNBR1oGbzyCgwd2li+9FI466w8\nRtm2nuxwYGbtK3jLJiJeAV6S1PDNNAF4GrgbmJrqpgJ3peW7gROU2R9Yky63zQMOlbRj6hhwKDAv\nrVsraf/Uk+2EnGNZD+h0y+CCC5ommlWrippozKzwijXO5uvAHElbAy8AJ5ElvtsknQxUA19I2/6O\nrNvzErKuzycBRMQbki4C/pa2+15EvJGWv0pj1+f/Ti/rQR1qGbz2Guy+e2P5kkuyTgFt6GiX6tzt\nKsoGIUHtunoP0DTrpeThJ5nx48fHggULih1Gr9XpcTUzZsC3v91YXrkSdtml3XO0dHmueauppe1y\ntbSPmeWHpMciYnx723nWZ2tXp8bVvP561tOsIdFcfHHW06ydRAMd71Ld0nbt7WNmxeVkY+3q8Lia\nSy6B3F59r76a9T7roI52qe5IF2sP0DTrXZxsrF3tJoE33shaM+elHuzf/W7Wmtltt06dp6NdqivK\nW2/VtHcsMysOJxtrV5tJ4NJLYeedGytffjnrfdYFHR1sWTvwRjbxTqvH8QBNs97Hycba1VIS2G1j\nHQ+dPwG++c2sYvp0iGDuyxs5cNb97Hnebzlw1v2dmi+to12ql9XfyapB/8UGvUawiY2sYSNrCDZ5\ngKZZL+VHDFi7mj+B8/Snf89Z91zZuEFNDQwb1iNT93ekS/XIipFUr/kz6wb+uUl9ZUUlD525tIPv\nyswKyS0b65DJ44bz0FfH8+IlRzQmmm99K7s3M2wY0IMTdLZjxoQZlA8qb1JXPqicGRNm9Oh5zKzn\nONlYx1x9ddOJMpcta5xIMynU1P1Txk6halIVlRWVCFFZUUnVpCqmjJ3So+cxs57jy2jWtjffhB12\naCyfeSZcdlmLmw4bUkZNC4klHz3Dpoyd4uRiVkLcsrHWXXNN00SzdGmriQY8db+Ztc4tG9vS22/D\ndts1lk87Da68svXtk+YdCTxPmZk1cLKxpm64AU48sbH8wguw554d3t1T95tZS5xsLLNuHWy/PWza\nlJVPOQWqqoobk5n1Gb5nYzBnDmy7bWOief55Jxoz61Fu2fRn77wDO+0EdakH2dSpcP31RQ3JzPom\nt2z6q1tvhbKyxkTz7LNONGaWN27Z9DfvvpvNxrx2bVY+/vjsMpqZWR65ZdOf/PrXMHhwY6J5+mkn\nGjMrCLds+oP167P5y1atyspHHw23317cmMysX3HLpq+76y7YZpvGRPPEE040ZlZwbtn0VfX1UFmZ\nPcwM4Mgj4Te/yZ6oaWZWYG7Z9EX33w9bb92YaB5/HObOdaIxs6JxsumLZs3Kfn7mM9lAzX33LW48\nZtbv+TJaHs1dWFOcSSlvvz1LMjvumP9zmZl1gJNNnvTEI5K7LPchZ2ZmvYAvo+VJoR6RbGZWCpxs\n8qRQj0g2MysFTjZ50tqjkPPxiGQzs97OySZP/IhkM7NG7iCQJ/l+RHLRerqZmXWBk00e5esRyUXt\n6WZm1gW+jFaC3NPNzEqNk00Jck83Mys1TjYlyD3dzKzUONmUIPd0M7NS4w4CJSjfPd3MzHqak02J\nyldPNzOzfCjaZTRJAyQtlHRvKu8p6a+Slki6VdLWqX6bVF6S1o/KOcb5qX6xpIk59YeluiWSziv0\neyumOYvmMOryUWx14VaMunwUcxbNKXZIZmZFvWdzBvBMTvkS4LKI2AtYDZyc6k8GVqf6y9J2SNob\nOBb4AHAYcFVKYAOAnwCHA3sDx6Vt82buwhoOnHU/e573Ww6cdT9zF9bk83StmrNoDtPumUb1mmqC\noHpNNdPumeaEY2ZFV5RkI2kE8FngmlQW8CngjrTJDcDktHxkKpPWT0jbHwncEhHvRsSLwBJgv/Ra\nEhEvRMR64Ja0bV40DLCsqa0jaBxgWYyEM33+dNbVr2tSt65+HdPnTy94LGZmuYrVsrkcOAfYlMo7\nA7URsSGVlwMNNySGAy8BpPVr0vab65vt01r9FiRNk7RA0oKVK1d26Y30pgGWy9Ys61S9mVmhFDzZ\nSDoCeC0iHiv0uZuLiKqIGB8R43fdddcuHaM3DbAcWTGyU/VmZoVSjJbNgcDnJC0lu8T1KeAKYIik\nht5xI4CG61A1wB4AaX0FsCq3vtk+rdXnRVcHWObjPs+MCTMoH1TepK58UDkzJszo9rHNzLqj4Mkm\nIs6PiBERMYrsBv/9ETEFeAA4Om02FbgrLd+dyqT190dEpPpjU2+1PYHRwKPA34DRqXfb1ukcd+fr\n/XRlgGW+7vNMGTuFqklVVFZUIkRlRSVVk6qYMnZKt45rZtZdvWmczbnALZIuBhYC16b6a4FfSloC\nvEGWPIiIpyTdBjwNbABOi4iNAJK+BswDBgDXRcRT+Qq6KwMs27rP092xM1PGTnFyMbNeR1kjwcaP\nHx8LFiwoyLn2PO+3tPSpC3hx1mcLEoOZWU+Q9FhEjG9vO8+NVgSeSNPM+hsnmyLwRJpm1t/0pns2\n/YYn0jSz/sbJpkg8kaaZ9Se+jGZmZnnnZGNmZnnnZGNmZnnnZGNmZnnnZGNmZnnnZGNmZnnnZGNm\nZnnnudESSSuB6mLH0QW7AK8XO4gi8vv3+/f7L67KiGj3gWBONiVO0oKOTILXV/n9+/37/ZfG+/dl\nNDMzyzsnGzMzyzsnm9JXVewAiszvv3/z+y8RvmdjZmZ555aNmZnlnZONmZnlnZNNCZK0h6QHJD0t\n6SlJZxQ7pmKQNEDSQkn3FjuWQpM0RNIdkp6V9IykjxY7pkKSdFb6t/+kpJslDS52TPkm6TpJr0l6\nMqduJ0n3SXo+/dyxmDG2xcmmNG0AvhkRewP7A6dJ2rvIMRXDGcAzxQ6iSK4Afh8R7wf2pR99DpKG\nA6cD4yNiH2AAcGxxoyqI64HDmtWdB8yPiNHA/FTulZxsSlBEvBwRf0/Lb5J90fSrx35KGgF8Frim\n2LEUmqQK4BPAtQARsT4iaosbVcENBMokDQTKgRVFjifvIuJB4I1m1UcCN6TlG4DJBQ2qE5xsSpyk\nUcA44K/FjaTgLgfOATYVO5Ai2BNYCfwiXUa8RtK2xQ6qUCKiBvghsAx4GVgTEX8oblRFs3tEvJyW\nXwF2L2YwbXGyKWGStgN+DZwZEWuLHU+hSDoCeC0iHit2LEUyEPgwcHVEjAPephdfPulp6b7EkWRJ\ndxiwraT/KG5UxRfZOJZeO5bFyaZESRpElmjmRMSdxY6nwA4EPidpKXAL8ClJvypuSAW1HFgeEQ2t\n2TvIkk9/cQjwYkSsjIh64E7ggCLHVCyvShoKkH6+VuR4WuVkU4Ikiex6/TMRcWmx4ym0iDg/IkZE\nxCiyG8P3R0S/+cs2Il4BXpI0JlVNAJ4uYkiFtgzYX1J5+r8wgX7UQaKZu4GpaXkqcFcRY2mTk01p\nOhD4Itlf9I+n12eKHZQV1NeBOZKeAD4EfL/I8RRMatHdAfwdWET2PVYy07Z0laSbgf8FxkhaLulk\nYBbwaUnPk7X4ZhUzxrZ4uhozM8s7t2zMzCzvnGzMzCzvnGzMzCzvnGzMzCzvnGzMzCzvnGysX1Lm\nfyQdnlP375J+X8y48kXSRyRdVuw4rP9y12frtyTtA9xONrfcQGAhcFhE/KMbxxwYERt6KMQe0Rtj\nsv7HLRvrtyLiSeAe4FzgP4EbI+IfkqZKejQNlr1K0lYAkqokLUjPUfnPhuOkAXbfkfQQ8Pn0rJWn\nJT3R2jQ6kv5V0p8lPSbpvyXtLmlQKn8sbTNb0oU555iV4vqrpPem+t0l3ZnielTS/qn+Ykk/k3Qf\n2YSdh0iam9ZtJ+n6tP1CSZNS/ZfTM3LmpeejzMyJ97OS/i7p/yT9oa3jmLUoIvzyq9++gG2BxWQj\n0bcB9gHmAgPT+irg+LS8U/o5EPgLsHcqLwe+kXPMl4Gt0/KQFs65DfAwsEsqTwGq0vIHyaaeORRY\nAAzKOce5aflLwNy0fCuwf1oeBTyZli8GHgUGp/IhOfv8ADg2Le8IPAcMBr4MPA/sAJQBL5FNdPke\nsiliKpt9Di0ep9i/U79652tgJ3OTWZ8SEW9LuhV4KyLelXQI8K/Agmzarc1fugDHpSlCBpJ9Ce9N\n45xkt+Yc9ingV5LuIktczf0z8AHgj+kcA8iSCRHxhKRbyOa4+khkE002uDn9nEPjtCSHkE1f0rDN\njpLK0vJdEfFOC+c/FDhcUsNM0YOBkWn5j5FmEJf0bKofCjwQEdUpxjfaOc5zLZzT+jknG7PsmTgN\nz8URcF1EfCd3A0mjyZ4Mul9E1KbLY7mPIn47Z3kicBDZNPjfTveG5gG7AI8APwOeiIiPtxLPPsAa\nYLdm9S3dYFWKaX2zeJvH1HyfydHs3pSkTwDv5lRtpO3viBaPY9YS37Mxa+qPwBck7QIgaWdJI8ku\nLb0JrE1TuU9saWdJA4AREXE/cDawK1AeEYdExIci4lSy1tBwSfulfbaW9IG0fAywHXAw8BNJO+Qc\n/pj08zjgoZx4T8s5/4c68B7nkU3k2bDPuHa2fxj4pKTKtP1OXTyO9WNu2ZjliIhF6ab8H1PHgHrg\nVLL7J08DTwIv0Phl39xA4CZJ25P9MXdJZI/uzj3Hu5KOBn6ckskA4EeSVgIXAQdHxApJPwMuA05O\nu5ZLepSshXNcqjsNuFrSSencD5CTfFpxIXC5pIYZk5eQtcJa+0xelfQV4K40pf8K4PDOHsf6N3d9\nNisBkpYD+0REbbFjMesKX0YzM7O8c8vGzMzyzi0bMzPLOycbMzPLOycbMzPLOycbMzPLOycbMzPL\nu/8HY3rjaSFMGIEAAAAASUVORK5CYII=\n", 437 | "text/plain": [ 438 | "" 439 | ] 440 | }, 441 | "metadata": {}, 442 | "output_type": "display_data" 443 | }, 444 | { 445 | "data": { 446 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYVdWZ9v/vLaApHCgVolIImNYmrZIEJWrHztDBCEZR\n4s84hChJ203bbQaTvE5NBk1Cqy/pOHQbO7SaaIJTjEFEE2I7ZXAsID9xaCKdAIITiiAKQonP+8de\nhzpV1nCq6pyz61Tdn+s6V+299vSciqmHZ++11lZEYGZmVknb5R2AmZn1fU42ZmZWcU42ZmZWcU42\nZmZWcU42ZmZWcU42ZmZWcU421u9I+k9J3+hge0jat5oxmfV1TjbW50haLmmTpNclvSDpx5J2KmyP\niDMi4jt5xtgWSRdIakpxr5P0oKS/zjsus3JwsrG+anJE7AR8ABgHnJ9zPKW6OcU9FLgP+FklLiJp\nYCXOWyvXt+pzsrE+LSJeABaQJR0AUqXz3aL1syU9L+k5SX9XfLyk3SXdIek1SY9J+q6k3xVtf6+k\nuyWtlbRU0ollivstYA7QIGlY0fWOkfSHosrnfUXbDpK0WNIGST+TdHPhe0r6mKRVks6V9ALwoxLO\nd66k1el8SyVNSO2HSGpMv5MXJX2/6JhjJT2Zzne/pL8q2rY8nfNx4A0nnP7Fycb6NEkjgKOAZe1s\nnwT8H+ATwH7AEa12uRJ4A9gTmJY+hWN3BO4GbgDeDZwM/EDS/mWIe3vgNOAV4NXUNg64FvhHYHfg\nh8A8STuk/X8B/BjYDbgR+FSr0+6Zto0CpndyvjHAF4APRsTOwERgeTrP5cDlEbEL8BfALSm+v0zX\nPQsYBtwF3JFiKzgFOBqoTwnV+gknG+ur5kraADwLvAR8q539TgR+FBFPRMQbwAWFDZIGAP8f8K2I\n2BgRTwHXFR17DLA8In4UEW9FxGLg58CnexD3iZLWAZuAfwBOKPqjPB34YUQ8EhFbI+I6YDNwWPoM\nBK6IiKaIuA14tNW5307fZXNEbOrkfFuBHYD9JQ2KiOUR8b/pPE3AvpKGRsTrEfFwaj8JuDMi7o6I\nJuB7QB3woaIYroiIZ9P1rR9xsrG+akr6F/nHgPeSPQNpy3CyhFSwomh5GNkf8OLtxcujgEPTLaN1\nKUlMJasgWpD04fTg/3VJT3YQ9y0RUQ/sATwBHNzqel9rdb2903cYDqyOljPrFscKsCYi3izlfBGx\njKxCuQB4SdJNkoan404H/hL4n3Rr8ZjUPpyi319EvJ1iaOggJusnnGysT4uIB8huLX2vnV2eJ/sD\nWzCyaHkN8BYwoqiteN9ngQcior7os1NE/FMbcfw2bdspIg4oIe6XySqPCyTtVXS9ma2uNzgibkzf\no0GS2okVoPUU7x2dj4i4ISL+hiwpBXBJan8mIk4hu3V4CXBruqX4XNoXgBTL3sDqDmKwfsLJxvqD\ny4BPSHp/G9tuAT4naX9Jgym63RYRW4HbyP7gD5b0XrLnKAXzgb+UdKqkQenzweKH4j0REUvJOjec\nk5r+CzhD0qHK7CjpaEk7Aw+R3fr6gqSBko4DDunkEu2eT9IYSR+XtAPwJtltvbcBJH1W0rBUuaxL\n53qb7Hd5tKQJkgYBXyO7LfdgOX4fVtucbKzPi4g1wPXAN9vY9kuyZHQvWSeCe1vt8gVgCPAC8BOy\nB+Cb07EbgCPJOgY8l/a5hOxZR7nMInuY/+6IaCR7jvMfZJ0GlgGfS7FsAY4nu8W1DvgsWTLc3N6J\nOzpf+g4XAy+n7/VumruPTwKelPQ6WWeBkyNiU0qOnwX+PR03mawL+pae/hKs9skvTzMrnaRLgD0j\nYlqnO+dM0iPAf0bEj/KOxcyVjVkH0jia96XbTIeQVQ6/yDuutkj6qKQ90220acD7gF/lHZcZZD1t\nzKx9O5PdOhsOvAj8G3B7rhG1bwzZc5MdgT+RdZt+Pt+QzDK+jWZmZhXn22hmZlZxvo2WDB06NEaP\nHp13GGZmNWXhwoUvR8SwzvZzsklGjx5NY2Nj3mGYmdUUSSs638u30czMrAqcbMzMrOKcbMzMrOKc\nbMzMrOKcbMzMrOKcbMzMrOKcbMzMrOI8zsbMrI+au3g1sxYs5bl1mxheX8fZE8cwZVxD5wdWgJON\nmVkfNHfxas6/bQmbmrYCsHrdJs6/bcm27dVOQk42ZmZ90KwFS7clmoJNTVu58I4nebPp7TaTUCUT\njp/ZmJn1Qc+t29Rm+6sbm9pMQrMWLK1oPE42ZmZ90PD6ui7t315yKpeKJRtJ10p6SdITRW2zJP2P\npMcl/UJSfdG28yUtk7RU0sSi9kmpbZmk84ra95H0SGq/WdL2qX2HtL4sbR9dqe9oZtZbnT1xDHWD\nBrRoqxs0gPq6QW3u39Xk1FWVrGx+DExq1XY3cGBEvA/4I3A+gKT9gZOBA9IxP5A0QNIA4ErgKGB/\n4JS0L8AlwKURsS/wKtnrekk/X03tl6b9zMz6lSnjGrjo+LE01NchoKG+jouOH8sFxx7QZhI6e+KY\nisZTsQ4CEfGb1lVFRPy6aPVh4IS0fBxwU0RsBv4saRlwSNq2LCL+BCDpJuA4SU8DHwc+k/a5DrgA\nuCqd64LUfivwH5IUfiWpmfUzU8Y1tPvQvz/1Rvs74Oa03ECWfApWpTaAZ1u1HwrsDqyLiLfa2L+h\ncExEvCVpfdr/5dYBSJoOTAcYOXJkD7+OmVlt6CgJVUouHQQkzQDeAubkcf2CiJgdEeMjYvywYZ2+\naM7MzLqp6pWNpM8BxwATim5trQb2LtptRGqjnfZXgHpJA1N1U7x/4VyrJA0EhqT9zcwsJ1WtbCRN\nAs4Bjo2IjUWb5gEnp55k+wD7AY8CjwH7pZ5n25N1IpiXktR9ND/zmQbcXnSuaWn5BOBeP68xM8tX\nxSobSTcCHwOGSloFfIus99kOwN2SAB6OiDMi4klJtwBPkd1eOzMitqbzfAFYAAwAro2IJ9MlzgVu\nkvRdYDFwTWq/BvhJ6mSwlixBmZn1Gr1pzrJqkf/Rnxk/fnw0NjbmHYaZ9XGt5yyDrOvxRcePrcmE\nI2lhRIzvbD/PIGBmVkXtzVlW6eli8uZkY2ZWRe1NC1Pp6WLy5mRjZlZF7U0LU+npYvLmZGNmVkXt\nzVlW6eli8ub32ZiZVVGhE0B/643mZGNmVmV5TBeTNycbM+sz+uP4lVrhZGNmfULr8SvVet2xlcbJ\nxsz6hI7Gr/TmZNNfqjEnGzPrE2px/Ep/qsbc9dnM+oRaHL/Sn2YTcLIxsz6hFsev1GI11l1ONmbW\nJ0wZ18BFx4+lob4OAQ31db1+cstarMa6y89szKzPqOr4lc2bYcsW2Hnnbp/i7Ilj2pwBujdXY93l\nysbMrKsaG+Hgg+GLX+zRaWqxGusuVzZmZqXavBm+/W245BLYYw848cQen7K/zCbgysbMLJmzZA6j\nLxvNdhdux+jLRjNnyZzmjYVq5l//FU49FZ58Ej75yfyCrTGubMzMyBLN9Dums7FpIwAr1q9g+h3T\n2W5LE6fc9kxzNXPnnU4y3eBkY2YGzLhnxrZEU/BXyzcybvJ0eL4JPvc5+P73Yddd8wmwxjnZmJkB\nK9ev3La8/VvwzQfg3N/Bizs1wfz5cPTROUZX+5xszMyAkUNGsmL9Ci6/C770aNb2ow/ApZ8eweNO\nND3mZGNmBsw6+Dw+/ZF/2rZ+zClw34GDmT354hyj6jvcG83M7IwzWiSag/4RnjhkFLMnz2bq2Kk5\nBtZ3uLIxs/7rlVdg6NDm9VGjYPlyFuUXUZ/lysbM+qczzmiZaBYtguXLcwunr3OyMbP+Ze1akOCH\nP8zWR46ECBg3ruRTdDj409rkZGNm/cc//zPsvnvz+qJFsGJFl05RGPy5Yv0Kgtg2+NMJp2NONmbW\n9xWqmauuytb33rvL1UxBW4M/NzZtZMY9M8oRaZ/lZGNmfVtb1czKle3v34niwZ+ltFvGvdHMrG9a\nu7Zlktl77x4lmYLC4M+22q19rmzMrO8588yWiWbhwrIkGoCZE2YyeNDgFm2DBw1m5oSZZTl/X+XK\nxsz6jtbVzIgR8OyzZb1EYZDnjHtmsHL9SkYOGcnMCTM9+LMTioi8Y+gVxo8fH42NjXmHYWbddeaZ\n8IMfNK8X3j9jFSVpYUSM72w/VzZmVttaVzMNDbBqVX7xWJucbMysJsxdvJpZC5by3LpNDK+v4+yJ\nY5hyzUVw5ZXNO7ma6bWcbMys15u7eDXn37aETU1bAdjwwhqmHDSheYfhw2H16pyis1K4N5qZ9Xqz\nFizdlmguuPs/efzyk5s3NjY60dSAiiUbSddKeknSE0Vtu0m6W9Iz6eeuqV2SrpC0TNLjkg4qOmZa\n2v8ZSdOK2g+WtCQdc4UkdXQNM6tdz63bxC5vvs7yS47hc4vmA/DiTruxz7nzfdusRlSysvkxMKlV\n23nAPRGxH3BPWgc4CtgvfaYDV0GWOIBvAYcChwDfKkoeVwH/UHTcpE6uYWY1atZvrm5RzUw+7VIO\nPfN6htfX5RiVdUXFkk1E/AZY26r5OOC6tHwdMKWo/frIPAzUS9oLmAjcHRFrI+JV4G5gUtq2S0Q8\nHFnf7etbnauta5hZrXn1VZA44aG5ALy0466MPnc+S/baj7pBAzh74picA7RSVfuZzR4R8XxafgHY\nIy03AMUjr1alto7aV7XR3tE13kHSdEmNkhrXrFnTja9jZhXzpS/BbrttW73/p3fyqa/fioCG+jou\nOn4sU8Y1tH+89Sq59UaLiJBU0RGlnV0jImYDsyEb1FnJWMz6qja7JPckCbz6aoskw557wvPP8zHg\n9z0N1nJT7crmxXQLjPTzpdS+Gti7aL8Rqa2j9hFttHd0DTMrs0KX5NXrNhHA6nWbOP+2Jcxd3M3e\nYV/+cstE89hj8Pzz7e9vNaPayWYeUOhRNg24vaj9tNQr7TBgfboVtgA4UtKuqWPAkcCCtO01SYel\nXmintTpXW9cwszIr7pJcsKlpK7MWLO3aidKzGa64IlvfY4/sfTPjO50FxWpEJbs+3wg8BIyRtErS\n6cDFwCckPQMckdYB7gL+BCwD/gv4Z4CIWAt8B3gsfb6d2kj7XJ2O+V/gl6m9vWuYWZk9t25Tl9rb\ndNZZLauZRx+FF17oYWTW21TsmU1EnNLOpgmtG1KPsjPbOc+1wLVttDcCB7bR/kpb1zCz8hteX8fq\nNhJLSV2S162DXYuGwb373fDii2WMznoTzyBgZt129sQx1A0a0KKtpC7JZ53VMtE8+qgTTR/nudHM\nrNsKvc5K7o3WupoZNgxech+e/sDJxsx6ZMq4htK6Op91Flx+efP6I4/AIYdULjDrVZxszKyyXM0Y\nfmZjZpX0la+0TDSPPOJE00+5sjGrQWUftV9urauZ3XeHl1/OLx7LnSsbsxpT9lH75fbVr76zmnGi\n6fdc2ZjVmI5G7eda3bSuZnbbDV55Jb94rFdxZWNWY8oyar/cvva1lonm4YedaKwFVzZmNaZHo/bL\nbf16qK9vXnc1Y+1wZWNWY7o9ar+L5i5ezeEX38s+593J4Rff+85nQl/7WstE42rGOuDKxqzGdHnU\nfjcUOiEUng0VOiEATHnPTi2TzK67wtrWL+U1a8nJxqwGlTxqv5va64Sw8ctfgd/+rLnxoYfgsMMq\nFof1HU42ZvYOrTsb7Lz5DZZcdlJzQ3199g4asxJ1+MxG0rsl7ZiW6yTNkHRx4U2YZtY3FXc2+Jd7\nr2mZaB56yInGuqyzDgI3Abun5QuBfYFXgRsqGZSZ5evsiWMYtvVNll9yDNMf+wUAr+2wI3MXrfJt\nM+uWdpONpGnAXwAfS8snAY3AC8AoSadJel91wjSzappyw2U89r0Ttq1PP+MK7n0o50GjVtM6emZz\nP7AJeBoYArwI3AGI7K2a9wPrKxuemVXVa6/BkCHN67vsAuvXMzu/iKyPaLeyiYgVwOXAfOAW4NsR\nsRII4JWIWBkRTjZmfcU557RMNA8+mA3aNCuDDnujRcRVkn4CvB0RG1PzK8ApFY/MzKqjdTWz885Z\nm1kZdTqDQES8XpRoiIg3ImJdZcMys6o499x3VjNONFYBHmdj1h+1rmZ22gk2bMgvHuvzPDeaWX/T\nupr5/e+daKziXNmY9ReuZixHnVY2ko6X9Iyk9ZJek7RBkm/qmtWS885zNWO5KqWy+b/A5Ih4utLB\nmFmZta5mdtwRXn89v3is3yrlmc2LTjRmNaitasaJxnJSSmXTKOlmYC6wudAYEbdVLCoz677W1czg\nwfDGG/nFY0Zplc0uwEbgSGBy+hxTyaDMrJvOP79lovnd75xorFfotLKJiM9XIxAz64ENG7J5zArq\n6mDjxvb3N6uyjmZ9Pif9/HdJV7T+VC9Es9o0Z8kcRl82mu0u3I7Rl41mzpI5lbnQ+ee3TDS//a0T\njfU6HVU2hU4BjdUIxKwvmbNkDtPvmM7GpuyP/or1K5h+x3QApo6dWp6LtK5m3vUu2LSp/f3NcqSI\nyDuGXmH8+PHR2Oi8auUx+rLRrFi/4h3to4aMYvlZy3t+gX/5F7jooub13/4W/uZven5esy6StDAi\nxne2n2cQMKuAletXdqm9ZK2rmR12gDff7Nk5zarAc6OZVcDIISO71F6Kpad/qUWi+c01tznRWM1w\nsjGrgJkTZjJ40OAWbYMHDWbmhJldP9mGDSAx5tp/B2DzgIGMPnc+//jnOuYuXl2OcM0qzsnGrAKm\njp3K7MmzGTVkFEKMGjKK2ZNnd71zwIwZLaqZE6Zewpj/MxeATU1bmbVgaTnDNquYXJ7ZSPoK8Pdk\nr5heAnwe2Au4CdgdWAicGhFbJO0AXA8cTPaW0JMiYnk6z/nA6cBW4EsRsSC1TyJ7pfUA4OqIuLh6\n384sM3Xs1O73PGv1bGbzgIHbkkyx59a595nVhlJmfR5QzgtKagC+BIyPiAPJEsLJwCXApRGxL/Aq\nWRIh/Xw1tV+a9kPS/um4A4BJwA8kDUjxXgkcBewPnJL2NasNX/96y04Av/kNH//ugjZ3HV5fV6Wg\nzHqmlNtoz0iaVeY/2AOBOkkDgcHA88DHgVvT9uuAKWn5uLRO2j5BklL7TRGxOSL+DCwDDkmfZRHx\np4jYQlYtHVfG2M0q4/XXQYKZ6bnOoEEQAR/+MGdPHEPdoJb/7qsbNICzJ47JIVCzrisl2bwf+CNw\ntaSHJU2XtEtnB7UnIlYD3wNWkiWZ9WS3zdZFxFtpt1VAQ1puAJ5Nx76V9t+9uL3VMe21v0P6Lo2S\nGtesWdPdr2TWc9/4Buy8c/P6Aw/Ali3bVqeMa+Ci48fSUF+HgIb6Oi46fixTxrX5n7ZZr1PK3Ggb\ngP8C/kvSR4EbgEsl3Qp8JyKWdeWCknYlqzT2AdYBPyO7DVZ1ETEbmA3ZoM48YrB+7vXXWyaZgQOh\nqanNXaeMa3BysZpV0jMbScdK+gVwGfBvwHuAO4C7unHNI4A/R8SaiGgCbgMOB+rTbTWAEUChT+dq\nYO8Uy0BgCFlHgW3trY5pr92sd2mrmmkn0ZjVulJ6oz0D3AfMiogHi9pvlfSRblxzJXCYpMHAJmAC\n2fxr9wEnkD1jmQbcnvafl9YfStvvjYiQNA+4QdL3geHAfsCjgID9JO1DlmROBj7TjTjNKqN1NTNg\nALz1Vvv7m/UBHVY2qWfXjyPi9FaJBoCI+FJXLxgRj5A96F9E1u15O7JbWecCX5W0jOyZzDXpkGuA\n3VP7V4Hz0nmeBG4BngJ+BZwZEVvTc50vAAvIJhO9Je1rlr/W1cz99zvRWL/Q6UScku6LiL+tUjy5\n8UScVlGtq5nttoOtW/OLx6xMSp2Is5TeaA9K+g9JH5Z0UOFThhjN+odvfvOd1YwTjfUzpTyz+VD6\n+e2itiAbF2Nm7XnlFRg6tHldgrffzi8esxyV0vW5z99CMyu7Aw6Ap55qXr//fvjoR3MLxyxvJc2N\nJulosmlh3lVoi4hvt3+EWT/VupqBbBYAs36ulHE2/wmcBHyRrFvxp4FRFY7LrPYccEDLRHPllU40\nZklJz2wi4n2SHo+ICyX9G9lATDMDWLsWdt+9ZZuTjFkLpfRGK8xhvlHScKCJbKoZMzvxxJaJxtWM\nWZtKqWzmS6oHZpENxAzg6opGZdbbrV8P9fUt25xkzNrVaWUTEd+JiHUR8XOyZzXvjYhvVD40s17q\npJNaJpqbb3aiMetEu5WNpOM72EZE+LmN9S+vvQZDhrRsc5IxK0lHt9Emd7AtcCcB609OOgluuaV5\n/aabsjYzK0m7ySYiPl/NQMx6JVczZmXhQZ1m7Tn55Ox5TIGrGbNu6zTZpEGdg4G/JeuFdgLZe2PM\n+qa2qpm3387mNjOzbillnM2HIuI04NWIuBD4a1q+CdOs7zjllJaJ5sYbs9tmTjRmPVLKbbTWgzpf\nwYM6ra9xNWNWUaVUNq0HdS4HbqxkUGZV9ZnPuJoxq7BSXjHwnbT4c0nzgXdFxPrKhmVWBa5mzKqm\n3cpG0gcl7Vm0fhpwC/AdSbtVIzizipk6tWWiueEGVzNmFdRRZfND4AgASR8BLiZ7zcAHgNlkvdLM\nasuGDbDLLi3bSqxm5i5ezawFS3lu3SaG19dx9sQxTBnXUKFAzfqWjp7ZDIiItWn5JGB2RPw8zYu2\nb+VDMyuzz362ZaLpQjUzd/Fqzr9tCavXbSKA1es2cf5tS5i7eHXl4jXrQzqqbAZIGhgRbwETgOkl\nHmfWu/SgmimYtWApm5q2tmjb1LSVWQuWuroxK0FHlc2NwAOSbifr/vxbAEn7Au4gYLXh1FNbJpo5\nc7r1bOa5dZu61G5mLXU0N9pMSfcAewG/jtg2IdR2ZM9uzHqvMlQzxYbX17G6jcQyvL6OOUvmMOOe\nGaxcv5KRQ0Yyc8JMpo6d2q3rmPVVHY6ziYiHI+IXEfFGUdsfI2JR5UMz66bW1cxPf9rjnmZnTxxD\n3aABLdrqBg3gQ/u/zPQ7prNi/QqCYMX6FUy/Yzpzlszp9rXM+iI/e7G+o8zVTLHCc5nWvdHOeuBw\nNjZtbLHvxqaNzLhnhqsbsyJONtY3nHpqVsEU/OQnWe+zMpoyruEdnQGOn7eyzX1Xrm+73ay/crKx\nsqr6WJTXX4edd27ZVsVZAEYOGcmK9SvabDezZqXMjWZWkqqPRTnttJaJ5ic/qfosADMnzGTwoMEt\n2gYPGszMCTOrFoNZLXBlY2VTtbEoW7bA0KHZM5qCnOY0KzyXcW80s4452VjZVHIsSuH23AGP3svs\nXxRVDRV4NtNVU8dOdXIx64STjZVNR2NRemLu4tV882eLuPfyUxm6MRtPfM9fHsaGG3/GlING9Ojc\nZlYdfmZjZdPeWJSzJ47p0Xkf/v41PH7RsdsSzaTP/zunf+rrzPr1H3t0XjOrHlc2VjbtjUXp9vOa\nLVtgxAguXrMGgLv3PZR/OP7r257NeKoYs9rhZGNl1dZYlG6ZOxc+9altq0d9/gqefvd7WuzS09tz\nZlY9TjbWu2zZAnvvDS+9lK1PnszcC37A8l88AUU93Xp6e87vpjGrLicb6z1uvx2mTGle/8Mf4P3v\nZwqAVLbkUBgPVOimXRgPBDjhmFVILslGUj1wNXAgEMDfAUuBm4HRwHLgxIh4VZKAy4FPAhuBzxUm\nApU0Dfh6Ou13I+K61H4w8GOgDrgL+HLRrNVWBmWtDNqoZrj99hbjZsp2ew6/m8YsD3n1Rrsc+FVE\nvBd4P/A0cB5wT0TsB9yT1gGOAvZLn+nAVQCSdgO+BRwKHAJ8S9Ku6ZirgH8oOm5SFb5Tv1HWmQJu\nvx122KE50fzhDzBvXkUHaPrdNGbVV/VkI2kI8BHgGoCI2BIR64DjgOvSbtcBhfspxwHXR+ZhoF7S\nXsBE4O6IWBsRrwJ3A5PStl3S6xECuL7oXFYGHVUGJduyBfbcs/m22THHZLMAvP/9ZYy0be11LHCH\nA7PKyaOy2QdYA/xI0mJJV0vaEdgjIp5P+7wA7JGWG4Bni45fldo6al/VRvs7SJouqVFS45rUvdY6\n1+PKoFDNvPhitr54MdxxR9Wmm6nUeCAza18eyWYgcBBwVUSMA96g+ZYZAKkiqfgzloiYHRHjI2L8\nsGHDKn25PqPblcGWLbDXXu+sZj7wgTJH2LEp4xq46PixNNTXIaChvo6Ljh/r5zVmFZRHB4FVwKqI\neCSt30qWbF6UtFdEPJ9uhaWb+KwG9i46fkRqWw18rFX7/al9RBv7W5mcPXFMi95cUEJl0Lqn2eLF\nVU8yxcrZ4cDMOlf1yiYiXgCelVT4yzQBeAqYB0xLbdOA29PyPOA0ZQ4D1qfbbQuAIyXtmjoGHAks\nSNtek3RY6sl2WtG5rAy6VBm0rmaOPjqXasbM8pXXOJsvAnMkbQ/8Cfg8WeK7RdLpwArgxLTvXWTd\nnpeRdX3+PEBErJX0HeCxtN+3I2JtWv5nmrs+/zJ9rIxKqgzmzYPjjmteL6GaKbVLdfF+Q+oGIcG6\njU0eoGnWS8nDTzLjx4+PxsbGvMPotbo0rmbLFhg1Cl54IVv/5Cdh/vxOOwC0HmwJ2e251lVTW/sV\na+sYM6sMSQsjYnxn+3nWZ+tUl8bVzJuX9TQrJJrFi+HOO0vqaVZql+q29uvsGDPLl5ONdaqkJLBl\nCwwf3nzb7JOf7PKzmVK7VJfSxdoDNM16Fycb61SnSaBQzTyfhkktWlRyNVOs1C7VQwa3X9V0di4z\ny4eTjXWqvT/cI3ca2LKaOeqorJoZN65b1yl1sOW6gdfzNm+2ex4P0DTrfTzrs3WqrXE1R/35Ma66\n5cLmnRYtgnHjejRBZ6kvX1vZdBt1g15m17emMSCG8jYbANiOnRlRv6N7o5n1Qk421qniJPDSKxt4\ncPbpDHvtlWzjUUdtu2VWjqn7S+lSPXLISFasf4CNAx9o0T5qyCh+f9byLnwzM6sW30azkkwZ18Dv\nD3iDZ743pTnRLFoEd9217dlMWSboLMHMCTMZPGhwi7bBgwYzc8LMsl7HzMrHycY619QEDQ1w7LHZ\n+qRJbT6bqdbU/VPHTmX25NmMGjIKIUYNGcXsybOZOnZqWa9jZuXj22jWsfnzs5eZFSxcCAcd1Oau\nw+vrWN2OqX7wAAALHUlEQVRGYqlEz7CpY6c6uZjVEFc21ramJhgxojnRFKqZdhINeOp+M2ufKxt7\npy5UM8VK7U1mZv2Pk401a2qC97wHVqV3z02cCL/8ZZcGZ3rqfjNri2+jWWb+fNh+++ZEs3Ah/OpX\nVXt7ppn1ba5s+rvW1cyRRzrJmFnZubLpz+68853VzIIFTjRmVnaubPqjpib4i7+AZ5/N1l3NmFmF\nubLpbwrVTCHRNDa6mjGzinNl01+0rmY+8QknGTOrGlc2/UFb1cyvf+1EY2ZV48qmL2tqgn33hZUr\ns3VXM2aWE1c2fdVdd2XVTCHRuJoxsxw52fRF55wDRx+dLR9xRDan2cEH5xuTmfVrvo3WF+2wQ/bz\nscdg/Ph8YzEzAxQRecfQK4wfPz4aGxvLes6evCLZzKwWSFoYEZ3+q9aVTYWU4xXJZmZ9hZ/ZVEi1\nXpFsZlYLnGwqpFqvSDYzqwVONhXS3quQK/GKZDOz3s7JpkL8imQzs2buIFAhlX5Fsnu6mVktcbKp\noEq9Itk93cys1vg2Wg1yTzczqzVONjXIPd3MrNY42dQg93Qzs1rjZFOD3NPNzGqNOwjUoEr3dDMz\nKzcnmxpVqZ5uZmaVkNttNEkDJC2WND+t7yPpEUnLJN0safvUvkNaX5a2jy46x/mpfamkiUXtk1Lb\nMknnVfu75WnOkjmMvmw02124HaMvG82cJXPyDsnMLNdnNl8Gni5avwS4NCL2BV4FTk/tpwOvpvZL\n035I2h84GTgAmAT8ICWwAcCVwFHA/sApad+Kmbt4NYdffC/7nHcnh198L3MXr67k5do1Z8kcpt8x\nnRXrVxAEK9avYPod051wzCx3uSQbSSOAo4Gr07qAjwO3pl2uA6ak5ePSOmn7hLT/ccBNEbE5Iv4M\nLAMOSZ9lEfGniNgC3JT2rYjCAMvV6zYRNA+wzCPhzLhnBhubNrZo29i0kRn3zKh6LGZmxfKqbC4D\nzgHeTuu7A+si4q20vgooPJBoAJ4FSNvXp/23tbc6pr32d5A0XVKjpMY1a9Z064v0pgGWK9ev7FK7\nmVm1VD3ZSDoGeCkiFlb72q1FxOyIGB8R44cNG9atc/SmAZYjh4zsUruZWbXkUdkcDhwraTnZLa6P\nA5cD9ZIKveNGAIX7UKuBvQHS9iHAK8XtrY5pr70iujvAshLPeWZOmMngQYNbtA0eNJiZE2b2+Nxm\nZj1R9WQTEedHxIiIGE32gP/eiJgK3AeckHabBtyeluelddL2eyMiUvvJqbfaPsB+wKPAY8B+qXfb\n9uka8yr1fbozwLJSz3mmjp3K7MmzGTVkFEKMGjKK2ZNnM3Xs1B6d18ysp3rTOJtzgZskfRdYDFyT\n2q8BfiJpGbCWLHkQEU9KugV4CngLODMitgJI+gKwABgAXBsRT1Yq6O4MsOzoOU9Px85MHTvVycXM\neh1lRYKNHz8+Ghsbq3Ktfc67k7Z+6wL+fPHRVYnBzKwcJC2MiPGd7ee50XLgiTTNrL9xssmBJ9I0\ns/6mNz2z6Tc8kaaZ9TdONjnxRJpm1p/4NpqZmVWck42ZmVWck42ZmVWck42ZmVWck42ZmVWck42Z\nmVWck42ZmVWc50ZLJK0BVuQdRzcMBV7OO4gc+fv7+/v752tURHT6QjAnmxonqbGUSfD6Kn9/f39/\n/9r4/r6NZmZmFedkY2ZmFedkU/tm5x1Azvz9+zd//xrhZzZmZlZxrmzMzKzinGzMzKzinGxqkKS9\nJd0n6SlJT0r6ct4x5UHSAEmLJc3PO5Zqk1Qv6VZJ/yPpaUl/nXdM1STpK+m//Sck3SjpXXnHVGmS\nrpX0kqQnitp2k3S3pGfSz13zjLEjTja16S3gaxGxP3AYcKak/XOOKQ9fBp7OO4icXA78KiLeC7yf\nfvR7kNQAfAkYHxEHAgOAk/ONqip+DExq1XYecE9E7Afck9Z7JSebGhQRz0fEorS8gewPTb967aek\nEcDRwNV5x1JtkoYAHwGuAYiILRGxLt+oqm4gUCdpIDAYeC7neCouIn4DrG3VfBxwXVq+DphS1aC6\nwMmmxkkaDYwDHsk3kqq7DDgHeDvvQHKwD7AG+FG6jXi1pB3zDqpaImI18D1gJfA8sD4ifp1vVLnZ\nIyKeT8svAHvkGUxHnGxqmKSdgJ8DZ0XEa3nHUy2SjgFeioiFeceSk4HAQcBVETEOeINefPuk3NJz\niePIku5wYEdJn803qvxFNo6l145lcbKpUZIGkSWaORFxW97xVNnhwLGSlgM3AR+X9NN8Q6qqVcCq\niChUs7eSJZ/+4gjgzxGxJiKagNuAD+UcU15elLQXQPr5Us7xtMvJpgZJEtn9+qcj4vt5x1NtEXF+\nRIyIiNFkD4bvjYh+8y/biHgBeFbSmNQ0AXgqx5CqbSVwmKTB6f8LE+hHHSRamQdMS8vTgNtzjKVD\nTja16XDgVLJ/0f8hfT6Zd1BWVV8E5kh6HPgA8K85x1M1qaK7FVgELCH7O1Yz07Z0l6QbgYeAMZJW\nSToduBj4hKRnyCq+i/OMsSOersbMzCrOlY2ZmVWck42ZmVWck42ZmVWck42ZmVWck42ZmVWck431\nS8r8TtJRRW2flvSrPOOqFEmHSro07zis/3LXZ+u3JB0I/IxsbrmBwGJgUkT8bw/OOTAi3ipTiGXR\nG2Oy/seVjfVbEfEEcAdwLvBN4PqI+F9J0yQ9mgbL/kDSdgCSZktqTO9R+WbhPGmA3Tck/R74VHrX\nylOSHm9vGh1JH5T0gKSFkn4paQ9Jg9L636R9Zkm6sOgaF6e4HpH0ntS+h6TbUlyPSjostX9X0g8l\n3U02YecRkuambTtJ+nHaf7Gkyan979M7chak96NcVBTv0ZIWSfr/Jf26o/OYtSki/PGn336AHYGl\nZCPRdwAOBOYCA9P22cBn0vJu6edA4LfA/ml9FfDVonM+D2yfluvbuOYOwIPA0LQ+FZidlt9HNvXM\nkUAjMKjoGuem5b8D5qblm4HD0vJo4Im0/F3gUeBdaf2IomP+L3ByWt4V+CPwLuDvgWeAXYA64Fmy\niS73JJsiZlSr30Ob58n7f1N/eudnYBdzk1mfEhFvSLoZeD0iNks6Avgg0JhNu7Xtjy7AKWmKkIFk\nf4T3p3lOspuLTvsk8FNJt5Mlrtb+CjgA+O90jQFkyYSIeFzSTWRzXB0a2USTBTemn3NonpbkCLLp\nSwr77CqpLi3fHhFvtnH9I4GjJBVmin4XMDIt/3ekGcQl/U9q3wu4LyJWpBjXdnKeP7ZxTevnnGzM\nsnfiFN6LI+DaiPhG8Q6S9iN7M+ghEbEu3R4rfhXxG0XLE4GPkk2D//X0bGgBMBR4GPgh8HhEfLid\neA4E1gPvbtXe1gNWpZi2tIq3dUytj5kSrZ5NSfoIsLmoaSsd/41o8zxmbfEzG7OW/hs4UdJQAEm7\nSxpJdmtpA/Bamsp9YlsHSxoAjIiIe4GzgWHA4Ig4IiI+EBFnkFVDDZIOScdsL+mAtHwSsBPwMeBK\nSbsUnf6k9PMU4PdF8Z5ZdP0PlPAdF5BN5Fk4Zlwn+z8I/K2kUWn/3bp5HuvHXNmYFYmIJemh/H+n\njgFNwBlkz0+eAp4A/kTzH/vWBgI3SNqZ7B9zl0T26u7ia2yWdAJwRUomA4B/k7QG+A7wsYh4TtIP\ngUuB09OhgyU9SlbhnJLazgSukvT5dO37KEo+7bgQuExSYcbkZWRVWHu/kxcl/RNwe5rS/zngqK6e\nx/o3d302qwGSVgEHRsS6vGMx6w7fRjMzs4pzZWNmZhXnysbMzCrOycbMzCrOycbMzCrOycbMzCrO\nycbMzCru/wHs3ZHs7RM8WgAAAABJRU5ErkJggg==\n", 447 | "text/plain": [ 448 | "" 449 | ] 450 | }, 451 | "metadata": {}, 452 | "output_type": "display_data" 453 | } 454 | ], 455 | "source": [ 456 | "plt.scatter(X_test, y_test, c='g')\n", 457 | "plt.scatter(X_train, y_train)\n", 458 | "plt.plot(X_test, pred_1, c='r')\n", 459 | "plt.title(\"Linear Regression - Regressor\")\n", 460 | "plt.xlabel(\"Years-experience\")\n", 461 | "plt.ylabel(\"Salary in $\")\n", 462 | "plt.show()\n", 463 | "\n", 464 | "plt.scatter(X_test, y_test, c='g')\n", 465 | "plt.scatter(X_train, y_train)\n", 466 | "plt.plot(X_test, pred_2, c='r')\n", 467 | "plt.title(\"Ridge - Regressor\")\n", 468 | "plt.xlabel(\"Years-experience\")\n", 469 | "plt.ylabel(\"Salary in $\")\n", 470 | "plt.show()" 471 | ] 472 | }, 473 | { 474 | "cell_type": "markdown", 475 | "metadata": {}, 476 | "source": [ 477 | "#### Let's try to improve Ridge regression with Grid Search CV" 478 | ] 479 | }, 480 | { 481 | "cell_type": "code", 482 | "execution_count": 31, 483 | "metadata": { 484 | "collapsed": true 485 | }, 486 | "outputs": [], 487 | "source": [ 488 | "from sklearn.grid_search import GridSearchCV" 489 | ] 490 | }, 491 | { 492 | "cell_type": "code", 493 | "execution_count": 32, 494 | "metadata": { 495 | "collapsed": true 496 | }, 497 | "outputs": [], 498 | "source": [ 499 | "params = {'solver':['auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag'], 'max_iter':[1000, 10000]}" 500 | ] 501 | }, 502 | { 503 | "cell_type": "code", 504 | "execution_count": 33, 505 | "metadata": { 506 | "collapsed": false 507 | }, 508 | "outputs": [ 509 | { 510 | "name": "stdout", 511 | "output_type": "stream", 512 | "text": [ 513 | "Fitting 5 folds for each of 12 candidates, totalling 60 fits\n", 514 | "[CV] solver=auto, max_iter=1000 ......................................\n", 515 | "[CV] ............. solver=auto, max_iter=1000, score=0.957247 - 0.0s\n", 516 | "[CV] solver=auto, max_iter=1000 ......................................\n", 517 | "[CV] ............. solver=auto, max_iter=1000, score=0.937050 - 0.0s\n", 518 | "[CV] solver=auto, max_iter=1000 ......................................\n", 519 | "[CV] ............. solver=auto, max_iter=1000, score=0.881413 - 0.0s\n", 520 | "[CV] solver=auto, max_iter=1000 ......................................\n", 521 | "[CV] ............. solver=auto, max_iter=1000, score=0.813687 - 0.0s\n", 522 | "[CV] solver=auto, max_iter=1000 ......................................\n", 523 | "[CV] ............. solver=auto, max_iter=1000, score=0.474748 - 0.0s\n", 524 | "[CV] solver=svd, max_iter=1000 .......................................\n", 525 | "[CV] .............. solver=svd, max_iter=1000, score=0.957247 - 0.0s\n", 526 | "[CV] solver=svd, max_iter=1000 .......................................\n", 527 | "[CV] .............. solver=svd, max_iter=1000, score=0.937050 - 0.0s\n", 528 | "[CV] solver=svd, max_iter=1000 .......................................\n", 529 | "[CV] .............. solver=svd, max_iter=1000, score=0.881413 - 0.0s\n", 530 | "[CV] solver=svd, max_iter=1000 .......................................\n", 531 | "[CV] .............. solver=svd, max_iter=1000, score=0.813687 - 0.0s\n", 532 | "[CV] solver=svd, max_iter=1000 .......................................\n", 533 | "[CV] .............. solver=svd, max_iter=1000, score=0.474748 - 0.0s\n", 534 | "[CV] solver=cholesky, max_iter=1000 ..................................\n", 535 | "[CV] ......... solver=cholesky, max_iter=1000, score=0.957247 - 0.0s\n", 536 | "[CV] solver=cholesky, max_iter=1000 ..................................\n", 537 | "[CV] ......... solver=cholesky, max_iter=1000, score=0.937050 - 0.0s\n", 538 | "[CV] solver=cholesky, max_iter=1000 ..................................\n", 539 | "[CV] ......... solver=cholesky, max_iter=1000, score=0.881413 - 0.0s\n", 540 | "[CV] solver=cholesky, max_iter=1000 ..................................\n", 541 | "[CV] ......... solver=cholesky, max_iter=1000, score=0.813687 - 0.0s\n", 542 | "[CV] solver=cholesky, max_iter=1000 ..................................\n", 543 | "[CV] ......... solver=cholesky, max_iter=1000, score=0.474748 - 0.0s\n", 544 | "[CV] solver=lsqr, max_iter=1000 ......................................\n", 545 | "[CV] ............. solver=lsqr, max_iter=1000, score=0.957247 - 0.0s\n", 546 | "[CV] solver=lsqr, max_iter=1000 ......................................\n", 547 | "[CV] ............. solver=lsqr, max_iter=1000, score=0.937050 - 0.0s\n", 548 | "[CV] solver=lsqr, max_iter=1000 ......................................\n", 549 | "[CV] ............. solver=lsqr, max_iter=1000, score=0.881413 - 0.0s\n", 550 | "[CV] solver=lsqr, max_iter=1000 ......................................\n", 551 | "[CV] ............. solver=lsqr, max_iter=1000, score=0.813687 - 0.0s\n", 552 | "[CV] solver=lsqr, max_iter=1000 ......................................\n", 553 | "[CV] ............. solver=lsqr, max_iter=1000, score=0.474748 - 0.0s\n", 554 | "[CV] solver=sparse_cg, max_iter=1000 .................................\n", 555 | "[CV] ........ solver=sparse_cg, max_iter=1000, score=0.957247 - 0.0s\n", 556 | "[CV] solver=sparse_cg, max_iter=1000 .................................\n", 557 | "[CV] ........ solver=sparse_cg, max_iter=1000, score=0.937050 - 0.0s\n", 558 | "[CV] solver=sparse_cg, max_iter=1000 .................................\n", 559 | "[CV] ........ solver=sparse_cg, max_iter=1000, score=0.881413 - 0.0s\n", 560 | "[CV] solver=sparse_cg, max_iter=1000 .................................\n", 561 | "[CV] ........ solver=sparse_cg, max_iter=1000, score=0.813687 - 0.0s\n", 562 | "[CV] solver=sparse_cg, max_iter=1000 .................................\n", 563 | "[CV] ........ solver=sparse_cg, max_iter=1000, score=0.474748 - 0.0s\n", 564 | "[CV] solver=sag, max_iter=1000 .......................................\n", 565 | "[CV] .............. solver=sag, max_iter=1000, score=0.957257 - 0.0s\n", 566 | "[CV] solver=sag, max_iter=1000 .......................................\n", 567 | "[CV] .............. solver=sag, max_iter=1000, score=0.936993 - 0.0s\n", 568 | "[CV] solver=sag, max_iter=1000 .......................................\n", 569 | "[CV] .............. solver=sag, max_iter=1000, score=0.881199 - 0.0s\n", 570 | "[CV] solver=sag, max_iter=1000 .......................................\n", 571 | "[CV] .............. solver=sag, max_iter=1000, score=0.813680 - 0.0s\n", 572 | "[CV] solver=sag, max_iter=1000 .......................................\n", 573 | "[CV] .............. solver=sag, max_iter=1000, score=0.470442 - 0.0s\n", 574 | "[CV] solver=auto, max_iter=10000 .....................................\n", 575 | "[CV] ............ solver=auto, max_iter=10000, score=0.957247 - 0.0s\n", 576 | "[CV] solver=auto, max_iter=10000 .....................................\n", 577 | "[CV] ............ solver=auto, max_iter=10000, score=0.937050 - 0.0s\n", 578 | "[CV] solver=auto, max_iter=10000 .....................................\n", 579 | "[CV] ............ solver=auto, max_iter=10000, score=0.881413 - 0.0s\n", 580 | "[CV] solver=auto, max_iter=10000 .....................................\n", 581 | "[CV] ............ solver=auto, max_iter=10000, score=0.813687 - 0.0s\n", 582 | "[CV] solver=auto, max_iter=10000 .....................................\n", 583 | "[CV] ............ solver=auto, max_iter=10000, score=0.474748 - 0.0s\n", 584 | "[CV] solver=svd, max_iter=10000 ......................................\n", 585 | "[CV] ............. solver=svd, max_iter=10000, score=0.957247 - 0.0s\n", 586 | "[CV] solver=svd, max_iter=10000 ......................................\n", 587 | "[CV] ............. solver=svd, max_iter=10000, score=0.937050 - 0.0s\n", 588 | "[CV] solver=svd, max_iter=10000 ......................................\n", 589 | "[CV] ............. solver=svd, max_iter=10000, score=0.881413 - 0.0s\n", 590 | "[CV] solver=svd, max_iter=10000 ......................................\n", 591 | "[CV] ............. solver=svd, max_iter=10000, score=0.813687 - 0.0s\n", 592 | "[CV] solver=svd, max_iter=10000 ......................................\n", 593 | "[CV] ............. solver=svd, max_iter=10000, score=0.474748 - 0.0s\n", 594 | "[CV] solver=cholesky, max_iter=10000 .................................\n", 595 | "[CV] ........ solver=cholesky, max_iter=10000, score=0.957247 - 0.0s\n", 596 | "[CV] solver=cholesky, max_iter=10000 .................................\n", 597 | "[CV] ........ solver=cholesky, max_iter=10000, score=0.937050 - 0.0s\n", 598 | "[CV] solver=cholesky, max_iter=10000 .................................\n", 599 | "[CV] ........ solver=cholesky, max_iter=10000, score=0.881413 - 0.0s\n", 600 | "[CV] solver=cholesky, max_iter=10000 .................................\n", 601 | "[CV] ........ solver=cholesky, max_iter=10000, score=0.813687 - 0.0s\n", 602 | "[CV] solver=cholesky, max_iter=10000 .................................\n", 603 | "[CV] ........ solver=cholesky, max_iter=10000, score=0.474748 - 0.0s\n", 604 | "[CV] solver=lsqr, max_iter=10000 .....................................\n", 605 | "[CV] ............ solver=lsqr, max_iter=10000, score=0.957247 - 0.0s\n", 606 | "[CV] solver=lsqr, max_iter=10000 .....................................\n", 607 | "[CV] ............ solver=lsqr, max_iter=10000, score=0.937050 - 0.0s\n", 608 | "[CV] solver=lsqr, max_iter=10000 .....................................\n", 609 | "[CV] ............ solver=lsqr, max_iter=10000, score=0.881413 - 0.0s\n", 610 | "[CV] solver=lsqr, max_iter=10000 .....................................\n", 611 | "[CV] ............ solver=lsqr, max_iter=10000, score=0.813687 - 0.0s\n", 612 | "[CV] solver=lsqr, max_iter=10000 .....................................\n", 613 | "[CV] ............ solver=lsqr, max_iter=10000, score=0.474748 - 0.0s\n", 614 | "[CV] solver=sparse_cg, max_iter=10000 ................................\n", 615 | "[CV] ....... solver=sparse_cg, max_iter=10000, score=0.957247 - 0.0s\n", 616 | "[CV] solver=sparse_cg, max_iter=10000 ................................\n", 617 | "[CV] ....... solver=sparse_cg, max_iter=10000, score=0.937050 - 0.0s\n", 618 | "[CV] solver=sparse_cg, max_iter=10000 ................................\n", 619 | "[CV] ....... solver=sparse_cg, max_iter=10000, score=0.881413 - 0.0s\n", 620 | "[CV] solver=sparse_cg, max_iter=10000 ................................\n", 621 | "[CV] ....... solver=sparse_cg, max_iter=10000, score=0.813687 - 0.0s\n", 622 | "[CV] solver=sparse_cg, max_iter=10000 ................................\n", 623 | "[CV] ....... solver=sparse_cg, max_iter=10000, score=0.474748 - 0.0s\n", 624 | "[CV] solver=sag, max_iter=10000 ......................................\n", 625 | "[CV] ............. solver=sag, max_iter=10000, score=0.957279 - 0.0s\n", 626 | "[CV] solver=sag, max_iter=10000 ......................................\n", 627 | "[CV] ............. solver=sag, max_iter=10000, score=0.936810 - 0.0s\n", 628 | "[CV] solver=sag, max_iter=10000 ......................................\n", 629 | "[CV] ............. solver=sag, max_iter=10000, score=0.880743 - 0.0s\n", 630 | "[CV] solver=sag, max_iter=10000 ......................................\n", 631 | "[CV] ............. solver=sag, max_iter=10000, score=0.810971 - 0.0s\n", 632 | "[CV] solver=sag, max_iter=10000 ......................................\n", 633 | "[CV] ............. solver=sag, max_iter=10000, score=0.473753 - 0.0s\n" 634 | ] 635 | }, 636 | { 637 | "name": "stderr", 638 | "output_type": "stream", 639 | "text": [ 640 | "[Parallel(n_jobs=1)]: Done 1 tasks | elapsed: 0.0s\n", 641 | "[Parallel(n_jobs=1)]: Done 4 tasks | elapsed: 0.0s\n", 642 | "[Parallel(n_jobs=1)]: Done 7 tasks | elapsed: 0.0s\n", 643 | "[Parallel(n_jobs=1)]: Done 12 tasks | elapsed: 0.0s\n", 644 | "[Parallel(n_jobs=1)]: Done 17 tasks | elapsed: 0.0s\n", 645 | "[Parallel(n_jobs=1)]: Done 24 tasks | elapsed: 0.0s\n", 646 | "[Parallel(n_jobs=1)]: Done 31 tasks | elapsed: 0.0s\n", 647 | "[Parallel(n_jobs=1)]: Done 40 tasks | elapsed: 0.0s\n", 648 | "[Parallel(n_jobs=1)]: Done 49 tasks | elapsed: 0.0s\n", 649 | "[Parallel(n_jobs=1)]: Done 60 tasks | elapsed: 0.0s\n", 650 | "[Parallel(n_jobs=1)]: Done 60 out of 60 | elapsed: 0.0s finished\n" 651 | ] 652 | }, 653 | { 654 | "data": { 655 | "text/plain": [ 656 | "GridSearchCV(cv=5, error_score='raise',\n", 657 | " estimator=Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,\n", 658 | " normalize=False, random_state=None, solver='auto', tol=0.001),\n", 659 | " fit_params={}, iid=True, n_jobs=1,\n", 660 | " param_grid={'solver': ['auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag'], 'max_iter': [1000, 10000]},\n", 661 | " pre_dispatch='2*n_jobs', refit=True, scoring=None, verbose=10)" 662 | ] 663 | }, 664 | "execution_count": 33, 665 | "metadata": {}, 666 | "output_type": "execute_result" 667 | } 668 | ], 669 | "source": [ 670 | "gs = GridSearchCV(rig, params, cv=5, verbose=10)\n", 671 | "gs.fit(X_train, y_train)" 672 | ] 673 | }, 674 | { 675 | "cell_type": "code", 676 | "execution_count": 34, 677 | "metadata": { 678 | "collapsed": false 679 | }, 680 | "outputs": [ 681 | { 682 | "name": "stdout", 683 | "output_type": "stream", 684 | "text": [ 685 | "{'solver': 'auto', 'max_iter': 1000}\n" 686 | ] 687 | } 688 | ], 689 | "source": [ 690 | "print(gs.best_params_)" 691 | ] 692 | }, 693 | { 694 | "cell_type": "code", 695 | "execution_count": 35, 696 | "metadata": { 697 | "collapsed": true 698 | }, 699 | "outputs": [], 700 | "source": [ 701 | "improved_rig = Ridge(solver='auto', max_iter=1000, alpha=1.0)" 702 | ] 703 | }, 704 | { 705 | "cell_type": "code", 706 | "execution_count": 36, 707 | "metadata": { 708 | "collapsed": false 709 | }, 710 | "outputs": [ 711 | { 712 | "data": { 713 | "text/plain": [ 714 | "Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=1000,\n", 715 | " normalize=False, random_state=None, solver='auto', tol=0.001)" 716 | ] 717 | }, 718 | "execution_count": 36, 719 | "metadata": {}, 720 | "output_type": "execute_result" 721 | } 722 | ], 723 | "source": [ 724 | "improved_rig.fit(X_train, y_train)" 725 | ] 726 | }, 727 | { 728 | "cell_type": "code", 729 | "execution_count": 37, 730 | "metadata": { 731 | "collapsed": false 732 | }, 733 | "outputs": [ 734 | { 735 | "name": "stdout", 736 | "output_type": "stream", 737 | "text": [ 738 | "R^2 on the Ridge regressor, tuned: 0.925002520196\n" 739 | ] 740 | } 741 | ], 742 | "source": [ 743 | "print(\"R^2 on the Ridge regressor, tuned: \", improved_rig.score(X_test, y_test))" 744 | ] 745 | }, 746 | { 747 | "cell_type": "markdown", 748 | "metadata": {}, 749 | "source": [ 750 | "### Tree Regression" 751 | ] 752 | }, 753 | { 754 | "cell_type": "code", 755 | "execution_count": 38, 756 | "metadata": { 757 | "collapsed": true 758 | }, 759 | "outputs": [], 760 | "source": [ 761 | "from sklearn.tree import DecisionTreeRegressor" 762 | ] 763 | }, 764 | { 765 | "cell_type": "code", 766 | "execution_count": 61, 767 | "metadata": { 768 | "collapsed": true 769 | }, 770 | "outputs": [], 771 | "source": [ 772 | "dtr = DecisionTreeRegressor(max_depth=100)" 773 | ] 774 | }, 775 | { 776 | "cell_type": "code", 777 | "execution_count": 62, 778 | "metadata": { 779 | "collapsed": false 780 | }, 781 | "outputs": [ 782 | { 783 | "data": { 784 | "text/plain": [ 785 | "DecisionTreeRegressor(criterion='mse', max_depth=100, max_features=None,\n", 786 | " max_leaf_nodes=None, min_samples_leaf=1, min_samples_split=2,\n", 787 | " min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n", 788 | " splitter='best')" 789 | ] 790 | }, 791 | "execution_count": 62, 792 | "metadata": {}, 793 | "output_type": "execute_result" 794 | } 795 | ], 796 | "source": [ 797 | "dtr.fit(X_train, y_train)" 798 | ] 799 | }, 800 | { 801 | "cell_type": "code", 802 | "execution_count": 78, 803 | "metadata": { 804 | "collapsed": false 805 | }, 806 | "outputs": [ 807 | { 808 | "data": { 809 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VUX6x/HPQ1NjAcSyKiZx17L6UxSNCnbFgrrY2xqs\nrKy9u7JmLajYFXuJYiWiqKjgCoiIoihgUBdQFsGFUCwUJYJIf35/zITchFTIzU35vl+v+7rnzJlz\n7twoeTLnzDxj7o6IiEgyNUl1A0REpOFTsBERkaRTsBERkaRTsBERkaRTsBERkaRTsBERkaRTsBGR\nMpnZe2aWnep2SMNgmmcjjYGZLUrYTQOWAivj/t/dPa/2WyXSeCjYSKNjZtOBv7n7+xXUaebuK5L0\n+TV67WS2VaSm6DaaCGBmt5vZq2bWz8wWAl3NrImZ3WBm35nZPDN7xcxaJ5yzv5mNNrMFZvaVmR1U\nwfVnmdl1ZjYB+C2WtTWzN81srplNM7NLEuqnmVnfeO1vzKxHDJJre70OZvaFmf1qZj+Z2b0Jn/Oy\nmc2PnzXWzDaLxz4xs3PjdhMzu8nMCsxsjpk9b2abxGPbm5mb2dmxXXPNrEcN/GeRBkTBRqTYicDL\nQEvgVeAq4FjgIKAtsAh4GMDMtgUGAjcDmwI9gAFm1qaC658BHA20MrMmwDvA58A2wBHAdWbWKda9\nFdgayASOArqu4/UeAe51902A7YHXY/l5hNuKbYE2wMXAkjI+62+xDYcAfwJaAw+VqrNfvPZRQE8z\n26GCn4U0Mgo2IsU+cfdB7r7K3X8HLgRucPfZ7r4E6AmcGn+xnw0MdPehsf4Q4D9A5wqu/5C7z4rX\n7ghs4u53uPsyd58K9CEEEIDTgF7uvsDdZwKPruP1lgM7mFkbd1/o7mMSyjcDtnf3le6e7+6L1vgk\nyAbuc/dp7r4QuAE4M/4sitzi7kvc/Qvga2D3Cn4W0sg0S3UDROqQmaX204FBZraqVPkWQAbwVzM7\nMaG8OTCkitfPANLNbEFCWVPgw7i9Van6pdtW3eudRwiWk83sf4TA8C7wPKEH1T/eFnsJ+FcZz4C2\nBgoS9guAFsDmRQXu/mPC8cXARmW0WRopBRuRYqVHy8wCzkzoBaxmZjOB59z9orW8/kxgirvvXE7d\nHwm3tr6N+9uuy/XcfTJwRuyJnAq8YWatY4/tFuAWM9uOECwnAS+UusT3hIBWJB1YBswFNi7nO4is\npttoIuV7ErjDzNIBzGwLMzsuHnsJONHMjjCzpma2vpkdamZbV/HanwHLzOyaeG5TM9vNzPaKx/sD\nN5hZKzNrC1xS/qUqv56ZnWVmm7n7KqCQEKhWmdlhZrZrDEK/Em6rle7JAfQDrjazTDPbGOgF9IvX\nE6mUgo1I+R4g/KU/PI5Q+xTYG8DdpxMGFNxI+Ot+BnANVfw3FW9THQPsA0wH5gFPAZvEKjcDP8Vj\n7xGCz9J1uN4xwKT4Pe4DTnf3ZYTbYwMIgeZr4H3CIInSniYMmvgY+B+wELiiKt9VBDTPRqReMLPL\ngBPcvVOllUXqIPVsROogM9vGzPaL81t2JgzDfjPV7RJZWxogIFI3rUe4dZUJ/EJ4ZvJUKhsksi50\nG01ERJJOt9FERCTpdBst2myzzTwzMzPVzRARqVfGjRs3z903r6yegk2UmZlJfn5+qpshIlKvmFlB\n5bV0G01ERGqBgo2IiCSdgo2IiCSdgo2IiCSdgo2IiCSdgo2ISAOWlweZmdCkSXjPy0tNOxRsREQa\nqLw86N4dCgrAPbx37x7KazsIaZ6NiEgDlZMDixeXLFu8GK64An7/vfhYURACyM5OTlvUsxERaaBm\nzCi7fP78soNQTk7y2qJgIyLSQKWnV69+ecGpJijYiIg0UL16QVpaybK0NGjTpuz61Q1O1ZG0YGNm\nz5rZHDObmFB2r5n918zGm9mbZtYq4dg/zWyqmU02s6MSyjvHsqlm1iOhfDszGxPLXzWzFrF8vbg/\nNR7PTNZ3FBGpy7KzITcXMjLALLzn5sJDD5UdhHr1Sl5bktmzeR7oXKpsGLCru7cDvgX+CWBmuwBn\nAP8Xz3nczJqaWVPgMeBoYBfgr7EuwN1Ab3ffnrC4VLdY3g34JZb3jvVERBql7GyYPh1WrQrv2dnl\nB6FkDQ6AJAYbdx8J/Fyq7D13XxF3RwNt4/bxwCvuvtTdpwFTgX3ia6q7/8/dlwGvAMebmQGHAa/H\n818ATki41gtx+3WgU6wvIiJRWUEomVL5zOZ8YHDc3gaYmXBsViwrr7wNsCAhcBWVl7hWPF4Y66/B\nzLqbWb6Z5c+dO3edv5CIiJQtJcHGzHKAFUCK5rIG7p7r7lnunrX55pWu/SMiImup1id1mtm5wF+A\nTu7usXg2sG1CtbaxjHLK5wOtzKxZ7L0k1i+61iwzawa0jPVFRCRFarVnY2adgX8Ax7l74pSigcAZ\ncSTZdsAOwFjgc2CHOPKsBWEQwcAYpEYAp8TzzwHeTrjWOXH7FOCDhKAmIiIpkMyhz/2Az4CdzGyW\nmXUDHgU2BoaZ2Vdm9iSAu38N9Ae+AYYAl7j7ythruRQYCkwC+se6ANcDV5vZVMIzmT6xvA/QJpZf\nDaweLi0iUlfUlQSZtcX0R3+QlZXl+fn5qW6GiNQnS5dCixZh/HA1FCXITEwZk5aW/OHHyWBm49w9\nq7J6yiAgIrI25syBgw+Ge+6p9qnlJchMZm6yVFOwERGprsmToUMHGD8edtyx2qeXl4MsmbnJUk3B\nRkSkOkaOhI4dYdEiGDECTjyx2pcoLwdZMnOTpZqCjYhIVb38MhxxBGyxBYweDfvuu1aXKS9BZjJz\nk6Wago2ISGXc4fbbw9P7jh3h00/hj39c68ulIjdZqinYiEiDUuNDipcvh7/9DW68Ebp2haFDYdNN\n17mdtZ2bLNW0LLSINBilhxSv83LHhYVwyinw/vsh2PTsWe1hzhKoZyMiDUaNDSn+z3+gXz844AD4\n8EN49lm49VYFmnWgYCMiDcY6Dyl2h8cfhz32gDPPDCcOGQLnnVdjbSxLY8gmoNtoItJgpKeHW2dl\nlVdq8WK48EJ46aWwv8EGMGoU7LprjbaxtBq/9VdHqWcjIg3GWg8p/u67MMqsKNDssksoS3KggcaT\nTUDBRkQajLUaUjxoELRvH7IBABxzDIwZA1ttVSttbizZBBRsRKRBqfKQ4pUrwwiz446DhQtD2aWX\nwsCBsNFGtdTaxpNNQMFGRBqf+fPh2GPDRM0ivXvDww9D06a12pTGkk1AwUZEGpdx42CvvcLkzCJv\nvAFXXpmSoc2NJZuARqOJSOPRpw9ccklYh6bIOuQ4qynZ2Q0vuJSmno2INHxLlsAFF4S0M0WBZrPN\nwoizUoEmb0IemQ9m0qRnEzIfzCRvQgOc9JIC6tmISMNWUAAnnxxunxXZd1949901cpzlTcij+6Du\nLF4exiIXFBbQfVCY9JK9WwPveiSZejYi0nANHQp77lky0Jx5Jnz0UZnJNHOG56wONEUWL19MzvAG\nNuklBRRsRKThWbUqjDQ7+mj4+efi8htvhL59Yb31yjxtRmHZk1vKK5eq0200EWlYFiyAs88OkzUT\nPftspTnO0lumU1C4Zr6b9JYNbNJLCqhnIyINx/jxkJW1ZqB5//0qJdPs1akXac1LTnpJa55Gr04N\nbNJLCijYiEjD0LcvdOgQRpglmjABOnWq0iWyd8smt0suGS0zMIyMlhnkdsnV4IAaYO6e6jbUCVlZ\nWZ6fn5/qZohIdS1bBldfDY89VrJ8u+1C1uZaynHWWJnZOHfPqqyeejYiUn/NmgUHH7xmoDniiHBL\nLYmBRvNxqkfBRkTqpxEjQtqZMWNKll98MQwenNRkmkXzcQoKC3B89XwcBZzyKdiISP3iDvfeC4cf\nDsuXh/0ivXvDo48mPZmm5uNUn4Y+i0j9sXBhGFX2xhthYbOJE4uPvfEGnHRSrTRD83GqTz0bEakf\nJk2CffaBt94Ka9AkBprRo2st0ED58240H6d8CjYiUvf17w977x2yAXTrFhY4A1h//TKTaSab5uNU\nn4KNiNRdy5fDNdfA6adDu3Zw2mlhsRcI+7Nnwx//WOvN0nyc6tM8m0jzbETqmB9/DEFm5Ei46KLw\nvKZv33Ds1FPhpZfKzXEmtUfzbESk/ho1KmRr/vxzePJJ+M9/igNNTg68+qoCTT2j0WgiUne4wyOP\nhFtnmZnw3HPQtSvMmxeOVyGZptRN6tmISN3w228hsFxxBRxzDDz/PHTuvDrQDO8xjMye59GkSYhD\neZo/Wa+oZyMiqTdlShi6/PXX0KtXGF12wAGrD79z5wROv21XFsd5lAUF0D0soEm2nsnXC+rZiEhq\nvf12WBbghx9gyBBITw/ZAQA22wy+/55LnywONEUWLw6Pb6R+SFqwMbNnzWyOmU1MKNvUzIaZ2ZT4\n3jqWm5k9bGZTzWy8me2ZcM45sf4UMzsnoXwvM5sQz3nYzKyizxCROmblSrjhBjjhBNhxx7B08+jR\ncNZZ4fiBB8K0abDVVswoZ2J+eeVS9ySzZ/M80LlUWQ9guLvvAAyP+wBHAzvEV3fgCQiBA7gZ2BfY\nB7g5IXg8AVyQcF7nSj5DROqKuXPD85g77wz3w0aMCIHn5pvD8QsuCGUxmWZ6ORPzyyuXuidpwcbd\nRwI/lyo+Hnghbr8AnJBQ/qIHo4FWZrYVcBQwzN1/dvdfgGFA53hsE3cf7WGi0IulrlXWZ4hIXTB2\nbMjW/PHH0KdPSKp50EHw8svh+P33w1NPlUim2asXpJWcsE9aWiiX+qG2Bwhs6e4/xO0fgS3j9jbA\nzIR6s2JZReWzyiiv6DNEJJXc4emn4bLLYOut4dNPYcstw2vJklCnnGSaRYMAcnLCrbP09BBoNDig\n/kjZaDR3dzNLavqCyj7DzLoTbtuRrv64SPL8/jtcckmYN3PUUWHc8vffQ9u2xXVGj64wx1l2toJL\nfVbbo9F+irfAiO9zYvlsYNuEem1jWUXlbcsor+gz1uDuue6e5e5Zm2+++Vp/KZHGLC8vzHspd/7L\ntGmw//4h0Nx0E/z73/DVVyG3WZEUJNOU2lXbwWYgUDSi7Bzg7YTys+OotA5AYbwVNhQ40sxax4EB\nRwJD47FfzaxDHIV2dqlrlfUZIlLD8vLC8/2CgnCXrGj+y+qAM3hweD4zbRoMGgQ9e4a0M0VDm//4\nR5g/PyXJNKWWuXtSXkA/4AdgOeGZSjegDWGE2BTgfWDTWNeAx4DvgAlAVsJ1zgemxtd5CeVZwMR4\nzqMUJxUt8zMqe+21114uItWTkeEewkzJV2b6SvdbbnE3c999d/epU91XrXL/17+KK3Xp4r5kSaq/\ngqwjIN+r8DtWWZ8jZX0Wqb4mTUquygzQmp95ibM4lnfh7LPhiSegWTP4619hwIBQ6frrw7DnMD1O\n6rGqZn1WuhoRWWvp6eHWWZE9+JI3OJm2zApB5u9/h19/DRmcJ08Olfr0gfPPT02DJWWUrkZE1lri\n/JezeYFP2Y8WtpwPbvkYLrwwLG7WqlVxoBk2TIGmkVLPRkTWWnY2NFm+lJWXXUnXRU/y6XqH8v0D\nr3DKxVuENWj22KO48oQJsOuuqWuspJSCjYisvZkz+esTp8CisXD99ex3++3h+cx774X5NADNm4d7\nbVttldq2SkrpNpqIrJ3hw8OzmEmTwsz/u+4KgeaZZ4oDTfv28PPPCjSiYCMi1eQeAsuRR8IWW4Sl\nm086KZRfd11IoglhJNrnn69OpimNm4KNSD1U6az9ZCksDIHln/+EU0+FMWNgp51g2bKwuuZ994V6\nd98dVtpMSKYpjZue2YjUM0Wz9mt91cqJE0OgmTYNHnwQLr88zJNZsCA8+J8dM0aVk0xTGjf1bETq\nmZwcan/Vyn79Qu6yhQvhgw/giitCoJkxA1q3Lg40o0cr0EiZFGxE6plaXbVy+XK48ko488wwGOCL\nL8IKmgBffgkZGcV1lUxTKqBgI1LP1NqqlT/8AIceCg89FALOBx8UjyobPDgEH4DNNlMyTamUgo1I\nPVMrq1aOHAnt27Mi/0su26wfTR7qTeYOzcNAhCeeCIMBAA47DGbNgk03rcEPl4ZIwUaknsnOhtzc\ncAfLLLzn5tbQ4AB36N0bDjuMX60lHWwsj847Iy4f4Px67mVw8cWh7pVXwvvvw3rr1cAHS0On0Wgi\n9VBSVq1ctAi6dYP+/eHEE9kv/3m+XrIJAC1YyjCO4KAVH4e6ubnF82lEqkDBRkRCoswTTwzvd98N\n113HN01D+v9W/MJ0MmnJrwAcwTCGXXB4Klsr9ZCCjUhjN2AAnHtuuB02bFh4DkMccFAwnelst7rq\nrkxgUYaSaUr16ZmNSGO1YkVYxOzkk2HnncOw5hhoAJ7427gSgWYrvmda2q41OxBBGo0Kg42ZNSm1\nn21mF5pZWnnniEg9MGdOyG12zz1w0UVh9Nm22xYfHzSIo28Miy9Ob/YnNmYh62VsVXMDEaTRqaxn\n828z2xnAzHKAs4HdgVeS3TARSZLRo8Mcmc8+C/nLHn+85Iiy3r3huOPC9sknk7lkMgt9I6ZPV6CR\ntVdusDGzg4EdgM3j9lnAU4RA82czO8jManoamYgki3sILAcdBC1ahGBzzjnFx1etCqtoXn112L/1\nVnjtNSXTlBpR2QCBJsAmwIbASmAeYMDv8bglr2kiUmMWLw7LNL/0Ehx7bHhv3br4+NKl0LFjSEED\n8OqrcNppqWmrNEjlBht3/8jM+gJ3AxsBt7j7SDNrA8xz95G11UgRWQfffReSY06YEHorOTlhbYIi\nP/8MbdoU748erRxnUuMq7Nm4+01m1g9Y7u5TY3ETQLO5ROqDd96Brl1DcHn3XejcueTxadNK5jT7\n7jvlOJOkqHTos7tPSgg0uPtcd/9fcpslIutk5Uq46Sbo0iUEj3Hj1gw0n39eHFjMlExTkkrzbEQa\nmvnzw3OZ226D886DUaNgu+1K1hkwAPbZJ2xnZcHvvyuZpiSVgo1IQzJuHOy1F4wYEfKX9ekDG2xQ\nss6dd4aJnBCW+Bw7Vsk0JemUrkakoejTBy65BLbYAj75BPbeu+TxVavCImivvhr2H3001BepBZUG\nGzPbH7gFyIj1DXB3181dkbpgyRK47DJ45hk44gh4+eWwoFnpOrvuGgYAAAwZAkcdVfttlUarKj2b\nPsBVwDjCXBsRqSsKCsItsXHj4IYbwtDm0pMw588vGXwmTAiBR6QWVSXYFLr74KS3RESq57334K9/\nDQk133oLjj9+zTrffQfbb1+8//33xUs7i9SiqgwQGGFm95pZRzPbs+iV9JaJSNlWrQprQHfuDFtv\nDfn5ZQeazz4rDjStW8PChQo0kjJVCTb7AlnAHcD98XVfMhsl0hDkTcgj88FMmvRsQuaDmeRNyFv3\niy5YACecAP/6V+jVjB4NO+ywZr1+/WC//cL2EUeELM8bbbTuny+yliq9jebuh9ZGQ0QakrwJeXQf\n1J3FyxcDUFBYQPdB3QHI3m0tUyePHx/SzhQUwCOPhJFkVkZ6whtvhNtvD9s9esAdd5RdT6QWlRts\nzKyru/c1s6vLOu7uDySvWSL1W87wnNWBpsji5YvJGZ6zdsGmb98wJ6Z1a/joo+JeS6JVq0LGgHff\nDfsvvRRS1YjUARX1bDaM7xvXRkNEGpIZhTOqVV6uZctCyv/HHoODD4ZXXoE//GHNer//DhkZMHdu\n2B81quyAJJIiFWV9fiq+96y95og0DOkt0ykoLCizvMpmz4ZTTw0P+q+5hpfb3cUNHZoxYwakp4cx\nAtnZhACzxRbF5ymZptRBSlcjkgS9OvUirXnJ1dPTmqfRq1Ovql3gww/Daprjx0P//uS1v48LLmpG\nQUFYA62gINxVG3j/lJKBRsk0pY5SsBFJguzdssntkktGywwMI6NlBrldcit/XuMO990Hhx8eEmN+\n/jmceio5OWH9s0R7LR7JcdfuGHb++MeQJUDJNKWOqjTYmFmNrwlrZleZ2ddmNtHM+pnZ+ma2nZmN\nMbOpZvaqmbWIddeL+1Pj8cyE6/wzlk82s6MSyjvHsqlm1qOm2y9SFdm7ZTP9yumsunkV06+cXnmg\nWbgw3Da77jo48cSQIHPnnQGYUepRz7k8x0gODjtnnAFTpyqZptRpVenZTImTOnepiQ80s22Ay4Es\nd98VaAqcQVgRtLe7bw/8AnSLp3QDfonlvWM9YnvOAP4P6Aw8bmZNY3B8DDga2AX4a021XSRpJk0K\nKf/feiv0bPr3h42Lx+akJzzquZ+reY7zAbij1T1hTo2GNksdV5VgszvwLfCMmY02s+5mtsk6fm4z\nYAMzawakAT8AhwGvx+MvACfE7ePjPvF4JzOzWP6Kuy9192nAVGCf+Jrq7v9z92XAK7GuSN3Uv3/I\n0Pzzz/D++3DNNWsEj169YKMNVjKSA7ma3gCc0mIgGY9el4oWi1RbVVbqXOjuT7v7fsD1wM3AD2b2\ngpltX8npZV1vNiEDwQxCkCkkJPlc4O4rYrVZwDZxextgZjx3RazfJrG81Dnlla8hBs58M8ufWzRk\nVKS2rFgRAsvpp0O7dvDFF3DIIWVWzT5xMQuWbcCBfAJA563+w4nPdgmj0UTqgaosMdAUOBY4D8gk\npKvJAw4E3gV2rM4HmllrQk9jO2AB8BrhNlitc/dcIBcgKyvLU9EGaaR+/DE8a/noI7j0Urj/fmjR\nouy6c+bAlluy+uHp7NkM2Xrr2mqpSI2oStbnKcAI4F53/zSh/HUzO2gtPvNwYJq7zwUwswHA/kAr\nM2sWey9tgdmx/mxgW2BWvO3WEpifUF4k8ZzyykVSb9SoMBBgwYLKZ/n/97+rBwkAYRCBcpxJPVTh\nbbTYq3ne3buVCjQAuPvla/GZM4AOZpYWn710Ar4hBLRTYp1zgLfj9sC4Tzz+gbt7LD8jjlbbDtgB\nGAt8DuwQR7e1IAwiGLgW7RSpWe4hp9khh0BaWkiiWVGgGTasONBkZcHy5Qo0Um9VGGzcfSVQo4k4\n3X0M4UH/F8CE2IZcwvOgq81sKuGZTJ94Sh+gTSy/GugRr/M10J8QqIYAl7j7ytgzuhQYCkwC+se6\nIqnz228hsFx+ORx9dFgWoF278us//jgceWTYvuSSMAy6mVZxl/rLQiehggpmvQi3rl4Ffisqd/cv\nktu02pWVleX5+fmpboY0RFOmhGzNX38dsjH36AFNyvk7zz2kBnjmmbCfmwsXXFB7bRWpJjMb5+5Z\nldWryp9KRdn8bk0oc8JQZRGpyNtvw9lnQ/PmMGRIcW+lLCtXFqeogZCy5uCDa6WZIsmm9WxEkmHl\nyrCuzJ13huctr78esjKX57ffSj6PmTKl5HLOIvVclW4Cm9mxhJn66xeVufut5Z8h0ojNmxdW0Xz/\n/XAL7OGHYf31y6//008llw2YP185zqTBqUputCeB04HLAANOBSr4E02kERs7NtwK+/jj8NwlN7fi\nQDNhQnGg2XhjJdOUBqsq6Wr2c/ezCfnJegIdKTmPRUTcQ2A58MDw8H/UKOjWreJz3nmneERa585Q\nWKhkmtJgVSXY/B7fF5vZ1sBywux/EYGwSma3bvD3v8Ohh8K4cbDXXhWfc++9YQlnCM92Bg9WMk1p\n0KryzOYdM2sF3EuYG+PAM0ltlUh9MW0anHwyfPkl3HRTeDWtYFUO9/A859VXw37//iGbgEgDV5XR\naLfFzTfM7B1gfXcvTG6zROqBwYPDuszuMGgQ/OUvFddfsSIscjYz5onNz6+8ByTSQJQbbMzspAqO\n4e4DktMkkTpu1Sq47Tbo2TM8c3njDfjTnyo+Z9GiEuvTMGsWbFNmMnKRBqmink2XCo45oGAjjc8v\nv4S0M+++C2edBU8+GfKcVWT2bGjbtnhfyTSlESo32Lj7ebXZEJE678svw/OZWbNC7rILL6z8of64\ncWFSJ0BmZpisqRxn0ghpUqdIVbzwQggubdrAyJHQoUPl57z2Gpx2WtjOzg7LCWjEmTRSmtQpUpGl\nS+Gii+Dcc6Fjx7CaZlUCzc03FweaBx6Avn0VaKRRq1IiTndvZ2bj3b2nmd2PntdIYzBzJpxySsgK\ncP31IWNzZbfA3MMSAkOHhv3Bg8OETZFGrirBpvSkzvloUqc0dMOHh2Wbly4No81OKndwZrEVK6B1\n6zDyDGDSJPjzn5PbTpF6oioZBEpP6pwO9Etmo0RSxh3uuissBbDFFvD551ULNAsXhmUEigLNvHkK\nNCIJNKlTpEhhIZx3Hrz5Jpx+ekikWZUhygUFYaRZkSVLlONMpJRyezZmtreZ/SFh/2zCMsy3mZnS\n0krDMnEi7L03DBwIvXtDv35VCzSfflocaLKywoRPBRqRNVR0G+0pYBmAmR0E3AW8CBQCuclvmkgt\n6dcP9t033AobMQKuvLJqI8eefx723z9sX3ZZuOWmEWciZaoo2DR195/j9ulArru/4e43AlpCUOq/\n5ctDYDnzzLAGzRdfhCUCypCXFzowTZqE90lHXx1uuQH06RMWSBORclX0zKapmTVz9xVAJ6B7Fc8T\nqft++CFkWx41KgSce+4JD/jLkJcH3bvD4sUAzisFHdm5YEw4+Mknxb0bESlXRUGjH/CRmc0jDH/+\nGMDMtifcShOpnz7+OEy4/PXXcAvtjDMqrJ6TEwJNM5aznBary/ffZjqj9s8IQ577VTBAc5ddlN1Z\nGr2KcqP1MrPhwFbAe+7u8VATQjYBkfrFHR56CK69NqT6HzYMdt210tNmzICWLGABrVeXbcyv/PZ9\nzOK8fDmcfXb5F2jfPtyiE2nEKpxn4+6j3f1Nd/8toexbd9e/HKlfFi0Ki5ZddVVYIfPzz6sUaAAO\n2mrK6kCzmA1oxnIWsTHp6bHCeuvB1KnhNXky3HBD8Yi0rl3DCDeRRq4qkzpF6rfJk8Nos9deCxM2\nBwyAli2rdu7w4Xz4/Y4ADKYzG/IbK2lGWhr06hWq5H3dj8xBnTj0tu35+rBd4Y47woCDMWNC8s3E\n5QVEGikFG2nYBgwI82fmzIH33gs5zqo6PPmxx+DwwwEYf+JNXJQxGDMjIwNyc0Mi57wJedz5/N94\n4KkCRrwk9pBVAAAXzklEQVQAGy1azjmntyDvyYthn32S+MVE6heNKpOGacWK8GT/nnvCL/3XX4dt\nt636+eefD889F7Zfe412p5zC9NJ1CgtZdMXFjBu5hOVNIOcweKAjLGm+jI8++BfZ7brW0JcRqf8U\nbKThmTMnjDAbMSIsD9C7d9Vn9bvDTjuFRc4gLJi2xx4l66xYEVLZ3HQTF8z9lef3gJxO8GPCqs8z\nCmfUzHcRaSB0G01qTOmJj3l5KWjE6NHheclnn4UZ/o8/XvVAs2xZaHxRoPnxxzUDzbBhYXTZRRfB\nzjvT5eo/0O2EkoEGIL1lOiJSTMFGakTRxMeCgtA5KCgI+7UWcNzhiSfgoIOgRYsQbM45p+rnz5tX\nMij9/jtsuWXx/uTJYRTbkUfCb7+F23IffsiZ595HWvO0EpdKa55Gr0691vELiTQsCjZSI4omPiZa\nvDiU14QKe02LF4eVNC++GI44AsaNW7NHUpGvv4bNNw/b224bkmmuH1dA/+WXMFx6113ho4/g7rvh\nm2/g5JPBjOzdssntkktGywwMI6NlBrldcsneLbtmvrhIQ+Huermz1157uaw9M/fQvSj5Mlv3a/ft\n656WVvK6aWmh3KdOdW/XLnxQz57uK1dW7+IDBxZf9Mwzi8uXLXN/+GH3TTd1b9LEvXt39x9/XPcv\nI9LAAPlehd+x6tlIjUgv5xFFeeXVUV6vafhV74Q0MDNnwr//DTfdFLo+VXXXXXDccWG7d+/i7tLg\nwdCuHVx+eXg+8+WX8NRTJW+riUi1KNhIjejVC9JKProoMfFxXcwoNbCrCSvpyU08O7dLSDszbhwc\nfXT1LnrSSfDPf4btoUNDMs6vv4bOneGYY2DlSnj77TAgoF27df8SIo2chj5LjciOjyhyckJwSE8P\ngSa7Bh5dpKeHAQcAmzKfPLLpzFD6b3gep416DDbYoOoXc4c2bcKzGAgP/jfdFC65JPReNtoIHngg\n7LdoUfG1RKTK1LORGpOdDdOnh+fr06fXTKCB4l7TnoxjHHtxKCO4pEUuy5/sU71As2RJuM0WA82W\n/MRtHf7NsoztQ6C58MKQ3+yqqxRoRGqYejZS52Vnwx8/epY9n76Yn9iCU//wCafft3f1gtkPP8DW\nW6/ePYk3+JgD2PGXKbzXpDNL77yfLv/YpeYbLyKAejZS1y1ZAt270/Hpbqx3+IGkz/2CgT9UM9Dk\n568ONCtoynAOYwAns4JmHM27HLVqMJc9rkAjkkwpCTZm1srMXjez/5rZJDPraGabmtkwM5sS31vH\numZmD5vZVDMbb2Z7JlznnFh/ipmdk1C+l5lNiOc8bKaF4WtSrWUKKCiAAw6Ap58OafuHDIHNNqve\nNV55JSTijAxnD77iUh5hd/7DEMLAgtKDEESkhlVlfHRNv4AXgL/F7RZAK+AeoEcs6wHcHbePAQYD\nBnQAxsTyTYH/xffWcbt1PDY21rV47tGVtUnzbKqmwjkvNWno0DDHZZNN3N96a+2u0aNHyYY2a+bP\nbHylt+LnNeYDZWTUaOtFGg3q6jwbM2sJHAT0AXD3Ze6+ADieEISI7yfE7eOBF+P3Gg20MrOtgKOA\nYe7+s7v/AgwDOsdjm3hY+M2BFxOuJeso2ZkCWLUqjAjo3Dnc+srPh+OPr/51DjkkzKMp0qULTJzI\n+k/0Zlla6xJVa2qItoiULxUDBLYD5gLPmdnuwDjgCmBLd/8h1vkRKJpBtw0wM+H8WbGsovJZZZSv\nwcy6A90B0mti9mEjUN7tphq5DbVgQVheedAgOPPMsGjMhhtW7xqrVkHTpsX7LVuGPGZxXZrsnUJx\nMoZoi0j5UvHMphmwJ/CEu7cHfiPcNlst9kg82Q1x91x3z3L3rM2LcmNJhZKWKWD8eMjKCrP3H34Y\n+vatfqCZMqVkoHnggZBgMwaaIskaoi0i5UtFsJkFzHL3MXH/dULw+SneAiO+z4nHZwOJq161jWUV\nlbcto1xqQFIyBfTtCx06hPtxH34Il11W9dU0IWRo7t4ddtyxuKwogWYzje4XqQtqPdi4+4/ATDOL\nNzToBHwDDASKRpSdA7wdtwcCZ8dRaR2Awni7bShwpJm1jiPXjgSGxmO/mlmHOArt7IRryTrKzg53\ntzIyQjxIXCK52pYtC4HlrLNg7715I+cLMrP3r/ooN/cw2iwtLYxYA9huu1DeqtVaNEhEkqYqowhq\n+gXsAeQD44G3CKPJ2gDDgSnA+8Cmsa4BjwHfAROArITrnA9Mja/zEsqzgInxnEcBq6xNGo1Wy2bN\ncu/YMQwFu+Yaz3t+WZVHufXt6378H0b7KDqWOOGhlv+q+VFxIlIhqjgaLSXBpi6+FGwq1rdvGB5s\nFt7X6Zf6iBHuW2zhvuGG7v37u3u4ZllLFJQekvzGQzP95aZd16h4Cv2TNwxbRMpV1WBjoa5kZWV5\nfn5+qptRJxWtwpk45DktbS1un7nD/fdDjx6www4wYADsvDMQJoiW9b+iWXiQz2+/wb338vut92C+\nisWksSkhx1l7vuAr2q8+JyMjPPgXkeQzs3HunlVZPaWrkUrVyNyahQvh1FPhuuvghBNg7NjVgQbK\nH82Wse0qeOkl2Gkn6NmTt/04/rzTXXQ9vjkzN4EtL2jLV7t9U+IcZQMQqXsUbKRS6zy3ZtIk2Gcf\nePNNuPdeeO012HjjElXKGuV22Hqj+LxZhzD3Zuut4ZNPuPSQLhScnMPg9nNIvxrmbDMLunSH3YpH\nE2jKlEjdo2AjlVqnuTWvvRYCzc8/w/vvw7XXljmsOXGUWybTGZR2OsOXHsBmS2bDiy/C6NGw//5w\neA60KNXNarEYOoVulrIBiNRNCjZSqarOrUlM0PmnjBV8c+y1cNppsNtu8MUXcOihFX5O9nELmZ6d\nw7T1/sxffBDcfDN8+20YGh2Xe/55RTndqZYz1m0YtogklWa8SaWqsgpn4iCCLfmRPjPOYJcZHzH5\niEvZ6Z37K16MbOVKeOGF8AE//ghdu8Kdd0LbtmtUTW+ZTkFhwRrlGa3SNShApA5Tz0aqpLIUL0WD\nCPZjFF+wJ/swlq68xFHfPlJxoPnoo7AEQLduYULmmDFhQEAZgQagV6depDUv2c1Ka55Gr066dyZS\nlynYSI2YUeBcyiN8yCEsJo0OjCaPruUPIvjuOzj55JCdef586NcPRo0Kz3cqkL1bNrldcslomYFh\nZLTMILdLLtm76d6ZSF2m22iy7n77jQFp3Tlh8csMpAtn8yKFhHQxawwiKCwM9+AeegiaN4fbb4er\nr4YNNqjyx2Xvlq3gIlLPqGcj62bKFOjQgeN/78ctzW/nBN5aHWhKDCJYsQKeeipM5rzvvnAf7ttv\nw/23agQaEamfFGxk7b39dlgW4PvvsSFD2OG5HNIzmqyZoPP992HPPeHCC8NEzvx8ePbZMHdGRBoF\n3UaT6lu5Em66Ce64IwSb11+HjAyyKTVw4Ntv4bhrw2Jo220X6p10UvWWDxCRBkE9G6meefPCks13\n3AEXXAAffxy6MYmK1pL5v/8L69PcfTd8800YEKBAI9IoqWcjVTd2LJxyCsyZA888E4YrJ1q+PDyX\nufnmsMTz3/4Gt94KW25Z9vVEpNFQz0Yq5x4ewBx4YJjJP2rUmoFm8GDYffewGNoee4SMAU89pUAj\nIoCCjVTm999DYPn730O6mXHjYK+9io9/8w0cfTQcc0wYcfb222FAwO67p67NIlLnKNhI+aZNC8kv\nn3sObrwR/v1vaNMmHJs3Dy69FNq1g88+gwcegIkT4bjj9FxGRNagZzZStsGDw9CyVavCaLK//CWU\nL1sGjz0WnsUsXBiGM99yC2y2WUqbKyJ1m3o2UtKqVSGQHHtsmP4/blwINO4h6Oy6a5jxv+++MH48\nPPqoAo2IVErBRor98gt06RJGk3XtCp9+Cn/6UwgqRxwRbpE1bQrvvgtDhsAuu6S6xSJSTyjYSPDl\nl+HB/7Bh8PjjIeX/okVhYED79uH4I4+EwHP00alurYjUMwo2EgLLfvuF5zEjR8L554flm7ffPqSV\nufxymDo1DAho3jzVrRWRekjBpjFbuhQuugjOPRc6dgzPZ2bPDrfHrr8+pP+fOBF694bWrVPdWhGp\nxxRskiRxieTMzLBfp8ycCQcdBE8+Cf/4R0gpc9ppIUNAWhq89x4MHAg77ZTqlopIA6Chz0mQuEQy\nQEFB2Ic1V7hMieHD4YwzQs/m0UdDj2bffcMcmiefDJM4m+l/DRGpOebuqW5DnZCVleX5+fk1cq3M\nzBBgSsvICEsqp4w73HMP3HBDaMwBB8CAAeFZzZVXhrVlWrZMYQNFpL4xs3HunlVZPf35mgTlLYVc\n7hLJtaGwEM47D958E1q0gF9/hZdeCin/77knDHEWEUkSBZskSE8vu2ezxhLJtWXixBBUpkwJ+8uW\nwbbbhvVlDjkkRY0SkcZEAwSSoFev8Iw9UYklkmtTv37heUxRoNlyS+jTJ6yWqUAjIrVEwSYJsrND\nRv6MDNZcInkdVXmU2/Ll4TnMmWeGkQrrrRee1UyZEubRNG267o0REaki3UZLkuzsmh95VuVRbj/8\nEIYxf/JJ2D/9dLjrrhCdRERSQD2beiQnpzjQFFm8OJSvNno07LlnCDR77x3eX3lFgUZEUko9m3qk\nSqPcevcO99hefDF0d5ro7wkRST0Fm3qkSqPcXnopBBhNyhSROkR/9tYjVRrl1qKFAo2I1DkKNvVI\nMke5iYgkk/4ErmeSMcpNRCTZUtazMbOmZvalmb0T97czszFmNtXMXjWzFrF8vbg/NR7PTLjGP2P5\nZDM7KqG8cyybamY9avu7iYhISam8jXYFMClh/26gt7tvD/wCdIvl3YBfYnnvWA8z2wU4A/g/oDPw\neAxgTYHHgKOBXYC/xroNXt6EPDIfzKRJzyZkPphJ3oS6tq6BiDRWKQk2ZtYWOBZ4Ju4bcBjweqzy\nAnBC3D4+7hOPd4r1jwdecfel7j4NmArsE19T3f1/7r4MeCXWrXF1ac2avAl5dB/UnYLCAhynoLCA\n7oO6K+CISJ2Qqp7Ng8A/gFVxvw2wwN1XxP1ZwDZxextgJkA8Xhjrry4vdU555TWqaDZ/QUHI3F80\nmz9VASdneA6Ll5ec8bl4+WJyhueUc4aISO2p9WBjZn8B5rj7uNr+7DLa0t3M8s0sf+7cudU6t0qz\n+WvRjMKyZ3yWVy4iUptS0bPZHzjOzKYTbnEdBjwEtDKzotFxbYHZcXs2sC1APN4SmJ9YXuqc8srX\n4O657p7l7lmbb755tb5EXVuzJr1l2esXlFcuIlKbaj3YuPs/3b2tu2cSHvB/4O7ZwAjglFjtHODt\nuD0w7hOPf+BhedGBwBlxtNp2wA7AWOBzYIc4uq1F/IyBNf09ylubprI1a5L1nKdXp16kNS854zOt\neRq9OqViXQMRkZLq0qTO64GrzWwq4ZlMn1jeB2gTy68GegC4+9dAf+AbYAhwibuvjM91LgWGEka7\n9Y91a9TarFmTzOc82btlk9sll4yWGRhGRssMcrvkkr2bJuWISOpZ6CRIVlaW5+fnV+ucvLzwjGbG\njNCj6dWr4gmXmZll5zbLyIDp06v10SIidYKZjXP3rErrKdgEaxNsqqtJk9CjKc0MVq1as1xEpK6r\narCpS7fRGry1fc4jIlLfKdjUorV5ziMi0hAo2NQiZW0WkcZKWZ9rmbI2i0hjpJ6NiIgknYKNiIgk\nnYKNiIgknYKNiIgknYKNiIgknTIIRGY2FygjmUydtRkwL9WNSDH9DPQzaOzfH1L/M8hw90rT5ivY\n1FNmll+VFBENmX4G+hk09u8P9ednoNtoIiKSdAo2IiKSdAo29VduqhtQB+hnoJ9BY//+UE9+Bnpm\nIyIiSaeejYiIJJ2CjYiIJJ2CTT1jZtua2Qgz+8bMvjazK1LdplQws6Zm9qWZvZPqtqSCmbUys9fN\n7L9mNsnMOqa6TbXNzK6K/wYmmlk/M1s/1W1KNjN71szmmNnEhLJNzWyYmU2J761T2cbyKNjUPyuA\na9x9F6ADcImZ7ZLiNqXCFcCkVDcihR4Chrj7n4HdaWQ/CzPbBrgcyHL3XYGmwBmpbVWteB7oXKqs\nBzDc3XcAhsf9OkfBpp5x9x/c/Yu4vZDwS2ab1LaqdplZW+BY4JlUtyUVzKwlcBDQB8Ddl7n7gtS2\nKiWaARuYWTMgDfg+xe1JOncfCfxcqvh44IW4/QJwQq02qooUbOoxM8sE2gNjUtuSWvcg8A9gVaob\nkiLbAXOB5+KtxGfMbMNUN6o2ufts4D5gBvADUOju76W2VSmzpbv/ELd/BLZMZWPKo2BTT5nZRsAb\nwJXu/muq21NbzOwvwBx3H5fqtqRQM2BP4Al3bw/8Rh29dZIs8bnE8YTAuzWwoZl1TW2rUs/DXJY6\nOZ9FwaYeMrPmhECT5+4DUt2eWrY/cJyZTQdeAQ4zs76pbVKtmwXMcveiHu3rhODTmBwOTHP3ue6+\nHBgA7JfiNqXKT2a2FUB8n5Pi9pRJwaaeMTMj3Kuf5O4PpLo9tc3d/+nubd09k/BA+AN3b1R/0br7\nj8BMM9spFnUCvklhk1JhBtDBzNLiv4lONLJBEgkGAufE7XOAt1PYlnIp2NQ/+wNnEf6i/yq+jkl1\no6TWXQbkmdl4YA/gjhS3p1bFXt3rwBfABMLvsnqRtmVdmFk/4DNgJzObZWbdgLuAI8xsCqHHd1cq\n21gepasREZGkU89GRESSTsFGRESSTsFGRESSTsFGRESSTsFGRESSTsFGpJaZ2Xpm9n4ctn56ituy\ntZm9nso2SOPQLNUNEKmPzKypu69cy9PbA83dfY+abFN1mVkzd/8eOCWV7ZDGQT0badDM7FYzuzJh\nv1fRGkBmdp2ZfW5m482sZ0Kdt8xsXFwrpXtC+aJ4vTFARzO7K64rNN7M7ivjszeN1xpvZqPNrJ2Z\nbQH0BfaIPZs/lTrnT2Y2JH7+x2b251j+tpmdHbf/bmZ5cftDM3vQzD6N67rsE8s3jGufjI3JOo+P\n5eea2WtmNgh4z8wyi9ZGiWsE3ZvwM/l7LD8kfk7R+jl5cdY+ZrZ3/Oz/xM/auLzrSCPn7nrp1WBf\nQCbwRdxuAnwHtAGOJMw4t1j+DnBQrLdpfN8AmAi0ifsOnBa32wCTKZ4Y3aqMz34EuDluHwZ8FbcP\nAd4pp73DgR3i9r6EdDwQMvlOBQ4Evk1o44fA03H7IGBi3L4D6FrUtnjOhsC5hNxqmyb8fIrO6Q78\nK26vB+QTEl0eAhQCbePP6jPgAKAF8D9g73jOJoS7JWVeJ9X/L+iV2pduo0mD5u7TzWy+mbUn/ML+\n0t3nm9mRhIDzZay6EbADMBK43MxOjOXbxvL5wEpCAlQIv3yXAH0srBZa1oqhBwAnx3Z8YGZtzGyT\n8toaM3nvB7wWOw4Qflnj7j+Z2U3ACOBEd09c06RfrDPSzDYxs1bxux1nZtfGOusD6XF7WKnzixwJ\ntDOzottqLeN3XwaMdfdZsZ1fEYJUIfCDu38eP//XeLy860wr77tLw6dgI43BM4S/6P8APBvLDLjT\n3Z9KrGhmhxDyS3V098Vm9iHhFzXAEo/Padx9Rbxl1YmQEPRSQu9lXTQBFnj5z3J2IwS9rUuVl845\n5YTvd7K7T048YGb7EpYkKIsBl7n70FLnHAIsTShaScW/O8q8jjRuemYjjcGbhKV09waKfgEOBc6P\nvQnMbJv4PKUl8EsMNH8mLL29hnheS3d/F7iSkAyztI+B7Fj/EGCeV7D2UDw2zcxOjeeYme0et/cB\njiYMLrjWzLZLOPX0WOcAwiJihfH7XZbwbKV9BT+fIkOBiywsYYGZ7WgVL8o2GdjKzPaO9Te2sGpm\nda8jjYB6NtLgufsyMxtB6DUU9UzeM7Odgc/i7+NFQFdgCHChhWzKk4HR5Vx2Y+BtM1uf8Jf8VWXU\nuQV4Nl5rMcVp4CuSDTxhZv8CmgOvmNl/gaeB89z9ezO7Jl63qCf1i5l9Snhmcn4su42woul4M2tC\nuIX1l0o++xniM64YpOZSwRLD8ed6OvCImW0A/E7oFVbrOtI4KOuzNHjxl+0XwKnuPiXV7alJ8Tbf\nte6en+q2iFREt9GkQTOzXQijuIY3tEAjUp+oZyMiIkmnno2IiCSdgo2IiCSdgo2IiCSdgo2IiCSd\ngo2IiCTd/wPFVBwuyVOlMgAAAABJRU5ErkJggg==\n", 810 | "text/plain": [ 811 | "" 812 | ] 813 | }, 814 | "metadata": {}, 815 | "output_type": "display_data" 816 | } 817 | ], 818 | "source": [ 819 | "plt.scatter(X_train, y_train, c='b')\n", 820 | "plt.scatter(X_test, y_test, c='g')\n", 821 | "plt.plot(X_test, dtr.predict(X_test), c='r')\n", 822 | "plt.title(\"Tree regression\")\n", 823 | "plt.xlabel(\"years of experience\")\n", 824 | "plt.ylabel(\"Salary in $\")\n", 825 | "plt.show()" 826 | ] 827 | }, 828 | { 829 | "cell_type": "code", 830 | "execution_count": 64, 831 | "metadata": { 832 | "collapsed": false 833 | }, 834 | "outputs": [ 835 | { 836 | "data": { 837 | "text/plain": [ 838 | "array([ 116969., 56957., 113812., 43525., 67938., 67938.])" 839 | ] 840 | }, 841 | "execution_count": 64, 842 | "metadata": {}, 843 | "output_type": "execute_result" 844 | } 845 | ], 846 | "source": [ 847 | "dtr.predict(X_test)" 848 | ] 849 | }, 850 | { 851 | "cell_type": "code", 852 | "execution_count": 71, 853 | "metadata": { 854 | "collapsed": false 855 | }, 856 | "outputs": [ 857 | { 858 | "name": "stdout", 859 | "output_type": "stream", 860 | "text": [ 861 | "R^2 score for Tree: 0.897047881196\n" 862 | ] 863 | } 864 | ], 865 | "source": [ 866 | "print(\"R^2 score for Tree: \", dtr.score(X_test, y_test))" 867 | ] 868 | }, 869 | { 870 | "cell_type": "markdown", 871 | "metadata": {}, 872 | "source": [ 873 | "### Improve Tree with AdaBoost" 874 | ] 875 | }, 876 | { 877 | "cell_type": "code", 878 | "execution_count": 65, 879 | "metadata": { 880 | "collapsed": true 881 | }, 882 | "outputs": [], 883 | "source": [ 884 | "from sklearn.ensemble import AdaBoostRegressor" 885 | ] 886 | }, 887 | { 888 | "cell_type": "code", 889 | "execution_count": 67, 890 | "metadata": { 891 | "collapsed": false 892 | }, 893 | "outputs": [ 894 | { 895 | "data": { 896 | "text/plain": [ 897 | "AdaBoostRegressor(base_estimator=DecisionTreeRegressor(criterion='mse', max_depth=100, max_features=None,\n", 898 | " max_leaf_nodes=None, min_samples_leaf=1, min_samples_split=2,\n", 899 | " min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n", 900 | " splitter='best'),\n", 901 | " learning_rate=1.0, loss='linear', n_estimators=300,\n", 902 | " random_state=None)" 903 | ] 904 | }, 905 | "execution_count": 67, 906 | "metadata": {}, 907 | "output_type": "execute_result" 908 | } 909 | ], 910 | "source": [ 911 | "ada_reg = AdaBoostRegressor(DecisionTreeRegressor(max_depth=100), n_estimators=300)\n", 912 | "ada_reg.fit(X_train, y_train)" 913 | ] 914 | }, 915 | { 916 | "cell_type": "code", 917 | "execution_count": 68, 918 | "metadata": { 919 | "collapsed": false 920 | }, 921 | "outputs": [ 922 | { 923 | "data": { 924 | "text/plain": [ 925 | "array([ 113812., 58820., 113812., 43525., 67938., 67938.])" 926 | ] 927 | }, 928 | "execution_count": 68, 929 | "metadata": {}, 930 | "output_type": "execute_result" 931 | } 932 | ], 933 | "source": [ 934 | "ada_reg.predict(X_test)" 935 | ] 936 | }, 937 | { 938 | "cell_type": "code", 939 | "execution_count": 77, 940 | "metadata": { 941 | "collapsed": false 942 | }, 943 | "outputs": [ 944 | { 945 | "data": { 946 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmczeUewPHPd6xNiy2tmFHJzS0tRintShSl7dZtpEWp\nbuq2b9Ne06Ju2ouSwkRCISG0KBFD96IkihERshRjnfneP77P5MyYDXPmzPJ9v17ndc55fs/5neec\ndL7z/J7n+T6iqjjnnHPRFBfrBjjnnKv4PNg455yLOg82zjnnos6DjXPOuajzYOOccy7qPNg455yL\nOg82zu0gEUkUERWRqrFui3PlhQcbV+aIyEIR2SAi60RktYiMEpGGsW5XcYQ259yyIz7HOhFJLoX3\nXxzxnstE5C0R2T3a7+tcUTzYuLKqo6ruAewP/Aa8FOP2FIuq7pFzAxYRPke4peWtH6XeUfvw/scA\nxwJ3ReE9EJEq0TjvDry/9yzLEQ82rkxT1Y3AEKBZTpmI1BKRfiKyQkQyROR+EYkLx+LC8wwRWR7q\n1QrHaorIABH5XUTWiMg0Edk34px9RGSpiCwRkcdzfkxFpIqIPCsiK0XkZ+Ccnf084bzvichAEfkT\n6BzafJ+I/BTeY5CI1Il4TWsRmRLa/F8RObmY392vwCfAURHnqikiz4nILyLym4i8KiI1I47fG3pE\nS0Tk2nC5MDEcGyAir4jIGBFZD5xU2PlEZB8R+Ti0e5WITIx4n/tE5FcR+UNEfhCRUyPa92LEf4fn\nRKR6OHZG6PXeJyLLgDd29r+DK30ebFyZJiLxwCXAlIjil4BawEHAKUAX4Kpw7MpwOy0c3wN4ORy7\nIryuIVAPuB7YEI69DWwFDgGOBtoC14Rj1wIdQnkScNEufqzzgXdDW94DbsUC2MlAA2Ad8CJAuHw4\nAngIqAvcAwwTkXpFvUl4bTtgfkTxM0BjoDnQBEgEUkL9DsBN2Hd3KHB6Pqe9DHgE2BOYXNj5gDuB\nn4H6wH7A/eF9/g5cBxyjqnsB7bFeIMCD2HfcHPu+WwP3Rrx/A+y/aSPgX0V9B64MUVW/+a1M3YCF\n2A/uGmAL8CtwRDhWBdgMNIuofx3weXg8AfhXxLGm4RxVgauBr4Hmed5vX2ATsFtE2T+Bz8LjT4Hr\nI461BRSoWozPcUaesseBT/OUzQNOiXjeENiI/TGYAvTNU38CkFzAey4O392foY2fALXCsbhw3oSI\n+icB88LjfsBjEcf+Fs6RGJ4PAN6KOF7U+Z4AhgEH52ljU+zSaJu83yGQAbSNeH4OMD88PiO8X/VY\n/xv1247fvGfjyqpOqlobqAl0B74Qkf2AvYFq2I9SjgzgwPD4gHyOVcUCSn9gLDAoXMLpISLVgIRw\nzqXhks8aoBewT8Q5f8lzzl3xS57njYCREe89K5TvE9r2z5xj4Xir0KaCdFDVPbEf82ZYjwisd1ED\n+F/EuT6i4M+Zt515y4o631PYdzUhXCK8E0BV5wK3A48Cy8Mlxf0i2lDQf1uA31R1cyGf3ZVRHmxc\nmaaqWao6DMgCTgRWYj2VhIhqjYAl4fGv+Rzbiv1IbVHVR1S1GXACdmmsC/YDugnYW1Vrh9teqvr3\ncI6lWG8j8py79LHyPF8MnBnx3rVVtaaqLgtt65vn2O6q+kyRb6L6KZCGXeoC601sBppGnKuWqtYK\nx5dil6ly5DcDMLLthZ5PVf9Q1VtVNRHoBNwtIqeEYwNUtTV2Ca4K8GQ4Z37//ZZEPPc09eWUBxtX\npok5D6gDzFHVLGAwkCoie4pIAnAbdokHYCBwq4g0FpE9sEs576nqVhE5TUSOCAP/f2BBK1tVl2KX\nm/4jInuFAfuDc34Yw/vdLCINwsD9PSX8MV8HnhCRRuEz7yMi54Zj/YHzReTMMFGhZvgchfVsIvUE\nzhaRw8N39ybwvIjUD99tAxFpG+oOBrqKSNMwVvZAYScu6nwi0jF8jwKsxf5gyBaRw8JnqIGNmW0A\nssNpBwIPisjeIlI/tGFA3vd25Y8HG1dWjRSRdVhQSAWuUNXvwrGbgPXY4PNX2GD7W+HYW9gP9ERg\nAXaN/6ZwbD9sZtsfwBzgi1AXrIdTHfgeWB3q7R+OvYFdfvsfMAMbhyhJzwFjsMtNf2LjSi0BVHUh\nNqHgAWAFNpB+O8X8fzf0jtLYFjhuxy5NTcUCwCfYwD6qOhJ4Dfvu5gGTwms2FfIWBZ4PG5v5FBtD\nmgS8oKpfYpfeemC91GXYHxI5kwoewb7n2cBM4Bu29XpcOSaq3it1zm1PRI7AgmsNVc0uqr5zhfGe\njXPuLyJyvohUF5G62AD/cA80riR4sHHORboRu7w1H7sEeWNsm+MqCr+M5pxzLuq8Z+Occy7qPJFd\nsPfee2tiYmKsm+Gcc+XK9OnTV6pq/aLqebAJEhMTSU9Pj3UznHOuXBGRYmXU8Mtozjnnos6DjXPO\nuajzYOOccy7qPNg455yLOg82zjnnos6DjXPOVWBpaZCYCHFxdp+WFpt2eLBxzrkKKi0NunWDjAxQ\ntftu3ay8tIOQr7NxzrkKKiUFMjNzl2Vmwr//DRs2bDuWE4QAkpOj0xbv2TjnXAW1aFH+5b//nn8Q\nSknJv35J8GDjnHMVVKMd3MC8oOBUEjzYOOdcBZWaCvHxucvi46Fevfzr72hw2hFRCzYi8paILBeR\n2RFlz4jIDyIyU0Q+EJHaEcfuFZH5IjJXRM6KKG8XyuaLyD0R5Y1F5JtQ/p6IVA/lNcLz+eF4YrQ+\no3POlWXJydC7NyQkgIjd9+4NL7yQfxBKTY1eW6LZs3kbaJenbBxwuKo2B34E7gUQkWbApcDfw2te\nFZEqIlIFeAVoDzQD/hnqAjwN9FTVQ7A947uG8q7A6lDeM9RzzrlKKTkZFi6E7Gy7T04uOAhFa3IA\nRDHYqOpEYFWesk9UdWt4OgVoEB6fBwxS1U2qugDbJfDYcJuvqj+r6mZgEHCeiAhwOjAkvP4doFPE\nud4Jj4cAbUJ955xzQX5BKJpiOWZzNTA6PD4Q+CXi2OJQVlB5PWBNRODKKc91rnB8bai/HRHpJiLp\nIpK+YsWKXf5Azjnn8heTYCMiKcBWIEZrWY2q9lbVJFVNql+/yL1/nHPO7aRSX9QpIlcCHYA2qqqh\neAnQMKJag1BGAeW/A7VFpGrovUTWzznXYhGpCtQK9Z1zzsVIqfZsRKQdcBdwrqpGLikaAVwaZpI1\nBpoAU4FpQJMw86w6NolgRAhSnwEXhddfAQyPONcV4fFFwKcRQc0551wMRHPq80BgMtBURBaLSFfg\nZWBPYJyI/FdEXgdQ1e+AwcD3wBjgRlXNCr2W7sBYYA4wONQFuBu4TUTmY2MyfUJ5H6BeKL8N+Gu6\ntHPOlRVlJUFmaRH/o98kJSVpenp6rJvhnKsEchJkRqaMiY+P/vTjaBCR6aqaVFQ9zyDgnHOlrKAE\nmdHMTRZrHmycc66UFZSDLJq5yWLNg41zzpWygnKQRTM3Wax5sHHOuVJWUILMaOYmizUPNs45V8pi\nkZss1jzYOOcqlPIypbi0c5PFmm8L7ZyrMPJOKS6N7Y5d8XjPxjlXYZTYlOL//Q8mTSq8jiq89BJM\nn76DJ6+cPNg45yqMXZ5SrAqvvgotWxY+Wp+ZCV26wM03w1tv7XA78yovl/52hQcb51yFsUtTijMz\n4Yor4MYboVUr6Ncv/3oLF0Lr1hYRHnvMeje7IOfSX0aGxbqcS38VLeB4sHHOVRg7PaX4p5/g+OOh\nf38LOOPGwd57b19v3Dho0QIWLICPPoL777fuyC6oLNkEPNg45yqMnZpSPHKkBZCZM+HJJ6FvX6hR\nI3cdVejRA9q1gwMOgPR0OPvsEmlzZckm4LPRnHMVSnJyMWeeZWXBww/D44/DbrvB0KFwwQXb11u3\nDq6+Gt5/H/7xD+jTB/bYo8Ta26iRXTrLr7wi8Z6Nc67y+f13OOccCzT77w9ffpl/oJk/38Zvhg61\nns2gQSUaaKDyZBPwYOOcq1ymT7fLZmPHwtFHw9Sp9jyvUaMgKQmWLbO6d95p1+ZKWGXJJuDBxjlX\nefTpYzPJMjLgvPNg4kRo0CB3nexsm2XWsSMcdJCNz5xxRlSbVRmyCXiwcc5VfBs3wrXXwjXXwKZN\ncNddMGzY9pfE1q7llzYt4cEH6XeE0vTSlaT9WcTiTlcsPkHAOVexZWTAhRfa5bOqVeH116Fr1+3r\nzZnD2rPbsN+ipdzUHl4+FtjwC91GWr6b5CMqYHejFHnPxjlXcY0dC8ccY4GmTh1bJ5NfoPngAzj2\nWDav/I02XeDl44AwPJO5JZOUCRVs0UsMeLBxzlU82dk206x9e1i1Cpo0gW++gVNPzV0vK8tWT15w\nATRrxjHXZvNl4vanW7S2gi16iQEPNs65imXNGujUCR54wBZjnnYaTJliASfS6tXQoQM88YSN5Uyc\nSJVGCfmeslGtCrboJQY82DjnKo6ZM2268siR9rxrVxgzBurWzb/ehAnQqxe88QbUqEFqm1Tiq+Ve\n9BJfLZ7UNhVs0UsMeLBxzlUMAwbYAsyffrIFK88+a0GkevXc9QYNsjxoGzfCF19s2/AGmwTQu2Nv\nEmolIAgJtRLo3bG3Tw4oAaKqsW5DmZCUlKTp6emxboZzbkdt3gy33QavvGLP4+Ph3XdtHU2krVvh\nnnvgP/+BE0+09DP77Vf67a1gRGS6qiYVVc97Ns658mvxYjjllG2B5sAD4auvtg80K1bAWWdZoOne\n3S6f7WKgSZuVRuLzicQ9Ekfi84mkzapgewKUMF9n45wrnz77DC69FJYvt+dJSTB8uGVljjR9us02\n++03ePtt20JgF6XNSqPbyG5kbrG9ATLWZvh6nCJ4z8Y5V76owjPPWAqZnEBz4YU2/pI30LzzjqWn\nAdvmuQQCDUDKhJS/Ak0OX49TOA82zrny488/4eKLLd1MdraV3XcfDB6cO3Xy5s12uezKKy3YpKfn\nn2xzJxW07sbX4xTMg41zrnyYMweOPdbS/QNUq2Y9l9TU3LtlLlsGbdrYOM7tt1sWgfr1S7QpBa27\n8fU4BfNg45wr+wYPhpYt4Ycf7Hm9ejbI36VL7npTplgPZsYMGDjQpj9XLfmhaV+Ps+M82Djnyq4t\nW6x3csklsH69lTVtakHlpJNy1+3dG04+GWrWhMmTbfJAlPh6nB3n62wCX2fjXBmzbJkFmYkTt5W1\naWPrY+rU2Va2aZONz7z5JrRrB2lp22cMcFHj62ycc+XXpEmWrTky0Fx3HYwenTvQ5KyzefNNS6j5\n0UceaMooDzbOubJDFV580bIzr1ljkwBEoGdPeO01e55j4kQbn/nuO9sI7fHHoUqVmDXdFc6DjXOu\nbFi/Hjp3hn//21b3x8VBjRowYgTccgtp7wqJiRAnysN1XyT79DbWy5k6Fc4/P9atd0XwDALOudib\nN89W+X/3nc06mz0b9t7bLos1b05amuXLzM7cwNtcR5fV/fmoyrlk3t6PfxxWK9atd8XgPRvnXGwN\nH26pZpYutQkA6elwxBHWY2neHLDhmPqZC5lEazozgAd4lHOzPuCuVA805UXUgo2IvCUiy0VkdkRZ\nXREZJyLzwn2dUC4i8qKIzBeRmSJyTMRrrgj154nIFRHlLURkVnjNiyIihb2Hc66Mycqy1f+dOkFi\nIhx9NIwfbxkCPv88V6LMJhnjSSeJg/iZDnzE4zyAEsciX7BfbkSzZ/M20C5P2T3ABFVtAkwIzwHa\nA03CrRvwGljgAB4CjgOOBR6KCB6vAddGvK5dEe/hnCsrVqywacpPPml5zeLjLdA88IAtxtxtN6sX\n8qCN4SyWsj9JpDOas/86TSNfsF9uRC3YqOpEYFWe4vOAd8Ljd4BOEeX91EwBaovI/sBZwDhVXaWq\nq4FxQLtwbC9VnaK2UKhfnnPl9x7OubJg6lSbRfbll3D33bZr5owZtvnZo49uSz2zbp0tzLzrLhYf\neyFtdpvMTxzy12ni4y1TjSsfSnvMZl9VXRoeLwP2DY8PBH6JqLc4lBVWvjif8sLewzkXS6q2yv+k\nkyygPP64PV+zBj79FJIjVt/Pn2+7aQ4ZAj16kDDlPXq+sQcJCTYTOiHBXprsC/bLjZjNRlNVFZGo\npi8o6j1EpBt22Y5G3h93Lno2bIAbb4S+fW0Ts9NPh3vvhUMPtRlnjRtvq/vxxxZF4uIsieYZZwBW\n5MGl/Crtns1v4RIY4T5sRsESoGFEvQahrLDyBvmUF/Ye21HV3qqapKpJ9Us4K6xzlUVamo3vx8XZ\nfVreDSsXLLA0/3372rSyZs3s8lmbNvD119sCTXY2PPYYdOhgJ5o+/a9A48q/0g42I4CcGWVXAMMj\nyruEWWmtgLXhUthYoK2I1AkTA9oCY8OxP0SkVZiF1iXPufJ7D+dcCctZ/5KRYVfJMjLs+V8BZ/Ro\nG59ZsAAGDYJZsywbwI03Wo+mVpi6/Mcfts7mwQet+zJpkgUcV3GoalRuwEBgKbAFG1PpCtTDZojN\nA8YDdUNdAV4BfgJmAUkR57kamB9uV0WUJwGzw2teZltS0Xzfo6hbixYt1Dm3YxISVC3M5L4lNspS\nffhhVRHVI49U/fxz1aOOUo2LU33ppdwnmTNHtWlT1SpVVF98UTU7Oyafxe0cIF2L8RvrWZ8Dz/rs\n3I6Li7PwEqkOq+jP5ZzDx3D55XDNNTarbN0625emXcSKiA8/tD1pdtvNjp1ySul+ALfLPOuzcy7q\n8s6rOYpvSSeJMxkHr74K555rwaVGDRufyQk0WVlw//2W0+yww2x8xgNNhebBxjm301JTbb0LQBfe\n4WtOoIZs5tOHJ9qU5osvhqOOgm++gcMPt4qrV0PHjvbia66x7M0NGhT8Jq5C8ESczrmdlpwMcVs2\nkXXTLXRe9zpf1ziNZT36ccH0FOjXDy67DPr0sd0zwSYInH8+LFoEvXrZbAJXKXiwcc7tvF9+4Z+v\nXQTrpsLdd3PCLbdYb+arr+CRRyz9jKUthPfeg6uvhtq14YsvbNGmqzQ82Djnds6ECTbwv2kTDB1q\nYy+tW8OSJZbf7NJLrd7WrbaA89ln4cQTbVvniCSbrnLwMRvn3I5RhaeegrZtYZ99YNo02HNP66ms\nW2cZm3MCzcqVNing2Wdtbc2ECR5oKikPNs6VQ0Wu2o+WtWtt8eW999rlsm++gc8+g/btbWra1KnQ\nqpXVnTHDFnR+9RW8/Ta8/DJUr15KDXVljQcb58qZIlftR8vs2baL5siRlgVgwAAbk7nhBst39tVX\nliETbHJA69b2eNIkuOKKgs/rKgUPNs6VMykpkJmZuywz08qjZuBAOO44+PNP68l07Wqbnj3/PNxy\nC4wYAXvtBVu2wE03WXA5/njbdbNFiyg2zJUXHmycK2cK2p0yKrtWbtliweSyy+CYY+zSWKNG1msZ\nMwZee816OVWqwLJlllzz5Zfh9tvhk0/AE9y6wIONc+VMQbthlPguGUuXwmmnwQsvWMD59FOLaMcd\nZ/ejR8P111vdKVOsB5Oebr2gZ5+Fqj7Z1W3jwca5ciZy1X6OEt+1cuJEOPpotqZ/y017DyTuhZ50\nP2AYWSedArvvDpMnw5lnWt3evS3VTM2aFnRyZqI5F8GDjXPlTHKy/b5HZddKVbssdvrp/CF7cbx8\nw8srL+F+fZSXV17KlKyWDLnzG1tTs2mTzUy47jrrAU2bBs2bl0AjXEXkWZ8Dz/rsKr1162zgf/Bg\n6NSJI6a/zbxfatCHriTzLu/QhW70Zv+EGiyctAQuvNCmPt93Hzz6qI3buEqnuFmf/aKqcw7mzrWc\nZXPn2oLNu+5iedwKPqU9JzCZe3mCp7gHEBplfAktLob16y1zwAUXxLr1rhzwYONcZTdsGFx5pW0D\n8MknNqNs9mzSq3SkXtZvXMT7DOUiQOnOSzzHbVDrIJsw0KxZrFvvygkfs3Gustq6Fe6+2y6HHXaY\nTWtu08amNJ9wAnvvsZGzanzBUC6iJht4myt5iZtZdszZlinAA43bAYUGGxGJy/M8WUSuF5H4gl7j\nnCsHli+33GY9elgGgIkToWFDWyNzzjlw8MHsNmsq1/dpyQkHZvAVJ3I5/fnfhY/ScNoHUKtWrD+B\nK2eK6tmMEpHDAEQkBegCHAkMinbDnHNRMmWKLdCcPNlylr36qg3ud+9uq/87dIAvv4SGDUnebwKT\nNragRa2fiPtoJEcOecASsjm3gwr8VyMipwBNgPrh8eVALyzQ/E1EThaRkl5G5pyLFlULLCefbAkx\nJ0+2tDJr11qAeeUVuOMOG8PZfXdbmNm2rWVpnjbNejzO7aSiJgjEAXsBuwNZwEpAgA3huESvac65\nEpOZaav9+/eHs8+2JJp16sCCBRZofvwR3njDtmlev96mQL/3Hlx0EfTtC3vsEetP4Mq5AoONqn4h\nIgOAp4E9gIdVdaKI1ANWqurE0mqkc24X/PSTTU+eNct2z7z/frsU9vXXlkxzyxYYOxZOPx3mz7cp\n0N9/D08/DXfeuW2nTed2QaE9G1V9UEQGAltUdX4ojgOujXrLnHO77qOPoHNnCy6jRtm+M2D7EVx9\ntSVUGzUKDj3Ucp1ddpnVHTNmWzoa50pAkSN9qjonItCgqitU9efoNss5t0uysuDBB6FjRzjoIJg+\n3QJNdraVd+5sWwBMmQKHHAKPP25jMomJlkzTA40rYb6o07mK5vffLVHa2LFw1VU28L/bbrBhgy3e\nHDzYejWvvQYbN9o6mw8/tADUq9f2WT6dKwEebJyrSKZPt+CxdKll57zmGhtzWbYMzjvPZpX16GGz\nznJS1MybZ9sI3HSTj8+4qPFg41xF0acP3Hgj7LOPbdHcsqWVz5xpM85+/92mNXfqZD2ZLl1sW4AJ\nE2yLAOeiqMgxGxFpLSLjRORHEflZRBaIiI/ZOFdWbNwI115rvZiTTrK0MzmBZtQo21UzK8sWanbs\nCA88YD2anBQ1HmhcKShOz6YPcCswHVtr45wrKzIy7LLZ9Om5U/2r2qWx22+Ho46CESNsLKZjR5t1\n1rWrpaapWTPWn8BVEsUJNmtVdXTUW+Kc2zGffAL//Kcl1PzwQxuTAVs3c9NNNth/wQXQrx/8/LP1\nYBYtgtdft03PfHzGlaLiJDn6TESeEZHjReSYnFvUW+acy192tu0B3a4dHHCATVXOCTRr1liGgF69\n4J574P337VJaq1aWReCLL2xnTQ80rpQVp2dzXLiP3IlNgdNLvjnOVRxps9JImZDCorWLaFSrEalt\nUkk+Yhf3bl6zxgb2R460BZi9e1seM7BMAR062H3fvjaV+Z574JlnbNzm/fdh//13/YM5txOKDDaq\nelppNMS5iiRtVhrdRnYjc0smABlrM+g2shvAzgecmTPtslhGBrz4omVpzumhTJxox1Rh/Hjba6Zd\nO5tpduON8NxzlnzTuRgpMNiISGdVHSAit+V3XFWfi16znCvfUiak/BVocmRuySRlQsrOBZsBA2yc\npXZt+Pxz66nkeOcdm4120EGWnuaPPyApydbW9O1rCzmdi7HCxmxC35w9C7g55wqwaO2iHSov0ObN\n1oO5/HKbzjxjxrZAk51tM9CuvNK2DZg82W6tW9uxr77yQOPKjMKyPvcK94+UXnOcqxga1WpExtqM\nfMuLbckSuPhiCyC33867RzzJfa2qsWgRHNogk0/2u5xG04ZZj6dnTxufeeklOPVUS0lTv37JfSDn\ndpFvuedcFKS2SSW+Wu4cY/HV4kltk1q8E3z+ue2mOXMmDB5M2tHPcu2/qpGRAfvprwz45WQaTPuA\n6cnP2dqadu0s0Nx2G4wb54HGlTmersa5KMgZl9nh2Wiq8J//WC+lSRMLOocdRkqizVw+im8ZSUdq\ns4bzGE6VCfvwYYsWsGoVvPuurbtxrixS1UJvQJWi6uzoDctI8B0wGxgI1AQaA98A84H3gOqhbo3w\nfH44nhhxnntD+VzgrIjydqFsPnBPcdrUokULdS6m/vhD9cILVUH1oovseSCiei4f6jriNYOG2pz/\n6jX01o1UV23cWPW//41hw11lBqRrMX5ji3MZbV5Y1NmsBGIbInIgcDOQpKqHA1WAS7EdQXuq6iHA\naqBreElXYHUo7xnqEdpzKfD3EFxeFZEqIlIFeAVoDzQD/llSbXcuaubMgWOPhQ8+sHUxgwfDnmEe\njiqP1XqWDzif7/g7J/El/+JV3qAbU2qeZos6jzwytu13rgjFCTZHAj8Cb4rIFBHpJiJ77eL7VgV2\nE5GqQDywFFskOiQcfwfoFB6fF54TjrcREQnlg1R1k6ouwHoxx4bbfFX9WVU3A4NCXefKpsGDbabZ\nqlW2RuaOO7atn9m8Ga69lpQ1d/JBlYu4jHcZxKVcR296VL2PJb1HQd26sW2/c8VQnJ06/1TVN1T1\nBOBu4CFgqYi8IyKH7OgbquoS4FlgERZk1mJJPteo6tZQbTFwYHh8IPBLeO3WUL9eZHme1xRUvp0Q\nONNFJH3FihU7+lGc2zVbt1qizEsugebNbVrzaRFrqFetgrPOsq0D7r+fve7tzuS4Ezmc2VxXfygH\nvp3KZZdXiV37ndsBRU4QCJelzgGuAhKB/wBpwEnAx8ChO/KGIlIH62k0BtYA72OXwUqdqvYGegMk\nJSVpLNrgKqlly+DSSy1XWffuNikgcoX/jz9a6pmMDOjfH9as4cxb28AhB8EHn9KrmV8ZduVLcWaj\nzQM+A55R1a8jyoeIyMk78Z5nAAtUdQWAiAwDWgO1RaRq6L00AJaE+kuAhsDicNmtFvB7RHmOyNcU\nVO5c7E2aZOtn1qyxQNK5c+7jn31m2wZUqWJJNPv3t8zNHTva41q1YtNu53ZBoZfRQq/mbVXtmifQ\nAKCqN+/Eey4CWolIfBh7aQN8jwW0i0KdK4Dh4fGI8Jxw/NMwA2IEcKmI1BCRxkATYCowDWgiIo1F\npDo2iWDETrTTuZKlum3RZXw8TJmyfaDp0wfatrWEme+/D3ffbQHmkUdsGwEPNK6cKrRno6pZInIa\n8GhJvaGqfiMiQ4AZwFbgW+xS1ihgkIg8Hsr6hJf0AfqLyHxgFRY8UNXvRGQwFqi2AjeqahaAiHQH\nxmIz3d7vNYhIAAAe9ElEQVRS1e9Kqv3O7ZT1622l/7vvWg+lXz/Lc5YjKwvuvddmorVtC9dfDxdd\nZOM6I0fCOefEru3OlQCxTkIhFURSsUtX7wHrc8pVdUZ0m1a6kpKSND09PdbNcBXRvHmWkfm77+Cx\nxyyoxEVcVFi3zno4w4fDv/4FjRpZzrO//c16M02axK7tzhVBRKaralJR9YozZnNCuI/s3fh+Ns4V\nx/Dhtv9M1aowZoz1WiItXmw9nZkz4ckn4X//g1dftV5N376wxx6xabdzJcz3s3EuGrKy4IEHLIAk\nJcGQIZCQkLtOejqce671bF54wTZC++47ePppuPNO303TVSjFyo0mIudgK/Vr5pSpaomN4zhXoaxc\naTnKxo+3fWZefBFq1sxdZ+hQ2zZgn33gwQctMMXFWe/nzDNj027noqjIRZ0i8jpwCXATIMDFQEKh\nL3Kuspo61bI1f/klvPmm9VYiA42q9XYuusgWcp53Htx1FyQmWk/HA42roIqTruYEVe2C5Sd7BDie\n3OtYnHOqFlhOOsl6KJMmQdeuuets2gRXXWWD/x06QJ061uu57DKr37hxbNruXCkozmW0DeE+U0QO\nwBZU+v8VzuXYsAFuvNEG9M86C9LSoF693HVWrrQZaV9+acFl+nSYPx+efx5uvtnHZ1yFV5xg85GI\n1AaewdbGKPBmVFvlXHmxYIGt9v/2Wxt3eeghW/kf6YcfrCezeLGlqBk50i6tjR9vCzydqwSKMxvt\nsfBwqIh8BNRU1bXRbZZz5cDo0ZCcDNnZFkA6dNi+zvjxNj5TrRqcfDIMGmQZnocOhYZ+NdpVHgUG\nGxG5oJBjqOqw6DTJuTIuO9sWZz7yiA3yDx0KBx+8fb3XX7ckm/vvb2lmxo2Dq6+GV17ZfnaacxVc\nYT2bjoUcU8CDjat8Vq+21f4ff2xTl19/3fKcRcrKsj1pnn/eei+bNlkW59deg+uu8/EZVykVGGxU\n9arSbIhzZd6339r4zOLFtsr/+uu3Dxx//mlrbEaNsh7N8uW2udkXX8Dxx8em3c6VAb6o07nieOcd\nCy716sHEidCq1fZ1MjIs9cysWbal89KlcMIJlj1g//1Lv83OlSG+qNO5wmzaBDfcAFdeaT2TGTPy\nDzTffAPHHWeBBqyH869/2d40Hmic80WdzhXol19sBtnrr9sq/08+sfQyeb33HpxyCvz2mz2vUcPW\n3LzySu7dN52rxHxRp3P5mTDB1sRs2mSXwS68cPs6qjYr7aGHtpU1bAjDhlnyTefcX4rTs8m7qHMh\nMDCajXIuZlThqadsK4B99oFp0/IPNBs32qy0yEBz6qmW38wDjXPb8UWdzuVYu9Zyl33wAVxyiSXS\nzG8/meXLoVMnmDx5W9ltt9nWAFWLNefGuUqnwJ6NiLQUkf0inncBBgOPiUjd0micc6Vm9mxb2T9i\nBPTsCQMH5h9oZs+2iQA5gWa33SwX2n/+44HGuUIUdhmtF7AZQEROBp4C+gFrgd7Rb5pzpWTgQAsg\nf/5ps8duuSX/hZdjxthU5oUL7XnjxhZ0LrusVJvrXHlUWLCpoqqrwuNLgN6qOlRVHwAOiX7TnIuy\nLVsssFx2GRx9tE1rPumkfKtOu+JlstqfYwEJ+PWItjY+c+SRpdli58qtQoONiORcF2gDfBpxzK8X\nuPJt6VI47TTbjvnf/y54PczWrcxt252W/W6iCtkAPMG9NJ3/MWmj/Wqyc8VVWNAYCHwhIiux6c9f\nAojIIdilNOfKpy+/hH/8A/74A95919LL5GftWrjkEpqOGwvAn+zBlbzNMC6EDZCSYkmf2brVLsUV\npFkzaNGi5D+Hc+VIYbnRUkVkArA/8ImqajgUh2UTcK58UbWezB13wEEHWRbmww/Pv+6CBbZlwPff\nA/AjTejEh8yh2V9VFi0KD7ZsgS5dCn7fnEt0zlViha6zUdUpqvqBqq6PKPtRVf3/HFe+rFtnPZhb\nb7X8ZdOmFRxovv7aJgyEQDNut460ZFquQAPQqFF4UKOG7bo5fz7MnWvbPteoYcc6d7YZbs5Vcj72\n4iq+uXNtS+YffrAFm3fdVXCa/7Q023Nm82Z7/vDDLD/oAbZeHweZ26rFx0NqanjJdwNJmZBC4/9m\n8PL4avx9yRbLo/b883DssdH9bM6VEx5sXMU2bJgl0axRw3KbtWmTf73sbHj4YUs/A7DXXhZ4OnQg\nGSDOxmgWLbIeTWqqjdekzUrjybev4bmPN3LBD5BRawtXXFKdtvf9i+TmHmicyyHbhmIqt6SkJE1P\nT491M1xJ2brVokOPHta7GDKk4G2YN2ywgDR4sD1v1syyCBx6aOHvsXYtvc5vxJUT/2BLHDx5Ejx3\nPGysBgm1Elh4y8KS/ETOlUkiMl1Vi8zRVJzcaM6VL8uXW26zHj1sD5qJEwsONMuWWU6znEBz0UUw\nZUrhgWbrVssE3aQJ1372B2lHQJOb4YmTLdAALFq7qODXO1cJebBxJSYtDRITIS7O7tPSYtCIKVPg\nmGNsZX/fvrYVc85gfV4zZ1qvZ+pUa/RTT1nQ2XPPgs8/bpzNLrvhBjjsMDreth9dO8GyPC9pVKtR\n/q93rpLyYONKRFoadOtmm1Wq2n23bqUYcFQtsJx8su0hM3myXRoryKhR0Lq17VlTpw6MHg13313w\nxIG5c20WW9u2sH69XZb7/HMuu/JZ4qvF56oaXy2e1DapJffZnKsIVNVvqrRo0ULdzktIULVf/Ny3\nhISSOf+AAXYuEbsfMCDi4Pr1ql262BuefbbqqlUFnyg7W7VnT9W4OKt/5JGqP/9ccP1Vq1RvuUW1\nalXVPfdUffpp1Q0bcrdt5gBN6Jmg8rBoQs8EHTBzQAEnc67iAdK1GL+xMf+RLys3Dza7RiT/YCOy\n6+ceMEA1Pj73eePjQ8CZP1+1eXN7o0ceUc3KKvhEmzerXnfdtpMkJ1ugKqjuiy+q1q1rgalbN9Vl\ny3b9wzhXwRQ32PjUZ1ciGjWyS2f5le+qlBTIzMxdlpkJE279iOTNnW28ZdQoaN++4JOsWQMXXwzj\nx0OVKvDss5YTLb/LZqNH2/40P/xgU6Wfew6aN9/1D+JcJeZjNq5EpKbaQsdIkQsfd8WiPBO74sji\nER7krRUdLe3M9OmFB5qffrJFluPHQ/36dp/fNgLffQft2sHZZ0NWFgwfbhMCPNA4t8s82LgSkZwM\nvXtDQoL9hick2PPk5F0/d2TvqC6/M4pzeJDHeH/3K2HSJNtXpiATJ1rqmR9+sO2ap0+3qc6RVq6E\nG2+07QKmTLGezOzZcO65BU8YcM7tEA82rsQkJ9u+YtnZdl8SgQa29ZqOYTrTacFpfEb36r3Y/Ppb\ntlNmQd55B844A37/3bZ7/vJLaNjwrynaNWQzj9XtyeaEQ6BXL1uTM3++5U+rXr1kGu+cAzzYuHIg\nORnGJ7/F17Qmjmwu3u8rjn+rG8mdC+h1ZGdbMsycqc+vvgp9+kDNmjZF+1qlecYIZnE4D6y+jc83\nHs/IJ2bCyy/D3nuX2udyrjLxCQKubNu4EW6+mePfeAPOOINGAwcyorCAkJkJl19uOdH22w+GDrWt\nnIP+d85kxIZbacOnfM9htOdjxmS3J+FV6HhXKXwe5yqpmPRsRKS2iAwRkR9EZI6IHC8idUVknIjM\nC/d1Ql0RkRdFZL6IzBSRYyLOc0WoP09ErogobyEis8JrXhTxC+8lqdQyBWRkwIknwhtvWE9lzJjC\nex6//mqLOocNswAzY8a2QPPbb9CtG6OWHs1R/JfuvMSR/I8x2MSCvJMQnHMlrDjzo0v6BrwDXBMe\nVwdqAz2Ae0LZPcDT4fHZwGhAgFbAN6G8LvBzuK8THtcJx6aGuhJe276oNvk6m+IpdM1LSRo71ta4\n7LWX6ocfFl1/xgzVAw+0Bt1wg+qmTVa+caMtxNxzT9WqVfXNPW/R2qyK2uJT5yobyuqiTqAWsICQ\ncTqifC6wf3i8PzA3PO4F/DNvPeCfQK+I8l6hbH/gh4jyXPUKunmwKZ5oZwrQrCzVxx+3RZqHH676\n449Fv+bDDy3i1aih+tZbVpadrTpkiGrjxtbAjh1Vf/ih9IKlc5VEcYNNLC6jNQZWAH1F5FsReVNE\ndgf2VdWloc4yYN/w+EDgl4jXLw5lhZUvzqd8OyLSTUTSRSR9xYoVu/ixKoeCLjeVyGWoNWugUye4\n/37bVXPKFGjSpOD6qrY48/zzoW5dm2121VV2+ezUUy2D8+6721qZESOgadOoTtF2zhUsFsGmKnAM\n8JqqHg2sxy6b/SVEy6hvtKOqvVU1SVWT6tevH+23qxAKygiwy5kCZs60dTCjR8OLL8KAARYoCrJ5\nM1x7Ldx5J5xyiq2fadDAdtlMSoI5c2wbgG+/tenPEaI1Rds5V7BYBJvFwGJV/SY8H4IFn99EZH+A\ncL88HF8CRG5G0iCUFVbeIJ9yVwKikilgwABo1cpmkn3+Odx0U9GLKV94waYz33qr9VreeMN6QQMG\nwB13wLx5cN11UNUnXDpXFpR6sFHVZcAvItI0FLUBvgdGADkzyq4AhofHI4AuYVZaK2BtuNw2Fmgr\nInXCzLW2wNhw7A8RaRVmoXWJOJfbRSV6GWrzZgssl18OLVsyNGUGicmtizfL7bLLLDvAscfC4Yfb\npbezzrIeTY8eUKvWTn5C51xUFGdgp6RvwFFAOjAT+BCbTVYPmADMA8YDdUNdAV4BfgJmAUkR57ka\nmB9uV0WUJwGzw2teJs9khPxuPkGglC1erHr88TZCf/vtmvb25mIP3A8YoHreflN0Evb6/1U5Sk/l\ns+23HnDORR1ldTZaWb15sClcofvJ7KjPPlPdZx/V3XdXHTxYVYs/y23oC7/ou1U6q4L+yn56FX00\njq0+s8y5GClusPF0Na5IJbYLp4bZY2ecYbtjTptmaf8pxiy39evh4Ydpf8uhnJ/1Pqncx6H8SF+u\nJpsqf9XPzLQtCZxzZYsHG1ekgvaT2aEf9T//tMBy5502vXnqVDjssL8OFzSbLaFhNvTvD02bwiOP\nMFzP5W9Nn+L+W9JY91AtuCURjsgd9TwbgHNljwcbV6RdXlszZ44N5H/wATzzDLz/Puy1V64q+c1y\nO73GJKZVbQVdusABB8BXX9H91I5kXJgCtTNA1O47dssVcEpiwzbnXMnyYOOKtEtra95/3wLNqlW2\nadkdd+Q7rTlyllsiCxkZfwkTNp3I3huXQL9+tsCzdWs4IwWq5+lmVc+ENtbNKqkN25xzJcuDjStS\ncdfWRCboPDhhK9+fcwf84x9wxBG2qv+00wp9n+Rz/2RhcgoLavyNDjoSHnoIfvzRpkbH2T/VVVsL\n6E7VWuTZAJwrw3zFmytSzo93SopdOmvUyAJN5I96ziSCzEzYl2X0WXQpzRZ9wdwzu9P0o/8UvhlZ\nVpZtdJaSAsuWQefO8OSTlhEgj0a1GpGxNmO78oTajVi4cBc/qHMuarxn44qlqBQvOZMITmASMziG\nY5lKZ/pz1o8vFR5ovvgCWraErl1te+dvvrEJAfkEGoDUNqnEV8vdzYqvFk9qG7925lxZ5sHGlYhF\nGUp3XuJzTiWTeFoxhTQ6FzyJ4Kef4MILLWHm77/DwIEwaZKN7xQi+YhkenfsTUKtBAQhoVYCvTv2\nJvkIv3bmXFnml9Hcrlu/nmHx3eiU+S4j6EgX+rGW2kA+kwjWrrVrcC+8ANWqweOPw223wW67Ffvt\nko9I9uDiXDnjPRu3a+bNg1atOG/DQB6u9jid+PCvQJNrEsHWrdCrlyXLfPZZuw734492/W0HAo1z\nrnzyYON23vDhls7/11+RMWNo0jeFRglx2yfoHD8ejjkGrr/eFnKmp8Nbb9naGedcpeCX0dyOy8qC\nBx+EJ56wYDNkCCQkkEyeiQM//gjn3gEjR9rg/5AhcMEFRW8f4JyrcLxn43bMypXQrp0Fmmuvtd0x\nExJy11m92vaZ+fvfbX+ap5+G77+3CQEeaJyrlLxn44pv6lTbann5cnjzTZuuHGnLFhuXeegh2+L5\nmmvg0Udh333zP59zrtLwno0rmqoNwJx0kq3knzRp+0AzejQceaRthnbUUZYxoFcvDzTOOcCDjSvK\nhg0WWK67ztLNTJ8OLVpsO/7999C+PZx9ts04Gz7cJgQceWTs2uycK3M82LiCLVhgyS/79oUHHoBR\no6BePTu2ciV07w7Nm8PkyfDcczB7Npx7ro/LOOe242M2Ln+jR9vUsuxsm03WoYOVb94Mr7xiYzF/\n/mnTmR9+GPbeO6bNdc6Vbd6zcbllZ1sgOeccW/4/fboFGlULOocfbiv+jzsOZs6El1/2QOOcK5IH\nG7fN6tXQsaPNJuvcGb7+Gg4+2ILKmWfaJbIqVeDjj2HMGGjWLNYtds6VEx5snPn2Wxv4HzcOXn3V\nUv6vW2cTA44+2o6/9JIFnvbtY91a51w548HGWWA54QQbj5k4Ea6+2rZvPuQQSytz880wf75NCKhW\nLdatdc6VQx5sKrNNm+CGG+DKK+H44218ZskSuzx2992W/n/2bOjZE+rUiXVrnXPlmAebKIncIjkx\n0Z6XKb/8AiefDK+/DnfdZSll/vEPyxAQHw+ffAIjRkDTprFuqXOuAvCpz1EQuUUyQEaGPYftd7iM\niQkT4NJLrWfz8svWoznuOFtD8/rrtoizqv/TcM6VHFHVWLehTEhKStL09PQSOVdiogWYvBISbEvl\nmFGFHj3gvvusMSeeCMOG2VjNLbfY3jK1asWwgc658kZEpqtqUlH1/M/XKChoK+QCt0guDWvXwlVX\nwQcfQPXq8Mcf0L+/pfzv0cOmODvnXJR4sImCRo3y79lst0VyaZk924LKvHn2fPNmaNjQ9pc59dQY\nNco5V5n4BIEoSE21MfZIubZILk0DB9p4TE6g2Xdf6NPHdsv0QOOcKyUebKIgOdky8icksP0Wybuo\n2LPctmyxcZjLLrOZCjVq2FjNvHm2jqZKlV1vjHPOFZNfRouS5OSSn3lW7FluS5faNOavvrLnl1wC\nTz1l0ck552LAezblSErKtkCTIzPTyv8yZQocc4wFmpYt7X7QIA80zrmY8p5NOVKsWW49e9o1tn79\nrLsT539POOdiz4NNOVKsWW79+1uA8UWZzrkyxP/sLUeKNcutenUPNM65MseDTTkSzVluzjkXTf4n\ncDkTjVluzjkXbTHr2YhIFRH5VkQ+Cs8bi8g3IjJfRN4TkeqhvEZ4Pj8cT4w4x72hfK6InBVR3i6U\nzReRe0r7sznnnMstlpfR/g3MiXj+NNBTVQ8BVgNdQ3lXYHUo7xnqISLNgEuBvwPtgFdDAKsCvAK0\nB5oB/wx1K7y0WWkkPp9I3CNxJD6fSNqssravgXOusopJsBGRBsA5wJvhuQCnA0NClXeATuHxeeE5\n4XibUP88YJCqblLVBcB84Nhwm6+qP6vqZmBQqFviytKeNWmz0ug2shsZazNQlIy1GXQb2c0DjnOu\nTIhVz+Z54C4gOzyvB6xR1a3h+WLgwPD4QOAXgHB8baj/V3me1xRUXqJyVvNnZFjm/pzV/LEKOCkT\nUsjcknvFZ+aWTFImpBTwCuecKz2lHmxEpAOwXFWnl/Z759OWbiKSLiLpK1as2KHXFms1fylatDb/\nFZ8FlTvnXGmKRc+mNXCuiCzELnGdDrwA1BaRnNlxDYAl4fESoCFAOF4L+D2yPM9rCirfjqr2VtUk\nVU2qX7/+Dn2IsrZnTaNa+e9fUFC5c86VplIPNqp6r6o2UNVEbID/U1VNBj4DLgrVrgCGh8cjwnPC\n8U/VthcdAVwaZqs1BpoAU4FpQJMwu616eI8RJf05Ctqbpqg9a6I1zpPaJpX4arlXfMZXiye1TSz2\nNXDOudzK0qLOu4HbRGQ+NibTJ5T3AeqF8tuAewBU9TtgMPA9MAa4UVWzwrhOd2AsNtttcKhbonZm\nz5pojvMkH5FM7469SaiVgCAk1Eqgd8feJB/hi3Kcc7En1klwSUlJmp6evkOvSUuzMZpFi6xHk5pa\n+ILLxMT8c5slJMDChTv01s45VyaIyHRVTSqyngcbszPBZkfFxVmPJi8RyM7evtw558q64gabsnQZ\nrcLb2XEe55wr7zzYlKKdGedxzrmKwINNKfKszc65ysqzPpcyz9rsnKuMvGfjnHMu6jzYOOecizoP\nNs4556LOg41zzrmo82DjnHMu6jyDQCAiK4B8ksmUWXsDK2PdiBjz78C/g8r++SH230GCqhaZNt+D\nTTklIunFSRFRkfl34N9BZf/8UH6+A7+M5pxzLuo82DjnnIs6DzblV+9YN6AM8O/Av4PK/vmhnHwH\nPmbjnHMu6rxn45xzLuo82DjnnIs6DzbljIg0FJHPROR7EflORP4d6zbFgohUEZFvReSjWLclFkSk\ntogMEZEfRGSOiBwf6zaVNhG5Nfw/MFtEBopIzVi3KdpE5C0RWS4isyPK6orIOBGZF+7rxLKNBfFg\nU/5sBW5X1WZAK+BGEWkW4zbFwr+BObFuRAy9AIxR1b8BR1LJvgsRORC4GUhS1cOBKsClsW1VqXgb\naJen7B5ggqo2ASaE52WOB5tyRlWXquqM8PhP7EfmwNi2qnSJSAPgHODNWLclFkSkFnAy0AdAVTer\n6prYtiomqgK7iUhVIB74NcbtiTpVnQisylN8HvBOePwO0KlUG1VMHmzKMRFJBI4GvoltS0rd88Bd\nQHasGxIjjYEVQN9wKfFNEdk91o0qTaq6BHgWWAQsBdaq6iexbVXM7KuqS8PjZcC+sWxMQTzYlFMi\nsgcwFLhFVf+IdXtKi4h0AJar6vRYtyWGqgLHAK+p6tHAesropZNoCeMS52GB9wBgdxHpHNtWxZ7a\nWpYyuZ7Fg005JCLVsECTpqrDYt2eUtYaOFdEFgKDgNNFZEBsm1TqFgOLVTWnRzsECz6VyRnAAlVd\noapbgGHACTFuU6z8JiL7A4T75TFuT7482JQzIiLYtfo5qvpcrNtT2lT1XlVtoKqJ2IDwp6paqf6i\nVdVlwC8i0jQUtQG+j2GTYmER0EpE4sP/E22oZJMkIowArgiPrwCGx7AtBfJgU/60Bi7H/qL/b7id\nHetGuVJ3E5AmIjOBo4AnYtyeUhV6dUOAGcAs7LesXKRt2RUiMhCYDDQVkcUi0hV4CjhTROZhPb6n\nYtnGgni6Guecc1HnPRvnnHNR58HGOedc1Hmwcc45F3UebJxzzkWdBxvnnHNR58HGuVImIjVEZHyY\ntn5JjNtygIgMiWUbXOVQNdYNcK48EpEqqpq1ky8/GqimqkeVZJt2lIhUVdVfgYti2Q5XOXjPxlVo\nIvKoiNwS8Tw1Zw8gEblTRKaJyEwReSSizociMj3sldItonxdON83wPEi8lTYV2imiDybz3vXDeea\nKSJTRKS5iOwDDACOCj2bg/O85mARGRPe/0sR+VsoHy4iXcLj60QkLTz+XESeF5Gvw74ux4by3cPe\nJ1NDss7zQvmVIvK+iIwEPhGRxJy9UcIeQc9EfCfXhfJTw/vk7J+TFlbtIyItw3v/L7zXngWdx1Vy\nquo3v1XYG5AIzAiP44CfgHpAW2zFuYTyj4CTQ7264X43YDZQLzxX4B/hcT1gLtsWRtfO571fAh4K\nj08H/hsenwp8VEB7JwBNwuPjsHQ8YJl85wMnAT9GtPFz4I3w+GRgdnj8BNA5p23hNbsDV2K51epG\nfD85r+kG3B8e1wDSsUSXpwJrgQbhu5oMnAhUB34GWobX7IVdLcn3PLH+t+C32N78Mpqr0FR1oYj8\nLiJHYz/Y36rq7yLSFgs434aqewBNgInAzSJyfihvGMp/B7KwBKhgP74bgT5iu4Xmt2PoicCFoR2f\nikg9EdmroLaGTN4nAO+HjgPYjzWq+puIPAh8BpyvqpF7mgwMdSaKyF4iUjt8tnNF5I5QpybQKDwe\nl+f1OdoCzUUk57JarfDZNwNTVXVxaOd/sSC1FliqqtPC+/8Rjhd0ngUFfXZX8XmwcZXBm9hf9PsB\nb4UyAZ5U1V6RFUXkVCy/1PGqmikin2M/1AAbNYzTqOrWcMmqDZYQtDvWe9kVccAaLXgs5wgs6B2Q\npzxvzinFPt+Fqjo38oCIHIdtSZAfAW5S1bF5XnMqsCmiKIvCfzvyPY+r3HzMxlUGH2Bb6bYEcn4A\nxwJXh94EInJgGE+pBawOgeZv2Nbb2wmvq6WqHwO3YMkw8/oSSA71TwVWaiF7D4VjC0Tk4vAaEZEj\nw+NjgfbY5II7RKRxxEsvCXVOxDYRWxs+300RYytHF/L95BgL3CC2hQUicqgUvinbXGB/EWkZ6u8p\ntmvmjp7HVQLes3EVnqpuFpHPsF5DTs/kExE5DJgcfo/XAZ2BMcD1YtmU5wJTCjjtnsBwEamJ/SV/\naz51HgbeCufKZFsa+MIkA6+JyP1ANWCQiPwAvAFcpaq/isjt4bw5PanVIvI1NmZydSh7DNvRdKaI\nxGGXsDoU8d5vEsa4QpBaQSFbDIfv9RLgJRHZDdiA9Qp36DyucvCsz67CCz+2M4CLVXVerNtTksJl\nvjtUNT3WbXGuMH4ZzVVoItIMm8U1oaIFGufKE+/ZOOecizrv2TjnnIs6DzbOOeeizoONc865qPNg\n45xzLuo82DjnnIu6/wMC9U5fwUIgQwAAAABJRU5ErkJggg==\n", 947 | "text/plain": [ 948 | "" 949 | ] 950 | }, 951 | "metadata": {}, 952 | "output_type": "display_data" 953 | } 954 | ], 955 | "source": [ 956 | "plt.scatter(X_train, y_train, c='b')\n", 957 | "plt.scatter(X_test, y_test, c='g')\n", 958 | "plt.plot(X_test, ada_reg.predict(X_test), c='r')\n", 959 | "plt.title(\"Boosed Tree Regressor\")\n", 960 | "plt.xlabel(\"years of experience\")\n", 961 | "plt.ylabel(\"Salary in $\")\n", 962 | "plt.show()" 963 | ] 964 | }, 965 | { 966 | "cell_type": "code", 967 | "execution_count": 70, 968 | "metadata": { 969 | "collapsed": false 970 | }, 971 | "outputs": [ 972 | { 973 | "name": "stdout", 974 | "output_type": "stream", 975 | "text": [ 976 | "R^2 score for Tree boosted witha Ada: 0.911099102673\n" 977 | ] 978 | } 979 | ], 980 | "source": [ 981 | "print(\"R^2 score for Tree boosted witha Ada: \", ada_reg.score(X_test, y_test))" 982 | ] 983 | }, 984 | { 985 | "cell_type": "markdown", 986 | "metadata": {}, 987 | "source": [ 988 | "By boosting Tree we achieved improvement of 2%." 989 | ] 990 | }, 991 | { 992 | "cell_type": "markdown", 993 | "metadata": {}, 994 | "source": [ 995 | "### (Optional) Testing Liner Regression from scratch" 996 | ] 997 | }, 998 | { 999 | "cell_type": "code", 1000 | "execution_count": 72, 1001 | "metadata": { 1002 | "collapsed": true 1003 | }, 1004 | "outputs": [], 1005 | "source": [ 1006 | "from linear_regression_numpy import *" 1007 | ] 1008 | }, 1009 | { 1010 | "cell_type": "code", 1011 | "execution_count": 73, 1012 | "metadata": { 1013 | "collapsed": true 1014 | }, 1015 | "outputs": [], 1016 | "source": [ 1017 | "lr_numpy = LinearRegressioN()" 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "code", 1022 | "execution_count": 74, 1023 | "metadata": { 1024 | "collapsed": false 1025 | }, 1026 | "outputs": [ 1027 | { 1028 | "data": { 1029 | "text/plain": [ 1030 | "(array([ 26747.18889182]), array([ 9407.5255039]))" 1031 | ] 1032 | }, 1033 | "execution_count": 74, 1034 | "metadata": {}, 1035 | "output_type": "execute_result" 1036 | } 1037 | ], 1038 | "source": [ 1039 | "lr_numpy.train(X_train, y_train)" 1040 | ] 1041 | }, 1042 | { 1043 | "cell_type": "code", 1044 | "execution_count": 75, 1045 | "metadata": { 1046 | "collapsed": true 1047 | }, 1048 | "outputs": [], 1049 | "source": [ 1050 | "pred_numpy_lr = lr_numpy.predict(X_test)" 1051 | ] 1052 | }, 1053 | { 1054 | "cell_type": "code", 1055 | "execution_count": 76, 1056 | "metadata": { 1057 | "collapsed": false 1058 | }, 1059 | "outputs": [ 1060 | { 1061 | "data": { 1062 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmY3FWZ9vHvnYWEhpBAjEAg3R0FUZA9bIMyQBiF0Yij\nzIjTCihjuyJuozCZGRBtB3RGFgfFBhR8aUAMMgIiCBFEZA2LhJ0I6SaAJGxBkgBZnvePczpd3em9\nu6q6qu/PddVVdc5vO1Xp1FPn/M6iiMDMzKyYxpS7AGZmVv0cbMzMrOgcbMzMrOgcbMzMrOgcbMzM\nrOgcbMzMrOgcbGxQJL1b0qPlLkc1kFQr6VVJY8tw7S0l3Szpr5L+p9TXH6kkLZZ0SLnLUU0cbKxX\nPf2ni4g/RMQO5ShTV5JOlrQ6f2G/LOlWSfuVu1z9FRFtEbFpRKwtw+UbgeeBzSLiq6W+uKRtJV0u\n6XlJyyU9IOmYIlynXlJIGjfc57b+cbCxitLLl8XPI2JT4E3AjcAvSnz9SlUHPBQ9jO4uwfv9f8BT\nuRxTgY8Dzw30JOWoFdrAONjYoEg6UNKSgvRiSV+TdH/+hfpzSRMLtr9f0n0FNY9dCradIOnPuSnn\nIUn/ULDtGEl/lHS6pBeAk3srV0SsAVqAbSRN6+f195B0b77+L3LZv134PiV9Q9JfgJ/243zfkPR0\nPt+jkmbn/L0lLZD0iqTnJH0/53f61S1puqQrJb0oaZGkTxWc+2RJl0n6WT7/g5Jm9fsfroCkC4Cj\nga/nWuEh+fzzJF0k6RXgGEkTJJ0h6Zn8OEPShC6fz9clLZX0rKQPSvp7SY/l9/BvvRRjL+CCiFgR\nEWsi4t6I+E1BGd+VP9+XJT3VXuuRdIGkH0m6RtIK4CBJ78v/jq/kfU8uuM7N+fnl/F73y+f5lKSH\nC/729ig4Zree/p5tECLCDz96fACLgUO6yT8QWNJlvzuB6cAWwMPAZ/K23YGlwD7AWNIX3GJgQt7+\nj/m4McBHgBXA1nnbMcAa4DhgHLBxN2U5Gbgov94IOJXUNDSur+vn/VuB44HxwIeAN4BvF7zPNcBp\nef+N+zjfDqRf6tPz8fXAW/Pr24CP59ebAvsW7BMF5b0Z+CEwEdgNWAYcXPBeXwP+Pl/7v4Dbh/Dv\ne0H7ey04/2rgg/nfY2PgFOB24M3ANOBW4FtdPp//zJ/fp3J5LwYmATsBq4CZPVz/BuCPwJFAbZdt\ndcBfgY/mc08Fdiso93Jg/1zOibksO+f0LqQa0ge7+4wL/u6eJgU8AdsBdX39PfsxyL+1chfAj5H9\nYGDB5mMF6e8C5+TXP2r/cirY/ijwtz1c8z7g8Pz6GKCtjzKeTAoQLwNrgReAAwu293h94ID8haOC\nbbfQOdi8AUzs5/m2IwWiQ4DxXfa5Gfgm8KYu+eu/CIEZ+T1MKtj+X6Rf/+3v9YaCbTsCq4bw73sB\nGwabm7vs82fg7wvS7wUWF3w+q4CxOT0pv5d9Cva/m/yl3831Nyf9OHgwv+/7gL3ythOBK3op98/6\neG9nAKd3/YwLtl8HHN/L3323f89+DO7hZjQbTn8peL2S9Osd0i/Ur+amkJclvUz6Up0OIOmogiap\nl4F3ku69tHuqH9e+LCKmAFsCDwB7Fmzr7frTgacjf6P0cL1lEfFaf84XEYuAL5G+tJdKulTS9Hzc\nscDbgEck3SXp/d28j+nAixHx14K8VmCbgnTXz3miurm3IunfcpPRq5LO6eZaPen6/qfnMhSWZ3pB\n+oXo6NywKj8X3ndZRcffQicR8VJEnBARO5H+7e4D/k+SSJ/pn/tbTkn7SLpR0jJJy4HP0PnvqKu+\nzt/T37MNgoONlcJTQFNETCl41ETEJZLqgHOBLwBTc8B4gNSs0a7fU5NHxPOkHlYnS9q6r+sDz5Lu\n7xReb0bX0/b3/eQyXBwR7yIFpSA1wRERj0fER0nNUacB8yRt0uXczwBbSJpUkFdLqn0NSER8J1Iv\nt00j4jMDObSbMtV1Kc8zAy1PnxdN/3b/TUfT1VPAW3s7pEv6YuBKYEZETAbOoePvqLu/ob7Ob8PI\nwcb6Y7ykiQWPgfZQOhf4TP7lKUmb5Ju5k4BNSF8EywAkfYJUsxm0iHiU1ETy9X5c/zZS880XJI2T\ndDiw92Dfj6QdJB2cb6C/RvpVvy6/t49JmhYR60hNfrRvKyj7U6R7Iv+VP+tdSDWii4bymQzRJcC/\nS5om6U2k+zPDUh5Jp0l6Z/7sJwGfBRZFxAukjh6HSPqnvH2qpN16Od0kUq3wNUl7A/9csG0Z6bN+\nS0HeecDXJO2Z/x23yz9+rAgcbKw/riF9abY/Th7IwRGxgHTj+H+Bl4BFpHsxRMRDwP+QvvSfI93g\n/eMwlPl7QKOkN/dx/TdInQKOJQWAjwFXA68P5v2QOgm0d1D4C6kWc2LedijwoKRXgTOBIyNiFRv6\nKOkewzPAFcBJEXHDAN//cPo2sAC4H1gI3JPzhkMN6T2+DDxBqkF9ANL4I1JHiK8CL5Ka2Hbt5Vyf\nA06R9FdSQLysfUNErASagD/mps99I+IXOe9iUkeE/yPVqKwI1Lmp2swk3UG6GfzTcpfFrFq4ZmOj\nnqS/lbRVbqo5mtRt9tpyl8usmlTbaGizwdiB1OSyCakp54iIeLa8RTKrLm5GMzOzonMzmpmZFZ2b\n0bI3velNUV9fX+5imJlVlLvvvvv5iJjW134ONll9fT0LFiwodzHMzCqKpNa+93IzmpmZlYCDjZmZ\nFZ2DjZmZFZ2DjZmZFZ2DjZmZFZ2DjZlZFWtpgfp6GDMmPbe0lKccDjZmZlWqpQUaG6G1FSLSc2Nj\nyi91EPI4GzOzKjV3Lqxc2Tlv5Uo4/nhYtapjW3sQAmhoKE5ZXLMxM6tSbW3d57/wQvdBaO7c4pXF\nwcbMrErV1g5s/56C03BwsDEzq1JNTVBT0zmvpgamTu1+/4EGp4EoWrCR9BNJSyU9UJD3PUmPSLpf\n0hWSphRsO1HSIkmPSnpvQf6hOW+RpBMK8mdKuiPn/1zSRjl/Qk4vytvri/UezcxGsoYGaG6GujqQ\n0nNzM5x5ZvdBqKmpeGUpZs3mAtKa64WuB94ZEbsAj5HXZpe0I3AksFM+5oeSxkoaC5wNHAbsCHw0\n7wtwGnB6RGxHWgf+2Jx/LPBSzj8972dmNio1NMDixbBuXXpuaOg5CBWrcwAUMdhExM3Ai13yfhsR\na3LydmDb/Ppw4NKIeD0ingQWAXvnx6KIeCIi3gAuBQ6XJOBgYF4+/kLggwXnujC/ngfMzvubmVnW\nXRAqpnLes/kk8Jv8ehvgqYJtS3JeT/lTgZcLAld7fqdz5e3L8/4bkNQoaYGkBcuWLRvyGzIzs+6V\nJdhImgusAco0ljWJiOaImBURs6ZN63PtHzMzG6SSD+qUdAzwfmB2RETOfhqYUbDbtjmPHvJfAKZI\nGpdrL4X7t59riaRxwOS8v5mZlUlJazaSDgW+DnwgIgqHFF0JHJl7ks0EtgfuBO4Cts89zzYidSK4\nMgepG4Ej8vFHA78qONfR+fURwO8KgpqZmZVBMbs+XwLcBuwgaYmkY4H/BSYB10u6T9I5ABHxIHAZ\n8BBwLfD5iFibay1fAK4DHgYuy/sCfAP4iqRFpHsy5+f884GpOf8rwPru0mZmI8VImSCzVOQf/cms\nWbNiwYIF5S6GmY0C7RNkFk4ZU1NT/O7HxSDp7oiY1dd+nkHAzKzEepogs5hzk5Wbg42ZWYn1NAdZ\nMecmKzcHGzOzEutpDrJizk1Wbg42ZmYl1tMEmcWcm6zcHGzMzEqsHHOTlZuDjZlVlUrpUlzqucnK\nzctCm1nV6NqluBTLHVv/uGZjZlVjNHYprhQONmZWNSq1S3GlNP0NhYONmVWNSuxS3N7019oKER1N\nf9UWcBxszKxqVGKX4tHS9OdgY2ZVoxK7FFdq099AuTeamVWVhoYSBpfVq1NUGzf4r9La2tR01l1+\nNXHNxsxsMB54APbZB773vSGdphKb/gbDwcbMbCDWroXTToM994QlS+Ad7xjS6Sqx6W8w3IxmZtZf\njz0GxxwDt90GH/oQnHMOTJs25NOWtOmvTFyzMTMr0LKwhfoz6hnzzTHUn1FPy8KWNKfMD34Au+0G\nDz+c+iXPmzcsgWa08EqdmVfqNLOWhS00XtXIytUdfZF3eHUiN/3+LWx110Nw2GFw3nkwfXoZSzmy\n9HelTjejmZllc+fP7Qg0AZ+8F06/9jXG6GE491w49th0Y8UGzMHGzCxrW54Gt7x7Mcy7DN68Em6s\nh08eHjz5L/9S1rJVOgcbM7OsdrMZLP5Kx2jKLx4K/7s31G5eV8ZSVQcHGzMzgJtv7hRovvxe+MG+\nUDO+hqbZVTbopQwcbMzMpkyB5cvXJ99+2gweW7WEusm1NM1uomHnKu+XXAIONmY2et17L+yxR0f6\nxBPhO9/hkfKVqGp5nI2ZjU7vfGfnQPPSS/Cd7/T78G7H41iPHGzMbHR5/PHUffnBB1P62GPTQjJT\npvT7FO3jcVqXtxIErctbabyq0QGnFw42ZjZ6/N3fwdve1pF+9tk0SHOAOo3HyVauXsnc+VW2CM0w\ncrAxs+r39NOpNnPDDSk9Z06qzWy11aBO1z4ep7/55mBjZtXu4x+HbbftSD/xBFx55ZBOWTu5+8Vm\neso3Bxszq1YvvphqMxddlNJ77JFqMzNnDvnUTbObqBnfeREaj8fpnYONmVWfr30Npk7tSN9/P9x9\n97CdvmHnBprnNFM3uQ4h6ibX0Tyn2eNxeuFZnzPP+mxWBVasgE037UhvtVXqBGBF099Zn12zMbPq\ncOqpnQPNrbc60IwgnkHAzCrbG2/AhAmd89xiM+K4ZmNmFaGlBerrYcyY9NzSQlpjpjDQ/OY3DjQj\nlGs2ZjbitbRAYyOszOMon2pdS8PHunx9rVvnhc1GMNdszGzEmzu3I9B8mHmsLfydfMklqTbjQDOi\nFS3YSPqJpKWSHijI20LS9ZIez8+b53xJOkvSIkn3S9qj4Jij8/6PSzq6IH9PSQvzMWdJ6S+tp2uY\nWeVqawMIAjGPf1yfP57VcOSRZSuX9V8xazYXAId2yTsBmB8R2wPzcxrgMGD7/GgEfgQpcAAnAfsA\newMnFQSPHwGfKjju0D6uYWYV6qPTbiAKvq4+x9mIYJs63wmoFEX7l4qImyXVd8k+HDgwv74QuAn4\nRs7/WaRBP7dLmiJp67zv9RHxIoCk64FDJd0EbBYRt+f8nwEfBH7TyzXMrBKNG0fL2rXrkxN4jTeY\nQE0NNHnAfsUo9T2bLSOiveP7X4At8+ttgKcK9luS83rLX9JNfm/XMLNKcued6T5MDjR/+vAp1NcF\nqzWBujpoboYGD9ivGGWrg0ZESCpqH8W+riGpkdRsR22tJ9AzGzFmzoTFizvSr7zCrpMmsbin/W3E\nK3XN5rncPEZ+XprznwZmFOy3bc7rLX/bbvJ7u8YGIqI5ImZFxKxp06YN+k2ZjWbdjn8ZrIceSrWZ\n9kBz3HGpp9mkSUMvqJVVqYPNlUB7j7KjgV8V5B+Ve6XtCyzPTWHXAe+RtHnuGPAe4Lq87RVJ++Ze\naEd1OVd31zCzYdY+/qW1NcWE1taUHlTA2X9/2GmnjvSyZXDWWcNWViuvYnZ9vgS4DdhB0hJJxwKn\nAn8n6XHgkJwGuAZ4AlgEnAt8DiB3DPgWcFd+nNLeWSDvc14+5s+kzgH0cg0zG2aF41/arVyZ8vut\ntTXVZm69NaU/8pEUud70pmErp5WfZ33OPOuz2cCNGdP97DBSGtDfpw99CK64oiPd1gYzZvS8v404\nnvXZzIqup341ffa3Wbo0RaT2QHPAASlqOdBULQcbMxu0piao6bxgZd/jXz77WdiyYETCI4/A739f\nlPLZyOHht2Y2aO3jXObOTS1gtbUp0HQ7/uWVV2Dy5I70dtvB44+XpJxWfq7ZmNmQNDSknsrr1qXn\nbgPNySd3DjR33eVAM8q4ZmNmxfPaa7Dxxh3piRNh1arylcfKxjUbMyuOs87qHGh+9zsHmlHMwcas\nAg3rqP3htmZN6ml2/PEdeevWwUEHla9MVnYONmYVZlhH7Q+3iy6C8eM70ldc4YXNDPCgzvU8qNMq\nRX19CjBd1dV1nruypCJSNavQ2rUb5lnV8aBOsyqVVq3sf37R/frXnYPK+ed3H3xsVHNvNLMKU1vb\nfc2m5KtkdBdQXn8dNtqoxAWxSuCfHmYVZlCj9gepx44It9zSOdB897sp+DjQWA9cszGrMAMatT8E\n7R0R2md1bu+IcMRnpjLh1Rc7dlyxYsPoZ9aFazZmFahfo/aHqOvyAbvwJ1asVEeg+cY3Um3Ggcb6\nwTUbM+tWYYeDP7ELu7CwI+PFF2HzzUtfKKtYrtmYWbdqa+Et/JlA6wPNT/gE9XXhQGMD5pqNmXXr\n1s0OZTrXrU9vzTO8UrM1zUXoiGDVr9eajaQxXdINkj4jyY20ZtXq2WdBYvrCFGhunHgYYxRMqNua\n5ubi3B+y6tdXM9qvJb0DQNJc4ChgV+DSYhfMzMrgmGNg+vSO9KJFHLTqmqJ2RLDRocdmNEl/C2wP\nTJP0ZuDjwL8BLwDnSjoAWBwR5Rq3bGbD5cUXYerUjvTuu8M995SvPFZ1+qrZjAE2A7YC1gLP5/z2\necI9u55ZpfvXf+0caP70JwcaG3Y91mwi4veSLgJOAzYFTo6ImyVNBZ6PiJtLVUgzK4IVK2DTTTvS\n06bB0qXlK49VtV5rNhHxn8ARwOyI+GnBMZ8qdsHMrIi++93OgeaWWxxorKj67PocEQ93SS8DlhWt\nRGZWPG+8ARMmdM7zMiNWAh7UaTZanH9+50Dz61870FjJeFCnWbVbtw7Gjt0wz6tnWgm5ZmNWzS6/\nvHOgufhiL9NsZdFnzUbS/sDJQF3eX0BExFuKWzQzG7TuFjZbvRrGuTHDyqM/NZvzge8D7wL2Ambl\nZzMbiebP7xxofvCDFHwcaKyM+vPXtzwiflP0kpjZ0E2YkHqctVu1CiZOLF95zLL+1GxulPQ9SftJ\n2qP9UfSSmVn/3XVXug/THmhOPjnVZhxobIToT81mn/w8qyAvgIOHvzhm1aNlYQtz58+lbXkbtZNr\naZrdRMPORZjJ8q1vhSee6EgvXw6bbTb81zEbgv4M6jyoFAUxqyYtC1tovKqRlavTusqty1tpvKoR\nYPgCziOPwDve0ZH+3Ofg7LOH59xmw6y3WZ8/FhEXSfpKd9sj4vvFK5ZZZZs7f+76QNNu5eqVzJ0/\nd3iCzbveBX/8Y0d66dI0t5nZCNXbPZtN8vOkHh5m1oO25d2vvNFTfv9P3JbuzbQHmiOOSPdmHGhs\nhOtt1ucf5+dvlq44ZtWhdnItrctbu80frNZ9/om6O3+xPn3FGa38w/GDP59ZKXkGAbMiaJrdRM34\nzqun14yvoWl208BPtmwZSOsDzS3sjwg+9m+1tLQMR2nNis/BxqwIGnZuoHlOM3WT6xCibnIdzXOa\nB36/5gtfgDe/eX3yHTzEu7kFgJUrYe7c4Sy1WfEo+pj1VdLYiFg7rBeVvgz8C6kL9ULgE8DWwKXA\nVOBu4OMR8YakCcDPgD1JS1J/JCIW5/OcCBxLWkX0ixFxXc4/FDgTGAucFxGn9lWmWbNmxYIFC4bz\nbZoN3iuvwOTJ65NPMJO38sQGu0lpTk2zcpF0d0TM6mu//tRsHs+DOncchnIhaRvgi8CsiHgnKSAc\nSVoR9PSI2A54iRREyM8v5fzT837k8hwJ7AQcCvxQ0lhJY4GzgcOAHYGPDlfZzUrim9/sFGi4804O\nrtsw0ADU+paNVYj+BJtdgceA8yTdLqlR0lBHjI0DNpY0DqgBniUNEp2Xt18IfDC/PjynydtnS1LO\nvzQiXo+IJ4FFwN75sSginoiIN0i1pcOHWF6z4nvttVRVOfnklN5oo9TTbK+9aGqCms63gKipgaZB\n3AIyK4c+g01E/DUizo2IvwG+AZwEPCvpQknbDfSCEfE08N9AGynILCc1m70cEWvybkuAbfLrbYCn\n8rFr8v5TC/O7HNNT/gZy4FwgacGyZV581Mro7LNh44070jfcAK+/vj7Z0ADNzVBXl+JRXV1KNxRh\nQgKzYujPEgNjgfeR7qvUA/8DtADvBq4B3jaQC0ranFTTmAm8DPyC1AxWchHRDDRDumdTjjLYKLdm\nDYwf3zmvh4XNGhocXKxy9eueDSk4fC8ido+I70fEcxExD7h2ENc8BHgyIpZFxGrgl8D+wJTcrAaw\nLfB0fv00MAMgb59M6iiwPr/LMT3lm40sF1/cOdBcfrkXNrOq1WvNJtdqLoiIU7rbHhFfHMQ124B9\nJdUAq4DZwALgRuAI0j2Wo4Ff5f2vzOnb8vbfRURIuhK4WNL3genA9sCdpMXdtpc0kxRkjgT+eRDl\nNCuO7hY2W7Nmw6WbzapIrzWb3OV5WCfijIg7SDf67yF1ex5Dasr6BvAVSYtI92TOz4ecD0zN+V8B\nTsjneRC4DHiIVMP6fESszfd1vgBcBzwMXJb3NSu/3/ymc6A599wUfBxorMr1Z5xNE6np6ufAivb8\niLinuEUrLY+zsaLr2jz2+uupx5lZBRvOcTZ/QxrLcgqpc8D/kHqTmVl//PGPnQPNqaem2owDjY0i\nXs/GrJi61mZefRU22aT7fc2qWL/mRpP0Pklfl/Sf7Y9iF8ysov36150Dzde+lmozDjQ2SvVnnM05\npFH+BwHnkXqE3VnkcplVrq61mbY2mDGj+33NRol+3bOJiKNI85N9E9iPzuNYzAzgtts2DDQRDjRm\n9KNmQxoLA7BS0nTSgMqZxSuSWQXqGmQeeAB22qk8ZTEbgfoTbK6WNAX4HmlsTJCa08zsySfhLW/p\nnNfHcAKz0ag/E3F+KyJejojLgTrg7RHxH8UvmtkIt8sunQPNLbc40Jj1oMeajaQP9bKNiPhlcYpk\nNsItXQpbbtk5z0HGrFe9NaPN6WVbkCbQNBtd3v/+1K253Y03woEHlq04ZpWix2ATEZ8oZUHMRrQu\nyzQDrs2YDYAHdZr15VOf6hxorrjCgcZsgDyo06wnr78OEyd2zuthYTMz650HdZp156STOgea9qUA\nHGjMBsWDOs0KrV0L48ZtmNd1sTMzG5D+/A/qOqhzMXBJMQtlVhY//GHnQPPtb3e/qqaZDVh/lhj4\nVn55uaSrgYkRsby4xTIroe4Cihc2MxtWPf5kk7SXpK0K0keRlmH+lqQtSlE4s6L7+c87B5rjjvPC\nZmZF0FvN5sfAIQCSDgBOBY4DdgOaSb3SzCqXFzYzK5neGqPHRsSL+fVHgOaIuDzPi7Zd8YtmViQ3\n3NA50Hz4w30ubNbSAvX1qRJUX5/SZtZ/vdVsxkoaFxFrgNlAYz+PMxu5utZmnn8epk7t9ZCWFmhs\nhJUrU7q1NaUBGhqKUEazKtRbzeYS4PeSfkXq/vwHAEnbAe4gYJVlwYLOgWavvVJtpo9AAzB3bkeg\nabdyZco3s/7pbW60Jknzga2B30asn59jDOnejVllGOIyzW1tA8s3sw312hwWEbd3k/dY8YpjNowe\nfRTe/vaO9FZbwbPPDvg0tbWp6ay7fDPrH49Ws+pUV9c50Dz00KACDUBTE9TUdM6rqUn5AC0LW6g/\no54x3xxD/Rn1tCx07wGzrnyj36rL00/Dttt2zhviDM3tnQDmzk1NZ7W1KdA0NKRA03hVIytXp5s6\nrctbabwq9R5o2Nm9B8zaKTxVOgCzZs2KBQsWlLsYNhQHHAB/+ENH+rbbYN99i3rJ+jPqaV2+YRtb\n3eQ6Fn9pcVGvbTYSSLo7Imb1tZ9rNlb5XnoJtugyqUWJfkS1Le++l0BP+Wajle/Z2LApy8DHhobO\ngeaaa0q6sFnt5O57CfSUbzZauWZjw6LkAx9Xrdrwrn0ZmoSbZjd1umcDUDO+hqbZTSUvi9lI5pqN\nDYtiD3wsrDUdteV1nQPNRReVbZnmhp0baJ7TTN3kOoSom1xH85xmdw4w68IdBDJ3EBiaMWO6/76X\n0krKQ9Fea3pt5Vr+xK68kwc7NnqZZrOy6m8HAddsbFj0NMBxOAY+zp0Lu668lbWMWx9o9uU26uu8\nTLNZpXCwsWHR18DHQYvgp60HcSv7A3APuzOGtdzBvp4uxqyCONjYsGhogObmNHBfSs/NzUPsHHDf\nfTBmDAdxEwCHcD17cg+R/2w9XYxZ5XCwsWHT0ACLF6fbKIsXDzHQHHEE7L47ACumzmCzjVczP63l\nBwyt1uS1acxKz8HGRpbHHktVo8svT+lf/pJNnm/jR+eOG5ZaU3tng9bW1KGhvYu2A45Zcbk3Wube\naCPApz+dogikVTOffx4mThzWS9TXdz+Dc11dqo2Z2cCM6N5okqZImifpEUkPS9pP0haSrpf0eH7e\nPO8rSWdJWiTpfkl7FJzn6Lz/45KOLsjfU9LCfMxZkrssDadhb4Zqa0tVlvZAc+GF8Oqrwx5o2i81\nkHwzGx7lakY7E7g2It4O7Ao8DJwAzI+I7YH5OQ1wGLB9fjQCPwKQtAVwErAPsDdwUnuAyvt8quC4\nQ0vwnkaFYW+GOvHEVK1o98orcNRRw1LW7hSzi7aZ9azkwUbSZOAA4HyAiHgjIl4GDgcuzLtdCHww\nvz4c+FkktwNTJG0NvBe4PiJejIiXgOuBQ/O2zSLi9ry66M8KzmVDNGwzBSxdmmozp56a0medlaLX\npEnDUs6eFK2Ltpn1qhw1m5nAMuCnku6VdJ6kTYAtI6J9dau/AFvm19sATxUcvyTn9Za/pJv8DUhq\nlLRA0oJly5YN8W2NDsPSDHXaabDllh3pF16A40qz0nhRumibWZ/KEWzGAXsAP4qI3YEVdDSZAZBr\nJEXvuRARzRExKyJmTZs2rdiXqwpDaoZ6+eX0DX9C/uc+5ZRUm+m6PECRDWsXbTPrl3IEmyXAkoi4\nI6fnkYLPc7kJjPy8NG9/GphRcPy2Oa+3/G27ybdhMOhmqHPOgc0370g/8wz8x38Me/nMbGQqebCJ\niL8AT0naIWfNBh4CrgTae5QdDfwqv74SOCr3StsXWJ6b264D3iNp89wx4D3AdXnbK5L2zb3Qjio4\nlw3RgJu6YiLWAAAOQ0lEQVShVqxIO372syn95S+n2szWW2+wqwdbmlWxiCj5A9gNWADcD/wfsDkw\nldQL7XHgBmCLvK+As4E/AwuBWQXn+SSwKD8+UZA/C3ggH/O/5PFEvT323HPPsGF28cURKbSkxxNP\n9LjrRRdF1NR03r2mJuV3t29dXYQUMXVqekgpr7v9zax4gAXRj+99D+rMPKizdy0tqcdZW1u6P9PU\n1Ett5o03YKut0nLNkLoyX3hhDzsn/R1s2XWRtq5qanzD36yU+juo08Emc7DpWXdf8D1+qV99NcyZ\n05F+8EHYccc+r9Hf9XB6CkqFPBuAWemM6BkErLL0a2zN2rWw3XYdgWbOnBQl+hFooP+93NragJ1b\n4Ev1cNKY9Lxzy4b7mNmI4mBjfepzbM1NN8G4cfDnP6f0XXfBlVcOaGGz/vZy2+JvW2BOI0xpBUV6\nntPYKeB4NgCzkcfBxvrUY61jRsC++8JBB6WM/fZLtZlZfdaoN9DvXm6HzIWNulSzNloJs1M1y7MB\nmI1MDjbWp+5qHe+acBeL28bAHXm41E030fL5W6mfqUF3Xe7PYMsX1/RQzZrc5tkAzEawceUugI18\n7V/ec+dCW2tww8bv5+BV16TM7beHhx+m5dKxnToRtE/QWXj8cKidXEvr8g17CNRNqXWnALMRzDUb\n65eGBlh8zUOsY0xHoLn66rTY2dixwzdBZx+aZjdRM75zNatmfA1Ns912ZjaSOdhY/xx1FOy0U3o9\ndSq8/jq8733rN5dqnZiGnRtontNM3eQ6hKibXEfznGYadnbbmdlI5mY0692TT8Jb3tKRvuQSOPLI\nDXarre1+/EsxeoY17Nzg4GJWYVyzsZ596UudA82KFd0GGvA6MWbWOwcb29Azz6T+x2eemdI//nEa\n3t81mhTwOjFm1hs3o1lnp5wCJ53UkX7pJZgypV+HNjQ4uJhZ9xxsLHnxxXTjv91pp8HXv16+8phZ\nVXGwMTjrLDj++I70c8/Bm99cvvKYWdXxPZvR7K9/TTdY2gPNiSemezMONGY2zFyzGa0uuAA+8YmO\ndFsbzJjR4+5mZkPhYDPavPZauuH/+usp/elPwznnlLdMZlb13Iw2mvzyl7Dxxh2B5rHHHGjMrCRc\nsxkNVq+GmTPh6adT+ogj4Be/KG+ZzGxUcbCpdtdfD+95T0f6vvtg113LVx4zG5UcbKrVunWwxx7w\npz+l9MEHww03DGj1TDOz4eJgU41uvRX2379zer/9ylceMxv13EGg2nzmMx2BZtddYe1aBxozKzsH\nmyJpaUlLIw92ieRB+/GP0/Nvf5vuz4zxP7GZlZ+b0YqgpYWSLJHcrYgiX8DMbOD8s7cISrVEsplZ\npXCwKYJSLZFsZlYpHGyKoKelkIuxRLKZWSVwsCkCL5FsZtaZg00RFHOJ5LL1cjMzGwL3RiuSYiyR\nXNZebmZmQ+CaTQVxLzczq1QONhXEvdzMrFI52FQQ93Izs0rlYFNB3MvNzCqVg00FKWYvNzOzYnJv\ntApTjF5uZmbFVraajaSxku6VdHVOz5R0h6RFkn4uaaOcPyGnF+Xt9QXnODHnPyrpvQX5h+a8RZJO\nKPV7MzOzzsrZjHY88HBB+jTg9IjYDngJODbnHwu8lPNPz/shaUfgSGAn4FDghzmAjQXOBg4DdgQ+\nmvetei0LW6g/o54x3xxD/Rn1tCz0iE8zGxnKEmwkbQu8DzgvpwUcDMzLu1wIfDC/Pjynydtn5/0P\nBy6NiNcj4klgEbB3fiyKiCci4g3g0rzvsBtJo/lbFrbQeFUjrctbCYLW5a00XtXogGNmI0K5ajZn\nAF8H1uX0VODliFiT00uAbfLrbYCnAPL25Xn/9fldjukpf1i1j+ZvbU1LyLSP5i9XwJk7fy4rV3ce\n8bly9UrmzveITzMrv5IHG0nvB5ZGxN2lvnY3ZWmUtEDSgmXLlg3o2JE2mr9tefcjO3vKNzMrpXLU\nbPYHPiBpMamJ62DgTGCKpPbecdsCT+fXTwMzAPL2ycALhfldjukpfwMR0RwRsyJi1rRp0wb0Jkba\naP7ayd2P7Owp38yslEoebCLixIjYNiLqSTf4fxcRDcCNwBF5t6OBX+XXV+Y0efvvIiJy/pG5t9pM\nYHvgTuAuYPvcu22jfI0rh/t9DHY0f7Hu8zTNbqJmfOcRnzXja2ia7RGfZlZ+I2lQ5zeAr0haRLon\nc37OPx+YmvO/ApwAEBEPApcBDwHXAp+PiLX5vs4XgOtIvd0uy/sOq8GM5i/mfZ6GnRtontNM3eQ6\nhKibXEfznGYadvagHDMrP6VKgs2aNSsWLFgwoGNaWtI9mra2VKNpaup9wGV9fQowXdXVweLFA7q0\nmdmIIOnuiJjV534ONslggs1AjRmTajRdSbBu3Yb5ZmYjXX+DzUhqRqt6nrXZzEYrB5sS8qzNZjZa\nOdiUkGdtNrPRyrM+l5hnbTaz0cg1GzMzKzoHGzMzKzoHGzMzKzoHGzMzKzoHGzMzKzrPIJBJWgZ0\nM5nMiPUm4PlyF6LM/Bn4Mxjt7x/K/xnURUSf0+Y72FQoSQv6M0VENfNn4M9gtL9/qJzPwM1oZmZW\ndA42ZmZWdA42lau53AUYAfwZ+DMY7e8fKuQz8D0bMzMrOtdszMys6BxszMys6BxsKoykGZJulPSQ\npAclHV/uMpWDpLGS7pV0dbnLUg6SpkiaJ+kRSQ9L2q/cZSo1SV/O/wcekHSJpInlLlOxSfqJpKWS\nHijI20LS9ZIez8+bl7OMPXGwqTxrgK9GxI7AvsDnJe1Y5jKVw/HAw+UuRBmdCVwbEW8HdmWUfRaS\ntgG+CMyKiHcCY4Ejy1uqkrgAOLRL3gnA/IjYHpif0yOOg02FiYhnI+Ke/PqvpC+ZbcpbqtKStC3w\nPuC8cpelHCRNBg4AzgeIiDci4uXylqosxgEbSxoH1ADPlLk8RRcRNwMvdsk+HLgwv74Q+GBJC9VP\nDjYVTFI9sDtwR3lLUnJnAF8H1pW7IGUyE1gG/DQ3JZ4naZNyF6qUIuJp4L+BNuBZYHlE/La8pSqb\nLSPi2fz6L8CW5SxMTxxsKpSkTYHLgS9FxCvlLk+pSHo/sDQi7i53WcpoHLAH8KOI2B1YwQhtOimW\nfF/icFLgnQ5sIulj5S1V+UUayzIix7M42FQgSeNJgaYlIn5Z7vKU2P7AByQtBi4FDpZ0UXmLVHJL\ngCUR0V6jnUcKPqPJIcCTEbEsIlYDvwT+psxlKpfnJG0NkJ+Xlrk83XKwqTCSRGqrfzgivl/u8pRa\nRJwYEdtGRD3phvDvImJU/aKNiL8AT0naIWfNBh4qY5HKoQ3YV1JN/j8xm1HWSaLAlcDR+fXRwK/K\nWJYeOdhUnv2Bj5N+0d+XH39f7kJZyR0HtEi6H9gN+E6Zy1NSuVY3D7gHWEj6LquIaVuGQtIlwG3A\nDpKWSDoWOBX4O0mPk2p8p5azjD3xdDVmZlZ0rtmYmVnROdiYmVnROdiYmVnROdiYmVnROdiYmVnR\nOdiYlZikCZJuyN3WP1LmskyXNK+cZbDRYVy5C2BWiSSNjYi1gzx8d2B8ROw2nGUaKEnjIuIZ4Ihy\nlsNGB9dsrKpJOkXSlwrSTe1rAEn6V0l3Sbpf0jcL9vk/SXfntVIaC/Jfzee7A9hP0ql5XaH7Jf13\nN9feIp/rfkm3S9pF0puBi4Ddcs3mrV2Oeauka/P1/yDp7Tn/V5KOyq8/Laklv75J0hmSbs3ruuyd\n8zfJa5/cmSfrPDznHyPpF5KuAn4rqb59bZS8RtD3Cj6TT+f8A/N12tfPacmj9pG0V772n/K1JvV0\nHhvlIsIPP6r2AdQD9+TXY4A/A1OB95BGnCvnXw0ckPfbIj9vDDwATM3pAP4pv54KPErHwOgp3Vz7\nB8BJ+fXBwH359YHA1T2Udz6wfX69D2k6Hkgz+S4C3g08VlDGm4Bz8+sDgAfy6+8AH2svWz5mE+AY\n0txqWxR8Pu3HNAL/nl9PABaQJro8EFgObJs/q9uAdwEbAU8Ae+VjNiO1lnR7nnL/LfhR3oeb0ayq\nRcRiSS9I2p30hX1vRLwg6T2kgHNv3nVTYHvgZuCLkv4h58/I+S8Aa0kToEL68n0NOF9ptdDuVgx9\nF/DhXI7fSZoqabOeyppn8v4b4Be54gDpy5qIeE7SfwI3Av8QEYVrmlyS97lZ0maSpuT39gFJX8v7\nTARq8+vruxzf7j3ALpLam9Um5/f+BnBnRCzJ5byPFKSWA89GxF35+q/k7T2d58me3rtVPwcbGw3O\nI/2i3wr4Sc4T8F8R8ePCHSUdSJpfar+IWCnpJtIXNcBrke/TRMSa3GQ1mzQh6BdItZehGAO8HD3f\ny9mZFPSmd8nvOudUkN7fhyPi0cINkvYhLUnQHQHHRcR1XY45EHi9IGstvX93dHseG918z8ZGgytI\nS+nuBbR/AV4HfDLXJpC0Tb6fMhl4KQeat5OW3t5APm5yRFwDfIk0GWZXfwAa8v4HAs9HL2sP5W1P\nSvrHfIwk7Zpf7w0cRupc8DVJMwsO/Uje512kRcSW5/d3XMG9ld17+XzaXQd8VmkJCyS9Tb0vyvYo\nsLWkvfL+k5RWzRzoeWwUcM3Gql5EvCHpRlKtob1m8ltJ7wBuy9/HrwIfA64FPqM0m/KjwO09nHYS\n8CtJE0m/5L/czT4nAz/J51pJxzTwvWkAfiTp34HxwKWSHgHOBT4REc9I+mo+b3tN6iVJt5LumXwy\n532LtKLp/ZLGkJqw3t/Htc8j3+PKQWoZvSwxnD/XjwA/kLQxsIpUKxzQeWx08KzPVvXyl+09wD9G\nxOPlLs9wys18X4uIBeUui1lv3IxmVU3SjqReXPOrLdCYVRLXbMzMrOhcszEzs6JzsDEzs6JzsDEz\ns6JzsDEzs6JzsDEzs6L7/8nK+MKNzoOnAAAAAElFTkSuQmCC\n", 1063 | "text/plain": [ 1064 | "" 1065 | ] 1066 | }, 1067 | "metadata": {}, 1068 | "output_type": "display_data" 1069 | } 1070 | ], 1071 | "source": [ 1072 | "plt.scatter(X_train, y_train, c='b')\n", 1073 | "plt.scatter(X_test, y_test, c='g')\n", 1074 | "plt.plot(X_test, pred_numpy_lr, c='r')\n", 1075 | "plt.title(\"Linear Regression - from Scratch\")\n", 1076 | "plt.xlabel(\"years of experience\")\n", 1077 | "plt.ylabel(\"Salary in $\")\n", 1078 | "plt.show()" 1079 | ] 1080 | } 1081 | ], 1082 | "metadata": { 1083 | "kernelspec": { 1084 | "display_name": "Python 3", 1085 | "language": "python", 1086 | "name": "python3" 1087 | }, 1088 | "language_info": { 1089 | "codemirror_mode": { 1090 | "name": "ipython", 1091 | "version": 3 1092 | }, 1093 | "file_extension": ".py", 1094 | "mimetype": "text/x-python", 1095 | "name": "python", 1096 | "nbconvert_exporter": "python", 1097 | "pygments_lexer": "ipython3", 1098 | "version": "3.5.3" 1099 | } 1100 | }, 1101 | "nbformat": 4, 1102 | "nbformat_minor": 0 1103 | } 1104 | -------------------------------------------------------------------------------- /algorithms_numpy/linear_regression_numpy.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | class LinearRegressioN(object): 4 | 5 | #in constructor - defining all hyperparameter 6 | def __init__(self): 7 | self.learning_rate = 0.0001 8 | #for accurecy play with how many iterations our gradient descent will run 9 | #100k = same output as in sklearn 10 | self.no_of_iter = 100000 11 | self.starting_b = 0 12 | self.starting_m = 0 13 | 14 | def train(self, X, y): 15 | #y = m*x + b 16 | self.X_train = X 17 | self.y_train = y 18 | 19 | #calling gradient descent function, and output of it is going to be our the best possible (according to our dataset) M and B 20 | self.new_b, self.new_m = self.gradient_descent(self.starting_b, self.starting_m, self.X_train, self.y_train, self.learning_rate, self.no_of_iter) 21 | 22 | return self.new_b, self.new_m 23 | 24 | #main function for gradient descent 25 | #INPUTS: STARTING b and m (alwyas start with b=0 and m=0) 26 | # X and y - Our training features and labels 27 | # learn_rate - is another hyperparam which will define "how fast" are it is going to find good B and M 28 | # no_of_iter - how many times our algorith is going to run 29 | def gradient_descent(self, b, m, X, y, learn_rate, no_of_iter): 30 | bG = b 31 | mG = m 32 | 33 | for i in range(no_of_iter): 34 | bG, mG = self.gradient_descent_step(bG, mG, X, y, learn_rate) 35 | 36 | return bG, mG 37 | #helper function for Gradient descent 38 | # current m and b - for each iteration its going to use that B and M 39 | # X, y - training features and labels 40 | # learn_rate - how fast it is going to move and to find the best possible M and B 41 | def gradient_descent_step(self, b, m, X, y, learn_rate): 42 | gradient_b = 0 43 | gradient_m = 0 44 | 45 | for i in range(len(X)): 46 | gradient_b += -(2/float(len(X)) * (y[i] - ((m*X[i]) + b))) 47 | gradient_m += (2/float(len(X))) * (-X[i]) * (y[i] - ((m*X[i]) + b)) 48 | 49 | new_b = b - (learn_rate * gradient_b) 50 | new_m = m - (learn_rate * gradient_m) 51 | return new_b, new_m 52 | 53 | #predict function to predict labels (y - values) for TEST set 54 | def predict(self, X): 55 | y_pred = [] 56 | for i in range(len(X)): 57 | #y = m*x + b 58 | y_pred.append(self.new_m*X[i] + self.new_b) 59 | 60 | return y_pred -------------------------------------------------------------------------------- /algorithms_numpy/multi_linear_regression_numpy.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | class LinearRegressionN(object): 4 | 5 | def __init__(self, number_of_interations= 100000, learning_rate= 0.01): 6 | self.number_of_interations = number_of_interations 7 | self.learning_rate = learning_rate 8 | 9 | def train(self, X, y): 10 | ''' 11 | This function is used to find the best parameters for our M values and B values in linear regression 12 | 13 | Input: X - Feature matrix with size NxM (N = number of samples, M = number of features for each sample) 14 | y - Values for each sample that we are trying to find (Example: Price of house(s)) 15 | ''' 16 | 17 | #Step 1. Define parameters for Linear Regression formula 18 | # y = m*X + b <-- Simple Linear Regression 19 | # y = m1*X1 + m2*X2 + m3*X3 + ... + mn*Xn + b <-- Multiple Linear Regression 20 | 21 | #Get shape[1] == number of features of our train set X and get vector of zeros for our M parameter. 22 | #In this way we can handle simple linear regresion and Multiple Linear Regression 23 | self.m = np.array(np.zeros(X.shape[1])) 24 | self.b = 0 25 | 26 | #Training loop 27 | for i in range(self.number_of_interations): 28 | 29 | gradient_m = np.zeros(X.shape[1]) 30 | gradient_b = 0 31 | 32 | b = self.b 33 | m = self.m 34 | 35 | #for each feature 36 | # for j in range(X.shape[1]): 37 | gradient_m = np.sum((2/X.shape[0]) * (-X) * (y - (m*X + b))) 38 | gradient_b = ((2/X.shape[0]) * (-(y - (m*X + b)))) 39 | 40 | b = b - (gradient_b * self.learning_rate) 41 | m = m - (gradient_m * self.learning_rate) 42 | 43 | 44 | self.m = m - (gradient_m * self.learning_rate) 45 | self.b = b - (gradient_b * self.learning_rate) 46 | 47 | 48 | 49 | def predict(self, X): 50 | predicted = [] 51 | for i in range(len(X)): 52 | predicted.append(np.sum(self.m * X[i] + self.b)) 53 | 54 | return predicted 55 | 56 | 57 | 58 | # To test Linear Regression uncomment this part 59 | # np.random.seed(0) 60 | # X = np.random.randn(100, 1) 61 | # y = np.random.randn(100, 1) 62 | 63 | # li = LinearRegressionN() 64 | # li.train(X, y) 65 | 66 | # # kurcina = LinearRegressioN() 67 | # # kurcina.train(X, y) 68 | # import matplotlib.pyplot as plt 69 | 70 | # plt.scatter(X, y) 71 | # plt.plot(X, li.predict(X)) 72 | # # plt.plot(X, kurcina.predict(X)) 73 | # plt.show() 74 | 75 | # m = np.ones(X.shape[1]) 76 | # b = 0 77 | # # print(m*X + b) --------------------------------------------------------------------------------