├── .gitignore ├── 01 Introduction of Classification ├── 01 Introduction of Classification.ipynb ├── README.md └── images │ ├── ANN.jpeg │ ├── CV.png │ ├── DT.png │ ├── KNN.png │ ├── KNN1.png │ ├── NB.png │ ├── README.md │ └── ROC.png ├── 02 Types of Classification Algorithm ├── 02 Types of Classification Algorithm.ipynb └── README.md ├── 03 Classification Algorithms ├── Intro & Types of Classifier.ipynb ├── README.md ├── data │ ├── README.md │ └── df.csv └── images │ ├── AS.png │ ├── DT.png │ ├── EDA.jpg │ ├── KNN.png │ ├── LR.png │ ├── NB.png │ ├── README.md │ ├── RF.png │ ├── SCD.png │ └── SVM.png ├── 04 Exploratory Data Analysis ├── 01 Data_Prep.sas ├── 02 Exploratory Data Analysis.ipynb ├── 02 Exploratory Data Analysis.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 05 Decision Tree ├── 01 Data_Prep.sas ├── 02 Decision Tree.ipynb ├── 02 Decision Tree.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 06 Exploratory Data Analysis ├── 01 Data_Prep.sas ├── 02 Exploratory Data Analysis.ipynb ├── 02 Exploratory Data Analysis.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 07 K-Nearest Neighbors ├── 01 Data_Prep.sas ├── 02 K-Neighbors Classifier.ipynb ├── 02 K-Neighbors Classifier.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 08 Logistic Regression Classifier ├── 01 Data_Prep.sas ├── 02 Logistic Regression.ipynb ├── 02 Logistic Regression.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 09 Naive_Bayes classifier ├── 01 Data_Prep.sas ├── 02 Naive_Bayes classifier.ipynb ├── 02 Naive_Bayes classifier.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 10 Random Forest ├── 01 Data_Prep.sas ├── 02 Random Forest.ipynb ├── 02 Random Forest.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 11 Stochastic Gradient Descent ├── 01 Data_Prep.sas ├── 02 Stochastic Gradient Descent.ipynb ├── 02 Stochastic Gradient Descent.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── 12 Support Vector Machine ├── 01 Data_Prep.sas ├── 02 Support Vector Machine.ipynb ├── 02 Support Vector Machine.py ├── README.md └── data │ ├── 00 df.csv │ └── README.md ├── LICENSE └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | target/ 76 | 77 | # Jupyter Notebook 78 | .ipynb_checkpoints 79 | 80 | # IPython 81 | profile_default/ 82 | ipython_config.py 83 | 84 | # pyenv 85 | .python-version 86 | 87 | # pipenv 88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 91 | # install all needed dependencies. 92 | #Pipfile.lock 93 | 94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 95 | __pypackages__/ 96 | 97 | # Celery stuff 98 | celerybeat-schedule 99 | celerybeat.pid 100 | 101 | # SageMath parsed files 102 | *.sage.py 103 | 104 | # Environments 105 | .env 106 | .venv 107 | env/ 108 | venv/ 109 | ENV/ 110 | env.bak/ 111 | venv.bak/ 112 | 113 | # Spyder project settings 114 | .spyderproject 115 | .spyproject 116 | 117 | # Rope project settings 118 | .ropeproject 119 | 120 | # mkdocs documentation 121 | /site 122 | 123 | # mypy 124 | .mypy_cache/ 125 | .dmypy.json 126 | dmypy.json 127 | 128 | # Pyre type checker 129 | .pyre/ 130 | -------------------------------------------------------------------------------- /01 Introduction of Classification/01 Introduction of Classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "---\n", 8 | "

Machine Learning Classifier

\n", 9 | "\n", 10 | "---" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": {}, 16 | "source": [ 17 | "---\n", 18 | "# **What is Classification?**\n", 19 | "\n", 20 | "---" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "### Classification is the process of predicting the class of given data points. Classes are sometimes called as targets/ labels or categories. Classification predictive modeling is the task of approximating a mapping function (f) from input variables (X) to discrete output variables (y).\n", 28 | "### For example, spam detection in email service providers can be identified as a classification problem. This is s binary classification since there are only 2 classes as spam and not spam. A classifier utilizes some training data to understand how given input variables relate to the class. In this case, known spam and non-spam emails have to be used as the training data. When the classifier is trained accurately, it can be used to detect an unknown email.\n", 29 | "### Classification belongs to the category of supervised learning where the targets also provided with the input data. There are many applications in classification in many domains such as in credit approval, medical diagnosis, target marketing etc.\n", 30 | "### There are two types of learners in classification as lazy learners and eager learners." 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "## **1.Lazy learners**" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "### Lazy learners simply store the training data and wait until a testing data appear. When it does, classification is conducted based on the most related data in the stored training data. Compared to eager learners, lazy learners have less training time but more time in predicting.\n", 45 | "### *Ex. k-nearest neighbor, Case-based reasoning*" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "## **2. Eager learners**" 53 | ] 54 | }, 55 | { 56 | "cell_type": "markdown", 57 | "metadata": {}, 58 | "source": [ 59 | "### Eager learners construct a classification model based on the given training data before receiving data for classification. It must be able to commit to a single hypothesis that covers the entire instance space. Due to the model construction, eager learners take a long time for train and less time to predict.\n", 60 | "### *Ex. Decision Tree, Naive Bayes, Artificial Neural Networks*" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "---\n", 68 | "# **Classification algorithms**\n", 69 | "---" 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "### There is a lot of classification algorithms available now but it is not possible to conclude which one is superior to other. It depends on the application and nature of available data set. For example, if the classes are linearly separable, the linear classifiers like Logistic regression, Fisher’s linear discriminant can outperform sophisticated models and vice versa." 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "---\n", 84 | "## **1. Decision Tree**" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "### Decision tree builds classification or regression models in the form of a tree structure. It utilizes an if-then rule set which is mutually exclusive and exhaustive for classification. The rules are learned sequentially using the training data one at a time. Each time a rule is learned, the tuples covered by the rules are removed. This process is continued on the training set until meeting a termination condition." 99 | ] 100 | }, 101 | { 102 | "cell_type": "markdown", 103 | "metadata": {}, 104 | "source": [ 105 | "### The tree is constructed in a top-down recursive divide-and-conquer manner. All the attributes should be categorical. Otherwise, they should be discretized in advance. Attributes in the top of the tree have more impact towards in the classification and they are identified using the information gain concept." 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "### A decision tree can be easily over-fitted generating too many branches and may reflect anomalies due to noise or outliers. An over-fitted model has a very poor performance on the unseen data even though it gives an impressive performance on training data. This can be avoided by pre-pruning which halts tree construction early or post-pruning which removes branches from the fully grown tree." 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "---\n", 120 | "## **2. Naive Bayes**" 121 | ] 122 | }, 123 | { 124 | "cell_type": "markdown", 125 | "metadata": {}, 126 | "source": [ 127 | "### Naive Bayes is a probabilistic classifier inspired by the Bayes theorem under a simple assumption which is the attributes are conditionally independent." 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "" 135 | ] 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "metadata": {}, 140 | "source": [ 141 | "### The classification is conducted by deriving the maximum posterior which is the maximal P(Ci|X) with the above assumption applying to Bayes theorem. This assumption greatly reduces the computational cost by only counting the class distribution. Even though the assumption is not valid in most cases since the attributes are dependent, surprisingly Naive Bayes has able to perform impressively." 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "### Naive Bayes is a very simple algorithm to implement and good results have obtained in most cases. It can be easily scalable to larger datasets since it takes linear time, rather than by expensive iterative approximation as used for many other types of classifiers." 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": {}, 154 | "source": [ 155 | "### Naive Bayes can suffer from a problem called the zero probability problem. When the conditional probability is zero for a particular attribute, it fails to give a valid prediction. This needs to be fixed explicitly using a Laplacian estimator." 156 | ] 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "metadata": {}, 161 | "source": [ 162 | "---\n", 163 | "## **3. Artificial Neural Networks**" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "" 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": {}, 176 | "source": [ 177 | "### Artificial Neural Network is a set of connected input/output units where each connection has a weight associated with it started by psychologists and neurobiologists to develop and test computational analogs of neurons. During the learning phase, the network learns by adjusting the weights so as to be able to predict the correct class label of the input tuples." 178 | ] 179 | }, 180 | { 181 | "cell_type": "markdown", 182 | "metadata": {}, 183 | "source": [ 184 | "### There are many network architectures available now like Feed-forward, Convolutional, Recurrent etc. The appropriate architecture depends on the application of the model. For most cases feed-forward models give reasonably accurate results and especially for image processing applications, convolutional networks perform better." 185 | ] 186 | }, 187 | { 188 | "cell_type": "markdown", 189 | "metadata": {}, 190 | "source": [ 191 | "### There can be multiple hidden layers in the model depending on the complexity of the function which is going to be mapped by the model. Having more hidden layers will enable to model complex relationships such as deep neural networks.\n" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "### However, when there are many hidden layers, it takes a lot of time to train and adjust wights. The other disadvantage of is the poor interpretability of model compared to other models like Decision Trees due to the unknown symbolic meaning behind the learned weights.\n" 199 | ] 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "### But Artificial Neural Networks have performed impressively in most of the real world applications. It is high tolerance to noisy data and able to classify untrained patterns. Usually, Artificial Neural Networks perform better with continuous-valued inputs and outputs." 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "metadata": {}, 211 | "source": [ 212 | "### *All of the above algorithms are eager learners since they train a model in advance to generalize the training data and use it for prediction later.*" 213 | ] 214 | }, 215 | { 216 | "cell_type": "markdown", 217 | "metadata": {}, 218 | "source": [ 219 | "---\n", 220 | "## **4. k-Nearest Neighbor (KNN)**" 221 | ] 222 | }, 223 | { 224 | "cell_type": "markdown", 225 | "metadata": {}, 226 | "source": [ 227 | "" 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "### k-Nearest Neighbor is a lazy learning algorithm which stores all instances correspond to training data points in n-dimensional space. When an unknown discrete data is received, it analyzes the closest k number of instances saved (nearest neighbors)and returns the most common class as the prediction and for real-valued data it returns the mean of k nearest neighbors.\n" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": {}, 240 | "source": [ 241 | "### In the distance-weighted nearest neighbor algorithm, it weights the contribution of each of the k neighbors according to their distance using the following query giving greater weight to the closest neighbors.\n" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "### Usually KNN is robust to noisy data since it is averaging the k-nearest neighbors.\n" 256 | ] 257 | }, 258 | { 259 | "cell_type": "markdown", 260 | "metadata": {}, 261 | "source": [ 262 | "---\n", 263 | "# **Evaluating a classifier**\n", 264 | "---" 265 | ] 266 | }, 267 | { 268 | "cell_type": "markdown", 269 | "metadata": {}, 270 | "source": [ 271 | "### After training the model the most important part is to evaluate the classifier to verify its applicability.\n" 272 | ] 273 | }, 274 | { 275 | "cell_type": "markdown", 276 | "metadata": {}, 277 | "source": [ 278 | "---\n", 279 | "## **Holdout method**" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "### There are several methods exists and the most common method is the holdout method. In this method, the given data set is divided into 2 partitions as test and train 20% and 80% respectively. The train set will be used to train the model and the unseen test data will be used to test its predictive power.\n" 287 | ] 288 | }, 289 | { 290 | "cell_type": "markdown", 291 | "metadata": {}, 292 | "source": [ 293 | "---\n", 294 | "## **Cross-validation**" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "" 302 | ] 303 | }, 304 | { 305 | "cell_type": "markdown", 306 | "metadata": {}, 307 | "source": [ 308 | "### Over-fitting is a common problem in machine learning which can occur in most models. k-fold cross-validation can be conducted to verify that the model is not over-fitted. In this method, the data-set is randomly partitioned into k mutually exclusive subsets, each approximately equal size and one is kept for testing while others are used for training. This process is iterated throughout the whole k folds.\n" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "---\n", 316 | "## **Precision and Recall**" 317 | ] 318 | }, 319 | { 320 | "cell_type": "markdown", 321 | "metadata": {}, 322 | "source": [ 323 | "### Precision is the fraction of relevant instances among the retrieved instances, while recall is the fraction of relevant instances that have been retrieved over the total amount of relevant instances. Precision and Recall are used as a measurement of the relevance.\n" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "metadata": {}, 329 | "source": [ 330 | "---\n", 331 | "## **ROC curve ( Receiver Operating Characteristics)**" 332 | ] 333 | }, 334 | { 335 | "cell_type": "markdown", 336 | "metadata": {}, 337 | "source": [ 338 | "### ROC curve is used for visual comparison of classification models which shows the trade-off between the true positive rate and the false positive rate. The area under the ROC curve is a measure of the accuracy of the model. When a model is closer to the diagonal, it is less accurate and the model with perfect accuracy will have an area of 1.0\n" 339 | ] 340 | }, 341 | { 342 | "cell_type": "markdown", 343 | "metadata": {}, 344 | "source": [ 345 | "" 346 | ] 347 | }, 348 | { 349 | "cell_type": "code", 350 | "execution_count": null, 351 | "metadata": {}, 352 | "outputs": [], 353 | "source": [] 354 | } 355 | ], 356 | "metadata": { 357 | "kernelspec": { 358 | "display_name": "Python 3", 359 | "language": "python", 360 | "name": "python3" 361 | }, 362 | "language_info": { 363 | "codemirror_mode": { 364 | "name": "ipython", 365 | "version": 3 366 | }, 367 | "file_extension": ".py", 368 | "mimetype": "text/x-python", 369 | "name": "python", 370 | "nbconvert_exporter": "python", 371 | "pygments_lexer": "ipython3", 372 | "version": "3.6.8" 373 | } 374 | }, 375 | "nbformat": 4, 376 | "nbformat_minor": 4 377 | } 378 | -------------------------------------------------------------------------------- /01 Introduction of Classification/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /01 Introduction of Classification/images/ANN.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/ANN.jpeg -------------------------------------------------------------------------------- /01 Introduction of Classification/images/CV.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/CV.png -------------------------------------------------------------------------------- /01 Introduction of Classification/images/DT.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/DT.png -------------------------------------------------------------------------------- /01 Introduction of Classification/images/KNN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/KNN.png -------------------------------------------------------------------------------- /01 Introduction of Classification/images/KNN1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/KNN1.png -------------------------------------------------------------------------------- /01 Introduction of Classification/images/NB.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/NB.png -------------------------------------------------------------------------------- /01 Introduction of Classification/images/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /01 Introduction of Classification/images/ROC.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/01 Introduction of Classification/images/ROC.png -------------------------------------------------------------------------------- /02 Types of Classification Algorithm/02 Types of Classification Algorithm.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "---\n", 8 | "

Types of Machine Learning Classifier

\n", 9 | "\n", 10 | "---" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": {}, 16 | "source": [ 17 | "### In machine learning and statistics, classification is a supervised learning approach in which the computer program learns from the data input given to it and then uses this learning to classify new observation. This data set may simply be bi-class (like identifying whether the person is male or female or that the mail is spam or non-spam) or it may be multi-class too. Some examples of classification problems are: speech recognition, handwriting recognition, bio metric identification, document classification etc.\n", 18 | "### Here we have the types of classification algorithms in Machine Learning:\n", 19 | "### 1. Linear Classifiers: Logistic Regression, Naive Bayes Classifier\n", 20 | "### 2. Nearest Neighbor\n", 21 | "### 3. Support Vector Machines\n", 22 | "### 4. Decision Trees\n", 23 | "### 5. Boosted Trees\n", 24 | "### 6. Random Forest\n", 25 | "### 7. Neural Networks" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "---\n", 33 | "## **Naive Bayes Classifier (Generative Learning Model) :**" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "### It is a classification technique based on Bayes’ Theorem with an assumption of independence among predictors. In simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. Even if these features depend on each other or upon the existence of the other features, all of these properties independently contribute to the probability. Naive Bayes model is easy to build and particularly useful for very large data sets. Along with simplicity, Naive Bayes is known to outperform even highly sophisticated classification methods." 41 | ] 42 | }, 43 | { 44 | "cell_type": "markdown", 45 | "metadata": {}, 46 | "source": [ 47 | "---\n", 48 | "## **Nearest Neighbor:**" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "### The k-nearest-neighbors algorithm is a classification algorithm, and it is supervised: it takes a bunch of labelled points and uses them to learn how to label other points. To label a new point, it looks at the labelled points closest to that new point (those are its nearest neighbors), and has those neighbors vote, so whichever label the most of the neighbors have is the label for the new point (the “k” is the number of neighbors it checks)." 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "---\n", 63 | "## **Logistic Regression (Predictive Learning Model) :**" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "### It is a statistical method for analysing a data set in which there are one or more independent variables that determine an outcome. The outcome is measured with a dichotomous variable (in which there are only two possible outcomes). The goal of logistic regression is to find the best fitting model to describe the relationship between the dichotomous characteristic of interest (dependent variable = response or outcome variable) and a set of independent (predictor or explanatory) variables. This is better than other binary classification like nearest neighbor since it also explains quantitatively the factors that lead to classification." 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": {}, 76 | "source": [ 77 | "---\n", 78 | "## **Decision Trees:**" 79 | ] 80 | }, 81 | { 82 | "cell_type": "markdown", 83 | "metadata": {}, 84 | "source": [ 85 | "### Decision tree builds classification or regression models in the form of a tree structure. It breaks down a data set into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. The final result is a tree with decision nodes and leaf nodes. A decision node has two or more branches and a leaf node represents a classification or decision. The topmost decision node in a tree which corresponds to the best predictor called root node. Decision trees can handle both categorical and numerical data." 86 | ] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "metadata": {}, 91 | "source": [ 92 | "---\n", 93 | "## **Random Forest:**" 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "### Random forests or random decision forests are an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Random decision forests correct for decision trees’ habit of over fitting to their training set." 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "---\n", 108 | "## **Neural Network:**" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [ 115 | "### A neural network consists of units (neurons), arranged in layers, which convert an input vector into some output. Each unit takes an input, applies a (often nonlinear) function to it and then passes the output on to the next layer. Generally the networks are defined to be feed-forward: a unit feeds its output to all the units on the next layer, but there is no feedback to the previous layer. Weightings are applied to the signals passing from one unit to another, and it is these weightings which are tuned in the training phase to adapt a neural network to the particular problem at hand.\n" 116 | ] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "execution_count": null, 121 | "metadata": {}, 122 | "outputs": [], 123 | "source": [] 124 | } 125 | ], 126 | "metadata": { 127 | "kernelspec": { 128 | "display_name": "Python 3", 129 | "language": "python", 130 | "name": "python3" 131 | }, 132 | "language_info": { 133 | "codemirror_mode": { 134 | "name": "ipython", 135 | "version": 3 136 | }, 137 | "file_extension": ".py", 138 | "mimetype": "text/x-python", 139 | "name": "python", 140 | "nbconvert_exporter": "python", 141 | "pygments_lexer": "ipython3", 142 | "version": "3.6.8" 143 | } 144 | }, 145 | "nbformat": 4, 146 | "nbformat_minor": 4 147 | } 148 | -------------------------------------------------------------------------------- /02 Types of Classification Algorithm/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /03 Classification Algorithms/Intro & Types of Classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### the 7 most commonly used classification algorithms along with the python code: Logistic Regression, Naïve Bayes, Stochastic Gradient Descent, K-Nearest Neighbours, Decision Tree, Random Forest, and Support Vector Machine" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "---\n", 15 | "# **1. Introduction**" 16 | ] 17 | }, 18 | { 19 | "cell_type": "markdown", 20 | "metadata": {}, 21 | "source": [ 22 | "---\n", 23 | "## **1.1 Structured Data Classification**" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "### Classification can be performed on structured or unstructured data. Classification is a technique where we categorize data into a given number of classes. The main goal of a classification problem is to identify the category/class to which a new data will fall under." 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "### Few of the terminologies encountered in machine learning – classification:" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "### **1.Classifier:** An algorithm that maps the input data to a specific category." 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "### **2. Classification model:** A classification model tries to draw some conclusion from the input values given for training. It will predict the class labels/categories for the new data.\n", 52 | "### **3. Feature:** A feature is an individual measurable property of a phenomenon being observed.\n", 53 | "### **4. Binary Classification:** Classification task with two possible outcomes. Eg: Gender classification (Male / Female)\n", 54 | "### **5. Multi class classification:** Classification with more than two classes. In multi class classification each sample is assigned to one and only one target label. Eg: An animal can be cat or dog but not both at the same time\n", 55 | "### **6. Multi label classification:** Classification task where each sample is mapped to a set of target labels (more than one class). Eg: A news article can be about sports, a person, and location at the same time." 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "---\n", 63 | "### The following are the steps involved in building a classification model:" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "### **1. Initialize** the classifier to be used.\n", 71 | "### **2. Train the classifier:** All classifiers in scikit-learn uses a fit(X, y) method to fit the model(training) for the given train data X and train label y.\n", 72 | "### **3. Predict the target:** Given an unlabeled observation X, the predict(X) returns the predicted label y.\n", 73 | "### **4. Evaluate** the classifier model" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "---\n", 81 | "## **1.2 Dataset Source and Contents**" 82 | ] 83 | }, 84 | { 85 | "cell_type": "markdown", 86 | "metadata": {}, 87 | "source": [ 88 | "### The dataset contains salaries. The following is a description of our dataset:" 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": {}, 94 | "source": [ 95 | "### **of Classes:** 2 (‘>50K’ and ‘<=50K’)\n", 96 | "### **of attributes (Columns):** 7\n", 97 | "### **of instances (Rows):** 48,842" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "---\n", 105 | "## **1.3 Exploratory Data Analysis**" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "---\n", 120 | "# **2 Classification Algorithms (Python)**\n", 121 | "---" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "## **2.1 Logistic Regression**" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": {}, 141 | "source": [ 142 | "### **Definition:** Logistic regression is a machine learning algorithm for classification. In this algorithm, the probabilities describing the possible outcomes of a single trial are modelled using a logistic function.\n", 143 | "\n", 144 | "### **Advantages:** Logistic regression is designed for this purpose (classification), and is most useful for understanding the influence of several independent variables on a single outcome variable.\n", 145 | "\n", 146 | "### **Disadvantages:** Works only when the predicted variable is binary, assumes all predictors are independent of each other, and assumes data is free of missing values." 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "---\n", 154 | "## **2.2 Naïve Bayes**" 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "metadata": {}, 160 | "source": [ 161 | "" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "### **Definition:** Naive Bayes algorithm based on Bayes’ theorem with the assumption of independence between every pair of features. Naive Bayes classifiers work well in many real-world situations such as document classification and spam filtering.\n", 169 | "\n", 170 | "### **Advantages:** This algorithm requires a small amount of training data to estimate the necessary parameters. Naive Bayes classifiers are extremely fast compared to more sophisticated methods.\n", 171 | "\n", 172 | "### **Disadvantages:** Naive Bayes is is known to be a bad estimator." 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "metadata": {}, 178 | "source": [ 179 | "---\n", 180 | "## **2.3 Stochastic Gradient Descent**" 181 | ] 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "metadata": {}, 186 | "source": [ 187 | "" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "### **Definition:** Stochastic gradient descent is a simple and very efficient approach to fit linear models. It is particularly useful when the number of samples is very large. It supports different loss functions and penalties for classification.\n", 195 | "\n", 196 | "### **Advantages:** Efficiency and ease of implementation.\n", 197 | "\n", 198 | "### **Disadvantages:** Requires a number of hyper-parameters and it is sensitive to feature scaling." 199 | ] 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "---\n", 206 | "## **2.4 K-Nearest Neighbours**" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": {}, 219 | "source": [ 220 | "### **Definition:** Neighbours based classification is a type of lazy learning as it does not attempt to construct a general internal model, but simply stores instances of the training data. Classification is computed from a simple majority vote of the k nearest neighbours of each point.\n", 221 | "\n", 222 | "### **Advantages:** This algorithm is simple to implement, robust to noisy training data, and effective if training data is large.\n", 223 | "\n", 224 | "### **Disadvantages:** Need to determine the value of K and the computation cost is high as it needs to computer the distance of each instance to all the training samples." 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "---\n", 232 | "## **2.5 Decision Tree**" 233 | ] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": {}, 238 | "source": [ 239 | "" 240 | ] 241 | }, 242 | { 243 | "cell_type": "markdown", 244 | "metadata": {}, 245 | "source": [ 246 | "### **Definition:** Given a data of attributes together with its classes, a decision tree produces a sequence of rules that can be used to classify the data.\n", 247 | "\n", 248 | "### **Advantages:** Decision Tree is simple to understand and visualise, requires little data preparation, and can handle both numerical and categorical data.\n", 249 | "\n", 250 | "### **Disadvantages:** Decision tree can create complex trees that do not generalise well, and decision trees can be unstable because small variations in the data might result in a completely different tree being generated." 251 | ] 252 | }, 253 | { 254 | "cell_type": "markdown", 255 | "metadata": {}, 256 | "source": [ 257 | "---\n", 258 | "## **2.6 Random Forest**" 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "### **Definition:** Random forest classifier is a meta-estimator that fits a number of decision trees on various sub-samples of datasets and uses average to improve the predictive accuracy of the model and controls over-fitting. The sub-sample size is always the same as the original input sample size but the samples are drawn with replacement.\n", 273 | "\n", 274 | "### **Advantages:** Reduction in over-fitting and random forest classifier is more accurate than decision trees in most cases.\n", 275 | "\n", 276 | "### **Disadvantages:** Slow real time prediction, difficult to implement, and complex algorithm." 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "---\n", 284 | "## **2.7 Support Vector Machine**" 285 | ] 286 | }, 287 | { 288 | "cell_type": "markdown", 289 | "metadata": {}, 290 | "source": [ 291 | "" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "### **Definition:** Support vector machine is a representation of the training data as points in space separated into categories by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall.\n", 299 | "\n", 300 | "### **Advantages:** Effective in high dimensional spaces and uses a subset of training points in the decision function so it is also memory efficient.\n", 301 | "\n", 302 | "### **Disadvantages:** The algorithm does not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation." 303 | ] 304 | }, 305 | { 306 | "cell_type": "markdown", 307 | "metadata": {}, 308 | "source": [ 309 | "---\n", 310 | "# **3 Conclusion**\n", 311 | "---" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "### **3.1 Comparison Matrix**" 319 | ] 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "metadata": {}, 324 | "source": [ 325 | ">- **Accuracy: (True Positive + True Negative) / Total Population**\n", 326 | ">>- Accuracy is a ratio of correctly predicted observation to the total observations. Accuracy is the most intuitive performance measure.\n", 327 | ">>- True Positive: The number of correct predictions that the occurrence is positive\n", 328 | ">>- True Negative: The number of correct predictions that the occurrence is negative" 329 | ] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": {}, 334 | "source": [ 335 | ">- **F1-Score: (2 x Precision x Recall) / (Precision + Recall)**\n", 336 | "\n", 337 | ">>- F1-Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. F1-Score is usually more useful than accuracy, especially if you have an uneven class distribution.\n", 338 | ">>- Precision: When a positive value is predicted, how often is the prediction correct?\n", 339 | ">>- Recall: When the actual value is positive, how often is the prediction correct?" 340 | ] 341 | }, 342 | { 343 | "cell_type": "raw", 344 | "metadata": {}, 345 | "source": [] 346 | }, 347 | { 348 | "cell_type": "raw", 349 | "metadata": {}, 350 | "source": [ 351 | "Classification Algorithm | Accuracy | F1-Score\n", 352 | "\n", 353 | "Logistic Regression | 84.60% | 0.6337\n", 354 | "Naïve Bayes | 80.11% | 0.6005\n", 355 | "Stochastic Gradient Descent | 82.20% | 0.5780\n", 356 | "K-Nearest Neighbours | 83.56% | 0.5924\n", 357 | "Decision Tree | 84.23% | 0.6308\n", 358 | "Random Forest | 84.33% | 0.6275\n", 359 | "Support Vector Machine | 84.09% | 0.6145" 360 | ] 361 | }, 362 | { 363 | "cell_type": "markdown", 364 | "metadata": {}, 365 | "source": [ 366 | "---\n", 367 | "## **3.2 Algorithm Selection**" 368 | ] 369 | }, 370 | { 371 | "cell_type": "markdown", 372 | "metadata": {}, 373 | "source": [ 374 | "" 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": null, 380 | "metadata": {}, 381 | "outputs": [], 382 | "source": [] 383 | } 384 | ], 385 | "metadata": { 386 | "kernelspec": { 387 | "display_name": "Python 3", 388 | "language": "python", 389 | "name": "python3" 390 | }, 391 | "language_info": { 392 | "codemirror_mode": { 393 | "name": "ipython", 394 | "version": 3 395 | }, 396 | "file_extension": ".py", 397 | "mimetype": "text/x-python", 398 | "name": "python", 399 | "nbconvert_exporter": "python", 400 | "pygments_lexer": "ipython3", 401 | "version": "3.6.8" 402 | } 403 | }, 404 | "nbformat": 4, 405 | "nbformat_minor": 4 406 | } 407 | -------------------------------------------------------------------------------- /03 Classification Algorithms/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /03 Classification Algorithms/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /03 Classification Algorithms/images/AS.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/AS.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/DT.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/DT.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/EDA.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/EDA.jpg -------------------------------------------------------------------------------- /03 Classification Algorithms/images/KNN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/KNN.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/LR.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/LR.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/NB.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/NB.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /03 Classification Algorithms/images/RF.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/RF.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/SCD.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/SCD.png -------------------------------------------------------------------------------- /03 Classification Algorithms/images/SVM.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Awesome-Machine-Learning/Machine-Learning-Classifications/9e22e12dfc3da18d18478f13a1b96c47680ab497/03 Classification Algorithms/images/SVM.png -------------------------------------------------------------------------------- /04 Exploratory Data Analysis/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /04 Exploratory Data Analysis/02 Exploratory Data Analysis.py: -------------------------------------------------------------------------------- 1 | # import libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | import seaborn as sns 7 | import matplotlib.pyplot as plt 8 | %matplotlib inline 9 | plt.style.use('seaborn-whitegrid') 10 | 11 | 12 | # Import the data 13 | df = pd.read_csv('data/00 df.csv') 14 | df = df[df['flag']=='train'] 15 | # print(df.info()) 16 | 17 | 18 | # Exploratory Data Analysis & plot the data 19 | #age_bin 20 | x_chart = df.pivot_table(values=['flag'], index=['age_bin'], columns=['y'], aggfunc='count') 21 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 22 | x_chart.plot(kind="bar",stacked=True) 23 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 24 | 25 | #capital_gl_bin 26 | x_chart = df.pivot_table(values=['flag'], index=['capital_gl_bin'], columns=['y'], aggfunc='count') 27 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 28 | x_chart.plot(kind="bar",stacked=True) 29 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 30 | 31 | #education_bin 32 | x_chart = df.pivot_table(values=['flag'], index=['education_bin'], columns=['y'], aggfunc='count') 33 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 34 | x_chart.plot(kind="bar",stacked=True) 35 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 36 | 37 | 38 | #hours_per_week_bin 39 | x_chart = df.pivot_table(values=['flag'], index=['hours_per_week_bin'], columns=['y'], aggfunc='count') 40 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 41 | x_chart.plot(kind="bar",stacked=True) 42 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 43 | 44 | 45 | #msr_bin 46 | x_chart = df.pivot_table(values=['flag'], index=['msr_bin'], columns=['y'], aggfunc='count') 47 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 48 | x_chart.plot(kind="bar",stacked=True) 49 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 50 | 51 | 52 | #occupation_bin 53 | x_chart = df.pivot_table(values=['flag'], index=['occupation_bin'], columns=['y'], aggfunc='count') 54 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 55 | x_chart.plot(kind="bar",stacked=True) 56 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 57 | 58 | 59 | #race_sex_bin 60 | x_chart = df.pivot_table(values=['flag'], index=['race_sex_bin'], columns=['y'], aggfunc='count') 61 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 62 | x_chart.plot(kind="bar",stacked=True) 63 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 64 | 65 | 66 | 67 | 68 | 69 | -------------------------------------------------------------------------------- /04 Exploratory Data Analysis/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /04 Exploratory Data Analysis/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /05 Decision Tree/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /05 Decision Tree/02 Decision Tree.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import math\n", 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from datetime import datetime\n", 13 | "\n", 14 | "import seaborn as sns\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "%matplotlib inline \n", 17 | "plt.style.use('seaborn-whitegrid')\n", 18 | "\n", 19 | "from sklearn.tree import DecisionTreeClassifier\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.metrics import confusion_matrix" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Get the Data" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "df = pd.read_csv('data/00 df.csv')\n", 38 | "train = df[df['flag']=='train']\n", 39 | "test = df[df['flag']=='test']" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 3, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']\n", 49 | "\n", 50 | "y_train = train['y']\n", 51 | "x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 52 | "x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True)\n", 53 | "\n", 54 | "y_test = test['y']\n", 55 | "x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 56 | "x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# Decision Tree" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [ 71 | { 72 | "data": { 73 | "text/plain": [ 74 | "" 75 | ] 76 | }, 77 | "execution_count": 4, 78 | "metadata": {}, 79 | "output_type": "execute_result" 80 | }, 81 | { 82 | "data": { 83 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe8AAAD0CAYAAACy764hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deVxU9f4/8NfMwLCPiCumgCAi4EJoLve6leGa3QyTLc0lNbNcbvnTAJPcMOvrVUvNoaJCEK83b0loLuSGmimmCIxmieCeK8M+zMz5/aHOlcVY5cwMr+fj0cNhzgy83uH4mjnzmXMkgiAIICIiIpMhFTsAERER1Q7Lm4iIyMSwvImIiEwMy5uIiMjEsLyJiIhMDMubiIjIxFiIHaCitLQ0sSMQERE1qp49e9bq9kZX3kDthzBGKpUK3t7eYseoF3OYATCPOcxhBoBzGBNzmAEwjznq8qKVu82JiIhMDMubiIjIxLC8iYiITAzLm4iIyMSwvImIiEwMy5vIiGXFx0Pp5oZkX18o3dyQFR8vdiQiMgJG+VExIrpf3LunTYO2qAgAoM7Jwe5p0wAAPmFhYkYjIpHxlTeREdEUFuLa8ePI+Oor7H3zTUNxP6QtKkJqRIRI6YjIWPCVN5EIyoqKcFulwu3MTNzKzDT8qb54sdr7qnNykLN3L1yGDIFEInnyYYnI6LC8iZ6gsuJi3Dl7tlJJ52VnA4IAAJDJ5Wju5YV2ffui25QpaOnrixa+vtgaEID83NxK31MilWJrQABa+Pqi55w58A4Lg6WNTWOPRkQiYnkTNQBtSQnunDtXuaQvXICg1wMApBYWaO7lhba9esH3tdcMJd28UydILSo/FAcsX17uPW8AsLC1xfPr1wMATq5ejd1Tp+LQggXoPn06/N58Ew5PPdU4AxORqFjeRLWgLS3F3d9+q1TS937/3VDSEpkMzTt3Rms/P3iHhf2vpD09IbO0rPHPergoLTUiAurcXChcXNB/2TLD9b4TJuDywYNIW70ax6KjcXzlSniNGwf/2bPh3Lt3ww9PREaD5U1UBZ1Gg7vnz5cr6NuZmbh7/jwEnQ7Ag5Lu1Aktu3aFV1CQoaSdOneGTC5vkBw+YWHwCQur8uQLEokEHQYNQodBg3DvwgX8+umnOPP551AlJKBdv37wnzMHnV9+ucpX9URk2vioJrOVFR9veNV6qMKr1od0ZWW4e/585VfS589Dr9UCuP8es6OHB1r4+qJzYCBa+Pqipa8vmnt5wcLKSozRKnF0d8ezq1bhb1FRyPzqK5xcuxY/BAXBoUMH+M2cie5Tp8LGyUnsmETUQFjeZJaq+oz0rtdfx/VffoF1ixaGkr7722/Ql5Xdv5NEAkd3d7Tw9YXnSy+VK2lTWRBmpVDAf9Ys+M2ciewdO5C2ejUOLViAox98AN/XXoP/rFloYeKnTyQiljeZqdSIiEqfkdaVlODk2rUAgGYdO6KFry88XnjBUNJOXbrA0tZWjLgNTiqTwWP0aHiMHo2b6elIW7MGGbGxOP3ZZ3AbPhw9Z8+G29ChkEh5qAciU8TyJrOkruIjVgAAiQSz8vMht7Nr3EAiatW9O4Z/8QUGrliB0xs34tS6dfh2xAg4dekC/9mz4TN+fJP6/0FkDvi0m8yKtqQE++bONXyGuiKFi0uTLSrbVq3QLzIS03JyMDIuDpZ2dtg7YwaUHTrg4IIFUF+6JHZEIqohljeZjZtnzmBT795IW70aLgEBsKjwPrWFrS36L1smUjrjIZPL4fPqq3j1+HGEpKbCZcgQHP/oI8R07IikoCBcPXoUwmOe/BCRcWB5k8kT9Hqc+Ne/sKlXLxT9+SdeTk7GuN27MTQmBgpXV0AigcLVFUOVSp7Q4xESiQRP/f3veHHrVrx+4QJ6zp2Li7t2IeFvf0N8nz5QJSRAp9GIHZOIqsDyJpOWf+UK/jNsGPb/859wGzYMr6Wnw33kSAD3PyM97eJFjMrMxLSLF1ncf6GZqysGf/QRpl++jCHr1qE0Lw/JYWGI6dgRPy9fjqJbt8SOSESPYHmTyfrt22/xdffuuHLkCAI2bsRL338Pu9atxY5l0uT29nj6zTcxWaXCy8nJaOHri9SICCg7dMCuqVNxMyND7IhEBJY3mSBNfj52TpqE7WPHopm7Oyb8+it6TJvGM2w1IIlUCveRI/HK7t2YmJEBnwkToNq0CV9364atAQH4IznZcDhYImp8LG8yKVeOHMHXfn7I+uYb9I2IQOiRI3Dq3FnsWGatpa8vhm7ciOmXL2PA8uW4rVLhvy+8gC+7dMHJTz+FpqBA7IhETQ7Lm0yCrqwMhxctQuKAARD0egQdOID+S5fW6kQfVD82LVqgz3vvYWp2Nl7YvBlWzZvjp7ffxsb27bH/3XeRV4NzkRNRw2B5k9G7+/vvSBwwAEcXL4b3q6/itVOn0L5/f7FjNVkyS0t0CQ7Gq8eOIfToUbgNH4601avxuYcHvg8MxOVDh/hRM6InjOVNRksQBKR//jm+8fPDnXPn8MKWLRj59dewatZM7Gj0QLu+fTE6MRFTs7PxzP/7f7i0bx8SBw7Epl69kBkXB21pKYD7x5pXurkh2dcXSjc3ZMXHi5ycyLSxvMkoFd26he9ffhm7p06Fc58+mHjmDLqMGyd2LHoMRYcOGBgdjemXLyPgs89QVlyMnRMmIMbNDd8HBmL31KlQ5+QAggB1Tg52T5vGAqd6aepPCFneZHSyd+3C1926IXvHDgz6+GO8smcPHNq3FzsW1YClrS16TJ+OSZmZGLtrF1o//TTOb9sGbXFxudtpi4qQGhEhUsqmzRxK7+FZA5vyE0KemISMRllxMQ4tWICTa9eihY8PAn/8Ea179BA7FtWBRCKB29ChcBs6FB9LpVUea16dk4PEwYPh6O4ORw8PNHvkT5sWLfjRvyegqlPl7p42DQBqfBAjQa+HTqOBTqOBvqzsf5c1GujKyu7/WWG7vorbV7f9r7ZdPngQugdvyTykLSrCnjfewM30dFg1awa5QgErhcJwWf7IZSuFAhbW1g37P7eRsbzJKPx5+jSSw8JwOzMT/rNmYcCKFSZzDm36awoXl/uvkCqwsLODoNUi+8cfUXjtWrltcoUCju7u5Qr9Yck7uLjwUwY1pC0tReH16yi8ehUFV68i5e23K50qV1tUhF1TpuDk2rU1KuAn+fl+qaUlZHI5ZHK54bK0wtcyubxScT9UVlCAk2vWPHb7o2RyeZWl/riyf9z1Mrm8zvNmxccjNSICPb/9ttb3ZXmTqAS9HidWrUJqRASsnZwQ+OOP6DhsmNixqAH1X7as3Ks94P5JYoZu3Gh4tVdWVIS87GzkXbiAe3/8gXsXLiDvwgXczsrCheTkcv8YS2QyKFxcDIXezMOj3Kt3a0fHRp+xsenKysqVcsG1a/+7/OC/wmvXUFzDw9rqSkth7eRUqTirK1GpXA6ZpaVhW33uK7WwqPHeFqWbW5VPCBWurph28SK0paXQqNXQqNUoVatRmpf3v68fXK7q+vxLl3ArMxOavDyU5uVBr9VWm8XC2rrKUpc3a3b/z8dcfzk1FT8vWVLpLaWaYnmTaNSXLmHna6/h0r596PTSSxgaEwPbli3FjkUN7GFBp0ZEQJ2bC4WLC/ovW1ZuN62lrS1a+vqipa9vpfsLej0Krl41FPq9P/4w/Hn+u+9QfPNmudtbOzmVe6X+6Kt3h/btIZXJnuzA9aDXalF448b98v2LYq44M3D/SY29szPsnJ3h6O6Op/r3h327drBv1w52zs6wb9cO2154AQWXL1e6r8LVFWN37myMERvE454QPjxroIWVFSxatYJtq1Z1/hmCIEBXWlqu7DV5eff/rPAkoOL1ednZ5a4XdLp6z1wRy5tEcfbf/8ae6dOhLyvDsM8/R9fJk/kepxnzCQuDT1gYVCoVvL29a3VfiVQKh/bt4dC+PToMHFhpuyY/v1yxP7x84+RJnN+2rdyrJ6mlJZq5uVW5O76Zuzvk9vZ/meXhbk51bi4OVfEk5HH0Oh2K/vyz2lIu+vPPSusDJFIp7Nq2hZ2zMxQuLmjXty/sKpSyfbt2sGnZstonJgNXrPjL0jMVNXlCWF8SiQQW1tawsLaGXZs2df4+giBAW1xcqey3Dh1a5VqQmmJ5U6MqVauR8tZbyIqLQ9vevTFq0yY09/QUOxaZMLmDA1r36FHl4ka9Tof8S5fuF3uFV+3Xf/kFJXfvlru9bevWj90df2nfPuyePr3SYq/SvDy069fvf8VcVSnfuFH5vWKJBLatWxvKt22vXvdL+UEhPyxo29atG2xvQWOUXmOpzxPCxiSRSGBpawtLW1vA2dlw/ePWgtQUy5sazeXUVOwYPx75ubno9/776BsZyYVH9ERJZbL7r7Td3ODy3HOVtpfcvVt5d/yFC7hy5AjOJiZWuzhLW1SElJkzK11v06qVoZRb+/lVWcp2bdpAatH4/wSbSumZu6p2/ddGtX9z9Ho9oqKicO7cOcjlcixduhSurq6G7du3b0dsbCykUikCAwMRGhoKnU6HyMhIZGdnQyaTITo6Gi4uLsjMzMQbb7wBNzc3AEBISAhGPjj3MpkvXVkZjn7wAY5FR0Ph5oaQ1FS069dP7FhEsG7eHG179kTbnj0rbdOVlUGdk2Mo9L0zZjz2+/xj2zZDMdu1aVOvFcjUNDy6F6Quqi3vvXv3QqPRYMuWLTh16hRWrFiBDRs2GLavXLkSP/zwA2xtbTFq1CiMGjUKx48fBwAkJibi2LFjiI6OxoYNG5CVlYVJkyZh8uTJdQpLpufOb78hOSwMN06cQNdJk/DcmjWQOziIHYuoWjJLSzTv1AnNO3UCAPyyYsVjVzh7jhnT2PHIDDzcC5KWllbr+1Zb3mlpaRgwYAAAwM/PDxkZGeW2e3l5IT8/HxYWFhAEARKJBM8//zwGDx4MALh69SpaPlhBnJGRgezsbKSkpMDV1RXh4eGwr2aBCJkmQRCQHhODfXPnwsLKCqO3boXX2LFixyKqs+pWOBM1pmrLu6CgoFzBymQyaLVaWDx4r8bT0xOBgYGwsbFBQEAAFArF/W9sYYH58+djz549WLt2LQCge/fueOWVV9C1a1ds2LAB69atw/z58yv9TJVK1SDDiamkpMTk56jrDKV37uDMwoW4sW8fWvbrhx7Ll0Pfpo1o/z+a8u/C2JjyHBJ/f3SNisK5f/0Lxdevw6ZtW3jNnQuJv79JzmTKv4tHmcsctSZUY/ny5UJycrLh6wEDBhguq1QqYdiwYYJarRa0Wq0wd+5cYceOHeXu/+effwqDBw8WCgsLhby8PMP158+fFyZMmFDp5504caK6SCYhKytL7Aj1VpcZ/khOFta1aSOsksuF46tWCXqd7gkkq52m+rswRpzDeJjDDIJgHnPUpfeqPTGJv78/Dh48CAA4deoUOnfubNjm4OAAa2trWFlZQSaTwcnJCWq1Gt999x02btwIALCxsYFEIoFMJsOUKVOQnp4OADh69Ch8qzggA5mmsqIi7H3rLWwbNQq2rVrh1RMn0GvuXEikPPcNEVFDq3a3eUBAAA4fPozg4GAIgoDly5cjKSkJRUVFCAoKQlBQEEJDQ2FpaQkXFxeMGTMGWq0W7733HsLCwqDVahEeHg4rKytERUVhyZIlsLS0RMuWLbFkyZLGmJGesBu//orksDDcUanQc+5cDFi+3OQP+k9EZMyqLW+pVIrFixeXu87Dw8NwOSQkBCEhIeW2y+VyrFmzptL38vX1RWJiYl2zkpHR63Q4/vHHOLxwIWxbtcLY3bvhFhAgdiwiIrPHg7RQnahzc7FjwgRcPnAAnoGBGLpxI2xatBA7FhFRk8DyplpTbd6MvTNmQK/TYXhsLHxfe43HJSciakQsb6qxknv3kPLWW1DFx6Ndv34YGRcHx0feQiEiosbB8qYauXTwIHaMH4+CK1fwt6go9I2IEOW4zERExPKmKjx62sODHTqgVY8euPDDD3B0d79/XPK+fcWOSETUpLG8qZys+Phyh4DMz81Ffm4u2g8ahJeTknhcciIiI8AjaFA5qRERVZ6iTn3xIoubiMhIsLypHHVubq2uJyKixsfyJgBA4Z9/YuekSYAgVLld4eLSyImIiOhxWN5NnF6nw6/r1+NLLy+oNm2C+wsvwMLGptxteNpDIiLjwvJuwq798gvi+/RBysyZaP3003gtPR0vJyVhaEwMFK6ugEQChasrhiqV8AkLEzsuERE9wNXmTVDx7ds4FB6O9JgY2LVti1EJCegSHGw4SppPWBh8wsKgUqng7e0tcloiIqqI5d2ECHo9znz5JQ4tWICSe/fQc84c/C0qClYKhdjRiIioFljeTcSNX3/F3jffxLWff8ZT/fvj+XXr0Kp7d7FjERFRHbC8zVzJvXs4vHAhTq1fD5uWLTHi66/hM348TyRCRGTCWN5mShAEZMXF4cC8eSi+dQs9ZsxA/6VLYe3oKHY0IiKqJ5a3Gbp55gz2vvkmrqSmwrlPHwTu3Ik2/v5ixyIiogbC8jYjpWo1jkRF4eTatbB2dMTQmBh0mzwZEik/EUhEZE5Y3mZAEASc27IF+/75TxRev47ur7+OAdHRsGnRQuxoRET0BLC8TdxtlQopb72F3J9+Qht/f7z03//CuU8fsWMREdETxPI2UZrCQvy8ZAlOrFoFSzs7DFm3Dj2mT4dUJhM7GhERPWEsbxMjCALOb9uGfXPnIv/SJfhOnIiBH34Iu9atxY5GRESNhOVtQu6eP4+Ut9/GxV270Kp7d4xKSED7/v3FjkVERI2M5W0CyoqLcSw6Gsc//BAyKys8u3o1np45E1IL/vqIiJoi/utv5P5ISkLKrFlQX7wI79BQDPr4Y9g7O4sdi4iIRMTyNlL3srOxb/Zs/JGUBCdvb4z76Se4PPus2LGIiMgIsLyNjLakBMc/+gjHli+HRCbDwJUr0XP2bMjkcrGjERGRkWB5G5HsXbuQ8tZbuPf77+g8diwGr1oFRYcOYsciIiIjw/I2AupLl7Bv7lyc//ZbNPf0xNhdu+A2dKjYsYiIyEixvEWk02hw4l//wtHFiwFBQP+lS9Hr3XdhYWUldjQiIjJiLG+R5P70E/a+9RbuqFTwePFFPLdmDZq5uYkdi4iITADLu5EVXL2K/e+8g7OJiWjWsSPGJCXB44UXxI5FREQmhOXdSHRlZfj1009xZNEi6DQa9Hv/ffResACWNjZiRyMiIhPD8m4Elw8dwt6ZM3HrzBm4DR+OIZ98guadOokdi4iITBTLu4FlxccjNSIC6txcHHjqKSjc3HA1NRUOHTrgH9u2odNLL0EikYgdk4iITBjLuwFlxcdj97Rp0BYVAQAKLl9GweXL8HjxRYxKSIDczk7khEREZA6kYgcwJ6kREYbiftTN06dZ3ERE1GCqfeWt1+sRFRWFc+fOQS6XY+nSpXB1dTVs3759O2JjYyGVShEYGIjQ0FDodDpERkYiOzsbMpkM0dHRcHFxQU5ODhYsWACJRAJPT08sWrQIUqn5PH9Q5+bW6noiIqK6qLY59+7dC41Ggy1btuCdd97BihUrym1fuXIlYmNjsXnzZsTGxiIvLw/79u0DACQmJmLWrFmIjo4GAERHR2POnDlISEiAIAhISUl5AiOJx+ExhzJVuLg0chIiIjJn1ZZ3WloaBgwYAADw8/NDRkZGue1eXl7Iz8+HRqOBIAiQSCR4/vnnsWTJEgDA1atX0bJlSwBAZmYmevfuDQAYOHAgjhw50qDDiM13woRK11nY2qL/smUipCEiInNV7W7zgoIC2NvbG76WyWTQarWwsLh/V09PTwQGBsLGxgYBAQFQKBT3v7GFBebPn489e/Zg7dq1AGAodwCws7NDfn5+lT9TpVLVbyqR5J46BYmVFayaN0fJjRuwadsWXnPnQuLvb5IzlZSUmGTuisxhDnOYAeAcxsQcZgDMZ47aqra87e3tUVhYaPhar9cbivvs2bPYv38/UlJSYGtri3nz5mHnzp0YMWIEAODDDz/Eu+++i3HjxiE5Obnc+9uFhYWGoq/I29u7XkOJofj2bfy4Zw/8pk7FkE8+gUqlMsk5HmUOMwDmMYc5zABwDmNiDjMA5jFHWlpare9T7W5zf39/HDx4EABw6tQpdO7c2bDNwcEB1tbWsLKygkwmg5OTE9RqNb777jts3LgRAGBjYwOJRAKZTAYfHx8cO3YMAHDw4EH06tWr1oGNVVZcHHSlpeg+bZrYUYiIyMxV+8o7ICAAhw8fRnBwMARBwPLly5GUlISioiIEBQUhKCgIoaGhsLS0hIuLC8aMGQOtVov33nsPYWFh0Gq1CA8Ph5WVFebPn4+FCxdi1apVcHd3x7BhwxpjxidOEAScVirh3LcvWnXrJnYcIiIyc9WWt1QqxeLFi8td5+HhYbgcEhKCkJCQctvlcjnWrFlT6Xt17NgRmzZtqmtWo3UlNRV3VCoM+/JLsaMQEVETYD4fshZRulIJuUIBr3HjxI5CRERNAMu7norv3MG5rVvh8+qrPIoaERE1CpZ3PXGhGhERNTaWdz0IgoB0pRJte/dG6x49xI5DRERNBMu7Hq4eOYLbWVnowVfdRETUiFje9XBaqYTcwQFeQUFiRyEioiaE5V1HJXfv4rd//xveYWGQP3L4WCIioieN5V1HWZs2QVtSgh7Tp4sdhYiImhiWdx0YFqo98wxa+/mJHYeIiJoYlncdXPv5Z9zKyODHw4iISBQs7zo4rVTC0t4eXYKDxY5CRERNEMu7lkru3cO5LVvgw4VqREQkEpZ3LWVt2gRtcTF3mRMRkWhY3rXwcKFam5490cbfX+w4RETURLG8a+HasWO4deYMX3UTEZGoWN61kK5UwtLODt4Vzl9ORETUmFjeNVSal4eziYnwDg2F3MFB7DhERNSEsbxrKCs+ngvViIjIKLC8a0AQBKRv3IjWTz+NNj17ih2HiIiaOJZ3DVw/fhw309PRY/p0SCQSseMQEVETx/KugYcL1bpwoRoRERkBlnc1StVqqDZvRpeQEFgpFGLHISIiYnlXRxUfD21REReqERGR0WB5/wVBEHB640a09vND2169xI5DREQEgOX9l66fOIGbp0+j+7RpXKhGRERGg+X9F9KVSljY2sI7NFTsKERERAYs78coVatxdvNmdAkOhlWzZmLHISIiMmB5P8bZzZtRVliIHlyoRkRERobl/RjpSiVade+Otr17ix2FiIioHJZ3Fa6npeHGyZNcqEZEREaJ5V2FdKUSFjY28A4LEzsKERFRJSzvCjT5+VAlJMArKAjWjo5ixyEiIqqE5V3B2cRElBUUoMf06WJHISIiqhLLu4LTSiVadusG5z59xI5CRERUJZb3I26cPIkbJ05woRoRERk1lvcj0pVKWFhbw+fVV8WOQkRE9Fgs7wc0BQXIio/nQjUiIjJ6FtXdQK/XIyoqCufOnYNcLsfSpUvh6upq2L59+3bExsZCKpUiMDAQoaGhKCsrQ3h4OK5cuQKNRoMZM2ZgyJAhyMzMxBtvvAE3NzcAQEhICEaOHPnEhquNhwvVeOpPIiIydtWW9969e6HRaLBlyxacOnUKK1aswIYNGwzbV65ciR9++AG2trYYNWoURo0ahb1798LR0REfffQR7t69izFjxmDIkCHIysrCpEmTMHny5Cc6VF2kK5Vo4euLdv36iR2FiIjoL1Vb3mlpaRgwYAAAwM/PDxkZGeW2e3l5IT8/HxYWFhAEARKJBMOHD8ewYcMMt5HJZACAjIwMZGdnIyUlBa6urggPD4e9vX1DzlMnN379FdePH8dza9ZwoRoRERm9asu7oKCgXMHKZDJotVpYWNy/q6enJwIDA2FjY4OAgAAoFIpy9501axbmzJkDAOjevTteeeUVdO3aFRs2bMC6deswf/78Sj9TpVLVe7DaOLNyJaRWVpA980yD/eySkpJGn6OhmcMMgHnMYQ4zAJzDmJjDDID5zFFb1Za3vb09CgsLDV/r9XpDcZ89exb79+9HSkoKbG1tMW/ePOzcuRMjRozAtWvXMHPmTISGhmL06NEAUK7cAwICsGTJkip/pre3d70HqylNYSH2JCejy7hx6NGAu8xVKlWjzvEkmMMMgHnMYQ4zAJzDmJjDDIB5zJGWllbr+1S72tzf3x8HDx4EAJw6dQqdO3c2bHNwcIC1tTWsrKwgk8ng5OQEtVqNW7duYfLkyZg3bx7Gjh1ruP2UKVOQnp4OADh69Ch8fX1rHbihnduyBZr8fC5UIyIik1HtK++AgAAcPnwYwcHBEAQBy5cvR1JSEoqKihAUFISgoCCEhobC0tISLi4uGDNmDFauXAm1Wo3169dj/fr1AICYmBhERUVhyZIlsLS0RMuWLR/7yrsxpSuVcPL2xlN//7vYUYiIiGqk2vKWSqVYvHhxues8PDwMl0NCQhASElJue2RkJCIjIyt9L19fXyQmJtY1a4P78/RpXDt2DM+uXs2FakREZDKa9EFa0mNiILOygs/48WJHISIiqrEmW96awkJkxcXB65VXYOPkJHYcIiKiGmuy5X3u3/+GRq3mQjUiIjI5Tba805VKOHXpgqf69xc7ChERUa00yfK+mZ6Oaz//zFN/EhGRSWqS5Z0eEwOZXA7fCRPEjkJERFRrTa68y4qKkBUXh85jx8KmRQux4xAREdVakyvvc1u3ojQvjwvViIjIZDW58k5XKtG8c2e0HzhQ7ChERER10qTK+2ZGBq4eOcKFakREZNKaVHmfebhQ7bXXxI5CRERUZ02mvMuKi5H5zTfwDAyEbcuWYschIiKqsyZT3r9t3YrSe/e4UI2IiExekynvdKUSzT090WHQILGjEBER1UuTKO9bmZm4cvgwF6oREZFZaBLlnR4TA6mlJReqERGRWTD78i4rLkbWN9/A8+WXYduqldhxiIiI6s3sy/v8t9+i5O5d9OBCNSIiMhNmX97pSiUcO3VCh8GDxY5CRETUIMy6vG+rVLh86BC6T50KidSsRyUioibErBvNsFBt4kSxoxARETUYsy1vbUkJMr/+Gp1eegl2rVuLHYeIiKjBmG15n9+2DSV37nChGhERmR2zLe/TSiUcPTzg8txzYkchIiJqUGZZ3vD9HY8AAAwDSURBVLfPnsXlAwfQjQvViIjIDJlls6XHxEBqYYGuXKhGRERmyOzKW1tSgqyHC9XatBE7DhERUYMzu/I+/9//ovj2bZ76k4iIzJbZlXe6UolmHTvCdcgQsaMQERE9EWZV3nd++w2X9u/nEdWIiMismVXDGRaqTZokdhQiIqInxmzKW1taisyvvoLHiy/Crm1bseMQERE9MWZT3r9/9x2Kb93iQjUiIjJ7ZlPe6UolFK6ucAsIEDsKERHRE2UW5X33/Hnk/vQTF6oREVGTYBZNlx4TA4lMhq6TJ4sdhYiI6Ikz+fLWlpYiIzYWnV58EfbOzmLHISIieuIsqruBXq9HVFQUzp07B7lcjqVLl8LV1dWwffv27YiNjYVUKkVgYCBCQ0NRVlaG8PBwXLlyBRqNBjNmzMCQIUOQk5ODBQsWQCKRwNPTE4sWLYK0nru5f//+ey5UIyKiJqXa5ty7dy80Gg22bNmCd955BytWrCi3feXKlYiNjcXmzZsRGxuLvLw8bN++HY6OjkhISEBMTAyWLFkCAIiOjsacOXOQkJAAQRCQkpJS7wEeLlRz5UI1IiJqIqot77S0NAwYMAAA4Ofnh4yMjHLbvby8kJ+fD41GA0EQIJFIMHz4cMyePdtwG5lMBgDIzMxE7969AQADBw7EkSNH6hX+7u+/IzclBd1efx3SBz+DiIjI3FW727ygoAD29vaGr2UyGbRaLSws7t/V09MTgYGBsLGxQUBAABQKRbn7zpo1C3PmzAEAQ7kDgJ2dHfLz8+sV/sznn99fqMYjqhERURNSbXnb29ujsLDQ8LVerzcU99mzZ7F//36kpKTA1tYW8+bNw86dOzFixAhcu3YNM2fORGhoKEaPHg0A5d7fLiwsLFf0j1KpVNUG12s0OPX552g9aBAuq9WAWl3tfRpTSUlJjeYwZuYwA2Aec5jDDADnMCbmMANgPnPUVrXl7e/vj3379mHkyJE4deoUOnfubNjm4OAAa2trWFlZQSaTwcnJCWq1Grdu3cLkyZPx/vvvo1+/fobb+/j44NixY+jTpw8OHjyIvn37Vvkzvb29qw1+7j//geb2bfz9nXfgXoPbNzaVSlWjOYyZOcwAmMcc5jADwDmMiTnMAJjHHGlpabW+T7XlHRAQgMOHDyM4OBiCIGD58uVISkpCUVERgoKCEBQUhNDQUFhaWsLFxQVjxozBypUroVarsX79eqxfvx4AEBMTg/nz52PhwoVYtWoV3N3dMWzYsNpP+UC6UgmHDh3gVo/vQUREZIqqLW+pVIrFixeXu87Dw8NwOSQkBCEhIeW2R0ZGIjIystL36tixIzZt2lTXrAb3LlxAzp49+NsHH3ChGhERNTkmeZCWM59/DolUim48ohoRETVBJlfeurIynPnyS7iPGgWH9u3FjkNERNToTK68/9i+HUU3bqD79OliRyEiIhKFyZV3ulIJh/bt0XH4cLGjEBERicKkyvtedjYu7t7NI6oREVGTZlLl/XChGk/9SURETZnJlLeurAwZX36JjiNHQtGhg9hxiIiIRGMy5X3hhx9QeP06evDUn0RE1MSZTHmfViph/9RT6DhihNhRiIiIRGUS5Z138SIu7tqFblOmQGpR7UHhiIiIzJpJlPeZL74AAHSbMkXkJEREROIz+vLWa7U488UX6DhiBBQuLmLHISIiEp3Rl/eF5GQUXrvGhWpEREQPGH15n964Efbt2sF91CixoxARERkFoy7vvJwcZP/4I7pyoRoREZGBUZc3F6oRERFVZrTlrddqkfHFF+g4fDiaubqKHYeIiMhoGG15X9ixAwVXr6I7F6oRERGVY7Tlna5Uws7ZmQvViIiIKjDK8lbn5iJ75050mzwZMktLseMQEREZFaMs7zNffglBENCVC9WIiIgqMcryzvjiC7gNHQrHjh3FjkJERGR0jLK88y9f5kI1IiKixzDK8pZIpdAUFIgdg4iIyCgZZXkLej32zpiBrPh4saMQEREZHaMsbwDQFhUhNSJC7BhERERGx2jLG7j/kTEiIiIqz6jLm+fvJiIiqsxoy9vC1hb9ly0TOwYREZHRMcryVri6YqhSCZ+wMLGjEBERGR2jPEn2tIsXxY5ARERktIzylTcRERE9HsubiIjIxLC8iYiITAzLm4iIyMSwvImIiEyMRBAEQewQj0pLSxM7AhERUaPq2bNnrW5vdOVNREREf427zYmIiEwMy5uIiMjEGEV5l5WVYd68eQgNDcXYsWORkpIidqR6uX37NgYNGoQ//vhD7Ch1tnHjRgQFBeHll1/G1q1bxY5Ta2VlZXjnnXcQHByM0NBQk/xdnD59GuPHjwcA5OTkICQkBKGhoVi0aBH0er3I6Wrm0RlUKhVCQ0Mxfvx4TJkyBbdu3RI5Xc09OsdDSUlJCAoKEilR3Tw6x+3btzFjxgyEhYUhODgYuSZyFseKf6fGjRuHkJAQvPfeeybxuKiq7+ry+DaK8t6+fTscHR2RkJCAmJgYLFmyROxIdVZWVob3338f1tbWYkeps2PHjuHXX3/F5s2bERcXh+vXr4sdqdYOHDgArVaLxMREzJw5E6tXrxY7Uq3ExMQgMjISpaWlAIDo6GjMmTMHCQkJEATBJJ7gVpxh2bJlWLhwIeLi4hAQEICYmBiRE9ZMxTmA+6Xxn//8B6a0ZKjiHB999BFGjx6N+Ph4zJkzBxcuXBA5YfUqzvDpp59i5syZ2Lx5MzQaDfbv3y9uwBqoqu/q8vg2ivIePnw4Zs+ebfhaJpOJmKZ+PvzwQwQHB6N169ZiR6mz1NRUdO7cGTNnzsQbb7yBwYMHix2p1jp27AidTge9Xo+CggJYWBjlYfwfy8XFBZ988onh68zMTPTu3RsAMHDgQBw5ckSsaDVWcYZVq1bB29sbAKDT6WBlZSVWtFqpOMfdu3fx8ccfIzw8XMRUtVdxjpMnT+LGjRuYOHEikpKSDH+/jFnFGby9vXHv3j0IgoDCwkKTeJxX1Xd1eXwbRXnb2dnB3t4eBQUFmDVrFubMmSN2pDrZtm0bnJycMGDAALGj1Mvdu3eRkZGBNWvW4IMPPsC7775rUq8wAMDW1hZXrlzBiBEjsHDhwkq7PI3dsGHDyv1DJAgCJBIJgPuPl/z8fLGi1VjFGR4+oT158iQ2bdqEiRMnipSsdh6dQ6fTISIiAuHh4bCzsxM5We1U/H1cuXIFCoUCX331FZydnU1iT0jFGdzc3LBs2TKMGDECt2/fRp8+fURMVzNV9V1dHt9GUd4AcO3aNUyYMAH/+Mc/MHr0aLHj1Mm3336LI0eOYPz48VCpVJg/fz5u3rwpdqxac3R0RP/+/SGXy+Hu7g4rKyvcuXNH7Fi18tVXX6F///7YtWsXvv/+eyxYsKDcbk9TI5X+76FaWFgIhUIhYpq627FjBxYtWgSlUgknJyex49RaZmYmcnJyEBUVhX/+85/4/fffsWzZMrFj1YmjoyOee+45AMBzzz2HjIwMkRPV3rJlyxAfH48ff/wRL730ElasWCF2pBqp2Hd1eXwbRXnfunULkydPxrx58zB27Fix49RZfHw8Nm3ahLi4OHh7e+PDDz9Eq1atxI5Vaz179sShQ4cgCAJu3LiB4uJiODo6ih2rVhQKBRwcHAAAzZo1g1arhU6nEzlV3fn4+ODYsWMAgIMHD6JXr14iJ6q977//3vD46NChg9hx6qR79+5ITk5GXFwcVq1ahU6dOiEiIkLsWHXSs2dPHDhwAABw/PhxdOrUSeREtdesWTPY29sDuL9nR61Wi5yoelX1XV0e30bxBsFnn30GtVqN9evXY/369QDuL0ww5UVfpuzZZ5/F8ePHMXbsWAiCgPfff9/k1iFMnDgR4eHhCA0NRVlZGebOnQtbW1uxY9XZ/PnzsXDhQqxatQru7u4YNmyY2JFqRafTYdmyZXB2dsbbb78NAHjmmWcwa9YskZM1XfPnz0dkZCQSExNhb2+P//u//xM7Uq0tXboUc+fOhYWFBSwtLU1isXNVfRcREYGlS5fW6vHNI6wRERGZGKPYbU5EREQ1x/ImIiIyMSxvIiIiE8PyJiIiMjEsbyIiIhPD8iYiIjIxLG8iIiITw/ImIiIyMf8fQFBQeUOFfDsAAAAASUVORK5CYII=\n", 84 | "text/plain": [ 85 | "
" 86 | ] 87 | }, 88 | "metadata": {}, 89 | "output_type": "display_data" 90 | } 91 | ], 92 | "source": [ 93 | "results = []\n", 94 | "max_depth_options = [2,4,6,8,10,12,14,16,18,20]\n", 95 | "for trees in max_depth_options:\n", 96 | " model = DecisionTreeClassifier(max_depth=trees, random_state=101)\n", 97 | " model.fit(x_train, y_train)\n", 98 | " y_pred = model.predict(x_test)\n", 99 | " accuracy = np.mean(y_test==y_pred)\n", 100 | " results.append(accuracy)\n", 101 | "\n", 102 | "plt.figure(figsize=(8,4))\n", 103 | "pd.Series(results, max_depth_options).plot(color=\"darkred\",marker=\"o\")" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 5, 109 | "metadata": {}, 110 | "outputs": [ 111 | { 112 | "data": { 113 | "text/plain": [ 114 | "" 115 | ] 116 | }, 117 | "execution_count": 5, 118 | "metadata": {}, 119 | "output_type": "execute_result" 120 | }, 121 | { 122 | "data": { 123 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAEGCAYAAABB3G3AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3dfXAU933H8ffpTg+gkwwqJHHLWE8jESFC1IPYrluR2vSKi7BjEHo4EdNBOC4ZU48qxRYoPMiSKqn4gT4E5MRjY1sYo0JijJwh05GorFamNHPxmUgcalxjgh2Hwii1njBC3PUP1xcrQj4jnbjV6vP663Z/+7v9fmcZPre7dyuL3+/3IyIiIoYTEe4CRERE5NoU0iIiIgalkBYRETEohbSIiIhBKaRFREQMSiEtIiJiULZgG/h8PiorK+nu7iYqKoqamhoSExMD44cPH+bZZ58lLi6OVatWkZeXN+acs2fPsnnzZiwWC2lpaezYsYOICH1OEBERuZagCdnS0sLQ0BBNTU2UlZVRX18fGOvp6eEf/uEfaGxsZN++fTQ3N/Pee++NOaeuro6SkhL279+P3++ntbV18joTERGZ4oKGtNvtJjs7G4CsrCw6OzsDY++99x5f/vKXmTVrFhEREXzlK1/hrbfeGnNOV1cXt956KwBLly7ljTfeCHlDIiIiZhH0cnd/fz92uz2wbLVaGR4exmazkZiYyNtvv83FixeJjY3l+PHjJCUljTnH7/djsVgAiI2Npa+vb9T+3G53KPoSERGZMhYvXnzN9UFD2m63MzAwEFj2+XzYbB9Pu+mmm9iyZQt//dd/zZe+9CUyMzOZPXv2mHM+ff95YGCA+Pj46yp2Mni9XjIyMm7Y/m40M/dn5t5A/U116m/qutG9fdbJadDL3Q6Hg/b2dgA8Hg/p6emBseHhYd566y1eeukl/u7v/o533nkHh8Mx5pwFCxZw4sQJANrb21myZMn4uxIRETG5oGfSTqeTjo4OCgsL8fv91NbW0tzczODgIAUFBURGRrJ69Wqio6NZv349CQkJ15wDUF5ezrZt23jqqadISUlh+fLlk96giIjIVBU0pCMiIqiqqhqxLjU1NfB606ZNbNq0KegcgOTkZPbt2zfeWkVERKYV/UhZRETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMShbsA18Ph+VlZV0d3cTFRVFTU0NiYmJgfEjR46wd+9eIiIiyM3NpaioiB/96Ee88sorAFy+fBmv10tHRwfnzp1j48aNJCUlAeByuVixYsXkdCYiIjLFBQ3plpYWhoaGaGpqwuPxUF9fT0NDQ2B8586dvPbaa8ycOZOcnBxycnJYvXo1q1evBuCxxx4jNzeX+Ph4Tp06xfr16ykuLp68jkREREwi6OVut9tNdnY2AFlZWXR2do4Ynz9/Pn19fQwNDeH3+7FYLIGxn//857z99tsUFBQA0NnZSVtbG2vXrqWiooL+/v5Q9iIiImIqQc+k+/v7sdvtgWWr1crw8DA228dT09LSyM3NZcaMGTidTuLj4wPbfv/73+ehhx4KLC9atIi8vDwWLlxIQ0MDu3fvpry8fNQ+vV7vhJq6Hh999NEN3d+NZub+zNwbqL+pTv1NXUbqLWhI2+12BgYGAss+ny8Q0KdPn6atrY3W1lZmzpzJI488wtGjR/mLv/gLent7eeedd7j99tsDcz8d4k6nk+rq6mvuMyMjY0JNXQ+v13tD93ejmbk/M/cG6m+qU39T143uze12jzkW9HK3w+Ggvb0dAI/HQ3p6emAsLi6OmJgYoqOjsVqtJCQk0NvbC8BPf/pT7rjjjhHvtWHDBk6ePAnA8ePHyczMvP5uREREpomgZ9JOp5OOjg4KCwvx+/3U1tbS3NzM4OAgBQUFFBQUUFRURGRkJLfccgurVq0C4MyZM8ybN2/Ee1VWVlJdXU1kZCRz5swZ80xaREREPkdIR0REUFVVNWJdampq4LXL5cLlco2a98ADD4xal5mZyYEDB8ZTp4iIyLSjh5mIiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMShbsA18Ph+VlZV0d3cTFRVFTU0NiYmJgfEjR46wd+9eIiIiyM3NpaioCID77ruPuLg4AObNm0ddXR1nz55l8+bNWCwW0tLS2LFjBxER+pwgIiJyLUFDuqWlhaGhIZqamvB4PNTX19PQ0BAY37lzJ6+99hozZ84kJyeHnJwcYmJiAGhsbBzxXnV1dZSUlHDbbbexfft2WltbcTqdIW5JRETEHIKGtNvtJjs7G4CsrCw6OztHjM+fP5++vj5sNht+vx+LxcLp06e5dOkSxcXFDA8PU1paSlZWFl1dXdx6660ALF26lI6OjmuGtNfrDUVvn8tHH310Q/d3o5m5PzP3BupvqlN/U5eRegsa0v39/djt9sCy1WpleHgYm+3jqWlpaeTm5jJjxgycTifx8fHExMSwYcMG8vLyePfdd/nWt77FT37yk0CIA8TGxtLX13fNfWZkZISit8/F6/Xe0P3daGbuz8y9gfqb6tTf1HWje3O73WOOBb0hbLfbGRgYCCz7fL5AQJ8+fZq2tjZaW1s5duwYPT09HD16lOTkZO69914sFgvJycnMmjWLCxcujLj/PDAwQHx8/ET6EhERMbWgIe1wOGhvbwfA4/GQnp4eGIuLiyMmJobo6GisVisJCQn09vZy6NAh6uvrATh//jz9/f3MnTuXBQsWcOLECQDa29tZsmTJZPQkIiJiCkEvdzudTjo6OigsLMTv91NbW0tzczODg4MUFBRQUFBAUVERkZGR3HLLLaxatQqALVu24HK5sFgs1NbWYrPZKC8vZ9u2bTz11FOkpKSwfPnySW9QRERkqgoa0hEREVRVVY1Yl5qaGnjtcrlwuVyj5j355JOj1iUnJ7Nv377x1CkiIjLt6EfKIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYVNDfSYsxPPH/zzwfjx+PY853/P5x709G0rETkfFSSIuIfAZ9yJJw0uVuERERg9KZtIhMiM40RSaPzqRFREQMSmfSIiLTmK6EGJvOpEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBhX0J1g+n4/Kykq6u7uJioqipqaGxMTEwPiRI0fYu3cvERER5ObmUlRUxJUrV6ioqOD9999naGiIb3/72yxbtoyuri42btxIUlISAC6XixUrVoSkEf2MYGob7/HTsRMRMwsa0i0tLQwNDdHU1ITH46G+vp6GhobA+M6dO3nttdeYOXMmOTk55OTk0NLSwqxZs3j88cf5zW9+w6pVq1i2bBmnTp1i/fr1FBcXT2pTIiIiZhA0pN1uN9nZ2QBkZWXR2dk5Ynz+/Pn09fVhs9nw+/1YLBbuvvtuli9fHtjGarUC0NnZyZkzZ2htbSUxMZGKigrsdvuofXq93gk1dSNMhRonwsz9mbk3UH9TnfoLv48++sgwdQYN6f7+/hFBarVaGR4exmb7eGpaWhq5ubnMmDEDp9NJfHz8iLkPP/wwJSUlACxatIi8vDwWLlxIQ0MDu3fvpry8fNQ+MzIyrruR8Vz2nIjx1DgR6i90zNwbqL9QU3+hdaP7Gw+v13tD63S73WOOBf3imN1uZ2BgILDs8/kCAX369Gna2tpobW3l2LFj9PT0cPToUQA++OAD1q1bxze+8Q3uueceAJxOJwsXLgy8PnXq1Pi7EhERMbmgIe1wOGhvbwfA4/GQnp4eGIuLiyMmJobo6GisVisJCQn09vZy8eJFiouLeeSRR1izZk1g+w0bNnDy5EkAjh8/TmZmZqj7ERERMY2gl7udTicdHR0UFhbi9/upra2lubmZwcFBCgoKKCgooKioiMjISG655RZWrVrFzp076e3tZc+ePezZsweAZ555hsrKSqqrq4mMjGTOnDlUV1dPeoMiIiJTVdCQjoiIoKqqasS61NTUwGuXy4XL5RoxvnXrVrZu3TrqvTIzMzlw4MB4axUREZlW9DATERERg1JIi4iIGJRCWkRExKAU0iIiIgalkBYRETEohbSIiIhBKaRFREQMSiEtIiJiUAppERERg1JIi4iIGJRCWkRExKCCPrtbRERkqnrCYhnXvPH8ne3v+P3j2tdn0Zm0iIiIQSmkRUREDEohLSIiYlAKaREREYNSSIuIiBiUQlpERMSgFNIiIiIGFfR30j6fj8rKSrq7u4mKiqKmpobExMTA+JEjR9i7dy8RERHk5uZSVFQ05pyzZ8+yefNmLBYLaWlp7Nixg4gIfU4QERG5lqAJ2dLSwtDQEE1NTZSVlVFfXz9ifOfOnezdu5eXX36ZvXv38uGHH445p66ujpKSEvbv34/f76e1tXVyuhIRETGBoCHtdrvJzs4GICsri87OzhHj8+fPp6+vj6GhIfx+PxaLZcw5XV1d3HrrrQAsXbqUN954I6TNiIiImEnQy939/f3Y7fbAstVqZXh4GJvt46lpaWnk5uYyY8YMnE4n8fHxY875JMQBYmNj6evru+Y+vV7vhJq6EaZCjRNh5v7M3Buov6lO/U1dk9Fb0JC22+0MDAwEln0+XyCgT58+TVtbG62trcycOZNHHnmEo0ePjjnn0/efBwYGiI+Pv+Y+MzIyrruR8TxndSLGU+NEqL/QMXNvoP5CTf2Flpn7G29vbrd7zLGgl7sdDgft7e0AeDwe0tPTA2NxcXHExMQQHR2N1WolISGB3t7eMecsWLCAEydOANDe3s6SJUvG1ZCIiMh0EPRM2ul00tHRQWFhIX6/n9raWpqbmxkcHKSgoICCggKKioqIjIzklltuYdWqVdhstlFzAMrLy9m2bRtPPfUUKSkpLF++fNIbFBERmaqChnRERARVVVUj1qWmpgZeu1wuXC7XqHm/OwcgOTmZffv2jadOERGRaUc/UhYRETEohbSIiIhBKaRFREQMSiEtIiJiUAppERERg1JIi4iIGJRCWkRExKAU0iIiIgalkBYRETEohbSIiIhBKaRFREQMSiEtIiJiUAppERERg1JIi4iIGJRCWkRExKAU0iIiIgalkBYRETEohbSIiIhBKaRFREQMyhZsA5/PR2VlJd3d3URFRVFTU0NiYiIAFy5coLS0NLCt1+ulrKyM6OhoXnnlFQAuX76M1+ulo6ODc+fOsXHjRpKSkgBwuVysWLFiEtoSERGZ+oKGdEtLC0NDQzQ1NeHxeKivr6ehoQGAuXPn0tjYCMCbb77Jrl27yM/Px2q1snr1agAee+wxcnNziY+P59SpU6xfv57i4uJJbElERMQcgl7udrvdZGdnA5CVlUVnZ+eobfx+P9XV1VRWVmK1WgPrf/7zn/P2229TUFAAQGdnJ21tbaxdu5aKigr6+/tD1YeIiIjpBD2T7u/vx263B5atVivDw8PYbL+deuzYMdLS0khJSRkx9/vf/z4PPfRQYHnRokXk5eWxcOFCGhoa2L17N+Xl5aP26fV6x9XMjTQVapwIM/dn5t5A/U116m/qmozegoa03W5nYGAgsOzz+UYENMCRI0dYt27diHW9vb2888473H777YF1TqeT+Pj4wOvq6upr7jMjI+Pzd/D/fnzdMyZmPDVOhPoLHTP3Buov1NRfaJm5v/H25na7xxwLernb4XDQ3t4OgMfjIT09fdQ2XV1dOByOEet++tOfcscdd4xYt2HDBk6ePAnA8ePHyczMDF69iIjINBX0TNrpdNLR0UFhYSF+v5/a2lqam5sZHBykoKCAnp4eYmNjsVgsI+adOXOGefPmjVhXWVlJdXU1kZGRzJkzZ8wzaREREfkcIR0REUFVVdWIdampqYHXCQkJvPrqq6PmPfDAA6PWZWZmcuDAgfHUKSIiMu3oYSYiIiIGpZAWERExKIW0iIiIQSmkRUREDEohLSIiYlAKaREREYNSSIuIiBiUQlpERMSgFNIiIiIGpZAWERExKIW0iIiIQSmkRUREDEohLSIiYlAKaREREYNSSIuIiBiUQlpERMSgFNIiIiIGpZAWERExKIW0iIiIQdmCbeDz+aisrKS7u5uoqChqampITEwE4MKFC5SWlga29Xq9lJWV4XK5uO+++4iLiwNg3rx51NXVcfbsWTZv3ozFYiEtLY0dO3YQEaHPCSIiItcSNKRbWloYGhqiqakJj8dDfX09DQ0NAMydO5fGxkYA3nzzTXbt2kV+fj6XL18GCIx9oq6ujpKSEm677Ta2b99Oa2srTqcz1D2JiIiYQtDTWLfbTXZ2NgBZWVl0dnaO2sbv91NdXU1lZSVWq5XTp09z6dIliouLWbduHR6PB4Curi5uvfVWAJYuXcobb7wRyl5ERERMJeiZdH9/P3a7PbBstVoZHh7GZvvt1GPHjpGWlkZKSgoAMTExbNiwgby8PN59912+9a1v8ZOf/AS/34/FYgEgNjaWvr6+a+7T6/VOqKkbYSrUOBFm7s/MvYH6m+rU39Q1Gb0FDWm73c7AwEBg2efzjQhogCNHjrBu3brAcnJyMomJiVgsFpKTk5k1axYXLlwYcf95YGCA+Pj4a+4zIyPjuhv58XXPmJjx1DgR6i90zNwbqL9QU3+hZeb+xtub2+0ecyzo5W6Hw0F7ezsAHo+H9PT0Udt0dXXhcDgCy4cOHaK+vh6A8+fP09/fz9y5c1mwYAEnTpwAoL29nSVLllxfJyIiItNI0JB2Op1ERUVRWFhIXV0dW7Zsobm5maamJgB6enqIjY0NXMYGWLNmDX19fbhcLv7mb/6G2tpabDYb5eXl/NM//RMFBQVcuXKF5cuXT15nIiIiU1zQy90RERFUVVWNWJeamhp4nZCQwKuvvjpiPCoqiieffHLUeyUnJ7Nv377x1ioiIjKt6EfKIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEHZgm3g8/morKyku7ubqKgoampqSExMBODChQuUlpYGtvV6vZSVlbFmzRoqKip4//33GRoa4tvf/jbLli2jq6uLjRs3kpSUBIDL5WLFihWT05mIiMgUFzSkW1paGBoaoqmpCY/HQ319PQ0NDQDMnTuXxsZGAN5880127dpFfn4+hw8fZtasWTz++OP85je/YdWqVSxbtoxTp06xfv16iouLJ7crEREREwga0m63m+zsbACysrLo7OwctY3f76e6uponnngCq9XK3XffzfLlywPjVqsVgM7OTs6cOUNrayuJiYlUVFRgt9tD1YuIiIipBA3p/v7+EUFqtVoZHh7GZvvt1GPHjpGWlkZKSgoAsbGxgbkPP/wwJSUlACxatIi8vDwWLlxIQ0MDu3fvpry8fNQ+vV7vxLq6AaZCjRNh5v7M3Buov6lO/U1dk9Fb0JC22+0MDAwEln0+34iABjhy5Ajr1q0bse6DDz7goYceoqioiHvuuQcAp9NJfHx84HV1dfU195mRkXF9XQA/vu4ZEzOeGidC/YWOmXsD9Rdq6i+0zNzfeHtzu91jjgX9drfD4aC9vR0Aj8dDenr6qG26urpwOByB5YsXL1JcXMwjjzzCmjVrAus3bNjAyZMnATh+/DiZmZmfvwsREZFpJuiZtNPppKOjg8LCQvx+P7W1tTQ3NzM4OEhBQQE9PT3ExsZisVgCc55++ml6e3vZs2cPe/bsAeCZZ56hsrKS6upqIiMjmTNnzphn0iIiIvI5QjoiIoKqqqoR61JTUwOvExISePXVV0eMb926la1bt456r8zMTA4cODDeWkVERKYVPcxERETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEEppEVERAxKIS0iImJQCmkRERGDUkiLiIgYlEJaRETEoBTSIiIiBqWQFhERMSiFtIiIiEHZgm3g8/morKyku7ubqKgoampqSExMBODChQuUlpYGtvV6vZSVlVFQUHDNOWfPnmXz5s1YLBbS0tLYsWMHERH6nCAiInItQROypaWFoaEhmpqaKCsro76+PjA2d+5cGhsbaWxspLS0lAULFpCfnz/mnLq6OkpKSti/fz9+v5/W1tbJ60xERGSKs/j9fv9nbVBXV8eiRYvIyckBIDs7m3/7t38bsY3f7yc3N5cnnniClJSUMedkZ2fT3t6OxWKhpaWFjo4OduzYMeK93G53KPsTERExvMWLF19zfdDL3f39/djt9sCy1WpleHgYm+23U48dO0ZaWhopKSmfOcfv92OxWACIjY2lr6/vcxcqIiIy3QS93G232xkYGAgs+3y+EQENcOTIEfLz84PO+fT954GBAeLj4ydUvIiIiJkFDWmHw0F7ezsAHo+H9PT0Udt0dXXhcDiCzlmwYAEnTpwAoL29nSVLlky8AxEREZMKek/6k293/9d//Rd+v5/a2lpOnTrF4OAgBQUF9PT0sH79el599dXPnJOamsqZM2fYtm0bV65cISUlhZqaGqxW66Q3KSIiMhUFDWkREREJj2n7I+Vf/OIXHD16FK/XG+5SREbw+XycP38en88X7lImhdn7MzsdvxtrWob0iy++yHe/+11+9rOfsW3bNp599tlwlxRSV69e5eDBg/zjP/4jJ06coKenJ9wlhUxVVdWI5UcffTRMlYRWRUUFAG+99RbLly9n06ZNrFy5Eo/HE+bKQsPs/b377ruB16+//joNDQ2B7+WYgdmPn5EF/QmWGf34xz9m//792Gw2rly5QmFhIRs2bAh3WSGzfft2vvCFL/DGG2+wcOFCysvLeeaZZ8Jd1oS89NJLNDQ08L//+7/8y7/8S2B9ampqGKsKnffeew+AXbt28cwzz5CUlMT58+cpKytj3759Ya5u4sze3/bt23nxxRf5wQ9+gNvt5utf/zqHDh3i5MmTbNq0KdzlTZiZj9+ZM2fGHEtOTr6BlVzbtAxpv98f+BlZZGQkkZGRYa4otH75y1/yt3/7t7jdbu666y5+8IMfhLukCVu7di1r167l6aefZuPGjeEuZ9JYrVaSkpIA+OIXv2i6S4pm76+trY0XX3wRm82Gy+Xim9/8pilC+hNmPH4VFRWcO3eOlJQUPv0VLYvFwosvvhjGyj42LUPa4XDw8MMPs3jxYtxuN3/4h38Y7pJC6urVq4FL3P39/aZ6Pvrrr79uypDu6+tj9erVDA4OcvDgQe69917q6+v5/d///XCXFhJm76+np4dTp04xd+5c+vv7mTVrFh999BGXL18Od2khMdbxu/nmm8Nd2oQ999xzfPOb3+Txxx/ni1/8YrjLGWXafru7ra2N//7v/yY1NZU//dM/DXc5IfWf//mfbNu2jQsXLnDzzTdTUVHBH//xH4e7rJDYuHEjf/RHf0RycnLgw8ef/MmfhLmq0BgaGuL06dPMmDGDxMREfvjDH7JmzRrTXOn5pL+YmBiSkpJM1d/zzz9PV1cXXV1d5OTk8Jd/+ZesXLmS0tJS7r333nCXFxLXOn55eXmjHm41FXV2dnLlyhVDnrBNq5C+evUqV69epbS0lF27duH3+/H5fDz44IOGuKwRaj09PcyePTvwKFYz2LJly6h1dXV1Yagk9H71q1/h8Xi4dOkSs2fPxuFwMGvWrHCXFRJNTU3k5+eb6t9iML/7eOSp7l//9V+Jjo7mjjvuCKxraWnhz/7sz8JYlflNq5D+53/+Z55++mkuXrzI3Llz8fv9WK1WFi9ePOKve011HR0dPP/88yMutZnxQ4iZHDp0iObmZr7yla9w/PhxMjMzOXPmDPfffz9//ud/Hu7yJuxrX/samZmZPPbYY4E/dTsdHDx4kLy8vHCXMWGVlZX09fUxPDzMpUuX+N73vkdUVBTr1q0z7f8tRjl2U/86xXXIz88nPz+fQ4cOsWbNmnCXM2nq6uqoqKjgS1/6UrhLCbnly5fzwQcf8Ad/8Af8+te/JjIykqioKAD+/d//PczVjd/hw4dpbGzEYrFw6dIltm7dynPPPWeakP7yl79MSUkJpaWlpKenk5+fb8hLi6Hg8/kCt2JmzJgR5mpCo7u7m5dffhmAxsZGSkpK2LNnD2Y7xzPisZtWIf2JX//613zve98bsc5M38C8+eabR1ySMpOFCxfy7LPPMm/ePC5evEhNTQ1///d/H+6yJqy3t5f+/n7i4uK4dOlS4AOIWb54ZLFYyMrK4oc//CHHjh3jhRde4NFHH8Vut/PKK6+Eu7wJO3fuHHV1dXR2dmKz2fD5fKSnp1/z9sxUdPXqVYaGhoiKiuL+++/nV7/6FTU1NeEuKySMfuymZUjPmTMH+PinWKdOnTLFzwg+7fd+7/fYvn07CxYsCNwDLCgoCHNVofH+++8zb9484OPjeP78+TBXFBrFxcV84xvfICMjg7fffpvNmzeza9culi1bFu7SQuLTZ1x33XUXd911F4BpHrTz3e9+l7KyMr761a8G1nk8HrZs2cKBAwfCWFlorFu3jpUrV3LgwAESEhJ49NFH2bZtG263O9ylTZjRj920DOnCwsIRyw888ECYKpkcn4TYxYsXw1xJ6CUnJ/Od73yHr371q/zsZz8jKysr3CWFxH333cfXv/51zp07R1JSEvHx8SxdutQ0f4Bm165d11yfkJBwgyuZHENDQyP+kwdM828TYOXKlTidzsCtJYvFQk1NDUVFRWGubOKMfuymZUh/+gkz//M//8MHH3wQxmpCb9OmTbS1tfGLX/yC5ORkU337sidKzecAAALwSURBVKSkhEOHDvHhhx9y8uRJHnzwwXCXFDKzZ89m9uzZgWWr1WqYL69M1CdXr36XWfqbP38+W7ZsITs7m7i4OAYGBnj99deZP39+uEsLmejo6FHrurq6WLBgQRiqCR2jH7tpGdLbt28PXAaOjo6mrKwszBWF1pNPPsnZs2dxOBwcPnwYt9tNeXl5uMsKic2bN/NXf/VX7N+/n9LSUmpra2lsbAx3WSFlxC+vhJIZ+6usrKSlpQW32x346dWdd96J0+kMd2khZ7bjZ/RjN61+gvWJl19+meeff54rV64AYLPZRjwPeqorLCwM3Evx+/3k5+dz8ODBMFcVGvfffz/PP/88DzzwAHv37mXt2rW89NJL4S5rwj7ryytGeH7wRJm9P7PT8QufaXkmffDgQRobG2loaODuu+/mhRdeCHdJITU8PBz4tOv3+031AIkrV65QV1fH4sWL+Y//+A+uXr0a7pJCwuhfXpkos/dndjp+4WOehzpfh9mzZ/OFL3yBgYEBbrvtNj788MNwlxRSOTk5uFwuamtrKSoqYsWKFeEuKWTq6+tJTk7mwQcfpKenh8cffzzcJYWE0b+8MlFm78/sdPzCZ1qeScfFxdHS0oLFYuHAgQOm+RnI4cOHgY8/hNxzzz1cvnyZlStXmurRhElJSYG/wmOmDx9G//LKRJm9P7PT8QufaXlPur+/n1/+8pfMmTOH5557jjvvvJPbbrst3GVN2JNPPjli2e/386Mf/YiYmBiOHTsWpqrk8/D7/aO+vOJwOHA6naa4XWH2/sxOxy98pmVITwdnz55l8+bNJCcnU1FRYaqzaRGR6WJaXu42u5deeokXXniBLVu2cOedd4a7HBERGSeFtImcP3+eLVu2cNNNN3Hw4EFuuummcJckIiIToMvdJvK1r32NyMhIbr/99lH3iX73frWIiBifzqRNZPfu3eEuQUREQkhn0iIiIgY1LR9mIiIiMhUopEVERAxKIS0iImJQCmkRERGD+j/oIrfcriHGfwAAAABJRU5ErkJggg==\n", 124 | "text/plain": [ 125 | "
" 126 | ] 127 | }, 128 | "metadata": {}, 129 | "output_type": "display_data" 130 | } 131 | ], 132 | "source": [ 133 | "results = []\n", 134 | "max_features_options = ['auto',None,'sqrt',0.95,0.75,0.5,0.25,0.10]\n", 135 | "for trees in max_features_options:\n", 136 | " model = DecisionTreeClassifier(max_depth=10, random_state=101, max_features = trees)\n", 137 | " model.fit(x_train, y_train)\n", 138 | " y_pred = model.predict(x_test)\n", 139 | " accuracy = np.mean(y_test==y_pred)\n", 140 | " results.append(accuracy)\n", 141 | "\n", 142 | "plt.figure(figsize=(8,4))\n", 143 | "pd.Series(results, max_features_options).plot(kind=\"bar\",color=\"darkred\",ylim=(0.7,0.9))" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": 6, 149 | "metadata": {}, 150 | "outputs": [ 151 | { 152 | "data": { 153 | "text/plain": [ 154 | "" 155 | ] 156 | }, 157 | "execution_count": 6, 158 | "metadata": {}, 159 | "output_type": "execute_result" 160 | }, 161 | { 162 | "data": { 163 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAD3CAYAAADrNaaQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdeVhUdfvH8fcw7JuG4lIJLrkAaoZ7iraIuxKgrGKZadqeS6QmmQoKlfXYQpJGPQguiZobjwqpuJsYqThUWmBp7imbOMDM749sfpIYoshhhvt1XV0Xc7b53H7Tm3PmzPeo9Hq9HiGEEEIYNTOlAwghhBDi7klDF0IIIUyANHQhhBDCBEhDF0IIIUyANHQhhBDCBEhDF0IIIUyAeWUb6HQ6Zs2axY8//oilpSVz587F1dXVsH7dunXEx8djZmaGv78/ISEhhnUXL17Ez8+PL774glatWqHRaJgzZw5qtRpLS0uio6Np2LAhc+fO5dChQ9jZ2QHw6aef4uDgcA/KFUIIIUxTpQ09NTUVrVbLihUryMzMZP78+cTGxhrWx8TEsGHDBmxtbRkyZAhDhgyhXr16lJSUEBERgbW1tWHbyMhIZs6ciZubG8uXL+fzzz9n2rRpZGVlsXjxYpycnO5NlUIIIYSJq/SSe0ZGBl5eXgB06tSJo0ePllvftm1b8vPz0Wq16PV6VCoVANHR0QQFBdGoUSPDtgsWLMDNzQ2AsrIyrKys0Ol05ObmEhERQVBQEKtWraq24oQQQoi6otIz9IKCAuzt7Q2v1Wo1paWlmJv/tWvr1q3x9/fHxsYGb29vHB0dWb16NU5OTnh5eREXF2fY9+/mfujQIZYuXUpiYiJFRUWMGjWKMWPGUFZWxujRo2nfvj3t2rUrlyMjI6NaChZCCCGMRefOnW9720obur29PYWFhYbXOp3O0Myzs7PZvn07aWlp2NraMnXqVFJSUkhOTkalUrF37140Gg3h4eHExsbi7OzMpk2biI2NJS4uDicnJ0MTt7GxAaBHjx5kZ2ff1NCrWlhtpNFoDFcojJkp1GEKNYDUUZuYQg1gGnWYQg1Q9RPZSi+5e3p6kp6eDkBmZiZt2rQxrHNwcMDa2horKyvUajVOTk7k5eWRmJjI0qVLSUhIwM3NjejoaJydnfnmm28My5s1awZATk4OISEhlJWVUVJSwqFDh/Dw8KhSEUIIIURdV+kZure3N7t37yYoKAi9Xk9UVBTr16+nqKiIwMBAAgMDCQkJwcLCAhcXF3x9fSs8TllZGZGRkTRt2pSXX34ZgK5du/LKK68wbNgwAgICsLCwwMfHh9atW1dvlUIIIYSJq7Shm5mZMXv27HLLWrVqZfg5ODiY4ODgW+6fkJBg+PnAgQMVbjNu3DjGjRtXaVghhBBCVEwmlhFCCCFMgDR0IYQQwgRIQxe37VhiInHNm7PRw4O45s05lpiodCQhhBDXVfoZuhDwVzPfMn48pUVFAOTl5rJl/HgA3ENDlYwmhBACOUMXt2nXjBmGZv630qIids2YoVAiIYQQN5KGLm5L3smTFS/PzeWHRYu4evFiDScSQghxI2no4rbYNm5c4XIzc3O2TphAbJMmrB46FE1SEtqCghpOJ4QQQj5DF5XSlZaitrAAlQr0esNyc1tb+sfF0dDDA01SEtnLl/PLxo2Y29ry0PDhtAsOpsXAgagtLRVML4QQdYOcoYtKZfznP+T/9huer7yCo6srqFQ4urrSPy4O99BQGnXqRN+YGMbn5BCUno7H00+Tu3Ura318iG3ShM3jxnFy2zZ0ZWVKlyKEECZLztDFv8o7eZLdERG0HDqUxz/4gCc+/PCWDz5QmZnxoJcXD3p58cR//kNuairZ18/cjyxejF3TprQLCqJdcDBNunQxPGpXCCHE3ZOGLv5V2vV595/8+OMqNWC1hQUtBw2i5aBBlBQVcWLDBrKXLSPzk0/I+OAD6j/0EO2Cg3ELDqaBCTwVSQghlCaX3MUt/bx2LSfWraPXO+9Qz9X1jo9jYWtLu4AAnlqzholnzzJgyRIcXV3ZHxlJvLs7/33kEQ68+y55v/1WjemFEKJukYYuKqTNzyftpZdw7tgRz1dfrbbjWtevT4dnnyUgNZXnf/+dxz/8ELWVFelvvEGciwvL+/QhMzaWogsXqu09hRCiLpCGLiq0OyKCgtOn8V606K873O8B+6ZN6fzqq4Tu28dzx4/Te+5crl68SOoLL/BZ06YkDx7MsaVL0ebn35P3F0IIUyINXdzk7KFDHFq4kIeff577e/Sokfes36oVPWbM4JmjR3n6hx/oMnkyF48dY1NYGJ82bsz6wEB+XruW0mvXaiSPEEIYG7kpTpSjKytjy/PPY+PsjNe8eTX+/iqVCueOHXHu2BGvqChO79uHJimJH1eu5MeVK7GqV4/W/v64BQfT7PHHMVOrazyjEELURtLQRTmZsbGcPXiQIUlJWNevr2gWlZkZDzz6KA88+ihPfPghuWlpaJKS+Onrrzn6xRfYNWlC28BA3IKDadKtm3wNTghRp8kld2GQf+oUu6ZPp3n//rQLClI6Tjlm5ua0GDCAwV99xcSzZxn29dfc/+ij/PDZZyT26MHihx5i11tvceHYMaWjCiGEIqShC4Ntr72GrqSEfp9+WqvPdi1sbGg7YgQ+ycm8cPYsA+Pjqd+qFfvnzeNLDw++evhh9kdHcyU3V+moQghRY6ShCwB+2bSJn1atosdbb1G/VSul49w2q3r1aP/MM4zcsoUJp0/zxMKFWNjZsfPNN/m8eXOW9e7N9598QuG5c8Bfz3WPa96cjR4exDVvzrHERIUrEEKI6lFpQ9fpdERERBAYGEhYWBi5/zjrWbduHb6+vvj7+5OUlFRu3cWLF+nbty8nTpwAQKPREBISQlhYGGPHjuXC9e8ar1y5Ej8/PwICAti2bVt11SZuU0lREakvvoiTmxtdp05VOs4ds2vcGM+XXyZkzx6e++UXvKKiuHblCmkvvcRn99/Plx078r9nnyUvNxf0evJyc9kyfrw0dSGESai0oaempqLValmxYgWTJ09m/vz55dbHxMQQHx/PsmXLiI+P58qVKwCUlJQQERGBtbW1YdvIyEhmzpxJQkIC3t7efP7555w/f56EhASWL1/OkiVLWLBgAVqttprLFP9m7+zZ5OXk4P3ZZybzZLT6LVrQfdo0njlyhKcPH6bbG29w8dgxdP/4f6u0qIhdM2YolFIIIapPpQ09IyMDLy8vADp16sTRo0fLrW/bti35+flotVr0er3hs9fo6GiCgoJo1KiRYdsFCxYYHupRVlaGlZUVhw8f5pFHHsHS0hIHBwdcXFzIzs6utgLFvzt/5AgH33+f9s8+S7M+fZSOc084d+iAV1QUep2uwvV5J0/WcCIhhKh+lX5traCgAHt7e8NrtVpNaWkp5uZ/7dq6dWv8/f2xsbHB29sbR0dHVq9ejZOTE15eXsTFxRn2/bu5Hzp0iKVLl5KYmMjOnTtxcHAwbGNnZ0dBQUGFWTQazZ1VWUsUFxfXqhr0Oh17Ro/G3MGBps89d9vZalsdt8umSROu/vHHTcutmzQxynrAeMfin0yhDlOoAUyjDlOo4U5U2tDt7e0pLCw0vNbpdIZmnp2dzfbt20lLS8PW1papU6eSkpJCcnIyKpWKvXv3otFoCA8PJzY2FmdnZzZt2kRsbCxxcXE4OTnddPzCwsJyDf5GFT2y05jc6rGjSvkhLo7LmZkM/PJL2vfsedv71bY6bpf+3XfZMn48pUVF5ZY7NG5Mm4ceumdT3N5LxjoW/2QKdZhCDWAadZhCDfDXFfKqqPSSu6enJ+np6QBkZmbSpk0bwzoHBwesra2xsrJCrVbj5OREXl4eiYmJLF26lISEBNzc3IiOjsbZ2ZlvvvnGsLxZs2YAdOzYkYyMDK5du0Z+fj4nTpwo9x7i3ig8e5b08HCaPfYYHqNHKx2nRriHhtI/Lg5HV1dQqXB0dcVjzBjOZ2ayadQodGVlSkcUQog7VukZure3N7t37yYoKAi9Xk9UVBTr16+nqKiIwMBAAgMDCQkJwcLCAhcXF3x9fSs8TllZGZGRkTRt2pSXrz9ju2vXrrzyyiuEhYUREhKCXq/n9ddfx8rKqnqrFDfZPnkyJYWF9IuNrdXfOa9u7qGhuIeGlvsNvqGHBzumTMHc1paBS5agMpNvcwohjE+lDd3MzIzZs2eXW9bqhu8pBwcHExwcfMv9ExISDD8fOHCgwm0CAgIICAioNKyoHrmpqWgSE+kZEUGDdu2UjqO4rpMnU1JQwJ5Zs7C0t+eJhQvr1C85QgjTIHO51zGlxcVsnTiR+1q3pvu0aUrHqTV6RkSgLSjg4HvvYWFnh9e8edLUhRBGRRp6HbMvKorLx48zcutWzG+YI6CuU6lU9I2JoaSggAPR0Vg6ONBDvp8uhDAi0tDrkIvZ2RyYPx+30FBc+/VTOk6to1Kp6PfJJ5QUFrLrrbewsLOj82uvKR1LCCFuizT0OkKv17N1wgQs7Ox47P33lY5Ta6nMzBj4xReUFBWx7fXXsbCzo+O4cUrHEkKISsntvHVE1n//y+87dtAnOhq7xo2VjlOrmZmbMzQpiRaDBrHl+efR/OMZBUIIURtJQ68Dii5cYMfkydz/6KN0fO45peMYBbWlJcOTk2nWty+bRo/m5zVrlI4khBD/Shp6HZD+xhtcu3IF70WL5DvWVWBhY4PvunU06dqV9YGB/Pq//ykdSQghbkn+dTdxv6WnczQ+ni6TJ+Pcvr3ScYyOpYMD/ikpNPTw4BtfX37bsUPpSEIIUSFp6CasTKtl64QJODZvTs+ICKXjGC3r+vUZsWULji1asHroUP64xQRJQgihJGnoJuy7d9/lkkZDv08+wcLWVuk4Rs3W2ZmA1FRsGzVi1YABnPvhB6UjCSFEOdLQTdSfx4+zd84c2owcScvBg5WOYxLs77+fgLQ0LO3t+drbm4vZ2UpHEkIIA2noJkiv15P6wguorax44sMPlY5jUuo1b87ItDRUZmZ83a8fl3/9VelIQggBSEM3SdnLl5O7dSu9IyOxv/9+peOYHKc2bRi5dSulRUV8/eST5J86pXQkIYSQhm5qii9fZtvrr9O4Sxc6TZyodByT5dyhA/6bN3P1wgW+7tePwnPnlI4khKjjpKGbmJ3TpnH1/Hn6L1qEmVqtdByT1rRrV/w2biQvN5dV/ftT/OefSkcSQtRh0tBNyOm9e/lh0SI8X3mFxp6eSsepEx708uKptWu5pNGQPGgQ2vx8pSMJIeooaegmoqykhC3PP4/DAw/Qa/ZspePUKc3792fYypWcOXiQ1UOHUlJUpHQkIUQdJA3dRGR8+CEXjhzhiY8+wtLBQek4dc5DPj4MWbqU33fu5Bt/f0qvXVM6khCijpGGbgKu5OayZ9YsWg0fTuunnlI6Tp3VLiiIAYsXk/O//7ExOBhdaanSkYQQdYg0dCOn1+tJe+klVCoVT370kdJx6rwOzz7LE//5Dz+vWUPKM8+g1+mUjiSEqCPMlQ4g7s7Pa9bwy4YN9H3vPRxdXJSOIwDPV16hpLCQndOnY2Fnh/dnn6FSqZSOJYQwcZU2dJ1Ox6xZs/jxxx+xtLRk7ty5uLq6GtavW7eO+Ph4zMzM8Pf3JyQkxLDu4sWL+Pn58cUXX9CqVSvD8qioKFq0aEFwcDAAc+fO5dChQ9jZ2QHw6aef4iCfA1fqWl4e3778Ms4PP0znV19VOo64Qfdp09AWFLA/KgoLOzsee/99aepCiHuq0oaempqKVqtlxYoVZGZmMn/+fGJjYw3rY2Ji2LBhA7a2tgwZMoQhQ4ZQr149SkpKiIiIwNra2rDtpUuXeOONN8jJyWHs2LGG5VlZWSxevBgnJ6dqLs+07Z45k4I//sBnzRrMzOViS23Te+5cSgoKyPjgAywdHOj1zjtKRxJCmLBKP0PPyMjAy8sLgE6dOnH06NFy69u2bUt+fj5arRa9Xm84C4mOjiYoKIhGjRoZti0sLOTll1/Gx8fHsEyn05Gbm0tERARBQUGsWrWqWgozdWcyMvj+44/pNHEiTbt1UzqOqIBKpeLxDz6gw9ix7J09mwMxMUpHEkKYsEpP6woKCrC3tze8VqvVlJaWYn79jLB169b4+/tjY2ODt7c3jo6OrF69GicnJ7y8vIiLizPs26xZM5o1a0Z6erphWVFREaNGjWLMmDGUlZUxevRo2rdvT7t27W7KotFo7qpYpRUXF1dLDfqyMnY//TSWTk40evrpGv9zqa46lFSTNTR77TUu/vEH6eHhXCwooPn1j5qqgymMBZhGHaZQA5hGHaZQw52otKHb29tTWFhoeK3T6QzNPDs7m+3bt5OWloatrS1Tp04lJSWF5ORkVCoVe/fuRaPREB4eTmxsLM7Ozjcd38bGhtGjR2NjYwNAjx49yM7OrrChu7m53XGhtYFGo6mWGg4tXMiVrCyGLl9OOwXOzqurDiXVdA1t165l3YgRZM2ZQ7OWLWn/zDPVclxTGAswjTpMoQYwjTpMoQb46wp5VVR6yd3T09NwRp2ZmUmbNm0M6xwcHLC2tsbKygq1Wo2TkxN5eXkkJiaydOlSEhIScHNzIzo6usJmDpCTk0NISAhlZWWUlJRw6NAhPDw8qlREXZL/++/snDGD5gMH0jYgQOk44japLSwYtmIFrt7ebB47luyVK5WOJIQwMZWeoXt7e7N7926CgoLQ6/VERUWxfv16ioqKCAwMJDAwkJCQECwsLHBxccHX17dKAVq1asWwYcMICAjAwsICHx8fWrdufccFmbpvX30VfWkp/T75RO6aNjLm1tY8tXYtyQMHsik0FAtbW1oNHap0LCGEiai0oZuZmTH7H3OD3/gVtODgYMPXzyqSkJBw07KXX3653Otx48Yxbty4SsPWdSc2bODn1avxioqifsuWSscRd8DC1hbfDRtY+eSTrBsxAr+NG3F98kmlYwkhTIDMFGcktIWFpL74Ig3c3ekyebLSccRdsHJ0ZMTmzdzXpg1rhg/n1O7dSkcSQpgAaehGYu8775B/8iTeixahtrRUOo64SzZOTozcuhWHBx8kefBgzh46pHQkIYSRk4ZuBM798AMHFyygw9ixPNi7t9JxRDWxa9yYkampWN93H6v69+dCVpbSkYQQRkwaei2n1+nYOmEC1k5O9JGJSUyOY7NmBKSlYWZpydf9+vHn8eNKRxJCGClp6LXcD3Fx/LFvH4+9/z42MjWuSarfqhUjU1PRlZay8sknyTt5UulIQggjJA29Fis8c4adb76JyxNP4D5qlNJxxD3U0N2dEVu2oL1yhZVPPknBH38oHUkIYWSkoddi2yZNovTqVfp9+ql857wOaPzII/inpFD4xx987e1N0YULSkcSQhgRaei1VM6WLWQvW0a3adNwattW6Tiihtzfsye+GzZw5cQJkgcM4NqVK0pHEkIYCWnotVDJ1atsnTiR+9q0ofubbyodR9Qwl8ceY3hyMuePHCF58GC0NzxLQQghbkUaei20PzKSK7/8gvdnn2F+w/PkRd3RcvBghi5bxh/79rHWx4fS4mKlIwkhajlp6LXMRY2GAzExuIeF4fL440rHEQpq4+/PwC+/5GRaGutGjqSspETpSEKIWkwaei2i1+vZOmEClvb29H3vPaXjiFrAIyyMfrGx/LJhA5tGjUJXVqZ0JCFELVXpw1lEzTn65Zf8np5O/88/x65RI6XjiFqi04QJlBQWsmPKFMxtbRm4ZAkqM/ldXAhRnjT0WqLo/Hl2TJnCA7160eHZZ5WOI2qZrpMnU1JQwJ5Zs7C0t+eJhQvlq4xCiHKkodcSO6ZORZuXh/eiRXL2JSrUMyICbUEBB997jys5OZw/fJj8335jp4sLvSMjcQ8NVTqiEEJB0tBrgZPbt5P11Vd0nzaNhh4eSscRtZRKpaJvTAxnv/+eXzZsMCzPy81ly/jxANLUhajD5FRQYaXXrpE6YQL1WrSgx1tvKR1H1HIqlYrLP/980/LSoiJ2zZihQCIhRG0hZ+gK+y4mhks//oh/SgoWtrZKxxFGIP+33ypcLg91EaJukzN0Bf3588/si4ykbUAALQYOVDqOMBKOLi5VWi6EqBukoStEr9ezdeJE1FZWPP7hh0rHEUakd2Qk5v+8mqNS0fPtt5UJJISoFSpt6DqdjoiICAIDAwkLCyM3N7fc+nXr1uHr64u/vz9JSUnl1l28eJG+ffty4sSJcsujoqJYtmyZ4fXKlSvx8/MjICCAbdu23U09RkOTlMTJtDS85s3DvmlTpeMII+IeGkr/uDgcXV1BpcLG2Rn0en7btg29Xq90PCGEQipt6KmpqWi1WlasWMHkyZOZP39+ufUxMTHEx8ezbNky4uPjuXL96VAlJSVERERgfcNc5JcuXeK5557j22+/NSw7f/48CQkJLF++nCVLlrBgwQK0Wm111VcrFf/5J9snTaJJt248/PzzSscRRsg9NJTxOTkMycrixXPn6DV7NscSEvj+44+VjiaEUEilDT0jIwMvLy8AOnXqxNGjR8utb9u2Lfn5+Wi1WvR6vWGyi+joaIKCgmh0w4xnhYWFvPzyy/j4+BiWHT58mEceeQRLS0scHBxwcXEhOzu7WoqrrdLffJOrFy/Sf9EizNRqpeMIE9BjxgxaDR/O9kmT+C09Xek4QggFVHqXe0FBAfb29obXarWa0tJSzM3/2rV169b4+/tjY2ODt7c3jo6OrF69GicnJ7y8vIiLizPs26xZM5o1a0b6Df/gFBQU4ODgYHhtZ2dHQUFBhVk0Gk3VK6xFiouL2Z2UxOG4OFo8/TQXray4aIQ1FRcXm8RYGHsNUL6OVjNmcObIEdb4+dH766+xadJE4XS3zxTGwxRqANOowxRquBOVNnR7e3sKb3ges06nMzTz7Oxstm/fTlpaGra2tkydOpWUlBSSk5NRqVTs3bsXjUZDeHg4sbGxODs7V3r8wsLCcg3+Rm5ublUusDbJOnyYH+bNw6FZM4Z9/DGWN/yiZEw0Go3Rj4Up1AA31/HAxo0s7daNY2++SVB6OuZWVgqmu32mMB6mUAOYRh2mUAP8dYW8Kiq95O7p6Wk4o87MzKRNmzaGdQ4ODlhbW2NlZYVarcbJyYm8vDwSExNZunQpCQkJuLm5ER0dXWEzB+jYsSMZGRlcu3aN/Px8Tpw4Ue49TMGxxETimjcn5eGHuXD0KG1GjjTaZi5qtwZubgz+7385c+AAaS+9pHQcIUQNqvQM3dvbm927dxMUFIRerycqKor169dTVFREYGAggYGBhISEYGFhgYuLC76+vlUK4OzsTFhYGCEhIej1el5//XWsjOSs4nYcS0xky/jxlBYVGZb98NlnNPb0lGk6xT3R2teXHjNmsC8ykiZdu/Lw9WlhhRCmTaU3ku+5ZGRk0LlzZ6VjVNkiV1fyK5jBy9HVlfE5OTUfqBqYwuUsU6gBbl2HrqyM1UOHcjItjaAdO7i/Z08F0t0+UxgPU6gBTKMOU6gBqt73ZGKZe0Sv13N83boKmznINJ3i3jJTqxmalIRDs2asGzGCwjNnlI4khLjHpKHfA2e//56VTz7JWh8fzMwr/lRDpukU95r1fffx1Nq1FF++zLqRIykz8fkdhKjrpKFXo/xTp0h55hkSOnfmwuHDPPnxx/RfsuSmaTrNbW3pHRmpUEpRlzh36MDAJUs4tWsX2yZNUjqOEOIekqetVQNtQQEHYmI4+N576MvK6DplCt2nT8e6fn3gr8ufu2bMIO/kSRxdXOgdGSk3xIka0y4oiDMZGRx87z2adOlC+2eeUTqSEOIekIZ+F3RlZWR9+SW73nqLwjNnaBsYiNe8edRv0aLcdu6hobiHhprMjRrC+PSZN49z33/P1gkTaNi+PU26dFE6khCimskl9zuUs3UrCZ6ebH7uORybNydkzx6GLV9+UzMXojYwMzdn6PLl2DVpwjd+fhSdP690JCFENZOGXkUXjh0jecgQVvXvjzY/n6ErVhCyZ0+t/1qQELYNG+KzejVXz59nfWAgutJSpSMJIaqRNPTbVHjuHFsnTuSrjh05vXs3fd99lzEaDe0CAgwPpBGitmvs6Yn3okX8tm0bO8LDlY4jhKhG8hl6JUquXuXQhx+yf948SoqK6DRxIj3ffhvbhg2VjibEHfEYPZozBw+SsWABTbp0wS04WOlIQohqIA39FvQ6HdnLl5M+bRr5J0/Savhw+sbE4NS2rdLRhLhrj73/PuczM9k8diwN3N1p9PDDSkcSQtwlueRegd937SKxRw82hoZi06ABAd9+i+8330gzFyZDbWHBsK+/xtrJiW98fbl66ZLSkYQQd0ka+g3+PH6cb/z9We7lRcHp0wz66ivCDh7E5fHHlY4mRLWza9wYn+RkCk6dYmNICLqyMqUjCSHugjR04OqlS2ybNIl4d3dyNm+m1+zZjP3pJzxGj0ZlJn9EwnQ17d6dJz/+mJzNm9k9c6bScYQQd6FOf4ZeptWS+emn7J09m2tXrtD+2WfpNXs29k2bKh1NiBrTcdw4zhw8yP5582jcuTNt/P2VjiSEuAN1sqHr9XqOr13Ljjfe4PLx47h6e/PYe+/h3LGj0tGEUMQTCxdy/vBhUp55Bic3Nxq6uysdSQhRRXXuevKZgwdZ0bcv3/j5oba0xG/TJkZs3izNXNRp5lZWDF+1Cgs7O7556imuXbmidCQhRBXVmYaed/IkG0eNYmnXrlz68Ue8P/uMp3/4gZaDBsnEMEIADg88wPBVq7jy669sCgtDr9MpHUkIUQUm39Cv5eWxc/p0vmjblp+Tk+k+bRpjf/6Zh59//pbPKheirnqwd28e++ADTqxfz945c5SOI4SoApPtaLrSUo4sWcLuiAiKzp3DLTQUr6goHF1clI4mRK32yIsvcvbgQfbMmkXjzp1pNXSo0pGEELfB5M7Q9Xo9v6Sk8NXDD7N1wgSc2rYl9MABhixdKs1ciNugUqnoFxtLY09PNoaGcumnn5SOJIS4DZU2dJ1OR0REBIGBgYSFhZGbm1tu/bp16/D19cXf35+kpKRy6y5evEjfvn05ceIEALm5uQQHBxMSEsLbb7+N7vpndHPnzsXPz+i2ErwAACAASURBVI+wsDDCwsLIz8+/o2LOHznCqgEDWD14MGVaLcOTkwncsYOmXbve0fGEqKssbGwYvno1aktLvvH1RXuHfyeFEDWn0oaempqKVqtlxYoVTJ48mfnz55dbHxMTQ3x8PMuWLSM+Pp4r1++OLSkpISIiAmtra8O28+bN47XXXiMpKQm9Xk9aWhoAWVlZLF68mISEBBISEnBwcKhSEYVnzrB53Dj+26kTZw8e5PEPPmBMVhZt/Pzkhjch7lA9V1eGrVjBpexsUsaMQa/XKx1JCPEvKm3oGRkZeHl5AdCpUyeOHj1abn3btm3Jz89Hq9Wi1+sNDTQ6OpqgoCAaNWpk2DYrK4tu3boB0KdPH/bs2YNOpyM3N5eIiAiCgoJYtWrVbYcvKSpi75w5LH7oIbK++grPV19l7PHjdH7tNdSWlrd9HCFExVyeeII+MTH8nJzMgehopeMIIf5FpTfFFRQUYG9vb3itVqspLS3F/Pod4q1bt8bf3x8bGxu8vb1xdHRk9erVODk54eXlRVxcnGHfGxu+nZ0d+fn5FBUVMWrUKMaMGUNZWRmjR4+mffv2tGvX7qYsGo3mr+PodJxav54fP/yQ4rNnaeLtTbtJk7BzdSXn7Fk4e/bu/lTukeLiYkMNxswU6jCFGqBm6rAbNIj7v/2WndOnc61BA5x796729zCF8TCFGsA06jCFGu5EpQ3d3t6ewsJCw2udTmdo5tnZ2Wzfvp20tDRsbW2ZOnUqKSkpJCcno1Kp2Lt3LxqNhvDwcGJjYzG7YV70wsJCHB0dsbGxYfTo0djY2ADQo0cPsrOzK2zoOwcNwm3UKHJSUjh76BBNunblsa+/5sHrVxBqO41Gg5ubm9Ix7pop1GEKNUDN1fHQypUse/RRDoeHM+rgQeq3bFmtxzfm8TiWmMiuGTPIO3kSRxcXekdG4h4aqnSsO2bMY/E3U6gB/rpCXhWVXnL39PQkPT0dgMzMTNq0aWNY5+DggLW1NVZWVqjVapycnMjLyyMxMZGlS5eSkJCAm5sb0dHRODs74+7uzv79+wFIT0+nS5cu5OTkEBISQllZGSUlJRw6dAgPD48Ks+Tl5rI/MpLLOTkMXrqU0H37jKaZC2HMLO3s8Fm9GuCvm+Ru+CW/LjuWmMiW8ePJy80FvZ683Fy2jB/PscREpaOJOqjShu7t7Y2lpSVBQUHMmzePadOmsX79elasWMEDDzxAYGAgISEhBAcHk5+fj6+v7y2PFR4ezkcffURgYCAlJSUMGDCAVq1aMWzYMAICAggLC8PHx4fWrVv/ayZLOzvcQ0PlSWhC1KD6rVoxJCmJ80eOsGXcOLlJDtg1YwalRUXllpUWFbFrxgyFEom6rNJL7mZmZsyePbvcslatWhl+Dg4OJjg4+Jb7JyQkGH5u0aIFS5cuvWmbcePGMW7cuNsKDJD/+++3va0Qovq0GDiQ3nPnsmvGDJp07UqX119XOpJi9Ho9eSdPVrjuVsuFuJeM8hRXJogRQjndp02jtZ8fO6ZO5eS2bUrHqXF6vZ4T69eT2L073OIqhf0DD9RwKiGMsKGb29rSOzJS6RhC1FkqlYpBX37JfW3asD4goM6cjep1On5KTibB05M1w4dz9cIF2o8di7mt7U3bll27xoVjxxRIKeoyo2rojq6u9I+LM+o7SIUwBZYODjy1Zg1lWi3f+PlRcvWq0pHuGV1ZGdnLl/Nlx46sGzGCksJCBn75Jc/++CMDFy+mf1wcjq6uoFLh6OrKo7NmoTIzI6lnT37dvFnp+KIOMaqGPj4nR5q5ELWEU9u2DE5I4GxGBqkvvGByN8npSkvJSkjgSw8PNgQHg17PkKQkxmg0tH/6adQWFgC4h4YyPieHIVlZjM/J4dG332bUgQPUa9GC1YMHc+jjjxWuRNQVRtXQhRC1y0PDh9MzIoKsL78kMzZW6TjVokyr5ciSJXzRrh0po0ejtrJi2Ndf88yRI7gFB2OmVld6DEcXF4J37aLl0KF8+/LLpL74ImUlJTWQXtRlJvv4VCFEzXj07bc5e+gQ2159FeeOHXnwHswkVxNKr13jaHw8B+bPJy83l8adO/PU2rW0Gjbsjr4ia2lvj8/q1eycNo3v3n2XP3/6iWFff411/fr3IL0QcoYuhLhLKjMzBickUK9FC9aPHEnB6dNKR6qSkqtXObRwIYtbtSJ14kTsmjbFb9MmRn33HQ/5+NzVfBdmajV9Y2IY8MUX/LZjB0k9evDn8ePVmF6I/ycNXQhx16zr18dnzRq0+fmsGzGC0mvXlI5UKW1hId+9/z6ft2jBt6++Sv1WrRi5dSshe/bQctCgan1SY4cxYwhIS+PqhQskdu/Oye3bq+3YQvxNGroQolo09PBg4JdfcnrvXra9+qrScW5Jm5/P/vnz+bx5c3ZMmULD9u0J3L6doB07cO3X7549cvlBLy9CDxzAtnFjVnl7c3jx4nvyPqLukoYuhKg2bUeMoFt4OD8sWsSRJUuUjlNO8eXL7Jk9mzhXV3ZOm0aTrl0J2bOHgNRUmvXtWyMZ6rdsSejevbg8+SRbxo1j2+TJ6MrKauS9hemTm+KEENWqd2Qk577/ntQXXqBhhw407dZN0TxXL14k44MPOPTRR2jz8mg1fDg93nqLpl27KpLHql49/DZsYNukSWQsWMCfP/3E0KQkLB0cFMkjTIecoQshqpWZWs2QpCTs7r+fb/z8KDx7VpEchefOsSM8nLjmzdkXGYmrtzejv/8e32++UayZ/83M3JwnFy6k36ef8mtKCkm9enElN1fRTML4SUMXQlQ7mwYNeGrNGoovXWJ9QECNfge74I8/2DZpEp83b853775Lq2HDeOboUXxWraJRp041luN2dJo4Ef+UFPJPniSxWzdO7dmjdCRhxKShCyHuiUadOjFg8WJ+T09nx5Qp9/z98n77jdSXXuLzFi04tHAhbUaO5FmNhqFJSTT08Ljn73+nmnt7E7JvH5YODqx8/HF5lrq4Y/IZuhDinnELCeHMd9+R8eGHNOnaFfdRo6r9PS7/+isH5s/naHw86PV4PP003adNo/4Nj3mu7Rq0a0fo/v2sGzGCTaNGcUmjodfs2Xf1HXhR90hDF0LcU31iYjiXmcmWceNo4OFB40ceqZbj/nn8OPujosj6738xU6vp8NxzdAsPp56ra7Ucv6bZNGjAiM2bSX3xRfZFRnIxO5tBX32FpZ2d0tGEkZBf/4QQ95TawoKhK1Zg07Ah3/j6UnThwl0d76JGw8ZRo/iibVuyly3jkRdf5LlffsH700+Ntpn/TW1pSf+4OB5bsICfV69mRZ8+5J86pXQsYSSkoQsh7jm7Ro0Yvno1hWfOsCEoCF1paZWPcf7IEdYHBhLv4cHPa9bQedIkxv36K0/85z84PPDAPUitDJVKRZfXX8d3/Xou/fQTid26cSYjQ+lYwghIQxdC1IimXbvSLzaWk2lp7Jw+/bb3O3voEGt9ffmqY0d+TUmh+5tvMj4nh8fefRe7Jk3uYWJltRoyhJA9ezCzsGC5lxc/rlqldCRRy0lDF0LUmA5jxvDwxIl89+67ZK9c+a/b/rF/P6uHDiWhc2d+27aNnhERjMvJwSsqCltn5xpKrCznDh0IPXCARp06sX7kSPZFRprcc+dF9am0oet0OiIiIggMDCQsLIzcf0x+sG7dOnx9ffH39ycpKancuosXL9K3b19OnDgBQG5uLsHBwYSEhPD222+j0+kAWLlyJX5+fgQEBLBt27bqqk0IUQs98eGH3P/oo/xvzBjOHzly0/rfd+7k6/79SezRg9N799J77lzG5+bS6513sHFyUiCxsuwaNSLg229xCw1l11tvsSksjNLiYqVjiVqo0rvcU1NT0Wq1rFixgszMTObPn09sbKxhfUxMDBs2bMDW1pYhQ4YwZMgQ6tWrR0lJCREREVhbWxu2nTdvHq+99hrdu3cnIiKCtLQ0OnXqREJCAsnJyVy7do2QkBB69eqFpaXlvalYCKEotaUlw1etIqFzZ1b264e5pSX5p06xzdkZ6wYNuKTRYOPsTJ/oaDpNnChTogLm1tYMTkiggbs7u2bM4Movv+CzZg12jRsrHU3UIpWeoWdkZODl5QVAp06dOHr0aLn1bdu2JT8/H61Wi16vNzypKDo6mqCgIBo1amTYNisri27X53Xu06cPe/bs4fDhwzzyyCNYWlri4OCAi4sL2dnZ1VagEKL2sW/alI7jx3P13Dnyf/8d9HqKzp3jkkaDW2go43Ny6PbGG9LMb6BSqegxfTrDV63iXGYmS7t14/zhw0rHErVIpWfoBQUF2NvbG16r1WpKS0sxN/9r19atW+Pv74+NjQ3e3t44OjqyevVqnJyc8PLyIi4uzrDvjQ3fzs6O/Px8CgoKcLjhL62dnR0FBQUVZtFoNHdWZS1RXFxs9DWAadRhCjWAcdeRecO/DTfK+fZbjhvhvOY1Nhbu7vT473/57sUXWdqzJ4+8+y6NH3+82g5vzP9P/c0UargTlTZ0e3t7CgsLDa91Op2hmWdnZ7N9+3bS0tKwtbVl6tSppKSkkJycjEqlYu/evWg0GsLDw4mNjcXshlmPCgsLcXR0vOn4hYWF5Rr8jdzc3O640NpAo9EYfQ1gGnWYQg1g3HVsPHOmwuVXz5wxyppqdCzc3PDo2ZO1Pj4cfOkl+r77Ll0mTaqWZ7kb8/9TfzOFGuCvK+RVUekld09PT9LT0wHIzMykTZs2hnUODg5YW1tjZWWFWq3GycmJvLw8EhMTWbp0KQkJCbi5uREdHY2zszPu7u7s378fgPT0dLp06ULHjh3JyMjg2rVr5Ofnc+LEiXLvIYQwTY4uLlVaLspzeOABgtLTaePvz44pU9gybhxlWq3SsYSCKj1D9/b2Zvfu3QQFBaHX64mKimL9+vUUFRURGBhIYGAgISEhWFhY4OLigq+v7y2PFR4ezsyZM1mwYAEtW7ZkwIABqNVqwsLCCAkJQa/X8/rrr2NlZVWtRQohap/ekZFsGT+e0qIiwzJzW1t6R0YqmMq4WNjaMmzFCna//Tb75s7lz+PH8UlOxqZBA6WjCQWo9EbypcaMjAw6d+6sdIy7YiqXgUyhDlOoAYy/jmOJieyaMYO8kydxdHGhd2Qk7qGhSse6I0qPxbHERDaPHYvDgw/iu2EDDdq1u6PjKF1HdTCFGqDqfU8mlhFCKMb9+h3tQ7KyGJ+TY7TNvDZwDw0lcNs2tPn5JPXoQc7WrUpHEjVMGroQQpiI+3v2JPTAARxcXEgeNIjvP/1U6UiiBklDF0IIE1LP1ZWQ3btpMWgQaS++SNrLL9/Rw3CE8ZGGLoQQJsbSwYGn1q6ly+TJfP/xx6weMoTiy5eVjiXuMWnoQghhgszUah577z0GLF7MyW+/JalnTy5ff66GME3S0IUQwoR1GDuWEVu3UnTuHIndu/Pb9XlFhOmRhi6EECbO5bHHCN2/HxtnZ77u148j8fFKRxL3gDR0IYSoA+576CFC9u6l2WOPsfnZZ9k+dSq6sjKlY4lqJA1dCCHqCOv69fHftIlOL7zAwffe4xs/P7S3eBiWMD7S0IUQog4xMzen3yef8OTHH/PLxo0s69WLvJMnlY4lqoE0dCGEqIMeefFF/Ddt4kpODku7dWP3O+8Q17w5Gz08iGvenGOJiUpHFFUkDV0IIeqo5v37E7pvH3qdjr2zZpGXmwt6PXm5uWwZP16aupGRhi6EEHVYAzc31BU84bK0qIhdM2YokEjcKWnoQghRxxWcOlXhcvls3bhIQxdCiDrO0cWlwuUOzZrVcBJxN6ShCyFEHdc7MhJzW9ubllvY23MtL0+BROJOSEMXQog6zj00lP5xcTi6uoJKhaOrKx0nTODyTz+x8vHHKTx7VumI4jZIQxdCCIF7aCjjc3IYkpXF+Jwc+sfG8tS6dVzMzmZZr15c/uUXpSOKSkhDF0IIUaGWgwYRkJZG8Z9/kvToo5zLzFQ6kvgX0tCFEELc0v09ehC8axdqS0uW9+3Lye3blY4kbkEauhBCiH/VwM2N4N27cXjwQZIHDOCn5GSlI4kKVNrQdTodERERBAYGEhYWRm5ubrn169atw9fXF39/f5KSkgAoKytj2rRpBAUFERoaysnr32XMyspixIgRhISEMGfOHHQ6HQBz587Fz8+PsLAwwsLCyM/Pr+46hRBC3AXHZs0I2rmTxp07s27kSH5YtEjpSOIfKm3oqampaLVaVqxYweTJk5k/f3659TExMcTHx7Ns2TLi4+O5cuUK27ZtA2D58uW88sorzJs3D4CZM2cyffp0kpKSsLe3Z/369cBfjX7x4sUkJCSQkJCAg4NDddcphBDiLtk4OTEyNZWWgwezdcIE9syejV6vVzqWuK7Shp6RkYGXlxcAnTp14ujRo+XWt23blvz8fLRaLXq9HpVKRb9+/ZgzZw4Ap0+fpmHDhgCcPXsWT09PADw9PcnIyECn05Gbm0tERARBQUGsWrWqWgsUQghRfSxsbfFZswaPp59mz9tvk/bSS/Jc9VrCvLINCgoKsLe3N7xWq9WUlpZibv7Xrq1bt8bf3x8bGxu8vb1xdHT868Dm5oSHh7N161YWLlwIQLNmzThw4ADdunVj27ZtXL16laKiIkaNGsWYMWMoKytj9OjRtG/fnnbt2t2URaPRVEvRSikuLjb6GsA06jCFGkDqqE1MoQa4/Tpc33iDq+bmZH76KWeOH6dTTAxqS8saSFg5UxmLqqq0odvb21NYWGh4rdPpDM08Ozub7du3k5aWhq2tLVOnTiUlJYVBgwYBEB0dzZQpUwgICGDjxo1ERUURGRnJ4sWL6dChA5aWltjY2DB69GhsbGwA6NGjB9nZ2RU2dDc3t2opWikajcboawDTqMMUagCpozYxhRqganW4L17Md25u7JgyhWOlpfisWYPV9ZM6JZnKWGRkZFRp+0ovuXt6epKeng5AZmYmbdq0MaxzcHDA2toaKysr1Go1Tk5O5OXlsXbtWhZdv2HCxsYGlUqFWq1mx44dREVFERcXx+XLl+nVqxc5OTmEhIRQVlZGSUkJhw4dwsPDo0pFCCGEUEbXyZMZnJDA7+nprHjsMZlVTkGVnqF7e3uze/dugoKC0Ov1REVFsX79eoqKiggMDCQwMJCQkBAsLCxwcXHB19eX0tJSpk2bRmhoKKWlpUyfPh0rKytcXV0ZP348NjY2dO/enb59+wIwbNgwAgICsLCwwMfHh9atW9/zwoUQQlQP91GjsG7QgHUjRrCsVy9GbNlC/ZYtlY5V56j0RnKLYkZGBp07d1Y6xl0xlctAplCHKdQAUkdtYgo1wN3VcXrfPlYPGYKZhQUj/vc/GnXqVM3pbo+pjEVV+55MLCOEEKJalJtVrk8fmVWuhklDF0IIUW0Ms8o1ayazytUwaehCCCGqlWFWuS5dWDdyJJmffaZ0pDpBGroQQohqZ+PkxMitW2k5eDCpEyey5513ZFa5e0wauhBCiHui3Kxys2bJrHL3WKVfWxNCCCHulNrCgoHx8dg2bsx3MTEUnTvH4KVLMbeyUjqayZGGLoQQ4p5SqVT0jY7GrnFjtk+ezNWLF3lq7dpaMaucKZFL7kIIIWpEl0mTGJyQwKmdO2VWuXtAGroQQoga4z5qFE+tW8elH39kWa9eXD5xQulIJkMauhBCiBrVctAgAr/9luI//ySpVy/Ofv+90pFMgjR0IYQQNa5p9+6GWeVW9O3LyW3blI5k9KShCyGEUEQDNzdC9uz5a1a5gQP5cdUqpSMZNWnoQgghFOPw4IOGWeXWBwTIrHJ3QRq6EEIIRcmsctVDGroQQgjFGWaVe+YZ9syaReqLL8qsclUkE8sIIYSoFdQWFgz84gtsGzXiu5gYrp4/L7PKVYE0dCGEELWGzCp35+SSuxBCiFpHZpWrOmnoQgghaiWZVa5qpKELIYSotf6eVe7a5csyq1wlKm3oOp2OiIgIAgMDCQsLIzc3t9z6devW4evri7+/P0lJSQCUlZUxbdo0goKCCA0N5eTJkwBkZWUxYsQIQkJCmDNnDjqdDoCVK1fi5+dHQEAA22S2ICGEEDdo2r07QTKrXKUqbeipqalotVpWrFjB5MmTmT9/frn1MTExxMfHs2zZMuLj47ly5YqhKS9fvpxXXnmFefPmATBz5kymT59OUlIS9vb2rF+/nvPnz5OQkMDy5ctZsmQJCxYsQKvV3oNShRBCGKsG7drJrHKVqLShZ2Rk4OXlBUCnTp04evRoufVt27YlPz8frVaLXq9HpVLRr18/5syZA8Dp06dp2LAhAGfPnsXT0xMAT09PMjIyOHz4MI888giWlpY4ODjg4uJCdnZ2tRYphBDC+Mmscv+u0q+tFRQUYG9vb3itVqspLS3F3PyvXVu3bo2/vz82NjZ4e3vjeP2rBebm5oSHh7N161YWLlwIQLNmzThw4ADdunVj27ZtXL16lYKCAhwcHAzHt7Ozo6CgoMIsGo3mziutBYqLi42+BjCNOkyhBpA6ahNTqAGMo46OH31E6eTJpE6cSO6RI7R+8UVUKpVhvTHUcC9U2tDt7e0pLCw0vNbpdIZmnp2dzfbt20lLS8PW1papU6eSkpLCoEGDAIiOjmbKlCkEBASwceNGoqKiiIyMZPHixXTo0AFLS8ubjl9YWFiuwd/Izc3tropVmkajMfoawDTqMIUaQOqoTUyhBjCeOty2bGHL+PFkffoptno9T370EWZqNWA8NVQmIyOjSttXesnd09OT9PR0ADIzM2nTpo1hnYODA9bW1lhZWaFWq3FyciIvL4+1a9eyaNEiAGxsbFCpVKjVanbs2EFUVBRxcXFcvnyZXr160bFjRzIyMrh27Rr5+fmcOHGi3HsIIYQQ//T3rHLdwsP5ITaW9YGBlBYXKx1LUZWeoXt7e7N7926CgoLQ6/VERUWxfv16ioqKCAwMJDAwkJCQECwsLHBxccHX15fS0lKmTZtGaGgopaWlTJ8+HSsrK1xdXRk/fjw2NjZ0796dvn37AhAWFkZISAh6vZ7XX38dK5nmTwghRCVUKhV95s/HtnFjtk+axH+PHaOkoID8339np4sLvSMjcQ8NVTpmjVHpjeSRNhkZGXTu3FnpGHfFVC4DmUIdplADSB21iSnUAMZbx9aJE/nhHzfJmdva0j8uzmibelX7nszlLoQQwuj9mpJy07LSoiK2TpjAJY0Gayenv/67777///n6f6by8Bdp6EIIIYxe3vUJzP6ppKCA/fPno/+XR7Ga29pi848mX1Hz/+c2FnZ25e6ury7HEhPZNWMGnZOTq7SfNHQhhBBGz9HFhbx/zGQK4Ojqyrhff0Wbn0/xpUuG/67e8HPxpUsU//mn4ec/f/rpr20uXqTs2rVbvqeZhcVtNf4bl1vddx9W9eoZ7sj/p2OJiWwZP57SoqIq/xlIQxdCCGH0ekdG3tQIzW1t6R0ZiUqlwsrREStHR+o1b16l45ZcvVq+8f+j+d/4y0HBqVNcOHKE4kuX0Obn3/qgKhXW9etX2PSPLV16R80cpKELIYQwAX/f+LZrxgzyTp7EsZrucrewscHigQdweOCBKu1XVlLCtcuXb31F4B/Lr/zyy1+/CFy5csdZpaELIYQwCe6hobiHhtaKO/XVFhbYOjtj6+xcpf0WubqSf4v7ASojj08VQgghagmvqCjMbW3vaF85QxdCCCFqiRs/OqgqOUMXQgghahH30FDG5+RUeT9p6EIIIYQJkIYuhBBCmABp6EIIIYQJkIYuhBBCmABp6EIIIYQJMKrHpwohhBB1SVUen2o0DV0IIYQQtyaX3IUQQggTIA1dCCGEMAG1furXp556CgcHBwAefPBB5s2bp3Ciqvnhhx947733SEhIIDc3lzfffBOVSkXr1q15++23MTOr/b9T3VhDVlYWEyZMoPn1RxAGBwczePBgZQNWoqSkhOnTp3Pq1Cm0Wi0TJ07koYceMrqxqKiOJk2aGNV4lJWV8dZbb/Hrr7+iVquZN28eer3e6Maiojry8/ONaiz+dvHiRfz8/Pjiiy8wNzc3urH42411FBcXG+VY/LPfTZgwoWrjoa/FiouL9T4+PkrHuGNxcXH6oUOH6keOHKnX6/X6559/Xr9v3z69Xq/Xz5w5U79lyxYl492Wf9awcuVK/ZIlSxROVTWrVq3Sz507V6/X6/WXLl3S9+3b1yjHoqI6jG08tm7dqn/zzTf1er1ev2/fPv2ECROMciwqqsPYxkKv1+u1Wq3+hRde0Pfv319//PhxoxwLvf7mOoxxLCrqd1Udj1r9q1d2djZXr17l2WefZfTo0WRmZiodqUpcXFz46KOPDK+zsrLo1q0bAH369GHPnj1KRbtt/6zh6NGjbN++ndDQUKZPn05BQYGC6W7PwIEDefXVVw2v1Wq1UY5FRXUY23j069ePOXPmAHD69GkaNmxolGNRUR3GNhYA0dHRBAUF0ahRI8A4/42Cm+swxrGoqN9VdTxqdUO3trZm7NixLFmyhHfeeYcpU6ZQWlqqdKzbNmDAAMzN//9TDb1ej0qlAsDOzo78/Hylot22f9bQsWNH3njjDRITE2nWrBmffPKJguluj52dHfb29hQUFPDKK6/w2muvGeVYVFSHMY6Hubk54eHhzJkzhwEDBhjlWMDNdRjbWKxevRonJye8vLwMy4xxLCqqw9jGAirud1Udj1rd0Fu0aMHw4cNRqVS0aNGC+vXrc/78eaVj3bEbP/soLCzE0dFRwTR3xtvbm/bt2xt+PnbsmMKJbs8ff/zB6NGj8fHxYdiwYUY7Fv+sw1jHIzo6ms2bNzNz5kyuXbtmWG5MYwHl6+jdu7dRjUVycjJ79uwhLCwMjUZDeHg4ly5dMqw3lrGoqI4+ffoY1VhAxf3u4sWLhvW3Mx61uqGvWrWK+fPnA3D27FkKCgpwdnZWONWdc3d3Z//+/QCkp6fTpUsXhRNVDhMWngAAAYRJREFU3dixYzl8+DAAe/fuxcPDQ+FElbtw4QLPPvssU6dOZcSIEYBxjkVFdRjbeKxdu5ZFixYBYGNjg0qlon379kY3FhXV8dJLLxnVWCQmJrJ06VISEhJwc3MjOjqaPn36GN1YVFTHCy+8YFRjARX3u169elVpPGr1xDJarZZp06Zx+vRpVCoVU6ZMwdPTU+lYVfL7778zadIkVq5cya+//srMmTMpKSmhZcuWzJ07F7VarXTESt1YQ1ZWFnPmzMHCwoKGDRsyZ84c7O3tlY74r+bOnUtKSgotW7Y0LJsxYwZz5841qrGoqI7XXnuNd99912jGo6ioiGnTpnHhwgVKS0sZN24crVq1Mrq/FxXV0bRpU6P7u/G3sLAwZs2ahZmZmdGNxY3+rqO4uNjoxqKifnffffdVaTxqdUMXQgghxO2p1ZfchRBCCHF7pKELIYQQJkAauhBCCGECpKELIYQQJkAauhBC/F97dSADAAAAMMjf+h5fSQQDQgeAAaEDwIDQAWAgvJiJ7OtlI6AAAAAASUVORK5CYII=\n", 164 | "text/plain": [ 165 | "
" 166 | ] 167 | }, 168 | "metadata": {}, 169 | "output_type": "display_data" 170 | } 171 | ], 172 | "source": [ 173 | "results = []\n", 174 | "min_samples_leaf_options = [5,10,15,20,25,30,35,40,45,50]\n", 175 | "for trees in min_samples_leaf_options:\n", 176 | " model = DecisionTreeClassifier(max_depth=10, random_state=101, max_features = None, min_samples_leaf = trees)\n", 177 | " model.fit(x_train, y_train)\n", 178 | " y_pred = model.predict(x_test)\n", 179 | " accuracy = np.mean(y_test==y_pred)\n", 180 | " results.append(accuracy)\n", 181 | "\n", 182 | "plt.figure(figsize=(8,4))\n", 183 | "pd.Series(results, min_samples_leaf_options).plot(color=\"darkred\",marker=\"o\")" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": 7, 189 | "metadata": {}, 190 | "outputs": [], 191 | "source": [ 192 | "dtree = DecisionTreeClassifier(max_depth=10, random_state=101, max_features = None, min_samples_leaf = 15)\n", 193 | "dtree.fit(x_train, y_train)\n", 194 | "y_pred=dtree.predict(x_test)" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": 8, 200 | "metadata": {}, 201 | "outputs": [ 202 | { 203 | "name": "stdout", 204 | "output_type": "stream", 205 | "text": [ 206 | "[[11521 914]\n", 207 | " [ 1653 2193]]\n", 208 | "accuracy: 0.8423315521159634\n", 209 | "precision: 0.7058255551979401\n", 210 | "recall: 0.5702028081123245\n", 211 | "f1 score: 0.6308068459657701\n" 212 | ] 213 | } 214 | ], 215 | "source": [ 216 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 217 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 218 | "\n", 219 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 220 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 221 | "print (df_table)\n", 222 | "\n", 223 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 224 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 225 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 226 | "\n", 227 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 228 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 229 | "print('f1 score: ', (2*p*r)/(p+r))" 230 | ] 231 | } 232 | ], 233 | "metadata": { 234 | "kernelspec": { 235 | "display_name": "Python 3", 236 | "language": "python", 237 | "name": "python3" 238 | }, 239 | "language_info": { 240 | "codemirror_mode": { 241 | "name": "ipython", 242 | "version": 3 243 | }, 244 | "file_extension": ".py", 245 | "mimetype": "text/x-python", 246 | "name": "python", 247 | "nbconvert_exporter": "python", 248 | "pygments_lexer": "ipython3", 249 | "version": "3.6.8" 250 | } 251 | }, 252 | "nbformat": 4, 253 | "nbformat_minor": 4 254 | } 255 | -------------------------------------------------------------------------------- /05 Decision Tree/02 Decision Tree.py: -------------------------------------------------------------------------------- 1 | # Import the libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.tree import DecisionTreeClassifier 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | # Import the data 17 | df = pd.read_csv('data/00 df.csv') 18 | 19 | 20 | # Split data into Train & test 21 | train = df[df['flag']=='train'] 22 | test = df[df['flag']=='test'] 23 | 24 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 25 | 26 | y_train = train['y'] 27 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 28 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 29 | 30 | y_test = test['y'] 31 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 32 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 33 | 34 | 35 | 36 | # Decision Tree 37 | results = [] 38 | max_depth_options = [2,4,6,8,10,12,14,16,18,20] 39 | for trees in max_depth_options: 40 | model = DecisionTreeClassifier(max_depth=trees, random_state=101) 41 | model.fit(x_train, y_train) 42 | y_pred = model.predict(x_test) 43 | accuracy = np.mean(y_test==y_pred) 44 | results.append(accuracy) 45 | 46 | 47 | # Plot the data 48 | plt.figure(figsize=(8,4)) 49 | pd.Series(results, max_depth_options).plot(color="darkred",marker="o") 50 | 51 | 52 | 53 | results = [] 54 | max_features_options = ['auto',None,'sqrt',0.95,0.75,0.5,0.25,0.10] 55 | for trees in max_features_options: 56 | model = DecisionTreeClassifier(max_depth=10, random_state=101, max_features = trees) 57 | model.fit(x_train, y_train) 58 | y_pred = model.predict(x_test) 59 | accuracy = np.mean(y_test==y_pred) 60 | results.append(accuracy) 61 | 62 | plt.figure(figsize=(8,4)) 63 | pd.Series(results, max_features_options).plot(kind="bar",color="darkred",ylim=(0.7,0.9)) 64 | 65 | 66 | 67 | results = [] 68 | min_samples_leaf_options = [5,10,15,20,25,30,35,40,45,50] 69 | for trees in min_samples_leaf_options: 70 | model = DecisionTreeClassifier(max_depth=10, random_state=101, max_features = None, min_samples_leaf = trees) 71 | model.fit(x_train, y_train) 72 | y_pred = model.predict(x_test) 73 | accuracy = np.mean(y_test==y_pred) 74 | results.append(accuracy) 75 | 76 | plt.figure(figsize=(8,4)) 77 | pd.Series(results, min_samples_leaf_options).plot(color="darkred",marker="o") 78 | 79 | 80 | 81 | # DTC 82 | dtree = DecisionTreeClassifier(max_depth=10, random_state=101, max_features = None, min_samples_leaf = 15) 83 | dtree.fit(x_train, y_train) 84 | y_pred=dtree.predict(x_test) 85 | 86 | 87 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 88 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 89 | 90 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 91 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 92 | print (df_table) 93 | 94 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 95 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 96 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 97 | 98 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 99 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 100 | print('f1 score: ', (2*p*r)/(p+r)) -------------------------------------------------------------------------------- /05 Decision Tree/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /05 Decision Tree/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /06 Exploratory Data Analysis/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /06 Exploratory Data Analysis/02 Exploratory Data Analysis.py: -------------------------------------------------------------------------------- 1 | # import libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | import seaborn as sns 7 | import matplotlib.pyplot as plt 8 | %matplotlib inline 9 | plt.style.use('seaborn-whitegrid') 10 | 11 | 12 | # Import the data 13 | df = pd.read_csv('data/00 df.csv') 14 | df = df[df['flag']=='train'] 15 | # print(df.info()) 16 | 17 | 18 | # Exploratory Data Analysis & plot the data 19 | #age_bin 20 | x_chart = df.pivot_table(values=['flag'], index=['age_bin'], columns=['y'], aggfunc='count') 21 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 22 | x_chart.plot(kind="bar",stacked=True) 23 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 24 | 25 | #capital_gl_bin 26 | x_chart = df.pivot_table(values=['flag'], index=['capital_gl_bin'], columns=['y'], aggfunc='count') 27 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 28 | x_chart.plot(kind="bar",stacked=True) 29 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 30 | 31 | #education_bin 32 | x_chart = df.pivot_table(values=['flag'], index=['education_bin'], columns=['y'], aggfunc='count') 33 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 34 | x_chart.plot(kind="bar",stacked=True) 35 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 36 | 37 | 38 | #hours_per_week_bin 39 | x_chart = df.pivot_table(values=['flag'], index=['hours_per_week_bin'], columns=['y'], aggfunc='count') 40 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 41 | x_chart.plot(kind="bar",stacked=True) 42 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 43 | 44 | 45 | #msr_bin 46 | x_chart = df.pivot_table(values=['flag'], index=['msr_bin'], columns=['y'], aggfunc='count') 47 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 48 | x_chart.plot(kind="bar",stacked=True) 49 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 50 | 51 | 52 | #occupation_bin 53 | x_chart = df.pivot_table(values=['flag'], index=['occupation_bin'], columns=['y'], aggfunc='count') 54 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 55 | x_chart.plot(kind="bar",stacked=True) 56 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 57 | 58 | 59 | #race_sex_bin 60 | x_chart = df.pivot_table(values=['flag'], index=['race_sex_bin'], columns=['y'], aggfunc='count') 61 | x_chart = x_chart.apply(lambda c: c / c.sum() * 100, axis=1) 62 | x_chart.plot(kind="bar",stacked=True) 63 | plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5)) 64 | 65 | 66 | 67 | 68 | 69 | -------------------------------------------------------------------------------- /06 Exploratory Data Analysis/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /06 Exploratory Data Analysis/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /07 K-Nearest Neighbors/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /07 K-Nearest Neighbors/02 K-Neighbors Classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import math\n", 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from datetime import datetime\n", 13 | "\n", 14 | "import seaborn as sns\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "%matplotlib inline \n", 17 | "plt.style.use('seaborn-whitegrid')\n", 18 | "\n", 19 | "from sklearn.neighbors import KNeighborsClassifier\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.metrics import confusion_matrix" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Get the Data" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "df = pd.read_csv('data/00 df.csv')\n", 38 | "train = df[df['flag']=='train']\n", 39 | "test = df[df['flag']=='test']" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 3, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']\n", 49 | "\n", 50 | "y_train = train['y']\n", 51 | "x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 52 | "x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True)\n", 53 | "\n", 54 | "y_test = test['y']\n", 55 | "x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 56 | "x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# KNN" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "error_rate = []\n", 73 | "for i in range(1,51): \n", 74 | " knn = KNeighborsClassifier(n_neighbors=i)\n", 75 | " knn.fit(x_train,y_train)\n", 76 | " pred_i = knn.predict(x_test)\n", 77 | " error_rate.append(np.mean(pred_i != y_test))" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 5, 83 | "metadata": {}, 84 | "outputs": [ 85 | { 86 | "data": { 87 | "text/plain": [ 88 | "Text(0, 0.5, 'Error Rate')" 89 | ] 90 | }, 91 | "execution_count": 5, 92 | "metadata": {}, 93 | "output_type": "execute_result" 94 | }, 95 | { 96 | "data": { 97 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAEPCAYAAAC9aJYqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de1xUdf4/8NdcwXG4eCnXFTMhKdRcU765Sbjbb+VhllopCQ8RMSt3y81VsdRCIiS1rK+6eKm1vGGbGJmBZvUgLQ2LlEQgsRTNNE3zRg4DzAxnfn8Q8x1kbjDnzAVez8fDx8OZc87Mmw+X15zPOZ/PR2Y2m80gIiIivyL3dgFERETUegxwIiIiP8QAJyIi8kMMcCIiIj/EACciIvJDDHAiIiI/pPR2AUT+7Pbbb0dkZCTk8uafhVevXo2wsDCPvK9MJkNtbS20Wi0yMjJw5513Ojz2vffeg8FgQFJSkmT1OZKdnY2rV68iPT0dAGAwGDB37lxcunQJa9euRUhIiGXf5ORkxMbGYvr06c1eY/369Th48CDWrl1r933mz5+Pfv364fHHH5fmCyHyMgY4kZs2bdqErl27ev193377bWRlZSE3N9fhcSUlJejXr5/U5blEr9fjmWeeQWBgIDZs2ICAgIBm2ydNmoQVK1a0CPBt27YhLS3Nk6US+RwGOJFEiouL8fLLL0Oj0aCmpgbPPfccli1bZnn8/vvv44MPPkBOTg7kcjm6d++OhQsXom/fvpg/fz6uXbuGM2fO4K9//SueffZZh+9lMplw/vx5y9nrpUuXkJ6ejsuXL+PXX39Fr169sGLFCnz77bfYs2cPioqKEBgYiKSkJKxduxaffvopBEFAr1698OKLL6JHjx7NXj8xMRGPPfYYRo0aBQBYtmwZAGDq1KmYN28erl69CgD4y1/+glmzZrnUPtXV1fj73/+OO+64A+np6S16MQAgLi4OixcvxqFDhxAdHQ0A+Oabb2A2mxETEwNBELB48WIcOXIENTU1MJvNyMrKwtChQ5u9zu23346vvvrK8oHH+vGePXuwdu1aGI1GBAYGYt68ebjrrrtc+hqIvIkBTuSmlJSUZuETFhaG1atXAwCOHz+OwsJC9OrVC8XFxc0ef/XVV3jrrbeQm5uLrl27Yvv27ZgxYwZ27doFAKirq7P83977AsDVq1cREBCA++67D0uWLAEA7Nq1C4MHD8b06dNhNpsxffp0fPjhh5g2bRo+++wz9OvXD0lJSdixYwd++OEHvPfee1AqlcjNzUVaWhrWrVvX7L0effRRbN++HaNGjUJDQwPy8/ORk5ODbdu2ISwsDOvXr4der8cLL7yA69evIygoyGGbXbp0CcnJyTh79iyys7NthjcAKJVKTJw4EXl5eZYAz83NxaRJkyCTyVBaWoqLFy8iNzcXcrkc//nPf7Bu3boWAW7Pjz/+iOXLl2Pz5s3o0qULjh8/jsceewyffvopNBqNS69B5C0McCI3OepC79mzJ3r16mXz8f79+/HAAw9Yjh0/fjxefvllnD17FgCchlDT+3733XeYPn06hg0bhm7dugFoDPdDhw5hw4YN+PHHH3H8+HH86U9/avEae/fuRXl5OSZMmAAAEAQBtbW1LfZ74IEH8Oqrr+LXX3/F0aNHceutt+LWW2+1XJ8+f/48hg8fjtTUVKfhDQCFhYXIyMhAeXk5Zs2ahY0bN0KlUtncd+LEiXjwwQeh0+lgMpnw5ZdfIiMjAwBw1113ISQkBFu3bsWZM2dQXFyMzp07O33/JkVFRbh48SKmTp1qeU4mk+Gnn37CHXfc4fLrEHkDA5xIQjeexVk/FgShxf5msxkmk8nmsfYMGDAACxYswPz58xEVFYWwsDAsW7YMZWVlmDBhAoYNGwaTyQRbyx4IgoAnnngCkyZNAtB4Q1l1dXWL/Tp16oRRo0Zh586dOHz4MB599FEAwKBBg/DZZ5/hq6++wtdff41HH30U69atw8CBAx3WPH78eEycOBEPPfQQJk6ciFdeecXuNe0ePXpg+PDh+Oijj6DX6zFq1CjLh4TPP/8cL7/8Mh577DH87W9/Q3h4OPLz8x2+t8FgaPb133PPPVixYoXlufPnz+Pmm292+BpEvoDDyIi8JDY2Fh999BGuXLkCAHj//fcRGhqKPn36tPq1xowZg0GDBlm60L/88kukpKTg4YcfRrdu3XDgwAE0NDQAABQKheVDwr333ou8vDzodDoAwMqVK/Hcc8/ZfI+JEyfigw8+wLfffmu5Fv7aa69hzZo1GDlyJF544QXcdtttOH78uNN61Wo1ACAgIAArV67EBx98gA8//NDu/klJSSgoKMCOHTua3T1fVFSE++67D5MmTcLAgQNRWFho+Tqtde3aFeXl5QCAnTt3Wp6/5557UFRUhKqqKgDAF198gXHjxqGurs7p10DkbTwDJ3LTjdfAAWDOnDkIDAx0eFxMTAymTp2KlJQUCIKArl274s0337R7PdiZhQsXYty4cdi/fz9mzJiBV199FStXroRKpcKQIUPw008/AQBGjBiBpUuXAgCefPJJXLhwARMnToRMJkPPnj0t2240cOBAKBQK3H///Za7xVNSUjB//nyMGTMGarUat99+Ox588EEAwEMPPYSsrCynw9puvfVWZGVlYcGCBYiMjERUVFSLfYYNG4asrCyEhITg9ttvtzyfmJiI1NRUjB07FiaTCTExMZYb8qylpaUhMzMTwcHBGD58OG666SYAwG233YbMzEzMmTMHZrMZSqUSa9eubVU3PJG3yLicKBERkf9hFzoREZEfYoATERH5IQY4ERGRH2KAExER+SEGOBERkR/yq2FkJSUl3i6BiIjI42zNzChJgAuCgIyMDHz//fdQq9XIyspqNjnFzp07sWnTJigUCkRGRiIjIwMNDQ14/vnn8fPPP8NgMOCpp57C3/72N5e+CHsqKyttjiml1mNbiodtKR62pXjYluKQoh3tnbxKEuCFhYUwGAzIzc1FaWkpli5dalm3t66uDitWrEBBQQE6deqEOXPmYO/evbh27RpCQ0OxbNkyXL16FY888ojNACciIiKJArykpASxsbEAgMGDB6OiosKyTa1WY+vWrejUqROAxmUQAwICcP/991umZwQap3skIiIi2yQJcJ1OB61Wa3ncNPeyUqm0rHsMADk5OdDr9YiJiYFMJrMcO3PmTJfXFCYiIuqIJAlwrVaLmpoay2NBEKBUKps9XrZsGU6dOoXs7GxLeJ8/fx4zZszApEmTMHbsWJuvXVlZ6XIddXV1Nvev+eknnNy4EecKCmDS66HUaPDHsWMRPnUqOt9yi8uv35HYa0tqPbaleNiW4mFbisOT7ShJgA8ZMgR79+7FAw88gNLSUkRGRjbbnp6eDrVajTVr1lgWbrh06RKmTZuG9PR03HPPPXZfuzU3B9i6meDk7t34JD4egtEIwWgEAJhqanD2/fdxLj8f4/LyED56tMvv0VHwBhfxsC3Fw7YUD9tSHH5/E1tcXByKioqQmJgIs9mMxYsXo6CgAHq9HgMHDkReXh6io6ORkpICAJgyZQqKi4vx22+/Yc2aNVizZg0AYN26dU5XdGqNa1VVyI+Ph0mvb7GtKdDz4+MxtawMoRERor0vERGR2CQJcLlcjszMzGbPRVgF4rFjx1ocExcXh7S0NCnKsTj0+uuWs257BKMRh5Yvx8hVqySthYiIyB0daia2o1u2uBTgR3NyPFQRERFR23SoADfodKLuR0RE5C0dKsDVVkPbxNiPiIjIWzpUgPefPBlylcrhPnKVCv2Tkz1UERERUdt0qACPTk11KcCjZ8/2UEVERERt06ECPDQiAuPy8qDUaFoEuVylglKjwbi8PA4hIyIin9ehAhwAwkePxtSyMgyaPh2KgAAAgDo4GIOmT8fUsjJO4kJERH7Br9YDF0toRARGrlqFHkOH4pNp0zDl8GGEhod7uywiIiKXdbgzcGvBvXsDAK6fOePlSoiIiFqnQwd4EAOciIj8FAMcDHAiIvI/HTrAVRoNArt2xW8McCIi8jMdOsCBxrNwnoETEZG/YYAzwImIyA8xwBngRETkhzp8gAf37o26K1dg1Ou9XQoREZHLOnyA8050IiLyRwxwBjgREfkhBvjvAc6hZERE5E86fIBre/UCwDNwIiLyLx0+wJUBAdD06MEAJyIiv9LhAxzgUDIiIvI/DHA0DiVjgBMRkT9hgINn4ERE5H8Y4GgMcMP166ivrvZ2KURERC5hgINDyYiIyP8wwAEEhYUB4FAyIiLyHwxwcDY2IiLyP5IEuCAISE9PR0JCApKTk3H69Olm23fu3IlHH30UiYmJSE9PhyAIlm1HjhxBcnKyFGXZpf3jHyGTyxngRETkNyQJ8MLCQhgMBuTm5iI1NRVLly61bKurq8OKFSuwefNmbN26FTqdDnv37gUArFu3Dmlpaaivr5eiLLvkSiU69+zJACciIr8hSYCXlJQgNjYWADB48GBUVFRYtqnVamzduhWdOnUCAJhMJgQEBAAAbrnlFmRnZ0tRklMcSkZERP5EKcWL6nQ6aLVay2OFQgGTyQSlUgm5XI7u3bsDAHJycqDX6xETEwMAGDVqFM6ePevwtSsrK12uo66uzvX9Q0Jw+dixVr1+R9KqtiSH2JbiYVuKh20pDk+2oyQBrtVqUVNTY3ksCAKUSmWzx8uWLcOpU6eQnZ0NmUzm8mtHRUW5vG9lZaXL+/8yYAB+/eIL3HHHHa2qp6NoTVuSY2xL8bAtxcO2FIcU7VhSUmLzeUm60IcMGYJ9+/YBAEpLSxEZGdlse3p6Ourr67FmzRpLV7q3BffuDVNdHWovX/Z2KURERE5JcgYeFxeHoqIiJCYmwmw2Y/HixSgoKIBer8fAgQORl5eH6OhopKSkAACmTJmCuLg4KUpxWdNQMt3Zs9D83sVPRETkqyQJcLlcjszMzGbPRUREWP5/7Ngxu8eGhYVh27ZtUpTlkPVY8JsHD/b4+xMREbUGJ3L5HadTJSIif8IA/13nHj0gV6k4lIyIiPwCA/x3Mrkc2l69GOBEROQXGOBWgjmZCxER+QkGuBXOxkZERP6CAW4lqHdvXD97FmarxVWIiIh8EQPcSlDv3hCMRugvXvR2KURERA4xwK1wKBkREfkLBrgV68lciIiIfBkD3AoDnIiI/AUD3Eqnbt2gDAxkgBMRkc9jgFuRyWQcSkZERH6BAX4DBjgREfkDBvgNGOBEROQPGOA3COrdG7pz5yCYTN4uhYiIyC4G+A2CeveGWRCgO3/e26UQERHZxQC/QVBYGAAOJSMiIt/GAL8Bx4ITEZE/YIDfgAFORET+gAF+g4CQEKi0WgY4ERH5NAb4DTiZCxER+QMGuA3BvXtzRTIiIvJpDHAbeAZORES+jgFuQ1Dv3tBfuABTfb23SyEiIrKJAW5D053oup9/9nIlREREtjHAbeBQMiIi8nUMcBsY4ERE5OuUUryoIAjIyMjA999/D7VajaysLPTp08eyfefOndi0aRMUCgUiIyORkZEBAA6P8SQGOBER+TpJzsALCwthMBiQm5uL1NRULF261LKtrq4OK1aswObNm7F161bodDrs3bvX4TGepu7cGYFdunAoGRER+SxJArykpASxsbEAgMGDB6OiosKyTa1WY+vWrejUqRMAwGQyISAgwOEx3sChZERE5Msk6ULX6XTQarWWxwqFAiaTCUqlEnK5HN27dwcA5OTkQK/XIyYmBrt377Z7jLXKykqX66irq2vV/tZkXbrg0vHjbT6+vXGnLak5tqV42JbiYVuKw5PtKEmAa7Va1NTUWB4LgtAsiAVBwLJly3Dq1ClkZ2dDJpM5PaZJVFSUy3VUVla2an9rZ6Oi8H15eZuPb2/caUtqjm0pHraleNiW4pCiHUtKSmw+L0kX+pAhQ7Bv3z4AQGlpKSIjI5ttT09PR319PdasWWPpSnd2jKcF9e6NuitXYNTrvVoHERGRLZKcgcfFxaGoqAiJiYkwm81YvHgxCgoKoNfrMXDgQOTl5SE6OhopKSkAgClTptg8xluuVVXh9GefAQBWarVQa7XoP3kyolNTERoR4bW6iIiImkgS4HK5HJmZmc2ei7AKvmPHjtk87sZjvOHk7t3Ij49HQ9M0qmYzDNevo+ytt1CxaRPG5eUhfPRo7xZJREQdHidysXKtqgr58fEw6fUwNzQ02yYYjTDp9ciPj8e1qiovVUhERNSIAW7l0OuvQzAaHe4jGI04tHy5hyoiIiKyjQFu5eiWLS4F+NGcHA9VREREZBsD3IpBpxN1PyIiIqkwwK2orSaSEWM/IiIiqTDArfSfPBlylcrhPnKVCv2Tkz1UERERkW0McCvRqakuBXj07NkeqoiIiMg2pwF+4cIFzJ07F48//ji2bduGI0eOeKIurwiNiMC4vDwoNZoWQS5XqaDUaDAuL4+TuRARkdc5DfCFCxdiwoQJMBgMiI6Oxssvv+yJurwmfPRoTC0rw6Dp06EODm58Ui7HoOnTMbWsjJO4EBGRT3Aa4PX19bjnnnsgk8kQHh6OgIAAT9TlVaERERi5ahVmVlcjdskSQBAQ89JLPPMmIiKf4TTA1Wo19u/fD0EQUFpaCrVa7Ym6fEbPYcMAAL8cPOjlSoiIiP6P0wBftGgRtm/fjqtXr2L9+vV46aWXPFGXz+gxdCggk+F8cbG3SyEiIrJwupjJ/v37sdxq6tDNmzdjypQpkhblSwKCg9Gtf38GOBER+RS7Ab5z507s2bMHxcXF+PrrrwEAgiDghx9+6FABDjR2o1d9+CHMZjNkMpm3yyEiIrIf4LGxsbjppptw7do1JCQkAGhcJrR3794eK85X9Bw2DBXr1+NaVRW63Habt8shIiKyH+AhISEYNmwYhg0bhosXL8JkMsFsNuPcuXPo0aOHJ2v0uqYb2c4XFzPAiYjIJzi9Bv7888+jtLQUtbW1qK2txS233IJt27Z5ojaf0X3AACg1GvzyzTfon5Tk7XKIiIic34V+8uRJ7Nq1C/feey8++uijDjEO/EZypRJ/GDqUN7IREZHPcBrgnTt3hkwmg16vR9euXWF0sl52e/WHYcNw8fBhmOrrvV0KERGR8wAfMGAA3n77bdx8882YPXs2TCaTJ+ryOT2HDUODwYBf2/Fc8ERE5D+cXgOfM2cOdDodAgMDsW/fPtx5552eqMvnWN/I1vPuu71cDRERdXR2z8CvX7+ODRs24P3334dGo4FSqcQf//hHPPPMM56sz2cEhYWhc8+e+OWbb7xdChERkf0z8H/9618YOHAgjh49ivPnz6N79+5YtWoV5s2b58n6fIZMJkPPYcN4IxsREfkEuwFeU1ODOXPmwGw24/7770evXr3w4Ycfolu3bp6sz6f0vPtunNixA7VXrqBT167eLoeIiDowu13oTauOyWQyBAQEYO3atR06vAGrlcnYjU5ERF5mN8Ct5/wODQ3tkOO/b9QjOporkxERkU+w24X+3XffITExEWazGSdOnLD8XyaTYevWrZ6s0WdwZTIiIvIVdgM8Pz/fk3X4Da5MRkREvsBuF3qvXr3s/nNGEASkp6cjISEBycnJOH36dIt9amtrkZiYiKqqKgCAwWBAamoqJk6ciGnTpuHHH39s+1cloZ53343ay5dRffKkt0shIqIOzOlMbG1RWFgIg8GA3NxcpKamYunSpc22l5eXIykpCWfOnLE8t23bNmg0Gmzbtg1paWlYtGiRFKW5zXpCFyIiIm9xGuBff/11q1+0pKQEsbGxAIDBgwejoqKi2XaDwYDVq1cjPDzc8tyJEycwYsQIAEB4eLjlzNzXdB84EEqNhgFORERe5XQq1ezsbPz5z39u1YvqdDpotVrLY4VCAZPJBKWy8e2GDh3a4pioqCjs3bsXI0eOxJEjR3DhwgU0NDRAoVA026+ystLlOurq6lq1v6uCo6Jw8vPP0VOC1/ZVUrVlR8S2FA/bUjxsS3F4sh2dBrhMJsOMGTPQt29fyOWNJ+xz5sxxeIxWq0VNTY3lsSAIlvC2Z8KECaiqqsKUKVMwZMgQDBgwoEV4A41B76rKyspW7e+qC/fdh8P//jciIyKg+H28fHsnVVt2RGxL8bAtxcO2FIcU7VhSUmLzeacBPmHChFa/2ZAhQ7B371488MADKC0tRWRkpNNjysvLMXToUDz//PMoLy/HTz/91Or39ZSew4bhkMGAi0eOoOf//I+3yyEiog7I6TXwsWPHQq/Xo6ysDL/99hsefPBBpy8aFxcHtVqNxMRELFmyBAsWLEBBQQFyc3PtHtOnTx+8++67SEhIwMqVKzF//vzWfSUe1LQaGa+DExGRtzg9A09PT0dwcDBiYmLwzTffIC0tDa+++qrDY+RyOTIzM5s9FxER0WK/nJwcy/+7du2KjRs3uli2dwX17o3Of/gDfikuBv75T2+XQ0REHZDTAD99+jTeeecdAMDIkSORmJgoeVG+jiuTERGRtzntQq+vr0dtbS2AxrvrGhoaJC/K112rqkLNL7/g6vHjeE0ux7+Dg1H49NO45qND34iIqP1xegaekpKChx56CP369cOJEycwc+ZMT9Tls07u3o38+Hg01Nc3PmE2w3D9OsreegsVmzZhXF4ewkeP9m6RRETU7jkN8Jtuugnbtm3DmTNnEBYWhi5duniiLp90raoK+fHxMOn1LbYJRiMEoxH58fGYWlaGUBvX/ImIiMTitAs9OzsboaGhuPPOOzt0eAPAoddfh2A0OtxHMBpxaPlyD1VEREQdlSQTubRXR7dscSnAj+bkYOSqVR6qioiIOiKnAf7www/bnBGtIzLodKLuR0RE1FZOA/yjjz7C+vXrPVGLz1NrtTBcv+7SfkRERFJyeg08KCgIn332GaqqqnDq1CmcOnXKE3X5pP6TJ0OuUjncR65SoX9ysocqIiKijsrpGfiVK1eazZAmk8mwefNmKWvyWdGpqajYtMnhdXC5SoXo2bM9WBUREXVETgPcerpToHFil44qNCIC4/LykB8fbxk2Zk2uVGJcXh6HkBERkeTsdqHPmjXL8n/ra+BPPvmktBX5uPDRozG1rAyDpk+HOjgYkMuhDg5GtwEDYAbQjcvxERGRB9gN8MuXL1v+//nnn1v+bzabJS3IH4RGRGDkqlWYWV2NuQ0NmFldjfhPPoFcocCBl17ydnlERNQBOL2JDWge2jKZTLJi/FlQr16465//xNHNm3H52DFvl0NERO2c3QC3DmqGtmvunj8fSo0GRenp3i6FiIjaObs3sZ04cQKpqakwm83N/l/FFbfs0nTvjug5c/BVZiYuHD6MHnfd5e2SiIionbIb4CtWrLD833oNcK4H7lj0nDkoWbkSOx5+GPVXr8Kg00Gt1aL/5MmITk3lHepERCQKuwF+9913e7KOduPnAwdg0uthqK62PMflRomISGwu3cRGrmlabtTWRC+C0QiTXo/8+Hhc42UIIiJyEwNcRFxulIiIPIUBLqLWLDdKRETkDga4iLjcKBEReQoDXESuLiPK5UaJiMhdDHARcblRIiLyFAa4iKJTU10KcC43SkRE7mKAi6hpuVGlRtMyyOVyKDUaLjdKRESiYICLzNZyozK5HJ179sTUsjJO4kJERKKQJMAFQUB6ejoSEhKQnJyM06dPt9intrYWiYmJlrnVjUYjUlNTkZiYiEmTJvn1nOs3Ljf6P889h9oLFxDYrZu3SyMionZCkgAvLCyEwWBAbm4uUlNTsXTp0mbby8vLkZSUhDNnzlie++KLL2AymbB161bMmDGj2Vzs/i5i7FgIJhN+/PRTb5dCRETthCQBXlJSgtjYWADA4MGDUVFR0Wy7wWDA6tWrER4ebnmub9++aGhogCAI0Ol0UCrtTtPud3oOG4ZO3brh5M6d3i6FiIjaCUlSUqfTQWs11lmhUMBkMllCeejQoS2O0Wg0+PnnnzF69GhcvXoVb7zxhs3XrqysdLmOurq6Vu0vpa4xMTheUICjFRWQKRTeLqfVfKkt/R3bUjxsS/GwLcXhyXaUJMC1Wi1qamosjwVBcHpGvXHjRtx7771ITU3F+fPnkZKSgoKCAgQEBDTbLyoqyuU6KisrW7W/lGRJSdiZn4+Q6mr0ionxdjmt5ktt6e/YluJhW4qHbSkOKdqxpKTE5vOSdKEPGTIE+/btAwCUlpYiMjLS6THBwcEICgoCAISEhMBkMqGhoUGK8ryi76hRkCuVqGI3OhERiUCSAI+Li4NarUZiYiKWLFmCBQsWoKCgALm5uXaPmTp1Kr777jtMmjQJKSkpmD17NjQajRTleUVASAjCRozgdXAiIhKFJF3ocrkcmZmZzZ6LsDF5SY7VqlydO3fGypUrpSjHZ4SPGYPP58xB9Y8/IuTWW71dDhER+TFO5OJBEWPGAAC70YmIyG0McA/q0q8fukRGshudiIjcxgD3sIixY3Fm714Yrl/3dilEROTHGOAeFj5mDBoMBpwuLPR2KURE5McY4B7WKyYGASEhvA5ORERuYYB7mEKlwq3334+Tu3bBLAjeLoeIiPwUA9wLIsaMgf7CBfxy6JC3SyEiIj/FAPeCvqNHQyaXsxudiIjajAHuBZ26dcMfhw/ncDIiImozBriX9LznHlw8fBgrg4LwmlyOfwcHo/Dpp3GtqgoAcK2qCoVPP41/Bwfb3E5ERB0bA9wLTu7ejdLsbACAUacDzGYYrl9H2VtvYeOgQTiQmYmNgwah7K23GseL37D95O7dXv4KiIjI2xjgHnatqgr58fEw1dW12CYYjTDp9Tjw4osw6fUQjEab2/Pj43kmTkTUwUmymAnZd+j111sEc2sJRiOKMjIQEBSEo1u2wKDTQa3Vov/kyYhOTUWojYVjiIiofeEZuIcd3bJFlACv3LLFaRc7r6MTEbVfDHAPM+h0or2Woy728vXreR2diKgdY4B7mFqrlfw9GgwGfPr3v/M6OhFRO8YA97D+kydDrlJJ+h5mkwlmk8nhPoLRiEPLl0taBxERSYcB7mHRqamSB7grBKMRR3NyvF0GERG1EQPcw0IjIjAuLw9KjaZFkMtVKig1Ggx/6SW728Uk5vV4IiLyLAa4F4SPHo2pZWUYNH061MHBgCaOwKgAABIpSURBVFwOdXAwBk2fjqllZRienm53e1RSkmhB7onr8UREJA2OA/eS0IgIjFy1CiNXrWrV9mtVVTj+wQcOh6LJFApAJnN4HVyuUqF/cnLbiiciIq/jGbifcaULPu4//4FCrXb8QjIZIseP5zhxIiI/xQD3Q8664AdNm+Yw5GUKBQSDAXmjRzscJ249EcyuAQMY8EREPoRd6H7KWRd8U8gfWr4cR3Ny/m+61eRkRIwdi+0PPgjBYGhxnGA0QjAaseORRyCXyyGYTJbu+qaAr9i0CePy8hA+erSkXyMREdnHAG/H7IV84dNPQyaXw9zQYPdYob4egq3nfw/4/Ph4TC0r47zrRERewi70Dkis+dg5EQwRkfcwwDsgMcZ/cyIYIiLvkiTABUFAeno6EhISkJycjNOnT7fYp7a2FomJiaj6/Yao7du3Izk5GcnJyZg4cSLuvPNO/Pbbb1KU1+GJNf6bE8EQEXmPJAFeWFgIg8GA3NxcpKamYunSpc22l5eXIykpCWfOnLE8N378eOTk5CAnJwcDBgxAWloagoODpSivwxNrPnZOBENE5D2SBHhJSQliY2MBAIMHD0ZFRUWz7QaDAatXr0Z4eHiLY8vLy3HixAkkJCRIURpBnPnYZQoFIsaN4zhyIiIvkeQudJ1OB63V2ZlCoYDJZIJS2fh2Q4cOtXvsm2++iRkzZtjdXllZ6XIddXV1rdq/I7nrf/8X386aBeGGlctkSiVk8sbPdbaGmTUxNzTg2LvvNpvxzXD9Oo6sW4fyDRswZMUK3DxihLRfhJ/iz6V42JbiYVuKw5PtKEmAa7Va1NTUWB4LgmAJb0d+++03nDx5En/+85/t7hMVFeVyHZWVla3avyOJiorCn0aOtDlOPHr2bFz54Qfkx8dbho01sUwEYzTaHIZmNpnQYDLh8Jw5HGZmB38uxcO2FA/bUhxStGNJSYnN5yXpQh8yZAj27dsHACgtLUVkZKRLxx08eBDDhw+XoiSyoWmc+MzqasxtaMDM6mqMXLUKoRERLWd7k8kss71FTphgOUu3h8PMiIikJckZeFxcHIqKipCYmAiz2YzFixejoKAAer3e4bXtU6dOISwsTIqSqA2sJ4Kx/lT57+Bgp+PIm4aZ2ZspjoiI3CNJgMvlcmRmZjZ7LsJGV2rODeOIn3jiCSnKIZG5OnzMoNPhWlUVDr3+Oo5u2fJ/3fSTJyM6NZXd60REbuBELtRqrg4fUwYEYOOgQQ4XTCEicof1oksdbTQMA5xazdVx5KbaWpj0+hbd7YLRCJNej/z4+A7xS0ZE0ji5e7dHThJ89UMCA5xazZVx5DKFwunrCEYjijIyfPIXw5qv/PK6W4cYX4evtAWJwxeWDG7rz9S1qirkx8dLfpLgyocEb7WjzGw2myV9BxGVlJQ4HEN+Iw6LEM+NbXly9267w8zkKhVgNsNUW+vSa8tVKpuv4QtLljr7OttSY1t+Lt2tQ4yvQ4q2cBd/x9vOF76f7tRQ+PTTKHvrLYc31MpVKgyaPr3NN9Neq6rCxkGDYNLr7b9HQECLpZdd/RpcZS/7eAZObdJimJlcbhlmNrWsDKa6OpdfS+oudl//hO+Mu3WI8XX4SluQOHzh++luDa6sqtg0GsbZ3wB724tefNH5e9TXw1Rb65V2ZIBTmzkaRy7GPOlijCVvbfdXq395RRrv7ugPzKHXX3erDnePF+s1yHe4+v10donLnUsq7v5MuTwa5vp1h38DDmRm2t1e+c47Pr30MrvQySWtbUtXurdcoQ4Oxszq6jYd6273l6u1t7ZGqS5H2Kvj38HBjX+Y2ni8WK8hhY78O+7OEE1Xv5+A/Utcd8+bh29eecVh93fXyEi7NW6+6y6XalBptRiQnNziNb7bvBlGqxk/fZm7vxf2sk+SceBE0ampqNi0ye0Ad2fJUpc+4dfXQ7D1fCvqdqdG625EWzWIUUdrxu23ZVtb9iP32PrQ13TWWLFpk9PwbM33yVbXsGA04sCLL9rdXzAaseORR1p8OLau0dV7ZIw6XbOTgabXEKzWcPB1Uv1esAudJBEaEYFxeXlQajQt7lhvzUpoKo2mzV10rlwjE4M7lwtc+ZDhbh2u1udoPzFeAxD/Tnhv3TntTa5cO97xyCPYeOedtruNBw4EZDLJ63R2bRit6Py19RqtOd7bpFp6mQFOknF0o1tUUpJrY8nr6to8xtMTZ4MyhQL9k5NbdZOMdeh8t3mzKAEuV6nQPznZ5raopCSnf7Blcrnd5WGvHj+O0Ntuc6sGQJwxu54a9+vLXO1ZshuedXWAIABO1jPwBzKFAjKl0uZJglKj8cgHFWec/V64g9fAySVit6Ur16edUWo0mFpWBgA2uwo9dY0savJkHN++3e61QEfXCsU6+1YEBmLCrl34IS+vWTtEJSWh+vRp/OhCsMmUSshkshY1mgUB5oYGyJTKZkvP2jp+wu7dOLF9e4vvRWR8PLaPHevw+930/bR3/daVnxlnr+ErPHX92i6FAjKrpYD9mUqrxYCUFJurKrp6nd0Vtu4FkCkUgNmMhvp6u8eJ8TNpL/sY4OQSKdrS0c1bZkGA2WxuPFOwQ65S4dZRo/DTnj0tXsNZ2LSWvRt5QiMicKm8XLT3sUfZqRPwe7i2+DobGiBXqxv/IDc0NP9QIJcDgoBef/kLLhw82OrlYa33G7l2LfbMnGnzNYDGszuZQgGZXG7z+wnA6Xs4GrMr1rhfb8/P7+rYZ3t1lr7xhijdx/Z+psT8YOmUTAZlp07u1SCXY66dnysxbqaVq1S4PSEBASEhrV56mePAqd1y1MWuDAx0GN5AY2Cc3LnT5rVAscJbrlIhavJku+Pde8XESN4VKVepMHDaNJtt9ae//x0PbtkCwWBAQ11dyz9Uv7fhhYMHMb6gwObX4crysABw8fBhu9+v+I8/bgzqGz9AAJYPB47Cu2k/R2N2Xbnc0PQa9ojVBS/13ALl69fbrVOsa7+m+nq7Swa7eonLXeqgILs/U6rOnV17DQfXl12ZNdIZuUqFmIyMNi29PLWsTNLJcHgGTi7xdFu+JpeL8odKplAAMpnNrmExur9E6c50wlkNhU8/jSPr1jn80OLozFSMIWJiDRuU8oxMrC54qWcPkykbBwdJ3b194/fT+ndcjEtczniqx8XZ98uV4XCtCWEp/lbyDJz8ilh3bZobGqAMDLT5Cf+xigo89MEHdu+UV2o0GJeX5/CPuZg3yrW1hqNbtjj9Y+/ozFSMIWKi3fFvNts9O3WVvZELYkzM4+oZ9E979rS5F8FsMkke3s5urHI2ikSp0WD4Sy/Z3a4IDIQiIMBpDdGzZ9vd7srZs7PXAJzPGjk8Pd3hdm9P5+wIz8DJJZ5uS9HO6ACH18iA36+JLl9u8/qWR24oguMbcZzV4HJvhZ12EOMMXKweE7fJ5ZDJ5TZ7XMSYmMelM2gHvT5iX1sW88YqW7/jzn43HG0X49qwL8zX3lqePANngJNLPN2WYnbhSTk7mFg3ybiz4IK7ASxGV6UnLiV4jFyOJ374wasjG1wik2Hw00+LFp5S/I678+FYzNfwJAa4HQxw7/FGWzr79N37vvtw+tNP3b5G5g4xh8O19Y+RuwEsxrVhd89M5SpV48xcLv45snXmaRaExrvd3fyT5hN3Z7vA2QfT1gYf/16Kg9fAieD82tXfVq4U5RqZO9y9VujKNW5n3L1W6MrX4KxGV2pQBATg0U8+sfv9dPW+B5VWa/OOX2VgoCjd+A319W5fh3dH0+QkjrgyOYijxYaofeAZOLnEV9vSV66Rtepa4fXrUAcFidoNKEY7uNtV6W4NbelJsP659Jnr8C6yN7fA/8vOxp5nnvH4hDW++jvub9iFbgcD3Ht8uS15jayRL7SDOzW0pSvfui3FWGHL1ZXf3OXspkVvfDD15d9xf8IAt4MB7j1sS/GwLe1rbXBZt6WrZ/COZtV6q18/t6/DA+7NONfE0x/I+HMpDk8GOJcTJSKf0XTfQ1uCy5UlbJtm1Wq6PnwjtVbr+hrVNs6gI8ePdzrnu6v3ZTTVKNUNmOT/GOBE5FPaGlxNN+M5O4N39CGg/+TJLp3FD0hJsVujuzUQuYp3oRNRu+Fs5IKz68ZizP7lbg1EruIZOBG1K+50PYtxFu9uDUSu4hk4EZEVnkGTv+AZOBHRDXgGTf5AkjNwQRCQnp6OhIQEJCcn4/Tp0y32qa2tRWJiIqqs1s998803kZCQgPHjx+O9996TojQiIqJ2QZIALywshMFgQG5uLlJTU7F06dJm28vLy5GUlIQzZ85YnisuLsbhw4fx7rvvIicnB7/88osUpREREbULkgR4SUkJYmNjAQCDBw9GRUVFs+0GgwGrV69GeHi45bkvv/wSkZGRmDFjBv7xj3/gr3/9qxSlERERtQuSXAPX6XTQWi1MoFAoYDKZoPx9gn5bM8pcvXoV586dwxtvvIGzZ8/iqaeewscffwyZTNZsv5KSklbV0tr9yT62pXjYluJhW4qHbSkOT7WjJAGu1WpRY7VmriAIlvC2JzQ0FOHh4VCr1QgPD0dAQACuXLmCbt26WfZpzTSqRERE7ZkkXehDhgzBvn37AAClpaWIjIx0eszQoUOxf/9+mM1mXLhwAbW1tQgNDZWiPCIiIr8nyRl4XFwcioqKkJiYCLPZjMWLF6OgoAB6vR4JCQk2j7nvvvtw8OBBxMfHw2w2Iz09HQqFQoryiIiI/J5frUbmKkEQkJGRge+//x5qtRpZWVno06ePt8vyO0eOHMFrr72GnJwcnD59GvPnz4dMJkO/fv3w4osvQi7nPEDOGI1GPP/88/j5559hMBjw1FNP4bbbbmNbtkFDQwPS0tJw6tQpKBQKLFmyBGazmW3ZRpcvX8b48eOxfv16KJVKtmMbPfzwwwgKCgIAhIWF4R//+IfH2rJdfoecDWMj59atW4e0tDTU19cDAJYsWYJZs2bhv//9L8xmMz777DMvV+gf8vPzERoaiv/+979Yt24dFi1axLZso7179wIAtm7dipkzZ2LJkiVsyzYyGo1IT09HYGAgAP5+t1XT38ecnBzk5OR4/GeyXQa4s2Fs5Nwtt9yC7Oxsy+PvvvsOd999NwBgxIgROHDggLdK8yv3338//vWvf1keKxQKtmUbjRw5EosWLQIAnDt3Dt27d2dbttErr7yCxMRE3HzzzQD4+91Wx44dQ21tLaZNm4YpU6agtLTUo23ZLgPc3jA2ct2oUaOajRwwm82WIX2dO3fGdRfWTKbGttJqtdDpdJg5cyZmzZrFtnSDUqnEvHnzsGjRIowaNYpt2Qbbt29H165dLSc5AH+/2yowMBCPP/443n77bbz00kuYO3euR9uyXQZ4W4axkWPW13BqamoQHBzsxWr8y/nz5zFlyhQ89NBDGDt2LNvSTa+88go++eQTLFy40NKFCbAtXfX+++/jwIEDSE5ORmVlJebNm4crV65YtrMdXde3b1+MGzcOMpkMffv2RWhoKC5fvmzZLnVbtssAb8swNnKsf//+KC4uBgDs27cP0dHRXq7IP1y6dAnTpk3Ds88+i/j4eABsy7basWMH3nzzTQBAp06dIJPJMHDgQLZlK73zzjvYsmULcnJyEBUVhVdeeQUjRoxgO7ZBXl6e5R6rCxcuQKfTISYmxmNt2a7vQv/hhx8sw9ginKzfSy2dPXsWc+bMwbZt23Dq1CksXLgQRqMR4eHhyMrK4jA/F2RlZWH37t3Npg1+4YUXkJWVxbZsJb1ejwULFuDSpUswmUx48sknERERwZ9LNyQnJyMjIwNyuZzt2AYGgwELFizAuXPnIJPJMHfuXHTp0sVjbdkuA5yIiKi9a5dd6ERERO0dA5yIiMgPMcCJiIj8EAOciIjIDzHAiYiI/BADnIjsKi4uxuzZsy2PP/74Y4wZMwbnzp3zYlVEBEi0nCgRtT+7du3C22+/jY0bN6J79+7eLoeow2OAE5FTO3bswJYtW7BhwwaEhIR4uxwiAgOciJw4dOgQLly4gOrqajQ0NHi7HCL6Ha+BE5FDN910EzZs2ICUlBQ8++yzEATB2yURERjgROREnz59EBAQgMmTJ0OlUmHt2rXeLomIwAAnolZYvHgxcnNz8fXXX3u7FKIOj4uZEBER+SGegRMREfkhBjgREZEfYoATERH5IQY4ERGRH2KAExER+SEGOBERkR9igBMREfkhBjgREZEf+v9x5p5wvmUrjgAAAABJRU5ErkJggg==\n", 98 | "text/plain": [ 99 | "
" 100 | ] 101 | }, 102 | "metadata": {}, 103 | "output_type": "display_data" 104 | } 105 | ], 106 | "source": [ 107 | "plt.figure(figsize=(8,4))\n", 108 | "plt.plot(range(1,51),error_rate,color='darkred', marker='o',markersize=10)\n", 109 | "plt.title('Error Rate vs. K Value')\n", 110 | "plt.xlabel('K')\n", 111 | "plt.ylabel('Error Rate')" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": 6, 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "knn = KNeighborsClassifier(n_neighbors=15)\n", 121 | "knn.fit(x_train,y_train)\n", 122 | "y_pred=knn.predict(x_test)" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": 7, 128 | "metadata": {}, 129 | "outputs": [ 130 | { 131 | "name": "stdout", 132 | "output_type": "stream", 133 | "text": [ 134 | "[[11660 775]\n", 135 | " [ 1901 1945]]\n", 136 | "accuracy: 0.8356366316565321\n", 137 | "precision: 0.7150735294117647\n", 138 | "recall: 0.5057202288091524\n", 139 | "f1 score: 0.5924459335973195\n" 140 | ] 141 | } 142 | ], 143 | "source": [ 144 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 145 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 146 | "\n", 147 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 148 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 149 | "print (df_table)\n", 150 | "\n", 151 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 152 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 153 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 154 | "\n", 155 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 156 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 157 | "print('f1 score: ', (2*p*r)/(p+r))" 158 | ] 159 | } 160 | ], 161 | "metadata": { 162 | "kernelspec": { 163 | "display_name": "Python 3", 164 | "language": "python", 165 | "name": "python3" 166 | }, 167 | "language_info": { 168 | "codemirror_mode": { 169 | "name": "ipython", 170 | "version": 3 171 | }, 172 | "file_extension": ".py", 173 | "mimetype": "text/x-python", 174 | "name": "python", 175 | "nbconvert_exporter": "python", 176 | "pygments_lexer": "ipython3", 177 | "version": "3.6.8" 178 | } 179 | }, 180 | "nbformat": 4, 181 | "nbformat_minor": 4 182 | } 183 | -------------------------------------------------------------------------------- /07 K-Nearest Neighbors/02 K-Neighbors Classifier.py: -------------------------------------------------------------------------------- 1 | # Import the libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.neighbors import KNeighborsClassifier 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | # Import the data 17 | df = pd.read_csv('data/00 df.csv') 18 | 19 | 20 | # Split data into Train & test 21 | train = df[df['flag']=='train'] 22 | test = df[df['flag']=='test'] 23 | 24 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 25 | 26 | y_train = train['y'] 27 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 28 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 29 | 30 | y_test = test['y'] 31 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 32 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 33 | 34 | 35 | # KNN 36 | error_rate = [] 37 | for i in range(1,51): 38 | knn = KNeighborsClassifier(n_neighbors=i) 39 | knn.fit(x_train,y_train) 40 | pred_i = knn.predict(x_test) 41 | error_rate.append(np.mean(pred_i != y_test)) 42 | 43 | 44 | # plot the data 45 | plt.figure(figsize=(8,4)) 46 | plt.plot(range(1,51),error_rate,color='darkred', marker='o',markersize=10) 47 | plt.title('Error Rate vs. K Value') 48 | plt.xlabel('K') 49 | plt.ylabel('Error Rate') 50 | plt.show() 51 | 52 | # KNN 53 | knn = KNeighborsClassifier(n_neighbors=15) 54 | knn.fit(x_train,y_train) 55 | y_pred=knn.predict(x_test) 56 | 57 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 58 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 59 | 60 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 61 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 62 | print (df_table) 63 | 64 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 65 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 66 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 67 | 68 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 69 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 70 | print('f1 score: ', (2*p*r)/(p+r)) -------------------------------------------------------------------------------- /07 K-Nearest Neighbors/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /07 K-Nearest Neighbors/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /08 Logistic Regression Classifier/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /08 Logistic Regression Classifier/02 Logistic Regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import math\n", 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from datetime import datetime\n", 13 | "\n", 14 | "import seaborn as sns\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "%matplotlib inline \n", 17 | "plt.style.use('seaborn-whitegrid')\n", 18 | "\n", 19 | "from sklearn.linear_model import LogisticRegression\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.metrics import confusion_matrix" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Get the Data" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "df = pd.read_csv('data/00 df.csv')\n", 38 | "train = df[df['flag']=='train']\n", 39 | "test = df[df['flag']=='test']" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 3, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']\n", 49 | "\n", 50 | "y_train = train['y']\n", 51 | "x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 52 | "x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True)\n", 53 | "\n", 54 | "y_test = test['y']\n", 55 | "x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 56 | "x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# Logistic Regression" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "lr = LogisticRegression()\n", 73 | "lr.fit(x_train, y_train)\n", 74 | "y_pred=lr.predict(x_test)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 5, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "name": "stdout", 84 | "output_type": "stream", 85 | "text": [ 86 | "[[11602 833]\n", 87 | " [ 1676 2170]]\n", 88 | "accuracy: 0.8458939868558443\n", 89 | "precision: 0.7226107226107226\n", 90 | "recall: 0.5642225689027561\n", 91 | "f1 score: 0.6336691487808439\n" 92 | ] 93 | } 94 | ], 95 | "source": [ 96 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 97 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 98 | "\n", 99 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 100 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 101 | "print (df_table)\n", 102 | "\n", 103 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 104 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 105 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 106 | "\n", 107 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 108 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 109 | "print('f1 score: ', (2*p*r)/(p+r))" 110 | ] 111 | } 112 | ], 113 | "metadata": { 114 | "kernelspec": { 115 | "display_name": "Python 3", 116 | "language": "python", 117 | "name": "python3" 118 | }, 119 | "language_info": { 120 | "codemirror_mode": { 121 | "name": "ipython", 122 | "version": 3 123 | }, 124 | "file_extension": ".py", 125 | "mimetype": "text/x-python", 126 | "name": "python", 127 | "nbconvert_exporter": "python", 128 | "pygments_lexer": "ipython3", 129 | "version": "3.6.8" 130 | } 131 | }, 132 | "nbformat": 4, 133 | "nbformat_minor": 4 134 | } 135 | -------------------------------------------------------------------------------- /08 Logistic Regression Classifier/02 Logistic Regression.py: -------------------------------------------------------------------------------- 1 | # import the libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.linear_model import LogisticRegression 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | 17 | # import the data 18 | df = pd.read_csv('data/00 df.csv') 19 | 20 | # split data into train n test 21 | train = df[df['flag']=='train'] 22 | test = df[df['flag']=='test'] 23 | 24 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 25 | 26 | y_train = train['y'] 27 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 28 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 29 | 30 | y_test = test['y'] 31 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 32 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 33 | 34 | 35 | # Logistic Regression Model 36 | lr=LogisticRegression() 37 | lr.fit(x_train, y_train) 38 | y_pred=lr.predict(x_test) 39 | 40 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 41 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 42 | 43 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 44 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 45 | print (df_table) 46 | 47 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 48 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 49 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 50 | 51 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 52 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 53 | print('f1 score: ', (2*p*r)/(p+r)) 54 | 55 | 56 | -------------------------------------------------------------------------------- /08 Logistic Regression Classifier/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /08 Logistic Regression Classifier/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /09 Naive_Bayes classifier/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /09 Naive_Bayes classifier/02 Naive_Bayes classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import math\n", 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from datetime import datetime\n", 13 | "\n", 14 | "import seaborn as sns\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "%matplotlib inline \n", 17 | "plt.style.use('seaborn-whitegrid')\n", 18 | "\n", 19 | "from sklearn.naive_bayes import GaussianNB\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.metrics import confusion_matrix" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Get the Data" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "df = pd.read_csv('data/00 df.csv')\n", 38 | "train = df[df['flag']=='train']\n", 39 | "test = df[df['flag']=='test']" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 3, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']\n", 49 | "\n", 50 | "y_train = train['y']\n", 51 | "x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 52 | "x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True)\n", 53 | "\n", 54 | "y_test = test['y']\n", 55 | "x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 56 | "x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# Naive Bayes" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "nb = GaussianNB()\n", 73 | "nb.fit(x_train, y_train)\n", 74 | "y_pred=nb.predict(x_test)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 5, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "name": "stdout", 84 | "output_type": "stream", 85 | "text": [ 86 | "[[10608 1827]\n", 87 | " [ 1412 2434]]\n", 88 | "accuracy: 0.8010564461642405\n", 89 | "precision: 0.5712274114057733\n", 90 | "recall: 0.6328653146125846\n", 91 | "f1 score: 0.6004687307265326\n" 92 | ] 93 | } 94 | ], 95 | "source": [ 96 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 97 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 98 | "\n", 99 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 100 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 101 | "print (df_table)\n", 102 | "\n", 103 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 104 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 105 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 106 | "\n", 107 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 108 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 109 | "print('f1 score: ', (2*p*r)/(p+r))" 110 | ] 111 | } 112 | ], 113 | "metadata": { 114 | "kernelspec": { 115 | "display_name": "Python 3", 116 | "language": "python", 117 | "name": "python3" 118 | }, 119 | "language_info": { 120 | "codemirror_mode": { 121 | "name": "ipython", 122 | "version": 3 123 | }, 124 | "file_extension": ".py", 125 | "mimetype": "text/x-python", 126 | "name": "python", 127 | "nbconvert_exporter": "python", 128 | "pygments_lexer": "ipython3", 129 | "version": "3.6.8" 130 | } 131 | }, 132 | "nbformat": 4, 133 | "nbformat_minor": 4 134 | } 135 | -------------------------------------------------------------------------------- /09 Naive_Bayes classifier/02 Naive_Bayes classifier.py: -------------------------------------------------------------------------------- 1 | # Import the libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.naive_bayes import GaussianNB 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | # Import the data 17 | df = pd.read_csv('data/00 df.csv') 18 | 19 | # Split data into train & test 20 | train = df[df['flag']=='train'] 21 | test = df[df['flag']=='test'] 22 | 23 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 24 | 25 | y_train = train['y'] 26 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 27 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 28 | 29 | y_test = test['y'] 30 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 31 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 32 | 33 | 34 | # Naive Bayes 35 | nb = GaussianNB() 36 | nb.fit(x_train, y_train) 37 | y_pred=nb.predict(x_test) 38 | 39 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 40 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 41 | 42 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 43 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 44 | print (df_table) 45 | 46 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 47 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 48 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 49 | 50 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 51 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 52 | print('f1 score: ', (2*p*r)/(p+r)) -------------------------------------------------------------------------------- /09 Naive_Bayes classifier/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /09 Naive_Bayes classifier/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /10 Random Forest/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /10 Random Forest/02 Random Forest.py: -------------------------------------------------------------------------------- 1 | # Import the libararies 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.ensemble import RandomForestClassifier 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | # Import the data 17 | df = pd.read_csv('data/00 df.csv') 18 | 19 | 20 | # split the data into train & test 21 | train = df[df['flag']=='train'] 22 | test = df[df['flag']=='test'] 23 | 24 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 25 | 26 | y_train = train['y'] 27 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 28 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 29 | 30 | y_test = test['y'] 31 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 32 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 33 | 34 | 35 | # Random Forest 36 | results = [] 37 | n_estimaor_options = [20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100] 38 | for trees in n_estimaor_options: 39 | model = RandomForestClassifier(trees, oob_score=True, n_jobs=-1, random_state=101) 40 | model.fit(x_train, y_train) 41 | y_pred = model.predict(x_test) 42 | accuracy = np.mean(y_test==y_pred) 43 | results.append(accuracy) 44 | 45 | plt.figure(figsize=(8,4)) 46 | pd.Series(results, n_estimaor_options).plot(color="darkred",marker="o") 47 | 48 | 49 | 50 | results = [] 51 | max_features_options = ['auto',None,'sqrt',0.95,0.75,0.5,0.25,0.10] 52 | for trees in max_features_options: 53 | model = RandomForestClassifier(n_estimators=70, oob_score=True, n_jobs=-1, random_state=101, max_features = trees) 54 | model.fit(x_train, y_train) 55 | y_pred = model.predict(x_test) 56 | accuracy = np.mean(y_test==y_pred) 57 | results.append(accuracy) 58 | 59 | plt.figure(figsize=(8,4)) 60 | pd.Series(results, max_features_options).plot(kind="bar",color="darkred",ylim=(0.7,0.9)) 61 | 62 | 63 | 64 | results = [] 65 | min_samples_leaf_options = [5,10,15,20,25,30,35,40,45,50] 66 | for trees in min_samples_leaf_options: 67 | model = RandomForestClassifier(n_estimators=70, oob_score=True, n_jobs=-1, random_state=101, max_features = None, min_samples_leaf = trees) 68 | model.fit(x_train, y_train) 69 | y_pred = model.predict(x_test) 70 | accuracy = np.mean(y_test==y_pred) 71 | results.append(accuracy) 72 | 73 | plt.figure(figsize=(8,4)) 74 | pd.Series(results, min_samples_leaf_options).plot(color="darkred",marker="o") 75 | 76 | 77 | 78 | rfm = RandomForestClassifier(n_estimators=70, oob_score=True, n_jobs=-1, random_state=101, max_features = None, min_samples_leaf = 30) 79 | rfm.fit(x_train, y_train) 80 | y_pred=rfm.predict(x_test) 81 | 82 | 83 | 84 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 85 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 86 | 87 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 88 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 89 | print (df_table) 90 | 91 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 92 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 93 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 94 | 95 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 96 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 97 | print('f1 score: ', (2*p*r)/(p+r)) -------------------------------------------------------------------------------- /10 Random Forest/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /10 Random Forest/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /11 Stochastic Gradient Descent/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /11 Stochastic Gradient Descent/02 Stochastic Gradient Descent.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import math\n", 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from datetime import datetime\n", 13 | "\n", 14 | "import seaborn as sns\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "%matplotlib inline \n", 17 | "plt.style.use('seaborn-whitegrid')\n", 18 | "\n", 19 | "from sklearn.linear_model import SGDClassifier\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.metrics import confusion_matrix" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Get the Data" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "df = pd.read_csv('data/00 df.csv')\n", 38 | "train = df[df['flag']=='train']\n", 39 | "test = df[df['flag']=='test']" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 3, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']\n", 49 | "\n", 50 | "y_train = train['y']\n", 51 | "x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 52 | "x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True)\n", 53 | "\n", 54 | "y_test = test['y']\n", 55 | "x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 56 | "x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# Stochastic Gradient Descent" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "sgd = SGDClassifier(loss='modified_huber', shuffle=True,random_state=101)\n", 73 | "sgd.fit(x_train, y_train)\n", 74 | "y_pred=sgd.predict(x_test)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 5, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "name": "stdout", 84 | "output_type": "stream", 85 | "text": [ 86 | "[[11244 1191]\n", 87 | " [ 1456 2390]]\n", 88 | "accuracy: 0.8374178490264725\n", 89 | "precision: 0.6674113376151913\n", 90 | "recall: 0.6214248569942797\n", 91 | "f1 score: 0.643597684125488\n" 92 | ] 93 | } 94 | ], 95 | "source": [ 96 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 97 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 98 | "\n", 99 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 100 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 101 | "print (df_table)\n", 102 | "\n", 103 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 104 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 105 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 106 | "\n", 107 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 108 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 109 | "print('f1 score: ', (2*p*r)/(p+r))" 110 | ] 111 | } 112 | ], 113 | "metadata": { 114 | "kernelspec": { 115 | "display_name": "Python 3", 116 | "language": "python", 117 | "name": "python3" 118 | }, 119 | "language_info": { 120 | "codemirror_mode": { 121 | "name": "ipython", 122 | "version": 3 123 | }, 124 | "file_extension": ".py", 125 | "mimetype": "text/x-python", 126 | "name": "python", 127 | "nbconvert_exporter": "python", 128 | "pygments_lexer": "ipython3", 129 | "version": "3.6.8" 130 | } 131 | }, 132 | "nbformat": 4, 133 | "nbformat_minor": 4 134 | } 135 | -------------------------------------------------------------------------------- /11 Stochastic Gradient Descent/02 Stochastic Gradient Descent.py: -------------------------------------------------------------------------------- 1 | # Import the libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.linear_model import SGDClassifier 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | # import the data 17 | df = pd.read_csv('data/00 df.csv') 18 | 19 | 20 | # split data into train & test 21 | train = df[df['flag']=='train'] 22 | test = df[df['flag']=='test'] 23 | 24 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 25 | 26 | y_train = train['y'] 27 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 28 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 29 | 30 | y_test = test['y'] 31 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 32 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 33 | 34 | 35 | # Stochastic Gradient Descent 36 | sgd = SGDClassifier(loss='modified_huber', shuffle=True,random_state=101) 37 | sgd.fit(x_train, y_train) 38 | y_pred=sgd.predict(x_test) 39 | 40 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 41 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 42 | 43 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 44 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 45 | print (df_table) 46 | 47 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 48 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 49 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 50 | 51 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 52 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 53 | print('f1 score: ', (2*p*r)/(p+r)) -------------------------------------------------------------------------------- /11 Stochastic Gradient Descent/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /11 Stochastic Gradient Descent/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /12 Support Vector Machine/01 Data_Prep.sas: -------------------------------------------------------------------------------- 1 | options compress=yes; 2 | 3 | data train; 4 | format flag $20.; 5 | set 'ADULT.DATA'n; 6 | flag = 'train'; 7 | run; 8 | 9 | data test (rename=('|1x3 Cross validator'n = F1)); 10 | format flag $20.; 11 | set 'ADULT.TEST'n; 12 | flag = 'test'; 13 | run; 14 | 15 | data rg.df (rename=( 16 | F1 = age 17 | F2 = workclass 18 | F3 = fnlwgt 19 | F4 = education 20 | F5 = education_num 21 | F6 = marital_status 22 | F7 = occupation 23 | F8 = relationship 24 | F9 = race 25 | F10 = sex 26 | F11 = capital_gain 27 | F12 = capital_loss 28 | F13 = hours_per_week 29 | F14 = native_country 30 | F15 = income 31 | )); 32 | set train test; 33 | if compress(F15) ^= ""; 34 | run; 35 | 36 | data rg.df; 37 | set rg.df; 38 | if compress(income) in (">50K",">50K.") then y = 1; 39 | else y = 0; 40 | run; 41 | 42 | /*age*/ 43 | data rg.df; 44 | format age_bin $20.; 45 | set rg.df; 46 | if age <= 25 then age_bin = 'a. 0-25'; 47 | else if age <= 30 then age_bin = 'b. 26-30 & 71-100'; 48 | else if age <= 35 then age_bin = 'c. 31-35 & 61-70'; 49 | else if age <= 40 then age_bin = 'd. 36-40 & 56-60'; 50 | else if age <= 55 then age_bin = 'e. 40-55'; 51 | else if age <= 60 then age_bin = 'd. 36-40 & 56-60'; 52 | else if age <= 70 then age_bin = 'c. 31-35 & 61-70'; 53 | else age_bin = 'b. 26-30 & 71-100'; 54 | run; 55 | 56 | /*workclass*/ 57 | data rg.df; 58 | format workclass_bin $20.; 59 | set rg.df; 60 | if workclass in ('?','Never-worked','Without-pay') then workclass_bin = 'a. no income'; 61 | else workclass_bin = 'b. income'; 62 | run; 63 | 64 | /*education*/ 65 | data rg.df; 66 | format education_bin $20.; 67 | set rg.df; 68 | if education in ('10th','11th','12th','1st-4th','5th-6th','7th-8th','9th','Preschool') then education_bin = 'a. Low'; 69 | else if education in ('HS-grad','Some-college','Assoc-acdm','Assoc-voc') then education_bin = 'b. Mid'; 70 | else if education in ('Bachelors') then education_bin = 'c. Bachelors'; 71 | else if education in ('Masters') then education_bin = 'd. Masters'; 72 | else education_bin = 'e. High'; 73 | run; 74 | 75 | /*education_num*/ 76 | data rg.df; 77 | format education_num_bin $20.; 78 | set rg.df; 79 | if education_num <= 8 then education_num_bin = 'a. 0-8'; 80 | else if education_num <= 12 then education_num_bin = 'b. 9-12'; 81 | else if education_num <= 13 then education_num_bin = 'c. 13'; 82 | else if education_num <= 14 then education_num_bin = 'd. 14'; 83 | else education_num_bin = 'e. 15+'; 84 | run; 85 | 86 | /*race & sex*/ 87 | data rg.df; 88 | format race_sex $50.; 89 | format race_sex_bin $20.; 90 | set rg.df; 91 | race_sex = compress(race)||' - '||compress(sex); 92 | if race_sex in ('Asian-Pac-Islander - Male','White - Male') then race_sex_bin = 'c. High'; 93 | else if race_sex in ('White - Female','Asian-Pac-Islander - Female','Amer-Indian-Eskimo - Male','Other - Male','Black - Male') then race_sex_bin = 'b. Mid'; 94 | else race_sex_bin = 'a. Low'; 95 | run; 96 | 97 | /*capital_gain & capital_loss*/ 98 | data rg.df; 99 | format capital_gl_bin $20.; 100 | set rg.df; 101 | if capital_gain = . then capital_gain = 0; 102 | if capital_loss = . then capital_loss = 0; 103 | capital_gl = capital_gain - capital_loss; 104 | if capital_gl > 0 then capital_gl_bin = "c. > 0"; 105 | else if capital_gl < 0 then capital_gl_bin = "b. < 0"; 106 | else capital_gl_bin = "a. = 0"; 107 | run; 108 | 109 | /*marital_status & relationship*/ 110 | data rg.df; 111 | format msr $50.; 112 | format msr_bin $20.; 113 | set rg.df; 114 | msr = compress(marital_status)||' - '||compress(relationship); 115 | if msr in ('Married-AF-spouse - Wife','Married-civ-spouse - Husband','Married-civ-spouse - Wife','Married-AF-spouse - Husband') then msr_bin = 'c. High'; 116 | else if msr in ('Widowed - Not-in-family','Divorced - Unmarried','Never-married - Not-in-family','Widowed - Unmarried','Separated - Not-in-family','Married-spouse-absent - Not-in-family','Divorced - Not-in-family','Married-civ-spouse - Other-relative','Married-civ-spouse - Own-child','Married-civ-spouse - Not-in-family') then msr_bin = 'b. Mid'; 117 | else msr_bin = 'a. Low'; 118 | run; 119 | 120 | /*occupation*/ 121 | data rg.df; 122 | format occupation_bin $20.; 123 | set rg.df; 124 | if occupation in ('Priv-house-serv','Other-service','Handlers-cleaners') then occupation_bin = 'a. Low'; 125 | else if occupation in ('Armed-Forces','?','Farming-fishing','Machine-op-inspct','Adm-clerical') then occupation_bin = 'b. Mid - Low'; 126 | else if occupation in ('Transport-moving','Craft-repair','Sales') then occupation_bin = 'c. Mid - Mid'; 127 | else if occupation in ('Tech-support','Protective-serv') then occupation_bin = 'd. Mid - High'; 128 | else occupation_bin = 'e. High'; 129 | run; 130 | 131 | /*hours_per_week*/ 132 | data rg.df; 133 | format hours_per_week_bin $20.; 134 | set rg.df; 135 | if hours_per_week <= 30 then hours_per_week_bin = 'a. 0-30'; 136 | else if hours_per_week <= 40 then hours_per_week_bin = 'b. 31-40'; 137 | else if hours_per_week <= 50 then hours_per_week_bin = 'd. 41-50 & 61-70'; 138 | else if hours_per_week <= 60 then hours_per_week_bin = 'e. 51-60'; 139 | else if hours_per_week <= 70 then hours_per_week_bin = 'd. 41-50 & 61-70'; 140 | else hours_per_week_bin = 'c. 71-100'; 141 | run; 142 | 143 | %macro rg_bin (var); 144 | proc sql; 145 | select &var., count(y) as cnt_y, mean(y) as avg_y 146 | from rg.df 147 | group by &var.; 148 | quit; 149 | %mend; 150 | 151 | %rg_bin(age_bin); 152 | %rg_bin(workclass_bin); 153 | %rg_bin(education_bin); 154 | %rg_bin(education_num_bin); 155 | %rg_bin(race_sex_bin); 156 | %rg_bin(capital_gl_bin); 157 | %rg_bin(msr_bin); 158 | %rg_bin(occupation_bin); 159 | %rg_bin(hours_per_week_bin); 160 | 161 | data rg.df (keep= 162 | y 163 | flag 164 | age_bin 165 | workclass_bin 166 | education_bin 167 | education_num_bin 168 | race_sex_bin 169 | capital_gl_bin 170 | msr_bin 171 | occupation_bin 172 | hours_per_week_bin 173 | ); 174 | set rg.df; 175 | run; -------------------------------------------------------------------------------- /12 Support Vector Machine/02 Support Vector Machine.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 10, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import math\n", 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from datetime import datetime\n", 13 | "\n", 14 | "import seaborn as sns\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "%matplotlib inline \n", 17 | "plt.style.use('seaborn-whitegrid')\n", 18 | "\n", 19 | "from sklearn.svm import SVC\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.metrics import confusion_matrix" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "# Get the Data" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 11, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "df = pd.read_csv('data/00 df.csv')\n", 38 | "train = df[df['flag']=='train']\n", 39 | "test = df[df['flag']=='test']" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 12, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']\n", 49 | "\n", 50 | "y_train = train['y']\n", 51 | "x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 52 | "x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True)\n", 53 | "\n", 54 | "y_test = test['y']\n", 55 | "x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']]\n", 56 | "x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# Support Vector Machine" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 13, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "svm = SVC(kernel=\"rbf\", C=0.025,random_state=101)\n", 73 | "svm.fit(x_train, y_train)\n", 74 | "y_pred=svm.predict(x_test)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 14, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "name": "stdout", 84 | "output_type": "stream", 85 | "text": [ 86 | "[[11769 666]\n", 87 | " [ 2019 1827]]\n", 88 | "accuracy: 0.8350838400589644\n", 89 | "precision: 0.7328519855595668\n", 90 | "recall: 0.4750390015600624\n", 91 | "f1 score: 0.5764316138192144\n" 92 | ] 93 | } 94 | ], 95 | "source": [ 96 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 97 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 98 | "\n", 99 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 100 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 101 | "print (df_table)\n", 102 | "\n", 103 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 104 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 105 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 106 | "\n", 107 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 108 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 109 | "print('f1 score: ', (2*p*r)/(p+r))" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": 15, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "svm = SVC(kernel=\"linear\", C=0.025,random_state=101)\n", 119 | "svm.fit(x_train, y_train)\n", 120 | "y_pred=svm.predict(x_test)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 16, 126 | "metadata": {}, 127 | "outputs": [ 128 | { 129 | "name": "stdout", 130 | "output_type": "stream", 131 | "text": [ 132 | "[[11625 810]\n", 133 | " [ 1781 2065]]\n", 134 | "accuracy: 0.8408574411891161\n", 135 | "precision: 0.7182608695652174\n", 136 | "recall: 0.5369214768590743\n", 137 | "f1 score: 0.6144918910876358\n" 138 | ] 139 | } 140 | ], 141 | "source": [ 142 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 143 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 144 | "\n", 145 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 146 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 147 | "print (df_table)\n", 148 | "\n", 149 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 150 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 151 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 152 | "\n", 153 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 154 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 155 | "print('f1 score: ', (2*p*r)/(p+r))" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": 17, 161 | "metadata": {}, 162 | "outputs": [], 163 | "source": [ 164 | "svm = SVC(kernel=\"poly\", C=0.025,random_state=101)\n", 165 | "svm.fit(x_train, y_train)\n", 166 | "y_pred=svm.predict(x_test)" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 18, 172 | "metadata": {}, 173 | "outputs": [ 174 | { 175 | "name": "stdout", 176 | "output_type": "stream", 177 | "text": [ 178 | "[[11705 730]\n", 179 | " [ 1838 2008]]\n", 180 | "accuracy: 0.8422701308273448\n", 181 | "precision: 0.733382030679328\n", 182 | "recall: 0.5221008840353614\n", 183 | "f1 score: 0.6099635479951396\n" 184 | ] 185 | } 186 | ], 187 | "source": [ 188 | "test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1)\n", 189 | "test_calc.rename(columns={0: 'predicted'}, inplace=True)\n", 190 | "\n", 191 | "test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0)\n", 192 | "df_table = confusion_matrix(test_calc['y'],test_calc['predicted'])\n", 193 | "print (df_table)\n", 194 | "\n", 195 | "print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1]))\n", 196 | "print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1]))\n", 197 | "print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0]))\n", 198 | "\n", 199 | "p = df_table[1,1] / (df_table[1,1] + df_table[0,1])\n", 200 | "r = df_table[1,1] / (df_table[1,1] + df_table[1,0])\n", 201 | "print('f1 score: ', (2*p*r)/(p+r))" 202 | ] 203 | } 204 | ], 205 | "metadata": { 206 | "kernelspec": { 207 | "display_name": "Python 3", 208 | "language": "python", 209 | "name": "python3" 210 | }, 211 | "language_info": { 212 | "codemirror_mode": { 213 | "name": "ipython", 214 | "version": 3 215 | }, 216 | "file_extension": ".py", 217 | "mimetype": "text/x-python", 218 | "name": "python", 219 | "nbconvert_exporter": "python", 220 | "pygments_lexer": "ipython3", 221 | "version": "3.6.8" 222 | } 223 | }, 224 | "nbformat": 4, 225 | "nbformat_minor": 4 226 | } 227 | -------------------------------------------------------------------------------- /12 Support Vector Machine/02 Support Vector Machine.py: -------------------------------------------------------------------------------- 1 | # Import the libraries 2 | import math 3 | import numpy as np 4 | import pandas as pd 5 | from datetime import datetime 6 | 7 | import seaborn as sns 8 | import matplotlib.pyplot as plt 9 | %matplotlib inline 10 | plt.style.use('seaborn-whitegrid') 11 | 12 | from sklearn.svm import SVC 13 | from sklearn.metrics import classification_report 14 | from sklearn.metrics import confusion_matrix 15 | 16 | 17 | # Import the dataset 18 | df = pd.read_csv('data/00 df.csv') 19 | 20 | 21 | # SPlit data into Train & test 22 | train = df[df['flag']=='train'] 23 | test = df[df['flag']=='test'] 24 | 25 | cat_feats = ['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin'] 26 | 27 | y_train = train['y'] 28 | x_train = train[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 29 | x_train = pd.get_dummies(x_train,columns=cat_feats,drop_first=True) 30 | 31 | y_test = test['y'] 32 | x_test = test[['age_bin','capital_gl_bin','education_bin','hours_per_week_bin','msr_bin','occupation_bin','race_sex_bin']] 33 | x_test = pd.get_dummies(x_test,columns=cat_feats,drop_first=True) 34 | 35 | 36 | 37 | # Support Vector Machine 38 | svm = SVC(kernel="rbf", C=0.025,random_state=101) 39 | svm.fit(x_train, y_train) 40 | y_pred=svm.predict(x_test) 41 | 42 | 43 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 44 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 45 | 46 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 47 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 48 | print (df_table) 49 | 50 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 51 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 52 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 53 | 54 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 55 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 56 | print('f1 score: ', (2*p*r)/(p+r)) 57 | 58 | 59 | # SVC 60 | svm = SVC(kernel="linear", C=0.025,random_state=101) 61 | svm.fit(x_train, y_train) 62 | y_pred=svm.predict(x_test) 63 | 64 | 65 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 66 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 67 | 68 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 69 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 70 | print (df_table) 71 | 72 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 73 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 74 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 75 | 76 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 77 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 78 | print('f1 score: ', (2*p*r)/(p+r)) 79 | 80 | 81 | # SVC 82 | svm = SVC(kernel="poly", C=0.025,random_state=101) 83 | svm.fit(x_train, y_train) 84 | y_pred=svm.predict(x_test) 85 | 86 | 87 | test_calc = pd.concat([pd.DataFrame(y_test).reset_index(drop=True),pd.DataFrame(y_pred).reset_index(drop=True)],axis=1) 88 | test_calc.rename(columns={0: 'predicted'}, inplace=True) 89 | 90 | test_calc['predicted'] = test_calc['predicted'].apply(lambda x: 1 if x > 0.5 else 0) 91 | df_table = confusion_matrix(test_calc['y'],test_calc['predicted']) 92 | print (df_table) 93 | 94 | print('accuracy:', (df_table[0,0] + df_table[1,1]) / (df_table[0,0] + df_table[0,1] + df_table[1,0] + df_table[1,1])) 95 | print ('precision:', df_table[1,1] / (df_table[1,1] + df_table[0,1])) 96 | print('recall:', df_table[1,1] / (df_table[1,1] + df_table[1,0])) 97 | 98 | p = df_table[1,1] / (df_table[1,1] + df_table[0,1]) 99 | r = df_table[1,1] / (df_table[1,1] + df_table[1,0]) 100 | print('f1 score: ', (2*p*r)/(p+r)) 101 | 102 | -------------------------------------------------------------------------------- /12 Support Vector Machine/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /12 Support Vector Machine/data/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Free Machine Learning 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | --- 2 |

Machine-Learning-Classification

3 | 4 | --- 5 | --- 6 | ![1_9iNsnOm0I3YFF-A_ryp5Mg](https://user-images.githubusercontent.com/42931974/71458425-af86a200-27c8-11ea-9643-8e849b990cf9.jpeg) 7 | 8 | --- 9 | --- 10 |

Classification Process

11 | 12 | ![1_PAqzvCxPjpDN8RC9HQw45w](https://user-images.githubusercontent.com/42931974/71458473-de047d00-27c8-11ea-9bed-e3d4d3752f0b.jpeg) 13 | 14 | --- 15 | --- 16 | #### Classification is the process of predicting the class of given data points. Classes are sometimes called as targets/ labels or categories. Classification predictive modeling is the task of approximating a mapping function (f) from input variables (X) to discrete output variables (y). 17 | #### For example, spam detection in email service providers can be identified as a classification problem. This is s binary classification since there are only 2 classes as spam and not spam. A classifier utilizes some training data to understand how given input variables relate to the class. In this case, known spam and non-spam emails have to be used as the training data. When the classifier is trained accurately, it can be used to detect an unknown email. 18 | #### Classification belongs to the category of supervised learning where the targets also provided with the input data. There are many applications in classification in many domains such as in credit approval, medical diagnosis, target marketing etc. 19 | --------------------------------------------------------------------------------